Skip to contentSkip to conversation

OCaml fixes

PATCH In review Edwin Török 5 participants 23 messages Dec 16, 2022 – Dec 19, 2022

lore
Share & tools
lore
#12 lore raw

PATCH v4 6/11 tools/ocaml: add .clang-format

In reply to Edwin Török

Add a .clang-format configuration that tries to match CODING_STYLE where possible.

I was not able to express the special casing of braces after 'do' though, this can only be controlled generally for all control statements.
It is imperfect, but should be better than the existing bindings, which do not follow Xen coding style.

Add this to tools/ocaml first because:
* there are relatively few C files here, and it is a good place to start with
* it'd be useful to make these follow Xen's CODING_STYLE
(which they currently do not because they use tabs for example)
* they change relatively infrequently, so shouldn't cause issues with backporting security fixes (could either backport the reindentation patch too, or use git cherry-pick with -Xignore-space-change)

Does not yet reformat any code.

No functional change.

---
Changes since v1:
* change commit title to reflect this is for OCaml subtree only
* don't mention stdint.h here, that may be fixed in a different way elsewhere
* add Acked-by line

---
 tools/ocaml/.clang-format | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 tools/ocaml/.clang-format
tools/ocaml/.clang-format new file +9 -0
new file mode 100644
index 0000000000..7ff88ee043
@@ -0,0 +1,9 @@
+BasedOnStyle: GNU
+IndentWidth: 4
+
+# override GNU to match Xen ../../CODING_STYLE more closely
+AlwaysBreakAfterDefinitionReturnType: None
+AlwaysBreakAfterReturnType: None
+SpacesInConditionalStatement: true
+SpaceBeforeParens: ControlStatements
+BreakBeforeBraces: Allman
Signature · 1 line
2.34.1
#17 lore raw

PATCH v4 9/11 tools/ocaml/libs/xb: check for unmapped ring before accessing it

In reply to Edwin Török

Xenmmap can unmap the ring, check for this condition before accessing it to avoid crashing on an unmapped page.

Note that we cannot use the usual OCaml finalizers (like bigarray would) to perform the unmap, because that might keep a reference count to a foreign domain's memory that we want to release before destroying the domain.

---
Changes:
* new patch

---
 tools/ocaml/libs/xb/xs_ring_stubs.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
tools/ocaml/libs/xb/xs_ring_stubs.c +8 -3
index 1e472d0bbf..28c79ee139 100644
@@ -35,7 +35,14 @@
 #include <sys/mman.h>
 #include "mmap_stubs.h"
 
-#define GET_C_STRUCT(a) ((struct mmap_interface *) Data_abstract_val(a))
+static struct mmap_interface* check_addr(struct mmap_interface *interface)
+{
+    if (!interface->addr || interface->addr == MAP_FAILED)
+        caml_failwith("ring is not mapped");
+    return interface;
+}
+
+#define GET_C_STRUCT(a) check_addr((struct mmap_interface *) Data_abstract_val(a))
 
 /*
  * Bytes_val has been introduced by Ocaml 4.06.1. So define our own version
@@ -167,8 +174,6 @@ CAMLprim value ml_interface_set_server_features(value interface, value v)
 {
     CAMLparam2(interface, v);
     struct xenstore_domain_interface *intf = GET_C_STRUCT(interface)->addr;
-    if (intf == (void*)MAP_FAILED)
-        caml_failwith("Interface closed");
 
     intf->server_features = Int_val(v);
 
Signature · 1 line
2.34.1
#0 lore raw

v4 cover OCaml fixes

From: Edwin Török <[email protected]>

Various OCaml fixes, some of which got Acked already.

Note: the Data_abstract_val got Acked but not yet commited because we thought there might be a better way with finalizers/etc.
It is not possible to use finalizers here but there is a new followup patch in this series that detects use-after-free and raises an exception without trigerring undefined behaviour or crashes (see 'xb: check for unmapped ring').

The formatting patch was controversial during the 4.17 freeze, but master is open now, and both the maintainer and myself agree with it,
so I think we should be able to do it now.
I've split it up into 2: add the formatting rules, and the actual formatting, so that it is easier to redo the formatting patch when this gets backported.

I think we should just commit it rather than worry that there might be more bugfixes to come after and attempt to rebase the bugfixes without the formatting patch (introducing bugs in the process), especially that one of the patches not yet posted to fix use of enter/leave changes quite a significant part of all C stubs, and I don't have a way of developing those patches without the formatting patch.

The rest are various bugfixes to the C bindings, and I have more to send after I've tested them
(almost all uses of enter/leave blocking section are
wrong because they access OCaml values in non-safe ways with _H, but the GC may have already moved them which would cause a crash, that changes a lot of bindings and needs more testing before sending out, and I've got a few more OCaml 5 fixes too).

Edwin Török (11):
tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper tools/ocaml/xenstored/Makefile: use ocamldep -sort for linking order CODING_STYLE(tools/ocaml): add 'make format' and remove tabs tools/ocaml: run "make format"
CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation
uses spaces
tools/ocaml: add .clang-format
fixup! tools/ocaml/xenctrl: OCaml 5 support, fix use-after-free tools/ocaml/libs/mmap: mark mmap/munmap as blocking and raise
Unix_error on failure
tools/ocaml/libs/xb: check for unmapped ring before accessing it tools/ocaml/xenstored: validate config file before live update tools/ocaml/libs/xb: drop Xs_ring.write

 .editorconfig                                 |   20 +
 tools/ocaml/.clang-format                     |    9 +
 tools/ocaml/Makefile                          |    5 +
 tools/ocaml/libs/eventchn/xeneventchn_stubs.c |  194 +-
 tools/ocaml/libs/mmap/mmap_stubs.h            |    8 +-
 tools/ocaml/libs/mmap/xenmmap.ml              |    2 +-
 tools/ocaml/libs/mmap/xenmmap.mli             |    4 +-
 tools/ocaml/libs/mmap/xenmmap_stubs.c         |  123 +-
 tools/ocaml/libs/xb/op.ml                     |   76 +-
 tools/ocaml/libs/xb/packet.ml                 |   30 +-
 tools/ocaml/libs/xb/partial.ml                |   48 +-
 tools/ocaml/libs/xb/xb.ml                     |  416 ++--
 tools/ocaml/libs/xb/xb.mli                    |  106 +-
 tools/ocaml/libs/xb/xenbus_stubs.c            |   50 +-
 tools/ocaml/libs/xb/xs_ring.ml                |   31 +-
 tools/ocaml/libs/xb/xs_ring_stubs.c           |  221 +-
 tools/ocaml/libs/xc/xenctrl.ml                |  326 +--
 tools/ocaml/libs/xc/xenctrl.mli               |   12 +-
 tools/ocaml/libs/xc/xenctrl_stubs.c           | 1556 ++++++-------
 tools/ocaml/libs/xentoollog/caml_xentoollog.h |    6 +-
 .../ocaml/libs/xentoollog/xentoollog_stubs.c  |  196 +-
 tools/ocaml/libs/xl/xenlight_stubs.c          | 2022 ++++++++---------
 tools/ocaml/libs/xs/queueop.ml                |   48 +-
 tools/ocaml/libs/xs/xs.ml                     |  220 +-
 tools/ocaml/libs/xs/xs.mli                    |   46 +-
 tools/ocaml/libs/xs/xsraw.ml                  |  300 +--
 tools/ocaml/libs/xs/xst.ml                    |   76 +-
 tools/ocaml/libs/xs/xst.mli                   |   20 +-
 tools/ocaml/test/dmesg.ml                     |   26 +-
 tools/ocaml/test/list_domains.ml              |    4 +-
 tools/ocaml/test/raise_exception.ml           |    4 +-
 tools/ocaml/test/xtl.ml                       |   28 +-
 tools/ocaml/xenstored/Makefile                |    6 +-
 tools/ocaml/xenstored/config.ml               |  156 +-
 tools/ocaml/xenstored/connection.ml           |  578 ++---
 tools/ocaml/xenstored/connections.ml          |  294 +--
 tools/ocaml/xenstored/disk.ml                 |  218 +-
 tools/ocaml/xenstored/domain.ml               |  116 +-
 tools/ocaml/xenstored/domains.ml              |  298 +--
 tools/ocaml/xenstored/event.ml                |   28 +-
 tools/ocaml/xenstored/history.ml              |   62 +-
 tools/ocaml/xenstored/logging.ml              |  478 ++--
 tools/ocaml/xenstored/packet.ml               |   20 +-
 tools/ocaml/xenstored/parse_arg.ml            |  132 +-
 tools/ocaml/xenstored/perms.ml                |  216 +-
 tools/ocaml/xenstored/poll.ml                 |   68 +-
 tools/ocaml/xenstored/poll.mli                |    4 +-
 tools/ocaml/xenstored/process.ml              | 1210 +++++-----
 tools/ocaml/xenstored/quota.ml                |   74 +-
 tools/ocaml/xenstored/select_stubs.c          |   62 +-
 tools/ocaml/xenstored/stdext.ml               |  190 +-
 tools/ocaml/xenstored/store.ml                |  752 +++---
 tools/ocaml/xenstored/symbol.ml               |    2 +-
 tools/ocaml/xenstored/syslog.ml               |   48 +-
 tools/ocaml/xenstored/syslog_stubs.c          |   68 +-
 tools/ocaml/xenstored/systemd_stubs.c         |   10 +-
 tools/ocaml/xenstored/transaction.ml          |  352 +--
 tools/ocaml/xenstored/trie.ml                 |  222 +-
 tools/ocaml/xenstored/trie.mli                |   22 +-
 tools/ocaml/xenstored/utils.ml                |  146 +-
 tools/ocaml/xenstored/xenstored.ml            |  987 ++++----
 61 files changed, 6569 insertions(+), 6483 deletions(-)
 create mode 100644 .editorconfig
 create mode 100644 tools/ocaml/.clang-format

--
2.34.1

#16 lore raw

PATCH v4 8/11 tools/ocaml/libs/mmap: mark mmap/munmap as blocking and raise Unix_error on failure

These functions can potentially take some time, so allow other OCaml code to proceed meanwhile (if any).

Also raise a Unix_error based on errno if mmap fails, instead of just calling failwith (which would lose the error reason).

---

Changes since v2:
* repost of lost patch from 2020 (posted to ML on 2021
https://patchwork.kernel.org/project/xen-devel/patch/294a60be29027d33b0a1d154b7d576237c7dd420.1620755942.git.edvin.torok@citrix.com/)

---
 tools/ocaml/libs/mmap/xenmmap_stubs.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
tools/ocaml/libs/mmap/xenmmap_stubs.c +12 -3
index bf864a7c32..5b8c240ef9 100644
@@ -27,6 +27,8 @@
 #include <caml/custom.h>
 #include <caml/fail.h>
 #include <caml/callback.h>
+#include <caml/unixsupport.h>
+#include <caml/signals.h>
 
 #define Intf_val(a) ((struct mmap_interface *) Data_abstract_val(a))
 
@@ -35,7 +37,9 @@ static int mmap_interface_init(struct mmap_interface *intf,
                                int len, int offset)
 {
     intf->len = len;
+    caml_enter_blocking_section();
     intf->addr = mmap(NULL, len, pflag, mflag, fd, offset);
+    caml_leave_blocking_section();
     return (intf->addr == MAP_FAILED) ? errno : 0;
 }
 
@@ -64,17 +68,22 @@ CAMLprim value stub_mmap_init(value fd, value pflag, value mflag,
     if (mmap_interface_init(Intf_val(result), Int_val(fd),
                             c_pflag, c_mflag,
                             Int_val(len), Int_val(offset)))
-        caml_failwith("mmap");
+        uerror("mmap", Nothing);
     CAMLreturn(result);
 }
 
 CAMLprim value stub_mmap_final(value intf)
 {
     CAMLparam1(intf);
+    struct mmap_interface interface = *Intf_val(intf);
 
-    if (Intf_val(intf)->addr != MAP_FAILED)
-        munmap(Intf_val(intf)->addr, Intf_val(intf)->len);
+    /* mark it as freed, in case munmap below fails, so we don't retry it */
     Intf_val(intf)->addr = MAP_FAILED;
+    if (interface.addr != MAP_FAILED) {
+        caml_enter_blocking_section();
+        munmap(interface.addr, interface.len);
+        caml_leave_blocking_section();
+    }
 
     CAMLreturn(Val_unit);
 }
Signature · 1 line
2.34.1
#1 lore raw

PATCH v4 1/11 tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper

In reply to Edwin Török

This is not strictly necessary since it is essentially a no-op
currently: a cast to void* and value*, even in OCaml 5.0.

However it does make it clearer that what we have here is not a regular OCaml value, but one allocated with Abstract_tag or Custom_tag, and follows the example from the manual more closely:
https://v2.ocaml.org/manual/intfc.html#ss:c-outside-head

It also makes it clearer that these modules have been reviewed for compat with OCaml 5.0.

We cannot use OCaml finalizers here, because we want control over when to unmap these pages from remote domains.
A follow-up commit will add use-after-free detection instead.

No functional change.

---
Reason for inclusion in 4.17:
- make code follow best practice for upcoming OCaml 5.0 compiler (already in beta)

Changes since v2:
- add Acked-by line

Changes since v3:
- mention that use-after-free is fixed in another commit, and we cannot use finalizers here

---
 tools/ocaml/libs/mmap/mmap_stubs.h    | 4 ++++
 tools/ocaml/libs/mmap/xenmmap_stubs.c | 2 +-
 tools/ocaml/libs/xb/xs_ring_stubs.c   | 2 +-
 3 files changed, 6 insertions(+), 2 deletions(-)
tools/ocaml/libs/mmap/mmap_stubs.h +4 -0
index 65e4239890..66f18d4406 100644
@@ -30,4 +30,8 @@ struct mmap_interface
 	int len;
 };
 
+#ifndef Data_abstract_val
+#define Data_abstract_val(x) ((void*)(value*)(x))
+#endif
+
 #endif
tools/ocaml/libs/mmap/xenmmap_stubs.c +1 -1
index e2ce088e25..141dedb78c 100644
@@ -28,7 +28,7 @@
 #include <caml/fail.h>
 #include <caml/callback.h>
 
-#define Intf_val(a) ((struct mmap_interface *) a)
+#define Intf_val(a) ((struct mmap_interface *) Data_abstract_val(a))
 
 static int mmap_interface_init(struct mmap_interface *intf,
                                int fd, int pflag, int mflag,
tools/ocaml/libs/xb/xs_ring_stubs.c +1 -1
index 7a91fdee75..cc9114029f 100644
@@ -35,7 +35,7 @@
 #include <sys/mman.h>
 #include "mmap_stubs.h"
 
-#define GET_C_STRUCT(a) ((struct mmap_interface *) a)
+#define GET_C_STRUCT(a) ((struct mmap_interface *) Data_abstract_val(a))
 
 /*
  * Bytes_val has been introduced by Ocaml 4.06.1. So define our own version
Signature · 1 line
2.34.1
#10 lore raw

PATCH v4 5/11 CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation uses spaces

In reply to Edwin Török

Add an .editorconfig to make it easier to keep patches compatible with Xen's coding style, and to reemphasize what Xen's coding style is.

I thought that Xen demands tabs rather than spaces (which is more difficult with OCaml because indentation tools use spaces, and the use of tabs requires changing editor settings), however CODING-STYLE says it is spaces.

Document this explicitly by adding a .editorconfig file (see editorconfig.org), which is an editor agnostic format for specifying basic style properties like indentation, either with native support in editors or via plugins.

It is safer than modelines because it only supports controlling a restricted set of editor properties and not arbitrary commands as Vim modelines would have, and works with editors other than Vim too. (Vim has a deny list for modeline sandboxing, which is error-prone because every time a new command gets added it needs to be added to the deny list, which has been the source of a few CVEs in the past and I disable Vim modelines everywhere as a precaution).

This file is added as a convenience for those who might have an editor that supports it, and its presence should have no impact on those that do not (want to) use it.
It also won't cause re-indentation of existing files when edited, only newly added lines would follow the convention.

No functional change.

Cc Christian Lindig <[email protected]>
---
 .editorconfig | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 .editorconfig
.editorconfig new file +20 -0
new file mode 100644
index 0000000000..cb2f27c581
@@ -0,0 +1,20 @@
+# See ./CODING_STYLE
+root = true
+
+[*]
+end_of_line = lf
+indent_style = space
+charset = utf-8
+max_line_length = 79
+trim_trailing_whitespace = true
+insert_final_newline = true
+
+# Makefiles must use tabs, otherwise they don't work
+[{Makefile,*.mk,Makefile.rules}]
+indent_style = tabs
+
+[*.{c,h}]
+indent_size = 4
+
+[*.{ml,mli}]
+indent_size = 2
Signature · 1 line
2.34.1
#5 lore raw

PATCH v4 3/11 CODING_STYLE(tools/ocaml): add 'make format' and remove tabs

In reply to Edwin Török

See CODING_STYLE: Xen uses spaces, not tabs.

* OCaml code:

Using ocp-indent for now to just make minimal modifications in tabs vs spaces and get the right indentation.
We can introduce ocamlformat later.

* C stubs:

just replace tabs with spaces now, using indent or clang-format would change code too much for 4.17.

This avoids perpetuating a formatting style that is inconsistent with the rest of Xen, and that makes preparing and submitting patches more difficult (OCaml indentation tools usually only support spaces, not tabs).

No functional change.

--
Reason for inclusion:
- avoid perpetuating a different coding style (I thought tabs were mandated by Xen, and was about to fix up my editor config to match when I realized Xen already mandates the use of spaces)
- should make submitting patches for OCaml easier (OCaml indentation tools know only about spaces, so I either can't use them, or have to manually adjust indentation every time I submit a patch)
- it can be verified that the only change here is the Makefile change for the new rule, 'git log -p -1 -w' should be otherwise empty

Changes since v3:
- this didn't make it into 4.17.0, we'll reconsider for 4.17.1, for now apply just to master which is open again
- separate introducing the rule from actual reformatting

Cc Christian Lindig <[email protected]>
---
 tools/ocaml/Makefile | 5 +++++
 1 file changed, 5 insertions(+)
tools/ocaml/Makefile +5 -0
index a7c04b6546..274ba15d75 100644
@@ -34,3 +34,8 @@ build-tools-oxenstored:
 	$(MAKE) -s -C libs/xb
 	$(MAKE) -s -C libs/xc
 	$(MAKE) -C xenstored
+
+.PHONY: format
+format:
+	git ls-files '*.ml' '*.mli' | xargs -n1 ocp-indent -i
+	git ls-files '*.c' '*.h' | xargs -n1 sed -ie 's/\t/    /g'
Signature · 1 line
2.34.1
#15 lore raw

PATCH v4 7/11 fixup! tools/ocaml/xenctrl: OCaml 5 support, fix use-after-free

In reply to Edwin Török

Fix unused value warning.

Fixes 8b3c06a3e5 ("tools/ocaml/xenctrl: OCaml 5 support, fix use-after-free")
Cc Andrew Cooper <[email protected]>

---
Changes:
* new patch

---
 tools/ocaml/libs/xc/xenctrl.ml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/ocaml/libs/xc/xenctrl.ml +1 -1
index b02be07429..a59dee0938 100644
@@ -182,7 +182,7 @@ let get_handle () = !handle
 
 let close_handle () =
   match !handle with
-  | Some h -> handle := None
+  | Some _ -> handle := None
   | None -> ()
 
 let with_intf f =
Signature · 1 line
2.34.1
#21 lore raw

PATCH v4 11/11 tools/ocaml/libs/xb: drop Xs_ring.write

In reply to Edwin Török

Unused, only Xs_ring.write_substring is used.
Also the bytes/string conversion here is backwards: the C stub implements the bytes version and then we use a Bytes.unsafe_of_string to convert a string into bytes.

However the operation here really is read-only: we read from the string and write it to the ring, so the C stub should implement the read-only string version, and if needed we could use Bytes.unsafe_to_string to be able to send 'bytes'. However that is not necessary as the 'bytes' version is dropped above.

---
Changes:
* new patch

---
 tools/ocaml/libs/xb/xs_ring.ml      | 5 +----
 tools/ocaml/libs/xb/xs_ring_stubs.c | 2 +-
 2 files changed, 2 insertions(+), 5 deletions(-)
tools/ocaml/libs/xb/xs_ring.ml +1 -4
index 2a27aa56c2..87c6b243e4 100644
@@ -25,14 +25,11 @@ module Server_features = Set.Make(struct
   end)
 
 external read: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_read"
-external write: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_write"
+external write_substring: Xenmmap.mmap_interface -> string -> int -> int = "ml_interface_write"
 
 external _internal_set_server_features: Xenmmap.mmap_interface -> int -> unit = "ml_interface_set_server_features" [@@noalloc]
 external _internal_get_server_features: Xenmmap.mmap_interface -> int = "ml_interface_get_server_features" [@@noalloc]
 
-let write_substring mmap buff len =
-  write mmap (Bytes.unsafe_of_string buff) len
-
 let get_server_features mmap =
   (* NB only one feature currently defined above *)
   let x = _internal_get_server_features mmap in
tools/ocaml/libs/xb/xs_ring_stubs.c +1 -1
index 28c79ee139..dca6059b0d 100644
@@ -119,7 +119,7 @@ CAMLprim value ml_interface_write(value ml_interface,
     CAMLlocal1(ml_result);
 
     struct mmap_interface *interface = GET_C_STRUCT(ml_interface);
-    const unsigned char *buffer = Bytes_val(ml_buffer);
+    const char *buffer = String_val(ml_buffer);
     int len = Int_val(ml_len);
     int result;
 
Signature · 1 line
2.34.1
#18 lore raw

PATCH v4 10/11 tools/ocaml/xenstored: validate config file before live update

In reply to Edwin Török

The configuration file can contain typos or various errors that could prevent live update from succeeding (e.g. a flag only valid on a different version). Unknown entries in the config file would be ignored on startup normally, add a strict --config-test that live-update can use to check that the config file is valid *for the new binary*.

For compatibility with running old code during live update recognize --live --help as an equivalent to --config-test.

---
Changes since v2:
* repost of lost patch from 2021: https://patchwork.kernel.org/project/xen-devel/patch/a53934dfa8ef984bffa858cc573cc7a6445bbdc0.1620755942.git.edvin.torok@citrix.com/

---
 tools/ocaml/xenstored/parse_arg.ml | 26 ++++++++++++++++++++++++++
 tools/ocaml/xenstored/xenstored.ml | 11 +++++++++--
 2 files changed, 35 insertions(+), 2 deletions(-)
tools/ocaml/xenstored/parse_arg.ml +26 -0
index 1a85b14ef5..b159b91f00 100644
@@ -26,8 +26,14 @@ type config =
     restart: bool;
     live_reload: bool;
     disable_socket: bool;
+    config_test: bool;
   }
 
+let get_config_filename config_file =
+  match config_file with
+  | Some name -> name
+  | None      -> Define.default_config_dir ^ "/oxenstored.conf"
+
 let do_argv =
   let pidfile = ref "" and tracefile = ref "" (* old xenstored compatibility *)
   and domain_init = ref true
@@ -38,6 +44,8 @@ let do_argv =
   and restart = ref false
   and live_reload = ref false
   and disable_socket = ref false
+  and config_test = ref false
+  and help = ref false
   in
 
   let speclist =
@@ -55,10 +63,27 @@ let do_argv =
       ("-T", Arg.Set_string tracefile, ""); (* for compatibility *)
       ("--restart", Arg.Set restart, "Read database on starting");
       ("--live", Arg.Set live_reload, "Read live dump on startup");
+      ("--config-test", Arg.Set config_test, "Test validity of config file");
       ("--disable-socket", Arg.Unit (fun () -> disable_socket := true), "Disable socket");
+      ("--help", Arg.Set help, "Display this list of options")
     ] in
   let usage_msg = "usage : xenstored [--config-file <filename>] [--no-domain-init] [--help] [--no-fork] [--reraise-top-level] [--restart] [--disable-socket]" in
   Arg.parse speclist (fun _ -> ()) usage_msg;
+  let () =
+    if !help then begin
+      if !live_reload then
+        (*
+          Transform --live --help into --config-test for backward compat with
+          running code during live update.
+          Caller will validate config and exit
+        *)
+        config_test := true
+      else begin
+        Arg.usage_string speclist usage_msg |> print_endline;
+        exit 0
+      end
+    end
+  in
   {
     domain_init = !domain_init;
     activate_access_log = !activate_access_log;
@@ -70,4 +95,5 @@ let do_argv =
     restart = !restart;
     live_reload = !live_reload;
     disable_socket = !disable_socket;
+    config_test = !config_test;
   }
tools/ocaml/xenstored/xenstored.ml +9 -2
index 366437b396..1aaa3e995e 100644
@@ -88,7 +88,7 @@ let default_pidfile = Paths.xen_run_dir ^ "/xenstored.pid"
 
 let ring_scan_interval = ref 20
 
-let parse_config filename =
+let parse_config ?(strict=false) filename =
   let pidfile = ref default_pidfile in
   let options = [
     ("merge-activate", Config.Set_bool Transaction.do_coalesce);
@@ -129,11 +129,12 @@ let parse_config filename =
     ("xenstored-port", Config.Set_string Domains.xenstored_port); ] in
   begin try Config.read filename options (fun _ _ -> raise Not_found)
     with
-    | Config.Error err -> List.iter (fun (k, e) ->
+    | Config.Error err as e -> List.iter (fun (k, e) ->
         match e with
         | "unknown key" -> eprintf "config: unknown key %s\n" k
         | _             -> eprintf "config: %s: %s\n" k e
       ) err;
+      if strict then raise e
     | Sys_error m -> eprintf "error: config: %s\n" m;
   end;
   !pidfile
@@ -358,6 +359,12 @@ let tweak_gc () =
 let () =
   Printexc.set_uncaught_exception_handler Logging.fallback_exception_handler;
   let cf = do_argv in
+  if cf.config_test then begin
+    let path = config_filename cf in
+    let _pidfile:string = parse_config ~strict:true path in
+    Printf.printf "Configuration valid at %s\n%!" path;
+    exit 0
+  end;
   let pidfile =
     if Sys.file_exists (config_filename cf) then
       parse_config (config_filename cf)
Signature · 1 line
2.34.1
#4 lore raw

PATCH v4 2/11 tools/ocaml/xenstored/Makefile: use ocamldep -sort for linking order

In reply to Edwin Török

A better solution is being worked on for master,
but for now use ocamldep -sort instead of a manually established link order.
The manually established link order will be wrong when (security) patches introduce new dependencies between files that would require changing the link order.

If dune was used as a build system this wouldn't be a problem, but we can't use Dune yet due to OSSTest, which is stuck on Debian oldstable.

No functional change.

---
Reason for inclusion in 4.17:

Avoids having to put this patch as a prerequisite into a security update. Earlier versions of XSA-326 needed this, and although latest version didn't we don't know whether it might be needed again in the future or not.

Changes since v2:
- new in v3 (was previously emailed to security team though)

Changes since v3:
- add Acked-by line

---
 tools/ocaml/xenstored/Makefile | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/ocaml/xenstored/Makefile +5 -1
index 6f7333926e..e8aaecf2e6 100644
@@ -72,7 +72,11 @@ XENSTOREDLIBS = \
 PROGRAMS = oxenstored
 
 oxenstored_LIBS = $(XENSTOREDLIBS)
-oxenstored_OBJS = $(OBJS)
+# use ocamldep to figure out link order, otherwise the Makefile would have
+# to be continously adjusted for security patches that introduce new
+# dependencies between files
+oxenstored_MLSORTED = $(shell $(OCAMLDEP) -sort $(OBJS:=.ml))
+oxenstored_OBJS = $(oxenstored_MLSORTED:.ml=)
 
 OCAML_PROGRAM = oxenstored
 
Signature · 1 line
2.34.1
#9 lore raw

PATCH v4 4/11 tools/ocaml: run "make format"

In reply to Edwin Török

From: Edwin Török <[email protected]>

git log -p -1 -w proves the change here is only whitespace.

No functional change.

---
Changes since v3
* split into its own commit to make it easier to refresh when rebasing

---
 tools/ocaml/libs/eventchn/xeneventchn_stubs.c |  194 +-
 tools/ocaml/libs/mmap/mmap_stubs.h            |    4 +-
 tools/ocaml/libs/mmap/xenmmap.ml              |    2 +-
 tools/ocaml/libs/mmap/xenmmap.mli             |    4 +-
 tools/ocaml/libs/mmap/xenmmap_stubs.c         |  112 +-
 tools/ocaml/libs/xb/op.ml                     |   76 +-
 tools/ocaml/libs/xb/packet.ml                 |   30 +-
 tools/ocaml/libs/xb/partial.ml                |   48 +-
 tools/ocaml/libs/xb/xb.ml                     |  416 ++--
 tools/ocaml/libs/xb/xb.mli                    |  106 +-
 tools/ocaml/libs/xb/xenbus_stubs.c            |   50 +-
 tools/ocaml/libs/xb/xs_ring.ml                |   28 +-
 tools/ocaml/libs/xb/xs_ring_stubs.c           |  214 +-
 tools/ocaml/libs/xc/xenctrl.ml                |  326 +--
 tools/ocaml/libs/xc/xenctrl.mli               |   12 +-
 tools/ocaml/libs/xc/xenctrl_stubs.c           | 1556 ++++++-------
 tools/ocaml/libs/xentoollog/caml_xentoollog.h |    6 +-
 .../ocaml/libs/xentoollog/xentoollog_stubs.c  |  196 +-
 tools/ocaml/libs/xl/xenlight_stubs.c          | 2022 ++++++++---------
 tools/ocaml/libs/xs/queueop.ml                |   48 +-
 tools/ocaml/libs/xs/xs.ml                     |  220 +-
 tools/ocaml/libs/xs/xs.mli                    |   46 +-
 tools/ocaml/libs/xs/xsraw.ml                  |  300 +--
 tools/ocaml/libs/xs/xst.ml                    |   76 +-
 tools/ocaml/libs/xs/xst.mli                   |   20 +-
 tools/ocaml/test/dmesg.ml                     |   26 +-
 tools/ocaml/test/list_domains.ml              |    4 +-
 tools/ocaml/test/raise_exception.ml           |    4 +-
 tools/ocaml/test/xtl.ml                       |   28 +-
 tools/ocaml/xenstored/config.ml               |  156 +-
 tools/ocaml/xenstored/connection.ml           |  578 ++---
 tools/ocaml/xenstored/connections.ml          |  294 +--
 tools/ocaml/xenstored/disk.ml                 |  218 +-
 tools/ocaml/xenstored/domain.ml               |  116 +-
 tools/ocaml/xenstored/domains.ml              |  298 +--
 tools/ocaml/xenstored/event.ml                |   28 +-
 tools/ocaml/xenstored/history.ml              |   62 +-
 tools/ocaml/xenstored/logging.ml              |  478 ++--
 tools/ocaml/xenstored/packet.ml               |   20 +-
 tools/ocaml/xenstored/parse_arg.ml            |  106 +-
 tools/ocaml/xenstored/perms.ml                |  216 +-
 tools/ocaml/xenstored/poll.ml                 |   68 +-
 tools/ocaml/xenstored/poll.mli                |    4 +-
 tools/ocaml/xenstored/process.ml              | 1210 +++++-----
 tools/ocaml/xenstored/quota.ml                |   74 +-
 tools/ocaml/xenstored/select_stubs.c          |   62 +-
 tools/ocaml/xenstored/stdext.ml               |  190 +-
 tools/ocaml/xenstored/store.ml                |  752 +++---
 tools/ocaml/xenstored/symbol.ml               |    2 +-
 tools/ocaml/xenstored/syslog.ml               |   48 +-
 tools/ocaml/xenstored/syslog_stubs.c          |   68 +-
 tools/ocaml/xenstored/systemd_stubs.c         |   10 +-
 tools/ocaml/xenstored/transaction.ml          |  352 +--
 tools/ocaml/xenstored/trie.ml                 |  222 +-
 tools/ocaml/xenstored/trie.mli                |   22 +-
 tools/ocaml/xenstored/utils.ml                |  146 +-
 tools/ocaml/xenstored/xenstored.ml            |  978 ++++----
 57 files changed, 6476 insertions(+), 6476 deletions(-)
tools/ocaml/libs/eventchn/xeneventchn_stubs.c +97 -97
index aa8a69cc1e..707406cbd1 100644
@@ -37,171 +37,171 @@
 
 static void stub_evtchn_finalize(value v)
 {
-	xenevtchn_close(_H(v));
+    xenevtchn_close(_H(v));
 }
 
 static struct custom_operations xenevtchn_ops = {
-	.identifier  = "xenevtchn",
-	.finalize    = stub_evtchn_finalize,
-	.compare     = custom_compare_default,     /* Can't compare     */
-	.hash        = custom_hash_default,        /* Can't hash        */
-	.serialize   = custom_serialize_default,   /* Can't serialize   */
-	.deserialize = custom_deserialize_default, /* Can't deserialize */
-	.compare_ext = custom_compare_ext_default, /* Can't compare     */
+    .identifier  = "xenevtchn",
+    .finalize    = stub_evtchn_finalize,
+    .compare     = custom_compare_default,     /* Can't compare     */
+    .hash        = custom_hash_default,        /* Can't hash        */
+    .serialize   = custom_serialize_default,   /* Can't serialize   */
+    .deserialize = custom_deserialize_default, /* Can't deserialize */
+    .compare_ext = custom_compare_ext_default, /* Can't compare     */
 };
 
 CAMLprim value stub_eventchn_init(value cloexec)
 {
-	CAMLparam1(cloexec);
-	CAMLlocal1(result);
-	xenevtchn_handle *xce;
-	unsigned int flags = 0;
+    CAMLparam1(cloexec);
+    CAMLlocal1(result);
+    xenevtchn_handle *xce;
+    unsigned int flags = 0;
 
-	if ( !Bool_val(cloexec) )
-		flags |= XENEVTCHN_NO_CLOEXEC;
+    if ( !Bool_val(cloexec) )
+        flags |= XENEVTCHN_NO_CLOEXEC;
 
-	caml_enter_blocking_section();
-	xce = xenevtchn_open(NULL, flags);
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    xce = xenevtchn_open(NULL, flags);
+    caml_leave_blocking_section();
 
-	if (xce == NULL)
-		caml_failwith("open failed");
+    if (xce == NULL)
+        caml_failwith("open failed");
 
-	result = caml_alloc_custom(&xenevtchn_ops, sizeof(xce), 0, 1);
-	_H(result) = xce;
+    result = caml_alloc_custom(&xenevtchn_ops, sizeof(xce), 0, 1);
+    _H(result) = xce;
 
-	CAMLreturn(result);
+    CAMLreturn(result);
 }
 
 CAMLprim value stub_eventchn_fdopen(value fdval)
 {
-	CAMLparam1(fdval);
-	CAMLlocal1(result);
-	xenevtchn_handle *xce;
+    CAMLparam1(fdval);
+    CAMLlocal1(result);
+    xenevtchn_handle *xce;
 
-	caml_enter_blocking_section();
-	xce = xenevtchn_fdopen(NULL, Int_val(fdval), 0);
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    xce = xenevtchn_fdopen(NULL, Int_val(fdval), 0);
+    caml_leave_blocking_section();
 
-	if (xce == NULL)
-		caml_failwith("evtchn fdopen failed");
+    if (xce == NULL)
+        caml_failwith("evtchn fdopen failed");
 
-	result = caml_alloc_custom(&xenevtchn_ops, sizeof(xce), 0, 1);
-	_H(result) = xce;
+    result = caml_alloc_custom(&xenevtchn_ops, sizeof(xce), 0, 1);
+    _H(result) = xce;
 
-	CAMLreturn(result);
+    CAMLreturn(result);
 }
 
 CAMLprim value stub_eventchn_fd(value xce)
 {
-	CAMLparam1(xce);
-	CAMLlocal1(result);
-	int fd;
+    CAMLparam1(xce);
+    CAMLlocal1(result);
+    int fd;
 
-	fd = xenevtchn_fd(_H(xce));
-	if (fd == -1)
-		caml_failwith("evtchn fd failed");
+    fd = xenevtchn_fd(_H(xce));
+    if (fd == -1)
+        caml_failwith("evtchn fd failed");
 
-	result = Val_int(fd);
+    result = Val_int(fd);
 
-	CAMLreturn(result);
+    CAMLreturn(result);
 }
 
 CAMLprim value stub_eventchn_notify(value xce, value port)
 {
-	CAMLparam2(xce, port);
-	int rc;
+    CAMLparam2(xce, port);
+    int rc;
 
-	caml_enter_blocking_section();
-	rc = xenevtchn_notify(_H(xce), Int_val(port));
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    rc = xenevtchn_notify(_H(xce), Int_val(port));
+    caml_leave_blocking_section();
 
-	if (rc == -1)
-		caml_failwith("evtchn notify failed");
+    if (rc == -1)
+        caml_failwith("evtchn notify failed");
 
-	CAMLreturn(Val_unit);
+    CAMLreturn(Val_unit);
 }
 
 CAMLprim value stub_eventchn_bind_interdomain(value xce, value domid,
                                               value remote_port)
 {
-	CAMLparam3(xce, domid, remote_port);
-	CAMLlocal1(port);
-	xenevtchn_port_or_error_t rc;
+    CAMLparam3(xce, domid, remote_port);
+    CAMLlocal1(port);
+    xenevtchn_port_or_error_t rc;
 
-	caml_enter_blocking_section();
-	rc = xenevtchn_bind_interdomain(_H(xce), Int_val(domid), Int_val(remote_port));
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    rc = xenevtchn_bind_interdomain(_H(xce), Int_val(domid), Int_val(remote_port));
+    caml_leave_blocking_section();
 
-	if (rc == -1)
-		caml_failwith("evtchn bind_interdomain failed");
-	port = Val_int(rc);
+    if (rc == -1)
+        caml_failwith("evtchn bind_interdomain failed");
+    port = Val_int(rc);
 
-	CAMLreturn(port);
+    CAMLreturn(port);
 }
 
 CAMLprim value stub_eventchn_bind_virq(value xce, value virq_type)
 {
-	CAMLparam2(xce, virq_type);
-	CAMLlocal1(port);
-	xenevtchn_port_or_error_t rc;
+    CAMLparam2(xce, virq_type);
+    CAMLlocal1(port);
+    xenevtchn_port_or_error_t rc;
 
-	caml_enter_blocking_section();
-	rc = xenevtchn_bind_virq(_H(xce), Int_val(virq_type));
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    rc = xenevtchn_bind_virq(_H(xce), Int_val(virq_type));
+    caml_leave_blocking_section();
 
-	if (rc == -1)
-		caml_failwith("evtchn bind_virq failed");
-	port = Val_int(rc);
+    if (rc == -1)
+        caml_failwith("evtchn bind_virq failed");
+    port = Val_int(rc);
 
-	CAMLreturn(port);
+    CAMLreturn(port);
 }
 
 CAMLprim value stub_eventchn_unbind(value xce, value port)
 {
-	CAMLparam2(xce, port);
-	int rc;
+    CAMLparam2(xce, port);
+    int rc;
 
-	caml_enter_blocking_section();
-	rc = xenevtchn_unbind(_H(xce), Int_val(port));
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    rc = xenevtchn_unbind(_H(xce), Int_val(port));
+    caml_leave_blocking_section();
 
-	if (rc == -1)
-		caml_failwith("evtchn unbind failed");
+    if (rc == -1)
+        caml_failwith("evtchn unbind failed");
 
-	CAMLreturn(Val_unit);
+    CAMLreturn(Val_unit);
 }
 
 CAMLprim value stub_eventchn_pending(value xce)
 {
-	CAMLparam1(xce);
-	CAMLlocal1(result);
-	xenevtchn_port_or_error_t port;
+    CAMLparam1(xce);
+    CAMLlocal1(result);
+    xenevtchn_port_or_error_t port;
 
-	caml_enter_blocking_section();
-	port = xenevtchn_pending(_H(xce));
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    port = xenevtchn_pending(_H(xce));
+    caml_leave_blocking_section();
 
-	if (port == -1)
-		caml_failwith("evtchn pending failed");
-	result = Val_int(port);
+    if (port == -1)
+        caml_failwith("evtchn pending failed");
+    result = Val_int(port);
 
-	CAMLreturn(result);
+    CAMLreturn(result);
 }
 
 CAMLprim value stub_eventchn_unmask(value xce, value _port)
 {
-	CAMLparam2(xce, _port);
-	evtchn_port_t port;
-	int rc;
+    CAMLparam2(xce, _port);
+    evtchn_port_t port;
+    int rc;
 
-	port = Int_val(_port);
+    port = Int_val(_port);
 
-	caml_enter_blocking_section();
-	rc = xenevtchn_unmask(_H(xce), port);
-	caml_leave_blocking_section();
+    caml_enter_blocking_section();
+    rc = xenevtchn_unmask(_H(xce), port);
+    caml_leave_blocking_section();
 
-	if (rc)
-		caml_failwith("evtchn unmask failed");
-	CAMLreturn(Val_unit);
+    if (rc)
+        caml_failwith("evtchn unmask failed");
+    CAMLreturn(Val_unit);
 }
tools/ocaml/libs/mmap/mmap_stubs.h +2 -2
index 66f18d4406..df22631c72 100644
@@ -26,8 +26,8 @@
 
 struct mmap_interface
 {
-	void *addr;
-	int len;
+    void *addr;
+    int len;
 };
 
 #ifndef Data_abstract_val
tools/ocaml/libs/mmap/xenmmap.ml +1 -1
index 44b67c89d2..fd6735649f 100644
@@ -21,7 +21,7 @@ type mmap_map_flag = SHARED | PRIVATE
 
 (* mmap: fd -> prot_flag -> map_flag -> length -> offset -> interface *)
 external mmap: Unix.file_descr -> mmap_prot_flag -> mmap_map_flag
-		-> int -> int -> mmap_interface = "stub_mmap_init"
+  -> int -> int -> mmap_interface = "stub_mmap_init"
 external unmap: mmap_interface -> unit = "stub_mmap_final"
 (* read: interface -> start -> length -> data *)
 external read: mmap_interface -> int -> int -> string = "stub_mmap_read"
tools/ocaml/libs/mmap/xenmmap.mli +2 -2
index 8f92ed6310..d097b68a8f 100644
@@ -19,10 +19,10 @@ type mmap_prot_flag = RDONLY | WRONLY | RDWR
 type mmap_map_flag = SHARED | PRIVATE
 
 external mmap : Unix.file_descr -> mmap_prot_flag -> mmap_map_flag -> int -> int
-             -> mmap_interface = "stub_mmap_init"
+  -> mmap_interface = "stub_mmap_init"
 external unmap : mmap_interface -> unit = "stub_mmap_final"
 external read : mmap_interface -> int -> int -> string = "stub_mmap_read"
 external write : mmap_interface -> string -> int -> int -> unit
-               = "stub_mmap_write"
+  = "stub_mmap_write"
 
 external getpagesize : unit -> int = "stub_mmap_getpagesize"
tools/ocaml/libs/mmap/xenmmap_stubs.c +56 -56

153 lines not on this page · raw message

tools/ocaml/libs/xb/op.ml +38 -38

88 lines not on this page · raw message

tools/ocaml/libs/xb/packet.ml +15 -15
index 74c04bb7ae..cd169c066b 100644
@@ -15,12 +15,12 @@
  *)
 
 type t =
-{
-	tid: int;
-	rid: int;
-	ty: Op.operation;
-	data: string;
-}
+  {
+    tid: int;
+    rid: int;
+    ty: Op.operation;
+    data: string;
+  }
 
 exception Error of string
 exception DataError of string
@@ -30,21 +30,21 @@ external string_of_header: int -> int -> int -> int -> string = "stub_string_of_
 let create tid rid ty data = { tid = tid; rid = rid; ty = ty; data = data; }
 
 let of_partialpkt ppkt =
-	create ppkt.Partial.tid ppkt.Partial.rid ppkt.Partial.ty (Buffer.contents ppkt.Partial.buf)
+  create ppkt.Partial.tid ppkt.Partial.rid ppkt.Partial.ty (Buffer.contents ppkt.Partial.buf)
 
 let to_string pkt =
-	let header = string_of_header pkt.tid pkt.rid (Op.to_cval pkt.ty) (String.length pkt.data) in
-	header ^ pkt.data
+  let header = string_of_header pkt.tid pkt.rid (Op.to_cval pkt.ty) (String.length pkt.data) in
+  header ^ pkt.data
 
 let unpack pkt =
-	pkt.tid, pkt.rid, pkt.ty, pkt.data
+  pkt.tid, pkt.rid, pkt.ty, pkt.data
 
 let get_tid pkt = pkt.tid
 let get_ty pkt = pkt.ty
 let get_data pkt =
-	let l = String.length pkt.data in
-	if l > 0 && pkt.data.[l - 1] = '\000' then
-		String.sub pkt.data 0 (l - 1)
-	else
-		pkt.data
+  let l = String.length pkt.data in
+  if l > 0 && pkt.data.[l - 1] = '\000' then
+    String.sub pkt.data 0 (l - 1)
+  else
+    pkt.data
 let get_rid pkt = pkt.rid
\ No newline at end of file
tools/ocaml/libs/xb/partial.ml +24 -24

63 lines not on this page · raw message

tools/ocaml/libs/xb/xb.ml +208 -208

512 lines not on this page · raw message

tools/ocaml/libs/xb/xb.mli +53 -53

111 lines not on this page · raw message

tools/ocaml/libs/xb/xenbus_stubs.c +25 -25

65 lines not on this page · raw message

tools/ocaml/libs/xb/xs_ring.ml +14 -14
index db7f86bd27..2a27aa56c2 100644
@@ -15,14 +15,14 @@
  *)
 
 module Server_feature = struct
-	type t =
-	| Reconnection
+  type t =
+    | Reconnection
 end
 
 module Server_features = Set.Make(struct
-	type t = Server_feature.t
-	let compare = compare
-end)
+    type t = Server_feature.t
+    let compare = compare
+  end)
 
 external read: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_read"
 external write: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_write"
@@ -31,18 +31,18 @@ external _internal_set_server_features: Xenmmap.mmap_interface -> int -> unit =
 external _internal_get_server_features: Xenmmap.mmap_interface -> int = "ml_interface_get_server_features" [@@noalloc]
 
 let write_substring mmap buff len =
-	write mmap (Bytes.unsafe_of_string buff) len
+  write mmap (Bytes.unsafe_of_string buff) len
 
 let get_server_features mmap =
-	(* NB only one feature currently defined above *)
-	let x = _internal_get_server_features mmap in
-	if x = 0
-	then Server_features.empty
-	else Server_features.singleton Server_feature.Reconnection
+  (* NB only one feature currently defined above *)
+  let x = _internal_get_server_features mmap in
+  if x = 0
+  then Server_features.empty
+  else Server_features.singleton Server_feature.Reconnection
 
 let set_server_features mmap set =
-	(* NB only one feature currently defined above *)
-	let x = if set = Server_features.empty then 0 else 1 in
-	_internal_set_server_features mmap x
+  (* NB only one feature currently defined above *)
+  let x = if set = Server_features.empty then 0 else 1 in
+  _internal_set_server_features mmap x
 
 external close: Xenmmap.mmap_interface -> unit = "ml_interface_close" [@@noalloc]
tools/ocaml/libs/xb/xs_ring_stubs.c +107 -107

260 lines not on this page · raw message

tools/ocaml/libs/xc/xenctrl.ml +163 -163

450 lines not on this page · raw message

tools/ocaml/libs/xc/xenctrl.mli +6 -6
index 82def5a17c..3154e90f4f 100644
@@ -182,11 +182,11 @@ external domain_getinfo : handle -> domid -> domaininfo
 external domain_get_vcpuinfo : handle -> int -> int -> vcpuinfo
   = "stub_xc_vcpu_getinfo"
 external domain_ioport_permission: handle -> domid -> int -> int -> bool -> unit
-       = "stub_xc_domain_ioport_permission"
+  = "stub_xc_domain_ioport_permission"
 external domain_iomem_permission: handle -> domid -> nativeint -> nativeint -> bool -> unit
-       = "stub_xc_domain_iomem_permission"
+  = "stub_xc_domain_iomem_permission"
 external domain_irq_permission: handle -> domid -> int -> bool -> unit
-       = "stub_xc_domain_irq_permission"
+  = "stub_xc_domain_irq_permission"
 external vcpu_affinity_set : handle -> domid -> int -> bool array -> unit
   = "stub_xc_vcpu_setaffinity"
 external vcpu_affinity_get : handle -> domid -> int -> bool array
@@ -284,11 +284,11 @@ external hvm_param_set: handle -> domid -> hvm_param -> int64 -> unit
   = "stub_xc_hvm_param_set"
 
 external domain_assign_device: handle -> domid -> (int * int * int * int) -> unit
-       = "stub_xc_domain_assign_device"
+  = "stub_xc_domain_assign_device"
 external domain_deassign_device: handle -> domid -> (int * int * int * int) -> unit
-       = "stub_xc_domain_deassign_device"
+  = "stub_xc_domain_deassign_device"
 external domain_test_assign_device: handle -> domid -> (int * int * int * int) -> bool
-       = "stub_xc_domain_test_assign_device"
+  = "stub_xc_domain_test_assign_device"
 
 external version : handle -> version = "stub_xc_version_version"
 external version_compile_info : handle -> compile_info
tools/ocaml/libs/xc/xenctrl_stubs.c +778 -778

877 lines not on this page · raw message

tools/ocaml/libs/xentoollog/caml_xentoollog.h +3 -3
index 0eb7618512..5ad4a4d5b2 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/libs/xentoollog/xentoollog_stubs.c +98 -98

196 lines not on this page · raw message

tools/ocaml/libs/xl/xenlight_stubs.c +1011 -1011

2,022 lines not on this page · raw message

tools/ocaml/libs/xs/queueop.ml +24 -24

48 lines not on this page · raw message

tools/ocaml/libs/xs/xs.ml +110 -110

220 lines not on this page · raw message

tools/ocaml/libs/xs/xs.mli +23 -23

46 lines not on this page · raw message

tools/ocaml/libs/xs/xsraw.ml +150 -150

300 lines not on this page · raw message

tools/ocaml/libs/xs/xst.ml +38 -38

76 lines not on this page · raw message

tools/ocaml/libs/xs/xst.mli +10 -10
index 5ae560424c..08d737bc36 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/test/dmesg.ml +13 -13
index c868024c52..f9efe5dc30 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/test/list_domains.ml +2 -2
index c8974957fd..94f1cec050 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/test/raise_exception.ml +2 -2
index 5ef7fc0f13..8c24c3555b 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/test/xtl.ml +14 -14
index 5f94a76a82..458b11bbaa 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/config.ml +78 -78

156 lines not on this page · raw message

tools/ocaml/xenstored/connection.ml +289 -289

578 lines not on this page · raw message

tools/ocaml/xenstored/connections.ml +147 -147

294 lines not on this page · raw message

tools/ocaml/xenstored/disk.ml +109 -109

218 lines not on this page · raw message

tools/ocaml/xenstored/domain.ml +58 -58

116 lines not on this page · raw message

tools/ocaml/xenstored/domains.ml +149 -149

298 lines not on this page · raw message

tools/ocaml/xenstored/event.ml +14 -14
index 629dc6041b..3523c397f4 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/history.ml +31 -31

62 lines not on this page · raw message

tools/ocaml/xenstored/logging.ml +239 -239

478 lines not on this page · raw message

tools/ocaml/xenstored/packet.ml +10 -10
index aeae0a4f57..7c25117937 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/parse_arg.ml +53 -53

106 lines not on this page · raw message

tools/ocaml/xenstored/perms.ml +108 -108

216 lines not on this page · raw message

tools/ocaml/xenstored/poll.ml +34 -34

68 lines not on this page · raw message

tools/ocaml/xenstored/poll.mli +2 -2
index f73465b99f..e759608b1a 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/process.ml +605 -605

1,210 lines not on this page · raw message

tools/ocaml/xenstored/quota.ml +37 -37

74 lines not on this page · raw message

tools/ocaml/xenstored/select_stubs.c +31 -31

62 lines not on this page · raw message

tools/ocaml/xenstored/stdext.ml +95 -95

190 lines not on this page · raw message

tools/ocaml/xenstored/store.ml +376 -376

752 lines not on this page · raw message

tools/ocaml/xenstored/symbol.ml +1 -1
index 72a84ebf80..02298a04ca 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/syslog.ml +24 -24

48 lines not on this page · raw message

tools/ocaml/xenstored/syslog_stubs.c +34 -34

68 lines not on this page · raw message

tools/ocaml/xenstored/systemd_stubs.c +5 -5
index f4c875075a..f36f7300cf 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/transaction.ml +176 -176

352 lines not on this page · raw message

tools/ocaml/xenstored/trie.ml +111 -111

222 lines not on this page · raw message

tools/ocaml/xenstored/trie.mli +11 -11
index 27785154f5..2f1b364b6a 100644

Lines of this file are not shown. View the raw message.

tools/ocaml/xenstored/utils.ml +73 -73

146 lines not on this page · raw message

tools/ocaml/xenstored/xenstored.ml +489 -489

978 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.34.1

Part of the diff is not shown here. View the raw message.

#2 lore raw

Re: [PATCH v4 01/11] tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper

In reply to Edwin Török

tools/ocaml/libs/mmap/mmap_stubs.h struct mmap_interface · line 34

Quoted 10 lines · Edwin Török
tools/ocaml/libs/mmap/mmap_stubs.h +2 -0
index 65e4239890..66f18d4406 100644
@@ -30,4 +30,8 @@ struct mmap_interface
 	int len;
 };
 
+#ifndef Data_abstract_val
+#define Data_abstract_val(x) ((void*)(value*)(x))

((void *)(x))

I take it this has come from the Ocaml headers?  The cast to (value *) in the middle is entirely cancelled out.

Can be fixed on commit.

~Andrew

Acked-by on v4 10/11
#19 lore raw

Re: [PATCH v4 10/11] tools/ocaml/xenstored: validate config file before live update

In reply to Edwin Török
Quoted 9 lines · Edwin Török

The configuration file can contain typos or various errors that could prevent live update from succeeding (e.g. a flag only valid on a different version). Unknown entries in the config file would be ignored on startup normally, add a strict --config-test that live-update can use to check that the config file is valid *for the new binary*.

Is the configuration tested, checked, or validated? If feel “check" or “validate" would convey better what is happening.

Quoted 4 lines

For compatibility with running old code during live update recognize --live --help as an equivalent to --config-test.

Quoted 14 lines
Quoted 1 line

---
Changes since v2:
* repost of lost patch from 2021: https://patchwork.kernel.org/project/xen-devel/patch/a53934dfa8ef984bffa858cc573cc7a6445bbdc0.1620755942.git.edvin.torok@citrix.com/
---
tools/ocaml/xenstored/parse_arg.ml | 26 ++++++++++++++++++++++++++
tools/ocaml/xenstored/xenstored.ml | 11 +++++++++--

2 files changed, 35 insertions(+), 2 deletions(-)
tools/ocaml/xenstored/parse_arg.ml +0 -0
index 1a85b14ef5..b159b91f00 100644

tools/ocaml/xenstored/parse_arg.ml type config = · line 35

Quoted 10 lines
tools/ocaml/xenstored/parse_arg.ml +5 -0
@@ -26,8 +26,14 @@ type config =
    restart: bool;
    live_reload: bool;
    disable_socket: bool;
+    config_test: bool;
  }
 
+let get_config_filename config_file =
+  match config_file with
+  | Some name -> name
+  | None      -> Define.default_config_dir ^ "/oxenstored.conf"

I’d use Filename.concat.

Quoted 14 lines
tools/ocaml/xenstored/parse_arg.ml +1 -0
+

let do_argv =
let pidfile = ref "" and tracefile = ref "" (* old xenstored compatibility *) and domain_init = ref true

tools/ocaml/xenstored/parse_arg.ml +2 -0
@@ -38,6 +44,8 @@ let do_argv =
  and restart = ref false
  and live_reload = ref false
  and disable_socket = ref false
+  and config_test = ref false
+  and help = ref false
  in
 
  let speclist =

tools/ocaml/xenstored/parse_arg.ml let do_argv = · line 66

Quoted 4 lines
tools/ocaml/xenstored/parse_arg.ml +1 -0
@@ -55,10 +63,27 @@ let do_argv =
      ("-T", Arg.Set_string tracefile, ""); (* for compatibility *)
      ("--restart", Arg.Set restart, "Read database on starting");
      ("--live", Arg.Set live_reload, "Read live dump on startup");
+      ("--config-test", Arg.Set config_test, "Test validity of config file");

I see the logic how this flag was named but feel starting with a verb (“validate”, “check”, “test”) leads to a clearer invocation pattern.

Quoted 72 lines
tools/ocaml/xenstored/parse_arg.ml +17 -0
      ("--disable-socket", Arg.Unit (fun () -> disable_socket := true), "Disable socket");
+      ("--help", Arg.Set help, "Display this list of options")
    ] in
  let usage_msg = "usage : xenstored [--config-file <filename>] [--no-domain-init] [--help] [--no-fork] [--reraise-top-level] [--restart] [--disable-socket]" in
  Arg.parse speclist (fun _ -> ()) usage_msg;
+  let () =
+    if !help then begin
+      if !live_reload then
+        (*
+          Transform --live --help into --config-test for backward compat with
+          running code during live update.
+          Caller will validate config and exit
+        *)
+        config_test := true
+      else begin
+        Arg.usage_string speclist usage_msg |> print_endline;
+        exit 0
+      end
+    end
+  in
  {
    domain_init = !domain_init;
    activate_access_log = !activate_access_log;
@@ -70,4 +95,5 @@ let do_argv =
    restart = !restart;
    live_reload = !live_reload;
    disable_socket = !disable_socket;
+    config_test = !config_test;
  }
tools/ocaml/xenstored/xenstored.ml +0 -0
index 366437b396..1aaa3e995e 100644
@@ -88,7 +88,7 @@ let default_pidfile = Paths.xen_run_dir ^ "/xenstored.pid"
 

let ring_scan_interval = ref 20

tools/ocaml/xenstored/xenstored.ml +3 -2
-let parse_config filename =
+let parse_config ?(strict=false) filename =
  let pidfile = ref default_pidfile in
  let options = [
    ("merge-activate", Config.Set_bool Transaction.do_coalesce);
@@ -129,11 +129,12 @@ let parse_config filename =
    ("xenstored-port", Config.Set_string Domains.xenstored_port); ] in
  begin try Config.read filename options (fun _ _ -> raise Not_found)
    with
-    | Config.Error err -> List.iter (fun (k, e) ->
+    | Config.Error err as e -> List.iter (fun (k, e) ->
        match e with
        | "unknown key" -> eprintf "config: unknown key %s\n" k
        | _             -> eprintf "config: %s: %s\n" k e
      ) err;
+      if strict then raise e
    | Sys_error m -> eprintf "error: config: %s\n" m;
  end;
  !pidfile
@@ -358,6 +359,12 @@ let tweak_gc () =

let () =
Printexc.set_uncaught_exception_handler Logging.fallback_exception_handler; let cf = do_argv in

tools/ocaml/xenstored/xenstored.ml +6 -0
+  if cf.config_test then begin
+    let path = config_filename cf in
+    let _pidfile:string = parse_config ~strict:true path in
+    Printf.printf "Configuration valid at %s\n%!" path;
+    exit 0
+  end;
  let pidfile =
    if Sys.file_exists (config_filename cf) then
      parse_config (config_filename cf)
Signature · 1 line
2.34.1
Acked-by on v4 3/11
#6 lore raw

Re: [PATCH v4 03/11] CODING_STYLE(tools/ocaml): add 'make format' and remove tabs

In reply to Edwin Török
Quoted 17 lines · Edwin Török

See CODING_STYLE: Xen uses spaces, not tabs.

* OCaml code:

Using ocp-indent for now to just make minimal modifications in tabs vs spaces and get the right indentation.
We can introduce ocamlformat later.

* C stubs:

just replace tabs with spaces now, using indent or clang-format would change code too much for 4.17.

Quoted 7 lines

This avoids perpetuating a formatting style that is inconsistent with the rest of Xen, and that makes preparing and submitting patches more difficult (OCaml indentation tools usually only support spaces, not tabs).

No functional change.

I am in favour of automating the code formatting, and moving to OCamlformat in particular. The main drawback is that it can make back porting more difficult because formatting can change not just indentation but placement of parentheses and similar grouping syntax as well, leading to changes between the current code and older code that have to me resolved manually.

— C

Acked-by on v4 5/11
#11 lore raw

Re: [PATCH v4 05/11] CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation uses spaces

In reply to Edwin Török
Quoted 30 lines · Edwin Török

Add an .editorconfig to make it easier to keep patches compatible with Xen's coding style, and to reemphasize what Xen's coding style is.

I thought that Xen demands tabs rather than spaces (which is more difficult with OCaml because indentation tools use spaces, and the use of tabs requires changing editor settings), however CODING-STYLE says it is spaces.

Document this explicitly by adding a .editorconfig file (see editorconfig.org), which is an editor agnostic format for specifying basic style properties like indentation, either with native support in editors or via plugins.

It is safer than modelines because it only supports controlling a restricted set of editor properties and not arbitrary commands as Vim modelines would have, and works with editors other than Vim too. (Vim has a deny list for modeline sandboxing, which is error-prone because every time a new command gets added it needs to be added to the deny list, which has been the source of a few CVEs in the past and I disable Vim modelines everywhere as a precaution).

This file is added as a convenience for those who might have an editor that supports it, and its presence should have no impact on those that do not (want to) use it.
It also won't cause re-indentation of existing files when edited, only newly added lines would follow the convention.

Quoted 4 lines

No functional change.

Cc Christian Lindig <[email protected]>

I am agnostic about this change. Automatic code formatting should take care of this but I see no harm including this file. The number of developers contributing to the OCaml code base is minuscule and we should try to keep barriers as low as possible. The wrangling about all these minutiae is already too much.

- C

#3 lore raw

Re: [PATCH v4 01/11] tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper

In reply to Andrew Cooper
Quoted 18 lines · Andrew Cooper

tools/ocaml/libs/mmap/mmap_stubs.h struct mmap_interface · line 34

Quoted 10 lines · Edwin Török
tools/ocaml/libs/mmap/mmap_stubs.h +2 -0
index 65e4239890..66f18d4406 100644
@@ -30,4 +30,8 @@ struct mmap_interface
 int len;
 };
 
+#ifndef Data_abstract_val
+#define Data_abstract_val(x) ((void*)(value*)(x))

((void *)(x))

Quoted 5 lines

I take it this has come from the Ocaml headers? The cast to (value *) in the middle is entirely cancelled out.

Can be fixed on commit.

In the OCaml headers it looks like:
+#define Data_abstract_val(v) ((void*) Op_val(v))

where Op_val is:
+#define Op_val(x) ((value *) (x))

However I haven't realized that Op_val has been in OCaml for a long time (at least 1995), so we can safely use it, and do this instead:

tools/ocaml/libs/mmap/mmap_stubs.h +4 -0
index 65e4239890..6c33f14138 100644
@@ -30,4 +30,8 @@ struct mmap_interface
    int len;
 };
 
+#ifndef Data_abstract_val
+#define Data_abstract_val(v) ((void*) Op_val(v))
+#endif
+
 #endif

There is an updated commit here for convenience: https://github.com/edwintorok/xen/commit/9855ed237f3f85ac30972bfd0a601c6746ba2353

Best regards,
--Edwin

#20 lore raw

Re: [PATCH v4 10/11] tools/ocaml/xenstored: validate config file before live update

In reply to Christian Lindig
Quoted 8 lines · Christian Lindig
Quoted 5 lines · Edwin Török
Quoted 7 lines
Quoted 5 lines

The configuration file can contain typos or various errors that could prevent live update from succeeding (e.g. a flag only valid on a different version). Unknown entries in the config file would be ignored on startup normally, add a strict --config-test that live-update can use to check that the config file is valid *for the new binary*.

Is the configuration tested, checked, or validated? If feel “check" or “validate" would convey better what is happening.

The rest of the code talks about validation, so I've renamed this to config-validate.

Quoted 8 lines
Quoted 6 lines

For compatibility with running old code during live update recognize --live --help as an equivalent to --config-test.

Thanks, I've pushed an update commit with this change here: https://github.com/edwintorok/xen/compare/staging...private/edvint/review5,
in particular https://github.com/edwintorok/xen/commit/f1a9153bb867bbb5df0f5e17b1ed3348e7ea79f8

Best regards,
--Edwin

Quoted 122 lines
Quoted 15 lines
Quoted 2 lines

---
Changes since v2:
* repost of lost patch from 2021: https://patchwork.kernel.org/project/xen-devel/patch/a53934dfa8ef984bffa858cc573cc7a6445bbdc0.1620755942.git.edvin.torok@citrix.com/
---
tools/ocaml/xenstored/parse_arg.ml | 26 ++++++++++++++++++++++++++
tools/ocaml/xenstored/xenstored.ml | 11 +++++++++--

2 files changed, 35 insertions(+), 2 deletions(-)
tools/ocaml/xenstored/parse_arg.ml +0 -0
index 1a85b14ef5..b159b91f00 100644

tools/ocaml/xenstored/parse_arg.ml type config = · line 35

Quoted 10 lines
tools/ocaml/xenstored/parse_arg.ml +5 -0
@@ -26,8 +26,14 @@ type config =
   restart: bool;
   live_reload: bool;
   disable_socket: bool;
+    config_test: bool;
 }
 
+let get_config_filename config_file =
+  match config_file with
+  | Some name -> name
+  | None      -> Define.default_config_dir ^ "/oxenstored.conf"

I’d use Filename.concat.

Quoted 14 lines
tools/ocaml/xenstored/parse_arg.ml +1 -0
+

let do_argv =
let pidfile = ref "" and tracefile = ref "" (* old xenstored compatibility *) and domain_init = ref true

tools/ocaml/xenstored/parse_arg.ml +2 -0
@@ -38,6 +44,8 @@ let do_argv =
 and restart = ref false
 and live_reload = ref false
 and disable_socket = ref false
+  and config_test = ref false
+  and help = ref false
 in
 
 let speclist =

tools/ocaml/xenstored/parse_arg.ml let do_argv = · line 66

Quoted 4 lines
tools/ocaml/xenstored/parse_arg.ml +1 -0
@@ -55,10 +63,27 @@ let do_argv =
     ("-T", Arg.Set_string tracefile, ""); (* for compatibility *)
     ("--restart", Arg.Set restart, "Read database on starting");
     ("--live", Arg.Set live_reload, "Read live dump on startup");
+      ("--config-test", Arg.Set config_test, "Test validity of config file");

I see the logic how this flag was named but feel starting with a verb (“validate”, “check”, “test”) leads to a clearer invocation pattern.

Quoted 72 lines
tools/ocaml/xenstored/parse_arg.ml +17 -0
     ("--disable-socket", Arg.Unit (fun () -> disable_socket := true), "Disable socket");
+      ("--help", Arg.Set help, "Display this list of options")
   ] in
 let usage_msg = "usage : xenstored [--config-file <filename>] [--no-domain-init] [--help] [--no-fork] [--reraise-top-level] [--restart] [--disable-socket]" in
 Arg.parse speclist (fun _ -> ()) usage_msg;
+  let () =
+    if !help then begin
+      if !live_reload then
+        (*
+          Transform --live --help into --config-test for backward compat with
+          running code during live update.
+          Caller will validate config and exit
+        *)
+        config_test := true
+      else begin
+        Arg.usage_string speclist usage_msg |> print_endline;
+        exit 0
+      end
+    end
+  in
 {
   domain_init = !domain_init;
   activate_access_log = !activate_access_log;
@@ -70,4 +95,5 @@ let do_argv =
   restart = !restart;
   live_reload = !live_reload;
   disable_socket = !disable_socket;
+    config_test = !config_test;
 }
tools/ocaml/xenstored/xenstored.ml +0 -0
index 366437b396..1aaa3e995e 100644
@@ -88,7 +88,7 @@ let default_pidfile = Paths.xen_run_dir ^ "/xenstored.pid"
 

let ring_scan_interval = ref 20

tools/ocaml/xenstored/xenstored.ml +3 -2
-let parse_config filename =
+let parse_config ?(strict=false) filename =
 let pidfile = ref default_pidfile in
 let options = [
   ("merge-activate", Config.Set_bool Transaction.do_coalesce);
@@ -129,11 +129,12 @@ let parse_config filename =
   ("xenstored-port", Config.Set_string Domains.xenstored_port); ] in
 begin try Config.read filename options (fun _ _ -> raise Not_found)
   with
-    | Config.Error err -> List.iter (fun (k, e) ->
+    | Config.Error err as e -> List.iter (fun (k, e) ->
       match e with
       | "unknown key" -> eprintf "config: unknown key %s\n" k
       | _             -> eprintf "config: %s: %s\n" k e
     ) err;
+      if strict then raise e
   | Sys_error m -> eprintf "error: config: %s\n" m;
 end;
 !pidfile
@@ -358,6 +359,12 @@ let tweak_gc () =

let () =
Printexc.set_uncaught_exception_handler Logging.fallback_exception_handler; let cf = do_argv in

tools/ocaml/xenstored/xenstored.ml +6 -0
+  if cf.config_test then begin
+    let path = config_filename cf in
+    let _pidfile:string = parse_config ~strict:true path in
+    Printf.printf "Configuration valid at %s\n%!" path;
+    exit 0
+  end;
 let pidfile =
   if Sys.file_exists (config_filename cf) then
     parse_config (config_filename cf)
Signature · 1 line
2.34.1
Acked-by on the v4 series
#22 lore raw
In reply to Edwin Török
Quoted 8 lines · Edwin Török

From: Edwin Török <[email protected]>

Various OCaml fixes, some of which got Acked already.

Quoted 14 lines

Edwin Török (11):
tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper tools/ocaml/xenstored/Makefile: use ocamldep -sort for linking order CODING_STYLE(tools/ocaml): add 'make format' and remove tabs tools/ocaml: run "make format"
CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation
uses spaces
tools/ocaml: add .clang-format
fixup! tools/ocaml/xenctrl: OCaml 5 support, fix use-after-free tools/ocaml/libs/mmap: mark mmap/munmap as blocking and raise
Unix_error on failure
tools/ocaml/libs/xb: check for unmapped ring before accessing it tools/ocaml/xenstored: validate config file before live update tools/ocaml/libs/xb: drop Xs_ring.write

I’m going to Ack this whole sale and provide individual comments.

#7 lore raw

Re: [PATCH v4 03/11] CODING_STYLE(tools/ocaml): add 'make format' and remove tabs

In reply to Edwin Török

tools/ocaml/Makefile build-tools-oxenstored: · line 41

Quoted 1 line · Edwin Török
tools/ocaml/Makefile +1 -0
+	git ls-files '*.c' '*.h' | xargs -n1 sed -ie 's/\t/    /g'

Seen as there's a patch adding .clang-format, what the point of this command?

"-ie" means to ask sed to change file in-place an keep a copy of the original file with "e" as suffix. So please replace that by "-i~" or "-i -e" or just "-i", with the first one create a "~" backup, the last two not creating a backup at all and probably what you wanted. ("-e" is optional as there's only a single command)

Thanks,

Signature · 1 line
Anthony PERARD
#13 lore raw

Re: [PATCH v4 06/11] tools/ocaml: add .clang-format

In reply to Edwin Török
Quoted 2 lines · Edwin Török

Add a .clang-format configuration that tries to match CODING_STYLE where possible.

Is there going to be a patch applying this to "tools/ocaml", like you did with "make format"?

Quoted 5 lines

I was not able to express the special casing of braces after 'do' though, this can only be controlled generally for all control statements.
It is imperfect, but should be better than the existing bindings, which do not follow Xen coding style.

There isn't a single CODING_STYLE for all code in the repo so it isn't an issue if the binding where having a different coding style. But having as few coding style as possible in the repo is probably helpful.

You could maybe add a CODING_STYLE in "tools/ocaml" to say that the coding style is running make format or clang-format .... And if there other rules like how to name certain variable, that could go in this file as well.

Cheers,

Signature · 1 line
Anthony PERARD
#14 lore raw

Re: [PATCH v4 06/11] tools/ocaml: add .clang-format

Quoted 10 lines · Anthony PERARD
Quoted 2 lines · Edwin Török

Add a .clang-format configuration that tries to match CODING_STYLE where possible.

Is there going to be a patch applying this to "tools/ocaml", like you did with "make format"?

Long term probably, not yet.

For now I just want to experiment with it to see how well and consistently it works (especially with different versions of the tool). (I've been using ocp-indent and ocamlformat for years now, clang-format would be a brand new addition.
There are some solutions for ocamlformat that might be useful for backports, e.g. there is a git merge driver
that reformats all 3 sides of a merge and then attempt to solve conflict/do the merge on the resulting formatted files, so even if old code was formatted differently or with a different version it might be able to apply the changes. I've yet to try it though, and I'm not yet sure how easy it would be to integrate something like that with 'guilt' or 'stgit' which is what Xen uses to manage its patch-queue for backports, at least internally.

If there was an ocp-indent equivalent for C that just re-indents that might be something we could apply right now, but I'm not aware of one.
)

Quoted 10 lines
Quoted 6 lines

I was not able to expr
ess the special casing of braces after 'do'
though, this can only be controlled generally for all control statements.
It is imperfect, but should be better than the existing bindings, which do not follow Xen coding style.

There isn't a single CODING_STYLE for all code in the repo so it isn't an issue if the binding where having a different coding style. But having as few coding style as possible in the repo is probably helpful.

Indeed, having *a* coding style (and automated checking/formatting) is what I'm more interested in rather than which particular one. The current one used by the bindings is not documented, and although I can guess at based on surrounding code, that surrounding code is subtly different based on when it was written, so it seemed better to adopt what is already documented in CODING_STYLE.

Quoted 6 lines

You could maybe add a CODING_STYLE in "tools/ocaml" to say that the coding style is running make format or clang-format .... And if there other rules like how to name certain variable, that could go in this file as well.

Having a way to easily run these formatters would be a plus.
Perhaps extending one of the container images that Xen already has to include (a recent enough version of) ocp-indent and clang-format, and similarly for the .spec file it might be useful to update its BuildRequires to bring in the necessary formatters to make it easier to use this in backports.

And then all that can be documented in a CODING_STYLE or CONTRIBUTING.txt/CONTRIBUTING.md file in tools/ocaml.

Best regards,
--Edwin

#8 lore raw

Re: [PATCH v4 03/11] CODING_STYLE(tools/ocaml): add 'make format' and remove tabs

In reply to Anthony PERARD
Quoted 9 lines · Anthony PERARD
Quoted 1 line · Edwin Török

+ git ls-files '*.c' '*.h' | xargs -n1 sed -ie 's/\t/ /g'

Seen as there's a patch adding .clang-format, what the point of this command?

The diff to change tabs to spaces (and the equivalent one from ocp-indent) can be proven
to introduce 0 changes by looking at the diff with ignore-whitespace.
Proving the same with ocamlformat or clang-format is more difficult (and in particular if you keep rebasing commits from after the reformat commit to before it you risk losing the change if you don't redo the format commit correctly).

So I intended to do this gradually: first get indentation to be consistent, and then get formatting to be consistent later once the former has been accepted and committed.

Quoted 7 lines

"-ie" means to ask sed to change file in-place an keep a copy of the original file with "e" as suffix. So please replace that by "-i~" or "-i -e" or just "-i", with the first one create a "~" backup, the last two not creating a backup at all and probably what you wanted. ("-e" is optional as there's only a single command)

Thanks for pointing it out, what I wanted is 'sed -i -e', and you can usually merge multiple single char flags into a single one, except that is not the case for -i, and I keep making this mistake.

.PHONY: format
format:

	git ls-files '*.ml' '*.mli' | xargs -n1 ocp-indent -i
	git ls-files '*.c' '*.h' | xargs -n1 sed -i 's/\t/    /g'

--Edwin