OCaml fixes
v4 cover OCaml fixes
From: Edwin Török <[email protected]>
Various OCaml fixes, some of which got Acked already.
Note: the Data_abstract_val got Acked but not yet commited because we thought there might be a better way with finalizers/etc.
It is not possible to use finalizers here but there is a new followup patch in this series that detects use-after-free and raises an exception without trigerring undefined behaviour or crashes (see 'xb: check for unmapped ring').
The formatting patch was controversial during the 4.17 freeze, but master is open now, and both the maintainer and myself agree with it,
so I think we should be able to do it now.
I've split it up into 2: add the formatting rules, and the actual formatting, so that it is easier to redo the formatting patch when this gets backported.
I think we should just commit it rather than worry that there might be more bugfixes to come after and attempt to rebase the bugfixes without the formatting patch (introducing bugs in the process), especially that one of the patches not yet posted to fix use of enter/leave changes quite a significant part of all C stubs, and I don't have a way of developing those patches without the formatting patch.
The rest are various bugfixes to the C bindings, and I have more to send after I've tested them
(almost all uses of enter/leave blocking section are
wrong because they access OCaml values in non-safe ways with _H, but the GC may have already moved them which would cause a crash, that changes a lot of bindings and needs more testing before sending out, and I've got a few more OCaml 5 fixes too).
Edwin Török (11):
tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper tools/ocaml/xenstored/Makefile: use ocamldep -sort for linking order CODING_STYLE(tools/ocaml): add 'make format' and remove tabs tools/ocaml: run "make format"
CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation
uses spaces
tools/ocaml: add .clang-format
fixup! tools/ocaml/xenctrl: OCaml 5 support, fix use-after-free tools/ocaml/libs/mmap: mark mmap/munmap as blocking and raise
Unix_error on failure
tools/ocaml/libs/xb: check for unmapped ring before accessing it tools/ocaml/xenstored: validate config file before live update tools/ocaml/libs/xb: drop Xs_ring.write
.editorconfig | 20 + tools/ocaml/.clang-format | 9 + tools/ocaml/Makefile | 5 + tools/ocaml/libs/eventchn/xeneventchn_stubs.c | 194 +- tools/ocaml/libs/mmap/mmap_stubs.h | 8 +- tools/ocaml/libs/mmap/xenmmap.ml | 2 +- tools/ocaml/libs/mmap/xenmmap.mli | 4 +- tools/ocaml/libs/mmap/xenmmap_stubs.c | 123 +- tools/ocaml/libs/xb/op.ml | 76 +- tools/ocaml/libs/xb/packet.ml | 30 +- tools/ocaml/libs/xb/partial.ml | 48 +- tools/ocaml/libs/xb/xb.ml | 416 ++-- tools/ocaml/libs/xb/xb.mli | 106 +- tools/ocaml/libs/xb/xenbus_stubs.c | 50 +- tools/ocaml/libs/xb/xs_ring.ml | 31 +- tools/ocaml/libs/xb/xs_ring_stubs.c | 221 +- tools/ocaml/libs/xc/xenctrl.ml | 326 +-- tools/ocaml/libs/xc/xenctrl.mli | 12 +- tools/ocaml/libs/xc/xenctrl_stubs.c | 1556 ++++++------- tools/ocaml/libs/xentoollog/caml_xentoollog.h | 6 +- .../ocaml/libs/xentoollog/xentoollog_stubs.c | 196 +- tools/ocaml/libs/xl/xenlight_stubs.c | 2022 ++++++++--------- tools/ocaml/libs/xs/queueop.ml | 48 +- tools/ocaml/libs/xs/xs.ml | 220 +- tools/ocaml/libs/xs/xs.mli | 46 +- tools/ocaml/libs/xs/xsraw.ml | 300 +-- tools/ocaml/libs/xs/xst.ml | 76 +- tools/ocaml/libs/xs/xst.mli | 20 +- tools/ocaml/test/dmesg.ml | 26 +- tools/ocaml/test/list_domains.ml | 4 +- tools/ocaml/test/raise_exception.ml | 4 +- tools/ocaml/test/xtl.ml | 28 +- tools/ocaml/xenstored/Makefile | 6 +- tools/ocaml/xenstored/config.ml | 156 +- tools/ocaml/xenstored/connection.ml | 578 ++--- tools/ocaml/xenstored/connections.ml | 294 +-- tools/ocaml/xenstored/disk.ml | 218 +- tools/ocaml/xenstored/domain.ml | 116 +- tools/ocaml/xenstored/domains.ml | 298 +-- tools/ocaml/xenstored/event.ml | 28 +- tools/ocaml/xenstored/history.ml | 62 +- tools/ocaml/xenstored/logging.ml | 478 ++-- tools/ocaml/xenstored/packet.ml | 20 +- tools/ocaml/xenstored/parse_arg.ml | 132 +- tools/ocaml/xenstored/perms.ml | 216 +- tools/ocaml/xenstored/poll.ml | 68 +- tools/ocaml/xenstored/poll.mli | 4 +- tools/ocaml/xenstored/process.ml | 1210 +++++----- tools/ocaml/xenstored/quota.ml | 74 +- tools/ocaml/xenstored/select_stubs.c | 62 +- tools/ocaml/xenstored/stdext.ml | 190 +- tools/ocaml/xenstored/store.ml | 752 +++--- tools/ocaml/xenstored/symbol.ml | 2 +- tools/ocaml/xenstored/syslog.ml | 48 +- tools/ocaml/xenstored/syslog_stubs.c | 68 +- tools/ocaml/xenstored/systemd_stubs.c | 10 +- tools/ocaml/xenstored/transaction.ml | 352 +-- tools/ocaml/xenstored/trie.ml | 222 +- tools/ocaml/xenstored/trie.mli | 22 +- tools/ocaml/xenstored/utils.ml | 146 +- tools/ocaml/xenstored/xenstored.ml | 987 ++++---- 61 files changed, 6569 insertions(+), 6483 deletions(-) create mode 100644 .editorconfig create mode 100644 tools/ocaml/.clang-format
--
2.34.1
PATCH v4 1/11 tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper
This is not strictly necessary since it is essentially a no-op
currently: a cast to void* and value*, even in OCaml 5.0.
However it does make it clearer that what we have here is not a regular OCaml value, but one allocated with Abstract_tag or Custom_tag, and follows the example from the manual more closely:
https://v2.ocaml.org/manual/intfc.html#ss:c-outside-head
It also makes it clearer that these modules have been reviewed for compat with OCaml 5.0.
We cannot use OCaml finalizers here, because we want control over when to unmap these pages from remote domains.
A follow-up commit will add use-after-free detection instead.
No functional change.
---
Reason for inclusion in 4.17:
- make code follow best practice for upcoming OCaml 5.0 compiler (already in beta)
Changes since v2:
- add Acked-by line
Changes since v3:
- mention that use-after-free is fixed in another commit, and we cannot use finalizers here
--- tools/ocaml/libs/mmap/mmap_stubs.h | 4 ++++ tools/ocaml/libs/mmap/xenmmap_stubs.c | 2 +- tools/ocaml/libs/xb/xs_ring_stubs.c | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-)
tools/ocaml/libs/mmap/mmap_stubs.h +4 -0
@@ -30,4 +30,8 @@ struct mmap_interface int len; }; +#ifndef Data_abstract_val +#define Data_abstract_val (x) ((void*)(value*)(x)) +#endif +#endif
tools/ocaml/libs/mmap/xenmmap_stubs.c +1 -1
@@ -28,7 +28,7 @@#include <caml/fail.h> #include <caml/callback.h> -#define Intf_val (a) ((struct mmap_interface *) a) +#define Intf_val (a) ((struct mmap_interface *) Data_abstract_val(a)) static int mmap_interface_init(struct mmap_interface *intf, int fd, int pflag, int mflag,
tools/ocaml/libs/xb/xs_ring_stubs.c +1 -1
@@ -35,7 +35,7 @@#include <sys/mman.h> #include "mmap_stubs.h" -#define GET_C_STRUCT (a) ((struct mmap_interface *) a) +#define GET_C_STRUCT (a) ((struct mmap_interface *) Data_abstract_val(a))/* * Bytes_val has been introduced by Ocaml 4.06.1. So define our own version
Signature · 1 line
2.34.1
Re: [PATCH v4 01/11] tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper
tools/ocaml/libs/mmap/mmap_stubs.h struct mmap_interface · line 34
Quoted 10 lines · Edwin Török
tools/ocaml/libs/mmap/mmap_stubs.h +2
-0@@ -30,4 +30,8 @@ struct mmap_interfaceint len; }; +#ifndef Data_abstract_val +#define Data_abstract_val (x) ((void*)(value*)(x))
((void *)(x))
I take it this has come from the Ocaml headers? The cast to (value *) in the middle is entirely cancelled out.
Can be fixed on commit.
~Andrew
Re: [PATCH v4 01/11] tools/ocaml/libs/{xb, mmap}: use Data_abstract_val wrapper
Quoted 18 lines · Andrew Cooper
tools/ocaml/libs/mmap/mmap_stubs.h struct mmap_interface · line 34
Quoted 10 lines · Edwin Török
tools/ocaml/libs/mmap/mmap_stubs.h +2
-0@@ -30,4 +30,8 @@ struct mmap_interfaceint len; }; +#ifndef Data_abstract_val +#define Data_abstract_val (x) ((void*)(value*)(x))((void *)(x))
Quoted 5 lines
I take it this has come from the Ocaml headers? The cast to (value *) in the middle is entirely cancelled out.
Can be fixed on commit.
In the OCaml headers it looks like:
+#define Data_abstract_val(v) ((void*) Op_val(v))
where Op_val is:
+#define Op_val(x) ((value *) (x))
However I haven't realized that Op_val has been in OCaml for a long time (at least 1995), so we can safely use it, and do this instead:
tools/ocaml/libs/mmap/mmap_stubs.h +4 -0
@@ -30,4 +30,8 @@ struct mmap_interface int len; }; +#ifndef Data_abstract_val +#define Data_abstract_val (v) ((void*) Op_val(v)) +#endif +#endif
There is an updated commit here for convenience: https://github.com/edwintorok/xen/commit/9855ed237f3f85ac30972bfd0a601c6746ba2353
Best regards,
--Edwin
PATCH v4 2/11 tools/ocaml/xenstored/Makefile: use ocamldep -sort for linking order
In reply to Edwin TörökA better solution is being worked on for master,
but for now use ocamldep -sort instead of a manually established link order.
The manually established link order will be wrong when (security) patches introduce new dependencies between files that would require changing the link order.
If dune was used as a build system this wouldn't be a problem, but we can't use Dune yet due to OSSTest, which is stuck on Debian oldstable.
No functional change.
---
Reason for inclusion in 4.17:
Avoids having to put this patch as a prerequisite into a security update. Earlier versions of XSA-326 needed this, and although latest version didn't we don't know whether it might be needed again in the future or not.
Changes since v2:
- new in v3 (was previously emailed to security team though)
Changes since v3:
- add Acked-by line
--- tools/ocaml/xenstored/Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tools/ocaml/xenstored/Makefile +5 -1
@@ -72,7 +72,11 @@ XENSTOREDLIBS = \ PROGRAMS = oxenstored oxenstored_LIBS = $(XENSTOREDLIBS) -oxenstored_OBJS = $(OBJS) +# use ocamldep to figure out link order, otherwise the Makefile would have +# to be continously adjusted for security patches that introduce new +# dependencies between files +oxenstored_MLSORTED = $(shell $(OCAMLDEP) -sort $(OBJS:=.ml)) +oxenstored_OBJS = $(oxenstored_MLSORTED:.ml=) OCAML_PROGRAM = oxenstored
Signature · 1 line
2.34.1
PATCH v4 3/11 CODING_STYLE(tools/ocaml): add 'make format' and remove tabs
In reply to Edwin TörökSee CODING_STYLE: Xen uses spaces, not tabs.
* OCaml code:
Using ocp-indent for now to just make minimal modifications in tabs vs spaces and get the right indentation.
We can introduce ocamlformat later.
* C stubs:
just replace tabs with spaces now, using indent or clang-format would change code too much for 4.17.
This avoids perpetuating a formatting style that is inconsistent with the rest of Xen, and that makes preparing and submitting patches more difficult (OCaml indentation tools usually only support spaces, not tabs).
No functional change.
--
Reason for inclusion:
- avoid perpetuating a different coding style (I thought tabs were mandated by Xen, and was about to fix up my editor config to match when I realized Xen already mandates the use of spaces)
- should make submitting patches for OCaml easier (OCaml indentation tools know only about spaces, so I either can't use them, or have to manually adjust indentation every time I submit a patch)
- it can be verified that the only change here is the Makefile change for the new rule, 'git log -p -1 -w' should be otherwise empty
Changes since v3:
- this didn't make it into 4.17.0, we'll reconsider for 4.17.1, for now apply just to master which is open again
- separate introducing the rule from actual reformatting
--- tools/ocaml/Makefile | 5 +++++ 1 file changed, 5 insertions(+)
tools/ocaml/Makefile +5 -0
@@ -34,3 +34,8 @@ build-tools-oxenstored: $(MAKE) -s -C libs/xb $(MAKE) -s -C libs/xc $(MAKE) -C xenstored + +.PHONY: format +format: + git ls-files '*.ml' '*.mli' | xargs -n1 ocp-indent -i + git ls-files '*.c' '*.h' | xargs -n1 sed -ie 's/\t/ /g'
Signature · 1 line
2.34.1
Re: [PATCH v4 03/11] CODING_STYLE(tools/ocaml): add 'make format' and remove tabs
Quoted 17 lines · Edwin Török
See
CODING_STYLE: Xen uses spaces, not tabs.* OCaml code:
Using
ocp-indentfor now to just make minimal modifications in tabs vs spaces and get the right indentation.
We can introduceocamlformatlater.* C stubs:
just replace tabs with spaces now, using
indentorclang-formatwould change code too much for 4.17.
Quoted 7 lines
This avoids perpetuating a formatting style that is inconsistent with the rest of Xen, and that makes preparing and submitting patches more difficult (OCaml indentation tools usually only support spaces, not tabs).
No functional change.
Signed-off-by Edwin Török <[email protected]>
I am in favour of automating the code formatting, and moving to OCamlformat in particular. The main drawback is that it can make back porting more difficult because formatting can change not just indentation but placement of parentheses and similar grouping syntax as well, leading to changes between the current code and older code that have to me resolved manually.
— C
Re: [PATCH v4 03/11] CODING_STYLE(tools/ocaml): add 'make format' and remove tabs
In reply to Edwin Töröktools/ocaml/Makefile build-tools-oxenstored: · line 41
Quoted 1 line · Edwin Török
tools/ocaml/Makefile +1
-0+ git ls-files '*.c' '*.h' | xargs -n1 sed -ie 's/\t/ /g'
Seen as there's a patch adding .clang-format, what the point of this command?
"-ie" means to ask sed to change file in-place an keep a copy of the original file with "e" as suffix. So please replace that by "-i~" or "-i -e" or just "-i", with the first one create a "~" backup, the last two not creating a backup at all and probably what you wanted. ("-e" is optional as there's only a single command)
Thanks,
Signature · 1 line
Anthony PERARD
Re: [PATCH v4 03/11] CODING_STYLE(tools/ocaml): add 'make format' and remove tabs
Quoted 9 lines · Anthony PERARD
Quoted 1 line · Edwin Török
+ git ls-files '*.c' '*.h' | xargs -n1 sed -ie 's/\t/ /g'
Seen as there's a patch adding .clang-format, what the point of this command?
The diff to change tabs to spaces (and the equivalent one from ocp-indent) can be proven
to introduce 0 changes by looking at the diff with ignore-whitespace.
Proving the same with ocamlformat or clang-format is more difficult (and in particular if you keep rebasing commits from after the reformat commit to before it you risk losing the change if you don't redo the format commit correctly).
So I intended to do this gradually: first get indentation to be consistent, and then get formatting to be consistent later once the former has been accepted and committed.
Quoted 7 lines
"-ie" means to ask sed to change file in-place an keep a copy of the original file with "e" as suffix. So please replace that by "-i~" or "-i -e" or just "-i", with the first one create a "~" backup, the last two not creating a backup at all and probably what you wanted. ("-e" is optional as there's only a single command)
Thanks for pointing it out, what I wanted is 'sed -i -e', and you can usually merge multiple single char flags into a single one, except that is not the case for -i, and I keep making this mistake.
.PHONY: format
format:
git ls-files '*.ml' '*.mli' | xargs -n1 ocp-indent -i git ls-files '*.c' '*.h' | xargs -n1 sed -i 's/\t/ /g'
--Edwin
PATCH v4 4/11 tools/ocaml: run "make format"
In reply to Edwin TörökFrom: Edwin Török <[email protected]>
git log -p -1 -w proves the change here is only whitespace.
No functional change.
---
Changes since v3
* split into its own commit to make it easier to refresh when rebasing
--- tools/ocaml/libs/eventchn/xeneventchn_stubs.c | 194 +- tools/ocaml/libs/mmap/mmap_stubs.h | 4 +- tools/ocaml/libs/mmap/xenmmap.ml | 2 +- tools/ocaml/libs/mmap/xenmmap.mli | 4 +- tools/ocaml/libs/mmap/xenmmap_stubs.c | 112 +- tools/ocaml/libs/xb/op.ml | 76 +- tools/ocaml/libs/xb/packet.ml | 30 +- tools/ocaml/libs/xb/partial.ml | 48 +- tools/ocaml/libs/xb/xb.ml | 416 ++-- tools/ocaml/libs/xb/xb.mli | 106 +- tools/ocaml/libs/xb/xenbus_stubs.c | 50 +- tools/ocaml/libs/xb/xs_ring.ml | 28 +- tools/ocaml/libs/xb/xs_ring_stubs.c | 214 +- tools/ocaml/libs/xc/xenctrl.ml | 326 +-- tools/ocaml/libs/xc/xenctrl.mli | 12 +- tools/ocaml/libs/xc/xenctrl_stubs.c | 1556 ++++++------- tools/ocaml/libs/xentoollog/caml_xentoollog.h | 6 +- .../ocaml/libs/xentoollog/xentoollog_stubs.c | 196 +- tools/ocaml/libs/xl/xenlight_stubs.c | 2022 ++++++++--------- tools/ocaml/libs/xs/queueop.ml | 48 +- tools/ocaml/libs/xs/xs.ml | 220 +- tools/ocaml/libs/xs/xs.mli | 46 +- tools/ocaml/libs/xs/xsraw.ml | 300 +-- tools/ocaml/libs/xs/xst.ml | 76 +- tools/ocaml/libs/xs/xst.mli | 20 +- tools/ocaml/test/dmesg.ml | 26 +- tools/ocaml/test/list_domains.ml | 4 +- tools/ocaml/test/raise_exception.ml | 4 +- tools/ocaml/test/xtl.ml | 28 +- tools/ocaml/xenstored/config.ml | 156 +- tools/ocaml/xenstored/connection.ml | 578 ++--- tools/ocaml/xenstored/connections.ml | 294 +-- tools/ocaml/xenstored/disk.ml | 218 +- tools/ocaml/xenstored/domain.ml | 116 +- tools/ocaml/xenstored/domains.ml | 298 +-- tools/ocaml/xenstored/event.ml | 28 +- tools/ocaml/xenstored/history.ml | 62 +- tools/ocaml/xenstored/logging.ml | 478 ++-- tools/ocaml/xenstored/packet.ml | 20 +- tools/ocaml/xenstored/parse_arg.ml | 106 +- tools/ocaml/xenstored/perms.ml | 216 +- tools/ocaml/xenstored/poll.ml | 68 +- tools/ocaml/xenstored/poll.mli | 4 +- tools/ocaml/xenstored/process.ml | 1210 +++++----- tools/ocaml/xenstored/quota.ml | 74 +- tools/ocaml/xenstored/select_stubs.c | 62 +- tools/ocaml/xenstored/stdext.ml | 190 +- tools/ocaml/xenstored/store.ml | 752 +++--- tools/ocaml/xenstored/symbol.ml | 2 +- tools/ocaml/xenstored/syslog.ml | 48 +- tools/ocaml/xenstored/syslog_stubs.c | 68 +- tools/ocaml/xenstored/systemd_stubs.c | 10 +- tools/ocaml/xenstored/transaction.ml | 352 +-- tools/ocaml/xenstored/trie.ml | 222 +- tools/ocaml/xenstored/trie.mli | 22 +- tools/ocaml/xenstored/utils.ml | 146 +- tools/ocaml/xenstored/xenstored.ml | 978 ++++---- 57 files changed, 6476 insertions(+), 6476 deletions(-)
tools/ocaml/libs/eventchn/xeneventchn_stubs.c +97 -97
@@ -37,171 +37,171 @@static void stub_evtchn_finalize (value v) { -xenevtchn_close (_H (v)); +xenevtchn_close (_H (v)); }static struct custom_operations xenevtchn_ops = { - .identifier ="xenevtchn" , - .finalize = stub_evtchn_finalize, - .compare = custom_compare_default,/* Can't compare */ - .hash = custom_hash_default,/* Can't hash */ - .serialize = custom_serialize_default,/* Can't serialize */ - .deserialize = custom_deserialize_default,/* Can't deserialize */ - .compare_ext = custom_compare_ext_default,/* Can't compare */ + .identifier ="xenevtchn" , + .finalize = stub_evtchn_finalize, + .compare = custom_compare_default,/* Can't compare */ + .hash = custom_hash_default,/* Can't hash */ + .serialize = custom_serialize_default,/* Can't serialize */ + .deserialize = custom_deserialize_default,/* Can't deserialize */ + .compare_ext = custom_compare_ext_default,/* Can't compare */ }; CAMLprim value stub_eventchn_init(value cloexec) { -CAMLparam1 (cloexec); -CAMLlocal1 (result); -xenevtchn_handle *xce; -unsigned int flags =0 ; +CAMLparam1 (cloexec); +CAMLlocal1 (result); +xenevtchn_handle *xce; +unsigned int flags =0 ; -if ( !Bool_val (cloexec) ) - flags |=XENEVTCHN_NO_CLOEXEC ; +if ( !Bool_val (cloexec) ) + flags |=XENEVTCHN_NO_CLOEXEC ; -caml_enter_blocking_section (); - xce =xenevtchn_open (NULL , flags); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + xce =xenevtchn_open (NULL , flags); +caml_leave_blocking_section (); -if (xce ==NULL ) -caml_failwith ("open failed" ); +if (xce ==NULL ) +caml_failwith ("open failed" ); - result =caml_alloc_custom (&xenevtchn_ops,sizeof (xce),0 ,1 ); -_H (result) = xce; + result =caml_alloc_custom (&xenevtchn_ops,sizeof (xce),0 ,1 ); +_H (result) = xce; -CAMLreturn (result); +CAMLreturn (result); } CAMLprim value stub_eventchn_fdopen(value fdval) { -CAMLparam1 (fdval); -CAMLlocal1 (result); -xenevtchn_handle *xce; +CAMLparam1 (fdval); +CAMLlocal1 (result); +xenevtchn_handle *xce; -caml_enter_blocking_section (); - xce =xenevtchn_fdopen (NULL ,Int_val (fdval),0 ); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + xce =xenevtchn_fdopen (NULL ,Int_val (fdval),0 ); +caml_leave_blocking_section (); -if (xce ==NULL ) -caml_failwith ("evtchn fdopen failed" ); +if (xce ==NULL ) +caml_failwith ("evtchn fdopen failed" ); - result =caml_alloc_custom (&xenevtchn_ops,sizeof (xce),0 ,1 ); -_H (result) = xce; + result =caml_alloc_custom (&xenevtchn_ops,sizeof (xce),0 ,1 ); +_H (result) = xce; -CAMLreturn (result); +CAMLreturn (result); } CAMLprim value stub_eventchn_fd(value xce) { -CAMLparam1 (xce); -CAMLlocal1 (result); -int fd; +CAMLparam1 (xce); +CAMLlocal1 (result); +int fd; - fd =xenevtchn_fd (_H (xce)); -if (fd ==-1 ) -caml_failwith ("evtchn fd failed" ); + fd =xenevtchn_fd (_H (xce)); +if (fd ==-1 ) +caml_failwith ("evtchn fd failed" ); - result =Val_int (fd); + result =Val_int (fd); -CAMLreturn (result); +CAMLreturn (result); } CAMLprim value stub_eventchn_notify(value xce, value port) { -CAMLparam2 (xce, port); -int rc; +CAMLparam2 (xce, port); +int rc; -caml_enter_blocking_section (); - rc =xenevtchn_notify (_H (xce),Int_val (port)); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + rc =xenevtchn_notify (_H (xce),Int_val (port)); +caml_leave_blocking_section (); -if (rc ==-1 ) -caml_failwith ("evtchn notify failed" ); +if (rc ==-1 ) +caml_failwith ("evtchn notify failed" ); -CAMLreturn (Val_unit); +CAMLreturn (Val_unit); } CAMLprim value stub_eventchn_bind_interdomain(value xce, value domid,value remote_port) { -CAMLparam3 (xce, domid, remote_port); -CAMLlocal1 (port); -xenevtchn_port_or_error_t rc; +CAMLparam3 (xce, domid, remote_port); +CAMLlocal1 (port); +xenevtchn_port_or_error_t rc; -caml_enter_blocking_section (); - rc =xenevtchn_bind_interdomain (_H (xce),Int_val (domid),Int_val (remote_port)); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + rc =xenevtchn_bind_interdomain (_H (xce),Int_val (domid),Int_val (remote_port)); +caml_leave_blocking_section (); -if (rc ==-1 ) -caml_failwith ("evtchn bind_interdomain failed" ); - port =Val_int (rc); +if (rc ==-1 ) +caml_failwith ("evtchn bind_interdomain failed" ); + port =Val_int (rc); -CAMLreturn (port); +CAMLreturn (port); } CAMLprim value stub_eventchn_bind_virq(value xce, value virq_type) { -CAMLparam2 (xce, virq_type); -CAMLlocal1 (port); -xenevtchn_port_or_error_t rc; +CAMLparam2 (xce, virq_type); +CAMLlocal1 (port); +xenevtchn_port_or_error_t rc; -caml_enter_blocking_section (); - rc =xenevtchn_bind_virq (_H (xce),Int_val (virq_type)); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + rc =xenevtchn_bind_virq (_H (xce),Int_val (virq_type)); +caml_leave_blocking_section (); -if (rc ==-1 ) -caml_failwith ("evtchn bind_virq failed" ); - port =Val_int (rc); +if (rc ==-1 ) +caml_failwith ("evtchn bind_virq failed" ); + port =Val_int (rc); -CAMLreturn (port); +CAMLreturn (port); } CAMLprim value stub_eventchn_unbind(value xce, value port) { -CAMLparam2 (xce, port); -int rc; +CAMLparam2 (xce, port); +int rc; -caml_enter_blocking_section (); - rc =xenevtchn_unbind (_H (xce),Int_val (port)); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + rc =xenevtchn_unbind (_H (xce),Int_val (port)); +caml_leave_blocking_section (); -if (rc ==-1 ) -caml_failwith ("evtchn unbind failed" ); +if (rc ==-1 ) +caml_failwith ("evtchn unbind failed" ); -CAMLreturn (Val_unit); +CAMLreturn (Val_unit); } CAMLprim value stub_eventchn_pending(value xce) { -CAMLparam1 (xce); -CAMLlocal1 (result); -xenevtchn_port_or_error_t port; +CAMLparam1 (xce); +CAMLlocal1 (result); +xenevtchn_port_or_error_t port; -caml_enter_blocking_section (); - port =xenevtchn_pending (_H (xce)); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + port =xenevtchn_pending (_H (xce)); +caml_leave_blocking_section (); -if (port ==-1 ) -caml_failwith ("evtchn pending failed" ); - result =Val_int (port); +if (port ==-1 ) +caml_failwith ("evtchn pending failed" ); + result =Val_int (port); -CAMLreturn (result); +CAMLreturn (result); } CAMLprim value stub_eventchn_unmask(value xce, value _port) { -CAMLparam2 (xce, _port); -evtchn_port_t port; -int rc; +CAMLparam2 (xce, _port); +evtchn_port_t port; +int rc; - port =Int_val (_port); + port =Int_val (_port); -caml_enter_blocking_section (); - rc =xenevtchn_unmask (_H (xce), port); -caml_leave_blocking_section (); +caml_enter_blocking_section (); + rc =xenevtchn_unmask (_H (xce), port); +caml_leave_blocking_section (); -if (rc) -caml_failwith ("evtchn unmask failed" ); -CAMLreturn (Val_unit); +if (rc) +caml_failwith ("evtchn unmask failed" ); +CAMLreturn (Val_unit); }
tools/ocaml/libs/mmap/mmap_stubs.h +2 -2
@@ -26,8 +26,8 @@struct mmap_interface { -void *addr; -int len; +void *addr; +int len; };#ifndef Data_abstract_val
tools/ocaml/libs/mmap/xenmmap.ml +1 -1
@@ -21,7 +21,7 @@ type mmap_map_flag = SHARED | PRIVATE (* mmap: fd -> prot_flag -> map_flag -> length -> offset -> interface *) external mmap: Unix.file_descr -> mmap_prot_flag -> mmap_map_flag - -> int -> int -> mmap_interface = "stub_mmap_init" + -> int -> int -> mmap_interface = "stub_mmap_init" external unmap: mmap_interface -> unit = "stub_mmap_final" (* read: interface -> start -> length -> data *) external read: mmap_interface -> int -> int -> string = "stub_mmap_read"
tools/ocaml/libs/mmap/xenmmap.mli +2 -2
@@ -19,10 +19,10 @@ type mmap_prot_flag = RDONLY | WRONLY | RDWR type mmap_map_flag = SHARED | PRIVATE external mmap : Unix.file_descr -> mmap_prot_flag -> mmap_map_flag -> int -> int - -> mmap_interface = "stub_mmap_init" + -> mmap_interface = "stub_mmap_init" external unmap : mmap_interface -> unit = "stub_mmap_final" external read : mmap_interface -> int -> int -> string = "stub_mmap_read" external write : mmap_interface -> string -> int -> int -> unit - = "stub_mmap_write" + = "stub_mmap_write" external getpagesize : unit -> int = "stub_mmap_getpagesize"
tools/ocaml/libs/mmap/xenmmap_stubs.c +56 -56
153 lines not on this page · raw message
tools/ocaml/libs/xb/op.ml +38 -38
88 lines not on this page · raw message
tools/ocaml/libs/xb/packet.ml +15 -15
@@ -15,12 +15,12 @@ *) type t = -{ - tid: int; - rid: int; - ty: Op.operation; - data: string; -} + { + tid: int; + rid: int; + ty: Op.operation; + data: string; + } exception Error of string exception DataError of string
@@ -30,21 +30,21 @@ external string_of_header: int -> int -> int -> int -> string = "stub_string_of_ let create tid rid ty data = { tid = tid; rid = rid; ty = ty; data = data; } let of_partialpkt ppkt = - create ppkt.Partial.tid ppkt.Partial.rid ppkt.Partial.ty (Buffer.contents ppkt.Partial.buf) + create ppkt.Partial.tid ppkt.Partial.rid ppkt.Partial.ty (Buffer.contents ppkt.Partial.buf) let to_string pkt = - let header = string_of_header pkt.tid pkt.rid (Op.to_cval pkt.ty) (String.length pkt.data) in - header ^ pkt.data + let header = string_of_header pkt.tid pkt.rid (Op.to_cval pkt.ty) (String.length pkt.data) in + header ^ pkt.data let unpack pkt = - pkt.tid, pkt.rid, pkt.ty, pkt.data + pkt.tid, pkt.rid, pkt.ty, pkt.data let get_tid pkt = pkt.tid let get_ty pkt = pkt.ty let get_data pkt = - let l = String.length pkt.data in - if l > 0 && pkt.data.[l - 1] = '\000' then - String.sub pkt.data 0 (l - 1) - else - pkt.data + let l = String.length pkt.data in + if l > 0 && pkt.data.[l - 1] = '\000' then + String.sub pkt.data 0 (l - 1) + else + pkt.data let get_rid pkt = pkt.rid
tools/ocaml/libs/xb/partial.ml +24 -24
63 lines not on this page · raw message
tools/ocaml/libs/xb/xb.ml +208 -208
512 lines not on this page · raw message
tools/ocaml/libs/xb/xb.mli +53 -53
111 lines not on this page · raw message
tools/ocaml/libs/xb/xenbus_stubs.c +25 -25
65 lines not on this page · raw message
tools/ocaml/libs/xb/xs_ring.ml +14 -14
@@ -15,14 +15,14 @@ *) module Server_feature = struct - type t = - | Reconnection + type t = + | Reconnection end module Server_features = Set.Make(struct - type t = Server_feature.t - let compare = compare -end) + type t = Server_feature.t + let compare = compare + end) external read: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_read" external write: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_write"
@@ -31,18 +31,18 @@ external _internal_set_server_features: Xenmmap.mmap_interface -> int -> unit = external _internal_get_server_features: Xenmmap.mmap_interface -> int = "ml_interface_get_server_features" [@@noalloc] let write_substring mmap buff len = - write mmap (Bytes.unsafe_of_string buff) len + write mmap (Bytes.unsafe_of_string buff) len let get_server_features mmap = - (* NB only one feature currently defined above *) - let x = _internal_get_server_features mmap in - if x = 0 - then Server_features.empty - else Server_features.singleton Server_feature.Reconnection + (* NB only one feature currently defined above *) + let x = _internal_get_server_features mmap in + if x = 0 + then Server_features.empty + else Server_features.singleton Server_feature.Reconnection let set_server_features mmap set = - (* NB only one feature currently defined above *) - let x = if set = Server_features.empty then 0 else 1 in - _internal_set_server_features mmap x + (* NB only one feature currently defined above *) + let x = if set = Server_features.empty then 0 else 1 in + _internal_set_server_features mmap x external close: Xenmmap.mmap_interface -> unit = "ml_interface_close" [@@noalloc]
tools/ocaml/libs/xb/xs_ring_stubs.c +107 -107
260 lines not on this page · raw message
tools/ocaml/libs/xc/xenctrl.ml +163 -163
450 lines not on this page · raw message
tools/ocaml/libs/xc/xenctrl.mli +6 -6
@@ -182,11 +182,11 @@ external domain_getinfo : handle -> domid -> domaininfo external domain_get_vcpuinfo : handle -> int -> int -> vcpuinfo = "stub_xc_vcpu_getinfo" external domain_ioport_permission: handle -> domid -> int -> int -> bool -> unit - = "stub_xc_domain_ioport_permission" + = "stub_xc_domain_ioport_permission" external domain_iomem_permission: handle -> domid -> nativeint -> nativeint -> bool -> unit - = "stub_xc_domain_iomem_permission" + = "stub_xc_domain_iomem_permission" external domain_irq_permission: handle -> domid -> int -> bool -> unit - = "stub_xc_domain_irq_permission" + = "stub_xc_domain_irq_permission" external vcpu_affinity_set : handle -> domid -> int -> bool array -> unit = "stub_xc_vcpu_setaffinity" external vcpu_affinity_get : handle -> domid -> int -> bool array
@@ -284,11 +284,11 @@ external hvm_param_set: handle -> domid -> hvm_param -> int64 -> unit = "stub_xc_hvm_param_set" external domain_assign_device: handle -> domid -> (int * int * int * int) -> unit - = "stub_xc_domain_assign_device" + = "stub_xc_domain_assign_device" external domain_deassign_device: handle -> domid -> (int * int * int * int) -> unit - = "stub_xc_domain_deassign_device" + = "stub_xc_domain_deassign_device" external domain_test_assign_device: handle -> domid -> (int * int * int * int) -> bool - = "stub_xc_domain_test_assign_device" + = "stub_xc_domain_test_assign_device" external version : handle -> version = "stub_xc_version_version" external version_compile_info : handle -> compile_info
tools/ocaml/libs/xc/xenctrl_stubs.c +778 -778
877 lines not on this page · raw message
tools/ocaml/libs/xentoollog/caml_xentoollog.h +3 -3
Lines of this file are not shown. View the raw message.
tools/ocaml/libs/xentoollog/xentoollog_stubs.c +98 -98
196 lines not on this page · raw message
tools/ocaml/libs/xl/xenlight_stubs.c +1011 -1011
2,022 lines not on this page · raw message
tools/ocaml/libs/xs/queueop.ml +24 -24
48 lines not on this page · raw message
tools/ocaml/libs/xs/xs.ml +110 -110
220 lines not on this page · raw message
tools/ocaml/libs/xs/xs.mli +23 -23
46 lines not on this page · raw message
tools/ocaml/libs/xs/xsraw.ml +150 -150
300 lines not on this page · raw message
tools/ocaml/libs/xs/xst.ml +38 -38
76 lines not on this page · raw message
tools/ocaml/xenstored/config.ml +78 -78
156 lines not on this page · raw message
tools/ocaml/xenstored/connection.ml +289 -289
578 lines not on this page · raw message
tools/ocaml/xenstored/connections.ml +147 -147
294 lines not on this page · raw message
tools/ocaml/xenstored/disk.ml +109 -109
218 lines not on this page · raw message
tools/ocaml/xenstored/domain.ml +58 -58
116 lines not on this page · raw message
tools/ocaml/xenstored/domains.ml +149 -149
298 lines not on this page · raw message
tools/ocaml/xenstored/history.ml +31 -31
62 lines not on this page · raw message
tools/ocaml/xenstored/logging.ml +239 -239
478 lines not on this page · raw message
tools/ocaml/xenstored/parse_arg.ml +53 -53
106 lines not on this page · raw message
tools/ocaml/xenstored/perms.ml +108 -108
216 lines not on this page · raw message
tools/ocaml/xenstored/poll.ml +34 -34
68 lines not on this page · raw message
tools/ocaml/xenstored/process.ml +605 -605
1,210 lines not on this page · raw message
tools/ocaml/xenstored/quota.ml +37 -37
74 lines not on this page · raw message
tools/ocaml/xenstored/select_stubs.c +31 -31
62 lines not on this page · raw message
tools/ocaml/xenstored/stdext.ml +95 -95
190 lines not on this page · raw message
tools/ocaml/xenstored/store.ml +376 -376
752 lines not on this page · raw message
tools/ocaml/xenstored/syslog.ml +24 -24
48 lines not on this page · raw message
tools/ocaml/xenstored/syslog_stubs.c +34 -34
68 lines not on this page · raw message
tools/ocaml/xenstored/transaction.ml +176 -176
352 lines not on this page · raw message
tools/ocaml/xenstored/trie.ml +111 -111
222 lines not on this page · raw message
tools/ocaml/xenstored/utils.ml +73 -73
146 lines not on this page · raw message
tools/ocaml/xenstored/xenstored.ml +489 -489
978 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.34.1
Part of the diff is not shown here. View the raw message.
PATCH v4 5/11 CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation uses spaces
In reply to Edwin TörökAdd an .editorconfig to make it easier to keep patches compatible with Xen's coding style, and to reemphasize what Xen's coding style is.
I thought that Xen demands tabs rather than spaces (which is more difficult with OCaml because indentation tools use spaces, and the use of tabs requires changing editor settings), however CODING-STYLE says it is spaces.
Document this explicitly by adding a .editorconfig file (see editorconfig.org), which is an editor agnostic format for specifying basic style properties like indentation, either with native support in editors or via plugins.
It is safer than modelines because it only supports controlling a restricted set of editor properties and not arbitrary commands as Vim modelines would have, and works with editors other than Vim too. (Vim has a deny list for modeline sandboxing, which is error-prone because every time a new command gets added it needs to be added to the deny list, which has been the source of a few CVEs in the past and I disable Vim modelines everywhere as a precaution).
This file is added as a convenience for those who might have an editor that supports it, and its presence should have no impact on those that do not (want to) use it.
It also won't cause re-indentation of existing files when edited, only newly added lines would follow the convention.
No functional change.
--- .editorconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 .editorconfig
.editorconfig new file +20 -0
@@ -0,0 +1,20 @@ +# See ./CODING_STYLE +root = true + +[*] +end_of_line = lf +indent_style = space +charset = utf-8 +max_line_length = 79 +trim_trailing_whitespace = true +insert_final_newline = true + +# Makefiles must use tabs, otherwise they don't work +[{Makefile,*.mk,Makefile.rules}] +indent_style = tabs + +[*.{c,h}] +indent_size = 4 + +[*.{ml,mli}] +indent_size = 2
Signature · 1 line
2.34.1
Re: [PATCH v4 05/11] CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation uses spaces
Quoted 30 lines · Edwin Török
Add an .editorconfig to make it easier to keep patches compatible with Xen's coding style, and to reemphasize what Xen's coding style is.
I thought that Xen demands tabs rather than spaces (which is more difficult with OCaml because indentation tools use spaces, and the use of tabs requires changing editor settings), however CODING-STYLE says it is spaces.
Document this explicitly by adding a .editorconfig file (see editorconfig.org), which is an editor agnostic format for specifying basic style properties like indentation, either with native support in editors or via plugins.
It is safer than modelines because it only supports controlling a restricted set of editor properties and not arbitrary commands as Vim modelines would have, and works with editors other than Vim too. (Vim has a deny list for modeline sandboxing, which is error-prone because every time a new command gets added it needs to be added to the deny list, which has been the source of a few CVEs in the past and I disable Vim modelines everywhere as a precaution).
This file is added as a convenience for those who might have an editor that supports it, and its presence should have no impact on those that do not (want to) use it.
It also won't cause re-indentation of existing files when edited, only newly added lines would follow the convention.
Quoted 4 lines
No functional change.
Signed-off-by Edwin Török <[email protected]>Cc Christian Lindig <[email protected]>
I am agnostic about this change. Automatic code formatting should take care of this but I see no harm including this file. The number of developers contributing to the OCaml code base is minuscule and we should try to keep barriers as low as possible. The wrangling about all these minutiae is already too much.
- C
PATCH v4 6/11 tools/ocaml: add .clang-format
In reply to Edwin TörökAdd a .clang-format configuration that tries to match CODING_STYLE where possible.
I was not able to express the special casing of braces after 'do' though, this can only be controlled generally for all control statements.
It is imperfect, but should be better than the existing bindings, which do not follow Xen coding style.
Add this to tools/ocaml first because:
* there are relatively few C files here, and it is a good place to start with
* it'd be useful to make these follow Xen's CODING_STYLE
(which they currently do not because they use tabs for example)
* they change relatively infrequently, so shouldn't cause issues with backporting security fixes (could either backport the reindentation patch too, or use git cherry-pick with -Xignore-space-change)
Does not yet reformat any code.
No functional change.
---
Changes since v1:
* change commit title to reflect this is for OCaml subtree only
* don't mention stdint.h here, that may be fixed in a different way elsewhere
* add Acked-by line
--- tools/ocaml/.clang-format | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 tools/ocaml/.clang-format
tools/ocaml/.clang-format new file +9 -0
@@ -0,0 +1,9 @@ +BasedOnStyle: GNU +IndentWidth: 4 + +# override GNU to match Xen ../../CODING_STYLE more closely +AlwaysBreakAfterDefinitionReturnType: None +AlwaysBreakAfterReturnType: None +SpacesInConditionalStatement: true +SpaceBeforeParens: ControlStatements +BreakBeforeBraces: Allman
Signature · 1 line
2.34.1
Re: [PATCH v4 06/11] tools/ocaml: add .clang-format
Quoted 2 lines · Edwin Török
Add a .clang-format configuration that tries to match
CODING_STYLEwhere possible.
Is there going to be a patch applying this to "tools/ocaml", like you did with "make format"?
Quoted 5 lines
I was not able to express the special casing of braces after 'do' though, this can only be controlled generally for all control statements.
It is imperfect, but should be better than the existing bindings, which do not follow Xen coding style.
There isn't a single CODING_STYLE for all code in the repo so it isn't an issue if the binding where having a different coding style. But having as few coding style as possible in the repo is probably helpful.
You could maybe add a CODING_STYLE in "tools/ocaml" to say that the coding style is running make format or clang-format .... And if there other rules like how to name certain variable, that could go in this file as well.
Cheers,
Signature · 1 line
Anthony PERARD
Re: [PATCH v4 06/11] tools/ocaml: add .clang-format
Quoted 10 lines · Anthony PERARD
Quoted 2 lines · Edwin Török
Add a .clang-format configuration that tries to match
CODING_STYLEwhere possible.Is there going to be a patch applying this to "
tools/ocaml", like you did with "make format"?
Long term probably, not yet.
For now I just want to experiment with it to see how well and consistently it works (especially with different versions of the tool). (I've been using ocp-indent and ocamlformat for years now, clang-format would be a brand new addition.
There are some solutions for ocamlformat that might be useful for backports, e.g. there is a git merge driver
that reformats all 3 sides of a merge and then attempt to solve conflict/do the merge on the resulting formatted files, so even if old code was formatted differently or with a different version it might be able to apply the changes. I've yet to try it though, and I'm not yet sure how easy it would be to integrate something like that with 'guilt' or 'stgit' which is what Xen uses to manage its patch-queue for backports, at least internally.
If there was an ocp-indent equivalent for C that just re-indents that might be something we could apply right now, but I'm not aware of one.
)
Quoted 10 lines
Quoted 6 lines
I was not able to expr
ess the special casing of braces after 'do'
though, this can only be controlled generally for all control statements.
It is imperfect, but should be better than the existing bindings, which do not follow Xen coding style.There isn't a single
CODING_STYLEfor all code in the repo so it isn't an issue if the binding where having a different coding style. But having as few coding style as possible in the repo is probably helpful.
Indeed, having *a* coding style (and automated checking/formatting) is what I'm more interested in rather than which particular one. The current one used by the bindings is not documented, and although I can guess at based on surrounding code, that surrounding code is subtly different based on when it was written, so it seemed better to adopt what is already documented in CODING_STYLE.
Quoted 6 lines
You could maybe add a
CODING_STYLEin "tools/ocaml" to say that the coding style is runningmake formatorclang-format .... And if there other rules like how to name certain variable, that could go in this file as well.
Having a way to easily run these formatters would be a plus.
Perhaps extending one of the container images that Xen already has to include (a recent enough version of) ocp-indent and clang-format, and similarly for the .spec file it might be useful to update its BuildRequires to bring in the necessary formatters to make it easier to use this in backports.
And then all that can be documented in a CODING_STYLE or CONTRIBUTING.txt/CONTRIBUTING.md file in tools/ocaml.
Best regards,
--Edwin
PATCH v4 7/11 fixup! tools/ocaml/xenctrl: OCaml 5 support, fix use-after-free
In reply to Edwin TörökFix unused value warning.
---
Changes:
* new patch
--- tools/ocaml/libs/xc/xenctrl.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/ocaml/libs/xc/xenctrl.ml +1 -1
@@ -182,7 +182,7 @@ let get_handle () = !handle let close_handle () = match !handle with - | Some h -> handle := None + | Some _ -> handle := None | None -> () let with_intf f =
Signature · 1 line
2.34.1
PATCH v4 8/11 tools/ocaml/libs/mmap: mark mmap/munmap as blocking and raise Unix_error on failure
In reply to Edwin TörökThese functions can potentially take some time, so allow other OCaml code to proceed meanwhile (if any).
Also raise a Unix_error based on errno if mmap fails, instead of just calling failwith (which would lose the error reason).
---
Changes since v2:
* repost of lost patch from 2020 (posted to ML on 2021
https://patchwork.kernel.org/project/xen-devel/patch/294a60be29027d33b0a1d154b7d576237c7dd420.1620755942.git.edvin.torok@citrix.com/)
--- tools/ocaml/libs/mmap/xenmmap_stubs.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
tools/ocaml/libs/mmap/xenmmap_stubs.c +12 -3
@@ -27,6 +27,8 @@#include <caml/custom.h> #include <caml/fail.h> #include <caml/callback.h> +#include <caml/unixsupport.h> +#include <caml/signals.h> #define Intf_val (a) ((struct mmap_interface *) Data_abstract_val(a))
@@ -35,7 +37,9 @@ static int mmap_interface_init(struct mmap_interface *intf, int len, int offset) { intf->len = len; +caml_enter_blocking_section (); intf->addr =mmap (NULL , len, pflag, mflag, fd, offset); +caml_leave_blocking_section ();return (intf->addr ==MAP_FAILED ) ? errno :0 ; }
@@ -64,17 +68,22 @@ CAMLprim value stub_mmap_init(value fd, value pflag, value mflag,if (mmap_interface_init (Intf_val (result),Int_val (fd), c_pflag, c_mflag,Int_val (len),Int_val (offset))) -caml_failwith ("mmap" ); +uerror ("mmap" , Nothing);CAMLreturn (result); } CAMLprim value stub_mmap_final(value intf) {CAMLparam1 (intf); +struct mmap_interface interface = *Intf_val (intf); -if (Intf_val (intf)->addr !=MAP_FAILED ) -munmap (Intf_val (intf)->addr,Intf_val (intf)->len); +/* mark it as freed, in case munmap below fails, so we don't retry it */ Intf_val (intf)->addr =MAP_FAILED ; +if (interface.addr !=MAP_FAILED ) { +caml_enter_blocking_section (); +munmap (interface.addr, interface.len); +caml_leave_blocking_section (); + }CAMLreturn (Val_unit); }
Signature · 1 line
2.34.1
PATCH v4 9/11 tools/ocaml/libs/xb: check for unmapped ring before accessing it
In reply to Edwin TörökXenmmap can unmap the ring, check for this condition before accessing it to avoid crashing on an unmapped page.
Note that we cannot use the usual OCaml finalizers (like bigarray would) to perform the unmap, because that might keep a reference count to a foreign domain's memory that we want to release before destroying the domain.
---
Changes:
* new patch
--- tools/ocaml/libs/xb/xs_ring_stubs.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
tools/ocaml/libs/xb/xs_ring_stubs.c +8 -3
@@ -35,7 +35,14 @@#include <sys/mman.h> #include "mmap_stubs.h" -#define GET_C_STRUCT (a) ((struct mmap_interface *) Data_abstract_val(a)) +static struct mmap_interface *check_addr (struct mmap_interface *interface) +{ +if (!interface->addr || interface->addr ==MAP_FAILED ) +caml_failwith ("ring is not mapped" ); +return interface; +} + +#define GET_C_STRUCT (a) check_addr((struct mmap_interface *) Data_abstract_val(a))/* * Bytes_val has been introduced by Ocaml 4.06.1. So define our own version
@@ -167,8 +174,6 @@ CAMLprim value ml_interface_set_server_features(value interface, value v) {CAMLparam2 (interface, v);struct xenstore_domain_interface *intf =GET_C_STRUCT (interface)->addr; -if (intf == (void *)MAP_FAILED ) -caml_failwith ("Interface closed" ); intf->server_features =Int_val (v);
Signature · 1 line
2.34.1
PATCH v4 10/11 tools/ocaml/xenstored: validate config file before live update
In reply to Edwin TörökThe configuration file can contain typos or various errors that could prevent live update from succeeding (e.g. a flag only valid on a different version). Unknown entries in the config file would be ignored on startup normally, add a strict --config-test that live-update can use to check that the config file is valid *for the new binary*.
For compatibility with running old code during live update recognize --live --help as an equivalent to --config-test.
---
Changes since v2:
* repost of lost patch from 2021: https://patchwork.kernel.org/project/xen-devel/patch/a53934dfa8ef984bffa858cc573cc7a6445bbdc0.1620755942.git.edvin.torok@citrix.com/
--- tools/ocaml/xenstored/parse_arg.ml | 26 ++++++++++++++++++++++++++ tools/ocaml/xenstored/xenstored.ml | 11 +++++++++-- 2 files changed, 35 insertions(+), 2 deletions(-)
tools/ocaml/xenstored/parse_arg.ml +26 -0
@@ -26,8 +26,14 @@ type config = restart: bool; live_reload: bool; disable_socket: bool; + config_test: bool; } +let get_config_filename config_file = + match config_file with + | Some name -> name + | None -> Define.default_config_dir ^ "/oxenstored.conf" + let do_argv = let pidfile = ref "" and tracefile = ref "" (* old xenstored compatibility *) and domain_init = ref true
@@ -38,6 +44,8 @@ let do_argv = and restart = ref false and live_reload = ref false and disable_socket = ref false + and config_test = ref false + and help = ref false in let speclist =
@@ -55,10 +63,27 @@ let do_argv = ("-T", Arg.Set_string tracefile, ""); (* for compatibility *) ("--restart", Arg.Set restart, "Read database on starting"); ("--live", Arg.Set live_reload, "Read live dump on startup"); + ("--config-test", Arg.Set config_test, "Test validity of config file"); ("--disable-socket", Arg.Unit (fun () -> disable_socket := true), "Disable socket"); + ("--help", Arg.Set help, "Display this list of options") ] in let usage_msg = "usage : xenstored [--config-file <filename>] [--no-domain-init] [--help] [--no-fork] [--reraise-top-level] [--restart] [--disable-socket]" in Arg.parse speclist (fun _ -> ()) usage_msg; + let () = + if !help then begin + if !live_reload then + (* + Transform --live --help into --config-test for backward compat with + running code during live update. + Caller will validate config and exit + *) + config_test := true + else begin + Arg.usage_string speclist usage_msg |> print_endline; + exit 0 + end + end + in { domain_init = !domain_init; activate_access_log = !activate_access_log;
@@ -70,4 +95,5 @@ let do_argv = restart = !restart; live_reload = !live_reload; disable_socket = !disable_socket; + config_test = !config_test; }
tools/ocaml/xenstored/xenstored.ml +9 -2
@@ -88,7 +88,7 @@ let default_pidfile = Paths.xen_run_dir ^ "/xenstored.pid" let ring_scan_interval = ref 20 -let parse_config filename = +let parse_config ?(strict=false) filename = let pidfile = ref default_pidfile in let options = [ ("merge-activate", Config.Set_bool Transaction.do_coalesce);
@@ -129,11 +129,12 @@ let parse_config filename = ("xenstored-port", Config.Set_string Domains.xenstored_port); ] in begin try Config.read filename options (fun _ _ -> raise Not_found) with - | Config.Error err -> List.iter (fun (k, e) -> + | Config.Error err as e -> List.iter (fun (k, e) -> match e with | "unknown key" -> eprintf "config: unknown key %s\n" k | _ -> eprintf "config: %s: %s\n" k e ) err; + if strict then raise e | Sys_error m -> eprintf "error: config: %s\n" m; end; !pidfile
@@ -358,6 +359,12 @@ let tweak_gc () = let () = Printexc.set_uncaught_exception_handler Logging.fallback_exception_handler; let cf = do_argv in + if cf.config_test then begin + let path = config_filename cf in + let _pidfile:string = parse_config ~strict:true path in + Printf.printf "Configuration valid at %s\n%!" path; + exit 0 + end; let pidfile = if Sys.file_exists (config_filename cf) then parse_config (config_filename cf)
Signature · 1 line
2.34.1
Re: [PATCH v4 10/11] tools/ocaml/xenstored: validate config file before live update
Quoted 9 lines · Edwin Török
The configuration file can contain typos or various errors that could prevent live update from succeeding (e.g. a flag only valid on a different version). Unknown entries in the config file would be ignored on startup normally, add a strict --config-test that live-update can use to check that the config file is valid *for the new binary*.
Is the configuration tested, checked, or validated? If feel “check" or “validate" would convey better what is happening.
Quoted 4 lines
For compatibility with running old code during live update recognize --live --help as an equivalent to --config-test.
Signed-off-by Edwin Török <[email protected]
Quoted 14 lines
Quoted 1 line
---
Changes since v2:
* repost of lost patch from 2021: https://patchwork.kernel.org/project/xen-devel/patch/a53934dfa8ef984bffa858cc573cc7a6445bbdc0.1620755942.git.edvin.torok@citrix.com/
---tools/ocaml/xenstored/parse_arg.ml| 26 ++++++++++++++++++++++++++tools/ocaml/xenstored/xenstored.ml| 11 +++++++++--2 files changed, 35 insertions(+), 2 deletions(-)tools/ocaml/xenstored/parse_arg.ml +0
-0
tools/ocaml/xenstored/parse_arg.ml type config = · line 35
Quoted 10 lines
tools/ocaml/xenstored/parse_arg.ml +5
-0@@ -26,8 +26,14 @@ type config = restart: bool; live_reload: bool; disable_socket: bool; + config_test: bool; } +let get_config_filename config_file = + match config_file with + | Some name -> name + | None -> Define.default_config_dir ^ "/oxenstored.conf"
I’d use Filename.concat.
Quoted 14 lines
tools/ocaml/xenstored/parse_arg.ml +1
-0+let
do_argv=
let pidfile = ref "" and tracefile = ref "" (* old xenstored compatibility *) anddomain_init= ref truetools/ocaml/xenstored/parse_arg.ml +2
-0@@ -38,6 +44,8 @@ let do_argv = and restart = ref false and live_reload = ref false and disable_socket = ref false + and config_test = ref false + and help = ref false in let speclist =
tools/ocaml/xenstored/parse_arg.ml let do_argv = · line 66
Quoted 4 lines
tools/ocaml/xenstored/parse_arg.ml +1
-0@@ -55,10 +63,27 @@ let do_argv = ("-T", Arg.Set_string tracefile, ""); (* for compatibility *) ("--restart", Arg.Set restart, "Read database on starting"); ("--live", Arg.Set live_reload, "Read live dump on startup"); + ("--config-test", Arg.Set config_test, "Test validity of config file");
I see the logic how this flag was named but feel starting with a verb (“validate”, “check”, “test”) leads to a clearer invocation pattern.
Quoted 72 lines
tools/ocaml/xenstored/parse_arg.ml +17
-0("--disable-socket", Arg.Unit (fun () -> disable_socket := true), "Disable socket"); + ("--help", Arg.Set help, "Display this list of options") ] in let usage_msg = "usage : xenstored [--config-file <filename>] [--no-domain-init] [--help] [--no-fork] [--reraise-top-level] [--restart] [--disable-socket]" in Arg.parse speclist (fun _ -> ()) usage_msg; + let () = + if !help then begin + if !live_reload then + (* + Transform --live --help into --config-test for backward compat with + running code during live update. + Caller will validate config and exit + *) + config_test := true + else begin + Arg.usage_string speclist usage_msg |> print_endline; + exit 0 + end + end + in { domain_init = !domain_init; activate_access_log = !activate_access_log;@@ -70,4 +95,5 @@ let do_argv = restart = !restart; live_reload = !live_reload; disable_socket = !disable_socket; + config_test = !config_test; }tools/ocaml/xenstored/xenstored.ml +0
-0@@ -88,7 +88,7 @@ let default_pidfile = Paths.xen_run_dir ^ "/xenstored.pid"let
ring_scan_interval= ref 20tools/ocaml/xenstored/xenstored.ml +3
-2-let parse_config filename = +let parse_config ?(strict=false) filename = let pidfile = ref default_pidfile in let options = [ ("merge-activate", Config.Set_bool Transaction.do_coalesce);@@ -129,11 +129,12 @@ let parse_config filename = ("xenstored-port", Config.Set_string Domains.xenstored_port); ] in begin try Config.read filename options (fun _ _ -> raise Not_found) with - | Config.Error err -> List.iter (fun (k, e) -> + | Config.Error err as e -> List.iter (fun (k, e) -> match e with | "unknown key" -> eprintf "config: unknown key %s\n" k | _ -> eprintf "config: %s: %s\n" k e ) err; + if strict then raise e | Sys_error m -> eprintf "error: config: %s\n" m; end; !pidfile@@ -358,6 +359,12 @@ let tweak_gc () =let () =
Printexc.set_uncaught_exception_handler Logging.fallback_exception_handler; let cf =do_argvintools/ocaml/xenstored/xenstored.ml +6
-0+ if cf.config_test then begin + let path = config_filename cf in + let _pidfile:string = parse_config ~strict:true path in + Printf.printf "Configuration valid at %s\n%!" path; + exit 0 + end; let pidfile = if Sys.file_exists (config_filename cf) then parse_config (config_filename cf)Signature · 1 line
2.34.1
Re: [PATCH v4 10/11] tools/ocaml/xenstored: validate config file before live update
Quoted 8 lines · Christian Lindig
Quoted 5 lines · Edwin Török
Quoted 7 lines
Quoted 5 lines
The configuration file can contain typos or various errors that could prevent live update from succeeding (e.g. a flag only valid on a different version). Unknown entries in the config file would be ignored on startup normally, add a strict --config-test that live-update can use to check that the config file is valid *for the new binary*.
Is the configuration tested, checked, or validated? If feel “check" or “validate" would convey better what is happening.
The rest of the code talks about validation, so I've renamed this to config-validate.
Quoted 8 lines
Quoted 6 lines
For compatibility with running old code during live update recognize --live --help as an equivalent to --config-test.
Signed-off-by Edwin Török <[email protected]Acked-by Christian Lindig <[email protected]>
Thanks, I've pushed an update commit with this change here: https://github.com/edwintorok/xen/compare/staging...private/edvint/review5,
in particular https://github.com/edwintorok/xen/commit/f1a9153bb867bbb5df0f5e17b1ed3348e7ea79f8
Best regards,
--Edwin
Quoted 122 lines
Quoted 15 lines
Quoted 2 lines
---
Changes since v2:
* repost of lost patch from 2021: https://patchwork.kernel.org/project/xen-devel/patch/a53934dfa8ef984bffa858cc573cc7a6445bbdc0.1620755942.git.edvin.torok@citrix.com/
---tools/ocaml/xenstored/parse_arg.ml| 26 ++++++++++++++++++++++++++tools/ocaml/xenstored/xenstored.ml| 11 +++++++++--2 files changed, 35 insertions(+), 2 deletions(-)tools/ocaml/xenstored/parse_arg.ml +0
-0tools/ocaml/xenstored/parse_arg.ml type config = · line 35
Quoted 10 lines
tools/ocaml/xenstored/parse_arg.ml +5
-0@@ -26,8 +26,14 @@ type config = restart: bool; live_reload: bool; disable_socket: bool; + config_test: bool; } +let get_config_filename config_file = + match config_file with + | Some name -> name + | None -> Define.default_config_dir ^ "/oxenstored.conf"I’d use Filename.concat.
Quoted 14 lines
tools/ocaml/xenstored/parse_arg.ml +1
-0+let
do_argv=
let pidfile = ref "" and tracefile = ref "" (* old xenstored compatibility *) anddomain_init= ref truetools/ocaml/xenstored/parse_arg.ml +2
-0@@ -38,6 +44,8 @@ let do_argv = and restart = ref false and live_reload = ref false and disable_socket = ref false + and config_test = ref false + and help = ref false in let speclist =tools/ocaml/xenstored/parse_arg.ml let do_argv = · line 66
Quoted 4 lines
tools/ocaml/xenstored/parse_arg.ml +1
-0@@ -55,10 +63,27 @@ let do_argv = ("-T", Arg.Set_string tracefile, ""); (* for compatibility *) ("--restart", Arg.Set restart, "Read database on starting"); ("--live", Arg.Set live_reload, "Read live dump on startup"); + ("--config-test", Arg.Set config_test, "Test validity of config file");I see the logic how this flag was named but feel starting with a verb (“validate”, “check”, “test”) leads to a clearer invocation pattern.
Quoted 72 lines
tools/ocaml/xenstored/parse_arg.ml +17
-0("--disable-socket", Arg.Unit (fun () -> disable_socket := true), "Disable socket"); + ("--help", Arg.Set help, "Display this list of options") ] in let usage_msg = "usage : xenstored [--config-file <filename>] [--no-domain-init] [--help] [--no-fork] [--reraise-top-level] [--restart] [--disable-socket]" in Arg.parse speclist (fun _ -> ()) usage_msg; + let () = + if !help then begin + if !live_reload then + (* + Transform --live --help into --config-test for backward compat with + running code during live update. + Caller will validate config and exit + *) + config_test := true + else begin + Arg.usage_string speclist usage_msg |> print_endline; + exit 0 + end + end + in { domain_init = !domain_init; activate_access_log = !activate_access_log;@@ -70,4 +95,5 @@ let do_argv = restart = !restart; live_reload = !live_reload; disable_socket = !disable_socket; + config_test = !config_test; }tools/ocaml/xenstored/xenstored.ml +0
-0@@ -88,7 +88,7 @@ let default_pidfile = Paths.xen_run_dir ^ "/xenstored.pid"let
ring_scan_interval= ref 20tools/ocaml/xenstored/xenstored.ml +3
-2-let parse_config filename = +let parse_config ?(strict=false) filename = let pidfile = ref default_pidfile in let options = [ ("merge-activate", Config.Set_bool Transaction.do_coalesce);@@ -129,11 +129,12 @@ let parse_config filename = ("xenstored-port", Config.Set_string Domains.xenstored_port); ] in begin try Config.read filename options (fun _ _ -> raise Not_found) with - | Config.Error err -> List.iter (fun (k, e) -> + | Config.Error err as e -> List.iter (fun (k, e) -> match e with | "unknown key" -> eprintf "config: unknown key %s\n" k | _ -> eprintf "config: %s: %s\n" k e ) err; + if strict then raise e | Sys_error m -> eprintf "error: config: %s\n" m; end; !pidfile@@ -358,6 +359,12 @@ let tweak_gc () =let () =
Printexc.set_uncaught_exception_handler Logging.fallback_exception_handler; let cf =do_argvintools/ocaml/xenstored/xenstored.ml +6
-0+ if cf.config_test then begin + let path = config_filename cf in + let _pidfile:string = parse_config ~strict:true path in + Printf.printf "Configuration valid at %s\n%!" path; + exit 0 + end; let pidfile = if Sys.file_exists (config_filename cf) then parse_config (config_filename cf)Signature · 1 line
2.34.1
PATCH v4 11/11 tools/ocaml/libs/xb: drop Xs_ring.write
In reply to Edwin TörökUnused, only Xs_ring.write_substring is used.
Also the bytes/string conversion here is backwards: the C stub implements the bytes version and then we use a Bytes.unsafe_of_string to convert a string into bytes.
However the operation here really is read-only: we read from the string and write it to the ring, so the C stub should implement the read-only string version, and if needed we could use Bytes.unsafe_to_string to be able to send 'bytes'. However that is not necessary as the 'bytes' version is dropped above.
---
Changes:
* new patch
--- tools/ocaml/libs/xb/xs_ring.ml | 5 +---- tools/ocaml/libs/xb/xs_ring_stubs.c | 2 +- 2 files changed, 2 insertions(+), 5 deletions(-)
tools/ocaml/libs/xb/xs_ring.ml +1 -4
@@ -25,14 +25,11 @@ module Server_features = Set.Make(struct end) external read: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_read" -external write: Xenmmap.mmap_interface -> bytes -> int -> int = "ml_interface_write" +external write_substring: Xenmmap.mmap_interface -> string -> int -> int = "ml_interface_write" external _internal_set_server_features: Xenmmap.mmap_interface -> int -> unit = "ml_interface_set_server_features" [@@noalloc] external _internal_get_server_features: Xenmmap.mmap_interface -> int = "ml_interface_get_server_features" [@@noalloc] -let write_substring mmap buff len = - write mmap (Bytes.unsafe_of_string buff) len - let get_server_features mmap = (* NB only one feature currently defined above *) let x = _internal_get_server_features mmap in
tools/ocaml/libs/xb/xs_ring_stubs.c +1 -1
@@ -119,7 +119,7 @@ CAMLprim value ml_interface_write(value ml_interface,CAMLlocal1 (ml_result);struct mmap_interface *interface =GET_C_STRUCT (ml_interface); -const unsigned char *buffer =Bytes_val (ml_buffer); +const char *buffer =String_val (ml_buffer);int len =Int_val (ml_len);int result;
Signature · 1 line
2.34.1
Quoted 8 lines · Edwin Török
From: Edwin Török <[email protected]>
Various OCaml fixes, some of which got Acked already.
Quoted 14 lines
Edwin Török (11):
tools/ocaml/libs/{xb, mmap}: useData_abstract_valwrappertools/ocaml/xenstored/Makefile: use ocamldep -sort for linking orderCODING_STYLE(tools/ocaml): add 'make format' and remove tabstools/ocaml: run "make format"
CODING-STYLE(tools/ocaml): add .editorconfig to clarify indentation
uses spaces
tools/ocaml: add .clang-format
fixup!tools/ocaml/xenctrl: OCaml 5 support, fix use-after-freetools/ocaml/libs/mmap: mark mmap/munmap as blocking and raise
Unix_erroron failure
tools/ocaml/libs/xb: check for unmapped ring before accessing ittools/ocaml/xenstored: validate config file before live updatetools/ocaml/libs/xb: drop Xs_ring.write
I’m going to Ack this whole sale and provide individual comments.