Skip to contentSkip to conversation

tools/xenstore: support live update for xenstored

PATCH Superseded Juergen Gross 4 participants 34 messages Dec 15, 2020 – Jan 13, 2021

v10 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

v10 cover tools/xenstore: support live update for xenstored

Today Xenstore is not restartable. This means a Xen server needing an update of xenstored has to be rebooted in order to let this update become effective.

This patch series is changing that: The internal state of xenstored (the contents of Xenstore, all connections to various clients like programs or other domains, and watches) is saved in a defined format and a new binary is being activated consuming the old state. All connections are being restored and the new Xenstore binary will continue where the old one stopped.

This patch series has been under (secret) development for quite some time. It hasn't been posted to xen-devel until now due to the various Xenstore related security issues which have become public only today.

There will be a similar series for oxenstored posted.

Xenstore-stubdom is not yet supported, but I'm planning to start working on that soon.

Changes in V10 (for the members of the security team):
- dropped patch 6 as requested by Andrew

Juergen Gross (24):
tools/xenstore: switch barf[_perror]() to use syslog() tools/xenstore: make set_tdb_key() non-static tools/xenstore: remove unused cruft from xenstored_domain.c tools/libxenevtchn: add possibility to not close file descriptor on
exec
tools/xenstore: refactor XS_CONTROL handling tools/xenstore: add live update command to xenstore-control tools/xenstore: add basic live-update command parsing tools/xenstore: introduce live update status block tools/xenstore: save new binary for live update tools/xenstore: add command line handling for live update tools/xenstore: add the basic framework for doing the live update tools/xenstore: allow live update only with no transaction active
docs: update the xenstore migration stream documentation tools/xenstore: add include file for state structure definitions tools/xenstore: dump the xenstore state for live update tools/xenstore: handle CLOEXEC flag for local files and pipes tools/xenstore: split off domain introduction from do_introduce() tools/xenstore: evaluate the live update flag when starting tools/xenstore: read internal state when doing live upgrade tools/xenstore: add reading global state for live update tools/xenstore: add read connection state for live update tools/xenstore: add read node state for live update tools/xenstore: add read watch state for live update tools/xenstore: activate new binary for live update

Julien Grall (1):
tools/xenstore: handle dying domains in live update

 docs/designs/xenstore-migration.md      |  19 +-
 docs/misc/xenstore.txt                  |  21 +
 tools/include/xenevtchn.h               |  16 +-
 tools/libs/evtchn/Makefile              |   2 +-
 tools/libs/evtchn/core.c                |  45 +-
 tools/libs/evtchn/freebsd.c             |   9 +-
 tools/libs/evtchn/libxenevtchn.map      |   4 +
 tools/libs/evtchn/linux.c               |   9 +-
 tools/libs/evtchn/minios.c              |   6 +-
 tools/libs/evtchn/netbsd.c              |   2 +-
 tools/libs/evtchn/private.h             |   2 +-
 tools/libs/evtchn/solaris.c             |   2 +-
 tools/xenstore/Makefile                 |   3 +-
 tools/xenstore/include/xenstore_state.h | 131 +++++
 tools/xenstore/utils.c                  |  20 +
 tools/xenstore/utils.h                  |   6 +
 tools/xenstore/xenstore_control.c       | 332 ++++++++++++-
 tools/xenstore/xenstored_control.c      | 612 +++++++++++++++++++++++-
 tools/xenstore/xenstored_control.h      |   1 +
 tools/xenstore/xenstored_core.c         | 510 ++++++++++++++++++--
 tools/xenstore/xenstored_core.h         |  40 ++
 tools/xenstore/xenstored_domain.c       | 312 +++++++++---
 tools/xenstore/xenstored_domain.h       |  14 +-
 tools/xenstore/xenstored_posix.c        |  13 +-
 tools/xenstore/xenstored_transaction.c  |  11 +-
 tools/xenstore/xenstored_watch.c        | 171 +++++--
 tools/xenstore/xenstored_watch.h        |   5 +
 27 files changed, 2103 insertions(+), 215 deletions(-)
 create mode 100644 tools/xenstore/include/xenstore_state.h
Signature · 1 line
2.26.2
#1 lore raw

PATCH v10 1/25 tools/xenstore: switch barf[_perror]() to use syslog()

When xenstored crashes due to an unrecoverable condition it is calling either barf() or barf_perror() to issue a message and then exit().

Make sure the message is visible somewhere by using syslog() additionally to xprintf(), as the latter will be visible only with tracing active.

---
V4:
- new patch

---
 tools/xenstore/utils.c | 3 +++
 1 file changed, 3 insertions(+)
tools/xenstore/utils.c +3 -0
index a1ac12584a..633ce3b4fc 100644
@@ -3,6 +3,7 @@
 #include <stdarg.h>
 #include <stdlib.h>
 #include <string.h>
+#include <syslog.h>
 #include <errno.h>
 #include <unistd.h>
 #include <fcntl.h>
@@ -35,6 +36,7 @@ void barf(const char *fmt, ...)
 	va_end(arglist);
 
  	if (bytes >= 0) {
+		syslog(LOG_CRIT, "%s\n", str);
 		xprintf("%s\n", str);
 		free(str);
 	}
@@ -54,6 +56,7 @@ void barf_perror(const char *fmt, ...)
 	va_end(arglist);
 
  	if (bytes >= 0) {
+		syslog(LOG_CRIT, "%s: %s\n", str, strerror(err));
 		xprintf("%s: %s\n", str, strerror(err));
 		free(str);
 	}
Signature · 1 line
2.26.2
#2 lore raw

PATCH v10 2/25 tools/xenstore: make set_tdb_key() non-static

In reply to Juergen Gross

set_tdb_key() can be used by destroy_node(), too. So remove the static attribute and move it to xenstored_core.c.

---
V5:
- new patch

V6:
- add comment (Julien Grall)

---
 tools/xenstore/xenstored_core.c        | 14 +++++++++++---
 tools/xenstore/xenstored_core.h        |  2 ++
 tools/xenstore/xenstored_transaction.c |  6 ------
 3 files changed, 13 insertions(+), 9 deletions(-)
tools/xenstore/xenstored_core.c +11 -3
index 3082a36d3a..ab1c7835b8 100644
@@ -352,6 +352,16 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout)
 	}
 }
 
+void set_tdb_key(const char *name, TDB_DATA *key)
+{
+	/*
+	 * Dropping const is fine here, as the key will never be modified
+	 * by TDB.
+	 */
+	key->dptr = (char *)name;
+	key->dsize = strlen(name);
+}
+
 /*
  * If it fails, returns NULL and sets errno.
  * Temporary memory allocations will be done with ctx.
@@ -985,9 +995,7 @@ static int destroy_node(void *_node)
 	if (streq(node->name, "/"))
 		corrupt(NULL, "Destroying root node!");
 
-	key.dptr = (void *)node->name;
-	key.dsize = strlen(node->name);
-
+	set_tdb_key(node->name, &key);
 	tdb_delete(tdb_ctx, key);
 
 	domain_entry_dec(talloc_parent(node), node);
tools/xenstore/xenstored_core.h +2 -0
index 4c6c3d6f20..fb59d862a2 100644
@@ -220,6 +220,8 @@ extern xengnttab_handle **xgt_handle;
 
 int remember_string(struct hashtable *hash, const char *str);
 
+void set_tdb_key(const char *name, TDB_DATA *key);
+
 #endif /* _XENSTORED_CORE_H */
 
 /*
tools/xenstore/xenstored_transaction.c +0 -6
index 2881f3b2e4..52355f4ed8 100644
@@ -168,12 +168,6 @@ struct transaction
 extern int quota_max_transaction;
 uint64_t generation;
 
-static void set_tdb_key(const char *name, TDB_DATA *key)
-{
-	key->dptr = (char *)name;
-	key->dsize = strlen(name);
-}
-
 static struct accessed_node *find_accessed_node(struct transaction *trans,
 						const char *name)
 {
Signature · 1 line
2.26.2
#3 lore raw

PATCH v10 3/25 tools/xenstore: remove unused cruft from xenstored_domain.c

In reply to Juergen Gross

domain->remote_port and restore_existing_connections() are useless and can be removed.

---
V7:
- new patch

---
 tools/xenstore/xenstored_core.c   |  3 ---
 tools/xenstore/xenstored_domain.c | 11 -----------
 tools/xenstore/xenstored_domain.h |  3 ---
 3 files changed, 17 deletions(-)
tools/xenstore/xenstored_core.c +0 -3
index ab1c7835b8..50986f8b29 100644
@@ -2087,9 +2087,6 @@ int main(int argc, char *argv[])
 	if (!no_domain_init)
 		domain_init();
 
-	/* Restore existing connections. */
-	restore_existing_connections();
-
 	if (outputpid) {
 		printf("%ld\n", (long)getpid());
 		fflush(stdout);
tools/xenstore/xenstored_domain.c +0 -11
index 7d348d57f3..ed8e83b06b 100644
@@ -54,10 +54,6 @@ struct domain
 	/* Event channel port */
 	evtchn_port_t port;
 
-	/* The remote end of the event channel, used only to validate
-	   repeated domain introductions. */
-	evtchn_port_t remote_port;
-
 	/* Domain path in store. */
 	char *path;
 
@@ -382,7 +378,6 @@ static int new_domain(struct domain *domain, int port)
 	domain->conn->domain = domain;
 	domain->conn->id = domain->domid;
 
-	domain->remote_port = port;
 	domain->nbentry = 0;
 	domain->nbwatch = 0;
 
@@ -470,7 +465,6 @@ int do_introduce(struct connection *conn, struct buffered_data *in)
 			xenevtchn_unbind(xce_handle, domain->port);
 		rc = xenevtchn_bind_interdomain(xce_handle, domid, port);
 		domain->port = (rc == -1) ? 0 : rc;
-		domain->remote_port = port;
 	}
 
 	domain_conn_reset(domain);
@@ -636,11 +630,6 @@ const char *get_implicit_path(const struct connection *conn)
 	return conn->domain->path;
 }
 
-/* Restore existing connections. */
-void restore_existing_connections(void)
-{
-}
-
 static int set_dom_perms_default(struct node_perms *perms)
 {
 	perms->num = 1;
tools/xenstore/xenstored_domain.h +0 -3
index 5e00087206..66e0a12654 100644
@@ -47,9 +47,6 @@ void domain_init(void);
 /* Returns the implicit path of a connection (only domains have this) */
 const char *get_implicit_path(const struct connection *conn);
 
-/* Read existing connection information from store. */
-void restore_existing_connections(void);
-
 /* Can connection attached to domain read/write. */
 bool domain_can_read(struct connection *conn);
 bool domain_can_write(struct connection *conn);
Signature · 1 line
2.26.2
#4 lore raw

PATCH v10 4/25 tools/libxenevtchn: add possibility to not close file descriptor on exec

In reply to Juergen Gross

Today the file descriptor for the access of the event channel driver is being closed in case of exec(2). For the support of live update of a daemon using libxenevtchn this can be problematic, so add a way to keep that file descriptor open.

Add support of a flag XENEVTCHN_NO_CLOEXEC for xenevtchn_open() which will result in _not_ setting O_CLOEXEC when opening the event channel driver node.

The caller can then obtain the file descriptor via xenevtchn_fd().

Add an alternative open function xenevtchn_open_fd() which takes that file descriptor as an additional parameter. This allows to allocate a xenevtchn_handle and to associate it with that file descriptor.

---
V7:
- new patch

V8:
- some minor comments by Julien Grall addressed

---
 tools/include/xenevtchn.h          | 16 ++++++++++-
 tools/libs/evtchn/Makefile         |  2 +-
 tools/libs/evtchn/core.c           | 45 ++++++++++++++++++++++++------
 tools/libs/evtchn/freebsd.c        |  9 ++++--
 tools/libs/evtchn/libxenevtchn.map |  4 +++
 tools/libs/evtchn/linux.c          |  9 ++++--
 tools/libs/evtchn/minios.c         |  6 +++-
 tools/libs/evtchn/netbsd.c         |  2 +-
 tools/libs/evtchn/private.h        |  2 +-
 tools/libs/evtchn/solaris.c        |  2 +-
 10 files changed, 79 insertions(+), 18 deletions(-)
tools/include/xenevtchn.h +15 -1
index 91821ee56d..dadc46ea36 100644
@@ -64,11 +64,25 @@ struct xentoollog_logger;
  *
  * Calling xenevtchn_close() is the only safe operation on a
  * xenevtchn_handle which has been inherited.
+ *
+ * Setting XENEVTCHN_NO_CLOEXEC allows to keep the file descriptor used
+ * for the event channel driver open across exec(2). In order to be able
+ * to use that file descriptor the new binary activated via exec(2) has
+ * to call xenevtchn_open_fd() with that file descriptor as parameter in
+ * order to associate it with a new handle. The file descriptor can be
+ * obtained via xenevtchn_fd() before calling exec(2).
  */
-/* Currently no flags are defined */
+
+/* Don't set O_CLOEXEC when opening event channel driver node. */
+#define XENEVTCHN_NO_CLOEXEC 0x01
+
 xenevtchn_handle *xenevtchn_open(struct xentoollog_logger *logger,
                                  unsigned open_flags);
 
+/* Flag XENEVTCHN_NO_CLOEXEC is ignored by xenevtchn_open_fd(). */
+xenevtchn_handle *xenevtchn_open_fd(struct xentoollog_logger *logger,
+                                    int fd, unsigned open_flags);
+
 /*
  * Close a handle previously allocated with xenevtchn_open().
  */
tools/libs/evtchn/Makefile +1 -1
index ad01a17b3d..b8c37b5b97 100644
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR    = 1
-MINOR    = 1
+MINOR    = 2
 
 SRCS-y                 += core.c
 SRCS-$(CONFIG_Linux)   += linux.c
tools/libs/evtchn/core.c +37 -8

72 lines not on this page · raw message

tools/libs/evtchn/freebsd.c +7 -2
index 6564ed4c44..635c10f09f 100644
@@ -31,9 +31,14 @@
 
 #define EVTCHN_DEV      "/dev/xen/evtchn"
 
-int osdep_evtchn_open(xenevtchn_handle *xce)
+int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int open_flags)
 {
-    int fd = open(EVTCHN_DEV, O_RDWR|O_CLOEXEC);
+    int flags = O_RDWR;
+    int fd;
+
+    if ( !(open_flags & XENEVTCHN_NO_CLOEXEC) )
+        flags |= O_CLOEXEC;
+    fd = open(EVTCHN_DEV, flags);
     if ( fd == -1 )
         return -1;
     xce->fd = fd;
tools/libs/evtchn/libxenevtchn.map +4 -0
index 33a38f953a..722fa026f9 100644
@@ -21,3 +21,7 @@ VERS_1.1 {
 	global:
 		xenevtchn_restrict;
 } VERS_1.0;
+VERS_1.2 {
+	global:
+		xenevtchn_open_fd;
+} VERS_1.1;
tools/libs/evtchn/linux.c +7 -2
index 17e64aea32..2297488f88 100644
@@ -34,9 +34,14 @@
 #define O_CLOEXEC 0
 #endif
 
-int osdep_evtchn_open(xenevtchn_handle *xce)
+int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int open_flags)
 {
-    int fd = open("/dev/xen/evtchn", O_RDWR|O_CLOEXEC);
+    int flags = O_RDWR;
+    int fd;
+
+    if ( !(open_flags & XENEVTCHN_NO_CLOEXEC) )
+        flags |= O_CLOEXEC;
+    fd = open("/dev/xen/evtchn", flags);
     if ( fd == -1 )
         return -1;
     xce->fd = fd;
tools/libs/evtchn/minios.c +5 -1
index 9cd7636fc5..f5db021747 100644
@@ -63,7 +63,11 @@ static void port_dealloc(struct evtchn_port_info *port_info) {
     free(port_info);
 }
 
-int osdep_evtchn_open(xenevtchn_handle *xce)
+/*
+ * XENEVTCHN_NO_CLOEXEC is being ignored, as there is no exec() call supported
+ * in Mini-OS.
+ */
+int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int open_flags)
 {
     int fd = alloc_fd(FTYPE_EVTCHN);
     if ( fd == -1 )
tools/libs/evtchn/netbsd.c +1 -1
index 8b8545d2f9..7c73d1c599 100644
@@ -31,7 +31,7 @@
 
 #define EVTCHN_DEV_NAME  "/dev/xenevt"
 
-int osdep_evtchn_open(xenevtchn_handle *xce)
+int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int open_flags)
 {
     int fd = open(EVTCHN_DEV_NAME, O_NONBLOCK|O_RDWR);
     if ( fd == -1 )
tools/libs/evtchn/private.h +1 -1
index 31e595bea2..bcac2a191d 100644
@@ -14,7 +14,7 @@ struct xenevtchn_handle {
     Xentoolcore__Active_Handle tc_ah;
 };
 
-int osdep_evtchn_open(xenevtchn_handle *xce);
+int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int open_flags);
 int osdep_evtchn_close(xenevtchn_handle *xce);
 int osdep_evtchn_restrict(xenevtchn_handle *xce, domid_t domid);
 
tools/libs/evtchn/solaris.c +1 -1
index dd41f62a24..7e22f7906a 100644
@@ -29,7 +29,7 @@
 
 #include "private.h"
 
-int osdep_evtchn_open(xenevtchn_handle *xce)
+int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int open_flags)
 {
     int fd;
 
Signature · 1 line
2.26.2
#5 lore raw

Re: [PATCH v10 04/25] tools/libxenevtchn: add possibility to not close file descriptor on exec

Quoted 11 lines · Juergen Gross

---
V7:
- new patch

V8:
- some minor comments by Julien Grall addressed

Various of your patches still have double SoB.  (Just as a note to be careful to anyone committing...)

Quoted 6 lines
tools/include/xenevtchn.h +0 -0
index 91821ee56d..dadc46ea36 100644
@@ -64,11 +64,25 @@ struct xentoollog_logger;
  *

tools/include/xenevtchn.h struct xentoollog_logger; · line 74

Quoted 10 lines
tools/include/xenevtchn.h +7 -0
  * Calling xenevtchn_close() is the only safe operation on a
  * xenevtchn_handle which has been inherited.
+ *
+ * Setting XENEVTCHN_NO_CLOEXEC allows to keep the file descriptor used
+ * for the event channel driver open across exec(2). In order to be able
+ * to use that file descriptor the new binary activated via exec(2) has
+ * to call xenevtchn_open_fd() with that file descriptor as parameter in
+ * order to associate it with a new handle. The file descriptor can be
+ * obtained via xenevtchn_fd() before calling exec(2).
  */

More of the comment block than this needs adjusting in light of the exec() changes.

tools/include/xenevtchn.h struct xentoollog_logger; · line 85

Quoted 12 lines
tools/include/xenevtchn.h +8 -1
-/* Currently no flags are defined */
+
+/* Don't set O_CLOEXEC when opening event channel driver node. */
+#define XENEVTCHN_NO_CLOEXEC 0x01
+
 xenevtchn_handle *xenevtchn_open(struct xentoollog_logger *logger,
                                  unsigned open_flags);
 
+/* Flag XENEVTCHN_NO_CLOEXEC is ignored by xenevtchn_open_fd(). */
+xenevtchn_handle *xenevtchn_open_fd(struct xentoollog_logger *logger,
+                                    int fd, unsigned open_flags);
+

I spotted this before, but didn't have time to reply.

This isn't "open fd".  It is "construct a xenevtchn_handle object around an already-open fd".  As such, open_flags appears bogus because at no point are we making an open() call.  (I'd argue that it irrespective of other things, it wants naming xenevtchn_fdopen() for API familiarity.)

However, the root of the problem is actually the ambiguity in the name. These are not flags to the open() system call, but general flags for xenevtchn.

Therefore, I recommend a prep patch which renames open_flags to just flags, and while at it, changes to unsigned int rather than a naked "unsigned" type.  There are no API/ABI implications for this, but it will help massively with code clarity.

I'd also possibly go as far as to say that plumbing 'flags' down into osdep ought to split out into a separate patch.  There is also a wild mix of coding styles even within the hunks here.

Additionally, something in core.c should check for unknown flags and reject them them with EINVAL.  It was buggy that this wasn't done before, and really needs to be implemented before we start having cases where people might plausibly pass something other than 0.

~Andrew

P.S. if you don't fancy doing all of this, my brain could do with a break from the complicated work, and I can see about organising this cleanup.

#6 lore raw

Re: [PATCH v10 04/25] tools/libxenevtchn: add possibility to not close file descriptor on exec

Quoted 8 lines · Andrew Cooper
Quoted 7 lines · Juergen Gross

---
V7:
- new patch

Quoted 7 lines
Quoted 4 lines

V8:
- some minor comments by Julien Grall addressed

Various of your patches still have double SoB.  (Just as a note to be careful to anyone committing...)

Yeah, this is annoying.

They are all after the first "---" mark, so they shouldn't end up in git AFAICS.

Why git is adding those duplicates I don't know. I had this problem before, but some git update made it disappear. Now it is back, but not for all patches. :-(

Quoted 36 lines
Quoted 7 lines
tools/include/xenevtchn.h +0 -0
index 91821ee56d..dadc46ea36 100644
@@ -64,11 +64,25 @@ struct xentoollog_logger;
   *

tools/include/xenevtchn.h struct xentoollog_logger; · line 74

Quoted 10 lines
tools/include/xenevtchn.h +7 -0
   * Calling xenevtchn_close() is the only safe operation on a
   * xenevtchn_handle which has been inherited.
+ *
+ * Setting XENEVTCHN_NO_CLOEXEC allows to keep the file descriptor used
+ * for the event channel driver open across exec(2). In order to be able
+ * to use that file descriptor the new binary activated via exec(2) has
+ * to call xenevtchn_open_fd() with that file descriptor as parameter in
+ * order to associate it with a new handle. The file descriptor can be
+ * obtained via xenevtchn_fd() before calling exec(2).
   */

More of the comment block than this needs adjusting in light of the exec() changes.

tools/include/xenevtchn.h struct xentoollog_logger; · line 85

Quoted 12 lines
tools/include/xenevtchn.h +8 -1
-/* Currently no flags are defined */
+
+/* Don't set O_CLOEXEC when opening event channel driver node. */
+#define XENEVTCHN_NO_CLOEXEC 0x01
+
  xenevtchn_handle *xenevtchn_open(struct xentoollog_logger *logger,
                                   unsigned open_flags);
 
+/* Flag XENEVTCHN_NO_CLOEXEC is ignored by xenevtchn_open_fd(). */
+xenevtchn_handle *xenevtchn_open_fd(struct xentoollog_logger *logger,
+                                    int fd, unsigned open_flags);
+

I spotted this before, but didn't have time to reply.

Quoted 4 lines

This isn't "open fd".  It is "construct a xenevtchn_handle object around an already-open fd".  As such, open_flags appears bogus because at no point are we making an open() call.  (I'd argue that it irrespective of other things, it wants naming xenevtchn_fdopen() for API familiarity.)

Okay.

Quoted 9 lines

However, the root of the problem is actually the ambiguity in the name. These are not flags to the open() system call, but general flags for xenevtchn.

Therefore, I recommend a prep patch which renames open_flags to just flags, and while at it, changes to unsigned int rather than a naked "unsigned" type.  There are no API/ABI implications for this, but it will help massively with code clarity.

Okay.

Quoted 4 lines

I'd also possibly go as far as to say that plumbing 'flags' down into osdep ought to split out into a separate patch.  There is also a wild mix of coding styles even within the hunks here.

Fine with me.

Quoted 5 lines

Additionally, something in core.c should check for unknown flags and reject them them with EINVAL.  It was buggy that this wasn't done before, and really needs to be implemented before we start having cases where people might plausibly pass something other than 0.

Are you sure this is safe? I'm not arguing against it, but we considered to do that and didn't dare to.

Quoted 7 lines

~Andrew

P.S. if you don't fancy doing all of this, my brain could do with a break from the complicated work, and I can see about organising this cleanup.

I'm fine doing it. I'm sure you'll find some other no-brainer to work on :-)

Juergen

#7 lore raw

Re: [PATCH v10 04/25] tools/libxenevtchn: add possibility to not close file descriptor on exec

Quoted 9 lines · Jürgen Groß
Quoted 5 lines · Andrew Cooper

Additionally, something in core.c should check for unknown flags and reject them them with EINVAL.  It was buggy that this wasn't done before, and really needs to be implemented before we start having cases where people might plausibly pass something other than 0.

Are you sure this is safe? I'm not arguing against it, but we considered to do that and didn't dare to.

Well - you're already breaking things by adding meaning to bit 0 where it was previously ignored.

But fundamentally - any caller passing non-zero to begin with is buggy, and it will be less bad to fix up our input validation and given them a clean EINVAL now.

The alternative is no error and some weird side effect when we implement whichever bit they were settings.

~Andrew

#8 lore raw

Re: [PATCH v10 04/25] tools/libxenevtchn: add possibility to not close file descriptor on exec

Quoted 8 lines · Andrew Cooper
Quoted 7 lines · Jürgen Groß
Quoted 5 lines · Andrew Cooper

Additionally, something in core.c should check for unknown flags and reject them them with EINVAL.  It was buggy that this wasn't done before, and really needs to be implemented before we start having cases where people might plausibly pass something other than 0.

Quoted 5 lines
Quoted 2 lines

Are you sure this is safe? I'm not arguing against it, but we considered to do that and didn't dare to.

Well - you're already breaking things by adding meaning to bit 0 where it was previously ignored.

Fair enough.

Quoted 3 lines

But fundamentally - any caller passing non-zero to begin with is buggy, and it will be less bad to fix up our input validation and given them a clean EINVAL now.

Fine with me.

Quoted 2 lines

The alternative is no error and some weird side effect when we implement whichever bit they were settings.

Hmm, yes. I'll add the check.

Juergen

#9 lore raw

PATCH v10 5/25 tools/xenstore: refactor XS_CONTROL handling

In reply to Juergen Gross

In order to allow control commands with binary data refactor handling of XS_CONTROL:

- get primary command first
- add maximum number of additional parameters to pass to command handler

---
V2:
- add comment regarding max_pars (Pawel Wieczorkiewicz)

V3:
- addressed Paul's comments

---
 tools/xenstore/xenstored_control.c | 34 ++++++++++++++++++++----------
 tools/xenstore/xenstored_core.c    |  3 +--
 tools/xenstore/xenstored_core.h    |  1 +
 3 files changed, 25 insertions(+), 13 deletions(-)
tools/xenstore/xenstored_control.c +23 -11
index 8d48ab4820..8d29db8270 100644
@@ -30,6 +30,14 @@ struct cmd_s {
 	char *cmd;
 	int (*func)(void *, struct connection *, char **, int);
 	char *pars;
+	/*
+	 * max_pars can be used to limit the size of the parameter vector,
+	 * e.g. in case of large binary parts in the parameters.
+	 * The command is included in the count, so 1 means just the command
+	 * without any parameter.
+	 * 0 == no limit (the default)
+	 */
+	unsigned int max_pars;
 };
 
 static int do_control_check(void *ctx, struct connection *conn,
@@ -194,25 +202,29 @@ static int do_control_help(void *ctx, struct connection *conn,
 
 int do_control(struct connection *conn, struct buffered_data *in)
 {
-	int num;
-	int cmd;
-	char **vec;
+	unsigned int cmd, num, off;
+	char **vec = NULL;
 
 	if (conn->id != 0)
 		return EACCES;
 
-	num = xs_count_strings(in->buffer, in->used);
-	if (num < 1)
+	off = get_string(in, 0);
+	if (!off)
+		return EINVAL;
+	for (cmd = 0; cmd < ARRAY_SIZE(cmds); cmd++)
+		if (streq(in->buffer, cmds[cmd].cmd))
+			break;
+	if (cmd == ARRAY_SIZE(cmds))
 		return EINVAL;
+
+	num = xs_count_strings(in->buffer, in->used);
+	if (cmds[cmd].max_pars)
+		num = min(num, cmds[cmd].max_pars);
 	vec = talloc_array(in, char *, num);
 	if (!vec)
 		return ENOMEM;
-	if (get_strings(in, vec, num) != num)
+	if (get_strings(in, vec, num) < num)
 		return EIO;
 
-	for (cmd = 0; cmd < ARRAY_SIZE(cmds); cmd++)
-		if (streq(vec[0], cmds[cmd].cmd))
-			return cmds[cmd].func(in, conn, vec + 1, num - 1);
-
-	return EINVAL;
+	return cmds[cmd].func(in, conn, vec + 1, num - 1);
 }
tools/xenstore/xenstored_core.c +1 -2
index 50986f8b29..e1b92c3dc8 100644
@@ -620,8 +620,7 @@ static struct buffered_data *new_buffer(void *ctx)
 /* Return length of string (including nul) at this offset.
  * If there is no nul, returns 0 for failure.
  */
-static unsigned int get_string(const struct buffered_data *data,
-			       unsigned int offset)
+unsigned int get_string(const struct buffered_data *data, unsigned int offset)
 {
 	const char *nul;
 
tools/xenstore/xenstored_core.h +1 -0
index fb59d862a2..27826c125c 100644
@@ -142,6 +142,7 @@ const char *onearg(struct buffered_data *in);
 /* Break input into vectors, return the number, fill in up to num of them. */
 unsigned int get_strings(struct buffered_data *data,
 			 char *vec[], unsigned int num);
+unsigned int get_string(const struct buffered_data *data, unsigned int offset);
 
 void send_reply(struct connection *conn, enum xsd_sockmsg_type type,
 		const void *data, unsigned int len);
Signature · 1 line
2.26.2
#10 lore raw

PATCH v10 6/25 tools/xenstore: add live update command to xenstore-control

In reply to Juergen Gross

Add the "live-update" command to xenstore-control enabling updating xenstored to a new version in a running Xen system.

With -c <arg> it is possible to pass a different command line to the new instance of xenstored. This will replace the command line used for the invocation of the just running xenstored instance.

The running xenstored (or xenstore-stubdom) needs to support live updating, of course.

For now just add a small dummy handler to C xenstore denying any live update action.

---
V2:
- add 0 byte after kernel chunk
- add comment regrading add_to_buf() semantics (Pawel Wieczorkiewicz)
- use %u for unsigned in format (Pawel Wieczorkiewicz)
- explain buffer size better (Pawel Wieczorkiewicz)
- add loop around "-s" option for client side retry in case of timeout

V3:
- add live-update command to docs/misc/xenstore.txt (Paul Durrant)
- fix indent (Paul Durrant)

V4:
- made several parameters const (Julien Grall)
- added more details to xenstore.txt (Julien Grall)

V5:
- set old_binary to NULL initially (Paul Durrant)

V6:
- use strerror(errno) in error message (Julien Grall)

V10:
- make binary specification mandatory (Andrew Cooper)

---
 docs/misc/xenstore.txt             |  21 ++
 tools/xenstore/Makefile            |   3 +-
 tools/xenstore/xenstore_control.c  | 332 +++++++++++++++++++++++++++--
 tools/xenstore/xenstored_control.c |  30 +++
 4 files changed, 369 insertions(+), 17 deletions(-)
docs/misc/xenstore.txt +21 -0
index 2081f20f55..1480742330 100644
@@ -317,6 +317,27 @@ CONTROL			<command>|[<parameters>|]
 	Current commands are:
 	check
 		checks xenstored innards
+	live-update|<params>|+
+		perform a live-update of the Xenstore daemon, only to
+		be used via xenstore-control command.
+		<params> are implementation specific and are used for
+		different steps of the live-update processing. Currently
+		supported <params> are:
+		-f <file>  specify new daemon binary
+		-b <size>  specify size of new stubdom binary
+		-d <chunk-size> <binary-chunk>  transfer chunk of new
+			stubdom binary
+		-c <pars>  specify new command line to use
+		-s [-t <sec>] [-F]  start live update process (-t specifies
+			timeout in seconds to wait for active transactions
+			to finish, default is 60 seconds; -F will force
+			live update to happen even with running transactions
+			after timeout elapsed)
+		-a  abort live update handling
+		All sub-options will return "OK" in case of success or an
+		error string in case of failure. -s can return "BUSY" in case
+		of an active transaction, a retry of -s can be done in that
+		case.
 	log|on
 		turn xenstore logging on
 	log|off
tools/xenstore/Makefile +2 -1
index 9a0f0d012d..ab89e22d3a 100644
@@ -11,6 +11,7 @@ CFLAGS += -include $(XEN_ROOT)/tools/config.h
 CFLAGS += -I./include
 CFLAGS += $(CFLAGS_libxenevtchn)
 CFLAGS += $(CFLAGS_libxenctrl)
+CFLAGS += $(CFLAGS_libxenguest)
 CFLAGS += $(CFLAGS_libxentoolcore)
 CFLAGS += -DXEN_LIB_STORED="\"$(XEN_LIB_STORED)\""
 CFLAGS += -DXEN_RUN_STORED="\"$(XEN_RUN_STORED)\""
@@ -81,7 +82,7 @@ xenstore: xenstore_client.o
 	$(CC) $< $(LDFLAGS) $(LDLIBS_libxenstore) $(LDLIBS_libxentoolcore) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
 
 xenstore-control: xenstore_control.o
-	$(CC) $< $(LDFLAGS) $(LDLIBS_libxenstore) $(LDLIBS_libxentoolcore) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
+	$(CC) $< $(LDFLAGS) $(LDLIBS_libxenstore) $(LDLIBS_libxenctrl) $(LDLIBS_libxenguest) $(LDLIBS_libxentoolcore) $(SOCKET_LIBS) -o $@ $(APPEND_LDFLAGS)
 
 xs_tdb_dump: xs_tdb_dump.o utils.o tdb.o talloc.o
 	$(CC) $^ $(LDFLAGS) -o $@ $(APPEND_LDFLAGS)
tools/xenstore/xenstore_control.c +316 -16

359 lines not on this page · raw message

tools/xenstore/xenstored_control.c +30 -0
index 8d29db8270..00fda5acdb 100644
@@ -149,11 +149,41 @@ static int do_control_print(void *ctx, struct connection *conn,
 	return 0;
 }
 
+static int do_control_lu(void *ctx, struct connection *conn,
+			 char **vec, int num)
+{
+	const char *resp;
+
+	resp = talloc_strdup(ctx, "NYI");
+	send_reply(conn, XS_CONTROL, resp, strlen(resp) + 1);
+	return 0;
+}
+
 static int do_control_help(void *, struct connection *, char **, int);
 
 static struct cmd_s cmds[] = {
 	{ "check", do_control_check, "" },
 	{ "log", do_control_log, "on|off" },
+
+	/*
+	 * The parameters are those of the xenstore-control utility!
+	 * Depending on environment (Mini-OS or daemon) the live-update
+	 * sequence is split into several sub-operations:
+	 * 1. Specification of new binary
+	 *    daemon:  -f <filename>
+	 *    Mini-OS: -b <binary-size>
+	 *             -d <size> <data-bytes> (multiple of those)
+	 * 2. New command-line (optional): -c <cmdline>
+	 * 3. Start of update: -s [-F] [-t <timeout>]
+	 * Any sub-operation needs to respond with the string "OK" in case
+	 * of success, any other response indicates failure.
+	 * A started live-update sequence can be aborted via "-a" (not
+	 * needed in case of failure for the first or last live-update
+	 * sub-operation).
+	 */
+	{ "live-update", do_control_lu,
+		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
+		"    Default timeout is 60 seconds.", 4 },
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
Signature · 1 line
2.26.2
#11 lore raw

Re: [PATCH v10 06/25] tools/xenstore: add live update command to xenstore-control

Quoted 16 lines · Juergen Gross

[...]
+static int live_update_start(struct xs_handle *xsh, bool force, unsigned int to)

tools/xenstore/xenstore_control.c +13 -0
+{
+    int len = 0;
+    char *buf = NULL, *ret;
+    time_t time_start;
+
+    if (asprintf(&ret, "%u", to) < 0)
+        return 1;
+    len = add_to_buf(&buf, "-s", len);
+    len = add_to_buf(&buf, "-t", len);
+    len = add_to_buf(&buf, ret, len);
+    free(ret);
+    if (force)
+        len = add_to_buf(&buf, "-F", len);

tools/xenstore/xenstore_control.c line 52

Quoted 10 lines
tools/xenstore/xenstore_control.c +10 -0
+    if (len < 0)
+        return 1;
+
+    for (time_start = time(NULL); time(NULL) - time_start < to;) {
+        ret = xs_control_command(xsh, "live-update", buf, len);
+        if (!ret)
+            goto err;
+        if (strcmp(ret, "BUSY"))
+            break;
+    }

We've discovered issues with this during testing: when a live-update is not possible (e.g. guest with active transaction held open on purpose) xenstored gets flooded with live-update requests until the timeout is reached.

This is problematic for multiple reasons:
* flooding xenstored reduces its throughput, and makes it use 100% CPU. Depending on the implementation and configuration it may also flood the logs (which isn't fatal, the system still stays alive if you ENOSPC on /var/log, but it makes it difficult to debug issues if a live update gets you to ENOSPC as you may not see actual failures from after the live update).
* flooding xenstored causes the evtchn to overflow and AFAICT neither xenstored or oxenstored is capable of coping with that (oxenstored infinite loops when that happens). IIUC this needs to be fixed in the kernel, so it doesn't return EFBIG in the first place. As a workaround I added a sleep(1) in this loop
* the timeout is hit on both client and server sides, but depending on rounding almost always happens on the client side first which prevents us from displaying a more informative error message from the server. As a workaround I increased the client side timeout by 2s to cope with rounding and give the server a chance to reply. Oxenstored can reply with a list of domains preventing the live-update for example.

My workarounds looked like this:
@@ -42,6 +43,9 @@ static int live_update_start(struct xs_handle *xsh, bool force, unsigned int to)

         len = add_to_buf(&buf, "-F", len);
     if (len < 0)
         return 1;

+ /* +1 for rounding issues
+ * +1 to give oxenstored a chance to timeout and report back first
*/
+ to += 2;

     for (time_start = time(NULL); time(NULL) - time_start < to;) {
         ret = xs_control_command(xsh, "live-update", buf, len);
@@ -49,6 +53,12 @@ static int live_update_start(struct xs_handle *xsh,
bool force, unsigned int to)
             goto err;
         if (strcmp(ret, "BUSY"))
             break;

+ /* TODO: use task ID for commands, avoid busy loop polling
here
+ * oxenstored checks BUSY condition internally on every main
loop iteration anyway.
+ * Avoid flooding xenstored with live-update requests.
+ * The flooding can also cause the evtchn to overflow in
xenstored which makes
+ * xenstored enter an infinite loop */

+        sleep(1);
     }

This also required various heuristics in oxenstored to differentiate between a new live-update command and querying the status of an already completed live-update command, otherwise I almost always ended up doing 2 live-updates in a row (first one queued up, returned BUSY, completed after a while, and then another one from all the repeated live-update requests).

I'd prefer it if there was a more asynchronous protocol available for live-update:
* the live-update on its own queues up the live update request and returns a generation ID. xenstored retries on its own during each of its main loop iterations whether the conditions for live-update are now met
* when a generation ID is specified with the live-update command it acts as a query to return the status. A query for generation ID < current ID return success, and for generation ID = current ID it returns either BUSY, or a specific error if known (e.g. timeout reached and list of domains preventing live update)
* the generation ID gets incremented every time a live update succeeds

This would eliminiate the need for a client-side timeout, since each of these commands would be non-blocking.
It'd also avoid the busy-poll flood.

Obviously any change here has to be backwards compatible with the already deployed xenstored and oxenstored which doesn't know about generation IDs, but you can tell them apart based on the reply: if you get back OK/BUSY/some error it is the old version, if you get back a generation ID it is the new version.

I ran out of time to implement this before the embargo was up, should I go ahead with prototyping a patch for this now, or do you see an alternative way to make the live-update command more robust?

Best regards,
--Edwin

#12 lore raw

Re: [PATCH v10 06/25] tools/xenstore: add live update command to xenstore-control

Quoted 112 lines · Edwin Torok
Quoted 16 lines · Juergen Gross

[...]
+static int live_update_start(struct xs_handle *xsh, bool force, unsigned int to)

tools/xenstore/xenstore_control.c +13 -0
+{
+    int len = 0;
+    char *buf = NULL, *ret;
+    time_t time_start;
+
+    if (asprintf(&ret, "%u", to) < 0)
+        return 1;
+    len = add_to_buf(&buf, "-s", len);
+    len = add_to_buf(&buf, "-t", len);
+    len = add_to_buf(&buf, ret, len);
+    free(ret);
+    if (force)
+        len = add_to_buf(&buf, "-F", len);

tools/xenstore/xenstore_control.c line 52

Quoted 10 lines
tools/xenstore/xenstore_control.c +10 -0
+    if (len < 0)
+        return 1;
+
+    for (time_start = time(NULL); time(NULL) - time_start < to;) {
+        ret = xs_control_command(xsh, "live-update", buf, len);
+        if (!ret)
+            goto err;
+        if (strcmp(ret, "BUSY"))
+            break;
+    }

We've discovered issues with this during testing: when a live-update is not possible (e.g. guest with active transaction held open on purpose) xenstored gets flooded with live-update requests until the timeout is reached.

This is problematic for multiple reasons:
* flooding xenstored reduces its throughput, and makes it use 100% CPU. Depending on the implementation and configuration it may also flood the logs (which isn't fatal, the system still stays alive if you ENOSPC on /var/log, but it makes it difficult to debug issues if a live update gets you to ENOSPC as you may not see actual failures from after the live update).
* flooding xenstored causes the evtchn to overflow and AFAICT neither xenstored or oxenstored is capable of coping with that (oxenstored infinite loops when that happens). IIUC this needs to be fixed in the kernel, so it doesn't return EFBIG in the first place. As a workaround I added a sleep(1) in this loop
* the timeout is hit on both client and server sides, but depending on rounding almost always happens on the client side first which prevents us from displaying a more informative error message from the server. As a workaround I increased the client side timeout by 2s to cope with rounding and give the server a chance to reply. Oxenstored can reply with a list of domains preventing the live-update for example.

My workarounds looked like this:

(quoted diff) +3 -0
@@ -42,6 +43,9 @@ static int live_update_start(struct xs_handle *xsh,
 bool force, unsigned int to)
          len = add_to_buf(&buf, "-F", len);
      if (len < 0)
          return 1;
+    /* +1 for rounding issues
+     * +1 to give oxenstored a chance to timeout and report back first
 */
+    to += 2;
 
      for (time_start = time(NULL); time(NULL) - time_start < to;) {
          ret = xs_control_command(xsh, "live-update", buf, len);
(quoted diff) +1 -0
@@ -49,6 +53,12 @@ static int live_update_start(struct xs_handle *xsh,
 bool force, unsigned int to)
              goto err;
          if (strcmp(ret, "BUSY"))
              break;
+        /* TODO: use task ID for commands, avoid busy loop polling

here
+ * oxenstored checks BUSY condition internally on every main
loop iteration anyway.

(quoted diff) +2 -0
+         * Avoid flooding xenstored with live-update requests.
+         * The flooding can also cause the evtchn to overflow in

xenstored which makes

(quoted diff) +2 -0
+         * xenstored enter an infinite loop */
+        sleep(1);
      }

This also required various heuristics in oxenstored to differentiate between a new live-update command and querying the status of an already completed live-update command, otherwise I almost always ended up doing 2 live-updates in a row (first one queued up, returned BUSY, completed after a while, and then another one from all the repeated live-update requests).

I'd prefer it if there was a more asynchronous protocol available for live-update:
* the live-update on its own queues up the live update request and returns a generation ID. xenstored retries on its own during each of its main loop iterations whether the conditions for live-update are now met
* when a generation ID is specified with the live-update command it acts as a query to return the status. A query for generation ID < current ID return success, and for generation ID = current ID it returns either BUSY, or a specific error if known (e.g. timeout reached and list of domains preventing live update)
* the generation ID gets incremented every time a live update succeeds

This would eliminiate the need for a client-side timeout, since each of these commands would be non-blocking.
It'd also avoid the busy-poll flood.

Obviously any change here has to be backwards compatible with the already deployed xenstored and oxenstored which doesn't know about generation IDs, but you can tell them apart based on the reply: if you get back OK/BUSY/some error it is the old version, if you get back a generation ID it is the new version.

Quoted 3 lines

I ran out of time to implement this before the embargo was up, should I go ahead with prototyping a patch for this now, or do you see an alternative way to make the live-update command more robust?

I think this can be made much easier:

The live-update should be handled completely in the daemon, so returning only with success or failure. Returning BUSY wouldn't occur this way, but either "OK" (after the successful LU) or a failure reason (e.g. list of domains blocking) would be returned.

I'll have a try with this approach.

Juergen

#13 lore raw

Re: [PATCH v10 06/25] tools/xenstore: add live update command to xenstore-control

Quoted 41 lines · Jürgen Groß
Quoted 33 lines · Edwin Torok

[...]

I'd prefer it if there was a more asynchronous protocol available for
live-update:
* the live-update on its own queues up the live update request and returns a generation ID. xenstored retries on its own during each of
its main loop iterations whether the conditions for live-update are now
met
* when a generation ID is specified with the live-update command it acts as a query to return the status. A query for generation ID < current ID return success, and for generation ID = current ID it returns either BUSY, or a specific error if known (e.g. timeout reached
and list of domains preventing live update)
* the generation ID gets incremented every time a live update succeeds

This would eliminiate the need for a client-side timeout, since each of
these commands would be non-blocking.
It'd also avoid the busy-poll flood.

Obviously any change here has to be backwards compatible with the already deployed xenstored and oxenstored which doesn't know about generation IDs, but you can tell them apart based on the reply: if you
get back OK/BUSY/some error it is the old version, if you get back a
generation ID it is the new version.

Quoted 4 lines

I ran out of time to implement this before the embargo was up, should I
go ahead with prototyping a patch for this now, or do you see an alternative way to make the live-update command more robust?

I think this can be made much easier:

Quoted 9 lines

The live-update should be handled completely in the daemon, so returning only with success or failure. Returning BUSY wouldn't occur this way, but either "OK" (after the successful LU) or a failure reason (e.g. list of domains blocking) would be returned.

I'll have a try with this approach.

In oxenstored each request wants an immediate reply, so delaying the reply to after the live-update is not trivial.
Should be possible to do though (e.g. mark the socket such that no further xenstore protocol is processed on it, and "put back" the live update request, to be replied by the new xenstored after the live update completes, which would clear the 'do not process this socket' flag), I'll be curious to see what your approach will look like.

Best regards,
--Edwin

#14 lore raw

PATCH v10 7/25 tools/xenstore: add basic live-update command parsing

In reply to Juergen Gross

Add the basic parts for parsing the live-update control command.

For now only add the parameter evaluation and calling appropriate functions. Those function only print a message for now and return success.

---
V2:
- keep consistent style in lu_arch() (Pawel Wieczorkiewicz)
- fix handling of force flag (Pawel Wieczorkiewicz)
- use xprintf() instead of trace() for better stubdom diag
- add conn parameter to subfunctions

V4:
- make several parameters/variables const (Julien Grall)
- don't reject an option specified multiple times (Julien Grall)
- use syslog() for messages

---
 tools/xenstore/xenstored_control.c | 105 ++++++++++++++++++++++++++++-
 1 file changed, 104 insertions(+), 1 deletion(-)
tools/xenstore/xenstored_control.c +104 -1
index 00fda5acdb..e3f0d34528 100644
@@ -19,7 +19,9 @@
 #include <errno.h>
 #include <stdarg.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
+#include <syslog.h>
 
 #include "utils.h"
 #include "talloc.h"
@@ -149,12 +151,113 @@ static int do_control_print(void *ctx, struct connection *conn,
 	return 0;
 }
 
+static const char *lu_abort(const void *ctx, struct connection *conn)
+{
+	syslog(LOG_INFO, "live-update: abort\n");
+	return NULL;
+}
+
+static const char *lu_cmdline(const void *ctx, struct connection *conn,
+			      const char *cmdline)
+{
+	syslog(LOG_INFO, "live-update: cmdline %s\n", cmdline);
+	return NULL;
+}
+
+#ifdef __MINIOS__
+static const char *lu_binary_alloc(const void *ctx, struct connection *conn,
+				   unsigned long size)
+{
+	syslog(LOG_INFO, "live-update: binary size %lu\n", size);
+	return NULL;
+}
+
+static const char *lu_binary_save(const void *ctx, struct connection *conn,
+				  unsigned int size, const char *data)
+{
+	return NULL;
+}
+
+static const char *lu_arch(const void *ctx, struct connection *conn,
+			   char **vec, int num)
+{
+	if (num == 2 && !strcmp(vec[0], "-b"))
+		return lu_binary_alloc(ctx, conn, atol(vec[1]));
+	if (num > 2 && !strcmp(vec[0], "-d"))
+		return lu_binary_save(ctx, conn, atoi(vec[1]), vec[2]);
+
+	errno = EINVAL;
+	return NULL;
+}
+#else
+static const char *lu_binary(const void *ctx, struct connection *conn,
+			     const char *filename)
+{
+	syslog(LOG_INFO, "live-update: binary %s\n", filename);
+	return NULL;
+}
+
+static const char *lu_arch(const void *ctx, struct connection *conn,
+			   char **vec, int num)
+{
+	if (num == 2 && !strcmp(vec[0], "-f"))
+		return lu_binary(ctx, conn, vec[1]);
+
+	errno = EINVAL;
+	return NULL;
+}
+#endif
+
+static const char *lu_start(const void *ctx, struct connection *conn,
+			    bool force, unsigned int to)
+{
+	syslog(LOG_INFO, "live-update: start, force=%d, to=%u\n", force, to);
+	return NULL;
+}
+
 static int do_control_lu(void *ctx, struct connection *conn,
 			 char **vec, int num)
 {
 	const char *resp;
+	const char *ret = NULL;
+	unsigned int i;
+	bool force = false;
+	unsigned int to = 0;
+
+	if (num < 1)
+		return EINVAL;
+
+	if (!strcmp(vec[0], "-a")) {
+		if (num == 1)
+			ret = lu_abort(ctx, conn);
+		else
+			return EINVAL;
+	} else if (!strcmp(vec[0], "-c")) {
+		if (num == 2)
+			ret = lu_cmdline(ctx, conn, vec[1]);
+		else
+			return EINVAL;
+	} else if (!strcmp(vec[0], "-s")) {
+		for (i = 1; i < num; i++) {
+			if (!strcmp(vec[i], "-F"))
+				force = true;
+			else if (!strcmp(vec[i], "-t") && i < num - 1) {
+				i++;
+				to = atoi(vec[i]);
+			} else
+				return EINVAL;
+		}
+		ret = lu_start(ctx, conn, force, to);
+	} else {
+		errno = 0;
+		ret = lu_arch(ctx, conn, vec, num);
+		if (errno)
+			return errno;
+	}
 
-	resp = talloc_strdup(ctx, "NYI");
+	if (!ret)
+		ret = "OK";
+	resp = talloc_strdup(ctx, ret);
 	send_reply(conn, XS_CONTROL, resp, strlen(resp) + 1);
 	return 0;
 }
Signature · 1 line
2.26.2
#15 lore raw

PATCH v10 8/25 tools/xenstore: introduce live update status block

In reply to Juergen Gross

Live update of Xenstore is done in multiple steps. It needs a status block holding the current state of live update and related data. It is allocated as child of the connection live update was started over in order to abort live update in case the connection is closed.

Allocation of the block is done in lu_binary[_alloc](), freeing in lu_abort() (and for now in lu_start() as long as no real live-update is happening).

Add tests in all live-update command handlers other than lu_abort() and lu_binary[_alloc]() for being started via the same connection as the begin of live-update.

---
V2:
- use talloc_zero() for allocating the status area (Julien Grall)

V4:
- const (Julien Grall)

---
 tools/xenstore/xenstored_control.c | 63 ++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)
tools/xenstore/xenstored_control.c +63 -0
index e3f0d34528..7854b7f46f 100644
@@ -28,6 +28,34 @@
 #include "xenstored_core.h"
 #include "xenstored_control.h"
 
+struct live_update {
+	/* For verification the correct connection is acting. */
+	struct connection *conn;
+};
+
+static struct live_update *lu_status;
+
+static int lu_destroy(void *data)
+{
+	lu_status = NULL;
+
+	return 0;
+}
+
+static const char *lu_begin(struct connection *conn)
+{
+	if (lu_status)
+		return "live-update session already active.";
+
+	lu_status = talloc_zero(conn, struct live_update);
+	if (!lu_status)
+		return "Allocation failure.";
+	lu_status->conn = conn;
+	talloc_set_destructor(lu_status, lu_destroy);
+
+	return NULL;
+}
+
 struct cmd_s {
 	char *cmd;
 	int (*func)(void *, struct connection *, char **, int);
@@ -154,6 +182,13 @@ static int do_control_print(void *ctx, struct connection *conn,
 static const char *lu_abort(const void *ctx, struct connection *conn)
 {
 	syslog(LOG_INFO, "live-update: abort\n");
+
+	if (!lu_status)
+		return "No live-update session active.";
+
+	/* Destructor will do the real abort handling. */
+	talloc_free(lu_status);
+
 	return NULL;
 }
 
@@ -161,6 +196,10 @@ static const char *lu_cmdline(const void *ctx, struct connection *conn,
 			      const char *cmdline)
 {
 	syslog(LOG_INFO, "live-update: cmdline %s\n", cmdline);
+
+	if (!lu_status || lu_status->conn != conn)
+		return "Not in live-update session.";
+
 	return NULL;
 }
 
@@ -168,13 +207,23 @@ static const char *lu_cmdline(const void *ctx, struct connection *conn,
 static const char *lu_binary_alloc(const void *ctx, struct connection *conn,
 				   unsigned long size)
 {
+	const char *ret;
+
 	syslog(LOG_INFO, "live-update: binary size %lu\n", size);
+
+	ret = lu_begin(conn);
+	if (ret)
+		return ret;
+
 	return NULL;
 }
 
 static const char *lu_binary_save(const void *ctx, struct connection *conn,
 				  unsigned int size, const char *data)
 {
+	if (!lu_status || lu_status->conn != conn)
+		return "Not in live-update session.";
+
 	return NULL;
 }
 
@@ -193,7 +242,14 @@ static const char *lu_arch(const void *ctx, struct connection *conn,
 static const char *lu_binary(const void *ctx, struct connection *conn,
 			     const char *filename)
 {
+	const char *ret;
+
 	syslog(LOG_INFO, "live-update: binary %s\n", filename);
+
+	ret = lu_begin(conn);
+	if (ret)
+		return ret;
+
 	return NULL;
 }
 
@@ -212,6 +268,13 @@ static const char *lu_start(const void *ctx, struct connection *conn,
 			    bool force, unsigned int to)
 {
 	syslog(LOG_INFO, "live-update: start, force=%d, to=%u\n", force, to);
+
+	if (!lu_status || lu_status->conn != conn)
+		return "Not in live-update session.";
+
+	/* Will be replaced by real live-update later. */
+	talloc_free(lu_status);
+
 	return NULL;
 }
 
Signature · 1 line
2.26.2
#16 lore raw

PATCH v10 9/25 tools/xenstore: save new binary for live update

In reply to Juergen Gross

Save the new binary name for the daemon case and the new kernel for stubdom in order to support live update of Xenstore..

---
 tools/xenstore/xenstored_control.c | 41 +++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)
tools/xenstore/xenstored_control.c +40 -1
index 7854b7f46f..95ac1a1648 100644
@@ -1,5 +1,5 @@
 /*
-    Interactive commands for Xen Store Daemon.
+Interactive commands for Xen Store Daemon.
     Copyright (C) 2017 Juergen Gross, SUSE Linux GmbH
 
     This program is free software; you can redistribute it and/or modify
@@ -22,6 +22,9 @@
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
 
 #include "utils.h"
 #include "talloc.h"
@@ -31,6 +34,14 @@
 struct live_update {
 	/* For verification the correct connection is acting. */
 	struct connection *conn;
+
+#ifdef __MINIOS__
+	void *kernel;
+	unsigned int kernel_size;
+	unsigned int kernel_off;
+#else
+	char *filename;
+#endif
 };
 
 static struct live_update *lu_status;
@@ -215,6 +226,13 @@ static const char *lu_binary_alloc(const void *ctx, struct connection *conn,
 	if (ret)
 		return ret;
 
+	lu_status->kernel = talloc_size(lu_status, size);
+	if (!lu_status->kernel)
+		return "Allocation failure.";
+
+	lu_status->kernel_size = size;
+	lu_status->kernel_off = 0;
+
 	return NULL;
 }
 
@@ -224,6 +242,12 @@ static const char *lu_binary_save(const void *ctx, struct connection *conn,
 	if (!lu_status || lu_status->conn != conn)
 		return "Not in live-update session.";
 
+	if (lu_status->kernel_off + size > lu_status->kernel_size)
+		return "Too much kernel data.";
+
+	memcpy(lu_status->kernel + lu_status->kernel_off, data, size);
+	lu_status->kernel_off += size;
+
 	return NULL;
 }
 
@@ -243,13 +267,23 @@ static const char *lu_binary(const void *ctx, struct connection *conn,
 			     const char *filename)
 {
 	const char *ret;
+	struct stat statbuf;
 
 	syslog(LOG_INFO, "live-update: binary %s\n", filename);
 
+	if (stat(filename, &statbuf))
+		return "File not accessible.";
+	if (!(statbuf.st_mode & (S_IXOTH | S_IXGRP | S_IXUSR)))
+		return "File not executable.";
+
 	ret = lu_begin(conn);
 	if (ret)
 		return ret;
 
+	lu_status->filename = talloc_strdup(lu_status, filename);
+	if (!lu_status->filename)
+		return "Allocation failure.";
+
 	return NULL;
 }
 
@@ -272,6 +306,11 @@ static const char *lu_start(const void *ctx, struct connection *conn,
 	if (!lu_status || lu_status->conn != conn)
 		return "Not in live-update session.";
 
+#ifdef __MINIOS__
+	if (lu_status->kernel_size != lu_status->kernel_off)
+		return "Kernel not complete.";
+#endif
+
 	/* Will be replaced by real live-update later. */
 	talloc_free(lu_status);
 
Signature · 1 line
2.26.2
#17 lore raw

PATCH v10 10/25 tools/xenstore: add command line handling for live update

In reply to Juergen Gross

Updating an instance of Xenstore via live update needs to hand over the command line parameters to the updated instance. Those can be either the parameters used by the updated instance or new ones when supplied when starting the live update.

So when supplied store the new command line parameters in lu_status.

As it is related add a new option -U (or --live-update") to the command line of xenstored which will be added when starting the new instance. This enables to perform slightly different initializations when started as a result of live update.

---
 tools/xenstore/xenstored_control.c | 6 ++++++
 tools/xenstore/xenstored_core.c    | 7 ++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)
tools/xenstore/xenstored_control.c +6 -0
index 95ac1a1648..2e0827b9ef 100644
@@ -42,6 +42,8 @@ struct live_update {
 #else
 	char *filename;
 #endif
+
+	char *cmdline;
 };
 
 static struct live_update *lu_status;
@@ -211,6 +213,10 @@ static const char *lu_cmdline(const void *ctx, struct connection *conn,
 	if (!lu_status || lu_status->conn != conn)
 		return "Not in live-update session.";
 
+	lu_status->cmdline = talloc_strdup(lu_status, cmdline);
+	if (!lu_status->cmdline)
+		return "Allocation failure.";
+
 	return NULL;
 }
 
tools/xenstore/xenstored_core.c +6 -1
index e1b92c3dc8..0dddf24327 100644
@@ -1969,6 +1969,7 @@ static struct option options[] = {
 	{ "internal-db", 0, NULL, 'I' },
 	{ "verbose", 0, NULL, 'V' },
 	{ "watch-nb", 1, NULL, 'W' },
+	{ "live-update", 0, NULL, 'U' },
 	{ NULL, 0, NULL, 0 } };
 
 extern void dump_conn(struct connection *conn); 
@@ -1983,11 +1984,12 @@ int main(int argc, char *argv[])
 	bool dofork = true;
 	bool outputpid = false;
 	bool no_domain_init = false;
+	bool live_update = false;
 	const char *pidfile = NULL;
 	int timeout;
 
 
-	while ((opt = getopt_long(argc, argv, "DE:F:HNPS:t:A:M:T:RVW:", options,
+	while ((opt = getopt_long(argc, argv, "DE:F:HNPS:t:A:M:T:RVW:U", options,
 				  NULL)) != -1) {
 		switch (opt) {
 		case 'D':
@@ -2045,6 +2047,9 @@ int main(int argc, char *argv[])
 		case 'p':
 			priv_domid = strtol(optarg, NULL, 10);
 			break;
+		case 'U':
+			live_update = true;
+			break;
 		}
 	}
 	if (optind != argc)
Signature · 1 line
2.26.2
#18 lore raw

PATCH v10 11/25 tools/xenstore: add the basic framework for doing the live update

In reply to Juergen Gross

Add the main framework for executing the live update. This for now only defines the basic execution steps with empty dummy functions. This final step returning means failure, as in case of success the new executable will have taken over.

---
V4:
- const (Julien Grall)

---
 tools/xenstore/xenstored_control.c | 39 ++++++++++++++++++++++++++++--
 1 file changed, 37 insertions(+), 2 deletions(-)
tools/xenstore/xenstored_control.c +37 -2
index 2e0827b9ef..940b717741 100644
@@ -304,9 +304,27 @@ static const char *lu_arch(const void *ctx, struct connection *conn,
 }
 #endif
 
+static const char *lu_check_lu_allowed(const void *ctx, bool force,
+				       unsigned int to)
+{
+	return NULL;
+}
+
+static const char *lu_dump_state(const void *ctx, struct connection *conn)
+{
+	return NULL;
+}
+
+static const char *lu_activate_binary(const void *ctx)
+{
+	return "Not yet implemented.";
+}
+
 static const char *lu_start(const void *ctx, struct connection *conn,
 			    bool force, unsigned int to)
 {
+	const char *ret;
+
 	syslog(LOG_INFO, "live-update: start, force=%d, to=%u\n", force, to);
 
 	if (!lu_status || lu_status->conn != conn)
@@ -317,10 +335,27 @@ static const char *lu_start(const void *ctx, struct connection *conn,
 		return "Kernel not complete.";
 #endif
 
-	/* Will be replaced by real live-update later. */
+	/* Check for state to allow live update. */
+	ret = lu_check_lu_allowed(ctx, force, to);
+	if (ret) {
+		if (!strcmp(ret, "BUSY"))
+			return ret;
+		goto out;
+	}
+
+	/* Dump out internal state, including "OK" for live update. */
+	ret = lu_dump_state(ctx, conn);
+	if (ret)
+		goto out;
+
+	/* Perform the activation of new binary. */
+	ret = lu_activate_binary(ctx);
+	/* We will reach this point only in case of failure. */
+
+ out:
 	talloc_free(lu_status);
 
-	return NULL;
+	return ret;
 }
 
 static int do_control_lu(void *ctx, struct connection *conn,
Signature · 1 line
2.26.2
#19 lore raw

PATCH v10 12/25 tools/xenstore: allow live update only with no transaction active

In reply to Juergen Gross

In order to simplify live update state dumping only allow live update to happen when no transaction is active.

A timeout is used to detect guests which have a transaction active for longer periods of time. In case such a guest is detected when trying to do a live update it will be reported and the update will fail.

The admin can then either use a longer timeout, or use the force flag to just ignore the transactions of such a guest, or kill the guest before retrying.

Transactions that have been active for a shorter time than the timeout will end in the live update starting to respond "BUSY" without aborting the complete live update process. The xenstore-control program will then just repeat the live update starting until a different result is returned.

---
 tools/xenstore/xenstored_control.c     | 20 +++++++++++++++++++-
 tools/xenstore/xenstored_core.h        |  1 +
 tools/xenstore/xenstored_transaction.c |  5 +++++
 3 files changed, 25 insertions(+), 1 deletion(-)
tools/xenstore/xenstored_control.c +19 -1
index 940b717741..af64a9a2d4 100644
@@ -22,6 +22,7 @@ Interactive commands for Xen Store Daemon.
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
+#include <time.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <unistd.h>
@@ -307,7 +308,24 @@ static const char *lu_arch(const void *ctx, struct connection *conn,
 static const char *lu_check_lu_allowed(const void *ctx, bool force,
 				       unsigned int to)
 {
-	return NULL;
+	char *ret = NULL;
+	struct connection *conn;
+	time_t now = time(NULL);
+	bool busy = false;
+
+	list_for_each_entry(conn, &connections, list) {
+		if (conn->ta_start_time - now >= to && !force) {
+			ret = talloc_asprintf(ctx, "%s\nDomain %u: %ld s",
+					      ret ? : "Domains with long running transactions:",
+					      conn->id,
+					      conn->ta_start_time - now);
+			if (!ret)
+				busy = true;
+		} else if (conn->ta_start_time)
+			busy = true;
+	}
+
+	return ret ? (const char *)ret : (busy ? "BUSY" : NULL);
 }
 
 static const char *lu_dump_state(const void *ctx, struct connection *conn)
tools/xenstore/xenstored_core.h +1 -0
index 27826c125c..a009b182fd 100644
@@ -93,6 +93,7 @@ struct connection
 	struct list_head transaction_list;
 	uint32_t next_transaction_id;
 	unsigned int transaction_started;
+	time_t ta_start_time;
 
 	/* The domain I'm associated with, if any. */
 	struct domain *domain;
tools/xenstore/xenstored_transaction.c +5 -0
index 52355f4ed8..cd07fb0f21 100644
@@ -473,6 +473,8 @@ int do_transaction_start(struct connection *conn, struct buffered_data *in)
 	list_add_tail(&trans->list, &conn->transaction_list);
 	talloc_steal(conn, trans);
 	talloc_set_destructor(trans, destroy_transaction);
+	if (!conn->transaction_started)
+		conn->ta_start_time = time(NULL);
 	conn->transaction_started++;
 	wrl_ntransactions++;
 
@@ -511,6 +513,8 @@ int do_transaction_end(struct connection *conn, struct buffered_data *in)
 	conn->transaction = NULL;
 	list_del(&trans->list);
 	conn->transaction_started--;
+	if (!conn->transaction_started)
+		conn->ta_start_time = 0;
 
 	/* Attach transaction to in for auto-cleanup */
 	talloc_steal(in, trans);
@@ -589,6 +593,7 @@ void conn_delete_all_transactions(struct connection *conn)
 	assert(conn->transaction == NULL);
 
 	conn->transaction_started = 0;
+	conn->ta_start_time = 0;
 }
 
 int check_transactions(struct hashtable *hash)
Signature · 1 line
2.26.2
#20 lore raw

PATCH v10 13/25 docs: update the xenstore migration stream documentation

In reply to Juergen Gross

For live update of Xenstore some records defined in the migration stream document need to be changed:

- Support of the read-only socket has been dropped from all Xenstore implementations, so ro-socket-fd in the global record can be removed.

- Some guests require the event channel to Xenstore to remain the same on Xenstore side, so Xenstore has to keep the event channel interface open across a live update. For this purpose an evtchn-fd needs to be added to the global record.

- With no read-only support the flags field in the connection record can be dropped.

- The evtchn field in the connection record needs to be switched to hold the port of the Xenstore side of the event channel.

- A flags field needs to be added to permission specifiers in order to be able to mark a permission as stale (XSA-322).

---
V8:
- split off from following patch (Julien Grall)

---
 docs/designs/xenstore-migration.md | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
docs/designs/xenstore-migration.md +10 -9
index 2ce2c836f5..1a5b94b31d 100644
@@ -116,7 +116,7 @@ xenstored state that needs to be restored.
 +-------+-------+-------+-------+
 | rw-socket-fd                  |
 +-------------------------------+
-| ro-socket-fd                  |
+| evtchn-fd                     |
 +-------------------------------+
 ```
 
@@ -126,8 +126,8 @@ xenstored state that needs to be restored.
 | `rw-socket-fd` | The file descriptor of the socket accepting  |
 |                | read-write connections                       |
 |                |                                              |
-| `ro-socket-fd` | The file descriptor of the socket accepting  |
-|                | read-only connections                        |
+| `evtchn-fd`    | The file descriptor used to communicate with |
+|                | the event channel driver                     |
 
 xenstored will resume in the original process context. Hence `rw-socket-fd` and
 `ro-socket-fd` simply specify the file descriptors of the sockets. Sockets
@@ -147,7 +147,7 @@ the domain being migrated.
 ```
     0       1       2       3       4       5       6       7    octet
 +-------+-------+-------+-------+-------+-------+-------+-------+
-| conn-id                       | conn-type     | flags         |
+| conn-id                       | conn-type     |               |
 +-------------------------------+---------------+---------------+
 | conn-spec
 ...
@@ -169,9 +169,6 @@ the domain being migrated.
 |                | 0x0001: socket                               |
 |                | 0x0002 - 0xFFFF: reserved for future use     |
 |                |                                              |
-| `flags`        | A bit-wise OR of:                            |
-|                | 0001: read-only                              |
-|                |                                              |
 | `conn-spec`    | See below                                    |
 |                |                                              |
 | `in-data-len`  | The length (in octets) of any data read      |
@@ -216,7 +213,7 @@ For `shared ring` connections it is as follows:
 |           | operation [2] or DOMID_INVALID [3] otherwise      |
 |           |                                                   |
 | `evtchn`  | The port number of the interdomain channel used   |
-|           | by `domid` to communicate with xenstored          |
+|           | by xenstored to communicate with `domid`          |
 |           |                                                   |
 
 Since the ABI guarantees that entry 1 in `domid`'s grant table will always
@@ -386,7 +383,7 @@ A node permission specifier has the following format:
 ```
     0       1       2       3    octet
 +-------+-------+-------+-------+
-| perm  | pad   | domid         |
+| perm  | flags | domid         |
 +-------+-------+---------------+
 ```
 
@@ -395,6 +392,10 @@ A node permission specifier has the following format:
 | `perm`  | One of the ASCII values `w`, `r`, `b` or `n` as     |
 |         | specified for the `SET_PERMS` operation [2]         |
 |         |                                                     |
+| `flags` | A bit-wise OR of:                                   |
+|         | 0x01: stale permission, ignore when checking        |
+|         |       permissions                                   |
+|         |                                                     |
 | `domid` | The domain-id to which the permission relates       |
 
 Note that perm1 defines the domain owning the code. See [4] for more
Signature · 1 line
2.26.2
#21 lore raw

PATCH v10 14/25 tools/xenstore: add include file for state structure definitions

In reply to Juergen Gross

Add an include file containing all structures and defines needed for dumping and restoring the internal Xenstore state.

---
V4:
- drop mfn from connection record (rebase to V5 of state doc patch)
- add #ifdef __MINIOS__ (Julien Grall)
- correct comments (Julien Grall)
- add data_in_len

V5:
- add data_resp_len

V6:
- add flag byte to node permissions (Julien Grall)
- update migration document

V7:
- add evtchn_fd

V8:
- remove ro-socket and read-only connection flag
- split off documentation part

V9:
- add htobe32() macro if needed (Wei Liu)

---
 tools/xenstore/include/xenstore_state.h | 131 ++++++++++++++++++++++++
 1 file changed, 131 insertions(+)
 create mode 100644 tools/xenstore/include/xenstore_state.h
tools/xenstore/include/xenstore_state.h new file +131 -0
new file mode 100644
index 0000000000..d2a9307400
@@ -0,0 +1,131 @@
+/*
+ * Xenstore internal state dump definitions.
+ * Copyright (C) Juergen Gross, SUSE Software Solutions Germany GmbH
+ *
+ * Used for live-update and migration, possibly across Xenstore implementations.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef XENSTORE_STATE_H
+#define XENSTORE_STATE_H
+
+#include <endian.h>
+#include <sys/types.h>
+
+#ifndef htobe32
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define htobe32(x) __builtin_bswap32(x)
+#else
+#define htobe32(x) (x)
+#endif
+#endif
+
+struct xs_state_preamble {
+    char ident[8];
+#define XS_STATE_IDENT    "xenstore"  /* To be used without the NUL byte. */
+    uint32_t version;                 /* Version in big endian format. */
+#define XS_STATE_VERSION  0x00000001
+    uint32_t flags;                   /* Endianess. */
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define XS_STATE_FLAGS    0x00000000  /* Little endian. */
+#else
+#define XS_STATE_FLAGS    0x00000001  /* Big endian. */
+#endif
+};
+
+/*
+ * Each record is starting with xs_state_record_header.
+ * All records have a length of a multiple of 8 bytes.
+ */
+
+/* Common record layout: */
+struct xs_state_record_header {
+    uint32_t type;
+#define XS_STATE_TYPE_END        0x00000000
+#define XS_STATE_TYPE_GLOBAL     0x00000001
+#define XS_STATE_TYPE_CONN       0x00000002
+#define XS_STATE_TYPE_WATCH      0x00000003
+#define XS_STATE_TYPE_TA         0x00000004
+#define XS_STATE_TYPE_NODE       0x00000005
+    uint32_t length;         /* Length of record in bytes. */
+};
+
+/* Global state of Xenstore: */
+struct xs_state_global {
+    int32_t socket_fd;      /* File descriptor for socket connections or -1. */
+    int32_t evtchn_fd;      /* File descriptor for event channel operations. */
+};
+
+/* Connection to Xenstore: */
+struct xs_state_connection {
+    uint32_t conn_id;       /* Used as reference in watch and TA records. */
+    uint16_t conn_type;
+#define XS_STATE_CONN_TYPE_RING   0
+#define XS_STATE_CONN_TYPE_SOCKET 1
+    uint16_t pad;
+    union {
+        struct {
+            uint16_t domid;  /* Domain-Id. */
+            uint16_t tdomid; /* Id of target domain or DOMID_INVALID. */
+            uint32_t evtchn; /* Event channel port. */
+        } ring;
+        int32_t socket_fd;   /* File descriptor for socket connections. */
+    } spec;
+    uint16_t data_in_len;    /* Number of unprocessed bytes read from conn. */
+    uint16_t data_resp_len;  /* Size of partial response pending for conn. */
+    uint32_t data_out_len;   /* Number of bytes not yet written to conn. */
+    uint8_t  data[];         /* Pending data (read, written) + 0-7 pad bytes. */
+};
+
+/* Watch: */
+struct xs_state_watch {
+    uint32_t conn_id;       /* Connection this watch is associated with. */
+    uint16_t path_length;   /* Number of bytes of path watched (incl. 0). */
+    uint16_t token_length;  /* Number of bytes of watch token (incl. 0). */
+    uint8_t data[];         /* Path bytes, token bytes, 0-7 pad bytes. */
+};
+
+/* Transaction: */
+struct xs_state_transaction {
+    uint32_t conn_id;       /* Connection this TA is associated with. */
+    uint32_t ta_id;         /* Transaction Id. */
+};
+
+/* Node (either XS_STATE_TYPE_NODE or XS_STATE_TYPE_TANODE[_MOD]): */
+struct xs_state_node_perm {
+    uint8_t access;         /* Access rights. */
+#define XS_STATE_NODE_PERM_NONE   'n'
+#define XS_STATE_NODE_PERM_READ   'r'
+#define XS_STATE_NODE_PERM_WRITE  'w'
+#define XS_STATE_NODE_PERM_BOTH   'b'
+    uint8_t flags;
+#define XS_STATE_NODE_PERM_IGNORE 0x01 /* Stale permission, ignore for check. */
+    uint16_t domid;         /* Domain-Id. */
+};
+struct xs_state_node {
+    uint32_t conn_id;       /* Connection in case of transaction or 0. */
+    uint32_t ta_id;         /* Transaction Id or 0. */
+    uint16_t path_len;      /* Length of path string including NUL byte. */
+    uint16_t data_len;      /* Length of node data. */
+    uint16_t ta_access;
+#define XS_STATE_NODE_TA_DELETED  0x0000
+#define XS_STATE_NODE_TA_READ     0x0001
+#define XS_STATE_NODE_TA_WRITTEN  0x0002
+    uint16_t perm_n;        /* Number of permissions (0 in TA: node deleted). */
+    /* Permissions (first is owner, has full access). */
+    struct xs_state_node_perm perms[];
+    /* Path and data follows, plus 0-7 pad bytes. */
+};
+#endif /* XENSTORE_STATE_H */
Signature · 1 line
2.26.2
#22 lore raw

PATCH v10 15/25 tools/xenstore: dump the xenstore state for live update

In reply to Juergen Gross

Dump the complete Xenstore status to a file (daemon case) or memory (stubdom case).

As we don't know the exact length of the needed area in advance we are using an anonymous rather large mapping in stubdom case, which will use only virtual address space until accessed. And as we are writing the area in a sequential manner this is fine. As the initial size we are choosing the double size of the memory allocated via talloc(), which should be more than enough.

---
V4:
- directly write to state file in daemon case
- add pending read data (Julien Grall)

V5:
- fix 0 length buffered data body (Julien Grall)
- simplify dump_state_buffered_data() (Paul Durrant)
- add data_resp_len field handling
- add comments (Paul Durrant)
- move dump_state_align() call out of dump_state_buffered_data() (Paul Durrant)
- move constant assignments out of loops (Paul Durrant)
- use set_tdb_key() (Paul Durrant)

V6:
- rename "first" to "Partial" (Paul Durrant)
- make sure data_resp_len is written (Paul Durrant)
- don't leak node memory in dump_state_nodes()
- add permission flag byte handling (Julien Grall)
- drop global state buffer (Julien Grall)
- add and correct comments (Julien Grall)
- add const (Julien Grall)
- add path buffer overrun check (Julien Grall)
- move get_watch_path() from later patch, correct dump_state_watches() to use it (Julien Grall)

V7:
- add glb.evtchn_fd, switch evtchn to local port
- remove definition of ROUNDUP() from utils.h due to rebase
- create state file with 0600 permissions

---
 tools/xenstore/utils.c             |  17 +++
 tools/xenstore/utils.h             |   6 +
 tools/xenstore/xenstored_control.c | 102 +++++++++++++-
 tools/xenstore/xenstored_core.c    | 213 +++++++++++++++++++++++++++++
 tools/xenstore/xenstored_core.h    |  12 ++
 tools/xenstore/xenstored_domain.c  | 105 ++++++++++++++
 tools/xenstore/xenstored_domain.h  |   3 +
 tools/xenstore/xenstored_watch.c   |  57 +++++++-
 tools/xenstore/xenstored_watch.h   |   3 +
 9 files changed, 512 insertions(+), 6 deletions(-)
tools/xenstore/utils.c +17 -0
index 633ce3b4fc..0d80cb6de8 100644
@@ -62,3 +62,20 @@ void barf_perror(const char *fmt, ...)
 	}
 	exit(1);
 }
+
+const char *dump_state_align(FILE *fp)
+{
+	long len;
+	static char nul[8] = {};
+
+	len = ftell(fp);
+	if (len < 0)
+		return "Dump state align error";
+	len &= 7;
+	if (!len)
+		return NULL;
+
+	if (fwrite(nul, 8 - len, 1, fp) != 1)
+		return "Dump state align error";
+	return NULL;
+}
tools/xenstore/utils.h +6 -0
index 6a1b5de9bd..df1cb9a3ba 100644
@@ -3,6 +3,7 @@
 #include <stdbool.h>
 #include <string.h>
 #include <stdint.h>
+#include <stdio.h>
 
 #include <xen-tools/libs.h>
 
@@ -21,6 +22,11 @@ static inline bool strends(const char *a, const char *b)
 	return streq(a + strlen(a) - strlen(b), b);
 }
 
+/*
+ * Write NUL bytes for aligning state data to 8 bytes.
+ */
+const char *dump_state_align(FILE *fp);
+
 void barf(const char *fmt, ...) __attribute__((noreturn));
 void barf_perror(const char *fmt, ...) __attribute__((noreturn));
 
tools/xenstore/xenstored_control.c +101 -1

144 lines not on this page · raw message

tools/xenstore/xenstored_core.c +213 -0

219 lines not on this page · raw message

tools/xenstore/xenstored_core.h +12 -0
index a009b182fd..32b306e161 100644
@@ -30,6 +30,7 @@
 #include <errno.h>
 
 #include "xenstore_lib.h"
+#include "xenstore_state.h"
 #include "list.h"
 #include "tdb.h"
 #include "hashtable.h"
@@ -41,6 +42,8 @@ typedef int32_t wrl_creditt;
 #define WRL_CREDIT_MAX (1000*1000*1000)
 /* ^ satisfies non-overflow condition for wrl_xfer_credit */
 
+struct xs_state_connection;
+
 struct buffered_data
 {
 	struct list_head list;
@@ -224,6 +227,15 @@ int remember_string(struct hashtable *hash, const char *str);
 
 void set_tdb_key(const char *name, TDB_DATA *key);
 
+const char *dump_state_global(FILE *fp);
+const char *dump_state_buffered_data(FILE *fp, const struct connection *c,
+				     const struct connection *conn,
+				     struct xs_state_connection *sc);
+const char *dump_state_nodes(FILE *fp, const void *ctx);
+const char *dump_state_node_perms(FILE *fp, struct xs_state_node *sn,
+				  const struct xs_permissions *perms,
+				  unsigned int n_perms);
+
 #endif /* _XENSTORED_CORE_H */
 
 /*
tools/xenstore/xenstored_domain.c +105 -0

111 lines not on this page · raw message

tools/xenstore/xenstored_domain.h +3 -0
index 66e0a12654..413b974375 100644
@@ -97,4 +97,7 @@ void wrl_log_periodic(struct wrl_timestampt now);
 void wrl_apply_debit_direct(struct connection *conn);
 void wrl_apply_debit_trans_commit(struct connection *conn);
 
+const char *dump_state_connections(FILE *fp, struct connection *conn);
+const char *dump_state_special_nodes(FILE *fp);
+
 #endif /* _XENSTORED_DOMAIN_H */
tools/xenstore/xenstored_watch.c +52 -5

75 lines not on this page · raw message

tools/xenstore/xenstored_watch.h +3 -0
index 03094374f3..3d81645f45 100644
@@ -30,4 +30,7 @@ void fire_watches(struct connection *conn, const void *tmp, const char *name,
 
 void conn_delete_all_watches(struct connection *conn);
 
+const char *dump_state_watches(FILE *fp, struct connection *conn,
+			       unsigned int conn_id);
+
 #endif /* _XENSTORED_WATCH_H */
Signature · 1 line
2.26.2
#23 lore raw

PATCH v10 16/25 tools/xenstore: handle CLOEXEC flag for local files and pipes

In reply to Juergen Gross

For support of live update the locally used files need to have the "close on exec" flag set. Fortunately the used Xen libraries are already doing this, so only the logging and tdb related files and pipes are affected. openlog() has the close on exec attribute, too.

In order to be able to keep the event channels open specify the XENEVTCHN_NO_CLOEXEC flag when calling xenevtchn_open().

---
V4:
- disable LU in case of O_CLOEXEC not supported (Julien Grall)

V5:
- add comment (Paul Durrant)

V7:
- set XENEVTCHN_NO_CLOEXEC

---
 tools/xenstore/xenstored_control.c |  6 ++++++
 tools/xenstore/xenstored_core.c    |  6 ++++--
 tools/xenstore/xenstored_core.h    |  8 ++++++++
 tools/xenstore/xenstored_domain.c  |  2 +-
 tools/xenstore/xenstored_posix.c   | 12 ++++++++++++
 5 files changed, 31 insertions(+), 3 deletions(-)
tools/xenstore/xenstored_control.c +6 -0
index 38550a559e..437276de8d 100644
@@ -41,6 +41,7 @@ Interactive commands for Xen Store Daemon.
 #define MAP_ANONYMOUS MAP_ANON
 #endif
 
+#ifndef NO_LIVE_UPDATE
 struct live_update {
 	/* For verification the correct connection is acting. */
 	struct connection *conn;
@@ -85,6 +86,7 @@ static const char *lu_begin(struct connection *conn)
 
 	return NULL;
 }
+#endif
 
 struct cmd_s {
 	char *cmd;
@@ -209,6 +211,7 @@ static int do_control_print(void *ctx, struct connection *conn,
 	return 0;
 }
 
+#ifndef NO_LIVE_UPDATE
 static const char *lu_abort(const void *ctx, struct connection *conn)
 {
 	syslog(LOG_INFO, "live-update: abort\n");
@@ -522,6 +525,7 @@ static int do_control_lu(void *ctx, struct connection *conn,
 	send_reply(conn, XS_CONTROL, resp, strlen(resp) + 1);
 	return 0;
 }
+#endif
 
 static int do_control_help(void *, struct connection *, char **, int);
 
@@ -529,6 +533,7 @@ static struct cmd_s cmds[] = {
 	{ "check", do_control_check, "" },
 	{ "log", do_control_log, "on|off" },
 
+#ifndef NO_LIVE_UPDATE
 	/*
 	 * The parameters are those of the xenstore-control utility!
 	 * Depending on environment (Mini-OS or daemon) the live-update
@@ -548,6 +553,7 @@ static struct cmd_s cmds[] = {
 	{ "live-update", do_control_lu,
 		"[-c <cmdline>] [-F] [-t <timeout>] <file>\n"
 		"    Default timeout is 60 seconds.", 4 },
+#endif
 #ifdef __MINIOS__
 	{ "memreport", do_control_memreport, "" },
 #else
tools/xenstore/xenstored_core.c +4 -2
index 064109c393..0f4e10815a 100644
@@ -197,7 +197,8 @@ void reopen_log(void)
 	if (tracefile) {
 		close_log();
 
-		tracefd = open(tracefile, O_WRONLY|O_CREAT|O_APPEND, 0600);
+		tracefd = open(tracefile,
+			       O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0600);
 
 		if (tracefd < 0)
 			perror("Could not open tracefile");
@@ -1646,7 +1647,8 @@ static void setup_structure(void)
 	if (!(tdb_flags & TDB_INTERNAL))
 		unlink(tdbname);
 
-	tdb_ctx = tdb_open_ex(tdbname, 7919, tdb_flags, O_RDWR|O_CREAT|O_EXCL,
+	tdb_ctx = tdb_open_ex(tdbname, 7919, tdb_flags,
+			      O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC,
 			      0640, &tdb_logger, NULL);
 	if (!tdb_ctx)
 		barf_perror("Could not create tdb file %s", tdbname);
tools/xenstore/xenstored_core.h +8 -0
index 32b306e161..e40e0e6806 100644
@@ -35,6 +35,14 @@
 #include "tdb.h"
 #include "hashtable.h"
 
+#ifndef O_CLOEXEC
+#define O_CLOEXEC 0
+/* O_CLOEXEC support is needed for Live Update in the daemon case. */
+#ifndef __MINIOS__
+#define NO_LIVE_UPDATE
+#endif
+#endif
+
 /* DEFAULT_BUFFER_SIZE should be large enough for each errno string. */
 #define DEFAULT_BUFFER_SIZE 16
 
tools/xenstore/xenstored_domain.c +1 -1
index 919a4d98cf..38d250fbed 100644
@@ -743,7 +743,7 @@ void domain_init(void)
 
 	talloc_set_destructor(xgt_handle, close_xgt_handle);
 
-	xce_handle = xenevtchn_open(NULL, 0);
+	xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
 
 	if (xce_handle == NULL)
 		barf_perror("Failed to open evtchn device");
tools/xenstore/xenstored_posix.c +12 -0
index 1f9603fea2..ae3e63e07f 100644
@@ -90,9 +90,21 @@ void finish_daemonize(void)
 
 void init_pipe(int reopen_log_pipe[2])
 {
+	int flags;
+	unsigned int i;
+
 	if (pipe(reopen_log_pipe)) {
 		barf_perror("pipe");
 	}
+
+	for (i = 0; i < 2; i++) {
+		flags = fcntl(reopen_log_pipe[i], F_GETFD);
+		if (flags < 0)
+			barf_perror("pipe get flags");
+		flags |= FD_CLOEXEC;
+		if (fcntl(reopen_log_pipe[i],  F_SETFD, flags) < 0)
+			barf_perror("pipe set flags");
+	}
 }
 
 void unmap_xenbus(void *interface)
Signature · 1 line
2.26.2
#24 lore raw

PATCH v10 17/25 tools/xenstore: split off domain introduction from do_introduce()

In reply to Juergen Gross

For live update the functionality to introduce a new domain similar to the XS_INTRODUCE command is needed, so split that functionality off into a dedicated function introduce_domain().

Switch initial dom0 initialization to use this function, too.

---
V8:
- new patch

---
 tools/xenstore/xenstored_domain.c | 95 ++++++++++++++++++-------------
 1 file changed, 55 insertions(+), 40 deletions(-)
tools/xenstore/xenstored_domain.c +55 -40
index 38d250fbed..71b078caf3 100644
@@ -326,7 +326,7 @@ static struct domain *find_domain_struct(unsigned int domid)
 	return NULL;
 }
 
-static struct domain *alloc_domain(void *context, unsigned int domid)
+static struct domain *alloc_domain(const void *context, unsigned int domid)
 {
 	struct domain *domain;
 
@@ -347,6 +347,14 @@ static struct domain *alloc_domain(void *context, unsigned int domid)
 	return domain;
 }
 
+static struct domain *find_or_alloc_domain(const void *ctx, unsigned int domid)
+{
+	struct domain *domain;
+
+	domain = find_domain_struct(domid);
+	return domain ? : alloc_domain(ctx, domid);
+}
+
 static int new_domain(struct domain *domain, int port)
 {
 	int rc;
@@ -413,52 +421,41 @@ static void domain_conn_reset(struct domain *domain)
 	domain->interface->rsp_cons = domain->interface->rsp_prod = 0;
 }
 
-/* domid, gfn, evtchn, path */
-int do_introduce(struct connection *conn, struct buffered_data *in)
+static struct domain *introduce_domain(const void *ctx,
+				       unsigned int domid,
+				       evtchn_port_t port)
 {
 	struct domain *domain;
-	char *vec[3];
-	unsigned int domid;
-	evtchn_port_t port;
 	int rc;
 	struct xenstore_domain_interface *interface;
+	bool is_master_domain = (domid == xenbus_master_domid());
 
-	if (get_strings(in, vec, ARRAY_SIZE(vec)) < ARRAY_SIZE(vec))
-		return EINVAL;
-
-	domid = atoi(vec[0]);
-	/* Ignore the gfn, we don't need it. */
-	port = atoi(vec[2]);
-
-	/* Sanity check args. */
-	if (port <= 0)
-		return EINVAL;
-
-	domain = find_domain_struct(domid);
-
-	if (domain == NULL) {
-		/* Hang domain off "in" until we're finished. */
-		domain = alloc_domain(in, domid);
-		if (domain == NULL)
-			return ENOMEM;
-	}
+	domain = find_or_alloc_domain(ctx, domid);
+	if (!domain)
+		return NULL;
 
 	if (!domain->introduced) {
-		interface = map_interface(domid);
+		interface = is_master_domain ? xenbus_map()
+					     : map_interface(domid);
 		if (!interface)
-			return errno;
-		/* Hang domain off "in" until we're finished. */
+			return NULL;
 		if (new_domain(domain, port)) {
 			rc = errno;
-			unmap_interface(interface);
-			return rc;
+			if (is_master_domain)
+				unmap_xenbus(interface);
+			else
+				unmap_interface(interface);
+			errno = rc;
+			return NULL;
 		}
 		domain->interface = interface;
 
 		/* Now domain belongs to its connection. */
 		talloc_steal(domain->conn, domain);
 
-		fire_watches(NULL, in, "@introduceDomain", NULL, false, NULL);
+		if (!is_master_domain)
+			fire_watches(NULL, ctx, "@introduceDomain", NULL,
+				     false, NULL);
 	} else {
 		/* Use XS_INTRODUCE for recreating the xenbus event-channel. */
 		if (domain->port)
@@ -467,6 +464,32 @@ int do_introduce(struct connection *conn, struct buffered_data *in)
 		domain->port = (rc == -1) ? 0 : rc;
 	}
 
+	return domain;
+}
+
+/* domid, gfn, evtchn, path */
+int do_introduce(struct connection *conn, struct buffered_data *in)
+{
+	struct domain *domain;
+	char *vec[3];
+	unsigned int domid;
+	evtchn_port_t port;
+
+	if (get_strings(in, vec, ARRAY_SIZE(vec)) < ARRAY_SIZE(vec))
+		return EINVAL;
+
+	domid = atoi(vec[0]);
+	/* Ignore the gfn, we don't need it. */
+	port = atoi(vec[2]);
+
+	/* Sanity check args. */
+	if (port <= 0)
+		return EINVAL;
+
+	domain = introduce_domain(in, domid, port);
+	if (!domain)
+		return errno;
+
 	domain_conn_reset(domain);
 
 	send_ack(conn, XS_INTRODUCE);
@@ -692,17 +715,9 @@ static int dom0_init(void)
 	if (port == -1)
 		return -1;
 
-	dom0 = alloc_domain(NULL, xenbus_master_domid());
+	dom0 = introduce_domain(NULL, xenbus_master_domid(), port);
 	if (!dom0)
 		return -1;
-	if (new_domain(dom0, port))
-		return -1;
-
-	dom0->interface = xenbus_map();
-	if (dom0->interface == NULL)
-		return -1;
-
-	talloc_steal(dom0->conn, dom0); 
 
 	xenevtchn_notify(xce_handle, dom0->port);
 
Signature · 1 line
2.26.2
#25 lore raw

PATCH v10 18/25 tools/xenstore: evaluate the live update flag when starting

In reply to Juergen Gross

In the live update case several initialization steps of xenstore must be omitted or modified. Add the proper handling for that.

---
V4:
- set xprintf = trace in daemon case (Julien Grall)
- only update /tool/xenstored node contents

V7:
- some restructuring to enable keeping event channel fd

V8:
- pass evtfd to domain_init() as parameter (Julien Grall)
- call dom0_init() from main()

V10:
- remove support for remembering binary name (Andrew Cooper)

---
 tools/xenstore/xenstored_control.c |  5 ++++
 tools/xenstore/xenstored_control.h |  1 +
 tools/xenstore/xenstored_core.c    | 43 +++++++++++++++++++++---------
 tools/xenstore/xenstored_domain.c  | 26 +++++++++---------
 tools/xenstore/xenstored_domain.h  |  3 ++-
 tools/xenstore/xenstored_posix.c   |  1 -
 6 files changed, 50 insertions(+), 29 deletions(-)
tools/xenstore/xenstored_control.c +5 -0
index 437276de8d..a539666410 100644
@@ -436,6 +436,11 @@ static const char *lu_dump_state(const void *ctx, struct connection *conn)
 	return ret;
 }
 
+void lu_read_state(void)
+{
+	xprintf("live-update: read state\n");
+}
+
 static const char *lu_activate_binary(const void *ctx)
 {
 	return "Not yet implemented.";
tools/xenstore/xenstored_control.h +1 -0
index 207e0a6fa3..aac61f0590 100644
@@ -17,3 +17,4 @@
 */
 
 int do_control(struct connection *conn, struct buffered_data *in);
+void lu_read_state(void);
tools/xenstore/xenstored_core.c +30 -13

98 lines not on this page · raw message

tools/xenstore/xenstored_domain.c +12 -14
index 71b078caf3..94dd501a3b 100644
@@ -706,29 +706,23 @@ bool check_perms_special(const char *name, struct connection *conn)
 	return perm_for_conn(conn, p) & XS_PERM_READ;
 }
 
-static int dom0_init(void) 
-{ 
+void dom0_init(void)
+{
 	evtchn_port_t port;
 	struct domain *dom0;
 
 	port = xenbus_evtchn();
 	if (port == -1)
-		return -1;
+		barf_perror("Failed to initialize dom0 port");
 
 	dom0 = introduce_domain(NULL, xenbus_master_domid(), port);
 	if (!dom0)
-		return -1;
+		barf_perror("Failed to initialize dom0");
 
 	xenevtchn_notify(xce_handle, dom0->port);
-
-	if (set_dom_perms_default(&dom_release_perms) ||
-	    set_dom_perms_default(&dom_introduce_perms))
-		return -1;
-
-	return 0; 
 }
 
-void domain_init(void)
+void domain_init(int evtfd)
 {
 	int rc;
 
@@ -758,13 +752,17 @@ void domain_init(void)
 
 	talloc_set_destructor(xgt_handle, close_xgt_handle);
 
-	xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
+	if (evtfd < 0)
+		xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
+	else
+		xce_handle = xenevtchn_open_fd(NULL, evtfd, 0);
 
 	if (xce_handle == NULL)
 		barf_perror("Failed to open evtchn device");
 
-	if (dom0_init() != 0) 
-		barf_perror("Failed to initialize dom0 state"); 
+	if (set_dom_perms_default(&dom_release_perms) ||
+	    set_dom_perms_default(&dom_introduce_perms))
+		barf_perror("Failed to set special permissions");
 
 	if ((rc = xenevtchn_bind_virq(xce_handle, VIRQ_DOM_EXC)) == -1)
 		barf_perror("Failed to bind to domain exception virq port");
tools/xenstore/xenstored_domain.h +2 -1
index 413b974375..b20269b038 100644
@@ -42,7 +42,8 @@ int do_get_domain_path(struct connection *conn, struct buffered_data *in);
 /* Allow guest to reset all watches */
 int do_reset_watches(struct connection *conn, struct buffered_data *in);
 
-void domain_init(void);
+void domain_init(int evtfd);
+void dom0_init(void);
 
 /* Returns the implicit path of a connection (only domains have this) */
 const char *get_implicit_path(const struct connection *conn);
tools/xenstore/xenstored_posix.c +0 -1
index ae3e63e07f..48c37ffe3e 100644
@@ -85,7 +85,6 @@ void finish_daemonize(void)
 	dup2(devnull, STDOUT_FILENO);
 	dup2(devnull, STDERR_FILENO);
 	close(devnull);
-	xprintf = trace;
 }
 
 void init_pipe(int reopen_log_pipe[2])
Signature · 1 line
2.26.2
#26 lore raw

PATCH v10 19/25 tools/xenstore: read internal state when doing live upgrade

In reply to Juergen Gross

When started due to a live upgrade read the internal state and apply it to the data base and internal structures.

Add the main control functions for that.

For now only handle the daemon case.

---
V4:
- directly mmap dump state file (Julien Grall)
- use syslog() instead of xprintf() (Julien Grall)

V8:
- remove state file after reading it (Julien Grall)

---
 tools/xenstore/xenstored_control.c | 102 ++++++++++++++++++++++++++++-
 1 file changed, 101 insertions(+), 1 deletion(-)
tools/xenstore/xenstored_control.c +101 -1
index a539666410..129d2b44bb 100644
@@ -62,6 +62,14 @@ struct live_update {
 
 static struct live_update *lu_status;
 
+struct lu_dump_state {
+	void *buf;
+	unsigned int size;
+#ifndef __MINIOS__
+	int fd;
+#endif
+};
+
 static int lu_destroy(void *data)
 {
 #ifdef __MINIOS__
@@ -313,6 +321,14 @@ static void lu_dump_close(FILE *fp)
 
 	fclose(fp);
 }
+
+static void lu_get_dump_state(struct lu_dump_state *state)
+{
+}
+
+static void lu_close_dump_state(struct lu_dump_state *state)
+{
+}
 #else
 static const char *lu_binary(const void *ctx, struct connection *conn,
 			     const char *filename)
@@ -368,6 +384,50 @@ static void lu_dump_close(FILE *fp)
 {
 	fclose(fp);
 }
+
+static void lu_get_dump_state(struct lu_dump_state *state)
+{
+	char *filename;
+	struct stat statbuf;
+
+	state->size = 0;
+
+	filename = talloc_asprintf(NULL, "%s/state_dump", xs_daemon_rootdir());
+	if (!filename)
+		barf("Allocation failure");
+
+	state->fd = open(filename, O_RDONLY);
+	talloc_free(filename);
+	if (state->fd < 0)
+		return;
+	if (fstat(state->fd, &statbuf) != 0)
+		goto out_close;
+	state->size = statbuf.st_size;
+
+	state->buf = mmap(NULL, state->size, PROT_READ, MAP_PRIVATE,
+			  state->fd, 0);
+	if (state->buf == MAP_FAILED) {
+		state->size = 0;
+		goto out_close;
+	}
+
+	return;
+
+ out_close:
+	close(state->fd);
+}
+
+static void lu_close_dump_state(struct lu_dump_state *state)
+{
+	char *filename;
+
+	munmap(state->buf, state->size);
+	close(state->fd);
+
+	filename = talloc_asprintf(NULL, "%s/state_dump", xs_daemon_rootdir());
+	unlink(filename);
+	talloc_free(filename);
+}
 #endif
 
 static const char *lu_check_lu_allowed(const void *ctx, bool force,
@@ -438,7 +498,47 @@ static const char *lu_dump_state(const void *ctx, struct connection *conn)
 
 void lu_read_state(void)
 {
-	xprintf("live-update: read state\n");
+	struct lu_dump_state state;
+	struct xs_state_record_header *head;
+	void *ctx = talloc_new(NULL); /* Work context for subfunctions. */
+	struct xs_state_preamble *pre;
+
+	syslog(LOG_INFO, "live-update: read state\n");
+	lu_get_dump_state(&state);
+	if (state.size == 0)
+		barf_perror("No state found after live-update");
+
+	pre = state.buf;
+	if (memcmp(pre->ident, XS_STATE_IDENT, sizeof(pre->ident)) ||
+	    pre->version != htobe32(XS_STATE_VERSION) ||
+	    pre->flags != XS_STATE_FLAGS)
+		barf("Unknown record identifier");
+	for (head = state.buf + sizeof(*pre);
+	     head->type != XS_STATE_TYPE_END &&
+		(void *)head - state.buf < state.size;
+	     head = (void *)head + sizeof(*head) + head->length) {
+		switch (head->type) {
+		case XS_STATE_TYPE_GLOBAL:
+			break;
+		case XS_STATE_TYPE_CONN:
+			break;
+		case XS_STATE_TYPE_WATCH:
+			break;
+		case XS_STATE_TYPE_TA:
+			xprintf("live-update: ignore transaction record\n");
+			break;
+		case XS_STATE_TYPE_NODE:
+			break;
+		default:
+			xprintf("live-update: unknown state record %08x\n",
+				head->type);
+			break;
+		}
+	}
+
+	lu_close_dump_state(&state);
+
+	talloc_free(ctx);
 }
 
 static const char *lu_activate_binary(const void *ctx)
Signature · 1 line
2.26.2
#27 lore raw

PATCH v10 20/25 tools/xenstore: add reading global state for live update

In reply to Juergen Gross

Add reading the global state for live update.

---
 tools/xenstore/xenstored_control.c | 1 +
 tools/xenstore/xenstored_core.c    | 9 +++++++++
 tools/xenstore/xenstored_core.h    | 2 ++
 3 files changed, 12 insertions(+)
tools/xenstore/xenstored_control.c +1 -0
index 129d2b44bb..f6c4ab3d8a 100644
@@ -519,6 +519,7 @@ void lu_read_state(void)
 	     head = (void *)head + sizeof(*head) + head->length) {
 		switch (head->type) {
 		case XS_STATE_TYPE_GLOBAL:
+			read_state_global(ctx, head + 1);
 			break;
 		case XS_STATE_TYPE_CONN:
 			break;
tools/xenstore/xenstored_core.c +9 -0
index d6f8373ee0..5922a03a98 100644
@@ -2451,6 +2451,15 @@ const char *dump_state_nodes(FILE *fp, const void *ctx)
 	return dump_state_node_tree(fp, path);
 }
 
+void read_state_global(const void *ctx, const void *state)
+{
+	const struct xs_state_global *glb = state;
+
+	sock = glb->socket_fd;
+
+	domain_init(glb->evtchn_fd);
+}
+
 /*
  * Local variables:
  *  mode: C
tools/xenstore/xenstored_core.h +2 -0
index e40e0e6806..6c9d838f11 100644
@@ -244,6 +244,8 @@ const char *dump_state_node_perms(FILE *fp, struct xs_state_node *sn,
 				  const struct xs_permissions *perms,
 				  unsigned int n_perms);
 
+void read_state_global(const void *ctx, const void *state);
+
 #endif /* _XENSTORED_CORE_H */
 
 /*
Signature · 1 line
2.26.2
#28 lore raw

PATCH v10 21/25 tools/xenstore: add read connection state for live update

In reply to Juergen Gross

Add the needed functions for reading connection state for live update.

As the connection is identified by a unique connection id in the state records we need to add this to struct connection. Add a new function to return the connection based on a connection id.

---
V2:
- fixed condition in introduce_domain() (Julien Grall)

V4:
- set pending data msg type to XS_INVALID (Julien Grall)
- add buffered read data (Julien Grall)

V5:
- really read buffered read data (Julien Grall)
- drop conn parameter from introduce_domain() (Paul Durrant)
- split pending write data into individual buffers

V6:
- rename "first" to "partial" (Paul Durrant)

V7:
- use local port from connection data

V8:
- remove dom0 special handling

---
 tools/xenstore/xenstored_control.c |   1 +
 tools/xenstore/xenstored_core.c    | 102 ++++++++++++++++++++++++++++-
 tools/xenstore/xenstored_core.h    |  10 +++
 tools/xenstore/xenstored_domain.c  |  60 +++++++++++++----
 tools/xenstore/xenstored_domain.h  |   2 +
 5 files changed, 162 insertions(+), 13 deletions(-)
tools/xenstore/xenstored_control.c +1 -0
index f6c4ab3d8a..4bf075ad79 100644
@@ -522,6 +522,7 @@ void lu_read_state(void)
 			read_state_global(ctx, head + 1);
 			break;
 		case XS_STATE_TYPE_CONN:
+			read_state_connection(ctx, head + 1);
 			break;
 		case XS_STATE_TYPE_WATCH:
 			break;
tools/xenstore/xenstored_core.c +100 -2

125 lines not on this page · raw message

tools/xenstore/xenstored_core.h +10 -0
index 6c9d838f11..cb256626fe 100644
@@ -118,6 +118,9 @@ struct connection
 	/* Methods for communicating over this connection: write can be NULL */
 	connwritefn_t *write;
 	connreadfn_t *read;
+
+	/* Support for live update: connection id. */
+	unsigned int conn_id;
 };
 extern struct list_head connections;
 
@@ -178,6 +181,7 @@ struct node *read_node(struct connection *conn, const void *ctx,
 		       const char *name);
 
 struct connection *new_connection(connwritefn_t *write, connreadfn_t *read);
+struct connection *get_connection_by_id(unsigned int conn_id);
 void check_store(void);
 void corrupt(struct connection *conn, const char *fmt, ...);
 enum xs_perm_type perm_for_conn(struct connection *conn,
@@ -229,6 +233,10 @@ void finish_daemonize(void);
 /* Open a pipe for signal handling */
 void init_pipe(int reopen_log_pipe[2]);
 
+int writefd(struct connection *conn, const void *data, unsigned int len);
+int readfd(struct connection *conn, void *data, unsigned int len);
+
+extern struct interface_funcs socket_funcs;
 extern xengnttab_handle **xgt_handle;
 
 int remember_string(struct hashtable *hash, const char *str);
@@ -245,6 +253,8 @@ const char *dump_state_node_perms(FILE *fp, struct xs_state_node *sn,
 				  unsigned int n_perms);
 
 void read_state_global(const void *ctx, const void *state);
+void read_state_buffered_data(const void *ctx, struct connection *conn,
+			      const struct xs_state_connection *sc);
 
 #endif /* _XENSTORED_CORE_H */
 
tools/xenstore/xenstored_domain.c +49 -11

108 lines not on this page · raw message

tools/xenstore/xenstored_domain.h +2 -0
index b20269b038..8f3b4e0f8b 100644
@@ -101,4 +101,6 @@ void wrl_apply_debit_trans_commit(struct connection *conn);
 const char *dump_state_connections(FILE *fp, struct connection *conn);
 const char *dump_state_special_nodes(FILE *fp);
 
+void read_state_connection(const void *ctx, const void *state);
+
 #endif /* _XENSTORED_DOMAIN_H */
Signature · 1 line
2.26.2
#29 lore raw

PATCH v10 22/25 tools/xenstore: add read node state for live update

In reply to Juergen Gross

Add the needed functions for reading node state for live update.

This requires some refactoring of current node handling in Xenstore in order to avoid repeating the same code patterns multiple times.

---
V4:
- drop local node handling (Julien Grall)

V5:
- use set_tdb_key (Paul Durrant)

V6:
- add permission flag handling (Julien Grall)

---
 tools/xenstore/xenstored_control.c |   1 +
 tools/xenstore/xenstored_core.c    | 105 ++++++++++++++++++++++++++---
 tools/xenstore/xenstored_core.h    |   1 +
 3 files changed, 96 insertions(+), 11 deletions(-)
tools/xenstore/xenstored_control.c +1 -0
index 4bf075ad79..a978ccf17e 100644
@@ -530,6 +530,7 @@ void lu_read_state(void)
 			xprintf("live-update: ignore transaction record\n");
 			break;
 		case XS_STATE_TYPE_NODE:
+			read_state_node(ctx, head + 1);
 			break;
 		default:
 			xprintf("live-update: unknown state record %08x\n",
tools/xenstore/xenstored_core.c +94 -11

128 lines not on this page · raw message

tools/xenstore/xenstored_core.h +1 -0
index cb256626fe..0af2f364bf 100644
@@ -255,6 +255,7 @@ const char *dump_state_node_perms(FILE *fp, struct xs_state_node *sn,
 void read_state_global(const void *ctx, const void *state);
 void read_state_buffered_data(const void *ctx, struct connection *conn,
 			      const struct xs_state_connection *sc);
+void read_state_node(const void *ctx, const void *state);
 
 #endif /* _XENSTORED_CORE_H */
 
Signature · 1 line
2.26.2
#30 lore raw

PATCH v10 23/25 tools/xenstore: add read watch state for live update

In reply to Juergen Gross

Add reading the watch state records for live update.

This requires factoring out some of the add watch functionality into a dedicated function.

---
V4:
- add comment (Julien Grall)

V6:
- correct check_watch_path() (setting errno)

---
 tools/xenstore/xenstored_control.c |   2 +
 tools/xenstore/xenstored_watch.c   | 114 +++++++++++++++++++++--------
 tools/xenstore/xenstored_watch.h   |   2 +
 3 files changed, 88 insertions(+), 30 deletions(-)
tools/xenstore/xenstored_control.c +2 -0
index a978ccf17e..8a1e3b35fe 100644
@@ -35,6 +35,7 @@ Interactive commands for Xen Store Daemon.
 #include "xenstored_core.h"
 #include "xenstored_control.h"
 #include "xenstored_domain.h"
+#include "xenstored_watch.h"
 
 /* Mini-OS only knows about MAP_ANON. */
 #ifndef MAP_ANONYMOUS
@@ -525,6 +526,7 @@ void lu_read_state(void)
 			read_state_connection(ctx, head + 1);
 			break;
 		case XS_STATE_TYPE_WATCH:
+			read_state_watch(ctx, head + 1);
 			break;
 		case XS_STATE_TYPE_TA:
 			xprintf("live-update: ignore transaction record\n");
tools/xenstore/xenstored_watch.c +84 -30

141 lines not on this page · raw message

tools/xenstore/xenstored_watch.h +2 -0
index 3d81645f45..0e693f0839 100644
@@ -33,4 +33,6 @@ void conn_delete_all_watches(struct connection *conn);
 const char *dump_state_watches(FILE *fp, struct connection *conn,
 			       unsigned int conn_id);
 
+void read_state_watch(const void *ctx, const void *state);
+
 #endif /* _XENSTORED_WATCH_H */
Signature · 1 line
2.26.2
#31 lore raw

PATCH v10 24/25 tools/xenstore: handle dying domains in live update

In reply to Juergen Gross

From: Julien Grall <[email protected]>

A domain could just be dying when live updating Xenstore, so the case of not being able to map the ring page or to connect to the event channel must be handled gracefully.

---
V4:
- new patch (Julien, I hope adding the Sob: is okay?)

V10:
- removed "XXX..." comment (Julien Grall)

---
 tools/xenstore/xenstored_control.c |  7 +++++++
 tools/xenstore/xenstored_domain.c  | 25 +++++++++++++++++--------
 tools/xenstore/xenstored_domain.h  |  2 ++
 3 files changed, 26 insertions(+), 8 deletions(-)
tools/xenstore/xenstored_control.c +7 -0
index 8a1e3b35fe..dee55de264 100644
@@ -544,6 +544,13 @@ void lu_read_state(void)
 	lu_close_dump_state(&state);
 
 	talloc_free(ctx);
+
+	/*
+	 * We may have missed the VIRQ_DOM_EXC notification and a domain may
+	 * have died while we were live-updating. So check all the domains are
+	 * still alive.
+	 */
+	check_domains(true);
 }
 
 static const char *lu_activate_binary(const void *ctx)
tools/xenstore/xenstored_domain.c +17 -8
index 0cd8234bd1..dfda90c791 100644
@@ -220,7 +220,7 @@ static bool get_domain_info(unsigned int domid, xc_dominfo_t *dominfo)
 	       dominfo->domid == domid;
 }
 
-static void domain_cleanup(void)
+void check_domains(bool restore)
 {
 	xc_dominfo_t dominfo;
 	struct domain *domain;
@@ -244,7 +244,14 @@ static void domain_cleanup(void)
 				domain->shutdown = true;
 				notify = 1;
 			}
-			if (!dominfo.dying)
+			/*
+			 * On Restore, we may have been unable to remap the
+			 * interface and the port. As we don't know whether
+			 * this was because of a dying domain, we need to
+			 * check if the interface and port are still valid.
+			 */
+			if (!dominfo.dying && domain->port &&
+			    domain->interface)
 				continue;
 		}
 		if (domain->conn) {
@@ -270,7 +277,7 @@ void handle_event(void)
 		barf_perror("Failed to read from event fd");
 
 	if (port == virq_port)
-		domain_cleanup();
+		check_domains(false);
 
 	if (xenevtchn_unmask(xce_handle, port) == -1)
 		barf_perror("Failed to write to event fd");
@@ -442,14 +449,16 @@ static struct domain *introduce_domain(const void *ctx,
 	if (!domain->introduced) {
 		interface = is_master_domain ? xenbus_map()
 					     : map_interface(domid);
-		if (!interface)
+		if (!interface && !restore)
 			return NULL;
 		if (new_domain(domain, port, restore)) {
 			rc = errno;
-			if (is_master_domain)
-				unmap_xenbus(interface);
-			else
-				unmap_interface(interface);
+			if (interface) {
+				if (is_master_domain)
+					unmap_xenbus(interface);
+				else
+					unmap_interface(interface);
+			}
 			errno = rc;
 			return NULL;
 		}
tools/xenstore/xenstored_domain.h +2 -0
index 8f3b4e0f8b..1cc1c03ed8 100644
@@ -21,6 +21,8 @@
 
 void handle_event(void);
 
+void check_domains(bool restore);
+
 /* domid, mfn, eventchn, path */
 int do_introduce(struct connection *conn, struct buffered_data *in);
 
Signature · 1 line
2.26.2
#32 lore raw

PATCH v10 25/25 tools/xenstore: activate new binary for live update

In reply to Juergen Gross

Add activation of the new binary for live update. The daemon case is handled completely, while for stubdom we only add stubs.

---
V7:
- added unbinding dom0 and virq event channels

V8:
- no longer close dom0 evtchn (Julien Grall)

V10:
- remember original argc and argv (taken from deleted patch)

---
 tools/xenstore/xenstored_control.c | 61 +++++++++++++++++++++++++++++-
 tools/xenstore/xenstored_core.c    |  5 +++
 tools/xenstore/xenstored_core.h    |  3 ++
 tools/xenstore/xenstored_domain.c  |  6 +++
 tools/xenstore/xenstored_domain.h  |  1 +
 5 files changed, 75 insertions(+), 1 deletion(-)
tools/xenstore/xenstored_control.c +60 -1
index dee55de264..a5d8185c41 100644
@@ -16,6 +16,7 @@ Interactive commands for Xen Store Daemon.
     along with this program; If not, see <http://www.gnu.org/licenses/>.
 */
 
+#include <ctype.h>
 #include <errno.h>
 #include <stdarg.h>
 #include <stdio.h>
@@ -330,6 +331,11 @@ static void lu_get_dump_state(struct lu_dump_state *state)
 static void lu_close_dump_state(struct lu_dump_state *state)
 {
 }
+
+static char *lu_exec(const void *ctx, int argc, char **argv)
+{
+	return "NYI";
+}
 #else
 static const char *lu_binary(const void *ctx, struct connection *conn,
 			     const char *filename)
@@ -429,6 +435,14 @@ static void lu_close_dump_state(struct lu_dump_state *state)
 	unlink(filename);
 	talloc_free(filename);
 }
+
+static char *lu_exec(const void *ctx, int argc, char **argv)
+{
+	argv[0] = lu_status->filename;
+	execvp(argv[0], argv);
+
+	return "Error activating new binary.";
+}
 #endif
 
 static const char *lu_check_lu_allowed(const void *ctx, bool force,
@@ -555,7 +569,52 @@ void lu_read_state(void)
 
 static const char *lu_activate_binary(const void *ctx)
 {
-	return "Not yet implemented.";
+	int argc;
+	char **argv;
+	unsigned int i;
+
+	if (lu_status->cmdline) {
+		argc = 4;   /* At least one arg + progname + "-U" + NULL. */
+		for (i = 0; lu_status->cmdline[i]; i++)
+			if (isspace(lu_status->cmdline[i]))
+				argc++;
+		argv = talloc_array(ctx, char *, argc);
+		if (!argv)
+			return "Allocation failure.";
+
+		i = 0;
+		argc = 1;
+		argv[1] = strtok(lu_status->cmdline, " \t");
+		while (argv[argc]) {
+			if (!strcmp(argv[argc], "-U"))
+				i = 1;
+			argc++;
+			argv[argc] = strtok(NULL, " \t");
+		}
+
+		if (!i) {
+			argv[argc++] = "-U";
+			argv[argc] = NULL;
+		}
+	} else {
+		for (i = 0; i < orig_argc; i++)
+			if (!strcmp(orig_argv[i], "-U"))
+				break;
+
+		argc = orig_argc;
+		argv = talloc_array(ctx, char *, orig_argc + 2);
+		if (!argv)
+			return "Allocation failure.";
+
+		memcpy(argv, orig_argv, orig_argc * sizeof(*argv));
+		if (i == orig_argc)
+			argv[argc++] = "-U";
+		argv[argc] = NULL;
+	}
+
+	domain_deinit();
+
+	return lu_exec(ctx, argc, argv);
 }
 
 static const char *lu_start(const void *ctx, struct connection *conn,
tools/xenstore/xenstored_core.c +5 -0
index 649dfb534a..7174a9288a 100644
@@ -72,6 +72,9 @@ static unsigned int nr_fds;
 
 static int sock = -1;
 
+int orig_argc;
+char **orig_argv;
+
 static bool verbose = false;
 LIST_HEAD(connections);
 int tracefd = -1;
@@ -2026,6 +2029,8 @@ int main(int argc, char *argv[])
 	const char *pidfile = NULL;
 	int timeout;
 
+	orig_argc = argc;
+	orig_argv = argv;
 
 	while ((opt = getopt_long(argc, argv, "DE:F:HNPS:t:A:M:T:RVW:U", options,
 				  NULL)) != -1) {
tools/xenstore/xenstored_core.h +3 -0
index 0af2f364bf..91e036f49f 100644
@@ -201,6 +201,9 @@ void dtrace_io(const struct connection *conn, const struct buffered_data *data,
 void reopen_log(void);
 void close_log(void);
 
+extern int orig_argc;
+extern char **orig_argv;
+
 extern char *tracefile;
 extern int tracefd;
 
tools/xenstore/xenstored_domain.c +6 -0
index dfda90c791..317427b7cb 100644
@@ -783,6 +783,12 @@ void domain_init(int evtfd)
 	virq_port = rc;
 }
 
+void domain_deinit(void)
+{
+	if (virq_port)
+		xenevtchn_unbind(xce_handle, virq_port);
+}
+
 void domain_entry_inc(struct connection *conn, struct node *node)
 {
 	struct domain *d;
tools/xenstore/xenstored_domain.h +1 -0
index 1cc1c03ed8..dc97591713 100644
@@ -46,6 +46,7 @@ int do_reset_watches(struct connection *conn, struct buffered_data *in);
 
 void domain_init(int evtfd);
 void dom0_init(void);
+void domain_deinit(void);
 
 /* Returns the implicit path of a connection (only domains have this) */
 const char *get_implicit_path(const struct connection *conn);
Signature · 1 line
2.26.2
#33 lore raw
In reply to Juergen Gross
Quoted 15 lines · Juergen Gross

Today Xenstore is not restartable. This means a Xen server needing an update of xenstored has to be rebooted in order to let this update become effective.

This patch series is changing that: The internal state of xenstored (the contents of Xenstore, all connections to various clients like programs or other domains, and watches) is saved in a defined format and a new binary is being activated consuming the old state. All connections are being restored and the new Xenstore binary will continue where the old one stopped.

This patch series has been under (secret) development for quite some time. It hasn't been posted to xen-devel until now due to the various Xenstore related security issues which have become public only today.

Quoted 7 lines

There will be a similar series for oxenstored posted.

Xenstore-stubdom is not yet supported, but I'm planning to start working on that soon.

Changes in V10 (for the members of the security team):
- dropped patch 6 as requested by Andrew

I went through this series when it was posted to security@ and didn't find that many issues. I guess I will wait for Andrew's comment to be addressed and have a look again.

Wei.