Skip to contentSkip to conversation

Enable Xenstore-stubdom Live Update

PATCH Superseded Juergen Gross 3 participants 23 messages Jul 2, 2025 – Jul 21, 2025

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover Enable Xenstore-stubdom Live Update

With this series applied I was able to perform Live Update of Xenstore running in a PVH stubdom.

This series is based on a related Mini-OS patch series AND it needs the Mini-OS commit id updated accordingly in Config.mk.

One relevant change in this series is how the Xenstore state is being read after activating the new binary. Instead of using mmap() the state records are read one by one via fread(). This might have a small performance impact for the daemon live update case, but it should be neglectable.

Juergen Gross (9):
stubdom: configure kexec for PVH Xenstore stubdom tools/xenstore: perform kexec for stubdom live update
libevtchn: add O_CLOEXEC support for Mini-OS
libxengnttab: set the cloexec flag on Mini-OS
xenstored: use read() instead of mmap() for reading live update state
xenstored: don't use xenevtchn_fdopen() in stubdom tools/libxenevtchn: add xenevtchn_bind() under Mini-OS
xenstored: rebind event channels after live update in stubdom tools/xenstored: make stubdom_init() live update aware

 stubdom/xenstorepvh-minios.cfg |  2 +
 tools/include/xenevtchn.h      | 11 ++++
 tools/libs/evtchn/minios.c     | 23 +++++++--
 tools/libs/gnttab/minios.c     |  1 +
 tools/xenstored/core.c         |  5 +-
 tools/xenstored/core.h         | 13 ++++-
 tools/xenstored/domain.c       | 25 +++++----
 tools/xenstored/domain.h       |  2 +-
 tools/xenstored/lu.c           | 93 +++++++++++++++++++++-------------
 tools/xenstored/lu_minios.c    | 33 +++++++++++-
 tools/xenstored/minios.c       | 22 ++++++--
 tools/xenstored/posix.c        | 10 ++++
 12 files changed, 183 insertions(+), 57 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/9 stubdom: configure kexec for PVH Xenstore stubdom

Add the required configuration settings to enable kexec in the PVH variant of xenstore-stubdom. This is required to support live update.

Set the number of kexec module pages to 18 in order to support using the kexec module memory for the stubdom's 9pfs granted pages.

In Xenstore code test the CONFIG_KEXEC setting in order to determine whether stubdom live update is supported (PV xenstore-stubdom doesn't have live update support, as it lacks kexec).

---
 stubdom/xenstorepvh-minios.cfg | 2 ++
 tools/xenstored/core.h         | 5 +++++
 2 files changed, 7 insertions(+)
stubdom/xenstorepvh-minios.cfg +2 -0
index 62a228f33d..ed2461d7fa 100644
@@ -8,4 +8,6 @@ CONFIG_CONSFRONT=n
 CONFIG_LWIP=n
 CONFIG_9PFRONT=y
 CONFIG_BALLOON=y
+CONFIG_KEXEC=y
+CONFIG_KEXEC_MODULE_PAGES=18
 XEN_INTERFACE_VERSION=__XEN_LATEST_INTERFACE_VERSION__
tools/xenstored/core.h +5 -0
index 632886cecf..1ba9592d16 100644
@@ -45,6 +45,11 @@
 #endif
 #endif
 
+/* Live update in stubdom case needs kexec support. */
+#if defined(__MINIOS__) && !defined(CONFIG_KEXEC)
+#define NO_LIVE_UPDATE
+#endif
+
 /* DEFAULT_BUFFER_SIZE should be large enough for each errno string. */
 #define DEFAULT_BUFFER_SIZE 16
 
Signature · 1 line
2.43.0
Reviewed-by on v1 1/9
#2 lore raw

Re: [PATCH 1/9] stubdom: configure kexec for PVH Xenstore stubdom

Quoted 11 lines · Juergen Gross

Add the required configuration settings to enable kexec in the PVH variant of xenstore-stubdom. This is required to support live update.

Set the number of kexec module pages to 18 in order to support using the kexec module memory for the stubdom's 9pfs granted pages.

In Xenstore code test the CONFIG_KEXEC setting in order to determine whether stubdom live update is supported (PV xenstore-stubdom doesn't have live update support, as it lacks kexec).

#3 lore raw

PATCH 2/9 tools/xenstore: perform kexec for stubdom live update

In reply to Juergen Gross

For the final step of live-update in stubdom, call kexec().

---
 tools/xenstored/lu_minios.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)
tools/xenstored/lu_minios.c +32 -1
index b14a0b29d5..b8ca631610 100644
@@ -8,13 +8,44 @@
 #include <stdlib.h>
 #include <syslog.h>
 
+#include <mini-os/kexec.h>
+
 #include "talloc.h"
 #include "lu.h"
+#include "core.h"
 
 #ifndef NO_LIVE_UPDATE
 char *lu_exec(const void *ctx, int argc, char **argv)
 {
-	return "NYI";
+	int i;
+	int ret;
+	char *errbuf;
+	char *cmdline;
+
+	if (!lu_status->kernel)
+		return "No new kernel";
+
+	cmdline = talloc_strdup(ctx, "");
+	if (!cmdline)
+		return "Allocation failure";
+	for (i = 1; argv[i]; i++) {
+		if (i > 1) {
+			cmdline = talloc_append_string(ctx, cmdline, " ");
+			if (!cmdline)
+				return "Allocation failure";
+		}
+		cmdline = talloc_append_string(ctx, cmdline, argv[i]);
+		if (!cmdline)
+			return "Allocation failure";
+	}
+
+	ret = kexec(lu_status->kernel, lu_status->kernel_size, cmdline);
+
+	errbuf = talloc_asprintf(ctx, "kexec() returned %d", ret);
+	if (!errbuf)
+		errbuf = "kexec() returned";
+
+	return errbuf;
 }
 
 static const char *lu_binary_alloc(const void *ctx, struct connection *conn,
Signature · 1 line
2.43.0
#5 lore raw

PATCH 3/9 libevtchn: add O_CLOEXEC support for Mini-OS

In reply to Juergen Gross

With Mini-OS now supporting O_CLOEXEC for the kexec case, no longer ignore the XENEVTCHN_NO_CLOEXEC flag in minios.c.

---
 tools/libs/evtchn/minios.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
tools/libs/evtchn/minios.c +2 -4
index 046cd28d37..0d2a201438 100644
@@ -114,10 +114,6 @@ static void evtchn_initialize(void)
     ftype_evtchn = alloc_file_type(&evtchn_ops);
 }
 
-/*
- * XENEVTCHN_NO_CLOEXEC is being ignored, as there is no exec() call supported
- * in Mini-OS.
- */
 int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int flags)
 {
     int fd;
@@ -137,6 +133,8 @@ int osdep_evtchn_open(xenevtchn_handle *xce, unsigned int flags)
         return -1;
     }
 
+    if ( !(flags & XENEVTCHN_NO_CLOEXEC) )
+        file->cloexec = true;
     file->dev = ports;
     XEN_LIST_INIT(&ports->list);
     xce->fd = fd;
Signature · 1 line
2.43.0
#7 lore raw

PATCH 4/9 libxengnttab: set the cloexec flag on Mini-OS

In reply to Juergen Gross

With Mini-OS now supporting the O_CLOEXEC flag for the kexec case, set the related cloexec flag in minios.c.

---
 tools/libs/gnttab/minios.c | 1 +
 1 file changed, 1 insertion(+)
tools/libs/gnttab/minios.c +1 -0
index cac89f0ba6..63bbdc7719 100644
@@ -74,6 +74,7 @@ int osdep_gnttab_open(xengnttab_handle *xgt)
     }
 
     file->dev = gntmap;
+    file->cloexec = true;
     gntmap_init(gntmap);
     xgt->fd = fd;
     return 0;
Signature · 1 line
2.43.0
#9 lore raw

PATCH 5/9 xenstored: use fread() instead of mmap() for reading live update state

In reply to Juergen Gross

Mini-OS doesn't support using mmap() for accessing a file. In order to support reading the live update state from a 9pfs based file, use fread() instead of mmap().

---
 tools/xenstored/lu.c | 93 +++++++++++++++++++++++++++-----------------
 1 file changed, 58 insertions(+), 35 deletions(-)
tools/xenstored/lu.c +58 -35
index 4fccbbc195..029c4fab72 100644
@@ -25,9 +25,11 @@
 
 struct lu_dump_state {
 	void *buf;
+	unsigned int buf_size;
 	unsigned int size;
-	int fd;
+	unsigned int offset;
 	char *filename;
+	FILE *fp;
 };
 
 struct live_update *lu_status;
@@ -80,9 +82,10 @@ bool lu_is_pending(void)
 	return lu_status != NULL;
 }
 
-static void lu_get_dump_state(struct lu_dump_state *state)
+static void lu_get_dump_state(void *ctx, struct lu_dump_state *state)
 {
 	struct stat statbuf;
+	int fd;
 
 	state->size = 0;
 
@@ -91,77 +94,102 @@ static void lu_get_dump_state(struct lu_dump_state *state)
 	if (!state->filename)
 		barf("Allocation failure");
 
-	state->fd = open(state->filename, O_RDONLY);
-	if (state->fd < 0)
-		return;
-	if (fstat(state->fd, &statbuf) != 0)
-		goto out_close;
+	fd = open(state->filename, O_RDONLY);
+	if (fd < 0)
+		barf("No state file found");
+	if (fstat(fd, &statbuf) != 0)
+		barf("Could not fstat state file");
 	state->size = statbuf.st_size;
 
-	state->buf = mmap(NULL, state->size, PROT_READ, MAP_PRIVATE,
-			  state->fd, 0);
-	if (state->buf == MAP_FAILED) {
-		state->size = 0;
-		goto out_close;
-	}
+	/* Start with a 4k buffer. If needed we'll reallocate a larger one. */
+	state->buf_size = 4096;
+	state->buf = talloc_size(ctx, state->buf_size);
+	if (!state->buf)
+		barf("Allocation failure");
 
-	return;
+	state->fp = fdopen(fd, "r");
+}
 
- out_close:
-	close(state->fd);
+static void lu_dump_close(FILE *fp)
+{
+	fclose(fp);
 }
 
 static void lu_close_dump_state(struct lu_dump_state *state)
 {
 	assert(state->filename != NULL);
 
-	munmap(state->buf, state->size);
-	close(state->fd);
+	lu_dump_close(state->fp);
 
 	unlink(state->filename);
 	talloc_free(state->filename);
+	talloc_free(state->buf);
+}
+
+static void lu_read_data(void *ctx, struct lu_dump_state *state,
+			 unsigned int size)
+{
+	if (state->offset + size > state->size)
+		barf("Inconsistent state data");
+
+	if (size > state->buf_size) {
+		state->buf = talloc_realloc_size(ctx, state->buf, size);
+		if (!state->buf)
+			barf("Allocation failure");
+		state->buf_size = size;
+	}
+
+	if (fread(state->buf, size, 1, state->fp) != 1)
+		barf("State read error");
+
+	state->offset += size;
 }
 
 void lu_read_state(void)
 {
 	struct lu_dump_state state = {};
-	struct xs_state_record_header *head;
+	struct xs_state_record_header head;
 	void *ctx = talloc_new(NULL); /* Work context for subfunctions. */
 	struct xs_state_preamble *pre;
 
 	syslog(LOG_INFO, "live-update: read state\n");
-	lu_get_dump_state(&state);
+	lu_get_dump_state(ctx, &state);
 	if (state.size == 0)
 		barf_perror("No state found after live-update");
 
+	lu_read_data(ctx, &state, sizeof(*pre));
 	pre = state.buf;
 	if (memcmp(pre->ident, XS_STATE_IDENT, sizeof(pre->ident)) ||
 	    pre->version != htobe32(XS_STATE_VERSION) ||
 	    pre->flags != XS_STATE_FLAGS)
 		barf("Unknown record identifier");
-	for (head = state.buf + sizeof(*pre);
-	     head->type != XS_STATE_TYPE_END &&
-		(void *)head - state.buf < state.size;
-	     head = (void *)head + sizeof(*head) + head->length) {
-		switch (head->type) {
+
+	for (;;) {
+		lu_read_data(ctx, &state, sizeof(head));
+		head = *(struct xs_state_record_header *)(state.buf);
+		if (head.type == XS_STATE_TYPE_END)
+			break;
+		lu_read_data(ctx, &state, head.length);
+
+		switch (head.type) {
 		case XS_STATE_TYPE_GLOBAL:
-			read_state_global(ctx, head + 1);
+			read_state_global(ctx, state.buf);
 			break;
 		case XS_STATE_TYPE_CONN:
-			read_state_connection(ctx, head + 1);
+			read_state_connection(ctx, state.buf);
 			break;
 		case XS_STATE_TYPE_WATCH:
-			read_state_watch(ctx, head + 1);
+			read_state_watch(ctx, state.buf);
 			break;
 		case XS_STATE_TYPE_TA:
 			xprintf("live-update: ignore transaction record\n");
 			break;
 		case XS_STATE_TYPE_NODE:
-			read_state_node(ctx, head + 1);
+			read_state_node(ctx, state.buf);
 			break;
 		default:
 			xprintf("live-update: unknown state record %08x\n",
-				head->type);
+				head.type);
 			break;
 		}
 	}
@@ -266,11 +294,6 @@ static FILE *lu_dump_open(const void *ctx)
 	return fdopen(fd, "w");
 }
 
-static void lu_dump_close(FILE *fp)
-{
-	fclose(fp);
-}
-
 static const char *lu_dump_state(const void *ctx, struct connection *conn)
 {
 	FILE *fp;
Signature · 1 line
2.43.0
#11 lore raw

PATCH 6/9 xenstored: don't use xenevtchn_fdopen() in stubdom

In reply to Juergen Gross

When running in a stubdom environment xenevtchn_fdopen() won't work, as any file descriptor state is lost across kexec().

Use a wrapper to replace the call of xenevtchn_fdopen() with the really needed xenevtchn_open() when running on top of Mini-OS.

---
 tools/xenstored/core.h   | 3 +++
 tools/xenstored/domain.c | 2 +-
 tools/xenstored/minios.c | 5 +++++
 tools/xenstored/posix.c  | 5 +++++
 4 files changed, 14 insertions(+), 1 deletion(-)
tools/xenstored/core.h +3 -0
index 1ba9592d16..bef24a688c 100644
@@ -29,6 +29,7 @@
 #include <stdint.h>
 #include <time.h>
 #include <errno.h>
+#include <xenevtchn.h>
 
 #include "xenstore_lib.h"
 #include "xenstore_state.h"
@@ -405,6 +406,8 @@ void handle_special_fds(void);
 int get_socket_fd(void);
 void set_socket_fd(int fd);
 
+xenevtchn_handle *evtchn_fdopen(int fd);
+
 #ifdef __MINIOS__
 void mount_9pfs(void);
 #endif
tools/xenstored/domain.c +1 -1
index e1d5e8d614..e0c2342edb 100644
@@ -1326,7 +1326,7 @@ void domain_init(int evtfd)
 	if (evtfd < 0)
 		xce_handle = xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
 	else
-		xce_handle = xenevtchn_fdopen(NULL, evtfd, 0);
+		xce_handle = evtchn_fdopen(evtfd);
 
 	if (xce_handle == NULL)
 		barf_perror("Failed to open evtchn device");
tools/xenstored/minios.c +5 -0
index a229954cf4..aa1f03fd6b 100644
@@ -85,6 +85,11 @@ void set_socket_fd(int fd)
 {
 }
 
+xenevtchn_handle *evtchn_fdopen(int fd)
+{
+	return xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
+}
+
 static void mount_thread(void *p)
 {
 	xenbus_event_queue events = NULL;
tools/xenstored/posix.c +5 -0
index 6037d739d0..ebdec82215 100644
@@ -408,6 +408,11 @@ void set_socket_fd(int fd)
 	sock = fd;
 }
 
+xenevtchn_handle *evtchn_fdopen(int fd)
+{
+	return xenevtchn_fdopen(NULL, fd, 0);
+}
+
 const char *xenstore_rundir(void)
 {
 	return xenstore_daemon_rundir();
Signature · 1 line
2.43.0
Reviewed-by on v1 6/9
#12 lore raw

Re: [PATCH 6/9] xenstored: don't use xenevtchn_fdopen() in stubdom

Quoted 7 lines · Juergen Gross

When running in a stubdom environment xenevtchn_fdopen() won't work, as any file descriptor state is lost across kexec().

Use a wrapper to replace the call of xenevtchn_fdopen() with the really needed xenevtchn_open() when running on top of Mini-OS.

#13 lore raw

PATCH 7/9 tools/libxenevtchn: add xenevtchn_bind() under Mini-OS

In reply to Juergen Gross

In order to reactivate an event channel after kexec() of Mini-OS, libxenevtchn needs to allocate the port data for the event channel and set the handler again. Add a new interface xenevtchn_bind() for that purpose, available under Mini-OS only.

---
 tools/include/xenevtchn.h  | 11 +++++++++++
 tools/libs/evtchn/minios.c | 17 +++++++++++++++++
 2 files changed, 28 insertions(+)
tools/include/xenevtchn.h +11 -0
index 1255c85178..113a718d67 100644
@@ -173,6 +173,17 @@ int xenevtchn_unmask(xenevtchn_handle *xce, evtchn_port_t port);
  */
 int xenevtchn_restrict(xenevtchn_handle *xce, domid_t domid);
 
+/**
+ * Bind an event channel under Mini-OS.
+ *
+ * Bind an event channel specified by its known port after a kexec() of
+ * Mini-OS. This function is available under Mini-OS only!
+ *
+ * @parm xce handle to the open evtchn interface
+ * @parm port the event channel to bind again
+ * @return 0 on success, -1 on failure with errno set appropriately.
+ */
+int xenevtchn_bind(xenevtchn_handle *xce, evtchn_port_t port);
 #endif
 
 /*
tools/libs/evtchn/minios.c +17 -0
index 0d2a201438..36e4201249 100644
@@ -259,6 +259,23 @@ xenevtchn_port_or_error_t xenevtchn_bind_interdomain(xenevtchn_handle *xce,
     return local_port;
 }
 
+int xenevtchn_bind(xenevtchn_handle *xce, evtchn_port_t port)
+{
+    struct port_info *port_info;
+    port_info = port_alloc(xce);
+    if ( port_info == NULL )
+        return -1;
+
+    printf("xenevtchn_bind(%"PRId32")\n", port);
+    bind_evtchn(port, evtchn_handler, xce);
+
+    port_info->bound = true;
+    port_info->port = port;
+    unmask_evtchn(port);
+
+    return 0;
+}
+
 int xenevtchn_unbind(xenevtchn_handle *xce, evtchn_port_t port)
 {
     int fd = xce->fd;
Signature · 1 line
2.43.0
Reviewed-by on v1 7/9
#14 lore raw

Re: [PATCH 7/9] tools/libxenevtchn: add xenevtchn_bind() under Mini-OS

Quoted 6 lines · Juergen Gross

In order to reactivate an event channel after kexec() of Mini-OS, libxenevtchn needs to allocate the port data for the event channel and set the handler again. Add a new interface xenevtchn_bind() for that purpose, available under Mini-OS only.

#15 lore raw

PATCH 8/9 xenstored: rebind event channels after live update in stubdom

In reply to Juergen Gross

After performing a live update in stubdom environment some information about inter-domain event channels is lost. In order to set this information again, call xenevtchn_bind() from new_domain() in the restore case.

---
 tools/xenstored/core.c   | 1 -
 tools/xenstored/core.h   | 3 +++
 tools/xenstored/domain.c | 8 ++++++--
 tools/xenstored/minios.c | 5 +++++
 tools/xenstored/posix.c  | 5 +++++
 5 files changed, 19 insertions(+), 3 deletions(-)
tools/xenstored/core.c +0 -1
index 37e4dd5a5b..09c838d6e8 100644
@@ -49,7 +49,6 @@
 #include "control.h"
 #include "lu.h"
 
-extern xenevtchn_handle *xce_handle; /* in domain.c */
 static int xce_pollfd_idx = -1;
 struct pollfd *poll_fds;
 static unsigned int current_array_size;
tools/xenstored/core.h +3 -0
index bef24a688c..3f7426fbe3 100644
@@ -394,6 +394,8 @@ static inline bool domain_is_unprivileged(const struct connection *conn)
 	return conn && domid_is_unprivileged(conn->id);
 }
 
+extern xenevtchn_handle *xce_handle; /* in domain.c */
+
 /* Return the event channel used by xenbus. */
 evtchn_port_t get_xenbus_evtchn(void);
 void early_init(bool live_update, bool dofork, const char *pidfile);
@@ -407,6 +409,7 @@ int get_socket_fd(void);
 void set_socket_fd(int fd);
 
 xenevtchn_handle *evtchn_fdopen(int fd);
+int evtchn_rebind(int port);
 
 #ifdef __MINIOS__
 void mount_9pfs(void);
tools/xenstored/domain.c +6 -2
index e0c2342edb..ca0744d6bb 100644
@@ -855,9 +855,13 @@ static int new_domain(struct domain *domain, int port, bool restore)
 
 	wrl_domain_new(domain);
 
-	if (restore)
+	if (restore) {
+		if (evtchn_rebind(port)) {
+			errno = ENOMEM;
+			return errno;
+		}
 		domain->port = port;
-	else {
+	} else {
 		/* Tell kernel we're interested in this event. */
 		rc = xenevtchn_bind_interdomain(xce_handle, domain->domid,
 						port);
tools/xenstored/minios.c +5 -0
index aa1f03fd6b..ec5faebaf3 100644
@@ -90,6 +90,11 @@ xenevtchn_handle *evtchn_fdopen(int fd)
 	return xenevtchn_open(NULL, XENEVTCHN_NO_CLOEXEC);
 }
 
+int evtchn_rebind(int port)
+{
+	return xenevtchn_bind(xce_handle, port);
+}
+
 static void mount_thread(void *p)
 {
 	xenbus_event_queue events = NULL;
tools/xenstored/posix.c +5 -0
index ebdec82215..97561701ae 100644
@@ -413,6 +413,11 @@ xenevtchn_handle *evtchn_fdopen(int fd)
 	return xenevtchn_fdopen(NULL, fd, 0);
 }
 
+int evtchn_rebind(int port)
+{
+	return 0;
+}
+
 const char *xenstore_rundir(void)
 {
 	return xenstore_daemon_rundir();
Signature · 1 line
2.43.0
Reviewed-by on v1 8/9
#16 lore raw

Re: [PATCH 8/9] xenstored: rebind event channels after live update in stubdom

Quoted 6 lines · Juergen Gross

After performing a live update in stubdom environment some information about inter-domain event channels is lost. In order to set this information again, call xenevtchn_bind() from new_domain() in the restore case.

#17 lore raw

PATCH 9/9 tools/xenstored: make stubdom_init() live update aware

In reply to Juergen Gross

Pass the live_update bool to stubdom_init() in order to be able to handle it properly under Mini-OS.

As stubdom_init() will (re-)create the 9pfs device needed for reading the saved state, call stubdom_init() before lu_read_state().

---
 tools/xenstored/core.c   |  4 ++--
 tools/xenstored/core.h   |  2 +-
 tools/xenstored/domain.c | 15 +++++++++------
 tools/xenstored/domain.h |  2 +-
 tools/xenstored/minios.c | 12 +++++++++---
 5 files changed, 22 insertions(+), 13 deletions(-)
tools/xenstored/core.c +2 -2
index 09c838d6e8..18c5643029 100644
@@ -2770,14 +2770,14 @@ int main(int argc, char *argv[])
 	if (tracefile)
 		tracefile = absolute_filename(NULL, tracefile);
 
+	stubdom_init(live_update);
+
 #ifndef NO_LIVE_UPDATE
 	/* Read state in case of live update. */
 	if (live_update)
 		lu_read_state();
 #endif
 
-	stubdom_init();
-
 	check_store();
 
 	/* Get ready to listen to the tools. */
tools/xenstored/core.h +1 -1
index 3f7426fbe3..5071f1dedd 100644
@@ -412,7 +412,7 @@ xenevtchn_handle *evtchn_fdopen(int fd);
 int evtchn_rebind(int port);
 
 #ifdef __MINIOS__
-void mount_9pfs(void);
+void mount_9pfs(bool live_update);
 #endif
 
 const char *xenstore_rundir(void);
tools/xenstored/domain.c +9 -6
index ca0744d6bb..877a14e858 100644
@@ -1267,7 +1267,7 @@ void dom0_init(void)
 	xenevtchn_notify(xce_handle, dom0->port);
 }
 
-void stubdom_init(void)
+void stubdom_init(bool live_update)
 {
 #ifdef __MINIOS__
 	struct domain *stubdom;
@@ -1275,13 +1275,16 @@ void stubdom_init(void)
 	if (stub_domid < 0)
 		return;
 
-	stubdom = introduce_domain(NULL, stub_domid, xenbus_evtchn, false);
-	if (!stubdom)
-		barf_perror("Failed to initialize stubdom");
+	if (!live_update) {
+		stubdom = introduce_domain(NULL, stub_domid, xenbus_evtchn,
+					   false);
+		if (!stubdom)
+			barf_perror("Failed to initialize stubdom");
 
-	xenevtchn_notify(xce_handle, stubdom->port);
+		xenevtchn_notify(xce_handle, stubdom->port);
+	}
 
-	mount_9pfs();
+	mount_9pfs(live_update);
 #endif
 }
 
tools/xenstored/domain.h +1 -1
index 844ac11510..02b261ad47 100644
@@ -85,7 +85,7 @@ int do_reset_watches(const void *ctx, struct connection *conn,
 void domain_early_init(void);
 void domain_init(int evtfd);
 void dom0_init(void);
-void stubdom_init(void);
+void stubdom_init(bool live_update);
 void domain_deinit(void);
 void ignore_connection(struct connection *conn, unsigned int err);
 
tools/xenstored/minios.c +9 -3
index ec5faebaf3..f04423fe09 100644
@@ -131,15 +131,21 @@ static void mount_thread(void *p)
 		free(err);
 	}
 
-	p9_device = init_9pfront(0, XENSTORE_LIB_DIR);
+	p9_device = init_9pfront2(0, XENSTORE_LIB_DIR, INIT9P_FLAG_KEXEC);
 
 	/* Start logging if selected. */
 	reopen_log();
 }
 
-void mount_9pfs(void)
+void mount_9pfs(bool live_update)
 {
-	create_thread("mount-9pfs", mount_thread, NULL);
+	if (!live_update)
+		create_thread("mount-9pfs", mount_thread, NULL);
+	else {
+		p9_device = init_9pfront2(0, XENSTORE_LIB_DIR,
+					  INIT9P_FLAG_REINIT);
+		reopen_log();
+	}
 }
 
 const char *xenstore_rundir(void)
Signature · 1 line
2.43.0
Reviewed-by on v1 9/9
#18 lore raw

Re: [PATCH 9/9] tools/xenstored: make stubdom_init() live update aware

Quoted 7 lines · Juergen Gross

Pass the live_update bool to stubdom_init() in order to be able to handle it properly under Mini-OS.

As stubdom_init() will (re-)create the 9pfs device needed for reading the saved state, call stubdom_init() before lu_read_state().

#19 lore raw
In reply to Juergen Gross
Quoted 23 lines · Juergen Gross

With this series applied I was able to perform Live Update of Xenstore running in a PVH stubdom.

This series is based on a related Mini-OS patch series AND it needs the Mini-OS commit id updated accordingly in Config.mk.

One relevant change in this series is how the Xenstore state is being read after activating the new binary. Instead of using mmap() the state records are read one by one via fread(). This might have a small performance impact for the daemon live update case, but it should be neglectable.

Juergen Gross (9):
stubdom: configure kexec for PVH Xenstore stubdom tools/xenstore: perform kexec for stubdom live update
libevtchn: add O_CLOEXEC support for Mini-OS
libxengnttab: set the cloexec flag on Mini-OS
xenstored: use read() instead of mmap() for reading live update state
xenstored: don't use xenevtchn_fdopen() in stubdom tools/libxenevtchn: add xenevtchn_bind() under Mini-OS
xenstored: rebind event channels after live update in stubdom tools/xenstored: make stubdom_init() live update aware

Quoted 14 lines
  stubdom/xenstorepvh-minios.cfg |  2 +
  tools/include/xenevtchn.h      | 11 ++++
  tools/libs/evtchn/minios.c     | 23 +++++++--
  tools/libs/gnttab/minios.c     |  1 +
  tools/xenstored/core.c         |  5 +-
  tools/xenstored/core.h         | 13 ++++-
  tools/xenstored/domain.c       | 25 +++++----
  tools/xenstored/domain.h       |  2 +-
  tools/xenstored/lu.c           | 93 +++++++++++++++++++++-------------
  tools/xenstored/lu_minios.c    | 33 +++++++++++-
  tools/xenstored/minios.c       | 22 ++++++--
  tools/xenstored/posix.c        | 10 ++++
  12 files changed, 183 insertions(+), 57 deletions(-)

Could I get some feedback for this series, please?

Juergen

#20 lore raw
In reply to Juergen Gross
Quoted 5 lines · Juergen Gross

With this series applied I was able to perform Live Update of Xenstore running in a PVH stubdom.

This series is based on a related Mini-OS patch series AND it needs the Mini-OS commit id updated accordingly in Config.mk.

I was about to commit the first two patches here, but then went back to check here. Aiui such a patch wasn't sent yet, so I need to hold off ...

Jan

#21 lore raw
Quoted 9 lines · Jan Beulich
Quoted 5 lines · Juergen Gross

With this series applied I was able to perform Live Update of Xenstore running in a PVH stubdom.

This series is based on a related Mini-OS patch series AND it needs the Mini-OS commit id updated accordingly in Config.mk.

I was about to commit the first two patches here, but then went back to check here. Aiui such a patch wasn't sent yet, so I need to hold off ...

I was about to send that patch, now that the Mini-OS side has gone in.

Juergen

#22 lore raw
In reply to Juergen Gross

Anthony,

could you please have a look at patches 3, 4 and 7? They are lacking a tools maintainer's comment.

Juergen

Quoted 37 lines · Juergen Gross

With this series applied I was able to perform Live Update of Xenstore running in a PVH stubdom.

This series is based on a related Mini-OS patch series AND it needs the Mini-OS commit id updated accordingly in Config.mk.

One relevant change in this series is how the Xenstore state is being read after activating the new binary. Instead of using mmap() the state records are read one by one via fread(). This might have a small performance impact for the daemon live update case, but it should be neglectable.

Juergen Gross (9):
stubdom: configure kexec for PVH Xenstore stubdom tools/xenstore: perform kexec for stubdom live update
libevtchn: add O_CLOEXEC support for Mini-OS
libxengnttab: set the cloexec flag on Mini-OS
xenstored: use read() instead of mmap() for reading live update state
xenstored: don't use xenevtchn_fdopen() in stubdom tools/libxenevtchn: add xenevtchn_bind() under Mini-OS
xenstored: rebind event channels after live update in stubdom tools/xenstored: make stubdom_init() live update aware

  stubdom/xenstorepvh-minios.cfg |  2 +
  tools/include/xenevtchn.h      | 11 ++++
  tools/libs/evtchn/minios.c     | 23 +++++++--
  tools/libs/gnttab/minios.c     |  1 +
  tools/xenstored/core.c         |  5 +-
  tools/xenstored/core.h         | 13 ++++-
  tools/xenstored/domain.c       | 25 +++++----
  tools/xenstored/domain.h       |  2 +-
  tools/xenstored/lu.c           | 93 +++++++++++++++++++++-------------
  tools/xenstored/lu_minios.c    | 33 +++++++++++-
  tools/xenstored/minios.c       | 22 ++++++--
  tools/xenstored/posix.c        | 10 ++++
  12 files changed, 183 insertions(+), 57 deletions(-)