Skip to contentSkip to conversation

Cleanup IOREQ server on exit

PATCH In review Maximilian Heyne 3 participants 11 messages Mar 13, 2020 – Apr 24, 2020 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Cleanup IOREQ server on exit

Following up on commit 9c0eed61 ("qemu-trad: stop using the default IOREQ server"), clean up the IOREQ server on exit. This fixes a bug with soft-reset that shows up as "bind interdomain ioctl error 22" because the event channels were not closed at the soft-reset and can't be bound again.

For this I used the exit notifiers from QEMU that I backported together with the required generic notifier lists.

Anthony Liguori (1):
Add support for generic notifier lists

Gerd Hoffmann (1):
Add exit notifiers.

Maximilian Heyne (1):
xen: cleanup IOREQ server on exit

 Makefile            |  1 +
 hw/xen_machine_fv.c | 11 +++++++++++
 notify.c            | 39 +++++++++++++++++++++++++++++++++++++++
 notify.h            | 43 +++++++++++++++++++++++++++++++++++++++++++
 sys-queue.h         |  5 +++++
 sysemu.h            |  5 +++++
 vl.c                | 20 ++++++++++++++++++++
 7 files changed, 124 insertions(+)
 create mode 100644 notify.c
 create mode 100644 notify.h
Signature · 20 lines
2.16.6




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#5 lore raw

PATCH 3/3 xen: cleanup IOREQ server on exit

Use the backported Notifier interface to register an atexit handler to cleanup the IOREQ server. This is required since Xen commit a5a180f9 ("x86/domain: don't destroy IOREQ servers on soft reset") is introduced which requires Qemu to explicitly close the IOREQ server.

This is can be seen as a backport of ba7fdd64 ("xen: cleanup IOREQ server on exit").

---
 hw/xen_machine_fv.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
hw/xen_machine_fv.c +11 -0
index f0989fad4..66eb4a1eb 100644
@@ -31,6 +31,7 @@
 #include "qemu-aio.h"
 #include "xen_backend.h"
 #include "pci.h"
+#include "sysemu.h"
 
 #include <xen/hvm/params.h>
 #include <sys/mman.h>
@@ -67,6 +68,8 @@ TAILQ_HEAD(map_cache_head, map_cache_rev) locked_entries = TAILQ_HEAD_INITIALIZE
 static unsigned long last_address_page = ~0UL;
 static uint8_t      *last_address_vaddr;
 
+static Notifier exit_notifier;
+
 static int qemu_map_cache_init(void)
 {
     unsigned long size;
@@ -283,6 +286,11 @@ void xen_disable_io(void)
     xc_hvm_set_ioreq_server_state(xc_handle, domid, ioservid, 0);
 }
 
+static void xen_exit_notifier(Notifier *n)
+{
+    xc_hvm_destroy_ioreq_server(xc_handle, domid, ioservid);
+}
+
 static void xen_init_fv(ram_addr_t ram_size, int vga_ram_size,
 			const char *boot_device,
 			const char *kernel_filename,const char *kernel_cmdline,
@@ -317,6 +325,9 @@ static void xen_init_fv(ram_addr_t ram_size, int vga_ram_size,
         exit(-1);
     }
 
+    exit_notifier.notify = xen_exit_notifier;
+    qemu_add_exit_notifier(&exit_notifier);
+
     if (xc_hvm_get_ioreq_server_info(xc_handle, domid, ioservid,
                                      &ioreq_pfn, &bufioreq_pfn,
                                      &bufioreq_evtchn)) {
Signature · 20 lines
2.16.6




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH 2/3 Add exit notifiers.

In reply to Maximilian Heyne

From: Gerd Hoffmann <[email protected]>

Hook up any cleanup work which needs to be done here. Advantages over using atexit(3):

(1) You get passed in a pointer to the notifier. If you embed that into your state struct you can use container_of() to get get your state info.
(2) You can unregister, say when un-plugging a device.

[ v2: move code out of #ifndef _WIN32 ]

(cherry picked from commit fd42deeb4cb42f90084046e3ebdb4383953195e3)
---
 sysemu.h |  5 +++++
 vl.c     | 20 ++++++++++++++++++++
 2 files changed, 25 insertions(+)
sysemu.h +5 -0
index 968258a84..759d0e9d5 100644
@@ -2,6 +2,8 @@
 #define SYSEMU_H
 /* Misc. things related to the system emulator.  */
 
+#include "notify.h"
+
 /* vl.c */
 extern const char *bios_name;
 extern const char *bios_dir;
@@ -39,6 +41,9 @@ void qemu_system_powerdown(void);
 #endif
 void qemu_system_reset(void);
 
+void qemu_add_exit_notifier(Notifier *notify);
+void qemu_remove_exit_notifier(Notifier *notify);
+
 void do_savevm(const char *name);
 void do_loadvm(const char *name);
 void do_delvm(const char *name);
vl.c +20 -0
index c3c5d630e..2163217ec 100644
@@ -282,6 +282,9 @@ uint8_t qemu_uuid[16];
 
 #include "xen-vl-extra.c"
 
+static NotifierList exit_notifiers =
+    NOTIFIER_LIST_INITIALIZER(exit_notifiers);
+
 /***********************************************************/
 /* x86 ISA bus support */
 
@@ -4843,6 +4846,21 @@ static void vcpu_hex_str_to_bitmap(const char *optarg)
     }
 }
 
+void qemu_add_exit_notifier(Notifier *notify)
+{
+    notifier_list_add(&exit_notifiers, notify);
+}
+
+void qemu_remove_exit_notifier(Notifier *notify)
+{
+    notifier_list_remove(notify);
+}
+
+static void qemu_run_exit_notifiers(void)
+{
+    notifier_list_notify(&exit_notifiers);
+}
+
 int main(int argc, char **argv, char **envp)
 {
 #ifdef CONFIG_GDBSTUB
@@ -4887,6 +4905,8 @@ int main(int argc, char **argv, char **envp)
     const char *chroot_dir = NULL;
     const char *run_as = NULL;
 
+    atexit(qemu_run_exit_notifiers);
+
     qemu_cache_utils_init(envp);
     logfile = stderr; /* initial value */
 
Signature · 20 lines
2.16.6




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH 1/3 Add support for generic notifier lists

In reply to Maximilian Heyne

From: Anthony Liguori <[email protected]>

Notifiers are data-less callbacks and a notifier list is a list of registered notifiers that all are interested in a particular event.

We'll use this in a few patches to implement mouse change notification.

---
v1 -> v2
- Do not do memory allocations by placing list nodes in notifier

[cherry-picked from d1e70c5e6d1472856c52969301247fe8c3c8389d
conflicts: used the sys-qeue interface and added required LIST_REMOVE_SAFE function to that]

---
 Makefile    |  1 +
 notify.c    | 39 +++++++++++++++++++++++++++++++++++++++
 notify.h    | 43 +++++++++++++++++++++++++++++++++++++++++++
 sys-queue.h |  5 +++++
 4 files changed, 88 insertions(+)
 create mode 100644 notify.c
 create mode 100644 notify.h
Makefile +1 -0
index 0fbec990b..d921bcdf8 100644
@@ -93,6 +93,7 @@ OBJS+=sd.o ssi-sd.o
 OBJS+=bt.o bt-host.o bt-vhci.o bt-l2cap.o bt-sdp.o bt-hci.o bt-hid.o usb-bt.o
 OBJS+=buffered_file.o migration.o migration-tcp.o net.o qemu-sockets.o
 OBJS+=qemu-char.o aio.o net-checksum.o savevm.o cache-utils.o
+OBJS+=notify.o
 
 ifdef CONFIG_BRLAPI
 OBJS+= baum.o
notify.c new file +39 -0
new file mode 100644
index 000000000..59e1e7c7d
@@ -0,0 +1,39 @@
+/*
+ * Notifier lists
+ *
+ * Copyright IBM, Corp. 2010
+ *
+ * Authors:
+ *  Anthony Liguori   <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "notify.h"
+
+void notifier_list_init(NotifierList *list)
+{
+    LIST_INIT(&list->notifiers);
+}
+
+void notifier_list_add(NotifierList *list, Notifier *notifier)
+{
+    LIST_INSERT_HEAD(&list->notifiers, notifier, node);
+}
+
+void notifier_list_remove(Notifier *notifier)
+{
+    LIST_REMOVE(notifier, node);
+}
+
+void notifier_list_notify(NotifierList *list)
+{
+    Notifier *notifier, *next;
+
+    LIST_FOREACH_SAFE(notifier, &list->notifiers, node, next) {
+        notifier->notify(notifier);
+    }
+}
notify.h new file +43 -0

43 lines not on this page · raw message

sys-queue.h +5 -0
index 55c26fe7f..81ab044a8 100644
@@ -132,6 +132,11 @@ struct {                                                                \
                 (var);                                                  \
                 (var) = ((var)->field.le_next))
 
+#define LIST_FOREACH_SAFE(var, head, field, next_var)                   \
+        for ((var) = ((head)->lh_first);                                \
+                (var) && ((next_var) = ((var)->field.le_next), 1);      \
+                (var) = (next_var))
+
 /*
  * List access methods.
  */
Signature · 20 lines
2.16.6




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#7 lore raw
In reply to Maximilian Heyne

Could someone please have a look at this patch? It solves an actual issue:
Try soft-reset with qemu-xen-traditional and it will fail.

Quoted 25 lines · Maximilian Heyne

Following up on commit 9c0eed61 ("qemu-trad: stop using the default IOREQ server"), clean up the IOREQ server on exit. This fixes a bug with soft-reset that shows up as "bind interdomain ioctl error 22" because the event channels were not closed at the soft-reset and can't be bound again.

For this I used the exit notifiers from QEMU that I backported together with the required generic notifier lists.

Anthony Liguori (1):
Add support for generic notifier lists

Gerd Hoffmann (1):
Add exit notifiers.

Maximilian Heyne (1):
xen: cleanup IOREQ server on exit

  Makefile            |  1 +
  hw/xen_machine_fv.c | 11 +++++++++++
  notify.c            | 39 +++++++++++++++++++++++++++++++++++++++
  notify.h            | 43 +++++++++++++++++++++++++++++++++++++++++++
  sys-queue.h         |  5 +++++
  sysemu.h            |  5 +++++
  vl.c                | 20 ++++++++++++++++++++
  7 files changed, 124 insertions(+)
Quoted 3 lines

create mode 100644 notify.c
create mode 100644 notify.h

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

#8 lore raw
Quoted 10 lines

-----Original Message-----
From: Maximilian Heyne <[email protected]>
Sent: 07 April 2020 10:16
To: [email protected]
Cc: Ian Jackson <[email protected]>; Paul Durrant <[email protected]>
Subject: Re: [PATCH 0/3] Cleanup IOREQ server on exit

Could someone please have a look at this patch? It solves an actual issue:
Try soft-reset with qemu-xen-traditional and it will fail.

I'll take a look today.

Ian, obviously we know that qemu trad is largely dead but this series does address a serious shortcoming. Could you take a look?

Paul

Reviewed-by on v1 3/3
#6 lore raw

RE: [PATCH 3/3] xen: cleanup IOREQ server on exit

In reply to Maximilian Heyne
Quoted 12 lines

-----Original Message-----
From: Maximilian Heyne <[email protected]>
Sent: 13 March 2020 12:33
To: [email protected]
Cc: Ian Jackson <[email protected]>; Paul Durrant <[email protected]>
Subject: [PATCH 3/3] xen: cleanup IOREQ server on exit

Use the backported Notifier interface to register an atexit handler to cleanup the IOREQ server. This is required since Xen commit a5a180f9 ("x86/domain: don't destroy IOREQ servers on soft reset") is introduced which requires Qemu to explicitly close the IOREQ server.

Quoted 4 lines

This is can be seen as a backport of ba7fdd64 ("xen: cleanup IOREQ server on exit").

Reviewed-by on v1 2/3
#4 lore raw

RE: [PATCH 2/3] Add exit notifiers.

In reply to Maximilian Heyne
Quoted 17 lines

-----Original Message-----
From: Maximilian Heyne <[email protected]>
Sent: 13 March 2020 12:33
To: [email protected]
Cc: Ian Jackson <[email protected]>; Paul Durrant <[email protected]>
Subject: [PATCH 2/3] Add exit notifiers.

From: Gerd Hoffmann <[email protected]>

Hook up any cleanup work which needs to be done here. Advantages over using atexit(3):

(1) You get passed in a pointer to the notifier. If you embed that into your state struct you can use container_of() to get get your state info.
(2) You can unregister, say when un-plugging a device.

Quoted 5 lines

[ v2: move code out of #ifndef _WIN32 ]

(cherry picked from commit fd42deeb4cb42f90084046e3ebdb4383953195e3)
Reviewed-by on v1 1/3
#2 lore raw

RE: [PATCH 1/3] Add support for generic notifier lists

In reply to Maximilian Heyne
Quoted 15 lines

-----Original Message-----
From: Maximilian Heyne <[email protected]>
Sent: 13 March 2020 12:33
To: [email protected]
Cc: Ian Jackson <[email protected]>; Paul Durrant <[email protected]>
Subject: [PATCH 1/3] Add support for generic notifier lists

From: Anthony Liguori <[email protected]>

Notifiers are data-less callbacks and a notifier list is a list of registered notifiers that all are interested in a particular event.

We'll use this in a few patches to implement mouse change notification.

Quoted 8 lines

---
v1 -> v2
- Do not do memory allocations by placing list nodes in notifier

[cherry-picked from d1e70c5e6d1472856c52969301247fe8c3c8389d
conflicts: used the sys-qeue interface and added required LIST_REMOVE_SAFE function to that]

#9 lore raw
In reply to Maximilian Heyne

Ping v2?

Quoted 50 lines

-----Original Message-----
From: Maximilian Heyne <[email protected]>
Sent: 07 April 2020 10:16
To: [email protected]
Cc: Ian Jackson <[email protected]>; Paul Durrant <[email protected]>
Subject: Re: [PATCH 0/3] Cleanup IOREQ server on exit

Could someone please have a look at this patch? It solves an actual issue:
Try soft-reset with qemu-xen-traditional and it will fail.

Quoted 25 lines · Maximilian Heyne

Following up on commit 9c0eed61 ("qemu-trad: stop using the default IOREQ server"), clean up the IOREQ server on exit. This fixes a bug with soft-reset that shows up as "bind interdomain ioctl error 22" because the event channels were not closed at the soft-reset and can't be bound again.

For this I used the exit notifiers from QEMU that I backported together with the required generic notifier lists.

Anthony Liguori (1):
Add support for generic notifier lists

Gerd Hoffmann (1):
Add exit notifiers.

Maximilian Heyne (1):
xen: cleanup IOREQ server on exit

  Makefile            |  1 +
  hw/xen_machine_fv.c | 11 +++++++++++
  notify.c            | 39 +++++++++++++++++++++++++++++++++++++++
  notify.h            | 43 +++++++++++++++++++++++++++++++++++++++++++
  sys-queue.h         |  5 +++++
  sysemu.h            |  5 +++++
  vl.c                | 20 ++++++++++++++++++++
  7 files changed, 124 insertions(+)
Quoted 3 lines

create mode 100644 notify.c
create mode 100644 notify.h

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

Acked-by
#10 lore raw
In reply to Maximilian Heyne

Maximilian Heyne writes ("Re: [PATCH 0/3] Cleanup IOREQ server on exit"):

Quoted 2 lines

Could someone please have a look at this patch? It solves an actual issue:
Try soft-reset with qemu-xen-traditional and it will fail.

Thanks. I reviewed this.

qemu is in deep freeze but the changes looked correct and are indeed solving a regression. I convinced myself that they were appropriately low risk, so

for all three and I have pushed them.

In theory a backport might be appropriate since this is a bugfix but my inclination is to leave existing releases where they are, since anyone using qemu-trad probably wants things super-stable. Contrary opinions welcome.

It has been a very long time since I did an update of qemu trad so it is possible that I have mangled the process somehow. We will see I guess...

Thanks also to Paul for chasing me about this.

Regards,
Ian.