xen: fix a potential crash in xen-bus
cover xen: fix a potential crash in xen-bus
This series fixes a potential segfault caused by NotifierList corruption in xen-bus. The first two patches lay the groundwork and the third actually fixes the problem.
Paul Durrant (3):
xen / notify: introduce a new XenWatchList abstraction
xen: introduce separate XenWatchList for XenDevice objects
xen: perform XenDevice clean-up in XenBus watch handler
hw/xen/trace-events | 9 +- hw/xen/xen-bus.c | 266 ++++++++++++++++++++++++++++----------- include/hw/xen/xen-bus.h | 7 +- include/qemu/notify.h | 2 + util/notify.c | 5 + 5 files changed, 212 insertions(+), 77 deletions(-)
---
Cc: Anthony Perard <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/3 xen / notify: introduce a new XenWatchList abstraction
Xenstore watch call-backs are already abstracted away from XenBus using the XenWatch data structure but the associated NotifierList manipulation and file handle registation is still open coded in various xen_bus_...() functions.
This patch creates a new XenWatchList data structure to allow these interactions to be abstracted away from XenBus as well. This is in preparation for a subsequent patch which will introduce separate watch lists for XenBus and XenDevice objects.
NOTE: This patch also introduces a new NotifierListEmpty() helper function
for the purposes of adding an assertion that a XenWatchList is not freed whilst its associated NotifierList is still occupied.
---
Cc: Stefano Stabellini <[email protected]>
Cc: Anthony Perard <[email protected]>
--- hw/xen/trace-events | 5 +- hw/xen/xen-bus.c | 117 +++++++++++++++++++++++++-------------- include/hw/xen/xen-bus.h | 3 +- include/qemu/notify.h | 2 + util/notify.c | 5 ++ 5 files changed, 87 insertions(+), 45 deletions(-)
hw/xen/trace-events +2 -3
@@ -19,9 +19,8 @@ xen_bus_unrealize(void) "" xen_bus_enumerate(void) "" xen_bus_type_enumerate(const char *type) "type: %s" xen_bus_backend_create(const char *type, const char *path) "type: %s path: %s" -xen_bus_add_watch(const char *node, const char *key, char *token) "node: %s key: %s token: %s" -xen_bus_remove_watch(const char *node, const char *key, char *token) "node: %s key: %s token: %s" -xen_bus_watch(const char *token) "token: %s" +xen_bus_add_watch(const char *node, const char *key) "node: %s key: %s" +xen_bus_remove_watch(const char *node, const char *key) "node: %s key: %s" xen_device_realize(const char *type, char *name) "type: %s name: %s" xen_device_unrealize(const char *type, char *name) "type: %s name: %s" xen_device_backend_state(const char *type, char *name, const char *state) "type: %s name: %s -> %s"
hw/xen/xen-bus.c +76 -41
156 lines not on this page · raw message
include/hw/xen/xen-bus.h +2 -1
@@ -14,6 +14,7 @@typedef void (*XenWatchHandler )(void *opaque); +typedef struct XenWatchList XenWatchList ;typedef struct XenWatch XenWatch ;typedef struct XenEventChannel XenEventChannel ;
@@ -63,7 +64,7 @@ typedef struct XenBus {BusState qbus;domid_t backend_id;struct xs_handle *xsh; -NotifierList watch_notifiers; +XenWatchList *watch_list;XenWatch *backend_watch; } XenBus;
include/qemu/notify.h +2 -0
@@ -40,6 +40,8 @@ void notifier_remove(Notifier *notifier);void notifier_list_notify (NotifierList *list,void *data); +bool notifier_list_empty (NotifierList *list); +/* Same as Notifier but allows .notify() to return errors */ typedef struct NotifierWithReturn NotifierWithReturn ;
util/notify.c +5 -0
@@ -40,6 +40,11 @@ void notifier_list_notify(NotifierList *list, void *data) } } +bool notifier_list_empty (NotifierList *list) +{ +return QLIST_EMPTY (&list->notifiers); +} +void notifier_with_return_list_init (NotifierWithReturnList *list) {QLIST_INIT (&list->notifiers);
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Qemu-devel] [PATCH 1/3] xen / notify: introduce a new XenWatchList abstraction
Quoted 3 lines · Paul Durrant
Xenstore watch call-backs are already abstracted away from XenBus using the XenWatch data structure but the associated NotifierList manipulation and file handle registation is still open coded in various xen_bus_...()
^ registration
Quoted 7 lines
functions.
This patch creates a new XenWatchList data structure to allow these interactions to be abstracted away from XenBus as well. This is in preparation for a subsequent patch which will introduce separate watch lists for XenBus and XenDevice objects.NOTE: This patch also introduces a new
NotifierListEmpty()helper function
^ notifier_list_empty() ?
Quoted 4 lines
for the purposes of adding an assertion that a XenWatchList is not freed whilst its associated NotifierList is still occupied.
Signed-off-by Paul Durrant <[email protected]>
Signature · 6 lines
Anthony PERARD _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Qemu-devel] [PATCH 1/3] xen / notify: introduce a new XenWatchList abstraction
Quoted 7 lines
-----Original Message-----
From: Anthony PERARD <[email protected]>
Sent: 12 September 2019 11:17
To: Paul Durrant <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 1/3] xen / notify: introduce a new XenWatchList abstraction
Quoted 5 lines
Quoted 3 lines · Paul Durrant
Xenstore watch call-backs are already abstracted away from XenBus using the XenWatch data structure but the associated NotifierList manipulation and file handle registation is still open coded in various xen_bus_...()
^ registration
Ok.
Quoted 9 lines
Quoted 7 lines
functions.
This patch creates a new XenWatchList data structure to allow these interactions to be abstracted away from XenBus as well. This is in preparation for a subsequent patch which will introduce separate watch lists for XenBus and XenDevice objects.NOTE: This patch also introduces a new
NotifierListEmpty()helper function^
notifier_list_empty()?
Oops, yes :-)
Quoted 7 lines
Quoted 4 lines
for the purposes of adding an assertion that a XenWatchList is not freed whilst its associated NotifierList is still occupied.
Signed-off-by Paul Durrant <[email protected]>Reviewed-by Anthony PERARD <[email protected]>
Thanks,
Paul
Quoted 2 lines
--
Anthony PERARD
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/3 xen: introduce separate XenWatchList for XenDevice objects
In reply to Paul DurrantThis patch uses the XenWatchList abstraction to add a separate watch list for each device. This is more scalable than walking a single notifier list for all watches and is also necessary to implement a bug-fix in a subsequent patch.
---
Cc: Stefano Stabellini <[email protected]>
Cc: Anthony Perard <[email protected]>
--- hw/xen/trace-events | 2 ++ hw/xen/xen-bus.c | 72 ++++++++++++++++++++++++++++++++-------- include/hw/xen/xen-bus.h | 2 ++ 3 files changed, 62 insertions(+), 14 deletions(-)
hw/xen/trace-events +2 -0
@@ -29,6 +29,8 @@ xen_device_backend_changed(const char *type, char *name) "type: %s name: %s" xen_device_frontend_state(const char *type, char *name, const char *state) "type: %s name: %s -> %s" xen_device_frontend_changed(const char *type, char *name) "type: %s name: %s" xen_device_unplug(const char *type, char *name) "type: %s name: %s" +xen_device_add_watch(const char *type, char *name, const char *node, const char *key) "type: %s name: %s node: %s key: %s" +xen_device_remove_watch(const char *type, char *name, const char *node, const char *key) "type: %s name: %s node: %s key: %s" # xen-bus-helper.c xs_node_create(const char *node) "%s"
hw/xen/xen-bus.c +58 -14
139 lines not on this page · raw message
include/hw/xen/xen-bus.h +2 -0
@@ -22,6 +22,8 @@ typedef struct XenDevice {DeviceState qdev;domid_t frontend_id;char *name; +struct xs_handle *xsh; +XenWatchList *watch_list;char *backend_path, *frontend_path;enum xenbus_state backend_state, frontend_state;Notifier exit;
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Qemu-devel] [PATCH 2/3] xen: introduce separate XenWatchList for XenDevice objects
Quoted 6 lines · Paul Durrant
This patch uses the XenWatchList abstraction to add a separate watch list for each device. This is more scalable than walking a single notifier list for all watches and is also necessary to implement a bug-fix in a subsequent patch.
Signed-off-by Paul Durrant <[email protected]>
Signature · 6 lines
Anthony PERARD _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/3 xen: perform XenDevice clean-up in XenBus watch handler
In reply to Paul DurrantCleaning up offine XenDevice objects directly in xen_device_backend_changed() is dangerous as xen_device_unrealize() will modify the watch list that is being walked. Even the QLIST_FOREACH_SAFE() used in notifier_list_notify() is insufficient as *two* notifiers (for the frontend and backend watches) are removed, thus potentially rendering the 'next' pointer unsafe.
The solution is to use the XenBus backend_watch handler to do the clean-up instead, as it is invoked whilst walking a separate watch list.
This patch therefore adds a new 'offline_devices' list to XenBus, to which offline devices are added by xen_device_backend_changed(). The XenBus backend_watch registration is also changed to not only invoke xen_bus_enumerate() but also a new xen_bus_cleanup() function, which will walk 'offline_devices' and perform the necessary actions.
For safety a an extra 'online' check is also added to xen_bus_type_enumerate() to make sure that no attempt is made to create a new XenDevice object for a backend that is offline.
NOTE: This patch also include some cosmetic changes:
- substitute the local variable name 'backend_state'
in xen_bus_type_enumerate() with 'state', since there is no ambiguity with any other state in that context.
- change xen_device_state_is_active() to xen_device_frontend_is_active() (and pass a XenDevice directly) since the state tests contained therein only apply to a frontend.
- use 'state' rather then 'xendev->backend_state' in xen_device_backend_changed() to shorten the code.
---
Cc: Stefano Stabellini <[email protected]>
Cc: Anthony Perard <[email protected]>
--- hw/xen/trace-events | 2 + hw/xen/xen-bus.c | 91 +++++++++++++++++++++++++++++----------- include/hw/xen/xen-bus.h | 2 + 3 files changed, 70 insertions(+), 25 deletions(-)
hw/xen/trace-events +2 -0
@@ -17,8 +17,10 @@ xen_domid_restrict(int err) "err: %u" xen_bus_realize(void) "" xen_bus_unrealize(void) "" xen_bus_enumerate(void) "" +xen_bus_cleanup(void) "" xen_bus_type_enumerate(const char *type) "type: %s" xen_bus_backend_create(const char *type, const char *path) "type: %s path: %s" +xen_bus_device_cleanup(const char *type, char *name) "type: %s name: %s" xen_bus_add_watch(const char *node, const char *key) "node: %s key: %s" xen_bus_remove_watch(const char *node, const char *key) "node: %s key: %s" xen_device_realize(const char *type, char *name) "type: %s name: %s"
hw/xen/xen-bus.c +66 -25
140 lines not on this page · raw message
include/hw/xen/xen-bus.h +2 -0
@@ -33,6 +33,7 @@ typedef struct XenDevice {xengnttab_handle *xgth;bool feature_grant_copy; QLIST_HEAD(, XenEventChannel) event_channels; +QLIST_ENTRY (XenDevice) list; } XenDevice;typedef char *(*XenDeviceGetName )(XenDevice *xendev,Error **errp);
@@ -68,6 +69,7 @@ typedef struct XenBus {struct xs_handle *xsh;XenWatchList *watch_list;XenWatch *backend_watch; + QLIST_HEAD(, XenDevice) offline_devices; } XenBus; typedef struct XenBusClass {
Signature · 7 lines
2.20.1.2.gb21ebb6 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Qemu-devel] [PATCH 3/3] xen: perform XenDevice clean-up in XenBus watch handler
Quoted 190 lines
-----Original Message-----
From: Paul Durrant <[email protected]>
Sent: 11 September 2019 15:36
To: [email protected]; [email protected]
Cc: Paul Durrant <[email protected]>; Stefano Stabellini <[email protected]>; Anthony Perard <[email protected]>
Subject: [PATCH 3/3] xen: perform XenDevice clean-up in XenBus watch handlerCleaning up offine XenDevice objects directly in
xen_device_backend_changed()is dangerous asxen_device_unrealize()will modify the watch list that is being walked. Even theQLIST_FOREACH_SAFE()used innotifier_list_notify()is insufficient as *two* notifiers (for the frontend and backend watches) are removed, thus potentially rendering the 'next' pointer unsafe.The solution is to use the XenBus
backend_watchhandler to do the clean-up instead, as it is invoked whilst walking a separate watch list.This patch therefore adds a new '
offline_devices' list to XenBus, to which offline devices are added byxen_device_backend_changed(). The XenBusbackend_watchregistration is also changed to not only invokexen_bus_enumerate()but also a newxen_bus_cleanup()function, which will walk 'offline_devices' and perform the necessary actions.
For safety a an extra 'online' check is also added toxen_bus_type_enumerate()to make sure that no attempt is made to create a new XenDevice object for a backend that is offline.NOTE: This patch also include some cosmetic changes:
- substitute the local variable name 'backend_state'
inxen_bus_type_enumerate()with 'state', since there is no ambiguity with any other state in that context.
- changexen_device_state_is_active()toxen_device_frontend_is_active()(and pass a XenDevice directly) since the state tests contained therein only apply to a frontend.
- use 'state' rather then 'xendev->backend_state' inxen_device_backend_changed()to shorten the code.Signed-off-by Paul Durrant <[email protected]>---
Cc: Stefano Stabellini <[email protected]>
Cc: Anthony Perard <[email protected]>--- hw/xen/trace-events | 2 + hw/xen/xen-bus.c | 91 +++++++++++++++++++++++++++++----------- include/hw/xen/xen-bus.h | 2 + 3 files changed, 70 insertions(+), 25 deletions(-)hw/xen/trace-events +2
-0@@ -17,8 +17,10 @@ xen_domid_restrict(int err) "err: %u" xen_bus_realize(void) "" xen_bus_unrealize(void) "" xen_bus_enumerate(void) "" +xen_bus_cleanup(void) "" xen_bus_type_enumerate(const char *type) "type: %s" xen_bus_backend_create(const char *type, const char *path) "type: %s path: %s" +xen_bus_device_cleanup(const char *type, char *name) "type: %s name: %s" xen_bus_add_watch(const char *node, const char *key) "node: %s key: %s" xen_bus_remove_watch(const char *node, const char *key) "node: %s key: %s" xen_device_realize(const char *type, char *name) "type: %s name: %s"hw/xen/xen-bus.c +56
-18@@ -340,13 +340,18 @@ static void xen_bus_type_enumerate(XenBus *xenbus, const char *type)for (i =0 ; i < n; i++) {char *backend_path =g_strdup_printf ("%s/%s" , domain_path, backend[i]); -enum xenbus_state backend_state; +enum xenbus_state state; +unsigned int online;if (xs_node_scanf (xenbus->xsh,XBT_NULL , backend_path,"state" , -NULL ,"%u" , &backend_state) !=1 ) - backend_state = XenbusStateUnknown; +NULL ,"%u" , &state) !=1 ) + state = XenbusStateUnknown; -if (backend_state == XenbusStateInitialising) { +if (xs_node_scanf (xenbus->xsh,XBT_NULL , backend_path,"online" , +NULL ,"%u" , &online) !=1 ) + online =0 ; + +if (online && state == XenbusStateInitialising) {Error *local_err =NULL ; xen_bus_backend_create(xenbus, type, backend[i], backend_path,@@ -365,9 +370,8 @@ out:g_free (domain_path); } -static void xen_bus_enumerate (void *opaque) +static void xen_bus_enumerate (XenBus *xenbus) { -XenBus *xenbus = opaque;char **type;unsigned int i, n;@@ -385,6 +389,44 @@ static void xen_bus_enumerate(void *opaque)free (type); } +static void xen_bus_device_cleanup (XenDevice *xendev) +{ +const char *type =object_get_typename (OBJECT (xendev)); +Error *local_err =NULL ; + +trace_xen_bus_device_cleanup (type, xendev->name); + +g_assert (!xendev->backend_online); + +if (!xen_backend_try_device_destroy (xendev, &local_err)) { +object_unparent (OBJECT (xendev)); + } + +if (local_err) { +error_report_err (local_err); + } +} + +static void xen_bus_cleanup (XenBus *xenbus) +{ +XenDevice *xendev, *next; + +trace_xen_bus_cleanup (); + + QLIST_FOREACH_SAFE(xendev, &xenbus->offline_devices, list, next) { +QLIST_REMOVE (xendev, list); +xen_bus_device_cleanup (xendev); + } +} + +static void xen_bus_backend_changed (void *opaque) +{ +XenBus *xenbus = opaque; + +xen_bus_enumerate (xenbus); +xen_bus_cleanup (xenbus); +} +static void xen_bus_unrealize (BusState *bus,Error **errp) {XenBus *xenbus =XEN_BUS (bus);@@ -433,7 +475,7 @@ static void xen_bus_realize(BusState *bus, Error **errp) xenbus->backend_watch =xen_bus_add_watch (xenbus,"" ,/* domain root node */ -"backend" , xen_bus_enumerate, &local_err); +"backend" , xen_bus_backend_changed, &local_err);if (local_err) {/* This need not be treated as a hard error so don't propagate */ error_reportf_err(local_err,@@ -555,9 +597,9 @@ static void xen_device_backend_set_online(XenDevice *xendev, bool online)* Tell from the state whether the frontend is likely alive, * i.e. it will react to a change of state of the backend. */ -static bool xen_device_state_is_active (enum xenbus_state state) +static bool xen_device_frontend_is_active (XenDevice *xendev) { -switch (state) { +switch (xendev->frontend_state) {case XenbusStateInitWait:case XenbusStateInitialised:case XenbusStateConnected:@@ -594,30 +636,29 @@ static void xen_device_backend_changed(void *opaque)* state to Closing, but there is no active frontend then set the * backend state to Closed. */ -if (xendev->backend_state == XenbusStateClosing && - !xen_device_state_is_active (xendev->frontend_state)) { +if (state == XenbusStateClosing && + !xen_device_frontend_is_active (xendev)) {xen_device_backend_set_state (xendev, XenbusStateClosed); }/* * If a backend is still 'online' then we should leave it alone but, -* if a backend is not 'online', then the device should be destroyed -* once the state is Closed. +* if a backend is not 'online', then the device is a candidate +* for destruction. Hence add it to the 'offline' list to be cleaned +* by xen_bus_cleanup(). */ -if (!xendev->backend_online && - (xendev->backend_state == XenbusStateClosed || - xendev->backend_state == XenbusStateInitialising || - xendev->backend_state == XenbusStateInitWait || - xendev->backend_state == XenbusStateUnknown)) {
hw/xen/xen-bus.c static void xen_device_backend_changed(void *opaque) · line 655
Quoted 10 lines
hw/xen/xen-bus.c +5
-4-Error *local_err =NULL ; +if (!online && + (state == XenbusStateClosed || state == XenbusStateInitialising || + state == XenbusStateInitWait || state == XenbusStateUnknown)) { +XenBus *xenbus =XEN_BUS (qdev_get_parent_bus (DEVICE (xendev))); -if (!xen_backend_try_device_destroy (xendev, &local_err)) { -object_unparent (OBJECT (xendev)); - } +QLIST_INSERT_HEAD (&xenbus->offline_devices, xendev, list);
I realize now that I should have a check here to make sure the above does not happen twice. I'll send a v2.
Paul
Quoted 33 lines
hw/xen/xen-bus.c +5
-3-if (local_err) { -error_report_err (local_err); - } +/* +* Re-write the state to cause a XenBus backend_watch notification, +* resulting in a call to xen_bus_cleanup(). +*/ +xen_device_backend_printf (xendev,"state" ,"%u" , state); } }include/hw/xen/xen-bus.h +2
-1@@ -33,6 +33,7 @@ typedef struct XenDevice {xengnttab_handle *xgth;bool feature_grant_copy; QLIST_HEAD(, XenEventChannel) event_channels; +QLIST_ENTRY (XenDevice) list; } XenDevice;typedef char *(*XenDeviceGetName )(XenDevice *xendev,Error **errp);@@ -68,6 +69,7 @@ typedef struct XenBus {struct xs_handle *xsh;XenWatchList *watch_list;XenWatch *backend_watch; + QLIST_HEAD(, XenDevice) offline_devices; } XenBus; typedef struct XenBusClass { --
Quoted 1 line
2.20.1.2.gb21ebb6
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel