Skip to contentSkip to conversation

virtio-blk: schedule virtio_notify_config to run on main context

PATCH Superseded Sergio Lopez 3 participants 3 messages Sep 13, 2019 – Sep 16, 2019

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 virtio-blk: schedule virtio_notify_config to run on main context

virtio_notify_config() needs to acquire the global mutex, which isn't allowed from an iothread, and may lead to a deadlock like this:

- main thead
* Has acquired: qemu_global_mutex.
* Is trying the acquire: iothread AioContext lock via AIO_WAIT_WHILE (after aio_poll).

- iothread
* Has acquired: AioContext lock.
* Is trying to acquire: qemu_global_mutex (via

    virtio_notify_config->prepare_mmio_access).

If virtio_blk_resize() is called from an iothread, schedule virtio_notify_config() to be run in the main context BH.

---
Changelog

v2:
- Use aio_bh_schedule_oneshot instead of scheduling a coroutine (thanks Kevin Wolf).
- Switch from RFC to v2 patch.

---
 hw/block/virtio-blk.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
hw/block/virtio-blk.c +20 -1
index 18851601cb..669dc60f5b 100644
@@ -16,6 +16,7 @@
 #include "qemu/iov.h"
 #include "qemu/module.h"
 #include "qemu/error-report.h"
+#include "qemu/main-loop.h"
 #include "trace.h"
 #include "hw/block/block.h"
 #include "hw/qdev-properties.h"
@@ -1086,11 +1087,29 @@ static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f,
     return 0;
 }
 
+static void virtio_resize_cb(void *opaque)
+{
+    VirtIODevice *vdev = opaque;
+
+    assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+    virtio_notify_config(vdev);
+}
+
 static void virtio_blk_resize(void *opaque)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(opaque);
 
-    virtio_notify_config(vdev);
+    if (qemu_get_current_aio_context() != qemu_get_aio_context()) {
+        /*
+         * virtio_notify_config() needs to acquire the global mutex,
+         * so it can't be called from an iothread. Instead, schedule
+         * it to be run in the main context BH.
+         */
+        aio_bh_schedule_oneshot(qemu_get_aio_context(),
+                                virtio_resize_cb, vdev);
+    } else {
+        virtio_notify_config(vdev);
+    }
 }
 
 static const BlockDevOps virtio_block_ops = {
Signature · 1 line
2.21.0
Reviewed-by on v2 1/1
#1 lore raw
Quoted 62 lines · Sergio Lopez

virtio_notify_config() needs to acquire the global mutex, which isn't allowed from an iothread, and may lead to a deadlock like this:

- main thead
* Has acquired: qemu_global_mutex.
* Is trying the acquire: iothread AioContext lock via AIO_WAIT_WHILE (after aio_poll).

- iothread
* Has acquired: AioContext lock.
* Is trying to acquire: qemu_global_mutex (via

    virtio_notify_config->prepare_mmio_access).

If virtio_blk_resize() is called from an iothread, schedule virtio_notify_config() to be run in the main context BH.

---
Changelog

v2:
- Use aio_bh_schedule_oneshot instead of scheduling a coroutine (thanks Kevin Wolf).
- Switch from RFC to v2 patch.

---
 hw/block/virtio-blk.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
hw/block/virtio-blk.c +13 -1
index 18851601cb..669dc60f5b 100644
@@ -16,6 +16,7 @@
 #include "qemu/iov.h"
 #include "qemu/module.h"
 #include "qemu/error-report.h"
+#include "qemu/main-loop.h"
 #include "trace.h"
 #include "hw/block/block.h"
 #include "hw/qdev-properties.h"
@@ -1086,11 +1087,29 @@ static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f,
     return 0;
 }
 
+static void virtio_resize_cb(void *opaque)
+{
+    VirtIODevice *vdev = opaque;
+
+    assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+    virtio_notify_config(vdev);
+}
+
 static void virtio_blk_resize(void *opaque)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(opaque);
 
-    virtio_notify_config(vdev);
+    if (qemu_get_current_aio_context() != qemu_get_aio_context()) {
+        /*
+         * virtio_notify_config() needs to acquire the global mutex,
+         * so it can't be called from an iothread. Instead, schedule

hw/block/virtio-blk.c static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f, · line 1115

Quoted 10 lines
hw/block/virtio-blk.c +7 -0
+         * it to be run in the main context BH.
+         */
+        aio_bh_schedule_oneshot(qemu_get_aio_context(),
+                                virtio_resize_cb, vdev);
+    } else {
+        virtio_notify_config(vdev);
+    }
 }
 
 static const BlockDevOps virtio_block_ops = {

Do we need to bother to check our current context before firing off the oneshot? It's simpler to just fire off the oneshot. I can't imagine that resize is a hot path that needs to worry about being that efficient.

Well, I guess it doesn't hurt either.

#2 lore raw
In reply to Sergio Lopez

Am 13.09.2019 um 12:56 hat Sergio Lopez geschrieben:

Quoted 57 lines

virtio_notify_config() needs to acquire the global mutex, which isn't allowed from an iothread, and may lead to a deadlock like this:

- main thead
* Has acquired: qemu_global_mutex.
* Is trying the acquire: iothread AioContext lock via AIO_WAIT_WHILE (after aio_poll).

- iothread
* Has acquired: AioContext lock.
* Is trying to acquire: qemu_global_mutex (via

    virtio_notify_config->prepare_mmio_access).

If virtio_blk_resize() is called from an iothread, schedule virtio_notify_config() to be run in the main context BH.

---
Changelog

v2:
- Use aio_bh_schedule_oneshot instead of scheduling a coroutine (thanks Kevin Wolf).
- Switch from RFC to v2 patch.

---
 hw/block/virtio-blk.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
hw/block/virtio-blk.c +9 -1
index 18851601cb..669dc60f5b 100644
@@ -16,6 +16,7 @@
 #include "qemu/iov.h"
 #include "qemu/module.h"
 #include "qemu/error-report.h"
+#include "qemu/main-loop.h"
 #include "trace.h"
 #include "hw/block/block.h"
 #include "hw/qdev-properties.h"
@@ -1086,11 +1087,29 @@ static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f,
     return 0;
 }
 
+static void virtio_resize_cb(void *opaque)
+{
+    VirtIODevice *vdev = opaque;
+
+    assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+    virtio_notify_config(vdev);
+}
+
 static void virtio_blk_resize(void *opaque)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(opaque);
 
-    virtio_notify_config(vdev);

hw/block/virtio-blk.c static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f, · line 1111

Quoted 10 lines
hw/block/virtio-blk.c +10 -0
+    if (qemu_get_current_aio_context() != qemu_get_aio_context()) {
+        /*
+         * virtio_notify_config() needs to acquire the global mutex,
+         * so it can't be called from an iothread. Instead, schedule
+         * it to be run in the main context BH.
+         */
+        aio_bh_schedule_oneshot(qemu_get_aio_context(),
+                                virtio_resize_cb, vdev);
+    } else {
+        virtio_notify_config(vdev);

Let's call virtio_resize_cb() instead to keep both code paths the same. Otherwise, we might add more code to virtio_resize_cb() later and miss that it must be duplicated here.

Kevin