Skip to contentSkip to conversation

monitor: Optionally run handlers in coroutines

PATCH Superseded Kevin Wolf 5 participants 62 messages Sep 9, 2020 – Oct 2, 2020

Subsystems (3)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

v7 cover monitor: Optionally run handlers in coroutines

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

v7:
- Added patch 2 to add a Monitor parameter to monitor_get_cpu_index(), too [Markus]
- Let monitor_set_cur() return the old monitor [Markus]
- Fixed comment about linking stub objects in test-util-sockets [Markus]
- More detailed commit message for per-coroutine current monitor and document that monitor_set_cur(NULL) must be called eventually [Markus]
- Resolve some merge conflicts on rebase

v6:
- Fixed cur_mon behaviour: It should always point to the Monitor while we're in the handler coroutine, but be NULL while the handler coroutines has yielded. [Markus]
- Give HMP handlers the coroutine option, too, because they will call QMP handlers, and life is easier when we know whether we are in coroutine context or not.
- Fixed block_resize for block devices in iothreads and for HMP
- Resolved some merge conflict with QAPI generator and monitor refactorings that were merged in the meantime

v5:
- Improved comments and documentation [Markus]

v4:
- Forbid 'oob': true, 'coroutine': true [Markus]
- Removed Python type hints [Markus]
- Introduced separate bool qmp_dispatcher_co_shutdown to make it clearer how a shutdown request is signalled to the dispatcher [Markus]
- Allow using aio_poll() with iohandler_ctx and use that instead of aio_bh_poll() [Markus]
- Removed coroutine_fn from qmp_block_resize() again because at least one caller (HMP) calls it outside of coroutine context [Markus]
- Tried to make the synchronisation between dispatcher and the monitor thread clearer, and fixed a race condition
- Improved documentation and comments

v3:
- Fix race between monitor thread and dispatcher that could schedule the dispatcher coroutine twice if a second requests comes in before the dispatcher can wake up [Patchew]

v2:
- Fix typo in a commit message [Eric]
- Use hyphen instead of underscore for the test command [Eric]
- Mark qmp_block_resize() as coroutine_fn [Stefan]

Kevin Wolf (13):
monitor: Add Monitor parameter to monitor_set_cpu()
monitor: Add Monitor parameter to monitor_get_cpu_index()
monitor: Use getter/setter functions for cur_mon
hmp: Update current monitor only in handle_hmp_command()
qmp: Assert that no other monitor is active
qmp: Call monitor_set_cur() only in qmp_dispatch()
monitor: Make current monitor a per-coroutine property
qapi: Add a 'coroutine' flag for commands
qmp: Move dispatcher to a coroutine
hmp: Add support for coroutine command handlers util/async: Add aio_co_reschedule_self()
block: Add bdrv_co_move_to_aio_context()
block: Convert 'block_resize' to coroutine

 qapi/block-core.json                    |   3 +-
 docs/devel/qapi-code-gen.txt            |  12 +++
 include/block/aio.h                     |  10 ++
 include/block/block.h                   |   6 ++
 include/monitor/monitor.h               |   7 +-
 include/qapi/qmp/dispatch.h             |   5 +-
 monitor/monitor-internal.h              |   7 +-
 audio/wavcapture.c                      |   8 +-
 block.c                                 |  10 ++
 blockdev.c                              |  13 ++-
 dump/dump.c                             |   2 +-
 hw/core/machine-hmp-cmds.c              |   2 +-
 hw/scsi/vhost-scsi.c                    |   2 +-
 hw/virtio/vhost-vsock.c                 |   2 +-
 migration/fd.c                          |   4 +-
 monitor/hmp-cmds.c                      |   4 +-
 monitor/hmp.c                           |  44 ++++++--
 monitor/misc.c                          |  38 ++++---
 monitor/monitor.c                       | 101 ++++++++++++++++--
 monitor/qmp-cmds-control.c              |   2 +
 monitor/qmp-cmds.c                      |   2 +-
 monitor/qmp.c                           | 130 +++++++++++++++++-------
 net/socket.c                            |   2 +-
 net/tap.c                               |   6 +-
 qapi/qmp-dispatch.c                     |  61 ++++++++++-
 qapi/qmp-registry.c                     |   3 +
 qga/main.c                              |   2 +-
 softmmu/cpus.c                          |   2 +-
 stubs/monitor-core.c                    |  10 +-
 tests/test-qmp-cmds.c                   |  10 +-
 tests/test-util-sockets.c               |  12 +--
 trace/control.c                         |   2 +-
 util/aio-posix.c                        |   8 +-
 util/async.c                            |  30 ++++++
 util/qemu-error.c                       |   6 +-
 util/qemu-print.c                       |   3 +-
 util/qemu-sockets.c                     |   1 +
 scripts/qapi/commands.py                |  10 +-
 scripts/qapi/doc.py                     |   2 +-
 scripts/qapi/expr.py                    |  10 +-
 scripts/qapi/introspect.py              |   2 +-
 scripts/qapi/schema.py                  |  12 ++-
 tests/qapi-schema/test-qapi.py          |   7 +-
 hmp-commands.hx                         |   1 +
 tests/qapi-schema/meson.build           |   1 +
 tests/qapi-schema/oob-coroutine.err     |   2 +
 tests/qapi-schema/oob-coroutine.json    |   2 +
 tests/qapi-schema/oob-coroutine.out     |   0
 tests/qapi-schema/qapi-schema-test.json |   1 +
 tests/qapi-schema/qapi-schema-test.out  |   2 +
 50 files changed, 481 insertions(+), 143 deletions(-)
 create mode 100644 tests/qapi-schema/oob-coroutine.err
 create mode 100644 tests/qapi-schema/oob-coroutine.json
 create mode 100644 tests/qapi-schema/oob-coroutine.out
Signature · 1 line
2.25.4
author
#1 lore raw

PATCH v7 1/13 monitor: Add Monitor parameter to monitor_set_cpu()

Most callers actually don't have to rely on cur_mon, but already know for which monitor they call monitor_set_cpu().

---
 include/monitor/monitor.h |  2 +-
 monitor/hmp-cmds.c        |  2 +-
 monitor/misc.c            | 10 +++++-----
 3 files changed, 7 insertions(+), 7 deletions(-)
include/monitor/monitor.h +1 -1
index 1018d754a6..0dcaefd4f9 100644
@@ -33,7 +33,7 @@ int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
     GCC_FMT_ATTR(2, 0);
 int monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
 void monitor_flush(Monitor *mon);
-int monitor_set_cpu(int cpu_index);
+int monitor_set_cpu(Monitor *mon, int cpu_index);
 int monitor_get_cpu_index(void);
 
 void monitor_read_command(MonitorHMP *mon, int show_prompt);
monitor/hmp-cmds.c +1 -1
index 7711726fd2..f608b5b27b 100644
@@ -996,7 +996,7 @@ void hmp_cpu(Monitor *mon, const QDict *qdict)
     /* XXX: drop the monitor_set_cpu() usage when all HMP commands that
             use it are converted to the QAPI */
     cpu_index = qdict_get_int(qdict, "index");
-    if (monitor_set_cpu(cpu_index) < 0) {
+    if (monitor_set_cpu(mon, cpu_index) < 0) {
         monitor_printf(mon, "invalid CPU index\n");
     }
 }
monitor/misc.c +5 -5
index e847b58a8c..b4f779b8d3 100644
@@ -129,7 +129,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
     cur_mon = &hmp.common;
 
     if (has_cpu_index) {
-        int ret = monitor_set_cpu(cpu_index);
+        int ret = monitor_set_cpu(&hmp.common, cpu_index);
         if (ret < 0) {
             cur_mon = old_mon;
             error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
@@ -255,7 +255,7 @@ static void monitor_init_qmp_commands(void)
 }
 
 /* Set the current CPU defined by the user. Callers must hold BQL. */
-int monitor_set_cpu(int cpu_index)
+int monitor_set_cpu(Monitor *mon, int cpu_index)
 {
     CPUState *cpu;
 
@@ -263,8 +263,8 @@ int monitor_set_cpu(int cpu_index)
     if (cpu == NULL) {
         return -1;
     }
-    g_free(cur_mon->mon_cpu_path);
-    cur_mon->mon_cpu_path = object_get_canonical_path(OBJECT(cpu));
+    g_free(mon->mon_cpu_path);
+    mon->mon_cpu_path = object_get_canonical_path(OBJECT(cpu));
     return 0;
 }
 
@@ -285,7 +285,7 @@ static CPUState *mon_get_cpu_sync(bool synchronize)
         if (!first_cpu) {
             return NULL;
         }
-        monitor_set_cpu(first_cpu->cpu_index);
+        monitor_set_cpu(cur_mon, first_cpu->cpu_index);
         cpu = first_cpu;
     }
     assert(cpu != NULL);
Signature · 1 line
2.25.4
author
#2 lore raw

PATCH v7 2/13 monitor: Add Monitor parameter to monitor_get_cpu_index()

In reply to Kevin Wolf

Most callers actually don't have to rely on cur_mon, but already know for which monitor they call monitor_get_cpu_index().

---
 include/monitor/monitor.h  |  2 +-
 hw/core/machine-hmp-cmds.c |  2 +-
 monitor/hmp-cmds.c         |  2 +-
 monitor/misc.c             | 20 ++++++++++----------
 softmmu/cpus.c             |  2 +-
 5 files changed, 14 insertions(+), 14 deletions(-)
include/monitor/monitor.h +1 -1
index 0dcaefd4f9..a64502fbb2 100644
@@ -34,7 +34,7 @@ int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
 int monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
 void monitor_flush(Monitor *mon);
 int monitor_set_cpu(Monitor *mon, int cpu_index);
-int monitor_get_cpu_index(void);
+int monitor_get_cpu_index(Monitor *mon);
 
 void monitor_read_command(MonitorHMP *mon, int show_prompt);
 int monitor_read_password(MonitorHMP *mon, ReadLineFunc *readline_func,
hw/core/machine-hmp-cmds.c +1 -1
index 39999c47c5..da7d2e27b6 100644
@@ -34,7 +34,7 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)
     for (cpu = cpu_list; cpu; cpu = cpu->next) {
         int active = ' ';
 
-        if (cpu->value->cpu_index == monitor_get_cpu_index()) {
+        if (cpu->value->cpu_index == monitor_get_cpu_index(mon)) {
             active = '*';
         }
 
monitor/hmp-cmds.c +1 -1
index f608b5b27b..1e5e611ed9 100644
@@ -1007,7 +1007,7 @@ void hmp_memsave(Monitor *mon, const QDict *qdict)
     const char *filename = qdict_get_str(qdict, "filename");
     uint64_t addr = qdict_get_int(qdict, "val");
     Error *err = NULL;
-    int cpu_index = monitor_get_cpu_index();
+    int cpu_index = monitor_get_cpu_index(mon);
 
     if (cpu_index < 0) {
         monitor_printf(mon, "No CPU available\n");
monitor/misc.c +10 -10
index b4f779b8d3..972726061c 100644
@@ -269,23 +269,23 @@ int monitor_set_cpu(Monitor *mon, int cpu_index)
 }
 
 /* Callers must hold BQL. */
-static CPUState *mon_get_cpu_sync(bool synchronize)
+static CPUState *mon_get_cpu_sync(Monitor *mon, bool synchronize)
 {
     CPUState *cpu = NULL;
 
-    if (cur_mon->mon_cpu_path) {
-        cpu = (CPUState *) object_resolve_path_type(cur_mon->mon_cpu_path,
+    if (mon->mon_cpu_path) {
+        cpu = (CPUState *) object_resolve_path_type(mon->mon_cpu_path,
                                                     TYPE_CPU, NULL);
         if (!cpu) {
-            g_free(cur_mon->mon_cpu_path);
-            cur_mon->mon_cpu_path = NULL;
+            g_free(mon->mon_cpu_path);
+            mon->mon_cpu_path = NULL;
         }
     }
-    if (!cur_mon->mon_cpu_path) {
+    if (!mon->mon_cpu_path) {
         if (!first_cpu) {
             return NULL;
         }
-        monitor_set_cpu(cur_mon, first_cpu->cpu_index);
+        monitor_set_cpu(mon, first_cpu->cpu_index);
         cpu = first_cpu;
     }
     assert(cpu != NULL);
@@ -297,7 +297,7 @@ static CPUState *mon_get_cpu_sync(bool synchronize)
 
 CPUState *mon_get_cpu(void)
 {
-    return mon_get_cpu_sync(true);
+    return mon_get_cpu_sync(cur_mon, true);
 }
 
 CPUArchState *mon_get_cpu_env(void)
@@ -307,9 +307,9 @@ CPUArchState *mon_get_cpu_env(void)
     return cs ? cs->env_ptr : NULL;
 }
 
-int monitor_get_cpu_index(void)
+int monitor_get_cpu_index(Monitor *mon)
 {
-    CPUState *cs = mon_get_cpu_sync(false);
+    CPUState *cs = mon_get_cpu_sync(mon, false);
 
     return cs ? cs->cpu_index : UNASSIGNED_CPU_INDEX;
 }
softmmu/cpus.c +1 -1
index e3b98065c9..ae643447e9 100644
@@ -2224,7 +2224,7 @@ exit:
 
 void qmp_inject_nmi(Error **errp)
 {
-    nmi_monitor_handle(monitor_get_cpu_index(), errp);
+    nmi_monitor_handle(monitor_get_cpu_index(cur_mon), errp);
 }
 
 void dump_drift_info(void)
Signature · 1 line
2.25.4
author
#3 lore raw

PATCH v7 3/13 monitor: Use getter/setter functions for cur_mon

In reply to Kevin Wolf

cur_mon really needs to be coroutine-local as soon as we move monitor command handlers to coroutines and let them yield. As a first step, just remove all direct accesses to cur_mon so that we can implement this in the getter function later.

---
 include/monitor/monitor.h  |  3 ++-
 audio/wavcapture.c         |  8 ++++----
 dump/dump.c                |  2 +-
 hw/scsi/vhost-scsi.c       |  2 +-
 hw/virtio/vhost-vsock.c    |  2 +-
 migration/fd.c             |  4 ++--
 monitor/hmp.c              | 11 +++++------
 monitor/misc.c             | 13 +++++++------
 monitor/monitor.c          | 24 +++++++++++++++++++++++-
 monitor/qmp-cmds-control.c |  2 ++
 monitor/qmp-cmds.c         |  2 +-
 monitor/qmp.c              |  7 ++-----
 net/socket.c               |  2 +-
 net/tap.c                  |  6 +++---
 softmmu/cpus.c             |  2 +-
 stubs/monitor-core.c       |  5 ++++-
 tests/test-util-sockets.c  | 12 +++++-------
 trace/control.c            |  2 +-
 util/qemu-error.c          |  6 +++---
 util/qemu-print.c          |  3 ++-
 util/qemu-sockets.c        |  1 +
 21 files changed, 72 insertions(+), 47 deletions(-)
include/monitor/monitor.h +2 -1
index a64502fbb2..7b0ad1de12 100644
@@ -5,7 +5,6 @@
 #include "qapi/qapi-types-misc.h"
 #include "qemu/readline.h"
 
-extern __thread Monitor *cur_mon;
 typedef struct MonitorHMP MonitorHMP;
 typedef struct MonitorOptions MonitorOptions;
 
@@ -13,6 +12,8 @@ typedef struct MonitorOptions MonitorOptions;
 
 extern QemuOptsList qemu_mon_opts;
 
+Monitor *monitor_cur(void);
+Monitor *monitor_set_cur(Monitor *mon);
 bool monitor_cur_is_qmp(void);
 
 void monitor_init_globals(void);
audio/wavcapture.c +4 -4
index 17e87ed6f4..c60286e162 100644
@@ -1,5 +1,5 @@
 #include "qemu/osdep.h"
-#include "monitor/monitor.h"
+#include "qemu/qemu-print.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "audio.h"
@@ -94,9 +94,9 @@ static void wav_capture_info (void *opaque)
     WAVState *wav = opaque;
     char *path = wav->path;
 
-    monitor_printf (cur_mon, "Capturing audio(%d,%d,%d) to %s: %d bytes\n",
-                    wav->freq, wav->bits, wav->nchannels,
-                    path ? path : "<not available>", wav->bytes);
+    qemu_printf("Capturing audio(%d,%d,%d) to %s: %d bytes\n",
+                wav->freq, wav->bits, wav->nchannels,
+                path ? path : "<not available>", wav->bytes);
 }
 
 static struct capture_ops wav_capture_ops = {
dump/dump.c +1 -1
index 383bc7876b..232027e92c 100644
@@ -1986,7 +1986,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
 
 #if !defined(WIN32)
     if (strstart(file, "fd:", &p)) {
-        fd = monitor_get_fd(cur_mon, p, errp);
+        fd = monitor_get_fd(monitor_cur(), p, errp);
         if (fd == -1) {
             return;
         }
hw/scsi/vhost-scsi.c +1 -1
index a83ffeefc8..4d70fa036b 100644
@@ -177,7 +177,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
     }
 
     if (vs->conf.vhostfd) {
-        vhostfd = monitor_fd_param(cur_mon, vs->conf.vhostfd, errp);
+        vhostfd = monitor_fd_param(monitor_cur(), vs->conf.vhostfd, errp);
         if (vhostfd == -1) {
             error_prepend(errp, "vhost-scsi: unable to parse vhostfd: ");
             return;
hw/virtio/vhost-vsock.c +1 -1
index c8f0699b4f..f9db4beb47 100644
@@ -143,7 +143,7 @@ static void vhost_vsock_device_realize(DeviceState *dev, Error **errp)
     }
 
     if (vsock->conf.vhostfd) {
-        vhostfd = monitor_fd_param(cur_mon, vsock->conf.vhostfd, errp);
+        vhostfd = monitor_fd_param(monitor_cur(), vsock->conf.vhostfd, errp);
         if (vhostfd == -1) {
             error_prepend(errp, "vhost-vsock: unable to parse vhostfd: ");
             return;
migration/fd.c +2 -2
index 0a29ecdebf..6f2f50475f 100644
@@ -26,7 +26,7 @@
 void fd_start_outgoing_migration(MigrationState *s, const char *fdname, Error **errp)
 {
     QIOChannel *ioc;
-    int fd = monitor_get_fd(cur_mon, fdname, errp);
+    int fd = monitor_get_fd(monitor_cur(), fdname, errp);
     if (fd == -1) {
         return;
     }
@@ -55,7 +55,7 @@ static gboolean fd_accept_incoming_migration(QIOChannel *ioc,
 void fd_start_incoming_migration(const char *fdname, Error **errp)
 {
     QIOChannel *ioc;
-    int fd = monitor_fd_param(cur_mon, fdname, errp);
+    int fd = monitor_fd_param(monitor_cur(), fdname, errp);
     if (fd == -1) {
         return;
     }
monitor/hmp.c +5 -6
index d598dd02bb..561e32d02f 100644
@@ -1300,12 +1300,11 @@ cleanup:
 
 static void monitor_read(void *opaque, const uint8_t *buf, int size)
 {
-    MonitorHMP *mon;
-    Monitor *old_mon = cur_mon;
+    MonitorHMP *mon = container_of(opaque, MonitorHMP, common);
+    Monitor *old_mon;
     int i;
 
-    cur_mon = opaque;
-    mon = container_of(cur_mon, MonitorHMP, common);
+    old_mon = monitor_set_cur(&mon->common);
 
     if (mon->rs) {
         for (i = 0; i < size; i++) {
@@ -1313,13 +1312,13 @@ static void monitor_read(void *opaque, const uint8_t *buf, int size)
         }
     } else {
         if (size == 0 || buf[size - 1] != 0) {
-            monitor_printf(cur_mon, "corrupted command\n");
+            monitor_printf(&mon->common, "corrupted command\n");
         } else {
             handle_hmp_command(mon, (char *)buf);
         }
     }
 
-    cur_mon = old_mon;
+    monitor_set_cur(old_mon);
 }
 
 static void monitor_event(void *opaque, QEMUChrEvent event)
monitor/misc.c +7 -6
index 972726061c..a5eb3025f8 100644
@@ -125,13 +125,12 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
 
     monitor_data_init(&hmp.common, false, true, false);
 
-    old_mon = cur_mon;
-    cur_mon = &hmp.common;
+    old_mon = monitor_set_cur(&hmp.common);
 
     if (has_cpu_index) {
         int ret = monitor_set_cpu(&hmp.common, cpu_index);
         if (ret < 0) {
-            cur_mon = old_mon;
+            monitor_set_cur(old_mon);
             error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
                        "a CPU number");
             goto out;
@@ -139,7 +138,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
     }
 
     handle_hmp_command(&hmp, command_line);
-    cur_mon = old_mon;
+    monitor_set_cur(old_mon);
 
     qemu_mutex_lock(&hmp.common.mon_lock);
     if (qstring_get_length(hmp.common.outbuf) > 0) {
@@ -297,7 +296,7 @@ static CPUState *mon_get_cpu_sync(Monitor *mon, bool synchronize)
 
 CPUState *mon_get_cpu(void)
 {
-    return mon_get_cpu_sync(cur_mon, true);
+    return mon_get_cpu_sync(monitor_cur(), true);
 }
 
 CPUArchState *mon_get_cpu_env(void)
@@ -1232,6 +1231,7 @@ static void hmp_acl_remove(Monitor *mon, const QDict *qdict)
 
 void qmp_getfd(const char *fdname, Error **errp)
 {
+    Monitor *cur_mon = monitor_cur();
     mon_fd_t *monfd;
     int fd, tmp_fd;
 
@@ -1272,6 +1272,7 @@ void qmp_getfd(const char *fdname, Error **errp)
 
 void qmp_closefd(const char *fdname, Error **errp)
 {
+    Monitor *cur_mon = monitor_cur();
     mon_fd_t *monfd;
     int tmp_fd;
 
@@ -1361,7 +1362,7 @@ AddfdInfo *qmp_add_fd(bool has_fdset_id, int64_t fdset_id, bool has_opaque,
                       const char *opaque, Error **errp)
 {
     int fd;
-    Monitor *mon = cur_mon;
+    Monitor *mon = monitor_cur();
     AddfdInfo *fdinfo;
 
     fd = qemu_chr_fe_get_msgfd(&mon->chr);
monitor/monitor.c +23 -1
index b385a3d569..be3839a7aa 100644
@@ -66,13 +66,31 @@ MonitorList mon_list;
 int mon_refcount;
 static bool monitor_destroyed;
 
-__thread Monitor *cur_mon;
+static __thread Monitor *cur_monitor;
+
+Monitor *monitor_cur(void)
+{
+    return cur_monitor;
+}
+
+/**
+ * Sets a new current monitor and returns the old one.
+ */
+Monitor *monitor_set_cur(Monitor *mon)
+{
+    Monitor *old_monitor = cur_monitor;
+
+    cur_monitor = mon;
+    return old_monitor;
+}
 
 /**
  * Is the current monitor, if any, a QMP monitor?
  */
 bool monitor_cur_is_qmp(void)
 {
+    Monitor *cur_mon = monitor_cur();
+
     return cur_mon && monitor_is_qmp(cur_mon);
 }
 
@@ -209,6 +227,8 @@ int monitor_printf(Monitor *mon, const char *fmt, ...)
  */
 int error_vprintf(const char *fmt, va_list ap)
 {
+    Monitor *cur_mon = monitor_cur();
+
     if (cur_mon && !monitor_cur_is_qmp()) {
         return monitor_vprintf(cur_mon, fmt, ap);
     }
@@ -217,6 +237,8 @@ int error_vprintf(const char *fmt, va_list ap)
 
 int error_vprintf_unless_qmp(const char *fmt, va_list ap)
 {
+    Monitor *cur_mon = monitor_cur();
+
     if (!cur_mon) {
         return vfprintf(stderr, fmt, ap);
     }
monitor/qmp-cmds-control.c +2 -0
index 8f04cfa6e6..a456762f6a 100644
@@ -69,6 +69,7 @@ static bool qmp_caps_accept(MonitorQMP *mon, QMPCapabilityList *list,
 void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
                           Error **errp)
 {
+    Monitor *cur_mon = monitor_cur();
     MonitorQMP *mon;
 
     assert(monitor_is_qmp(cur_mon));
@@ -119,6 +120,7 @@ static void query_commands_cb(const QmpCommand *cmd, void *opaque)
 CommandInfoList *qmp_query_commands(Error **errp)
 {
     CommandInfoList *list = NULL;
+    Monitor *cur_mon = monitor_cur();
     MonitorQMP *mon;
 
     assert(monitor_is_qmp(cur_mon));
monitor/qmp-cmds.c +1 -1
index 864cbfa32e..c7bf6bb4dc 100644
@@ -327,7 +327,7 @@ void qmp_add_client(const char *protocol, const char *fdname,
     Chardev *s;
     int fd;
 
-    fd = monitor_get_fd(cur_mon, fdname, errp);
+    fd = monitor_get_fd(monitor_cur(), fdname, errp);
     if (fd < 0) {
         return;
     }
monitor/qmp.c +2 -5
index d433ceae5b..bb2d9d0cc7 100644
@@ -139,12 +139,9 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
     QDict *rsp;
     QDict *error;
 
-    old_mon = cur_mon;
-    cur_mon = &mon->common;
-
+    old_mon = monitor_set_cur(&mon->common);
     rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    cur_mon = old_mon;
+    monitor_set_cur(old_mon);
 
     if (mon->commands == &qmp_cap_negotiation_commands) {
         error = qdict_get_qdict(rsp, "error");
net/socket.c +1 -1
index 2d21fddd9c..15b410e8d8 100644
@@ -727,7 +727,7 @@ int net_init_socket(const Netdev *netdev, const char *name,
     if (sock->has_fd) {
         int fd, ret;
 
-        fd = monitor_fd_param(cur_mon, sock->fd, errp);
+        fd = monitor_fd_param(monitor_cur(), sock->fd, errp);
         if (fd == -1) {
             return -1;
         }
net/tap.c +3 -3
index 14dc904fca..688dcfafdf 100644
@@ -696,7 +696,7 @@ static void net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
         if (vhostfdname) {
             int ret;
 
-            vhostfd = monitor_fd_param(cur_mon, vhostfdname, &err);
+            vhostfd = monitor_fd_param(monitor_cur(), vhostfdname, &err);
             if (vhostfd == -1) {
                 if (tap->has_vhostforce && tap->vhostforce) {
                     error_propagate(errp, err);
@@ -802,7 +802,7 @@ int net_init_tap(const Netdev *netdev, const char *name,
             return -1;
         }
 
-        fd = monitor_fd_param(cur_mon, tap->fd, errp);
+        fd = monitor_fd_param(monitor_cur(), tap->fd, errp);
         if (fd == -1) {
             return -1;
         }
@@ -856,7 +856,7 @@ int net_init_tap(const Netdev *netdev, const char *name,
         }
 
         for (i = 0; i < nfds; i++) {
-            fd = monitor_fd_param(cur_mon, fds[i], errp);
+            fd = monitor_fd_param(monitor_cur(), fds[i], errp);
             if (fd == -1) {
                 ret = -1;
                 goto free_fail;
softmmu/cpus.c +1 -1
index ae643447e9..aa1b6956f3 100644
@@ -2224,7 +2224,7 @@ exit:
 
 void qmp_inject_nmi(Error **errp)
 {
-    nmi_monitor_handle(monitor_get_cpu_index(cur_mon), errp);
+    nmi_monitor_handle(monitor_get_cpu_index(monitor_cur()), errp);
 }
 
 void dump_drift_info(void)
stubs/monitor-core.c +4 -1
index 6cff1c4e1d..0cd2d864b2 100644
@@ -3,7 +3,10 @@
 #include "qemu-common.h"
 #include "qapi/qapi-emit-events.h"
 
-__thread Monitor *cur_mon;
+Monitor *monitor_cur(void)
+{
+    return NULL;
+}
 
 void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
 {
tests/test-util-sockets.c +5 -7
index af9f5c0c70..6c50dbf051 100644
@@ -52,6 +52,7 @@ static void test_fd_is_socket_good(void)
 
 static int mon_fd = -1;
 static const char *mon_fdname;
+__thread Monitor *cur_mon;
 
 int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
 {
@@ -66,15 +67,12 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
 
 /*
  * Syms of stubs in libqemuutil.a are discarded at .o file granularity.
- * To replace monitor_get_fd() we must ensure everything in
- * stubs/monitor.c is defined, to make sure monitor.o is discarded
- * otherwise we get duplicate syms at link time.
+ * To replace monitor_get_fd() and monitor_cur(), we must ensure that we also
+ * replace any other symbol that is used in the binary and would be taken from
+ * the same stub object file, otherwise we get duplicate syms at link time.
  */
-__thread Monitor *cur_mon;
+Monitor *monitor_cur(void) { return cur_mon; }
 int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) { abort(); }
-void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp) {}
-void monitor_init_hmp(Chardev *chr, bool use_readline, Error **errp) {}
-
 
 static void test_socket_fd_pass_name_good(void)
 {
trace/control.c +1 -1
index c63a4de732..b35e512dce 100644
@@ -176,7 +176,7 @@ void trace_enable_events(const char *line_buf)
 {
     if (is_help_option(line_buf)) {
         trace_list_events();
-        if (cur_mon == NULL) {
+        if (monitor_cur() == NULL) {
             exit(0);
         }
     } else {
util/qemu-error.c +3 -3
index 3ee41438e9..aa30f03564 100644
@@ -171,7 +171,7 @@ static void print_loc(void)
     int i;
     const char *const *argp;
 
-    if (!cur_mon && progname) {
+    if (!monitor_cur() && progname) {
         fprintf(stderr, "%s:", progname);
         sep = " ";
     }
@@ -208,7 +208,7 @@ static void vreport(report_type type, const char *fmt, va_list ap)
     GTimeVal tv;
     gchar *timestr;
 
-    if (error_with_timestamp && !cur_mon) {
+    if (error_with_timestamp && !monitor_cur()) {
         g_get_current_time(&tv);
         timestr = g_time_val_to_iso8601(&tv);
         error_printf("%s ", timestr);
@@ -216,7 +216,7 @@ static void vreport(report_type type, const char *fmt, va_list ap)
     }
 
     /* Only prepend guest name if -msg guest-name and -name guest=... are set */
-    if (error_with_guestname && error_guest_name && !cur_mon) {
+    if (error_with_guestname && error_guest_name && !monitor_cur()) {
         error_printf("%s ", error_guest_name);
     }
 
util/qemu-print.c +2 -1
index e79d6b8396..69ba612f56 100644
@@ -20,6 +20,7 @@
  */
 int qemu_vprintf(const char *fmt, va_list ap)
 {
+    Monitor *cur_mon = monitor_cur();
     if (cur_mon) {
         return monitor_vprintf(cur_mon, fmt, ap);
     }
@@ -48,7 +49,7 @@ int qemu_printf(const char *fmt, ...)
 int qemu_vfprintf(FILE *stream, const char *fmt, va_list ap)
 {
     if (!stream) {
-        return monitor_vprintf(cur_mon, fmt, ap);
+        return monitor_vprintf(monitor_cur(), fmt, ap);
     }
     return vfprintf(stream, fmt, ap);
 }
util/qemu-sockets.c +1 -0
index b37d288866..40c18ba142 100644
@@ -1092,6 +1092,7 @@ fail:
 
 static int socket_get_fd(const char *fdstr, int num, Error **errp)
 {
+    Monitor *cur_mon = monitor_cur();
     int fd;
     if (num != 1) {
         error_setg_errno(errp, EINVAL, "socket_get_fd: too many connections");
Signature · 1 line
2.25.4
#4 lore raw

Re: [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon

Additional nitpick detail on Kevin's request.

Quoted 6 lines · Kevin Wolf

cur_mon really needs to be coroutine-local as soon as we move monitor command handlers to coroutines and let them yield. As a first step, just remove all direct accesses to cur_mon so that we can implement this in the getter function later.

[...]

Quoted 13 lines
tests/test-util-sockets.c +1 -0
index af9f5c0c70..6c50dbf051 100644
@@ -52,6 +52,7 @@ static void test_fd_is_socket_good(void)
 
 static int mon_fd = -1;
 static const char *mon_fdname;
+__thread Monitor *cur_mon;
 
 int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
 {

tests/test-util-sockets.c int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp) · line 72

Quoted 9 lines
tests/test-util-sockets.c +3 -3
@@ -66,15 +67,12 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
 
 /*
  * Syms of stubs in libqemuutil.a are discarded at .o file granularity.
- * To replace monitor_get_fd() we must ensure everything in
- * stubs/monitor.c is defined, to make sure monitor.o is discarded
- * otherwise we get duplicate syms at link time.
+ * To replace monitor_get_fd() and monitor_cur(), we must ensure that we also
+ * replace any other symbol that is used in the binary and would be taken from
+ * the same stub object file, otherwise we get duplicate syms at link time.

Wrapping the comment around column 70 or so would make it easier to read. File has no maintainers. Up to you.

Quoted 10 lines
tests/test-util-sockets.c +1 -4
  */
-__thread Monitor *cur_mon;
+Monitor *monitor_cur(void) { return cur_mon; }
 int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) { abort(); }
-void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp) {}
-void monitor_init_hmp(Chardev *chr, bool use_readline, Error **errp) {}
-
 
 static void test_socket_fd_pass_name_good(void)
 {

[...]

author
#5 lore raw

PATCH v7 4/13 hmp: Update current monitor only in handle_hmp_command()

In reply to Kevin Wolf

The current monitor is updated relatively early in the command handling code even though only the command handler actually needs it.

The current monitor will become coroutine-local later, so we can only update it when we know in which coroutine the command will be exectued. Move it to handle_hmp_command() where this information will be available.

---
 monitor/hmp.c  | 10 +++++-----
 monitor/misc.c |  5 -----
 2 files changed, 5 insertions(+), 10 deletions(-)
monitor/hmp.c +5 -5
index 561e32d02f..896c670183 100644
@@ -1061,6 +1061,7 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
     QDict *qdict;
     const HMPCommand *cmd;
     const char *cmd_start = cmdline;
+    Monitor *old_mon;
 
     trace_handle_hmp_command(mon, cmdline);
 
@@ -1079,7 +1080,11 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
+    /* old_mon is non-NULL when called from qmp_human_monitor_command() */
+    old_mon = monitor_set_cur(&mon->common);
     cmd->cmd(&mon->common, qdict);
+    monitor_set_cur(old_mon);
+
     qobject_unref(qdict);
 }
 
@@ -1301,11 +1306,8 @@ cleanup:
 static void monitor_read(void *opaque, const uint8_t *buf, int size)
 {
     MonitorHMP *mon = container_of(opaque, MonitorHMP, common);
-    Monitor *old_mon;
     int i;
 
-    old_mon = monitor_set_cur(&mon->common);
-
     if (mon->rs) {
         for (i = 0; i < size; i++) {
             readline_handle_byte(mon->rs, buf[i]);
@@ -1317,8 +1319,6 @@ static void monitor_read(void *opaque, const uint8_t *buf, int size)
             handle_hmp_command(mon, (char *)buf);
         }
     }
-
-    monitor_set_cur(old_mon);
 }
 
 static void monitor_event(void *opaque, QEMUChrEvent event)
monitor/misc.c +0 -5
index a5eb3025f8..ba1063024c 100644
@@ -120,17 +120,13 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
                                 int64_t cpu_index, Error **errp)
 {
     char *output = NULL;
-    Monitor *old_mon;
     MonitorHMP hmp = {};
 
     monitor_data_init(&hmp.common, false, true, false);
 
-    old_mon = monitor_set_cur(&hmp.common);
-
     if (has_cpu_index) {
         int ret = monitor_set_cpu(&hmp.common, cpu_index);
         if (ret < 0) {
-            monitor_set_cur(old_mon);
             error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
                        "a CPU number");
             goto out;
@@ -138,7 +134,6 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
     }
 
     handle_hmp_command(&hmp, command_line);
-    monitor_set_cur(old_mon);
 
     qemu_mutex_lock(&hmp.common.mon_lock);
     if (qstring_get_length(hmp.common.outbuf) > 0) {
Signature · 1 line
2.25.4
author
#6 lore raw

PATCH v7 5/13 qmp: Assert that no other monitor is active

In reply to Kevin Wolf

monitor_qmp_dispatch() is never supposed to be called in the context of another monitor, so assert that monitor_cur() is NULL instead of saving and restoring it.

---
 monitor/qmp.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
monitor/qmp.c +4 -1
index bb2d9d0cc7..8469970c69 100644
@@ -140,8 +140,11 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
     QDict *error;
 
     old_mon = monitor_set_cur(&mon->common);
+    assert(old_mon == NULL);
+
     rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-    monitor_set_cur(old_mon);
+
+    monitor_set_cur(NULL);
 
     if (mon->commands == &qmp_cap_negotiation_commands) {
         error = qdict_get_qdict(rsp, "error");
Signature · 1 line
2.25.4
author
#7 lore raw

PATCH v7 6/13 qmp: Call monitor_set_cur() only in qmp_dispatch()

In reply to Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +1 -7
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
 
     if (mon->commands == &qmp_cap_negotiation_commands) {
         error = qdict_get_qdict(rsp, "error");
qapi/qmp-dispatch.c +7 -1
index 79347e0864..2fdbc0fba4 100644
@@ -89,7 +89,7 @@ bool qmp_is_oob(const QDict *dict)
 }
 
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob)
+                    bool allow_oob, Monitor *cur_mon)
 {
     Error *err = NULL;
     bool oob;
@@ -152,7 +152,13 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
         args = qdict_get_qdict(dict, "arguments");
         qobject_ref(args);
     }
+
+    assert(monitor_cur() == NULL);
+    monitor_set_cur(cur_mon);
+
     cmd->fn(args, &ret, &err);
+
+    monitor_set_cur(NULL);
     qobject_unref(args);
     if (err) {
         /* or assert(!ret) after reviewing all handlers: */
qga/main.c +1 -1
index 3febf3b0fd..241779a1d6 100644
@@ -578,7 +578,7 @@ static void process_event(void *opaque, QObject *obj, Error *err)
     }
 
     g_debug("processing command");
-    rsp = qmp_dispatch(&ga_commands, obj, false);
+    rsp = qmp_dispatch(&ga_commands, obj, false, NULL);
 
 end:
     ret = send_response(s, rsp);
stubs/monitor-core.c +5 -0
index 0cd2d864b2..dc1748bf13 100644
@@ -8,6 +8,11 @@ Monitor *monitor_cur(void)
     return NULL;
 }
 
+Monitor *monitor_set_cur(Monitor *mon)
+{
+    return NULL;
+}
+
 void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
 {
 }
tests/test-qmp-cmds.c +3 -3
index d12ff47e26..5f1b245e19 100644
@@ -152,7 +152,7 @@ static QObject *do_qmp_dispatch(bool allow_oob, const char *template, ...)
     req = qdict_from_vjsonf_nofail(template, ap);
     va_end(ap);
 
-    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob);
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob, NULL);
     g_assert(resp);
     ret = qdict_get(resp, "return");
     g_assert(ret);
@@ -175,7 +175,7 @@ static void do_qmp_dispatch_error(bool allow_oob, ErrorClass cls,
     req = qdict_from_vjsonf_nofail(template, ap);
     va_end(ap);
 
-    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob);
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob, NULL);
     g_assert(resp);
     error = qdict_get_qdict(resp, "error");
     g_assert(error);
@@ -231,7 +231,7 @@ static void test_dispatch_cmd_success_response(void)
     QDict *resp;
 
     qdict_put_str(req, "execute", "cmd-success-response");
-    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
+    resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false, NULL);
     g_assert_null(resp);
     qobject_unref(req);
 }
Signature · 1 line
2.25.4
#8 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 44 lines · Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 10 lines
monitor/qmp.c +1 -6
     QDict *rsp;
     QDict *error;
 
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 35 lines
monitor/qmp.c +0 -0
     if (mon->commands == &qmp_cap_negotiation_commands) {
         error = qdict_get_qdict(rsp, "error");
qapi/qmp-dispatch.c +7 -1
index 79347e0864..2fdbc0fba4 100644
@@ -89,7 +89,7 @@ bool qmp_is_oob(const QDict *dict)
 }
 
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob)
+                    bool allow_oob, Monitor *cur_mon)
 {
     Error *err = NULL;
     bool oob;
@@ -152,7 +152,13 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
         args = qdict_get_qdict(dict, "arguments");
         qobject_ref(args);
     }
+
+    assert(monitor_cur() == NULL);
+    monitor_set_cur(cur_mon);
+
     cmd->fn(args, &ret, &err);
+
+    monitor_set_cur(NULL);
     qobject_unref(args);
     if (err) {
         /* or assert(!ret) after reviewing all handlers: */
qga/main.c +0 -0
index 3febf3b0fd..241779a1d6 100644

qga/main.c static void process_event(void *opaque, QObject *obj, Error *err) · line 584

Quoted 8 lines
qga/main.c +1 -1
@@ -578,7 +578,7 @@ static void process_event(void *opaque, QObject *obj, Error *err)
     }
 
     g_debug("processing command");
-    rsp = qmp_dispatch(&ga_commands, obj, false);
+    rsp = qmp_dispatch(&ga_commands, obj, false, NULL);
 
 end:
     ret = send_response(s, rsp);

I dislike how this entangles qemu-ga even more with the monitor. But I want to get this wrapped more than I want it to be spotless. Perhaps I can clean up some on top.

[...]

author
#9 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 49 lines
Quoted 47 lines · Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Kevin

#10 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 60 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 49 lines
Quoted 47 lines · Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Would you mind me wrapping this line in my tree?

A few more in PATCH 08-11.

author
#11 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 67 lines
Quoted 60 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 49 lines
Quoted 47 lines · Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

Kevin

#12 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 77 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 67 lines
Quoted 60 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 49 lines
Quoted 47 lines · Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
Quoted 7 lines

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

The applicable coding style document is PEP 8.

Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

author
#13 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:

Quoted 83 lines
Quoted 81 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 67 lines
Quoted 60 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 49 lines
Quoted 47 lines · Kevin Wolf

The correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().

---
 include/qapi/qmp/dispatch.h | 3 ++-
 monitor/qmp.c               | 8 +-------
 qapi/qmp-dispatch.c         | 8 +++++++-
 qga/main.c                  | 2 +-
 stubs/monitor-core.c        | 5 +++++
 tests/test-qmp-cmds.c       | 6 +++---
 6 files changed, 19 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
index 5a9cf82472..0c2f467028 100644
@@ -14,6 +14,7 @@
 #ifndef QAPI_QMP_DISPATCH_H
 #define QAPI_QMP_DISPATCH_H
 
+#include "monitor/monitor.h"
 #include "qemu/queue.h"
 
 typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
 bool qmp_has_success_response(const QmpCommand *cmd);
 QDict *qmp_error_response(Error *err);
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
-                    bool allow_oob);
+                    bool allow_oob, Monitor *cur_mon);
 bool qmp_is_oob(const QDict *dict);
 
 typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Quoted 5 lines
Quoted 3 lines

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

The applicable coding style document is PEP 8.

I'll happily apply PEP 8 to Python code, but this is C. I don't think PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style guide, but we're not writing C code for the Python project here...)

Quoted 11 lines
Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

The project style guide for C is defined in CODING_STYLE.rst. Missing consistency with it is what I'm complaining about.

I also agree that consistency within one module or function is most important, which is why I allow you to reformat my code. But I don't think it means that local coding style rules shouldn't be documented, especially if you can't just look at the code and see immediately how it's supposed to be.

Quoted 11 lines
Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

Quoted 7 lines

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Ok, that's finally clear limits at least.

Any other rules from PEP 8 that you want to see applied to C code?

Would you mind documenting this somewhere?

Quoted 17 lines

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

Quoted 8 lines

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

As I said, I don't mind the exact number much. I do mind predictability, though. (And ideally also consistency across the project because otherwise I need to change my editor settings for individual files.)

So if you don't like 79 columns, give me any other number. But please, do give me something specific I can work with. "illegibly wide" is not something I can work with because it's highly subjective.

Kevin

#14 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 8 lines · Kevin Wolf

Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:

Quoted 7 lines
Quoted 5 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 4 lines
Quoted 2 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 1 line

Kevin Wolf <[email protected]> writes:

[...]

Quoted 44 lines
Quoted 44 lines
Quoted 43 lines
Quoted 27 lines
Quoted 22 lines
Quoted 12 lines
Quoted 12 lines
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

Quoted 5 lines
Quoted 1 line

The applicable coding style document is PEP 8.

I'll happily apply PEP 8 to Python code, but this is C. I don't think PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style guide, but we're not writing C code for the Python project here...)

I got confused (too much Python code review), my apologies.

Quoted 36 lines
Quoted 11 lines
Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

The project style guide for C is defined in CODING_STYLE.rst. Missing consistency with it is what I'm complaining about.

I also agree that consistency within one module or function is most important, which is why I allow you to reformat my code. But I don't think it means that local coding style rules shouldn't be documented, especially if you can't just look at the code and see immediately how it's supposed to be.

Quoted 15 lines
Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

Quoted 7 lines
Quoted 3 lines

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Ok, that's finally clear limits at least.

Any other rules from PEP 8 that you want to see applied to C code?

PEP 8 does not apply to C.

Quoted 28 lines

Would you mind documenting this somewhere?

Quoted 17 lines

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

Quoted 8 lines

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

Quoted 7 lines

As I said, I don't mind the exact number much. I do mind predictability, though. (And ideally also consistency across the project because otherwise I need to change my editor settings for individual files.)

So if you don't like 79 columns, give me any other number. But please, do give me something specific I can work with. "illegibly wide" is not something I can work with because it's highly subjective.

Taste is subjective.

We can always make CODING_STYLE.rst more detailed. I view that as a last resort when we waste too much time arguing.

Back to line length.

CODING_STYLE.rst sets a *limit*.

Going over the limit violates CODING_STYLE.rst. There are (rare) cases where that is justified.

CODING_STYLE.rst neither demands nor prohibits breaking lines before the limit is reached.

Until CODING_STYLE.rst prohibits breaking lines unless they exceed the limit, I will continue to ask for breaking lines when that makes the code easier to read and more consistent with the code around it, for code I maintain, and admittedly in my opinion.

These requests appear to irk you a great deal. I don't understand, but I'm sorry about it all the same. By arguing about it repeatedly, you've irked some back. Brought it on myself, I guess. However, if that's what it takes to keep the code I maintain legible and consistent, I'll pay the price.

author
#15 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:

Quoted 165 lines
Quoted 8 lines · Kevin Wolf

Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:

Quoted 7 lines
Quoted 5 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 4 lines
Quoted 2 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 1 line

Kevin Wolf <[email protected]> writes:

[...]

Quoted 44 lines
Quoted 44 lines
Quoted 43 lines
Quoted 27 lines
Quoted 22 lines
Quoted 12 lines
Quoted 12 lines
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

Quoted 5 lines
Quoted 1 line

The applicable coding style document is PEP 8.

I'll happily apply PEP 8 to Python code, but this is C. I don't think PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style guide, but we're not writing C code for the Python project here...)

I got confused (too much Python code review), my apologies.

Quoted 36 lines
Quoted 11 lines
Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

The project style guide for C is defined in CODING_STYLE.rst. Missing consistency with it is what I'm complaining about.

I also agree that consistency within one module or function is most important, which is why I allow you to reformat my code. But I don't think it means that local coding style rules shouldn't be documented, especially if you can't just look at the code and see immediately how it's supposed to be.

Quoted 15 lines
Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

Quoted 7 lines
Quoted 3 lines

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Ok, that's finally clear limits at least.

Any other rules from PEP 8 that you want to see applied to C code?

PEP 8 does not apply to C.

Quoted 28 lines

Would you mind documenting this somewhere?

Quoted 17 lines

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

Quoted 8 lines

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

Quoted 7 lines

As I said, I don't mind the exact number much. I do mind predictability, though. (And ideally also consistency across the project because otherwise I need to change my editor settings for individual files.)

So if you don't like 79 columns, give me any other number. But please, do give me something specific I can work with. "illegibly wide" is not something I can work with because it's highly subjective.

Taste is subjective.

We can always make CODING_STYLE.rst more detailed. I view that as a last resort when we waste too much time arguing.

Back to line length.

CODING_STYLE.rst sets a *limit*.

Going over the limit violates CODING_STYLE.rst. There are (rare) cases where that is justified.

CODING_STYLE.rst neither demands nor prohibits breaking lines before the limit is reached.

Until CODING_STYLE.rst prohibits breaking lines unless they exceed the limit, I will continue to ask for breaking lines when that makes the code easier to read and more consistent with the code around it, for code I maintain, and admittedly in my opinion.

Quoted 5 lines

These requests appear to irk you a great deal. I don't understand, but I'm sorry about it all the same. By arguing about it repeatedly, you've irked some back. Brought it on myself, I guess. However, if that's what it takes to keep the code I maintain legible and consistent, I'll pay the price.

I conclude that I'll never be able to submit code that passes your review in the first attempt because I don't know the specific criteria (and you don't seem to know them either before you see the patch).

Fine, I'll live with it. It's just one of the things that makes working in your subsystems more frustrating than in others.

Kevin

#16 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 172 lines · * Kevin Wolf ([email protected]) wrote:

Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:

Quoted 165 lines
Quoted 8 lines · Kevin Wolf

Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:

Quoted 7 lines
Quoted 5 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 4 lines
Quoted 2 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 1 line

Kevin Wolf <[email protected]> writes:

[...]

Quoted 44 lines
Quoted 44 lines
Quoted 43 lines
Quoted 27 lines
Quoted 22 lines
Quoted 12 lines
Quoted 12 lines
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

Quoted 5 lines
Quoted 1 line

The applicable coding style document is PEP 8.

I'll happily apply PEP 8 to Python code, but this is C. I don't think PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style guide, but we're not writing C code for the Python project here...)

I got confused (too much Python code review), my apologies.

Quoted 36 lines
Quoted 11 lines
Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

The project style guide for C is defined in CODING_STYLE.rst. Missing consistency with it is what I'm complaining about.

I also agree that consistency within one module or function is most important, which is why I allow you to reformat my code. But I don't think it means that local coding style rules shouldn't be documented, especially if you can't just look at the code and see immediately how it's supposed to be.

Quoted 15 lines
Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

Quoted 7 lines
Quoted 3 lines

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Ok, that's finally clear limits at least.

Any other rules from PEP 8 that you want to see applied to C code?

PEP 8 does not apply to C.

Quoted 28 lines

Would you mind documenting this somewhere?

Quoted 17 lines

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

Quoted 8 lines

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

Quoted 7 lines

As I said, I don't mind the exact number much. I do mind predictability, though. (And ideally also consistency across the project because otherwise I need to change my editor settings for individual files.)

So if you don't like 79 columns, give me any other number. But please, do give me something specific I can work with. "illegibly wide" is not something I can work with because it's highly subjective.

Taste is subjective.

We can always make CODING_STYLE.rst more detailed. I view that as a last resort when we waste too much time arguing.

Back to line length.

CODING_STYLE.rst sets a *limit*.

Going over the limit violates CODING_STYLE.rst. There are (rare) cases where that is justified.

CODING_STYLE.rst neither demands nor prohibits breaking lines before the limit is reached.

Until CODING_STYLE.rst prohibits breaking lines unless they exceed the limit, I will continue to ask for breaking lines when that makes the code easier to read and more consistent with the code around it, for code I maintain, and admittedly in my opinion.

Quoted 5 lines

These requests appear to irk you a great deal. I don't understand, but I'm sorry about it all the same. By arguing about it repeatedly, you've irked some back. Brought it on myself, I guess. However, if that's what it takes to keep the code I maintain legible and consistent, I'll pay the price.

Quoted 6 lines

I conclude that I'll never be able to submit code that passes your review in the first attempt because I don't know the specific criteria (and you don't seem to know them either before you see the patch).

Fine, I'll live with it. It's just one of the things that makes working in your subsystems more frustrating than in others.

Hmm,
IMHO the thing here is that there's two different things here:

a) A CODING_STYLE limit - and personally I use every last character
of that when appropriate
b) For this particular case, Markus is saying he prefers the wrap
there.

I don't think I see (b) as incompatible as a preference, but lets be sensible; if it's something you want to change in merge that seems reasonable, if it's something that you ask to change in a respin that's kind of reasonable, just don't hold up a big patch series for an argument over something that's legal in the coding style and isn't particularly offensive!

Dave

Quoted 1 line

Kevin

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
author
#17 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Am 30.09.2020 um 19:20 hat Dr. David Alan Gilbert geschrieben:

Quoted 188 lines · * Kevin Wolf ([email protected]) wrote:
Quoted 172 lines

Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:

Quoted 165 lines
Quoted 8 lines · Kevin Wolf

Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:

Quoted 7 lines
Quoted 5 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 4 lines
Quoted 2 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 1 line

Kevin Wolf <[email protected]> writes:

[...]

Quoted 44 lines
Quoted 44 lines
Quoted 43 lines
Quoted 27 lines
Quoted 22 lines
Quoted 12 lines
Quoted 12 lines
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

Quoted 5 lines
Quoted 1 line

The applicable coding style document is PEP 8.

I'll happily apply PEP 8 to Python code, but this is C. I don't think PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style guide, but we're not writing C code for the Python project here...)

I got confused (too much Python code review), my apologies.

Quoted 36 lines
Quoted 11 lines
Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

The project style guide for C is defined in CODING_STYLE.rst. Missing consistency with it is what I'm complaining about.

I also agree that consistency within one module or function is most important, which is why I allow you to reformat my code. But I don't think it means that local coding style rules shouldn't be documented, especially if you can't just look at the code and see immediately how it's supposed to be.

Quoted 15 lines
Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

Quoted 7 lines
Quoted 3 lines

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Ok, that's finally clear limits at least.

Any other rules from PEP 8 that you want to see applied to C code?

PEP 8 does not apply to C.

Quoted 28 lines

Would you mind documenting this somewhere?

Quoted 17 lines

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

Quoted 8 lines

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

Quoted 7 lines

As I said, I don't mind the exact number much. I do mind predictability, though. (And ideally also consistency across the project because otherwise I need to change my editor settings for individual files.)

So if you don't like 79 columns, give me any other number. But please, do give me something specific I can work with. "illegibly wide" is not something I can work with because it's highly subjective.

Taste is subjective.

We can always make CODING_STYLE.rst more detailed. I view that as a last resort when we waste too much time arguing.

Back to line length.

CODING_STYLE.rst sets a *limit*.

Going over the limit violates CODING_STYLE.rst. There are (rare) cases where that is justified.

CODING_STYLE.rst neither demands nor prohibits breaking lines before the limit is reached.

Until CODING_STYLE.rst prohibits breaking lines unless they exceed the limit, I will continue to ask for breaking lines when that makes the code easier to read and more consistent with the code around it, for code I maintain, and admittedly in my opinion.

Quoted 5 lines

These requests appear to irk you a great deal. I don't understand, but I'm sorry about it all the same. By arguing about it repeatedly, you've irked some back. Brought it on myself, I guess. However, if that's what it takes to keep the code I maintain legible and consistent, I'll pay the price.

Quoted 6 lines

I conclude that I'll never be able to submit code that passes your review in the first attempt because I don't know the specific criteria (and you don't seem to know them either before you see the patch).

Fine, I'll live with it. It's just one of the things that makes working in your subsystems more frustrating than in others.

Hmm,
IMHO the thing here is that there's two different things here:

a) A CODING_STYLE limit - and personally I use every last character
of that when appropriate
b) For this particular case, Markus is saying he prefers the wrap
there.

Quoted 6 lines

I don't think I see (b) as incompatible as a preference, but lets be sensible; if it's something you want to change in merge that seems reasonable, if it's something that you ask to change in a respin that's kind of reasonable, just don't hold up a big patch series for an argument over something that's legal in the coding style and isn't particularly offensive!

I'll just change this one in the next version. Changing a single well-known instance not a big problem. It's just unfortunate that there are "A few more in PATCH 08-11" and I don't know how to identify them.

So Markus will have to comment again in the next version (as he did in other places in previous versions of this series) and potentially modify my patches while applying to match his taste. The latter is something I try hard to avoid as a maintainer, but I admit this means compromising instead of perfectionism. And the former is just a bit tiring when with every version of a series you get additional comments about style preferences.

Kevin

#18 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 195 lines · Kevin Wolf

Am 30.09.2020 um 19:20 hat Dr. David Alan Gilbert geschrieben:

Quoted 194 lines · * Kevin Wolf ([email protected]) wrote:
Quoted 172 lines

Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:

Quoted 165 lines
Quoted 8 lines · Kevin Wolf

Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:

Quoted 7 lines
Quoted 5 lines · Kevin Wolf

Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:

Quoted 4 lines
Quoted 2 lines · Kevin Wolf

Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:

Quoted 1 line

Kevin Wolf <[email protected]> writes:

[...]

Quoted 44 lines
Quoted 44 lines
Quoted 43 lines
Quoted 27 lines
Quoted 22 lines
Quoted 12 lines
Quoted 12 lines
monitor/qmp.c +0 -1
index 8469970c69..922fdb5541 100644
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
 
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
-    Monitor *old_mon;
     QDict *rsp;
     QDict *error;
 
Quoted 9 lines

monitor/qmp.c static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp) · line 141

Quoted 7 lines
monitor/qmp.c +1 -6
-    old_mon = monitor_set_cur(&mon->common);
-    assert(old_mon == NULL);
-
-    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
-    monitor_set_cur(NULL);
+    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.

Quoted 2 lines

It's 79 characters. Should be fine even with your local deviation from the coding style to require less than that for comments?

Let me rephrase my remark.

Quoted 8 lines

For me,

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
                       &mon->common);

is significantly easier to read than

    rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);

I guess this is highly subjective. I find wrapped lines harder to read. For answering subjective questions like this, we generally use the coding style document.

Anyway, I guess following an idiosyncratic coding style that is different from every other subsystem in QEMU is possible (if inconvenient) if I know what it is.

Quoted 5 lines
Quoted 1 line

The applicable coding style document is PEP 8.

I'll happily apply PEP 8 to Python code, but this is C. I don't think PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style guide, but we're not writing C code for the Python project here...)

I got confused (too much Python code review), my apologies.

Quoted 36 lines
Quoted 11 lines
Quoted 3 lines

My problem is more that I don't know what the exact rules are. Can they only be figured out experimentally by submitting patches and seeing whether you like them or not?

PEP 8:

A style guide is about consistency. Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is the most important.

In other words, you should make a reasonable effort to blend in.

The project style guide for C is defined in CODING_STYLE.rst. Missing consistency with it is what I'm complaining about.

I also agree that consistency within one module or function is most important, which is why I allow you to reformat my code. But I don't think it means that local coding style rules shouldn't be documented, especially if you can't just look at the code and see immediately how it's supposed to be.

Quoted 15 lines
Quoted 4 lines
Quoted 1 line

Would you mind me wrapping this line in my tree?

I have no say in this subsystem and I take it that you want all code to look as if you had written it yourself, so do as you wish.

I'm refusing the bait.

Quoted 3 lines

But I understand that I'll have to respin anyway, so if you could explain what you're after, I might be able to apply the rules for the next version of the series.

First, PEP 8 again:

Limit all lines to a maximum of 79 characters.

Quoted 7 lines
Quoted 3 lines

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Ok, that's finally clear limits at least.

Any other rules from PEP 8 that you want to see applied to C code?

PEP 8 does not apply to C.

Quoted 28 lines

Would you mind documenting this somewhere?

Quoted 17 lines

Second, an argument we two had on this list, during review of a prior version of this patch series, talking about C:

Legibility. Humans tend to have trouble following long lines with their eyes (I sure do). Typographic manuals suggest to limit columns to roughly 60 characters for exactly that reason[*].

Code is special. It's typically indented, and long identifiers push it further to the right, function arguments in particular. We compromised at 80 columns.

[...]

[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style

The width of the line not counting indentation matters for legibility.

Quoted 8 lines

The line I flagged as long is 75 characters wide not counting indentation. That's needlessly hard to read for me.

PEP 8's line length limit is a *limit*, not a sacred right to push right to the limit.

Since I get to read this code a lot, I've taken care to avoid illegibly wide lines, and I've guided contributors to blend in.

Quoted 7 lines

As I said, I don't mind the exact number much. I do mind predictability, though. (And ideally also consistency across the project because otherwise I need to change my editor settings for individual files.)

So if you don't like 79 columns, give me any other number. But please, do give me something specific I can work with. "illegibly wide" is not something I can work with because it's highly subjective.

Taste is subjective.

We can always make CODING_STYLE.rst more detailed. I view that as a last resort when we waste too much time arguing.

Back to line length.

CODING_STYLE.rst sets a *limit*.

Going over the limit violates CODING_STYLE.rst. There are (rare) cases where that is justified.

CODING_STYLE.rst neither demands nor prohibits breaking lines before the limit is reached.

Until CODING_STYLE.rst prohibits breaking lines unless they exceed the limit, I will continue to ask for breaking lines when that makes the code easier to read and more consistent with the code around it, for code I maintain, and admittedly in my opinion.

Quoted 5 lines

These requests appear to irk you a great deal. I don't understand, but I'm sorry about it all the same. By arguing about it repeatedly, you've irked some back. Brought it on myself, I guess. However, if that's what it takes to keep the code I maintain legible and consistent, I'll pay the price.

Quoted 6 lines

I conclude that I'll never be able to submit code that passes your review in the first attempt because I don't know the specific criteria (and you don't seem to know them either before you see the patch).

Fine, I'll live with it. It's just one of the things that makes working in your subsystems more frustrating than in others.

Hmm,
IMHO the thing here is that there's two different things here:

a) A CODING_STYLE limit - and personally I use every last character
of that when appropriate
b) For this particular case, Markus is saying he prefers the wrap
there.

I don't think I see (b) as incompatible as a preference, but lets be sensible; if it's something you want to change in merge that seems reasonable, if it's something that you ask to change in a respin that's kind of reasonable, just don't hold up a big patch series for an argument over something that's legal in the coding style and isn't particularly offensive!

I don't think I ever asked for a respin just to adjust style. I always offer to adjust style myself in my tree. If a respin is needed for some other reason, also making the style adjustments I requested is courteous and appreciated.

I don't think I ever rejected a patch just due to differences over style. If a patch submitter refused to make the style adjustments I want, and refused to permit me to make them, I'd commit as is, then maye adjust on top. This is hypothetical.

Quoted 3 lines

I'll just change this one in the next version. Changing a single well-known instance not a big problem. It's just unfortunate that there are "A few more in PATCH 08-11" and I don't know how to identify them.

When I do that, and you'd rather have a complete list, just ask. Out of time for today, but I can get it for you first thing tomorrow.

[...]

#19 lore raw

Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()

Quoted 1 line · Markus Armbruster

Kevin Wolf <[email protected]> writes:

[...]

Quoted 8 lines
Quoted 3 lines

I'll just change this one in the next version. Changing a single well-known instance not a big problem. It's just unfortunate that there are "A few more in PATCH 08-11" and I don't know how to identify them.

When I do that, and you'd rather have a complete list, just ask. Out of time for today, but I can get it for you first thing tomorrow.

[...]

Done, with the detail level cranked up to "lots" ;)

author
#20 lore raw

PATCH v7 7/13 monitor: Make current monitor a per-coroutine property

In reply to Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

Approaches to implement some form of coroutine local storage directly in the coroutine core code have been considered and discarded because they didn't end up being much more generic than the hash table and their performance impact on coroutines not using coroutine local storage was unclear. As the block layer uses a coroutine per I/O request, this is a fast path and we have to be careful. It's safest to just stay out of this path with code only used by the monitor.

---
 include/monitor/monitor.h |  2 +-
 monitor/hmp.c             |  4 ++--
 monitor/monitor.c         | 34 +++++++++++++++++++++++++++-------
 qapi/qmp-dispatch.c       |  4 ++--
 stubs/monitor-core.c      |  2 +-
 5 files changed, 33 insertions(+), 13 deletions(-)
include/monitor/monitor.h +1 -1
index 7b0ad1de12..026f8a31b2 100644
@@ -13,7 +13,7 @@ typedef struct MonitorOptions MonitorOptions;
 extern QemuOptsList qemu_mon_opts;
 
 Monitor *monitor_cur(void);
-Monitor *monitor_set_cur(Monitor *mon);
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon);
 bool monitor_cur_is_qmp(void);
 
 void monitor_init_globals(void);
monitor/hmp.c +2 -2
index 896c670183..4b66ca1cd6 100644
@@ -1081,9 +1081,9 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
     }
 
     /* old_mon is non-NULL when called from qmp_human_monitor_command() */
-    old_mon = monitor_set_cur(&mon->common);
+    old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
     cmd->cmd(&mon->common, qdict);
-    monitor_set_cur(old_mon);
+    monitor_set_cur(qemu_coroutine_self(), old_mon);
 
     qobject_unref(qdict);
 }
monitor/monitor.c +27 -7
index be3839a7aa..629aa073ee 100644
@@ -58,29 +58,48 @@ IOThread *mon_iothread;
 /* Bottom half to dispatch the requests received from I/O thread */
 QEMUBH *qmp_dispatcher_bh;
 
-/* Protects mon_list, monitor_qapi_event_state, monitor_destroyed.  */
+/*
+ * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
+ * monitor_destroyed.
+ */
 QemuMutex monitor_lock;
 static GHashTable *monitor_qapi_event_state;
+static GHashTable *coroutine_mon; /* Maps Coroutine* to Monitor* */
 
 MonitorList mon_list;
 int mon_refcount;
 static bool monitor_destroyed;
 
-static __thread Monitor *cur_monitor;
-
 Monitor *monitor_cur(void)
 {
-    return cur_monitor;
+    Monitor *mon;
+
+    qemu_mutex_lock(&monitor_lock);
+    mon = g_hash_table_lookup(coroutine_mon, qemu_coroutine_self());
+    qemu_mutex_unlock(&monitor_lock);
+
+    return mon;
 }
 
 /**
  * Sets a new current monitor and returns the old one.
+ *
+ * If a non-NULL monitor is set for a coroutine, another call resetting it to
+ * NULL is required before the coroutine terminates, otherwise a stale entry
+ * would remain in the hash table.
  */
-Monitor *monitor_set_cur(Monitor *mon)
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
 {
-    Monitor *old_monitor = cur_monitor;
+    Monitor *old_monitor = monitor_cur();
+
+    qemu_mutex_lock(&monitor_lock);
+    if (mon) {
+        g_hash_table_replace(coroutine_mon, co, mon);
+    } else {
+        g_hash_table_remove(coroutine_mon, co);
+    }
+    qemu_mutex_unlock(&monitor_lock);
 
-    cur_monitor = mon;
     return old_monitor;
 }
 
@@ -623,6 +642,7 @@ void monitor_init_globals_core(void)
 {
     monitor_qapi_event_init();
     qemu_mutex_init(&monitor_lock);
+    coroutine_mon = g_hash_table_new(NULL, NULL);
 
     /*
      * The dispatcher BH must run in the main loop thread, since we
qapi/qmp-dispatch.c +2 -2
index 2fdbc0fba4..5677ba92ca 100644
@@ -154,11 +154,11 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
     }
 
     assert(monitor_cur() == NULL);
-    monitor_set_cur(cur_mon);
+    monitor_set_cur(qemu_coroutine_self(), cur_mon);
 
     cmd->fn(args, &ret, &err);
 
-    monitor_set_cur(NULL);
+    monitor_set_cur(qemu_coroutine_self(), NULL);
     qobject_unref(args);
     if (err) {
         /* or assert(!ret) after reviewing all handlers: */
stubs/monitor-core.c +1 -1
index dc1748bf13..d058a2a00d 100644
@@ -8,7 +8,7 @@ Monitor *monitor_cur(void)
     return NULL;
 }
 
-Monitor *monitor_set_cur(Monitor *mon)
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
 {
     return NULL;
 }
Signature · 1 line
2.25.4
#21 lore raw

Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property

Quoted 7 lines · Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

In qemu-system-FOO, the hash table can have only these entries:

* (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while executing a QMP command out-of-band.

* (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main thread) to a QMP monitor, while executing a QMP command in-band and in coroutine context.

* (QMP) One mapping the main thread's leader to a QMP monitor, while executing a QMP command in-band and out of coroutine context, in a bottom half.

* (HMP) One mapping the main thread's leader to an HMP monitor, while executing an HMP command out of coroutine context.

* (HMP-CO) One mapping a transient coroutine in the main thread to an HMP monitor, while executing an HMP command in coroutine context.

In-band execution is one command after the other.

Therefore, at most one monitor command can be executing in-band at any time.

Therefore, the hash table has at most *two* entries: one (OOB), and one of the other four.

Can you shoot any holes into my argument?

I suspect there's a simpler solution struggling to get out. But this solution works, so in it goes. Should the simpler one succeed at getting out, it can go in on top. If not, I'll probably add even more comments to remind myself of these facts.

Quoted 10 lines

Approaches to implement some form of coroutine local storage directly in the coroutine core code have been considered and discarded because they didn't end up being much more generic than the hash table and their performance impact on coroutines not using coroutine local storage was unclear. As the block layer uses a coroutine per I/O request, this is a fast path and we have to be careful. It's safest to just stay out of this path with code only used by the monitor.

author
#22 lore raw

Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property

Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:

Quoted 31 lines
Quoted 7 lines · Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

In qemu-system-FOO, the hash table can have only these entries:

* (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while executing a QMP command out-of-band.

* (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main thread) to a QMP monitor, while executing a QMP command in-band and in coroutine context.

* (QMP) One mapping the main thread's leader to a QMP monitor, while executing a QMP command in-band and out of coroutine context, in a bottom half.

* (HMP) One mapping the main thread's leader to an HMP monitor, while executing an HMP command out of coroutine context.

* (HMP-CO) One mapping a transient coroutine in the main thread to an HMP monitor, while executing an HMP command in coroutine context.

In-band execution is one command after the other.

Quoted 7 lines

Therefore, at most one monitor command can be executing in-band at any time.

Therefore, the hash table has at most *two* entries: one (OOB), and one of the other four.

Can you shoot any holes into my argument?

I think with human-monitor-command, you can have three mappings:

1. The main thread's leader (it is a non-coroutine QMP command) to the QMP monitor

2. With a coroutine HMP command, one mapping from the transient HMP coroutine to the transient HMP monitor (with a non-coroutine HMP command, we'd instead temporarily change the mapping from 1.)

3. The OOB entry

Quoted 4 lines

I suspect there's a simpler solution struggling to get out. But this solution works, so in it goes. Should the simpler one succeed at getting out, it can go in on top. If not, I'll probably add even more comments to remind myself of these facts.

I think the simple approach you had posted could work if you can fill in the HMP part, but I think it wasn't completely trivial and you told me not to bother for now, so I didn't. It may still be a viable path forward if you like it better.

Kevin

#23 lore raw

Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property

Quoted 38 lines · Kevin Wolf

Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:

Quoted 37 lines
Quoted 7 lines · Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

In qemu-system-FOO, the hash table can have only these entries:

* (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while executing a QMP command out-of-band.

* (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main thread) to a QMP monitor, while executing a QMP command in-band and in coroutine context.

* (QMP) One mapping the main thread's leader to a QMP monitor, while executing a QMP command in-band and out of coroutine context, in a bottom half.

* (HMP) One mapping the main thread's leader to an HMP monitor, while executing an HMP command out of coroutine context.

* (HMP-CO) One mapping a transient coroutine in the main thread to an HMP monitor, while executing an HMP command in coroutine context.

In-band execution is one command after the other.

Therefore, at most one monitor command can be executing in-band at any time.

Therefore, the hash table has at most *two* entries: one (OOB), and one of the other four.

Quoted 6 lines
Quoted 1 line

Can you shoot any holes into my argument?

I think with human-monitor-command, you can have three mappings:

1. The main thread's leader (it is a non-coroutine QMP command) to the QMP monitor

This is (QMP).

Quoted 3 lines

2. With a coroutine HMP command, one mapping from the transient HMP coroutine to the transient HMP monitor (with a non-coroutine HMP command, we'd instead temporarily change the mapping from 1.)

This is (HMP-CO).

Quoted 1 line

3. The OOB entry

This is (OOB).

To get 1. (QMP) and 2, (HMP-CO) at the same time, the in-band, non-coroutine QMP command needs to execute interleaved with the in-band, coroutine HMP command.

Such an interleaving contradicts "In-band execution is one command after the other", which is a fundamental assumption in-band commands may make. If the assumption is invalid, we got a problem. Is it?

Quoted 8 lines
Quoted 4 lines

I suspect there's a simpler solution struggling to get out. But this solution works, so in it goes. Should the simpler one succeed at getting out, it can go in on top. If not, I'll probably add even more comments to remind myself of these facts.

I think the simple approach you had posted could work if you can fill in the HMP part, but I think it wasn't completely trivial and you told me not to bother for now,

Correct. I decided to go with the code you already tested.

Quoted 2 lines

so I didn't. It may still be a viable path
forward if you like it better.

author
#24 lore raw

Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property

Am 28.09.2020 um 09:47 hat Markus Armbruster geschrieben:

Quoted 59 lines
Quoted 38 lines · Kevin Wolf

Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:

Quoted 37 lines
Quoted 7 lines · Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

In qemu-system-FOO, the hash table can have only these entries:

* (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while executing a QMP command out-of-band.

* (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main thread) to a QMP monitor, while executing a QMP command in-band and in coroutine context.

* (QMP) One mapping the main thread's leader to a QMP monitor, while executing a QMP command in-band and out of coroutine context, in a bottom half.

* (HMP) One mapping the main thread's leader to an HMP monitor, while executing an HMP command out of coroutine context.

* (HMP-CO) One mapping a transient coroutine in the main thread to an HMP monitor, while executing an HMP command in coroutine context.

In-band execution is one command after the other.

Therefore, at most one monitor command can be executing in-band at any time.

Therefore, the hash table has at most *two* entries: one (OOB), and one of the other four.

Quoted 6 lines
Quoted 1 line

Can you shoot any holes into my argument?

I think with human-monitor-command, you can have three mappings:

1. The main thread's leader (it is a non-coroutine QMP command) to the QMP monitor

This is (QMP).

Quoted 3 lines

2. With a coroutine HMP command, one mapping from the transient HMP coroutine to the transient HMP monitor (with a non-coroutine HMP command, we'd instead temporarily change the mapping from 1.)

This is (HMP-CO).

Quoted 1 line

3. The OOB entry

This is (OOB).

Quoted 7 lines

To get 1. (QMP) and 2, (HMP-CO) at the same time, the in-band, non-coroutine QMP command needs to execute interleaved with the in-band, coroutine HMP command.

Such an interleaving contradicts "In-band execution is one command after the other", which is a fundamental assumption in-band commands may make. If the assumption is invalid, we got a problem. Is it?

Interleaving, or rather executing another command in the middle of its implementation is the very purpose of human-monitor-command (which is what I was talking about, so "the in-band non-coroutine QMP command" is a very specific one).

It's the only command I can think of that is exceptional in this way and would lead to three mappings.

Kevin

#25 lore raw

Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property

Quoted 64 lines · Kevin Wolf

Am 28.09.2020 um 09:47 hat Markus Armbruster geschrieben:

Quoted 63 lines
Quoted 38 lines · Kevin Wolf

Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:

Quoted 37 lines
Quoted 7 lines · Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

In qemu-system-FOO, the hash table can have only these entries:

* (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while executing a QMP command out-of-band.

* (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main thread) to a QMP monitor, while executing a QMP command in-band and in coroutine context.

* (QMP) One mapping the main thread's leader to a QMP monitor, while executing a QMP command in-band and out of coroutine context, in a bottom half.

* (HMP) One mapping the main thread's leader to an HMP monitor, while executing an HMP command out of coroutine context.

* (HMP-CO) One mapping a transient coroutine in the main thread to an HMP monitor, while executing an HMP command in coroutine context.

In-band execution is one command after the other.

Therefore, at most one monitor command can be executing in-band at any time.

Therefore, the hash table has at most *two* entries: one (OOB), and one of the other four.

Quoted 6 lines
Quoted 1 line

Can you shoot any holes into my argument?

I think with human-monitor-command, you can have three mappings:

1. The main thread's leader (it is a non-coroutine QMP command) to the QMP monitor

This is (QMP).

Quoted 3 lines

2. With a coroutine HMP command, one mapping from the transient HMP coroutine to the transient HMP monitor (with a non-coroutine HMP command, we'd instead temporarily change the mapping from 1.)

This is (HMP-CO).

Quoted 1 line

3. The OOB entry

This is (OOB).

To get 1. (QMP) and 2, (HMP-CO) at the same time, the in-band, non-coroutine QMP command needs to execute interleaved with the in-band, coroutine HMP command.

Quoted 8 lines
Quoted 3 lines

Such an interleaving contradicts "In-band execution is one command after the other", which is a fundamental assumption in-band commands may make. If the assumption is invalid, we got a problem. Is it?

Interleaving, or rather executing another command in the middle of its implementation is the very purpose of human-monitor-command (which is what I was talking about, so "the in-band non-coroutine QMP command" is a very specific one).

Got it now, thanks.

Quoted 4 lines

It's the only command I can think of that is exceptional in this way and would lead to three mappings.

Kevin

#26 lore raw

Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property

In reply to Kevin Wolf

Additional nitpick detail on Kevin's request.

Quoted 86 lines · Kevin Wolf

This way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().

This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.

Approaches to implement some form of coroutine local storage directly in the coroutine core code have been considered and discarded because they didn't end up being much more generic than the hash table and their performance impact on coroutines not using coroutine local storage was unclear. As the block layer uses a coroutine per I/O request, this is a fast path and we have to be careful. It's safest to just stay out of this path with code only used by the monitor.

---
 include/monitor/monitor.h |  2 +-
 monitor/hmp.c             |  4 ++--
 monitor/monitor.c         | 34 +++++++++++++++++++++++++++-------
 qapi/qmp-dispatch.c       |  4 ++--
 stubs/monitor-core.c      |  2 +-
 5 files changed, 33 insertions(+), 13 deletions(-)
include/monitor/monitor.h +1 -1
index 7b0ad1de12..026f8a31b2 100644
@@ -13,7 +13,7 @@ typedef struct MonitorOptions MonitorOptions;
 extern QemuOptsList qemu_mon_opts;
 
 Monitor *monitor_cur(void);
-Monitor *monitor_set_cur(Monitor *mon);
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon);
 bool monitor_cur_is_qmp(void);
 
 void monitor_init_globals(void);
monitor/hmp.c +2 -2
index 896c670183..4b66ca1cd6 100644
@@ -1081,9 +1081,9 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
     }
 
     /* old_mon is non-NULL when called from qmp_human_monitor_command() */
-    old_mon = monitor_set_cur(&mon->common);
+    old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
     cmd->cmd(&mon->common, qdict);
-    monitor_set_cur(old_mon);
+    monitor_set_cur(qemu_coroutine_self(), old_mon);
 
     qobject_unref(qdict);
 }
monitor/monitor.c +10 -4
index be3839a7aa..629aa073ee 100644
@@ -58,29 +58,48 @@ IOThread *mon_iothread;
 /* Bottom half to dispatch the requests received from I/O thread */
 QEMUBH *qmp_dispatcher_bh;
 
-/* Protects mon_list, monitor_qapi_event_state, monitor_destroyed.  */
+/*
+ * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
+ * monitor_destroyed.
+ */
 QemuMutex monitor_lock;
 static GHashTable *monitor_qapi_event_state;
+static GHashTable *coroutine_mon; /* Maps Coroutine* to Monitor* */
 
 MonitorList mon_list;
 int mon_refcount;
 static bool monitor_destroyed;
 
-static __thread Monitor *cur_monitor;
-
 Monitor *monitor_cur(void)
 {
-    return cur_monitor;
+    Monitor *mon;
+
+    qemu_mutex_lock(&monitor_lock);
+    mon = g_hash_table_lookup(coroutine_mon, qemu_coroutine_self());
+    qemu_mutex_unlock(&monitor_lock);

monitor/monitor.c IOThread *mon_iothread; · line 89

Quoted 10 lines
monitor/monitor.c +6 -0
+
+    return mon;
 }
 
 /**
  * Sets a new current monitor and returns the old one.
+ *
+ * If a non-NULL monitor is set for a coroutine, another call resetting it to
+ * NULL is required before the coroutine terminates, otherwise a stale entry
+ * would remain in the hash table.

Wrapping the comment around column 70 or so would make it easier to read.

Quoted 19 lines
monitor/monitor.c +10 -3
  */
-Monitor *monitor_set_cur(Monitor *mon)
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
 {
-    Monitor *old_monitor = cur_monitor;
+    Monitor *old_monitor = monitor_cur();
+
+    qemu_mutex_lock(&monitor_lock);
+    if (mon) {
+        g_hash_table_replace(coroutine_mon, co, mon);
+    } else {
+        g_hash_table_remove(coroutine_mon, co);
+    }
+    qemu_mutex_unlock(&monitor_lock);
 
-    cur_monitor = mon;
     return old_monitor;
 }
 

[...]

author
#27 lore raw

PATCH v7 8/13 qapi: Add a 'coroutine' flag for commands

In reply to Kevin Wolf

This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.

The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.

---
 docs/devel/qapi-code-gen.txt            | 12 ++++++++++++
 include/qapi/qmp/dispatch.h             |  1 +
 tests/test-qmp-cmds.c                   |  4 ++++
 scripts/qapi/commands.py                | 10 +++++++---
 scripts/qapi/doc.py                     |  2 +-
 scripts/qapi/expr.py                    | 10 ++++++++--
 scripts/qapi/introspect.py              |  2 +-
 scripts/qapi/schema.py                  | 12 ++++++++----
 tests/qapi-schema/test-qapi.py          |  7 ++++---
 tests/qapi-schema/meson.build           |  1 +
 tests/qapi-schema/oob-coroutine.err     |  2 ++
 tests/qapi-schema/oob-coroutine.json    |  2 ++
 tests/qapi-schema/oob-coroutine.out     |  0
 tests/qapi-schema/qapi-schema-test.json |  1 +
 tests/qapi-schema/qapi-schema-test.out  |  2 ++
 15 files changed, 54 insertions(+), 14 deletions(-)
 create mode 100644 tests/qapi-schema/oob-coroutine.err
 create mode 100644 tests/qapi-schema/oob-coroutine.json
 create mode 100644 tests/qapi-schema/oob-coroutine.out
docs/devel/qapi-code-gen.txt +12 -0
index f3e7ced212..36daa9b5f8 100644
@@ -472,6 +472,7 @@ Syntax:
                 '*gen': false,
                 '*allow-oob': true,
                 '*allow-preconfig': true,
+                '*coroutine': true,
                 '*if': COND,
                 '*features': FEATURES }
 
@@ -596,6 +597,17 @@ before the machine is built.  It defaults to false.  For example:
 QMP is available before the machine is built only when QEMU was
 started with --preconfig.
 
+Member 'coroutine' tells the QMP dispatcher whether the command handler
+is safe to be run in a coroutine.  It defaults to false.  If it is true,
+the command handler is called from coroutine context and may yield while
+waiting for an external event (such as I/O completion) in order to avoid
+blocking the guest and other background operations.
+
+It is an error to specify both 'coroutine': true and 'allow-oob': true
+for a command.  We don't currently have a use case for both together and
+without a use case, it's not entirely clear what the semantics should
+be.
+
 The optional 'if' member specifies a conditional.  See "Configuring
 the schema" below for more on this.
 
include/qapi/qmp/dispatch.h +1 -0
index 0c2f467028..9fd2b720a7 100644
@@ -25,6 +25,7 @@ typedef enum QmpCommandOptions
     QCO_NO_SUCCESS_RESP       =  (1U << 0),
     QCO_ALLOW_OOB             =  (1U << 1),
     QCO_ALLOW_PRECONFIG       =  (1U << 2),
+    QCO_COROUTINE             =  (1U << 3),
 } QmpCommandOptions;
 
 typedef struct QmpCommand
tests/test-qmp-cmds.c +4 -0
index 5f1b245e19..d3413bfef0 100644
@@ -36,6 +36,10 @@ void qmp_cmd_success_response(Error **errp)
 {
 }
 
+void qmp_coroutine_cmd(Error **errp)
+{
+}
+
 Empty2 *qmp_user_def_cmd0(Error **errp)
 {
     return g_new0(Empty2, 1);
scripts/qapi/commands.py +7 -3
index 3cf9e1110b..6e6fc94a14 100644
@@ -176,7 +176,8 @@ out:
     return ret
 
 
-def gen_register_command(name, success_response, allow_oob, allow_preconfig):
+def gen_register_command(name, success_response, allow_oob, allow_preconfig,
+                         coroutine):
     options = []
 
     if not success_response:
@@ -185,6 +186,8 @@ def gen_register_command(name, success_response, allow_oob, allow_preconfig):
         options += ['QCO_ALLOW_OOB']
     if allow_preconfig:
         options += ['QCO_ALLOW_PRECONFIG']
+    if coroutine:
+        options += ['QCO_COROUTINE']
 
     if not options:
         options = ['QCO_NO_OPTIONS']
@@ -267,7 +270,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
 
     def visit_command(self, name, info, ifcond, features,
                       arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+                      allow_oob, allow_preconfig, coroutine):
         if not gen:
             return
         # FIXME: If T is a user-defined type, the user is responsible
@@ -285,7 +288,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
             self._genh.add(gen_marshal_decl(name))
             self._genc.add(gen_marshal(name, arg_type, boxed, ret_type))
             self._regy.add(gen_register_command(name, success_response,
-                                                allow_oob, allow_preconfig))
+                                                allow_oob, allow_preconfig,
+                                                coroutine))
 
 
 def gen_commands(schema, output_dir, prefix):
scripts/qapi/doc.py +1 -1
index 92f584edcf..11771de923 100644
@@ -264,7 +264,7 @@ class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
 
     def visit_command(self, name, info, ifcond, features,
                       arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+                      allow_oob, allow_preconfig, coroutine):
         doc = self.cur_doc
         self._gen.add(texi_msg('Command', doc, ifcond,
                                texi_arguments(doc,
scripts/qapi/expr.py +8 -2
index 2942520399..928cd1eb5c 100644
@@ -88,10 +88,16 @@ def check_flags(expr, info):
         if key in expr and expr[key] is not False:
             raise QAPISemError(
                 info, "flag '%s' may only use false value" % key)
-    for key in ['boxed', 'allow-oob', 'allow-preconfig']:
+    for key in ['boxed', 'allow-oob', 'allow-preconfig', 'coroutine']:
         if key in expr and expr[key] is not True:
             raise QAPISemError(
                 info, "flag '%s' may only use true value" % key)
+    if 'allow-oob' in expr and 'coroutine' in expr:
+        # This is not necessarily a fundamental incompatibility, but we don't
+        # have a use case and the desired semantics isn't obvious. The simplest
+        # solution is to forbid it until we get a use case for it.
+        raise QAPISemError(info, "flags 'allow-oob' and 'coroutine' "
+                                 "are incompatible")
 
 
 def check_if(expr, info, source):
@@ -342,7 +348,7 @@ def check_exprs(exprs):
                        ['command'],
                        ['data', 'returns', 'boxed', 'if', 'features',
                         'gen', 'success-response', 'allow-oob',
-                        'allow-preconfig'])
+                        'allow-preconfig', 'coroutine'])
             normalize_members(expr.get('data'))
             check_command(expr, info)
         elif meta == 'event':
scripts/qapi/introspect.py +1 -1
index 23652be810..5907b09cd5 100644
@@ -216,7 +216,7 @@ const QLitObject %(c_name)s = %(c_string)s;
 
     def visit_command(self, name, info, ifcond, features,
                       arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+                      allow_oob, allow_preconfig, coroutine):
         arg_type = arg_type or self._schema.the_empty_object_type
         ret_type = ret_type or self._schema.the_empty_object_type
         obj = {'arg-type': self._use_type(arg_type),
scripts/qapi/schema.py +8 -4
index 78309a00f0..c44d391c3f 100644
@@ -128,7 +128,7 @@ class QAPISchemaVisitor:
 
     def visit_command(self, name, info, ifcond, features,
                       arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+                      allow_oob, allow_preconfig, coroutine):
         pass
 
     def visit_event(self, name, info, ifcond, features, arg_type, boxed):
@@ -713,7 +713,8 @@ class QAPISchemaCommand(QAPISchemaEntity):
 
     def __init__(self, name, info, doc, ifcond, features,
                  arg_type, ret_type,
-                 gen, success_response, boxed, allow_oob, allow_preconfig):
+                 gen, success_response, boxed, allow_oob, allow_preconfig,
+                 coroutine):
         super().__init__(name, info, doc, ifcond, features)
         assert not arg_type or isinstance(arg_type, str)
         assert not ret_type or isinstance(ret_type, str)
@@ -726,6 +727,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
         self.boxed = boxed
         self.allow_oob = allow_oob
         self.allow_preconfig = allow_preconfig
+        self.coroutine = coroutine
 
     def check(self, schema):
         super().check(schema)
@@ -768,7 +770,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
         visitor.visit_command(
             self.name, self.info, self.ifcond, self.features,
             self.arg_type, self.ret_type, self.gen, self.success_response,
-            self.boxed, self.allow_oob, self.allow_preconfig)
+            self.boxed, self.allow_oob, self.allow_preconfig, self.coroutine)
 
 
 class QAPISchemaEvent(QAPISchemaEntity):
@@ -1074,6 +1076,7 @@ class QAPISchema:
         boxed = expr.get('boxed', False)
         allow_oob = expr.get('allow-oob', False)
         allow_preconfig = expr.get('allow-preconfig', False)
+        coroutine = expr.get('coroutine', False)
         ifcond = expr.get('if')
         features = self._make_features(expr.get('features'), info)
         if isinstance(data, OrderedDict):
@@ -1086,7 +1089,8 @@ class QAPISchema:
         self._def_entity(QAPISchemaCommand(name, info, doc, ifcond, features,
                                            data, rets,
                                            gen, success_response,
-                                           boxed, allow_oob, allow_preconfig))
+                                           boxed, allow_oob, allow_preconfig,
+                                           coroutine))
 
     def _def_event(self, expr, info, doc):
         name = expr['event']
tests/qapi-schema/test-qapi.py +4 -3
index f396b471eb..e8db9d09d9 100755
@@ -68,12 +68,13 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
 
     def visit_command(self, name, info, ifcond, features,
                       arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+                      allow_oob, allow_preconfig, coroutine):
         print('command %s %s -> %s'
               % (name, arg_type and arg_type.name,
                  ret_type and ret_type.name))
-        print('    gen=%s success_response=%s boxed=%s oob=%s preconfig=%s'
-              % (gen, success_response, boxed, allow_oob, allow_preconfig))
+        print('    gen=%s success_response=%s boxed=%s oob=%s preconfig=%s%s'
+              % (gen, success_response, boxed, allow_oob, allow_preconfig,
+                 " coroutine=True" if coroutine else ""))
         self._print_if(ifcond)
         self._print_features(features)
 
tests/qapi-schema/meson.build +1 -0
index c87d141417..7a9bc69cc0 100644
@@ -141,6 +141,7 @@ schemas = [
   'nested-struct-data.json',
   'nested-struct-data-invalid-dict.json',
   'non-objects.json',
+  'oob-coroutine.json',
   'oob-test.json',
   'allow-preconfig-test.json',
   'pragma-doc-required-crap.json',
tests/qapi-schema/oob-coroutine.err new file +2 -0
new file mode 100644
index 0000000000..c01a4992bd
@@ -0,0 +1,2 @@
+oob-coroutine.json: In command 'oob-command-1':
+oob-coroutine.json:2: flags 'allow-oob' and 'coroutine' are incompatible
tests/qapi-schema/oob-coroutine.json new file +2 -0
new file mode 100644
index 0000000000..0f67663bcd
@@ -0,0 +1,2 @@
+# Check that incompatible flags allow-oob and coroutine are rejected
+{ 'command': 'oob-command-1', 'allow-oob': true, 'coroutine': true }
tests/qapi-schema/oob-coroutine.out new file +0 -0
new file mode 100644
index 0000000000..e69de29bb2
tests/qapi-schema/qapi-schema-test.json +1 -0
index 3a9f2cbb33..63f92adf68 100644
@@ -148,6 +148,7 @@
   'returns': 'UserDefTwo' }
 
 { 'command': 'cmd-success-response', 'data': {}, 'success-response': false }
+{ 'command': 'coroutine-cmd', 'data': {}, 'coroutine': true }
 
 # Returning a non-dictionary requires a name from the whitelist
 { 'command': 'guest-get-time', 'data': {'a': 'int', '*b': 'int' },
tests/qapi-schema/qapi-schema-test.out +2 -0
index 891b4101e0..8868ca0dca 100644
@@ -203,6 +203,8 @@ command user_def_cmd2 q_obj_user_def_cmd2-arg -> UserDefTwo
     gen=True success_response=True boxed=False oob=False preconfig=False
 command cmd-success-response None -> None
     gen=True success_response=False boxed=False oob=False preconfig=False
+command coroutine-cmd None -> None
+    gen=True success_response=True boxed=False oob=False preconfig=False coroutine=True
 object q_obj_guest-get-time-arg
     member a: int optional=False
     member b: int optional=True
Signature · 1 line
2.25.4
#28 lore raw

Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands

Quoted 45 lines · Kevin Wolf

This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.

The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.

---
 docs/devel/qapi-code-gen.txt            | 12 ++++++++++++
 include/qapi/qmp/dispatch.h             |  1 +
 tests/test-qmp-cmds.c                   |  4 ++++
 scripts/qapi/commands.py                | 10 +++++++---
 scripts/qapi/doc.py                     |  2 +-
 scripts/qapi/expr.py                    | 10 ++++++++--
 scripts/qapi/introspect.py              |  2 +-
 scripts/qapi/schema.py                  | 12 ++++++++----
 tests/qapi-schema/test-qapi.py          |  7 ++++---
 tests/qapi-schema/meson.build           |  1 +
 tests/qapi-schema/oob-coroutine.err     |  2 ++
 tests/qapi-schema/oob-coroutine.json    |  2 ++
 tests/qapi-schema/oob-coroutine.out     |  0
 tests/qapi-schema/qapi-schema-test.json |  1 +
 tests/qapi-schema/qapi-schema-test.out  |  2 ++
 15 files changed, 54 insertions(+), 14 deletions(-)
 create mode 100644 tests/qapi-schema/oob-coroutine.err
 create mode 100644 tests/qapi-schema/oob-coroutine.json
 create mode 100644 tests/qapi-schema/oob-coroutine.out
docs/devel/qapi-code-gen.txt +1 -0
index f3e7ced212..36daa9b5f8 100644
@@ -472,6 +472,7 @@ Syntax:
                 '*gen': false,
                 '*allow-oob': true,
                 '*allow-preconfig': true,
+                '*coroutine': true,
                 '*if': COND,
                 '*features': FEATURES }
 

docs/devel/qapi-code-gen.txt before the machine is built. It defaults to false. For example: · line 601

Quoted 5 lines
docs/devel/qapi-code-gen.txt +2 -0
@@ -596,6 +597,17 @@ before the machine is built.  It defaults to false.  For example:
 QMP is available before the machine is built only when QEMU was
 started with --preconfig.
 
+Member 'coroutine' tells the QMP dispatcher whether the command handler
+is safe to be run in a coroutine.

We need to document what exactly makes a command handler coroutine safe / unsafe. We discussed this at some length in review of PATCH v4 1/4:

    Message-ID: <[email protected]>
    https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05015.html

I'd be willing to accept a follow-up patch, if that's more convenient for you.

docs/devel/qapi-code-gen.txt before the machine is built. It defaults to false. For example: · line 603

Quoted 2 lines
docs/devel/qapi-code-gen.txt +1 -0
                                    It defaults to false.  If it is true,
+the command handler is called from coroutine context and may yield while

Is it *always* called from coroutine context, or could it be called outside coroutine context, too? I guess the former, thanks to PATCH 10, and as long as we diligently mark HMP commands that such call QMP handlers, like you do in PATCH 13.

What's the worst than can happen when we neglect to mark such an HMP command?

Quoted 10 lines
docs/devel/qapi-code-gen.txt +8 -0
+waiting for an external event (such as I/O completion) in order to avoid
+blocking the guest and other background operations.
+
+It is an error to specify both 'coroutine': true and 'allow-oob': true
+for a command.  We don't currently have a use case for both together and
+without a use case, it's not entirely clear what the semantics should
+be.
+
 The optional 'if' member specifies a conditional.  See "Configuring
 the schema" below for more on this.

[...]

author
#29 lore raw

Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands

Am 14.09.2020 um 17:15 hat Markus Armbruster geschrieben:

Quoted 56 lines
Quoted 45 lines · Kevin Wolf

This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.

The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.

---
 docs/devel/qapi-code-gen.txt            | 12 ++++++++++++
 include/qapi/qmp/dispatch.h             |  1 +
 tests/test-qmp-cmds.c                   |  4 ++++
 scripts/qapi/commands.py                | 10 +++++++---
 scripts/qapi/doc.py                     |  2 +-
 scripts/qapi/expr.py                    | 10 ++++++++--
 scripts/qapi/introspect.py              |  2 +-
 scripts/qapi/schema.py                  | 12 ++++++++----
 tests/qapi-schema/test-qapi.py          |  7 ++++---
 tests/qapi-schema/meson.build           |  1 +
 tests/qapi-schema/oob-coroutine.err     |  2 ++
 tests/qapi-schema/oob-coroutine.json    |  2 ++
 tests/qapi-schema/oob-coroutine.out     |  0
 tests/qapi-schema/qapi-schema-test.json |  1 +
 tests/qapi-schema/qapi-schema-test.out  |  2 ++
 15 files changed, 54 insertions(+), 14 deletions(-)
 create mode 100644 tests/qapi-schema/oob-coroutine.err
 create mode 100644 tests/qapi-schema/oob-coroutine.json
 create mode 100644 tests/qapi-schema/oob-coroutine.out
docs/devel/qapi-code-gen.txt +1 -0
index f3e7ced212..36daa9b5f8 100644
@@ -472,6 +472,7 @@ Syntax:
                 '*gen': false,
                 '*allow-oob': true,
                 '*allow-preconfig': true,
+                '*coroutine': true,
                 '*if': COND,
                 '*features': FEATURES }
 

docs/devel/qapi-code-gen.txt before the machine is built. It defaults to false. For example: · line 601

Quoted 5 lines
docs/devel/qapi-code-gen.txt +2 -0
@@ -596,6 +597,17 @@ before the machine is built.  It defaults to false.  For example:
 QMP is available before the machine is built only when QEMU was
 started with --preconfig.
 
+Member 'coroutine' tells the QMP dispatcher whether the command handler
+is safe to be run in a coroutine.

We need to document what exactly makes a command handler coroutine safe / unsafe. We discussed this at some length in review of PATCH v4 1/4:

Quoted 5 lines
    Message-ID: <[email protected]>
    https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05015.html

I'd be willing to accept a follow-up patch, if that's more convenient for you.

Did we ever arrive at a conclusion for a good definition?

I mean I can give a definition like "it's coroutine safe if it results in the right behaviour even when called in coroutine context", but that's not really helpful.

FWIW, I would also have a hard time giving a much better definition than this for thread safety.

Quoted 7 lines

docs/devel/qapi-code-gen.txt before the machine is built. It defaults to false. For example: · line 603

Quoted 2 lines
docs/devel/qapi-code-gen.txt +1 -0
                                    It defaults to false.  If it is true,
+the command handler is called from coroutine context and may yield while

Is it *always* called from coroutine context, or could it be called outside coroutine context, too? I guess the former, thanks to PATCH 10, and as long as we diligently mark HMP commands that such call QMP handlers, like you do in PATCH 13.

Yes, it must *always* be called from coroutine context. This is the reason why I included HMP after all, even though I'm really mostly just interested in QMP.

Quoted 2 lines

What's the worst than can happen when we neglect to mark such an HMP command?

When the command handler tries to yield and it's not in coroutine context, it will abort(). If it never tries to yield, I think it would just work - but of course, the ability to yield is the only reason why you would want to run a command handler in a coroutine.

Kevin

#30 lore raw

Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands

Quoted 60 lines · Kevin Wolf

Am 14.09.2020 um 17:15 hat Markus Armbruster geschrieben:

Quoted 59 lines
Quoted 45 lines · Kevin Wolf

This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.

The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.

---
 docs/devel/qapi-code-gen.txt            | 12 ++++++++++++
 include/qapi/qmp/dispatch.h             |  1 +
 tests/test-qmp-cmds.c                   |  4 ++++
 scripts/qapi/commands.py                | 10 +++++++---
 scripts/qapi/doc.py                     |  2 +-
 scripts/qapi/expr.py                    | 10 ++++++++--
 scripts/qapi/introspect.py              |  2 +-
 scripts/qapi/schema.py                  | 12 ++++++++----
 tests/qapi-schema/test-qapi.py          |  7 ++++---
 tests/qapi-schema/meson.build           |  1 +
 tests/qapi-schema/oob-coroutine.err     |  2 ++
 tests/qapi-schema/oob-coroutine.json    |  2 ++
 tests/qapi-schema/oob-coroutine.out     |  0
 tests/qapi-schema/qapi-schema-test.json |  1 +
 tests/qapi-schema/qapi-schema-test.out  |  2 ++
 15 files changed, 54 insertions(+), 14 deletions(-)
 create mode 100644 tests/qapi-schema/oob-coroutine.err
 create mode 100644 tests/qapi-schema/oob-coroutine.json
 create mode 100644 tests/qapi-schema/oob-coroutine.out
docs/devel/qapi-code-gen.txt +1 -0
index f3e7ced212..36daa9b5f8 100644
@@ -472,6 +472,7 @@ Syntax:
                 '*gen': false,
                 '*allow-oob': true,
                 '*allow-preconfig': true,
+                '*coroutine': true,
                 '*if': COND,
                 '*features': FEATURES }
 

docs/devel/qapi-code-gen.txt before the machine is built. It defaults to false. For example: · line 601

Quoted 5 lines
docs/devel/qapi-code-gen.txt +2 -0
@@ -596,6 +597,17 @@ before the machine is built.  It defaults to false.  For example:
 QMP is available before the machine is built only when QEMU was
 started with --preconfig.
 
+Member 'coroutine' tells the QMP dispatcher whether the command handler
+is safe to be run in a coroutine.

We need to document what exactly makes a command handler coroutine safe / unsafe. We discussed this at some length in review of PATCH v4 1/4:

    Message-ID: <[email protected]>
    https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05015.html
Quoted 8 lines
Quoted 2 lines

I'd be willing to accept a follow-up patch, if that's more convenient for you.

Did we ever arrive at a conclusion for a good definition?

I mean I can give a definition like "it's coroutine safe if it results in the right behaviour even when called in coroutine context", but that's not really helpful.

If an actual definition is out of reach, we should still say *something*. Even a mere "it's complicated" would help, because it warns the reader he's on thin ice.

Can we give examples for "unsafe"? You mentioned nested event loops.

Quoted 2 lines

FWIW, I would also have a hard time giving a much better definition than this for thread safety.

For what it's worth, here's how POSIX.1-2017 defined "thread-safe":

A thread-safe function can be safely invoked concurrently with other calls to the same function, or with calls to any other thread-safe functions, by multiple threads. Each function defined in the System Interfaces volume of POSIX.1-2017 is thread-safe unless explicitly stated otherwise. Examples are any "pure" function, a function which holds a mutex locked while it is accessing static storage, or objects shared among threads.

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_407

Quoted 12 lines
Quoted 7 lines

docs/devel/qapi-code-gen.txt before the machine is built. It defaults to false. For example: · line 603

Quoted 2 lines
docs/devel/qapi-code-gen.txt +1 -0
                                    It defaults to false.  If it is true,
+the command handler is called from coroutine context and may yield while

Is it *always* called from coroutine context, or could it be called outside coroutine context, too? I guess the former, thanks to PATCH 10, and as long as we diligently mark HMP commands that such call QMP handlers, like you do in PATCH 13.

Yes, it must *always* be called from coroutine context. This is the reason why I included HMP after all, even though I'm really mostly just interested in QMP.

Quoted 7 lines
Quoted 2 lines

What's the worst than can happen when we neglect to mark such an HMP command?

When the command handler tries to yield and it's not in coroutine context, it will abort(). If it never tries to yield, I think it would just work - but of course, the ability to yield is the only reason why you would want to run a command handler in a coroutine.

Let's spell this out. After your paragraph

Member 'coroutine' tells the QMP dispatcher whether the command handler is safe to be run in a coroutine. It defaults to false. If it is true, the command handler is called from coroutine context and may yield while waiting for an external event (such as I/O completion) in order to avoid blocking the guest and other background operations.

add something like

Since the command handler may assume coroutine-context, any callers other than the QMP dispatcher must also call it in coroutine context. In particular, HMP commands calling such a QMP command handler must be marked .coroutine = true in hmp-commands.hx.

Patch ordering issue: this becomes possible only in PATCH 10. Possible solutions:

* Add the last sentence only in PATCH 10.

* Write "HMP commands cannot call such a QMP command handler" in PATCH 8, replace it in PATCH 10.

* Add a "TODO make that possible" line here, drop it in PATCH 10.

* Reorder patches.

I don't like the first one much. Anyway, up to you.

#31 lore raw

Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands

In reply to Kevin Wolf

Additional nitpick detail on Kevin's request.

Quoted 11 lines · Kevin Wolf

This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.

The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.

[...]

Quoted 9 lines
scripts/qapi/expr.py +0 -1
index 2942520399..928cd1eb5c 100644
@@ -88,10 +88,16 @@ def check_flags(expr, info):
         if key in expr and expr[key] is not False:
             raise QAPISemError(
                 info, "flag '%s' may only use false value" % key)
-    for key in ['boxed', 'allow-oob', 'allow-preconfig']:

scripts/qapi/expr.py def check_flags(expr, info): · line 100

Quoted 10 lines
scripts/qapi/expr.py +7 -0
+    for key in ['boxed', 'allow-oob', 'allow-preconfig', 'coroutine']:
         if key in expr and expr[key] is not True:
             raise QAPISemError(
                 info, "flag '%s' may only use true value" % key)
+    if 'allow-oob' in expr and 'coroutine' in expr:
+        # This is not necessarily a fundamental incompatibility, but we don't
+        # have a use case and the desired semantics isn't obvious. The simplest
+        # solution is to forbid it until we get a use case for it.
+        raise QAPISemError(info, "flags 'allow-oob' and 'coroutine' "
+                                 "are incompatible")

PEP 8: For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

PEP 8: You should use two spaces after a sentence-ending period in multi- sentence comments, except after the final sentence.

Quoted 3 lines
scripts/qapi/expr.py +0 -0
 def check_if(expr, info, source):

[...]

#32 lore raw

Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands

In reply to Kevin Wolf

Hit send too fast.

Quoted 11 lines · Kevin Wolf

This patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.

The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.

[...]

Quoted 32 lines
scripts/qapi/schema.py +4 -2
index 78309a00f0..c44d391c3f 100644
@@ -128,7 +128,7 @@ class QAPISchemaVisitor:
 
     def visit_command(self, name, info, ifcond, features,
                       arg_type, ret_type, gen, success_response, boxed,
-                      allow_oob, allow_preconfig):
+                      allow_oob, allow_preconfig, coroutine):
         pass
 
     def visit_event(self, name, info, ifcond, features, arg_type, boxed):
@@ -713,7 +713,8 @@ class QAPISchemaCommand(QAPISchemaEntity):
 
     def __init__(self, name, info, doc, ifcond, features,
                  arg_type, ret_type,
-                 gen, success_response, boxed, allow_oob, allow_preconfig):
+                 gen, success_response, boxed, allow_oob, allow_preconfig,
+                 coroutine):
         super().__init__(name, info, doc, ifcond, features)
         assert not arg_type or isinstance(arg_type, str)
         assert not ret_type or isinstance(ret_type, str)
@@ -726,6 +727,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
         self.boxed = boxed
         self.allow_oob = allow_oob
         self.allow_preconfig = allow_preconfig
+        self.coroutine = coroutine
 
     def check(self, schema):
         super().check(schema)

scripts/qapi/schema.py class QAPISchemaCommand(QAPISchemaEntity): · line 773

Quoted 5 lines
scripts/qapi/schema.py +1 -1
@@ -768,7 +770,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
         visitor.visit_command(
             self.name, self.info, self.ifcond, self.features,
             self.arg_type, self.ret_type, self.gen, self.success_response,
-            self.boxed, self.allow_oob, self.allow_preconfig)
+            self.boxed, self.allow_oob, self.allow_preconfig, self.coroutine)

Recommend to break the line after preconfig, like you do elsewhere.

Quoted 12 lines
scripts/qapi/schema.py +1 -0
 class QAPISchemaEvent(QAPISchemaEntity):
@@ -1074,6 +1076,7 @@ class QAPISchema:
         boxed = expr.get('boxed', False)
         allow_oob = expr.get('allow-oob', False)
         allow_preconfig = expr.get('allow-preconfig', False)
+        coroutine = expr.get('coroutine', False)
         ifcond = expr.get('if')
         features = self._make_features(expr.get('features'), info)
         if isinstance(data, OrderedDict):

scripts/qapi/schema.py class QAPISchema: · line 1093

Quoted 6 lines
scripts/qapi/schema.py +2 -1
@@ -1086,7 +1089,8 @@ class QAPISchema:
         self._def_entity(QAPISchemaCommand(name, info, doc, ifcond, features,
                                            data, rets,
                                            gen, success_response,
-                                           boxed, allow_oob, allow_preconfig))
+                                           boxed, allow_oob, allow_preconfig,
+                                           coroutine))

Preexisting: the arguments are kind of squeezed onto the right margin. Hanging indent would avoid that. Feel free to ignore.

Quoted 3 lines
scripts/qapi/schema.py +0 -0
     def _def_event(self, expr, info, doc):
         name = expr['event']

[...]

author
#33 lore raw

PATCH v7 9/13 qmp: Move dispatcher to a coroutine

In reply to Kevin Wolf

This moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events.

For commands that are not declared safe to run in a coroutine, the dispatcher drops out of coroutine context by calling the QMP command handler from a bottom half.

---
 include/qapi/qmp/dispatch.h |   1 +
 monitor/monitor-internal.h  |   6 +-
 monitor/monitor.c           |  55 +++++++++++++---
 monitor/qmp.c               | 122 +++++++++++++++++++++++++++---------
 qapi/qmp-dispatch.c         |  61 ++++++++++++++++--
 qapi/qmp-registry.c         |   3 +
 util/aio-posix.c            |   8 ++-
 7 files changed, 210 insertions(+), 46 deletions(-)
include/qapi/qmp/dispatch.h +1 -0
index 9fd2b720a7..af8d96c570 100644
@@ -31,6 +31,7 @@ typedef enum QmpCommandOptions
 typedef struct QmpCommand
 {
     const char *name;
+    /* Runs in coroutine context if QCO_COROUTINE is set */
     QmpCommandFunc *fn;
     QmpCommandOptions options;
     QTAILQ_ENTRY(QmpCommand) node;
monitor/monitor-internal.h +4 -2
index b39e03b744..b55d6df07f 100644
@@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
 
 typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;
 extern IOThread *mon_iothread;
-extern QEMUBH *qmp_dispatcher_bh;
+extern Coroutine *qmp_dispatcher_co;
+extern bool qmp_dispatcher_co_shutdown;
+extern bool qmp_dispatcher_co_busy;
 extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
 extern QemuMutex monitor_lock;
 extern MonitorList mon_list;
@@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);
 
 void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
 void monitor_data_destroy_qmp(MonitorQMP *mon);
-void monitor_qmp_bh_dispatcher(void *data);
+void coroutine_fn monitor_qmp_dispatcher_co(void *data);
 
 int get_monitor_def(int64_t *pval, const char *name);
 void help_cmd(Monitor *mon, const char *name);
monitor/monitor.c +47 -8

73 lines not on this page · raw message

monitor/qmp.c +92 -30

148 lines not on this page · raw message

qapi/qmp-dispatch.c +56 -5

86 lines not on this page · raw message

qapi/qmp-registry.c +3 -0
index d0f9a1d3e3..58c65b5052 100644
@@ -20,6 +20,9 @@ void qmp_register_command(QmpCommandList *cmds, const char *name,
 {
     QmpCommand *cmd = g_malloc0(sizeof(*cmd));
 
+    /* QCO_COROUTINE and QCO_ALLOW_OOB are incompatible for now */
+    assert(!((options & QCO_COROUTINE) && (options & QCO_ALLOW_OOB)));
+
     cmd->name = name;
     cmd->fn = fn;
     cmd->enabled = true;
util/aio-posix.c +7 -1
index f7f13ebfc2..30bb21d699 100644
@@ -15,6 +15,7 @@
 
 #include "qemu/osdep.h"
 #include "block/block.h"
+#include "qemu/main-loop.h"
 #include "qemu/rcu.h"
 #include "qemu/rcu_queue.h"
 #include "qemu/sockets.h"
@@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
      * There cannot be two concurrent aio_poll calls for the same AioContext (or
      * an aio_poll concurrent with a GSource prepare/check/dispatch callback).
      * We rely on this below to avoid slow locked accesses to ctx->notify_me.
+     *
+     * aio_poll() may only be called in the AioContext's thread. iohandler_ctx
+     * is special in that it runs in the main thread, but that thread's context
+     * is qemu_aio_context.
      */
-    assert(in_aio_context_home_thread(ctx));
+    assert(in_aio_context_home_thread(ctx == iohandler_get_aio_context() ?
+                                      qemu_get_aio_context() : ctx));
 
     qemu_lockcnt_inc(&ctx->list_lock);
 
Signature · 1 line
2.25.4
#34 lore raw

Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine

Quoted 358 lines · Kevin Wolf

This moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events.

For commands that are not declared safe to run in a coroutine, the dispatcher drops out of coroutine context by calling the QMP command handler from a bottom half.

---
 include/qapi/qmp/dispatch.h |   1 +
 monitor/monitor-internal.h  |   6 +-
 monitor/monitor.c           |  55 +++++++++++++---
 monitor/qmp.c               | 122 +++++++++++++++++++++++++++---------
 qapi/qmp-dispatch.c         |  61 ++++++++++++++++--
 qapi/qmp-registry.c         |   3 +
 util/aio-posix.c            |   8 ++-
 7 files changed, 210 insertions(+), 46 deletions(-)
include/qapi/qmp/dispatch.h +1 -0
index 9fd2b720a7..af8d96c570 100644
@@ -31,6 +31,7 @@ typedef enum QmpCommandOptions
 typedef struct QmpCommand
 {
     const char *name;
+    /* Runs in coroutine context if QCO_COROUTINE is set */
     QmpCommandFunc *fn;
     QmpCommandOptions options;
     QTAILQ_ENTRY(QmpCommand) node;
monitor/monitor-internal.h +4 -2
index b39e03b744..b55d6df07f 100644
@@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
 
 typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;
 extern IOThread *mon_iothread;
-extern QEMUBH *qmp_dispatcher_bh;
+extern Coroutine *qmp_dispatcher_co;
+extern bool qmp_dispatcher_co_shutdown;
+extern bool qmp_dispatcher_co_busy;
 extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
 extern QemuMutex monitor_lock;
 extern MonitorList mon_list;
@@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);
 
 void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
 void monitor_data_destroy_qmp(MonitorQMP *mon);
-void monitor_qmp_bh_dispatcher(void *data);
+void coroutine_fn monitor_qmp_dispatcher_co(void *data);
 
 int get_monitor_def(int64_t *pval, const char *name);
 void help_cmd(Monitor *mon, const char *name);
monitor/monitor.c +47 -8
index 629aa073ee..ac2722bf91 100644
@@ -55,8 +55,32 @@ typedef struct {
 /* Shared monitor I/O thread */
 IOThread *mon_iothread;
 
-/* Bottom half to dispatch the requests received from I/O thread */
-QEMUBH *qmp_dispatcher_bh;
+/* Coroutine to dispatch the requests received from I/O thread */
+Coroutine *qmp_dispatcher_co;
+
+/* Set to true when the dispatcher coroutine should terminate */
+bool qmp_dispatcher_co_shutdown;
+
+/*
+ * qmp_dispatcher_co_busy is used for synchronisation between the
+ * monitor thread and the main thread to ensure that the dispatcher
+ * coroutine never gets scheduled a second time when it's already
+ * scheduled (scheduling the same coroutine twice is forbidden).
+ *
+ * It is true if the coroutine is active and processing requests.
+ * Additional requests may then be pushed onto mon->qmp_requests,
+ * and @qmp_dispatcher_co_shutdown may be set without further ado.
+ * @qmp_dispatcher_co_busy must not be woken up in this case.
+ *
+ * If false, you also have to set @qmp_dispatcher_co_busy to true and
+ * wake up @qmp_dispatcher_co after pushing the new requests.
+ *
+ * The coroutine will automatically change this variable back to false
+ * before it yields.  Nobody else may set the variable to false.
+ *
+ * Access must be atomic for thread safety.
+ */
+bool qmp_dispatcher_co_busy;
 
 /*
  * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
@@ -623,9 +647,24 @@ void monitor_cleanup(void)
     }
     qemu_mutex_unlock(&monitor_lock);
 
-    /* QEMUBHs needs to be deleted before destroying the I/O thread */
-    qemu_bh_delete(qmp_dispatcher_bh);
-    qmp_dispatcher_bh = NULL;
+    /*
+     * The dispatcher needs to stop before destroying the I/O thread.
+     *
+     * We need to poll both qemu_aio_context and iohandler_ctx to make
+     * sure that the dispatcher coroutine keeps making progress and
+     * eventually terminates.  qemu_aio_context is automatically
+     * polled by calling AIO_WAIT_WHILE on it, but we must poll
+     * iohandler_ctx manually.
+     */
+    qmp_dispatcher_co_shutdown = true;
+    if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+        aio_co_wake(qmp_dispatcher_co);
+    }
+
+    AIO_WAIT_WHILE(qemu_get_aio_context(),
+                   (aio_poll(iohandler_get_aio_context(), false),
+                    atomic_mb_read(&qmp_dispatcher_co_busy)));
+
     if (mon_iothread) {
         iothread_destroy(mon_iothread);
         mon_iothread = NULL;
@@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
      * have commands assuming that context.  It would be nice to get
      * rid of those assumptions.
      */
-    qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
-                                   monitor_qmp_bh_dispatcher,
-                                   NULL);
+    qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);
+    atomic_mb_set(&qmp_dispatcher_co_busy, true);
+    aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
 }
 
 int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)
monitor/qmp.c +92 -30
index 922fdb5541..69f6e93f38 100644
@@ -133,6 +133,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
     }
 }
 
+/*
+ * Runs outside of coroutine context for OOB commands, but in
+ * coroutine context for everything else.
+ */
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
     QDict *rsp;
@@ -205,43 +209,99 @@ static QMPRequest *monitor_qmp_requests_pop_any_with_lock(void)
     return req_obj;
 }
 
-void monitor_qmp_bh_dispatcher(void *data)
+void coroutine_fn monitor_qmp_dispatcher_co(void *data)
 {
-    QMPRequest *req_obj = monitor_qmp_requests_pop_any_with_lock();
+    QMPRequest *req_obj = NULL;
     QDict *rsp;
     bool need_resume;
     MonitorQMP *mon;
 
-    if (!req_obj) {
-        return;
-    }
+    while (true) {
+        assert(atomic_mb_read(&qmp_dispatcher_co_busy) == true);
 
-    mon = req_obj->mon;
-    /*  qmp_oob_enabled() might change after "qmp_capabilities" */
-    need_resume = !qmp_oob_enabled(mon) ||
-        mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
-    qemu_mutex_unlock(&mon->qmp_queue_lock);
-    if (req_obj->req) {
-        QDict *qdict = qobject_to(QDict, req_obj->req);
-        QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
-        trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
-        monitor_qmp_dispatch(mon, req_obj->req);
-    } else {
-        assert(req_obj->err);
-        rsp = qmp_error_response(req_obj->err);
-        req_obj->err = NULL;
-        monitor_qmp_respond(mon, rsp);
-        qobject_unref(rsp);
-    }
+        /*
+         * Mark the dispatcher as not busy already here so that we
+         * don't miss any new requests coming in the middle of our
+         * processing.
+         */
+        atomic_mb_set(&qmp_dispatcher_co_busy, false);
+
+        while (!(req_obj = monitor_qmp_requests_pop_any_with_lock())) {
+            /*
+             * No more requests to process.  Wait to be reentered from
+             * handle_qmp_command() when it pushes more requests, or
+             * from monitor_cleanup() when it requests shutdown.
+             */
+            if (!qmp_dispatcher_co_shutdown) {
+                qemu_coroutine_yield();
+
+                /*
+                 * busy must be set to true again by whoever
+                 * rescheduled us to avoid double scheduling
+                 */
+                assert(atomic_xchg(&qmp_dispatcher_co_busy, false) == true);
+            }
+
+            /*
+             * qmp_dispatcher_co_shutdown may have changed if we
+             * yielded and were reentered from monitor_cleanup()
+             */
+            if (qmp_dispatcher_co_shutdown) {
+                return;
+            }
+        }
 
-    if (need_resume) {
-        /* Pairs with the monitor_suspend() in handle_qmp_command() */
-        monitor_resume(&mon->common);
-    }
-    qmp_request_free(req_obj);
+        if (atomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
+            /*
+             * Someone rescheduled us (probably because a new requests
+             * came in), but we didn't actually yield. Do that now,
+             * only to be immediately reentered and removed from the
+             * list of scheduled coroutines.
+             */
+            qemu_coroutine_yield();
+        }
 
-    /* Reschedule instead of looping so the main loop stays responsive */
-    qemu_bh_schedule(qmp_dispatcher_bh);
+        /*
+         * Move the coroutine from iohandler_ctx to qemu_aio_context for
+         * executing the command handler so that it can make progress if it
+         * involves an AIO_WAIT_WHILE().
+         */
+        aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
+        qemu_coroutine_yield();
+
+        mon = req_obj->mon;
+        /* qmp_oob_enabled() might change after "qmp_capabilities" */
+        need_resume = !qmp_oob_enabled(mon) ||
+            mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
+        qemu_mutex_unlock(&mon->qmp_queue_lock);
+        if (req_obj->req) {
+            QDict *qdict = qobject_to(QDict, req_obj->req);
+            QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
+            trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
+            monitor_qmp_dispatch(mon, req_obj->req);
+        } else {
+            assert(req_obj->err);
+            rsp = qmp_error_response(req_obj->err);
+            req_obj->err = NULL;
+            monitor_qmp_respond(mon, rsp);
+            qobject_unref(rsp);
+        }
+
+        if (need_resume) {
+            /* Pairs with the monitor_suspend() in handle_qmp_command() */
+            monitor_resume(&mon->common);
+        }
+        qmp_request_free(req_obj);
+
+        /*
+         * Yield and reschedule so the main loop stays responsive.
+         *
+         * Move back to iohandler_ctx so that nested event loops for
+         * qemu_aio_context don't start new monitor commands.
+         */
+        aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
+        qemu_coroutine_yield();
+    }
 }
 
 static void handle_qmp_command(void *opaque, QObject *req, Error *err)
@@ -302,7 +362,9 @@ static void handle_qmp_command(void *opaque, QObject *req, Error *err)
     qemu_mutex_unlock(&mon->qmp_queue_lock);
 
     /* Kick the dispatcher routine */
-    qemu_bh_schedule(qmp_dispatcher_bh);
+    if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+        aio_co_wake(qmp_dispatcher_co);
+    }
 }
 
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
qapi/qmp-dispatch.c +32 -5
index 5677ba92ca..754f7b854c 100644
@@ -12,12 +12,16 @@
  */
 
 #include "qemu/osdep.h"
+
+#include "block/aio.h"
 #include "qapi/error.h"
 #include "qapi/qmp/dispatch.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
 #include "sysemu/runstate.h"
 #include "qapi/qmp/qbool.h"
+#include "qemu/coroutine.h"
+#include "qemu/main-loop.h"
 
 static QDict *qmp_dispatch_check_obj(QDict *dict, bool allow_oob,
                                      Error **errp)
@@ -88,6 +92,30 @@ bool qmp_is_oob(const QDict *dict)
         && !qdict_haskey(dict, "execute");
 }
 
+typedef struct QmpDispatchBH {
+    const QmpCommand *cmd;
+    Monitor *cur_mon;
+    QDict *args;
+    QObject **ret;
+    Error **errp;
+    Coroutine *co;
+} QmpDispatchBH;
+
+static void do_qmp_dispatch_bh(void *opaque)
+{
+    QmpDispatchBH *data = opaque;
+
+    assert(monitor_cur() == NULL);
+    monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
+    data->cmd->fn(data->args, data->ret, data->errp);
+    monitor_set_cur(qemu_coroutine_self(), NULL);
+    aio_co_wake(data->co);
+}
+
+/*
+ * Runs outside of coroutine context for OOB commands, but in coroutine
+ * context for everything else.
+ */
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
                     bool allow_oob, Monitor *cur_mon)
 {
@@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
         qobject_ref(args);
     }
 
+    assert(!(oob && qemu_in_coroutine()));
     assert(monitor_cur() == NULL);
-    monitor_set_cur(qemu_coroutine_self(), cur_mon);
-
-    cmd->fn(args, &ret, &err);
-
-    monitor_set_cur(qemu_coroutine_self(), NULL);
+    if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
+        monitor_set_cur(qemu_coroutine_self(), cur_mon);
+        cmd->fn(args, &ret, &err);

qapi/qmp-dispatch.c QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request, · line 198

Quoted 10 lines
qapi/qmp-dispatch.c +10 -0
+        monitor_set_cur(qemu_coroutine_self(), NULL);
+    } else {
+        /*
+         * Not being in coroutine context implies that we're handling
+         * an OOB command, which must not have QCO_COROUTINE.
+         *
+         * This implies that we are in coroutine context, but the
+         * command doesn't have QCO_COROUTINE. We must drop out of
+         * coroutine context for this one.
+         */

I had to read this several times to get it. The first sentence leads me into coroutine context, and then the next sentence tells me the opposite, throwing me into confusion.

Perhaps something like this:

/*
* Actual context doesn't match the one the command needs.
* Case 1: we are in coroutine context, but command does not
* have QCO_COROUTINE. We need to drop out of coroutine
* context for executing it.
* Case 2: we are outside coroutine context, but command has
* QCO_COROUTINE. Can't actually happen, because we get here
* outside coroutine context only when executing a command
* out of band, and OOB commands never have QCO_COROUTINE. */

Quoted 57 lines
qapi/qmp-dispatch.c +14 -0
+        assert(!oob && qemu_in_coroutine() && !(cmd->options & QCO_COROUTINE));
+
+        QmpDispatchBH data = {
+            .cur_mon    = cur_mon,
+            .cmd        = cmd,
+            .args       = args,
+            .ret        = &ret,
+            .errp       = &err,
+            .co         = qemu_coroutine_self(),
+        };
+        aio_bh_schedule_oneshot(qemu_get_aio_context(), do_qmp_dispatch_bh,
+                                &data);
+        qemu_coroutine_yield();
+    }
     qobject_unref(args);
     if (err) {
         /* or assert(!ret) after reviewing all handlers: */
qapi/qmp-registry.c +3 -0
index d0f9a1d3e3..58c65b5052 100644
@@ -20,6 +20,9 @@ void qmp_register_command(QmpCommandList *cmds, const char *name,
 {
     QmpCommand *cmd = g_malloc0(sizeof(*cmd));
 
+    /* QCO_COROUTINE and QCO_ALLOW_OOB are incompatible for now */
+    assert(!((options & QCO_COROUTINE) && (options & QCO_ALLOW_OOB)));
+
     cmd->name = name;
     cmd->fn = fn;
     cmd->enabled = true;
util/aio-posix.c +7 -1
index f7f13ebfc2..30bb21d699 100644
@@ -15,6 +15,7 @@
 
 #include "qemu/osdep.h"
 #include "block/block.h"
+#include "qemu/main-loop.h"
 #include "qemu/rcu.h"
 #include "qemu/rcu_queue.h"
 #include "qemu/sockets.h"
@@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
      * There cannot be two concurrent aio_poll calls for the same AioContext (or
      * an aio_poll concurrent with a GSource prepare/check/dispatch callback).
      * We rely on this below to avoid slow locked accesses to ctx->notify_me.
+     *
+     * aio_poll() may only be called in the AioContext's thread. iohandler_ctx
+     * is special in that it runs in the main thread, but that thread's context
+     * is qemu_aio_context.
      */
-    assert(in_aio_context_home_thread(ctx));
+    assert(in_aio_context_home_thread(ctx == iohandler_get_aio_context() ?
+                                      qemu_get_aio_context() : ctx));
 
     qemu_lockcnt_inc(&ctx->list_lock);
author
#35 lore raw

Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine

Am 14.09.2020 um 17:30 hat Markus Armbruster geschrieben:

Quoted 377 lines
Quoted 358 lines · Kevin Wolf

This moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events.

For commands that are not declared safe to run in a coroutine, the dispatcher drops out of coroutine context by calling the QMP command handler from a bottom half.

---
 include/qapi/qmp/dispatch.h |   1 +
 monitor/monitor-internal.h  |   6 +-
 monitor/monitor.c           |  55 +++++++++++++---
 monitor/qmp.c               | 122 +++++++++++++++++++++++++++---------
 qapi/qmp-dispatch.c         |  61 ++++++++++++++++--
 qapi/qmp-registry.c         |   3 +
 util/aio-posix.c            |   8 ++-
 7 files changed, 210 insertions(+), 46 deletions(-)
include/qapi/qmp/dispatch.h +1 -0
index 9fd2b720a7..af8d96c570 100644
@@ -31,6 +31,7 @@ typedef enum QmpCommandOptions
 typedef struct QmpCommand
 {
     const char *name;
+    /* Runs in coroutine context if QCO_COROUTINE is set */
     QmpCommandFunc *fn;
     QmpCommandOptions options;
     QTAILQ_ENTRY(QmpCommand) node;
monitor/monitor-internal.h +4 -2
index b39e03b744..b55d6df07f 100644
@@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
 
 typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;
 extern IOThread *mon_iothread;
-extern QEMUBH *qmp_dispatcher_bh;
+extern Coroutine *qmp_dispatcher_co;
+extern bool qmp_dispatcher_co_shutdown;
+extern bool qmp_dispatcher_co_busy;
 extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
 extern QemuMutex monitor_lock;
 extern MonitorList mon_list;
@@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);
 
 void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
 void monitor_data_destroy_qmp(MonitorQMP *mon);
-void monitor_qmp_bh_dispatcher(void *data);
+void coroutine_fn monitor_qmp_dispatcher_co(void *data);
 
 int get_monitor_def(int64_t *pval, const char *name);
 void help_cmd(Monitor *mon, const char *name);
monitor/monitor.c +47 -8
index 629aa073ee..ac2722bf91 100644
@@ -55,8 +55,32 @@ typedef struct {
 /* Shared monitor I/O thread */
 IOThread *mon_iothread;
 
-/* Bottom half to dispatch the requests received from I/O thread */
-QEMUBH *qmp_dispatcher_bh;
+/* Coroutine to dispatch the requests received from I/O thread */
+Coroutine *qmp_dispatcher_co;
+
+/* Set to true when the dispatcher coroutine should terminate */
+bool qmp_dispatcher_co_shutdown;
+
+/*
+ * qmp_dispatcher_co_busy is used for synchronisation between the
+ * monitor thread and the main thread to ensure that the dispatcher
+ * coroutine never gets scheduled a second time when it's already
+ * scheduled (scheduling the same coroutine twice is forbidden).
+ *
+ * It is true if the coroutine is active and processing requests.
+ * Additional requests may then be pushed onto mon->qmp_requests,
+ * and @qmp_dispatcher_co_shutdown may be set without further ado.
+ * @qmp_dispatcher_co_busy must not be woken up in this case.
+ *
+ * If false, you also have to set @qmp_dispatcher_co_busy to true and
+ * wake up @qmp_dispatcher_co after pushing the new requests.
+ *
+ * The coroutine will automatically change this variable back to false
+ * before it yields.  Nobody else may set the variable to false.
+ *
+ * Access must be atomic for thread safety.
+ */
+bool qmp_dispatcher_co_busy;
 
 /*
  * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
@@ -623,9 +647,24 @@ void monitor_cleanup(void)
     }
     qemu_mutex_unlock(&monitor_lock);
 
-    /* QEMUBHs needs to be deleted before destroying the I/O thread */
-    qemu_bh_delete(qmp_dispatcher_bh);
-    qmp_dispatcher_bh = NULL;
+    /*
+     * The dispatcher needs to stop before destroying the I/O thread.
+     *
+     * We need to poll both qemu_aio_context and iohandler_ctx to make
+     * sure that the dispatcher coroutine keeps making progress and
+     * eventually terminates.  qemu_aio_context is automatically
+     * polled by calling AIO_WAIT_WHILE on it, but we must poll
+     * iohandler_ctx manually.
+     */
+    qmp_dispatcher_co_shutdown = true;
+    if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+        aio_co_wake(qmp_dispatcher_co);
+    }
+
+    AIO_WAIT_WHILE(qemu_get_aio_context(),
+                   (aio_poll(iohandler_get_aio_context(), false),
+                    atomic_mb_read(&qmp_dispatcher_co_busy)));
+
     if (mon_iothread) {
         iothread_destroy(mon_iothread);
         mon_iothread = NULL;
@@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
      * have commands assuming that context.  It would be nice to get
      * rid of those assumptions.
      */
-    qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
-                                   monitor_qmp_bh_dispatcher,
-                                   NULL);
+    qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);
+    atomic_mb_set(&qmp_dispatcher_co_busy, true);
+    aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
 }
 
 int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)
monitor/qmp.c +92 -30
index 922fdb5541..69f6e93f38 100644
@@ -133,6 +133,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
     }
 }
 
+/*
+ * Runs outside of coroutine context for OOB commands, but in
+ * coroutine context for everything else.
+ */
 static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
 {
     QDict *rsp;
@@ -205,43 +209,99 @@ static QMPRequest *monitor_qmp_requests_pop_any_with_lock(void)
     return req_obj;
 }
 
-void monitor_qmp_bh_dispatcher(void *data)
+void coroutine_fn monitor_qmp_dispatcher_co(void *data)
 {
-    QMPRequest *req_obj = monitor_qmp_requests_pop_any_with_lock();
+    QMPRequest *req_obj = NULL;
     QDict *rsp;
     bool need_resume;
     MonitorQMP *mon;
 
-    if (!req_obj) {
-        return;
-    }
+    while (true) {
+        assert(atomic_mb_read(&qmp_dispatcher_co_busy) == true);
 
-    mon = req_obj->mon;
-    /*  qmp_oob_enabled() might change after "qmp_capabilities" */
-    need_resume = !qmp_oob_enabled(mon) ||
-        mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
-    qemu_mutex_unlock(&mon->qmp_queue_lock);
-    if (req_obj->req) {
-        QDict *qdict = qobject_to(QDict, req_obj->req);
-        QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
-        trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
-        monitor_qmp_dispatch(mon, req_obj->req);
-    } else {
-        assert(req_obj->err);
-        rsp = qmp_error_response(req_obj->err);
-        req_obj->err = NULL;
-        monitor_qmp_respond(mon, rsp);
-        qobject_unref(rsp);
-    }
+        /*
+         * Mark the dispatcher as not busy already here so that we
+         * don't miss any new requests coming in the middle of our
+         * processing.
+         */
+        atomic_mb_set(&qmp_dispatcher_co_busy, false);
+
+        while (!(req_obj = monitor_qmp_requests_pop_any_with_lock())) {
+            /*
+             * No more requests to process.  Wait to be reentered from
+             * handle_qmp_command() when it pushes more requests, or
+             * from monitor_cleanup() when it requests shutdown.
+             */
+            if (!qmp_dispatcher_co_shutdown) {
+                qemu_coroutine_yield();
+
+                /*
+                 * busy must be set to true again by whoever
+                 * rescheduled us to avoid double scheduling
+                 */
+                assert(atomic_xchg(&qmp_dispatcher_co_busy, false) == true);
+            }
+
+            /*
+             * qmp_dispatcher_co_shutdown may have changed if we
+             * yielded and were reentered from monitor_cleanup()
+             */
+            if (qmp_dispatcher_co_shutdown) {
+                return;
+            }
+        }
 
-    if (need_resume) {
-        /* Pairs with the monitor_suspend() in handle_qmp_command() */
-        monitor_resume(&mon->common);
-    }
-    qmp_request_free(req_obj);
+        if (atomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
+            /*
+             * Someone rescheduled us (probably because a new requests
+             * came in), but we didn't actually yield. Do that now,
+             * only to be immediately reentered and removed from the
+             * list of scheduled coroutines.
+             */
+            qemu_coroutine_yield();
+        }
 
-    /* Reschedule instead of looping so the main loop stays responsive */
-    qemu_bh_schedule(qmp_dispatcher_bh);
+        /*
+         * Move the coroutine from iohandler_ctx to qemu_aio_context for
+         * executing the command handler so that it can make progress if it
+         * involves an AIO_WAIT_WHILE().
+         */
+        aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
+        qemu_coroutine_yield();
+
+        mon = req_obj->mon;
+        /* qmp_oob_enabled() might change after "qmp_capabilities" */
+        need_resume = !qmp_oob_enabled(mon) ||
+            mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
+        qemu_mutex_unlock(&mon->qmp_queue_lock);
+        if (req_obj->req) {
+            QDict *qdict = qobject_to(QDict, req_obj->req);
+            QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
+            trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
+            monitor_qmp_dispatch(mon, req_obj->req);
+        } else {
+            assert(req_obj->err);
+            rsp = qmp_error_response(req_obj->err);
+            req_obj->err = NULL;
+            monitor_qmp_respond(mon, rsp);
+            qobject_unref(rsp);
+        }
+
+        if (need_resume) {
+            /* Pairs with the monitor_suspend() in handle_qmp_command() */
+            monitor_resume(&mon->common);
+        }
+        qmp_request_free(req_obj);
+
+        /*
+         * Yield and reschedule so the main loop stays responsive.
+         *
+         * Move back to iohandler_ctx so that nested event loops for
+         * qemu_aio_context don't start new monitor commands.
+         */
+        aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
+        qemu_coroutine_yield();
+    }
 }
 
 static void handle_qmp_command(void *opaque, QObject *req, Error *err)
@@ -302,7 +362,9 @@ static void handle_qmp_command(void *opaque, QObject *req, Error *err)
     qemu_mutex_unlock(&mon->qmp_queue_lock);
 
     /* Kick the dispatcher routine */
-    qemu_bh_schedule(qmp_dispatcher_bh);
+    if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+        aio_co_wake(qmp_dispatcher_co);
+    }
 }
 
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
qapi/qmp-dispatch.c +32 -5
index 5677ba92ca..754f7b854c 100644
@@ -12,12 +12,16 @@
  */
 
 #include "qemu/osdep.h"
+
+#include "block/aio.h"
 #include "qapi/error.h"
 #include "qapi/qmp/dispatch.h"
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qjson.h"
 #include "sysemu/runstate.h"
 #include "qapi/qmp/qbool.h"
+#include "qemu/coroutine.h"
+#include "qemu/main-loop.h"
 
 static QDict *qmp_dispatch_check_obj(QDict *dict, bool allow_oob,
                                      Error **errp)
@@ -88,6 +92,30 @@ bool qmp_is_oob(const QDict *dict)
         && !qdict_haskey(dict, "execute");
 }
 
+typedef struct QmpDispatchBH {
+    const QmpCommand *cmd;
+    Monitor *cur_mon;
+    QDict *args;
+    QObject **ret;
+    Error **errp;
+    Coroutine *co;
+} QmpDispatchBH;
+
+static void do_qmp_dispatch_bh(void *opaque)
+{
+    QmpDispatchBH *data = opaque;
+
+    assert(monitor_cur() == NULL);
+    monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
+    data->cmd->fn(data->args, data->ret, data->errp);
+    monitor_set_cur(qemu_coroutine_self(), NULL);
+    aio_co_wake(data->co);
+}
+
+/*
+ * Runs outside of coroutine context for OOB commands, but in coroutine
+ * context for everything else.
+ */
 QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
                     bool allow_oob, Monitor *cur_mon)
 {
@@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
         qobject_ref(args);
     }
 
+    assert(!(oob && qemu_in_coroutine()));
     assert(monitor_cur() == NULL);
-    monitor_set_cur(qemu_coroutine_self(), cur_mon);
-
-    cmd->fn(args, &ret, &err);
-
-    monitor_set_cur(qemu_coroutine_self(), NULL);
+    if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
+        monitor_set_cur(qemu_coroutine_self(), cur_mon);
+        cmd->fn(args, &ret, &err);

qapi/qmp-dispatch.c QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request, · line 198

Quoted 10 lines
qapi/qmp-dispatch.c +10 -0
+        monitor_set_cur(qemu_coroutine_self(), NULL);
+    } else {
+        /*
+         * Not being in coroutine context implies that we're handling
+         * an OOB command, which must not have QCO_COROUTINE.
+         *
+         * This implies that we are in coroutine context, but the
+         * command doesn't have QCO_COROUTINE. We must drop out of
+         * coroutine context for this one.
+         */

I had to read this several times to get it. The first sentence leads me into coroutine context, and then the next sentence tells me the opposite, throwing me into confusion.

Perhaps something like this:

Quoted 10 lines

/*
* Actual context doesn't match the one the command needs.
* Case 1: we are in coroutine context, but command does not
* have QCO_COROUTINE. We need to drop out of coroutine
* context for executing it.
* Case 2: we are outside coroutine context, but command has
* QCO_COROUTINE. Can't actually happen, because we get here
* outside coroutine context only when executing a command
* out of band, and OOB commands never have QCO_COROUTINE. */

Works for me. Can you squash this in while applying?

Kevin

#36 lore raw

Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine

Quoted 15 lines · Kevin Wolf

Am 14.09.2020 um 17:30 hat Markus Armbruster geschrieben:

Quoted 14 lines
Quoted 12 lines · Kevin Wolf

This moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events.

For commands that are not declared safe to run in a coroutine, the dispatcher drops out of coroutine context by calling the QMP command handler from a bottom half.

---

[...]

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
qapi/qmp-dispatch.c +0 -0
index 5677ba92ca..754f7b854c 100644

[...]

Quoted 42 lines
Quoted 31 lines
Quoted 14 lines
qapi/qmp-dispatch.c +4 -5
@@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
         qobject_ref(args);
     }
 
+    assert(!(oob && qemu_in_coroutine()));
     assert(monitor_cur() == NULL);
-    monitor_set_cur(qemu_coroutine_self(), cur_mon);
-
-    cmd->fn(args, &ret, &err);
-
-    monitor_set_cur(qemu_coroutine_self(), NULL);
+    if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
+        monitor_set_cur(qemu_coroutine_self(), cur_mon);
+        cmd->fn(args, &ret, &err);

qapi/qmp-dispatch.c QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request, · line 198

Quoted 10 lines
qapi/qmp-dispatch.c +10 -0
+        monitor_set_cur(qemu_coroutine_self(), NULL);
+    } else {
+        /*
+         * Not being in coroutine context implies that we're handling
+         * an OOB command, which must not have QCO_COROUTINE.
+         *
+         * This implies that we are in coroutine context, but the
+         * command doesn't have QCO_COROUTINE. We must drop out of
+         * coroutine context for this one.
+         */

I had to read this several times to get it. The first sentence leads me into coroutine context, and then the next sentence tells me the opposite, throwing me into confusion.

Perhaps something like this:

Quoted 10 lines

/*
* Actual context doesn't match the one the command needs.
* Case 1: we are in coroutine context, but command does not
* have QCO_COROUTINE. We need to drop out of coroutine
* context for executing it.
* Case 2: we are outside coroutine context, but command has
* QCO_COROUTINE. Can't actually happen, because we get here
* outside coroutine context only when executing a command
* out of band, and OOB commands never have QCO_COROUTINE. */

Quoted 1 line

Works for me. Can you squash this in while applying?

Sure!

#37 lore raw

Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine

In reply to Kevin Wolf

Additional nitpick detail on Kevin's request.

Quoted 11 lines · Kevin Wolf

This moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events.

For commands that are not declared safe to run in a coroutine, the dispatcher drops out of coroutine context by calling the QMP command handler from a bottom half.

[...]

Quoted 68 lines
monitor/monitor.c +44 -5
index 629aa073ee..ac2722bf91 100644
@@ -55,8 +55,32 @@ typedef struct {
 /* Shared monitor I/O thread */
 IOThread *mon_iothread;
 
-/* Bottom half to dispatch the requests received from I/O thread */
-QEMUBH *qmp_dispatcher_bh;
+/* Coroutine to dispatch the requests received from I/O thread */
+Coroutine *qmp_dispatcher_co;
+
+/* Set to true when the dispatcher coroutine should terminate */
+bool qmp_dispatcher_co_shutdown;
+
+/*
+ * qmp_dispatcher_co_busy is used for synchronisation between the
+ * monitor thread and the main thread to ensure that the dispatcher
+ * coroutine never gets scheduled a second time when it's already
+ * scheduled (scheduling the same coroutine twice is forbidden).
+ *
+ * It is true if the coroutine is active and processing requests.
+ * Additional requests may then be pushed onto mon->qmp_requests,
+ * and @qmp_dispatcher_co_shutdown may be set without further ado.
+ * @qmp_dispatcher_co_busy must not be woken up in this case.
+ *
+ * If false, you also have to set @qmp_dispatcher_co_busy to true and
+ * wake up @qmp_dispatcher_co after pushing the new requests.
+ *
+ * The coroutine will automatically change this variable back to false
+ * before it yields.  Nobody else may set the variable to false.
+ *
+ * Access must be atomic for thread safety.
+ */
+bool qmp_dispatcher_co_busy;
 
 /*
  * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
@@ -623,9 +647,24 @@ void monitor_cleanup(void)
     }
     qemu_mutex_unlock(&monitor_lock);
 
-    /* QEMUBHs needs to be deleted before destroying the I/O thread */
-    qemu_bh_delete(qmp_dispatcher_bh);
-    qmp_dispatcher_bh = NULL;
+    /*
+     * The dispatcher needs to stop before destroying the I/O thread.
+     *
+     * We need to poll both qemu_aio_context and iohandler_ctx to make
+     * sure that the dispatcher coroutine keeps making progress and
+     * eventually terminates.  qemu_aio_context is automatically
+     * polled by calling AIO_WAIT_WHILE on it, but we must poll
+     * iohandler_ctx manually.
+     */
+    qmp_dispatcher_co_shutdown = true;
+    if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+        aio_co_wake(qmp_dispatcher_co);
+    }
+
+    AIO_WAIT_WHILE(qemu_get_aio_context(),
+                   (aio_poll(iohandler_get_aio_context(), false),
+                    atomic_mb_read(&qmp_dispatcher_co_busy)));
+
     if (mon_iothread) {
         iothread_destroy(mon_iothread);
         mon_iothread = NULL;

monitor/monitor.c void monitor_init_globals_core(void) · line 691

Quoted 7 lines
monitor/monitor.c +1 -3
@@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
      * have commands assuming that context.  It would be nice to get
      * rid of those assumptions.
      */
-    qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
-                                   monitor_qmp_bh_dispatcher,
-                                   NULL);
+    qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);

Rather long line, caused by rather long identifiers.

Not your fault; you imitated the existing pattern static variable qmp_dispatcher_bh / extern function monitor_qmp_bh_dispatcher(). But the prefix monitor_qmp_ is awfully long, and not just here. Let's leave this for another day.

Quoted 5 lines
monitor/monitor.c +2 -0
+    atomic_mb_set(&qmp_dispatcher_co_busy, true);
+    aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
 }
 
 int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)

[...]

Quoted 13 lines
util/aio-posix.c +1 -0
index f7f13ebfc2..30bb21d699 100644
@@ -15,6 +15,7 @@
 
 #include "qemu/osdep.h"
 #include "block/block.h"
+#include "qemu/main-loop.h"
 #include "qemu/rcu.h"
 #include "qemu/rcu_queue.h"
 #include "qemu/sockets.h"

util/aio-posix.c bool aio_poll(AioContext *ctx, bool blocking) · line 565

Quoted 7 lines
util/aio-posix.c +4 -0
@@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
      * There cannot be two concurrent aio_poll calls for the same AioContext (or
      * an aio_poll concurrent with a GSource prepare/check/dispatch callback).
      * We rely on this below to avoid slow locked accesses to ctx->notify_me.
+     *
+     * aio_poll() may only be called in the AioContext's thread. iohandler_ctx
+     * is special in that it runs in the main thread, but that thread's context
+     * is qemu_aio_context.

Wrapping the comment around column 70 or so would make it easier to read. Up to you.

Quoted 6 lines
util/aio-posix.c +2 -1
      */
-    assert(in_aio_context_home_thread(ctx));
+    assert(in_aio_context_home_thread(ctx == iohandler_get_aio_context() ?
+                                      qemu_get_aio_context() : ctx));
 
     qemu_lockcnt_inc(&ctx->list_lock);
author
#38 lore raw

PATCH v7 10/13 hmp: Add support for coroutine command handlers

In reply to Kevin Wolf

Often, QMP command handlers are not only called to handle QMP commands, but also from a corresponding HMP command handler. In order to give them a consistent environment, optionally run HMP command handlers in a coroutine, too.

The implementation is a lot simpler than in QMP because for HMP, we still block the VM while the coroutine is running.

---
 monitor/monitor-internal.h |  1 +
 monitor/hmp.c              | 37 ++++++++++++++++++++++++++++++++-----
 2 files changed, 33 insertions(+), 5 deletions(-)
monitor/monitor-internal.h +1 -0
index b55d6df07f..ad2e64be13 100644
@@ -74,6 +74,7 @@ typedef struct HMPCommand {
     const char *help;
     const char *flags; /* p=preconfig */
     void (*cmd)(Monitor *mon, const QDict *qdict);
+    bool coroutine;
     /*
      * @sub_table is a list of 2nd level of commands. If it does not exist,
      * cmd should be used. If it exists, sub_table[?].cmd should be
monitor/hmp.c +32 -5
index 4b66ca1cd6..b858b0dbde 100644
@@ -1056,12 +1056,26 @@ fail:
     return NULL;
 }
 
+typedef struct HandleHmpCommandCo {
+    Monitor *mon;
+    const HMPCommand *cmd;
+    QDict *qdict;
+    bool done;
+} HandleHmpCommandCo;
+
+static void handle_hmp_command_co(void *opaque)
+{
+    HandleHmpCommandCo *data = opaque;
+    data->cmd->cmd(data->mon, data->qdict);
+    monitor_set_cur(qemu_coroutine_self(), NULL);
+    data->done = true;
+}
+
 void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
 {
     QDict *qdict;
     const HMPCommand *cmd;
     const char *cmd_start = cmdline;
-    Monitor *old_mon;
 
     trace_handle_hmp_command(mon, cmdline);
 
@@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
-    /* old_mon is non-NULL when called from qmp_human_monitor_command() */
-    old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
-    cmd->cmd(&mon->common, qdict);
-    monitor_set_cur(qemu_coroutine_self(), old_mon);
+    if (!cmd->coroutine) {
+        /* old_mon is non-NULL when called from qmp_human_monitor_command() */
+        Monitor *old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
+        cmd->cmd(&mon->common, qdict);
+        monitor_set_cur(qemu_coroutine_self(), old_mon);
+    } else {
+        HandleHmpCommandCo data = {
+            .mon = &mon->common,
+            .cmd = cmd,
+            .qdict = qdict,
+            .done = false,
+        };
+        Coroutine *co = qemu_coroutine_create(handle_hmp_command_co, &data);
+        monitor_set_cur(co, &mon->common);
+        aio_co_enter(qemu_get_aio_context(), co);
+        AIO_WAIT_WHILE(qemu_get_aio_context(), !data.done);
+    }
 
     qobject_unref(qdict);
 }
Signature · 1 line
2.25.4
Reviewed-by on v7 10/13
#39 lore raw

Re: [PATCH v7 10/13] hmp: Add support for coroutine command handlers

Quoted 9 lines · * Kevin Wolf ([email protected]) wrote:

Often, QMP command handlers are not only called to handle QMP commands, but also from a corresponding HMP command handler. In order to give them a consistent environment, optionally run HMP command handlers in a coroutine, too.

The implementation is a lot simpler than in QMP because for HMP, we still block the VM while the coroutine is running.

Quoted 80 lines
---
 monitor/monitor-internal.h |  1 +
 monitor/hmp.c              | 37 ++++++++++++++++++++++++++++++++-----
 2 files changed, 33 insertions(+), 5 deletions(-)
monitor/monitor-internal.h +1 -0
index b55d6df07f..ad2e64be13 100644
@@ -74,6 +74,7 @@ typedef struct HMPCommand {
     const char *help;
     const char *flags; /* p=preconfig */
     void (*cmd)(Monitor *mon, const QDict *qdict);
+    bool coroutine;
     /*
      * @sub_table is a list of 2nd level of commands. If it does not exist,
      * cmd should be used. If it exists, sub_table[?].cmd should be
monitor/hmp.c +32 -5
index 4b66ca1cd6..b858b0dbde 100644
@@ -1056,12 +1056,26 @@ fail:
     return NULL;
 }
 
+typedef struct HandleHmpCommandCo {
+    Monitor *mon;
+    const HMPCommand *cmd;
+    QDict *qdict;
+    bool done;
+} HandleHmpCommandCo;
+
+static void handle_hmp_command_co(void *opaque)
+{
+    HandleHmpCommandCo *data = opaque;
+    data->cmd->cmd(data->mon, data->qdict);
+    monitor_set_cur(qemu_coroutine_self(), NULL);
+    data->done = true;
+}
+
 void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
 {
     QDict *qdict;
     const HMPCommand *cmd;
     const char *cmd_start = cmdline;
-    Monitor *old_mon;
 
     trace_handle_hmp_command(mon, cmdline);
 
@@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
-    /* old_mon is non-NULL when called from qmp_human_monitor_command() */
-    old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
-    cmd->cmd(&mon->common, qdict);
-    monitor_set_cur(qemu_coroutine_self(), old_mon);
+    if (!cmd->coroutine) {
+        /* old_mon is non-NULL when called from qmp_human_monitor_command() */
+        Monitor *old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
+        cmd->cmd(&mon->common, qdict);
+        monitor_set_cur(qemu_coroutine_self(), old_mon);
+    } else {
+        HandleHmpCommandCo data = {
+            .mon = &mon->common,
+            .cmd = cmd,
+            .qdict = qdict,
+            .done = false,
+        };
+        Coroutine *co = qemu_coroutine_create(handle_hmp_command_co, &data);
+        monitor_set_cur(co, &mon->common);
+        aio_co_enter(qemu_get_aio_context(), co);
+        AIO_WAIT_WHILE(qemu_get_aio_context(), !data.done);
+    }
 
     qobject_unref(qdict);
 }
Signature · 1 line
2.25.4
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#40 lore raw

Re: [PATCH v7 10/13] hmp: Add support for coroutine command handlers

In reply to Kevin Wolf

Additional nitpick detail on Kevin's request.

Quoted 59 lines · Kevin Wolf

Often, QMP command handlers are not only called to handle QMP commands, but also from a corresponding HMP command handler. In order to give them a consistent environment, optionally run HMP command handlers in a coroutine, too.

The implementation is a lot simpler than in QMP because for HMP, we still block the VM while the coroutine is running.

---
 monitor/monitor-internal.h |  1 +
 monitor/hmp.c              | 37 ++++++++++++++++++++++++++++++++-----
 2 files changed, 33 insertions(+), 5 deletions(-)
monitor/monitor-internal.h +1 -0
index b55d6df07f..ad2e64be13 100644
@@ -74,6 +74,7 @@ typedef struct HMPCommand {
     const char *help;
     const char *flags; /* p=preconfig */
     void (*cmd)(Monitor *mon, const QDict *qdict);
+    bool coroutine;
     /*
      * @sub_table is a list of 2nd level of commands. If it does not exist,
      * cmd should be used. If it exists, sub_table[?].cmd should be
monitor/hmp.c +15 -1
index 4b66ca1cd6..b858b0dbde 100644
@@ -1056,12 +1056,26 @@ fail:
     return NULL;
 }
 
+typedef struct HandleHmpCommandCo {
+    Monitor *mon;
+    const HMPCommand *cmd;
+    QDict *qdict;
+    bool done;
+} HandleHmpCommandCo;
+
+static void handle_hmp_command_co(void *opaque)
+{
+    HandleHmpCommandCo *data = opaque;
+    data->cmd->cmd(data->mon, data->qdict);
+    monitor_set_cur(qemu_coroutine_self(), NULL);
+    data->done = true;
+}
+
 void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
 {
     QDict *qdict;
     const HMPCommand *cmd;
     const char *cmd_start = cmdline;
-    Monitor *old_mon;
 
     trace_handle_hmp_command(mon, cmdline);
 

monitor/hmp.c void handle_hmp_command(MonitorHMP *mon, const char *cmdline) · line 1099

Quoted 10 lines
monitor/hmp.c +3 -4
@@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
         return;
     }
 
-    /* old_mon is non-NULL when called from qmp_human_monitor_command() */
-    old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
-    cmd->cmd(&mon->common, qdict);
-    monitor_set_cur(qemu_coroutine_self(), old_mon);
+    if (!cmd->coroutine) {
+        /* old_mon is non-NULL when called from qmp_human_monitor_command() */
+        Monitor *old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);

Long line. David seems fine with it, and he's the maintainer.

Quoted 17 lines
monitor/hmp.c +14 -0
+        cmd->cmd(&mon->common, qdict);
+        monitor_set_cur(qemu_coroutine_self(), old_mon);
+    } else {
+        HandleHmpCommandCo data = {
+            .mon = &mon->common,
+            .cmd = cmd,
+            .qdict = qdict,
+            .done = false,
+        };
+        Coroutine *co = qemu_coroutine_create(handle_hmp_command_co, &data);
+        monitor_set_cur(co, &mon->common);
+        aio_co_enter(qemu_get_aio_context(), co);
+        AIO_WAIT_WHILE(qemu_get_aio_context(), !data.done);
+    }
 
     qobject_unref(qdict);
 }
author
#41 lore raw

PATCH v7 11/13 util/async: Add aio_co_reschedule_self()

In reply to Kevin Wolf

Add a function that can be used to move the currently running coroutine to a different AioContext (and therefore potentially a different thread).

---
 include/block/aio.h | 10 ++++++++++
 util/async.c        | 30 ++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)
include/block/aio.h +10 -0
index b2f703fa3f..c37617b404 100644
@@ -17,6 +17,7 @@
 #ifdef CONFIG_LINUX_IO_URING
 #include <liburing.h>
 #endif
+#include "qemu/coroutine.h"
 #include "qemu/queue.h"
 #include "qemu/event_notifier.h"
 #include "qemu/thread.h"
@@ -654,6 +655,15 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)
  */
 void aio_co_schedule(AioContext *ctx, struct Coroutine *co);
 
+/**
+ * aio_co_reschedule_self:
+ * @new_ctx: the new context
+ *
+ * Move the currently running coroutine to new_ctx. If the coroutine is already
+ * running in new_ctx, do nothing.
+ */
+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx);
+
 /**
  * aio_co_wake:
  * @co: the coroutine
util/async.c +30 -0
index 4266745dee..a609e18693 100644
@@ -569,6 +569,36 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
     aio_context_unref(ctx);
 }
 
+typedef struct AioCoRescheduleSelf {
+    Coroutine *co;
+    AioContext *new_ctx;
+} AioCoRescheduleSelf;
+
+static void aio_co_reschedule_self_bh(void *opaque)
+{
+    AioCoRescheduleSelf *data = opaque;
+    aio_co_schedule(data->new_ctx, data->co);
+}
+
+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx)
+{
+    AioContext *old_ctx = qemu_get_current_aio_context();
+
+    if (old_ctx != new_ctx) {
+        AioCoRescheduleSelf data = {
+            .co = qemu_coroutine_self(),
+            .new_ctx = new_ctx,
+        };
+        /*
+         * We can't directly schedule the coroutine in the target context
+         * because this would be racy: The other thread could try to enter the
+         * coroutine before it has yielded in this one.
+         */
+        aio_bh_schedule_oneshot(old_ctx, aio_co_reschedule_self_bh, &data);
+        qemu_coroutine_yield();
+    }
+}
+
 void aio_co_wake(struct Coroutine *co)
 {
     AioContext *ctx;
Signature · 1 line
2.25.4
Reviewed-by on v7 11/13
#42 lore raw

Re: [PATCH v7 11/13] util/async: Add aio_co_reschedule_self()

Quoted 9 lines · Kevin Wolf

Add a function that can be used to move the currently running coroutine to a different AioContext (and therefore potentially a different thread).

---
 include/block/aio.h | 10 ++++++++++
 util/async.c        | 30 ++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)
#43 lore raw

Re: [PATCH v7 11/13] util/async: Add aio_co_reschedule_self()

In reply to Kevin Wolf

Additional nitpick detail on Kevin's request.

Quoted 23 lines · Kevin Wolf

Add a function that can be used to move the currently running coroutine to a different AioContext (and therefore potentially a different thread).

---
 include/block/aio.h | 10 ++++++++++
 util/async.c        | 30 ++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+)
include/block/aio.h +1 -0
index b2f703fa3f..c37617b404 100644
@@ -17,6 +17,7 @@
 #ifdef CONFIG_LINUX_IO_URING
 #include <liburing.h>
 #endif
+#include "qemu/coroutine.h"
 #include "qemu/queue.h"
 #include "qemu/event_notifier.h"
 #include "qemu/thread.h"

include/block/aio.h static inline bool aio_node_check(AioContext *ctx, bool is_external) · line 663

Quoted 9 lines
include/block/aio.h +6 -0
@@ -654,6 +655,15 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)
  */
 void aio_co_schedule(AioContext *ctx, struct Coroutine *co);
 
+/**
+ * aio_co_reschedule_self:
+ * @new_ctx: the new context
+ *
+ * Move the currently running coroutine to new_ctx. If the coroutine is already
+ * running in new_ctx, do nothing.

Wrapping the comment around column 70 or so would make it easier to read. Up to you.

Quoted 29 lines
include/block/aio.h +3 -0
+ */
+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx);
+
 /**
  * aio_co_wake:
  * @co: the coroutine
util/async.c +15 -0
index 4266745dee..a609e18693 100644
@@ -569,6 +569,36 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
     aio_context_unref(ctx);
 }
 
+typedef struct AioCoRescheduleSelf {
+    Coroutine *co;
+    AioContext *new_ctx;
+} AioCoRescheduleSelf;
+
+static void aio_co_reschedule_self_bh(void *opaque)
+{
+    AioCoRescheduleSelf *data = opaque;
+    aio_co_schedule(data->new_ctx, data->co);
+}
+
+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx)
+{
+    AioContext *old_ctx = qemu_get_current_aio_context();
+

util/async.c void aio_co_schedule(AioContext *ctx, Coroutine *co) · line 596

Quoted 10 lines
util/async.c +10 -0
+    if (old_ctx != new_ctx) {
+        AioCoRescheduleSelf data = {
+            .co = qemu_coroutine_self(),
+            .new_ctx = new_ctx,
+        };
+        /*
+         * We can't directly schedule the coroutine in the target context
+         * because this would be racy: The other thread could try to enter the
+         * coroutine before it has yielded in this one.
+         */

Likewise.

Quoted 8 lines
util/async.c +5 -0
+        aio_bh_schedule_oneshot(old_ctx, aio_co_reschedule_self_bh, &data);
+        qemu_coroutine_yield();
+    }
+}
+
 void aio_co_wake(struct Coroutine *co)
 {
     AioContext *ctx;
author
#44 lore raw

PATCH v7 12/13 block: Add bdrv_co_move_to_aio_context()

In reply to Kevin Wolf

Add a function to move the current coroutine to the AioContext of a given BlockDriverState.

---
 include/block/block.h |  6 ++++++
 block.c               | 10 ++++++++++
 2 files changed, 16 insertions(+)
include/block/block.h +6 -0
index 981ab5b314..80ab322f11 100644
@@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
  */
 AioContext *bdrv_get_aio_context(BlockDriverState *bs);
 
+/**
+ * Move the current coroutine to the AioContext of @bs and return the old
+ * AioContext of the coroutine.
+ */
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);
+
 /**
  * Transfer control to @co in the aio context of @bs
  */
block.c +10 -0
index 9538af4884..81403e00d1 100644
@@ -6372,6 +6372,16 @@ AioContext *bdrv_get_aio_context(BlockDriverState *bs)
     return bs ? bs->aio_context : qemu_get_aio_context();
 }
 
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs)
+{
+    Coroutine *self = qemu_coroutine_self();
+    AioContext *old_ctx = qemu_coroutine_get_aio_context(self);
+    AioContext *new_ctx = bdrv_get_aio_context(bs);
+
+    aio_co_reschedule_self(new_ctx);
+    return old_ctx;
+}
+
 void bdrv_coroutine_enter(BlockDriverState *bs, Coroutine *co)
 {
     aio_co_enter(bdrv_get_aio_context(bs), co);
Signature · 1 line
2.25.4
Reviewed-by on v7 12/13
#45 lore raw

Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()

Quoted 14 lines · Kevin Wolf

Add a function to move the current coroutine to the AioContext of a given BlockDriverState.

---
 include/block/block.h |  6 ++++++
 block.c               | 10 ++++++++++
 2 files changed, 16 insertions(+)
include/block/block.h +0 -0
index 981ab5b314..80ab322f11 100644

include/block/block.h bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag); · line 633

Quoted 8 lines
include/block/block.h +5 -0
@@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
  */
 AioContext *bdrv_get_aio_context(BlockDriverState *bs);
 
+/**
+ * Move the current coroutine to the AioContext of @bs and return the old
+ * AioContext of the coroutine.
+ */
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);

I'm not sure this function handles all cases:
1. Being called without the BQL (i.e. from an IOThread).
2. Being called while a device stops using its IOThread.

The races that come to mind are fetching the AioContext for bs and then scheduling a BH. The BH is executed later on by the event loop. There might be cases where the AioContext for bs is updated before the BH runs.

I didn't investigate these cases but wanted to mention them in case you want to add doc comments about when this function can be used or if you'd like to verify them yourself.

author
#46 lore raw

Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()

Am 15.09.2020 um 16:31 hat Stefan Hajnoczi geschrieben:

Quoted 28 lines · Kevin Wolf
Quoted 14 lines

Add a function to move the current coroutine to the AioContext of a given BlockDriverState.

---
 include/block/block.h |  6 ++++++
 block.c               | 10 ++++++++++
 2 files changed, 16 insertions(+)
include/block/block.h +0 -0
index 981ab5b314..80ab322f11 100644

include/block/block.h bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag); · line 633

Quoted 8 lines
include/block/block.h +5 -0
@@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
  */
 AioContext *bdrv_get_aio_context(BlockDriverState *bs);
 
+/**
+ * Move the current coroutine to the AioContext of @bs and return the old
+ * AioContext of the coroutine.
+ */
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);

I'm not sure this function handles all cases:
1. Being called without the BQL (i.e. from an IOThread).
2. Being called while a device stops using its IOThread.

Quoted 4 lines

The races that come to mind are fetching the AioContext for bs and then scheduling a BH. The BH is executed later on by the event loop. There might be cases where the AioContext for bs is updated before the BH runs.

Updating the AioContext of a node drains the BDS first, so it would execute any BHs still pending in the old AioContext. So this part looks safe to me.

I'm not sure what you mean with the BQL part. I don't think I'm accessing anything protected by the BQL?

Quoted 3 lines

I didn't investigate these cases but wanted to mention them in case you want to add doc comments about when this function can be used or if you'd like to verify them yourself.

One thing that I'm not completely sure about (but that isn't really related to this specific patch) is AioContext changes later in the process when the actual command handler has yielded. We may have to be careful to prevent those for coroutine based QMP commands in the block layer.

By sheer luck, qmp_block_resize() creates a new BlockBackend that has blk->allow_aio_context_change = false. So we're actually good in the only command I'm converting. Phew.

Kevin

#47 lore raw

Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()

Quoted 33 lines · Kevin Wolf

Am 15.09.2020 um 16:31 hat Stefan Hajnoczi geschrieben:

Quoted 32 lines · Kevin Wolf
Quoted 14 lines

Add a function to move the current coroutine to the AioContext of a given BlockDriverState.

---
 include/block/block.h |  6 ++++++
 block.c               | 10 ++++++++++
 2 files changed, 16 insertions(+)
include/block/block.h +0 -0
index 981ab5b314..80ab322f11 100644

include/block/block.h bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag); · line 633

Quoted 8 lines
include/block/block.h +5 -0
@@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
  */
 AioContext *bdrv_get_aio_context(BlockDriverState *bs);
 
+/**
+ * Move the current coroutine to the AioContext of @bs and return the old
+ * AioContext of the coroutine.
+ */
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);

I'm not sure this function handles all cases:
1. Being called without the BQL (i.e. from an IOThread).
2. Being called while a device stops using its IOThread.

The races that come to mind are fetching the AioContext for bs and then scheduling a BH. The BH is executed later on by the event loop. There might be cases where the AioContext for bs is updated before the BH runs.

The scenario I'm thinking about is where bs' AioContext changes while we are trying to move there.

There is a window of time between fetching bs' AioContext, scheduling a BH in our old AioContext (not in bs' AioContext), and then scheduling the coroutine into the AioContext we previously fetched for bs.

Is it possible for the AioContext value we stashed to be outdated by the time we use it?

I think the answer is that it's safe to use this function from the main loop thread under the BQL. That way nothing else will change bs' AioContext while we're running. But it's probably not safe to use this function from an arbitrary IOThread (without the BQL).

I think this limitation is okay but it needs to be documented. Maybe an assertion can verify that it holds.

Stefan

author
#48 lore raw

Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()

Am 28.09.2020 um 10:59 hat Stefan Hajnoczi geschrieben:

Quoted 42 lines · Kevin Wolf
Quoted 33 lines

Am 15.09.2020 um 16:31 hat Stefan Hajnoczi geschrieben:

Quoted 32 lines · Kevin Wolf
Quoted 14 lines

Add a function to move the current coroutine to the AioContext of a given BlockDriverState.

---
 include/block/block.h |  6 ++++++
 block.c               | 10 ++++++++++
 2 files changed, 16 insertions(+)
include/block/block.h +0 -0
index 981ab5b314..80ab322f11 100644

include/block/block.h bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag); · line 633

Quoted 8 lines
include/block/block.h +5 -0
@@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
  */
 AioContext *bdrv_get_aio_context(BlockDriverState *bs);
 
+/**
+ * Move the current coroutine to the AioContext of @bs and return the old
+ * AioContext of the coroutine.
+ */
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);

I'm not sure this function handles all cases:
1. Being called without the BQL (i.e. from an IOThread).
2. Being called while a device stops using its IOThread.

The races that come to mind are fetching the AioContext for bs and then scheduling a BH. The BH is executed later on by the event loop. There might be cases where the AioContext for bs is updated before the BH runs.

The scenario I'm thinking about is where bs' AioContext changes while we are trying to move there.

There is a window of time between fetching bs' AioContext, scheduling a BH in our old AioContext (not in bs' AioContext), and then scheduling the coroutine into the AioContext we previously fetched for bs.

Quoted 7 lines

Is it possible for the AioContext value we stashed to be outdated by the time we use it?

I think the answer is that it's safe to use this function from the main loop thread under the BQL. That way nothing else will change bs' AioContext while we're running. But it's probably not safe to use this function from an arbitrary IOThread (without the BQL).

It's probably the safest to treat it as such. The first part of it (the window between fetching bs->aio_context and using it) is actually also true for this ubiquitous sequence:

    AioContext *ctx = bdrv_get_aio_context(bs);
    aio_context_acquire(ctx);

I never really thought about this, but this is only safe in the main thread. Most of its users are of course monitor command handlers, which always run in the main thread.

Quoted 2 lines

I think this limitation is okay but it needs to be documented. Maybe an assertion can verify that it holds.

Yes, why not.

Maybe we should actually change the interface into a pair of bdrv_co_enter/leave() functions that also increase bs->in_flight so that the whole section will complete before the AioContext of bs changes (changing the AioContext while the handle coroutine has yielded and will continue to run in the old context would be bad).

block_resize is safe without it, but it might be better to introduce patterns that will be safe without being extra careful in each command.

Kevin

author
#49 lore raw

PATCH v7 13/13 block: Convert 'block_resize' to coroutine

In reply to Kevin Wolf

block_resize performs some I/O that could potentially take quite some time, so use it as an example for the new 'coroutine': true annotation in the QAPI schema.

bdrv_truncate() requires that we're already in the right AioContext for the BlockDriverState if called in coroutine context. So instead of just taking the AioContext lock, move the QMP handler coroutine to the context.

Call blk_unref() only after switching back because blk_unref() may only be called in the main thread.

---
 qapi/block-core.json |  3 ++-
 blockdev.c           | 13 ++++++-------
 hmp-commands.hx      |  1 +
 3 files changed, 9 insertions(+), 8 deletions(-)
qapi/block-core.json +2 -1
index 0345f6f2d2..d3e49c9419 100644
@@ -1302,7 +1302,8 @@
 { 'command': 'block_resize',
   'data': { '*device': 'str',
             '*node-name': 'str',
-            'size': 'int' } }
+            'size': 'int' },
+  'coroutine': true }
 
 ##
 # @NewImageMode:
blockdev.c +6 -7
index 7f2561081e..064989fc2d 100644
@@ -2439,14 +2439,14 @@ BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,
     return ret;
 }
 
-void qmp_block_resize(bool has_device, const char *device,
-                      bool has_node_name, const char *node_name,
-                      int64_t size, Error **errp)
+void coroutine_fn qmp_block_resize(bool has_device, const char *device,
+                                   bool has_node_name, const char *node_name,
+                                   int64_t size, Error **errp)
 {
     Error *local_err = NULL;
     BlockBackend *blk = NULL;
     BlockDriverState *bs;
-    AioContext *aio_context;
+    AioContext *old_ctx;
 
     bs = bdrv_lookup_bs(has_device ? device : NULL,
                         has_node_name ? node_name : NULL,
@@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
         return;
     }
 
-    aio_context = bdrv_get_aio_context(bs);
-    aio_context_acquire(aio_context);
+    old_ctx = bdrv_co_move_to_aio_context(bs);
 
     if (size < 0) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");
@@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
     bdrv_drained_end(bs);
 
 out:
+    aio_co_reschedule_self(old_ctx);
     blk_unref(blk);
-    aio_context_release(aio_context);
 }
 
 void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
hmp-commands.hx +1 -0
index 60f395c276..ac360b73f6 100644
@@ -76,6 +76,7 @@ ERST
         .params     = "device size",
         .help       = "resize a block image",
         .cmd        = hmp_block_resize,
+        .coroutine  = true,
     },
 
 SRST
Signature · 1 line
2.25.4
#50 lore raw

Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2462

Quoted 10 lines · Kevin Wolf
blockdev.c +1 -2
@@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
         return;
     }
 
-    aio_context = bdrv_get_aio_context(bs);
-    aio_context_acquire(aio_context);
+    old_ctx = bdrv_co_move_to_aio_context(bs);
 
     if (size < 0) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");

Is it safe to call blk_new() outside the BQL since it mutates global state?

In other words, could another thread race with us?

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2483

Quoted 7 lines
blockdev.c +1 -1
@@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
     bdrv_drained_end(bs);
 
 out:
+    aio_co_reschedule_self(old_ctx);
     blk_unref(blk);
-    aio_context_release(aio_context);

The following precondition is violated by the blk_unref -> bdrv_drain -> AIO_WAIT_WHILE() call if blk->refcnt is 1 here:

* The caller's thread must be the IOThread that owns @ctx or the main loop
* thread (with @ctx acquired exactly once).

blk_unref() is called from the main loop thread without having acquired blk's AioContext.

Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but I'm not sure if that can be guaranteed.

The following seems safer although it's uglier:

  aio_context = bdrv_get_aio_context(bs);
  aio_context_acquire(aio_context);
  blk_unref(blk);
  aio_context_release(aio_context);
author
#51 lore raw

Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine

Am 15.09.2020 um 16:57 hat Stefan Hajnoczi geschrieben:

Quoted 9 lines · Kevin Wolf
Quoted 8 lines
blockdev.c +1 -2
@@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
         return;
     }
 
-    aio_context = bdrv_get_aio_context(bs);
-    aio_context_acquire(aio_context);
+    old_ctx = bdrv_co_move_to_aio_context(bs);
 
Quoted 6 lines

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2462

Quoted 2 lines
blockdev.c +0 -0
     if (size < 0) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");

Is it safe to call blk_new() outside the BQL since it mutates global state?

In other words, could another thread race with us?

Hm, probably not.

Would it be safer to have the bdrv_co_move_to_aio_context() call only immediately before the drain?

Quoted 20 lines

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2483

Quoted 7 lines
blockdev.c +1 -1
@@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
     bdrv_drained_end(bs);
 
 out:
+    aio_co_reschedule_self(old_ctx);
     blk_unref(blk);
-    aio_context_release(aio_context);

The following precondition is violated by the blk_unref -> bdrv_drain -> AIO_WAIT_WHILE() call if blk->refcnt is 1 here:

* The caller's thread must be the IOThread that owns @ctx or the main loop
* thread (with @ctx acquired exactly once).

blk_unref() is called from the main loop thread without having acquired blk's AioContext.

Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but I'm not sure if that can be guaranteed.

Quoted 6 lines

The following seems safer although it's uglier:

  aio_context = bdrv_get_aio_context(bs);
  aio_context_acquire(aio_context);
  blk_unref(blk);
  aio_context_release(aio_context);

May we actually acquire aio_context if blk is in the main thread? I think we must only do this if it's in a different iothread because we'd end up with a recursive lock and drain would hang.

Kevin

#52 lore raw

Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine

Quoted 15 lines · Kevin Wolf

Am 15.09.2020 um 16:57 hat Stefan Hajnoczi geschrieben:

Quoted 14 lines · Kevin Wolf

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2462

Quoted 10 lines
blockdev.c +1 -2
@@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
         return;
     }
 
-    aio_context = bdrv_get_aio_context(bs);
-    aio_context_acquire(aio_context);
+    old_ctx = bdrv_co_move_to_aio_context(bs);
 
     if (size < 0) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");

Is it safe to call blk_new() outside the BQL since it mutates global state?

Quoted 6 lines
Quoted 1 line

In other words, could another thread race with us?

Hm, probably not.

Would it be safer to have the bdrv_co_move_to_aio_context() call only immediately before the drain?

Yes, sounds good.

Quoted 22 lines
Quoted 22 lines

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2483

Quoted 7 lines
blockdev.c +1 -1
@@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
     bdrv_drained_end(bs);
 
 out:
+    aio_co_reschedule_self(old_ctx);
     blk_unref(blk);
-    aio_context_release(aio_context);

The following precondition is violated by the blk_unref -> bdrv_drain -> AIO_WAIT_WHILE() call if blk->refcnt is 1 here:

* The caller's thread must be the IOThread that owns @ctx or the main loop
* thread (with @ctx acquired exactly once).

blk_unref() is called from the main loop thread without having acquired blk's AioContext.

Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but I'm not sure if that can be guaranteed.

The following seems safer although it's uglier:

Quoted 8 lines
Quoted 4 lines
  aio_context = bdrv_get_aio_context(bs);
  aio_context_acquire(aio_context);
  blk_unref(blk);
  aio_context_release(aio_context);

May we actually acquire aio_context if blk is in the main thread? I think we must only do this if it's in a different iothread because we'd end up with a recursive lock and drain would hang.

Right :). Maybe an aio_context_acquire_once() API would help.

Stefan

author
#53 lore raw

Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine

Am 28.09.2020 um 11:05 hat Stefan Hajnoczi geschrieben:

Quoted 52 lines · Kevin Wolf
Quoted 15 lines

Am 15.09.2020 um 16:57 hat Stefan Hajnoczi geschrieben:

Quoted 14 lines · Kevin Wolf

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2462

Quoted 10 lines
blockdev.c +1 -2
@@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
         return;
     }
 
-    aio_context = bdrv_get_aio_context(bs);
-    aio_context_acquire(aio_context);
+    old_ctx = bdrv_co_move_to_aio_context(bs);
 
     if (size < 0) {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");

Is it safe to call blk_new() outside the BQL since it mutates global state?

Quoted 6 lines
Quoted 1 line

In other words, could another thread race with us?

Hm, probably not.

Would it be safer to have the bdrv_co_move_to_aio_context() call only immediately before the drain?

Yes, sounds good.

Quoted 27 lines
Quoted 26 lines

blockdev.c void qmp_block_resize(bool has_device, const char *device, · line 2483

Quoted 7 lines
blockdev.c +1 -1
@@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
     bdrv_drained_end(bs);
 
 out:
+    aio_co_reschedule_self(old_ctx);
     blk_unref(blk);
-    aio_context_release(aio_context);

The following precondition is violated by the blk_unref -> bdrv_drain -> AIO_WAIT_WHILE() call if blk->refcnt is 1 here:

* The caller's thread must be the IOThread that owns @ctx or the main loop
* thread (with @ctx acquired exactly once).

blk_unref() is called from the main loop thread without having acquired blk's AioContext.

Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but I'm not sure if that can be guaranteed.

The following seems safer although it's uglier:

  aio_context = bdrv_get_aio_context(bs);
  aio_context_acquire(aio_context);
  blk_unref(blk);
  aio_context_release(aio_context);
Quoted 5 lines
Quoted 3 lines

May we actually acquire aio_context if blk is in the main thread? I think we must only do this if it's in a different iothread because we'd end up with a recursive lock and drain would hang.

Right :). Maybe an aio_context_acquire_once() API would help.

If you want it to work in the general case, how would you implement this? As far as I know there is no way to tell whether we already own the lock or not.

Something like aio_context_acquire_unless_self() might be easier to implement.

Kevin

#54 lore raw
In reply to Kevin Wolf
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed build test on FreeBSD host. Please find the details below.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with # HEAD pointing to a commit that has the patches applied on top of "base"

# branch
if qemu-system-x86_64 --help >/dev/null 2>&1; then
  QEMU=qemu-system-x86_64
elif /usr/libexec/qemu-kvm --help >/dev/null 2>&1; then
  QEMU=/usr/libexec/qemu-kvm
else
  exit 1
fi
make vm-build-freebsd J=21 QEMU=$QEMU
exit 0
=== TEST SCRIPT END ===

The full log is available at
http://patchew.org/logs/[email protected]/testing.FreeBSD/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Acked-by on the v7 series
#55 lore raw
In reply to Kevin Wolf
Quoted 11 lines · Kevin Wolf

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?
* Are QMP events reported while the command is pending?

#56 lore raw
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Kevin Wolf

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

Quoted 6 lines
Quoted 2 lines

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?
* Are QMP events reported while the command is pending?

Good points. Kevin, I'd be willing to take this as a follow-up patch, if that's more convenient for you.

Quoted 1 line

Stefan, I could use your proper review of PATCH 11-13. Pretty-please?

#57 lore raw
Quoted 19 lines · Markus Armbruster
Quoted 10 lines · Stefan Hajnoczi
Quoted 9 lines · Kevin Wolf

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

Quoted 6 lines
Quoted 2 lines

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?
* Are QMP events reported while the command is pending?

Quoted 6 lines

Good points. Kevin, I'd be willing to take this as a follow-up patch, if that's more convenient for you.

Quoted 1 line

Stefan, I could use your proper review of PATCH 11-13. Pretty-please?

Sounds good. I have reviewed the patches 11-13 and left questions for Kevin.

author
#58 lore raw
In reply to Stefan Hajnoczi

Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:

Quoted 7 lines · Kevin Wolf
Quoted 6 lines

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

Quoted 8 lines
Quoted 5 lines

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?

Suspended as in monitor_suspend()? No.

Quoted 1 line

* Are QMP events reported while the command is pending?

Hm, I don't know to be honest. But I think so.

Does it matter, though? I don't think events have a defined order compared to command results, and the client can't respond to the event anyway until the current command has completed.

Kevin

#59 lore raw
Quoted 19 lines · Kevin Wolf

Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:

Quoted 7 lines · Kevin Wolf
Quoted 6 lines

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

Quoted 8 lines
Quoted 5 lines

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?

Suspended as in monitor_suspend()? No.

Quoted 7 lines
Quoted 1 line

* Are QMP events reported while the command is pending?

Hm, I don't know to be honest. But I think so.

Does it matter, though? I don't think events have a defined order compared to command results, and the client can't respond to the event anyway until the current command has completed.

You're right, I don't think it matters because the client must expect QMP events at any time.

I was trying to understand the semantics of coroutine monitor commands from two perspectives:

1. The QMP client - do coroutine commands behave differently from non-coroutine commands? I think the answer is no. The monitor will not process more commands until the coroutine finishes?

2. The command implementation - which thread does the coroutine run in? I guess it runs in the main loop thread with the BQL and the AioContext acquired?

author
#60 lore raw

Am 28.09.2020 um 10:46 hat Stefan Hajnoczi geschrieben:

Quoted 31 lines · Kevin Wolf
Quoted 19 lines

Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:

Quoted 7 lines · Kevin Wolf
Quoted 6 lines

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

Quoted 8 lines
Quoted 5 lines

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?

Suspended as in monitor_suspend()? No.

Quoted 7 lines
Quoted 1 line

* Are QMP events reported while the command is pending?

Hm, I don't know to be honest. But I think so.

Does it matter, though? I don't think events have a defined order compared to command results, and the client can't respond to the event anyway until the current command has completed.

You're right, I don't think it matters because the client must expect QMP events at any time.

Quoted 6 lines

I was trying to understand the semantics of coroutine monitor commands from two perspectives:

1. The QMP client - do coroutine commands behave differently from non-coroutine commands? I think the answer is no. The monitor will not process more commands until the coroutine finishes?

No, on the wire, things should look exactly the same. If you consider more than the QMP traffic and the client communicates with the guest, it might see that the guest isn't blocked any more, of course.

Quoted 3 lines

2. The command implementation - which thread does the coroutine run in? I guess it runs in the main loop thread with the BQL and the AioContext acquired?

By default, yes. But the coroutine can reschedule itself to another thread. Block-related handlers will want to reschedule themselves to bs->aio_context because you can't just hold the AioContext lock from another thread across yields. This is what block_resize does after this series.

Kevin

#61 lore raw
In reply to Kevin Wolf
Quoted 11 lines · Kevin Wolf

Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:

Quoted 10 lines · Kevin Wolf
Quoted 9 lines

Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.

This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.

Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?

Suspended as in monitor_suspend()? No.

A suspended monitor doesn't read monitor input.

We suspend

* a QMP monitor while the request queue is full

* an HMP monitor while it executes a command

* a multiplexed HMP monitor while the "mux-focus" is elsewhere

* an HMP monitor when it executes command "quit", forever

* an HMP monitor while it executes command "migrate" without -d

Let me explain the first item in a bit more detail. Before OOB, a QMP monitor was also suspended while it executed a command. To make OOB work, we moved the QMP monitors to an I/O thread and added a request queue, drained by the main loop. QMP monitors continue to read commands, execute right away if OOB, else queue, suspend when queue gets full, resume when it gets non-full.

The "run command in coroutine context" feature does not affect any of this.

qapi-code-gen.txt does not talk about monitor suspension at all. It's instead discussed in qmp-spec.txt section 2.3.1 Out-of-band execution.

Stefan, what would you like us to clarify, and where?

Quoted 3 lines
Quoted 1 line

* Are QMP events reported while the command is pending?

Hm, I don't know to be honest. But I think so.

Yes, events should be reported while a command is being executed.

Sending events takes locks. Their critical sections are all short-lived. Another possible delay is the underlying character device failing the send with EAGAIN. That's all.

Fine print: qapi_event_emit() takes @monitor_lock. It sends to each QMP monitor with qmp_send_response(), which uses monitor_puts(), which takes the monitor's @mon_lock.

The "run command in coroutine context" feature does not affect any of this.

Quoted 3 lines

Does it matter, though? I don't think events have a defined order compared to command results, and the client can't respond to the event anyway until the current command has completed.

Stefan, what would you like us to clarify, and where?