monitor: Optionally run handlers in coroutines
Subsystems (3)
v8 cover monitor: Optionally run handlers in coroutines
Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.
This series adds the infrastructure to allow this and switches block_resize to run in a coroutine as a first example.
This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.
v8:
- Replaced bdrv_co_move_to_aio_context() with bdrv_co_enter()/leave(), added bdrv_co_lock()/unlock() and fixed the iothread support in qmp_block_resize() with the new functions [Stefan]
- More detailed documentation of 'coroutine' flag for commands [Markus]
- Improved comment for dropping out of coroutine context in qmp_dispatch() [Markus]
- Coding style changes [Markus]
- Resolve some more merge conflicts on rebase
v7:
- Added patch 2 to add a Monitor parameter to monitor_get_cpu_index(), too [Markus]
- Let monitor_set_cur() return the old monitor [Markus]
- Fixed comment about linking stub objects in test-util-sockets [Markus]
- More detailed commit message for per-coroutine current monitor and document that monitor_set_cur(NULL) must be called eventually [Markus]
- Resolve some merge conflicts on rebase
v6:
- Fixed cur_mon behaviour: It should always point to the Monitor while we're in the handler coroutine, but be NULL while the handler coroutines has yielded. [Markus]
- Give HMP handlers the coroutine option, too, because they will call QMP handlers, and life is easier when we know whether we are in coroutine context or not.
- Fixed block_resize for block devices in iothreads and for HMP
- Resolved some merge conflict with QAPI generator and monitor refactorings that were merged in the meantime
v5:
- Improved comments and documentation [Markus]
v4:
- Forbid 'oob': true, 'coroutine': true [Markus]
- Removed Python type hints [Markus]
- Introduced separate bool qmp_dispatcher_co_shutdown to make it clearer how a shutdown request is signalled to the dispatcher [Markus]
- Allow using aio_poll() with iohandler_ctx and use that instead of aio_bh_poll() [Markus]
- Removed coroutine_fn from qmp_block_resize() again because at least one caller (HMP) calls it outside of coroutine context [Markus]
- Tried to make the synchronisation between dispatcher and the monitor thread clearer, and fixed a race condition
- Improved documentation and comments
v3:
- Fix race between monitor thread and dispatcher that could schedule the dispatcher coroutine twice if a second requests comes in before the dispatcher can wake up [Patchew]
v2:
- Fix typo in a commit message [Eric]
- Use hyphen instead of underscore for the test command [Eric]
- Mark qmp_block_resize() as coroutine_fn [Stefan]
Kevin Wolf (14):
monitor: Add Monitor parameter to monitor_set_cpu()
monitor: Add Monitor parameter to monitor_get_cpu_index()
monitor: Use getter/setter functions for cur_mon
hmp: Update current monitor only in handle_hmp_command()
qmp: Assert that no other monitor is active
qmp: Call monitor_set_cur() only in qmp_dispatch()
monitor: Make current monitor a per-coroutine property
qapi: Add a 'coroutine' flag for commands
qmp: Move dispatcher to a coroutine
hmp: Add support for coroutine command handlers util/async: Add aio_co_reschedule_self()
block: Add bdrv_co_enter()/leave()
block: Add bdrv_lock()/unlock()
block: Convert 'block_resize' to coroutine
qapi/block-core.json | 3 +- docs/devel/qapi-code-gen.txt | 29 ++++++ docs/sphinx/qapidoc.py | 2 +- include/block/aio.h | 10 ++ include/block/block.h | 31 ++++++ include/monitor/monitor.h | 7 +- include/qapi/qmp/dispatch.h | 5 +- monitor/monitor-internal.h | 7 +- audio/wavcapture.c | 8 +- block.c | 50 +++++++++ blockdev.c | 16 +-- dump/dump.c | 2 +- hw/core/machine-hmp-cmds.c | 2 +- hw/scsi/vhost-scsi.c | 2 +- hw/virtio/vhost-vsock.c | 2 +- migration/fd.c | 4 +- monitor/hmp-cmds.c | 4 +- monitor/hmp.c | 44 ++++++-- monitor/misc.c | 38 +++---- monitor/monitor.c | 101 ++++++++++++++++-- monitor/qmp-cmds-control.c | 2 + monitor/qmp-cmds.c | 2 +- monitor/qmp.c | 131 +++++++++++++++++------- net/socket.c | 2 +- net/tap.c | 6 +- qapi/qmp-dispatch.c | 65 +++++++++++- qapi/qmp-registry.c | 3 + qga/main.c | 2 +- softmmu/cpus.c | 2 +- stubs/monitor-core.c | 10 +- tests/test-qmp-cmds.c | 10 +- tests/test-util-sockets.c | 12 +-- trace/control.c | 2 +- util/aio-posix.c | 8 +- util/async.c | 30 ++++++ util/qemu-error.c | 6 +- util/qemu-print.c | 3 +- util/qemu-sockets.c | 1 + scripts/qapi/commands.py | 10 +- scripts/qapi/expr.py | 11 +- scripts/qapi/introspect.py | 2 +- scripts/qapi/schema.py | 13 ++- tests/qapi-schema/test-qapi.py | 7 +- hmp-commands.hx | 1 + tests/qapi-schema/meson.build | 1 + tests/qapi-schema/oob-coroutine.err | 2 + tests/qapi-schema/oob-coroutine.json | 2 + tests/qapi-schema/oob-coroutine.out | 0 tests/qapi-schema/qapi-schema-test.json | 1 + tests/qapi-schema/qapi-schema-test.out | 2 + 50 files changed, 573 insertions(+), 143 deletions(-) create mode 100644 tests/qapi-schema/oob-coroutine.err create mode 100644 tests/qapi-schema/oob-coroutine.json create mode 100644 tests/qapi-schema/oob-coroutine.out
Signature · 1 line
2.25.4
PATCH v8 1/14 monitor: Add Monitor parameter to monitor_set_cpu()
Most callers actually don't have to rely on cur_mon, but already know for which monitor they call monitor_set_cpu().
--- include/monitor/monitor.h | 2 +- monitor/hmp-cmds.c | 2 +- monitor/misc.c | 10 +++++----- 3 files changed, 7 insertions(+), 7 deletions(-)
include/monitor/monitor.h +1 -1
@@ -33,7 +33,7 @@ int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)GCC_FMT_ATTR (2 ,0 ); int monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);void monitor_flush (Monitor *mon); -int monitor_set_cpu (int cpu_index); +int monitor_set_cpu (Monitor *mon,int cpu_index);int monitor_get_cpu_index (void );void monitor_read_command (MonitorHMP *mon,int show_prompt);
monitor/hmp-cmds.c +1 -1
@@ -998,7 +998,7 @@ void hmp_cpu(Monitor *mon, const QDict *qdict)/* XXX: drop the monitor_set_cpu() usage when all HMP commands that use it are converted to the QAPI */ cpu_index =qdict_get_int (qdict,"index" ); -if (monitor_set_cpu (cpu_index) <0 ) { +if (monitor_set_cpu (mon, cpu_index) <0 ) {monitor_printf (mon,"invalid CPU index\n" ); } }
monitor/misc.c +5 -5
@@ -129,7 +129,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index, cur_mon = &hmp.common;if (has_cpu_index) { -int ret =monitor_set_cpu (cpu_index); +int ret =monitor_set_cpu (&hmp.common, cpu_index);if (ret <0 ) { cur_mon = old_mon; error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
@@ -255,7 +255,7 @@ static void monitor_init_qmp_commands(void) }/* Set the current CPU defined by the user. Callers must hold BQL. */ -int monitor_set_cpu (int cpu_index) +int monitor_set_cpu (Monitor *mon,int cpu_index) {CPUState *cpu;
@@ -263,8 +263,8 @@ int monitor_set_cpu(int cpu_index)if (cpu ==NULL ) {return -1 ; } -g_free (cur_mon->mon_cpu_path); - cur_mon->mon_cpu_path =object_get_canonical_path (OBJECT (cpu)); +g_free (mon->mon_cpu_path); + mon->mon_cpu_path =object_get_canonical_path (OBJECT (cpu));return 0 ; }
@@ -285,7 +285,7 @@ static CPUState *mon_get_cpu_sync(bool synchronize)if (!first_cpu) {return NULL ; } -monitor_set_cpu (first_cpu->cpu_index); +monitor_set_cpu (cur_mon, first_cpu->cpu_index); cpu = first_cpu; }assert (cpu !=NULL );
Signature · 1 line
2.25.4
PATCH v8 2/14 monitor: Add Monitor parameter to monitor_get_cpu_index()
In reply to Kevin WolfMost callers actually don't have to rely on cur_mon, but already know for which monitor they call monitor_get_cpu_index().
--- include/monitor/monitor.h | 2 +- hw/core/machine-hmp-cmds.c | 2 +- monitor/hmp-cmds.c | 2 +- monitor/misc.c | 20 ++++++++++---------- softmmu/cpus.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-)
include/monitor/monitor.h +1 -1
@@ -34,7 +34,7 @@ int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) int monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);void monitor_flush (Monitor *mon);int monitor_set_cpu (Monitor *mon,int cpu_index); -int monitor_get_cpu_index (void ); +int monitor_get_cpu_index (Monitor *mon);void monitor_read_command (MonitorHMP *mon,int show_prompt); int monitor_read_password(MonitorHMP *mon, ReadLineFunc *readline_func,
hw/core/machine-hmp-cmds.c +1 -1
@@ -34,7 +34,7 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)for (cpu = cpu_list; cpu; cpu = cpu->next) {int active =' ' ; -if (cpu->value->cpu_index ==monitor_get_cpu_index ()) { +if (cpu->value->cpu_index ==monitor_get_cpu_index (mon)) { active ='*' ; }
monitor/hmp-cmds.c +1 -1
@@ -1009,7 +1009,7 @@ void hmp_memsave(Monitor *mon, const QDict *qdict)const char *filename =qdict_get_str (qdict,"filename" );uint64_t addr =qdict_get_int (qdict,"val" );Error *err =NULL ; -int cpu_index =monitor_get_cpu_index (); +int cpu_index =monitor_get_cpu_index (mon);if (cpu_index <0 ) {monitor_printf (mon,"No CPU available\n" );
monitor/misc.c +10 -10
@@ -269,23 +269,23 @@ int monitor_set_cpu(Monitor *mon, int cpu_index) }/* Callers must hold BQL. */ -static CPUState *mon_get_cpu_sync (bool synchronize) +static CPUState *mon_get_cpu_sync (Monitor *mon,bool synchronize) {CPUState *cpu =NULL ; -if (cur_mon->mon_cpu_path) { - cpu = (CPUState *)object_resolve_path_type (cur_mon->mon_cpu_path, +if (mon->mon_cpu_path) { + cpu = (CPUState *)object_resolve_path_type (mon->mon_cpu_path,TYPE_CPU ,NULL );if (!cpu) { -g_free (cur_mon->mon_cpu_path); - cur_mon->mon_cpu_path =NULL ; +g_free (mon->mon_cpu_path); + mon->mon_cpu_path =NULL ; } } -if (!cur_mon->mon_cpu_path) { +if (!mon->mon_cpu_path) {if (!first_cpu) {return NULL ; } -monitor_set_cpu (cur_mon, first_cpu->cpu_index); +monitor_set_cpu (mon, first_cpu->cpu_index); cpu = first_cpu; }assert (cpu !=NULL );
@@ -297,7 +297,7 @@ static CPUState *mon_get_cpu_sync(bool synchronize)CPUState *mon_get_cpu (void ) { -return mon_get_cpu_sync (true); +return mon_get_cpu_sync (cur_mon, true); }CPUArchState *mon_get_cpu_env (void )
@@ -307,9 +307,9 @@ CPUArchState *mon_get_cpu_env(void)return cs ? cs->env_ptr :NULL ; } -int monitor_get_cpu_index (void ) +int monitor_get_cpu_index (Monitor *mon) { -CPUState *cs =mon_get_cpu_sync (false); +CPUState *cs =mon_get_cpu_sync (mon, false);return cs ? cs->cpu_index :UNASSIGNED_CPU_INDEX ; }
softmmu/cpus.c +1 -1
@@ -2224,7 +2224,7 @@ exit:void qmp_inject_nmi (Error **errp) { -nmi_monitor_handle (monitor_get_cpu_index (), errp); +nmi_monitor_handle (monitor_get_cpu_index (cur_mon), errp); }void dump_drift_info (void )
Signature · 1 line
2.25.4
Re: [PATCH v8 02/14] monitor: Add Monitor parameter to monitor_get_cpu_index()
Quoted 12 lines · Kevin Wolf
Most callers actually don't have to rely on
cur_mon, but already know for which monitor they callmonitor_get_cpu_index().Signed-off-by Kevin Wolf <[email protected]>--- include/monitor/monitor.h | 2 +- hw/core/machine-hmp-cmds.c | 2 +- monitor/hmp-cmds.c | 2 +- monitor/misc.c | 20 ++++++++++---------- softmmu/cpus.c | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v8 3/14 monitor: Use getter/setter functions for cur_mon
In reply to Kevin Wolfcur_mon really needs to be coroutine-local as soon as we move monitor command handlers to coroutines and let them yield. As a first step, just remove all direct accesses to cur_mon so that we can implement this in the getter function later.
--- include/monitor/monitor.h | 3 ++- audio/wavcapture.c | 8 ++++---- dump/dump.c | 2 +- hw/scsi/vhost-scsi.c | 2 +- hw/virtio/vhost-vsock.c | 2 +- migration/fd.c | 4 ++-- monitor/hmp.c | 11 +++++------ monitor/misc.c | 13 +++++++------ monitor/monitor.c | 24 +++++++++++++++++++++++- monitor/qmp-cmds-control.c | 2 ++ monitor/qmp-cmds.c | 2 +- monitor/qmp.c | 7 ++----- net/socket.c | 2 +- net/tap.c | 6 +++--- softmmu/cpus.c | 2 +- stubs/monitor-core.c | 5 ++++- tests/test-util-sockets.c | 12 ++++++------ trace/control.c | 2 +- util/qemu-error.c | 6 +++--- util/qemu-print.c | 3 ++- util/qemu-sockets.c | 1 + 21 files changed, 73 insertions(+), 46 deletions(-)
include/monitor/monitor.h +2 -1
@@ -5,7 +5,6 @@#include "qapi/qapi-types-misc.h" #include "qemu/readline.h" -extern __threadMonitor *cur_mon;typedef struct MonitorHMP MonitorHMP ;typedef struct MonitorOptions MonitorOptions ;
@@ -13,6 +12,8 @@ typedef struct MonitorOptions MonitorOptions;extern QemuOptsList qemu_mon_opts; +Monitor *monitor_cur (void ); +Monitor *monitor_set_cur (Monitor *mon);bool monitor_cur_is_qmp (void );void monitor_init_globals (void );
audio/wavcapture.c +4 -4
@@ -1,5 +1,5 @@#include "qemu/osdep.h" -#include "monitor/monitor.h" +#include "qemu/qemu-print.h" #include "qapi/error.h" #include "qemu/error-report.h" #include "audio.h"
@@ -94,9 +94,9 @@ static void wav_capture_info (void *opaque)WAVState *wav = opaque;char *path = wav->path; -monitor_printf (cur_mon,"Capturing audio(%d,%d,%d) to %s: %d bytes\n" , - wav->freq, wav->bits, wav->nchannels, - path ? path :"<not available>" , wav->bytes); +qemu_printf ("Capturing audio(%d,%d,%d) to %s: %d bytes\n" , + wav->freq, wav->bits, wav->nchannels, + path ? path :"<not available>" , wav->bytes); }static struct capture_ops wav_capture_ops = {
dump/dump.c +1 -1
@@ -1986,7 +1986,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,#if !defined(WIN32 )if (strstart (file,"fd:" , &p)) { - fd =monitor_get_fd (cur_mon, p, errp); + fd =monitor_get_fd (monitor_cur (), p, errp);if (fd ==-1 ) {return ; }
hw/scsi/vhost-scsi.c +1 -1
@@ -177,7 +177,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp) }if (vs->conf.vhostfd) { - vhostfd =monitor_fd_param (cur_mon, vs->conf.vhostfd, errp); + vhostfd =monitor_fd_param (monitor_cur (), vs->conf.vhostfd, errp);if (vhostfd ==-1 ) {error_prepend (errp,"vhost-scsi: unable to parse vhostfd: " );return ;
hw/virtio/vhost-vsock.c +1 -1
@@ -143,7 +143,7 @@ static void vhost_vsock_device_realize(DeviceState *dev, Error **errp) }if (vsock->conf.vhostfd) { - vhostfd =monitor_fd_param (cur_mon, vsock->conf.vhostfd, errp); + vhostfd =monitor_fd_param (monitor_cur (), vsock->conf.vhostfd, errp);if (vhostfd ==-1 ) {error_prepend (errp,"vhost-vsock: unable to parse vhostfd: " );return ;
migration/fd.c +2 -2
@@ -26,7 +26,7 @@void fd_start_outgoing_migration (MigrationState *s,const char *fdname,Error **errp) {QIOChannel *ioc; -int fd =monitor_get_fd (cur_mon, fdname, errp); +int fd =monitor_get_fd (monitor_cur (), fdname, errp);if (fd ==-1 ) {return ; }
@@ -55,7 +55,7 @@ static gboolean fd_accept_incoming_migration(QIOChannel *ioc,void fd_start_incoming_migration (const char *fdname,Error **errp) {QIOChannel *ioc; -int fd =monitor_fd_param (cur_mon, fdname, errp); +int fd =monitor_fd_param (monitor_cur (), fdname, errp);if (fd ==-1 ) {return ; }
monitor/hmp.c +5 -6
@@ -1300,12 +1300,11 @@ cleanup:static void monitor_read (void *opaque,const uint8_t *buf,int size) { -MonitorHMP *mon; -Monitor *old_mon = cur_mon; +MonitorHMP *mon =container_of (opaque, MonitorHMP, common); +Monitor *old_mon;int i; - cur_mon = opaque; - mon =container_of (cur_mon, MonitorHMP, common); + old_mon =monitor_set_cur (&mon->common);if (mon->rs) {for (i =0 ; i < size; i++) {
@@ -1313,13 +1312,13 @@ static void monitor_read(void *opaque, const uint8_t *buf, int size) } }else {if (size ==0 || buf[size -1 ] !=0 ) { -monitor_printf (cur_mon,"corrupted command\n" ); +monitor_printf (&mon->common,"corrupted command\n" ); }else {handle_hmp_command (mon, (char *)buf); } } - cur_mon = old_mon; +monitor_set_cur (old_mon); }static void monitor_event (void *opaque,QEMUChrEvent event)
monitor/misc.c +7 -6
@@ -125,13 +125,12 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,monitor_data_init (&hmp.common, false, true, false); - old_mon = cur_mon; - cur_mon = &hmp.common; + old_mon =monitor_set_cur (&hmp.common);if (has_cpu_index) {int ret =monitor_set_cpu (&hmp.common, cpu_index);if (ret <0 ) { - cur_mon = old_mon; +monitor_set_cur (old_mon);error_setg (errp,QERR_INVALID_PARAMETER_VALUE ,"cpu-index" ,"a CPU number" ); goto out;
@@ -139,7 +138,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index, }handle_hmp_command (&hmp, command_line); - cur_mon = old_mon; +monitor_set_cur (old_mon); WITH_QEMU_LOCK_GUARD(&hmp.common.mon_lock) {if (qstring_get_length (hmp.common.outbuf) >0 ) {
@@ -297,7 +296,7 @@ static CPUState *mon_get_cpu_sync(Monitor *mon, bool synchronize)CPUState *mon_get_cpu (void ) { -return mon_get_cpu_sync (cur_mon, true); +return mon_get_cpu_sync (monitor_cur (), true); }CPUArchState *mon_get_cpu_env (void )
@@ -1232,6 +1231,7 @@ static void hmp_acl_remove(Monitor *mon, const QDict *qdict)void qmp_getfd (const char *fdname,Error **errp) { +Monitor *cur_mon =monitor_cur ();mon_fd_t *monfd;int fd, tmp_fd;
@@ -1270,6 +1270,7 @@ void qmp_getfd(const char *fdname, Error **errp)void qmp_closefd (const char *fdname,Error **errp) { +Monitor *cur_mon =monitor_cur ();mon_fd_t *monfd;int tmp_fd;
@@ -1356,7 +1357,7 @@ AddfdInfo *qmp_add_fd(bool has_fdset_id, int64_t fdset_id, bool has_opaque, const char *opaque, Error **errp) { int fd; -Monitor *mon = cur_mon; +Monitor *mon =monitor_cur ();AddfdInfo *fdinfo; fd =qemu_chr_fe_get_msgfd (&mon->chr);
monitor/monitor.c +23 -1
@@ -66,13 +66,31 @@ MonitorList mon_list;int mon_refcount;static bool monitor_destroyed; -__threadMonitor *cur_mon; +static __threadMonitor *cur_monitor; + +Monitor *monitor_cur (void ) +{ +return cur_monitor; +} + +/** +* Sets a new current monitor and returns the old one. +*/ +Monitor *monitor_set_cur (Monitor *mon) +{ +Monitor *old_monitor = cur_monitor; + + cur_monitor = mon; +return old_monitor; +}/** * Is the current monitor, if any, a QMP monitor? */ bool monitor_cur_is_qmp (void ) { +Monitor *cur_mon =monitor_cur (); +return cur_mon &&monitor_is_qmp (cur_mon); }
@@ -209,6 +227,8 @@ int monitor_printf(Monitor *mon, const char *fmt, ...)*/ int error_vprintf (const char *fmt,va_list ap) { +Monitor *cur_mon =monitor_cur (); +if (cur_mon && !monitor_cur_is_qmp ()) {return monitor_vprintf (cur_mon, fmt, ap); }
@@ -217,6 +237,8 @@ int error_vprintf(const char *fmt, va_list ap)int error_vprintf_unless_qmp (const char *fmt,va_list ap) { +Monitor *cur_mon =monitor_cur (); +if (!cur_mon) {return vfprintf (stderr, fmt, ap); }
monitor/qmp-cmds-control.c +2 -0
@@ -69,6 +69,7 @@ static bool qmp_caps_accept(MonitorQMP *mon, QMPCapabilityList *list,void qmp_qmp_capabilities (bool has_enable,QMPCapabilityList *enable,Error **errp) { +Monitor *cur_mon =monitor_cur ();MonitorQMP *mon;assert (monitor_is_qmp (cur_mon));
@@ -119,6 +120,7 @@ static void query_commands_cb(const QmpCommand *cmd, void *opaque)CommandInfoList *qmp_query_commands (Error **errp) {CommandInfoList *list =NULL ; +Monitor *cur_mon =monitor_cur ();MonitorQMP *mon;assert (monitor_is_qmp (cur_mon));
monitor/qmp-cmds.c +1 -1
@@ -328,7 +328,7 @@ void qmp_add_client(const char *protocol, const char *fdname,Chardev *s;int fd; - fd =monitor_get_fd (cur_mon, fdname, errp); + fd =monitor_get_fd (monitor_cur (), fdname, errp);if (fd <0 ) {return ; }
monitor/qmp.c +2 -5
@@ -139,12 +139,9 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)QDict *rsp;QDict *error; - old_mon = cur_mon; - cur_mon = &mon->common; - + old_mon =monitor_set_cur (&mon->common); rsp =qmp_dispatch (mon->commands, req,qmp_oob_enabled (mon)); - - cur_mon = old_mon; +monitor_set_cur (old_mon);if (mon->commands == &qmp_cap_negotiation_commands) { error =qdict_get_qdict (rsp,"error" );
net/socket.c +1 -1
@@ -727,7 +727,7 @@ int net_init_socket(const Netdev *netdev, const char *name,if (sock->has_fd) {int fd, ret; - fd =monitor_fd_param (cur_mon, sock->fd, errp); + fd =monitor_fd_param (monitor_cur (), sock->fd, errp);if (fd ==-1 ) {return -1 ; }
net/tap.c +3 -3
@@ -700,7 +700,7 @@ static void net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,if (vhostfdname) {int ret; - vhostfd =monitor_fd_param (cur_mon, vhostfdname, &err); + vhostfd =monitor_fd_param (monitor_cur (), vhostfdname, &err);if (vhostfd ==-1 ) {if (tap->has_vhostforce && tap->vhostforce) {error_propagate (errp, err);
@@ -808,7 +808,7 @@ int net_init_tap(const Netdev *netdev, const char *name,return -1 ; } - fd =monitor_fd_param (cur_mon, tap->fd, errp); + fd =monitor_fd_param (monitor_cur (), tap->fd, errp);if (fd ==-1 ) {return -1 ; }
@@ -862,7 +862,7 @@ int net_init_tap(const Netdev *netdev, const char *name, }for (i =0 ; i < nfds; i++) { - fd =monitor_fd_param (cur_mon, fds[i], errp); + fd =monitor_fd_param (monitor_cur (), fds[i], errp);if (fd ==-1 ) { ret =-1 ; goto free_fail;
softmmu/cpus.c +1 -1
@@ -2224,7 +2224,7 @@ exit:void qmp_inject_nmi (Error **errp) { -nmi_monitor_handle (monitor_get_cpu_index (cur_mon), errp); +nmi_monitor_handle (monitor_get_cpu_index (monitor_cur ()), errp); }void dump_drift_info (void )
stubs/monitor-core.c +4 -1
@@ -3,7 +3,10 @@#include "qemu-common.h" #include "qapi/qapi-emit-events.h" -__threadMonitor *cur_mon; +Monitor *monitor_cur (void ) +{ +return NULL ; +}void monitor_init_qmp (Chardev *chr,bool pretty,Error **errp) {
tests/test-util-sockets.c +6 -6
@@ -52,6 +52,7 @@ static void test_fd_is_socket_good(void)static int mon_fd =-1 ;static const char *mon_fdname; +__threadMonitor *cur_mon;int monitor_get_fd (Monitor *mon,const char *fdname,Error **errp) {
@@ -65,15 +66,14 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp) }/* -* Syms of stubs in libqemuutil.a are discarded at .o file granularity. -* To replace monitor_get_fd() we must ensure everything in -* stubs/monitor.c is defined, to make sure monitor.o is discarded +* Syms of stubs in libqemuutil.a are discarded at .o file +* granularity. To replace monitor_get_fd() and monitor_cur(), we +* must ensure that we also replace any other symbol that is used in +* the binary and would be taken from the same stub object file, * otherwise we get duplicate syms at link time. */ -__threadMonitor *cur_mon; +Monitor *monitor_cur (void ) {return cur_mon; }int monitor_vprintf (Monitor *mon,const char *fmt,va_list ap) {abort (); } -void monitor_init_qmp (Chardev *chr,bool pretty,Error **errp) {} -void monitor_init_hmp (Chardev *chr,bool use_readline,Error **errp) {}#ifndef _WIN32static void test_socket_fd_pass_name_good (void )
trace/control.c +1 -1
@@ -176,7 +176,7 @@ void trace_enable_events(const char *line_buf) {if (is_help_option (line_buf)) {trace_list_events (); -if (cur_mon ==NULL ) { +if (monitor_cur () ==NULL ) {exit (0 ); } }else {
util/qemu-error.c +3 -3
@@ -171,7 +171,7 @@ static void print_loc(void)int i;const char *const *argp; -if (!cur_mon && progname) { +if (!monitor_cur () && progname) {fprintf (stderr,"%s:" , progname); sep =" " ; }
@@ -208,7 +208,7 @@ static void vreport(report_type type, const char *fmt, va_list ap)GTimeVal tv;gchar *timestr; -if (error_with_timestamp && !cur_mon) { +if (error_with_timestamp && !monitor_cur ()) {g_get_current_time (&tv); timestr =g_time_val_to_iso8601 (&tv);error_printf ("%s " , timestr);
@@ -216,7 +216,7 @@ static void vreport(report_type type, const char *fmt, va_list ap) }/* Only prepend guest name if -msg guest-name and -name guest=... are set */ -if (error_with_guestname && error_guest_name && !cur_mon) { +if (error_with_guestname && error_guest_name && !monitor_cur ()) {error_printf ("%s " , error_guest_name); }
util/qemu-print.c +2 -1
@@ -20,6 +20,7 @@*/ int qemu_vprintf (const char *fmt,va_list ap) { +Monitor *cur_mon =monitor_cur ();if (cur_mon) {return monitor_vprintf (cur_mon, fmt, ap); }
@@ -48,7 +49,7 @@ int qemu_printf(const char *fmt, ...)int qemu_vfprintf (FILE *stream,const char *fmt,va_list ap) {if (!stream) { -return monitor_vprintf (cur_mon, fmt, ap); +return monitor_vprintf (monitor_cur (), fmt, ap); }return vfprintf (stream, fmt, ap); }
util/qemu-sockets.c +1 -0
@@ -1092,6 +1092,7 @@ fail:static int socket_get_fd (const char *fdstr,int num,Error **errp) { +Monitor *cur_mon =monitor_cur ();int fd;if (num !=1 ) {error_setg_errno (errp,EINVAL ,"socket_get_fd: too many connections" );
Signature · 1 line
2.25.4
PATCH v8 4/14 hmp: Update current monitor only in handle_hmp_command()
In reply to Kevin WolfThe current monitor is updated relatively early in the command handling code even though only the command handler actually needs it.
The current monitor will become coroutine-local later, so we can only update it when we know in which coroutine the command will be exectued. Move it to handle_hmp_command() where this information will be available.
--- monitor/hmp.c | 10 +++++----- monitor/misc.c | 5 ----- 2 files changed, 5 insertions(+), 10 deletions(-)
monitor/hmp.c +5 -5
@@ -1061,6 +1061,7 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)QDict *qdict;const HMPCommand *cmd;const char *cmd_start = cmdline; +Monitor *old_mon;trace_handle_hmp_command (mon, cmdline);
@@ -1079,7 +1080,11 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)return ; } +/* old_mon is non-NULL when called from qmp_human_monitor_command() */ + old_mon =monitor_set_cur (&mon->common); cmd->cmd (&mon->common, qdict); +monitor_set_cur (old_mon); +qobject_unref (qdict); }
@@ -1301,11 +1306,8 @@ cleanup:static void monitor_read (void *opaque,const uint8_t *buf,int size) {MonitorHMP *mon =container_of (opaque, MonitorHMP, common); -Monitor *old_mon;int i; - old_mon =monitor_set_cur (&mon->common); -if (mon->rs) {for (i =0 ; i < size; i++) {readline_handle_byte (mon->rs, buf[i]);
@@ -1317,8 +1319,6 @@ static void monitor_read(void *opaque, const uint8_t *buf, int size)handle_hmp_command (mon, (char *)buf); } } - -monitor_set_cur (old_mon); }static void monitor_event (void *opaque,QEMUChrEvent event)
monitor/misc.c +0 -5
@@ -120,17 +120,13 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index, int64_t cpu_index, Error **errp) { char *output = NULL; -Monitor *old_mon;MonitorHMP hmp = {};monitor_data_init (&hmp.common, false, true, false); - old_mon =monitor_set_cur (&hmp.common); -if (has_cpu_index) {int ret =monitor_set_cpu (&hmp.common, cpu_index);if (ret <0 ) { -monitor_set_cur (old_mon);error_setg (errp,QERR_INVALID_PARAMETER_VALUE ,"cpu-index" ,"a CPU number" ); goto out;
@@ -138,7 +134,6 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index, }handle_hmp_command (&hmp, command_line); -monitor_set_cur (old_mon); WITH_QEMU_LOCK_GUARD(&hmp.common.mon_lock) {if (qstring_get_length (hmp.common.outbuf) >0 ) {
Signature · 1 line
2.25.4
PATCH v8 5/14 qmp: Assert that no other monitor is active
In reply to Kevin Wolfmonitor_qmp_dispatch() is never supposed to be called in the context of another monitor, so assert that monitor_cur() is NULL instead of saving and restoring it.
--- monitor/qmp.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
monitor/qmp.c +4 -1
@@ -140,8 +140,11 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)QDict *error; old_mon =monitor_set_cur (&mon->common); +assert (old_mon ==NULL ); + rsp =qmp_dispatch (mon->commands, req,qmp_oob_enabled (mon)); -monitor_set_cur (old_mon); + +monitor_set_cur (NULL );if (mon->commands == &qmp_cap_negotiation_commands) { error =qdict_get_qdict (rsp,"error" );
Signature · 1 line
2.25.4
PATCH v8 6/14 qmp: Call monitor_set_cur() only in qmp_dispatch()
In reply to Kevin WolfThe correct way to set the current monitor for a coroutine handler will be different than for a blocking handler, so monitor_set_cur() needs to be called in qmp_dispatch().
--- include/qapi/qmp/dispatch.h | 3 ++- monitor/qmp.c | 9 ++------- qapi/qmp-dispatch.c | 8 +++++++- qga/main.c | 2 +- stubs/monitor-core.c | 5 +++++ tests/test-qmp-cmds.c | 6 +++--- 6 files changed, 20 insertions(+), 13 deletions(-)
include/qapi/qmp/dispatch.h +2 -1
@@ -14,6 +14,7 @@#ifndef QAPI_QMP_DISPATCH_H #define QAPI_QMP_DISPATCH_H +#include "monitor/monitor.h" #include "qemu/queue.h" typedef void (QmpCommandFunc )(QDict *,QObject **,Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);bool qmp_has_success_response (const QmpCommand *cmd);QDict *qmp_error_response (Error *err);QDict *qmp_dispatch (const QmpCommandList *cmds,QObject *request, -bool allow_oob); +bool allow_oob,Monitor *cur_mon);bool qmp_is_oob (const QDict *dict);typedef void (*qmp_cmd_callback_fn )(const QmpCommand *cmd,void *opaque);
monitor/qmp.c +2 -7
@@ -135,16 +135,11 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)static void monitor_qmp_dispatch (MonitorQMP *mon,QObject *req) { -Monitor *old_mon;QDict *rsp;QDict *error; - old_mon =monitor_set_cur (&mon->common); -assert (old_mon ==NULL ); - - rsp =qmp_dispatch (mon->commands, req,qmp_oob_enabled (mon)); - -monitor_set_cur (NULL ); + rsp =qmp_dispatch (mon->commands, req,qmp_oob_enabled (mon), + &mon->common);if (mon->commands == &qmp_cap_negotiation_commands) { error =qdict_get_qdict (rsp,"error" );
qapi/qmp-dispatch.c +7 -1
@@ -89,7 +89,7 @@ bool qmp_is_oob(const QDict *dict) }QDict *qmp_dispatch (const QmpCommandList *cmds,QObject *request, -bool allow_oob) +bool allow_oob,Monitor *cur_mon) {Error *err =NULL ;bool oob;
@@ -152,7 +152,13 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request, args =qdict_get_qdict (dict,"arguments" );qobject_ref (args); } + +assert (monitor_cur () ==NULL ); +monitor_set_cur (cur_mon); + cmd->fn (args, &ret, &err); + +monitor_set_cur (NULL );qobject_unref (args);if (err) {/* or assert(!ret) after reviewing all handlers: */
qga/main.c +1 -1
@@ -579,7 +579,7 @@ static void process_event(void *opaque, QObject *obj, Error *err) }g_debug ("processing command" ); - rsp =qmp_dispatch (&ga_commands, obj, false); + rsp =qmp_dispatch (&ga_commands, obj, false,NULL ); end: ret =send_response (s, rsp);
stubs/monitor-core.c +5 -0
@@ -8,6 +8,11 @@ Monitor *monitor_cur(void)return NULL ; } +Monitor *monitor_set_cur (Monitor *mon) +{ +return NULL ; +} +void monitor_init_qmp (Chardev *chr,bool pretty,Error **errp) { }
tests/test-qmp-cmds.c +3 -3
@@ -152,7 +152,7 @@ static QObject *do_qmp_dispatch(bool allow_oob, const char *template, ...) req =qdict_from_vjsonf_nofail (template, ap);va_end (ap); - resp =qmp_dispatch (&qmp_commands,QOBJECT (req), allow_oob); + resp =qmp_dispatch (&qmp_commands,QOBJECT (req), allow_oob,NULL );g_assert (resp); ret =qdict_get (resp,"return" );g_assert (ret);
@@ -175,7 +175,7 @@ static void do_qmp_dispatch_error(bool allow_oob, ErrorClass cls, req =qdict_from_vjsonf_nofail (template, ap);va_end (ap); - resp =qmp_dispatch (&qmp_commands,QOBJECT (req), allow_oob); + resp =qmp_dispatch (&qmp_commands,QOBJECT (req), allow_oob,NULL );g_assert (resp); error =qdict_get_qdict (resp,"error" );g_assert (error);
@@ -231,7 +231,7 @@ static void test_dispatch_cmd_success_response(void)QDict *resp;qdict_put_str (req,"execute" ,"cmd-success-response" ); - resp =qmp_dispatch (&qmp_commands,QOBJECT (req), false); + resp =qmp_dispatch (&qmp_commands,QOBJECT (req), false,NULL );g_assert_null (resp);qobject_unref (req); }
Signature · 1 line
2.25.4
PATCH v8 7/14 monitor: Make current monitor a per-coroutine property
In reply to Kevin WolfThis way, a monitor command handler will still be able to access the current monitor, but when it yields, all other code code will correctly get NULL from monitor_cur().
This uses a hash table to map the coroutine pointer to the current monitor of that coroutine. Outside of coroutine context, we associate the current monitor with the leader coroutine of the current thread.
Approaches to implement some form of coroutine local storage directly in the coroutine core code have been considered and discarded because they didn't end up being much more generic than the hash table and their performance impact on coroutines not using coroutine local storage was unclear. As the block layer uses a coroutine per I/O request, this is a fast path and we have to be careful. It's safest to just stay out of this path with code only used by the monitor.
--- include/monitor/monitor.h | 2 +- monitor/hmp.c | 4 ++-- monitor/monitor.c | 34 +++++++++++++++++++++++++++------- qapi/qmp-dispatch.c | 4 ++-- stubs/monitor-core.c | 2 +- 5 files changed, 33 insertions(+), 13 deletions(-)
include/monitor/monitor.h +1 -1
@@ -13,7 +13,7 @@ typedef struct MonitorOptions MonitorOptions;extern QemuOptsList qemu_mon_opts;Monitor *monitor_cur (void ); -Monitor *monitor_set_cur (Monitor *mon); +Monitor *monitor_set_cur (Coroutine *co,Monitor *mon);bool monitor_cur_is_qmp (void );void monitor_init_globals (void );
monitor/hmp.c +2 -2
@@ -1081,9 +1081,9 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline) }/* old_mon is non-NULL when called from qmp_human_monitor_command() */ - old_mon =monitor_set_cur (&mon->common); + old_mon =monitor_set_cur (qemu_coroutine_self (), &mon->common); cmd->cmd (&mon->common, qdict); -monitor_set_cur (old_mon); +monitor_set_cur (qemu_coroutine_self (), old_mon);qobject_unref (qdict); }
monitor/monitor.c +27 -7
@@ -58,29 +58,48 @@ IOThread *mon_iothread;/* Bottom half to dispatch the requests received from I/O thread */ QEMUBH *qmp_dispatcher_bh; -/* Protects mon_list, monitor_qapi_event_state, monitor_destroyed. */ +/* +* Protects mon_list, monitor_qapi_event_state, coroutine_mon, +* monitor_destroyed. +*/ QemuMutex monitor_lock;static GHashTable *monitor_qapi_event_state; +static GHashTable *coroutine_mon;/* Maps Coroutine* to Monitor* */ MonitorList mon_list;int mon_refcount;static bool monitor_destroyed; -static __threadMonitor *cur_monitor; -Monitor *monitor_cur (void ) { -return cur_monitor; +Monitor *mon; + +qemu_mutex_lock (&monitor_lock); + mon =g_hash_table_lookup (coroutine_mon,qemu_coroutine_self ()); +qemu_mutex_unlock (&monitor_lock); + +return mon; }/** * Sets a new current monitor and returns the old one. +* +* If a non-NULL monitor is set for a coroutine, another call +* resetting it to NULL is required before the coroutine terminates, +* otherwise a stale entry would remain in the hash table. */ -Monitor *monitor_set_cur (Monitor *mon) +Monitor *monitor_set_cur (Coroutine *co,Monitor *mon) { -Monitor *old_monitor = cur_monitor; +Monitor *old_monitor =monitor_cur (); + +qemu_mutex_lock (&monitor_lock); +if (mon) { +g_hash_table_replace (coroutine_mon, co, mon); + }else { +g_hash_table_remove (coroutine_mon, co); + } +qemu_mutex_unlock (&monitor_lock); - cur_monitor = mon;return old_monitor; }
@@ -623,6 +642,7 @@ void monitor_init_globals_core(void) {monitor_qapi_event_init ();qemu_mutex_init (&monitor_lock); + coroutine_mon =g_hash_table_new (NULL ,NULL );/* * The dispatcher BH must run in the main loop thread, since we
qapi/qmp-dispatch.c +2 -2
@@ -154,11 +154,11 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request, }assert (monitor_cur () ==NULL ); -monitor_set_cur (cur_mon); +monitor_set_cur (qemu_coroutine_self (), cur_mon); cmd->fn (args, &ret, &err); -monitor_set_cur (NULL ); +monitor_set_cur (qemu_coroutine_self (),NULL );qobject_unref (args);if (err) {/* or assert(!ret) after reviewing all handlers: */
stubs/monitor-core.c +1 -1
@@ -8,7 +8,7 @@ Monitor *monitor_cur(void)return NULL ; } -Monitor *monitor_set_cur (Monitor *mon) +Monitor *monitor_set_cur (Coroutine *co,Monitor *mon) {return NULL ; }
Signature · 1 line
2.25.4
PATCH v8 8/14 qapi: Add a 'coroutine' flag for commands
In reply to Kevin WolfThis patch adds a new 'coroutine' flag to QMP command definitions that tells the QMP dispatcher that the command handler is safe to be run in a coroutine.
The documentation of the new flag pretends that this flag is already used as intended, which it isn't yet after this patch. We'll implement this in another patch in this series.
--- docs/devel/qapi-code-gen.txt | 29 +++++++++++++++++++++++++ docs/sphinx/qapidoc.py | 2 +- include/qapi/qmp/dispatch.h | 1 + tests/test-qmp-cmds.c | 4 ++++ scripts/qapi/commands.py | 10 ++++++--- scripts/qapi/expr.py | 11 ++++++++-- scripts/qapi/introspect.py | 2 +- scripts/qapi/schema.py | 13 +++++++---- tests/qapi-schema/test-qapi.py | 7 +++--- tests/qapi-schema/meson.build | 1 + tests/qapi-schema/oob-coroutine.err | 2 ++ tests/qapi-schema/oob-coroutine.json | 2 ++ tests/qapi-schema/oob-coroutine.out | 0 tests/qapi-schema/qapi-schema-test.json | 1 + tests/qapi-schema/qapi-schema-test.out | 2 ++ 15 files changed, 73 insertions(+), 14 deletions(-) create mode 100644 tests/qapi-schema/oob-coroutine.err create mode 100644 tests/qapi-schema/oob-coroutine.json create mode 100644 tests/qapi-schema/oob-coroutine.out
docs/devel/qapi-code-gen.txt +29 -0
@@ -472,6 +472,7 @@ Syntax: '*gen': false, '*allow-oob': true, '*allow-preconfig': true, + '*coroutine': true, '*if': COND, '*features': FEATURES }
@@ -596,6 +597,34 @@ before the machine is built. It defaults to false. For example: QMP is available before the machine is built only when QEMU was started with --preconfig. +Member 'coroutine' tells the QMP dispatcher whether the command handler +is safe to be run in a coroutine. It defaults to false. If it is true, +the command handler is called from coroutine context and may yield while +waiting for an external event (such as I/O completion) in order to avoid +blocking the guest and other background operations. + +Coroutine safety can be hard to prove, similar to thread safety. Common +pitfalls are: + +- The global mutex isn't held across qemu_coroutine_yield(), so + operations that used to assume that they execute atomically may have + to be more careful to protect against changes in the global state. + +- Nested event loops (AIO_WAIT_WHILE() etc.) are problematic in + coroutine context and can easily lead to deadlocks. They should be + replaced by yielding and reentering the coroutine when the condition + becomes false. + +Since the command handler may assume coroutine context, any callers +other than the QMP dispatcher must also call it in coroutine context. +In particular, HMP commands calling such a QMP command handler must +enter coroutine context before calling the handler. + +It is an error to specify both 'coroutine': true and 'allow-oob': true +for a command. We don't currently have a use case for both together and +without a use case, it's not entirely clear what the semantics should +be. + The optional 'if' member specifies a conditional. See "Configuring the schema" below for more on this.
docs/sphinx/qapidoc.py +1 -1
@@ -330,7 +330,7 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor): def visit_command(self, name, info, ifcond, features, arg_type, ret_type, gen, success_response, boxed, allow_oob, - allow_preconfig): + allow_preconfig, coroutine): doc = self._cur_doc self._add_doc('Command', self._nodes_for_arguments(doc,
include/qapi/qmp/dispatch.h +1 -0
@@ -25,6 +25,7 @@ typedef enum QmpCommandOptionsQCO_NO_SUCCESS_RESP = (1U <<0 ),QCO_ALLOW_OOB = (1U <<1 ),QCO_ALLOW_PRECONFIG = (1U <<2 ), +QCO_COROUTINE = (1U <<3 ), } QmpCommandOptions; typedef struct QmpCommand
tests/test-qmp-cmds.c +4 -0
@@ -36,6 +36,10 @@ void qmp_cmd_success_response(Error **errp) { } +void qmp_coroutine_cmd (Error **errp) +{ +} +Empty2 *qmp_user_def_cmd0 (Error **errp) {return g_new0 (Empty2,1 );
scripts/qapi/commands.py +7 -3
@@ -176,7 +176,8 @@ out: return ret -def gen_register_command(name, success_response, allow_oob, allow_preconfig): +def gen_register_command(name, success_response, allow_oob, allow_preconfig, + coroutine): options = [] if not success_response:
@@ -185,6 +186,8 @@ def gen_register_command(name, success_response, allow_oob, allow_preconfig): options += ['QCO_ALLOW_OOB'] if allow_preconfig: options += ['QCO_ALLOW_PRECONFIG'] + if coroutine: + options += ['QCO_COROUTINE'] if not options: options = ['QCO_NO_OPTIONS']
@@ -267,7 +270,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); def visit_command(self, name, info, ifcond, features, arg_type, ret_type, gen, success_response, boxed, - allow_oob, allow_preconfig): + allow_oob, allow_preconfig, coroutine): if not gen: return # FIXME: If T is a user-defined type, the user is responsible
@@ -285,7 +288,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds); self._genh.add(gen_marshal_decl(name)) self._genc.add(gen_marshal(name, arg_type, boxed, ret_type)) self._regy.add(gen_register_command(name, success_response, - allow_oob, allow_preconfig)) + allow_oob, allow_preconfig, + coroutine)) def gen_commands(schema, output_dir, prefix):
scripts/qapi/expr.py +9 -2
@@ -88,10 +88,17 @@ def check_flags(expr, info): if key in expr and expr[key] is not False: raise QAPISemError( info, "flag '%s' may only use false value" % key) - for key in ['boxed', 'allow-oob', 'allow-preconfig']: + for key in ['boxed', 'allow-oob', 'allow-preconfig', 'coroutine']: if key in expr and expr[key] is not True: raise QAPISemError( info, "flag '%s' may only use true value" % key) + if 'allow-oob' in expr and 'coroutine' in expr: + # This is not necessarily a fundamental incompatibility, but + # we don't have a use case and the desired semantics isn't + # obvious. The simplest solution is to forbid it until we get + # a use case for it. + raise QAPISemError(info, "flags 'allow-oob' and 'coroutine' " + "are incompatible") def check_if(expr, info, source):
@@ -342,7 +349,7 @@ def check_exprs(exprs): ['command'], ['data', 'returns', 'boxed', 'if', 'features', 'gen', 'success-response', 'allow-oob', - 'allow-preconfig']) + 'allow-preconfig', 'coroutine']) normalize_members(expr.get('data')) check_command(expr, info) elif meta == 'event':
scripts/qapi/introspect.py +1 -1
@@ -216,7 +216,7 @@ const QLitObject %(c_name)s = %(c_string)s; def visit_command(self, name, info, ifcond, features, arg_type, ret_type, gen, success_response, boxed, - allow_oob, allow_preconfig): + allow_oob, allow_preconfig, coroutine): arg_type = arg_type or self._schema.the_empty_object_type ret_type = ret_type or self._schema.the_empty_object_type obj = {'arg-type': self._use_type(arg_type),
scripts/qapi/schema.py +9 -4
@@ -128,7 +128,7 @@ class QAPISchemaVisitor: def visit_command(self, name, info, ifcond, features, arg_type, ret_type, gen, success_response, boxed, - allow_oob, allow_preconfig): + allow_oob, allow_preconfig, coroutine): pass def visit_event(self, name, info, ifcond, features, arg_type, boxed):
@@ -713,7 +713,8 @@ class QAPISchemaCommand(QAPISchemaEntity): def __init__(self, name, info, doc, ifcond, features, arg_type, ret_type, - gen, success_response, boxed, allow_oob, allow_preconfig): + gen, success_response, boxed, allow_oob, allow_preconfig, + coroutine): super().__init__(name, info, doc, ifcond, features) assert not arg_type or isinstance(arg_type, str) assert not ret_type or isinstance(ret_type, str)
@@ -726,6 +727,7 @@ class QAPISchemaCommand(QAPISchemaEntity): self.boxed = boxed self.allow_oob = allow_oob self.allow_preconfig = allow_preconfig + self.coroutine = coroutine def check(self, schema): super().check(schema)
@@ -768,7 +770,8 @@ class QAPISchemaCommand(QAPISchemaEntity): visitor.visit_command( self.name, self.info, self.ifcond, self.features, self.arg_type, self.ret_type, self.gen, self.success_response, - self.boxed, self.allow_oob, self.allow_preconfig) + self.boxed, self.allow_oob, self.allow_preconfig, + self.coroutine) class QAPISchemaEvent(QAPISchemaEntity):
@@ -1074,6 +1077,7 @@ class QAPISchema: boxed = expr.get('boxed', False) allow_oob = expr.get('allow-oob', False) allow_preconfig = expr.get('allow-preconfig', False) + coroutine = expr.get('coroutine', False) ifcond = expr.get('if') features = self._make_features(expr.get('features'), info) if isinstance(data, OrderedDict):
@@ -1086,7 +1090,8 @@ class QAPISchema: self._def_entity(QAPISchemaCommand(name, info, doc, ifcond, features, data, rets, gen, success_response, - boxed, allow_oob, allow_preconfig)) + boxed, allow_oob, allow_preconfig, + coroutine)) def _def_event(self, expr, info, doc): name = expr['event']
tests/qapi-schema/test-qapi.py +4 -3
@@ -68,12 +68,13 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor): def visit_command(self, name, info, ifcond, features, arg_type, ret_type, gen, success_response, boxed, - allow_oob, allow_preconfig): + allow_oob, allow_preconfig, coroutine): print('command %s %s -> %s' % (name, arg_type and arg_type.name, ret_type and ret_type.name)) - print(' gen=%s success_response=%s boxed=%s oob=%s preconfig=%s' - % (gen, success_response, boxed, allow_oob, allow_preconfig)) + print(' gen=%s success_response=%s boxed=%s oob=%s preconfig=%s%s' + % (gen, success_response, boxed, allow_oob, allow_preconfig, + " coroutine=True" if coroutine else "")) self._print_if(ifcond) self._print_features(features)
tests/qapi-schema/meson.build +1 -0
@@ -142,6 +142,7 @@ schemas = [ 'nested-struct-data.json', 'nested-struct-data-invalid-dict.json', 'non-objects.json', + 'oob-coroutine.json', 'oob-test.json', 'allow-preconfig-test.json', 'pragma-doc-required-crap.json',
tests/qapi-schema/oob-coroutine.err new file +2 -0
@@ -0,0 +1,2 @@ +oob-coroutine.json: In command 'oob-command-1': +oob-coroutine.json:2: flags 'allow-oob' and 'coroutine' are incompatible
tests/qapi-schema/oob-coroutine.json new file +2 -0
@@ -0,0 +1,2 @@ +# Check that incompatible flags allow-oob and coroutine are rejected +{ 'command': 'oob-command-1', 'allow-oob': true, 'coroutine': true }
tests/qapi-schema/oob-coroutine.out new file +0 -0
tests/qapi-schema/qapi-schema-test.json +1 -0
@@ -148,6 +148,7 @@ 'returns': 'UserDefTwo' } { 'command': 'cmd-success-response', 'data': {}, 'success-response': false } +{ 'command': 'coroutine-cmd', 'data': {}, 'coroutine': true } # Returning a non-dictionary requires a name from the whitelist { 'command': 'guest-get-time', 'data': {'a': 'int', '*b': 'int' },
tests/qapi-schema/qapi-schema-test.out +2 -0
@@ -203,6 +203,8 @@ command user_def_cmd2 q_obj_user_def_cmd2-arg -> UserDefTwo gen=True success_response=True boxed=False oob=False preconfig=False command cmd-success-response None -> None gen=True success_response=False boxed=False oob=False preconfig=False +command coroutine-cmd None -> None + gen=True success_response=True boxed=False oob=False preconfig=False coroutine=True object q_obj_guest-get-time-arg member a: int optional=False member b: int optional=True
Signature · 1 line
2.25.4
PATCH v8 9/14 qmp: Move dispatcher to a coroutine
In reply to Kevin WolfThis moves the QMP dispatcher to a coroutine and runs all QMP command handlers that declare 'coroutine': true in coroutine context so they can avoid blocking the main loop while doing I/O or waiting for other events.
For commands that are not declared safe to run in a coroutine, the dispatcher drops out of coroutine context by calling the QMP command handler from a bottom half.
--- include/qapi/qmp/dispatch.h | 1 + monitor/monitor-internal.h | 6 +- monitor/monitor.c | 55 +++++++++++++--- monitor/qmp.c | 122 +++++++++++++++++++++++++++--------- qapi/qmp-dispatch.c | 65 +++++++++++++++++-- qapi/qmp-registry.c | 3 + util/aio-posix.c | 8 ++- 7 files changed, 214 insertions(+), 46 deletions(-)
include/qapi/qmp/dispatch.h +1 -0
@@ -31,6 +31,7 @@ typedef enum QmpCommandOptions typedef struct QmpCommand { const char *name; + /* Runs in coroutine context if QCO_COROUTINE is set */ QmpCommandFunc *fn; QmpCommandOptions options; QTAILQ_ENTRY(QmpCommand) node;
monitor/monitor-internal.h +4 -2
@@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon) typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;extern IOThread *mon_iothread; -extern QEMUBH *qmp_dispatcher_bh; +extern Coroutine *qmp_dispatcher_co; +extern bool qmp_dispatcher_co_shutdown; +extern bool qmp_dispatcher_co_busy;extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;extern QemuMutex monitor_lock;extern MonitorList mon_list;
@@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);void qmp_send_response (MonitorQMP *mon,const QDict *rsp);void monitor_data_destroy_qmp (MonitorQMP *mon); -void monitor_qmp_bh_dispatcher (void *data); +void coroutine_fn monitor_qmp_dispatcher_co(void *data);int get_monitor_def (int64_t *pval,const char *name);void help_cmd (Monitor *mon,const char *name);
monitor/monitor.c +47 -8
73 lines not on this page · raw message
monitor/qmp.c +92 -30
148 lines not on this page · raw message
qapi/qmp-dispatch.c +60 -5
90 lines not on this page · raw message
qapi/qmp-registry.c +3 -0
@@ -20,6 +20,9 @@ void qmp_register_command(QmpCommandList *cmds, const char *name, {QmpCommand *cmd =g_malloc0 (sizeof (*cmd)); +/* QCO_COROUTINE and QCO_ALLOW_OOB are incompatible for now */ +assert (!((options &QCO_COROUTINE ) && (options &QCO_ALLOW_OOB ))); + cmd->name = name; cmd->fn = fn; cmd->enabled = true;
util/aio-posix.c +7 -1
@@ -15,6 +15,7 @@#include "qemu/osdep.h" #include "block/block.h" +#include "qemu/main-loop.h" #include "qemu/rcu.h" #include "qemu/rcu_queue.h" #include "qemu/sockets.h"
@@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)* There cannot be two concurrent aio_poll calls for the same AioContext (or * an aio_poll concurrent with a GSource prepare/check/dispatch callback). * We rely on this below to avoid slow locked accesses to ctx->notify_me. +* +* aio_poll() may only be called in the AioContext's thread. iohandler_ctx +* is special in that it runs in the main thread, but that thread's context +* is qemu_aio_context. */ -assert (in_aio_context_home_thread (ctx)); +assert (in_aio_context_home_thread (ctx ==iohandler_get_aio_context () ? +qemu_get_aio_context () : ctx));qemu_lockcnt_inc (&ctx->list_lock);
Signature · 1 line
2.25.4
PATCH v8 10/14 hmp: Add support for coroutine command handlers
In reply to Kevin WolfOften, QMP command handlers are not only called to handle QMP commands, but also from a corresponding HMP command handler. In order to give them a consistent environment, optionally run HMP command handlers in a coroutine, too.
The implementation is a lot simpler than in QMP because for HMP, we still block the VM while the coroutine is running.
--- docs/devel/qapi-code-gen.txt | 4 ++-- monitor/monitor-internal.h | 1 + monitor/hmp.c | 37 +++++++++++++++++++++++++++++++----- 3 files changed, 35 insertions(+), 7 deletions(-)
docs/devel/qapi-code-gen.txt +2 -2
@@ -617,8 +617,8 @@ pitfalls are: Since the command handler may assume coroutine context, any callers other than the QMP dispatcher must also call it in coroutine context. -In particular, HMP commands calling such a QMP command handler must -enter coroutine context before calling the handler. +In particular, HMP commands calling such a QMP command handler must be +marked .coroutine = true in hmp-commands.hx. It is an error to specify both 'coroutine': true and 'allow-oob': true for a command. We don't currently have a use case for both together and
monitor/monitor-internal.h +1 -0
@@ -74,6 +74,7 @@ typedef struct HMPCommand {const char *help;const char *flags;/* p=preconfig */ void (*cmd)(Monitor *mon,const QDict *qdict); +bool coroutine;/* * @sub_table is a list of 2nd level of commands. If it does not exist, * cmd should be used. If it exists, sub_table[?].cmd should be
monitor/hmp.c +32 -5
@@ -1056,12 +1056,26 @@ fail:return NULL ; } +typedef struct HandleHmpCommandCo { +Monitor *mon; +const HMPCommand *cmd; +QDict *qdict; +bool done; +}HandleHmpCommandCo ; + +static void handle_hmp_command_co (void *opaque) +{ +HandleHmpCommandCo *data = opaque; + data->cmd->cmd (data->mon, data->qdict); +monitor_set_cur (qemu_coroutine_self (),NULL ); + data->done = true; +} +void handle_hmp_command (MonitorHMP *mon,const char *cmdline) {QDict *qdict;const HMPCommand *cmd;const char *cmd_start = cmdline; -Monitor *old_mon;trace_handle_hmp_command (mon, cmdline);
@@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)return ; } -/* old_mon is non-NULL when called from qmp_human_monitor_command() */ - old_mon =monitor_set_cur (qemu_coroutine_self (), &mon->common); - cmd->cmd (&mon->common, qdict); -monitor_set_cur (qemu_coroutine_self (), old_mon); +if (!cmd->coroutine) { +/* old_mon is non-NULL when called from qmp_human_monitor_command() */ +Monitor *old_mon =monitor_set_cur (qemu_coroutine_self (), &mon->common); + cmd->cmd (&mon->common, qdict); +monitor_set_cur (qemu_coroutine_self (), old_mon); + }else { +HandleHmpCommandCo data = { + .mon = &mon->common, + .cmd = cmd, + .qdict = qdict, + .done = false, + }; +Coroutine *co =qemu_coroutine_create (handle_hmp_command_co, &data); +monitor_set_cur (co, &mon->common); +aio_co_enter (qemu_get_aio_context (), co); +AIO_WAIT_WHILE (qemu_get_aio_context (), !data.done); + }qobject_unref (qdict); }
Signature · 1 line
2.25.4
PATCH v8 11/14 util/async: Add aio_co_reschedule_self()
In reply to Kevin WolfAdd a function that can be used to move the currently running coroutine to a different AioContext (and therefore potentially a different thread).
--- include/block/aio.h | 10 ++++++++++ util/async.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+)
include/block/aio.h +10 -0
@@ -17,6 +17,7 @@#ifdef CONFIG_LINUX_IO_URING #include <liburing.h> #endif +#include "qemu/coroutine.h" #include "qemu/queue.h" #include "qemu/event_notifier.h" #include "qemu/thread.h"
@@ -654,6 +655,15 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)*/ void aio_co_schedule (AioContext *ctx,struct Coroutine *co); +/** +* aio_co_reschedule_self: +* @new_ctx: the new context +* +* Move the currently running coroutine to new_ctx. If the coroutine is already +* running in new_ctx, do nothing. +*/ +void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx); +/** * aio_co_wake: * @co: the coroutine
util/async.c +30 -0
@@ -569,6 +569,36 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)aio_context_unref (ctx); } +typedef struct AioCoRescheduleSelf { +Coroutine *co; +AioContext *new_ctx; +}AioCoRescheduleSelf ; + +static void aio_co_reschedule_self_bh (void *opaque) +{ +AioCoRescheduleSelf *data = opaque; +aio_co_schedule (data->new_ctx, data->co); +} + +void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx) +{ +AioContext *old_ctx =qemu_get_current_aio_context (); + +if (old_ctx != new_ctx) { +AioCoRescheduleSelf data = { + .co =qemu_coroutine_self (), + .new_ctx = new_ctx, + }; +/* +* We can't directly schedule the coroutine in the target context +* because this would be racy: The other thread could try to enter the +* coroutine before it has yielded in this one. +*/ +aio_bh_schedule_oneshot (old_ctx, aio_co_reschedule_self_bh, &data); +qemu_coroutine_yield (); + } +} +void aio_co_wake (struct Coroutine *co) {AioContext *ctx;
Signature · 1 line
2.25.4
PATCH v8 12/14 block: Add bdrv_co_enter()/leave()
In reply to Kevin WolfAdd a pair of functions to temporarily move the current coroutine to the AioContext of a given BlockDriverState.
--- include/block/block.h | 17 +++++++++++++++++ block.c | 23 +++++++++++++++++++++++ 2 files changed, 40 insertions(+)
include/block/block.h +17 -0
@@ -626,6 +626,23 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);*/ AioContext *bdrv_get_aio_context (BlockDriverState *bs); +/** +* Move the current coroutine to the AioContext of @bs and return the old +* AioContext of the coroutine. Increase bs->in_flight so that draining @bs +* will wait for the operation to proceed until the corresponding +* bdrv_co_leave(). +* +* Consequently, you can't call drain inside a bdrv_co_enter/leave() section as +* this will deadlock. +*/ +AioContext *coroutine_fn bdrv_co_enter(BlockDriverState *bs); + +/** +* Ends a section started by bdrv_co_enter(). Move the current coroutine back +* to old_ctx and decrease bs->in_flight again. +*/ +void coroutine_fn bdrv_co_leave(BlockDriverState *bs, AioContext *old_ctx); +/** * Transfer control to @co in the aio context of @bs */
block.c +23 -0
@@ -6372,6 +6372,29 @@ AioContext *bdrv_get_aio_context(BlockDriverState *bs)return bs ? bs->aio_context :qemu_get_aio_context (); } +AioContext *coroutine_fn bdrv_co_enter(BlockDriverState *bs) +{ +Coroutine *self =qemu_coroutine_self (); +AioContext *old_ctx =qemu_coroutine_get_aio_context (self); +AioContext *new_ctx; + +/* +* Increase bs->in_flight to ensure that this operation is completed before +* moving the node to a different AioContext. Read new_ctx only afterwards. +*/ +bdrv_inc_in_flight (bs); + + new_ctx =bdrv_get_aio_context (bs); +aio_co_reschedule_self (new_ctx); +return old_ctx; +} + +void coroutine_fn bdrv_co_leave(BlockDriverState *bs, AioContext *old_ctx) +{ +aio_co_reschedule_self (old_ctx); +bdrv_dec_in_flight (bs); +} +void bdrv_coroutine_enter (BlockDriverState *bs,Coroutine *co) {aio_co_enter (bdrv_get_aio_context (bs), co);
Signature · 1 line
2.25.4
PATCH v8 13/14 block: Add bdrv_lock()/unlock()
In reply to Kevin WolfInside of coroutine context, we can't directly use aio_context_acquire() for the AioContext of a block node because we already own the lock of the current AioContext and we need to avoid double locking to prevent deadlocks.
This provides helper functions to lock the AioContext of a node only if it's not the same as the current AioContext.
--- include/block/block.h | 14 ++++++++++++++ block.c | 27 +++++++++++++++++++++++++++ 2 files changed, 41 insertions(+)
include/block/block.h +14 -0
@@ -643,6 +643,20 @@ AioContext *coroutine_fn bdrv_co_enter(BlockDriverState *bs);*/ void coroutine_fn bdrv_co_leave(BlockDriverState *bs, AioContext *old_ctx); +/** +* Locks the AioContext of @bs if it's not the current AioContext. This avoids +* double locking which could lead to deadlocks: This is a coroutine_fn, so we +* know we already own the lock of the current AioContext. +* +* May only be called in the main thread. +*/ +void coroutine_fn bdrv_co_lock(BlockDriverState *bs); + +/** +* Unlocks the AioContext of @bs if it's not the current AioContext. +*/ +void coroutine_fn bdrv_co_unlock(BlockDriverState *bs); +/** * Transfer control to @co in the aio context of @bs */
block.c +27 -0
@@ -6395,6 +6395,33 @@ void coroutine_fn bdrv_co_leave(BlockDriverState *bs, AioContext *old_ctx)bdrv_dec_in_flight (bs); } +void coroutine_fn bdrv_co_lock(BlockDriverState *bs) +{ +AioContext *ctx =bdrv_get_aio_context (bs); + +/* In the main thread, bs->aio_context won't change concurrently */ +assert (qemu_get_current_aio_context () ==qemu_get_aio_context ()); + +/* +* We're in coroutine context, so we already hold the lock of the main +* loop AioContext. Don't lock it twice to avoid deadlocks. +*/ +assert (qemu_in_coroutine ()); +if (ctx !=qemu_get_aio_context ()) { +aio_context_acquire (ctx); + } +} + +void coroutine_fn bdrv_co_unlock(BlockDriverState *bs) +{ +AioContext *ctx =bdrv_get_aio_context (bs); + +assert (qemu_in_coroutine ()); +if (ctx !=qemu_get_aio_context ()) { +aio_context_release (ctx); + } +} +void bdrv_coroutine_enter (BlockDriverState *bs,Coroutine *co) {aio_co_enter (bdrv_get_aio_context (bs), co);
Signature · 1 line
2.25.4
PATCH v8 14/14 block: Convert 'block_resize' to coroutine
In reply to Kevin Wolfblock_resize performs some I/O that could potentially take quite some time, so use it as an example for the new 'coroutine': true annotation in the QAPI schema.
bdrv_truncate() requires that we're already in the right AioContext for the BlockDriverState if called in coroutine context. So instead of just taking the AioContext lock, move the QMP handler coroutine to the context.
Call blk_unref() only after switching back because blk_unref() may only be called in the main thread.
--- qapi/block-core.json | 3 ++- blockdev.c | 16 ++++++++-------- hmp-commands.hx | 1 + 3 files changed, 11 insertions(+), 9 deletions(-)
qapi/block-core.json +2 -1
@@ -1304,7 +1304,8 @@ { 'command': 'block_resize', 'data': { '*device': 'str', '*node-name': 'str', - 'size': 'int' } } + 'size': 'int' }, + 'coroutine': true } ## # @NewImageMode:
blockdev.c +8 -8
@@ -2439,14 +2439,14 @@ BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,return ret; } -void qmp_block_resize (bool has_device,const char *device, -bool has_node_name,const char *node_name, -int64_t size,Error **errp) +void coroutine_fn qmp_block_resize(bool has_device, const char *device, +bool has_node_name,const char *node_name, +int64_t size,Error **errp) {Error *local_err =NULL ;BlockBackend *blk =NULL ;BlockDriverState *bs; -AioContext *aio_context; +AioContext *old_ctx; bs = bdrv_lookup_bs(has_device ? device : NULL, has_node_name ? node_name : NULL,
@@ -2456,9 +2456,6 @@ void qmp_block_resize(bool has_device, const char *device,return ; } - aio_context =bdrv_get_aio_context (bs); -aio_context_acquire (aio_context); -if (size <0 ) {error_setg (errp,QERR_INVALID_PARAMETER_VALUE ,"size" ,"a >0 size" ); goto out;
@@ -2475,12 +2472,15 @@ void qmp_block_resize(bool has_device, const char *device, }bdrv_drained_begin (bs); + old_ctx =bdrv_co_enter (bs);blk_truncate (blk, size, false,PREALLOC_MODE_OFF ,0 , errp); +bdrv_co_leave (bs, old_ctx);bdrv_drained_end (bs); out: +bdrv_co_lock (bs);blk_unref (blk); -aio_context_release (aio_context); +bdrv_co_unlock (bs); } void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
hmp-commands.hx +1 -0
@@ -76,6 +76,7 @@ ERST .params = "device size", .help = "resize a block image", .cmd = hmp_block_resize, + .coroutine = true, }, SRST
Signature · 1 line
2.25.4
Quoted 11 lines · Kevin Wolf
Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.
This series adds the infrastructure to allow this and switches
block_resizeto run in a coroutine as a first example.This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.
PATCH 01-10:
PATCH 11-14 are for Stefan to review.
I can take this through my tree once that's done.
Marc-André, I figure rebasing your "[PATCH] console: make QMP screendump use coroutine" on top of this is now low risk. No rush, of course.
Quoted 131 lines · Kevin Wolf
Some QMP command handlers can block the main loop for a relatively long time, for example because they perform some I/O. This is quite nasty. Allowing such handlers to run in a coroutine where they can yield (and therefore release the BQL) while waiting for an event such as I/O completion solves the problem.
This series adds the infrastructure to allow this and switches
block_resizeto run in a coroutine as a first example.This is an alternative solution to Marc-André's "monitor: add asynchronous command type" series.
v8:
- Replacedbdrv_co_move_to_aio_context()with bdrv_co_enter()/leave(), added bdrv_co_lock()/unlock() and fixed the iothread support inqmp_block_resize()with the new functions [Stefan]
- More detailed documentation of 'coroutine' flag for commands [Markus]
- Improved comment for dropping out of coroutine context inqmp_dispatch()[Markus]
- Coding style changes [Markus]
- Resolve some more merge conflicts on rebasev7:
- Added patch 2 to add a Monitor parameter tomonitor_get_cpu_index(), too [Markus]
- Letmonitor_set_cur()return the old monitor [Markus]
- Fixed comment about linking stub objects in test-util-sockets [Markus]
- More detailed commit message for per-coroutine current monitor and document thatmonitor_set_cur(NULL)must be called eventually [Markus]
- Resolve some merge conflicts on rebasev6:
- Fixedcur_monbehaviour: It should always point to the Monitor while we're in the handler coroutine, but be NULL while the handler coroutines has yielded. [Markus]
- Give HMP handlers the coroutine option, too, because they will call QMP handlers, and life is easier when we know whether we are in coroutine context or not.
- Fixedblock_resizefor block devices in iothreads and for HMP
- Resolved some merge conflict with QAPI generator and monitor refactorings that were merged in the meantimev5:
- Improved comments and documentation [Markus]v4:
- Forbid 'oob': true, 'coroutine': true [Markus]
- Removed Python type hints [Markus]
- Introduced separate boolqmp_dispatcher_co_shutdownto make it clearer how a shutdown request is signalled to the dispatcher [Markus]
- Allow usingaio_poll()withiohandler_ctxand use that instead ofaio_bh_poll()[Markus]
- Removedcoroutine_fnfromqmp_block_resize()again because at least one caller (HMP) calls it outside of coroutine context [Markus]
- Tried to make the synchronisation between dispatcher and the monitor thread clearer, and fixed a race condition
- Improved documentation and commentsv3:
- Fix race between monitor thread and dispatcher that could schedule the dispatcher coroutine twice if a second requests comes in before the dispatcher can wake up [Patchew]v2:
- Fix typo in a commit message [Eric]
- Use hyphen instead of underscore for the test command [Eric]
- Markqmp_block_resize()ascoroutine_fn[Stefan]Kevin Wolf (14):
monitor: Add Monitor parameter tomonitor_set_cpu()
monitor: Add Monitor parameter tomonitor_get_cpu_index()
monitor: Use getter/setter functions forcur_mon
hmp: Update current monitor only inhandle_hmp_command()
qmp: Assert that no other monitor is active
qmp: Callmonitor_set_cur()only inqmp_dispatch()
monitor: Make current monitor a per-coroutine property
qapi: Add a 'coroutine' flag for commands
qmp: Move dispatcher to a coroutine
hmp: Add support for coroutine command handlers util/async: Addaio_co_reschedule_self()
block: Add bdrv_co_enter()/leave()
block: Add bdrv_lock()/unlock()
block: Convert 'block_resize' to coroutineqapi/block-core.json | 3 +- docs/devel/qapi-code-gen.txt | 29 ++++++ docs/sphinx/qapidoc.py | 2 +- include/block/aio.h | 10 ++ include/block/block.h | 31 ++++++ include/monitor/monitor.h | 7 +- include/qapi/qmp/dispatch.h | 5 +- monitor/monitor-internal.h | 7 +- audio/wavcapture.c | 8 +- block.c | 50 +++++++++ blockdev.c | 16 +-- dump/dump.c | 2 +- hw/core/machine-hmp-cmds.c | 2 +- hw/scsi/vhost-scsi.c | 2 +- hw/virtio/vhost-vsock.c | 2 +- migration/fd.c | 4 +- monitor/hmp-cmds.c | 4 +- monitor/hmp.c | 44 ++++++-- monitor/misc.c | 38 +++---- monitor/monitor.c | 101 ++++++++++++++++-- monitor/qmp-cmds-control.c | 2 + monitor/qmp-cmds.c | 2 +- monitor/qmp.c | 131 +++++++++++++++++------- net/socket.c | 2 +- net/tap.c | 6 +- qapi/qmp-dispatch.c | 65 +++++++++++- qapi/qmp-registry.c | 3 + qga/main.c | 2 +- softmmu/cpus.c | 2 +- stubs/monitor-core.c | 10 +- tests/test-qmp-cmds.c | 10 +- tests/test-util-sockets.c | 12 +-- trace/control.c | 2 +- util/aio-posix.c | 8 +- util/async.c | 30 ++++++ util/qemu-error.c | 6 +- util/qemu-print.c | 3 +- util/qemu-sockets.c | 1 + scripts/qapi/commands.py | 10 +- scripts/qapi/expr.py | 11 +- scripts/qapi/introspect.py | 2 +- scripts/qapi/schema.py | 13 ++- tests/qapi-schema/test-qapi.py | 7 +- hmp-commands.hx | 1 + tests/qapi-schema/meson.build | 1 + tests/qapi-schema/oob-coroutine.err | 2 +
Quoted 12 lines
tests/qapi-schema/oob-coroutine.json | 2 + tests/qapi-schema/oob-coroutine.out | 0 tests/qapi-schema/qapi-schema-test.json | 1 + tests/qapi-schema/qapi-schema-test.out | 2 + 50 files changed, 573 insertions(+), 143 deletions(-) create mode 100644 tests/qapi-schema/oob-coroutine.err create mode 100644 tests/qapi-schema/oob-coroutine.json create mode 100644 tests/qapi-schema/oob-coroutine.outSignature · 1 line
2.25.4