Skip to content

block: Convert qmp_query_block into a coroutine

Fabiano Rosas qemu-devel 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v3 Latest11 patches 0 repliesNew

Changes in v3 from the cover letter

- Do the changes more gradually to make it easier to reason about the safety of the change.

- Patch 4 addresses the issue I asked about recently on the ml [1] about how to avoid dispatching the QMP command during an aio_poll().

- Converted qmp_query_block and qmp_query_named_block_nodes in a single patch to avoid having hmp_info_block call a coroutine_fn out of coroutine context.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11block: Allow the wrapper script to see functions declared in qapi.hblock/meson.build scripts/block-coroutine-wrapper.py+2 -00 replies
2/11block: Temporarily mark bdrv_co_get_allocated_file_size as mixedinclude/block/block-io.h+1 -10 replies
3/11block: Take the graph lock in bdrv_snapshot_listblock/snapshot.c+1 -10 replies
4/11block: Reschedule query-block during qcow2 invalidationblock.c block/mirror.c block/qcow2.c +4 more+26 -00 replies
5/11block: Run bdrv_do_query_node_info in a coroutineblock/qapi.c include/block/qapi.h+14 -50 replies
6/11block: Convert bdrv_query_block_graph_info to coroutineblock/qapi.c include/block/qapi.h qemu-img.c+13 -100 replies
7/11block: Convert bdrv_query_image_info to coroutineblock/qapi.c include/block/qapi.h+11 -130 replies
8/11block: Convert bdrv_block_device_info into co_wrapperblock/qapi.c include/block/qapi.h+11 -120 replies
9/11block: Don't query all block devices at hmp_nbd_server_startblock/monitor/block-hmp-cmds.c+24 -80 replies
10/11block: Convert qmp_query_block and qmp_query_named_block_nodes to coroutineblock.c block/monitor/block-hmp-cmds.c block/qapi.c +7 more+23 -230 replies
11/11block: Add a thread-pool version of fstatblock/file-posix.c include/block/raw-aio.h+40 -40 replies

Reviews trailers per patch

4 of 11 patches have a Reviewed-by or Acked-by.

PatchHanna CzenczekEric BlakeClaudio Fontana
1/11R-b
2/11R-b R-b
3/11
4/11
5/11
6/11R-b
7/11
8/11
9/11
10/11
11/11R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v3 thread

Cover letter [PATCH v3 00/11] block: Convert qmp_query_block into a coroutine

Hi, it's been a while since the last version, so a recap:

This series converts qmp_query_block() & qmp_query_named_block_nodes() to coroutines so we can yield from them all the way back into the main loop. This addresses a vcpu softlockup encountered when querying a disk placed on NFS.

If the NFS server happens to have high latency, an fstat() issued from raw_co_get_allocated_file_size() could take seconds while the whole QMP command is holding the BQL and blocks a vcpu thread going out of the guest to handle IO.

This scenario is clearly undesireable since a query command is of much lower priority than the vcpu thread doing actual work.

Move the 'fstat' call into the thread-pool and make the necessary adaptations to ensure the whole QMP command that calls raw_co_get_allocated_file_size() runs in a coroutine.

Changes since v2:

- Do the changes more gradually to make it easier to reason about the safety of the change.

- Patch 4 addresses the issue I asked about recently on the ml [1] about how to avoid dispatching the QMP command during an aio_poll().

- Converted qmp_query_block and qmp_query_named_block_nodes in a single patch to avoid having hmp_info_block call a coroutine_fn out of coroutine context.

On v2, Hanna asked:

"I wonder how the threading is actually supposed to work.  I assume QMP coroutines run in the main thread, so now we run bdrv_co_get_allocated_file_size() in the main thread – is that correct, or do we need to use bdrv_co_enter() like qmp_block_resize() does?  And so, if we run it in the main thread, is it OK not to acquire the AioContext around it to prevent interference from a potential I/O thread?"

The QMP coroutines and also bdrv_co_get_allocated_file_size() run in the main thread. This series doesn't change that. The difference is that instead of bdrv_co_get_allocated_file_size() yielding back to bdrv_poll(), it now yields back to the main loop.

As for thread safety, that's basically what I asked about in [1], so I'm still gathering information and don't have a definite answer for it. Since we don't have the AioContext lock anymore, it seems that safety is now dependant on not dispatching the QMP command while other operations are ongoing.

Still, for this particular case of fstat(), I don't think interference of an I/O thread could cause any problems, as long as the file descriptor is not closed prematurely. The fstat() manual already mentions that it is succeptible to return old information in some cases.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/1244905208

1- Advice on block QMP command coroutines
https://lore.kernel.org/r/[email protected]

Initial discussion:
https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg03141.html
v1:
https://lore.kernel.org/r/[email protected]
v2:
https://lore.kernel.org/r/[email protected]

Fabiano Rosas (9):
block: Allow the wrapper script to see functions declared in qapi.h
block: Temporarily mark bdrv_co_get_allocated_file_size as mixed
block: Take the graph lock in bdrv_snapshot_list
block: Reschedule query-block during qcow2 invalidation
block: Run bdrv_do_query_node_info in a coroutine
block: Convert bdrv_query_block_graph_info to coroutine
block: Convert bdrv_query_image_info to coroutine
block: Convert bdrv_block_device_info into co_wrapper
block: Don't query all block devices at hmp_nbd_server_start

João Silva (1):
block: Add a thread-pool version of fstat

Lin Ma (1):
block: Convert qmp_query_block and qmp_query_named_block_nodes to
coroutine

 block.c                            |  9 +++--
 block/file-posix.c                 | 40 +++++++++++++++++--
 block/meson.build                  |  1 +
 block/mirror.c                     |  1 +
 block/monitor/block-hmp-cmds.c     | 34 +++++++++++-----
 block/qapi.c                       | 63 +++++++++++++++---------------
 block/qcow2.c                      | 20 ++++++++++
 block/replication.c                |  1 +
 block/snapshot.c                   |  2 +-
 blockdev.c                         |  8 ++--
 blockjob.c                         |  1 +
 hmp-commands-info.hx               |  1 +
 include/block/block-common.h       |  1 +
 include/block/block-global-state.h |  3 +-
 include/block/block-hmp-cmds.h     |  2 +-
 include/block/qapi.h               | 24 ++++++++----
 include/block/raw-aio.h            |  4 +-
 migration/block.c                  |  1 +
 qapi/block-core.json               |  5 ++-
 qemu-img.c                         |  3 --
 scripts/block-coroutine-wrapper.py |  1 +
 21 files changed, 157 insertions(+), 68 deletions(-)
Signature · 1 line
2.35.3