monitor: only run coroutine commands in qemu_aio_context
cover monitor: only run coroutine commands in qemu_aio_context
Several bugs have been reported related to how QMP commands are rescheduled in qemu_aio_context:
- https://gitlab.com/qemu-project/qemu/-/issues/1933
- https://issues.redhat.com/browse/RHEL-17369
- https://bugzilla.redhat.com/show_bug.cgi?id=2215192
- https://bugzilla.redhat.com/show_bug.cgi?id=2214985
The first instance of the bug interacted with drain_call_rcu() temporarily dropping the BQL and resulted in vCPU threads entering device emulation code simultaneously (something that should never happen). I set out to make drain_call_rcu() safe to use in this environment, but Paolo and Kevin discussed the possibility of avoiding rescheduling the monitor_qmp_dispatcher_co() coroutine for non-coroutine commands. This would prevent monitor commands from running during vCPU thread aio_poll() entirely and addresses the root cause.
This patch series implements this idea. qemu-iotests is sensitive to the exact order in which QMP events and responses are emitted. Running QMP handlers in the iohandler AioContext causes some QMP events to be ordered differently than before. It is therefore necessary to adjust the reference output in many test cases. The actual QMP code change is small and everything else is just to make qemu-iotests happy.
If you have bugs related to the same issue, please retest them with these patches. Thanks!
Stefan Hajnoczi (3):
iotests: add filter_qmp_generated_node_ids()
iotests: port 141 to Python for reliable QMP testing
monitor: only run coroutine commands in qemu_aio_context
monitor/qmp.c | 17 - qapi/qmp-dispatch.c | 24 +- tests/qemu-iotests/060.out | 4 +- tests/qemu-iotests/071.out | 4 +- tests/qemu-iotests/081.out | 16 +- tests/qemu-iotests/087.out | 12 +- tests/qemu-iotests/108.out | 2 +- tests/qemu-iotests/109 | 4 +- tests/qemu-iotests/109.out | 78 ++--- tests/qemu-iotests/117.out | 2 +- tests/qemu-iotests/120.out | 2 +- tests/qemu-iotests/127.out | 2 +- tests/qemu-iotests/140.out | 2 +- tests/qemu-iotests/141 | 297 +++++++----------- tests/qemu-iotests/141.out | 190 +++-------- tests/qemu-iotests/143.out | 2 +- tests/qemu-iotests/156.out | 2 +- tests/qemu-iotests/176.out | 16 +- tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/183.out | 4 +- tests/qemu-iotests/184.out | 32 +- tests/qemu-iotests/185 | 6 +- tests/qemu-iotests/185.out | 45 ++- tests/qemu-iotests/191.out | 16 +- tests/qemu-iotests/195.out | 16 +- tests/qemu-iotests/223.out | 16 +- tests/qemu-iotests/227.out | 32 +- tests/qemu-iotests/247.out | 2 +- tests/qemu-iotests/273.out | 8 +- tests/qemu-iotests/308 | 4 +- tests/qemu-iotests/308.out | 4 +- tests/qemu-iotests/iotests.py | 7 + tests/qemu-iotests/tests/file-io-error | 5 +- tests/qemu-iotests/tests/iothreads-resize.out | 2 +- tests/qemu-iotests/tests/qsd-jobs.out | 4 +- 35 files changed, 375 insertions(+), 506 deletions(-)
Signature · 1 line
2.43.0
PATCH 1/3 iotests: add filter_qmp_generated_node_ids()
Add a filter function for QMP responses that contain QEMU's automatically generated node ids. The ids change between runs and must be masked in the reference output.
The next commit will use this new function.
--- tests/qemu-iotests/iotests.py | 7 +++++++ 1 file changed, 7 insertions(+)
tests/qemu-iotests/iotests.py +7 -0
@@ -651,6 +651,13 @@ def _filter(_key, value): def filter_generated_node_ids(msg): return re.sub("#block[0-9]+", "NODE_NAME", msg) +def filter_qmp_generated_node_ids(qmsg): + def _filter(_key, value): + if is_str(value): + return filter_generated_node_ids(value) + return value + return filter_qmp(qmsg, _filter) + def filter_img_info(output: str, filename: str, drop_child_info: bool = True) -> str: lines = []
Signature · 1 line
2.43.0
Re: [PATCH 1/3] iotests: add filter_qmp_generated_node_ids()
Am 16.01.2024 um 20:00 hat Stefan Hajnoczi geschrieben:
Quoted 7 lines
Add a filter function for QMP responses that contain QEMU's automatically generated node ids. The ids change between runs and must be masked in the reference output.
The next commit will use this new function.
Signed-off-by Stefan Hajnoczi <[email protected]>
PATCH 2/3 iotests: port 141 to Python for reliable QMP testing
In reply to Stefan HajnocziThe common.qemu bash functions allow tests to interact with the QMP monitor of a QEMU process. I spent two days trying to update 141 when the order of the test output changed, but found it would still fail occassionally because printf() and QMP events race with synchronous QMP communication.
I gave up and ported 141 to the existing Python API for QMP tests. The Python API is less affected by the order in which QEMU prints output because it does not print all QMP traffic by default.
The next commit changes the order in which QMP messages are received. Make 141 reliable first.
--- tests/qemu-iotests/141 | 297 +++++++++++++++---------------------- tests/qemu-iotests/141.out | 190 +++++------------------- 2 files changed, 161 insertions(+), 326 deletions(-)
tests/qemu-iotests/141 +121 -176
@@ -1,9 +1,12 @@ -#!/usr/bin/env bash +#!/usr/bin/env python3 # group: rw auto quick # # Test case for ejecting BDSs with block jobs still running on them # -# Copyright (C) 2016 Red Hat, Inc. +# Originally written in bash by Hanna Czenczek, ported to Python by Stefan +# Hajnoczi. +# +# Copyright Red Hat # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by
@@ -19,177 +22,119 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # -# creator [email protected] - -seq="$(basename $0)" -echo "QA output created by $seq" - -status=1 # failure is the default! - -_cleanup() -{ - _cleanup_qemu - _cleanup_test_img - for img in "$TEST_DIR"/{b,m,o}.$IMGFMT; do - _rm_test_img "$img" - done -} -trap "_cleanup; exit \$status" 0 1 2 3 15 - -# get standard environment, filters and checks -. ./common.rc -. ./common.filter -. ./common.qemu - -# Needs backing file and backing format support -_supported_fmt qcow2 qed -_supported_proto file -_supported_os Linux - - -test_blockjob() -{ - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': '$IMGFMT', - 'file': { - 'driver': 'file', - 'filename': '$TEST_IMG' - }}}" \ - 'return' - - # If "$2" is an event, we may or may not see it before the - # {"return": {}}. Therefore, filter the {"return": {}} out both - # here and in the next command. (Naturally, if we do not see it - # here, we will see it before the next command can be executed, - # so it will appear in the next _send_qemu_cmd's output.) - _send_qemu_cmd $QEMU_HANDLE \ - "$1" \ - "$2" \ - | _filter_img_create | _filter_qmp_empty_return - - # We want this to return an error because the block job is still running - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}}" \ - 'error' | _filter_generated_node_ids | _filter_qmp_empty_return - - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'block-job-cancel', - 'arguments': {'device': 'job0'}}" \ - "$3" - - _send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'blockdev-del', - 'arguments': {'node-name': 'drv0'}}" \ - 'return' -} - - -TEST_IMG="$TEST_DIR/b.$IMGFMT" _make_test_img 1M -TEST_IMG="$TEST_DIR/m.$IMGFMT" _make_test_img -b "$TEST_DIR/b.$IMGFMT" -F $IMGFMT 1M -_make_test_img -b "$TEST_DIR/m.$IMGFMT" 1M -F $IMGFMT - -_launch_qemu -nodefaults - -_send_qemu_cmd $QEMU_HANDLE \ - "{'execute': 'qmp_capabilities'}" \ - 'return' - -echo -echo '=== Testing drive-backup ===' -echo - -# drive-backup will not send BLOCK_JOB_READY by itself, and cancelling the job -# will consequently result in BLOCK_JOB_CANCELLED being emitted. - -test_blockjob \ - "{'execute': 'drive-backup', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'target': '$TEST_DIR/o.$IMGFMT', - 'format': '$IMGFMT', - 'sync': 'none'}}" \ - 'return' \ - '"status": "null"' - -echo -echo '=== Testing drive-mirror ===' -echo - -# drive-mirror will send BLOCK_JOB_READY basically immediately, and cancelling -# the job will consequently result in BLOCK_JOB_COMPLETED being emitted. - -test_blockjob \ - "{'execute': 'drive-mirror', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'target': '$TEST_DIR/o.$IMGFMT', - 'format': '$IMGFMT', - 'sync': 'none'}}" \ - 'BLOCK_JOB_READY' \ - '"status": "null"' - -echo -echo '=== Testing active block-commit ===' -echo - -# An active block-commit will send BLOCK_JOB_READY basically immediately, and -# cancelling the job will consequently result in BLOCK_JOB_COMPLETED being -# emitted. - -test_blockjob \ - "{'execute': 'block-commit', - 'arguments': {'job-id': 'job0', 'device': 'drv0'}}" \ - 'BLOCK_JOB_READY' \ - '"status": "null"' - -echo -echo '=== Testing non-active block-commit ===' -echo - -# Give block-commit something to work on, otherwise it would be done -# immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would work just -# fine without the block job still running. - -$QEMU_IO -c 'write 0 1M' "$TEST_DIR/m.$IMGFMT" | _filter_qemu_io - -test_blockjob \ - "{'execute': 'block-commit', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'top': '$TEST_DIR/m.$IMGFMT', - 'speed': 1}}" \ - 'return' \ - '"status": "null"' - -echo -echo '=== Testing block-stream ===' -echo - -# Give block-stream something to work on, otherwise it would be done -# immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would work just -# fine without the block job still running. - -$QEMU_IO -c 'write 0 1M' "$TEST_DIR/b.$IMGFMT" | _filter_qemu_io - -# With some data to stream (and @speed set to 1), block-stream will not complete -# until we send the block-job-cancel command. - -test_blockjob \ - "{'execute': 'block-stream', - 'arguments': {'job-id': 'job0', - 'device': 'drv0', - 'speed': 1}}" \ - 'return' \ - '"status": "null"' - -_cleanup_qemu - -# success, all done -echo "*** done" -rm -f $seq.full -status=0 +import iotests + + +class TestCase: + def __init__(self, name, vm, image_path, cancel_event): + self.name = name + self.vm = vm + self.image_path = image_path + self.cancel_event = cancel_event + + def __enter__(self): + iotests.log(f'=== Testing {self.name} ===') + self.vm.qmp_log('blockdev-add', \ + node_name='drv0', \ + driver=iotests.imgfmt, \ + file={'driver': 'file', 'filename': self.image_path}, \ + filters=[iotests.filter_qmp_testfiles]) + + def __exit__(self, *exc_details): + # This is expected to fail because the job still exists + self.vm.qmp_log('blockdev-del', node_name='drv0', \ + filters=[iotests.filter_qmp_generated_node_ids]) + + self.vm.qmp_log('block-job-cancel', device='job0') + event = self.vm.event_wait(self.cancel_event) + iotests.log(event, filters=[iotests.filter_qmp_event]) + + # This time it succeeds + self.vm.qmp_log('blockdev-del', node_name='drv0') + + # Separate test cases in output + iotests.log('') + + +def main() -> None: + with iotests.FilePath('bottom', 'middle', 'top', 'target') as \ + (bottom_path, middle_path, top_path, target_path), \ + iotests.VM() as vm: + + iotests.log('Creating bottom <- middle <- top backing file chain...') + IMAGE_SIZE='1M' + iotests.qemu_img_create('-f', iotests.imgfmt, bottom_path, IMAGE_SIZE) + iotests.qemu_img_create('-f', iotests.imgfmt, \ + '-F', iotests.imgfmt, \ + '-b', bottom_path, \ + middle_path, \ + IMAGE_SIZE) + iotests.qemu_img_create('-f', iotests.imgfmt, \ + '-F', iotests.imgfmt, \ + '-b', middle_path, \ + top_path, \ + IMAGE_SIZE) + + iotests.log('Starting VM...') + vm.add_args('-nodefaults') + vm.launch() + + # drive-backup will not send BLOCK_JOB_READY by itself, and cancelling the job + # will consequently result in BLOCK_JOB_CANCELLED being emitted. + with TestCase('drive-backup', vm, top_path, 'BLOCK_JOB_CANCELLED'): + vm.qmp_log('drive-backup', \ + job_id='job0', \ + device='drv0', \ + target=target_path, \ + format=iotests.imgfmt, \ + sync='none', \ + filters=[iotests.filter_qmp_testfiles]) + + # drive-mirror will send BLOCK_JOB_READY basically immediately, and cancelling + # the job will consequently result in BLOCK_JOB_COMPLETED being emitted. + with TestCase('drive-mirror', vm, top_path, 'BLOCK_JOB_COMPLETED'): + vm.qmp_log('drive-mirror', \ + job_id='job0', \ + device='drv0', \ + target=target_path, \ + format=iotests.imgfmt, \ + sync='none', \ + filters=[iotests.filter_qmp_testfiles]) + event = vm.event_wait('BLOCK_JOB_READY') + iotests.log(event, filters=[iotests.filter_qmp_event]) + + # An active block-commit will send BLOCK_JOB_READY basically immediately, and + # cancelling the job will consequently result in BLOCK_JOB_COMPLETED being + # emitted. + with TestCase('active block-commit', vm, top_path, 'BLOCK_JOB_COMPLETED'): + vm.qmp_log('block-commit', \ + job_id='job0', \ + device='drv0') + event = vm.event_wait('BLOCK_JOB_READY') + iotests.log(event, filters=[iotests.filter_qmp_event]) + + # Give block-commit something to work on, otherwise it would be done + # immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would work just + # fine without the block job still running. + iotests.qemu_io(middle_path, '-c', f'write 0 {IMAGE_SIZE}') + with TestCase('non-active block-commit', vm, top_path, 'BLOCK_JOB_CANCELLED'): + vm.qmp_log('block-commit', \ + job_id='job0', \ + device='drv0', \ + top=middle_path, \ + speed=1, \ + filters=[iotests.filter_qmp_testfiles]) + + # Give block-stream something to work on, otherwise it would be done + # immediately, send a BLOCK_JOB_COMPLETED and ejecting the BDS would work just + # fine without the block job still running. + iotests.qemu_io(bottom_path, '-c', f'write 0 {IMAGE_SIZE}') + with TestCase('block-stream', vm, top_path, 'BLOCK_JOB_CANCELLED'): + vm.qmp_log('block-stream', \ + job_id='job0', \ + device='drv0', \ + speed=1) + +if __name__ == '__main__': + iotests.script_main(main, supported_fmts=['qcow2', 'qed'], + supported_protocols=['file'])
tests/qemu-iotests/141.out +40 -150
219 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH 2/3] iotests: port 141 to Python for reliable QMP testing
Am 16.01.2024 um 20:00 hat Stefan Hajnoczi geschrieben:
Quoted 31 lines
The common.qemu bash functions allow tests to interact with the QMP monitor of a QEMU process. I spent two days trying to update 141 when the order of the test output changed, but found it would still fail occassionally because
printf()and QMP events race with synchronous QMP communication.I gave up and ported 141 to the existing Python API for QMP tests. The Python API is less affected by the order in which QEMU prints output because it does not print all QMP traffic by default.
The next commit changes the order in which QMP messages are received. Make 141 reliable first.
Cc Hanna Czenczek <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>tests/qemu-iotests/141.out +2
-7@@ -1,179 +1,69 @@ -QA output created by 141 -Formatting 'TEST_DIR/b.IMGFMT', fmt=IMGFMT size=1048576 -Formatting 'TEST_DIR/m.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/b.IMGFMT backing_fmt=IMGFMT -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/m.IMGFMT backing_fmt=IMGFMT -{'execute': 'qmp_capabilities'} -{"return": {}} - +Creating bottom <- middle <- top backing file chain... +Starting VM... === Testing drive-backup ===
tests/qemu-iotests/141.out line 4
Quoted 10 lines
tests/qemu-iotests/141.out +1
-9- -{'execute': 'blockdev-add', - 'arguments': { - 'node-name': 'drv0', - 'driver': 'IMGFMT', - 'file': { - 'driver': 'file', - 'filename': 'TEST_DIR/t.IMGFMT' - }}} +{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-top"}, "node-name": "drv0"}}
This and other lines in the test output need image format filtering. You have a literal "qcow2" in there, but the test case also runs (and fails because of this) for qed.
Apart from this, the patch looks good to me.
Kevin
Re: [PATCH 2/3] iotests: port 141 to Python for reliable QMP testing
In reply to Stefan HajnocziAm 16.01.2024 um 20:00 hat Stefan Hajnoczi geschrieben:
Quoted 10 lines
The common.qemu bash functions allow tests to interact with the QMP monitor of a QEMU process. I spent two days trying to update 141 when the order of the test output changed, but found it would still fail occassionally because
printf()and QMP events race with synchronous QMP communication.I gave up and ported 141 to the existing Python API for QMP tests. The Python API is less affected by the order in which QEMU prints output because it does not print all QMP traffic by default.
Quoted 5 lines
The next commit changes the order in which QMP messages are received. Make 141 reliable first.
Cc Hanna Czenczek <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>
iotests 297 finds problems with the new Python code:
=== pylint ===
+************* Module 141
+141:82:0: C0301: Line too long (86/79) (line-too-long) +141:93:0: C0301: Line too long (86/79) (line-too-long) +141:94:0: C0301: Line too long (80/79) (line-too-long) +141:106:0: C0301: Line too long (85/79) (line-too-long) +141:107:0: C0301: Line too long (82/79) (line-too-long) +141:109:0: C0301: Line too long (82/79) (line-too-long) +141:117:0: C0301: Line too long (86/79) (line-too-long) +141:120:0: C0301: Line too long (86/79) (line-too-long) +141:129:0: C0301: Line too long (86/79) (line-too-long)
=== mypy ===
+141:104: error: Value of type variable "Msg" of "log" cannot be "dict[str, Any] | None" [type-var] +141:114: error: Value of type variable "Msg" of "log" cannot be "dict[str, Any] | None" [type-var]
+Found 2 errors in 1 file (checked 37 source files)
Kevin
Re: [PATCH 2/3] iotests: port 141 to Python for reliable QMP testing
Quoted 24 lines · Kevin Wolf
Am 16.01.2024 um 20:00 hat Stefan Hajnoczi geschrieben:
Quoted 10 lines
The common.qemu bash functions allow tests to interact with the QMP monitor of a QEMU process. I spent two days trying to update 141 when the order of the test output changed, but found it would still fail occassionally because
printf()and QMP events race with synchronous QMP communication.I gave up and ported 141 to the existing Python API for QMP tests. The Python API is less affected by the order in which QEMU prints output because it does not print all QMP traffic by default.
Quoted 5 lines
The next commit changes the order in which QMP messages are received. Make 141 reliable first.
Cc Hanna Czenczek <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>iotests 297 finds problems with the new Python code:
=== pylint ===
(quoted diff) +4
-0+************* Module 141 +141:82:0: C0301: Line too long (86/79) (line-too-long) +141:93:0: C0301: Line too long (86/79) (line-too-long) +141:94:0: C0301: Line too long (80/79) (line-too-long)
Review
Quoted 10 lines
(quoted diff) +9
-0+141:106:0: C0301: Line too long (85/79) (line-too-long) +141:107:0: C0301: Line too long (82/79) (line-too-long) +141:109:0: C0301: Line too long (82/79) (line-too-long) +141:117:0: C0301: Line too long (86/79) (line-too-long) +141:120:0: C0301: Line too long (86/79) (line-too-long) +141:129:0: C0301: Line too long (86/79) (line-too-long) === mypy === +141:104: error: Value of type variable "Msg" of "log" cannot be "dict[str, Any] | None" [type-var] +141:114: error: Value of type variable "Msg" of "log" cannot be "dict[str, Any] | None" [type-var] +Found 2 errors in 1 file (checked 37 source files)
I'm not sure how to run this but I manually ran pylint and mypy and fixed those errors. Can you share how you ran this?
The mypy errors are because the type system cannot express that event_wait() only returns None when timeout=0. That's not the case in 141 but mypy complains anyway, so I added an assertion that event is not None to silence mypy.
Thanks,
Stefan
Re: [PATCH 2/3] iotests: port 141 to Python for reliable QMP testing
Am 18.01.2024 um 15:55 hat Stefan Hajnoczi geschrieben:
Quoted 30 lines · Kevin Wolf
Quoted 29 lines
Am 16.01.2024 um 20:00 hat Stefan Hajnoczi geschrieben:
Quoted 10 lines
The common.qemu bash functions allow tests to interact with the QMP monitor of a QEMU process. I spent two days trying to update 141 when the order of the test output changed, but found it would still fail occassionally because
printf()and QMP events race with synchronous QMP communication.I gave up and ported 141 to the existing Python API for QMP tests. The Python API is less affected by the order in which QEMU prints output because it does not print all QMP traffic by default.
Quoted 5 lines
The next commit changes the order in which QMP messages are received. Make 141 reliable first.
Cc Hanna Czenczek <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>iotests 297 finds problems with the new Python code:
=== pylint ===
+************* Module 141
+141:82:0: C0301: Line too long (86/79) (line-too-long) +141:93:0: C0301: Line too long (86/79) (line-too-long) +141:94:0: C0301: Line too long (80/79) (line-too-long) +141:106:0: C0301: Line too long (85/79) (line-too-long) +141:107:0: C0301: Line too long (82/79) (line-too-long) +141:109:0: C0301: Line too long (82/79) (line-too-long) +141:117:0: C0301: Line too long (86/79) (line-too-long) +141:120:0: C0301: Line too long (86/79) (line-too-long)
Quoted 8 lines
Quoted 5 lines
+141:129:0: C0301: Line too long (86/79) (line-too-long)
=== mypy ===+141:104: error: Value of type variable "Msg" of "log" cannot be "dict[str, Any] | None" [type-var] +141:114: error: Value of type variable "Msg" of "log" cannot be "dict[str, Any] | None" [type-var]+Found 2 errors in 1 file (checked 37 source files)
I'm not sure how to run this but I manually ran pylint and mypy and fixed those errors. Can you share how you ran this?
I already answered this in IRC, but for the record: This is from running qemu-iotests 297, which is kind of a meta-test that runs the linters on Python files in qemu-iotests.
Quoted 4 lines
The mypy errors are because the type system cannot express that
event_wait()only returns None when timeout=0. That's not the case in 141 but mypy complains anyway, so I added an assertion that event is not None to silence mypy.
Sounds like the right solution to me.
Kevin
PATCH 3/3 monitor: only run coroutine commands in qemu_aio_context
In reply to Stefan Hajnoczimonitor_qmp_dispatcher_co() runs in the iohandler AioContext that is not polled during nested event loops. The coroutine currently reschedules itself in the main loop's qemu_aio_context AioContext, which is polled during nested event loops. One known problem is that QMP device-add calls drain_call_rcu(), which temporarily drops the BQL, leading to all sorts of havoc like other vCPU threads re-entering device emulation code while another vCPU thread is waiting in device emulation code with aio_poll().
Paolo Bonzini suggested running non-coroutine QMP handlers in the iohandler AioContext. This avoids trouble with nested event loops. His original idea was to move coroutine rescheduling to monitor_qmp_dispatch(), but I resorted to moving it to qmp_dispatch() because we don't know if the QMP handler needs to run in coroutine context in monitor_qmp_dispatch(). monitor_qmp_dispatch() would have been nicer since it's associated with the monitor implementation and not as general as qmp_dispatch(), which is also used by qemu-ga.
A number of qemu-iotests need updated .out files because the order of QMP events vs QMP responses has changed.
Solves Issue #1933.
--- monitor/qmp.c | 17 ---- qapi/qmp-dispatch.c | 24 +++++- tests/qemu-iotests/060.out | 4 +- tests/qemu-iotests/071.out | 4 +- tests/qemu-iotests/081.out | 16 ++-- tests/qemu-iotests/087.out | 12 +-- tests/qemu-iotests/108.out | 2 +- tests/qemu-iotests/109 | 4 +- tests/qemu-iotests/109.out | 78 ++++++++----------- tests/qemu-iotests/117.out | 2 +- tests/qemu-iotests/120.out | 2 +- tests/qemu-iotests/127.out | 2 +- tests/qemu-iotests/140.out | 2 +- tests/qemu-iotests/143.out | 2 +- tests/qemu-iotests/156.out | 2 +- tests/qemu-iotests/176.out | 16 ++-- tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/183.out | 4 +- tests/qemu-iotests/184.out | 32 ++++---- tests/qemu-iotests/185 | 6 +- tests/qemu-iotests/185.out | 45 +++++++++-- tests/qemu-iotests/191.out | 16 ++-- tests/qemu-iotests/195.out | 16 ++-- tests/qemu-iotests/223.out | 16 ++-- tests/qemu-iotests/227.out | 32 ++++---- tests/qemu-iotests/247.out | 2 +- tests/qemu-iotests/273.out | 8 +- tests/qemu-iotests/308 | 4 +- tests/qemu-iotests/308.out | 4 +- tests/qemu-iotests/tests/file-io-error | 5 +- tests/qemu-iotests/tests/iothreads-resize.out | 2 +- tests/qemu-iotests/tests/qsd-jobs.out | 4 +- 32 files changed, 207 insertions(+), 180 deletions(-)
monitor/qmp.c +0 -17
@@ -321,14 +321,6 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data)qemu_coroutine_yield (); } -/* -* Move the coroutine from iohandler_ctx to qemu_aio_context for -* executing the command handler so that it can make progress if it -* involves an AIO_WAIT_WHILE(). -*/ -aio_co_schedule (qemu_get_aio_context (), qmp_dispatcher_co); -qemu_coroutine_yield (); -/* Process request */ if (req_obj->req) {if (trace_event_get_state (TRACE_MONITOR_QMP_CMD_IN_BAND )) {
@@ -355,15 +347,6 @@ void coroutine_fn monitor_qmp_dispatcher_co(void *data) }qmp_request_free (req_obj); - -/* -* Yield and reschedule so the main loop stays responsive. -* -* Move back to iohandler_ctx so that nested event loops for -* qemu_aio_context don't start new monitor commands. -*/ -aio_co_schedule (iohandler_get_aio_context (), qmp_dispatcher_co); -qemu_coroutine_yield (); }qatomic_set (&qmp_dispatcher_co,NULL ); }
qapi/qmp-dispatch.c +23 -1
@@ -206,9 +206,31 @@ QDict *coroutine_mixed_fn qmp_dispatch(const QmpCommandList *cmds, QObject *requassert (!(oob &&qemu_in_coroutine ()));assert (monitor_cur () ==NULL );if (!!(cmd->options &QCO_COROUTINE ) ==qemu_in_coroutine ()) { +if (qemu_in_coroutine ()) { +/* +* Move the coroutine from iohandler_ctx to qemu_aio_context for +* executing the command handler so that it can make progress if it +* involves an AIO_WAIT_WHILE(). +*/ +aio_co_schedule (qemu_get_aio_context (),qemu_coroutine_self ()); +qemu_coroutine_yield (); + } +monitor_set_cur (qemu_coroutine_self (), cur_mon); cmd->fn (args, &ret, &err);monitor_set_cur (qemu_coroutine_self (),NULL ); + +if (qemu_in_coroutine ()) { +/* +* Yield and reschedule so the main loop stays responsive. +* +* Move back to iohandler_ctx so that nested event loops for +* qemu_aio_context don't start new monitor commands. +*/ +aio_co_schedule (iohandler_get_aio_context (), +qemu_coroutine_self ()); +qemu_coroutine_yield (); + } }else {/* * Actual context doesn't match the one the command needs.
@@ -232,7 +254,7 @@ QDict *coroutine_mixed_fn qmp_dispatch(const QmpCommandList *cmds, QObject *requ .errp = &err, .co = qemu_coroutine_self(), }; -aio_bh_schedule_oneshot (qemu_get_aio_context (), do_qmp_dispatch_bh, +aio_bh_schedule_oneshot (iohandler_get_aio_context (), do_qmp_dispatch_bh, &data);qemu_coroutine_yield (); }
tests/qemu-iotests/060.out +2 -2
@@ -421,8 +421,8 @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "none0", "msg": "Preventing invalid write on metadata (overlaps with refcount table)", "offset": 65536, "node-name": "drive", "fatal": true, "size": 65536}} write failed: Input/output error {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Testing incoming inactive corrupted image ===
@@ -432,8 +432,8 @@ QMP_VERSION qcow2: Image is corrupt: L2 table offset 0x2a2a2a00 unaligned (L1 index: 0); further non-fatal corruption events will be suppressed {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "", "msg": "L2 table offset 0x2a2a2a00 unaligned (L1 index: 0)", "node-name": "drive", "fatal": false}} {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} corrupt: false *** done
tests/qemu-iotests/071.out +2 -2
@@ -45,8 +45,8 @@ QMP_VERSION {"return": {}} read failed: Input/output error {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Testing blkverify on existing block device ===
@@ -84,9 +84,9 @@ wrote 512/512 bytes at offset 0 {"return": ""} read failed: Input/output error {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} QEMU_PROG: Failed to flush the L2 table cache: Input/output error QEMU_PROG: Failed to flush the refcount block cache: Input/output error +{"return": {}} *** done
tests/qemu-iotests/081.out +8 -8
@@ -35,8 +35,8 @@ QMP_VERSION read 10485760/10485760 bytes at offset 0 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} == using quorum rewrite corrupted mode ==
@@ -67,8 +67,8 @@ QMP_VERSION read 10485760/10485760 bytes at offset 0 10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} -- checking that the image has been corrected -- read 10485760/10485760 bytes at offset 0
@@ -106,8 +106,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION
@@ -115,8 +115,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "Cannot add a child to a quorum in blkverify mode"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} == dynamically removing a child from a quorum ==
@@ -125,31 +125,31 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "The number of children cannot be lower than the vote threshold 2"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "blkverify=on can only be set if there are exactly two files and vote-threshold is 2"}} {"error": {"class": "GenericError", "desc": "Cannot find device='drive0-quorum' nor node-name='drive0-quorum'"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} Testing: QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "The number of children cannot be lower than the vote threshold 2"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/087.out +6 -6
@@ -7,8 +7,8 @@ Testing: QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "'node-name' must be specified for the root node"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Duplicate ID ===
@@ -18,8 +18,8 @@ QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "node-name=disk is conflicting with a device id"}} {"error": {"class": "GenericError", "desc": "Duplicate nodes with node-name='test-node'"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === aio=native without O_DIRECT ===
@@ -28,8 +28,8 @@ Testing: QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "aio=native was specified, but it requires cache.direct=on, which was not specified."}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Encrypted image QCow ===
@@ -40,8 +40,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"error": {"class": "GenericError", "desc": "Use of AES-CBC encrypted IMGFMT images is no longer supported in system emulators"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Encrypted image LUKS ===
@@ -52,8 +52,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Missing driver ===
@@ -63,7 +63,7 @@ Testing: -S QMP_VERSION {"return": {}} {"error": {"class": "GenericError", "desc": "Parameter 'driver' is missing"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/108.out +1 -1
@@ -173,8 +173,8 @@ OK: Reftable is where we expect it {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "create"}} {"return": {}} { "execute": "quit" } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/qemu-iotests/109 +2 -2
@@ -57,13 +57,13 @@ run_qemu() _launch_qemu -drive file="${source_img}",format=raw,cache=${CACHEMODE},aio=${AIOMODE},id=src _send_qemu_cmd $QEMU_HANDLE "{ 'execute': 'qmp_capabilities' }" "return" - _send_qemu_cmd $QEMU_HANDLE \ + capture_events="$qmp_event" _send_qemu_cmd $QEMU_HANDLE \ "{'execute':'drive-mirror', 'arguments':{ 'device': 'src', 'target': '$raw_img', $qmp_format 'mode': 'existing', 'sync': 'full'}}" \ "return" - _send_qemu_cmd $QEMU_HANDLE '' "$qmp_event" + capture_events="$qmp_event JOB_STATUS_CHANGE" _wait_event $QEMU_HANDLE "$qmp_event" if test "$qmp_event" = BLOCK_JOB_ERROR; then _send_qemu_cmd $QEMU_HANDLE '' '"status": "null"' fi
tests/qemu-iotests/109.out +33 -45
405 lines not on this page · raw message
tests/qemu-iotests/117.out +1 -1
@@ -18,8 +18,8 @@ wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} No errors were found on the image. read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/qemu-iotests/120.out +1 -1
@@ -5,8 +5,8 @@ QMP_VERSION wrote 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 65536/65536 bytes at offset 0
tests/qemu-iotests/127.out +1 -1
@@ -28,6 +28,6 @@ wrote 42/42 bytes at offset 0 { 'execute': 'quit' } {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "mirror"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "mirror"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/140.out +1 -1
@@ -19,6 +19,6 @@ read 65536/65536 bytes at offset 0 qemu-io: can't open device nbd+unix:///drv?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'drv' not present { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/143.out +1 -1
@@ -10,6 +10,6 @@ server reported: export 'no_such_export' not present qemu-io: can't open device nbd+unix:///aa--aa1?socket=SOCK_DIR/nbd: Requested export not available server reported: export 'aa--aa...' not present { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/156.out +1 -1
@@ -72,8 +72,8 @@ read 65536/65536 bytes at offset 196608 {"return": ""} { 'execute': 'quit' } -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} read 65536/65536 bytes at offset 0 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
tests/qemu-iotests/176.out +8 -8
@@ -169,8 +169,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576
@@ -206,8 +206,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Test pass bitmap.1 ===
@@ -218,8 +218,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576
@@ -256,8 +256,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Test pass bitmap.2 ===
@@ -268,8 +268,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576
@@ -306,8 +306,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Test pass bitmap.3 ===
@@ -318,8 +318,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} wrote 196608/196608 bytes at offset 2147287040 192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 131072/131072 bytes at offset 2147352576
@@ -353,6 +353,6 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {"sha256": HASH}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/182.out +1 -1
@@ -53,6 +53,6 @@ Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 cluster_size=65536 extended_l2= {'execute': 'qmp_capabilities'} {"return": {}} {'execute': 'quit'} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/183.out +2 -2
@@ -53,11 +53,11 @@ wrote 65536/65536 bytes at offset 1048576 === Shut down and check image === {"execute":"quit"} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"return": {}} {"execute":"quit"} +{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"return": {}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} No errors were found on the image. No errors were found on the image. wrote 65536/65536 bytes at offset 1048576
tests/qemu-iotests/184.out +16 -16
@@ -89,10 +89,6 @@ Testing: "return": [ ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -104,6 +100,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} == property changes in ThrottleGroup ==
@@ -169,10 +169,6 @@ Testing: "iops-total-max": 0 } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -184,6 +180,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} == object creation/set errors ==
@@ -211,10 +211,6 @@ Testing: "desc": "bps/iops/max total values and read/write values cannot be used at the same time" } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -226,6 +222,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} == don't specify group ==
@@ -247,10 +247,6 @@ Testing: "desc": "Parameter 'throttle-group' is missing" } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -262,6 +258,10 @@ Testing: "reason": "host-qmp-quit" } } +{ + "return": { + } +} *** done
tests/qemu-iotests/185 +3 -3
@@ -344,14 +344,14 @@ wait_for_job_and_quit() { sleep 1 + # List of expected events + capture_events='BLOCK_JOB_CANCELLED JOB_STATUS_CHANGE SHUTDOWN' + _send_qemu_cmd $h \ '{"execute": "quit"}' \ 'return' - # List of expected events - capture_events='BLOCK_JOB_CANCELLED JOB_STATUS_CHANGE SHUTDOWN' _wait_event $h 'SHUTDOWN' - QEMU_EVENTS= # Ignore all JOB_STATUS_CHANGE events that came before SHUTDOWN _wait_event $h 'JOB_STATUS_CHANGE' # standby _wait_event $h 'JOB_STATUS_CHANGE' # ready _wait_event $h 'JOB_STATUS_CHANGE' # standby
tests/qemu-iotests/185.out +40 -5
85 lines not on this page · raw message
tests/qemu-iotests/191.out +8 -8
@@ -378,10 +378,6 @@ wrote 65536/65536 bytes at offset 1048576 ] } { 'execute': 'quit' } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -393,6 +389,10 @@ wrote 65536/65536 bytes at offset 1048576 "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 64 MiB (67108864 bytes)
@@ -796,10 +796,6 @@ wrote 65536/65536 bytes at offset 1048576 ] } { 'execute': 'quit' } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -811,6 +807,10 @@ wrote 65536/65536 bytes at offset 1048576 "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 64 MiB (67108864 bytes)
tests/qemu-iotests/195.out +8 -8
@@ -17,10 +17,6 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,backing.node-name=mid "return": { } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -32,6 +28,10 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,backing.node-name=mid "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT.mid file format: IMGFMT
@@ -55,10 +55,6 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,node-name=top "return": { } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -70,6 +66,10 @@ Testing: -drive if=none,file=TEST_DIR/t.IMGFMT,node-name=top "reason": "host-qmp-quit" } } +{ + "return": { + } +} image: TEST_DIR/t.IMGFMT file format: IMGFMT
tests/qemu-iotests/223.out +8 -8
@@ -11,8 +11,8 @@ QMP_VERSION {"return": {}} {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Write part of the file under active bitmap ===
@@ -145,14 +145,14 @@ read 2097152/2097152 bytes at offset 2097152 {"execute":"nbd-server-remove", "arguments":{"name":"n"}} -{"return": {}} -{"execute":"nbd-server-remove", - "arguments":{"name":"n2"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n"}} {"return": {}} {"execute":"nbd-server-remove", "arguments":{"name":"n2"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n2"}} +{"return": {}} +{"execute":"nbd-server-remove", + "arguments":{"name":"n2"}} {"error": {"class": "GenericError", "desc": "Export 'n2' is not found"}} {"execute":"nbd-server-stop"} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n3"}}
@@ -267,14 +267,14 @@ read 2097152/2097152 bytes at offset 2097152 {"execute":"nbd-server-remove", "arguments":{"name":"n"}} -{"return": {}} -{"execute":"nbd-server-remove", - "arguments":{"name":"n2"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n"}} {"return": {}} {"execute":"nbd-server-remove", "arguments":{"name":"n2"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n2"}} +{"return": {}} +{"execute":"nbd-server-remove", + "arguments":{"name":"n2"}} {"error": {"class": "GenericError", "desc": "Export 'n2' is not found"}} {"execute":"nbd-server-stop"} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "n3"}}
@@ -282,8 +282,8 @@ read 2097152/2097152 bytes at offset 2097152 {"execute":"nbd-server-stop"} {"error": {"class": "GenericError", "desc": "NBD server not running"}} {"execute":"quit"} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} === Use qemu-nbd as server ===
tests/qemu-iotests/227.out +16 -16
@@ -54,10 +54,6 @@ Testing: -drive driver=null-co,read-zeroes=on,if=virtio } ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -69,6 +65,10 @@ Testing: -drive driver=null-co,read-zeroes=on,if=virtio "reason": "host-qmp-quit" } } +{ + "return": { + } +} === blockstats with -drive if=none ===
@@ -124,10 +124,6 @@ Testing: -drive driver=null-co,if=none } ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -139,6 +135,10 @@ Testing: -drive driver=null-co,if=none "reason": "host-qmp-quit" } } +{ + "return": { + } +} === blockstats with -blockdev ===
@@ -155,10 +155,6 @@ Testing: -blockdev driver=null-co,node-name=null "return": [ ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -170,6 +166,10 @@ Testing: -blockdev driver=null-co,node-name=null "reason": "host-qmp-quit" } } +{ + "return": { + } +} === blockstats with -blockdev and -device ===
@@ -226,10 +226,6 @@ Testing: -blockdev driver=null-co,read-zeroes=on,node-name=null -device virtio-b } ] } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -241,5 +237,9 @@ Testing: -blockdev driver=null-co,read-zeroes=on,node-name=null -device virtio-b "reason": "host-qmp-quit" } } +{ + "return": { + } +} *** done
tests/qemu-iotests/247.out +1 -1
@@ -17,6 +17,6 @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 134217728, "offset": 134217728, "speed": 0, "type": "commit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} *** done
tests/qemu-iotests/273.out +4 -4
@@ -282,10 +282,6 @@ Testing: -blockdev file,node-name=base,filename=TEST_DIR/t.IMGFMT.base -blockdev ] } } -{ - "return": { - } -} { "timestamp": { "seconds": TIMESTAMP,
@@ -297,5 +293,9 @@ Testing: -blockdev file,node-name=base,filename=TEST_DIR/t.IMGFMT.base -blockdev "reason": "host-qmp-quit" } } +{ + "return": { + } +} *** done
tests/qemu-iotests/308 +2 -2
@@ -77,6 +77,7 @@ fuse_export_add() # $1: Export ID fuse_export_del() { + capture_events="BLOCK_EXPORT_DELETED" \ _send_qemu_cmd $QEMU_HANDLE \ "{'execute': 'block-export-del', 'arguments': {
@@ -84,8 +85,7 @@ fuse_export_del() } }" \ 'return' - _send_qemu_cmd $QEMU_HANDLE \ - '' \ + _wait_event $QEMU_HANDLE \ 'BLOCK_EXPORT_DELETED' }
tests/qemu-iotests/308.out +2 -2
@@ -165,9 +165,9 @@ OK: Post-truncate image size is as expected === Tear down === {'execute': 'quit'} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export-mp"}} +{"return": {}} === Compare copy with original === Images are identical.
@@ -201,9 +201,9 @@ wrote 67108864/67108864 bytes at offset 0 read 67108864/67108864 bytes at offset 0 64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {'execute': 'quit'} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export"}} +{"return": {}} read 67108864/67108864 bytes at offset 0 64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done
tests/qemu-iotests/tests/file-io-error +2 -3
@@ -99,13 +99,12 @@ echo $QEMU_IO -f file -c 'write 0 64M' "$TEST_DIR/fuse-export" | _filter_qemu_io echo -_send_qemu_cmd $QEMU_HANDLE \ +capture_events=BLOCK_EXPORT_DELETED _send_qemu_cmd $QEMU_HANDLE \ "{'execute': 'block-export-del', 'arguments': {'id': 'exp0'}}" \ 'return' -_send_qemu_cmd $QEMU_HANDLE \ - '' \ +_wait_event $QEMU_HANDLE \ 'BLOCK_EXPORT_DELETED' _send_qemu_cmd $QEMU_HANDLE \
tests/qemu-iotests/tests/iothreads-resize.out +1 -1
@@ -3,8 +3,8 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 QMP_VERSION {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-qmp-quit"}} +{"return": {}} image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes)
tests/qemu-iotests/tests/qsd-jobs.out +2 -2
@@ -7,8 +7,8 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/ QMP_VERSION {"return": {}} {"return": {}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} +{"return": {}} === Streaming can't get permission on base node ===
@@ -17,6 +17,6 @@ QMP_VERSION {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} {"error": {"class": "GenericError", "desc": "Permission conflict on node 'fmt_base': permissions 'write' are both required by an unnamed block device (uses node 'fmt_base' as 'root' child) and unshared by stream job 'job0' (uses node 'fmt_base' as 'intermediate node' child)."}} -{"return": {}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_EXPORT_DELETED", "data": {"id": "export1"}} +{"return": {}} *** done
Signature · 1 line
2.43.0
Am 16.01.2024 um 20:00 hat Stefan Hajnoczi geschrieben:
Quoted 21 lines
monitor_qmp_dispatcher_co()runs in the iohandler AioContext that is not polled during nested event loops. The coroutine currently reschedules itself in the main loop'sqemu_aio_contextAioContext, which is polled during nested event loops. One known problem is that QMP device-add callsdrain_call_rcu(), which temporarily drops the BQL, leading to all sorts of havoc like other vCPU threads re-entering device emulation code while another vCPU thread is waiting in device emulation code withaio_poll().Paolo Bonzini suggested running non-coroutine QMP handlers in the iohandler AioContext. This avoids trouble with nested event loops. His original idea was to move coroutine rescheduling to
monitor_qmp_dispatch(), but I resorted to moving it toqmp_dispatch()because we don't know if the QMP handler needs to run in coroutine context inmonitor_qmp_dispatch().monitor_qmp_dispatch()would have been nicer since it's associated with the monitor implementation and not as general asqmp_dispatch(), which is also used by qemu-ga.A number of qemu-iotests need updated .out files because the order of QMP events vs QMP responses has changed.
Quoted 7 lines
Solves Issue #1933.
Fixes 7bed89958bfbf40df9ca681cefbdca63abdde39d ("device_core: use drain_call_rcu in in qmp_device_add")Signed-off-by Stefan Hajnoczi <[email protected]>
Am 16.01.24 um 20:00 schrieb Stefan Hajnoczi:
Quoted 21 lines
monitor_qmp_dispatcher_co()runs in the iohandler AioContext that is not polled during nested event loops. The coroutine currently reschedules itself in the main loop'sqemu_aio_contextAioContext, which is polled during nested event loops. One known problem is that QMP device-add callsdrain_call_rcu(), which temporarily drops the BQL, leading to all sorts of havoc like other vCPU threads re-entering device emulation code while another vCPU thread is waiting in device emulation code withaio_poll().Paolo Bonzini suggested running non-coroutine QMP handlers in the iohandler AioContext. This avoids trouble with nested event loops. His original idea was to move coroutine rescheduling to
monitor_qmp_dispatch(), but I resorted to moving it toqmp_dispatch()because we don't know if the QMP handler needs to run in coroutine context inmonitor_qmp_dispatch().monitor_qmp_dispatch()would have been nicer since it's associated with the monitor implementation and not as general asqmp_dispatch(), which is also used by qemu-ga.A number of qemu-iotests need updated .out files because the order of QMP events vs QMP responses has changed.
Quoted 7 lines
Solves Issue #1933.
Fixes 7bed89958bfbf40df9ca681cefbdca63abdde39d ("device_core: use drain_call_rcu in in qmp_device_add")Signed-off-by Stefan Hajnoczi <[email protected]>
With the patch I can no longer see any do_qmp_dispatch_bh() calls run in vCPU threads.
I also did a bit of smoke testing with some other QMP and QGA commands and didn't find any obvious breakage, so:
P.S.
Unfortunately, the patch does not solve the other issue I came across back then [0] with snapshot_save_job_bh() being executed during a vCPU thread's aio_poll(). See also [1].
I suppose this would need some other mechanism to solve or could it also be scheduled to the iohandler AioContext? It's not directly related to your patch of course, just mentioning it, because it's a similar theme.
[0]:
https://lore.kernel.org/qemu-devel/[email protected]/
[1]: https://gitlab.com/qemu-project/qemu/-/issues/2111
Best Regards,
Fiona
16.01.2024 22:00, Stefan Hajnoczi пишет:
Quoted 21 lines
monitor_qmp_dispatcher_co()runs in the iohandler AioContext that is not polled during nested event loops. The coroutine currently reschedules itself in the main loop'sqemu_aio_contextAioContext, which is polled during nested event loops. One known problem is that QMP device-add callsdrain_call_rcu(), which temporarily drops the BQL, leading to all sorts of havoc like other vCPU threads re-entering device emulation code while another vCPU thread is waiting in device emulation code withaio_poll().Paolo Bonzini suggested running non-coroutine QMP handlers in the iohandler AioContext. This avoids trouble with nested event loops. His original idea was to move coroutine rescheduling to
monitor_qmp_dispatch(), but I resorted to moving it toqmp_dispatch()because we don't know if the QMP handler needs to run in coroutine context inmonitor_qmp_dispatch().monitor_qmp_dispatch()would have been nicer since it's associated with the monitor implementation and not as general asqmp_dispatch(), which is also used by qemu-ga.A number of qemu-iotests need updated .out files because the order of QMP events vs QMP responses has changed.
Quoted 7 lines
Solves Issue #1933.
Fixes 7bed89958bfbf40df9ca681cefbdca63abdde39d ("device_core: use drain_call_rcu in in qmp_device_add")Signed-off-by Stefan Hajnoczi <[email protected]>
This change broke something in 7.2. I'm still debugging it, will come with a follow-up once some more details are found, I'll also check current master with and without this commit.
The prob happens with multiple suspend-resume cycles, - with this change applied, guest does not work as expected after *second* suspend-resume.
JFYI for now.
Thanks,
/mjt
03.02.2024 12:01, Michael Tokarev wrote:
...
Quoted 7 lines
This change broke something in 7.2. I'm still debugging it, will come with a follow-up once some more details are found, I'll also check current master with and without this commit.
The prob happens with multiple suspend-resume cycles, - with this change applied, guest does not work as expected after *second* suspend-resume.
So, it turned out the prob here exists on master too, and manifests itself the same way on 7.2.9 or on 8.2.1, - in all cases where we have this change applied it works (or breaks) equally.
A (simple) reproducer so far is a hibernate test, - it fails *only* after suspend-to-ram, but works fine after just hibernate.
I used just an initrd (with a drive image used for swap - for hibernation space).
qemu-img create s.img 256M
mkswap s.img
qemu-system-x86_64 \
-serial stdio -vga none -display none -parallel none -net none \
-machine q35 \
-drive file=s.img,if=ide,format=raw \
-m 256 \
-monitor unix:ttyS0,server,nowait \
-kernel /boot/vmlinuz-6.1.0-15-amd64 \
-initrd /boot/initrd.img-6.1.0-15-amd64 \
-append "shell=/bin/sh console=ttyS0 root=none"
There, in the guest (it has busybox only here):
# swapon /dev/sda # echo mem > /sys/power/state (system_wakeup on the monitor) # echo disk > /sys/power/state
The system will hibernate but *not* turn off power, qemu will continue running, while all console messages are the same as when it works fine. qemu process is spinning up with 100% cpu usage at this stage.
Without the intermediate suspend-to-ram or without the commit in question, qemu process will exit normally at this stage.
This is a somewhat patalogical test case, but I see it as an indicator of something else being wrong, like we aren't saving or restoring some state now which we should do.
The tight loop also suggests we're not having success in there.
Thanks,
/mjt
Quoted 48 lines · Michael Tokarev
03.02.2024 12:01, Michael Tokarev wrote:
...Quoted 7 lines
This change broke something in 7.2. I'm still debugging it, will come with a follow-up once some more details are found, I'll also check current master with and without this commit.
The prob happens with multiple suspend-resume cycles, - with this change applied, guest does not work as expected after *second* suspend-resume.
So, it turned out the prob here exists on master too, and manifests itself the same way on 7.2.9 or on 8.2.1, - in all cases where we have this change applied it works (or breaks) equally.
A (simple) reproducer so far is a hibernate test, - it fails *only* after suspend-to-ram, but works fine after just hibernate.
I used just an initrd (with a drive image used for swap - for hibernation space).
qemu-img create s.img 256M mkswap s.img qemu-system-x86_64 \ -serial stdio -vga none -display none -parallel none -net none \ -machine q35 \ -drive file=s.img,if=ide,format=raw \ -m 256 \ -monitor unix:ttyS0,server,nowait \ -kernel /boot/vmlinuz-6.1.0-15-amd64 \ -initrd /boot/initrd.img-6.1.0-15-amd64 \ -append "shell=/bin/sh console=ttyS0 root=none"There, in the guest (it has busybox only here):
# swapon /dev/sda # echo mem > /sys/power/state (system_wakeup on the monitor) # echo disk > /sys/power/stateThe system will hibernate but *not* turn off power, qemu will continue running, while all console messages are the same as when it works fine. qemu process is spinning up with 100% cpu usage at this stage.
Without the intermediate suspend-to-ram or without the commit in question, qemu process will exit normally at this stage.
Quoted 5 lines
This is a somewhat patalogical test case, but I see it as an indicator of something else being wrong, like we aren't saving or restoring some state now which we should do.
The tight loop also suggests we're not having success in there.
I'm unable to reproduce this. QEMU v8.2.0, v8.1.0, and even v7.2.0 all spin with 100% CPU usage on my machine. It looks to me like this behavior is not a regression caused by this commit.
Do v7.2.0, v8.1.0, and v8.2.0 terminate QEMU on your machine?
Stefan
Quoted 15 lines · Stefan Hajnoczi
Several bugs have been reported related to how QMP commands are rescheduled in
qemu_aio_context:
- https://gitlab.com/qemu-project/qemu/-/issues/1933
- https://issues.redhat.com/browse/RHEL-17369
- https://bugzilla.redhat.com/show_bug.cgi?id=2215192
- https://bugzilla.redhat.com/show_bug.cgi?id=2214985The first instance of the bug interacted with
drain_call_rcu()temporarily dropping the BQL and resulted in vCPU threads entering device emulation code simultaneously (something that should never happen). I set out to makedrain_call_rcu()safe to use in this environment, but Paolo and Kevin discussed the possibility of avoiding rescheduling themonitor_qmp_dispatcher_co()coroutine for non-coroutine commands. This would prevent monitor commands from running during vCPU threadaio_poll()entirely and addresses the root cause.
Quoted 6 lines
This patch series implements this idea. qemu-iotests is sensitive to the exact order in which QMP events and responses are emitted. Running QMP handlers in the iohandler AioContext causes some QMP events to be ordered differently than before. It is therefore necessary to adjust the reference output in many test cases. The actual QMP code change is small and everything else is just to make qemu-iotests happy.
This seems to be -stable material too, once it is applied to master. It's difficult for me to catch the issue (#1933 @gitlab) locally, but I did have some success there, and it always work after this patch is applied. It also seem to work fine on 7.2, not only on 8.2, fwiw.
Thanks,
/mjt
Quoted 16 lines · Stefan Hajnoczi
Several bugs have been reported related to how QMP commands are rescheduled in
qemu_aio_context:
- https://gitlab.com/qemu-project/qemu/-/issues/1933
- https://issues.redhat.com/browse/RHEL-17369
- https://bugzilla.redhat.com/show_bug.cgi?id=2215192
- https://bugzilla.redhat.com/show_bug.cgi?id=2214985The first instance of the bug interacted with
drain_call_rcu()temporarily dropping the BQL and resulted in vCPU threads entering device emulation code simultaneously (something that should never happen). I set out to makedrain_call_rcu()safe to use in this environment, but Paolo and Kevin discussed the possibility of avoiding rescheduling themonitor_qmp_dispatcher_co()coroutine for non-coroutine commands. This would prevent monitor commands from running during vCPU threadaio_poll()entirely and addresses the root cause.
Quoted 6 lines
This patch series implements this idea. qemu-iotests is sensitive to the exact order in which QMP events and responses are emitted. Running QMP handlers in the iohandler AioContext causes some QMP events to be ordered differently than before. It is therefore necessary to adjust the reference output in many test cases. The actual QMP code change is small and everything else is just to make qemu-iotests happy.
Hi; we have a suspicion that this change has resulted in a flaky-CI
test: iotest-144 sometimes fails, apparently because a "return" result from QMP isn't always returned at the same place in relation to other QMP events. Could you have a look at it?
https://gitlab.com/qemu-project/qemu/-/issues/2126
thanks
-- PMM
YangHang Liu
Tested-byIt's easily for me to encounter " ../block/qcow2.c:5263:
ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *, Error **): Assertion `false' failed" issue during 1Q vhost-user interface + RT VM + post-copy migration
After applying this patch, the issue is still not reproduced even if I repeat the same migration test for 60 times.
Best Regards,
YangHang Liu
Quoted 35 lines · Peter Maydell
Quoted 16 lines · Stefan Hajnoczi
Several bugs have been reported related to how QMP commands are rescheduled in
qemu_aio_context:
- https://gitlab.com/qemu-project/qemu/-/issues/1933
- https://issues.redhat.com/browse/RHEL-17369
- https://bugzilla.redhat.com/show_bug.cgi?id=2215192
- https://bugzilla.redhat.com/show_bug.cgi?id=2214985The first instance of the bug interacted with
drain_call_rcu()temporarily dropping the BQL and resulted in vCPU threads entering device emulation code simultaneously (something that should never happen). I set out to makedrain_call_rcu()safe to use in this environment, but Paolo and Kevin discussed the possibility of avoiding rescheduling themonitor_qmp_dispatcher_co()coroutine for non-coroutine commands. This would prevent monitor commands from running during vCPU threadaio_poll()entirely and addresses the root cause.Quoted 6 lines
This patch series implements this idea. qemu-iotests is sensitive to the exact order in which QMP events and responses are emitted. Running QMP handlers in the iohandler AioContext causes some QMP events to be ordered differently than before. It is therefore necessary to adjust the reference output in many test cases. The actual QMP code change is small and everything else is just to make qemu-iotests happy.
Hi; we have a suspicion that this change has resulted in a flaky-CI
test: iotest-144 sometimes fails, apparently because a "return" result from QMP isn't always returned at the same place in relation to other QMP events. Could you have a look at it?https://gitlab.com/qemu-project/qemu/-/issues/2126
thanks
-- PMM