block-backend: Queue requests while drained
v2 cover block-backend: Queue requests while drained
This series fixes the problem that devices like IDE, which submit requests as a direct result of I/O from the CPU thread, can continue to submit new requests even in a drained section.
v2:
- Rebased on top of block-next
- Replaced patch 2 with draining mirror_top_bs instead [Max]
- Removed wait_while_drained parameter from patch 3 [Max]
- Covered blk_co_flush() [Max]
- Fixed some typos [Eric]
Kevin Wolf (3):
block: Remove blk_pread_unthrottled()
mirror: Keep mirror_top_bs drained after dropping permissions
block-backend: Queue requests while drained
include/sysemu/block-backend.h | 3 +- block/backup.c | 1 + block/block-backend.c | 69 ++++++++++++++++++++++++---------- block/commit.c | 2 + block/mirror.c | 7 +++- blockjob.c | 3 ++ hw/block/hd-geometry.c | 7 +--- tests/test-bdrv-drain.c | 1 + 8 files changed, 65 insertions(+), 28 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/3 block: Remove blk_pread_unthrottled()
The functionality offered by blk_pread_unthrottled() goes back to commit 498e386c584. Then, we couldn't perform I/O throttling with synchronous requests because timers wouldn't be executed in polling loops. So the commit automatically disabled I/O throttling as soon as a synchronous request was issued.
However, for geometry detection during disk initialisation, we always used (and still use) synchronous requests even if guest requests use AIO later. Geometry detection was not wanted to disable I/O throttling, so bdrv_pread_unthrottled() was introduced which disabled throttling only temporarily.
All of this isn't necessary any more because we do run timers in polling loop and even synchronous requests are now using coroutine infrastructure internally. For this reason, commit 90c78624f already removed the automatic disabling of I/O throttling.
It's time to get rid of the workaround for the removed code, and its abuse of blk_root_drained_begin()/end(), as well.
--- include/sysemu/block-backend.h | 2 -- block/block-backend.c | 16 ---------------- hw/block/hd-geometry.c | 7 +------ 3 files changed, 1 insertion(+), 24 deletions(-)
include/sysemu/block-backend.h +0 -2
@@ -117,8 +117,6 @@ char *blk_get_attached_dev_id(BlockBackend *blk);BlockBackend *blk_by_dev (void *dev);BlockBackend *blk_by_qdev_id (const char *id,Error **errp);void blk_set_dev_ops (BlockBackend *blk,const BlockDevOps *ops,void *opaque); -int blk_pread_unthrottled (BlockBackend *blk,int64_t offset,uint8_t *buf, -int bytes); int coroutine_fn blk_co_preadv(BlockBackend *blk, int64_t offset,unsigned int bytes,QEMUIOVector *qiov,BdrvRequestFlags flags);
block/block-backend.c +0 -16
@@ -1237,22 +1237,6 @@ static int blk_prw(BlockBackend *blk, int64_t offset, uint8_t *buf,return rwco.ret; } -int blk_pread_unthrottled (BlockBackend *blk,int64_t offset,uint8_t *buf, -int count) -{ -int ret; - - ret =blk_check_byte_request (blk, offset, count); -if (ret <0 ) { -return ret; - } - -blk_root_drained_begin (blk->root); - ret =blk_pread (blk, offset, buf, count); -blk_root_drained_end (blk->root,NULL ); -return ret; -} -int blk_pwrite_zeroes (BlockBackend *blk,int64_t offset,int bytes,BdrvRequestFlags flags) {
hw/block/hd-geometry.c +1 -6
@@ -63,12 +63,7 @@ static int guess_disk_lchs(BlockBackend *blk,blk_get_geometry (blk, &nb_sectors); -/** -* The function will be invoked during startup not only in sync I/O mode, -* but also in async I/O mode. So the I/O throttling function has to -* be disabled temporarily here, not permanently. -*/ -if (blk_pread_unthrottled (blk,0 , buf,BDRV_SECTOR_SIZE ) <0 ) { +if (blk_pread (blk,0 , buf,BDRV_SECTOR_SIZE ) <0 ) {return -1 ; }/* test msdos magic */
Signature · 1 line
2.20.1
PATCH v2 2/3 mirror: Keep mirror_top_bs drained after dropping permissions
In reply to Kevin Wolfmirror_top_bs is currently implicitly drained through its connection to the source or the target node. However, the drain section for target_bs ends early after moving mirror_top_bs from src to target_bs, so that requests can already be restarted while mirror_top_bs is still present in the chain, but has dropped all permissions and therefore runs into an assertion failure like this:
qemu-system-x86_64: block/io.c:1634: bdrv_co_write_req_prepare:
Assertion `child->perm & BLK_PERM_WRITE' failed.Keep mirror_top_bs drained until all graph changes have completed.
--- block/mirror.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
block/mirror.c +5 -1
@@ -656,7 +656,10 @@ static int mirror_exit_common(Job *job) s->target =NULL ;/* We don't access the source any more. Dropping any WRITE/RESIZE is -* required before it could become a backing file of target_bs. */ +* required before it could become a backing file of target_bs. Not having +* these permissions any more means that we can't allow any new requests on +* mirror_top_bs from now on, so keep it drained. */ +bdrv_drained_begin (mirror_top_bs); bs_opaque->stop = true;bdrv_child_refresh_perms (mirror_top_bs, mirror_top_bs->backing, &error_abort);
@@ -724,6 +727,7 @@ static int mirror_exit_common(Job *job) bs_opaque->job =NULL ;bdrv_drained_end (src); +bdrv_drained_end (mirror_top_bs); s->in_drain = false;bdrv_unref (mirror_top_bs);bdrv_unref (src);
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v2 2/3] mirror: Keep mirror_top_bs drained after dropping permissions
Quoted 22 lines · Kevin Wolf
mirror_top_bsis currently implicitly drained through its connection to the source or the target node. However, the drain section fortarget_bsends early after movingmirror_top_bsfrom src totarget_bs, so that requests can already be restarted whilemirror_top_bsis still present in the chain, but has dropped all permissions and therefore runs into an assertion failure like this:qemu-system-x86_64: block/io.c:1634: bdrv_co_write_req_prepare: Assertion `child->perm & BLK_PERM_WRITE' failed.Keep
mirror_top_bsdrained until all graph changes have completed.Signed-off-by Kevin Wolf <[email protected]>--- block/mirror.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)block/mirror.c +0
-0
block/mirror.c static int mirror_exit_common(Job *job) · line 661
Quoted 7 lines
block/mirror.c +3
-1@@ -656,7 +656,10 @@ static int mirror_exit_common(Job *job) s->target =NULL ;/* We don't access the source any more. Dropping any WRITE/RESIZE is -* required before it could become a backing file of target_bs. */ +* required before it could become a backing file of target_bs. Not having +* these permissions any more means that we can't allow any new requests on +* mirror_top_bs from now on, so keep it drained. */
You’re lucky Patchew broke or it would have complained about multi-line comments needing /* and */ on separate lines. :-)
I’m perfectly happy with this style, though:
Quoted 13 lines
block/mirror.c +2
-0+bdrv_drained_begin (mirror_top_bs); bs_opaque->stop = true;bdrv_child_refresh_perms (mirror_top_bs, mirror_top_bs->backing, &error_abort);@@ -724,6 +727,7 @@ static int mirror_exit_common(Job *job) bs_opaque->job =NULL ;bdrv_drained_end (src); +bdrv_drained_end (mirror_top_bs); s->in_drain = false;bdrv_unref (mirror_top_bs);bdrv_unref (src);
PATCH v2 3/3 block-backend: Queue requests while drained
In reply to Kevin WolfThis fixes devices like IDE that can still start new requests from I/O handlers in the CPU thread while the block backend is drained.
The basic assumption is that in a drain section, no new requests should be allowed through a BlockBackend (blk_drained_begin/end don't exist, we get drain sections only on the node level). However, there are two special cases where requests should not be queued:
1. Block jobs: We already make sure that block jobs are paused in a drain section, so they won't start new requests. However, if the drain_begin is called on the job's BlockBackend first, it can happen that we deadlock because the job stays busy until it reaches a pause point - which it can't if its requests aren't processed any more.
The proper solution here would be to make all requests through the job's filter node instead of using a BlockBackend. For now, just disabling request queuing on the job BlockBackend is simpler.
2. In test cases where making requests through bdrv_* would be cumbersome because we'd need a BdrvChild. As we already got the functionality to disable request queuing from 1., use it in tests, too, for convenience.
--- include/sysemu/block-backend.h | 1 + block/backup.c | 1 + block/block-backend.c | 53 ++++++++++++++++++++++++++++++++-- block/commit.c | 2 ++ block/mirror.c | 1 + blockjob.c | 3 ++ tests/test-bdrv-drain.c | 1 + 7 files changed, 59 insertions(+), 3 deletions(-)
include/sysemu/block-backend.h +1 -0
@@ -104,6 +104,7 @@ void blk_get_perm(BlockBackend *blk, uint64_t *perm, uint64_t *shared_perm);void blk_set_allow_write_beyond_eof (BlockBackend *blk,bool allow);void blk_set_allow_aio_context_change (BlockBackend *blk,bool allow); +void blk_set_disable_request_queuing (BlockBackend *blk,bool disable);void blk_iostatus_enable (BlockBackend *blk);bool blk_iostatus_is_enabled (const BlockBackend *blk);BlockDeviceIoStatus blk_iostatus (const BlockBackend *blk);
block/backup.c +1 -0
@@ -644,6 +644,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,if (ret <0 ) { goto error; } +blk_set_disable_request_queuing (job->target, true); job->on_source_error = on_source_error; job->on_target_error = on_target_error;
block/block-backend.c +50 -3
126 lines not on this page · raw message
block/commit.c +2 -0
@@ -350,6 +350,7 @@ void commit_start(const char *job_id, BlockDriverState *bs,if (ret <0 ) { goto fail; } +blk_set_disable_request_queuing (s->base, true); s->base_bs = base;/* Required permissions are already taken with block_job_add_bdrv() */
@@ -358,6 +359,7 @@ void commit_start(const char *job_id, BlockDriverState *bs,if (ret <0 ) { goto fail; } +blk_set_disable_request_queuing (s->top, true); s->backing_file_str =g_strdup (backing_file_str); s->on_error = on_error;
block/mirror.c +1 -0
@@ -1636,6 +1636,7 @@ static BlockJob *mirror_start_job(blk_set_force_allow_inactivate (s->target); }blk_set_allow_aio_context_change (s->target, true); +blk_set_disable_request_queuing (s->target, true); s->replaces =g_strdup (replaces); s->on_source_error = on_source_error;
blockjob.c +3 -0
@@ -445,6 +445,9 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver,bdrv_op_unblock (bs,BLOCK_OP_TYPE_DATAPLANE , job->blocker); +/* Disable request queuing in the BlockBackend to avoid deadlocks on drain: +* The job reports that it's busy until it reaches a pause point. */ +blk_set_disable_request_queuing (blk, true);blk_set_allow_aio_context_change (blk, true);/* Only set speed when necessary to avoid NotSupported error */
tests/test-bdrv-drain.c +1 -0
@@ -686,6 +686,7 @@ static void test_iothread_common(enum drain_type drain_type, int drain_thread) &error_abort); s = bs->opaque;blk_insert_bs (blk, bs, &error_abort); +blk_set_disable_request_queuing (blk, true);blk_set_aio_context (blk, ctx_a, &error_abort);aio_context_acquire (ctx_a);
Signature · 1 line
2.20.1
Quoted 24 lines · Kevin Wolf
This fixes devices like IDE that can still start new requests from I/O handlers in the CPU thread while the block backend is drained.
The basic assumption is that in a drain section, no new requests should be allowed through a BlockBackend (blk_drained_begin/end don't exist, we get drain sections only on the node level). However, there are two special cases where requests should not be queued:
1. Block jobs: We already make sure that block jobs are paused in a drain section, so they won't start new requests. However, if the
drain_beginis called on the job's BlockBackend first, it can happen that we deadlock because the job stays busy until it reaches a pause point - which it can't if its requests aren't processed any more.The proper solution here would be to make all requests through the job's filter node instead of using a BlockBackend. For now, just disabling request queuing on the job BlockBackend is simpler.
2. In test cases where making requests through bdrv_* would be cumbersome because we'd need a BdrvChild. As we already got the functionality to disable request queuing from 1., use it in tests, too, for convenience.
Signed-off-by Kevin Wolf <[email protected]>
Quoted 9 lines
--- include/sysemu/block-backend.h | 1 + block/backup.c | 1 + block/block-backend.c | 53 ++++++++++++++++++++++++++++++++-- block/commit.c | 2 ++ block/mirror.c | 1 + blockjob.c | 3 ++ tests/test-bdrv-drain.c | 1 + 7 files changed, 59 insertions(+), 3 deletions(-)
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Type: series
Subject: [Qemu-devel] [PATCH v2 0/3] block-backend: Queue requests while drained
Message-id: [email protected]
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/[email protected] -> patchew/[email protected]
Submodule 'capstone' (https://git.qemu.org/git/capstone.git) registered for path 'capstone'
Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc'
Submodule 'roms/QemuMacDrivers' (https://git.qemu.org/git/QemuMacDrivers.git) registered for path 'roms/QemuMacDrivers' Submodule 'roms/SLOF' (https://git.qemu.org/git/SLOF.git) registered for path 'roms/SLOF'
Submodule 'roms/edk2' (https://git.qemu.org/git/edk2.git) registered for path 'roms/edk2'
Submodule 'roms/ipxe' (https://git.qemu.org/git/ipxe.git) registered for path 'roms/ipxe'
Submodule 'roms/openbios' (https://git.qemu.org/git/openbios.git) registered for path 'roms/openbios'
Submodule 'roms/openhackware' (https://git.qemu.org/git/openhackware.git) registered for path 'roms/openhackware'
Submodule 'roms/opensbi' (https://git.qemu.org/git/opensbi.git) registered for path 'roms/opensbi'
Submodule 'roms/qemu-palcode' (https://git.qemu.org/git/qemu-palcode.git) registered for path 'roms/qemu-palcode'
Submodule 'roms/seabios' (https://git.qemu.org/git/seabios.git/) registered for path 'roms/seabios'
Submodule 'roms/seabios-hppa' (https://git.qemu.org/git/seabios-hppa.git) registered for path 'roms/seabios-hppa'
Submodule 'roms/sgabios' (https://git.qemu.org/git/sgabios.git) registered for path 'roms/sgabios'
Submodule 'roms/skiboot' (https://git.qemu.org/git/skiboot.git) registered for path 'roms/skiboot'
Submodule 'roms/u-boot' (https://git.qemu.org/git/u-boot.git) registered for path 'roms/u-boot'
Submodule 'roms/u-boot-sam460ex' (https://git.qemu.org/git/u-boot-sam460ex.git) registered for path 'roms/u-boot-sam460ex' Submodule 'slirp' (https://git.qemu.org/git/libslirp.git) registered for path 'slirp'
Submodule 'tests/fp/berkeley-softfloat-3' (https://git.qemu.org/git/berkeley-softfloat-3.git) registered for path 'tests/fp/berkeley-softfloat-3' Submodule 'tests/fp/berkeley-testfloat-3' (https://git.qemu.org/git/berkeley-testfloat-3.git) registered for path 'tests/fp/berkeley-testfloat-3' Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into 'capstone'...
Submodule path 'capstone': checked out '22ead3e0bfdb87516656453336160e0a37b066bf'
Cloning into 'dtc'...
Submodule path 'dtc': checked out '88f18909db731a627456f26d779445f84e449536'
Cloning into 'roms/QemuMacDrivers'...
Submodule path 'roms/QemuMacDrivers': checked out '90c488d5f4a407342247b9ea869df1c2d9c8e266'
Cloning into 'roms/SLOF'...
Submodule path 'roms/SLOF': checked out 'ba1ab360eebe6338bb8d7d83a9220ccf7e213af3'
Cloning into 'roms/edk2'...
Submodule path 'roms/edk2': checked out '20d2e5a125e34fc8501026613a71549b2a1a3e54'
Submodule 'SoftFloat' (https://github.com/ucb-bar/berkeley-softfloat-3.git) registered for path 'ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3' Submodule 'CryptoPkg/Library/OpensslLib/openssl' (https://github.com/openssl/openssl) registered for path 'CryptoPkg/Library/OpensslLib/openssl' Cloning into 'ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3'...
Submodule path 'roms/edk2/ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3': checked out 'b64af41c3276f97f0e181920400ee056b9c88037' Cloning into 'CryptoPkg/Library/OpensslLib/openssl'...
Submodule path 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl': checked out '50eaac9f3337667259de725451f201e784599687' Submodule 'boringssl' (https://boringssl.googlesource.com/boringssl) registered for path 'boringssl'
Submodule 'krb5' (https://github.com/krb5/krb5) registered for path 'krb5'
Submodule 'pyca.cryptography' (https://github.com/pyca/cryptography.git) registered for path 'pyca-cryptography'
Cloning into 'boringssl'...
Submodule path 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl/boringssl': checked out '2070f8ad9151dc8f3a73bffaa146b5e6937a583f' Cloning into 'krb5'...
Submodule path 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl/krb5': checked out 'b9ad6c49505c96a088326b62a52568e3484f2168' Cloning into 'pyca-cryptography'...
Submodule path 'roms/edk2/CryptoPkg/Library/OpensslLib/openssl/pyca-cryptography': checked out '09403100de2f6f1cdd0d484dcb8e620f1c335c8f' Cloning into 'roms/ipxe'...
Submodule path 'roms/ipxe': checked out 'de4565cbe76ea9f7913a01f331be3ee901bb6e17'
Cloning into 'roms/openbios'...
Submodule path 'roms/openbios': checked out 'c79e0ecb84f4f1ee3f73f521622e264edd1bf174'
Cloning into 'roms/openhackware'...
Submodule path 'roms/openhackware': checked out 'c559da7c8eec5e45ef1f67978827af6f0b9546f5'
Cloning into 'roms/opensbi'...
Submodule path 'roms/opensbi': checked out 'ce228ee0919deb9957192d723eecc8aaae2697c6'
Cloning into 'roms/qemu-palcode'...
Submodule path 'roms/qemu-palcode': checked out 'bf0e13698872450164fa7040da36a95d2d4b326f'
Cloning into 'roms/seabios'...
Submodule path 'roms/seabios': checked out 'a5cab58e9a3fb6e168aba919c5669bea406573b4'
Cloning into 'roms/seabios-hppa'...
Submodule path 'roms/seabios-hppa': checked out '0f4fe84658165e96ce35870fd19fc634e182e77b'
Cloning into 'roms/sgabios'...
Submodule path 'roms/sgabios': checked out 'cbaee52287e5f32373181cff50a00b6c4ac9015a'
Cloning into 'roms/skiboot'...
Submodule path 'roms/skiboot': checked out '261ca8e779e5138869a45f174caa49be6a274501'
Cloning into 'roms/u-boot'...
Submodule path 'roms/u-boot': checked out 'd3689267f92c5956e09cc7d1baa4700141662bff'
Cloning into 'roms/u-boot-sam460ex'...
error: RPC failed; result=18, HTTP code = 200
fatal: The remote end hung up unexpectedly
fatal: protocol error: bad pack header
Clone of 'https://git.qemu.org/git/u-boot-sam460ex.git' into submodule path 'roms/u-boot-sam460ex' failed
Traceback (most recent call last):
File "./patchew-cli", line 504, in test_one
git_clone_repo(clone, r["repo"], r["head"], logf)
File "./patchew-cli", line 50, in git_clone_repo
stderr=logf, stdout=logf)
File "/usr/lib64/python3.4/subprocess.py", line 558, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'clone', '-q', '--recursive', '/home/patchew/.cache/patchew-git-cache/httpsgithubcompatchewprojectqemu-3c8cf5a9c21ff8782164d1def7f44bd888713384', '/var/tmp/patchew-tester-tmp-ecmgcz62/src']' returned non-zero exit status 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]