nbd/server: attach client channel to the export's AioContext
PATCH v2 1/1 nbd/server: attach client channel to the export's AioContext
On creation, the export's AioContext is set to the same one as the BlockBackend, while the AioContext in the client QIOChannel is left untouched.
As a result, when using data-plane, nbd_client_receive_next_request() schedules coroutines in the IOThread AioContext, while the client's QIOChannel is serviced from the main_loop, potentially triggering the assertion at qio_channel_restart_[read|write].
To fix this, as soon we have the export corresponding to the client, we call qio_channel_attach_aio_context() to attach the QIOChannel context to the export's AioContext. This matches with the logic at blk_aio_attached().
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1748253
---
Changelog
v2:
- Attach the channel once after negotiation completes, avoiding duplication. (thanks Kevin Wolf).
--- nbd/server.c | 5 +++++ 1 file changed, 5 insertions(+)
nbd/server.c +5 -0
@@ -1297,6 +1297,11 @@ static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)return ret; } +/* Attach the channel to the same AioContext as the export */ +if (client->exp && client->exp->ctx) { +qio_channel_attach_aio_context (client->ioc, client->exp->ctx); + } +assert (!client->optlen);trace_nbd_negotiate_success ();
Signature · 1 line
2.21.0
Quoted 31 lines · Sergio Lopez
On creation, the export's AioContext is set to the same one as the BlockBackend, while the AioContext in the client QIOChannel is left untouched.
As a result, when using data-plane,
nbd_client_receive_next_request()schedules coroutines in the IOThread AioContext, while the client's QIOChannel is serviced from themain_loop, potentially triggering the assertion at qio_channel_restart_[read|write].To fix this, as soon we have the export corresponding to the client, we call
qio_channel_attach_aio_context()to attach the QIOChannel context to the export's AioContext. This matches with the logic atblk_aio_attached().RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1748253
Signed-off-by Sergio Lopez <[email protected]>---
Changelogv2:
- Attach the channel once after negotiation completes, avoiding duplication. (thanks Kevin Wolf).--- nbd/server.c | 5 +++++ 1 file changed, 5 insertions(+)nbd/server.c +0
-0
nbd/server.c static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp) · line 1304
Quoted 8 lines
nbd/server.c +5
-0@@ -1297,6 +1297,11 @@ static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)return ret; } +/* Attach the channel to the same AioContext as the export */ +if (client->exp && client->exp->ctx) { +qio_channel_attach_aio_context (client->ioc, client->exp->ctx); + } +
Will queue through my NBD tree.
Quoted 4 lines
nbd/server.c +0
-0assert (!client->optlen);trace_nbd_negotiate_success ();
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Quoted 33 lines · Sergio Lopez
On creation, the export's AioContext is set to the same one as the BlockBackend, while the AioContext in the client QIOChannel is left untouched.
As a result, when using data-plane,
nbd_client_receive_next_request()schedules coroutines in the IOThread AioContext, while the client's QIOChannel is serviced from themain_loop, potentially triggering the assertion at qio_channel_restart_[read|write].To fix this, as soon we have the export corresponding to the client, we call
qio_channel_attach_aio_context()to attach the QIOChannel context to the export's AioContext. This matches with the logic atblk_aio_attached().RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1748253
Signed-off-by Sergio Lopez <[email protected]>---
Changelogv2:
- Attach the channel once after negotiation completes, avoiding duplication. (thanks Kevin Wolf).--- nbd/server.c | 5 +++++ 1 file changed, 5 insertions(+)nbd/server.c +0
-0@@ -1297,6 +1297,11 @@ static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)return ret;
nbd/server.c static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp) · line 1307
Quoted 10 lines
nbd/server.c +5
-0} +/* Attach the channel to the same AioContext as the export */ +if (client->exp && client->exp->ctx) { +qio_channel_attach_aio_context (client->ioc, client->exp->ctx); + } +assert (!client->optlen);trace_nbd_negotiate_success ();
I assume this patch has been superseded by Eric's later patches?
--js
Quoted 37 lines · John Snow
Quoted 34 lines · Sergio Lopez
On creation, the export's AioContext is set to the same one as the BlockBackend, while the AioContext in the client QIOChannel is left untouched.
As a result, when using data-plane,
nbd_client_receive_next_request()schedules coroutines in the IOThread AioContext, while the client's QIOChannel is serviced from themain_loop, potentially triggering the assertion at qio_channel_restart_[read|write].To fix this, as soon we have the export corresponding to the client, we call
qio_channel_attach_aio_context()to attach the QIOChannel context to the export's AioContext. This matches with the logic atblk_aio_attached().RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1748253
Signed-off-by Sergio Lopez <[email protected]>---
Changelogv2:
- Attach the channel once after negotiation completes, avoiding duplication. (thanks Kevin Wolf).--- nbd/server.c | 5 +++++ 1 file changed, 5 insertions(+)nbd/server.c +0
-0@@ -1297,6 +1297,11 @@ static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)return ret; }
Quoted 11 lines
nbd/server.c static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp) · line 1307
Quoted 9 lines
nbd/server.c +5
-0+/* Attach the channel to the same AioContext as the export */ +if (client->exp && client->exp->ctx) { +qio_channel_attach_aio_context (client->ioc, client->exp->ctx); + } +assert (!client->optlen);trace_nbd_negotiate_success ();I assume this patch has been superseded by Eric's later patches?
Nevermind -- my filtering got messed up slightly and I missed the followup. I see that Eric staged this.
--js
Quoted 9 lines · John Snow
Quoted 9 lines
Quoted 7 lines
To fix this, as soon we have the export corresponding to the client, we call
qio_channel_attach_aio_context()to attach the QIOChannel context to the export's AioContext. This matches with the logic atblk_aio_attached().
Quoted 4 lines
Quoted 1 line
I assume this patch has been superseded by Eric's later patches?
Nevermind -- my filtering got messed up slightly and I missed the followup. I see that Eric staged this.
I actually think both patches are needed: this one covers transactions, while my later patch was on top of this to protect shutdown. But now you've made me curious; I'll see if my patch hoisted in front still solves everything, or if we really do need both.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Quoted 12 lines · Eric Blake
Quoted 11 lines · John Snow
Quoted 10 lines
Quoted 7 lines
To fix this, as soon we have the export corresponding to the client, we call
qio_channel_attach_aio_context()to attach the QIOChannel context to the export's AioContext. This matches with the logic atblk_aio_attached().I assume this patch has been superseded by Eric's later patches?
Quoted 8 lines
Quoted 2 lines
Nevermind -- my filtering got messed up slightly and I missed the followup. I see that Eric staged this.
I actually think both patches are needed: this one covers transactions, while my later patch was on top of this to protect shutdown. But now you've made me curious; I'll see if my patch hoisted in front still solves everything, or if we really do need both.
Nope, both patches are still needed. Sergio's fixes the assertion:
(qemu) qemu-kvm: io/channel.c:411: qio_channel_restart_read:
Assertion `qemu_get_current_aio_context() == qemu_coroutine_get_aio_context(co)' failed.
while mine fixes:
+qemu: qemu_mutex_unlock_impl: Operation not permitted
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org