migration: Fix rare hang of migration_channel_read_peek()
PATCH v3 1/1 migration: Fix rare hang of migration_channel_read_peek()
In an unlikely case, when a migration stream is attached to the destination QEMU and only send <4 bytes to the channel as magic, it's possible that migration_channel_read_peek() may spin forever.
Fix it by adding a manual sleep for partial read.
Since the path isn't attached to a coroutine, it means when partial read happens, there's yet not much we can do but hang the main thread, it will happen even for len==0 case. It means monitors can hang due to this, either partial read or no data arrived (but connection established).
Leave this for later, the hope is this is extremely rare in production.
Resolves: https://gitlab.com/qemu-project/qemu/-/work_items/3889
--- migration/channel.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
migration/channel.c +9 -2
@@ -296,9 +296,16 @@ int migration_channel_read_peek(QIOChannel *ioc,if (len == buflen) {break ; + }else if (len ==QIO_CHANNEL_ERR_BLOCK ) { +qio_channel_wait_cond (ioc,G_IO_IN ); + }else { +/* +* When partially ready, we can't use qio_channel_wait_cond() +* because it will return immediately. Apply a manual wait. +*/ +assert (!qemu_in_coroutine ()); +g_usleep (1000 ); } - -qio_channel_wait_cond (ioc,G_IO_IN ); }return 0 ;
Signature · 1 line
2.54.0
Quoted 13 lines · Peter Xu
In an unlikely case, when a migration stream is attached to the destination QEMU and only send <4 bytes to the channel as magic, it's possible that
migration_channel_read_peek()may spin forever.Fix it by adding a manual sleep for partial read.
Since the path isn't attached to a coroutine, it means when partial read happens, there's yet not much we can do but hang the main thread, it will happen even for len==0 case. It means monitors can hang due to this, either partial read or no data arrived (but connection established).
Leave this for later, the hope is this is extremely rare in production.
Quoted 7 lines
Resolves: https://gitlab.com/qemu-project/qemu/-/work_items/3889
Reported-by Feifan Qian <[email protected]>Cc Daniel P. Berrangé <[email protected]>Signed-off-by Peter Xu <[email protected]>--- migration/channel.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
Quoted 13 lines · Peter Xu
In an unlikely case, when a migration stream is attached to the destination QEMU and only send <4 bytes to the channel as magic, it's possible that
migration_channel_read_peek()may spin forever.Fix it by adding a manual sleep for partial read.
Since the path isn't attached to a coroutine, it means when partial read happens, there's yet not much we can do but hang the main thread, it will happen even for len==0 case. It means monitors can hang due to this, either partial read or no data arrived (but connection established).
Leave this for later, the hope is this is extremely rare in production.
Quoted 7 lines
Resolves: https://gitlab.com/qemu-project/qemu/-/work_items/3889
Reported-by Feifan Qian <[email protected]>Cc Daniel P. Berrangé <[email protected]>Signed-off-by Peter Xu <[email protected]>--- migration/channel.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)