migration: fix crash in when incoming client channel setup fails
PATCH v2 1/1 migration: fix crash in when incoming client channel setup fails
The way we determine if we can start the incoming migration was changed to use migration_has_all_channels() in:
commit 428d89084c709e568f9cd301c2f6416a54c53d6d
Author: Juan Quintela <[email protected]>
Date: Mon Jul 24 13:06:25 2017 +0200
migration: Create migration_has_all_channels
This method in turn calls multifd_recv_all_channels_created() which is hardcoded to always return 'true' when multifd is not in use. This is a latent bug...
...activated in in a following commit where that return result ends up acting as the flag to indicate whether it is possible to start processing the migration:
commit 36c2f8be2c4eb0003ac77a14910842b7ddd7337e
Author: Juan Quintela <[email protected]>
Date: Wed Mar 7 08:40:52 2018 +0100
migration: Delay start of migration main routines
This means that if channel initialization fails with normal migration, it'll never notice and attempt to start the incoming migration regardless and crash on a NULL pointer.
This can be seen, for example, if a client connects to a server requiring TLS, but has an invalid x509 certificate:
qemu-system-x86_64: The certificate hasn't got a known issuer
qemu-system-x86_64: migration/migration.c:386: process_incoming_migration_co: Assertion `mis->from_src_file' failed.
#0 0x00007fffebd24f2b in raise () at /lib64/libc.so.6 #1 0x00007fffebd0f561 in abort () at /lib64/libc.so.6 #2 0x00007fffebd0f431 in _nl_load_domain.cold.0 () at /lib64/libc.so.6 #3 0x00007fffebd1d692 in () at /lib64/libc.so.6 #4 0x0000555555ad027e in process_incoming_migration_co (opaque=<optimized out>) at migration/migration.c:386 #5 0x0000555555c45e8b in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at util/coroutine-ucontext.c:116 #6 0x00007fffebd3a6a0 in __start_context () at /lib64/libc.so.6 #7 0x0000000000000000 in ()
To handle the non-multifd case, we check whether mis->from_src_file is non-NULL. With this in place, the migration server drops the rejected client and stays around waiting for another, hopefully valid, client to arrive.
---
Changed in v2:
- Clarify exactly when it broke
- Clarify that we expect the server to stay active
migration/migration.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
migration/migration.c +2 -1
@@ -518,11 +518,12 @@ void migration_ioc_process_incoming(QIOChannel *ioc)*/ bool migration_has_all_channels (void ) { +MigrationIncomingState *mis =migration_incoming_get_current ();bool all_channels; all_channels =multifd_recv_all_channels_created (); -return all_channels; +return all_channels && mis->from_src_file !=NULL ; }/*
Signature · 1 line
2.17.0
Quoted 42 lines · * Daniel P. Berrangé ([email protected]) wrote:
The way we determine if we can start the incoming migration was changed to use
migration_has_all_channels()in:commit
428d89084c709e568f9cd301c2f6416a54c53d6d
Author: Juan Quintela <[email protected]>
Date: Mon Jul 24 13:06:25 2017 +0200migration: Create
migration_has_all_channelsThis method in turn calls
multifd_recv_all_channels_created()which is hardcoded to always return 'true' when multifd is not in use. This is a latent bug......activated in in a following commit where that return result ends up acting as the flag to indicate whether it is possible to start processing the migration:
commit
36c2f8be2c4eb0003ac77a14910842b7ddd7337e
Author: Juan Quintela <[email protected]>
Date: Wed Mar 7 08:40:52 2018 +0100migration: Delay start of migration main routines
This means that if channel initialization fails with normal migration, it'll never notice and attempt to start the incoming migration regardless and crash on a NULL pointer.
This can be seen, for example, if a client connects to a server requiring TLS, but has an invalid x509 certificate:
qemu-system-x86_64: The certificate hasn't got a known issuer
qemu-system-x86_64:migration/migration.c:386:process_incoming_migration_co: Assertion `mis->from_src_file' failed.#0 0x00007fffebd24f2b in raise () at /lib64/libc.so.6 #1 0x00007fffebd0f561 in abort () at /lib64/libc.so.6 #2 0x00007fffebd0f431 in _nl_load_domain.cold.0 () at /lib64/libc.so.6 #3 0x00007fffebd1d692 in () at /lib64/libc.so.6 #4 0x0000555555ad027e in process_incoming_migration_co (opaque=<optimized out>) at migration/migration.c:386 #5 0x0000555555c45e8b in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at util/coroutine-ucontext.c:116 #6 0x00007fffebd3a6a0 in __start_context () at /lib64/libc.so.6 #7 0x0000000000000000 in ()
Quoted 6 lines
To handle the non-multifd case, we check whether
mis->from_src_fileis non-NULL. With this in place, the migration server drops the rejected client and stays around waiting for another, hopefully valid, client to arrive.Signed-off-by Daniel P. Berrangé <[email protected]>
Quoted 31 lines
---
Changed in v2:
- Clarify exactly when it broke
- Clarify that we expect the server to stay activemigration/migration.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)migration/migration.c +2
-1@@ -518,11 +518,12 @@ void migration_ioc_process_incoming(QIOChannel *ioc)*/ bool migration_has_all_channels (void ) { +MigrationIncomingState *mis =migration_incoming_get_current ();bool all_channels; all_channels =multifd_recv_all_channels_created (); -return all_channels; +return all_channels && mis->from_src_file !=NULL ; }/* Signature · 1 line
2.17.0
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 7 lines · Daniel P. Berrangé
The way we determine if we can start the incoming migration was changed to use
migration_has_all_channels()in:commit
428d89084c709e568f9cd301c2f6416a54c53d6d
Author: Juan Quintela <[email protected]>
Date: Mon Jul 24 13:06:25 2017 +0200
Quoted 7 lines
migration: Create
migration_has_all_channelsThis method in turn calls
multifd_recv_all_channels_created()which is hardcoded to always return 'true' when multifd is not in use. This is a latent bug......activated in in a following commit where that return result
s/in in/in/
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Quoted 42 lines · Daniel P. Berrangé
The way we determine if we can start the incoming migration was changed to use
migration_has_all_channels()in:commit
428d89084c709e568f9cd301c2f6416a54c53d6d
Author: Juan Quintela <[email protected]>
Date: Mon Jul 24 13:06:25 2017 +0200migration: Create
migration_has_all_channelsThis method in turn calls
multifd_recv_all_channels_created()which is hardcoded to always return 'true' when multifd is not in use. This is a latent bug......activated in in a following commit where that return result ends up acting as the flag to indicate whether it is possible to start processing the migration:
commit
36c2f8be2c4eb0003ac77a14910842b7ddd7337e
Author: Juan Quintela <[email protected]>
Date: Wed Mar 7 08:40:52 2018 +0100migration: Delay start of migration main routines
This means that if channel initialization fails with normal migration, it'll never notice and attempt to start the incoming migration regardless and crash on a NULL pointer.
This can be seen, for example, if a client connects to a server requiring TLS, but has an invalid x509 certificate:
qemu-system-x86_64: The certificate hasn't got a known issuer
qemu-system-x86_64:migration/migration.c:386:process_incoming_migration_co: Assertion `mis->from_src_file' failed.#0 0x00007fffebd24f2b in raise () at /lib64/libc.so.6 #1 0x00007fffebd0f561 in abort () at /lib64/libc.so.6 #2 0x00007fffebd0f431 in _nl_load_domain.cold.0 () at /lib64/libc.so.6 #3 0x00007fffebd1d692 in () at /lib64/libc.so.6 #4 0x0000555555ad027e in process_incoming_migration_co (opaque=<optimized out>) at migration/migration.c:386 #5 0x0000555555c45e8b in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at util/coroutine-ucontext.c:116 #6 0x00007fffebd3a6a0 in __start_context () at /lib64/libc.so.6 #7 0x0000000000000000 in ()
Quoted 6 lines
To handle the non-multifd case, we check whether
mis->from_src_fileis non-NULL. With this in place, the migration server drops the rejected client and stays around waiting for another, hopefully valid, client to arrive.Signed-off-by Daniel P. Berrangé <[email protected]>