Auto exit source QEMU process after a successful migration
cover Auto exit source QEMU process after a successful migration
Hi all,
This serial include patches as below:
Patch 1:
- add a new shutdown cause 'migration-completed', which used for automatically exit of source QEMU process after a successful migration
Patch 2:
- add a new migration capability 'auto-quit' to control whether to automatically exit source QEMU process after a successful migration
Kunkun Jiang (2):
qapi/run-state: Add a new shutdown cause 'migration-completed' qapi/migration: Add a new migration capability 'auto-quit'
migration/migration.c | 13 +++++++++++++ migration/migration.h | 1 + qapi/migration.json | 6 +++++- qapi/run-state.json | 4 +++- 4 files changed, 22 insertions(+), 2 deletions(-)
Signature · 1 line
2.23.0
PATCH 1/2 qapi/run-state: Add a new shutdown cause 'migration-completed'
In the current version, the source QEMU process does not automatic exit after a successful migration. Additional action is required, such as sending { "execute": "quit" } or ctrl+c. For simplify, add a new shutdown cause 'migration-completed' to exit the source QEMU process after a successful migration.
--- migration/migration.c | 1 + qapi/run-state.json | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
migration/migration.c +1 -0
@@ -3539,6 +3539,7 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED );break ;case MIGRATION_STATUS_ACTIVE :
qapi/run-state.json +3 -1
@@ -86,12 +86,14 @@ # ignores --no-reboot. This is useful for sanitizing # hypercalls on s390 that are used during kexec/kdump/boot # +# @migration-completed: Reaction to the successful migration +# ## { 'enum': 'ShutdownCause', # Beware, shutdown_caused_by_guest() depends on enumeration order 'data': [ 'none', 'host-error', 'host-qmp-quit', 'host-qmp-system-reset', 'host-signal', 'host-ui', 'guest-shutdown', 'guest-reset', - 'guest-panic', 'subsystem-reset'] } + 'guest-panic', 'subsystem-reset', 'migration-completed'] } ## # @StatusInfo:
Signature · 1 line
2.23.0
Re: [PATCH 1/2] qapi/run-state: Add a new shutdown cause 'migration-completed'
Quoted 5 lines · Kunkun Jiang
In the current version, the source QEMU process does not automatic exit after a successful migration. Additional action is required, such as sending { "execute": "quit" } or ctrl+c. For simplify, add a new shutdown cause 'migration-completed' to exit the source QEMU process after a successful migration.
IIUC, 'STATUS_COMPLETED' state is entered on the source host once it has finished sending all VM state, and thus does not guarantee that the target host has successfully received and loaded all VM state.
Typically a mgmt app will need to directly confirm that the target host QEMU has succesfully started running, before it will tell the source QEMU to quit.
So, AFAICT, this automatic exit after STATUS_COMPLETED is not safe and could lead to total loss of the running VM in error scenarios.
Quoted 43 lines
Signed-off-by Kunkun Jiang <[email protected]>--- migration/migration.c | 1 + qapi/run-state.json | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)migration/migration.c +1
-0@@ -3539,6 +3539,7 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED );break ;case MIGRATION_STATUS_ACTIVE :qapi/run-state.json +3
-1@@ -86,12 +86,14 @@ # ignores --no-reboot. This is useful for sanitizing # hypercalls on s390 that are used during kexec/kdump/boot # +# @migration-completed: Reaction to the successful migration +# ## { 'enum': 'ShutdownCause', # Beware, shutdown_caused_by_guest() depends on enumeration order 'data': [ 'none', 'host-error', 'host-qmp-quit', 'host-qmp-system-reset', 'host-signal', 'host-ui', 'guest-shutdown', 'guest-reset', - 'guest-panic', 'subsystem-reset'] } + 'guest-panic', 'subsystem-reset', 'migration-completed'] } ## # @StatusInfo:Signature · 1 line
2.23.0
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 1/2] qapi/run-state: Add a new shutdown cause 'migration-completed'
Hi Daniel,
Quoted 10 lines · Daniel P. Berrangé
Quoted 5 lines · Kunkun Jiang
In the current version, the source QEMU process does not automatic exit after a successful migration. Additional action is required, such as sending { "execute": "quit" } or ctrl+c. For simplify, add a new shutdown cause 'migration-completed' to exit the source QEMU process after a successful migration.
IIUC, '
STATUS_COMPLETED' state is entered on the source host once it has finished sending all VM state, and thus does not guarantee that the target host has successfully received and loaded all VM state.
Thanks for your reply.
If the target host doesn't successfully receive and load all VM state, we can send { "execute": "cont" } to resume the soruce in time to ensure that VM will not lost?
Quoted 3 lines
Typically a mgmt app will need to directly confirm that the target host QEMU has succesfully started running, before it will tell the source QEMU to quit.
'a mgmt app', such as libvirt?
Thanks,
Kunkun Jiang
Quoted 50 lines
So, AFAICT, this automatic exit after
STATUS_COMPLETEDis not safe and could lead to total loss of the running VM in error scenarios.Quoted 42 lines
Signed-off-by Kunkun Jiang <[email protected]>--- migration/migration.c | 1 + qapi/run-state.json | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)migration/migration.c +1
-0@@ -3539,6 +3539,7 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED );break ;case MIGRATION_STATUS_ACTIVE :qapi/run-state.json +3
-1@@ -86,12 +86,14 @@ # ignores --no-reboot. This is useful for sanitizing # hypercalls on s390 that are used during kexec/kdump/boot # +# @migration-completed: Reaction to the successful migration +# ## { 'enum': 'ShutdownCause', # Beware, shutdown_caused_by_guest() depends on enumeration order 'data': [ 'none', 'host-error', 'host-qmp-quit', 'host-qmp-system-reset', 'host-signal', 'host-ui', 'guest-shutdown', 'guest-reset', - 'guest-panic', 'subsystem-reset'] } + 'guest-panic', 'subsystem-reset', 'migration-completed'] } ## # @StatusInfo:Signature · 1 line
2.23.0Regards,
Daniel
Re: [PATCH 1/2] qapi/run-state: Add a new shutdown cause 'migration-completed'
Quoted 13 lines · * Kunkun Jiang ([email protected]) wrote:
Hi Daniel,
Quoted 10 lines · Daniel P. Berrangé
Quoted 5 lines · Kunkun Jiang
In the current version, the source QEMU process does not automatic exit after a successful migration. Additional action is required, such as sending { "execute": "quit" } or ctrl+c. For simplify, add a new shutdown cause 'migration-completed' to exit the source QEMU process after a successful migration.
IIUC, '
STATUS_COMPLETED' state is entered on the source host once it has finished sending all VM state, and thus does not guarantee that the target host has successfully received and loaded all VM state.
Quoted 5 lines
Thanks for your reply.
If the target host doesn't successfully receive and load all VM state, we can send { "execute": "cont" } to resume the soruce in time to ensure that VM will not lost?
Yes, that's pretty common at the moment; the failed migration can happen at lots of different points:
a) The last part of the actual migration stream/loading the devices
- that's pretty easy, since the destination hasn't actually got the full migration stream.
b) If the migration itself completes, but then the management system then tries to reconfigure the networking/storage on the destination, and something goes wrong in that, then it can roll that back and cont on the source.
So, it's a pretty common type of failure/recovery - the management application has to be a bit careful not to do anything destructive until as late as possible, so it knows it can switch back.
Quoted 4 lines
Quoted 3 lines
Typically a mgmt app will need to directly confirm that the target host QEMU has succesfully started running, before it will tell the source QEMU to quit.
'a mgmt app', such as libvirt?
Yes, it's currently libvirt that does that; but any of the control things could (it's just libvirt has been going long enough so it knows about lots and lots of nasty cases of migration failure, and recovering properly).
Can you explain why did you want to get the source to automatically quit? In a real setup where does it help?
Dave
Quoted 52 lines
Thanks,
Kunkun JiangQuoted 48 lines
So, AFAICT, this automatic exit after
STATUS_COMPLETEDis not safe and could lead to total loss of the running VM in error scenarios.Quoted 40 lines
Signed-off-by Kunkun Jiang <[email protected]>--- migration/migration.c | 1 + qapi/run-state.json | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)migration/migration.c +1
-0@@ -3539,6 +3539,7 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED );break ;case MIGRATION_STATUS_ACTIVE :qapi/run-state.json +3
-1@@ -86,12 +86,14 @@ # ignores --no-reboot. This is useful for sanitizing # hypercalls on s390 that are used during kexec/kdump/boot # +# @migration-completed: Reaction to the successful migration +# ## { 'enum': 'ShutdownCause', # Beware, shutdown_caused_by_guest() depends on enumeration order 'data': [ 'none', 'host-error', 'host-qmp-quit', 'host-qmp-system-reset', 'host-signal', 'host-ui', 'guest-shutdown', 'guest-reset', - 'guest-panic', 'subsystem-reset'] } + 'guest-panic', 'subsystem-reset', 'migration-completed'] } ## # @StatusInfo:Signature · 1 line
2.23.0Regards,
Daniel
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Re: [PATCH 1/2] qapi/run-state: Add a new shutdown cause 'migration-completed'
Quoted 25 lines · Dr. David Alan Gilbert
Quoted 13 lines · * Kunkun Jiang ([email protected]) wrote:
Hi Daniel,
Quoted 10 lines · Daniel P. Berrangé
Quoted 5 lines · Kunkun Jiang
In the current version, the source QEMU process does not automatic exit after a successful migration. Additional action is required, such as sending { "execute": "quit" } or ctrl+c. For simplify, add a new shutdown cause 'migration-completed' to exit the source QEMU process after a successful migration.
IIUC, '
STATUS_COMPLETED' state is entered on the source host once it has finished sending all VM state, and thus does not guarantee that the target host has successfully received and loaded all VM state.Quoted 5 lines
Thanks for your reply.
If the target host doesn't successfully receive and load all VM state, we can send { "execute": "cont" } to resume the soruce in time to ensure that VM will not lost?
Yes, that's pretty common at the moment; the failed migration can happen at lots of different points:
a) The last part of the actual migration stream/loading the devices
- that's pretty easy, since the destination hasn't actually got the full migration stream.
Quoted 8 lines
b) If the migration itself completes, but then the management system then tries to reconfigure the networking/storage on the destination, and something goes wrong in that, then it can roll that back and cont on the source.
So, it's a pretty common type of failure/recovery - the management application has to be a bit careful not to do anything destructive until as late as possible, so it knows it can switch back.
Okay, I see.
Quoted 11 lines
Quoted 4 lines
Quoted 3 lines
Typically a mgmt app will need to directly confirm that the target host QEMU has succesfully started running, before it will tell the source QEMU to quit.
'a mgmt app', such as libvirt?
Yes, it's currently libvirt that does that; but any of the control things could (it's just libvirt has been going long enough so it knows about lots and lots of nasty cases of migration failure, and recovering properly).
Can you explain why did you want to get the source to automatically quit? In a real setup where does it help?
Sorry, my thoughts on live migration scenarios are not comprehensive enough.
Thanks,
Kunkun Jiang
Quoted 54 lines
Dave
Quoted 51 lines
Thanks,
Kunkun JiangQuoted 48 lines
So, AFAICT, this automatic exit after
STATUS_COMPLETEDis not safe and could lead to total loss of the running VM in error scenarios.Quoted 40 lines
Signed-off-by Kunkun Jiang <[email protected]>--- migration/migration.c | 1 + qapi/run-state.json | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)migration/migration.c +1
-0@@ -3539,6 +3539,7 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED );break ;case MIGRATION_STATUS_ACTIVE :qapi/run-state.json +3
-1@@ -86,12 +86,14 @@ # ignores --no-reboot. This is useful for sanitizing # hypercalls on s390 that are used during kexec/kdump/boot # +# @migration-completed: Reaction to the successful migration +# ## { 'enum': 'ShutdownCause', # Beware, shutdown_caused_by_guest() depends on enumeration order 'data': [ 'none', 'host-error', 'host-qmp-quit', 'host-qmp-system-reset', 'host-signal', 'host-ui', 'guest-shutdown', 'guest-reset', - 'guest-panic', 'subsystem-reset'] } + 'guest-panic', 'subsystem-reset', 'migration-completed'] } ## # @StatusInfo:Signature · 1 line
2.23.0Regards,
Daniel
Re: [PATCH 1/2] qapi/run-state: Add a new shutdown cause 'migration-completed'
Quoted 39 lines · * Kunkun Jiang ([email protected]) wrote:
Quoted 25 lines · Dr. David Alan Gilbert
Quoted 13 lines · * Kunkun Jiang ([email protected]) wrote:
Hi Daniel,
Quoted 10 lines · Daniel P. Berrangé
Quoted 5 lines · Kunkun Jiang
In the current version, the source QEMU process does not automatic exit after a successful migration. Additional action is required, such as sending { "execute": "quit" } or ctrl+c. For simplify, add a new shutdown cause 'migration-completed' to exit the source QEMU process after a successful migration.
IIUC, '
STATUS_COMPLETED' state is entered on the source host once it has finished sending all VM state, and thus does not guarantee that the target host has successfully received and loaded all VM state.Quoted 5 lines
Thanks for your reply.
If the target host doesn't successfully receive and load all VM state, we can send { "execute": "cont" } to resume the soruce in time to ensure that VM will not lost?
Yes, that's pretty common at the moment; the failed migration can happen at lots of different points:
a) The last part of the actual migration stream/loading the devices
- that's pretty easy, since the destination hasn't actually got the full migration stream.Quoted 8 lines
b) If the migration itself completes, but then the management system then tries to reconfigure the networking/storage on the destination, and something goes wrong in that, then it can roll that back and cont on the source.
So, it's a pretty common type of failure/recovery - the management application has to be a bit careful not to do anything destructive until as late as possible, so it knows it can switch back.
Okay, I see.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Typically a mgmt app will need to directly confirm that the target host QEMU has succesfully started running, before it will tell the source QEMU to quit.
'a mgmt app', such as libvirt?
Quoted 8 lines
Quoted 7 lines
Yes, it's currently libvirt that does that; but any of the control things could (it's just libvirt has been going long enough so it knows about lots and lots of nasty cases of migration failure, and recovering properly).
Can you explain why did you want to get the source to automatically quit? In a real setup where does it help?
Sorry, my thoughts on live migration scenarios are not comprehensive enough.
That's OK; it takes a little while to understand all of the recovery and error cases; people *really* want to recover from a failed migration; so we try and be very careful about not throwing away the source.
Dave
Quoted 57 lines
Thanks,
Kunkun JiangQuoted 54 lines
Dave
Quoted 51 lines
Thanks,
Kunkun JiangQuoted 48 lines
So, AFAICT, this automatic exit after
STATUS_COMPLETEDis not safe and could lead to total loss of the running VM in error scenarios.Quoted 40 lines
Signed-off-by Kunkun Jiang <[email protected]>--- migration/migration.c | 1 + qapi/run-state.json | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)migration/migration.c +1
-0@@ -3539,6 +3539,7 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED );break ;case MIGRATION_STATUS_ACTIVE :qapi/run-state.json +3
-1@@ -86,12 +86,14 @@ # ignores --no-reboot. This is useful for sanitizing # hypercalls on s390 that are used during kexec/kdump/boot # +# @migration-completed: Reaction to the successful migration +# ## { 'enum': 'ShutdownCause', # Beware, shutdown_caused_by_guest() depends on enumeration order 'data': [ 'none', 'host-error', 'host-qmp-quit', 'host-qmp-system-reset', 'host-signal', 'host-ui', 'guest-shutdown', 'guest-reset', - 'guest-panic', 'subsystem-reset'] } + 'guest-panic', 'subsystem-reset', 'migration-completed'] } ## # @StatusInfo:Signature · 1 line
2.23.0Regards,
Daniel
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
PATCH 2/2 qapi/migration: Add a new migration capability 'auto-quit'
In reply to Kunkun JiangFor compatibility, a new migration capability 'auto-quit' is added to control the exit of source QEMU after a successful migration.
--- migration/migration.c | 14 +++++++++++++- migration/migration.h | 1 + qapi/migration.json | 6 +++++- 3 files changed, 19 insertions(+), 2 deletions(-)
migration/migration.c +13 -1
@@ -2567,6 +2567,15 @@ bool migrate_background_snapshot(void)return s->enabled_capabilities[MIGRATION_CAPABILITY_BACKGROUND_SNAPSHOT ]; } +bool migrate_auto_quit (void ) +{ +MigrationState *s; + + s =migrate_get_current (); + +return s->enabled_capabilities[MIGRATION_CAPABILITY_AUTO_QUIT ]; +} +/* migration thread support */ /* * Something bad happened to the RP stream, mark an error
@@ -3539,7 +3548,10 @@ static void migration_iteration_finish(MigrationState *s)case MIGRATION_STATUS_COMPLETED :migration_calculate_complete (s);runstate_set (RUN_STATE_POSTMIGRATE ); -qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED ); + +if (migrate_auto_quit ()) { +qemu_system_shutdown_request (SHUTDOWN_CAUSE_MIGRATION_COMPLETED ); + }break ;case MIGRATION_STATUS_ACTIVE :
migration/migration.h +1 -0
@@ -349,6 +349,7 @@ int migrate_decompress_threads(void);bool migrate_use_events (void );bool migrate_postcopy_blocktime (void );bool migrate_background_snapshot (void ); +bool migrate_auto_quit (void );/* Sending on the return path - generic and then for each message type */ void migrate_send_rp_shut(MigrationIncomingState *mis,
qapi/migration.json +5 -1
@@ -452,6 +452,9 @@ # procedure starts. The VM RAM is saved with running VM. # (since 6.0) # +# @auto-quit: If enabled, QEMU process will exit after a successful migration. +# (since 6.1) +# # Since: 1.2 ## { 'enum': 'MigrationCapability',
@@ -459,7 +462,8 @@ 'compress', 'events', 'postcopy-ram', 'x-colo', 'release-ram', 'block', 'return-path', 'pause-before-switchover', 'multifd', 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate', - 'x-ignore-shared', 'validate-uuid', 'background-snapshot'] } + 'x-ignore-shared', 'validate-uuid', 'background-snapshot', + 'auto-quit'] } ## # @MigrationCapabilityStatus:
Signature · 1 line
2.23.0