migration: small cleanup series
v2 cover migration: small cleanup series
CI: https://gitlab.com/peterx/qemu/-/pipelines/2289719627
v1: https://lore.kernel.org/r/[email protected]
This is the v2 of the small cleanup series. It used to be only for COLO, but now for some other things too around dumping vmstates.
The small series growed a little bit, because I wanted to try Fabiano's request to deduplicate qemu_save_device_state() and precopy's similar version to dump non-iterable devices.
Changelog skipped, as too many things added. Old patches should almost the same with R-bs collected, though. I still touched up some small things, though (where I dropped the R-bs).
This should pass the new COLO unit test too, I hope I didn't overlook something once more. Please anyone let me know if it fails.
Comments welcomed, thanks.
Peter Xu (24):
migration: Introduce qemu_savevm_send_* helpers
migration: Use qemu_savevm_send_header() in qemu_save_device_state()
migration: Remove one migration_in_colo_state() occurance migration/savevm: Remove SaveStateEntry.is_ram migration/colo: Unwrap qemu_savevm_live_state()
migration: Remove call to send switchover start event in colo/savevm
colo: Forbid VM resume during checkpointing migration/colo: Use the RAM iterable helper directly migration/colo: Document qemu_fflush(fb)
migration: Drop iterable_only in qemu_savevm_state_complete_precopy
migration: Drop qemu_file_set_error() when save non-iterable fails migration/colo: Send device states without copying buffer migration/postcopy: Send device states without copying buffer
migration: Introduce qemu_savevm_state_end()
migration: Provide helper for save vm description
migration: Split qemu_savevm_state_complete_precopy_non_iterable()
migration: qemu_savevm_state_complete_precopy() take MigrationState*
migration: Cleanup error propagates in qemu_savevm_state_setup()
migration: Refactor qemu_savevm_state_setup()
migration: Introduce qemu_savevm_state_active() migration/bg-snapshot: Cleanup error paths
migration: Make qemu_savevm_state_non_iterable() take errp
migration: Simplify qemu_save_device_state() migration/colo/xen: Use generic helpers in qemu_save_device_state()
migration/savevm.h | 18 +-- migration/colo.c | 16 ++- migration/migration.c | 48 +++---- migration/savevm.c | 312 ++++++++++++++++++++++-------------------- monitor/qmp-cmds.c | 3 + 5 files changed, 213 insertions(+), 184 deletions(-)
Signature · 1 line
2.50.1
PATCH v2 1/24 migration: Introduce qemu_savevm_send_* helpers
Split qemu_savevm_state_header() into two parts. This paves way for a
reuse elsewhere.
--- migration/savevm.h | 1 + migration/savevm.c | 58 +++++++++++++++++++++++++++------------------- 2 files changed, 35 insertions(+), 24 deletions(-)
migration/savevm.h +1 -0
@@ -37,6 +37,7 @@ int qemu_savevm_state_prepare(Error **errp);int qemu_savevm_state_setup (QEMUFile *f,Error **errp);bool qemu_savevm_state_guest_unplug_pending (void );int qemu_savevm_state_resume_prepare (MigrationState *s); +void qemu_savevm_send_header (QEMUFile *f);void qemu_savevm_state_header (QEMUFile *f);int qemu_savevm_state_iterate (QEMUFile *f,bool postcopy);void qemu_savevm_state_cleanup (void );
migration/savevm.c +34 -24
72 lines not on this page · raw message
Signature · 1 line
2.50.1
PATCH v2 2/24 migration: Use qemu_savevm_send_header() in qemu_save_device_state()
In reply to Peter XuReduces duplication of the other path where we also send the same header.
--- migration/savevm.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
migration/savevm.c +1 -2
@@ -1872,8 +1872,7 @@ int qemu_save_device_state(QEMUFile *f)SaveStateEntry *se;if (!migration_in_colo_state ()) { -qemu_put_be32 (f,QEMU_VM_FILE_MAGIC ); -qemu_put_be32 (f,QEMU_VM_FILE_VERSION ); +qemu_savevm_send_header (f); }cpu_synchronize_all_states ();
Signature · 1 line
2.50.1
PATCH v2 3/24 migration: Remove one migration_in_colo_state() occurance
In reply to Peter XuMove the send header operation directly into Xen's QMP command, as COLO doesn't need it.
--- migration/savevm.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
migration/savevm.c +1 -3
@@ -1871,9 +1871,6 @@ int qemu_save_device_state(QEMUFile *f)Error *local_err =NULL ;SaveStateEntry *se; -if (!migration_in_colo_state ()) { -qemu_savevm_send_header (f); - }cpu_synchronize_all_states (); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {
@@ -3335,6 +3332,7 @@ void qmp_xen_save_devices_state(const char *filename, bool has_live, bool live,qio_channel_set_name (QIO_CHANNEL (ioc),"migration-xen-save-state" ); f =qemu_file_new_output (QIO_CHANNEL (ioc));object_unref (OBJECT (ioc)); +qemu_savevm_send_header (f); ret =qemu_save_device_state (f);if (ret <0 ||qemu_fclose (f) <0 ) {error_setg (errp,"saving Xen device state failed" );
Signature · 1 line
2.50.1
PATCH v2 4/24 migration/savevm: Remove SaveStateEntry.is_ram
In reply to Peter XuIt's neither accurate nor necessary. Use a proper helper to detect if it's an iterable savevm state entry instead.
--- migration/savevm.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
migration/savevm.c +8 -7
@@ -249,7 +249,6 @@ typedef struct SaveStateEntry {const VMStateDescription *vmsd;void *opaque;CompatEntry *compat; -int is_ram; } SaveStateEntry; typedef struct SaveState {
@@ -816,10 +815,6 @@ int register_savevm_live(const char *idstr, se->ops = ops; se->opaque = opaque; se->vmsd =NULL ; -/* if this is a live_savem then set is_ram */ -if (ops->save_setup !=NULL ) { - se->is_ram =1 ; - }pstrcat (se->idstr,sizeof (se->idstr), idstr);
@@ -1866,6 +1861,12 @@ void qemu_savevm_live_state(QEMUFile *f)qemu_put_byte (f,QEMU_VM_EOF ); } +/* Is a save state entry iterable (e.g. RAM)? */ +static bool qemu_savevm_se_iterable (SaveStateEntry *se) +{ +return se->ops && se->ops->save_setup; +} +int qemu_save_device_state (QEMUFile *f) {Error *local_err =NULL ;
@@ -1876,7 +1877,7 @@ int qemu_save_device_state(QEMUFile *f) QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {int ret; -if (se->is_ram) { +if (qemu_savevm_se_iterable (se)) {continue ; } ret =vmstate_save (f, se,NULL , &local_err);
@@ -2648,7 +2649,7 @@ qemu_loadvm_section_start_full(QEMUFile *f, uint8_t type, Error **errp) se->load_section_id = section_id;/* Validate if it is a device's state */ -if (xen_enabled () && se->is_ram) { +if (xen_enabled () &&qemu_savevm_se_iterable (se)) {error_setg (errp,"loadvm: %s RAM loading not allowed on Xen" , idstr);return -EINVAL ; }
Signature · 1 line
2.50.1
PATCH v2 5/24 migration/colo: Unwrap qemu_savevm_live_state()
In reply to Peter XuIt's only used in COLO path and only contains two calls. Unwrap the function. It paves way for further reduce special COLO paths on sync.
--- migration/savevm.h | 1 - migration/colo.c | 3 ++- migration/savevm.c | 7 ------- 3 files changed, 2 insertions(+), 9 deletions(-)
migration/savevm.h +0 -1
@@ -64,7 +64,6 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,uint64_t *start_list, uint64_t *length_list);void qemu_savevm_send_colo_enable (QEMUFile *f); -void qemu_savevm_live_state (QEMUFile *f);int qemu_save_device_state (QEMUFile *f);int qemu_loadvm_state (QEMUFile *f,Error **errp);
migration/colo.c +2 -1
@@ -471,7 +471,8 @@ static int colo_do_checkpoint_transaction(MigrationState *s,* TODO: We may need a timeout mechanism to prevent COLO process * to be blocked here. */ -qemu_savevm_live_state (s->to_dst_file); +qemu_savevm_state_complete_precopy (s->to_dst_file, true); +qemu_put_byte (s->to_dst_file,QEMU_VM_EOF );qemu_fflush (fb);
migration/savevm.c +0 -7
@@ -1854,13 +1854,6 @@ cleanup:return ret; } -void qemu_savevm_live_state (QEMUFile *f) -{ -/* save QEMU_VM_SECTION_END section */ -qemu_savevm_state_complete_precopy (f, true); -qemu_put_byte (f,QEMU_VM_EOF ); -} -/* Is a save state entry iterable (e.g. RAM)? */ static bool qemu_savevm_se_iterable (SaveStateEntry *se) {
Signature · 1 line
2.50.1
PATCH v2 6/24 migration: Remove call to send switchover start event in colo/savevm
In reply to Peter XuCOLO (in case of periodically checkpointing) already have switchover happened before hand. This switchover_start feature never applies to COLO.
Savevm for snapshot doesn't have switchover phase and VM is stopped for the whole process.
Remove both.
--- migration/colo.c | 2 -- migration/savevm.c | 1 - 2 files changed, 3 deletions(-)
migration/colo.c +0 -2
@@ -453,8 +453,6 @@ static int colo_do_checkpoint_transaction(MigrationState *s, goto out; } -qemu_savevm_maybe_send_switchover_start (s->to_dst_file); -/* Note: device state is saved into buffer */ ret =qemu_save_device_state (fb);
migration/savevm.c +0 -1
@@ -1830,7 +1830,6 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) ret =qemu_file_get_error (f);if (ret ==0 ) { -qemu_savevm_maybe_send_switchover_start (f);qemu_savevm_state_complete_precopy (f, false); ret =qemu_file_get_error (f); }
Signature · 1 line
2.50.1
Re: [PATCH v2 06/24] migration: Remove call to send switchover start event in colo/savevm
Quoted 9 lines · Peter Xu
COLO (in case of periodically checkpointing) already have switchover happened before hand. This
switchover_startfeature never applies to COLO.Savevm for snapshot doesn't have switchover phase and VM is stopped for the whole process.
Remove both.
Signed-off-by Peter Xu <[email protected]>
PATCH v2 7/24 colo: Forbid VM resume during checkpointing
In reply to Peter XuCOLO will stop the VM during each checkpoint on either PVM or SVM.
Accidentally resuming the VM during the window might be fatal because it may cause the RAM and devices state to misalign, corrupting the checkpoint.
Hence forbid VM resume during the process.
--- monitor/qmp-cmds.c | 3 +++ 1 file changed, 3 insertions(+)
monitor/qmp-cmds.c +3 -0
@@ -84,6 +84,9 @@ void qmp_cont(Error **errp) }else if (runstate_check (RUN_STATE_FINISH_MIGRATE )) {error_setg (errp,"Migration is not finalized yet" );return ; + }else if (runstate_check (RUN_STATE_COLO )) { +error_setg (errp,"COLO checkpoint in progress" ); +return ; }for (blk =blk_next (NULL ); blk; blk =blk_next (blk)) {
Signature · 1 line
2.50.1
PATCH v2 8/24 migration/colo: Use the RAM iterable helper directly
In reply to Peter Xuqemu_savevm_state_complete_precopy() has a weird parameter called "iterable_only". It's needed because COLO saves device states in advance.
To make dropping that weird parameter easier, let COLO directly use the RAM iterator helper instead, which should make the code easier to read too.
--- migration/colo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
migration/colo.c +1 -1
@@ -469,7 +469,7 @@ static int colo_do_checkpoint_transaction(MigrationState *s,* TODO: We may need a timeout mechanism to prevent COLO process * to be blocked here. */ -qemu_savevm_state_complete_precopy (s->to_dst_file, true); +qemu_savevm_state_complete_precopy_iterable (s->to_dst_file, false);qemu_put_byte (s->to_dst_file,QEMU_VM_EOF );qemu_fflush (fb);
Signature · 1 line
2.50.1
PATCH v2 9/24 migration/colo: Document qemu_fflush(fb)
In reply to Peter XuCOLO caches all device states in a buffer channel fb'. Add some comments explaining the flush, that (1) it's the fb' not the main channel, (2) on what it updates.
--- migration/colo.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
migration/colo.c +4 -2
@@ -472,12 +472,14 @@ static int colo_do_checkpoint_transaction(MigrationState *s,qemu_savevm_state_complete_precopy_iterable (s->to_dst_file, false);qemu_put_byte (s->to_dst_file,QEMU_VM_EOF ); -qemu_fflush (fb); -/* * We need the size of the VMstate data in Secondary side, * With which we can decide how much data should be read. +* +* Flush the qemufile cache to make sure both bioc->usage and +* bioc->data contains the latest info. */ +qemu_fflush (fb);colo_send_message_value (s->to_dst_file,COLO_MESSAGE_VMSTATE_SIZE , bioc->usage, &local_err);if (local_err) {
Signature · 1 line
2.50.1
Re: [PATCH v2 09/24] migration/colo: Document qemu_fflush(fb)
Quoted 5 lines · Peter Xu
COLO caches all device states in a buffer channel
fb'. Add some comments explaining the flush, that (1) it's thefb' not the main channel, (2) on what it updates.Signed-off-by Peter Xu <[email protected]>
PATCH v2 10/24 migration: Drop iterable_only in qemu_savevm_state_complete_precopy
In reply to Peter XuNow after removing the special case in COLO, we can drop this parameter.
--- migration/savevm.h | 2 +- migration/migration.c | 2 +- migration/savevm.c | 12 +++++------- 3 files changed, 7 insertions(+), 9 deletions(-)
migration/savevm.h +1 -1
@@ -42,7 +42,7 @@ void qemu_savevm_state_header(QEMUFile *f);int qemu_savevm_state_iterate (QEMUFile *f,bool postcopy);void qemu_savevm_state_cleanup (void );void qemu_savevm_state_complete_postcopy (QEMUFile *f); -int qemu_savevm_state_complete_precopy (QEMUFile *f,bool iterable_only); +int qemu_savevm_state_complete_precopy (QEMUFile *f);void qemu_savevm_state_pending_exact (uint64_t *must_precopy,uint64_t *can_postcopy); void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,
migration/migration.c +1 -1
@@ -2752,7 +2752,7 @@ static int migration_completion_precopy(MigrationState *s) goto out_unlock; } - ret =qemu_savevm_state_complete_precopy (s->to_dst_file, false); + ret =qemu_savevm_state_complete_precopy (s->to_dst_file); out_unlock:bql_unlock ();return ret;
migration/savevm.c +5 -7
@@ -1717,7 +1717,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,return 0 ; } -int qemu_savevm_state_complete_precopy (QEMUFile *f,bool iterable_only) +int qemu_savevm_state_complete_precopy (QEMUFile *f) {int ret;
@@ -1726,11 +1726,9 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f, bool iterable_only)return ret; } -if (!iterable_only) { - ret =qemu_savevm_state_complete_precopy_non_iterable (f, false); -if (ret) { -return ret; - } + ret =qemu_savevm_state_complete_precopy_non_iterable (f, false); +if (ret) { +return ret; }return qemu_fflush (f);
@@ -1830,7 +1828,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) ret =qemu_file_get_error (f);if (ret ==0 ) { -qemu_savevm_state_complete_precopy (f, false); +qemu_savevm_state_complete_precopy (f); ret =qemu_file_get_error (f); }if (ret !=0 ) {
Signature · 1 line
2.50.1
PATCH v2 11/24 migration: Drop qemu_file_set_error() when save non-iterable fails
In reply to Peter XuAll users of qemu_savevm_state_complete_precopy_non_iterable() process return values. There's no need to set error on qemufile (which we likely should remove gradually across the tree). Remove it for possible code dedup to happen later.
--- migration/savevm.c | 1 - 1 file changed, 1 deletion(-)
migration/savevm.c +0 -1
@@ -1688,7 +1688,6 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,if (ret) {migrate_error_propagate (ms,error_copy (local_err));error_report_err (local_err); -qemu_file_set_error (f, ret);return ret; }
Signature · 1 line
2.50.1
Re: [PATCH v2 11/24] migration: Drop qemu_file_set_error() when save non-iterable fails
Quoted 15 lines · Peter Xu
All users of
qemu_savevm_state_complete_precopy_non_iterable()process return values. There's no need to set error on qemufile (which we likely should remove gradually across the tree). Remove it for possible code dedup to happen later.Signed-off-by Peter Xu <[email protected]>--- migration/savevm.c | 1 - 1 file changed, 1 deletion(-)migration/savevm.c +0
-0
migration/savevm.c int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, · line 1692
Quoted 6 lines
migration/savevm.c +0
-1@@ -1688,7 +1688,6 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,if (ret) {migrate_error_propagate (ms,error_copy (local_err));error_report_err (local_err); -qemu_file_set_error (f, ret);return ret; }
PATCH v2 12/24 migration/colo: Send device states without copying buffer
In reply to Peter XuWe can safely use the async version of put buffer here because the qemufile will be flushed right away.
--- migration/colo.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
migration/colo.c +2 -1
@@ -486,7 +486,8 @@ static int colo_do_checkpoint_transaction(MigrationState *s, goto out; } -qemu_put_buffer (s->to_dst_file, bioc->data, bioc->usage); +/* We can use async put because flush happens right away */ +qemu_put_buffer_async (s->to_dst_file, bioc->data, bioc->usage, false); ret =qemu_fflush (s->to_dst_file);if (ret <0 ) { goto out;
Signature · 1 line
2.50.1
Re: [PATCH v2 12/24] migration/colo: Send device states without copying buffer
Quoted 14 lines · Peter Xu
We can safely use the async version of put buffer here because the qemufile will be flushed right away.
Suggested-by Fabiano Rosas <[email protected]>Signed-off-by Peter Xu <[email protected]>--- migration/colo.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)migration/colo.c +0
-0
migration/colo.c static int colo_do_checkpoint_transaction(MigrationState *s, · line 493
Quoted 9 lines
migration/colo.c +2
-1@@ -486,7 +486,8 @@ static int colo_do_checkpoint_transaction(MigrationState *s, goto out; } -qemu_put_buffer (s->to_dst_file, bioc->data, bioc->usage); +/* We can use async put because flush happens right away */ +qemu_put_buffer_async (s->to_dst_file, bioc->data, bioc->usage, false); ret =qemu_fflush (s->to_dst_file);if (ret <0 ) { goto out;
PATCH v2 13/24 migration/postcopy: Send device states without copying buffer
In reply to Peter XuPut buffer can be async as long as the flush happens before the buffer will be recycled / reused. Do it for postcopy package data. Quick measurement shows a small VM the time to push / flush the package shrinks from 91us to 38us.
--- migration/savevm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
migration/savevm.c +2 -1
@@ -1136,7 +1136,8 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len)trace_qemu_savevm_send_packaged ();qemu_savevm_command_send (f,MIG_CMD_PACKAGED ,4 , (uint8_t *)&tmp); -qemu_put_buffer (f, buf, len); +/* We can use async put because the qemufile will be flushed right away */ +qemu_put_buffer_async (f, buf, len, false);qemu_fflush (f);return 0 ;
Signature · 1 line
2.50.1
Re: [PATCH v2 13/24] migration/postcopy: Send device states without copying buffer
Quoted 15 lines · Peter Xu
Put buffer can be async as long as the flush happens before the buffer will be recycled / reused. Do it for postcopy package data. Quick measurement shows a small VM the time to push / flush the package shrinks from 91us to 38us.
Signed-off-by Peter Xu <[email protected]>--- migration/savevm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)migration/savevm.c +0
-0
migration/savevm.c int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len) · line 1143
Quoted 9 lines
migration/savevm.c +2
-1@@ -1136,7 +1136,8 @@ int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len)trace_qemu_savevm_send_packaged ();qemu_savevm_command_send (f,MIG_CMD_PACKAGED ,4 , (uint8_t *)&tmp); -qemu_put_buffer (f, buf, len); +/* We can use async put because the qemufile will be flushed right away */ +qemu_put_buffer_async (f, buf, len, false);qemu_fflush (f);return 0 ;
PATCH v2 14/24 migration: Introduce qemu_savevm_state_end()
In reply to Peter XuIntroduce a helper to end a migration stream.
--- migration/savevm.h | 1 + migration/colo.c | 2 +- migration/savevm.c | 12 +++++++++--- 3 files changed, 11 insertions(+), 4 deletions(-)
migration/savevm.h +1 -0
@@ -49,6 +49,7 @@ void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, uint64_t *can_postcopy);int qemu_savevm_state_complete_precopy_iterable (QEMUFile *f,bool in_postcopy);bool qemu_savevm_state_postcopy_prepare (QEMUFile *f,Error **errp); +void qemu_savevm_state_end (QEMUFile *f);void qemu_savevm_send_ping (QEMUFile *f,uint32_t value);void qemu_savevm_send_open_return_path (QEMUFile *f);int qemu_savevm_send_packaged (QEMUFile *f,const uint8_t *buf,size_t len);
migration/colo.c +1 -1
@@ -470,7 +470,7 @@ static int colo_do_checkpoint_transaction(MigrationState *s,* to be blocked here. */ qemu_savevm_state_complete_precopy_iterable (s->to_dst_file, false); -qemu_put_byte (s->to_dst_file,QEMU_VM_EOF ); +qemu_savevm_state_end (s->to_dst_file);/* * We need the size of the VMstate data in Secondary side,
migration/savevm.c +9 -3
@@ -1065,6 +1065,12 @@ static int vmstate_save(QEMUFile *f, SaveStateEntry *se, JSONWriter *vmdesc, }return 0 ; } + +void qemu_savevm_state_end (QEMUFile *f) +{ +qemu_put_byte (f,QEMU_VM_EOF ); +} +/** * qemu_savevm_command_send: Send a 'QEMU_VM_COMMAND' type element with the * command and associated data.
@@ -1555,7 +1561,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) } } -qemu_put_byte (f,QEMU_VM_EOF ); +qemu_savevm_state_end (f);qemu_fflush (f); }
@@ -1699,7 +1705,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,if (!in_postcopy) {/* Postcopy stream will still be going */ -qemu_put_byte (f,QEMU_VM_EOF ); +qemu_savevm_state_end (f);if (vmdesc) {json_writer_end_array (vmdesc);
@@ -1879,7 +1885,7 @@ int qemu_save_device_state(QEMUFile *f) } } -qemu_put_byte (f,QEMU_VM_EOF ); +qemu_savevm_state_end (f);return qemu_file_get_error (f); }
Signature · 1 line
2.50.1
Re: [PATCH v2 14/24] migration: Introduce qemu_savevm_state_end()
Quoted 70 lines · Peter Xu
Introduce a helper to end a migration stream.
Signed-off-by Peter Xu <[email protected]>--- migration/savevm.h | 1 + migration/colo.c | 2 +- migration/savevm.c | 12 +++++++++--- 3 files changed, 11 insertions(+), 4 deletions(-)migration/savevm.h +1
-0@@ -49,6 +49,7 @@ void qemu_savevm_state_pending_estimate(uint64_t *must_precopy, uint64_t *can_postcopy);int qemu_savevm_state_complete_precopy_iterable (QEMUFile *f,bool in_postcopy);bool qemu_savevm_state_postcopy_prepare (QEMUFile *f,Error **errp); +void qemu_savevm_state_end (QEMUFile *f);void qemu_savevm_send_ping (QEMUFile *f,uint32_t value);void qemu_savevm_send_open_return_path (QEMUFile *f);int qemu_savevm_send_packaged (QEMUFile *f,const uint8_t *buf,size_t len);migration/colo.c +1
-1@@ -470,7 +470,7 @@ static int colo_do_checkpoint_transaction(MigrationState *s,* to be blocked here. */ qemu_savevm_state_complete_precopy_iterable (s->to_dst_file, false); -qemu_put_byte (s->to_dst_file,QEMU_VM_EOF ); +qemu_savevm_state_end (s->to_dst_file);/* * We need the size of the VMstate data in Secondary side, migration/savevm.c +8
-2@@ -1065,6 +1065,12 @@ static int vmstate_save(QEMUFile *f, SaveStateEntry *se, JSONWriter *vmdesc, }return 0 ; } + +void qemu_savevm_state_end (QEMUFile *f) +{ +qemu_put_byte (f,QEMU_VM_EOF ); +} +/** * qemu_savevm_command_send: Send a 'QEMU_VM_COMMAND' type element with the * command and associated data. @@ -1555,7 +1561,7 @@ void qemu_savevm_state_complete_postcopy(QEMUFile *f) } } -qemu_put_byte (f,QEMU_VM_EOF ); +qemu_savevm_state_end (f);qemu_fflush (f); }@@ -1699,7 +1705,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,if (!in_postcopy) {/* Postcopy stream will still be going */ -qemu_put_byte (f,QEMU_VM_EOF ); +qemu_savevm_state_end (f);if (vmdesc) {json_writer_end_array (vmdesc);
migration/savevm.c int qemu_save_device_state(QEMUFile *f) · line 1891
Quoted 8 lines
migration/savevm.c +1
-1@@ -1879,7 +1885,7 @@ int qemu_save_device_state(QEMUFile *f) } } -qemu_put_byte (f,QEMU_VM_EOF ); +qemu_savevm_state_end (f);return qemu_file_get_error (f); }
PATCH v2 15/24 migration: Provide helper for save vm description
In reply to Peter XuProvide two smaller helpers to dump the vm desc. Preparing to move it out and generalize device state dump.
--- migration/savevm.h | 1 + migration/savevm.c | 35 +++++++++++++++++++++++------------ 2 files changed, 24 insertions(+), 12 deletions(-)
migration/savevm.h +1 -0
@@ -50,6 +50,7 @@ void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,int qemu_savevm_state_complete_precopy_iterable (QEMUFile *f,bool in_postcopy);bool qemu_savevm_state_postcopy_prepare (QEMUFile *f,Error **errp);void qemu_savevm_state_end (QEMUFile *f); +void qemu_savevm_state_end_precopy (MigrationState *s,QEMUFile *f);void qemu_savevm_send_ping (QEMUFile *f,uint32_t value);void qemu_savevm_send_open_return_path (QEMUFile *f);int qemu_savevm_send_packaged (QEMUFile *f,const uint8_t *buf,size_t len);
migration/savevm.c +23 -12
@@ -1669,13 +1669,34 @@ ret_fail_abort_threads:return -1 ; } +static void qemu_savevm_state_vm_desc (MigrationState *s,QEMUFile *f) +{ +JSONWriter *vmdesc = s->vmdesc; +int vmdesc_len; + +if (vmdesc) { +json_writer_end_array (vmdesc); +json_writer_end_object (vmdesc); + vmdesc_len =strlen (json_writer_get (vmdesc)); + +qemu_put_byte (f,QEMU_VM_VMDESCRIPTION ); +qemu_put_be32 (f, vmdesc_len); +qemu_put_buffer (f, (uint8_t *)json_writer_get (vmdesc), vmdesc_len); + } +} + +void qemu_savevm_state_end_precopy (MigrationState *s,QEMUFile *f) +{ +qemu_savevm_state_end (f); +qemu_savevm_state_vm_desc (s, f); +} +int qemu_savevm_state_complete_precopy_non_iterable (QEMUFile *f,bool in_postcopy) {MigrationState *ms =migrate_get_current ();int64_t start_ts_each, end_ts_each;JSONWriter *vmdesc = ms->vmdesc; -int vmdesc_len;SaveStateEntry *se;Error *local_err =NULL ;int ret;
@@ -1705,17 +1726,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,if (!in_postcopy) {/* Postcopy stream will still be going */ -qemu_savevm_state_end (f); - -if (vmdesc) { -json_writer_end_array (vmdesc); -json_writer_end_object (vmdesc); - vmdesc_len =strlen (json_writer_get (vmdesc)); - -qemu_put_byte (f,QEMU_VM_VMDESCRIPTION ); -qemu_put_be32 (f, vmdesc_len); -qemu_put_buffer (f, (uint8_t *)json_writer_get (vmdesc), vmdesc_len); - } +qemu_savevm_state_end_precopy (ms, f); }trace_vmstate_downtime_checkpoint ("src-non-iterable-saved" );
Signature · 1 line
2.50.1
Re: [PATCH v2 15/24] migration: Provide helper for save vm description
Quoted 70 lines · Peter Xu
Provide two smaller helpers to dump the vm desc. Preparing to move it out and generalize device state dump.
Signed-off-by Peter Xu <[email protected]>--- migration/savevm.h | 1 + migration/savevm.c | 35 +++++++++++++++++++++++------------ 2 files changed, 24 insertions(+), 12 deletions(-)migration/savevm.h +1
-0@@ -50,6 +50,7 @@ void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,int qemu_savevm_state_complete_precopy_iterable (QEMUFile *f,bool in_postcopy);bool qemu_savevm_state_postcopy_prepare (QEMUFile *f,Error **errp);void qemu_savevm_state_end (QEMUFile *f); +void qemu_savevm_state_end_precopy (MigrationState *s,QEMUFile *f);void qemu_savevm_send_ping (QEMUFile *f,uint32_t value);void qemu_savevm_send_open_return_path (QEMUFile *f);int qemu_savevm_send_packaged (QEMUFile *f,const uint8_t *buf,size_t len);migration/savevm.c +22
-6@@ -1669,13 +1669,34 @@ ret_fail_abort_threads:return -1 ; } +static void qemu_savevm_state_vm_desc (MigrationState *s,QEMUFile *f) +{ +JSONWriter *vmdesc = s->vmdesc; +int vmdesc_len; + +if (vmdesc) { +json_writer_end_array (vmdesc); +json_writer_end_object (vmdesc); + vmdesc_len =strlen (json_writer_get (vmdesc)); + +qemu_put_byte (f,QEMU_VM_VMDESCRIPTION ); +qemu_put_be32 (f, vmdesc_len); +qemu_put_buffer (f, (uint8_t *)json_writer_get (vmdesc), vmdesc_len); + } +} + +void qemu_savevm_state_end_precopy (MigrationState *s,QEMUFile *f) +{ +qemu_savevm_state_end (f); +qemu_savevm_state_vm_desc (s, f); +} +int qemu_savevm_state_complete_precopy_non_iterable (QEMUFile *f,bool in_postcopy) {MigrationState *ms =migrate_get_current ();int64_t start_ts_each, end_ts_each;JSONWriter *vmdesc = ms->vmdesc; -int vmdesc_len;SaveStateEntry *se;Error *local_err =NULL ;int ret;@@ -1705,17 +1726,7 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f,if (!in_postcopy) {/* Postcopy stream will still be going */ -qemu_savevm_state_end (f); - -if (vmdesc) { -json_writer_end_array (vmdesc); -json_writer_end_object (vmdesc);
migration/savevm.c int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, · line 1732
Quoted 10 lines
migration/savevm.c +1
-6- vmdesc_len =strlen (json_writer_get (vmdesc)); - -qemu_put_byte (f,QEMU_VM_VMDESCRIPTION ); -qemu_put_be32 (f, vmdesc_len); -qemu_put_buffer (f, (uint8_t *)json_writer_get (vmdesc), vmdesc_len); - } +qemu_savevm_state_end_precopy (ms, f); }trace_vmstate_downtime_checkpoint ("src-non-iterable-saved" );
PATCH v2 16/24 migration: Split qemu_savevm_state_complete_precopy_non_iterable()
In reply to Peter XuSplit the function, making itself to be the helper to dump all non-iterable device states (early_vmsd excluded). Move the precopy end logic out to the two callers that need it.
With it, we can remove the in_postcopy parameter. Meanwhile, renaming the function to be qemu_savevm_state_non_iterable(): we don't need the keyword "complete" because non-iterable doesn't iterate anyway, and we don't need precopy because we moved precopy specialties out.
NOTE: this patch introduced one new migrate_get_current() user; will be removed in follow up patch.
--- migration/savevm.h | 3 +-- migration/migration.c | 7 +++++-- migration/savevm.c | 12 ++++-------- 3 files changed, 10 insertions(+), 12 deletions(-)
migration/savevm.h +1 -2
@@ -74,8 +74,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, Error **errp);int qemu_load_device_state (QEMUFile *f,Error **errp);int qemu_loadvm_approve_switchover (void ); -int qemu_savevm_state_complete_precopy_non_iterable (QEMUFile *f, -bool in_postcopy); +int qemu_savevm_state_non_iterable (QEMUFile *f);bool qemu_loadvm_load_state_buffer (const char *idstr,uint32_t instance_id,char *buf,size_t len,Error **errp);
migration/migration.c +5 -2
@@ -2545,7 +2545,7 @@ static int postcopy_start(MigrationState *ms, Error **errp)*/ qemu_savevm_send_postcopy_listen (fb); - ret =qemu_savevm_state_complete_precopy_non_iterable (fb, true); + ret =qemu_savevm_state_non_iterable (fb);if (ret) {error_setg (errp,"Postcopy save non-iterable device states failed" ); goto fail_closefb;
@@ -3678,9 +3678,12 @@ static void *bg_migration_thread(void *opaque) goto fail; } -if (qemu_savevm_state_complete_precopy_non_iterable (fb, false)) { +if (qemu_savevm_state_non_iterable (fb)) { goto fail; } + +qemu_savevm_state_end_precopy (s, fb); +/* * Since we are going to get non-iterable state data directly * from s->bioc->data, explicit flush is needed here.
migration/savevm.c +4 -8
@@ -1691,8 +1691,7 @@ void qemu_savevm_state_end_precopy(MigrationState *s, QEMUFile *f)qemu_savevm_state_vm_desc (s, f); } -int qemu_savevm_state_complete_precopy_non_iterable (QEMUFile *f, -bool in_postcopy) +int qemu_savevm_state_non_iterable (QEMUFile *f) {MigrationState *ms =migrate_get_current ();int64_t start_ts_each, end_ts_each;
@@ -1724,11 +1723,6 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, end_ts_each - start_ts_each); } -if (!in_postcopy) { -/* Postcopy stream will still be going */ -qemu_savevm_state_end_precopy (ms, f); - } -trace_vmstate_downtime_checkpoint ("src-non-iterable-saved" );return 0 ;
@@ -1743,11 +1737,13 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f)return ret; } - ret =qemu_savevm_state_complete_precopy_non_iterable (f, false); + ret =qemu_savevm_state_non_iterable (f);if (ret) {return ret; } +qemu_savevm_state_end_precopy (migrate_get_current (), f); +return qemu_fflush (f); }
Signature · 1 line
2.50.1
Re: [PATCH v2 16/24] migration: Split qemu_savevm_state_complete_precopy_non_iterable()
Quoted 90 lines · Peter Xu
Split the function, making itself to be the helper to dump all non-iterable device states (
early_vmsdexcluded). Move the precopy end logic out to the two callers that need it.With it, we can remove the
in_postcopyparameter. Meanwhile, renaming the function to beqemu_savevm_state_non_iterable(): we don't need the keyword "complete" because non-iterable doesn't iterate anyway, and we don't need precopy because we moved precopy specialties out.NOTE: this patch introduced one new
migrate_get_current()user; will be removed in follow up patch.Signed-off-by Peter Xu <[email protected]>--- migration/savevm.h | 3 +-- migration/migration.c | 7 +++++-- migration/savevm.c | 12 ++++-------- 3 files changed, 10 insertions(+), 12 deletions(-)migration/savevm.h +1
-2@@ -74,8 +74,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, Error **errp);int qemu_load_device_state (QEMUFile *f,Error **errp);int qemu_loadvm_approve_switchover (void ); -int qemu_savevm_state_complete_precopy_non_iterable (QEMUFile *f, -bool in_postcopy); +int qemu_savevm_state_non_iterable (QEMUFile *f);bool qemu_loadvm_load_state_buffer (const char *idstr,uint32_t instance_id,char *buf,size_t len,Error **errp);migration/migration.c +5
-2@@ -2545,7 +2545,7 @@ static int postcopy_start(MigrationState *ms, Error **errp)*/ qemu_savevm_send_postcopy_listen (fb); - ret =qemu_savevm_state_complete_precopy_non_iterable (fb, true); + ret =qemu_savevm_state_non_iterable (fb);if (ret) {error_setg (errp,"Postcopy save non-iterable device states failed" ); goto fail_closefb;@@ -3678,9 +3678,12 @@ static void *bg_migration_thread(void *opaque) goto fail; } -if (qemu_savevm_state_complete_precopy_non_iterable (fb, false)) { +if (qemu_savevm_state_non_iterable (fb)) { goto fail; } + +qemu_savevm_state_end_precopy (s, fb); +/* * Since we are going to get non-iterable state data directly * from s->bioc->data, explicit flush is needed here. migration/savevm.c +1
-7@@ -1691,8 +1691,7 @@ void qemu_savevm_state_end_precopy(MigrationState *s, QEMUFile *f)qemu_savevm_state_vm_desc (s, f); } -int qemu_savevm_state_complete_precopy_non_iterable (QEMUFile *f, -bool in_postcopy) +int qemu_savevm_state_non_iterable (QEMUFile *f) {MigrationState *ms =migrate_get_current ();int64_t start_ts_each, end_ts_each;@@ -1724,11 +1723,6 @@ int qemu_savevm_state_complete_precopy_non_iterable(QEMUFile *f, end_ts_each - start_ts_each); } -if (!in_postcopy) { -/* Postcopy stream will still be going */ -qemu_savevm_state_end_precopy (ms, f); - } -trace_vmstate_downtime_checkpoint ("src-non-iterable-saved" );return 0 ;@@ -1743,11 +1737,13 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f)return ret; }
migration/savevm.c int qemu_savevm_state_complete_precopy(QEMUFile *f) · line 1748
Quoted 10 lines
migration/savevm.c +3
-1- ret =qemu_savevm_state_complete_precopy_non_iterable (f, false); + ret =qemu_savevm_state_non_iterable (f);if (ret) {return ret; } +qemu_savevm_state_end_precopy (migrate_get_current (), f); +return qemu_fflush (f); }
PATCH v2 17/24 migration: qemu_savevm_state_complete_precopy() take MigrationState*
In reply to Peter XuMake it pass in MigrationState* instead of s->to_dst_file, so as to drop the internal migrate_get_current().
--- migration/savevm.h | 2 +- migration/migration.c | 2 +- migration/savevm.c | 7 ++++--- 3 files changed, 6 insertions(+), 5 deletions(-)
migration/savevm.h +1 -1
@@ -42,7 +42,7 @@ void qemu_savevm_state_header(QEMUFile *f);int qemu_savevm_state_iterate (QEMUFile *f,bool postcopy);void qemu_savevm_state_cleanup (void );void qemu_savevm_state_complete_postcopy (QEMUFile *f); -int qemu_savevm_state_complete_precopy (QEMUFile *f); +int qemu_savevm_state_complete_precopy (MigrationState *s);void qemu_savevm_state_pending_exact (uint64_t *must_precopy,uint64_t *can_postcopy); void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,
migration/migration.c +1 -1
@@ -2752,7 +2752,7 @@ static int migration_completion_precopy(MigrationState *s) goto out_unlock; } - ret =qemu_savevm_state_complete_precopy (s->to_dst_file); + ret =qemu_savevm_state_complete_precopy (s); out_unlock:bql_unlock ();return ret;
migration/savevm.c +4 -3
@@ -1728,8 +1728,9 @@ int qemu_savevm_state_non_iterable(QEMUFile *f)return 0 ; } -int qemu_savevm_state_complete_precopy (QEMUFile *f) +int qemu_savevm_state_complete_precopy (MigrationState *s) { +QEMUFile *f = s->to_dst_file;int ret; ret =qemu_savevm_state_complete_precopy_iterable (f, false);
@@ -1742,7 +1743,7 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f)return ret; } -qemu_savevm_state_end_precopy (migrate_get_current (), f); +qemu_savevm_state_end_precopy (s, f);return qemu_fflush (f); }
@@ -1841,7 +1842,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) ret =qemu_file_get_error (f);if (ret ==0 ) { -qemu_savevm_state_complete_precopy (f); +qemu_savevm_state_complete_precopy (ms); ret =qemu_file_get_error (f); }if (ret !=0 ) {
Signature · 1 line
2.50.1
Re: [PATCH v2 17/24] migration: qemu_savevm_state_complete_precopy() take MigrationState*
Quoted 61 lines · Peter Xu
Make it pass in MigrationState* instead of
s->to_dst_file, so as to drop the internalmigrate_get_current().Signed-off-by Peter Xu <[email protected]>--- migration/savevm.h | 2 +- migration/migration.c | 2 +- migration/savevm.c | 7 ++++--- 3 files changed, 6 insertions(+), 5 deletions(-)migration/savevm.h +1
-1@@ -42,7 +42,7 @@ void qemu_savevm_state_header(QEMUFile *f);int qemu_savevm_state_iterate (QEMUFile *f,bool postcopy);void qemu_savevm_state_cleanup (void );void qemu_savevm_state_complete_postcopy (QEMUFile *f); -int qemu_savevm_state_complete_precopy (QEMUFile *f); +int qemu_savevm_state_complete_precopy (MigrationState *s);void qemu_savevm_state_pending_exact (uint64_t *must_precopy,uint64_t *can_postcopy); void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,migration/migration.c +1
-1@@ -2752,7 +2752,7 @@ static int migration_completion_precopy(MigrationState *s) goto out_unlock; } - ret =qemu_savevm_state_complete_precopy (s->to_dst_file); + ret =qemu_savevm_state_complete_precopy (s); out_unlock:bql_unlock ();return ret;migration/savevm.c +3
-2@@ -1728,8 +1728,9 @@ int qemu_savevm_state_non_iterable(QEMUFile *f)return 0 ; } -int qemu_savevm_state_complete_precopy (QEMUFile *f) +int qemu_savevm_state_complete_precopy (MigrationState *s) { +QEMUFile *f = s->to_dst_file;int ret; ret =qemu_savevm_state_complete_precopy_iterable (f, false);@@ -1742,7 +1743,7 @@ int qemu_savevm_state_complete_precopy(QEMUFile *f)return ret; } -qemu_savevm_state_end_precopy (migrate_get_current (), f); +qemu_savevm_state_end_precopy (s, f);return qemu_fflush (f); }
migration/savevm.c static int qemu_savevm_state(QEMUFile *f, Error **errp) · line 1848
Quoted 8 lines
migration/savevm.c +1
-1@@ -1841,7 +1842,7 @@ static int qemu_savevm_state(QEMUFile *f, Error **errp) ret =qemu_file_get_error (f);if (ret ==0 ) { -qemu_savevm_state_complete_precopy (f); +qemu_savevm_state_complete_precopy (ms); ret =qemu_file_get_error (f); }if (ret !=0 ) {
PATCH v2 18/24 migration: Cleanup error propagates in qemu_savevm_state_setup()
In reply to Peter XuWe did two unnecessary error propagations in qemu_savevm_state_setup(), on either propagate it to MigrationState*, or set qemufile with error.
Error propagation is not needed because:
- Two live migration callers ([bg_]migration_thread) will propagate error if this function returned with an error.
- Save snapshot (qemu_savevm_state) doesn't need to persist error; it got returned directly from save_snapshot().
QEMUFile set error is not needed because the callers always check for errors explicitly.
--- migration/savevm.c | 3 --- 1 file changed, 3 deletions(-)
migration/savevm.c +0 -3
@@ -1385,8 +1385,6 @@ int qemu_savevm_state_setup(QEMUFile *f, Error **errp)if (se->vmsd && se->vmsd->early_setup) { ret =vmstate_save (f, se, vmdesc, errp);if (ret) { -migrate_error_propagate (ms,error_copy (*errp)); -qemu_file_set_error (f, ret);break ; }continue ;
@@ -1405,7 +1403,6 @@ int qemu_savevm_state_setup(QEMUFile *f, Error **errp) ret = se->ops->save_setup (f, se->opaque, errp);save_section_footer (f, se);if (ret <0 ) { -qemu_file_set_error (f, ret);break ; } }
Signature · 1 line
2.50.1
Re: [PATCH v2 18/24] migration: Cleanup error propagates in qemu_savevm_state_setup()
Quoted 5 lines · Peter Xu
We did two unnecessary error propagations in
qemu_savevm_state_setup(), on either propagate it to MigrationState*, or set qemufile with error.Error propagation is not needed because:
Quoted 9 lines
- Two live migration callers ([bg_]migration_thread) will propagate error if this function returned with an error.
- Save snapshot (
qemu_savevm_state) doesn't need to persist error; it got returned directly fromsave_snapshot().QEMUFile set error is not needed because the callers always check for errors explicitly.
Nice
Quoted 26 lines
Signed-off-by Peter Xu <[email protected]>--- migration/savevm.c | 3 --- 1 file changed, 3 deletions(-)migration/savevm.c +0
-3@@ -1385,8 +1385,6 @@ int qemu_savevm_state_setup(QEMUFile *f, Error **errp)if (se->vmsd && se->vmsd->early_setup) { ret =vmstate_save (f, se, vmdesc, errp);if (ret) { -migrate_error_propagate (ms,error_copy (*errp)); -qemu_file_set_error (f, ret);break ; }continue ;@@ -1405,7 +1403,6 @@ int qemu_savevm_state_setup(QEMUFile *f, Error **errp) ret = se->ops->save_setup (f, se->opaque, errp);save_section_footer (f, se);if (ret <0 ) { -qemu_file_set_error (f, ret);break ; } }
PATCH v2 19/24 migration: Refactor qemu_savevm_state_setup()
In reply to Peter XuSplit it into two smaller chunks:
- Dump of early_setup VMSDs
- Dump of save_setup() sections
They're mutual exclusive, hence we can run two loops and do them sequentially. This will cause migration thread to loop one more time, but it should be fine when migration just started and only do it once. It's needed because we will need to reuse the early_vmsd helper later to deduplicate code elsewhere.
QEMU almost sticks with qemu_savevm_state_XXX() to represent the dump of vmstates's section XXX. With that in mind, this patch renamed the original qemu_savevm_state_setup() to qemu_savevm_state_do_setup() instead.
So after this patch:
- qemu_savevm_state_non_iterable_early() dumps early_vmsds only,
- qemu_savevm_state_setup() dumps save_setup() sections only,
- qemu_savevm_state_do_setup() does all things needed during setup phase (including migration SETUP notifies)
--- migration/savevm.h | 6 +++-- migration/migration.c | 4 ++-- migration/savevm.c | 56 ++++++++++++++++++++++++++++++------------- 3 files changed, 46 insertions(+), 20 deletions(-)
migration/savevm.h +4 -2
@@ -34,7 +34,7 @@bool qemu_savevm_state_blocked (Error **errp);void qemu_savevm_non_migratable_list (strList **reasons);int qemu_savevm_state_prepare (Error **errp); -int qemu_savevm_state_setup (QEMUFile *f,Error **errp); +int qemu_savevm_state_do_setup (QEMUFile *f,Error **errp);bool qemu_savevm_state_guest_unplug_pending (void );int qemu_savevm_state_resume_prepare (MigrationState *s);void qemu_savevm_send_header (QEMUFile *f);
@@ -75,7 +75,9 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,int qemu_load_device_state (QEMUFile *f,Error **errp);int qemu_loadvm_approve_switchover (void );int qemu_savevm_state_non_iterable (QEMUFile *f); - +int qemu_savevm_state_non_iterable_early (QEMUFile *f, +JSONWriter *vmdesc, +Error **errp);bool qemu_loadvm_load_state_buffer (const char *idstr,uint32_t instance_id,char *buf,size_t len,Error **errp);
migration/migration.c +2 -2
@@ -3528,7 +3528,7 @@ static void *migration_thread(void *opaque) }bql_lock (); - ret =qemu_savevm_state_setup (s->to_dst_file, &local_err); + ret =qemu_savevm_state_do_setup (s->to_dst_file, &local_err);bql_unlock (); qemu_savevm_wait_unplug(s, MIGRATION_STATUS_SETUP,
@@ -3651,7 +3651,7 @@ static void *bg_migration_thread(void *opaque)bql_lock ();qemu_savevm_state_header (s->to_dst_file); - ret =qemu_savevm_state_setup (s->to_dst_file, &local_err); + ret =qemu_savevm_state_do_setup (s->to_dst_file, &local_err);bql_unlock (); qemu_savevm_wait_unplug(s, MIGRATION_STATUS_SETUP,
migration/savevm.c +40 -16
90 lines not on this page · raw message
Signature · 1 line
2.50.1
Re: [PATCH v2 19/24] migration: Refactor qemu_savevm_state_setup()
Quoted 16 lines · Peter Xu
Split it into two smaller chunks:
- Dump of
early_setupVMSDs
- Dump ofsave_setup()sectionsThey're mutual exclusive, hence we can run two loops and do them sequentially. This will cause migration thread to loop one more time, but it should be fine when migration just started and only do it once. It's needed because we will need to reuse the
early_vmsdhelper later to deduplicate code elsewhere.QEMU almost sticks with
qemu_savevm_state_XXX()to represent the dump of vmstates's section XXX.
With that in mind, this patch renamed the originalqemu_savevm_state_setup()toqemu_savevm_state_do_setup()instead.
Quoted 8 lines
So after this patch:
-
qemu_savevm_state_non_iterable_early()dumpsearly_vmsdsonly,
-qemu_savevm_state_setup()dumpssave_setup()sections only,
-qemu_savevm_state_do_setup()does all things needed during setup phase (including migration SETUP notifies)Signed-off-by Peter Xu <[email protected]>
Not for this series, but I think we could try to standardize the naming in savevm.c a bit. Here's a list of functions from savevm.c plus the main function called by them. You'll see that there's room from improvement.
Removing this "qemu" prefix and choosing when to put "state" in the name would already be a win.
Also maybe making more clear what calls vmstate code and what deals with se->ops only.
---
qemu_loadvm_state_setup { se->ops->load_setup }
qemu_savevm_state_active { se->ops->is_active }
qemu_savevm_state_blocked { se->vmsd->unmigratable }
qemu_savevm_state_cleanup { se->ops->save_cleanup }
qemu_savevm_state_guest_unplug_pending { se->vmsd->dev_unplug_pending }
qemu_savevm_state_iterate { se->ops->save_live_iterate }
qemu_savevm_state_non_iterable_early { se->vmsd->early_setup; vmstate_save }
qemu_savevm_state_pending_estimate { se->ops->state_pending_estimate }
qemu_savevm_state_pending_exact { se->ops->state_pending_exact }
qemu_savevm_state_postcopy_prepare { se->ops->save_postcopy_prepare }
qemu_savevm_state_prepare { se->ops->save_prepare }
qemu_savevm_state_resume_prepare { se->ops->resume_prepare }
loadvm_postcopy_handle_switchover_start { se->ops->switchover_start }
qemu_loadvm_load_state_buffer { se->ops->load_state_buffer }
qemu_loadvm_state_switchover_ack_needed { se->ops->switchover_ack_needed }
qemu_savevm_complete { se->ops->save_complete }
qemu_savevm_complete_exists { se->ops->save_complete }
qemu_savevm_non_migratable_list { se->vmsd->unmigratable }
qemu_savevm_se_iterable { se->ops->save_setup }
save_state_priority { se->vmsd->priority }
vmstate_load { se->ops->load_state }
vmstate_save_old_style { se->ops->save_state }loadvm_handle_cmd_packaged { qemu_loadvm_state_main }
qemu_load_device_state { qemu_loadvm_state_main }
qemu_savevm_maybe_send_switchover_start { qemu_savevm_send_switchover_start }
qemu_savevm_send_* { qemu_savevm_command_send }
qemu_savevm_state_complete_postcopy { qemu_savevm_complete }qemu_savevm_state_complete_precopy_iterable { qemu_savevm_complete }qemu_savevm_state_end_precopy { qemu_savevm_state_end }qemu_savevm_state_header { qemu_savevm_send_header }
qemu_savevm_state_do_setup { qemu_savevm_state_non_iterable_early;
qemu_savevm_state_setup }qemu_loadvm_state { qemu_loadvm_state_header;
qemu_loadvm_state_setup;
qemu_loadvm_state_switchover_ack_needed;
qemu_loadvm_state_main;
qemu_loadvm_thread_pool_wait }qemu_loadvm_state_main { qemu_loadvm_section_start_full;
qemu_loadvm_section_part_end;
loadvm_process_command }qemu_savevm_state { qemu_savevm_state_header;
qemu_savevm_state_do_setup;
qemu_savevm_state_iterate;
qemu_savevm_state_complete_precopy;
qemu_savevm_state_cleanup }qemu_savevm_state_complete_precopy { qemu_savevm_state_complete_precopy_iterable;
qemu_savevm_state_non_iterable;
qemu_savevm_state_end_precopy }qemu_save_device_state { qemu_savevm_state_non_iterable_early;
qemu_savevm_state_non_iterable;
qemu_savevm_state_end }loadvm_handle_recv_bitmap { migrate_send_rp_recv_bitmap }
loadvm_postcopy_handle_advise { ram_postcopy_incoming_init }
loadvm_postcopy_handle_listen { postcopy_incoming_setup }
loadvm_postcopy_handle_resume { mis->postcopy_pause_sem_fault }
loadvm_postcopy_handle_run { loadvm_postcopy_handle_run_bh }
loadvm_postcopy_handle_run_bh { vm_start }
loadvm_postcopy_ram_handle_discard { postcopy_ram_prepare_discard }
loadvm_process_command { switch (cmd) }
loadvm_process_enable_colo { colo_init_ram_cache }
qemu_loadvm_approve_switchover { migrate_send_rp_switchover_ack }
qemu_loadvm_load_thread { data->function }
qemu_loadvm_section_part_end { vmstate_load }
qemu_loadvm_section_start_full { vmstate_load }
qemu_loadvm_start_load_thread { thread_pool_submit }
qemu_loadvm_state_cleanup { se->ops->load_cleanup }
qemu_loadvm_state_header { QEMU_VM_FILE_MAGIC }
qemu_loadvm_thread_pool_create { thread_pool_new }
qemu_loadvm_thread_pool_destroy { thread_pool_free }
qemu_loadvm_thread_pool_wait { thread_pool_wait }
qemu_savevm_command_send { QEMU_VM_COMMAND }
qemu_savevm_send_colo_enable { MIG_CMD_ENABLE_COLO }
qemu_savevm_send_configuration { vmstate_save_state(&vmstate_configuration) }
qemu_savevm_send_header { QEMU_VM_FILE_MAGIC }
qemu_savevm_state_end { QEMU_VM_EOF }
qemu_savevm_state_non_iterable { vmstate_save }
qemu_savevm_state_setup { se->ops->save_setup }
qemu_savevm_state_vm_desc { QEMU_VM_VMDESCRIPTION }
register_savevm_live { savevm_state_handler_insert }
save_section_footer { QEMU_VM_SECTION_FOOTER }
save_section_header { section_type }
savevm_state_handler_insert { QTAILQ_INSERT_TAIL }
savevm_state_handler_remove { QTAILQ_REMOVE }
unregister_savevm { savevm_state_handler_remove }
vmstate_register_with_alias_id { savevm_state_handler_insert }
vmstate_save { vmstate_save_old_style; vmstate_save_state }
vmstate_unregister { savevm_state_handler_remove }Re: [PATCH v2 19/24] migration: Refactor qemu_savevm_state_setup()
Quoted 29 lines · Fabiano Rosas
Quoted 16 lines · Peter Xu
Split it into two smaller chunks:
- Dump of
early_setupVMSDs
- Dump ofsave_setup()sectionsThey're mutual exclusive, hence we can run two loops and do them sequentially. This will cause migration thread to loop one more time, but it should be fine when migration just started and only do it once. It's needed because we will need to reuse the
early_vmsdhelper later to deduplicate code elsewhere.QEMU almost sticks with
qemu_savevm_state_XXX()to represent the dump of vmstates's section XXX.
With that in mind, this patch renamed the originalqemu_savevm_state_setup()toqemu_savevm_state_do_setup()instead.Quoted 8 lines
So after this patch:
-
qemu_savevm_state_non_iterable_early()dumpsearly_vmsdsonly,
-qemu_savevm_state_setup()dumpssave_setup()sections only,
-qemu_savevm_state_do_setup()does all things needed during setup phase (including migration SETUP notifies)Signed-off-by Peter Xu <[email protected]>Reviewed-by Fabiano Rosas <[email protected]>
Quoted 7 lines
Not for this series, but I think we could try to standardize the naming in
savevm.ca bit. Here's a list of functions fromsavevm.cplus the main function called by them. You'll see that there's room from improvement.Removing this "qemu" prefix and choosing when to put "state" in the name would already be a win.
True.
IMHO if we choose it again we can use prefix vmstate_. The current savevm_state_ is pretty vague on its own when loadvm needs to use it. Then it can be vmstate_save_* on save side, and vmstate_load_* on load side (and vmstate_XXX) when valid on both.
Quoted 3 lines
Also maybe making more clear what calls vmstate code and what deals with
se->opsonly.
Yes we can try, but there'll be some tricky ones, inline below (assuming we can shorten the prefix to "vmstate_").
Quoted 3 lines
---
qemu_loadvm_state_setup{se->ops->load_setup}
Taking this one as example.
We could rename this to vmstate_ops_load_setup(), however "load_setup" is actually a concept that is higher than the impl. IOW it'll be also reasonable if we want to add a load_setup() hook to VMSDs some day. Maybe still good to rename it to vmstate_load_setup().
Quoted 1 line
qemu_savevm_state_active{se->ops->is_active}
Similarly, here I'd think vmstate_active() (it applies to both save/load) better than vmstate_ops_active() because active describes the vmstate instance, and it's fine one day we want to introduce VMSD.active.
...
Quoted 11 lines
qemu_savevm_state_blocked { se->vmsd->unmigratable }
qemu_savevm_state_cleanup { se->ops->save_cleanup }
qemu_savevm_state_guest_unplug_pending { se->vmsd->dev_unplug_pending }
qemu_savevm_state_iterate { se->ops->save_live_iterate }
qemu_savevm_state_non_iterable_early { se->vmsd->early_setup; vmstate_save }
qemu_savevm_state_pending_estimate { se->ops->state_pending_estimate }
qemu_savevm_state_pending_exact { se->ops->state_pending_exact }
qemu_savevm_state_postcopy_prepare { se->ops->save_postcopy_prepare }
qemu_savevm_state_prepare { se->ops->save_prepare }
qemu_savevm_state_resume_prepare { se->ops->resume_prepare }
loadvm_postcopy_handle_switchover_start { se->ops->switchover_start }Quoted 8 lines
qemu_loadvm_load_state_buffer { se->ops->load_state_buffer }
qemu_loadvm_state_switchover_ack_needed { se->ops->switchover_ack_needed }
qemu_savevm_complete { se->ops->save_complete }
qemu_savevm_complete_exists { se->ops->save_complete }
qemu_savevm_non_migratable_list { se->vmsd->unmigratable }
qemu_savevm_se_iterable { se->ops->save_setup }
save_state_priority { se->vmsd->priority }
vmstate_load { se->ops->load_state }This one (and vmstate_save()) is special because it handles both impls (VMSD and OPS). The current names are actually suitable.
Quoted 10 lines
vmstate_save_old_style{se->ops->save_state}loadvm_handle_cmd_packaged { qemu_loadvm_state_main } qemu_load_device_state { qemu_loadvm_state_main } qemu_savevm_maybe_send_switchover_start { qemu_savevm_send_switchover_start } qemu_savevm_send_* { qemu_savevm_command_send } qemu_savevm_state_complete_postcopy { qemu_savevm_complete }
qemu_savevm_state_complete_precopy_iterable{qemu_savevm_complete}qemu_savevm_state_end_precopy{qemu_savevm_state_end}qemu_savevm_state_header{qemu_savevm_send_header}
...
Quoted 20 lines
qemu_savevm_state_do_setup { qemu_savevm_state_non_iterable_early; qemu_savevm_state_setup }qemu_loadvm_state { qemu_loadvm_state_header; qemu_loadvm_state_setup; qemu_loadvm_state_switchover_ack_needed; qemu_loadvm_state_main; qemu_loadvm_thread_pool_wait }qemu_loadvm_state_main { qemu_loadvm_section_start_full; qemu_loadvm_section_part_end; loadvm_process_command }qemu_savevm_state { qemu_savevm_state_header; qemu_savevm_state_do_setup; qemu_savevm_state_iterate; qemu_savevm_state_complete_precopy; qemu_savevm_state_cleanup }
Quoted 7 lines
qemu_savevm_state_complete_precopy { qemu_savevm_state_complete_precopy_iterable; qemu_savevm_state_non_iterable; qemu_savevm_state_end_precopy }qemu_save_device_state { qemu_savevm_state_non_iterable_early; qemu_savevm_state_non_iterable; qemu_savevm_state_end }
Most of above are special functions higher than vmstate alone, hence IMHO not vmstate_*() APIs. We can name it whatever way we like, or leave them be.
Quoted 10 lines
loadvm_handle_recv_bitmap { migrate_send_rp_recv_bitmap }
loadvm_postcopy_handle_advise { ram_postcopy_incoming_init }
loadvm_postcopy_handle_listen { postcopy_incoming_setup }
loadvm_postcopy_handle_resume { mis->postcopy_pause_sem_fault }
loadvm_postcopy_handle_run { loadvm_postcopy_handle_run_bh }
loadvm_postcopy_handle_run_bh { vm_start }
loadvm_postcopy_ram_handle_discard { postcopy_ram_prepare_discard }
loadvm_process_command { switch (cmd) }
loadvm_process_enable_colo { colo_init_ram_cache }These are almost not vmstate relevant but about MIG_CMD*. We can likely leave them alone. Currently the loadvm_ prefix actually sounds not too bad.
...
[will skip most of below; we could start with vmstate API first]
Quoted 28 lines
qemu_loadvm_approve_switchover { migrate_send_rp_switchover_ack }
qemu_loadvm_load_thread { data->function }
qemu_loadvm_section_part_end { vmstate_load }
qemu_loadvm_section_start_full { vmstate_load }
qemu_loadvm_start_load_thread { thread_pool_submit }
qemu_loadvm_state_cleanup { se->ops->load_cleanup }
qemu_loadvm_state_header { QEMU_VM_FILE_MAGIC }
qemu_loadvm_thread_pool_create { thread_pool_new }
qemu_loadvm_thread_pool_destroy { thread_pool_free }
qemu_loadvm_thread_pool_wait { thread_pool_wait }
qemu_savevm_command_send { QEMU_VM_COMMAND }
qemu_savevm_send_colo_enable { MIG_CMD_ENABLE_COLO }
qemu_savevm_send_configuration { vmstate_save_state(&vmstate_configuration) }
qemu_savevm_send_header { QEMU_VM_FILE_MAGIC }
qemu_savevm_state_end { QEMU_VM_EOF }
qemu_savevm_state_non_iterable { vmstate_save }
qemu_savevm_state_setup { se->ops->save_setup }
qemu_savevm_state_vm_desc { QEMU_VM_VMDESCRIPTION }
register_savevm_live { savevm_state_handler_insert }
save_section_footer { QEMU_VM_SECTION_FOOTER }
save_section_header { section_type }
savevm_state_handler_insert { QTAILQ_INSERT_TAIL }
savevm_state_handler_remove { QTAILQ_REMOVE }
unregister_savevm { savevm_state_handler_remove }
vmstate_register_with_alias_id { savevm_state_handler_insert }
vmstate_save { vmstate_save_old_style; vmstate_save_state }
vmstate_unregister { savevm_state_handler_remove }Signature · 1 line
Peter Xu
Re: [PATCH v2 19/24] migration: Refactor qemu_savevm_state_setup()
Quoted 38 lines · Peter Xu
Quoted 29 lines · Fabiano Rosas
Quoted 16 lines · Peter Xu
Split it into two smaller chunks:
- Dump of
early_setupVMSDs
- Dump ofsave_setup()sectionsThey're mutual exclusive, hence we can run two loops and do them sequentially. This will cause migration thread to loop one more time, but it should be fine when migration just started and only do it once. It's needed because we will need to reuse the
early_vmsdhelper later to deduplicate code elsewhere.QEMU almost sticks with
qemu_savevm_state_XXX()to represent the dump of vmstates's section XXX.
With that in mind, this patch renamed the originalqemu_savevm_state_setup()toqemu_savevm_state_do_setup()instead.Quoted 8 lines
So after this patch:
-
qemu_savevm_state_non_iterable_early()dumpsearly_vmsdsonly,
-qemu_savevm_state_setup()dumpssave_setup()sections only,
-qemu_savevm_state_do_setup()does all things needed during setup phase (including migration SETUP notifies)Signed-off-by Peter Xu <[email protected]>Reviewed-by Fabiano Rosas <[email protected]>Quoted 7 lines
Not for this series, but I think we could try to standardize the naming in
savevm.ca bit. Here's a list of functions fromsavevm.cplus the main function called by them. You'll see that there's room from improvement.Removing this "qemu" prefix and choosing when to put "state" in the name would already be a win.
Quoted 7 lines
True.
IMHO if we choose it again we can use prefix
vmstate_. The currentsavevm_state_is pretty vague on its own when loadvm needs to use it. Then it can be vmstate_save_* on save side, and vmstate_load_* on load side (andvmstate_XXX) when valid on both.
It's slightly annoying that vmstate is also used in the macros and virtually every single VMStateDescription all over QEMU. I wish we had a separate name for the implementation. But vmstate_ is definitely better than qemu_savevm_state_.
Or perhaps something like migstate, migvm, etc.
I think we could see great benefit from a small-ish change doing:
s/qemu_savevm_state_/new_name_/ and s/SaveStateEntry/NewNameEntry/
Because then it becomes clear there's a high level thing that deals with SaveVMHandlers and VMStateDescription which are two different ways of migrating state.
Quoted 7 lines
Quoted 3 lines
Also maybe making more clear what calls vmstate code and what deals with
se->opsonly.Yes we can try, but there'll be some tricky ones, inline below (assuming we can shorten the prefix to "
vmstate_").
Yeah, I see.
Quoted 79 lines
Quoted 3 lines
---
qemu_loadvm_state_setup{se->ops->load_setup}Taking this one as example.
We could rename this to
vmstate_ops_load_setup(), however "load_setup" is actually a concept that is higher than the impl. IOW it'll be also reasonable if we want to add aload_setup()hook to VMSDs some day. Maybe still good to rename it tovmstate_load_setup().Quoted 1 line
qemu_savevm_state_active{se->ops->is_active}Similarly, here I'd think
vmstate_active()(it applies to both save/load) better thanvmstate_ops_active()because active describes the vmstate instance, and it's fine one day we want to introduce VMSD.active....
Quoted 11 lines
qemu_savevm_state_blocked { se->vmsd->unmigratable } qemu_savevm_state_cleanup { se->ops->save_cleanup } qemu_savevm_state_guest_unplug_pending { se->vmsd->dev_unplug_pending } qemu_savevm_state_iterate { se->ops->save_live_iterate } qemu_savevm_state_non_iterable_early { se->vmsd->early_setup; vmstate_save } qemu_savevm_state_pending_estimate { se->ops->state_pending_estimate } qemu_savevm_state_pending_exact { se->ops->state_pending_exact } qemu_savevm_state_postcopy_prepare { se->ops->save_postcopy_prepare } qemu_savevm_state_prepare { se->ops->save_prepare } qemu_savevm_state_resume_prepare { se->ops->resume_prepare } loadvm_postcopy_handle_switchover_start { se->ops->switchover_start }Quoted 8 lines
qemu_loadvm_load_state_buffer { se->ops->load_state_buffer } qemu_loadvm_state_switchover_ack_needed { se->ops->switchover_ack_needed } qemu_savevm_complete { se->ops->save_complete } qemu_savevm_complete_exists { se->ops->save_complete } qemu_savevm_non_migratable_list { se->vmsd->unmigratable } qemu_savevm_se_iterable { se->ops->save_setup } save_state_priority { se->vmsd->priority } vmstate_load { se->ops->load_state }This one (and
vmstate_save()) is special because it handles both impls (VMSD and OPS). The current names are actually suitable.Quoted 10 lines
vmstate_save_old_style{se->ops->save_state}loadvm_handle_cmd_packaged { qemu_loadvm_state_main } qemu_load_device_state { qemu_loadvm_state_main } qemu_savevm_maybe_send_switchover_start { qemu_savevm_send_switchover_start } qemu_savevm_send_* { qemu_savevm_command_send } qemu_savevm_state_complete_postcopy { qemu_savevm_complete }
qemu_savevm_state_complete_precopy_iterable{qemu_savevm_complete}qemu_savevm_state_end_precopy{qemu_savevm_state_end}qemu_savevm_state_header{qemu_savevm_send_header}...
Quoted 24 lines
qemu_savevm_state_do_setup { qemu_savevm_state_non_iterable_early; qemu_savevm_state_setup }qemu_loadvm_state { qemu_loadvm_state_header; qemu_loadvm_state_setup; qemu_loadvm_state_switchover_ack_needed; qemu_loadvm_state_main; qemu_loadvm_thread_pool_wait }qemu_loadvm_state_main { qemu_loadvm_section_start_full; qemu_loadvm_section_part_end; loadvm_process_command }qemu_savevm_state { qemu_savevm_state_header; qemu_savevm_state_do_setup; qemu_savevm_state_iterate; qemu_savevm_state_complete_precopy; qemu_savevm_state_cleanup }qemu_savevm_state_complete_precopy { qemu_savevm_state_complete_precopy_iterable; qemu_savevm_state_non_iterable; qemu_savevm_state_end_precopy }
Quoted 8 lines
Quoted 3 lines
qemu_save_device_state { qemu_savevm_state_non_iterable_early; qemu_savevm_state_non_iterable; qemu_savevm_state_end }Most of above are special functions higher than vmstate alone, hence IMHO not vmstate_*() APIs. We can name it whatever way we like, or leave them be.
They definitely need a better name, it's only a question of "when".
Quoted 11 lines
Quoted 10 lines
loadvm_handle_recv_bitmap { migrate_send_rp_recv_bitmap } loadvm_postcopy_handle_advise { ram_postcopy_incoming_init } loadvm_postcopy_handle_listen { postcopy_incoming_setup } loadvm_postcopy_handle_resume { mis->postcopy_pause_sem_fault } loadvm_postcopy_handle_run { loadvm_postcopy_handle_run_bh } loadvm_postcopy_handle_run_bh { vm_start } loadvm_postcopy_ram_handle_discard { postcopy_ram_prepare_discard } loadvm_process_command { switch (cmd) } loadvm_process_enable_colo { colo_init_ram_cache }
Quoted 4 lines
These are almost not vmstate relevant but about MIG_CMD*. We can likely leave them alone. Currently the
loadvm_prefix actually sounds not too bad.
I agree.
Quoted 32 lines
...
[will skip most of below; we could start with vmstate API first]
Quoted 28 lines
qemu_loadvm_approve_switchover { migrate_send_rp_switchover_ack } qemu_loadvm_load_thread { data->function } qemu_loadvm_section_part_end { vmstate_load } qemu_loadvm_section_start_full { vmstate_load } qemu_loadvm_start_load_thread { thread_pool_submit } qemu_loadvm_state_cleanup { se->ops->load_cleanup } qemu_loadvm_state_header { QEMU_VM_FILE_MAGIC } qemu_loadvm_thread_pool_create { thread_pool_new } qemu_loadvm_thread_pool_destroy { thread_pool_free } qemu_loadvm_thread_pool_wait { thread_pool_wait } qemu_savevm_command_send { QEMU_VM_COMMAND } qemu_savevm_send_colo_enable { MIG_CMD_ENABLE_COLO } qemu_savevm_send_configuration { vmstate_save_state(&vmstate_configuration) } qemu_savevm_send_header { QEMU_VM_FILE_MAGIC } qemu_savevm_state_end { QEMU_VM_EOF } qemu_savevm_state_non_iterable { vmstate_save } qemu_savevm_state_setup { se->ops->save_setup } qemu_savevm_state_vm_desc { QEMU_VM_VMDESCRIPTION } register_savevm_live { savevm_state_handler_insert } save_section_footer { QEMU_VM_SECTION_FOOTER } save_section_header { section_type } savevm_state_handler_insert { QTAILQ_INSERT_TAIL } savevm_state_handler_remove { QTAILQ_REMOVE } unregister_savevm { savevm_state_handler_remove } vmstate_register_with_alias_id { savevm_state_handler_insert } vmstate_save { vmstate_save_old_style; vmstate_save_state } vmstate_unregister { savevm_state_handler_remove }
PATCH v2 20/24 migration: Introduce qemu_savevm_state_active()
In reply to Peter XuIntroduce this helper to detect if a SaveStateEntry is active.
Note that this helper can actually also be used in loadvm paths, but let's stick with this name for now because we still use SaveStateEntry for the shared structure that both savevm/loadvm uses, where this name still suites.
--- migration/savevm.c | 63 ++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 36 deletions(-)
migration/savevm.c +27 -36
@@ -1071,6 +1071,16 @@ void qemu_savevm_state_end(QEMUFile *f)qemu_put_byte (f,QEMU_VM_EOF ); } +static inline bool qemu_savevm_state_active (SaveStateEntry *se) +{ +/* When no is_active() hook, always treat it as ACTIVE */ +if (!se->ops->is_active) { +return true; + } + +return se->ops->is_active (se->opaque); +} +/** * qemu_savevm_command_send: Send a 'QEMU_VM_COMMAND' type element with the * command and associated data.
@@ -1352,12 +1362,9 @@ int qemu_savevm_state_prepare(Error **errp)if (!se->ops || !se->ops->save_prepare) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } - ret = se->ops->save_prepare (se->opaque, errp);if (ret <0 ) {return ret;
@@ -1397,10 +1404,8 @@ static int qemu_savevm_state_setup(QEMUFile *f, Error **errp)if (!se->ops || !se->ops->save_setup) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; }save_section_header (f, se,QEMU_VM_SECTION_START ); ret = se->ops->save_setup (f, se->opaque, errp);
@@ -1450,10 +1455,8 @@ int qemu_savevm_state_resume_prepare(MigrationState *s)if (!se->ops || !se->ops->resume_prepare) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } ret = se->ops->resume_prepare (s, se->opaque);if (ret <0 ) {
@@ -1481,8 +1484,7 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy)if (!se->ops || !se->ops->save_live_iterate) {continue ; } -if (se->ops->is_active && - !se->ops->is_active (se->opaque)) { +if (!qemu_savevm_state_active (se)) {continue ; } if (se->ops->is_active_iterate &&
@@ -1543,10 +1545,8 @@ static int qemu_savevm_complete(SaveStateEntry *se, QEMUFile *f) {int ret; -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -return 0 ; - } +if (!qemu_savevm_state_active (se)) { +return 0 ; }trace_savevm_section_start (se->idstr, se->section_id);
@@ -1596,10 +1596,8 @@ bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp)continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; }trace_savevm_section_start (se->idstr, se->section_id);
@@ -1785,10 +1783,8 @@ void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,if (!se->ops || !se->ops->state_pending_estimate) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } se->ops->state_pending_estimate (se->opaque, must_precopy, can_postcopy); }
@@ -1806,10 +1802,8 @@ void qemu_savevm_state_pending_exact(uint64_t *must_precopy,if (!se->ops || !se->ops->state_pending_exact) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } se->ops->state_pending_exact (se->opaque, must_precopy, can_postcopy); }
@@ -2829,12 +2823,9 @@ static int qemu_loadvm_state_setup(QEMUFile *f, Error **errp)if (!se->ops || !se->ops->load_setup) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } - ret = se->ops->load_setup (f, se->opaque, errp);if (ret <0 ) { error_prepend(errp, "Load state of device %s failed: ",
Signature · 1 line
2.50.1
Re: [PATCH v2 20/24] migration: Introduce qemu_savevm_state_active()
Quoted 140 lines · Peter Xu
Introduce this helper to detect if a SaveStateEntry is active.
Note that this helper can actually also be used in loadvm paths, but let's stick with this name for now because we still use SaveStateEntry for the shared structure that both savevm/loadvm uses, where this name still suites.
Signed-off-by Peter Xu <[email protected]>--- migration/savevm.c | 63 ++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 36 deletions(-)migration/savevm.c +25
-32@@ -1071,6 +1071,16 @@ void qemu_savevm_state_end(QEMUFile *f)qemu_put_byte (f,QEMU_VM_EOF ); } +static inline bool qemu_savevm_state_active (SaveStateEntry *se) +{ +/* When no is_active() hook, always treat it as ACTIVE */ +if (!se->ops->is_active) { +return true; + } + +return se->ops->is_active (se->opaque); +} +/** * qemu_savevm_command_send: Send a 'QEMU_VM_COMMAND' type element with the * command and associated data. @@ -1352,12 +1362,9 @@ int qemu_savevm_state_prepare(Error **errp)if (!se->ops || !se->ops->save_prepare) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } - ret = se->ops->save_prepare (se->opaque, errp);if (ret <0 ) {return ret;@@ -1397,10 +1404,8 @@ static int qemu_savevm_state_setup(QEMUFile *f, Error **errp)if (!se->ops || !se->ops->save_setup) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; }save_section_header (f, se,QEMU_VM_SECTION_START ); ret = se->ops->save_setup (f, se->opaque, errp);@@ -1450,10 +1455,8 @@ int qemu_savevm_state_resume_prepare(MigrationState *s)if (!se->ops || !se->ops->resume_prepare) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } ret = se->ops->resume_prepare (s, se->opaque);if (ret <0 ) {@@ -1481,8 +1484,7 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy)if (!se->ops || !se->ops->save_live_iterate) {continue ; } -if (se->ops->is_active && - !se->ops->is_active (se->opaque)) { +if (!qemu_savevm_state_active (se)) {continue ; } if (se->ops->is_active_iterate &&@@ -1543,10 +1545,8 @@ static int qemu_savevm_complete(SaveStateEntry *se, QEMUFile *f) {int ret; -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -return 0 ; - } +if (!qemu_savevm_state_active (se)) { +return 0 ; }trace_savevm_section_start (se->idstr, se->section_id);@@ -1596,10 +1596,8 @@ bool qemu_savevm_state_postcopy_prepare(QEMUFile *f, Error **errp)continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; }trace_savevm_section_start (se->idstr, se->section_id);@@ -1785,10 +1783,8 @@ void qemu_savevm_state_pending_estimate(uint64_t *must_precopy,if (!se->ops || !se->ops->state_pending_estimate) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } se->ops->state_pending_estimate (se->opaque, must_precopy, can_postcopy); }@@ -1806,10 +1802,8 @@ void qemu_savevm_state_pending_exact(uint64_t *must_precopy,if (!se->ops || !se->ops->state_pending_exact) {continue ; } -if (se->ops->is_active) { -if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } se->ops->state_pending_exact (se->opaque, must_precopy, can_postcopy); }@@ -2829,12 +2823,9 @@ static int qemu_loadvm_state_setup(QEMUFile *f, Error **errp)if (!se->ops || !se->ops->load_setup) {continue ; } -if (se->ops->is_active) {
migration/savevm.c static int qemu_loadvm_state_setup(QEMUFile *f, Error **errp) · line 2831
Quoted 10 lines
migration/savevm.c +2
-4-if (!se->ops->is_active (se->opaque)) { -continue ; - } +if (!qemu_savevm_state_active (se)) { +continue ; } - ret = se->ops->load_setup (f, se->opaque, errp);if (ret <0 ) { error_prepend(errp, "Load state of device %s failed: ",
PATCH v2 21/24 migration/bg-snapshot: Cleanup error paths
In reply to Peter XuCleanup bg_migration_thread() function on error handling. First of all, early_fail is almost only used to say if BQL is taken. Since we already have separate jumping labels, we don't really need it, hence removed.
Also, since local_err is around, making sure every failure path will set a proper error string for the failure, then propagate to MigrationState.error.
--- migration/migration.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-)
migration/migration.c +16 -17
@@ -3616,7 +3616,6 @@ static void *bg_migration_thread(void *opaque)int64_t setup_start;MigThrError thr_error;QEMUFile *fb; -bool early_fail = true;Error *local_err =NULL ;int ret;
@@ -3662,10 +3661,7 @@ static void *bg_migration_thread(void *opaque)* devices to unplug. This to preserve migration state transitions. */ if (ret) { -migrate_error_propagate (s, local_err); -migrate_set_state (&s->state,MIGRATION_STATUS_ACTIVE , -MIGRATION_STATUS_FAILED ); - goto fail_setup; + goto fail; } s->setup_time =qemu_clock_get_ms (QEMU_CLOCK_HOST ) - setup_start;
@@ -3675,11 +3671,13 @@ static void *bg_migration_thread(void *opaque)bql_lock ();if (migration_stop_vm (s,RUN_STATE_PAUSED )) { - goto fail; +error_setg (&local_err,"Failed to stop the VM" ); + goto fail_with_bql; }if (qemu_savevm_state_non_iterable (fb)) { - goto fail; +error_setg (&local_err,"Failed to save non-iterable devices" ); + goto fail_with_bql; }qemu_savevm_state_end_precopy (s, fb);
@@ -3692,9 +3690,9 @@ static void *bg_migration_thread(void *opaque)/* Now initialize UFFD context and start tracking RAM writes */ if (ram_write_tracking_start ()) { - goto fail; +error_setg (&local_err,"Failed to start write tracking" ); + goto fail_with_bql; } - early_fail = false;/* * Start VM from BH handler to avoid write-fault lock here.
@@ -3726,21 +3724,22 @@ static void *bg_migration_thread(void *opaque) }trace_migration_thread_after_loop (); + goto done; + +fail_with_bql: +bql_unlock (); fail: -if (early_fail) { -migrate_set_state (&s->state,MIGRATION_STATUS_ACTIVE , -MIGRATION_STATUS_FAILED ); -bql_unlock (); - } +/* local_err is guaranteed to be set when reaching here */ +migrate_error_propagate (s, local_err); +migrate_set_state (&s->state,MIGRATION_STATUS_ACTIVE , +MIGRATION_STATUS_FAILED ); -fail_setup: +done:bg_migration_iteration_finish (s); -qemu_fclose (fb);object_unref (OBJECT (s));rcu_unregister_thread (); -return NULL ; }
Signature · 1 line
2.50.1
Re: [PATCH v2 21/24] migration/bg-snapshot: Cleanup error paths
Quoted 8 lines · Peter Xu
Cleanup
bg_migration_thread()function on error handling. First of all,early_failis almost only used to say if BQL is taken. Since we already have separate jumping labels, we don't really need it, hence removed.Also, since
local_erris around, making sure every failure path will set a proper error string for the failure, then propagate to MigrationState.error.Signed-off-by Peter Xu <[email protected]>
PATCH v2 22/24 migration: Make qemu_savevm_state_non_iterable() take errp
In reply to Peter XuLet the function report errors to upper layers. Out of three current users, two of them already process the errors, except one outlier, qemu_savevm_state_complete_precopy(), where we do it manually for now with a comment for TODO.
--- migration/savevm.h | 2 +- migration/migration.c | 8 ++++---- migration/savevm.c | 13 +++++++------ 3 files changed, 12 insertions(+), 11 deletions(-)
migration/savevm.h +1 -1
@@ -74,7 +74,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, Error **errp);int qemu_load_device_state (QEMUFile *f,Error **errp);int qemu_loadvm_approve_switchover (void ); -int qemu_savevm_state_non_iterable (QEMUFile *f); +int qemu_savevm_state_non_iterable (QEMUFile *f,Error **errp);int qemu_savevm_state_non_iterable_early (QEMUFile *f,JSONWriter *vmdesc,Error **errp);
migration/migration.c +4 -4
@@ -2545,9 +2545,9 @@ static int postcopy_start(MigrationState *ms, Error **errp)*/ qemu_savevm_send_postcopy_listen (fb); - ret =qemu_savevm_state_non_iterable (fb); + ret =qemu_savevm_state_non_iterable (fb, errp);if (ret) { -error_setg (errp,"Postcopy save non-iterable device states failed" ); +error_prepend (errp,"Postcopy save non-iterable states failed: " ); goto fail_closefb; }
@@ -3675,8 +3675,8 @@ static void *bg_migration_thread(void *opaque) goto fail_with_bql; } -if (qemu_savevm_state_non_iterable (fb)) { -error_setg (&local_err,"Failed to save non-iterable devices" ); +if (qemu_savevm_state_non_iterable (fb, &local_err)) { +error_prepend (&local_err,"Failed to save non-iterable devices" ); goto fail_with_bql; }
migration/savevm.c +7 -6
@@ -1710,13 +1710,12 @@ void qemu_savevm_state_end_precopy(MigrationState *s, QEMUFile *f)qemu_savevm_state_vm_desc (s, f); } -int qemu_savevm_state_non_iterable (QEMUFile *f) +int qemu_savevm_state_non_iterable (QEMUFile *f,Error **errp) {MigrationState *ms =migrate_get_current ();int64_t start_ts_each, end_ts_each;JSONWriter *vmdesc = ms->vmdesc;SaveStateEntry *se; -Error *local_err =NULL ;int ret;/* Making sure cpu states are synchronized before saving non-iterable */
@@ -1730,10 +1729,8 @@ int qemu_savevm_state_non_iterable(QEMUFile *f) start_ts_each =qemu_clock_get_us (QEMU_CLOCK_REALTIME ); - ret =vmstate_save (f, se, vmdesc, &local_err); + ret =vmstate_save (f, se, vmdesc, errp);if (ret) { -migrate_error_propagate (ms,error_copy (local_err)); -error_report_err (local_err);return ret; }
@@ -1750,6 +1747,7 @@ int qemu_savevm_state_non_iterable(QEMUFile *f)int qemu_savevm_state_complete_precopy (MigrationState *s) {QEMUFile *f = s->to_dst_file; +Error *local_err =NULL ;int ret; ret =qemu_savevm_state_complete_precopy_iterable (f, false);
@@ -1757,8 +1755,11 @@ int qemu_savevm_state_complete_precopy(MigrationState *s)return ret; } - ret =qemu_savevm_state_non_iterable (f); +/* TODO: pass error upper */ + ret =qemu_savevm_state_non_iterable (f, &local_err);if (ret) { +migrate_error_propagate (s,error_copy (local_err)); +error_report_err (local_err);return ret; }
Signature · 1 line
2.50.1
Re: [PATCH v2 22/24] migration: Make qemu_savevm_state_non_iterable() take errp
Quoted 42 lines · Peter Xu
Let the function report errors to upper layers. Out of three current users, two of them already process the errors, except one outlier,
qemu_savevm_state_complete_precopy(), where we do it manually for now with a comment for TODO.Signed-off-by Peter Xu <[email protected]>--- migration/savevm.h | 2 +- migration/migration.c | 8 ++++---- migration/savevm.c | 13 +++++++------ 3 files changed, 12 insertions(+), 11 deletions(-)migration/savevm.h +1
-1@@ -74,7 +74,7 @@ int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis, Error **errp);int qemu_load_device_state (QEMUFile *f,Error **errp);int qemu_loadvm_approve_switchover (void ); -int qemu_savevm_state_non_iterable (QEMUFile *f); +int qemu_savevm_state_non_iterable (QEMUFile *f,Error **errp);int qemu_savevm_state_non_iterable_early (QEMUFile *f,JSONWriter *vmdesc,Error **errp);migration/migration.c +2
-2@@ -2545,9 +2545,9 @@ static int postcopy_start(MigrationState *ms, Error **errp)*/ qemu_savevm_send_postcopy_listen (fb); - ret =qemu_savevm_state_non_iterable (fb); + ret =qemu_savevm_state_non_iterable (fb, errp);if (ret) { -error_setg (errp,"Postcopy save non-iterable device states failed" ); +error_prepend (errp,"Postcopy save non-iterable states failed: " ); goto fail_closefb; }
migration/migration.c static void *bg_migration_thread(void *opaque) · line 3679
Quoted 7 lines
migration/migration.c +2
-2@@ -3675,8 +3675,8 @@ static void *bg_migration_thread(void *opaque) goto fail_with_bql; } -if (qemu_savevm_state_non_iterable (fb)) { -error_setg (&local_err,"Failed to save non-iterable devices" ); +if (qemu_savevm_state_non_iterable (fb, &local_err)) { +error_prepend (&local_err,"Failed to save non-iterable devices" );
No space or punctuation at the end?
Quoted 54 lines
migration/migration.c +0
-0goto fail_with_bql; }migration/savevm.c +7
-6@@ -1710,13 +1710,12 @@ void qemu_savevm_state_end_precopy(MigrationState *s, QEMUFile *f)qemu_savevm_state_vm_desc (s, f); } -int qemu_savevm_state_non_iterable (QEMUFile *f) +int qemu_savevm_state_non_iterable (QEMUFile *f,Error **errp) {MigrationState *ms =migrate_get_current ();int64_t start_ts_each, end_ts_each;JSONWriter *vmdesc = ms->vmdesc;SaveStateEntry *se; -Error *local_err =NULL ;int ret;/* Making sure cpu states are synchronized before saving non-iterable */ @@ -1730,10 +1729,8 @@ int qemu_savevm_state_non_iterable(QEMUFile *f) start_ts_each =qemu_clock_get_us (QEMU_CLOCK_REALTIME ); - ret =vmstate_save (f, se, vmdesc, &local_err); + ret =vmstate_save (f, se, vmdesc, errp);if (ret) { -migrate_error_propagate (ms,error_copy (local_err)); -error_report_err (local_err);return ret; }@@ -1750,6 +1747,7 @@ int qemu_savevm_state_non_iterable(QEMUFile *f)int qemu_savevm_state_complete_precopy (MigrationState *s) {QEMUFile *f = s->to_dst_file; +Error *local_err =NULL ;int ret; ret =qemu_savevm_state_complete_precopy_iterable (f, false);@@ -1757,8 +1755,11 @@ int qemu_savevm_state_complete_precopy(MigrationState *s)return ret; } - ret =qemu_savevm_state_non_iterable (f); +/* TODO: pass error upper */ + ret =qemu_savevm_state_non_iterable (f, &local_err);if (ret) { +migrate_error_propagate (s,error_copy (local_err)); +error_report_err (local_err);return ret; }
PATCH v2 23/24 migration: Simplify qemu_save_device_state()
In reply to Peter XuThis function is used by both COLO and Xen. Simplify it with two changes:
- Remove checks on qemu_savevm_se_iterable(): this is not needed as vmstate_save() also checks for "save_state() || vmsd" instead. Here, save_setup() (or say, iterable states) should be mutual exclusive to "save_state() || vmsd" [*].
- Remove migrate_error_propagate(): both of the users are not using live migration framework, but raw vmstate operations. Error propagation is not needed for query-migrate persistence.
[*] One tricky user is VFIO, who provided _both_ save_state() and save_setup(). However VFIO mustn't have been used in these paths or it means both COLO and Xen have ignored VFIO data instead (that is, qemu_savevm_se_iterable() will return true for VFIO). Hence, this change is safe.
--- migration/savevm.c | 5 ----- 1 file changed, 5 deletions(-)
migration/savevm.c +0 -5
@@ -1897,13 +1897,8 @@ int qemu_save_device_state(QEMUFile *f) QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {int ret; -if (qemu_savevm_se_iterable (se)) { -continue ; - } ret =vmstate_save (f, se,NULL , &local_err);if (ret) { -migrate_error_propagate (migrate_get_current (), -error_copy (local_err));error_report_err (local_err);return ret; }
Signature · 1 line
2.50.1
Re: [PATCH v2 23/24] migration: Simplify qemu_save_device_state()
Quoted 11 lines · Peter Xu
This function is used by both COLO and Xen. Simplify it with two changes:
- Remove checks on
qemu_savevm_se_iterable(): this is not needed asvmstate_save()also checks for "save_state()|| vmsd" instead. Here,save_setup()(or say, iterable states) should be mutual exclusive to "save_state()|| vmsd" [*].- Remove
migrate_error_propagate(): both of the users are not using live migration framework, but raw vmstate operations. Error propagation is not needed for query-migrate persistence.
s/not needed/only needed/
I can fixup if no repost.
Quoted 9 lines
[*] One tricky user is VFIO, who provided _both_
save_state()andsave_setup(). However VFIO mustn't have been used in these paths or it means both COLO and Xen have ignored VFIO data instead (that is,qemu_savevm_se_iterable()will return true for VFIO). Hence, this change is safe.Cc David Woodhouse <[email protected]>Cc Paul Durrant <[email protected]>Signed-off-by Peter Xu <[email protected]>
Quoted 22 lines
--- migration/savevm.c | 5 ----- 1 file changed, 5 deletions(-)migration/savevm.c +0
-5@@ -1897,13 +1897,8 @@ int qemu_save_device_state(QEMUFile *f) QTAILQ_FOREACH(se, &savevm_state.handlers, entry) {int ret; -if (qemu_savevm_se_iterable (se)) { -continue ; - } ret =vmstate_save (f, se,NULL , &local_err);if (ret) { -migrate_error_propagate (migrate_get_current (), -error_copy (local_err));error_report_err (local_err);return ret; }
Re: [PATCH v2 23/24] migration: Simplify qemu_save_device_state()
Quoted 9 lines · Fabiano Rosas
Quoted 7 lines · Peter Xu
This function is used by both COLO and Xen. Simplify it with two changes:
- Remove checks on
qemu_savevm_se_iterable(): this is not needed asvmstate_save()also checks for "save_state()|| vmsd" instead. Here,save_setup()(or say, iterable states) should be mutual exclusive to "save_state()|| vmsd" [*].
Quoted 8 lines
Quoted 4 lines
- Remove
migrate_error_propagate(): both of the users are not using live migration framework, but raw vmstate operations. Error propagation is not needed for query-migrate persistence.s/not needed/only needed/
I can fixup if no repost.
I'll fix both issues, including the one in the previous patch.
I'll wait for your review on the last patch to complete to see if I should repost.
Quoted 7 lines
Quoted 7 lines
[*] One tricky user is VFIO, who provided _both_
save_state()andsave_setup(). However VFIO mustn't have been used in these paths or it means both COLO and Xen have ignored VFIO data instead (that is,qemu_savevm_se_iterable()will return true for VFIO). Hence, this change is safe.
Quoted 5 lines
Quoted 3 lines
Cc David Woodhouse <[email protected]>Cc Paul Durrant <[email protected]>Signed-off-by Peter Xu <[email protected]>Reviewed-by Fabiano Rosas <[email protected]>
Thanks.
Signature · 1 line
Peter Xu
PATCH v2 24/24 migration/colo/xen: Use generic helpers in qemu_save_device_state()
In reply to Peter XuUse qemu_savevm_state_non_iterable*() helpers for saving device states, rather than walking the vmstate handlers on its own.
Non-iterables can be either early_setup devices, or otherwise.
Note that QEMU only has one early_setup device currently, which is virtio-mem, and I highly doubt if it is used in either COLO or Xen users.. However this step is still better needed to provide full coverage of all non-iterable vmstates.
When at it, allow it to report errors.
--- migration/savevm.h | 3 +-- migration/colo.c | 2 +- migration/savevm.c | 30 +++++++++++++++--------------- 3 files changed, 17 insertions(+), 18 deletions(-)
migration/savevm.h +1 -2
@@ -66,8 +66,7 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,uint64_t *start_list, uint64_t *length_list);void qemu_savevm_send_colo_enable (QEMUFile *f); -int qemu_save_device_state (QEMUFile *f); - +int qemu_save_device_state (QEMUFile *f,Error **errp);int qemu_loadvm_state (QEMUFile *f,Error **errp);void qemu_loadvm_state_cleanup (MigrationIncomingState *mis); int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
migration/colo.c +1 -1
@@ -454,7 +454,7 @@ static int colo_do_checkpoint_transaction(MigrationState *s, }/* Note: device state is saved into buffer */ - ret =qemu_save_device_state (fb); + ret =qemu_save_device_state (fb, &local_err);bql_unlock ();if (ret <0 ) {
migration/savevm.c +15 -15
@@ -1887,26 +1887,24 @@ static bool qemu_savevm_se_iterable(SaveStateEntry *se)return se->ops && se->ops->save_setup; } -int qemu_save_device_state (QEMUFile *f) +int qemu_save_device_state (QEMUFile *f,Error **errp) { -Error *local_err =NULL ; -SaveStateEntry *se; - -cpu_synchronize_all_states (); +int ret; - QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { -int ret; +/* Both COLO and Xen never use vmdesc, hence NULL. */ + ret =qemu_savevm_state_non_iterable_early (f,NULL , errp); +if (ret) { +return ret; + } - ret =vmstate_save (f, se,NULL , &local_err); -if (ret) { -error_report_err (local_err); -return ret; - } + ret =qemu_savevm_state_non_iterable (f, errp); +if (ret) { +return ret; }qemu_savevm_state_end (f); -return qemu_file_get_error (f); +return 0 ; }static SaveStateEntry *find_se (const char *idstr,uint32_t instance_id)
@@ -3346,9 +3344,11 @@ void qmp_xen_save_devices_state(const char *filename, bool has_live, bool live, f =qemu_file_new_output (QIO_CHANNEL (ioc));object_unref (OBJECT (ioc));qemu_savevm_send_header (f); - ret =qemu_save_device_state (f); + ret =qemu_save_device_state (f, errp);if (ret <0 ||qemu_fclose (f) <0 ) { -error_setg (errp,"saving Xen device state failed" ); +if (*errp ==NULL ) { +error_setg (errp,"saving Xen device state failed" ); + } }else {/* libxl calls the QMP command "stop" before calling * "xen-save-devices-state" and in case of migration failure, libxl
Signature · 1 line
2.50.1
Re: [PATCH v2 24/24] migration/colo/xen: Use generic helpers in qemu_save_device_state()
Quoted 10 lines · Peter Xu
Use qemu_savevm_state_non_iterable*() helpers for saving device states, rather than walking the vmstate handlers on its own.
Non-iterables can be either
early_setupdevices, or otherwise.Note that QEMU only has one
early_setupdevice currently, which is virtio-mem, and I highly doubt if it is used in either COLO or Xen users.. However this step is still better needed to provide full coverage of all non-iterable vmstates.
Quoted 5 lines
When at it, allow it to report errors.
Cc David Woodhouse <[email protected]>Cc Paul Durrant <[email protected]>Signed-off-by Peter Xu <[email protected]>
Quoted 11 lines · On Tue, 27 Jan 2026 13:52:30 -0500 Peter Xu <[email protected]> wrote:
CI: https://gitlab.com/peterx/qemu/-/pipelines/2289719627
v1: https://lore.kernel.org/r/[email protected]
This is the v2 of the small cleanup series. It used to be only for COLO, but now for some other things too around dumping vmstates.
The small series growed a little bit, because I wanted to try Fabiano's request to deduplicate
qemu_save_device_state()and precopy's similar version to dump non-iterable devices.
Quoted 8 lines
Changelog skipped, as too many things added. Old patches should almost the same with R-bs collected, though. I still touched up some small things, though (where I dropped the R-bs).
This should pass the new COLO unit test too, I hope I didn't overlook something once more. Please anyone let me know if it fails.
Comments welcomed, thanks.
This now works well in my testing:
Quoted 34 lines
Peter Xu (24):
migration: Introduce qemu_savevm_send_* helpers
migration: Useqemu_savevm_send_header()inqemu_save_device_state()
migration: Remove onemigration_in_colo_state()occurance migration/savevm: Remove SaveStateEntry.is_ram migration/colo: Unwrapqemu_savevm_live_state()
migration: Remove call to send switchover start event in colo/savevm
colo: Forbid VM resume during checkpointing migration/colo: Use the RAM iterable helper directly migration/colo: Documentqemu_fflush(fb)
migration: Dropiterable_onlyinqemu_savevm_state_complete_precopy
migration: Dropqemu_file_set_error()when save non-iterable fails migration/colo: Send device states without copying buffer migration/postcopy: Send device states without copying buffer
migration: Introduceqemu_savevm_state_end()
migration: Provide helper for save vm description
migration: Splitqemu_savevm_state_complete_precopy_non_iterable()
migration:qemu_savevm_state_complete_precopy()take MigrationState*
migration: Cleanup error propagates inqemu_savevm_state_setup()
migration: Refactorqemu_savevm_state_setup()
migration: Introduceqemu_savevm_state_active()migration/bg-snapshot: Cleanup error paths
migration: Makeqemu_savevm_state_non_iterable()take errp
migration: Simplifyqemu_save_device_state()migration/colo/xen: Use generic helpers inqemu_save_device_state()migration/savevm.h | 18 +-- migration/colo.c | 16 ++- migration/migration.c | 48 +++---- migration/savevm.c | 312 ++++++++++++++++++++++-------------------- monitor/qmp-cmds.c | 3 + 5 files changed, 213 insertions(+), 184 deletions(-)