Skip to contentSkip to conversation

Migration: Small fixes and improvements

PATCH In review Dr. David Alan Gilbert (git) 4 participants 13 messages Dec 16, 2015 – Jan 13, 2016

lore
Share & tools
lore
#0 lore raw

v2 cover Migration: Small fixes and improvements

From: "Dr. David Alan Gilbert" <[email protected]>

This is a set of 4 small fixes and improvements I've been collecting during the 2.5 freeze; I've added in a couple of Zhanghailiang's patches that are easier to merge before rather than after.
(Of which one I've added a small fix of my own, see the message)

Jiri was after events from the incoming postcopy side to make it easy for libvirt; I'd forgotten to add them - Zhanghailiang's patches change the way the events get sent on the incoming side.

Jiri also wanted an event on the outgoing side at each pass, which makes it easy for libvirt to trigger the transition to postcopy.

The other two patches avoid a couple of buffer copies on the incoming side; although in both xbzrle and multithread compression it's rare that the destination is the slow path (but one patch is trivial and the other saves code).

Dave

v2
1/6 update text based on Juan's comments
4/6 Document 'pass' component of event, update qmp-events.txt

Dr. David Alan Gilbert (4):
Postcopy: Send events/change state on incoming side
Migration: Emit event at start of pass
Use qemu_get_buffer_in_place for xbzrle data multithread decompression: Avoid one copy

zhanghailiang (2):
migration: Export migrate_set_state()
migration: Add state records for migration incoming

 docs/qmp-events.txt           | 14 ++++++++++++
 include/migration/migration.h |  3 +++
 migration/migration.c         | 52 ++++++++++++++++++++++++++-----------------
 migration/ram.c               | 21 ++++++++---------
 migration/savevm.c            | 22 ++++++++++++------
 qapi/event.json               | 13 +++++++++++
 6 files changed, 87 insertions(+), 38 deletions(-)
Signature · 1 line
2.5.0
#1 lore raw

PATCH v2 1/6 migration: Export migrate_set_state()

From: zhanghailiang <[email protected]>

Change the first parameter of migrate_set_state(), and export it. We will use it in a later patch to update incoming state.

dgilbert: Updated comment as per Juan's review

---
 include/migration/migration.h |  2 ++
 migration/migration.c         | 36 +++++++++++++++++++++---------------
 2 files changed, 23 insertions(+), 15 deletions(-)
include/migration/migration.h +2 -0
index fd018b7..8340432 100644
@@ -169,6 +169,8 @@ struct MigrationState
     RAMBlock *last_req_rb;
 };
 
+void migrate_set_state(int *state, int old_state, int new_state);
+
 void process_incoming_migration(QEMUFile *f);
 
 void qemu_start_incoming_migration(const char *uri, Error **errp);
migration/migration.c +21 -15
index adc6b6f..ba6d4f6 100644
@@ -787,9 +787,9 @@ void qmp_migrate_start_postcopy(Error **errp)
 
 /* shared migration helpers */
 
-static void migrate_set_state(MigrationState *s, int old_state, int new_state)
+void migrate_set_state(int *state, int old_state, int new_state)
 {
-    if (atomic_cmpxchg(&s->state, old_state, new_state) == old_state) {
+    if (atomic_cmpxchg(state, old_state, new_state) == old_state) {
         trace_migrate_set_state(new_state);
         migrate_generate_event(new_state);
     }
@@ -822,7 +822,7 @@ static void migrate_fd_cleanup(void *opaque)
            (s->state != MIGRATION_STATUS_POSTCOPY_ACTIVE));
 
     if (s->state == MIGRATION_STATUS_CANCELLING) {
-        migrate_set_state(s, MIGRATION_STATUS_CANCELLING,
+        migrate_set_state(&s->state, MIGRATION_STATUS_CANCELLING,
                           MIGRATION_STATUS_CANCELLED);
     }
 
@@ -833,7 +833,8 @@ void migrate_fd_error(MigrationState *s)
 {
     trace_migrate_fd_error();
     assert(s->file == NULL);
-    migrate_set_state(s, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_FAILED);
+    migrate_set_state(&s->state, MIGRATION_STATUS_SETUP,
+                      MIGRATION_STATUS_FAILED);
     notifier_list_notify(&migration_state_notifiers, s);
 }
 
@@ -853,7 +854,7 @@ static void migrate_fd_cancel(MigrationState *s)
         if (!migration_is_setup_or_active(old_state)) {
             break;
         }
-        migrate_set_state(s, old_state, MIGRATION_STATUS_CANCELLING);
+        migrate_set_state(&s->state, old_state, MIGRATION_STATUS_CANCELLING);
     } while (s->state != MIGRATION_STATUS_CANCELLING);
 
     /*
@@ -927,7 +928,7 @@ MigrationState *migrate_init(const MigrationParams *params)
     s->migration_thread_running = false;
     s->last_req_rb = NULL;
 
-    migrate_set_state(s, MIGRATION_STATUS_NONE, MIGRATION_STATUS_SETUP);
+    migrate_set_state(&s->state, MIGRATION_STATUS_NONE, MIGRATION_STATUS_SETUP);
 
     QSIMPLEQ_INIT(&s->src_page_requests);
 
@@ -1026,7 +1027,8 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk,
     } else {
         error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "uri",
                    "a valid migration protocol");
-        migrate_set_state(s, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_FAILED);
+        migrate_set_state(&s->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_FAILED);
         return;
     }
 
@@ -1405,7 +1407,7 @@ static int postcopy_start(MigrationState *ms, bool *old_vm_running)
     int ret;
     const QEMUSizedBuffer *qsb;
     int64_t time_at_stop = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
-    migrate_set_state(ms, MIGRATION_STATUS_ACTIVE,
+    migrate_set_state(&ms->state, MIGRATION_STATUS_ACTIVE,
                       MIGRATION_STATUS_POSTCOPY_ACTIVE);
 
     trace_postcopy_start();
@@ -1496,7 +1498,7 @@ static int postcopy_start(MigrationState *ms, bool *old_vm_running)
     ret = qemu_file_get_error(ms->file);
     if (ret) {
         error_report("postcopy_start: Migration stream errored");
-        migrate_set_state(ms, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+        migrate_set_state(&ms->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
                               MIGRATION_STATUS_FAILED);
     }
 
@@ -1505,7 +1507,7 @@ static int postcopy_start(MigrationState *ms, bool *old_vm_running)
 fail_closefb:
     qemu_fclose(fb);
 fail:
-    migrate_set_state(ms, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+    migrate_set_state(&ms->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
                           MIGRATION_STATUS_FAILED);
     qemu_mutex_unlock_iothread();
     return -1;
@@ -1574,11 +1576,13 @@ static void migration_completion(MigrationState *s, int current_active_state,
         goto fail;
     }
 
-    migrate_set_state(s, current_active_state, MIGRATION_STATUS_COMPLETED);
+    migrate_set_state(&s->state, current_active_state,
+                      MIGRATION_STATUS_COMPLETED);
     return;
 
 fail:
-    migrate_set_state(s, current_active_state, MIGRATION_STATUS_FAILED);
+    migrate_set_state(&s->state, current_active_state,
+                      MIGRATION_STATUS_FAILED);
 }
 
 /*
@@ -1623,7 +1627,8 @@ static void *migration_thread(void *opaque)
 
     s->setup_time = qemu_clock_get_ms(QEMU_CLOCK_HOST) - setup_start;
     current_active_state = MIGRATION_STATUS_ACTIVE;
-    migrate_set_state(s, MIGRATION_STATUS_SETUP, MIGRATION_STATUS_ACTIVE);
+    migrate_set_state(&s->state, MIGRATION_STATUS_SETUP,
+                      MIGRATION_STATUS_ACTIVE);
 
     trace_migration_thread_setup_complete();
 
@@ -1666,7 +1671,8 @@ static void *migration_thread(void *opaque)
         }
 
         if (qemu_file_get_error(s->file)) {
-            migrate_set_state(s, current_active_state, MIGRATION_STATUS_FAILED);
+            migrate_set_state(&s->state, current_active_state,
+                              MIGRATION_STATUS_FAILED);
             trace_migration_thread_file_err();
             break;
         }
@@ -1747,7 +1753,7 @@ void migrate_fd_connect(MigrationState *s)
     if (migrate_postcopy_ram()) {
         if (open_return_path_on_source(s)) {
             error_report("Unable to open return-path for postcopy");
-            migrate_set_state(s, MIGRATION_STATUS_SETUP,
+            migrate_set_state(&s->state, MIGRATION_STATUS_SETUP,
                               MIGRATION_STATUS_FAILED);
             migrate_fd_cleanup(s);
             return;
Signature · 1 line
2.5.0
#3 lore raw

PATCH v2 2/6 migration: Add state records for migration incoming

In reply to Dr. David Alan Gilbert (git)

From: zhanghailiang <[email protected]>

For migration destination, we also need to know its state, we will use it in COLO.

Here we add a new member 'state' for MigrationIncomingState, and also use migrate_set_state() to modify its value.

dgilbert: Fixed early free of MigraitonIncomingState

---
 include/migration/migration.h |  1 +
 migration/migration.c         | 16 ++++++++++------
 2 files changed, 11 insertions(+), 6 deletions(-)
include/migration/migration.h +1 -0
index 8340432..4912e7a 100644
@@ -105,6 +105,7 @@ struct MigrationIncomingState {
     QemuMutex rp_mutex;    /* We send replies from multiple threads */
     void     *postcopy_tmp_page;
 
+    int state;
     /* See savevm.c */
     LoadStateEntry_Head loadvm_handlers;
 };
migration/migration.c +10 -6
index ba6d4f6..50fe218 100644
@@ -111,6 +111,7 @@ MigrationIncomingState *migration_incoming_state_new(QEMUFile* f)
 {
     mis_current = g_new0(MigrationIncomingState, 1);
     mis_current->from_src_file = f;
+    mis_current->state = MIGRATION_STATUS_NONE;
     QLIST_INIT(&mis_current->loadvm_handlers);
     qemu_mutex_init(&mis_current->rp_mutex);
     qemu_event_init(&mis_current->main_thread_load_event, false);
@@ -331,8 +332,8 @@ static void process_incoming_migration_co(void *opaque)
 
     mis = migration_incoming_state_new(f);
     postcopy_state_set(POSTCOPY_INCOMING_NONE);
-    migrate_generate_event(MIGRATION_STATUS_ACTIVE);
-
+    migrate_set_state(&mis->state, MIGRATION_STATUS_NONE,
+                      MIGRATION_STATUS_ACTIVE);
     ret = qemu_loadvm_state(f);
 
     ps = postcopy_state_get();
@@ -358,10 +359,10 @@ static void process_incoming_migration_co(void *opaque)
 
     qemu_fclose(f);
     free_xbzrle_decoded_buf();
-    migration_incoming_state_destroy();
 
     if (ret < 0) {
-        migrate_generate_event(MIGRATION_STATUS_FAILED);
+        migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
+                          MIGRATION_STATUS_FAILED);
         error_report("load of migration failed: %s", strerror(-ret));
         migrate_decompress_threads_join();
         exit(EXIT_FAILURE);
@@ -370,7 +371,8 @@ static void process_incoming_migration_co(void *opaque)
     /* Make sure all file formats flush their mutable metadata */
     bdrv_invalidate_cache_all(&local_err);
     if (local_err) {
-        migrate_generate_event(MIGRATION_STATUS_FAILED);
+        migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
+                          MIGRATION_STATUS_FAILED);
         error_report_err(local_err);
         migrate_decompress_threads_join();
         exit(EXIT_FAILURE);
@@ -402,7 +404,9 @@ static void process_incoming_migration_co(void *opaque)
      * observer sees this event they might start to prod at the VM assuming
      * it's ready to use.
      */
-    migrate_generate_event(MIGRATION_STATUS_COMPLETED);
+    migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
+                      MIGRATION_STATUS_COMPLETED);
+    migration_incoming_state_destroy();
 }
 
 void process_incoming_migration(QEMUFile *f)
Signature · 1 line
2.5.0
#4 lore raw

PATCH v2 3/6 Postcopy: Send events/change state on incoming side

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

I missed the calls to send migration events on the destination side as we enter postcopy.
Take care when adding them not to do it after state has been freed.

---
 migration/savevm.c | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)
migration/savevm.c +15 -7
index 0ad1b93..c469bad 100644
@@ -1399,6 +1399,8 @@ static void *postcopy_ram_listen_thread(void *opaque)
     MigrationIncomingState *mis = migration_incoming_get_current();
     int load_res;
 
+    migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE,
+                                   MIGRATION_STATUS_POSTCOPY_ACTIVE);
     qemu_sem_post(&mis->listen_thread_sem);
     trace_postcopy_ram_listen_thread_start();
 
@@ -1415,6 +1417,8 @@ static void *postcopy_ram_listen_thread(void *opaque)
     if (load_res < 0) {
         error_report("%s: loadvm failed: %d", __func__, load_res);
         qemu_file_set_error(f, load_res);
+        migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+                                       MIGRATION_STATUS_FAILED);
     } else {
         /*
          * This looks good, but it's possible that the device loading in the
@@ -1424,13 +1428,6 @@ static void *postcopy_ram_listen_thread(void *opaque)
         qemu_event_wait(&mis->main_thread_load_event);
     }
     postcopy_ram_incoming_cleanup(mis);
-    /*
-     * If everything has worked fine, then the main thread has waited
-     * for us to start, and we're the last use of the mis.
-     * (If something broke then qemu will have to exit anyway since it's
-     * got a bad migration state).
-     */
-    migration_incoming_state_destroy();
 
     if (load_res < 0) {
         /*
@@ -1442,6 +1439,17 @@ static void *postcopy_ram_listen_thread(void *opaque)
         exit(EXIT_FAILURE);
     }
 
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+                                   MIGRATION_STATUS_COMPLETED);
+    /*
+     * If everything has worked fine, then the main thread has waited
+     * for us to start, and we're the last use of the mis.
+     * (If something broke then qemu will have to exit anyway since it's
+     * got a bad migration state).
+     */
+    migration_incoming_state_destroy();
+
+
     return NULL;
 }
 
Signature · 1 line
2.5.0
#5 lore raw

PATCH v2 4/6 Migration: Emit event at start of pass

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy.

---
 docs/qmp-events.txt | 14 ++++++++++++++
 migration/ram.c     |  4 ++++
 qapi/event.json     | 13 +++++++++++++
 3 files changed, 31 insertions(+)
docs/qmp-events.txt +14 -0
index d2f1ce4..52eb7e2 100644
@@ -496,6 +496,20 @@ Example:
 {"timestamp": {"seconds": 1432121972, "microseconds": 744001},
  "event": "MIGRATION", "data": {"status": "completed"}}
 
+MIGRATION_PASS
+--------------
+
+Emitted from the source side of a migration at the start of each pass
+(when it syncs the dirty bitmap)
+
+Data: None.
+
+  - "pass": An incrementing count (starting at 1 on the first pass)
+
+Example:
+{"timestamp": {"seconds": 1449669631, "microseconds": 239225},
+ "event": "MIGRATION_PASS", "data": {"pass": 2}}
+
 STOP
 ----
 
migration/ram.c +4 -0
index 0490f00..102d1f2 100644
@@ -27,6 +27,7 @@
  */
 #include <stdint.h>
 #include <zlib.h>
+#include "qapi-event.h"
 #include "qemu/bitops.h"
 #include "qemu/bitmap.h"
 #include "qemu/timer.h"
@@ -682,6 +683,9 @@ static void migration_bitmap_sync(void)
         num_dirty_pages_period = 0;
     }
     s->dirty_sync_count = bitmap_sync_count;
+    if (migrate_use_events()) {
+        qapi_event_send_migration_pass(bitmap_sync_count, NULL);
+    }
 }
 
 /**
qapi/event.json +13 -0
index f0cef01..390fd45 100644
@@ -255,6 +255,19 @@
   'data': {'status': 'MigrationStatus'}}
 
 ##
+# @MIGRATION_PASS
+#
+# Emitted from the source side of a migration at the start of each pass
+# (when it syncs the dirty bitmap)
+#
+# @pass: An incrementing count (starting at 1 on the first pass)
+#
+# Since: 2.6
+##
+{ 'event': 'MIGRATION_PASS',
+  'data': { 'pass': 'int' } }
+
+##
 # @ACPI_DEVICE_OST
 #
 # Emitted when guest executes ACPI _OST method.
Signature · 1 line
2.5.0
Reviewed-by on v2 4/6
#6 lore raw

Re: [Qemu-devel] [PATCH v2 4/6] Migration: Emit event at start of pass

Quoted 7 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy.

Reviewed-by on v2 4/6
#7 lore raw

Re: [Qemu-devel] [PATCH v2 4/6] Migration: Emit event at start of pass

In reply to Dr. David Alan Gilbert (git)
Quoted 18 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy.

---
 docs/qmp-events.txt | 14 ++++++++++++++
 migration/ram.c     |  4 ++++
 qapi/event.json     | 13 +++++++++++++
 3 files changed, 31 insertions(+)
docs/qmp-events.txt +3 -0
+MIGRATION_PASS
+--------------
+

docs/qmp-events.txt Example: · line 511

Quoted 10 lines
docs/qmp-events.txt +10 -0
+Emitted from the source side of a migration at the start of each pass
+(when it syncs the dirty bitmap)
+
+Data: None.
+
+  - "pass": An incrementing count (starting at 1 on the first pass)
+
+Example:
+{"timestamp": {"seconds": 1449669631, "microseconds": 239225},
+ "event": "MIGRATION_PASS", "data": {"pass": 2}}

How frequently can this event be emitted? It is not under guest control, so it's harder to argue that it could be abused as a way for the guest to DoS the host. And hopefully it is slow enough (considering that each pass also requires lots more effort for actually transmitting state from source to dest), so I don't think we have to worry about adding rate-limiting to avoid overwhelming the monitor client.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#8 lore raw

Re: [Qemu-devel] [PATCH v2 4/6] Migration: Emit event at start of pass

Quoted 28 lines · * Eric Blake ([email protected]) wrote:
Quoted 27 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy.

---
 docs/qmp-events.txt | 14 ++++++++++++++
 migration/ram.c     |  4 ++++
 qapi/event.json     | 13 +++++++++++++
 3 files changed, 31 insertions(+)
docs/qmp-events.txt +12 -0
+MIGRATION_PASS
+--------------
+
+Emitted from the source side of a migration at the start of each pass
+(when it syncs the dirty bitmap)
+
+Data: None.
+
+  - "pass": An incrementing count (starting at 1 on the first pass)
+
+Example:
+{"timestamp": {"seconds": 1449669631, "microseconds": 239225},
Quoted 8 lines
Quoted 1 line

+ "event": "MIGRATION_PASS", "data": {"pass": 2}}

How frequently can this event be emitted? It is not under guest control, so it's harder to argue that it could be abused as a way for the guest to DoS the host. And hopefully it is slow enough (considering that each pass also requires lots more effort for actually transmitting state from source to dest), so I don't think we have to worry about adding rate-limiting to avoid overwhelming the monitor client.

The only way that you'd get a rapid repetition of the events is if the pass was very short. However a very short pass is normally the signal to exit the iterative loop of migration and send the last blob at the end of migration. And that definition of 'short' is determined by the downtime set by the management layer anyway.

Quoted 2 lines

Thanks.

Dave

Quoted 5 lines
Signature · 3 lines
-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#9 lore raw

Re: [Qemu-devel] [PATCH v2 4/6] Migration: Emit event at start of pass

Quoted 38 lines · Dr. David Alan Gilbert
Quoted 28 lines · * Eric Blake ([email protected]) wrote:
Quoted 27 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Emit an event each time we sync the dirty bitmap on the source; this helps libvirt use postcopy by giving it a kick when it might be a good idea to start the postcopy.

---
 docs/qmp-events.txt | 14 ++++++++++++++
 migration/ram.c     |  4 ++++
 qapi/event.json     | 13 +++++++++++++
 3 files changed, 31 insertions(+)
docs/qmp-events.txt +12 -0
+MIGRATION_PASS
+--------------
+
+Emitted from the source side of a migration at the start of each pass
+(when it syncs the dirty bitmap)
+
+Data: None.
+
+  - "pass": An incrementing count (starting at 1 on the first pass)
+
+Example:
+{"timestamp": {"seconds": 1449669631, "microseconds": 239225},
Quoted 8 lines
Quoted 1 line

+ "event": "MIGRATION_PASS", "data": {"pass": 2}}

How frequently can this event be emitted? It is not under guest control, so it's harder to argue that it could be abused as a way for the guest to DoS the host. And hopefully it is slow enough (considering that each pass also requires lots more effort for actually transmitting state from source to dest), so I don't think we have to worry about adding rate-limiting to avoid overwhelming the monitor client.

Quoted 5 lines

The only way that you'd get a rapid repetition of the events is if the pass was very short. However a very short pass is normally the signal to exit the iterative loop of migration and send the last blob at the end of migration. And that definition of 'short' is determined by the downtime set by the management layer anyway.

Yes, and this can also give us stats on how quickly migration is progressing.

Amit

#10 lore raw

PATCH v2 5/6 Use qemu_get_buffer_in_place for xbzrle data

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Avoid a data copy (if we're lucky) in the xbzrle code.

---
 migration/ram.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
migration/ram.c +4 -2
index 102d1f2..994552c 100644
@@ -2088,10 +2088,12 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr, void *host)
 {
     unsigned int xh_len;
     int xh_flags;
+    uint8_t *loaded_data;
 
     if (!xbzrle_decoded_buf) {
         xbzrle_decoded_buf = g_malloc(TARGET_PAGE_SIZE);
     }
+    loaded_data = xbzrle_decoded_buf;
 
     /* extract RLE header */
     xh_flags = qemu_get_byte(f);
@@ -2107,10 +2109,10 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr, void *host)
         return -1;
     }
     /* load data and decode */
-    qemu_get_buffer(f, xbzrle_decoded_buf, xh_len);
+    qemu_get_buffer_in_place(f, &loaded_data, xh_len);
 
     /* decode RLE */
-    if (xbzrle_decode_buffer(xbzrle_decoded_buf, xh_len, host,
+    if (xbzrle_decode_buffer(loaded_data, xh_len, host,
                              TARGET_PAGE_SIZE) == -1) {
         error_report("Failed to load XBZRLE page - decode error!");
         return -1;
Signature · 1 line
2.5.0
#11 lore raw

PATCH v2 6/6 multithread decompression: Avoid one copy

In reply to Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

qemu_get_buffer does a copy, we can avoid the memcpy, and we can then remove the extra buffer.

---
 migration/ram.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
migration/ram.c +3 -8
index 994552c..2da3b51 100644
@@ -286,7 +286,6 @@ static bool quit_comp_thread;
 static bool quit_decomp_thread;
 static DecompressParam *decomp_param;
 static QemuThread *decompress_threads;
-static uint8_t *compressed_data_buf;
 
 static int do_compress_ram_page(CompressParam *param);
 
@@ -2207,7 +2206,6 @@ void migrate_decompress_threads_create(void)
     thread_count = migrate_decompress_threads();
     decompress_threads = g_new0(QemuThread, thread_count);
     decomp_param = g_new0(DecompressParam, thread_count);
-    compressed_data_buf = g_malloc0(compressBound(TARGET_PAGE_SIZE));
     quit_decomp_thread = false;
     for (i = 0; i < thread_count; i++) {
         qemu_mutex_init(&decomp_param[i].mutex);
@@ -2238,13 +2236,11 @@ void migrate_decompress_threads_join(void)
     }
     g_free(decompress_threads);
     g_free(decomp_param);
-    g_free(compressed_data_buf);
     decompress_threads = NULL;
     decomp_param = NULL;
-    compressed_data_buf = NULL;
 }
 
-static void decompress_data_with_multi_threads(uint8_t *compbuf,
+static void decompress_data_with_multi_threads(QEMUFile *f,
                                                void *host, int len)
 {
     int idx, thread_count;
@@ -2253,7 +2249,7 @@ static void decompress_data_with_multi_threads(uint8_t *compbuf,
     while (true) {
         for (idx = 0; idx < thread_count; idx++) {
             if (!decomp_param[idx].start) {
-                memcpy(decomp_param[idx].compbuf, compbuf, len);
+                qemu_get_buffer(f, decomp_param[idx].compbuf, len);
                 decomp_param[idx].des = host;
                 decomp_param[idx].len = len;
                 start_decompression(&decomp_param[idx]);
@@ -2498,8 +2494,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
                 ret = -EINVAL;
                 break;
             }
-            qemu_get_buffer(f, compressed_data_buf, len);
-            decompress_data_with_multi_threads(compressed_data_buf, host, len);
+            decompress_data_with_multi_threads(f, host, len);
             break;
 
         case RAM_SAVE_FLAG_XBZRLE:
Signature · 1 line
2.5.0
Reviewed-by on the v2 series
#12 lore raw
In reply to Dr. David Alan Gilbert (git)
Quoted 11 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

This is a set of 4 small fixes and improvements I've been collecting during the 2.5 freeze; I've added in a couple of Zhanghailiang's patches that are easier to merge before rather than after.
(Of which one I've added a small fix of my own, see the message)

Jiri was after events from the incoming postcopy side to make it easy for libvirt; I'd forgotten to add them - Zhanghailiang's patches change the way the events get sent on the incoming side.

Quoted 7 lines

Jiri also wanted an event on the outgoing side at each pass, which makes it easy for libvirt to trigger the transition to postcopy.

The other two patches avoid a couple of buffer copies on the incoming side; although in both xbzrle and multithread compression it's rare that the destination is the slow path (but one patch is trivial and the other saves code).

Amit