Skip to contentSkip to conversation

migration: fast snapshot load

RFC PATCH Superseded Aadeshveer Singh 3 participants 24 messages Jun 18 – Jun 26

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

cover migration: fast snapshot load

This RFC implements a "fast snapshot load" mechanism to significantly reduce the perceived resume time of a VM from a snapshot file.

Currently, resuming a VM from a snapshot file requires loading all RAM pages into the QEMU instance before execution begins. This extension allows the user to run the VM nearly instantly by loading only the required device states up front and loading RAM pages lazily, by trapping access to pages that have not yet been loaded.

Using the Linux userfaultfd syscall, a fault thread catches all page faults caused by the guest and loads in the pages required to keep the VM running. Concurrently, an eager background thread iteratively loads all remaining pages into RAM so the guest does not have to depend on the fault thread indefinitely.

Much of code is reused from postcopy for fault handling and precopy for reading mapped ram file. Implementation revolves around two threads named the fault thread and eager load thread. Fault thread as name suggests catches page faults by the guest and serves them using userfaultfd. Postcopy fault thread is reused but instead of requesting source for a page it loads the page directly by reading form file. In order to remove the dependency of guest on fault thread indefinitely the eager load thread loads in the entire RAM sequentially, and after iterating through the entire RAM signals fault thread to exit and calls cleanup.

In order to prevent the case of a page being loaded twice(in the case when eager load thread is loading it and fault thread also tries to serve fault on same page) a bitmap called pending_bmap is used to track pages which are pending and not being loaded by any thread. Atomic operations on this bitmap allows coordination between threads to prevent any unwanted behaviours

This patch was tested using a Debian 13 bare minimum system and Fedora 44 KDE, snapshots for both are loaded successfully with no error.

Next Steps:
- Add testing framework, in qtest and unit tests
- Add support for postcopy-blocktime
- Update documentation

Future direction:
- Add support for hugepages
- Add support for multifd
- Add support for vhost-user

Aadeshveer Singh (5):
migration: add RAM Block fields and helpers for fast snapshot load
migration: add support for fault thread to load pages from disk
migration: add eager load thread for fast snapshot load
migration: write up code to run fast snapshot load in
qemu_loadvm_state
migration/tests: remove capability conflict test
postcopy-ram+mapped-ram

 include/system/ramblock.h          |   8 ++
 migration/migration.c              |  10 +-
 migration/migration.h              |   5 +
 migration/options.c                |  11 +-
 migration/options.h                |   1 +
 migration/postcopy-ram.c           | 167 ++++++++++++++++++++++++++---
 migration/postcopy-ram.h           |   2 +
 migration/qemu-file.c              |  10 +-
 migration/ram.c                    |  61 +++++++++--
 migration/savevm.c                 |  52 ++++++++-
 migration/savevm.h                 |   2 +
 migration/trace-events             |   2 +
 tests/qtest/migration/misc-tests.c |  52 ---------
 13 files changed, 283 insertions(+), 100 deletions(-)
Signature · 1 line
2.54.0
#1 lore raw

RFC PATCH 1/5 migration: add RAM Block fields and helpers for fast snapshot load

Add two fields per RAMBlock:

- nonzeropages: Mirrors the mapped-ram bitmap for storing which pages are present in file and which are zero.
- pending_bmap: Bitmap to store internal state of which pages have been read by some thread to ensure coordination between threads.

Both fields are allocated and initialized in ram_load_setup and freed in ram_load_cleanup. nonzeropages is populated in parse_ramblock_mapped_ram eliminating the use of a temporary bitmap.

Change ram_load() to load using ram_load_precopy() in case of fast snapshot load.

Also add migrate_fast_snapshot_load() returning true when both postcopy-ram and mapped-ram capabilities are set.

Update qemu_get_buffer_at() to not set error to make it thread safe. All the callers of qemu_get_buffer_at(), take care of error handling.

---
 include/system/ramblock.h |  8 +++++
 migration/options.c       |  5 ++++
 migration/options.h       |  1 +
 migration/qemu-file.c     | 10 +------
 migration/ram.c           | 61 ++++++++++++++++++++++++++++++++-------
 5 files changed, 65 insertions(+), 20 deletions(-)
include/system/ramblock.h +8 -0
index 4435f8d55f..73275d0459 100644
@@ -60,6 +60,14 @@ struct RAMBlock {
 
     /* Bitmap of already received pages.  Only used on destination side. */
     unsigned long *receivedmap;
+    /* Bitmap of zero pages. Used for fast snapshot load. */
+    unsigned long *nonzeropages;
+    /*
+     * Bitmap for pages that are yet to be read from disk. It is required for
+     * fault thread and eager thread to keep note of which pages are currently
+     * being read. Used by fast snapshot load.
+     */
+    unsigned long *pending_bmap;
 
     /*
      * bitmap to track already cleared dirty bitmap.  When the bit is
migration/options.c +5 -0
index 5cbfd29099..5f80dd5b42 100644
@@ -467,6 +467,11 @@ bool migrate_rdma(void)
     return s->rdma_migration;
 }
 
+bool migrate_fast_snapshot_load(void)
+{
+    return migrate_mapped_ram() && migrate_postcopy_ram();
+}
+
 typedef enum WriteTrackingSupport {
     WT_SUPPORT_UNKNOWN = 0,
     WT_SUPPORT_ABSENT,
migration/options.h +1 -0
index b46221998a..a81ca40d23 100644
@@ -54,6 +54,7 @@ bool migrate_multifd_flush_after_each_section(void);
 bool migrate_postcopy(void);
 bool migrate_rdma(void);
 bool migrate_tls(void);
+bool migrate_fast_snapshot_load(void);
 
 /* capabilities helpers */
 
migration/qemu-file.c +1 -9
index d5a48115bd..602ece1b74 100644
@@ -553,17 +553,9 @@ void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen,
 size_t qemu_get_buffer_at(QEMUFile *f, uint8_t *buf, size_t buflen,
                           off_t pos)
 {
-    Error *err = NULL;
-
-    if (f->last_error) {
-        return 0;
-    }
-
-    if (qio_channel_pread_all(f->ioc, buf, buflen, pos, &err) < 0) {
-        qemu_file_set_error_obj(f, -EIO, err);
+    if (qio_channel_pread_all(f->ioc, buf, buflen, pos, NULL) < 0) {
         return 0;
     }
-
     return buflen;
 }
 
migration/ram.c +50 -11

123 lines not on this page · raw message

Signature · 1 line
2.54.0
#2 lore raw

Re: [RFC PATCH 1/5] migration: add RAM Block fields and helpers for fast snapshot load

Quoted 34 lines · Aadeshveer Singh

Add two fields per RAMBlock:

- nonzeropages: Mirrors the mapped-ram bitmap for storing which pages are present in file and which are zero.
- pending_bmap: Bitmap to store internal state of which pages have been read by some thread to ensure coordination between threads.

Both fields are allocated and initialized in ram_load_setup and freed in ram_load_cleanup. nonzeropages is populated in parse_ramblock_mapped_ram eliminating the use of a temporary bitmap.

Change ram_load() to load using ram_load_precopy() in case of fast snapshot load.

Also add migrate_fast_snapshot_load() returning true when both postcopy-ram and mapped-ram capabilities are set.

Update qemu_get_buffer_at() to not set error to make it thread safe. All the callers of qemu_get_buffer_at(), take care of error handling.

---
 include/system/ramblock.h |  8 +++++
 migration/options.c       |  5 ++++
 migration/options.h       |  1 +
 migration/qemu-file.c     | 10 +------
 migration/ram.c           | 61 ++++++++++++++++++++++++++++++++-------
 5 files changed, 65 insertions(+), 20 deletions(-)
include/system/ramblock.h +0 -0
index 4435f8d55f..73275d0459 100644

include/system/ramblock.h struct RAMBlock { · line 64

Quoted 5 lines
include/system/ramblock.h +2 -0
@@ -60,6 +60,14 @@ struct RAMBlock {
 
     /* Bitmap of already received pages.  Only used on destination side. */
     unsigned long *receivedmap;
+    /* Bitmap of zero pages. Used for fast snapshot load. */
+    unsigned long *nonzeropages;

We have file_bmap, only used on source for now. I think we can safely reuse it by caching the pointer allocated.

include/system/ramblock.h struct RAMBlock { · line 70

Quoted 6 lines
include/system/ramblock.h +6 -0
+    /*
+     * Bitmap for pages that are yet to be read from disk. It is required for
+     * fault thread and eager thread to keep note of which pages are currently
+     * being read. Used by fast snapshot load.
+     */
+    unsigned long *pending_bmap;

We have receivedmap right above, and it's always allocated on dest. IIUC we can directly use it.

It's also already set by uffd helpers, see qemu_ufd_copy_ioctl(). Currently it's a bit ugly put under a "if (!ret)".. if you want you can clean it up a bit.

There, we may want to skip page_requested or page_request_mutex operations for file load because they're not necessary.

Quoted 36 lines
include/system/ramblock.h +0 -0
     /*
      * bitmap to track already cleared dirty bitmap.  When the bit is
migration/options.c +5 -0
index 5cbfd29099..5f80dd5b42 100644
@@ -467,6 +467,11 @@ bool migrate_rdma(void)
     return s->rdma_migration;
 }
 
+bool migrate_fast_snapshot_load(void)
+{
+    return migrate_mapped_ram() && migrate_postcopy_ram();
+}
+
 typedef enum WriteTrackingSupport {
     WT_SUPPORT_UNKNOWN = 0,
     WT_SUPPORT_ABSENT,
migration/options.h +1 -0
index b46221998a..a81ca40d23 100644
@@ -54,6 +54,7 @@ bool migrate_multifd_flush_after_each_section(void);
 bool migrate_postcopy(void);
 bool migrate_rdma(void);
 bool migrate_tls(void);
+bool migrate_fast_snapshot_load(void);
 
 /* capabilities helpers */
 
migration/qemu-file.c +0 -0
index d5a48115bd..602ece1b74 100644

migration/qemu-file.c void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, · line 560

Quoted 8 lines
migration/qemu-file.c +0 -5
@@ -553,17 +553,9 @@ void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen,
 size_t qemu_get_buffer_at(QEMUFile *f, uint8_t *buf, size_t buflen,
                           off_t pos)
 {
-    Error *err = NULL;
-
-    if (f->last_error) {
-        return 0;
-    }

Let's keep this line, this is thread-safe and if we have a prior error it seems still reasonable to return immediately. Then leave below one line change to be a small separate patch would be nicer.

Quoted 15 lines
migration/qemu-file.c +1 -4
-
-    if (qio_channel_pread_all(f->ioc, buf, buflen, pos, &err) < 0) {
-        qemu_file_set_error_obj(f, -EIO, err);
+    if (qio_channel_pread_all(f->ioc, buf, buflen, pos, NULL) < 0) {
         return 0;
     }
-
     return buflen;
 }
 
migration/ram.c +0 -0
index fc38ffbf8a..c2bacf3dfc 100644

migration/ram.c int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque) · line 260

Quoted 9 lines
migration/ram.c +6 -0
@@ -252,6 +252,31 @@ int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque)
     return ret;
 }
 
+static void ramblock_non_zero_map_init(void)
+{
+    RAMBlock *rb;
+
+    RAMBLOCK_FOREACH_NOT_IGNORED(rb)
+    {

Let's follow QEMU's coding style, see:

https://qemu-project.gitlab.io/qemu/devel/style.html#block-structure

So:

    RAMBLOCK_FOREACH_NOT_IGNORED(rb) {
        ...
    }

The other thing is, if we pre-allocate this bmap, then it might be wise to double check this size suites what to be read later in the snapshot file header, in parse_ramblock_mapped_ram(). We can fail the load immediately if we found the size requested in header is larger than what is allocated here (aka, max_length).

We have some length checks, like:

    if (length != block->used_length) {
        ret = qemu_ram_resize(block, length, &local_err);
        if (local_err) {
            error_report_err(local_err);
            return ret;
        }
    }

But I think that != isn't as safe.. we should likely check max_length first. This can also be a separate patch just to introduce the file_bmap to RAMBlock, so that sync mapped-ram can already use it.

migration/ram.c int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque) · line 272

Quoted 12 lines
migration/ram.c +12 -0
+        assert(!rb->nonzeropages);
+        size_t size = rb->max_length >> qemu_target_page_bits();
+        rb->nonzeropages = bitmap_new(size);
+    }
+}
+
+static void ramblock_pending_bmap_init(void)
+{
+    RAMBlock *rb;
+
+    RAMBLOCK_FOREACH_NOT_IGNORED(rb)
+    {

Similar indent issue, IIUC this function can be dropped after reusing receivedmap.

Quoted 24 lines
migration/ram.c +13 -0
+        assert(!rb->pending_bmap);
+        size_t size = rb->max_length >> qemu_target_page_bits();
+        rb->pending_bmap = bitmap_new(size);
+        bitmap_set(rb->pending_bmap, 0, size);
+    }
+}
+
 static void ramblock_recv_map_init(void)
 {
     RAMBlock *rb;
@@ -3749,6 +3774,12 @@ static int ram_load_setup(QEMUFile *f, void *opaque, Error **errp)
 {
     xbzrle_load_setup();
     ramblock_recv_map_init();
+    if (migrate_mapped_ram()) {
+        ramblock_non_zero_map_init();
+    }
+    if (migrate_fast_snapshot_load()) {
+        ramblock_pending_bmap_init();
+    }
 
     return 0;
 }

migration/ram.c static int ram_load_cleanup(void *opaque) · line 3805

Quoted 7 lines
migration/ram.c +4 -0
@@ -3768,6 +3799,10 @@ static int ram_load_cleanup(void *opaque)
     RAMBLOCK_FOREACH_NOT_IGNORED(rb) {
         g_free(rb->receivedmap);
         rb->receivedmap = NULL;
+        g_free(rb->pending_bmap);
+        rb->pending_bmap = NULL;
+        g_free(rb->nonzeropages);
+        rb->nonzeropages = NULL;

For new code, IMHO we can use g_clear_pointer().

migration/ram.c static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, · line 4140

Quoted 9 lines
migration/ram.c +1 -1
     }
 
     return 0;
@@ -4102,7 +4137,7 @@ static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
             host = host_from_ram_block_offset(block, offset);
             if (!host) {
                 error_setg(errp, "page outside of ramblock %s range",
-                           block->idstr);
+                            block->idstr);

Looks like irrelevant line changes, let's try to not touch them if they're not needed.

migration/ram.c static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, · line 4151

Quoted 13 lines
migration/ram.c +2 -2
                 return false;
             }
 
@@ -4110,10 +4145,10 @@ static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
 
             if (migrate_multifd()) {
                 read = ram_load_multifd_pages(host, size,
-                                              block->pages_offset + offset);
+                                                block->pages_offset + offset);
             } else {
                 read = qemu_get_buffer_at(f, host, size,
-                                          block->pages_offset + offset);
+                                            block->pages_offset + offset);

Same here.

Quoted 16 lines
migration/ram.c +0 -2
             }
 
             if (!read) {
@@ -4142,7 +4177,6 @@ err:
 static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
                                       ram_addr_t length, Error **errp)
 {
-    g_autofree unsigned long *bitmap = NULL;
     MappedRamHeader header;
     size_t bitmap_size;
     long num_pages;
@@ -4174,15 +4208,18 @@ static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
     num_pages = length / header.page_size;
     bitmap_size = BITS_TO_LONGS(num_pages) * sizeof(unsigned long);
 
-    bitmap = g_malloc0(bitmap_size);

migration/ram.c static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, · line 4217

Quoted 10 lines
migration/ram.c +2 -3
-    if (qemu_get_buffer_at(f, (uint8_t *)bitmap, bitmap_size,
+    if (qemu_get_buffer_at(f, (uint8_t *)block->nonzeropages, bitmap_size,
                            header.bitmap_offset) != bitmap_size) {
         error_setg(errp, "Error reading dirty bitmap");
         return;
     }
 
-    if (!read_ramblock_mapped_ram(f, block, num_pages, bitmap, errp)) {
-        return;
+    if (!migrate_fast_snapshot_load()) {

Nitpick: when reaching here it must have mapped-ram enabled, we can directly check migrate_postcopy_ram() here.

Quoted 9 lines
migration/ram.c +5 -0
+        /* Do not load RAM during setup for fast snapshot load */
+        if (!read_ramblock_mapped_ram(f, block, num_pages, block->nonzeropages,
+                                      errp)) {
+            return;
+        }
     }
 
     /* Skip pages array */

migration/ram.c static int ram_load(QEMUFile *f, void *opaque, int version_id) · line 4504

Quoted 10 lines
migration/ram.c +4 -2
@@ -4460,9 +4497,11 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
     static uint64_t seq_iter;
     /*
      * If system is running in postcopy mode, page inserts to host memory must
-     * be atomic
+     * be atomic. However, fast snapshot load uses the mapped ram precopy like
+     * path to read block headers and populating bitmaps.
      */
-    bool postcopy_running = postcopy_is_running();
+    bool load_using_postcopy =
+        postcopy_is_running() && !migrate_fast_snapshot_load();

I think this change is correct, it's just that I believe it'll be hard to follow for most readers.

Here, what you really need is to parse the ramblock headers only, it's just that we used to have both steps (RAM setup + precopy) all processed in the same ram_load_precopy() function, and here you want to leverage the "RAM setup" phase only.

Maybe rename the bool to load_postcopy_pages? That may help to explain why in your case postcopy-ram=on but you don't set this bool to true: it's because your case doesn't need to load any page in postcopy way.

Maybe something like this to be slightly more verbose:

  bool ram_should_load_postcopy_pages(void)
  {
      /* This is pure precopy, we don't need to load pages in postcopy way */
      if (!postcopy_is_running()) {
          return false;
      }
      /*
       * This is postcopy, but when with mapped-ram, pages are not loaded
       * in the migration stream here, but done separately in a thread eagerly
       * reading pages from the snapshot.  Here, we only need to read the
       * ram headers, reusing the precopy code.  TODO: when we have separate 
       * function to parse RAM headers we should switch to that.
       */
       if (migrate_mapped_ram()) {
           return false;
       }

       /*
        * Genuine network postcopy, we will load pages in this current stream
        * and they need to be done in postcopy way.
        */
       return true;

}

  ...
  bool load_postcopy_pages = ram_expects_postcopy_pages();

Thanks,

Quoted 15 lines
migration/ram.c +1 -1
     seq_iter++;
 
@@ -4478,7 +4517,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
      */
     trace_ram_load_start();
     WITH_RCU_READ_LOCK_GUARD() {
-        if (postcopy_running) {
+        if (load_using_postcopy) {
             /*
              * Note!  Here RAM_CHANNEL_PRECOPY is the precopy channel of
              * postcopy migration, we have another RAM_CHANNEL_POSTCOPY to
Signature · 1 line
2.54.0
Signature · 1 line
Peter Xu
#3 lore raw

Re: [RFC PATCH 1/5] migration: add RAM Block fields and helpers for fast snapshot load

Quoted 37 lines · Peter Xu
Quoted 35 lines · Aadeshveer Singh

Add two fields per RAMBlock:

- nonzeropages: Mirrors the mapped-ram bitmap for storing which pages are present in file and which are zero.
- pending_bmap: Bitmap to store internal state of which pages have been read by some thread to ensure coordination between threads.

Both fields are allocated and initialized in ram_load_setup and freed in ram_load_cleanup. nonzeropages is populated in parse_ramblock_mapped_ram eliminating the use of a temporary bitmap.

Change ram_load() to load using ram_load_precopy() in case of fast snapshot load.

Also add migrate_fast_snapshot_load() returning true when both postcopy-ram and mapped-ram capabilities are set.

Update qemu_get_buffer_at() to not set error to make it thread safe. All the callers of qemu_get_buffer_at(), take care of error handling.

---
 include/system/ramblock.h |  8 +++++
 migration/options.c       |  5 ++++
 migration/options.h       |  1 +
 migration/qemu-file.c     | 10 +------
 migration/ram.c           | 61 ++++++++++++++++++++++++++++++++-------
 5 files changed, 65 insertions(+), 20 deletions(-)
include/system/ramblock.h +0 -0
index 4435f8d55f..73275d0459 100644
@@ -60,6 +60,14 @@ struct RAMBlock {
 
Quoted 7 lines

include/system/ramblock.h struct RAMBlock { · line 64

Quoted 4 lines
include/system/ramblock.h +2 -0
     /* Bitmap of already received pages.  Only used on destination side. */
     unsigned long *receivedmap;
+    /* Bitmap of zero pages. Used for fast snapshot load. */
+    unsigned long *nonzeropages;

We have file_bmap, only used on source for now. I think we can safely reuse it by caching the pointer allocated.

Agreed will drop nonzeropages and reuse file_bmap for this in v2

Quoted 11 lines

include/system/ramblock.h struct RAMBlock { · line 70

Quoted 7 lines
include/system/ramblock.h +6 -0
+    /*
+     * Bitmap for pages that are yet to be read from disk. It is required for
+     * fault thread and eager thread to keep note of which pages are currently
+     * being read. Used by fast snapshot load.
+     */
+    unsigned long *pending_bmap;

We have receivedmap right above, and it's always allocated on dest. IIUC we can directly use it.

Quoted 7 lines

It's also already set by uffd helpers, see qemu_ufd_copy_ioctl(). Currently it's a bit ugly put under a "if (!ret)".. if you want you can clean it up a bit.

There, we may want to skip page_requested or page_request_mutex operations for file load because they're not necessary.

I looked into reusing receivedmap here instead of creating pending_bmap, but doing so creates a race condition that breaks the postcopy blocktime calculations.

receivedmap strictly tracks pages that have been successfully placed in the VM. My pending_bmap tracks pages that any thread is actively loading (reading from disk).

If we use receivedmap to track in-load pages, and a vCPU faults on a page that the eager thread is currently reading, the fault thread will see the bit set in receivedmap. It will then incorrectly assume the page is already in the VM and skip tracking the blocktime for that fault, even though the eager thread hasn't actually called place_page yet. Keeping pending_bmap separate ensures the fault thread accurately tracks blocktime until placement is actually complete.

Regarding the cleanup in qemu_ufd_copy_ioctl(), I agree we can skip the page_requested operations for the file load. I also noticed a potential race where if the eager thread places a page just before the fault thread attempts to, we might hit an assert there. I will look into returning safely instead of asserting to harden this, and I will include the cleanup in v2.

Quoted 296 lines
Quoted 36 lines
include/system/ramblock.h +0 -0
     /*
      * bitmap to track already cleared dirty bitmap.  When the bit is
migration/options.c +5 -0
index 5cbfd29099..5f80dd5b42 100644
@@ -467,6 +467,11 @@ bool migrate_rdma(void)
     return s->rdma_migration;
 }
 
+bool migrate_fast_snapshot_load(void)
+{
+    return migrate_mapped_ram() && migrate_postcopy_ram();
+}
+
 typedef enum WriteTrackingSupport {
     WT_SUPPORT_UNKNOWN = 0,
     WT_SUPPORT_ABSENT,
migration/options.h +1 -0
index b46221998a..a81ca40d23 100644
@@ -54,6 +54,7 @@ bool migrate_multifd_flush_after_each_section(void);
 bool migrate_postcopy(void);
 bool migrate_rdma(void);
 bool migrate_tls(void);
+bool migrate_fast_snapshot_load(void);
 
 /* capabilities helpers */
 
migration/qemu-file.c +0 -0
index d5a48115bd..602ece1b74 100644

migration/qemu-file.c void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen, · line 560

Quoted 8 lines
migration/qemu-file.c +0 -5
@@ -553,17 +553,9 @@ void qemu_put_buffer_at(QEMUFile *f, const uint8_t *buf, size_t buflen,
 size_t qemu_get_buffer_at(QEMUFile *f, uint8_t *buf, size_t buflen,
                           off_t pos)
 {
-    Error *err = NULL;
-
-    if (f->last_error) {
-        return 0;
-    }

Let's keep this line, this is thread-safe and if we have a prior error it seems still reasonable to return immediately. Then leave below one line change to be a small separate patch would be nicer.

Quoted 15 lines
migration/qemu-file.c +1 -4
-
-    if (qio_channel_pread_all(f->ioc, buf, buflen, pos, &err) < 0) {
-        qemu_file_set_error_obj(f, -EIO, err);
+    if (qio_channel_pread_all(f->ioc, buf, buflen, pos, NULL) < 0) {
         return 0;
     }
-
     return buflen;
 }
 
migration/ram.c +0 -0
index fc38ffbf8a..c2bacf3dfc 100644

migration/ram.c int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque) · line 260

Quoted 9 lines
migration/ram.c +6 -0
@@ -252,6 +252,31 @@ int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque)
     return ret;
 }
 
+static void ramblock_non_zero_map_init(void)
+{
+    RAMBlock *rb;
+
+    RAMBLOCK_FOREACH_NOT_IGNORED(rb)
+    {

Let's follow QEMU's coding style, see:

https://qemu-project.gitlab.io/qemu/devel/style.html#block-structure

So:

    RAMBLOCK_FOREACH_NOT_IGNORED(rb) {
        ...
    }

The other thing is, if we pre-allocate this bmap, then it might be wise to double check this size suites what to be read later in the snapshot file header, in parse_ramblock_mapped_ram(). We can fail the load immediately if we found the size requested in header is larger than what is allocated here (aka, max_length).

We have some length checks, like:

    if (length != block->used_length) {
        ret = qemu_ram_resize(block, length, &local_err);
        if (local_err) {
            error_report_err(local_err);
            return ret;
        }
    }

But I think that != isn't as safe.. we should likely check max_length first. This can also be a separate patch just to introduce the file_bmap to RAMBlock, so that sync mapped-ram can already use it.

migration/ram.c int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque) · line 272

Quoted 12 lines
migration/ram.c +12 -0
+        assert(!rb->nonzeropages);
+        size_t size = rb->max_length >> qemu_target_page_bits();
+        rb->nonzeropages = bitmap_new(size);
+    }
+}
+
+static void ramblock_pending_bmap_init(void)
+{
+    RAMBlock *rb;
+
+    RAMBLOCK_FOREACH_NOT_IGNORED(rb)
+    {

Similar indent issue, IIUC this function can be dropped after reusing receivedmap.

Quoted 24 lines
migration/ram.c +13 -0
+        assert(!rb->pending_bmap);
+        size_t size = rb->max_length >> qemu_target_page_bits();
+        rb->pending_bmap = bitmap_new(size);
+        bitmap_set(rb->pending_bmap, 0, size);
+    }
+}
+
 static void ramblock_recv_map_init(void)
 {
     RAMBlock *rb;
@@ -3749,6 +3774,12 @@ static int ram_load_setup(QEMUFile *f, void *opaque, Error **errp)
 {
     xbzrle_load_setup();
     ramblock_recv_map_init();
+    if (migrate_mapped_ram()) {
+        ramblock_non_zero_map_init();
+    }
+    if (migrate_fast_snapshot_load()) {
+        ramblock_pending_bmap_init();
+    }
 
     return 0;
 }

migration/ram.c static int ram_load_cleanup(void *opaque) · line 3805

Quoted 7 lines
migration/ram.c +4 -0
@@ -3768,6 +3799,10 @@ static int ram_load_cleanup(void *opaque)
     RAMBLOCK_FOREACH_NOT_IGNORED(rb) {
         g_free(rb->receivedmap);
         rb->receivedmap = NULL;
+        g_free(rb->pending_bmap);
+        rb->pending_bmap = NULL;
+        g_free(rb->nonzeropages);
+        rb->nonzeropages = NULL;

For new code, IMHO we can use g_clear_pointer().

migration/ram.c static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, · line 4140

Quoted 9 lines
migration/ram.c +1 -1
     }
 
     return 0;
@@ -4102,7 +4137,7 @@ static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
             host = host_from_ram_block_offset(block, offset);
             if (!host) {
                 error_setg(errp, "page outside of ramblock %s range",
-                           block->idstr);
+                            block->idstr);

Looks like irrelevant line changes, let's try to not touch them if they're not needed.

migration/ram.c static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, · line 4151

Quoted 13 lines
migration/ram.c +2 -2
                 return false;
             }
 
@@ -4110,10 +4145,10 @@ static bool read_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
 
             if (migrate_multifd()) {
                 read = ram_load_multifd_pages(host, size,
-                                              block->pages_offset + offset);
+                                                block->pages_offset + offset);
             } else {
                 read = qemu_get_buffer_at(f, host, size,
-                                          block->pages_offset + offset);
+                                            block->pages_offset + offset);

Same here.

Quoted 16 lines
migration/ram.c +0 -2
             }
 
             if (!read) {
@@ -4142,7 +4177,6 @@ err:
 static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
                                       ram_addr_t length, Error **errp)
 {
-    g_autofree unsigned long *bitmap = NULL;
     MappedRamHeader header;
     size_t bitmap_size;
     long num_pages;
@@ -4174,15 +4208,18 @@ static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block,
     num_pages = length / header.page_size;
     bitmap_size = BITS_TO_LONGS(num_pages) * sizeof(unsigned long);
 
-    bitmap = g_malloc0(bitmap_size);

migration/ram.c static void parse_ramblock_mapped_ram(QEMUFile *f, RAMBlock *block, · line 4217

Quoted 10 lines
migration/ram.c +2 -3
-    if (qemu_get_buffer_at(f, (uint8_t *)bitmap, bitmap_size,
+    if (qemu_get_buffer_at(f, (uint8_t *)block->nonzeropages, bitmap_size,
                            header.bitmap_offset) != bitmap_size) {
         error_setg(errp, "Error reading dirty bitmap");
         return;
     }
 
-    if (!read_ramblock_mapped_ram(f, block, num_pages, bitmap, errp)) {
-        return;
+    if (!migrate_fast_snapshot_load()) {

Nitpick: when reaching here it must have mapped-ram enabled, we can directly check migrate_postcopy_ram() here.

Quoted 9 lines
migration/ram.c +5 -0
+        /* Do not load RAM during setup for fast snapshot load */
+        if (!read_ramblock_mapped_ram(f, block, num_pages, block->nonzeropages,
+                                      errp)) {
+            return;
+        }
     }
 
     /* Skip pages array */

migration/ram.c static int ram_load(QEMUFile *f, void *opaque, int version_id) · line 4504

Quoted 10 lines
migration/ram.c +4 -2
@@ -4460,9 +4497,11 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
     static uint64_t seq_iter;
     /*
      * If system is running in postcopy mode, page inserts to host memory must
-     * be atomic
+     * be atomic. However, fast snapshot load uses the mapped ram precopy like
+     * path to read block headers and populating bitmaps.
      */
-    bool postcopy_running = postcopy_is_running();
+    bool load_using_postcopy =
+        postcopy_is_running() && !migrate_fast_snapshot_load();

I think this change is correct, it's just that I believe it'll be hard to follow for most readers.

Here, what you really need is to parse the ramblock headers only, it's just that we used to have both steps (RAM setup + precopy) all processed in the same ram_load_precopy() function, and here you want to leverage the "RAM setup" phase only.

Maybe rename the bool to load_postcopy_pages? That may help to explain why in your case postcopy-ram=on but you don't set this bool to true: it's because your case doesn't need to load any page in postcopy way.

Maybe something like this to be slightly more verbose:

  bool ram_should_load_postcopy_pages(void)
  {
      /* This is pure precopy, we don't need to load pages in postcopy way */
      if (!postcopy_is_running()) {
          return false;
      }
      /*
       * This is postcopy, but when with mapped-ram, pages are not loaded
       * in the migration stream here, but done separately in a thread eagerly
       * reading pages from the snapshot.  Here, we only need to read the
       * ram headers, reusing the precopy code.  TODO: when we have separate
       * function to parse RAM headers we should switch to that.
       */
       if (migrate_mapped_ram()) {
           return false;
       }

       /*
        * Genuine network postcopy, we will load pages in this current stream
        * and they need to be done in postcopy way.
        */
       return true;

}

  ...
  bool load_postcopy_pages = ram_expects_postcopy_pages();

Thanks,

Quoted 13 lines
migration/ram.c +1 -2
     seq_iter++;
 
@@ -4478,7 +4517,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
      */
     trace_ram_load_start();
     WITH_RCU_READ_LOCK_GUARD() {
-        if (postcopy_running) {
+        if (load_using_postcopy) {
             /*
              * Note!  Here RAM_CHANNEL_PRECOPY is the precopy channel of
              * postcopy migration, we have another RAM_CHANNEL_POSTCOPY to
--
Quoted 2 lines

2.54.0

--
Peter Xu

#4 lore raw

Re: [RFC PATCH 1/5] migration: add RAM Block fields and helpers for fast snapshot load

Quoted 66 lines · Aadeshveer Singh
Quoted 37 lines · Peter Xu
Quoted 35 lines · Aadeshveer Singh

Add two fields per RAMBlock:

- nonzeropages: Mirrors the mapped-ram bitmap for storing which pages are present in file and which are zero.
- pending_bmap: Bitmap to store internal state of which pages have been read by some thread to ensure coordination between threads.

Both fields are allocated and initialized in ram_load_setup and freed in ram_load_cleanup. nonzeropages is populated in parse_ramblock_mapped_ram eliminating the use of a temporary bitmap.

Change ram_load() to load using ram_load_precopy() in case of fast snapshot load.

Also add migrate_fast_snapshot_load() returning true when both postcopy-ram and mapped-ram capabilities are set.

Update qemu_get_buffer_at() to not set error to make it thread safe. All the callers of qemu_get_buffer_at(), take care of error handling.

---
 include/system/ramblock.h |  8 +++++
 migration/options.c       |  5 ++++
 migration/options.h       |  1 +
 migration/qemu-file.c     | 10 +------
 migration/ram.c           | 61 ++++++++++++++++++++++++++++++++-------
 5 files changed, 65 insertions(+), 20 deletions(-)
include/system/ramblock.h +0 -0
index 4435f8d55f..73275d0459 100644
@@ -60,6 +60,14 @@ struct RAMBlock {
 
Quoted 7 lines

include/system/ramblock.h struct RAMBlock { · line 64

Quoted 4 lines
include/system/ramblock.h +2 -0
     /* Bitmap of already received pages.  Only used on destination side. */
     unsigned long *receivedmap;
+    /* Bitmap of zero pages. Used for fast snapshot load. */
+    unsigned long *nonzeropages;

We have file_bmap, only used on source for now. I think we can safely reuse it by caching the pointer allocated.

Agreed will drop nonzeropages and reuse file_bmap for this in v2

Quoted 11 lines

include/system/ramblock.h struct RAMBlock { · line 70

Quoted 7 lines
include/system/ramblock.h +6 -0
+    /*
+     * Bitmap for pages that are yet to be read from disk. It is required for
+     * fault thread and eager thread to keep note of which pages are currently
+     * being read. Used by fast snapshot load.
+     */
+    unsigned long *pending_bmap;

We have receivedmap right above, and it's always allocated on dest. IIUC we can directly use it.

Quoted 7 lines

It's also already set by uffd helpers, see qemu_ufd_copy_ioctl(). Currently it's a bit ugly put under a "if (!ret)".. if you want you can clean it up a bit.

There, we may want to skip page_requested or page_request_mutex operations for file load because they're not necessary.

Quoted 7 lines

I looked into reusing receivedmap here instead of creating pending_bmap, but doing so creates a race condition that breaks the postcopy blocktime calculations.

receivedmap strictly tracks pages that have been successfully placed in the VM. My pending_bmap tracks pages that any thread is actively loading (reading from disk).

Yes, I think if we want to reuse the bitmap, we need to redefine receivedmap a little bit.

Remote postcopy needs to remember it not only to maintain the GTree for page_requested, but also when postcopy is interrupted it needs to sync the bitmap to source to know what exactly have landed destination. Here setting the bit _after_ the placement is critical.

In your case, you don't need to remember "what pages we have loaded".

So if we could redefine receivedmap in your case to be "who will place this page, eager load thread or fault thread", then it could work.

Said that, maybe this will need some work to make it work, you can evaluate how much needed before going too deep. Then if you think it's still easier to have the new bitmap, then it's still ok to consider having it.

It may also matter on how you plan to refactor qemu_ufd_copy_ioctl(), maybe. So you can take all these problems together in mind when considering the final decision.

Quoted 9 lines

If we use receivedmap to track in-load pages, and a vCPU faults on a page that the eager thread is currently reading, the fault thread will see the bit set in receivedmap. It will then incorrectly assume the page is already in the VM and skip tracking the blocktime for that fault, even though the eager thread hasn't actually called place_page yet. Keeping pending_bmap separate ensures the fault thread accurately tracks blocktime until placement is actually complete.

Quoted 6 lines

Regarding the cleanup in qemu_ufd_copy_ioctl(), I agree we can skip the page_requested operations for the file load. I also noticed a potential race where if the eager thread places a page just before the fault thread attempts to, we might hit an assert there. I will look into returning safely instead of asserting to harden this, and I will include the cleanup in v2.

What is the assertion about?

Logically if the fault thread will atomically fetch and set the bitmap bits (either your new bitmap, or reuse receivedmap), then it should already see the eager thread set the bit already, so it should avoid further playing with the page. Vice versa.

But as long as you have clue on how to resolve it and move on, we're good! You can mention that in the next cover letter then we can review it in the next version directly.

Thanks,

Signature · 1 line
Peter Xu
#5 lore raw

RFC PATCH 2/5 migration: add support for fault thread to load pages from disk

In reply to Aadeshveer Singh

In fast snapshot load, we would like to serve faults as soon as possible hence loading pages directly instead of requesting a source

Add postcopy_mapped_ram_load_page() function which serves single page fault by reading the snapshot file. It uses bitmap_test_and_clear_atomic on pending_bmap to coordinate between threads so each page is loaded exactly once. Non-zero pages are read using qemu_get_buffer_at into a temporary page (for loading page atomically), which is then placed using postcopy_place_page. Zero pages are placed directly using postcopy_place_page_zero.

Update postcopy_ram_fault_thread to call postcopy_mapped_ram_load_page instead of requesting source in case of fast snapshot load. to_src_file check is bypassed in fast snapshot load case as there is no source

Allocate another channel in postcopy_temp_pages_setup(like the preempt case), for both the fault thread and eager thread to load pages independently.

In case of failure to read required page crash the system using assert as disk failure is critical and VM cannot be recovered.

---
 migration/postcopy-ram.c | 92 ++++++++++++++++++++++++++++++++--------
 1 file changed, 75 insertions(+), 17 deletions(-)
migration/postcopy-ram.c +75 -17
index f5ef93f193..1ec20a07dd 100644
@@ -949,6 +949,53 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd,
                        pagesize);
 }
 
+/*
+ * Load a page from RAMBlock at offset at given host address.
+ * Used by postcopy ram fault thread and eager thread in fast snapshot load
+ * case. rb_offset: Offset of page in RAMBlock haddr: Base of page where to load
+ * in page Channel: Used to identify between threads and use corresponding temp
+ * pages Returns 0 on success
+ */
+static int postcopy_mapped_ram_load_page(MigrationIncomingState *mis,
+                                         RAMBlock *rb, ram_addr_t rb_offset,
+                                         uint64_t haddr, int channel)
+{
+    int ret = 0;
+    unsigned long page;
+    void *host = (void *)haddr;
+    void *place_source = mis->postcopy_tmp_pages[channel].tmp_huge_page;
+    size_t read;
+
+    page = rb_offset >> TARGET_PAGE_BITS;
+
+    if (bitmap_test_and_clear_atomic(rb->pending_bmap, page, 1)) {
+        if (test_bit(page, rb->nonzeropages)) {
+            /*
+             * qemu_get_buffer_at uses preadv which is thread safe we do not
+             * need different channels
+             */
+            read = qemu_get_buffer_at(mis->from_src_file, place_source,
+                                      TARGET_PAGE_SIZE,
+                                      rb->pages_offset + rb_offset);
+
+            g_assert(read == TARGET_PAGE_SIZE);
+
+            ret = postcopy_place_page(mis, host, place_source, rb);
+            if (ret) {
+                return ret;
+            }
+
+        } else {
+            /* zero page */
+            ret = postcopy_place_page_zero(mis, host, rb);
+            if (ret) {
+                return ret;
+            }
+        }
+    }
+    return ret;
+}
+
 /*
  * NOTE: @tid is only used when postcopy-blocktime feature is enabled, and
  * also optional: when zero is provided, the fault accounting will be ignored.
@@ -1320,11 +1367,11 @@ static void *postcopy_ram_fault_thread(void *opaque)
             break;
         }
 
-        if (!mis->to_src_file) {
+        if (!migrate_fast_snapshot_load() && !mis->to_src_file) {
             /*
-             * Possibly someone tells us that the return path is
-             * broken already using the event. We should hold until
-             * the channel is rebuilt.
+             * Fast snapshot load has no to src file or in other case someone
+             * possibly tells us that the return path is broken already using
+             * the event. We should hold until the channel is rebuilt.
              */
             postcopy_pause_fault_thread(mis);
         }
@@ -1387,18 +1434,26 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 qemu_ram_get_idstr(rb),
                                                 rb_offset,
                                                 msg.arg.pagefault.feat.ptid);
+
+            if (migrate_fast_snapshot_load()) {
+                if (postcopy_mapped_ram_load_page(
+                        mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {
+                    break;
+                }
+            } else {
 retry:
-            /*
-             * Send the request to the source - we want to request one
-             * of our host page sizes (which is >= TPS)
-             */
-            ret = postcopy_request_page(mis, rb, rb_offset,
-                                        msg.arg.pagefault.address,
-                                        msg.arg.pagefault.feat.ptid);
-            if (ret) {
-                /* May be network failure, try to wait for recovery */
-                postcopy_pause_fault_thread(mis);
-                goto retry;
+                /*
+                 * Send the request to the source - we want to request one
+                 * of our host page sizes (which is >= TPS)
+                 */
+                ret = postcopy_request_page(mis, rb, rb_offset,
+                                            msg.arg.pagefault.address,
+                                            msg.arg.pagefault.feat.ptid);
+                if (ret) {
+                    /* May be network failure, try to wait for recovery */
+                    postcopy_pause_fault_thread(mis);
+                    goto retry;
+                }
             }
         }
 
@@ -1471,8 +1526,11 @@ static int postcopy_temp_pages_setup(MigrationIncomingState *mis)
     unsigned i, channels;
     void *temp_page;
 
-    if (migrate_postcopy_preempt()) {
-        /* If preemption enabled, need extra channel for urgent requests */
+    if (migrate_postcopy_preempt() || migrate_fast_snapshot_load()) {
+        /*
+         * If preemption enabled or it is fast snapshot load, need extra channel
+         * for urgent requests/faults
+         */
         mis->postcopy_channels = RAM_CHANNEL_MAX;
     } else {
         /* Both precopy/postcopy on the same channel */
Signature · 1 line
2.54.0
#6 lore raw

Re: [RFC PATCH 2/5] migration: add support for fault thread to load pages from disk

Quoted 32 lines · Aadeshveer Singh

In fast snapshot load, we would like to serve faults as soon as possible hence loading pages directly instead of requesting a source

Add postcopy_mapped_ram_load_page() function which serves single page fault by reading the snapshot file. It uses bitmap_test_and_clear_atomic on pending_bmap to coordinate between threads so each page is loaded exactly once. Non-zero pages are read using qemu_get_buffer_at into a temporary page (for loading page atomically), which is then placed using postcopy_place_page. Zero pages are placed directly using postcopy_place_page_zero.

Update postcopy_ram_fault_thread to call postcopy_mapped_ram_load_page instead of requesting source in case of fast snapshot load. to_src_file check is bypassed in fast snapshot load case as there is no source

Allocate another channel in postcopy_temp_pages_setup(like the preempt case), for both the fault thread and eager thread to load pages independently.

In case of failure to read required page crash the system using assert as disk failure is critical and VM cannot be recovered.

---
 migration/postcopy-ram.c | 92 ++++++++++++++++++++++++++++++++--------
 1 file changed, 75 insertions(+), 17 deletions(-)
migration/postcopy-ram.c +0 -0
index f5ef93f193..1ec20a07dd 100644

migration/postcopy-ram.c int postcopy_wake_shared(struct PostCopyFD *pcfd, · line 958

Quoted 10 lines
migration/postcopy-ram.c +7 -0
@@ -949,6 +949,53 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd,
                        pagesize);
 }
 
+/*
+ * Load a page from RAMBlock at offset at given host address.
+ * Used by postcopy ram fault thread and eager thread in fast snapshot load
+ * case. rb_offset: Offset of page in RAMBlock haddr: Base of page where to load
+ * in page Channel: Used to identify between threads and use corresponding temp
+ * pages Returns 0 on success
+ */

Somehow this paragraph is not properly formatted with newlines. If you want to provide a full doc of the function, you can follow kernel-doc format:

https://docs.kernel.org/doc-guide/kernel-doc.html

/**
* function_name() - desc
*
* @arg1: desc for @arg1
* @arg2: desc for @arg2
* ...

migration/postcopy-ram.c int postcopy_wake_shared(struct PostCopyFD *pcfd, · line 965

Quoted 7 lines
migration/postcopy-ram.c +7 -0
+static int postcopy_mapped_ram_load_page(MigrationIncomingState *mis,
+                                         RAMBlock *rb, ram_addr_t rb_offset,
+                                         uint64_t haddr, int channel)
+{
+    int ret = 0;
+    unsigned long page;
+    void *host = (void *)haddr;

Can drop this var.

migration/postcopy-ram.c int postcopy_wake_shared(struct PostCopyFD *pcfd, · line 967

Quoted 2 lines
migration/postcopy-ram.c +2 -0
+    void *place_source = mis->postcopy_tmp_pages[channel].tmp_huge_page;
+    size_t read;

Nit: can use reverse christmas tree.

migration/postcopy-ram.c int postcopy_wake_shared(struct PostCopyFD *pcfd, · line 976

Quoted 9 lines
migration/postcopy-ram.c +9 -0
+
+    page = rb_offset >> TARGET_PAGE_BITS;
+
+    if (bitmap_test_and_clear_atomic(rb->pending_bmap, page, 1)) {
+        if (test_bit(page, rb->nonzeropages)) {
+            /*
+             * qemu_get_buffer_at uses preadv which is thread safe we do not
+             * need different channels
+             */

Slightly misleading when the two threads do not use the same "channel" (or say, temp pages..). Maybe what you wanted to emphasize is QEMU might have more than one thread using this function to install pages. In that case, it can be put as:

/*
* This can happen concurrently, but it's thread-safe because
* qemu_get_buffer_at() is thread-safe, and the caller will be
* using different temporary buffers.
*/

migration/postcopy-ram.c int postcopy_wake_shared(struct PostCopyFD *pcfd, · line 981

Quoted 5 lines
migration/postcopy-ram.c +5 -0
+            read = qemu_get_buffer_at(mis->from_src_file, place_source,
+                                      TARGET_PAGE_SIZE,
+                                      rb->pages_offset + rb_offset);
+
+            g_assert(read == TARGET_PAGE_SIZE);

Two things can be improved on errors:

- When an error can be reached with user input, logically we shouldn't assert(), assert() is only for program errors. Here it's possible an user specified a broken image, then we should cleanly exit with an err code.

- Still better to report the error to the caller and only handle error at the very top caller after throwing the error to stderr.

I did suggest that we can assert on loading failures when we talked, but I confess I was not clear on how to do, sorry. Let's use assert() only if it's a programming error.

Postcopy didn't do as good on error reporting, normally nowadays QEMU suggests to use "bool function(..., Error **errp)" as function interface, return false if failure hit. You can do it with the new functions like this, then return the Error** object to the top caller and do one error_report() before exit().

Alone the way if you want to convert some postcopy code to start using Error** it'll be even better. Feel free to have a look at the comments at the start of include/qapi/error.h on the suggested way to handle errors in QEMU.

migration/postcopy-ram.c int postcopy_wake_shared(struct PostCopyFD *pcfd, · line 989

Quoted 8 lines
migration/postcopy-ram.c +8 -0
+
+            ret = postcopy_place_page(mis, host, place_source, rb);
+            if (ret) {
+                return ret;
+            }
+
+        } else {
+            /* zero page */

Nit, we can drop this comment.

Quoted 13 lines
migration/postcopy-ram.c +9 -0
+            ret = postcopy_place_page_zero(mis, host, rb);
+            if (ret) {
+                return ret;
+            }
+        }
+    }
+    return ret;
+}
+
 /*
  * NOTE: @tid is only used when postcopy-blocktime feature is enabled, and
  * also optional: when zero is provided, the fault accounting will be ignored.

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1369

Quoted 3 lines
migration/postcopy-ram.c +0 -0
@@ -1320,11 +1367,11 @@ static void *postcopy_ram_fault_thread(void *opaque)
             break;
         }
 

We can leave below comments as-is, then add one line here to explain:

/*
* Fast snapshot load doesn't support pause and recover, because
* it's not necessary: we can fail right away when QEMU just booted
* with nothing to lose.
*/

Quoted 13 lines
migration/postcopy-ram.c +4 -4
-        if (!mis->to_src_file) {
+        if (!migrate_fast_snapshot_load() && !mis->to_src_file) {
             /*
-             * Possibly someone tells us that the return path is
-             * broken already using the event. We should hold until
-             * the channel is rebuilt.
+             * Fast snapshot load has no to src file or in other case someone
+             * possibly tells us that the return path is broken already using
+             * the event. We should hold until the channel is rebuilt.
              */
             postcopy_pause_fault_thread(mis);
         }

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1440

Quoted 7 lines
migration/postcopy-ram.c +4 -0
@@ -1387,18 +1434,26 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 qemu_ram_get_idstr(rb),
                                                 rb_offset,
                                                 msg.arg.pagefault.feat.ptid);
+
+            if (migrate_fast_snapshot_load()) {
+                if (postcopy_mapped_ram_load_page(
+                        mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {

s/1/RAM_CHANNEL_POSTCOPY/? I agree it's not ideal with the current names, but it is still kind of suitable.

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1442

Quoted 2 lines
migration/postcopy-ram.c +2 -0
+                    break;
+                }

With above, we can error_report() here and exit() when error happens.

Quoted 45 lines
migration/postcopy-ram.c +18 -13
+            } else {
 retry:
-            /*
-             * Send the request to the source - we want to request one
-             * of our host page sizes (which is >= TPS)
-             */
-            ret = postcopy_request_page(mis, rb, rb_offset,
-                                        msg.arg.pagefault.address,
-                                        msg.arg.pagefault.feat.ptid);
-            if (ret) {
-                /* May be network failure, try to wait for recovery */
-                postcopy_pause_fault_thread(mis);
-                goto retry;
+                /*
+                 * Send the request to the source - we want to request one
+                 * of our host page sizes (which is >= TPS)
+                 */
+                ret = postcopy_request_page(mis, rb, rb_offset,
+                                            msg.arg.pagefault.address,
+                                            msg.arg.pagefault.feat.ptid);
+                if (ret) {
+                    /* May be network failure, try to wait for recovery */
+                    postcopy_pause_fault_thread(mis);
+                    goto retry;
+                }
             }
         }
 
@@ -1471,8 +1526,11 @@ static int postcopy_temp_pages_setup(MigrationIncomingState *mis)
     unsigned i, channels;
     void *temp_page;
 
-    if (migrate_postcopy_preempt()) {
-        /* If preemption enabled, need extra channel for urgent requests */
+    if (migrate_postcopy_preempt() || migrate_fast_snapshot_load()) {
+        /*
+         * If preemption enabled or it is fast snapshot load, need extra channel
+         * for urgent requests/faults
+         */
         mis->postcopy_channels = RAM_CHANNEL_MAX;
     } else {
         /* Both precopy/postcopy on the same channel */
Signature · 1 line
2.54.0
Signature · 1 line
Peter Xu
#7 lore raw

Re: [RFC PATCH 2/5] migration: add support for fault thread to load pages from disk

In reply to Aadeshveer Singh
Quoted 102 lines · Aadeshveer Singh

In fast snapshot load, we would like to serve faults as soon as possible hence loading pages directly instead of requesting a source

Add postcopy_mapped_ram_load_page() function which serves single page fault by reading the snapshot file. It uses bitmap_test_and_clear_atomic on pending_bmap to coordinate between threads so each page is loaded exactly once. Non-zero pages are read using qemu_get_buffer_at into a temporary page (for loading page atomically), which is then placed using postcopy_place_page. Zero pages are placed directly using postcopy_place_page_zero.

Update postcopy_ram_fault_thread to call postcopy_mapped_ram_load_page instead of requesting source in case of fast snapshot load. to_src_file check is bypassed in fast snapshot load case as there is no source

Allocate another channel in postcopy_temp_pages_setup(like the preempt case), for both the fault thread and eager thread to load pages independently.

In case of failure to read required page crash the system using assert as disk failure is critical and VM cannot be recovered.

---
 migration/postcopy-ram.c | 92 ++++++++++++++++++++++++++++++++--------
 1 file changed, 75 insertions(+), 17 deletions(-)
migration/postcopy-ram.c +51 -4
index f5ef93f193..1ec20a07dd 100644
@@ -949,6 +949,53 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd,
                        pagesize);
 }
 
+/*
+ * Load a page from RAMBlock at offset at given host address.
+ * Used by postcopy ram fault thread and eager thread in fast snapshot load
+ * case. rb_offset: Offset of page in RAMBlock haddr: Base of page where to load
+ * in page Channel: Used to identify between threads and use corresponding temp
+ * pages Returns 0 on success
+ */
+static int postcopy_mapped_ram_load_page(MigrationIncomingState *mis,
+                                         RAMBlock *rb, ram_addr_t rb_offset,
+                                         uint64_t haddr, int channel)
+{
+    int ret = 0;
+    unsigned long page;
+    void *host = (void *)haddr;
+    void *place_source = mis->postcopy_tmp_pages[channel].tmp_huge_page;
+    size_t read;
+
+    page = rb_offset >> TARGET_PAGE_BITS;
+
+    if (bitmap_test_and_clear_atomic(rb->pending_bmap, page, 1)) {
+        if (test_bit(page, rb->nonzeropages)) {
+            /*
+             * qemu_get_buffer_at uses preadv which is thread safe we do not
+             * need different channels
+             */
+            read = qemu_get_buffer_at(mis->from_src_file, place_source,
+                                      TARGET_PAGE_SIZE,
+                                      rb->pages_offset + rb_offset);
+
+            g_assert(read == TARGET_PAGE_SIZE);
+
+            ret = postcopy_place_page(mis, host, place_source, rb);
+            if (ret) {
+                return ret;
+            }
+
+        } else {
+            /* zero page */
+            ret = postcopy_place_page_zero(mis, host, rb);
+            if (ret) {
+                return ret;
+            }
+        }
+    }
+    return ret;
+}
+
 /*
  * NOTE: @tid is only used when postcopy-blocktime feature is enabled, and
  * also optional: when zero is provided, the fault accounting will be ignored.
@@ -1320,11 +1367,11 @@ static void *postcopy_ram_fault_thread(void *opaque)
             break;
         }
 
-        if (!mis->to_src_file) {
+        if (!migrate_fast_snapshot_load() && !mis->to_src_file) {
             /*
-             * Possibly someone tells us that the return path is
-             * broken already using the event. We should hold until
-             * the channel is rebuilt.
+             * Fast snapshot load has no to src file or in other case someone
+             * possibly tells us that the return path is broken already using
+             * the event. We should hold until the channel is rebuilt.
              */
             postcopy_pause_fault_thread(mis);
         }

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1438

Quoted 5 lines
migration/postcopy-ram.c +2 -0
@@ -1387,18 +1434,26 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 qemu_ram_get_idstr(rb),
                                                 rb_offset,
                                                 msg.arg.pagefault.feat.ptid);
+
+            if (migrate_fast_snapshot_load()) {

You probably want to check mapped_ram here instead as this is already under the postcopy path.

I haven't reviewed the entire series yet, but I'm questioning whether we need the new term (fast snapshot) at all.

Quoted 46 lines
migration/postcopy-ram.c +22 -13
+                if (postcopy_mapped_ram_load_page(
+                        mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {
+                    break;
+                }
+            } else {
 retry:
-            /*
-             * Send the request to the source - we want to request one
-             * of our host page sizes (which is >= TPS)
-             */
-            ret = postcopy_request_page(mis, rb, rb_offset,
-                                        msg.arg.pagefault.address,
-                                        msg.arg.pagefault.feat.ptid);
-            if (ret) {
-                /* May be network failure, try to wait for recovery */
-                postcopy_pause_fault_thread(mis);
-                goto retry;
+                /*
+                 * Send the request to the source - we want to request one
+                 * of our host page sizes (which is >= TPS)
+                 */
+                ret = postcopy_request_page(mis, rb, rb_offset,
+                                            msg.arg.pagefault.address,
+                                            msg.arg.pagefault.feat.ptid);
+                if (ret) {
+                    /* May be network failure, try to wait for recovery */
+                    postcopy_pause_fault_thread(mis);
+                    goto retry;
+                }
             }
         }
 
@@ -1471,8 +1526,11 @@ static int postcopy_temp_pages_setup(MigrationIncomingState *mis)
     unsigned i, channels;
     void *temp_page;
 
-    if (migrate_postcopy_preempt()) {
-        /* If preemption enabled, need extra channel for urgent requests */
+    if (migrate_postcopy_preempt() || migrate_fast_snapshot_load()) {
+        /*
+         * If preemption enabled or it is fast snapshot load, need extra channel
+         * for urgent requests/faults
+         */
         mis->postcopy_channels = RAM_CHANNEL_MAX;
     } else {
         /* Both precopy/postcopy on the same channel */
#8 lore raw

Re: [RFC PATCH 2/5] migration: add support for fault thread to load pages from disk

Quoted 114 lines · Fabiano Rosas
Quoted 102 lines · Aadeshveer Singh

In fast snapshot load, we would like to serve faults as soon as possible hence loading pages directly instead of requesting a source

Add postcopy_mapped_ram_load_page() function which serves single page fault by reading the snapshot file. It uses bitmap_test_and_clear_atomic on pending_bmap to coordinate between threads so each page is loaded exactly once. Non-zero pages are read using qemu_get_buffer_at into a temporary page (for loading page atomically), which is then placed using postcopy_place_page. Zero pages are placed directly using postcopy_place_page_zero.

Update postcopy_ram_fault_thread to call postcopy_mapped_ram_load_page instead of requesting source in case of fast snapshot load. to_src_file check is bypassed in fast snapshot load case as there is no source

Allocate another channel in postcopy_temp_pages_setup(like the preempt case), for both the fault thread and eager thread to load pages independently.

In case of failure to read required page crash the system using assert as disk failure is critical and VM cannot be recovered.

---
 migration/postcopy-ram.c | 92 ++++++++++++++++++++++++++++++++--------
 1 file changed, 75 insertions(+), 17 deletions(-)
migration/postcopy-ram.c +51 -4
index f5ef93f193..1ec20a07dd 100644
@@ -949,6 +949,53 @@ int postcopy_wake_shared(struct PostCopyFD *pcfd,
                        pagesize);
 }
 
+/*
+ * Load a page from RAMBlock at offset at given host address.
+ * Used by postcopy ram fault thread and eager thread in fast snapshot load
+ * case. rb_offset: Offset of page in RAMBlock haddr: Base of page where to load
+ * in page Channel: Used to identify between threads and use corresponding temp
+ * pages Returns 0 on success
+ */
+static int postcopy_mapped_ram_load_page(MigrationIncomingState *mis,
+                                         RAMBlock *rb, ram_addr_t rb_offset,
+                                         uint64_t haddr, int channel)
+{
+    int ret = 0;
+    unsigned long page;
+    void *host = (void *)haddr;
+    void *place_source = mis->postcopy_tmp_pages[channel].tmp_huge_page;
+    size_t read;
+
+    page = rb_offset >> TARGET_PAGE_BITS;
+
+    if (bitmap_test_and_clear_atomic(rb->pending_bmap, page, 1)) {
+        if (test_bit(page, rb->nonzeropages)) {
+            /*
+             * qemu_get_buffer_at uses preadv which is thread safe we do not
+             * need different channels
+             */
+            read = qemu_get_buffer_at(mis->from_src_file, place_source,
+                                      TARGET_PAGE_SIZE,
+                                      rb->pages_offset + rb_offset);
+
+            g_assert(read == TARGET_PAGE_SIZE);
+
+            ret = postcopy_place_page(mis, host, place_source, rb);
+            if (ret) {
+                return ret;
+            }
+
+        } else {
+            /* zero page */
+            ret = postcopy_place_page_zero(mis, host, rb);
+            if (ret) {
+                return ret;
+            }
+        }
+    }
+    return ret;
+}
+
 /*
  * NOTE: @tid is only used when postcopy-blocktime feature is enabled, and
  * also optional: when zero is provided, the fault accounting will be ignored.
@@ -1320,11 +1367,11 @@ static void *postcopy_ram_fault_thread(void *opaque)
             break;
         }
 
-        if (!mis->to_src_file) {
+        if (!migrate_fast_snapshot_load() && !mis->to_src_file) {
             /*
-             * Possibly someone tells us that the return path is
-             * broken already using the event. We should hold until
-             * the channel is rebuilt.
+             * Fast snapshot load has no to src file or in other case someone
+             * possibly tells us that the return path is broken already using
+             * the event. We should hold until the channel is rebuilt.
              */
             postcopy_pause_fault_thread(mis);
         }

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1438

Quoted 5 lines
migration/postcopy-ram.c +2 -0
@@ -1387,18 +1434,26 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 qemu_ram_get_idstr(rb),
                                                 rb_offset,
                                                 msg.arg.pagefault.feat.ptid);
+
+            if (migrate_fast_snapshot_load()) {

You probably want to check mapped_ram here instead as this is already under the postcopy path.

Quoted 2 lines

I haven't reviewed the entire series yet, but I'm questioning whether we need the new term (fast snapshot) at all.

You are right. I initially introduced the migrate_fast_snapshot_load() wrapper to improve readability, as checking multiple capabilities might make it non-trivial for someone new to the codebase to understand.

However, I agree that introducing a new wrapper unnecessarily complicates capabilities. Adding comments around these checks will achieve that same readability without increasing the API footprint.

I will drop the custom wrapper and consolidate the checks for v2.

Quoted 47 lines
Quoted 47 lines
migration/postcopy-ram.c +22 -13
+                if (postcopy_mapped_ram_load_page(
+                        mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {
+                    break;
+                }
+            } else {
 retry:
-            /*
-             * Send the request to the source - we want to request one
-             * of our host page sizes (which is >= TPS)
-             */
-            ret = postcopy_request_page(mis, rb, rb_offset,
-                                        msg.arg.pagefault.address,
-                                        msg.arg.pagefault.feat.ptid);
-            if (ret) {
-                /* May be network failure, try to wait for recovery */
-                postcopy_pause_fault_thread(mis);
-                goto retry;
+                /*
+                 * Send the request to the source - we want to request one
+                 * of our host page sizes (which is >= TPS)
+                 */
+                ret = postcopy_request_page(mis, rb, rb_offset,
+                                            msg.arg.pagefault.address,
+                                            msg.arg.pagefault.feat.ptid);
+                if (ret) {
+                    /* May be network failure, try to wait for recovery */
+                    postcopy_pause_fault_thread(mis);
+                    goto retry;
+                }
             }
         }
 
@@ -1471,8 +1526,11 @@ static int postcopy_temp_pages_setup(MigrationIncomingState *mis)
     unsigned i, channels;
     void *temp_page;
 
-    if (migrate_postcopy_preempt()) {
-        /* If preemption enabled, need extra channel for urgent requests */
+    if (migrate_postcopy_preempt() || migrate_fast_snapshot_load()) {
+        /*
+         * If preemption enabled or it is fast snapshot load, need extra channel
+         * for urgent requests/faults
+         */
         mis->postcopy_channels = RAM_CHANNEL_MAX;
     } else {
         /* Both precopy/postcopy on the same channel */
#9 lore raw

RFC PATCH 3/5 migration: add eager load thread for fast snapshot load

In reply to Aadeshveer Singh

In fast snapshot load a thread is needed for actively loading in pages along with the fault path so that the guest is not dependent on fault thread indefinitely.

Add postcopy_ram_eager_load_thread(), for the eager thread which iterates over all non ignored blocks calling ram_block_load_eager() each. ram_block_load_eager then iterates to load in all pages using postcopy_mapped_ram_load_page(), with a different channel, which takes care of not loading in pages already loaded by fault thread. On completion the thread schedules postcopy_ram_eager_load_bh() to destroy the incoming migration state and set states to completed/end.

Add postcopy_ram_eager_load_setup() to create the thread. Added joining logic in postcopy_incoming_cleanup().

Add tracepoints for entry and exit to eager load thread.

---
 migration/migration.h    |  5 +++
 migration/postcopy-ram.c | 75 ++++++++++++++++++++++++++++++++++++++++
 migration/postcopy-ram.h |  2 ++
 migration/trace-events   |  2 ++
 4 files changed, 84 insertions(+)
migration/migration.h +5 -0
index 841f49b215..7bb54a6584 100644
@@ -42,6 +42,7 @@
 #define  MIGRATION_THREAD_DST_FAULT         "mig/dst/fault"
 #define  MIGRATION_THREAD_DST_LISTEN        "mig/dst/listen"
 #define  MIGRATION_THREAD_DST_PREEMPT       "mig/dst/preempt"
+#define  MIGRATION_THREAD_DST_EAGER         "mig/dst/eager"
 
 struct PostcopyBlocktimeContext;
 typedef struct ThreadPool ThreadPool;
@@ -120,6 +121,10 @@ struct MigrationIncomingState {
     bool           have_listen_thread;
     QemuThread     listen_thread;
 
+    /* Thread to load pages eagerly in fast snapshot load case */
+    bool have_eager_load_thread;
+    QemuThread eager_load_thread;
+
     /* For the kernel to send us notifications */
     int       userfault_fd;
     /* To notify the fault_thread to wake, e.g., when need to quit */
migration/postcopy-ram.c +75 -0

84 lines not on this page · raw message

migration/postcopy-ram.h +2 -0
index a080dd65a7..b3ba42e447 100644
@@ -202,4 +202,6 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
 int postcopy_incoming_setup(MigrationIncomingState *mis, Error **errp);
 int postcopy_incoming_cleanup(MigrationIncomingState *mis);
 
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis);
+
 #endif
migration/trace-events +2 -0
index de99d976ab..38f11e1e9f 100644
@@ -314,6 +314,8 @@ postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u"
 postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu, bool exists) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d, exist: %d"
 postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu, int affected_non_cpus) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d, affected_non_cpus: %d"
 postcopy_blocktime_end_one(int cpu, uint8_t left_faults) "cpu: %d, left_faults: %" PRIu8
+postcopy_ram_eager_load_thread_entry(void) ""
+postcopy_ram_eager_load_thread_exit(void) ""
 
 # exec.c
 migration_exec_outgoing(const char *cmd) "cmd=%s"
Signature · 1 line
2.54.0
#10 lore raw

Re: [RFC PATCH 3/5] migration: add eager load thread for fast snapshot load

Quoted 30 lines · Aadeshveer Singh

In fast snapshot load a thread is needed for actively loading in pages along with the fault path so that the guest is not dependent on fault thread indefinitely.

Add postcopy_ram_eager_load_thread(), for the eager thread which iterates over all non ignored blocks calling ram_block_load_eager() each. ram_block_load_eager then iterates to load in all pages using postcopy_mapped_ram_load_page(), with a different channel, which takes care of not loading in pages already loaded by fault thread. On completion the thread schedules postcopy_ram_eager_load_bh() to destroy the incoming migration state and set states to completed/end.

Add postcopy_ram_eager_load_setup() to create the thread. Added joining logic in postcopy_incoming_cleanup().

Add tracepoints for entry and exit to eager load thread.

---
 migration/migration.h    |  5 +++
 migration/postcopy-ram.c | 75 ++++++++++++++++++++++++++++++++++++++++
 migration/postcopy-ram.h |  2 ++
 migration/trace-events   |  2 ++
 4 files changed, 84 insertions(+)
migration/migration.h +0 -0
index 841f49b215..7bb54a6584 100644

migration/migration.h line 45

Quoted 4 lines
migration/migration.h +1 -0
@@ -42,6 +42,7 @@
 #define  MIGRATION_THREAD_DST_FAULT         "mig/dst/fault"
 #define  MIGRATION_THREAD_DST_LISTEN        "mig/dst/listen"
 #define  MIGRATION_THREAD_DST_PREEMPT       "mig/dst/preempt"
+#define  MIGRATION_THREAD_DST_EAGER         "mig/dst/eager"

The name isn't easy to digest when someone saw it the 1st time. Maybe "snapshot_load" / DST_SNAPSHOT_LOAD?

Quoted 37 lines
migration/migration.h +4 -0
 struct PostcopyBlocktimeContext;
 typedef struct ThreadPool ThreadPool;
@@ -120,6 +121,10 @@ struct MigrationIncomingState {
     bool           have_listen_thread;
     QemuThread     listen_thread;
 
+    /* Thread to load pages eagerly in fast snapshot load case */
+    bool have_eager_load_thread;
+    QemuThread eager_load_thread;
+
     /* For the kernel to send us notifications */
     int       userfault_fd;
     /* To notify the fault_thread to wake, e.g., when need to quit */
migration/postcopy-ram.c +9 -0
index 1ec20a07dd..0ee294a381 100644
@@ -2289,9 +2289,84 @@ int postcopy_incoming_cleanup(MigrationIncomingState *mis)
         mis->have_listen_thread = false;
     }
 
+    if (mis->have_eager_load_thread) {
+        qemu_thread_join(&mis->eager_load_thread);
+        mis->have_eager_load_thread = false;
+    }
+
     if (migrate_postcopy_ram()) {
         rc = postcopy_ram_incoming_cleanup(mis);
     }
 
     return rc;
 }
+
+/*
+ * Called by postcopy_ram_eager_load_thread over all blocks to load in all the
+ * pending pages of given ram block

migration/postcopy-ram.c int postcopy_incoming_cleanup(MigrationIncomingState *mis) · line 2316

Quoted 10 lines
migration/postcopy-ram.c +10 -0
+ */
+static int ram_block_load_eager(RAMBlock *rb, void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+    void *host = qemu_ram_get_host_addr(rb);
+    void *target;
+    int ret = 0;
+
+    for (ram_addr_t page_loc = 0; page_loc < rb->used_length;
+         page_loc += TARGET_PAGE_SIZE) {

I think you can directly use rb->page_size here, then huge page will also work.

migration/postcopy-ram.c int postcopy_incoming_cleanup(MigrationIncomingState *mis) · line 2319

Quoted 3 lines
migration/postcopy-ram.c +3 -0
+        target = (uint8_t *)host + page_loc;
+        ret = postcopy_mapped_ram_load_page(mis, rb, page_loc, (uint64_t)target,
+                                            0);

RAM_CHANNEL_PRECOPY

Quoted 6 lines
migration/postcopy-ram.c +6 -0
+        if (ret) {
+            break;
+        }
+    }
+    return ret;
+}

migration/postcopy-ram.c int postcopy_incoming_cleanup(MigrationIncomingState *mis) · line 2335

Quoted 10 lines
migration/postcopy-ram.c +10 -0
+
+/*
+ * Bottom half for fast snapshot load, scheduled by eager load thread
+ */
+static void postcopy_ram_eager_load_bh(void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+    postcopy_state_set(POSTCOPY_INCOMING_END);
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+                      MIGRATION_STATUS_COMPLETED);

If you follow what postcopy_listen_thread() does, this should be put in the eager load thread when all things succeeded.

migration/postcopy-ram.c int postcopy_incoming_cleanup(MigrationIncomingState *mis) · line 2337

Quoted 2 lines
migration/postcopy-ram.c +2 -0
+    migration_incoming_state_destroy();
+}

IIUC we should reuse what postcopy does.

Say, you can have one prior patch making postcopy_listen_thread_bh() to be what you want. IIUC we can safely move these two lines into it:

    postcopy_state_set(POSTCOPY_INCOMING_END);

Then I believe you can reuse it. Maybe when at it, rename it to postcopy_complete_bh() so it detaches from "listen thread".

That also handles exit_on_error case, so IIUC that's also what you should
do: when any failure happens, set s->error, then set FAILED status, finally kickoff this BH to handle the rest, so exit() will be done in the BH.

PS: on incoming side so far we still sometimes reuse MigrationState->error. It's a historical "slight" misuse.. but let's stick with it so far; I think the whole point is to make error visible in query-migrate if exit_on_error=false. It's another problem to solve in the future.

Quoted 5 lines
migration/postcopy-ram.c +5 -0
+
+/*
+ * Used by fast snapshot load to eagerly load in all pages of RAM and schedule
+ * cleanup after entire RAM is loaded
+ */

migration/postcopy-ram.c int postcopy_incoming_cleanup(MigrationIncomingState *mis) · line 2352

Quoted 10 lines
migration/postcopy-ram.c +10 -0
+static void *postcopy_ram_eager_load_thread(void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+
+    trace_postcopy_ram_eager_load_thread_entry();
+    rcu_register_thread();
+    qemu_event_set(&mis->thread_sync_event);
+
+    if (foreach_not_ignored_block(ram_block_load_eager, mis)) {
+        error_report("ram_block_load_eager failed");

We can set error to MigrationState here with migrate_error_propagate(), if you would reuse the BH I mentioned above.

Quoted 10 lines
migration/postcopy-ram.c +10 -0
+    }
+
+    migration_bh_schedule(postcopy_ram_eager_load_bh, mis);
+
+    rcu_unregister_thread();
+    trace_postcopy_ram_eager_load_thread_exit();
+    return NULL;
+}
+
+/*

migration/postcopy-ram.c int postcopy_incoming_cleanup(MigrationIncomingState *mis) · line 2372

Quoted 10 lines
migration/postcopy-ram.c +10 -0
+ * Create thread for eager loading in fast snapshot load case
+ */
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis)
+{
+    postcopy_thread_create(
+        mis, &mis->eager_load_thread, MIGRATION_THREAD_DST_EAGER,
+        postcopy_ram_eager_load_thread, QEMU_THREAD_JOINABLE);
+    mis->have_eager_load_thread = true;
+    return 0;
+}

Then here this patch introduced these functions without using them. Then this patch will be almost not possible to do proper review because the reviewer won't know how it will be used, and where.

IMHO you can squash this patch with the one that will use this function.

Thanks,

Quoted 27 lines
migration/postcopy-ram.h +2 -0
index a080dd65a7..b3ba42e447 100644
@@ -202,4 +202,6 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
 int postcopy_incoming_setup(MigrationIncomingState *mis, Error **errp);
 int postcopy_incoming_cleanup(MigrationIncomingState *mis);
 
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis);
+
 #endif
migration/trace-events +2 -0
index de99d976ab..38f11e1e9f 100644
@@ -314,6 +314,8 @@ postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u"
 postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu, bool exists) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d, exist: %d"
 postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu, int affected_non_cpus) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d, affected_non_cpus: %d"
 postcopy_blocktime_end_one(int cpu, uint8_t left_faults) "cpu: %d, left_faults: %" PRIu8
+postcopy_ram_eager_load_thread_entry(void) ""
+postcopy_ram_eager_load_thread_exit(void) ""
 
 # exec.c
 migration_exec_outgoing(const char *cmd) "cmd=%s"
Signature · 1 line
2.54.0
Signature · 1 line
Peter Xu
#11 lore raw

Re: [RFC PATCH 3/5] migration: add eager load thread for fast snapshot load

In reply to Aadeshveer Singh
Quoted 4 lines · Aadeshveer Singh

In fast snapshot load a thread is needed for actively loading in pages along with the fault path so that the guest is not dependent on fault thread indefinitely.

In the paragraph above, "dependent on fault thread indefinitely" is a bit vague: it's not clear why that condition might be undesirable or why it's indefinte.

How is that different from normal postcopy? Is the point that postcopy will have transferred most of the memory in precopy so the fault thread has way less pages to deal with whereas here is the opposite, just a few pages have been potentially loaded with a large chunk still waiting to be faulted in?

Is this new eager thread something that could be used even for normal postcopy? Kind of sounds like a separate sub-feature. Does the fast snapshot work at all without this?

Quoted 8 lines

Add postcopy_ram_eager_load_thread(), for the eager thread which iterates over all non ignored blocks calling ram_block_load_eager() each. ram_block_load_eager then iterates to load in all pages using postcopy_mapped_ram_load_page(), with a different channel, which takes care of not loading in pages already loaded by fault thread. On completion the thread schedules postcopy_ram_eager_load_bh() to destroy the incoming migration state and set states to completed/end.

Ok, so that answers my question from the cover letter, we're holding the migration until all pages have been loaded. So it's yet another reason to have the eager thread. Still, the fast snapshot load can be fast while having a slow migration overall I suppose.

To be clear, I'm not arguing against the design, only trying to frame the dependencies.

Quoted 149 lines

Add postcopy_ram_eager_load_setup() to create the thread. Added joining logic in postcopy_incoming_cleanup().

Add tracepoints for entry and exit to eager load thread.

---
 migration/migration.h    |  5 +++
 migration/postcopy-ram.c | 75 ++++++++++++++++++++++++++++++++++++++++
 migration/postcopy-ram.h |  2 ++
 migration/trace-events   |  2 ++
 4 files changed, 84 insertions(+)
migration/migration.h +5 -0
index 841f49b215..7bb54a6584 100644
@@ -42,6 +42,7 @@
 #define  MIGRATION_THREAD_DST_FAULT         "mig/dst/fault"
 #define  MIGRATION_THREAD_DST_LISTEN        "mig/dst/listen"
 #define  MIGRATION_THREAD_DST_PREEMPT       "mig/dst/preempt"
+#define  MIGRATION_THREAD_DST_EAGER         "mig/dst/eager"
 
 struct PostcopyBlocktimeContext;
 typedef struct ThreadPool ThreadPool;
@@ -120,6 +121,10 @@ struct MigrationIncomingState {
     bool           have_listen_thread;
     QemuThread     listen_thread;
 
+    /* Thread to load pages eagerly in fast snapshot load case */
+    bool have_eager_load_thread;
+    QemuThread eager_load_thread;
+
     /* For the kernel to send us notifications */
     int       userfault_fd;
     /* To notify the fault_thread to wake, e.g., when need to quit */
migration/postcopy-ram.c +75 -0
index 1ec20a07dd..0ee294a381 100644
@@ -2289,9 +2289,84 @@ int postcopy_incoming_cleanup(MigrationIncomingState *mis)
         mis->have_listen_thread = false;
     }
 
+    if (mis->have_eager_load_thread) {
+        qemu_thread_join(&mis->eager_load_thread);
+        mis->have_eager_load_thread = false;
+    }
+
     if (migrate_postcopy_ram()) {
         rc = postcopy_ram_incoming_cleanup(mis);
     }
 
     return rc;
 }
+
+/*
+ * Called by postcopy_ram_eager_load_thread over all blocks to load in all the
+ * pending pages of given ram block
+ */
+static int ram_block_load_eager(RAMBlock *rb, void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+    void *host = qemu_ram_get_host_addr(rb);
+    void *target;
+    int ret = 0;
+
+    for (ram_addr_t page_loc = 0; page_loc < rb->used_length;
+         page_loc += TARGET_PAGE_SIZE) {
+        target = (uint8_t *)host + page_loc;
+        ret = postcopy_mapped_ram_load_page(mis, rb, page_loc, (uint64_t)target,
+                                            0);
+        if (ret) {
+            break;
+        }
+    }
+    return ret;
+}
+
+/*
+ * Bottom half for fast snapshot load, scheduled by eager load thread
+ */
+static void postcopy_ram_eager_load_bh(void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+    postcopy_state_set(POSTCOPY_INCOMING_END);
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+                      MIGRATION_STATUS_COMPLETED);
+    migration_incoming_state_destroy();
+}
+
+/*
+ * Used by fast snapshot load to eagerly load in all pages of RAM and schedule
+ * cleanup after entire RAM is loaded
+ */
+static void *postcopy_ram_eager_load_thread(void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+
+    trace_postcopy_ram_eager_load_thread_entry();
+    rcu_register_thread();
+    qemu_event_set(&mis->thread_sync_event);
+
+    if (foreach_not_ignored_block(ram_block_load_eager, mis)) {
+        error_report("ram_block_load_eager failed");
+    }
+
+    migration_bh_schedule(postcopy_ram_eager_load_bh, mis);
+
+    rcu_unregister_thread();
+    trace_postcopy_ram_eager_load_thread_exit();
+    return NULL;
+}
+
+/*
+ * Create thread for eager loading in fast snapshot load case
+ */
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis)
+{
+    postcopy_thread_create(
+        mis, &mis->eager_load_thread, MIGRATION_THREAD_DST_EAGER,
+        postcopy_ram_eager_load_thread, QEMU_THREAD_JOINABLE);
+    mis->have_eager_load_thread = true;
+    return 0;
+}
migration/postcopy-ram.h +2 -0
index a080dd65a7..b3ba42e447 100644
@@ -202,4 +202,6 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
 int postcopy_incoming_setup(MigrationIncomingState *mis, Error **errp);
 int postcopy_incoming_cleanup(MigrationIncomingState *mis);
 
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis);
+
 #endif
migration/trace-events +2 -0
index de99d976ab..38f11e1e9f 100644
@@ -314,6 +314,8 @@ postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u"
 postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu, bool exists) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d, exist: %d"
 postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu, int affected_non_cpus) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d, affected_non_cpus: %d"
 postcopy_blocktime_end_one(int cpu, uint8_t left_faults) "cpu: %d, left_faults: %" PRIu8
+postcopy_ram_eager_load_thread_entry(void) ""
+postcopy_ram_eager_load_thread_exit(void) ""
 
 # exec.c
 migration_exec_outgoing(const char *cmd) "cmd=%s"
#12 lore raw

Re: [RFC PATCH 3/5] migration: add eager load thread for fast snapshot load

Quoted 18 lines · Fabiano Rosas
Quoted 4 lines · Aadeshveer Singh

In fast snapshot load a thread is needed for actively loading in pages along with the fault path so that the guest is not dependent on fault thread indefinitely.

In the paragraph above, "dependent on fault thread indefinitely" is a bit vague: it's not clear why that condition might be undesirable or why it's indefinte.

How is that different from normal postcopy? Is the point that postcopy will have transferred most of the memory in precopy so the fault thread has way less pages to deal with whereas here is the opposite, just a few pages have been potentially loaded with a large chunk still waiting to be faulted in?

Quoted 4 lines

Is this new eager thread something that could be used even for normal postcopy? Kind of sounds like a separate sub-feature. Does the fast snapshot work at all without this?

Yes, the difference is the starting state of the RAM. In normal network postcopy, the precopy phase has already transferred the vast majority of the memory. The fault thread only has to fetch a small, finite number of remaining active pages.

In this snapshot load, we skip precopy entirely. The VM starts with 0% of its RAM loaded.

If we relied only on the fault thread, a page would load only if the guest OS explicitly accessed it. The hypervisor should not make any assumptions about the guest; this is what introduces the indefiniteness of load time.
If the guest OS never touches background pages, those pages will never load from disk. This would cause the migration state to hang indefinitely in POSTCOPY_ACTIVE, waiting for pages that might never be requested.

As you noted in Patch 0, this indefinite POSTCOPY_ACTIVE state would permanently block the user from ever issuing a subsequent savevm or migration command. The eager thread guarantees a deterministic end to the migration state machine.

Quoted 9 lines
Quoted 8 lines

Add postcopy_ram_eager_load_thread(), for the eager thread which iterates over all non ignored blocks calling ram_block_load_eager() each. ram_block_load_eager then iterates to load in all pages using postcopy_mapped_ram_load_page(), with a different channel, which takes care of not loading in pages already loaded by fault thread. On completion the thread schedules postcopy_ram_eager_load_bh() to destroy the incoming migration state and set states to completed/end.

Quoted 8 lines

Ok, so that answers my question from the cover letter, we're holding the migration until all pages have been loaded. So it's yet another reason to have the eager thread. Still, the fast snapshot load can be fast while having a slow migration overall I suppose.

To be clear, I'm not arguing against the design, only trying to frame the dependencies.

Yes, your summary is correct.

The "fast" in this feature refers strictly to the VM downtime (the time it takes for the guest to resume execution). The overall migration time is still fundamentally bound by the disk I/O required to load the entire RAM, so the total migration duration is essentially the same as a standard load or even slower than the current snapshot load.

Holding the migration state open until the eager thread finishes ensures we have a deterministic endpoint, keeping the VM safe from conflicting operations (like a second savevm) while its memory is still being populated in the background.

Quoted 149 lines
Quoted 149 lines

Add postcopy_ram_eager_load_setup() to create the thread. Added joining logic in postcopy_incoming_cleanup().

Add tracepoints for entry and exit to eager load thread.

---
 migration/migration.h    |  5 +++
 migration/postcopy-ram.c | 75 ++++++++++++++++++++++++++++++++++++++++
 migration/postcopy-ram.h |  2 ++
 migration/trace-events   |  2 ++
 4 files changed, 84 insertions(+)
migration/migration.h +5 -0
index 841f49b215..7bb54a6584 100644
@@ -42,6 +42,7 @@
 #define  MIGRATION_THREAD_DST_FAULT         "mig/dst/fault"
 #define  MIGRATION_THREAD_DST_LISTEN        "mig/dst/listen"
 #define  MIGRATION_THREAD_DST_PREEMPT       "mig/dst/preempt"
+#define  MIGRATION_THREAD_DST_EAGER         "mig/dst/eager"
 
 struct PostcopyBlocktimeContext;
 typedef struct ThreadPool ThreadPool;
@@ -120,6 +121,10 @@ struct MigrationIncomingState {
     bool           have_listen_thread;
     QemuThread     listen_thread;
 
+    /* Thread to load pages eagerly in fast snapshot load case */
+    bool have_eager_load_thread;
+    QemuThread eager_load_thread;
+
     /* For the kernel to send us notifications */
     int       userfault_fd;
     /* To notify the fault_thread to wake, e.g., when need to quit */
migration/postcopy-ram.c +75 -0
index 1ec20a07dd..0ee294a381 100644
@@ -2289,9 +2289,84 @@ int postcopy_incoming_cleanup(MigrationIncomingState *mis)
         mis->have_listen_thread = false;
     }
 
+    if (mis->have_eager_load_thread) {
+        qemu_thread_join(&mis->eager_load_thread);
+        mis->have_eager_load_thread = false;
+    }
+
     if (migrate_postcopy_ram()) {
         rc = postcopy_ram_incoming_cleanup(mis);
     }
 
     return rc;
 }
+
+/*
+ * Called by postcopy_ram_eager_load_thread over all blocks to load in all the
+ * pending pages of given ram block
+ */
+static int ram_block_load_eager(RAMBlock *rb, void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+    void *host = qemu_ram_get_host_addr(rb);
+    void *target;
+    int ret = 0;
+
+    for (ram_addr_t page_loc = 0; page_loc < rb->used_length;
+         page_loc += TARGET_PAGE_SIZE) {
+        target = (uint8_t *)host + page_loc;
+        ret = postcopy_mapped_ram_load_page(mis, rb, page_loc, (uint64_t)target,
+                                            0);
+        if (ret) {
+            break;
+        }
+    }
+    return ret;
+}
+
+/*
+ * Bottom half for fast snapshot load, scheduled by eager load thread
+ */
+static void postcopy_ram_eager_load_bh(void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+    postcopy_state_set(POSTCOPY_INCOMING_END);
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+                      MIGRATION_STATUS_COMPLETED);
+    migration_incoming_state_destroy();
+}
+
+/*
+ * Used by fast snapshot load to eagerly load in all pages of RAM and schedule
+ * cleanup after entire RAM is loaded
+ */
+static void *postcopy_ram_eager_load_thread(void *opaque)
+{
+    MigrationIncomingState *mis = opaque;
+
+    trace_postcopy_ram_eager_load_thread_entry();
+    rcu_register_thread();
+    qemu_event_set(&mis->thread_sync_event);
+
+    if (foreach_not_ignored_block(ram_block_load_eager, mis)) {
+        error_report("ram_block_load_eager failed");
+    }
+
+    migration_bh_schedule(postcopy_ram_eager_load_bh, mis);
+
+    rcu_unregister_thread();
+    trace_postcopy_ram_eager_load_thread_exit();
+    return NULL;
+}
+
+/*
+ * Create thread for eager loading in fast snapshot load case
+ */
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis)
+{
+    postcopy_thread_create(
+        mis, &mis->eager_load_thread, MIGRATION_THREAD_DST_EAGER,
+        postcopy_ram_eager_load_thread, QEMU_THREAD_JOINABLE);
+    mis->have_eager_load_thread = true;
+    return 0;
+}
migration/postcopy-ram.h +2 -0
index a080dd65a7..b3ba42e447 100644
@@ -202,4 +202,6 @@ void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
 int postcopy_incoming_setup(MigrationIncomingState *mis, Error **errp);
 int postcopy_incoming_cleanup(MigrationIncomingState *mis);
 
+int postcopy_ram_eager_load_setup(MigrationIncomingState *mis);
+
 #endif
migration/trace-events +2 -0
index de99d976ab..38f11e1e9f 100644
@@ -314,6 +314,8 @@ postcopy_blocktime_tid_cpu_map(int cpu, uint32_t tid) "cpu: %d, tid: %u"
 postcopy_blocktime_begin(uint64_t addr, uint64_t time, int cpu, bool exists) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", cpu: %d, exist: %d"
 postcopy_blocktime_end(uint64_t addr, uint64_t time, int affected_cpu, int affected_non_cpus) "addr: 0x%" PRIx64 ", time: %" PRIu64 ", affected_cpus: %d, affected_non_cpus: %d"
 postcopy_blocktime_end_one(int cpu, uint8_t left_faults) "cpu: %d, left_faults: %" PRIu8
+postcopy_ram_eager_load_thread_entry(void) ""
+postcopy_ram_eager_load_thread_exit(void) ""
 
 # exec.c
 migration_exec_outgoing(const char *cmd) "cmd=%s"
#13 lore raw

RFC PATCH 4/5 migration: write up code to run fast snapshot load in qemu_loadvm_state

In reply to Aadeshveer Singh

When both mapped-ram and postcopy-ram are set, divert from qemu_loadvm_state to run fast snapshot load

Initialize postcopy RAM state and register RAM Blocks with userfaultfd via ram_postcopy_incoming_init() and postcopy_ram_incoming_setup(). Launch fault thread before VM to serve faults for some hardwares emulation that need to read RAM (like vapic devices). Populate bitmaps and offset tables while reading file in qemu_loadvm_state_main. Call to qemu_loadvm_state_postcopy() which starts the VM using loadvm_postcopy_handle_run_bh() and launches eager load thread.

Skip scheduling process_incoming_migration_bh() in process_incoming_migration_co(), for fast snapshot load as the state cleanup is managed by eager load thread on completion.

Skip setting migration status to ACTIVE in process_incoming_migration_co and set set it to POSTCOPY_DEVICE in qemu_loadvm_state() itself.

Remove the capability check that rejected mapped-ram and postcopy-ram being set simultaneously, as this combination now corresponds to fast snapshot load. The corresponding test will be updated in following patch.

---
 migration/migration.c | 10 ++++++---
 migration/options.c   |  6 -----
 migration/savevm.c    | 52 +++++++++++++++++++++++++++++++++++++++++--
 migration/savevm.h    |  2 ++
 4 files changed, 59 insertions(+), 11 deletions(-)
migration/migration.c +7 -3
index 074d3f2c69..e1ac310e20 100644
@@ -756,8 +756,10 @@ process_incoming_migration_co(void *opaque)
 
     mis->largest_page_size = qemu_ram_pagesize_largest();
     postcopy_state_set(POSTCOPY_INCOMING_NONE);
-    migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
-                      MIGRATION_STATUS_ACTIVE);
+    if (!migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_ACTIVE);
+    }
 
     mis->loadvm_co = qemu_coroutine_self();
     ret = qemu_loadvm_state(mis->from_src_file, &local_err);
@@ -786,7 +788,9 @@ process_incoming_migration_co(void *opaque)
         colo_incoming_co();
     }
 
-    migration_bh_schedule(process_incoming_migration_bh, mis);
+    if (!migrate_fast_snapshot_load()) {
+        migration_bh_schedule(process_incoming_migration_bh, mis);
+    }
     goto out;
 
 fail:
migration/options.c +0 -6
index 5f80dd5b42..3f447cf7b2 100644
@@ -732,12 +732,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp)
                        "Mapped-ram migration is incompatible with xbzrle");
             return false;
         }
-
-        if (new_caps[MIGRATION_CAPABILITY_POSTCOPY_RAM]) {
-            error_setg(errp,
-                       "Mapped-ram migration is incompatible with postcopy");
-            return false;
-        }
     }
 
     /*
migration/savevm.c +50 -2

64 lines not on this page · raw message

migration/savevm.h +2 -0
index 96fdf96d4e..9656acd7fe 100644
@@ -67,6 +67,8 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,
 int qemu_save_device_state(QEMUFile *f, Error **errp);
 int qemu_loadvm_state(QEMUFile *f, Error **errp);
 void qemu_loadvm_state_cleanup(MigrationIncomingState *mis);
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp);
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp);
 int qemu_load_device_state(QEMUFile *f, Error **errp);
Signature · 1 line
2.54.0
#14 lore raw

Re: [RFC PATCH 4/5] migration: write up code to run fast snapshot load in qemu_loadvm_state

Quoted 36 lines · Aadeshveer Singh

When both mapped-ram and postcopy-ram are set, divert from qemu_loadvm_state to run fast snapshot load

Initialize postcopy RAM state and register RAM Blocks with userfaultfd via ram_postcopy_incoming_init() and postcopy_ram_incoming_setup(). Launch fault thread before VM to serve faults for some hardwares emulation that need to read RAM (like vapic devices). Populate bitmaps and offset tables while reading file in qemu_loadvm_state_main. Call to qemu_loadvm_state_postcopy() which starts the VM using loadvm_postcopy_handle_run_bh() and launches eager load thread.

Skip scheduling process_incoming_migration_bh() in process_incoming_migration_co(), for fast snapshot load as the state cleanup is managed by eager load thread on completion.

Skip setting migration status to ACTIVE in process_incoming_migration_co and set set it to POSTCOPY_DEVICE in qemu_loadvm_state() itself.

Remove the capability check that rejected mapped-ram and postcopy-ram being set simultaneously, as this combination now corresponds to fast snapshot load. The corresponding test will be updated in following patch.

---
 migration/migration.c | 10 ++++++---
 migration/options.c   |  6 -----
 migration/savevm.c    | 52 +++++++++++++++++++++++++++++++++++++++++--
 migration/savevm.h    |  2 ++
 4 files changed, 59 insertions(+), 11 deletions(-)
migration/migration.c +0 -0
index 074d3f2c69..e1ac310e20 100644

migration/migration.c process_incoming_migration_co(void *opaque) · line 762

Quoted 9 lines
migration/migration.c +4 -2
@@ -756,8 +756,10 @@ process_incoming_migration_co(void *opaque)
 
     mis->largest_page_size = qemu_ram_pagesize_largest();
     postcopy_state_set(POSTCOPY_INCOMING_NONE);
-    migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
-                      MIGRATION_STATUS_ACTIVE);
+    if (!migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_ACTIVE);
+    }

Is this really needed? The less common code we touch the better.

I think it's OK having ACTIVE during early load, set POSTCOPY_ACTIVE only until postcopy setup.

migration/migration.c process_incoming_migration_co(void *opaque) · line 793

Quoted 11 lines
migration/migration.c +3 -1
     mis->loadvm_co = qemu_coroutine_self();
     ret = qemu_loadvm_state(mis->from_src_file, &local_err);
@@ -786,7 +788,9 @@ process_incoming_migration_co(void *opaque)
         colo_incoming_co();
     }
 
-    migration_bh_schedule(process_incoming_migration_bh, mis);
+    if (!migrate_fast_snapshot_load()) {
+        migration_bh_schedule(process_incoming_migration_bh, mis);
+    }

IMHO we should reuse as much postcopy path as possible, like:

+static bool +migration_incoming_has_postcopy_thread(MigrationIncomingState *mis) +{
+ return mis->have_listen_thread || mis->have_eager_load_thread;
+}
+
static void coroutine_fn process_incoming_migration_co(void *opaque)
{
@@ -768,7 +774,7 @@ process_incoming_migration_co(void *opaque)

     trace_vmstate_downtime_checkpoint("dst-precopy-loadvm-completed");
 
     trace_process_incoming_migration_co_end(ret);

- if (mis->have_listen_thread) {
+ if (migration_incoming_has_postcopy_thread(mis)) {
/*
* Postcopy was started, cleanup should happen at the end of the
* postcopy listen thread.
@@ -788,9 +794,7 @@ process_incoming_migration_co(void *opaque)

         colo_incoming_co();
     }

- if (!migrate_fast_snapshot_load()) {
- migration_bh_schedule(process_incoming_migration_bh, mis);
- }

+    migration_bh_schedule(process_incoming_migration_bh, mis);                                                                          
     goto out;
Quoted 8 lines
migration/migration.c +0 -0
     goto out;
 
 fail:
migration/options.c +0 -0
index 5f80dd5b42..3f447cf7b2 100644

migration/options.c bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) · line 740

Quoted 9 lines
migration/options.c +0 -6
@@ -732,12 +732,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp)
                        "Mapped-ram migration is incompatible with xbzrle");
             return false;
         }
-
-        if (new_caps[MIGRATION_CAPABILITY_POSTCOPY_RAM]) {
-            error_setg(errp,
-                       "Mapped-ram migration is incompatible with postcopy");
-            return false;
-        }

Maybe you can split this change into the test file change that reverts that check, then it can be the last patch.

Quoted 41 lines
migration/options.c +0 -0
     }
 
     /*
migration/savevm.c +26 -0
index 23adaf9dd9..f10cc3c2fc 100644
@@ -2959,6 +2959,32 @@ static bool postcopy_pause_incoming(MigrationIncomingState *mis)
     return true;
 }
 
+/*
+ * Starts the VM and launches the eager thread for fast snapshot load
+ */
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp)
+{
+    ERRP_GUARD();
+    int ret = 0;
+
+    postcopy_state_set(POSTCOPY_INCOMING_RUNNING);
+
+    migration_bh_schedule(loadvm_postcopy_handle_run_bh, mis);
+
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_DEVICE,
+                      MIGRATION_STATUS_POSTCOPY_ACTIVE);
+
+    ret = postcopy_ram_eager_load_setup(mis);
+    if (ret) {
+        error_prepend(errp,
+                      "Failed to setup eager load for fast snapshot load: ");
+        return ret;
+    }
+
+    return ret;
+}
+
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp)
 {

migration/savevm.c int qemu_loadvm_state(QEMUFile *f, Error **errp) · line 3098

Quoted 8 lines
migration/savevm.c +3 -2
@@ -3067,8 +3093,30 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp)
 
     cpu_synchronize_all_pre_loadvm();
 
-    ret = qemu_loadvm_state_main(f, mis, errp);
-    qemu_event_set(&mis->main_thread_load_event);
+    if (migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_POSTCOPY_DEVICE);

Yes, switching to POSTCOPY_DEVICE seems reasonable here.

migration/savevm.c int qemu_loadvm_state(QEMUFile *f, Error **errp) · line 3106

Quoted 8 lines
migration/savevm.c +8 -0
+
+        if (ram_postcopy_incoming_init(mis, errp)) {
+            return -EINVAL;
+        }
+
+        postcopy_state_set(POSTCOPY_INCOMING_LISTENING);
+        if (postcopy_ram_incoming_setup(mis)) {
+            return -EINVAL;

Careful; when returning an error we must make sure *errp is set. See postcopy_incoming_setup().

migration/savevm.c int qemu_loadvm_state(QEMUFile *f, Error **errp) · line 3119

Quoted 13 lines
migration/savevm.c +13 -0
+        }
+
+        ret = qemu_loadvm_state_main(f, mis, errp);
+
+        qemu_event_set(&mis->main_thread_load_event);
+
+        if (ret == 0) {
+            ret = qemu_loadvm_state_postcopy(f, mis, errp);
+        }
+    } else {
+        ret = qemu_loadvm_state_main(f, mis, errp);
+        qemu_event_set(&mis->main_thread_load_event);
+    }

This chunk may not belong here, qemu_loadvm_state() has three callers, only process_incoming_migration_co() will use this logic. Let's move it out into process_incoming_migration_co() instead.

Quoted 19 lines
migration/savevm.c +0 -0
     trace_qemu_loadvm_state_post_main(ret);
 
migration/savevm.h +2 -0
index 96fdf96d4e..9656acd7fe 100644
@@ -67,6 +67,8 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,
 int qemu_save_device_state(QEMUFile *f, Error **errp);
 int qemu_loadvm_state(QEMUFile *f, Error **errp);
 void qemu_loadvm_state_cleanup(MigrationIncomingState *mis);
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp);
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp);
 int qemu_load_device_state(QEMUFile *f, Error **errp);
Signature · 1 line
2.54.0
Signature · 1 line
Peter Xu
#15 lore raw

Re: [RFC PATCH 4/5] migration: write up code to run fast snapshot load in qemu_loadvm_state

In reply to Aadeshveer Singh
Quoted 49 lines · Aadeshveer Singh

When both mapped-ram and postcopy-ram are set, divert from qemu_loadvm_state to run fast snapshot load

Initialize postcopy RAM state and register RAM Blocks with userfaultfd via ram_postcopy_incoming_init() and postcopy_ram_incoming_setup(). Launch fault thread before VM to serve faults for some hardwares emulation that need to read RAM (like vapic devices). Populate bitmaps and offset tables while reading file in qemu_loadvm_state_main. Call to qemu_loadvm_state_postcopy() which starts the VM using loadvm_postcopy_handle_run_bh() and launches eager load thread.

Skip scheduling process_incoming_migration_bh() in process_incoming_migration_co(), for fast snapshot load as the state cleanup is managed by eager load thread on completion.

Skip setting migration status to ACTIVE in process_incoming_migration_co and set set it to POSTCOPY_DEVICE in qemu_loadvm_state() itself.

Remove the capability check that rejected mapped-ram and postcopy-ram being set simultaneously, as this combination now corresponds to fast snapshot load. The corresponding test will be updated in following patch.

---
 migration/migration.c | 10 ++++++---
 migration/options.c   |  6 -----
 migration/savevm.c    | 52 +++++++++++++++++++++++++++++++++++++++++--
 migration/savevm.h    |  2 ++
 4 files changed, 59 insertions(+), 11 deletions(-)
migration/migration.c +4 -2
index 074d3f2c69..e1ac310e20 100644
@@ -756,8 +756,10 @@ process_incoming_migration_co(void *opaque)
 
     mis->largest_page_size = qemu_ram_pagesize_largest();
     postcopy_state_set(POSTCOPY_INCOMING_NONE);
-    migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
-                      MIGRATION_STATUS_ACTIVE);
+    if (!migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_ACTIVE);
+    }
 
     mis->loadvm_co = qemu_coroutine_self();
     ret = qemu_loadvm_state(mis->from_src_file, &local_err);

migration/migration.c process_incoming_migration_co(void *opaque) · line 793

Quoted 7 lines
migration/migration.c +3 -1
@@ -786,7 +788,9 @@ process_incoming_migration_co(void *opaque)
         colo_incoming_co();
     }
 
-    migration_bh_schedule(process_incoming_migration_bh, mis);
+    if (!migrate_fast_snapshot_load()) {
+        migration_bh_schedule(process_incoming_migration_bh, mis);
+    }

This inconspicuous 4 line hunk will cause bugs for years to come. =D

We should try to include the fast_snapshot actions within the incoming BH so the paths are the same in both types of migration. Ideally we'd merge all the BHs in a single one.

Quoted 8 lines
migration/migration.c +0 -0
     goto out;
 
 fail:
migration/options.c +0 -0
index 5f80dd5b42..3f447cf7b2 100644

migration/options.c bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) · line 740

Quoted 9 lines
migration/options.c +0 -6
@@ -732,12 +732,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp)
                        "Mapped-ram migration is incompatible with xbzrle");
             return false;
         }
-
-        if (new_caps[MIGRATION_CAPABILITY_POSTCOPY_RAM]) {
-            error_setg(errp,
-                       "Mapped-ram migration is incompatible with postcopy");
-            return false;
-        }

Just checking: Does this allow any type of migration that is not covered by this series?

Quoted 86 lines
migration/options.c +0 -0
     }
 
     /*
migration/savevm.c +50 -2
index 23adaf9dd9..f10cc3c2fc 100644
@@ -2959,6 +2959,32 @@ static bool postcopy_pause_incoming(MigrationIncomingState *mis)
     return true;
 }
 
+/*
+ * Starts the VM and launches the eager thread for fast snapshot load
+ */
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp)
+{
+    ERRP_GUARD();
+    int ret = 0;
+
+    postcopy_state_set(POSTCOPY_INCOMING_RUNNING);
+
+    migration_bh_schedule(loadvm_postcopy_handle_run_bh, mis);
+
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_DEVICE,
+                      MIGRATION_STATUS_POSTCOPY_ACTIVE);
+
+    ret = postcopy_ram_eager_load_setup(mis);
+    if (ret) {
+        error_prepend(errp,
+                      "Failed to setup eager load for fast snapshot load: ");
+        return ret;
+    }
+
+    return ret;
+}
+
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp)
 {
@@ -3067,8 +3093,30 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp)
 
     cpu_synchronize_all_pre_loadvm();
 
-    ret = qemu_loadvm_state_main(f, mis, errp);
-    qemu_event_set(&mis->main_thread_load_event);
+    if (migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_POSTCOPY_DEVICE);
+
+        if (ram_postcopy_incoming_init(mis, errp)) {
+            return -EINVAL;
+        }
+
+        postcopy_state_set(POSTCOPY_INCOMING_LISTENING);
+        if (postcopy_ram_incoming_setup(mis)) {
+            return -EINVAL;
+        }
+
+        ret = qemu_loadvm_state_main(f, mis, errp);
+
+        qemu_event_set(&mis->main_thread_load_event);
+
+        if (ret == 0) {
+            ret = qemu_loadvm_state_postcopy(f, mis, errp);
+        }
+    } else {
+        ret = qemu_loadvm_state_main(f, mis, errp);
+        qemu_event_set(&mis->main_thread_load_event);
+    }
 
     trace_qemu_loadvm_state_post_main(ret);
 
migration/savevm.h +2 -0
index 96fdf96d4e..9656acd7fe 100644
@@ -67,6 +67,8 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,
 int qemu_save_device_state(QEMUFile *f, Error **errp);
 int qemu_loadvm_state(QEMUFile *f, Error **errp);
 void qemu_loadvm_state_cleanup(MigrationIncomingState *mis);
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp);
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp);
 int qemu_load_device_state(QEMUFile *f, Error **errp);
#16 lore raw

Re: [RFC PATCH 4/5] migration: write up code to run fast snapshot load in qemu_loadvm_state

Quoted 62 lines · Fabiano Rosas
Quoted 49 lines · Aadeshveer Singh

When both mapped-ram and postcopy-ram are set, divert from qemu_loadvm_state to run fast snapshot load

Initialize postcopy RAM state and register RAM Blocks with userfaultfd via ram_postcopy_incoming_init() and postcopy_ram_incoming_setup(). Launch fault thread before VM to serve faults for some hardwares emulation that need to read RAM (like vapic devices). Populate bitmaps and offset tables while reading file in qemu_loadvm_state_main. Call to qemu_loadvm_state_postcopy() which starts the VM using loadvm_postcopy_handle_run_bh() and launches eager load thread.

Skip scheduling process_incoming_migration_bh() in process_incoming_migration_co(), for fast snapshot load as the state cleanup is managed by eager load thread on completion.

Skip setting migration status to ACTIVE in process_incoming_migration_co and set set it to POSTCOPY_DEVICE in qemu_loadvm_state() itself.

Remove the capability check that rejected mapped-ram and postcopy-ram being set simultaneously, as this combination now corresponds to fast snapshot load. The corresponding test will be updated in following patch.

---
 migration/migration.c | 10 ++++++---
 migration/options.c   |  6 -----
 migration/savevm.c    | 52 +++++++++++++++++++++++++++++++++++++++++--
 migration/savevm.h    |  2 ++
 4 files changed, 59 insertions(+), 11 deletions(-)
migration/migration.c +4 -2
index 074d3f2c69..e1ac310e20 100644
@@ -756,8 +756,10 @@ process_incoming_migration_co(void *opaque)
 
     mis->largest_page_size = qemu_ram_pagesize_largest();
     postcopy_state_set(POSTCOPY_INCOMING_NONE);
-    migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
-                      MIGRATION_STATUS_ACTIVE);
+    if (!migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_ACTIVE);
+    }
 
     mis->loadvm_co = qemu_coroutine_self();
     ret = qemu_loadvm_state(mis->from_src_file, &local_err);

migration/migration.c process_incoming_migration_co(void *opaque) · line 793

Quoted 7 lines
migration/migration.c +3 -1
@@ -786,7 +788,9 @@ process_incoming_migration_co(void *opaque)
         colo_incoming_co();
     }
 
-    migration_bh_schedule(process_incoming_migration_bh, mis);
+    if (!migrate_fast_snapshot_load()) {
+        migration_bh_schedule(process_incoming_migration_bh, mis);
+    }

This inconspicuous 4 line hunk will cause bugs for years to come. =D

Quoted 4 lines

We should try to include the fast_snapshot actions within the incoming BH so the paths are the same in both types of migration. Ideally we'd merge all the BHs in a single one.

Agreed! Bypassing the cleanup BH was definitely fragile.

I am working on adding the checks within the BH itself. Will resolve this in v2.

Quoted 12 lines
Quoted 12 lines
migration/migration.c +0 -0
     goto out;
 
 fail:
migration/options.c +0 -1
index 5f80dd5b42..3f447cf7b2 100644
@@ -732,12 +732,6 @@ bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp)
                        "Mapped-ram migration is incompatible with xbzrle");
             return false;
         }
-
Quoted 9 lines

migration/options.c bool migrate_caps_check(bool *old_caps, bool *new_caps, Error **errp) · line 740

Quoted 5 lines
migration/options.c +0 -5
-        if (new_caps[MIGRATION_CAPABILITY_POSTCOPY_RAM]) {
-            error_setg(errp,
-                       "Mapped-ram migration is incompatible with postcopy");
-            return false;
-        }

Just checking: Does this allow any type of migration that is not covered by this series?

By removing this check, we are specifically allowing the combination of mapped-ram and postcopy-ram.

Since mapped-ram fundamentally requires a file based migration (and enforces this elsewhere in the capability checks), this combination triggers the file-based postcopy behavior introduced in this series. It does not enable this combination for network-based postcopy because the underlying mapped-ram constraints still apply and block non-file transports.

Quoted 86 lines
Quoted 86 lines
migration/options.c +0 -0
     }
 
     /*
migration/savevm.c +50 -2
index 23adaf9dd9..f10cc3c2fc 100644
@@ -2959,6 +2959,32 @@ static bool postcopy_pause_incoming(MigrationIncomingState *mis)
     return true;
 }
 
+/*
+ * Starts the VM and launches the eager thread for fast snapshot load
+ */
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp)
+{
+    ERRP_GUARD();
+    int ret = 0;
+
+    postcopy_state_set(POSTCOPY_INCOMING_RUNNING);
+
+    migration_bh_schedule(loadvm_postcopy_handle_run_bh, mis);
+
+    migrate_set_state(&mis->state, MIGRATION_STATUS_POSTCOPY_DEVICE,
+                      MIGRATION_STATUS_POSTCOPY_ACTIVE);
+
+    ret = postcopy_ram_eager_load_setup(mis);
+    if (ret) {
+        error_prepend(errp,
+                      "Failed to setup eager load for fast snapshot load: ");
+        return ret;
+    }
+
+    return ret;
+}
+
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp)
 {
@@ -3067,8 +3093,30 @@ int qemu_loadvm_state(QEMUFile *f, Error **errp)
 
     cpu_synchronize_all_pre_loadvm();
 
-    ret = qemu_loadvm_state_main(f, mis, errp);
-    qemu_event_set(&mis->main_thread_load_event);
+    if (migrate_fast_snapshot_load()) {
+        migrate_set_state(&mis->state, MIGRATION_STATUS_SETUP,
+                          MIGRATION_STATUS_POSTCOPY_DEVICE);
+
+        if (ram_postcopy_incoming_init(mis, errp)) {
+            return -EINVAL;
+        }
+
+        postcopy_state_set(POSTCOPY_INCOMING_LISTENING);
+        if (postcopy_ram_incoming_setup(mis)) {
+            return -EINVAL;
+        }
+
+        ret = qemu_loadvm_state_main(f, mis, errp);
+
+        qemu_event_set(&mis->main_thread_load_event);
+
+        if (ret == 0) {
+            ret = qemu_loadvm_state_postcopy(f, mis, errp);
+        }
+    } else {
+        ret = qemu_loadvm_state_main(f, mis, errp);
+        qemu_event_set(&mis->main_thread_load_event);
+    }
 
     trace_qemu_loadvm_state_post_main(ret);
 
migration/savevm.h +2 -0
index 96fdf96d4e..9656acd7fe 100644
@@ -67,6 +67,8 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name,
 int qemu_save_device_state(QEMUFile *f, Error **errp);
 int qemu_loadvm_state(QEMUFile *f, Error **errp);
 void qemu_loadvm_state_cleanup(MigrationIncomingState *mis);
+int qemu_loadvm_state_postcopy(QEMUFile *f, MigrationIncomingState *mis,
+                               Error **errp);
 int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis,
                            Error **errp);
 int qemu_load_device_state(QEMUFile *f, Error **errp);
#17 lore raw

RFC PATCH 5/5 migration/tests: remove capability conflict test postcopy-ram+mapped-ram

In reply to Aadeshveer Singh

Remove the test test_validate_caps_pair, which asserted postcopy-ram and mapped-ram capabilities cannot be active together. The new fast snapshot load feature is exactly this pair of capabilities active together, with the previous patches in this series, this combination is now supported and functional.

No replacement test for fast snapshot load is included in this RFC. A test exercising the full save/load flow will be added in a follow-up.

---
 tests/qtest/migration/misc-tests.c | 52 ------------------------------
 1 file changed, 52 deletions(-)
tests/qtest/migration/misc-tests.c +0 -52
index ec6d438cdc..4e0deb7f18 100644
@@ -201,55 +201,6 @@ static void do_test_validate_uri_channel(MigrateCommon *args)
     migrate_end(from, to, false);
 }
 
-static void validate_caps_pair(QTestState *from,
-                               const char *first_capability,
-                               const char *second_capability,
-                               const char *expected_error)
-{
-    QDict *rsp;
-    const char *error_desc;
-
-    migrate_set_capability(from, first_capability, true);
-
-    rsp = qtest_qmp_assert_failure_ref(
-        from,
-        "{ 'execute': 'migrate-set-capabilities',"
-        "  'arguments': { 'capabilities': [ { "
-        "      'capability': %s, 'state': true } ] } }",
-        second_capability);
-
-    error_desc = qdict_get_str(rsp, "desc");
-    g_assert_cmpstr(error_desc, ==, expected_error);
-    qobject_unref(rsp);
-
-    migrate_set_capability(from, first_capability, false);
-}
-
-static void test_validate_caps_pair(char *test_path, MigrateCommon *args)
-{
-    g_autofree char *serial_path = g_strconcat(tmpfs, "/src_serial", NULL);
-    g_autofree char *cap_pair = g_path_get_basename(test_path);
-    QTestState *from, *to;
-
-    args->start.hide_stderr = true;
-    args->start.only_source = true;
-
-    if (migrate_start(&from, &to, &args->start)) {
-        return;
-    }
-
-    if (g_str_equal(cap_pair, "mapped_ram_postcopy")) {
-        const char *error =
-            "Mapped-ram migration is incompatible with postcopy";
-
-        validate_caps_pair(from, "mapped-ram", "postcopy-ram", error);
-        validate_caps_pair(from, "postcopy-ram", "mapped-ram", error);
-    }
-
-    qtest_quit(from);
-    unlink(serial_path);
-}
-
 static void test_validate_uri_channels_both_set(char *name, MigrateCommon *args)
 {
     args->uri = "tcp:127.0.0.1:0",
@@ -309,7 +260,4 @@ void migration_test_add_misc(MigrationTestEnv *env)
                        test_validate_uri_channels_both_set);
     migration_test_add("/migration/validate_uri/channels/none_set",
                        test_validate_uri_channels_none_set);
-    migration_test_add_suffix("/migration/validate_caps/",
-                              "mapped_ram_postcopy",
-                              test_validate_caps_pair);
 }
Signature · 1 line
2.54.0
#18 lore raw

Re: [RFC PATCH 5/5] migration/tests: remove capability conflict test postcopy-ram+mapped-ram

Quoted 10 lines · Aadeshveer Singh

Remove the test test_validate_caps_pair, which asserted postcopy-ram and mapped-ram capabilities cannot be active together. The new fast snapshot load feature is exactly this pair of capabilities active together, with the previous patches in this series, this combination is now supported and functional.

No replacement test for fast snapshot load is included in this RFC. A test exercising the full save/load flow will be added in a follow-up.

When sending formal patch, let's make this the 1st one otherwise previous patches will start to break qtest.

Signature · 1 line
Peter Xu
#19 lore raw
In reply to Aadeshveer Singh

Hi Everyone,

Adding this small patch on top to enable blocktime support. I will include it directly when I repost the v2 RFC.
Running cold cache tests most faults lie in the [64 us - 128 us] range(most likely pages loaded from disk) and a secondary mean is around [4 us - 8 us](likely for directly served zero pages).
Per CPU blocktime is a fraction of a second and offers a visible ergonomic improvement over precopy migration for snapshot loads, which previously required a few seconds to load the guest.

Test context:
- Guest: Fedora 44 (4 core, 16 GB RAM)
- Host: Fedora 43
- Host CPU: Intel ultra9 185H(22 cores)
- Host RAM: 32 GB DDR5x
- Host Drive: Gen 4 NVMe SSD
- Acceleration: KVM enabled

Cold Cache Output:
Postcopy Blocktime (ms): 0
Postcopy vCPU Blocktime (ms):
[128, 117, 111, 97]
Postcopy Latency (ns): 153571
Postcopy non-vCPU Latencies (ns): 149885
Postcopy vCPU Latencies (ns):
[234229, 225562, 195604, 209678]
Postcopy Latency Distribution:

    [     1 us -     2 us ]:         25
    [     2 us -     4 us ]:        370
    [     4 us -     8 us ]:       3938
    [     8 us -    16 us ]:       3599
    [    16 us -    32 us ]:        520
    [    32 us -    64 us ]:        192
    [    64 us -   128 us ]:      18643
    [   128 us -   256 us ]:       6199
    [   256 us -   512 us ]:       2768
    [   512 us -     1 ms ]:       1096
    [     1 ms -     2 ms ]:        486
    [     2 ms -     4 ms ]:         78
    [     4 ms -     8 ms ]:          5
    [     8 ms -    16 ms ]:          0
    [    16 ms -    32 ms ]:          0
    [    32 ms -    65 ms ]:          0
    [    65 ms -   131 ms ]:          0
    [   131 ms -   262 ms ]:          0
    [   262 ms -   524 ms ]:          0
    [   524 ms -    1 sec ]:          0
    [    1 sec -    2 sec ]:          0
    [    2 sec -    4 sec ]:          0
    [    4 sec -    8 sec ]:          0
    [    8 sec -   16 sec ]:          0

Thank you,
Aadeshveer

Quoted 70 lines · Aadeshveer Singh

This RFC implements a "fast snapshot load" mechanism to significantly reduce the perceived resume time of a VM from a snapshot file.

Currently, resuming a VM from a snapshot file requires loading all RAM pages into the QEMU instance before execution begins. This extension allows the user to run the VM nearly instantly by loading only the required device states up front and loading RAM pages lazily, by trapping access to pages that have not yet been loaded.

Using the Linux userfaultfd syscall, a fault thread catches all page faults caused by the guest and loads in the pages required to keep the VM running. Concurrently, an eager background thread iteratively loads all remaining pages into RAM so the guest does not have to depend on the fault thread indefinitely.

Much of code is reused from postcopy for fault handling and precopy for reading mapped ram file. Implementation revolves around two threads named the fault thread and eager load thread. Fault thread as name suggests catches page faults by the guest and serves them using userfaultfd. Postcopy fault thread is reused but instead of requesting source for a page it loads the page directly by reading form file. In order to remove the dependency of guest on fault thread indefinitely the eager load thread loads in the entire RAM sequentially, and after iterating through the entire RAM signals fault thread to exit and calls cleanup.

In order to prevent the case of a page being loaded twice(in the case when eager load thread is loading it and fault thread also tries to serve fault on same page) a bitmap called pending_bmap is used to track pages which are pending and not being loaded by any thread. Atomic operations on this bitmap allows coordination between threads to prevent any unwanted behaviours

This patch was tested using a Debian 13 bare minimum system and Fedora 44 KDE, snapshots for both are loaded successfully with no error.

Next Steps:
- Add testing framework, in qtest and unit tests
- Add support for postcopy-blocktime
- Update documentation

Future direction:
- Add support for hugepages
- Add support for multifd
- Add support for vhost-user

Aadeshveer Singh (5):
migration: add RAM Block fields and helpers for fast snapshot load
migration: add support for fault thread to load pages from disk
migration: add eager load thread for fast snapshot load
migration: write up code to run fast snapshot load in
qemu_loadvm_state
migration/tests: remove capability conflict test
postcopy-ram+mapped-ram

 include/system/ramblock.h          |   8 ++
 migration/migration.c              |  10 +-
 migration/migration.h              |   5 +
 migration/options.c                |  11 +-
 migration/options.h                |   1 +
 migration/postcopy-ram.c           | 167 ++++++++++++++++++++++++++---
 migration/postcopy-ram.h           |   2 +
 migration/qemu-file.c              |  10 +-
 migration/ram.c                    |  61 +++++++++--
 migration/savevm.c                 |  52 ++++++++-
 migration/savevm.h                 |   2 +
 migration/trace-events             |   2 +
 tests/qtest/migration/misc-tests.c |  52 ---------
 13 files changed, 283 insertions(+), 100 deletions(-)
Quoted 4 lines

--
2.54.0

From 70ab2949ef99968c2fc16e6a0d9860a993514367 Mon Sep 17 00:00:00 2001
From: Aadeshveer Singh <[email protected]>
Date: Fri, 19 Jun 2026 18:12:36 +0530
Subject: [PATCH] migration: postcopy-blocktime support for fast snapshot load

Add postcopy-blocktime support to fast snapshot load by calling mark_postcopy_blocktime_begin(), on all page faults intercepted by fault thread.

There is no need to call mark_postcopy_blocktime_end(), as postcopy_mapped_ram_load_page() calls postcopy_place_page() and postcopy_place_page_zero() which call the end marking internally.

---
 migration/postcopy-ram.c | 6 ++++++
 1 file changed, 6 insertions(+)
migration/postcopy-ram.c +6 -0
index 0ee294a381..2f4698fbed 100644
@@ -1436,6 +1436,12 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 msg.arg.pagefault.feat.ptid);
 
             if (migrate_fast_snapshot_load()) {
+                WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex)
+                {
+                    mark_postcopy_blocktime_begin(msg.arg.pagefault.address,
+                                                  msg.arg.pagefault.feat.ptid,
+                                                  rb);
+                }
                 if (postcopy_mapped_ram_load_page(
                         mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {
                     break;
Signature · 1 line
2.54.0
#20 lore raw
Quoted 25 lines · Aadeshveer Singh

From 70ab2949ef99968c2fc16e6a0d9860a993514367 Mon Sep 17 00:00:00 2001
From: Aadeshveer Singh <[email protected]>
Date: Fri, 19 Jun 2026 18:12:36 +0530
Subject: [PATCH] migration: postcopy-blocktime support for fast snapshot load

Add postcopy-blocktime support to fast snapshot load by calling mark_postcopy_blocktime_begin(), on all page faults intercepted by fault thread.

There is no need to call mark_postcopy_blocktime_end(), as postcopy_mapped_ram_load_page() calls postcopy_place_page() and postcopy_place_page_zero() which call the end marking internally.

---
 migration/postcopy-ram.c | 6 ++++++
 1 file changed, 6 insertions(+)
migration/postcopy-ram.c +0 -0
index 0ee294a381..2f4698fbed 100644
@@ -1436,6 +1436,12 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 msg.arg.pagefault.feat.ptid);
 

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1447

Quoted 10 lines
migration/postcopy-ram.c +6 -0
             if (migrate_fast_snapshot_load()) {
+                WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex)
+                {
+                    mark_postcopy_blocktime_begin(msg.arg.pagefault.address,
+                                                  msg.arg.pagefault.feat.ptid,
+                                                  rb);
+                }
                 if (postcopy_mapped_ram_load_page(
                         mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {
                     break;

Let's squash this directly to your core patch 4, then mention it in the commit log.

Even if I left quite some comments, most of them are small nitpicks. It's good to know there're only a few postcopy functions need some touch, and most logics can be reused.

The RFC series looks a great start, thank you!

Signature · 1 line
Peter Xu
#21 lore raw

Hi Peter,

Thank you for the detailed review of the RFC.
I agree with the feedback across the series. For v2, I will:
- Squash the blocktime patch directly into Patch 4 as you suggested.
- Apply the QEMU styling fixes (reverse christmas tree, g_clear_pointer, macro braces).
- Replace the g_assert() calls with standard Error errp propagation to gracefully handle disk read failures.
- Move the capability conflict test removal to Patch 1.
I have one architectural clarification regarding the bitmap usage on Patch 1, which I will reply to inline on that specific thread. I will get to work on v2.

Thank you,
Aadeshveer Singh

Quoted 50 lines · Peter Xu
Quoted 25 lines · Aadeshveer Singh

From 70ab2949ef99968c2fc16e6a0d9860a993514367 Mon Sep 17 00:00:00 2001
From: Aadeshveer Singh <[email protected]>
Date: Fri, 19 Jun 2026 18:12:36 +0530
Subject: [PATCH] migration: postcopy-blocktime support for fast snapshot load

Add postcopy-blocktime support to fast snapshot load by calling mark_postcopy_blocktime_begin(), on all page faults intercepted by fault thread.

There is no need to call mark_postcopy_blocktime_end(), as postcopy_mapped_ram_load_page() calls postcopy_place_page() and postcopy_place_page_zero() which call the end marking internally.

---
 migration/postcopy-ram.c | 6 ++++++
 1 file changed, 6 insertions(+)
migration/postcopy-ram.c +0 -0
index 0ee294a381..2f4698fbed 100644
@@ -1436,6 +1436,12 @@ static void *postcopy_ram_fault_thread(void *opaque)
                                                 msg.arg.pagefault.feat.ptid);
 

migration/postcopy-ram.c static void *postcopy_ram_fault_thread(void *opaque) · line 1447

Quoted 10 lines
migration/postcopy-ram.c +6 -0
             if (migrate_fast_snapshot_load()) {
+                WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex)
+                {
+                    mark_postcopy_blocktime_begin(msg.arg.pagefault.address,
+                                                  msg.arg.pagefault.feat.ptid,
+                                                  rb);
+                }
                 if (postcopy_mapped_ram_load_page(
                         mis, rb, rb_offset, msg.arg.pagefault.address, 1)) {
                     break;

Let's squash this directly to your core patch 4, then mention it in the commit log.

Even if I left quite some comments, most of them are small nitpicks. It's good to know there're only a few postcopy functions need some touch, and most logics can be reused.

The RFC series looks a great start, thank you!

--
Peter Xu

#22 lore raw
In reply to Aadeshveer Singh
Quoted 26 lines · Aadeshveer Singh

This RFC implements a "fast snapshot load" mechanism to significantly reduce the perceived resume time of a VM from a snapshot file.

Currently, resuming a VM from a snapshot file requires loading all RAM pages into the QEMU instance before execution begins. This extension allows the user to run the VM nearly instantly by loading only the required device states up front and loading RAM pages lazily, by trapping access to pages that have not yet been loaded.

Using the Linux userfaultfd syscall, a fault thread catches all page faults caused by the guest and loads in the pages required to keep the VM running. Concurrently, an eager background thread iteratively loads all remaining pages into RAM so the guest does not have to depend on the fault thread indefinitely.

Much of code is reused from postcopy for fault handling and precopy for reading mapped ram file. Implementation revolves around two threads named the fault thread and eager load thread. Fault thread as name suggests catches page faults by the guest and serves them using userfaultfd. Postcopy fault thread is reused but instead of requesting source for a page it loads the page directly by reading form file. In order to remove the dependency of guest on fault thread indefinitely the eager load thread loads in the entire RAM sequentially, and after iterating through the entire RAM signals fault thread to exit and calls cleanup.

Quoted 7 lines

In order to prevent the case of a page being loaded twice(in the case when eager load thread is loading it and fault thread also tries to serve fault on same page) a bitmap called pending_bmap is used to track pages which are pending and not being loaded by any thread. Atomic operations on this bitmap allows coordination between threads to prevent any unwanted behaviours

How does it work if a second savevm happens while the RAM has not been yet entirely loaded?

Quoted 36 lines

This patch was tested using a Debian 13 bare minimum system and Fedora 44 KDE, snapshots for both are loaded successfully with no error.

Next Steps:
- Add testing framework, in qtest and unit tests
- Add support for postcopy-blocktime
- Update documentation

Future direction:
- Add support for hugepages
- Add support for multifd
- Add support for vhost-user

Aadeshveer Singh (5):
migration: add RAM Block fields and helpers for fast snapshot load
migration: add support for fault thread to load pages from disk
migration: add eager load thread for fast snapshot load
migration: write up code to run fast snapshot load in
qemu_loadvm_state
migration/tests: remove capability conflict test
postcopy-ram+mapped-ram

 include/system/ramblock.h          |   8 ++
 migration/migration.c              |  10 +-
 migration/migration.h              |   5 +
 migration/options.c                |  11 +-
 migration/options.h                |   1 +
 migration/postcopy-ram.c           | 167 ++++++++++++++++++++++++++---
 migration/postcopy-ram.h           |   2 +
 migration/qemu-file.c              |  10 +-
 migration/ram.c                    |  61 +++++++++--
 migration/savevm.c                 |  52 ++++++++-
 migration/savevm.h                 |   2 +
 migration/trace-events             |   2 +
 tests/qtest/migration/misc-tests.c |  52 ---------
 13 files changed, 283 insertions(+), 100 deletions(-)
#23 lore raw
Quoted 37 lines · Fabiano Rosas
Quoted 26 lines · Aadeshveer Singh

This RFC implements a "fast snapshot load" mechanism to significantly reduce the perceived resume time of a VM from a snapshot file.

Currently, resuming a VM from a snapshot file requires loading all RAM pages into the QEMU instance before execution begins. This extension allows the user to run the VM nearly instantly by loading only the required device states up front and loading RAM pages lazily, by trapping access to pages that have not yet been loaded.

Using the Linux userfaultfd syscall, a fault thread catches all page faults caused by the guest and loads in the pages required to keep the VM running. Concurrently, an eager background thread iteratively loads all remaining pages into RAM so the guest does not have to depend on the fault thread indefinitely.

Much of code is reused from postcopy for fault handling and precopy for reading mapped ram file. Implementation revolves around two threads named the fault thread and eager load thread. Fault thread as name suggests catches page faults by the guest and serves them using userfaultfd. Postcopy fault thread is reused but instead of requesting source for a page it loads the page directly by reading form file. In order to remove the dependency of guest on fault thread indefinitely the eager load thread loads in the entire RAM sequentially, and after iterating through the entire RAM signals fault thread to exit and calls cleanup.

Quoted 7 lines

In order to prevent the case of a page being loaded twice(in the case when eager load thread is loading it and fault thread also tries to serve fault on same page) a bitmap called pending_bmap is used to track pages which are pending and not being loaded by any thread. Atomic operations on this bitmap allows coordination between threads to prevent any unwanted behaviours

Quoted 2 lines

How does it work if a second savevm happens while the RAM has not been yet entirely loaded?

Hi Fabiano,
Thanks for the review across the series.

To answer this: If a second savevm is issued before the RAM is fully loaded, it will be blocked. Because the migration state remains MIGRATION_STATUS_POSTCOPY_ACTIVE until the eager thread finishes reading the last page from disk, standard QEMU migration checks will reject any new savevm or migration requests.

Quoted 37 lines
Quoted 37 lines

This patch was tested using a Debian 13 bare minimum system and Fedora 44 KDE, snapshots for both are loaded successfully with no error.

Next Steps:
- Add testing framework, in qtest and unit tests
- Add support for postcopy-blocktime
- Update documentation

Future direction:
- Add support for hugepages
- Add support for multifd
- Add support for vhost-user

Aadeshveer Singh (5):
migration: add RAM Block fields and helpers for fast snapshot load
migration: add support for fault thread to load pages from disk
migration: add eager load thread for fast snapshot load
migration: write up code to run fast snapshot load in
qemu_loadvm_state
migration/tests: remove capability conflict test
postcopy-ram+mapped-ram

 include/system/ramblock.h          |   8 ++
 migration/migration.c              |  10 +-
 migration/migration.h              |   5 +
 migration/options.c                |  11 +-
 migration/options.h                |   1 +
 migration/postcopy-ram.c           | 167 ++++++++++++++++++++++++++---
 migration/postcopy-ram.h           |   2 +
 migration/qemu-file.c              |  10 +-
 migration/ram.c                    |  61 +++++++++--
 migration/savevm.c                 |  52 ++++++++-
 migration/savevm.h                 |   2 +
 migration/trace-events             |   2 +
 tests/qtest/migration/misc-tests.c |  52 ---------
 13 files changed, 283 insertions(+), 100 deletions(-)