Skip to contentSkip to conversation

migration: cleanup ram_load

PATCH Superseded Wei Yang 3 participants 8 messages Jul 22, 2019 – Jul 24, 2019

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Wei Yang

author
#0 lore raw

cover migration: cleanup ram_load

Two cleanup for ram_load:

* return -EINVAL for version_id mismatch
* extract ram_load_precopy for better readability

Wei Yang (2):
migration: return -EINVAL directly when version_id mismatch
migration: extract ram_load_precopy

 migration/ram.c | 73 +++++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 27 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/2 migration: return -EINVAL directly when version_id mismatch

It is not reasonable to continue when version_id mismatch.

---
 migration/ram.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
migration/ram.c +1 -1
index 7e34c82a72..6bfdfae16e 100644
@@ -4216,7 +4216,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
     seq_iter++;
 
     if (version_id != 4) {
-        ret = -EINVAL;
+        return -EINVAL;
     }
 
     if (!migrate_use_compression()) {
Signature · 1 line
2.17.1
Reviewed-by on v1 1/2
#2 lore raw

Re: [Qemu-devel] [PATCH 1/2] migration: return -EINVAL directly when version_id mismatch

Quoted 12 lines · * Wei Yang ([email protected]) wrote:

It is not reasonable to continue when version_id mismatch.

---
 migration/ram.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
migration/ram.c +0 -0
index 7e34c82a72..6bfdfae16e 100644

migration/ram.c static int ram_load(QEMUFile *f, void *opaque, int version_id) · line 4220

Quoted 6 lines
migration/ram.c +1 -1
@@ -4216,7 +4216,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
     seq_iter++;
 
     if (version_id != 4) {
-        ret = -EINVAL;
+        return -EINVAL;
     }

Oh yes that's nice; it's before the lock gets taken so we can just return

Quoted 4 lines
migration/ram.c +0 -0
     if (!migrate_use_compression()) {
Signature · 1 line
2.17.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

author
#3 lore raw

PATCH 2/2 migration: extract ram_load_precopy

In reply to Wei Yang

After cleanup, it would be clear to audience there are two cases ram_load:

* precopy
* postcopy

And it is not necessary to check postcopy_running on each iteration for precopy.

---
 migration/ram.c | 73 +++++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 27 deletions(-)
migration/ram.c +46 -27
index 6bfdfae16e..5f6f07b255 100644
@@ -4200,40 +4200,26 @@ static void colo_flush_ram_cache(void)
     trace_colo_flush_ram_cache_end();
 }
 
-static int ram_load(QEMUFile *f, void *opaque, int version_id)
+/**
+ * ram_load_precopy: load a page in precopy case
+ *
+ * Returns 0 for success or -errno in case of error
+ *
+ * Called in precopy mode by ram_load().
+ * rcu_read_lock is taken prior to this being called.
+ *
+ * @f: QEMUFile where to send the data
+ */
+static int ram_load_precopy(QEMUFile *f)
 {
-    int flags = 0, ret = 0, invalid_flags = 0;
-    static uint64_t seq_iter;
-    int len = 0;
-    /*
-     * If system is running in postcopy mode, page inserts to host memory must
-     * be atomic
-     */
-    bool postcopy_running = postcopy_is_running();
+    int flags = 0, ret = 0, invalid_flags = 0, len = 0;
     /* ADVISE is earlier, it shows the source has the postcopy capability on */
     bool postcopy_advised = postcopy_is_advised();
-
-    seq_iter++;
-
-    if (version_id != 4) {
-        return -EINVAL;
-    }
-
     if (!migrate_use_compression()) {
         invalid_flags |= RAM_SAVE_FLAG_COMPRESS_PAGE;
     }
-    /* This RCU critical section can be very long running.
-     * When RCU reclaims in the code start to become numerous,
-     * it will be necessary to reduce the granularity of this
-     * critical section.
-     */
-    rcu_read_lock();
-
-    if (postcopy_running) {
-        ret = ram_load_postcopy(f);
-    }
 
-    while (!postcopy_running && !ret && !(flags & RAM_SAVE_FLAG_EOS)) {
+    while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) {
         ram_addr_t addr, total_ram_bytes;
         void *host = NULL;
         uint8_t ch;
@@ -4390,6 +4376,39 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
         }
     }
 
+    return ret;
+}
+
+static int ram_load(QEMUFile *f, void *opaque, int version_id)
+{
+    int ret = 0;
+    static uint64_t seq_iter;
+    /*
+     * If system is running in postcopy mode, page inserts to host memory must
+     * be atomic
+     */
+    bool postcopy_running = postcopy_is_running();
+
+    seq_iter++;
+
+    if (version_id != 4) {
+        return -EINVAL;
+    }
+
+    /*
+     * This RCU critical section can be very long running.
+     * When RCU reclaims in the code start to become numerous,
+     * it will be necessary to reduce the granularity of this
+     * critical section.
+     */
+    rcu_read_lock();
+
+    if (postcopy_running) {
+        ret = ram_load_postcopy(f);
+    } else {
+        ret = ram_load_precopy(f);
+    }
+
     ret |= wait_for_decompress_done();
     rcu_read_unlock();
     trace_ram_load_complete(ret, seq_iter);
Signature · 1 line
2.17.1
Reviewed-by on v1 2/2
#4 lore raw

Re: [Qemu-devel] [PATCH 2/2] migration: extract ram_load_precopy

Quoted 19 lines · * Wei Yang ([email protected]) wrote:

After cleanup, it would be clear to audience there are two cases ram_load:

* precopy
* postcopy

And it is not necessary to check postcopy_running on each iteration for precopy.

---
 migration/ram.c | 73 +++++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 27 deletions(-)
migration/ram.c +0 -0
index 6bfdfae16e..5f6f07b255 100644

migration/ram.c static void colo_flush_ram_cache(void) · line 4204

Quoted 6 lines
migration/ram.c +2 -1
@@ -4200,40 +4200,26 @@ static void colo_flush_ram_cache(void)
     trace_colo_flush_ram_cache_end();
 }
 
-static int ram_load(QEMUFile *f, void *opaque, int version_id)
+/**
+ * ram_load_precopy: load a page in precopy case

This comment is wrong - although I realise you copied it from the postcopy case; they don't just load a single page; they load 'pages'

Other than that, I think it's OK, so:

Quoted 89 lines
migration/ram.c +43 -26
+ * Returns 0 for success or -errno in case of error
+ *
+ * Called in precopy mode by ram_load().
+ * rcu_read_lock is taken prior to this being called.
+ *
+ * @f: QEMUFile where to send the data
+ */
+static int ram_load_precopy(QEMUFile *f)
 {
-    int flags = 0, ret = 0, invalid_flags = 0;
-    static uint64_t seq_iter;
-    int len = 0;
-    /*
-     * If system is running in postcopy mode, page inserts to host memory must
-     * be atomic
-     */
-    bool postcopy_running = postcopy_is_running();
+    int flags = 0, ret = 0, invalid_flags = 0, len = 0;
     /* ADVISE is earlier, it shows the source has the postcopy capability on */
     bool postcopy_advised = postcopy_is_advised();
-
-    seq_iter++;
-
-    if (version_id != 4) {
-        return -EINVAL;
-    }
-
     if (!migrate_use_compression()) {
         invalid_flags |= RAM_SAVE_FLAG_COMPRESS_PAGE;
     }
-    /* This RCU critical section can be very long running.
-     * When RCU reclaims in the code start to become numerous,
-     * it will be necessary to reduce the granularity of this
-     * critical section.
-     */
-    rcu_read_lock();
-
-    if (postcopy_running) {
-        ret = ram_load_postcopy(f);
-    }
 
-    while (!postcopy_running && !ret && !(flags & RAM_SAVE_FLAG_EOS)) {
+    while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) {
         ram_addr_t addr, total_ram_bytes;
         void *host = NULL;
         uint8_t ch;
@@ -4390,6 +4376,39 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
         }
     }
 
+    return ret;
+}
+
+static int ram_load(QEMUFile *f, void *opaque, int version_id)
+{
+    int ret = 0;
+    static uint64_t seq_iter;
+    /*
+     * If system is running in postcopy mode, page inserts to host memory must
+     * be atomic
+     */
+    bool postcopy_running = postcopy_is_running();
+
+    seq_iter++;
+
+    if (version_id != 4) {
+        return -EINVAL;
+    }
+
+    /*
+     * This RCU critical section can be very long running.
+     * When RCU reclaims in the code start to become numerous,
+     * it will be necessary to reduce the granularity of this
+     * critical section.
+     */
+    rcu_read_lock();
+
+    if (postcopy_running) {
+        ret = ram_load_postcopy(f);
+    } else {
+        ret = ram_load_precopy(f);
+    }
+
     ret |= wait_for_decompress_done();
     rcu_read_unlock();
     trace_ram_load_complete(ret, seq_iter);
Signature · 1 line
2.17.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

author
#5 lore raw

Re: [Qemu-devel] [PATCH 2/2] migration: extract ram_load_precopy

Quoted 23 lines · Dr. David Alan Gilbert
Quoted 22 lines · * Wei Yang ([email protected]) wrote:

After cleanup, it would be clear to audience there are two cases ram_load:

* precopy
* postcopy

And it is not necessary to check postcopy_running on each iteration for precopy.

---
 migration/ram.c | 73 +++++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 27 deletions(-)
migration/ram.c +0 -0
index 6bfdfae16e..5f6f07b255 100644
@@ -4200,40 +4200,26 @@ static void colo_flush_ram_cache(void)
     trace_colo_flush_ram_cache_end();
 }
 
Quoted 7 lines

migration/ram.c static void colo_flush_ram_cache(void) · line 4204

Quoted 3 lines
migration/ram.c +2 -1
-static int ram_load(QEMUFile *f, void *opaque, int version_id)
+/**
+ * ram_load_precopy: load a page in precopy case

This comment is wrong - although I realise you copied it from the postcopy case; they don't just load a single page; they load 'pages'

Thanks for pointing out.

Actually, I got one confusion in these two load. Compare these two cases, I found precopy would handle two more cases:

* precopy: RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE
* postcopy: RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE

Why postcopy doesn't need to handle the other two cases? Function ram_save_target_page() does the same thing in precopy and postcopy. I don't find the reason the behavior differs. Would you mind giving me a hint?

Quoted 5 lines

Other than that, I think it's OK, so:

Signature · 2 lines
Wei Yang
Help you, Help me
#6 lore raw

Re: [Qemu-devel] [PATCH 2/2] migration: extract ram_load_precopy

Quoted 37 lines · * Wei Yang ([email protected]) wrote:
Quoted 23 lines · Dr. David Alan Gilbert
Quoted 22 lines · * Wei Yang ([email protected]) wrote:

After cleanup, it would be clear to audience there are two cases ram_load:

* precopy
* postcopy

And it is not necessary to check postcopy_running on each iteration for precopy.

---
 migration/ram.c | 73 +++++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 27 deletions(-)
migration/ram.c +0 -0
index 6bfdfae16e..5f6f07b255 100644
@@ -4200,40 +4200,26 @@ static void colo_flush_ram_cache(void)
     trace_colo_flush_ram_cache_end();
 }
 
Quoted 7 lines

migration/ram.c static void colo_flush_ram_cache(void) · line 4204

Quoted 3 lines
migration/ram.c +2 -1
-static int ram_load(QEMUFile *f, void *opaque, int version_id)
+/**
+ * ram_load_precopy: load a page in precopy case

This comment is wrong - although I realise you copied it from the postcopy case; they don't just load a single page; they load 'pages'

Thanks for pointing out.

Actually, I got one confusion in these two load. Compare these two cases, I found precopy would handle two more cases:

Quoted 7 lines

* precopy: RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE
* postcopy: RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE

Why postcopy doesn't need to handle the other two cases? Function ram_save_target_page() does the same thing in precopy and postcopy. I don't find the reason the behavior differs. Would you mind giving me a hint?

Because we don't support either compression or xbzrle with postcopy. Compression could be fixed, but it needs to make sure it uses the place-page function to atomically place the page.

xbzrle never gets used during the postcopy stage; it gets used in the precopy stage in a migration that might switch to postcopy though. Since xbzrle relies on optimising differences between passes, it's
1) Not needed in postcopy where there's only one final pass
2) Since the destination is changing RAM, you can't transmit deltas relative to the old data, since that data may have changed.

Dave

Quoted 9 lines
Quoted 5 lines

Other than that, I think it's OK, so:

Signature · 2 lines
Wei Yang
Help you, Help me

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#7 lore raw

Re: [Qemu-devel] [PATCH 2/2] migration: extract ram_load_precopy

Quoted 42 lines · Dr. David Alan Gilbert
Quoted 41 lines · * Wei Yang ([email protected]) wrote:
Quoted 23 lines · Dr. David Alan Gilbert
Quoted 22 lines · * Wei Yang ([email protected]) wrote:

After cleanup, it would be clear to audience there are two cases ram_load:

* precopy
* postcopy

And it is not necessary to check postcopy_running on each iteration for precopy.

---
 migration/ram.c | 73 +++++++++++++++++++++++++++++++------------------
 1 file changed, 46 insertions(+), 27 deletions(-)
migration/ram.c +0 -0
index 6bfdfae16e..5f6f07b255 100644
@@ -4200,40 +4200,26 @@ static void colo_flush_ram_cache(void)
     trace_colo_flush_ram_cache_end();
 }
 
Quoted 7 lines

migration/ram.c static void colo_flush_ram_cache(void) · line 4204

Quoted 3 lines
migration/ram.c +2 -1
-static int ram_load(QEMUFile *f, void *opaque, int version_id)
+/**
+ * ram_load_precopy: load a page in precopy case

This comment is wrong - although I realise you copied it from the postcopy case; they don't just load a single page; they load 'pages'

Thanks for pointing out.

Actually, I got one confusion in these two load. Compare these two cases, I found precopy would handle two more cases:

* precopy: RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE
* postcopy: RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE

Quoted 8 lines
Quoted 3 lines

Why postcopy doesn't need to handle the other two cases? Function ram_save_target_page() does the same thing in precopy and postcopy. I don't find the reason the behavior differs. Would you mind giving me a hint?

Because we don't support either compression or xbzrle with postcopy. Compression could be fixed, but it needs to make sure it uses the place-page function to atomically place the page.

Thanks for the explanation. Sounds I missed some point.

The place-page function to use is postcopy_place_page()?

Quoted 22 lines

xbzrle never gets used during the postcopy stage; it gets used in the precopy stage in a migration that might switch to postcopy though. Since xbzrle relies on optimising differences between passes, it's
1) Not needed in postcopy where there's only one final pass
2) Since the destination is changing RAM, you can't transmit deltas relative to the old data, since that data may have changed.

Dave

Quoted 9 lines
Quoted 5 lines

Other than that, I think it's OK, so:

Signature · 2 lines
Wei Yang
Help you, Help me

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 2 lines
Wei Yang
Help you, Help me