Skip to contentSkip to conversation

migration/ram: Merge zero page handling

PATCH In review Fabiano Rosas 3 participants 9 messages Aug 16, 2023 – Aug 22, 2023

lore
Share & tools
lore
#0 lore raw

v2 cover migration/ram: Merge zero page handling

For v2 I fixed patch 3 which had a hunk belonging to patch 5.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/969706915

v1:
https://lore.kernel.org/r/[email protected]

Hi,

This is another small series that I extracted from my fixed-ram series and that could be already considered for merging.

This is just code movement, no functional change. The objective is to consolidate the zero page handling in the same routine that saves the page header and does accounting. Then in the future I'll be able to just return early because fixed-ram ignores zero pages.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/968300062

Fabiano Rosas (4):
migration/ram: Remove RAMState from xbzrle_cache_zero_page migration/ram: Stop passing QEMUFile around in save_zero_page migration/ram: Move xbzrle zero page handling into save_zero_page migration/ram: Merge save_zero_page functions

 migration/ram.c | 75 ++++++++++++++++++++-----------------------------
 1 file changed, 30 insertions(+), 45 deletions(-)
Signature · 1 line
2.35.3
#1 lore raw

PATCH v2 1/4 migration/ram: Remove RAMState from xbzrle_cache_zero_page

'rs' is not used in that function. It's a leftover from commit 9360447d34 ("ram: Use MigrationStats for statistics").

---
 migration/ram.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
migration/ram.c +2 -3
index 9040d66e61..87efab72e8 100644
@@ -561,7 +561,6 @@ void mig_throttle_counter_reset(void)
 /**
  * xbzrle_cache_zero_page: insert a zero page in the XBZRLE cache
  *
- * @rs: current RAM state
  * @current_addr: address for the zero page
  *
  * Update the xbzrle cache to reflect a page that's been sent as all 0.
@@ -570,7 +569,7 @@ void mig_throttle_counter_reset(void)
  * As a bonus, if the page wasn't in the cache it gets added so that
  * when a small write is made into the 0'd page it gets XBZRLE sent.
  */
-static void xbzrle_cache_zero_page(RAMState *rs, ram_addr_t current_addr)
+static void xbzrle_cache_zero_page(ram_addr_t current_addr)
 {
     /* We don't care if this fails to allocate a new cache page
      * as long as it updated an old one */
@@ -2148,7 +2147,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
          */
         if (rs->xbzrle_started) {
             XBZRLE_cache_lock();
-            xbzrle_cache_zero_page(rs, block->offset + offset);
+            xbzrle_cache_zero_page(block->offset + offset);
             XBZRLE_cache_unlock();
         }
         return res;
Signature · 1 line
2.35.3
#2 lore raw

PATCH v2 2/4 migration/ram: Stop passing QEMUFile around in save_zero_page

In reply to Fabiano Rosas

We don't need the QEMUFile when we're already passing the PageSearchStatus.

---
 migration/ram.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
migration/ram.c +6 -5
index 87efab72e8..761f43dc34 100644
@@ -1138,10 +1138,11 @@ void ram_release_page(const char *rbname, uint64_t offset)
  * @block: block that contains the page we want to send
  * @offset: offset inside the block for the page
  */
-static int save_zero_page_to_file(PageSearchStatus *pss, QEMUFile *file,
-                                  RAMBlock *block, ram_addr_t offset)
+static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
+                                  ram_addr_t offset)
 {
     uint8_t *p = block->host + offset;
+    QEMUFile *file = pss->pss_channel;
     int len = 0;
 
     if (buffer_is_zero(p, TARGET_PAGE_SIZE)) {
@@ -1162,10 +1163,10 @@ static int save_zero_page_to_file(PageSearchStatus *pss, QEMUFile *file,
  * @block: block that contains the page we want to send
  * @offset: offset inside the block for the page
  */
-static int save_zero_page(PageSearchStatus *pss, QEMUFile *f, RAMBlock *block,
+static int save_zero_page(PageSearchStatus *pss, RAMBlock *block,
                           ram_addr_t offset)
 {
-    int len = save_zero_page_to_file(pss, f, block, offset);
+    int len = save_zero_page_to_file(pss, block, offset);
 
     if (len) {
         stat64_add(&mig_stats.zero_pages, 1);
@@ -2140,7 +2141,7 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return 1;
     }
 
-    res = save_zero_page(pss, pss->pss_channel, block, offset);
+    res = save_zero_page(pss, block, offset);
     if (res > 0) {
         /* Must let xbzrle know, otherwise a previous (now 0'd) cached
          * page would be stale
Signature · 1 line
2.35.3
#3 lore raw

PATCH v2 3/4 migration/ram: Move xbzrle zero page handling into save_zero_page

In reply to Fabiano Rosas

It makes a bit more sense to have the zero page handling of xbzrle right where we save the zero page.

Also invert the exit condition to remove one level of indentation which makes the next patch easier to grasp.

---
 migration/ram.c | 35 ++++++++++++++++++++---------------
 1 file changed, 20 insertions(+), 15 deletions(-)
migration/ram.c +20 -15
index 761f43dc34..82ff53beec 100644
@@ -1159,21 +1159,34 @@ static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
  *
  * Returns the number of pages written.
  *
+ * @rs: current RAM state
  * @pss: current PSS channel
  * @block: block that contains the page we want to send
  * @offset: offset inside the block for the page
  */
-static int save_zero_page(PageSearchStatus *pss, RAMBlock *block,
+static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
                           ram_addr_t offset)
 {
     int len = save_zero_page_to_file(pss, block, offset);
 
-    if (len) {
-        stat64_add(&mig_stats.zero_pages, 1);
-        ram_transferred_add(len);
-        return 1;
+    if (!len) {
+        return -1;
     }
-    return -1;
+
+    stat64_add(&mig_stats.zero_pages, 1);
+    ram_transferred_add(len);
+
+    /*
+     * Must let xbzrle know, otherwise a previous (now 0'd) cached
+     * page would be stale.
+     */
+    if (rs->xbzrle_started) {
+        XBZRLE_cache_lock();
+        xbzrle_cache_zero_page(block->offset + offset);
+        XBZRLE_cache_unlock();
+    }
+
+    return 1;
 }
 
 /*
@@ -2141,16 +2154,8 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return 1;
     }
 
-    res = save_zero_page(pss, block, offset);
+    res = save_zero_page(rs, pss, block, offset);
     if (res > 0) {
-        /* Must let xbzrle know, otherwise a previous (now 0'd) cached
-         * page would be stale
-         */
-        if (rs->xbzrle_started) {
-            XBZRLE_cache_lock();
-            xbzrle_cache_zero_page(block->offset + offset);
-            XBZRLE_cache_unlock();
-        }
         return res;
     }
 
Signature · 1 line
2.35.3
#4 lore raw

PATCH v2 4/4 migration/ram: Merge save_zero_page functions

In reply to Fabiano Rosas

We don't need to do this in two pieces. One single function makes it easier to grasp, specially since it removes the indirection on the return value handling.

---
 migration/ram.c | 46 +++++++++++++---------------------------------
 1 file changed, 13 insertions(+), 33 deletions(-)
migration/ram.c +13 -33
index 82ff53beec..13935ead1c 100644
@@ -1128,32 +1128,6 @@ void ram_release_page(const char *rbname, uint64_t offset)
     ram_discard_range(rbname, offset, TARGET_PAGE_SIZE);
 }
 
-/**
- * save_zero_page_to_file: send the zero page to the file
- *
- * Returns the size of data written to the file, 0 means the page is not
- * a zero page
- *
- * @pss: current PSS channel
- * @block: block that contains the page we want to send
- * @offset: offset inside the block for the page
- */
-static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
-                                  ram_addr_t offset)
-{
-    uint8_t *p = block->host + offset;
-    QEMUFile *file = pss->pss_channel;
-    int len = 0;
-
-    if (buffer_is_zero(p, TARGET_PAGE_SIZE)) {
-        len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
-        qemu_put_byte(file, 0);
-        len += 1;
-        ram_release_page(block->idstr, offset);
-    }
-    return len;
-}
-
 /**
  * save_zero_page: send the zero page to the stream
  *
@@ -1167,12 +1141,19 @@ static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
 static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
                           ram_addr_t offset)
 {
-    int len = save_zero_page_to_file(pss, block, offset);
+    uint8_t *p = block->host + offset;
+    QEMUFile *file = pss->pss_channel;
+    int len = 0;
 
-    if (!len) {
-        return -1;
+    if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) {
+        return 0;
     }
 
+    len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
+    qemu_put_byte(file, 0);
+    len += 1;
+    ram_release_page(block->idstr, offset);
+
     stat64_add(&mig_stats.zero_pages, 1);
     ram_transferred_add(len);
 
@@ -1186,7 +1167,7 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
         XBZRLE_cache_unlock();
     }
 
-    return 1;
+    return len;
 }
 
 /*
@@ -2154,9 +2135,8 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return 1;
     }
 
-    res = save_zero_page(rs, pss, block, offset);
-    if (res > 0) {
-        return res;
+    if (save_zero_page(rs, pss, block, offset)) {
+        return 1;
     }
 
     /*
Signature · 1 line
2.35.3
Reviewed-by on v2 4/4
#5 lore raw

Re: [PATCH v2 4/4] migration/ram: Merge save_zero_page functions

Quoted 5 lines · Fabiano Rosas

We don't need to do this in two pieces. One single function makes it easier to grasp, specially since it removes the indirection on the return value handling.

Signature · 1 line
Peter Xu
#6 lore raw

Re: [PATCH v2 4/4] migration/ram: Merge save_zero_page functions

In reply to Fabiano Rosas

Hello,

this patch would still need a review,

and is needed as a precondition for further work to improve dramatically the performance of virsh save, virsh restore when migrating to disk, since Peter Xu already reviewed PATCH 1, 2, 3, maybe it makes sense to look at 4 too?

Thanks,

Claudio

Quoted 90 lines · Fabiano Rosas

We don't need to do this in two pieces. One single function makes it easier to grasp, specially since it removes the indirection on the return value handling.

---
 migration/ram.c | 46 +++++++++++++---------------------------------
 1 file changed, 13 insertions(+), 33 deletions(-)
migration/ram.c +13 -33
index 82ff53beec..13935ead1c 100644
@@ -1128,32 +1128,6 @@ void ram_release_page(const char *rbname, uint64_t offset)
     ram_discard_range(rbname, offset, TARGET_PAGE_SIZE);
 }
 
-/**
- * save_zero_page_to_file: send the zero page to the file
- *
- * Returns the size of data written to the file, 0 means the page is not
- * a zero page
- *
- * @pss: current PSS channel
- * @block: block that contains the page we want to send
- * @offset: offset inside the block for the page
- */
-static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
-                                  ram_addr_t offset)
-{
-    uint8_t *p = block->host + offset;
-    QEMUFile *file = pss->pss_channel;
-    int len = 0;
-
-    if (buffer_is_zero(p, TARGET_PAGE_SIZE)) {
-        len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
-        qemu_put_byte(file, 0);
-        len += 1;
-        ram_release_page(block->idstr, offset);
-    }
-    return len;
-}
-
 /**
  * save_zero_page: send the zero page to the stream
  *
@@ -1167,12 +1141,19 @@ static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
 static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
                           ram_addr_t offset)
 {
-    int len = save_zero_page_to_file(pss, block, offset);
+    uint8_t *p = block->host + offset;
+    QEMUFile *file = pss->pss_channel;
+    int len = 0;
 
-    if (!len) {
-        return -1;
+    if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) {
+        return 0;
     }
 
+    len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
+    qemu_put_byte(file, 0);
+    len += 1;
+    ram_release_page(block->idstr, offset);
+
     stat64_add(&mig_stats.zero_pages, 1);
     ram_transferred_add(len);
 
@@ -1186,7 +1167,7 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
         XBZRLE_cache_unlock();
     }
 
-    return 1;
+    return len;
 }
 
 /*
@@ -2154,9 +2135,8 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return 1;
     }
 
-    res = save_zero_page(rs, pss, block, offset);
-    if (res > 0) {
-        return res;
+    if (save_zero_page(rs, pss, block, offset)) {
+        return 1;
     }
 
     /*
#7 lore raw

Re: [PATCH v2 4/4] migration/ram: Merge save_zero_page functions

Apologies, already reviewed, will ping for the merge of the series momentarily,

Claudio

Quoted 103 lines · Claudio Fontana

Hello,

this patch would still need a review,

and is needed as a precondition for further work to improve dramatically the performance of virsh save, virsh restore when migrating to disk, since Peter Xu already reviewed PATCH 1, 2, 3, maybe it makes sense to look at 4 too?

Thanks,

Claudio

Quoted 90 lines · Fabiano Rosas

We don't need to do this in two pieces. One single function makes it easier to grasp, specially since it removes the indirection on the return value handling.

---
 migration/ram.c | 46 +++++++++++++---------------------------------
 1 file changed, 13 insertions(+), 33 deletions(-)
migration/ram.c +13 -33
index 82ff53beec..13935ead1c 100644
@@ -1128,32 +1128,6 @@ void ram_release_page(const char *rbname, uint64_t offset)
     ram_discard_range(rbname, offset, TARGET_PAGE_SIZE);
 }
 
-/**
- * save_zero_page_to_file: send the zero page to the file
- *
- * Returns the size of data written to the file, 0 means the page is not
- * a zero page
- *
- * @pss: current PSS channel
- * @block: block that contains the page we want to send
- * @offset: offset inside the block for the page
- */
-static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
-                                  ram_addr_t offset)
-{
-    uint8_t *p = block->host + offset;
-    QEMUFile *file = pss->pss_channel;
-    int len = 0;
-
-    if (buffer_is_zero(p, TARGET_PAGE_SIZE)) {
-        len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
-        qemu_put_byte(file, 0);
-        len += 1;
-        ram_release_page(block->idstr, offset);
-    }
-    return len;
-}
-
 /**
  * save_zero_page: send the zero page to the stream
  *
@@ -1167,12 +1141,19 @@ static int save_zero_page_to_file(PageSearchStatus *pss, RAMBlock *block,
 static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
                           ram_addr_t offset)
 {
-    int len = save_zero_page_to_file(pss, block, offset);
+    uint8_t *p = block->host + offset;
+    QEMUFile *file = pss->pss_channel;
+    int len = 0;
 
-    if (!len) {
-        return -1;
+    if (!buffer_is_zero(p, TARGET_PAGE_SIZE)) {
+        return 0;
     }
 
+    len += save_page_header(pss, file, block, offset | RAM_SAVE_FLAG_ZERO);
+    qemu_put_byte(file, 0);
+    len += 1;
+    ram_release_page(block->idstr, offset);
+
     stat64_add(&mig_stats.zero_pages, 1);
     ram_transferred_add(len);
 
@@ -1186,7 +1167,7 @@ static int save_zero_page(RAMState *rs, PageSearchStatus *pss, RAMBlock *block,
         XBZRLE_cache_unlock();
     }
 
-    return 1;
+    return len;
 }
 
 /*
@@ -2154,9 +2135,8 @@ static int ram_save_target_page_legacy(RAMState *rs, PageSearchStatus *pss)
         return 1;
     }
 
-    res = save_zero_page(rs, pss, block, offset);
-    if (res > 0) {
-        return res;
+    if (save_zero_page(rs, pss, block, offset)) {
+        return 1;
     }
 
     /*
#8 lore raw
In reply to Fabiano Rosas

Hello,

this series is all reviewed,

and is needed as a precondition for further work to improve dramatically the performance of virsh save, virsh restore when migrating to disk, can it be merged?

Thanks,

Claudio

Quoted 28 lines · Fabiano Rosas

For v2 I fixed patch 3 which had a hunk belonging to patch 5.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/969706915

v1:
https://lore.kernel.org/r/[email protected]

Hi,

This is another small series that I extracted from my fixed-ram series and that could be already considered for merging.

This is just code movement, no functional change. The objective is to consolidate the zero page handling in the same routine that saves the page header and does accounting. Then in the future I'll be able to just return early because fixed-ram ignores zero pages.

CI run: https://gitlab.com/farosas/qemu/-/pipelines/968300062

Fabiano Rosas (4):
migration/ram: Remove RAMState from xbzrle_cache_zero_page migration/ram: Stop passing QEMUFile around in save_zero_page migration/ram: Move xbzrle zero page handling into save_zero_page migration/ram: Merge save_zero_page functions

 migration/ram.c | 75 ++++++++++++++++++++-----------------------------
 1 file changed, 30 insertions(+), 45 deletions(-)