Skip to contentSkip to conversation

system/physmem: Assert migration invariants

PATCH In review Akihiko Odaki 3 participants 3 messages Jun 4

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 system/physmem: Assert migration invariants

The migration stream assumes that the set of migratable RAMBlocks does not change while migration is running. Assert that RAMBlocks are not made migratable or non-migratable during migration, and that migratable RAMBlocks are not freed during migration.

Non-migratable RAMBlocks may still be allocated or freed during migration; for example, QMP object-add of memory-backend-ram creates a non-migratable RAMBlock without exposing it to the guest.

---
 system/physmem.c | 5 +++++
 1 file changed, 5 insertions(+)
system/physmem.c +5 -0
index 7bcbf8757361..42a39141645e 100644
@@ -39,6 +39,7 @@
 #include "hw/core/qdev.h"
 #include "hw/core/qdev-properties.h"
 #include "hw/core/boards.h"
+#include "migration/misc.h"
 #include "system/xen.h"
 #include "system/kvm.h"
 #include "system/tcg.h"
@@ -1911,11 +1912,13 @@ bool qemu_ram_is_migratable(const RAMBlock *rb)
 
 void qemu_ram_set_migratable(RAMBlock *rb)
 {
+    assert(!migration_is_running());
     rb->flags |= RAM_MIGRATABLE;
 }
 
 void qemu_ram_unset_migratable(RAMBlock *rb)
 {
+    assert(!migration_is_running());
     rb->flags &= ~RAM_MIGRATABLE;
 }
 
@@ -2599,6 +2602,8 @@ void qemu_ram_free(RAMBlock *block)
         return;
     }
 
+    assert(!migration_is_running() || !qemu_ram_is_migratable(block));
+
     if (block->host) {
         ram_block_notify_remove(block->host, block->used_length,
                                 block->max_length);

---
base-commit: 2db91528542672cf0db78b3f2cc0e22b36302b38
change-id: 20260602-migration-15128dbe4036

Best regards,

Signature · 1 line
Akihiko Odaki <[email protected]>
Reviewed-by on v1 1/1
#1 lore raw
Quoted 5 lines · Akihiko Odaki

The migration stream assumes that the set of migratable RAMBlocks does not change while migration is running. Assert that RAMBlocks are not made migratable or non-migratable during migration, and that migratable RAMBlocks are not freed during migration.

Quoted 8 lines

Non-migratable RAMBlocks may still be allocated or freed during migration; for example, QMP object-add of memory-backend-ram creates a non-migratable RAMBlock without exposing it to the guest.

---
  system/physmem.c | 5 +++++
  1 file changed, 5 insertions(+)
#2 lore raw
In reply to Akihiko Odaki
Quoted 41 lines · Akihiko Odaki

The migration stream assumes that the set of migratable RAMBlocks does not change while migration is running. Assert that RAMBlocks are not made migratable or non-migratable during migration, and that migratable RAMBlocks are not freed during migration.

Non-migratable RAMBlocks may still be allocated or freed during migration; for example, QMP object-add of memory-backend-ram creates a non-migratable RAMBlock without exposing it to the guest.

---
 system/physmem.c | 5 +++++
 1 file changed, 5 insertions(+)
system/physmem.c +3 -0
index 7bcbf8757361..42a39141645e 100644
@@ -39,6 +39,7 @@
 #include "hw/core/qdev.h"
 #include "hw/core/qdev-properties.h"
 #include "hw/core/boards.h"
+#include "migration/misc.h"
 #include "system/xen.h"
 #include "system/kvm.h"
 #include "system/tcg.h"
@@ -1911,11 +1912,13 @@ bool qemu_ram_is_migratable(const RAMBlock *rb)
 
 void qemu_ram_set_migratable(RAMBlock *rb)
 {
+    assert(!migration_is_running());
     rb->flags |= RAM_MIGRATABLE;
 }
 
 void qemu_ram_unset_migratable(RAMBlock *rb)
 {
+    assert(!migration_is_running());
     rb->flags &= ~RAM_MIGRATABLE;
 }
 

system/physmem.c void qemu_ram_free(RAMBlock *block) · line 2606

Quoted 5 lines
system/physmem.c +2 -0
@@ -2599,6 +2602,8 @@ void qemu_ram_free(RAMBlock *block)
         return;
     }
 
+    assert(!migration_is_running() || !qemu_ram_is_migratable(block));
+

The intention makes sense, however these asserts are slightly risky to me on a few things..

Firstly, migration_is_running() covers a wide range, the tricky part is when CANCELLING and FAILING: when QEMU shutdowns with qemu_cleanup(), it only kicks the migration out with migration_cancel(), it won't wait for migration to finish completely. I didn't verify the possibility, but IIUC it means during QEMU tearing down we may have ramblock freed while migration_is_running() returns true.

Above will also include possibility of changing migration migratable. In this specific case, it's about marking ramblock to be unmigratable during tearing down with a call to vmstate_unregister_ram().

The other thing is, strictly speaking the migration ABI is not ramblock pointers, but a combination of ramblock ID+page_size+addr+... info, per ram_save_setup(). I don't know if there's legal user (I'm having some form of ROMs in mind..) that may silently replace a ramblock pointer but kept everything else intact..

Thanks,

Quoted 12 lines
system/physmem.c +0 -1
     if (block->host) {
         ram_block_notify_remove(block->host, block->used_length,
                                 block->max_length);
 
---

base-commit: 2db91528542672cf0db78b3f2cc0e22b36302b38
change-id: 20260602-migration-15128dbe4036

Best regards,

Signature · 1 line
Akihiko Odaki <[email protected]>
Signature · 1 line
Peter Xu