Skip to contentSkip to conversation

migration/postcopy: ensure preempt channel is ready before loading states

PATCH In review Wei Wang 3 participants 5 messages Apr 5, 2024 – Apr 7, 2024

lore
Share & tools
lore

Wei Wang

author
#0 lore raw

PATCH v2 1/1 migration/postcopy: ensure preempt channel is ready before loading states

Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event hasn't been dispatched.

---
 migration/savevm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
migration/savevm.c +17 -0
index 388d7af7cd..63f9991a8a 100644
@@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 
     QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
 
+    /*
+     * Before loading the guest states, ensure that the preempt channel has
+     * been ready to use, as some of the states (e.g. via virtio_load) might
+     * trigger page faults that will be handled through the preempt channel.
+     * So yield to the main thread in the case that the channel create event
+     * hasn't been dispatched.
+     */
+    do {
+        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
+            mis->postcopy_qemufile_dst) {
+            break;
+        }
+
+        aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self());
+        qemu_coroutine_yield();
+    } while (1);
+
     ret = qemu_loadvm_state_main(packf, mis);
     trace_loadvm_handle_cmd_packaged_main(ret);
     qemu_fclose(packf);
Signature · 1 line
2.27.0
#1 lore raw
Quoted 46 lines · Wei W

Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event hasn't been dispatched.

[email protected]/T/

---
 migration/savevm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
migration/savevm.c index renamed from migration/savevm.c +0 -0

388d7af7cd..63f9991a8a 100644

migration/savevm.c +17 -0
@@ -2342,6 +2342,23 @@ static int
 loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 
     QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
 
+    /*
+     * Before loading the guest states, ensure that the preempt channel has
+     * been ready to use, as some of the states (e.g. via virtio_load) might
+     * trigger page faults that will be handled through the preempt channel.
+     * So yield to the main thread in the case that the channel create event
+     * hasn't been dispatched.
+     */
+    do {
+        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
+            mis->postcopy_qemufile_dst) {
+            break;
+        }
+
+        aio_co_schedule(qemu_get_current_aio_context(),
 qemu_coroutine_self());
+        qemu_coroutine_yield();
+    } while (1);
+
     ret = qemu_loadvm_state_main(packf, mis);
     trace_loadvm_handle_cmd_packaged_main(ret);
     qemu_fclose(packf);
Quoted 2 lines

--
2.27.0

Main change from v1 is the drop of the wait on sem. It's still patched to loadvm_handle_cmd_packaged, as the sem issue (possible twice wait) isn't there now.

#2 lore raw
In reply to Wei Wang
Quoted 9 lines · Wei Wang

Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event hasn't been dispatched.

The current version doesn't have any of my credits. :) Thanks, but I'll just drop it to reflect reality, so we keep the credit to the right ones.

Quoted 11 lines
---
 migration/savevm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
migration/savevm.c +0 -0
index 388d7af7cd..63f9991a8a 100644

migration/savevm.c static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis) · line 2350

Quoted 9 lines
migration/savevm.c +6 -0
@@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 
     QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
 
+    /*
+     * Before loading the guest states, ensure that the preempt channel has
+     * been ready to use, as some of the states (e.g. via virtio_load) might
+     * trigger page faults that will be handled through the preempt channel.
+     * So yield to the main thread in the case that the channel create event
+     * hasn't been dispatched.

I'll squash below into it. If any of you disagree please shoot: valid until this Sunday.

+     * TODO: if we can move migration loadvm out of main thread, then we
+     * won't block main thread from polling the accept() fds.  We can drop
+     * this as a whole when that is done.

Huge thanks to both!

Quoted 17 lines
migration/savevm.c +11 -0
+     */
+    do {
+        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
+            mis->postcopy_qemufile_dst) {
+            break;
+        }
+
+        aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self());
+        qemu_coroutine_yield();
+    } while (1);
+
     ret = qemu_loadvm_state_main(packf, mis);
     trace_loadvm_handle_cmd_packaged_main(ret);
     qemu_fclose(packf);
Signature · 1 line
2.27.0
Signature · 1 line
Peter Xu
#3 lore raw
Quoted 42 lines · Peter Xu
Quoted 11 lines · Wei Wang

Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event hasn't been dispatched.

Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel
.com/T/

The current version doesn't have any of my credits. :) Thanks, but I'll just drop it to reflect reality, so we keep the credit to the right ones.

Quoted 11 lines
---
 migration/savevm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
migration/savevm.c index renamed from migration/savevm.c +0 -0

388d7af7cd..63f9991a8a 100644

migration/savevm.c static int · line 2351

Quoted 10 lines
migration/savevm.c +6 -0
@@ -2342,6 +2342,23 @@ static int
 loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 
     QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
 
+    /*
+     * Before loading the guest states, ensure that the preempt channel has
+     * been ready to use, as some of the states (e.g. via virtio_load) might
+     * trigger page faults that will be handled through the preempt channel.
+     * So yield to the main thread in the case that the channel create event
+     * hasn't been dispatched.

I'll squash below into it. If any of you disagree please shoot: valid until this Sunday.

Review

Quoted 3 lines
(quoted diff) +3 -0
+     * TODO: if we can move migration loadvm out of main thread, then we
+     * won't block main thread from polling the accept() fds.  We can drop
+     * this as a whole when that is done.

Looks good to me.

#4 lore raw
In reply to Peter Xu
Quoted 27 lines · Wei Wang
Quoted 9 lines

Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event hasn't been dispatched.

The current version doesn't have any of my credits. :) Thanks, but I'll just drop it to reflect reality, so we keep the credit to the right ones.

Quoted 14 lines
---
 migration/savevm.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
migration/savevm.c +0 -0
index 388d7af7cd..63f9991a8a 100644
@@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)
 
     QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc));
 
Quoted 9 lines

migration/savevm.c static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis) · line 2350

Quoted 6 lines
migration/savevm.c +6 -0
+    /*
+     * Before loading the guest states, ensure that the preempt channel has
+     * been ready to use, as some of the states (e.g. via virtio_load) might
+     * trigger page faults that will be handled through the preempt channel.
+     * So yield to the main thread in the case that the channel create event
+     * hasn't been dispatched.

I'll squash below into it. If any of you disagree please shoot: valid until this Sunday.

I'm OK with it.

Quoted 6 lines
(quoted diff) +3 -0
+     * TODO: if we can move migration loadvm out of main thread, then we
+     * won't block main thread from polling the accept() fds.  We can drop
+     * this as a whole when that is done.

Huge thanks to both!

Thank you Peter :)

Quoted 19 lines
Quoted 18 lines
migration/savevm.c +11 -0
+     */
+    do {
+        if (!migrate_postcopy_preempt() || !qemu_in_coroutine() ||
+            mis->postcopy_qemufile_dst) {
+            break;
+        }
+
+        aio_co_schedule(qemu_get_current_aio_context(), qemu_coroutine_self());
+        qemu_coroutine_yield();
+    } while (1);
+
     ret = qemu_loadvm_state_main(packf, mis);
     trace_loadvm_handle_cmd_packaged_main(ret);
     qemu_fclose(packf);
Signature · 1 line
2.27.0