migration/postcopy: ensure preempt channel is ready before loading states
PATCH 1/1 migration/postcopy: ensure preempt channel is ready before loading states
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event has been dispatched.
--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
migration/savevm.c +17 -0
@@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)QEMUFile *packf =qemu_file_new_input (QIO_CHANNEL (bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has +* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel. +* So yield to the main thread in the case that the channel create event +* has been dispatched. +*/ +do { +if (!migrate_postcopy_preempt () || !qemu_in_coroutine () || + mis->postcopy_qemufile_dst) { +break ; + } + +aio_co_schedule (qemu_get_current_aio_context (),qemu_coroutine_self ()); +qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done,1 )); + ret =qemu_loadvm_state_main (packf, mis);trace_loadvm_handle_cmd_packaged_main (ret);qemu_fclose (packf);
Signature · 1 line
2.27.0
Quoted 29 lines · Wei Wang
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event has been dispatched.Originally-by Lei Wang <[email protected]>Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c +6
-0@@ -2342,6 +2342,23 @@ static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis)QEMUFile *packf =qemu_file_new_input (QIO_CHANNEL (bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has +* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel. +* So yield to the main thread in the case that the channel create event +* has been dispatched.
migration/savevm.c static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis) · line 2360
Quoted 10 lines
migration/savevm.c +10
-0+*/ +do { +if (!migrate_postcopy_preempt () || !qemu_in_coroutine () || + mis->postcopy_qemufile_dst) { +break ; + } + +aio_co_schedule (qemu_get_current_aio_context (),qemu_coroutine_self ()); +qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done,1 ));
I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of qemu_sem_timedwait() at some point later..
Besides, this patch kept the sem_wait() in postcopy_preempt_thread() so it will wait() on this sem again. If this qemu_sem_timedwait() accidentally consumed the sem count then I think the other thread can hang forever?
That's why I put the wait before creation of the preempt thread (in postcopy_ram_incoming_setup()), as we can only consume the sem once, so we must prepare the qemufile when the thread is created.
Thanks,
Quoted 7 lines
migration/savevm.c +1
-0+ ret =qemu_loadvm_state_main (packf, mis);trace_loadvm_handle_cmd_packaged_main (ret);qemu_fclose (packf);Signature · 1 line
2.27.0
Signature · 1 line
Peter Xu
Quoted 42 lines · Peter Xu
Quoted 30 lines · Wei Wang
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event has been dispatched.Originally-by Lei Wang <[email protected]>Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel
.com/T/Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c index renamed from migration/savevm.c +0
-0
388d7af7cd..fbc9f2bdd4100644migration/savevm.c +4
-0@@ -2342,6 +2342,23 @@ static intloadvm_handle_cmd_packaged (MigrationIncomingState *mis) QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has +* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel. migration/savevm.c static int · line 2359
Quoted 10 lines
migration/savevm.c +10
-0+* So yield to the main thread in the case that the channel create event +* has been dispatched. +*/ + do { + if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || + mis->postcopy_qemufile_dst) { + break; + } + + aio_co_schedule(qemu_get_current_aio_context(),
qemu_coroutine_self());
Quoted 6 lines
migration/savevm.c static int · line 2362
Quoted 3 lines
migration/savevm.c +3
-0+qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done, +1 ));I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of
qemu_sem_timedwait()at some point later..
No. qemu_sem_timedwait returns false when timeout, which means sem isn’t posted yet. So it needs to go back to the loop. (the patch was tested)
Quoted 4 lines
Besides, this patch kept the
sem_wait()inpostcopy_preempt_thread()so it willwait()on this sem again. If thisqemu_sem_timedwait()accidentally consumed the sem count then I think the other thread can hang forever?
I can get the issue you mentioned, and seems better to be placed before the creation of the preempt thread. Then we probably don’t need to wait_sem in the preempt thread, as the channel is guaranteed to be ready when it runs?
Update will be:
migration/postcopy-ram.c +9 -5
@@ -1254,6 +1254,15 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis) }if (migrate_postcopy_preempt ()) { +do { +if (!migrate_postcopy_preempt () || !qemu_in_coroutine () || + mis->postcopy_qemufile_dst) { +break ; + } +aio_co_schedule (qemu_get_current_aio_context (),qemu_coroutine_self ()); +qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done,1 )); +/* * This thread needs to be created after the temp pages because * it'll fetch RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately.
@@ -1743,12 +1752,6 @@ void *postcopy_preempt_thread(void *opaque)qemu_sem_post (&mis->thread_sync_sem); -/* -* The preempt channel is established in asynchronous way. Wait -* for its completion. -*/ -qemu_sem_wait (&mis->postcopy_qemufile_dst_done);
Quoted 46 lines · Peter Xu
Quoted 46 lines · Wei Wang
Quoted 30 lines
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event has been dispatched.Originally-by Lei Wang <[email protected]>Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel
.com/T/Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c index renamed from migration/savevm.c +0
-0
388d7af7cd..fbc9f2bdd4100644migration/savevm.c +4
-0@@ -2342,6 +2342,23 @@ static intloadvm_handle_cmd_packaged (MigrationIncomingState *mis) QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has +* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel. migration/savevm.c static int · line 2359
Quoted 10 lines
migration/savevm.c +10
-0+* So yield to the main thread in the case that the channel create event +* has been dispatched. +*/ + do { + if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || + mis->postcopy_qemufile_dst) { + break; + } + + aio_co_schedule(qemu_get_current_aio_context(),
qemu_coroutine_self());Quoted 3 lines
migration/savevm.c +3
-0+qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done, +1 ));
Quoted 5 lines
Quoted 2 lines
I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of
qemu_sem_timedwait()at some point later..No.
qemu_sem_timedwaitreturns false when timeout, which means sem isn’t posted yet. So it needs to go back to the loop. (the patch was tested)
When timeout, qemu_sem_timedwait() will return -1. I think the patch test passed may because you will always have at least one yield (the first yield in the do ...while ...) when loadvm_handle_cmd_packaged()?
Quoted 48 lines
Quoted 5 lines
Besides, this patch kept the
sem_wait()inpostcopy_preempt_thread()so it willwait()on this sem again. If thisqemu_sem_timedwait()accidentally consumed the sem count then I think the other thread can hang forever?I can get the issue you mentioned, and seems better to be placed before the creation of the preempt thread. Then we probably don’t need to
wait_semin the preempt thread, as the channel is guaranteed to be ready when it runs?Update will be:
migration/postcopy-ram.c +9
-5@@ -1254,6 +1254,15 @@ int postcopy_ram_incoming_setup(MigrationIncomingState *mis) }if (migrate_postcopy_preempt ()) { +do { +if (!migrate_postcopy_preempt () || !qemu_in_coroutine () || + mis->postcopy_qemufile_dst) { +break ; + } +aio_co_schedule (qemu_get_current_aio_context (),qemu_coroutine_self ()); +qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done,1 )); +/* * This thread needs to be created after the temp pages because * it'll fetch RAM_CHANNEL_POSTCOPY PostcopyTmpPage immediately. @@ -1743,12 +1752,6 @@ void *postcopy_preempt_thread(void *opaque)qemu_sem_post (&mis->thread_sync_sem); -/* -* The preempt channel is established in asynchronous way. Wait -* for its completion. -*/ -qemu_sem_wait (&mis->postcopy_qemufile_dst_done);
Quoted 51 lines · Lei
Quoted 49 lines · Peter Xu
Quoted 48 lines · Wei Wang
Quoted 30 lines
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through the preempt channel. So yield to the main thread in the case that the channel create event has been dispatched.Originally-by Lei Wang <[email protected]>Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@intel
.com/T/Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c index renamed from migration/savevm.c +0
-0
388d7af7cd..fbc9f2bdd4100644migration/savevm.c +4
-0@@ -2342,6 +2342,23 @@ static intloadvm_handle_cmd_packaged (MigrationIncomingState *mis) QEMUFile *packf = qemu_file_new_input(QIO_CHANNEL(bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has +* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel. migration/savevm.c static int · line 2359
Quoted 10 lines
migration/savevm.c +10
-0+* So yield to the main thread in the case that the channel create event +* has been dispatched. +*/ + do { + if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || + mis->postcopy_qemufile_dst) { + break; + } + + aio_co_schedule(qemu_get_current_aio_context(),
qemu_coroutine_self());migration/savevm.c static int · line 2362
Quoted 3 lines
migration/savevm.c +3
-0+qemu_coroutine_yield (); + }while (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done, +1 ));I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of
qemu_sem_timedwait()at some point later..
Quoted 6 lines
Quoted 2 lines
No.
qemu_sem_timedwaitreturns false when timeout, which means sem isn’t posted yet. So it needs to go back to the loop. (the patch was tested)When timeout,
qemu_sem_timedwait()will return -1. I think the patch test passed may because you will always have at least one yield (the first yield in the do ...while ...) whenloadvm_handle_cmd_packaged()?
My guess is that here the kick will work and qemu_sem_timedwait() later will ETIMEOUT -> qemu_sem_timedwait() returns -1, then the loop just broke. That aio schedule should make sure anyway that the file is ready; the preempt thread must run before this to not hang that thread.
I think it more kind of justifies that the retval needs to be properly defined. :( It's confusion is on top of when I know libpthread returns positive error codes.
Thans,
Signature · 1 line
Peter Xu
Quoted 62 lines · Peter Xu
Quoted 6 lines · Lei
Quoted 4 lines · Peter Xu
Quoted 4 lines · Wei Wang
Quoted 3 lines
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through thepreempt channel.
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
So yield to the main thread in the case that the channel create event has been dispatched.
Originally-by Lei Wang <[email protected]>Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@i
ntel
.com/T/Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c index renamed from migration/savevm.c +0
-0
388d7af7cd..fbc9f2bdd4100644migration/savevm.c +0
-0@@ -2342,6 +2342,23 @@ static intloadvm_handle_cmd_packaged (MigrationIncomingState *mis)Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
migration/savevm.c static int · line 2347
Quoted 4 lines
migration/savevm.c +2
-0QEMUFile *packf =qemu_file_new_input (QIO_CHANNEL (bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
migration/savevm.c static int · line 2349
Quoted 2 lines
migration/savevm.c +2
-0+* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel.
Quoted 1 line
Quoted 1 line
Quoted 1 line
migration/savevm.c static int · line 2350
Quoted 1 line
migration/savevm.c +1
-0+* So yield to the main thread in the case that the channel create event
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
migration/savevm.c static int · line 2359
Quoted 9 lines
migration/savevm.c +9
-0+* has been dispatched. +*/ + do { + if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || + mis->postcopy_qemufile_dst) { + break; + } + + aio_co_schedule(qemu_get_current_aio_context(),
qemu_coroutine_self());Quoted 4 lines
migration/savevm.c +4
-0+qemu_coroutine_yield (); + }while + (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done, +1 ));Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of
qemu_sem_timedwait()at some point later..No.
qemu_sem_timedwaitreturns false when timeout, which means semisn’t posted yet.
Quoted 2 lines
Quoted 1 line
So it needs to go back to the loop. (the patch was tested)
Quoted 8 lines
Quoted 3 lines
When timeout,
qemu_sem_timedwait()will return -1. I think the patch test passed may because you will always have at least one yield (the first yield in the do ...while ...) whenloadvm_handle_cmd_packaged()?My guess is that here the kick will work and
qemu_sem_timedwait()later will ETIMEOUT ->qemu_sem_timedwait()returns -1, then the loop just broke. That aio schedule should make sure anyway that the file is ready; the preempt thread must run before this to not hang that thread.
Yes, misread of the return value. It still worked because the loop broke at the "if (mis->postcopy_qemufile_dst)" check.
Even below will work:
do {
if (mis->postcopy_qemufile_dst) {
break;
}...
} while (1);
I still don’t see the value of using postcopy_qemufile_dst_done sem in the code though It simplify blocks the main thread from creating the preempt channel for 1ms (regardless of the possibility about whether the sem has been be posted or not. We add it for the case it is not posted and need to go back to the loop).
Quoted 80 lines · Wei W
Quoted 62 lines · Peter Xu
Quoted 6 lines · Lei
Quoted 4 lines · Peter Xu
Quoted 4 lines · Wei Wang
Quoted 3 lines
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through thepreempt channel.
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
So yield to the main thread in the case that the channel create event has been dispatched.
Originally-by Lei Wang <[email protected]>Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced249@i
ntel
.com/T/Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c index renamed from migration/savevm.c +0
-0
388d7af7cd..fbc9f2bdd4100644migration/savevm.c +0
-0@@ -2342,6 +2342,23 @@ static intloadvm_handle_cmd_packaged (MigrationIncomingState *mis)Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
migration/savevm.c static int · line 2347
Quoted 4 lines
migration/savevm.c +2
-0QEMUFile *packf =qemu_file_new_input (QIO_CHANNEL (bioc)); +/* +* Before loading the guest states, ensure that the preempt channel has
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
migration/savevm.c static int · line 2349
Quoted 2 lines
migration/savevm.c +2
-0+* been ready to use, as some of the states (e.g. via virtio_load) might +* trigger page faults that will be handled through the preempt channel.
Quoted 1 line
Quoted 1 line
Quoted 1 line
migration/savevm.c static int · line 2350
Quoted 1 line
migration/savevm.c +1
-0+* So yield to the main thread in the case that the channel create event
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
migration/savevm.c static int · line 2359
Quoted 9 lines
migration/savevm.c +9
-0+* has been dispatched. +*/ + do { + if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || + mis->postcopy_qemufile_dst) { + break; + } + + aio_co_schedule(qemu_get_current_aio_context(),
qemu_coroutine_self());Quoted 4 lines
migration/savevm.c +4
-0+qemu_coroutine_yield (); + }while + (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done, +1 ));Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of
qemu_sem_timedwait()at some point later..No.
qemu_sem_timedwaitreturns false when timeout, which means semisn’t posted yet.
Quoted 2 lines
Quoted 1 line
So it needs to go back to the loop. (the patch was tested)
Quoted 8 lines
Quoted 3 lines
When timeout,
qemu_sem_timedwait()will return -1. I think the patch test passed may because you will always have at least one yield (the first yield in the do ...while ...) whenloadvm_handle_cmd_packaged()?My guess is that here the kick will work and
qemu_sem_timedwait()later will ETIMEOUT ->qemu_sem_timedwait()returns -1, then the loop just broke. That aio schedule should make sure anyway that the file is ready; the preempt thread must run before this to not hang that thread.Yes, misread of the return value. It still worked because the loop broke at the "if (
mis->postcopy_qemufile_dst)" check.Even below will work:
do { if (mis->postcopy_qemufile_dst) { break; }
Quoted 7 lines
...
} while (1);I still don’t see the value of using
postcopy_qemufile_dst_donesem in the code though It simplify blocks the main thread from creating the preempt channel for 1ms (regardless of the possibility about whether the sem has been be posted or not. We add it for the case it is not posted and need to go back to the loop).
I think it used to only wait() in the preempt thread, so that is needed. It's also needed when postcopy is interrupted and need a recover, see loadvm_postcopy_handle_resume(), in that case it's the postcopy ram load thread that waits for it rather than the main thread or preempt thread.
Indeed if we move channel creation out of the preempt thread then it seems we don't need the sem in this path. However the other path will still need it, then when the new channel created (postcopy_preempt_new_channel) we'll need to identify a "switch to postcopy" case or "postcopy recovery" case, only post the sem when the former. I think it might complicate the code, I'll think again tomorrow after a sleep so my brain will work better, but I doubt this is what we want to do at rc3.
If you feel comfortable, please feel free to send a version that you think is the most correct so far (if you prefer no timedwait it's fine), and make sure the test works the best on your side. Then I'll smoke it a bit during weekends. Please always keep that in mind if that will be for rc3 it should be non-intrusive change, or we keep it slow for 9.1 and we don't need to rush.
Thanks,
Signature · 1 line
Peter Xu
Quoted 105 lines · Peter Xu
Quoted 33 lines · Wei W
Quoted 32 lines · Peter Xu
Quoted 7 lines · Lei
Quoted 5 lines · Peter Xu
Quoted 5 lines · Wei Wang
Quoted 4 lines
Before loading the guest states, ensure that the preempt channel has been ready to use, as some of the states (e.g. via
virtio_load) might trigger page faults that will be handled through thepreempt channel.
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
So yield to the main thread in the case that the channel create event has been dispatched.
Originally-by Lei Wang <[email protected]>Link:
https://lore.kernel.org/all/9aa5d1be-7801-40dd-83fd-f7e041ced2
49@i
ntel
.com/T/Suggested-by Peter Xu <[email protected]>Signed-off-by Lei Wang <[email protected]>Signed-off-by Wei Wang <[email protected]>--- migration/savevm.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)migration/savevm.c index renamed from migration/savevm.c +0
-0
388d7af7cd..fbc9f2bdd4100644migration/savevm.c +0
-0@@ -2342,6 +2342,23 @@ static intloadvm_handle_cmd_packaged (MigrationIncomingState *mis)Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
migration/savevm.c static int · line 2348
Quoted 5 lines
migration/savevm.c +3
-0QEMUFile *packf =qemu_file_new_input (QIO_CHANNEL (bioc)); +/* +* Before loading the guest states, ensure that the +preempt channel has
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
migration/savevm.c +1
-0+* been ready to use, as some of the states (e.g. via virtio_load) might
Quoted 15 lines
Quoted 15 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
migration/savevm.c static int · line 2351
Quoted 2 lines
migration/savevm.c +2
-0+ * trigger page faults that will be handled through the + preemptchannel.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
migration/savevm.c static int · line 2353
Quoted 2 lines
migration/savevm.c +2
-0+ * So yield to the main thread in the case that the + channel createevent
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
migration/savevm.c +9
-0+* has been dispatched. +*/ + do { + if (!migrate_postcopy_preempt() || !qemu_in_coroutine() || + mis->postcopy_qemufile_dst) { + break; + } + + aio_co_schedule(qemu_get_current_aio_context(),Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
qemu_coroutine_self());migration/savevm.c static int
Quoted 4 lines
migration/savevm.c +4
-0+qemu_coroutine_yield (); + }while + (!qemu_sem_timedwait (&mis->postcopy_qemufile_dst_done, +1 ));I think we need s/!// here, so the same mistake I made? I think we need to rework the retval of
qemu_sem_timedwait()at somepoint later..
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
No.
qemu_sem_timedwaitreturns false when timeout, which means semisn’t posted yet.
Quoted 5 lines
Quoted 1 line
So it needs to go back to the loop. (the patch was tested)
When timeout,
qemu_sem_timedwait()will return -1. I think the patch test passed may because you will always have at least one yield (the first yield in the do ...while ...) when
loadvm_handle_cmd_packaged()?Quoted 3 lines
Quoted 3 lines
My guess is that here the kick will work and
qemu_sem_timedwait()later will ETIMEOUT ->qemu_sem_timedwait()returns -1, then the loopjust broke.
Quoted 11 lines
Quoted 2 lines
That aio schedule should make sure anyway that the file is ready; the preempt thread must run before this to not hang that thread.
Yes, misread of the return value. It still worked because the loop broke at the "if (
mis->postcopy_qemufile_dst)" check.Even below will work:
do { if (mis->postcopy_qemufile_dst) { break; }Quoted 7 lines
...
} while (1);I still don’t see the value of using
postcopy_qemufile_dst_donesem in the code though It simplify blocks the main thread from creating the preempt channel for 1ms (regardless of the possibility about whether the sem has been be posted or not. We add it for the case it is not postedand need to go back to the loop).
I think it used to only
wait()in the preempt thread, so that is needed. It's also needed when postcopy is interrupted and need a recover, seeloadvm_postcopy_handle_resume(), in that case it's the postcopy ram load thread that waits for it rather than the main thread or preempt thread.
Quoted 7 lines
Indeed if we move channel creation out of the preempt thread then it seems we don't need the sem in this path. However the other path will still need it, then when the new channel created (
postcopy_preempt_new_channel) we'll need to identify a "switch to postcopy" case or "postcopy recovery" case, only post the sem when the former. I think it might complicate the code, I'll think again tomorrow after a sleep so my brain will work better, but I doubt this is what we want to do at rc3.
Yes, it's a bit rushed (no need to remove it completely at rc3).
Quoted 7 lines
If you feel comfortable, please feel free to send a version that you think is the most correct so far (if you prefer no timedwait it's fine), and make sure the test works the best on your side. Then I'll smoke it a bit during weekends. Please always keep that in mind if that will be for rc3 it should be non-intrusive change, or we keep it slow for 9.1 and we don't need to rush.
Sounds good.