migration/rdma disconnect fixes
cover migration/rdma disconnect fixes
From: "Dr. David Alan Gilbert" <[email protected]>
Hi,
This fixes a deadlock that can occur on the source after
a failed RDMA migration and cleans up some warning messages that can appear during normal completion.
https://bugzilla.redhat.com/show_bug.cgi?id=1746787
Dr. David Alan Gilbert (2):
migration/rdma: Don't moan about disconnects at the end migration/rdma.c: Swap synchronize_rcu for call_rcu
migration/rdma.c | 51 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 16 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/2 migration/rdma: Don't moan about disconnects at the end
From: "Dr. David Alan Gilbert" <[email protected]>
If we've already finished the migration or something has already gone wrong, don't moan about the migration stream disconnecting.
--- migration/rdma.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
migration/rdma.c +8 -4
@@ -3253,10 +3253,14 @@ static void rdma_cm_poll_handler(void *opaque)if (cm_event->event ==RDMA_CM_EVENT_DISCONNECTED || cm_event->event ==RDMA_CM_EVENT_DEVICE_REMOVAL ) { -error_report ("receive cm event, cm event is %d" , cm_event->event); - rdma->error_state = -EPIPE ; -if (rdma->return_path) { - rdma->return_path->error_state = -EPIPE ; +if (!rdma->error_state && +migration_incoming_get_current ()->state != +MIGRATION_STATUS_COMPLETED ) { +error_report ("receive cm event, cm event is %d" , cm_event->event); + rdma->error_state = -EPIPE ; +if (rdma->return_path) { + rdma->return_path->error_state = -EPIPE ; + } }if (mis->migration_incoming_co) {
Signature · 1 line
2.21.0
PATCH 2/2 migration/rdma.c: Swap synchronize_rcu for call_rcu
In reply to Dr. David Alan Gilbert (git)From: "Dr. David Alan Gilbert" <[email protected]>
This fixes a deadlock that can occur on the migration source after a failed RDMA migration; as the source tries to cleanup it clears a pair of pointers and uses synchronize_rcu to wait; this is happening on the main thread. With the CPUs running
a CPU thread can be an rcu reader and attempt to grab the main lock (kvm_handle_io->address_space_write->flatview_write->flatview_write_continue-> prepare_mmio_access->qemu_mutex_lock_iothread_impl)
Replace the synchronize_rcu with a call_rcu to postpone the freeing.
( https://bugzilla.redhat.com/show_bug.cgi?id=1746787 )
--- migration/rdma.c | 39 +++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-)
migration/rdma.c +27 -12
@@ -3017,11 +3017,35 @@ static void qio_channel_rdma_set_aio_fd_handler(QIOChannel *ioc, } } +struct rdma_close_rcu { +struct rcu_head rcu; +RDMAContext *rdmain; +RDMAContext *rdmaout; +}; + +/* callback from qio_channel_rdma_close via call_rcu */ +static void qio_channel_rdma_close_rcu (struct rdma_close_rcu *rcu) +{ +if (rcu->rdmain) { +qemu_rdma_cleanup (rcu->rdmain); + } + +if (rcu->rdmaout) { +qemu_rdma_cleanup (rcu->rdmaout); + } + +g_free (rcu->rdmain); +g_free (rcu->rdmaout); +g_free (rcu); +} +static int qio_channel_rdma_close (QIOChannel *ioc,Error **errp) {QIOChannelRDMA *rioc =QIO_CHANNEL_RDMA (ioc);RDMAContext *rdmain, *rdmaout; + struct rdma_close_rcu *rcu = g_new(struct rdma_close_rcu, 1); +trace_qemu_rdma_close (); rdmain = rioc->rdmain;
@@ -3034,18 +3058,9 @@ static int qio_channel_rdma_close(QIOChannel *ioc,atomic_rcu_set (&rioc->rdmaout,NULL ); } -synchronize_rcu (); - -if (rdmain) { -qemu_rdma_cleanup (rdmain); - } - -if (rdmaout) { -qemu_rdma_cleanup (rdmaout); - } - -g_free (rdmain); -g_free (rdmaout); + rcu->rdmain = rdmain; + rcu->rdmaout = rdmaout; +call_rcu (rcu, qio_channel_rdma_close_rcu, rcu);return 0 ; }
Signature · 1 line
2.21.0
Quoted 8 lines · Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" <[email protected]>
Hi,
This fixes a deadlock that can occur on the source after
a failed RDMA migration and cleans up some warning messages that can appear during normal completion.
Signature · 1 line
Peter Xu
Quoted 8 lines · * Dr. David Alan Gilbert (git) ([email protected]) wrote:
From: "Dr. David Alan Gilbert" <[email protected]>
Hi,
This fixes a deadlock that can occur on the source after
a failed RDMA migration and cleans up some warning messages that can appear during normal completion.
Queued
Quoted 11 lines
Dr. David Alan Gilbert (2):
migration/rdma: Don't moan about disconnects at the endmigration/rdma.c: Swapsynchronize_rcuforcall_rcumigration/rdma.c | 51 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 16 deletions(-)Signature · 1 line
2.21.0
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK