Skip to contentSkip to conversation

migration/rdma: unegister fd handler

PATCH In review Dr. David Alan Gilbert (git) 4 participants 8 messages Jan 22, 2019 – Feb 14, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 migration/rdma: unegister fd handler

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

---
 migration/rdma.c | 1 +
 1 file changed, 1 insertion(+)
migration/rdma.c +1 -0
index 9b2e7e10aa..54a3c11540 100644
@@ -2321,6 +2321,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
         rdma->connected = false;
     }
 
+    qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
     g_free(rdma->dest_blocks);
     rdma->dest_blocks = NULL;
 
Signature · 1 line
2.20.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 6 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

Quoted 6 lines

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

(Could the crash happened because the same fd number is re-used after
the RDMA channel was destroyed? Then when the fd has an event, it'll be delivered to rdma_cm_poll_handler() while the fd is not really the RDMA channel handle any more)

Regards,

Signature · 1 line
Peter Xu
#2 lore raw
Quoted 10 lines · * Peter Xu ([email protected]) wrote:
Quoted 9 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Quoted 8 lines
Quoted 3 lines

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

(Could the crash happened because the same fd number is re-used after
the RDMA channel was destroyed? Then when the fd has an event, it'll be delivered to rdma_cm_poll_handler() while the fd is not really the RDMA channel handle any more)

That's an interesting thought, I'd assumed it was just a race, but being dependent on the fd numbering would explain why it was so delicate to reproduce it.

Quoted 1 line

Thanks!

Dave

Quoted 4 lines

Regards,

Signature · 1 line
Peter Xu

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#3 lore raw
In reply to Dr. David Alan Gilbert (git)
Quoted 6 lines · * Dr. David Alan Gilbert (git) ([email protected]) wrote:

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

Quoted 6 lines

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

Queued

Quoted 20 lines
---
 migration/rdma.c | 1 +
 1 file changed, 1 insertion(+)
migration/rdma.c +1 -0
index 9b2e7e10aa..54a3c11540 100644
@@ -2321,6 +2321,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
         rdma->connected = false;
     }
 
+    qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
     g_free(rdma->dest_blocks);
     rdma->dest_blocks = NULL;
 
Signature · 1 line
2.20.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#4 lore raw
Quoted 7 lines · Dr. David Alan Gilbert
Quoted 6 lines · * Dr. David Alan Gilbert (git) ([email protected]) wrote:

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

Quoted 8 lines
Quoted 6 lines

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

Queued

Did you fixed the patch subject typo? "un(r)egister"

Quoted 23 lines
Quoted 20 lines
---
 migration/rdma.c | 1 +
 1 file changed, 1 insertion(+)
migration/rdma.c +1 -0
index 9b2e7e10aa..54a3c11540 100644
@@ -2321,6 +2321,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
         rdma->connected = false;
     }
 
+    qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
     g_free(rdma->dest_blocks);
     rdma->dest_blocks = NULL;
 
Signature · 1 line
2.20.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#5 lore raw
Quoted 11 lines · * Philippe Mathieu-Daudé ([email protected]) wrote:
Quoted 10 lines · Dr. David Alan Gilbert
Quoted 9 lines · * Dr. David Alan Gilbert (git) ([email protected]) wrote:

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

Queued

Did you fixed the patch subject typo? "un(r)egister"

Yes; fortunately I spotted it during building the pull :-)

Dave

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
---
 migration/rdma.c | 1 +
 1 file changed, 1 insertion(+)
migration/rdma.c +1 -0
index 9b2e7e10aa..54a3c11540 100644
@@ -2321,6 +2321,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
         rdma->connected = false;
     }
 
+    qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
     g_free(rdma->dest_blocks);
     rdma->dest_blocks = NULL;
 
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Signature · 1 line
2.20.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#6 lore raw
In reply to Dr. David Alan Gilbert (git)
Quoted 22 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

---
 migration/rdma.c | 1 +
 1 file changed, 1 insertion(+)
migration/rdma.c +0 -0
index 9b2e7e10aa..54a3c11540 100644

migration/rdma.c static void qemu_rdma_cleanup(RDMAContext *rdma) · line 2326

Quoted 6 lines
migration/rdma.c +1 -0
@@ -2321,6 +2321,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
         rdma->connected = false;
     }
 
+    qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
     g_free(rdma->dest_blocks);
     rdma->dest_blocks = NULL;

Hi -- this patch makes coverity complain (CID 1398634), because here we use rdma->channel without checking that it is NULL, but later in the function we have an "if (rdma->channel)" test. Should this code be conditional on rmda->channel being non-NULL, or is the later test incorrect?

thanks
-- PMM

#7 lore raw
Quoted 27 lines · * Peter Maydell ([email protected]) wrote:
Quoted 25 lines · Dr. David Alan Gilbert (git)

From: "Dr. David Alan Gilbert" <[email protected]>

Unregister the fd handler before we destroy the channel, otherwise we've got a race where we might land in the fd handler just as we're closing the device.

(The race is quite data dependent, you just have to have the right set of devices for it to trigger).

Corresponds to RH bz: https://bugzilla.redhat.com/show_bug.cgi?id=1666601

---
 migration/rdma.c | 1 +
 1 file changed, 1 insertion(+)
migration/rdma.c +0 -0
index 9b2e7e10aa..54a3c11540 100644
@@ -2321,6 +2321,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma)
         rdma->connected = false;
     }
 
Quoted 9 lines

migration/rdma.c static void qemu_rdma_cleanup(RDMAContext *rdma) · line 2326

Quoted 3 lines
migration/rdma.c +1 -0
+    qemu_set_fd_handler(rdma->channel->fd, NULL, NULL, NULL);
     g_free(rdma->dest_blocks);
     rdma->dest_blocks = NULL;

Hi -- this patch makes coverity complain (CID 1398634), because here we use rdma->channel without checking that it is NULL, but later in the function we have an "if (rdma->channel)" test. Should this code be conditional on rmda->channel being non-NULL, or is the later test incorrect?

Yes, it's got a point - I can seg that.

I'll post a fix.

Dave

Quoted 2 lines

thanks
-- PMM

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK