Skip to contentSkip to conversation

nvme-rdma: flush async events before freeing qe

PATCH In review David Milburn 2 participants 3 messages Aug 31, 2020 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme-rdma: flush async events before freeing qe

Flush async event workqueue in case async event has been queued up before freeing qe and setting ctrl->async_event_sqe.data to NULL. Otherwise, we can get into a state were nvme_rdma_submit_ async_event() runs with sqe->data set to NULL leading to crash.

---
 drivers/nvme/host/rdma.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvme/host/rdma.c +1 -0
index 8e5ffe2f117d..76f117462d5b 100644
@@ -835,6 +835,7 @@ static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
 		blk_mq_free_tag_set(ctrl->ctrl.admin_tagset);
 	}
 	if (ctrl->async_event_sqe.data) {
+		flush_work(&ctrl->ctrl.async_event_work);
 		nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,
 				sizeof(struct nvme_command), DMA_TO_DEVICE);
 		ctrl->async_event_sqe.data = NULL;
Signature · 7 lines
2.18.1


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#1 lore raw
Quoted 16 lines

Flush async event workqueue in case async event has been queued up before freeing qe and setting ctrl->async_event_sqe.data to NULL. Otherwise, we can get into a state were nvme_rdma_submit_ async_event() runs with sqe->data set to NULL leading to crash.

---
  drivers/nvme/host/rdma.c | 1 +
  1 file changed, 1 insertion(+)
drivers/nvme/host/rdma.c +0 -0
index 8e5ffe2f117d..76f117462d5b 100644

drivers/nvme/host/rdma.c static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl, · line 841

Quoted 7 lines
drivers/nvme/host/rdma.c +1 -0
@@ -835,6 +835,7 @@ static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
  		blk_mq_free_tag_set(ctrl->ctrl.admin_tagset);
  	}
  	if (ctrl->async_event_sqe.data) {
+		flush_work(&ctrl->ctrl.async_event_work);
  		nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,
  				sizeof(struct nvme_command), DMA_TO_DEVICE);
  		ctrl->async_event_sqe.data = NULL;

Why not cancel the work?

I also think that we will want to have this in nvme-core, not specifically to nvme-rdma?

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#2 lore raw

Hi Sagi,

Quoted 11 lines · Sagi Grimberg
Quoted 11 lines

Flush async event workqueue in case async event has been queued up before freeing qe and setting ctrl->async_event_sqe.data to NULL. Otherwise, we can get into a state were nvme_rdma_submit_ async_event() runs with sqe->data set to NULL leading to crash.

---
  drivers/nvme/host/rdma.c | 1 +

  1 file changed, 1 insertion(+)
Quoted 15 lines
Quoted 12 lines
drivers/nvme/host/rdma.c +1 -0
index 8e5ffe2f117d..76f117462d5b 100644
@@ -835,6 +835,7 @@ static void nvme_rdma_destroy_admin_queue(struct 
 nvme_rdma_ctrl *ctrl,
           blk_mq_free_tag_set(ctrl->ctrl.admin_tagset);
       }
       if (ctrl->async_event_sqe.data) {
+        flush_work(&ctrl->ctrl.async_event_work);
           nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,
                   sizeof(struct nvme_command), DMA_TO_DEVICE);
Quoted 1 line

          ctrl->async_event_sqe.data = NULL;

Why not cancel the work?

Ok, cancel_work_sync would have been a better choice.

Quoted 4 lines

I also think that we will want to have this in nvme-core, not specifically to nvme-rdma?

I see how tcp.c may run into the same problem with nvme_tcp_free_admin_queue(), but since we queue err_work from rdma.c, tcp.c, fc.c, not sure how to move the change to core.c?

Each calls nvme_complete_async_event(), but doesn't look like we could check there.

Thanks,
David

_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme