Skip to contentSkip to conversation

nvme-rdma: Avoid double freeing of async event data

PATCH Applied Prabhath Sajeepa 4 participants 4 messages Mar 9, 2020 – Mar 10, 2020 NVMe

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nvme-rdma: Avoid double freeing of async event data

The timeout of identify cmd, which is invoked as part of admin queue creation, can result in freeing of async event data both in nvme_rdma_timeout handler and error handling path of nvme_rdma_configure_admin queue thus causing NULL pointer reference.

Call Trace:
 ? nvme_rdma_setup_ctrl+0x223/0x800 [nvme_rdma]
 nvme_rdma_create_ctrl+0x2ba/0x3f7 [nvme_rdma]
 nvmf_dev_write+0xa54/0xcc6 [nvme_fabrics]
 __vfs_write+0x1b/0x40
 vfs_write+0xb2/0x1b0
 ksys_write+0x61/0xd0
 __x64_sys_write+0x1a/0x20
 do_syscall_64+0x60/0x1e0
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
---
 drivers/nvme/host/rdma.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/nvme/host/rdma.c +5 -3
index 3e85c5c..0fe08c4 100644
@@ -850,9 +850,11 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
 	if (new)
 		blk_mq_free_tag_set(ctrl->ctrl.admin_tagset);
 out_free_async_qe:
-	nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,
-		sizeof(struct nvme_command), DMA_TO_DEVICE);
-	ctrl->async_event_sqe.data = NULL;
+	if (ctrl->async_event_sqe.data) {
+		nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,
+			sizeof(struct nvme_command), DMA_TO_DEVICE);
+		ctrl->async_event_sqe.data = NULL;
+	}
 out_free_queue:
 	nvme_rdma_free_queue(&ctrl->queues[0]);
 	return error;
Signature · 1 line
2.7.4
Reviewed-by on v1 1/1
#1 lore raw
Quoted 30 lines · Prabhath Sajeepa

The timeout of identify cmd, which is invoked as part of admin queue creation, can result in freeing of async event data both in nvme_rdma_timeout handler and error handling path of nvme_rdma_configure_admin queue thus causing NULL pointer reference.

Call Trace:
  ? nvme_rdma_setup_ctrl+0x223/0x800 [nvme_rdma]
  nvme_rdma_create_ctrl+0x2ba/0x3f7 [nvme_rdma]
  nvmf_dev_write+0xa54/0xcc6 [nvme_fabrics]
  __vfs_write+0x1b/0x40
  vfs_write+0xb2/0x1b0
  ksys_write+0x61/0xd0
  __x64_sys_write+0x1a/0x20
  do_syscall_64+0x60/0x1e0
  entry_SYSCALL_64_after_hwframe+0x44/0xa9
---
  drivers/nvme/host/rdma.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)
drivers/nvme/host/rdma.c +0 -1
index 3e85c5c..0fe08c4 100644
@@ -850,9 +850,11 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
  	if (new)
  		blk_mq_free_tag_set(ctrl->ctrl.admin_tagset);
  out_free_async_qe:
-	nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,

drivers/nvme/host/rdma.c static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl, · line 860

Quoted 10 lines
drivers/nvme/host/rdma.c +5 -2
-		sizeof(struct nvme_command), DMA_TO_DEVICE);
-	ctrl->async_event_sqe.data = NULL;
+	if (ctrl->async_event_sqe.data) {
+		nvme_rdma_free_qe(ctrl->device->dev, &ctrl->async_event_sqe,
+			sizeof(struct nvme_command), DMA_TO_DEVICE);
+		ctrl->async_event_sqe.data = NULL;
+	}
  out_free_queue:
  	nvme_rdma_free_queue(&ctrl->queues[0]);
  	return error;

Looks good,

We did the same fix in-house yesterday and planed to send it tomorrow :)

Reviewed-by on v1 1/1
#2 lore raw
In reply to Prabhath Sajeepa
Quoted 15 lines · Prabhath Sajeepa

The timeout of identify cmd, which is invoked as part of admin queue creation, can result in freeing of async event data both in nvme_rdma_timeout handler and error handling path of nvme_rdma_configure_admin queue thus causing NULL pointer reference.

Call Trace:
 ? nvme_rdma_setup_ctrl+0x223/0x800 [nvme_rdma]
 nvme_rdma_create_ctrl+0x2ba/0x3f7 [nvme_rdma]
 nvmf_dev_write+0xa54/0xcc6 [nvme_fabrics]
 __vfs_write+0x1b/0x40
 vfs_write+0xb2/0x1b0
 ksys_write+0x61/0xd0
 __x64_sys_write+0x1a/0x20
 do_syscall_64+0x60/0x1e0
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
Quoted 2 lines

This looks good as a hot fix:

But I really think we need to do something about init vs timeout in the state machine. Otherwise we're going to deal with racing resource deallocation forever.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

maintainer Applied
#3 lore raw
In reply to Prabhath Sajeepa
Quoted 15 lines · Prabhath Sajeepa

The timeout of identify cmd, which is invoked as part of admin queue creation, can result in freeing of async event data both in nvme_rdma_timeout handler and error handling path of nvme_rdma_configure_admin queue thus causing NULL pointer reference.

Call Trace:
 ? nvme_rdma_setup_ctrl+0x223/0x800 [nvme_rdma]
 nvme_rdma_create_ctrl+0x2ba/0x3f7 [nvme_rdma]
 nvmf_dev_write+0xa54/0xcc6 [nvme_fabrics]
 __vfs_write+0x1b/0x40
 vfs_write+0xb2/0x1b0
 ksys_write+0x61/0xd0
 __x64_sys_write+0x1a/0x20
 do_syscall_64+0x60/0x1e0
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
Quoted 2 lines

Thanks, applied to new branch for 5.6-rc6.