Skip to contentSkip to conversation

nvme-rdma: fix -EIO cleanup order in queue_rq

PATCH Superseded Xixin Liu 3 participants 6 messages Aug 17 – Aug 19 NVMe

lore
Share & tools
lore

Xixin Liu

author
#0 lore raw

PATCH 1/1 nvme-rdma: fix -EIO cleanup order in queue_rq

On -EIO, the RDMA queue_rq path reports a host path error and then still cleans up the command and unmaps the SQE DMA. The path error helper completes the request, so that is double cleanup and DMA unmap after the request is already complete.

Unmap the SQE first, then report the host path error. Skip the outer command cleanup on that path.

Fixes 62eca39722fd ("nvme-rdma: handle nvme_rdma_post_send failures better")
---
 drivers/nvme/host/rdma.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
drivers/nvme/host/rdma.c +7 -7
index 56cd228af1d5..d80b81676527 100644
@@ -2061,16 +2061,16 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 err_unmap:
 	nvme_rdma_unmap_data(queue, rq);
 err:
-	if (err == -EIO)
-		ret = nvme_host_path_error(rq);
-	else if (err == -ENOMEM || err == -EAGAIN)
-		ret = BLK_STS_RESOURCE;
-	else
-		ret = BLK_STS_IOERR;
-	nvme_cleanup_cmd(rq);
+	if (err != -EIO) {
+		nvme_cleanup_cmd(rq);
+		ret = (err == -ENOMEM || err == -EAGAIN) ?
+			BLK_STS_RESOURCE : BLK_STS_IOERR;
+	}
 unmap_qe:
 	ib_dma_unmap_single(dev, req->sqe.dma, sizeof(struct nvme_command),
 			    DMA_TO_DEVICE);
+	if (err == -EIO)
+		return nvme_host_path_error(rq);
 	return ret;
 }
 
Signature · 1 line
2.53.0
#1 lore raw

drivers/nvme/host/rdma.c static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx, · line 2067

Quoted 5 lines · Xixin Liu
drivers/nvme/host/rdma.c +4 -1
-	nvme_cleanup_cmd(rq);
+	if (err != -EIO) {
+		nvme_cleanup_cmd(rq);
+		ret = (err == -ENOMEM || err == -EAGAIN) ?
+			BLK_STS_RESOURCE : BLK_STS_IOERR;

Please stick to the much more readable if/else here.

author
#2 lore raw

v2 cover nvme-rdma: fix -EIO cleanup order in queue_rq

In reply to Xixin Liu

Hi,

Thanks Christoph for the if/else readability suggestion on the RESOURCE versus IOERR branch.

Changes since v1:
- use if/else for RESOURCE versus IOERR as suggested by Christoph
- initialize err to 0 for the shared unmap_qe path

Thanks,
Xixin Liu

---

Xixin Liu (1):
nvme-rdma: fix -EIO cleanup order in queue_rq

 drivers/nvme/host/rdma.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
Signature · 1 line
2.53.0
author
#3 lore raw

PATCH v2 1/1 nvme-rdma: fix -EIO cleanup order in queue_rq

On -EIO, the RDMA queue_rq path reports a host path error and then still cleans up the command and unmaps the SQE DMA. The path error helper completes the request, so that is double cleanup and DMA unmap after the request is already complete.

Unmap the SQE first, then report the host path error. Skip the outer command cleanup on that path.

Fixes 62eca39722fd ("nvme-rdma: handle nvme_rdma_post_send failures better")
---
 drivers/nvme/host/rdma.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/nvme/host/rdma.c +10 -8
index 56cd228af1d5..a1b2c3d4e5f6 100644
@@ -2005,7 +2005,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 	struct ib_device *dev;
 	bool queue_ready = test_bit(NVME_RDMA_Q_LIVE, &queue->flags);
 	blk_status_t ret;
-	int err;
+	int err = 0;
 
 	WARN_ON_ONCE(rq->tag < 0);
 
@@ -2061,16 +2061,18 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx,
 err_unmap:
 	nvme_rdma_unmap_data(queue, rq);
 err:
-	if (err == -EIO)
-		ret = nvme_host_path_error(rq);
-	else if (err == -ENOMEM || err == -EAGAIN)
-		ret = BLK_STS_RESOURCE;
-	else
-		ret = BLK_STS_IOERR;
-	nvme_cleanup_cmd(rq);
+	if (err != -EIO) {
+		nvme_cleanup_cmd(rq);
+		if (err == -ENOMEM || err == -EAGAIN)
+			ret = BLK_STS_RESOURCE;
+		else
+			ret = BLK_STS_IOERR;
+	}
 unmap_qe:
 	ib_dma_unmap_single(dev, req->sqe.dma, sizeof(struct nvme_command),
 			    DMA_TO_DEVICE);
+	if (err == -EIO)
+		return nvme_host_path_error(rq);
 	return ret;
 }
 
Signature · 1 line
2.53.0
maintainer Applied
#5 lore raw
In reply to Xixin Liu
Quoted 7 lines · Xixin Liu

On -EIO, the RDMA queue_rq path reports a host path error and then still cleans up the command and unmaps the SQE DMA. The path error helper completes the request, so that is double cleanup and DMA unmap after the request is already complete.

Unmap the SQE first, then report the host path error. Skip the outer command cleanup on that path.

Thanks, applied to nvme-7.3.