nvmet-rdma: fix null dereference under heavy load
Subsystems (3)
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
Under heavy load if we don't have any pre-allocated rsps left, we dynamically allocate a rsp, but we are not actually allocating memory for nvme_completion (rsp->req.rsp). In such a case, accessing pointer fields (req->rsp->status) in nvmet_req_init() will result in crash.
To fix this, allocate the memory for nvme_completion by calling nvmet_rdma_alloc_rsp()
--
Changes from v2:
- Dropped the unneeded initialization of ret variable
Changes from v1:
- Moved integer to 'if' block
- Used 'unlikely' in datapath flow condition
--- drivers/nvme/target/rdma.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/nvme/target/rdma.c +14 -1
@@ -139,6 +139,10 @@ static void nvmet_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc);static void nvmet_rdma_read_data_done (struct ib_cq *cq,struct ib_wc *wc);static void nvmet_rdma_qp_event (struct ib_event *event,void *priv);static void nvmet_rdma_queue_disconnect (struct nvmet_rdma_queue *queue); +static void nvmet_rdma_free_rsp (struct nvmet_rdma_device *ndev, +struct nvmet_rdma_rsp *r); +static int nvmet_rdma_alloc_rsp (struct nvmet_rdma_device *ndev, +struct nvmet_rdma_rsp *r);static const struct nvmet_fabrics_ops nvmet_rdma_ops;
@@ -182,9 +186,17 @@ nvmet_rdma_get_rsp(struct nvmet_rdma_queue *queue)spin_unlock_irqrestore (&queue->rsps_lock, flags);if (unlikely (!rsp)) { - rsp =kmalloc (sizeof (*rsp),GFP_KERNEL ); +int ret; + + rsp =kzalloc (sizeof (*rsp),GFP_KERNEL );if (unlikely (!rsp))return NULL ; + ret =nvmet_rdma_alloc_rsp (queue->dev, rsp); +if (unlikely (ret)) { +kfree (rsp); +return NULL ; + } + rsp->allocated = true; }
@@ -197,6 +209,7 @@ nvmet_rdma_put_rsp(struct nvmet_rdma_rsp *rsp)unsigned long flags;if (unlikely (rsp->allocated)) { +nvmet_rdma_free_rsp (rsp->queue->dev, rsp);kfree (rsp);return ; }
Signature · 1 line
2.12.0
Sagi Grimberg
maintainerPATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
Wonder how the original fix got a tested-by...
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
Quoted 3 lines · Sagi Grimberg
Wonder how the original fix got a tested-by...
Reviewed-by Sagi Grimberg <[email protected]>
That was my bad. But the testers said they couldn't reproduce the issue, so I assumed that meant it was fixed. I guess they weren't stressing it enough.
Steve.
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
Hi Christoph,
In case you missed, could you please pull this patch ?
-Raju
-----Original Message-----
From: Steve Wise <[email protected]>
Sent: 07 January 2019 21:29
To: Sagi Grimberg <sagi at grimberg.me>; Raju Rangoju <rajur at chelsio.com>; linux-nvme at lists.infradead.org
Cc: maxg at mellanox.com; stable at vger.kernel.org
Subject: Re: [PATCH v3] nvmet-rdma: fix null dereference under heavy load
Quoted 3 lines · Sagi Grimberg
Wonder how the original fix got a tested-by...
Reviewed-by Sagi Grimberg <sagi at grimberg.me>
That was my bad.? But the testers said they couldn't reproduce the issue, so I assumed that meant it was fixed.? I guess they weren't stressing it enough.
Steve.
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
In reply to Raju RangojuThis looks ok to me:
In the long run I wonder if we should get rid of the ?tatic preallocation entirely and just use a free pool to avoid constant reallocations..
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
Quoted 3 lines
This looks ok to me:
Reviewed-by Christoph Hellwig <hch at lst.de>
Applied to nvme-5.0
Quoted 3 lines
In the long run I wonder if we should get rid of the ?tatic preallocation entirely and just use a free pool to avoid constant reallocations..
you mean a kmem_cache? very possible, we can also make the command struct to contain the rsp behind it to avoid the extra allocation...
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
Quoted 4 lines · Sagi Grimberg
Quoted 3 lines
This looks ok to me:
Reviewed-by Christoph Hellwig <hch at lst.de>
Quoted 8 lines
Applied to nvme-5.0
Quoted 3 lines
In the long run I wonder if we should get rid of the ?tatic preallocation entirely and just use a free pool to avoid constant reallocations..
you mean a
kmem_cache? very possible, we can also make the command struct to contain the rsp behind it to avoid the extra allocation...
yes, the container idea sounds good..
PATCH v3 1/1 nvmet-rdma: fix null dereference under heavy load
In reply to Sagi GrimbergQuoted 4 lines · Sagi Grimberg
Quoted 3 lines
This looks ok to me:
Reviewed-by Christoph Hellwig <hch at lst.de>
Quoted 8 lines
Applied to nvme-5.0
Quoted 3 lines
In the long run I wonder if we should get rid of the ?tatic preallocation entirely and just use a free pool to avoid constant reallocations..
you mean a
kmem_cache? very possible, we can also make the command struct to contain the rsp behind it to avoid the extra allocation...
Hey guys,
If every command struct requires a rsp, then it seems simpler to contain it in the command struct.
My 2 cents...
Steve.