nvme-pci: fix dma mapping leak on data setup error
PATCH v2 1/1 nvme-pci: fix dma mapping leak on data setup error
From: Keith Busch <[email protected]>
We're leaking the initial DMA mapping during iteration if we fail to allocate the tracking descriptor for both PRP and SGL. Unmap the iterator directly; we can't use the existing unmap helper because it depends on the tracking descriptor being successfully allocated.
The mappings were also leaking when the driver detects an invalid bio_vec when mapping PRPs, so fix that too.
---v1->v2:
Check the conditions that indicate we have to unmap. We know we don't have a coalesced iova in this path, so we can skip that, but need to check P2P and device dma requirements. And introduce a helper function for this so PRP and SGL don't duplicate the open coding for it.
Add fixes tags.
drivers/nvme/host/pci.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
drivers/nvme/host/pci.c +20 -2
@@ -996,6 +996,18 @@ static bool nvme_pci_prp_iter_next(struct request *req, struct device *dma_dev,return nvme_pci_prp_save_mapping (req, dma_dev, iter); } +static void nvme_unmap_iter (struct request *req,struct blk_dma_iter *iter) +{ +struct nvme_queue *nvmeq = req->mq_hctx->driver_data; +struct nvme_iod *iod =blk_mq_rq_to_pdu (req); + +if (iod->flags &IOD_DATA_P2P || !dma_need_unmap (dev)) +return ; + +dma_unmap_phys (nvmeq->dev->dev, iter->addr, iter->len,rq_dma_dir (req), + iod->flags &IOD_DATA_MMIO ?DMA_ATTR_MMIO :0 ); +} +static blk_status_t nvme_pci_setup_data_prp (struct request *req,struct blk_dma_iter *iter) {
@@ -1006,8 +1018,10 @@ static blk_status_t nvme_pci_setup_data_prp(struct request *req,unsigned int prp_len, i;__le64 *prp_list; -if (!nvme_pci_prp_save_mapping (req, nvmeq->dev->dev, iter)) +if (!nvme_pci_prp_save_mapping (req, nvmeq->dev->dev, iter)) { +nvme_unmap_iter (req, iter);return iter->status; + }/* * PRP1 always points to the start of the DMA transfers.
@@ -1112,6 +1126,7 @@ static blk_status_t nvme_pci_setup_data_prp(struct request *req,dev_err_once (nvmeq->dev->dev,"Incorrectly formed request for payload:%d nents:%d\n" ,blk_rq_payload_bytes (req),blk_rq_nr_phys_segments (req)); +nvme_unmap_data (req);return BLK_STS_IOERR ; }
@@ -1155,8 +1170,11 @@ static blk_status_t nvme_pci_setup_data_sgl(struct request *req, sg_list =dma_pool_alloc (nvme_dma_pool (nvmeq, iod),GFP_ATOMIC , &sgl_dma); -if (!sg_list) +if (!sg_list) { +nvme_unmap_iter (req, iter);return BLK_STS_RESOURCE ; + } + iod->descriptors[iod->nr_descriptors++] = sg_list; do {
Signature · 1 line
2.53.0-Meta
Keith Busch
maintainerQuoted 4 lines · Keith Busch
We're leaking the initial DMA mapping during iteration if we fail to allocate the tracking descriptor for both PRP and SGL. Unmap the iterator directly; we can't use the existing unmap helper because it depends on the tracking descriptor being successfully allocated.
I hate to do another rapid respin here, but I gotta do it. The metadata sgl setup also can leak for the same reason, so need to close that one too. And there's also a leak for a P2P case that needs fixing. So v3 coming shortly.