Skip to contentSkip to conversation

nvme: set dma alignment to qword

PATCH Applied Keith Busch 4 participants 8 messages May 21, 2020 – May 26, 2020 NVMe

lore
Share & tools
lore

Keith Busch

author maintainer
#0 lore raw

PATCH 1/1 nvme: set dma alignment to qword

The default dma alignment mask is 511, which is much larger than any nvme controller requires. NVMe controllers accept qword aligned DMA addresses, so set the request_queue constraints to that. This can help avoid bounce buffers on user passthrough commands.

---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
 		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
 	}
 	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);
 	if (ctrl->vwc & NVME_CTRL_VWC_PRESENT)
 		vwc = true;
 	blk_queue_write_cache(q, vwc, vwc);
Signature · 7 lines
2.24.1


_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer
#1 lore raw

drivers/nvme/host/core.c static void nvme_set_queue_limits(struct nvme_ctrl *ctrl, · line 2286

Quoted 10 lines
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
  		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
  	}
  	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);

Shouldn't be an issue for rdma/tcp, think that it should also be OK for FC as well but not sure.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

author maintainer
#2 lore raw
Quoted 13 lines · Sagi Grimberg

drivers/nvme/host/core.c static void nvme_set_queue_limits(struct nvme_ctrl *ctrl, · line 2286

Quoted 10 lines
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
  		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
  	}
  	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);

Shouldn't be an issue for rdma/tcp, think that it should also be OK for FC as well but not sure.

Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#3 lore raw
Quoted 15 lines · Keith Busch
Quoted 13 lines · Sagi Grimberg

drivers/nvme/host/core.c static void nvme_set_queue_limits(struct nvme_ctrl *ctrl, · line 2286

Quoted 10 lines
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
  		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
  	}
  	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);

Shouldn't be an issue for rdma/tcp, think that it should also be OK for FC as well but not sure.

Quoted 1 line

Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.

No callbacks please, especially without users. If FC needs an override we should just add a field to struct nvme_ctrl.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

author maintainer
#4 lore raw
Quoted 16 lines · Christoph Hellwig
Quoted 15 lines · Keith Busch
Quoted 13 lines · Sagi Grimberg

drivers/nvme/host/core.c static void nvme_set_queue_limits(struct nvme_ctrl *ctrl, · line 2286

Quoted 10 lines
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
  		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
  	}
  	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);

Shouldn't be an issue for rdma/tcp, think that it should also be OK for FC as well but not sure.

Quoted 4 lines
Quoted 1 line

Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.

No callbacks please, especially without users. If FC needs an override we should just add a field to struct nvme_ctrl.

Well, pci would be a user. But if you're okay with having the driver default to this alignment, would you consider taking this original patch, or do you want confirmation from FC developers on whether or not this is safe?

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#5 lore raw
Quoted 19 lines · Keith Busch
Quoted 18 lines · Christoph Hellwig
Quoted 16 lines · Keith Busch
Quoted 13 lines · Sagi Grimberg

drivers/nvme/host/core.c static void nvme_set_queue_limits(struct nvme_ctrl *ctrl, · line 2286

Quoted 10 lines
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
  		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
  	}
  	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);

Shouldn't be an issue for rdma/tcp, think that it should also be OK for FC as well but not sure.

Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.

Quoted 7 lines
Quoted 2 lines

No callbacks please, especially without users. If FC needs an override we should just add a field to struct nvme_ctrl.

Well, pci would be a user. But if you're okay with having the driver default to this alignment, would you consider taking this original patch, or do you want confirmation from FC developers on whether or not this is safe?

Lets wait for feedback from the FC folks.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#6 lore raw
Quoted 22 lines · Christoph Hellwig
Quoted 14 lines · Keith Busch
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Keith Busch
Quoted 11 lines · Sagi Grimberg

drivers/nvme/host/core.c static void nvme_set_queue_limits(struct nvme_ctrl *ctrl, · line 2286

Quoted 9 lines
drivers/nvme/host/core.c +1 -0
index 805d289e6cd9..ba860efd250d 100644
@@ -2283,6 +2283,7 @@ static void nvme_set_queue_limits(struct nvme_ctrl *ctrl,
   		blk_queue_max_segments(q, min_t(u32, max_segments, USHRT_MAX));
   	}
   	blk_queue_virt_boundary(q, ctrl->page_size - 1);
+	blk_queue_dma_alignment(q, 7);

Shouldn't be an issue for rdma/tcp, think that it should also be OK for FC as well but not sure.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Okay, just to be safe, I'll make this a nvme_ctrl_ops callback.

No callbacks please, especially without users. If FC needs an override we should just add a field to struct nvme_ctrl.

Well, pci would be a user. But if you're okay with having the driver default to this alignment, would you consider taking this original patch, or do you want confirmation from FC developers on whether or not this is safe?

Quoted 6 lines

Lets wait for feedback from the FC folks.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

It won't bother the FC devices as they are typically byte-based for payload dma requirements.

-- james

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme