nvme-pci: Set min align mask before calculating max_hw_sectors
PATCH 1/1 nvme-pci: Set min align mask before calculating max_hw_sectors
If swiotlb is force enabled dma_max_mapping_size ends up calling swiotlb_max_mapping_size which takes into account the min align mask for the device.
Set the min align mask for nvme driver before calling dma_max_mapping_size while calculating max hw sectors.
--- drivers/nvme/host/pci.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/nvme/host/pci.c +2 -1
@@ -2834,6 +2834,8 @@ static void nvme_reset_work(struct work_struct *work)nvme_start_admin_queue (&dev->ctrl); } +dma_set_min_align_mask (dev->dev,NVME_CTRL_PAGE_SIZE -1 ); +/* * Limit the max command size to prevent iod->sg allocations going * over a single page.
@@ -2846,7 +2848,6 @@ static void nvme_reset_work(struct work_struct *work)* Don't limit the IOMMU merged segment size. */ dma_set_max_seg_size (dev->dev,0xffffffff ); -dma_set_min_align_mask (dev->dev,NVME_CTRL_PAGE_SIZE -1 );mutex_unlock (&dev->shutdown_lock);
Signature · 1 line
2.37.1
Sagi Grimberg
maintainerQuoted 6 lines
If swiotlb is force enabled
dma_max_mapping_sizeends up callingswiotlb_max_mapping_sizewhich takes into account the min align mask for the device.
Set the min align mask for nvme driver before callingdma_max_mapping_sizewhile calculating max hw sectors.
Does this fix a specific bug? if so it needs a fixes tag so it can go to stable backports as well.
Quoted 27 lines
Signed-off-by Rishabh Bhatnagar <[email protected]>--- drivers/nvme/host/pci.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/nvme/host/pci.c +2
-1@@ -2834,6 +2834,8 @@ static void nvme_reset_work(struct work_struct *work)nvme_start_admin_queue (&dev->ctrl); } +dma_set_min_align_mask (dev->dev,NVME_CTRL_PAGE_SIZE -1 ); +/* * Limit the max command size to prevent iod->sg allocations going * over a single page. @@ -2846,7 +2848,6 @@ static void nvme_reset_work(struct work_struct *work)* Don't limit the IOMMU merged segment size. */ dma_set_max_seg_size (dev->dev,0xffffffff ); -dma_set_min_align_mask (dev->dev,NVME_CTRL_PAGE_SIZE -1 );mutex_unlock (&dev->shutdown_lock);
Christoph Hellwig
maintainerQuoted 9 lines · Sagi Grimberg
Quoted 6 lines
If swiotlb is force enabled
dma_max_mapping_sizeends up callingswiotlb_max_mapping_sizewhich takes into account the min align mask for the device.
Set the min align mask for nvme driver before callingdma_max_mapping_sizewhile calculating max hw sectors.Does this fix a specific bug? if so it needs a fixes tag so it can go to stable backports as well.
Hmm. Basically this is another fixlet for nvme on swiotlb, which already drove adding the max_mapping size and the align_mask, and it seems we still hadn't covered all corner cases properly. So I think it basically has been broken since day 1, but nvme on swiotlb only started becoming "interesting" with the trusted hypervisor schemes.
On 9/21/22, 10:58 PM, "Christoph Hellwig" <[email protected]> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
On Wed, Sep 21, 2022 at 11:10:58AM +0300, Sagi Grimberg wrote:
>
>> If swiotlb is force enabled dma_max_mapping_size ends up calling >> swiotlb_max_mapping_size which takes into account the min align >> mask for the device.
>> Set the min align mask for nvme driver before calling >> dma_max_mapping_size while calculating max hw sectors. >
> Does this fix a specific bug? if so it needs a fixes tag so it can > go to stable backports as well.
Hmm. Basically this is another fixlet for nvme on swiotlb, which already drove adding the max_mapping size and the align_mask, and it seems we still hadn't covered all corner cases properly. So I think it basically has been broken since day 1, but nvme on swiotlb only started becoming "interesting" with the trusted hypervisor schemes.
Yes this fixes a bug that we saw in nvme driver(after enabling swiotlb) that resembles what was fixed in 82806744fd7dde603b64c151eeddaa4ee62193fd. But not setting themin_align_mask before deriving max hw sectors was causing the bug to still exist.
Also we would probably need to backport the above fix along with (82806744fd7dde603b64c151eeddaa4ee62193fd) to 5.10 and 5.15 kernels. Let me know what you think is the best way to achieve that.