mpt3sas: Limit NVMe request size to 2 MiB
Subsystems (3)
Ranjan Kumar
maintainerPATCH 1/1 mpt3sas: Limit NVMe request size to 2 MiB
Some firmware reports NVMe maximum transfer sizes that follow the drive capability. When those values are very large, the block layer may build I/O that this driver cannot handle, which can cause a kernel oops.
When an NVMe device is set up, cap how large a single transfer may be to the smaller of the firmware-reported limit and roughly two mebibytes with a small margin. If no valid limit is reported, apply the same upper bound.
--- drivers/scsi/mpt3sas/mpt3sas_scsih.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/scsi/mpt3sas/mpt3sas_scsih.c +11 -1
@@ -54,6 +54,7 @@#include <linux/interrupt.h> #include <linux/raid_class.h> #include <linux/unaligned.h> +#include <linux/sizes.h> #include "mpt3sas_base.h"
@@ -2738,8 +2739,17 @@ scsih_sdev_configure(struct scsi_device *sdev, struct queue_limits *lim) pcie_device->enclosure_level, pcie_device->connector_name); +/* +* Firmware may report NVMe MDTS from the drive; values above +* what the driver can handle can cause a kernel oops. Cap queue +* I/O in sectors to min(MDTS, 2 MiB - 4096 B). +*/ if (pcie_device->nvme_mdts) - lim->max_hw_sectors = pcie_device->nvme_mdts /512 ; + lim->max_hw_sectors =min_t (u32, + pcie_device->nvme_mdts /512 , + (SZ_2M /512 ) -8 ); +else + lim->max_hw_sectors = (SZ_2M /512 ) -8 ;pcie_device_put (pcie_device);spin_unlock_irqrestore (&ioc->pcie_device_lock, flags);
Signature · 1 line
2.47.3
Quoted 33 lines · Ranjan Kumar
Some firmware reports NVMe maximum transfer sizes that follow the drive capability. When those values are very large, the block layer may build I/O that this driver cannot handle, which can cause a kernel oops.
When an NVMe device is set up, cap how large a single transfer may be to the smaller of the firmware-reported limit and roughly two mebibytes with a small margin. If no valid limit is reported, apply the same upper bound.
Fixes 9b8b84879d4a ("block: Increase BLK_DEF_MAX_SECTORS_CAP")Reported-by Mira Limbeck <[email protected]>Suggested-by Keith Busch <[email protected]>Signed-off-by Ranjan Kumar <[email protected]>--- drivers/scsi/mpt3sas/mpt3sas_scsih.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)drivers/scsi/mpt3sas/mpt3sas_scsih.c +1
-0@@ -54,6 +54,7 @@#include <linux/interrupt.h> #include <linux/raid_class.h> #include <linux/unaligned.h> +#include <linux/sizes.h> #include "mpt3sas_base.h"
drivers/scsi/mpt3sas/mpt3sas_scsih.c scsih_sdev_configure(struct scsi_device *sdev, struct queue_limits *lim) · line 2746
Quoted 8 lines
drivers/scsi/mpt3sas/mpt3sas_scsih.c +5
-0@@ -2738,8 +2739,17 @@ scsih_sdev_configure(struct scsi_device *sdev, struct queue_limits *lim) pcie_device->enclosure_level, pcie_device->connector_name); +/* +* Firmware may report NVMe MDTS from the drive; values above +* what the driver can handle can cause a kernel oops. Cap queue +* I/O in sectors to min(MDTS, 2 MiB - 4096 B). +*/
This comment has some grammar issues and is really hard to parse/understand.
drivers/scsi/mpt3sas/mpt3sas_scsih.c scsih_sdev_configure(struct scsi_device *sdev, struct queue_limits *lim) · line 2752
Quoted 7 lines
drivers/scsi/mpt3sas/mpt3sas_scsih.c +5
-1if (pcie_device->nvme_mdts) - lim->max_hw_sectors = pcie_device->nvme_mdts /512 ; + lim->max_hw_sectors =min_t (u32, + pcie_device->nvme_mdts /512 , + (SZ_2M /512 ) -8 ); +else + lim->max_hw_sectors = (SZ_2M /512 ) -8 ;
I am very confused here: SZ_2MB assumes that you have an SSD with a minimum page size of 4K, which can fit 4K / 8 = 512 PRP entries, each referencing 4K (one page), so a maximum of 2MiB. However, if I am not mistaken, there is nothing in nvme specs that forces the MPS field to be 0 (which leads to a page size of 4K). So this seems incorrect to me, even though that will probably work for the vast majority of SSDs out there, some exotic ones will not be correctly supported.
Keith ? Am I missing something here ?
Or do we simply do not care about SSDs with a minimum page size > 4K having their maximum command size truncated ?
Quoted 3 lines
drivers/scsi/mpt3sas/mpt3sas_scsih.c +0
-0pcie_device_put (pcie_device);spin_unlock_irqrestore (&ioc->pcie_device_lock, flags);
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 15 lines · Damien Le Moal
drivers/scsi/mpt3sas/mpt3sas_scsih.c scsih_sdev_configure(struct scsi_device *sdev, struct queue_limits *lim) · line 2752
Quoted 7 lines · Ranjan Kumar
drivers/scsi/mpt3sas/mpt3sas_scsih.c +5
-1if (pcie_device->nvme_mdts) - lim->max_hw_sectors = pcie_device->nvme_mdts /512 ; + lim->max_hw_sectors =min_t (u32, + pcie_device->nvme_mdts /512 , + (SZ_2M /512 ) -8 ); +else + lim->max_hw_sectors = (SZ_2M /512 ) -8 ;I am very confused here:
SZ_2MBassumes that you have an SSD with a minimum page size of 4K, which can fit 4K / 8 = 512 PRP entries, each referencing 4K (one page), so a maximum of 2MiB. However, if I am not mistaken, there is nothing in nvme specs that forces the MPS field to be 0 (which leads to a page size of 4K).
Quoted 7 lines
So this seems incorrect to me, even though that will probably work for the vast majority of SSDs out there, some exotic ones will not be correctly supported.
Keith ? Am I missing something here ?
Or do we simply do not care about SSDs with a minimum page size > 4K having their maximum command size truncated ?
Spec doesn't require it, but industry converged on that as always being the minimum supported page size. The nvme driver rejects any device that doesn't support 4k pages because they can't be reliably supported on a lot of archs, even ones with larger page sizes. So it should be a safe assumption that everyone supports 4k since no on is complaining. :)
On the patch, I initially left the "- 8" in the calculation to account for page offsets. But it's not necessary because that gets absorbed in PRP1 within the command, so we'd have at most 512 entries in the PRP list for a 2M transfer.
Ranjan Kumar
maintainerHi Damien and Keith,
Thank you both for the review and the clarifications.I will submit the v2 patch.
Quoted 35 lines · Keith Busch
Quoted 15 lines · Damien Le Moal
drivers/scsi/mpt3sas/mpt3sas_scsih.c scsih_sdev_configure(struct scsi_device *sdev, struct queue_limits *lim) · line 2752
Quoted 7 lines · Ranjan Kumar
if (pcie_device->nvme_mdts)drivers/scsi/mpt3sas/mpt3sas_scsih.c +5
-1- lim->max_hw_sectors = pcie_device->nvme_mdts /512 ; + lim->max_hw_sectors =min_t (u32, + pcie_device->nvme_mdts /512 , + (SZ_2M /512 ) -8 ); + else + lim->max_hw_sectors = (SZ_2M / 512) - 8;I am very confused here:
SZ_2MBassumes that you have an SSD with a minimum page size of 4K, which can fit 4K / 8 = 512 PRP entries, each referencing 4K (one page), so a maximum of 2MiB. However, if I am not mistaken, there is nothing in nvme specs that forces the MPS field to be 0 (which leads to a page size of 4K).Quoted 7 lines
So this seems incorrect to me, even though that will probably work for the vast majority of SSDs out there, some exotic ones will not be correctly supported.
Keith ? Am I missing something here ?
Or do we simply do not care about SSDs with a minimum page size > 4K having their maximum command size truncated ?
Spec doesn't require it, but industry converged on that as always being the minimum supported page size. The nvme driver rejects any device that doesn't support 4k pages because they can't be reliably supported on a lot of archs, even ones with larger page sizes. So it should be a safe assumption that everyone supports 4k since no on is complaining. :)
On the patch, I initially left the "- 8" in the calculation to account for page offsets. But it's not necessary because that gets absorbed in PRP1 within the command, so we'd have at most 512 entries in the PRP list for a 2M transfer.