nvme: improve discard_granularity spec compliance
Subsystems (2)
cover nvme: improve discard_granularity spec compliance
NVMe block devices always report the logical block size for the discard_granularity queue limit. However, more accurate values may be available in the NPDG/NPDA fields of the Identify Namespace structure or the NPDGL/NPDAL fields of the NVM Command Set Specific Identify Namespace structure. So use these values to compute discard_granularity.
Also fix the use of the OPTPERF field to better comply with version 2.1 of the NVMe spec.
Update the target side to report NPDGL and NPDAL as wel, in case the discard granularity doesn't fit in the 16-bit NPDG and NPDA fields.
Caleb Sander Mateos (6):
nvme: add preferred I/O size fields to struct nvme_id_ns_nvm
nvme: update nvme_id_ns OPTPERF constants
nvme: always issue I/O Command Set specific Identify Namespace
nvme: set discard_granularity from NPDG/NPDA
nvmet: use NVME_NS_FEAT_OPTPERF_SHIFT
nvmet: report NPDGL and NPDAL
drivers/nvme/host/core.c | 21 ++++++++++++++++----- drivers/nvme/target/admin-cmd.c | 2 ++ drivers/nvme/target/io-cmd-bdev.c | 19 +++++++++++++++---- drivers/nvme/target/nvmet.h | 2 ++ include/linux/nvme.h | 10 ++++++++-- 5 files changed, 43 insertions(+), 11 deletions(-)
Signature · 1 line
2.45.2
PATCH 1/6 nvme: add preferred I/O size fields to struct nvme_id_ns_nvm
A subsequent change will use the NPDGL and NPDAL fields of the NVM Command Set Specific Identify Namespace structure, so add them (and the handful of intervening fields) to struct nvme_id_ns_nvm.
--- include/linux/nvme.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
include/linux/nvme.h +6 -1
@@ -511,11 +511,16 @@ struct nvme_id_ctrl_zns {struct nvme_id_ns_nvm {__le64 lbstm;__u8 pic;__u8 rsvd9[3 ];__le32 elbaf[64 ]; -__u8 rsvd268[3828 ]; +__le32 npdgl; +__le32 nprg; +__le32 npra; +__le32 nors; +__le32 npdal; +__u8 rsvd288[3808 ]; };enum {NVME_ID_NS_NVM_STS_MASK =0x7f ,NVME_ID_NS_NVM_GUARD_SHIFT =7 ,
Signature · 1 line
2.45.2
Christoph Hellwig
maintainerRe: [PATCH 1/6] nvme: add preferred I/O size fields to struct nvme_id_ns_nvm
Quoted 3 lines · Caleb Sander Mateos
A subsequent change will use the NPDGL and NPDAL fields of the NVM Command Set Specific Identify Namespace structure, so add them (and the handful of intervening fields) to struct
nvme_id_ns_nvm.
Looks good. Btw, as the kernel now has a the global scope static_assert, can you add that too any on the wire format struct you touch to ensure our modifications aren't changing the sizses?
Re: [PATCH 1/6] nvme: add preferred I/O size fields to struct nvme_id_ns_nvm
Quoted 9 lines · Christoph Hellwig
Quoted 3 lines · Caleb Sander Mateos
A subsequent change will use the NPDGL and NPDAL fields of the NVM Command Set Specific Identify Namespace structure, so add them (and the handful of intervening fields) to struct
nvme_id_ns_nvm.Looks good. Btw, as the kernel now has a the global scope
static_assert, can you add that too any on the wire format struct you touch to ensure our modifications aren't changing the sizses?
Looks like static_assert() was added 3 years before struct nvme_id_ns_nvm :) Definitely agree it's good to assert on these struct sizes.
Quoted 2 lines
Signed-off-by Christoph Hellwig <[email protected]>
Assuming you meant "Reviewed-by". Let me know if you meant something else.
Thanks,
Caleb
Christoph Hellwig
maintainerRe: [PATCH 1/6] nvme: add preferred I/O size fields to struct nvme_id_ns_nvm
Quoted 7 lines · Caleb Sander Mateos
Looks like
static_assert()was added 3 years before structnvme_id_ns_nvm:) Definitely agree it's good to assert on these struct sizes.Quoted 2 lines
Signed-off-by Christoph Hellwig <[email protected]>Assuming you meant "Reviewed-by". Let me know if you meant something else.
Yes, sorry.
PATCH 2/6 nvme: update nvme_id_ns OPTPERF constants
In reply to Caleb Sander MateosSince NVM Command Spec 1.1, OPTPERF comprises both bits 4 and 5 of NSFEAT in the Identify Namespace structure. Replace NVME_NS_FEAT_IO_OPT, which represented only bit 4, with NVME_NS_FEAT_OPTPERF_SHIFT and NVME_NS_FEAT_OPTPERF_MASK.
Update nvme_update_disk_info() to check both OPTPERF bits, as NPWG and NOWS are supported even if only bit 5 is set.
--- drivers/nvme/host/core.c | 3 ++- include/linux/nvme.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)
drivers/nvme/host/core.c +2 -1
@@ -2095,11 +2095,12 @@ static bool nvme_update_disk_info(struct nvme_ns *ns, struct nvme_id_ns *id, } phys_bs = bs; atomic_bs =nvme_configure_atomic_write (ns, id, lim, bs); -if (id->nsfeat &NVME_NS_FEAT_IO_OPT ) { +if (id->nsfeat >>NVME_NS_FEAT_OPTPERF_SHIFT & +NVME_NS_FEAT_OPTPERF_MASK ) {/* NPWG = Namespace Preferred Write Granularity */ phys_bs = bs * (1 +le16_to_cpu (id->npwg));/* NOWS = Namespace Optimal Write Size */ if (id->nows) io_opt = bs * (1 +le16_to_cpu (id->nows));
include/linux/nvme.h +2 -1
@@ -593,11 +593,12 @@ enum { };enum {NVME_NS_FEAT_THIN =1 <<0 ,NVME_NS_FEAT_ATOMICS =1 <<1 , -NVME_NS_FEAT_IO_OPT =1 <<4 , +NVME_NS_FEAT_OPTPERF_SHIFT =4 , +NVME_NS_FEAT_OPTPERF_MASK =0x3 ,NVME_NS_ATTR_RO =1 <<0 ,NVME_NS_FLBAS_LBA_MASK =0xf ,NVME_NS_FLBAS_LBA_UMASK =0x60 ,NVME_NS_FLBAS_LBA_SHIFT =1 ,NVME_NS_FLBAS_META_EXT =0x10 ,
Signature · 1 line
2.45.2
Christoph Hellwig
maintainerRe: [PATCH 2/6] nvme: update nvme_id_ns OPTPERF constants
Quoted 4 lines · Caleb Sander Mateos
Since NVM Command Spec 1.1, OPTPERF comprises both bits 4 and 5 of NSFEAT in the Identify Namespace structure. Replace
NVME_NS_FEAT_IO_OPT, which represented only bit 4, withNVME_NS_FEAT_OPTPERF_SHIFTandNVME_NS_FEAT_OPTPERF_MASK.
As you're deeper in this than me: do the older specs require the other bit to be cleared to zero and this is all compatible? Or do we need to guard it somehow?
Otherwise looks good.
Re: [PATCH 2/6] nvme: update nvme_id_ns OPTPERF constants
Quoted 10 lines · Christoph Hellwig
Quoted 4 lines · Caleb Sander Mateos
Since NVM Command Spec 1.1, OPTPERF comprises both bits 4 and 5 of NSFEAT in the Identify Namespace structure. Replace
NVME_NS_FEAT_IO_OPT, which represented only bit 4, withNVME_NS_FEAT_OPTPERF_SHIFTandNVME_NS_FEAT_OPTPERF_MASK.As you're deeper in this than me: do the older specs require the other bit to be cleared to zero and this is all compatible? Or do we need to guard it somehow?
In NVM Command Set spec 1.0 (NVMe version 2.0), OPTPERF is only bit 4 of NSFEAT and the higher bits are reserved. I think it seems relatively safe to assume bit 5 will be reported as 0 on controllers advertising older NVMe versions, but you're correct that the host should technically check for version >= 2.1 before using bit 5. Let me know which approach you'd prefer.
Thanks,
Caleb
Christoph Hellwig
maintainerRe: [PATCH 2/6] nvme: update nvme_id_ns OPTPERF constants
Quoted 12 lines · Caleb Sander Mateos
Quoted 10 lines · Christoph Hellwig
Quoted 4 lines · Caleb Sander Mateos
Since NVM Command Spec 1.1, OPTPERF comprises both bits 4 and 5 of NSFEAT in the Identify Namespace structure. Replace
NVME_NS_FEAT_IO_OPT, which represented only bit 4, withNVME_NS_FEAT_OPTPERF_SHIFTandNVME_NS_FEAT_OPTPERF_MASK.As you're deeper in this than me: do the older specs require the other bit to be cleared to zero and this is all compatible? Or do we need to guard it somehow?
Quoted 6 lines
In NVM Command Set spec 1.0 (NVMe version 2.0), OPTPERF is only bit 4 of NSFEAT and the higher bits are reserved. I think it seems relatively safe to assume bit 5 will be reported as 0 on controllers advertising older NVMe versions, but you're correct that the host should technically check for version >= 2.1 before using bit 5. Let me know which approach you'd prefer.
Not entirely sure, but either way the code should have a big fat comment about this.
PATCH 3/6 nvme: always issue I/O Command Set specific Identify Namespace
In reply to Caleb Sander MateosCurrently, the I/O Command Set specific Identify Namespace structure is only fetched for controllers that support extended LBA formats. This is because struct nvme_id_ns_nvm is only used by nvme_configure_pi_elbas(), which is only called when the ELBAS bit is set in the CTRATT field of the Identify Controller structure.
However, the I/O Command Set specific Identify Namespace structure will soon be used in nvme_config_discard(), so always try to obtain it in nvme_update_ns_info_block() if the controller supports NVMe version 2.0 or later.
--- drivers/nvme/host/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/nvme/host/core.c +1 -1
@@ -2351,11 +2351,11 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, ret = -ENXIO ; goto out; } lbaf =nvme_lbaf_index (id->flbas); -if (ns->ctrl->ctratt &NVME_CTRL_ATTR_ELBAS ) { +if (ns->ctrl->vs >=NVME_VS (2 ,0 ,0 )) { ret =nvme_identify_ns_nvm (ns->ctrl, info->nsid, &nvm);if (ret <0 ) goto out; }
Signature · 1 line
2.45.2
Re: [PATCH 3/6] nvme: always issue I/O Command Set specific Identify Namespace
Quoted 10 lines · Caleb Sander Mateos
Currently, the I/O Command Set specific Identify Namespace structure is only fetched for controllers that support extended LBA formats. This is because struct
nvme_id_ns_nvmis only used bynvme_configure_pi_elbas(), which is only called when the ELBAS bit is set in the CTRATT field of the Identify Controller structure.However, the I/O Command Set specific Identify Namespace structure will soon be used in
nvme_config_discard(), so always try to obtain it innvme_update_ns_info_block()if the controller supports NVMe version 2.0 or later.
This was done kinda intentionally to avoid extra roundtrips and tripping over buggy implementations. But I think you have a good use case to extent this, so:
Quoted 5 lines
(quoted diff) +0
-0goto out; }Signature · 2 lines
-- 2.45.2
---end quoted text---
PATCH 4/6 nvme: set discard_granularity from NPDG/NPDA
In reply to Caleb Sander MateosCurrently, nvme_config_discard() always sets the discard_granularity queue limit to the logical block size. However, NVMe namespaces can advertise a larger preferred discard granularity in the NPDG or NPDA field of the Identify Namespace structure or the NPDGL or NPDAL fields of the I/O Command Set Specific Identify Namespace structure.
Use these fields to compute the discard_granularity limit. The logic is somewhat involved. First, the fields are optional. NPDG is only reported if the low bit of OPTPERF is set in NSFEAT. NPDA is reported if any bit of OPTPERF is set. And NPDGL and NPDAL are reported if the high bit of OPTPERF is set. NPDGL and NPDAL can also each be set to 0 to opt out of reporting a limit. I/O Command Set Specific Identify Namespace may also not be supported by older NVMe controllers. Another complication is that multiple values may be reported among NPDG, NPDGL, NPDA, and NPDAL. The spec says to prefer the values reported in the L variants. The spec says NPDG should be a multiple of NPDA and NPDGL should be a multiple of NPDAL, but it doesn't specify a relationship between NPDG and NPDAL or NPDGL and NPDA. So use the maximum of the reported NPDG(L) and NPDA(L) values as the discard_granularity.
--- drivers/nvme/host/core.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
drivers/nvme/host/core.c +13 -3
@@ -1880,23 +1880,33 @@ static bool nvme_init_integrity(struct nvme_ns_head *head, bi->pi_offset = info->pi_offset; }return true; } -static void nvme_config_discard (struct nvme_ns *ns,struct queue_limits *lim) +static void nvme_config_discard (struct nvme_ns *ns,struct nvme_id_ns *id, +struct nvme_id_ns_nvm *nvm, +struct queue_limits *lim) {struct nvme_ctrl *ctrl = ns->ctrl; +u32 npdg, npda; +u8 optperf;if (ctrl->dmrsl && ctrl->dmrsl <=nvme_sect_to_lba (ns->head,UINT_MAX )) lim->max_hw_discard_sectors =nvme_lba_to_sect (ns->head, ctrl->dmrsl);else if (ctrl->oncs &NVME_CTRL_ONCS_DSM ) lim->max_hw_discard_sectors =UINT_MAX ;else lim->max_hw_discard_sectors =0 ; - lim->discard_granularity = lim->logical_block_size; + optperf = id->nsfeat >>NVME_NS_FEAT_OPTPERF_SHIFT & +NVME_NS_FEAT_OPTPERF_MASK ; + npdg = optperf &0x2 && nvm && nvm->npdgl ?le32_to_cpu (nvm->npdgl) : + optperf &0x1 ?le16_to_cpu (id->npdg) +1 :1 ; + npda = optperf &0x2 && nvm && nvm->npdal ?le32_to_cpu (nvm->npdal) : + optperf ?le16_to_cpu (id->npda) +1 :1 ; + lim->discard_granularity =max (npdg, npda) * lim->logical_block_size;if (ctrl->dmrl) lim->max_discard_segments = ctrl->dmrl;else lim->max_discard_segments =NVME_DSM_MAX_RANGES ;
@@ -2382,11 +2392,11 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns,nvme_configure_metadata (ns->ctrl, ns->head, id, nvm, info);nvme_set_chunk_sectors (ns, id, &lim);if (!nvme_update_disk_info (ns, id, &lim)) capacity =0 ; -nvme_config_discard (ns, &lim); +nvme_config_discard (ns, id, nvm, &lim);if (IS_ENABLED (CONFIG_BLK_DEV_ZONED ) && ns->head->ids.csi ==NVME_CSI_ZNS )nvme_update_zone_info (ns, &lim, &zi);if ((ns->ctrl->vwc &NVME_CTRL_VWC_PRESENT ) && !info->no_vwc)
Signature · 1 line
2.45.2
Christoph Hellwig
maintainerRe: [PATCH 4/6] nvme: set discard_granularity from NPDG/NPDA
drivers/nvme/host/core.c static bool nvme_init_integrity(struct nvme_ns_head *head, · line 1887
Quoted 4 lines · Caleb Sander Mateos
drivers/nvme/host/core.c +3
-1-static void nvme_config_discard (struct nvme_ns *ns,struct queue_limits *lim) +static void nvme_config_discard (struct nvme_ns *ns,struct nvme_id_ns *id, +struct nvme_id_ns_nvm *nvm, +struct queue_limits *lim)
The indentation here looks weird, this should be just two tabs.
drivers/nvme/host/core.c static bool nvme_init_integrity(struct nvme_ns_head *head, · line 1906
Quoted 7 lines
drivers/nvme/host/core.c +6
-1- lim->discard_granularity = lim->logical_block_size; + optperf = id->nsfeat >>NVME_NS_FEAT_OPTPERF_SHIFT & +NVME_NS_FEAT_OPTPERF_MASK ; + npdg = optperf &0x2 && nvm && nvm->npdgl ?le32_to_cpu (nvm->npdgl) : + optperf &0x1 ?le16_to_cpu (id->npdg) +1 :1 ; + npda = optperf &0x2 && nvm && nvm->npdal ?le32_to_cpu (nvm->npdal) : + optperf ?le16_to_cpu (id->npda) +1 :1 ;
Please undinw this into a few if statemens. (And I really with the spec would given the bits names..)
PATCH 5/6 nvmet: use NVME_NS_FEAT_OPTPERF_SHIFT
In reply to Caleb Sander MateosUse the NVME_NS_FEAT_OPTPERF_SHIFT constant in nvmet_bdev_set_limits() to set the OPTPERF bits of the nvme_id_ns NSFEAT field instead of the magic number 4.
--- drivers/nvme/target/io-cmd-bdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/nvme/target/io-cmd-bdev.c +2 -2
@@ -28,15 +28,15 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->nawun = lpp0b; id->nawupf = lpp0b; id->nacwu = lpp0b;/* -* Bit 4 indicates that the fields NPWG, NPWA, NPDG, NPDA, and +* OPTPERF = 01b indicates that the fields NPWG, NPWA, NPDG, NPDA, and * NOWS are defined for this namespace and should be used by * the host for I/O optimization. */ - id->nsfeat |=1 <<4 ; + id->nsfeat |=0x1 <<NVME_NS_FEAT_OPTPERF_SHIFT ;/* NPWG = Namespace Preferred Write Granularity. 0's based */ id->npwg =to0based (bdev_io_min (bdev) /bdev_logical_block_size (bdev));/* NPWA = Namespace Preferred Write Alignment. 0's based */ id->npwa = id->npwg;/* NPDG = Namespace Preferred Deallocate Granularity. 0's based */
Signature · 1 line
2.45.2
Re: [PATCH 5/6] nvmet: use NVME_NS_FEAT_OPTPERF_SHIFT
Quoted 3 lines · Caleb Sander Mateos
Use the
NVME_NS_FEAT_OPTPERF_SHIFTconstant innvmet_bdev_set_limits()to set the OPTPERF bits of thenvme_id_nsNSFEAT field instead of the magic number 4.
Looks good:
PATCH 6/6 nvmet: report NPDGL and NPDAL
In reply to Caleb Sander MateosA block device with a very large discard_granularity queue limit may not be able to report it in the 16-bit NPDG and NPDA fields in the Identify Namespace data structure. For this reason, version 2.1 of the NVMe specs added 32-bit fields NPDGL and NPDAL to the NVM Command Set Specific Identify Namespace structure. So report the discard_granularity there too and set OPTPERF to 11b to indicate those fields are supported.
--- drivers/nvme/target/admin-cmd.c | 2 ++ drivers/nvme/target/io-cmd-bdev.c | 19 +++++++++++++++---- drivers/nvme/target/nvmet.h | 2 ++ 3 files changed, 19 insertions(+), 4 deletions(-)
drivers/nvme/target/admin-cmd.c +2 -0
@@ -1056,10 +1056,12 @@ static void nvme_execute_identify_ns_nvm(struct nvmet_req *req) id =kzalloc (sizeof (*id),GFP_KERNEL );if (!id) { status =NVME_SC_INTERNAL ; goto out; } +if (req->ns->bdev) +nvmet_bdev_set_nvm_limits (req->ns->bdev, id); status =nvmet_copy_to_sgl (req,0 , id,sizeof (*id));kfree (id); out:nvmet_req_complete (req, status); }
drivers/nvme/target/io-cmd-bdev.c +15 -4
@@ -28,15 +28,15 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->nawun = lpp0b; id->nawupf = lpp0b; id->nacwu = lpp0b;/* -* OPTPERF = 01b indicates that the fields NPWG, NPWA, NPDG, NPDA, and -* NOWS are defined for this namespace and should be used by -* the host for I/O optimization. +* OPTPERF = 11b indicates that the fields NPWG, NPWA, NPDG, NPDA, +* NPDGL, NPDAL, and NOWS are defined for this namespace and should be +* used by the host for I/O optimization. */ - id->nsfeat |=0x1 <<NVME_NS_FEAT_OPTPERF_SHIFT ; + id->nsfeat |=0x3 <<NVME_NS_FEAT_OPTPERF_SHIFT ;/* NPWG = Namespace Preferred Write Granularity. 0's based */ id->npwg =to0based (bdev_io_min (bdev) /bdev_logical_block_size (bdev));/* NPWA = Namespace Preferred Write Alignment. 0's based */ id->npwa = id->npwg;/* NPDG = Namespace Preferred Deallocate Granularity. 0's based */
@@ -50,10 +50,21 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id)/* Set WZDS and DRB if device supports unmapped write zeroes */ if (bdev_write_zeroes_unmap_sectors (bdev)) id->dlfeat = (1 <<3 ) |0x1 ; } +void nvmet_bdev_set_nvm_limits (struct block_device *bdev, +struct nvme_id_ns_nvm *id) +{ +/* +* NPDGL = Namespace Preferred Deallocate Granularity Large +* NPDAL = Namespace Preferred Deallocate Alignment Large +*/ + id->npdgl = id->npdal =cpu_to_le32 (bdev_discard_granularity (bdev) / +bdev_logical_block_size (bdev)); +} +void nvmet_bdev_ns_disable (struct nvmet_ns *ns) {if (ns->bdev_file) {fput (ns->bdev_file); ns->bdev =NULL ;
drivers/nvme/target/nvmet.h +2 -0
@@ -547,10 +547,12 @@ void nvmet_start_keep_alive_timer(struct nvmet_ctrl *ctrl);void nvmet_stop_keep_alive_timer (struct nvmet_ctrl *ctrl);u16 nvmet_parse_connect_cmd (struct nvmet_req *req);u32 nvmet_connect_cmd_data_len (struct nvmet_req *req);void nvmet_bdev_set_limits (struct block_device *bdev,struct nvme_id_ns *id); +void nvmet_bdev_set_nvm_limits (struct block_device *bdev, +struct nvme_id_ns_nvm *id);u16 nvmet_bdev_parse_io_cmd (struct nvmet_req *req);u16 nvmet_file_parse_io_cmd (struct nvmet_req *req);u16 nvmet_bdev_zns_parse_io_cmd (struct nvmet_req *req);u32 nvmet_admin_cmd_data_len (struct nvmet_req *req);u16 nvmet_parse_admin_cmd (struct nvmet_req *req);
Signature · 1 line
2.45.2
Christoph Hellwig
maintainerRe: [PATCH 6/6] nvmet: report NPDGL and NPDAL
Quoted 6 lines · Caleb Sander Mateos
A block device with a very large
discard_granularityqueue limit may not be able to report it in the 16-bit NPDG and NPDA fields in the Identify Namespace data structure. For this reason, version 2.1 of the NVMe specs added 32-bit fields NPDGL and NPDAL to the NVM Command Set Specific Identify Namespace structure. So report thediscard_granularitythere too and set OPTPERF to 11b to indicate those fields are supported.
From a pure mechanics point of view this looks fine. I'll not that actually aligning the discard to these granularities probably won't really work for most file systems, though.
Christoph Hellwig
maintainer"Improving spec compliance" sounds a bit harsh for just adding support for new optional fields / field extension.