nvmet: FDP bridge for NVMe-oF targets
Subsystems (3)
cover nvmet: FDP bridge for NVMe-oF targets
Flexible Data Placement (FDP) is defined in the NVMe base specification and therefore applies to fabrics as well. An Ethernet Bunch Of Flash (EBOF) style
storage system that exposes its local FDP-capable NVMe drives to remote hosts over NVMe-oF currently has no way to pass the capability through:
nvmet neither advertises FDP nor translates or forwards any FDP command, so a host connected through the target loses the write amplification benefits that the local drives provide.
This series adds an FDP bridge to the NVMe target. A namespace backed by an NVMe namespace opts in through the fdp_enable configfs attribute; the bridge then advertises the FDPS capability, reports the backing FDP configuration through the feature and log pages, translates the placement identifiers of write commands to the backing write streams, and forwards the FDP management commands and log pages to the backing controller. The backing geometry is verified at namespace enable time, so the bridge only comes up when the backing placement identifiers are identity mapped, which the translation relies on.
The FDP commands are served by the bridge directly instead of relying on the existing passthru support, for several reasons:
- Passthru subsystems do not support reservations; the reservation commands are rejected and the controller is not advertised as reservation capable, because the underlying device cannot differentiate the hosts that connect over fabrics. A regular namespace goes through the target's own reservation implementation, which is not available to a passthru subsystem.
- A passthru subsystem is bound to a single backend controller: the whole drive, with all of its namespaces, is passed through as one unit, and no regular namespaces can be added to the subsystem. An EBOF system needs to compose a storage pool out of individual namespaces from multiple backend drives, which only the regular namespace path provides.
- The regular namespace path already handles the target side of reservations, ANA reporting and per-namespace configuration; the bridge extends that path rather than replacing it.
- With passthru, the namespace identity of the backend drive is exposed to the host as is, and the host would talk to the backend controller's FDP state through the passthrough of every command. The bridge instead owns the FDP presentation of the exported namespace, while the state that the backing controller owns is forwarded to it: the reclaim unit handle status hosts read and the reclaim unit handle updates they issue are carried out by the controller that actually manages the media.
The Linux host side needs no changes: the existing FDP discovery path (FDPS bit, endurance group identifier, FDP feature, FDP configurations log page and reclaim unit handle status) works against the bridge unmodified.
The series is structured as follows:
1-2 wire structures and block layer helper 3 nvme_bdev_ops export, used to recognize an NVMe backing device 4 per-namespace bridge state and configuration 5 endurance group identifier in the CS-INDEP identify data 6 FDPS advertisement 7 FDP feature (FID 1Dh) 8 I/O management command forwarding 9 FDP log pages (LID 20h-23h) 10 placement identifier translation on the write path 11 tracepoint for the translation
This was tested with a QEMU FDP backing device through an nvmet-tcp loopback setup: the host discovers and registers the backing write streams over the bridge, writes with placement identifiers land in the reclaim units of the backend device they select (verified through the reclaim unit handle status of the backend), and the forwarded log pages match the backend byte for byte.
Guixin Liu (11):
nvme: add the FDP wire structures for the I/O management send command
block: add bdev_write_stream_granularity()
nvme: export nvme_bdev_ops
nvmet: add per-namespace FDP bridge state and configuration
nvmet: report the Endurance Group Identifier in the CS-INDEP identify
data
nvmet: advertise the FDP capability
nvmet: implement the FDP feature
nvmet: forward the FDP I/O management commands to the backing device
nvmet: serve the FDP log pages
nvmet: translate placement identifiers of writes to write streams
nvmet: add an FDP write translation tracepoint
drivers/nvme/host/core.c | 1 + drivers/nvme/target/Kconfig | 14 + drivers/nvme/target/Makefile | 1 + drivers/nvme/target/admin-cmd.c | 33 +++ drivers/nvme/target/configfs.c | 31 +++ drivers/nvme/target/core.c | 38 +++- drivers/nvme/target/fdp.c | 431 ++++++++++++++++++++++++++++++++++++++ drivers/nvme/target/io-cmd-bdev.c | 8 + drivers/nvme/target/nvmet.h | 93 ++++++++ drivers/nvme/target/trace.h | 23 ++ include/linux/blkdev.h | 8 + include/linux/nvme.h | 24 + 12 files changed, 703 insertions(+), 2 deletions(-)
Signature · 1 line
2.50.0
PATCH 1/11 nvme: add the FDP wire structures for the I/O management send command
Add the command definition of the I/O Management Send command (opcode 0x1D) to the command opcode enum and the command union, together with its management operation, and the log page identifiers of the FDP pages that the target serves, so that initiators and targets share the same wire format.
No in-kernel initiator uses the command yet; the NVMe target FDP support that consumes these structures follows in this series.
--- include/linux/nvme.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
include/linux/nvme.h +24 -0
@@ -967,6 +967,7 @@ enum nvme_opcode { nvme_cmd_resv_acquire =0x11 , nvme_cmd_io_mgmt_recv =0x12 , nvme_cmd_resv_release =0x15 , + nvme_cmd_io_mgmt_send =0x1d , nvme_cmd_zone_mgmt_send =0x79 , nvme_cmd_zone_mgmt_recv =0x7a , nvme_cmd_zone_append = 0x7d,
@@ -989,6 +990,7 @@ enum nvme_opcode {nvme_opcode_name (nvme_cmd_resv_acquire), \nvme_opcode_name (nvme_cmd_io_mgmt_recv), \nvme_opcode_name (nvme_cmd_resv_release), \ +nvme_opcode_name (nvme_cmd_io_mgmt_send), \nvme_opcode_name (nvme_cmd_zone_mgmt_send), \nvme_opcode_name (nvme_cmd_zone_mgmt_recv), \ nvme_opcode_name(nvme_cmd_zone_append))
@@ -1245,6 +1247,23 @@ enum { NVME_IO_MGMT_RECV_MO_RUHS = 1, }; +enum { +NVME_IO_MGMT_SEND_MO_RUH_UPDATE =1 , +}; + +struct nvme_io_mgmt_send_cmd { +__u8 opcode; +__u8 flags; +__le16 command_id; +__le32 nsid; +__le64 rsvd2[2 ]; +union nvme_data_ptr dptr; +__u8 mo; +__u8 rsvd10; +__le16 mos; +__le32 rsvd11[5 ]; +}; +struct nvme_fdp_ruh_status_desc {__le16 pid;__le16 ruhid;
@@ -1416,6 +1435,9 @@ enum {NVME_LOG_FEATURES =0x12 ,NVME_LOG_RMI =0x16 ,NVME_LOG_FDP_CONFIGS =0x20 , +NVME_LOG_FDP_RUH_USAGE =0x21 , +NVME_LOG_FDP_STATS =0x22 , +NVME_LOG_FDP_EVENTS =0x23 ,NVME_LOG_DISC =0x70 ,NVME_LOG_RESERVATION =0x80 , NVME_FWACT_REPL = (0 << 3),
@@ -1437,6 +1459,7 @@ struct nvme_supported_features_log {enum {NVME_FIS_FSUPP =1 <<0 , +NVME_FIS_EGSCPE =1 <<23 ,NVME_FIS_NSCPE =1 <<20 ,NVME_FIS_CSCPE =1 <<21 , };
@@ -2016,6 +2039,7 @@ struct nvme_command {struct nvme_dbbuf dbbuf;struct nvme_directive_cmd directive;struct nvme_io_mgmt_recv_cmd imr; +struct nvme_io_mgmt_send_cmd ims; }; };
Signature · 1 line
2.43.7
PATCH 2/11 block: add bdev_write_stream_granularity()
In reply to Guixin LiuThe write stream granularity queue limit is readable through the queue_limits structure only. Add a block device accessor next to bdev_max_write_streams(), so that holders of a block device can read it without going through the request queue.
--- include/linux/blkdev.h | 8 ++++++++ 1 file changed, 8 insertions(+)
include/linux/blkdev.h +8 -0
@@ -1307,6 +1307,14 @@ static inline unsigned short bdev_max_write_streams(struct block_device *bdev)return bdev_limits (bdev)->max_write_streams; } +static inline unsigned int +bdev_write_stream_granularity (struct block_device *bdev) +{ +if (bdev_is_partition (bdev)) +return 0 ; +return bdev_limits (bdev)->write_stream_granularity; +} +static inline unsigned queue_logical_block_size (const struct request_queue *q) {return q->limits.logical_block_size;
Signature · 1 line
2.43.7
PATCH 3/11 nvme: export nvme_bdev_ops
In reply to Guixin LiuExport the NVMe block device operations so that other subsystems can recognize a block device as an NVMe namespace through its operations pointer and reach the NVMe structures behind it. The NVMe target FDP bridge uses this to detect an NVMe backing device of a namespace.
--- drivers/nvme/host/core.c | 1 + 1 file changed, 1 insertion(+)
drivers/nvme/host/core.c +1 -0
@@ -2712,6 +2712,7 @@ const struct block_device_operations nvme_bdev_ops = { .report_zones = nvme_report_zones, .pr_ops = &nvme_pr_ops, }; +EXPORT_SYMBOL_GPL (nvme_bdev_ops);static int nvme_wait_ready (struct nvme_ctrl *ctrl,u32 mask,u32 val,u32 timeout,const char *op)
Signature · 1 line
2.43.7
PATCH 4/11 nvmet: add per-namespace FDP bridge state and configuration
In reply to Guixin LiuFlexible Data Placement lets a host direct writes into reclaim units through placement identifiers, so that data with different lifetimes stays separated in the media. It is defined in the NVMe base specification and therefore applies to fabrics as well; a storage system that exposes its local FDP-capable NVMe devices to remote hosts over NVMe-oF currently has no way to pass that capability
through: nvmet neither advertises FDP nor forwards any FDP command.
Introduce the per-namespace FDP state that the following bridge patches build on. A namespace opts in through a new fdp_enable configfs attribute, settable only while the namespace is disabled, like resv_enable. When the namespace is enabled, the FDP configuration is snapshotted from the block device write stream limits of the backing device: the number of reclaim unit handles and the reclaim unit nominal size, together with the queues of the backing NVMe namespace, which the FDP command forwarding needs.
The bridge lives behind its own NVME_TARGET_FDP Kconfig option, which depends on NVME_CORE like the passthru support does: it reads the host driver internals behind the block device operations pointer to recognize an NVMe backing device and to reach its controller. The plain NVMe target keeps no dependency on the NVMe host driver.
The NVMe host driver is the only one that reports write streams today, so the enable is rejected when the backing device is not an NVMe namespace. The snapshot is constrained to what the bridge can
serve: the namespace identifier has to fit the 16 bit Endurance Group Identifier it is reported as, and the number of reclaim unit handles is limited to the 8 bit write stream identifiers of the block layer.
--- drivers/nvme/target/Kconfig | 14 ++++++ drivers/nvme/target/Makefile | 1 + drivers/nvme/target/configfs.c | 31 +++++++++++++ drivers/nvme/target/core.c | 12 ++++- drivers/nvme/target/fdp.c | 85 ++++++++++++++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 37 +++++++++++++++ 6 files changed, 179 insertions(+), 1 deletion(-) create mode 100644 drivers/nvme/target/fdp.c
drivers/nvme/target/Kconfig +14 -0
@@ -26,6 +26,20 @@ config NVME_TARGET_DEBUGFS If unsure, say N. +config NVME_TARGET_FDP + bool "NVMe Target Flexible Data Placement bridge support" + depends on NVME_TARGET + depends on NVME_CORE=y || NVME_CORE=NVME_TARGET + help + This enables the target side bridge for the Flexible Data + Placement (FDP) capability: namespaces backed by an FDP capable + NVMe namespace advertise the capability to hosts over NVMe-oF, + translate the placement identifiers of write commands to + backing write streams, and forward the FDP management commands + and log pages to the backing controller. + + If unsure, say N. + config NVME_TARGET_PASSTHRU bool "NVMe Target Passthrough support" depends on NVME_TARGET
drivers/nvme/target/Makefile +1 -0
@@ -13,6 +13,7 @@ obj-$(CONFIG_NVME_TARGET_PCI_EPF) += nvmet-pci-epf.o nvmet-y += core.o configfs.o admin-cmd.o fabrics-cmd.o \ discovery.o io-cmd-file.o io-cmd-bdev.o pr.o nvmet-$(CONFIG_NVME_TARGET_DEBUGFS) += debugfs.o +nvmet-$(CONFIG_NVME_TARGET_FDP) += fdp.o nvmet-$(CONFIG_NVME_TARGET_PASSTHRU) += passthru.o nvmet-$(CONFIG_BLK_DEV_ZONED) += zns.o nvmet-$(CONFIG_NVME_TARGET_AUTH) += fabrics-cmd-auth.o auth.o
drivers/nvme/target/configfs.c +31 -0
@@ -824,6 +824,34 @@ static ssize_t nvmet_ns_resv_enable_store(struct config_item *item, }CONFIGFS_ATTR (nvmet_ns_, resv_enable); +#ifdef CONFIG_NVME_TARGET_FDP +static ssize_t nvmet_ns_fdp_enable_show (struct config_item *item,char *page) +{ +return sysfs_emit (page,"%d\n" ,nvmet_fdp_enabled (to_nvmet_ns (item))); +} + +static ssize_t nvmet_ns_fdp_enable_store (struct config_item *item, +const char *page,size_t count) +{ +struct nvmet_ns *ns =to_nvmet_ns (item); +bool val; + +if (kstrtobool (page, &val)) +return -EINVAL ; + +mutex_lock (&ns->subsys->lock); +if (ns->enabled) { +pr_err ("the ns:%u is already enabled.\n" , ns->nsid); +mutex_unlock (&ns->subsys->lock); +return -EINVAL ; + } + ns->fdp.enable = val; +mutex_unlock (&ns->subsys->lock); +return count; +} +CONFIGFS_ATTR (nvmet_ns_, fdp_enable); +#endif +static struct configfs_attribute *nvmet_ns_attrs[] = { &nvmet_ns_attr_device_path, &nvmet_ns_attr_device_nguid,
@@ -833,6 +861,9 @@ static struct configfs_attribute *nvmet_ns_attrs[] = { &nvmet_ns_attr_buffered_io, &nvmet_ns_attr_revalidate_size, &nvmet_ns_attr_resv_enable, +#ifdef CONFIG_NVME_TARGET_FDP + &nvmet_ns_attr_fdp_enable, +#endif #ifdef CONFIG_PCI_P2PDMA &nvmet_ns_attr_p2pmem, #endif
drivers/nvme/target/core.c +11 -1
@@ -610,9 +610,15 @@ int nvmet_ns_enable(struct nvmet_ns *ns) goto out_dev_put; } +if (nvmet_fdp_enabled (ns)) { + ret =nvmet_fdp_init_ns (ns); +if (ret) + goto out_pr_exit; + } + ret =percpu_ref_init (&ns->ref, nvmet_destroy_namespace,0 ,GFP_KERNEL );if (ret) - goto out_pr_exit; + goto out_fdp_exit;nvmet_ns_changed (subsys, ns->nsid); ns->enabled = true;
@@ -622,6 +628,8 @@ int nvmet_ns_enable(struct nvmet_ns *ns) out_unlock:mutex_unlock (&subsys->lock);return ret; +out_fdp_exit: +nvmet_fdp_exit_ns (ns); out_pr_exit:if (ns->pr.enable)nvmet_pr_exit_ns (ns);
@@ -667,6 +675,8 @@ void nvmet_ns_disable(struct nvmet_ns *ns)if (ns->pr.enable)nvmet_pr_exit_ns (ns); +nvmet_fdp_exit_ns (ns); +mutex_lock (&subsys->lock);nvmet_ns_changed (subsys, ns->nsid);nvmet_ns_dev_disable (ns);
drivers/nvme/target/fdp.c new file +85 -0
85 lines not on this page · raw message
drivers/nvme/target/nvmet.h +37 -0
@@ -97,6 +97,14 @@ struct nvmet_pr_per_ctrl_ref {uuid_t hostid; }; +struct nvmet_fdp { +bool enable; +u16 nruh; +u64 runs; +struct request_queue *backend_q; +struct nvme_ctrl *backend_ctrl; +}; +struct nvmet_ns {struct percpu_ref ref;struct file *bdev_file;
@@ -128,6 +136,9 @@ struct nvmet_ns {u8 csi;struct nvmet_pr pr;struct xarray pr_per_ctrl_refs; +#ifdef CONFIG_NVME_TARGET_FDP +struct nvmet_fdp fdp; +#endif #ifdef CONFIG_NVME_TARGET_DEBUGFS struct dentry *debugfs_dir;#endif
@@ -978,6 +989,32 @@ static inline void nvmet_pr_put_ns_pc_ref(struct nvmet_pr_per_ctrl_ref *pc_ref)percpu_ref_put (&pc_ref->ref); } +#ifdef CONFIG_NVME_TARGET_FDP +static inline bool nvmet_fdp_enabled (struct nvmet_ns *ns) +{ +return ns->fdp.enable; +} +int nvmet_fdp_init_ns (struct nvmet_ns *ns); +void nvmet_fdp_exit_ns (struct nvmet_ns *ns); +bool nvmet_fdp_subsys_configured (struct nvmet_subsys *subsys); +#else +static inline bool nvmet_fdp_enabled (struct nvmet_ns *ns) +{ +return false; +} +static inline int nvmet_fdp_init_ns (struct nvmet_ns *ns) +{ +return -EINVAL ; +} +static inline void nvmet_fdp_exit_ns (struct nvmet_ns *ns) +{ +} +static inline bool nvmet_fdp_subsys_configured (struct nvmet_subsys *subsys) +{ +return false; +} +#endif +/* * Data for the get_feature() and set_feature() operations of PCI target * controllers.
Signature · 1 line
2.43.7
PATCH 5/11 nvmet: report the Endurance Group Identifier in the CS-INDEP identify data
In reply to Guixin LiuThe Identify Namespace data structure (CNS 00h) reports the Endurance Group Identifier of the namespace, but the I/O Command Set Independent Identify Namespace data structure (CNS 08h) leaves the field zeroed. Hosts that implement NVMe 2.0 or later, or that cannot use CNS 00h, take the Endurance Group Identifier from the CS-INDEP data structure and thus always see 0, which breaks the consumers of the identifier:
the FDP feature and log pages, and the Rotational Media Information log page, are addressed by it.
Report the same Endurance Group Identifier as CNS 00h: each namespace is its own endurance group, so the identifier equals the namespace identifier.
--- drivers/nvme/target/admin-cmd.c | 1 + 1 file changed, 1 insertion(+)
drivers/nvme/target/admin-cmd.c +1 -0
@@ -1081,6 +1081,7 @@ static void nvmet_execute_id_cs_indep(struct nvmet_req *req) id->nstat =NVME_NSTAT_NRDY ; id->anagrpid =cpu_to_le32 (req->ns->anagrpid); id->nmic =NVME_NS_NMIC_SHARED ; + id->endgid =cpu_to_le16 (req->ns->nsid);if (req->ns->readonly) id->nsattr |=NVME_NS_ATTR_RO ;if (req->ns->bdev &&bdev_rot (req->ns->bdev))
Signature · 1 line
2.43.7
PATCH 6/11 nvmet: advertise the FDP capability
In reply to Guixin LiuSet the Flexible Data Placement Supported (FDPS) bit in the Identify Controller ctratt field whenever at least one enabled namespace of the subsystem bridges FDP. Hosts use the FDPS bit to decide whether to query the FDP feature and configuration for each namespace, so without it the bridge stays invisible. The FDP log pages are listed in the Supported Log Pages log page by the patches that serve them.
--- drivers/nvme/target/admin-cmd.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/nvme/target/admin-cmd.c +2 -0
@@ -699,6 +699,8 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req) ctratt =NVME_CTRL_ATTR_HID_128_BIT |NVME_CTRL_ATTR_TBKAS ;if (nvmet_is_pci_ctrl (ctrl)) ctratt |=NVME_CTRL_ATTR_RHII ; +if (nvmet_fdp_subsys_configured (ctrl->subsys)) + ctratt |=NVME_CTRL_ATTR_FDPS ; id->ctratt =cpu_to_le32 (ctratt); id->oacs =0 ;
Signature · 1 line
2.43.7
PATCH 7/11 nvmet: implement the FDP feature
In reply to Guixin LiuImplement Set Features and Get Features for FID 1Dh. The Endurance Group Identifier in Command Dword 11 names the namespace directly, since nvmet identifies each namespace as its own endurance group (endgid = nsid). Get Features reports FDPE set with FDPCIDX 0.
Set Features is served locally: the bridge runs the one configuration that the backing device is already in, so setting the feature to that same value is a no-op that succeeds, and any other value is a change of a feature whose endurance group has namespaces, which the spec aborts with Command Sequence Error. That also covers disabling, which would strand the write streams already handed out to hosts.
The namespace lookup takes a namespace reference, so that a namespace being disabled concurrently with the command cannot go away under it. Report the feature in the Features log page under the same per-subsystem condition as the FDPS bit.
--- drivers/nvme/target/admin-cmd.c | 9 ++++ drivers/nvme/target/fdp.c | 77 +++++++++++++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 10 +++++ 3 files changed, 96 insertions(+)
drivers/nvme/target/admin-cmd.c +9 -0
@@ -600,6 +600,9 @@ static void nvmet_execute_get_log_page_features(struct nvmet_req *req)cpu_to_le32 (NVME_FIS_FSUPP |NVME_FIS_NSCPE ); features->fis[NVME_FEAT_RESV_MASK ] =cpu_to_le32 (NVME_FIS_FSUPP |NVME_FIS_NSCPE ); +if (IS_ENABLED (CONFIG_NVME_TARGET_FDP )) + features->fis[NVME_FEAT_FDP ] = +cpu_to_le32 (NVME_FIS_FSUPP |NVME_FIS_EGSCPE ); status =nvmet_copy_to_sgl (req,0 , features,sizeof (*features));kfree (features);
@@ -1385,6 +1388,9 @@ void nvmet_execute_set_features(struct nvmet_req *req)case NVME_FEAT_RESV_MASK : status =nvmet_set_feat_resv_notif_mask (req, cdw11);break ; +case NVME_FEAT_FDP : + status =nvmet_set_feat_fdp (req); +break ;default : req->error_loc = offsetof(struct nvme_common_command , cdw10); status =NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ;
@@ -1562,6 +1568,9 @@ void nvmet_execute_get_features(struct nvmet_req *req)case NVME_FEAT_RESV_MASK : status =nvmet_get_feat_resv_notif_mask (req);break ; +case NVME_FEAT_FDP : + status =nvmet_get_feat_fdp (req); +break ;default : req->error_loc = offsetof(struct nvme_common_command , cdw10);
drivers/nvme/target/fdp.c +77 -0
86 lines not on this page · raw message
drivers/nvme/target/nvmet.h +10 -0
@@ -997,6 +997,8 @@ static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)int nvmet_fdp_init_ns (struct nvmet_ns *ns);void nvmet_fdp_exit_ns (struct nvmet_ns *ns);bool nvmet_fdp_subsys_configured (struct nvmet_subsys *subsys); +u16 nvmet_set_feat_fdp (struct nvmet_req *req); +u16 nvmet_get_feat_fdp (struct nvmet_req *req);#else static inline bool nvmet_fdp_enabled (struct nvmet_ns *ns) {
@@ -1013,6 +1015,14 @@ static inline bool nvmet_fdp_subsys_configured(struct nvmet_subsys *subsys) {return false; } +static inline u16 nvmet_set_feat_fdp (struct nvmet_req *req) +{ +return NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ; +} +static inline u16 nvmet_get_feat_fdp (struct nvmet_req *req) +{ +return NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ; +}#endif /*
Signature · 1 line
2.43.7
PATCH 8/11 nvmet: forward the FDP I/O management commands to the backing device
In reply to Guixin LiuDispatch the I/O Management Receive and Send commands from the I/O command parser, with the transfer length derived from NUMD and NPID, respectively, and report both commands as supported in the Commands Supported and Effects log page.
Both are forwarded to the backing namespace: the data is staged through a linear kernel buffer, which is what the backend submission path takes, so the reclaim unit handle status hosts read reflects the real reclaim units of the backing device, and a Reclaim Unit Handle Update is carried out by the controller that owns that state.
When the namespace is enabled, learn the backing FDP geometry that the forwarding needs: the Endurance Group Identifier the backing FDP state is addressed by, and the placement identifier list of the Reclaim Unit Handle Status. The bridge translates placement identifiers to write streams by their index, so the enable fails when the backing placement identifiers are not identity mapped.
--- drivers/nvme/target/admin-cmd.c | 5 + drivers/nvme/target/core.c | 26 ++++- drivers/nvme/target/fdp.c | 177 +++++++++++++++++++++++++++++++- drivers/nvme/target/nvmet.h | 12 +++ 4 files changed, 217 insertions(+), 3 deletions(-)
drivers/nvme/target/admin-cmd.c +5 -0
@@ -394,6 +394,11 @@ static void nvmet_get_cmd_effects_nvm(struct nvme_effects_log *log) log->iocs[nvme_cmd_write] = log->iocs[nvme_cmd_write_zeroes] =cpu_to_le32 (NVME_CMD_EFFECTS_CSUPP |NVME_CMD_EFFECTS_LBCC ); +if (IS_ENABLED (CONFIG_NVME_TARGET_FDP )) { + log->iocs[nvme_cmd_io_mgmt_recv] = + log->iocs[nvme_cmd_io_mgmt_send] = +cpu_to_le32 (NVME_CMD_EFFECTS_CSUPP ); + } }static void nvmet_get_cmd_effects_zns (struct nvme_effects_log *log)
drivers/nvme/target/core.c +25 -1
@@ -1097,9 +1097,22 @@ static u32 nvmet_io_cmd_transfer_len(struct nvmet_req *req)return nvmet_dsm_len (req);case nvme_cmd_zone_mgmt_recv:return (le32_to_cpu (req->cmd->zmr.numd) +1 ) <<2 ; +case nvme_cmd_io_mgmt_recv: +if (IS_ENABLED (CONFIG_NVME_TARGET_FDP )) +return (le32_to_cpu (req->cmd->imr.numd) +1 ) <<2 ; +break ; +case nvme_cmd_io_mgmt_send: +if (IS_ENABLED (CONFIG_NVME_TARGET_FDP )) { +u32 npid =le16_to_cpu (req->cmd->ims.mos); + +return (npid +1 ) *sizeof (__le16); + } +break ;default : -return 0 ; +break ; } + +return 0 ; }static u16 nvmet_parse_io_cmd (struct nvmet_req *req)
@@ -1141,6 +1154,17 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)return ret; } +if (IS_ENABLED (CONFIG_NVME_TARGET_FDP )) { +switch (req->cmd->common.opcode) { +case nvme_cmd_io_mgmt_recv: + req->execute = nvmet_execute_io_mgmt_recv; +return 0 ; +case nvme_cmd_io_mgmt_send: + req->execute = nvmet_execute_io_mgmt_send; +return 0 ; + } + } +switch (req->ns->csi) {case NVME_CSI_NVM :if (req->ns->file)
drivers/nvme/target/fdp.c +175 -2
199 lines not on this page · raw message
drivers/nvme/target/nvmet.h +12 -0
@@ -103,6 +103,8 @@ struct nvmet_fdp {u64 runs;struct request_queue *backend_q;struct nvme_ctrl *backend_ctrl; +u32 backend_nsid; +u16 backend_endgid; };struct nvmet_ns {
@@ -999,6 +1001,8 @@ void nvmet_fdp_exit_ns(struct nvmet_ns *ns);bool nvmet_fdp_subsys_configured (struct nvmet_subsys *subsys);u16 nvmet_set_feat_fdp (struct nvmet_req *req);u16 nvmet_get_feat_fdp (struct nvmet_req *req); +void nvmet_execute_io_mgmt_recv (struct nvmet_req *req); +void nvmet_execute_io_mgmt_send (struct nvmet_req *req);#else static inline bool nvmet_fdp_enabled (struct nvmet_ns *ns) {
@@ -1023,6 +1027,14 @@ static inline u16 nvmet_get_feat_fdp(struct nvmet_req *req) {return NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ; } +static inline void nvmet_execute_io_mgmt_recv (struct nvmet_req *req) +{ +nvmet_req_complete (req,NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ); +} +static inline void nvmet_execute_io_mgmt_send (struct nvmet_req *req) +{ +nvmet_req_complete (req,NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ); +}#endif /*
Signature · 1 line
2.43.7
PATCH 9/11 nvmet: serve the FDP log pages
In reply to Guixin LiuThe bridge advertises the FDP log pages 20h to 23h as supported; serve them.
The FDP Configurations log page (LID 20h) is constructed from the snapshot taken at namespace enable time: one valid configuration with a single reclaim group, the backing number of reclaim unit handles, all of the Initially Isolated type, and the backing reclaim unit nominal size. The page is addressed by the Endurance Group Identifier in the Log Specific Identifier field. Hosts first request the 16 byte header to learn the page size and then the full page; both request sizes are served from the same construction path.
The remaining pages are owned by the backing controller and are forwarded to it over its admin queue, addressed by the Endurance Group Identifier of the backing endurance group, so hosts read the real reclaim unit handle usage, counters and events of the backing device.
--- drivers/nvme/target/admin-cmd.c | 16 ++++++ drivers/nvme/target/fdp.c | 98 ++++++++++++++++++++++++++++++++- drivers/nvme/target/nvmet.h | 14 +++++ 3 files changed, 127 insertions(+), 1 deletion(-)
drivers/nvme/target/admin-cmd.c +16 -0
@@ -220,6 +220,16 @@ static void nvmet_execute_get_supported_log_pages(struct nvmet_req *req) logs->lids[NVME_LOG_FEATURES ] =cpu_to_le32 (NVME_LIDS_LSUPP ); logs->lids[NVME_LOG_RMI ] =cpu_to_le32 (NVME_LIDS_LSUPP ); logs->lids[NVME_LOG_RESERVATION ] =cpu_to_le32 (NVME_LIDS_LSUPP ); +if (nvmet_fdp_subsys_configured (nvmet_req_subsys (req))) { + logs->lids[NVME_LOG_FDP_CONFIGS ] = +cpu_to_le32 (NVME_LIDS_LSUPP ); + logs->lids[NVME_LOG_FDP_RUH_USAGE ] = +cpu_to_le32 (NVME_LIDS_LSUPP ); + logs->lids[NVME_LOG_FDP_STATS ] = +cpu_to_le32 (NVME_LIDS_LSUPP ); + logs->lids[NVME_LOG_FDP_EVENTS ] = +cpu_to_le32 (NVME_LIDS_LSUPP ); + } status =nvmet_copy_to_sgl (req,0 , logs,sizeof (*logs));kfree (logs);
@@ -648,6 +658,12 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)return nvmet_execute_get_log_page_rmi (req);case NVME_LOG_RESERVATION :return nvmet_execute_get_log_page_resv (req); +case NVME_LOG_FDP_CONFIGS : +return nvmet_execute_get_log_page_fdp_configs (req); +case NVME_LOG_FDP_RUH_USAGE : +case NVME_LOG_FDP_STATS : +case NVME_LOG_FDP_EVENTS : +return nvmet_fdp_log_forward (req); }pr_debug ("unhandled lid %d on qid %d\n" , req->cmd->get_log_page.lid, req->sq->qid);
drivers/nvme/target/fdp.c +97 -1
107 lines not on this page · raw message
drivers/nvme/target/nvmet.h +14 -0
@@ -1003,6 +1003,8 @@ u16 nvmet_set_feat_fdp(struct nvmet_req *req);u16 nvmet_get_feat_fdp (struct nvmet_req *req);void nvmet_execute_io_mgmt_recv (struct nvmet_req *req);void nvmet_execute_io_mgmt_send (struct nvmet_req *req); +void nvmet_execute_get_log_page_fdp_configs (struct nvmet_req *req); +void nvmet_fdp_log_forward (struct nvmet_req *req);#else static inline bool nvmet_fdp_enabled (struct nvmet_ns *ns) {
@@ -1035,6 +1037,18 @@ static inline void nvmet_execute_io_mgmt_send(struct nvmet_req *req) {nvmet_req_complete (req,NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ); } +static inline void nvmet_execute_get_log_page_fdp_configs (struct nvmet_req *req) +{ +nvmet_req_complete (req,NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ); +} +static inline void nvmet_fdp_log_forward (struct nvmet_req *req) +{ +nvmet_req_complete (req,NVME_SC_INVALID_FIELD |NVME_STATUS_DNR ); +} +static inline u8 nvmet_fdp_write_stream (struct nvmet_req *req) +{ +return 0 ; +}#endif /*
Signature · 1 line
2.43.7
PATCH 10/11 nvmet: translate placement identifiers of writes to write streams
In reply to Guixin LiuA write command that carries the Data Placement Directive names a placement identifier in the Directive Specific field. Both sides of the bridge use identity mapped handles: the write streams of the backing device map to the placement handles one by one, in the order the reclaim unit handle status reports them. Translate the placement identifier to the matching backing write stream on the write path, so the data lands in the reclaim unit of the backing device that the host selected. The enable of the namespace has verified that the backing placement identifiers are identity mapped.
Writes without the directive, and writes with a placement identifier that is out of range, keep the default stream 0: the spec lets the controller place such data itself.
--- drivers/nvme/target/io-cmd-bdev.c | 4 ++++ drivers/nvme/target/nvmet.h | 20 ++++++++++++++++++++ 2 files changed, 24 insertions(+)
drivers/nvme/target/io-cmd-bdev.c +4 -0
@@ -300,6 +300,10 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req) bio->bi_iter.bi_sector = sector; bio->bi_private = req; bio->bi_end_io = nvmet_bio_done; +if (req->cmd->rw.opcode == nvme_cmd_write && +nvmet_fdp_enabled (req->ns)) { + bio->bi_write_stream =nvmet_fdp_write_stream (req); + }blk_start_plug (&plug);if (req->metadata_len)
drivers/nvme/target/nvmet.h +20 -0
@@ -1005,6 +1005,26 @@ void nvmet_execute_io_mgmt_recv(struct nvmet_req *req);void nvmet_execute_io_mgmt_send (struct nvmet_req *req);void nvmet_execute_get_log_page_fdp_configs (struct nvmet_req *req);void nvmet_fdp_log_forward (struct nvmet_req *req); + +/* +* Both sides of the bridge use identity mapped handles, so placement +* identifier i routes to backing write stream i+1; stream 0, the +* default, places the data without a directive. +*/ +static inline u8 nvmet_fdp_write_stream (struct nvmet_req *req) +{ +u16 control =le16_to_cpu (req->cmd->rw.control); +u16 pid; + +if ((control &0xf0 ) != (NVME_RW_DTYPE_DPLCMT &0xf0 )) +return 0 ; + + pid =le32_to_cpu (req->cmd->rw.dsmgmt) >>16 ; +if (pid >= req->ns->fdp.nruh) +return 0 ; + +return pid +1 ; +}#else static inline bool nvmet_fdp_enabled (struct nvmet_ns *ns) {
Signature · 1 line
2.43.7
PATCH 11/11 nvmet: add an FDP write translation tracepoint
In reply to Guixin LiuAdd a tracepoint to the write path that reports the placement identifier carried by a write command with the Data Placement Directive and the backing write stream it is translated to.
--- drivers/nvme/target/io-cmd-bdev.c | 4 ++++ drivers/nvme/target/trace.h | 23 +++++++++++++++++++++++ 2 files changed, 27 insertions(+)
drivers/nvme/target/io-cmd-bdev.c +4 -0
@@ -9,6 +9,7 @@#include <linux/memremap.h> #include <linux/module.h> #include "nvmet.h" +#include "trace.h" void nvmet_bdev_set_limits (struct block_device *bdev,struct nvme_id_ns *id) {
@@ -302,7 +303,10 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req) bio->bi_end_io = nvmet_bio_done;if (req->cmd->rw.opcode == nvme_cmd_write &&nvmet_fdp_enabled (req->ns)) { +u16 pid =le32_to_cpu (req->cmd->rw.dsmgmt) >>16 ; + bio->bi_write_stream =nvmet_fdp_write_stream (req); +trace_nvmet_fdp_write_translate (req, pid, bio->bi_write_stream); }blk_start_plug (&plug);
drivers/nvme/target/trace.h +23 -0
@@ -131,6 +131,29 @@ TRACE_EVENT(nvmet_req_complete, ); +TRACE_EVENT (nvmet_fdp_write_translate, + TP_PROTO(struct nvmet_req *req, u16 pid, u8 stream), +TP_ARGS (req, pid, stream), +TP_STRUCT__entry ( + __field(u16, ctrl_id) + __array(char, disk, DISK_NAME_LEN) + __field(u32, nsid) + __field(u16, pid) +__field (u8, stream) + ), +TP_fast_assign ( + __entry->ctrl_id = nvmet_req_to_ctrl_id(req); + __assign_req_name(__entry->disk, req); + __entry->nsid = req->ns->nsid; + __entry->pid = pid; + __entry->stream = stream; + ), +TP_printk ("nvmet%s: %snsid=%u, pid=%u, write_stream=%u" , +__print_ctrl_id (__entry->ctrl_id), +__print_disk_name (__entry->disk), + __entry->nsid, __entry->pid, __entry->stream) +); +#define aer_name (aer) { aer, #aer } TRACE_EVENT(nvmet_async_event,
Signature · 1 line
2.43.7