Skip to contentSkip to conversation

nvmet: FDP bridge for NVMe-oF targets

PATCH In review Guixin Liu 2 participants 17 messages Oct 29 – Nov 16

Subsystems (3)
lore
Share & tools
lore

Guixin Liu

author
#0 lore raw

v3 cover nvmet: FDP bridge for NVMe-oF targets

Flexible Data Placement (FDP) is defined in the NVMe base specification and therefore applies to fabrics as well. An Ethernet Bunch Of Flash (EBOF) style storage system that exposes its local FDP-capable NVMe drives to remote hosts over NVMe-oF currently has no way to pass that capability through: nvmet neither advertises FDP nor translates or forwards any FDP command, so a host connected through the target loses the write amplification benefits that the local drives provide.

This series adds an FDP bridge to the NVMe target. A namespace backed by an NVMe namespace opts in through the fdp_enable configfs attribute; the bridge then advertises the FDPS capability, reports the backing FDP configuration through the feature and log pages, translates the placement identifiers of write commands to the backing write streams, and forwards the FDP management commands and log pages to the backing controller. The backing geometry is verified at namespace enable time, so the bridge only comes up when the backing placement identifiers are identity mapped, which the translation relies on.

The FDP commands are served by the bridge directly instead of relying on the existing passthru support, for several reasons:

- Passthru subsystems do not support reservations; the reservation commands are rejected and the controller is not advertised as reservation capable, because the underlying device cannot differentiate the hosts that connect over fabrics. A regular namespace goes through the target's own reservation implementation, which is not available to a passthru subsystem.

- A passthru subsystem is bound to a single backend controller: the whole drive, with all of its namespaces, is passed through as one unit, and no regular namespaces can be added to the subsystem. An EBOF system needs to compose a storage pool out of individual namespaces from multiple backend drives, which only the regular namespace path provides.

- The regular namespace path already handles the target side of reservations, ANA reporting and per-namespace configuration; the bridge extends that path rather than replacing it.

- With passthru, the namespace identity of the backend drive is exposed to the host as is, and the host would talk to the backend controller's FDP state through the passthrough of every command. The bridge instead owns the FDP presentation of the exported namespace, while the state that the backing controller owns is forwarded to it: the reclaim unit handle status hosts read and the reclaim unit handle updates they issue are carried out by the controller that actually manages the media.

The Linux host side needs no changes: the existing FDP discovery path (FDPS bit, endurance group identifier, FDP feature, FDP configurations log page and reclaim unit handle status) works against the bridge unmodified.

The series is structured as follows:

  1-2   wire structures and block layer helper
  3     nvme_bdev_ops export, used to recognize an NVMe backing device
  4     per-namespace bridge state and configuration
  5     fdp_enable ABI documentation
  6     endurance group identifier in the CS-INDEP identify data
  7     FDPS advertisement
  8     FDP feature (FID 1Dh)
  9     I/O management command forwarding
  10    FDP log pages (LID 20h-23h)
  11    placement identifier translation on the write path
  12    tracepoint for the translation

This was tested with a QEMU FDP backing device through an nvmet-tcp loopback setup, with persistent reservations enabled on the exported
namespace: the host discovers and registers the backing write streams over the bridge, writes with placement identifiers land in the reclaim units of the backend device they select (verified through the reclaim unit handle status of the backend, for single bio and chained bio writes alike), and the forwarded log pages match the backend byte for byte. Short and offset reads of the configurations log match the full page, and Set Features on a namespace without FDP accepts the unchanged disabled configuration. I/O management receive requests whose NUMD arithmetic would wrap, zero-length forwards, and requests above the 1 MiB staging bound are rejected with Invalid Field. A second namespace without FDP in the same subsystem comes up normally and reports FDPE clear. With a second controller registered, the FDP commands are served while a write exclusive reservation is held, writes from the non-holder conflict, and preempt-and-abort completes with I/O management commands in flight.

Changes since v2:

https://lore.kernel.org/all/[email protected]/

- Take the namespace reference of the FDP commands under NVMET_NS_IO_LIVE with percpu_ref_tryget_live_rcu(), closing a use-after-free against a concurrently disabled namespace.
- Report FDPE clear from Get Features for namespaces without FDP, so that a subsystem mixing bridged and regular namespaces does not fail the host's per-namespace discovery; Set Features accepts that same disabled configuration as a no-op.
- Evaluate the I/O management receive transfer length in 64 bits instead of letting the 32-bit NUMD arithmetic wrap, bound all staged FDP transfers to 1 MiB, and reject zero-length forwards.
- Re-encode NUMD of the forwarded log pages from the staged transfer length instead of carrying over the host supplied value, and clear the reserved metadata field of the forwarded commands.
- Fall through to the persistent reservation access checks for the I/O management commands instead of returning before them.
- Require the backing reclaim unit handle count to match the write streams exactly instead of accepting a larger count.
- Read the backing FDP configurations log at enable time so the advertised handle types and the 64-bit reclaim unit nominal size are the backing device's own values, selected by the active FDPCIDX rather than by descriptor order, and validate every descriptor before dereferencing it.
- Serve short reads and nonzero Log Page Offsets of the configurations log as windows of the complete page.
- Run the FDP command handlers from a workqueue so that lpfc cannot sleep in hardirq context.
- Track the FDP capability as a per-subsystem count, incremented at namespace enable and decremented at namespace disable.
- Advertise the I/O management commands in the command-effects log only when the subsystem actually bridges FDP, and set error_loc for invalid FDP feature requests.
- Document fdp_enable in the ABI file.

Changes since v1:

https://lore.kernel.org/all/[email protected]/

- Verify the transfer length encoded in the I/O management commands against the length the host supplied, and do not carry the host data pointer over to the backing command.
- Assign the translated write stream to every bio of a write, not just the first one.
- Run the namespace lookup of the FDP commands under RCU.
- Report the FDP feature in the Features log page under the same per-subsystem condition as the FDPS bit.
- Only emit the write translation tracepoint for writes that carry the Data Placement Directive.
- Dispatch the I/O management commands after the persistent reservation checks.
- Report backing command failures during the namespace enable as errno, and clean up the backend state when the enable fails.
- Drop the Assisted-by: LLM tag from all commits.

Guixin Liu (12):
nvme: add the FDP wire structures for the I/O management send command
block: add bdev_write_stream_granularity()
nvme: export nvme_bdev_ops
nvmet: add per-namespace FDP bridge state and configuration
nvmet: document the fdp_enable configfs attribute
nvmet: report the Endurance Group Identifier in the CS-INDEP identify
data
nvmet: advertise the FDP capability
nvmet: implement the FDP feature
nvmet: forward the FDP I/O management commands to the backing device
nvmet: serve the FDP log pages
nvmet: translate placement identifiers of writes to write streams
nvmet: add an FDP write translation tracepoint

 Documentation/ABI/stable/configfs-nvmet |  13 +
 drivers/nvme/host/core.c                |   1 +
 drivers/nvme/target/Kconfig             |  14 +
 drivers/nvme/target/Makefile            |   1 +
 drivers/nvme/target/admin-cmd.c         |  45 +-
 drivers/nvme/target/configfs.c          |  32 ++
 drivers/nvme/target/core.c              |  43 +-
 drivers/nvme/target/fdp.c               | 682 ++++++++++++++++++++++++
 drivers/nvme/target/io-cmd-bdev.c       |  15 +
 drivers/nvme/target/nvmet.h             |  91 ++++
 drivers/nvme/target/trace.h             |  23 +
 include/linux/blkdev.h                  |   8 +
 include/linux/nvme.h                    |  24 +
 13 files changed, 987 insertions(+), 5 deletions(-)
 create mode 100644 drivers/nvme/target/fdp.c
Signature · 1 line
2.43.7
author
#1 lore raw

PATCH v3 1/12 nvme: add the FDP wire structures for the I/O management send command

Add the command definition of the I/O Management Send command (opcode 0x1D) to the command opcode enum and the command union, together with its management operation, and the log page identifiers of the FDP pages that the target serves, so that initiators and targets share the same wire format.

No in-kernel initiator uses the command yet, but the FDP support that consumes these structures is following.

---
 include/linux/nvme.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
include/linux/nvme.h +24 -0
index 6248711784b3..ad77af010d37 100644
@@ -967,6 +967,7 @@ enum nvme_opcode {
 	nvme_cmd_resv_acquire	= 0x11,
 	nvme_cmd_io_mgmt_recv	= 0x12,
 	nvme_cmd_resv_release	= 0x15,
+	nvme_cmd_io_mgmt_send	= 0x1d,
 	nvme_cmd_zone_mgmt_send	= 0x79,
 	nvme_cmd_zone_mgmt_recv	= 0x7a,
 	nvme_cmd_zone_append	= 0x7d,
@@ -989,6 +990,7 @@ enum nvme_opcode {
 		nvme_opcode_name(nvme_cmd_resv_acquire),	\
 		nvme_opcode_name(nvme_cmd_io_mgmt_recv),	\
 		nvme_opcode_name(nvme_cmd_resv_release),	\
+		nvme_opcode_name(nvme_cmd_io_mgmt_send),	\
 		nvme_opcode_name(nvme_cmd_zone_mgmt_send),	\
 		nvme_opcode_name(nvme_cmd_zone_mgmt_recv),	\
 		nvme_opcode_name(nvme_cmd_zone_append))
@@ -1245,6 +1247,23 @@ enum {
 	NVME_IO_MGMT_RECV_MO_RUHS	= 1,
 };
 
+enum {
+	NVME_IO_MGMT_SEND_MO_RUH_UPDATE	= 1,
+};
+
+struct nvme_io_mgmt_send_cmd {
+	__u8			opcode;
+	__u8			flags;
+	__le16			command_id;
+	__le32			nsid;
+	__le64			rsvd2[2];
+	union nvme_data_ptr	dptr;
+	__u8			mo;
+	__u8			rsvd10;
+	__le16			mos;
+	__le32			rsvd11[5];
+};
+
 struct nvme_fdp_ruh_status_desc {
 	__le16			pid;
 	__le16			ruhid;
@@ -1416,6 +1435,9 @@ enum {
 	NVME_LOG_FEATURES	= 0x12,
 	NVME_LOG_RMI		= 0x16,
 	NVME_LOG_FDP_CONFIGS	= 0x20,
+	NVME_LOG_FDP_RUH_USAGE	= 0x21,
+	NVME_LOG_FDP_STATS	= 0x22,
+	NVME_LOG_FDP_EVENTS	= 0x23,
 	NVME_LOG_DISC		= 0x70,
 	NVME_LOG_RESERVATION	= 0x80,
 	NVME_FWACT_REPL		= (0 << 3),
@@ -1437,6 +1459,7 @@ struct nvme_supported_features_log {
 
 enum {
 	NVME_FIS_FSUPP	= 1 << 0,
+	NVME_FIS_EGSCPE	= 1 << 23,
 	NVME_FIS_NSCPE	= 1 << 20,
 	NVME_FIS_CSCPE	= 1 << 21,
 };
@@ -2016,6 +2039,7 @@ struct nvme_command {
 		struct nvme_dbbuf dbbuf;
 		struct nvme_directive_cmd directive;
 		struct nvme_io_mgmt_recv_cmd imr;
+		struct nvme_io_mgmt_send_cmd ims;
 	};
 };
 
Signature · 1 line
2.43.7
author
#2 lore raw

PATCH v3 2/12 block: add bdev_write_stream_granularity()

In reply to Guixin Liu

The write stream granularity queue limit is readable through the queue_limits structure only. Add a block device accessor next to bdev_max_write_streams(), so that holders of a block device can read it without going through the request queue.

---
 include/linux/blkdev.h | 8 ++++++++
 1 file changed, 8 insertions(+)
include/linux/blkdev.h +8 -0
index 4f7905c3412b..6eca09058dec 100644
@@ -1307,6 +1307,14 @@ static inline unsigned short bdev_max_write_streams(struct block_device *bdev)
 	return bdev_limits(bdev)->max_write_streams;
 }
 
+static inline unsigned int
+bdev_write_stream_granularity(struct block_device *bdev)
+{
+	if (bdev_is_partition(bdev))
+		return 0;
+	return bdev_limits(bdev)->write_stream_granularity;
+}
+
 static inline unsigned queue_logical_block_size(const struct request_queue *q)
 {
 	return q->limits.logical_block_size;
Signature · 1 line
2.43.7
author
#3 lore raw

PATCH v3 3/12 nvme: export nvme_bdev_ops

In reply to Guixin Liu

Export the NVMe block device operations so that other subsystems can recognize a block device as an NVMe namespace through its operations pointer and reach the NVMe structures behind it. The NVMe target FDP bridge uses this to detect an NVMe backing device of a namespace.

---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvme/host/core.c +1 -0
index 7eb0f9fe6b18..0099ce5730b2 100644
@@ -2719,6 +2719,7 @@ const struct block_device_operations nvme_bdev_ops = {
 	.report_zones	= nvme_report_zones,
 	.pr_ops		= &nvme_pr_ops,
 };
+EXPORT_SYMBOL_GPL(nvme_bdev_ops);
 
 static int nvme_wait_ready(struct nvme_ctrl *ctrl, u32 mask, u32 val,
 		u32 timeout, const char *op)
Signature · 1 line
2.43.7
author
#4 lore raw

PATCH v3 4/12 nvmet: add per-namespace FDP bridge state and configuration

In reply to Guixin Liu

Flexible Data Placement lets a host direct writes into reclaim units through placement identifiers, so that data with different lifetimes stays separated in the media. It is defined in the NVMe base specification and therefore applies to fabrics as well; a storage system that exposes its local FDP-capable NVMe devices to remote hosts over NVMe-oF currently has no way to pass that capability
through: nvmet neither advertises FDP nor forwards any FDP command.

Introduce the per-namespace FDP state that the following bridge patches build on. A namespace opts in through a new fdp_enable configfs attribute, settable only while the namespace is disabled, like resv_enable. When the namespace is enabled, the FDP configuration is snapshotted from the block device write stream limits of the backing device: the number of reclaim unit handles and the reclaim unit nominal size, together with the queues of the backing NVMe namespace, which the FDP command forwarding needs.

The bridge lives behind its own NVME_TARGET_FDP Kconfig option, which depends on NVME_CORE like the passthru support does: it reads the host driver internals behind the block device operations pointer to recognize an NVMe backing device and to reach its controller. The plain NVMe target keeps no dependency on the NVMe host driver.

The NVMe host driver is the only one that reports write streams today, so the enable is rejected when the backing device is not an NVMe namespace. The snapshot is constrained to what the bridge can
serve: the namespace identifier has to fit the 16 bit Endurance Group Identifier it is reported as, and the number of reclaim unit handles is limited to the 8 bit write stream identifiers of the block layer.

---
 drivers/nvme/target/Kconfig    | 14 +++++++
 drivers/nvme/target/Makefile   |  1 +
 drivers/nvme/target/configfs.c | 32 ++++++++++++++++
 drivers/nvme/target/core.c     | 16 +++++++-
 drivers/nvme/target/fdp.c      | 67 ++++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h    | 36 ++++++++++++++++++
 6 files changed, 165 insertions(+), 1 deletion(-)
 create mode 100644 drivers/nvme/target/fdp.c
drivers/nvme/target/Kconfig +14 -0
index 69bde270115e..974878e345cb 100644
@@ -26,6 +26,20 @@ config NVME_TARGET_DEBUGFS
 
 	  If unsure, say N.
 
+config NVME_TARGET_FDP
+	bool "NVMe Target Flexible Data Placement bridge support"
+	depends on NVME_TARGET
+	depends on NVME_CORE=y || NVME_CORE=NVME_TARGET
+	help
+	  This enables the target side bridge for the Flexible Data
+	  Placement (FDP) capability: namespaces backed by an FDP capable
+	  NVMe namespace advertise the capability to hosts over NVMe-oF,
+	  translate the placement identifiers of write commands to
+	  backing write streams, and forward the FDP management commands
+	  and log pages to the backing controller.
+
+	  If unsure, say N.
+
 config NVME_TARGET_PASSTHRU
 	bool "NVMe Target Passthrough support"
 	depends on NVME_TARGET
drivers/nvme/target/Makefile +1 -0
index ed8522911d1f..94814e015c2f 100644
@@ -13,6 +13,7 @@ obj-$(CONFIG_NVME_TARGET_PCI_EPF)	+= nvmet-pci-epf.o
 nvmet-y		+= core.o configfs.o admin-cmd.o fabrics-cmd.o \
 			discovery.o io-cmd-file.o io-cmd-bdev.o pr.o
 nvmet-$(CONFIG_NVME_TARGET_DEBUGFS)	+= debugfs.o
+nvmet-$(CONFIG_NVME_TARGET_FDP)		+= fdp.o
 nvmet-$(CONFIG_NVME_TARGET_PASSTHRU)	+= passthru.o
 nvmet-$(CONFIG_BLK_DEV_ZONED)		+= zns.o
 nvmet-$(CONFIG_NVME_TARGET_AUTH)	+= fabrics-cmd-auth.o auth.o
drivers/nvme/target/configfs.c +32 -0
index b03b5d1c2dc3..732fce67c594 100644
@@ -827,6 +827,35 @@ static ssize_t nvmet_ns_resv_enable_store(struct config_item *item,
 }
 CONFIGFS_ATTR(nvmet_ns_, resv_enable);
 
+#ifdef CONFIG_NVME_TARGET_FDP
+static ssize_t nvmet_ns_fdp_enable_show(struct config_item *item,
+					char *page)
+{
+	return sysfs_emit(page, "%d\n", nvmet_fdp_enabled(to_nvmet_ns(item)));
+}
+
+static ssize_t nvmet_ns_fdp_enable_store(struct config_item *item,
+		const char *page, size_t count)
+{
+	struct nvmet_ns *ns = to_nvmet_ns(item);
+	bool val;
+
+	if (kstrtobool(page, &val))
+		return -EINVAL;
+
+	mutex_lock(&ns->subsys->lock);
+	if (ns->enabled) {
+		pr_err("the ns:%u is already enabled.\n", ns->nsid);
+		mutex_unlock(&ns->subsys->lock);
+		return -EINVAL;
+	}
+	ns->fdp.enable = val;
+	mutex_unlock(&ns->subsys->lock);
+	return count;
+}
+CONFIGFS_ATTR(nvmet_ns_, fdp_enable);
+#endif
+
 static struct configfs_attribute *nvmet_ns_attrs[] = {
 	&nvmet_ns_attr_device_path,
 	&nvmet_ns_attr_device_nguid,
@@ -836,6 +865,9 @@ static struct configfs_attribute *nvmet_ns_attrs[] = {
 	&nvmet_ns_attr_buffered_io,
 	&nvmet_ns_attr_revalidate_size,
 	&nvmet_ns_attr_resv_enable,
+#ifdef CONFIG_NVME_TARGET_FDP
+	&nvmet_ns_attr_fdp_enable,
+#endif
 #ifdef CONFIG_PCI_P2PDMA
 	&nvmet_ns_attr_p2pmem,
 #endif
drivers/nvme/target/core.c +15 -1
index 07a74c2e1f58..14e2a908857c 100644
@@ -622,19 +622,29 @@ int nvmet_ns_enable(struct nvmet_ns *ns)
 			goto out_dev_put;
 	}
 
+	if (nvmet_fdp_enabled(ns)) {
+		ret = nvmet_fdp_init_ns(ns);
+		if (ret)
+			goto out_pr_exit;
+	}
+
 	ret = percpu_ref_init(&ns->ref, nvmet_destroy_namespace, 0, GFP_KERNEL);
 	if (ret)
-		goto out_pr_exit;
+		goto out_fdp_exit;
 
 	nvmet_ns_changed(subsys, ns->nsid);
 	ns->enabled = true;
 	xa_set_mark(&subsys->namespaces, ns->nsid, NVMET_NS_ENABLED);
 	nvmet_debugfs_ns_setup(ns);
 	set_bit(NVMET_NS_IO_LIVE, &ns->flags);
+	if (nvmet_fdp_enabled(ns))
+		subsys->nr_ns_fdp_enabled++;
 	ret = 0;
 out_unlock:
 	mutex_unlock(&subsys->lock);
 	return ret;
+out_fdp_exit:
+	nvmet_fdp_exit_ns(ns);
 out_pr_exit:
 	if (ns->pr.enable)
 		nvmet_pr_exit_ns(ns);
@@ -680,10 +690,14 @@ void nvmet_ns_disable(struct nvmet_ns *ns)
 	if (ns->pr.enable)
 		nvmet_pr_exit_ns(ns);
 
+	nvmet_fdp_exit_ns(ns);
+
 	mutex_lock(&subsys->lock);
 	nvmet_ns_changed(subsys, ns->nsid);
 	nvmet_ns_dev_disable(ns);
 	ns->enabled = false;
+	if (nvmet_fdp_enabled(ns))
+		subsys->nr_ns_fdp_enabled--;
 	mutex_unlock(&subsys->lock);
 }
 
drivers/nvme/target/fdp.c new file +67 -0

67 lines not on this page · raw message

drivers/nvme/target/nvmet.h +36 -0
index 162e2fdd848e..81defb4abfdd 100644
@@ -97,6 +97,17 @@ struct nvmet_pr_per_ctrl_ref {
 	uuid_t			hostid;
 };
 
+struct nvmet_fdp {
+	bool			enable;
+	u16			nruh;
+	u64			runs;
+	u8			ruht[U8_MAX];
+	struct request_queue	*backend_q;
+	struct nvme_ctrl	*backend_ctrl;
+	u32			backend_nsid;
+	u16			backend_endgid;
+};
+
 struct nvmet_ns {
 	struct percpu_ref	ref;
 	struct file		*bdev_file;
@@ -130,6 +141,9 @@ struct nvmet_ns {
 	u8			csi;
 	struct nvmet_pr		pr;
 	struct xarray		pr_per_ctrl_refs;
+#ifdef CONFIG_NVME_TARGET_FDP
+	struct nvmet_fdp	fdp;
+#endif
 #ifdef CONFIG_NVME_TARGET_DEBUGFS
 	struct dentry		*debugfs_dir;
 #endif
@@ -356,6 +370,7 @@ struct nvmet_subsys {
 	char			*model_number;
 	u32			ieee_oui;
 	char			*firmware_rev;
+	unsigned int		nr_ns_fdp_enabled;
 
 #ifdef CONFIG_NVME_TARGET_PASSTHRU
 	struct nvme_ctrl	*passthru_ctrl;
@@ -982,6 +997,27 @@ static inline void nvmet_pr_put_ns_pc_ref(struct nvmet_pr_per_ctrl_ref *pc_ref)
 	percpu_ref_put(&pc_ref->ref);
 }
 
+#ifdef CONFIG_NVME_TARGET_FDP
+static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
+{
+	return ns->fdp.enable;
+}
+int nvmet_fdp_init_ns(struct nvmet_ns *ns);
+void nvmet_fdp_exit_ns(struct nvmet_ns *ns);
+#else
+static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
+{
+	return false;
+}
+static inline int nvmet_fdp_init_ns(struct nvmet_ns *ns)
+{
+	return -EINVAL;
+}
+static inline void nvmet_fdp_exit_ns(struct nvmet_ns *ns)
+{
+}
+#endif
+
 /*
  * Data for the get_feature() and set_feature() operations of PCI target
  * controllers.
Signature · 1 line
2.43.7
author
#5 lore raw

PATCH v3 5/12 nvmet: document the fdp_enable configfs attribute

In reply to Guixin Liu

Document the boolean fdp_enable attribute: it is only changeable while the namespace is disabled, and enabling it requires a backing NVMe namespace whose write streams and placement identifier layout pass the enable-time checks.

---
 Documentation/ABI/stable/configfs-nvmet | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Documentation/ABI/stable/configfs-nvmet +13 -0
index 36b587404ee7..093435439dab 100644
@@ -302,6 +302,19 @@ Description:
 		enabled for this namespace. Accepts boolean value. The
 		namespace must be disabled before modification.
 
+What:		/config/nvmet/subsystems/NAME/namespaces/NSID/fdp_enable
+Date:		October 2026
+KernelVersion:	7.4
+Contact:	Guixin Liu <[email protected]>
+Description:
+		Shows or sets whether Flexible Data Placement bridging is
+		enabled for this namespace. Accepts boolean value and
+		requires CONFIG_NVME_TARGET_FDP. The namespace must be
+		disabled before modification. Enabling requires a backing
+		NVMe namespace that reports write streams and an identity
+		mapped placement identifier list, which is verified when
+		the namespace is enabled.
+
 What:		/config/nvmet/subsystems/NAME/attr_vendor_id
 What:		/config/nvmet/subsystems/NAME/attr_subsys_vendor_id
 Date:		January 2025
Signature · 1 line
2.43.7
author
#6 lore raw

PATCH v3 6/12 nvmet: report the Endurance Group Identifier in the CS-INDEP identify data

In reply to Guixin Liu

The Identify Namespace data structure (CNS 00h) reports the Endurance Group Identifier of the namespace, but the I/O Command Set Independent Identify Namespace data structure (CNS 08h) leaves the field zeroed. Hosts that implement NVMe 2.0 or later, or that cannot use CNS 00h, take the Endurance Group Identifier from the CS-INDEP data structure and thus always see 0, which breaks every consumer of the identifier:
the FDP feature and log pages and the Rotational Media Information log page are addressed by it.

Report the same Endurance Group Identifier as CNS 00h: each namespace is its own endurance group, so the identifier equals the namespace identifier.

---
 drivers/nvme/target/admin-cmd.c | 1 +
 1 file changed, 1 insertion(+)
drivers/nvme/target/admin-cmd.c +1 -0
index 7764a3c0195c..aad5abdb6ef9 100644
@@ -1081,6 +1081,7 @@ static void nvmet_execute_id_cs_indep(struct nvmet_req *req)
 	id->nstat = NVME_NSTAT_NRDY;
 	id->anagrpid = cpu_to_le32(req->ns->anagrpid);
 	id->nmic = NVME_NS_NMIC_SHARED;
+	id->endgid = cpu_to_le16(req->ns->nsid);
 	if (req->ns->readonly)
 		id->nsattr |= NVME_NS_ATTR_RO;
 	if (req->ns->bdev && bdev_rot(req->ns->bdev))
Signature · 1 line
2.43.7
author
#7 lore raw

PATCH v3 7/12 nvmet: advertise the FDP capability

In reply to Guixin Liu

Set the Flexible Data Placement Supported (FDPS) bit in the Identify Controller ctratt field whenever at least one enabled namespace of the subsystem bridges FDP. Hosts use the FDPS bit to decide whether to query the FDP feature and configuration for each namespace, so without it the bridge stays invisible. The FDP log pages are listed in the Supported Log Pages log page by the patches that serve them.

---
 drivers/nvme/target/admin-cmd.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/nvme/target/admin-cmd.c +2 -0
index aad5abdb6ef9..8a8ab2af3882 100644
@@ -699,6 +699,8 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
 	ctratt = NVME_CTRL_ATTR_HID_128_BIT | NVME_CTRL_ATTR_TBKAS;
 	if (nvmet_is_pci_ctrl(ctrl))
 		ctratt |= NVME_CTRL_ATTR_RHII;
+	if (ctrl->subsys->nr_ns_fdp_enabled)
+		ctratt |= NVME_CTRL_ATTR_FDPS;
 	id->ctratt = cpu_to_le32(ctratt);
 
 	id->oacs = 0;
Signature · 1 line
2.43.7
author
#8 lore raw

PATCH v3 8/12 nvmet: implement the FDP feature

In reply to Guixin Liu

Implement Set Features and Get Features for FID 1Dh. The Endurance Group Identifier in Command Dword 11 names the namespace directly, since nvmet identifies each namespace as its own endurance group (endgid = nsid). Get Features reports FDPE set with FDPCIDX 0 for bridged namespaces, and FDPE clear for the others, so that a subsystem mixing both kinds does not fail the per-namespace discovery the host runs on every namespace.

Set Features is served locally: the bridge runs the one configuration that the backing device is already in, so setting the feature to that same value is a no-op that succeeds, and any other value is a change of a feature whose endurance group has namespaces, which the spec aborts with Command Sequence Error. That also covers disabling, which would strand the write streams already handed out to hosts.

The namespace lookup runs under RCU and takes a namespace reference, so that a namespace being disabled concurrently with the command cannot go away under it. Report the feature in the Features log page under the same per-subsystem condition as the FDPS bit.

---
 drivers/nvme/target/admin-cmd.c |   9 +++
 drivers/nvme/target/fdp.c       | 111 ++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h     |  10 +++
 3 files changed, 130 insertions(+)
drivers/nvme/target/admin-cmd.c +9 -0
index 8a8ab2af3882..6d0b1df3ae96 100644
@@ -600,6 +600,9 @@ static void nvmet_execute_get_log_page_features(struct nvmet_req *req)
 		cpu_to_le32(NVME_FIS_FSUPP | NVME_FIS_NSCPE);
 	features->fis[NVME_FEAT_RESV_MASK] =
 		cpu_to_le32(NVME_FIS_FSUPP | NVME_FIS_NSCPE);
+	if (nvmet_req_subsys(req)->nr_ns_fdp_enabled)
+		features->fis[NVME_FEAT_FDP] =
+			cpu_to_le32(NVME_FIS_FSUPP | NVME_FIS_EGSCPE);
 
 	status = nvmet_copy_to_sgl(req, 0, features, sizeof(*features));
 	kfree(features);
@@ -1385,6 +1388,9 @@ void nvmet_execute_set_features(struct nvmet_req *req)
 	case NVME_FEAT_RESV_MASK:
 		status = nvmet_set_feat_resv_notif_mask(req, cdw11);
 		break;
+	case NVME_FEAT_FDP:
+		status = nvmet_set_feat_fdp(req);
+		break;
 	default:
 		req->error_loc = offsetof(struct nvme_common_command, cdw10);
 		status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
@@ -1562,6 +1568,9 @@ void nvmet_execute_get_features(struct nvmet_req *req)
 	case NVME_FEAT_RESV_MASK:
 		status = nvmet_get_feat_resv_notif_mask(req);
 		break;
+	case NVME_FEAT_FDP:
+		status = nvmet_get_feat_fdp(req);
+		break;
 	default:
 		req->error_loc =
 			offsetof(struct nvme_common_command, cdw10);
drivers/nvme/target/fdp.c +111 -0

120 lines not on this page · raw message

drivers/nvme/target/nvmet.h +10 -0
index 81defb4abfdd..5620492dd431 100644
@@ -1004,6 +1004,8 @@ static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
 }
 int nvmet_fdp_init_ns(struct nvmet_ns *ns);
 void nvmet_fdp_exit_ns(struct nvmet_ns *ns);
+u16 nvmet_set_feat_fdp(struct nvmet_req *req);
+u16 nvmet_get_feat_fdp(struct nvmet_req *req);
 #else
 static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
 {
@@ -1016,6 +1018,14 @@ static inline int nvmet_fdp_init_ns(struct nvmet_ns *ns)
 static inline void nvmet_fdp_exit_ns(struct nvmet_ns *ns)
 {
 }
+static inline u16 nvmet_set_feat_fdp(struct nvmet_req *req)
+{
+	return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
+}
+static inline u16 nvmet_get_feat_fdp(struct nvmet_req *req)
+{
+	return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
+}
 #endif
 
 /*
Signature · 1 line
2.43.7
author
#9 lore raw

PATCH v3 9/12 nvmet: forward the FDP I/O management commands to the backing device

In reply to Guixin Liu

Dispatch the I/O Management Receive and Send commands from the I/O command parser, after the persistent reservation checks so that the commands go through the same access path as the other I/O commands, with the transfer length derived from NUMD and NPID, respectively, and report both commands as supported in the Commands Supported and Effects log page. The transfer length encoded in the command is verified against the length the host actually supplied, so that the forwarded command cannot carry a length that disagrees with the data buffer.

Both are forwarded to the backing namespace: the data is staged through a linear kernel buffer, which is what the backend submission path takes, so the reclaim unit handle status hosts read reflects the real reclaim units of the backing device, and a Reclaim Unit Handle Update is carried out by the controller that owns that state. The host data pointer is not carried over to the backing command.

When the namespace is enabled, learn the backing FDP geometry that the forwarding needs: the Endurance Group Identifier the backing FDP state is addressed by, and the placement identifier list of the Reclaim Unit Handle Status. The bridge translates placement identifiers to write streams by their index, so the enable fails when the backing placement identifiers are not identity mapped. The backing command failures are reported as errno; the NVMe status of the backing controller is logged but not propagated, as the enable path may only fail with a negative error.

The enable also reads the backing FDP configurations log page, so that the reclaim unit nominal size and the handle types presented to hosts are the values of the backing device itself, and the Commands Supported and Effects advertisement of the new opcodes is limited to subsystems that actually bridge FDP.

The FDP command handlers run from a workqueue, since lpfc can invoke the execute callbacks in hardirq context, and the backing geometry is read from the active FDP configuration selected by FDPCIDX rather than from the first descriptor of the configurations log.

---
 drivers/nvme/target/admin-cmd.c |  17 +-
 drivers/nvme/target/core.c      |  27 ++-
 drivers/nvme/target/fdp.c       | 370 +++++++++++++++++++++++++++++++-
 drivers/nvme/target/nvmet.h     |  10 +
 4 files changed, 418 insertions(+), 6 deletions(-)
drivers/nvme/target/admin-cmd.c +14 -3
index 6d0b1df3ae96..523083eb0cee 100644
@@ -381,7 +381,8 @@ static void nvmet_get_cmd_effects_admin(struct nvmet_ctrl *ctrl,
 		cpu_to_le32(NVME_CMD_EFFECTS_CSUPP);
 }
 
-static void nvmet_get_cmd_effects_nvm(struct nvme_effects_log *log)
+static void nvmet_get_cmd_effects_nvm(struct nvmet_req *req,
+				      struct nvme_effects_log *log)
 {
 	log->iocs[nvme_cmd_read] =
 	log->iocs[nvme_cmd_flush] =
@@ -394,6 +395,16 @@ static void nvmet_get_cmd_effects_nvm(struct nvme_effects_log *log)
 	log->iocs[nvme_cmd_write] =
 	log->iocs[nvme_cmd_write_zeroes] =
 		cpu_to_le32(NVME_CMD_EFFECTS_CSUPP | NVME_CMD_EFFECTS_LBCC);
+	/*
+	 * This log page has no namespace context (req->ns is always
+	 * NULL), so the advertisement can only follow the subsystem-wide
+	 * FDP state, just like FDPS and the Features log page.
+	 */
+	if (nvmet_req_subsys(req)->nr_ns_fdp_enabled) {
+		log->iocs[nvme_cmd_io_mgmt_recv] =
+		log->iocs[nvme_cmd_io_mgmt_send] =
+			cpu_to_le32(NVME_CMD_EFFECTS_CSUPP);
+	}
 }
 
 static void nvmet_get_cmd_effects_zns(struct nvme_effects_log *log)
@@ -420,7 +431,7 @@ static void nvmet_execute_get_log_cmd_effects_ns(struct nvmet_req *req)
 	switch (req->cmd->get_log_page.csi) {
 	case NVME_CSI_NVM:
 		nvmet_get_cmd_effects_admin(ctrl, log);
-		nvmet_get_cmd_effects_nvm(log);
+		nvmet_get_cmd_effects_nvm(req, log);
 		break;
 	case NVME_CSI_ZNS:
 		if (!IS_ENABLED(CONFIG_BLK_DEV_ZONED)) {
@@ -428,7 +439,7 @@ static void nvmet_execute_get_log_cmd_effects_ns(struct nvmet_req *req)
 			goto free;
 		}
 		nvmet_get_cmd_effects_admin(ctrl, log);
-		nvmet_get_cmd_effects_nvm(log);
+		nvmet_get_cmd_effects_nvm(req, log);
 		nvmet_get_cmd_effects_zns(log);
 		break;
 	default:
drivers/nvme/target/core.c +26 -1
index 14e2a908857c..2bae9ee86611 100644
@@ -1114,9 +1114,22 @@ static u32 nvmet_io_cmd_transfer_len(struct nvmet_req *req)
 		return nvmet_dsm_len(req);
 	case nvme_cmd_zone_mgmt_recv:
 		return (le32_to_cpu(req->cmd->zmr.numd) + 1) << 2;
+	case nvme_cmd_io_mgmt_recv:
+		if (IS_ENABLED(CONFIG_NVME_TARGET_FDP))
+			return (le32_to_cpu(req->cmd->imr.numd) + 1) << 2;
+		break;
+	case nvme_cmd_io_mgmt_send:
+		if (IS_ENABLED(CONFIG_NVME_TARGET_FDP)) {
+			u32 npid = le16_to_cpu(req->cmd->ims.mos);
+
+			return (npid + 1) * sizeof(__le16);
+		}
+		break;
 	default:
-		return 0;
+		break;
 	}
+
+	return 0;
 }
 
 static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
@@ -1158,6 +1171,17 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 			return ret;
 	}
 
+	if (IS_ENABLED(CONFIG_NVME_TARGET_FDP)) {
+		switch (req->cmd->common.opcode) {
+		case nvme_cmd_io_mgmt_recv:
+			req->execute = nvmet_execute_io_mgmt_recv;
+			goto check_pr;
+		case nvme_cmd_io_mgmt_send:
+			req->execute = nvmet_execute_io_mgmt_send;
+			goto check_pr;
+		}
+	}
+
 	switch (req->ns->csi) {
 	case NVME_CSI_NVM:
 		if (req->ns->file)
@@ -1177,6 +1201,7 @@ static u16 nvmet_parse_io_cmd(struct nvmet_req *req)
 	if (ret)
 		return ret;
 
+check_pr:
 	if (req->ns->pr.enable) {
 		ret = nvmet_pr_check_cmd_access(req);
 		if (ret)
drivers/nvme/target/fdp.c +368 -2

402 lines not on this page · raw message

drivers/nvme/target/nvmet.h +10 -0
index 5620492dd431..0a20c22b0b23 100644
@@ -1006,6 +1006,8 @@ int nvmet_fdp_init_ns(struct nvmet_ns *ns);
 void nvmet_fdp_exit_ns(struct nvmet_ns *ns);
 u16 nvmet_set_feat_fdp(struct nvmet_req *req);
 u16 nvmet_get_feat_fdp(struct nvmet_req *req);
+void nvmet_execute_io_mgmt_recv(struct nvmet_req *req);
+void nvmet_execute_io_mgmt_send(struct nvmet_req *req);
 #else
 static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
 {
@@ -1026,6 +1028,14 @@ static inline u16 nvmet_get_feat_fdp(struct nvmet_req *req)
 {
 	return NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
 }
+static inline void nvmet_execute_io_mgmt_recv(struct nvmet_req *req)
+{
+	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_STATUS_DNR);
+}
+static inline void nvmet_execute_io_mgmt_send(struct nvmet_req *req)
+{
+	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_STATUS_DNR);
+}
 #endif
 
 /*
Signature · 1 line
2.43.7
author
#10 lore raw

PATCH v3 10/12 nvmet: serve the FDP log pages

In reply to Guixin Liu

The bridge advertises the FDP log pages 20h to 23h as supported; serve them.

The FDP Configurations log page (LID 20h) is constructed from the snapshot taken at namespace enable time: one valid configuration with a single reclaim group, the backing number of reclaim unit handles, the handle types and the reclaim unit nominal size reported by the backing device. The page is addressed by the Endurance Group Identifier in the Log Specific Identifier field. Hosts first request the 16 byte header to learn the page size and then the full page; both request sizes are served from the same construction path. Short reads and nonzero Log Page Offsets are served as windows of the complete page, and a zero length forward is rejected so that no command can reach the backing controller without a staging buffer.

The remaining pages are owned by the backing controller and are forwarded to it over its admin queue, addressed by the Endurance Group Identifier of the backing endurance group, so hosts read the real reclaim unit handle usage, counters and events of the backing device.

---
 drivers/nvme/target/admin-cmd.c |  16 ++++
 drivers/nvme/target/fdp.c       | 138 ++++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h     |  11 +++
 3 files changed, 165 insertions(+)
drivers/nvme/target/admin-cmd.c +16 -0
index 523083eb0cee..e4efee44f5f4 100644
@@ -220,6 +220,16 @@ static void nvmet_execute_get_supported_log_pages(struct nvmet_req *req)
 	logs->lids[NVME_LOG_FEATURES] = cpu_to_le32(NVME_LIDS_LSUPP);
 	logs->lids[NVME_LOG_RMI] = cpu_to_le32(NVME_LIDS_LSUPP);
 	logs->lids[NVME_LOG_RESERVATION] = cpu_to_le32(NVME_LIDS_LSUPP);
+	if (nvmet_req_subsys(req)->nr_ns_fdp_enabled) {
+		logs->lids[NVME_LOG_FDP_CONFIGS] =
+			cpu_to_le32(NVME_LIDS_LSUPP);
+		logs->lids[NVME_LOG_FDP_RUH_USAGE] =
+			cpu_to_le32(NVME_LIDS_LSUPP);
+		logs->lids[NVME_LOG_FDP_STATS] =
+			cpu_to_le32(NVME_LIDS_LSUPP);
+		logs->lids[NVME_LOG_FDP_EVENTS] =
+			cpu_to_le32(NVME_LIDS_LSUPP);
+	}
 
 	status = nvmet_copy_to_sgl(req, 0, logs, sizeof(*logs));
 	kfree(logs);
@@ -654,6 +664,12 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)
 		return nvmet_execute_get_log_page_rmi(req);
 	case NVME_LOG_RESERVATION:
 		return nvmet_execute_get_log_page_resv(req);
+	case NVME_LOG_FDP_CONFIGS:
+		return nvmet_execute_get_log_page_fdp_configs(req);
+	case NVME_LOG_FDP_RUH_USAGE:
+	case NVME_LOG_FDP_STATS:
+	case NVME_LOG_FDP_EVENTS:
+		return nvmet_fdp_log_forward(req);
 	}
 	pr_debug("unhandled lid %d on qid %d\n",
 	       req->cmd->get_log_page.lid, req->sq->qid);
drivers/nvme/target/fdp.c +138 -0

141 lines not on this page · raw message

drivers/nvme/target/nvmet.h +11 -0
index 0a20c22b0b23..c61cfe66f336 100644
@@ -1008,6 +1008,8 @@ u16 nvmet_set_feat_fdp(struct nvmet_req *req);
 u16 nvmet_get_feat_fdp(struct nvmet_req *req);
 void nvmet_execute_io_mgmt_recv(struct nvmet_req *req);
 void nvmet_execute_io_mgmt_send(struct nvmet_req *req);
+void nvmet_execute_get_log_page_fdp_configs(struct nvmet_req *req);
+void nvmet_fdp_log_forward(struct nvmet_req *req);
 #else
 static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
 {
@@ -1036,6 +1038,15 @@ static inline void nvmet_execute_io_mgmt_send(struct nvmet_req *req)
 {
 	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_STATUS_DNR);
 }
+static inline void nvmet_execute_get_log_page_fdp_configs(
+		struct nvmet_req *req)
+{
+	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_STATUS_DNR);
+}
+static inline void nvmet_fdp_log_forward(struct nvmet_req *req)
+{
+	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_STATUS_DNR);
+}
 #endif
 
 /*
Signature · 1 line
2.43.7
author
#11 lore raw

PATCH v3 11/12 nvmet: translate placement identifiers of writes to write streams

In reply to Guixin Liu

A write command that carries the Data Placement Directive names a placement identifier in the Directive Specific field. Both sides of the bridge use identity mapped handles: the write streams of the backing device map to the placement handles one by one, in the order the reclaim unit handle status reports them. Translate the placement identifier to the matching backing write stream on the write path, so the data lands in the reclaim unit of the backing device that the host selected. The enable of the namespace has verified that the backing placement identifiers are identity mapped.

The translated stream is assigned to every bio of the write, not just the first one, so that a write larger than a single bio keeps its placement over the whole chain.

Writes without the directive, and writes with a placement identifier that is out of range, keep the default stream 0: the spec lets the controller place such data itself.

---
 drivers/nvme/target/io-cmd-bdev.c |  7 +++++++
 drivers/nvme/target/nvmet.h       | 24 ++++++++++++++++++++++++
 2 files changed, 31 insertions(+)
drivers/nvme/target/io-cmd-bdev.c +7 -0
index f2d9e8901df4..1390f70748af 100644
@@ -256,6 +256,7 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req)
 	struct bio *bio;
 	struct scatterlist *sg;
 	struct blk_plug plug;
+	u8 write_stream = 0;
 	sector_t sector;
 	blk_opf_t opf;
 	int i, rc;
@@ -300,6 +301,11 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req)
 	bio->bi_iter.bi_sector = sector;
 	bio->bi_private = req;
 	bio->bi_end_io = nvmet_bio_done;
+	if (req->cmd->rw.opcode == nvme_cmd_write &&
+	    nvmet_fdp_enabled(req->ns)) {
+		write_stream = nvmet_fdp_write_stream(req);
+		bio->bi_write_stream = write_stream;
+	}
 
 	blk_start_plug(&plug);
 	if (req->metadata_len)
@@ -323,6 +329,7 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req)
 			bio = bio_alloc(req->ns->bdev, bio_max_segs(sg_cnt),
 					opf, GFP_KERNEL);
 			bio->bi_iter.bi_sector = sector;
+			bio->bi_write_stream = write_stream;
 
 			bio_chain(bio, prev);
 			submit_bio(prev);
drivers/nvme/target/nvmet.h +24 -0
index c61cfe66f336..2013c72c3bcc 100644
@@ -1010,6 +1010,26 @@ void nvmet_execute_io_mgmt_recv(struct nvmet_req *req);
 void nvmet_execute_io_mgmt_send(struct nvmet_req *req);
 void nvmet_execute_get_log_page_fdp_configs(struct nvmet_req *req);
 void nvmet_fdp_log_forward(struct nvmet_req *req);
+
+/*
+ * Both sides of the bridge use identity mapped handles, so placement
+ * identifier i routes to backing write stream i+1; stream 0, the
+ * default, places the data without a directive.
+ */
+static inline u8 nvmet_fdp_write_stream(struct nvmet_req *req)
+{
+	u16 control = le16_to_cpu(req->cmd->rw.control);
+	u16 pid;
+
+	if ((control & 0xf0) != (NVME_RW_DTYPE_DPLCMT & 0xf0))
+		return 0;
+
+	pid = le32_to_cpu(req->cmd->rw.dsmgmt) >> 16;
+	if (pid >= req->ns->fdp.nruh)
+		return 0;
+
+	return pid + 1;
+}
 #else
 static inline bool nvmet_fdp_enabled(struct nvmet_ns *ns)
 {
@@ -1047,6 +1067,10 @@ static inline void nvmet_fdp_log_forward(struct nvmet_req *req)
 {
 	nvmet_req_complete(req, NVME_SC_INVALID_FIELD | NVME_STATUS_DNR);
 }
+static inline u8 nvmet_fdp_write_stream(struct nvmet_req *req)
+{
+	return 0;
+}
 #endif
 
 /*
Signature · 1 line
2.43.7
author
#12 lore raw

PATCH v3 12/12 nvmet: add an FDP write translation tracepoint

In reply to Guixin Liu

Add a tracepoint to the write path that reports the placement identifier carried by a write command with the Data Placement Directive and the backing write stream it is translated to.

---
 drivers/nvme/target/io-cmd-bdev.c |  8 ++++++++
 drivers/nvme/target/trace.h       | 23 +++++++++++++++++++++++
 2 files changed, 31 insertions(+)
drivers/nvme/target/io-cmd-bdev.c +8 -0
index 1390f70748af..6757d099b66a 100644
@@ -9,6 +9,7 @@
 #include <linux/memremap.h>
 #include <linux/module.h>
 #include "nvmet.h"
+#include "trace.h"
 
 void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id)
 {
@@ -305,6 +306,13 @@ static void nvmet_bdev_execute_rw(struct nvmet_req *req)
 	    nvmet_fdp_enabled(req->ns)) {
 		write_stream = nvmet_fdp_write_stream(req);
 		bio->bi_write_stream = write_stream;
+
+		if (write_stream) {
+			u16 pid = le32_to_cpu(req->cmd->rw.dsmgmt) >> 16;
+
+			trace_nvmet_fdp_write_translate(req, pid,
+							write_stream);
+		}
 	}
 
 	blk_start_plug(&plug);
drivers/nvme/target/trace.h +23 -0
index 7f7ebf9558e5..0ab725296079 100644
@@ -131,6 +131,29 @@ TRACE_EVENT(nvmet_req_complete,
 
 );
 
+TRACE_EVENT(nvmet_fdp_write_translate,
+	TP_PROTO(struct nvmet_req *req, u16 pid, u8 stream),
+	TP_ARGS(req, pid, stream),
+	TP_STRUCT__entry(
+		__field(u16, ctrl_id)
+		__array(char, disk, DISK_NAME_LEN)
+		__field(u32, nsid)
+		__field(u16, pid)
+		__field(u8, stream)
+	),
+	TP_fast_assign(
+		__entry->ctrl_id = nvmet_req_to_ctrl_id(req);
+		__assign_req_name(__entry->disk, req);
+		__entry->nsid = req->ns->nsid;
+		__entry->pid = pid;
+		__entry->stream = stream;
+	),
+	TP_printk("nvmet%s: %snsid=%u, pid=%u, write_stream=%u",
+		__print_ctrl_id(__entry->ctrl_id),
+		__print_disk_name(__entry->disk),
+		__entry->nsid, __entry->pid, __entry->stream)
+);
+
 #define aer_name(aer) { aer, #aer }
 
 TRACE_EVENT(nvmet_async_event,
Signature · 1 line
2.43.7
author
#13 lore raw
In reply to Guixin Liu

Hi guys,
  All the issues Sashiko raised are addressed in this v3, and it looks like Sashiko has no further comments on the current version now. Please review this series, all comments are welcome. Or if can tell me this approach is reasonable, I'll keep iterating on the implementation details.

Best Regards,
Guixin Liu

Quoted 173 lines · Guixin Liu

Flexible Data Placement (FDP) is defined in the NVMe base specification and therefore applies to fabrics as well. An Ethernet Bunch Of Flash (EBOF) style storage system that exposes its local FDP-capable NVMe drives to remote hosts over NVMe-oF currently has no way to pass that capability through: nvmet neither advertises FDP nor translates or forwards any FDP command, so a host connected through the target loses the write amplification benefits that the local drives provide.

This series adds an FDP bridge to the NVMe target. A namespace backed by an NVMe namespace opts in through the fdp_enable configfs attribute; the bridge then advertises the FDPS capability, reports the backing FDP configuration through the feature and log pages, translates the placement identifiers of write commands to the backing write streams, and forwards the FDP management commands and log pages to the backing controller. The backing geometry is verified at namespace enable time, so the bridge only comes up when the backing placement identifiers are identity mapped, which the translation relies on.

The FDP commands are served by the bridge directly instead of relying on the existing passthru support, for several reasons:

- Passthru subsystems do not support reservations; the reservation commands are rejected and the controller is not advertised as reservation capable, because the underlying device cannot differentiate the hosts that connect over fabrics. A regular namespace goes through the target's own reservation implementation, which is not available to a passthru subsystem.

- A passthru subsystem is bound to a single backend controller: the whole drive, with all of its namespaces, is passed through as one unit, and no regular namespaces can be added to the subsystem. An EBOF system needs to compose a storage pool out of individual namespaces from multiple backend drives, which only the regular namespace path provides.

- The regular namespace path already handles the target side of reservations, ANA reporting and per-namespace configuration; the bridge extends that path rather than replacing it.

- With passthru, the namespace identity of the backend drive is exposed to the host as is, and the host would talk to the backend controller's FDP state through the passthrough of every command. The bridge instead owns the FDP presentation of the exported namespace, while the state that the backing controller owns is forwarded to it: the reclaim unit handle status hosts read and the reclaim unit handle updates they issue are carried out by the controller that actually manages the media.

The Linux host side needs no changes: the existing FDP discovery path (FDPS bit, endurance group identifier, FDP feature, FDP configurations log page and reclaim unit handle status) works against the bridge unmodified.

The series is structured as follows:

  1-2   wire structures and block layer helper
  3     nvme_bdev_ops export, used to recognize an NVMe backing device
  4     per-namespace bridge state and configuration
  5     fdp_enable ABI documentation
  6     endurance group identifier in the CS-INDEP identify data
  7     FDPS advertisement
  8     FDP feature (FID 1Dh)
  9     I/O management command forwarding
  10    FDP log pages (LID 20h-23h)
  11    placement identifier translation on the write path
  12    tracepoint for the translation

This was tested with a QEMU FDP backing device through an nvmet-tcp loopback setup, with persistent reservations enabled on the exported
namespace: the host discovers and registers the backing write streams over the bridge, writes with placement identifiers land in the reclaim units of the backend device they select (verified through the reclaim unit handle status of the backend, for single bio and chained bio writes alike), and the forwarded log pages match the backend byte for byte. Short and offset reads of the configurations log match the full page, and Set Features on a namespace without FDP accepts the unchanged disabled configuration. I/O management receive requests whose NUMD arithmetic would wrap, zero-length forwards, and requests above the 1 MiB staging bound are rejected with Invalid Field. A second namespace without FDP in the same subsystem comes up normally and reports FDPE clear. With a second controller registered, the FDP commands are served while a write exclusive reservation is held, writes from the non-holder conflict, and preempt-and-abort completes with I/O management commands in flight.

Changes since v2:

https://lore.kernel.org/all/[email protected]/

- Take the namespace reference of the FDP commands under NVMET_NS_IO_LIVE with percpu_ref_tryget_live_rcu(), closing a use-after-free against a concurrently disabled namespace.
- Report FDPE clear from Get Features for namespaces without FDP, so that a subsystem mixing bridged and regular namespaces does not fail the host's per-namespace discovery; Set Features accepts that same disabled configuration as a no-op.
- Evaluate the I/O management receive transfer length in 64 bits instead of letting the 32-bit NUMD arithmetic wrap, bound all staged FDP transfers to 1 MiB, and reject zero-length forwards.
- Re-encode NUMD of the forwarded log pages from the staged transfer length instead of carrying over the host supplied value, and clear the reserved metadata field of the forwarded commands.
- Fall through to the persistent reservation access checks for the I/O management commands instead of returning before them.
- Require the backing reclaim unit handle count to match the write streams exactly instead of accepting a larger count.
- Read the backing FDP configurations log at enable time so the advertised handle types and the 64-bit reclaim unit nominal size are the backing device's own values, selected by the active FDPCIDX rather than by descriptor order, and validate every descriptor before dereferencing it.
- Serve short reads and nonzero Log Page Offsets of the configurations log as windows of the complete page.
- Run the FDP command handlers from a workqueue so that lpfc cannot sleep in hardirq context.
- Track the FDP capability as a per-subsystem count, incremented at namespace enable and decremented at namespace disable.
- Advertise the I/O management commands in the command-effects log only when the subsystem actually bridges FDP, and set error_loc for invalid FDP feature requests.
- Document fdp_enable in the ABI file.

Changes since v1:

https://lore.kernel.org/all/[email protected]/

- Verify the transfer length encoded in the I/O management commands against the length the host supplied, and do not carry the host data pointer over to the backing command.
- Assign the translated write stream to every bio of a write, not just the first one.
- Run the namespace lookup of the FDP commands under RCU.
- Report the FDP feature in the Features log page under the same per-subsystem condition as the FDPS bit.
- Only emit the write translation tracepoint for writes that carry the Data Placement Directive.
- Dispatch the I/O management commands after the persistent reservation checks.
- Report backing command failures during the namespace enable as errno, and clean up the backend state when the enable fails.
- Drop the Assisted-by: LLM tag from all commits.

Guixin Liu (12):
nvme: add the FDP wire structures for the I/O management send command
block: add bdev_write_stream_granularity()
nvme: export nvme_bdev_ops
nvmet: add per-namespace FDP bridge state and configuration
nvmet: document the fdp_enable configfs attribute
nvmet: report the Endurance Group Identifier in the CS-INDEP identify
data
nvmet: advertise the FDP capability
nvmet: implement the FDP feature
nvmet: forward the FDP I/O management commands to the backing device
nvmet: serve the FDP log pages
nvmet: translate placement identifiers of writes to write streams
nvmet: add an FDP write translation tracepoint

 Documentation/ABI/stable/configfs-nvmet |  13 +
 drivers/nvme/host/core.c                |   1 +
 drivers/nvme/target/Kconfig             |  14 +
 drivers/nvme/target/Makefile            |   1 +
 drivers/nvme/target/admin-cmd.c         |  45 +-
 drivers/nvme/target/configfs.c          |  32 ++
 drivers/nvme/target/core.c              |  43 +-
 drivers/nvme/target/fdp.c               | 682 ++++++++++++++++++++++++
 drivers/nvme/target/io-cmd-bdev.c       |  15 +
 drivers/nvme/target/nvmet.h             |  91 ++++
 drivers/nvme/target/trace.h             |  23 +
 include/linux/blkdev.h                  |   8 +
 include/linux/nvme.h                    |  24 +
 13 files changed, 987 insertions(+), 5 deletions(-)
 create mode 100644 drivers/nvme/target/fdp.c
#14 lore raw

Hi,

Quoted 5 lines · Guixin Liu

  All the issues Sashiko raised are addressed in this v3, and it looks like Sashiko has no further comments on the current version now. Please review this series, all comments are welcome. Or if can tell me this approach is reasonable, I'll keep iterating on the implementation details.

Before diving deeper into the implementation details, I want to make sure we're aligned on the higher-level direction. As far as I recall from previous discussions, there has been significant hesitation around supporting FDP.

I am happy to help review, but to respect everyone's time, it might be best to first confirm that this has a change to be accepted.

Thanks,
Daniel

author
#15 lore raw
Quoted 8 lines · Daniel Wagner

Hi,

Quoted 5 lines · Guixin Liu

  All the issues Sashiko raised are addressed in this v3, and it looks like Sashiko has no further comments on the current version now. Please review this series, all comments are welcome. Or if can tell me this approach is reasonable, I'll keep iterating on the implementation details.

Quoted 4 lines

Before diving deeper into the implementation details, I want to make sure we're aligned on the higher-level direction. As far as I recall from previous discussions, there has been significant hesitation around supporting FDP.

Has FDP passthrough support in the nvme target module been discussed before?

I'm not familiar with the history here — could you fill me in so we can align on whether we want to support it?

Best Regards,

Guixin Liu

Quoted 6 lines

I am happy to help review, but to respect everyone's time, it might be best to first confirm that this has a change to be accepted.

Thanks,
Daniel

author
#16 lore raw
In reply to Guixin Liu

Hi,
  A gentle ping on this series, all comments are welcome. If you have any concerns about this feature, please let me know. Best Regards, Guixin Liu

Quoted 173 lines · Guixin Liu

Flexible Data Placement (FDP) is defined in the NVMe base specification and therefore applies to fabrics as well. An Ethernet Bunch Of Flash (EBOF) style storage system that exposes its local FDP-capable NVMe drives to remote hosts over NVMe-oF currently has no way to pass that capability through: nvmet neither advertises FDP nor translates or forwards any FDP command, so a host connected through the target loses the write amplification benefits that the local drives provide.

This series adds an FDP bridge to the NVMe target. A namespace backed by an NVMe namespace opts in through the fdp_enable configfs attribute; the bridge then advertises the FDPS capability, reports the backing FDP configuration through the feature and log pages, translates the placement identifiers of write commands to the backing write streams, and forwards the FDP management commands and log pages to the backing controller. The backing geometry is verified at namespace enable time, so the bridge only comes up when the backing placement identifiers are identity mapped, which the translation relies on.

The FDP commands are served by the bridge directly instead of relying on the existing passthru support, for several reasons:

- Passthru subsystems do not support reservations; the reservation commands are rejected and the controller is not advertised as reservation capable, because the underlying device cannot differentiate the hosts that connect over fabrics. A regular namespace goes through the target's own reservation implementation, which is not available to a passthru subsystem.

- A passthru subsystem is bound to a single backend controller: the whole drive, with all of its namespaces, is passed through as one unit, and no regular namespaces can be added to the subsystem. An EBOF system needs to compose a storage pool out of individual namespaces from multiple backend drives, which only the regular namespace path provides.

- The regular namespace path already handles the target side of reservations, ANA reporting and per-namespace configuration; the bridge extends that path rather than replacing it.

- With passthru, the namespace identity of the backend drive is exposed to the host as is, and the host would talk to the backend controller's FDP state through the passthrough of every command. The bridge instead owns the FDP presentation of the exported namespace, while the state that the backing controller owns is forwarded to it: the reclaim unit handle status hosts read and the reclaim unit handle updates they issue are carried out by the controller that actually manages the media.

The Linux host side needs no changes: the existing FDP discovery path (FDPS bit, endurance group identifier, FDP feature, FDP configurations log page and reclaim unit handle status) works against the bridge unmodified.

The series is structured as follows:

  1-2   wire structures and block layer helper
  3     nvme_bdev_ops export, used to recognize an NVMe backing device
  4     per-namespace bridge state and configuration
  5     fdp_enable ABI documentation
  6     endurance group identifier in the CS-INDEP identify data
  7     FDPS advertisement
  8     FDP feature (FID 1Dh)
  9     I/O management command forwarding
  10    FDP log pages (LID 20h-23h)
  11    placement identifier translation on the write path
  12    tracepoint for the translation

This was tested with a QEMU FDP backing device through an nvmet-tcp loopback setup, with persistent reservations enabled on the exported
namespace: the host discovers and registers the backing write streams over the bridge, writes with placement identifiers land in the reclaim units of the backend device they select (verified through the reclaim unit handle status of the backend, for single bio and chained bio writes alike), and the forwarded log pages match the backend byte for byte. Short and offset reads of the configurations log match the full page, and Set Features on a namespace without FDP accepts the unchanged disabled configuration. I/O management receive requests whose NUMD arithmetic would wrap, zero-length forwards, and requests above the 1 MiB staging bound are rejected with Invalid Field. A second namespace without FDP in the same subsystem comes up normally and reports FDPE clear. With a second controller registered, the FDP commands are served while a write exclusive reservation is held, writes from the non-holder conflict, and preempt-and-abort completes with I/O management commands in flight.

Changes since v2:

https://lore.kernel.org/all/[email protected]/

- Take the namespace reference of the FDP commands under NVMET_NS_IO_LIVE with percpu_ref_tryget_live_rcu(), closing a use-after-free against a concurrently disabled namespace.
- Report FDPE clear from Get Features for namespaces without FDP, so that a subsystem mixing bridged and regular namespaces does not fail the host's per-namespace discovery; Set Features accepts that same disabled configuration as a no-op.
- Evaluate the I/O management receive transfer length in 64 bits instead of letting the 32-bit NUMD arithmetic wrap, bound all staged FDP transfers to 1 MiB, and reject zero-length forwards.
- Re-encode NUMD of the forwarded log pages from the staged transfer length instead of carrying over the host supplied value, and clear the reserved metadata field of the forwarded commands.
- Fall through to the persistent reservation access checks for the I/O management commands instead of returning before them.
- Require the backing reclaim unit handle count to match the write streams exactly instead of accepting a larger count.
- Read the backing FDP configurations log at enable time so the advertised handle types and the 64-bit reclaim unit nominal size are the backing device's own values, selected by the active FDPCIDX rather than by descriptor order, and validate every descriptor before dereferencing it.
- Serve short reads and nonzero Log Page Offsets of the configurations log as windows of the complete page.
- Run the FDP command handlers from a workqueue so that lpfc cannot sleep in hardirq context.
- Track the FDP capability as a per-subsystem count, incremented at namespace enable and decremented at namespace disable.
- Advertise the I/O management commands in the command-effects log only when the subsystem actually bridges FDP, and set error_loc for invalid FDP feature requests.
- Document fdp_enable in the ABI file.

Changes since v1:

https://lore.kernel.org/all/[email protected]/

- Verify the transfer length encoded in the I/O management commands against the length the host supplied, and do not carry the host data pointer over to the backing command.
- Assign the translated write stream to every bio of a write, not just the first one.
- Run the namespace lookup of the FDP commands under RCU.
- Report the FDP feature in the Features log page under the same per-subsystem condition as the FDPS bit.
- Only emit the write translation tracepoint for writes that carry the Data Placement Directive.
- Dispatch the I/O management commands after the persistent reservation checks.
- Report backing command failures during the namespace enable as errno, and clean up the backend state when the enable fails.
- Drop the Assisted-by: LLM tag from all commits.

Guixin Liu (12):
nvme: add the FDP wire structures for the I/O management send command
block: add bdev_write_stream_granularity()
nvme: export nvme_bdev_ops
nvmet: add per-namespace FDP bridge state and configuration
nvmet: document the fdp_enable configfs attribute
nvmet: report the Endurance Group Identifier in the CS-INDEP identify
data
nvmet: advertise the FDP capability
nvmet: implement the FDP feature
nvmet: forward the FDP I/O management commands to the backing device
nvmet: serve the FDP log pages
nvmet: translate placement identifiers of writes to write streams
nvmet: add an FDP write translation tracepoint

 Documentation/ABI/stable/configfs-nvmet |  13 +
 drivers/nvme/host/core.c                |   1 +
 drivers/nvme/target/Kconfig             |  14 +
 drivers/nvme/target/Makefile            |   1 +
 drivers/nvme/target/admin-cmd.c         |  45 +-
 drivers/nvme/target/configfs.c          |  32 ++
 drivers/nvme/target/core.c              |  43 +-
 drivers/nvme/target/fdp.c               | 682 ++++++++++++++++++++++++
 drivers/nvme/target/io-cmd-bdev.c       |  15 +
 drivers/nvme/target/nvmet.h             |  91 ++++
 drivers/nvme/target/trace.h             |  23 +
 include/linux/blkdev.h                  |   8 +
 include/linux/nvme.h                    |  24 +
 13 files changed, 987 insertions(+), 5 deletions(-)
 create mode 100644 drivers/nvme/target/fdp.c