Skip to contentSkip to conversation

nvme: introduce the service-time iopolicy

PATCH Superseded Guixin Liu 4 participants 12 messages Oct 19

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Guixin Liu

author
#0 lore raw

v2 cover nvme: introduce the service-time iopolicy

The queue-depth iopolicy counts requests, so a 128K I/O weighs the same as a 4K one, and it cannot express that one path is simply faster than another. This series adds a service-time iopolicy for NVMe native multipath, adapted from dm-ps-service-time: it picks the path with the lowest estimated service time, in_flight_bytes / relative_throughput, so balancing follows actual bytes in flight and per-path throughput weights.

Weights live in a per-path relative_throughput sysfs attribute (0-100, default 1, 0 makes the path a standby). Patch 3 documents the new attributes in the stable sysfs ABI; patch 1 carries the benchmark data in its commit log.

Changes since v1:

- Rebased onto v7.3. iopolicy parsing was refactored into nvme_iopolicy_parse() in the meantime, which picks up the new policy for both the module parameter and the per-subsystem attribute.

- The comparison helper now returns a sign-safe -1/0/1. The in-flight sizes are 64-bit, and returning the plain difference truncated to int can report the wrong sign once loads grow. Selection is unchanged in the ranges the benchmarks cover.

- relative_throughput now defaults to 1 in nvme_mpath_init_ctrl(). v1 re-applied the default on every controller reset in nvme_mpath_init_identify(), which silently re-enabled standby paths after routine reconnects.

- Dropped the unused "incoming" argument of the comparison helper.

- Added ABI documentation for in_flight_bytes and relative_throughput.

On the adaptive I/O policy dependency Hannes raised during v1: that series has become the latency I/O policy and is still under revision, so this version stays on plain master. The two are
complementary: the latency policy infers path speed from observed completions, service-time takes configured weights and byte counts. I can rebase once the latency series lands.

Guixin Liu (3):
nvme-multipath: implement service-time iopolicy
nvme-multipath: document service-time iopolicy
nvme: document the service-time iopolicy sysfs attributes

 Documentation/ABI/stable/sysfs-nvme          |  22 +++
 Documentation/admin-guide/nvme-multipath.rst |  31 +++-
 drivers/nvme/host/multipath.c                | 162 ++++++++++++++++++-
 drivers/nvme/host/nvme.h                     |   6 +
 drivers/nvme/host/sysfs.c                    |   5 +-
 5 files changed, 221 insertions(+), 5 deletions(-)

v1 of this series:
https://lore.kernel.org/r/[email protected]

latency I/O policy v7:
https://lore.kernel.org/r/[email protected]

Signature · 1 line
2.43.7
author
#1 lore raw

PATCH v2 1/3 nvme-multipath: implement service-time iopolicy

The queue-depth iopolicy balances paths by the number of in-flight requests. This misjudges the load in two common cases: a 128K request counts the same as a 4K one, and there is no way to tell the selector that one path is simply faster than another. With mixed I/O sizes or paths of different bandwidth, large I/Os keep piling onto the slow path, driving up tail latencies.

Add a "service-time" iopolicy adapted from the dm-ps-service-time path selector. It selects the path with the lowest estimated service time, computed as in_flight_bytes / relative_throughput, so it tracks actual bytes in flight rather than request counts, and users can weight paths according to their throughput.

Each path keeps the total in-flight I/O size and a relative throughput weight as controller state, exposed through two new sysfs attributes on the path namespace devices: in_flight_bytes (read-only) reports the tracked load, and relative_throughput (read-write, 0-100, default 1) sets the weight. Setting relative_throughput to 0 makes the path a standby that is only selected when no path with a positive value is available, matching DM service-time semantics.

As in dm-ps-service-time, the path comparison cross-multiplies to avoid division, and shifts the load counters down when they grow too large to keep the products in range.

Usage:
echo service-time > /sys/module/nvme_core/parameters/iopolicy echo 4 > /sys/block/nvmeXcYnZ/relative_throughput

Test environment:
Two QEMU/KVM VMs connected via NVMe-oF TCP, 32 vCPU / 32G RAM each. The target exports a 1G RAM disk via nvmet-tcp on two ports, and the initiator connects both ports to form a native NVMe multipath device.

Asymmetric bandwidth is created by shaping the second path with tc-tbf (rate 500mbit, burst 128kb, latency 5ms). Measured single path bandwidth: ~237 MiB/s unshaped vs ~56 MiB/s shaped, a ratio of approximately 4:1.

Test 1 - Uniform 4k random read (iodepth=64, numjobs=4, 15s):

  iopolicy              IOPS   BW(MiB/s)  avg_lat(us)  lat_stddev(us)
  --------------------  -----  ---------  -----------  --------------
  round-robin           59.1k  231        4316         3109
  queue-depth           59.0k  230        4323         3046
  service-time 1:1      59.3k  232        4302         3057
  service-time 4:1      61.0k  238        4179         1294

With uniform I/O size, service-time 1:1 performs similarly to queue-depth since per-byte and per-request tracking are equivalent for fixed-size I/Os. service-time 4:1 with correct throughput weights improves IOPS by 3.4% and reduces latency stddev by 57%.

Test 2 - Mixed random read 4k/128k (bssplit=4k/50:128k/50,

          iodepth=64, numjobs=4, 15s):
  iopolicy              IOPS  BW(MiB/s)  avg_lat(us)  lat_stddev(us)
  --------------------  ----  ---------  -----------  --------------
  round-robin           7689  276        33209        46817
  queue-depth           7642  276        33429        45868
  service-time 1:1      7456  269        34279        47433
  service-time 4:1      7747  277        32998        27102

With mixed I/O sizes on asymmetric paths, queue-depth treats 128K and 4K requests identically when counting in-flight depth, leading to suboptimal load distribution. service-time tracks actual bytes in flight, producing significantly more consistent tail latencies. service-time 4:1 vs queue-depth: latency stddev -41%.

Standby path (relative_throughput=0):
Setting a path's relative_throughput to 0 correctly excludes it from selection. With the slow path set to 0, all I/O is directed to the fast path only (IO split 118052:0), verified via diskstats.

---
 drivers/nvme/host/multipath.c | 162 +++++++++++++++++++++++++++++++++-
 drivers/nvme/host/nvme.h      |   6 ++
 drivers/nvme/host/sysfs.c     |   5 +-
 3 files changed, 170 insertions(+), 3 deletions(-)
drivers/nvme/host/multipath.c +160 -2
index 99ecc73393e58..df7ebb6f9aa1d 100644
@@ -69,6 +69,7 @@ static const char *nvme_iopolicy_names[] = {
 	[NVME_IOPOLICY_NUMA]	= "numa",
 	[NVME_IOPOLICY_RR]	= "round-robin",
 	[NVME_IOPOLICY_QD]      = "queue-depth",
+	[NVME_IOPOLICY_ST]	= "service-time",
 };
 
 static int iopolicy = NVME_IOPOLICY_NUMA;
@@ -107,7 +108,7 @@ static int nvme_get_iopolicy(char *buf, const struct kernel_param *kp)
 module_param_call(iopolicy, nvme_set_iopolicy, nvme_get_iopolicy,
 	&iopolicy, 0644);
 MODULE_PARM_DESC(iopolicy,
-	"Default multipath I/O policy; 'numa' (default), 'round-robin' or 'queue-depth'");
+	"Default multipath I/O policy; 'numa' (default), 'round-robin', 'queue-depth' or 'service-time'");
 
 void nvme_mpath_default_iopolicy(struct nvme_subsystem *subsys)
 {
@@ -179,11 +180,16 @@ void nvme_mpath_start_request(struct request *rq)
 {
 	struct nvme_ns *ns = rq->q->queuedata;
 	struct gendisk *disk = ns->head->disk;
+	int iopolicy = READ_ONCE(ns->head->subsys->iopolicy);
 
-	if ((READ_ONCE(ns->head->subsys->iopolicy) == NVME_IOPOLICY_QD) &&
+	if (iopolicy == NVME_IOPOLICY_QD &&
 	    !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE)) {
 		atomic_inc(&ns->ctrl->nr_active);
 		nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE;
+	} else if (iopolicy == NVME_IOPOLICY_ST &&
+		   !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES)) {
+		atomic64_add(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
+		nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE_BYTES;
 	}
 
 	if (!blk_queue_io_stat(disk->queue) ||
@@ -205,6 +211,10 @@ void nvme_mpath_end_request(struct request *rq)
 
 	if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE)
 		atomic_dec_if_positive(&ns->ctrl->nr_active);
+	if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES) {
+		atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
+		nvme_req(rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES;
+	}
 
 	if (!(nvme_req(rq)->flags & NVME_MPATH_IO_STATS))
 		return;
@@ -464,6 +474,109 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)
 	return best_opt ? best_opt : best_nonopt;
 }
 
+#define ST_MAX_RELATIVE_THROUGHPUT	100
+#define ST_MAX_RELATIVE_THROUGHPUT_SHIFT	7
+#define ST_MAX_INFLIGHT_SIZE \
+	((size_t)-1 >> ST_MAX_RELATIVE_THROUGHPUT_SHIFT)
+
+/*
+ * Compare estimated service time of two paths.
+ * Returns negative if ns1 is better, positive if ns2 is better, 0 if equal.
+ *
+ * Service time = in_flight_size / relative_throughput
+ * Cross-multiply to avoid division.
+ */
+static int nvme_st_compare_load(struct nvme_ns *ns1, struct nvme_ns *ns2)
+{
+	size_t sz1, sz2, st1, st2;
+	unsigned int tp1 = ns1->ctrl->relative_throughput;
+	unsigned int tp2 = ns2->ctrl->relative_throughput;
+
+	sz1 = atomic64_read(&ns1->ctrl->in_flight_size);
+	sz2 = atomic64_read(&ns2->ctrl->in_flight_size);
+
+	/* Case 1: same throughput — compare load directly */
+	if (tp1 == tp2)
+		return (sz1 > sz2) - (sz1 < sz2);
+
+	/*
+	 * Case 2a: same load — prefer higher throughput.
+	 * Case 2b: one path has zero throughput — prefer the other.
+	 */
+	if (sz1 == sz2 || !tp1 || !tp2)
+		return tp2 - tp1;
+
+	/*
+	 * Case 3: general comparison via cross-multiplication.
+	 *   st1 = sz1 / tp1
+	 *   st2 = sz2 / tp2
+	 * Equivalent (since tp > 0):
+	 *   sz1 * tp2 <=> sz2 * tp1
+	 *
+	 * Use comparisons instead of subtractions when returning: the
+	 * intermediate values do not fit into the int return type.
+	 */
+	if (unlikely(sz1 >= ST_MAX_INFLIGHT_SIZE ||
+		     sz2 >= ST_MAX_INFLIGHT_SIZE)) {
+		sz1 >>= ST_MAX_RELATIVE_THROUGHPUT_SHIFT;
+		sz2 >>= ST_MAX_RELATIVE_THROUGHPUT_SHIFT;
+	}
+	st1 = sz1 * tp2;
+	st2 = sz2 * tp1;
+	if (st1 != st2)
+		return (st1 > st2) - (st1 < st2);
+
+	/* Case 4: equal service time — prefer higher throughput */
+	return tp2 - tp1;
+}
+
+static struct nvme_ns *nvme_service_time_path(struct nvme_ns_head *head)
+{
+	struct nvme_ns *best_opt = NULL, *best_nonopt = NULL, *ns;
+	struct nvme_ns *fallback_opt = NULL, *fallback_nonopt = NULL;
+
+	list_for_each_entry_srcu(ns, &head->list, siblings,
+				 srcu_read_lock_held(&head->srcu)) {
+		if (nvme_path_is_disabled(ns))
+			continue;
+
+		/*
+		 * Paths with relative_throughput == 0 are only used when no
+		 * path with a positive value is available (matching DM
+		 * service-time semantics).
+		 */
+		if (!ns->ctrl->relative_throughput) {
+			if (ns->ana_state == NVME_ANA_OPTIMIZED && !fallback_opt)
+				fallback_opt = ns;
+			else if (ns->ana_state == NVME_ANA_NONOPTIMIZED &&
+				 !fallback_nonopt)
+				fallback_nonopt = ns;
+			continue;
+		}
+
+		switch (ns->ana_state) {
+		case NVME_ANA_OPTIMIZED:
+			if (!best_opt ||
+			    nvme_st_compare_load(ns, best_opt) < 0)
+				best_opt = ns;
+			break;
+		case NVME_ANA_NONOPTIMIZED:
+			if (!best_nonopt ||
+			    nvme_st_compare_load(ns, best_nonopt) < 0)
+				best_nonopt = ns;
+			break;
+		default:
+			break;
+		}
+	}
+
+	if (best_opt)
+		return best_opt;
+	if (best_nonopt)
+		return best_nonopt;
+	return fallback_opt ? fallback_opt : fallback_nonopt;
+}
+
 static inline bool nvme_path_is_optimized(struct nvme_ns *ns)
 {
 	return nvme_ctrl_state(ns->ctrl) == NVME_CTRL_LIVE &&
@@ -491,6 +604,8 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)
 		return nvme_queue_depth_path(head);
 	case NVME_IOPOLICY_RR:
 		return nvme_round_robin_path(head);
+	case NVME_IOPOLICY_ST:
+		return nvme_service_time_path(head);
 	default:
 		return nvme_numa_path(head);
 	}
@@ -1110,6 +1225,47 @@ static ssize_t queue_depth_show(struct device *dev,
 }
 DEVICE_ATTR_RO(queue_depth);
 
+static ssize_t in_flight_bytes_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct nvme_ns *ns = nvme_get_ns_from_dev(dev);
+
+	if (ns->head->subsys->iopolicy != NVME_IOPOLICY_ST)
+		return 0;
+
+	return sysfs_emit(buf, "%lld\n", atomic64_read(&ns->ctrl->in_flight_size));
+}
+DEVICE_ATTR_RO(in_flight_bytes);
+
+static ssize_t relative_throughput_show(struct device *dev,
+		struct device_attribute *attr, char *buf)
+{
+	struct nvme_ns *ns = nvme_get_ns_from_dev(dev);
+
+	if (ns->head->subsys->iopolicy != NVME_IOPOLICY_ST)
+		return 0;
+
+	return sysfs_emit(buf, "%u\n", ns->ctrl->relative_throughput);
+}
+
+static ssize_t relative_throughput_store(struct device *dev,
+		struct device_attribute *attr, const char *buf, size_t count)
+{
+	struct nvme_ns *ns = nvme_get_ns_from_dev(dev);
+	unsigned int val;
+	int ret;
+
+	ret = kstrtouint(buf, 0, &val);
+	if (ret < 0)
+		return ret;
+	if (val > ST_MAX_RELATIVE_THROUGHPUT)
+		return -EINVAL;
+
+	ns->ctrl->relative_throughput = val;
+	return count;
+}
+DEVICE_ATTR_RW(relative_throughput);
+
 static ssize_t numa_nodes_show(struct device *dev, struct device_attribute *attr,
 		char *buf)
 {
@@ -1435,6 +1591,7 @@ void nvme_mpath_init_ctrl(struct nvme_ctrl *ctrl)
 	mutex_init(&ctrl->ana_lock);
 	timer_setup(&ctrl->anatt_timer, nvme_anatt_timeout, 0);
 	INIT_WORK(&ctrl->ana_work, nvme_ana_work);
+	ctrl->relative_throughput = 1;
 }
 
 int nvme_mpath_init_identify(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
@@ -1450,6 +1607,7 @@ int nvme_mpath_init_identify(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
 
 	/* initialize this in the identify path to cover controller resets */
 	atomic_set(&ctrl->nr_active, 0);
+	atomic64_set(&ctrl->in_flight_size, 0);
 
 	if (!ctrl->max_namespaces ||
 	    ctrl->max_namespaces > le32_to_cpu(id->nn)) {
drivers/nvme/host/nvme.h +6 -0
index a643c4723a69b..02cdd55350911 100644
@@ -268,6 +268,7 @@ enum {
 	NVME_REQ_USERCMD		= (1 << 1),
 	NVME_MPATH_IO_STATS		= (1 << 2),
 	NVME_MPATH_CNT_ACTIVE		= (1 << 3),
+	NVME_MPATH_CNT_ACTIVE_BYTES	= (1 << 4),
 };
 
 static inline struct nvme_request *nvme_req(struct request *req)
@@ -438,6 +439,8 @@ struct nvme_ctrl {
 	struct timer_list anatt_timer;
 	struct work_struct ana_work;
 	atomic_t nr_active;
+	atomic64_t in_flight_size;
+	u8 relative_throughput;
 #endif
 
 #ifdef CONFIG_NVME_HOST_AUTH
@@ -491,6 +494,7 @@ enum nvme_iopolicy {
 	NVME_IOPOLICY_NUMA,
 	NVME_IOPOLICY_RR,
 	NVME_IOPOLICY_QD,
+	NVME_IOPOLICY_ST,
 };
 
 struct nvme_subsystem {
@@ -1095,6 +1099,8 @@ extern bool multipath;
 extern struct device_attribute dev_attr_ana_grpid;
 extern struct device_attribute dev_attr_ana_state;
 extern struct device_attribute dev_attr_queue_depth;
+extern struct device_attribute dev_attr_in_flight_bytes;
+extern struct device_attribute dev_attr_relative_throughput;
 extern struct device_attribute dev_attr_numa_nodes;
 extern struct device_attribute dev_attr_delayed_removal_secs;
 extern struct device_attribute dev_attr_multipath_failover_count;
drivers/nvme/host/sysfs.c +4 -1
index e1e3dcfd084be..a46e7db4934a7 100644
@@ -262,6 +262,8 @@ static struct attribute *nvme_ns_attrs[] = {
 	&dev_attr_ana_grpid.attr,
 	&dev_attr_ana_state.attr,
 	&dev_attr_queue_depth.attr,
+	&dev_attr_in_flight_bytes.attr,
+	&dev_attr_relative_throughput.attr,
 	&dev_attr_numa_nodes.attr,
 	&dev_attr_delayed_removal_secs.attr,
 #endif
@@ -296,7 +298,8 @@ static umode_t nvme_ns_attrs_are_visible(struct kobject *kobj,
 		if (!nvme_ctrl_use_ana(nvme_get_ns_from_dev(dev)->ctrl))
 			return 0;
 	}
-	if (a == &dev_attr_queue_depth.attr || a == &dev_attr_numa_nodes.attr) {
+	if (a == &dev_attr_queue_depth.attr || a == &dev_attr_in_flight_bytes.attr ||
+	    a == &dev_attr_relative_throughput.attr || a == &dev_attr_numa_nodes.attr) {
 		if (nvme_disk_is_ns_head(dev_to_disk(dev)))
 			return 0;
 	}
Signature · 1 line
2.43.7
#2 lore raw

Re: [PATCH v2 1/3] nvme-multipath: implement service-time iopolicy

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] nvme-multipath: in_flight_size leak in nvme_mpath_end_request() on failover - [Medium]…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] nvme-multipath: in_flight_size leak in nvme_mpath_end_request() on failover
- [Medium] nvme-multipath: in_flight_size underflow wraps in nvme_st_compare_load() --

commit 211891d7cca0785689865f3a7650f5305abaef90
Author: Guixin Liu <[email protected]>

nvme-multipath: implement service-time iopolicy

This commit adds a service-time multipath I/O policy that selects paths based on the ratio of in-flight bytes to a configurable relative throughput weight, exposing in_flight_bytes and relative_throughput sysfs attributes.

Quoted 4 lines
drivers/nvme/host/multipath.c +0 -0
index 75dbb58286a32..3901654d8226b 100644

[ ... ]

Quoted 19 lines
drivers/nvme/host/multipath.c +6 -1
@@ -179,11 +180,16 @@ void nvme_mpath_start_request(struct request *rq)
 {
 	struct nvme_ns *ns = rq->q->queuedata;
 	struct gendisk *disk = ns->head->disk;
+	int iopolicy = READ_ONCE(ns->head->subsys->iopolicy);
 
-	if ((READ_ONCE(ns->head->subsys->iopolicy) == NVME_IOPOLICY_QD) &&
+	if (iopolicy == NVME_IOPOLICY_QD &&
 	    !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE)) {
 		atomic_inc(&ns->ctrl->nr_active);
 		nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE;
+	} else if (iopolicy == NVME_IOPOLICY_ST &&
+		   !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES)) {
+		atomic64_add(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
+		nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE_BYTES;
 	}
 
 	if (!blk_queue_io_stat(disk->queue) ||

drivers/nvme/host/multipath.c void nvme_mpath_end_request(struct request *rq) · line 217

Quoted 7 lines
drivers/nvme/host/multipath.c +4 -0
@@ -205,6 +211,10 @@ void nvme_mpath_end_request(struct request *rq)
 
 	if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE)
 		atomic_dec_if_positive(&ns->ctrl->nr_active);
+	if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES) {
+		atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
+		nvme_req(rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES;
+	}

[Severity: Medium]
Can this leak bytes in ns->ctrl->in_flight_size when a request fails over to another path?

When a request fails with a path error (such as NVME_SC_ANA_TRANSITION, NVME_SC_ANA_INACCESSIBLE, NVME_SC_ANA_PERSISTENT_LOSS, NVME_SC_HOST_PATH_ERROR, or NVME_SC_HOST_ABORTED_CMD) or on a dying queue, __nvme_complete_rq() calls nvme_failover_req():

drivers/nvme/host/multipath.c:nvme_failover_req() {
	...
	spin_lock_irqsave(&ns->head->requeue_lock, flags);
	for (bio = req->bio; bio; bio = bio->bi_next)
		bio_set_dev(bio, ns->head->disk->part0);
	blk_steal_bios(&ns->head->requeue_list, req);
	spin_unlock_irqrestore(&ns->head->requeue_lock, flags);

	nvme_req(req)->status = 0;
	nvme_end_req(req);
}

blk_steal_bios() detaches all bios from the request and sets req->__data_len = 0 before nvme_end_req() -> __nvme_end_req() -> nvme_mpath_end_request() is called:

block/blk-mq.c:blk_steal_bios() {
	...
	rq->bio = NULL;
	rq->biotail = NULL;
	rq->__data_len = 0;
}

When nvme_mpath_end_request() subsequently calls blk_rq_bytes(rq), it reads rq->__data_len as 0 and subtracts 0 from ns->ctrl->in_flight_size while clearing NVME_MPATH_CNT_ACTIVE_BYTES. Whenever failover occurs without a controller reset (for example during an ANA state transition, or on a multipath controller without NVME_CTRL_CMIC_ANA where nvme_mpath_init_identify() returns early), those bytes remain permanently counted in ns->ctrl->in_flight_size, skewing path selection in nvme_service_time_path() and leaving in_flight_bytes non-zero when idle.

Quoted 17 lines
drivers/nvme/host/multipath.c +10 -0
 	if (!(nvme_req(rq)->flags & NVME_MPATH_IO_STATS))
 		return;
@@ -464,6 +474,109 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)
 	return best_opt ? best_opt : best_nonopt;
 }
 
+#define ST_MAX_RELATIVE_THROUGHPUT	100
+#define ST_MAX_RELATIVE_THROUGHPUT_SHIFT	7
+#define ST_MAX_INFLIGHT_SIZE \
+	((size_t)-1 >> ST_MAX_RELATIVE_THROUGHPUT_SHIFT)
+
+/*
+ * Compare estimated service time of two paths.
+ * Returns negative if ns1 is better, positive if ns2 is better, 0 if equal.
+ *
+ * Service time = in_flight_size / relative_throughput

drivers/nvme/host/multipath.c static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head) · line 496

Quoted 10 lines
drivers/nvme/host/multipath.c +10 -0
+ * Cross-multiply to avoid division.
+ */
+static int nvme_st_compare_load(struct nvme_ns *ns1, struct nvme_ns *ns2)
+{
+	size_t sz1, sz2, st1, st2;
+	unsigned int tp1 = ns1->ctrl->relative_throughput;
+	unsigned int tp2 = ns2->ctrl->relative_throughput;
+
+	sz1 = atomic64_read(&ns1->ctrl->in_flight_size);
+	sz2 = atomic64_read(&ns2->ctrl->in_flight_size);

[Severity: Medium]
What happens here if ns->ctrl->in_flight_size underflows below zero after a controller reset?

When a request is started under NVME_IOPOLICY_ST, nvme_setup_cmd() calls nvme_clear_nvme_request() (which sets RQF_DONTPREP), and nvme_mpath_start_request() adds blk_rq_bytes(rq) to ns->ctrl->in_flight_size and sets NVME_MPATH_CNT_ACTIVE_BYTES.

If that request is then placed onto q->requeue_list in MQ_RQ_IDLE state without calling nvme_mpath_end_request() (either via nvme_retry_req() -> blk_mq_requeue_request(), or when a transport queue_rq callback such as nvme_rdma_queue_rq(), nvme_fc_start_fcp_op(), or nvme_loop_queue_rq() returns BLK_STS_RESOURCE after nvme_start_request()) and a controller reset occurs:

1. nvme_cancel_tagset() -> nvme_cancel_request() skips the request because blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT.
2. nvme_mpath_init_identify() resets ctrl->in_flight_size to 0 via atomic64_set(&ctrl->in_flight_size, 0).
3. When the queue is unquiesced and the request is re-dispatched, nvme_setup_cmd() preserves nvme_req(req)->flags because RQF_DONTPREP is set, and nvme_mpath_start_request() skips adding blk_rq_bytes(rq) because NVME_MPATH_CNT_ACTIVE_BYTES is already set.
4. When the request completes, nvme_mpath_end_request() unconditionally executes atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size), driving ctrl->in_flight_size negative (unlike ctrl->nr_active, which uses atomic_dec_if_positive()).

In nvme_st_compare_load(), the negative s64 returned by atomic64_read() is assigned to the unsigned size_t variables sz1 and sz2, wrapping to a large positive value near SIZE_MAX and causing nvme_service_time_path() to treat the path as heavily overloaded until enough new I/O leaks or resets it. Should in_flight_size be prevented from going negative or clamped when read?

[ ... ]

Quoted 8 lines
drivers/nvme/host/multipath.c +1 -0
@@ -1453,6 +1610,7 @@ int nvme_mpath_init_identify(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)
 
 	/* initialize this in the identify path to cover controller resets */
 	atomic_set(&ctrl->nr_active, 0);
+	atomic64_set(&ctrl->in_flight_size, 0);
 
 	if (!ctrl->max_namespaces ||
 	    ctrl->max_namespaces > le32_to_cpu(id->nn)) {

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author
#3 lore raw

PATCH v2 2/3 nvme-multipath: document service-time iopolicy

In reply to Guixin Liu

Add documentation for the service-time path selection policy: its algorithm, the in_flight_bytes and relative_throughput sysfs attributes, and guidance on when to prefer it over queue-depth.

Document that setting relative_throughput to 0 makes the path a standby that is only selected when no path with a positive value is available.

---
 Documentation/admin-guide/nvme-multipath.rst | 31 ++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)
Documentation/admin-guide/nvme-multipath.rst +29 -2
index 97ca1ccef459b..2acfceaf3d652 100644
@@ -24,8 +24,8 @@ Policies
 
 All policies follow the ANA (Asymmetric Namespace Access) mechanism, meaning
 that when an optimized path is available, it will be chosen over a non-optimized
-one. Current the NVMe multipath policies include numa(default), round-robin and
-queue-depth.
+one. Current the NVMe multipath policies include numa(default), round-robin,
+queue-depth and service-time.
 
 To set the desired policy (e.g., round-robin), use one of the following methods:
    1. echo -n "round-robin" > /sys/module/nvme_core/parameters/iopolicy
@@ -70,3 +70,30 @@ When to use the queue-depth policy:
   1. High load with small I/Os: Effectively balances load across paths when
      the load is high, and I/O operations consist of small, relatively
      fixed-sized requests.
+
+
+Service-Time
+------------
+
+The service-time policy selects the path with the lowest estimated service time.
+It calculates service time as ``in_flight_bytes / relative_throughput`` for each
+path, preferring the path that would complete I/O fastest. Unlike queue-depth
+which counts requests regardless of size, service-time tracks actual bytes in
+flight, making it aware of I/O sizes.
+
+Each path exposes two sysfs attributes under
+``/sys/class/nvme/nvmeX/nvmeXcYnZ/``:
+
+  - ``in_flight_bytes`` (read-only): Current bytes in flight on this path.
+  - ``relative_throughput`` (read-write): Relative throughput weight for this
+    path, default 1. The valid range is 0-100. Set higher values for faster
+    paths. If set to 0, the path is not selected while other paths with
+    positive values are available.
+
+When to use the service-time policy:
+  1. Asymmetric Link Speeds: When paths have different bandwidths, set
+     ``relative_throughput`` proportionally (e.g., 2 for a link twice as fast)
+     to steer more traffic to faster paths.
+  2. Mixed I/O Sizes: When workloads mix small and large I/Os (e.g., 4K and
+     128K), service-time distributes load more accurately than queue-depth
+     because it accounts for actual bytes rather than request count.
Signature · 1 line
2.43.7
author
#4 lore raw

PATCH v2 3/3 nvme: document the service-time iopolicy sysfs attributes

In reply to Guixin Liu

The service-time iopolicy adds two sysfs attributes to the NVMe path namespace devices: in_flight_bytes, reporting the bytes currently in flight on the path's controller, and relative_throughput, the per-path throughput weight used by the service-time estimation.

Document both in the stable sysfs ABI, including the 0-100 value range, the default of 1, and the standby semantics of a relative_throughput of 0.

---
 Documentation/ABI/stable/sysfs-nvme | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
Documentation/ABI/stable/sysfs-nvme +22 -0
index a2f5d0710db40..adaf292c8c67b 100644
@@ -326,6 +326,28 @@ Description:
 		currently selected path. Returns empty if iopolicy is not
 		"numa". Requires CONFIG_NVME_MULTIPATH.
 
+What:		/sys/block/nvmeXcYnZ/in_flight_bytes
+What:		/sys/block/nvmeXcYnZ/relative_throughput
+Date:		October 2026
+KernelVersion:	7.5
+Contact:	Guixin Liu <[email protected]>
+Description:
+		in_flight_bytes: Shows the number of bytes currently in
+		flight on this path's controller. Returns empty if
+		iopolicy is not "service-time".
+		Requires CONFIG_NVME_MULTIPATH.
+
+		relative_throughput: Shows or sets the relative
+		throughput weight of this path, which the "service-time"
+		iopolicy uses to estimate the path service time as
+		in_flight_bytes / relative_throughput. The value is an
+		integer in the range 0-100 and defaults to 1; assign
+		higher values to faster paths. A path with a value of 0
+		is only selected when no path with a positive value is
+		available. Reading returns empty if iopolicy is not
+		"service-time"; writing is accepted regardless of the
+		active policy. Requires CONFIG_NVME_MULTIPATH.
+
 What:		/sys/block/nvmeXnY/delayed_removal_secs
 Date:		May 2025
 KernelVersion:	6.16
Signature · 1 line
2.43.7
#5 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] nvme-multipath: nvme_failover_req() clears rq bytes before in_flight_size sub - [High]…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] nvme-multipath: nvme_failover_req() clears rq bytes before in_flight_size sub
- [High] nvme-multipath: in_flight_size underflow in nvme_mpath_end_request()
- [Low] nvme: missing service-time in sysfs-nvme iopolicy ABI documentation --

commit 5acf4057a11f40dc3e2cd32f04002286c4084ff4
Author: Guixin Liu <[email protected]>

nvme: document the service-time iopolicy sysfs attributes

This commit documents the new in_flight_bytes and relative_throughput sysfs attributes added for the service-time NVMe multipath I/O policy in Documentation/ABI/stable/sysfs-nvme.

Quoted 8 lines
Documentation/ABI/stable/sysfs-nvme +0 -0
index a2f5d0710db40..adaf292c8c67b 100644
@@ -326,6 +326,28 @@ Description:
 		currently selected path. Returns empty if iopolicy is not
 		"numa". Requires CONFIG_NVME_MULTIPATH.
 

Documentation/ABI/stable/sysfs-nvme Description: · line 338

Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +10 -0
+What:		/sys/block/nvmeXcYnZ/in_flight_bytes
+What:		/sys/block/nvmeXcYnZ/relative_throughput
+Date:		October 2026
+KernelVersion:	7.5
+Contact:	Guixin Liu <[email protected]>
+Description:
+		in_flight_bytes: Shows the number of bytes currently in
+		flight on this path's controller. Returns empty if
+		iopolicy is not "service-time".
+		Requires CONFIG_NVME_MULTIPATH.

[Severity: High]
Can ns->ctrl->in_flight_size permanently leak in-flight bytes when a request fails over in nvme_failover_req()?

In commit 211891d7cca0 ("nvme-multipath: implement service-time iopolicy"), nvme_mpath_start_request() adds blk_rq_bytes(rq) to ns->ctrl->in_flight_size and sets NVME_MPATH_CNT_ACTIVE_BYTES:

drivers/nvme/host/multipath.c:nvme_mpath_start_request() {
	...
	} else if (iopolicy == NVME_IOPOLICY_ST &&
		   !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES)) {
		atomic64_add(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
		nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE_BYTES;
	}
}

When a request fails over (for example, on NVME_SC_ANA_TRANSITION, NVME_SC_HOST_PATH_ERROR, or a dying queue), __nvme_complete_rq() calls nvme_failover_req(), which steals the bios from req before calling nvme_end_req():

drivers/nvme/host/multipath.c:nvme_failover_req() {
	...
	blk_steal_bios(&ns->head->requeue_list, req);
	spin_unlock_irqrestore(&ns->head->requeue_lock, flags);

	nvme_req(req)->status = 0;
	nvme_end_req(req);
}

Because blk_steal_bios() clears rq->__data_len:

block/blk-mq.c:blk_steal_bios() {
	...
	rq->__data_len = 0;
}

by the time nvme_end_req() -> __nvme_end_req() -> nvme_mpath_end_request() runs:

drivers/nvme/host/multipath.c:nvme_mpath_end_request() {
	...
	if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES) {
		atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
		nvme_req(rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES;
	}
}

blk_rq_bytes(rq) is 0, so the failed-over request's bytes are never subtracted from ns->ctrl->in_flight_size. When failover occurs without a controller reset, does ns->ctrl->in_flight_size remain permanently inflated, skewing nvme_st_compare_load() and the in_flight_bytes sysfs attribute?

[Severity: High]
Also in nvme_mpath_end_request(), can ns->ctrl->in_flight_size underflow to a negative value if a requeued request completes after a controller reset?

When a request started under NVME_IOPOLICY_ST is requeued via nvme_retry_req() -> blk_mq_requeue_request() (or via BLK_STS_RESOURCE after nvme_start_request() in nvme_rdma_queue_rq() or nvme_fc_start_fcp_op()), the request returns to MQ_RQ_IDLE while retaining RQF_DONTPREP and NVME_MPATH_CNT_ACTIVE_BYTES.

If a controller reset occurs before that request is re-dispatched, nvme_mpath_init_identify() resets ctrl->in_flight_size to 0:

drivers/nvme/host/multipath.c:nvme_mpath_init_identify() {
	...
	/* initialize this in the identify path to cover controller resets */
	atomic_set(&ctrl->nr_active, 0);
	atomic64_set(&ctrl->in_flight_size, 0);
}

When the request is later re-dispatched, nvme_setup_cmd() skips nvme_clear_nvme_request() because RQF_DONTPREP is set, and nvme_mpath_start_request() skips atomic64_add() because NVME_MPATH_CNT_ACTIVE_BYTES is already set. When the request finishes, nvme_mpath_end_request() unconditionally subtracts blk_rq_bytes(rq) from ns->ctrl->in_flight_size (unlike nr_active, which uses atomic_dec_if_positive()), driving ctrl->in_flight_size negative.

In nvme_st_compare_load(), the negative s64 is then assigned to unsigned size_t:

drivers/nvme/host/multipath.c:nvme_st_compare_load() {
	size_t sz1, sz2, st1, st2;
	unsigned int tp1 = ns1->ctrl->relative_throughput;
	unsigned int tp2 = ns2->ctrl->relative_throughput;

	sz1 = atomic64_read(&ns1->ctrl->in_flight_size);
	sz2 = atomic64_read(&ns2->ctrl->in_flight_size);

Would this wrap sz1 or sz2 to near SIZE_MAX, causing nvme_service_time_path() to avoid that controller's path while reporting a negative value in in_flight_bytes?

Quoted 5 lines
Documentation/ABI/stable/sysfs-nvme +5 -0
+
+		relative_throughput: Shows or sets the relative
+		throughput weight of this path, which the "service-time"
+		iopolicy uses to estimate the path service time as
+		in_flight_bytes / relative_throughput. The value is an

Documentation/ABI/stable/sysfs-nvme Description: · line 353

Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +7 -0
+		integer in the range 0-100 and defaults to 1; assign
+		higher values to faster paths. A path with a value of 0
+		is only selected when no path with a positive value is
+		available. Reading returns empty if iopolicy is not
+		"service-time"; writing is accepted regardless of the
+		active policy. Requires CONFIG_NVME_MULTIPATH.
+
 What:		/sys/block/nvmeXnY/delayed_removal_secs
 Date:		May 2025
 KernelVersion:	6.16

[Severity: Low]
In the same file (Documentation/ABI/stable/sysfs-nvme), the entry for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy still lists only the three previous policies:

What: /sys/class/nvme-subsystem/nvme-subsysX/iopolicy
...
Description:
Shows or sets the multipath I/O path selection policy for this subsystem. Accepted values: "numa", "round-robin", "queue-depth". Changing the policy clears all current path selections. Only available when CONFIG_NVME_MULTIPATH is enabled.

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

هایی

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author
#6 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

Quoted 82 lines · 在 2026/10/19 16:13, [email protected] 写道:

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] nvme-multipath: nvme_failover_req() clears rq bytes before in_flight_size sub
- [High] nvme-multipath: in_flight_size underflow in nvme_mpath_end_request()
- [Low] nvme: missing service-time in sysfs-nvme iopolicy ABI documentation --

commit 5acf4057a11f40dc3e2cd32f04002286c4084ff4
Author: Guixin Liu <[email protected]>

nvme: document the service-time iopolicy sysfs attributes

This commit documents the new in_flight_bytes and relative_throughput sysfs attributes added for the service-time NVMe multipath I/O policy in Documentation/ABI/stable/sysfs-nvme.

Quoted 8 lines
Documentation/ABI/stable/sysfs-nvme +0 -0
index a2f5d0710db40..adaf292c8c67b 100644
@@ -326,6 +326,28 @@ Description:
 		currently selected path. Returns empty if iopolicy is not
 		"numa". Requires CONFIG_NVME_MULTIPATH.
 

Documentation/ABI/stable/sysfs-nvme Description: · line 338

Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +10 -0
+What:		/sys/block/nvmeXcYnZ/in_flight_bytes
+What:		/sys/block/nvmeXcYnZ/relative_throughput
+Date:		October 2026
+KernelVersion:	7.5
+Contact:	Guixin Liu <[email protected]>
+Description:
+		in_flight_bytes: Shows the number of bytes currently in
+		flight on this path's controller. Returns empty if
+		iopolicy is not "service-time".
+		Requires CONFIG_NVME_MULTIPATH.

[Severity: High]
Can ns->ctrl->in_flight_size permanently leak in-flight bytes when a request fails over in nvme_failover_req()?

In commit 211891d7cca0 ("nvme-multipath: implement service-time iopolicy"), nvme_mpath_start_request() adds blk_rq_bytes(rq) to ns->ctrl->in_flight_size and sets NVME_MPATH_CNT_ACTIVE_BYTES:

drivers/nvme/host/multipath.c:nvme_mpath_start_request() {
	...
	} else if (iopolicy == NVME_IOPOLICY_ST &&
		   !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES)) {
		atomic64_add(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
		nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE_BYTES;
	}
}

When a request fails over (for example, on NVME_SC_ANA_TRANSITION, NVME_SC_HOST_PATH_ERROR, or a dying queue), __nvme_complete_rq() calls nvme_failover_req(), which steals the bios from req before calling nvme_end_req():

drivers/nvme/host/multipath.c:nvme_failover_req() {
	...
	blk_steal_bios(&ns->head->requeue_list, req);
	spin_unlock_irqrestore(&ns->head->requeue_lock, flags);

	nvme_req(req)->status = 0;
	nvme_end_req(req);
}

Because blk_steal_bios() clears rq->__data_len:

block/blk-mq.c:blk_steal_bios() {
	...
	rq->__data_len = 0;
}

by the time nvme_end_req() -> __nvme_end_req() -> nvme_mpath_end_request() runs:

drivers/nvme/host/multipath.c:nvme_mpath_end_request() {
	...
	if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES) {
		atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
		nvme_req(rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES;
	}
}
Quoted 5 lines

blk_rq_bytes(rq) is 0, so the failed-over request's bytes are never subtracted from ns->ctrl->in_flight_size. When failover occurs without a controller reset, does ns->ctrl->in_flight_size remain permanently inflated, skewing nvme_st_compare_load() and the in_flight_bytes sysfs attribute?

Right, fixed in v3.

Quoted 41 lines

[Severity: High]
Also in nvme_mpath_end_request(), can ns->ctrl->in_flight_size underflow to a negative value if a requeued request completes after a controller reset?

When a request started under NVME_IOPOLICY_ST is requeued via nvme_retry_req() -> blk_mq_requeue_request() (or via BLK_STS_RESOURCE after nvme_start_request() in nvme_rdma_queue_rq() or nvme_fc_start_fcp_op()), the request returns to MQ_RQ_IDLE while retaining RQF_DONTPREP and NVME_MPATH_CNT_ACTIVE_BYTES.

If a controller reset occurs before that request is re-dispatched, nvme_mpath_init_identify() resets ctrl->in_flight_size to 0:

drivers/nvme/host/multipath.c:nvme_mpath_init_identify() {
	...
	/* initialize this in the identify path to cover controller resets */
	atomic_set(&ctrl->nr_active, 0);
	atomic64_set(&ctrl->in_flight_size, 0);
}

When the request is later re-dispatched, nvme_setup_cmd() skips nvme_clear_nvme_request() because RQF_DONTPREP is set, and nvme_mpath_start_request() skips atomic64_add() because NVME_MPATH_CNT_ACTIVE_BYTES is already set. When the request finishes, nvme_mpath_end_request() unconditionally subtracts blk_rq_bytes(rq) from ns->ctrl->in_flight_size (unlike nr_active, which uses atomic_dec_if_positive()), driving ctrl->in_flight_size negative.

In nvme_st_compare_load(), the negative s64 is then assigned to unsigned size_t:

drivers/nvme/host/multipath.c:nvme_st_compare_load() {
	size_t sz1, sz2, st1, st2;
	unsigned int tp1 = ns1->ctrl->relative_throughput;
	unsigned int tp2 = ns2->ctrl->relative_throughput;

	sz1 = atomic64_read(&ns1->ctrl->in_flight_size);
	sz2 = atomic64_read(&ns2->ctrl->in_flight_size);
Quoted 3 lines

Would this wrap sz1 or sz2 to near SIZE_MAX, causing nvme_service_time_path() to avoid that controller's path while reporting a negative value in in_flight_bytes?

Yes, fixed in v3.

Quoted 29 lines
Quoted 5 lines
Documentation/ABI/stable/sysfs-nvme +5 -0
+
+		relative_throughput: Shows or sets the relative
+		throughput weight of this path, which the "service-time"
+		iopolicy uses to estimate the path service time as
+		in_flight_bytes / relative_throughput. The value is an

Documentation/ABI/stable/sysfs-nvme Description: · line 353

Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +7 -0
+		integer in the range 0-100 and defaults to 1; assign
+		higher values to faster paths. A path with a value of 0
+		is only selected when no path with a positive value is
+		available. Reading returns empty if iopolicy is not
+		"service-time"; writing is accepted regardless of the
+		active policy. Requires CONFIG_NVME_MULTIPATH.
+
 What:		/sys/block/nvmeXnY/delayed_removal_secs
 Date:		May 2025
 KernelVersion:	6.16

[Severity: Low]
In the same file (Documentation/ABI/stable/sysfs-nvme), the entry for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy still lists only the three previous policies:

What: /sys/class/nvme-subsystem/nvme-subsysX/iopolicy
...
Description:
Shows or sets the multipath I/O path selection policy for this subsystem. Accepted values: "numa", "round-robin", "queue-depth". Changing the policy clears all current path selections. Only available when CONFIG_NVME_MULTIPATH is enabled.

Quoted 4 lines

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

هایی

Sure, changed in v3.

#7 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

Quoted 6 lines · Guixin Liu
Quoted 5 lines

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

هایی

Sure, changed in v3.

Not just /sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in /sys/class/nvme- subsystem/nvme-subsysX/iopolicies as well.

-Martin

#8 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

Quoted 5 lines · Martin George
Quoted 4 lines · Guixin Liu
Quoted 4 lines

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

Quoted 8 lines
Quoted 2 lines
Quoted 1 line

هایی

Sure, changed in v3.

Not just /sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in /sys/class/nvme- subsystem/nvme-subsysX/iopolicies as well.

-Martin

Hello

There should be no changes needed for my original iopolicies patch

iopolicies_show() walks ARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up in iopolicies automatically. The existing iopolicy store path already matches against the same array, so both attributes stay in sync without code changes

Thanks
Laurence

author
#9 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

Quoted 14 lines · Laurence Oberman
Quoted 11 lines · Martin George
Quoted 5 lines · Guixin Liu
Quoted 4 lines

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

هایی

Sure, changed in v3.

Not just /sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in /sys/class/nvme- subsystem/nvme-subsysX/iopolicies as well.

-Martin

Hello

Quoted 8 lines

There should be no changes needed for my original iopolicies patch

iopolicies_show() walks ARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up in iopolicies automatically. The existing iopolicy store path already matches against the same array, so both attributes stay in sync without code changes

Yes, it is.

Best Regards,
Guixin Liu

Quoted 3 lines

Thanks
Laurence

#10 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

In reply to Laurence Oberman
Quoted 18 lines · Laurence Oberman
Quoted 5 lines · Martin George
Quoted 4 lines · Guixin Liu
Quoted 4 lines

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

Quoted 8 lines
Quoted 2 lines
Quoted 1 line

هایی

Sure, changed in v3.

Not just /sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in /sys/class/nvme- subsystem/nvme-subsysX/iopolicies as well.

-Martin

Hello

There should be no changes needed for my original iopolicies patch

Quoted 9 lines

iopolicies_show() walks ARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up
in iopolicies automatically. The existing iopolicy store path already matches against the same array, so both attributes stay in sync without
code changes

I meant adding to the iopolicies documentation here.

-Martin

#11 lore raw

Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes

Quoted 32 lines · Martin George
Quoted 20 lines · Laurence Oberman
Quoted 7 lines · Martin George
Quoted 6 lines · Guixin Liu
Quoted 6 lines

Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?

هایی

Quoted 8 lines
Quoted 1 line

Sure, changed in v3.

Not just /sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this
should be added to the accepted values in /sys/class/nvme- subsystem/nvme-subsysX/iopolicies as well.

-Martin

Hello

There should be no changes needed for my original iopolicies patch

Quoted 10 lines

iopolicies_show() walks ARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up
in iopolicies automatically. The existing iopolicy store path already
matches against the same array, so both attributes stay in sync without
code changes

Quoted 3 lines

I meant adding to the iopolicies documentation here.

-Martin

Thanks, makes perfect sense
Regards
Laurence