nvme: introduce the service-time iopolicy
Subsystems (2)
v2 cover nvme: introduce the service-time iopolicy
The queue-depth iopolicy counts requests, so a 128K I/O weighs the same as a 4K one, and it cannot express that one path is simply faster than another. This series adds a service-time iopolicy for NVMe native multipath, adapted from dm-ps-service-time: it picks the path with the lowest estimated service time, in_flight_bytes / relative_throughput, so balancing follows actual bytes in flight and per-path throughput weights.
Weights live in a per-path relative_throughput sysfs attribute (0-100, default 1, 0 makes the path a standby). Patch 3 documents the new attributes in the stable sysfs ABI; patch 1 carries the benchmark data in its commit log.
Changes since v1:
- Rebased onto v7.3. iopolicy parsing was refactored into nvme_iopolicy_parse() in the meantime, which picks up the new policy for both the module parameter and the per-subsystem attribute.
- The comparison helper now returns a sign-safe -1/0/1. The in-flight sizes are 64-bit, and returning the plain difference truncated to int can report the wrong sign once loads grow. Selection is unchanged in the ranges the benchmarks cover.
- relative_throughput now defaults to 1 in nvme_mpath_init_ctrl(). v1 re-applied the default on every controller reset in nvme_mpath_init_identify(), which silently re-enabled standby paths after routine reconnects.
- Dropped the unused "incoming" argument of the comparison helper.
- Added ABI documentation for in_flight_bytes and relative_throughput.
On the adaptive I/O policy dependency Hannes raised during v1: that series has become the latency I/O policy and is still under revision, so this version stays on plain master. The two are
complementary: the latency policy infers path speed from observed completions, service-time takes configured weights and byte counts. I can rebase once the latency series lands.
Guixin Liu (3):
nvme-multipath: implement service-time iopolicy
nvme-multipath: document service-time iopolicy
nvme: document the service-time iopolicy sysfs attributes
Documentation/ABI/stable/sysfs-nvme | 22 +++ Documentation/admin-guide/nvme-multipath.rst | 31 +++- drivers/nvme/host/multipath.c | 162 ++++++++++++++++++- drivers/nvme/host/nvme.h | 6 + drivers/nvme/host/sysfs.c | 5 +- 5 files changed, 221 insertions(+), 5 deletions(-)
v1 of this series:
https://lore.kernel.org/r/[email protected]
latency I/O policy v7:
https://lore.kernel.org/r/[email protected]
Signature · 1 line
2.43.7
PATCH v2 1/3 nvme-multipath: implement service-time iopolicy
The queue-depth iopolicy balances paths by the number of in-flight requests. This misjudges the load in two common cases: a 128K request counts the same as a 4K one, and there is no way to tell the selector that one path is simply faster than another. With mixed I/O sizes or paths of different bandwidth, large I/Os keep piling onto the slow path, driving up tail latencies.
Add a "service-time" iopolicy adapted from the dm-ps-service-time path selector. It selects the path with the lowest estimated service time, computed as in_flight_bytes / relative_throughput, so it tracks actual bytes in flight rather than request counts, and users can weight paths according to their throughput.
Each path keeps the total in-flight I/O size and a relative throughput weight as controller state, exposed through two new sysfs attributes on the path namespace devices: in_flight_bytes (read-only) reports the tracked load, and relative_throughput (read-write, 0-100, default 1) sets the weight. Setting relative_throughput to 0 makes the path a standby that is only selected when no path with a positive value is available, matching DM service-time semantics.
As in dm-ps-service-time, the path comparison cross-multiplies to avoid division, and shifts the load counters down when they grow too large to keep the products in range.
Usage:
echo service-time > /sys/module/nvme_core/parameters/iopolicy echo 4 > /sys/block/nvmeXcYnZ/relative_throughput
Test environment:
Two QEMU/KVM VMs connected via NVMe-oF TCP, 32 vCPU / 32G RAM each. The target exports a 1G RAM disk via nvmet-tcp on two ports, and the initiator connects both ports to form a native NVMe multipath device.
Asymmetric bandwidth is created by shaping the second path with tc-tbf (rate 500mbit, burst 128kb, latency 5ms). Measured single path bandwidth: ~237 MiB/s unshaped vs ~56 MiB/s shaped, a ratio of approximately 4:1.
Test 1 - Uniform 4k random read (iodepth=64, numjobs=4, 15s):
iopolicy IOPS BW(MiB/s) avg_lat(us) lat_stddev(us) -------------------- ----- --------- ----------- -------------- round-robin 59.1k 231 4316 3109 queue-depth 59.0k 230 4323 3046 service-time 1:1 59.3k 232 4302 3057 service-time 4:1 61.0k 238 4179 1294
With uniform I/O size, service-time 1:1 performs similarly to queue-depth since per-byte and per-request tracking are equivalent for fixed-size I/Os. service-time 4:1 with correct throughput weights improves IOPS by 3.4% and reduces latency stddev by 57%.
Test 2 - Mixed random read 4k/128k (bssplit=4k/50:128k/50,
iodepth=64, numjobs=4, 15s):
iopolicy IOPS BW(MiB/s) avg_lat(us) lat_stddev(us) -------------------- ---- --------- ----------- -------------- round-robin 7689 276 33209 46817 queue-depth 7642 276 33429 45868 service-time 1:1 7456 269 34279 47433 service-time 4:1 7747 277 32998 27102
With mixed I/O sizes on asymmetric paths, queue-depth treats 128K and 4K requests identically when counting in-flight depth, leading to suboptimal load distribution. service-time tracks actual bytes in flight, producing significantly more consistent tail latencies. service-time 4:1 vs queue-depth: latency stddev -41%.
Standby path (relative_throughput=0):
Setting a path's relative_throughput to 0 correctly excludes it from selection. With the slow path set to 0, all I/O is directed to the fast path only (IO split 118052:0), verified via diskstats.
--- drivers/nvme/host/multipath.c | 162 +++++++++++++++++++++++++++++++++- drivers/nvme/host/nvme.h | 6 ++ drivers/nvme/host/sysfs.c | 5 +- 3 files changed, 170 insertions(+), 3 deletions(-)
drivers/nvme/host/multipath.c +160 -2
@@ -69,6 +69,7 @@ static const char *nvme_iopolicy_names[] = { [NVME_IOPOLICY_NUMA] = "numa", [NVME_IOPOLICY_RR] = "round-robin", [NVME_IOPOLICY_QD] = "queue-depth", + [NVME_IOPOLICY_ST] = "service-time", };static int iopolicy =NVME_IOPOLICY_NUMA ;
@@ -107,7 +108,7 @@ static int nvme_get_iopolicy(char *buf, const struct kernel_param *kp)module_param_call (iopolicy, nvme_set_iopolicy, nvme_get_iopolicy, &iopolicy,0644 );MODULE_PARM_DESC (iopolicy, -"Default multipath I/O policy; 'numa' (default), 'round-robin' or 'queue-depth'" ); +"Default multipath I/O policy; 'numa' (default), 'round-robin', 'queue-depth' or 'service-time'" );void nvme_mpath_default_iopolicy (struct nvme_subsystem *subsys) {
@@ -179,11 +180,16 @@ void nvme_mpath_start_request(struct request *rq) {struct nvme_ns *ns = rq->q->queuedata;struct gendisk *disk = ns->head->disk; +int iopolicy =READ_ONCE (ns->head->subsys->iopolicy); -if ((READ_ONCE (ns->head->subsys->iopolicy) ==NVME_IOPOLICY_QD ) && +if (iopolicy ==NVME_IOPOLICY_QD && !(nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE )) {atomic_inc (&ns->ctrl->nr_active);nvme_req (rq)->flags |=NVME_MPATH_CNT_ACTIVE ; + }else if (iopolicy ==NVME_IOPOLICY_ST && + !(nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE_BYTES )) { +atomic64_add (blk_rq_bytes (rq), &ns->ctrl->in_flight_size); +nvme_req (rq)->flags |=NVME_MPATH_CNT_ACTIVE_BYTES ; } if (!blk_queue_io_stat(disk->queue) ||
@@ -205,6 +211,10 @@ void nvme_mpath_end_request(struct request *rq)if (nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE )atomic_dec_if_positive (&ns->ctrl->nr_active); +if (nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE_BYTES ) { +atomic64_sub (blk_rq_bytes (rq), &ns->ctrl->in_flight_size); +nvme_req (rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES ; + }if (!(nvme_req (rq)->flags &NVME_MPATH_IO_STATS ))return ;
@@ -464,6 +474,109 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)return best_opt ? best_opt : best_nonopt; } +#define ST_MAX_RELATIVE_THROUGHPUT 100 +#define ST_MAX_RELATIVE_THROUGHPUT_SHIFT 7 +#define ST_MAX_INFLIGHT_SIZE \ + ((size_t)-1 >> ST_MAX_RELATIVE_THROUGHPUT_SHIFT) + +/* +* Compare estimated service time of two paths. +* Returns negative if ns1 is better, positive if ns2 is better, 0 if equal. +* +* Service time = in_flight_size / relative_throughput +* Cross-multiply to avoid division. +*/ +static int nvme_st_compare_load (struct nvme_ns *ns1,struct nvme_ns *ns2) +{ +size_t sz1, sz2, st1, st2; +unsigned int tp1 = ns1->ctrl->relative_throughput; +unsigned int tp2 = ns2->ctrl->relative_throughput; + + sz1 =atomic64_read (&ns1->ctrl->in_flight_size); + sz2 =atomic64_read (&ns2->ctrl->in_flight_size); + +/* Case 1: same throughput — compare load directly */ +if (tp1 == tp2) +return (sz1 > sz2) - (sz1 < sz2); + +/* +* Case 2a: same load — prefer higher throughput. +* Case 2b: one path has zero throughput — prefer the other. +*/ +if (sz1 == sz2 || !tp1 || !tp2) +return tp2 - tp1; + +/* +* Case 3: general comparison via cross-multiplication. +* st1 = sz1 / tp1 +* st2 = sz2 / tp2 +* Equivalent (since tp > 0): +* sz1 * tp2 <=> sz2 * tp1 +* +* Use comparisons instead of subtractions when returning: the +* intermediate values do not fit into the int return type. +*/ +if (unlikely (sz1 >=ST_MAX_INFLIGHT_SIZE || + sz2 >=ST_MAX_INFLIGHT_SIZE )) { + sz1 >>=ST_MAX_RELATIVE_THROUGHPUT_SHIFT ; + sz2 >>=ST_MAX_RELATIVE_THROUGHPUT_SHIFT ; + } + st1 = sz1 * tp2; + st2 = sz2 * tp1; +if (st1 != st2) +return (st1 > st2) - (st1 < st2); + +/* Case 4: equal service time — prefer higher throughput */ +return tp2 - tp1; +} + +static struct nvme_ns *nvme_service_time_path (struct nvme_ns_head *head) +{ +struct nvme_ns *best_opt =NULL , *best_nonopt =NULL , *ns; +struct nvme_ns *fallback_opt =NULL , *fallback_nonopt =NULL ; + +list_for_each_entry_srcu (ns, &head->list, siblings, +srcu_read_lock_held (&head->srcu)) { +if (nvme_path_is_disabled (ns)) +continue ; + +/* +* Paths with relative_throughput == 0 are only used when no +* path with a positive value is available (matching DM +* service-time semantics). +*/ +if (!ns->ctrl->relative_throughput) { +if (ns->ana_state ==NVME_ANA_OPTIMIZED && !fallback_opt) + fallback_opt = ns; +else if (ns->ana_state ==NVME_ANA_NONOPTIMIZED && + !fallback_nonopt) + fallback_nonopt = ns; +continue ; + } + +switch (ns->ana_state) { +case NVME_ANA_OPTIMIZED : +if (!best_opt || +nvme_st_compare_load (ns, best_opt) <0 ) + best_opt = ns; +break ; +case NVME_ANA_NONOPTIMIZED : +if (!best_nonopt || +nvme_st_compare_load (ns, best_nonopt) <0 ) + best_nonopt = ns; +break ; +default : +break ; + } + } + +if (best_opt) +return best_opt; +if (best_nonopt) +return best_nonopt; +return fallback_opt ? fallback_opt : fallback_nonopt; +} +static inline bool nvme_path_is_optimized (struct nvme_ns *ns) { return nvme_ctrl_state(ns->ctrl) == NVME_CTRL_LIVE &&
@@ -491,6 +604,8 @@ inline struct nvme_ns *nvme_find_path(struct nvme_ns_head *head)return nvme_queue_depth_path (head);case NVME_IOPOLICY_RR :return nvme_round_robin_path (head); +case NVME_IOPOLICY_ST : +return nvme_service_time_path (head);default :return nvme_numa_path (head); }
@@ -1110,6 +1225,47 @@ static ssize_t queue_depth_show(struct device *dev, }DEVICE_ATTR_RO (queue_depth); +static ssize_t in_flight_bytes_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct nvme_ns *ns =nvme_get_ns_from_dev (dev); + +if (ns->head->subsys->iopolicy !=NVME_IOPOLICY_ST ) +return 0 ; + +return sysfs_emit (buf,"%lld\n" ,atomic64_read (&ns->ctrl->in_flight_size)); +} +DEVICE_ATTR_RO (in_flight_bytes); + +static ssize_t relative_throughput_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct nvme_ns *ns =nvme_get_ns_from_dev (dev); + +if (ns->head->subsys->iopolicy !=NVME_IOPOLICY_ST ) +return 0 ; + +return sysfs_emit (buf,"%u\n" , ns->ctrl->relative_throughput); +} + +static ssize_t relative_throughput_store (struct device *dev, +struct device_attribute *attr,const char *buf,size_t count) +{ +struct nvme_ns *ns =nvme_get_ns_from_dev (dev); +unsigned int val; +int ret; + + ret =kstrtouint (buf,0 , &val); +if (ret <0 ) +return ret; +if (val >ST_MAX_RELATIVE_THROUGHPUT ) +return -EINVAL ; + + ns->ctrl->relative_throughput = val; +return count; +} +DEVICE_ATTR_RW (relative_throughput); +static ssize_t numa_nodes_show (struct device *dev,struct device_attribute *attr,char *buf) {
@@ -1435,6 +1591,7 @@ void nvme_mpath_init_ctrl(struct nvme_ctrl *ctrl)mutex_init (&ctrl->ana_lock);timer_setup (&ctrl->anatt_timer, nvme_anatt_timeout,0 );INIT_WORK (&ctrl->ana_work, nvme_ana_work); + ctrl->relative_throughput =1 ; }int nvme_mpath_init_identify (struct nvme_ctrl *ctrl,struct nvme_id_ctrl *id)
@@ -1450,6 +1607,7 @@ int nvme_mpath_init_identify(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)/* initialize this in the identify path to cover controller resets */ atomic_set (&ctrl->nr_active,0 ); +atomic64_set (&ctrl->in_flight_size,0 );if (!ctrl->max_namespaces || ctrl->max_namespaces >le32_to_cpu (id->nn)) {
drivers/nvme/host/nvme.h +6 -0
@@ -268,6 +268,7 @@ enum {NVME_REQ_USERCMD = (1 <<1 ),NVME_MPATH_IO_STATS = (1 <<2 ),NVME_MPATH_CNT_ACTIVE = (1 <<3 ), + NVME_MPATH_CNT_ACTIVE_BYTES = (1 << 4), };static inline struct nvme_request *nvme_req (struct request *req)
@@ -438,6 +439,8 @@ struct nvme_ctrl {struct timer_list anatt_timer;struct work_struct ana_work;atomic_t nr_active; +atomic64_t in_flight_size; +u8 relative_throughput;#endif #ifdef CONFIG_NVME_HOST_AUTH
@@ -491,6 +494,7 @@ enum nvme_iopolicy {NVME_IOPOLICY_NUMA ,NVME_IOPOLICY_RR ,NVME_IOPOLICY_QD , + NVME_IOPOLICY_ST, }; struct nvme_subsystem {
@@ -1095,6 +1099,8 @@ extern bool multipath;extern struct device_attribute dev_attr_ana_grpid;extern struct device_attribute dev_attr_ana_state;extern struct device_attribute dev_attr_queue_depth; +extern struct device_attribute dev_attr_in_flight_bytes; +extern struct device_attribute dev_attr_relative_throughput;extern struct device_attribute dev_attr_numa_nodes;extern struct device_attribute dev_attr_delayed_removal_secs;extern struct device_attribute dev_attr_multipath_failover_count;
drivers/nvme/host/sysfs.c +4 -1
@@ -262,6 +262,8 @@ static struct attribute *nvme_ns_attrs[] = { &dev_attr_ana_grpid.attr, &dev_attr_ana_state.attr, &dev_attr_queue_depth.attr, + &dev_attr_in_flight_bytes.attr, + &dev_attr_relative_throughput.attr, &dev_attr_numa_nodes.attr, &dev_attr_delayed_removal_secs.attr, #endif
@@ -296,7 +298,8 @@ static umode_t nvme_ns_attrs_are_visible(struct kobject *kobj,if (!nvme_ctrl_use_ana (nvme_get_ns_from_dev (dev)->ctrl))return 0 ; } -if (a == &dev_attr_queue_depth.attr || a == &dev_attr_numa_nodes.attr) { +if (a == &dev_attr_queue_depth.attr || a == &dev_attr_in_flight_bytes.attr || + a == &dev_attr_relative_throughput.attr || a == &dev_attr_numa_nodes.attr) {if (nvme_disk_is_ns_head (dev_to_disk (dev)))return 0 ; }
Signature · 1 line
2.43.7
[email protected]
AI reviewRe: [PATCH v2 1/3] nvme-multipath: implement service-time iopolicy
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] nvme-multipath: in_flight_size leak in nvme_mpath_end_request() on failover - [Medium]…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] nvme-multipath: in_flight_size leak in nvme_mpath_end_request() on failover
- [Medium] nvme-multipath: in_flight_size underflow wraps in nvme_st_compare_load() --
commit 211891d7cca0785689865f3a7650f5305abaef90
Author: Guixin Liu <[email protected]>
nvme-multipath: implement service-time iopolicy
This commit adds a service-time multipath I/O policy that selects paths based on the ratio of in-flight bytes to a configurable relative throughput weight, exposing in_flight_bytes and relative_throughput sysfs attributes.
Quoted 4 lines
drivers/nvme/host/multipath.c +0
-0
[ ... ]
Quoted 19 lines
drivers/nvme/host/multipath.c +6
-1@@ -179,11 +180,16 @@ void nvme_mpath_start_request(struct request *rq) {struct nvme_ns *ns = rq->q->queuedata;struct gendisk *disk = ns->head->disk; +int iopolicy =READ_ONCE (ns->head->subsys->iopolicy); -if ((READ_ONCE (ns->head->subsys->iopolicy) ==NVME_IOPOLICY_QD ) && +if (iopolicy ==NVME_IOPOLICY_QD && !(nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE )) {atomic_inc (&ns->ctrl->nr_active);nvme_req (rq)->flags |=NVME_MPATH_CNT_ACTIVE ; + }else if (iopolicy ==NVME_IOPOLICY_ST && + !(nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE_BYTES )) { +atomic64_add (blk_rq_bytes (rq), &ns->ctrl->in_flight_size); +nvme_req (rq)->flags |=NVME_MPATH_CNT_ACTIVE_BYTES ; } if (!blk_queue_io_stat(disk->queue) ||
drivers/nvme/host/multipath.c void nvme_mpath_end_request(struct request *rq) · line 217
Quoted 7 lines
drivers/nvme/host/multipath.c +4
-0@@ -205,6 +211,10 @@ void nvme_mpath_end_request(struct request *rq)if (nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE )atomic_dec_if_positive (&ns->ctrl->nr_active); +if (nvme_req (rq)->flags &NVME_MPATH_CNT_ACTIVE_BYTES ) { +atomic64_sub (blk_rq_bytes (rq), &ns->ctrl->in_flight_size); +nvme_req (rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES ; + }
[Severity: Medium]
Can this leak bytes in ns->ctrl->in_flight_size when a request fails over to another path?
When a request fails with a path error (such as NVME_SC_ANA_TRANSITION, NVME_SC_ANA_INACCESSIBLE, NVME_SC_ANA_PERSISTENT_LOSS, NVME_SC_HOST_PATH_ERROR, or NVME_SC_HOST_ABORTED_CMD) or on a dying queue, __nvme_complete_rq() calls nvme_failover_req():
drivers/nvme/host/multipath.c:nvme_failover_req() {
...
spin_lock_irqsave(&ns->head->requeue_lock, flags);
for (bio = req->bio; bio; bio = bio->bi_next)
bio_set_dev(bio, ns->head->disk->part0);
blk_steal_bios(&ns->head->requeue_list, req);
spin_unlock_irqrestore(&ns->head->requeue_lock, flags);
nvme_req(req)->status = 0;
nvme_end_req(req);
}blk_steal_bios() detaches all bios from the request and sets req->__data_len = 0 before nvme_end_req() -> __nvme_end_req() -> nvme_mpath_end_request() is called:
block/blk-mq.c:blk_steal_bios() {
...
rq->bio = NULL;
rq->biotail = NULL;
rq->__data_len = 0;
}When nvme_mpath_end_request() subsequently calls blk_rq_bytes(rq), it reads rq->__data_len as 0 and subtracts 0 from ns->ctrl->in_flight_size while clearing NVME_MPATH_CNT_ACTIVE_BYTES. Whenever failover occurs without a controller reset (for example during an ANA state transition, or on a multipath controller without NVME_CTRL_CMIC_ANA where nvme_mpath_init_identify() returns early), those bytes remain permanently counted in ns->ctrl->in_flight_size, skewing path selection in nvme_service_time_path() and leaving in_flight_bytes non-zero when idle.
Quoted 17 lines
drivers/nvme/host/multipath.c +10
-0if (!(nvme_req (rq)->flags &NVME_MPATH_IO_STATS ))return ;@@ -464,6 +474,109 @@ static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head)return best_opt ? best_opt : best_nonopt; } +#define ST_MAX_RELATIVE_THROUGHPUT 100 +#define ST_MAX_RELATIVE_THROUGHPUT_SHIFT 7 +#define ST_MAX_INFLIGHT_SIZE \ + ((size_t)-1 >> ST_MAX_RELATIVE_THROUGHPUT_SHIFT) + +/* +* Compare estimated service time of two paths. +* Returns negative if ns1 is better, positive if ns2 is better, 0 if equal. +* +* Service time = in_flight_size / relative_throughput
drivers/nvme/host/multipath.c static struct nvme_ns *nvme_queue_depth_path(struct nvme_ns_head *head) · line 496
Quoted 10 lines
drivers/nvme/host/multipath.c +10
-0+* Cross-multiply to avoid division. +*/ +static int nvme_st_compare_load (struct nvme_ns *ns1,struct nvme_ns *ns2) +{ +size_t sz1, sz2, st1, st2; +unsigned int tp1 = ns1->ctrl->relative_throughput; +unsigned int tp2 = ns2->ctrl->relative_throughput; + + sz1 =atomic64_read (&ns1->ctrl->in_flight_size); + sz2 =atomic64_read (&ns2->ctrl->in_flight_size);
[Severity: Medium]
What happens here if ns->ctrl->in_flight_size underflows below zero after a controller reset?
When a request is started under NVME_IOPOLICY_ST, nvme_setup_cmd() calls nvme_clear_nvme_request() (which sets RQF_DONTPREP), and nvme_mpath_start_request() adds blk_rq_bytes(rq) to ns->ctrl->in_flight_size and sets NVME_MPATH_CNT_ACTIVE_BYTES.
If that request is then placed onto q->requeue_list in MQ_RQ_IDLE state without calling nvme_mpath_end_request() (either via nvme_retry_req() -> blk_mq_requeue_request(), or when a transport queue_rq callback such as nvme_rdma_queue_rq(), nvme_fc_start_fcp_op(), or nvme_loop_queue_rq() returns BLK_STS_RESOURCE after nvme_start_request()) and a controller reset occurs:
1. nvme_cancel_tagset() -> nvme_cancel_request() skips the request because blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT.
2. nvme_mpath_init_identify() resets ctrl->in_flight_size to 0 via atomic64_set(&ctrl->in_flight_size, 0).
3. When the queue is unquiesced and the request is re-dispatched, nvme_setup_cmd() preserves nvme_req(req)->flags because RQF_DONTPREP is set, and nvme_mpath_start_request() skips adding blk_rq_bytes(rq) because NVME_MPATH_CNT_ACTIVE_BYTES is already set.
4. When the request completes, nvme_mpath_end_request() unconditionally executes atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size), driving ctrl->in_flight_size negative (unlike ctrl->nr_active, which uses atomic_dec_if_positive()).
In nvme_st_compare_load(), the negative s64 returned by atomic64_read() is assigned to the unsigned size_t variables sz1 and sz2, wrapping to a large positive value near SIZE_MAX and causing nvme_service_time_path() to treat the path as heavily overloaded until enough new I/O leaks or resets it. Should in_flight_size be prevented from going negative or clamped when read?
[ ... ]
Quoted 8 lines
drivers/nvme/host/multipath.c +1
-0@@ -1453,6 +1610,7 @@ int nvme_mpath_init_identify(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id)/* initialize this in the identify path to cover controller resets */ atomic_set (&ctrl->nr_active,0 ); +atomic64_set (&ctrl->in_flight_size,0 );if (!ctrl->max_namespaces || ctrl->max_namespaces >le32_to_cpu (id->nn)) {
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH v2 2/3 nvme-multipath: document service-time iopolicy
In reply to Guixin LiuAdd documentation for the service-time path selection policy: its algorithm, the in_flight_bytes and relative_throughput sysfs attributes, and guidance on when to prefer it over queue-depth.
Document that setting relative_throughput to 0 makes the path a standby that is only selected when no path with a positive value is available.
--- Documentation/admin-guide/nvme-multipath.rst | 31 ++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-)
Documentation/admin-guide/nvme-multipath.rst +29 -2
@@ -24,8 +24,8 @@ Policies All policies follow the ANA (Asymmetric Namespace Access) mechanism, meaning that when an optimized path is available, it will be chosen over a non-optimized -one. Current the NVMe multipath policies include numa(default), round-robin and -queue-depth. +one. Current the NVMe multipath policies include numa(default), round-robin, +queue-depth and service-time. To set the desired policy (e.g., round-robin), use one of the following methods: 1. echo -n "round-robin" > /sys/module/nvme_core/parameters/iopolicy
@@ -70,3 +70,30 @@ When to use the queue-depth policy: 1. High load with small I/Os: Effectively balances load across paths when the load is high, and I/O operations consist of small, relatively fixed-sized requests. + + +Service-Time +------------ + +The service-time policy selects the path with the lowest estimated service time. +It calculates service time as ``in_flight_bytes / relative_throughput`` for each +path, preferring the path that would complete I/O fastest. Unlike queue-depth +which counts requests regardless of size, service-time tracks actual bytes in +flight, making it aware of I/O sizes. + +Each path exposes two sysfs attributes under +``/sys/class/nvme/nvmeX/nvmeXcYnZ/``: + + - ``in_flight_bytes`` (read-only): Current bytes in flight on this path. + - ``relative_throughput`` (read-write): Relative throughput weight for this + path, default 1. The valid range is 0-100. Set higher values for faster + paths. If set to 0, the path is not selected while other paths with + positive values are available. + +When to use the service-time policy: + 1. Asymmetric Link Speeds: When paths have different bandwidths, set + ``relative_throughput`` proportionally (e.g., 2 for a link twice as fast) + to steer more traffic to faster paths. + 2. Mixed I/O Sizes: When workloads mix small and large I/Os (e.g., 4K and + 128K), service-time distributes load more accurately than queue-depth + because it accounts for actual bytes rather than request count.
Signature · 1 line
2.43.7
PATCH v2 3/3 nvme: document the service-time iopolicy sysfs attributes
In reply to Guixin LiuThe service-time iopolicy adds two sysfs attributes to the NVMe path namespace devices: in_flight_bytes, reporting the bytes currently in flight on the path's controller, and relative_throughput, the per-path throughput weight used by the service-time estimation.
Document both in the stable sysfs ABI, including the 0-100 value range, the default of 1, and the standby semantics of a relative_throughput of 0.
--- Documentation/ABI/stable/sysfs-nvme | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
Documentation/ABI/stable/sysfs-nvme +22 -0
@@ -326,6 +326,28 @@ Description: currently selected path. Returns empty if iopolicy is not "numa". Requires CONFIG_NVME_MULTIPATH. +What: /sys/block/nvmeXcYnZ/in_flight_bytes +What: /sys/block/nvmeXcYnZ/relative_throughput +Date: October 2026 +KernelVersion: 7.5 +Contact: Guixin Liu <[email protected]> +Description: + in_flight_bytes: Shows the number of bytes currently in + flight on this path's controller. Returns empty if + iopolicy is not "service-time". + Requires CONFIG_NVME_MULTIPATH. + + relative_throughput: Shows or sets the relative + throughput weight of this path, which the "service-time" + iopolicy uses to estimate the path service time as + in_flight_bytes / relative_throughput. The value is an + integer in the range 0-100 and defaults to 1; assign + higher values to faster paths. A path with a value of 0 + is only selected when no path with a positive value is + available. Reading returns empty if iopolicy is not + "service-time"; writing is accepted regardless of the + active policy. Requires CONFIG_NVME_MULTIPATH. + What: /sys/block/nvmeXnY/delayed_removal_secs Date: May 2025 KernelVersion: 6.16
Signature · 1 line
2.43.7
[email protected]
AI reviewRe: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] nvme-multipath: nvme_failover_req() clears rq bytes before in_flight_size sub - [High]…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] nvme-multipath: nvme_failover_req() clears rq bytes before in_flight_size sub
- [High] nvme-multipath: in_flight_size underflow in nvme_mpath_end_request()
- [Low] nvme: missing service-time in sysfs-nvme iopolicy ABI documentation --
commit 5acf4057a11f40dc3e2cd32f04002286c4084ff4
Author: Guixin Liu <[email protected]>
nvme: document the service-time iopolicy sysfs attributes
This commit documents the new in_flight_bytes and relative_throughput sysfs attributes added for the service-time NVMe multipath I/O policy in Documentation/ABI/stable/sysfs-nvme.
Quoted 8 lines
Documentation/ABI/stable/sysfs-nvme +0
-0@@ -326,6 +326,28 @@ Description: currently selected path. Returns empty if iopolicy is not "numa". Requires CONFIG_NVME_MULTIPATH.
Documentation/ABI/stable/sysfs-nvme Description: · line 338
Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +10
-0+What: /sys/block/nvmeXcYnZ/in_flight_bytes +What: /sys/block/nvmeXcYnZ/relative_throughput +Date: October 2026 +KernelVersion: 7.5 +Contact: Guixin Liu <[email protected]> +Description: + in_flight_bytes: Shows the number of bytes currently in + flight on this path's controller. Returns empty if + iopolicy is not "service-time". + Requires CONFIG_NVME_MULTIPATH.
[Severity: High]
Can ns->ctrl->in_flight_size permanently leak in-flight bytes when a request fails over in nvme_failover_req()?
In commit 211891d7cca0 ("nvme-multipath: implement service-time iopolicy"), nvme_mpath_start_request() adds blk_rq_bytes(rq) to ns->ctrl->in_flight_size and sets NVME_MPATH_CNT_ACTIVE_BYTES:
drivers/nvme/host/multipath.c:nvme_mpath_start_request() {
...
} else if (iopolicy == NVME_IOPOLICY_ST &&
!(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES)) {
atomic64_add(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE_BYTES;
}
}When a request fails over (for example, on NVME_SC_ANA_TRANSITION, NVME_SC_HOST_PATH_ERROR, or a dying queue), __nvme_complete_rq() calls nvme_failover_req(), which steals the bios from req before calling nvme_end_req():
drivers/nvme/host/multipath.c:nvme_failover_req() {
...
blk_steal_bios(&ns->head->requeue_list, req);
spin_unlock_irqrestore(&ns->head->requeue_lock, flags);
nvme_req(req)->status = 0;
nvme_end_req(req);
}Because blk_steal_bios() clears rq->__data_len:
block/blk-mq.c:blk_steal_bios() {
...
rq->__data_len = 0;
}by the time nvme_end_req() -> __nvme_end_req() -> nvme_mpath_end_request() runs:
drivers/nvme/host/multipath.c:nvme_mpath_end_request() {
...
if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES) {
atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size);
nvme_req(rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES;
}
}blk_rq_bytes(rq) is 0, so the failed-over request's bytes are never subtracted from ns->ctrl->in_flight_size. When failover occurs without a controller reset, does ns->ctrl->in_flight_size remain permanently inflated, skewing nvme_st_compare_load() and the in_flight_bytes sysfs attribute?
[Severity: High]
Also in nvme_mpath_end_request(), can ns->ctrl->in_flight_size underflow to a negative value if a requeued request completes after a controller reset?
When a request started under NVME_IOPOLICY_ST is requeued via nvme_retry_req() -> blk_mq_requeue_request() (or via BLK_STS_RESOURCE after nvme_start_request() in nvme_rdma_queue_rq() or nvme_fc_start_fcp_op()), the request returns to MQ_RQ_IDLE while retaining RQF_DONTPREP and NVME_MPATH_CNT_ACTIVE_BYTES.
If a controller reset occurs before that request is re-dispatched, nvme_mpath_init_identify() resets ctrl->in_flight_size to 0:
drivers/nvme/host/multipath.c:nvme_mpath_init_identify() {
...
/* initialize this in the identify path to cover controller resets */
atomic_set(&ctrl->nr_active, 0);
atomic64_set(&ctrl->in_flight_size, 0);
}When the request is later re-dispatched, nvme_setup_cmd() skips nvme_clear_nvme_request() because RQF_DONTPREP is set, and nvme_mpath_start_request() skips atomic64_add() because NVME_MPATH_CNT_ACTIVE_BYTES is already set. When the request finishes, nvme_mpath_end_request() unconditionally subtracts blk_rq_bytes(rq) from ns->ctrl->in_flight_size (unlike nr_active, which uses atomic_dec_if_positive()), driving ctrl->in_flight_size negative.
In nvme_st_compare_load(), the negative s64 is then assigned to unsigned size_t:
drivers/nvme/host/multipath.c:nvme_st_compare_load() {
size_t sz1, sz2, st1, st2;
unsigned int tp1 = ns1->ctrl->relative_throughput;
unsigned int tp2 = ns2->ctrl->relative_throughput;
sz1 = atomic64_read(&ns1->ctrl->in_flight_size);
sz2 = atomic64_read(&ns2->ctrl->in_flight_size);Would this wrap sz1 or sz2 to near SIZE_MAX, causing nvme_service_time_path() to avoid that controller's path while reporting a negative value in in_flight_bytes?
Quoted 5 lines
Documentation/ABI/stable/sysfs-nvme +5
-0+ + relative_throughput: Shows or sets the relative + throughput weight of this path, which the "service-time" + iopolicy uses to estimate the path service time as + in_flight_bytes / relative_throughput. The value is an
Documentation/ABI/stable/sysfs-nvme Description: · line 353
Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +7
-0+ integer in the range 0-100 and defaults to 1; assign + higher values to faster paths. A path with a value of 0 + is only selected when no path with a positive value is + available. Reading returns empty if iopolicy is not + "service-time"; writing is accepted regardless of the + active policy. Requires CONFIG_NVME_MULTIPATH. + What: /sys/block/nvmeXnY/delayed_removal_secs Date: May 2025 KernelVersion: 6.16
[Severity: Low]
In the same file (Documentation/ABI/stable/sysfs-nvme), the entry for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy still lists only the three previous policies:
What: /sys/class/nvme-subsystem/nvme-subsysX/iopolicy
...
Description:
Shows or sets the multipath I/O path selection policy for this subsystem. Accepted values: "numa", "round-robin", "queue-depth". Changing the policy clears all current path selections. Only available when CONFIG_NVME_MULTIPATH is enabled.
Should "service-time" also be added to the accepted values for /sys/class/nvme-subsystem/nvme-subsysX/iopolicy?
هایی
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
Quoted 82 lines · 在 2026/10/19 16:13, [email protected] 写道:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] nvme-multipath:nvme_failover_req()clears rq bytes beforein_flight_sizesub
- [High] nvme-multipath:in_flight_sizeunderflow innvme_mpath_end_request()
- [Low] nvme: missing service-time in sysfs-nvme iopolicy ABI documentation --commit
5acf4057a11f40dc3e2cd32f04002286c4084ff4
Author: Guixin Liu <[email protected]>nvme: document the service-time iopolicy sysfs attributes
This commit documents the new
in_flight_bytesandrelative_throughputsysfs attributes added for the service-time NVMe multipath I/O policy inDocumentation/ABI/stable/sysfs-nvme.Quoted 8 lines
Documentation/ABI/stable/sysfs-nvme +0
-0@@ -326,6 +326,28 @@ Description: currently selected path. Returns empty if iopolicy is not "numa". Requires CONFIG_NVME_MULTIPATH.Documentation/ABI/stable/sysfs-nvme Description: · line 338
Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +10
-0+What: /sys/block/nvmeXcYnZ/in_flight_bytes +What: /sys/block/nvmeXcYnZ/relative_throughput +Date: October 2026 +KernelVersion: 7.5 +Contact: Guixin Liu <[email protected]> +Description: + in_flight_bytes: Shows the number of bytes currently in + flight on this path's controller. Returns empty if + iopolicy is not "service-time". + Requires CONFIG_NVME_MULTIPATH.[Severity: High]
Canns->ctrl->in_flight_sizepermanently leak in-flight bytes when a request fails over innvme_failover_req()?In commit
211891d7cca0("nvme-multipath: implement service-time iopolicy"),nvme_mpath_start_request()addsblk_rq_bytes(rq)tons->ctrl->in_flight_sizeand setsNVME_MPATH_CNT_ACTIVE_BYTES:drivers/nvme/host/multipath.c:nvme_mpath_start_request() { ... } else if (iopolicy == NVME_IOPOLICY_ST && !(nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES)) { atomic64_add(blk_rq_bytes(rq), &ns->ctrl->in_flight_size); nvme_req(rq)->flags |= NVME_MPATH_CNT_ACTIVE_BYTES; } }When a request fails over (for example, on
NVME_SC_ANA_TRANSITION,NVME_SC_HOST_PATH_ERROR, or a dying queue),__nvme_complete_rq()callsnvme_failover_req(), which steals the bios from req before callingnvme_end_req():drivers/nvme/host/multipath.c:nvme_failover_req() { ... blk_steal_bios(&ns->head->requeue_list, req); spin_unlock_irqrestore(&ns->head->requeue_lock, flags); nvme_req(req)->status = 0; nvme_end_req(req); }Because
blk_steal_bios()clearsrq->__data_len:block/blk-mq.c:blk_steal_bios() { ... rq->__data_len = 0; }by the time
nvme_end_req()->__nvme_end_req()->nvme_mpath_end_request()runs:drivers/nvme/host/multipath.c:nvme_mpath_end_request() { ... if (nvme_req(rq)->flags & NVME_MPATH_CNT_ACTIVE_BYTES) { atomic64_sub(blk_rq_bytes(rq), &ns->ctrl->in_flight_size); nvme_req(rq)->flags &= ~NVME_MPATH_CNT_ACTIVE_BYTES; } }
Quoted 5 lines
blk_rq_bytes(rq)is 0, so the failed-over request's bytes are never subtracted fromns->ctrl->in_flight_size. When failover occurs without a controller reset, doesns->ctrl->in_flight_sizeremain permanently inflated, skewingnvme_st_compare_load()and thein_flight_bytessysfs attribute?
Right, fixed in v3.
Quoted 41 lines
[Severity: High]
Also innvme_mpath_end_request(), canns->ctrl->in_flight_sizeunderflow to a negative value if a requeued request completes after a controller reset?When a request started under
NVME_IOPOLICY_STis requeued vianvme_retry_req()->blk_mq_requeue_request()(or viaBLK_STS_RESOURCEafternvme_start_request()innvme_rdma_queue_rq()ornvme_fc_start_fcp_op()), the request returns toMQ_RQ_IDLEwhile retainingRQF_DONTPREPandNVME_MPATH_CNT_ACTIVE_BYTES.If a controller reset occurs before that request is re-dispatched,
nvme_mpath_init_identify()resetsctrl->in_flight_sizeto 0:drivers/nvme/host/multipath.c:nvme_mpath_init_identify() { ... /* initialize this in the identify path to cover controller resets */ atomic_set(&ctrl->nr_active, 0); atomic64_set(&ctrl->in_flight_size, 0); }When the request is later re-dispatched,
nvme_setup_cmd()skipsnvme_clear_nvme_request()becauseRQF_DONTPREPis set, andnvme_mpath_start_request()skipsatomic64_add()becauseNVME_MPATH_CNT_ACTIVE_BYTESis already set. When the request finishes,nvme_mpath_end_request()unconditionally subtractsblk_rq_bytes(rq)fromns->ctrl->in_flight_size(unlikenr_active, which usesatomic_dec_if_positive()), drivingctrl->in_flight_sizenegative.In
nvme_st_compare_load(), the negative s64 is then assigned to unsignedsize_t:drivers/nvme/host/multipath.c:nvme_st_compare_load() { size_t sz1, sz2, st1, st2; unsigned int tp1 = ns1->ctrl->relative_throughput; unsigned int tp2 = ns2->ctrl->relative_throughput; sz1 = atomic64_read(&ns1->ctrl->in_flight_size); sz2 = atomic64_read(&ns2->ctrl->in_flight_size);
Quoted 3 lines
Would this wrap sz1 or sz2 to near
SIZE_MAX, causingnvme_service_time_path()to avoid that controller's path while reporting a negative value inin_flight_bytes?
Yes, fixed in v3.
Quoted 29 lines
Quoted 5 lines
Documentation/ABI/stable/sysfs-nvme +5
-0+ + relative_throughput: Shows or sets the relative + throughput weight of this path, which the "service-time" + iopolicy uses to estimate the path service time as + in_flight_bytes / relative_throughput. The value is anDocumentation/ABI/stable/sysfs-nvme Description: · line 353
Quoted 10 lines
Documentation/ABI/stable/sysfs-nvme +7
-0+ integer in the range 0-100 and defaults to 1; assign + higher values to faster paths. A path with a value of 0 + is only selected when no path with a positive value is + available. Reading returns empty if iopolicy is not + "service-time"; writing is accepted regardless of the + active policy. Requires CONFIG_NVME_MULTIPATH. + What: /sys/block/nvmeXnY/delayed_removal_secs Date: May 2025 KernelVersion: 6.16[Severity: Low]
In the same file (Documentation/ABI/stable/sysfs-nvme), the entry for/sys/class/nvme-subsystem/nvme-subsysX/iopolicystill lists only the three previous policies:What:
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy
...
Description:
Shows or sets the multipath I/O path selection policy for this subsystem. Accepted values: "numa", "round-robin", "queue-depth". Changing the policy clears all current path selections. Only available whenCONFIG_NVME_MULTIPATHis enabled.
Quoted 4 lines
Should "service-time" also be added to the accepted values for
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy?هایی
Sure, changed in v3.
Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
Quoted 6 lines · Guixin Liu
Quoted 5 lines
Should "service-time" also be added to the accepted values for
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy?هایی
Sure, changed in v3.
Not just /sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in /sys/class/nvme- subsystem/nvme-subsysX/iopolicies as well.
-Martin
Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
Quoted 5 lines · Martin George
Quoted 4 lines · Guixin Liu
Quoted 4 lines
Should "service-time" also be added to the accepted values for
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy?
Quoted 8 lines
Quoted 2 lines
Quoted 1 line
هایی
Sure, changed in v3.
Not just
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in/sys/class/nvme-subsystem/nvme-subsysX/iopolicies as well.-Martin
Hello
There should be no changes needed for my original iopolicies patch
iopolicies_show() walks ARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up in iopolicies automatically. The existing iopolicy store path already matches against the same array, so both attributes stay in sync without code changes
Thanks
Laurence
Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
Quoted 14 lines · Laurence Oberman
Quoted 11 lines · Martin George
Quoted 5 lines · Guixin Liu
Quoted 4 lines
Should "service-time" also be added to the accepted values for
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy?هایی
Sure, changed in v3.
Not just
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in/sys/class/nvme-subsystem/nvme-subsysX/iopolicies as well.-Martin
Hello
Quoted 8 lines
There should be no changes needed for my original iopolicies patch
iopolicies_show()walksARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up in iopolicies automatically. The existing iopolicy store path already matches against the same array, so both attributes stay in sync without code changes
Yes, it is.
Best Regards,
Guixin Liu
Quoted 3 lines
Thanks
Laurence
Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
In reply to Laurence ObermanQuoted 18 lines · Laurence Oberman
Quoted 5 lines · Martin George
Quoted 4 lines · Guixin Liu
Quoted 4 lines
Should "service-time" also be added to the accepted values for
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy?Quoted 8 lines
Quoted 2 lines
Quoted 1 line
هایی
Sure, changed in v3.
Not just
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this should be added to the accepted values in/sys/class/nvme-subsystem/nvme-subsysX/iopolicies as well.-Martin
Hello
There should be no changes needed for my original iopolicies patch
Quoted 9 lines
iopolicies_show()walksARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up
in iopolicies automatically. The existing iopolicy store path already matches against the same array, so both attributes stay in sync without
code changes
I meant adding to the iopolicies documentation here.
-Martin
Re: [PATCH v2 3/3] nvme: document the service-time iopolicy sysfs attributes
Quoted 32 lines · Martin George
Quoted 20 lines · Laurence Oberman
Quoted 7 lines · Martin George
Quoted 6 lines · Guixin Liu
Quoted 6 lines
Should "service-time" also be added to the accepted values for
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy?هایی
Quoted 8 lines
Quoted 1 line
Sure, changed in v3.
Not just
/sys/class/nvme-subsystem/nvme-subsysX/iopolicy, but this
should be added to the accepted values in/sys/class/nvme-subsystem/nvme-subsysX/iopolicies as well.-Martin
Hello
There should be no changes needed for my original iopolicies patch
Quoted 10 lines
iopolicies_show()walksARRAY_SIZE(nvme_iopolicy_names), so whoever adds a service-time policy adds its enum value and its string to nvme_iopolicy_names[] as part of that work. The new name then shows up
in iopolicies automatically. The existing iopolicy store path already
matches against the same array, so both attributes stay in sync without
code changes
Quoted 3 lines
I meant adding to the iopolicies documentation here.
-Martin
Thanks, makes perfect sense
Regards
Laurence