nvme: introduce the service-time iopolicy
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest3 patches 2 repliesIn review
- v33 patches 2 repliesSuperseded
- v23 patches 6 repliesSuperseded
Earlier versions · 1
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v1 | 2 patches | 3 replies | Superseded |
Changes in v4 from the cover letter
- Rebased onto nvme-7.4, which adds the iopolicies and path_state attributes and the discovery-subsystem "none" handling.
- The iopolicies ABI entry now lists service-time too. nvme-7.4 already documents the attribute (with a hardcoded list of the three older policies); patch 3 extends that list along with the iopolicy accepted values.
Patches in v4 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | nvme-multipath: implement service-time iopolicydrivers/ drivers/ drivers/ | +198 | 0 replies | |
| 2/3 | nvme-multipath: document service-time iopolicyDocumentation/ | +29 | 0 replies | |
| 3/3 | nvme: document the service-time iopolicy sysfs attributesDocumentation/ | +29 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.
Cover letter [PATCH v4 0/3] nvme: introduce the service-time iopolicy
The queue-depth iopolicy counts requests, so a 128K I/O weighs the same as a 4K one, and it cannot express that one path is simply faster than another. This series adds a service-time iopolicy for NVMe native multipath, adapted from dm-ps-service-time: it picks the path with the lowest estimated service time, in_flight_bytes / relative_throughput, so balancing follows actual bytes in flight and per-path throughput weights.
Weights live in a per-path relative_throughput sysfs attribute (0-100, default 1, 0 makes the path a standby). Patch 3 documents the new attributes in the stable sysfs ABI; patch 1 carries the benchmark data in its commit log.
This version is rebased onto the nvme-7.4 branch ofgit://git.infradead.org/nvme.git.
Changes since v3:
- Rebased onto nvme-7.4, which adds the iopolicies and path_state attributes and the discovery-subsystem "none" handling.
- The iopolicies ABI entry now lists service-time too. nvme-7.4 already documents the attribute (with a hardcoded list of the three older policies); patch 3 extends that list along with the iopolicy accepted values.
Changes since v2:
- in_flight_bytes no longer leaks on failover: the accounted bytes are now released in nvme_failover_req() before blk_steal_bios() clears the request data length. Before, every failover (ANA transition, path error, queue_rq unwind) permanently left the request's bytes in the counter, skewing path selection until a controller reset. Reproduced with an ANA transition during load on the NVMe-oF/TCP test setup: the counter kept 512K of phantom bytes after all I/O completed; with the fix it returns to zero.
- The release clamps at zero instead of a plain subtraction. A request requeued across a controller reset is accounted against the pre-reset counter but completes after nvme_mpath_init_identify() re-zeroed it, and the plain subtraction would wrap negative, making the comparator treat the path as pathologically overloaded. This mirrors atomic_dec_if_positive() in the queue-depth accounting.
- The comparison operates on u64 instead of size_t, so the byte counts and the shift guard keep their magnitude on 32-bit builds, where size_t would truncate the atomic64 reads.
- Added "service-time" to the accepted values of the subsystem iopolicy ABI entry and to the policies listed by the iopolicies ABI entry.
No public v3 was posted; this version folds the review feedback from the v2 discussion and a rebase onto nvme-7.4.
On the adaptive I/O policy dependency Hannes raised during v1: that series has become the latency I/O policy and is still under revision. The two are complementary: the latency policy infers path speed from observed completions, service-time takes configured weights and byte counts.
Guixin Liu (3):
nvme-multipath: implement service-time iopolicy
nvme-multipath: document service-time iopolicy
nvme: document the service-time iopolicy sysfs attributes
Documentation/ABI/stable/sysfs-nvme | 35 +++- Documentation/admin-guide/nvme-multipath.rst | 31 +++- drivers/nvme/host/multipath.c | 189 ++++++++++++++++++- drivers/nvme/host/nvme.h | 6 + drivers/nvme/host/sysfs.c | 7 +- 5 files changed, 256 insertions(+), 12 deletions(-)
v2 of this series:
https://lore.kernel.org/r/[email protected]
v1 of this series:
https://lore.kernel.org/r/[email protected]
latency I/O policy v7:
https://lore.kernel.org/r/[email protected]
Signature · 1 line
2.43.7