block: make changing to the current I/O scheduler cheap again
v2 cover block: make changing to the current I/O scheduler cheap again
Writing the scheduler a queue already uses to queue/scheduler takes about 10 ms and freezes the queue, which stalls I/O on live disks every time a udev rule or tuning tool sets the scheduler unconditionally.
Patch 1 stops requesting a "none-iosched" module on every write of "none". Patch 2 returns before freezing the queue when the elevator wouldn't change.
On a null_blk disk (median of 20 writes, all kernels measured in the same run):
operation base (ms) patch 1 (ms) patch 1+2 (ms)
none -> none 13.10 9.11 0.08 mq-deadline -> mq-deadline 10.95 9.05 0.08 mq-deadline -> none 13.00 7.97 6.03
---
V1 -> V2:
- Keep the elevator check under the freeze and note the race in the comment, as Christoph suggested
- Say "change" instead of "rewrite" in the cover letter subject
- Rebase onto v7.4-rc2-470-g7e1392a42ea2 and update the numbers from a new run
v1: https://lore.kernel.org/r/20261111-bug-block-elevator-unchanged-v1-0-583198bd4e21@linux.dev
---
Qiliang Yuan (2):
block: don't request a module for the none elevator
block: return early when the elevator doesn't change
block/elevator.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
---
base-commit: 7e1392a42ea288d70a266f6553fe06f57f8647ea
change-id: 20261111-bug-block-elevator-unchanged-a6cd2d761d38
Best regards,
Signature · 1 line
Qiliang Yuan <[email protected]>
PATCH v2 1/2 block: don't request a module for the none elevator
elv_iosched_store() calls elv_iosched_load_module() for the requested name before anything else, and loads "<name>-iosched" if no elevator type of that name is registered. "none" never is, so every write of "none" to queue/scheduler ends up in request_module("none-iosched").
That runs a modprobe helper which can never find the module, and adds several milliseconds to every switch to "none", including rewrites of it by udev rules or tuning tools that set the scheduler on each disk.
Return early for "none", like elevator_change() did before the module loading moved into elv_iosched_load_module().
Writing queue/scheduler of a null_blk disk 20 times takes (median):
operation before (ms) after (ms) change
none -> none 13.10 9.11 -30% mq-deadline -> none 13.00 7.97 -39%
--- block/elevator.c | 3 +++ 1 file changed, 3 insertions(+)
block/elevator.c +3 -0
@@ -775,6 +775,9 @@ static void elv_iosched_load_module(const char *elevator_name) {struct elevator_type *found; +if (!strncmp (elevator_name,"none" ,4 )) +return ; +spin_lock (&elv_list_lock); found =__elevator_find (elevator_name);spin_unlock (&elv_list_lock);
Signature · 1 line
2.43.0
PATCH v2 2/2 block: return early when the elevator doesn't change
In reply to Qiliang YuanWriting the scheduler a queue already uses to queue/scheduler used to be a no-op. elevator_change() now only checks for an unchanged elevator after freezing the queue, and doesn't treat "none" on a queue without an elevator as unchanged at all, so that case also quiesces the queue in elevator_switch() and resets nr_requests.
Every such write therefore drains in-flight I/O and waits for an RCU grace period. udev rules and tuning tools that set the scheduler unconditionally stall I/O on live disks each time they run.
Check under elevator_lock whether anything would change before allocating scheduler resources and freezing the queue. The check can race with another elevator change, which at worst leads to a redundant switch, so keep the one done under the freeze as well.
Writing the current scheduler to queue/scheduler of a null_blk disk 20 times takes (median):
operation before (ms) after (ms) change
none -> none 9.11 0.08 -99% mq-deadline -> mq-deadline 9.05 0.08 -99%
4k random reads at queue depth 1 for 10 seconds, while "none" is written to queue/scheduler every 50 ms:
metric before after change
IOPS 340562 390392 +15%
max latency (us) 16302 1224 -92%
--- block/elevator.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
block/elevator.c +14 -0
@@ -654,10 +654,24 @@ static int elevator_change(struct request_queue *q, struct elv_change_ctx *ctx) {unsigned int memflags;struct blk_mq_tag_set *set = q->tag_set; +bool unchanged;int ret =0 ;lockdep_assert_held (&set->update_nr_hwq_lock); +/* +* Skip freezing the queue if nothing would change. This can race with +* another elevator change, which at worst leads to a redundant switch. +*/ +mutex_lock (&q->elevator_lock); +if (q->elevator) + unchanged =elevator_match (q->elevator->type, ctx->name); +else + unchanged = !strncmp (ctx->name,"none" ,4 ); +mutex_unlock (&q->elevator_lock); +if (unchanged) +return 0 ; +if (strncmp (ctx->name,"none" ,4 )) { ret =blk_mq_alloc_sched_res (q, ctx->type, &ctx->res, set->nr_hw_queues);
Signature · 1 line
2.43.0
Re: [PATCH v2 2/2] block: return early when the elevator doesn't change
Looks good:
Quoted 1 line · Qiliang Yuan
Fixes b07a889e83355 ("block: move q->sysfs_lock and queue-freeze under show/store method")
But this fixes tag is not correct, that commit just moved code around.
Quoted 9 lines · Qiliang Yuan
Writing the scheduler a queue already uses to queue/scheduler takes about 10 ms and freezes the queue, which stalls I/O on live disks every time a udev rule or tuning tool sets the scheduler unconditionally.
Patch 1 stops requesting a "none-iosched" module on every write of "none". Patch 2 returns before freezing the queue when the elevator wouldn't change.
[...]
Applied, thanks!
[1/2] block: don't request a module for the none elevator
commit: c85f570121c94679854b570cae2c0dbcc0eae43b
[2/2] block: return early when the elevator doesn't change
commit: 375a88c6176ffe05e521c2df31b1ceaf4ae5fbafBest regards,
Signature · 1 line
Jens Axboe
Quoted 2 lines · Jens Axboe
Applied, thanks!
[...]
Quoted 2 lines
[2/2] block: return early when the elevator doesn't change
commit: 375a88c6176ffe05e521c2df31b1ceaf4ae5fbaf
This one went in from v2, whose Fixes tag Christoph pointed out was wrong. v3 changed it to the commit that added the freeze, with the code unchanged:
Could you update it in for-next?
v3: https://lore.kernel.org/r/20261116-bug-block-elevator-unchanged-v3-2-cb4dda2dadf4@linux.dev
Thanks,
Qiliang