Skip to contentSkip to conversation

block: make changing to the current I/O scheduler cheap again

PATCH Applied Qiliang Yuan 3 participants 6 messages Nov 15 – Nov 19 Block layer

v2 of 3 Applied Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover block: make changing to the current I/O scheduler cheap again

Writing the scheduler a queue already uses to queue/scheduler takes about 10 ms and freezes the queue, which stalls I/O on live disks every time a udev rule or tuning tool sets the scheduler unconditionally.

Patch 1 stops requesting a "none-iosched" module on every write of "none". Patch 2 returns before freezing the queue when the elevator wouldn't change.

On a null_blk disk (median of 20 writes, all kernels measured in the same run):

operation base (ms) patch 1 (ms) patch 1+2 (ms)

  none -> none                    13.10          9.11            0.08
  mq-deadline -> mq-deadline      10.95          9.05            0.08
  mq-deadline -> none             13.00          7.97            6.03

---
V1 -> V2:
- Keep the elevator check under the freeze and note the race in the comment, as Christoph suggested
- Say "change" instead of "rewrite" in the cover letter subject
- Rebase onto v7.4-rc2-470-g7e1392a42ea2 and update the numbers from a new run

v1: https://lore.kernel.org/r/20261111-bug-block-elevator-unchanged-v1-0-583198bd4e21@linux.dev

---
Qiliang Yuan (2):
block: don't request a module for the none elevator
block: return early when the elevator doesn't change

 block/elevator.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

---
base-commit: 7e1392a42ea288d70a266f6553fe06f57f8647ea
change-id: 20261111-bug-block-elevator-unchanged-a6cd2d761d38

Best regards,

Signature · 1 line
Qiliang Yuan <[email protected]>
#1 lore raw

PATCH v2 1/2 block: don't request a module for the none elevator

elv_iosched_store() calls elv_iosched_load_module() for the requested name before anything else, and loads "<name>-iosched" if no elevator type of that name is registered. "none" never is, so every write of "none" to queue/scheduler ends up in request_module("none-iosched").

That runs a modprobe helper which can never find the module, and adds several milliseconds to every switch to "none", including rewrites of it by udev rules or tuning tools that set the scheduler on each disk.

Return early for "none", like elevator_change() did before the module loading moved into elv_iosched_load_module().

Writing queue/scheduler of a null_blk disk 20 times takes (median):

operation before (ms) after (ms) change

  none -> none                13.10        9.11    -30%
  mq-deadline -> none         13.00        7.97    -39%
Fixes 734e1a8603128 ("block: Prevent deadlocks when switching elevators")
---
 block/elevator.c | 3 +++
 1 file changed, 3 insertions(+)
block/elevator.c +3 -0
index 66edf1a5768a8..9d9b2e84a1288 100644
@@ -775,6 +775,9 @@ static void elv_iosched_load_module(const char *elevator_name)
 {
 	struct elevator_type *found;
 
+	if (!strncmp(elevator_name, "none", 4))
+		return;
+
 	spin_lock(&elv_list_lock);
 	found = __elevator_find(elevator_name);
 	spin_unlock(&elv_list_lock);
Signature · 1 line
2.43.0
#2 lore raw

PATCH v2 2/2 block: return early when the elevator doesn't change

In reply to Qiliang Yuan

Writing the scheduler a queue already uses to queue/scheduler used to be a no-op. elevator_change() now only checks for an unchanged elevator after freezing the queue, and doesn't treat "none" on a queue without an elevator as unchanged at all, so that case also quiesces the queue in elevator_switch() and resets nr_requests.

Every such write therefore drains in-flight I/O and waits for an RCU grace period. udev rules and tuning tools that set the scheduler unconditionally stall I/O on live disks each time they run.

Check under elevator_lock whether anything would change before allocating scheduler resources and freezing the queue. The check can race with another elevator change, which at worst leads to a redundant switch, so keep the one done under the freeze as well.

Writing the current scheduler to queue/scheduler of a null_blk disk 20 times takes (median):

operation before (ms) after (ms) change

  none -> none                     9.11        0.08    -99%
  mq-deadline -> mq-deadline       9.05        0.08    -99%

4k random reads at queue depth 1 for 10 seconds, while "none" is written to queue/scheduler every 50 ms:

metric before after change
IOPS 340562 390392 +15%
max latency (us) 16302 1224 -92%

Fixes b07a889e83355 ("block: move q->sysfs_lock and queue-freeze under show/store method")
---
 block/elevator.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
block/elevator.c +14 -0
index 9d9b2e84a1288..549cf51543a8b 100644
@@ -654,10 +654,24 @@ static int elevator_change(struct request_queue *q, struct elv_change_ctx *ctx)
 {
 	unsigned int memflags;
 	struct blk_mq_tag_set *set = q->tag_set;
+	bool unchanged;
 	int ret = 0;
 
 	lockdep_assert_held(&set->update_nr_hwq_lock);
 
+	/*
+	 * Skip freezing the queue if nothing would change. This can race with
+	 * another elevator change, which at worst leads to a redundant switch.
+	 */
+	mutex_lock(&q->elevator_lock);
+	if (q->elevator)
+		unchanged = elevator_match(q->elevator->type, ctx->name);
+	else
+		unchanged = !strncmp(ctx->name, "none", 4);
+	mutex_unlock(&q->elevator_lock);
+	if (unchanged)
+		return 0;
+
 	if (strncmp(ctx->name, "none", 4)) {
 		ret = blk_mq_alloc_sched_res(q, ctx->type, &ctx->res,
 				set->nr_hw_queues);
Signature · 1 line
2.43.0
Reviewed-by on v2 2/2
#3 lore raw

Re: [PATCH v2 2/2] block: return early when the elevator doesn't change

Looks good:

Quoted 1 line · Qiliang Yuan
Fixes b07a889e83355 ("block: move q->sysfs_lock and queue-freeze under show/store method")

But this fixes tag is not correct, that commit just moved code around.

maintainer Applied
#4 lore raw
In reply to Qiliang Yuan
Quoted 9 lines · Qiliang Yuan

Writing the scheduler a queue already uses to queue/scheduler takes about 10 ms and freezes the queue, which stalls I/O on live disks every time a udev rule or tuning tool sets the scheduler unconditionally.

Patch 1 stops requesting a "none-iosched" module on every write of "none". Patch 2 returns before freezing the queue when the elevator wouldn't change.

[...]

Applied, thanks!

[1/2] block: don't request a module for the none elevator

      commit: c85f570121c94679854b570cae2c0dbcc0eae43b
[2/2] block: return early when the elevator doesn't change
      commit: 375a88c6176ffe05e521c2df31b1ceaf4ae5fbaf

Best regards,

Signature · 1 line
Jens Axboe
#5 lore raw
Quoted 2 lines · Jens Axboe

Applied, thanks!

[...]

Quoted 2 lines

[2/2] block: return early when the elevator doesn't change

      commit: 375a88c6176ffe05e521c2df31b1ceaf4ae5fbaf

This one went in from v2, whose Fixes tag Christoph pointed out was wrong. v3 changed it to the commit that added the freeze, with the code unchanged:

Fixes af2814149883e ("block: freeze the queue in queue_attr_store")

Could you update it in for-next?

v3: https://lore.kernel.org/r/20261116-bug-block-elevator-unchanged-v3-2-cb4dda2dadf4@linux.dev

Thanks,
Qiliang