block: check more requests for multiple_queues in blk_attempt_plug_merge
Subsystems (3)
Song Liu
maintainerPATCH 1/1 block: check more requests for multiple_queues in blk_attempt_plug_merge
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when plug->multiple_queues == true.
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
--- block/blk-merge.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
block/blk-merge.c +8 -6
@@ -1089,12 +1089,14 @@ bool blk_attempt_plug_merge(struct request_queue *q, struct bio *bio,if (!plug ||rq_list_empty (plug->mq_list))return false; -/* check the previously added entry for a quick merge attempt */ - rq =rq_list_peek (&plug->mq_list); -if (rq->q == q) { -if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == -BIO_MERGE_OK ) -return true; +rq_list_for_each (&plug->mq_list, rq) { +if (rq->q == q) { +if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == +BIO_MERGE_OK ) +return true; + } +if (!plug->multiple_queues) +break ; }return false; }
Signature · 1 line
2.30.2
Quoted 7 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.
But this also means really significant CPU overhead for all other workloads.
Quoted 3 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
What driver uses multiple queues for HDDs?
Can you explain the workload submitted by a md a bit better? I wonder if we can easily do the right thing straight in the md driver.
Song Liu
maintainerQuoted 7 lines · Christoph Hellwig
Quoted 5 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Quoted 5 lines
Quoted 2 lines
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.But this also means really significant CPU overhead for all other workloads.
Would the following check help with these workloads?
if (!plug->multiple_queues)
break;
Quoted 9 lines
Quoted 4 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
What driver uses multiple queues for HDDs?
Can you explain the workload submitted by a md a bit better? I wonder if we can easily do the right thing straight in the md driver.
It is the md sync_thread doing check and repair. Basically, the md thread reads all
the disks and computes parity from data.
Maybe we should add a new flag to struct blk_plug for this special case?
Song
Song Liu
maintainerQuoted 27 lines · Song Liu
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Quoted 5 lines
Quoted 2 lines
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.But this also means really significant CPU overhead for all other workloads.
Would the following check help with these workloads?
if (!plug->multiple_queues)
break;Quoted 7 lines
Quoted 4 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
What driver uses multiple queues for HDDs?
Quoted 8 lines
Quoted 2 lines
Can you explain the workload submitted by a md a bit better? I wonder if we can easily do the right thing straight in the md driver.
It is the md
sync_threaddoing check and repair. Basically, the md thread reads all
the disks and computes parity from data.Maybe we should add a new flag to struct
blk_plugfor this special case?
I meant something like:
block/blk-core.c +6 -0
@@ -1303,6 +1303,12 @@ void blk_finish_plug(struct blk_plug *plug) }EXPORT_SYMBOL (blk_finish_plug); +void blk_plug_merge_aggressively (struct blk_plug *plug) +{ + plug->aggresive_merge = true; +} +EXPORT_SYMBOL (blk_plug_merge_aggressively); +void blk_io_schedule (void ) {/* Prevent hang_check timer from firing at us during very long I/O */
block/blk-merge.c +0 -0
@@ -1089,12 +1089,14 @@ bool blk_attempt_plug_merge(struct
request_queue *q, struct bio *bio,
if (!plug || rq_list_empty(plug->mq_list))
return false;- /* check the previously added entry for a quick merge attempt */
- rq = rq_list_peek(&plug->mq_list);
- if (rq->q == q) {- if (blk_attempt_bio_merge(q, rq, bio, nr_segs, false) ==
- BIO_MERGE_OK)
- return true;
+ rq_list_for_each(&plug->mq_list, rq) {
+ if (rq->q == q) {
+ if (blk_attempt_bio_merge(q, rq, bio, nr_segs, false) ==
+ BIO_MERGE_OK)
+ return true;
+ }
+ if (!plug->aggresive_merge)
+ break;
}
return false;}
drivers/md/md.c +1 -0
@@ -8901,6 +8901,7 @@ void md_do_sync(struct md_thread *thread) update_time = jiffies;blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug);while (j < max_sectors) {sector_t sectors;
drivers/md/raid1.c +2 -0
@@ -838,6 +838,7 @@ static void flush_pending_writes(struct r1conf *conf)*/ __set_current_state (TASK_RUNNING );blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug);flush_bio_list (conf, bio);blk_finish_plug (&plug); }else
@@ -2591,6 +2592,7 @@ static void raid1d(struct md_thread *thread) }blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug);for (;;) {flush_pending_writes (conf);
drivers/md/raid10.c +2 -0
@@ -876,6 +876,7 @@ static void flush_pending_writes(struct r10conf *conf)__set_current_state (TASK_RUNNING );blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug);/* flush any pending bitmap writes to disk * before proceeding w/ I/O */ md_bitmap_unplug (conf->mddev->bitmap);
@@ -3088,6 +3089,7 @@ static void raid10d(struct md_thread *thread) }blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug);for (;;) {flush_pending_writes (conf);
drivers/md/raid5.c +2 -0
@@ -6447,6 +6447,7 @@ static void raid5_do_work(struct work_struct *work)pr_debug ("+++ raid5worker active\n" );blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug); handled =0 ;spin_lock_irq (&conf->device_lock);while (1 ) {
@@ -6497,6 +6498,7 @@ static void raid5d(struct md_thread *thread)md_check_recovery (mddev);blk_start_plug (&plug); +blk_plug_merge_aggressively (&plug); handled =0 ;spin_lock_irq (&conf->device_lock);while (1 ) {
include/linux/blkdev.h +1 -0
@@ -775,6 +775,7 @@ struct blk_plug {bool multiple_queues;bool has_elevator;bool nowait; +bool aggresive_merge;struct list_head cb_list;/* md requires an unplug callback */ };
@@ -791,6 +792,7 @@ extern struct blk_plug_cb
*blk_check_plugged(blk_plug_cb_fn unplug,
extern void blk_start_plug(struct blk_plug *);
extern void blk_start_plug_nr_ios(struct blk_plug *, unsigned short); extern void blk_finish_plug(struct blk_plug *);
+void blk_plug_merge_aggressively(struct blk_plug *plug);
void blk_flush_plug(struct blk_plug *plug, bool from_schedule);
Song Liu
maintainerQuoted 47 lines · Song Liu
Quoted 27 lines · Song Liu
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Quoted 5 lines
Quoted 2 lines
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.But this also means really significant CPU overhead for all other workloads.
Would the following check help with these workloads?
if (!plug->multiple_queues)
break;Quoted 7 lines
Quoted 4 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
What driver uses multiple queues for HDDs?
Quoted 8 lines
Quoted 2 lines
Can you explain the workload submitted by a md a bit better? I wonder if we can easily do the right thing straight in the md driver.
It is the md
sync_threaddoing check and repair. Basically, the md thread reads all
the disks and computes parity from data.Maybe we should add a new flag to struct
blk_plugfor this special case?I meant something like:
block/blk-core.c +0
-0@@ -1303,6 +1303,12 @@ void blk_finish_plug(struct blk_plug *plug) }EXPORT_SYMBOL (blk_finish_plug);
block/blk-core.c void blk_finish_plug(struct blk_plug *plug) · line 1314
Quoted 10 lines
block/blk-core.c +6
-0+void blk_plug_merge_aggressively (struct blk_plug *plug) +{ + plug->aggresive_merge = true; +} +EXPORT_SYMBOL (blk_plug_merge_aggressively); +void blk_io_schedule (void ) {/* Prevent hang_check timer from firing at us during very long I/O */
Missed one change:
block/blk-core.c +0 -0
@@ -1188,6 +1188,7 @@ void blk_start_plug_nr_ios(struct blk_plug
*plug, unsigned short nr_ios)
plug->multiple_queues = false;
plug->has_elevator = false;
plug->nowait = false;
+ plug->aggresive_merge = false;
INIT_LIST_HEAD(&plug->cb_list);
/*Jens Axboe
maintainerQuoted 10 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
Between your two approaches, I do greatly prefer the first one though.
Signature · 1 line
Jens Axboe
Song Liu
maintainerQuoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.
Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 1 line
Between your two approaches, I do greatly prefer the first one though.
I also like the first one better. But I am not sure whether it will slow down other workloads. We can probably also make the second one cleaner with a new variation of blk_start_plug.
Thanks,
Song
Jens Axboe
maintainerQuoted 20 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
Quoted 5 lines
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging. Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
Signature · 1 line
Jens Axboe
Song Liu
maintainerQuoted 28 lines · Jens Axboe
Quoted 20 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
Quoted 5 lines
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced blk_plug_max_rq_count() to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.
Thanks,
Song
Jens Axboe
maintainerQuoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...
Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.
Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Signature · 1 line
Jens Axboe
Song Liu
maintainerQuoted 39 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...
Quoted 6 lines
Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.
Quoted 8 lines
Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Let me look more into this. Maybe we messed something up in the scheduler.
Thanks,
Song
Jens Axboe
maintainerQuoted 43 lines · Song Liu
Quoted 42 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Quoted 7 lines
Quoted 3 lines
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Right, you'd need to stick some sort of caching in between so instead of reading 4k directly, you ask the cache for 4k and that can manage read-ahead.
Quoted 15 lines
Quoted 15 lines
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
Quoted 6 lines
Quoted 3 lines
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Let me look more into this. Maybe we messed something up in the scheduler.
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But blk_mq_flush_plug_list() should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerQuoted 5 lines · Jens Axboe
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But
blk_mq_flush_plug_list()should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.
Maybe you need a list sort in blk_mq_flush_plug_list(). If you round-robin all the drives, then we'll hit that "run queue" path for each of them when we flush.
Signature · 1 line
Jens Axboe
Quoted 75 lines · Jens Axboe
Quoted 43 lines · Song Liu
Quoted 42 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Quoted 7 lines
Quoted 3 lines
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Right, you'd need to stick some sort of caching in between so instead of reading 4k directly, you ask the cache for 4k and that can manage read-ahead.
Quoted 19 lines
Quoted 18 lines
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Quoted 8 lines
Quoted 2 lines
Let me look more into this. Maybe we messed something up in the scheduler.
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But
blk_mq_flush_plug_list()should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.
requests are shared, when running out of request, plug list will be flushed early.
Maybe we can just put bios into plug list directly, then handle them all in blk_mq_flush_plug_list.
thanks,
Ming
Jens Axboe
maintainerQuoted 79 lines · Ming Lei
Quoted 78 lines · Jens Axboe
Quoted 43 lines · Song Liu
Quoted 42 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Quoted 7 lines
Quoted 3 lines
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Right, you'd need to stick some sort of caching in between so instead of reading 4k directly, you ask the cache for 4k and that can manage read-ahead.
Quoted 21 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.Quoted 8 lines
Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Let me look more into this. Maybe we messed something up in the scheduler.
Quoted 8 lines
Quoted 5 lines
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But
blk_mq_flush_plug_list()should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.requests are shared, when running out of request, plug list will be flushed early.
That is true, but I don't think that's the problem here with the round robin approach. Seems like it'd drive a pretty low queue depth, even considering SATA.
Quoted 2 lines
Maybe we can just put bios into plug list directly, then handle them all in
blk_mq_flush_plug_list.
That might not be a bad idea regardless. And should be trivial now, with the plug list being singly linked anyway.
Signature · 1 line
Jens Axboe
Quoted 86 lines · Jens Axboe
Quoted 85 lines · Ming Lei
Quoted 83 lines · Jens Axboe
Quoted 43 lines · Song Liu
Quoted 42 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Quoted 7 lines
Quoted 3 lines
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Right, you'd need to stick some sort of caching in between so instead of reading 4k directly, you ask the cache for 4k and that can manage read-ahead.
Quoted 15 lines
Quoted 15 lines
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
Quoted 6 lines
Quoted 3 lines
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Let me look more into this. Maybe we messed something up in the scheduler.
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But
blk_mq_flush_plug_list()should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.
Quoted 6 lines
Quoted 2 lines
requests are shared, when running out of request, plug list will be flushed early.
That is true, but I don't think that's the problem here with the round robin approach. Seems like it'd drive a pretty low queue depth, even considering SATA.
Another one may be plug list not sorted before inserting requests to scheduler in blk_mq_flush_plug_list(), looks you have mentioned.
Thanks,
Ming
Jens Axboe
maintainerQuoted 90 lines · Ming Lei
Quoted 89 lines · Jens Axboe
Quoted 87 lines · Ming Lei
Quoted 75 lines · Jens Axboe
Quoted 43 lines · Song Liu
Quoted 42 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Quoted 7 lines
Quoted 3 lines
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Right, you'd need to stick some sort of caching in between so instead of reading 4k directly, you ask the cache for 4k and that can manage read-ahead.
Quoted 19 lines
Quoted 18 lines
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Quoted 8 lines
Quoted 2 lines
Let me look more into this. Maybe we messed something up in the scheduler.
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But
blk_mq_flush_plug_list()should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.requests are shared, when running out of request, plug list will be flushed early.
Quoted 6 lines
Quoted 3 lines
That is true, but I don't think that's the problem here with the round robin approach. Seems like it'd drive a pretty low queue depth, even considering SATA.
Another one may be plug list not sorted before inserting requests to scheduler in
blk_mq_flush_plug_list(), looks you have mentioned.
Yep, it'd probably be the first thing I'd try... The way the IO is issued, it's pretty much guaranteed that the plug list will be fully interleaved with different queues and we're then issuine one-by-one and running the queue each time.
Signature · 1 line
Jens Axboe
Quoted 95 lines · Jens Axboe
Quoted 94 lines · Ming Lei
Quoted 92 lines · Jens Axboe
Quoted 79 lines · Ming Lei
Quoted 78 lines · Jens Axboe
Quoted 43 lines · Song Liu
Quoted 42 lines · Jens Axboe
Quoted 29 lines · Song Liu
Quoted 28 lines · Jens Axboe
Quoted 25 lines · Song Liu
Quoted 10 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests. If we only check the previous entry for merge attempt, many merge will be missed. As a result, significant regression is observed for RAID check and repair.
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.Quoted 8 lines
Quoted 2 lines
This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to 103 MB/s.
Do the underlying disks not have an IO scheduler attached? Curious why the merges aren't being done there, would be trivial when the list is flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a plug worth of IO.
The disks have mq-deadline by default. I also tried kyber, the result is the same. Raid repair work sends IOs to all the HDDs in a round-robin manner. If we only check the previous request, there isn't much opportunity for merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 3 lines
Round robin one at the time? I feel like there's something odd or suboptimal with the raid rebuild, if it's that sensitive to plug merging.
It is not one request at a time, but more like (for raid456):
read 4kB from HDD1, HDD2, HDD3...,
then read another 4kB from HDD1, HDD2, HDD3, ...Ehm, that very much looks like one-at-the-time from each drive, which is pretty much the worst way to do it :-)
Quoted 7 lines
Quoted 3 lines
Is there a reason for that? Why isn't it using 64k chunks or something like that? You could still do that as a kind of read-ahead, even if you're still processing in chunks of 4k.
raid456 handles logic in the granularity of stripe. Each stripe is 4kB from every HDD in the array. AFAICT, we need some non-trivial change to enable the read ahead.
Right, you'd need to stick some sort of caching in between so instead of reading 4k directly, you ask the cache for 4k and that can manage read-ahead.
Quoted 21 lines
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Plug merging is mainly meant to reduce the overhead of merging, complement what the scheduler would do. If there's a big drop in performance just by not getting as efficient merging on the plug side, that points to an issue with something else.
We introduced
blk_plug_max_rq_count()to give md more opportunities to merge at plug side, so I guess the behavior has been like this for a long time. I will take a look at the scheduler side and see whether we can just merge later, but I am not very optimistic about it.Quoted 8 lines
Yeah I remember, and that also kind of felt like a work-around for some underlying issue. Maybe there's something about how the IO is issued that makes it go straight to disk and we never get any merging? Is it because they are sync reads?
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Let me look more into this. Maybe we messed something up in the scheduler.
Quoted 8 lines
Quoted 5 lines
I'm assuming you have a plug setup for doing the reads, which is why you see the big difference (or there would be none). But
blk_mq_flush_plug_list()should really take care of this when the plug is flushed, requests should be merged at that point. And from your description, doesn't sound like they are at all.requests are shared, when running out of request, plug list will be flushed early.
That is true, but I don't think that's the problem here with the round robin approach. Seems like it'd drive a pretty low queue depth, even considering SATA.
Quoted 8 lines
Quoted 2 lines
Another one may be plug list not sorted before inserting requests to scheduler in
blk_mq_flush_plug_list(), looks you have mentioned.Yep, it'd probably be the first thing I'd try... The way the IO is issued, it's pretty much guaranteed that the plug list will be fully interleaved with different queues and we're then issuine one-by-one and running the queue each time.
Naive question, but can you make the flush flush the first one, then scan the queue for all bios for the same device, then go back and start again? Simple approach if it'll work, at the expense of scanning the queue once per device.
Cheers,
Wol
Quoted 35 lines · Song Liu
Quoted 17 lines · Jens Axboe
Quoted 8 lines · Song Liu
RAID arrays check/repair operations benefit a lot from merging requests.
If we only check the previous entry for merge attempt, many merge will be
missed. As a result, significant regression is observed for RAID check
and repair.Quoted 6 lines
Fix this by checking more than just the previous entry when
plug->multiple_queues== true.This improves the check/repair speed of a 20-HDD raid6 from 19 MB/s to
103 MB/s.Quoted 8 lines
Do the underlying disks not have an IO scheduler attached? Curious why
the merges aren't being done there, would be trivial when the list is
flushed out. Because if the perf difference is that big, then other workloads would be suffering they are that sensitive to being within a
plug worth of IO.The disks have mq-deadline by default. I also tried kyber, the result is the
same. Raid repair work sends IOs to all the HDDs in a round-robin manner.
If we only check the previous request, there isn't much opportunity for
merge. I guess other workloads may have different behavior?
Quoted 7 lines
Quoted 2 lines
Between your two approaches, I do greatly prefer the first one though.
I also like the first one better. But I am not sure whether it will slow down
other workloads. We can probably also make the second one cleaner with a new variation ofblk_start_plug.
As a matter of note and purely anecdotal: Before the raid "check" slow down/regression my system would be responsive but delayed (opening a program or opening the xface application menu or switching a file in VLC would take longer than normal, fractions of seconds to a second but slugish and notacable) and with the regression that slow down went from annoying to unbearable.
The slowdowns (in programs and menus and file changes) also *seems* to get worse (in both pre & post regression) the longer the check has been running and the slower a run naturally gets (I assume as the check moves from the outer portion of the disk to the inner portion?) and the lower the KB's reported in cat /proc/mdstat/.
In the post regression situation it wasn't just that the check was taking much longer and was much slower it was also that it slowed down everything else to the point that it was painful to try and use the computer as it was so much less responsive (multiple seconds for anything to load/run/swtch; even web pages). A laggy annoyance had become an actual hindrance.
I have no idea why the speed of the "check" would seemingly affect the apparent responsiveness of the computer and why it would appear that the slower the check the slower the responsiveness.
Quoted 3 lines
Thanks,
Song
Jens Axboe
maintainerQuoted 3 lines · Jens Axboe
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Song, can you try this one? It'll do the dispatch in a somewhat saner fashion, bundling identical queues. And we'll keep iterating the plug list for a merge if we have multiple disks, until we've seen a queue match and checked.
block/blk-merge.c +14 -6
@@ -1073,12 +1073,20 @@ bool blk_attempt_plug_merge(struct request_queue *q, struct bio *bio,if (!plug ||rq_list_empty (plug->mq_list))return false; -/* check the previously added entry for a quick merge attempt */ - rq =rq_list_peek (&plug->mq_list); -if (rq->q == q) { -if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == -BIO_MERGE_OK ) -return true; +rq_list_for_each (&plug->mq_list, rq) { +if (rq->q == q) { +if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == +BIO_MERGE_OK ) +return true; +break ; + } + +/* +* Only keep iterating plug list for merges if we have multiple +* queues +*/ +if (!plug->multiple_queues) +break ; }return false; }
block/blk-mq.c +28 -31
76 lines not on this page · raw message
Signature · 1 line
Jens Axboe
Quoted 3 lines · Wilson Jonathan
I have no idea why the speed of the "check" would seemingly affect the apparent responsiveness of the computer and why it would appear that the slower the check the slower the responsiveness.
The buffers were filling up, nicking your free ram, and you were having to wait longer and longer before they could be flushed to swap your applications back in?
Cheers,
Wol
Song Liu
maintainerHi Jens,
Quoted 10 lines · Jens Axboe
Quoted 3 lines · Jens Axboe
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Song, can you try this one? It'll do the dispatch in a somewhat saner fashion, bundling identical queues. And we'll keep iterating the plug list for a merge if we have multiple disks, until we've seen a queue match and checked.
This one works great! We are seeing 99% read request merge and 500kB+ average read size. The original patch in this thread only got 88% and 34kB for these two metrics.
Thanks,
Song
[...]
Song Liu
maintainerQuoted 18 lines · Paul Menzel
Dear Song,
Am 11.03.22 um 17:59 schrieb Song Liu:
Quoted 12 lines · Jens Axboe
Quoted 10 lines · Jens Axboe
Quoted 3 lines
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Song, can you try this one? It'll do the dispatch in a somewhat saner fashion, bundling identical queues. And we'll keep iterating the plug list for a merge if we have multiple disks, until we've seen a queue match and checked.
Quoted 6 lines
Quoted 3 lines
This one works great! We are seeing 99% read request merge and 500kB+ average read size. The original patch in this thread only got 88% and 34kB for these two metrics.
Nice. I am curious, how these metrics can be obtained?
We can use tools as iostat:
iostat -mx 2
Device r/s w/s rMB/s wMB/s rrqm/s wrqm/s
%rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
sdb 3176.50 1.00 100.57 0.00 22503.00 0.00
87.63 0.00 10.22 3.50 32.46 32.42 4.00 0.24 76.60
sdi 3167.00 1.00 100.57 0.00 22512.50 0.00
87.67 0.00 11.58 4.00 36.68 32.52 4.00 0.24 77.55
The two metrics we used here are %rrqm and rareq-sz.
Thanks,
Song
Dear Song,
Am 11.03.22 um 17:59 schrieb Song Liu:
Quoted 7 lines · Jens Axboe
Quoted 6 lines · Jens Axboe
Quoted 3 lines
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Quoted 8 lines
Quoted 4 lines
Song, can you try this one? It'll do the dispatch in a somewhat saner fashion, bundling identical queues. And we'll keep iterating the plug list for a merge if we have multiple disks, until we've seen a queue match and checked.
This one works great! We are seeing 99% read request merge and 500kB+ average read size. The original patch in this thread only got 88% and 34kB for these two metrics.
Nice. I am curious, how these metrics can be obtained?
[…]
Kind regards,
Paul
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
--Larkin
Quoted 19 lines · Song Liu
Hi Jens,
Quoted 8 lines · Jens Axboe
Quoted 3 lines · Jens Axboe
In any case, just doing larger reads would likely help quite a bit, but would still be nice to get to the bottom of why we're not seeing the level of merging we expect.
Song, can you try this one? It'll do the dispatch in a somewhat saner fashion, bundling identical queues. And we'll keep iterating the plug list for a merge if we have multiple disks, until we've seen a queue match and checked.
This one works great! We are seeing 99% read request merge and 500kB+ average read size. The original patch in this thread only got 88% and 34kB for these two metrics.
Thanks,
Song[...]
Quoted 5 lines · Larkin Lowrey
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
The two patches are merged for 5.18, and I just checked and they apply directly to 5.17 as well.
Greg, care to queue up the two attached patches for 5.17-stable?
Signature · 1 line
Jens Axboe
From 26fed4ac4eab09c27fbae1859696cc38f0536407 Mon Sep 17 00:00:00 2001
From: Jens Axboe <[email protected]>
Date: Fri, 11 Mar 2022 10:24:17 -0700
Subject: [PATCH 2/2] block: flush plug based on hardware and software queue
order
We used to sort the plug list if we had multiple queues before dispatching requests to the IO scheduler. This usually isn't needed, but for certain workloads that interleave requests to disks, it's a less efficient to process the plug list one-by-one if everything is interleaved.
Don't sort the list, but skip through it and flush out entries that have the same target at the same time.
--- block/blk-mq.c | 59 ++++++++++++++++++++++++-------------------------- 1 file changed, 28 insertions(+), 31 deletions(-)
block/blk-mq.c +28 -31
@@ -2573,13 +2573,36 @@ static void __blk_mq_flush_plug_list(struct request_queue *q, q->mq_ops->queue_rqs (&plug->mq_list); } +static void blk_mq_dispatch_plug_list (struct blk_plug *plug,bool from_sched) +{ +struct blk_mq_hw_ctx *this_hctx =NULL ; +struct blk_mq_ctx *this_ctx =NULL ; +struct request *requeue_list =NULL ; +unsigned int depth =0 ; +LIST_HEAD (list); + +do { +struct request *rq =rq_list_pop (&plug->mq_list); + +if (!this_hctx) { + this_hctx = rq->mq_hctx; + this_ctx = rq->mq_ctx; + }else if (this_hctx != rq->mq_hctx || this_ctx != rq->mq_ctx) { +rq_list_add (&requeue_list, rq); +continue ; + } +list_add_tail (&rq->queuelist, &list); + depth++; + }while (!rq_list_empty (plug->mq_list)); + + plug->mq_list = requeue_list; +trace_block_unplug (this_hctx->queue, depth, !from_sched); +blk_mq_sched_insert_requests (this_hctx, this_ctx, &list, from_sched); +} +void blk_mq_flush_plug_list (struct blk_plug *plug,bool from_schedule) { -struct blk_mq_hw_ctx *this_hctx; -struct blk_mq_ctx *this_ctx;struct request *rq; -unsigned int depth; -LIST_HEAD (list);if (rq_list_empty (plug->mq_list))return ;
@@ -2615,35 +2638,9 @@ void blk_mq_flush_plug_list(struct blk_plug *plug, bool from_schedule)return ; } - this_hctx =NULL ; - this_ctx =NULL ; - depth =0 ;do { - rq =rq_list_pop (&plug->mq_list); - -if (!this_hctx) { - this_hctx = rq->mq_hctx; - this_ctx = rq->mq_ctx; - }else if (this_hctx != rq->mq_hctx || this_ctx != rq->mq_ctx) { -trace_block_unplug (this_hctx->queue, depth, - !from_schedule); -blk_mq_sched_insert_requests (this_hctx, this_ctx, - &list, from_schedule); - depth =0 ; - this_hctx = rq->mq_hctx; - this_ctx = rq->mq_ctx; - - } - -list_add (&rq->queuelist, &list); - depth++; +blk_mq_dispatch_plug_list (plug, from_schedule); }while (!rq_list_empty (plug->mq_list)); - -if (!list_empty (&list)) { -trace_block_unplug (this_hctx->queue, depth, !from_schedule); -blk_mq_sched_insert_requests (this_hctx, this_ctx, &list, - from_schedule); - } } void blk_mq_try_issue_list_directly(struct blk_mq_hw_ctx *hctx,
Signature · 1 line
2.35.1
From 5b2050718d095cd3242d1f42aaaea3a2fec8e6f0 Mon Sep 17 00:00:00 2001
From: Jens Axboe <[email protected]>
Date: Fri, 11 Mar 2022 10:21:43 -0700
Subject: [PATCH 1/2] block: ensure plug merging checks the correct queue at
least once
Song reports that a RAID rebuild workload runs much slower recently, and it is seeing a lot less merging than it did previously. The reason is that a previous commit reduced the amount of work we do for plug merging. RAID rebuild interleaves requests between disks, so a last-entry check in plug merging always misses a merge opportunity since we always find a different disk than what we are looking for.
Modify the logic such that it's still a one-hit cache, but ensure that we check enough to find the right target before giving up.
--- block/blk-merge.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
block/blk-merge.c +14 -6
@@ -1087,12 +1087,20 @@ bool blk_attempt_plug_merge(struct request_queue *q, struct bio *bio,if (!plug ||rq_list_empty (plug->mq_list))return false; -/* check the previously added entry for a quick merge attempt */ - rq =rq_list_peek (&plug->mq_list); -if (rq->q == q) { -if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == -BIO_MERGE_OK ) -return true; +rq_list_for_each (&plug->mq_list, rq) { +if (rq->q == q) { +if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == +BIO_MERGE_OK ) +return true; +break ; + } + +/* +* Only keep iterating plug list for merges if we have multiple +* queues +*/ +if (!plug->multiple_queues) +break ; }return false; }
Signature · 1 line
2.35.1
Greg Kroah-Hartman
maintainerQuoted 11 lines · Jens Axboe
Quoted 5 lines · Larkin Lowrey
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
The two patches are merged for 5.18, and I just checked and they apply directly to 5.17 as well.
Greg, care to queue up the two attached patches for 5.17-stable?
Now queued up, but shouldn't they also work for 5.16? They don't apply there, but the Fixes: tag says it should.
thanks,
greg k-h
Jens Axboe
maintainerQuoted 8 lines · Greg Kroah-Hartman
Quoted 7 lines · Jens Axboe
Quoted 5 lines · Larkin Lowrey
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
Quoted 7 lines
Quoted 4 lines
The two patches are merged for 5.18, and I just checked and they apply directly to 5.17 as well.
Greg, care to queue up the two attached patches for 5.17-stable?
Now queued up, but shouldn't they also work for 5.16? They don't apply there, but the Fixes: tag says it should.
Yes, they should go to 5.16-stable as well. Song, do you have time to port and test on 5.16?
Signature · 1 line
Jens Axboe
Quoted 24 lines · Roger Heflin
If someone sends a patch that will apply for 5.16 I can patch and compile, and run normal IO and a few raid checks tests.
Quoted 20 lines · Jens Axboe
Quoted 8 lines · Greg Kroah-Hartman
Quoted 7 lines · Jens Axboe
Quoted 5 lines · Larkin Lowrey
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
Quoted 7 lines
Quoted 4 lines
The two patches are merged for 5.18, and I just checked and they apply directly to 5.17 as well.
Greg, care to queue up the two attached patches for 5.17-stable?
Now queued up, but shouldn't they also work for 5.16? They don't apply there, but the Fixes: tag says it should.
Yes, they should go to 5.16-stable as well. Song, do you have time to port and test on 5.16?
I ported the two patches on top of 5.16.18 (attached). They look good in my tests.
Thanks,
Song
From 6ab65bec8897d0b9cdbe573bff9f7977e1a0fbdf Mon Sep 17 00:00:00 2001
From: Jens Axboe <[email protected]>
Date: Fri, 11 Mar 2022 10:21:43 -0700
Subject: [PATCH 1/2] block: ensure plug merging checks the correct queue at
least once
Song reports that a RAID rebuild workload runs much slower recently, and it is seeing a lot less merging than it did previously. The reason is that a previous commit reduced the amount of work we do for plug merging. RAID rebuild interleaves requests between disks, so a last-entry check in plug merging always misses a merge opportunity since we always find a different disk than what we are looking for.
Modify the logic such that it's still a one-hit cache, but ensure that we check enough to find the right target before giving up.
--- block/blk-merge.c | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
block/blk-merge.c +13 -10
@@ -1093,18 +1093,21 @@ bool blk_attempt_plug_merge(struct request_queue *q, struct bio *bio,if (!plug ||rq_list_empty (plug->mq_list))return false; -/* check the previously added entry for a quick merge attempt */ - rq =rq_list_peek (&plug->mq_list); -if (rq->q == q) { +rq_list_for_each (&plug->mq_list, rq) { +if (rq->q == q) { + *same_queue_rq = true; +if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == +BIO_MERGE_OK ) +return true; +break ; + } +/* -* Only blk-mq multiple hardware queues case checks the rq in -* the same queue, there should be only one such rq in a queue +* Only keep iterating plug list for merges if we have multiple +* queues */ - *same_queue_rq = true; - -if (blk_attempt_bio_merge (q, rq, bio, nr_segs, false) == -BIO_MERGE_OK ) -return true; +if (!plug->multiple_queues) +break ; }return false; }
Signature · 1 line
2.30.2
From d84de1971d5a512b7bc36d8df169660e8ce7dd63 Mon Sep 17 00:00:00 2001
From: Jens Axboe <[email protected]>
Date: Fri, 11 Mar 2022 10:24:17 -0700
Subject: [PATCH 2/2] block: flush plug based on hardware and software queue
order
We used to sort the plug list if we had multiple queues before dispatching requests to the IO scheduler. This usually isn't needed, but for certain workloads that interleave requests to disks, it's a less efficient to process the plug list one-by-one if everything is interleaved.
Don't sort the list, but skip through it and flush out entries that have the same target at the same time.
--- block/blk-mq.c | 60 +++++++++++++++++++++++--------------------------- 1 file changed, 27 insertions(+), 33 deletions(-)
block/blk-mq.c +27 -33
77 lines not on this page · raw message
Signature · 1 line
2.30.2
Jens Axboe
maintainerQuoted 20 lines · Song Liu
Quoted 19 lines · Roger Heflin
If someone sends a patch that will apply for 5.16 I can patch and compile, and run normal IO and a few raid checks tests.
Quoted 15 lines · Jens Axboe
Quoted 13 lines · Greg Kroah-Hartman
Quoted 11 lines · Jens Axboe
Quoted 5 lines · Larkin Lowrey
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
The two patches are merged for 5.18, and I just checked and they apply directly to 5.17 as well.
Greg, care to queue up the two attached patches for 5.17-stable?
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Now queued up, but shouldn't they also work for 5.16? They don't apply there, but the Fixes: tag says it should.
Yes, they should go to 5.16-stable as well. Song, do you have time to port and test on 5.16?
I ported the two patches on top of 5.16.18 (attached). They look good in my tests.
Thanks!
Signature · 1 line
Jens Axboe
Greg Kroah-Hartman
maintainerQuoted 24 lines · Jens Axboe
Quoted 23 lines · Song Liu
Quoted 22 lines · Roger Heflin
If someone sends a patch that will apply for 5.16 I can patch and compile, and run normal IO and a few raid checks tests.
Quoted 18 lines · Jens Axboe
Quoted 15 lines · Greg Kroah-Hartman
Quoted 11 lines · Jens Axboe
Quoted 5 lines · Larkin Lowrey
Thank you for investigating and resolving this issue. Your effort is very much appreciated.
I am interested in when this patch will end up in a release. Is it going to make it into a 5.17.x release or will it not come until 5.18?
The two patches are merged for 5.18, and I just checked and they apply directly to 5.17 as well.
Greg, care to queue up the two attached patches for 5.17-stable?
Now queued up, but shouldn't they also work for 5.16? They don't apply there, but the Fixes: tag says it should.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
Yes, they should go to 5.16-stable as well. Song, do you have time to port and test on 5.16?
I ported the two patches on top of 5.16.18 (attached). They look good in my tests.
Thanks!
Thanks, both queued up now.
greg k-h