blk-mq: modify hybrid sleep time to aggressive
PATCH 1/1 blk-mq: modify hybrid sleep time to aggressive
Current sleep time for hybrid polling is half of mean time. The 'half' sleep time is good for minimizing the cpu utilization. But, the problem is that its cpu utilization is still high. this patch can help to minimize the cpu utilization side.
Below 1,2 is my test hardware sets.
1. Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz + Samsung 970 pro 1Tb
2. Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz + INTEL SSDPED1D480GA 480G
| Classic Polling | Hybrid Polling | this Patch
-----------------------------------------------------------------
cpu util | IOPS(k) | cpu util | IOPS | cpu util | IOPS |
-----------------------------------------------------------------
1. 99.96 | 491 | 56.98 | 467 | 35.98 | 442 |
-----------------------------------------------------------------
2. 99.94 | 582 | 56.3 | 582 | 35.28 | 582 |
cpu util means that sum of sys and user util.
I used 4k rand read for this test.
because that case is worst case of I/O performance side. below one is my fio setup.
name=pollTest
ioengine=pvsync2
hipri
direct=1
size=100%
randrepeat=0time_based
ramp_time=0
norandommaprefill_buffers
log_avg_msec=1000
log_max_value=1group_reporting
filename=/dev/nvme0n1
[rd_rnd_qd_1_4k_1w]
bs=4k
iodepth=32
numjobs=[num of cpus]
rw=randread
runtime=60
write_bw_log=bw_rd_rnd_qd_1_4k_1w
write_iops_log=iops_rd_rnd_qd_1_4k_1w
write_lat_log=lat_rd_rnd_qd_1_4k_1w
Thanks
--- block/blk-mq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
block/blk-mq.c +1 -2
@@ -3749,8 +3749,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q,return ret;if (q->poll_stat[bucket].nr_samples) - ret = (q->poll_stat[bucket].mean +1 ) /2 ; - + ret = (q->poll_stat[bucket].mean +1 ) *3 /4 ;return ret; }
Signature · 1 line
2.17.1
Adding Damien who wrote this code.
Quoted 72 lines · Dongjoo Seo
Current sleep time for hybrid polling is half of mean time. The 'half' sleep time is good for minimizing the cpu utilization. But, the problem is that its cpu utilization is still high. this patch can help to minimize the cpu utilization side.
Below 1,2 is my test hardware sets.
1. Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz + Samsung 970 pro 1Tb
2. Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz + INTEL SSDPED1D480GA 480G| Classic Polling | Hybrid Polling | this Patch
-----------------------------------------------------------------
cpu util | IOPS(k) | cpu util | IOPS | cpu util | IOPS |
-----------------------------------------------------------------
1. 99.96 | 491 | 56.98 | 467 | 35.98 | 442 |
-----------------------------------------------------------------
2. 99.94 | 582 | 56.3 | 582 | 35.28 | 582 |cpu util means that sum of sys and user util.
I used 4k rand read for this test.
because that case is worst case of I/O performance side. below one is my fio setup.name=pollTest
ioengine=pvsync2
hipri
direct=1
size=100%
randrepeat=0time_based
ramp_time=0
norandommaprefill_buffers
log_avg_msec=1000
log_max_value=1group_reporting
filename=/dev/nvme0n1
[rd_rnd_qd_1_4k_1w]
bs=4k
iodepth=32
numjobs=[num of cpus]
rw=randread
runtime=60
write_bw_log=bw_rd_rnd_qd_1_4k_1w
write_iops_log=iops_rd_rnd_qd_1_4k_1w
write_lat_log=lat_rd_rnd_qd_1_4k_1wThanks
Signed-off-by Dongjoo Seo <[email protected]>--- block/blk-mq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)block/blk-mq.c +1
-2@@ -3749,8 +3749,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q,return ret;if (q->poll_stat[bucket].nr_samples) - ret = (q->poll_stat[bucket].mean +1 ) /2 ; - + ret = (q->poll_stat[bucket].mean +1 ) *3 /4 ;return ret; }Signature · 1 line
2.17.1
---end quoted text---
Quoted 1 line · Christoph Hellwig
Adding Damien who wrote this code.
Nope. It wasn't me. I think it was Stephen Bates:
commit 720b8ccc4500 ("blk-mq: Add a polling specific stats function")
So +Stephen.
Quoted 76 lines · Dongjoo Seo
Quoted 72 lines
Current sleep time for hybrid polling is half of mean time. The 'half' sleep time is good for minimizing the cpu utilization. But, the problem is that its cpu utilization is still high. this patch can help to minimize the cpu utilization side.
Below 1,2 is my test hardware sets.
1. Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz + Samsung 970 pro 1Tb
2. Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz + INTEL SSDPED1D480GA 480G| Classic Polling | Hybrid Polling | this Patch
-----------------------------------------------------------------
cpu util | IOPS(k) | cpu util | IOPS | cpu util | IOPS |
-----------------------------------------------------------------
1. 99.96 | 491 | 56.98 | 467 | 35.98 | 442 |
-----------------------------------------------------------------
2. 99.94 | 582 | 56.3 | 582 | 35.28 | 582 |cpu util means that sum of sys and user util.
I used 4k rand read for this test.
because that case is worst case of I/O performance side. below one is my fio setup.name=pollTest
ioengine=pvsync2
hipri
direct=1
size=100%
randrepeat=0time_based
ramp_time=0
norandommaprefill_buffers
log_avg_msec=1000
log_max_value=1group_reporting
filename=/dev/nvme0n1
[rd_rnd_qd_1_4k_1w]
bs=4k
iodepth=32
numjobs=[num of cpus]
rw=randread
runtime=60
write_bw_log=bw_rd_rnd_qd_1_4k_1w
write_iops_log=iops_rd_rnd_qd_1_4k_1w
write_lat_log=lat_rd_rnd_qd_1_4k_1wThanks
Signed-off-by Dongjoo Seo <[email protected]>--- block/blk-mq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)block/blk-mq.c +1
-2@@ -3749,8 +3749,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q,return ret;if (q->poll_stat[bucket].nr_samples) - ret = (q->poll_stat[bucket].mean +1 ) /2 ; - + ret = (q->poll_stat[bucket].mean +1 ) *3 /4 ;return ret; }Signature · 1 line
2.17.1---end quoted text---
Signature · 2 lines
Damien Le Moal Western Digital Research
Quoted 14 lines · Damien Le Moal
Quoted 1 line · Christoph Hellwig
Adding Damien who wrote this code.
Nope. It wasn't me. I think it was Stephen Bates:
commit
720b8ccc4500("blk-mq: Add a polling specific stats function")So +Stephen.
Quoted 6 lines · Dongjoo Seo
Quoted 4 lines
Current sleep time for hybrid polling is half of mean time. The 'half' sleep time is good for minimizing the cpu utilization. But, the problem is that its cpu utilization is still high. this patch can help to minimize the cpu utilization side.
This won't work well. When I was experimenting I saw that half mean is actually is too much for fast enough requests, like <20us 4K writes, it's oversleeping them. Even more I'm afraid of getting in a vicious cycle, when oversleeping increases statistical mean, that increases sleep time, that again increases stat mean, and so on. That what happened for me when the scheme was too aggressive.
I actually sent once patches [1] for automatic dynamic sleep time adjustment, but nobody cared.
[1] https://lkml.org/lkml/2019/4/30/117
Quoted 72 lines
Quoted 70 lines
Quoted 68 lines
Below 1,2 is my test hardware sets.
1. Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz + Samsung 970 pro 1Tb
2. Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz + INTEL SSDPED1D480GA 480G| Classic Polling | Hybrid Polling | this Patch
-----------------------------------------------------------------
cpu util | IOPS(k) | cpu util | IOPS | cpu util | IOPS |
-----------------------------------------------------------------
1. 99.96 | 491 | 56.98 | 467 | 35.98 | 442 |
-----------------------------------------------------------------
2. 99.94 | 582 | 56.3 | 582 | 35.28 | 582 |cpu util means that sum of sys and user util.
I used 4k rand read for this test.
because that case is worst case of I/O performance side. below one is my fio setup.name=pollTest
ioengine=pvsync2
hipri
direct=1
size=100%
randrepeat=0time_based
ramp_time=0
norandommaprefill_buffers
log_avg_msec=1000
log_max_value=1group_reporting
filename=/dev/nvme0n1
[rd_rnd_qd_1_4k_1w]
bs=4k
iodepth=32
numjobs=[num of cpus]
rw=randread
runtime=60
write_bw_log=bw_rd_rnd_qd_1_4k_1w
write_iops_log=iops_rd_rnd_qd_1_4k_1w
write_lat_log=lat_rd_rnd_qd_1_4k_1wThanks
Signed-off-by Dongjoo Seo <[email protected]>--- block/blk-mq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)block/blk-mq.c +1
-2@@ -3749,8 +3749,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q,return ret;if (q->poll_stat[bucket].nr_samples) - ret = (q->poll_stat[bucket].mean +1 ) /2 ; - + ret = (q->poll_stat[bucket].mean +1 ) *3 /4 ;return ret; }Signature · 1 line
2.17.1---end quoted text---
Signature · 1 line
Pavel Begunkov
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · dongjoo seo
I agree with your opinion. And your patch is also good approach. How about combine it? Adaptive solution with 3/4.
I couldn't disclose numbers back then, but thanks to a steep skewed latency distribution of NAND/SSDs, it actually was automatically adjusting it to ~3/4 for QD1 and long enough requests (~75+ us). Also, if "max(sleep_ns, half_mean)" is removed, it was keeping the time below 1/2 for fast requests (less than ~30us), and that is a good thing because it was constantly oversleeping them. Though new ultra low-latency SSDs came since then.
The real problem is to find anyone who actually uses it, otherwise it's just a chunk of dead code. Do you? Anyone? I remember once it was completely broken for months, but that was barely noticed.
Quoted 112 lines
Because, if we get the intensive workloads then we need to decrease the whole cpu utilization even with [1].
[1] https://lkml.org/lkml/2019/4/30/117 <https://lkml.org/lkml/2019/4/30/117>
Quoted 105 lines · Pavel Begunkov
Quoted 14 lines · Damien Le Moal
Quoted 1 line · Christoph Hellwig
Adding Damien who wrote this code.
Nope. It wasn't me. I think it was Stephen Bates:
commit
720b8ccc4500("blk-mq: Add a polling specific stats function")So +Stephen.
Quoted 6 lines · Dongjoo Seo
Quoted 4 lines
Current sleep time for hybrid polling is half of mean time. The 'half' sleep time is good for minimizing the cpu utilization. But, the problem is that its cpu utilization is still high. this patch can help to minimize the cpu utilization side.
This won't work well. When I was experimenting I saw that half mean is actually is too much for fast enough requests, like <20us 4K writes, it's oversleeping them. Even more I'm afraid of getting in a vicious cycle, when oversleeping increases statistical mean, that increases sleep time, that again increases stat mean, and so on. That what happened for me when the scheme was too aggressive.
I actually sent once patches [1] for automatic dynamic sleep time adjustment, but nobody cared.
[1] https://lkml.org/lkml/2019/4/30/117 <https://lkml.org/lkml/2019/4/30/117>
Quoted 72 lines
Quoted 70 lines
Quoted 68 lines
Below 1,2 is my test hardware sets.
1. Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz + Samsung 970 pro 1Tb
2. Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz + INTEL SSDPED1D480GA 480G| Classic Polling | Hybrid Polling | this Patch
-----------------------------------------------------------------
cpu util | IOPS(k) | cpu util | IOPS | cpu util | IOPS |
-----------------------------------------------------------------
1. 99.96 | 491 | 56.98 | 467 | 35.98 | 442 |
-----------------------------------------------------------------
2. 99.94 | 582 | 56.3 | 582 | 35.28 | 582 |cpu util means that sum of sys and user util.
I used 4k rand read for this test.
because that case is worst case of I/O performance side. below one is my fio setup.name=pollTest
ioengine=pvsync2
hipri
direct=1
size=100%
randrepeat=0time_based
ramp_time=0
norandommaprefill_buffers
log_avg_msec=1000
log_max_value=1group_reporting
filename=/dev/nvme0n1
[rd_rnd_qd_1_4k_1w]
bs=4k
iodepth=32
numjobs=[num of cpus]
rw=randread
runtime=60
write_bw_log=bw_rd_rnd_qd_1_4k_1w
write_iops_log=iops_rd_rnd_qd_1_4k_1w
write_lat_log=lat_rd_rnd_qd_1_4k_1wThanks
Signed-off-by Dongjoo Seo <[email protected]>---
block/blk-mq.c| 3 +--1 file changed, 1 insertion(+), 2 deletions(-)block/blk-mq.c +1
-2@@ -3749,8 +3749,7 @@ static unsigned long blk_mq_poll_nsecs(struct request_queue *q,return ret;if (q->poll_stat[bucket].nr_samples) - ret = (q->poll_stat[bucket].mean +1 ) /2 ; - + ret = (q->poll_stat[bucket].mean +1 ) *3 /4 ;return ret; }Signature · 1 line
2.17.1---end quoted text---
Signature · 1 line
Pavel Begunkov
Signature · 1 line
Pavel Begunkov
A message that is not in the archive (sent off-list or not yet received): CABM9hu3FE6ZZL=oWznbJUw2i9i8qJ1AYKotg_uEeAe1Vu+8Ong@mail.gmail.com
Quoted 3 lines · dongjoo seo
Actually I worked with a many-core machine with an NVMe devices. It have 120~192 cpu core and lots of memory and disk device. For that, I checked the scalability of block layer and NVMe device driver.
I'm rather mean not test setups but production servers or large sets of user actually using it.
AFAIK "normal users" like laptops and so on don't even use iopoll, that's too wasteful. Maybe someone here can correct me.
Quoted 1 line
During that process, I found the polling methodologies like
io_uring, SPDK.
Just to notice, io_uring is not iopoll centric, and internally it goes through similar code path as HIPRI reads/writes, i.e. using mentioned NVMe driver polling.
Quoted 7 lines
But, for applying that engines, we need to modify the application's code. Thats why I and we interest in the polling in NVMe device driver.
And, you are right, normal user can not recognize that difference btw previous one and polling based approach. However, I believe that ultra low-latency dissemination will make this active.
Ultra low-latency devices make hybrid polling not as useful as before because a) it's harder to get right statistics with all discrepancies in the host system, b) tighter gaps makes relative overhead on that sleep higher + easier to oversleep.
Quoted 144 lines
Anyway, Thank you for your opinion.
2020년 11월 18일 (수) 오후 11:20, Pavel Begunkov <[email protected]>님이 작성:
Quoted 138 lines · dongjoo seo
Quoted 2 lines
I agree with your opinion. And your patch is also good approach. How about combine it? Adaptive solution with 3/4.
I couldn't disclose numbers back then, but thanks to a steep skewed latency distribution of NAND/SSDs, it actually was automatically adjusting it to ~3/4 for QD1 and long enough requests (~75+ us). Also, if "
max(sleep_ns, half_mean)" is removed, it was keeping the time below 1/2 for fast requests (less than ~30us), and that is a good thing because it was constantly oversleeping them. Though new ultra low-latency SSDs came since then.The real problem is to find anyone who actually uses it, otherwise it's just a chunk of dead code. Do you? Anyone? I remember once it was completely broken for months, but that was barely noticed.
Quoted 4 lines
Because, if we get the intensive workloads then we need to decrease the whole cpu utilization even with [1].
https://lkml.org/lkml/2019/4/30/117>
Quoted 2 lines
Quoted 2 lines
On Nov 18, 2020, at 6:26 PM, Pavel Begunkov <[email protected]>
Quoted 24 lines · wrote:
Quoted 24 lines · Damien Le Moal
Quoted 14 lines · Christoph Hellwig
Quoted 1 line
Adding Damien who wrote this code.
Nope. It wasn't me. I think it was Stephen Bates:
commit
720b8ccc4500("blk-mq: Add a polling specific stats function")So +Stephen.
Quoted 6 lines · Dongjoo Seo
Quoted 4 lines
Current sleep time for hybrid polling is half of mean time. The 'half' sleep time is good for minimizing the cpu utilization. But, the problem is that its cpu utilization is still high. this patch can help to minimize the cpu utilization side.
This won't work well. When I was experimenting I saw that half mean is actually is too much for fast enough requests, like <20us 4K writes, it's oversleeping them. Even more I'm afraid of getting in a vicious cycle, when oversleeping increases statistical mean, that increases sleep time, that again increases stat mean, and so on. That what happened for me when the scheme was too aggressive.
Quoted 4 lines
Quoted 4 lines
I actually sent once patches [1] for automatic dynamic sleep time adjustment, but nobody cared.
https://lkml.org/lkml/2019/4/30/117>
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Below 1,2 is my test hardware sets.
1. Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz + Samsung 970 pro 1Tb
2. Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz + INTEL SSDPED1D480GA480G
Quoted 46 lines
Quoted 46 lines
Quoted 46 lines
Quoted 46 lines
Quoted 46 lines
| Classic Polling | Hybrid Polling | this Patch
-----------------------------------------------------------------
cpu util | IOPS(k) | cpu util | IOPS | cpu util | IOPS |
-----------------------------------------------------------------
1. 99.96 | 491 | 56.98 | 467 | 35.98 | 442 |
-----------------------------------------------------------------
2. 99.94 | 582 | 56.3 | 582 | 35.28 | 582 |cpu util means that sum of sys and user util.
I used 4k rand read for this test.
because that case is worst case of I/O performance side. below one is my fio setup.name=pollTest
ioengine=pvsync2
hipri
direct=1
size=100%
randrepeat=0time_based
ramp_time=0
norandommaprefill_buffers
log_avg_msec=1000
log_max_value=1group_reporting
filename=/dev/nvme0n1
[rd_rnd_qd_1_4k_1w]
bs=4k
iodepth=32
numjobs=[num of cpus]
rw=randread
runtime=60
write_bw_log=bw_rd_rnd_qd_1_4k_1w
write_iops_log=iops_rd_rnd_qd_1_4k_1w
write_lat_log=lat_rd_rnd_qd_1_4k_1wThanks
Signed-off-by Dongjoo Seo <[email protected]>---
block/blk-mq.c| 3 +--1 file changed, 1 insertion(+), 2 deletions(-)Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
block/blk-mq.c static unsigned long blk_mq_poll_nsecs(struct
Quoted 5 lines
block/blk-mq.c +0
-0@@ -3749,8 +3749,7 @@ static unsigned long blk_mq_poll_nsecs(struct
request_queue*q,Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
block/blk-mq.c +1
-3return ret;if (q->poll_stat[bucket].nr_samples) - ret = (q->poll_stat[bucket].mean +1 ) /2 ; - + ret = (q->poll_stat[bucket].mean +1 ) *3 /4 ;return ret; } --2.17.1
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
---end quoted text---
--
Pavel Begunkov--
Pavel Begunkov
Signature · 1 line
Pavel Begunkov