cfq-iosched: quantum check tweak
RFC 1/1 cfq-iosched: quantum check tweak
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limit
2. we could keep the check only when cfq_latency is on. For uses who don't care about latency should be happy to have device fully piped on.
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/s
--- block/cfq-iosched.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
block/cfq-iosched.c +14 -1
@@ -2242,6 +2242,18 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq)) +return 1 ; +if (time_after (jiffies + cfqd->cfq_slice_idle, cfqq->slice_end)) +return 1 ; + +return 0 ; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;
@@ -2275,7 +2287,8 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq) + && cfqd->cfq_latency)return false;/*
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limit
ok. You can even scale the limit proportionally to the remaining slice (see below).
Quoted 2 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.
I wouldn't overload low_latency with this meaning. You can obtain the same by setting the quantum to 32.
Quoted 5 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/s
Please, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
Quoted 11 lines
Signed-off-by Shaohua Li <[email protected]>---
block/cfq-iosched.c | 15 ++++++++++++++-1 file changed, 14 insertions(+), 1 deletion(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================
block/cfq-iosched.c static int cfq_forced_dispatch(struct cf · line 2251
Quoted 10 lines
block/cfq-iosched.c +7
-0@@ -2242,6 +2242,18 @@ static int cfq_forced_dispatch(struct cf return dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, + struct cfq_queue *cfqq) +{ + /* the queue hasn't finished any request, can't estimate */ + if (cfq_cfqq_slice_new(cfqq)) + return 1; + if (time_after(jiffies + cfqd->cfq_slice_idle, cfqq->slice_end))
here I would use
jiffies + cfqd->cfq_slice_idle * (cfqq->queued[0] + cfqq->queued[1]) to obtain the proper scaling
block/cfq-iosched.c static int cfq_forced_dispatch(struct cf · line 2259
Quoted 8 lines
block/cfq-iosched.c +5
-0+ return 1; + + return 0; +} + static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) { unsigned int max_dispatch;
Thanks,
Corrado
Signature · 5 lines
__________________________________________________________________________ dott. Corrado Zoccolo mailto:[email protected] PhD - Department of Computer Science - University of Pisa, Italy --------------------------------------------------------------------------
Quoted 8 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale. cfq_slice_used_soon() means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope a cfq_slice_idle time is enough to finish the dispatched requests.
Quoted 4 lines
Quoted 2 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.
As this impact fairness, so natually thought we could use low_latency. I'll remove the check in next post.
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
I'll do more checks. The time is hard to choose (I choose cfq_slice-idle here) to balance thoughput and latency. Do we have creteria to measure this? See the patch passes some tests, so it's ok for latency.
Thanks,
Shaohua
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limit
Quoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idle is 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1 cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle * ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.
Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.
Quoted 3 lines
I'll do more checks. The time is hard to choose (I choose cfq_slice-idle here) to balance thoughput and latency. Do we have creteria to measure this? See the patch passes some tests, so it's ok for latency.
Max latency should be near 300ms (compare with and without the patch), for a reasonable number of concurrent processes (300ms/#proc < 2*idle_slice).
Thanks,
Corrado
Quoted 4 lines
Thanks,
Shaohua
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle*ilog2(nr_dispatched+1)
cfq_slice_idle* (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the slice_idle * dispatched approach, which I thought should be safe.
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
For async io, 40ms/8ms = 5 - quantum = 1, we only send extra 1 request in maxium. For sync io, 100ms/8ms = 12 - quantum = 8, we might send extra 8 requests in maxium. This might cause latency issue if the queue is preempted at the very beginning.
This patch boost my workload from 78m/s to 102m/s, which isn't that big as my last post, but also is a big improvement.
--- block/cfq-iosched.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
block/cfq-iosched.c +14 -1
@@ -2242,6 +2242,19 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq)) +return true; +if (time_after (jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) +return true; + +return false; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;
@@ -2275,7 +2288,7 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;/*
[global]
runtime=120
ioscheduler=cfq
overwrite=1
loops=10group_reporting
[seq-read]
directory=/mnt/b1
bs=4k
rw=read
ioengine=sync
size=1G
[seq-write]
directory=/mnt/b1
bs=4k
rw=write
ioengine=sync
size=1G
[rand-read]
directory=/mnt/b1
bs=4k
rw=randread
ioengine=sync
size=1G
numjobs=4
[rand-read-aio]
directory=/mnt/b1
bs=4k
iodepth=32
rw=randread
ioengine=libaio
direct=1
size=1G
numjobs=4
Corrado Zoccolo
Acked-byHi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.
Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :slice_idle * dispatched / 2. It should give a performance boost, without noticeable impact on latency.
Quoted 11 lines
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
For async io, 40ms/8ms = 5 - quantum = 1, we only send extra 1 request in maxium. For sync io, 100ms/8ms = 12 - quantum = 8, we might send extra 8 requests in maxium. This might cause latency issue if the queue is preempted at the very beginning.This patch boost my workload from 78m/s to 102m/s, which isn't that big as my last post, but also is a big improvement.
Quoted 41 lines
Signed-off-by Shaohua Li <[email protected]>---
block/cfq-iosched.c | 15 ++++++++++++++-1 file changed, 14 insertions(+), 1 deletion(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +14
-1@@ -2242,6 +2242,19 @@ static int cfq_forced_dispatch(struct cf return dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, + struct cfq_queue *cfqq) +{ + /* the queue hasn't finished any request, can't estimate */ + if (cfq_cfqq_slice_new(cfqq)) + return true; + if (time_after(jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) + return true; + + return false; +} + static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) { unsigned int max_dispatch;@@ -2275,7 +2288,7 @@ static bool cfq_may_dispatch(struct cfq_ /* * We have other queues, don't allow more IO from this one */ - if (cfqd->busy_queues > 1) + if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; /*
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.
Quoted 5 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency
Thanks for looking at it. can you forward your tests to me so I can check here? I'll do more aggressive formula next. It shouldn't impact my SSD which is very fast, each request takes less than 1ms. We'd better have a mechanism to measure device speed, but jiffies isn't good. I'm thinking using sched_clock() which has its issue too like having drift between CPUs.
Thanks,
Shaohua
Quoted 63 lines · Shaohua Li
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 58 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.
Quoted 7 lines
Quoted 5 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latencyThanks for looking at it. can you forward your tests to me so I can check here?
Sure, you can find them at:
* simple seek time: http://dl.dropbox.com/u/3525644/stride.c
* avg seek time with NCQ: http://dl.dropbox.com/u/3525644/ncq.c
Quoted 4 lines
I'll do more aggressive formula next. It shouldn't impact
my SSD which is very fast, each request takes less than 1ms. We'd better have a mechanism to measure device speed, but jiffies isn't good. I'm thinking usingsched_clock()which has its issue too like having drift between CPUs.
I'm using ktime_get() successfully. Measuring is the easy part. The difficult one is decide what to do with the value :)
Thanks
Corrado
Quoted 4 lines
Thanks,
Shaohua
Signature · 10 lines
__________________________________________________________________________ dott. Corrado Zoccolo mailto:[email protected] PhD - Department of Computer Science - University of Pisa, Italy -------------------------------------------------------------------------- The self-confidence of a warrior is not the self-confidence of the average man. The average man seeks certainty in the eyes of the onlooker and calls that self-confidence. The warrior seeks impeccability in his own eyes and calls that humbleness. Tales of Power - C. Castaneda
Quoted 69 lines · Corrado Zoccolo
Quoted 68 lines · Shaohua Li
Quoted 67 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency
Quoted 5 lines
Quoted 2 lines
Thanks for looking at it. can you forward your tests to me so I can check here?
Sure, you can find them at:
* simple seek time: http://dl.dropbox.com/u/3525644/stride.c
* avg seek time with NCQ: http://dl.dropbox.com/u/3525644/ncq.c
great, I'll do some tests.
Quoted 6 lines
Quoted 4 lines
I'll do more aggressive formula next. It shouldn't impact
my SSD which is very fast, each request takes less than 1ms. We'd better have a mechanism to measure device speed, but jiffies isn't good. I'm thinking usingsched_clock()which has its issue too like having drift between CPUs.I'm using
ktime_get()successfully. Measuring is the easy part. The difficult one is decide what to do with the value :)
If we can get the average time a request takes, we can further tweak the quantum check. In fast device, a slice_idle can finish 100 requests.
Thanks,
Shaohua
Quoted 74 lines · Corrado Zoccolo
Quoted 63 lines · Shaohua Li
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 58 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 7 lines
Quoted 5 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latencyThanks for looking at it. can you forward your tests to me so I can check here?
Sure, you can find them at:
* simple seek time: http://dl.dropbox.com/u/3525644/stride.c
* avg seek time with NCQ: http://dl.dropbox.com/u/3525644/ncq.c
Quoted 6 lines
Quoted 4 lines
I'll do more aggressive formula next. It shouldn't impact
my SSD which is very fast, each request takes less than 1ms. We'd better have a mechanism to measure device speed, but jiffies isn't good. I'm thinking usingsched_clock()which has its issue too like having drift between CPUs.I'm using
ktime_get()successfully. Measuring is the easy part. The difficult one is decide what to do with the value :)
BTW, is ktime_get() light enough for a 40k/s call? I thought we should have a very light clock for counting.
Thanks,
Shaohua
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.
Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.
But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch). Were you running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.
I think this patch breaks the meaning of cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respecting cfq_quantum. Especially it can help in testing. With this patch cfq_quantum will more or less loose its meaning.
Vivek
Quoted 60 lines
Quoted 11 lines
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
For async io, 40ms/8ms = 5 - quantum = 1, we only send extra 1 request in maxium. For sync io, 100ms/8ms = 12 - quantum = 8, we might send extra 8 requests in maxium. This might cause latency issue if the queue is preempted at the very beginning.This patch boost my workload from 78m/s to 102m/s, which isn't that big as my last post, but also is a big improvement.
Acked-by Corrado Zoccolo <[email protected]>Quoted 33 lines
Signed-off-by Shaohua Li <[email protected]>---
block/cfq-iosched.c | 15 ++++++++++++++-1 file changed, 14 insertions(+), 1 deletion(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +13
-0@@ -2242,6 +2242,19 @@ static int cfq_forced_dispatch(struct cf return dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, + struct cfq_queue *cfqq) +{ + /* the queue hasn't finished any request, can't estimate */ + if (cfq_cfqq_slice_new(cfqq)) + return true; + if (time_after(jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) + return true; + + return false; +} + static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) { unsigned int max_dispatch;block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2294
Quoted 8 lines
block/cfq-iosched.c +1
-1@@ -2275,7 +2288,7 @@ static bool cfq_may_dispatch(struct cfq_ /* * We have other queues, don't allow more IO from this one */ - if (cfqd->busy_queues > 1) + if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; /*--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 76 lines · Vivek Goyal
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch). Were you running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.
Quoted 5 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
I guess this is a question of soft limit and hard limit. May be we can bump up default cfq_quantum to 8 and internally define a soft limit of of 50% of cfq_quantum. So we will start throttling number of requests from a queue when cfqq->dispatched reaches 4. But will allow more dispatches up to cfq_quantum based on how much slice is left and what's the possiblity that already dispatched request will finish with-in the slice.
That way we will maintain existing behavior, meaning of cfq_quantum as well as possibly get performance improvement in the said case.
Vivek
Quoted 60 lines
Quoted 60 lines
Quoted 11 lines
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
For async io, 40ms/8ms = 5 - quantum = 1, we only send extra 1 request in maxium. For sync io, 100ms/8ms = 12 - quantum = 8, we might send extra 8 requests in maxium. This might cause latency issue if the queue is preempted at the very beginning.This patch boost my workload from 78m/s to 102m/s, which isn't that big as my last post, but also is a big improvement.
Acked-by Corrado Zoccolo <[email protected]>Quoted 33 lines
Signed-off-by Shaohua Li <[email protected]>---
block/cfq-iosched.c | 15 ++++++++++++++-1 file changed, 14 insertions(+), 1 deletion(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +13
-0@@ -2242,6 +2242,19 @@ static int cfq_forced_dispatch(struct cf return dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, + struct cfq_queue *cfqq) +{ + /* the queue hasn't finished any request, can't estimate */ + if (cfq_cfqq_slice_new(cfqq)) + return true; + if (time_after(jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) + return true; + + return false; +} + static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) { unsigned int max_dispatch;block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2294
Quoted 8 lines
block/cfq-iosched.c +1
-1@@ -2275,7 +2288,7 @@ static bool cfq_may_dispatch(struct cfq_ /* * We have other queues, don't allow more IO from this one */ - if (cfqd->busy_queues > 1) + if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; /*--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 70 lines · Vivek Goyal
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.
Quoted 2 lines
But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch).
I think here we want to determine the average cost of a request, when there are many submitted.
Quoted 4 lines
Were you
running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.
Seeky queues. Seeks where rather small (not more than 1/64 of the whole disk), but already meaningful for comparison.
Quoted 5 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
cfq_quantum will still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.
One can argue, instead, that this reduces a bit the effectiveness of preemption on ncq disks.
However, I don't think preemption is the solution for low latency, while cfq_quantum reduction is.
With this change in place, we could change the default cfq_quantum to a smaller number (ideally 1), to have lowest number of leftovers when the slice finishes, while still driving deep queues at the beginning of the slice.
This needs thorough testing, though. Maybe it is better to delay those changes to 2.6.34...
Thanks,
Corrado
Quoted 56 lines
Vivek
Quoted 53 lines
Quoted 11 lines
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
For async io, 40ms/8ms = 5 - quantum = 1, we only send extra 1 request in maxium. For sync io, 100ms/8ms = 12 - quantum = 8, we might send extra 8 requests in maxium. This might cause latency issue if the queue is preempted at the very beginning.This patch boost my workload from 78m/s to 102m/s, which isn't that big as my last post, but also is a big improvement.
Acked-by Corrado Zoccolo <[email protected]>Quoted 39 lines
Signed-off-by Shaohua Li <[email protected]>---
block/cfq-iosched.c | 15 ++++++++++++++-1 file changed, 14 insertions(+), 1 deletion(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +14
-1@@ -2242,6 +2242,19 @@ static int cfq_forced_dispatch(struct cf return dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, + struct cfq_queue *cfqq) +{ + /* the queue hasn't finished any request, can't estimate */ + if (cfq_cfqq_slice_new(cfqq)) + return true; + if (time_after(jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) + return true; + + return false; +} + static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) { unsigned int max_dispatch;@@ -2275,7 +2288,7 @@ static bool cfq_may_dispatch(struct cfq_ /* * We have other queues, don't allow more IO from this one */ - if (cfqd->busy_queues > 1) + if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; /*
Quoted 86 lines · Corrado Zoccolo
Quoted 70 lines · Vivek Goyal
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.Quoted 2 lines
But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch).
I think here we want to determine the average cost of a request, when there are many submitted.
Quoted 4 lines
Were you
running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.Seeky queues. Seeks where rather small (not more than 1/64 of the whole disk), but already meaningful for comparison.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
cfq_quantumwill still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.
Not always and it will depend how accurate your approximation of service time is. If per request completion time is more than approximation (in this case slice_idle), than you will end up with more requests in dispatch queue from one cfqq at the time of slice expiry.
Quoted 9 lines
One can argue, instead, that this reduces a bit the effectiveness of preemption on ncq disks.
However, I don't think preemption is the solution for low latency, whilecfq_quantumreduction is.
With this change in place, we could change the defaultcfq_quantumto a smaller number (ideally 1), to have lowest number of leftovers when the slice finishes, while still driving deep queues at the beginning of the slice.
I think using cfq_quantum as hard limit might be a better idea as it gives more predictable control. Instead of treating it as soft limit and trying to meet it at the end of slice expiry based on our approximation of predicted completion time.
Quoted 3 lines
This needs thorough testing, though. Maybe it is better to delay those changes to 2.6.34...
Agreed. This should be tested more throughly and should be candidate for 2.6.34.
Thanks
Vivek
Quoted 60 lines
Thanks,
CorradoQuoted 56 lines
Vivek
Quoted 53 lines
Quoted 11 lines
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
For async io, 40ms/8ms = 5 - quantum = 1, we only send extra 1 request in maxium. For sync io, 100ms/8ms = 12 - quantum = 8, we might send extra 8 requests in maxium. This might cause latency issue if the queue is preempted at the very beginning.This patch boost my workload from 78m/s to 102m/s, which isn't that big as my last post, but also is a big improvement.
Acked-by Corrado Zoccolo <[email protected]>Quoted 39 lines
Signed-off-by Shaohua Li <[email protected]>---
block/cfq-iosched.c | 15 ++++++++++++++-1 file changed, 14 insertions(+), 1 deletion(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +14
-1@@ -2242,6 +2242,19 @@ static int cfq_forced_dispatch(struct cf return dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, + struct cfq_queue *cfqq) +{ + /* the queue hasn't finished any request, can't estimate */ + if (cfq_cfqq_slice_new(cfqq)) + return true; + if (time_after(jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) + return true; + + return false; +} + static bool cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq) { unsigned int max_dispatch;@@ -2275,7 +2288,7 @@ static bool cfq_may_dispatch(struct cfq_ /* * We have other queues, don't allow more IO from this one */ - if (cfqd->busy_queues > 1) + if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; /*
Quoted 91 lines · Vivek Goyal
Quoted 90 lines · Corrado Zoccolo
Quoted 70 lines · Vivek Goyal
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.Quoted 2 lines
But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch).
I think here we want to determine the average cost of a request, when there are many submitted.
Quoted 4 lines
Were you
running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.Seeky queues. Seeks where rather small (not more than 1/64 of the whole disk), but already meaningful for comparison.
Quoted 5 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
Quoted 8 lines
Quoted 3 lines
cfq_quantumwill still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.Not always and it will depend how accurate your approximation of service time is. If per request completion time is more than approximation (in this case
slice_idle), than you will end up with more requests in dispatch queue from one cfqq at the time of slice expiry.
we use slice_idle for a long time and no complain. So assume the approximation of service time is good.
Quoted 10 lines
Quoted 9 lines
One can argue, instead, that this reduces a bit the effectiveness of preemption on ncq disks.
However, I don't think preemption is the solution for low latency, whilecfq_quantumreduction is.
With this change in place, we could change the defaultcfq_quantumto a smaller number (ideally 1), to have lowest number of leftovers when the slice finishes, while still driving deep queues at the beginning of the slice.
Quoted 4 lines
I think using
cfq_quantumas hard limit might be a better idea as it gives more predictable control. Instead of treating it as soft limit and trying to meet it at the end of slice expiry based on our approximation of predicted completion time.
Current patch has such hard limit too (100ms/8m = 12 for sync io and 40ms/8
= 5 for async io).
Quoted 5 lines
Quoted 2 lines
This needs thorough testing, though. Maybe it is better to delay those changes to 2.6.34...
Agreed. This should be tested more throughly and should be candidate for 2.6.34.
Sure, this needs a lot of test.
Thanks,
Shaohua
Quoted 96 lines · Shaohua Li
Quoted 95 lines · Vivek Goyal
Quoted 93 lines · Corrado Zoccolo
Quoted 70 lines · Vivek Goyal
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.Quoted 2 lines
But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch).
I think here we want to determine the average cost of a request, when there are many submitted.
Quoted 4 lines
Were you
running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.Seeky queues. Seeks where rather small (not more than 1/64 of the whole disk), but already meaningful for comparison.
Quoted 5 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
cfq_quantumwill still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.
Quoted 6 lines
Quoted 4 lines
Not always and it will depend how accurate your approximation of service time is. If per request completion time is more than approximation (in this case
slice_idle), than you will end up with more requests in dispatch queue from one cfqq at the time of slice expiry.we use
slice_idlefor a long time and no complain. So assume the approximation of service time is good.
slice_idle is a variable and user can easily change it to 1ms and even 0. In that case you will be theoritically be ready to dispatch 100/1 requests from the cfqq?
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
One can argue, instead, that this reduces a bit the effectiveness of preemption on ncq disks.
However, I don't think preemption is the solution for low latency, whilecfq_quantumreduction is.
With this change in place, we could change the defaultcfq_quantumto a smaller number (ideally 1), to have lowest number of leftovers when the slice finishes, while still driving deep queues at the beginning of the slice.
Quoted 6 lines
Quoted 4 lines
I think using
cfq_quantumas hard limit might be a better idea as it gives more predictable control. Instead of treating it as soft limit and trying to meet it at the end of slice expiry based on our approximation of predicted completion time.Current patch has such hard limit too (100ms/8m = 12 for sync io and 40ms/8
= 5 for async io).
This is software logic driven and not cfq_quantum driven. We can always keep on changing how to approximate service time completions. So a user first needs to read the code, derive internal limits and then do testing?
I think than tunable looses its significance. That's why I am advocating of treating cfq_quantum as hard limit and derive an internal soft limit based on certain % of hard limit and use that as default max queue depth for cfqq.
In this case user knows no matter what, you are not dispatching more than cfq_quantum requests from a queue at a time.
Quoted 10 lines
Quoted 6 lines
Quoted 3 lines
This needs thorough testing, though. Maybe it is better to delay those changes to 2.6.34...
Agreed. This should be tested more throughly and should be candidate for 2.6.34.
Sure, this needs a lot of test.
Thanks,
Shaohua
Quoted 101 lines · Vivek Goyal
Quoted 100 lines · Shaohua Li
Quoted 99 lines · Vivek Goyal
Quoted 86 lines · Corrado Zoccolo
Quoted 70 lines · Vivek Goyal
Quoted 62 lines · Corrado Zoccolo
Hi Shahoua,
Quoted 52 lines · Shaohua Li
Quoted 41 lines · Corrado Zoccolo
Hi Shaohua,
Quoted 7 lines · Shaohua Li
Quoted 6 lines · Corrado Zoccolo
Quoted 5 lines · Shaohua Li
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do some tweaks:
1. if the queue still has a lot of slice left, sounds we could ignore the limitQuoted 6 lines
Quoted 2 lines
ok. You can even scale the limit proportionally to the remaining slice (see below).
I can't understand the meaning of below scale.
cfq_slice_used_soon()means dispatched requests can finish before slice is used, so other queues will not be impacted. I thought/hope acfq_slice_idletime is enough to finish the dispatched requests.
cfq_slice_idleis 8ms, that is the average time to complete 1 request on most disks. If you have more requests dispatched on a NCQ-rotational disk (non-RAID), it will take more time. Probably a linear formula is not the most accurate, but still more accurate than taking just 1cfq_slice_idle. If you can experiment a bit, you could also try:
cfq_slice_idle *ilog2(nr_dispatched+1)
cfq_slice_idle * (1<<(ilog2(nr_dispatched+1)>>1))Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
2. we could keep the check only when
cfq_latencyis on. For uses who don't care about latency should be happy to have device fully piped on.I wouldn't overload
low_latencywith this meaning. You can obtain the same by setting the quantum to 32.As this impact fairness, so natually thought we could use
low_latency. I'll remove the check in next post.Great.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
I have a test of random direct io of two threads, each has 32 requests one time without patch: 78m/s
with tweak 1: 138m/s
with two tweaks and disable latency: 156m/sPlease, test also with competing seq/random(depth1)/async workloads, and measure also introduced latencies.
depth1 should be ok, as if device can only send one request, it should not require more requests from ioscheduler.
Quoted 10 lines
I mean have a run with, at the same time:
* one seq reader,
* h random readers with depth 1 (non-aio)
* one async seq writer
* k random readers with large depth.
In this way, you can see if the changes you introduce to boost your workload affect more realistic scenarios, in which various workloads are mixed.
I explicitly add the depth1 random readers, since they are sceduled differently than the large (>4) depth ones.Quoted 6 lines
I tried a fio script which does like your description, but the data isn't stable, especially the write speed, other kind of io speed is stable. Apply below patch doesn't make things worse (still write speed isn't stable, other io is stable), so I can't say if the patch passes the test, but it appears latency reported by fio hasn't change. I adopt the
slice_idle* dispatched approach, which I thought should be safe.Quoted 6 lines
I'm doing some tests right now on a single ncq rotational disk, and the average service time when submitting with a high depth is halved w.r.t. depth 1, so I think you could test also with the formula :
slice_idle* dispatched / 2. It should give a performance boost, without noticeable impact on latency.Quoted 2 lines
But I guess the right comparison here would service times vary when we push queue depths from 4 to higher (as done by this patch).
I think here we want to determine the average cost of a request, when there are many submitted.
Quoted 4 lines
Were you
running deep seeky queues or sequential queues. Curious to know whether service times reduced even in case of deep seeky queues on this single disk.Seeky queues. Seeks where rather small (not more than 1/64 of the whole disk), but already meaningful for comparison.
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
cfq_quantumwill still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.Not always and it will depend how accurate your approximation of service time is. If per request completion time is more than approximation (in this case
slice_idle), than you will end up with more requests in dispatch queue from one cfqq at the time of slice expiry.
Quoted 6 lines
Quoted 2 lines
we use
slice_idlefor a long time and no complain. So assume the approximation of service time is good.
slice_idleis a variable and user can easily change it to 1ms and even 0. In that case you will be theoritically be ready to dispatch 100/1 requests from the cfqq?
User changing it should know what he does. A less-experienced user can mess a lot of things, which we don't care.
Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
One can argue, instead, that this reduces a bit the effectiveness of preemption on ncq disks.
However, I don't think preemption is the solution for low latency, whilecfq_quantumreduction is.
With this change in place, we could change the defaultcfq_quantumto a smaller number (ideally 1), to have lowest number of leftovers when the slice finishes, while still driving deep queues at the beginning of the slice.Quoted 6 lines
Quoted 4 lines
I think using
cfq_quantumas hard limit might be a better idea as it gives more predictable control. Instead of treating it as soft limit and trying to meet it at the end of slice expiry based on our approximation of predicted completion time.Current patch has such hard limit too (100ms/8m = 12 for sync io and 40ms/8
= 5 for async io).This is software logic driven and not
cfq_quantumdriven. We can always keep on changing how to approximate service time completions. So a user first needs to read the code, derive internal limits and then do testing?
Quoted 7 lines
I think than tunable looses its significance. That's why I am advocating of treating
cfq_quantumas hard limit and derive an internal soft limit based on certain % of hard limit and use that as default max queue depth for cfqq.In this case user knows no matter what, you are not dispatching more than
cfq_quantumrequests from a queue at a time.
ok, then the question is which value should cfq_quantum have. I have a test with below patch. Its performance still is good. With hard limit 8, speed is 100m/s. without hard limit, speed is 102m/s.
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
Test shows this boost my workload (two thread randread of a SSD) from 78m/s to 100m/s.
--- block/cfq-iosched.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-)
block/cfq-iosched.c +21 -3
@@ -19,7 +19,7 @@* tunables */ /* max queue in one round of service */ -static const int cfq_quantum =4 ; +static const int cfq_quantum =8 ;static const int cfq_fifo_expire[2 ] = {HZ /4 ,HZ /8 };/* maximum backwards seek, in KiB */ static const int cfq_back_max =16 *1024 ;
@@ -32,6 +32,8 @@ static int cfq_slice_idle = HZ / 125;static const int cfq_target_latency =HZ *3 /10 ;/* 300 ms */ static const int cfq_hist_divisor =4 ; +#define CFQ_SOFT_QUANTUM (4) +/* * offset from end of service tree */
@@ -2242,6 +2244,19 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq) || cfqq->dispatched >= cfqd->cfq_quantum) +return 1 ; +if (time_after (jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) +return 1 ; + +return 0 ; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;
@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false; - max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum /2 ; +if (max_dispatch <CFQ_SOFT_QUANTUM ) + max_dispatch = min_t(unsigned int, CFQ_SOFT_QUANTUM, + cfqd->cfq_quantum);if (cfq_class_idle (cfqq)) max_dispatch =1 ;
@@ -2275,7 +2293,7 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;/*
On Tue, Jan 12, 2010 at 11:07:56AM +0800, Shaohua Li wrote:
[..]
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 7 lines
Quoted 4 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
cfq_quantumwill still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.Not always and it will depend how accurate your approximation of service time is. If per request completion time is more than approximation (in this case
slice_idle), than you will end up with more requests in dispatch queue from one cfqq at the time of slice expiry.
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
we use
slice_idlefor a long time and no complain. So assume the approximation of service time is good.
slice_idleis a variable and user can easily change it to 1ms and even 0. In that case you will be theoritically be ready to dispatch 100/1 requests from the cfqq?User changing it should know what he does. A less-experienced user can mess a lot of things, which we don't care.
The point is that there is no obivious co-relation between slice_idle and cfq_quantum. Even an experienced user would not expect that changing slice_idle silently will enable dispatching more requests from each cfqq.
Quoted 88 lines
Quoted 20 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
One can argue, instead, that this reduces a bit the effectiveness of preemption on ncq disks.
However, I don't think preemption is the solution for low latency, whilecfq_quantumreduction is.
With this change in place, we could change the defaultcfq_quantumto a smaller number (ideally 1), to have lowest number of leftovers when the slice finishes, while still driving deep queues at the beginning of the slice.Quoted 6 lines
Quoted 4 lines
I think using
cfq_quantumas hard limit might be a better idea as it gives more predictable control. Instead of treating it as soft limit and trying to meet it at the end of slice expiry based on our approximation of predicted completion time.Current patch has such hard limit too (100ms/8m = 12 for sync io and 40ms/8
= 5 for async io).This is software logic driven and not
cfq_quantumdriven. We can always keep on changing how to approximate service time completions. So a user first needs to read the code, derive internal limits and then do testing?Quoted 7 lines
I think than tunable looses its significance. That's why I am advocating of treating
cfq_quantumas hard limit and derive an internal soft limit based on certain % of hard limit and use that as default max queue depth for cfqq.In this case user knows no matter what, you are not dispatching more than
cfq_quantumrequests from a queue at a time.ok, then the question is which value should
cfq_quantumhave. I have a test with below patch. Its performance still is good. With hard limit 8, speed is 100m/s. without hard limit, speed is 102m/s.Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
Test shows this boost my workload (two thread randread of a SSD) from 78m/s to 100m/s.Signed-off-by Shaohua Li <[email protected]>--- block/cfq-iosched.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +16
-1@@ -19,7 +19,7 @@* tunables */ /* max queue in one round of service */ -static const int cfq_quantum =4 ; +static const int cfq_quantum =8 ;static const int cfq_fifo_expire[2 ] = {HZ /4 ,HZ /8 };/* maximum backwards seek, in KiB */ static const int cfq_back_max =16 *1024 ;@@ -32,6 +32,8 @@ static int cfq_slice_idle = HZ / 125;static const int cfq_target_latency =HZ *3 /10 ;/* 300 ms */ static const int cfq_hist_divisor =4 ; +#define CFQ_SOFT_QUANTUM (4) +/* * offset from end of service tree */ @@ -2242,6 +2244,19 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq) || cfqq->dispatched >= cfqd->cfq_quantum) +return 1 ; +if (time_after (jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) +return 1 ; + +return 0 ; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 6 lines
block/cfq-iosched.c +2
-1@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false; - max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum /2 ; +if (max_dispatch <CFQ_SOFT_QUANTUM )
We don't have to hardcode CFQ_SOFT_QUANTUM or in fact we don't need it. We can derive the soft limit from hard limit (cfq_quantum). Say soft limit will be 50% of cfq_quantum value.
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2297
Quoted 12 lines
block/cfq-iosched.c +3
-1+ max_dispatch = min_t(unsigned int, CFQ_SOFT_QUANTUM, + cfqd->cfq_quantum);if (cfq_class_idle (cfqq)) max_dispatch =1 ;@@ -2275,7 +2293,7 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;
So I guess here we can write something as follows.
if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; if (cfqd->busy_queues == 1) max_dispatch = -1; else /* * Normally we start throttling cfqq when cfq_quantum/2 * requests have been dispatched. But we can drive * deeper queue depths at the beginning of slice * subjected to upper limit of cfq_quantum. */ max_dispatch = cfqd->cfq_quantum;
Thanks
Vivek
Quoted 21 lines · Vivek Goyal
On Tue, Jan 12, 2010 at 11:07:56AM +0800, Shaohua Li wrote:
[..]
Quoted 18 lines
Quoted 18 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
I think this patch breaks the meaning of
cfq_quantum? Now we can allow dispatch of more requests from the same queue. I had kind of liked the idea of respectingcfq_quantum. Especially it can help in testing. With this patchcfq_quantumwill more or less loose its meaning.
cfq_quantumwill still be enforced at the end of the slice, so its meaning of how many requests can be still pending when you finish your slice is preserved.Quoted 6 lines
Quoted 4 lines
Not always and it will depend how accurate your approximation of service time is. If per request completion time is more than approximation (in this case
slice_idle), than you will end up with more requests in dispatch queue from one cfqq at the time of slice expiry.we use
slice_idlefor a long time and no complain. So assume the approximation of service time is good.
slice_idleis a variable and user can easily change it to 1ms and even 0. In that case you will be theoritically be ready to dispatch 100/1 requests from the cfqq?
Quoted 7 lines
Quoted 3 lines
User changing it should know what he does. A less-experienced user can mess a lot of things, which we don't care.
The point is that there is no obivious co-relation between
slice_idleandcfq_quantum. Even an experienced user would not expect that changingslice_idlesilently will enable dispatching more requests from each cfqq.
Agree slice_idle hasn't relationship with cfq_quantum. Yes, there are more requests dispatched, but it shouldn't impact user experience. If it does, then the patch fails.
Quoted 48 lines
Quoted 48 lines
1 file changed, 21 insertions(+), 3 deletions(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +16
-1@@ -19,7 +19,7 @@* tunables */ /* max queue in one round of service */ -static const int cfq_quantum =4 ; +static const int cfq_quantum =8 ;static const int cfq_fifo_expire[2 ] = {HZ /4 ,HZ /8 };/* maximum backwards seek, in KiB */ static const int cfq_back_max =16 *1024 ;@@ -32,6 +32,8 @@ static int cfq_slice_idle = HZ / 125;static const int cfq_target_latency =HZ *3 /10 ;/* 300 ms */ static const int cfq_hist_divisor =4 ; +#define CFQ_SOFT_QUANTUM (4) +/* * offset from end of service tree */ @@ -2242,6 +2244,19 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq) || cfqq->dispatched >= cfqd->cfq_quantum) +return 1 ; +if (time_after (jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) +return 1 ; + +return 0 ; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false;
Quoted 7 lines
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 3 lines
block/cfq-iosched.c +2
-1- max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum /2 ; +if (max_dispatch <CFQ_SOFT_QUANTUM )We don't have to hardcode
CFQ_SOFT_QUANTUMor in fact we don't need it. We can derive the soft limit from hard limit (cfq_quantum). Say soft limit will be 50% ofcfq_quantumvalue.
I'm hoping this doesn't give user a surprise. Say cfq_quantum sets to 7, then we start doing throttling from 3 requests. Adding the CFQ_SOFT_QUANTUM gives a compatibility against old behavior at least. Am I over thinking?
Quoted 15 lines
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2297
Quoted 12 lines
block/cfq-iosched.c +3
-1+ max_dispatch = min_t(unsigned int, CFQ_SOFT_QUANTUM, + cfqd->cfq_quantum);if (cfq_class_idle (cfqq)) max_dispatch =1 ;@@ -2275,7 +2293,7 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;So I guess here we can write something as follows.
Quoted 13 lines
if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; if (cfqd->busy_queues == 1) max_dispatch = -1; else /* * Normally we start throttling cfqq when cfq_quantum/2 * requests have been dispatched. But we can drive * deeper queue depths at the beginning of slice * subjected to upper limit of cfq_quantum. */ max_dispatch = cfqd->cfq_quantum;
ok.
Thanks,
Shaohua
On Wed, Jan 13, 2010 at 04:17:35PM +0800, Shaohua Li wrote:
[..]
Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
static bool
cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq){block/cfq-iosched.c +0
-0unsigned int max_dispatch;@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false;Quoted 7 lines
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 3 lines
block/cfq-iosched.c +2
-1- max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum /2 ; +if (max_dispatch <CFQ_SOFT_QUANTUM )We don't have to hardcode
CFQ_SOFT_QUANTUMor in fact we don't need it. We can derive the soft limit from hard limit (cfq_quantum). Say soft limit will be 50% ofcfq_quantumvalue.
Quoted 4 lines
I'm hoping this doesn't give user a surprise. Say
cfq_quantumsets to 7, then we start doing throttling from 3 requests. Adding theCFQ_SOFT_QUANTUMgives a compatibility against old behavior at least. Am I over thinking?
I would not worry too much about that. If you are really worried about that, then create one Documentation/block/cfq-iosched.txt and document how cfq_quantum works so that users know that cfq_quantum is upper hard limit and internal soft limit is cfq_quantum/2.
Thanks
Vivek
Quoted 32 lines
Quoted 15 lines
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2297
Quoted 12 lines
block/cfq-iosched.c +3
-1+ max_dispatch = min_t(unsigned int, CFQ_SOFT_QUANTUM, + cfqd->cfq_quantum);if (cfq_class_idle (cfqq)) max_dispatch =1 ;@@ -2275,7 +2293,7 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;So I guess here we can write something as follows.
Quoted 13 lines
if (cfqd->busy_queues > 1 && cfq_slice_used_soon(cfqd, cfqq)) return false; if (cfqd->busy_queues == 1) max_dispatch = -1; else /* * Normally we start throttling cfqq when cfq_quantum/2 * requests have been dispatched. But we can drive * deeper queue depths at the beginning of slice * subjected to upper limit of cfq_quantum. */ max_dispatch = cfqd->cfq_quantum;ok.
Thanks,
Shaohua
Quoted 21 lines · Vivek Goyal
On Wed, Jan 13, 2010 at 04:17:35PM +0800, Shaohua Li wrote:
[..]Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
static bool
cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq){(quoted diff) +0
-0unsigned int max_dispatch;@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_ if (cfqd->sync_flight && !cfq_cfqq_sync(cfqq)) return false;Quoted 7 lines
Review static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 3 lines
(quoted diff) +2
-1- max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum / 2; + if (max_dispatch < CFQ_SOFT_QUANTUM)We don't have to hardcode
CFQ_SOFT_QUANTUMor in fact we don't need it. We can derive the soft limit from hard limit (cfq_quantum). Say soft limit will be 50% ofcfq_quantumvalue.Quoted 4 lines
I'm hoping this doesn't give user a surprise. Say
cfq_quantumsets to 7, then we start doing throttling from 3 requests. Adding theCFQ_SOFT_QUANTUMgives a compatibility against old behavior at least. Am I over thinking?
Quoted 4 lines
I would not worry too much about that. If you are really worried about that, then create one
Documentation/block/cfq-iosched.txtand document howcfq_quantumworks so that users know thatcfq_quantumis upper hard limit and internal soft limit is cfq_quantum/2.
Good idea. Looks we don't document cfq tunnables, I'll try to do it later.
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
Test shows this boost my workload (two thread randread of a SSD) from 78m/s to 100m/s.
--- block/cfq-iosched.c | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-)
block/cfq-iosched.c +26 -4
@@ -19,7 +19,7 @@* tunables */ /* max queue in one round of service */ -static const int cfq_quantum =4 ; +static const int cfq_quantum =8 ;static const int cfq_fifo_expire[2 ] = {HZ /4 ,HZ /8 };/* maximum backwards seek, in KiB */ static const int cfq_back_max =16 *1024 ;
@@ -2215,6 +2215,19 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq)) +return 1 ; +if (time_after (jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) +return 1 ; + +return 0 ; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;
@@ -2231,7 +2244,7 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false; - max_dispatch = cfqd->cfq_quantum; + max_dispatch = max_t(unsigned int, cfqd->cfq_quantum / 2, 1);if (cfq_class_idle (cfqq)) max_dispatch =1 ;
@@ -2248,13 +2261,22 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;/* * Sole queue user, no limit */ - max_dispatch =-1 ; +if (cfqd->busy_queues ==1 ) + max_dispatch =-1 ; +else +/* +* Normally we start throttling cfqq when cfq_quantum/2 +* requests have been dispatched. But we can drive +* deeper queue depths at the beginning of slice +* subjected to upper limit of cfq_quantum. +* */ + max_dispatch = cfqd->cfq_quantum; }/*
Quoted 26 lines · Shaohua Li
Quoted 21 lines · Vivek Goyal
On Wed, Jan 13, 2010 at 04:17:35PM +0800, Shaohua Li wrote:
[..]Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
static bool
cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq){block/cfq-iosched.c +0
-0unsigned int max_dispatch;@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false;Quoted 7 lines
Review static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 3 lines
(quoted diff) +2
-1- max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum / 2; + if (max_dispatch < CFQ_SOFT_QUANTUM)We don't have to hardcode
CFQ_SOFT_QUANTUMor in fact we don't need it. We can derive the soft limit from hard limit (cfq_quantum). Say soft limit will be 50% ofcfq_quantumvalue.Quoted 4 lines
I'm hoping this doesn't give user a surprise. Say
cfq_quantumsets to 7, then we start doing throttling from 3 requests. Adding theCFQ_SOFT_QUANTUMgives a compatibility against old behavior at least. Am I over thinking?Quoted 4 lines
I would not worry too much about that. If you are really worried about that, then create one
Documentation/block/cfq-iosched.txtand document howcfq_quantumworks so that users know thatcfq_quantumis upper hard limit and internal soft limit is cfq_quantum/2.
Quoted 7 lines
Good idea. Looks we don't document cfq tunnables, I'll try to do it later.
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
Hi Shaohua,
This looks much better. Though usage of "slice_idle" as measure of service times, I find little un-intutive. Especially, I do some testing with slice_idle=0, in that case, we will be allowing dispatch of 8 requests from each queue even if slice is about to expire.
But I guess that's fine for the time being as upper limit is still controlld by cfq_quantum.
Quoted 2 lines
Test shows this boost my workload (two thread randread of a SSD) from 78m/s to 100m/s.
Are these deep queue random reads (with higher iodepths, using libaio)?
Have you done similar test on some slower NCQ rotational hardware also and seen the impact on throughput and *max latency* of readers, especially in the presence of buffered writers.
Thanks
Vivek
Quoted 73 lines
Signed-off-by Shaohua Li <[email protected]>--- block/cfq-iosched.c | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-)Index:
linux-2.6/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +26
-4@@ -19,7 +19,7 @@* tunables */ /* max queue in one round of service */ -static const int cfq_quantum =4 ; +static const int cfq_quantum =8 ;static const int cfq_fifo_expire[2 ] = {HZ /4 ,HZ /8 };/* maximum backwards seek, in KiB */ static const int cfq_back_max =16 *1024 ;@@ -2215,6 +2215,19 @@ static int cfq_forced_dispatch(struct cfreturn dispatched; } +static inline bool cfq_slice_used_soon (struct cfq_data *cfqd, +struct cfq_queue *cfqq) +{ +/* the queue hasn't finished any request, can't estimate */ +if (cfq_cfqq_slice_new (cfqq)) +return 1 ; +if (time_after (jiffies + cfqd->cfq_slice_idle * cfqq->dispatched, + cfqq->slice_end)) +return 1 ; + +return 0 ; +} +static bool cfq_may_dispatch (struct cfq_data *cfqd,struct cfq_queue *cfqq) {unsigned int max_dispatch;@@ -2231,7 +2244,7 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false; - max_dispatch = cfqd->cfq_quantum; + max_dispatch = max_t(unsigned int, cfqd->cfq_quantum / 2, 1);if (cfq_class_idle (cfqq)) max_dispatch =1 ;@@ -2248,13 +2261,22 @@ static bool cfq_may_dispatch(struct cfq_/* * We have other queues, don't allow more IO from this one */ -if (cfqd->busy_queues >1 ) +if (cfqd->busy_queues >1 &&cfq_slice_used_soon (cfqd, cfqq))return false;/* * Sole queue user, no limit */ - max_dispatch =-1 ; +if (cfqd->busy_queues ==1 ) + max_dispatch =-1 ; +else +/* +* Normally we start throttling cfqq when cfq_quantum/2 +* requests have been dispatched. But we can drive +* deeper queue depths at the beginning of slice +* subjected to upper limit of cfq_quantum. +* */ + max_dispatch = cfqd->cfq_quantum; }/*
Quoted 35 lines · Vivek Goyal
Quoted 26 lines · Shaohua Li
Quoted 21 lines · Vivek Goyal
On Wed, Jan 13, 2010 at 04:17:35PM +0800, Shaohua Li wrote:
[..]Quoted 14 lines
Quoted 7 lines
Quoted 7 lines
static bool
cfq_may_dispatch(struct cfq_data *cfqd, struct cfq_queue *cfqq){block/cfq-iosched.c +0
-0unsigned int max_dispatch;@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false;Quoted 7 lines
Review static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 3 lines
(quoted diff) +2
-1- max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum / 2; + if (max_dispatch < CFQ_SOFT_QUANTUM)We don't have to hardcode
CFQ_SOFT_QUANTUMor in fact we don't need it. We can derive the soft limit from hard limit (cfq_quantum). Say soft limit will be 50% ofcfq_quantumvalue.Quoted 4 lines
I'm hoping this doesn't give user a surprise. Say
cfq_quantumsets to 7, then we start doing throttling from 3 requests. Adding theCFQ_SOFT_QUANTUMgives a compatibility against old behavior at least. Am I over thinking?Quoted 4 lines
I would not worry too much about that. If you are really worried about that, then create one
Documentation/block/cfq-iosched.txtand document howcfq_quantumworks so that users know thatcfq_quantumis upper hard limit and internal soft limit is cfq_quantum/2.Quoted 7 lines
Good idea. Looks we don't document cfq tunnables, I'll try to do it later.
Currently a queue can only dispatch up to 4 requests if there are other queues. This isn't optimal, device can handle more requests, for example, AHCI can handle 31 requests. I can understand the limit is for fairness, but we could do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
Quoted 6 lines
Hi Shaohua,
This looks much better. Though usage of "
slice_idle" as measure of service times, I find little un-intutive. Especially, I do some testing with slice_idle=0, in that case, we will be allowing dispatch of 8 requests from each queue even if slice is about to expire.
I agree this is problematic, but I also think we need to do something about the control of queuing depth. For most users, keeping it low is what they want - performance doesn't change much with higher depths, you only pay a latency cost when switching to a new queue. And they don't want that.
But for other hardware, driving up the queue depth to what the hardware supports (potentially) can be a big win, and CFQ definitely needs to be able to do that.
Write caches are again problematic in this area... For reads and writes on write through caching, just looking at what this cfqq has already dispatched and completed in this slice would be sufficient. It could even be carried over to the next slice as a seed value, so you could dispatch more earlier. What we want to avoid is stuffing the device queue with tons of writes that complete immediately, only to move the pentalty of those requests into the slices of other queues.
Signature · 1 line
Jens Axboe
Quoted 64 lines
-----Original Message-----
From: Vivek Goyal [mailto:[email protected]]
Sent: Thursday, January 14, 2010 7:31 PM
To: Li, Shaohua
Cc: Corrado Zoccolo; [email protected]; [email protected]; Zhang, Yanmin
Subject: Re: [RFC]cfq-iosched: quantum check tweakQuoted 4 lines · Shaohua Li
Quoted 3 lines · Vivek Goyal
On Wed, Jan 13, 2010 at 04:17:35PM +0800, Shaohua Li wrote:
[..]Quoted 1 line
Quoted 1 line
Quoted 1 line
static bool cfq_may_dispatch(struct
cfq_data*cfqd, struct
cfq_queue*cfqq)Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
block/cfq-iosched.c static bool cfq_may_dispatch(struct cfq_ · line 2277
Quoted 9 lines
{
block/cfq-iosched.c +2
-1unsigned int max_dispatch;@@ -2258,7 +2273,10 @@ static bool cfq_may_dispatch(struct cfq_if (cfqd->sync_flight && !cfq_cfqq_sync (cfqq))return false; - max_dispatch = cfqd->cfq_quantum; + max_dispatch = cfqd->cfq_quantum /2 ; +if (max_dispatch <CFQ_SOFT_QUANTUM )We don't have to hardcode
CFQ_SOFT_QUANTUMor in fact we don'tneed it. We can
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
derive the soft limit from hard limit (
cfq_quantum). Say softlimit will be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
50% of
cfq_quantumvalue.I'm hoping this doesn't give user a surprise. Say
cfq_quantumsetsto 7, then we
Quoted 1 line
Quoted 1 line
Quoted 1 line
start doing throttling from 3 requests. Adding the
CFQ_SOFT_QUANTUMgives a compatibility
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
against old behavior at least. Am I over thinking?
I would not worry too much about that. If you are really worried about that, then create one
Documentation/block/cfq-iosched.txtand document howcfq_quantumworks so that users know thatcfq_quantumis upperhard
Quoted 2 lines
Quoted 1 line
limit and internal soft limit is cfq_quantum/2.
Good idea. Looks we don't document cfq tunnables, I'll try to do it
later.
Quoted 2 lines
Currently a queue can only dispatch up to 4 requests if there are other
queues.
Quoted 1 line
This isn't optimal, device can handle more requests, for example, AHCI
can
Quoted 1 line
handle 31 requests. I can understand the limit is for fairness, but we
could
Quoted 2 lines
do a tweak: if the queue still has a lot of slice left, sounds we could ignore the limit.
Hi Shaohua,
This looks much better. Though usage of "
slice_idle" as measure of service times, I find little un-intutive. Especially, I do some testing with slice_idle=0, in that case, we will be allowing dispatch of 8 requests from each queue even if slice is about to expire.But I guess that's fine for the time being as upper limit is still controlld by
cfq_quantum.Quoted 1 line
Test shows this boost my workload (two thread randread of a SSD) from
Quoted 8 lines
78m/s
Quoted 1 line
to 100m/s.
Are these deep queue random reads (with higher iodepths, using libaio)?
Have you done similar test on some slower NCQ rotational hardware also and seen the impact on throughput and *max latency* of readers, especially in the presence of buffered writers.
Tested in a 320g hardidisk (ST3320620AS). The throughput improves about 6% and average latency drops 6% too. Below is the fio output, I tested 3 run for each case, the result is similar.
No patch case:
sdb: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
sdb: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32 Starting 2 processes
sdb: (groupid=0, jobs=2): err= 0: pid=3389
read : io=90,900KiB, bw=755KiB/s, iops=188, runt=120336msec
slat (usec): min=8, max=527K, avg=679.01, stdev=6101.05
clat (usec): min=0, max=0, avg= 0.00, stdev= 0.00
bw (KiB/s) : min= 0, max= 837, per=47.35%, avg=357.50, stdev=78.71
cpu : usr=0.02%, sys=0.13%, ctx=22661, majf=0, minf=169
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w: total=22725/0, short=0/0
lat (msec): 10=0.04%, 20=1.34%, 50=7.42%, 100=8.05%, 250=31.58%
lat (msec): 500=30.38%, 750=13.79%, 1000=5.27%, 2000=2.14%Run status group 0 (all jobs):
READ: io=90,900KiB, aggrb=755KiB/s, minb=755KiB/s, maxb=755KiB/s, mint=120336msec, maxt=120336msec
--------------------------------------------------------------------------
Patched case:
sdb: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
sdb: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=32
Starting 2 processes
sdb: (groupid=0, jobs=2): err= 0: pid=4776
read : io=98,140KiB, bw=815KiB/s, iops=203, runt=120323msec
slat (usec): min=9, max=68, avg=11.23, stdev= 1.03
clat (usec): min=0, max=0, avg= 0.00, stdev= 0.00
bw (KiB/s) : min= 0, max= 534, per=47.28%, avg=385.32, stdev=74.37
cpu : usr=0.04%, sys=0.13%, ctx=24523, majf=0, minf=188
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.7%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued r/w: total=24535/0, short=0/0
lat (msec): 10=0.01%, 20=0.93%, 50=6.50%, 100=7.65%, 250=36.40%
lat (msec): 500=31.81%, 750=11.24%, 1000=4.08%, 2000=1.38%Run status group 0 (all jobs):
READ: io=98,140KiB, aggrb=815KiB/s, minb=815KiB/s, maxb=815KiB/s, mint=120323msec, maxt=120323msec