cfq-iosched: improve latency for no-idle queues (v3)
cover cfq-iosched: improve latency for no-idle queues (v3)
[rebased on top of Jeff's latest changes for 2.6.33. Various code style improvements over v1 & v2]
This patch series is intended to improve I/O latency, addressing an often neglected, important subset of workloads: the ones for which cfq currently prefers not to do any idling.
Those are the ones that would benefit most from having low latency, in fact they are any of:
* processes with large think times (e.g. interactive ones like file managers)
* seeky (e.g. programs faulting in their code at startup)
* or marked as no-idle from upper levels.
The patch series addresses this by:
* reducing queues' timeslice when many queues have pending I/O
* separating queues with different priorities and different characteristics in different service trees, each with an allocated time slice
* enable idling when switching between service trees, even for queues that would not have idling enabled otherwise.
This provides various benefits:
* service tree insertion code is simplified, since it doesn't need to cope with priorities any more.
* high priority no_idle queues are no longer penalized when competing with lower priority, idling queues
* seeky and no_idle queues have their fair share of disk time, without penalizing NCQ drives' performances, since they can all dispatch together, filling up the available NCQ slots.
On a non-NCQ capable drive, a workload of 4 random readers competing with sequential writer, the maximum latency experienced by readers decreased from > 500ms to about 160ms.
Quoted 28 lines · Corrado Zoccolo
[rebased on top of Jeff's latest changes for 2.6.33. Various code style improvements over v1 & v2]
This patch series is intended to improve I/O latency, addressing an often neglected, important subset of workloads: the ones for which cfq currently prefers not to do any idling.
Those are the ones that would benefit most from having low latency, in fact they are any of:
* processes with large think times (e.g. interactive ones like file managers)
* seeky (e.g. programs faulting in their code at startup)
* or marked as no-idle from upper levels.The patch series addresses this by:
* reducing queues' timeslice when many queues have pending I/O
* separating queues with different priorities and different characteristics in different service trees, each with an allocated time slice
* enable idling when switching between service trees, even for queues that would not have idling enabled otherwise.This provides various benefits:
* service tree insertion code is simplified, since it doesn't need to cope with priorities any more.
* high priorityno_idlequeues are no longer penalized when competing with lower priority, idling queues
* seeky andno_idlequeues have their fair share of disk time, without penalizing NCQ drives' performances, since they can all dispatch together, filling up the available NCQ slots.
Quoted 3 lines
On a non-NCQ capable drive, a workload of 4 random readers competing with sequential writer, the maximum latency experienced by readers decreased from > 500ms to about 160ms.
Thanks Corrado, this is indeed good stuff. Only style issue left was the one in cfq_get_avg_queues(), I just corrected that manually.
I have committed this in a test branch based off for-2.6.33 and will do some testing with it, then merge it into for-2.6.33 if it looks good.
Signature · 1 line
Jens Axboe
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
cover cfq-iosched: improve latency for no-idle queues (v3)
Hi Jens,
Jeff did some testing of this patchset on his NCQ-enabled SSD (the 30GB OCZ Vertex).
The test suite contained various multiple competing workloads scenarios, and was run on for-2.6.33 and cfq-2.6.33 branches.
Max latencies were reduced in most cases, and we had also improvements on bandwidth side in some scenarios, especially
for multiple random readers, either alone or competing with writes. 2 random readers aggregate bw increased from 48356 to 74205 and 4 random readers vs 1 seq writer:
* aggregate reader bw increased from 35242 to 56400
* writer bandwidth increased from 33269 to 55127
* maximum latency on read decreased from 535 to 324
* maximum latency on writes decreased from 22243 to 1153 It's a win on all measures.
The effect increasing the number of readers to 32 (latency_test_2.fio) is even more visible (max read latency reduced from 3305 to 268, aggregated read BW increased from 32894 to 164571).
The only case where I see an increased max latency is for 2 random readers vs 1 seq reader:
for-2.6.33:
randomread.0: read_bw = 15,418K
randomread.1: read_bw = 15,399K
seqread: read_bw = 409K
0: read_bw = 31226
0: read_lat_max = 11.589
0: read_lat_avg = 3.22366666666667
cfq-2.6.33:
randomread.0: read_bw = 10,065K
randomread.1: read_bw = 10,067K
seqread: read_bw = 101M
0: read_bw = 121132
0: read_lat_max = 303
0: read_lat_avg = 0.282333333333333
but here the increased latency is paid back by a large increase in sequential read BW (the max latency is, btw, experienced by the seq reader, so I think it is a fair behaviour).
Jeff observed that the for-2.6.33 numbers were worse than his baseline runs, probably due to changed hw_tag detection.
My patchset is much less sensible to hw_tag on SSDs (since there are much less situations in which it would idle), so my numbers are unaffected.
Corrado
Quoted 43 lines · Corrado Zoccolo
Hi Jens,
Jeff did some testing of this patchset on his NCQ-enabled SSD (the 30GB OCZ Vertex).
The test suite contained various multiple competing workloads scenarios, and was run on for-2.6.33 and cfq-2.6.33 branches.Max latencies were reduced in most cases, and we had also improvements on bandwidth side in some scenarios, especially
for multiple random readers, either alone or competing with writes. 2 random readers aggregate bw increased from 48356 to 74205 and 4 random readers vs 1 seq writer:
* aggregate reader bw increased from 35242 to 56400
* writer bandwidth increased from 33269 to 55127
* maximum latency on read decreased from 535 to 324
* maximum latency on writes decreased from 22243 to 1153 It's a win on all measures.
The effect increasing the number of readers to 32 (latency_test_2.fio) is even more visible (max read latency reduced from 3305 to 268, aggregated read BW increased from 32894 to 164571).The only case where I see an increased max latency is for 2 random readers vs 1 seq reader:
for-2.6.33:
randomread.0:read_bw= 15,418K
randomread.1:read_bw= 15,399K
seqread:read_bw= 409K
0:read_bw= 31226
0:read_lat_max= 11.589
0:read_lat_avg= 3.22366666666667cfq-2.6.33:
randomread.0:read_bw= 10,065K
randomread.1:read_bw= 10,067K
seqread:read_bw= 101M
0:read_bw= 121132
0:read_lat_max= 303
0:read_lat_avg= 0.282333333333333but here the increased latency is paid back by a large increase in sequential read BW (the max latency is, btw, experienced by the seq reader, so I think it is a fair behaviour).
Quoted 5 lines
Jeff observed that the for-2.6.33 numbers were worse than his baseline runs, probably due to changed
hw_tagdetection.
My patchset is much less sensible tohw_tagon SSDs (since there are much less situations in which it would idle), so my numbers are unaffected.
Thanks a lot for your testing. My testing on cfq-2.6.33 looks good too, so I pulled it into for-2.6.33 today.
Since for-linus contains conflicting changes, can you and Jeff please double check that everything is still in order? The interesting bit here is the merge with for-2.6.33 and the coop limit from Shaohua Li. I did the straight forward merge, but we likely just need to drop that logic since the coop concept is radically different given that we merge and break queues in for-2.6.33.
Signature · 1 line
Jens Axboe
Quoted 6 lines · Jens Axboe
Since for-linus contains conflicting changes, can you and Jeff please double check that everything is still in order? The interesting bit here is the merge with for-2.6.33 and the coop limit from Shaohua Li. I did the straight forward merge, but we likely just need to drop that logic since the coop concept is radically different given that we merge and break queues in for-2.6.33.
Yeah, since I changed the meaning of the cfqq_coop flag, a lot of those tests are just plain wrong. Let me play with it and I'll send you an incremental patch in a bit.
Cheers,
Jeff
Quoted 9 lines · Jeff Moyer
Quoted 6 lines · Jens Axboe
Since for-linus contains conflicting changes, can you and Jeff please double check that everything is still in order? The interesting bit here is the merge with for-2.6.33 and the coop limit from Shaohua Li. I did the straight forward merge, but we likely just need to drop that logic since the coop concept is radically different given that we merge and break queues in for-2.6.33.
Quoted 3 lines
Yeah, since I changed the meaning of the
cfqq_coopflag, a lot of those tests are just plain wrong. Let me play with it and I'll send you an incremental patch in a bit.
Thanks, here's what I have. It's basically a revert of the commit in question.
block/cfq-iosched.c +2 -19
@@ -253,7 +253,6 @@ enum cfqq_state_flags { CFQ_CFQQ_FLAG_slice_new,/* no requests dispatched in slice */ CFQ_CFQQ_FLAG_sync,/* synchronous queue */ CFQ_CFQQ_FLAG_coop, /* cfqq is shared */ - CFQ_CFQQ_FLAG_coop_preempt, /* coop preempt */ };#define CFQ_CFQQ_FNS (name) \
@@ -280,7 +279,6 @@ CFQ_CFQQ_FNS(prio_changed);CFQ_CFQQ_FNS (slice_new);CFQ_CFQQ_FNS (sync);CFQ_CFQQ_FNS (coop); -CFQ_CFQQ_FNS (coop_preempt);#undef CFQ_CFQQ_FNS #define cfq_log_cfqq(cfqd, cfqq, fmt, args...) \
@@ -1070,16 +1068,9 @@ static struct cfq_queue *cfq_get_next_queue(struct cfq_data *cfqd)static struct cfq_queue *cfq_set_active_queue (struct cfq_data *cfqd,struct cfq_queue *cfqq) { -if (!cfqq) { +if (!cfqq) cfqq =cfq_get_next_queue (cfqd); -if (cfqq && !cfq_cfqq_coop_preempt (cfqq)) -cfq_clear_cfqq_coop (cfqq); - } - -if (cfqq) -cfq_clear_cfqq_coop_preempt (cfqq); -__cfq_set_active_queue (cfqd, cfqq);return cfqq; }
@@ -2433,16 +2424,8 @@ cfq_should_preempt(struct cfq_data *cfqd, struct cfq_queue *new_cfqq,* if this request is as-good as one we would expect from the * current cfqq, let it preempt */ -if (cfq_rq_close (cfqd, cfqq, rq) && (!cfq_cfqq_coop (new_cfqq) || - cfqd->busy_queues ==1 )) { -/* -* Mark new queue coop_preempt, so its coop flag will not be -* cleared when new queue gets scheduled at the very first time -*/ -cfq_mark_cfqq_coop_preempt (new_cfqq); -cfq_mark_cfqq_coop (new_cfqq); +if (cfq_rq_close (cfqd, cfqq, rq))return true; - }return false; }
Signature · 1 line
Jens Axboe
Quoted 10 lines · Jens Axboe
Quoted 9 lines · Jeff Moyer
Quoted 6 lines · Jens Axboe
Since for-linus contains conflicting changes, can you and Jeff please double check that everything is still in order? The interesting bit here is the merge with for-2.6.33 and the coop limit from Shaohua Li. I did the straight forward merge, but we likely just need to drop that logic since the coop concept is radically different given that we merge and break queues in for-2.6.33.
Quoted 6 lines
Quoted 3 lines
Yeah, since I changed the meaning of the
cfqq_coopflag, a lot of those tests are just plain wrong. Let me play with it and I'll send you an incremental patch in a bit.Thanks, here's what I have. It's basically a revert of the commit in question.
Your patch looks like a straight-forward revert. I still think we need some guards in place, though. For now, I think we can go with what you have, and I'll come up with some other mechanism to deal with this case.
Cheers,
Jeff
Quoted 16 lines · Jeff Moyer
Quoted 14 lines · Jens Axboe
Quoted 12 lines · Jeff Moyer
Quoted 6 lines · Jens Axboe
Since for-linus contains conflicting changes, can you and Jeff please double check that everything is still in order? The interesting bit here is the merge with for-2.6.33 and the coop limit from Shaohua Li. I did the straight forward merge, but we likely just need to drop that logic since the coop concept is radically different given that we merge and break queues in for-2.6.33.
Yeah, since I changed the meaning of the
cfqq_coopflag, a lot of those tests are just plain wrong. Let me play with it and I'll send you an incremental patch in a bit.
Quoted 6 lines
Quoted 2 lines
Thanks, here's what I have. It's basically a revert of the commit in question.
Your patch looks like a straight-forward revert. I still think we need some guards in place, though. For now, I think we can go with what you have, and I'll come up with some other mechanism to deal with this case.
Thanks Jeff, I'll merge it up and we can get things straightened out in due time.
Signature · 1 line
Jens Axboe