IO scheduler based IO controller V9
Subsystems (20)
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
In reply to a message not in the archiveQuoted 68 lines · Ryo Tsuruta
Hi Vivek,
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
Quoted 7 lines
Quoted 3 lines
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.So in above case IO is still charged to G2 but you keep a track if page is
PG_Reclaimthen releae the this bio before other bios queued up in the group?Yes, the bio with
PG_Reclaimpage is given priority over the other bios.Quoted 15 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
Quoted 8 lines
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
If all buffered writes are treated as system wide activities, it does not mean that bandwidth is being controlled. It is true that pdflush doesn't do I/O according to weight, but bandwidth (including for bufferd writes) should be reserved for each cgroup.
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 6 lines
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 3 lines
No. In the current blkio-cgroup, a process which uses a large amount of memory gets penalty, not a memory requester.
At ioband level you just get to see bio and page. How do you decide wheter this bio is being issued by a process which is a memory hog?
In fact requester of memory could be anybody. It could be memory hog or a different process. So are you saying that you got a mechanism where you can detect that a process is memory hog and charge swap activity to it. IOW, if there are two processes A and B and assume A is the memory hog and then B requests for memory which triggers lot of swap IO, then you can charge all that IO to memory hog A?
Can you please point me to the relevant code in dm-ioband?
IMHO, to keep things simple, all swapping activity should be charged to root group and be considered as kernel activity and user space not be charged for that.
Thanks
Vivek
Quoted 29 lines
As you wrote, using both io-controller and memory controller are required to prevent swap-out caused by memory consumption on another cgroup.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
Quoted 5 lines
Quoted 2 lines
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
I'm sorry, I wrote wrong on the previous mail, IO for writing out page-cache pages is not issued immediately, it is throttled by dm-ioband.
Anyway, there is a case where a higher weight group task is made to sleep, but if we reserve the memory for each cgroup by memory controller in advance, we can avoid the task put to sleep.
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): 20090901.160004.226800357.ryov-jCdQPDEk3idL9jVzuh4AOg@public.gmane.org
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 24 lines · Ryo Tsuruta
Hi,
Quoted 22 lines
Quoted 22 lines
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
In order to avoid this wait, dm-ioband issues IO which has a page with
PG_Reclaimas early as possible.
So in above case IO is still charged to G2 but you keep a track if page is PG_Reclaim then releae the this bio before other bios queued up in the group?
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
But it is hard to achieve fairness for buffered writes becase we don't create complete parallel IO paths and not necessarily higher weight process dispatches more buffered writes to IO scheduler. (Due to page cache buffered write logic).
So in some cases we might see buffered write fairness and in other cases not. For example, run two dd processes in two groups doing buffered writes and it is hard to achieve fairness between these.
That's why the idea that if we can't ensure Buffered write vs Buffered write fairness in all the cases, then does it make sense to attribute buffered writes to pdflush and put pdflush threads into a separate group to limit system wide write out activity.
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
A process requesting memory gets IO penalty? IMHO, swapping is a kernel mechanism and kernel's way of providing extended RAM. If we want to solve the issue of memory hogging by a process then right way to solve is to use memory controller and not by charging the process for IO activity. Instead, proabably a more suitable way is to charge swap activity to root group (where by default all the kernel related activity goes).
Quoted 11 lines
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
But you are doing it only for selected pages and not for all buffered writes?
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
Quoted 7 lines
Quoted 4 lines
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Yes, I think that is true.
But anyway we are not able to gurantee this isolation in all the cases. Again I go back to example of two dd threads doing buffered writes in two groups.
I don't mind keeping it. Just wanted to make sure that we agree and understand that keeping it does not mean that we get buffered write vs buffered write isolation/fairness in all the cases.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Thanks for bringing it forward.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): e98e18940908311651s26de5b70ye6f4a82402956309-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Hi,
Quoted 22 lines
Quoted 22 lines
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.
Quoted 3 lines
Quoted 3 lines
But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
In order to avoid this wait, dm-ioband issues IO which has a page with PG_Reclaim as early as possible.
Quoted 5 lines
Quoted 5 lines
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
I think that buffered writes also should be controlled per cgroup as well as synchronous writes.
Quoted 4 lines
Quoted 4 lines
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
I think that swap writeouts should be charged to the memory hogging process, because the process consumes more resources and it should get a penalty.
Quoted 8 lines
Quoted 8 lines
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
As mentioned above, in dm-ioband, the bio is charged to the page owner and issued immediately.
Quoted 14 lines
Quoted 6 lines
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
Quoted 7 lines
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
Quoted 4 lines
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Yes, I think that is true.
Quoted 12 lines
Quoted 12 lines
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
Quoted 2 lines
Quoted 2 lines
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Thanks for bringing it forward.
Thanks,
Ryo Tsuruta
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 67 lines · Vivek Goyal
Quoted 6 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
o
blkio_cgrouppatches from Ryo to track async bios.o This functionality is used to determine the group of async IO from page instead of context of submitting task.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Hirokazu Takahashi <[email protected]>Signed-off-by Ryo Tsuruta <[email protected]>Signed-off-by Vivek Goyal <[email protected]>This seems to be the most complex part of the code so far, but I see why this code is necessary.
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence
blkio_cgrouppages.But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
Quoted 7 lines
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
What if we wanted to control buffered write activity per group? If a group keeps dirtying pages, we wouldn't want it to dominate the disk IO capacity at the expense of other cgroups (by dominating the writes sent down by pdflush).
Quoted 23 lines
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Ryo, any thoughts on this?
Thanks
VivekQuoted 1 line
Acked-by Rik van Riel <[email protected]>
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 6 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
o
blkio_cgrouppatches from Ryo to track async bios.o This functionality is used to determine the group of async IO from page instead of context of submitting task.
Quoted 7 lines
Quoted 3 lines
Signed-off-by Hirokazu Takahashi <[email protected]>Signed-off-by Ryo Tsuruta <[email protected]>Signed-off-by Vivek Goyal <[email protected]>This seems to be the most complex part of the code so far, but I see why this code is necessary.
Hi Rik,
Thanks for reviewing the patches. I wanted to have better understanding of where all does it help to associate a bio to the group of process who created/owned the page. Hence few thoughts.
When a bio is submitted to IO scheduler, it needs to determine the group bio belongs to and group which should be charged to. There seem to be two methods.
- Attribute the bio to cgroup submitting process belongs to.
- For async requests, track the original owner hence cgroup of the page and charge that group for the bio.
One can think of pros/cons of both the approaches.
- The primary use case of tracking async context seems be that if a process T1 in group G1 mmaps a big file and then another process T2 in group G2, asks for memory and triggers reclaim and generates writes of the file pages mapped by T1, then these writes should not be charged to T2, hence blkio_cgroup pages.
But the flip side of this might be that group G2 is a low weight group and probably too busy also right now, which will delay the write out and possibly T2 will wait longer for memory to be allocated.
- At one point of time Andrew mentioned that buffered writes are generally a big problem and one needs to map these to owner's group. Though I am not very sure what specific problem he was referring to. Can we attribute buffered writes to pdflush threads and move all pdflush threads in a cgroup to limit system wide write out activity?
- Somebody also gave an example where there is a memory hogging process and possibly pushes out some processes to swap. It does not sound fair to charge those proccess for that swap writeout. These processes never requested swap IO.
- If there are multiple buffered writers in the system, then those writers can also be forced to writeout some pages to disk before they are allowed to dirty more pages. As per the page cache design, any writer can pick any inode and start writing out pages. So it can happen a weight group task is writting out pages dirtied by a lower weight group task. If, async bio is mapped to owner's group, it might happen that higher weight group task might be made to sleep on lower weight group task because request descriptors are all consumed up.
It looks like there does not seem to be a clean way which covers all the cases without issues. I am just trying to think, what is a simple way which covers most of the cases. Can we just stick to using submitting task context to determine a bio's group (as cfq does). Which can result in following.
- Less code and reduced complexity.
- Buffered writes will be charged to pdflush and its group. If one wish to limit buffered write activity for pdflush, one can move all the pdflush threads into a group and assign desired weight. Writes submitted in process context will continue to be charged to that process irrespective of the fact who dirtied that page.
- swap activity will be charged to kswapd and its group. If swap writes are coming from process context, it gets charged to process and its group.
- If one is worried about the case of one process being charged for write out of file mapped by another process during reclaim, then we can probably make use of memory controller and mount memory controller and io controller together on same hierarchy. I am told that with memory controller, group's memory will be reclaimed by the process requesting more memory. If that's the case, then IO will automatically be charged to right group if we use submitting task context.
I just wanted to bring this point forward for more discussions to know what is the right thing to do? Use bio tracking or not.
Ryo, any thoughts on this?
Thanks
Vivek
Quoted 1 line
Acked-by Rik van Riel <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-19-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 18/23] io-controller: blkio_cgroup patches from Ryo to track async bios.
Quoted 8 lines · Vivek Goyal
o
blkio_cgrouppatches from Ryo to track async bios.o This functionality is used to determine the group of async IO from page instead of context of submitting task.
Signed-off-by Hirokazu Takahashi <[email protected]>Signed-off-by Ryo Tsuruta <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
This seems to be the most complex part of the code so far, but I see why this code is necessary.
PATCH 19/23 io-controller: map async requests to appropriate cgroup
In reply to Vivek Goyalo So far we were assuming that a bio/rq belongs to the task who is submitting it. It did not hold good in case of async writes. This patch makes use of blkio_cgroup pataches to attribute the aysnc writes to right group instead of task submitting the bio.
o For sync requests, we continue to assume that io belongs to the task submitting it. Only in case of async requests, we make use of io tracking patches to track the owner cgroup.
o So far cfq always caches the async queue pointer. With async requests now not necessarily being tied to submitting task io context, caching the pointer will not help for async queues. This patch introduces a new config option CONFIG_TRACK_ASYNC_CONTEXT. If this option is not set, cfq retains old behavior where async queue pointer is cached in task context. If it is set, async queue pointer is not cached and we take help of bio tracking patches to determine group bio belongs to and then map it to async queue of that group.
--- block/Kconfig.iosched | 16 +++++ block/as-iosched.c | 2 +- block/blk-core.c | 7 +- block/cfq-iosched.c | 152 ++++++++++++++++++++++++++++++++++++---------- block/deadline-iosched.c | 2 +- block/elevator-fq.c | 91 +++++++++++++++++++++++---- block/elevator-fq.h | 31 ++++++--- block/elevator.c | 15 +++-- include/linux/elevator.h | 22 ++++++- 9 files changed, 266 insertions(+), 72 deletions(-)
block/Kconfig.iosched +16 -0
@@ -132,6 +132,22 @@ config DEBUG_GROUP_IOSCHED Enable some debugging hooks for hierarchical scheduling support. Currently it just outputs more information in blktrace output. +config TRACK_ASYNC_CONTEXT + bool "Determine async request context from bio" + depends on GROUP_IOSCHED + select CGROUP_BLKIO + default n + ---help--- + Normally async request is attributed to the task submitting the + request. With group ioscheduling, for accurate accounting of + async writes, one needs to map the request to original task/cgroup + which originated the request and not the submitter of the request. + + Currently there are generic io tracking patches to provide facility + to map bio to original owner. If this option is set, for async + request, original owner of the bio is decided by using io tracking + patches otherwise we continue to attribute the request to the + submitting thread. endmenu endif
block/as-iosched.c +1 -1
@@ -1515,7 +1515,7 @@ as_merge(struct request_queue *q, struct request **req, struct bio *bio) {sector_t rb_key = bio->bi_sector +bio_sectors (bio);struct request *__rq; -struct as_queue *asq =elv_get_sched_queue_current (q); +struct as_queue *asq =elv_get_sched_queue_bio (q, bio);if (!asq)return ELEVATOR_NO_MERGE ;
block/blk-core.c +4 -3
@@ -619,7 +619,8 @@ static inline void blk_free_request(struct request_queue *q, struct request *rq) }static struct request * -blk_alloc_request (struct request_queue *q,int flags,int priv,gfp_t gfp_mask) +blk_alloc_request (struct request_queue *q,struct bio *bio,int flags,int priv, +gfp_t gfp_mask) {struct request *rq =mempool_alloc (q->rq.rq_pool, gfp_mask);
@@ -631,7 +632,7 @@ blk_alloc_request(struct request_queue *q, int flags, int priv, gfp_t gfp_mask) rq->cmd_flags = flags |REQ_ALLOCED ;if (priv) { -if (unlikely (elv_set_request (q, rq, gfp_mask))) { +if (unlikely (elv_set_request (q, rq, bio, gfp_mask))) {mempool_free (rq, q->rq.rq_pool);return NULL ; }
@@ -772,7 +773,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags, rw_flags |=REQ_IO_STAT ;spin_unlock_irq (q->queue_lock); - rq =blk_alloc_request (q, rw_flags, priv, gfp_mask); + rq =blk_alloc_request (q, bio, rw_flags, priv, gfp_mask);if (unlikely (!rq)) {/* * Allocation failed presumably due to memory. Undo anything
block/cfq-iosched.c +119 -33
252 lines not on this page · raw message
block/deadline-iosched.c +1 -1
@@ -134,7 +134,7 @@ deadline_merge(struct request_queue *q, struct request **req, struct bio *bio)int ret;struct deadline_queue *dq; - dq =elv_get_sched_queue_current (q); + dq =elv_get_sched_queue_bio (q, bio);if (!dq)return ELEVATOR_NO_MERGE ;
block/elevator-fq.c +77 -14
171 lines not on this page · raw message
block/elevator-fq.h +21 -10
@@ -414,7 +414,9 @@ static inline struct io_queue *elv_get_oom_ioq(struct elevator_queue *eq)extern int elv_io_group_allow_merge (struct request *rq,struct bio *bio);extern void elv_put_iog (struct io_group *iog);extern struct io_group *elv_io_get_io_group (struct request_queue *q, -int create); +struct page *page,int create); +extern struct io_group *elv_io_get_io_group_bio (struct request_queue *q, +struct bio *bio,int create);extern ssize_t elv_group_idle_show (struct elevator_queue *q,char *name);extern ssize_t elv_group_idle_store (struct elevator_queue *q,const char *name,size_t count);
@@ -424,9 +426,10 @@ static inline void elv_get_iog(struct io_group *iog) }extern int elv_set_request_ioq (struct request_queue *q,struct request *rq, -gfp_t gfp_mask); +struct bio *bio,gfp_t gfp_mask);extern void elv_reset_request_ioq (struct request_queue *q,struct request *rq); -extern struct io_queue *elv_lookup_ioq_current (struct request_queue *q); +extern struct io_queue *elv_lookup_ioq_bio (struct request_queue *q, +struct bio *bio);#else /* !GROUP_IOSCHED */
@@ -439,14 +442,20 @@ static inline void elv_get_iog(struct io_group *iog) {}static inline void elv_put_iog (struct io_group *iog) {}static inline struct io_group * -elv_io_get_io_group (struct request_queue *q,int create) +elv_io_get_io_group (struct request_queue *q,struct page *page,int create) {/* In flat mode, there is only root group */ return q->elevator->efqd->root_group; } -static inline int -elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +static inline struct io_group * +elv_io_get_io_group_bio (struct request_queue *q,struct bio *bio,int create) +{ +return q->elevator->efqd->root_group; +} + +static inline int elv_set_request_ioq (struct request_queue *q, +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
@@ -454,7 +463,8 @@ elv_set_request_ioq(struct request_queue *q, struct request *rq, gfp_t gfp_mask)static inline void elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } -static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +static inline struct io_queue * +elv_lookup_ioq_bio (struct request_queue *q,struct bio *bio) {return NULL ; }
@@ -553,8 +563,8 @@ static inline int elv_io_group_allow_merge(struct request *rq, struct bio *bio) {return 1 ; } -static inline int -elv_set_request_ioq (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +static inline int elv_set_request_ioq (struct request_queue *q, +struct request *rq,struct bio *bio,gfp_t gfp_mask) {return 0 ; }
@@ -562,7 +572,8 @@ elv_set_request_ioq(struct request_queue *q, struct request *rq, gfp_t gfp_mask)static inline void elv_reset_request_ioq (struct request_queue *q,struct request *rq) { } -static inline struct io_queue *elv_lookup_ioq_current (struct request_queue *q) +static inline struct io_queue *elv_lookup_ioq_bio (struct request_queue *q, +struct bio *bio) {return NULL ; }
block/elevator.c +8 -7
@@ -865,7 +865,8 @@ struct request *elv_former_request(struct request_queue *q, struct request *rq)return NULL ; } -int elv_set_request (struct request_queue *q,struct request *rq,gfp_t gfp_mask) +int elv_set_request (struct request_queue *q,struct request *rq, +struct bio *bio,gfp_t gfp_mask) {struct elevator_queue *e = q->elevator;
@@ -874,10 +875,10 @@ int elv_set_request(struct request_queue *q, struct request *rq, gfp_t gfp_mask)* ioq per io group */ if (elv_iosched_single_ioq (e)) -return elv_set_request_ioq (q, rq, gfp_mask); +return elv_set_request_ioq (q, rq, bio, gfp_mask);if (e->ops->elevator_set_req_fn) -return e->ops->elevator_set_req_fn (q, rq, gfp_mask); +return e->ops->elevator_set_req_fn (q, rq, bio, gfp_mask); rq->elevator_private =NULL ;return 0 ;
@@ -1279,19 +1280,19 @@ void *elv_select_sched_queue(struct request_queue *q, int force)EXPORT_SYMBOL (elv_select_sched_queue);/* -* Get the io scheduler queue pointer for current task. +* Get the io scheduler queue pointer for the group bio belongs to. * * If fair queuing is enabled, determine the io group of task and retrieve * the ioq pointer from that. This is used by only single queue ioschedulers * for retrieving the queue associated with the group to decide whether the * new bio can do a front merge or not. */ -void *elv_get_sched_queue_current (struct request_queue *q) +void *elv_get_sched_queue_bio (struct request_queue *q,struct bio *bio) {/* Fair queuing is not enabled. There is only one queue. */ if (!elv_iosched_fair_queuing_enabled (q->elevator))return q->elevator->sched_queue; -return elv_ioq_sched_queue (elv_lookup_ioq_current (q)); +return elv_ioq_sched_queue (elv_lookup_ioq_bio (q, bio)); } -EXPORT_SYMBOL (elv_get_sched_queue_current); +EXPORT_SYMBOL (elv_get_sched_queue_bio);
include/linux/elevator.h +19 -3
@@ -22,7 +22,8 @@ typedef struct request *(elevator_request_list_fn) (struct request_queue *, strutypedef void (elevator_completed_req_fn ) (struct request_queue *,struct request *);typedef int (elevator_may_queue_fn ) (struct request_queue *,int ); -typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *,gfp_t ); +typedef int (elevator_set_req_fn ) (struct request_queue *,struct request *, +struct bio *bio,gfp_t );typedef void (elevator_put_req_fn ) (struct request *);typedef void (elevator_activate_req_fn ) (struct request_queue *,struct request *);typedef void (elevator_deactivate_req_fn ) (struct request_queue *,struct request *);
@@ -146,7 +147,8 @@ extern void elv_unregister_queue(struct request_queue *q);extern int elv_may_queue (struct request_queue *,int );extern void elv_abort_queue (struct request_queue *);extern void elv_completed_request (struct request_queue *,struct request *); -extern int elv_set_request (struct request_queue *,struct request *,gfp_t ); +extern int elv_set_request (struct request_queue *,struct request *, +struct bio *bio,gfp_t );extern void elv_put_request (struct request_queue *,struct request *);extern void elv_drain_elevator (struct request_queue *);
@@ -275,6 +277,20 @@ static inline int elv_iosched_single_ioq(struct elevator_queue *e)#endif /* ELV_IOSCHED_FAIR_QUEUING */ extern void *elv_get_sched_queue (struct request_queue *q,struct request *rq);extern void *elv_select_sched_queue (struct request_queue *q,int force); -extern void *elv_get_sched_queue_current (struct request_queue *q); +extern void *elv_get_sched_queue_bio (struct request_queue *q,struct bio *bio); + +/* +* This is equivalent of rq_is_sync()/cfq_bio_sync() function where we +* determine whether an rq/bio is sync or not. There are cases like during +* merging and during * request allocation, where we don't have rq but bio +* and needs to find out * if this bio will be considered as sync or async by +* elevator/iosched. This function is useful in such cases. +*/ +static inline int elv_bio_sync (struct bio *bio) +{ +if ((bio_data_dir (bio) ==READ ) ||bio_sync (bio)) +return 1 ; +return 0 ; +}#endif /* CONFIG_BLOCK */ #endif
Signature · 1 line
1.6.0.6
Re: [PATCH 19/23] io-controller: map async requests to appropriate cgroup
Quoted 18 lines · Vivek Goyal
o So far we were assuming that a bio/rq belongs to the task who is submitting it. It did not hold good in case of async writes. This patch makes use of
blkio_cgrouppataches to attribute the aysnc writes to right group instead of task submitting the bio.o For sync requests, we continue to assume that io belongs to the task submitting it. Only in case of async requests, we make use of io tracking patches to track the owner cgroup.
o So far cfq always caches the async queue pointer. With async requests now not necessarily being tied to submitting task io context, caching the pointer will not help for async queues. This patch introduces a new config option
CONFIG_TRACK_ASYNC_CONTEXT. If this option is not set, cfq retains old behavior where async queue pointer is cached in task context. If it is set, async queue pointer is not cached and we take help of bio tracking patches to determine group bio belongs to and then map it to async queue of that group.
Quoted 3 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-20-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 19/23] io-controller: map async requests to appropriate cgroup
Quoted 18 lines · Vivek Goyal
o So far we were assuming that a bio/rq belongs to the task who is submitting it. It did not hold good in case of async writes. This patch makes use of
blkio_cgrouppataches to attribute the aysnc writes to right group instead of task submitting the bio.o For sync requests, we continue to assume that io belongs to the task submitting it. Only in case of async requests, we make use of io tracking patches to track the owner cgroup.
o So far cfq always caches the async queue pointer. With async requests now not necessarily being tied to submitting task io context, caching the pointer will not help for async queues. This patch introduces a new config option
CONFIG_TRACK_ASYNC_CONTEXT. If this option is not set, cfq retains old behavior where async queue pointer is cached in task context. If it is set, async queue pointer is not cached and we take help of bio tracking patches to determine group bio belongs to and then map it to async queue of that group.
Quoted 3 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Gui Jianfeng <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
PATCH 20/23 io-controller: Per cgroup request descriptor support
In reply to Vivek Goyalo Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively request_list becomes per io group property and not a global request queue feature.
o Currently one can define q->nr_requests to limit request descriptors allocated for the queue. Now there is another tunable q->nr_group_requests which controls the requests descriptr limit per group. q->nr_requests supercedes q->nr_group_requests to make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.
--- block/blk-core.c | 317 +++++++++++++++++++++++++++++++++--------- block/blk-settings.c | 1 + block/blk-sysfs.c | 59 ++++++-- block/elevator-fq.c | 36 +++++ block/elevator-fq.h | 29 ++++ block/elevator.c | 7 +- include/linux/blkdev.h | 47 ++++++- include/trace/events/block.h | 6 +- kernel/trace/blktrace.c | 6 +- 9 files changed, 421 insertions(+), 87 deletions(-)
block/blk-core.c +250 -65
@@ -460,20 +460,53 @@ void blk_cleanup_queue(struct request_queue *q) }EXPORT_SYMBOL (blk_cleanup_queue); -static int blk_init_free_list (struct request_queue *q) +struct request_list * +blk_get_request_list (struct request_queue *q,struct bio *bio) +{ +#ifdef CONFIG_GROUP_IOSCHED +/* +* Determine which request list bio will be allocated from. This +* is dependent on which io group bio belongs to +*/ +return elv_get_request_list_bio (q, bio); +#else +return &q->rq; +#endif +} + +static struct request_list *rq_rl (struct request_queue *q,struct request *rq) +{ +#ifdef CONFIG_GROUP_IOSCHED +int priv = rq->cmd_flags &REQ_ELVPRIV ; + +return elv_get_request_list_rq (q, rq, priv); +#else +return &q->rq; +#endif +} + +void blk_init_request_list (struct request_list *rl) { -struct request_list *rl = &q->rq; rl->count[BLK_RW_SYNC ] = rl->count[BLK_RW_ASYNC ] =0 ; - rl->starved[BLK_RW_SYNC ] = rl->starved[BLK_RW_ASYNC ] =0 ; - rl->elvpriv =0 ;init_waitqueue_head (&rl->wait[BLK_RW_SYNC ]);init_waitqueue_head (&rl->wait[BLK_RW_ASYNC ]); +} - rl->rq_pool =mempool_create_node (BLKDEV_MIN_RQ , mempool_alloc_slab, - mempool_free_slab, request_cachep, q->node); +static int blk_init_free_list (struct request_queue *q) +{ +/* +* In case of group scheduling, request list is inside group and is +* initialized when group is instanciated. +*/ +#ifndef CONFIG_GROUP_IOSCHED +blk_init_request_list (&q->rq); +#endif + q->rq_data.rq_pool =mempool_create_node (BLKDEV_MIN_RQ , + mempool_alloc_slab, mempool_free_slab, + request_cachep, q->node); -if (!rl->rq_pool) +if (!q->rq_data.rq_pool)return -ENOMEM ;return 0 ;
@@ -581,6 +614,9 @@ blk_init_queue_node(request_fn_proc *rfn, spinlock_t *lock, int node_id) q->queue_flags =QUEUE_FLAG_DEFAULT ; q->queue_lock = lock; +/* init starved waiter wait queue */ +init_waitqueue_head (&q->rq_data.starved_wait); +/* * This also sets hw/phys segments, boundary and size */
@@ -615,14 +651,14 @@ static inline void blk_free_request(struct request_queue *q, struct request *rq) {if (rq->cmd_flags &REQ_ELVPRIV )elv_put_request (q, rq); -mempool_free (rq, q->rq.rq_pool); +mempool_free (rq, q->rq_data.rq_pool); }static struct request *blk_alloc_request (struct request_queue *q,struct bio *bio,int flags,int priv,gfp_t gfp_mask) { -struct request *rq =mempool_alloc (q->rq.rq_pool, gfp_mask); +struct request *rq =mempool_alloc (q->rq_data.rq_pool, gfp_mask);if (!rq)return NULL ;
@@ -633,7 +669,7 @@ blk_alloc_request(struct request_queue *q, struct bio *bio, int flags, int priv,if (priv) {if (unlikely (elv_set_request (q, rq, bio, gfp_mask))) { -mempool_free (rq, q->rq.rq_pool); +mempool_free (rq, q->rq_data.rq_pool);return NULL ; } rq->cmd_flags |=REQ_ELVPRIV ;
@@ -676,18 +712,18 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc) ioc->last_waited = jiffies; } -static void __freed_request (struct request_queue *q,int sync) +static void __freed_request (struct request_queue *q,int sync, +struct request_list *rl) { -struct request_list *rl = &q->rq; - -if (rl->count[sync] <queue_congestion_off_threshold (q)) +if (q->rq_data.count[sync] <queue_congestion_off_threshold (q))blk_clear_queue_congested (q, sync); -if (rl->count[sync] +1 <= q->nr_requests) { +if (q->rq_data.count[sync] +1 <= q->nr_requests) +blk_clear_queue_full (q, sync); + +if (rl->count[sync] +1 <= q->nr_group_requests) {if (waitqueue_active (&rl->wait[sync]))wake_up (&rl->wait[sync]); - -blk_clear_queue_full (q, sync); } }
@@ -695,63 +731,130 @@ static void __freed_request(struct request_queue *q, int sync)* A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request (struct request_queue *q,int sync,int priv) +static void freed_request (struct request_queue *q,int sync,int priv, +struct request_list *rl) { -struct request_list *rl = &q->rq; +/* +* There is a window during request allocation where request is +* mapped to one group but by the time a queue for the group is +* allocated, it is possible that original cgroup/io group has been +* deleted and now io queue is allocated in a different group (root) +* altogether. +* +* One solution to the problem is that rq should take io group +* reference. But it looks too much to do that to solve this issue. +* The only side affect to the hard to hit issue seems to be that +* we will try to decrement the rl->count for a request list which +* did not allocate that request. Chcek for rl->count going less than +* zero and do not decrement it if that's the case. +*/ + +if (priv && rl->count[sync] >0 ) + rl->count[sync]--; + +BUG_ON (!q->rq_data.count[sync]); + q->rq_data.count[sync]--; - rl->count[sync]--;if (priv) - rl->elvpriv--; + q->rq_data.elvpriv--; -__freed_request (q, sync); +__freed_request (q, sync, rl);if (unlikely (rl->starved[sync ^1 ])) -__freed_request (q, sync ^1 ); +__freed_request (q, sync ^1 , rl); + +/* Wake up the starved process on global list, if any */ +if (unlikely (q->rq_data.starved)) { +if (waitqueue_active (&q->rq_data.starved_wait)) +wake_up (&q->rq_data.starved_wait); + q->rq_data.starved--; + } +} + +/* +* Returns whether one can sleep on this request list or not. There are +* cases (elevator switch) where request list might not have allocated +* any request descriptor but we deny request allocation due to gloabl +* limits. In that case one should sleep on global list as on this request +* list no wakeup will take place. +* +* Also sets the request list starved flag if there are no requests pending +* in the direction of rq. +* +* Return 1 --> sleep on request list, 0 --> sleep on global list +*/ +static int can_sleep_on_request_list (struct request_list *rl,int is_sync) +{ +if (unlikely (rl->count[is_sync] ==0 )) { +/* +* If there is a request pending in other direction +* in same io group, then set the starved flag of +* the group request list. Otherwise, we need to +* make this process sleep in global starved list +* to make sure it will not sleep indefinitely. +*/ +if (rl->count[is_sync ^1 ] !=0 ) { + rl->starved[is_sync] =1 ; +return 1 ; + }else +return 0 ; + } + +return 1 ; }/* * Get a free request, queue_lock must be held. -* Returns NULL on failure, with queue_lock held. +* Returns NULL on failure, with queue_lock held. Also sets the "reason" field +* in case of failure. This reason field helps caller decide to whether sleep +* on per group list or global per queue list. +* reason = 0 sleep on per group list +* reason = 1 sleep on global list +* * Returns !NULL on success, with queue_lock *not held*. */ static struct request *get_request (struct request_queue *q,int rw_flags, -struct bio *bio,gfp_t gfp_mask) +struct bio *bio,gfp_t gfp_mask, +struct request_list *rl,int *reason) {struct request *rq =NULL ; -struct request_list *rl = &q->rq;struct io_context *ioc =NULL ;const bool is_sync =rw_is_sync (rw_flags) !=0 ;int may_queue, priv; +int sleep_on_global =0 ; may_queue =elv_may_queue (q, rw_flags);if (may_queue ==ELV_MQUEUE_NO ) goto rq_starved; -if (rl->count[is_sync]+1 >=queue_congestion_on_threshold (q)) { -if (rl->count[is_sync]+1 >= q->nr_requests) { - ioc =current_io_context (GFP_ATOMIC , q->node); -/* -* The queue will fill after this allocation, so set -* it as full, and mark this process as "batching". -* This process will be allowed to complete a batch of -* requests, others will be blocked. -*/ -if (!blk_queue_full (q, is_sync)) { -ioc_set_batching (q, ioc); -blk_set_queue_full (q, is_sync); - }else { -if (may_queue !=ELV_MQUEUE_MUST - && !ioc_batching (q, ioc)) { -/* -* The queue is full and the allocating -* process is not a "batcher", and not -* exempted by the IO scheduler -*/ - goto out; - } +if (q->rq_data.count[is_sync]+1 >=queue_congestion_on_threshold (q)) +blk_set_queue_congested (q, is_sync); + +/* queue full seems redundant now */ +if (q->rq_data.count[is_sync]+1 >= q->nr_requests) +blk_set_queue_full (q, is_sync); + +if (rl->count[is_sync]+1 >= q->nr_group_requests) { + ioc =current_io_context (GFP_ATOMIC , q->node); +/* +* The queue request descriptor group will fill after this +* allocation, so set it as full, and mark this process as +* "batching". This process will be allowed to complete a +* batch of requests, others will be blocked. +*/ +if (rl->count[is_sync] <= q->nr_group_requests) +ioc_set_batching (q, ioc); +else { +if (may_queue !=ELV_MQUEUE_MUST + && !ioc_batching (q, ioc)) { +/* +* The queue is full and the allocating +* process is not a "batcher", and not +* exempted by the IO scheduler +*/ + goto out; } } -blk_set_queue_congested (q, is_sync); }/*
@@ -759,21 +862,60 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* limit of requests, otherwise we could have thousands of requests * allocated with any setting of ->nr_requests */ -if (rl->count[is_sync] >= (3 * q->nr_requests /2 )) + +if (q->rq_data.count[is_sync] >= (3 * q->nr_requests /2 )) { +/* +* Queue is too full for allocation. On which request queue +* the task should sleep? Generally it should sleep on its +* request list but if elevator switch is happening, in that +* window, request descriptors are allocated from global +* pool and are not accounted against any particular request +* list as group is going away. +* +* So it might happen that request list does not have any +* requests allocated at all and if process sleeps on per +* group request list, it will not be woken up. In such case, +* make it sleep on global starved list. +*/ +if (test_bit (QUEUE_FLAG_ELVSWITCH , &q->queue_flags) + || !can_sleep_on_request_list (rl, is_sync)) + sleep_on_global =1 ; + goto out; + } + +/* +* Allocation of request is allowed from queue perspective. Now check +* from per group request list +*/ + +if (rl->count[is_sync] >= (3 * q->nr_group_requests /2 )) goto out; - rl->count[is_sync]++; rl->starved[is_sync] =0 ; + q->rq_data.count[is_sync]++; + priv = !test_bit (QUEUE_FLAG_ELVSWITCH , &q->queue_flags); -if (priv) - rl->elvpriv++; +if (priv) { + q->rq_data.elvpriv++; +/* +* Account the request to request list only if request is +* going to elevator. During elevator switch, there will +* be small window where group is going away and new group +* will not be allocated till elevator switch is complete. +* So till then instead of slowing down the application, +* we will continue to allocate request from total common +* pool instead of per group limit +*/ + rl->count[is_sync]++; + }if (blk_queue_io_stat (q)) rw_flags |=REQ_IO_STAT ;spin_unlock_irq (q->queue_lock); rq =blk_alloc_request (q, bio, rw_flags, priv, gfp_mask); +if (unlikely (!rq)) {/* * Allocation failed presumably due to memory. Undo anything
@@ -783,7 +925,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* wait queue, but this is pretty rare. */ spin_lock_irq (q->queue_lock); -freed_request (q, is_sync, priv); +freed_request (q, is_sync, priv, rl);/* * in the very unlikely event that allocation failed and no
@@ -793,9 +935,8 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* rq mempool into READ and WRITE */ rq_starved: -if (unlikely (rl->count[is_sync] ==0 )) - rl->starved[is_sync] =1 ; - +if (!can_sleep_on_request_list (rl, is_sync)) + sleep_on_global =1 ; goto out; }
@@ -810,6 +951,8 @@ rq_starved:trace_block_getrq (q, bio, rw_flags &1 ); out: +if (reason && sleep_on_global) + *reason =1 ;return rq; }
@@ -823,16 +966,39 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, struct bio *bio) {const bool is_sync =rw_is_sync (rw_flags) !=0 ; +int sleep_on_global =0 ;struct request *rq; +struct request_list *rl =blk_get_request_list (q, bio); - rq =get_request (q, rw_flags, bio,GFP_NOIO ); + rq =get_request (q, rw_flags, bio,GFP_NOIO , rl, &sleep_on_global);while (!rq) {DEFINE_WAIT (wait);struct io_context *ioc; -struct request_list *rl = &q->rq; -prepare_to_wait_exclusive (&rl->wait[is_sync], &wait, -TASK_UNINTERRUPTIBLE ); +if (sleep_on_global) { +/* +* Task failed allocation and needs to wait and +* try again. There are no requests pending from +* the io group hence need to sleep on global +* wait queue. Most likely the allocation failed +* because of memory issues. +*/ + + q->rq_data.starved++; +prepare_to_wait_exclusive (&q->rq_data.starved_wait, + &wait,TASK_UNINTERRUPTIBLE ); + }else { +/* +* We are about to sleep on a request list and we +* drop queue lock. After waking up, we will do +* finish_wait() on request list and in the mean +* time group might be gone. Take a reference to +* the group now. +*/ +prepare_to_wait_exclusive (&rl->wait[is_sync], &wait, +TASK_UNINTERRUPTIBLE ); +elv_get_rl_iog (rl); + }trace_block_sleeprq (q, bio, rw_flags &1 );
@@ -850,9 +1016,25 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags,ioc_set_batching (q, ioc);spin_lock_irq (q->queue_lock); -finish_wait (&rl->wait[is_sync], &wait); - rq =get_request (q, rw_flags, bio,GFP_NOIO ); +if (sleep_on_global) { +finish_wait (&q->rq_data.starved_wait, &wait); + sleep_on_global =0 ; + }else { +/* +* We had taken a reference to the rl/iog. Put that now +*/ +finish_wait (&rl->wait[is_sync], &wait); +elv_put_rl_iog (rl); + } + +/* +* After the sleep check the rl again in case cgrop bio +* belonged to is gone and it is mapped to root group now +*/ + rl =blk_get_request_list (q, bio); + rq =get_request (q, rw_flags, bio,GFP_NOIO , rl, + &sleep_on_global); };return rq;
@@ -861,14 +1043,16 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags,struct request *blk_get_request (struct request_queue *q,int rw,gfp_t gfp_mask) {struct request *rq; +struct request_list *rl;BUG_ON (rw !=READ && rw !=WRITE );spin_lock_irq (q->queue_lock); + rl =blk_get_request_list (q,NULL );if (gfp_mask & __GFP_WAIT) { rq =get_request_wait (q, rw,NULL ); }else { - rq =get_request (q, rw,NULL , gfp_mask); + rq =get_request (q, rw,NULL , gfp_mask, rl,NULL );if (!rq)spin_unlock_irq (q->queue_lock); }
@@ -1085,12 +1269,13 @@ void __blk_put_request(struct request_queue *q, struct request *req)if (req->cmd_flags &REQ_ALLOCED ) {int is_sync =rq_is_sync (req) !=0 ;int priv = req->cmd_flags &REQ_ELVPRIV ; +struct request_list *rl =rq_rl (q, req);BUG_ON (!list_empty (&req->queuelist));BUG_ON (!hlist_unhashed (&req->hash));blk_free_request (q, req); -freed_request (q, is_sync, priv); +freed_request (q, is_sync, priv, rl); } }EXPORT_SYMBOL_GPL (__blk_put_request);
block/blk-settings.c +1 -0
@@ -149,6 +149,7 @@ void blk_queue_make_request(struct request_queue *q, make_request_fn *mfn)* set defaults */ q->nr_requests =BLKDEV_MAX_RQ ; + q->nr_group_requests =BLKDEV_MAX_GROUP_RQ ; q->make_request_fn = mfn;blk_queue_dma_alignment (q,511 );
block/blk-sysfs.c +47 -12
113 lines not on this page · raw message
block/elevator-fq.c +36 -0
@@ -925,6 +925,39 @@ static struct io_cgroup *cgroup_to_io_cgroup(struct cgroup *cgroup) struct io_cgroup, css); } +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio) +{ +struct io_group *iog; + +if (!elv_iosched_fair_queuing_enabled (q->elevator)) + iog = q->elevator->efqd->root_group; +else + iog =elv_io_get_io_group_bio (q, bio,1 ); + +BUG_ON (!iog); +return &iog->rl; +} + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv) +{ +struct io_group *iog; + +if (!elv_iosched_fair_queuing_enabled (q->elevator)) +return &q->elevator->efqd->root_group->rl; + +BUG_ON (priv && !rq->ioq); + +if (priv) + iog =ioq_to_io_group (rq->ioq); +else + iog = q->elevator->efqd->root_group; + +BUG_ON (!iog); +return &iog->rl; +} +/* * Search the io_group for efqd into the hash table (by now only a list) * of bgrp. Must be called under rcu_read_lock().
@@ -1281,6 +1314,8 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup)elv_get_iog (iog);io_group_path (iog); +blk_init_request_list (&iog->rl); +if (leaf ==NULL ) { leaf = iog; prev = leaf;
@@ -1502,6 +1537,7 @@ static struct io_group *io_alloc_root_group(struct request_queue *q,for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) iog->sched_data.service_tree[i] =ELV_SERVICE_TREE_INIT ; +blk_init_request_list (&iog->rl);spin_lock_irq (&iocg->lock);rcu_assign_pointer (iog->key, key);hlist_add_head_rcu (&iog->group_node, &iocg->group_data);
block/elevator-fq.h +29 -0
@@ -128,6 +128,9 @@ struct io_group {/* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + +/* request list associated with the group */ +struct request_list rl; };struct io_cgroup {
@@ -425,11 +428,31 @@ static inline void elv_get_iog(struct io_group *iog)atomic_inc (&iog->ref); } +static inline struct io_group *rl_iog (struct request_list *rl) +{ + return container_of(rl, struct io_group, rl); +} + +static inline void elv_get_rl_iog (struct request_list *rl) +{ +elv_get_iog (rl_iog (rl)); +} + +static inline void elv_put_rl_iog (struct request_list *rl) +{ +elv_put_iog (rl_iog (rl)); +} +extern int elv_set_request_ioq (struct request_queue *q,struct request *rq,struct bio *bio,gfp_t gfp_mask);extern void elv_reset_request_ioq (struct request_queue *q,struct request *rq);extern struct io_queue *elv_lookup_ioq_bio (struct request_queue *q,struct bio *bio); +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio); + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv);#else /* !GROUP_IOSCHED */
@@ -469,6 +492,9 @@ elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio) return NULL; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } +#endif /* GROUP_IOSCHED */ extern ssize_t elv_slice_sync_show (struct elevator_queue *q,char *name);
@@ -578,6 +604,9 @@ static inline struct io_queue *elv_lookup_ioq_bio(struct request_queue *q,return NULL ; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } +#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */
block/elevator.c +4 -3
@@ -678,7 +678,7 @@ void elv_quiesce_start(struct request_queue *q)* make sure we don't have any requests in flight */ elv_drain_elevator (q); -while (q->rq.elvpriv) { +while (q->rq_data.elvpriv) {__blk_run_queue (q);spin_unlock_irq (q->queue_lock);msleep (10 );
@@ -777,8 +777,9 @@ void elv_insert(struct request_queue *q, struct request *rq, int where) }if (unplug_it &&blk_queue_plugged (q)) { -int nrq = q->rq.count[BLK_RW_SYNC ] + q->rq.count[BLK_RW_ASYNC ] - -queue_in_flight (q); +int nrq = q->rq_data.count[BLK_RW_SYNC ] + + q->rq_data.count[BLK_RW_ASYNC ] - +queue_in_flight (q);if (nrq >= q->unplug_thresh)__generic_unplug_device (q);
include/linux/blkdev.h +45 -2
88 lines not on this page · raw message
include/trace/events/block.h +4 -2
@@ -397,7 +397,8 @@ TRACE_EVENT(block_unplug_timer, ),TP_fast_assign ( - __entry->nr_rq = q->rq.count[READ] + q->rq.count[WRITE]; + __entry->nr_rq = q->rq_data.count[READ ] + + q->rq_data.count[WRITE];memcpy (__entry->comm, current->comm,TASK_COMM_LEN ); ),
@@ -416,7 +417,8 @@ TRACE_EVENT(block_unplug_io, ),TP_fast_assign ( - __entry->nr_rq = q->rq.count[READ] + q->rq.count[WRITE]; + __entry->nr_rq = q->rq_data.count[READ ] + + q->rq_data.count[WRITE];memcpy (__entry->comm, current->comm,TASK_COMM_LEN ); ),
kernel/trace/blktrace.c +4 -2
@@ -786,7 +786,8 @@ static void blk_add_trace_unplug_io(struct request_queue *q)struct blk_trace *bt = q->blk_trace;if (bt) { -unsigned int pdu = q->rq.count[READ ] + q->rq.count[WRITE ]; +unsigned int pdu = q->rq_data.count[READ ] + + q->rq_data.count[WRITE ];__be64 rpdu =cpu_to_be64 (pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_IO, 0,
@@ -799,7 +800,8 @@ static void blk_add_trace_unplug_timer(struct request_queue *q)struct blk_trace *bt = q->blk_trace;if (bt) { -unsigned int pdu = q->rq.count[READ ] + q->rq.count[WRITE ]; +unsigned int pdu = q->rq_data.count[READ ] + + q->rq_data.count[WRITE ];__be64 rpdu =cpu_to_be64 (pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_TIMER, 0,
Signature · 1 line
1.6.0.6
Re: [PATCH 20/23] io-controller: Per cgroup request descriptor support
Quoted 23 lines · Vivek Goyal
o Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively
request_listbecomes per io group property and not a global request queue feature.o Currently one can define
q->nr_requeststo limit request descriptors allocated for the queue. Now there is another tunableq->nr_group_requestswhich controls the requests descriptr limit per group.q->nr_requestssupercedesq->nr_group_requeststo make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.
Quoted 2 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-21-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 20/23] io-controller: Per cgroup request descriptor support
Quoted 23 lines · Vivek Goyal
o Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively
request_listbecomes per io group property and not a global request queue feature.o Currently one can define
q->nr_requeststo limit request descriptors allocated for the queue. Now there is another tunableq->nr_group_requestswhich controls the requests descriptr limit per group.q->nr_requestssupercedesq->nr_group_requeststo make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.
Quoted 2 lines
Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Re: [PATCH 20/23] io-controller: Per cgroup request descriptor support
In reply to a message not in the archiveQuoted 514 lines · Vivek Goyal
o Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively
request_listbecomes per io group property and not a global request queue feature.o Currently one can define
q->nr_requeststo limit request descriptors allocated for the queue. Now there is another tunableq->nr_group_requestswhich controls the requests descriptr limit per group.q->nr_requestssupercedesq->nr_group_requeststo make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>---
block/blk-core.c | 317 +++++++++++++++++++++++++++++++++---------block/blk-settings.c| 1 +
block/blk-sysfs.c | 59 ++++++--
block/elevator-fq.c | 36 +++++
block/elevator-fq.h | 29 ++++
block/elevator.c | 7 +-
include/linux/blkdev.h | 47 ++++++-include/trace/events/block.h| 6 +-
kernel/trace/blktrace.c | 6 +-9 files changed, 421 insertions(+), 87 deletions(-)block/blk-core.c +44
-8@@ -460,20 +460,53 @@ void blk_cleanup_queue(struct request_queue *q) } EXPORT_SYMBOL(blk_cleanup_queue); -static int blk_init_free_list (struct request_queue *q) +struct request_list * +blk_get_request_list (struct request_queue *q,struct bio *bio) +{ +#ifdef CONFIG_GROUP_IOSCHED + /* + * Determine which request list bio will be allocated from. This + * is dependent on which io group bio belongs to + */ + return elv_get_request_list_bio(q, bio); +#else + return &q->rq; +#endif +} + +static struct request_list *rq_rl (struct request_queue *q,struct request *rq) +{ +#ifdef CONFIG_GROUP_IOSCHED + int priv = rq->cmd_flags & REQ_ELVPRIV; + + return elv_get_request_list_rq(q, rq, priv); +#else + return &q->rq; +#endif +} + +void blk_init_request_list (struct request_list *rl) { - struct request_list *rl = &q->rq; rl->count[BLK_RW_SYNC] = rl->count[BLK_RW_ASYNC] = 0; - rl->starved[BLK_RW_SYNC] = rl->starved[BLK_RW_ASYNC] = 0; - rl->elvpriv = 0; init_waitqueue_head(&rl->wait[BLK_RW_SYNC]); init_waitqueue_head(&rl->wait[BLK_RW_ASYNC]); +} - rl->rq_pool = mempool_create_node(BLKDEV_MIN_RQ, mempool_alloc_slab, - mempool_free_slab, request_cachep, q->node); +static int blk_init_free_list (struct request_queue *q) +{ + /* +* In case of group scheduling, request list is inside group and is +* initialized when group is instanciated. +*/ +#ifndef CONFIG_GROUP_IOSCHED + blk_init_request_list(&q->rq); +#endif + q->rq_data.rq_pool = mempool_create_node(BLKDEV_MIN_RQ, + mempool_alloc_slab, mempool_free_slab, + request_cachep, q->node); - if (!rl->rq_pool) + if (!q->rq_data.rq_pool) return -ENOMEM; return 0;@@ -581,6 +614,9 @@ blk_init_queue_node(request_fn_proc *rfn, spinlock_t *lock, int node_id) q->queue_flags = QUEUE_FLAG_DEFAULT; q->queue_lock = lock; + /* init starved waiter wait queue */ + init_waitqueue_head(&q->rq_data.starved_wait); + /** This also sets hw/phys segments, boundary and size */ @@ -615,14 +651,14 @@ static inline void blk_free_request(struct request_queue *q, struct request *rq) { if (rq->cmd_flags & REQ_ELVPRIV) elv_put_request(q, rq); - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); }static struct request *
blk_alloc_request(structrequest_queue*q, struct bio *bio, int flags, int priv,gfp_tgfp_mask)
{
- struct request *rq =mempool_alloc(q->rq.rq_pool, gfp_mask);
+ struct request *rq =mempool_alloc(q->rq_data.rq_pool, gfp_mask);if (!rq)
return NULL;block/blk-core.c +151
-45@@ -633,7 +669,7 @@ blk_alloc_request(struct request_queue *q, struct bio *bio, int flags, int priv, if (priv) { if (unlikely(elv_set_request(q, rq, bio, gfp_mask))) { - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); return NULL; } rq->cmd_flags |= REQ_ELVPRIV;@@ -676,18 +712,18 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc) ioc->last_waited = jiffies; } -static void __freed_request (struct request_queue *q,int sync) +static void __freed_request (struct request_queue *q,int sync, + struct request_list *rl) { - struct request_list *rl = &q->rq; - - if (rl->count[sync] < queue_congestion_off_threshold(q)) + if (q->rq_data.count[sync] < queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, sync); - if (rl->count[sync] + 1 <= q->nr_requests) { + if (q->rq_data.count[sync] + 1 <= q->nr_requests) + blk_clear_queue_full(q, sync); + + if (rl->count[sync] + 1 <= q->nr_group_requests) { if (waitqueue_active(&rl->wait[sync])) wake_up(&rl->wait[sync]); - - blk_clear_queue_full(q, sync); } }@@ -695,63 +731,130 @@ static void __freed_request(struct request_queue *q, int sync)* A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request (struct request_queue *q,int sync,int priv) +static void freed_request (struct request_queue *q,int sync,int priv, + struct request_list *rl) { - struct request_list *rl = &q->rq; + /* + * There is a window during request allocation where request is + * mapped to one group but by the time a queue for the group is + * allocated, it is possible that original cgroup/io group has been + * deleted and now io queue is allocated in a different group (root) + * altogether. + * + * One solution to the problem is that rq should take io group + * reference. But it looks too much to do that to solve this issue. + * The only side affect to the hard to hit issue seems to be that + * we will try to decrement the rl->count for a request list which + * did not allocate that request. Chcek for rl->count going less than + * zero and do not decrement it if that's the case. + */ + + if (priv && rl->count[sync] > 0) + rl->count[sync]--; + + BUG_ON(!q->rq_data.count[sync]); + q->rq_data.count[sync]--; - rl->count[sync]--; if (priv) - rl->elvpriv--; + q->rq_data.elvpriv--; - __freed_request(q, sync); + __freed_request(q, sync, rl); if (unlikely(rl->starved[sync ^ 1])) - __freed_request(q, sync ^ 1); + __freed_request(q, sync ^ 1, rl); + + /* Wake up the starved process on global list, if any */ + if (unlikely(q->rq_data.starved)) { + if (waitqueue_active(&q->rq_data.starved_wait)) + wake_up(&q->rq_data.starved_wait); + q->rq_data.starved--; + } +} + +/* +* Returns whether one can sleep on this request list or not. There are +* cases (elevator switch) where request list might not have allocated +* any request descriptor but we deny request allocation due to gloabl +* limits. In that case one should sleep on global list as on this request +* list no wakeup will take place. +* +* Also sets the request list starved flag if there are no requests pending +* in the direction of rq. +* +* Return 1 --> sleep on request list, 0 --> sleep on global list +*/ +static int can_sleep_on_request_list (struct request_list *rl,int is_sync) +{ + if (unlikely(rl->count[is_sync] == 0)) { + /* + * If there is a request pending in other direction + * in same io group, then set the starved flag of + * the group request list. Otherwise, we need to + * make this process sleep in global starved list + * to make sure it will not sleep indefinitely. + */ + if (rl->count[is_sync ^ 1] != 0) { + rl->starved[is_sync] = 1; + return 1; + } else + return 0; + } + + return 1; } /* * Get a free request, queue_lock must be held. - * Returns NULL on failure, with queue_lock held. + * Returns NULL on failure, with queue_lock held. Also sets the "reason" field + * in case of failure. This reason field helps caller decide to whether sleep + * on per group list or global per queue list. + * reason = 0 sleep on per group list + * reason = 1 sleep on global list + * * Returns !NULL on success, with queue_lock *not held*. */ static struct request *get_request(struct request_queue *q, int rw_flags, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, gfp_t gfp_mask, + struct request_list *rl, int *reason) { struct request *rq = NULL; - struct request_list *rl = &q->rq; struct io_context *ioc = NULL; const bool is_sync = rw_is_sync(rw_flags) != 0; int may_queue, priv; + int sleep_on_global = 0; may_queue = elv_may_queue(q, rw_flags); if (may_queue == ELV_MQUEUE_NO) goto rq_starved; - if (rl->count[is_sync]+1 >= queue_congestion_on_threshold(q)) { - if (rl->count[is_sync]+1 >= q->nr_requests) { - ioc = current_io_context(GFP_ATOMIC, q->node); - /* - * The queue will fill after this allocation, so set - * it as full, and mark this process as "batching". - * This process will be allowed to complete a batch of - * requests, others will be blocked. - */ - if (!blk_queue_full(q, is_sync)) { - ioc_set_batching(q, ioc); - blk_set_queue_full(q, is_sync); - } else { - if (may_queue != ELV_MQUEUE_MUST - && !ioc_batching(q, ioc)) { - /* - * The queue is full and the allocating - * process is not a "batcher", and not - * exempted by the IO scheduler - */ - goto out; - } + if (q->rq_data.count[is_sync]+1 >= queue_congestion_on_threshold(q)) + blk_set_queue_congested(q, is_sync); + + /* queue full seems redundant now */ + if (q->rq_data.count[is_sync]+1 >= q->nr_requests) + blk_set_queue_full(q, is_sync); + + if (rl->count[is_sync]+1 >= q->nr_group_requests) { + ioc = current_io_context(GFP_ATOMIC, q->node); + /* + * The queue request descriptor group will fill after this + * allocation, so set it as full, and mark this process as + * "batching". This process will be allowed to complete a + * batch of requests, others will be blocked. + */ + if (rl->count[is_sync] <= q->nr_group_requests) + ioc_set_batching(q, ioc); + else { + if (may_queue != ELV_MQUEUE_MUST + && !ioc_batching(q, ioc)) { + /* + * The queue is full and the allocating + * process is not a "batcher", and not + * exempted by the IO scheduler + */ + goto out; } } - blk_set_queue_congested(q, is_sync); } /*@@ -759,21 +862,60 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* limit of requests, otherwise we could have thousands of requests * allocated with any setting of ->nr_requests */ - if (rl->count[is_sync] >= (3 * q->nr_requests / 2)) + + if (q->rq_data.count[is_sync] >= (3 * q->nr_requests / 2)) { + /* + * Queue is too full for allocation. On which request queue + * the task should sleep? Generally it should sleep on its + * request list but if elevator switch is happening, in that + * window, request descriptors are allocated from global + * pool and are not accounted against any particular request + * list as group is going away. + * + * So it might happen that request list does not have any + * requests allocated at all and if process sleeps on per + * group request list, it will not be woken up. In such case, + * make it sleep on global starved list. + */ + if (test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags) + || !can_sleep_on_request_list(rl, is_sync)) + sleep_on_global = 1; + goto out; + } + + /* + * Allocation of request is allowed from queue perspective. Now check + * from per group request list + */ + + if (rl->count[is_sync] >= (3 * q->nr_group_requests / 2)) goto out; - rl->count[is_sync]++; rl->starved[is_sync] = 0; + q->rq_data.count[is_sync]++; + priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags); - if (priv) - rl->elvpriv++; + if (priv) { + q->rq_data.elvpriv++; + /* + * Account the request to request list only if request is + * going to elevator. During elevator switch, there will + * be small window where group is going away and new group + * will not be allocated till elevator switch is complete. + * So till then instead of slowing down the application, + * we will continue to allocate request from total common + * pool instead of per group limit + */ + rl->count[is_sync]++; + } if (blk_queue_io_stat(q)) rw_flags |= REQ_IO_STAT; spin_unlock_irq(q->queue_lock); rq = blk_alloc_request(q, bio, rw_flags, priv, gfp_mask); + if (unlikely(!rq)) { /** Allocation failed presumably due to memory. Undo anything @@ -783,7 +925,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* wait queue, but this is pretty rare. */ spin_lock_irq(q->queue_lock); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl); /** in the very unlikely event that allocation failed and no @@ -793,9 +935,8 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* rq mempool into READ and WRITE */ rq_starved:
- if (unlikely(rl->count[is_sync] == 0))
- rl->starved[is_sync] = 1; -
+ if (!can_sleep_on_request_list(rl,is_sync))
+sleep_on_global= 1;
goto out;
}block/blk-core.c +0
-0@@ -810,6 +951,8 @@ rq_starved: trace_block_getrq(q, bio, rw_flags & 1);out:
+ if (reason &&sleep_on_global)
+ *reason = 1;
return rq;
}block/blk-core.c +48
-7@@ -823,16 +966,39 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, struct bio *bio) { const bool is_sync = rw_is_sync(rw_flags) != 0; + int sleep_on_global = 0; struct request *rq; + struct request_list *rl = blk_get_request_list(q, bio); - rq = get_request(q, rw_flags, bio, GFP_NOIO); + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl, &sleep_on_global); while (!rq) { DEFINE_WAIT(wait); struct io_context *ioc; - struct request_list *rl = &q->rq; - prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, - TASK_UNINTERRUPTIBLE); + if (sleep_on_global) { + /* + * Task failed allocation and needs to wait and + * try again. There are no requests pending from + * the io group hence need to sleep on global + * wait queue. Most likely the allocation failed + * because of memory issues. + */ + + q->rq_data.starved++; + prepare_to_wait_exclusive(&q->rq_data.starved_wait, + &wait, TASK_UNINTERRUPTIBLE); + } else { + /* + * We are about to sleep on a request list and we + * drop queue lock. After waking up, we will do + * finish_wait() on request list and in the mean + * time group might be gone. Take a reference to + * the group now. + */ + prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, + TASK_UNINTERRUPTIBLE); + elv_get_rl_iog(rl); + } trace_block_sleeprq(q, bio, rw_flags & 1);@@ -850,9 +1016,25 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, ioc_set_batching(q, ioc); spin_lock_irq(q->queue_lock); - finish_wait(&rl->wait[is_sync], &wait); - rq = get_request(q, rw_flags, bio, GFP_NOIO); + if (sleep_on_global) { + finish_wait(&q->rq_data.starved_wait, &wait); + sleep_on_global = 0; + } else { + /* + * We had taken a reference to the rl/iog. Put that now + */ + finish_wait(&rl->wait[is_sync], &wait); + elv_put_rl_iog(rl); + } + + /* + * After the sleep check the rl again in case cgrop bio + * belonged to is gone and it is mapped to root group now + */ + rl = blk_get_request_list(q, bio); + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl, + &sleep_on_global); }; return rq;@@ -861,14 +1043,16 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, struct request *blk_get_request(struct request_queue *q, int rw, gfp_t gfp_mask) { struct request *rq; + struct request_list *rl; BUG_ON(rw != READ && rw != WRITE); spin_lock_irq(q->queue_lock); + rl = blk_get_request_list(q, NULL); if (gfp_mask & __GFP_WAIT) { rq = get_request_wait(q, rw, NULL); } else { - rq = get_request(q, rw, NULL, gfp_mask); + rq = get_request(q, rw, NULL, gfp_mask, rl, NULL); if (!rq) spin_unlock_irq(q->queue_lock); }@@ -1085,12 +1269,13 @@ void __blk_put_request(struct request_queue *q, struct request *req) if (req->cmd_flags & REQ_ALLOCED) {
block/blk-core.c void __blk_put_request(struct request_queue *q, struct request *req) · line 1278
Quoted 10 lines
block/blk-core.c +2
-1int is_sync = rq_is_sync(req) != 0; int priv = req->cmd_flags & REQ_ELVPRIV; + struct request_list *rl = rq_rl(q, req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash)); blk_free_request(q, req); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl);
We have a potential memory bug here. freed_request should be called before blk_free_request as blk_free_request might result in release of cgroup, and request_list. Calling freed_request after blk_free_request would result in operations on freed memory.
Quoted 436 lines
block/blk-core.c +0
-0} } EXPORT_SYMBOL_GPL(__blk_put_request);block/blk-settings.c +1
-0@@ -149,6 +149,7 @@ void blk_queue_make_request(struct request_queue *q, make_request_fn *mfn)* set defaults */ q->nr_requests = BLKDEV_MAX_RQ; + q->nr_group_requests = BLKDEV_MAX_GROUP_RQ; q->make_request_fn = mfn; blk_queue_dma_alignment(q, 511);block/blk-sysfs.c +0
-0@@ -38,42 +38,67 @@ static ssize_t queue_requests_show(struct request_queue *q, char *page)static
ssize_t
queue_requests_store(structrequest_queue*q, const char *page,size_tcount) {
- structrequest_list*rl =&q->rq;
+ structrequest_list*rl;
unsigned long nr;
int ret =queue_var_store(&nr, page, count);
if (nr <BLKDEV_MIN_RQ)
nr =BLKDEV_MIN_RQ;spin_lock_irq(q->queue_lock);
+ rl =blk_get_request_list(q, NULL);
q->nr_requests = nr;
blk_queue_congestion_threshold(q);- if (rl->count[BLK_RW_SYNC] >=
queue_congestion_on_threshold(q))
+ if (q->rq_data.count[BLK_RW_SYNC] >=queue_congestion_on_threshold(q))blk_set_queue_congested(q, BLK_RW_SYNC);
- else if (rl->count[BLK_RW_SYNC] <queue_congestion_off_threshold(q))
+ else if (q->rq_data.count[BLK_RW_SYNC] <
+queue_congestion_off_threshold(q))blk_clear_queue_congested(q, BLK_RW_SYNC);- if (rl->count[BLK_RW_ASYNC] >=
queue_congestion_on_threshold(q))
+ if (q->rq_data.count[BLK_RW_ASYNC] >=queue_congestion_on_threshold(q))blk_set_queue_congested(q, BLK_RW_ASYNC);
- else if (rl->count[BLK_RW_ASYNC] <queue_congestion_off_threshold(q))
+ else if (q->rq_data.count[BLK_RW_ASYNC] <
+queue_congestion_off_threshold(q))blk_clear_queue_congested(q, BLK_RW_ASYNC);- if (rl->count[BLK_RW_SYNC] >=
q->nr_requests) {
+ if (q->rq_data.count[BLK_RW_SYNC] >=q->nr_requests) {blk_set_queue_full(q, BLK_RW_SYNC);
- } else if (rl->count[BLK_RW_SYNC]+1 <=q->nr_requests) {
+ } else if (q->rq_data.count[BLK_RW_SYNC]+1 <=q->nr_requests) {blk_clear_queue_full(q, BLK_RW_SYNC);wake_up(&rl->wait[BLK_RW_SYNC]);
}- if (rl->count[BLK_RW_ASYNC] >=
q->nr_requests) {
+ if (q->rq_data.count[BLK_RW_ASYNC] >=q->nr_requests) {blk_set_queue_full(q, BLK_RW_ASYNC);
- } else if (rl->count[BLK_RW_ASYNC]+1 <=q->nr_requests) {
+ } else if (q->rq_data.count[BLK_RW_ASYNC]+1 <=q->nr_requests) {blk_clear_queue_full(q, BLK_RW_ASYNC);wake_up(&rl->wait[BLK_RW_ASYNC]);
}
spin_unlock_irq(q->queue_lock);
return ret;
}block/blk-sysfs.c +22
-0+#ifdef CONFIG_GROUP_IOSCHED +static ssize_t queue_group_requests_show (struct request_queue *q,char *page) +{ + return queue_var_show(q->nr_group_requests, (page)); +} + +static ssize_t +queue_group_requests_store (struct request_queue *q,const char *page, + size_t count) +{ + unsigned long nr; + int ret = queue_var_store(&nr, page, count); + + if (nr < BLKDEV_MIN_RQ) + nr = BLKDEV_MIN_RQ; + + spin_lock_irq(q->queue_lock); + q->nr_group_requests = nr; + spin_unlock_irq(q->queue_lock); + return ret; +} +#endif static
ssize_tqueue_ra_show(struct request_queue *q, char *page){block/blk-sysfs.c +11
-0@@ -240,6 +265,14 @@ static struct queue_sysfs_entry queue_requests_entry = { .store = queue_requests_store, }; +#ifdef CONFIG_GROUP_IOSCHED +static struct queue_sysfs_entry queue_group_requests_entry = { + .attr = {.name = "nr_group_requests", .mode = S_IRUGO | S_IWUSR }, + .show = queue_group_requests_show, + .store = queue_group_requests_store, +}; +#endif + static struct queue_sysfs_entry queue_ra_entry = { .attr = {.name = "read_ahead_kb", .mode = S_IRUGO | S_IWUSR }, .show = queue_ra_show,@@ -314,6 +347,9 @@ static struct queue_sysfs_entry queue_iostats_entry = { static struct attribute *default_attrs[] = { &queue_requests_entry.attr, +#ifdef CONFIG_GROUP_IOSCHED + &queue_group_requests_entry.attr, +#endif &queue_ra_entry.attr, &queue_max_hw_sectors_entry.attr, &queue_max_sectors_entry.attr,@@ -393,12 +429,11 @@ static void blk_release_queue(struct kobject *kobj) {struct
request_queue*q =
container_of(kobj, structrequest_queue, kobj);
- structrequest_list*rl =&q->rq;blk_sync_queue(q);
- if (
rl->rq_pool)
-mempool_destroy(rl->rq_pool);
+ if (q->rq_data.rq_pool)
+mempool_destroy(q->rq_data.rq_pool);if (
q->queue_tags)
__blk_queue_free_tags(q);block/elevator-fq.c +36
-0@@ -925,6 +925,39 @@ static struct io_cgroup *cgroup_to_io_cgroup(struct cgroup *cgroup) struct io_cgroup, css); } +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio) +{ + struct io_group *iog; + + if (!elv_iosched_fair_queuing_enabled(q->elevator)) + iog = q->elevator->efqd->root_group; + else + iog = elv_io_get_io_group_bio(q, bio, 1); + + BUG_ON(!iog); + return &iog->rl; +} + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv) +{ + struct io_group *iog; + + if (!elv_iosched_fair_queuing_enabled(q->elevator)) + return &q->elevator->efqd->root_group->rl; + + BUG_ON(priv && !rq->ioq); + + if (priv) + iog = ioq_to_io_group(rq->ioq); + else + iog = q->elevator->efqd->root_group; + + BUG_ON(!iog); + return &iog->rl; +} + /** Search the io_group for efqd into the hash table (by now only a list) * of bgrp. Must be called under rcu_read_lock(). @@ -1281,6 +1314,8 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup) elv_get_iog(iog); io_group_path(iog); + blk_init_request_list(&iog->rl); + if (leaf == NULL) { leaf = iog; prev = leaf;@@ -1502,6 +1537,7 @@ static struct io_group *io_alloc_root_group(struct request_queue *q, for (i = 0; i < IO_IOPRIO_CLASSES; i++) iog->sched_data.service_tree[i] = ELV_SERVICE_TREE_INIT; + blk_init_request_list(&iog->rl); spin_lock_irq(&iocg->lock); rcu_assign_pointer(iog->key, key); hlist_add_head_rcu(&iog->group_node, &iocg->group_data);block/elevator-fq.h +29
-0@@ -128,6 +128,9 @@ struct io_group { /* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + + /* request list associated with the group */ + struct request_list rl; }; struct io_cgroup {@@ -425,11 +428,31 @@ static inline void elv_get_iog(struct io_group *iog) atomic_inc(&iog->ref); } +static inline struct io_group *rl_iog (struct request_list *rl) +{ + return container_of(rl, struct io_group, rl); +} + +static inline void elv_get_rl_iog (struct request_list *rl) +{ + elv_get_iog(rl_iog(rl)); +} + +static inline void elv_put_rl_iog (struct request_list *rl) +{ + elv_put_iog(rl_iog(rl)); +} + extern int elv_set_request_ioq(struct request_queue *q, struct request *rq, struct bio *bio, gfp_t gfp_mask); extern void elv_reset_request_ioq(struct request_queue *q, struct request *rq); extern struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio); +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio); + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv); #else /* !GROUP_IOSCHED */@@ -469,6 +492,9 @@ elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio) return NULL; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } + #endif /* GROUP_IOSCHED */ extern ssize_t elv_slice_sync_show(struct elevator_queue *q, char *name);@@ -578,6 +604,9 @@ static inline struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, return NULL; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } + #endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */block/elevator.c +4
-3@@ -678,7 +678,7 @@ void elv_quiesce_start(struct request_queue *q)* make sure we don't have any requests in flight */ elv_drain_elevator(q); - while (q->rq.elvpriv) { + while (q->rq_data.elvpriv) { __blk_run_queue(q); spin_unlock_irq(q->queue_lock); msleep(10);@@ -777,8 +777,9 @@ void elv_insert(struct request_queue *q, struct request *rq, int where) } if (unplug_it && blk_queue_plugged(q)) { - int nrq = q->rq.count[BLK_RW_SYNC] + q->rq.count[BLK_RW_ASYNC] - - queue_in_flight(q); + int nrq = q->rq_data.count[BLK_RW_SYNC] + + q->rq_data.count[BLK_RW_ASYNC] - + queue_in_flight(q); if (nrq >= q->unplug_thresh) __generic_unplug_device(q);include/linux/blkdev.h +39
-2@@ -32,21 +32,51 @@ struct request; struct sg_io_hdr; #define BLKDEV_MIN_RQ 4 + +#ifdef CONFIG_GROUP_IOSCHED +#define BLKDEV_MAX_RQ 512/* Default maximum for queue */ +#define BLKDEV_MAX_GROUP_RQ 128/* Default maximum per group*/ +#else #define BLKDEV_MAX_RQ 128 /* Default maximum */ +/* +* This is eqivalent to case of only one group present (root group). Let +* it consume all the request descriptors available on the queue . +*/ +#define BLKDEV_MAX_GROUP_RQ BLKDEV_MAX_RQ/* Default maximum */ +#endif struct request; typedef void (rq_end_io_fn)(struct request *, int); struct request_list { /* - * count[], starved[], and wait[] are indexed by + * count[], starved and wait[] are indexed by * BLK_RW_SYNC/BLK_RW_ASYNC */ int count[2]; int starved[2]; + wait_queue_head_t wait[2]; +}; + +/* +* This data structures keeps track of mempool of requests for the queue +* and some overall statistics. +*/ +struct request_data { + /* + * Per queue request descriptor count. This is in addition to per + * cgroup count + */ + int count[2]; int elvpriv; mempool_t *rq_pool; - wait_queue_head_t wait[2]; + int starved; + /* + * Global list for starved tasks. A task will be queued here if + * it could not allocate request descriptor and the associated + * group request list does not have any requests pending. + */ + wait_queue_head_t starved_wait; }; /*@@ -339,10 +369,17 @@ struct request_queue struct request *last_merge; struct elevator_queue *elevator; +#ifndef CONFIG_GROUP_IOSCHED /* * the queue request freelist, one for reads and one for writes + * In case of group io scheduling, this request list is per group + * and is present in group data structure. */ struct request_list rq; +#endif + + /* Contains request pool and other data like starved data */ + struct request_data rq_data; request_fn_proc *request_fn; make_request_fn *make_request_fn;@@ -405,6 +442,8 @@ struct request_queue* queue settings */ unsigned long
nr_requests; /* Max # of requests */
+ /* Max # of per io group requests */
+ unsigned longnr_group_requests;
unsigned int nr_congestion_on;
unsigned int nr_congestion_off;
unsigned int nr_batching;include/linux/blkdev.h +4
-0@@ -784,6 +823,10 @@ extern int scsi_cmd_ioctl(struct request_queue *, struct gendisk *, fmode_t, extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t, struct scsi_ioctl_command __user *); +extern void blk_init_request_list (struct request_list *rl); + +extern struct request_list *blk_get_request_list (struct request_queue *q, + struct bio *bio); /** A queue has just exitted congestion. Note this in the global counter of * congested queues, and wake up anyone who was waiting for requests to be include/trace/events/block.h +0
-0@@ -397,7 +397,8 @@ TRACE_EVENT(block_unplug_timer, ),TP_fast_assign(
-__entry->nr_rq= q->rq.count[READ] + q->rq.count[WRITE];
+__entry->nr_rq= q->rq_data.count[READ] +
+ q->rq_data.count[WRITE];memcpy(__entry->comm, current->comm, TASK_COMM_LEN); ),include/trace/events/block.h +0
-0@@ -416,7 +417,8 @@ TRACE_EVENT(block_unplug_io, ),TP_fast_assign(
-__entry->nr_rq= q->rq.count[READ] + q->rq.count[WRITE];
+__entry->nr_rq= q->rq_data.count[READ] +
+ q->rq_data.count[WRITE];memcpy(__entry->comm, current->comm, TASK_COMM_LEN); ),kernel/trace/blktrace.c +4
-3@@ -786,7 +786,8 @@ static void blk_add_trace_unplug_io(struct request_queue *q) struct blk_trace *bt = q->blk_trace; if (bt) { - unsigned int pdu = q->rq.count[READ] + q->rq.count[WRITE]; + unsigned int pdu = q->rq_data.count[READ] + + q->rq_data.count[WRITE]; __be64 rpdu = cpu_to_be64(pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_IO, 0,@@ -799,7 +800,8 @@ static void blk_add_trace_unplug_timer(struct request_queue *q) struct blk_trace *bt = q->blk_trace; if (bt) { - unsigned int pdu = q->rq.count[READ] + q->rq.count[WRITE]; + unsigned int pdu = q->rq_data.count[READ] + + q->rq_data.count[WRITE]; __be64 rpdu = cpu_to_be64(pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_TIMER, 0, --1.6.0.6
Re: [PATCH 20/23] io-controller: Per cgroup request descriptor support
In reply to Vivek GoyalQuoted 514 lines · Vivek Goyal
o Currently a request queue has got fixed number of request descriptors for sync and async requests. Once the request descriptors are consumed, new processes are put to sleep and they effectively become serialized. Because sync and async queues are separate, async requests don't impact sync ones but if one is looking for fairness between async requests, that is not achievable if request queue descriptors become bottleneck.
o Make request descriptor's per io group so that if there is lots of IO going on in one cgroup, it does not impact the IO of other group.
o This patch implements the per cgroup request descriptors. request pool per queue is still common but every group will have its own wait list and its own count of request descriptors allocated to that group for sync and async queues. So effectively
request_listbecomes per io group property and not a global request queue feature.o Currently one can define
q->nr_requeststo limit request descriptors allocated for the queue. Now there is another tunableq->nr_group_requestswhich controls the requests descriptr limit per group.q->nr_requestssupercedesq->nr_group_requeststo make sure if there are lots of groups present, we don't end up allocating too many request descriptors on the queue.Signed-off-by Nauman Rafique <[email protected]>Signed-off-by Vivek Goyal <[email protected]>---
block/blk-core.c | 317 +++++++++++++++++++++++++++++++++---------block/blk-settings.c| 1 +
block/blk-sysfs.c | 59 ++++++--
block/elevator-fq.c | 36 +++++
block/elevator-fq.h | 29 ++++
block/elevator.c | 7 +-
include/linux/blkdev.h | 47 ++++++-include/trace/events/block.h| 6 +-
kernel/trace/blktrace.c | 6 +-9 files changed, 421 insertions(+), 87 deletions(-)block/blk-core.c +44
-8@@ -460,20 +460,53 @@ void blk_cleanup_queue(struct request_queue *q) } EXPORT_SYMBOL(blk_cleanup_queue); -static int blk_init_free_list (struct request_queue *q) +struct request_list * +blk_get_request_list (struct request_queue *q,struct bio *bio) +{ +#ifdef CONFIG_GROUP_IOSCHED + /* + * Determine which request list bio will be allocated from. This + * is dependent on which io group bio belongs to + */ + return elv_get_request_list_bio(q, bio); +#else + return &q->rq; +#endif +} + +static struct request_list *rq_rl (struct request_queue *q,struct request *rq) +{ +#ifdef CONFIG_GROUP_IOSCHED + int priv = rq->cmd_flags & REQ_ELVPRIV; + + return elv_get_request_list_rq(q, rq, priv); +#else + return &q->rq; +#endif +} + +void blk_init_request_list (struct request_list *rl) { - struct request_list *rl = &q->rq; rl->count[BLK_RW_SYNC] = rl->count[BLK_RW_ASYNC] = 0; - rl->starved[BLK_RW_SYNC] = rl->starved[BLK_RW_ASYNC] = 0; - rl->elvpriv = 0; init_waitqueue_head(&rl->wait[BLK_RW_SYNC]); init_waitqueue_head(&rl->wait[BLK_RW_ASYNC]); +} - rl->rq_pool = mempool_create_node(BLKDEV_MIN_RQ, mempool_alloc_slab, - mempool_free_slab, request_cachep, q->node); +static int blk_init_free_list (struct request_queue *q) +{ + /* +* In case of group scheduling, request list is inside group and is +* initialized when group is instanciated. +*/ +#ifndef CONFIG_GROUP_IOSCHED + blk_init_request_list(&q->rq); +#endif + q->rq_data.rq_pool = mempool_create_node(BLKDEV_MIN_RQ, + mempool_alloc_slab, mempool_free_slab, + request_cachep, q->node); - if (!rl->rq_pool) + if (!q->rq_data.rq_pool) return -ENOMEM; return 0;@@ -581,6 +614,9 @@ blk_init_queue_node(request_fn_proc *rfn, spinlock_t *lock, int node_id) q->queue_flags = QUEUE_FLAG_DEFAULT; q->queue_lock = lock; + /* init starved waiter wait queue */ + init_waitqueue_head(&q->rq_data.starved_wait); + /** This also sets hw/phys segments, boundary and size */ @@ -615,14 +651,14 @@ static inline void blk_free_request(struct request_queue *q, struct request *rq) { if (rq->cmd_flags & REQ_ELVPRIV) elv_put_request(q, rq); - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); }static struct request *
blk_alloc_request(structrequest_queue*q, struct bio *bio, int flags, int priv,gfp_tgfp_mask)
{
- struct request *rq =mempool_alloc(q->rq.rq_pool, gfp_mask);
+ struct request *rq =mempool_alloc(q->rq_data.rq_pool, gfp_mask);if (!rq)
return NULL;block/blk-core.c +151
-45@@ -633,7 +669,7 @@ blk_alloc_request(struct request_queue *q, struct bio *bio, int flags, int priv, if (priv) { if (unlikely(elv_set_request(q, rq, bio, gfp_mask))) { - mempool_free(rq, q->rq.rq_pool); + mempool_free(rq, q->rq_data.rq_pool); return NULL; } rq->cmd_flags |= REQ_ELVPRIV;@@ -676,18 +712,18 @@ static void ioc_set_batching(struct request_queue *q, struct io_context *ioc) ioc->last_waited = jiffies; } -static void __freed_request (struct request_queue *q,int sync) +static void __freed_request (struct request_queue *q,int sync, + struct request_list *rl) { - struct request_list *rl = &q->rq; - - if (rl->count[sync] < queue_congestion_off_threshold(q)) + if (q->rq_data.count[sync] < queue_congestion_off_threshold(q)) blk_clear_queue_congested(q, sync); - if (rl->count[sync] + 1 <= q->nr_requests) { + if (q->rq_data.count[sync] + 1 <= q->nr_requests) + blk_clear_queue_full(q, sync); + + if (rl->count[sync] + 1 <= q->nr_group_requests) { if (waitqueue_active(&rl->wait[sync])) wake_up(&rl->wait[sync]); - - blk_clear_queue_full(q, sync); } }@@ -695,63 +731,130 @@ static void __freed_request(struct request_queue *q, int sync)* A request has just been released. Account for it, update the full and * congestion status, wake up any waiters. Called under q->queue_lock. */ -static void freed_request (struct request_queue *q,int sync,int priv) +static void freed_request (struct request_queue *q,int sync,int priv, + struct request_list *rl) { - struct request_list *rl = &q->rq; + /* + * There is a window during request allocation where request is + * mapped to one group but by the time a queue for the group is + * allocated, it is possible that original cgroup/io group has been + * deleted and now io queue is allocated in a different group (root) + * altogether. + * + * One solution to the problem is that rq should take io group + * reference. But it looks too much to do that to solve this issue. + * The only side affect to the hard to hit issue seems to be that + * we will try to decrement the rl->count for a request list which + * did not allocate that request. Chcek for rl->count going less than + * zero and do not decrement it if that's the case. + */ + + if (priv && rl->count[sync] > 0) + rl->count[sync]--; + + BUG_ON(!q->rq_data.count[sync]); + q->rq_data.count[sync]--; - rl->count[sync]--; if (priv) - rl->elvpriv--; + q->rq_data.elvpriv--; - __freed_request(q, sync); + __freed_request(q, sync, rl); if (unlikely(rl->starved[sync ^ 1])) - __freed_request(q, sync ^ 1); + __freed_request(q, sync ^ 1, rl); + + /* Wake up the starved process on global list, if any */ + if (unlikely(q->rq_data.starved)) { + if (waitqueue_active(&q->rq_data.starved_wait)) + wake_up(&q->rq_data.starved_wait); + q->rq_data.starved--; + } +} + +/* +* Returns whether one can sleep on this request list or not. There are +* cases (elevator switch) where request list might not have allocated +* any request descriptor but we deny request allocation due to gloabl +* limits. In that case one should sleep on global list as on this request +* list no wakeup will take place. +* +* Also sets the request list starved flag if there are no requests pending +* in the direction of rq. +* +* Return 1 --> sleep on request list, 0 --> sleep on global list +*/ +static int can_sleep_on_request_list (struct request_list *rl,int is_sync) +{ + if (unlikely(rl->count[is_sync] == 0)) { + /* + * If there is a request pending in other direction + * in same io group, then set the starved flag of + * the group request list. Otherwise, we need to + * make this process sleep in global starved list + * to make sure it will not sleep indefinitely. + */ + if (rl->count[is_sync ^ 1] != 0) { + rl->starved[is_sync] = 1; + return 1; + } else + return 0; + } + + return 1; } /* * Get a free request, queue_lock must be held. - * Returns NULL on failure, with queue_lock held. + * Returns NULL on failure, with queue_lock held. Also sets the "reason" field + * in case of failure. This reason field helps caller decide to whether sleep + * on per group list or global per queue list. + * reason = 0 sleep on per group list + * reason = 1 sleep on global list + * * Returns !NULL on success, with queue_lock *not held*. */ static struct request *get_request(struct request_queue *q, int rw_flags, - struct bio *bio, gfp_t gfp_mask) + struct bio *bio, gfp_t gfp_mask, + struct request_list *rl, int *reason) { struct request *rq = NULL; - struct request_list *rl = &q->rq; struct io_context *ioc = NULL; const bool is_sync = rw_is_sync(rw_flags) != 0; int may_queue, priv; + int sleep_on_global = 0; may_queue = elv_may_queue(q, rw_flags); if (may_queue == ELV_MQUEUE_NO) goto rq_starved; - if (rl->count[is_sync]+1 >= queue_congestion_on_threshold(q)) { - if (rl->count[is_sync]+1 >= q->nr_requests) { - ioc = current_io_context(GFP_ATOMIC, q->node); - /* - * The queue will fill after this allocation, so set - * it as full, and mark this process as "batching". - * This process will be allowed to complete a batch of - * requests, others will be blocked. - */ - if (!blk_queue_full(q, is_sync)) { - ioc_set_batching(q, ioc); - blk_set_queue_full(q, is_sync); - } else { - if (may_queue != ELV_MQUEUE_MUST - && !ioc_batching(q, ioc)) { - /* - * The queue is full and the allocating - * process is not a "batcher", and not - * exempted by the IO scheduler - */ - goto out; - } + if (q->rq_data.count[is_sync]+1 >= queue_congestion_on_threshold(q)) + blk_set_queue_congested(q, is_sync); + + /* queue full seems redundant now */ + if (q->rq_data.count[is_sync]+1 >= q->nr_requests) + blk_set_queue_full(q, is_sync); + + if (rl->count[is_sync]+1 >= q->nr_group_requests) { + ioc = current_io_context(GFP_ATOMIC, q->node); + /* + * The queue request descriptor group will fill after this + * allocation, so set it as full, and mark this process as + * "batching". This process will be allowed to complete a + * batch of requests, others will be blocked. + */ + if (rl->count[is_sync] <= q->nr_group_requests) + ioc_set_batching(q, ioc); + else { + if (may_queue != ELV_MQUEUE_MUST + && !ioc_batching(q, ioc)) { + /* + * The queue is full and the allocating + * process is not a "batcher", and not + * exempted by the IO scheduler + */ + goto out; } } - blk_set_queue_congested(q, is_sync); } /*@@ -759,21 +862,60 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* limit of requests, otherwise we could have thousands of requests * allocated with any setting of ->nr_requests */ - if (rl->count[is_sync] >= (3 * q->nr_requests / 2)) + + if (q->rq_data.count[is_sync] >= (3 * q->nr_requests / 2)) { + /* + * Queue is too full for allocation. On which request queue + * the task should sleep? Generally it should sleep on its + * request list but if elevator switch is happening, in that + * window, request descriptors are allocated from global + * pool and are not accounted against any particular request + * list as group is going away. + * + * So it might happen that request list does not have any + * requests allocated at all and if process sleeps on per + * group request list, it will not be woken up. In such case, + * make it sleep on global starved list. + */ + if (test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags) + || !can_sleep_on_request_list(rl, is_sync)) + sleep_on_global = 1; + goto out; + } + + /* + * Allocation of request is allowed from queue perspective. Now check + * from per group request list + */ + + if (rl->count[is_sync] >= (3 * q->nr_group_requests / 2)) goto out; - rl->count[is_sync]++; rl->starved[is_sync] = 0; + q->rq_data.count[is_sync]++; + priv = !test_bit(QUEUE_FLAG_ELVSWITCH, &q->queue_flags); - if (priv) - rl->elvpriv++; + if (priv) { + q->rq_data.elvpriv++; + /* + * Account the request to request list only if request is + * going to elevator. During elevator switch, there will + * be small window where group is going away and new group + * will not be allocated till elevator switch is complete. + * So till then instead of slowing down the application, + * we will continue to allocate request from total common + * pool instead of per group limit + */ + rl->count[is_sync]++; + } if (blk_queue_io_stat(q)) rw_flags |= REQ_IO_STAT; spin_unlock_irq(q->queue_lock); rq = blk_alloc_request(q, bio, rw_flags, priv, gfp_mask); + if (unlikely(!rq)) { /** Allocation failed presumably due to memory. Undo anything @@ -783,7 +925,7 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* wait queue, but this is pretty rare. */ spin_lock_irq(q->queue_lock); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl); /** in the very unlikely event that allocation failed and no @@ -793,9 +935,8 @@ static struct request *get_request(struct request_queue *q, int rw_flags,* rq mempool into READ and WRITE */ rq_starved:
- if (unlikely(rl->count[is_sync] == 0))
- rl->starved[is_sync] = 1; -
+ if (!can_sleep_on_request_list(rl,is_sync))
+sleep_on_global= 1;
goto out;
}block/blk-core.c +0
-0@@ -810,6 +951,8 @@ rq_starved: trace_block_getrq(q, bio, rw_flags & 1);out:
+ if (reason &&sleep_on_global)
+ *reason = 1;
return rq;
}block/blk-core.c +48
-7@@ -823,16 +966,39 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, struct bio *bio) { const bool is_sync = rw_is_sync(rw_flags) != 0; + int sleep_on_global = 0; struct request *rq; + struct request_list *rl = blk_get_request_list(q, bio); - rq = get_request(q, rw_flags, bio, GFP_NOIO); + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl, &sleep_on_global); while (!rq) { DEFINE_WAIT(wait); struct io_context *ioc; - struct request_list *rl = &q->rq; - prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, - TASK_UNINTERRUPTIBLE); + if (sleep_on_global) { + /* + * Task failed allocation and needs to wait and + * try again. There are no requests pending from + * the io group hence need to sleep on global + * wait queue. Most likely the allocation failed + * because of memory issues. + */ + + q->rq_data.starved++; + prepare_to_wait_exclusive(&q->rq_data.starved_wait, + &wait, TASK_UNINTERRUPTIBLE); + } else { + /* + * We are about to sleep on a request list and we + * drop queue lock. After waking up, we will do + * finish_wait() on request list and in the mean + * time group might be gone. Take a reference to + * the group now. + */ + prepare_to_wait_exclusive(&rl->wait[is_sync], &wait, + TASK_UNINTERRUPTIBLE); + elv_get_rl_iog(rl); + } trace_block_sleeprq(q, bio, rw_flags & 1);@@ -850,9 +1016,25 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, ioc_set_batching(q, ioc); spin_lock_irq(q->queue_lock); - finish_wait(&rl->wait[is_sync], &wait); - rq = get_request(q, rw_flags, bio, GFP_NOIO); + if (sleep_on_global) { + finish_wait(&q->rq_data.starved_wait, &wait); + sleep_on_global = 0; + } else { + /* + * We had taken a reference to the rl/iog. Put that now + */ + finish_wait(&rl->wait[is_sync], &wait); + elv_put_rl_iog(rl); + } + + /* + * After the sleep check the rl again in case cgrop bio + * belonged to is gone and it is mapped to root group now + */ + rl = blk_get_request_list(q, bio); + rq = get_request(q, rw_flags, bio, GFP_NOIO, rl, + &sleep_on_global); }; return rq;@@ -861,14 +1043,16 @@ static struct request *get_request_wait(struct request_queue *q, int rw_flags, struct request *blk_get_request(struct request_queue *q, int rw, gfp_t gfp_mask) { struct request *rq; + struct request_list *rl; BUG_ON(rw != READ && rw != WRITE); spin_lock_irq(q->queue_lock); + rl = blk_get_request_list(q, NULL); if (gfp_mask & __GFP_WAIT) { rq = get_request_wait(q, rw, NULL); } else { - rq = get_request(q, rw, NULL, gfp_mask); + rq = get_request(q, rw, NULL, gfp_mask, rl, NULL); if (!rq) spin_unlock_irq(q->queue_lock); }@@ -1085,12 +1269,13 @@ void __blk_put_request(struct request_queue *q, struct request *req) if (req->cmd_flags & REQ_ALLOCED) {
block/blk-core.c void __blk_put_request(struct request_queue *q, struct request *req) · line 1278
Quoted 10 lines
block/blk-core.c +2
-1int is_sync = rq_is_sync(req) != 0; int priv = req->cmd_flags & REQ_ELVPRIV; + struct request_list *rl = rq_rl(q, req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash)); blk_free_request(q, req); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl);
We have a potential memory bug here. freed_request should be called before blk_free_request as blk_free_request might result in release of cgroup, and request_list. Calling freed_request after blk_free_request would result in operations on freed memory.
Quoted 436 lines
block/blk-core.c +0
-0} } EXPORT_SYMBOL_GPL(__blk_put_request);block/blk-settings.c +1
-0@@ -149,6 +149,7 @@ void blk_queue_make_request(struct request_queue *q, make_request_fn *mfn)* set defaults */ q->nr_requests = BLKDEV_MAX_RQ; + q->nr_group_requests = BLKDEV_MAX_GROUP_RQ; q->make_request_fn = mfn; blk_queue_dma_alignment(q, 511);block/blk-sysfs.c +0
-0@@ -38,42 +38,67 @@ static ssize_t queue_requests_show(struct request_queue *q, char *page)static
ssize_t
queue_requests_store(structrequest_queue*q, const char *page,size_tcount) {
- structrequest_list*rl =&q->rq;
+ structrequest_list*rl;
unsigned long nr;
int ret =queue_var_store(&nr, page, count);
if (nr <BLKDEV_MIN_RQ)
nr =BLKDEV_MIN_RQ;spin_lock_irq(q->queue_lock);
+ rl =blk_get_request_list(q, NULL);
q->nr_requests = nr;
blk_queue_congestion_threshold(q);- if (rl->count[BLK_RW_SYNC] >=
queue_congestion_on_threshold(q))
+ if (q->rq_data.count[BLK_RW_SYNC] >=queue_congestion_on_threshold(q))blk_set_queue_congested(q, BLK_RW_SYNC);
- else if (rl->count[BLK_RW_SYNC] <queue_congestion_off_threshold(q))
+ else if (q->rq_data.count[BLK_RW_SYNC] <
+queue_congestion_off_threshold(q))blk_clear_queue_congested(q, BLK_RW_SYNC);- if (rl->count[BLK_RW_ASYNC] >=
queue_congestion_on_threshold(q))
+ if (q->rq_data.count[BLK_RW_ASYNC] >=queue_congestion_on_threshold(q))blk_set_queue_congested(q, BLK_RW_ASYNC);
- else if (rl->count[BLK_RW_ASYNC] <queue_congestion_off_threshold(q))
+ else if (q->rq_data.count[BLK_RW_ASYNC] <
+queue_congestion_off_threshold(q))blk_clear_queue_congested(q, BLK_RW_ASYNC);- if (rl->count[BLK_RW_SYNC] >=
q->nr_requests) {
+ if (q->rq_data.count[BLK_RW_SYNC] >=q->nr_requests) {blk_set_queue_full(q, BLK_RW_SYNC);
- } else if (rl->count[BLK_RW_SYNC]+1 <=q->nr_requests) {
+ } else if (q->rq_data.count[BLK_RW_SYNC]+1 <=q->nr_requests) {blk_clear_queue_full(q, BLK_RW_SYNC);wake_up(&rl->wait[BLK_RW_SYNC]);
}- if (rl->count[BLK_RW_ASYNC] >=
q->nr_requests) {
+ if (q->rq_data.count[BLK_RW_ASYNC] >=q->nr_requests) {blk_set_queue_full(q, BLK_RW_ASYNC);
- } else if (rl->count[BLK_RW_ASYNC]+1 <=q->nr_requests) {
+ } else if (q->rq_data.count[BLK_RW_ASYNC]+1 <=q->nr_requests) {blk_clear_queue_full(q, BLK_RW_ASYNC);wake_up(&rl->wait[BLK_RW_ASYNC]);
}
spin_unlock_irq(q->queue_lock);
return ret;
}block/blk-sysfs.c +22
-0+#ifdef CONFIG_GROUP_IOSCHED +static ssize_t queue_group_requests_show (struct request_queue *q,char *page) +{ + return queue_var_show(q->nr_group_requests, (page)); +} + +static ssize_t +queue_group_requests_store (struct request_queue *q,const char *page, + size_t count) +{ + unsigned long nr; + int ret = queue_var_store(&nr, page, count); + + if (nr < BLKDEV_MIN_RQ) + nr = BLKDEV_MIN_RQ; + + spin_lock_irq(q->queue_lock); + q->nr_group_requests = nr; + spin_unlock_irq(q->queue_lock); + return ret; +} +#endif static
ssize_tqueue_ra_show(struct request_queue *q, char *page){block/blk-sysfs.c +11
-0@@ -240,6 +265,14 @@ static struct queue_sysfs_entry queue_requests_entry = { .store = queue_requests_store, }; +#ifdef CONFIG_GROUP_IOSCHED +static struct queue_sysfs_entry queue_group_requests_entry = { + .attr = {.name = "nr_group_requests", .mode = S_IRUGO | S_IWUSR }, + .show = queue_group_requests_show, + .store = queue_group_requests_store, +}; +#endif + static struct queue_sysfs_entry queue_ra_entry = { .attr = {.name = "read_ahead_kb", .mode = S_IRUGO | S_IWUSR }, .show = queue_ra_show,@@ -314,6 +347,9 @@ static struct queue_sysfs_entry queue_iostats_entry = { static struct attribute *default_attrs[] = { &queue_requests_entry.attr, +#ifdef CONFIG_GROUP_IOSCHED + &queue_group_requests_entry.attr, +#endif &queue_ra_entry.attr, &queue_max_hw_sectors_entry.attr, &queue_max_sectors_entry.attr,@@ -393,12 +429,11 @@ static void blk_release_queue(struct kobject *kobj) {struct
request_queue*q =
container_of(kobj, structrequest_queue, kobj);
- structrequest_list*rl =&q->rq;blk_sync_queue(q);
- if (
rl->rq_pool)
-mempool_destroy(rl->rq_pool);
+ if (q->rq_data.rq_pool)
+mempool_destroy(q->rq_data.rq_pool);if (
q->queue_tags)
__blk_queue_free_tags(q);block/elevator-fq.c +36
-0@@ -925,6 +925,39 @@ static struct io_cgroup *cgroup_to_io_cgroup(struct cgroup *cgroup) struct io_cgroup, css); } +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio) +{ + struct io_group *iog; + + if (!elv_iosched_fair_queuing_enabled(q->elevator)) + iog = q->elevator->efqd->root_group; + else + iog = elv_io_get_io_group_bio(q, bio, 1); + + BUG_ON(!iog); + return &iog->rl; +} + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv) +{ + struct io_group *iog; + + if (!elv_iosched_fair_queuing_enabled(q->elevator)) + return &q->elevator->efqd->root_group->rl; + + BUG_ON(priv && !rq->ioq); + + if (priv) + iog = ioq_to_io_group(rq->ioq); + else + iog = q->elevator->efqd->root_group; + + BUG_ON(!iog); + return &iog->rl; +} + /** Search the io_group for efqd into the hash table (by now only a list) * of bgrp. Must be called under rcu_read_lock(). @@ -1281,6 +1314,8 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup) elv_get_iog(iog); io_group_path(iog); + blk_init_request_list(&iog->rl); + if (leaf == NULL) { leaf = iog; prev = leaf;@@ -1502,6 +1537,7 @@ static struct io_group *io_alloc_root_group(struct request_queue *q, for (i = 0; i < IO_IOPRIO_CLASSES; i++) iog->sched_data.service_tree[i] = ELV_SERVICE_TREE_INIT; + blk_init_request_list(&iog->rl); spin_lock_irq(&iocg->lock); rcu_assign_pointer(iog->key, key); hlist_add_head_rcu(&iog->group_node, &iocg->group_data);block/elevator-fq.h +29
-0@@ -128,6 +128,9 @@ struct io_group { /* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; + + /* request list associated with the group */ + struct request_list rl; }; struct io_cgroup {@@ -425,11 +428,31 @@ static inline void elv_get_iog(struct io_group *iog) atomic_inc(&iog->ref); } +static inline struct io_group *rl_iog (struct request_list *rl) +{ + return container_of(rl, struct io_group, rl); +} + +static inline void elv_get_rl_iog (struct request_list *rl) +{ + elv_get_iog(rl_iog(rl)); +} + +static inline void elv_put_rl_iog (struct request_list *rl) +{ + elv_put_iog(rl_iog(rl)); +} + extern int elv_set_request_ioq(struct request_queue *q, struct request *rq, struct bio *bio, gfp_t gfp_mask); extern void elv_reset_request_ioq(struct request_queue *q, struct request *rq); extern struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio); +struct request_list * +elv_get_request_list_bio (struct request_queue *q,struct bio *bio); + +struct request_list * +elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv); #else /* !GROUP_IOSCHED */@@ -469,6 +492,9 @@ elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio) return NULL; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } + #endif /* GROUP_IOSCHED */ extern ssize_t elv_slice_sync_show(struct elevator_queue *q, char *name);@@ -578,6 +604,9 @@ static inline struct io_queue *elv_lookup_ioq_bio(struct request_queue *q, return NULL; } +static inline void elv_get_rl_iog (struct request_list *rl) { } +static inline void elv_put_rl_iog (struct request_list *rl) { } + #endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */ #endif /* CONFIG_BLOCK */block/elevator.c +4
-3@@ -678,7 +678,7 @@ void elv_quiesce_start(struct request_queue *q)* make sure we don't have any requests in flight */ elv_drain_elevator(q); - while (q->rq.elvpriv) { + while (q->rq_data.elvpriv) { __blk_run_queue(q); spin_unlock_irq(q->queue_lock); msleep(10);@@ -777,8 +777,9 @@ void elv_insert(struct request_queue *q, struct request *rq, int where) } if (unplug_it && blk_queue_plugged(q)) { - int nrq = q->rq.count[BLK_RW_SYNC] + q->rq.count[BLK_RW_ASYNC] - - queue_in_flight(q); + int nrq = q->rq_data.count[BLK_RW_SYNC] + + q->rq_data.count[BLK_RW_ASYNC] - + queue_in_flight(q); if (nrq >= q->unplug_thresh) __generic_unplug_device(q);include/linux/blkdev.h +39
-2@@ -32,21 +32,51 @@ struct request; struct sg_io_hdr; #define BLKDEV_MIN_RQ 4 + +#ifdef CONFIG_GROUP_IOSCHED +#define BLKDEV_MAX_RQ 512/* Default maximum for queue */ +#define BLKDEV_MAX_GROUP_RQ 128/* Default maximum per group*/ +#else #define BLKDEV_MAX_RQ 128 /* Default maximum */ +/* +* This is eqivalent to case of only one group present (root group). Let +* it consume all the request descriptors available on the queue . +*/ +#define BLKDEV_MAX_GROUP_RQ BLKDEV_MAX_RQ/* Default maximum */ +#endif struct request; typedef void (rq_end_io_fn)(struct request *, int); struct request_list { /* - * count[], starved[], and wait[] are indexed by + * count[], starved and wait[] are indexed by * BLK_RW_SYNC/BLK_RW_ASYNC */ int count[2]; int starved[2]; + wait_queue_head_t wait[2]; +}; + +/* +* This data structures keeps track of mempool of requests for the queue +* and some overall statistics. +*/ +struct request_data { + /* + * Per queue request descriptor count. This is in addition to per + * cgroup count + */ + int count[2]; int elvpriv; mempool_t *rq_pool; - wait_queue_head_t wait[2]; + int starved; + /* + * Global list for starved tasks. A task will be queued here if + * it could not allocate request descriptor and the associated + * group request list does not have any requests pending. + */ + wait_queue_head_t starved_wait; }; /*@@ -339,10 +369,17 @@ struct request_queue struct request *last_merge; struct elevator_queue *elevator; +#ifndef CONFIG_GROUP_IOSCHED /* * the queue request freelist, one for reads and one for writes + * In case of group io scheduling, this request list is per group + * and is present in group data structure. */ struct request_list rq; +#endif + + /* Contains request pool and other data like starved data */ + struct request_data rq_data; request_fn_proc *request_fn; make_request_fn *make_request_fn;@@ -405,6 +442,8 @@ struct request_queue* queue settings */ unsigned long
nr_requests; /* Max # of requests */
+ /* Max # of per io group requests */
+ unsigned longnr_group_requests;
unsigned int nr_congestion_on;
unsigned int nr_congestion_off;
unsigned int nr_batching;include/linux/blkdev.h +4
-0@@ -784,6 +823,10 @@ extern int scsi_cmd_ioctl(struct request_queue *, struct gendisk *, fmode_t, extern int sg_scsi_ioctl(struct request_queue *, struct gendisk *, fmode_t, struct scsi_ioctl_command __user *); +extern void blk_init_request_list (struct request_list *rl); + +extern struct request_list *blk_get_request_list (struct request_queue *q, + struct bio *bio); /** A queue has just exitted congestion. Note this in the global counter of * congested queues, and wake up anyone who was waiting for requests to be include/trace/events/block.h +0
-0@@ -397,7 +397,8 @@ TRACE_EVENT(block_unplug_timer, ),TP_fast_assign(
-__entry->nr_rq= q->rq.count[READ] + q->rq.count[WRITE];
+__entry->nr_rq= q->rq_data.count[READ] +
+ q->rq_data.count[WRITE];memcpy(__entry->comm, current->comm, TASK_COMM_LEN); ),include/trace/events/block.h +0
-0@@ -416,7 +417,8 @@ TRACE_EVENT(block_unplug_io, ),TP_fast_assign(
-__entry->nr_rq= q->rq.count[READ] + q->rq.count[WRITE];
+__entry->nr_rq= q->rq_data.count[READ] +
+ q->rq_data.count[WRITE];memcpy(__entry->comm, current->comm, TASK_COMM_LEN); ),kernel/trace/blktrace.c +4
-3@@ -786,7 +786,8 @@ static void blk_add_trace_unplug_io(struct request_queue *q) struct blk_trace *bt = q->blk_trace; if (bt) { - unsigned int pdu = q->rq.count[READ] + q->rq.count[WRITE]; + unsigned int pdu = q->rq_data.count[READ] + + q->rq_data.count[WRITE]; __be64 rpdu = cpu_to_be64(pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_IO, 0,@@ -799,7 +800,8 @@ static void blk_add_trace_unplug_timer(struct request_queue *q) struct blk_trace *bt = q->blk_trace; if (bt) { - unsigned int pdu = q->rq.count[READ] + q->rq.count[WRITE]; + unsigned int pdu = q->rq_data.count[READ] + + q->rq_data.count[WRITE]; __be64 rpdu = cpu_to_be64(pdu); __blk_add_trace(bt, 0, 0, 0, BLK_TA_UNPLUG_TIMER, 0, --1.6.0.6
Re: [PATCH 20/23] io-controller: Per cgroup request descriptor support
On Mon, Sep 14, 2009 at 11:33:37AM -0700, Nauman Rafique wrote:
[..]
Quoted 26 lines
Quoted 26 lines
struct request *blk_get_request(struct
request_queue*q, int rw,gfp_tgfp_mask) {
struct request *rq;
+ structrequest_list*rl;BUG_ON(rw != READ && rw != WRITE);
spin_lock_irq(q->queue_lock);
+ rl =blk_get_request_list(q, NULL);
if (gfp_mask&__GFP_WAIT) {
rq =get_request_wait(q, rw, NULL);
} else {
- rq =get_request(q, rw, NULL, gfp_mask);
+ rq =get_request(q, rw, NULL, gfp_mask, rl, NULL); if (!rq)
spin_unlock_irq(q->queue_lock); }block/blk-core.c +1
-0@@ -1085,12 +1269,13 @@ void __blk_put_request(struct request_queue *q, struct request *req) if (req->cmd_flags & REQ_ALLOCED) { int is_sync = rq_is_sync(req) != 0; int priv = req->cmd_flags & REQ_ELVPRIV; + struct request_list *rl = rq_rl(q, req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash));
Quoted 9 lines
block/blk-core.c void __blk_put_request(struct request_queue *q, struct request *req) · line 1278
Quoted 3 lines
block/blk-core.c +1
-1blk_free_request(q, req); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl);We have a potential memory bug here.
freed_requestshould be called beforeblk_free_requestasblk_free_requestmight result in release of cgroup, andrequest_list. Callingfreed_requestafterblk_free_requestwould result in operations on freed memory.
Good point Nauman. freeing rq will drop reference on the io queue. Which in turn will drop reference on io group and if associated cgroup is already gone, then io group will be freed hence request list pointer is no more valid and any operation on that is bad.
So either we can take a reference on the queue, call free_request() and then drop the reference or we can call freed_request() before blk_free_request(). Calling freed_request() before blk_free_request() sounds better to me. The only thing is that this function name and other dependent function names should now be freeing_request() instead of freed_request(). :-)
Will move freed_request() before blk_free_request() in next version.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): e98e18940909141133m5186b780r3215ce15141e4f87-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH 20/23] io-controller: Per cgroup request descriptor support
On Mon, Sep 14, 2009 at 11:33:37AM -0700, Nauman Rafique wrote:
[..]
Quoted 26 lines
Quoted 26 lines
struct request *blk_get_request(struct
request_queue*q, int rw,gfp_tgfp_mask) {
struct request *rq;
+ structrequest_list*rl;BUG_ON(rw != READ && rw != WRITE);
spin_lock_irq(q->queue_lock);
+ rl =blk_get_request_list(q, NULL);
if (gfp_mask&__GFP_WAIT) {
rq =get_request_wait(q, rw, NULL);
} else {
- rq =get_request(q, rw, NULL, gfp_mask);
+ rq =get_request(q, rw, NULL, gfp_mask, rl, NULL); if (!rq)
spin_unlock_irq(q->queue_lock); }block/blk-core.c +1
-0@@ -1085,12 +1269,13 @@ void __blk_put_request(struct request_queue *q, struct request *req) if (req->cmd_flags & REQ_ALLOCED) { int is_sync = rq_is_sync(req) != 0; int priv = req->cmd_flags & REQ_ELVPRIV; + struct request_list *rl = rq_rl(q, req); BUG_ON(!list_empty(&req->queuelist)); BUG_ON(!hlist_unhashed(&req->hash));
Quoted 9 lines
block/blk-core.c void __blk_put_request(struct request_queue *q, struct request *req) · line 1278
Quoted 3 lines
block/blk-core.c +1
-1blk_free_request(q, req); - freed_request(q, is_sync, priv); + freed_request(q, is_sync, priv, rl);We have a potential memory bug here.
freed_requestshould be called beforeblk_free_requestasblk_free_requestmight result in release of cgroup, andrequest_list. Callingfreed_requestafterblk_free_requestwould result in operations on freed memory.
Good point Nauman. freeing rq will drop reference on the io queue. Which in turn will drop reference on io group and if associated cgroup is already gone, then io group will be freed hence request list pointer is no more valid and any operation on that is bad.
So either we can take a reference on the queue, call free_request() and then drop the reference or we can call freed_request() before blk_free_request(). Calling freed_request() before blk_free_request() sounds better to me. The only thing is that this function name and other dependent function names should now be freeing_request() instead of freed_request(). :-)
Will move freed_request() before blk_free_request() in next version.
Thanks
Vivek
PATCH 21/23 io-controller: Per io group bdi congestion interface
In reply to Vivek Goyalo So far there used to be only one pair or queue of request descriptors (one for sync and one for async) per device and number of requests allocated used to decide whether associated bdi is congested or not.
Now with per io group request descriptor infrastructure, there is a pair of request descriptor queue per io group per device. So it might happen that overall request queue is not congested but a particular io group bio belongs to is congested.
Or, it could be otherwise that group is not congested but overall queue is congested. This can happen if user has not properly set the request descriptors limits for queue and groups.
(q->nr_requests < nr_groups * q->nr_group_requests)
Hence there is a need for new interface which can query deivce congestion status per group. This group is determined by the "struct page" IO will be done for. If page is null, then group is determined from the current task context.
o This patch introduces new set of function bdi_*_congested_group(), which take "struct page" as addition argument. These functions will call the block layer and in trun elevator to find out if the io group the page will go into is congested or not.
o Currently I have introduced the core functions and migrated most of the users. But there might be still some left. This is an ongoing TODO item.
--- block/blk-core.c | 26 ++++++++ block/blk-sysfs.c | 6 +- block/elevator-fq.c | 135 +++++++++++++++++++++++++++++++++++++++++++ block/elevator-fq.h | 24 +++++++- drivers/md/dm-table.c | 11 ++- drivers/md/dm.c | 7 +- drivers/md/dm.h | 3 +- drivers/md/linear.c | 7 ++- drivers/md/multipath.c | 7 ++- drivers/md/raid0.c | 6 +- drivers/md/raid1.c | 9 ++- drivers/md/raid10.c | 6 +- drivers/md/raid5.c | 2 +- fs/afs/write.c | 8 ++- fs/btrfs/disk-io.c | 6 +- fs/btrfs/extent_io.c | 12 ++++ fs/btrfs/volumes.c | 8 ++- fs/cifs/file.c | 11 ++++ fs/ext2/ialloc.c | 2 +- fs/gfs2/aops.c | 12 ++++ fs/nilfs2/segbuf.c | 3 +- fs/xfs/linux-2.6/xfs_aops.c | 2 +- fs/xfs/linux-2.6/xfs_buf.c | 2 +- include/linux/backing-dev.h | 63 +++++++++++++++++++- include/linux/blkdev.h | 5 ++ mm/backing-dev.c | 74 ++++++++++++++++++++++- mm/page-writeback.c | 11 ++++ mm/readahead.c | 2 +- 28 files changed, 430 insertions(+), 40 deletions(-)
block/blk-core.c +26 -0
@@ -90,6 +90,27 @@ void blk_queue_congestion_threshold(struct request_queue *q) q->nr_congestion_off = nr; } +#ifdef CONFIG_GROUP_IOSCHED +int blk_queue_io_group_congested (struct backing_dev_info *bdi,int bdi_bits, +struct page *page) +{ +int ret =0 ; +struct request_queue *q = bdi->unplug_io_data; + +if (!q || !q->elevator) +return bdi_congested (bdi, bdi_bits); + +/* Do we need to hold queue lock? */ +if (bdi_bits & (1 << BDI_sync_congested)) + ret |=elv_page_io_group_congested (q, page,1 ); + +if (bdi_bits & (1 << BDI_async_congested)) + ret |=elv_page_io_group_congested (q, page,0 ); + +return ret; +} +#endif +/** * blk_get_backing_dev_info - get the address of a queue's backing_dev_info * @bdev: device
@@ -721,6 +742,8 @@ static void __freed_request(struct request_queue *q, int sync,if (q->rq_data.count[sync] +1 <= q->nr_requests)blk_clear_queue_full (q, sync); +elv_freed_request (rl, sync); +if (rl->count[sync] +1 <= q->nr_group_requests) {if (waitqueue_active (&rl->wait[sync]))wake_up (&rl->wait[sync]);
@@ -830,6 +853,9 @@ static struct request *get_request(struct request_queue *q, int rw_flags,if (q->rq_data.count[is_sync]+1 >=queue_congestion_on_threshold (q))blk_set_queue_congested (q, is_sync); +/* check if io group will get congested after this allocation*/ +elv_get_request (rl, is_sync); +/* queue full seems redundant now */ if (q->rq_data.count[is_sync]+1 >= q->nr_requests)blk_set_queue_full (q, is_sync);
block/blk-sysfs.c +3 -3
@@ -83,9 +83,8 @@ static ssize_t queue_group_requests_show(struct request_queue *q, char *page)return queue_var_show (q->nr_group_requests, (page)); } -static ssize_t -queue_group_requests_store (struct request_queue *q,const char *page, -size_t count) +static ssize_t queue_group_requests_store (struct request_queue *q, +const char *page,size_t count) {unsigned long nr;int ret =queue_var_store (&nr, page, count);
@@ -95,6 +94,7 @@ queue_group_requests_store(struct request_queue *q, const char *page,spin_lock_irq (q->queue_lock); q->nr_group_requests = nr; +elv_updated_nr_group_requests (q);spin_unlock_irq (q->queue_lock);return ret; }
block/elevator-fq.c +135 -0
153 lines not on this page · raw message
block/elevator-fq.h +23 -1
@@ -95,6 +95,13 @@ struct io_queue { }; #ifdef CONFIG_GROUP_IOSCHED /* CONFIG_GROUP_IOSCHED */ + +enum io_group_state { + IOG_async_congested, /* The async queue of group is getting full */ + IOG_sync_congested, /* The sync queue of group is getting full */ + IOG_unused, /* Available bits start here */ +}; + struct io_group { struct io_entity entity; atomic_t ref;
@@ -129,6 +136,11 @@ struct io_group {/* Single ioq per group, used for noop, deadline, anticipatory */ struct io_queue *ioq; +/* io group congestion on and off threshold for request descriptors */ +unsigned int nr_congestion_on; +unsigned int nr_congestion_off; + +unsigned long state;/* request list associated with the group */ struct request_list rl; };
@@ -453,6 +465,11 @@ elv_get_request_list_bio(struct request_queue *q, struct bio *bio);struct request_list *elv_get_request_list_rq (struct request_queue *q,struct request *rq,int priv); +extern int elv_page_io_group_congested (struct request_queue *q, +struct page *page,int sync); +extern void elv_freed_request (struct request_list *rl,int sync); +extern void elv_get_request (struct request_list *rl,int sync); +extern void elv_updated_nr_group_requests (struct request_queue *q);#else /* !GROUP_IOSCHED */
@@ -491,9 +508,11 @@ elv_lookup_ioq_bio(struct request_queue *q, struct bio *bio) {return NULL ; } -static inline void elv_get_rl_iog (struct request_list *rl) { }static inline void elv_put_rl_iog (struct request_list *rl) { } +static inline void elv_updated_nr_group_requests (struct request_queue *q) { } +static inline void elv_freed_request (struct request_list *rl,int sync) { } +static inline void elv_get_request (struct request_list *rl,int sync) { }#endif /* GROUP_IOSCHED */
@@ -606,6 +625,9 @@ static inline struct io_queue *elv_lookup_ioq_bio(struct request_queue *q,static inline void elv_get_rl_iog (struct request_list *rl) { }static inline void elv_put_rl_iog (struct request_list *rl) { } +static inline void elv_updated_nr_group_requests (struct request_queue *q) { } +static inline void elv_freed_request (struct request_list *rl,int sync) { } +static inline void elv_get_request (struct request_list *rl,int sync) { }#endif /* CONFIG_ELV_FAIR_QUEUING */ #endif /* _ELV_SCHED_H */
drivers/md/dm-table.c +7 -4
@@ -1170,7 +1170,8 @@ int dm_table_resume_targets(struct dm_table *t)return 0 ; } -int dm_table_any_congested (struct dm_table *t,int bdi_bits) +int dm_table_any_congested (struct dm_table *t,int bdi_bits,struct page *page, +int group) {struct dm_dev_internal *dd;struct list_head *devices =dm_table_get_devices (t);
@@ -1180,9 +1181,11 @@ int dm_table_any_congested(struct dm_table *t, int bdi_bits)struct request_queue *q =bdev_get_queue (dd->dm_dev.bdev);char b[BDEVNAME_SIZE ]; -if (likely (q)) - r |=bdi_congested (&q->backing_dev_info, bdi_bits); -else +if (likely (q)) { +struct backing_dev_info *bdi = &q->backing_dev_info; + r |= group ?bdi_congested_group (bdi, bdi_bits, page) + :bdi_congested (bdi, bdi_bits); + }else DMWARN_LIMIT ("%s: any_congested: nonexistent device %s" ,dm_device_name (t->md),bdevname (dd->dm_dev.bdev, b));
drivers/md/dm.c +4 -3
@@ -1608,7 +1608,8 @@ static void dm_unplug_all(struct request_queue *q) } } -static int dm_any_congested (void *congested_data,int bdi_bits) +static int dm_any_congested (void *congested_data,int bdi_bits, +struct page *page,int group) {int r = bdi_bits;struct mapped_device *md = congested_data;
@@ -1625,8 +1626,8 @@ static int dm_any_congested(void *congested_data, int bdi_bits) r = md->queue->backing_dev_info.state & bdi_bits; else - r =dm_table_any_congested (map, bdi_bits); - + r =dm_table_any_congested (map, bdi_bits, page, + group);dm_table_put (map); } }
drivers/md/dm.h +2 -1
@@ -57,7 +57,8 @@ struct list_head *dm_table_get_devices(struct dm_table *t);void dm_table_presuspend_targets (struct dm_table *t);void dm_table_postsuspend_targets (struct dm_table *t);int dm_table_resume_targets (struct dm_table *t); -int dm_table_any_congested (struct dm_table *t,int bdi_bits); +int dm_table_any_congested (struct dm_table *t,int bdi_bits,struct page *page, +int group);int dm_table_any_busy_target (struct dm_table *t);int dm_table_set_type (struct dm_table *t);unsigned dm_table_get_type (struct dm_table *t);
drivers/md/linear.c +5 -2
@@ -102,7 +102,7 @@ static void linear_unplug(struct request_queue *q)rcu_read_unlock (); } -static int linear_congested (void *data,int bits) +static int linear_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;linear_conf_t *conf;
@@ -113,7 +113,10 @@ static int linear_congested(void *data, int bits)for (i =0 ; i < mddev->raid_disks && !ret ; i++) {struct request_queue *q =bdev_get_queue (conf->disks[i].rdev->bdev); - ret |=bdi_congested (&q->backing_dev_info, bits); +struct backing_dev_info *bdi = &q->backing_dev_info; + + ret |= group ?bdi_congested_group (bdi, bits, page) : +bdi_congested (bdi, bits); }rcu_read_unlock ();
drivers/md/multipath.c +5 -2
@@ -192,7 +192,8 @@ static void multipath_status (struct seq_file *seq, mddev_t *mddev)seq_printf (seq,"]" ); } -static int multipath_congested (void *data,int bits) +static int multipath_congested (void *data,int bits,struct page *page, +int group) {mddev_t *mddev = data;multipath_conf_t *conf = mddev->private;
@@ -203,8 +204,10 @@ static int multipath_congested(void *data, int bits)mdk_rdev_t *rdev =rcu_dereference (conf->multipaths[i].rdev);if (rdev && !test_bit (Faulty, &rdev->flags)) {struct request_queue *q =bdev_get_queue (rdev->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info; - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, page) + :bdi_congested (bdi, bits);/* Just like multipath_map, we just check the * first available device */
drivers/md/raid0.c +4 -2
@@ -37,7 +37,7 @@ static void raid0_unplug(struct request_queue *q) } } -static int raid0_congested (void *data,int bits) +static int raid0_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;raid0_conf_t *conf = mddev->private;
@@ -46,8 +46,10 @@ static int raid0_congested(void *data, int bits)for (i =0 ; i < mddev->raid_disks && !ret ; i++) {struct request_queue *q =bdev_get_queue (devlist[i]->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info; - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, page) + :bdi_congested (bdi, bits); }return ret; }
drivers/md/raid1.c +6 -3
@@ -570,7 +570,7 @@ static void raid1_unplug(struct request_queue *q)md_wakeup_thread (mddev->thread); } -static int raid1_congested (void *data,int bits) +static int raid1_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;conf_t *conf = mddev->private;
@@ -581,14 +581,17 @@ static int raid1_congested(void *data, int bits)mdk_rdev_t *rdev =rcu_dereference (conf->mirrors[i].rdev);if (rdev && !test_bit (Faulty, &rdev->flags)) {struct request_queue *q =bdev_get_queue (rdev->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info;/* Note the '|| 1' - when read_balance prefers * non-congested targets, it can be removed */ if ((bits & (1 <<BDI_async_congested)) ||1 ) - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, + page) :bdi_congested (bdi, bits);else - ret &=bdi_congested (&q->backing_dev_info, bits); + ret &= group ?bdi_congested_group (bdi, bits, + page) :bdi_congested (bdi, bits); } }rcu_read_unlock ();
drivers/md/raid10.c +4 -2
@@ -625,7 +625,7 @@ static void raid10_unplug(struct request_queue *q)md_wakeup_thread (mddev->thread); } -static int raid10_congested (void *data,int bits) +static int raid10_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;conf_t *conf = mddev->private;
@@ -636,8 +636,10 @@ static int raid10_congested(void *data, int bits)mdk_rdev_t *rdev =rcu_dereference (conf->mirrors[i].rdev);if (rdev && !test_bit (Faulty, &rdev->flags)) {struct request_queue *q =bdev_get_queue (rdev->bdev); +struct backing_dev_info *bdi = &q->backing_dev_info; - ret |=bdi_congested (&q->backing_dev_info, bits); + ret |= group ?bdi_congested_group (bdi, bits, page) + :bdi_congested (bdi, bits); } }rcu_read_unlock ();
drivers/md/raid5.c +1 -1
@@ -3323,7 +3323,7 @@ static void raid5_unplug_device(struct request_queue *q)unplug_slaves (mddev); } -static int raid5_congested (void *data,int bits) +static int raid5_congested (void *data,int bits,struct page *page,int group) {mddev_t *mddev = data;raid5_conf_t *conf = mddev->private;
fs/afs/write.c +7 -1
@@ -455,7 +455,7 @@ int afs_writepage(struct page *page, struct writeback_control *wbc) } wbc->nr_to_write -= ret; -if (wbc->nonblocking &&bdi_write_congested (bdi)) +if (wbc->nonblocking &&bdi_or_group_write_congested (bdi, page)) wbc->encountered_congestion =1 ;_leave (" = 0" );
@@ -491,6 +491,12 @@ static int afs_writepages_region(struct address_space *mapping,return 0 ; } +if (wbc->nonblocking &&bdi_write_congested_group (bdi, page)) { + wbc->encountered_congestion =1 ; +page_cache_release (page); +break ; + } +/* at this point we hold neither mapping->tree_lock nor lock on * the page itself: the page may be truncated or invalidated * (changing page->mapping to NULL), or even swizzled back from
fs/btrfs/disk-io.c +4 -2
@@ -1249,7 +1249,8 @@ struct btrfs_root *btrfs_read_fs_root(struct btrfs_fs_info *fs_info,return root; } -static int btrfs_congested_fn (void *congested_data,int bdi_bits) +static int btrfs_congested_fn (void *congested_data,int bdi_bits, +struct page *page,int group) {struct btrfs_fs_info *info = (struct btrfs_fs_info *)congested_data;int ret =0 ;
@@ -1260,7 +1261,8 @@ static int btrfs_congested_fn(void *congested_data, int bdi_bits)if (!device->bdev)continue ; bdi =blk_get_backing_dev_info (device->bdev); -if (bdi &&bdi_congested (bdi, bdi_bits)) { +if (bdi && (group ?bdi_congested_group (bdi, bdi_bits, page) : +bdi_congested (bdi, bdi_bits))) { ret =1 ;break ; }
fs/btrfs/extent_io.c +12 -0
@@ -2368,6 +2368,18 @@ retry:unsigned i; scanned =1 ; + +/* +* If the io group page will go into is congested, bail out. +*/ +if (wbc->nonblocking + &&bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } +for (i =0 ; i < nr_pages; i++) {struct page *page = pvec.pages[i];
fs/btrfs/volumes.c +6 -2
@@ -165,6 +165,7 @@ static noinline int run_scheduled_bios(struct btrfs_device *device)unsigned long limit;unsigned long last_waited =0 ;int force_reg =0 ; +struct page *page; bdi =blk_get_backing_dev_info (device->bdev); fs_info = device->dev_root->fs_info;
@@ -276,8 +277,11 @@ loop_lock:* is now congested. Back off and let other work structs * run instead */ -if (pending &&bdi_write_congested (bdi) && batch_run >32 && - fs_info->fs_devices->open_devices >1 ) { +if (pending) + page =bio_iovec_idx (pending,0 )->bv_page; + +if (pending &&bdi_or_group_write_congested (bdi, page) && + num_run >32 && fs_info->fs_devices->open_devices >1 ) {struct io_context *ioc; ioc = current->io_context;
fs/cifs/file.c +11 -0
@@ -1470,6 +1470,17 @@ retry: n_iov =0 ; bytes_to_write =0 ; +/* +* If the io group page will go into is congested, bail out. +*/ +if (wbc->nonblocking && +bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } +for (i =0 ; i < nr_pages; i++) { page = pvec.pages[i];/*
fs/ext2/ialloc.c +1 -1
@@ -179,7 +179,7 @@ static void ext2_preread_inode(struct inode *inode)struct backing_dev_info *bdi; bdi = inode->i_mapping->backing_dev_info; -if (bdi_read_congested (bdi)) +if (bdi_or_group_read_congested (bdi,NULL ))return ;if (bdi_write_congested (bdi))return ;
fs/gfs2/aops.c +12 -0
@@ -371,6 +371,18 @@ retry: PAGECACHE_TAG_DIRTY, min(end - index, (pgoff_t)PAGEVEC_SIZE-1) + 1))) { scanned =1 ; + +/* +* If io group page belongs to is congested. bail out. +*/ +if (wbc->nonblocking + &&bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } + ret =gfs2_write_jdata_pagevec (mapping, wbc, &pvec, nr_pages, end);if (ret) done =1 ;
fs/nilfs2/segbuf.c +2 -1
@@ -266,8 +266,9 @@ static int nilfs_submit_seg_bio(struct nilfs_write_info *wi, int mode) {struct bio *bio = wi->bio;int err; +struct page *page =bio_iovec_idx (bio,0 )->bv_page; -if (wi->nbio >0 &&bdi_write_congested (wi->bdi)) { +if (wi->nbio >0 &&bdi_or_group_write_congested (wi->bdi, page)) {wait_for_completion (&wi->bio_event); wi->nbio--;if (unlikely (atomic_read (&wi->err))) {
fs/xfs/linux-2.6/xfs_aops.c +1 -1
@@ -891,7 +891,7 @@ xfs_convert_page( bdi = inode->i_mapping->backing_dev_info; wbc->nr_to_write--; -if (bdi_write_congested (bdi)) { +if (bdi_or_group_write_congested (bdi, page)) { wbc->encountered_congestion =1 ; done =1 ; }else if (wbc->nr_to_write <=0 ) {
fs/xfs/linux-2.6/xfs_buf.c +1 -1
@@ -714,7 +714,7 @@ xfs_buf_readahead(struct backing_dev_info *bdi; bdi = target->bt_mapping->backing_dev_info; -if (bdi_read_congested (bdi)) +if (bdi_or_group_read_congested (bdi,NULL ))return ; flags |= (XBF_TRYLOCK |XBF_ASYNC |XBF_READ_AHEAD );
include/linux/backing-dev.h +60 -3
87 lines not on this page · raw message
include/linux/blkdev.h +5 -0
@@ -846,6 +846,11 @@ static inline void blk_set_queue_congested(struct request_queue *q, int sync)set_bdi_congested (&q->backing_dev_info, sync); } +#ifdef CONFIG_GROUP_IOSCHED +extern int blk_queue_io_group_congested (struct backing_dev_info *bdi, +int bdi_bits,struct page *page); +#endif +extern void blk_start_queue (struct request_queue *q);extern void blk_stop_queue (struct request_queue *q);extern void blk_sync_queue (struct request_queue *q);
mm/backing-dev.c +71 -3
96 lines not on this page · raw message
mm/page-writeback.c +11 -0
@@ -985,6 +985,17 @@ retry:if (nr_pages ==0 )break ; +/* +* If the io group page will go into is congested, bail out. +*/ +if (wbc->nonblocking + &&bdi_write_congested_group (bdi, pvec.pages[0 ])) { + wbc->encountered_congestion =1 ; + done =1 ; +pagevec_release (&pvec); +break ; + } +for (i =0 ; i < nr_pages; i++) {struct page *page = pvec.pages[i];
mm/readahead.c +1 -1
@@ -542,7 +542,7 @@ page_cache_async_readahead(struct address_space *mapping,/* * Defer asynchronous read-ahead on IO congestion. */ -if (bdi_read_congested (mapping->backing_dev_info)) +if (bdi_or_group_read_congested (mapping->backing_dev_info,NULL ))return ;/* do read-ahead */
Signature · 1 line
1.6.0.6
Re: [PATCH 21/23] io-controller: Per io group bdi congestion interface
Quoted 3 lines · Vivek Goyal
o So far there used to be only one pair or queue of request descriptors (one for sync and one for async) per device and number of requests allocated used to decide whether associated bdi is congested or not.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-22-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 21/23] io-controller: Per io group bdi congestion interface
Quoted 3 lines · Vivek Goyal
o So far there used to be only one pair or queue of request descriptors (one for sync and one for async) per device and number of requests allocated used to decide whether associated bdi is congested or not.
PATCH 22/23 io-controller: Support per cgroup per device weights and io class
In reply to Vivek GoyalThis patch enables per-cgroup per-device weight and ioprio_class handling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight and ioprio_class for each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
You can use the following format to play with the new interface. #echo dev_major:dev_minor weight ioprio_class > /patch/to/cgroup/policy weight=0 means removing the policy for device.
Examples:
Configure weight=300 ioprio_class=2 on /dev/hdb (8:16) in this cgroup
# echo "8:16 300 2" > io.policy # cat io.policy dev weight class 8:16 300 2
Configure weight=500 ioprio_class=1 on /dev/hda (8:0) in this cgroup
# echo "8:0 500 1" > io.policy # cat io.policy dev weight class 8:0 500 1 8:16 300 2
Remove the policy for /dev/hda in this cgroup
# echo 8:0 0 1 > io.policy # cat io.policy dev weight class 8:16 300 2
Changelog (v1 -> v2)
- Rename some structures
- Use spin_lock_irqsave() and spin_lock_irqrestore() version to prevent from enabling the interrupts unconditionally.
- Fix policy setup bug when switching to another io scheduler.
- If a policy is available for a specific device, don't update weight and io class when writing "weight" and "iprio_class".
- Fix a bug when parsing policy string.
--- block/elevator-fq.c | 263 ++++++++++++++++++++++++++++++++++++++++++++++++++- block/elevator-fq.h | 10 ++ 2 files changed, 269 insertions(+), 4 deletions(-)
block/elevator-fq.c +259 -4
@@ -15,6 +15,7 @@#include <linux/blktrace_api.h> #include <linux/seq_file.h> #include <linux/biotrack.h> +#include <linux/genhd.h> #include "elevator-fq.h" const int elv_slice_sync =HZ /10 ;
@@ -866,12 +867,26 @@ EXPORT_SYMBOL(elv_io_group_set_async_queue);#ifdef CONFIG_GROUP_IOSCHED static void iocg_destroy (struct cgroup_subsys *subsys,struct cgroup *cgroup); -static void io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog) +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev); +static void +io_group_init_entity (struct io_cgroup *iocg,struct io_group *iog,dev_t dev) {struct io_entity *entity = &iog->entity; +struct io_policy_node *pn; +unsigned long flags; + +spin_lock_irqsave (&iocg->lock, flags); + pn =policy_search_node (iocg, dev); +if (pn) { + entity->weight = pn->weight; + entity->ioprio_class = pn->ioprio_class; + }else { + entity->weight = iocg->weight; + entity->ioprio_class = iocg->ioprio_class; + } +spin_unlock_irqrestore (&iocg->lock, flags); - entity->weight = iocg->weight; - entity->ioprio_class = iocg->ioprio_class; entity->ioprio_changed =1 ; entity->my_sd = &iog->sched_data; }
@@ -1111,6 +1126,229 @@ io_cgroup_lookup_group(struct io_cgroup *iocg, void *key)return NULL ; } +static int io_cgroup_policy_read (struct cgroup *cgrp,struct cftype *cft, +struct seq_file *m) +{ +struct io_cgroup *iocg; +struct io_policy_node *pn; + + iocg =cgroup_to_io_cgroup (cgrp); + +if (list_empty (&iocg->policy_list)) + goto out; + +seq_printf (m,"dev\tweight\tclass\n" ); + +spin_lock_irq (&iocg->lock); +list_for_each_entry (pn, &iocg->policy_list, node) { +seq_printf (m,"%u:%u\t%u\t%hu\n" ,MAJOR (pn->dev), +MINOR (pn->dev), pn->weight, pn->ioprio_class); + } +spin_unlock_irq (&iocg->lock); +out: +return 0 ; +} + +static inline void policy_insert_node (struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +list_add (&pn->node, &iocg->policy_list); +} + +/* Must be called with iocg->lock held */ +static inline void policy_delete_node (struct io_policy_node *pn) +{ +list_del (&pn->node); +} + +/* Must be called with iocg->lock held */ +static struct io_policy_node *policy_search_node (const struct io_cgroup *iocg, +dev_t dev) +{ +struct io_policy_node *pn; + +if (list_empty (&iocg->policy_list)) +return NULL ; + +list_for_each_entry (pn, &iocg->policy_list, node) { +if (pn->dev == dev) +return pn; + } + +return NULL ; +} + +static int check_dev_num (dev_t dev) +{ +int part =0 ; +struct gendisk *disk; + + disk =get_gendisk (dev, &part); +if (!disk || part) +return -ENODEV ; + +return 0 ; +} + +static int policy_parse_and_set (char *buf,struct io_policy_node *newpn) +{ +char *s[4 ], *p, *major_s =NULL , *minor_s =NULL ; +int ret; +unsigned long major, minor, temp; +int i =0 ; +dev_t dev; + +memset (s,0 ,sizeof (s)); +while ((p =strsep (&buf," " )) !=NULL ) { +if (!*p) +continue ; + s[i++] = p; + +/* Prevent from inputing too many things */ +if (i ==4 ) +break ; + } + +if (i !=3 ) +return -EINVAL ; + + p =strsep (&s[0 ],":" ); +if (p !=NULL ) + major_s = p; +else +return -EINVAL ; + + minor_s = s[0 ]; +if (!minor_s) +return -EINVAL ; + + ret =strict_strtoul (major_s,10 , &major); +if (ret) +return -EINVAL ; + + ret =strict_strtoul (minor_s,10 , &minor); +if (ret) +return -EINVAL ; + + dev =MKDEV (major, minor); + + ret =check_dev_num (dev); +if (ret) +return ret; + + newpn->dev = dev; + +if (s[1 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[1 ],10 , &temp); +if (ret || temp >IO_WEIGHT_MAX ) +return -EINVAL ; + + newpn->weight = temp; + +if (s[2 ] ==NULL ) +return -EINVAL ; + + ret =strict_strtoul (s[2 ],10 , &temp); +if (ret || temp <IOPRIO_CLASS_RT || temp >IOPRIO_CLASS_IDLE ) +return -EINVAL ; + newpn->ioprio_class = temp; + +return 0 ; +} + +static void update_iog_weight_prio (struct io_group *iog,struct io_cgroup *iocg, +struct io_policy_node *pn) +{ +if (pn->weight) { + iog->entity.weight = pn->weight; + iog->entity.ioprio_class = pn->ioprio_class; +/* +* iog weight and ioprio_class updating actually happens if +* ioprio_changed is set. So ensure ioprio_changed is not set +* until new weight and new ioprio_class are updated. +*/ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + }else { + iog->entity.weight = iocg->weight; + iog->entity.ioprio_class = iocg->ioprio_class; + +/* The same as above */ +smp_wmb (); + iog->entity.ioprio_changed =1 ; + } +} + +static int io_cgroup_policy_write (struct cgroup *cgrp,struct cftype *cft, +const char *buffer) +{ +struct io_cgroup *iocg; +struct io_policy_node *newpn, *pn; +char *buf; +int ret =0 ; +int keep_newpn =0 ; +struct hlist_node *n; +struct io_group *iog; + + buf =kstrdup (buffer,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + newpn =kzalloc (sizeof (*newpn),GFP_KERNEL ); +if (!newpn) { + ret = -ENOMEM ; + goto free_buf; + } + + ret =policy_parse_and_set (buf, newpn); +if (ret) + goto free_newpn; + +if (!cgroup_lock_live_group (cgrp)) { + ret = -ENODEV ; + goto free_newpn; + } + + iocg =cgroup_to_io_cgroup (cgrp); +spin_lock_irq (&iocg->lock); + + pn =policy_search_node (iocg, newpn->dev); +if (!pn) { +if (newpn->weight !=0 ) { +policy_insert_node (iocg, newpn); + keep_newpn =1 ; + } + goto update_io_group; + } + +if (newpn->weight ==0 ) { +/* weight == 0 means deleteing a policy */ +policy_delete_node (pn); + goto update_io_group; + } + + pn->weight = newpn->weight; + pn->ioprio_class = newpn->ioprio_class; + +update_io_group: +hlist_for_each_entry (iog, n, &iocg->group_data, group_node) { +if (iog->dev == newpn->dev) +update_iog_weight_prio (iog, iocg, newpn); + } +spin_unlock_irq (&iocg->lock); + +cgroup_unlock (); + +free_newpn: +if (!keep_newpn) +kfree (newpn); +free_buf: +kfree (buf); +return ret; +} +#define SHOW_FUNCTION (__VAR) \ static u64 io_cgroup_##__VAR##_read(struct cgroup *cgroup, \ struct cftype *cftype) \
@@ -1143,6 +1381,7 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \struct io_cgroup *iocg; \struct io_group *iog; \struct hlist_node *n; \ +struct io_policy_node *pn; \ \if (val < (__MIN) || val > (__MAX)) \return -EINVAL ; \
@@ -1155,6 +1394,9 @@ static int io_cgroup_##__VAR##_write(struct cgroup *cgroup, \spin_lock_irq (&iocg->lock); \ iocg->__VAR = (unsigned long )val; \ hlist_for_each_entry(iog, n, &iocg->group_data, group_node) { \ + pn =policy_search_node (iocg, iog->dev); \ +if (pn) \ +continue ; \ iog->entity.__VAR = (unsigned long )val; \smp_wmb (); \ iog->entity.ioprio_changed =1 ; \
@@ -1290,6 +1532,12 @@ static int io_cgroup_disk_dequeue_read(struct cgroup *cgroup,struct cftype io_files[] = { { + .name ="policy" , + .read_seq_string = io_cgroup_policy_read, + .write_string = io_cgroup_policy_write, + .max_write_len =256 , + }, + { .name ="weight" , .read_u64 = io_cgroup_weight_read, .write_u64 = io_cgroup_weight_write,
@@ -1340,6 +1588,7 @@ static struct cgroup_subsys_state *iocg_create(struct cgroup_subsys *subsys,INIT_HLIST_HEAD (&iocg->group_data); iocg->weight =IO_WEIGHT_DEFAULT ; iocg->ioprio_class =IOPRIO_CLASS_BE ; +INIT_LIST_HEAD (&iocg->policy_list);return &iocg->css; }
@@ -1433,7 +1682,7 @@ io_group_chain_alloc(struct request_queue *q, void *key, struct cgroup *cgroup)sscanf (dev_name (bdi->dev),"%u:%u" , &major, &minor); iog->dev =MKDEV (major, minor); -io_group_init_entity (iocg, iog); +io_group_init_entity (iocg, iog, iog->dev);atomic_set (&iog->ref,0 );
@@ -1761,6 +2010,7 @@ static void iocg_destroy(struct cgroup_subsys *subsys, struct cgroup *cgroup)struct io_group *iog;struct elv_fq_data *efqd;unsigned long uninitialized_var (flags ); +struct io_policy_node *pn, *pntmp;/* * io groups are linked in two lists. One list is maintained
@@ -1800,6 +2050,11 @@ remove_entry: goto remove_entry; done: +list_for_each_entry_safe (pn, pntmp, &iocg->policy_list, node) { +policy_delete_node (pn); +kfree (pn); + } +free_css_id (&io_subsys, &iocg->css);rcu_read_unlock ();BUG_ON (!hlist_empty (&iocg->group_data));
block/elevator-fq.h +10 -0
@@ -145,12 +145,22 @@ struct io_group {struct request_list rl; }; +struct io_policy_node { +struct list_head node; +dev_t dev; +unsigned int weight; +unsigned short ioprio_class; +}; +struct io_cgroup {struct cgroup_subsys_state css;unsigned int weight;unsigned short ioprio_class; +/* list of io_policy_node */ +struct list_head policy_list; +spinlock_t lock;struct hlist_head group_data; };
Signature · 1 line
1.6.0.6
Re: [PATCH 22/23] io-controller: Support per cgroup per device weights and io class
Quoted 6 lines · Vivek Goyal
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-23-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Rik van Riel
Acked-byRe: [PATCH 22/23] io-controller: Support per cgroup per device weights and io class
Quoted 6 lines · Vivek Goyal
This patch enables per-cgroup per-device weight and
ioprio_classhandling. A new cgroup interface "policy" is introduced. You can make use of this file to configure weight andioprio_classfor each device in a given cgroup. The original "weight" and "ioprio_class" files are still available. If you don't do special configuration for a particular device, "weight" and "ioprio_class" are used as default values in this device.
PATCH 23/23 io-controller: debug elevator fair queuing support
In reply to Vivek Goyalo More debugging help to debug elevator fair queuing support. Enabled under CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.
--- block/Kconfig.iosched | 9 +++++++++ block/elevator-fq.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 55 insertions(+), 1 deletions(-)
block/Kconfig.iosched +9 -0
@@ -15,6 +15,15 @@ config ELV_FAIR_QUEUING other ioschedulers can make use of it. If unsure, say N. +config DEBUG_ELV_FAIR_QUEUING + bool "Debug elevator fair queuing" + depends on ELV_FAIR_QUEUING + default n + ---help--- + Enable some debugging hooks for elevator fair queuing support. + Currently it just outputs more information about vdisktime in + blktrace output . + config IOSCHED_NOOP bool default y
block/elevator-fq.c +46 -1
73 lines not on this page · raw message
Signature · 1 line
1.6.0.6
Re: [PATCH 23/23] io-controller: debug elevator fair queuing support
Quoted 5 lines · Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under
CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.Signed-off-by Vivek Goyal <[email protected]>
Is this meant for merging upstream, or just as a temporary debugging help while this sits in a subsystem tree or -mm?
Re: [PATCH 23/23] io-controller: debug elevator fair queuing support
Quoted 9 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under
CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.Signed-off-by Vivek Goyal <[email protected]>Is this meant for merging upstream, or just as a temporary debugging help while this sits in a subsystem tree or -mm?
I think it would be good if this also is merged upstream. A useful debugging help to track fairness and latecy related issues.
Thanks
Vivek
Rik van Riel
Acked-byRe: [PATCH 23/23] io-controller: debug elevator fair queuing support
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Rik van Riel
Quoted 4 lines · Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under
CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Vivek Goyal <[email protected]>Is this meant for merging upstream, or just as a temporary debugging help while this sits in a subsystem tree or -mm?
I think it would be good if this also is merged upstream. A useful debugging help to track fairness and latecy related issues.
Fair enough. The code is small enough, anyway.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Rik van Riel
Acked-byRe: [PATCH 23/23] io-controller: debug elevator fair queuing support
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Rik van Riel
Quoted 4 lines · Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under
CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Vivek Goyal <[email protected]>Is this meant for merging upstream, or just as a temporary debugging help while this sits in a subsystem tree or -mm?
I think it would be good if this also is merged upstream. A useful debugging help to track fairness and latecy related issues.
Fair enough. The code is small enough, anyway.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 23/23] io-controller: debug elevator fair queuing support
Quoted 9 lines · Rik van Riel
Quoted 5 lines · Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under
CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.Signed-off-by Vivek Goyal <[email protected]>Is this meant for merging upstream, or just as a temporary debugging help while this sits in a subsystem tree or -mm?
I think it would be good if this also is merged upstream. A useful debugging help to track fairness and latecy related issues.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): 1251495072-7780-24-git-send-email-vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
Re: [PATCH 23/23] io-controller: debug elevator fair queuing support
Quoted 5 lines · Vivek Goyal
o More debugging help to debug elevator fair queuing support. Enabled under
CONFIG_DEBUG_ELV_FAIR_QUEUING. Currently it prints vdisktime related trace messages in blktrace.Signed-off-by Vivek Goyal <[email protected]>
Is this meant for merging upstream, or just as a temporary debugging help while this sits in a subsystem tree or -mm?
Quoted 14 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes belownr_congestion_off.
Quoted 5 lines
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
Ok, will do.
Signature · 2 lines
Regards Gui Jianfeng
Quoted 14 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes belownr_congestion_off.
Quoted 5 lines
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
Hi Vivek,
Here are some performance numbers generated by fio test. It seems V9 performance is better than V8 especially for write case.
Normal Read | Random Read | Normal Write | Random Write
V8(Avg of 3 timers) 64667 KiB/s 3387 KiB/s 59197 KiB/s 9327 KiB/s
V9(Avg of 3 timers) 65947 KiB/s 3528 KiB/s 61654 KiB/s 9744 KiB/s
Performance +2.0% +4.1% +4.2% +4.5%
Quoted 26 lines · Gui Jianfeng
Quoted 14 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes belownr_congestion_off.Quoted 5 lines
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
Hi Vivek,
Here are some performance numbers generated by fio test. It seems V9 performance is better than V8 especially for write case.
Quoted 7 lines
Normal Read | Random Read | Normal Write | Random Write
V8(Avg of 3 timers) 64667 KiB/s 3387 KiB/s 59197 KiB/s 9327 KiB/s
V9(Avg of 3 timers) 65947 KiB/s 3528 KiB/s 61654 KiB/s 9744 KiB/s
Performance +2.0% +4.1% +4.2% +4.5%
Thanks Gui. I was also keen to know how does the vanilla kernel and V9 comparision look like? Can you please run the same tests with vanilla kernel also?
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 26 lines · Gui Jianfeng
Quoted 14 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Changes from V8
===============
- Implemented bdi like congestion semantics for io group also. Now once an io group gets congested, we don't clear the congestion flag until number of requests goes belownr_congestion_off.Quoted 5 lines
This helps in getting rid of Buffered write performance regression we were observing with io controller patches.
Gui, can you please test it and see if this version is better in terms of your buffered write tests.
Hi Vivek,
Here are some performance numbers generated by fio test. It seems V9 performance is better than V8 especially for write case.
Quoted 7 lines
Normal Read | Random Read | Normal Write | Random Write
V8(Avg of 3 timers) 64667 KiB/s 3387 KiB/s 59197 KiB/s 9327 KiB/s
V9(Avg of 3 timers) 65947 KiB/s 3528 KiB/s 61654 KiB/s 9744 KiB/s
Performance +2.0% +4.1% +4.2% +4.5%
Thanks Gui. I was also keen to know how does the vanilla kernel and V9 comparision look like? Can you please run the same tests with vanilla kernel also?
Thanks
Vivek
Hi Vivek,
Here is the vanilla kernel and V9 comparision.
Normal Read | Random Read | Normal Write | Random Write
vanilla(Avg of 3 times) 67580 KiB/s 3540 KiB/s 61964 KiB/s 9823 KiB/s
V9(Avg of 3 times) 68954 KiB/s 3567 KiB/s 60654 KiB/s 9858 KiB/s
Performance +2.0% +0.7% -2.1% +0.3%
Hi Vivek,
Here is the vanilla kernel and V9 comparision.
Normal Read | Random Read | Normal Write | Random Write
vanilla(Avg of 3 times) 67580 KiB/s 3540 KiB/s 61964 KiB/s 9823 KiB/s
V9(Avg of 3 times) 68954 KiB/s 3567 KiB/s 60654 KiB/s 9858 KiB/s
Performance +2.0% +0.7% -2.1% +0.3%
Quoted 4 lines · Gui Jianfeng
Hi Vivek,
Here is the vanilla kernel and V9 comparision.
Quoted 8 lines
Normal Read | Random Read | Normal Write | Random Write
vanilla(Avg of 3 times) 67580 KiB/s 3540 KiB/s 61964 KiB/s 9823 KiB/s
V9(Avg of 3 times) 68954 KiB/s 3567 KiB/s 60654 KiB/s 9858 KiB/s
Performance +2.0% +0.7% -2.1% +0.3%
Thanks Gui. So that's lot of improvements from previous versions. I remember you were noticing 7% regressions in buffered write performance.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · Gui Jianfeng
Hi Vivek,
Here is the vanilla kernel and V9 comparision.
Quoted 8 lines
Normal Read | Random Read | Normal Write | Random Write
vanilla(Avg of 3 times) 67580 KiB/s 3540 KiB/s 61964 KiB/s 9823 KiB/s
V9(Avg of 3 times) 68954 KiB/s 3567 KiB/s 60654 KiB/s 9858 KiB/s
Performance +2.0% +0.7% -2.1% +0.3%
Thanks Gui. So that's lot of improvements from previous versions. I remember you were noticing 7% regressions in buffered write performance.
Thanks
Vivek
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
===========
#!/bin/sh
mkdir /cgroup
mount -t cgroup -o io,blkio io /cgroup
echo anticipatory > /sys/block/hdb/queue/scheduler
mkdir /cgroup/test1
echo 100 > /cgroup/test1/io.weight
mkdir /cgroup/test2
echo 400 > /cgroup/test2/io.weight
mkdir /cgroup/test2/test3
echo 400 > /cgroup/test2/test3/io.weight
mkdir /cgroup/test2/test4
echo 400 > /cgroup/test2/test4/io.weight
#./rwio -r -f /hdb2/2000M.3 &
dd if=/hdb2/2000M.3 of=/dev/null &
pid4=$!
echo $pid4 > /cgroup/test2/test3/tasks
echo "pid4: $pid4"
#./rwio -r -f /hdb2/2000M.1 &
dd if=/hdb2/2000M.1 of=/dev/null &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
echo "pid1 $pid1"
#./rwio -r -f /hdb2/2000M.2 &
dd if=/hdb2/2000M.2 of=/dev/null &
pid2=$!
echo $pid2 > /cgroup/test2/test4/tasks
echo "pid2 $pid2"
sleep 20
for ((;1;))
{
ps -p $pid1 > /dev/null 2>&1
if [ $? -ne 0 ]; then
break
fi
kill -9 $pid1 > /dev/null 2>&1
}
for ((;1;))
{
ps -p $pid2 > /dev/null 2>&1
if [ $? -ne 0 ]; then
break
fi
kill -9 $pid2 > /dev/null 2>&1
}kill -9 $pid4 > /dev/null 2>&1
rmdir /cgroup/test2/test3
rmdir /cgroup/test2/test4
rmdir /cgroup/test2
rmdir /cgroup/test1
umount /cgroup
rmdir /cgroup
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Thanks for the testing it out Gui. I will run this test case on my machine and see if I can reproduce this issue on my box and try to fix it.
Is your box completely hung or IO scheduler don't seem to be doing anything. Can you try to swith the io scheduler to something else (after it appears to be hung), and see if switch is successful and new scheduler starts working?
Thanks
Vivek
Quoted 69 lines
===========
#!/bin/shmkdir /cgroup
mount -t cgroup -o io,blkio io /cgroupecho anticipatory >
/sys/block/hdb/queue/schedulermkdir /cgroup/test1
echo 100 > /cgroup/test1/io.weightmkdir /cgroup/test2
echo 400 > /cgroup/test2/io.weightmkdir /cgroup/test2/test3
echo 400 > /cgroup/test2/test3/io.weightmkdir /cgroup/test2/test4
echo 400 > /cgroup/test2/test4/io.weight#./rwio -r -f /hdb2/2000M.3 &
dd if=/hdb2/2000M.3 of=/dev/null &
pid4=$!
echo $pid4 > /cgroup/test2/test3/tasks
echo "pid4: $pid4"#./rwio -r -f /hdb2/2000M.1 &
dd if=/hdb2/2000M.1 of=/dev/null &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
echo "pid1 $pid1"#./rwio -r -f /hdb2/2000M.2 &
dd if=/hdb2/2000M.2 of=/dev/null &
pid2=$!
echo $pid2 > /cgroup/test2/test4/tasks
echo "pid2 $pid2"sleep 20
for ((;1;))
{ ps -p $pid1 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid1 > /dev/null 2>&1 } for ((;1;)) { ps -p $pid2 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid2 > /dev/null 2>&1 }kill -9 $pid4 >
/dev/null2>&1rmdir /cgroup/test2/test3
rmdir /cgroup/test2/test4
rmdir /cgroup/test2
rmdir /cgroup/test1umount /cgroup
rmdir /cgroup
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Thanks for the testing it out Gui. I will run this test case on my machine and see if I can reproduce this issue on my box and try to fix it.
Is your box completely hung or IO scheduler don't seem to be doing anything. Can you try to swith the io scheduler to something else (after it appears to be hung), and see if switch is successful and new scheduler starts working?
Thanks
Vivek
Quoted 69 lines
===========
#!/bin/shmkdir /cgroup
mount -t cgroup -o io,blkio io /cgroupecho anticipatory >
/sys/block/hdb/queue/schedulermkdir /cgroup/test1
echo 100 > /cgroup/test1/io.weightmkdir /cgroup/test2
echo 400 > /cgroup/test2/io.weightmkdir /cgroup/test2/test3
echo 400 > /cgroup/test2/test3/io.weightmkdir /cgroup/test2/test4
echo 400 > /cgroup/test2/test4/io.weight#./rwio -r -f /hdb2/2000M.3 &
dd if=/hdb2/2000M.3 of=/dev/null &
pid4=$!
echo $pid4 > /cgroup/test2/test3/tasks
echo "pid4: $pid4"#./rwio -r -f /hdb2/2000M.1 &
dd if=/hdb2/2000M.1 of=/dev/null &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
echo "pid1 $pid1"#./rwio -r -f /hdb2/2000M.2 &
dd if=/hdb2/2000M.2 of=/dev/null &
pid2=$!
echo $pid2 > /cgroup/test2/test4/tasks
echo "pid2 $pid2"sleep 20
for ((;1;))
{ ps -p $pid1 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid1 > /dev/null 2>&1 } for ((;1;)) { ps -p $pid2 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid2 > /dev/null 2>&1 }kill -9 $pid4 >
/dev/null2>&1rmdir /cgroup/test2/test3
rmdir /cgroup/test2/test4
rmdir /cgroup/test2
rmdir /cgroup/test1umount /cgroup
rmdir /cgroup
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
- Does io scheduler switch on the device work
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
Thanks
Vivek
Quoted 69 lines
===========
#!/bin/shmkdir /cgroup
mount -t cgroup -o io,blkio io /cgroupecho anticipatory >
/sys/block/hdb/queue/schedulermkdir /cgroup/test1
echo 100 > /cgroup/test1/io.weightmkdir /cgroup/test2
echo 400 > /cgroup/test2/io.weightmkdir /cgroup/test2/test3
echo 400 > /cgroup/test2/test3/io.weightmkdir /cgroup/test2/test4
echo 400 > /cgroup/test2/test4/io.weight#./rwio -r -f /hdb2/2000M.3 &
dd if=/hdb2/2000M.3 of=/dev/null &
pid4=$!
echo $pid4 > /cgroup/test2/test3/tasks
echo "pid4: $pid4"#./rwio -r -f /hdb2/2000M.1 &
dd if=/hdb2/2000M.1 of=/dev/null &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
echo "pid1 $pid1"#./rwio -r -f /hdb2/2000M.2 &
dd if=/hdb2/2000M.2 of=/dev/null &
pid2=$!
echo $pid2 > /cgroup/test2/test4/tasks
echo "pid2 $pid2"sleep 20
for ((;1;))
{ ps -p $pid1 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid1 > /dev/null 2>&1 } for ((;1;)) { ps -p $pid2 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid2 > /dev/null 2>&1 }kill -9 $pid4 >
/dev/null2>&1rmdir /cgroup/test2/test3
rmdir /cgroup/test2/test4
rmdir /cgroup/test2
rmdir /cgroup/test1umount /cgroup
rmdir /cgroup
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
- Does io scheduler switch on the device work
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
Thanks
Vivek
Quoted 69 lines
===========
#!/bin/shmkdir /cgroup
mount -t cgroup -o io,blkio io /cgroupecho anticipatory >
/sys/block/hdb/queue/schedulermkdir /cgroup/test1
echo 100 > /cgroup/test1/io.weightmkdir /cgroup/test2
echo 400 > /cgroup/test2/io.weightmkdir /cgroup/test2/test3
echo 400 > /cgroup/test2/test3/io.weightmkdir /cgroup/test2/test4
echo 400 > /cgroup/test2/test4/io.weight#./rwio -r -f /hdb2/2000M.3 &
dd if=/hdb2/2000M.3 of=/dev/null &
pid4=$!
echo $pid4 > /cgroup/test2/test3/tasks
echo "pid4: $pid4"#./rwio -r -f /hdb2/2000M.1 &
dd if=/hdb2/2000M.1 of=/dev/null &
pid1=$!
echo $pid1 > /cgroup/test1/tasks
echo "pid1 $pid1"#./rwio -r -f /hdb2/2000M.2 &
dd if=/hdb2/2000M.2 of=/dev/null &
pid2=$!
echo $pid2 > /cgroup/test2/test4/tasks
echo "pid2 $pid2"sleep 20
for ((;1;))
{ ps -p $pid1 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid1 > /dev/null 2>&1 } for ((;1;)) { ps -p $pid2 > /dev/null 2>&1 if [ $? -ne 0 ]; then break fi kill -9 $pid2 > /dev/null 2>&1 }kill -9 $pid4 >
/dev/null2>&1rmdir /cgroup/test2/test3
rmdir /cgroup/test2/test4
rmdir /cgroup/test2
rmdir /cgroup/test1umount /cgroup
rmdir /cgroup
Quoted 14 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing getdents() syscall.
kernel generates the following message:
INFO: task find:3260 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
find D a1e95787 1912 3260 2897 0x00000004
f6af2db8 00000096 f660075c a1e95787 00000032 f6600270 f6600508 c2037820 00000000 c09e0820 f655f0c0 f6af2d8c fffebbf1 00000000 c0447323 f7152a1c 0006a144 f7152a1c 0006a144 f6af2e04 f6af2db0 c04438df c2037820 c2037820
Call Trace:
[<c0447323>] ? getnstimeofday+0x57/0xe0
[<c04438df>] ? ktime_get_ts+0x4a/0x4e
[<c068ab68>] io_schedule+0x47/0x79
[<c04c12ee>] sync_buffer+0x36/0x3a
[<c068ae14>] __wait_on_bit+0x36/0x5d
[<c04c12b8>] ? sync_buffer+0x0/0x3a
[<c068ae93>] out_of_line_wait_on_bit+0x58/0x60
[<c04c12b8>] ? sync_buffer+0x0/0x3a
[<c0440fa4>] ? wake_bit_function+0x0/0x43
[<c04c1249>] __wait_on_buffer+0x19/0x1c
[<f81e4186>] ext3_bread+0x5e/0x79 [ext3]
[<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3]
[<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3]
[<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3
[<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f
[<c044c45b>] ? trace_hardirqs_on+0xb/0xd
[<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3]
[<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3]
[<c04b1100>] ? filldir64+0x0/0xcd
[<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5
[<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5
[<c04b12db>] ? vfs_readdir+0x46/0x94
[<c04b12fd>] vfs_readdir+0x68/0x94
[<c04b1100>] ? filldir64+0x0/0xcd
[<c04b1387>] sys_getdents64+0x5e/0x9f
[<c04028b4>] sysenter_do_call+0x12/0x32
1 lock held by find/3260:
#0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94ext3 calls wait_on_buffer() to wait buffer, and schedule the task out in TASK_UNINTERRUPTIBLE state, and I found this task will be resumed after a quite long period(more than 10 mins).
Signature · 2 lines
Regards Gui Jianfeng
Quoted 73 lines · Gui Jianfeng
Quoted 14 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94
Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).
Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Thanks
Vivek
Quoted 73 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).
Quoted 6 lines
Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Ok, I'll try.
Quoted 3 lines
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
yes, i used the old ide driver. So I switch to libata instead(deactivate the whole “ATA/ATAPI/MFM/RLL support”), the bug still exists and I can reproduce it by that script.
Quoted 6 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 73 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).
Quoted 6 lines
Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Ok, I'll try.
Quoted 3 lines
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
yes, i used the old ide driver. So I switch to libata instead(deactivate the whole “ATA/ATAPI/MFM/RLL support”), the bug still exists and I can reproduce it by that script.
Quoted 6 lines
Thanks
Vivek
Signature · 7 lines
Regards Gui Jianfeng _______________________________________________ Containers mailing list [email protected] https://lists.linux-foundation.org/mailman/listinfo/containers
PATCH 1/1 io-controller: Fix task hanging when there are more than one groups
In reply to a message not in the archiveQuoted 77 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
--- block/elevator-fq.c | 13 +++++++------ 1 files changed, 7 insertions(+), 6 deletions(-)
block/elevator-fq.c +7 -6
@@ -2338,9 +2338,10 @@ void elv_reset_request_ioq(struct request_queue *q, struct request *rq) } } -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) { -if (list_empty (&io_root_cgroup.css.cgroup->children)) +if (list_empty (&io_root_cgroup.css.cgroup->children) && + efqd->busy_queues ==1 && efqd->root_group->ioq)return 1 ;return 0 ;
@@ -2383,7 +2384,7 @@ static void io_free_root_group(struct elevator_queue *e)int elv_iog_should_idle (struct io_queue *ioq) {return 0 ; }EXPORT_SYMBOL (elv_iog_should_idle); -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) {return 1 ; }
@@ -2547,7 +2548,7 @@ elv_iosched_expire_ioq(struct request_queue *q, int slice_expired, int force)struct elevator_queue *e = q->elevator;struct io_queue *ioq =elv_active_ioq (q->elevator);int ret =1 ; - +if (e->ops->elevator_expire_ioq_fn) { ret = e->ops->elevator_expire_ioq_fn (q, ioq->sched_queue, slice_expired, force);
@@ -2969,7 +2970,7 @@ void *elv_select_ioq(struct request_queue *q, int force)* single queue ioschedulers (noop, deadline, AS). */ -if (is_only_root_group () &&elv_iosched_single_ioq (q->elevator)) +if (is_only_root_group (efqd) &&elv_iosched_single_ioq (q->elevator)) goto keep_queue;/* We are waiting for this group to become busy before it expires.*/
@@ -3180,7 +3181,7 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq)* unnecessary overhead. */ -if (is_only_root_group () && +if (is_only_root_group (ioq->efqd) &&elv_iosched_single_ioq (q->elevator)) {elv_log_ioq (efqd, ioq,"select: only root group," " no expiry" );
Signature · 1 line
1.5.4.rc3
PATCH 1/1 io-controller: Fix task hanging when there are more than one groups
In reply to Vivek GoyalQuoted 77 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
--- block/elevator-fq.c | 13 +++++++------ 1 files changed, 7 insertions(+), 6 deletions(-)
block/elevator-fq.c +7 -6
@@ -2338,9 +2338,10 @@ void elv_reset_request_ioq(struct request_queue *q, struct request *rq) } } -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) { -if (list_empty (&io_root_cgroup.css.cgroup->children)) +if (list_empty (&io_root_cgroup.css.cgroup->children) && + efqd->busy_queues ==1 && efqd->root_group->ioq)return 1 ;return 0 ;
@@ -2383,7 +2384,7 @@ static void io_free_root_group(struct elevator_queue *e)int elv_iog_should_idle (struct io_queue *ioq) {return 0 ; }EXPORT_SYMBOL (elv_iog_should_idle); -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) {return 1 ; }
@@ -2547,7 +2548,7 @@ elv_iosched_expire_ioq(struct request_queue *q, int slice_expired, int force)struct elevator_queue *e = q->elevator;struct io_queue *ioq =elv_active_ioq (q->elevator);int ret =1 ; - +if (e->ops->elevator_expire_ioq_fn) { ret = e->ops->elevator_expire_ioq_fn (q, ioq->sched_queue, slice_expired, force);
@@ -2969,7 +2970,7 @@ void *elv_select_ioq(struct request_queue *q, int force)* single queue ioschedulers (noop, deadline, AS). */ -if (is_only_root_group () &&elv_iosched_single_ioq (q->elevator)) +if (is_only_root_group (efqd) &&elv_iosched_single_ioq (q->elevator)) goto keep_queue;/* We are waiting for this group to become busy before it expires.*/
@@ -3180,7 +3181,7 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq)* unnecessary overhead. */ -if (is_only_root_group () && +if (is_only_root_group (ioq->efqd) &&elv_iosched_single_ioq (q->elevator)) {elv_log_ioq (efqd, ioq,"select: only root group," " no expiry" );
Signature · 1 line
1.5.4.rc3
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
On Fri, Sep 11, 2009 at 09:15:42AM +0800, Gui Jianfeng wrote:
[..]
Quoted 11 lines
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Signed-off-by Gui Jianfeng <[email protected]>
Cool. Good catch Gui. Queued for next posting.
Thanks
Vivek
Quoted 64 lines
--- block/elevator-fq.c | 13 +++++++------ 1 files changed, 7 insertions(+), 6 deletions(-)block/elevator-fq.c +7
-6@@ -2338,9 +2338,10 @@ void elv_reset_request_ioq(struct request_queue *q, struct request *rq) } } -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) { -if (list_empty (&io_root_cgroup.css.cgroup->children)) +if (list_empty (&io_root_cgroup.css.cgroup->children) && + efqd->busy_queues ==1 && efqd->root_group->ioq)return 1 ;return 0 ;@@ -2383,7 +2384,7 @@ static void io_free_root_group(struct elevator_queue *e)int elv_iog_should_idle (struct io_queue *ioq) {return 0 ; }EXPORT_SYMBOL (elv_iog_should_idle); -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) {return 1 ; }@@ -2547,7 +2548,7 @@ elv_iosched_expire_ioq(struct request_queue *q, int slice_expired, int force)struct elevator_queue *e = q->elevator;struct io_queue *ioq =elv_active_ioq (q->elevator);int ret =1 ; - +if (e->ops->elevator_expire_ioq_fn) { ret = e->ops->elevator_expire_ioq_fn (q, ioq->sched_queue, slice_expired, force);@@ -2969,7 +2970,7 @@ void *elv_select_ioq(struct request_queue *q, int force)* single queue ioschedulers (noop, deadline, AS). */ -if (is_only_root_group () &&elv_iosched_single_ioq (q->elevator)) +if (is_only_root_group (efqd) &&elv_iosched_single_ioq (q->elevator)) goto keep_queue;/* We are waiting for this group to become busy before it expires.*/ @@ -3180,7 +3181,7 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq)* unnecessary overhead. */ -if (is_only_root_group () && +if (is_only_root_group (ioq->efqd) &&elv_iosched_single_ioq (q->elevator)) {elv_log_ioq (efqd, ioq,"select: only root group," " no expiry" );Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
On Fri, Sep 11, 2009 at 09:15:42AM +0800, Gui Jianfeng wrote:
[..]
Quoted 11 lines
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Signed-off-by Gui Jianfeng <[email protected]>
Cool. Good catch Gui. Queued for next posting.
Thanks
Vivek
Quoted 64 lines
--- block/elevator-fq.c | 13 +++++++------ 1 files changed, 7 insertions(+), 6 deletions(-)block/elevator-fq.c +7
-6@@ -2338,9 +2338,10 @@ void elv_reset_request_ioq(struct request_queue *q, struct request *rq) } } -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) { -if (list_empty (&io_root_cgroup.css.cgroup->children)) +if (list_empty (&io_root_cgroup.css.cgroup->children) && + efqd->busy_queues ==1 && efqd->root_group->ioq)return 1 ;return 0 ;@@ -2383,7 +2384,7 @@ static void io_free_root_group(struct elevator_queue *e)int elv_iog_should_idle (struct io_queue *ioq) {return 0 ; }EXPORT_SYMBOL (elv_iog_should_idle); -static inline int is_only_root_group (void ) +static inline int is_only_root_group (struct elv_fq_data *efqd) {return 1 ; }@@ -2547,7 +2548,7 @@ elv_iosched_expire_ioq(struct request_queue *q, int slice_expired, int force)struct elevator_queue *e = q->elevator;struct io_queue *ioq =elv_active_ioq (q->elevator);int ret =1 ; - +if (e->ops->elevator_expire_ioq_fn) { ret = e->ops->elevator_expire_ioq_fn (q, ioq->sched_queue, slice_expired, force);@@ -2969,7 +2970,7 @@ void *elv_select_ioq(struct request_queue *q, int force)* single queue ioschedulers (noop, deadline, AS). */ -if (is_only_root_group () &&elv_iosched_single_ioq (q->elevator)) +if (is_only_root_group (efqd) &&elv_iosched_single_ioq (q->elevator)) goto keep_queue;/* We are waiting for this group to become busy before it expires.*/ @@ -3180,7 +3181,7 @@ void elv_ioq_completed_request(struct request_queue *q, struct request *rq)* unnecessary overhead. */ -if (is_only_root_group () && +if (is_only_root_group (ioq->efqd) &&elv_iosched_single_ioq (q->elevator)) {elv_log_ioq (efqd, ioq,"select: only root group," " no expiry" );Signature · 1 line
1.5.4.rc3
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
In reply to a message not in the archiveQuoted 95 lines · Gui Jianfeng
Quoted 77 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>
Hi Gui,
I have modified your patch a bit to improve readability. Looking at the issue closely I realized that this optimization of not expiring the queue can lead to other issues like high vdisktime in certain scenarios. While fixing that also noticed the issue of high rate of as queue expiration in certain cases which could have been avoided.
Here is a patch which should fix all that. I am still testing this patch to make sure that something is not obiviously broken. Will merge it if there are no issues.
Thanks
Vivek
o Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.
o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.
- elevator layer does not know that As is anticipating hence initiates expiry requests in select_ioq() thinking queue is empty.
- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts ANITC_WAIT_NEXT anticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.
o Moved some of the code in separate functions to improve readability.
--- block/as-iosched.c | 93 +++++++++++++++++++++++++++++--- block/elevator-fq.c | 150 +++++++++++++++++++++++++++++++++++++++++----------- block/elevator-fq.h | 3 + 3 files changed, 210 insertions(+), 36 deletions(-)
block/elevator-fq.c +121 -29
@@ -472,19 +472,18 @@ static inline void debug_entity_vdisktim unsigned long served, u64 delta) {}#endif /* DEBUG_ELV_FAIR_QUEUING */ -static void -entity_served (struct io_entity *entity,unsigned long served, -unsigned long nr_sectors) +static void entity_served (struct io_entity *entity,unsigned long real_served, +unsigned long virtual_served,unsigned long nr_sectors) {for_each_entity (entity) {u64 delta; - delta =elv_delta_fair (served, entity); + delta =elv_delta_fair (virtual_served, entity); entity->vdisktime += delta;update_min_vdisktime (entity->st); - entity->total_time += served; + entity->total_time += real_served; entity->total_sectors += nr_sectors; -debug_entity_vdisktime (entity, served, delta); +debug_entity_vdisktime (entity, virtual_served, delta); } }
@@ -928,7 +927,24 @@ EXPORT_SYMBOL(elv_put_ioq);static void elv_ioq_served (struct io_queue *ioq,unsigned long served) { -entity_served (&ioq->entity, served, ioq->nr_sectors); + +unsigned long virtual_served = served, allocated_slice; + +/* +* For single ioq schedulers we don't expire the queue if there are +* no other competing groups. It might happen that once a queue has +* not been expired for a long time, suddenly a new group is created +* and IO comes in that new group. In that case, we don't want to +* charge the old queue for whole of the period it was not expired. +*/ +if (elv_ioq_charge_one_slice (ioq)) { + allocated_slice =elv_prio_to_slice (ioq->efqd, ioq); +if (served > allocated_slice) + virtual_served = allocated_slice; +elv_clear_ioq_charge_one_slice (ioq); + } + +entity_served (&ioq->entity, served, virtual_served, ioq->nr_sectors);elv_log_ioq (ioq->efqd, ioq,"ioq served: QSt=%lu QSs=%lu qued=%lu" , served, ioq->nr_sectors, ioq->nr_queued);print_ioq_service_stats (ioq);
@@ -2543,6 +2559,22 @@ alloc_sched_q:elv_init_ioq_io_group (ioq, iog);elv_init_ioq_sched_queue (e, ioq, sched_q); +/* +* For AS, also mark the group queue idle_window. This will +* make sure that select_ioq() will not try to expire an +* AS queue if there are dispatched request from the queue but +* queue is empty. This gives a chance to asq to anticipate +* after request completion, otherwise select_ioq() will +* mark it must_expire and soon asq will be expired. +* +* Not doing it for noop and deadline yet as they don't have +* any anticpation logic and this will slow down queue +* switching in a NCQ supporting hardware. +*/ +if (!strcmp (e->elevator_type->elevator_name,"anticipatory" )) { +elv_mark_ioq_idle_window (ioq); + } +elv_io_group_set_ioq (iog, ioq);elv_mark_ioq_sync (ioq);elv_get_iog (iog);
@@ -2664,6 +2696,12 @@ static inline int is_only_root_group(voi#endif /* CONFIG_GROUP_IOSCHED */ +static inline int ioq_is_idling (struct io_queue *ioq) +{ +return (elv_ioq_wait_request (ioq) || +timer_pending (&ioq->efqd->idle_slice_timer)); +} +/* * Should be called after ioq prio and class has been initialized as prio * class data will be used to determine which service tree in the group
@@ -2835,7 +2873,6 @@ elv_iosched_expire_ioq(struct request_quif (!ret)elv_mark_ioq_must_expire (ioq); } -return ret; }
@@ -3078,6 +3115,7 @@ void elv_ioq_request_add(struct request_*/ if (group_wait_req ||elv_ioq_wait_request (ioq)) {del_timer (&efqd->idle_slice_timer); +elv_clear_ioq_wait_request (ioq);if (blk_rq_bytes (rq) >PAGE_CACHE_SIZE || efqd->busy_queues >1 || !blk_queue_plugged (q))__blk_run_queue (q);
@@ -3121,6 +3159,7 @@ static void elv_idle_slice_timer(unsigneif (ioq) {struct io_group *iog =ioq_to_io_group (ioq); +elv_clear_ioq_wait_request (ioq);elv_clear_iog_wait_request (iog);if (elv_iog_wait_busy (iog)) {
@@ -3222,6 +3261,28 @@ static inline struct io_queue *elv_closereturn new_ioq; } +/* +* One can do some optimizations for single ioq scheduler, when one does +* not have to expire the queue after every time slice is used. This avoids +* some unnecessary overhead, especially in AS where we wait for requests to +* finish from last queue before new queue is scheduled in. +*/ +static inline int single_ioq_no_timed_expiry (struct request_queue *q) +{ +struct elv_fq_data *efqd = q->elevator->efqd; + +if (!elv_iosched_single_ioq (q->elevator)) +return 0 ; + +if (!is_only_root_group ()) +return 0 ; + +if (efqd->busy_queues ==1 ) +return 1 ; + +return 0 ; +} +/* Common layer function to select the next queue to dispatch from */ void *elv_select_ioq (struct request_queue *q,int force) {
@@ -3229,7 +3290,7 @@ void *elv_select_ioq(struct request_queustruct io_queue *new_ioq =NULL , *ioq =elv_active_ioq (q->elevator);struct io_group *iog;struct elevator_type *e = q->elevator->elevator_type; -int slice_expired =1 ; +int slice_expired =0 ;if (!elv_nr_busy_ioq (q->elevator))return NULL ;
@@ -3255,16 +3316,20 @@ void *elv_select_ioq(struct request_queu }/* This queue has been marked for expiry. Try to expire it */ -if (elv_ioq_must_expire (ioq)) +if (elv_ioq_must_expire (ioq)) { +elv_log_ioq (efqd, ioq,"select: ioq must_expire. expire" ); goto expire; + }/* * If there is only root group present, don't expire the queue for * single queue ioschedulers (noop, deadline, AS). */ -if (is_only_root_group () &&elv_iosched_single_ioq (q->elevator)) +if (single_ioq_no_timed_expiry (q)) { +elv_mark_ioq_charge_one_slice (ioq); goto keep_queue; + }/* We are waiting for this group to become busy before it expires.*/ if (elv_iog_wait_busy (iog)) {
@@ -3301,6 +3366,7 @@ void *elv_select_ioq(struct request_queu* from queue and is not proportional to group's weight, it * harms the fairness of the group. */ + slice_expired =1 ;if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog)) { ioq =NULL ; goto keep_queue;
@@ -3332,7 +3398,7 @@ void *elv_select_ioq(struct request_queu* conditions to happen (or time out) before selecting a new queue. */ -if (timer_pending (&efqd->idle_slice_timer) || +if (ioq_is_idling (ioq) || (elv_ioq_nr_dispatched (ioq) &&elv_ioq_idle_window (ioq))) { ioq =NULL ; goto keep_queue;
@@ -3344,7 +3410,6 @@ void *elv_select_ioq(struct request_queu goto keep_queue; } - slice_expired =0 ; expire:if (efqd->fairness && !force && ioq && ioq->dispatched &&strcmp (e->elevator_name,"anticipatory" )) {
@@ -3439,6 +3504,43 @@ void elv_deactivate_rq_fair(struct reque efqd->rq_in_driver); } +/* +* if this is only queue and it has completed all its requests and has nothing +* to dispatch, expire it. We don't want to keep it around idle otherwise later +* when it is expired, all this idle time will be added to queue's disk time +* used and queue might not get a chance to run for a long time. +*/ +static inline void +check_expire_last_empty_queue (struct request_queue *q,struct io_queue *ioq) +{ +struct elv_fq_data *efqd = q->elevator->efqd; + +if (efqd->busy_queues !=1 ) +return ; + +if (ioq->dispatched || ioq->nr_queued) +return ; + +/* +* Anticipation is on. Don't expire queue. Either a new request will +* come or it is up to io scheduler to expire the queue once idle +* timer fires +*/ + +if (ioq_is_idling (ioq)) +return ; + +/* +* If IO scheduler denies expiration here, it is up to io scheduler +* to expire the queue when possible. Otherwise all the idle time +* will be charged to the queue when queue finally expires. +*/ +if (elv_iosched_expire_ioq (q,0 ,0 )) { +elv_log_ioq (efqd, ioq,"expire last empty queue" ); +elv_slice_expired (q); + } +} +/* A request got completed from io_queue. Do the accounting. */ void elv_ioq_completed_request (struct request_queue *q,struct request *rq) {
@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/* * If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) { +elv_mark_ioq_charge_one_slice (ioq); +elv_log_ioq (efqd, ioq,"single ioq no timed expiry" ); goto done; }
@@ -3519,7 +3621,7 @@ void elv_ioq_completed_request(struct re* decide to idle on queue, idle on group. */ if (elv_iog_should_idle (ioq) && !elv_ioq_nr_dispatched (ioq) - && !timer_pending (&efqd->idle_slice_timer)) { + && !ioq_is_idling (ioq)) {/* * If queue has used up its slice, wait for the * one extra group_idle period to let the group
@@ -3532,17 +3634,7 @@ void elv_ioq_completed_request(struct reelv_iog_arm_slice_timer (q, iog,0 ); } -/* -* if this is only queue and it has completed all its requests -* and has nothing to dispatch, expire it. We don't want to -* keep it around idle otherwise later when it is expired, all -* this idle time will be added to queue's disk time used. -*/ -if (efqd->busy_queues ==1 && !ioq->dispatched && - !ioq->nr_queued && !timer_pending (&efqd->idle_slice_timer)) { -if (elv_iosched_expire_ioq (q,0 ,0 )) -elv_slice_expired (q); - } +check_expire_last_empty_queue (q, ioq); } done:if (!efqd->rq_in_driver)
block/as-iosched.c +86 -7
162 lines not on this page · raw message
block/elevator-fq.h +3 -0
@@ -264,6 +264,8 @@ enum elv_queue_state_flags { ELV_QUEUE_FLAG_slice_new,/* no requests dispatched in slice */ ELV_QUEUE_FLAG_sync,/* synchronous queue */ ELV_QUEUE_FLAG_must_expire,/* expire queue even slice is left */ + ELV_QUEUE_FLAG_charge_one_slice, /* Charge the queue for only one + * time slice length */ };#define ELV_IO_QUEUE_FLAG_FNS (name) \
@@ -287,6 +289,7 @@ ELV_IO_QUEUE_FLAG_FNS(idle_window) ELV_IO_QUEUE_FLAG_FNS(slice_new) ELV_IO_QUEUE_FLAG_FNS(sync) ELV_IO_QUEUE_FLAG_FNS(must_expire) +ELV_IO_QUEUE_FLAG_FNS(charge_one_slice)#ifdef CONFIG_GROUP_IOSCHED
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
In reply to Gui JianfengQuoted 95 lines · Gui Jianfeng
Quoted 77 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>
Hi Gui,
I have modified your patch a bit to improve readability. Looking at the issue closely I realized that this optimization of not expiring the queue can lead to other issues like high vdisktime in certain scenarios. While fixing that also noticed the issue of high rate of as queue expiration in certain cases which could have been avoided.
Here is a patch which should fix all that. I am still testing this patch to make sure that something is not obiviously broken. Will merge it if there are no issues.
Thanks
Vivek
o Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.
o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.
- elevator layer does not know that As is anticipating hence initiates expiry requests in select_ioq() thinking queue is empty.
- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts ANITC_WAIT_NEXT anticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.
o Moved some of the code in separate functions to improve readability.
--- block/as-iosched.c | 93 +++++++++++++++++++++++++++++--- block/elevator-fq.c | 150 +++++++++++++++++++++++++++++++++++++++++----------- block/elevator-fq.h | 3 + 3 files changed, 210 insertions(+), 36 deletions(-)
block/elevator-fq.c +121 -29
@@ -472,19 +472,18 @@ static inline void debug_entity_vdisktim unsigned long served, u64 delta) {}#endif /* DEBUG_ELV_FAIR_QUEUING */ -static void -entity_served (struct io_entity *entity,unsigned long served, -unsigned long nr_sectors) +static void entity_served (struct io_entity *entity,unsigned long real_served, +unsigned long virtual_served,unsigned long nr_sectors) {for_each_entity (entity) {u64 delta; - delta =elv_delta_fair (served, entity); + delta =elv_delta_fair (virtual_served, entity); entity->vdisktime += delta;update_min_vdisktime (entity->st); - entity->total_time += served; + entity->total_time += real_served; entity->total_sectors += nr_sectors; -debug_entity_vdisktime (entity, served, delta); +debug_entity_vdisktime (entity, virtual_served, delta); } }
@@ -928,7 +927,24 @@ EXPORT_SYMBOL(elv_put_ioq);static void elv_ioq_served (struct io_queue *ioq,unsigned long served) { -entity_served (&ioq->entity, served, ioq->nr_sectors); + +unsigned long virtual_served = served, allocated_slice; + +/* +* For single ioq schedulers we don't expire the queue if there are +* no other competing groups. It might happen that once a queue has +* not been expired for a long time, suddenly a new group is created +* and IO comes in that new group. In that case, we don't want to +* charge the old queue for whole of the period it was not expired. +*/ +if (elv_ioq_charge_one_slice (ioq)) { + allocated_slice =elv_prio_to_slice (ioq->efqd, ioq); +if (served > allocated_slice) + virtual_served = allocated_slice; +elv_clear_ioq_charge_one_slice (ioq); + } + +entity_served (&ioq->entity, served, virtual_served, ioq->nr_sectors);elv_log_ioq (ioq->efqd, ioq,"ioq served: QSt=%lu QSs=%lu qued=%lu" , served, ioq->nr_sectors, ioq->nr_queued);print_ioq_service_stats (ioq);
@@ -2543,6 +2559,22 @@ alloc_sched_q:elv_init_ioq_io_group (ioq, iog);elv_init_ioq_sched_queue (e, ioq, sched_q); +/* +* For AS, also mark the group queue idle_window. This will +* make sure that select_ioq() will not try to expire an +* AS queue if there are dispatched request from the queue but +* queue is empty. This gives a chance to asq to anticipate +* after request completion, otherwise select_ioq() will +* mark it must_expire and soon asq will be expired. +* +* Not doing it for noop and deadline yet as they don't have +* any anticpation logic and this will slow down queue +* switching in a NCQ supporting hardware. +*/ +if (!strcmp (e->elevator_type->elevator_name,"anticipatory" )) { +elv_mark_ioq_idle_window (ioq); + } +elv_io_group_set_ioq (iog, ioq);elv_mark_ioq_sync (ioq);elv_get_iog (iog);
@@ -2664,6 +2696,12 @@ static inline int is_only_root_group(voi#endif /* CONFIG_GROUP_IOSCHED */ +static inline int ioq_is_idling (struct io_queue *ioq) +{ +return (elv_ioq_wait_request (ioq) || +timer_pending (&ioq->efqd->idle_slice_timer)); +} +/* * Should be called after ioq prio and class has been initialized as prio * class data will be used to determine which service tree in the group
@@ -2835,7 +2873,6 @@ elv_iosched_expire_ioq(struct request_quif (!ret)elv_mark_ioq_must_expire (ioq); } -return ret; }
@@ -3078,6 +3115,7 @@ void elv_ioq_request_add(struct request_*/ if (group_wait_req ||elv_ioq_wait_request (ioq)) {del_timer (&efqd->idle_slice_timer); +elv_clear_ioq_wait_request (ioq);if (blk_rq_bytes (rq) >PAGE_CACHE_SIZE || efqd->busy_queues >1 || !blk_queue_plugged (q))__blk_run_queue (q);
@@ -3121,6 +3159,7 @@ static void elv_idle_slice_timer(unsigneif (ioq) {struct io_group *iog =ioq_to_io_group (ioq); +elv_clear_ioq_wait_request (ioq);elv_clear_iog_wait_request (iog);if (elv_iog_wait_busy (iog)) {
@@ -3222,6 +3261,28 @@ static inline struct io_queue *elv_closereturn new_ioq; } +/* +* One can do some optimizations for single ioq scheduler, when one does +* not have to expire the queue after every time slice is used. This avoids +* some unnecessary overhead, especially in AS where we wait for requests to +* finish from last queue before new queue is scheduled in. +*/ +static inline int single_ioq_no_timed_expiry (struct request_queue *q) +{ +struct elv_fq_data *efqd = q->elevator->efqd; + +if (!elv_iosched_single_ioq (q->elevator)) +return 0 ; + +if (!is_only_root_group ()) +return 0 ; + +if (efqd->busy_queues ==1 ) +return 1 ; + +return 0 ; +} +/* Common layer function to select the next queue to dispatch from */ void *elv_select_ioq (struct request_queue *q,int force) {
@@ -3229,7 +3290,7 @@ void *elv_select_ioq(struct request_queustruct io_queue *new_ioq =NULL , *ioq =elv_active_ioq (q->elevator);struct io_group *iog;struct elevator_type *e = q->elevator->elevator_type; -int slice_expired =1 ; +int slice_expired =0 ;if (!elv_nr_busy_ioq (q->elevator))return NULL ;
@@ -3255,16 +3316,20 @@ void *elv_select_ioq(struct request_queu }/* This queue has been marked for expiry. Try to expire it */ -if (elv_ioq_must_expire (ioq)) +if (elv_ioq_must_expire (ioq)) { +elv_log_ioq (efqd, ioq,"select: ioq must_expire. expire" ); goto expire; + }/* * If there is only root group present, don't expire the queue for * single queue ioschedulers (noop, deadline, AS). */ -if (is_only_root_group () &&elv_iosched_single_ioq (q->elevator)) +if (single_ioq_no_timed_expiry (q)) { +elv_mark_ioq_charge_one_slice (ioq); goto keep_queue; + }/* We are waiting for this group to become busy before it expires.*/ if (elv_iog_wait_busy (iog)) {
@@ -3301,6 +3366,7 @@ void *elv_select_ioq(struct request_queu* from queue and is not proportional to group's weight, it * harms the fairness of the group. */ + slice_expired =1 ;if (elv_iog_should_idle (ioq) && !elv_iog_wait_busy_done (iog)) { ioq =NULL ; goto keep_queue;
@@ -3332,7 +3398,7 @@ void *elv_select_ioq(struct request_queu* conditions to happen (or time out) before selecting a new queue. */ -if (timer_pending (&efqd->idle_slice_timer) || +if (ioq_is_idling (ioq) || (elv_ioq_nr_dispatched (ioq) &&elv_ioq_idle_window (ioq))) { ioq =NULL ; goto keep_queue;
@@ -3344,7 +3410,6 @@ void *elv_select_ioq(struct request_queu goto keep_queue; } - slice_expired =0 ; expire:if (efqd->fairness && !force && ioq && ioq->dispatched &&strcmp (e->elevator_name,"anticipatory" )) {
@@ -3439,6 +3504,43 @@ void elv_deactivate_rq_fair(struct reque efqd->rq_in_driver); } +/* +* if this is only queue and it has completed all its requests and has nothing +* to dispatch, expire it. We don't want to keep it around idle otherwise later +* when it is expired, all this idle time will be added to queue's disk time +* used and queue might not get a chance to run for a long time. +*/ +static inline void +check_expire_last_empty_queue (struct request_queue *q,struct io_queue *ioq) +{ +struct elv_fq_data *efqd = q->elevator->efqd; + +if (efqd->busy_queues !=1 ) +return ; + +if (ioq->dispatched || ioq->nr_queued) +return ; + +/* +* Anticipation is on. Don't expire queue. Either a new request will +* come or it is up to io scheduler to expire the queue once idle +* timer fires +*/ + +if (ioq_is_idling (ioq)) +return ; + +/* +* If IO scheduler denies expiration here, it is up to io scheduler +* to expire the queue when possible. Otherwise all the idle time +* will be charged to the queue when queue finally expires. +*/ +if (elv_iosched_expire_ioq (q,0 ,0 )) { +elv_log_ioq (efqd, ioq,"expire last empty queue" ); +elv_slice_expired (q); + } +} +/* A request got completed from io_queue. Do the accounting. */ void elv_ioq_completed_request (struct request_queue *q,struct request *rq) {
@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/* * If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) { +elv_mark_ioq_charge_one_slice (ioq); +elv_log_ioq (efqd, ioq,"single ioq no timed expiry" ); goto done; }
@@ -3519,7 +3621,7 @@ void elv_ioq_completed_request(struct re* decide to idle on queue, idle on group. */ if (elv_iog_should_idle (ioq) && !elv_ioq_nr_dispatched (ioq) - && !timer_pending (&efqd->idle_slice_timer)) { + && !ioq_is_idling (ioq)) {/* * If queue has used up its slice, wait for the * one extra group_idle period to let the group
@@ -3532,17 +3634,7 @@ void elv_ioq_completed_request(struct reelv_iog_arm_slice_timer (q, iog,0 ); } -/* -* if this is only queue and it has completed all its requests -* and has nothing to dispatch, expire it. We don't want to -* keep it around idle otherwise later when it is expired, all -* this idle time will be added to queue's disk time used. -*/ -if (efqd->busy_queues ==1 && !ioq->dispatched && - !ioq->nr_queued && !timer_pending (&efqd->idle_slice_timer)) { -if (elv_iosched_expire_ioq (q,0 ,0 )) -elv_slice_expired (q); - } +check_expire_last_empty_queue (q, ioq); } done:if (!efqd->rq_in_driver)
block/as-iosched.c +86 -7
162 lines not on this page · raw message
block/elevator-fq.h +3 -0
@@ -264,6 +264,8 @@ enum elv_queue_state_flags { ELV_QUEUE_FLAG_slice_new,/* no requests dispatched in slice */ ELV_QUEUE_FLAG_sync,/* synchronous queue */ ELV_QUEUE_FLAG_must_expire,/* expire queue even slice is left */ + ELV_QUEUE_FLAG_charge_one_slice, /* Charge the queue for only one + * time slice length */ };#define ELV_IO_QUEUE_FLAG_FNS (name) \
@@ -287,6 +289,7 @@ ELV_IO_QUEUE_FLAG_FNS(idle_window) ELV_IO_QUEUE_FLAG_FNS(slice_new) ELV_IO_QUEUE_FLAG_FNS(sync) ELV_IO_QUEUE_FLAG_FNS(must_expire) +ELV_IO_QUEUE_FLAG_FNS(charge_one_slice)#ifdef CONFIG_GROUP_IOSCHED
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
Quoted 133 lines · Vivek Goyal
Quoted 93 lines · Gui Jianfeng
Quoted 76 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>Hi Gui,
I have modified your patch a bit to improve readability. Looking at the issue closely I realized that this optimization of not expiring the queue can lead to other issues like high vdisktime in certain scenarios. While fixing that also noticed the issue of high rate of as queue expiration in certain cases which could have been avoided.
Here is a patch which should fix all that. I am still testing this patch to make sure that something is not obiviously broken. Will merge it if there are no issues.
Thanks
Viveko Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
Quoted 8 lines
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
Signed-off-by Vivek Goyal <[email protected]>
I'd like to have a try this patch :)
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
Quoted 133 lines · Vivek Goyal
Quoted 93 lines · Gui Jianfeng
Quoted 76 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>Hi Gui,
I have modified your patch a bit to improve readability. Looking at the issue closely I realized that this optimization of not expiring the queue can lead to other issues like high vdisktime in certain scenarios. While fixing that also noticed the issue of high rate of as queue expiration in certain cases which could have been avoided.
Here is a patch which should fix all that. I am still testing this patch to make sure that something is not obiviously broken. Will merge it if there are no issues.
Thanks
Viveko Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
Quoted 8 lines
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
Signed-off-by Vivek Goyal <[email protected]>
I'd like to have a try this patch :)
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
In reply to a message not in the archiveQuoted 140 lines · Vivek Goyal
Quoted 93 lines · Gui Jianfeng
Quoted 76 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>Hi Gui,
I have modified your patch a bit to improve readability. Looking at the issue closely I realized that this optimization of not expiring the queue can lead to other issues like high vdisktime in certain scenarios. While fixing that also noticed the issue of high rate of as queue expiration in certain cases which could have been avoided.
Here is a patch which should fix all that. I am still testing this patch to make sure that something is not obiviously broken. Will merge it if there are no issues.
Thanks
Viveko Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
...
Quoted 9 lines
/* A request got completed from
io_queue. Do the accounting. */ voidelv_ioq_completed_request(struct request_queue *q, struct request *rq){block/elevator-fq.c +1
-0@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/*
block/elevator-fq.c void elv_ioq_completed_request(struct re · line 3582
Quoted 10 lines
block/elevator-fq.c +1
-4* If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) {
Hi Vivek,
So we make use of single_ioq_no_timed_expiry() to decide whether there is only root ioq to be busy, right? But single_ioq_no_timed_expiry() only checks if the root cgroup is the only group and if there is only one busy_ioq there. As I explained in previous mail, these two checks are not sufficient to say the current active ioq comes from root group. Because when the child cgroup is just removed, and the ioq which belongs to child group is still there(maybe some requests are in flight). In this case, only root cgroup and only one active ioq (child ioq) checks are satisfied. So IMHO, in single_ioq_no_timed_expiry() we still need to check "efqd->root_group->ioq" is already created to ensure the only ioq comes from root group. Am i missing something?
Quoted 5 lines
block/elevator-fq.c +2
-0+elv_mark_ioq_charge_one_slice (ioq); +elv_log_ioq (efqd, ioq,"single ioq no timed expiry" ); goto done; }
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
In reply to a message not in the archiveHi Vivek,
Currently, we just set this flag when anticipating next request. So make sure we remove this flag also in this case.
--- block/as-iosched.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-)
block/as-iosched.c +3 -2
@@ -728,9 +728,10 @@ static void as_antic_stop(struct as_data *ad)as_log (ad,"as_antic_stop antic_status=%d" , ad->antic_status);if (status ==ANTIC_WAIT_REQ || status ==ANTIC_WAIT_NEXT ) { -if (status ==ANTIC_WAIT_NEXT ) +if (status ==ANTIC_WAIT_NEXT ) {del_timer (&ad->antic_timer); -as_clear_active_asq_wait_request (ad); +as_clear_active_asq_wait_request (ad); + } ad->antic_status =ANTIC_FINISHED ;/* see as_work_handler */ kblockd_schedule_work (ad->q, &ad->antic_work);
Signature · 1 line
1.5.4.rc3
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
In reply to Vivek GoyalQuoted 140 lines · Vivek Goyal
Quoted 93 lines · Gui Jianfeng
Quoted 76 lines · Vivek Goyal
Quoted 69 lines · Gui Jianfeng
Quoted 13 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Quoted 6 lines
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Hi Vivek, Jens,
Currently, If there's only the root cgroup and no other child cgroup available, io-controller will optimize to stop expiring the current ioq, and we thought the current ioq belongs to root group. But in some cases, this assumption is not true. Consider the following scenario, if there is a child cgroup located in root cgroup, and task A is running in the child cgroup, and task A issues some IOs. Then we kill task A and remove the child cgroup, at this time, there is only root cgroup available. But the ioq is still under service, and from now on, this ioq won't expire because "only root" optimization. The following patch ensures the ioq do belongs to the root group if there's only root group existing.
Quoted 1 line
Signed-off-by Gui Jianfeng <[email protected]>Hi Gui,
I have modified your patch a bit to improve readability. Looking at the issue closely I realized that this optimization of not expiring the queue can lead to other issues like high vdisktime in certain scenarios. While fixing that also noticed the issue of high rate of as queue expiration in certain cases which could have been avoided.
Here is a patch which should fix all that. I am still testing this patch to make sure that something is not obiviously broken. Will merge it if there are no issues.
Thanks
Viveko Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
...
Quoted 9 lines
/* A request got completed from
io_queue. Do the accounting. */ voidelv_ioq_completed_request(struct request_queue *q, struct request *rq){block/elevator-fq.c +1
-0@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/*
block/elevator-fq.c void elv_ioq_completed_request(struct re · line 3582
Quoted 10 lines
block/elevator-fq.c +1
-4* If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) {
Hi Vivek,
So we make use of single_ioq_no_timed_expiry() to decide whether there is only root ioq to be busy, right? But single_ioq_no_timed_expiry() only checks if the root cgroup is the only group and if there is only one busy_ioq there. As I explained in previous mail, these two checks are not sufficient to say the current active ioq comes from root group. Because when the child cgroup is just removed, and the ioq which belongs to child group is still there(maybe some requests are in flight). In this case, only root cgroup and only one active ioq (child ioq) checks are satisfied. So IMHO, in single_ioq_no_timed_expiry() we still need to check "efqd->root_group->ioq" is already created to ensure the only ioq comes from root group. Am i missing something?
Quoted 5 lines
block/elevator-fq.c +2
-0+elv_mark_ioq_charge_one_slice (ioq); +elv_log_ioq (efqd, ioq,"single ioq no timed expiry" ); goto done; }
Signature · 2 lines
Regards Gui Jianfeng
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
On Wed, Sep 16, 2009 at 10:58:10AM +0800, Gui Jianfeng wrote:
[..]
Quoted 53 lines
Quoted 29 lines
o Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
...
Quoted 9 lines
/* A request got completed from
io_queue. Do the accounting. */ voidelv_ioq_completed_request(struct request_queue *q, struct request *rq){block/elevator-fq.c +1
-0@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/* block/elevator-fq.c void elv_ioq_completed_request(struct re · line 3582
Quoted 10 lines
block/elevator-fq.c +1
-4* If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) {Hi Vivek,
Quoted 11 lines
So we make use of
single_ioq_no_timed_expiry()to decide whether there is only root ioq to be busy, right? Butsingle_ioq_no_timed_expiry()only checks if the root cgroup is the only group and if there is only onebusy_ioqthere. As I explained in previous mail, these two checks are not sufficient to say the current active ioq comes from root group. Because when the child cgroup is just removed, and the ioq which belongs to child group is still there(maybe some requests are in flight). In this case, only root cgroup and only one active ioq (child ioq) checks are satisfied. So IMHO, insingle_ioq_no_timed_expiry()we still need to check "efqd->root_group->ioq" is already created to ensure the only ioq comes from root group. Am i missing something?
Hi Gui,
The only side effect of not checking for "efqd->root_group->ioq" seems to be that this ioq hence io group of the child will not be freed immediately and release will be delayed until a some other queue in the system gets backlogged or ioscheduler exits. At that point of time, this child queue will be expired and ioq and iog will be freed.
The advantage is that if there is IO happening only in child group in that case we can avoid expiring that queue.
So may be keeping the queue around for sometime is not a very idea.
Am I missing something?
Thanks
Vivek
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
Quoted 75 lines · Vivek Goyal
On Wed, Sep 16, 2009 at 10:58:10AM +0800, Gui Jianfeng wrote:
[..]
Quoted 52 lines
Quoted 29 lines
o Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
...
Quoted 9 lines
/* A request got completed from
io_queue. Do the accounting. */ voidelv_ioq_completed_request(struct request_queue *q, struct request *rq){block/elevator-fq.c +1
-0@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/* block/elevator-fq.c void elv_ioq_completed_request(struct re · line 3582
Quoted 10 lines
block/elevator-fq.c +1
-4* If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) {Hi Vivek,
Quoted 11 lines
So we make use of
single_ioq_no_timed_expiry()to decide whether there is only root ioq to be busy, right? Butsingle_ioq_no_timed_expiry()only checks if the root cgroup is the only group and if there is only onebusy_ioqthere. As I explained in previous mail, these two checks are not sufficient to say the current active ioq comes from root group. Because when the child cgroup is just removed, and the ioq which belongs to child group is still there(maybe some requests are in flight). In this case, only root cgroup and only one active ioq (child ioq) checks are satisfied. So IMHO, insingle_ioq_no_timed_expiry()we still need to check "efqd->root_group->ioq" is already created to ensure the only ioq comes from root group. Am i missing something?Hi Gui,
The only side effect of not checking for "
efqd->root_group->ioq" seems to be that this ioq hence io group of the child will not be freed immediately and release will be delayed until a some other queue in the system gets backlogged or ioscheduler exits. At that point of time, this child queue will be expired and ioq and iog will be freed.
Quoted 6 lines
The advantage is that if there is IO happening only in child group in that case we can avoid expiring that queue.
So may be keeping the queue around for sometime is not a very idea.
Am I missing something?
Hi Vivek,
I think you're right.
A thought comes to my mind, whether can we extend optimization for not expiring an ioq not only for root group, but for single ioq case? IOW, if there is only one ioq in hierarchy, we don't exipre it until another ioq gets backlogged, and we just charge only one time slice for that ioq when it schedule out. This might help if IO only happens in child group.
Quoted 6 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
Quoted 75 lines · Vivek Goyal
On Wed, Sep 16, 2009 at 10:58:10AM +0800, Gui Jianfeng wrote:
[..]
Quoted 52 lines
Quoted 29 lines
o Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
...
Quoted 9 lines
/* A request got completed from
io_queue. Do the accounting. */ voidelv_ioq_completed_request(struct request_queue *q, struct request *rq){block/elevator-fq.c +1
-0@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/* block/elevator-fq.c void elv_ioq_completed_request(struct re · line 3582
Quoted 10 lines
block/elevator-fq.c +1
-4* If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) {Hi Vivek,
Quoted 11 lines
So we make use of
single_ioq_no_timed_expiry()to decide whether there is only root ioq to be busy, right? Butsingle_ioq_no_timed_expiry()only checks if the root cgroup is the only group and if there is only onebusy_ioqthere. As I explained in previous mail, these two checks are not sufficient to say the current active ioq comes from root group. Because when the child cgroup is just removed, and the ioq which belongs to child group is still there(maybe some requests are in flight). In this case, only root cgroup and only one active ioq (child ioq) checks are satisfied. So IMHO, insingle_ioq_no_timed_expiry()we still need to check "efqd->root_group->ioq" is already created to ensure the only ioq comes from root group. Am i missing something?Hi Gui,
The only side effect of not checking for "
efqd->root_group->ioq" seems to be that this ioq hence io group of the child will not be freed immediately and release will be delayed until a some other queue in the system gets backlogged or ioscheduler exits. At that point of time, this child queue will be expired and ioq and iog will be freed.
Quoted 6 lines
The advantage is that if there is IO happening only in child group in that case we can avoid expiring that queue.
So may be keeping the queue around for sometime is not a very idea.
Am I missing something?
Hi Vivek,
I think you're right.
A thought comes to my mind, whether can we extend optimization for not expiring an ioq not only for root group, but for single ioq case? IOW, if there is only one ioq in hierarchy, we don't exipre it until another ioq gets backlogged, and we just charge only one time slice for that ioq when it schedule out. This might help if IO only happens in child group.
Quoted 6 lines
Thanks
Vivek
Signature · 2 lines
Regards Gui Jianfeng
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
On Wed, Sep 16, 2009 at 10:58:10AM +0800, Gui Jianfeng wrote:
[..]
Quoted 53 lines
Quoted 29 lines
o Fixed the issue of not expiring the queue for single ioq schedulers. Reported and fixed by Gui.
o If an AS queue is not expired for a long time and suddenly somebody decides to create a group and launch a job there, in that case old AS queue will be expired with a very high value of slice used and will get a very high disk time. Fix it by marking the queue as "
charge_one_slice" and charge the queue only for a single time slice and not for whole of the duration when queue was running.o There are cases where in case of AS, excessive queue expiration will take place by elevator fair queuing layer because of few reasons.
- AS does not anticipate on a queue if there are no competing requests. So if only a single reader is present in a group, anticipation does not get turn on.- elevator layer does not know that As is anticipating hence initiates expiry requests in
select_ioq()thinking queue is empty.- elevaotr layer tries to aggressively expire last empty queue. This can lead to lof of queue expiry
o This patch now starts
ANITC_WAIT_NEXTanticipation if last request in the queue completed and associated io context is eligible to anticipate. Also AS lets elevatory layer know that it is anticipating (elv_ioq_wait_request()) . This solves above mentioned issues.o Moved some of the code in separate functions to improve readability.
...
Quoted 9 lines
/* A request got completed from
io_queue. Do the accounting. */ voidelv_ioq_completed_request(struct request_queue *q, struct request *rq){block/elevator-fq.c +1
-0@@ -3470,16 +3572,16 @@ void elv_ioq_completed_request(struct reelv_set_prio_slice (q->elevator->efqd, ioq);elv_clear_ioq_slice_new (ioq); } +/* block/elevator-fq.c void elv_ioq_completed_request(struct re · line 3582
Quoted 10 lines
block/elevator-fq.c +1
-4* If there is only root group present, don't expire the queue * for single queue ioschedulers (noop, deadline, AS). It is * unnecessary overhead. */ -if (is_only_root_group () && -elv_iosched_single_ioq (q->elevator)) { -elv_log_ioq (efqd, ioq,"select: only root group," -" no expiry" ); +if (single_ioq_no_timed_expiry (q)) {Hi Vivek,
Quoted 11 lines
So we make use of
single_ioq_no_timed_expiry()to decide whether there is only root ioq to be busy, right? Butsingle_ioq_no_timed_expiry()only checks if the root cgroup is the only group and if there is only onebusy_ioqthere. As I explained in previous mail, these two checks are not sufficient to say the current active ioq comes from root group. Because when the child cgroup is just removed, and the ioq which belongs to child group is still there(maybe some requests are in flight). In this case, only root cgroup and only one active ioq (child ioq) checks are satisfied. So IMHO, insingle_ioq_no_timed_expiry()we still need to check "efqd->root_group->ioq" is already created to ensure the only ioq comes from root group. Am i missing something?
Hi Gui,
The only side effect of not checking for "efqd->root_group->ioq" seems to be that this ioq hence io group of the child will not be freed immediately and release will be delayed until a some other queue in the system gets backlogged or ioscheduler exits. At that point of time, this child queue will be expired and ioq and iog will be freed.
The advantage is that if there is IO happening only in child group in that case we can avoid expiring that queue.
So may be keeping the queue around for sometime is not a very idea.
Am I missing something?
Thanks
Vivek
Re: [PATCH] io-controller: Fix task hanging when there are more than one groups
In reply to Vivek GoyalHi Vivek,
Currently, we just set this flag when anticipating next request. So make sure we remove this flag also in this case.
--- block/as-iosched.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-)
block/as-iosched.c +3 -2
@@ -728,9 +728,10 @@ static void as_antic_stop(struct as_data *ad)as_log (ad,"as_antic_stop antic_status=%d" , ad->antic_status);if (status ==ANTIC_WAIT_REQ || status ==ANTIC_WAIT_NEXT ) { -if (status ==ANTIC_WAIT_NEXT ) +if (status ==ANTIC_WAIT_NEXT ) {del_timer (&ad->antic_timer); -as_clear_active_asq_wait_request (ad); +as_clear_active_asq_wait_request (ad); + } ad->antic_status =ANTIC_FINISHED ;/* see as_work_handler */ kblockd_schedule_work (ad->q, &ad->antic_work);
Signature · 1 line
1.5.4.rc3
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 73 lines · Gui Jianfeng
Quoted 14 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing
getdents()syscall.
kernel generates the following message:INFO: task find:3260 blocked for more than 120 seconds."echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
find Da1e957871912 3260 28970x00000004
f6af2db800000096f660075ca1e9578700000032f6600270f6600508c203782000000000c09e0820f655f0c0f6af2d8cfffebbf100000000c0447323f7152a1c0006a144f7152a1c0006a144f6af2e04f6af2db0c04438dfc2037820c2037820Call Trace: [<c0447323>] ? getnstimeofday+0x57/0xe0 [<c04438df>] ? ktime_get_ts+0x4a/0x4e [<c068ab68>] io_schedule+0x47/0x79 [<c04c12ee>] sync_buffer+0x36/0x3a [<c068ae14>] __wait_on_bit+0x36/0x5d [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c068ae93>] out_of_line_wait_on_bit+0x58/0x60 [<c04c12b8>] ? sync_buffer+0x0/0x3a [<c0440fa4>] ? wake_bit_function+0x0/0x43 [<c04c1249>] __wait_on_buffer+0x19/0x1c [<f81e4186>] ext3_bread+0x5e/0x79 [ext3] [<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3] [<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3] [<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3 [<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f [<c044c45b>] ? trace_hardirqs_on+0xb/0xd [<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3] [<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3] [<c04b1100>] ? filldir64+0x0/0xcd [<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5 [<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5 [<c04b12db>] ? vfs_readdir+0x46/0x94 [<c04b12fd>] vfs_readdir+0x68/0x94 [<c04b1100>] ? filldir64+0x0/0xcd [<c04b1387>] sys_getdents64+0x5e/0x9f [<c04028b4>] sysenter_do_call+0x12/0x32 1 lock held by find/3260: #0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94
Quoted 2 lines
ext3 calls
wait_on_buffer()to wait buffer, and schedule the task out inTASK_UNINTERRUPTIBLEstate, and I found this task will be resumed after a quite long period(more than 10 mins).
Thanks Gui. As Jens said, it does look like a case of missing queue restart somewhere and now we are stuck, no requests are being dispatched to the disk and queue is already unplugged.
Can you please also try capturing the trace of events at io scheduler (blktrace) to see how did we get into that situation.
Are you using ide drivers and not libata? As jens said, I will try to make use of ide drivers and see if I can reproduce it.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 14 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Quoted 8 lines
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
Can you please dig deeper a bit into it.
- If whole system hangs or it is just IO to disk seems to be hung.
Only when the task is trying do IO to disk it will hang up.
Quoted 1 line
- Does io scheduler switch on the device work
yes, io scheduler can be switched, and the hung task will be resumed.
Quoted 2 lines
- If the system is not hung, can you capture the blktrace on the device. Trace might give some idea, what's happening.
I run a "find" task to do some io on that disk, it seems that task hangs when it is issuing getdents() syscall.
kernel generates the following message:
INFO: task find:3260 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
find D a1e95787 1912 3260 2897 0x00000004
f6af2db8 00000096 f660075c a1e95787 00000032 f6600270 f6600508 c2037820 00000000 c09e0820 f655f0c0 f6af2d8c fffebbf1 00000000 c0447323 f7152a1c 0006a144 f7152a1c 0006a144 f6af2e04 f6af2db0 c04438df c2037820 c2037820
Call Trace:
[<c0447323>] ? getnstimeofday+0x57/0xe0
[<c04438df>] ? ktime_get_ts+0x4a/0x4e
[<c068ab68>] io_schedule+0x47/0x79
[<c04c12ee>] sync_buffer+0x36/0x3a
[<c068ae14>] __wait_on_bit+0x36/0x5d
[<c04c12b8>] ? sync_buffer+0x0/0x3a
[<c068ae93>] out_of_line_wait_on_bit+0x58/0x60
[<c04c12b8>] ? sync_buffer+0x0/0x3a
[<c0440fa4>] ? wake_bit_function+0x0/0x43
[<c04c1249>] __wait_on_buffer+0x19/0x1c
[<f81e4186>] ext3_bread+0x5e/0x79 [ext3]
[<f81e77a8>] htree_dirblock_to_tree+0x1f/0x120 [ext3]
[<f81e7923>] ext3_htree_fill_tree+0x7a/0x1bb [ext3]
[<c04a01f9>] ? kmem_cache_alloc+0x86/0xf3
[<c044c428>] ? trace_hardirqs_on_caller+0x107/0x12f
[<c044c45b>] ? trace_hardirqs_on+0xb/0xd
[<f81e09e4>] ? ext3_readdir+0x9e/0x692 [ext3]
[<f81e0b34>] ext3_readdir+0x1ee/0x692 [ext3]
[<c04b1100>] ? filldir64+0x0/0xcd
[<c068b86a>] ? mutex_lock_killable_nested+0x2b1/0x2c5
[<c068b874>] ? mutex_lock_killable_nested+0x2bb/0x2c5
[<c04b12db>] ? vfs_readdir+0x46/0x94
[<c04b12fd>] vfs_readdir+0x68/0x94
[<c04b1100>] ? filldir64+0x0/0xcd
[<c04b1387>] sys_getdents64+0x5e/0x9f
[<c04028b4>] sysenter_do_call+0x12/0x32
1 lock held by find/3260:
#0: (&sb->s_type->i_mutex_key#7){+.+.+.}, at: [<c04b12db>] vfs_readdir+0x46/0x94ext3 calls wait_on_buffer() to wait buffer, and schedule the task out in TASK_UNINTERRUPTIBLE state, and I found this task will be resumed after a quite long period(more than 10 mins).
Signature · 2 lines
Regards Gui Jianfeng
Quoted 10 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 8 lines
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
The key factor is likely the old IDE driver, since this is IO scheduler related. It probably works if Gui uses libata instead, and you can probably reproduce if you could use drivers/ide/ on a disk.
So the likely culprit is probably a missing queue restart somewhere on IO completion.
Signature · 1 line
Jens Axboe
Quoted 10 lines · Vivek Goyal
Quoted 9 lines · Gui Jianfeng
Hi Vivek,
I happened to encount a bug when i test IO Controller V9. When there are three tasks to run concurrently in three group, that is, one is parent group, and other two tasks are running in two different child groups respectively to read or write files in some disk, say disk "hdb", The task may hang up, and other tasks which access into "hdb" will also hang up.
Quoted 8 lines
Quoted 3 lines
The bug only happens when using AS io scheduler. The following scirpt can reproduce this bug in my box.
Hi Gui,
I tried reproducing this on my system and can't reproduce it. All the three processes get killed and system does not hang.
The key factor is likely the old IDE driver, since this is IO scheduler related. It probably works if Gui uses libata instead, and you can probably reproduce if you could use drivers/ide/ on a disk.
So the likely culprit is probably a missing queue restart somewhere on IO completion.
Signature · 1 line
Jens Axboe
Quoted 9 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
For ease of patching, a consolidated patch is available here.
http://people.redhat.com/~vgoyal/io-controller/io-scheduler-based-io-controller-v9.patch
Found few issues during testing. Sending 3 more patches for this series. After some more testing, will merge these patches in higher level patches and post V10.
Thanks
Vivek
[PATCH 24/23] io-controller: Don't leave a queue active when a disk is idle
In reply to Vivek Goyalo It is possible that when there is only a single queue in the system, it remains unexpired for a long time (because there is no IO activity on the disk). So when next request comes in after a long time, it might make scheduler think that all this while queue used the disk and it will assign a high vdisktime to the queue. Hence make sure queue is expired once all the requests have completed from the queue.
o Also avoid unnecessarily expiring a queue when it has got one request dispatched to the queue and waiting for it to finish and it does not have more requests queued to dispatch.
--- block/elevator-fq.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
block/elevator-fq.c +15 -0
@@ -2947,6 +2947,10 @@ void *elv_select_ioq(struct request_queuif (ioq ==NULL ) goto new_queue; +/* There is only one active queue which is empty. Nothing to dispatch */ +if (elv_nr_busy_ioq (q->elevator) ==1 && !ioq->nr_queued) +return NULL ; + iog =ioq_to_io_group (ioq);/*
@@ -3236,6 +3240,17 @@ void elv_ioq_completed_request(struct reelse elv_iog_arm_slice_timer (q, iog,0 ); } + +/* +* if this is only queue and it has completed all its requests +* and has nothing to dispatch, expire it. We don't want to +* keep it around idle otherwise later when it is expired, all +* this idle time will be added to queue's disk time used. +*/ +if (efqd->busy_queues ==1 && !ioq->dispatched && + !ioq->nr_queued && !timer_pending (&efqd->idle_slice_timer)) { +elv_slice_expired (q); + } } done:if (!efqd->rq_in_driver)
Rik van Riel
Acked-byRe: [PATCH 24/23] io-controller: Don't leave a queue active when a disk is idle
Quoted 7 lines · Vivek Goyal
o It is possible that when there is only a single queue in the system, it remains unexpired for a long time (because there is no IO activity on the disk). So when next request comes in after a long time, it might make scheduler think that all this while queue used the disk and it will assign a high vdisktime to the queue. Hence make sure queue is expired once all the requests have completed from the queue.
Quoted 5 lines
o Also avoid unnecessarily expiring a queue when it has got one request dispatched to the queue and waiting for it to finish and it does not have more requests queued to dispatch.
Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Rik van Riel
Acked-byRe: [PATCH 24/23] io-controller: Don't leave a queue active when a disk is idle
Quoted 7 lines · Vivek Goyal
o It is possible that when there is only a single queue in the system, it remains unexpired for a long time (because there is no IO activity on the disk). So when next request comes in after a long time, it might make scheduler think that all this while queue used the disk and it will assign a high vdisktime to the queue. Hence make sure queue is expired once all the requests have completed from the queue.
Quoted 5 lines
o Also avoid unnecessarily expiring a queue when it has got one request dispatched to the queue and waiting for it to finish and it does not have more requests queued to dispatch.
Signed-off-by Vivek Goyal <[email protected]>
Signature · 1 line
All rights reversed.
[PATCH 25/23] io-controller: fix queue vs group fairness
In reply to Vivek Goyalo I found an issue during test and that is if there is a mix of queue and group at same level, there can be fairness issue. For example, consider following case.
root / \ T1 G1 | T2
T1 and T2 are two tasks with prio 0 and 7 respectively and G1 is the group with weight 900.
Task T1 prio 0 is mapped to weight 900 and it will get slice length of 180ms and then queue will expire and will be put after G1. (Note, in case of reader most liekly next request will come after queue expiry hence queue will be deleted and once the request comes in again, it will added to tree fresh. A fresh queue is added at the end of the tree. So it will be put after G1.).
Now G1 will get to run (effectivly T2 will run), T2 has prio 7, which will map to weight 200 and get slice length of 40ms and will expire after that. Now G1 will a new vtime which is effectively charge of 40ms.
Now to get fairness G1 should run more but instead T1 will be running as we gave it a vtime, same as G1.
The core issue here is that for readers, when slice expires, queue is empty and not backlogged hence it gets deleted from the tree. Because CFQ only operates in flat mode, it did a smart thing and did not keep a track of history. Instead it provides slice lenghts according to prio and if in one round of dispatch one gets fairness it is fine, otherwise upon queue expiry you will be placed at the end of service tree.
This does not work in hierarchical setups where group's slice lenght is determined not by group' weight but by the weight of the queue which will run under the group.
Hence we need to keep track of histroy and assign a new vtime based on disk time used by the current queue at the time of expiry.
But here io scheduler is little different from CFS that at the time of expiry most of the time reader's queue is empty. So one will end up deleting it from the service tree and next request comes with-in 1 ms and it gets into the tree again like a new process.
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree min_vtime wrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).
Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
--- block/cfq-iosched.c | 2 block/elevator-fq.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++---- block/elevator-fq.h | 9 + 3 files changed, 246 insertions(+), 17 deletions(-)
block/elevator-fq.c +236 -16
@@ -52,6 +52,8 @@ static struct kmem_cache *elv_ioq_pool; #define elv_log_entity(entity, fmt, args...)#endif +static void check_idle_tree_release (struct io_service_tree *st); +static inline struct io_queue *ioq_of (struct io_entity *entity) {if (entity->my_sd ==NULL )
@@ -109,6 +111,11 @@ elv_prio_to_slice(struct elv_fq_data *efreturn elv_weight_slice (efqd,elv_ioq_sync (ioq), ioq->entity.weight); } +static inline int vdisktime_gt (u64 a,u64 b) +{ +return (s64)(a - b) >0 ; +} +static inline u64 max_vdisktime (u64 min_vdisktime,u64 vdisktime) {s64 delta = (s64)(vdisktime - min_vdisktime);
@@ -145,6 +152,7 @@ static void update_min_vdisktime(struct } st->min_vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); +check_idle_tree_release (st); }static inline struct io_entity *parent_entity (struct io_entity *entity)
@@ -411,27 +419,46 @@ static void place_entity(struct io_servistruct rb_node *parent;struct io_entity *entry;int nr_active = st->nr_active -1 ; +struct io_queue *ioq =ioq_of (entity); +int sync =1 ; + +if (ioq) + sync =elv_ioq_sync (ioq); + +if (add_front || !nr_active) { + vdisktime = st->min_vdisktime; + goto done; + } + +if (sync && entity->vdisktime + &&vdisktime_gt (entity->vdisktime, st->min_vdisktime)) { +/* vdisktime still in future. Use old vdisktime */ + vdisktime = entity->vdisktime; + goto done; + }/* -* Currently put entity at the end of last entity. This probably will -* require adjustments as we move along +* Effectively a new queue. Assign sync queue a lower vdisktime so +* we can achieve better latencies for small file readers. For async +* queues, put them at the end of the existing queue. +* Group entities are always considered sync. */ -if (io_entity_class_idle (entity)) { - vdisktime =elv_delta_fair (ELV_IDLE_DELAY , entity); - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime += entry->vdisktime; - } - }else if (!add_front && nr_active) { - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime = entry->vdisktime; - } - }else +if (sync) { vdisktime = st->min_vdisktime; + goto done; + } +/* +* Put entity at the end of the tree. Effectively async queues use +* this path. +*/ + parent =rb_last (&st->active); +if (parent) { + entry = rb_entry(parent, struct io_entity, rb_node); + vdisktime = entry->vdisktime; + }else + vdisktime = st->min_vdisktime; +done: entity->vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); elv_log_entity(entity, "place_entity: vdisktime=%llu" " min_vdisktime=%llu", entity->vdisktime,
@@ -447,6 +474,122 @@ static inline void io_entity_update_prio*/ init_io_entity_service_tree (entity,parent_entity (entity)); entity->ioprio_changed =0 ; + +/* +* Assign this entity a fresh vdisktime instead of using +* previous one as prio class will lead to service tree +* change and this vdisktime will not be valid on new +* service tree. +* +* TODO: Handle the case of only prio change. +*/ + entity->vdisktime =0 ; + } +} + +static void +__dequeue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +if (st->rb_leftmost_idle == &entity->rb_node) { +struct rb_node *next_node; + + next_node =rb_next (&entity->rb_node); + st->rb_leftmost_idle = next_node; + } + +rb_erase (&entity->rb_node, &st->idle); +RB_CLEAR_NODE (&entity->rb_node); +} + +static void dequeue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); + +__dequeue_io_entity_idle (entity->st, entity); + entity->on_idle_st =0 ; +if (ioq) +elv_put_ioq (ioq); +} + +static void +__enqueue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +struct rb_node **node = &st->idle.rb_node; +struct rb_node *parent =NULL ; +struct io_entity *entry; +int leftmost =1 ; + +while (*node !=NULL ) { + parent = *node; + entry = rb_entry(parent, struct io_entity, rb_node); + +if (vdisktime_gt (entry->vdisktime, entity->vdisktime)) + node = &parent->rb_left; +else { + node = &parent->rb_right; + leftmost =0 ; + } + } + +/* +* Maintain a cache of leftmost tree entries (it is frequently +* used) +*/ +if (leftmost) + st->rb_leftmost_idle = &entity->rb_node; + +rb_link_node (&entity->rb_node, parent, node); +rb_insert_color (&entity->rb_node, &st->idle); +} + +static void enqueue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); +struct io_group *parent_iog; + +/* +* Don't put an entity on idle tree if it has been marked for deletion. +* We are not expecting more io from this entity. No need to cache it +*/ + +if (entity->exiting) +return ; + +/* +* If parent group is exiting, don't put on idle tree. May be task got +* moved to a different cgroup and original cgroup got deleted +*/ + parent_iog =iog_of (parent_entity (entity)); +if (parent_iog->entity.exiting) +return ; + +if (ioq) +elv_get_ioq (ioq); +__enqueue_io_entity_idle (entity->st, entity); + entity->on_idle_st =1 ; +} + +static void check_idle_tree_release (struct io_service_tree *st) +{ +struct io_entity *leftmost; + +if (!st->rb_leftmost_idle) +return ; + + leftmost = rb_entry(st->rb_leftmost_idle, struct io_entity, rb_node); + +if (vdisktime_gt (st->min_vdisktime, leftmost->vdisktime)) +dequeue_io_entity_idle (leftmost); +} + +static void flush_idle_tree (struct io_service_tree *st) +{ +struct io_entity *entity; + +while (st->rb_leftmost_idle) { + entity = rb_entry(st->rb_leftmost_idle, struct io_entity, + rb_node); +dequeue_io_entity_idle (entity); } }
@@ -483,6 +626,9 @@ static void dequeue_io_entity(struct io_ st->nr_active--; sd->nr_active--;debug_update_stats_dequeue (entity); + +if (vdisktime_gt (entity->vdisktime, st->min_vdisktime)) +enqueue_io_entity_idle (entity); } static void
@@ -524,6 +670,16 @@ static void enqueue_io_entity(struct io_struct io_service_tree *st;struct io_sched_data *sd =io_entity_sched_data (entity); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +else +/* +* This entity was not in idle tree cache. Zero out vdisktime +* so that we don't rely on old vdisktime instead assign a +* fresh one. +*/ + entity->vdisktime =0 ; +io_entity_update_prio (entity); st = entity->st; st->nr_active++;
@@ -574,6 +730,8 @@ static void requeue_io_entity(struct io_struct io_service_tree *st = entity->st;struct io_entity *next_entity; + entity->vdisktime =0 ; +if (add_front) { next_entity =__lookup_next_io_entity (st);
@@ -1937,11 +2095,18 @@ static void io_free_root_group(struct el {struct io_group *iog = e->efqd->root_group;struct io_cgroup *iocg = &io_root_cgroup; +struct io_service_tree *st; +int i;spin_lock_irq (&iocg->lock);hlist_del_rcu (&iog->group_node);spin_unlock_irq (&iocg->lock); +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +put_io_group_queues (e, iog);elv_put_iog (iog); }
@@ -2039,9 +2204,29 @@ EXPORT_SYMBOL(elv_put_iog);*/ static void __io_destroy_group (struct elv_fq_data *efqd,struct io_group *iog) { +struct io_service_tree *st; +int i; +struct io_entity *entity = &iog->entity; + +/* +* Mark io group for deletion so that no new entry goes in +* idle tree. Any active queue which is removed from active +* tree will not be put in to idle tree. +*/ + entity->exiting =1 ; + +/* We flush idle tree now, and don't put things in there any more. */ +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +hlist_del (&iog->elv_data_node);put_io_group_queues (efqd->eq, iog); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +/* * Put the reference taken at the time of creation so that when all * queues are gone, group can be destroyed.
@@ -2374,7 +2559,13 @@ static struct io_group *io_alloc_root_grstatic void io_free_root_group (struct elevator_queue *e) {struct io_group *iog = e->efqd->root_group; +struct io_service_tree *st; +int i; +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + }put_io_group_queues (e, iog);kfree (iog); }
@@ -3257,6 +3448,35 @@ done:elv_schedule_dispatch (q); } +/* +* The process associted with ioq (in case of cfq), is going away. Mark it +* for deletion. +*/ +void elv_exit_ioq (struct io_queue *ioq) +{ +struct io_entity *entity = &ioq->entity; + +/* +* Async ioq's belong to io group and are cleaned up once group is +* being deleted. Not need to do any cleanup here even if cfq has +* dropped the reference to the queue +*/ +if (!elv_ioq_sync (ioq)) +return ; + +/* +* This queue is still under service. Just mark it so that once all +* the IO from queue is done, it is not put back in idle tree. +*/ +if (entity->on_st) { + entity->exiting =1 ; +return ; + }else if (entity->on_idle_st) { +/* Remove ioq from idle tree */ +dequeue_io_entity_idle (entity); + } +} +EXPORT_SYMBOL (elv_exit_ioq);static void elv_slab_kill (void ) {/*
block/cfq-iosched.c +2 -0
@@ -1138,6 +1138,7 @@ static void cfq_exit_cfqq(struct cfq_datelv_schedule_dispatch (cfqd->queue); } +elv_exit_ioq (cfqq->ioq);cfq_put_queue (cfqq); }
@@ -1373,6 +1374,7 @@ static void changed_cgroup(struct io_con*/ if (iog != __iog) {cic_set_cfqq (cic,NULL ,1 ); +elv_exit_ioq (sync_cfqq->ioq);cfq_put_queue (sync_cfqq); } }
block/elevator-fq.h +8 -1
@@ -33,6 +33,10 @@ struct io_service_tree {u64 min_vdisktime;struct rb_node *rb_leftmost;unsigned int nr_active; + +/* A cache of io entities which were served and expired */ +struct rb_root idle; +struct rb_node *rb_leftmost_idle; };struct io_sched_data {
@@ -44,9 +48,12 @@ struct io_sched_data {struct io_entity {struct rb_node rb_node;int on_st; +int on_idle_st;u64 vdisktime;unsigned int weight;struct io_entity *parent; +/* This io entity (queue or group) has been marked for deletion */ +unsigned int exiting;struct io_sched_data *my_sd;struct io_service_tree *st;
@@ -572,7 +579,7 @@ extern struct io_queue *elv_alloc_ioq(stextern void elv_free_ioq (struct io_queue *ioq);extern struct io_group *ioq_to_io_group (struct io_queue *ioq);extern int elv_iog_should_idle (struct io_queue *ioq); - +extern void elv_exit_ioq (struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ static inline struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e)
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
patches 25, and 26 both have checkpatch errors.. Could you run them through checkpatch and clean up any errors you find?
Daniel
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
Quoted 4 lines · Daniel Walker
patches 25, and 26 both have checkpatch errors.. Could you run them through checkpatch and clean up any errors you find?
Sure. Will cleanup those errors in V10 of posting.
Thanks
Vivek
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
In reply to Daniel WalkerQuoted 4 lines · Daniel Walker
patches 25, and 26 both have checkpatch errors.. Could you run them through checkpatch and clean up any errors you find?
Sure. Will cleanup those errors in V10 of posting.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
patches 25, and 26 both have checkpatch errors.. Could you run them through checkpatch and clean up any errors you find?
Daniel
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
In reply to a message not in the archiveHi,
Quoted 5 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 06:28:27PM -0400o I found an issue during test and that is if there is a mix of queue and group
...
Quoted 13 lines
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree
min_vtimewrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
Quoted 8 lines
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
just a little note: this patch seems to introduce a special case for
vdisktime = 0, assigning it the meaning of "bad timestamp," but the virtual time space wraps, so 0 is a perfectly legal value, which can be reached by service. I have no idea if it can produce visible effects, but it doesn't seem to be correct.
Quoted 418 lines
Signed-off-by Vivek Goyal <[email protected]>--- block/cfq-iosched.c | 2 block/elevator-fq.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++---- block/elevator-fq.h | 9 + 3 files changed, 246 insertions(+), 17 deletions(-)Index:
linux16/block/elevator-fq.c
===================================================================block/elevator-fq.c +236
-16@@ -52,6 +52,8 @@ static struct kmem_cache *elv_ioq_pool; #define elv_log_entity(entity, fmt, args...)#endif +static void check_idle_tree_release (struct io_service_tree *st); +static inline struct io_queue *ioq_of (struct io_entity *entity) {if (entity->my_sd ==NULL )@@ -109,6 +111,11 @@ elv_prio_to_slice(struct elv_fq_data *efreturn elv_weight_slice (efqd,elv_ioq_sync (ioq), ioq->entity.weight); } +static inline int vdisktime_gt (u64 a,u64 b) +{ +return (s64)(a - b) >0 ; +} +static inline u64 max_vdisktime (u64 min_vdisktime,u64 vdisktime) {s64 delta = (s64)(vdisktime - min_vdisktime);@@ -145,6 +152,7 @@ static void update_min_vdisktime(struct } st->min_vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); +check_idle_tree_release (st); }static inline struct io_entity *parent_entity (struct io_entity *entity)@@ -411,27 +419,46 @@ static void place_entity(struct io_servistruct rb_node *parent;struct io_entity *entry;int nr_active = st->nr_active -1 ; +struct io_queue *ioq =ioq_of (entity); +int sync =1 ; + +if (ioq) + sync =elv_ioq_sync (ioq); + +if (add_front || !nr_active) { + vdisktime = st->min_vdisktime; + goto done; + } + +if (sync && entity->vdisktime + &&vdisktime_gt (entity->vdisktime, st->min_vdisktime)) { +/* vdisktime still in future. Use old vdisktime */ + vdisktime = entity->vdisktime; + goto done; + }/* -* Currently put entity at the end of last entity. This probably will -* require adjustments as we move along +* Effectively a new queue. Assign sync queue a lower vdisktime so +* we can achieve better latencies for small file readers. For async +* queues, put them at the end of the existing queue. +* Group entities are always considered sync. */ -if (io_entity_class_idle (entity)) { - vdisktime =elv_delta_fair (ELV_IDLE_DELAY , entity); - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime += entry->vdisktime; - } - }else if (!add_front && nr_active) { - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime = entry->vdisktime; - } - }else +if (sync) { vdisktime = st->min_vdisktime; + goto done; + } +/* +* Put entity at the end of the tree. Effectively async queues use +* this path. +*/ + parent =rb_last (&st->active); +if (parent) { + entry = rb_entry(parent, struct io_entity, rb_node); + vdisktime = entry->vdisktime; + }else + vdisktime = st->min_vdisktime; +done: entity->vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); elv_log_entity(entity, "place_entity: vdisktime=%llu" " min_vdisktime=%llu", entity->vdisktime,@@ -447,6 +474,122 @@ static inline void io_entity_update_prio*/ init_io_entity_service_tree (entity,parent_entity (entity)); entity->ioprio_changed =0 ; + +/* +* Assign this entity a fresh vdisktime instead of using +* previous one as prio class will lead to service tree +* change and this vdisktime will not be valid on new +* service tree. +* +* TODO: Handle the case of only prio change. +*/ + entity->vdisktime =0 ; + } +} + +static void +__dequeue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +if (st->rb_leftmost_idle == &entity->rb_node) { +struct rb_node *next_node; + + next_node =rb_next (&entity->rb_node); + st->rb_leftmost_idle = next_node; + } + +rb_erase (&entity->rb_node, &st->idle); +RB_CLEAR_NODE (&entity->rb_node); +} + +static void dequeue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); + +__dequeue_io_entity_idle (entity->st, entity); + entity->on_idle_st =0 ; +if (ioq) +elv_put_ioq (ioq); +} + +static void +__enqueue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +struct rb_node **node = &st->idle.rb_node; +struct rb_node *parent =NULL ; +struct io_entity *entry; +int leftmost =1 ; + +while (*node !=NULL ) { + parent = *node; + entry = rb_entry(parent, struct io_entity, rb_node); + +if (vdisktime_gt (entry->vdisktime, entity->vdisktime)) + node = &parent->rb_left; +else { + node = &parent->rb_right; + leftmost =0 ; + } + } + +/* +* Maintain a cache of leftmost tree entries (it is frequently +* used) +*/ +if (leftmost) + st->rb_leftmost_idle = &entity->rb_node; + +rb_link_node (&entity->rb_node, parent, node); +rb_insert_color (&entity->rb_node, &st->idle); +} + +static void enqueue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); +struct io_group *parent_iog; + +/* +* Don't put an entity on idle tree if it has been marked for deletion. +* We are not expecting more io from this entity. No need to cache it +*/ + +if (entity->exiting) +return ; + +/* +* If parent group is exiting, don't put on idle tree. May be task got +* moved to a different cgroup and original cgroup got deleted +*/ + parent_iog =iog_of (parent_entity (entity)); +if (parent_iog->entity.exiting) +return ; + +if (ioq) +elv_get_ioq (ioq); +__enqueue_io_entity_idle (entity->st, entity); + entity->on_idle_st =1 ; +} + +static void check_idle_tree_release (struct io_service_tree *st) +{ +struct io_entity *leftmost; + +if (!st->rb_leftmost_idle) +return ; + + leftmost = rb_entry(st->rb_leftmost_idle, struct io_entity, rb_node); + +if (vdisktime_gt (st->min_vdisktime, leftmost->vdisktime)) +dequeue_io_entity_idle (leftmost); +} + +static void flush_idle_tree (struct io_service_tree *st) +{ +struct io_entity *entity; + +while (st->rb_leftmost_idle) { + entity = rb_entry(st->rb_leftmost_idle, struct io_entity, + rb_node); +dequeue_io_entity_idle (entity); } }@@ -483,6 +626,9 @@ static void dequeue_io_entity(struct io_ st->nr_active--; sd->nr_active--;debug_update_stats_dequeue (entity); + +if (vdisktime_gt (entity->vdisktime, st->min_vdisktime)) +enqueue_io_entity_idle (entity); } static void@@ -524,6 +670,16 @@ static void enqueue_io_entity(struct io_struct io_service_tree *st;struct io_sched_data *sd =io_entity_sched_data (entity); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +else +/* +* This entity was not in idle tree cache. Zero out vdisktime +* so that we don't rely on old vdisktime instead assign a +* fresh one. +*/ + entity->vdisktime =0 ; +io_entity_update_prio (entity); st = entity->st; st->nr_active++;@@ -574,6 +730,8 @@ static void requeue_io_entity(struct io_struct io_service_tree *st = entity->st;struct io_entity *next_entity; + entity->vdisktime =0 ; +if (add_front) { next_entity =__lookup_next_io_entity (st);@@ -1937,11 +2095,18 @@ static void io_free_root_group(struct el {struct io_group *iog = e->efqd->root_group;struct io_cgroup *iocg = &io_root_cgroup; +struct io_service_tree *st; +int i;spin_lock_irq (&iocg->lock);hlist_del_rcu (&iog->group_node);spin_unlock_irq (&iocg->lock); +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +put_io_group_queues (e, iog);elv_put_iog (iog); }@@ -2039,9 +2204,29 @@ EXPORT_SYMBOL(elv_put_iog);*/ static void __io_destroy_group (struct elv_fq_data *efqd,struct io_group *iog) { +struct io_service_tree *st; +int i; +struct io_entity *entity = &iog->entity; + +/* +* Mark io group for deletion so that no new entry goes in +* idle tree. Any active queue which is removed from active +* tree will not be put in to idle tree. +*/ + entity->exiting =1 ; + +/* We flush idle tree now, and don't put things in there any more. */ +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +hlist_del (&iog->elv_data_node);put_io_group_queues (efqd->eq, iog); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +/* * Put the reference taken at the time of creation so that when all * queues are gone, group can be destroyed. @@ -2374,7 +2559,13 @@ static struct io_group *io_alloc_root_grstatic void io_free_root_group (struct elevator_queue *e) {struct io_group *iog = e->efqd->root_group; +struct io_service_tree *st; +int i; +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + }put_io_group_queues (e, iog);kfree (iog); }@@ -3257,6 +3448,35 @@ done:elv_schedule_dispatch (q); } +/* +* The process associted with ioq (in case of cfq), is going away. Mark it +* for deletion. +*/ +void elv_exit_ioq (struct io_queue *ioq) +{ +struct io_entity *entity = &ioq->entity; + +/* +* Async ioq's belong to io group and are cleaned up once group is +* being deleted. Not need to do any cleanup here even if cfq has +* dropped the reference to the queue +*/ +if (!elv_ioq_sync (ioq)) +return ; + +/* +* This queue is still under service. Just mark it so that once all +* the IO from queue is done, it is not put back in idle tree. +*/ +if (entity->on_st) { + entity->exiting =1 ; +return ; + }else if (entity->on_idle_st) { +/* Remove ioq from idle tree */ +dequeue_io_entity_idle (entity); + } +} +EXPORT_SYMBOL (elv_exit_ioq);static void elv_slab_kill (void ) {/* Index:
linux16/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +2
-0@@ -1138,6 +1138,7 @@ static void cfq_exit_cfqq(struct cfq_datelv_schedule_dispatch (cfqd->queue); } +elv_exit_ioq (cfqq->ioq);cfq_put_queue (cfqq); }@@ -1373,6 +1374,7 @@ static void changed_cgroup(struct io_con*/ if (iog != __iog) {cic_set_cfqq (cic,NULL ,1 ); +elv_exit_ioq (sync_cfqq->ioq);cfq_put_queue (sync_cfqq); } }Index:
linux16/block/elevator-fq.h
===================================================================block/elevator-fq.h +8
-1@@ -33,6 +33,10 @@ struct io_service_tree {u64 min_vdisktime;struct rb_node *rb_leftmost;unsigned int nr_active; + +/* A cache of io entities which were served and expired */ +struct rb_root idle; +struct rb_node *rb_leftmost_idle; };struct io_sched_data {@@ -44,9 +48,12 @@ struct io_sched_data {struct io_entity {struct rb_node rb_node;int on_st; +int on_idle_st;u64 vdisktime;unsigned int weight;struct io_entity *parent; +/* This io entity (queue or group) has been marked for deletion */ +unsigned int exiting;struct io_sched_data *my_sd;struct io_service_tree *st;@@ -572,7 +579,7 @@ extern struct io_queue *elv_alloc_ioq(stextern void elv_free_ioq (struct io_queue *ioq);extern struct io_group *ioq_to_io_group (struct io_queue *ioq);extern int elv_iog_should_idle (struct io_queue *ioq); - +extern void elv_exit_ioq (struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ static inline struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e)
Rik van Riel
Acked-byRe: [PATCH 25/23] io-controller: fix queue vs group fairness
In reply to a message not in the archiveQuoted 3 lines · Vivek Goyal
o I found an issue during test and that is if there is a mix of queue and group at same level, there can be fairness issue. For example, consider following case.
(assuming the cleanups are done in V10 - the artifact of
a wraparound hitting exactly 0 should not be an issue)
Signature · 1 line
All rights reversed.
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
In reply to Vivek GoyalHi,
Quoted 5 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 06:28:27PM -0400o I found an issue during test and that is if there is a mix of queue and group
...
Quoted 13 lines
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree
min_vtimewrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
Quoted 8 lines
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
just a little note: this patch seems to introduce a special case for
vdisktime = 0, assigning it the meaning of "bad timestamp," but the virtual time space wraps, so 0 is a perfectly legal value, which can be reached by service. I have no idea if it can produce visible effects, but it doesn't seem to be correct.
Quoted 418 lines
Signed-off-by Vivek Goyal <[email protected]>--- block/cfq-iosched.c | 2 block/elevator-fq.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++---- block/elevator-fq.h | 9 + 3 files changed, 246 insertions(+), 17 deletions(-)Index:
linux16/block/elevator-fq.c
===================================================================block/elevator-fq.c +236
-16@@ -52,6 +52,8 @@ static struct kmem_cache *elv_ioq_pool; #define elv_log_entity(entity, fmt, args...)#endif +static void check_idle_tree_release (struct io_service_tree *st); +static inline struct io_queue *ioq_of (struct io_entity *entity) {if (entity->my_sd ==NULL )@@ -109,6 +111,11 @@ elv_prio_to_slice(struct elv_fq_data *efreturn elv_weight_slice (efqd,elv_ioq_sync (ioq), ioq->entity.weight); } +static inline int vdisktime_gt (u64 a,u64 b) +{ +return (s64)(a - b) >0 ; +} +static inline u64 max_vdisktime (u64 min_vdisktime,u64 vdisktime) {s64 delta = (s64)(vdisktime - min_vdisktime);@@ -145,6 +152,7 @@ static void update_min_vdisktime(struct } st->min_vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); +check_idle_tree_release (st); }static inline struct io_entity *parent_entity (struct io_entity *entity)@@ -411,27 +419,46 @@ static void place_entity(struct io_servistruct rb_node *parent;struct io_entity *entry;int nr_active = st->nr_active -1 ; +struct io_queue *ioq =ioq_of (entity); +int sync =1 ; + +if (ioq) + sync =elv_ioq_sync (ioq); + +if (add_front || !nr_active) { + vdisktime = st->min_vdisktime; + goto done; + } + +if (sync && entity->vdisktime + &&vdisktime_gt (entity->vdisktime, st->min_vdisktime)) { +/* vdisktime still in future. Use old vdisktime */ + vdisktime = entity->vdisktime; + goto done; + }/* -* Currently put entity at the end of last entity. This probably will -* require adjustments as we move along +* Effectively a new queue. Assign sync queue a lower vdisktime so +* we can achieve better latencies for small file readers. For async +* queues, put them at the end of the existing queue. +* Group entities are always considered sync. */ -if (io_entity_class_idle (entity)) { - vdisktime =elv_delta_fair (ELV_IDLE_DELAY , entity); - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime += entry->vdisktime; - } - }else if (!add_front && nr_active) { - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime = entry->vdisktime; - } - }else +if (sync) { vdisktime = st->min_vdisktime; + goto done; + } +/* +* Put entity at the end of the tree. Effectively async queues use +* this path. +*/ + parent =rb_last (&st->active); +if (parent) { + entry = rb_entry(parent, struct io_entity, rb_node); + vdisktime = entry->vdisktime; + }else + vdisktime = st->min_vdisktime; +done: entity->vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); elv_log_entity(entity, "place_entity: vdisktime=%llu" " min_vdisktime=%llu", entity->vdisktime,@@ -447,6 +474,122 @@ static inline void io_entity_update_prio*/ init_io_entity_service_tree (entity,parent_entity (entity)); entity->ioprio_changed =0 ; + +/* +* Assign this entity a fresh vdisktime instead of using +* previous one as prio class will lead to service tree +* change and this vdisktime will not be valid on new +* service tree. +* +* TODO: Handle the case of only prio change. +*/ + entity->vdisktime =0 ; + } +} + +static void +__dequeue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +if (st->rb_leftmost_idle == &entity->rb_node) { +struct rb_node *next_node; + + next_node =rb_next (&entity->rb_node); + st->rb_leftmost_idle = next_node; + } + +rb_erase (&entity->rb_node, &st->idle); +RB_CLEAR_NODE (&entity->rb_node); +} + +static void dequeue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); + +__dequeue_io_entity_idle (entity->st, entity); + entity->on_idle_st =0 ; +if (ioq) +elv_put_ioq (ioq); +} + +static void +__enqueue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +struct rb_node **node = &st->idle.rb_node; +struct rb_node *parent =NULL ; +struct io_entity *entry; +int leftmost =1 ; + +while (*node !=NULL ) { + parent = *node; + entry = rb_entry(parent, struct io_entity, rb_node); + +if (vdisktime_gt (entry->vdisktime, entity->vdisktime)) + node = &parent->rb_left; +else { + node = &parent->rb_right; + leftmost =0 ; + } + } + +/* +* Maintain a cache of leftmost tree entries (it is frequently +* used) +*/ +if (leftmost) + st->rb_leftmost_idle = &entity->rb_node; + +rb_link_node (&entity->rb_node, parent, node); +rb_insert_color (&entity->rb_node, &st->idle); +} + +static void enqueue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); +struct io_group *parent_iog; + +/* +* Don't put an entity on idle tree if it has been marked for deletion. +* We are not expecting more io from this entity. No need to cache it +*/ + +if (entity->exiting) +return ; + +/* +* If parent group is exiting, don't put on idle tree. May be task got +* moved to a different cgroup and original cgroup got deleted +*/ + parent_iog =iog_of (parent_entity (entity)); +if (parent_iog->entity.exiting) +return ; + +if (ioq) +elv_get_ioq (ioq); +__enqueue_io_entity_idle (entity->st, entity); + entity->on_idle_st =1 ; +} + +static void check_idle_tree_release (struct io_service_tree *st) +{ +struct io_entity *leftmost; + +if (!st->rb_leftmost_idle) +return ; + + leftmost = rb_entry(st->rb_leftmost_idle, struct io_entity, rb_node); + +if (vdisktime_gt (st->min_vdisktime, leftmost->vdisktime)) +dequeue_io_entity_idle (leftmost); +} + +static void flush_idle_tree (struct io_service_tree *st) +{ +struct io_entity *entity; + +while (st->rb_leftmost_idle) { + entity = rb_entry(st->rb_leftmost_idle, struct io_entity, + rb_node); +dequeue_io_entity_idle (entity); } }@@ -483,6 +626,9 @@ static void dequeue_io_entity(struct io_ st->nr_active--; sd->nr_active--;debug_update_stats_dequeue (entity); + +if (vdisktime_gt (entity->vdisktime, st->min_vdisktime)) +enqueue_io_entity_idle (entity); } static void@@ -524,6 +670,16 @@ static void enqueue_io_entity(struct io_struct io_service_tree *st;struct io_sched_data *sd =io_entity_sched_data (entity); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +else +/* +* This entity was not in idle tree cache. Zero out vdisktime +* so that we don't rely on old vdisktime instead assign a +* fresh one. +*/ + entity->vdisktime =0 ; +io_entity_update_prio (entity); st = entity->st; st->nr_active++;@@ -574,6 +730,8 @@ static void requeue_io_entity(struct io_struct io_service_tree *st = entity->st;struct io_entity *next_entity; + entity->vdisktime =0 ; +if (add_front) { next_entity =__lookup_next_io_entity (st);@@ -1937,11 +2095,18 @@ static void io_free_root_group(struct el {struct io_group *iog = e->efqd->root_group;struct io_cgroup *iocg = &io_root_cgroup; +struct io_service_tree *st; +int i;spin_lock_irq (&iocg->lock);hlist_del_rcu (&iog->group_node);spin_unlock_irq (&iocg->lock); +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +put_io_group_queues (e, iog);elv_put_iog (iog); }@@ -2039,9 +2204,29 @@ EXPORT_SYMBOL(elv_put_iog);*/ static void __io_destroy_group (struct elv_fq_data *efqd,struct io_group *iog) { +struct io_service_tree *st; +int i; +struct io_entity *entity = &iog->entity; + +/* +* Mark io group for deletion so that no new entry goes in +* idle tree. Any active queue which is removed from active +* tree will not be put in to idle tree. +*/ + entity->exiting =1 ; + +/* We flush idle tree now, and don't put things in there any more. */ +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +hlist_del (&iog->elv_data_node);put_io_group_queues (efqd->eq, iog); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +/* * Put the reference taken at the time of creation so that when all * queues are gone, group can be destroyed. @@ -2374,7 +2559,13 @@ static struct io_group *io_alloc_root_grstatic void io_free_root_group (struct elevator_queue *e) {struct io_group *iog = e->efqd->root_group; +struct io_service_tree *st; +int i; +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + }put_io_group_queues (e, iog);kfree (iog); }@@ -3257,6 +3448,35 @@ done:elv_schedule_dispatch (q); } +/* +* The process associted with ioq (in case of cfq), is going away. Mark it +* for deletion. +*/ +void elv_exit_ioq (struct io_queue *ioq) +{ +struct io_entity *entity = &ioq->entity; + +/* +* Async ioq's belong to io group and are cleaned up once group is +* being deleted. Not need to do any cleanup here even if cfq has +* dropped the reference to the queue +*/ +if (!elv_ioq_sync (ioq)) +return ; + +/* +* This queue is still under service. Just mark it so that once all +* the IO from queue is done, it is not put back in idle tree. +*/ +if (entity->on_st) { + entity->exiting =1 ; +return ; + }else if (entity->on_idle_st) { +/* Remove ioq from idle tree */ +dequeue_io_entity_idle (entity); + } +} +EXPORT_SYMBOL (elv_exit_ioq);static void elv_slab_kill (void ) {/* Index:
linux16/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +2
-0@@ -1138,6 +1138,7 @@ static void cfq_exit_cfqq(struct cfq_datelv_schedule_dispatch (cfqd->queue); } +elv_exit_ioq (cfqq->ioq);cfq_put_queue (cfqq); }@@ -1373,6 +1374,7 @@ static void changed_cgroup(struct io_con*/ if (iog != __iog) {cic_set_cfqq (cic,NULL ,1 ); +elv_exit_ioq (sync_cfqq->ioq);cfq_put_queue (sync_cfqq); } }Index:
linux16/block/elevator-fq.h
===================================================================block/elevator-fq.h +8
-1@@ -33,6 +33,10 @@ struct io_service_tree {u64 min_vdisktime;struct rb_node *rb_leftmost;unsigned int nr_active; + +/* A cache of io entities which were served and expired */ +struct rb_root idle; +struct rb_node *rb_leftmost_idle; };struct io_sched_data {@@ -44,9 +48,12 @@ struct io_sched_data {struct io_entity {struct rb_node rb_node;int on_st; +int on_idle_st;u64 vdisktime;unsigned int weight;struct io_entity *parent; +/* This io entity (queue or group) has been marked for deletion */ +unsigned int exiting;struct io_sched_data *my_sd;struct io_service_tree *st;@@ -572,7 +579,7 @@ extern struct io_queue *elv_alloc_ioq(stextern void elv_free_ioq (struct io_queue *ioq);extern struct io_group *ioq_to_io_group (struct io_queue *ioq);extern int elv_iog_should_idle (struct io_queue *ioq); - +extern void elv_exit_ioq (struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ static inline struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e)
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
Quoted 30 lines · Fabio Checconi
Hi,
Quoted 5 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 06:28:27PM -0400o I found an issue during test and that is if there is a mix of queue and group
...
Quoted 13 lines
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree
min_vtimewrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
Quoted 8 lines
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
Quoted 7 lines
just a little note: this patch seems to introduce a special case for
vdisktime = 0, assigning it the meaning of "bad timestamp," but the virtual time space wraps, so 0 is a perfectly legal value, which can be reached by service. I have no idea if it can produce visible effects, but it doesn't seem to be correct.
Hi Fabio,
You are right that technically during wrap arounds one can hit value 0 as legal value. But I think it is hard to hit at the same time, the only side affect of it will be that a queue will be either placed favorably (in case of sync queues) or at the end of tree (if it is async queue).
Async queues anyway go at the end after every dispatch round. So only side affect is that once during wrap around cycle a sync queue will be placed favorably and can gain share once in a dispatch round.
I think it is not a big issue at this point of time. But if it becomes significant, I can introduce a new variable or start passing function parameter to denote whether we found the queue in cache or not.
But if you think that it is absolutely no no, let me know....
Thanks
Vivek
Quoted 418 lines
Quoted 418 lines
Signed-off-by Vivek Goyal <[email protected]>--- block/cfq-iosched.c | 2 block/elevator-fq.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++---- block/elevator-fq.h | 9 + 3 files changed, 246 insertions(+), 17 deletions(-)Index:
linux16/block/elevator-fq.c
===================================================================block/elevator-fq.c +236
-16@@ -52,6 +52,8 @@ static struct kmem_cache *elv_ioq_pool; #define elv_log_entity(entity, fmt, args...)#endif +static void check_idle_tree_release (struct io_service_tree *st); +static inline struct io_queue *ioq_of (struct io_entity *entity) {if (entity->my_sd ==NULL )@@ -109,6 +111,11 @@ elv_prio_to_slice(struct elv_fq_data *efreturn elv_weight_slice (efqd,elv_ioq_sync (ioq), ioq->entity.weight); } +static inline int vdisktime_gt (u64 a,u64 b) +{ +return (s64)(a - b) >0 ; +} +static inline u64 max_vdisktime (u64 min_vdisktime,u64 vdisktime) {s64 delta = (s64)(vdisktime - min_vdisktime);@@ -145,6 +152,7 @@ static void update_min_vdisktime(struct } st->min_vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); +check_idle_tree_release (st); }static inline struct io_entity *parent_entity (struct io_entity *entity)@@ -411,27 +419,46 @@ static void place_entity(struct io_servistruct rb_node *parent;struct io_entity *entry;int nr_active = st->nr_active -1 ; +struct io_queue *ioq =ioq_of (entity); +int sync =1 ; + +if (ioq) + sync =elv_ioq_sync (ioq); + +if (add_front || !nr_active) { + vdisktime = st->min_vdisktime; + goto done; + } + +if (sync && entity->vdisktime + &&vdisktime_gt (entity->vdisktime, st->min_vdisktime)) { +/* vdisktime still in future. Use old vdisktime */ + vdisktime = entity->vdisktime; + goto done; + }/* -* Currently put entity at the end of last entity. This probably will -* require adjustments as we move along +* Effectively a new queue. Assign sync queue a lower vdisktime so +* we can achieve better latencies for small file readers. For async +* queues, put them at the end of the existing queue. +* Group entities are always considered sync. */ -if (io_entity_class_idle (entity)) { - vdisktime =elv_delta_fair (ELV_IDLE_DELAY , entity); - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime += entry->vdisktime; - } - }else if (!add_front && nr_active) { - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime = entry->vdisktime; - } - }else +if (sync) { vdisktime = st->min_vdisktime; + goto done; + } +/* +* Put entity at the end of the tree. Effectively async queues use +* this path. +*/ + parent =rb_last (&st->active); +if (parent) { + entry = rb_entry(parent, struct io_entity, rb_node); + vdisktime = entry->vdisktime; + }else + vdisktime = st->min_vdisktime; +done: entity->vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); elv_log_entity(entity, "place_entity: vdisktime=%llu" " min_vdisktime=%llu", entity->vdisktime,@@ -447,6 +474,122 @@ static inline void io_entity_update_prio*/ init_io_entity_service_tree (entity,parent_entity (entity)); entity->ioprio_changed =0 ; + +/* +* Assign this entity a fresh vdisktime instead of using +* previous one as prio class will lead to service tree +* change and this vdisktime will not be valid on new +* service tree. +* +* TODO: Handle the case of only prio change. +*/ + entity->vdisktime =0 ; + } +} + +static void +__dequeue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +if (st->rb_leftmost_idle == &entity->rb_node) { +struct rb_node *next_node; + + next_node =rb_next (&entity->rb_node); + st->rb_leftmost_idle = next_node; + } + +rb_erase (&entity->rb_node, &st->idle); +RB_CLEAR_NODE (&entity->rb_node); +} + +static void dequeue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); + +__dequeue_io_entity_idle (entity->st, entity); + entity->on_idle_st =0 ; +if (ioq) +elv_put_ioq (ioq); +} + +static void +__enqueue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +struct rb_node **node = &st->idle.rb_node; +struct rb_node *parent =NULL ; +struct io_entity *entry; +int leftmost =1 ; + +while (*node !=NULL ) { + parent = *node; + entry = rb_entry(parent, struct io_entity, rb_node); + +if (vdisktime_gt (entry->vdisktime, entity->vdisktime)) + node = &parent->rb_left; +else { + node = &parent->rb_right; + leftmost =0 ; + } + } + +/* +* Maintain a cache of leftmost tree entries (it is frequently +* used) +*/ +if (leftmost) + st->rb_leftmost_idle = &entity->rb_node; + +rb_link_node (&entity->rb_node, parent, node); +rb_insert_color (&entity->rb_node, &st->idle); +} + +static void enqueue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); +struct io_group *parent_iog; + +/* +* Don't put an entity on idle tree if it has been marked for deletion. +* We are not expecting more io from this entity. No need to cache it +*/ + +if (entity->exiting) +return ; + +/* +* If parent group is exiting, don't put on idle tree. May be task got +* moved to a different cgroup and original cgroup got deleted +*/ + parent_iog =iog_of (parent_entity (entity)); +if (parent_iog->entity.exiting) +return ; + +if (ioq) +elv_get_ioq (ioq); +__enqueue_io_entity_idle (entity->st, entity); + entity->on_idle_st =1 ; +} + +static void check_idle_tree_release (struct io_service_tree *st) +{ +struct io_entity *leftmost; + +if (!st->rb_leftmost_idle) +return ; + + leftmost = rb_entry(st->rb_leftmost_idle, struct io_entity, rb_node); + +if (vdisktime_gt (st->min_vdisktime, leftmost->vdisktime)) +dequeue_io_entity_idle (leftmost); +} + +static void flush_idle_tree (struct io_service_tree *st) +{ +struct io_entity *entity; + +while (st->rb_leftmost_idle) { + entity = rb_entry(st->rb_leftmost_idle, struct io_entity, + rb_node); +dequeue_io_entity_idle (entity); } }@@ -483,6 +626,9 @@ static void dequeue_io_entity(struct io_ st->nr_active--; sd->nr_active--;debug_update_stats_dequeue (entity); + +if (vdisktime_gt (entity->vdisktime, st->min_vdisktime)) +enqueue_io_entity_idle (entity); } static void@@ -524,6 +670,16 @@ static void enqueue_io_entity(struct io_struct io_service_tree *st;struct io_sched_data *sd =io_entity_sched_data (entity); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +else +/* +* This entity was not in idle tree cache. Zero out vdisktime +* so that we don't rely on old vdisktime instead assign a +* fresh one. +*/ + entity->vdisktime =0 ; +io_entity_update_prio (entity); st = entity->st; st->nr_active++;@@ -574,6 +730,8 @@ static void requeue_io_entity(struct io_struct io_service_tree *st = entity->st;struct io_entity *next_entity; + entity->vdisktime =0 ; +if (add_front) { next_entity =__lookup_next_io_entity (st);@@ -1937,11 +2095,18 @@ static void io_free_root_group(struct el {struct io_group *iog = e->efqd->root_group;struct io_cgroup *iocg = &io_root_cgroup; +struct io_service_tree *st; +int i;spin_lock_irq (&iocg->lock);hlist_del_rcu (&iog->group_node);spin_unlock_irq (&iocg->lock); +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +put_io_group_queues (e, iog);elv_put_iog (iog); }@@ -2039,9 +2204,29 @@ EXPORT_SYMBOL(elv_put_iog);*/ static void __io_destroy_group (struct elv_fq_data *efqd,struct io_group *iog) { +struct io_service_tree *st; +int i; +struct io_entity *entity = &iog->entity; + +/* +* Mark io group for deletion so that no new entry goes in +* idle tree. Any active queue which is removed from active +* tree will not be put in to idle tree. +*/ + entity->exiting =1 ; + +/* We flush idle tree now, and don't put things in there any more. */ +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +hlist_del (&iog->elv_data_node);put_io_group_queues (efqd->eq, iog); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +/* * Put the reference taken at the time of creation so that when all * queues are gone, group can be destroyed. @@ -2374,7 +2559,13 @@ static struct io_group *io_alloc_root_grstatic void io_free_root_group (struct elevator_queue *e) {struct io_group *iog = e->efqd->root_group; +struct io_service_tree *st; +int i; +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + }put_io_group_queues (e, iog);kfree (iog); }@@ -3257,6 +3448,35 @@ done:elv_schedule_dispatch (q); } +/* +* The process associted with ioq (in case of cfq), is going away. Mark it +* for deletion. +*/ +void elv_exit_ioq (struct io_queue *ioq) +{ +struct io_entity *entity = &ioq->entity; + +/* +* Async ioq's belong to io group and are cleaned up once group is +* being deleted. Not need to do any cleanup here even if cfq has +* dropped the reference to the queue +*/ +if (!elv_ioq_sync (ioq)) +return ; + +/* +* This queue is still under service. Just mark it so that once all +* the IO from queue is done, it is not put back in idle tree. +*/ +if (entity->on_st) { + entity->exiting =1 ; +return ; + }else if (entity->on_idle_st) { +/* Remove ioq from idle tree */ +dequeue_io_entity_idle (entity); + } +} +EXPORT_SYMBOL (elv_exit_ioq);static void elv_slab_kill (void ) {/* Index:
linux16/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +2
-0@@ -1138,6 +1138,7 @@ static void cfq_exit_cfqq(struct cfq_datelv_schedule_dispatch (cfqd->queue); } +elv_exit_ioq (cfqq->ioq);cfq_put_queue (cfqq); }@@ -1373,6 +1374,7 @@ static void changed_cgroup(struct io_con*/ if (iog != __iog) {cic_set_cfqq (cic,NULL ,1 ); +elv_exit_ioq (sync_cfqq->ioq);cfq_put_queue (sync_cfqq); } }Index:
linux16/block/elevator-fq.h
===================================================================block/elevator-fq.h +8
-1@@ -33,6 +33,10 @@ struct io_service_tree {u64 min_vdisktime;struct rb_node *rb_leftmost;unsigned int nr_active; + +/* A cache of io entities which were served and expired */ +struct rb_root idle; +struct rb_node *rb_leftmost_idle; };struct io_sched_data {@@ -44,9 +48,12 @@ struct io_sched_data {struct io_entity {struct rb_node rb_node;int on_st; +int on_idle_st;u64 vdisktime;unsigned int weight;struct io_entity *parent; +/* This io entity (queue or group) has been marked for deletion */ +unsigned int exiting;struct io_sched_data *my_sd;struct io_service_tree *st;@@ -572,7 +579,7 @@ extern struct io_queue *elv_alloc_ioq(stextern void elv_free_ioq (struct io_queue *ioq);extern struct io_group *ioq_to_io_group (struct io_queue *ioq);extern int elv_iog_should_idle (struct io_queue *ioq); - +extern void elv_exit_ioq (struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ static inline struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e)
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
Quoted 53 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 09:32:05PM -0400Quoted 30 lines · Fabio Checconi
Hi,
Quoted 5 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 06:28:27PM -0400o I found an issue during test and that is if there is a mix of queue and group
...
Quoted 13 lines
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree
min_vtimewrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
Quoted 8 lines
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
Quoted 7 lines
just a little note: this patch seems to introduce a special case for
vdisktime = 0, assigning it the meaning of "bad timestamp," but the virtual time space wraps, so 0 is a perfectly legal value, which can be reached by service. I have no idea if it can produce visible effects, but it doesn't seem to be correct.Hi Fabio,
You are right that technically during wrap arounds one can hit value 0 as legal value. But I think it is hard to hit at the same time, the only side affect of it will be that a queue will be either placed favorably (in case of sync queues) or at the end of tree (if it is async queue).
Async queues anyway go at the end after every dispatch round. So only side affect is that once during wrap around cycle a sync queue will be placed favorably and can gain share once in a dispatch round.
Quoted 6 lines
I think it is not a big issue at this point of time. But if it becomes significant, I can introduce a new variable or start passing function parameter to denote whether we found the queue in cache or not.
But if you think that it is absolutely no no, let me know....
I don't think it's an issue at all, just wanted to make sure it gets noticed, because timestamping bugs may be hard to hit but often are hard to debug. Maybe it deserves a line of comment...
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
Quoted 53 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 09:32:05PM -0400Quoted 30 lines · Fabio Checconi
Hi,
Quoted 5 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 06:28:27PM -0400o I found an issue during test and that is if there is a mix of queue and group
...
Quoted 13 lines
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree
min_vtimewrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
Quoted 8 lines
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
Quoted 7 lines
just a little note: this patch seems to introduce a special case for
vdisktime = 0, assigning it the meaning of "bad timestamp," but the virtual time space wraps, so 0 is a perfectly legal value, which can be reached by service. I have no idea if it can produce visible effects, but it doesn't seem to be correct.Hi Fabio,
You are right that technically during wrap arounds one can hit value 0 as legal value. But I think it is hard to hit at the same time, the only side affect of it will be that a queue will be either placed favorably (in case of sync queues) or at the end of tree (if it is async queue).
Async queues anyway go at the end after every dispatch round. So only side affect is that once during wrap around cycle a sync queue will be placed favorably and can gain share once in a dispatch round.
Quoted 6 lines
I think it is not a big issue at this point of time. But if it becomes significant, I can introduce a new variable or start passing function parameter to denote whether we found the queue in cache or not.
But if you think that it is absolutely no no, let me know....
I don't think it's an issue at all, just wanted to make sure it gets noticed, because timestamping bugs may be hard to hit but often are hard to debug. Maybe it deserves a line of comment...
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 25/23] io-controller: fix queue vs group fairness
Quoted 30 lines · Fabio Checconi
Hi,
Quoted 5 lines
From: Vivek Goyal <[email protected]>
Date: Tue, Sep 08, 2009 06:28:27PM -0400o I found an issue during test and that is if there is a mix of queue and group
...
Quoted 13 lines
So we need to keep track of process io queue's vdisktime, even it after got deleted from io scheduler's service tree and use that same vdisktime if that queue gets backlogged again. But trusting a ioq's vdisktime is bad because it can lead to issues if a service tree
min_vtimewrap around takes place between two requests of the queue. (Agreed that it can be not that easy to hit but it is possible).Hence, keep a cache of io queues serviced recently and when a queue gets backlogged, if it is found in cache, use that vdisktime otherwise assign a new vdisktime. This cache of io queues (idle tree), is basically the idea implemented by BFQ guys. I had gotten rid of idle trees in V9 and now I am bringing it back. (Now I understand it better. :-)).
Quoted 8 lines
There is one good side affect of keeping the cache of recently service io queues. Now CFQ can differentiate between streaming readers and new processes doing IO. Now for a new queue (which is not in the cache), we can assign a lower vdisktime and for a streaming reader, we assign vdisktime based on disk time used. This way small file readers or the processes doing small amount of IO will have reduced latencies at the cost of little reduced throughput of streaming readers.
Quoted 7 lines
just a little note: this patch seems to introduce a special case for
vdisktime = 0, assigning it the meaning of "bad timestamp," but the virtual time space wraps, so 0 is a perfectly legal value, which can be reached by service. I have no idea if it can produce visible effects, but it doesn't seem to be correct.
Hi Fabio,
You are right that technically during wrap arounds one can hit value 0 as legal value. But I think it is hard to hit at the same time, the only side affect of it will be that a queue will be either placed favorably (in case of sync queues) or at the end of tree (if it is async queue).
Async queues anyway go at the end after every dispatch round. So only side affect is that once during wrap around cycle a sync queue will be placed favorably and can gain share once in a dispatch round.
I think it is not a big issue at this point of time. But if it becomes significant, I can introduce a new variable or start passing function parameter to denote whether we found the queue in cache or not.
But if you think that it is absolutely no no, let me know....
Thanks
Vivek
Quoted 418 lines
Quoted 418 lines
Signed-off-by Vivek Goyal <[email protected]>--- block/cfq-iosched.c | 2 block/elevator-fq.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++---- block/elevator-fq.h | 9 + 3 files changed, 246 insertions(+), 17 deletions(-)Index:
linux16/block/elevator-fq.c
===================================================================block/elevator-fq.c +236
-16@@ -52,6 +52,8 @@ static struct kmem_cache *elv_ioq_pool; #define elv_log_entity(entity, fmt, args...)#endif +static void check_idle_tree_release (struct io_service_tree *st); +static inline struct io_queue *ioq_of (struct io_entity *entity) {if (entity->my_sd ==NULL )@@ -109,6 +111,11 @@ elv_prio_to_slice(struct elv_fq_data *efreturn elv_weight_slice (efqd,elv_ioq_sync (ioq), ioq->entity.weight); } +static inline int vdisktime_gt (u64 a,u64 b) +{ +return (s64)(a - b) >0 ; +} +static inline u64 max_vdisktime (u64 min_vdisktime,u64 vdisktime) {s64 delta = (s64)(vdisktime - min_vdisktime);@@ -145,6 +152,7 @@ static void update_min_vdisktime(struct } st->min_vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); +check_idle_tree_release (st); }static inline struct io_entity *parent_entity (struct io_entity *entity)@@ -411,27 +419,46 @@ static void place_entity(struct io_servistruct rb_node *parent;struct io_entity *entry;int nr_active = st->nr_active -1 ; +struct io_queue *ioq =ioq_of (entity); +int sync =1 ; + +if (ioq) + sync =elv_ioq_sync (ioq); + +if (add_front || !nr_active) { + vdisktime = st->min_vdisktime; + goto done; + } + +if (sync && entity->vdisktime + &&vdisktime_gt (entity->vdisktime, st->min_vdisktime)) { +/* vdisktime still in future. Use old vdisktime */ + vdisktime = entity->vdisktime; + goto done; + }/* -* Currently put entity at the end of last entity. This probably will -* require adjustments as we move along +* Effectively a new queue. Assign sync queue a lower vdisktime so +* we can achieve better latencies for small file readers. For async +* queues, put them at the end of the existing queue. +* Group entities are always considered sync. */ -if (io_entity_class_idle (entity)) { - vdisktime =elv_delta_fair (ELV_IDLE_DELAY , entity); - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime += entry->vdisktime; - } - }else if (!add_front && nr_active) { - parent =rb_last (&st->active); -if (parent) { - entry = rb_entry(parent, struct io_entity, rb_node); - vdisktime = entry->vdisktime; - } - }else +if (sync) { vdisktime = st->min_vdisktime; + goto done; + } +/* +* Put entity at the end of the tree. Effectively async queues use +* this path. +*/ + parent =rb_last (&st->active); +if (parent) { + entry = rb_entry(parent, struct io_entity, rb_node); + vdisktime = entry->vdisktime; + }else + vdisktime = st->min_vdisktime; +done: entity->vdisktime =max_vdisktime (st->min_vdisktime, vdisktime); elv_log_entity(entity, "place_entity: vdisktime=%llu" " min_vdisktime=%llu", entity->vdisktime,@@ -447,6 +474,122 @@ static inline void io_entity_update_prio*/ init_io_entity_service_tree (entity,parent_entity (entity)); entity->ioprio_changed =0 ; + +/* +* Assign this entity a fresh vdisktime instead of using +* previous one as prio class will lead to service tree +* change and this vdisktime will not be valid on new +* service tree. +* +* TODO: Handle the case of only prio change. +*/ + entity->vdisktime =0 ; + } +} + +static void +__dequeue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +if (st->rb_leftmost_idle == &entity->rb_node) { +struct rb_node *next_node; + + next_node =rb_next (&entity->rb_node); + st->rb_leftmost_idle = next_node; + } + +rb_erase (&entity->rb_node, &st->idle); +RB_CLEAR_NODE (&entity->rb_node); +} + +static void dequeue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); + +__dequeue_io_entity_idle (entity->st, entity); + entity->on_idle_st =0 ; +if (ioq) +elv_put_ioq (ioq); +} + +static void +__enqueue_io_entity_idle (struct io_service_tree *st,struct io_entity *entity) +{ +struct rb_node **node = &st->idle.rb_node; +struct rb_node *parent =NULL ; +struct io_entity *entry; +int leftmost =1 ; + +while (*node !=NULL ) { + parent = *node; + entry = rb_entry(parent, struct io_entity, rb_node); + +if (vdisktime_gt (entry->vdisktime, entity->vdisktime)) + node = &parent->rb_left; +else { + node = &parent->rb_right; + leftmost =0 ; + } + } + +/* +* Maintain a cache of leftmost tree entries (it is frequently +* used) +*/ +if (leftmost) + st->rb_leftmost_idle = &entity->rb_node; + +rb_link_node (&entity->rb_node, parent, node); +rb_insert_color (&entity->rb_node, &st->idle); +} + +static void enqueue_io_entity_idle (struct io_entity *entity) +{ +struct io_queue *ioq =ioq_of (entity); +struct io_group *parent_iog; + +/* +* Don't put an entity on idle tree if it has been marked for deletion. +* We are not expecting more io from this entity. No need to cache it +*/ + +if (entity->exiting) +return ; + +/* +* If parent group is exiting, don't put on idle tree. May be task got +* moved to a different cgroup and original cgroup got deleted +*/ + parent_iog =iog_of (parent_entity (entity)); +if (parent_iog->entity.exiting) +return ; + +if (ioq) +elv_get_ioq (ioq); +__enqueue_io_entity_idle (entity->st, entity); + entity->on_idle_st =1 ; +} + +static void check_idle_tree_release (struct io_service_tree *st) +{ +struct io_entity *leftmost; + +if (!st->rb_leftmost_idle) +return ; + + leftmost = rb_entry(st->rb_leftmost_idle, struct io_entity, rb_node); + +if (vdisktime_gt (st->min_vdisktime, leftmost->vdisktime)) +dequeue_io_entity_idle (leftmost); +} + +static void flush_idle_tree (struct io_service_tree *st) +{ +struct io_entity *entity; + +while (st->rb_leftmost_idle) { + entity = rb_entry(st->rb_leftmost_idle, struct io_entity, + rb_node); +dequeue_io_entity_idle (entity); } }@@ -483,6 +626,9 @@ static void dequeue_io_entity(struct io_ st->nr_active--; sd->nr_active--;debug_update_stats_dequeue (entity); + +if (vdisktime_gt (entity->vdisktime, st->min_vdisktime)) +enqueue_io_entity_idle (entity); } static void@@ -524,6 +670,16 @@ static void enqueue_io_entity(struct io_struct io_service_tree *st;struct io_sched_data *sd =io_entity_sched_data (entity); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +else +/* +* This entity was not in idle tree cache. Zero out vdisktime +* so that we don't rely on old vdisktime instead assign a +* fresh one. +*/ + entity->vdisktime =0 ; +io_entity_update_prio (entity); st = entity->st; st->nr_active++;@@ -574,6 +730,8 @@ static void requeue_io_entity(struct io_struct io_service_tree *st = entity->st;struct io_entity *next_entity; + entity->vdisktime =0 ; +if (add_front) { next_entity =__lookup_next_io_entity (st);@@ -1937,11 +2095,18 @@ static void io_free_root_group(struct el {struct io_group *iog = e->efqd->root_group;struct io_cgroup *iocg = &io_root_cgroup; +struct io_service_tree *st; +int i;spin_lock_irq (&iocg->lock);hlist_del_rcu (&iog->group_node);spin_unlock_irq (&iocg->lock); +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +put_io_group_queues (e, iog);elv_put_iog (iog); }@@ -2039,9 +2204,29 @@ EXPORT_SYMBOL(elv_put_iog);*/ static void __io_destroy_group (struct elv_fq_data *efqd,struct io_group *iog) { +struct io_service_tree *st; +int i; +struct io_entity *entity = &iog->entity; + +/* +* Mark io group for deletion so that no new entry goes in +* idle tree. Any active queue which is removed from active +* tree will not be put in to idle tree. +*/ + entity->exiting =1 ; + +/* We flush idle tree now, and don't put things in there any more. */ +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + } +hlist_del (&iog->elv_data_node);put_io_group_queues (efqd->eq, iog); +if (entity->on_idle_st) +dequeue_io_entity_idle (entity); +/* * Put the reference taken at the time of creation so that when all * queues are gone, group can be destroyed. @@ -2374,7 +2559,13 @@ static struct io_group *io_alloc_root_grstatic void io_free_root_group (struct elevator_queue *e) {struct io_group *iog = e->efqd->root_group; +struct io_service_tree *st; +int i; +for (i =0 ; i <IO_IOPRIO_CLASSES ; i++) { + st = iog->sched_data.service_tree + i; +flush_idle_tree (st); + }put_io_group_queues (e, iog);kfree (iog); }@@ -3257,6 +3448,35 @@ done:elv_schedule_dispatch (q); } +/* +* The process associted with ioq (in case of cfq), is going away. Mark it +* for deletion. +*/ +void elv_exit_ioq (struct io_queue *ioq) +{ +struct io_entity *entity = &ioq->entity; + +/* +* Async ioq's belong to io group and are cleaned up once group is +* being deleted. Not need to do any cleanup here even if cfq has +* dropped the reference to the queue +*/ +if (!elv_ioq_sync (ioq)) +return ; + +/* +* This queue is still under service. Just mark it so that once all +* the IO from queue is done, it is not put back in idle tree. +*/ +if (entity->on_st) { + entity->exiting =1 ; +return ; + }else if (entity->on_idle_st) { +/* Remove ioq from idle tree */ +dequeue_io_entity_idle (entity); + } +} +EXPORT_SYMBOL (elv_exit_ioq);static void elv_slab_kill (void ) {/* Index:
linux16/block/cfq-iosched.c
===================================================================block/cfq-iosched.c +2
-0@@ -1138,6 +1138,7 @@ static void cfq_exit_cfqq(struct cfq_datelv_schedule_dispatch (cfqd->queue); } +elv_exit_ioq (cfqq->ioq);cfq_put_queue (cfqq); }@@ -1373,6 +1374,7 @@ static void changed_cgroup(struct io_con*/ if (iog != __iog) {cic_set_cfqq (cic,NULL ,1 ); +elv_exit_ioq (sync_cfqq->ioq);cfq_put_queue (sync_cfqq); } }Index:
linux16/block/elevator-fq.h
===================================================================block/elevator-fq.h +8
-1@@ -33,6 +33,10 @@ struct io_service_tree {u64 min_vdisktime;struct rb_node *rb_leftmost;unsigned int nr_active; + +/* A cache of io entities which were served and expired */ +struct rb_root idle; +struct rb_node *rb_leftmost_idle; };struct io_sched_data {@@ -44,9 +48,12 @@ struct io_sched_data {struct io_entity {struct rb_node rb_node;int on_st; +int on_idle_st;u64 vdisktime;unsigned int weight;struct io_entity *parent; +/* This io entity (queue or group) has been marked for deletion */ +unsigned int exiting;struct io_sched_data *my_sd;struct io_service_tree *st;@@ -572,7 +579,7 @@ extern struct io_queue *elv_alloc_ioq(stextern void elv_free_ioq (struct io_queue *ioq);extern struct io_group *ioq_to_io_group (struct io_queue *ioq);extern int elv_iog_should_idle (struct io_queue *ioq); - +extern void elv_exit_ioq (struct io_queue *ioq);#else /* CONFIG_ELV_FAIR_QUEUING */ static inline struct elv_fq_data *elv_alloc_fq_data (struct request_queue *q,struct elevator_queue *e)
Rik van Riel
Acked-byRe: [PATCH 25/23] io-controller: fix queue vs group fairness
In reply to Vivek GoyalQuoted 3 lines · Vivek Goyal
o I found an issue during test and that is if there is a mix of queue and group at same level, there can be fairness issue. For example, consider following case.
(assuming the cleanups are done in V10 - the artifact of
a wraparound hitting exactly 0 should not be an issue)
Signature · 1 line
All rights reversed.
[PATCH 26/23] io-controller: fix writer preemption with in a group
In reply to Vivek Goyalo Found another issue during testing. Consider following hierarchy.
root / \ R1 G1 /\ R2 W
Generally in CFQ when readers and writers are running, reader immediately preempts writers and hence reader gets the better bandwidth. In case of hierarchical setup, it becomes little more tricky. In above diagram, G1 is a group and R1, R2 are readers and W is writer tasks.
Now assume W runs and then R1 runs and then R2 runs. After R2 has used its time slice, if R1 is schedule in, after couple of ms, R1 will get backlogged again in group G1, (streaming reader). But it will not preempt R1 as R1 is also a reader and also because preemption across group is not allowed for isolation reasons. Hence R2 will get backlogged in G1 and will get a vdisktime much higher than W. So when G2 gets scheduled again, W will get to run its full slice length despite the fact R2 is queue on same service tree.
The core issue here is that apart from regular preemptions (preemption across classes), CFQ also has this special notion of preemption with-in class and that can lead to issues active task is running in a differnt group than where new queue gets backlogged.
To solve the issue keep a track of this event (I am calling it late preemption). When a group becomes eligible to run again, if late_preemption is set, check if there are sync readers backlogged, and if yes, expire the writer after one round of dispatch.
This solves the issue of reader not getting enough bandwidth in hierarchical setups.
--- block/elevator-fq.c | 125 ++++++++++++++++++++++++++++++++++++++++++++++------ block/elevator-fq.h | 2 2 files changed, 114 insertions(+), 13 deletions(-)
block/elevator-fq.h +2 -0
@@ -43,6 +43,7 @@ struct io_sched_data {struct io_entity *active_entity;int nr_active;struct io_service_tree service_tree[IO_IOPRIO_CLASSES ]; +int nr_sync; };struct io_entity {
@@ -150,6 +151,7 @@ struct io_group {unsigned long state;/* request list associated with the group */ struct request_list rl; +int late_preemption; };struct io_policy_node {
block/elevator-fq.c +112 -13
226 lines not on this page · raw message
Rik van Riel
Acked-byRe: [PATCH 26/23] io-controller: fix writer preemption with in a group
Quoted 32 lines · Vivek Goyal
o Found another issue during testing. Consider following hierarchy.
root / \ R1 G1 /\ R2 WGenerally in CFQ when readers and writers are running, reader immediately preempts writers and hence reader gets the better bandwidth. In case of hierarchical setup, it becomes little more tricky. In above diagram, G1 is a group and R1, R2 are readers and W is writer tasks.
Now assume W runs and then R1 runs and then R2 runs. After R2 has used its time slice, if R1 is schedule in, after couple of ms, R1 will get backlogged again in group G1, (streaming reader). But it will not preempt R1 as R1 is also a reader and also because preemption across group is not allowed for isolation reasons. Hence R2 will get backlogged in G1 and will get a vdisktime much higher than W. So when G2 gets scheduled again, W will get to run its full slice length despite the fact R2 is queue on same service tree.
The core issue here is that apart from regular preemptions (preemption across classes), CFQ also has this special notion of preemption with-in class and that can lead to issues active task is running in a differnt group than where new queue gets backlogged.
To solve the issue keep a track of this event (I am calling it late preemption). When a group becomes eligible to run again, if
late_preemptionis set, check if there are sync readers backlogged, and if yes, expire the writer after one round of dispatch.
Quoted 4 lines
This solves the issue of reader not getting enough bandwidth in hierarchical setups.
Signed-off-by Vivek Goyal <[email protected]>
Conceptually a nice solution. The code gets a little tricky, but I guess any code dealing with these situations would end up that way :)
Signature · 1 line
All rights reversed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Rik van Riel
Acked-byRe: [PATCH 26/23] io-controller: fix writer preemption with in a group
Quoted 32 lines · Vivek Goyal
o Found another issue during testing. Consider following hierarchy.
root / \ R1 G1 /\ R2 WGenerally in CFQ when readers and writers are running, reader immediately preempts writers and hence reader gets the better bandwidth. In case of hierarchical setup, it becomes little more tricky. In above diagram, G1 is a group and R1, R2 are readers and W is writer tasks.
Now assume W runs and then R1 runs and then R2 runs. After R2 has used its time slice, if R1 is schedule in, after couple of ms, R1 will get backlogged again in group G1, (streaming reader). But it will not preempt R1 as R1 is also a reader and also because preemption across group is not allowed for isolation reasons. Hence R2 will get backlogged in G1 and will get a vdisktime much higher than W. So when G2 gets scheduled again, W will get to run its full slice length despite the fact R2 is queue on same service tree.
The core issue here is that apart from regular preemptions (preemption across classes), CFQ also has this special notion of preemption with-in class and that can lead to issues active task is running in a differnt group than where new queue gets backlogged.
To solve the issue keep a track of this event (I am calling it late preemption). When a group becomes eligible to run again, if
late_preemptionis set, check if there are sync readers backlogged, and if yes, expire the writer after one round of dispatch.
Quoted 4 lines
This solves the issue of reader not getting enough bandwidth in hierarchical setups.
Signed-off-by Vivek Goyal <[email protected]>
Conceptually a nice solution. The code gets a little tricky, but I guess any code dealing with these situations would end up that way :)
Signature · 1 line
All rights reversed.
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:
$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.
Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_caches
Results with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DB
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5s
As you can see, there is no significant difference for CFQ scheduler. There is big improvement for noop and deadline schedulers (why is that happening?). The performance with anticipatory scheduler is a bit lower (~4%).
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6s
The result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0s
Here again, the result in term of fairness is very close from what we expect.
Thanks,
Jerome
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DB
Quoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).
I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 7 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6s
Quoted 5 lines
The result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.
Quoted 9 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
Quoted 48 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Thanks
Vivek
Quoted 50 lines
I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 7 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6sQuoted 5 lines
The result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.Quoted 9 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
Quoted 57 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).
Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Jerome
Quoted 48 lines
Thanks
VivekQuoted 45 lines
I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 11 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6sThe result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.Quoted 8 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
Quoted 68 lines · Jerome Marchand
Quoted 57 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Thanks
Vivek
Quoted 83 lines · Vivek Goyal
Quoted 68 lines · Jerome Marchand
Quoted 57 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 8 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Does your hard drive support command queuing? May be we are driving deeper queue depths for reads and during queue switch we will wait for requests to finish from last queue to finish before next queue is scheduled in (for AS) and that probably will cause more delay if we are driving deeper queue depth.
Can you please set queue depth to "1" (/sys/block/<disk>/device/queue_depth) on this disk and see time consumed in two cases are same or different. I think setting depth to "1" will bring down overall throughput but if times are same in two cases, at least we will know where the delay is coming from.
Thanks
Vivek
Quoted 96 lines · Vivek Goyal
Quoted 80 lines · Vivek Goyal
Quoted 66 lines · Jerome Marchand
Quoted 56 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 8 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Does your hard drive support command queuing? May be we are driving deeper queue depths for reads and during queue switch we will wait for requests to finish from last queue to finish before next queue is scheduled in (for AS) and that probably will cause more delay if we are driving deeper queue depth.
Quoted 7 lines
Can you please set queue depth to "1" (/sys/block/<disk>/device/queue_depth) on this disk and see time consumed in two cases are same or different. I think setting depth to "1" will bring down overall throughput but if times are same in two cases, at least we will know where the delay is coming from.
Thanks
Vivek
It looks like command queuing is supported but disabled. Queue depth is already 1 and the file /sys/block/<disk>/device/queue_depth is read-only.
Jerome
Quoted 102 lines · Jerome Marchand
Quoted 101 lines · Vivek Goyal
Quoted 80 lines · Vivek Goyal
Quoted 66 lines · Jerome Marchand
Quoted 56 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 8 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Does your hard drive support command queuing? May be we are driving deeper queue depths for reads and during queue switch we will wait for requests to finish from last queue to finish before next queue is scheduled in (for AS) and that probably will cause more delay if we are driving deeper queue depth.
Can you please set queue depth to "1" (/sys/block/<disk>/device/queue_depth) on this disk and see time consumed in two cases are same or different. I think setting depth to "1" will bring down overall throughput but if times are same in two cases, at least we will know where the delay is coming from.
Quoted 5 lines
Quoted 2 lines
Thanks
VivekIt looks like command queuing is supported but disabled. Queue depth is already 1 and the file /sys/block/<disk>/device/queue_depth is read-only.
Hmm..., time to run blktraces and in both the cases and compare the two cases and see what's the issue.
Would be great if you can capture and look at traces. Otherwise I will try to do it sometime soon..
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 102 lines · Jerome Marchand
Quoted 101 lines · Vivek Goyal
Quoted 80 lines · Vivek Goyal
Quoted 66 lines · Jerome Marchand
Quoted 56 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 8 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Does your hard drive support command queuing? May be we are driving deeper queue depths for reads and during queue switch we will wait for requests to finish from last queue to finish before next queue is scheduled in (for AS) and that probably will cause more delay if we are driving deeper queue depth.
Can you please set queue depth to "1" (/sys/block/<disk>/device/queue_depth) on this disk and see time consumed in two cases are same or different. I think setting depth to "1" will bring down overall throughput but if times are same in two cases, at least we will know where the delay is coming from.
Quoted 5 lines
Quoted 2 lines
Thanks
VivekIt looks like command queuing is supported but disabled. Queue depth is already 1 and the file /sys/block/<disk>/device/queue_depth is read-only.
Hmm..., time to run blktraces and in both the cases and compare the two cases and see what's the issue.
Would be great if you can capture and look at traces. Otherwise I will try to do it sometime soon..
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 83 lines · Vivek Goyal
Quoted 68 lines · Jerome Marchand
Quoted 57 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 8 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Does your hard drive support command queuing? May be we are driving deeper queue depths for reads and during queue switch we will wait for requests to finish from last queue to finish before next queue is scheduled in (for AS) and that probably will cause more delay if we are driving deeper queue depth.
Can you please set queue depth to "1" (/sys/block/<disk>/device/queue_depth) on this disk and see time consumed in two cases are same or different. I think setting depth to "1" will bring down overall throughput but if times are same in two cases, at least we will know where the delay is coming from.
Thanks
Vivek
Quoted 81 lines · Vivek Goyal
Quoted 66 lines · Jerome Marchand
Quoted 56 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 7 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Yes it's correct. I misunderstood you.
Jerome
Quoted 3 lines
Thanks
Vivek
Quoted 81 lines · Vivek Goyal
Quoted 66 lines · Jerome Marchand
Quoted 56 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
Quoted 7 lines
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Yes it's correct. I misunderstood you.
Jerome
Quoted 3 lines
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 68 lines · Jerome Marchand
Quoted 57 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Quoted 6 lines
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Hi Jerome,
Ok, so when both the DB's are in root group (with io-controller V9 patches), then you get 46.6 seconds time for both the DBs. That means there is no regression in this case. In this case there is only one queue of root group and AS is running timed read/write batches on this queue.
But when both the DBs are put in separate groups then you get 48.3 and 48.6 seconds respectively and we see regression. In this case there are two queues belonging to each group. Elevator layer takes care of queue group queue switch and AS runs timed read/write batches on these queues.
If it is correct, then it does not exclude the possiblity that it is queue switching overhead between groups?
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 57 lines · Vivek Goyal
Quoted 46 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).
Quoted 9 lines
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Hi Vivek,
I guess that's not the reason. I got 46.6s for both DB in root group with io-controller v9 patches. I also rerun the test with DB in different groups and found about the same result as above (48.3s and 48.6s).
Jerome
Quoted 48 lines
Thanks
VivekQuoted 45 lines
I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 11 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6sThe result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.Quoted 8 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 48 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
Quoted 9 lines
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).
Hi Jerome,
Can you also run the AS test with io controller patches and both the database in root group (basically don't put them in to separate group). I suspect that this regression might come from that fact that we now have to switch between queues and in AS we wait for request to finish from previous queue before next queue is scheduled in and probably that is slowing down things a bit.., just a wild guess..
Thanks
Vivek
Quoted 50 lines
I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 7 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6sQuoted 5 lines
The result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.Quoted 9 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
Quoted 73 lines · Vivek Goyal
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBQuoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 11 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6sThe result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Quoted 3 lines
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
I know and took it into account. Let me detail my calculations.
Both request are of the same size and takes alone a time T to complete (about 22.5s in our example). For sake of simplification, let's ignore switching cost. Then, the completion time of both requests running at the same time would be 2T, whatever are their weights or classes. If one group weights 1000 and the other 500 (resp. 250), the first group gets 2/3 (4/5) of the bandwidth as long as it is running, and thus finished in T/(2/3) = 2T*3/4 (resp. T/(4/5) = 2T*5/8 ) that is 3/4 (5/8) of the total time. The other always finish in about 2T.
The actual results above are pretty closed to these theoretical values and that how I concluded that the controller is pretty fair.
Quoted 6 lines
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.
Actually, I already check that and the result was good but I didn't keep the output, so I just rerun the (1000,500) weights. First column is the time spent by first group since last refresh (refresh period is 2s). The second column is the same for second group. The group test3 is not used. The first "ratios" column is the ratio between io time spent by first group and time spent by second group.
$ ./watch_cgroup.sh 2 test1: 0 test2: 0 test3: 0 ratios: -- -- -- test1: 805 test2: 301 test3: 0 ratios: 2.67441860465116279069 -- -- test1: 1209 test2: 714 test3: 0 ratios: 1.69327731092436974789 -- -- test1: 1306 test2: 503 test3: 0 ratios: 2.59642147117296222664 -- -- test1: 1210 test2: 604 test3: 0 ratios: 2.00331125827814569536 -- -- test1: 1207 test2: 605 test3: 0 ratios: 1.99504132231404958677 -- -- test1: 1209 test2: 605 test3: 0 ratios: 1.99834710743801652892 -- -- test1: 1206 test2: 606 test3: 0 ratios: 1.99009900990099009900 -- -- test1: 1109 test2: 607 test3: 0 ratios: 1.82701812191103789126 -- -- test1: 1213 test2: 603 test3: 0 ratios: 2.01160862354892205638 -- -- test1: 1214 test2: 608 test3: 0 ratios: 1.99671052631578947368 -- -- test1: 1211 test2: 603 test3: 0 ratios: 2.00829187396351575456 -- -- test1: 1110 test2: 603 test3: 0 ratios: 1.84079601990049751243 -- -- test1: 1210 test2: 605 test3: 0 ratios: 2.00000000000000000000 -- -- test1: 1211 test2: 601 test3: 0 ratios: 2.01497504159733777038 -- -- test1: 1210 test2: 607 test3: 0 ratios: 1.99341021416803953871 -- -- test1: 1204 test2: 604 test3: 0 ratios: 1.99337748344370860927 -- -- test1: 1207 test2: 605 test3: 0 ratios: 1.99504132231404958677 -- -- test1: 1089 test2: 708 test3: 0 ratios: 1.53813559322033898305 -- -- test1: 0 test2: 2124 test3: 0 ratios: 0 -- -- test1: 0 test2: 1915 test3: 0 ratios: 0 -- -- test1: 0 test2: 1919 test3: 0 ratios: 0 -- -- test1: 0 test2: 2023 test3: 0 ratios: 0 -- -- test1: 0 test2: 1925 test3: 0 ratios: 0 -- -- test1: 0 test2: 705 test3: 0 ratios: 0 -- -- test1: 0 test2: 0 test3: 0 ratios: -- -- --
As you can see, the ratio stays close to 2 as long as first request is running.
Regards,
Jerome
Quoted 23 lines
Quoted 9 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 29 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DB
Quoted 7 lines
CFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sAs you can see, there is no significant difference for CFQ scheduler.
Thanks Jerome.
Quoted 2 lines
There is big improvement for noop and deadline schedulers (why is that happening?).
I think because now related IO is in a single queue and it gets to run for 100ms or so (like CFQ). So previously, IO from both the instances will go into a single queue which should lead to more seeks as requests from two groups will kind of get interleaved.
With io controller, both groups have separate queues so requests from both the data based instances will not get interleaved (This almost becomes like CFQ where ther are separate queues for each io context and for sequential reader, one io context gets to run nicely for certain ms based on its priority).
Quoted 3 lines
The performance with anticipatory scheduler
is a bit lower (~4%).
I will run some tests with AS and see if I can reproduce this lower performance and attribute it to a particular piece of code.
Quoted 7 lines
Results with 2 groups of same io policy (BE), different io weights and CFQ scheduler:
io-scheduler v8 io-scheduler v9
weights = 1000, 500 35.6s 46.7s 35.6s 46.7s
weigths = 1000, 250 29.2s 45.8s 29.2s 45.6s
Quoted 5 lines
The result in term of fairness is close to what we can expect from the ideal theoric case: with io weights of 1000 and 500 (1000 and 250), the first request get 2/3 (4/5) of io time as long as it runs and thus finish in about 3/4 (5/8) of total time.
Jerome, after 36.6 seconds, disk will be fully given to second group. Hence these times might not reflect the accurate measure of who got how much of disk time.
Can you just capture the output of "io.disk_time" file in both the cgroups at the time of completion of task in higher weight group. Alternatively, you can just run this a script in a loop which prints the output of
"cat io.disk_time | grep major:minor" every 2 seconds. That way we can
see how disk times are being distributed between groups.
Quoted 9 lines
Results with 2 groups of different io policies, same io weight and CFQ scheduler:
io-scheduler v8 io-scheduler v9
policy = RT, BE 22.5s 45.3s 22.4s 45.0s
policy = BE, IDLE 22.6s 44.8s 22.4s 45.0sHere again, the result in term of fairness is very close from what we expect.
Same as above in this case too.
These seem to be good test for fairness measurement in case of streaming readers. I think one more interesting test case will be do how are the random read latencies in case of multiple streaming readers present.
So if we can launch 4-5 dd processes in one group and then issue some random small queueries on postgresql in second group, I am keen to see how quickly the query can be completed with and without io controller. Would be interesting to see at results for all 4 io schedulers.
Thanks
Vivek
Quoted 34 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBCFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5s
Quoted 5 lines
As you can see, there is no significant difference for CFQ scheduler. There is big improvement for noop and deadline schedulers (why is that happening?). The performance with anticipatory scheduler is a bit lower (~4%).
Ok, I think what's happening here is that by default slice lenght for a queue is 100ms. When you put two instances of DB in two different groups, one streaming reader can run at max for 100ms at a go and then we switch to next reader.
But when both the readers are in root group, then AS lets run one reader to run at max 250ms (sometimes 125ms and sometimes 250ms based on at what time as_fifo_expired() was invoked).
So because a reader gets to run longer at one stretch in root group, it reduces number of seeks and leads to little enhanced throughput.
If you change the /sys/block/<disk>/queue/iosched/slice_sync to 250 ms, then one group queue can run at max for 250ms before we switch the queue. In this case you should be able to get same performance as in root group.
Thanks
Vivek
Quoted 34 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBCFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5s
Quoted 5 lines
As you can see, there is no significant difference for CFQ scheduler. There is big improvement for noop and deadline schedulers (why is that happening?). The performance with anticipatory scheduler is a bit lower (~4%).
Ok, I think what's happening here is that by default slice lenght for a queue is 100ms. When you put two instances of DB in two different groups, one streaming reader can run at max for 100ms at a go and then we switch to next reader.
But when both the readers are in root group, then AS lets run one reader to run at max 250ms (sometimes 125ms and sometimes 250ms based on at what time as_fifo_expired() was invoked).
So because a reader gets to run longer at one stretch in root group, it reduces number of seeks and leads to little enhanced throughput.
If you change the /sys/block/<disk>/queue/iosched/slice_sync to 250 ms, then one group queue can run at max for 250ms before we switch the queue. In this case you should be able to get same performance as in root group.
Thanks
Vivek
Quoted 53 lines · Vivek Goyal
Quoted 34 lines · Jerome Marchand
Quoted 3 lines · Vivek Goyal
Hi All,
Here is the V9 of the IO controller patches generated on top of 2.6.31-rc7.
Hi Vivek,
I've run some postgresql benchmarks for io-controller. Tests have been made with 2.6.31-rc6 kernel, without io-controller patches (when relevant) and with io-controller v8 and v9 patches.
I set up two instances of the TPC-H database, each running in their own io-cgroup. I ran two clients to these databases and tested on each that simple request:$ select count(*) from LINEITEM; where LINEITEM is the biggest table of TPC-H (6001215 entries, 720MB). That request generates a steady stream of IOs.Time is measure by psql (\timing switched on). Each test is run twice or more if there is any significant difference between the first two runs. Before each run, the cache is flush:
$ echo 3 > /proc/sys/vm/drop_cachesResults with 2 groups of same io policy (BE) and same io weight (1000):
w/o io-scheduler io-scheduler v8 io-scheduler v9
first second first second first second
DB DB DB DB DB DBCFQ 48.4s 48.4s 48.2s 48.2s 48.1s 48.5s
Noop 138.0s 138.0s 48.3s 48.4s 48.5s 48.8s
AS 46.3s 47.0s 48.5s 48.7s 48.3s 48.5s
Deadl. 137.1s 137.1s 48.2s 48.3s 48.3s 48.5sQuoted 5 lines
As you can see, there is no significant difference for CFQ scheduler. There is big improvement for noop and deadline schedulers (why is that happening?). The performance with anticipatory scheduler is a bit lower (~4%).
Ok, I think what's happening here is that by default slice lenght for a queue is 100ms. When you put two instances of DB in two different groups, one streaming reader can run at max for 100ms at a go and then we switch to next reader.
But when both the readers are in root group, then AS lets run one reader to run at max 250ms (sometimes 125ms and sometimes 250ms based on at what time
as_fifo_expired()was invoked).So because a reader gets to run longer at one stretch in root group, it reduces number of seeks and leads to little enhanced throughput.
Quoted 6 lines
If you change the /sys/block/<disk>/queue/iosched/slice_sync to 250 ms, then one group queue can run at max for 250ms before we switch the queue. In this case you should be able to get same performance as in root group.
Thanks
Vivek
Indeed. When I run the benchmark with slice_sync = 250ms, I get results close to the one for both instances running within the root group:
first group 46.1s and second group 46.4s.
Jerome