blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
Subsystems (2)
PATCH 1/1 blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
o Jeff Moyer was doing some testing on a RAM backed disk and blkiocg_lookup_group() showed up high overhead after memcpy(). Similarly somebody else reported that blkiocg_lookup_group() is eating 6% extra cpu. Though looking at the code I can't think why the overhead of this function is so high. One thing is that it is called with very high frequency (once for every IO).
o For lot of folks blkio controller will be compiled in but they might not have actually created cgroups. Hence optimize the case of root cgroup where we can avoid calling blkiocg_lookup_group() if IO is happening in root group (common case).
--- block/blk-throttle.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
block/blk-throttle.c +9 -1
@@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a* tree of blkg (instead of traversing through hash list all * the time. */ - tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key)); + +/* +* This is the common case when there are no blkio cgroups. +* Avoid lookup in this case +*/ +if (blkcg == &blkio_root_cgroup) + tg = &td->root_tg; +else + tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key));/* Fill in device details for root group */ if (tg && !tg->blkg.dev && bdi->dev &&dev_name (bdi->dev)) {
Quoted 7 lines · Vivek Goyal
o Jeff Moyer was doing some testing on a RAM backed disk and
blkiocg_lookup_group()showed up high overhead aftermemcpy(). Similarly somebody else reported thatblkiocg_lookup_group()is eating 6% extra cpu. Though looking at the code I can't think why the overhead of this function is so high. One thing is that it is called with very high frequency (once for every IO).
Quoted 7 lines
o For lot of folks blkio controller will be compiled in but they might not have actually created cgroups. Hence optimize the case of root cgroup where we can avoid calling
blkiocg_lookup_group()if IO is happening in root group (common case).Reported-by Jeff Moyer <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
In case it wasn't obvious, this patch was tested and confirmed to fix the performance degradation.
Cheers,
Jeff
Quoted 25 lines
--- block/blk-throttle.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)Index:
linux-2.6/block/blk-throttle.c
===================================================================block/blk-throttle.c +9
-1@@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a* tree of blkg (instead of traversing through hash list all * the time. */ - tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key)); + +/* +* This is the common case when there are no blkio cgroups. +* Avoid lookup in this case +*/ +if (blkcg == &blkio_root_cgroup) + tg = &td->root_tg; +else + tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key));/* Fill in device details for root group */ if (tg && !tg->blkg.dev && bdi->dev &&dev_name (bdi->dev)) {
Quoted 7 lines · Vivek Goyal
o Jeff Moyer was doing some testing on a RAM backed disk and
blkiocg_lookup_group()showed up high overhead aftermemcpy(). Similarly somebody else reported thatblkiocg_lookup_group()is eating 6% extra cpu. Though looking at the code I can't think why the overhead of this function is so high. One thing is that it is called with very high frequency (once for every IO).
Quoted 7 lines
o For lot of folks blkio controller will be compiled in but they might not have actually created cgroups. Hence optimize the case of root cgroup where we can avoid calling
blkiocg_lookup_group()if IO is happening in root group (common case).Reported-by Jeff Moyer <[email protected]>Signed-off-by Vivek Goyal <[email protected]>
Hi Jens,
Do you have any concerns regarding this patch?
Thanks
Vivek
Quoted 26 lines
--- block/blk-throttle.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)Index:
linux-2.6/block/blk-throttle.c
===================================================================block/blk-throttle.c +9
-1@@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a* tree of blkg (instead of traversing through hash list all * the time. */ - tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key)); + +/* +* This is the common case when there are no blkio cgroups. +* Avoid lookup in this case +*/ +if (blkcg == &blkio_root_cgroup) + tg = &td->root_tg; +else + tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key));/* Fill in device details for root group */ if (tg && !tg->blkg.dev && bdi->dev &&dev_name (bdi->dev)) {
Jeff Moyer
Acked-byQuoted 13 lines · Vivek Goyal
Quoted 12 lines · Vivek Goyal
o Jeff Moyer was doing some testing on a RAM backed disk and
blkiocg_lookup_group()showed up high overhead aftermemcpy(). Similarly somebody else reported thatblkiocg_lookup_group()is eating 6% extra cpu. Though looking at the code I can't think why the overhead of this function is so high. One thing is that it is called with very high frequency (once for every IO).o For lot of folks blkio controller will be compiled in but they might not have actually created cgroups. Hence optimize the case of root cgroup where we can avoid calling
blkiocg_lookup_group()if IO is happening in root group (common case).
Quoted 6 lines
Quoted 2 lines
Reported-by Jeff Moyer <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Hi Jens,
Do you have any concerns regarding this patch?
Jens, this is a pretty easy performance regression fixup. I think it should be pulled in sooner rather than later. We've also witnessed this slowdown on big performance testing rigs, so it's not just a ramdisk issue.
Cheers,
Jeff
Quoted 26 lines
Quoted 26 lines
--- block/blk-throttle.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)Index:
linux-2.6/block/blk-throttle.c
===================================================================block/blk-throttle.c +9
-1@@ -168,7 +168,15 @@ static struct throtl_grp * throtl_find_a* tree of blkg (instead of traversing through hash list all * the time. */ - tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key)); + +/* +* This is the common case when there are no blkio cgroups. +* Avoid lookup in this case +*/ +if (blkcg == &blkio_root_cgroup) + tg = &td->root_tg; +else + tg =tg_of_blkg (blkiocg_lookup_group (blkcg, key));/* Fill in device details for root group */ if (tg && !tg->blkg.dev && bdi->dev &&dev_name (bdi->dev)) {
Jens Axboe
maintainerQuoted 20 lines · Jeff Moyer
Quoted 18 lines · Vivek Goyal
Quoted 7 lines · Vivek Goyal
o Jeff Moyer was doing some testing on a RAM backed disk and
blkiocg_lookup_group()showed up high overhead aftermemcpy(). Similarly somebody else reported thatblkiocg_lookup_group()is eating 6% extra cpu. Though looking at the code I can't think why the overhead of this function is so high. One thing is that it is called with very high frequency (once for every IO).Quoted 7 lines
o For lot of folks blkio controller will be compiled in but they might not have actually created cgroups. Hence optimize the case of root cgroup where we can avoid calling
blkiocg_lookup_group()if IO is happening in root group (common case).Reported-by Jeff Moyer <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Hi Jens,
Quoted 8 lines
Quoted 1 line
Do you have any concerns regarding this patch?
Acked-by Jeff Moyer <[email protected]>Jens, this is a pretty easy performance regression fixup. I think it should be pulled in sooner rather than later. We've also witnessed this slowdown on big performance testing rigs, so it's not just a ramdisk issue.
Yep, will fold it into 2.6.38, thanks!
Signature · 1 line
Jens Axboe
Quoted 23 lines · Jens Axboe
Quoted 22 lines · Jeff Moyer
Quoted 19 lines · Vivek Goyal
Quoted 7 lines · Vivek Goyal
o Jeff Moyer was doing some testing on a RAM backed disk and
blkiocg_lookup_group()showed up high overhead aftermemcpy(). Similarly somebody else reported thatblkiocg_lookup_group()is eating 6% extra cpu. Though looking at the code I can't think why the overhead of this function is so high. One thing is that it is called with very high frequency (once for every IO).Quoted 7 lines
o For lot of folks blkio controller will be compiled in but they might not have actually created cgroups. Hence optimize the case of root cgroup where we can avoid calling
blkiocg_lookup_group()if IO is happening in root group (common case).Reported-by Jeff Moyer <[email protected]>Signed-off-by Vivek Goyal <[email protected]>Hi Jens,
Do you have any concerns regarding this patch?
Quoted 8 lines
Quoted 6 lines
Acked-by Jeff Moyer <[email protected]>Jens, this is a pretty easy performance regression fixup. I think it should be pulled in sooner rather than later. We've also witnessed this slowdown on big performance testing rigs, so it's not just a ramdisk issue.
Yep, will fold it into 2.6.38, thanks!
Just to be clear, it's queued up in for-linus so it will go to 2.6.38 on the next push. It was updated last week, this is what it has so far:
Sergey Senozhatsky (1):
loop: queue_lock NULL pointer derefence in blk_throtl_exit
Stephen M. Cameron (1):
cciss: make cciss_revalidate not loop through CISS_MAX_LUNS volumes unnecessarily.
Tao Ma (1):
blktrace: Don't output messages if NOTIFY isn't set.
Tracey Dent (2):
drivers/block/Makefile: replace the use of <module>-objs with <module>-y drivers/block/aoe/Makefile: replace the use of <module>-objs with <module>-y
Vivek Goyal (2):
cfq: rename a function to give it more appropriate name
blkio-throttle: Avoid calling blkiocg_lookup_group() for root group
block/blk-throttle.c | 10 +++++++++- block/cfq-iosched.c | 6 +++--- drivers/block/Makefile | 2 +- drivers/block/aoe/Makefile | 2 +- drivers/block/cciss.c | 2 +- drivers/block/loop.c | 3 +++ kernel/trace/blktrace.c | 7 +++++++ 7 files changed, 25 insertions(+), 7 deletions(-)
Signature · 1 line
Jens Axboe
Quoted 1 line · Jens Axboe
Quoted 1 line
Yep, will fold it into 2.6.38, thanks!
Awesome, thanks Jens!
-Jeff