block: Per cpu merge stats and cfq fix

PATCH In review Vivek Goyal 3 participants 9 messages May 21, 2011 – May 23, 2011

Subsystems (2)

Block layer cgroup blkio

lore

Share & tools

lore

Vivek Goyal

author

2011-05-21 00:04 UTC #0 lore raw

cover block: Per cpu merge stats and cfq fix

Hi Jens,

Here is the patch for making merge stats per cpu. While scanning the code I also noticed that in one case I was not freeing up per cpu stats. First patch fixes it. I also included Namhyung Kim's bio merge fix.

Thanks
Vivek

Vivek Goyal (3):
cfq-iosched: Fix a memory leak of per cpu stats for root group
cfq-iosched: Make IO merge related stats per cpu
block: call elv_bio_merged() when merged

 block/blk-cgroup.c  |   22 +++++++++++++++++-----
 block/blk-cgroup.h  |    4 ++--
 block/blk-core.c    |    2 ++
 block/cfq-iosched.c |    5 +++++
 4 files changed, 26 insertions(+), 7 deletions(-)

Signature · 1 line

1.7.4.4

Vivek Goyal

author

2011-05-21 00:03 UTC #1 lore raw

PATCH 1/3 cfq-iosched: Fix a memory leak of per cpu stats for root group

We allocated per cpu stats struct for root group but did not free it. Fix it.

Signed-off-by Vivek Goyal <[email protected]>

---
 block/cfq-iosched.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

block/cfq-iosched.c +5 -0

index d646b27..0316f6e 100644
@@ -3940,6 +3940,11 @@ static void cfq_exit_queue(struct elevator_queue *e)
 	 */
 	if (wait)
 		synchronize_rcu();
+
+#ifdef CONFIG_CFQ_GROUP_IOSCHED
+	/* Free up per cpu stats for root group */
+	free_percpu(cfqd->root_group.blkg.stats_cpu);
+#endif
 	kfree(cfqd);
 }

Signature · 1 line

1.7.4.4

Vivek Goyal

author

2011-05-21 00:03 UTC #2 lore raw

PATCH 2/3 cfq-iosched: Make IO merge related stats per cpu

In reply to Vivek Goyal

Make BLKIO_STAT_MERGED per cpu hence gettring rid of need of taking blkg->stats_lock.

Signed-off-by Vivek Goyal <[email protected]>

---
 block/blk-cgroup.c |   22 +++++++++++++++++-----
 block/blk-cgroup.h |    4 ++--
 2 files changed, 19 insertions(+), 7 deletions(-)

block/blk-cgroup.c +17 -5

index e41cc6f..07371cf 100644
@@ -441,15 +441,27 @@ void blkiocg_update_completion_stats(struct blkio_group *blkg,
 }
 EXPORT_SYMBOL_GPL(blkiocg_update_completion_stats);
 
+/*  Merged stats are per cpu.  */
 void blkiocg_update_io_merged_stats(struct blkio_group *blkg, bool direction,
 					bool sync)
 {
+	struct blkio_group_stats_cpu *stats_cpu;
 	unsigned long flags;
 
-	spin_lock_irqsave(&blkg->stats_lock, flags);
-	blkio_add_stat(blkg->stats.stat_arr[BLKIO_STAT_MERGED], 1, direction,
-			sync);
-	spin_unlock_irqrestore(&blkg->stats_lock, flags);
+	/*
+	 * Disabling interrupts to provide mutual exclusion between two
+	 * writes on same cpu. It probably is not needed for 64bit. Not
+	 * optimizing that case yet.
+	 */
+	local_irq_save(flags);
+
+	stats_cpu = this_cpu_ptr(blkg->stats_cpu);
+
+	u64_stats_update_begin(&stats_cpu->syncp);
+	blkio_add_stat(stats_cpu->stat_arr_cpu[BLKIO_STAT_CPU_MERGED], 1,
+				direction, sync);
+	u64_stats_update_end(&stats_cpu->syncp);
+	local_irq_restore(flags);
 }
 EXPORT_SYMBOL_GPL(blkiocg_update_io_merged_stats);

@@ -1244,7 +1256,7 @@ static int blkiocg_file_read_map(struct cgroup *cgrp, struct cftype *cft,
 						BLKIO_STAT_WAIT_TIME, 1, 0);
 		case BLKIO_PROP_io_merged:
 			return blkio_read_blkg_stats(blkcg, cft, cb,
-						BLKIO_STAT_MERGED, 1, 0);
+						BLKIO_STAT_CPU_MERGED, 1, 1);
 		case BLKIO_PROP_io_queued:
 			return blkio_read_blkg_stats(blkcg, cft, cb,
 						BLKIO_STAT_QUEUED, 1, 0);

block/blk-cgroup.h +2 -2

index 2622267..a71d290 100644
@@ -39,8 +39,6 @@ enum stat_type {
 	BLKIO_STAT_SERVICE_TIME = 0,
 	/* Total time spent waiting in scheduler queue in ns */
 	BLKIO_STAT_WAIT_TIME,
-	/* Number of IOs merged */
-	BLKIO_STAT_MERGED,
 	/* Number of IOs queued up */
 	BLKIO_STAT_QUEUED,
 	/* All the single valued stats go below this */

@@ -63,6 +61,8 @@ enum stat_type_cpu {
 	BLKIO_STAT_CPU_SERVICE_BYTES,
 	/* Total IOs serviced, post merge */
 	BLKIO_STAT_CPU_SERVICED,
+	/* Number of IOs merged */
+	BLKIO_STAT_CPU_MERGED,
 	BLKIO_STAT_CPU_NR
 };

Signature · 1 line

1.7.4.4

Vivek Goyal

author

2011-05-21 00:04 UTC #3 lore raw

PATCH 3/3 block: call elv_bio_merged() when merged

In reply to Vivek Goyal

Commit 73c101011926 ("block: initial patch for on-stack per-task plugging") removed calls to elv_bio_merged() when @bio merged with @req. Re-add them.

This in turn will update merged stats in associated group. That should be safe as long as request has got reference to the blkio_group.

Signed-off-by Namhyung Kim <[email protected]>

Cc Divyesh Shah <[email protected]>

---
 block/blk-core.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

block/blk-core.c +2 -0

index 7369eee..c8303e9 100644
@@ -1122,6 +1122,7 @@ static bool bio_attempt_back_merge(struct request_queue *q, struct request *req,
 	req->ioprio = ioprio_best(req->ioprio, bio_prio(bio));
 
 	drive_stat_acct(req, 0);
+	elv_bio_merged(q, req, bio);
 	return true;
 }

@@ -1155,6 +1156,7 @@ static bool bio_attempt_front_merge(struct request_queue *q,
 	req->ioprio = ioprio_best(req->ioprio, bio_prio(bio));
 
 	drive_stat_acct(req, 0);
+	elv_bio_merged(q, req, bio);
 	return true;
 }

Signature · 1 line

1.7.4.4

Vivek Goyal

author

2011-05-21 00:07 UTC #4 lore raw

In reply to Vivek Goyal

Quoted 6 lines · Vivek Goyal

Hi Jens,
Here is the patch for making merge stats per cpu. While scanning the code I also noticed that in one case I was not freeing up per cpu stats. First patch fixes it. I also included Namhyung Kim's bio merge fix.

Quoted 6 lines

Thanks
Vivek
Vivek Goyal (3):
cfq-iosched: Fix a memory leak of per cpu stats for root group
cfq-iosched: Make IO merge related stats per cpu

Minor correction. Following patch author is Namhyung Kim.

Namhyung Kim(1)

Quoted 1 line

block: call elv_bio_merged() when merged

Thanks
Vivek

Quoted 9 lines

 block/blk-cgroup.c  |   22 +++++++++++++++++-----
 block/blk-cgroup.h  |    4 ++--
 block/blk-core.c    |    2 ++
 block/cfq-iosched.c |    5 +++++
 4 files changed, 26 insertions(+), 7 deletions(-)

Signature · 1 line

1.7.4.4

Jens Axboe

maintainer

2011-05-23 08:04 UTC #5 lore raw

Quoted 10 lines · Vivek Goyal

Quoted 9 lines · Vivek Goyal
Hi Jens,
Here is the patch for making merge stats per cpu. While scanning the code I also noticed that in one case I was not freeing up per cpu stats. First patch fixes it. I also included Namhyung Kim's bio merge fix.
Thanks
Vivek

Quoted 7 lines

Quoted 3 lines
Vivek Goyal (3):
cfq-iosched: Fix a memory leak of per cpu stats for root group
cfq-iosched: Make IO merge related stats per cpu
Minor correction. Following patch author is Namhyung Kim.
Namhyung Kim(1)

Ugh, already merged it. I should have noticed that too, sorry about that Namhyung.

Signature · 1 line

Jens Axboe

Namhyung Kim

2011-05-23 13:53 UTC #6 lore raw

2011-05-23 (월), 10:04 +0200, Jens Axboe:

Quoted 15 lines · Vivek Goyal

Quoted 14 lines · Vivek Goyal
Quoted 12 lines
Hi Jens,
Here is the patch for making merge stats per cpu. While scanning the code I also noticed that in one case I was not freeing up per cpu stats. First patch fixes it. I also included Namhyung Kim's bio merge fix.
Thanks
Vivek
Vivek Goyal (3):
cfq-iosched: Fix a memory leak of per cpu stats for root group
cfq-iosched: Make IO merge related stats per cpu

Quoted 7 lines

Quoted 3 lines
Minor correction. Following patch author is Namhyung Kim.
Namhyung Kim(1)
Ugh, already merged it. I should have noticed that too, sorry about that Namhyung.

Hi, Jens.
It's all right, I'm fine :)

Signature · 2 lines

Regards,
Namhyung Kim

Namhyung Kim

2011-05-23 11:07 UTC #7 lore raw

In reply to Vivek Goyal

2011-05-20 (금), 20:06 -0400, Vivek Goyal:

Quoted 14 lines · Vivek Goyal

Quoted 6 lines
Hi Jens,
Here is the patch for making merge stats per cpu. While scanning the code I also noticed that in one case I was not freeing up per cpu stats. First patch fixes it. I also included Namhyung Kim's bio merge fix.
Quoted 6 lines
Thanks
Vivek
Vivek Goyal (3):
cfq-iosched: Fix a memory leak of per cpu stats for root group
cfq-iosched: Make IO merge related stats per cpu

Quoted 9 lines

Minor correction. Following patch author is Namhyung Kim.
Namhyung Kim(1)
Quoted 1 line
block: call elv_bio_merged() when merged
Thanks
Vivek

Hi, Vivek. Thanks a lot for your work. :)

Signature · 2 lines

Regards,
Namhyung Kim

Jens Axboe

maintainer

2011-05-23 08:03 UTC #8 lore raw

In reply to Vivek Goyal

Quoted 5 lines · Vivek Goyal

Hi Jens,
Here is the patch for making merge stats per cpu. While scanning the code I also noticed that in one case I was not freeing up per cpu stats. First patch fixes it. I also included Namhyung Kim's bio merge fix.

Thanks, looks fine. Merged.

Signature · 1 line

Jens Axboe