md:Add blk_plug in sync_thread.

PATCH In review majianpeng 3 participants 5 messages Jun 11, 2012 – Jun 27, 2012 MD RAID

Share & tools

majianpeng

author

2012-06-11 09:16 UTC #0 lore raw

PATCH 1/1 md:Add blk_plug in sync_thread.

Add blk_plug in sync_thread will increase the performance of sync. Because sync_thread did not blk_plug,so when raid sync, the bio merge not well.

Testing environment:
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller.
OS:Linux xxx 3.5.0-rc2+ #340 SMP Tue Jun 12 09:00:25 CST 2012 x86_64 x86_64 x86_64 GNU/Linux.
RAID5: four ST31000524NS disk.

Without blk_plug:recovery speed about 63M/Sec;
Add blk_plug:recovery speed about 120M/Sec.

Using blktrace:
blktrace -d /dev/sdb -w 60 -o -|blkparse -i -

without blk_plug:
Total (8,16):

 Reads Queued:      309811,     1239MiB	 Writes Queued:           0,        0KiB
 Read Dispatches:   283583,     1189MiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:   273351,     1149MiB	 Writes Completed:        0,        0KiB
 Read Merges:        23533,    94132KiB	 Write Merges:            0,        0KiB

IO unplugs: 0 Timer unplugs: 0

add blk_plug:
Total (8,16):

 Reads Queued:      428697,     1714MiB	 Writes Queued:           0,        0KiB
 Read Dispatches:     3954,     1714MiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:     3956,     1715MiB	 Writes Completed:        0,        0KiB
 Read Merges:       424743,     1698MiB	 Write Merges:            0,        0KiB

IO unplugs: 0 Timer unplugs: 3384

The ratio of merge will be markedly increased.

Signed-off-by majianpeng <[email protected]>

---
 drivers/md/md.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

drivers/md/md.c +4 -1

index 1c2f904..90bdc37 100644
@@ -7298,6 +7298,7 @@ void md_do_sync(struct mddev *mddev)
 	int skipped = 0;
 	struct md_rdev *rdev;
 	char *desc;
+	struct blk_plug plug;
 
 	/* just incase thread restarts... */
 	if (test_bit(MD_RECOVERY_DONE, &mddev->recovery))

@@ -7446,7 +7447,8 @@ void md_do_sync(struct mddev *mddev)
 		mddev->curr_resync = j;
 	}
 	mddev->curr_resync_completed = j;
-
+
+	blk_start_plug(&plug);
 	while (j < max_sectors) {
 		sector_t sectors;

@@ -7586,6 +7588,7 @@ void md_do_sync(struct mddev *mddev)
 	}
  skip:
 	set_bit(MD_CHANGE_DEVS, &mddev->flags);
+	blk_finish_plug(&plug);
 
 	if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) {
 		/* We completed so min/max setting can be forgotten if used. */

Signature · 6 lines

1.7.5.4



--------------
majianpeng

Nagilum

2012-06-11 09:32 UTC #1 lore raw

Quoted 11 lines · majianpeng

Add blk_plug in sync_thread will increase the performance of sync. Because sync_thread did not blk_plug,so when raid sync, the bio merge not well.
Testing environment:
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller.
OS:Linux xxx 3.5.0-rc2+ #340 SMP Tue Jun 12 09:00:25 CST 2012 x86_64 x86_64 x86_64 GNU/Linux.
RAID5: four ST31000524NS disk.

Quoted 2 lines

Without blk_plug:recovery speed about 63M/Sec;
Add blk_plug:recovery speed about 120M/Sec.

Whahoo!! :)

majianpeng

author

2012-06-11 10:11 UTC #2 lore raw

Quoted 12 lines · On Mon, 11 Jun 2012 02:32:57 najilum wrote:

Quoted 11 lines · majianpeng
Add blk_plug in sync_thread will increase the performance of sync. Because sync_thread did not blk_plug,so when raid sync, the bio merge not well.
Testing environment:
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller.
OS:Linux xxx 3.5.0-rc2+ #340 SMP Tue Jun 12 09:00:25 CST 2012 x86_64 x86_64 x86_64 GNU/Linux.
RAID5: four ST31000524NS disk.

Quoted 3 lines

Quoted 2 lines
Without blk_plug:recovery speed about 63M/Sec;
Add blk_plug:recovery speed about 120M/Sec.
Whahoo!! :)

What's mean ?

Quoted 1 line

--------------
majianpeng

Nagilum

2012-06-11 10:30 UTC #3 lore raw

Quoted 13 lines · majianpeng

Quoted 12 lines · On Mon, 11 Jun 2012 02:32:57 najilum wrote:
Quoted 11 lines · majianpeng
Add blk_plug in sync_thread will increase the performance of sync. Because sync_thread did not blk_plug,so when raid sync, the bio merge not well.
Testing environment:
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller.
OS:Linux xxx 3.5.0-rc2+ #340 SMP Tue Jun 12 09:00:25 CST 2012 x86_64 x86_64 x86_64 GNU/Linux.
RAID5: four ST31000524NS disk.

Quoted 4 lines

Quoted 3 lines
Quoted 2 lines
Without blk_plug:recovery speed about 63M/Sec;
Add blk_plug:recovery speed about 120M/Sec.
Whahoo!! :)
What's mean ?

Almost doubled the performance with one simple change, that's great! :)

NeilBrown

2012-06-27 02:30 UTC #4 lore raw

In reply to majianpeng

Quoted 65 lines · On Mon, 11 Jun 2012 17:16:55 +0800 majianpeng <[email protected]> wrote:

Add blk_plug in sync_thread will increase the performance of sync. Because sync_thread did not blk_plug,so when raid sync, the bio merge not well.
Testing environment:
SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller.
OS:Linux xxx 3.5.0-rc2+ #340 SMP Tue Jun 12 09:00:25 CST 2012 x86_64 x86_64 x86_64 GNU/Linux.
RAID5: four ST31000524NS disk.
Without blk_plug:recovery speed about 63M/Sec;
Add blk_plug:recovery speed about 120M/Sec.
Using blktrace:
blktrace -d /dev/sdb -w 60 -o -|blkparse -i -
without blk_plug:
Total (8,16):
 Reads Queued:      309811,     1239MiB	 Writes Queued:           0,        0KiB
 Read Dispatches:   283583,     1189MiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:   273351,     1149MiB	 Writes Completed:        0,        0KiB
 Read Merges:        23533,    94132KiB	 Write Merges:            0,        0KiB
IO unplugs: 0 Timer unplugs: 0
add blk_plug:
Total (8,16):
 Reads Queued:      428697,     1714MiB	 Writes Queued:           0,        0KiB
 Read Dispatches:     3954,     1714MiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:     3956,     1715MiB	 Writes Completed:        0,        0KiB
 Read Merges:       424743,     1698MiB	 Write Merges:            0,        0KiB
IO unplugs: 0 Timer unplugs: 3384
The ratio of merge will be markedly increased.
Signed-off-by majianpeng <[email protected]>
---
 drivers/md/md.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
drivers/md/md.c +3 -1
index 1c2f904..90bdc37 100644
@@ -7298,6 +7298,7 @@ void md_do_sync(struct mddev *mddev)
 	int skipped = 0;
 	struct md_rdev *rdev;
 	char *desc;
+	struct blk_plug plug;
 
 	/* just incase thread restarts... */
 	if (test_bit(MD_RECOVERY_DONE, &mddev->recovery))
@@ -7446,7 +7447,8 @@ void md_do_sync(struct mddev *mddev)
 		mddev->curr_resync = j;
 	}
 	mddev->curr_resync_completed = j;
-
+
+	blk_start_plug(&plug);
 	while (j < max_sectors) {
 		sector_t sectors;
 

drivers/md/md.c void md_do_sync(struct mddev *mddev) · line 7594

Quoted 7 lines

drivers/md/md.c +1 -0

@@ -7586,6 +7588,7 @@ void md_do_sync(struct mddev *mddev)
 	}
  skip:
 	set_bit(MD_CHANGE_DEVS, &mddev->flags);
+	blk_finish_plug(&plug);
 
 	if (!test_bit(MD_RECOVERY_INTR, &mddev->recovery)) {
 		/* We completed so min/max setting can be forgotten if used. */

Thanks again.

I've moved the call to "blk_finish_plug" up a bit to just after the
out:
label, as it is possible to "goto skip" before calling blk_start_plug.

Thanks a lot,
NeilBrown