MD: improve raid1/10 write performance for fast storage

PATCH Superseded Shaohua Li 1 participant 4 messages May 28, 2012 MD RAID

Share & tools

Shaohua Li

author

2012-05-28 01:13 UTC #0 lore raw

v2 cover MD: improve raid1/10 write performance for fast storage

In raid1/10, all write requests are dispatched in a single thread. In fast storage, the thread is a bottleneck, because it dispatches request too slow. Also the thread migrates freely, which makes request completion cpu not match with submission cpu even driver/block layer has such capability. This will cause bad cache issue. Both these are not a big deal for slow storage.

Switching the dispatching to percpu/perthread based dramatically increases performance. The more raid disk number is, the more performance boosts. In a 4-disk raid10 setup, this can double the throughput.

percpu/perthread based dispatch doesn't harm slow storage. This is the way how raw device is accessed, and there is correct block plug set which can help do request merge and reduce lock contention.

V1->V2:
1. droped direct dispatch patches. That has better performance imporvement, but is hopelessly made correct.
2. Add a MD specific workqueue to do percpu dispatch.

Shaohua Li

author

2012-05-28 01:13 UTC #1 lore raw

PATCH v2 1/3 MD: add a specific workqueue to do dispatch

Add a specific workqueue to do dispatch. Later patches will use it to do per-cpu request queue dispatch.

Signed-off-by Shaohua Li <[email protected]>

---
 drivers/md/md.c |   14 ++++++++++++++
 drivers/md/md.h |    1 +
 2 files changed, 15 insertions(+)

drivers/md/md.c +14 -0

Index: linux/drivers/md/md.c
===================================================================
@@ -71,6 +71,7 @@ static void md_print_devices(void);
 static DECLARE_WAIT_QUEUE_HEAD(resync_wait);
 static struct workqueue_struct *md_wq;
 static struct workqueue_struct *md_misc_wq;
+static struct workqueue_struct *md_run_wq;
 
 #define MD_BUG(x...) { printk("md: bug in file %s, line %d\n", __FILE__, __LINE__); md_print_devices(); }

@@ -8446,6 +8447,12 @@ static void md_geninit(void)
 	proc_create("mdstat", S_IRUGO, NULL, &md_seq_fops);
 }
 
+int md_schedule_work_on(int cpu, struct work_struct *work)
+{
+	return queue_work_on(cpu, md_run_wq, work);
+}
+EXPORT_SYMBOL(md_schedule_work_on);
+
 static int __init md_init(void)
 {
 	int ret = -ENOMEM;

@@ -8458,6 +8465,10 @@ static int __init md_init(void)
 	if (!md_misc_wq)
 		goto err_misc_wq;
 
+	md_run_wq = alloc_workqueue("md_run", WQ_MEM_RECLAIM, 0);
+	if (!md_run_wq)
+		goto err_run_wq;
+
 	if ((ret = register_blkdev(MD_MAJOR, "md")) < 0)
 		goto err_md;

@@ -8479,6 +8490,8 @@ static int __init md_init(void)
 err_mdp:
 	unregister_blkdev(MD_MAJOR, "md");
 err_md:
+	destroy_workqueue(md_run_wq);
+err_run_wq:
 	destroy_workqueue(md_misc_wq);
 err_misc_wq:
 	destroy_workqueue(md_wq);

@@ -8571,6 +8584,7 @@ static __exit void md_exit(void)
 		export_array(mddev);
 		mddev->hold_active = 0;
 	}
+	destroy_workqueue(md_run_wq);
 	destroy_workqueue(md_misc_wq);
 	destroy_workqueue(md_wq);
 }

drivers/md/md.h +1 -0

Index: linux/drivers/md/md.h
===================================================================
@@ -616,6 +616,7 @@ extern int md_integrity_register(struct
 extern void md_integrity_add_rdev(struct md_rdev *rdev, struct mddev *mddev);
 extern int strict_strtoul_scaled(const char *cp, unsigned long *res, int scale);
 extern void restore_bitmap_write_access(struct file *file);
+extern int md_schedule_work_on(int cpu, struct work_struct *work);
 
 extern void mddev_init(struct mddev *mddev);
 extern int md_run(struct mddev *mddev);

Shaohua Li

author

2012-05-28 01:13 UTC #2 lore raw

PATCH v2 2/3 raid1: percpu dispatch for write request if bitmap supported

In reply to Shaohua Li

In raid1, all write requests are dispatched in raid1d thread. In fast storage, the raid1d thread is a bottleneck, because it dispatches request too slow. Also raid1d thread migrates freely, which makes request completion cpu not match with submission cpu even driver/block layer has such capability. This will cause bad cache issue.

If bitmap support is enabled, write requests can only be dispatched after dirty bitmap is flushed out. After bitmap is flushed, how write requests are dispatched doesn't impact correctness. A natural idea is to distribute request dispatch to several threads. With this patch, requests are added to a percpu list first. After bitmap is flushed, then the percpu list requests will dispatched in a workqueue. In this way, above bottleneck is removed.

In a 4k randwrite test with a 2 disks setup, below patch can provide 10% ~ 50% performance improvements depending on numa binding.

Signed-off-by Shaohua Li <[email protected]>

---
 drivers/md/raid1.c |   97 +++++++++++++++++++++++++++++++++++++++++++----------
 drivers/md/raid1.h |   10 ++++-
 2 files changed, 88 insertions(+), 19 deletions(-)

drivers/md/raid1.h +9 -1

Index: linux/drivers/md/raid1.h
===================================================================
@@ -22,6 +22,14 @@ struct pool_info {
 	int	raid_disks;
 };
 
+struct write_list {
+	struct bio_list	pending_bio_list;
+	struct bio_list	running_bio_list;
+	struct bio_list	tmp_bio_list;
+	struct work_struct work;
+	struct r1conf *conf;
+};
+
 struct r1conf {
 	struct mddev		*mddev;
 	struct mirror_info	*mirrors;	/* twice 'raid_disks' to

@@ -50,7 +58,7 @@ struct r1conf {
 	struct list_head	retry_list;
 
 	/* queue pending writes to be submitted on unplug */
-	struct bio_list		pending_bio_list;
+	struct write_list __percpu *write_list;
 	int			pending_count;
 
 	/* for use when syncing mirrors:

drivers/md/raid1.c +79 ~~-18~~

147 lines not on this page · raw message

Shaohua Li

author

2012-05-28 01:13 UTC #3 lore raw

PATCH v2 3/3 raid10: percpu dispatch for write request if bitmap supported

In reply to Shaohua Li

In raid10, all write requests are dispatched in raid10d thread. In fast storage, the raid10d thread is a bottleneck, because it dispatches request too slow. Also raid10d thread migrates freely, which makes request completion cpu not match with submission cpu even driver/block layer has such capability. This will cause bad cache issue.

In a 4k randwrite test with a 4 disks setup, below patch can provide about 95% performance improvements depending on numa binding.

Signed-off-by Shaohua Li <[email protected]>

---
 drivers/md/raid10.c |   98 ++++++++++++++++++++++++++++++++++++++++++----------
 drivers/md/raid10.h |   10 ++++-
 2 files changed, 89 insertions(+), 19 deletions(-)

drivers/md/raid10.h +9 -1

Index: linux/drivers/md/raid10.h
===================================================================
@@ -11,6 +11,14 @@ struct mirror_info {
 						 */
 };
 
+struct write_list {
+	struct bio_list	pending_bio_list;
+	struct bio_list	running_bio_list;
+	struct bio_list	tmp_bio_list;
+	struct work_struct work;
+	struct r10conf *conf;
+};
+
 struct r10conf {
 	struct mddev		*mddev;
 	struct mirror_info	*mirrors;

@@ -49,7 +57,7 @@ struct r10conf {
 
 	struct list_head	retry_list;
 	/* queue pending writes and submit them on unplug */
-	struct bio_list		pending_bio_list;
+	struct write_list __percpu *write_list;
 	int			pending_count;
 
 	spinlock_t		resync_lock;

drivers/md/raid10.c +80 ~~-18~~

154 lines not on this page · raw message