Skip to contentSkip to conversation

(2nd try) force parallel resync

PATCH In review Bernd Schubert 2 participants 3 messages Dec 6, 2007 – Dec 7, 2007 MD RAID

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 (2nd try) force parallel resync

Hello,

here is the second version of the patch. With this version also on setting /sys/block/*/md/sync_force_parallel the sync_thread is woken up. Though, I still don't understand why md_wakeup_thread() is not working.

drivers/md/md.c +39 -2
Index: linux-2.6.22/drivers/md/md.c
===================================================================
@@ -2843,6 +2843,41 @@ __ATTR(sync_speed_max, S_IRUGO|S_IWUSR, 
 
 
 static ssize_t
+sync_force_parallel_show(mddev_t *mddev, char *page)
+{
+        return sprintf(page, "%d\n", mddev->parallel_resync);
+}
+
+static ssize_t
+sync_force_parallel_store(mddev_t *mddev, const char *buf, size_t len)
+{
+	char *e;
+	unsigned long n = simple_strtoul(buf, &e, 10);
+
+	if (!*buf || (*e && *e != '\n') || (n != 0 && n != 1))
+		return -EINVAL;
+
+	mddev->parallel_resync = n;
+
+	if (mddev->sync_thread) {
+		dprintk("md: waking up MD thread %s.\n",
+		       mddev->sync_thread->tsk->comm);
+		set_bit(THREAD_WAKEUP, &mddev->sync_thread->flags);
+		wake_up_process(mddev->sync_thread->tsk);
+
+/*		FIXME: why does md_wakeup_thread() not work?,
+		somehow related to:  wake_up(&thread->wqueue);
+		md_wakeup_thread(mddev->sync_thread); */
+	}
+	return len;
+}
+
+/* force parallel resync, even with shared block devices */
+static struct md_sysfs_entry md_sync_force_parallel =
+__ATTR(sync_force_parallel, S_IRUGO|S_IWUSR,
+       sync_force_parallel_show, sync_force_parallel_store);
+
+static ssize_t
 sync_speed_show(mddev_t *mddev, char *page)
 {
 	unsigned long resync, dt, db;
@@ -2980,6 +3015,7 @@ static struct attribute *md_redundancy_a
 	&md_sync_min.attr,
 	&md_sync_max.attr,
 	&md_sync_speed.attr,
+	&md_sync_force_parallel.attr,
 	&md_sync_completed.attr,
 	&md_suspend_lo.attr,
 	&md_suspend_hi.attr,
@@ -5264,8 +5300,9 @@ void md_do_sync(mddev_t *mddev)
 		ITERATE_MDDEV(mddev2,tmp) {
 			if (mddev2 == mddev)
 				continue;
-			if (mddev2->curr_resync && 
-			    match_mddev_units(mddev,mddev2)) {
+			if (!mddev->parallel_resync
+			&&  mddev2->curr_resync
+			&&  match_mddev_units(mddev,mddev2)) {
 				DEFINE_WAIT(wq);
 				if (mddev < mddev2 && mddev->curr_resync == 2) {
 					/* arbitrarily yield */

Index: linux-2.6.22/include/linux/raid/md_k.h
===================================================================
--- linux-2.6.22.orig/include/linux/raid/md_k.h 2007-12-06 19:51:55.000000000
+0100
+++ linux-2.6.22/include/linux/raid/md_k.h 2007-12-06 19:52:33.000000000 +0100
@@ -170,6 +170,9 @@ struct mddev_s
int sync_speed_min;
int sync_speed_max;

+ /* resync even though the same disks are shared among md-devices */
+ int parallel_resync;
+
int ok_start_degraded;
/* recovery/resync flags
* NEEDED: we might need to start a resync/recover

Signature · 2 lines
Bernd Schubert
Q-Leap Networks GmbH
#1 lore raw
Quoted 5 lines · On Thursday December 6, [email protected] wrote:

Hello,

here is the second version of the patch. With this version also on setting /sys/block/*/md/sync_force_parallel the sync_thread is woken up. Though, I still don't understand why md_wakeup_thread() is not working.

Could give a little more detail on why you want this? When do you want multiple arrays on the same device to sync at the same time? What exactly is the hardware like?

md threads generally run for a little while to perform some task, then stop and wait to be needed again. md_wakeup_thread says "you are needed again".

The resync/recovery thread is a bit different. It just run md_do_sync once. md_wakeup_thread is not really meaningful in that context.

What you want is:

		wake_up(&resync_wait);

that will get any thread that is waiting for some other array to resync to wake up and see if something needs to be done.

NeilBrown

#2 lore raw

Hello Neil,

Quoted 10 lines · On Friday 07 December 2007 03:10:37 Neil Brown wrote:
Quoted 5 lines · On Thursday December 6, [email protected] wrote:

Hello,

here is the second version of the patch. With this version also on setting /sys/block/*/md/sync_force_parallel the sync_thread is woken up. Though, I still don't understand why md_wakeup_thread() is not working.

Could give a little more detail on why you want this? When do you want multiple arrays on the same device to sync at the same time? What exactly is the hardware like?

I did describe it in my first mail, I guess you just missed this one. So I'm pasting this part here:

Quoted 21 lines · On Monday 03 December 2007 14:37:03 Bernd Schubert wrote:

Hello Neil,

on doing software-raid over some Infortrend hardware raid boxes, we can easily bring a single cpu to 100%, while the hardware-raid boxes are still only utilized for 50% or less.
So we are using *several* software raids instead of only one. However, for us it then also doesn't make sense to have a resync for only one of the md-raids. Below is a patch to force parallel resync for specified md-sets via sysfs.
This still leaves a problem, though. If a resync starts before settings this parameter, I don't know how to wake up the sleeping resync-thread. Any ideas?

md threads generally run for a little while to perform some task, then stop and wait to be needed again. md_wakeup_thread says "you are needed again".

Quoted 8 lines

The resync/recovery thread is a bit different. It just run md_do_sync once. md_wakeup_thread is not really meaningful in that context.

What you want is:

		wake_up(&resync_wait);

that will get any thread that is waiting for some other array to resync to wake up and see if something needs to be done.

Ah thanks a lot! Now I understand. What about this?

drivers/md/md.c +2 -0
Index: linux-2.6.22/drivers/md/md.c
===================================================================
@@ -74,6 +74,8 @@ static DEFINE_SPINLOCK(pers_lock);
 
 static void md_print_devices(void);
 
+static DECLARE_WAIT_QUEUE_HEAD(resync_wait);
+
 #define MD_BUG(x...) { printk("md: bug in file %s, line %d\n", __FILE__, 

__LINE__); md_print_devices(); }

/*
@@ -2843,6 +2845,34 @@ __ATTR(sync_speed_max, S_IRUGO|S_IWUSR,

static ssize_t
+sync_force_parallel_show(mddev_t *mddev, char *page)
+{
+ return sprintf(page, "%d\n", mddev->parallel_resync);
+}
+
+static ssize_t
+sync_force_parallel_store(mddev_t *mddev, const char *buf, size_t len) +{
+ char *e;
+ unsigned long n = simple_strtoul(buf, &e, 10);
+
+ if (!*buf || (*e && *e != '\n') || (n != 0 && n != 1))
+ return -EINVAL;
+
+ mddev->parallel_resync = n;
+
+ if (mddev->sync_thread) {
+ wake_up(&resync_wait);
+ }
+ return len;
+}
+
+/* force parallel resync, even with shared block devices */ +static struct md_sysfs_entry md_sync_force_parallel = +__ATTR(sync_force_parallel, S_IRUGO|S_IWUSR,
+ sync_force_parallel_show, sync_force_parallel_store);
+
+static ssize_t
sync_speed_show(mddev_t *mddev, char *page)
{
unsigned long resync, dt, db;
@@ -2980,6 +3010,7 @@ static struct attribute *md_redundancy_a
&md_sync_min.attr,
&md_sync_max.attr,
&md_sync_speed.attr,
+ &md_sync_force_parallel.attr,
&md_sync_completed.attr,
&md_suspend_lo.attr,
&md_suspend_hi.attr,
@@ -5199,8 +5230,6 @@ void md_allow_write(mddev_t *mddev)
}
EXPORT_SYMBOL_GPL(md_allow_write);

-static DECLARE_WAIT_QUEUE_HEAD(resync_wait);
-
#define SYNC_MARKS 10
#define SYNC_MARK_STEP (3*HZ)
void md_do_sync(mddev_t *mddev)
@@ -5264,8 +5293,9 @@ void md_do_sync(mddev_t *mddev)
ITERATE_MDDEV(mddev2,tmp) {
if (mddev2 == mddev)
continue;
- if (mddev2->curr_resync &&
- match_mddev_units(mddev,mddev2)) {
+ if (!mddev->parallel_resync
+ && mddev2->curr_resync
+ && match_mddev_units(mddev,mddev2)) {
DEFINE_WAIT(wq);
if (mddev < mddev2 && mddev->curr_resync == 2) {
/* arbitrarily yield */
Index: linux-2.6.22/include/linux/raid/md_k.h
===================================================================
--- linux-2.6.22.orig/include/linux/raid/md_k.h 2007-12-06 19:51:55.000000000
+0100
+++ linux-2.6.22/include/linux/raid/md_k.h 2007-12-06 19:52:33.000000000 +0100
@@ -170,6 +170,9 @@ struct mddev_s
int sync_speed_min;
int sync_speed_max;

+ /* resync even though the same disks are shared among md-devices */
+ int parallel_resync;
+
int ok_start_degraded;
/* recovery/resync flags
* NEEDED: we might need to start a resync/recover

Signature · 2 lines
Bernd Schubert
Q-Leap Networks GmbH