Skip to contentSkip to conversation

Introduction

PATCH In review NeilBrown 3 participants 14 messages Feb 18, 2005 – Feb 20, 2005

Subsystems (2)
lore
Share & tools
lore

NeilBrown

author
#0 lore raw

PATCH 1/1 Introduction

9 patches for md in 2.6.11-rc3-mm2 follow.

1 - tightens up some locking to close a race that is extremely unlikely
to happen (famous last words). It can only happen when failed devices are being removed.
2 - Fixes a resync related problem for raid5 and raid6 arrays that have
lost more devices than they can cope with.
3 - Fixes a clumsy test that has very little real effect (just a printk).

These three can be considered stable.

4,5 Improve handling of 'safemode' which is where we mark the superblock
clean whenever there have been no writes for a little while, and mark it dirty before the next write.
6-9 Introduce "bitmap based intent logging" which allows a file to
store - as a bitmap - all block that may have been written recently. This allow resync to be optimised, which is more important as drives get larger.

These should not be considerred stable - certainly actually using the bitmap mode (which requires a 2.0 series version of mdadm) should be seen as 'experimental'.

mdadm-2.0-devel-1 was released today.

NeilBrown

author
#1 lore raw

PATCH 1/1 Remove possible oops in md/raid1

When we get a pointer and check it is non-null, we should not get the pointer again but should instead use the pointer we got in the first place...

### Diffstat output

 ./drivers/md/raid1.c |   40 ++++++++++++++++++++++++++++------------
 1 files changed, 28 insertions(+), 12 deletions(-)
drivers/md/raid1.c renamed from drivers/md/raid1.c~current~ +28 -12
diff ./drivers/md/raid1.c~current~ ./drivers/md/raid1.c
@@ -338,6 +338,7 @@ static int read_balance(conf_t *conf, r1
 	int new_disk = conf->last_used, disk = new_disk;
 	const int sectors = r1_bio->sectors;
 	sector_t new_distance, current_distance;
+	mdk_rdev_t *new_rdev, *rdev;
 
 	rcu_read_lock();
 	/*
@@ -345,13 +346,14 @@ static int read_balance(conf_t *conf, r1
 	 * device if no resync is going on, or below the resync window.
 	 * We take the first readable disk when above the resync window.
 	 */
+ retry:
 	if (conf->mddev->recovery_cp < MaxSector &&
 	    (this_sector + sectors >= conf->next_resync)) {
 		/* Choose the first operation device, for consistancy */
 		new_disk = 0;
 
-		while (!conf->mirrors[new_disk].rdev ||
-		       !conf->mirrors[new_disk].rdev->in_sync) {
+		while ((new_rdev=conf->mirrors[new_disk].rdev) == NULL ||
+		       !new_rdev->in_sync) {
 			new_disk++;
 			if (new_disk == conf->raid_disks) {
 				new_disk = -1;
@@ -363,8 +365,8 @@ static int read_balance(conf_t *conf, r1
 
 
 	/* make sure the disk is operational */
-	while (!conf->mirrors[new_disk].rdev ||
-	       !conf->mirrors[new_disk].rdev->in_sync) {
+	while ((new_rdev=conf->mirrors[new_disk].rdev) == NULL ||
+	       !new_rdev->in_sync) {
 		if (new_disk <= 0)
 			new_disk = conf->raid_disks;
 		new_disk--;
@@ -393,18 +395,20 @@ static int read_balance(conf_t *conf, r1
 			disk = conf->raid_disks;
 		disk--;
 
-		if (!conf->mirrors[disk].rdev ||
-		    !conf->mirrors[disk].rdev->in_sync)
+		if ((rdev=conf->mirrors[disk].rdev) == NULL ||
+		    !rdev->in_sync)
 			continue;
 
-		if (!atomic_read(&conf->mirrors[disk].rdev->nr_pending)) {
+		if (!atomic_read(&rdev->nr_pending)) {
 			new_disk = disk;
+			new_rdev = rdev;
 			break;
 		}
 		new_distance = abs(this_sector - conf->mirrors[disk].head_position);
 		if (new_distance < current_distance) {
 			current_distance = new_distance;
 			new_disk = disk;
+			new_rdev = rdev;
 		}
 	} while (disk != conf->last_used);
 
@@ -414,7 +418,14 @@ rb_out:
 	if (new_disk >= 0) {
 		conf->next_seq_sect = this_sector + sectors;
 		conf->last_used = new_disk;
-		atomic_inc(&conf->mirrors[new_disk].rdev->nr_pending);
+		atomic_inc(&new_rdev->nr_pending);
+		if (!new_rdev->in_sync) {
+			/* cannot risk returning a device that failed
+			 * before we inc'ed nr_pending
+			 */
+			atomic_dec(&new_rdev->nr_pending);
+			goto retry;
+		}
 	}
 	rcu_read_unlock();
 
@@ -512,6 +523,7 @@ static int make_request(request_queue_t 
 	r1bio_t *r1_bio;
 	struct bio *read_bio;
 	int i, disks;
+	mdk_rdev_t *rdev;
 
 	/*
 	 * Register the new request and wait if the reconstruction
@@ -585,10 +597,14 @@ static int make_request(request_queue_t 
 	disks = conf->raid_disks;
 	rcu_read_lock();
 	for (i = 0;  i < disks; i++) {
-		if (conf->mirrors[i].rdev &&
-		    !conf->mirrors[i].rdev->faulty) {
-			atomic_inc(&conf->mirrors[i].rdev->nr_pending);
-			r1_bio->bios[i] = bio;
+		if ((rdev=conf->mirrors[i].rdev) != NULL &&
+		    !rdev->faulty) {
+			atomic_inc(&rdev->nr_pending);
+			if (rdev->faulty) {
+				atomic_dec(&rdev->nr_pending);
+				r1_bio->bios[i] = NULL;
+			} else
+				r1_bio->bios[i] = bio;
 		} else
 			r1_bio->bios[i] = NULL;
 	}
author
#2 lore raw

PATCH 1/1 Make raid5 and raid6 robust against failure during recovery.

In reply to NeilBrown

Two problems are fixed here.
1/ if the array is known to require a resync (parity update),
but there are too many failed devices, the resync cannot complete but will be retried indefinitely.
2/ if the array has too many failed drives to be usable and a spare is
available, reconstruction will be attempted, but cannot work. This also is retried indefinitely.

### Diffstat output

 ./drivers/md/md.c        |   12 ++++++------
 ./drivers/md/raid5.c     |   13 +++++++++++++
 ./drivers/md/raid6main.c |   12 ++++++++++++
 3 files changed, 31 insertions(+), 6 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +6 -6
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -3655,18 +3655,18 @@ void md_check_recovery(mddev_t *mddev)
 
 		/* no recovery is running.
 		 * remove any failed drives, then
-		 * add spares if possible
+		 * add spares if possible.
+		 * Spare are also removed and re-added, to allow
+		 * the personality to fail the re-add.
 		 */
-		ITERATE_RDEV(mddev,rdev,rtmp) {
+		ITERATE_RDEV(mddev,rdev,rtmp)
 			if (rdev->raid_disk >= 0 &&
-			    rdev->faulty &&
+			    (rdev->faulty || ! rdev->in_sync) &&
 			    atomic_read(&rdev->nr_pending)==0) {
 				if (mddev->pers->hot_remove_disk(mddev, rdev->raid_disk)==0)
 					rdev->raid_disk = -1;
 			}
-			if (!rdev->faulty && rdev->raid_disk >= 0 && !rdev->in_sync)
-				spares++;
-		}
+
 		if (mddev->degraded) {
 			ITERATE_RDEV(mddev,rdev,rtmp)
 				if (rdev->raid_disk < 0
drivers/md/raid5.c renamed from drivers/md/raid5.c~current~ +13 -0
diff ./drivers/md/raid5.c~current~ ./drivers/md/raid5.c
@@ -1491,6 +1491,15 @@ static int sync_request (mddev_t *mddev,
 		unplug_slaves(mddev);
 		return 0;
 	}
+	/* if there is 1 or more failed drives and we are trying
+	 * to resync, then assert that we are finished, because there is
+	 * nothing we can do.
+	 */
+	if (mddev->degraded >= 1 && test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
+		int rv = (mddev->size << 1) - sector_nr;
+		md_done_sync(mddev, rv, 1);
+		return rv;
+	}
 
 	x = sector_nr;
 	chunk_offset = sector_div(x, sectors_per_chunk);
@@ -1882,6 +1891,10 @@ static int raid5_add_disk(mddev_t *mddev
 	int disk;
 	struct disk_info *p;
 
+	if (mddev->degraded > 1)
+		/* no point adding a device */
+		return 0;
+
 	/*
 	 * find the disk ...
 	 */
drivers/md/raid6main.c renamed from drivers/md/raid6main.c~current~ +12 -0
diff ./drivers/md/raid6main.c~current~ ./drivers/md/raid6main.c
@@ -1650,6 +1650,15 @@ static int sync_request (mddev_t *mddev,
 		unplug_slaves(mddev);
 		return 0;
 	}
+	/* if there are 2 or more failed drives and we are trying
+	 * to resync, then assert that we are finished, because there is
+	 * nothing we can do.
+	 */
+	if (mddev->degraded >= 2 && test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
+		int rv = (mddev->size << 1) - sector_nr;
+		md_done_sync(mddev, rv, 1);
+		return rv;
+	}
 
 	x = sector_nr;
 	chunk_offset = sector_div(x, sectors_per_chunk);
@@ -2048,6 +2057,9 @@ static int raid6_add_disk(mddev_t *mddev
 	int disk;
 	struct disk_info *p;
 
+	if (mddev->degraded > 2)
+		/* no point adding a device */
+		return 0;
 	/*
 	 * find the disk ...
 	 */
author
#3 lore raw

PATCH 1/1 Remove kludgy level check from md.c

In reply to NeilBrown

This test is overly specific, and misses raid10. Assume all levels >= 1 might need reconstruction instead.

### Diffstat output

 ./drivers/md/md.c |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +2 -3
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -1478,9 +1478,8 @@ static int analyze_sbs(mddev_t * mddev)
 
 
 
-	if ((mddev->recovery_cp != MaxSector) &&
-	    ((mddev->level == 1) ||
-	     ((mddev->level >= 4) && (mddev->level <= 6))))
+	if (mddev->recovery_cp != MaxSector &&
+	    mddev->level >= 1)
 		printk(KERN_ERR "md: %s: raid array is not clean"
 		       " -- starting background reconstruction\n",
 		       mdname(mddev));
author
#4 lore raw

PATCH 1/1 Merge md_enter_safemode into md_check_recovery

In reply to NeilBrown

md_enter_safemode checks if it is time to mark the md superblock as 'clean'. i.e. if all writes have completed and a suitable delay has passed.

This is currently called from md_handle_safemode which in-turn is called (almost) every time md_check_recovery is called, and from the end of md_do_sync which causes the mddev->thread to run, which will always call md_check_recovery as well.

So it doesn't need to be a separate function and fits quite well into md_check_recovery.

The "almost" is because multipathd calls md_check_recovery but not md_handle_safemode. This is OK because the code from md_enter_safemode is a no-op if mddev->safemode == 0, which it always is for a multipathd (providing we don't allow it to be set to 2 on a signal...)

### Diffstat output

 ./drivers/md/md.c        |   58 +++++++++++++++++++----------------------------
 ./drivers/md/raid1.c     |    1 
 ./drivers/md/raid10.c    |    1 
 ./drivers/md/raid5.c     |    1 
 ./drivers/md/raid6main.c |    1 
 5 files changed, 24 insertions(+), 38 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +24 -34
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -3323,37 +3323,6 @@ void md_write_end(mddev_t *mddev)
 	}
 }
 
-static inline void md_enter_safemode(mddev_t *mddev)
-{
-	if (!mddev->safemode) return;
-	if (mddev->safemode == 2 &&
-	    (atomic_read(&mddev->writes_pending) || mddev->in_sync ||
-		    mddev->recovery_cp != MaxSector))
-		return; /* avoid the lock */
-	mddev_lock_uninterruptible(mddev);
-	if (mddev->safemode && !atomic_read(&mddev->writes_pending) &&
-	    !mddev->in_sync && mddev->recovery_cp == MaxSector) {
-		mddev->in_sync = 1;
-		md_update_sb(mddev);
-	}
-	mddev_unlock(mddev);
-
-	if (mddev->safemode == 1)
-		mddev->safemode = 0;
-}
-
-void md_handle_safemode(mddev_t *mddev)
-{
-	if (signal_pending(current)) {
-		printk(KERN_INFO "md: %s in immediate safe mode\n",
-			mdname(mddev));
-		mddev->safemode = 2;
-		flush_signals(current);
-	}
-	md_enter_safemode(mddev);
-}
-
-
 DECLARE_WAIT_QUEUE_HEAD(resync_wait);
 
 #define SYNC_MARKS	10
@@ -3574,7 +3543,6 @@ static void md_do_sync(mddev_t *mddev)
 			mddev->recovery_cp = MaxSector;
 	}
 
-	md_enter_safemode(mddev);
  skip:
 	mddev->curr_resync = 0;
 	wake_up(&resync_wait);
@@ -3615,14 +3583,37 @@ void md_check_recovery(mddev_t *mddev)
 
 	if (mddev->ro)
 		return;
+
+	if (signal_pending(current)) {
+		if (mddev->pers->sync_request) {
+			printk(KERN_INFO "md: %s in immediate safe mode\n",
+			       mdname(mddev));
+			mddev->safemode = 2;
+		}
+		flush_signals(current);
+	}
+
 	if ( ! (
 		mddev->sb_dirty ||
 		test_bit(MD_RECOVERY_NEEDED, &mddev->recovery) ||
-		test_bit(MD_RECOVERY_DONE, &mddev->recovery)
+		test_bit(MD_RECOVERY_DONE, &mddev->recovery) ||
+		(mddev->safemode == 1) ||
+		(mddev->safemode == 2 && ! atomic_read(&mddev->writes_pending)
+		 && !mddev->in_sync && mddev->recovery_cp == MaxSector)
 		))
 		return;
+
 	if (mddev_trylock(mddev)==0) {
 		int spares =0;
+
+		if (mddev->safemode && !atomic_read(&mddev->writes_pending) &&
+		    !mddev->in_sync && mddev->recovery_cp == MaxSector) {
+			mddev->in_sync = 1;
+			mddev->sb_dirty = 1;
+		}
+		if (mddev->safemode == 1)
+			mddev->safemode = 0;
+
 		if (mddev->sb_dirty)
 			md_update_sb(mddev);
 		if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery) &&
@@ -3869,7 +3860,6 @@ EXPORT_SYMBOL(md_error);
 EXPORT_SYMBOL(md_done_sync);
 EXPORT_SYMBOL(md_write_start);
 EXPORT_SYMBOL(md_write_end);
-EXPORT_SYMBOL(md_handle_safemode);
 EXPORT_SYMBOL(md_register_thread);
 EXPORT_SYMBOL(md_unregister_thread);
 EXPORT_SYMBOL(md_wakeup_thread);
drivers/md/raid1.c renamed from drivers/md/raid1.c~current~ +0 -1
diff ./drivers/md/raid1.c~current~ ./drivers/md/raid1.c
@@ -931,7 +931,6 @@ static void raid1d(mddev_t *mddev)
 	mdk_rdev_t *rdev;
 
 	md_check_recovery(mddev);
-	md_handle_safemode(mddev);
 	
 	for (;;) {
 		char b[BDEVNAME_SIZE];
drivers/md/raid10.c renamed from drivers/md/raid10.c~current~ +0 -1
diff ./drivers/md/raid10.c~current~ ./drivers/md/raid10.c
@@ -1216,7 +1216,6 @@ static void raid10d(mddev_t *mddev)
 	mdk_rdev_t *rdev;
 
 	md_check_recovery(mddev);
-	md_handle_safemode(mddev);
 
 	for (;;) {
 		char b[BDEVNAME_SIZE];
drivers/md/raid5.c renamed from drivers/md/raid5.c~current~ +0 -1
diff ./drivers/md/raid5.c~current~ ./drivers/md/raid5.c
@@ -1544,7 +1544,6 @@ static void raid5d (mddev_t *mddev)
 	PRINTK("+++ raid5d active\n");
 
 	md_check_recovery(mddev);
-	md_handle_safemode(mddev);
 
 	handled = 0;
 	spin_lock_irq(&conf->device_lock);
drivers/md/raid6main.c renamed from drivers/md/raid6main.c~current~ +0 -1
diff ./drivers/md/raid6main.c~current~ ./drivers/md/raid6main.c
@@ -1703,7 +1703,6 @@ static void raid6d (mddev_t *mddev)
 	PRINTK("+++ raid6d active\n");
 
 	md_check_recovery(mddev);
-	md_handle_safemode(mddev);
 
 	handled = 0;
 	spin_lock_irq(&conf->device_lock);
author
#5 lore raw

PATCH 1/1 Improve locking on 'safemode' and move superblock writes

In reply to NeilBrown

When md marks the superblock dirty before a write, it calls generic_make_request (to write the superblock) from within generic_make_request (to write the first dirty block), which could cause problems later.
With this patch, the superblock write is always done by the helper thread, and write request are delayed until that write completes.

Also, the locking around marking the array dirty and writing the superblock is improved to avoid possible races.

### Diffstat output

 ./drivers/md/md.c           |   73 +++++++++++++++++++++++++++++++++++---------
 ./drivers/md/raid1.c        |    4 +-
 ./drivers/md/raid10.c       |    5 ++-
 ./drivers/md/raid5.c        |    6 ++-
 ./drivers/md/raid6main.c    |    6 ++-
 ./include/linux/raid/md.h   |    2 -
 ./include/linux/raid/md_k.h |    7 ++++
 7 files changed, 82 insertions(+), 21 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +59 -14
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -261,6 +261,8 @@ static mddev_t * mddev_find(dev_t unit)
 	INIT_LIST_HEAD(&new->all_mddevs);
 	init_timer(&new->safemode_timer);
 	atomic_set(&new->active, 1);
+	bio_list_init(&new->write_list);
+	spin_lock_init(&new->write_lock);
 
 	new->queue = blk_alloc_queue(GFP_KERNEL);
 	if (!new->queue) {
@@ -1294,9 +1296,11 @@ static void md_update_sb(mddev_t * mddev
 	int err, count = 100;
 	struct list_head *tmp;
 	mdk_rdev_t *rdev;
+	int sync_req;
 
-	mddev->sb_dirty = 0;
 repeat:
+	spin_lock(&mddev->write_lock);
+	sync_req = mddev->in_sync;
 	mddev->utime = get_seconds();
 	mddev->events ++;
 
@@ -1315,8 +1319,12 @@ repeat:
 	 * do not write anything to disk if using
 	 * nonpersistent superblocks
 	 */
-	if (!mddev->persistent)
+	if (!mddev->persistent) {
+		mddev->sb_dirty = 0;
+		spin_unlock(&mddev->write_lock);
 		return;
+	}
+	spin_unlock(&mddev->write_lock);
 
 	dprintk(KERN_INFO 
 		"md: updating %s RAID superblock on device (in sync %d)\n",
@@ -1347,6 +1355,15 @@ repeat:
 		printk(KERN_ERR \
 			"md: excessive errors occurred during superblock update, exiting\n");
 	}
+	spin_lock(&mddev->write_lock);
+	if (mddev->in_sync != sync_req) {
+		/* have to write it out again */
+		spin_unlock(&mddev->write_lock);
+		goto repeat;
+	}
+	mddev->sb_dirty = 0;
+	spin_unlock(&mddev->write_lock);
+
 }
 
 /*
@@ -3298,19 +3315,31 @@ void md_done_sync(mddev_t *mddev, int bl
 }
 
 
-void md_write_start(mddev_t *mddev)
+/* md_write_start(mddev, bi)
+ * If we need to update some array metadata (e.g. 'active' flag
+ * in superblock) before writing, queue bi for later writing
+ * and return 0, else return 1 and it will be written now
+ */
+int md_write_start(mddev_t *mddev, struct bio *bi)
 {
-	if (!atomic_read(&mddev->writes_pending)) {
-		mddev_lock_uninterruptible(mddev);
-		if (mddev->in_sync) {
-			mddev->in_sync = 0;
- 			del_timer(&mddev->safemode_timer);
-			md_update_sb(mddev);
-		}
-		atomic_inc(&mddev->writes_pending);
-		mddev_unlock(mddev);
-	} else
-		atomic_inc(&mddev->writes_pending);
+	if (bio_data_dir(bi) != WRITE)
+		return 1;
+
+	atomic_inc(&mddev->writes_pending);
+	spin_lock(&mddev->write_lock);
+	if (mddev->in_sync == 0 && mddev->sb_dirty == 0) {
+		spin_unlock(&mddev->write_lock);
+		return 1;
+	}
+	bio_list_add(&mddev->write_list, bi);
+
+	if (mddev->in_sync) {
+		mddev->in_sync = 0;
+		mddev->sb_dirty = 1;
+	}
+	spin_unlock(&mddev->write_lock);
+	md_wakeup_thread(mddev->thread);
+	return 0;
 }
 
 void md_write_end(mddev_t *mddev)
@@ -3597,6 +3626,7 @@ void md_check_recovery(mddev_t *mddev)
 		mddev->sb_dirty ||
 		test_bit(MD_RECOVERY_NEEDED, &mddev->recovery) ||
 		test_bit(MD_RECOVERY_DONE, &mddev->recovery) ||
+		mddev->write_list.head ||
 		(mddev->safemode == 1) ||
 		(mddev->safemode == 2 && ! atomic_read(&mddev->writes_pending)
 		 && !mddev->in_sync && mddev->recovery_cp == MaxSector)
@@ -3605,7 +3635,9 @@ void md_check_recovery(mddev_t *mddev)
 
 	if (mddev_trylock(mddev)==0) {
 		int spares =0;
+		struct bio *blist;
 
+		spin_lock(&mddev->write_lock);
 		if (mddev->safemode && !atomic_read(&mddev->writes_pending) &&
 		    !mddev->in_sync && mddev->recovery_cp == MaxSector) {
 			mddev->in_sync = 1;
@@ -3613,9 +3645,22 @@ void md_check_recovery(mddev_t *mddev)
 		}
 		if (mddev->safemode == 1)
 			mddev->safemode = 0;
+		blist = bio_list_get(&mddev->write_list);
+		spin_unlock(&mddev->write_lock);
 
 		if (mddev->sb_dirty)
 			md_update_sb(mddev);
+
+		while (blist) {
+			struct bio *b = blist;
+			blist = blist->bi_next;
+			b->bi_next = NULL;
+			generic_make_request(b);
+			/* we already counted this, so need to un-count */
+			md_write_end(mddev);
+		}
+
+
 		if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery) &&
 		    !test_bit(MD_RECOVERY_DONE, &mddev->recovery)) {
 			/* resync/recovery still happening */
drivers/md/raid1.c renamed from drivers/md/raid1.c~current~ +3 -1
diff ./drivers/md/raid1.c~current~ ./drivers/md/raid1.c
@@ -530,6 +530,8 @@ static int make_request(request_queue_t 
 	 * thread has put up a bar for new requests.
 	 * Continue immediately if no resync is active currently.
 	 */
+	if (md_write_start(mddev, bio)==0)
+		return 0;
 	spin_lock_irq(&conf->resync_lock);
 	wait_event_lock_irq(conf->wait_resume, !conf->barrier, conf->resync_lock, );
 	conf->nr_pending++;
@@ -611,7 +613,7 @@ static int make_request(request_queue_t 
 	rcu_read_unlock();
 
 	atomic_set(&r1_bio->remaining, 1);
-	md_write_start(mddev);
+
 	for (i = 0; i < disks; i++) {
 		struct bio *mbio;
 		if (!r1_bio->bios[i])
drivers/md/raid10.c renamed from drivers/md/raid10.c~current~ +4 -1
diff ./drivers/md/raid10.c~current~ ./drivers/md/raid10.c
@@ -700,6 +700,9 @@ static int make_request(request_queue_t 
 		return 0;
 	}
 
+	if (md_write_start(mddev, bio) == 0)
+		return 0;
+
 	/*
 	 * Register the new request and wait if the reconstruction
 	 * thread has put up a bar for new requests.
@@ -774,7 +777,7 @@ static int make_request(request_queue_t 
 	rcu_read_unlock();
 
 	atomic_set(&r10_bio->remaining, 1);
-	md_write_start(mddev);
+
 	for (i = 0; i < conf->copies; i++) {
 		struct bio *mbio;
 		int d = r10_bio->devs[i].devnum;
drivers/md/raid5.c renamed from drivers/md/raid5.c~current~ +4 -2
diff ./drivers/md/raid5.c~current~ ./drivers/md/raid5.c
@@ -1409,6 +1409,9 @@ static int make_request (request_queue_t
 	sector_t logical_sector, last_sector;
 	struct stripe_head *sh;
 
+	if (md_write_start(mddev, bi)==0)
+		return 0;
+
 	if (bio_data_dir(bi)==WRITE) {
 		disk_stat_inc(mddev->gendisk, writes);
 		disk_stat_add(mddev->gendisk, write_sectors, bio_sectors(bi));
@@ -1421,8 +1424,7 @@ static int make_request (request_queue_t
 	last_sector = bi->bi_sector + (bi->bi_size>>9);
 	bi->bi_next = NULL;
 	bi->bi_phys_segments = 1;	/* over-loaded to count active stripes */
-	if ( bio_data_dir(bi) == WRITE )
-		md_write_start(mddev);
+
 	for (;logical_sector < last_sector; logical_sector += STRIPE_SECTORS) {
 		DEFINE_WAIT(w);
 		
drivers/md/raid6main.c renamed from drivers/md/raid6main.c~current~ +4 -2
diff ./drivers/md/raid6main.c~current~ ./drivers/md/raid6main.c
@@ -1568,6 +1568,9 @@ static int make_request (request_queue_t
 	sector_t logical_sector, last_sector;
 	struct stripe_head *sh;
 
+	if (md_write_start(mddev, bi)==0)
+		return 0;
+
 	if (bio_data_dir(bi)==WRITE) {
 		disk_stat_inc(mddev->gendisk, writes);
 		disk_stat_add(mddev->gendisk, write_sectors, bio_sectors(bi));
@@ -1581,8 +1584,7 @@ static int make_request (request_queue_t
 
 	bi->bi_next = NULL;
 	bi->bi_phys_segments = 1;	/* over-loaded to count active stripes */
-	if ( bio_data_dir(bi) == WRITE )
-		md_write_start(mddev);
+
 	for (;logical_sector < last_sector; logical_sector += STRIPE_SECTORS) {
 		DEFINE_WAIT(w);
 
include/linux/raid/md.h renamed from include/linux/raid/md.h~current~ +1 -1
diff ./include/linux/raid/md.h~current~ ./include/linux/raid/md.h
@@ -69,7 +69,7 @@ extern mdk_thread_t * md_register_thread
 extern void md_unregister_thread (mdk_thread_t *thread);
 extern void md_wakeup_thread(mdk_thread_t *thread);
 extern void md_check_recovery(mddev_t *mddev);
-extern void md_write_start(mddev_t *mddev);
+extern int md_write_start(mddev_t *mddev, struct bio *bi);
 extern void md_write_end(mddev_t *mddev);
 extern void md_handle_safemode(mddev_t *mddev);
 extern void md_done_sync(mddev_t *mddev, int blocks, int ok);
include/linux/raid/md_k.h renamed from include/linux/raid/md_k.h~current~ +7 -0
diff ./include/linux/raid/md_k.h~current~ ./include/linux/raid/md_k.h
@@ -15,6 +15,9 @@
 #ifndef _MD_K_H
 #define _MD_K_H
 
+/* and dm-bio-list.h is not under include/linux because.... ??? */
+#include "../../../drivers/md/dm-bio-list.h"
+
 #define MD_RESERVED       0UL
 #define LINEAR            1UL
 #define RAID0             2UL
@@ -252,6 +255,10 @@ struct mddev_s
 	atomic_t			recovery_active; /* blocks scheduled, but not written */
 	wait_queue_head_t		recovery_wait;
 	sector_t			recovery_cp;
+
+	spinlock_t			write_lock;
+	struct bio_list			write_list;
+
 	unsigned int			safemode;	/* if set, update "clean" superblock
 							 * when no writes pending.
 							 */ 
author
#6 lore raw

PATCH 1/1 Improve the interface to sync_request

In reply to NeilBrown

1/ change the return value (which is number-of-sectors synced)
from 'int' to 'sector_t'.
The number of sectors is usually easily small enough to fit in an int, but if resync needs to abort, it may want to return the total number of remaining sectors, which could be large. Also errors cannot be returned as negative numbers now, so use 0 instead
2/ Add a 'skipped' return parameter to allow the array to report
that it skipped the sectors. This allows md to take this into account in the speed calculations.
Currently there is no important skipping, but the bitmap-based-resync that is coming will use this.

### Diffstat output

 ./drivers/md/md.c           |   39 ++++++++++++++++++++++++---------------
 ./drivers/md/raid1.c        |    8 ++++----
 ./drivers/md/raid10.c       |   19 ++++++++++++-------
 ./drivers/md/raid5.c        |    6 +++---
 ./drivers/md/raid6main.c    |    6 +++---
 ./include/linux/raid/md_k.h |    2 +-
 6 files changed, 47 insertions(+), 33 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +24 -15
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -3361,12 +3361,13 @@ static void md_do_sync(mddev_t *mddev)
 	mddev_t *mddev2;
 	unsigned int currspeed = 0,
 		 window;
-	sector_t max_sectors,j;
+	sector_t max_sectors,j, io_sectors;
 	unsigned long mark[SYNC_MARKS];
 	sector_t mark_cnt[SYNC_MARKS];
 	int last_mark,m;
 	struct list_head *tmp;
 	sector_t last_check;
+	int skipped = 0;
 
 	/* just incase thread restarts... */
 	if (test_bit(MD_RECOVERY_DONE, &mddev->recovery))
@@ -3432,7 +3433,7 @@ static void md_do_sync(mddev_t *mddev)
 
 	if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery))
 		/* resync follows the size requested by the personality,
-		 * which default to physical size, but can be virtual size
+		 * which defaults to physical size, but can be virtual size
 		 */
 		max_sectors = mddev->resync_max_sectors;
 	else
@@ -3451,9 +3452,10 @@ static void md_do_sync(mddev_t *mddev)
 		j = mddev->recovery_cp;
 	else
 		j = 0;
+	io_sectors = 0;
 	for (m = 0; m < SYNC_MARKS; m++) {
 		mark[m] = jiffies;
-		mark_cnt[m] = j;
+		mark_cnt[m] = io_sectors;
 	}
 	last_mark = 0;
 	mddev->resync_mark = mark[last_mark];
@@ -3478,26 +3480,32 @@ static void md_do_sync(mddev_t *mddev)
 	}
 
 	while (j < max_sectors) {
-		int sectors;
-
-		sectors = mddev->pers->sync_request(mddev, j, currspeed < sysctl_speed_limit_min);
-		if (sectors < 0) {
+		sector_t sectors;
+		skipped = 0;
+		sectors = mddev->pers->sync_request(mddev, j, &skipped,
+						    currspeed < sysctl_speed_limit_min);
+		if (sectors == 0) {
 			set_bit(MD_RECOVERY_ERR, &mddev->recovery);
 			goto out;
 		}
-		atomic_add(sectors, &mddev->recovery_active);
+		if (!skipped) { /* actual IO requested */
+			io_sectors += sectors;
+			atomic_add(sectors, &mddev->recovery_active);
+		}
+
 		j += sectors;
 		if (j>1) mddev->curr_resync = j;
+
+		if (last_check + window > io_sectors || j == max_sectors)
+			continue;
+
 		if (last_check==0)
 			/* This is the earliest that rebuild will be visible
 			 * in /proc/mdstat
 			 */
 			md_new_event();
 
-		if (last_check + window > j || j == max_sectors)
-			continue;
-
-		last_check = j;
+		last_check = io_sectors;
 
 		if (test_bit(MD_RECOVERY_INTR, &mddev->recovery) ||
 		    test_bit(MD_RECOVERY_ERR, &mddev->recovery))
@@ -3511,7 +3519,7 @@ static void md_do_sync(mddev_t *mddev)
 			mddev->resync_mark = mark[next];
 			mddev->resync_mark_cnt = mark_cnt[next];
 			mark[next] = jiffies;
-			mark_cnt[next] = j - atomic_read(&mddev->recovery_active);
+			mark_cnt[next] = io_sectors - atomic_read(&mddev->recovery_active);
 			last_mark = next;
 		}
 
@@ -3538,7 +3546,8 @@ static void md_do_sync(mddev_t *mddev)
 		mddev->queue->unplug_fn(mddev->queue);
 		cond_resched();
 
-		currspeed = ((unsigned long)(j-mddev->resync_mark_cnt))/2/((jiffies-mddev->resync_mark)/HZ +1) +1;
+		currspeed = ((unsigned long)(io_sectors-mddev->resync_mark_cnt))/2
+			/((jiffies-mddev->resync_mark)/HZ +1) +1;
 
 		if (currspeed > sysctl_speed_limit_min) {
 			if ((currspeed > sysctl_speed_limit_max) ||
@@ -3558,7 +3567,7 @@ static void md_do_sync(mddev_t *mddev)
 	wait_event(mddev->recovery_wait, !atomic_read(&mddev->recovery_active));
 
 	/* tell personality that we are finished */
-	mddev->pers->sync_request(mddev, max_sectors, 1);
+	mddev->pers->sync_request(mddev, max_sectors, &skipped, 1);
 
 	if (!test_bit(MD_RECOVERY_ERR, &mddev->recovery) &&
 	    mddev->curr_resync > 2 &&
drivers/md/raid1.c renamed from drivers/md/raid1.c~current~ +4 -4
diff ./drivers/md/raid1.c~current~ ./drivers/md/raid1.c
@@ -1010,7 +1010,7 @@ static int init_resync(conf_t *conf)
  * that can be installed to exclude normal IO requests.
  */
 
-static int sync_request(mddev_t *mddev, sector_t sector_nr, int go_faster)
+static sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, int go_faster)
 {
 	conf_t *conf = mddev_to_conf(mddev);
 	mirror_info_t *mirror;
@@ -1023,7 +1023,7 @@ static int sync_request(mddev_t *mddev, 
 
 	if (!conf->r1buf_pool)
 		if (init_resync(conf))
-			return -ENOMEM;
+			return 0;
 
 	max_sector = mddev->size << 1;
 	if (sector_nr >= max_sector) {
@@ -1107,8 +1107,8 @@ static int sync_request(mddev_t *mddev, 
 		/* There is nowhere to write, so all non-sync
 		 * drives must be failed - so we are finished
 		 */
-		int rv = max_sector - sector_nr;
-		md_done_sync(mddev, rv, 1);
+		sector_t rv = max_sector - sector_nr;
+		*skipped = 1;
 		put_buf(r1_bio);
 		rdev_dec_pending(conf->mirrors[disk].rdev, mddev);
 		return rv;
drivers/md/raid10.c renamed from drivers/md/raid10.c~current~ +12 -7
diff ./drivers/md/raid10.c~current~ ./drivers/md/raid10.c
@@ -1321,7 +1321,7 @@ static int init_resync(conf_t *conf)
  *
  */
 
-static int sync_request(mddev_t *mddev, sector_t sector_nr, int go_faster)
+static sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, int go_faster)
 {
 	conf_t *conf = mddev_to_conf(mddev);
 	r10bio_t *r10_bio;
@@ -1335,7 +1335,7 @@ static int sync_request(mddev_t *mddev, 
 
 	if (!conf->r10buf_pool)
 		if (init_resync(conf))
-			return -ENOMEM;
+			return 0;
 
  skipped:
 	max_sector = mddev->size << 1;
@@ -1343,15 +1343,15 @@ static int sync_request(mddev_t *mddev, 
 		max_sector = mddev->resync_max_sectors;
 	if (sector_nr >= max_sector) {
 		close_sync(conf);
+		*skipped = 1;
 		return sectors_skipped;
 	}
 	if (chunks_skipped >= conf->raid_disks) {
 		/* if there has been nothing to do on any drive,
 		 * then there is nothing to do at all..
 		 */
-		sector_t sec = max_sector - sector_nr;
-		md_done_sync(mddev, sec, 1);
-		return sec + sectors_skipped;
+		*skipped = 1;
+		return (max_sector - sector_nr) + sectors_skipped;
 	}
 
 	/* make sure whole request will fit in a chunk - if chunks
@@ -1565,17 +1565,22 @@ static int sync_request(mddev_t *mddev, 
 		}
 	}
 
+	if (sectors_skipped)
+		/* pretend they weren't skipped, it makes
+		 * no important difference in this case
+		 */
+		md_done_sync(mddev, sectors_skipped, 1);
+
 	return sectors_skipped + nr_sectors;
  giveup:
 	/* There is nowhere to write, so all non-sync
 	 * drives must be failed, so try the next chunk...
 	 */
 	{
-	int sec = max_sector - sector_nr;
+	sector_t sec = max_sector - sector_nr;
 	sectors_skipped += sec;
 	chunks_skipped ++;
 	sector_nr = max_sector;
-	md_done_sync(mddev, sec, 1);
 	goto skipped;
 	}
 }
drivers/md/raid5.c renamed from drivers/md/raid5.c~current~ +3 -3
diff ./drivers/md/raid5.c~current~ ./drivers/md/raid5.c
@@ -1475,7 +1475,7 @@ static int make_request (request_queue_t
 }
 
 /* FIXME go_faster isn't used */
-static int sync_request (mddev_t *mddev, sector_t sector_nr, int go_faster)
+static sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, int go_faster)
 {
 	raid5_conf_t *conf = (raid5_conf_t *) mddev->private;
 	struct stripe_head *sh;
@@ -1498,8 +1498,8 @@ static int sync_request (mddev_t *mddev,
 	 * nothing we can do.
 	 */
 	if (mddev->degraded >= 1 && test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
-		int rv = (mddev->size << 1) - sector_nr;
-		md_done_sync(mddev, rv, 1);
+		sector_t rv = (mddev->size << 1) - sector_nr;
+		*skipped = 1;
 		return rv;
 	}
 
drivers/md/raid6main.c renamed from drivers/md/raid6main.c~current~ +3 -3
diff ./drivers/md/raid6main.c~current~ ./drivers/md/raid6main.c
@@ -1634,7 +1634,7 @@ static int make_request (request_queue_t
 }
 
 /* FIXME go_faster isn't used */
-static int sync_request (mddev_t *mddev, sector_t sector_nr, int go_faster)
+static sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, int go_faster)
 {
 	raid6_conf_t *conf = (raid6_conf_t *) mddev->private;
 	struct stripe_head *sh;
@@ -1657,8 +1657,8 @@ static int sync_request (mddev_t *mddev,
 	 * nothing we can do.
 	 */
 	if (mddev->degraded >= 2 && test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
-		int rv = (mddev->size << 1) - sector_nr;
-		md_done_sync(mddev, rv, 1);
+		sector_t rv = (mddev->size << 1) - sector_nr;
+		*skipped = 1;
 		return rv;
 	}
 
include/linux/raid/md_k.h renamed from include/linux/raid/md_k.h~current~ +1 -1
diff ./include/linux/raid/md_k.h~current~ ./include/linux/raid/md_k.h
@@ -298,7 +298,7 @@ struct mdk_personality_s
 	int (*hot_add_disk) (mddev_t *mddev, mdk_rdev_t *rdev);
 	int (*hot_remove_disk) (mddev_t *mddev, int number);
 	int (*spare_active) (mddev_t *mddev);
-	int (*sync_request)(mddev_t *mddev, sector_t sector_nr, int go_faster);
+	sector_t (*sync_request)(mddev_t *mddev, sector_t sector_nr, int *skipped, int go_faster);
 	int (*resize) (mddev_t *mddev, sector_t sectors);
 	int (*reshape) (mddev_t *mddev, int raid_disks);
 	int (*reconfig) (mddev_t *mddev, int layout, int chunk_size);
author
#7 lore raw

PATCH 1/1 Optimised resync using Bitmap based intent logging

In reply to NeilBrown

With this patch, the intent to write to some block in the array can be logged to a bitmap file. Each bit represents some number of sectors and is set before any update happens, and only cleared when all writes relating to all sectors are complete.

After an unclean shutdown, information in this bitmap can be used to optimise resync - only sectors which could be out-of-sync need to be updated.

Also if a drive is removed and then added back into an array, the recovery can make use of the bitmap to optimise reconstruction. This is not implemented in this patch.

Currently the bitmap is stored in a file which must (obviously) be stored on a separate device.

The patch only provided infrastructure. It does not update any personalities to bitmap intent logging.

Md arrays can still be used with no bitmap file. This patch has minimal impact on such arrays.

This is still a work-in-progress. Don't trust data to it yet.

### Diffstat output

 ./drivers/md/Makefile         |    3 
 ./drivers/md/bitmap.c         | 1517 ++++++++++++++++++++++++++++++++++++++++++
 ./drivers/md/md.c             |  172 ++++
 ./include/linux/raid/bitmap.h |  280 +++++++
 ./include/linux/raid/md_k.h   |    4 
 ./include/linux/raid/md_u.h   |    7 
 6 files changed, 1968 insertions(+), 15 deletions(-)
drivers/md/Makefile renamed from drivers/md/Makefile~current~ +2 -1
diff ./drivers/md/Makefile~current~ ./drivers/md/Makefile
@@ -6,6 +6,7 @@ dm-mod-objs	:= dm.o dm-table.o dm-target
 		   dm-ioctl.o dm-io.o kcopyd.o
 dm-snapshot-objs := dm-snap.o dm-exception-store.o
 dm-mirror-objs	:= dm-log.o dm-raid1.o
+md-mod-objs     := md.o bitmap.o
 raid6-objs	:= raid6main.o raid6algos.o raid6recov.o raid6tables.o \
 		   raid6int1.o raid6int2.o raid6int4.o \
 		   raid6int8.o raid6int16.o raid6int32.o \
@@ -27,7 +28,7 @@ obj-$(CONFIG_MD_RAID5)		+= raid5.o xor.o
 obj-$(CONFIG_MD_RAID6)		+= raid6.o xor.o
 obj-$(CONFIG_MD_MULTIPATH)	+= multipath.o
 obj-$(CONFIG_MD_FAULTY)		+= faulty.o
-obj-$(CONFIG_BLK_DEV_MD)	+= md.o
+obj-$(CONFIG_BLK_DEV_MD)	+= md-mod.o
 obj-$(CONFIG_BLK_DEV_DM)	+= dm-mod.o
 obj-$(CONFIG_DM_CRYPT)		+= dm-crypt.o
 obj-$(CONFIG_DM_SNAPSHOT)	+= dm-snapshot.o
drivers/md/bitmap.c renamed from drivers/md/bitmap.c~current~ +1517 -0

1,517 lines not on this page · raw message

drivers/md/md.c renamed from drivers/md/md.c~current~ +158 -14

297 lines not on this page · raw message

include/linux/raid/bitmap.h renamed from include/linux/raid/bitmap.h~current~ +280 -0

280 lines not on this page · raw message

include/linux/raid/md_k.h renamed from include/linux/raid/md_k.h~current~ +4 -0
diff ./include/linux/raid/md_k.h~current~ ./include/linux/raid/md_k.h
@@ -267,6 +267,9 @@ struct mddev_s
 	atomic_t			writes_pending; 
 	request_queue_t			*queue;	/* for plugging ... */
 
+	struct bitmap                   *bitmap; /* the bitmap for the device */
+	struct file			*bitmap_file; /* the bitmap file */
+
 	struct list_head		all_mddevs;
 };
 
@@ -341,6 +344,7 @@ typedef struct mdk_thread_s {
 	unsigned long           flags;
 	struct completion	*event;
 	struct task_struct	*tsk;
+	unsigned long		timeout;
 	const char		*name;
 } mdk_thread_t;
 
include/linux/raid/md_u.h renamed from include/linux/raid/md_u.h~current~ +7 -0
diff ./include/linux/raid/md_u.h~current~ ./include/linux/raid/md_u.h
@@ -23,6 +23,7 @@
 #define GET_DISK_INFO		_IOR (MD_MAJOR, 0x12, mdu_disk_info_t)
 #define PRINT_RAID_DEBUG	_IO (MD_MAJOR, 0x13)
 #define RAID_AUTORUN		_IO (MD_MAJOR, 0x14)
+#define GET_BITMAP_FILE		_IOR (MD_MAJOR, 0x15, mdu_bitmap_file_t)
 
 /* configuration */
 #define CLEAR_ARRAY		_IO (MD_MAJOR, 0x20)
@@ -36,6 +37,7 @@
 #define HOT_ADD_DISK		_IO (MD_MAJOR, 0x28)
 #define SET_DISK_FAULTY		_IO (MD_MAJOR, 0x29)
 #define HOT_GENERATE_ERROR	_IO (MD_MAJOR, 0x2a)
+#define SET_BITMAP_FILE		_IOW (MD_MAJOR, 0x2b, int)
 
 /* usage */
 #define RUN_ARRAY		_IOW (MD_MAJOR, 0x30, mdu_param_t)
@@ -106,6 +108,11 @@ typedef struct mdu_start_info_s {
 
 } mdu_start_info_t;
 
+typedef struct mdu_bitmap_file_s
+{
+	char pathname[4096];
+} mdu_bitmap_file_t;
+
 typedef struct mdu_param_s
 {
 	int			personality;	/* 1,2,3,4 */
author
#8 lore raw

PATCH 1/1 raid1 support for bitmap intent logging

In reply to NeilBrown

### Diffstat output

 ./drivers/md/raid1.c         |  184 ++++++++++++++++++++++++++++++++++++-------
 ./include/linux/raid/raid1.h |   16 +++
 2 files changed, 169 insertions(+), 31 deletions(-)
drivers/md/raid1.c renamed from drivers/md/raid1.c~current~ +155 -29
diff ./drivers/md/raid1.c~current~ ./drivers/md/raid1.c
@@ -12,6 +12,15 @@
  * Fixes to reconstruction by Jakob Østergaard" <[email protected]>
  * Various fixes by Neil Brown <[email protected]>
  *
+ * Changes by Peter T. Breuer <[email protected]> 31/1/2003 to support
+ * bitmapped intelligence in resync:
+ *
+ *      - bitmap marked during normal i/o
+ *      - bitmap used to skip nondirty blocks during sync
+ *
+ * Additions to bitmap code, (C) 2003-2004 Paul Clements, SteelEye Technology:
+ * - persistent bitmap code
+ *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation; either version 2, or (at your option)
@@ -22,7 +31,16 @@
  * Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
  */
 
+#include "dm-bio-list.h"
 #include <linux/raid/raid1.h>
+#include <linux/raid/bitmap.h>
+
+#define DEBUG 0
+#if DEBUG
+#define PRINTK(x...) printk(x)
+#else
+#define PRINTK(x...)
+#endif
 
 /*
  * Number of guaranteed r1bios in case of extreme VM load:
@@ -287,9 +305,11 @@ static int raid1_end_write_request(struc
 	/*
 	 * this branch is our 'one mirror IO has finished' event handler:
 	 */
-	if (!uptodate)
+	if (!uptodate) {
 		md_error(r1_bio->mddev, conf->mirrors[mirror].rdev);
-	else
+		/* an I/O failed, we can't clear the bitmap */
+		set_bit(R1BIO_Degraded, &r1_bio->state);
+	} else
 		/*
 		 * Set R1BIO_Uptodate in our master bio, so that
 		 * we will return a good error code for to the higher
@@ -309,6 +329,10 @@ static int raid1_end_write_request(struc
 	 * already.
 	 */
 	if (atomic_dec_and_test(&r1_bio->remaining)) {
+		/* clear the bitmap if all writes complete successfully */
+		bitmap_endwrite(r1_bio->mddev->bitmap, r1_bio->sector,
+				r1_bio->sectors,
+				!test_bit(R1BIO_Degraded, &r1_bio->state));
 		md_write_end(r1_bio->mddev);
 		raid_end_bio_io(r1_bio);
 	}
@@ -458,7 +482,10 @@ static void unplug_slaves(mddev_t *mddev
 
 static void raid1_unplug(request_queue_t *q)
 {
-	unplug_slaves(q->queuedata);
+	mddev_t *mddev = q->queuedata;
+
+	unplug_slaves(mddev);
+	md_wakeup_thread(mddev->thread);
 }
 
 static int raid1_issue_flush(request_queue_t *q, struct gendisk *disk,
@@ -501,16 +528,16 @@ static void device_barrier(conf_t *conf,
 {
 	spin_lock_irq(&conf->resync_lock);
 	wait_event_lock_irq(conf->wait_idle, !waitqueue_active(&conf->wait_resume),
-			    conf->resync_lock, unplug_slaves(conf->mddev));
+			    conf->resync_lock, raid1_unplug(conf->mddev->queue));
 	
 	if (!conf->barrier++) {
 		wait_event_lock_irq(conf->wait_idle, !conf->nr_pending,
-				    conf->resync_lock, unplug_slaves(conf->mddev));
+				    conf->resync_lock, raid1_unplug(conf->mddev->queue));
 		if (conf->nr_pending)
 			BUG();
 	}
 	wait_event_lock_irq(conf->wait_resume, conf->barrier < RESYNC_DEPTH,
-			    conf->resync_lock, unplug_slaves(conf->mddev));
+			    conf->resync_lock, raid1_unplug(conf->mddev->queue));
 	conf->next_resync = sect;
 	spin_unlock_irq(&conf->resync_lock);
 }
@@ -522,8 +549,12 @@ static int make_request(request_queue_t 
 	mirror_info_t *mirror;
 	r1bio_t *r1_bio;
 	struct bio *read_bio;
-	int i, disks;
+	int i, targets = 0, disks;
 	mdk_rdev_t *rdev;
+	struct bitmap *bitmap = mddev->bitmap;
+	unsigned long flags;
+	struct bio_list bl;
+
 
 	/*
 	 * Register the new request and wait if the reconstruction
@@ -554,7 +585,7 @@ static int make_request(request_queue_t 
 
 	r1_bio->master_bio = bio;
 	r1_bio->sectors = bio->bi_size >> 9;
-
+	r1_bio->state = 0;
 	r1_bio->mddev = mddev;
 	r1_bio->sector = bio->bi_sector;
 
@@ -597,6 +628,13 @@ static int make_request(request_queue_t 
 	 * bios[x] to bio
 	 */
 	disks = conf->raid_disks;
+#if 0
+	{ static int first=1;
+	if (first) printk("First Write sector %llu disks %d\n",
+			  (unsigned long long)r1_bio->sector, disks);
+	first = 0;
+	}
+#endif
 	rcu_read_lock();
 	for (i = 0;  i < disks; i++) {
 		if ((rdev=conf->mirrors[i].rdev) != NULL &&
@@ -607,13 +645,21 @@ static int make_request(request_queue_t 
 				r1_bio->bios[i] = NULL;
 			} else
 				r1_bio->bios[i] = bio;
+			targets++;
 		} else
 			r1_bio->bios[i] = NULL;
 	}
 	rcu_read_unlock();
 
-	atomic_set(&r1_bio->remaining, 1);
+	if (targets < conf->raid_disks) {
+		/* array is degraded, we will not clear the bitmap
+		 * on I/O completion (see raid1_end_write_request) */
+		set_bit(R1BIO_Degraded, &r1_bio->state);
+	}
+
+	atomic_set(&r1_bio->remaining, 0);
 
+	bio_list_init(&bl);
 	for (i = 0; i < disks; i++) {
 		struct bio *mbio;
 		if (!r1_bio->bios[i])
@@ -629,14 +675,23 @@ static int make_request(request_queue_t 
 		mbio->bi_private = r1_bio;
 
 		atomic_inc(&r1_bio->remaining);
-		generic_make_request(mbio);
-	}
 
-	if (atomic_dec_and_test(&r1_bio->remaining)) {
-		md_write_end(mddev);
-		raid_end_bio_io(r1_bio);
+		bio_list_add(&bl, mbio);
 	}
 
+	bitmap_startwrite(bitmap, bio->bi_sector, r1_bio->sectors);
+	spin_lock_irqsave(&conf->device_lock, flags);
+	bio_list_merge(&conf->pending_bio_list, &bl);
+	bio_list_init(&bl);
+
+	blk_plug_device(mddev->queue);
+	spin_unlock_irqrestore(&conf->device_lock, flags);
+	
+#if 0
+	while ((bio = bio_list_pop(&bl)) != NULL) 
+		generic_make_request(bio);
+#endif
+
 	return 0;
 }
 
@@ -716,7 +771,7 @@ static void close_sync(conf_t *conf)
 {
 	spin_lock_irq(&conf->resync_lock);
 	wait_event_lock_irq(conf->wait_resume, !conf->barrier,
-			    conf->resync_lock, 	unplug_slaves(conf->mddev));
+			    conf->resync_lock, 	raid1_unplug(conf->mddev->queue));
 	spin_unlock_irq(&conf->resync_lock);
 
 	if (conf->barrier) BUG();
@@ -830,10 +885,11 @@ static int end_sync_read(struct bio *bio
 	 * or re-read if the read failed.
 	 * We don't do much here, just schedule handling by raid1d
 	 */
-	if (!uptodate)
+	if (!uptodate) {
 		md_error(r1_bio->mddev,
 			 conf->mirrors[r1_bio->read_disk].rdev);
-	else
+		set_bit(R1BIO_Degraded, &r1_bio->state);
+	} else
 		set_bit(R1BIO_Uptodate, &r1_bio->state);
 	rdev_dec_pending(conf->mirrors[r1_bio->read_disk].rdev, conf->mddev);
 	reschedule_retry(r1_bio);
@@ -857,8 +913,10 @@ static int end_sync_write(struct bio *bi
 			mirror = i;
 			break;
 		}
-	if (!uptodate)
+	if (!uptodate) {
 		md_error(mddev, conf->mirrors[mirror].rdev);
+		set_bit(R1BIO_Degraded, &r1_bio->state);
+	}
 	update_head_pos(mirror, r1_bio);
 
 	if (atomic_dec_and_test(&r1_bio->remaining)) {
@@ -878,6 +936,9 @@ static void sync_request_write(mddev_t *
 
 	bio = r1_bio->bios[r1_bio->read_disk];
 
+/*
+	if (r1_bio->sector == 0) printk("First sync write startss\n");
+*/
 	/*
 	 * schedule writes
 	 */
@@ -905,10 +966,12 @@ static void sync_request_write(mddev_t *
 		atomic_inc(&conf->mirrors[i].rdev->nr_pending);
 		atomic_inc(&r1_bio->remaining);
 		md_sync_acct(conf->mirrors[i].rdev->bdev, wbio->bi_size >> 9);
+
 		generic_make_request(wbio);
 	}
 
 	if (atomic_dec_and_test(&r1_bio->remaining)) {
+		/* if we're here, all write(s) have completed, so clean up */
 		md_done_sync(mddev, r1_bio->sectors, 1);
 		put_buf(r1_bio);
 	}
@@ -937,6 +1000,26 @@ static void raid1d(mddev_t *mddev)
 	for (;;) {
 		char b[BDEVNAME_SIZE];
 		spin_lock_irqsave(&conf->device_lock, flags);
+
+		if (conf->pending_bio_list.head) {
+			bio = bio_list_get(&conf->pending_bio_list);
+			blk_remove_plug(mddev->queue);
+			spin_unlock_irqrestore(&conf->device_lock, flags);
+			/* flush any pending bitmap writes to disk before proceeding w/ I/O */
+			if (bitmap_unplug(mddev->bitmap) != 0)
+				printk("%s: bitmap file write failed!\n", mdname(mddev));
+
+			while (bio) { /* submit pending writes */
+				struct bio *next = bio->bi_next;
+				bio->bi_next = NULL;
+				generic_make_request(bio);
+				bio = next;
+			}
+			unplug = 1;
+
+			continue;
+		}
+
 		if (list_empty(head))
 			break;
 		r1_bio = list_entry(head->prev, r1bio_t, retry_list);
@@ -1020,17 +1103,43 @@ static sector_t sync_request(mddev_t *md
 	int disk;
 	int i;
 	int write_targets = 0;
+	int sync_blocks;
 
-	if (!conf->r1buf_pool)
+	if (!conf->r1buf_pool) 
+	{
+/*
+		printk("sync start - bitmap %p\n", mddev->bitmap);
+*/
 		if (init_resync(conf))
 			return 0;
+	}
 
 	max_sector = mddev->size << 1;
 	if (sector_nr >= max_sector) {
+		/* If we aborted, we need to abort the
+		 * sync on the 'current' bitmap chunk (there will
+		 * only be one in raid1 resync.
+		 * We can find the current addess in mddev->curr_resync
+		 */
+		if (!conf->fullsync) {
+			if (mddev->curr_resync < max_sector)
+				bitmap_end_sync(mddev->bitmap,
+						mddev->curr_resync,
+						&sync_blocks, 1);
+			bitmap_close_sync(mddev->bitmap);
+		}
+		if (mddev->curr_resync >= max_sector)
+			conf->fullsync = 0;
 		close_sync(conf);
 		return 0;
 	}
 
+	if (!conf->fullsync &&
+	    !bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks)) {
+		/* We can skip this block, and probably several more */
+		*skipped = 1;
+		return sync_blocks;
+	}
 	/*
 	 * If there is non-resync activity waiting for us then
 	 * put in a delay to throttle resync.
@@ -1069,6 +1178,7 @@ static sector_t sync_request(mddev_t *md
 
 	r1_bio->mddev = mddev;
 	r1_bio->sector = sector_nr;
+	r1_bio->state = 0;
 	set_bit(R1BIO_IsSync, &r1_bio->state);
 	r1_bio->read_disk = disk;
 
@@ -1103,6 +1213,11 @@ static sector_t sync_request(mddev_t *md
 		bio->bi_bdev = conf->mirrors[i].rdev->bdev;
 		bio->bi_private = r1_bio;
 	}
+
+	if (write_targets + 1 < conf->raid_disks)
+		/* array degraded, can't clear bitmap */
+		set_bit(R1BIO_Degraded, &r1_bio->state);
+
 	if (write_targets == 0) {
 		/* There is nowhere to write, so all non-sync
 		 * drives must be failed - so we are finished
@@ -1122,6 +1237,14 @@ static sector_t sync_request(mddev_t *md
 			len = (max_sector - sector_nr) << 9;
 		if (len == 0)
 			break;
+		if (!conf->fullsync && sync_blocks == 0)
+			if (!bitmap_start_sync(mddev->bitmap, 
+					       sector_nr, &sync_blocks)) 
+				break;
+		if (sync_blocks < (PAGE_SIZE>>9))
+			BUG();
+		if (len > (sync_blocks<<9)) len = sync_blocks<<9;
+
 		for (i=0 ; i < conf->raid_disks; i++) {
 			bio = r1_bio->bios[i];
 			if (bio->bi_end_io) {
@@ -1144,6 +1267,7 @@ static sector_t sync_request(mddev_t *md
 		}
 		nr_sectors += len>>9;
 		sector_nr += len>>9;
+		sync_blocks -= (len>>9);
 	} while (r1_bio->bios[disk]->bi_vcnt < RESYNC_PAGES);
  bio_full:
 	bio = r1_bio->bios[disk];
@@ -1236,6 +1360,9 @@ static int run(mddev_t *mddev)
 	init_waitqueue_head(&conf->wait_idle);
 	init_waitqueue_head(&conf->wait_resume);
 
+	bio_list_init(&conf->pending_bio_list);
+	bio_list_init(&conf->flushing_bio_list);
+
 	if (!conf->working_disks) {
 		printk(KERN_ERR "raid1: no operational mirrors for %s\n",
 			mdname(mddev));
@@ -1264,16 +1391,15 @@ static int run(mddev_t *mddev)
 	conf->last_used = j;
 
 
-
-	{
-		mddev->thread = md_register_thread(raid1d, mddev, "%s_raid1");
-		if (!mddev->thread) {
-			printk(KERN_ERR 
-				"raid1: couldn't allocate thread for %s\n", 
-				mdname(mddev));
-			goto out_free_conf;
-		}
+	mddev->thread = md_register_thread(raid1d, mddev, "%s_raid1");
+	if (!mddev->thread) {
+		printk(KERN_ERR 
+		       "raid1: couldn't allocate thread for %s\n", 
+		       mdname(mddev));
+		goto out_free_conf;
 	}
+	if (mddev->bitmap) mddev->thread->timeout = mddev->bitmap->daemon_sleep * HZ;
+
 	printk(KERN_INFO 
 		"raid1: raid set %s active with %d out of %d mirrors\n",
 		mdname(mddev), mddev->raid_disks - mddev->degraded, 
@@ -1386,7 +1512,7 @@ static int raid1_reshape(mddev_t *mddev,
 	spin_lock_irq(&conf->resync_lock);
 	conf->barrier++;
 	wait_event_lock_irq(conf->wait_idle, !conf->nr_pending,
-			    conf->resync_lock, unplug_slaves(mddev));
+			    conf->resync_lock, raid1_unplug(mddev->queue));
 	spin_unlock_irq(&conf->resync_lock);
 
 	/* ok, everything is stopped */
include/linux/raid/raid1.h renamed from include/linux/raid/raid1.h~current~ +14 -2
diff ./include/linux/raid/raid1.h~current~ ./include/linux/raid/raid1.h
@@ -36,12 +36,21 @@ struct r1_private_data_s {
 	spinlock_t		device_lock;
 
 	struct list_head	retry_list;
+	/* queue pending writes and submit them on unplug */
+	struct bio_list		pending_bio_list;
+	/* queue of writes that have been unplugged */
+	struct bio_list		flushing_bio_list;
+
 	/* for use when syncing mirrors: */
 
 	spinlock_t		resync_lock;
-	int nr_pending;
-	int barrier;
+	int			nr_pending;
+	int			barrier;
 	sector_t		next_resync;
+	int			fullsync;  /* set to 1 if a full sync is needed,
+					    * (fresh device added).
+					    * Cleared when a sync completes.
+					    */
 
 	wait_queue_head_t	wait_idle;
 	wait_queue_head_t	wait_resume;
@@ -85,14 +94,17 @@ struct r1bio_s {
 	int			read_disk;
 
 	struct list_head	retry_list;
+	struct bitmap_update	*bitmap_update;
 	/*
 	 * if the IO is in WRITE direction, then multiple bios are used.
 	 * We choose the number when they are allocated.
 	 */
 	struct bio		*bios[0];
+	/* DO NOT PUT ANY NEW FIELDS HERE - bios array is contiguously alloced*/
 };
 
 /* bits for r1bio.state */
 #define	R1BIO_Uptodate	0
 #define	R1BIO_IsSync	1
+#define	R1BIO_Degraded	2
 #endif

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#9 lore raw

PATCH 1/1 Optimise reconstruction when re-adding a recently failed drive.

In reply to NeilBrown

When an array is degraded, bit in the intent-bitmap are never cleared. So if a recently failed drive is re-added, we only need to reconstruct the block that are still reflected in the bitmap.
This patch adds support for this re-adding.

### Diffstat output

 ./drivers/md/md.c           |   71 ++++++++++++++++++++++++++++++++++----------
 ./drivers/md/raid1.c        |    7 +++-
 ./include/linux/raid/md_k.h |    4 ++
 3 files changed, 65 insertions(+), 17 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +55 -16
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -620,6 +620,8 @@ static int super_90_validate(mddev_t *md
 	mdp_disk_t *desc;
 	mdp_super_t *sb = (mdp_super_t *)page_address(rdev->sb_page);
 
+	rdev->raid_disk = -1;
+	rdev->in_sync = 0;
 	if (mddev->raid_disks == 0) {
 		mddev->major_version = 0;
 		mddev->minor_version = sb->minor_version;
@@ -650,16 +652,24 @@ static int super_90_validate(mddev_t *md
 		memcpy(mddev->uuid+12,&sb->set_uuid3, 4);
 
 		mddev->max_disks = MD_SB_DISKS;
-	} else {
-		__u64 ev1;
-		ev1 = md_event(sb);
+	} else if (mddev->pers == NULL) {
+		/* Insist on good event counter while assembling */
+		__u64 ev1 = md_event(sb);
 		++ev1;
 		if (ev1 < mddev->events) 
 			return -EINVAL;
-	}
+	} else if (mddev->bitmap) {
+		/* if adding to array with a bitmap, then we can accept an
+		 * older device ... but not too old.
+		 */
+		__u64 ev1 = md_event(sb);
+		if (ev1 < mddev->bitmap->events_cleared)
+			return 0;
+	} else /* just a hot-add of a new device, leave raid_disk at -1 */
+		return 0;
+
 	if (mddev->level != LEVEL_MULTIPATH) {
-		rdev->raid_disk = -1;
-		rdev->in_sync = rdev->faulty = 0;
+		rdev->faulty = 0;
 		desc = sb->disks + rdev->desc_nr;
 
 		if (desc->state & (1<<MD_DISK_FAULTY))
@@ -669,7 +679,8 @@ static int super_90_validate(mddev_t *md
 			rdev->in_sync = 1;
 			rdev->raid_disk = desc->raid_disk;
 		}
-	}
+	} else /* MULTIPATH are always insync */
+		rdev->in_sync = 1;
 	return 0;
 }
 
@@ -911,6 +922,8 @@ static int super_1_validate(mddev_t *mdd
 {
 	struct mdp_superblock_1 *sb = (struct mdp_superblock_1*)page_address(rdev->sb_page);
 
+	rdev->raid_disk = -1;
+	rdev->in_sync = 0;
 	if (mddev->raid_disks == 0) {
 		mddev->major_version = 1;
 		mddev->patch_version = 0;
@@ -928,13 +941,21 @@ static int super_1_validate(mddev_t *mdd
 		memcpy(mddev->uuid, sb->set_uuid, 16);
 
 		mddev->max_disks =  (4096-256)/2;
-	} else {
-		__u64 ev1;
-		ev1 = le64_to_cpu(sb->events);
+	} else if (mddev->pers == NULL) {
+		/* Insist of good event counter while assembling */
+		__u64 ev1 = le64_to_cpu(sb->events);
 		++ev1;
 		if (ev1 < mddev->events)
 			return -EINVAL;
-	}
+	} else if (mddev->bitmap) {
+		/* If adding to array with a bitmap, then we can accept an
+		 * older device, but not too old.
+		 */
+		__u64 ev1 = le64_to_cpu(sb->events);
+		if (ev1 < mddev->bitmap->events_cleared)
+			return 0;
+	} else /* just a hot-add of a new device, leave raid_disk at -1 */
+		return 0;
 
 	if (mddev->level != LEVEL_MULTIPATH) {
 		int role;
@@ -942,14 +963,10 @@ static int super_1_validate(mddev_t *mdd
 		role = le16_to_cpu(sb->dev_roles[rdev->desc_nr]);
 		switch(role) {
 		case 0xffff: /* spare */
-			rdev->in_sync = 0;
 			rdev->faulty = 0;
-			rdev->raid_disk = -1;
 			break;
 		case 0xfffe: /* faulty */
-			rdev->in_sync = 0;
 			rdev->faulty = 1;
-			rdev->raid_disk = -1;
 			break;
 		default:
 			rdev->in_sync = 1;
@@ -957,7 +974,9 @@ static int super_1_validate(mddev_t *mdd
 			rdev->raid_disk = role;
 			break;
 		}
-	}
+	} else /* MULTIPATH are always insync */
+		rdev->in_sync = 1;
+
 	return 0;
 }
 
@@ -2211,6 +2230,18 @@ static int add_new_disk(mddev_t * mddev,
 				PTR_ERR(rdev));
 			return PTR_ERR(rdev);
 		}
+		/* set save_raid_disk if appropriate */
+		if (!mddev->persistent) {
+			if (info->state & (1<<MD_DISK_SYNC)  &&
+			    info->raid_disk < mddev->raid_disks)
+				rdev->raid_disk = info->raid_disk;
+			else
+				rdev->raid_disk = -1;
+		} else
+			super_types[mddev->major_version].
+				validate_super(mddev, rdev);
+		rdev->saved_raid_disk = rdev->raid_disk;
+
 		rdev->in_sync = 0; /* just to be sure */
 		rdev->raid_disk = -1;
 		err = bind_rdev_to_array(rdev, mddev);
@@ -3829,6 +3860,14 @@ void md_check_recovery(mddev_t *mddev)
 				mddev->pers->spare_active(mddev);
 			}
 			md_update_sb(mddev);
+
+			/* if array is no-longer degraded, then any saved_raid_disk
+			 * information must be scrapped
+			 */
+			if (!mddev->degraded)
+				ITERATE_RDEV(mddev,rdev,rtmp)
+					rdev->saved_raid_disk = -1;
+			
 			mddev->recovery = 0;
 			/* flag recovery needed just to double check */
 			set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
drivers/md/raid1.c renamed from drivers/md/raid1.c~current~ +6 -1
diff ./drivers/md/raid1.c~current~ ./drivers/md/raid1.c
@@ -811,9 +811,12 @@ static int raid1_add_disk(mddev_t *mddev
 {
 	conf_t *conf = mddev->private;
 	int found = 0;
-	int mirror;
+	int mirror = 0;
 	mirror_info_t *p;
 
+	if (rdev->saved_raid_disk >= 0 &&
+	    conf->mirrors[rdev->saved_raid_disk].rdev == NULL)
+		mirror = rdev->saved_raid_disk;
 	for (mirror=0; mirror < mddev->raid_disks; mirror++)
 		if ( !(p=conf->mirrors+mirror)->rdev) {
 
@@ -830,6 +833,8 @@ static int raid1_add_disk(mddev_t *mddev
 			p->head_position = 0;
 			rdev->raid_disk = mirror;
 			found = 1;
+			if (rdev->saved_raid_disk != mirror)
+				conf->fullsync = 1;
 			p->rdev = rdev;
 			break;
 		}
include/linux/raid/md_k.h renamed from include/linux/raid/md_k.h~current~ +4 -0
diff ./include/linux/raid/md_k.h~current~ ./include/linux/raid/md_k.h
@@ -183,6 +183,10 @@ struct mdk_rdev_s
 
 	int desc_nr;			/* descriptor index in the superblock */
 	int raid_disk;			/* role of device in array */
+	int saved_raid_disk;		/* role that device used to have in the 
+					 * array and could again if we did a partial
+					 * resync from the bitmap
+					 */
 
 	atomic_t	nr_pending;	/* number of pending requests.
 					 * only maintained for arrays that
#10 lore raw

Re: [PATCH md 9 of 9] Optimise reconstruction when re-adding a recently failed drive.

Quoted 5 lines · NeilBrown

When an array is degraded, bit in the intent-bitmap are never cleared. So if a recently failed drive is re-added, we only need to reconstruct the block that are still reflected in the bitmap.
This patch adds support for this re-adding.

Hi there -

If I understand this correctly, this means that:

1) if I had a raid1 mirror (for example) that has no writes to it since a resync
2) a drive fails out, and some writes occur
3) when I re-add the drive, only the areas where the writes occurred would be re-synced?

I can think of a bunch of peripheral questions around this scenario, and bad sectors / bad sector clearing, but I may not be understanding the basic idea, so I wanted to ask first.

-Mike

author
#11 lore raw

Re: [PATCH md 9 of 9] Optimise reconstruction when re-adding a recently failed drive.

Quoted 18 lines · On Thursday February 17, [email protected] wrote:
Quoted 5 lines · NeilBrown

When an array is degraded, bit in the intent-bitmap are never cleared. So if a recently failed drive is re-added, we only need to reconstruct the block that are still reflected in the bitmap.
This patch adds support for this re-adding.

Hi there -

If I understand this correctly, this means that:

1) if I had a raid1 mirror (for example) that has no writes to it since a resync
2) a drive fails out, and some writes occur
3) when I re-add the drive, only the areas where the writes occurred would be re-synced?

Quoted 3 lines

I can think of a bunch of peripheral questions around this scenario, and bad sectors / bad sector clearing, but I may not be understanding the basic idea, so I wanted to ask first.

You seem to understand the basic idea.
I believe one of the motivators for this code (I didn't originate it) is when a raid1 has one device locally and one device over a network connection.

If the network connection breaks, that device has to be thrown out. But when it comes back, we don't want to resync the whole array over the network. This functionality helps there (though there are a few other things needed before that scenario can work smoothly).

You would only re-add a device if you thought it was OK. i.e. if it was a connection problem rather than a media problem, or if you had resolved any media issues.

NeilBrown

#12 lore raw
In reply to NeilBrown

Would you recommend to apply this package
http://neilb.web.cse.unsw.edu.au/~neilb/patches/linux-devel/2.6/2005-02-18-00/patch-all-2005-02-18-00
To a 2.6.10 kernel?

Thank you for a bery good raid tool :)

"NeilBrown" <[email protected]> skrev i meddelandet news:20050218111131.790.patches@notabene...

Quoted 33 lines

9 patches for md in 2.6.11-rc3-mm2 follow.

1 - tightens up some locking to close a race that is extremely unlikely
to happen (famous last words). It can only happen when failed devices
are
being removed.
2 - Fixes a resync related problem for raid5 and raid6 arrays that have
lost more devices than they can cope with.
3 - Fixes a clumsy test that has very little real effect (just a printk).

These three can be considered stable.

4,5 Improve handling of 'safemode' which is where we mark the superblock
clean whenever there have been no writes for a little while, and mark it dirty before the next write.
6-9 Introduce "bitmap based intent logging" which allows a file to
store - as a bitmap - all block that may have been written recently. This allow resync to be optimised, which is more important as drives get larger.

These should not be considerred stable - certainly actually using the bitmap mode (which requires a 2.0 series version of mdadm) should be seen as 'experimental'.

mdadm-2.0-devel-1 was released today.

NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html