Skip to contentSkip to conversation

md: Introduction

PATCH In review NeilBrown 2 participants 7 messages Feb 2, 2006 – Feb 3, 2006 MD RAID

lore
Share & tools
lore

NeilBrown

author
#0 lore raw

PATCH 1/1 md: Introduction

Three patches for 2.6.lastest. All should go in 2.6.16. One won't apply against -rc1-git5 as it fixes a bug in a patch in -mm that hasn't quite got to -linus yes.

They are mostly little fixes. I've been doing some more testing, particularly creating a raid1 over 2 6TB arrays. This requires version-1 superblocks, so they have had a bit more testing too.

(No, I didn't try to resync the 6TB raid1. I created it --assume-clean and gave it a write-intent bitmap so a full resync would never be needed. If one side died, a recovery would take ages!!!! but for me it was only an experiment, not a serious configuration). (You will need mdadm-2.3 if you too want a 6TB raid1).

[PATCH 001 of 3] md: Handle overflow of mdu_array_info_t->size better. [PATCH 002 of 3] md: Assorted little md fixes:
[PATCH 003 of 3] md: Make sure rdev->size gets set for version-1 superblocks.

author
#1 lore raw

PATCH 1/1 md: Handle overflow of mdu_array_info_t->size better.

mdu_array_info_t->size is 'int', which isn't big enough for the size (in KB of each component in) some arrays.

So rather than a random overflow, set size to -1 when it cannot be set correctly.

To update aspect on an array, userspace will sometimes:
get_array_info
change one field
set_array_info

in this case, we don't want the '-1' in 'size' to change to size, or look like a size change at all. So test for that in update_array_info.

### Diffstat output

 ./drivers/md/md.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +4 -2
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -2943,6 +2943,8 @@ static int get_array_info(mddev_t * mdde
 	info.ctime         = mddev->ctime;
 	info.level         = mddev->level;
 	info.size          = mddev->size;
+	if (info.size != mddev->size) /* overflow */
+		info.size = -1;
 	info.nr_disks      = nr;
 	info.raid_disks    = mddev->raid_disks;
 	info.md_minor      = mddev->md_minor;
@@ -3524,7 +3526,7 @@ static int update_array_info(mddev_t *md
 		)
 		return -EINVAL;
 	/* Check there is only one change */
-	if (mddev->size != info->size) cnt++;
+	if (info->size >= 0 && mddev->size != info->size) cnt++;
 	if (mddev->raid_disks != info->raid_disks) cnt++;
 	if (mddev->layout != info->layout) cnt++;
 	if ((state ^ info->state) & (1<<MD_SB_BITMAP_PRESENT)) cnt++;
@@ -3541,7 +3543,7 @@ static int update_array_info(mddev_t *md
 		else
 			return mddev->pers->reconfig(mddev, info->layout, -1);
 	}
-	if (mddev->size != info->size)
+	if (info->size >= 0 && mddev->size != info->size)
 		rv = update_size(mddev, info->size);
 
 	if (mddev->raid_disks    != info->raid_disks)
author
#2 lore raw

PATCH 1/1 md: Assorted little md fixes:

In reply to NeilBrown

- version-1 superblock
+ The default_bitmap_offset is in sectors, not bytes.
+ the 'size' field in the superblock is in sectors, not KB
- raid0_run should return a negative number on error, not '1'
- raid10_read_balance should not return a valid 'disk' number if

     ->rdev turned out to be NULL

- kmem_cache_destroy doesn't like being passed a NULL.

### Diffstat output

 ./drivers/md/md.c        |    4 ++--
 ./drivers/md/raid0.c     |    2 +-
 ./drivers/md/raid10.c    |    2 ++
 ./drivers/md/raid5.c     |    3 ++-
 ./drivers/md/raid6main.c |    3 ++-
 5 files changed, 9 insertions(+), 5 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +2 -2
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -1082,7 +1082,7 @@ static int super_1_validate(mddev_t *mdd
 		mddev->size = le64_to_cpu(sb->size)/2;
 		mddev->events = le64_to_cpu(sb->events);
 		mddev->bitmap_offset = 0;
-		mddev->default_bitmap_offset = 1024;
+		mddev->default_bitmap_offset = 1024 >> 9;
 		
 		mddev->recovery_cp = le64_to_cpu(sb->resync_offset);
 		memcpy(mddev->uuid, sb->set_uuid, 16);
@@ -1163,7 +1163,7 @@ static void super_1_sync(mddev_t *mddev,
 	sb->cnt_corrected_read = atomic_read(&rdev->corrected_errors);
 
 	sb->raid_disks = cpu_to_le32(mddev->raid_disks);
-	sb->size = cpu_to_le64(mddev->size);
+	sb->size = cpu_to_le64(mddev->size<<1);
 
 	if (mddev->bitmap && mddev->bitmap_file == NULL) {
 		sb->bitmap_offset = cpu_to_le32((__u32)mddev->bitmap_offset);
drivers/md/raid0.c renamed from drivers/md/raid0.c~current~ +1 -1
diff ./drivers/md/raid0.c~current~ ./drivers/md/raid0.c
@@ -372,7 +372,7 @@ out_free_conf:
 	kfree(conf);
 	mddev->private = NULL;
 out:
-	return 1;
+	return -ENOMEM;
 }
 
 static int raid0_stop (mddev_t *mddev)
drivers/md/raid10.c renamed from drivers/md/raid10.c~current~ +2 -0
diff ./drivers/md/raid10.c~current~ ./drivers/md/raid10.c
@@ -565,6 +565,8 @@ rb_out:
 
 	if (disk >= 0 && (rdev=rcu_dereference(conf->mirrors[disk].rdev))!= NULL)
 		atomic_inc(&conf->mirrors[disk].rdev->nr_pending);
+	else
+		disk = -1;
 	rcu_read_unlock();
 
 	return disk;
drivers/md/raid5.c renamed from drivers/md/raid5.c~current~ +2 -1
diff ./drivers/md/raid5.c~current~ ./drivers/md/raid5.c
@@ -350,7 +350,8 @@ static void shrink_stripes(raid5_conf_t 
 	while (drop_one_stripe(conf))
 		;
 
-	kmem_cache_destroy(conf->slab_cache);
+	if (conf->slab_cache)
+		kmem_cache_destroy(conf->slab_cache);
 	conf->slab_cache = NULL;
 }
 
drivers/md/raid6main.c renamed from drivers/md/raid6main.c~current~ +2 -1
diff ./drivers/md/raid6main.c~current~ ./drivers/md/raid6main.c
@@ -366,7 +366,8 @@ static void shrink_stripes(raid6_conf_t 
 	while (drop_one_stripe(conf))
 		;
 
-	kmem_cache_destroy(conf->slab_cache);
+	if (conf->slab_cache)
+		kmem_cache_destroy(conf->slab_cache);
 	conf->slab_cache = NULL;
 }
 
author
#3 lore raw

PATCH 1/1 md: Make sure rdev->size gets set for version-1 superblocks.

In reply to NeilBrown

Sometimes it doesn't so make the code more like the version-0 code which works.

### Diffstat output

 ./drivers/md/md.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/md/md.c renamed from drivers/md/md.c~current~ +5 -3
diff ./drivers/md/md.c~current~ ./drivers/md/md.c
@@ -1025,7 +1025,7 @@ static int super_1_load(mdk_rdev_t *rdev
 		rdev-> sb_size = (rdev->sb_size | bmask)+1;
 
 	if (refdev == 0)
-		return 1;
+		ret = 1;
 	else {
 		__u64 ev1, ev2;
 		struct mdp_superblock_1 *refsb = 
@@ -1045,7 +1045,9 @@ static int super_1_load(mdk_rdev_t *rdev
 		ev2 = le64_to_cpu(refsb->events);
 
 		if (ev1 > ev2)
-			return 1;
+			ret = 1;
+		else
+			ret = 0;
 	}
 	if (minor_version) 
 		rdev->size = ((rdev->bdev->bd_inode->i_size>>9) - le64_to_cpu(sb->data_offset)) / 2;
@@ -1059,7 +1061,7 @@ static int super_1_load(mdk_rdev_t *rdev
 
 	if (le32_to_cpu(sb->size) > rdev->size*2)
 		return -EINVAL;
-	return 0;
+	return ret;
 }
 
 static int super_1_validate(mddev_t *mddev, mdk_rdev_t *rdev)
#4 lore raw
In reply to NeilBrown

Hello Neil ,

Quoted 14 lines · NeilBrown

Three patches for 2.6.lastest. All should go in 2.6.16. One won't apply against -rc1-git5 as it fixes a bug in a patch in -mm that hasn't quite got to -linus yes.

They are mostly little fixes. I've been doing some more testing, particularly creating a raid1 over 2 6TB arrays. This requires version-1 superblocks, so they have had a bit more testing too.

(No, I didn't try to resync the 6TB raid1. I created it --assume-clean and gave it a write-intent bitmap so a full resync would never be needed. If one side died, a recovery would take ages!!!! but for me it was only an experiment, not a serious configuration). (You will need mdadm-2.3 if you too want a 6TB raid1).

Quoted 3 lines

[PATCH 001 of 3] md: Handle overflow of mdu_array_info_t->size better. [PATCH 002 of 3] md: Assorted little md fixes:
[PATCH 003 of 3] md: Make sure rdev->size gets set for version-1 superblocks.

Would you be so kind and share the commands you ran in order to
build this array ? Please , Please !-) Tia , JimL

Signature · 6 lines
+------------------------------------------------------------------+
| James   W.   Laferriere | System    Techniques | Give me VMS     |
| Network        Engineer | 3542 Broken Yoke Dr. |  Give me Linux  |
| [email protected] | Billings , MT. 59105 |   only  on  AXP |
|  http://www.asteriskhelpdesk.com/cgi-bin/astlance/r.cgi?babydr   |
+------------------------------------------------------------------+
author
#5 lore raw
Quoted 20 lines · On Thursday February 2, [email protected] wrote:

Hello Neil ,

Quoted 17 lines · NeilBrown

Three patches for 2.6.lastest. All should go in 2.6.16. One won't apply against -rc1-git5 as it fixes a bug in a patch in -mm that hasn't quite got to -linus yes.

They are mostly little fixes. I've been doing some more testing, particularly creating a raid1 over 2 6TB arrays. This requires version-1 superblocks, so they have had a bit more testing too.

(No, I didn't try to resync the 6TB raid1. I created it --assume-clean and gave it a write-intent bitmap so a full resync would never be needed. If one side died, a recovery would take ages!!!! but for me it was only an experiment, not a serious configuration). (You will need mdadm-2.3 if you too want a 6TB raid1).

[PATCH 001 of 3] md: Handle overflow of mdu_array_info_t->size better. [PATCH 002 of 3] md: Assorted little md fixes:
[PATCH 003 of 3] md: Make sure rdev->size gets set for version-1 superblocks.

Review

Quoted 2 lines
(quoted diff) +0 -0
 	Would you be so kind and share the commands you ran in order to
 	build this array ?  Please ,  Please !-)   Tia ,  JimL

Well, I don't have 12TB of storage so:

# create some sparse 2G loopback devices
for i in 1 2 3 4 5 6
do
 dd if=/dev/full of=Big$i bs=1024k seek=2000000 count=1
 losetup /dev/loop$i Big$1
done
# combine them into 6G linear arrays
mdadm -C /dev/md4 -l linear -n 3 /dev/loop[123]
mdadm -C /dev/md5 -l linear -n 3 /dev/loop[456]
# now make the 6G mirrored pair.
mdadm -C /dev/md3 -l 1 -n2 --assume-clean --bitmap=internal /dev/md[45]

Providing you are using mdadm 2.3, this will automatically select version-1 superblocks and create the array marking it clean without any resync.

Ofcourse, as this particular array is based on sparse files, actually trying to store much data on it will quickly run you into trouble, but as a proof-of-implementation, it works OK.

Was that what you were after?

NeilBrown

#6 lore raw

Hello Neil ,

Quoted 42 lines · Neil Brown
Quoted 18 lines · On Thursday February 2, [email protected] wrote:
Quoted 17 lines · NeilBrown

Three patches for 2.6.lastest. All should go in 2.6.16. One won't apply against -rc1-git5 as it fixes a bug in a patch in -mm that hasn't quite got to -linus yes.

They are mostly little fixes. I've been doing some more testing, particularly creating a raid1 over 2 6TB arrays. This requires version-1 superblocks, so they have had a bit more testing too.

(No, I didn't try to resync the 6TB raid1. I created it --assume-clean and gave it a write-intent bitmap so a full resync would never be needed. If one side died, a recovery would take ages!!!! but for me it was only an experiment, not a serious configuration). (You will need mdadm-2.3 if you too want a 6TB raid1).

[PATCH 001 of 3] md: Handle overflow of mdu_array_info_t->size better. [PATCH 002 of 3] md: Assorted little md fixes:
[PATCH 003 of 3] md: Make sure rdev->size gets set for version-1 superblocks.

Review

Quoted 2 lines
(quoted diff) +0 -0
 	Would you be so kind and share the commands you ran in order to
 	build this array ?  Please ,  Please !-)   Tia ,  JimL

Well, I don't have 12TB of storage so:

# create some sparse 2G loopback devices
for i in 1 2 3 4 5 6
do
dd if=/dev/full of=Big$i bs=1024k seek=2000000 count=1
losetup /dev/loop$i Big$1
done
# combine them into 6G linear arrays
mdadm -C /dev/md4 -l linear -n 3 /dev/loop[123]
mdadm -C /dev/md5 -l linear -n 3 /dev/loop[456]
# now make the 6G mirrored pair.
mdadm -C /dev/md3 -l 1 -n2 --assume-clean --bitmap=internal /dev/md[45]

Providing you are using mdadm 2.3, this will automatically select version-1 superblocks and create the array marking it clean without any resync.

Quoted 5 lines

Ofcourse, as this particular array is based on sparse files, actually trying to store much data on it will quickly run you into trouble, but as a proof-of-implementation, it works OK.

Was that what you were after?

Yes , Exactly what I was looking for . The loop* stuff is not needed for me but the last to commands sure were .
Tnx agn , JimL

Signature · 6 lines
+------------------------------------------------------------------+
| James   W.   Laferriere | System    Techniques | Give me VMS     |
| Network        Engineer | 3542 Broken Yoke Dr. |  Give me Linux  |
| [email protected] | Billings , MT. 59105 |   only  on  AXP |
|  http://www.asteriskhelpdesk.com/cgi-bin/astlance/r.cgi?babydr   |
+------------------------------------------------------------------+