Skip to contentSkip to conversation

md: More block -> sector conversions.

PATCH In review Andre Noll 3 participants 10 messages Jan 26, 2009 – Feb 1, 2009 MD RAID

lore
Share & tools
lore

Andre Noll

author
#0 lore raw

cover md: More block -> sector conversions.

These two patches against Neil's for-next tree make the internal representaions of mddev->size and rdev->size sector-based. checkpatch.pl is happy with both patches, and the patched kernel passes all tests of the mdadm test suite on my i386 UP box.

Please review.

BTW: This completes the block -> sector conversion of the md code base because all remaining block-based quantities are either user-visible (via sysfs) or describe on-disk formats (version 0.90 superblocks) and hence can not be changed without breaking user space.

Andre

author
#1 lore raw

PATCH 1/2 md: Make mddev->size sector-based.

This patch renames the "size" field of struct mddev_s to "dev_sectors" and stores the number of 512-byte sectors instead of the number of 1K-blocks in it.

All users of that field, including raid levels 1,4-6,10, are adjusted accordingly. This simplifies the code a bit because it allows to get rid of a couple of divisions/multiplications by two.

In order to make checkpatch happy, some minor coding style issues have also been addressed. In particular, size_store() now uses strict_strtoull() instead of simple_strtoull().

---
 drivers/md/bitmap.c       |    2 +-
 drivers/md/faulty.c       |    2 +-
 drivers/md/md.c           |   97 ++++++++++++++++++++++++--------------------
 drivers/md/multipath.c    |    2 +-
 drivers/md/raid1.c        |   10 ++--
 drivers/md/raid10.c       |    6 +-
 drivers/md/raid5.c        |   24 ++++++------
 include/linux/raid/md_k.h |    3 +-
 8 files changed, 78 insertions(+), 68 deletions(-)
drivers/md/bitmap.c +1 -1
index 7199437..56a941a 100644
@@ -297,7 +297,7 @@ static int write_sb_page(struct bitmap *bitmap, struct page *page, int wait)
 				    + size/512 > 0)
 					/* bitmap runs in to metadata */
 					goto bad_alignment;
-				if (rdev->data_offset + mddev->size*2
+				if (rdev->data_offset + mddev->dev_sectors
 				    > rdev->sb_start + bitmap->offset)
 					/* data runs in to bitmap */
 					goto bad_alignment;
drivers/md/faulty.c +1 -1
index 86d9adf..b3b0437 100644
@@ -298,7 +298,7 @@ static int run(mddev_t *mddev)
 	list_for_each_entry(rdev, &mddev->disks, same_set)
 		conf->rdev = rdev;
 
-	mddev->array_sectors = mddev->size * 2;
+	mddev->array_sectors = mddev->dev_sectors;
 	mddev->private = conf;
 
 	reconfig(mddev, mddev->layout, -1);
drivers/md/md.c +53 -44

242 lines not on this page · raw message

drivers/md/multipath.c +1 -1
index f6d08f2..a21084b 100644
@@ -498,7 +498,7 @@ static int multipath_run (mddev_t *mddev)
 	/*
 	 * Ok, everything is just fine now
 	 */
-	mddev->array_sectors = mddev->size * 2;
+	mddev->array_sectors = mddev->dev_sectors;
 
 	mddev->queue->unplug_fn = multipath_unplug;
 	mddev->queue->backing_dev_info.congested_fn = multipath_congested;
drivers/md/raid1.c +5 -5
index 7b4f5f7..7b73bc9 100644
@@ -1721,7 +1721,7 @@ static sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, i
 			return 0;
 	}
 
-	max_sector = mddev->size << 1;
+	max_sector = mddev->dev_sectors;
 	if (sector_nr >= max_sector) {
 		/* If we aborted, we need to abort the
 		 * sync on the 'current' bitmap chunk (there will
@@ -2046,7 +2046,7 @@ static int run(mddev_t *mddev)
 	/*
 	 * Ok, everything is just fine now
 	 */
-	mddev->array_sectors = mddev->size * 2;
+	mddev->array_sectors = mddev->dev_sectors;
 
 	mddev->queue->unplug_fn = raid1_unplug;
 	mddev->queue->backing_dev_info.congested_fn = raid1_congested;
@@ -2111,12 +2111,12 @@ static int raid1_resize(mddev_t *mddev, sector_t sectors)
 	mddev->array_sectors = sectors;
 	set_capacity(mddev->gendisk, mddev->array_sectors);
 	mddev->changed = 1;
-	if (mddev->array_sectors / 2 > mddev->size &&
+	if (mddev->array_sectors > mddev->dev_sectors &&
 	    mddev->recovery_cp == MaxSector) {
-		mddev->recovery_cp = mddev->size << 1;
+		mddev->recovery_cp = mddev->dev_sectors;
 		set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
 	}
-	mddev->size = mddev->array_sectors / 2;
+	mddev->dev_sectors = mddev->array_sectors;
 	mddev->resync_max_sectors = sectors;
 	return 0;
 }
drivers/md/raid10.c +3 -3
index 6736d6d..0484c7e 100644
@@ -1694,7 +1694,7 @@ static sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *skipped, i
 			return 0;
 
  skipped:
-	max_sector = mddev->size << 1;
+	max_sector = mddev->dev_sectors;
 	if (test_bit(MD_RECOVERY_SYNC, &mddev->recovery))
 		max_sector = mddev->resync_max_sectors;
 	if (sector_nr >= max_sector) {
@@ -2075,7 +2075,7 @@ static int run(mddev_t *mddev)
 	conf->far_offset = fo;
 	conf->chunk_mask = (sector_t)(mddev->chunk_size>>9)-1;
 	conf->chunk_shift = ffz(~mddev->chunk_size) - 9;
-	size = mddev->size >> (conf->chunk_shift-1);
+	size = mddev->dev_sectors >> conf->chunk_shift;
 	sector_div(size, fc);
 	size = size * conf->raid_disks;
 	sector_div(size, nc);
@@ -2088,7 +2088,7 @@ static int run(mddev_t *mddev)
 	 */
 	stride += conf->raid_disks - 1;
 	sector_div(stride, conf->raid_disks);
-	mddev->size = stride  << (conf->chunk_shift-1);
+	mddev->dev_sectors = stride << conf->chunk_shift;
 
 	if (fo)
 		stride = 1;
drivers/md/raid5.c +12 -12
index a5ba080..99c5261 100644
@@ -3626,8 +3626,8 @@ static sector_t reshape_request(mddev_t *mddev, sector_t sector_nr, int *skipped
 				     *(new_data_disks) -1,
 				     raid_disks, data_disks,
 				     &dd_idx, &pd_idx, conf);
-	if (last_sector >= (mddev->size<<1))
-		last_sector = (mddev->size<<1)-1;
+	if (last_sector >= mddev->dev_sectors)
+		last_sector = mddev->dev_sectors - 1;
 	while (first_sector <= last_sector) {
 		pd_idx = stripe_to_pdidx(first_sector, conf,
 					 conf->previous_raid_disks);
@@ -3667,7 +3667,7 @@ static inline sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *ski
 	struct stripe_head *sh;
 	int pd_idx;
 	int raid_disks = conf->raid_disks;
-	sector_t max_sector = mddev->size << 1;
+	sector_t max_sector = mddev->dev_sectors;
 	int sync_blocks;
 	int still_degraded = 0;
 	int i;
@@ -3705,7 +3705,7 @@ static inline sector_t sync_request(mddev_t *mddev, sector_t sector_nr, int *ski
 	 */
 	if (mddev->degraded >= conf->max_degraded &&
 	    test_bit(MD_RECOVERY_SYNC, &mddev->recovery)) {
-		sector_t rv = (mddev->size << 1) - sector_nr;
+		sector_t rv = mddev->dev_sectors - sector_nr;
 		*skipped = 1;
 		return rv;
 	}
@@ -4143,8 +4143,8 @@ static int run(mddev_t *mddev)
 	conf->expand_progress = mddev->reshape_position;
 
 	/* device size must be a multiple of chunk size */
-	mddev->size &= ~(mddev->chunk_size/1024 -1);
-	mddev->resync_max_sectors = mddev->size << 1;
+	mddev->dev_sectors &= ~(mddev->chunk_size / 512 - 1);
+	mddev->resync_max_sectors = mddev->dev_sectors;
 
 	if (conf->level == 6 && conf->raid_disks < 4) {
 		printk(KERN_ERR "raid6: not enough configured devices for %s (%d, minimum 4)\n",
@@ -4251,8 +4251,8 @@ static int run(mddev_t *mddev)
 	mddev->queue->backing_dev_info.congested_data = mddev;
 	mddev->queue->backing_dev_info.congested_fn = raid5_congested;
 
-	mddev->array_sectors = 2 * mddev->size * (conf->previous_raid_disks -
-					    conf->max_degraded);
+	mddev->array_sectors = mddev->dev_sectors *
+		(conf->previous_raid_disks - conf->max_degraded);
 
 	blk_queue_merge_bvec(mddev->queue, raid5_mergeable_bvec);
 
@@ -4479,11 +4479,11 @@ static int raid5_resize(mddev_t *mddev, sector_t sectors)
 					  - conf->max_degraded);
 	set_capacity(mddev->gendisk, mddev->array_sectors);
 	mddev->changed = 1;
-	if (sectors/2  > mddev->size && mddev->recovery_cp == MaxSector) {
-		mddev->recovery_cp = mddev->size << 1;
+	if (sectors > mddev->dev_sectors && mddev->recovery_cp == MaxSector) {
+		mddev->recovery_cp = mddev->dev_sectors;
 		set_bit(MD_RECOVERY_NEEDED, &mddev->recovery);
 	}
-	mddev->size = sectors /2;
+	mddev->dev_sectors = sectors;
 	mddev->resync_max_sectors = sectors;
 	return 0;
 }
@@ -4612,7 +4612,7 @@ static void end_reshape(raid5_conf_t *conf)
 	struct block_device *bdev;
 
 	if (!test_bit(MD_RECOVERY_INTR, &conf->mddev->recovery)) {
-		conf->mddev->array_sectors = 2 * conf->mddev->size *
+		conf->mddev->array_sectors = conf->mddev->dev_sectors *
 			(conf->raid_disks - conf->max_degraded);
 		set_capacity(conf->mddev->gendisk, conf->mddev->array_sectors);
 		conf->mddev->changed = 1;
include/linux/raid/md_k.h +2 -1
index 9743e4d..ed2f24b 100644
@@ -155,7 +155,8 @@ struct mddev_s
 	char				clevel[16];
 	int				raid_disks;
 	int				max_disks;
-	sector_t			size; /* used size of component devices */
+	sector_t			dev_sectors; 	/* used size of
+							 * component devices */
 	sector_t			array_sectors; /* exported array size */
 	__u64				events;
 
Signature · 1 line
1.5.3.8
author
#2 lore raw

PATCH 2/2 md: Represent raid device size in sectors.

In reply to Andre Noll

This patch renames the "size" field of struct mdk_rdev_s to "num_sectors" and changes this field to store sectors instead of blocks.

All users of this field, linear.c, raid0.c and md.c, are fixed up accordingly which gets rid of many multiplications and divisions.

---
 drivers/md/linear.c       |    4 +-
 drivers/md/md.c           |   96 ++++++++++++++++++++++----------------------
 drivers/md/raid0.c        |   41 +++++++++----------
 include/linux/raid/md_k.h |    2 +-
 4 files changed, 71 insertions(+), 72 deletions(-)
drivers/md/linear.c +2 -2
index 09658b2..83110f8 100644
@@ -135,8 +135,8 @@ static linear_conf_t *linear_conf(mddev_t *mddev, int raid_disks)
 		    mddev->queue->max_sectors > (PAGE_SIZE>>9))
 			blk_queue_max_sectors(mddev->queue, PAGE_SIZE>>9);
 
-		disk->num_sectors = rdev->size * 2;
-		conf->array_sectors += rdev->size * 2;
+		disk->num_sectors = rdev->sectors;
+		conf->array_sectors += rdev->sectors;
 
 		cnt++;
 	}
drivers/md/md.c +48 -48

239 lines not on this page · raw message

drivers/md/raid0.c +20 -21

89 lines not on this page · raw message

include/linux/raid/md_k.h +1 -1
index ed2f24b..e0defd6 100644
@@ -49,7 +49,7 @@ struct mdk_rdev_s
 {
 	struct list_head same_set;	/* RAID devices within the same set */
 
-	sector_t size;			/* Device size (in blocks) */
+	sector_t sectors;		/* Device size (in 512bytes sectors) */
 	mddev_t *mddev;			/* RAID array if running */
 	long last_events;		/* IO event timestamp */
 
Signature · 1 line
1.5.3.8
#3 lore raw
In reply to Andre Noll
Quoted 5 lines · Andre Noll

These two patches against Neil's for-next tree make the internal representaions of mddev->size and rdev->size sector-based. checkpatch.pl is happy with both patches, and the patched kernel passes all tests of the mdadm test suite on my i386 UP box.

Quoted 9 lines

Please review.

BTW: This completes the block -> sector conversion of the md code base because all remaining block-based quantities are either user-visible (via sysfs) or describe on-disk formats (version 0.90 superblocks) and hence can not be changed without breaking user space.

Andre

Hi Andre,

Is it possible that the following issue is related to your earlier
patches: http://marc.info/?l=linux-raid&m=123202024116493&w=2

Thanks

author
#4 lore raw
Quoted 2 lines · Peter Rabbitson

Is it possible that the following issue is related to your earlier
patches: http://marc.info/?l=linux-raid&m=123202024116493&w=2

I don't think this is possible because you are using Linux 2.6.24.7 which does not contain any of the block -> sector conversion changes.

Moreover, those changes should be invisible to userspace. In particular, mdadm does not need to know whether the kernel uses blocks or sectors for its internal representations. In fact, no changes to mdadm were made wrt. the block -> sector conversions.

Can you reproduce the incorrect output for the used device size with the recent master branch of the mdadm git tree?

Thanks
Andre

Signature · 1 line
The only person who always got his work done by Friday was Robinson Crusoe
#5 lore raw
Quoted 8 lines · Andre Noll
Quoted 2 lines · Peter Rabbitson

Is it possible that the following issue is related to your earlier
patches: http://marc.info/?l=linux-raid&m=123202024116493&w=2

I don't think this is possible because you are using Linux 2.6.24.7 which does not contain any of the block -> sector conversion changes.

Quoted 8 lines

Moreover, those changes should be invisible to userspace. In particular, mdadm does not need to know whether the kernel uses blocks or sectors for its internal representations. In fact, no changes to mdadm were made wrt. the block -> sector conversions.

Can you reproduce the incorrect output for the used device size with the recent master branch of the mdadm git tree?

Hi Andre,

Yes I can:

root@Thesaurus:/home/rabbit/src/git/mdadm# ./mdadm --version
mdadm - v2.6.8 - 28th November 2008
root@Thesaurus:/home/rabbit/src/git/mdadm# ./mdadm -E /dev/sda1
/dev/sda1:
          Magic : a92b4efc
        Version : 1.1
    Feature Map : 0x0
     Array UUID : 6b11b1ba:78985745:b320fc1a:1db68bcf
           Name : Thesaurus:Crypta  (local to host Thesaurus)
  Creation Time : Sat Mar  8 16:33:41 2008
     Raid Level : raid5
   Raid Devices : 4

Avail Dev Size : 578130828 (275.67 GiB 296.00 GB)

     Array Size : 1730162688 (825.01 GiB 885.84 GB)
  Used Dev Size : 576720896 (275.00 GiB 295.28 GB)
    Data Offset : 264 sectors
   Super Offset : 0 sectors
          State : active
    Device UUID : 85e0e62b:56129e5f:b0d82e78:2ace7d37

    Update Time : Mon Jan 26 18:24:19 2009
       Checksum : 61b214ff - correct
         Events : 772406

         Layout : left-symmetric
     Chunk Size : 2048K

    Array Slot : 5 (failed, failed, 2, 3, 0, 1)

Array State : uUuu 2 failed

root@Thesaurus:/home/rabbit/src/git/mdadm# ./mdadm -D /dev/md5
/dev/md5:
        Version : 1.01
  Creation Time : Sat Mar  8 16:33:41 2008
     Raid Level : raid5
     Array Size : 865081344 (825.01 GiB 885.84 GB)
  Used Dev Size : 576720896 (550.00 GiB 590.56 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 5
    Persistence : Superblock is persistent

Update Time : Mon Jan 26 18:24:34 2009
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 2048K

Name : Thesaurus:Crypta (local to host Thesaurus) UUID : 6b11b1ba:78985745:b320fc1a:1db68bcf
Events : 772406

Number Major Minor RaidDevice State
4 8 67 0 active sync /dev/sde3
5 8 1 1 active sync /dev/sda1
2 8 19 2 active sync /dev/sdb3
3 8 35 3 active sync /dev/sdc3

author
#6 lore raw
Quoted 7 lines · Peter Rabbitson
Quoted 3 lines

Can you reproduce the incorrect output for the used device size with the recent master branch of the mdadm git tree?

Hi Andre,

Yes I can:

[snip]

Quoted 1 line

Used Dev Size : 576720896 (550.00 GiB 590.56 GB)

This is twice as much as it should be due to a bug in mdadm which bites only for version1 superblocks. The patch below should fix it. However, this might not be the most elegant solution because the real bug is IMHO that get_component_size() multiplies the value from sysfs (which is always in 1K units) by two, so it returns 2K units which looks a bit weird.

Since there are more callers of get_component_size() I have to take a deeper look to come up with a more reasonable patch.

Have fun
Andre

Detail.c +1 -1
index 3cee66f..cef1abb 100644
@@ -195,7 +195,7 @@ int Detail(char *dev, int brief, int export, int test, char *homehost)
 		if (array.level >= 1) {
 			if (array.major_version != 0 &&
 			    (larray_size >= 0xFFFFFFFFULL|| array.size == 0)) {
-				unsigned long long dsize = get_component_size(fd);
+				unsigned long long dsize = get_component_size(fd) / 2; /* KB */
 				if (dsize > 0)
 					printf("  Used Dev Size : %llu%s\n",
 					       dsize,
Signature · 1 line
The only person who always got his work done by Friday was Robinson Crusoe
author
#7 lore raw

Re: [PATCH] mdadm: Fix the used device size in mdadm -D output.

Quoted 6 lines · Andre Noll

This is twice as much as it should be due to a bug in mdadm which bites only for version1 superblocks. The patch below should fix it. However, this might not be the most elegant solution because the real bug is IMHO that get_component_size() multiplies the value from sysfs (which is always in 1K units) by two, so it returns 2K units which looks a bit weird.

This was of course a braino: Multiplying a value in 1K units by two yields 512byte units rather than 2K units, which is exactly what the comment to get_component_size() says ;)

That being said, I think my patch is correct and dividing the result by two in Detail() seems the best way to deal with the situation. So here's the patch again, this time with proper log message.

Andre

commit 2e9fd78bd09bf332ac86f2d288a0e7c3c1c3df4f

Author: Andre Noll <[email protected]>
Date:   Tue Jan 27 08:49:15 2009 +0100

Fix the used device size in mdadm -D output.

As get_component_size() returns the number of used sectors of a device while mdadm is supposed to print this size in kilobytes, we have to divide the return value of get_component_size() by two.

Detail.c +1 -1
index 3cee66f..cef1abb 100644
@@ -195,7 +195,7 @@ int Detail(char *dev, int brief, int export, int test, char *homehost)
 		if (array.level >= 1) {
 			if (array.major_version != 0 &&
 			    (larray_size >= 0xFFFFFFFFULL|| array.size == 0)) {
-				unsigned long long dsize = get_component_size(fd);
+				unsigned long long dsize = get_component_size(fd) / 2; /* KB */
 				if (dsize > 0)
 					printf("  Used Dev Size : %llu%s\n",
 					       dsize,
Signature · 1 line
The only person who always got his work done by Friday was Robinson Crusoe
#8 lore raw

Re: [PATCH] mdadm: Fix the used device size in mdadm -D output.

Quoted 8 lines · On Tuesday January 27, [email protected] wrote:
Quoted 6 lines · Andre Noll

This is twice as much as it should be due to a bug in mdadm which bites only for version1 superblocks. The patch below should fix it. However, this might not be the most elegant solution because the real bug is IMHO that get_component_size() multiplies the value from sysfs (which is always in 1K units) by two, so it returns 2K units which looks a bit weird.

Quoted 7 lines

This was of course a braino: Multiplying a value in 1K units by two yields 512byte units rather than 2K units, which is exactly what the comment to get_component_size() says ;)

That being said, I think my patch is correct and dividing the result by two in Detail() seems the best way to deal with the situation. So here's the patch again, this time with proper log message.

Thanks Andre.
Your patch is good... but I really like to use "sectors" as often as possible.
So I've changed it to leave 'dsize' as sectors, but device it by 2, or shift by 9, as appropriate.

Thanks,
NeilBrown

Quoted 30 lines

Andre

commit 2e9fd78bd09bf332ac86f2d288a0e7c3c1c3df4f

Author: Andre Noll <[email protected]>
Date:   Tue Jan 27 08:49:15 2009 +0100

Fix the used device size in mdadm -D output.

As get_component_size() returns the number of used sectors of a device while mdadm is supposed to print this size in kilobytes, we have to divide the return value of get_component_size() by two.

Detail.c +1 -1
index 3cee66f..cef1abb 100644
@@ -195,7 +195,7 @@ int Detail(char *dev, int brief, int export, int test, char *homehost)
 		if (array.level >= 1) {
 			if (array.major_version != 0 &&
 			    (larray_size >= 0xFFFFFFFFULL|| array.size == 0)) {
-				unsigned long long dsize = get_component_size(fd);
+				unsigned long long dsize = get_component_size(fd) / 2; /* KB */
 				if (dsize > 0)
 					printf("  Used Dev Size : %llu%s\n",
 					       dsize,
Signature · 1 line
The only person who always got his work done by Friday was Robinson Crusoe
#9 lore raw
In reply to Andre Noll
Quoted 5 lines · On Monday January 26, [email protected] wrote:

These two patches against Neil's for-next tree make the internal representaions of mddev->size and rdev->size sector-based. checkpatch.pl is happy with both patches, and the patched kernel passes all tests of the mdadm test suite on my i386 UP box.

Quoted 7 lines

Please review.

BTW: This completes the block -> sector conversion of the md code base because all remaining block-based quantities are either user-visible (via sysfs) or describe on-disk formats (version 0.90 superblocks) and hence can not be changed without breaking user space.

Thanks Andre.
These look fine.
The only problem I could find was that in the first patch, the change log says "num_sectors" and the code says "sectors" :-)

I'll queue these for the next release.

Thanks,
NeilBrown