md: 'size_limit' attribute
PATCH 1/1 md: 'size_limit' attribute
Subject: md: 'size_limit' attribute
From: Dan Williams <[email protected]>
Provide a sysfs attribute to allow a raid array to be truncated to an arbitrary size. This functionality is needed to support imsm raid arrays where the metadata format expects that the size of some arrays is rounded down to the nearest 1MB boundary.
--- drivers/md/linear.c | 2 + drivers/md/md.c | 64 +++++++++++++++++++++++++++++++++++---------- drivers/md/raid0.c | 2 + drivers/md/raid1.c | 2 + drivers/md/raid5.c | 6 ++-- include/linux/raid/md.h | 1 + include/linux/raid/md_k.h | 1 + 7 files changed, 58 insertions(+), 20 deletions(-)
Also available here:
git://git.kernel.org/pub/scm/linux/kernel/git/djbw/md.git for-neil
drivers/md/linear.c +1 -1
@@ -284,7 +284,7 @@ static int linear_add(mddev_t *mddev, mdk_rdev_t *rdev) mddev->private = newconf; mddev->raid_disks++; mddev->array_sectors = newconf->array_sectors; -set_capacity (mddev->gendisk, mddev->array_sectors); +set_capacity (mddev->gendisk,md_array_sectors (mddev));return 0 ; }
drivers/md/md.c +50 -14
112 lines not on this page · raw message
drivers/md/raid0.c +1 -1
@@ -296,7 +296,7 @@ static int raid0_run (mddev_t *mddev) mddev->array_sectors += rdev->size *2 ;printk (KERN_INFO "raid0 : md_size is %llu sectors.\n" , - (unsigned long long )mddev->array_sectors); + (unsigned long long )md_array_sectors (mddev));printk (KERN_INFO "raid0 : conf->spacing is %llu sectors.\n" , (unsigned long long )conf->spacing); {
drivers/md/raid1.c +1 -1
@@ -2110,7 +2110,7 @@ static int raid1_resize(mddev_t *mddev, sector_t sectors)* worth it. */ mddev->array_sectors = sectors; -set_capacity (mddev->gendisk, mddev->array_sectors); +set_capacity (mddev->gendisk,md_array_sectors (mddev)); mddev->changed =1 ;if (mddev->array_sectors /2 > mddev->size && mddev->recovery_cp == MaxSector) {
drivers/md/raid5.c +3 -3
@@ -4477,7 +4477,7 @@ static int raid5_resize(mddev_t *mddev, sector_t sectors) sectors &= ~((sector_t )mddev->chunk_size/512 -1 ); mddev->array_sectors = sectors * (mddev->raid_disks - conf->max_degraded); -set_capacity (mddev->gendisk, mddev->array_sectors); +set_capacity (mddev->gendisk,md_array_sectors (mddev)); mddev->changed =1 ;if (sectors/2 > mddev->size && mddev->recovery_cp == MaxSector) { mddev->recovery_cp = mddev->size <<1 ;
@@ -4614,14 +4614,14 @@ static void end_reshape(raid5_conf_t *conf)if (!test_bit (MD_RECOVERY_INTR , &conf->mddev->recovery)) { conf->mddev->array_sectors =2 * conf->mddev->size * (conf->raid_disks - conf->max_degraded); -set_capacity (conf->mddev->gendisk, conf->mddev->array_sectors); +set_capacity (conf->mddev->gendisk,md_array_sectors (conf->mddev)); conf->mddev->changed =1 ; bdev =bdget_disk (conf->mddev->gendisk,0 );if (bdev) {mutex_lock (&bdev->bd_inode->i_mutex);i_size_write (bdev->bd_inode, - (loff_t )conf->mddev->array_sectors <<9 ); + (loff_t )md_array_sectors (conf->mddev) <<9 );mutex_unlock (&bdev->bd_inode->i_mutex);bdput (bdev); }
include/linux/raid/md.h +1 -0
@@ -75,6 +75,7 @@ extern void md_do_sync(mddev_t *mddev);extern void md_new_event (mddev_t *mddev);extern int md_allow_write (mddev_t *mddev);extern void md_wait_for_blocked_rdev (mdk_rdev_t *rdev,mddev_t *mddev); +extern sector_t md_array_sectors (mddev_t *mddev);#endif /* CONFIG_MD */ #endif
include/linux/raid/md_k.h +1 -0
@@ -157,6 +157,7 @@ struct mddev_sint max_disks;sector_t size;/* used size of component devices */ sector_t array_sectors;/* exported array size */ +sector_t size_limit;/* limit for array_sectors */ __u64 events;char uuid[16 ];
Quoted 7 lines · On Friday February 13, [email protected] wrote:
Subject: md: '
size_limit' attribute
From: Dan Williams <[email protected]>Provide a sysfs attribute to allow a raid array to be truncated to an arbitrary size. This functionality is needed to support imsm raid arrays where the metadata format expects that the size of some arrays is rounded down to the nearest 1MB boundary.
Well it's not April 1st, so I assume you are serious.
It really truncates the array, not the individual drives? So you could have e.g. a raid0 in which only some of the last stripe was used?
Can you give me a concrete example of an array where this will make a required difference? I just want to be sure I understand.
I guess you couldn't just add an 'array_size' attribute which gave direct access to mddev->array_size because that gets set when the array is started, and we want to be able to impose the limit before starting the array....
How about a semantic where starting the array will only modify ->array_size if it's value is zero of if it would reduce the value.
How might this interact with array resizing? You add a drive to an array, reshape it, and then it doesn't get any bigger until thesize_limit is updated? I guess that could work but it might be
confusing.... though presumably mdadm/mdmon would know to look after all the details.
What would you think of renaming the attribute to 'array_size' with the semantic of "once user-space sets it, the kernel will never change it" ??
NeilBrown
Quoted 9 lines · Neil Brown
Quoted 7 lines · On Friday February 13, [email protected] wrote:
Subject: md: '
size_limit' attribute
From: Dan Williams <[email protected]>Provide a sysfs attribute to allow a raid array to be truncated to an arbitrary size. This functionality is needed to support imsm raid arrays where the metadata format expects that the size of some arrays is rounded down to the nearest 1MB boundary.
Quoted 5 lines
Well it's not April 1st, so I assume you are serious.
It really truncates the array, not the individual drives? So you could have e.g. a raid0 in which only some of the last stripe was used?
Unfortunately yes. It will even record the "correct" value for num_data_stripes based on the per device size, but the actual array_size recorded in the metadata is this weird rounded down value.
Quoted 2 lines
Can you give me a concrete example of an array where this will make a required difference? I just want to be sure I understand.
1/ Array created in orom
2/ User assembles, partitions, and formats the array in Linux
3/ User reboots into Windows and sees data missing off the end of the volume
So, it is purely an interoperability issue with other imsm drivers.
Quoted 16 lines
I guess you couldn't just add an '
array_size' attribute which gave direct access tomddev->array_sizebecause that gets set when the array is started, and we want to be able to impose the limit before starting the array....How about a semantic where starting the array will only modify ->array_size if it's value is zero of if it would reduce the value.
How might this interact with array resizing? You add a drive to an array, reshape it, and then it doesn't get any bigger until the
size_limitis updated? I guess that could work but it might be
confusing.... though presumably mdadm/mdmon would know to look after all the details.
Quoted 3 lines
What would you think of renaming the attribute to '
array_size' with the semantic of "once user-space sets it, the kernel will never change it" ??
To be be sure I understand, the differences from the current patch:
1/ The kernel will set ->array_size at array start time unless userspace has modified it from zero. If userspace has modified it we should probably refuse to run if ->array_size is > ->array_sectors? Upon successfully starting the array we record that userspace owns ->array_size.
2/ At reshape time the kernel sets ->array_size = ->array_sectors unless userspace owns ->array_size at which point we don't touch ->array_size. I assume we must then block attempts to reshape the array to a smaller than ->array_size size when userspace ->array_size is in effect?
3/ While an array is active userspace can set ->array_size only if the kernel has recorded that ->array_size is under userspace control, and then it can only set a value that is <= ->array_sectors?
Thanks,
Dan
Quoted 9 lines · On Tuesday February 17, [email protected] wrote:
Quoted 4 lines · Neil Brown
On Friday February 13, [email protected] wrote:
Can you give me a concrete example of an array where this will make a required difference? I just want to be sure I understand.
1/ Array created in orom
2/ User assembles, partitions, and formats the array in Linux
3/ User reboots into Windows and sees data missing off the end of the volume
I was hoping for raid level devices size, chunk size, array size etc, but it probably isn't important.
Quoted 11 lines
Quoted 4 lines
What would you think of renaming the attribute to '
array_size' with the semantic of "once user-space sets it, the kernel will never change it" ??To be be sure I understand, the differences from the current patch:
1/ The kernel will set ->array_size at array start time unless userspace has modified it from zero. If userspace has modified it we should probably refuse to run if ->array_size is > ->array_sectors? Upon successfully starting the array we record that userspace owns ->array_size.
I wasn't thinking of having two 'size' variables. Just the one "array_sectors" with a "externally modified" flag.
Yes, refuse to run if the specified size is larger that the array provides.
We record that user-space owns the size whenever they set it.
Quoted 5 lines
2/ At reshape time the kernel sets ->array_size = ->array_sectors unless userspace owns ->array_size at which point we don't touch ->array_size. I assume we must then block attempts to reshape the array to a smaller than ->array_size size when userspace ->array_size is in effect?
Yes, I think that makes sense.
Quoted 4 lines
3/ While an array is active userspace can set ->array_size only if the kernel has recorded that ->array_size is under userspace control, and then it can only set a value that is <= ->array_sectors?
I'm not sure about that restriction. I want to be able to 'take over' at any time. I'd also like to be able to 'let go' as well.
You see I have come up with another use for this.
One of the reasons I have been reticent to add support for shaping a RAID5 to have fewer devices is that it irreversibly destroys data. The moment you trigger the reshape, it will start over-writing the data at the end of the array.
But if I could explicitly set the size of the array to be smaller, or conversely, set the 'used-space' per device to be more without making the array larger, then it wouldn't be the shape that destroyed data.
If you are replacing the drives with fewer larger drives, then I never want the array to appear larger.
If you really are shrinking the array, then I want that to be an operation that is reversible.
If I could explicitly set the array size, then I would require that you first make the array smaller. If that causes your filesystem to start spewing errors, you can set it larger again and minimal harm done.
So: new attribute: ../md/size (or do we want array_size). Measured in ....
sectors would be nice.
K would be consistent with component_size
I suspect we should use K.
Write a number, and (if it isn't too big) the array size is pinned to that number.
Write 'default' and the array size is allowed to float to whatever is the available space.
Read always gives the current size. It gives no indication on whether the size is pinned or not. (Does it need to?)
I could probably live without the 'default' setting for now. Any version of 'mdadm' which ever sets /size would need to make sure to set /size for any --grow which made a difference.
It might be good to allow writing 'max' to get the max size??
Thanks,
NeilBrown