Skip to contentSkip to conversation

Subject: [PATCH 006/009]: raid1: chunk size check in run

PATCH raz ben yehuda 2 participants 6 messages May 19, 2009 – May 21, 2009 MD RAID

lore
Share & tools
lore
#0 lore raw

have raid1 check chunk size in run method instead of in md

 raid1.c |    9 +++++++++
 1 file changed, 9 insertions(+)
Signed-off-by [email protected]

---

drivers/md/raid1.c +9 -0
index 2c8d9e6..8609506 100644
@@ -1942,6 +1942,15 @@ static int run(mddev_t *mddev)
 		       mdname(mddev), mddev->level);
 		goto out;
 	}
+	/*
+	 * chunk-size has to be a power of 2
+	 */
+	if ((1 << ffz(~mddev->chunk_size)) != mddev->chunk_size) {
+		printk(KERN_ERR "%s chunk_size of %d not valid\n",
+			mdname(mddev),
+			mddev->chunk_size);
+		return -EINVAL;
+	}
 	if (mddev->reshape_position != MaxSector) {
 		printk("raid1: %s: reshape_position set but not supported\n",
 		       mdname(mddev));
#1 lore raw
Quoted 1 line · On Tuesday May 19, [email protected] wrote:

have raid1 check chunk size in run method instead of in md

I dropped this patch. chunk_size is not meaningful for raid1.

NeilBrown

Quoted 30 lines
 raid1.c |    9 +++++++++
 1 file changed, 9 insertions(+)
Signed-off-by [email protected]

---

drivers/md/raid1.c +9 -1
index 2c8d9e6..8609506 100644
@@ -1942,6 +1942,15 @@ static int run(mddev_t *mddev)
 		       mdname(mddev), mddev->level);
 		goto out;
 	}
+	/*
+	 * chunk-size has to be a power of 2
+	 */
+	if ((1 << ffz(~mddev->chunk_size)) != mddev->chunk_size) {
+		printk(KERN_ERR "%s chunk_size of %d not valid\n",
+			mdname(mddev),
+			mddev->chunk_size);
+		return -EINVAL;
+	}
 	if (mddev->reshape_position != MaxSector) {
 		printk("raid1: %s: reshape_position set but not supported\n",
 		       mdname(mddev));
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw

Neil
First I thank you for your effort. Now I can work in full steam on the reshape on top of the new raid0 code. Currently this is what I have in mind.If you have any design suggestions I would be happy to hear before the coding.

I added : raid0_add_hot that:
1. checks if the new disk size if smaller than the raid chunk size. if
so , reject.
2. check if new the disk size max_hw_sectors is smaller than the
raid's. if so generate a warning but do not reject.
3. adds a disk to raid0 disk list. and turns off its in_sync bit.

I will add raid0_check_reshape
This procedure prepares the raid for the reshape process.
1. Creates a temporary mddev with the same disks as the raid's and with
the new disks. This raid acts as a mere mappings so i will be able to map sectors to the new target raid in the reshape process. This means i have to work in create_strip_zones raid0_run ( separate patch ).
2. Sets the target raid transfer size.
3. Create an allocation scheme for reshape bio allocation. i reshape in
chunk size.
4. create raid0_reshape thread for writes.
5. wake up raid0_sync thread.

I will add raid0_sync: raid0_sync acts as the reshape read size process.

1. Allocates a read bio.
2. Map_bio target with find_zone and map_sector, both map_sector and
find_zone are using the old raid mappings.
3. Deactivate the raid.
3. Lock and wait for the raid to be emptied from any previous IOs.
4. Generate a read request.
5. Release the lock.

I will add reshape_read_endio:
if IO is successful then:
add the bio to reshape_list
else
add the bio to a retry list ( how many retries .. ?)

I will add raid0_reshape:
raid0_reshape is a md_thread that polls on the reshape_list and
commences writes based on the reads.
1. Grub a bio from reshape list.
2. map sector and find zone on the new raid mappings.
3. set bio direction to write.
4. generate a write.

if bio is in retry_list retry the bio.
if bio is in active_io list do the bio.

I will add a reshape_write_endio that just frees the bio and his pages.

raid0_make_request
I will add a check and see if the raid is in reshape.
if so then
if IO is in the new mappings area we generate the IO
from the new mappings.
if IO is in the old mappings then we generate the IO
from the old mappings ( race here .. no ?)
if IO is in the current reshape active area, we push the io to a
active_io list that will processed by raid0_reshape.

Quoted 4 lines · Neil Brown
Quoted 1 line · On Tuesday May 19, [email protected] wrote:

have raid1 check chunk size in run method instead of in md

I dropped this patch. chunk_size is not meaningful for raid1.

how embarrassing.

Quoted 32 lines

NeilBrown

Quoted 30 lines
 raid1.c |    9 +++++++++
 1 file changed, 9 insertions(+)
Signed-off-by [email protected]

---

drivers/md/raid1.c +9 -1
index 2c8d9e6..8609506 100644
@@ -1942,6 +1942,15 @@ static int run(mddev_t *mddev)
 		       mdname(mddev), mddev->level);
 		goto out;
 	}
+	/*
+	 * chunk-size has to be a power of 2
+	 */
+	if ((1 << ffz(~mddev->chunk_size)) != mddev->chunk_size) {
+		printk(KERN_ERR "%s chunk_size of %d not valid\n",
+			mdname(mddev),
+			mddev->chunk_size);
+		return -EINVAL;
+	}
 	if (mddev->reshape_position != MaxSector) {
 		printk("raid1: %s: reshape_position set but not supported\n",
 		       mdname(mddev));
 
 
--

To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 6 lines · On Wednesday May 20, [email protected] wrote:

Neil
First I thank you for your effort. Now I can work in full steam on the reshape on top of the new raid0 code. Currently this is what I have in mind.If you have any design suggestions I would be happy to hear before the coding.

Quoted 6 lines

I added : raid0_add_hot that:
1. checks if the new disk size if smaller than the raid chunk size. if
so , reject.
2. check if new the disk size max_hw_sectors is smaller than the
raid's. if so generate a warning but do not reject.
3. adds a disk to raid0 disk list. and turns off its in_sync bit.

I don't think the 'in_sync' bit is used in raid0 currently, so that bit seems irrelevant, but shouldn't hurt.

Quoted 12 lines

I will add raid0_check_reshape
This procedure prepares the raid for the reshape process.
1. Creates a temporary mddev with the same disks as the raid's and with
the new disks. This raid acts as a mere mappings so i will be able to map sectors to the new target raid in the reshape process. This means i have to work in create_strip_zones raid0_run ( separate patch ).
2. Sets the target raid transfer size.
3. Create an allocation scheme for reshape bio allocation. i reshape in
chunk size.
4. create raid0_reshape thread for writes.
5. wake up raid0_sync thread.

Do you really need a temporary mddev, or just a temporary 'conf'?? Having to create the raid0_reshape thread just for writes is a bit unfortunate, but it probably is the easiest approach. You might be able to get the raid0_sync thread to do them, but that would be messy I expect.

Quoted 10 lines

I will add raid0_sync: raid0_sync acts as the reshape read size process.

1. Allocates a read bio.
2. Map_bio target with find_zone and map_sector, both map_sector and
find_zone are using the old raid mappings.
3. Deactivate the raid.
3. Lock and wait for the raid to be emptied from any previous IOs.
4. Generate a read request.
5. Release the lock.

I think that sounds correct.

Quoted 6 lines

I will add reshape_read_endio:
if IO is successful then:
add the bio to reshape_list
else
add the bio to a retry list ( how many retries .. ?)

zero retries. The underlying block device has done all the retries that are appropriate. If you get a read error, then that block is gone. Probably the best you can do is write garbage to the destination and report the error.

Quoted 9 lines

I will add raid0_reshape:
raid0_reshape is a md_thread that polls on the reshape_list and
commences writes based on the reads.
1. Grub a bio from reshape list.
2. map sector and find zone on the new raid mappings.
3. set bio direction to write.
4. generate a write.

Quoted 4 lines

if bio is in retry_list retry the bio.
if bio is in active_io list do the bio.

I will add a reshape_write_endio that just frees the bio and his pages.

OK (except for the retry).

Quoted 10 lines

raid0_make_request
I will add a check and see if the raid is in reshape.
if so then
if IO is in the new mappings area we generate the IO
from the new mappings.
if IO is in the old mappings then we generate the IO
from the old mappings ( race here .. no ?)
if IO is in the current reshape active area, we push the io to a
active_io list that will processed by raid0_reshape.

This doesn't seem to match what you say above.
If you don't submit a read for 'reshape' until all IO has drained, then presumably you would just block any incoming IO until the current reshape requests have all finished. i.e. you only ever have IO or reshape, but not both.

Alternately you could have a sliding window covering there area that is currently being reshaped.
If an IO comes in for that area, you need to either
- close the window and perform the IO, or
- wait for the window to slide past.

I would favour the latter. But queueing the IO for raid0_reshape doesn't really gain you anything I think.

Issues that you haven't mentioned:
- metadata update: you need to record progress in the metadata as the window slides along, in case of an unclean restart
- Unless you only schedule one chunk at a time (which would be slow things down I expect), you need to ensure that you don't schedule a write to block for which the read hasn't completed yet. This is particularly an issues if you support changing the chunk size.
- I assume you are (currently) only supporting a reshape that increases the size of the array and the number of devices?

NeilBrown

#4 lore raw
Quoted 29 lines · Neil Brown
Quoted 6 lines · On Wednesday May 20, [email protected] wrote:

Neil
First I thank you for your effort. Now I can work in full steam on the reshape on top of the new raid0 code. Currently this is what I have in mind.If you have any design suggestions I would be happy to hear before the coding.

Quoted 6 lines

I added : raid0_add_hot that:
1. checks if the new disk size if smaller than the raid chunk size. if
so , reject.
2. check if new the disk size max_hw_sectors is smaller than the
raid's. if so generate a warning but do not reject.
3. adds a disk to raid0 disk list. and turns off its in_sync bit.

I don't think the 'in_sync' bit is used in raid0 currently, so that bit seems irrelevant, but shouldn't hurt.

Quoted 12 lines

I will add raid0_check_reshape
This procedure prepares the raid for the reshape process.
1. Creates a temporary mddev with the same disks as the raid's and with
the new disks. This raid acts as a mere mappings so i will be able to map sectors to the new target raid in the reshape process. This means i have to work in create_strip_zones raid0_run ( separate patch ).
2. Sets the target raid transfer size.
3. Create an allocation scheme for reshape bio allocation. i reshape in
chunk size.
4. create raid0_reshape thread for writes.
5. wake up raid0_sync thread.

Quoted 1 line

Do you really need a temporary mddev, or just a temporary 'conf'??

I need mddev because i want to use map_sector and create_strip. I certainly can fix map_sector and create_strip to work with conf and not mddev, though it will make create_strip quite cumbersome.
I will split create_strip to several independent functions. Do you agree ?

Quoted 4 lines

Having to create the raid0_reshape thread just for writes is a bit unfortunate, but it probably is the easiest approach. You might be able to get the raid0_sync thread to do them, but that would be messy I expect.

I will start with the easy approach, meaning , a different thread for the writes. Once i am done , i will see how can merge the reads and writes to work in md_sync.

Quoted 52 lines
Quoted 10 lines

I will add raid0_sync: raid0_sync acts as the reshape read size process.

1. Allocates a read bio.
2. Map_bio target with find_zone and map_sector, both map_sector and
find_zone are using the old raid mappings.
3. Deactivate the raid.
3. Lock and wait for the raid to be emptied from any previous IOs.
4. Generate a read request.
5. Release the lock.

I think that sounds correct.

Quoted 6 lines

I will add reshape_read_endio:
if IO is successful then:
add the bio to reshape_list
else
add the bio to a retry list ( how many retries .. ?)

zero retries. The underlying block device has done all the retries that are appropriate. If you get a read error, then that block is gone. Probably the best you can do is write garbage to the destination and report the error.

Quoted 9 lines

I will add raid0_reshape:
raid0_reshape is a md_thread that polls on the reshape_list and
commences writes based on the reads.
1. Grub a bio from reshape list.
2. map sector and find zone on the new raid mappings.
3. set bio direction to write.
4. generate a write.

Quoted 4 lines

if bio is in retry_list retry the bio.
if bio is in active_io list do the bio.

I will add a reshape_write_endio that just frees the bio and his pages.

OK (except for the retry).

Quoted 10 lines

raid0_make_request
I will add a check and see if the raid is in reshape.
if so then
if IO is in the new mappings area we generate the IO
from the new mappings.
if IO is in the old mappings then we generate the IO
from the old mappings ( race here .. no ?)
if IO is in the current reshape active area, we push the io to a
active_io list that will processed by raid0_reshape.

Quoted 5 lines

This doesn't seem to match what you say above.
If you don't submit a read for 'reshape' until all IO has drained, then presumably you would just block any incoming IO until the current reshape requests have all finished. i.e. you only ever have IO or reshape, but not both.

Where did i said that ? guess i wasn't clear.

Quoted 7 lines

Alternately you could have a sliding window covering there area that is currently being reshaped.
If an IO comes in for that area, you need to either
- close the window and perform the IO, or
- wait for the window to slide past.
I would favour the latter. But queueing the IO for raid0_reshape doesn't really gain you anything I think.

I wasn't clear enough.A "current reshape active area" is my sliding window. I wait for the window to slide past. this is exactly what i had in mind.so yes this is what am writing, a reshape_window.

Quoted 3 lines

Issues that you haven't mentioned:
- metadata update: you need to record progress in the metadata as the window slides along, in case of an unclean restart

I thought md does that for me. So it doesn't. Am i to call md_allow_write ( that calls md_update_sbs ) ? how frequent ?

Quoted 3 lines

- Unless you only schedule one chunk at a time (which would be slow things down I expect), you need to ensure that you don't schedule a write to block for which the read hasn't completed yet.

ah... yes. i thought of it but forgot. my question is how ? should i simply use an interruptable sleep ? what do you do in raid5 ?

Quoted 4 lines

This is particularly an issues if you support changing the chunk size.
- I assume you are (currently) only supporting a reshape that increases the size of the array and the number of devices?

neither changing the chunk size nor shrink is in my spec. so no.
Maybe when i finish my studies ( you can google for "offsched + raz" and follow the link ... ) then i will have some raid quality time.

Quoted 1 line

NeilBrown

#5 lore raw
Quoted 31 lines · On Thursday May 21, [email protected] wrote:
Quoted 29 lines · Neil Brown
Quoted 6 lines · On Wednesday May 20, [email protected] wrote:

Neil
First I thank you for your effort. Now I can work in full steam on the reshape on top of the new raid0 code. Currently this is what I have in mind.If you have any design suggestions I would be happy to hear before the coding.

Quoted 6 lines

I added : raid0_add_hot that:
1. checks if the new disk size if smaller than the raid chunk size. if
so , reject.
2. check if new the disk size max_hw_sectors is smaller than the
raid's. if so generate a warning but do not reject.
3. adds a disk to raid0 disk list. and turns off its in_sync bit.

I don't think the 'in_sync' bit is used in raid0 currently, so that bit seems irrelevant, but shouldn't hurt.

Quoted 12 lines

I will add raid0_check_reshape
This procedure prepares the raid for the reshape process.
1. Creates a temporary mddev with the same disks as the raid's and with
the new disks. This raid acts as a mere mappings so i will be able to map sectors to the new target raid in the reshape process. This means i have to work in create_strip_zones raid0_run ( separate patch ).
2. Sets the target raid transfer size.
3. Create an allocation scheme for reshape bio allocation. i reshape in
chunk size.
4. create raid0_reshape thread for writes.
5. wake up raid0_sync thread.

Quoted 4 lines
Quoted 1 line

Do you really need a temporary mddev, or just a temporary 'conf'??

I need mddev because i want to use map_sector and create_strip. I certainly can fix map_sector and create_strip to work with conf and not mddev, though it will make create_strip quite cumbersome.

I think it would actually be an improvement.
Maybe look at setup_conf in raid5.c. It does similar things for a similar reason.
Maybe:
- move the setting of mddev->queue->* from create_strip_zones to

    raid0_run.
 -  move the chunk_size check to raid0_run
 -  pass in raid_disks as an argument
 -  get create_strip_zones to return the conf, rather than setting
    ->private (raid0_run sets private if the returned conf was not an
    error).

Then the only part of mddev that create_strip_zones uses is ->disks which is the same list if both cases (if you ignore those with a ->raid_disk that is too large).

Quoted 2 lines

I will split create_strip to several independent functions. Do you agree ?

Hard to know without seeing the split. Maybe if you said what the new functions would be.

Quoted 7 lines
Quoted 4 lines

Having to create the raid0_reshape thread just for writes is a bit unfortunate, but it probably is the easiest approach. You might be able to get the raid0_sync thread to do them, but that would be messy I expect.

I will start with the easy approach, meaning , a different thread for the writes. Once i am done , i will see how can merge the reads and writes to work in md_sync.

Definitely sensible.

Quoted 65 lines
Quoted 52 lines
Quoted 10 lines

I will add raid0_sync: raid0_sync acts as the reshape read size process.

1. Allocates a read bio.
2. Map_bio target with find_zone and map_sector, both map_sector and
find_zone are using the old raid mappings.
3. Deactivate the raid.
3. Lock and wait for the raid to be emptied from any previous IOs.
4. Generate a read request.
5. Release the lock.

I think that sounds correct.

Quoted 6 lines

I will add reshape_read_endio:
if IO is successful then:
add the bio to reshape_list
else
add the bio to a retry list ( how many retries .. ?)

zero retries. The underlying block device has done all the retries that are appropriate. If you get a read error, then that block is gone. Probably the best you can do is write garbage to the destination and report the error.

Quoted 9 lines

I will add raid0_reshape:
raid0_reshape is a md_thread that polls on the reshape_list and
commences writes based on the reads.
1. Grub a bio from reshape list.
2. map sector and find zone on the new raid mappings.
3. set bio direction to write.
4. generate a write.

Quoted 4 lines

if bio is in retry_list retry the bio.
if bio is in active_io list do the bio.

I will add a reshape_write_endio that just frees the bio and his pages.

OK (except for the retry).

Quoted 10 lines

raid0_make_request
I will add a check and see if the raid is in reshape.
if so then
if IO is in the new mappings area we generate the IO
from the new mappings.
if IO is in the old mappings then we generate the IO
from the old mappings ( race here .. no ?)
if IO is in the current reshape active area, we push the io to a
active_io list that will processed by raid0_reshape.

Quoted 5 lines

This doesn't seem to match what you say above.
If you don't submit a read for 'reshape' until all IO has drained, then presumably you would just block any incoming IO until the current reshape requests have all finished. i.e. you only ever have IO or reshape, but not both.

Where did i said that ? guess i wasn't clear.

Quoted 7 lines

Alternately you could have a sliding window covering there area that is currently being reshaped.
If an IO comes in for that area, you need to either
- close the window and perform the IO, or
- wait for the window to slide past.
I would favour the latter. But queueing the IO for raid0_reshape doesn't really gain you anything I think.

Quoted 3 lines

I wasn't clear enough.A "current reshape active area" is my sliding window. I wait for the window to slide past. this is exactly what i had in mind.so yes this is what am writing, a reshape_window.

Excellent !!

Quoted 5 lines
Quoted 3 lines

Issues that you haven't mentioned:
- metadata update: you need to record progress in the metadata as the window slides along, in case of an unclean restart

I thought md does that for me. So it doesn't. Am i to call md_allow_write ( that calls md_update_sbs ) ? how frequent ?

Look at what raid5 does... though that might be a bit hard to follow. The metadata records the current 'reshape_position'. That must always be inside your sliding window. Once the trailing edge reaches the reshape_position currently record on disk, you need to update that to be the leading edge of the window...
Or something like that. It doesn't feel quite right. Maybe it is too late at night to be reasoning this sort of thing clearly :-)

With raid5, I
set MD_CHANGE_DEVS in mddev->flags
md_wakeup_thread(mddev->thread)

and that thread calls md_check_recovery which does:

  if (mddev->flags) {
     if (mddev_trylock(mddev)) {
        if (mddev->flags)
             md_update_sb(mddev, 0);
        mddev_unlock(mddev);
     }
Quoted 5 lines
Quoted 3 lines

- Unless you only schedule one chunk at a time (which would be slow things down I expect), you need to ensure that you don't schedule a write to block for which the read hasn't completed yet.

ah... yes. i thought of it but forgot. my question is how ? should i simply use an interruptable sleep ? what do you do in raid5 ?

If raid5, I read a whole stripe into the cache before releasing any of it to be written. Something like that is probably appropriate for raid0 too, though you don't have a cache in the same way, so the code will be very different.

Quoted 7 lines
Quoted 4 lines

This is particularly an issues if you support changing the chunk size.
- I assume you are (currently) only supporting a reshape that increases the size of the array and the number of devices?

neither changing the chunk size nor shrink is in my spec. so no.
Maybe when i finish my studies ( you can google for "offsched + raz" and follow the link ... ) then i will have some raid quality time.

:-)

Thanks,
NeilBrown