Creating a 3-disk RAID6 array
Hi,
is there any specific reason why md refuses to create a RAID6 array with 3 disks? My (probably naive) understanding suggests it should be the same as a 3-disk RAID1, similar to a 2-disk RAID5.
The reason I'm asking is that I currently have space on three disks for a new array, and would like to expand it when I add a fourth. I tried this scenario with a few loopback devices, but the only way to go from a 3-disk RAID1 to a 4-disk RAID6 seems to be via an intermediate 3-disk RAID5, requiring two reshapes. I'd like to avoid one of them, if at all possible.
Regards,
Oliver
Quoted 5 lines · On Thu, 17 May 2012 03:24:26 +0200 Oliver Martin <[email protected]>…
Hi,
is there any specific reason why md refuses to create a RAID6 array with 3 disks? My (probably naive) understanding suggests it should be the same as a 3-disk RAID1, similar to a 2-disk RAID5.
I'll have to leave for for hpa to answer. I've occasionally thought that maybe it should be fixed, but it never seemed worth the effort.
Quoted 7 lines
The reason I'm asking is that I currently have space on three disks for a new array, and would like to expand it when I add a fourth. I tried this scenario with a few loopback devices, but the only way to go from a 3-disk RAID1 to a 4-disk RAID6 seems to be via an intermediate 3-disk RAID5, requiring two reshapes. I'd like to avoid one of them, if at all possible.
Yes, not possible at present.
It might be as simple and finding the places that impose the limit and delete them...
NeilBrown
Quoted 7 lines
Regards,
Oliver
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 20 lines · NeilBrown
Quoted 5 lines · On Thu, 17 May 2012 03:24:26 +0200 Oliver Martin<[email protected]> wrote:
Hi,
is there any specific reason why md refuses to create a RAID6 array with 3 disks? My (probably naive) understanding suggests it should be the same as a 3-disk RAID1, similar to a 2-disk RAID5.
I'll have to leave for for hpa to answer. I've occasionally thought that maybe it should be fixed, but it never seemed worth the effort.
Quoted 7 lines
The reason I'm asking is that I currently have space on three disks for a new array, and would like to expand it when I add a fourth. I tried this scenario with a few loopback devices, but the only way to go from a 3-disk RAID1 to a 4-disk RAID6 seems to be via an intermediate 3-disk RAID5, requiring two reshapes. I'd like to avoid one of them, if at all possible.
Quoted 6 lines
Yes, not possible at present.
It might be as simple and finding the places that impose the limit and delete them...NeilBrown
Theoretically, a 3-disk RAID6 is like a 2-disk RAID5 or a 1-disk RAID1 - it is possible, but not much use except as a building block for expansion later. mdadm supports 1-disk RAID1 (you need a "--force" to persuade mdadm that you know what you are doing, which is nice), and I think it supports a 2-disk RAID5, though I haven't tried it. I can't think of any good reason for it /not/ to support 3-disk RAID6, as there is nothing in the algorithms to hinder it.
mvh.,
David
On 17/05/2012 14:18, David Brown wrote:
[...]
Quoted 1 line
Theoretically, a 3-disk RAID6 is like a 2-disk RAID5 or a 1-disk RAID1 -
[...]
Quoted 2 lines
I can't think of any good reason for it /not/ to support 3-disk RAID6, as there is nothing in the algorithms to hinder it.
I've a vague recollection of Peter Anvin saying the implementation is optimised in such a way that it won't work.
Cheers,
John.
Quoted 10 lines · John Robinson
On 17/05/2012 14:18, David Brown wrote:
[...]Quoted 1 line
Theoretically, a 3-disk RAID6 is like a 2-disk RAID5 or a 1-disk RAID1 -
[...]
Quoted 2 lines
I can't think of any good reason for it /not/ to support 3-disk RAID6, as there is nothing in the algorithms to hinder it.
I've a vague recollection of Peter Anvin saying the implementation is optimised in such a way that it won't work.
Okay, optimised implementation is a good reason for not supporting it! I only know the theory, I haven't studied the implementation well enough yet. (One day, when I get time... :-)
Quoted 4 lines
Cheers,
John.
Quoted 10 lines · John Robinson
On 17/05/2012 14:18, David Brown wrote:
[...]Quoted 1 line
Theoretically, a 3-disk RAID6 is like a 2-disk RAID5 or a 1-disk RAID1 -
[...]
Quoted 2 lines
I can't think of any good reason for it /not/ to support 3-disk RAID6, as there is nothing in the algorithms to hinder it.
I've a vague recollection of Peter Anvin saying the implementation is optimised in such a way that it won't work.
Yes, it would have required introducing some odd special cases. A 3-disk RAID-6 is bitwise identical to a 3-disk RAID-1, so if Neil wants to he could add "instant reshaping" support in mdadm (add a disk to a 3-disk RAID-1 turning it into a 4-disk RAID-6; and similar for 2-disk RAID-1 into 3-disk RAID-5.)
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 12 lines · On Fri, 18 May 2012 00:30:47 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 10 lines · John Robinson
On 17/05/2012 14:18, David Brown wrote:
[...]Quoted 1 line
Theoretically, a 3-disk RAID6 is like a 2-disk RAID5 or a 1-disk RAID1 -
[...]
Quoted 2 lines
I can't think of any good reason for it /not/ to support 3-disk RAID6, as there is nothing in the algorithms to hinder it.
I've a vague recollection of Peter Anvin saying the implementation is optimised in such a way that it won't work.
Quoted 8 lines
Yes, it would have required introducing some odd special cases. A 3-disk RAID-6 is bitwise identical to a 3-disk RAID-1, so if Neil wants to he could add "instant reshaping" support in mdadm (add a disk to a 3-disk RAID-1 turning it into a 4-disk RAID-6; and similar for 2-disk RAID-1 into 3-disk RAID-5.)
-hpa
Converting a 2-disk RAID-1 to a 3-disk RAID-5 is not instant.
You first convert a 2-disk RAID-1 to a 2-disk RAID-5. Then you re-stripe the RAID-5 to have 3-disks. During this restripe, part of the array looks like a 2-disk RAID-5, and part looks like a 3-disk RAID-4.
To convert some sort of RAID1 to RAID6 we would need to be able to support a 3-disk RAID-6, if only for a relatively short time.
Possibly we could special case the parity-generation code to just copy the data block when data-disks==1. That sound easy enough...
But the raid456 module needs to understand this 3-way-RAID1-pretending-to-be-RAID6, we cannot use the raid1 module to do it.
NeilBrown
Quoted 7 lines · NeilBrown
Possibly we could special case the parity-generation code to just copy the data block when data-disks==1. That sound easy enough... But the raid456 module needs to understand this 3-way-RAID1-pretending-to-be-RAID6, we cannot use the raid1 module to do it.
I actually think it will Just Work as-is... there might be some need to modify the loop control logic.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 11 lines · NeilBrown
Quoted 5 lines · On Thu, 17 May 2012 03:24:26 +0200 Oliver Martin <[email protected]>…
Hi,
is there any specific reason why md refuses to create a RAID6 array with 3 disks? My (probably naive) understanding suggests it should be the same as a 3-disk RAID1, similar to a 2-disk RAID5.
I'll have to leave for for hpa to answer. I've occasionally thought that maybe it should be fixed, but it never seemed worth the effort.
The math assumes 2 data disks.
Quoted 10 lines
Quoted 6 lines
The reason I'm asking is that I currently have space on three disks for a new array, and would like to expand it when I add a fourth. I tried this scenario with a few loopback devices, but the only way to go from a 3-disk RAID1 to a 4-disk RAID6 seems to be via an intermediate 3-disk RAID5, requiring two reshapes. I'd like to avoid one of them, if at all possible.
Yes, not possible at present.
It might be as simple and finding the places that impose the limit and delete them...
You'd certainly need to route around the acceleration code, because that increased the dependency on the assumption that there is always two data disk slots.
--
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Am 17.05.2012 23:02, schrieb Dan Williams:
Quoted 4 lines
Quoted 2 lines
I'll have to leave for for hpa to answer. I've occasionally thought that maybe it should be fixed, but it never seemed worth the effort.
The math assumes 2 data disks.
Aside from a BUG_ON in async_raid6_datap_recov, it seems like it should work based on a cursory glance. Especially the various gen_syndrome implementations and raid6_datap_recov look like they should produce the correct result.
Quoted 7 lines
Quoted 3 lines
Yes, not possible at present.
It might be as simple and finding the places that impose the limit and delete them...You'd certainly need to route around the acceleration code, because that increased the dependency on the assumption that there is always two data disk slots.
This, however, makes it seem like lots of hassle for very little gain, given that the same on-disk data can much more cheaply be produced by using a RAID1. I think the more sensible thing to do is to add support for reshaping a RAID1 into a 4-disk RAID6. This should hopefully not be too different from the existing RAID5-RAID6 reshape, and is probably what I'll (at least try to) implement when the time comes to expand my array.
Oliver
Quoted 6 lines · Oliver Martin
Hi,
is there any specific reason why md refuses to create a RAID6 array with 3 disks? My (probably naive) understanding suggests it should be the same as a 3-disk RAID1, similar to a 2-disk RAID5.
Quoted 6 lines
The reason I'm asking is that I currently have space on three disks for a new array, and would like to expand it when I add a fourth. I tried this scenario with a few loopback devices, but the only way to go from a 3-disk RAID1 to a 4-disk RAID6 seems to be via an intermediate 3-disk RAID5, requiring two reshapes. I'd like to avoid one of them, if at all possible.
If you are definitely planning to add another drive fairly soon, you can create a four drive array with one missing. Given that the performance of a three way raid-6 is not going to be stellar anyway, since you write to every drive with every write, you actually might find it runs better only writing to two on some chunks.
It avoids the whole double reshape issue.
Signature · 3 lines
Bill Davidsen <[email protected]> "We have more to fear from the bungling of the incompetent than from the machinations of the wicked." - from Slashdot
Quoted 6 lines · Oliver Martin
Hi,
is there any specific reason why md refuses to create a RAID6 array with 3 disks? My (probably naive) understanding suggests it should be the same as a 3-disk RAID1, similar to a 2-disk RAID5.
Quoted 6 lines
The reason I'm asking is that I currently have space on three disks for a new array, and would like to expand it when I add a fourth. I tried this scenario with a few loopback devices, but the only way to go from a 3-disk RAID1 to a 4-disk RAID6 seems to be via an intermediate 3-disk RAID5, requiring two reshapes. I'd like to avoid one of them, if at all possible.
Make a 3 partition (seems you're using partitions) md RAID 10 (RAID 1E) array with what you have now. When you add a disk down the road, backup your filesystem, create your 4 disk md RAID 6 array, format, restore.
Now for my $0.02: You're better served all around with a 4 partition RAID 10 or 1+0, than a 4 partition RAID 6. Random write performance is greatly superior, rebuild time is significantly lower, etc. Sure it would be nice if mdadm could reshape a 3 disk RAID10 into a 4 disk RAID10, but it can't. But since you should be doing a backup/restore anyway, it doesn't matter.
Signature · 1 line
Stan