Can't expand linear RAID on top of 2 x RAID1
I have expanded my system over time, I started with 2 x 2TB drives in RAID1 (md2)
I then added 2 x 750GB drives, configured as RAID1 (md1) Then created a third raid (md3) as linear with the md2 + md1
Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a time).
I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
The problem I am having is that I can't expand the linear (md3) array to grow the extra 250G of space.
Could anyone suggest how I might be able to get the extra 250G of space to become available?
keep:~# cat /proc/mdstat
Personalities : [raid1] [linear]
md3 : active linear md2[0] md1[1]
2686081193 blocks super 1.2 0k roundingmd1 : active (auto-read-only) raid1 sdb1[2] sdd1[3]
976758977 blocks super 1.2 [2/2] [UU]
md2 : active (auto-read-only) raid1 sda1[0] sdc1[2]
1953510841 blocks super 1.2 [2/2] [UU]
unused devices: <none>
keep:~# mdadm --grow /dev/md3 --size=max mdadm: component size of /dev/md3 has been set to 0K
There is nothing logged for dmesg after the above command, no change to /proc/mdstat.
I have tried a reboot with no change either.
Thanks for any assistance, I'd really appreciate it.
Regards,
Adam
Quoted 8 lines · Adam Goryachev
I have expanded my system over time, I started with 2 x 2TB drives in RAID1 (md2)
I then added 2 x 750GB drives, configured as RAID1 (md1) Then created a third raid (md3) as linear with the md2 + md1
Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a time).
Quoted 7 lines
I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
The problem I am having is that I can't expand the linear (md3) array to grow the extra 250G of space.
Could anyone suggest how I might be able to get the extra 250G of space to become available?
If you think about this for a few minutes more, and re-read how md --linear works, and thus how growing a linear array works, you'll surely understand why you can't do what you're attempting to do.
As for seeing that extra 250GB, I don't have an answer. Typically linear arrays are used in lieu of growing constituent member arrays. That's kinda the whole point of linear (concatenation).
You could try deleting the linear array and simply creating a new one. But surely the changed offsets would wreak havoc on the filesystem currently spanning this mess.
Signature · 1 line
Stan
Quoted 16 lines · Stan Hoeppner
Quoted 7 lines · Adam Goryachev
I have expanded my system over time, I started with 2 x 2TB drives in RAID1 (md2)
I then added 2 x 750GB drives, configured as RAID1 (md1) Then created a third raid (md3) as linear with the md2 + md1
Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
time).
Quoted 4 lines
I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
The problem I am having is that I can't expand the linear (md3) array
to
Quoted 2 lines
grow the extra 250G of space.
Quoted 8 lines
Quoted 1 line
Could anyone suggest how I might be able to get the extra 250G of
space
Quoted 1 line
to become available?
If you think about this for a few minutes more, and re-read how md --linear works, and thus how growing a linear array works, you'll surely
understand why you can't do what you're attempting to do.
Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
Quoted 3 lines
As for seeing that extra 250GB, I don't have an answer. Typically linear arrays are used in lieu of growing constituent member arrays. That's kinda the whole point of linear (concatenation).
Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).
Quoted 3 lines
You could try deleting the linear array and simply creating a new one. But surely the changed offsets would wreak havoc on the filesystem currently spanning this mess.
Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?
What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in /proc/mdstat just tell me what order the devices were added, not what order they are in the array?
Thank you for your advice.
Regards,
Adam
Quoted 27 lines · Adam Goryachev
Quoted 16 lines · Stan Hoeppner
Quoted 7 lines · Adam Goryachev
I have expanded my system over time, I started with 2 x 2TB drives in RAID1 (md2)
I then added 2 x 750GB drives, configured as RAID1 (md1) Then created a third raid (md3) as linear with the md2 + md1
Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
time).
Quoted 4 lines
I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
The problem I am having is that I can't expand the linear (md3) array
to
Quoted 2 lines
grow the extra 250G of space.
Quoted 8 lines
Quoted 1 line
Could anyone suggest how I might be able to get the extra 250G of
space
Quoted 1 line
to become available?
If you think about this for a few minutes more, and re-read how md --linear works, and thus how growing a linear array works, you'll surely
understand why you can't do what you're attempting to do.
Quoted 7 lines
Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
Quoted 3 lines
As for seeing that extra 250GB, I don't have an answer. Typically linear arrays are used in lieu of growing constituent member arrays. That's kinda the whole point of linear (concatenation).
Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).
This is the only sure fire way I see ATM to do this. But I agree, it's not very clean. I'm a big proponent of only one array per physical disk, as I stated recently, and the reasons for it. Given what I'm guessing your workload profile is, it wouldn't make a difference.
Quoted 9 lines
Quoted 3 lines
You could try deleting the linear array and simply creating a new one. But surely the changed offsets would wreak havoc on the filesystem currently spanning this mess.
Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?
What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in
/proc/mdstatjust tell me what order the devices were added, not what order they are in the array?Thank you for your advice.
The best advice I can give you is to wait for Neil Brown to weigh in before you make any such changes.
Signature · 1 line
Stan
Hay Neil,
Could you please take a look at this one? I'm at the limits of my knowledge in this case, not exactly sure what the OP's next step should be.
Thanks.
Signature · 1 line
Stan
Quoted 54 lines · Stan Hoeppner
Quoted 27 lines · Adam Goryachev
Quoted 16 lines · Stan Hoeppner
Quoted 7 lines · Adam Goryachev
I have expanded my system over time, I started with 2 x 2TB drives in RAID1 (md2)
I then added 2 x 750GB drives, configured as RAID1 (md1) Then created a third raid (md3) as linear with the md2 + md1
Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
time).
Quoted 4 lines
I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
The problem I am having is that I can't expand the linear (md3) array
to
Quoted 2 lines
grow the extra 250G of space.
Quoted 8 lines
Quoted 1 line
Could anyone suggest how I might be able to get the extra 250G of
space
Quoted 1 line
to become available?
If you think about this for a few minutes more, and re-read how md --linear works, and thus how growing a linear array works, you'll surely
understand why you can't do what you're attempting to do.Quoted 7 lines
Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
Quoted 3 lines
As for seeing that extra 250GB, I don't have an answer. Typically linear arrays are used in lieu of growing constituent member arrays. That's kinda the whole point of linear (concatenation).
Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).
This is the only sure fire way I see ATM to do this. But I agree, it's not very clean. I'm a big proponent of only one array per physical disk, as I stated recently, and the reasons for it. Given what I'm guessing your workload profile is, it wouldn't make a difference.
Quoted 9 lines
Quoted 3 lines
You could try deleting the linear array and simply creating a new one. But surely the changed offsets would wreak havoc on the filesystem currently spanning this mess.
Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?
What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in
/proc/mdstatjust tell me what order the devices were added, not what order they are in the array?Thank you for your advice.
The best advice I can give you is to wait for Neil Brown to weigh in before you make any such changes.
Quoted 33 lines · On Sat, 23 Jun 2012 12:53:12 +1000 Adam Goryachev…
Quoted 16 lines · Stan Hoeppner
Quoted 7 lines · Adam Goryachev
I have expanded my system over time, I started with 2 x 2TB drives in RAID1 (md2)
I then added 2 x 750GB drives, configured as RAID1 (md1) Then created a third raid (md3) as linear with the md2 + md1
Finally, I've upgraded the 2 x 750G to 2 x 1TB drives (one at a
time).
Quoted 4 lines
I then did a mdadm --grow to expand the RAID1 from 750G to 1TB
The problem I am having is that I can't expand the linear (md3) array
to
Quoted 2 lines
grow the extra 250G of space.
Quoted 8 lines
Quoted 1 line
Could anyone suggest how I might be able to get the extra 250G of
space
Quoted 1 line
to become available?
If you think about this for a few minutes more, and re-read how md --linear works, and thus how growing a linear array works, you'll surely
understand why you can't do what you're attempting to do.Actually, no I don't understand the problem... From my (probably limited) understanding, linear simply appends each drive to the array. I understand that you would not be able to increase the size of any constituent device which is not the last component of the array, but I don't see any reason why it is not possible to increase the size of the last component. (Which is what I am attempting to do)
Quoted 3 lines
As for seeing that extra 250GB, I don't have an answer. Typically linear arrays are used in lieu of growing constituent member arrays. That's kinda the whole point of linear (concatenation).
Quoted 7 lines
Well, the only other alternative I can think of is to reduce the size of the array back to 750G, reduce the size of the partitions for that array to 750G. Then create a new 250G partition on the two 1TB drives, create a fourth RAID1 array, and then add that 4th RAID1 array to the end of the existing linear array. This just didn't seem like a very good solution (ie, not clean).
Quoted 3 lines
You could try deleting the linear array and simply creating a new one. But surely the changed offsets would wreak havoc on the filesystem currently spanning this mess.
Perhaps this would work. If I delete the linear array and re-create, with assume-clean (or whatever the right flag is, I'll read the man page later when I'm doing it), then like I said, the first component device will be the same, the second component device is the same, but just happens to be bigger. Any comments on whether this should work?
The is no '--assume-clean' for linear. Linear arrays cannot be clean or dirty as md stores no redundant information.
Just re-creating the array would probably work, but there is an easier way.
md doesn't record the size of the array for Linear and RAID0. It just adds together the sizes of the devices that it finds.
With 1.x metadata, md does record the effective size of each devices, so adds those together.
You can update this effective size by assembling with
--update=devicesize
I just checked this will loop devices and it works as expected (assuming you have 1.1 or 1.2 metadata).
So:
mdadm -S /dev/md2
mdadm -A --update=devicesize /dev/md0 /dev/md1
(order doesn't matter with assemble).
Quoted 2 lines
What should I do to ensure that the component devices are ordered the way I think they are? Apparently the numbers in
/proc/mdstatjust tell me what order the devices were added, not what order they are in the array?
mdadm --detail or mdadm --examine would tell you what you want to know.
NeilBrown
Quoted 12 lines
Thank you for your advice.
Regards,
Adam--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 13 lines · NeilBrown
The is no '--assume-clean' for linear. Linear arrays cannot be clean or dirty as md stores no redundant information.
Just re-creating the array would probably work, but there is an easier way.
md doesn't record the size of the array for Linear and RAID0. It just adds together the sizes of the devices that it finds.
With 1.x metadata, md does record the effective size of each devices, so adds those together. You can update this effective size by assembling with --update=devicesize
Quoted 5 lines
I just checked this will loop devices and it works as expected (assuming you have 1.1 or 1.2 metadata). So: mdadm -S
/dev/md2mdadm -A --update=devicesize/dev/md0/dev/md1(order doesn't matter with assemble).
Hi,
Just a followup email on this to say Thank You very much for your expert assistance, it was greatly appreciated.
Quoted 1 line
mdadm -S
/dev/md2
I had some issues with this command, df showed that it was unmounted, lsof showed nothing looking at the device nor filesystem where it was mounted, etc... Eventually, I ran "mount" and realised I had bind mounted a sub-directory elsewhere... Once I umounted that I could stop the array... (Duh moment :)
Quoted 1 line
mdadm -A --update=devicesize
/dev/md0/dev/md1
I got an error that the device was in use, thankfully mdadm has the smarts to stop me from idly shooting myself in the foot when doing something as silly as copy & paste instructions from others without stopping and thinking first.
I adjusted the command to the correct:
mdadm -A --update=devicesize /dev/md2 /dev/md0 /dev/md1
and got:
Size was 3907019634
Size is 3907019634
Size was 1465142752
Size is 1953515906
I then checked the array, was told the filesystem was clean, and then resized that, and now I have the extra 250G space (just in time, I only had 6% free space there).
So, again, thank you for your assistance, and thanks for all the work you put into the drivers etc.
Regards,
Adam