Growing after replacing with larger discs
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
Cheers,
John.
I'm not sure if it is going to work or not, what you suggested.
But if you can run all disks at the same time, then why not create a new array and copy the files over?
Quoted 16 lines · John Robinson
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
Cheers,
John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
Majed B. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines · Majed B.
I'm not sure if it is going to work or not, what you suggested.
But if you can run all disks at the same time, then why not create a new array and copy the files over?
I can't, I only have the 3 drive caddies, so I was going to shut down, start up from a CD, copy each drive individually, then put the 3 larger drives in all at once.
Cheers,
John.
If you do this kind of thing a lot, or often enough, then it's better to get a portable NAS, copy filesystem contents to it, then dump the stuff back.
You can save yourself time with the new arrays by zeroing all the new disks, and then when creating the new array, use assume-clean. That'll prevent the array from resyncing, and you'll get full speed when copying the filesystem contents back to the new array.
dd allows parallel writing to output devices.
Quoted 16 lines · John Robinson
Quoted 5 lines · Majed B.
I'm not sure if it is going to work or not, what you suggested.
But if you can run all disks at the same time, then why not create a new array and copy the files over?
I can't, I only have the 3 drive caddies, so I was going to shut down, start up from a CD, copy each drive individually, then put the 3 larger drives in all at once.
Cheers,
John.
Signature · 1 line
Majed B.
Quoted 1 line · John Robinson
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
I believe it won't work
mdadm has no way to know where the metadata is in the above case.
you have to follow the painful procedure fail one drive (mdadm /dev/mdX --fail /dev/sdY) remove it (mdadm /dev/mdX --remove /dev/sdY) replace drive with larger one
create larger partition
add drive again (mdadm /dev/mdX --add /dev/sdY) wait for it to resync...
...and repeat until all drives have been replaced
L.
Signature · 6 lines
Luca Berra -- [email protected] Communication Media & Services S.r.l. /"\ \ / ASCII RIBBON CAMPAIGN X AGAINST HTML MAIL / \
Quoted 5 lines · Luca Berra
Quoted 1 line · John Robinson
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
I believe it won't work
mdadm has no way to know where the metadata is in the above case.
Quoted 8 lines
you have to follow the painful procedure fail one drive (mdadm
/dev/mdX--fail/dev/sdY) remove it (mdadm/dev/mdX--remove/dev/sdY) replace drive with larger one
create larger partition
add drive again (mdadm/dev/mdX--add/dev/sdY) wait for it to resync...
...and repeat until all drives have been replaced
Yeuch. That'll take forever, because I'd need to rebuild 3 times. I see two other options:
(1) dd the drives as I said originally, and when the system can't find the array on the bigger discs, recreate the array using the same options the original array had
(2) degrade the existing array to two discs, replace the soft-failed disc with one of the new discs, temporarily make a filesystem on it, copy the data; replace the other two discs, make a new array on them with a missing drive, copy the data back; scrub the temporarily-made filesystem and add the drive to the array.
Actually I think something along the lines of Majed B's suggestion - in this case hang a spare drive out the side of the chassis, and just copy twice - would be quicker than (2). But (1) should work, shouldn't it?
Cheers,
John.
You should never degrade the array to copy its contents. Its very risky. There might be bad sectors on one or more disks and if you don't have all disks at hand, you may not be able to rebuild the array on the new disk.
As for using dd, as others have pointed, the metadata won't be in place and if you create a new array after using dd, it'll still require a resync and the data will be destroyed.
Quoted 47 lines · John Robinson
Quoted 14 lines · Luca Berra
Quoted 8 lines · John Robinson
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
I believe it won't work
mdadm has no way to know where the metadata is in the above case.Quoted 8 lines
you have to follow the painful procedure fail one drive (mdadm
/dev/mdX--fail/dev/sdY) remove it (mdadm/dev/mdX--remove/dev/sdY) replace drive with larger one
create larger partition
add drive again (mdadm/dev/mdX--add/dev/sdY) wait for it to resync...
...and repeat until all drives have been replacedYeuch. That'll take forever, because I'd need to rebuild 3 times. I see two other options:
(1) dd the drives as I said originally, and when the system can't find the array on the bigger discs, recreate the array using the same options the original array had
(2) degrade the existing array to two discs, replace the soft-failed disc with one of the new discs, temporarily make a filesystem on it, copy the data; replace the other two discs, make a new array on them with a missing drive, copy the data back; scrub the temporarily-made filesystem and add the drive to the array.Actually I think something along the lines of Majed B's suggestion - in this case hang a spare drive out the side of the chassis, and just copy twice - would be quicker than (2). But (1) should work, shouldn't it?
Cheers,
John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
Majed B. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines · Majed B.
You should never degrade the array to copy its contents. Its very risky. There might be bad sectors on one or more disks and if you don't have all disks at hand, you may not be able to rebuild the array on the new disk.
OK that's fair, you're right I don't want to risk losing data because I was using a degraded array.
Quoted 3 lines
As for using dd, as others have pointed, the metadata won't be in place and if you create a new array after using dd, it'll still require a resync and the data will be destroyed.
I don't see whay either it'd require a resync - although I didn't say it, I was planning to zero the rest of the larger drives and recreate with --assume-clean - nor why data would be destroyed if I didn't create with --assume-clean, after all the data will all be in the right place.
But I think on balance I'm going to save the contents of the original array to an extra drive, create a new array on the big discs, and copy the data back. It shouldn't take too much more time than copying the discs individually with dd, even though I'm copying the data twice rather than once, and it's both safer and gives me a free defrag.
Cheers,
John.
What I meant was: If you use dd to clone an old disk to a new one of a larger size, metadata isn't where it should be and the new disk won't be recognized as part of an array.
So if you do that to all disks, then attempt to create a new array on the new disks, it will cause a resync. And even if it didn't resync, data mapping may be incorrect.
For argument's sake, let's assume that cloning the disks would work. It means cloning 3 disks.
The other option that you presented, which I hope you avoid, is degrading the array and presenting the new disks. This would also require 3 resyncs.
The option where you copy the data off the old array then back to the new one consist of 2 copy operations only. May not be as fast as a dd operation, but still should consume less time than 3 resyncs.
Also, while copying the original data, if the filesystem has problems, you'll see them right away and probably identify which files are affected. I hope everything goes smooth for you, but in case such problems occur, do NOT run fsck!
Clone the filesystem first for best assurance of data safety.
Good luck!
Quoted 33 lines · John Robinson
Quoted 5 lines · Majed B.
You should never degrade the array to copy its contents. Its very risky. There might be bad sectors on one or more disks and if you don't have all disks at hand, you may not be able to rebuild the array on the new disk.
OK that's fair, you're right I don't want to risk losing data because I was using a degraded array.
Quoted 3 lines
As for using dd, as others have pointed, the metadata won't be in place and if you create a new array after using dd, it'll still require a resync and the data will be destroyed.
I don't see whay either it'd require a resync - although I didn't say it, I was planning to zero the rest of the larger drives and recreate with --assume-clean - nor why data would be destroyed if I didn't create with --assume-clean, after all the data will all be in the right place.
But I think on balance I'm going to save the contents of the original array to an extra drive, create a new array on the big discs, and copy the data back. It shouldn't take too much more time than copying the discs individually with dd, even though I'm copying the data twice rather than once, and it's both safer and gives me a free defrag.
Cheers,
John.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
Majed B. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi all,
Am 13.03.2010 19:22, schrieb Majed B.:
Quoted 3 lines
What I meant was: If you use dd to clone an old disk to a new one of a larger size, metadata isn't where it should be and the new disk won't be recognized as part of an array.
I mean to have read that he uses 0.9 metadata. mdadm finds the device size (i.e. partition size) and locates the metadata according to that. So if he has partitioned the drives containing the raid, the dd-way should work on one hand (if he clones the complete drive, not only the partitions). On the other hand he'd have a backup at hand if something goes terribly wrong. Or might I be mistaken now?
Quoted 3 lines
So if you do that to all disks, then attempt to create a new array on the new disks, it will cause a resync. And even if it didn't resync, data mapping may be incorrect.
I second that - recreating is only good on same size devices - but
again: if the partition is same size, it should work.
Quoted 3 lines
For argument's sake, let's assume that cloning the disks would work. It means cloning 3 disks.
Yes, using sgp_dd this works very well and quickly. And you don't have problems arising from broken read-sectors being skipped. (yes, I know there is conv=noerror,sync; but I had my problems with that...)
Quoted 3 lines
The other option that you presented, which I hope you avoid, is degrading the array and presenting the new disks. This would also require 3 resyncs.
Yes, that is a bad idea.
Quoted 3 lines
The option where you copy the data off the old array then back to the new one consist of 2 copy operations only. May not be as fast as a dd operation, but still should consume less time than 3 resyncs.
right on one hand. the other hand is: on my laboratory computer (which is used for raid-recoveries and such) 4 parallel sgp_dd operations work at about 90MB/s (if the drives are quick enough, starting with 115MB/s, and sinking speed as the r/w-heads move outside). But one copy only reaches 50MB/s. Multiple parallel copies do not give any speedup.
Quoted 5 lines
Also, while copying the original data, if the filesystem has problems, you'll see them right away and probably identify which files are affected. I hope everything goes smooth for you, but in case such problems occur, do NOT run fsck!
second that - if fsck is needed, create a dd-backup to run it on.
Quoted 4 lines
Clone the filesystem first for best assurance of data safety.
Good luck!
[...]
Also my best wishes for the operation...
Stefan
A message that is not in the archive (sent off-list or not yet received): [email protected]
Forgot to send to the list...
Quoted 21 lines · Beolach
Quoted 8 lines · John Robinson
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
I have no idea if this would actually work, but maybe try partitioning the new, larger drives so the partitions are identical in size to the old discs, copying the old discs to the new partitions, starting the array with the new partitions, resizing the new partitions, and then running --grow. If mdadm can re-check the size of the member devices of a running array at --grow time, this might work, but if it only checks member device sizes at creation or assemble time, then it probably won't.
Quoted 3 lines
Good luck,
Conway S. Smith
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Beolach,
That involves double the work!
Quoted 32 lines · Beolach
Forgot to send to the list...
Quoted 21 lines · Beolach
Quoted 8 lines · John Robinson
Before I have to go to a customer's premises and try it, I wanted to ask if anyone had already done anything like this: I have a CentOS 5 system with 0.90 metadata and type fd partitions with RAID-5 over 3 discs. I want to grow it by replacing the drives. If I dd the small partitions onto larger ones on the new discs, then reboot, will the system recognise the RAID-5 - which now has its metadata somewhere in the middle of the paritions not at the end - so that I can then --grow?
I have no idea if this would actually work, but maybe try partitioning the new, larger drives so the partitions are identical in size to the old discs, copying the old discs to the new partitions, starting the array with the new partitions, resizing the new partitions, and then running --grow. If mdadm can re-check the size of the member devices of a running array at --grow time, this might work, but if it only checks member device sizes at creation or assemble time, then it probably won't.
Quoted 3 lines
Good luck,
Conway S. Smith--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
Majed B. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html