block level vs. file level
Greetings,
There seem to be two ways of doing software RAID, and I have had trouble finding information on this.
First, the raid is done with partitions, for example /dev/sda1 and /dev/sdb1 are partitioned as Linux type and the mirroring is done then between the two partitions. The writing takes place on a filesystem level, and the partition table is not actually mirrored, because it's not on /dev/sda1 or /dev/sdb1.
The hardware raid does the mirroring on the block level, so it's actually /dev/sda mirroring /dev/sdb - the whole drive, and not partitions. There is a way to set this up on software raid. It takes more configuration tweaking, but the mirroring then includes the partition table as well. This way, if a drive fails, one can replace it without pre-partitioning it.
This also raises another point, which is relevant for both cases - same exact models of hard disks have different number of cylinders, so if a RAID partition is created on a larger drive it cannot be mirrored to a smaller drive.
Does anyone have any experience with this?
A.
Quoted 6 lines · it
The hardware raid does the mirroring on the block level, so it's actually
/dev/sdamirroring/dev/sdb- the whole drive, and not partitions. There is a way to set this up on software raid. It takes more configuration tweaking, but the mirroring then includes the partition table as well. This way, if a drive fails, one can replace it without pre-partitioning it.
That can be less flexible though. If I have say 4 drives then I quite often want small /boot, / and swap under RAID-1 then the rest as a single large partition in RAID-5, -6 or -10 as an LVM PV.
Quoted 4 lines
This also raises another point, which is relevant for both cases - same exact models of hard disks have different number of cylinders, so if a RAID partition is created on a larger drive it cannot be mirrored to a smaller drive.
Same exact models don't usually have different block counts, but certainly if you replace a dead drive with a different one of the same advertised capacity you can end up getting one slightly smaller.
Quoted 1 line
Does anyone have any experience with this?
Yes, and it's a pain, but if you have to deal with it I think the wealth of options in md leaves you better able to handle it than with hardware RAID. Here's something that happened to me:
http://strugglers.net/wiki/becks.strugglers.net
Signature · 2 lines
http://strugglers.net/wiki/Xen_hosting -- A Xen VPS hosting hobby Encrypted mail welcome - keyid 0x604DE5DB
Quoted 8 lines
Yes, and it's a pain, but if you have to deal with it I think the wealth of options in md leaves you better able to handle it than with hardware RAID. Here's something that happened to me:
Ouch.
How does hardware raid deal with this? Does it?
Thanks,
A.
Quoted 36 lines · Andy Smith
Quoted 3 lines · it
The hardware raid does the mirroring on the block level, so it's actually
/dev/sdamirroring/dev/sdb- the whole drive, and not partitions. There is a way to set this up on software raid. It takes more configuration tweaking, but the mirroring then includes the partition table as well. This way, if a drive fails, one can replace it without pre-partitioning it.That can be less flexible though. If I have say 4 drives then I quite often want small /boot, / and swap under RAID-1 then the rest as a single large partition in RAID-5, -6 or -10 as an LVM PV.
Quoted 3 lines
This also raises another point, which is relevant for both cases - same exact models of hard disks have different number of cylinders, so if a RAID partition is created on a larger drive it cannot be mirrored to a smaller drive.
Same exact models don't usually have different block counts, but certainly if you replace a dead drive with a different one of the same advertised capacity you can end up getting one slightly smaller.
Quoted 3 lines
Does anyone have any experience with this?
Yes, and it's a pain, but if you have to deal with it I think the wealth of options in md leaves you better able to handle it than with hardware RAID. Here's something that happened to me:
Quoted 3 lines · it
Ouch.
How does hardware raid deal with this? Does it?
Hardware RAID controllers deal with this by rounding the size of participant devices down to nearest GB, on the assumption that no drive manufacturers would have the guts to actually sell eg. a 250 GB drive with less than exactly 250.000.000.000 bytes of space on it.
(It would be nice if the various flavors of Linux fdisk had an option to do this. It would be very nice if anaconda had an option to do this.)
Quoted 9 lines · Molle Bestefich
Quoted 5 lines · it
Ouch.
How does hardware raid deal with this? Does it?
Quoted 9 lines
Hardware RAID controllers deal with this by rounding the size of participant devices down to nearest GB, on the assumption that no drive manufacturers would have the guts to actually sell eg. a 250 GB drive with less than exactly 250.000.000.000 bytes of space on it.
(It would be nice if the various flavors of Linux fdisk had an option to do this. It would be very nice if anaconda had an option to do this.)
I guess if you care you specify the size of the partition instead of "use it all." I use fdisk usually, cfdisk when installing, both let me set size, fdisk let's me set starting track and even play with the partition table's idea of geometry. What kind of an option did you have in mind?
Signature · 3 lines
bill davidsen <[email protected]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979
Quoted 15 lines · Bill Davidsen
Quoted 5 lines · Molle Bestefich
Quoted 3 lines · it
Ouch.
How does hardware raid deal with this? Does it?
Quoted 8 lines
Hardware RAID controllers deal with this by rounding the size of participant devices down to nearest GB, on the assumption that no drive manufacturers would have the guts to actually sell eg. a 250 GB drive with less than exactly 250.000.000.000 bytes of space on it.
(It would be nice if the various flavors of Linux fdisk had an option to do this. It would be very nice if anaconda had an option to do this.)
Quoted 5 lines
I guess if you care you specify the size of the partition instead of "use it all." I use fdisk usually, cfdisk when installing, both let me set size, fdisk let's me set starting track and even play with the partition table's idea of geometry. What kind of an option did you have in mind?
I don't know. Examples good enough?
a.) "Do not use space beyond highest GB"
b.) "Do not use last cylinder"
Help texts could be:
a.) "Helps ensure that you can replace fx. a 300GB drive from one manufacturer with a 300GB from another."
b.) "Leave an area that Windows might use for disk metadata alone."
Quoted 28 lines · Andy Smith
Quoted 3 lines · it
The hardware raid does the mirroring on the block level, so it's actually
/dev/sdamirroring/dev/sdb- the whole drive, and not partitions. There is a way to set this up on software raid. It takes more configuration tweaking, but the mirroring then includes the partition table as well. This way, if a drive fails, one can replace it without pre-partitioning it.That can be less flexible though. If I have say 4 drives then I quite often want small /boot, / and swap under RAID-1 then the rest as a single large partition in RAID-5, -6 or -10 as an LVM PV.
Quoted 3 lines
This also raises another point, which is relevant for both cases - same exact models of hard disks have different number of cylinders, so if a RAID partition is created on a larger drive it cannot be mirrored to a smaller drive.
Same exact models don't usually have different block counts, but certainly if you replace a dead drive with a different one of the same advertised capacity you can end up getting one slightly smaller.
Quoted 3 lines
Does anyone have any experience with this?
Quoted 8 lines
Yes, and it's a pain, but if you have to deal with it I think the wealth of options in md leaves you better able to handle it than with hardware RAID. Here's something that happened to me:
I don't quite understand your problem with fdisk, you can specify starting sector and then either ending sector or size, and call it any partition number you want. So I don't see why any other tool is required and what "at the end" meant in you tale.
Glad you got it working, but you may have used more solution than you have problem.
Signature · 3 lines
bill davidsen <[email protected]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979
Quoted 3 lines
This also raises another point, which is relevant for both cases - same exact models of hard disks have different number of cylinders, so if a RAID partition is created on a larger drive it cannot be mirrored to a smaller drive.
I have a RAID5 with 5 250G drives, but some are 251 GiB (maxtors), some
are 250.059 GiB (seagate)... say, if I started with 5 Seagates, I could later replace one of them with a Maxtor, but not the other way around, as the Seagate are just a tiny bit smaller.
cfdisk says :
sdb1 250994,42
sdc1 250056,74
I suggest, when using software raid, to create partitions that are, say, 100 megabytes or even a gigabyte smaller than the size of the drive. You lose a bit of space, but if you ever need to change one, you won't feel stupid with a brand new drive that you can't use because it's a few sectors too short.
Quoted 5 lines · PFC
I suggest, when using software raid, to create partitions that are, say, 100 megabytes or even a gigabyte smaller than the size of the drive.
You lose a bit of space, but if you ever need to change one, you won't feel stupid with a brand new drive that you can't use because it's a few sectors too short.
After my previous experience what I tend to do now is set aside about 2GB on each disk to use as components of a RAID-0 that I use for scratch space (/tmp or whatever, anything that I don't care about losing) while the machine is running.
That way if I end up by bad luck getting a slightly smaller replacement drive then I can just do away with or shrink its RAID-0 component while keeping the other partitions the same, yet the space is not *totally* wasted.
Signature · 2 lines
http://strugglers.net/wiki/Xen_hosting -- A Xen VPS hosting hobby Encrypted mail welcome - keyid 0x604DE5DB