RAID1 On 3 Drives
Quoted 4 lines · Carlos Mennens
I was told by my distributions Wiki page that Grub doesn't support RAID 5 or RAID 6 so I would need to create a volume with three disks and set the level to RAID 1:
Quoted 8 lines
# mdadm --create /dev/md1 --level=1 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1 --spare-devices=1 /dev/sdd1Is this possible to mirror three identical drive partitions? I was talking to some co-workers and was told that I could only pair two drives on RAID 1.
Can anyone please help me understand this?
/boot must be RAID1 (or a normal partition), everything else can be RAIDwhatever. Grub has no problems reading /boot from a mirrors.
It is possible to make RAID1 to use more than two drives and keep the all active. In your case it is a matter of "believe" whether a spare is better or actually using all four actively in your raid, I prefer the latter. I have heavily tested more than two only with files via loopback yet, two drive, three drives, four drives and more. It does work, the testfile generated out of urandom kept the correct hash value throughout all my sick testings. The quick test with three SATA drives worked too, but it was just for the fun to see whether grub pukes or works (works if you don't forget the grub setup for drive 2, 3 etc).
My plan is: Use RAID1 with 4 drives (quad-mirroring) for /boot or / and raid10 or raid5 for the data upon my next linux server upgrade.
Joachim Otahal
I am glad I asked! Thanks for all the info. Now my question next is if I am creating a 3 disk raid with a 1 disk spare, do I need to use any special 'mdadm' parameters besides what u see below for /boot (Grub)?
# mdadm --create /dev/md0 --level=1 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1 --spare-devices=1 /dev/sdd1
Anything else I should add to the above command for my /boot partition?
Quoted 39 lines · Joachim Otahal
Quoted 4 lines · Carlos Mennens
I was told by my distributions Wiki page that Grub doesn't support RAID 5 or RAID 6 so I would need to create a volume with three disks and set the level to RAID 1:
Quoted 8 lines
# mdadm --create /dev/md1 --level=1 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1 --spare-devices=1 /dev/sdd1Is this possible to mirror three identical drive partitions? I was talking to some co-workers and was told that I could only pair two drives on RAID 1.
Can anyone please help me understand this?
/boot must be RAID1 (or a normal partition), everything else can be RAIDwhatever. Grub has no problems reading /boot from a mirrors.
It is possible to make RAID1 to use more than two drives and keep the all active. In your case it is a matter of "believe" whether a spare is better or actually using all four actively in your raid, I prefer the latter. I have heavily tested more than two only with files via loopback yet, two drive, three drives, four drives and more. It does work, the testfile generated out of urandom kept the correct hash value throughout all my sick testings.
The quick test with three SATA drives worked too, but it was just for the fun to see whether grub pukes or works (works if you don't forget the grub setup for drive 2, 3 etc).
My plan is: Use RAID1 with 4 drives (quad-mirroring) for /boot or / and raid10 or raid5 for the data upon my next linux server upgrade.Joachim Otahal
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 1 line
Sent from my mobile device
Quoted 9 lines · Carlos Mennens
I am glad I asked! Thanks for all the info. Now my question next is if I am creating a 3 disk raid with a 1 disk spare, do I need to use any special 'mdadm' parameters besides what u see below for /boot (Grub)?
# mdadm --create /dev/md0 --level=1 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1 --spare-devices=1 /dev/sdd1Anything else I should add to the above command for my /boot partition?
Having a spare on raid-1 is fairly pointless, it hurts performance and buys you nothing. Having one more copy of the data built and ready serves you better.
Signature · 3 lines
Bill Davidsen <[email protected]> "We can't solve today's problems by using the same thinking we used in creating them." - Einstein
Quoted 3 lines · Bill Davidsen
Having a spare on raid-1 is fairly pointless, it hurts performance and buys you nothing. Having one more copy of the data built and ready serves you better.
Can you explain this as I find this interesting. How does having a /boot partition on 3 drives with 1 spare hurt performance? Are you saying that I would get better drive performance if I had all 4 disk partitions active members of my RAID1 /boot? I just don't understand how the 4th disk doing nothing but acting as a spare would hinder performance.
Secondly, if the above statement also applies to my / partition? Would you suggest using all 4 drives as active partitions in a RAID5 array too? If I have a 3 disk RAID5 and one hot spare, do you think I would get less performance value for my configuration?
Thanks for the clarification!
Carlos,
Choosing your RAID setup depends on the purpose of your machine. Is it a web server, file server, archive, mail, video storage, ...etc.?
/boot is only used during boot up. Having a spare on stand by or not does not affect performance, except in the case a disk dies; in that case, the hot spare is engaged and becomes an active disk at which the array starts to resync the data to this new disk. During resyncing, performance will be degraded.
Including all disks in the array (no spare) means data is being written in parallel to all disks at all times. You'd still have to replace a dead disk and add it to the array and it will still have to resync.
As for your RAID5 question: I think if your usage of the server is write-mostly, you may find it to have better performance with 3 disks rather 4. If it's read-mostly, then 4 disks should perform better.
If you have physical access to the machine, try both cases. Setting them up won't take more than 5-10 minutes. Benchmarking wouldn't take more than 15 minutes in each setup.
Remember, there are parameters to fine-tune: NCQ, read-ahead, noatime, nodiratime, chunksize, ...etc.
Quoted 23 lines · Carlos Mennens
Quoted 3 lines · Bill Davidsen
Having a spare on raid-1 is fairly pointless, it hurts performance and buys you nothing. Having one more copy of the data built and ready serves you better.
Can you explain this as I find this interesting. How does having a /boot partition on 3 drives with 1 spare hurt performance? Are you saying that I would get better drive performance if I had all 4 disk partitions active members of my RAID1 /boot? I just don't understand how the 4th disk doing nothing but acting as a spare would hinder performance.
Secondly, if the above statement also applies to my / partition? Would you suggest using all 4 drives as active partitions in a RAID5 array too? If I have a 3 disk RAID5 and one hot spare, do you think I would get less performance value for my configuration?
Thanks for the clarification!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
Majed B. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines · Majed B.
Carlos,
Choosing your RAID setup depends on the purpose of your machine. Is it a web server, file server, archive, mail, video storage, ...etc.
Basically this is my everyday workstation that I keep everything on. It's not hosting Apache or MySQL or anything like that. Just my everyday workstation.
Quoted 5 lines
/boot is only used during boot up. Having a spare on stand by or not does not affect performance, except in the case a disk dies; in that case, the hot spare is engaged and becomes an active disk at which the array starts to resync the data to this new disk. During resyncing, performance will be degraded.
Oh, I was wondering why Bill noted previously that a spare would hurt performance but based on your info above, this is only if a drive fails and then the spare starts to synchronize. I hope that is what Bill meant.
Quoted 10 lines
As for your RAID5 question: I think if your usage of the server is write-mostly, you may find it to have better performance with 3 disks rather 4. If it's read-mostly, then 4 disks should perform better.
If you have physical access to the machine, try both cases. Setting them up won't take more than 5-10 minutes. Benchmarking wouldn't take more than 15 minutes in each setup.
Remember, there are parameters to fine-tune: NCQ, read-ahead, noatime, nodiratime, chunksize, ...etc.
I do have access to the machine as it's my new Desktop PC I am building today at some point. I just wanted to get some expert advise on how I should proceed. For now I am going to use all 4 disks with no spares.
I just don't know enough about fine tuning and how it could benefit or hinder disk performance for my setup so I guess I will omit them until I can understand what to use for what function. If you guys have any suggestions for what parameters to use for /boot, /, or swap...please feel free to chime in.
Thanks all!
100-200 MB is good enough for the /boot filesystem.
How much swap to use depends on how much RAM you have and how much RAM the applications you use demand.
If you have 4GB RAM or more, and most of your usage is about browsing the Internet, documents, watching videos, ..etc., then 2GB swap is more than enough even though I think you'll never use it. In case you ever need more than that, you could also create a swap-file -- whenever in need, create the file with the size you need on your array, activate the file as swap and that's it! When done with it, deactivated and delete the file to reclaim the capacity it once occupied.
Personally, I keep my /home on its own partition. This guarantees that whatever system upgrades or distribution installs occur, /home won't be affected nor touched, and can be seen by multiple Linux distros if you're into multi-booting.
I have a workstation with 4x 320GB disks with my root filesystem on it as RAID5. I have to say that it wasn't a good judgment going that way. One time one of the disks failed and the system refused to boot in degraded mode, so I had to boot from a CD, resync the array and then boot normally.
That event got me thinking: Why don't I put / on its own disk and /home be on the RAID5 array. I could copy / to the array as a backup, should the OS disk crash; put a new disk, make it bootable, install grub on it, then using a LiveCD I could boot and access my RAID5 array and copy the OS files back.
It's OK to have all disks running on the array without a spare. Just make sure you configure smartd and have it run tests on a daily basis (short tests) and a weekly basis (long tests) and to have it email you if problems occur.
For RAID performance enhancements, the page seem to have been removed but a cached version of it is available:
http://74.125.77.132/search?q=cache:5f7lyJQGL78J:www.linuxfoundation.org/collaborate/workgroups/linux-raid/performance+http://linux-raid.osdl.org/index.php/Performance&cd=1&hl=en&ct=clnk
For RAID benchmarks:
https://turing.phas.ubc.ca/mediawiki/index.php/RAID_benchmarks
http://peterkieser.com/2009/11/27/linux-raid-mdraid-raid6-raid5-benchmarks/
I hope these help.
Quoted 49 lines · Carlos Mennens
Quoted 4 lines · Majed B.
Carlos,
Choosing your RAID setup depends on the purpose of your machine. Is it a web server, file server, archive, mail, video storage, ...etc.
Basically this is my everyday workstation that I keep everything on. It's not hosting Apache or MySQL or anything like that. Just my everyday workstation.
Quoted 5 lines
/boot is only used during boot up. Having a spare on stand by or not does not affect performance, except in the case a disk dies; in that case, the hot spare is engaged and becomes an active disk at which the array starts to resync the data to this new disk. During resyncing, performance will be degraded.
Oh, I was wondering why Bill noted previously that a spare would hurt performance but based on your info above, this is only if a drive fails and then the spare starts to synchronize. I hope that is what Bill meant.
Quoted 10 lines
As for your RAID5 question: I think if your usage of the server is write-mostly, you may find it to have better performance with 3 disks rather 4. If it's read-mostly, then 4 disks should perform better.
If you have physical access to the machine, try both cases. Setting them up won't take more than 5-10 minutes. Benchmarking wouldn't take more than 15 minutes in each setup.
Remember, there are parameters to fine-tune: NCQ, read-ahead, noatime, nodiratime, chunksize, ...etc.
I do have access to the machine as it's my new Desktop PC I am building today at some point. I just wanted to get some expert advise on how I should proceed. For now I am going to use all 4 disks with no spares.
I just don't know enough about fine tuning and how it could benefit or hinder disk performance for my setup so I guess I will omit them until I can understand what to use for what function. If you guys have any suggestions for what parameters to use for /boot, /, or swap...please feel free to chime in.
Thanks all!
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
Majed B. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 29 lines · Majed B.
100-200 MB is good enough for the /boot filesystem.
How much swap to use depends on how much RAM you have and how much RAM the applications you use demand.
If you have 4GB RAM or more, and most of your usage is about browsing the Internet, documents, watching videos, ..etc., then 2GB swap is more than enough even though I think you'll never use it. In case you ever need more than that, you could also create a swap-file -- whenever in need, create the file with the size you need on your array, activate the file as swap and that's it! When done with it, deactivated and delete the file to reclaim the capacity it once occupied.Personally, I keep my /home on its own partition. This guarantees that whatever system upgrades or distribution installs occur, /home won't be affected nor touched, and can be seen by multiple Linux distros if you're into multi-booting.
I have a workstation with 4x 320GB disks with my root filesystem on it as RAID5. I have to say that it wasn't a good judgment going that way. One time one of the disks failed and the system refused to boot in degraded mode, so I had to boot from a CD, resync the array and then boot normally.
That event got me thinking: Why don't I put / on its own disk and /home be on the RAID5 array. I could copy / to the array as a backup, should the OS disk crash; put a new disk, make it bootable, install grub on it, then using a LiveCD I could boot and access my RAID5 array and copy the OS files back.
Quoted 8 lines
It's OK to have all disks running on the array without a spare. Just make sure you configure smartd and have it run tests on a daily basis (short tests) and a weekly basis (long tests) and to have it email you if problems occur.
For RAID performance enhancements, the page seem to have been removed but a cached version of it is available:
http://74.125.77.132/search?q=cache:5f7lyJQGL78J:www.linuxfoundation.org/collaborate/workgroups/linux-raid/performance+http://linux-raid.osdl.org/index.php/Performance&cd=1&hl=en&ct=clnk
Our wiki moved ... see
http://raid.wiki.kernel.org/index.php/Performance
Quoted 3 lines
For RAID benchmarks:
https://turing.phas.ubc.ca/mediawiki/index.php/RAID_benchmarks
does not involve Linux MD RAID. Linux MD RAID is generally faster then HW RAID.
Best regards
keld
Quoted 7 lines · Carlos Mennens
Quoted 4 lines · Bill Davidsen
Having a spare on raid-1 is fairly pointless, it hurts performance and buys you nothing. Having one more copy of the data built and ready serves you better.
Quoted 8 lines
Can you explain this as I find this interesting. How does having a /boot partition on 3 drives with 1 spare hurt performance? Are you saying that I would get better drive performance if I had all 4 disk partitions active members of my RAID1 /boot? I just don't understand how the 4th disk doing nothing but acting as a spare would hinder performance.
In RAID-1 any drive which is a member of the array is already a spare, they are all the same. So if a drive failed you would just stop using it. Having a spare makes no sense, since the spare would be rebuilt using the same data it would have contained as a member of the array. Note that while the "spare" is being rebuilt the array will be quite busy and response will be poor, not an issue if /boot is the only thing on the drives, but more of an issue if the array just uses partitions and the thrashing of a spare rebuild slows other arrays on the same drives.
Finally, RAID-1 supposedly will read from an unbusy drive if there are multiple reads against the array, allowing overlapping of seek and transfer. That's only an issue for /boot if you are trying for a ten second boot time, but it applies to all RAID-1 arrays.
Quoted 5 lines
Secondly, if the above statement also applies to my / partition? Would you suggest using all 4 drives as active partitions in a RAID5 array too? If I have a 3 disk RAID5 and one hot spare, do you think I would get less performance value for my configuration?
No, because the drives are not all the same, so you can't keep a copy of just the drive ready to fail next. For RAID-[456&10] you want at least one spare for reliability and fast return to full performance.
Ask again if I didn't give enough detail.
Signature · 3 lines
Bill Davidsen <[email protected]> "We can't solve today's problems by using the same thinking we used in creating them." - Einstein