Creating a RAID10 (near) for use in a CentOS 6 system
I come today, hat in hand, for some suggestions on a new install with RAID10.
I wish to install CentOS 6 on a new system.
I am using 4 identical drives, and do not see any need to partition them. They are 2TB Hitachi Ultrastar (enterprise)drives.
I wish to use RAID10, near
Questions:
Syntax: Creating near vs far. Is there a specific syntax to use?
I *think* this is what I need:
mdadm --create /dev/md0 --run --level=10 --chunk=4 --raid-devices=4 /dev/sda /dev/sdb /dev/sdc /dev/sdd
Next question is about mdadm versions and kernels. CentOS 6 currently uses 2.6.32-358.2.1.el6 Is there anything in that I should beware of?
OTOH, one may get newer kernels by enabling elrepo
http://elrepo.org/tiki/tiki-index.php
This will get me up to:
kernel-lt-3.0.70-1.el6.elrepo
kernel-ml-3.8.4-1.el6.elrepoIs that desirable in this case?
Any thoughts and suggestions are most welcome..
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Further to this, can anyone comment on creating partitions on the RAID10 device, versus making separate partitions on the devices, for installation of the OS?
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Quoted 2 lines · maurice
Further to this, can anyone comment on creating partitions on the RAID10 device, versus making separate partitions on the devices, for installation of the OS?
I'd partition the array, rather than make separate raid10 arrays from partitions. It's certainly easier to manage if there are issues. If a drive dies, you replace that drive, whereas if you partition the drives you have to partition it exactly right, and then add the partitions as replacements for each array you've created. I think that's a hassle.
The advantage of partitioning the drives with one partition (I'd use GPT for this), is that fdisk/gdisk/parted and other utils will see each drive as being partitioned (for linux software raid) rather than appearing as free space.
Chris Murphy
Quoted 10 lines · Chris Murphy
Quoted 2 lines · maurice
Further to this, can anyone comment on creating partitions on the RAID10 device, versus making separate partitions on the devices, for installation of the OS?
I'd partition the array, rather than make separate raid10 arrays from partitions. It's certainly easier to manage if there are issues. If a drive dies, you replace that drive, whereas if you partition the drives you have to partition it exactly right, and then add the partitions as replacements for each array you've created. I think that's a hassle.
The advantage of partitioning the drives with one partition (I'd use GPT for this), is that fdisk/gdisk/parted and other utils will see each drive as being partitioned (for linux software raid) rather than appearing as free space.
Chris Murphy
Thanks Chris.
Is there any problem with installing CentOS directly on an array in RAID10, as opposed to on to "regular" partitions?
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Quoted 2 lines · maurice
Is there any problem with installing CentOS directly on an array in RAID10, as opposed to on to "regular" partitions?
Oh that's a totally different scenario. I thought you wanted the raid10 array for /home or /data or whatever. That's pretty messy, actually, even on Fedora 18/19 which has the benefit of a new anaconda and GRUB2. GRUB2 definitely can assemble md raid10, and read most any file system, find /boot, and load the kernel and initramfs. That's not something I think GRUB Legacy can do, which is what CentOS 6 is using.
I don't know off hand if partitionable arrays are supported by GRUB2. I haven't tested this. And even if GRUB2 does recognize the partitioned array, the initramfs I think needs kpartx built into it, in order for the kernel to have the ability to find and then mount rootfs. And even if all of that it true, I don't think anaconda new or old will let you install the system the way you're thinking. Fedora 18/19's anaconda does not see arrays, it sees actual physical drives. And it wants to create partitions on those devices, not partitions on the array. It might be possible to do what you want via kickstart.
I have successfully created a bootable raid 10, where /boot, /, /home were all on a four disk raid10. But that's with Fedora 18, which uses GRUB2.
I'm unsure of your use case, but I'm thinking you may be better off with a small SSD for the boot, swap and system disk, and then use a separate raid10 array for /home, /opt, /data, whatever else you want. Otherwise it's a bit of a rabbit hole.
Chris Murphy
Quoted 6 lines · Chris Murphy
..Oh that's a totally different scenario.
I thought you wanted the raid10 array for /home or /data or whatever.
That's pretty messy, actually, even on Fedora 18/19 which has the benefit of a new anaconda and GRUB2. GRUB2 definitely can assemble md raid10, and read most any file system, find /boot, and load the kernel and initramfs. That's not something I think GRUB Legacy can do, which is what CentOS 6 is using.
Quoted 6 lines
..
I'm unsure of your use case, but I'm thinking you may be better off with a small SSD for the boot, swap and system disk, and then use a separate raid10 array for /home, /opt, /data, whatever else you want.
Otherwise it's a bit of a rabbit hole.
Fair enough then.
What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10? Do you think that the CentOS installer and GRUB would have any issues with that layout?
Thanks Chris.
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Quoted 2 lines · maurice
What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?
What's the advantage of this compared to a separate drive? In your proposed scenario, if the /boot drive dies, you have an unbootable system anyway.
Quoted 1 line
Do you think that the CentOS installer and GRUB would have any issues with that layout?
No. It'll be fine with that. It does partition drives though, not arrays.
Chris Murphy
Quoted 3 lines · Chris Murphy
Quoted 1 line · maurice
What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?
What's the advantage of this compared to a separate drive? In your proposed scenario, if the /boot drive dies, you have an unbootable system anyway.
Firstly, there would be 4 redundant copies of the boot. Easy enough to accomplish. If any one goes down, easy to pick one of the others. Secondly, there is only room for 4 drives in this chassis
Thirdly, cost
Now, back to my original question:
Making the RAID10, near.
In reading the man pages, it seems near 2 is the default option for create?
Is there any reason to be wary of the stock CentOS 6 2.6.32 kernel?
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Quoted 5 lines · Maurice Hilarius
Quoted 3 lines · Chris Murphy
Quoted 1 line · maurice
What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?
What's the advantage of this compared to a separate drive? In your proposed scenario, if the /boot drive dies, you have an unbootable system anyway.
Firstly, there would be 4 redundant copies of the boot.
AFAIK you'll have to make those 4 copies yourself, the installer won't do it unless you use kickstart, and it won't partition the drives correctly for it. And you'll have to keep them sync'd. And you'll have to manually install grub to all four disks, as the installer will only install to one disk.
Quoted 1 line
Secondly, there is only room for 4 drives in this chassis.
USB stick, ext2 to avoid journal wear. And only put boot on it. rootfs can go on the array. And you can setup a swapfile to go in rootfs instead of a swap partition.
Quoted 4 lines
In reading the man pages, it seems near 2 is the default option for create?
Is there any reason to be wary of the stock CentOS 6 2.6.32 kernel?
No idea. I'd probably use a newer one from EL repo, and keep the 6.4 kernel handy as a fallback which I think is now 2.6.32.358.
Chris Murphy
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · maurice
Quoted 1 line · Chris Murphy
..No. It'll be fine with that. It does partition drives though, not arrays. Chris Murphy
Are you saying that it could not create the partitions on the RAID?
The only partition layout I've seen any version of anaconda create is partitioning physical devices, and then adding those partitions as member devices in an array. Maybe partitionable arrays are possible with kickstart.
Quoted 1 line
So, would I have to prepare the RAID first on another machine?
I haven't tried doing this with the old anaconda, so I don't know if it will see a pre-created array that it doesn't create itself; let alone if it will honor the partitions in the array.
Chris Murphy
Quoted 5 lines · maurice
What about partitioning the drives first, installing a smallish partition for boot, then a second much larger partition on each for the RAID10?
Do you think that the CentOS installer and GRUB would have any issues with that layout?
This is how I have a Debian server set up.
The system is on 6 SSD's. Each ssd has a small partition and a big partition. All the small partitions are set up in a 6 way RAID1 as ext2 for /boot, and the big partitions are in a RAID10,n2. This RAID10 is then partitioned with GPT to provide / /home /opt.
Can't comment on the Centos part, but the method is sound.
Quoted 8 lines · Brad Campbell
..
The system is on 6 SSD's. Each ssd has a small partition and a big partition.
All the small partitions are set up in a 6 way RAID1 as ext2 for /boot, and the big partitions are in a RAID10,n2. This RAID10 is then partitioned with GPT to provide / /home /opt.Can't comment on the Centos part, but the method is sound.
Ah, good idea on the small RAID1. Keeps it all synchronized. Nice.
Thank you!
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
If you have 2x4TB and you want to setup the lot for booting and data, I'd recommend doing like the NAS boxes I've seen (well, only QNAP 410 IIRC). On that one, they chose to create partitions for / and swap (and perhaps /boot), and equal-ish partitions for those used for data. On those, you can place a RAID-10, or perhaps two RAID-1s and use LVM to stripe them. The latter is far more flexible, as RAID-10 lacks most of the flexibility other RAID leverls have.
If you don't need the extra IOPS from the RAID-10, I'd recommend using RAID-6 instead for even futher safety and flexibility (the ability to change to RAID-5 in need of more space, easier to grow etc).
roy
----- Opprinnelig melding -----
Quoted 43 lines
I come today, hat in hand, for some suggestions on a new install with RAID10.
I wish to install CentOS 6 on a new system.
I am using 4 identical drives, and do not see any need to partition them.
They are 2TB Hitachi Ultrastar (enterprise)drives.I wish to use RAID10, near
Questions:
Syntax: Creating near vs far. Is there a specific syntax to use?I *think* this is what I need:
mdadm --create/dev/md0--run --level=10 --chunk=4 --raid-devices=4/dev/sda/dev/sdb/dev/sdc/dev/sddNext question is about mdadm versions and kernels. CentOS 6 currently uses 2.6.32-358.2.1.el6 Is there anything in that I should beware of?
OTOH, one may get newer kernels by enabling elrepo
http://elrepo.org/tiki/tiki-index.php
This will get me up to:
kernel-lt-3.0.70-1.el6.elrepo
kernel-ml-3.8.4-1.el6.elrepoIs that desirable in this case?
Any thoughts and suggestions are most welcome..
--
Cheers,
Maurice Hilarius
eMail: /[email protected]/
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 15 lines
Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 98013356 [email protected] http://blogg.karlsbakk.net/ GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · Roy Sigurd Karlsbakk
If you have 2x4TB and you want to setup the lot for booting and data, I'd recommend doing like the NAS boxes I've seen (well, only QNAP 410 IIRC). On that one, they chose to create partitions for / and swap (and perhaps /boot), and equal-ish partitions for those used for data. On those, you can place a RAID-10, or perhaps two RAID-1s and use LVM to stripe them. The latter is far more flexible, as RAID-10 lacks most of the flexibility other RAID leverls have.
I have 4 x 2TB, actually.
I am in the middle of this install now
CentOS Anaconda installer seems to be happy with me creating a 4 disk RAID1 (thanks for the suggestion, Brad C!) for /boot. And I created a 4 disk RAID10 for swap, and another for /
Quoted 1 line
If you don't need the extra IOPS from the RAID-10, I'd recommend using RAID-6 instead for even futher safety and flexibility (the ability to change to RAID-5 in need of more space, easier to grow etc).
I want the performance.
Yes, RAID6 is an option, too, but R10 should substantially outperform it, and makes a lot less CPU load for the trouble. As far as growth goes, only 4 bays available , so not a factor in this case, at least.
Once done I will run some tests and confirm that the performance is good ( or not).
I still have heard nothing about any problems with the versions of kernels and mdadm in this distro version. So are there none??
Anyone?
Signature · 3 lines
Cheers, Maurice Hilarius eMail: /[email protected]/
Quoted 10 lines · Roy Sigurd Karlsbakk
Quoted 7 lines
If you have 2x4TB and you want to setup the lot for booting and data, I'd recommend doing like the NAS boxes I've seen (well, only QNAP 410 IIRC). On that one, they chose to create partitions for / and swap (and perhaps /boot), and equal-ish partitions for those used for data. On those, you can place a RAID-10, or perhaps two RAID-1s and use LVM to stripe them. The latter is far more flexible, as RAID-10 lacks most of the flexibility other RAID leverls have.
I have 4 x 2TB, actually.
Erm, that's what I meant. I just wouldn't recommend using RAID-10, since it isn't very flexible. LVM on top of multiple mirrors is better.
Vennlige hilsener / Best regards
roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
[email protected]
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. --
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html