Skip to contentSkip to conversation

Will LVM2 ever be able to do striped mirrors "raid 10"?

linux-lvm John Hughes 12 participants 17 messages Apr 8, 2009 – Apr 20, 2009

lore
Share & tools
lore
#0 lore raw
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

#1 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

Why you need RAID 10 at all ?

Usualy because the need of performance and additional redundancy/fault tollerance.
So if u need performance a "Software RAID Solution" would always be a bad choice. If you need some sort of redundancy and no performance, go for it. if u need performance youll better get yourself a real HARDWARE RAID CONTROLLER for your drives. Those cost a bit. but are worth it.

Best RAID Controllers around i do know are those from ICP VORTEX (meanwhile belongs to Adaptec). Best Linux Support ever! Accustic alarm, SNMP, GUI, CLI Interfaces and Text-oriented tools to handle/configure the raid controller within the running os and all.

2nd best are ADAPTEC Raid controllers,

also working good with linux are LSI Logic MEgaRaids. BUT... they dont support GNU Linux , only RH Enterprise and Suse Enterprise are supported. All software comes within RPM packages. (pretty bullshit!) - The only thing that works good is a CLI tool. with some scripting you can monitor your drives and raid within the os then. no accustic alarm. pretty crappy.

DO NOT USE CHEAP "PSEUDO HARDWARE RAID CONTROLLERS" like DAWI Controll, Silicon Chip crap or those other 150$ shity RAID 0 or RAID 1 crap. those cheap controllers are NO REAL HARDWARE RAIDs. Those are just simple ATA/SATA Adapters with a "more advanced driver". Those Raid controllers are NOTHING without their drivers. just a bunch of disks. And that is what you usualy can see if you boot a linux on such a controller. even if you have configured a RAID5 or RAID1 with only ONE logical drive, those controllers will still present ALL physical DRIVES to the OS as there would be no RAID configuration at all. Drop those controllers... trash em.

Get ICP Vortex (Adaptec) and Adaptec - Those Rock!

greets
Axel

Quoted 13 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

#2 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

Quoted 1 line · Axel Werner

Why you need RAID 10 at all ?

Because my data is important to me, and I need it quickly.

Quoted 2 lines

Usualy because the need of performance and additiona redundancy/fault tollerance. So if u need performance a "Software RAID Solution" would always be a bad choice.

Why? Do you have any benchmarks that show that software raid is slower than hardware raid?

Anyway, still doesn't answer my question.

#4 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

In reply to Axel Werner
Quoted 8 lines · Axel Werner

Why you need RAID 10 at all ?

Usualy because the need of performance and additional redundancy/fault tollerance.
So if u need performance a "Software RAID Solution" would always be a bad choice. If you need some sort of redundancy and no performance, go for it. if u need performance youll better get yourself a real HARDWARE RAID CONTROLLER for your drives. Those cost a bit. but are worth it.

Common myth; as John says please present benchmarks to support this. While there are some advantages to hardware RAID that can be a benefit in some situations it's not safe to assume that the hardware solution outperforms the software approach. It can do but usually only when the number of disks to manage reaches a level where PCI bus saturation becomes an issue. Below this point, particularly for RAID levels involving parity calculations software will often outperform a hardware solution (it uses the host CPU for these calculations instead of the itty-bitty embedded processor on the RAID card).

Quoted 4 lines

Best RAID Controllers around i do know are those from ICP VORTEX (meanwhile belongs to Adaptec). Best Linux Support ever! Accustic alarm, SNMP, GUI, CLI Interfaces and Text-oriented tools to handle/configure the raid controller within the running os and all.

ICP Vortex use the aacraid chipset from Adaptec (also shipped in different forms by a number of other OEMs and system integrators).

Quoted 2 lines

also working good with linux are LSI Logic MEgaRaids. BUT... they dont support GNU Linux , only RH Enterprise and Suse Enterprise are

What are you talking about? The megaraid driver has been in the kernel for years now. It's been a very long time since you had to use out-of-tree modules for these cards (even if the vendors still provide their own packaged binary modules).

Quoted 4 lines

DO NOT USE CHEAP "PSEUDO HARDWARE RAID CONTROLLERS" like DAWI Controll, Silicon Chip crap or those other 150$ shity RAID 0 or RAID 1 crap. those cheap controllers are NO REAL HARDWARE RAIDs. Those are just simple ATA/SATA Adapters with a "more advanced driver". Those Raid controllers

The vendor provided drivers for these cards are generally junk. Most ATA soft-RAID cards now work reasonably well with the dmraid tools.

Quoted 3 lines

are NOTHING without their drivers. just a bunch of disks. And that is what you usualy can see if you boot a linux on such a controller. even if you have configured a RAID5 or RAID1 with only ONE logical drive,

RAID5 with one drive?

Quoted 3 lines

those controllers will still present ALL physical DRIVES to the OS as there would be no RAID configuration at all. Drop those controllers... trash em.

Yep, that's where dmraid comes in - it interprets the on-disk metadata and generates an appropriate device-mapper table to map the arrays on the disks without any need for the proprietary drivers.

Regards,
Bryn.

#5 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

Quoted 8 lines · Bryn M. Reeves
Quoted 2 lines · Axel Werner

also working good with linux are LSI Logic MEgaRaids. BUT... they dont support GNU Linux , only RH Enterprise and Suse Enterprise are

What are you talking about? The megaraid driver has been in the kernel for years now. It's been a very long time since you had to use out-of-tree modules for these cards (even if the vendors still provide their own packaged binary modules).

He probably is referring to the userland tools for managing the controllers, which are useless on most distros. Which makes the hardware much less valuable, since you have to down the system and get into the adaptor's BIOS just to look at it, let alone adjust it. See for example *any* HP storage or system-management product. The tools will not even load -- they depend on versions of libstdc++ that do not and will never exist in most distros.

The megaraid (and cciss) drivers *have* been in-kernel for a long time now and *do* work well -- once you get the adaptor set up from BIOS. As long as you're happy to set-and-forget, they are good choices.

Signature · 2 lines
Mark H. Wood, Lead System Programmer   [email protected]
Friends don't let friends publish revisable-form documents.
#3 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

In reply to John Hughes
Quoted 4 lines · John Hughes
Quoted 2 lines

So if u need performance a "Software RAID Solution" would always be a bad choice.

Why? Do you have any benchmarks that show that software raid is slower than hardware raid?

For RAID 1 and 0, software RAID is as fast or faster in my experience. If you think about it, the software is just coordinating DMA transfers, no actual processing. It is RAID 5 and similar configurations that benefit from hardware implementation. My experience has been with the 'md' driver and AIX LVM, not with Linux LVM mirroring/striping, however.

A big drawback of controller based hardware RAID is the requirement to have same sized drives (or waste the difference). With software RAID (and with high end sub-system based hardware RAID), you can mix a variety of drive sizes. I typically use RAID 1, and I like to expand storage by just adding a drive in the size du jour (500G today, 1T tomorrow), and doing a few pvmoves to expand mirrored space. (And you can also move md mirrors while in operation with judicious use of mdadm -a and mdadm -f, the tricky part being no changes to a partition table while a partition is in use).

Signature · 4 lines
	      Stuart D. Gathman <[email protected]>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
#6 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

In reply to Axel Werner

I have to take issue with that. Software raid is almost always faster than hardware raid on any modern system. I've run into a few special cases where this isn't true, usually involving a database, replication, and very large onboard raid cache.

Not to mention that adding a raid card is a single point of failure and *poof* -- hope you had current backups.

Quoted 62 lines · Axel Werner

Why you need RAID 10 at all ?

Usualy because the need of performance and additional redundancy/fault tollerance.
So if u need performance a "Software RAID Solution" would always be a bad choice. If you need some sort of redundancy and no performance, go for it. if u need performance youll better get yourself a real HARDWARE RAID CONTROLLER for your drives. Those cost a bit. but are worth it.

Best RAID Controllers around i do know are those from ICP VORTEX (meanwhile belongs to Adaptec). Best Linux Support ever! Accustic alarm, SNMP, GUI, CLI Interfaces and Text-oriented tools to handle/configure the raid controller within the running os and all.

2nd best are ADAPTEC Raid controllers,

also working good with linux are LSI Logic MEgaRaids. BUT... they dont support GNU Linux , only RH Enterprise and Suse Enterprise are supported. All software comes within RPM packages. (pretty bullshit!) - The only thing that works good is a CLI tool. with some scripting you can monitor your drives and raid within the os then. no accustic alarm. pretty crappy.

DO NOT USE CHEAP "PSEUDO HARDWARE RAID CONTROLLERS" like DAWI Controll, Silicon Chip crap or those other 150$ shity RAID 0 or RAID 1 crap. those cheap controllers are NO REAL HARDWARE RAIDs. Those are just simple ATA/SATA Adapters with a "more advanced driver". Those Raid controllers are NOTHING without their drivers. just a bunch of disks. And that is what you usualy can see if you boot a linux on such a controller. even if you have configured a RAID5 or RAID1 with only ONE logical drive, those controllers will still present ALL physical DRIVES to the OS as there would be no RAID configuration at all. Drop those controllers... trash em.

Get ICP Vortex (Adaptec) and Adaptec - Those Rock!

greets
Axel

Quoted 6 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }
Quoted 7 lines

Should I just stick with mdadm for my mirroring and striping needs?

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Signature · 6 lines
Sent from my mobile device

Greg

Prosperity is only an instrument to be used, not a deity to be worshipped.
-- Calvin Coolidge
#7 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

In reply to Axel Werner

Short version: Go read http://linuxmafia.com/faq/Hardware/sata.html, and don't make softraid worse than it is.

Quoted 1 line · Axel Werner

DO NOT USE CHEAP "PSEUDO HARDWARE RAID CONTROLLERS" like DAWI Controll,

(lotsa yelling deleted)

Signature · 4 lines
Idiot Box, n.:
	The part of the envelope that tells a person where to place the
stamp when they can't quite figure it out for themselves.
		-- Rich Hall, "Sniglets"
#9 lore raw
In reply to John Hughes
Quoted 7 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

So, ignoring the hijacking of this thread by people proposing hardware solutions, the answer is "nobody who knows reads this list"?

#10 lore raw
Quoted 8 lines · John Hughes [[email protected]] wrote:
Quoted 7 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

You can do it today with some hassle or wait until someone implements a feature called 'generic layering'. The feature really means, treat some LVs as PVs!

How can you do raid10 today? Create two raid0 LVs. Lets us call these lvgroup0/lv0 and lvgroup0/lv1. Now create raid1 lv in lvgroup1 where lvgroup1's PVs are lvgroup0/lv0 and lvgroup0/lv1.

Isn't that a rai10 volume?

Thanks, Malahal.

#11 lore raw
Quoted 17 lines · [email protected] wrote:
Quoted 10 lines · John Hughes [[email protected]] wrote:
Quoted 8 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

You can do it today with some hassle or wait until someone implements a feature called 'generic layering'. The feature really means, treat some LVs as PVs!

Quoted 6 lines

How can you do raid10 today? Create two raid0 LVs. Lets us call these lvgroup0/lv0 and lvgroup0/lv1. Now create raid1 lv in lvgroup1 where lvgroup1's PVs are lvgroup0/lv0 and lvgroup0/lv1.

Isn't that a rai10 volume?

To increase the chances of surviving a double-disk failure it would be better to raid-0 a bunch of raid-1's.

Are we sure there are no deadlock problems with LVM2 layered on top of LVM2?

#12 lore raw

In my experience, on the linux platform it's easier to avoid LVM's raid-like features and let software raid, specifically md, handle the raid for you. It's another layer to manage but not much more complexity.

Not to knock on the LVM folks, they do awesome work, but imho the code for the software raid is better tested if only by virtue of how often it's used/tested/broken. LVM, as I see it, is designed to take physical disks/arrays and hide the underlying structure into one giant pool for the admin to manage.

Signature · 4 lines
Drew

"Nothing in life is to be feared. It is only to be understood."
--Marie Curie
#13 lore raw
In reply to John Hughes

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 18 lines · John Hughes
Quoted 17 lines · [email protected] wrote:
Quoted 10 lines · John Hughes [[email protected]] wrote:
Quoted 8 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

You can do it today with some hassle or wait until someone implements a feature called 'generic layering'. The feature really means, treat some LVs as PVs!

Quoted 5 lines
Quoted 5 lines

How can you do raid10 today? Create two raid0 LVs. Lets us call these lvgroup0/lv0 and lvgroup0/lv1. Now create raid1 lv in lvgroup1 where lvgroup1's PVs are lvgroup0/lv0 and lvgroup0/lv1.

Isn't that a rai10 volume?

No, it's a RAID 0+1 (RAID01/mirror of stripes).

Quoted 2 lines

To increase the chances of surviving a double-disk failure it would be better to raid-0 a bunch of raid-1's.

This is RAID1+0 (RAID10/stripe of mirrors) - it's usually preferable to 0+1 not only because of the improved redundancy but also the individual mirror sets can have failures and recover independently. When a strip set in a RAID0+1 fails the entire mirror must be re-synchronised giving longer recovery times and more performance degradation during the rebuild.

Quoted 2 lines

Are we sure there are no deadlock problems with LVM2 layered on top of LVM2?

Stacked volumes work fine, there's just limited support in the tools for creating them without going to a lot of manual effort at the moment.

Regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAknjL1cACgkQ6YSQoMYUY96uPACeNwylwf1aeZuAzYxHWGJhH7Je LHMAnjLy4NB77gUiABVUkiKeCAMpbGU4
=lv0v
-----END PGP SIGNATURE-----

#14 lore raw
Quoted 28 lines · Bryn M. Reeves

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 18 lines · John Hughes
Quoted 17 lines · [email protected] wrote:
Quoted 10 lines · John Hughes [[email protected]] wrote:
Quoted 8 lines · John Hughes

� � � � � � � �if (lp->stripes > 1) {
� � � � � � � � � � � �log_error("mirrors and stripes are currently " � � � � � � � � � � � � � � � � �"incompatible");
� � � � � � � � � � � �return 0;
� � � � � � � �}

Should I just stick with mdadm for my mirroring and striping needs?

You can do it today with some hassle or wait until someone implements a feature called 'generic layering'. The feature really means, treat some LVs as PVs!

Quoted 5 lines
Quoted 5 lines

How can you do raid10 today? Create two raid0 LVs. Lets us call these lvgroup0/lv0 and lvgroup0/lv1. Now create raid1 lv in lvgroup1 where lvgroup1's PVs are lvgroup0/lv0 and lvgroup0/lv1.

Isn't that a rai10 volume?

Quoted 7 lines

No, it's a RAID 0+1 (RAID01/mirror of stripes).

Quoted 2 lines

To increase the chances of surviving a double-disk failure it would be better to raid-0 a bunch of raid-1's.

This is RAID1+0 (RAID10/stripe of mirrors) - it's usually preferable to 0+1 not only because of the improved redundancy but also the individual

I disagree.

Look at this:
http://www.pcguide.com/ref/hdd/perf/raid/levels/multXY-c.html

Quoted 30 lines

mirror sets can have failures and recover independently. When a strip set in a RAID0+1 fails the entire mirror must be re-synchronised giving longer recovery times and more performance degradation during the rebuild.

Quoted 2 lines

Are we sure there are no deadlock problems with LVM2 layered on top of LVM2?

Stacked volumes work fine, there's just limited support in the tools for creating them without going to a lot of manual effort at the moment.

Regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAknjL1cACgkQ6YSQoMYUY96uPACeNwylwf1aeZuAzYxHWGJhH7Je LHMAnjLy4NB77gUiABVUkiKeCAMpbGU4
=lv0v
-----END PGP SIGNATURE-----

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Signature · 9 lines
Regards,
Sandeep.





 	
�To learn is to change. Education is a process that changes the learner.�
#15 lore raw

On Mon, Apr 13, 2009 at 02:30:27PM GMT, Sandeep K Sinha [[email protected]] said the following:

Quoted 11 lines
Quoted 6 lines
Quoted 3 lines

To increase the chances of surviving a double-disk failure it would be better to raid-0 a bunch of raid-1's.

This is RAID1+0 (RAID10/stripe of mirrors) - it's usually preferable to 0+1 not only because of the improved redundancy but also the individual

I disagree.

Look at this:
http://www.pcguide.com/ref/hdd/perf/raid/levels/multXY-c.html

I misread what Bryn had said at first as well, but re-reading it, he
means that RAID1+0 does better at recovery than RAID0+1, which does make sense. In RAID0+1, depending on how many mirror arrays you have and how smart your RAID controller is, you could have twice as many disk accesses happening during a disk recovery. The URL that you pointed too doesn't really talk about performance during a recovery anyways.

Signature · 4 lines
Mark S. Krenz
IT Director
Suso Technology Services, Inc.
http://suso.org/
#8 lore raw

Re: [linux-lvm] Will LVM2 ever be able to do striped mirrors "raid 10"? >>> Use HARDWARE RAID CONTROLLERS

In reply to Axel Werner
Quoted 4 lines · Axel Werner

Why you need RAID 10 at all ?

Usualy because the need of performance and additional redundancy/fault tollerance.
So if u need performance a "Software RAID Solution" would always be a bad choice. If you need some sort of redundancy and no performance, go for it. if u need performance youll better get yourself a real HARDWARE RAID CONTROLLER for your drives. Those cost a bit. but are worth it.

i have a couple usage cases for raid10 that make your point ridiculus

i have two san attached storages in two different buildings, they present a number of lun (each is raid-5 (3+1)). those will likely outperform your icp vortex by some order of magnitude, still it is not enough for me, so i would use striping, between different luns to achieve more iops.

now i said they were 2 and in two different buildings, this is why i use mirroring, in case an entire building is unavailable. yes i have servers in both building.

having this integrated seamlessly in lvm would be nice, so i could use lvm locking integration with cluster software.

the other scenario is when i already have a striped lv and want to mirror it for migration to a different storage.

L.

Signature · 6 lines
Luca Berra -- [email protected]
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \
#16 lore raw
In reply to Sandeep K Sinha

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 23 lines · Sandeep K Sinha
Quoted 21 lines · On Mon, Apr 13, 2009 at 5:55 PM, Bryn M. Reeves <[email protected]> wrote: John…
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines · [email protected] wrote:
Quoted 10 lines · John Hughes [[email protected]] wrote:
Quoted 8 lines · John Hughes
               if (lp->stripes > 1) {
                       log_error("mirrors and stripes are currently "
                                 "incompatible");
                       return 0;
               }

Should I just stick with mdadm for my mirroring and striping needs?

You can do it today with some hassle or wait until someone implements a feature called 'generic layering'. The feature really means, treat some LVs as PVs!

How can you do raid10 today? Create two raid0 LVs. Lets us call these lvgroup0/lv0 and lvgroup0/lv1. Now create raid1 lv in lvgroup1 where lvgroup1's PVs are lvgroup0/lv0 and lvgroup0/lv1.

Quoted 12 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Isn't that a rai10 volume?

No, it's a RAID 0+1 (RAID01/mirror of stripes).

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

To increase the chances of surviving a double-disk failure it would be better to raid-0 a bunch of raid-1's.

This is RAID1+0 (RAID10/stripe of mirrors) - it's usually preferable to 0+1 not only because of the improved redundancy but also the individual

Quoted 4 lines

I disagree.

Look at this:
http://www.pcguide.com/ref/hdd/perf/raid/levels/multXY-c.html

What do you disagree with?

The article you linked to points out the same robustness and recovery concerns with a 0+1 layout as I mentioned earlier. It then goes on to say that this could be mitigated for 0+1 by a smart RAID implementation that continues to stripe to partially-failed RAID 0 sets but that most actual RAID controllers don't chose to implement this.

The article seems to make the same claims as I did.

Regards,
Bryn.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAknsLCgACgkQ6YSQoMYUY94KswCeOahyHLXilItIA8B1AR2Z/Ujb aloAn0jftWyKXqr9QK51tYkwmOWU68LF
=mMPX
-----END PGP SIGNATURE-----