Mirror between different SAN fabrics

Hi,

Using lvm2, I would like to mirror a logical volume between two different SAN fabrics to guarantee high availability for my data, even if a whole SAN fabric crashes.
Is there a possibility in lvm2 to define two volume groups, put the disks from fabric1 into group 1 and the disks from fabric2 into group 2 and then create the mirror over the two groups? I didn't find much documentation for lvm2 about this, I just know that other Volume Managers can handle it.

It's essential to group the discs together because I use like 100 discs for each mirror and I have to be sure that mirroring takes place between the two different SAN storages.

Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter:
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

From: [email protected] [mailto:[email protected]] On Behalf Of Matt P

I can't think of any Volume Manager that would let you mirror disks from 2 separate VGs...

But other than that I wouldn't see why you couldn't use 2 different fabrics, they're just disks to LVM... If you're having trouble managing 200+ PVs, just build the LV first using only disks from 1 fabric, then make the second fabric available/visible, and add a mirror using only those disks... I think that would work...

Quoted 6 lines

From: [email protected] [mailto:[email protected]] On Behalf Of Matt P

I can't think of any Volume Manager that would let you mirror disks from 2 separate VGs...

Maybe not from 2 separate VGs but they add an additional level to group PVs together. With HP-UX you can use "lvcreate -m -s g" to use different PV-groups for the mirror. With Tru64 as far as I know you can use a Plex to create a PV-group (should be similar with Veritas)

Quoted 6 lines

But other than that I wouldn't see why you couldn't use 2 different fabrics, they're just disks to LVM... If you're having trouble managing 200+ PVs, just build the LV first using only disks from 1 fabric, then make the second fabric available/visible, and add a mirror using only those disks... I think that would work...

That's an idea, but when I'm adding 4 more disks (eg. disk1 and disk2 from fabric1, disk3 and disk4 from fabric2), how can I tell LVM to mirror Disk1,2 to Disk3,4 and not to mirror Disk1 to Disk2?

Regards Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter:
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Matt P

I think the proper way would be to use mdadm. Setup software RAID1 prior to getting LVM involved...

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Matt P

I think the proper way would be to use mdadm. Setup software RAID1 prior to getting LVM involved...

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of [email protected]
Sent: Donnerstag, 28 Dezember, 2006 09:49

Hi,

Here is a nice example from one of my RHEL 4 Oracle servers:

We have three layers:

first the LUNs from the SAN are multipathed to device aliases:

[root@ ~]# multipath -ll
sanb (XXXX60e8003f653000000XXXX000001c7)
[size=101 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [active] \_ 0:0:1:1 sdb 8:16 [active][ready] \_ 1:0:1:1 sdd 8:48 [active][ready]

sana (XXXX60e80039cbe000000XXXX000006ad)
[size=101 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [active] \_ 0:0:0:1 sda 8:0 [active][ready] \_ 1:0:0:1 sdc 8:32 [active][ready]

Next these multipath aliases are RAIDed:

[root@ ~]# mdadm --detail /dev/md0
/dev/md0:

        Version : 00.90.01
  Creation Time : Thu Nov  2 13:07:01 2006
     Raid Level : raid1
     Array Size : 106788160 (101.84 GiB 109.35 GB)
    Device Size : 106788160 (101.84 GiB 109.35 GB)

Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Thu Dec 28 09:36:19 2006
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

    Number   Major   Minor   RaidDevice State
       0     253        2        0      active sync   /dev/mapper/sana
       1     253        3        1      active sync   /dev/mapper/sanb
           UUID : b5ac4ae9:99da8114:744a7ebb:aba6f687
         Events : 0.4254576

And finally, the RAID device is used with LVM:

[root@ ~]# vgs -o +devices
VG #PV #LV #SN Attr VSize VFree Devices

  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(0)
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p4(0)
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(250)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(0)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5120)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5376)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5632)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(8192)

This works very well, as both paths and mirrors are able to break away without any disruption in disk access.

Cheers,

Christian

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of [email protected]
Sent: Donnerstag, 28 Dezember, 2006 09:49

Hi,

Here is a nice example from one of my RHEL 4 Oracle servers:

We have three layers:

first the LUNs from the SAN are multipathed to device aliases:

[root@ ~]# multipath -ll
sanb (XXXX60e8003f653000000XXXX000001c7)
[size=101 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [active] \_ 0:0:1:1 sdb 8:16 [active][ready] \_ 1:0:1:1 sdd 8:48 [active][ready]

sana (XXXX60e80039cbe000000XXXX000006ad)
[size=101 GB][features="1 queue_if_no_path"][hwhandler="0"] \_ round-robin 0 [active] \_ 0:0:0:1 sda 8:0 [active][ready] \_ 1:0:0:1 sdc 8:32 [active][ready]

Next these multipath aliases are RAIDed:

[root@ ~]# mdadm --detail /dev/md0
/dev/md0:

        Version : 00.90.01
  Creation Time : Thu Nov  2 13:07:01 2006
     Raid Level : raid1
     Array Size : 106788160 (101.84 GiB 109.35 GB)
    Device Size : 106788160 (101.84 GiB 109.35 GB)

Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Thu Dec 28 09:36:19 2006
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

    Number   Major   Minor   RaidDevice State
       0     253        2        0      active sync   /dev/mapper/sana
       1     253        3        1      active sync   /dev/mapper/sanb
           UUID : b5ac4ae9:99da8114:744a7ebb:aba6f687
         Events : 0.4254576

And finally, the RAID device is used with LVM:

[root@ ~]# vgs -o +devices
VG #PV #LV #SN Attr VSize VFree Devices

  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(0)
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p4(0)
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(250)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(0)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5120)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5376)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5632)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(8192)

This works very well, as both paths and mirrors are able to break away without any disruption in disk access.

Cheers,

Christian

I haven't tried it in a cluster yet. I was planning on using HP's MC-ServiceGuard to deal with HA clustering. I don't see why the LUNs that are used on one system with mdadm can't be used another, since the RAID block is on the disk and is readable even on a system upon which it wasn't created on. /etc/mdadm.conf will ofcourse need to be copied and kept current on all cluster nodes, but with the config file and the RAID block on the disk, an "mdadm --assemble" should work. Importing the LVM structures should then also not be a problem.

I haven't tried it in a cluster yet. I was planning on using HP's MC-ServiceGuard to deal with HA clustering. I don't see why the LUNs that are used on one system with mdadm can't be used another, since the RAID block is on the disk and is readable even on a system upon which it

created on. /etc/mdadm.conf will ofcourse need to be copied and kept current on all cluster nodes, but with the config file and the RAID block on the disk, an "mdadm --assemble" should work. Importing the LVM structures should then also not be a problem.

ahhh, I see where the difference in view comes from: MC-Service Guard is a HA clustering suite that does not contain a cluster FS; it simply does package switching upon failover between two or more nodes.

At that stage the md stuff is only ever accessed on a single node, and there's no problem.

[...]

Quoted 2 lines

At that stage the md stuff is only ever accessed on a single node, and there's no problem.

I will use GFS Filesystem with all cluster nodes up and running at the same time, sharing their disks.
So my problem with the lvm2 mirroring feature still exists. Think I have to use the expensive and not easy configurable Continuous Access Solution to mirror directly on SAN fabric level...

Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.
------------------------------------------------------------------------

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

I'm wondering how well a "stacked" lvm approach would work for this. Could you take LVM and make a two VGs called "Fabric1VG" and "Fabric2VG", where you put all of the "fabric 1 paths" to your PVs into Fabric1VG, and all your "fabric 2 paths" to your PVs into Fabric2VG. Then form volumes in each of those (it would be your responsibility to ensure that you have equal volumes in each fabric group). So you would have Fabric1VG/LVa and Fabric2VG/LVa as "equivilant" devices, going across your different fabric paths. Then you could create a new VG called MirrorVG and put both Fabric1VG/LVa and Fabric2VG/LVa into MirrorVG as PVs. Then I think you should be able to create a new mirrored LV from MirrorVG which would mirror across your two fabrics. This should move all of your management into LVM2, which is cluster aware, but it 1) makes management a bit messy (but it is all quite scriptable), and 2) adds a wierd LVM 2-layer setup.

Anyone know if this 2-layer LVM approach would kill performance (any more than a two layer approach of lvm->mdadm or mdadm->lvm)?

Hmm... Thinking about this further, I've been thinking of this 2-layer approach for striping and mirroring, but it looks like it might be more problematic for your case, where you really want multipath and mirroring. I'm not sure how to ensure that Fabric1VG/LVa and Fabric2VG/LVa are placed on the same PV blocks as the other one. When you create Fabric1VG/LVa it might end up on disks 0,1, and 2, but Fabric2VG/LVa might end up on disks 1, 2, and 3 (but by different paths). Anyone know any way to ensure that block placement is identical? Is the block allocation algorithm predictable and repeatable so that if you have two VGs with equal PVs in each, and you create LVs in the same order in each, they get the same PV mapping? Or is there a built-in randomness? You would likely be assured of the same failures, since a bad block on fabric 1 should also be a bad block on fabric 2, and a failed PV would fail in both LVs... Might work, if the placement routines are predictably defined.

-Ty!

Quoted 20 lines · [email protected] wrote:

[...]

Quoted 3 lines

At that stage the md stuff is only ever accessed on a single node, and there's no problem.

I will use GFS Filesystem with all cluster nodes up and running at the same time, sharing their disks.
So my problem with the lvm2 mirroring feature still exists. Think I have to use the expensive and not easy configurable Continuous Access Solution to mirror directly on SAN fabric level...

Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter:
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.
------------------------------------------------------------------------

Quoted 5 lines

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

--
-===========================-
Ty! Boyack
NREL Unix Network Manager
[email protected]
(970) 491-1186
-===========================-

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

This is basically the "messy" way I mentioned in my reply above. I found if you pvcreate the LV device, you end up mangling the lvm data (this probably comes as little surprise) and it breaks down after that. So, I ended up using losetup and an "image file", one for/on each fabric. Then did pvcreate on each loop device, and made a new VG containing both PVs and created the LV with mirroring.... It worked.... I did no performance, stability, or failure testing...

Actually, I'm quite surprised that this approach mangles the lvm data. It seems that when you do a pvcreate on a block device, LVM should (and I think, does) write the lvm metadata in a region of that device, and then never let anything touch that metadata. This way, if you do a 'dd if=/dev/zeros of=<PV-DEVICE>' it blanks out the device, but the metadata is intact.

So, if you do a 'pvcreate' on an LV, it should contain a second copy of the metadata, unique and independent from the first copy on the original block device. My tests on this has worked fine (although my tests have been for building two VGs that have striped volumes across the member disks, and then a VG that creates a mirrored LV of the striped volumes, so no multipathing is involved). I'm wondering if we can compare notes to see if I'm doing something that makes it look like it's working -- I don't want to be quietly destroying my lvm data and not knowing it!!!

I'm doing (roughly, block devices are a bit made-up):

# prepare the physical volumes
pvcreate /dev/sda
pvcreate /dev/sdb
pvcreate /dev/sdc
pvcreate /dev/sdd
pvcreate /dev/sde

# Create volume groups to contain uniquely striped volumes vgcreate Stripe1VG /dev/sda /dev/sdb
vgcreate Stripe2VG /dev/sdc /dev/sdd

# Create the striped volumes
lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG
lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG

# Make the striped volumes into PVs
pvcreate /dev/Stripe1VG/Stripe1LV
pvcreate /dev/Stripe2VG/Stripe2LV

# Create the volume group for mirrored volumes
vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV /dev/Stripe2VG/Stripe2LV /dev/sde
# (Had to use three PVs to have the mirror log in place)

# Create the mirrored volume
lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG

# Filesystem, test, etc. this will be GFS eventually, but testing with ext3 for now.
mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV
mkdir /mnt/mirror1lv
mount /dev/MirrorVG/Mirror1LV /mnt/mirror1

Is that about your procedure as well? When does the lvm data get mangled?

(Sorry if this is going off topic - but if this is solvable it might actually answer the original question...)

-Ty!

Quoted 7 lines · Matt P

This is basically the "messy" way I mentioned in my reply above. I found if you pvcreate the LV device, you end up mangling the lvm data (this probably comes as little surprise) and it breaks down after that. So, I ended up using losetup and an "image file", one for/on each fabric. Then did pvcreate on each loop device, and made a new VG containing both PVs and created the LV with mirroring.... It worked.... I did no performance, stability, or failure testing...

--
-===========================-
Ty! Boyack
NREL Unix Network Manager
[email protected]
(970) 491-1186
-===========================-

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Matt P
Sent: Freitag, 29 Dezember, 2006 00:25
To: LVM general discussion and development
Subject: Re: [linux-lvm] Mirror between different SAN fabrics

That's exactly the steps I used to test it out. The failure I encountered was at the point of adding the StripeLVs to the MirrorVG. I don't recall the error I got but I seem to recall it being the same error as if the PV hadn't been pvcreated. Although when I went through the process again just now, without any errors... It's quite likely I mistyped or skipped a step during my first attempt...

I've gone through the whole process 4 times now from raw disk to a mirrored LV without getting the error again.

It looks as though this could work and isn't nearly as crazy as my initial configuration. I wonder now if anything wonky will happen if we were to lose a disk or in the original post if we lost one of the fabrics...

On 12/28/06, Ty! Boyack <[email protected]> wrote:

Actually, I'm quite surprised that this approach
mangles the lvm data.
It seems that when you do a pvcreate on a block device,
LVM should (and
I think, does) write the lvm metadata in a region of
that device, and
then never let anything touch that metadata. This way,
if you do a 'dd
if=/dev/zeros of=<PV-DEVICE>' it blanks out the device,
but the metadata
is intact.

So, if you do a 'pvcreate' on an LV, it should contain
a second copy of
the metadata, unique and independent from the first
copy on the original
block device. My tests on this has worked fine
(although my tests have
been for building two VGs that have striped volumes
across the member
disks, and then a VG that creates a mirrored LV of the
striped volumes,
so no multipathing is involved). I'm wondering if we
can compare notes
to see if I'm doing something that makes it look like
it's working -- I
don't want to be quietly destroying my lvm data and not
knowing it!!!

I'm doing (roughly, block devices are a bit made-up):

# prepare the physical volumes
pvcreate /dev/sda
pvcreate /dev/sdb
pvcreate /dev/sdc
pvcreate /dev/sdd
pvcreate /dev/sde

# Create volume groups to contain uniquely striped volumes vgcreate Stripe1VG /dev/sda /dev/sdb
vgcreate Stripe2VG /dev/sdc /dev/sdd

# Create the striped volumes lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG

# Make the striped volumes into PVs
pvcreate /dev/Stripe1VG/Stripe1LV
pvcreate /dev/Stripe2VG/Stripe2LV

# Create the volume group for mirrored volumes vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV
/dev/Stripe2VG/Stripe2LV /dev/sde
# (Had to use three PVs to have the mirror log in place)

# Create the mirrored volume lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG

# Filesystem, test, etc. this will be GFS eventually,
but testing with
ext3 for now.
mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV mkdir /mnt/mirror1lv
mount /dev/MirrorVG/Mirror1LV /mnt/mirror1

Is that about your procedure as well? When does the
lvm data get mangled?

(Sorry if this is going off topic - but if this is
solvable it might
actually answer the original question...)

-Ty!

Matt P wrote:
> This is basically the "messy" way I mentioned in my
reply above. I
> found if you pvcreate the LV device, you end up
mangling the lvm data
> (this probably comes as little surprise) and it
breaks down after
> that. So, I ended up using losetup and an "image
file", one for/on
> each fabric. Then did pvcreate on each loop device,
and made a new VG
> containing both PVs and created the LV with mirroring.... It > worked.... I did no performance, stability, or
failure testing...

	--
	-===========================-
	  Ty! Boyack
	  NREL Unix Network Manager
	  [email protected] 
	  (970) 491-1186
	-===========================-
	
	_______________________________________________
	linux-lvm mailing list
	[email protected]
	https://www.redhat.com/mailman/listinfo/linux-lvm 

<https://www.redhat.com/mailman/listinfo/linux-lvm>
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Matt P
Sent: Freitag, 29 Dezember, 2006 00:25
To: LVM general discussion and development
Subject: Re: [linux-lvm] Mirror between different SAN fabrics

That's exactly the steps I used to test it out. The failure I encountered was at the point of adding the StripeLVs to the MirrorVG. I don't recall the error I got but I seem to recall it being the same error as if the PV hadn't been pvcreated. Although when I went through the process again just now, without any errors... It's quite likely I mistyped or skipped a step during my first attempt...

I've gone through the whole process 4 times now from raw disk to a mirrored LV without getting the error again.

It looks as though this could work and isn't nearly as crazy as my initial configuration. I wonder now if anything wonky will happen if we were to lose a disk or in the original post if we lost one of the fabrics...

On 12/28/06, Ty! Boyack <[email protected]> wrote:

Actually, I'm quite surprised that this approach
mangles the lvm data.
It seems that when you do a pvcreate on a block device,
LVM should (and
I think, does) write the lvm metadata in a region of
that device, and
then never let anything touch that metadata. This way,
if you do a 'dd
if=/dev/zeros of=<PV-DEVICE>' it blanks out the device,
but the metadata
is intact.

So, if you do a 'pvcreate' on an LV, it should contain
a second copy of
the metadata, unique and independent from the first
copy on the original
block device. My tests on this has worked fine
(although my tests have
been for building two VGs that have striped volumes
across the member
disks, and then a VG that creates a mirrored LV of the
striped volumes,
so no multipathing is involved). I'm wondering if we
can compare notes
to see if I'm doing something that makes it look like
it's working -- I
don't want to be quietly destroying my lvm data and not
knowing it!!!

I'm doing (roughly, block devices are a bit made-up):

# prepare the physical volumes
pvcreate /dev/sda
pvcreate /dev/sdb
pvcreate /dev/sdc
pvcreate /dev/sdd
pvcreate /dev/sde

# Create volume groups to contain uniquely striped volumes vgcreate Stripe1VG /dev/sda /dev/sdb
vgcreate Stripe2VG /dev/sdc /dev/sdd

# Create the striped volumes lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG

# Make the striped volumes into PVs
pvcreate /dev/Stripe1VG/Stripe1LV
pvcreate /dev/Stripe2VG/Stripe2LV

# Create the volume group for mirrored volumes vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV
/dev/Stripe2VG/Stripe2LV /dev/sde
# (Had to use three PVs to have the mirror log in place)

# Create the mirrored volume lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG

# Filesystem, test, etc. this will be GFS eventually,
but testing with
ext3 for now.
mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV mkdir /mnt/mirror1lv
mount /dev/MirrorVG/Mirror1LV /mnt/mirror1

Is that about your procedure as well? When does the
lvm data get mangled?

(Sorry if this is going off topic - but if this is
solvable it might
actually answer the original question...)

-Ty!

Matt P wrote:
> This is basically the "messy" way I mentioned in my
reply above. I
> found if you pvcreate the LV device, you end up
mangling the lvm data
> (this probably comes as little surprise) and it
breaks down after
> that. So, I ended up using losetup and an "image
file", one for/on
> each fabric. Then did pvcreate on each loop device,
and made a new VG
> containing both PVs and created the LV with mirroring.... It > worked.... I did no performance, stability, or
failure testing...

	--
	-===========================-
	  Ty! Boyack
	  NREL Unix Network Manager
	  [email protected] 
	  (970) 491-1186
	-===========================-
	
	_______________________________________________
	linux-lvm mailing list
	[email protected]
	https://www.redhat.com/mailman/listinfo/linux-lvm 

<https://www.redhat.com/mailman/listinfo/linux-lvm>
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/