HDD Failure
Hello List,
I have a 3x 120GB HDD that are all lumped into one LVM volume. Recently, one died so I removed it using.
# vgreduce --removemissing Vol1
This worked fine but now I can't mount the volume group:
root@nibiru:~# cat /etc/fstab | grep Vol /dev/mapper/Vol1-share /share ext3 defaults 0 2 root@nibiru:~# mount /share mount: special device /dev/mapper/Vol1-share does not exist root@nibiru:~#
I've no idea why it says it doesn't exist as I thought all I did was remove missing PE's from a VG - not remove the actual VG!
Can anyone help me get my data back? Thanks for any advice.
Nick
Nick -- if I understand your situation properly, you effectively took one of the platters (the bad drive) out of the drive (the vg) and now you still want to be able to access the entire drive ... ?? I don't think that's going to happen.
Off to the backups, I'd say ...
-- Mitch
Quoted 28 lines · Nick
Hello List,
I have a 3x 120GB HDD that are all lumped into one LVM volume. Recently, one died so I removed it using.
# vgreduce --removemissing Vol1This worked fine but now I can't mount the volume group:
root@nibiru:~# cat /etc/fstab | grep Vol /dev/mapper/Vol1-share /share ext3 defaults 0 2 root@nibiru:~# mount /share mount: special device /dev/mapper/Vol1-share does not exist root@nibiru:~#I've no idea why it says it doesn't exist as I thought all I did was remove missing PE's from a VG - not remove the actual VG!
Can anyone help me get my data back? Thanks for any advice.
Nick
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Hi Mitch,
Thanks for the response. I understand I won't be able to get all the data back but I have 2x120GB "good" drives left and want to get what's left - is this possible?
Thanks, Nick
Quoted 8 lines · Mitch Miller
Nick -- if I understand your situation properly, you effectively took one of the platters (the bad drive) out of the drive (the vg) and now you still want to be able to access the entire drive ... ?? I don't think that's going to happen.
Off to the backups, I'd say ...
-- Mitch
Quoted 18 lines · Nick
Hi Mitch,
Thanks for the response. I understand I won't be able to get all the data back but I have 2x120GB "good" drives left and want to get what's left - is this possible?
Thanks, Nick
Quoted 8 lines · Mitch Miller
Nick -- if I understand your situation properly, you effectively took one of the platters (the bad drive) out of the drive (the vg) and now you still want to be able to access the entire drive ... ?? I don't think that's going to happen.
Off to the backups, I'd say ...
-- Mitch
did you have only one lv in your vg ?
if yes, you can go to your backups.
what does pvdisplay and vgdisplay says ?
Signature · 1 line
Fabien Jakimowicz <[email protected]>
Hi Fabien,
Yes, just one LV - "Vol1-share".
Does this mean I've lost *everything*? I would have though I should be still be able to access everything on the two working disks?
I don't have a backup of this data.
Thanks, Nick
root@nibiru:~# pvdisplay --- Physical volume --- PV Name /dev/hda4 VG Name Vol1 PV Size 106.79 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 27339 Free PE 27339 Allocated PE 0 PV UUID Cq9xKF-W33m-BCLt-YyIc-EEfm-Btqc-eZLHNh
--- Physical volume ---
PV Name /dev/hdb1 VG Name Vol1 PV Size 111.75 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 28609 Free PE 28609 Allocated PE 0 PV UUID hwQrhH-iXHO-Bots-6zUQ-w8JG-Nmb3-shqZiX
root@nibiru:~# vgdisplay --- Volume group --- VG Name Vol1 System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 2 Act PV 2 VG Size 218.55 GB PE Size 4.00 MB Total PE 55948 Alloc PE / Size 0 / 0 Free PE / Size 55948 / 218.55 GB VG UUID RORj4f-LAOJ-83YS-34lD-4YRM-FKP8-8hgXLg
Quoted 10 lines · Fabien Jakimowicz
did you have only one lv in your vg ?
if yes, you can go to your backups.
what does pvdisplay and vgdisplay says ?
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Nick,
LVM != RAID
You should have been doing RAID if you wanted to be able to handle the
failure of one drive.
On Mon, Sep 18, 2006 at 07:34:37PM GMT, Nick [[email protected]] said the following:
Quoted 74 lines
Hi Fabien,
Yes, just one LV - "Vol1-share".
Does this mean I've lost *everything*? I would have though I should be still be able to access everything on the two working disks?
I don't have a backup of this data.
Thanks, Nick
root@nibiru:~# pvdisplay --- Physical volume --- PV Name /dev/hda4 VG Name Vol1 PV Size 106.79 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 27339 Free PE 27339 Allocated PE 0 PV UUID Cq9xKF-W33m-BCLt-YyIc-EEfm-Btqc-eZLHNh--- Physical volume ---
PV Name /dev/hdb1 VG Name Vol1 PV Size 111.75 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 28609 Free PE 28609 Allocated PE 0 PV UUID hwQrhH-iXHO-Bots-6zUQ-w8JG-Nmb3-shqZiXroot@nibiru:~# vgdisplay --- Volume group --- VG Name Vol1 System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 2 Act PV 2 VG Size 218.55 GB PE Size 4.00 MB Total PE 55948 Alloc PE / Size 0 / 0 Free PE / Size 55948 / 218.55 GB VG UUID RORj4f-LAOJ-83YS-34lD-4YRM-FKP8-8hgXLgQuoted 10 lines · Fabien Jakimowicz
did you have only one lv in your vg ?
if yes, you can go to your backups.
what does pvdisplay and vgdisplay says ?
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Signature · 4 lines
Mark S. Krenz IT Director Suso Technology Services, Inc. http://suso.org/
Re: [linux-lvm] Misleading documentation (was: HDD Failure)
Quoted 5 lines · Mark Krenz
LVM != RAID
You should have been doing RAID if you wanted to be able to
handle the
failure of one drive.
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
For example, the recipes chapter [1] of the HOWTO shows a non-trivial setup with four volume groups split across seven physical drives. There's no mention of RAID. This is a ridiculously bad idea - if *any* of those seven drives are lost, at least one volume group will fail. In some cases, more than one. This document should be showing best practices, and it's instead showing how to throw away your data.
The "lvcreate" manual page is pretty bad, too. It mentions the ability to tune stripe size, which on casual read, might suggest that it uses real RAID. Instead, I think this is just RAID-0.
[1] - http://tldp.org/HOWTO/LVM-HOWTO/recipeadddisk.html
Signature · 1 line
Scott Lamb <http://www.slamb.org/>
Re: [linux-lvm] Misleading documentation (was: HDD Failure)
Quoted 20 lines · Scott Lamb
Quoted 5 lines · Mark Krenz
LVM != RAID
You should have been doing RAID if you wanted to be able to
handle the
failure of one drive.This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
For example, the recipes chapter [1] of the HOWTO shows a non-trivial setup with four volume groups split across seven physical drives. There's no mention of RAID. This is a ridiculously bad idea - if *any* of those seven drives are lost, at least one volume group will fail. In some cases, more than one. This document should be showing best practices, and it's instead showing how to throw away your data.
Quoted 6 lines
The "lvcreate" manual page is pretty bad, too. It mentions the ability to tune stripe size, which on casual read, might suggest that it uses real RAID. Instead, I think this is just RAID-0.
LVM is just a Logical Volume Manager. It sounds like it add a layer between physical devices and logical volumes. A such layer can take care of physical devices failure, but lvm does not. Maybe you are right and it should be pointed out. Maybe not ...
Having a half block device make no sense because there are actually no filesystem capable of recovering this kind of failure.
Signature · 1 line
Fabien Jakimowicz <[email protected]>
Re: [linux-lvm] Misleading documentation (was: HDD Failure)
In reply to Scott Lamb Personally I like it when documentation is kept simple and uses simple
examples. There is nothing worse than when you are trying to learn something and it tells you how to how to intantiate a variable, and then it immediately goes on to show you how to make some complicated reference to it using some code.
I agree with you though, its probably a good idea to steer newcomers
in the right direction on disk management and a few notes about doing LVM ontop of RAID being a good idea couldn't hurt. This is especially so since I've heard three mentions of people using LVM on a server without doing RAID this week alone. :-/
I suppose it could also be said that people who are causually doing
LVM on their systems using something like a GUI are most likely not going to be referencing the man pages or LVM documentation until after their system is setup, at which point it is probably too late to put the physical volumes on a RAID array. So I think its more the responsibility of the GUI/ncurses installer to alert you to be using RAID.
Mark
On Tue, Sep 19, 2006 at 10:40:43PM GMT, Scott Lamb [[email protected]] said the following:
Quoted 35 lines · Mark Krenz
Quoted 5 lines
LVM != RAID
You should have been doing RAID if you wanted to be able to
handle the
failure of one drive.This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
For example, the recipes chapter [1] of the HOWTO shows a non-trivial setup with four volume groups split across seven physical drives. There's no mention of RAID. This is a ridiculously bad idea - if *any* of those seven drives are lost, at least one volume group will fail. In some cases, more than one. This document should be showing best practices, and it's instead showing how to throw away your data.
The "lvcreate" manual page is pretty bad, too. It mentions the ability to tune stripe size, which on casual read, might suggest that it uses real RAID. Instead, I think this is just RAID-0.
[1] - http://tldp.org/HOWTO/LVM-HOWTO/recipeadddisk.html
Signature · 8 lines
Scott Lamb <http://www.slamb.org/> _______________________________________________ linux-lvm mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Signature · 4 lines
Mark S. Krenz IT Director Suso Technology Services, Inc. http://suso.org/
Re: [linux-lvm] Misleading documentation (was: HDD Failure)
Quoted 11 lines · Mark Krenz
Personally I like it when documentation is kept simple and uses simple
examples. There is nothing worse than when you are trying to learn something and it tells you how to how to intantiate a variable, and then it immediately goes on to show you how to make some complicated reference to it using some code.I agree with you though, its probably a good idea to steer newcomers
in the right direction on disk management and a few notes about doing LVM ontop of RAID being a good idea couldn't hurt. This is especially so since I've heard three mentions of people using LVM on a server without doing RAID this week alone. :-/
We should add something in faq page
( http://tldp.org/HOWTO/LVM-HOWTO/lvm2faq.html ), like "i've lost one of my hard drive and i can't mount my lv, did i lost everything ?" followed by a quick explanation : lvm is NOT faulty tolerant like raid1/5, if you lose a PV, you lose every LV which was (even partially) on on it.
Adding a recipe with raid can't kill us.
A. Setting up LVM over software RAID on four disks
For this recipe, the setup has four disks that will be put into two raid arrays which will be used as PV.
The main goal of this configuration is to avoid any data loss if one of the hard drives fails.
A.1 RAID
A.1.1 Preparing the disks
You must partition your disks and set partition type to Linux raid autodetect (FD type), if your system can handle it. I recommand to make only one partition per hard drive. You can use cfdisk to do it.
# cfdisk /dev/sda
If your drives are identicals, you can save time using sfdisk :
# sfdisk -d /dev/sda | sfdisk /dev/sdb # sfdisk -d /dev/sda | sfdisk /dev/sdc # sfdisk -d /dev/sda | sfdisk /dev/sdd
This will partition sdb, sdc and sdd using sda partition table scheme.
A.1.2 Creating arrays
You can check if your system can handle raid1 by typing the following :
# cat /proc/mdstat Personalities : [raid1]
If not (file not found, no raid1 in list), then load raid1 module :
# modprobe raid1
You can now create raid arrays, assuming you have only one raid partiton per drive :
# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1 # mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1
You should wait for the raid arrays to be fully synchronized, check it with :
# cat /proc/mdstat
A.2 LVM
A.2.1 Create Physical Volumes
Run pvcreate on each raid array :
# pvcreate /dev/md0 # pvcreate /dev/md1
This creates a volume group descriptor area (VGDA) at the start of the raid arrays.
A.2.2 Setup a Volume Group
# vgcreate my_volume_group /dev/md0 /dev/md1
You should now see something like that :
# vgdisplay --- Volume group --- VG Name my_volume_group System ID Format lvm2 Metadata Areas 4 Metadata Sequence No 37 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 2 Act PV 2 VG Size 1.18 TB PE Size 4.00 MB Total PE 310014 Alloc PE / Size 0 / 0 TB Free PE / Size 310014 / 1.18 TB VG UUID LI3k9v-MnIA-lfY6-kdAB-nmpW-adjX-A5yKiF
You should check if 'VG Size' matchs your hard drive size (raid1 divide available space by two, so if you have four 300Go hard drives, you will have a ~600Go VG).
A.2.3 Create Logical Volumes
You can now create some LV on your VG :
# lvcreate -L10G -nmy_logical_volume my_volume_group Logical volume "my_logical_volume" created # lvcreate -L42G -nmy_cool_lv my_volume_group Logical volume "my_cool_lv" created
A.2.4 Create the File System
Create an XFS file system on each logical volume :
# mkfs.xfs /dev/my_volume_group/my_logical_volume
meta-data=/dev/my_volume_group/my_logical_volume isize=256
agcount=16, agsize=163840 blks
= sectsz=512
data = bsize=4096 blocks=2621440, imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2560, version=1
= sectsz=512 sunit=0 blks
realtime =none extsz=65536 blocks=0, rtextents=0# mkfs.xfs /dev/my_volume_group/my_cool_lv
meta-data=/dev/my_volume_group/my_cool_lv isize=256 agcount=16,
agsize=688128 blks
= sectsz=512
data = bsize=4096 blocks=11010048,
imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=5376, version=1
= sectsz=512 sunit=0 blks
realtime =none extsz=65536 blocks=0, rtextents=0A.2.5 Test File System
Mount logical volumes and check if everything is fine :
# mkdir -p /mnt/{my_logical_volume,my_cool_lv}
# mount /dev/my_volume_group/my_logical_volume /mnt/my_logical_volume
# mount /dev/my_volume_group/my_cool_lv /mnt/my_cool_lv
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/hda1 1.9G 78M 1.8G 5% /
/dev/mapper/my_volume_group-my_logical_volume
10G 0 10G 0% /mnt/my_logical_volume
/dev/mapper/my_volume_group-my_cool_lv
42G 0 42G 0% /mnt/my_cool_lvQuoted 39 lines
I suppose it could also be said that people who are causually doing
LVM on their systems using something like a GUI are most likely not going to be referencing the man pages or LVM documentation until after their system is setup, at which point it is probably too late to put the physical volumes on a RAID array. So I think its more the responsibility of the GUI/ncurses installer to alert you to be using RAID.Mark
On Tue, Sep 19, 2006 at 10:40:43PM GMT, Scott Lamb [[email protected]] said the following:
Quoted 26 lines · Mark Krenz
Quoted 5 lines
LVM != RAID
You should have been doing RAID if you wanted to be able to
handle the
failure of one drive.This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
For example, the recipes chapter [1] of the HOWTO shows a non-trivial setup with four volume groups split across seven physical drives. There's no mention of RAID. This is a ridiculously bad idea - if *any* of those seven drives are lost, at least one volume group will fail. In some cases, more than one. This document should be showing best practices, and it's instead showing how to throw away your data.
The "lvcreate" manual page is pretty bad, too. It mentions the ability to tune stripe size, which on casual read, might suggest that it uses real RAID. Instead, I think this is just RAID-0.
Signature · 1 line
Fabien Jakimowicz <[email protected]>
Re: [linux-lvm] Misleading documentation
Quoted 14 lines · Fabien Jakimowicz
Quoted 6 lines · Mark Krenz
Personally I like it when documentation is kept simple and uses simple
examples. There is nothing worse than when you are trying to learn something and it tells you how to how to intantiate a variable, and then it immediately goes on to show you how to make some complicated reference to it using some code.Quoted 6 lines
I agree with you though, its probably a good idea to steer newcomers
in the right direction on disk management and a few notes about doing LVM ontop of RAID being a good idea couldn't hurt. This is especially so since I've heard three mentions of people using LVM on a server without doing RAID this week alone. :-/
Quoted 6 lines
We should add something in faq page
( http://tldp.org/HOWTO/LVM-HOWTO/lvm2faq.html ), like "i've lost one of my hard drive and i can't mount my lv, did i lost everything ?" followed by a quick explanation : lvm is NOT faulty tolerant like raid1/5, if you lose a PV, you lose every LV which was (even partially) on on it.
<snip>
Not to nit-pick, but when one of my multi-PV LVs experienced a single PV failure, I did NOT lose all the data on the LV. The VG was using linear spanning, so using the --partial parameter with read-only file-system mounting (XFS in my case) I recovered all the data from the LV that wasn't physically spanned onto the failed PV.
If this was some sort of miracle and shouldn't have worked, well I suppose I'll count my blessings but it seemed perfectly reasonable at the time.
:)
-Barnaby
Re: [linux-lvm] Misleading documentation
In reply to Scott LambQuoted 1 line · Scott Lamb
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
Mirroring should be dealt with in LVM itself, by mapping more PEs on physical seperate PVs to each LE. From the little information I've seen about the new LVM mirroring, it seems that the mirrors are defined on LV level instead of LE/PE. But after some JFGI sessions nothing really enlightening has shown up.
Signature · 6 lines
//Morten Torstensen //Email: [email protected] //IM: [email protected] [email protected] And if it turns out that there is a God, I don't believe that he is evil. The worst that can be said is that he's an underachiever.
Re: [linux-lvm] Misleading documentation
Quoted 11 lines · Morten Torstensen
Quoted 4 lines · Scott Lamb
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
Mirroring should be dealt with in LVM itself, by mapping more PEs on physical seperate PVs to each LE. From the little information I've seen about the new LVM mirroring, it seems that the mirrors are defined on LV level instead of LE/PE. But after some JFGI sessions nothing really enlightening has shown up.
LVM2 does mirror by grouping multiple (hidden) logical volumes into a mirror set.
Quoted 15 lines
Signature · 14 lines
-- //Morten Torstensen //Email: [email protected] //IM: [email protected] [email protected] And if it turns out that there is a God, I don't believe that he is evil. The worst that can be said is that he's an underachiever. _______________________________________________ linux-lvm mailing list [email protected] https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Regards,
Heinz -- The LVM Guy --
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Heinz Mauelshagen Red Hat GmbH
Consulting Development Engineer Am Sonnenhang 11
Storage Development 56242 Marienrachdorf
Germany
[email protected] PHONE +49 171 7803392
FAX +49 2626 924446=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Re: [linux-lvm] Misleading documentation
Quoted 5 lines · Heinz Mauelshagen
Quoted 5 lines
Quoted 4 lines
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
Quoted 8 lines
Quoted 5 lines
Mirroring should be dealt with in LVM itself, by mapping more PEs on physical seperate PVs to each LE. From the little information I've seen about the new LVM mirroring, it seems that the mirrors are defined on LV level instead of LE/PE. But after some JFGI sessions nothing really enlightening has shown up.
LVM2 does mirror by grouping multiple (hidden) logical volumes into a mirror set.
Where would I find example commands to add a mirror, check the mirroring status, break it, reuse the other half, etc.? Is it possible to mirror a snapshot instead of the changing volume?
Signature · 2 lines
Les Mikesell
[email protected]Re: [linux-lvm] Misleading documentation
Quoted 13 lines · Les Mikesell
Quoted 11 lines · Heinz Mauelshagen
Quoted 10 lines
Quoted 4 lines
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
Mirroring should be dealt with in LVM itself, by mapping more PEs on physical seperate PVs to each LE. From the little information I've seen about the new LVM mirroring, it seems that the mirrors are defined on LV level instead of LE/PE. But after some JFGI sessions nothing really enlightening has shown up.
Quoted 5 lines
Quoted 2 lines
LVM2 does mirror by grouping multiple (hidden) logical volumes into a mirror set.
Where would I find example commands to add a mirror, check the mirroring status, break it, reuse the other half, etc.?
'man lvcreate' to start with.
Quoted 2 lines
Is it possible to mirror a snapshot instead of the changing volume?
Not yet.
Quoted 5 lines
Signature · 3 lines
-- Les Mikesell [email protected]
Signature · 12 lines
Regards,
Heinz -- The LVM Guy --
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Heinz Mauelshagen Red Hat GmbH
Consulting Development Engineer Am Sonnenhang 11
Storage Development 56242 Marienrachdorf
Germany
[email protected] PHONE +49 171 7803392
FAX +49 2626 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-Re: [linux-lvm] Misleading documentation
Quoted 13 lines · Heinz Mauelshagen
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Mirroring should be dealt with in LVM itself, by mapping more PEs on physical seperate PVs to each LE. From the little information I've seen about the new LVM mirroring, it seems that the mirrors are defined on LV level instead of LE/PE. But after some JFGI sessions nothing really enlightening has shown up.
Quoted 5 lines
Quoted 2 lines
LVM2 does mirror by grouping multiple (hidden) logical volumes into a mirror set.
Where would I find example commands to add a mirror, check the mirroring status, break it, reuse the other half, etc.?
Quoted 6 lines
'man lvcreate' to start with.
Quoted 2 lines
Is it possible to mirror a snapshot instead of the changing volume?
Not yet.
Are the parts that do work version-specific? What is the earliest version where the mirror support is reliable and has that made it into the popular distributions yet?
Signature · 2 lines
Les Mikesell [email protected]
Re: [linux-lvm] Misleading documentation (was: HDD Failure)
In reply to Scott LambScott Lamb wrote on 09/19/2006 06:40:43 PM:
Quoted 7 lines · Mark Krenz
Quoted 5 lines
LVM != RAID
You should have been doing RAID if you wanted to be able to
handle the
failure of one drive.
Quoted 5 lines
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
Good point. So if the docs make no mention of RAID or fail-over or redundency or spares, what made you assume that
LVM had such capabilities????
Re: [linux-lvm] Misleading documentation (was: HDD Failure)
Quoted 10 lines · Tobias Bluhm
Scott Lamb wrote on 09/19/2006 06:40:43 PM:
Quoted 5 lines
This is my biggest beef with LVM - why doesn't *any* of the documentation point this out? There are very few good reasons to use LVM without RAID, and "ignorance" certainly isn't among them. I don't see any mention of RAID or disk failures in the manual pages or in the HOWTO.
Good point. So if the docs make no mention of RAID or fail-over or redundency or spares, what made you assume that
LVM had such capabilities????
If that "you" is second-person singular, I didn't. I probed deep enough to discover that it did not do RAID, but there were enough suggestions that it could do redundancy that it took me a while to come to that conclusion:
(1) the many-disk examples, which would be a horrible idea otherwise
(2) the striping tuning options
(3) some vague mention of mirroring...Heinz has mentioned it in this thread, but I don't see any actual information about how it's done.
(4) ...probably more I'm forgetting...
So I don't know why you are surprised that people are assuming this and ending up without raid. (Did you see Mark Krenz's email? Three people this week, one of whom actually lost data because of it.)
All is fixable. Fabien's suggested changes are a great improvement.
Signature · 1 line
Scott Lamb <http://www.slamb.org/>
Quoted 8 lines · Nick
Hi Fabien,
Yes, just one LV - "Vol1-share".
Does this mean I've lost *everything*? I would have though I should be still be able to access everything on the two working disks?
I don't have a backup of this data.
I'm sure you will now have at least one backup of all your data.
Quoted 39 lines
Thanks, Nick
root@nibiru:~# pvdisplay --- Physical volume --- PV Name /dev/hda4 VG Name Vol1 PV Size 106.79 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 27339 Free PE 27339 Allocated PE 0 PV UUID Cq9xKF-W33m-BCLt-YyIc-EEfm-Btqc-eZLHNh--- Physical volume ---
PV Name /dev/hdb1 VG Name Vol1 PV Size 111.75 GB / not usable 0 Allocatable yes PE Size (KByte) 4096 Total PE 28609 Free PE 28609 Allocated PE 0 PV UUID hwQrhH-iXHO-Bots-6zUQ-w8JG-Nmb3-shqZiXroot@nibiru:~# vgdisplay --- Volume group --- VG Name Vol1 System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0
Quoted 8 lines
Cur PV 2 Act PV 2 VG Size 218.55 GB PE Size 4.00 MB Total PE 55948 Alloc PE / Size 0 / 0 Free PE / Size 55948 / 218.55 GB VG UUID RORj4f-LAOJ-83YS-34lD-4YRM-FKP8-8hgXLg
As you can see, you have no remaining LV in this VG. And both of your PV are empty (Total == Free).
Now if you read vgreduce manual page :
--removemissing
Removes all missing physical volumes from the volume
group and makes the volume group consistent again.
It's a good idea to run this option with --test first to
find out what it would remove before running it for real.
Any logical volumes and dependent snapshots that were
partly on the missing disks get removed completely. This includes those parts that lie on disks that are still present.
If your logical volumes spanned several disks including
the ones that are lost, you might want to try to salvage data first by activating your logical volumes with --partial as described in lvm (8).
you can see that you've just lost all of your data.
Signature · 1 line
Fabien Jakimowicz <[email protected]>
Oh well, better setup RAID this time!! Thanks for your (And the others who replied) help, I appreciate it even though it's bad news.
Nick
Quoted 6 lines · Fabien Jakimowicz
you can see that you've just lost all of your data.
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 4 lines · Nick
Oh well, better setup RAID this time!! Thanks for your (And the others who replied) help, I appreciate it even though it's bad news.
Nick
You can also run a lvm over raid configuration, but you should really test it before using it, just to exactly know what you are doing.
Signature · 1 line
Fabien Jakimowicz <[email protected]>
[linux-lvm] LVM on RAID
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
RE: [linux-lvm] LVM on RAID
I've run LVM on top of hardware raid (Areca SATA system with 8 drives in RAID-6) as well as LVM on top of software RAID-1 and RAID-5 (both IDE and SATA). LVM has been no less reliable in this configuration than in any other. IMHO, running LVM over RAID makes absolute sense. My preferred installation on a Linux box is to run s/w RAID-1 or RAID-5 with LVM on top and boot directly into an LVM root partition with initrd. I put a small (64MB) boot partition on each drive in the RAID array with an identical boot image and then any drive can boot into the kernel + initrd and start the RAID+LVM+root. I've never tried it with more sophisticated RAIDs such as your Fibre or SCSI
As always, your mileage may vary, but [IDE|SATA]+[HW|SW]RAID+LVM has worked very well for me.
- Roger
Quoted 25 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alexander Lazarevich
Sent: 19 September 2006 22:54
To: LVM general discussion and development
Subject: [linux-lvm] LVM on RAIDWe have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM on RAID
Quoted 1 line · Roger Lucas
and boot directly into an LVM root partition with initrd.
I do exactly the same thing, with multiple RAID-5 arrays and an LVM image striped over all of them at once. It works fine.
(I use initramfs for booting instead of initrd: my current boot setup is at <http://linux-raid.osdl.org/index.php/RAID_Boot>.)
Signature · 2 lines
`In typical emacs fashion, it is both absurdly ornate and still not really what one wanted.' --- jdev
Re: [linux-lvm] LVM on RAID
In reply to Alexander Lazarevich I couldn't imagine using LVM on a server without it being on RAID.
Although I've only been using LVM for less than a year, I've installed it ontop of hard and software RAID-1 on several different servers including on Xen virtual machines and it has all been fine.
Just make sure that you don't expect LVM to keep your data if you lose
a drive and are not doing RAID underneath. ;-)
On Tue, Sep 19, 2006 at 09:53:58PM GMT, Alexander Lazarevich [[email protected]] said the following:
Quoted 18 lines
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Signature · 4 lines
Mark S. Krenz IT Director Suso Technology Services, Inc. http://suso.org/
Re: [linux-lvm] LVM on RAID
In reply to Alexander LazarevichQuoted 8 lines · --On September 19, 2006 4:53:58 PM -0500 Alexander Lazarevich…
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
This is nearly the exact scenario we use. Works great. LVM is indifferent to the backend block device.
Re: [linux-lvm] LVM on RAID
In reply to Alexander LazarevichQuoted 6 lines · Alexander Lazarevich
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Quoted 6 lines
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
Works perfectly with severall machines, maybe having more interactions between raid and lvm could be a good thing (for example, while creating a new LV or extending existing one on mixed raid devices (degrated and not), it could be better if lvm don't use degrated devices first).
Signature · 1 line
Fabien Jakimowicz <[email protected]>
Re: [linux-lvm] LVM on RAID
In reply to Alexander LazarevichI have used LVM on top of software raid and ISCSI. It works well. It also helps keep track of what device is where. ISCSI does not export
its targets in the same order, some times sdb shows up as sdc.... LVM
will keep track of what is what.
Matt
Quoted 18 lines · Alexander Lazarevich
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM on RAID
I use LVM2 on software and hardware raid on a regular basis. Almost every server I configure has hardware raid and all the machines I support run LVM. No problems to report.
Quoted 7 lines · Matthew B. Brookover
I have used LVM on top of software raid and ISCSI. It works well. It also helps keep track of what device is where. ISCSI does not export its targets in the same order, some times sdb shows up as sdc.... LVM will keep track of what is what.
Quoted 4 lines · Alexander Lazarevich
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Re: [linux-lvm] LVM on RAID
In reply to Matthew B. BrookoverI should have been more clear. I'm not worried about LVM on one RAID. My questions is specifically about creating an LVM volume group ACROSS two RAID's.
For example, we have a 64bit linux server, with two different RAID devices attached to the host via Fiber. These RAID's are each 4TB volumes. The RAID is attached as /dev/sda and /dev/sdb. What I'm asking about is creating a LVM volume group, and joining /dev/sda AND /dev/sdb to that same volume group, creating the lv of 8TB (minus overhead of course), and then creating a filesystem on that lv. A 8TB filesystem, which is spanned (via LVM) across both RAID's.
Does anyone here do that? Reading all the reply's I realize I wasn't clear enough about that, and neither was anyone's responses.
Alex
Quoted 28 lines · Matthew B. Brookover
I have used LVM on top of software raid and ISCSI. It works well. It also helps keep track of what device is where. ISCSI does not export
its targets in the same order, some times sdb shows up as sdc.... LVM
will keep track of what is what.Matt
Quoted 18 lines · Alexander Lazarevich
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM on RAID
Quoted 12 lines · Alexander Lazarevich
I should have been more clear. I'm not worried about LVM on one RAID. My questions is specifically about creating an LVM volume group ACROSS two RAID's.
For example, we have a 64bit linux server, with two different RAID devices attached to the host via Fiber. These RAID's are each 4TB volumes. The RAID is attached as
/dev/sdaand/dev/sdb. What I'm asking about is creating a LVM volume group, and joining/dev/sdaAND/dev/sdbto that same volume group, creating the lv of 8TB (minus overhead of course), and then creating a filesystem on that lv. A 8TB filesystem, which is spanned (via LVM) across both RAID's.
Quoted 4 lines
Does anyone here do that? Reading all the reply's I realize I wasn't clear enough about that, and neither was anyone's responses.
Alex
I have a 'similar' configuration, with less expansive hardware ;)
i have an LVM VG built with 4 software raid arrays, each array is a two hard drives in raid1.
Signature · 1 line
Fabien Jakimowicz <[email protected]>
Re: [linux-lvm] LVM on RAID
In reply to Alexander LazarevichWe do this: we have, connected to an amd64 server via a QLogic QLA-2340 to an FC switch, several RAID devices (XServe RAID, Nexsan SATABeast) totalling 15 PVs in a single 27TB VG, with various XFS filesystems, several of which approach 8-10TB. These filesystems are then exported to a small lab of NFS clients. The vgdisplay:
--- Volume group ---
VG Name Storage2
System ID
Format lvm2 Metadata Areas 15 Metadata Sequence No 39 VG Access read/write VG Status resizable MAX LV 256 Cur LV 8 Open LV 7 Max PV 256 Cur PV 15 Act PV 15 VG Size 27.74 TB PE Size 32.00 MB Total PE 909076 Alloc PE / Size 905658 / 27.64 TB Free PE / Size 3418 / 106.81 GB
I'm only mostly (rather than completely) convinced of the righteousness of this approach, but it's been working for us, give or take a dramatic episode that we never managed to completely diagnose (the SCSI layer blew up in a way that acted like a hardware failure, but with ultimately no clear evidence that the heart of the fault wasn't an XFS corruption and no real device problem). We have lost and replaced drives in the RAID arrays, though it is a little heart-stopping.
We're doing no meaningful fail-over, other than having a second server configured and ready to replace the primary one in the case of server catastrophe (which has happened). No multipathing, no clustering.
Any commentary on the appropriateness of this approach? Occasionally I look over at ZFS and the way it collapses the software stack to a single component, and get a little jealous...
Regards,
Andy Boyko [email protected]
Quoted 45 lines · Alexander Lazarevich
I should have been more clear. I'm not worried about LVM on one RAID. My questions is specifically about creating an LVM volume group ACROSS two RAID's.
For example, we have a 64bit linux server, with two different RAID devices attached to the host via Fiber. These RAID's are each 4TB volumes. The RAID is attached as
/dev/sdaand/dev/sdb. What I'm asking about is creating a LVM volume group, and joining/dev/sdaAND/dev/sdbto that same volume group, creating the lv of 8TB (minus overhead of course), and then creating a filesystem on that lv. A 8TB filesystem, which is spanned (via LVM) across both RAID's.Does anyone here do that? Reading all the reply's I realize I wasn't clear enough about that, and neither was anyone's responses.
Alex
Quoted 5 lines · Matthew B. Brookover
I have used LVM on top of software raid and ISCSI. It works well. It also helps keep track of what device is where. ISCSI does not export
its targets in the same order, some times sdb shows up as sdc.... LVM
will keep track of what is what.Quoted 23 lines
Matt
Quoted 18 lines · Alexander Lazarevich
We have several RAID devices (16-24 drive Fiber/SCSI attached RAID) which are currently single devices on our 64bit linux servers (RHEL-4, core5). We are considering LVM'ing 2 or more of the RAID's into a LVM group. I don't doubt the reliability and robustness of LVM2 on single drives, but I worry about it on top of RAID devices.
Does anyone have any experience with LVM on to of RAID volumes, positive or negative?
Thanks,
Alex
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] LVM on RAID
In reply to Matthew B. Brookover-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
We've been doing that on a number of hosts, using Compaq SMART2, HP Smart Array, and Dell PERC II/PERC 2/PERC 4 hardware SCSI RAID cards for several years. The combination's never been a bit of trouble, and I'll add my vote to those who wouldn't think of doing a server without some sort of storage redundancy.
Likewise I wouldn't set up a server without LVM. I was spoiled by the flexibility of the Netware 4 storage management layer and am loath to give it up just because we don't run Netware anymore.
- -- Mark H. Wood, Lead System Programmer [email protected]
Typically when a software vendor says that a product is "intuitive" he means the exact opposite.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.4 (GNU/Linux)
Comment: pgpenvelope 2.10.2 - http://pgpenvelope.sourceforge.net/
iD8DBQFFEXlms/NR4JuTKG8RAiX9AKCq3hIMBz/uF3WnTNHzClnxfCdsXwCePY3K 9M8gfDdsD1ShjeJo+5RZ/JE=
=wQc5
-----END PGP SIGNATURE-----