Skip to contentSkip to conversation

my PV's disappearing

linux-lvm Richard Mollel 3 participants 6 messages Nov 13, 2006 – Nov 21, 2006

lore
Share & tools
lore
#0 lore raw

For some unknown reasons...my LVM PV's which have worked for several years are dropping!
Here is one example....the node is still up, the partition usable...but pvscan/pvdisplay don't display anything.
I know if i reboot, i wont be able to access this partition anymore! It has happened to one filesystem already. Any ideas what is going on here?

/dev/sdb1: unknown volume type
/dev/sdc1: unknown volume type
/dev/sdd1: unknown volume type
No physical volume label read from /dev/sdb1 Failed to read physical volume "/dev/sdb1" No physical volume label read from /dev/sdc1 Failed to read physical volume "/dev/sdc1" No physical volume label read from /dev/sdd1 Failed to read physical volume "/dev/sdd1"

Well, before i finished typing this message...the node locked up (un-related issue) and upon reboot...it does not find any PV's, thus VG's and LV's.

I followed a recovery procedure from
http://codeworks.gnomedia.com/archives/2005/general/lvm_recovery/
on a 5TB reiserfs filesystem...and the FS is still hosed, with --rebuild-tree failing. I will post a separate email for that. Now i am wondering how to recover this one successfully...and also, why is it happening? i have another node, that is still up, with same symptoms..i know as soon as i reboot, it wont work.

_Thanks

Rick...

____________________________________________________________________________________
Want to start your own business?
Learn how on Yahoo! Small Business.
http://smallbusiness.yahoo.com/r-index

#1 lore raw
Quoted 2 lines · Richard Mollel

For some unknown reasons...my LVM PV's which have
worked for several years are dropping!

A simple rule to follow when reporting a problem:
Tell people the precise version of the software you're using - and the distro!

And if things suddenly stopped working, tell people about anything possibly relevant that might have changed recently on the system.

Alasdair

Signature · 1 line
[email protected]
#2 lore raw

Sorry...
I am running Mandriva 2006.0, with LVM2 (lvm2-2.01.09-1mdk). I am in the process of migrating to Redhat ES 4.0. I have external storage, EonStor, virtually partitioned into 2+2+1TB to overcome 2TB limitation...and brought together with LVM with PV's sdb, sdc and sdd.
I have migrated one such partition to Redhat successfully, simple vgexport and vgimport on the other end after moving physical device.
So i rebooted one box to add storage so as to migrate data on reiserfs to new storage, lvm2+ext3. Only then, the old LVM group did not come up. pvscan shows "no matching devices". So i had to go thru LVM recovery...which has fsck'ed my LVM partition, it is no longer mountable, reiserfs --rebuild-tree could not complete, not finding metadata.
So i started looking at other nodes, 4 of them, all Mandriva 2006.0 with LVM2....i perform pvscan once, it shows output, then probably pvdisplay, and pvscan again...No output! Only option at that point has been to backup data as fast as possible...and upon reboot, all PV's as expected are not recognized,.....and going thru lvm_recovery (as detailed on
http://codeworks.gnomedia.com/archives/2005/general/lvm_recovery/)
messes up reiserfs completely.
I have ran pvscan on 4 different systems, only to kill LVM...it seems somehow it is removing LVM label from physical partition....
I have not been able to pin-point where the problem might be. On the external Raid5 device, it does not show any signs of fault...besides, we are talking of 4 different systems and four different (same make) storage devices.
As for mandriva 2006.0, i simply take all updates it recommends that i am missing...and i cannot see anything remotely close to causing problems. I have osiris running to detect rpm changes...and recently i can only see ssh related stuff, running really concerning.
So i am hurriedly backing up all filesystems that are on LVM....since it seems to be to common denominator...though i dont know why....

Just before i sent this out...i saw this thread::
https://www.redhat.com/archives/linux-lvm/2005-May/msg00032.html

And that was exactly my problem!! Not sure why!! I have now been able to perform::
losetup /dev/loop0 /dev/sdb
losetup /dev/loop1 /dev/sdc
losetup /dev/loop2 /dev/sdd

and, performed : vgchange -a y
(it complained of duplicate uuid's, recreated on sdb1,sdc1 and sdd1 since pvcreate was complaining of /dev/sdb,sdc & sdd not found!)
I have been able to mount my reiserfs, so i can continue with migration.
Issue which i have to take up with Mandriva guys is why the failure all of a sudden? and do i need to be performing losetup everytime? My original configuration had used whole disks...

_Thanks

Quoted 10 lines · --- Alasdair G Kergon
Quoted 2 lines · Richard Mollel

For some unknown reasons...my LVM PV's which have
worked for several years are dropping!

A simple rule to follow when reporting a problem:
Tell people the precise version of the software
you're using - and the
distro!

Quoted 16 lines

And if things suddenly stopped working, tell people about anything possibly
relevant that might have changed recently on the system.

Alasdair

Signature · 8 lines
[email protected]

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at
http://tldp.org/HOWTO/LVM-HOWTO/

____________________________________________________________________________________
Cheap talk?
Check out Yahoo! Messenger's low PC-to-Phone call rates.
http://voice.yahoo.com

#3 lore raw
Quoted 10 lines · Richard Mollel

Just before i sent this out...i saw this thread::
https://www.redhat.com/archives/linux-lvm/2005-May/msg00032.html

And that was exactly my problem!! Not sure why!! I have now been able to perform::
losetup /dev/loop0 /dev/sdb
losetup /dev/loop1 /dev/sdc
losetup /dev/loop2 /dev/sdd

can you please clarify if you are really using whole disks or did you by chance partition the devices.
the message you cite was the fix for someone that did partition the disk after creating the PVs and filling LVM with data.

if the device holds a partition table lvm commands will ignore the whole disk and try to find an LVM label on the partition. the loosetup trick will fool lvm tools (loop device are not partitionable)

Quoted 4 lines

and, performed : vgchange -a y
(it complained of duplicate uuid's, recreated on sdb1,sdc1 and sdd1 since pvcreate was complaining of /dev/sdb,sdc & sdd not found!)

pvcreate? did you pvcreate? on the partition?

Quoted 7 lines

I have been able to mount my reiserfs, so i can continue with migration.
Issue which i have to take up with Mandriva guys is why the failure all of a sudden? and do i need to be performing losetup everytime? My original configuration had used whole disks...

i did not see anything related to that on mandriva bugzilla, but in any case if the issue is that the whole device was partitioned after lvm creation there is not much i can do about it.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#4 lore raw
Quoted 1 line · Luca Berra

On Tue, Nov 14, 2006 at 09:56:22AM -0800, Richard Mollel wrote:

please ignore, i just found the rest of the thread.

L.

Signature · 6 lines
Luca Berra -- [email protected]
       Communication Media & Services S.r.l.
/"\
\ /     ASCII RIBBON CAMPAIGN
 X        AGAINST HTML MAIL
/ \
#5 lore raw
In reply to Luca Berra
Quoted 20 lines · --- Luca Berra
Quoted 2 lines · Richard Mollel

Just before i sent this out...i saw this thread::

https://www.redhat.com/archives/linux-lvm/2005-May/msg00032.html

Quoted 7 lines

And that was exactly my problem!! Not sure why!! I have now been able to perform::
losetup /dev/loop0 /dev/sdb
losetup /dev/loop1 /dev/sdc
losetup /dev/loop2 /dev/sdd

can you please clarify if you are really using whole disks or did you by
chance partition the devices.
the message you cite was the fix for someone that did partition the disk
after creating the PVs and filling LVM with data.

Quoted 7 lines

if the device holds a partition table lvm commands will ignore the
whole disk and try to find an LVM label on the partition.
the loosetup trick will fool lvm tools (loop device are not
partitionable)

Hi Luca,
It created these LVM volumes several years back on LVM1...and to admit, were my first LVM's experience then. So, i followed religiously the instructions, whereby i had to create a partition, label it type "8e" (LVM) on the partition. For some reason, back then, i then performed pvcreate on whole device (/dev/sdb, sdc,sdd) and not their partition. It all worked OK for years, even after OS upgrade from Mandrake 10.0 to Mandriva 2006.0, thus changing LVM to LVM2. Until i ran s fresh pvscan was when i broke the configuration. It seems to have been working on cache (/etc/lvm/.cache) configuration, and a pvscan forced a flush of that.

Quoted 17 lines
Quoted 4 lines

and, performed : vgchange -a y
(it complained of duplicate uuid's, recreated on sdb1,sdc1 and sdd1 since pvcreate was complaining

of

Quoted 1 line

/dev/sdb,sdc & sdd not found!)

pvcreate? did you pvcreate? on the partition?

Quoted 4 lines

I have been able to mount my reiserfs, so i can continue with migration.
Issue which i have to take up with Mandriva guys is why the failure all of a sudden? and do i need to

be

Quoted 3 lines

performing losetup everytime? My original configuration had used whole disks...

Quoted 5 lines

i did not see anything related to that on mandriva bugzilla, but in any
case if the issue is that the whole device was partitioned after lvm
creation there is not much i can do about it.

The whole device was partitioned before LVM creation...

Quoted 18 lines

L.

Signature · 13 lines
Luca Berra -- [email protected]
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at
http://tldp.org/HOWTO/LVM-HOWTO/

____________________________________________________________________________________
Sponsored Link

Compare mortgage rates for today.
Get up to 5 free quotes. www2.nextag.com