recover lost VG (VG data differs between PVs)

Help! :)

I think I messed up my VG trying to recover it... :( this is what I did.

First, I believe grub erased the first sector of a whole-disk PV (/dev/hde). luckily I had a lvm1 backup in /etc/lvmconf. Using (lvm1's) pvcreate and vgcfgrestore I managed to get my VG back online. Good.

Then I wanted to move the data from the PV on /dev/hde to a new PV, so I created a new PV (/dev/hdf3) and used vgextend to extend the VG. Without thinking I rebooted the computer and worse: I managed to overwrite that first sectore of /dev/hde again! This time I don't have any backups of the new configuration (still have the old ones), but I didn't move the PV or resized the fs on the vg.

I removed the newly added VG and using vgcfgrestore to recover the metadata on /dev/hde. but now vgdisplay tells me

VG data differs between PVs /dev/hde and /dev/hdf3 Volume group "datavg" doesn't exist

Any ideas on how to proceed next are highly appreciated...

Jelle

So hde should be in the VG at the moment, but not hdf3? Add a filter to lvm.conf so lvm only looks at hde. Run vgscan and see check if things look OK
eg vgcfgbackup and read the text backup file. If things *are* OK like that, then pvremove hdf3, remove the filter & vgscan.

Alasdair

[email protected]

Yes.

OK, I shall try that tomorrow. But a quick look at vgdisplay -P -v shows:

VG data differs between PVs /dev/hde and /dev/hdf3 Partial mode. Incomplete volume groups will be activated read-only.
Finding all volume groups
Finding volume group "datavg"
--- Volume group ---

  VG Name               datavg
  System ID             trigger1078793806
  Format                lvm1
  VG Access             read
  VG Status             resizable
  MAX LV                256
  Cur LV                1
  Open LV               0
  Max PV                256
  Cur PV                3
  Act PV                3
  VG Size               284.09 GB
  PE Size               32.00 MB
  Total PE              9091
  Alloc PE / Size       9091 / 284.09 GB
  Free  PE / Size       0 / 0   
  VG UUID               f49snm-PqWg-hfnp-Ms2k-dgyC-M72S-YqScuJ

--- Logical volume ---

  LV Name                /dev/datavg/datalv
  VG Name                datavg
  LV UUID                000000-0000-0000-0000-0000-0000-000000
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                284.09 GB
  Current LE             9091
  Segments               3
  Allocation             normal
  Read ahead sectors     1024

--- Physical volumes ---

  PV Name               /dev/hde     
  PV UUID               ifxE7h-Z1c6-9mox-VYz5-XTFy-aBw2-vAkdk4
  PV Status             allocatable
  Total PE / Free PE    1831 / 0

  PV Name               /dev/hdg3     
  PV UUID               3AnIvc-DdzR-JqtO-hYkV-3vqw-4Qm0-FBKsCS
  PV Status             allocatable
  Total PE / Free PE    3580 / 0

  PV Name               /dev/hdh     
  PV UUID               oxjwHR-Cm3S-d0GZ-5LJy-xqa7-X4X6-RAzX02
  PV Status             allocatable
  Total PE / Free PE    3680 / 0

All PVs that should be in the VG are there (hdf3 was the one I added (and removed again) later to move hde to).

So, *this* output looks OK to me as far as the PVs are concerned, but I shall have a look at the text backup file as you suggested.

thanks for the help
Jelle.

Hi Alasdair,

precisely.

Wow! *phew* that did the job! THANKS A LOT!!

Just a question out of curiosity, what does "VG data differs..." mean, and why does filtering hdf3 help (I thought vgreduce removed that PV from the VG)?

well, again, thanks a zillion
Jelle.

LVM2 checks its metadata is consistent on disk every time you access
a VG. LVM1 caches its metadata and has less-stringent checks.

That message means it found an anomaly that it couldn't
fix - the metadata on 3 of your devices formed a consistent VG after you restored the backup, but the metadata on the 4th device hdf3 still said it was part of the VG.

Filtering tells LVM2 to ignore hdf3 completely, then pvremove hdf3 wipes LVM metadata from it.

Alasdair

[email protected]

Nice system, this LVM. Easy to use, very stable, fool proof. I love it!

Thanks a lot for the clean explanation and the help with getting my VG back online. You don't know how happy I am with the positive outcome of this accident :-)

cheers!
Jelle