Hard disk failure, restore problem

Hi,
I've recently encountered some problems with my LVM.

I had (and still have) a bad hd /dev/hda (used part. /dev/hda1)

I started:
pvmove /dev/hda1 /dev/hdf1
yet the hd gave an error and I had to reboot.

I have a metadata backup made before I've started the move.

The volume group name is bigspace, yet the 'dir' /dev/bigspace/bigspace1 is also gone...

now I got
pvscan -- reading all physical volumes (this may take a while...)

pvscan -- inactive PV "/dev/hdg1" is in no VG  [74.53 GB]
pvscan -- inactive PV "/dev/hde1" is in no VG  [149.05 GB]
pvscan -- inactive PV "/dev/hdf1" is in no VG  [149.05 GB]
pvscan -- inactive PV "/dev/hdc1" is in no VG  [111.79 GB]
pvscan -- inactive PV "/dev/hdd1" is in no VG  [74.53 GB]
pvscan -- inactive PV "/dev/hda1" is in no VG  [74.53 GB]
pvscan -- inactive PV "/dev/hdb1" is in no VG  [74.53 GB]

pvscan -- total: 7 [708 GB] / in use: 0 [0] / in no VG: 7 [708 GB]

yet if I do the restore I get

I also did a pvcreate -ff /dev/hdx1 for every disk... I've read somewhere that should fix it. yet id did not.

pvscan -- inactive PV "/dev/hda1" is associated to unknown VG "bigspace" (run vgscan)

vgscan gives
vgscan -- reading all physical volumes (this may take a while...)
vgscan -- ERROR "vg_read_with_pv_and_lv(): current PV" can't get data of volume group "bigspace" from physical volume(s) vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
vgscan -- WARNING: This program does not do a VGDA backup of your volume group

there is a LOT of data on this array of disk (as you can imagine)

sometimes, hda is working good, so i hope it want to work long enough to restore the data.

My guess now is to make a new vg and add those disks... but I want to be sure, so I don't lose the data.

Any suggestions to recover the data or to replace the broken harddisk with a new one with as little data loss as possible.

Regards,
Thomas.

Thomas,

you ran 'pvcreate -ff' *and* vgcfgrestore ... on all PVs ok ? pvscan output looks like the vgcfgrestore went wrong.

Yes, you need "pvcreate -ff ..." before you vgcfgrestore to the devices.

If you want to activate "bigspace" just in order to backup date *and* remove/recreate it afterwards, you might want to use the following workaround:

o identify your actual metadata backup (say, bigspace.conf)

o mkdir /etc/lvmtab.d

o cp bigspace.conf /etc/lvmtab.d/bigspace

o echo -ne "bigspace\0" > /etc/lvmtab

o vgchange -ay

That won't work, because the PVs are falgged *new* after pvcreate -ff and you will just have empty PVs that way.

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Red Hat GmbH
Consulting Development Engineer                   Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
[email protected]                            +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Hello,

I've done

after being sure to restore the metadata I have

pvscan -- reading all physical volumes (this may take a while...) pvscan -- inactive PV "/dev/hdg1" of VG "bigspace" [74.47 GB / 768 MB free] pvscan -- inactive PV "/dev/hde1" of VG "bigspace" [149 GB / 0 free] pvscan -- inactive PV "/dev/hdf1" of VG "bigspace" [149 GB / 139.16 GB free] pvscan -- inactive PV "/dev/hdc1" of VG "bigspace" [111.75 GB / 0 free] pvscan -- inactive PV "/dev/hdd1" of VG "bigspace" [74.47 GB / 0 free] pvscan -- inactive PV "/dev/hda1" of VG "bigspace" [74.47 GB / 8.22 GB free] pvscan -- inactive PV "/dev/hdb1" of VG "bigspace" [74.47 GB / 0 free] pvscan -- total: 7 [708 GB] / in use: 7 [708 GB] / in no VG: 0 [0]

vgscan -- reading all physical volumes (this may take a while...) vgscan -- only found 17904 of 17876 LEs for LV /dev/bigspace/bigspace1 (0) vgscan -- ERROR "vg_read_with_pv_and_lv(): allocated LE of LV" can't get data of volume group "bigspace" from physical volume(s)
vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created vgscan -- WARNING: This program does not do a VGDA backup of your volume group

=> after the vgscan the echo -ne "bigspace\0" > /etc/lvmtab is gone gone again....

[root@ment root]# lvscan
lvscan -- ERROR: VGDA in kernel and lvmtab are NOT consistent; please run vgscan

after doing the vgscan, I get

pvscan -- reading all physical volumes (this may take a while...) pvscan -- inactive PV "/dev/hdg1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- inactive PV "/dev/hde1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- inactive PV "/dev/hdf1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- inactive PV "/dev/hdc1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- inactive PV "/dev/hdd1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- inactive PV "/dev/hda1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- inactive PV "/dev/hdb1" is associated to unknown VG "bigspace" (run vgscan)
pvscan -- total: 7 [708 GB] / in use: 7 [708 GB] / in no VG: 0 [0]

again.

the /dev/bigspace/bigspace1 doesn't exists, so I can't mount.

should I recreate bigspace and force an import ?

I 've alse rebooted once after the vgchange -ay, but he also didn't find the bigspace.

thanx again for your help!!

kind regards,
Thomas.

Thomas,

don't run vgscan after the procedure below.

Remember this is a workaround *just* to get you to the point to be able to backup your data and recreate your VG.

Run vgmknodes to create your device nodes in /dev/bigspace/ after step "vgchange -ay" below.

Regards,
Heinz -- The LVM Guy --

*** Software bugs are stupid.
Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen Red Hat GmbH
Consulting Development Engineer Am Sonnenhang 11
56242 Marienrachdorf Germany
[email protected] +49 2626 141200

                                                       FAX 924446

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-