LVs corrupted after pvresize
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Yohoo!
I'm currently in big troubles :-(
What happened?
I have a single VG running on three RAID arrays.
I resized one of these RAID arrays for additional 250GB. After this was done I needed to reboot for the kernel to pick up the new size. Then I went into fdisk, deleted the 8e-LVM partition and re-created a new one with the same starting sector but a later end sector. Again, for the kernel to pick up the new partition table I rebooted.
After the reboot my VG was inaccessible because it complained about a missing PV.
I checked
http://www.novell.com/coolsolutions/appnote/19386.html#DiskPermanentlyRemoved
Symptom2, Solution2.
vgcfgrestore showed the same error: "Couldn't find device with uuid ..."
So I re-created th PV with "pvcreate --uuid [...] /dev/sdd1.
vgscan and I had the new size and a running VG showing all my LVs. :)
Unfortunately ALL my LVs on the /dev/sdd1 PV are severely corrupted- the complain about missing partition tables, fsck.ext2 prints thousands of errors and so on.
I tried to reduce the PV again, but the LVs remain corrupted.
I don't expect my data to be recoverable- but I want to know what I did wrong.
Though, if you can recover my LVs it would be fine ;-)
Any ideas?
Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD8DBQFI96130XNIYlAXmzsRAqMGAKDK13dabQD/LFGl460R410PQlNFCQCgt7ZT iVCXd/QCUoSQIdfTaODAbPU=
=XMWS
-----END PGP SIGNATURE-----
Quoted 6 lines · Christian V�lker
I'm currently in big troubles :-(
What happened?I have a single VG running on three RAID arrays.
I resized one of these RAID arrays for additional 250GB. After this was
Resizing a RAID-5 array is often not what you think, and doing so often scrambles the data, depending on what RAID implementation you are using. That is why I stick with RAID-1. Resizing is simple. In any case, LVM was not your problem.
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Yohoo!
|> I resized one of these RAID arrays for additional 250GB. After this was | Resizing a RAID-5 array is often not what you think, and doing so often | scrambles the data, depending on what RAID implementation you are using.
I'm using a hardware RAID 3ware 9500S Controller. And this guy offers migration from RAID10 (4disks) to RAID5 (4disks, but with larger capacity). And I already did this task several times on these types of controllers.
| In any case, LVM was not your problem.
This answer is too simple to be true.
After the resize the partition and the PV were recognized correctly! So I'm pretty sure the migration was ok.
But after the deletion of the partion, recreation and pvcreate everything went wrong. So for me it is obvious, LVM had an issue. But which one?
Greets
Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD8DBQFI+A770XNIYlAXmzsRAicJAKCpogPKLWhuOtABRI/cBthIRVwlHACZAcoi b7+z+RaIKUKnui+b8XLqtw0=
=TSoe
-----END PGP SIGNATURE-----
Quoted 11 lines · Christian Völker
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1Yohoo!
|> I resized one of these RAID arrays for additional 250GB. After this was | Resizing a RAID-5 array is often not what you think, and doing so often | scrambles the data, depending on what RAID implementation you are using.I'm using a hardware RAID 3ware 9500S Controller. And this guy offers migration from RAID10 (4disks) to RAID5 (4disks, but with larger capacity). And I already did this task several times on these types of controllers.
That wasn't really the issue raised. Extending raids by adding new disks doesn't do a balanced increase. In other words, you don't get a very well functioning raid. Doing true raid migration is not straight forward. So unless you fully restructure all data, you aren't getting the performance you should be getting.
Quoted 2 lines
| In any case, LVM was not your problem.
This answer is too simple to be true.
Not really. Your approach was very poor.
Quoted 1 line
After the resize the partition and the PV were recognized correctly!
Because at that time, the PVs were intact.
Quoted 2 lines
So
I'm pretty sure the migration was ok.
Again, you misunderstood Stuart's comment. He didn't say your RAID migration didn't work. He said it wasn't optimal.
Quoted 1 line
But after the deletion of the partion, recreation and pvcreate
That's the error. There's a pvresize for a reason. You also need to use the same UUID. Removing the PV makes your vg inconsistent and you've basically @#$@# up LVM.
Quoted 2 lines
everything went wrong. So for me it is obvious, LVM had an issue. But which one?
No - your use of LVM was wrong. Not LVM itself.
---
Regards
Peter Larsen
"Right now I'm having amnesia and deja vu at the same time."
-- Steven Wright
Quoted 2 lines · Peter Larsen
Again, you misunderstood Stuart's comment. He didn't say your RAID migration didn't work. He said it wasn't optimal.
Actually, I *was* hypothesizing that the RAID migration scrambled the data. But apparently, that was not the problem because:
Quoted 3 lines
Quoted 1 line
But after the deletion of the partion, recreation and pvcreate
That's the error. There's a pvresize for a reason. You also need to use
Yes, the pvcreate just hosed your metadata. If you have a backup of the metadata (somewhere other than an affected LV), you can restore it, and then do the pvresize.
Quoted 1 line
No - your use of LVM was wrong. Not LVM itself.
Having used AIX LVM, I can say that Linux LVM has a way to go before it reaches that level of ease of use. But it keeps getting better, and I get to participate!
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.
Quoted 3 lines · Stuart D. Gathman
Having used AIX LVM, I can say that Linux LVM has a way to go before it reaches that level of ease of use. But it keeps getting better, and I get to participate!
I'm currious - my exposure to LVM like systems is mainly on Linux. Where's the catch? I find the basic LVM command structure pretty straight forward and easy. There's even a couple of GUI's available now to make the LVM management easier.
Can you give an example or two on how the LVM differs on the user-friendliness areas?
---
Regards
Peter Larsen
"Here's something to think about: How come you never see a headline like `Psychic Wins Lottery'?"
-- Jay Leno
Quoted 2 lines · Peter Larsen
Can you give an example or two on how the LVM differs on the user-friendliness areas?
Say you want to mirror a LV, then you just do a "mklvcopy mylvname 2" to have 2 copies of data (one extra, like raid-1). If you want to migrate your disks to a new system, add the new PV to your VG and do a "mklvcopy mylvname 3 newpv", then when it finish remove your old mirrors and delete the old PVs.
The AIX concept of physical extent/logical extent mapping is sorely missed on linux.
Generally speaking, LVM on AIX is also more mature. Exporting and importing VGs (to move VGs between systems) works better in my experience, clustering is more stable, but that is also something where YMMV can show.
Managing mirrors, data placement and hotspot detection are what I mainly miss on Linux.
Signature · 6 lines
//Morten Torstensen //Email: [email protected] //IM: [email protected] [email protected] And if it turns out that there is a God, I don't believe that he is evil. The worst that can be said is that he's an underachiever.
I thought lvdisplay --maps gave you the PV to LV mappings.
Larry Dickson
Cutting Edge Networked Storage
Quoted 38 lines · Morten Torstensen
Quoted 3 lines · Peter Larsen
Can you give an example or two on how the LVM differs on the user-friendliness areas?
Say you want to mirror a LV, then you just do a "mklvcopy mylvname 2" to have 2 copies of data (one extra, like raid-1). If you want to migrate your disks to a new system, add the new PV to your VG and do a "mklvcopy mylvname 3 newpv", then when it finish remove your old mirrors and delete the old PVs.
The AIX concept of physical extent/logical extent mapping is sorely missed on linux.
Generally speaking, LVM on AIX is also more mature. Exporting and importing VGs (to move VGs between systems) works better in my experience, clustering is more stable, but that is also something where YMMV can show.
Managing mirrors, data placement and hotspot detection are what I mainly miss on Linux.
--
//Morten Torstensen
//Email: [email protected]
//IM: [email protected] [email protected]And if it turns out that there is a God, I don't believe that he is evil. The worst that can be said is that he's an underachiever.
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Quoted 4 lines · Larry Dickson
I thought lvdisplay --maps gave you the PV to LV mappings.
Quoted 2 lines
The AIX concept of physical extent/logical extent mapping is sorely missed on linux.
He misses the AIX *flavor* of PE to LE mapping (not PV to LV). In AIX, each LE is mapped to 1 or more (for mirroring) PEs in arbitrary locations. Each LE is tracked independently in AIX - the focus is on LEs, with LVs being simply a collection of LEs. If a PV dies, some LEs in a LV may have mirrors and others not. When bringing a PV back on line, each LE is resynchronized independently.
BTW, AIX also has "migratepv mylv srcpv dstpv" which creates a copy of each LE on dstpv, then removes a copy of each LE on srcpv.
Why did we leave? It had nothing to do with quality of hardware or software from IBM. As a small VAR, we couldn't deal with IBM directly, and it was too expensive to deal with additional middlemen who knew less than we did and couldn't provide support.
Additionally, I love diving into the source when I get spare moments - something you can't do with AIX LVM.
Solaris on Sun hardware looks attractive, offering a robust LVM/FS (ZFS), fully supported enterprise hardware, and open source. But it is past the price point of our current customers, and Solaris on PCs doesn't justify the learning curve compared to Linux which we are already familiar with.
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.