LVM metadata corruption?
Hi
We had a LVM problem that we believe was caused by corrupted LVM-metadata. Can anyone confirm if this is a likely explanation?
1. Installed LVM
2. Created a single vg on a single device with a single lv
3. Used without problems for a while
4. Added a new device
5. Add the device to the vg, extend the lv and the filesystem
6. We experienced scsi-problems at this time.
7. Fixed scsi termination
After a reboot, mounting the filesystem(xfs) oopsed.
lvdisplay -v segfaulted, but there was no backtrace.
We recreated the lvm vg and lv, and the filesytem - everything worked fine.
My theory is that the vgextend or lvextend corrupted the lvm-metadata because of the scsi-termination problem. LVM doesn't update the metadata using a transactional log, does it? so if a write failed, the data will become corrupted? (yes, I know a copy is saved in the /etc/lvm files)
Unfortenately we needed the system back online right away, so the metadata is no gone. No way of debugging it futher.
Thank you.
Signature · 2 lines
Ragnar Kjorstad Big Storage
Quoted 4 lines · Ragnar Kj�rstad
Hi
We had a LVM problem that we believe was caused by corrupted LVM-metadata. Can anyone confirm if this is a likely explanation?
Oh, I forgot to mention LVM-version ++.
Used "lvm-0.9.1_beta7-2.4.5" patch, except one chunk that failed becaused it is also included in XFS-patch.
Binary utilities, 0.9.1_beta7.
Signature · 2 lines
Ragnar Kjorstad Big Storage
Quoted 17 lines · Ragnar Kj?rstad
Hi
We had a LVM problem that we believe was caused by corrupted LVM-metadata. Can anyone confirm if this is a likely explanation?
1. Installed LVM
2. Created a single vg on a single device with a single lv
3. Used without problems for a while
4. Added a new device
5. Add the device to the vg, extend the lv and the filesystem
6. We experienced scsi-problems at this time.
7. Fixed scsi terminationAfter a reboot, mounting the filesystem(xfs) oopsed.
lvdisplay -v segfaulted, but there was no backtrace.
Quoted 7 lines
We recreated the lvm vg and lv, and the filesytem - everything worked fine.
My theory is that the vgextend or lvextend corrupted the lvm-metadata because of the scsi-termination problem. LVM doesn't update the metadata using a transactional log, does it?
Ragnar,
no it doesn't so far.
That's a planned enhgancement for post 1.0.
Quoted 2 lines
so if a write failed, the data will
become corrupted? (yes, I know a copy is saved in the/etc/lvmfiles)
Yep, but this corruption could only take place, because the hardware wasn't able to spot the data corruption.
With that kind of silent data corruption even transaction oriented models will have problems while aiming to make nonatomic I/Os behave atomic. They can't garantee either that no silent data corruption took place.
In order to avoid such trouble after changing the SCSI HW configurations you could to write data to free areas on the disks and verify it multiple times while imposing read load to the disks.
SCSI is pushable over the edge far too easy :-(
Regards,
Heinz -- The LVM Guy --
Quoted 17 lines
Unfortenately we needed the system back online right away, so the metadata is no gone. No way of debugging it futher.
Thank you.
Signature · 7 lines
Ragnar Kjorstad Big Storage _______________________________________________ linux-lvm mailing list [email protected] http://lists.sistina.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
Signature · 12 lines
*** Software bugs are stupid.
Nevertheless it needs not so stupid people to solve them ***
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Heinz Mauelshagen Sistina Software Inc.
Senior Consultant/Developer Am Sonnenhang 11
56242 Marienrachdorf
Germany
[email protected] +49 2626 141200
FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-