hd read error / some PEs won't pvmove - how to resolve?
Hi people,
I've searched the net, the mailing lists, and the documentation for the solution to this, but I don't see anything obvious.
I had partitions on 45GB and 20GB drives in a volume group. Problems developed with the 20GB drive, where certain blocks became undreadable. I added a 13GB drive, and juggled some files to come up with another 8GB on the 45GB, and pvmoved all the data off the 20 GB drive.
All except for 8 PEs, that is. pvmove fails with an error when I try to move those particular PEs where the read errors are occuring.
I don't need to worry about recovering that data. None of the affected files are important enough to worry about. But I do need to remove the PV on the 20GB drive from the VG, which I can't do while these 8 PEs are still assigned to my LV.
So! The only hint at a solution I've found for this in the mailing lists is to replace the 20GB drive with an equal or greater size drive, and use vgcfgrestore to restore the PV information on that drive. I would assume the PEs will still be allocated, but they'll just be empty of data. From that point on, recovery is a filesystem layer issue.
The problem is, I don't have a 20GB or greater drive lying around to replace this one with until I get this one replaced on warranty, which I can't send in for warranty until I can remove it from the volume group. Chicken and egg.
The other option I can consider is a painful piece-by-piece move of the data from this VG into a new VG, gradually reducing one VG and expanding the other until there's no data left on the first VG. If there's no better option, I'll do this, but I'm not excited about it. Can I do this within the existing volume group, or will the currently allocated PEs cause problems later on when I try to remove that failed drive? (vgsplit?)
Any help appreciated, much thanks.
Rick
Rick, you write:
Quoted 2 lines
All except for 8 PEs, that is. pvmove fails with an error when I try to move those particular PEs where the read errors are occuring.
Yes, AIX always had this same problem. However, because you could mirror inside the LVM on AIX, you could mirror the bad LEs, and then remove the old PV because there were 2 copies of that LE. Generally you were better off throwing the whole filesystem out and restoring from a backup, if you didn't have a mirrored LV in the first place. AIX also warned you at the time of temporary (rather than permanent) read errors, so sometimes you had a chance to replace the disk before it totally crapped out.
Quoted 4 lines
I don't need to worry about recovering that data. None of the affected files are important enough to worry about. But I do need to remove the PV on the 20GB drive from the VG, which I can't do while these 8 PEs are still assigned to my LV.
If the bad PEs are in the middle of the filesystem, you are pretty much hosed in terms of recovery for this filesystem, regardless of whether you can force LVM to move the PEs for you (if you are lucky fsck may be able to recover, but it really depends on where/how big the bad PEs are and how you initially configured the filesystem). Hopefully the bad PEs are only in one LV.
I would simply back up what files you can (via tar) and make a new LV and filesystem on the other disk. You _should_ be able to delete the LV. Once the LV using those PEs is gone is gone you should be able to remove the PV.
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert