Determining filename from absolute sector failure in raid0
I have a raid0 comprised of 4x identical 300Gb drives
Disk /dev/hdf: 300.0 GB, 300001443840 bytes
255 heads, 63 sectors/track, 36473 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0009f25f
Device Boot Start End Blocks Id System /dev/hdf1 * 1 36473 292969341 fd Linux raid
autodetect
md1 : active raid0 hdk1[0] hdg1[2] hdf1[1] hde1[3]
1171876864 blocks 256k chunks
hdf recently wobbled and the kernel happily dumped out the following message (about 20 odd times)
hdf: dma_intr: status=0x51 { DriveReady SeekComplete Error }
hdf: dma_intr: error=0x40 { UncorrectableError }, LBAsect=216521307, high=12, low=15194715, sector=216521303
ide: failed opcode was: unknownend_request: I/O error, dev hdf, sector 216521303
terrific.
smartctl shows
5 Reallocated_Sector_Ct 0x0033 252 252 063 Pre-fail
Always - 16
196 Reallocated_Event_Count 0x0008 251 251 000 Old_age
Offline - 2
197 Current_Pending_Sector 0x0008 253 253 000 Old_age
Offline - 1
198 Offline_Uncorrectable 0x0008 252 252 000 Old_age
Offline - 1
so while a few sectors have been remapped the sector remapping table is barely used, so I'm reasonably confident the disk is 'ok' in terms of normal operation
Now my question about all this is how the hell do I determine what file it was trying to access? (I just want to be sure that whatever file it was fiddling with isn't a steaming pile of poo now)
normally if it were a single disk it would be reasonably straightforward to work out what file was being used from debugfs: icheck <block>, ncheck <inode from previews operation>
the issue I have with raid0 is, exactly what block should I be pointing debugfs at since the kernel has given an absolute address on a physical disk and no logical information as to where in the FS it was at the time ?
tune2fs -l /dev/md1
Block count: 292969216 Block size: 4096
mdadm --examine /dev/hdf1/dev/hdf1:
Magic : a92b4efc
Version : 00.90.00
UUID : 2132d2e6:a2955ac2:6ad2f3dc:52b3dfd6 Creation Time : Wed May 26 18:34:06 2004
Raid Level : raid0
Used Dev Size : 292969216 (279.40 GiB 300.00 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 1
Update Time : Wed May 26 18:34:06 2004
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : bd5e5781 - correct
Events : 0.5
Chunk Size : 256K
Number Major Minor RaidDevice State
this 1 33 65 1 active sync /dev/hdf1
0 0 33 1 0 active sync /dev/hde1 1 1 33 65 1 active sync /dev/hdf1 2 2 34 1 2 active sync /dev/hdg1 3 3 34 65 3 active sync <---
empty. bug? (expected /dev/hdk1)
mdadm --version
mdadm - v2.6.2 - 21st May 2007
Would I be correct in assuming that I start my offset from hde1? do I need some other funky math to account for striping/blocking?
Or I could just say 'to hell with it' and listen to elevator music all day.
Phil
=--=
Quoted 42 lines
-----Original Message-----
From: [email protected] [mailto:linux-raid- [email protected]] On Behalf Of Bryce
Sent: Tuesday, October 07, 2008 9:03 AM
To: Linux RAID Mailing List
Subject: Determining filename from absolute sector failure in raid0I have a raid0 comprised of 4x identical 300Gb drives
Disk
/dev/hdf: 300.0 GB, 300001443840 bytes
255 heads, 63 sectors/track, 36473 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier:0x0009f25fDevice Boot Start End Blocks Id System /dev/hdf1 * 1 36473 292969341 fd Linux raidautodetect
md1 : active raid0 hdk1[0] hdg1[2] hdf1[1] hde1[3]
1171876864 blocks 256k chunkshdf recently wobbled and the kernel happily dumped out the following message (about 20 odd times)
hdf:
dma_intr: status=0x51 { DriveReady SeekComplete Error }
hdf:dma_intr: error=0x40 { UncorrectableError }, LBAsect=216521307, high=12, low=15194715, sector=216521303
ide: failed opcode was: unknownend_request: I/O error, dev hdf, sector 216521303terrific.
smartctl shows
5Reallocated_Sector_Ct0x0033252 252 063 Pre-fail
Always - 16
196Reallocated_Event_Count0x0008251 251 000Old_age
Offline - 2
197Current_Pending_Sector0x0008253 253 000Old_age
Offline - 1
198Offline_Uncorrectable0x0008252 252 000Old_age
Offline - 1
Quoted 1 line
so while a few sectors have been remapped the sector remapping table
is
Quoted 3 lines
barely used, so I'm reasonably confident the disk is 'ok' in terms of normal operation
Now my question about all this is how the hell do I determine what
file
Quoted 8 lines
it was trying to access? (I just want to be sure that whatever file it was fiddling with isn't a steaming pile of poo now)
normally if it were a single disk it would be reasonably straightforward
to work out what file was being used from debugfs: icheck <block>, ncheck <inode from previews operation>
the issue I have with raid0 is, exactly what block should I be
pointing
Quoted 1 line
debugfs at since the kernel has given an absolute address on a
physical
Quoted 49 lines
disk and no logical information as to where in the FS it was at the time ?
tune2fs -l
/dev/md1Block count: 292969216 Block size: 4096mdadm --examine
/dev/hdf1/dev/hdf1:Magic : a92b4efc Version : 00.90.00 UUID : 2132d2e6:a2955ac2:6ad2f3dc:52b3dfd6Creation Time : Wed May 26 18:34:06 2004
Raid Level : raid0
Used Dev Size : 292969216 (279.40 GiB 300.00 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 1Update Time : Wed May 26 18:34:06 2004
State : active
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum :bd5e5781- correct
Events : 0.5Chunk Size : 256K
Number Major Minor RaidDevice State
this 1 33 65 1 active sync/dev/hdf10 0 33 1 0 active sync /dev/hde1 1 1 33 65 1 active sync /dev/hdf1 2 2 34 1 2 active sync /dev/hdg1 3 3 34 65 3 active sync <---empty. bug? (expected
/dev/hdk1)mdadm --version
mdadm - v2.6.2 - 21st May 2007Would I be correct in assuming that I start my offset from hde1? do I need some other funky math to account for striping/blocking?
Or I could just say 'to hell with it' and listen to elevator music all day.
Quoted 8 lines
Phil
=--=--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
------------------------------------------------------------------------
--------------------
What you need is a utility program that translates a physical device & offset and converts that into a logical block in md0. Then Something that maps a logical block in md0 to a file in a filesystem.
Unfortunately, I am not aware of any off-the-shelf utility to do this, but put me down for being very interested in such a solution myself. David
Quoted 6 lines · David Lethe
------------------------------------------------
--------------------
What you need is a utility program that translates a physical device & offset and converts that into a logical block in md0. Then Something that maps a logical block in md0 to a file in a filesystem.
Quoted 6 lines
Unfortunately, I am not aware of any off-the-shelf utility to do this, but put me down for being very interested in such a solution myself. David
well another part of the 'what do i do now?' question is that /proc/mdstat shows the drives assembled thusly hdk1[0] hdg1[2] hdf1[1] hde1[3]
and mdadm --examine shows the order as hde 0 hdf 1 hdg 2 hdk 3 err whats the actual order?
Phil
=--=
Quoted 6 lines
-----Original Message-----
From: [email protected] [mailto:linux-raid- [email protected]] On Behalf Of Bryce
Sent: Tuesday, October 07, 2008 11:35 AM
To: Linux RAID Mailing List
Subject: Re: Determining filename from absolute sector failure in
raid0
Quoted 8 lines
Quoted 3 lines · David Lethe
------------------------------------------------
--------------------
What you need is a utility program that translates a physical device&
Quoted 2 lines
offset and converts that into a logical block in md0. Then Something that maps a logical block in md0 to a file in a
filesystem.
Quoted 20 lines
Quoted 2 lines
Unfortunately, I am not aware of any off-the-shelf utility to do
this,
Quoted 5 lines
but put me down for being very interested in such a solution myself. David
well another part of the 'what do i do now?' question is that
/proc/mdstatshows the drives assembled thusly hdk1[0] hdg1[2] hdf1[1] hde1[3]and mdadm --examine shows the order as hde 0 hdf 1 hdg 2 hdk 3 err whats the actual order?
Phil
=--=
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
It is not that easy, you have to look at the topology of the RAID, factor in starting/ending block numbers, journaling, chunk/stripe size,
RAID level, etc.. in order to obtain the logical block number that
corresponds to any given physical disk number & raid configuration.
Lets call this magic routine PhysToLog(int mdNumber,char *PhysDev, size_t PhysBlockNumber);
That routine in perfect world would query the md config and return a 64-bit logical block number (let's future-proof the code), and if you had parity RAID, then would need to know the physical stripe number that would need to get rebuilt in event the bad block mapped to a parity (XOR) data.
Since you have RAID0, you have a lot of variables to eliminate, and can extrapolate this information. Part 2, which can be done independently, is the logical block to physical block.
Let's assume you manually take it apart and now have a logical block number in md0 that maps to the physical block ...
Look at this link: it will help to convert a known "bad block" to file's inode, which is what you need.
http://mail-index.netbsd.org/tech-userlevel/2003/04/17/0000.html
Too bad there isn't a mapbadblock2file command that ties this all up into a nice pretty package for everybody. It would be a big hit in dealing with data corruption and drive failures. I'm doing a 2TB rebuild as we speak scanning for corrupted blocks.