HDD Unrecovered readerror issue
Drive:WDC WD1003FZEX-00MK2A0
I have got this in logs:
ata1.00: failed command: READ FPDMA QUEUED
ata1.00: cmd 60/a0:a0:f0:c0:c5/00:00:04:00:00/40 tag 20 ncq 81920 in res 41/40:00:88:c1:c5/00:00:04:00:00/00 Emask 0x409 (media error) ata1.00: status: { DRDY ERR }
ata1.00: error: { UNC }
ata1.00: configured for UDMA/133
sd 0:0:0:0: [sda] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 0:0:0:0: [sda] Sense Key : Medium Error[current] [descriptor]
sd 0:0:0:0: [sda] Add. Sense: Unrecovered readerror - auto reallocate failed
sd 0:0:0:0: [sda] CDB: Read(10) 28 00 04 c5 c0 f0 00 00 a0 00blk_update_request: I/O error, dev sda, sector 80069000
ata1: EH complete
I can reproduce this easily
#xfs_io -c "pread $((80069000/2))k 4k" -d /dev/sda
pread64: Input/output error
##Got EIO
##Smartctl also detect this
#smartctl -t short /dev/sda
#smartctl -l selftest /dev/sda
....
Short offline Completed: read failure 90% 4682 80069000
But once I rewrite this block, problem goes away. #xfs_io -c "pwrite -S 0x0 $((80069000/2))k 4k" -d /dev/sda
Now I can read it w/o any errors and smartctl is happy #smartctl -t short /dev/sda
#smartctl -l selftest /dev/sda
Num Test_Description Status Remaining
LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 4683 -
So my disk is not dead right? Why the hell HDD fail read from very beginning Is this because HDD firmware detect internal crcXX sum corruption? How this can happen? Is this because of power failure?
AFAIK standard guarantees that sector will be updated atomically. But it happens! Please guide me how to fix such problems in general.
Quoted 11 lines · Dmitry Monakhov
But once I rewrite this block, problem goes away. #xfs_io -c "pwrite -S
0x0$((80069000/2))k 4k" -d/dev/sdaNow I can read it w/o any errors and smartctl is happy #smartctl -t short
/dev/sda
#smartctl -l selftest/dev/sda
NumTest_DescriptionStatus Remaining
LifeTime(hours)LBA_of_first_error
# 1 Short offline Completed without error 00% 4683 -So my disk is not dead right?
Correct.
Quoted 2 lines
Why the hell HDD fail read from very beginning
Is this because HDD firmware detect internal crcXX sum corruption?
Yes.
Quoted 1 line
How this can happen? Is this because of power failure?
Could be. If power was cut in the middle of a write, this can happen. There are other causes, though (bit rot, for example).
Quoted 1 line
AFAIK standard guarantees that sector will be updated atomically.
No, the SCSI and ATA standards most certainly do not guarantee that! NVMe is the only standard I know of that requires Atomic Write Unit Power Fail to be at lest one sector.
Quoted 1 line
But it happens! Please guide me how to fix such problems in general.
You fixed it. Overwriting the sector will clear the error.
Cheers,
Jeff
James Bottomley
maintainerQuoted 24 lines · Jeff Moyer
Quoted 11 lines · Dmitry Monakhov
But once I rewrite this block, problem goes away. #xfs_io -c "pwrite -S
0x0$((80069000/2))k 4k" -d/dev/sdaNow I can read it w/o any errors and smartctl is happy #smartctl -t short
/dev/sda
#smartctl -l selftest/dev/sda
NumTest_DescriptionStatus Remaining
LifeTime(hours)LBA_of_first_error
# 1 Short offline Completed without error 00%
4683 -Quoted 1 line
So my disk is not dead right?
Correct.
Quoted 2 lines
Why the hell HDD fail read from very beginning
Is this because HDD firmware detect internal crcXX sum corruption?Yes.
Quoted 1 line
How this can happen? Is this because of power failure?
Quoted 8 lines
Could be. If power was cut in the middle of a write, this can happen. There are other causes, though (bit rot, for example).
Quoted 1 line
AFAIK standard guarantees that sector will be updated atomically.
No, the SCSI and ATA standards most certainly do not guarantee that! NVMe is the only standard I know of that requires Atomic Write Unit Power Fail to be at lest one sector.
The mechanics of the drive mostly ensure atomic updates on the physical block level. You definitely get either the old data, the new data or an unreadable sector. The latter is a pretty rare event because surviving power usually ensures the writes complete, but it's not guaranteed.
Quoted 4 lines
Quoted 2 lines
But it happens! Please guide me how to fix such problems in general.
You fixed it. Overwriting the sector will clear the error.
Actually only "may clear the error" depending on what happened. If the hamming codes on the sector itself just failed (because of a torn write due to power fail) then a rewrite simply re-fixes the sector in situ.
Sometimes the magnetic substrate of the track is worn (so the sector
is permanently damaged) and the re-write forces a reallocation. If that's happening to your disk then eventually it will fail irrecoverably when the reallocation table is full.
You can monitor this with the smart Reallocated_Event_Count.
James