array sync looping
Hi -
I recently upgraded to the 2.6.17-2-686 SMP kernel image (debian).
Now, when I hot-add a disk (mdadm /dev/md1 --add /dev/hdc3) the array resyncs to 100%, a bunch of errors appear and then the array resync starts from 0% again.
mdadm --zero-superblock on the "new" disk doesn't help.
Any suggestions on what might be happening or how to debug/fix would be welcome.
Thanks -
Leni
$ dmesg # output truncated
...
md: md1: sync done.
hda: dma_intr: status=0x51 { DriveReady SeekComplete Error }
hda: dma_intr: error=0x40 { UncorrectableError }, LBAsect=390716106, high=23, low=4840138, sector=390716042
ide: failed opcode was: unknown
end_request: I/O error, dev hda, sector 390716042
...
raid1: hda: unrecoverable I/O read error for block 388499072
RAID1 conf printout:
--- wd:1 rd:3
disk 0, wo:1, o:1, dev:hdc3
disk 2, wo:0, o:1, dev:hda3
RAID1 conf printout:
--- wd:1 rd:3
disk 2, wo:0, o:1, dev:hda3
RAID1 conf printout:
--- wd:1 rd:3
disk 0, wo:1, o:1, dev:hdc3
disk 2, wo:0, o:1, dev:hda3
md: syncing RAID array md
...Frequent SATA errors / port timeouts in 2.6.18.3?
Hi all,
this may not be the best list for this question, but I figure that the number of disks connected to users here should be pretty big...
I upgraded from 2.6.17-rc4 to 2.6.18.3 about a week ago, and I've since had 3 drives kicked out of my 10-drive RAID5 array. Previously, I had no kicks over almost a year. The kernel message is:
ata7.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 ata7.00: (BMDMA stat 0x20)
ata7.00: tag 0 cmd 0xc8 Emask 0x1 stat 0x41 err 0x4 (device error)
ata7: EH complete
SCSI device sdc: 488397168 512-byte hdwr sectors (250059 MB)
sdc: Write Protect is off
sdc: Mode Sense: 00 3a 00 00
SCSI device sdc: drive cache: write back
ata7.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen ata7.00: (BMDMA stat 0x20)
ata7.00: tag 0 cmd 0xca Emask 0x4 stat 0x40 err 0x0 (timeout)
ata7: port is slow to respond, please be patient
ata7: port failed to respond (30 secs)
ata7: soft resetting port
ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata7.00: failed to IDENTIFY (I/O error, err_mask=0x2) ata7.00: revalidation failed (errno=-5)
ata7: failed to recover some devices, retrying in 5 secs
ata7: hard resetting port
ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata7.00: failed to IDENTIFY (I/O error, err_mask=0x2) ata7.00: revalidation failed (errno=-5)
ata7: failed to recover some devices, retrying in 5 secs
ata7: hard resetting port
ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata7.00: failed to IDENTIFY (I/O error, err_mask=0x2) ata7.00: revalidation failed (errno=-5)
ata7.00: disabled
ata7: EH complete
First I thought it was a cabling or card issue, because the same drive got kicked twice. That drive was connected to a 2-port SIG sata_sil24 card. However, I just had another drive kicked that's connected to sata_nv, which leads me to suspect that the upgraded kernel might have something to do with it. A quick googling seems to indicate that others are seeing this with 2.6.18, too, so I was wondering if anyone here knows more.
I did at the same time also install an Areca ARC1260 controller and connected a bunch of drives to it, so another idea I had was cable interference or something (there are now 18 drives in the machine).
Any ideas or thought would be appreciated,
/Patrik
Frequent SATA errors / port timeouts in 2.6.18.3?
In reply to [email protected]Hi all,
Hopefully someone here will know what's up with my machine. It's an nforce4 ultra box that's running a 10-drive RAID5 array. I upgraded from 2.6.17-rc4 to 2.6.18.3 about a week ago, and I've since had 3 drives kicked out. Previously, I had no kicks over almost a year. The kernel message is:
ata7.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 ata7.00: (BMDMA stat 0x20)
ata7.00: tag 0 cmd 0xc8 Emask 0x1 stat 0x41 err 0x4 (device error)
ata7: EH complete
SCSI device sdc: 488397168 512-byte hdwr sectors (250059 MB)
sdc: Write Protect is off
sdc: Mode Sense: 00 3a 00 00
SCSI device sdc: drive cache: write back
ata7.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen ata7.00: (BMDMA stat 0x20)
ata7.00: tag 0 cmd 0xca Emask 0x4 stat 0x40 err 0x0 (timeout)
ata7: port is slow to respond, please be patient
ata7: port failed to respond (30 secs)
ata7: soft resetting port
ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata7.00: failed to IDENTIFY (I/O error, err_mask=0x2) ata7.00: revalidation failed (errno=-5)
ata7: failed to recover some devices, retrying in 5 secs
ata7: hard resetting port
ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata7.00: failed to IDENTIFY (I/O error, err_mask=0x2) ata7.00: revalidation failed (errno=-5)
ata7: failed to recover some devices, retrying in 5 secs
ata7: hard resetting port
ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata7.00: failed to IDENTIFY (I/O error, err_mask=0x2) ata7.00: revalidation failed (errno=-5)
ata7.00: disabled
ata7: EH complete
First I thought it was a cabling or card issue, because the same drive got kicked twice. That drive was connected to a 2-port SIG sata_sil24 card. However, I just had another drive kicked that's connected to the onboard sata_nv, which leads me to suspect that the upgraded kernel might have something to do with it. A quick googling seems to indicate that others are seeing this with 2.6.18, too, so I was wondering if anyone knows more. The drives contain science data for analysis, so it would be a pain (though not a disaster) to lose it. Would it be advisable to revert to the previous 2.6.17 that I was running before or is this a problem that's fixed in a later kernel than the one I'm running now?
I did at the same time also install an Areca ARC1260 controller and connected a bunch of drives to it, so another idea I had was cable interference or something (there are now 18 drives in the machine).
Any ideas or thought would be appreciated,
/Patrik
Re: Frequent SATA errors / port timeouts in 2.6.18.3?
In reply to Patrik JonssonQuoted 8 lines · Patrik Jonsson
Hi all,
this may not be the best list for this question, but I figure that the number of disks connected to users here should be pretty big...I upgraded from 2.6.17-rc4 to 2.6.18.3 about a week ago, and I've since had 3 drives kicked out of my 10-drive RAID5 array. Previously, I had no kicks over almost a year. The kernel message is:
Quoted 6 lines
ata7.00: exception Emask
0x0SAct0x0SErr0x0action0x0ata7.00: (BMDMA stat0x20)
ata7.00: tag 0 cmd0xc8Emask0x1stat0x41err0x4(device error)
ata7: EH completeAny ideas or thought would be appreciated,
SMART?
Read the manpage and then try running:
smartctl -data -S on /dev/...
and
smartctl -data -s on /dev/...
Then look at your smartd timing and see if it's related; possibly just do a manual smartd poll.
I've had smart/libata problems (well, no, glitches) for about 2 years now but as the irq handler occasionally says "no one cared" ;)
It may well not be your problem but...
David
Re: Frequent SATA errors / port timeouts in 2.6.18.3?
In reply to Patrik JonssonQuoted 7 lines · Patrik Jonsson
First I thought it was a cabling or card issue, because the same drive got kicked twice. That drive was connected to a 2-port SIG
sata_sil24card. However, I just had another drive kicked that's connected to the onboardsata_nv, which leads me to suspect that the upgraded kernel might have something to do with it. A quick googling seems to indicate that others are seeing this with 2.6.18, too, so I was wondering if anyone knows more.
I don't think there has been any change which can affect both sata_sil24 and sata_nv to cause timeouts. Please post the result of 'smartctl -d ata -a /dev/sdX' of failed devices.
Quoted 11 lines
The drives contain science data for analysis, so it
would be a pain (though not a disaster) to lose it. Would it be advisable to revert to the previous 2.6.17 that I was running before or is this a problem that's fixed in a later kernel than the one I'm running now?I did at the same time also install an Areca ARC1260 controller and connected a bunch of drives to it, so another idea I had was cable interference or something (there are now 18 drives in the machine).
Any ideas or thought would be appreciated,
Power quality degradation can cause transmission failures which can result in timeouts. How are your power lines hooked?
Signature · 1 line
tejun
Re: Frequent SATA errors / port timeouts in 2.6.18.3?
Quoted 9 lines · Tejun Heo
Quoted 7 lines · Patrik Jonsson
First I thought it was a cabling or card issue, because the same drive got kicked twice. That drive was connected to a 2-port SIG
sata_sil24card. However, I just had another drive kicked that's connected to the onboardsata_nv, which leads me to suspect that the upgraded kernel might have something to do with it. A quick googling seems to indicate that others are seeing this with 2.6.18, too, so I was wondering if anyone knows more.
Quoted 3 lines
I don't think there has been any change which can affect both
sata_sil24andsata_nvto cause timeouts. Please post the result of 'smartctl -d ata -a/dev/sdX' of failed devices.
At the time it happened, none of the devices responded to smart commands ("not an ATA device"). After resetting, they do. The outputs are at the bottom.
Quoted 7 lines
Quoted 7 lines
The drives contain science data for analysis, so it
would be a pain (though not a disaster) to lose it. Would it be advisable to revert to the previous 2.6.17 that I was running before or is this a problem that's fixed in a later kernel than the one I'm running now?
Quoted 8 lines
Quoted 5 lines
I did at the same time also install an Areca ARC1260 controller and connected a bunch of drives to it, so another idea I had was cable interference or something (there are now 18 drives in the machine).
Any ideas or thought would be appreciated,
Power quality degradation can cause transmission failures which can result in timeouts. How are your power lines hooked?
The drives are connected to the Seasonic M12-600 PSU, connected to an APC BackUPS and then it's in the wall. It's in a dedicated computer room, but I'm not sure if the power in conditioned in any way.
cheers,
/Patrik
**** Drive 1 SMART output ****
=== START OF INFORMATION SECTION ===
Device Model: WDC WD2500SD-01KCC0 Serial Number: WD-WCAL74587210 Firmware Version: 08.02D08 User Capacity: 250,059,350,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 6 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Tue Dec 19 16:25:01 2006 PST SMART support is: Available - device has SMART capability. SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x05) Offline data collection activity
was aborted by an interrupting
command from host.
Auto Offline Data Collection:
Disabled.
Self-test execution status: ( 0) The previous self-test routine
completed
without error or no self-test
has ever
been run.
Total time to complete Offline
data collection: (6942) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection
on/off support.
Suspend Offline collection upon new command.
Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode. Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
No General Purpose Logging support.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 88) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE
UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000b 200 159 051 Pre-fail Always
- 0
3 Spin_Up_Time 0x0007 123 121 021 Pre-fail Always
- 6375
4 Start_Stop_Count 0x0032 100 100 040 Old_age Always
- 172
5 Reallocated_Sector_Ct 0x0033 183 183 140 Pre-fail Always
- 271
7 Seek_Error_Rate 0x000b 200 200 051 Pre-fail Always
- 0
9 Power_On_Hours 0x0032 083 083 000 Old_age Always
- 12742
10 Spin_Retry_Count 0x0013 100 100 051 Pre-fail Always
- 0
11 Calibration_Retry_Count 0x0013 100 100 051 Pre-fail Always
- 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always
- 171
194 Temperature_Celsius 0x0022 125 103 000 Old_age Always
- 25
196 Reallocated_Event_Count 0x0032 161 161 000 Old_age Always
- 39
197 Current_Pending_Sector 0x0012 200 200 000 Old_age Always
- 1
198 Offline_Uncorrectable 0x0012 200 200 000 Old_age Always
- 0
199 UDMA_CRC_Error_Count 0x000a 200 253 000 Old_age Always
- 0
200 Multi_Zone_Error_Rate 0x0009 200 200 051 Pre-fail
Offline - 0
SMART Error Log Version: 1
ATA Error Count: 899 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 899 occurred at disk power-on lifetime: 12737 hours (530 days + 17 hours)
When the command that caused the error occurred, the device was doing
SMART Offline or Self-test.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 08 2a c3 90 e5 Error: UNC 8 sectors at LBA = 0x0590c32a = 93373226
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 28 c3 90 05 00 5d+19:30:10.900 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:10.900 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:10.900 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:10.900 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:10.900 READ DMA
Error 898 occurred at disk power-on lifetime: 12737 hours (530 days + 17 hours)
When the command that caused the error occurred, the device was doing
SMART Offline or Self-test.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 08 2a c3 90 e5 Error: UNC 8 sectors at LBA = 0x0590c32a = 93373226
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 28 c3 90 05 00 5d+19:30:08.850 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:08.850 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:08.850 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:08.850 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:08.850 READ DMA
Error 897 occurred at disk power-on lifetime: 12737 hours (530 days + 17 hours)
When the command that caused the error occurred, the device was doing
SMART Offline or Self-test.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 08 2a c3 90 e5 Error: UNC 8 sectors at LBA = 0x0590c32a = 93373226
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 28 c3 90 05 00 5d+19:30:06.800 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:06.800 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:06.800 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:06.800 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:06.800 READ DMA
Error 896 occurred at disk power-on lifetime: 12737 hours (530 days + 17 hours)
When the command that caused the error occurred, the device was doing
SMART Offline or Self-test.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 08 2a c3 90 e5 Error: UNC 8 sectors at LBA = 0x0590c32a = 93373226
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 28 c3 90 05 00 5d+19:30:04.650 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:04.650 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:04.650 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:04.650 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:04.650 READ DMA
Error 895 occurred at disk power-on lifetime: 12737 hours (530 days + 17 hours)
When the command that caused the error occurred, the device was doing
SMART Offline or Self-test.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 08 2a c3 90 e5 Error: UNC 8 sectors at LBA = 0x0590c32a = 93373226
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 28 c3 90 05 00 5d+19:30:02.550 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:02.550 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:02.550 READ DMA ec 00 08 2a c3 90 00 00 5d+19:30:02.550 IDENTIFY DEVICE c8 00 08 28 c3 90 05 00 5d+19:30:02.550 READ DMA
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining
LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 12742
-
# 2 Extended offline Completed: read failure 70% 12738
110374951
# 3 Short offline Completed: read failure 10% 12729
93373219
# 4 Short offline Completed: read failure 10% 12705
93373219
# 5 Extended offline Completed: read failure 70% 12681
93373219
# 6 Short offline Completed: read failure 10% 12657
93373219
# 7 Short offline Completed: read failure 10% 12633
93373219
# 8 Short offline Completed: read failure 10% 12609
93373219
# 9 Extended offline Completed: read failure 70% 12598
93373219
#10 Short offline Completed without error 00% 12587
-
#11 Short offline Completed without error 00% 12563
-
#12 Short offline Completed without error 00% 12543
-
#13 Short offline Completed without error 00% 12519
-
#14 Extended offline Completed without error 00% 12396
-
#15 Short offline Completed without error 00% 12342
-
#16 Short offline Completed without error 00% 12318
-
#17 Short offline Completed without error 00% 12294
-
#18 Extended offline Completed without error 00% 12272
-
#19 Short offline Completed without error 00% 12247
-
#20 Short offline Completed without error 00% 12223
-
#21 Short offline Completed without error 00% 12202
-**** Drive 2 SMART output ****
=== START OF INFORMATION SECTION ===
Device Model: WDC WD2500SD-01KCC0 Serial Number: WD-WCAL74586965 Firmware Version: 08.02D08 User Capacity: 250,059,350,016 bytes Device is: Not in smartctl database [for details use: -P showall] ATA Version is: 6 ATA Standard is: Exact ATA specification draft version not indicated Local Time is: Tue Dec 19 16:27:31 2006 PST SMART support is: Available - device has SMART capability. SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x02) Offline data collection activity
was completed without error. Auto Offline Data Collection:
Disabled.
Self-test execution status: ( 0) The previous self-test routine
completed
without error or no self-test
has ever
been run.
Total time to complete Offline
data collection: (7275) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection
on/off support.
Suspend Offline collection upon new command.
Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode. Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
No General Purpose Logging support.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 91) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE
UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000b 200 158 051 Pre-fail Always
- 0
3 Spin_Up_Time 0x0007 146 126 021 Pre-fail Always
- 5225
4 Start_Stop_Count 0x0032 100 100 040 Old_age Always
- 94
5 Reallocated_Sector_Ct 0x0033 183 183 140 Pre-fail Always
- 258
7 Seek_Error_Rate 0x000b 200 200 051 Pre-fail Always
- 0
9 Power_On_Hours 0x0032 094 094 000 Old_age Always
- 4878
10 Spin_Retry_Count 0x0013 100 253 051 Pre-fail Always
- 0
11 Calibration_Retry_Count 0x0013 100 253 051 Pre-fail Always
- 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always
- 93
194 Temperature_Celsius 0x0022 123 101 000 Old_age Always
- 27
196 Reallocated_Event_Count 0x0032 175 175 000 Old_age Always
- 25
197 Current_Pending_Sector 0x0012 200 200 000 Old_age Always
- 0
198 Offline_Uncorrectable 0x0012 200 200 000 Old_age Always
- 0
199 UDMA_CRC_Error_Count 0x000a 200 253 000 Old_age Always
- 0
200 Multi_Zone_Error_Rate 0x0009 200 200 051 Pre-fail
Offline - 0
SMART Error Log Version: 1
ATA Error Count: 246 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 246 occurred at disk power-on lifetime: 2573 hours (107 days + 5 hours)
When the command that caused the error occurred, the device was active
or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 28 15 bd 69 e1 Error: UNC 40 sectors at LBA = 0x0169bd15 = 23706901
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 28 0f bd 69 01 00 30d+01:23:50.954 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:50.954 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:50.954 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:50.954 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:50.954 READ DMA
Error 245 occurred at disk power-on lifetime: 2573 hours (107 days + 5 hours)
When the command that caused the error occurred, the device was active
or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 28 15 bd 69 e1 Error: UNC 40 sectors at LBA = 0x0169bd15 = 23706901
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 28 0f bd 69 01 00 30d+01:23:48.904 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:48.904 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:48.904 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:48.904 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:48.904 READ DMA
Error 244 occurred at disk power-on lifetime: 2573 hours (107 days + 5 hours)
When the command that caused the error occurred, the device was active
or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 28 15 bd 69 e1 Error: UNC 40 sectors at LBA = 0x0169bd15 = 23706901
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 28 0f bd 69 01 00 30d+01:23:46.854 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:46.854 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:46.854 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:46.854 READ DMA c8 00 30 07 bd 69 01 00 30d+01:23:46.854 READ DMA
Error 243 occurred at disk power-on lifetime: 2573 hours (107 days + 5 hours)
When the command that caused the error occurred, the device was active
or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 28 15 bd 69 e1 Error: UNC 40 sectors at LBA = 0x0169bd15 = 23706901
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 28 0f bd 69 01 00 30d+01:23:44.754 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:44.754 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:44.754 READ DMA c8 00 30 07 bd 69 01 00 30d+01:23:44.754 READ DMA c8 00 30 07 bd 69 01 00 30d+01:23:44.754 READ DMA
Error 242 occurred at disk power-on lifetime: 2573 hours (107 days + 5 hours)
When the command that caused the error occurred, the device was active
or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 28 15 bd 69 e1 Error: UNC 40 sectors at LBA = 0x0169bd15 = 23706901
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 28 0f bd 69 01 00 30d+01:23:42.654 READ DMA c8 00 28 0f bd 69 01 00 30d+01:23:42.654 READ DMA c8 00 30 07 bd 69 01 00 30d+01:23:42.654 READ DMA c8 00 30 07 bd 69 01 00 30d+01:23:42.654 READ DMA c8 00 30 07 bd 69 01 00 30d+01:23:42.654 READ DMA
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining
LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 4865
-
# 2 Short offline Completed without error 00% 4841
-
# 3 Extended offline Completed without error 00% 4818
-
# 4 Short offline Completed without error 00% 4793
-
# 5 Short offline Completed without error 00% 4769
-
# 6 Short offline Completed without error 00% 4745
-
# 7 Short offline Completed without error 00% 4722
-
# 8 Short offline Completed without error 00% 4698
-
# 9 Short offline Completed: read failure 10% 4687
254963387
#10 Short offline Completed without error 00% 4678
-
#11 Short offline Completed without error 00% 4654
-
#12 Extended offline Completed without error 00% 4532
-
#13 Short offline Completed without error 00% 4478
-
#14 Short offline Completed without error 00% 4454
-
#15 Short offline Completed without error 00% 4430
-
#16 Extended offline Completed without error 00% 4408
-
#17 Short offline Completed without error 00% 4382
-
#18 Short offline Completed without error 00% 4358
-
#19 Short offline Completed without error 00% 4338
-
#20 Short offline Completed without error 00% 4314
-
#21 Short offline Completed without error 00% 4290
-Re: Frequent SATA errors / port timeouts in 2.6.18.3?
Quoted 3 lines · Patrik Jonsson
At the time it happened, none of the devices responded to smart commands ("not an ATA device"). After resetting, they do. The outputs are at the bottom.
Hmmm... Nothing indicative of power problem. The first drive seems to have suffered quite some number of read failures due to bad sectors tho.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
The drives contain science data for analysis, so it
would be a pain (though not a disaster) to lose it. Would it be advisable to revert to the previous 2.6.17 that I was running before or is this a problem that's fixed in a later kernel than the one I'm running now?I did at the same time also install an Areca ARC1260 controller and connected a bunch of drives to it, so another idea I had was cable interference or something (there are now 18 drives in the machine).
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Any ideas or thought would be appreciated,
Power quality degradation can cause transmission failures which can result in timeouts. How are your power lines hooked?
The drives are connected to the Seasonic M12-600 PSU, connected to an APC BackUPS and then it's in the wall. It's in a dedicated computer room, but I'm not sure if the power in conditioned in any way.
My primary suspect is still power. Please do one of the followings.
* Buy an extra power and hang 2/3 of drives there. Just cheap 350w one should do. Using PS without mainboard attached is easy, google it. In my experience, single-lane 300 ~ 350w PS is better than 450w multilane one for this purpose. More juice on molex/SATA 12v.
* If that's not an option, disconnect the extra drives and see how the system behaves.
Signature · 1 line
tejun