Problems w/ Sil3124 + Port Multiplier
Hi,
I have a system with a Sil3124 controller and two Sil3726 port multipliers.
I'm running 2.6.18.1 with the patches from http://home-tj.org/wiki/index.php/Libata-tj-stable
since I was not able to get the port multipler to work at all with 2.6.21 (perhaps the support for PMP is not fully integrated there).
One of my Sil3726 is freezing my system approx one time per hour when the disks are under load.
ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen ata5.00: tag 0 cmd 0xea Emask 0x4 stat 0x40 err 0x0 (timeout) ata5.15: hard resetting port
ata5.15: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata5.00: hard resetting port
ata5.00: SATA link up 3.0 Gbps (SStatus 113 SControl 300) ata5.01: hard resetting port
ata5.01: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata5.02: hard resetting port
ata5.02: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata5.03: hard resetting port
ata5.03: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata5.04: hard resetting port
ata5.04: SATA link up 1.5 Gbps (SStatus 113 SControl 300) ata5.00: configured for UDMA/100
ata5.01: configured for UDMA/100
ata5.02: configured for UDMA/100
ata5.03: configured for UDMA/100
ata5.04: configured for UDMA/100
ata5: EH complete
I suspect that the root-cause of the problem could be drive-related. All the disks on the other Sil3726 are WD4000KD's and the disks attached to this one is 3xWD4000KD and 1xWD4000KS and 1xSamsung HD501LJ. Is there a way of test this?
Is there an updated patch-set for port multipliers in a later kernel than 2.6.18.1?
Regards,
/LM
Quoted 11 lines · Lars Michael Jogbäck
I have a system with a Sil3124 controller and two Sil3726 port multipliers. I'm running 2.6.18.1 with the patches from
http://home-tj.org/wiki/index.php/Libata-tj-stable
since I was not able to get the port multipler to work at all with 2.6.21 (perhaps the support for PMP is not fully integrated there).One of my Sil3726 is freezing my system approx one time per hour when the disks are under load.
ata5.00: exception Emask
0x0SAct0x0SErr0x0action0x2frozen ata5.00: tag 0 cmd0xeaEmask0x4stat0x40err0x0(timeout)
Hmmm... that's FLUSH_CACHE timing out.
Quoted 4 lines
I suspect that the root-cause of the problem could be drive-related. All the disks on the other Sil3726 are WD4000KD's and the
disks attached to this one is 3xWD4000KD and 1xWD4000KS and 1xSamsung HD501LJ. Is there a way of test this?
I think the disk attached to port 0 might be bad. Please report the result of 'smartctl -d ata -a /dev/sdX' where sdX is the device attached to the failing port.
Quoted 2 lines
Is there an updated patch-set for port multipliers in a later kernel than 2.6.18.1?
Sorry but not yet.
Signature · 1 line
tejun
Quoted 8 lines · Tejun Heo
Quoted 6 lines · Lars Michael Jogbäck
I have a system with a Sil3124 controller and two Sil3726 port multipliers. I'm running 2.6.18.1 with the patches from
http://home-tj.org/wiki/index.php/Libata-tj-stable
since I was not able to get the port multipler to work at all with 2.6.21 (perhaps the support for PMP is not fully integrated there).
Quoted 9 lines
Quoted 6 lines
One of my Sil3726 is freezing my system approx one time per hour when the disks are under load.
ata5.00: exception Emask
0x0SAct0x0SErr0x0action0x2frozen ata5.00: tag 0 cmd0xeaEmask0x4stat0x40err0x0(timeout)Hmmm... that's
FLUSH_CACHEtiming out.
Is there a way that I can generate a FLUSH_CACHE event manually somehow?
Quoted 7 lines
Quoted 6 lines
I suspect that the root-cause of the problem could be drive-related. All the disks on the other Sil3726 are WD4000KD's and the
disks attached to this one is 3xWD4000KD and 1xWD4000KS and 1xSamsung HD501LJ. Is there a way of test this?
Quoted 5 lines
I think the disk attached to port 0 might be bad. Please report the result of 'smartctl -d ata -a
/dev/sdX' where sdX is the device attached to the failing port.
smartctl version 5.36 [i686-pc-linux-gnu] Copyright (C) 2002-6 Bruce Allen Home page is http://smartmontools.sourceforge.net/
=== START OF INFORMATION SECTION ===
Device Model: SAMSUNG HD501LJ Serial Number: S0VVJ1NP300009 Firmware Version: CR100-10 User Capacity: 500,107,862,016 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: Not recognized. Minor revision code: 0x52 Local Time is: Wed May 2 15:26:35 2007 CEST
==> WARNING: May need -F samsung or -F samsung2 enabled; see manual for details.
SMART support is: Available - device has SMART capability. SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection:
Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test
has ever
been run.
Total time to complete Offline
data collection: (8887) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection
on/off support.
Suspend Offline collection upon new command.
Offline surface scan supported. Self-test supported.
No Conveyance Self-test supported. Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode. Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 152) minutes.
SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 100 100 051 Pre-fail
Always - 2
3 Spin_Up_Time 0x0007 100 100 015 Pre-fail
Always - 7232
4 Start_Stop_Count 0x0032 100 100 000 Old_age
Always - 3
5 Reallocated_Sector_Ct 0x0033 253 253 010 Pre-fail
Always - 0
7 Seek_Error_Rate 0x000f 253 253 051 Pre-fail
Always - 0
8 Seek_Time_Performance 0x0025 253 253 015 Pre-fail
Offline - 0
9 Power_On_Hours 0x0032 253 253 000 Old_age
Always - 49
10 Spin_Retry_Count 0x0033 253 253 051 Pre-fail Always - 0 11 Calibration_Retry_Count 0x0012 253 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 3 187 Unknown_Attribute 0x0032 253 253 000 Old_age Always - 0 188 Unknown_Attribute 0x0032 253 253 000 Old_age Always - 0 190 Unknown_Attribute 0x0022 070 069 000 Old_age Always - 30 194 Temperature_Celsius 0x0022 148 145 000 Old_age Always - 30 195 Hardware_ECC_Recovered 0x001a 100 100 000 Old_age Always - 385707184 196 Reallocated_Event_Count 0x0032 253 253 000 Old_age Always - 0 197 Current_Pending_Sector 0x0012 253 253 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 253 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x000a 100 100 000 Old_age Always - 0 201 Soft_Read_Error_Rate 0x000a 100 100 000 Old_age Always - 0 202 TA_Increase_Count 0x0032 100 100 000 Old_age Always - 21
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 49 -
SMART Selective Self-Test Log Data Structure Revision Number (0) should be 1 SMART Selective self-test log data structure revision number 0
Warning: ATA Specification requires selective self-test log data structure revision number = 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Quoted 7 lines
Quoted 3 lines
Is there an updated patch-set for port multipliers in a later kernel than 2.6.18.1?
Sorry but not yet.
Regards,
/LM
Quoted 2 lines · Lars Michael Jogbäck
Is there a way that I can generate a
FLUSH_CACHEevent manually somehow?
hdparm -W0 /dev/sda
** requires hdparm-7.3; earlier versions won't do **
http://sourceforge.net/project/showfiles.php?group_id=136732
Quoted 7 lines · Lars Michael Jogbäck
Quoted 5 lines
I think the disk attached to port 0 might be bad. Please report the result of 'smartctl -d ata -a
/dev/sdX' where sdX is the device attached to the failing port.SMART Error Log Version: 1
No Errors Logged
I was hoping to see some error logs but no. Hardware_ECC_Recovered count seems high (385707184) but I dunno whether the value is normal or not. Different manufacturers use different norms in counting them. If you have other disks of the same model, you can compare the values and see whether if it's unusually high.
--
tejun
Quoted 12 lines · Tejun Heo
Quoted 9 lines · Lars Michael Jogbäck
Quoted 6 lines
I think the disk attached to port 0 might be bad. Please report the result of 'smartctl -d ata -a
/dev/sdX' where sdX is the device attached to the failing port.SMART Error Log Version: 1
No Errors Logged
Quoted 6 lines
I was hoping to see some error logs but no.
Hardware_ECC_Recoveredcount seems high (385707184) but I dunno whether the value is normal or not. Different manufacturers use different norms in counting them. If you have other disks of the same model, you can compare the values and see whether if it's unusually high.
I think that is normal for that kind of disk. This is another disk but the same model (this one is attached to a 3ware 9500-controller) and it shows the same.
smartctl version 5.36 [i686-pc-linux-gnu] Copyright (C) 2002-6 Bruce Allen Home page is http://smartmontools.sourceforge.net/
=== START OF INFORMATION SECTION ===
Device Model: SAMSUNG HD501LJ Serial Number: S0VVJ1NP300014 Firmware Version: CR100-10 User Capacity: 500,107,862,016 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 8 ATA Standard is: Not recognized. Minor revision code: 0x52 Local Time is: Thu May 3 12:36:32 2007 CEST
==> WARNING: May need -F samsung or -F samsung2 enabled; see manual for details.
SMART support is: Available - device has SMART capability. SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection:
Disabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test
has ever
been run.
Total time to complete Offline
data collection: (8852) seconds.
Offline data collection
capabilities: (0x5b) SMART execute Offline immediate.
Auto Offline data collection
on/off support.
Suspend Offline collection upon new command.
Offline surface scan supported. Self-test supported.
No Conveyance Self-test supported. Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode. Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 151) minutes.
SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 100 100 051 Pre-fail
Always - 3
3 Spin_Up_Time 0x0007 100 100 015 Pre-fail
Always - 7104
4 Start_Stop_Count 0x0032 100 100 000 Old_age
Always - 7
5 Reallocated_Sector_Ct 0x0033 253 253 010 Pre-fail
Always - 0
7 Seek_Error_Rate 0x000f 253 253 051 Pre-fail
Always - 0
8 Seek_Time_Performance 0x0025 253 253 015 Pre-fail
Offline - 0
9 Power_On_Hours 0x0032 253 253 000 Old_age
Always - 97
10 Spin_Retry_Count 0x0033 253 253 051 Pre-fail Always - 0 11 Calibration_Retry_Count 0x0012 253 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 7 187 Unknown_Attribute 0x0032 253 253 000 Old_age Always - 0 188 Unknown_Attribute 0x0032 253 253 000 Old_age Always - 0 190 Unknown_Attribute 0x0022 068 067 000 Old_age Always - 32 194 Temperature_Celsius 0x0022 142 139 000 Old_age Always - 32 195 Hardware_ECC_Recovered 0x001a 100 100 000 Old_age Always - 455228167 196 Reallocated_Event_Count 0x0032 253 253 000 Old_age Always - 0 197 Current_Pending_Sector 0x0012 253 253 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 253 253 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x000a 100 100 000 Old_age Always - 0 201 Soft_Read_Error_Rate 0x000a 100 100 000 Old_age Always - 0 202 TA_Increase_Count 0x0032 253 253 000 Old_age Always - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 97 -
SMART Selective Self-Test Log Data Structure Revision Number (0) should be 1 SMART Selective self-test log data structure revision number 0
Warning: ATA Specification requires selective self-test log data structure revision number = 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Also, something else happened with error after a while.
First it changed to show this instead:
May 1 23:52:25 cleopatra kernel: ata5.00: limiting SATA link speed to 1.5 Gbps May 1 23:52:25 cleopatra kernel: ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen May 1 23:52:25 cleopatra kernel: ata5.00: tag 2 cmd 0xea Emask 0x4 stat 0x40 err 0x0 (timeout) May 1 23:52:25 cleopatra kernel: ata5.15: hard resetting port May 1 23:52:27 cleopatra kernel: ata5.15: SATA link up 3.0 Gbps (SStatus 123 SControl 300) May 1 23:52:27 cleopatra kernel: ata5.00: hard resetting port May 1 23:52:28 cleopatra kernel: ata5.00: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 1 23:52:28 cleopatra kernel: ata5.01: hard resetting port May 1 23:52:28 cleopatra kernel: ata5.01: SATA link up 3.0 Gbps (SStatus 123 SControl 300) May 1 23:52:29 cleopatra kernel: ata5.02: hard resetting port May 1 23:52:29 cleopatra kernel: ata5.02: SATA link up 1.5 Gbps (SStatus 113 SControl 300) May 1 23:52:29 cleopatra kernel: ata5.03: hard resetting port May 1 23:52:30 cleopatra kernel: ata5.03: SATA link up 1.5 Gbps (SStatus 113 SControl 300) May 1 23:52:30 cleopatra kernel: ata5.04: hard resetting port May 1 23:52:30 cleopatra kernel: ata5.04: SATA link up 1.5 Gbps (SStatus 113 SControl 300) May 1 23:52:30 cleopatra kernel: ata5.00: configured for UDMA/100 May 1 23:52:30 cleopatra kernel: ata5.01: configured for UDMA/100 May 1 23:52:30 cleopatra kernel: ata5.02: configured for UDMA/100 May 1 23:52:30 cleopatra kernel: ata5.03: configured for UDMA/100 May 1 23:52:30 cleopatra kernel: ata5.04: configured for UDMA/100 May 1 23:52:30 cleopatra kernel: ata5: EH complete
and later it showed this:
May 1 23:53:36 cleopatra kernel: ata5.00: limiting speed to UDMA/66 May 1 23:53:36 cleopatra kernel: ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen May 1 23:53:36 cleopatra kernel: ata5.00: tag 0 cmd 0xea Emask 0x4 stat 0x40 err 0x0 (timeout) May 1 23:53:36 cleopatra kernel: ata5.15: hard resetting port May 1 23:53:38 cleopatra kernel: ata5.15: SATA link up 3.0 Gbps (SStatus 123 SControl 300) May 1 23:53:38 cleopatra kernel: ata5.00: hard resetting port May 1 23:53:39 cleopatra kernel: ata5.00: SATA link up 1.5 Gbps (SStatus 113 SControl 310) May 1 23:53:39 cleopatra kernel: ata5.01: hard resetting port May 1 23:53:40 cleopatra kernel: ata5.01: SATA link up 3.0 Gbps (SStatus 123 SControl 300) May 1 23:53:40 cleopatra kernel: ata5.02: hard resetting port May 1 23:53:40 cleopatra kernel: ata5.02: SATA link up 1.5 Gbps (SStatus 113 SControl 300) May 1 23:53:40 cleopatra kernel: ata5.03: hard resetting port May 1 23:53:41 cleopatra kernel: ata5.03: SATA link up 1.5 Gbps (SStatus 113 SControl 300) May 1 23:53:41 cleopatra kernel: ata5.04: hard resetting port May 1 23:53:41 cleopatra kernel: ata5.04: SATA link up 1.5 Gbps (SStatus 113 SControl 300) May 1 23:53:41 cleopatra kernel: ata5.00: configured for UDMA/66 May 1 23:53:41 cleopatra kernel: ata5.01: configured for UDMA/100 May 1 23:53:41 cleopatra kernel: ata5.02: configured for UDMA/100 May 1 23:53:41 cleopatra kernel: ata5.03: configured for UDMA/100 May 1 23:53:41 cleopatra kernel: ata5.04: configured for UDMA/100 May 1 23:53:41 cleopatra kernel: ata5: EH complete
and then it continues with approx 1/hour of the above. So it seems something is strange in the interface between the drive and the computer. I can swap the drive (it's in an raid5-array) to another drive of the same model if helps in any way; but I suspect that it will show the same result.
Regards,
/LM
Quoted 10 lines · Lars Michael Jogbäck
Quoted 6 lines · Tejun Heo
I was hoping to see some error logs but no.
Hardware_ECC_Recoveredcount seems high (385707184) but I dunno whether the value is normal or not. Different manufacturers use different norms in counting them. If you have other disks of the same model, you can compare the values and see whether if it's unusually high.I think that is normal for that kind of disk. This is another disk but the same model (this one is attached to a 3ware 9500-controller) and it shows the same.
I see.
Quoted 8 lines
May 1 23:53:36 cleopatra kernel: ata5.00: tag 0 cmd 0xea Emask 0x4 stat
0x40err0x0(timeout)and then it continues with approx 1/hour of the above. So it seems something is strange in the interface between the drive and the computer. I can swap the drive (it's in an raid5-array) to another drive of the same model if helps in any way; but I suspect that it will show the same result.
The fact that the first slot is always the one causing problem bothers me. Can you try to connect the disk to a different controller and use different power connector? Let's see to which the problem is attached - disk, port or something else.
Signature · 1 line
tejun