System freezes with kernel >2.6.19 - sata_nv
Hi folks,
yesterday I upgraded kernel 2.6.19 to 2.6.20 (gentoo kernel). Now my box locks up about 10 min after boot.
After that I tested with a vanilla 2.6.21.1 it shows the same behavior. I'm attaching a kern log file from the 2.6.20. The 2.6.21.1 locked up so hard, that there was no trace left in the log file.
If I switch back to 2.6.19 everything is fine again.
If necessary I will hook up a laptop to this box, so I can capture messages via netconsole.
This machine is running an AMD X2 64, NFORCE4 (ASUS A8N-E)
I haven't been following the development of libata for a while, but from the 2.6.20 changelog It looks like there have been some major changes.
Just let me know what kind of information you need in order to narrow it down.
Cheers Stefan
Quoted 10 lines · Stefan
Hi folks,
yesterday I upgraded kernel 2.6.19 to 2.6.20 (gentoo kernel). Now my box locks up about 10 min after boot.
After that I tested with a vanilla 2.6.21.1 it shows the same behavior. I'm attaching a kern log file from the 2.6.20. The 2.6.21.1 locked up so hard, that there was no trace left in the log file.
If I switch back to 2.6.19 everything is fine again.
If necessary I will hook up a laptop to this box, so I can capture messages via netconsole.
Yes please.
Quoted 7 lines
This machine is running an AMD X2 64, NFORCE4 (ASUS A8N-E)
I haven't been following the development of libata for a while, but from the 2.6.20 changelog It looks like there have been some major changes.
Just let me know what kind of information you need in order to narrow it down.
Does giving 'sata_nv.adma=0' kernel parameter make any difference?
Signature · 1 line
tejun
Quoted 16 lines · Tejun Heo
Quoted 10 lines · Stefan
Hi folks,
yesterday I upgraded kernel 2.6.19 to 2.6.20 (gentoo kernel). Now my box locks up about 10 min after boot.
After that I tested with a vanilla 2.6.21.1 it shows the same behavior. I'm attaching a kern log file from the 2.6.20. The 2.6.21.1 locked up so hard, that there was no trace left in the log file.
If I switch back to 2.6.19 everything is fine again.
If necessary I will hook up a laptop to this box, so I can capture messages via netconsole.Yes please.
Quoted 2 lines
This machine is running an AMD X2 64, NFORCE4 (ASUS A8N-E)
Quoted 7 lines
Quoted 5 lines
I haven't been following the development of libata for a while, but from the 2.6.20 changelog It looks like there have been some major changes.
Just let me know what kind of information you need in order to narrow it down.
Does giving 'sata_nv.adma=0' kernel parameter make any difference?
If adma=0 words, then that means it's either an ADMA related problem or that SAMSUNG HD401LJ drive has some problems with NCQ (since ADMA off means NCQ off as well). I would say the latter is more likely.
We should really have some kind of "noncq" kernel parameter we can use to help debugging these problems. Though, later kernels are supposed to switch it off automatically after too many errors..
Signature · 3 lines
Robert Hancock Saskatoon, SK, Canada To email, remove "nospam" from [email protected] Home Page: http://www.roberthancock.com/
Quoted 20 lines · Robert Hancock
Quoted 19 lines · Tejun Heo
Quoted 10 lines · Stefan
Hi folks,
yesterday I upgraded kernel 2.6.19 to 2.6.20 (gentoo kernel). Now my box locks up about 10 min after boot.
After that I tested with a vanilla 2.6.21.1 it shows the same behavior. I'm attaching a kern log file from the 2.6.20. The 2.6.21.1 locked up so hard, that there was no trace left in the log file.
If I switch back to 2.6.19 everything is fine again.
If necessary I will hook up a laptop to this box, so I can capture messages via netconsole.Yes please.
Quoted 5 lines
This machine is running an AMD X2 64, NFORCE4 (ASUS A8N-E)
I haven't been following the development of libata for a while, but from the 2.6.20 changelog It looks like there have been some major changes.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Just let me know what kind of information you need in order to narrow it down.
Does giving 'sata_nv.adma=0' kernel parameter make any difference?
If adma=0 words, then that means it's either an ADMA related problem or that SAMSUNG HD401LJ drive has some problems with NCQ (since ADMA off means NCQ off as well). I would say the latter is more likely.
I don't have first hand experience with the particular model but I'll be surprised if they screwed their firmware up with new generation of harddisks. Firmware on the previous generation drives was pretty good and they don't get worse usually.
Quoted 3 lines
We should really have some kind of "noncq" kernel parameter we can use to help debugging these problems. Though, later kernels are supposed to switch it off automatically after too many errors..
Till now there hasn't been any case where a broken NCQ prevented a machine from booting but, yeah, having such thing would be nice for debugging.
Signature · 1 line
tejun
Re: System freezes with kernel >2.6.19 - sata_nv [added crash info kern 2.6.21.1]
Quoted 17 lines · Tejun Heo
Quoted 15 lines · Robert Hancock
Quoted 13 lines · Tejun Heo
Quoted 11 lines · Stefan
Hi folks,
yesterday I upgraded kernel 2.6.19 to 2.6.20 (gentoo kernel). Now my box locks up about 10 min after boot.
After that I tested with a vanilla 2.6.21.1 it shows the same behavior. I'm attaching a kern log file from the 2.6.20. The 2.6.21.1 locked up so hard, that there was no trace left in the log file.
If I switch back to 2.6.19 everything is fine again.
If necessary I will hook up a laptop to this box, so I can capture messages via netconsole.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Yes please.
Okay, I had time to set this up. I'm attaching the log messages I got via netconsole.
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
This machine is running an AMD X2 64, NFORCE4 (ASUS A8N-E)
I haven't been following the development of libata for a while, but from the 2.6.20 changelog It looks like there have been some major changes.
Just let me know what kind of information you need in order to narrow it down.
Does giving 'sata_nv.adma=0' kernel parameter make any difference?
I tested about 20h with adma disabled, the crash won't occur.
If I remove
sata_nv.adma=0
from boot options again it doesn't take long until my machine locks up.
[Attached dmesg output with 2.6.21.1 kernel + crash info I got via netconsole]
I hope this is useful to you guys.
Cheers Stefan
Quoted 21 lines
Quoted 4 lines
If adma=0 words, then that means it's either an ADMA related problem or that SAMSUNG HD401LJ drive has some problems with NCQ (since ADMA off means NCQ off as well). I would say the latter is more likely.
I don't have first hand experience with the particular model but I'll be surprised if they screwed their firmware up with new generation of harddisks. Firmware on the previous generation drives was pretty good and they don't get worse usually.
Quoted 4 lines
We should really have some kind of "noncq" kernel parameter we can use to help debugging these problems. Though, later kernels are supposed to switch it off automatically after too many errors..
Till now there hasn't been any case where a broken NCQ prevented a machine from booting but, yeah, having such thing would be nice for debugging.
Re: System freezes with kernel >2.6.19 - sata_nv [added crash info kern 2.6.21.1]
Quoted 10 lines · Stefan
Okay, I had time to set this up. I'm attaching the log messages I got via netconsole.
I tested about 20h with adma disabled, the crash won't occur.
If I remove
sata_nv.adma=0
Quoted 7 lines
from boot options again it doesn't take long until my machine locks up.
[Attached dmesg output with 2.6.21.1 kernel + crash info I got via netconsole]
I hope this is useful to you guys.
It looks like you've got SError bits set from the controller, 0x200000 means link layer CRC error (btw, we really should be decoding that error and printing it in human readable form rather than making people pore through the SATA spec and count bits). First thing you should try is replacing the SATA cable to that drive.
Signature · 3 lines
Robert Hancock Saskatoon, SK, Canada To email, remove "nospam" from [email protected] Home Page: http://www.roberthancock.com/
Re: System freezes with kernel >2.6.19 - sata_nv [added crash info kern 2.6.21.1]
Quoted 15 lines · Robert Hancock
Quoted 14 lines · Stefan
Okay, I had time to set this up. I'm attaching the log messages I got via netconsole.
I tested about 20h with adma disabled, the crash won't occur.
If I remove
sata_nv.adma=0
from boot options again it doesn't take long until my machine locks up.
[Attached dmesg output with 2.6.21.1 kernel + crash info I got via netconsole]
Quoted 6 lines
Quoted 1 line
I hope this is useful to you guys.
It looks like you've got SError bits set from the controller,
0x200000means link layer CRC error (btw, we really should be decoding that error and printing it in human readable form rather than making people pore through the SATA spec and count bits).
Hmmm... Maybe, but most of the bits are nearly meaningless to end users anyway.
Quoted 1 line
First thing you should try is replacing the SATA cable to that drive.
Yeap, please apply some hardware debugging techniques - replacing / reseating SATA cables and connecting it to different power connector. But it's disturbing to see machine lock up even if CRC error occurs. sata_nv non-adma interface locks the whole machine up too after certain error conditions but I thought adma was saner than that. I hope we can work around this somehow.
Thanks.
Signature · 1 line
tejun
Re: System freezes with kernel >2.6.19 - sata_nv [added crash info kern 2.6.21.1]
Quoted 33 lines · Tejun Heo
Quoted 16 lines · Robert Hancock
Quoted 14 lines · Stefan
Okay, I had time to set this up. I'm attaching the log messages I got via netconsole.
I tested about 20h with adma disabled, the crash won't occur.
If I remove
sata_nv.adma=0
from boot options again it doesn't take long until my machine locks up.
[Attached dmesg output with 2.6.21.1 kernel + crash info I got via netconsole]
Quoted 7 lines
Quoted 2 lines
I hope this is useful to you guys.
It looks like you've got SError bits set from the controller,
0x200000means link layer CRC error (btw, we really should be decoding that error and printing it in human readable form rather than making people pore through the SATA spec and count bits).Hmmm... Maybe, but most of the bits are nearly meaningless to end users anyway.
Quoted 2 lines
First thing you should try is replacing the SATA cable to that drive.
Quoted 10 lines
Yeap, please apply some hardware debugging techniques - replacing / reseating SATA cables and connecting it to different power connector. But it's disturbing to see machine lock up even if CRC error occurs.
sata_nvnon-adma interface locks the whole machine up too after certain error conditions but I thought adma was saner than that. I hope we can work around this somehow.Thanks.
I have just replaced the cables with brand new ones, still no change the machine freezes ~10min after boot.
Sometimes I even have to completely power down the machine to be able to boot again.
Re: System freezes with kernel >2.6.19 - sata_nv [added crash info kern 2.6.21.1]
Quoted 9 lines · Stefan
Quoted 9 lines
Quoted 2 lines
First thing you should try is replacing the SATA cable to that drive.
Yeap, please apply some hardware debugging techniques - replacing / reseating SATA cables and connecting it to different power connector. But it's disturbing to see machine lock up even if CRC error occurs.
sata_nvnon-adma interface locks the whole machine up too after certain error conditions but I thought adma was saner than that. I hope we can work around this somehow.
Quoted 7 lines
Quoted 3 lines
Thanks.
I have just replaced the cables with brand new ones, still no change the machine freezes ~10min after boot.
Sometimes I even have to completely power down the machine to be able to boot again.
Power problems would be another possibility..
Signature · 3 lines
Robert Hancock Saskatoon, SK, Canada To email, remove "nospam" from [email protected] Home Page: http://www.roberthancock.com/
Re: System freezes with kernel >2.6.19 - sata_nv [added crash info kern 2.6.21.1]
In reply to Tejun HeoQuoted 33 lines · Tejun Heo
Quoted 16 lines · Robert Hancock
Quoted 14 lines · Stefan
Okay, I had time to set this up. I'm attaching the log messages I got via netconsole.
I tested about 20h with adma disabled, the crash won't occur.
If I remove
sata_nv.adma=0
from boot options again it doesn't take long until my machine locks up.
[Attached dmesg output with 2.6.21.1 kernel + crash info I got via netconsole]
Quoted 7 lines
Quoted 2 lines
I hope this is useful to you guys.
It looks like you've got SError bits set from the controller,
0x200000means link layer CRC error (btw, we really should be decoding that error and printing it in human readable form rather than making people pore through the SATA spec and count bits).Hmmm... Maybe, but most of the bits are nearly meaningless to end users anyway.
Quoted 2 lines
First thing you should try is replacing the SATA cable to that drive.
Quoted 10 lines
Yeap, please apply some hardware debugging techniques - replacing / reseating SATA cables and connecting it to different power connector. But it's disturbing to see machine lock up even if CRC error occurs.
sata_nvnon-adma interface locks the whole machine up too after certain error conditions but I thought adma was saner than that. I hope we can work around this somehow.Thanks.
Hi folks, got some news:
I replaced cables, which didn't change anything. My PSU is strong enough to take a lot more, so I don't think it could be a power problem. Therefore I replaced the SAMSUNG HD401LJ with ST3160812AS. With the seagate attached I don't get the crash. So this may be a problem in combination with the HD401LJ+ NFORCE4 + ADMA.
--Stefan
Quoted 19 lines · Tejun Heo
Quoted 11 lines · Stefan
Hi folks,
yesterday I upgraded kernel 2.6.19 to 2.6.20 (gentoo kernel). Now my box locks up about 10 min after boot.
After that I tested with a vanilla 2.6.21.1 it shows the same behavior. I'm attaching a kern log file from the 2.6.20. The 2.6.21.1 locked up so hard, that there was no trace left in the log file.
If I switch back to 2.6.19 everything is fine again.
If necessary I will hook up a laptop to this box, so I can capture messages via netconsole.Yes please.
Quoted 2 lines
This machine is running an AMD X2 64, NFORCE4 (ASUS A8N-E)
Quoted 10 lines
Quoted 6 lines
I haven't been following the development of libata for a while, but from the 2.6.20 changelog It looks like there have been some major changes.
Just let me know what kind of information you need in order to narrow it down.
Does giving 'sata_nv.adma=0' kernel parameter make any difference?
It seems like switching off adma works. I didn't experience any problems within the last few hours.
Just to mention I also use windows 2003 on this box with NCQ enabled (I think the nvidia driver there states that ncq is active).
Should I recompile the kernel with some debugging options enabled?