Athlon64 + Nforce4 MCE panic
Hello!
Tonight I had a kernel panic (full hang) with the following on the console:
CPU 0: Machine Check Exception: 0000000000000004
Bank 4: b200000000070f0f
Kernel panic - not syncing: CPU context corrupt
I tried to decode this:
# ./parsemce -e 0000000000000004 -b 4 -s b200000000070f0f -a 0
Status: (4) Machine Check in progress.
Restart IP invalid.
parsebank(4): b200000000070f0f @ 0
External tag parity error
CPU state corrupt. Restart not possible
Error enabled in control register
Error not corrected.
Bus and interconnect error
Participation: Generic
Timeout:
Request: Generic error
Transaction type : Invalid
Memory/IO : Other# echo 'CPU 0: Machine Check Exception: 0000000000000004 Bank 4: b200000000070f0f' | mcelog --ascii --k8
HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 0 4 northbridge Northbridge Watchdog error
bit57 = processor context corrupt
bit61 = error uncorrected
bus error 'generic participation, request timed out
generic error mem transaction
generic access, level generic'
STATUS b200000000070f0f MCGSTATUS 4I've been searching for and found some additional info, it looks like I'm not the first experiencing this problem:
http://kerneltrap.org/node/4993
Here ^^^ it is suggested that there was some thread about A64 MCEs on LKML but I failed to find it so I decided to post... sorry if it's redundant.
The kernel is 2.6.16-gentoo-r9 and the CPU is an A64 "Venice" 3500+ I can post further hw/sw environment information if req'd.
Thanks!
Regards,
Sab
Quoted 39 lines · Rumi Szabolcs
Hello!
Tonight I had a kernel panic (full hang) with the following on the console:
CPU 0: Machine Check Exception: 0000000000000004
Bank 4:b200000000070f0fKernel panic - not syncing: CPU context corruptI tried to decode this:
# ./parsemce -e 0000000000000004 -b 4 -s b200000000070f0f -a 0 Status: (4) Machine Check in progress. Restart IP invalid. parsebank(4): b200000000070f0f @ 0 External tag parity error CPU state corrupt. Restart not possible Error enabled in control register Error not corrected. Bus and interconnect error Participation: Generic Timeout: Request: Generic error Transaction type : Invalid Memory/IO : Other# echo 'CPU 0: Machine Check Exception: 0000000000000004 Bank 4: b200000000070f0f' | mcelog --ascii --k8 HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor CPU 0 4 northbridge Northbridge Watchdog error bit57 = processor context corrupt bit61 = error uncorrected bus error 'generic participation, request timed out generic error mem transaction generic access, level generic' STATUS b200000000070f0f MCGSTATUS 4I've been searching for and found some additional info, it looks like I'm not the first experiencing this problem:
Quoted 8 lines
http://kerneltrap.org/node/4993
Here ^^^ it is suggested that there was some thread about A64 MCEs on LKML but I failed to find it so I decided to post... sorry if it's redundant.
The kernel is 2.6.16-gentoo-r9 and the CPU is an A64 "Venice" 3500+ I can post further hw/sw environment information if req'd.
I had this same problem, search the lkml if you're interested in investigating this further, I had the problem, was told it was hardware, replaced my motherboard with the same model, the MCE continued to occur, replaced the motherboard with a different model and the MCE went away. It was some Biostar IIRC, but you can check the archives if you really want to find out.
Good luck
Signature · 3 lines
avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
Ar Iau, 2006-07-13 am 09:01 +0200, ysgrifennodd Rumi Szabolcs:
Quoted 3 lines
# echo 'CPU 0: Machine Check Exception: 0000000000000004 Bank 4: b200000000070f0f' | mcelog --ascii --k8 HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It prints this bit for a reason
MCE almost always occurs because your processor detected an internal inconsistency, in this case a parity error. So you almost certainly have a real hardware problem.
Fixing it tends to depend on the system. With bigger servers you usually find that the MCE info produces the correct response because the server people actually understand use and care about MCEs even in other-os.
For random cheap desktop PC systems it can be more fun, reporting an MCE is as likely to cause them to send you a new monitor as any other part unfortunately 8)
Quoted 6 lines
CPU 0: Machine Check Exception: 0000000000000004
Bank 4:b200000000070f0fKernel panic - not syncing: CPU context corruptI tried to decode this:
Quoted 14 lines
# ./parsemce -e 0000000000000004 -b 4 -s b200000000070f0f -a 0
Status: (4) Machine Check in progress.
Restart IP invalid.
parsebank(4): b200000000070f0f @ 0
External tag parity error
CPU state corrupt. Restart not possible
Error enabled in control register
Error not corrected.
Bus and interconnect error
Participation: Generic
Timeout:
Request: Generic error
Transaction type : Invalid
Memory/IO : OtherI'm not sure why parsemce says this is a parity error, it's a K8 virtual
northbridge watchdog timeout on an I/O transaction. In other words a
CPU PIO transaction to some device timed out, the timeout is usually set to 10s. Prior to K8 these types of errors would usually be system hardlocks.
I've debugged a few of these before and they usually end up being IDE device issues. The IDE controller is really thin (the ATA registers actually reside in the device) so for example if the CPU goes to read the ATA status register of some device and the device doesn't respond, the PCI transaction won't be terminated and the watchdog will fire.
-Allen
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
Quoted 6 lines · On Thu, 13 Jul 2006 21:30:28 -0700 "Allen Martin" <[email protected]> wrote:
I'm not sure why parsemce says this is a parity error, it's a K8 virtual
northbridge watchdog timeout on an I/O transaction. In other words a
CPU PIO transaction to some device timed out, the timeout is usually set to 10s. Prior to K8 these types of errors would usually be system hardlocks.
Quoted 7 lines
I've debugged a few of these before and they usually end up being IDE device issues. The IDE controller is really thin (the ATA registers actually reside in the device) so for example if the CPU goes to read the ATA status register of some device and the device doesn't respond, the PCI transaction won't be terminated and the watchdog will fire.
-Allen
Hmm, this is informative, thanks a lot!
Actually what I've g00gled about the issue so far it seems that only Nforce4 + A64 users used to get exactly the same error, some are indeed suggesting NF4 is buggy when it comes to IDE DMA... Isn't it possible that this is some NF4 specific problem, a chipset bug or a motherboard design issue (some sporadic signal distortion problem) that is specific to some but not all NF4-based motherboards?
In fact this particular system I'm having the problem with contains the following parts:
1x Asrock NF4G-SATA2 motherboard (http://www.asrock.com/product/939NF4G-SATA2.htm)
1x Athlon64 "Venice" 3500+ with a huge Arctic cooler
1x Corsair kit of 2 matched 512MB DDR400 modules
1x Seagate 160GB SATA drive
1x well ventilated Chieftec rackmount chassis w/PSU
The former three are brand new, none of them was ever overclocked or otherwise tortured, the latter two are reliably working since years. The environmental temperatures were optimal at the time the panic occurred, so it probably was not a heat related issue. In fact the system was rather unloaded (idling around 3AM). The uptime was about two weeks without any problem until the panic occurred. So I have a reason to believe that this could be a chipset specific problem which not only affects me but quite a number of NF4 users, most of which (using Windo$$$) will probably never know why their system suddenly hung after some weeks or months of use...
Or maybe just a neutrino hit to the CPU?
What do you think?
Regards,
Sab
Quoted 6 lines
1x Asrock NF4G-SATA2 motherboard
(http://www.asrock.com/product/939NF4G-SATA2.htm)
1x Athlon64 "Venice" 3500+ with a huge Arctic cooler 1x Corsair kit of 2 matched 512MB DDR400 modules 1x Seagate 160GB SATA drive
1x well ventilated Chieftec rackmount chassis w/PSU
You don't have any PATA devices at all? SATA is a lot more resilient to this type of problem.
Quoted 4 lines
So I have a reason to believe that this could be a chipset specific problem which not only affects me but quite a number of NF4 users, most of which (using Windo$$$) will probably never know why their system suddenly hung after some weeks or months of use...
Windows will generate a bugcheck on an MCA exception just like Linux. We have really detailed statistics on Windows bugchecks due to OCA, so I know this is not a widespread issue at least on Windows.
Quoted 2 lines
Or maybe just a neutrino hit to the CPU?
What do you think?
The stack trace will almost always tell you exactly what device timed out the PIO, you should start there.
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
Quoted 9 lines · On Fri, 14 Jul 2006 11:18:38 -0700 "Allen Martin" <[email protected]> wrote:
Quoted 6 lines
1x Asrock NF4G-SATA2 motherboard
(http://www.asrock.com/product/939NF4G-SATA2.htm)
1x Athlon64 "Venice" 3500+ with a huge Arctic cooler 1x Corsair kit of 2 matched 512MB DDR400 modules 1x Seagate 160GB SATA drive
1x well ventilated Chieftec rackmount chassis w/PSUYou don't have any PATA devices at all? SATA is a lot more resilient to this type of problem.
Exactly. I hooked up a PATA CDROM temporarily when I changed the mobo, but otherwise the only storage it has is that SATA disk.
Quoted 8 lines
Quoted 4 lines
So I have a reason to believe that this could be a chipset specific problem which not only affects me but quite a number of NF4 users, most of which (using Windo$$$) will probably never know why their system suddenly hung after some weeks or months of use...
Windows will generate a bugcheck on an MCA exception just like Linux. We have really detailed statistics on Windows bugchecks due to OCA, so I know this is not a widespread issue at least on Windows.
Do you think that this problem is caused by or at least triggered by Linux or it's disk usage patterns? At least if you type "b200000000070f0f" into g00gle you get a lot of hits all of which has something to do with Linux.
Quoted 2 lines
The stack trace will almost always tell you exactly what device timed out the PIO, you should start there.
Well I have to admit I'm not a kernel hacker but a simple user without a clue so I have no idea how I could have got a stack trace from a kernel that is paniced, frozen hard, and couldn't even sync it's disks so after I realized there is no reaction to any input devices on the system I've just hit the reset button and hoped it will come up again. I guess this is exactly what Windo$$$ users used to do when they get a blue screen... ;)
Thanks!
Regards,
Sab