APIC error on SMP machine

Linux-2.4.22-SMP, 1 GB RAM, devfs, gcc-3.2.3.

Hi,

Today, my dual PIII (Coppermine) refused to boot, and wrote a large number of these messages to the serial console instead:

APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)
APIC error on CPU1: 04(04)

Can anyone tell me what these might mean, please? The kernel source implies that it's a "Send accept error", but this doesn't help me in an "Ah, I can fix that!" sense.

Does this APIC error just mean that the CPU is unhappy in this slot, and is refusing to listen to the motherboard? Or is the motherboard refusing to listen to the CPU?

Background:
This machine has been misbehaving for a while. I thought I had worked around the problem by underclocking the FSB from 133 MHz to 100 MHz, but that now looks like it was just a "reprieve". I have tried running "nosmp", "pci=noacpi" and "noapic pci=noacpi" without success, and have resorted to yanking the CPU out of this slot entirely. (I suspect that the CPU is fine, however.) I have also restored the FSB to 133 MHz, so I am currently running the SMP kernel on a single 933 MHz PIII.

Cheers,
Chris

Neither. An APIC send accept error means that when trying to send an interrupt, it was not accepted by the target. In this case, the target is a CPU, either your other CPU or the same one (a CPU can send itself an interrupt).

While there are several reasons why this can happen, the most common ones are:

1) The target CPU is "full". The local APIC on P54Cs through P3s only has two interrupt latches per interrupt "level", which is the high nibble of the IRQ vector number. So, if a CPU had already latched interrupt vectors 0x30 and 0x3A, it would have to reject any other 0x3X vector that was sent until it could service one of the two latched vectors.

You can force this to happen by manually binding too many IRQs that happen to be on the same "level" to one CPU, then causing a lot of interrupt traffic on those devices.

In order to avoid this problem, Linux spreads the IRQs among as many vector levels as possible. Still, the vector assignment is done before any devices have requested interrupts. You may get unlucky and have 3 devices on one level.

2) The interrupt cannot be delivered because something is wrong with it. This can happen if the kernel screws up and picks "clustered" APIC mode on a "flat" system or vice versa. A dual P3 system should be flat. Check your dmesg log to make sure it was properly detected. (This seldom happens unless you're doing interrupt development work in Linux.)

3) Maybe the other CPU is broken and physically cannot accept the interrupt. Do any previous kernels boot?

James Cleverdon
IBM xSeries Linux Solutions
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

target.

...

Given the background, the most likely cause would seem to be bad a CPU/motherboard connection. I have realised that the APIC error is for CPU1, but I have actually removed CPU0. And a bad CPU0 would explain why "nosmp" didn't work either.

It's a pity that "nosmp" doesn't (logially cannot?) take a "boot CPU number" as a parameter.

Not any more. Everything started to hit the fan at the beginning of August, and I thought that I had "patched" things by underclocking the FSB. However, that only seems to have delayed the inevitable. CPU slot 2 on my motherboard just seems not to work any more. I have no idea why - it's not like I can see a lot of dust and dirt in there.

Oh well, I hear that Dell are selling dual 2.6 GHz Xeons with RedHat preinstalled nowadays. (These should have "hyperthreading support", right ;-) ?)

Cheers,
Chris

________________________________________________________________________
Want to chat instantly with your online friends? Get the FREE Yahoo! Messenger http://mail.messenger.yahoo.co.uk

I rewrote that code to make it spit out those messages that you see. That however doesn't mean I know what I'm doing....

The APIC chip has a bit register that indicates errors. The kernel, reads the register, stores it, and that should clear the error. Just to be sure, we read it again, and store the result. Then we print the two results.

In your case, the APIC seems to have a problem, and it doesn't go away when we read the register, as it should.

On my "BP6" motherboard, I often see 04(08) errors: The error changes after I read it once.

The code was printing the whole bitflag shebang before reading it again, allowing the system to generate another error in the meanwhile, and hanging the machine. To prevent this, I modified it to just print the raw bits, trusting that you'd be knowledgable enough to grep through the kernel sources to find the definitions of the bits. That proved true. And as expected, you (just like me) don't know what to do with the definition of that bit anyway.

On the BP6 it seems that the APIC bus is a bit noisy. So we get transmission errors on that bus, allowing for a variety of errors on the recieving end. In your case, the errors seem to end up happening faster than the machine can handle :-(

Roger.

** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
**** "Linux is like a wigwam -  no windows, no gates, apache inside!" ****