Skip to contentSkip to conversation

APIC routing broken on ASUS P2B-DS

2.4.21-pre3 Simon Kirby 3 participants 5 messages Jan 28, 2003 – Feb 19, 2003

lore
Share & tools
lore
#0 lore raw

Something broke between 2.4.20 and 2.4.21-pre3 which is causing interrupts to not be routed the second CPU. I saw the problem on one box and copied the kernel to another which then had the same problem (both ASUS P2B-DS boards, one with PIII CPUs, one with PII CPUs).

[sroot@devel:/]# cat /proc/interrupts
           CPU0       CPU1       
  0:     114480          0    IO-APIC-edge  timer
  1:          2          0    IO-APIC-edge  keyboard
  2:          0          0          XT-PIC  cascade
  8:          3          0    IO-APIC-edge  rtc
 16:      30707          0   IO-APIC-level  eth0
 19:       3260          0   IO-APIC-level  aic7xxx
NMI:          0          0 
LOC:     114405     114404 
ERR:          0
MIS:          0

With 2.4.20:

[sroot@devel:/root]# cat /proc/interrupts 
           CPU0       CPU1       
  0:      23627      27340    IO-APIC-edge  timer
  1:          0          2    IO-APIC-edge  keyboard
  2:          0          0          XT-PIC  cascade
  8:          2          1    IO-APIC-edge  rtc
 16:       1671          2   IO-APIC-level  eth0
 19:        965        979   IO-APIC-level  aic7xxx
NMI:          0          0 
LOC:      50893      50891 
ERR:          0
MIS:          0

...If nobody knows what happened immediately, I'll try narrowing it down to the patch.

Simon-

[        Simon Kirby        ][        Network Operations        ]
[     [email protected]     ][     NetNation Communications     ]

[ Opinions expressed are not necessarily those of my employer. ]

#1 lore raw
Quoted 5 lines · Simon Kirby

Something broke between 2.4.20 and 2.4.21-pre3 which is causing interrupts to not be routed the second CPU. I saw the problem on one box and copied the kernel to another which then had the same problem (both ASUS P2B-DS boards, one with PIII CPUs, one with PII CPUs).

Quoted 12 lines
[sroot@devel:/]# cat /proc/interrupts
           CPU0       CPU1       
  0:     114480          0    IO-APIC-edge  timer
  1:          2          0    IO-APIC-edge  keyboard
  2:          0          0          XT-PIC  cascade
  8:          3          0    IO-APIC-edge  rtc
 16:      30707          0   IO-APIC-level  eth0
 19:       3260          0   IO-APIC-level  aic7xxx
NMI:          0          0 
LOC:     114405     114404 
ERR:          0
MIS:          0

Hi !

Well, yesterday I've noticed the same problem here on an Asus A7M266-D. I haven't had time to test every previous version, but I've searched through the archives, and other similar reports have been posted since December. I believe that some people did have it with 2.4.21pre2 too.

Your report narrows the problem down to fewer releases than what I saw first, and if I have time I'll test 2.4.20 and 2.4.21pre1 here.

BTW, I don't know if -ac exhibits the same problem. Alan, would you check your SMP box, please ?

Cheers,
Willy

#2 lore raw

[Nearly Solved]: APIC routing broken on ASUS P2B-DS

Quoted 5 lines · Willy Tarreau
Quoted 4 lines · Simon Kirby

Something broke between 2.4.20 and 2.4.21-pre3 which is causing interrupts to not be routed the second CPU. I saw the problem on one box and copied the kernel to another which then had the same problem (both ASUS P2B-DS boards, one with PIII CPUs, one with PII CPUs).

Hi !

I noticed that 2.4.21-pre4 had the same problem whereas -ac1 and -aa1 worked fine. But unfortunately, this was unrelated since both use irq_balance which seems to work around or fix the problem. So I searched back the earlier versions, and finally narrowed this problem down to the introduction of CONFIG_X86_NUMA and associated code in 2.4.21pre1.

If I compile my kernel for an SMP K7, only CPU0 gets the interrupts. But if I enable CONFIG_X86_CLUSTERED_APIC by enabling either CONFIG_X86_NUMAQ or CONFIG_X86_SUMMIT (CONFIG_X86_NUMA alone isn't enough), then I get my interrupts distributed across both CPUs. This is on an Asus A7M266D with 2 Athlon XP 1800+. I don't know if this option can affect performance or stability. BTW, the system runs in Flat APIC mode, as reported at boot time. I can provide dmesg on request, but didn't want to pollute the list.

I looked through the code but since I don't know much about APIC, I didn't understand the changes nor how they would affect what I observed.

Anyone has any clue ?

Cheers,
Willy

#3 lore raw

Re: [Nearly Solved]: APIC routing broken on ASUS P2B-DS

Quoted 4 lines · Willy Tarreau

If I compile my kernel for an SMP K7, only CPU0 gets the interrupts. But if I enable CONFIG_X86_CLUSTERED_APIC by enabling either CONFIG_X86_NUMAQ or CONFIG_X86_SUMMIT (CONFIG_X86_NUMA alone isn't enough), then I get my interrupts distributed across both CPUs. This is on an Asus A7M266D with 2 Athlon XP 1800+.

did you ever get a response on this?

The answer is a big fat "don't do that" ;-)

Summit and Numa are two things your box definitely does not have... don't enable those options. If you still have problems with those options disabled, please re-post...

Jeff

#4 lore raw

Re: [Nearly Solved]: APIC routing broken on ASUS P2B-DS

Quoted 10 lines · Jeff Garzik
Quoted 6 lines · Willy Tarreau

If I compile my kernel for an SMP K7, only CPU0 gets the interrupts. But if I enable CONFIG_X86_CLUSTERED_APIC by enabling either CONFIG_X86_NUMAQ or CONFIG_X86_SUMMIT (CONFIG_X86_NUMA alone isn't enough), then I get my interrupts
distributed across both CPUs. This is on an Asus A7M266D with 2 Athlon XP 1800+.

did you ever get a response on this?

Well, Maciej Rozycki replied to me privately saying that he would look at the problem though he didn't have time right now. I can't blame him, I didn't either...

Quoted 5 lines

The answer is a big fat "don't do that" ;-)

Summit and Numa are two things your box definitely does not have... don't enable those options. If you still have problems with those options disabled, please re-post...

If I manage to get a few hours, I might compare the initialization code between these different modes and try to determine what should be added or removed in the classical code to make it work. Unfortunately, I'm fairly busy right now, and trying to understand this unknown area will take some time :-(

Willy