APIC routing broken on ASUS P2B-DS
Something broke between 2.4.20 and 2.4.21-pre3 which is causing interrupts to not be routed the second CPU. I saw the problem on one box and copied the kernel to another which then had the same problem (both ASUS P2B-DS boards, one with PIII CPUs, one with PII CPUs).
[sroot@devel:/]# cat /proc/interrupts
CPU0 CPU1
0: 114480 0 IO-APIC-edge timer
1: 2 0 IO-APIC-edge keyboard
2: 0 0 XT-PIC cascade
8: 3 0 IO-APIC-edge rtc
16: 30707 0 IO-APIC-level eth0
19: 3260 0 IO-APIC-level aic7xxx
NMI: 0 0
LOC: 114405 114404
ERR: 0
MIS: 0With 2.4.20:
[sroot@devel:/root]# cat /proc/interrupts
CPU0 CPU1
0: 23627 27340 IO-APIC-edge timer
1: 0 2 IO-APIC-edge keyboard
2: 0 0 XT-PIC cascade
8: 2 1 IO-APIC-edge rtc
16: 1671 2 IO-APIC-level eth0
19: 965 979 IO-APIC-level aic7xxx
NMI: 0 0
LOC: 50893 50891
ERR: 0
MIS: 0...If nobody knows what happened immediately, I'll try narrowing it down to the patch.
Simon-
[ Simon Kirby ][ Network Operations ] [ [email protected] ][ NetNation Communications ]
[ Opinions expressed are not necessarily those of my employer. ]
Quoted 5 lines · Simon Kirby
Something broke between 2.4.20 and 2.4.21-pre3 which is causing interrupts to not be routed the second CPU. I saw the problem on one box and copied the kernel to another which then had the same problem (both ASUS P2B-DS boards, one with PIII CPUs, one with PII CPUs).
Quoted 12 lines
[sroot@devel:/]# cat /proc/interrupts
CPU0 CPU1
0: 114480 0 IO-APIC-edge timer
1: 2 0 IO-APIC-edge keyboard
2: 0 0 XT-PIC cascade
8: 3 0 IO-APIC-edge rtc
16: 30707 0 IO-APIC-level eth0
19: 3260 0 IO-APIC-level aic7xxx
NMI: 0 0
LOC: 114405 114404
ERR: 0
MIS: 0Hi !
Well, yesterday I've noticed the same problem here on an Asus A7M266-D. I haven't had time to test every previous version, but I've searched through the archives, and other similar reports have been posted since December. I believe that some people did have it with 2.4.21pre2 too.
Your report narrows the problem down to fewer releases than what I saw first, and if I have time I'll test 2.4.20 and 2.4.21pre1 here.
BTW, I don't know if -ac exhibits the same problem. Alan, would you check your SMP box, please ?
Cheers,
Willy
[Nearly Solved]: APIC routing broken on ASUS P2B-DS
Quoted 5 lines · Willy Tarreau
Quoted 4 lines · Simon Kirby
Something broke between 2.4.20 and 2.4.21-pre3 which is causing interrupts to not be routed the second CPU. I saw the problem on one box and copied the kernel to another which then had the same problem (both ASUS P2B-DS boards, one with PIII CPUs, one with PII CPUs).
Hi !
I noticed that 2.4.21-pre4 had the same problem whereas -ac1 and -aa1 worked fine. But unfortunately, this was unrelated since both use irq_balance which seems to work around or fix the problem. So I searched back the earlier versions, and finally narrowed this problem down to the introduction of CONFIG_X86_NUMA and associated code in 2.4.21pre1.
If I compile my kernel for an SMP K7, only CPU0 gets the interrupts. But if I enable CONFIG_X86_CLUSTERED_APIC by enabling either CONFIG_X86_NUMAQ or CONFIG_X86_SUMMIT (CONFIG_X86_NUMA alone isn't enough), then I get my interrupts distributed across both CPUs. This is on an Asus A7M266D with 2 Athlon XP 1800+. I don't know if this option can affect performance or stability. BTW, the system runs in Flat APIC mode, as reported at boot time. I can provide dmesg on request, but didn't want to pollute the list.
I looked through the code but since I don't know much about APIC, I didn't understand the changes nor how they would affect what I observed.
Anyone has any clue ?
Cheers,
Willy
Re: [Nearly Solved]: APIC routing broken on ASUS P2B-DS
Quoted 4 lines · Willy Tarreau
If I compile my kernel for an SMP K7, only CPU0 gets the interrupts. But if I enable
CONFIG_X86_CLUSTERED_APICby enabling eitherCONFIG_X86_NUMAQorCONFIG_X86_SUMMIT(CONFIG_X86_NUMAalone isn't enough), then I get my interrupts distributed across both CPUs. This is on an Asus A7M266D with 2 Athlon XP 1800+.
did you ever get a response on this?
The answer is a big fat "don't do that" ;-)
Summit and Numa are two things your box definitely does not have... don't enable those options. If you still have problems with those options disabled, please re-post...
Jeff
Re: [Nearly Solved]: APIC routing broken on ASUS P2B-DS
Quoted 10 lines · Jeff Garzik
Quoted 6 lines · Willy Tarreau
If I compile my kernel for an SMP K7, only CPU0 gets the interrupts. But if I enable
CONFIG_X86_CLUSTERED_APICby enabling eitherCONFIG_X86_NUMAQorCONFIG_X86_SUMMIT(CONFIG_X86_NUMAalone isn't enough), then I get my interrupts
distributed across both CPUs. This is on an Asus A7M266D with 2 Athlon XP 1800+.did you ever get a response on this?
Well, Maciej Rozycki replied to me privately saying that he would look at the problem though he didn't have time right now. I can't blame him, I didn't either...
Quoted 5 lines
The answer is a big fat "don't do that" ;-)
Summit and Numa are two things your box definitely does not have... don't enable those options. If you still have problems with those options disabled, please re-post...
If I manage to get a few hours, I might compare the initialization code between these different modes and try to determine what should be added or removed in the classical code to make it work. Unfortunately, I'm fairly busy right now, and trying to understand this unknown area will take some time :-(
Willy