Readd BUG for SMP TLB IPI
PATCH 1/1 Readd BUG for SMP TLB IPI
Readd the BUG for an spurious SMP TLB IPI.
Rationale:
The condition is fatal and it's better to have a BUG than a hang.
The goto out code forgets to ack the IPI in the APIC. When the IPI would really arrive at the wrong CPU it would immediately deadlock because the non-Acked IPI is retriggered. Adding an ACK for this path is also no good, because then the SMP flusher would need to detect this case and "retransmit" the IPI, otherwise it would hang too in the loop waiting for other CPUs. But nobody has ever seen such a hang, so it's safe to assume that all hardware guarantees it cannot happen.
-Andi
arch/i386/kernel/smp.c renamed from arch/i386/kernel/smp.c~ +1 -9
@@ -312,15 +312,7 @@ cpu =get_cpu ();if (!test_bit (cpu, &flush_cpumask)) - goto out; -/* -* This was a BUG() but until someone can quote me the -* line from the intel manual that guarantees an IPI to -* multiple CPUs is retried _only_ on the erroring CPUs -* its staying as a return -* -* BUG(); -*/ +BUG ();if (flush_mm == cpu_tlbstate[cpu].active_mm) {if (cpu_tlbstate[cpu].state ==TLBSTATE_OK ) {
Quoted 4 lines · Andi Kleen
Adding an ACK for this path is also no good, because then the SMP flusher would need to detect this case and "retransmit" the IPI, otherwise it would hang too in the loop waiting for other CPUs. But nobody has ever seen such a hang, so it's safe to assume that all hardware guarantees it cannot happen.
We have recorded retransmitted IPI's on some boards (notably the infamous BP6). They do happen. Early 2.4 had a fix for handling the replay case too, but someone lost it and BP6 boards no longer work as reliably.
An IPI can be retried and the retry for dual PII at least seems to hit all the CPUs. Even on a 2 CPU box this has been observed - I assume because the error was raised by the IO APIC when it got a garbled IPI.
Go ask Intel if you doubt it.
Quoted 4 lines · On 09 Jul 2003 12:27:02 +0100 Alan Cox <[email protected]> wrote:
We have recorded retransmitted IPI's on some boards (notably the infamous BP6). They do happen. Early 2.4 had a fix for handling the replay case too, but someone lost it and BP6 boards no longer work as reliably.
Ok, all non broken hardware.
Quoted 4 lines
An IPI can be retried and the retry for dual PII at least seems to hit all the CPUs. Even on a 2 CPU box this has been observed - I assume because the error was raised by the IO APIC when it got a garbled IPI.
I don't believe it. It would cause an immediate hang because the goto out does not ack the IPI. Surely such hangs would have been reported?
-Andi
Quoted 6 lines · Andi Kleen
Quoted 4 lines
We have recorded retransmitted IPI's on some boards (notably the infamous BP6). They do happen. Early 2.4 had a fix for handling the replay case too, but someone lost it and BP6 boards no longer work as reliably.
Ok, all non broken hardware.
It can happen to any PII/PIII box, its just very very rare on others, so rare I guess such crashes are in the noise.
Quoted 2 lines
I don't believe it. It would cause an immediate hang because the goto out does not ack the IPI. Surely such hangs would have been reported?
See - trawling bug data is *useful* 8)
For 2.4.x crashes relating to IPI replay bugs are reported very occasionally, for 2.5 I've no idea at all
Quoted 10 lines · On 09 Jul 2003 17:53:28 +0100 Alan Cox <[email protected]> wrote:
Quoted 6 lines · Andi Kleen
Quoted 4 lines
We have recorded retransmitted IPI's on some boards (notably the infamous BP6). They do happen. Early 2.4 had a fix for handling the replay case too, but someone lost it and BP6 boards no longer work as reliably.
Ok, all non broken hardware.
It can happen to any PII/PIII box, its just very very rare on others, so rare I guess such crashes are in the noise.
How do you know it an happen on them? Do you have backtraces?
If the BUG was there they wouldn't be in the noise. With the incomplete/broken handling it's just a silent hang.
-Andi
Quoted 4 lines · Andi Kleen
Quoted 2 lines
It can happen to any PII/PIII box, its just very very rare on others, so rare I guess such crashes are in the noise.
How do you know it an happen on them? Do you have backtraces?
I sat down with a BP6 owner and did lots of debugging.
Quoted 2 lines
If the BUG was there they wouldn't be in the noise. With the incomplete/broken handling it's just a silent hang.
I'm not arguing BUG is better than hang, but working right is better than BUG 8)
Quoted 7 lines · On 09 Jul 2003 18:04:15 +0100 Alan Cox <[email protected]> wrote:
Quoted 4 lines · Andi Kleen
Quoted 2 lines
It can happen to any PII/PIII box, its just very very rare on others, so rare I guess such crashes are in the noise.
How do you know it an happen on them? Do you have backtraces?
I sat down with a BP6 owner and did lots of debugging.
I meant on the not known-to-be-nearly-unusable boards. Or are you saying that on the other boards the APIC bus could be lossy too, but it's very unlikely?
[my personal feeling would be to consider the lossy APIC bus to be a hardware problem, like an MCE that cannot be really handled]
Quoted 4 lines
Quoted 2 lines
If the BUG was there they wouldn't be in the noise. With the incomplete/broken handling it's just a silent hang.
I'm not arguing BUG is better than hang, but working right is better than BUG 8)
I suspect when you have a lossy APIC bug you will run into problems with other IPIs too, it's really an uphill fight which you are likely to lose.
Anyways, having a clear BUG will make it easier to evaluate if there is really a problem.
-Andi
Quoted 8 lines · Andi Kleen
Quoted 3 lines
Quoted 1 line
How do you know it an happen on them? Do you have backtraces?
I sat down with a BP6 owner and did lots of debugging.
I meant on the not known-to-be-nearly-unusable boards. Or are you saying that on the other boards the APIC bus could be lossy too, but it's very unlikely? [my personal feeling would be to consider the lossy APIC bus to be a hardware problem, like an MCE that cannot be really handled]
APIC errors can occur on any box. The checksum ensures they get retransmitted but does mean you can ger replay of events
Quoted 2 lines
I suspect when you have a lossy APIC bug you will run into problems with other IPIs too, it's really an uphill fight which you are likely to lose.
If you follow intels recommendations it ought to just work. Its basically about checking if the message is one you processed and not repeating the execution
Compile failure 2.4.22-pre3-ac1
In reply to Alan CoxHello Alan,
I've tryed to compile 2.4.22-pre3-ac1, but every time I get this error:
arch/i386/kernel/kernel.o(.text.init+0x7803): In function `setup_ioapic_ids_from_mpc':
: undefined reference to xapic_support' arch/i386/kernel/kernel.o(.text.init+0x7a16): In function setup_ioapic_ids_from_mpc':
: undefined reference to `xapic_support'
make: *** [vmlinux] Error 1
I've tryed to search for patches from the mailing list with no luck, is there some patches for this?
Regards
Signature · 1 line
Markus Hästbacka <[email protected]>
Re: Compile failure 2.4.22-pre3-ac1
Quoted 11 lines · Midian
Hello Alan,
I've tryed to compile 2.4.22-pre3-ac1, but every time I get this error:arch/i386/kernel/kernel.o(.text.init+0x7803): In function `setup_ioapic_ids_from_mpc':
: undefined reference toxapic_support' arch/i386/kernel/kernel.o(.text.init+0x7a16): In functionsetup_ioapic_ids_from_mpc':
: undefined reference to `xapic_support'
make: *** [vmlinux] Error 1
Quoted 4 lines
I've tryed to search for patches from the mailing list with no luck, is there some patches for this?
Regards
I posted this workaround here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=105760102522650&w=2
but as Adrian Bunk pointed out in a response, the problem is that "changes to arch/i386/kernel/mpparse.c got lost at the update of -ac to -pre3".
If you want to be slightly more adventurous than using my workaround patch, you could copy the mpparse.c file from 2.4.21-ac4. That was compile tested but not run tested.
Steven