Working nforce2, was Re: Fixes for nforce2 hard lockup, apic, io-apic, udma133 covered
Quoted 21 lines
Quoted 2 lines
APIC error on CPU0: 02(02) what?? no crash though.
[...]
Quoted 9 lines
bob@where cat
/proc/interruptsCPU0
0: 3350153 IO-APIC-edge timer 1: 5775 IO-APIC-edge i8042
2: 0 XT-PIC cascade
8: 1 IO-APIC-edge rtc
9: 0 IO-APIC-level acpi
12: 5385 IO-APIC-edge i8042 14: 10 IO-APIC-edge ide0Quoted 8 lines
15: 10 IO-APIC-edge ide1
16: 1717957 IO-APIC-level ide2, ide3, eth0 19: 472929 IO-APIC-level ide4, ide5 21: 0 IO-APIC-level NVidia nForce2
NMI: 822
LOC: 3350073
ERR: 35
MIS: 15818
Quoted 4 lines
It looks like the infamous APIC delivery bug -- the "MIS" counter shows how many level-triggered interrupts has been erronously delivered as edge-triggered ones. No wonder the system shows instability -- you have noise problems at the APIC bus.
Thanks Maciej
I was wondering about those, I had seen the work around code and would not have thought it need apply to recent athlon chipsets?
For comparison here is my proc/interrupts
CPU0
0: 50462204 IO-APIC-edge timer 1: 49153 IO-APIC-edge keyboard 2: 0 XT-PIC cascade 9: 0 IO-APIC-level acpi 12: 395912 IO-APIC-edge PS/2 Mouse 14: 995872 IO-APIC-edge ide0 15: 283 IO-APIC-edge ide1 16: 3921102 IO-APIC-level nvidia 18: 2 IO-APIC-level bttv 20: 136325 IO-APIC-level eth0, usb-ohci 21: 146903 IO-APIC-level ehci_hcd, NVIDIA nForce Audio 22: 0 IO-APIC-level usb-ohci NMI: 0 LOC: 50457798 ERR: 0 MIS: 0
Albatron KM18G-Pro, nforce2, pheonix bios, 2200XP, 255fsb, ddr400, ide0 is hard drive, ide1 is cdrom, nmi watchdog off
Report seems OK but this machine locks up hard without the apic delay patch.
I am currently trying the simpler v1 (always add a delay) patch but on all apic acks as per this posting
http://linux.derkeiler.com/Mailing-Lists/Kernel/2003-12/3291.html
which is a reply to an earlier posting of the same name but I accidently omitted the Re in the subject.
Regards,
Ross.
Just to give the status here ...
Im still running the original 2.6 test 11 patches for apic and ioapic. Uptime is now 2d 20h with lots of idle time and hard work too..
/proc/interrupts as follows:
CPU0
0: 245382420 IO-APIC-edge timer 1: 139577 IO-APIC-edge i8042 2: 0 XT-PIC cascade 8: 3 IO-APIC-edge rtc 9: 0 IO-APIC-level acpi 12: 1478615 IO-APIC-edge i8042 14: 1055548 IO-APIC-edge ide0 15: 737664 IO-APIC-edge ide1 19: 18405692 IO-APIC-level radeon@PCI:3:0:0 21: 5257090 IO-APIC-level ehci_hcd, NVidia nForce2, eth0 22: 3 IO-APIC-level ohci1394 NMI: 14944 LOC: 245087891 ERR: 0 MIS: 6
As for NMI.. I actually forget which I booted from... I think =1, but NMI is a small number now.. would it have wrapped?
Craig
A7N8X Deluxe V2 BIOS 1007
Quoted 72 lines · Ross Dickson
Quoted 21 lines
Quoted 2 lines
APIC error on CPU0: 02(02) what?? no crash though.
[...]
Quoted 9 lines
bob@where cat
/proc/interruptsCPU0
0: 3350153 IO-APIC-edge timer 1: 5775 IO-APIC-edge i8042
2: 0 XT-PIC cascade
8: 1 IO-APIC-edge rtc
9: 0 IO-APIC-level acpi
12: 5385 IO-APIC-edge i8042 14: 10 IO-APIC-edge ide0Quoted 8 lines
15: 10 IO-APIC-edge ide1
16: 1717957 IO-APIC-level ide2, ide3, eth0 19: 472929 IO-APIC-level ide4, ide5 21: 0 IO-APIC-level NVidia nForce2
NMI: 822
LOC: 3350073
ERR: 35
MIS: 15818Quoted 4 lines
It looks like the infamous APIC delivery bug -- the "MIS" counter shows how many level-triggered interrupts has been erronously delivered as edge-triggered ones. No wonder the system shows instability -- you have noise problems at the APIC bus.
Thanks Maciej
I was wondering about those, I had seen the work around code and would not have thought it need apply to recent athlon chipsets?For comparison here is my proc/interrupts
CPU00: 50462204 IO-APIC-edge timer 1: 49153 IO-APIC-edge keyboard 2: 0 XT-PIC cascade 9: 0 IO-APIC-level acpi 12: 395912 IO-APIC-edge PS/2 Mouse 14: 995872 IO-APIC-edge ide0 15: 283 IO-APIC-edge ide1 16: 3921102 IO-APIC-level nvidia 18: 2 IO-APIC-level bttv 20: 136325 IO-APIC-level eth0, usb-ohci 21: 146903 IO-APIC-level ehci_hcd, NVIDIA nForce Audio 22: 0 IO-APIC-level usb-ohci NMI: 0 LOC: 50457798 ERR: 0 MIS: 0Albatron KM18G-Pro, nforce2, pheonix bios, 2200XP, 255fsb, ddr400, ide0 is hard drive, ide1 is cdrom, nmi watchdog off
Report seems OK but this machine locks up hard without the apic delay patch.
I am currently trying the simpler v1 (always add a delay) patch but on all apic acks as per this posting
http://linux.derkeiler.com/Mailing-Lists/Kernel/2003-12/3291.html
which is a reply to an earlier posting of the same name but I accidently omitted the Re in the subject.
Regards,
Ross.-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 17 lines · Craig Bradney
CPU0
0: 245382420 IO-APIC-edge timer 1: 139577 IO-APIC-edge i8042 2: 0 XT-PIC cascade 8: 3 IO-APIC-edge rtc 9: 0 IO-APIC-level acpi 12: 1478615 IO-APIC-edge i8042 14: 1055548 IO-APIC-edge ide0 15: 737664 IO-APIC-edge ide1 19: 18405692 IO-APIC-level radeon@PCI:3:0:0 21: 5257090 IO-APIC-level ehci_hcd, NVidia nForce2, eth0 22: 3 IO-APIC-level ohci1394 NMI: 14944 LOC: 245087891 ERR: 0 MIS: 6
Quoted 1 line
As for NMI.. I actually forget which I booted from... I think =1, but NMI is a small number now.. would it have wrapped?
That's "=2" -- otherwise the NMI count would be rougly the same as the
sum of counts for IRQ 0 for all processors. And you can actually get your kernel's command line from /proc/cmdline.
Signature · 3 lines
+ Maciej W. Rozycki, Technical University of Gdansk, Poland + +--------------------------------------------------------------+ + e-mail: [email protected], PGP key available +
Quoted 7 lines · you
Just to give the status here ...
Im still running the original 2.6 test 11 patches for apic and ioapic. Uptime is now 2d 20h with lots of idle time and hard work too..
/proc/interruptsas follows:CPU0
Quoted 15 lines
0: 245382420 IO-APIC-edge timer 1: 139577 IO-APIC-edge i8042 2: 0 XT-PIC cascade 8: 3 IO-APIC-edge rtc 9: 0 IO-APIC-level acpi 12: 1478615 IO-APIC-edge i8042 14: 1055548 IO-APIC-edge ide0 15: 737664 IO-APIC-edge ide1 19: 18405692 IO-APIC-level radeon@PCI:3:0:0 21: 5257090 IO-APIC-level ehci_hcd, NVidia nForce2, eth0 22: 3 IO-APIC-level ohci1394 NMI: 14944 LOC: 245087891 ERR: 0 MIS: 6
Uptime sounds good so far.
I am not convinced my v2 apic patch is a great overall improvement, I am thinking v1 apic, is safer for now.
Having said that
Ian Kumlien currently has an uptime of
1 day, 15 hours +
on v2 patches but with the apic delay timeout increased from 600UL to 800UL. He has a Barton core - see below.
Quoted 5 lines
Craig
A7N8X Deluxe V2 BIOS 1007
<snip>
Quoted 8 lines
Quoted 8 lines
I am currently trying the simpler v1 (always add a delay) patch but on all apic acks as per this posting
http://linux.derkeiler.com/Mailing-Lists/Kernel/2003-12/3291.html
which is a reply to an earlier posting of the same name but I accidently omitted the Re in the subject.
I don't think it is necessary to put the delay in all apic acks - I just tried it to see if it worked and have not yet put my code back the way it was. My hard lockups went away with the original v1 apic
timer delay patch anyway.
Please note in that (above) posting I write that I stuffed up the #ifdefs in my v1 and v2 patches and adjust code accordingly. Patches worked but were only testing on the first config item after #ifdef
apic code should have had
#if defined(CONFIG_MK7) && defined(CONFIG_BLK_DEV_AMD74XX)
ioapic code should have had
#if defined(CONFIG_ACPI_BOOT) && defined(CONFIG_X86_UP_IOAPIC)
Brief summary at this point
1) 2? reports are in that latest award bios with "C1 disconnect" set to "auto?" may remove need for apic ack delay patch and still keep cpu thermo managed
2) apic ack delay v1 patch seems safe for all cpu cores but introduces a small delay of about half the time of an XTPIC access on each apic timer interrupt
3) apic ack delay v2 patch seems safe only on barton cores and gives more debugging info and wastes less time than apic v1 patch
4) io-apic v2 patch gives more debugging info but functions same as io-apic v1 patch
Regards
Ross
Ross,
my_make_script nf2-800UL 2>&1 | tee /tmp/make.err
#/tmp/make.err
<snip>
CC arch/i386/kernel/apic.oarch/i386/kernel/apic.c: In function `smp_apic_timer_interrupt':arch/i386/kernel/apic.c:1105: warning: unsigned int format, long unsigned int arg (arg 2)
...which is around the printk line here--
printk("..APIC TIMER ack delay, reload:%u, safe:%u\n",
+ if(!passno) { /* calculate timing */
+ safecnt = apic_read(APIC_TMICT) -
+ ( (800UL * apic_read(APIC_TMICT) ) /
+ (1000000000UL/HZ) );
+ printk("..APIC TIMER ack delay, reload:%u, safe:%u\n",
+ apic_read(APIC_TMICT), safecnt);
+ passno++;
Here are the two patches with "#ifdef N" to "#if defined(N)" change but not the unsigned int change --
arch/i386/kernel/apic.c +0 -0
@@ -1089,6 +1089,37 @@
*/ irq_stat[cpu].apic_timer_irqs++;
+#if defined(CONFIG_MK7) && defined(CONFIG_BLK_DEV_AMD74XX)
+ /*
+ * on 2200XP & nforce2 chipset we need 600ns? 800? 1000? 1100? + * from timer irq start to apic irq ack to prevent + * hard lockups, use apic timer itself. + * C1 disconnect bit related. Ross Dickson. + */
+ {
+ static unsigned int passno, safecnt;
+ if(!passno) { /* calculate timing */
+ safecnt = apic_read(APIC_TMICT) -
+ ( (800UL * apic_read(APIC_TMICT) ) /
+ (1000000000UL/HZ) );
+ printk("..APIC TIMER ack delay, reload:%u, safe:%u\n",
+ apic_read(APIC_TMICT), safecnt);
+ passno++;
+ }
+#if APIC_DEBUG
+ if(passno<12) {
+ unsigned int at1 = apic_read(APIC_TMCCT);
+ if( passno > 1 )
+ Dprintk("..APIC TIMER ack delay, predelay count:%u \n", at1 );
+ passno++;
+ }
+#endif
+ /* delay only if required */
+ while( apic_read(APIC_TMCCT) > safecnt )
+ ndelay(100);
+ }
+#endif
+
/*
* NOTE! We'd better ACK the irq immediately,
* because timer handling can be slow.*/
arch/i386/kernel/io_apic.c +0 -0
@@ -2128,6 +2128,54 @@
printk(KERN_ERR "..MP-BIOS bug: 8254 timer not connected to IO-APIC\n"); }
+#if defined (CONFIG_ACPI_BOOT) && (CONFIG_X86_UP_IOAPIC)
+ /* for nforce2 try vector 0 on pin0
+ * Note 8259a is already masked, also by default
+ * the io_apic_set_pci_routing call disables the 8259 irq 0
+ * so we must be connected directly to the 8254 timer if this works
+ * Note2: this violates the above comment re Subtle but works!
+ */
+ printk(KERN_INFO "..TIMER: Is timer irq0 connected to IOAPIC Pin0? ...\n");
+ if (pin1 != -1) {
+ extern spinlock_t i8259A_lock;
+ unsigned long flags;
+ int tok, saved_timer_ack = timer_ack;
+ /*
+ * Ok, does IRQ0 through the IOAPIC work?
+ */
+ io_apic_set_pci_routing ( 0, 0, 0, 0, 0); /* connect pin */
+ unmask_IO_APIC_irq(0);
+ timer_ack = 0;+
+ /*
+
+
+
+ * Ok, does IRQ0 through the IOAPIC work?
+ */
+ spin_lock_irqsave(&i8259A_lock, flags);
+ Dprintk("..TIMER check 8259 ints disabled, imr1:%02x, imr2:%02x\n", inb(0x21), inb(0xA1));
+ tok = timer_irq_works();
+ spin_unlock_irqrestore(&i8259A_lock, flags);
+ if (tok) {
+ if (nmi_watchdog == NMI_IO_APIC) {
+ disable_8259A_irq(0);
+ setup_nmi();
+ enable_8259A_irq(0);
+ check_nmi_watchdog();
+ }
+ printk(KERN_INFO "..TIMER: works OK on apic pin0 irq0\n" );
+ return;
+ }
+ /* failed */
+ timer_ack = saved_timer_ack;
+ clear_IO_APIC_pin(0, 0);
+ io_apic_set_pci_routing ( 0, pin1, 0, 0, 0);
+ printk(KERN_ERR "..MP-BIOS bug: 8254 timer not connected to IO-APIC Pin 0\n");+ }
+/* end new stuff for nforce2 */
+#endif
+
printk(KERN_INFO "...trying to set up timer (IRQ0) through the 8259A ... ");
if (pin2 != -1) {
printk("\n..... (found pin %d) ...", pin2);