Skip to contentSkip to conversation

x86_64: Test patch for ATI/Nvidia timer problems

PATCH In review Andi Kleen 3 participants 8 messages Nov 26, 2005 – Dec 8, 2005

Subsystems (2)
lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

PATCH 1/1 x86_64: Test patch for ATI/Nvidia timer problems

Everybody who saw timing problems with ATI IXP based boards with x86-64 or some Nvidia NForce4 boards please test this patch. Please send success/failure to me.

Patch against 2.6.15rc2

Len, Ingo, I would be grateful if you could review the patch.

-Andi

Remove support for interrupt 0/timer routing on non ACPI compliant boards. Timer interrupt is now always routed through the local APIC and the 8259 is always fully masked when APIC mode is active. All the special case code for timer interrupts has been removed and interrupt 0 is now handled like all other interrupts.

This should fix persistent timer problems on ATI and some Nvidia boards. I tested it on different machines with different chipsets I had available, but it needs more testing.

On an ATI IXP machine I still have some minor instability in the local APIC timing compared to other chipsets, but the pattern is quite regular and shouldn't cause big problems.

It mainly consists of removing code. I also removed support for the IO-APIC watchdog, because it complicated the code paths and wasn't used. Some other code also went that wasn't directly related like support for MCA and EISA busses.

32bit is not changed right now, although it has the same troubles. Problem is that it still has to support the old broken legacy systems with weird interrupt routing. Perhaps the best way would be there to check if the machine has ACPI tables (not necessarily using ACPI, just checking if it claims to be ACPI compliant hardware) and if yes disable all the code I removed here on 64bit.

 Documentation/nmi_watchdog.txt |    4 
 arch/x86_64/kernel/apic.c      |   24 -
 arch/x86_64/kernel/i8259.c     |   48 +--
 arch/x86_64/kernel/io_apic.c   |  538 -----------------------------------------
 arch/x86_64/kernel/nmi.c       |   25 -
 arch/x86_64/kernel/setup.c     |    5 
 arch/x86_64/kernel/smpboot.c   |    6 
 drivers/acpi/sleep/main.c      |    2 
 include/asm-x86_64/apic.h      |    8 
 include/asm-x86_64/hw_irq.h    |   18 -
 include/asm-x86_64/io_apic.h   |    2 
 include/asm-x86_64/nmi.h       |    0 
 12 files changed, 51 insertions(+), 629 deletions(-)
Documentation/nmi_watchdog.txt renamed from Documentation/nmi_watchdog.txt-o +2 -2
diff -u linux-2.6.15rc2-work/Documentation/nmi_watchdog.txt-o linux-2.6.15rc2-work/Documentation/nmi_watchdog.txt
@@ -72,8 +72,8 @@
 you have to enable it with a boot time parameter.  Prior to 2.4.2-ac18
 the NMI-oopser is enabled unconditionally on x86 SMP boxes.
 
-On x86-64 the NMI oopser is on by default. On 64bit Intel CPUs
-it uses IO-APIC by default and on AMD it uses local APIC.
+On x86-64 the NMI watchdog is on by default if the CPU is supported. 
+Only the local APIC watchdog exists.
 
 [ feel free to send bug reports, suggestions and patches to
   Ingo Molnar <[email protected]> or the Linux SMP mailing
arch/x86_64/kernel/nmi.c renamed from arch/x86_64/kernel/nmi.c-o +1 -24
diff -u linux-2.6.15rc2-work/arch/x86_64/kernel/nmi.c-o linux-2.6.15rc2-work/arch/x86_64/kernel/nmi.c
@@ -116,7 +116,7 @@
 	if (nmi_known_cpu())
 		nmi_watchdog = NMI_LOCAL_APIC;
 	else
-		nmi_watchdog = NMI_IO_APIC;
+		printk(KERN_WARNING "Unknown CPU: no NMI watchdog support\n");
 }
 
 #ifdef CONFIG_SMP
@@ -264,27 +264,6 @@
 		enable_lapic_nmi_watchdog();
 }
 
-void disable_timer_nmi_watchdog(void)
-{
-	if ((nmi_watchdog != NMI_IO_APIC) || (nmi_active <= 0))
-		return;
-
-	disable_irq(0);
-	unset_nmi_callback();
-	nmi_active = -1;
-	nmi_watchdog = NMI_NONE;
-}
-
-void enable_timer_nmi_watchdog(void)
-{
-	if (nmi_active < 0) {
-		nmi_watchdog = NMI_IO_APIC;
-		touch_nmi_watchdog();
-		nmi_active = 1;
-		enable_irq(0);
-	}
-}
-
 #ifdef CONFIG_PM
 
 static int nmi_pm_active; /* nmi_active before suspend */
@@ -585,6 +564,4 @@
 EXPORT_SYMBOL(nmi_watchdog);
 EXPORT_SYMBOL(reserve_lapic_nmi);
 EXPORT_SYMBOL(release_lapic_nmi);
-EXPORT_SYMBOL(disable_timer_nmi_watchdog);
-EXPORT_SYMBOL(enable_timer_nmi_watchdog);
 EXPORT_SYMBOL(touch_nmi_watchdog);
arch/x86_64/kernel/setup.c renamed from arch/x86_64/kernel/setup.c-o +0 -5
diff -u linux-2.6.15rc2-work/arch/x86_64/kernel/setup.c-o linux-2.6.15rc2-work/arch/x86_64/kernel/setup.c
@@ -336,11 +336,6 @@
 #endif
 #endif
 
-		if (!memcmp(from, "disable_timer_pin_1", 19))
-			disable_timer_pin_1 = 1;
-		if (!memcmp(from, "enable_timer_pin_1", 18))
-			disable_timer_pin_1 = -1;
-
 		if (!memcmp(from, "nolapic", 7) ||
 		    !memcmp(from, "disableapic", 11))
 			disable_apic = 1;
arch/x86_64/kernel/io_apic.c renamed from arch/x86_64/kernel/io_apic.c-o +11 -527

664 lines not on this page · raw message

arch/x86_64/kernel/apic.c renamed from arch/x86_64/kernel/apic.c-o +9 -15
diff -u linux-2.6.15rc2-work/arch/x86_64/kernel/apic.c-o linux-2.6.15rc2-work/arch/x86_64/kernel/apic.c
@@ -129,7 +129,7 @@
 	}
 }
 
-void disconnect_bsp_APIC(int virt_wire_setup)
+void disconnect_bsp_APIC(void)
 {
 	if (pic_mode) {
 		/*
@@ -153,20 +153,14 @@
 		value |= 0xf;
 		apic_write_around(APIC_SPIV, value);
 
-		if (!virt_wire_setup) {
-			/* For LVT0 make it edge triggered, active high, external and enabled */
-			value = apic_read(APIC_LVT0);
-			value &= ~(APIC_MODE_MASK | APIC_SEND_PENDING |
-				APIC_INPUT_POLARITY | APIC_LVT_REMOTE_IRR |
-				APIC_LVT_LEVEL_TRIGGER | APIC_LVT_MASKED );
-			value |= APIC_LVT_REMOTE_IRR | APIC_SEND_PENDING;
-			value = SET_APIC_DELIVERY_MODE(value, APIC_MODE_EXTINT);
-			apic_write_around(APIC_LVT0, value);
-		}
-		else {
-			/* Disable LVT0 */
-			apic_write_around(APIC_LVT0, APIC_LVT_MASKED);
-		}
+		/* For LVT0 make it edge triggered, active high, external and enabled */
+		value = apic_read(APIC_LVT0);
+		value &= ~(APIC_MODE_MASK | APIC_SEND_PENDING |
+			   APIC_INPUT_POLARITY | APIC_LVT_REMOTE_IRR |
+			   APIC_LVT_LEVEL_TRIGGER | APIC_LVT_MASKED );
+		value |= APIC_LVT_REMOTE_IRR | APIC_SEND_PENDING;
+		value = SET_APIC_DELIVERY_MODE(value, APIC_MODE_EXTINT);
+		apic_write_around(APIC_LVT0, value);
 
 		/* For LVT1 make it edge triggered, active high, nmi and enabled */
 		value = apic_read(APIC_LVT1);
arch/x86_64/kernel/i8259.c renamed from arch/x86_64/kernel/i8259.c-o +15 -33

90 lines not on this page · raw message

arch/x86_64/kernel/smpboot.c renamed from arch/x86_64/kernel/smpboot.c-o +0 -6
diff -u linux-2.6.15rc2-work/arch/x86_64/kernel/smpboot.c-o linux-2.6.15rc2-work/arch/x86_64/kernel/smpboot.c
@@ -516,12 +516,6 @@
 
 	Dprintk("cpu %d: enabling apic timer\n", smp_processor_id());
 
-	if (nmi_watchdog == NMI_IO_APIC) {
-		disable_8259A_irq(0);
-		enable_NMI_through_LVT0(NULL);
-		enable_8259A_irq(0);
-	}
-
 	enable_APIC_timer();
 
 	/*
drivers/acpi/sleep/main.c renamed from drivers/acpi/sleep/main.c-o +2 -0
diff -u linux-2.6.15rc2-work/drivers/acpi/sleep/main.c-o linux-2.6.15rc2-work/drivers/acpi/sleep/main.c
@@ -137,10 +137,12 @@
 	/* reset firmware waking vector */
 	acpi_set_firmware_waking_vector((acpi_physical_address) 0);
 
+#ifndef CONFIG_X86_64
 	if (init_8259A_after_S1) {
 		printk("Broken toshiba laptop -> kicking interrupts\n");
 		init_8259A(0);
 	}
+#endif
 	return 0;
 }
 
include/asm-x86_64/apic.h renamed from include/asm-x86_64/apic.h-o +4 -4
diff -u linux-2.6.15rc2-work/include/asm-x86_64/apic.h-o linux-2.6.15rc2-work/include/asm-x86_64/apic.h
@@ -77,7 +77,7 @@
 extern int get_maxlvt (void);
 extern void clear_local_APIC (void);
 extern void connect_bsp_APIC (void);
-extern void disconnect_bsp_APIC (int virt_wire_setup);
+extern void disconnect_bsp_APIC (void);
 extern void disable_local_APIC (void);
 extern int verify_local_APIC (void);
 extern void cache_APIC_registers (void);
@@ -91,8 +91,9 @@
 extern void setup_apic_nmi_watchdog (void);
 extern int reserve_lapic_nmi(void);
 extern void release_lapic_nmi(void);
-extern void disable_timer_nmi_watchdog(void);
-extern void enable_timer_nmi_watchdog(void);
+/* For oprofile. not needed anymore */
+static inline void enable_timer_nmi_watchdog(void) {}
+static inline void disable_timer_nmi_watchdog(void) {}
 extern void nmi_watchdog_tick (struct pt_regs * regs, unsigned reason);
 extern int APIC_init_uniprocessor (void);
 extern void disable_APIC_timer(void);
@@ -105,7 +106,6 @@
 extern unsigned int nmi_watchdog;
 #define NMI_DEFAULT	-1
 #define NMI_NONE	0
-#define NMI_IO_APIC	1
 #define NMI_LOCAL_APIC	2
 #define NMI_INVALID	3
 
include/asm-x86_64/hw_irq.h renamed from include/asm-x86_64/hw_irq.h-o +6 -12
diff -u linux-2.6.15rc2-work/include/asm-x86_64/hw_irq.h-o linux-2.6.15rc2-work/include/asm-x86_64/hw_irq.h
@@ -93,8 +93,7 @@
 extern void enable_8259A_irq(unsigned int irq);
 extern int i8259A_irq_pending(unsigned int irq);
 extern void make_8259A_irq(unsigned int irq);
-extern void init_8259A(int aeoi);
-extern void FASTCALL(send_IPI_self(int vector));
+extern void send_IPI_self(int vector);
 extern void init_VISWS_APIC_irqs(void);
 extern void setup_IO_APIC(void);
 extern void disable_IO_APIC(void);
@@ -102,13 +101,12 @@
 extern int IO_APIC_get_PCI_irq_vector(int bus, int slot, int fn);
 extern void send_IPI(int dest, int vector);
 extern void setup_ioapic_dest(void);
-
-extern unsigned long io_apic_irqs;
+extern void disable_8259A_completely(void);
 
 extern atomic_t irq_err_count;
 extern atomic_t irq_mis_count;
 
-#define IO_APIC_IRQ(x) (((x) >= 16) || ((1<<(x)) & io_apic_irqs))
+#define IO_APIC_IRQ(x) 1
 
 #define __STR(x) #x
 #define STR(x) __STR(x)
@@ -130,14 +128,10 @@
 	"push $" #nr "-256 ; " \
 	"jmp common_interrupt");
 
-#if defined(CONFIG_X86_IO_APIC)
-static inline void hw_resend_irq(struct hw_interrupt_type *h, unsigned int i) {
-	if (IO_APIC_IRQ(i))
-		send_IPI_self(IO_APIC_VECTOR(i));
+static inline void hw_resend_irq(struct hw_interrupt_type *h, unsigned int i) 
+{
+	send_IPI_self(IO_APIC_VECTOR(i));
 }
-#else
-static inline void hw_resend_irq(struct hw_interrupt_type *h, unsigned int i) {}
-#endif
 
 #define platform_legacy_irq(irq)	((irq) < 16)
 

diff -u linux-2.6.15rc2-work/include/asm-x86_64/nmi.h-o linux-2.6.15rc2-work/include/asm-x86_64/nmi.h

include/asm-x86_64/io_apic.h renamed from include/asm-x86_64/io_apic.h-o +1 -1
diff -u linux-2.6.15rc2-work/include/asm-x86_64/io_apic.h-o linux-2.6.15rc2-work/include/asm-x86_64/io_apic.h
@@ -199,7 +199,7 @@
  * If we use the IO-APIC for IRQ routing, disable automatic
  * assignment of PCI IRQ's.
  */
-#define io_apic_assign_pci_irqs (mp_irq_entries && !skip_ioapic_setup && io_apic_irqs)
+#define io_apic_assign_pci_irqs (!skip_ioapic_setup)
 
 #ifdef CONFIG_ACPI
 extern int io_apic_get_version (int ioapic);
#1 lore raw

Hi,

Quoted 3 lines · Andi Kleen

Everybody who saw timing problems with ATI IXP based boards with x86-64 or some Nvidia NForce4 boards please test this patch. Please send success/failure to me.

I try to give your patch a try on the ATI based MSI Megabook S270, today - however even with the workaround of "noapic" I had timer drift on resuem from ram if the cpu was scaled to a lower frequency when it was suspended.

The k8 cpufreq code failed to assert the current frequency and thus assumed a wrong one:

Restarting tasks...<3>powernow-k8: ignoring illegal change in lo freq table-0 to 0x0
powernow-k8: transition frequency failed
done

Also my ACPI table only has two frequency entries, 800000 and 1600000 MHz - I wonder if one could rework the powernow-k8 driver to interpolate values in between to get smoother adaption of the frequency?

Yours,

Signature · 3 lines
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            http://www.exactcode.de | http://www.t2-project.org
            +49 (0)30  255 897 45
#2 lore raw

Hi!

Quoted 12 lines · Andi Kleen
Quoted 3 lines

Everybody who saw timing problems with ATI IXP based boards with x86-64 or some Nvidia NForce4 boards please test this patch. Please send success/failure to me.

I try to give your patch a try on the ATI based MSI Megabook S270, today - however even with the workaround of "noapic" I had timer drift on resuem from ram if the cpu was scaled to a lower frequency when it was suspended.

The k8 cpufreq code failed to assert the current frequency and thus assumed a wrong one:

Quoted 8 lines

Restarting tasks...<3>powernow-k8: ignoring illegal change in lo freq table-0 to 0x0
powernow-k8: transition frequency failed
done

Also my ACPI table only has two frequency entries, 800000 and 1600000 MHz - I wonder if one could rework the powernow-k8 driver to interpolate values in between to get smoother adaption of the frequency?

Unfortunately, hardware can not go to anything between.
Pavel

Signature · 1 line
Thanks, Sharp!
author
#3 lore raw
In reply to René Rebe
Quoted 10 lines · Ren? Rebe

Hi,

Quoted 3 lines · Andi Kleen

Everybody who saw timing problems with ATI IXP based boards with x86-64 or some Nvidia NForce4 boards please test this patch. Please send success/failure to me.

I try to give your patch a try on the ATI based MSI Megabook S270, today - however even with the workaround of "noapic" I had timer drift on resuem from ram if the cpu was scaled to a lower frequency when it was suspended.

But it worked properly before suspend/resume without noapic?

-Andi

#4 lore raw

Hi,

Quoted 8 lines · Andi Kleen
Quoted 7 lines · Ren? Rebe

Hi,

Quoted 3 lines · Andi Kleen

Everybody who saw timing problems with ATI IXP based boards with x86-64 or some Nvidia NForce4 boards please test this patch. Please send success/failure to me.

Quoted 5 lines
Quoted 3 lines

I try to give your patch a try on the ATI based MSI Megabook S270, today - however even with the workaround of "noapic" I had timer drift on resuem from ram if the cpu was scaled to a lower frequency when it was suspended.

But it worked properly before suspend/resume without noapic?

Without noapic the timer has about the 2x speed compared to real-time. I only used the machien with noapic since otherwise it is barely useful.

Did you want to know if suspend/resume does behave differently without noapic?

Yours,

Signature · 3 lines
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            http://www.exactcode.de | http://www.t2-project.org
            +49 (0)30  255 897 45
author
#5 lore raw
Quoted 11 lines · Ren? Rebe

Hi,

Quoted 8 lines · Andi Kleen
Quoted 7 lines · Ren? Rebe

Hi,

Quoted 3 lines · Andi Kleen

Everybody who saw timing problems with ATI IXP based boards with x86-64 or some Nvidia NForce4 boards please test this patch. Please send success/failure to me.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

I try to give your patch a try on the ATI based MSI Megabook S270, today - however even with the workaround of "noapic" I had timer drift on resuem from ram if the cpu was scaled to a lower frequency when it was suspended.

But it worked properly before suspend/resume without noapic?

Without noapic the timer has about the 2x speed compared to real-time. I only used the machien with noapic since otherwise it is barely useful.

It has that still with the patch applied? The patch was supposed to fix that at least part of that problem on ATI systems (there seems to be also a timer miscalibration problem on some other laptops)

-Andi

#6 lore raw

Hi,

Quoted 9 lines · Andi Kleen
Quoted 4 lines
Quoted 1 line

But it worked properly before suspend/resume without noapic?

Without noapic the timer has about the 2x speed compared to real-time. I only used the machien with noapic since otherwise it is barely useful.

It has that still with the patch applied? The patch was supposed to fix that at least part of that problem on ATI systems (there seems to be also a timer miscalibration problem on some other laptops)

Sorry for the late reply, just too much to do ... It appears my MSI Megabook S270 with Ati chipset and AMD Turion freezes on boot with your patch applied to 2.6.14.2 after the io schedulers are registered. Without the patch it boots up fine.

Yours,

Signature · 3 lines
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            http://www.exactcode.de | http://www.t2-project.org
            +49 (0)30  255 897 45
author
#7 lore raw
Quoted 14 lines · Ren? Rebe

Hi,

Quoted 9 lines · Andi Kleen
Quoted 4 lines
Quoted 1 line

But it worked properly before suspend/resume without noapic?

Without noapic the timer has about the 2x speed compared to real-time. I only used the machien with noapic since otherwise it is barely useful.

It has that still with the patch applied? The patch was supposed to fix that at least part of that problem on ATI systems (there seems to be also a timer miscalibration problem on some other laptops)

Quoted 4 lines

Sorry for the late reply, just too much to do ... It appears my MSI Megabook S270 with Ati chipset and AMD Turion freezes on boot with your patch applied to 2.6.14.2 after the io schedulers are registered. Without the patch it boots up fine.

Ok thanks.

Does it work when booted with acpi_skip_timer_override ?

I sometimes wish this ATI chipset wouldn't exist - its timers are an endless headache. Admittedly the Linux code for this is somewhat screwy too, but their hardware also doesn't seem to be quite kosher.

-Andi