AMD C1E aware idle support
Subsystems (2)
cover AMD C1E aware idle support
AMD CPUs with C1E support are currently excluded from high resolution timers and NOHZ support. The reason is that C1E is a BIOS controlled C3 power state which switches off TSC and the local APIC timer. The ACPI C-State control manages the TSC/local APIC timer wreckage, but this does not include the C1 based ("halt" instruction) C1E mode. The BIOS/SMM controlled C1E state works on most systems even without enabling ACPI C-State control.
The fact that a system has C1E support enabled is advertised in a MSR, but the time during boot when the C1E bit is set by the BIOS varies:
1) Boot CPU has already C1E bit set
2) Secondary CPU sets C1E bit
3) C1E bit is set after ACPI C-State query
Case #1 and #2 are covered by the current implementation, but case #3 results in a complete system lockup due to missing timer interrupts. The current solution is to disable the local APIC timer and use the PIT in broadcast mode. This restricts the C1E enabled systems to periodic timer mode.
The following patch series implements a C1E aware idle function which also covers the late C1E enablement (case #3):
The function is selected during boot for CPUs which have possibly C1E support. The function checks the MSR which contains the C1E active bits before executing the halt instruction. When one of the C1E active bits is set, it makes the system C1E aware by enabling the timer broadcast mechanism for all CPUs. For high resolution timer and/or nohz enabled systems it calls the oneshot timer broadcast mechanism before executing the halt instruction. This is the same mechanism which is used in the ACPI C-State control for C2/C3 power states.
On my C1E affected X2 box these patches reduce the wakeups/sec down to 20 according to powertop.
The patches work fine on systems which are not affected by the dreaded ATI chipset timer wreckage. On those which have the problem, the box needs help from the keyboard to continue working.
The x86 changes for .27 contain a complete overhaul of the affected code, but this is out of scope for this patchset.
For those who are interested to test those patches on top of 2.6.26-rc I extracted a patch and added it to the c1e series. It's available from:
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
or
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2
@Macej: I bisected your patches and the commit which solves the mysterious hangs is:
x86: I/O APIC: timer through 8259A second-chance (7e3530cd98a0c6ab38f5898e855a5beffab26561 in linux-2.6-tip.git)
That's the patch which you worried about possible impacts, but it seems that it actually fixes the stupid timer irq issue finally. I have tested it on various machines which had timer irq problems in the past and they all run smothly. Great work!
Thanks,
tglx
PATCH 1/6 x86: simplify idle selection
default_idle is selected in cpu_idle(), when no other idle routine is selected. Select it in select_idle_routine() when mwait is not selected.
--- arch/x86/kernel/process.c | 18 +++++++----------- arch/x86/kernel/process_32.c | 7 +------ arch/x86/kernel/process_64.c | 7 ++----- 3 files changed, 10 insertions(+), 22 deletions(-)
arch/x86/kernel/process.c +7 -11
@@ -139,27 +139,23 @@ static int __cpuinit mwait_usable(constvoid __cpuinit select_idle_routine (const struct cpuinfo_x86 *c) { -static int selected; - -if (selected) -return ;#ifdef CONFIG_X86_SMP if (pm_idle == poll_idle && smp_num_siblings >1 ) {printk (KERN_WARNING "WARNING: polling idle and HT enabled," " performance may degrade.\n" ); }#endif +if (pm_idle) +return ; +if (cpu_has (c,X86_FEATURE_MWAIT ) &&mwait_usable (c)) {/* -* Skip, if setup has overridden idle. * One CPU supports mwait => All CPUs supports mwait */ -if (!pm_idle) { -printk (KERN_INFO "using mwait in idle threads.\n" ); - pm_idle = mwait_idle; - } - } - selected =1 ; +printk (KERN_INFO "using mwait in idle threads.\n" ); + pm_idle = mwait_idle; + }else + pm_idle = default_idle; }static int __init idle_setup (char *str)
arch/x86/kernel/process_32.c +1 -6
@@ -168,24 +168,19 @@ void cpu_idle(void)while (1 ) {tick_nohz_stop_sched_tick ();while (!need_resched ()) { -void (*idle)(void );check_pgt_cache ();rmb (); - idle = pm_idle;if (rcu_pending (cpu))rcu_check_callbacks (cpu,0 ); -if (!idle) - idle = default_idle; -if (cpu_is_offline (cpu))play_dead ();local_irq_disable ();__get_cpu_var (irq_stat).idle_timestamp = jiffies; -idle (); +pm_idle (); }tick_nohz_restart_sched_tick ();preempt_enable_no_resched ();
arch/x86/kernel/process_64.c +2 -5
@@ -150,12 +150,9 @@ void cpu_idle(void)while (1 ) {tick_nohz_stop_sched_tick ();while (!need_resched ()) { -void (*idle)(void );rmb (); - idle = pm_idle; -if (!idle) - idle = default_idle; +if (cpu_is_offline (smp_processor_id ()))play_dead ();/*
@@ -165,7 +162,7 @@ void cpu_idle(void)*/ local_irq_disable ();enter_idle (); -idle (); +pm_idle ();/* In many cases the interrupt that ended idle has already called exit_idle. But some idle loops can be woken up without interrupt. */
PATCH 2/6 x86: cleanup C1E enabled detection
In reply to Thomas GleixnerRename the MSR to INT_PENDING_MSG, which is the name in the data sheet as well. Move the C1E mask to the header file.
--- arch/x86/kernel/cpu/amd.c | 5 ++--- arch/x86/kernel/setup_64.c | 5 ++--- include/asm-x86/msr-index.h | 4 +++- 3 files changed, 7 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/amd.c +2 -3
@@ -25,7 +25,6 @@ extern void vide(void); __asm__(".align 4\nvide: ret" );#ifdef CONFIG_X86_LOCAL_APIC -#define ENABLE_C1E_MASK 0x18000000#define CPUID_PROCESSOR_SIGNATURE 1#define CPUID_XFAM 0x0ff00000#define CPUID_XFAM_K8 0x00000000
@@ -45,8 +44,8 @@ static __cpuinit int amd_apic_timer_brokbreak ;case CPUID_XFAM_10H :case CPUID_XFAM_11H : -rdmsr (MSR_K8_ENABLE_C1E , lo, hi); -if (lo &ENABLE_C1E_MASK ) { +rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); +if (lo &K8_INTP_C1E_ACTIVE_MASK ) {if (smp_processor_id () != boot_cpu_physical_apicid)printk (KERN_INFO "AMD C1E detected late. " " Force timer broadcast.\n" );
arch/x86/kernel/setup_64.c +2 -3
@@ -682,7 +682,6 @@ static void __cpuinit early_init_amd_mc(#endif } -#define ENABLE_C1E_MASK 0x18000000#define CPUID_PROCESSOR_SIGNATURE 1#define CPUID_XFAM 0x0ff00000#define CPUID_XFAM_K8 0x00000000
@@ -702,8 +701,8 @@ static __cpuinit int amd_apic_timer_brokbreak ;case CPUID_XFAM_10H :case CPUID_XFAM_11H : -rdmsr (MSR_K8_ENABLE_C1E , lo, hi); -if (lo &ENABLE_C1E_MASK ) +rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); +if (lo &K8_INTP_C1E_ACTIVE_MASK )return 1 ;break ;default :
include/asm-x86/msr-index.h +3 -1
@@ -111,7 +111,9 @@#define MSR_K8_TOP_MEM2 0xc001001d#define MSR_K8_SYSCFG 0xc0010010#define MSR_K8_HWCR 0xc0010015 -#define MSR_K8_ENABLE_C1E 0xc0010055 +#define MSR_K8_INT_PENDING_MSG 0xc0010055 +/* C1E active bits in int pending message */ +#define K8_INTP_C1E_ACTIVE_MASK 0x18000000#define MSR_K8_TSEG_ADDR 0xc0010112#define K8_MTRRFIXRANGE_DRAM_ENABLE 0x00040000/* MtrrFixDramEn bit */ #define K8_MTRRFIXRANGE_DRAM_MODIFY 0x00080000/* MtrrFixDramModEn bit */
PATCH 3/6 x86: use cpuinfo to check for interrupt pending message msr
In reply to Thomas GleixnerNo need to do a cpuid(1) again. The cpuinfo structure has all necessary information already.
--- arch/x86/kernel/cpu/amd.c | 41 +++++++++++++++-------------------------- arch/x86/kernel/setup_64.c | 38 +++++++++++++++----------------------- 2 files changed, 30 insertions(+), 49 deletions(-)
arch/x86/kernel/cpu/amd.c +15 -26
@@ -25,35 +25,24 @@ extern void vide(void); __asm__(".align 4\nvide: ret" );#ifdef CONFIG_X86_LOCAL_APIC -#define CPUID_PROCESSOR_SIGNATURE 1 -#define CPUID_XFAM 0x0ff00000 -#define CPUID_XFAM_K8 0x00000000 -#define CPUID_XFAM_10H 0x00100000 -#define CPUID_XFAM_11H 0x00200000 -#define CPUID_XMOD 0x000f0000 -#define CPUID_XMOD_REV_F 0x00040000/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */ -static __cpuinit int amd_apic_timer_broken (void ) +static __cpuinit int amd_apic_timer_broken (struct cpuinfo_x86 *c) {u32 lo, hi; -u32 eax =cpuid_eax (CPUID_PROCESSOR_SIGNATURE ); -switch (eax &CPUID_XFAM ) { -case CPUID_XFAM_K8 : -if ((eax &CPUID_XMOD ) <CPUID_XMOD_REV_F ) -break ; -case CPUID_XFAM_10H : -case CPUID_XFAM_11H : -rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); -if (lo &K8_INTP_C1E_ACTIVE_MASK ) { -if (smp_processor_id () != boot_cpu_physical_apicid) -printk (KERN_INFO "AMD C1E detected late. " -" Force timer broadcast.\n" ); -return 1 ; - } -break ; -default : -/* err on the side of caution */ + +if (c->x86 <0x0F ) +return 0 ; + +/* Family 0x0f models < rev F do not have this MSR */ +if (c->x86 ==0x0f && c->x86_model <0x40 ) +return 0 ; + +rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); +if (lo &K8_INTP_C1E_ACTIVE_MASK ) { +if (smp_processor_id () != boot_cpu_physical_apicid) +printk (KERN_INFO "AMD C1E detected late. " +"Force timer broadcast.\n" );return 1 ; }return 0 ;
@@ -297,7 +286,7 @@ static void __cpuinit init_amd(struct cp }#ifdef CONFIG_X86_LOCAL_APIC -if (amd_apic_timer_broken ()) +if (amd_apic_timer_broken (c)) local_apic_timer_disabled =1 ;#endif
arch/x86/kernel/setup_64.c +15 -23
@@ -682,31 +682,23 @@ static void __cpuinit early_init_amd_mc(#endif } -#define CPUID_PROCESSOR_SIGNATURE 1 -#define CPUID_XFAM 0x0ff00000 -#define CPUID_XFAM_K8 0x00000000 -#define CPUID_XFAM_10H 0x00100000 -#define CPUID_XFAM_11H 0x00200000 -#define CPUID_XMOD 0x000f0000 -#define CPUID_XMOD_REV_F 0x00040000 -/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */ -static __cpuinit int amd_apic_timer_broken (void ) +static __cpuinit int amd_apic_timer_broken (struct cpuinfo_x86 *c) { -u32 lo, hi, eax =cpuid_eax (CPUID_PROCESSOR_SIGNATURE ); +u32 lo, hi; -switch (eax &CPUID_XFAM ) { -case CPUID_XFAM_K8 : -if ((eax &CPUID_XMOD ) <CPUID_XMOD_REV_F ) -break ; -case CPUID_XFAM_10H : -case CPUID_XFAM_11H : -rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); -if (lo &K8_INTP_C1E_ACTIVE_MASK ) -return 1 ; -break ; -default : -/* err on the side of caution */ +if (c->x86 <0x0F ) +return 0 ; + +/* Family 0x0f models < rev F do not have this MSR */ +if (c->x86 ==0x0f && c->x86_model <0x40 ) +return 0 ; + +rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); +if (lo &K8_INTP_C1E_ACTIVE_MASK ) { +if (smp_processor_id () != boot_cpu_physical_apicid) +printk (KERN_INFO "AMD C1E detected late. " +"Force timer broadcast.\n" );return 1 ; }return 0 ;
@@ -789,7 +781,7 @@ static void __cpuinit init_amd(struct cpif (c->x86 ==0x10 )fam10h_check_enable_mmcfg (); -if (amd_apic_timer_broken ()) +if (amd_apic_timer_broken (c)) disable_apic_timer =1 ;if (c == &boot_cpu_data && c->x86 >=0xf && c->x86 <=0x11 ) {
Re: [patch 3/6] x86: use cpuinfo to check for interrupt pending message msr
Quoted 46 lines · Thomas Gleixner
No need to do a cpuid(1) again. The cpuinfo structure has all necessary information already.
Signed-off-by Thomas Gleixner <[email protected]>Cc Andreas Herrmann <[email protected]>--- arch/x86/kernel/cpu/amd.c | 41 +++++++++++++++-------------------------- arch/x86/kernel/setup_64.c | 38 +++++++++++++++----------------------- 2 files changed, 30 insertions(+), 49 deletions(-)Index:
linux-2.6/arch/x86/kernel/cpu/amd.c
===================================================================arch/x86/kernel/cpu/amd.c +1
-22@@ -25,35 +25,24 @@ extern void vide(void); __asm__(".align 4\nvide: ret" );#ifdef CONFIG_X86_LOCAL_APIC -#define CPUID_PROCESSOR_SIGNATURE 1 -#define CPUID_XFAM 0x0ff00000 -#define CPUID_XFAM_K8 0x00000000 -#define CPUID_XFAM_10H 0x00100000 -#define CPUID_XFAM_11H 0x00200000 -#define CPUID_XMOD 0x000f0000 -#define CPUID_XMOD_REV_F 0x00040000/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */ -static __cpuinit int amd_apic_timer_broken (void ) +static __cpuinit int amd_apic_timer_broken (struct cpuinfo_x86 *c) {u32 lo, hi; -u32 eax =cpuid_eax (CPUID_PROCESSOR_SIGNATURE ); -switch (eax &CPUID_XFAM ) { -case CPUID_XFAM_K8 : -if ((eax &CPUID_XMOD ) <CPUID_XMOD_REV_F ) -break ; -case CPUID_XFAM_10H : -case CPUID_XFAM_11H : -rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); -if (lo &K8_INTP_C1E_ACTIVE_MASK ) { -if (smp_processor_id () != boot_cpu_physical_apicid) -printk (KERN_INFO "AMD C1E detected late. " -" Force timer broadcast.\n" ); -return 1 ; - }
arch/x86/kernel/cpu/amd.c extern void vide(void); · line 39
Quoted 10 lines
arch/x86/kernel/cpu/amd.c +7
-3-break ; -default : -/* err on the side of caution */ + +if (c->x86 <0x0F ) +return 0 ; + +/* Family 0x0f models < rev F do not have this MSR */ +if (c->x86 ==0x0f && c->x86_model <0x40 ) +return 0 ;
Just some minor nitpicking.
Older AMD family 0xf CPUs have this Interrupt Pending Message Register. But they do not support C1E and thus bits 27 and 28 of this MSR are reserved.
Andreas
Re: [patch 3/6] x86: use cpuinfo to check for interrupt pending message msr
Quoted 8 lines · Andreas Herrmann
arch/x86/kernel/cpu/amd.c extern void vide(void); · line 39
Quoted 3 lines
arch/x86/kernel/cpu/amd.c +3
-0+/* Family 0x0f models < rev F do not have this MSR */ +if (c->x86 ==0x0f && c->x86_model <0x40 ) +return 0 ;Just some minor nitpicking.
Older AMD family0xfCPUs have this Interrupt Pending Message Register. But they do not support C1E and thus bits 27 and 28 of this MSR are reserved.
So the check can be simplified to always check the MSR for all family >= 0x0f CPUs ?
Thanks,
tglx
Re: [patch 3/6] x86: use cpuinfo to check for interrupt pending message msr
Quoted 10 lines · Thomas Gleixner
Quoted 8 lines · Andreas Herrmann
arch/x86/kernel/cpu/amd.c extern void vide(void); · line 39
Quoted 3 lines
arch/x86/kernel/cpu/amd.c +3
-0+/* Family 0x0f models < rev F do not have this MSR */ +if (c->x86 ==0x0f && c->x86_model <0x40 ) +return 0 ;Just some minor nitpicking.
Older AMD family0xfCPUs have this Interrupt Pending Message Register. But they do not support C1E and thus bits 27 and 28 of this MSR are reserved.
Quoted 2 lines
So the check can be simplified to always check the MSR for all family >=
0x0fCPUs ?
First of all I thought of changing the comment.
But now that you ask:
Documentation for older K8 CPUs says that reserved bits in that MSR are "Read as Zero". But otherwise it also says "Software must not depend on the state of a reserved field ..."
Maybe I am a little paranoid but I would keep the model check.
Andreas
PATCH 4/6 x86: use cpuid to check MWAIT support for C1
In reply to Thomas Gleixnercpuid(0x05) provides extended information about MWAIT in EDX when bit 0 of ECX is set. Bit 4-7 of EDX determine whether MWAIT is supported for C1. C1E enabled CPUs have these bits set to 0.
Based on an ealier patch from Andi Kleen.
--- arch/x86/kernel/process.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-)
arch/x86/kernel/process.c +20 -8
@@ -122,19 +122,31 @@ static void poll_idle(void)* * idle=mwait overrides this decision and forces the usage of mwait. */ + +#define MWAIT_INFO 0x05 +#define MWAIT_ECX_EXTENDED_INFO 0x01 +#define MWAIT_EDX_C1 0xf0 +static int __cpuinit mwait_usable (const struct cpuinfo_x86 *c) { +u32 eax, ebx, ecx, edx; +if (force_mwait)return 1 ; -if (c->x86_vendor ==X86_VENDOR_AMD ) { -switch (c->x86) { -case 0x10 : -case 0x11 : -return 0 ; - } - } -return 1 ; +if (c->cpuid_level <MWAIT_INFO ) +return 0 ; + +cpuid (MWAIT_INFO , &eax, &ebx, &ecx, &edx); +/* Check, whether EDX has extended info about MWAIT */ +if (!(ecx &MWAIT_ECX_EXTENDED_INFO )) +return 1 ; + +/* +* edx enumerates MONITOR/MWAIT extensions. Check, whether C1 +* supports MWAIT +*/ +return edx &MWAIT_EDX_C1 ; }void __cpuinit select_idle_routine (const struct cpuinfo_x86 *c)
PATCH 5/6 x86: move more common idle functions/variables to process.c
In reply to Thomas Gleixner--- arch/x86/kernel/process.c | 70 +++++++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/process_32.c | 54 --------------------------------- arch/x86/kernel/process_64.c | 28 ----------------- 3 files changed, 70 insertions(+), 82 deletions(-)
arch/x86/kernel/process.c +70 -0
@@ -45,6 +45,76 @@ void arch_task_cache_init(void) SLAB_PANIC, NULL); } +/* +* Idle related variables and functions +*/ +unsigned long boot_option_idle_override =0 ; +EXPORT_SYMBOL (boot_option_idle_override); + +/* +* Powermanagement idle function, if any.. +*/ +void (*pm_idle)(void ); +EXPORT_SYMBOL (pm_idle); + +#ifdef CONFIG_X86_32 +/* +* This halt magic was a workaround for ancient floppy DMA +* wreckage. It should be safe to remove. +*/ +static int hlt_counter; +void disable_hlt (void ) +{ + hlt_counter++; +} +EXPORT_SYMBOL (disable_hlt); + +void enable_hlt (void ) +{ + hlt_counter--; +} +EXPORT_SYMBOL (enable_hlt); + +static inline int hlt_use_halt (void ) +{ +return !hlt_counter && boot_cpu_data.hlt_works_ok; +} +#else +static inline int hlt_use_halt (void ) +{ +return 1 ; +} +#endif + +/* +* We use this if we don't have any better +* idle routine.. +*/ +void default_idle (void ) +{ +if (hlt_use_halt ()) { +current_thread_info ()->status &= ~TS_POLLING ; +/* +* TS_POLLING-cleared state must be visible before we +* test NEED_RESCHED: +*/ +smp_mb (); + +if (!need_resched ()) +safe_halt ();/* enables interrupts racelessly */ +else +local_irq_enable (); +current_thread_info ()->status |=TS_POLLING ; + }else { +local_irq_enable (); +/* loop is done by the caller */ +cpu_relax (); + } +} +#ifdef CONFIG_APM_MODULE +EXPORT_SYMBOL (default_idle); +#endif +static void do_nothing (void *unused) { }
arch/x86/kernel/process_32.c +0 -54
66 lines not on this page · raw message
arch/x86/kernel/process_64.c +0 -28
@@ -56,15 +56,6 @@ asmlinkage extern void ret_from_fork(voiunsigned long kernel_thread_flags =CLONE_VM |CLONE_UNTRACED ; -unsigned long boot_option_idle_override =0 ; -EXPORT_SYMBOL (boot_option_idle_override); - -/* -* Powermanagement idle function, if any.. -*/ -void (*pm_idle)(void ); -EXPORT_SYMBOL (pm_idle); -static ATOMIC_NOTIFIER_HEAD (idle_notifier);void idle_notifier_register (struct notifier_block *n)
@@ -94,25 +85,6 @@ void exit_idle(void)__exit_idle (); } -/* -* We use this if we don't have any better -* idle routine.. -*/ -void default_idle (void ) -{ -current_thread_info ()->status &= ~TS_POLLING ; -/* -* TS_POLLING-cleared state must be visible before we -* test NEED_RESCHED: -*/ -smp_mb (); -if (!need_resched ()) -safe_halt ();/* enables interrupts racelessly */ -else -local_irq_enable (); -current_thread_info ()->status |=TS_POLLING ; -} -#ifdef CONFIG_HOTPLUG_CPU DECLARE_PER_CPU (int, cpu_state);
PATCH 6/6 x86: add c1e aware idle function
In reply to Thomas GleixnerC1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.
To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting for C1E and covers the late enablement of C1E as well.
--- arch/x86/kernel/apic_32.c | 5 +-- arch/x86/kernel/apic_64.c | 26 ---------------- arch/x86/kernel/cpu/amd.c | 30 ------------------- arch/x86/kernel/process.c | 66 +++++++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/setup_64.c | 25 ---------------- include/asm-x86/apic.h | 3 - kernel/time/tick-broadcast.c | 6 ++- 7 files changed, 73 insertions(+), 88 deletions(-)
arch/x86/kernel/apic_32.c +2 -3
@@ -64,9 +64,8 @@ static int enable_local_apic __initdata;/* Local APIC timer verification ok */ static int local_apic_timer_verify_ok; -/* Disable local APIC timer from the kernel commandline or via dmi quirk -or using CPU MSR check */ -int local_apic_timer_disabled; +/* Disable local APIC timer from the kernel commandline or via dmi quirk */ +static int local_apic_timer_disabled;/* Local APIC timer works in C2 */ int local_apic_timer_c2_ok;EXPORT_SYMBOL_GPL (local_apic_timer_c2_ok);
arch/x86/kernel/apic_64.c +1 -25
@@ -43,7 +43,7 @@#include <mach_ipi.h> #include <mach_apic.h> -int disable_apic_timer__cpuinitdata ; +static int disable_apic_timer__cpuinitdata ;static int apic_calibrate_pmtmr__initdata ;int disable_apic;
@@ -422,32 +422,8 @@ void __init setup_boot_APIC_clock(void)setup_APIC_timer (); } -/* -* AMD C1E enabled CPUs have a real nasty problem: Some BIOSes set the -* C1E flag only in the secondary CPU, so when we detect the wreckage -* we already have enabled the boot CPU local apic timer. Check, if -* disable_apic_timer is set and the DUMMY flag is cleared. If yes, -* set the DUMMY flag again and force the broadcast mode in the -* clockevents layer. -*/ -static void __cpuinit check_boot_apic_timer_broadcast (void ) -{ -if (!disable_apic_timer || - (lapic_clockevent.features &CLOCK_EVT_FEAT_DUMMY )) -return ; - -printk (KERN_INFO "AMD C1E detected late. Force timer broadcast.\n" ); - lapic_clockevent.features |=CLOCK_EVT_FEAT_DUMMY ; - -local_irq_enable (); -clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_FORCE , - &boot_cpu_physical_apicid); -local_irq_disable (); -} -void __cpuinit setup_secondary_APIC_clock (void ) { -check_boot_apic_timer_broadcast ();setup_APIC_timer (); }
arch/x86/kernel/cpu/amd.c +0 -30
@@ -24,31 +24,6 @@extern void vide (void ); __asm__(".align 4\nvide: ret" ); -#ifdef CONFIG_X86_LOCAL_APIC - -/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */ -static __cpuinit int amd_apic_timer_broken (struct cpuinfo_x86 *c) -{ -u32 lo, hi; - -if (c->x86 <0x0F ) -return 0 ; - -/* Family 0x0f models < rev F do not have this MSR */ -if (c->x86 ==0x0f && c->x86_model <0x40 ) -return 0 ; - -rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); -if (lo &K8_INTP_C1E_ACTIVE_MASK ) { -if (smp_processor_id () != boot_cpu_physical_apicid) -printk (KERN_INFO "AMD C1E detected late. " -"Force timer broadcast.\n" ); -return 1 ; - } -return 0 ; -} -#endif -int force_mwait__cpuinitdata ;static void __cpuinit early_init_amd (struct cpuinfo_x86 *c)
@@ -285,11 +260,6 @@ static void __cpuinit init_amd(struct cp num_cache_leaves =3 ; } -#ifdef CONFIG_X86_LOCAL_APIC -if (amd_apic_timer_broken (c)) - local_apic_timer_disabled =1 ; -#endif -/* K6s reports MCEs but don't actually have all the MSRs */ if (c->x86 <6 )clear_cpu_cap (c,X86_FEATURE_MCE );
arch/x86/kernel/process.c +66 -0
84 lines not on this page · raw message
arch/x86/kernel/setup_64.c +0 -25
@@ -682,28 +682,6 @@ static void __cpuinit early_init_amd_mc(#endif } -/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */ -static __cpuinit int amd_apic_timer_broken (struct cpuinfo_x86 *c) -{ -u32 lo, hi; - -if (c->x86 <0x0F ) -return 0 ; - -/* Family 0x0f models < rev F do not have this MSR */ -if (c->x86 ==0x0f && c->x86_model <0x40 ) -return 0 ; - -rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); -if (lo &K8_INTP_C1E_ACTIVE_MASK ) { -if (smp_processor_id () != boot_cpu_physical_apicid) -printk (KERN_INFO "AMD C1E detected late. " -"Force timer broadcast.\n" ); -return 1 ; - } -return 0 ; -} -static void __cpuinit early_init_amd (struct cpuinfo_x86 *c) {early_init_amd_mc (c);
@@ -781,9 +759,6 @@ static void __cpuinit init_amd(struct cpif (c->x86 ==0x10 )fam10h_check_enable_mmcfg (); -if (amd_apic_timer_broken (c)) - disable_apic_timer =1 ; -if (c == &boot_cpu_data && c->x86 >=0xf && c->x86 <=0x11 ) {unsigned long long tseg;
include/asm-x86/apic.h +0 -3
@@ -38,12 +38,9 @@ extern void generic_apic_probe(void);extern int apic_verbosity;extern int timer_over_8254;extern int local_apic_timer_c2_ok; -extern int local_apic_timer_disabled; -extern int apic_runs_main_timer;extern int ioapic_force;extern int disable_apic; -extern int disable_apic_timer;/* * Basic functions accessing APICs.
kernel/time/tick-broadcast.c +4 -2
@@ -30,6 +30,7 @@struct tick_device tick_broadcast_device;static cpumask_t tick_broadcast_mask;static DEFINE_SPINLOCK (tick_broadcast_lock); +static int tick_broadcast_force;#ifdef CONFIG_TICK_ONESHOT static void tick_broadcast_clear_oneshot (int cpu);
@@ -232,10 +233,11 @@ static void tick_do_broadcast_on_off(voi CLOCK_EVT_MODE_SHUTDOWN); }if (*reason ==CLOCK_EVT_NOTIFY_BROADCAST_FORCE ) - dev->features |=CLOCK_EVT_FEAT_DUMMY ; + tick_broadcast_force =1 ;break ;case CLOCK_EVT_NOTIFY_BROADCAST_OFF : -if (cpu_isset (cpu, tick_broadcast_mask)) { +if (!tick_broadcast_force && +cpu_isset (cpu, tick_broadcast_mask)) {cpu_clear (cpu, tick_broadcast_mask);if (td->mode ==TICKDEV_MODE_PERIODIC )tick_setup_periodic (dev,0 );
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 25 lines · On Thu, 12 Jun 2008 10:29:00 -0000 Thomas Gleixner <[email protected]> wrote:
arch/x86/kernel/process.c +25
-0+static void c1e_idle (void ) +{ +static cpumask_t c1e_mask =CPU_MASK_NONE ; +static int c1e_detected; + +if (need_resched ()) +return ; + +if (!c1e_detected) { +u32 lo, hi; + +rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi); +if (lo &K8_INTP_C1E_ACTIVE_MASK ) { + c1e_detected =1 ; +mark_tsc_unstable ("TSC halt in C1E" ); +printk (KERN_INFO "System has C1E enabled\n" ); + } + } + +if (c1e_detected) { +int cpu =smp_processor_id (); + +if (!cpu_isset (cpu, c1e_mask)) { +cpu_set (cpu, c1e_mask); +/* Force broadcast so ACPI can not interfere */
arch/x86/kernel/process.c static int __cpuinit mwait_usable(const · line 280
Quoted 10 lines
arch/x86/kernel/process.c +10
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_FORCE , + &cpu); +printk (KERN_INFO "Switch to broadcast mode on CPU%d\n" , + cpu); + } +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu); +default_idle (); +local_irq_disable (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu); +local_irq_enable ();
I worked it out! It took me a while.
This function is called with local irqs disabled.
default_idle() is entered with local irqs disabled but returns with them enabled.
clockevents_notify() is supposed to be called with local irqs disabled.
This functions returns with local irqs enabled.
Was I right? None of any of that is documented anywhere. But it should be.
Quoted 3 lines
arch/x86/kernel/process.c +3
-0+ }else +default_idle (); +}
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 10 lines · Andrew Morton
arch/x86/kernel/process.c static int __cpuinit mwait_usable(const · line 280
Quoted 5 lines · On Thu, 12 Jun 2008 10:29:00 -0000 Thomas Gleixner <[email protected]> wrote:
arch/x86/kernel/process.c +5
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu); +default_idle (); +local_irq_disable (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu); +local_irq_enable ();I worked it out! It took me a while.
Quoted 8 lines
This function is called with local irqs disabled.
default_idle()is entered with local irqs disabled but returns with them enabled.
clockevents_notify()is supposed to be called with local irqs disabled.This functions returns with local irqs enabled.
Damn, you decoded my sekrit.
Quoted 2 lines
Was I right? None of any of that is documented anywhere. But it should be.
Yeah, needs a big comment. Will add one.
Thanks,
tglx
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 13 lines · On Fri, 13 Jun 2008 08:02:30 +0200 (CEST) Thomas Gleixner <[email protected]>…
Quoted 12 lines · Andrew Morton
arch/x86/kernel/process.c static int __cpuinit mwait_usable(const · line 280
Quoted 5 lines · On Thu, 12 Jun 2008 10:29:00 -0000 Thomas Gleixner <[email protected]> wrote:
arch/x86/kernel/process.c +5
-0+clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_ENTER , &cpu); +default_idle (); +local_irq_disable (); +clockevents_notify (CLOCK_EVT_NOTIFY_BROADCAST_EXIT , &cpu); +local_irq_enable ();I worked it out! It took me a while.
This function is called with local irqs disabled.
Quoted 8 lines
Quoted 6 lines
default_idle()is entered with local irqs disabled but returns with them enabled.
clockevents_notify()is supposed to be called with local irqs disabled.This functions returns with local irqs enabled.
Damn, you decoded my sekrit.
The wonders of the GPL.
Quoted 5 lines
Quoted 2 lines
Was I right? None of any of that is documented anywhere. But it should be.
Yeah, needs a big comment. Will add one.
Actually it need lots of little comments. One at default_idle(), one at clockevents_notify(), one at whatever-this-function-was.
Because "must be called with local irqs disabled" is as much a part of a function's interface as "must be passed a foo* and an integer and returns a bar*". Ditto "must be called under zot_lock". But people often forget these things.
I guess kerneldoc should have a standard "call environment" template. Maybe it does, dunno.
Of course, WARN_ON(!irqs_disabled()) and WARN_ON(!spin_is_locked(foo)) is robust documentation.
Re: [patch 6/6] x86: add c1e aware idle function
In reply to Thomas GleixnerQuoted 6 lines · Thomas Gleixner
C1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
Quoted 7 lines
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting
Entering idle is quite a common operation, and reading MSR is quite slow. Is it possible to do better here?
What happens if ACPI BIOS toggles MSR on all cpus *while* we are entering idle? This seems inherently racy...
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 15 lines · Pavel Machek
Quoted 6 lines · Thomas Gleixner
C1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
Quoted 7 lines
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting
Quoted 5 lines
Entering idle is quite a common operation, and reading MSR is quite slow. Is it possible to do better here?
What happens if ACPI BIOS toggles MSR on all cpus *while* we are entering idle? This seems inherently racy...
Yes, and that fits the picture I'm observing on the nx6325 (see the "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
Thanks,
Rafael
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 16 lines · Rafael J. Wysocki
Quoted 15 lines · Pavel Machek
Quoted 13 lines · Thomas Gleixner
C1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting
Quoted 5 lines
Quoted 5 lines
Entering idle is quite a common operation, and reading MSR is quite slow. Is it possible to do better here?
What happens if ACPI BIOS toggles MSR on all cpus *while* we are entering idle? This seems inherently racy...
Well, on most of the machines we see the C1E bit on the first idle entry either on CPU0 or on CPU1.
I know of exactly one machine which has the C1E thing enabled late when the ACPI stuff runs.
Quoted 2 lines
Yes, and that fits the picture I'm observing on the nx6325 (see the "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
Hmm, when is the C1E detected on current mainline ? Does the boot CPU have it or is it when the second CPU comes up ?
Thanks,
tglx
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 26 lines · Thomas Gleixner
Quoted 16 lines · Rafael J. Wysocki
Quoted 15 lines · Pavel Machek
Quoted 13 lines · Thomas Gleixner
C1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting
Quoted 5 lines
Quoted 5 lines
Entering idle is quite a common operation, and reading MSR is quite slow. Is it possible to do better here?
What happens if ACPI BIOS toggles MSR on all cpus *while* we are entering idle? This seems inherently racy...
Well, on most of the machines we see the C1E bit on the first idle entry either on CPU0 or on CPU1.
Quoted 8 lines
I know of exactly one machine which has the C1E thing enabled late when the ACPI stuff runs.
Quoted 2 lines
Yes, and that fits the picture I'm observing on the nx6325 (see the "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
Hmm, when is the C1E detected on current mainline ? Does the boot CPU have it or is it when the second CPU comes up ?
How can I check that?
Rafael
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 30 lines · Rafael J. Wysocki
Quoted 29 lines · Thomas Gleixner
Quoted 16 lines · Rafael J. Wysocki
Quoted 15 lines · Pavel Machek
Quoted 13 lines · Thomas Gleixner
C1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting
Quoted 5 lines
Quoted 5 lines
Entering idle is quite a common operation, and reading MSR is quite slow. Is it possible to do better here?
What happens if ACPI BIOS toggles MSR on all cpus *while* we are entering idle? This seems inherently racy...
Well, on most of the machines we see the C1E bit on the first idle entry either on CPU0 or on CPU1.
I know of exactly one machine which has the C1E thing enabled late when the ACPI stuff runs.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Yes, and that fits the picture I'm observing on the nx6325 (see the "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
Hmm, when is the C1E detected on current mainline ? Does the boot CPU have it or is it when the second CPU comes up ?
How can I check that?
When the boot cpu has it then it prints:
"Disabling APIC timer"
when the secondary CPU has it then it prints:
"AMD C1E detected late. Force timer broadcast."
Current mainline does not check for the late ACPI case so I'm pretty sure that on your box it is detected during early boot way before we can switch to highres/dyntick.
Thanks,
tglx
Re: [patch 6/6] x86: add c1e aware idle function
Quoted 34 lines · Thomas Gleixner
Quoted 33 lines · Rafael J. Wysocki
Quoted 32 lines · Thomas Gleixner
Quoted 16 lines · Rafael J. Wysocki
Quoted 15 lines · Pavel Machek
Quoted 13 lines · Thomas Gleixner
C1E on AMD machines is like C3 but without control from the OS. Up to now we disabled the local apic timer for those machines as it stops when the CPU goes into C1E. This excludes those machines from high resolution timers / dynamic ticks, which hurts especially the X2 based laptops.
The current boot time C1E detection has another more serious flaw:
some BIOSes do not enable C1E until the ACPI processor module is loaded. This causes systems to stop working after that point.To work nicely with C1E enabled machines we use a separate idle function, which checks on idle entry whether C1E was enabled in the Interrupt Pending Message MSR. This allows us to do timer broadcasting
Quoted 5 lines
Quoted 5 lines
Entering idle is quite a common operation, and reading MSR is quite slow. Is it possible to do better here?
What happens if ACPI BIOS toggles MSR on all cpus *while* we are entering idle? This seems inherently racy...
Well, on most of the machines we see the C1E bit on the first idle entry either on CPU0 or on CPU1.
I know of exactly one machine which has the C1E thing enabled late when the ACPI stuff runs.
Quoted 2 lines
Yes, and that fits the picture I'm observing on the nx6325 (see the "linux-next: Tree for June 13: IO APIC breakage on HP nx6325" thread).
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Hmm, when is the C1E detected on current mainline ? Does the boot CPU have it or is it when the second CPU comes up ?
How can I check that?
When the boot cpu has it then it prints:
"Disabling APIC timer"
It does that.
Quoted 7 lines
when the secondary CPU has it then it prints:
"AMD C1E detected late. Force timer broadcast."
Current mainline does not check for the late ACPI case so I'm pretty sure that on your box it is detected during early boot way before we can switch to highres/dyntick.
Yes, that's the case.
Thanks,
Rafael
Quoted 43 lines · Thomas Gleixner
AMD CPUs with C1E support are currently excluded from high resolution timers and NOHZ support. The reason is that C1E is a BIOS controlled C3 power state which switches off TSC and the local APIC timer. The ACPI C-State control manages the TSC/local APIC timer wreckage, but this does not include the C1 based ("halt" instruction) C1E mode. The BIOS/SMM controlled C1E state works on most systems even without enabling ACPI C-State control.
The fact that a system has C1E support enabled is advertised in a MSR, but the time during boot when the C1E bit is set by the BIOS varies:
1) Boot CPU has already C1E bit set
2) Secondary CPU sets C1E bit
3) C1E bit is set after ACPI C-State queryCase #1 and #2 are covered by the current implementation, but case #3 results in a complete system lockup due to missing timer interrupts. The current solution is to disable the local APIC timer and use the PIT in broadcast mode. This restricts the C1E enabled systems to periodic timer mode.
The following patch series implements a C1E aware idle function which also covers the late C1E enablement (case #3):
The function is selected during boot for CPUs which have possibly C1E support. The function checks the MSR which contains the C1E active bits before executing the halt instruction. When one of the C1E active bits is set, it makes the system C1E aware by enabling the timer broadcast mechanism for all CPUs. For high resolution timer and/or nohz enabled systems it calls the oneshot timer broadcast mechanism before executing the halt instruction. This is the same mechanism which is used in the ACPI C-State control for C2/C3 power states.
On my C1E affected X2 box these patches reduce the wakeups/sec down to 20 according to powertop.
The patches work fine on systems which are not affected by the dreaded ATI chipset timer wreckage. On those which have the problem, the box needs help from the keyboard to continue working.
The x86 changes for .27 contain a complete overhaul of the affected code, but this is out of scope for this patchset.
Quoted 7 lines
For those who are interested to test those patches on top of 2.6.26-rc I extracted a patch and added it to the c1e series. It's available from:
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
or
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2
I guess the x86-next-apic-timer-fixes.patch is a combination of all patches in this series. Is that correct?
Thanks,
Rafael
Quoted 6 lines · Rafael J. Wysocki
Quoted 3 lines
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
or
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2I guess the x86-next-apic-timer-fixes.patch is a combination of all patches in this series. Is that correct?
you can also try out those patches via running tip/master. (the patches from Thomas are queued up in the tip/x86/cpu topic branch)
Ingo
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Rafael J. Wysocki
Quoted 10 lines · Ingo Molnar
Quoted 6 lines · Rafael J. Wysocki
Quoted 3 lines
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
or
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2I guess the x86-next-apic-timer-fixes.patch is a combination of all patches in this series. Is that correct?
Quoted 7 lines
Quoted 2 lines
you can also try out those patches via running tip/master. (the patches from Thomas are queued up in the tip/x86/cpu topic branch)
I tested this one eventually. :-)
It seems to work quite well on the nx6325 that caused us so many headaches in the past, even suspend to RAM doesn't break which is a good sign.
Well, unfortunately, an attempt to suspend to RAM after a successful hibernation-restore cycle finally hanged the machine. :-(
Thanks,
Rafael
Quoted 11 lines · Rafael J. Wysocki
Quoted 7 lines · Thomas Gleixner
For those who are interested to test those patches on top of 2.6.26-rc I extracted a patch and added it to the c1e series. It's available from:
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.gz
or
http://www.kernel.org/pub/linux/kernel/people/tglx/c1e/2.6.26-rc5-c1e-patches.tar.bz2I guess the x86-next-apic-timer-fixes.patch is a combination of all patches in this series. Is that correct?
Yes. It's the combi patch of everything in the x86/apic branch of linux-2.6-tip.git.
Thanks,
tglx
Quoted 4 lines · Thomas Gleixner
...
The following patch series implements a C1E aware idle function which also covers the late C1E enablement (case #3):
...
Great work!
Quoted 1 line
From a first glance and test it looks really good.
Just about 10 wakeups from idle on a C1E enabled box here.
Will do some deeper review and more tests later.
Quoted 3 lines
The patches work fine on systems which are not affected by the dreaded ATI chipset timer wreckage. On those which have the problem, the box needs help from the keyboard to continue working.
I am currently not aware of that timer wreckage problem. Are you referring to one of the HPET related bugzillas at Kernel Bug Tracker?
If not, do you have a pointer to more information regarding this? (Say, problem description and lspci -vxxx output for such a system.)
Thanks,
Andreas
Quoted 7 lines · Andreas Herrmann
Quoted 4 lines · Thomas Gleixner
...
The following patch series implements a C1E aware idle function which also covers the late C1E enablement (case #3):
...Great work!
Thanks.
Quoted 5 lines
Quoted 1 line
From a first glance and test it looks really good.
Just about 10 wakeups from idle on a C1E enabled box here.
Will do some deeper review and more tests later.
Quoted 7 lines
Quoted 3 lines
The patches work fine on systems which are not affected by the dreaded ATI chipset timer wreckage. On those which have the problem, the box needs help from the keyboard to continue working.
I am currently not aware of that timer wreckage problem. Are you referring to one of the HPET related bugzillas at Kernel Bug Tracker?
There might be related ones. Need to look.
Quoted 2 lines
If not, do you have a pointer to more information regarding this? (Say, problem description and lspci -vxxx output for such a system.)
Usually those systems have the:
..MP-BIOS bug: 8254 timer not connected to IO-APIC
and the fixup machinery in current mainline is slightly insufficient.
The result on such a system is that with current mainline plus the C1E idle changes (just do not apply the apic patch from the tarball), those systems keep missing timer interrupts. I have not yet decoded the root cause, but the patch which is finally fixing that is:
lspci output below.
Thanks,
tglx
---
00:00.0 Host bridge: ATI Technologies Inc RS690 Host Bridge
Subsystem: Fujitsu Siemens Computers Unknown device 111d
Flags: bus master, 66MHz, medium devsel, latency 64 Memory at <ignored> (64-bit, non-prefetchable)
00: 02 10 10 79 06 00 20 22 00 00 00 06 00 40 00 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 e0 20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 1d 11 30: 00 00 00 00 c4 00 00 00 00 00 00 00 00 00 00 00 40: 00 00 00 00 00 00 00 00 00 00 00 00 42 20 05 00 50: 34 17 1d 11 00 00 00 00 00 00 00 00 00 00 00 00 60: 80 00 00 00 87 04 00 00 00 02 20 00 d8 d0 01 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 40 44 00 00 95 00 00 03 20 11 10 00 02 27 00 10 90: 00 00 00 38 40 c4 40 e3 00 c0 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 07 01 00 00 49 01 10 07
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 08 00 80 01 60 20 11 11 d0 00 00 00
d0: 25 05 65 00 02 00 00 00 00 00 00 00 00 00 00 00
e0: 10 00 00 00 01 30 03 80 7f 00 00 00 00 00 00 00
f0: 00 00 00 00 00 80 80 00 00 00 00 00 00 00 00 00
00:01.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (Internal gfx) (prog-if 00 [Normal decode])
Flags: bus master, 66MHz, medium devsel, latency 64 Bus: primary=00, secondary=01, subordinate=01, sec-latency=64 I/O behind bridge: 00009000-00009fff Memory behind bridge: f8200000-f83fffff Prefetchable memory behind bridge: 00000000f0000000-00000000f7ffffff Capabilities: [44] HyperTransport: MSI Mapping Enable+ Fixed+ Capabilities: [b0] Subsystem: ATI Technologies Inc RS690 PCI to PCI Bridge (Internal gfx) Kernel modules: shpchp
00: 02 10 12 79 07 00 30 02 00 00 04 06 00 40 01 00 10: 00 00 00 00 00 00 00 00 00 01 01 40 91 91 20 02 20: 20 f8 30 f8 01 f0 f1 f7 00 00 00 00 00 00 00 00 30: 00 00 00 00 44 00 00 00 00 00 00 00 ff 00 0c 00 40: 00 00 00 00 08 b0 03 a8 00 00 00 00 02 10 12 79 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d 00 00 00 02 10 12 79 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:06.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Port 2) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=02, subordinate=07, sec-latency=0
I/O behind bridge: 0000a000-0000afff
Memory behind bridge: f8400000-f84fffff
Prefetchable memory behind bridge: 00000000f8000000-00000000f80fffff
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Root Port (Slot+), MSI 00
Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Queue=0/0 Enable+
Capabilities: [b0] Subsystem: Fujitsu Siemens Computers Unknown device 111d
Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
Capabilities: [100] Virtual Channel <?>
Kernel driver in use: pcieport-driver
Kernel modules: shpchp
00: 02 10 16 79 07 04 10 00 00 00 04 06 08 00 01 00
10: 00 00 00 00 00 00 00 00 00 02 07 00 a1 a1 00 00
20: 40 f8 40 f8 01 f8 01 f8 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 04 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 58 03 c8 00 00 00 00 10 80 41 01 20 80 00 00
60: 1f 08 00 00 11 0c 10 03 42 00 11 30 e0 0c 30 00
70: c0 03 48 00 00 00 01 00 00 00 00 00 00 00 00 00
80: 05 b0 01 00 0c 30 e0 fe 49 41 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d b8 00 00 34 17 1d 11 08 00 03 a8 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:07.0 PCI bridge: ATI Technologies Inc RS690 PCI to PCI Bridge (PCI Express Port 3) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=08, subordinate=0d, sec-latency=0
I/O behind bridge: 0000b000-0000bfff
Memory behind bridge: f8500000-f85fffff
Prefetchable memory behind bridge: 00000000f8100000-00000000f81fffff
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Root Port (Slot+), MSI 00
Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Queue=0/0 Enable+
Capabilities: [b0] Subsystem: Fujitsu Siemens Computers Unknown device 111d
Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
Capabilities: [100] Virtual Channel <?>
Kernel driver in use: pcieport-driver
Kernel modules: shpchp
00: 02 10 17 79 07 04 10 00 00 00 04 06 08 00 01 00
10: 00 00 00 00 00 00 00 00 00 08 0d 00 b1 b1 00 00
20: 50 f8 50 f8 11 f8 11 f8 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 04 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 58 03 c8 00 00 00 00 10 80 41 01 20 80 00 00
60: 1f 08 00 00 11 0c 10 04 42 00 11 30 80 0c 30 00
70: c0 03 48 00 00 00 01 00 00 00 00 00 00 00 00 00
80: 05 b0 01 00 0c 30 e0 fe 51 41 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d b8 00 00 34 17 1d 11 08 00 03 a8 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:12.0 SATA controller: ATI Technologies Inc SB600 Non-Raid-5 SATA (prog-if 01 [AHCI 1.0])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1 Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 22 I/O ports at 8440 [size=8] I/O ports at 8434 [size=4] I/O ports at 8438 [size=8] I/O ports at 8430 [size=4] I/O ports at 8400 [size=16] Memory at f890a000 (32-bit, non-prefetchable) [size=1K] Capabilities: [60] Power Management version 2 Kernel driver in use: ahci Kernel modules: ahci
00: 02 10 80 43 07 00 30 02 00 01 06 01 00 40 00 00 10: 41 84 00 00 35 84 00 00 39 84 00 00 31 84 00 00 20: 01 84 00 00 00 a0 90 f8 00 00 00 00 34 17 d1 10 30: 00 00 00 00 60 00 00 00 00 00 00 00 0b 01 00 00 40: 10 00 83 02 01 00 10 00 01 00 00 00 00 00 00 00 50: 05 00 84 00 00 00 00 00 00 00 00 00 00 00 00 00 60: 01 00 22 00 00 00 00 00 00 00 00 00 00 00 00 00 70: 12 00 10 00 0f 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 06 00 00 2c d6 01 b4 00 d6 01 b4 00 90: d6 01 b4 00 d6 01 b4 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 b8 00 00 00 00 00 00 00 b8 00 00
b0: 00 00 00 00 00 b8 00 00 00 00 00 00 00 b8 00 00
c0: 00 20 00 00 80 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:13.0 USB Controller: ATI Technologies Inc SB600 USB (OHCI0) (prog-if 10 [OHCI])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16 Memory at f8904000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd
Kernel modules: ohci-hcd
00: 02 10 87 43 17 00 a0 02 00 10 03 0c 08 40 80 00
10: 00 40 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 0a 01 00 00
40: 80 1f 00 00 0a 84 b7 18 07 35 00 00 00 00 00 00
50: 00 9c 00 00 00 00 00 00 ff ff ff ff ff 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 ff 00 00 80 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:13.1 USB Controller: ATI Technologies Inc SB600 USB (OHCI1) (prog-if 10 [OHCI])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 17 Memory at f8905000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd
Kernel modules: ohci-hcd
00: 02 10 88 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 50 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 05 02 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:13.2 USB Controller: ATI Technologies Inc SB600 USB (OHCI2) (prog-if 10 [OHCI])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18 Memory at f8906000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd
Kernel modules: ohci-hcd
00: 02 10 89 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 60 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 0b 03 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:13.3 USB Controller: ATI Technologies Inc SB600 USB (OHCI3) (prog-if 10 [OHCI])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 17 Memory at f8907000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd
Kernel modules: ohci-hcd
00: 02 10 8a 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 70 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 05 02 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:13.4 USB Controller: ATI Technologies Inc SB600 USB (OHCI4) (prog-if 10 [OHCI])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18 Memory at f8908000 (32-bit, non-prefetchable) [size=4K] Kernel driver in use: ohci_hcd
Kernel modules: ohci-hcd
00: 02 10 8b 43 17 00 a0 02 00 10 03 0c 08 40 00 00
10: 00 80 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 00 00 00 00 00 00 00 00 0b 03 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:13.5 USB Controller: ATI Technologies Inc SB600 USB Controller (EHCI) (prog-if 20 [EHCI])
Subsystem: ATI Technologies Inc SB600 USB Controller (EHCI)
Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 19
Memory at f890a400 (32-bit, non-prefetchable) [size=256]
Capabilities: [c0] Power Management version 2
Capabilities: [e4] Debug port: BAR=1 offset=00e0
Kernel driver in use: ehci_hcd
Kernel modules: ehci-hcd
00: 02 10 86 43 17 00 b0 02 00 20 03 0c 10 40 00 00
10: 00 a4 90 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 86 43
30: 00 00 00 00 c0 00 00 00 00 00 00 00 0b 04 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 40 00 0e 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 01 00 00 01 00 00 08 80 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 e4 02 7e 00 00 40 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 0a 00 e0 20 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 14)
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: 66MHz, medium devsel
I/O ports at 8410 [size=16]
Capabilities: [b0] HyperTransport: MSI Mapping Enable- Fixed+ Kernel driver in use: piix4_smbus
Kernel modules: i2c-piix4
00: 02 10 85 43 03 00 30 02 14 00 05 0c 00 00 80 00
10: 11 84 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10
30: 00 00 00 00 b0 00 00 00 00 00 00 00 00 00 00 00
40: d4 bb 00 04 00 00 00 00 0f ff 00 00 00 00 00 00
50: f0 0d 00 00 00 00 f0 08 51 80 03 f3 00 00 00 00
60: 01 00 24 00 bf fd 9e 8f 3f 90 00 00 20 00 00 00
70: 00 01 00 00 08 00 c0 fe ff 6f 00 00 00 00 5a 0f
80: 4a 0a 84 7c 00 00 00 00 00 00 00 00 8c 00 00 80
90: 11 84 00 00 ff de f7 00 00 00 00 00 00 00 00 00
a0: 00 00 ff ff ff ff e0 08 14 28 1c 00 16 79 20 18
b0: 08 00 02 a8 00 00 00 00 00 00 00 00 f0 0f 08 1a
c0: 7d db b6 ed 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: d8 0c 00 00 7f 08 00 00 00 00 00 00 02 00 30 01
00:14.1 IDE interface: ATI Technologies Inc SB600 IDE (prog-if 8a [Master SecP PriP])
Subsystem: Fujitsu Siemens Computers Unknown device 10d1 Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 16 I/O ports at 01f0 [size=8] I/O ports at 03f4 [size=1] I/O ports at 0170 [size=8] I/O ports at 0374 [size=1] I/O ports at 8420 [size=16] Kernel driver in use: pata_atiixp Kernel modules: ata_generic, pata_acpi, pata_atiixp
00: 02 10 8c 43 15 00 20 02 00 8a 01 01 00 40 00 00 10: f1 01 00 00 f5 03 00 00 01 00 00 00 01 00 00 00 20: 21 84 00 00 00 00 00 00 00 00 00 00 34 17 d1 10 30: 00 00 00 00 00 00 00 00 00 00 00 00 ff 01 00 00 40: 5d 5d 00 00 ff ff 00 00 00 00 00 00 00 00 00 00 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 60: 00 00 40 00 10 2c 01 07 01 00 00 00 ff 03 00 00 70: 05 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia
Subsystem: Fujitsu Siemens Computers Unknown device 111e
Flags: bus master, slow devsel, latency 64, IRQ 16 Memory at f8900000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2 Kernel driver in use: HDA Intel
Kernel modules: snd-hda-intel
00: 02 10 83 43 06 00 10 04 00 00 03 04 08 40 00 00 10: 04 00 90 f8 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 1e 11 30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 00 00 00 40: 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 50: 01 00 42 c8 00 00 00 00 00 00 00 00 00 00 00 00 60: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:14.3 ISA bridge: ATI Technologies Inc SB600 PCI to LPC Bridge
Subsystem: Fujitsu Siemens Computers Unknown device 10d1
Flags: bus master, 66MHz, medium devsel, latency 0
00: 02 10 8d 43 0f 00 20 02 00 00 01 06 00 00 80 00 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 d1 10 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40: 04 00 00 00 f7 ff 43 ff 03 ff 42 01 00 00 00 00 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 60: 00 00 00 00 00 00 00 06 0e 00 0f 00 f0 ff ff ff 70: 67 45 23 01 00 00 00 00 01 00 00 00 05 00 00 00 80: 08 00 03 a8 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (prog-if 01 [Subtractive decode])
Flags: bus master, 66MHz, medium devsel, latency 64 Bus: primary=00, secondary=0e, subordinate=13, sec-latency=64 Memory behind bridge: f8600000-f86fffff
00: 02 10 84 43 07 00 a0 02 00 01 04 06 00 40 81 00 10: 00 00 00 00 00 00 00 00 00 0e 13 40 f0 00 80 22 20: 60 f8 60 f8 f0 ff 00 00 00 00 00 00 00 00 00 00 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 40: 26 00 3c ff 00 00 00 00 0c 01 3f d1 00 00 00 00 50: 01 00 00 00 08 00 03 a8 00 00 00 00 85 00 ff ff 60: ca 0e 17 00 ba 18 10 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 02 06
e0: 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
Flags: fast devsel
Capabilities: [80] HyperTransport: Host or Secondary Interface
00: 22 10 00 11 00 00 10 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 80 00 00 00 00 00 00 00 00 00 00 00
40: 01 01 01 00 01 01 01 00 01 01 01 00 01 01 01 00
50: 01 01 01 00 01 01 01 00 01 01 01 00 01 01 01 00
60: 00 00 01 00 e4 00 00 00 20 c8 2f 0f 3c 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 08 00 01 21 20 a8 11 11 22 05 35 80 02 00 00 00
90: 78 01 70 01 00 00 ff 00 07 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
Flags: fast devsel
00: 22 10 01 11 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
40: 03 00 00 00 00 00 3f 00 00 00 00 00 01 00 00 00
50: 00 00 00 00 02 00 00 00 00 00 00 00 03 00 00 00
60: 00 00 00 00 04 00 00 00 00 00 00 00 05 00 00 00
70: 00 00 00 00 06 00 00 00 00 00 00 00 07 00 00 00
80: 03 30 f8 00 00 ff ff 00 03 20 f8 00 80 2f f8 00
90: 03 00 f8 00 00 1f f8 00 03 00 f0 00 80 ff f7 00
a0: 03 0a 00 00 00 0b 00 00 03 00 f0 00 00 ff ef 00
b0: 03 00 e0 00 80 ff ef 00 03 00 40 00 00 ff df 00
c0: 00 00 00 00 00 00 00 00 13 10 00 00 00 f0 0f 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 03 00 00 ff 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
Flags: fast devsel
00: 22 10 02 11 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
40: 01 00 00 00 01 01 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: e0 3e 38 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 46 00 00 00 00 00 00 00
80: 01 00 00 00 00 00 00 00 24 c2 aa 5d 20 13 12 00
90: 20 0c 01 00 5a 80 00 a7 39 00 00 80 00 00 00 00
a0: ef 06 00 5d 00 00 00 00 00 00 00 00 00 00 00 00
b0: 7e b6 b5 e7 7e 00 00 00 9b 7e 1a 3f d8 3f 72 78
c0: 00 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 44 e0 dc e1 12 11 91 6f 21 17 c1 06 88 24 80 87
e0: a4 e2 76 81 2b 3a b7 af 60 4b 5a 04 0a 70 c9 ad
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
Flags: fast devsel
Capabilities: [f0] Secure device <?>
Kernel driver in use: k8temp
Kernel modules: k8temp
00: 22 10 03 11 00 00 10 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 f0 00 00 00 00 00 00 00 00 00 00 00
40: ff 3b 04 00 40 00 10 0a 00 00 00 00 00 00 00 00
50: 00 00 00 05 08 00 00 00 00 00 00 00 80 01 00 94
60: 88 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 01 32 51 21 40 30 50 00 2a 00 08 1b 21 00 00
80: 31 37 07 33 13 31 13 31 00 00 00 00 00 00 00 00
90: 00 00 00 00 26 56 00 00 50 01 d2 85 00 00 00 00
a0: 9e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 01 a7 0d 00 00 00 00 09 25 26 26 00
e0: 00 00 00 00 24 0e 4f 00 01 17 00 00 00 00 00 00
f0: 0f 00 10 00 00 00 00 00 00 00 00 00 82 0f 04 00
01:05.0 VGA compatible controller: ATI Technologies Inc RS690M [Radeon X1200 Series] (prog-if 00 [VGA controller])
Subsystem: Fujitsu Siemens Computers Unknown device 111d Flags: bus master, fast devsel, latency 64, IRQ 10 Memory at f0000000 (64-bit, prefetchable) [size=128M] Memory at f8300000 (64-bit, non-prefetchable) [size=64K] I/O ports at 9000 [size=256] Memory at f8200000 (32-bit, non-prefetchable) [size=1M] Capabilities: [50] Power Management version 2 Capabilities: [80] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable-
00: 02 10 1f 79 07 00 10 00 00 00 00 03 08 40 80 00 10: 0c 00 00 f0 00 00 00 00 04 00 30 f8 00 00 00 00 20: 01 90 00 00 00 00 20 f8 00 00 00 00 34 17 1d 11 30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 01 00 00 40: 00 00 00 00 00 00 00 00 00 00 00 00 34 17 1d 11 50: 01 80 02 06 00 00 00 00 00 00 00 00 00 00 00 00 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01:05.2 Audio device: ATI Technologies Inc Radeon X1200 Series Audio Controller
Subsystem: ATI Technologies Inc Radeon X1200 Series Audio Controller
Flags: bus master, fast devsel, latency 64, IRQ 17
Memory at f8310000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 2
Capabilities: [60] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable- Kernel driver in use: HDA Intel
Kernel modules: snd-hda-intel
00: 02 10 19 79 06 00 18 00 00 00 03 04 08 40 00 00
10: 04 00 31 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 19 79
30: 00 00 00 00 50 00 00 00 00 00 00 00 0b 02 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 02 10 19 79
50: 01 60 02 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
Subsystem: Fujitsu Siemens Computers Unknown device 111d
Flags: bus master, fast devsel, latency 0, IRQ 2301
I/O ports at a000 [size=256]
Memory at f8400000 (64-bit, non-prefetchable) [size=4K]
Memory at f8000000 (64-bit, prefetchable) [size=64K]
[virtual] Expansion ROM at f8020000 [disabled] [size=128K]
Capabilities: [40] Power Management version 3
Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
Capabilities: [d0] Vital Product Data <?>
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [140] Virtual Channel <?>
Capabilities: [160] Device Serial Number 81-68-10-ec-00-00-00-00
Kernel driver in use: r8169
Kernel modules: r8169
00: ec 10 68 81 07 04 10 00 02 00 00 02 10 00 00 00
10: 01 a0 00 00 00 00 00 00 04 00 40 f8 00 00 00 00
20: 0c 00 00 f8 00 00 00 00 00 00 00 00 34 17 1d 11
30: 00 00 00 00 40 00 00 00 00 00 00 00 05 01 00 00
40: 01 50 c3 ff 08 00 00 00 00 00 00 00 00 00 00 00
50: 05 70 81 00 0c 10 e0 fe 00 00 00 00 81 41 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 b0 01 02 c1 86 64 00 10 20 19 00 11 3c 07 00
80: 42 00 11 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 11 d0 01 00 04 00 00 00 04 08 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
08:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)
Subsystem: Fujitsu Siemens Computers Unknown device 111d
Flags: bus master, fast devsel, latency 0, IRQ 2300
I/O ports at b000 [size=256]
Memory at f8500000 (64-bit, non-prefetchable) [size=4K]
Memory at f8100000 (64-bit, prefetchable) [size=64K]
[virtual] Expansion ROM at f8120000 [disabled] [size=128K]
Capabilities: [40] Power Management version 3
Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable+
Capabilities: [70] Express Endpoint, MSI 01
Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
Capabilities: [d0] Vital Product Data <?>
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [140] Virtual Channel <?>
Capabilities: [160] Device Serial Number 81-68-10-ec-00-00-00-00
Kernel driver in use: r8169
Kernel modules: r8169
00: ec 10 68 81 07 04 10 00 02 00 00 02 10 00 00 00
10: 01 b0 00 00 00 00 00 00 04 00 50 f8 00 00 00 00
20: 0c 00 10 f8 00 00 00 00 00 00 00 00 34 17 1d 11
30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
40: 01 50 c3 ff 08 00 00 00 00 00 00 00 00 00 00 00
50: 05 70 81 00 0c 30 e0 fe 00 00 00 00 89 41 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 b0 01 02 c1 86 64 00 10 20 19 00 11 3c 07 00
80: 42 00 11 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 11 d0 01 00 04 00 00 00 04 08 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0e:04.0 FireWire (IEEE 1394): Agere Systems FW323 (rev 70) (prog-if 10 [OHCI])
Subsystem: Agere Systems FW323
Flags: bus master, fast Back2Back, medium devsel, latency 96, IRQ 23 Memory at f8600000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Power Management version 2
Kernel driver in use: firewire_ohci
Kernel modules: firewire-ohci
00: c1 11 11 58 16 02 90 02 70 10 00 0c 08 60 00 00
10: 00 00 60 f8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 c1 11 11 58
30: 00 00 00 00 44 00 00 00 00 00 00 00 0a 01 0c 18
40: 00 00 00 00 01 00 02 7e 00 80 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Quoted 10 lines · Thomas Gleixner
@Macej: I bisected your patches and the commit which solves the mysterious hangs is:
x86: I/O APIC: timer through 8259A second-chance (
7e3530cd98a0c6ab38f5898e855a5beffab26561in linux-2.6-tip.git)That's the patch which you worried about possible impacts, but it seems that it actually fixes the stupid timer irq issue finally. I have tested it on various machines which had timer irq problems in the past and they all run smothly. Great work!
Well, I did hope at least some impact would be positive! I am glad this
is the case and thanks for your report as well as appreciation.
Maciej
Quoted 7 lines · Thomas Gleixner
AMD CPUs with C1E support are currently excluded from high resolution timers and NOHZ support. The reason is that C1E is a BIOS controlled C3 power state which switches off TSC and the local APIC timer. The ACPI C-State control manages the TSC/local APIC timer wreckage, but this does not include the C1 based ("halt" instruction) C1E mode. The BIOS/SMM controlled C1E state works on most systems even without enabling ACPI C-State control.
What a mess.
What is the measured power savings that justifies this effort?
While I'm okay with platform specific idle states, I'm not okay with the use of therm C1E here.
C1E has been shipped for many years on Intel processors and it is completely transparent to the OS.
If AMD now has their own C1E and it breaks the OS, please call is something like AMD_C1E to make it totally clear in shared files like process.c that consulting that variable or running that routine on Intel hardware would be a Linux bug.
thanks
-Len
Quoted 10 lines · Len Brown
Quoted 7 lines · Thomas Gleixner
AMD CPUs with C1E support are currently excluded from high resolution timers and NOHZ support. The reason is that C1E is a BIOS controlled C3 power state which switches off TSC and the local APIC timer. The ACPI C-State control manages the TSC/local APIC timer wreckage, but this does not include the C1 based ("halt" instruction) C1E mode. The BIOS/SMM controlled C1E state works on most systems even without enabling ACPI C-State control.
Quoted 3 lines
What a mess.
What is the measured power savings that justifies this effort?
IMHO the power savings are not that important when such a kernel runs on bare metal:
(In the following C1E denotes "AMD-C1E" of course ;-)
- From what I've seen in some basic tests, power usage was not that different between C1E+NOHZ and C1E+250Hz. (I've done tests on a Turion X2 (RevF) Laptop, a desktop with Phenom (RevB3), and some Turion X2 Ultra (family 0x11) prototype.)
- Just the desktop showed 2-4 Watt larger power usage with C1E+250Hz in comparison to C1E+NOHZ.
- The Turion X2 (Ultra) systems showed almost no differences. Just when running with C1E+1000Hz the ACPI power usage estimate reported by powertop was slightly increased (by <1 Watt). (No differences shown on external power meter and removed battery.)
In conclusion, I think the big benefit from that code is not improved power savings for laptop users but that it allows highres timers and dynticks for CPUs with C1E.
E.g. this helps a lot with virtualization -- where dynticks is preferred for the host system.
And if you have lots of guests the number of timer interrupts can significantly be reduced for the entire system when all guests and the host use dynticks.
Some further (random) observations/notes from my tests:
- Using C1E (NOHZ or with periodic timer) was usually better than running w/o C1E on those machines. (Ok, I didn't expect anything else.)
- It did not matter which timer interrupt source was used (PIT or HPET) in NOHZ/one-shot mode.
+ One-shot programming of HPET in comparison to PIT was faster (on average) by a factor of 2.7 with the chipsets that I've used.
+ Of course using PIT in NOHZ mode resulted in sligtly more interrupts as the 16-bit counter overflows at least every 0.055 seconds - and Linux even uses a maximum delta of about 27ms.
But overall no measurable difference in power usage was seen.
- "wakeups from idle" as reported by powertop are somehow inaccurate (from my point of view). They are calculated as "number of interrupts"/"number of online CPUs". Thus powertop shows about 125 wakeups with 250 HZ on any dual core system or about 250 wakeups with 1000Hz on a quad-core system. But I had no time to look further into this.
- Doing battery tests where you fully charge the battery and then deeply discharge it are harmful, it will/might decrease full capacity of your lithium battery and most probably decrease its lifetime. ;-(
Quoted 2 lines
While I'm okay with platform specific idle states, I'm not okay with the use of therm C1E here.
Valid point.
I try to address this with some patch.
Regards,
Andreas
PATCH 1/1 x86: emphasize that c1e aware idle stuff is AMD specific
This is to clarify that AMD C1E has nothing to do with Intel's C1E or C2E.
Furthermore I changed some comment to clarify that currently just AMD family 0x10 CPUs support MWAIT. AMD family 0x11 has quite similar power management features like family 0x10 but it has no MONITOR/MWAIT instructions.
---
Patch is against current tip/master.
(git describe says: tip-history-2008-06-30_08.08_Mon-441-g7782052 whatever that means ;-)
Regards,
Andreas
arch/x86/kernel/process.c | 36 ++++++++++++++++++------------------ 1 files changed, 18 insertions(+), 18 deletions(-)
arch/x86/kernel/process.c +18 -18
@@ -189,12 +189,12 @@ static void poll_idle(void)/* * mwait selection logic: * -* It depends on the CPU. For AMD CPUs that support MWAIT this is -* wrong. Family 0x10 and 0x11 CPUs will enter C1 on HLT. Powersavings -* then depend on a clock divisor and current Pstate of the core. If -* all cores of a processor are in halt state (C1) the processor can -* enter the C1E (C1 enhanced) state. If mwait is used this will never -* happen. +* It depends on the CPU. For AMD CPUs that support MWAIT - currently +* AMD family 0x10 - this is wrong. Those CPUs will enter C1 on +* HLT. Powersavings then depend on a clock divisor and current Pstate +* of the core. If all cores of a processor are in halt state (C1) the +* processor can enter the AMD C1E (C1 enhanced) state. If mwait is +* used this will never happen. * * idle=mwait overrides this decision and forces the usage of mwait. */
@@ -219,16 +219,16 @@ static int __cpuinit mwait_usable(const struct cpuinfo_x86 *c)return 1 ;/* -* edx enumeratios MONITOR/MWAIT extensions. Check, whether -* C1 supports MWAIT +* EDX enumerates MONITOR/MWAIT extensions. Check, whether an +* C1 sub-state is supported using MONITOR/MWAIT. */ return (edx &MWAIT_EDX_C1 ); }/* -* Check for AMD CPUs, which have potentially C1E support +* Check for AMD CPUs, which have potentially AMD's C1E support */ -static int __cpuinit check_c1e_idle (const struct cpuinfo_x86 *c) +static int __cpuinit check_amd_c1e_idle (const struct cpuinfo_x86 *c) {if (c->x86_vendor !=X86_VENDOR_AMD )return 0 ;
@@ -236,7 +236,7 @@ static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c)if (c->x86 <0x0F )return 0 ; -/* Family 0x0f models < rev F do not have C1E */ +/* Family 0x0f models < rev F do not have AMD C1E */ if (c->x86 ==0x0f && c->x86_model <0x40 )return 0 ;
@@ -244,11 +244,11 @@ static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c) }/* -* C1E aware idle routine. We check for C1E active in the interrupt +* AMD C1E aware idle routine. We check for C1E active in the interrupt * pending message MSR. If we detect C1E, then we handle it the same * way as C3 power states (local apic timer and TSC stop) */ -static void c1e_idle (void ) +static void amd_c1e_idle (void ) {static cpumask_t c1e_mask =CPU_MASK_NONE ;static int c1e_detected;
@@ -262,8 +262,8 @@ static void c1e_idle(void)rdmsr (MSR_K8_INT_PENDING_MSG , lo, hi);if (lo &K8_INTP_C1E_ACTIVE_MASK ) { c1e_detected =1 ; -mark_tsc_unstable ("TSC halt in C1E" ); -printk (KERN_INFO "System has C1E enabled\n" ); +mark_tsc_unstable ("TSC halts in AMD C1E" ); +printk (KERN_INFO "System has AMD C1E enabled\n" ); } }
@@ -316,9 +316,9 @@ void __cpuinit select_idle_routine(const struct cpuinfo_x86 *c)*/ printk (KERN_INFO "using mwait in idle threads.\n" ); pm_idle = mwait_idle; - }else if (check_c1e_idle (c)) { -printk (KERN_INFO "using C1E aware idle routine\n" ); - pm_idle = c1e_idle; + }else if (check_amd_c1e_idle (c)) { +printk (KERN_INFO "using AMD C1E aware idle routine\n" ); + pm_idle = amd_c1e_idle; }else pm_idle = default_idle; }
Signature · 1 line
1.5.6.1
Hi!
Quoted 9 lines
Quoted 9 lines
Quoted 7 lines
AMD CPUs with C1E support are currently excluded from high resolution timers and NOHZ support. The reason is that C1E is a BIOS controlled C3 power state which switches off TSC and the local APIC timer. The ACPI C-State control manages the TSC/local APIC timer wreckage, but this does not include the C1 based ("halt" instruction) C1E mode. The BIOS/SMM controlled C1E state works on most systems even without enabling ACPI C-State control.
What a mess.
Yep, seems like AMD is breaking C1 semantics. Is it even valid from ACPI spec point of view?
Quoted 4 lines
Quoted 1 line
What is the measured power savings that justifies this effort?
IMHO the power savings are not that important when such a kernel runs on bare metal:
Ok, so maybe we should disable C1E to work around its misdesign?
It would be certainly nice to have noc1e command line option...
Quoted 1 line
But overall no measurable difference in power usage was seen.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Quoted 13 lines · Pavel Machek
Quoted 9 lines
Quoted 9 lines
Quoted 7 lines
AMD CPUs with C1E support are currently excluded from high resolution timers and NOHZ support. The reason is that C1E is a BIOS controlled C3 power state which switches off TSC and the local APIC timer. The ACPI C-State control manages the TSC/local APIC timer wreckage, but this does not include the C1 based ("halt" instruction) C1E mode. The BIOS/SMM controlled C1E state works on most systems even without enabling ACPI C-State control.
What a mess.
Yep, seems like AMD is breaking C1 semantics. Is it even valid from ACPI spec point of view?
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
What is the measured power savings that justifies this effort?
IMHO the power savings are not that important when such a kernel runs on bare metal:
Ok, so maybe we should disable C1E to work around its misdesign?
Disabling C1E is not an option. It saves the most power when you have a multicore AMD CPU. Neither C2 nor C3 are declared here. AMD C1E is hardware level power management if all cores are in the C1 state.
If you disable C1E and use a NOHZ kernel your power consumption is higher than using C1E with a periodic timer (say 250 HZ).
Quoted 1 line
It would be certainly nice to have noc1e command line option...
The sane way to disable it is in the BIOS -- if your BIOS provides such an option.
Thomas pointed already out that C1E might be enabled later during boot. So you would have to reset the respective bits whenever you enter idle. This is ugly and what would be the benefit of this?
Andreas