x86: Make offline cpus to go to deepest idle state using mwait
cover x86: Make offline cpus to go to deepest idle state using mwait
cpu hotplug has this long standing bug, with offline cpus going into hlt loop. That can result in offline CPU burning more power than while it is in online state (as when online it can potentially go to deeper C-state). Fix the bug using native mwait to enter deepest C-state in play_dead.
PATCH 1/2 x86: Add pm_play_dead funcptr to power-efficiently offline CPUs
Add a funcptr pm_play_dead (similar to pm_idle) that can take the offline CPUs to the most power efficient idle state.
This patch just adds the func pointer. The pointer will get initialized by patch that follows.
--- arch/x86/include/asm/smp.h | 2 ++ arch/x86/kernel/smpboot.c | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletions(-)
arch/x86/include/asm/smp.h +2 -0
@@ -148,6 +148,8 @@ static inline int num_booting_cpus(void) {return cpumask_weight (cpu_callout_mask); } + +extern void (*pm_play_dead)(void );#endif /* CONFIG_SMP */ extern unsigned disabled_cpus__cpuinitdata ;
arch/x86/kernel/smpboot.c +6 -1
@@ -1195,6 +1195,8 @@ __init void prefill_possible_map(void) nr_cpu_ids = possible; } +void (*pm_play_dead)(void ) =NULL ; +#ifdef CONFIG_HOTPLUG_CPU static void remove_siblinginfo (int cpu)
@@ -1313,7 +1315,10 @@ void play_dead_common(void)void native_play_dead (void ) {play_dead_common (); -wbinvd_halt (); +if (pm_play_dead) +pm_play_dead (); +else +wbinvd_halt (); }#else /* ... !CONFIG_HOTPLUG_CPU */
Signature · 3 lines
1.6.0.6 --
Re: [patch 1/2] x86: Add pm_play_dead funcptr to power-efficiently offline CPUs
Quoted 7 lines · On Fri, 2009-05-22 at 16:19 -0700, [email protected] wrote:
plain text document attachment (0001-x86-Add-pm_play_dead-funcptr-to-power-efficiently-o.patch) Add a funcptr
pm_play_dead(similar topm_idle) that can take the offline CPUs to the most power efficient idle state.This patch just adds the func pointer. The pointer will get initialized by patch that follows.
Since the pm_idle function pointer has given us so much grief, I don't think its wise to repeat that particular disaster.
I'd much rather see a framework where idle functions can be registered, and selected from based on criteria such as wakeup latency as provided by the pm_qos stuff, and power saving.
This framework should be shared between hotplug-idle and the regular idle routines. Hotplug would of course not care about things like wakeup latency and might therefore pick another idle routine.
Quoted 47 lines
Signed-off-by Venkatesh Pallipadi <[email protected]>--- arch/x86/include/asm/smp.h | 2 ++ arch/x86/kernel/smpboot.c | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletions(-)arch/x86/include/asm/smp.h +2
-0@@ -148,6 +148,8 @@ static inline int num_booting_cpus(void) {return cpumask_weight (cpu_callout_mask); } + +extern void (*pm_play_dead)(void );#endif /* CONFIG_SMP */ extern unsigned disabled_cpus__cpuinitdata ;arch/x86/kernel/smpboot.c +6
-1@@ -1195,6 +1195,8 @@ __init void prefill_possible_map(void) nr_cpu_ids = possible; } +void (*pm_play_dead)(void ) =NULL ; +#ifdef CONFIG_HOTPLUG_CPU static void remove_siblinginfo (int cpu)@@ -1313,7 +1315,10 @@ void play_dead_common(void)void native_play_dead (void ) {play_dead_common (); -wbinvd_halt (); +if (pm_play_dead) +pm_play_dead (); +else +wbinvd_halt (); }#else /* ... !CONFIG_HOTPLUG_CPU */ Signature · 1 line
1.6.0.6
RE: [patch 1/2] x86: Add pm_play_dead funcptr to power-efficiently offline CPUs
Quoted 25 lines
-----Original Message-----
From: Peter Zijlstra [mailto:[email protected]]
Sent: Saturday, May 23, 2009 3:45 AM
To: Pallipadi, Venkatesh
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Li, Shaohua; [email protected]
Subject: Re: [patch 1/2] x86: Addpm_play_deadfuncptr to power-efficiently offline CPUsQuoted 6 lines · On Fri, 2009-05-22 at 16:19 -0700, [email protected] wrote:
plain text document attachment (0001-x86-Add-pm_play_dead-funcptr-to-power-efficiently-o.patch) Add a funcptr
pm_play_dead(similar topm_idle) that can take the offline CPUs to the most power efficient idle state.This patch just adds the func pointer. The pointer will get
initialized
Quoted 1 line
by patch that follows.
Since the
pm_idlefunction pointer has given us so much grief, I don't think its wise to repeat that particular disaster.
Quoted 9 lines
I'd much rather see a framework where idle functions can be registered, and selected from based on criteria such as wakeup latency as provided by the
pm_qosstuff, and power saving.This framework should be shared between hotplug-idle and the regular idle routines. Hotplug would of course not care about things like wakeup
latency and might therefore pick another idle routine.
Yes. Thought about that approach. There are issues with that approach however.
- It would be ugly as we have to add logic in various low level idle handlers we have today mwait_idle, default_idle, acpi_mwait_idle, acpi_ioport_idle to make them aware of offline idle case and disable interrupt and not enable interrupt while going idle and not to wake up on interrupt.
- There are limitations on what we can and cannot do when offline. We have had issues with calling acpi methods and/or IO port accesses, during suspend before, so CPU going on suspend only uses C1. Specifically, with the above approach we will have:
1) CPU 1 going offline and starts using regular ACPI IO port C-states.
2) CPU 0 starts suspend and after some points it is unsafe to access ACPI io port accesses, which can potentially result in crash/hangs.
So, I would like to keep ACPI out of play_dead. And keeping idle and play_dead separate seemed to be cleaner way to do it.
Thanks,
Venki
RE: [patch 1/2] x86: Add pm_play_dead funcptr to power-efficiently offline CPUs
In reply to Peter ZijlstraQuoted 46 lines
From: Pallipadi, Venkatesh
Quoted 2 lines
From: Peter Zijlstra [mailto:[email protected]] On Fri, 2009-05-22 at 16:19 -0700,
Quoted 5 lines · [email protected] wrote:
Quoted 5 lines
plain text document attachment (0001-x86-Add-pm_play_dead-funcptr-to-power-efficiently-o.patch) Add a funcptr
pm_play_dead(similar topm_idle) that can take the offline CPUs to the most power efficient idle state.Quoted 8 lines
Quoted 1 line
This patch just adds the func pointer. The pointer will get
initialized
Quoted 1 line
by patch that follows.
Since the
pm_idlefunction pointer has given us so much grief, I don't think its wise to repeat that particular disaster.I'd much rather see a framework where idle functions can be
registered,
Quoted 8 lines
and selected from based on criteria such as wakeup latency as provided by the
pm_qosstuff, and power saving.This framework should be shared between hotplug-idle and the regular idle routines. Hotplug would of course not care about things like wakeup
latency and might therefore pick another idle routine.Yes. Thought about that approach. There are issues with that approach however.
- It would be ugly as we have to add logic in various low level idle handlers we have todaymwait_idle,default_idle,acpi_mwait_idle,acpi_ioport_idleto make them aware of offline idle case and disable interrupt and not enable interrupt while going idle and not to wake up on interrupt.
- There are limitations on what we can and cannot do when offline. We have had issues with calling acpi methods and/or IO port accesses, during suspend before, so CPU going on suspend only uses C1. Specifically, with the above approach we will have:
1) CPU 1 going offline and starts using regular ACPI IO port C-states.
2) CPU 0 starts suspend and after some points it is unsafe to access ACPI io port accesses, which can potentially result in crash/hangs.
Quoted 3 lines
So, I would like to keep ACPI out of
play_dead. And keeping idle andplay_deadseparate seemed to be cleaner way to do it.
Peter,
Any more comments on this approach? As I mentioned above, from my POV, it is simpler and cleaner to keep ACPI out of the CPU offline path. Do you agree?
Thanks,
Venki
PATCH 2/2 x86: put offline CPUs into deepest mwait cstate_subcstate
In reply to [email protected]Offline CPUs can save power by going into deepest cstate, subcstate instead of hlt loop.
--- arch/x86/kernel/acpi/cstate.c | 51 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 51 insertions(+), 0 deletions(-)
arch/x86/kernel/acpi/cstate.c +51 -0
@@ -150,6 +150,54 @@ void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) }EXPORT_SYMBOL_GPL (acpi_processor_ffh_cstate_enter); +static unsigned long mwait_play_dead_eax; + +static void mwait_play_dead (void ) +{ +if (boot_cpu_data.x86 >=4 ) +wbinvd (); + +while (1 ) { +__monitor ((void *)¤t_thread_info ()->flags,0 ,0 ); +smp_mb (); +__mwait (mwait_play_dead_eax,0 ); + } +} + +static void mwait_play_dead_init (void *unused) +{ +unsigned int eax, ebx, ecx, edx; +unsigned int highest_cstate =0 ; +unsigned int highest_subcstate =0 ; +int i; + +if (!boot_cpu_has (X86_FEATURE_MWAIT )) +return ; + +if (boot_cpu_data.cpuid_level <CPUID_MWAIT_LEAF ) +return ; + + pm_play_dead = mwait_play_dead; + +cpuid (CPUID_MWAIT_LEAF , &eax, &ebx, &ecx, &edx); +/* +* mwait_play_dead_eax will be 0 if EDX enumeration is not valid. +* Initialized below to cstate, sub_cstate value when EDX is valid. +*/ +if (!(ecx &0x1 )) +return ; + + edx >>=MWAIT_SUBSTATE_SIZE ; +for (i =0 ; i <7 && edx; i++, edx >>=MWAIT_SUBSTATE_SIZE ) { +if (edx &MWAIT_SUBSTATE_MASK ) { + highest_cstate = i; + highest_subcstate = edx &MWAIT_SUBSTATE_MASK ; + } + } + mwait_play_dead_eax = (highest_cstate <<MWAIT_SUBSTATE_SIZE ) | + (highest_subcstate -1 ); +} +static int __init ffh_cstate_init (void ) {struct cpuinfo_x86 *c = &boot_cpu_data;
@@ -157,6 +205,9 @@ static int __init ffh_cstate_init(void)return -1 ; cpu_cstate_entry = alloc_percpu(struct cstate_entry); + +smp_call_function_single (0 , mwait_play_dead_init,NULL ,1 ); +return 0 ; }
Signature · 3 lines
1.6.0.6 --
Re: [patch 2/2] x86: put offline CPUs into deepest mwait cstate_subcstate
Quoted 19 lines · Venkatesh
Offline CPUs can save power by going into deepest cstate, subcstate instead of hlt loop.
Signed-off-by Venkatesh Pallipadi <[email protected]>--- arch/x86/kernel/acpi/cstate.c | 51 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 51 insertions(+), 0 deletions(-)arch/x86/kernel/acpi/cstate.c +3
-0@@ -150,6 +150,54 @@ void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) }EXPORT_SYMBOL_GPL (acpi_processor_ffh_cstate_enter); +static unsigned long mwait_play_dead_eax; + +static void mwait_play_dead (void )
arch/x86/kernel/acpi/cstate.c void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) · line 165
Quoted 10 lines
arch/x86/kernel/acpi/cstate.c +10
-0+{ +if (boot_cpu_data.x86 >=4 ) +wbinvd (); + +while (1 ) { +__monitor ((void *)¤t_thread_info ()->flags,0 ,0 ); +smp_mb (); +__mwait (mwait_play_dead_eax,0 ); + } +}
CPU is dead, can current_thread_info() still be used? Maybe just monitor a never changed address.
Looks the patch will always take the highest native C-state, is this safe, considering BIOS usually limit C-state?
Thanks,
Shaohua
Re: [patch 2/2] x86: put offline CPUs into deepest mwait cstate_subcstate
Quoted 17 lines · Shaohua
Quoted 16 lines · Venkatesh
Offline CPUs can save power by going into deepest cstate, subcstate instead of hlt loop.
Signed-off-by Venkatesh Pallipadi <[email protected]>--- arch/x86/kernel/acpi/cstate.c | 51 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 51 insertions(+), 0 deletions(-)arch/x86/kernel/acpi/cstate.c +0
-0@@ -150,6 +150,54 @@ void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) }EXPORT_SYMBOL_GPL (acpi_processor_ffh_cstate_enter);
Quoted 16 lines
arch/x86/kernel/acpi/cstate.c void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) · line 165
Quoted 13 lines
arch/x86/kernel/acpi/cstate.c +13
-0+static unsigned long mwait_play_dead_eax; + +static void mwait_play_dead (void ) +{ +if (boot_cpu_data.x86 >=4 ) +wbinvd (); + +while (1 ) { +__monitor ((void *)¤t_thread_info ()->flags,0 ,0 ); +smp_mb (); +__mwait (mwait_play_dead_eax,0 ); + } +}CPU is dead, can
current_thread_info()still be used? Maybe just monitor a never changed address.
We still execute in this while loop even for an offline CPU, in case CPU just wakes up for whatever reason (HT sibling woke up etc). So, we should have stack and current_thread_info() even when offline, and we can use it here. I had a never changing variable in my earlier version of this patch. But removed it as I don't see why we should have couple of cache lines of data when we can avoid it.
Quoted 3 lines
Looks the patch will always take the highest native C-state, is this safe, considering BIOS usually limit C-state?
It may not be safe in terms of wakeup latency etc. So, we use BIOS CST for normal C-states. When offline, we don't really care about latency part. If CPU supports a C-state, we should be able to enter it and we can save most power with deepest C-state. If there are functionality issues with C-state, I am sure CPU feature (hw or microcode) will have it disabled, instead of depending on BIOSes to disable it.
Thanks,
Venki
[tip:x86/idle] x86, hotplug: Use mwait to offline a processor, fix the legacy case
In reply to [email protected]Commit-ID: ea53069231f9317062910d6e772cca4ce93de8c8 Gitweb: http://git.kernel.org/tip/ea53069231f9317062910d6e772cca4ce93de8c8 Author: H. Peter Anvin <[email protected]> AuthorDate: Fri, 17 Sep 2010 15:39:11 -0700 Committer: H. Peter Anvin <[email protected]> CommitDate: Fri, 17 Sep 2010 15:39:11 -0700
x86, hotplug: Use mwait to offline a processor, fix the legacy case
The code in native_play_dead() has a number of problems:
1. We should use MWAIT when available, to put ourselves into a deeper sleep state.
2. We use the existence of CLFLUSH to determine if WBINVD is safe, but that is totally bogus -- WBINVD is 486+, whereas CLFLUSH is a much later addition.
3. We should do WBINVD inside the loop, just in case of something like setting an A bit on page tables. Pointed out by Arjan van de Ven.
This code is based in part of a previous patch by Venki Pallipadi, but unlike that patch this one keeps all the detection code local instead of pre-caching a bunch of information. We're shutting down the CPU; there is absolutely no hurry.
This patch moves all the code to C and deletes the global wbinvd_halt() which is broken anyway.
LKML-Reference: <[email protected]>
--- arch/x86/include/asm/processor.h | 23 -------------- arch/x86/kernel/smpboot.c | 63 +++++++++++++++++++++++++++++++++++++- 2 files changed, 62 insertions(+), 24 deletions(-)
arch/x86/include/asm/processor.h +0 -23
@@ -764,29 +764,6 @@ extern unsigned long idle_halt;extern unsigned long idle_nomwait;extern bool c1e_detected; -/* -* on systems with caches, caches must be flashed as the absolute -* last instruction before going into a suspended halt. Otherwise, -* dirty data can linger in the cache and become stale on resume, -* leading to strange errors. -* -* perform a variety of operations to guarantee that the compiler -* will not reorder instructions. wbinvd itself is serializing -* so the processor will not reorder. -* -* Systems without cache can just go into halt. -*/ -static inline void wbinvd_halt (void ) -{ -mb (); -/* check for clflush to determine if wbinvd is legal */ -if (cpu_has_clflush) - asmvolatile ("cli; wbinvd; 1: hlt; jmp 1b" : : :"memory" ); -else -while (1 ) -halt (); -} -extern void enable_sep_cpu (void );extern int sysenter_setup (void );
arch/x86/kernel/smpboot.c +62 -1
79 lines not on this page · raw message
[tip:x86/idle] x86, hotplug: Move WBINVD back outside the play_dead loop
Commit-ID: a68e5c94f7d3dd64fef34dd5d97e365cae4bb42a Gitweb: http://git.kernel.org/tip/a68e5c94f7d3dd64fef34dd5d97e365cae4bb42a Author: H. Peter Anvin <[email protected]> AuthorDate: Fri, 17 Sep 2010 17:06:46 -0700 Committer: H. Peter Anvin <[email protected]> CommitDate: Fri, 17 Sep 2010 17:10:23 -0700
x86, hotplug: Move WBINVD back outside the play_dead loop
On processors with hyperthreading, when only one thread is offlined the other thread can cause a spurious wakeup on the idled thread. We do not want to re-WBINVD when that happens.
Ideally, we should simply skip WBINVD unless we're the last thread on a particular core to shut down, but there might be similar issues elsewhere in the system.
Thus, revert to previous behavior of only WBINVD outside the loop. Partly as a result, remove the mb()'s around it: they are not necessary since wbinvd() is a serializing instruction, but they were intended to make sure the compiler didn't do any funny loop optimizations.
LKML-Reference: <[email protected]>
--- arch/x86/kernel/smpboot.c | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-)
arch/x86/kernel/smpboot.c +5 -6
@@ -1422,9 +1422,9 @@ static inline void mwait_play_dead(void) (highest_subcstate -1 ); } +wbinvd (); +while (1 ) { -mb (); -wbinvd ();__monitor (¤t_thread_info ()->flags,0 ,0 );mb ();__mwait (eax,0 );
@@ -1433,11 +1433,10 @@ static inline void mwait_play_dead(void)static inline void hlt_play_dead (void ) { +if (current_cpu_data.x86 >=4 ) +wbinvd (); +while (1 ) { -mb (); -if (current_cpu_data.x86 >=4 ) -wbinvd (); -mb ();native_halt (); } }
Re: [tip:x86/idle] x86, hotplug: Move WBINVD back outside the play_dead loop
Quoted 39 lines · tip-bot for H. Peter Anvin
Commit-ID: a68e5c94f7d3dd64fef34dd5d97e365cae4bb42a
Gitweb: http://git.kernel.org/tip/a68e5c94f7d3dd64fef34dd5d97e365cae4bb42a
Author: H. Peter Anvin <[email protected]>
AuthorDate: Fri, 17 Sep 2010 17:06:46 -0700
Committer: H. Peter Anvin <[email protected]>
CommitDate: Fri, 17 Sep 2010 17:10:23 -0700x86, hotplug: Move WBINVD back outside the
play_deadloopOn processors with hyperthreading, when only one thread is offlined the other thread can cause a spurious wakeup on the idled thread. We do not want to re-WBINVD when that happens.
Ideally, we should simply skip WBINVD unless we're the last thread on a particular core to shut down, but there might be similar issues elsewhere in the system.
Thus, revert to previous behavior of only WBINVD outside the loop. Partly as a result, remove the mb()'s around it: they are not necessary since
wbinvd()is a serializing instruction, but they were intended to make sure the compiler didn't do any funny loop optimizations.Reported-by Asit Mallick <[email protected]>Signed-off-by H. Peter Anvin <[email protected]>Cc Arjan van de Ven <[email protected]>Cc Len Brown <[email protected]>Cc Venkatesh Pallipadi <[email protected]>Cc Peter Zijlstra <[email protected]>LKML-Reference: <[email protected]>
---
arch/x86/kernel/smpboot.c | 11 +++++------1 files changed, 5 insertions(+), 6 deletions(-)arch/x86/kernel/smpboot.c +0
-0
arch/x86/kernel/smpboot.c static inline void mwait_play_dead(void) · line 1429
Quoted 10 lines
arch/x86/kernel/smpboot.c +2
-2@@ -1422,9 +1422,9 @@ static inline void mwait_play_dead(void) (highest_subcstate - 1); } + wbinvd(); + while (1) { - mb(); - wbinvd(); __monitor(¤t_thread_info()->flags, 0, 0); mb();
Just one observation. There are some CPUs with errata that need clflush before monitor. So, if that CPU wakesup spuriously it may have problem reentering idle. Not sure whether that will be a problem as that errata also depended on read happening on the flag. May be its better to do monitor (0, 0, 0).
Thanks,
Venki
H. Peter Anvin
reviewerRe: [tip:x86/idle] x86, hotplug: Move WBINVD back outside the play_dead loop
Quoted 10 lines · Venkatesh Pallipadi
arch/x86/kernel/smpboot.c static inline void mwait_play_dead(void) · line 1429
Quoted 8 lines
arch/x86/kernel/smpboot.c +2
-2+wbinvd (); +while (1 ) { -mb (); -wbinvd ();__monitor (¤t_thread_info ()->flags,0 ,0 );mb ();
Quoted 6 lines
Just one observation. There are some CPUs with errata that need clflush before monitor. So, if that CPU wakesup spuriously it may have problem reentering idle. Not sure whether that will be a problem as that errata also depended on read happening on the flag. May be its better to do monitor (0, 0, 0).
It seems the easy way to deal with that would be to just add clflush before monitor... it is *probably* redundant, but it should be safe to do. It means depending on X86_FEATURE_CLFLUSH as well as X86_FEATURE_MWAIT, but I don't think there is any x86 processor which has MWAIT and not CLFLUSH, and I highly doubt there ever will be.
Does anyone see any downside?
-hpa
H. Peter Anvin
reviewerRe: [tip:x86/idle] x86, hotplug: Move WBINVD back outside the play_dead loop
Quoted 18 lines · H. Peter Anvin
Quoted 10 lines · Venkatesh Pallipadi
Review
Quoted 8 lines
(quoted diff) +2
-2+ wbinvd(); + while (1) { - mb(); - wbinvd(); __monitor(¤t_thread_info()->flags, 0, 0); mb();Quoted 6 lines
Just one observation. There are some CPUs with errata that need clflush before monitor. So, if that CPU wakesup spuriously it may have problem reentering idle. Not sure whether that will be a problem as that errata also depended on read happening on the flag. May be its better to do monitor (0, 0, 0).
Quoted 8 lines
It seems the easy way to deal with that would be to just add clflush before monitor... it is *probably* redundant, but it should be safe to do. It means depending on
X86_FEATURE_CLFLUSHas well asX86_FEATURE_MWAIT, but I don't think there is any x86 processor which has MWAIT and not CLFLUSH, and I highly doubt there ever will be.Does anyone see any downside?
Patch for review... anyone see any problem with it?
-hpa
Quoted 1 line
From
83c70ace9aad19ba7c41986f3c34a0808fbaf78fMon Sep 17 00:00:00 2001
From: H. Peter Anvin <[email protected]>
Date: Mon, 20 Sep 2010 13:04:45 -0700
Subject: [PATCH] x86, hotplug: In the MWAIT case of play_dead, CLFLUSH the cache line
When we're using MWAIT for play_dead, explicitly CLFLUSH the cache line before executing MONITOR. This is a potential workaround for the Xeon 7400 erratum AAI65 after having a spurious wakeup and returning around the loop. "Potential" here because it is not certain that that erratum could actually trigger; however, the CLFLUSH should be harmless.
--- arch/x86/kernel/smpboot.c | 20 +++++++++++++++++++- 1 files changed, 19 insertions(+), 1 deletions(-)
arch/x86/kernel/smpboot.c +19 -1
@@ -1394,9 +1394,12 @@ static inline void mwait_play_dead(void)unsigned int highest_cstate =0 ;unsigned int highest_subcstate =0 ;int i; +void *mwait_ptr;if (!cpu_has (¤t_cpu_data,X86_FEATURE_MWAIT ))return ; +if (!cpu_has (¤t_cpu_data,X86_FEATURE_CLFLSH )) +return ;if (current_cpu_data.cpuid_level <CPUID_MWAIT_LEAF )return ;
@@ -1422,10 +1425,25 @@ static inline void mwait_play_dead(void) (highest_subcstate -1 ); } +/* +* This should be a memory location in a cache line which is +* unlikely to be touched by other processors. The actual +* content is immaterial as it is not actually modified in any way. +*/ + mwait_ptr = ¤t_thread_info ()->flags; +wbinvd ();while (1 ) { -__monitor (¤t_thread_info ()->flags,0 ,0 ); +/* +* The CLFLUSH is a workaround for erratum AAI65 for +* the Xeon 7400 series. It's not clear it is actually +* needed, but it should be harmless in either case. +* The WBINVD is insufficient due to the spurious-wakeup +* case where we return around the loop. +*/ +clflush (mwait_ptr); +__monitor (mwait_ptr,0 ,0 );mb ();__mwait (eax,0 ); }
Signature · 1 line
1.7.2
Venkatesh Pallipadi
Acked-byRe: [tip:x86/idle] x86, hotplug: Move WBINVD back outside the play_dead loop
Quoted 25 lines · H. Peter Anvin
Quoted 24 lines · H. Peter Anvin
Quoted 10 lines · Venkatesh Pallipadi
Quoted 8 lines
+
wbinvd();
+
while (1) {
-mb();
-wbinvd();__monitor(¤t_thread_info()->flags, 0, 0);mb();Quoted 6 lines
Just one observation. There are some CPUs with errata that need clflush before monitor. So, if that CPU wakesup spuriously it may have problem reentering idle. Not sure whether that will be a problem as that errata also depended on read happening on the flag. May be its better to do monitor (0, 0, 0).
It seems the easy way to deal with that would be to just add clflush before monitor... it is *probably* redundant, but it should be safe to do. It means depending on
X86_FEATURE_CLFLUSHas well asX86_FEATURE_MWAIT, but I don't think there is any x86 processor which has MWAIT and not CLFLUSH, and I highly doubt there ever will be.
Quoted 5 lines
Quoted 2 lines
Does anyone see any downside?
Patch for review... anyone see any problem with it?
Looks good.
We can also close one of the longest standing bugs in bugzilla.kernel.org now :-)
https://bugzilla.kernel.org/show_bug.cgi?id=5471