x86: Arbitrary CPU hot(un)plug support
Subsystems (4)
v10 cover x86: Arbitrary CPU hot(un)plug support
From: Fenghua Yu <[email protected]>
CPU0 or BSP (Bootstrap Processor) has been the last processor that can not be hot removed on x86. This patchset implements CPU0 or BSP online and offline and removes this obstacle to CPU hotplug.
RAS needs the feature. If socket0 needs to be hotplugged for any reason (any thread on socket0 is bad, shared cache issue, uncore issue, etc), CPU0 is required to be offline or hot replaced to keep the system run. For example, starting with Core Duo, if you have a system that is reporting cache problems via the "yellow" status in the MCi_STATUS msr, then there is benefit in simply soft off-lining the cores that share that cache - assuming that leaves you at least one online core. A single socket system with L3 cache troubles is not helped - but problems in L1/L2 cache, or on multi-socket systems can be avoided. They are already being avoided for the cases where CPU0 is not involved. This patchset can help L1/L2 cache problem in CPU0 or L3 cache problem on the socket with CPU0 in a multi-socket system.
v10: Use first online CPU to handle retriggered irq in ioapic. Correct cpu0_logical_apicid for x2apic. Add wakeup_cpu_via_init_nmi() to make code more readable. Add ENDPROC(start_cpu0). Add comment for pm_notifier priority in cpu_hotplug_pm_callback.
v9: Add Intel vendor check to support the feature on Intel platforms only.
v8: Add smp_store_boot_cpu_info() and change smp_store_cpu_info() back to avoid a compilation error. Fix a few messy subject issues.
v7: Change smp_store_cpu_info() to allow store cpuinfo for CPU0 when online. Change PIC irq detection in native_cpu_disable().
v6: If CPU0 is offlined during boot time in CPU0 hotplug debug mode, put CPU0 online again before resuming from hibernation and disable x2apic and xapic.Don't set __CPUINIT for start_cpu0() in head_32.S. Clean up CPU0 wake up nmi handler after callin and callout sync. In a period (3 seconds), check if CPU0 wake up NMI is handled after offlined CPU0 exits from mwait.
v5: Add CONFIG_BOOTPARAM_HOTPLUG_CPU0 and CONFIG_DEBUG_HOTPLUG_CPU0. Simplify duplicate xstate_size init check. Wakeup CPU0 via nmi instead INITs. Add mcheck_cpu_init when CPU0 online. Change variable bsp_hotpluggable to cpu0_hotpluggable with __initdata qualifier.
v4: Add __read_mostly for internal bsp_hotpluggable variable. Add my email address in cpu-hotplug.txt document. A wording change in comment.
v3: Register a pm notifier to check if CPU0 is online before hibernate/suspend. Small wording changes in document and print info.
v2: Add locking changes between cpu hotplug and hibernate/suspend. Change PIC irq bound to CPU0 detection.
Fenghua Yu (14):
doc: Add x86 CPU0 online/offline feature
x86, Kconfig: Add config switch for CPU0 hotplug
x86, topology: Don't offline CPU0 if any PIC irq can not be migrated
out of it
x86, hotplug: Support functions for CPU0 online/offline x86, hotplug, suspend: Online CPU0 for suspend or hibernate kernel/cpu.c: Add comment for priority in cpu_hotplug_pm_callback x86-64, hotplug: Add start_cpu0() entry point to head_64.S x86-32, hotplug: Add start_cpu0() entry point to head_32.S x86, hotplug: Wake up CPU0 via NMI instead of INIT, SIPI, SIPI x86, hotplug: During CPU0 online, enable x2apic, set_numa_node. x86, hotplug: The first online processor saves the MTRR state x86, hotplug: Handle retrigger irq by the first available CPU x86/i387.c: Initialize thread xstate only on CPU0 only once x86, topology: Debug CPU00 hotplug
Documentation/cpu-hotplug.txt | 24 ++++++ Documentation/kernel-parameters.txt | 14 +++ arch/x86/Kconfig | 44 ++++++++++ arch/x86/include/asm/cpu.h | 4 + arch/x86/include/asm/smp.h | 1 + arch/x86/kernel/apic/io_apic.c | 4 +- arch/x86/kernel/cpu/common.c | 5 +- arch/x86/kernel/cpu/mtrr/main.c | 9 ++- arch/x86/kernel/head_32.S | 13 +++ arch/x86/kernel/head_64.S | 16 ++++ arch/x86/kernel/i387.c | 6 +- arch/x86/kernel/smpboot.c | 151 ++++++++++++++++++++++++++++------ arch/x86/kernel/topology.c | 101 ++++++++++++++++++++++-- arch/x86/power/cpu.c | 82 +++++++++++++++++++ kernel/cpu.c | 5 + 15 files changed, 438 insertions(+), 41 deletions(-)
Signature · 1 line
1.7.2
PATCH v10 1/14 doc: Add x86 CPU0 online/offline feature
From: Fenghua Yu <[email protected]>
If CONFIG_BOOTPARAM_HOTPLUG_CPU0 is turned on, CPU0 is hotpluggable. Otherwise, by default CPU0 is not hotpluggable and kernel parameter cpu0_hotplug enables CPU0 online/offline feature.
The documentations point out two known CPU0 dependencies. First, resume from hibernate or suspend always starts from CPU0. So hibernate and suspend are prevented if CPU0 is offline. Another dependency is PIC interrupts always go to CPU0.
It's said that some machines may depend on CPU0 to poweroff/reboot. But I haven't seen such dependency on a few tested machines.
Please let me know if you see any CPU0 dependencies on your machine.
--- Documentation/cpu-hotplug.txt | 24 ++++++++++++++++++++++++ Documentation/kernel-parameters.txt | 14 ++++++++++++++ 2 files changed, 38 insertions(+), 0 deletions(-)
Documentation/cpu-hotplug.txt +24 -0
@@ -207,6 +207,30 @@ by making it not-removable. In such cases you will also notice that the online file is missing under cpu0. +Q: Is CPU0 removable on X86? +A: Yes. If kernel is compiled with CONFIG_BOOTPARAM_HOTPLUG_CPU0=y, CPU0 is +removable by default. Otherwise, CPU0 is also removable by kernel option +cpu0_hotplug. + +But some features depend on CPU0. Two known dependencies are: + +1. Resume from hibernate/suspend depends on CPU0. Hibernate/suspend will fail if +CPU0 is offline and you need to online CPU0 before hibernate/suspend can +continue. +2. PIC interrupts also depend on CPU0. CPU0 can't be removed if a PIC interrupt +is detected. + +It's said poweroff/reboot may depend on CPU0 on some machines although I haven't +seen any poweroff/reboot failure so far after CPU0 is offline on a few tested +machines. + +Please let me know if you know or see any other dependencies of CPU0. + +If the dependencies are under your control, you can turn on CPU0 hotplug feature +either by CONFIG_BOOTPARAM_HOTPLUG_CPU0 or by kernel parameter cpu0_hotplug. + +--Fenghua Yu <[email protected]> + Q: How do i find out if a particular CPU is not removable? A: Depending on the implementation, some architectures may show this by the absence of the "online" file. This is done if it can be determined ahead of
Documentation/kernel-parameters.txt +14 -0
@@ -1988,6 +1988,20 @@ bytes respectively. Such letter suffixes can also be entirely omitted. nox2apic [X86-64,APIC] Do not enable x2APIC mode. + cpu0_hotplug [X86] Turn on CPU0 hotplug feature when + CONFIG_BOOTPARAM_HOTPLUG_CPU0 is off. + Some features depend on CPU0. Known dependencies are: + 1. Resume from suspend/hibernate depends on CPU0. + Suspend/hibernate will fail if CPU0 is offline and you + need to online CPU0 before suspend/hibernate. + 2. PIC interrupts also depend on CPU0. CPU0 can't be + removed if a PIC interrupt is detected. + It's said poweroff/reboot may depend on CPU0 on some + machines although I haven't seen such issues so far + after CPU0 is offline on a few tested machines. + If the dependencies are under your control, you can + turn on cpu0_hotplug. + nptcg= [IA-64] Override max number of concurrent global TLB purges which is reported from either PAL_VM_SUMMARY or SAL PALO.
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] doc: Add x86 CPU0 online/offline feature
Commit-ID: f78cff48c3d13636705851b0a9e733db117d58ee Gitweb: http://git.kernel.org/tip/f78cff48c3d13636705851b0a9e733db117d58ee Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:38 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:44 -0800
doc: Add x86 CPU0 online/offline feature
If CONFIG_BOOTPARAM_HOTPLUG_CPU0 is turned on, CPU0 is hotpluggable. Otherwise, by default CPU0 is not hotpluggable and kernel parameter cpu0_hotplug enables CPU0 online/offline feature.
The documentations point out two known CPU0 dependencies. First, resume from hibernate or suspend always starts from CPU0. So hibernate and suspend are prevented if CPU0 is offline. Another dependency is PIC interrupts always go to CPU0.
It's said that some machines may depend on CPU0 to poweroff/reboot. But I haven't seen such dependency on a few tested machines.
Please let me know if you see any CPU0 dependencies on your machine.
--- Documentation/cpu-hotplug.txt | 24 ++++++++++++++++++++++++ Documentation/kernel-parameters.txt | 14 ++++++++++++++ 2 files changed, 38 insertions(+)
Documentation/cpu-hotplug.txt +24 -0
@@ -207,6 +207,30 @@ by making it not-removable. In such cases you will also notice that the online file is missing under cpu0. +Q: Is CPU0 removable on X86? +A: Yes. If kernel is compiled with CONFIG_BOOTPARAM_HOTPLUG_CPU0=y, CPU0 is +removable by default. Otherwise, CPU0 is also removable by kernel option +cpu0_hotplug. + +But some features depend on CPU0. Two known dependencies are: + +1. Resume from hibernate/suspend depends on CPU0. Hibernate/suspend will fail if +CPU0 is offline and you need to online CPU0 before hibernate/suspend can +continue. +2. PIC interrupts also depend on CPU0. CPU0 can't be removed if a PIC interrupt +is detected. + +It's said poweroff/reboot may depend on CPU0 on some machines although I haven't +seen any poweroff/reboot failure so far after CPU0 is offline on a few tested +machines. + +Please let me know if you know or see any other dependencies of CPU0. + +If the dependencies are under your control, you can turn on CPU0 hotplug feature +either by CONFIG_BOOTPARAM_HOTPLUG_CPU0 or by kernel parameter cpu0_hotplug. + +--Fenghua Yu <[email protected]> + Q: How do i find out if a particular CPU is not removable? A: Depending on the implementation, some architectures may show this by the absence of the "online" file. This is done if it can be determined ahead of
Documentation/kernel-parameters.txt +14 -0
@@ -1984,6 +1984,20 @@ bytes respectively. Such letter suffixes can also be entirely omitted. nox2apic [X86-64,APIC] Do not enable x2APIC mode. + cpu0_hotplug [X86] Turn on CPU0 hotplug feature when + CONFIG_BOOTPARAM_HOTPLUG_CPU0 is off. + Some features depend on CPU0. Known dependencies are: + 1. Resume from suspend/hibernate depends on CPU0. + Suspend/hibernate will fail if CPU0 is offline and you + need to online CPU0 before suspend/hibernate. + 2. PIC interrupts also depend on CPU0. CPU0 can't be + removed if a PIC interrupt is detected. + It's said poweroff/reboot may depend on CPU0 on some + machines although I haven't seen such issues so far + after CPU0 is offline on a few tested machines. + If the dependencies are under your control, you can + turn on cpu0_hotplug. + nptcg= [IA-64] Override max number of concurrent global TLB purges which is reported from either PAL_VM_SUMMARY or SAL PALO.
PATCH v10 2/14 x86, Kconfig: Add config switch for CPU0 hotplug
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
New config switch CONFIG_BOOTPARAM_HOTPLUG_CPU0 sets default state of whether the CPU0 hotplug is on or off.
If the switch is off, CPU0 is not hotpluggable by default. But the CPU0 hotplug feature can still be turned on by kernel parameter cpu0_hotplug at boot.
If the switch is on, CPU0 is always hotpluggable.
The default value of the switch is off.
--- arch/x86/Kconfig | 29 +++++++++++++++++++++++++++++ 1 files changed, 29 insertions(+), 0 deletions(-)
arch/x86/Kconfig +29 -0
@@ -1730,6 +1730,35 @@ config HOTPLUG_CPU automatically on SMP systems. ) Say N if you want to disable CPU hotplug. +config BOOTPARAM_HOTPLUG_CPU0 + bool "Set default setting of cpu0_hotpluggable" + default n + depends on HOTPLUG_CPU && EXPERIMENTAL + ---help--- + Set whether default state of cpu0_hotpluggable is on or off. + + Say Y here to enable CPU0 hotplug by default. If this switch + is turned on, there is no need to give cpu0_hotplug kernel + parameter and the CPU0 hotplug feature is enabled by default. + + Please note: there are two known CPU0 dependencies if you want + to enable the CPU0 hotplug feature either by this switch or by + cpu0_hotplug kernel parameter. + + First, resume from hibernate or suspend always starts from CPU0. + So hibernate and suspend are prevented if CPU0 is offline. + + Second dependency is PIC interrupts always go to CPU0. CPU0 can not + offline if any interrupt can not migrate out of CPU0. There may + be other CPU0 dependencies. + + Please make sure the dependencies are under your control before + you enable this feature. + + Say N if you don't want to enable CPU0 hotplug feature by default. + You still can enable the CPU0 hotplug feature at boot by kernel + parameter cpu0_hotplug. + config COMPAT_VDSO def_bool y prompt "Compat VDSO support"
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, Kconfig: Add config switch for CPU0 hotplug
Commit-ID: 80aa1dff65717e7518647d4e27d1d3dcea5818e6 Gitweb: http://git.kernel.org/tip/80aa1dff65717e7518647d4e27d1d3dcea5818e6 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:39 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:46 -0800
x86, Kconfig: Add config switch for CPU0 hotplug
New config switch CONFIG_BOOTPARAM_HOTPLUG_CPU0 sets default state of whether the CPU0 hotplug is on or off.
If the switch is off, CPU0 is not hotpluggable by default. But the CPU0 hotplug feature can still be turned on by kernel parameter cpu0_hotplug at boot.
If the switch is on, CPU0 is always hotpluggable.
The default value of the switch is off.
--- arch/x86/Kconfig | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
arch/x86/Kconfig +29 -0
@@ -1698,6 +1698,35 @@ config HOTPLUG_CPU automatically on SMP systems. ) Say N if you want to disable CPU hotplug. +config BOOTPARAM_HOTPLUG_CPU0 + bool "Set default setting of cpu0_hotpluggable" + default n + depends on HOTPLUG_CPU && EXPERIMENTAL + ---help--- + Set whether default state of cpu0_hotpluggable is on or off. + + Say Y here to enable CPU0 hotplug by default. If this switch + is turned on, there is no need to give cpu0_hotplug kernel + parameter and the CPU0 hotplug feature is enabled by default. + + Please note: there are two known CPU0 dependencies if you want + to enable the CPU0 hotplug feature either by this switch or by + cpu0_hotplug kernel parameter. + + First, resume from hibernate or suspend always starts from CPU0. + So hibernate and suspend are prevented if CPU0 is offline. + + Second dependency is PIC interrupts always go to CPU0. CPU0 can not + offline if any interrupt can not migrate out of CPU0. There may + be other CPU0 dependencies. + + Please make sure the dependencies are under your control before + you enable this feature. + + Say N if you don't want to enable CPU0 hotplug feature by default. + You still can enable the CPU0 hotplug feature at boot by kernel + parameter cpu0_hotplug. + config COMPAT_VDSO def_bool y prompt "Compat VDSO support"
PATCH v10 3/14 x86, topology: Don't offline CPU0 if any PIC irq can not be migrated out of it
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
If CONFIG_BOOTPARAM_HOTPLUG_CPU is turned on, CPU0 hotplug feature is enabled by default.
If CONFIG_BOOTPARAM_HOTPLUG_CPU is not turned on, CPU0 hotplug feature is not enabled by default. The kernel parameter cpu0_hotplug can enable CPU0 hotplug feature at boot.
Currently the feature is supported on Intel platforms only.
--- arch/x86/kernel/topology.c | 50 +++++++++++++++++++++++++++++++++++++------ 1 files changed, 43 insertions(+), 7 deletions(-)
arch/x86/kernel/topology.c +43 -7
@@ -30,23 +30,59 @@#include <linux/mmzone.h> #include <linux/init.h> #include <linux/smp.h> +#include <linux/irq.h> #include <asm/cpu.h> static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);#ifdef CONFIG_HOTPLUG_CPU + +#ifdef CONFIG_BOOTPARAM_HOTPLUG_CPU0 +static int cpu0_hotpluggable =1 ; +#else +static int cpu0_hotpluggable; +static int __init enable_cpu0_hotplug (char *str) +{ + cpu0_hotpluggable =1 ; +return 1 ; +} + +__setup ("cpu0_hotplug" , enable_cpu0_hotplug); +#endif +int __ref arch_register_cpu (int num) { +struct cpuinfo_x86 *c = &cpu_data (num); + +/* +* Currently CPU0 is only hotpluggable on Intel platforms. Other +* vendors can add hotplug support later. +*/ +if (c->x86_vendor !=X86_VENDOR_INTEL ) + cpu0_hotpluggable =0 ; +/* -* CPU0 cannot be offlined due to several -* restrictions and assumptions in kernel. This basically -* doesn't add a control file, one cannot attempt to offline -* BSP. +* Two known BSP/CPU0 dependencies: Resume from suspend/hibernate +* depends on BSP. PIC interrupts depend on BSP. * -* Also certain PCI quirks require not to enable hotplug control -* for all CPU's. +* If the BSP depencies are under control, one can tell kernel to +* enable BSP hotplug. This basically adds a control file and +* one can attempt to offline BSP. */ -if (num) +if (num ==0 && cpu0_hotpluggable) { +unsigned int irq; +/* +* We won't take down the boot processor on i386 if some +* interrupts only are able to be serviced by the BSP in PIC. +*/ +for_each_active_irq (irq) { +if (!IO_APIC_IRQ (irq) &&irq_has_action (irq)) { + cpu0_hotpluggable =0 ; +break ; + } + } + } +if (num || cpu0_hotpluggable)per_cpu (cpu_devices, num).cpu.hotpluggable =1 ;return register_cpu (&per_cpu (cpu_devices, num).cpu, num);
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, topology: Don' t offline CPU0 if any PIC irq can not be migrated out of it
Commit-ID: 4d25031a81d3cd32edc00de6596db76cc4010685 Gitweb: http://git.kernel.org/tip/4d25031a81d3cd32edc00de6596db76cc4010685 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:40 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:47 -0800
x86, topology: Don't offline CPU0 if any PIC irq can not be migrated out of it
If CONFIG_BOOTPARAM_HOTPLUG_CPU is turned on, CPU0 hotplug feature is enabled by default.
If CONFIG_BOOTPARAM_HOTPLUG_CPU is not turned on, CPU0 hotplug feature is not enabled by default. The kernel parameter cpu0_hotplug can enable CPU0 hotplug feature at boot.
Currently the feature is supported on Intel platforms only.
--- arch/x86/kernel/topology.c | 50 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 43 insertions(+), 7 deletions(-)
arch/x86/kernel/topology.c +43 -7
@@ -30,23 +30,59 @@#include <linux/mmzone.h> #include <linux/init.h> #include <linux/smp.h> +#include <linux/irq.h> #include <asm/cpu.h> static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);#ifdef CONFIG_HOTPLUG_CPU + +#ifdef CONFIG_BOOTPARAM_HOTPLUG_CPU0 +static int cpu0_hotpluggable =1 ; +#else +static int cpu0_hotpluggable; +static int __init enable_cpu0_hotplug (char *str) +{ + cpu0_hotpluggable =1 ; +return 1 ; +} + +__setup ("cpu0_hotplug" , enable_cpu0_hotplug); +#endif +int __ref arch_register_cpu (int num) { +struct cpuinfo_x86 *c = &cpu_data (num); + +/* +* Currently CPU0 is only hotpluggable on Intel platforms. Other +* vendors can add hotplug support later. +*/ +if (c->x86_vendor !=X86_VENDOR_INTEL ) + cpu0_hotpluggable =0 ; +/* -* CPU0 cannot be offlined due to several -* restrictions and assumptions in kernel. This basically -* doesn't add a control file, one cannot attempt to offline -* BSP. +* Two known BSP/CPU0 dependencies: Resume from suspend/hibernate +* depends on BSP. PIC interrupts depend on BSP. * -* Also certain PCI quirks require not to enable hotplug control -* for all CPU's. +* If the BSP depencies are under control, one can tell kernel to +* enable BSP hotplug. This basically adds a control file and +* one can attempt to offline BSP. */ -if (num) +if (num ==0 && cpu0_hotpluggable) { +unsigned int irq; +/* +* We won't take down the boot processor on i386 if some +* interrupts only are able to be serviced by the BSP in PIC. +*/ +for_each_active_irq (irq) { +if (!IO_APIC_IRQ (irq) &&irq_has_action (irq)) { + cpu0_hotpluggable =0 ; +break ; + } + } + } +if (num || cpu0_hotpluggable)per_cpu (cpu_devices, num).cpu.hotpluggable =1 ;return register_cpu (&per_cpu (cpu_devices, num).cpu, num);
PATCH v10 4/14 x86, hotplug: Support functions for CPU0 online/offline
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
Add smp_store_boot_cpu_info() to store cpu info for BSP during boot time.
Now smp_store_cpu_info() stores cpu info for bringing up BSP or AP after it's offline.
Continue to online CPU0 in native_cpu_up().
Continue to offline CPU0 in native_cpu_disable().
--- arch/x86/include/asm/smp.h | 1 + arch/x86/kernel/smpboot.c | 38 ++++++++++++++++++-------------------- 2 files changed, 19 insertions(+), 20 deletions(-)
arch/x86/include/asm/smp.h +1 -0
@@ -166,6 +166,7 @@ void native_send_call_func_ipi(const struct cpumask *mask);void native_send_call_func_single_ipi (int cpu);void x86_idle_thread_init (unsigned int cpu,struct task_struct *idle); +void smp_store_boot_cpu_info (void );void smp_store_cpu_info (int id);#define cpu_physical_id (cpu) per_cpu(x86_cpu_to_apicid, cpu)
arch/x86/kernel/smpboot.c +18 -20
@@ -125,8 +125,8 @@ EXPORT_PER_CPU_SYMBOL(cpu_info);atomic_t init_deasserted;/* -* Report back to the Boot Processor. -* Running on AP. +* Report back to the Boot Processor during boot time or to the caller processor +* during CPU online. */ static void __cpuinit smp_callin (void ) {
@@ -279,19 +279,30 @@ notrace static void __cpuinit start_secondary(void *unused)cpu_idle (); } +void __init smp_store_boot_cpu_info (void ) +{ +int id =0 ;/* CPU 0 */ +struct cpuinfo_x86 *c = &cpu_data (id); + + *c = boot_cpu_data; + c->cpu_index = id; +} +/* * The bootstrap kernel entry code has set these up. Save them for * a given CPU */ -void __cpuinit smp_store_cpu_info (int id) {struct cpuinfo_x86 *c = &cpu_data (id); *c = boot_cpu_data; c->cpu_index = id; -if (id !=0 ) -identify_secondary_cpu (c); +/* +* During boot time, CPU0 has this setup already. Save the info when +* bringing up AP or offlined CPU0. +*/ +identify_secondary_cpu (c); } static bool __cpuinit
@@ -795,7 +806,7 @@ int __cpuinit native_cpu_up(unsigned int cpu, struct task_struct *tidle)pr_debug ("++++++++++++++++++++=_---CPU UP %u\n" , cpu); -if (apicid ==BAD_APICID || apicid == boot_cpu_physical_apicid || +if (apicid ==BAD_APICID || !physid_isset (apicid, phys_cpu_present_map) || !apic->apic_id_valid (apicid)) {pr_err ("%s: bad cpu %d\n" , __func__, cpu);
@@ -990,7 +1001,7 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus)/* * Setup boot CPU information */ -smp_store_cpu_info (0 );/* Final full version of the data */ +smp_store_boot_cpu_info ();/* Final full version of the data */ cpumask_copy (cpu_callin_mask,cpumask_of (0 ));mb ();
@@ -1214,19 +1225,6 @@ void cpu_disable_common(void)int native_cpu_disable (void ) { -int cpu =smp_processor_id (); - -/* -* Perhaps use cpufreq to drop frequency, but that could go -* into generic code. -* -* We won't take down the boot processor on i386 due to some -* interrupts only being able to be serviced by the BSP. -* Especially so if we're not using an IOAPIC -zwane -*/ -if (cpu ==0 ) -return -EBUSY ; -clear_local_APIC ();cpu_disable_common ();
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, hotplug: Support functions for CPU0 online/offline
Commit-ID: 30106c174311b8cfaaa3186c7f6f9c36c62d17da Gitweb: http://git.kernel.org/tip/30106c174311b8cfaaa3186c7f6f9c36c62d17da Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:41 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:48 -0800
x86, hotplug: Support functions for CPU0 online/offline
Add smp_store_boot_cpu_info() to store cpu info for BSP during boot time.
Now smp_store_cpu_info() stores cpu info for bringing up BSP or AP after it's offline.
Continue to online CPU0 in native_cpu_up().
Continue to offline CPU0 in native_cpu_disable().
--- arch/x86/include/asm/smp.h | 1 + arch/x86/kernel/smpboot.c | 38 ++++++++++++++++++-------------------- 2 files changed, 19 insertions(+), 20 deletions(-)
arch/x86/include/asm/smp.h +1 -0
@@ -166,6 +166,7 @@ void native_send_call_func_ipi(const struct cpumask *mask);void native_send_call_func_single_ipi (int cpu);void x86_idle_thread_init (unsigned int cpu,struct task_struct *idle); +void smp_store_boot_cpu_info (void );void smp_store_cpu_info (int id);#define cpu_physical_id (cpu) per_cpu(x86_cpu_to_apicid, cpu)
arch/x86/kernel/smpboot.c +18 -20
@@ -125,8 +125,8 @@ EXPORT_PER_CPU_SYMBOL(cpu_info);atomic_t init_deasserted;/* -* Report back to the Boot Processor. -* Running on AP. +* Report back to the Boot Processor during boot time or to the caller processor +* during CPU online. */ static void __cpuinit smp_callin (void ) {
@@ -279,19 +279,30 @@ notrace static void __cpuinit start_secondary(void *unused)cpu_idle (); } +void __init smp_store_boot_cpu_info (void ) +{ +int id =0 ;/* CPU 0 */ +struct cpuinfo_x86 *c = &cpu_data (id); + + *c = boot_cpu_data; + c->cpu_index = id; +} +/* * The bootstrap kernel entry code has set these up. Save them for * a given CPU */ -void __cpuinit smp_store_cpu_info (int id) {struct cpuinfo_x86 *c = &cpu_data (id); *c = boot_cpu_data; c->cpu_index = id; -if (id !=0 ) -identify_secondary_cpu (c); +/* +* During boot time, CPU0 has this setup already. Save the info when +* bringing up AP or offlined CPU0. +*/ +identify_secondary_cpu (c); } static bool __cpuinit
@@ -795,7 +806,7 @@ int __cpuinit native_cpu_up(unsigned int cpu, struct task_struct *tidle)pr_debug ("++++++++++++++++++++=_---CPU UP %u\n" , cpu); -if (apicid ==BAD_APICID || apicid == boot_cpu_physical_apicid || +if (apicid ==BAD_APICID || !physid_isset (apicid, phys_cpu_present_map) || !apic->apic_id_valid (apicid)) {pr_err ("%s: bad cpu %d\n" , __func__, cpu);
@@ -990,7 +1001,7 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus)/* * Setup boot CPU information */ -smp_store_cpu_info (0 );/* Final full version of the data */ +smp_store_boot_cpu_info ();/* Final full version of the data */ cpumask_copy (cpu_callin_mask,cpumask_of (0 ));mb ();
@@ -1214,19 +1225,6 @@ void cpu_disable_common(void)int native_cpu_disable (void ) { -int cpu =smp_processor_id (); - -/* -* Perhaps use cpufreq to drop frequency, but that could go -* into generic code. -* -* We won't take down the boot processor on i386 due to some -* interrupts only being able to be serviced by the BSP. -* Especially so if we're not using an IOAPIC -zwane -*/ -if (cpu ==0 ) -return -EBUSY ; -clear_local_APIC ();cpu_disable_common ();
PATCH v10 5/14 x86, hotplug, suspend: Online CPU0 for suspend or hibernate
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
Because x86 BIOS requires CPU0 to resume from sleep, suspend or hibernate can't be executed if CPU0 is detected offline. To make suspend or hibernate and further resume succeed, CPU0 must be online.
--- arch/x86/power/cpu.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 44 insertions(+), 0 deletions(-)
arch/x86/power/cpu.c +44 -0
@@ -237,3 +237,47 @@ void restore_processor_state(void)#ifdef CONFIG_X86_32 EXPORT_SYMBOL (restore_processor_state);#endif + +/* +* When bsp_check() is called in hibernate and suspend, cpu hotplug +* is disabled already. So it's unnessary to handle race condition between +* cpumask query and cpu hotplug. +*/ +static int bsp_check (void ) +{ +if (cpumask_first (cpu_online_mask) !=0 ) { +pr_warn ("CPU0 is offline.\n" ); +return -ENODEV ; + } + +return 0 ; +} + +static int bsp_pm_callback (struct notifier_block *nb,unsigned long action, +void *ptr) +{ +int ret =0 ; + +switch (action) { +case PM_SUSPEND_PREPARE : +case PM_HIBERNATION_PREPARE : + ret =bsp_check (); +break ; +default : +break ; + } +return notifier_from_errno (ret); +} + +static int __init bsp_pm_check_init (void ) +{ +/* +* Set this bsp_pm_callback as lower priority than +* cpu_hotplug_pm_callback. So cpu_hotplug_pm_callback will be called +* earlier to disable cpu hotplug before bsp online check. +*/ +pm_notifier (bsp_pm_callback, -INT_MAX ); +return 0 ; +} + +core_initcall (bsp_pm_check_init);
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, hotplug, suspend: Online CPU0 for suspend or hibernate
Commit-ID: 209efae12981f3d2d694499b761def10895c078c Gitweb: http://git.kernel.org/tip/209efae12981f3d2d694499b761def10895c078c Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:42 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:49 -0800
x86, hotplug, suspend: Online CPU0 for suspend or hibernate
Because x86 BIOS requires CPU0 to resume from sleep, suspend or hibernate can't be executed if CPU0 is detected offline. To make suspend or hibernate and further resume succeed, CPU0 must be online.
--- arch/x86/power/cpu.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)
arch/x86/power/cpu.c +44 -0
@@ -237,3 +237,47 @@ void restore_processor_state(void)#ifdef CONFIG_X86_32 EXPORT_SYMBOL (restore_processor_state);#endif + +/* +* When bsp_check() is called in hibernate and suspend, cpu hotplug +* is disabled already. So it's unnessary to handle race condition between +* cpumask query and cpu hotplug. +*/ +static int bsp_check (void ) +{ +if (cpumask_first (cpu_online_mask) !=0 ) { +pr_warn ("CPU0 is offline.\n" ); +return -ENODEV ; + } + +return 0 ; +} + +static int bsp_pm_callback (struct notifier_block *nb,unsigned long action, +void *ptr) +{ +int ret =0 ; + +switch (action) { +case PM_SUSPEND_PREPARE : +case PM_HIBERNATION_PREPARE : + ret =bsp_check (); +break ; +default : +break ; + } +return notifier_from_errno (ret); +} + +static int __init bsp_pm_check_init (void ) +{ +/* +* Set this bsp_pm_callback as lower priority than +* cpu_hotplug_pm_callback. So cpu_hotplug_pm_callback will be called +* earlier to disable cpu hotplug before bsp online check. +*/ +pm_notifier (bsp_pm_callback, -INT_MAX ); +return 0 ; +} + +core_initcall (bsp_pm_check_init);
PATCH v10 6/14 kernel/cpu.c: Add comment for priority in cpu_hotplug_pm_callback
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
cpu_hotplug_pm_callback should have higher priority than bsp_pm_callback which depends on cpu_hotplug_pm_callback to disable cpu hotplug to avoid race during bsp online checking.
This is to hightlight the priorities between the two callbacks in case people may overlook the order.
Ideally the priorities should be defined in macro/enum instead of fixed values. To do that, a seperate patchset may be pushed which will touch serveral other generic files and is out of scope of this patchset.
--- kernel/cpu.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-)
kernel/cpu.c +5 -0
@@ -601,6 +601,11 @@ cpu_hotplug_pm_callback(struct notifier_block *nb,static int __init cpu_hotplug_pm_sync_init (void ) { +/* +* cpu_hotplug_pm_callback has higher priority than x86 +* bsp_pm_callback which depends on cpu_hotplug_pm_callback +* to disable cpu hotplug to avoid cpu hotplug race. +*/ pm_notifier (cpu_hotplug_pm_callback,0 );return 0 ; }
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] kernel/cpu.c: Add comment for priority in cpu_hotplug_pm_callback
Commit-ID: 6e32d479db6079dd5d4309aa66aecbcf2664a5fe Gitweb: http://git.kernel.org/tip/6e32d479db6079dd5d4309aa66aecbcf2664a5fe Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:43 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:50 -0800
kernel/cpu.c: Add comment for priority in cpu_hotplug_pm_callback
cpu_hotplug_pm_callback should have higher priority than bsp_pm_callback which depends on cpu_hotplug_pm_callback to disable cpu hotplug to avoid race during bsp online checking.
This is to hightlight the priorities between the two callbacks in case people may overlook the order.
Ideally the priorities should be defined in macro/enum instead of fixed values. To do that, a seperate patchset may be pushed which will touch serveral other generic files and is out of scope of this patchset.
--- kernel/cpu.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/cpu.c +5 -0
@@ -601,6 +601,11 @@ cpu_hotplug_pm_callback(struct notifier_block *nb,static int __init cpu_hotplug_pm_sync_init (void ) { +/* +* cpu_hotplug_pm_callback has higher priority than x86 +* bsp_pm_callback which depends on cpu_hotplug_pm_callback +* to disable cpu hotplug to avoid cpu hotplug race. +*/ pm_notifier (cpu_hotplug_pm_callback,0 );return 0 ; }
PATCH v10 7/14 x86-64, hotplug: Add start_cpu0() entry point to head_64.S
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
start_cpu0() is defined in head_64.S for 64-bit. The function sets up stack and jumps to start_secondary() for CPU0 wake up.
--- arch/x86/kernel/head_64.S | 16 ++++++++++++++++ 1 files changed, 16 insertions(+), 0 deletions(-)
arch/x86/kernel/head_64.S +16 -0
@@ -252,6 +252,22 @@ ENTRY(secondary_startup_64) pushq %rax # target address in negative space lretq +#ifdef CONFIG_HOTPLUG_CPU +/* + * Boot CPU0 entry point. It's called from play_dead(). Everything has been set + * up already except stack. We just set up stack here. Then call + * start_secondary(). + */ +ENTRY(start_cpu0) + movq stack_start(%rip),%rsp + movq initial_code(%rip),%rax + pushq $0 # fake return address to stop unwinder + pushq $__KERNEL_CS # set correct cs + pushq %rax # target address in negative space + lretq +ENDPROC(start_cpu0) +#endif + /* SMP bootup changes these two */ __REFDATA .align 8
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86-64, hotplug: Add start_cpu0() entry point to head_64.S
Commit-ID: 42e78e9719aa0c76711e2731b19c90fe5ae05278 Gitweb: http://git.kernel.org/tip/42e78e9719aa0c76711e2731b19c90fe5ae05278 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:44 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:51 -0800
x86-64, hotplug: Add start_cpu0() entry point to head_64.S
start_cpu0() is defined in head_64.S for 64-bit. The function sets up stack and jumps to start_secondary() for CPU0 wake up.
--- arch/x86/kernel/head_64.S | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/x86/kernel/head_64.S +16 -0
@@ -252,6 +252,22 @@ ENTRY(secondary_startup_64) pushq %rax # target address in negative space lretq +#ifdef CONFIG_HOTPLUG_CPU +/* + * Boot CPU0 entry point. It's called from play_dead(). Everything has been set + * up already except stack. We just set up stack here. Then call + * start_secondary(). + */ +ENTRY(start_cpu0) + movq stack_start(%rip),%rsp + movq initial_code(%rip),%rax + pushq $0 # fake return address to stop unwinder + pushq $__KERNEL_CS # set correct cs + pushq %rax # target address in negative space + lretq +ENDPROC(start_cpu0) +#endif + /* SMP bootup changes these two */ __REFDATA .align 8
PATCH v10 8/14 x86-32, hotplug: Add start_cpu0() entry point to head_32.S
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
start_cpu0() is defined in head_32.S for 32-bit. The function sets up stack and jumps to start_secondary() for CPU0 wake up.
--- arch/x86/kernel/head_32.S | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-)
arch/x86/kernel/head_32.S +13 -0
@@ -266,6 +266,19 @@ num_subarch_entries = (. - subarch_entries) / 4 jmp default_entry #endif /* CONFIG_PARAVIRT */ +#ifdef CONFIG_HOTPLUG_CPU +/* + * Boot CPU0 entry point. It's called from play_dead(). Everything has been set + * up already except stack. We just set up stack here. Then call + * start_secondary(). + */ +ENTRY(start_cpu0) + movl stack_start, %ecx + movl %ecx, %esp + jmp *(initial_code) +ENDPROC(start_cpu0) +#endif + /* * Non-boot CPU entry point; entered from trampoline.S * We can't lgdt here, because lgdt itself uses a data segment, but
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86-32, hotplug: Add start_cpu0() entry point to head_32.S
Commit-ID: 3e2a0cc3cdc19e0518ae87583add40ea1bf55b67 Gitweb: http://git.kernel.org/tip/3e2a0cc3cdc19e0518ae87583add40ea1bf55b67 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:45 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:52 -0800
x86-32, hotplug: Add start_cpu0() entry point to head_32.S
start_cpu0() is defined in head_32.S for 32-bit. The function sets up stack and jumps to start_secondary() for CPU0 wake up.
--- arch/x86/kernel/head_32.S | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/kernel/head_32.S +13 -0
@@ -266,6 +266,19 @@ num_subarch_entries = (. - subarch_entries) / 4 jmp default_entry #endif /* CONFIG_PARAVIRT */ +#ifdef CONFIG_HOTPLUG_CPU +/* + * Boot CPU0 entry point. It's called from play_dead(). Everything has been set + * up already except stack. We just set up stack here. Then call + * start_secondary(). + */ +ENTRY(start_cpu0) + movl stack_start, %ecx + movl %ecx, %esp + jmp *(initial_code) +ENDPROC(start_cpu0) +#endif + /* * Non-boot CPU entry point; entered from trampoline.S * We can't lgdt here, because lgdt itself uses a data segment, but
PATCH v10 9/14 x86, hotplug: Wake up CPU0 via NMI instead of INIT, SIPI, SIPI
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
Instead of waiting for STARTUP after INITs, BSP will execute the BIOS boot-strap code which is not a desired behavior for waking up BSP. To avoid the boot-strap code, wake up CPU0 by NMI instead.
This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined (i.e. physically hot removed and then hot added), NMI won't wake it up. We'll change this code in the future to wake up hard offlined CPU0 if real platform and request are available.
AP is still waken up as before by INIT, SIPI, SIPI sequence.
--- arch/x86/include/asm/cpu.h | 1 + arch/x86/kernel/smpboot.c | 113 +++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 107 insertions(+), 7 deletions(-)
arch/x86/include/asm/cpu.h +1 -0
@@ -28,6 +28,7 @@ struct x86_cpu {#ifdef CONFIG_HOTPLUG_CPU extern int arch_register_cpu (int num);extern void arch_unregister_cpu (int ); +extern void __cpuinit start_cpu0 (void );#endif DECLARE_PER_CPU (int, cpu_state);
arch/x86/kernel/smpboot.c +106 -7
202 lines not on this page · raw message
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, hotplug: Wake up CPU0 via NMI instead of INIT, SIPI, SIPI
Commit-ID: f72614ed18fb4698d27a3d5cfdea96599b08a084 Gitweb: http://git.kernel.org/tip/f72614ed18fb4698d27a3d5cfdea96599b08a084 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:46 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:53 -0800
x86, hotplug: Wake up CPU0 via NMI instead of INIT, SIPI, SIPI
Instead of waiting for STARTUP after INITs, BSP will execute the BIOS boot-strap code which is not a desired behavior for waking up BSP. To avoid the boot-strap code, wake up CPU0 by NMI instead.
This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined (i.e. physically hot removed and then hot added), NMI won't wake it up. We'll change this code in the future to wake up hard offlined CPU0 if real platform and request are available.
AP is still waken up as before by INIT, SIPI, SIPI sequence.
--- arch/x86/include/asm/cpu.h | 1 + arch/x86/kernel/smpboot.c | 113 ++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 107 insertions(+), 7 deletions(-)
arch/x86/include/asm/cpu.h +1 -0
@@ -28,6 +28,7 @@ struct x86_cpu {#ifdef CONFIG_HOTPLUG_CPU extern int arch_register_cpu (int num);extern void arch_unregister_cpu (int ); +extern void __cpuinit start_cpu0 (void );#endif DECLARE_PER_CPU (int, cpu_state);
arch/x86/kernel/smpboot.c +106 -7
202 lines not on this page · raw message
PATCH v10 10/14 x86, hotplug: During CPU0 online, enable x2apic, set_numa_node.
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
Previously these functions were not run on the BSP (CPU 0, the boot processor) since the boot processor init would only be executed before this functionality was initialized.
--- arch/x86/kernel/cpu/common.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/common.c +2 -3
@@ -1237,7 +1237,7 @@ void __cpuinit cpu_init(void) oist = &per_cpu (orig_ist, cpu);#ifdef CONFIG_NUMA -if (cpu !=0 &&this_cpu_read (numa_node) ==0 && +if (this_cpu_read (numa_node) ==0 &&early_cpu_to_node (cpu) !=NUMA_NO_NODE )set_numa_node (early_cpu_to_node (cpu));#endif
@@ -1269,8 +1269,7 @@ void __cpuinit cpu_init(void)barrier ();x86_configure_nx (); -if (cpu !=0 ) -enable_x2apic (); +enable_x2apic ();/* * set up and load the per-CPU TSS
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, hotplug: During CPU0 online, enable x2apic, set_numa_node.
Commit-ID: 27ac537ab4c7f9675e93d1fa73df5f8f233b871d Gitweb: http://git.kernel.org/tip/27ac537ab4c7f9675e93d1fa73df5f8f233b871d Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:47 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:54 -0800
x86, hotplug: During CPU0 online, enable x2apic, set_numa_node.
Previously these functions were not run on the BSP (CPU 0, the boot processor) since the boot processor init would only be executed before this functionality was initialized.
--- arch/x86/kernel/cpu/common.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/common.c +2 -3
@@ -1237,7 +1237,7 @@ void __cpuinit cpu_init(void) oist = &per_cpu (orig_ist, cpu);#ifdef CONFIG_NUMA -if (cpu !=0 &&this_cpu_read (numa_node) ==0 && +if (this_cpu_read (numa_node) ==0 &&early_cpu_to_node (cpu) !=NUMA_NO_NODE )set_numa_node (early_cpu_to_node (cpu));#endif
@@ -1269,8 +1269,7 @@ void __cpuinit cpu_init(void)barrier ();x86_configure_nx (); -if (cpu !=0 ) -enable_x2apic (); +enable_x2apic ();/* * set up and load the per-CPU TSS
[tip:x86/bsp-hotplug] x86, hotplug: During CPU0 online, enable x2apic, set_numa_node.
In reply to Fenghua YuCommit-ID: 27fd185f3d9e83c64b7a596881d7751a638c9683 Gitweb: http://git.kernel.org/tip/27fd185f3d9e83c64b7a596881d7751a638c9683 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:47 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 15:28:10 -0800
x86, hotplug: During CPU0 online, enable x2apic, set_numa_node.
Previously these functions were not run on the BSP (CPU 0, the boot processor) since the boot processor init would only be executed before this functionality was initialized.
--- arch/x86/kernel/cpu/common.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/common.c +2 -3
@@ -1237,7 +1237,7 @@ void __cpuinit cpu_init(void) oist = &per_cpu (orig_ist, cpu);#ifdef CONFIG_NUMA -if (cpu !=0 &&this_cpu_read (numa_node) ==0 && +if (this_cpu_read (numa_node) ==0 &&early_cpu_to_node (cpu) !=NUMA_NO_NODE )set_numa_node (early_cpu_to_node (cpu));#endif
@@ -1269,8 +1269,7 @@ void __cpuinit cpu_init(void)barrier ();x86_configure_nx (); -if (cpu !=0 ) -enable_x2apic (); +enable_x2apic ();/* * set up and load the per-CPU TSS
PATCH v10 11/14 x86, hotplug: The first online processor saves the MTRR state
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
Ask the first online CPU to save mtrr instead of asking BSP. BSP could be offline when mtrr_save_state() is called.
--- arch/x86/kernel/cpu/mtrr/main.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/mtrr/main.c +7 -2
@@ -695,11 +695,16 @@ void mtrr_ap_init(void) }/** -* Save current fixed-range MTRR state of the BSP +* Save current fixed-range MTRR state of the first cpu in cpu_online_mask. */ void mtrr_save_state (void ) { -smp_call_function_single (0 , mtrr_save_fixed_ranges,NULL ,1 ); +int first_cpu; + +get_online_cpus (); + first_cpu =cpumask_first (cpu_online_mask); +smp_call_function_single (first_cpu, mtrr_save_fixed_ranges,NULL ,1 ); +put_online_cpus (); }void set_mtrr_aps_delayed_init (void )
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, hotplug: The first online processor saves the MTRR state
Commit-ID: e7c6d2368362461d1c9e5d58bedbed9c3862b607 Gitweb: http://git.kernel.org/tip/e7c6d2368362461d1c9e5d58bedbed9c3862b607 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:48 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:55 -0800
x86, hotplug: The first online processor saves the MTRR state
Ask the first online CPU to save mtrr instead of asking BSP. BSP could be offline when mtrr_save_state() is called.
--- arch/x86/kernel/cpu/mtrr/main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/mtrr/main.c +7 -2
@@ -695,11 +695,16 @@ void mtrr_ap_init(void) }/** -* Save current fixed-range MTRR state of the BSP +* Save current fixed-range MTRR state of the first cpu in cpu_online_mask. */ void mtrr_save_state (void ) { -smp_call_function_single (0 , mtrr_save_fixed_ranges,NULL ,1 ); +int first_cpu; + +get_online_cpus (); + first_cpu =cpumask_first (cpu_online_mask); +smp_call_function_single (first_cpu, mtrr_save_fixed_ranges,NULL ,1 ); +put_online_cpus (); }void set_mtrr_aps_delayed_init (void )
[tip:x86/bsp-hotplug] x86, hotplug: The first online processor saves the MTRR state
In reply to Fenghua YuCommit-ID: 30242aa6023b71325c6b8addac06faf544a85fd0 Gitweb: http://git.kernel.org/tip/30242aa6023b71325c6b8addac06faf544a85fd0 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:48 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 15:28:10 -0800
x86, hotplug: The first online processor saves the MTRR state
Ask the first online CPU to save mtrr instead of asking BSP. BSP could be offline when mtrr_save_state() is called.
--- arch/x86/kernel/cpu/mtrr/main.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/mtrr/main.c +7 -2
@@ -695,11 +695,16 @@ void mtrr_ap_init(void) }/** -* Save current fixed-range MTRR state of the BSP +* Save current fixed-range MTRR state of the first cpu in cpu_online_mask. */ void mtrr_save_state (void ) { -smp_call_function_single (0 , mtrr_save_fixed_ranges,NULL ,1 ); +int first_cpu; + +get_online_cpus (); + first_cpu =cpumask_first (cpu_online_mask); +smp_call_function_single (first_cpu, mtrr_save_fixed_ranges,NULL ,1 ); +put_online_cpus (); }void set_mtrr_aps_delayed_init (void )
PATCH v10 12/14 x86, hotplug: Handle retrigger irq by the first available CPU
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
The first cpu in irq cfg->domain is likely to be CPU 0 and may not be available when CPU 0 is offline. Instead of using CPU 0 to handle retriggered irq, we use first available CPU which is online and in this irq's domain.
--- arch/x86/kernel/apic/io_apic.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-)
arch/x86/kernel/apic/io_apic.c +3 -1
@@ -2199,9 +2199,11 @@ static int ioapic_retrigger_irq(struct irq_data *data) {struct irq_cfg *cfg = data->chip_data;unsigned long flags; +int cpu;raw_spin_lock_irqsave (&vector_lock, flags); - apic->send_IPI_mask (cpumask_of (cpumask_first (cfg->domain)), cfg->vector); + cpu =cpumask_first_and (cfg->domain, cpu_online_mask); + apic->send_IPI_mask (cpumask_of (cpu), cfg->vector);raw_spin_unlock_irqrestore (&vector_lock, flags);return 1 ;
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, hotplug: Handle retrigger irq by the first available CPU
Commit-ID: 41bfa818b30d595b1d9feee8eb8baee9befff44b Gitweb: http://git.kernel.org/tip/41bfa818b30d595b1d9feee8eb8baee9befff44b Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:49 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:55 -0800
x86, hotplug: Handle retrigger irq by the first available CPU
The first cpu in irq cfg->domain is likely to be CPU 0 and may not be available when CPU 0 is offline. Instead of using CPU 0 to handle retriggered irq, we use first available CPU which is online and in this irq's domain.
--- arch/x86/kernel/apic/io_apic.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kernel/apic/io_apic.c +3 -1
@@ -2199,9 +2199,11 @@ static int ioapic_retrigger_irq(struct irq_data *data) {struct irq_cfg *cfg = data->chip_data;unsigned long flags; +int cpu;raw_spin_lock_irqsave (&vector_lock, flags); - apic->send_IPI_mask (cpumask_of (cpumask_first (cfg->domain)), cfg->vector); + cpu =cpumask_first_and (cfg->domain, cpu_online_mask); + apic->send_IPI_mask (cpumask_of (cpu), cfg->vector);raw_spin_unlock_irqrestore (&vector_lock, flags);return 1 ;
[tip:x86/bsp-hotplug] x86, hotplug: Handle retrigger irq by the first available CPU
In reply to Fenghua YuCommit-ID: 8d966a04107e56993a051cd41ead0b4f23ba2414 Gitweb: http://git.kernel.org/tip/8d966a04107e56993a051cd41ead0b4f23ba2414 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:49 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 15:28:11 -0800
x86, hotplug: Handle retrigger irq by the first available CPU
The first cpu in irq cfg->domain is likely to be CPU 0 and may not be available when CPU 0 is offline. Instead of using CPU 0 to handle retriggered irq, we use first available CPU which is online and in this irq's domain.
--- arch/x86/kernel/apic/io_apic.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kernel/apic/io_apic.c +3 -1
@@ -2199,9 +2199,11 @@ static int ioapic_retrigger_irq(struct irq_data *data) {struct irq_cfg *cfg = data->chip_data;unsigned long flags; +int cpu;raw_spin_lock_irqsave (&vector_lock, flags); - apic->send_IPI_mask (cpumask_of (cpumask_first (cfg->domain)), cfg->vector); + cpu =cpumask_first_and (cfg->domain, cpu_online_mask); + apic->send_IPI_mask (cpumask_of (cpu), cfg->vector);raw_spin_unlock_irqrestore (&vector_lock, flags);return 1 ;
PATCH v10 13/14 x86/i387.c: Initialize thread xstate only on CPU0 only once
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
init_thread_xstate() is only called once to avoid overriding xstate_size during boot time or during CPU hotplug.
--- arch/x86/kernel/i387.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)
arch/x86/kernel/i387.c +5 -1
@@ -175,7 +175,11 @@ void __cpuinit fpu_init(void) cr0 |=X86_CR0_EM ;write_cr0 (cr0); -if (!smp_processor_id ()) +/* +* init_thread_xstate is only called once to avoid overriding +* xstate_size during boot time or during CPU hotplug. +*/ +if (xstate_size ==0 )init_thread_xstate ();mxcsr_feature_mask_init ();
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86/i387.c: Initialize thread xstate only on CPU0 only once
Commit-ID: cc5d167b58af1d858aa5a2ade158acb94af4df70 Gitweb: http://git.kernel.org/tip/cc5d167b58af1d858aa5a2ade158acb94af4df70 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:50 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 09:39:56 -0800
x86/i387.c: Initialize thread xstate only on CPU0 only once
init_thread_xstate() is only called once to avoid overriding xstate_size during boot time or during CPU hotplug.
--- arch/x86/kernel/i387.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kernel/i387.c +5 -1
@@ -175,7 +175,11 @@ void __cpuinit fpu_init(void) cr0 |=X86_CR0_EM ;write_cr0 (cr0); -if (!smp_processor_id ()) +/* +* init_thread_xstate is only called once to avoid overriding +* xstate_size during boot time or during CPU hotplug. +*/ +if (xstate_size ==0 )init_thread_xstate ();mxcsr_feature_mask_init ();
[tip:x86/bsp-hotplug] x86/i387.c: Initialize thread xstate only on CPU0 only once
In reply to Fenghua YuCommit-ID: 6f5298c2139b06925037490367906f3d73955b86 Gitweb: http://git.kernel.org/tip/6f5298c2139b06925037490367906f3d73955b86 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:50 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 15:28:11 -0800
x86/i387.c: Initialize thread xstate only on CPU0 only once
init_thread_xstate() is only called once to avoid overriding xstate_size during boot time or during CPU hotplug.
--- arch/x86/kernel/i387.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/kernel/i387.c +5 -1
@@ -175,7 +175,11 @@ void __cpuinit fpu_init(void) cr0 |=X86_CR0_EM ;write_cr0 (cr0); -if (!smp_processor_id ()) +/* +* init_thread_xstate is only called once to avoid overriding +* xstate_size during boot time or during CPU hotplug. +*/ +if (xstate_size ==0 )init_thread_xstate ();mxcsr_feature_mask_init ();
PATCH v10 14/14 x86, topology: Debug CPU00 hotplug
In reply to Fenghua YuFrom: Fenghua Yu <[email protected]>
CONFIG_DEBUG_HOTPLUG_CPU0 is for debugging the CPU0 hotplug feature. The switch offlines CPU0 as soon as possible and boots userspace up with CPU0 offlined. User can online CPU0 back after boot time. The default value of the switch is off.
To debug CPU0 hotplug, you need to enable CPU0 offline/online feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during compilation or giving cpu0_hotplug kernel parameter at boot.
It's safe and early place to take down CPU0 after all hotplug notifiers are installed and SMP is booted.
Please note that some applications or drivers, e.g. some versions of udevd, during boot time may put CPU0 online again in this CPU0 hotplug debug mode.
In this debug mode, setup_local_APIC() may report a warning on max_loops<=0 when CPU0 is onlined back after boot time. This is because pending interrupt in IRR can not move to ISR. The warning is not CPU0 specfic and it can happen on other CPUs as well. It is harmless except the first CPU0 online takes a bit longer time. And so this debug mode is useful to expose this issue. I'll send a seperate patch to fix this generic warning issue.
--- arch/x86/Kconfig | 15 +++++++++++++ arch/x86/include/asm/cpu.h | 3 ++ arch/x86/kernel/topology.c | 51 ++++++++++++++++++++++++++++++++++++++++++++ arch/x86/power/cpu.c | 38 ++++++++++++++++++++++++++++++++ 4 files changed, 107 insertions(+), 0 deletions(-)
arch/x86/Kconfig +15 -0
@@ -1759,6 +1759,21 @@ config BOOTPARAM_HOTPLUG_CPU0 You still can enable the CPU0 hotplug feature at boot by kernel parameter cpu0_hotplug. +config DEBUG_HOTPLUG_CPU0 + def_bool n + prompt "Debug CPU0 hotplug" + depends on HOTPLUG_CPU && EXPERIMENTAL + ---help--- + Enabling this option offlines CPU0 (if CPU0 can be offlined) as + soon as possible and boots up userspace with CPU0 offlined. User + can online CPU0 back after boot time. + + To debug CPU0 hotplug, you need to enable CPU0 offline/online + feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during + compilation or giving cpu0_hotplug kernel parameter at boot. + + If unsure, say N. + config COMPAT_VDSO def_bool y prompt "Compat VDSO support"
arch/x86/include/asm/cpu.h +3 -0
@@ -29,6 +29,9 @@ struct x86_cpu {extern int arch_register_cpu (int num);extern void arch_unregister_cpu (int );extern void __cpuinit start_cpu0 (void ); +#ifdef CONFIG_DEBUG_HOTPLUG_CPU0 +extern int _debug_hotplug_cpu (int cpu,int action); +#endif #endif DECLARE_PER_CPU (int, cpu_state);
arch/x86/kernel/topology.c +51 -0
57 lines not on this page · raw message
arch/x86/power/cpu.c +38 -0
@@ -21,6 +21,7 @@#include <asm/suspend.h> #include <asm/debugreg.h> #include <asm/fpu-internal.h> /* pcntxt_mask */ +#include <asm/cpu.h> #ifdef CONFIG_X86_32 static struct saved_context saved_context;
@@ -263,6 +264,43 @@ static int bsp_pm_callback(struct notifier_block *nb, unsigned long action,case PM_HIBERNATION_PREPARE : ret =bsp_check ();break ; +#ifdef CONFIG_DEBUG_HOTPLUG_CPU0 +case PM_RESTORE_PREPARE : +/* +* When system resumes from hibernation, online CPU0 because +* 1. it's required for resume and +* 2. the CPU was online before hibernation +*/ +if (!cpu_online (0 )) +_debug_hotplug_cpu (0 ,1 ); +break ; +case PM_POST_RESTORE : +/* +* When a resume really happens, this code won't be called. +* +* This code is called only when user space hibernation software +* prepares for snapshot device during boot time. So we just +* call _debug_hotplug_cpu() to restore to CPU0's state prior to +* preparing the snapshot device. +* +* This works for normal boot case in our CPU0 hotplug debug +* mode, i.e. CPU0 is offline and user mode hibernation +* software initializes during boot time. +* +* If CPU0 is online and user application accesses snapshot +* device after boot time, this will offline CPU0 and user may +* see different CPU0 state before and after accessing +* the snapshot device. But hopefully this is not a case when +* user debugging CPU0 hotplug. Even if users hit this case, +* they can easily online CPU0 back. +* +* To simplify this debug code, we only consider normal boot +* case. Otherwise we need to remember CPU0's state and restore +* to that state and resolve racy conditions etc. +*/ +_debug_hotplug_cpu (0 ,0 ); +break ; +#endif default :break ; }
Signature · 1 line
1.7.2
[tip:x86/bsp-hotplug] x86, topology: Debug CPU0 hotplug
Commit-ID: 69d130989b0f96508f02f508b88f545fec72cc08 Gitweb: http://git.kernel.org/tip/69d130989b0f96508f02f508b88f545fec72cc08 Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:51 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 10:29:04 -0800
x86, topology: Debug CPU0 hotplug
CONFIG_DEBUG_HOTPLUG_CPU0 is for debugging the CPU0 hotplug feature. The switch offlines CPU0 as soon as possible and boots userspace up with CPU0 offlined. User can online CPU0 back after boot time. The default value of the switch is off.
To debug CPU0 hotplug, you need to enable CPU0 offline/online feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during compilation or giving cpu0_hotplug kernel parameter at boot.
It's safe and early place to take down CPU0 after all hotplug notifiers are installed and SMP is booted.
Please note that some applications or drivers, e.g. some versions of udevd, during boot time may put CPU0 online again in this CPU0 hotplug debug mode.
In this debug mode, setup_local_APIC() may report a warning on max_loops<=0 when CPU0 is onlined back after boot time. This is because pending interrupt in IRR can not move to ISR. The warning is not CPU0 specfic and it can happen on other CPUs as well. It is harmless except the first CPU0 online takes a bit longer time. And so this debug mode is useful to expose this issue. I'll send a seperate patch to fix this generic warning issue.
--- arch/x86/Kconfig | 15 ++++++++++++++ arch/x86/include/asm/cpu.h | 3 +++ arch/x86/kernel/topology.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/power/cpu.c | 38 ++++++++++++++++++++++++++++++++++ 4 files changed, 107 insertions(+)
arch/x86/Kconfig +15 -0
@@ -1727,6 +1727,21 @@ config BOOTPARAM_HOTPLUG_CPU0 You still can enable the CPU0 hotplug feature at boot by kernel parameter cpu0_hotplug. +config DEBUG_HOTPLUG_CPU0 + def_bool n + prompt "Debug CPU0 hotplug" + depends on HOTPLUG_CPU && EXPERIMENTAL + ---help--- + Enabling this option offlines CPU0 (if CPU0 can be offlined) as + soon as possible and boots up userspace with CPU0 offlined. User + can online CPU0 back after boot time. + + To debug CPU0 hotplug, you need to enable CPU0 offline/online + feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during + compilation or giving cpu0_hotplug kernel parameter at boot. + + If unsure, say N. + config COMPAT_VDSO def_bool y prompt "Compat VDSO support"
arch/x86/include/asm/cpu.h +3 -0
@@ -29,6 +29,9 @@ struct x86_cpu {extern int arch_register_cpu (int num);extern void arch_unregister_cpu (int );extern void __cpuinit start_cpu0 (void ); +#ifdef CONFIG_DEBUG_HOTPLUG_CPU0 +extern int _debug_hotplug_cpu (int cpu,int action); +#endif #endif DECLARE_PER_CPU (int, cpu_state);
arch/x86/kernel/topology.c +51 -0
57 lines not on this page · raw message
arch/x86/power/cpu.c +38 -0
@@ -21,6 +21,7 @@#include <asm/suspend.h> #include <asm/debugreg.h> #include <asm/fpu-internal.h> /* pcntxt_mask */ +#include <asm/cpu.h> #ifdef CONFIG_X86_32 static struct saved_context saved_context;
@@ -263,6 +264,43 @@ static int bsp_pm_callback(struct notifier_block *nb, unsigned long action,case PM_HIBERNATION_PREPARE : ret =bsp_check ();break ; +#ifdef CONFIG_DEBUG_HOTPLUG_CPU0 +case PM_RESTORE_PREPARE : +/* +* When system resumes from hibernation, online CPU0 because +* 1. it's required for resume and +* 2. the CPU was online before hibernation +*/ +if (!cpu_online (0 )) +_debug_hotplug_cpu (0 ,1 ); +break ; +case PM_POST_RESTORE : +/* +* When a resume really happens, this code won't be called. +* +* This code is called only when user space hibernation software +* prepares for snapshot device during boot time. So we just +* call _debug_hotplug_cpu() to restore to CPU0's state prior to +* preparing the snapshot device. +* +* This works for normal boot case in our CPU0 hotplug debug +* mode, i.e. CPU0 is offline and user mode hibernation +* software initializes during boot time. +* +* If CPU0 is online and user application accesses snapshot +* device after boot time, this will offline CPU0 and user may +* see different CPU0 state before and after accessing +* the snapshot device. But hopefully this is not a case when +* user debugging CPU0 hotplug. Even if users hit this case, +* they can easily online CPU0 back. +* +* To simplify this debug code, we only consider normal boot +* case. Otherwise we need to remember CPU0's state and restore +* to that state and resolve racy conditions etc. +*/ +_debug_hotplug_cpu (0 ,0 ); +break ; +#endif default :break ; }
[tip:x86/bsp-hotplug] x86, topology: Debug CPU0 hotplug
In reply to Fenghua YuCommit-ID: a71c8bc5dfefbbf80ef90739791554ef7ea4401b Gitweb: http://git.kernel.org/tip/a71c8bc5dfefbbf80ef90739791554ef7ea4401b Author: Fenghua Yu <[email protected]> AuthorDate: Tue, 13 Nov 2012 11:32:51 -0800 Committer: H. Peter Anvin <[email protected]> CommitDate: Wed, 14 Nov 2012 15:28:11 -0800
x86, topology: Debug CPU0 hotplug
CONFIG_DEBUG_HOTPLUG_CPU0 is for debugging the CPU0 hotplug feature. The switch offlines CPU0 as soon as possible and boots userspace up with CPU0 offlined. User can online CPU0 back after boot time. The default value of the switch is off.
To debug CPU0 hotplug, you need to enable CPU0 offline/online feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during compilation or giving cpu0_hotplug kernel parameter at boot.
It's safe and early place to take down CPU0 after all hotplug notifiers are installed and SMP is booted.
Please note that some applications or drivers, e.g. some versions of udevd, during boot time may put CPU0 online again in this CPU0 hotplug debug mode.
In this debug mode, setup_local_APIC() may report a warning on max_loops<=0 when CPU0 is onlined back after boot time. This is because pending interrupt in IRR can not move to ISR. The warning is not CPU0 specfic and it can happen on other CPUs as well. It is harmless except the first CPU0 online takes a bit longer time. And so this debug mode is useful to expose this issue. I'll send a seperate patch to fix this generic warning issue.
--- arch/x86/Kconfig | 15 ++++++++++++++ arch/x86/include/asm/cpu.h | 3 +++ arch/x86/kernel/topology.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++ arch/x86/power/cpu.c | 38 ++++++++++++++++++++++++++++++++++ 4 files changed, 107 insertions(+)
arch/x86/Kconfig +15 -0
@@ -1727,6 +1727,21 @@ config BOOTPARAM_HOTPLUG_CPU0 You still can enable the CPU0 hotplug feature at boot by kernel parameter cpu0_hotplug. +config DEBUG_HOTPLUG_CPU0 + def_bool n + prompt "Debug CPU0 hotplug" + depends on HOTPLUG_CPU && EXPERIMENTAL + ---help--- + Enabling this option offlines CPU0 (if CPU0 can be offlined) as + soon as possible and boots up userspace with CPU0 offlined. User + can online CPU0 back after boot time. + + To debug CPU0 hotplug, you need to enable CPU0 offline/online + feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during + compilation or giving cpu0_hotplug kernel parameter at boot. + + If unsure, say N. + config COMPAT_VDSO def_bool y prompt "Compat VDSO support"
arch/x86/include/asm/cpu.h +3 -0
@@ -29,6 +29,9 @@ struct x86_cpu {extern int arch_register_cpu (int num);extern void arch_unregister_cpu (int );extern void __cpuinit start_cpu0 (void ); +#ifdef CONFIG_DEBUG_HOTPLUG_CPU0 +extern int _debug_hotplug_cpu (int cpu,int action); +#endif #endif DECLARE_PER_CPU (int, cpu_state);
arch/x86/kernel/topology.c +51 -0
57 lines not on this page · raw message
arch/x86/power/cpu.c +38 -0
@@ -21,6 +21,7 @@#include <asm/suspend.h> #include <asm/debugreg.h> #include <asm/fpu-internal.h> /* pcntxt_mask */ +#include <asm/cpu.h> #ifdef CONFIG_X86_32 static struct saved_context saved_context;
@@ -263,6 +264,43 @@ static int bsp_pm_callback(struct notifier_block *nb, unsigned long action,case PM_HIBERNATION_PREPARE : ret =bsp_check ();break ; +#ifdef CONFIG_DEBUG_HOTPLUG_CPU0 +case PM_RESTORE_PREPARE : +/* +* When system resumes from hibernation, online CPU0 because +* 1. it's required for resume and +* 2. the CPU was online before hibernation +*/ +if (!cpu_online (0 )) +_debug_hotplug_cpu (0 ,1 ); +break ; +case PM_POST_RESTORE : +/* +* When a resume really happens, this code won't be called. +* +* This code is called only when user space hibernation software +* prepares for snapshot device during boot time. So we just +* call _debug_hotplug_cpu() to restore to CPU0's state prior to +* preparing the snapshot device. +* +* This works for normal boot case in our CPU0 hotplug debug +* mode, i.e. CPU0 is offline and user mode hibernation +* software initializes during boot time. +* +* If CPU0 is online and user application accesses snapshot +* device after boot time, this will offline CPU0 and user may +* see different CPU0 state before and after accessing +* the snapshot device. But hopefully this is not a case when +* user debugging CPU0 hotplug. Even if users hit this case, +* they can easily online CPU0 back. +* +* To simplify this debug code, we only consider normal boot +* case. Otherwise we need to remember CPU0's state and restore +* to that state and resolve racy conditions etc. +*/ +_debug_hotplug_cpu (0 ,0 ); +break ; +#endif default :break ; }