Skip to content

cpu/hotplug, x86: Reworked parallel CPU bringup

Thomas Gleixner linux-arm-kernel linux-csky linux-mips linux-parisc linux-riscv lkml xen-devel 5 versions – Applied

Status from this message: The following commit has been merged into the smp/core branch of tip:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest37 patches 15 repliesApplied
  2. v2 posting 2/210 of 38 patches 5 repliesSuperseded
  3. v326 of 36 patches 27 repliesApplied
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v2 posting 1/238 patches5 repliesSuperseded
v137 patches41 repliesSuperseded

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/37x86/smpboot: Cleanup topology_phys_to_logical_pkg()/die()arch/x86/include/asm/topology.h arch/x86/kernel/smpboot.c+6 -70 replies
2/37cpu/hotplug: Mark arch_disable_smp_support() and bringup_nonboot_cpus() __initarch/x86/kernel/smpboot.c kernel/cpu.c kernel/smp.c+4 -40 replies
3/37x86/smpboot: Avoid pointless delay calibration if TSC is synchronizedarch/x86/kernel/smpboot.c arch/x86/kernel/tsc.c+41 -190 replies
4/37x86/smpboot: Rename start_cpu0() to soft_restart_cpu()arch/x86/include/asm/cpu.h arch/x86/kernel/callthunks.c arch/x86/kernel/head_32.S +2 more+13 -131 reply
5/37x86/topology: Remove CPU0 hotplug optionDocumentation/admin-guide/kernel-parameters.txt Documentation/core-api/cpu_hotplug.rst arch/x86/Kconfig +3 more+6 -2020 replies
6/37x86/smpboot: Remove the CPU0 hotplug kludgearch/x86/include/asm/apic.h arch/x86/include/asm/smp.h arch/x86/kernel/smpboot.c +1 more+14 -1620 replies
7/37x86/smpboot: Restrict soft_restart_cpu() to SEVarch/x86/kernel/callthunks.c arch/x86/kernel/head_32.S arch/x86/kernel/head_64.S+2 -161 reply
8/37x86/smpboot: Remove unnecessary barrier()arch/x86/kernel/smpboot.c+0 -20 replies
9/37x86/smpboot: Split up native_cpu_up() into separate phases and document themarch/x86/kernel/smpboot.c+119 -650 replies
10/37x86/smpboot: Get rid of cpu_init_secondary()arch/x86/include/asm/processor.h arch/x86/kernel/cpu/common.c arch/x86/kernel/smpboot.c+19 -331 reply
11/37x86/cpu/cacheinfo: Remove cpu_callout_mask dependencyarch/x86/kernel/cpu/cacheinfo.c+17 -40 replies
12/37x86/smpboot: Move synchronization masks to SMP boot codearch/x86/include/asm/cpumask.h arch/x86/kernel/cpu/common.c arch/x86/kernel/smpboot.c +1 more+16 -250 replies
13/37x86/smpboot: Make TSC synchronization function call basedarch/x86/include/asm/tsc.h arch/x86/kernel/smpboot.c arch/x86/kernel/tsc_sync.c+14 -440 replies
14/37x86/smpboot: Remove cpu_callin_maskarch/x86/kernel/smpboot.c+15 -590 replies
15/37cpu/hotplug: Rework sparse_irq locking in bringup_cpu()kernel/cpu.c+24 -100 replies
16/37x86/smpboot: Remove wait for cpu_online()arch/x86/kernel/smpboot.c+2 -240 replies
17/37x86/xen/smp_pv: Remove wait for CPU onlinearch/x86/xen/smp_pv.c+5 -50 replies
18/37x86/xen/hvm: Get rid of DEAD_FROZEN handlingarch/x86/xen/enlighten_hvm.c+5 -60 replies
19/37cpu/hotplug: Add CPU state tracking and synchronizationarch/Kconfig include/linux/cpuhotplug.h kernel/cpu.c +1 more+221 -10 replies
20/37x86/smpboot: Switch to hotplug core state synchronizationarch/x86/Kconfig arch/x86/include/asm/smp.h arch/x86/kernel/smp.c +3 more+75 -1540 replies
21/37cpu/hotplug: Remove cpu_report_state() and related unused cruftinclude/linux/cpu.h kernel/smpboot.c+0 -920 replies
22/37ARM: smp: Switch to hotplug core state synchronizationarch/arm/Kconfig arch/arm/include/asm/smp.h arch/arm/kernel/smp.c+9 -120 replies
23/37arm64: smp: Switch to hotplug core state synchronizationarch/arm64/Kconfig arch/arm64/include/asm/smp.h arch/arm64/kernel/smp.c+7 -100 replies
24/37csky/smp: Switch to hotplug core state synchronizationarch/csky/Kconfig arch/csky/include/asm/smp.h arch/csky/kernel/smp.c+4 -70 replies
25/37MIPS: SMP_CPS: Switch to hotplug core state synchronizationarch/mips/Kconfig arch/mips/cavium-octeon/smp.c arch/mips/include/asm/smp-ops.h +4 more+18 -90 replies
26/37parisc: Switch to hotplug core state synchronizationarch/parisc/Kconfig arch/parisc/kernel/process.c arch/parisc/kernel/smp.c+6 -60 replies
27/37riscv: Switch to hotplug core state synchronizationarch/riscv/Kconfig arch/riscv/include/asm/smp.h arch/riscv/kernel/cpu-hotplug.c+9 -80 replies
28/37cpu/hotplug: Remove unused state functionsinclude/linux/cpu.h kernel/smpboot.c+0 -770 replies
29/37cpu/hotplug: Reset task stack state in _cpu_up()kernel/cpu.c+6 -60 replies
30/37cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanismarch/Kconfig include/linux/cpuhotplug.h kernel/cpu.c+76 -20 replies
31/37x86/smpboot: Enable split CPU startuparch/x86/Kconfig arch/x86/include/asm/smp.h arch/x86/kernel/smp.c +2 more+10 -150 replies
32/37x86/apic: Provide cpu_primary_thread maskarch/x86/include/asm/apic.h arch/x86/include/asm/topology.h arch/x86/kernel/apic/apic.c +1 more+27 -260 replies
33/37cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATEDocumentation/admin-guide/kernel-parameters.txt arch/Kconfig include/linux/cpuhotplug.h +1 more+109 -55 replies
34/37x86/apic: Save the APIC virtual base addressarch/x86/include/asm/smp.h arch/x86/kernel/apic/apic.c+5 -00 replies
35/37x86/smpboot: Implement a bit spinlock to protect the realmode stackarch/x86/include/asm/realmode.h arch/x86/kernel/head_64.S arch/x86/realmode/init.c +1 more+36 -50 replies
36/37x86/smpboot: Support parallel startup of secondary CPUsarch/x86/include/asm/apic.h arch/x86/include/asm/apicdef.h arch/x86/include/asm/smp.h +4 more+83 -43 replies
37/37x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable itarch/x86/Kconfig arch/x86/kernel/cpu/common.c arch/x86/kernel/smpboot.c+75 -211 reply

Reviews trailers per patch

5 of 37 patches have a Reviewed-by or Acked-by.

PatchMichael KelleyPalmer DabbeltMark RutlandDavid WoodhouseOleksandr NatalenkoHelge DellerGuilherme G. PiccoliJeffrey HugoMark BrownPhilippe Mathieu-Daudé
SeriesT-b T-b T-b T-b
1/37T-b
2/37T-b
3/37T-b
4/37T-b R-b
5/37T-b
6/37T-b
7/37T-b R-b
8/37
9/37T-b
10/37T-b R-b
11/37T-b
12/37T-b
13/37T-b
14/37T-b
15/37T-b
16/37T-b
17/37T-b
18/37T-b
19/37T-b
20/37T-b
21/37T-b
22/37T-b
23/37T-b T-b
24/37T-b
25/37T-b
26/37T-b
27/37T-b A-b
28/37T-b
29/37T-b T-b R-b
30/37T-b
31/37T-b
32/37T-b
33/37T-b T-b
34/37T-b
35/37T-b
36/37T-b T-b
37/37T-b Originally-by

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 10 reviewers.

Read the v4 thread

Cover letter [patch V4 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup

Hi!

This is version 4 of the reworked parallel bringup series. Version 3 can be found here:

https://lore.kernel.org/lkml/[email protected]

This is just a reiteration to address the following details:

1) Address review feedback (Peter Zijlstra)

2) Fix a MIPS related build problem (0day)

Other than that there are no changes and the other details are all the same as in V3 and V2.

It's also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug

Diff to V3 below.

Thanks,

tglx
---

arch/mips/kernel/smp.c +1 -1
index f5e0f4235746..90c71d800b59 100644
@@ -690,7 +690,7 @@ void flush_tlb_one(unsigned long vaddr)
 EXPORT_SYMBOL(flush_tlb_page);
 EXPORT_SYMBOL(flush_tlb_one);
 
-#ifdef CONFIG_HOTPLUG_CPU
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
 void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
 {
 	if (mp_ops->cleanup_dead_cpu)
arch/x86/kernel/head_64.S +2 -4
index 0438802031c3..9cd77d319555 100644
@@ -290,8 +290,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL)
 
 	/*  APIC ID not found in the table. Drop the trampoline lock and bail. */
 	movq	trampoline_lock(%rip), %rax
-	lock
-	btrl	$0, (%rax)
+	movl	$0, (%rax)
 
 1:	cli
 	hlt
@@ -320,8 +319,7 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL)
 	movq	trampoline_lock(%rip), %rax
 	testq	%rax, %rax
 	jz	.Lsetup_gdt
-	lock
-	btrl	$0, (%rax)
+	movl	$0, (%rax)
 
 .Lsetup_gdt:
 	/*
arch/x86/kernel/smpboot.c +10 -16
index 5caf4897b507..660709e94823 100644
@@ -161,31 +161,28 @@ static inline void smpboot_restore_warm_reset_vector(void)
 
 }
 
-/*
- * Report back to the Boot Processor during boot time or to the caller processor
- * during CPU online.
- */
-static void smp_callin(void)
+/* Run the next set of setup steps for the upcoming CPU */
+static void ap_starting(void)
 {
 	int cpuid = smp_processor_id();
 
 	/*
-	 * If waken up by an INIT in an 82489DX configuration the alive
-	 * synchronization guarantees we don't get here before an
-	 * INIT_deassert IPI reaches our local APIC, so it is now safe to
-	 * touch our local APIC.
+	 * If woken up by an INIT in an 82489DX configuration the alive
+	 * synchronization guarantees that the CPU does not reach this
+	 * point before an INIT_deassert IPI reaches the local APIC, so it
+	 * is now safe to touch the local APIC.
 	 *
 	 * Set up this CPU, first the APIC, which is probably redundant on
 	 * most boards.
 	 */
 	apic_ap_setup();
 
-	/* Save our processor parameters. */
+	/* Save the processor parameters. */
 	smp_store_cpu_info(cpuid);
 
 	/*
 	 * The topology information must be up to date before
-	 * calibrate_delay() and notify_cpu_starting().
+	 * notify_cpu_starting().
 	 */
 	set_cpu_sibling_map(cpuid);
 
@@ -197,7 +194,7 @@ static void smp_callin(void)
 
 	/*
 	 * This runs the AP through all the cpuhp states to its target
-	 * state (CPUHP_ONLINE in the case of serial bringup).
+	 * state CPUHP_ONLINE.
 	 */
 	notify_cpu_starting(cpuid);
 }
@@ -274,10 +271,7 @@ static void notrace start_secondary(void *unused)
 	rcu_cpu_starting(raw_smp_processor_id());
 	x86_cpuinit.early_percpu_clock_init();
 
-	smp_callin();
-
-	/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
-	barrier();
+	ap_starting();
 
 	/* Check TSC synchronization with the control CPU. */
 	check_tsc_sync_target();
arch/x86/realmode/rm/trampoline_64.S +5 -9
index 2dfb1c400167..c6de4deec746 100644
@@ -40,17 +40,13 @@
 .macro LOAD_REALMODE_ESP
 	/*
 	 * Make sure only one CPU fiddles with the realmode stack
-	 */
+	*/
 .Llock_rm\@:
-	btl	$0, tr_lock
-	jnc	2f
-	pause
-	jmp	.Llock_rm\@
+        lock btsl       $0, tr_lock
+        jnc             2f
+        pause
+        jmp             .Llock_rm\@
 2:
-	lock
-	btsl	$0, tr_lock
-	jc	.Llock_rm\@
-
 	# Setup stack
 	movl	$rm_stack_end, %esp
 .endm
kernel/cpu.c +8 -4
index 60b4093fae9e..005f863a3d2b 100644
@@ -294,14 +294,14 @@ enum cpuhp_sync_state {
  * cpuhp_ap_update_sync_state - Update synchronization state during bringup/teardown
  * @state:	The synchronization state to set
  *
- * No synchronization point. Just update of the synchronization state.
+ * No synchronization point. Just update of the synchronization state, but implies
+ * a full barrier so that the AP changes are visible before the control CPU proceeds.
  */
 static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state)
 {
 	atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
-	int sync = atomic_read(st);
 
-	while (!atomic_try_cmpxchg(st, &sync, state));
+	(void)atomic_xchg(st, state);
 }
 
 void __weak arch_cpuhp_sync_state_poll(void) { cpu_relax(); }
@@ -829,7 +829,11 @@ static int bringup_cpu(unsigned int cpu)
 	/*
 	 * Some architectures have to walk the irq descriptors to
 	 * setup the vector space for the cpu which comes online.
-	 * Prevent irq alloc/free across the bringup.
+	 *
+	 * Prevent irq alloc/free across the bringup by acquiring the
+	 * sparse irq lock. Hold it until the upcoming CPU completes the
+	 * startup in cpuhp_online_idle() which allows to avoid
+	 * intermediate synchronization points in the architecture code.
 	 */
 	irq_lock_sparse();