x86/topology: Cure fallout
cover x86/topology: Cure fallout
Guenter reported a crash on UP which was caused by the per CPU rework and this unearthed a few other issues in the new topology evaluation code:
https://lore.kernel.org/all/[email protected]/
The following series addresses it. It is based on Linus tree and also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git x86/core
Thanks,
tglx
--- arch/x86/kernel/cpu/common.c | 9 +++++++++ arch/x86/kernel/cpu/topology.c | 11 +++++++++++ arch/x86/kernel/cpu/topology_common.c | 12 +++++++----- arch/x86/kernel/mpparse.c | 10 +++++----- arch/x86/kernel/setup.c | 10 ---------- arch/x86/kernel/smpboot.c | 32 +++++--------------------------- 6 files changed, 37 insertions(+), 47 deletions(-)
PATCH 1/4 x86/cpu: Ensure that CPU info updates are propagated on UP
The boot sequence evaluates CPUID information twice:
1) During early boot
2) When finalizing the early setup right before mitigations are selected and alternatives are patched.
In both cases the evaluation is stored in boot_cpu_data, but on UP the copying of boot_cpu_data to the per CPU info of the boot CPU happens between #1 and #2. So any update which happens in #2 is never propagated to the per CPU info instance.
Consolidate the whole logic and copy boot_cpu_data right before applying alternatives as that's the point where boot_cpu_data is in it's final state and not supposed to change anymore.
This also removes the voodoo mb() from smp_prepare_cpus_common() which had absolutely no purpose.
--- arch/x86/kernel/cpu/common.c | 9 +++++++++ arch/x86/kernel/setup.c | 10 ---------- arch/x86/kernel/smpboot.c | 32 +++++--------------------------- 3 files changed, 14 insertions(+), 37 deletions(-)
arch/x86/kernel/cpu/common.c +9 -0
@@ -2307,6 +2307,8 @@ void arch_smt_update(void)void __init arch_cpu_finalize_init (void ) { +struct cpuinfo_x86 *c =this_cpu_ptr (&cpu_info); +identify_boot_cpu ();select_idle_routine ();
@@ -2345,6 +2347,13 @@ void __init arch_cpu_finalize_init(void)fpu__init_system ();fpu__init_cpu (); +/* +* Ensure that access to the per CPU representation has the initial +* boot CPU configuration. +*/ + *c = boot_cpu_data; + c->initialized = true; +alternative_instructions ();if (IS_ENABLED (CONFIG_X86_64 )) {
arch/x86/kernel/setup.c +0 -10
@@ -1206,16 +1206,6 @@ void __init i386_reserve_resources(void)#endif /* CONFIG_X86_32 */ -#ifndef CONFIG_SMP -void __init smp_prepare_boot_cpu (void ) -{ -struct cpuinfo_x86 *c = &cpu_data (0 ); - - *c = boot_cpu_data; - c->initialized = true; -} -#endif -static struct notifier_block kernel_offset_notifier = { .notifier_call = dump_kernel_offset };
arch/x86/kernel/smpboot.c +5 -27
@@ -313,14 +313,6 @@ static void notrace start_secondary(voidcpu_startup_entry (CPUHP_AP_ONLINE_IDLE ); } -static void __init smp_store_boot_cpu_info (void ) -{ -struct cpuinfo_x86 *c = &cpu_data (0 ); - - *c = boot_cpu_data; - c->initialized = true; -} -/* * The bootstrap kernel entry code has set these up. Save them for * a given CPU
@@ -1039,29 +1031,15 @@ static __init void disable_smp(void)cpumask_set_cpu (0 ,topology_die_cpumask (0 )); } -static void __init smp_cpu_index_default (void ) -{ -int i; -struct cpuinfo_x86 *c; - -for_each_possible_cpu (i) { - c = &cpu_data (i); -/* mark all to hotplug */ - c->cpu_index = nr_cpu_ids; - } -} -void __init smp_prepare_cpus_common (void ) {unsigned int i; -smp_cpu_index_default (); - -/* -* Setup boot CPU information -*/ -smp_store_boot_cpu_info ();/* Final full version of the data */ -mb (); +/* Mark all except the boot CPU as hotpluggable */ +for_each_possible_cpu (i) { +if (i) +per_cpu (cpu_info.cpu_index, i) = nr_cpu_ids; + }for_each_possible_cpu (i) {zalloc_cpumask_var (&per_cpu (cpu_sibling_map, i),GFP_KERNEL );
Re: [patch 1/4] x86/cpu: Ensure that CPU info updates are propagated on UP
Quoted 16 lines · Thomas Gleixner
The boot sequence evaluates CPUID information twice:
1) During early boot
2) When finalizing the early setup right before mitigations are selected and alternatives are patched.
In both cases the evaluation is stored in
boot_cpu_data, but on UP the copying ofboot_cpu_datato the per CPU info of the boot CPU happens between #1 and #2. So any update which happens in #2 is never propagated to the per CPU info instance.Consolidate the whole logic and copy
boot_cpu_dataright before applying alternatives as that's the point whereboot_cpu_datais in it's final state and not supposed to change anymore.
Quoted 6 lines
This also removes the voodoo
mb()fromsmp_prepare_cpus_common()which had absolutely no purpose.Fixes 71eb4893cfaf ("x86/percpu: Cure per CPU madness on UP")Reported-by Guenter Roeck <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>
Guenter
[tip: x86/urgent] x86/cpu: Ensure that CPU info updates are propagated on UP
In reply to Thomas GleixnerThe following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: c90399fbd74a0713d5972a6d931e4a9918621e88 Gitweb: https://git.kernel.org/tip/c90399fbd74a0713d5972a6d931e4a9918621e88 Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 22 Mar 2024 19:56:35 +01:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sat, 23 Mar 2024 12:22:04 +01:00
x86/cpu: Ensure that CPU info updates are propagated on UP
The boot sequence evaluates CPUID information twice:
1) During early boot
2) When finalizing the early setup right before mitigations are selected and alternatives are patched.
In both cases the evaluation is stored in boot_cpu_data, but on UP the copying of boot_cpu_data to the per CPU info of the boot CPU happens between #1 and #2. So any update which happens in #2 is never propagated to the per CPU info instance.
Consolidate the whole logic and copy boot_cpu_data right before applying alternatives as that's the point where boot_cpu_data is in it's final state and not supposed to change anymore.
This also removes the voodoo mb() from smp_prepare_cpus_common() which had absolutely no purpose.
--- arch/x86/kernel/cpu/common.c | 9 +++++++++ arch/x86/kernel/setup.c | 10 ---------- arch/x86/kernel/smpboot.c | 32 +++++--------------------------- 3 files changed, 14 insertions(+), 37 deletions(-)
arch/x86/kernel/cpu/common.c +9 -0
@@ -2307,6 +2307,8 @@ void arch_smt_update(void)void __init arch_cpu_finalize_init (void ) { +struct cpuinfo_x86 *c =this_cpu_ptr (&cpu_info); +identify_boot_cpu ();select_idle_routine ();
@@ -2345,6 +2347,13 @@ void __init arch_cpu_finalize_init(void)fpu__init_system ();fpu__init_cpu (); +/* +* Ensure that access to the per CPU representation has the initial +* boot CPU configuration. +*/ + *c = boot_cpu_data; + c->initialized = true; +alternative_instructions ();if (IS_ENABLED (CONFIG_X86_64 )) {
arch/x86/kernel/setup.c +0 -10
@@ -1206,16 +1206,6 @@ void __init i386_reserve_resources(void)#endif /* CONFIG_X86_32 */ -#ifndef CONFIG_SMP -void __init smp_prepare_boot_cpu (void ) -{ -struct cpuinfo_x86 *c = &cpu_data (0 ); - - *c = boot_cpu_data; - c->initialized = true; -} -#endif -static struct notifier_block kernel_offset_notifier = { .notifier_call = dump_kernel_offset };
arch/x86/kernel/smpboot.c +5 -27
@@ -313,14 +313,6 @@ static void notrace start_secondary(void *unused)cpu_startup_entry (CPUHP_AP_ONLINE_IDLE ); } -static void __init smp_store_boot_cpu_info (void ) -{ -struct cpuinfo_x86 *c = &cpu_data (0 ); - - *c = boot_cpu_data; - c->initialized = true; -} -/* * The bootstrap kernel entry code has set these up. Save them for * a given CPU
@@ -1039,29 +1031,15 @@ static __init void disable_smp(void)cpumask_set_cpu (0 ,topology_die_cpumask (0 )); } -static void __init smp_cpu_index_default (void ) -{ -int i; -struct cpuinfo_x86 *c; - -for_each_possible_cpu (i) { - c = &cpu_data (i); -/* mark all to hotplug */ - c->cpu_index = nr_cpu_ids; - } -} -void __init smp_prepare_cpus_common (void ) {unsigned int i; -smp_cpu_index_default (); - -/* -* Setup boot CPU information -*/ -smp_store_boot_cpu_info ();/* Final full version of the data */ -mb (); +/* Mark all except the boot CPU as hotpluggable */ +for_each_possible_cpu (i) { +if (i) +per_cpu (cpu_info.cpu_index, i) = nr_cpu_ids; + }for_each_possible_cpu (i) {zalloc_cpumask_var (&per_cpu (cpu_sibling_map, i),GFP_KERNEL );
PATCH 2/4 x86/topology: Dont evaluate logical IDs during early boot
In reply to Thomas GleixnerThe local APICs have not yet been enumerated so the logical ID evaluation from the topology bitmaps does not work and would return an error code.
Skip the evaluation during the early boot CPUID evaluation and only apply it on the final run.
--- arch/x86/kernel/cpu/topology_common.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/topology_common.c +7 -5
@@ -140,7 +140,7 @@ static void parse_topology(struct topo_s } } -static void topo_set_ids (struct topo_scan *tscan) +static void topo_set_ids (struct topo_scan *tscan,bool early) {struct cpuinfo_x86 *c = tscan->c;u32 apicid = c->topo.apicid;
@@ -148,8 +148,10 @@ static void topo_set_ids(struct topo_sca c->topo.pkg_id =topo_shift_apicid (apicid,TOPO_PKG_DOMAIN ); c->topo.die_id =topo_shift_apicid (apicid,TOPO_DIE_DOMAIN ); - c->topo.logical_pkg_id =topology_get_logical_id (apicid,TOPO_PKG_DOMAIN ); - c->topo.logical_die_id =topology_get_logical_id (apicid,TOPO_DIE_DOMAIN ); +if (!early) { + c->topo.logical_pkg_id =topology_get_logical_id (apicid,TOPO_PKG_DOMAIN ); + c->topo.logical_die_id =topology_get_logical_id (apicid,TOPO_DIE_DOMAIN ); + }/* Package relative core ID */ c->topo.core_id = (apicid & topo_domain_mask(TOPO_PKG_DOMAIN)) >>
@@ -187,7 +189,7 @@ void cpu_parse_topology(struct cpuinfo_x tscan.dom_shifts[dom], x86_topo_system.dom_shifts[dom]); } -topo_set_ids (&tscan); +topo_set_ids (&tscan, false); }void __init cpu_init_topology (struct cpuinfo_x86 *c)
@@ -208,7 +210,7 @@ void __init cpu_init_topology(struct cpu x86_topo_system.dom_size[dom] =1U << sft; } -topo_set_ids (&tscan); +topo_set_ids (&tscan, true);/* * AMD systems have Nodes per package which cannot be mapped to
Re: [patch 2/4] x86/topology: Dont evaluate logical IDs during early boot
Quoted 8 lines · Thomas Gleixner
The local APICs have not yet been enumerated so the logical ID evaluation from the topology bitmaps does not work and would return an error code.
Skip the evaluation during the early boot CPUID evaluation and only apply it on the final run.
Fixes 380414be78bf ("x86/cpu/topology: Use topology logical mapping mechanism")Signed-off-by Thomas Gleixner <[email protected]>
Guenter
[tip: x86/urgent] x86/topology: Don't evaluate logical IDs during early boot
In reply to Thomas GleixnerThe following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 7af541cee1e0eb48c6eb439bc6309175339fa96f Gitweb: https://git.kernel.org/tip/7af541cee1e0eb48c6eb439bc6309175339fa96f Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 22 Mar 2024 19:56:36 +01:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sat, 23 Mar 2024 12:28:06 +01:00
x86/topology: Don't evaluate logical IDs during early boot
The local APICs have not yet been enumerated so the logical ID evaluation from the topology bitmaps does not work and would return an error code.
Skip the evaluation during the early boot CPUID evaluation and only apply it on the final run.
--- arch/x86/kernel/cpu/topology_common.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/topology_common.c +7 -5
@@ -140,7 +140,7 @@ static void parse_topology(struct topo_scan *tscan, bool early) } } -static void topo_set_ids (struct topo_scan *tscan) +static void topo_set_ids (struct topo_scan *tscan,bool early) {struct cpuinfo_x86 *c = tscan->c;u32 apicid = c->topo.apicid;
@@ -148,8 +148,10 @@ static void topo_set_ids(struct topo_scan *tscan) c->topo.pkg_id =topo_shift_apicid (apicid,TOPO_PKG_DOMAIN ); c->topo.die_id =topo_shift_apicid (apicid,TOPO_DIE_DOMAIN ); - c->topo.logical_pkg_id =topology_get_logical_id (apicid,TOPO_PKG_DOMAIN ); - c->topo.logical_die_id =topology_get_logical_id (apicid,TOPO_DIE_DOMAIN ); +if (!early) { + c->topo.logical_pkg_id =topology_get_logical_id (apicid,TOPO_PKG_DOMAIN ); + c->topo.logical_die_id =topology_get_logical_id (apicid,TOPO_DIE_DOMAIN ); + }/* Package relative core ID */ c->topo.core_id = (apicid & topo_domain_mask(TOPO_PKG_DOMAIN)) >>
@@ -187,7 +189,7 @@ void cpu_parse_topology(struct cpuinfo_x86 *c) tscan.dom_shifts[dom], x86_topo_system.dom_shifts[dom]); } -topo_set_ids (&tscan); +topo_set_ids (&tscan, false); }void __init cpu_init_topology (struct cpuinfo_x86 *c)
@@ -208,7 +210,7 @@ void __init cpu_init_topology(struct cpuinfo_x86 *c) x86_topo_system.dom_size[dom] =1U << sft; } -topo_set_ids (&tscan); +topo_set_ids (&tscan, true);/* * AMD systems have Nodes per package which cannot be mapped to
PATCH 3/4 x86/topology: Handle the !APIC case gracefully
In reply to Thomas GleixnerIf there is no local APIC enumerated and registered then the topology bitmaps are empty. Therefore topology_init_possible_cpus() will die with a division by zero exception.
Prevent this by registering a fake APIC id to populate the topology bitmap. This also allows to use all topology query interfaces unconditionally. It does not affect the actual APIC code because either the local APIC address was not registered or no local APIC could be detected.
--- arch/x86/kernel/cpu/topology.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/x86/kernel/cpu/topology.c +11 -0
@@ -415,6 +415,17 @@ void __init topology_init_possible_cpus(unsigned int total = assigned + disabled;u32 apicid, firstid; +/* +* If there was no APIC registered, then fake one so that the +* topology bitmap is populated. That ensures that the code below +* is valid and the various query interfaces can be used +* unconditionally. This does not affect the actual APIC code in +* any way because either the local APIC address has not been +* registered or the local APIC was disabled on the command line. +*/ +if (topo_info.boot_cpu_apic_id ==BAD_APICID ) +topology_register_boot_apic (0 ); +if (!restrict_to_up ()) {if (WARN_ON_ONCE (assigned > nr_cpu_ids)) { disabled += assigned - nr_cpu_ids;
Re: [patch 3/4] x86/topology: Handle the !APIC case gracefully
Quoted 9 lines · Thomas Gleixner
If there is no local APIC enumerated and registered then the topology bitmaps are empty. Therefore
topology_init_possible_cpus()will die with a division by zero exception.Prevent this by registering a fake APIC id to populate the topology bitmap. This also allows to use all topology query interfaces unconditionally. It does not affect the actual APIC code because either the local APIC address was not registered or no local APIC could be detected.
Quoted 4 lines
Fixes f1f758a80516 ("x86/topology: Add a mechanism to track topology via APIC IDs")Reported-by Guenter Roeck <[email protected]>Reported-by Linus Torvalds <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>
Guenter
[tip: x86/urgent] x86/topology: Handle the !APIC case gracefully
In reply to Thomas GleixnerThe following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 5e25eb25dae9fa0700bbe42aff0e2f105fcd096a Gitweb: https://git.kernel.org/tip/5e25eb25dae9fa0700bbe42aff0e2f105fcd096a Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 22 Mar 2024 19:56:38 +01:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sat, 23 Mar 2024 12:35:56 +01:00
x86/topology: Handle the !APIC case gracefully
If there is no local APIC enumerated and registered then the topology bitmaps are empty. Therefore, topology_init_possible_cpus() will die with a division by zero exception.
Prevent this by registering a fake APIC id to populate the topology bitmap. This also allows to use all topology query interfaces unconditionally. It does not affect the actual APIC code because either the local APIC address was not registered or no local APIC could be detected.
--- arch/x86/kernel/cpu/topology.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/x86/kernel/cpu/topology.c +11 -0
@@ -415,6 +415,17 @@ void __init topology_init_possible_cpus(void)unsigned int total = assigned + disabled;u32 apicid, firstid; +/* +* If there was no APIC registered, then fake one so that the +* topology bitmap is populated. That ensures that the code below +* is valid and the various query interfaces can be used +* unconditionally. This does not affect the actual APIC code in +* any way because either the local APIC address has not been +* registered or the local APIC was disabled on the command line. +*/ +if (topo_info.boot_cpu_apic_id ==BAD_APICID ) +topology_register_boot_apic (0 ); +if (!restrict_to_up ()) {if (WARN_ON_ONCE (assigned > nr_cpu_ids)) { disabled += assigned - nr_cpu_ids;
PATCH 4/4 x86/mpparse: Register APIC address only once
In reply to Thomas GleixnerThe APIC address is registered twice. First during the early detection and afterwards when actually scanning the table for APIC IDs. The APIC and topology core warn about the second attempt.
Restrict it to the early detection call.
--- arch/x86/kernel/mpparse.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kernel/mpparse.c +5 -5
@@ -197,12 +197,12 @@ static int __init smp_read_mpc(struct mpif (!smp_check_mpc (mpc, oem, str))return 0 ; -/* Initialize the lapic mapping */ -if (!acpi_lapic) -register_lapic_address (mpc->lapic); - -if (early) +if (early) { +/* Initialize the lapic mapping */ +if (!acpi_lapic) +register_lapic_address (mpc->lapic);return 1 ; + }/* Now process the configuration blocks. */ while (count < mpc->length) {
Re: [patch 4/4] x86/mpparse: Register APIC address only once
Quoted 8 lines · Thomas Gleixner
The APIC address is registered twice. First during the early detection and afterwards when actually scanning the table for APIC IDs. The APIC and topology core warn about the second attempt.
Restrict it to the early detection call.
Fixes 81287ad65da5 ("x86/apic: Sanitize APIC address setup")Signed-off-by Thomas Gleixner <[email protected]>
Guenter
[tip: x86/urgent] x86/mpparse: Register APIC address only once
In reply to Thomas GleixnerThe following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: f2208aa12c27bfada3c15c550c03ca81d42dcac2 Gitweb: https://git.kernel.org/tip/f2208aa12c27bfada3c15c550c03ca81d42dcac2 Author: Thomas Gleixner <[email protected]> AuthorDate: Fri, 22 Mar 2024 19:56:39 +01:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sat, 23 Mar 2024 12:41:48 +01:00
x86/mpparse: Register APIC address only once
The APIC address is registered twice. First during the early detection and afterwards when actually scanning the table for APIC IDs. The APIC and topology core warn about the second attempt.
Restrict it to the early detection call.
--- arch/x86/kernel/mpparse.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kernel/mpparse.c +5 -5
@@ -197,12 +197,12 @@ static int __init smp_read_mpc(struct mpc_table *mpc, unsigned early)if (!smp_check_mpc (mpc, oem, str))return 0 ; -/* Initialize the lapic mapping */ -if (!acpi_lapic) -register_lapic_address (mpc->lapic); - -if (early) +if (early) { +/* Initialize the lapic mapping */ +if (!acpi_lapic) +register_lapic_address (mpc->lapic);return 1 ; + }/* Now process the configuration blocks. */ while (count < mpc->length) {