x86/cpu/topology: don't write to immutable cpu_present_mask
PATCH 1/2 x86/cpu/topology: don't write to immutable cpu_present_mask
Workaround the following oops:
topology_hotplug_apic topo_set_cpuids set_cpu_possible(cpu, true); // write to __ro_after_init section after init
adobriyan: I'm not sure what's going on, can it set unset bit here? If not, then why does it repeat the job and set already set bits.
Anyhow, let's not oops peoples' machines for now.
--- arch/x86/kernel/cpu/topology.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/topology.c +9 -4
@@ -117,13 +117,18 @@ static __init int topo_get_cpunr(u32 apic_id)return topo_info.nr_assigned_cpus++; } -static void topo_set_cpuids (unsigned int cpu,u32 apic_id,u32 acpi_id) +static void topo_set_cpuids (unsigned int cpu,u32 apic_id,u32 acpi_id,bool from_init) {#if defined(CONFIG_SMP ) || defined(CONFIG_X86_64 )early_per_cpu (x86_cpu_to_apicid, cpu) = apic_id;early_per_cpu (x86_cpu_to_acpiid, cpu) = acpi_id;#endif -set_cpu_possible (cpu, true); +if (from_init) { +set_cpu_possible (cpu, true); + }else { +/* cpu_possible_mask is supposed to be fixed by now. */ +WARN_ON (!cpumask_test_cpu (cpu, cpu_possible_mask)); + }set_cpu_present (cpu, true); }
@@ -191,7 +196,7 @@ static __init void topo_register_apic(u32 apic_id, u32 acpi_id, bool present) cpu =topo_get_cpunr (apic_id); cpuid_to_apicid[cpu] = apic_id; -topo_set_cpuids (cpu, apic_id, acpi_id); +topo_set_cpuids (cpu, apic_id, acpi_id, true); }else {u32 pkgid =topo_apicid (apic_id,TOPO_PKG_DOMAIN );
@@ -343,7 +348,7 @@ int topology_hotplug_apic(u32 apic_id, u32 acpi_id)return -ENOSPC ;set_bit (apic_id, phys_cpu_present_map); -topo_set_cpuids (cpu, apic_id, acpi_id); +topo_set_cpuids (cpu, apic_id, acpi_id, false);cpu_mark_primary_thread (cpu, apic_id);return cpu; }
Signature · 1 line
2.43.2
Quoted 6 lines · Alexey Dobriyan
Workaround the following oops:
topology_hotplug_apic topo_set_cpuids set_cpu_possible(cpu, true); // write to __ro_after_init section after init
Duh, yes.
Quoted 4 lines
adobriyan: I'm not sure what's going on, can it set unset bit here? If not, then why does it repeat the job and set already set bits.
Anyhow, let's not oops peoples' machines for now.
Adding a bandaid to paper over the non-understood real problem is definitely not a good plan. I take this patch as a bug report.
Proper fix below.
Thanks,
tglx
---
Subject: x86/topology: Don't update cpu_possible_map in topo_set_cpuids()
From: Thomas Gleixner <[email protected]>
Date: Mon, 08 Apr 2024 15:22:01 +0200
topo_set_cpuids() updates cpu_present_map and cpu_possible map. It is invoked during enumeration and "physical hotplug" operations. In the latter case this results in a kernel crash because cpu_possible_map is marked read only after init completes.
There is no reason to update cpu_possible_map in that function. During enumeration cpu_possible_map is not relevant and gets fully initialized after enumeration completed. On "physical hotplug" the bit is already set because the kernel allows only CPUs to be plugged which have been enumerated and associated to a CPU number during early boot.
Remove the bogus update of cpu_possible_map.
--- arch/x86/kernel/cpu/topology.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/topology.c +5 -2
@@ -123,7 +123,6 @@ static void topo_set_cpuids(unsigned intearly_per_cpu (x86_cpu_to_apicid, cpu) = apic_id;early_per_cpu (x86_cpu_to_acpiid, cpu) = acpi_id;#endif -set_cpu_possible (cpu, true);set_cpu_present (cpu, true); }
@@ -210,7 +209,11 @@ static __init void topo_register_apic(u3 topo_info.nr_disabled_cpus++; } -/* Register present and possible CPUs in the domain maps */ +/* +* Register present and possible CPUs in the domain +* maps. cpu_possible_map will be updated in +* topology_init_possible_cpus() after enumeration is done. +*/ for (dom =TOPO_SMT_DOMAIN ; dom <TOPO_MAX_DOMAIN ; dom++)set_bit (topo_apicid (apic_id, dom), apic_maps[dom].map); }
Quoted 13 lines · Thomas Gleixner
Quoted 6 lines · Alexey Dobriyan
Workaround the following oops:
topology_hotplug_apic topo_set_cpuids set_cpu_possible(cpu, true); // write to __ro_after_init section after initDuh, yes.
Quoted 3 lines
adobriyan: I'm not sure what's going on, can it set unset bit here? If not, then why does it repeat the job and set already set bits.
Quoted 6 lines
Quoted 1 line
Anyhow, let's not oops peoples' machines for now.
Adding a bandaid to paper over the non-understood real problem is definitely not a good plan. I take this patch as a bug report.
Proper fix below.
BTW., independently of the fix, warning about a too late set_cpu_possible() might still make sense - clearly it *can* be called too late by architecture init code :-)
It's not a performance-sensitive function in any case, so the extra debugging code might not hurt.
Thanks,
Ingo
Quoted 18 lines · Ingo Molnar
Quoted 15 lines · Thomas Gleixner
Quoted 6 lines · Alexey Dobriyan
Workaround the following oops:
topology_hotplug_apic topo_set_cpuids set_cpu_possible(cpu, true); // write to __ro_after_init section after initDuh, yes.
Quoted 4 lines
adobriyan: I'm not sure what's going on, can it set unset bit here? If not, then why does it repeat the job and set already set bits.
Anyhow, let's not oops peoples' machines for now.
Quoted 8 lines
Quoted 4 lines
Adding a bandaid to paper over the non-understood real problem is definitely not a good plan. I take this patch as a bug report.
Proper fix below.
BTW., independently of the fix, warning about a too late
set_cpu_possible()might still make sense - clearly it *can* be called too late by architecture init code :-)
Make the function __init ? Then it goes away right when the data becomes RO.
[tip: x86/alternatives] x86/topology: Don't update cpu_possible_map in topo_set_cpuids()
In reply to Thomas GleixnerThe following commit has been merged into the x86/alternatives branch of tip:
The following commit has been merged into the x86/alternatives branch of tip:
Commit-ID: 675ae1aac0572595999e6f78a16e51351291f7aa Gitweb: https://git.kernel.org/tip/675ae1aac0572595999e6f78a16e51351291f7aa Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 08 Apr 2024 15:22:01 +02:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 10 Apr 2024 15:24:27 +02:00
x86/topology: Don't update cpu_possible_map in topo_set_cpuids()
topo_set_cpuids() updates cpu_present_map and cpu_possible map. It is invoked during enumeration and "physical hotplug" operations. In the latter case this results in a kernel crash because cpu_possible_map is marked read only after init completes.
There is no reason to update cpu_possible_map in that function. During enumeration cpu_possible_map is not relevant and gets fully initialized after enumeration completed. On "physical hotplug" the bit is already set because the kernel allows only CPUs to be plugged which have been enumerated and associated to a CPU number during early boot.
Remove the bogus update of cpu_possible_map.
--- arch/x86/kernel/cpu/topology.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/topology.c +5 -2
@@ -123,7 +123,6 @@ static void topo_set_cpuids(unsigned int cpu, u32 apic_id, u32 acpi_id)early_per_cpu (x86_cpu_to_apicid, cpu) = apic_id;early_per_cpu (x86_cpu_to_acpiid, cpu) = acpi_id;#endif -set_cpu_possible (cpu, true);set_cpu_present (cpu, true); }
@@ -210,7 +209,11 @@ static __init void topo_register_apic(u32 apic_id, u32 acpi_id, bool present) topo_info.nr_disabled_cpus++; } -/* Register present and possible CPUs in the domain maps */ +/* +* Register present and possible CPUs in the domain +* maps. cpu_possible_map will be updated in +* topology_init_possible_cpus() after enumeration is done. +*/ for (dom =TOPO_SMT_DOMAIN ; dom <TOPO_MAX_DOMAIN ; dom++)set_bit (topo_apicid (apic_id, dom), apic_maps[dom].map); }
[tip: x86/urgent] x86/topology: Don't update cpu_possible_map in topo_set_cpuids()
In reply to Thomas GleixnerThe following commit has been merged into the x86/urgent branch of tip:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: a9025cd1c673a8d6eefc79d911075b8b452eba8f Gitweb: https://git.kernel.org/tip/a9025cd1c673a8d6eefc79d911075b8b452eba8f Author: Thomas Gleixner <[email protected]> AuthorDate: Mon, 08 Apr 2024 15:22:01 +02:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 10 Apr 2024 15:31:38 +02:00
x86/topology: Don't update cpu_possible_map in topo_set_cpuids()
topo_set_cpuids() updates cpu_present_map and cpu_possible map. It is invoked during enumeration and "physical hotplug" operations. In the latter case this results in a kernel crash because cpu_possible_map is marked read only after init completes.
There is no reason to update cpu_possible_map in that function. During enumeration cpu_possible_map is not relevant and gets fully initialized after enumeration completed. On "physical hotplug" the bit is already set because the kernel allows only CPUs to be plugged which have been enumerated and associated to a CPU number during early boot.
Remove the bogus update of cpu_possible_map.
--- arch/x86/kernel/cpu/topology.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/topology.c +5 -2
@@ -123,7 +123,6 @@ static void topo_set_cpuids(unsigned int cpu, u32 apic_id, u32 acpi_id)early_per_cpu (x86_cpu_to_apicid, cpu) = apic_id;early_per_cpu (x86_cpu_to_acpiid, cpu) = acpi_id;#endif -set_cpu_possible (cpu, true);set_cpu_present (cpu, true); }
@@ -210,7 +209,11 @@ static __init void topo_register_apic(u32 apic_id, u32 acpi_id, bool present) topo_info.nr_disabled_cpus++; } -/* Register present and possible CPUs in the domain maps */ +/* +* Register present and possible CPUs in the domain +* maps. cpu_possible_map will be updated in +* topology_init_possible_cpus() after enumeration is done. +*/ for (dom =TOPO_SMT_DOMAIN ; dom <TOPO_MAX_DOMAIN ; dom++)set_bit (topo_apicid (apic_id, dom), apic_maps[dom].map); }