Fix panic in logical packages calculation
Subsystems (2)
v6 cover Fix panic in logical packages calculation
A system booted with a small number of cores enabled per package panics because the estimate of __max_logical_packages is too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.
Andi noted that the logical_packages calculation wastes 128k on 64-bit x86 systems and posted patches to fix the allocation. His patches result in significant memory size savings, and I have put my fix on top of those patches.
Here's an example of the panic:
smpboot: Booting Node 1, Processors #1 OK
smpboot: Package 1 of CPU 1 exceeds BIOS package data 1.
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cpu/common.c:1087!
invalid opcode: 0000 [#1] SMP
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.0-rc2+ #4This was tested by dropping the number of active cores across all packages to reproduce the bug above. Additional testing included 2 socket and 4 socket systems and hotplugging entire sockets in different order.
Andi Kleen (2):
perf/x86/intel/uncore: Cache logical pkg id in uncore driver x86/topology: Avoid wasting 128k for package id array
Prarit Bhargava (1):
x86/smpboot: Fix __max_logical_packages estimate
arch/x86/events/intel/uncore.c | 4 +- arch/x86/events/intel/uncore.h | 2 +- arch/x86/events/intel/uncore_snbep.c | 2 +- arch/x86/include/asm/processor.h | 1 + arch/x86/kernel/smpboot.c | 130 +++++++++++------------------------ 5 files changed, 44 insertions(+), 95 deletions(-)
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
PATCH v6 1/3 perf/x86/intel/uncore: Cache logical pkg id in uncore driver
From: Andi Kleen <[email protected]>
The SNB-EP uncore driver is the only user of topology_phys_to_logical_pkg in a performance critical path. Change it query the logical pkg ID only once at initialization time and then cache it in box structure.
[v2]: Used the existing pkgid, Logical package ID, which was already stored.
--- arch/x86/events/intel/uncore.c | 4 ++-- arch/x86/events/intel/uncore.h | 2 +- arch/x86/events/intel/uncore_snbep.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/events/intel/uncore.c +2 -2
@@ -975,10 +975,10 @@ static void uncore_pci_remove(struct pci_dev *pdev)int i, phys_id, pkg; phys_id =uncore_pcibus_to_physid (pdev->bus); - pkg =topology_phys_to_logical_pkg (phys_id); box =pci_get_drvdata (pdev);if (!box) { + pkg =topology_phys_to_logical_pkg (phys_id);for (i =0 ; i <UNCORE_EXTRA_PCI_DEV_MAX ; i++) {if (uncore_extra_pci_dev[pkg].dev[i] == pdev) { uncore_extra_pci_dev[pkg].dev[i] =NULL ;
@@ -994,7 +994,7 @@ static void uncore_pci_remove(struct pci_dev *pdev)return ;pci_set_drvdata (pdev,NULL ); - pmu->boxes[pkg] =NULL ; + pmu->boxes[box->pkgid] =NULL ;if (atomic_dec_return (&pmu->activeboxes) ==0 )uncore_pmu_unregister (pmu);uncore_box_exit (box);
arch/x86/events/intel/uncore.h +1 -1
@@ -100,7 +100,7 @@ struct intel_uncore_extra_reg {struct intel_uncore_box {int pci_phys_id; -int pkgid; +int pkgid;/* Logical package ID */ int n_active;/* number of active events */ int n_events;int cpu;/* cpu to collect events */
arch/x86/events/intel/uncore_snbep.c +1 -1
@@ -1057,7 +1057,7 @@ static void snbep_qpi_enable_event(struct intel_uncore_box *box, struct perf_eveif (reg1->idx !=EXTRA_REG_NONE ) {int idx = box->pmu->pmu_idx +SNBEP_PCI_QPI_PORT0_FILTER ; -int pkg =topology_phys_to_logical_pkg (box->pci_phys_id); +int pkg = box->pkgid;struct pci_dev *filter_pdev = uncore_extra_pci_dev[pkg].dev[idx];if (filter_pdev) {
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
tip-bot for Andi Kleen
merge bot[tip:x86/urgent] perf/x86/intel/uncore: Cache logical pkg id in uncore driver
perf/x86/intel/uncore: Cache logical pkg id in uncore driver
Commit-ID: d46b4c1ce5f0d9a13fb2318763076442669a2bdc Gitweb: https://git.kernel.org/tip/d46b4c1ce5f0d9a13fb2318763076442669a2bdc Author: Andi Kleen <[email protected]> AuthorDate: Tue, 14 Nov 2017 07:42:55 -0500 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 17 Nov 2017 16:22:30 +0100
perf/x86/intel/uncore: Cache logical pkg id in uncore driver
The SNB-EP uncore driver is the only user of topology_phys_to_logical_pkg in a performance critical path.
Change it query the logical pkg ID only once at initialization time and then cache it in box structure. This allows to change the logical package management without affecting the performance critical path.
--- arch/x86/events/intel/uncore.c | 4 ++-- arch/x86/events/intel/uncore.h | 2 +- arch/x86/events/intel/uncore_snbep.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/events/intel/uncore.c +2 -2
@@ -975,10 +975,10 @@ static void uncore_pci_remove(struct pci_dev *pdev)int i, phys_id, pkg; phys_id =uncore_pcibus_to_physid (pdev->bus); - pkg =topology_phys_to_logical_pkg (phys_id); box =pci_get_drvdata (pdev);if (!box) { + pkg =topology_phys_to_logical_pkg (phys_id);for (i =0 ; i <UNCORE_EXTRA_PCI_DEV_MAX ; i++) {if (uncore_extra_pci_dev[pkg].dev[i] == pdev) { uncore_extra_pci_dev[pkg].dev[i] =NULL ;
@@ -994,7 +994,7 @@ static void uncore_pci_remove(struct pci_dev *pdev)return ;pci_set_drvdata (pdev,NULL ); - pmu->boxes[pkg] =NULL ; + pmu->boxes[box->pkgid] =NULL ;if (atomic_dec_return (&pmu->activeboxes) ==0 )uncore_pmu_unregister (pmu);uncore_box_exit (box);
arch/x86/events/intel/uncore.h +1 -1
@@ -100,7 +100,7 @@ struct intel_uncore_extra_reg {struct intel_uncore_box {int pci_phys_id; -int pkgid; +int pkgid;/* Logical package ID */ int n_active;/* number of active events */ int n_events;int cpu;/* cpu to collect events */
arch/x86/events/intel/uncore_snbep.c +1 -1
@@ -1057,7 +1057,7 @@ static void snbep_qpi_enable_event(struct intel_uncore_box *box, struct perf_eveif (reg1->idx !=EXTRA_REG_NONE ) {int idx = box->pmu->pmu_idx +SNBEP_PCI_QPI_PORT0_FILTER ; -int pkg =topology_phys_to_logical_pkg (box->pci_phys_id); +int pkg = box->pkgid;struct pci_dev *filter_pdev = uncore_extra_pci_dev[pkg].dev[idx];if (filter_pdev) {
PATCH v6 2/3 x86/topology: Avoid wasting 128k for package id array
In reply to Prarit BhargavaFrom: Andi Kleen <[email protected]>
I was looking at large early boot allocations and noticed that since (1f12e32f x86/topology: Create logical package id) every 64bit system allocates a 128k array to convert logical package ids.
This happens because the array is sized for MAX_LOCAL_APIC and that is always 32k on 64bit systems, and it needs 4 bytes for each entry.
This is fairly wasteful, especially for the common case of having only one socket, where we need 128K just to store a single 4 byte value.
Store the logical processor id in cpudata. Add a set flag that indicates that the cpu's cpudata entry has been written.
[v2]: Decrease logical_packages when the last thread in a socket is removed.
[v3]: Add more logic to keep logical and physical package IDs in synch.
[v4]: Keep logical mapping static by using hybrid approach of a small logical
to physical array and keeping logical cpu information in cpu_data. [v5]: Change kmalloc to GFP_ATOMIC to fix "sleeping function" warning on virtual machines. Remove phys_pkg_id. Add spinlock to avoid concurrency issues.
[v6]: Revert back to storing logical processor ID in cpudata, and add set flag to indicate that boot_cpu_data was written to a cpu's cpu_data.
--- arch/x86/include/asm/processor.h | 1 + arch/x86/kernel/smpboot.c | 75 ++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 46 deletions(-)
arch/x86/include/asm/processor.h +1 -0
@@ -132,6 +132,7 @@ struct cpuinfo_x86 {/* Index into per_cpu list: */ u16 cpu_index;u32 microcode; + unsigned set : 1; } __randomize_layout;struct cpuid_regs {
arch/x86/kernel/smpboot.c +29 -46
133 lines not on this page · raw message
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
Re: [PATCH v6 2/3] x86/topology: Avoid wasting 128k for package id array
arch/x86/kernel/smpboot.c void __init smp_store_boot_cpu_info(void) · line 372
Quoted 5 lines · Prarit Bhargava
arch/x86/kernel/smpboot.c +1
-0@@ -389,6 +369,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id;smp_init_package_map (c, id); +cpu_data (id).set =1 ;
What's wrong with
c->set ?
Aside of that 'set' is not really descriptive. initialized might be better suited.
Quoted 7 lines
arch/x86/kernel/smpboot.c +0
-0}/* @@ -399,13 +380,15 @@ void smp_store_cpu_info(int id) {struct cpuinfo_x86 *c = &cpu_data (id);
arch/x86/kernel/smpboot.c void smp_store_cpu_info(int id) · line 391
Quoted 10 lines
arch/x86/kernel/smpboot.c +3
-1- *c = boot_cpu_data; +if (cpu_data (id).set ==0 ) + *c = boot_cpu_data; c->cpu_index = id;/* * During boot time, CPU0 has this setup already. Save the info when * bringing up AP or offlined CPU0. */ identify_secondary_cpu (c); +cpu_data (id).set =1 ;
See above.
No need to resend. I'll fix it up.
Thanks,
tglx
tip-bot for Andi Kleen
merge bot[tip:x86/urgent] x86/topology: Avoid wasting 128k for package id array
In reply to Prarit Bhargavax86/topology: Avoid wasting 128k for package id array
Commit-ID: 30bb9811856f667042e746d8033883b1091a46ce Gitweb: https://git.kernel.org/tip/30bb9811856f667042e746d8033883b1091a46ce Author: Andi Kleen <[email protected]> AuthorDate: Tue, 14 Nov 2017 07:42:56 -0500 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 17 Nov 2017 16:22:30 +0100
x86/topology: Avoid wasting 128k for package id array
Analyzing large early boot allocations unveiled the logical package id storage as a prominent memory waste. Since commit 1f12e32f4cd5 ("x86/topology: Create logical package id") every 64-bit system allocates a 128k array to convert logical package ids.
This happens because the array is sized for MAX_LOCAL_APIC which is always 32k on 64bit systems, and it needs 4 bytes for each entry.
This is fairly wasteful, especially for the common case of having only one socket, which uses exactly 4 byte out of 128K.
There is no user of the package id map which is performance critical, so the lookup is not required to be O(1). Store the logical processor id in cpu_data and use a loop based lookup.
To keep the mapping stable accross cpu hotplug operations, add a flag to cpu_data which is set when the CPU is brought up the first time. When the flag is set, then cpu_data is not reinitialized by copying boot_cpu_data on subsequent bringups.
[ tglx: Rename the flag to 'initialized', use proper pointers instead of
repeated cpu_data(x) evaluation and massage changelog. ]
--- arch/x86/include/asm/processor.h | 1 + arch/x86/kernel/smpboot.c | 73 ++++++++++++++++------------------------ 2 files changed, 30 insertions(+), 44 deletions(-)
arch/x86/include/asm/processor.h +1 -0
@@ -132,6 +132,7 @@ struct cpuinfo_x86 {/* Index into per_cpu list: */ u16 cpu_index;u32 microcode; + unsigned initialized : 1; } __randomize_layout;struct cpuid_regs {
arch/x86/kernel/smpboot.c +29 -44
129 lines not on this page · raw message
PATCH v6 3/3 x86/smpboot: Fix __max_logical_packages estimate
In reply to Prarit BhargavaA system booted with a small number of cores enabled per package panics because the estimate of __max_logical_packages is too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.
ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of __max_logical_packages is calculated as DIV_ROUND_UP(16 / 20) = 1 and not 4.
Calculate __max_logical_packages after the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.
--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)
arch/x86/kernel/smpboot.c +10 -45
@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",
@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */
@@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }
@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
tip-bot for Prarit Bhargava
merge bot[tip:x86/urgent] x86/smpboot: Fix __max_logical_packages estimate
x86/smpboot: Fix __max_logical_packages estimate
Commit-ID: b4c0a7326f5dc0ef7a64128b0ae7d081f4b2cbd1 Gitweb: https://git.kernel.org/tip/b4c0a7326f5dc0ef7a64128b0ae7d081f4b2cbd1 Author: Prarit Bhargava <[email protected]> AuthorDate: Tue, 14 Nov 2017 07:42:57 -0500 Committer: Thomas Gleixner <[email protected]> CommitDate: Fri, 17 Nov 2017 16:22:31 +0100
x86/smpboot: Fix __max_logical_packages estimate
A system booted with a small number of cores enabled per package panics because the estimate of __max_logical_packages is too low.
This occurs when the total number of active cores across all packages is less than the maximum core count for a single package. e.g.:
On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of __max_logical_packages is calculated as DIV_ROUND_UP(16 / 20) = 1 and not 4.
Calculate __max_logical_packages after the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.
--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)
arch/x86/kernel/smpboot.c +10 -45
@@ -310,12 +310,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) { pr_info("CPU %u Converting physical %u to logical package %u\n",
@@ -326,44 +320,6 @@ found:return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */
@@ -371,7 +327,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id); c->initialized = true; }
@@ -1341,7 +1297,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);
Re: [v6, 3/3] x86/smpboot: Fix __max_logical_packages estimate
In reply to Prarit BhargavaPrarit Bhargava:
Quoted 116 lines
A system booted with a small number of cores enabled per package panics because the estimate of
__max_logical_packagesis too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of
__max_logical_packagesis calculated asDIV_ROUND_UP(16 / 20)= 1 and not 4.Calculate
__max_logical_packagesafter the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.Signed-off-by Prarit Bhargava <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Andi Kleen <[email protected]>Cc Dave Hansen <[email protected]>Cc Piotr Luc <[email protected]>Cc Kan Liang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Stephane Eranian <[email protected]>Cc Prarit Bhargava <[email protected]>Cc Arvind Yadav <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Christian Borntraeger <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Tom Lendacky <[email protected]>Cc He Chen <[email protected]>Cc Mathias Krause <[email protected]>Cc Tim Chen <[email protected]>Cc Vitaly Kuznetsov <[email protected]>--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)arch/x86/kernel/smpboot.c +3
-45@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */ @@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" );
arch/x86/kernel/smpboot.c void __init native_smp_prepare_boot_cpu(void) · line 1342
Quoted 10 lines
arch/x86/kernel/smpboot.c +7
-0+/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);
This breaks booting as Xen PV domain for me. The problem seems to be that native_smp_cpus_done() is never called on a PV domain. So __max_logical_packages is uninitialized and this leads to a NULL pointer dereference in coretemp.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [v6,3/3] x86/smpboot: Fix __max_logical_packages estimate
In reply to Prarit BhargavaPrarit Bhargava:
Quoted 116 lines
A system booted with a small number of cores enabled per package panics because the estimate of
__max_logical_packagesis too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of
__max_logical_packagesis calculated asDIV_ROUND_UP(16 / 20)= 1 and not 4.Calculate
__max_logical_packagesafter the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.Signed-off-by Prarit Bhargava <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Andi Kleen <[email protected]>Cc Dave Hansen <[email protected]>Cc Piotr Luc <[email protected]>Cc Kan Liang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Stephane Eranian <[email protected]>Cc Prarit Bhargava <[email protected]>Cc Arvind Yadav <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Christian Borntraeger <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Tom Lendacky <[email protected]>Cc He Chen <[email protected]>Cc Mathias Krause <[email protected]>Cc Tim Chen <[email protected]>Cc Vitaly Kuznetsov <[email protected]>--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)arch/x86/kernel/smpboot.c +3
-45@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */ @@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" );
arch/x86/kernel/smpboot.c void __init native_smp_prepare_boot_cpu(void) · line 1342
Quoted 10 lines
arch/x86/kernel/smpboot.c +7
-0+/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);
This breaks booting as Xen PV domain for me. The problem seems to be that native_smp_cpus_done() is never called on a PV domain. So __max_logical_packages is uninitialized and this leads to a NULL pointer dereference in coretemp.
Re: [v6, 3/3] x86/smpboot: Fix __max_logical_packages estimate
Quoted 125 lines · Simon Gaiser
Prarit Bhargava:
Quoted 124 lines
A system booted with a small number of cores enabled per package panics because the estimate of
__max_logical_packagesis too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of
__max_logical_packagesis calculated asDIV_ROUND_UP(16 / 20)= 1 and not 4.Calculate
__max_logical_packagesafter the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.Signed-off-by Prarit Bhargava <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Andi Kleen <[email protected]>Cc Dave Hansen <[email protected]>Cc Piotr Luc <[email protected]>Cc Kan Liang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Stephane Eranian <[email protected]>Cc Prarit Bhargava <[email protected]>Cc Arvind Yadav <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Christian Borntraeger <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Tom Lendacky <[email protected]>Cc He Chen <[email protected]>Cc Mathias Krause <[email protected]>Cc Tim Chen <[email protected]>Cc Vitaly Kuznetsov <[email protected]>--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)arch/x86/kernel/smpboot.c +10
-45@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */ @@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);
Quoted 8 lines
arch/x86/kernel/smpboot.c void __init native_smp_prepare_boot_cpu(void) · line 1342
Quoted 2 lines
arch/x86/kernel/smpboot.c +0
-0if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);This breaks booting as Xen PV domain for me. The problem seems to be that
native_smp_cpus_done()is never called on a PV domain. So__max_logical_packagesis uninitialized and this leads to a NULL pointer dereference in coretemp.
I'll see if I can figure out a way to test that. Does 947134d9b00f ("x86/smpboot: Do not use smp_num_siblings in __max_logical_packages calculation") help?
P.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [v6,3/3] x86/smpboot: Fix __max_logical_packages estimate
In reply to Simon GaiserQuoted 125 lines · Simon Gaiser
Prarit Bhargava:
Quoted 124 lines
A system booted with a small number of cores enabled per package panics because the estimate of
__max_logical_packagesis too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of
__max_logical_packagesis calculated asDIV_ROUND_UP(16 / 20)= 1 and not 4.Calculate
__max_logical_packagesafter the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.Signed-off-by Prarit Bhargava <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Andi Kleen <[email protected]>Cc Dave Hansen <[email protected]>Cc Piotr Luc <[email protected]>Cc Kan Liang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Stephane Eranian <[email protected]>Cc Prarit Bhargava <[email protected]>Cc Arvind Yadav <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Christian Borntraeger <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Tom Lendacky <[email protected]>Cc He Chen <[email protected]>Cc Mathias Krause <[email protected]>Cc Tim Chen <[email protected]>Cc Vitaly Kuznetsov <[email protected]>--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)arch/x86/kernel/smpboot.c +10
-45@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */ @@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);
Quoted 8 lines
arch/x86/kernel/smpboot.c void __init native_smp_prepare_boot_cpu(void) · line 1342
Quoted 2 lines
arch/x86/kernel/smpboot.c +0
-0if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);This breaks booting as Xen PV domain for me. The problem seems to be that
native_smp_cpus_done()is never called on a PV domain. So__max_logical_packagesis uninitialized and this leads to a NULL pointer dereference in coretemp.
I'll see if I can figure out a way to test that. Does 947134d9b00f ("x86/smpboot: Do not use smp_num_siblings in __max_logical_packages calculation") help?
P.
Re: [v6, 3/3] x86/smpboot: Fix __max_logical_packages estimate
Prarit Bhargava:
Quoted 135 lines · Simon Gaiser
Quoted 125 lines
Prarit Bhargava:
Quoted 124 lines
A system booted with a small number of cores enabled per package panics because the estimate of
__max_logical_packagesis too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of
__max_logical_packagesis calculated asDIV_ROUND_UP(16 / 20)= 1 and not 4.Calculate
__max_logical_packagesafter the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.Signed-off-by Prarit Bhargava <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Andi Kleen <[email protected]>Cc Dave Hansen <[email protected]>Cc Piotr Luc <[email protected]>Cc Kan Liang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Stephane Eranian <[email protected]>Cc Prarit Bhargava <[email protected]>Cc Arvind Yadav <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Christian Borntraeger <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Tom Lendacky <[email protected]>Cc He Chen <[email protected]>Cc Mathias Krause <[email protected]>Cc Tim Chen <[email protected]>Cc Vitaly Kuznetsov <[email protected]>--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)arch/x86/kernel/smpboot.c +10
-45@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */ @@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);Quoted 8 lines
arch/x86/kernel/smpboot.c void __init native_smp_prepare_boot_cpu(void) · line 1342
Quoted 2 lines
arch/x86/kernel/smpboot.c +0
-0if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);This breaks booting as Xen PV domain for me. The problem seems to be that
native_smp_cpus_done()is never called on a PV domain. So__max_logical_packagesis uninitialized and this leads to a NULL pointer dereference in coretemp.
Quoted 1 line
I'll see if I can figure out a way to test that.
Ok, thanks. If you need some logs, or if I should test something just ask.
Quoted 3 lines
Does
947134d9b00f
("x86/smpboot: Do not usesmp_num_siblingsin__max_logical_packagescalculation") help?
No.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [v6,3/3] x86/smpboot: Fix __max_logical_packages estimate
In reply to Prarit BhargavaPrarit Bhargava:
Quoted 135 lines · Simon Gaiser
Quoted 125 lines
Prarit Bhargava:
Quoted 124 lines
A system booted with a small number of cores enabled per package panics because the estimate of
__max_logical_packagesis too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of
__max_logical_packagesis calculated asDIV_ROUND_UP(16 / 20)= 1 and not 4.Calculate
__max_logical_packagesafter the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.Signed-off-by Prarit Bhargava <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc "H. Peter Anvin" <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Andi Kleen <[email protected]>Cc Dave Hansen <[email protected]>Cc Piotr Luc <[email protected]>Cc Kan Liang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Stephane Eranian <[email protected]>Cc Prarit Bhargava <[email protected]>Cc Arvind Yadav <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Christian Borntraeger <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Tom Lendacky <[email protected]>Cc He Chen <[email protected]>Cc Mathias Krause <[email protected]>Cc Tim Chen <[email protected]>Cc Vitaly Kuznetsov <[email protected]>--- arch/x86/kernel/smpboot.c | 55 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 45 deletions(-)arch/x86/kernel/smpboot.c +10
-45@@ -308,12 +308,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;if (new != pkg) pr_info("CPU %u Converting physical %u to logical package %u\n",@@ -323,44 +317,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */ @@ -368,7 +324,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id);cpu_data (id).set =1 ; }@@ -1371,7 +1327,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);Quoted 8 lines
arch/x86/kernel/smpboot.c void __init native_smp_prepare_boot_cpu(void) · line 1342
Quoted 2 lines
arch/x86/kernel/smpboot.c +0
-0if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);This breaks booting as Xen PV domain for me. The problem seems to be that
native_smp_cpus_done()is never called on a PV domain. So__max_logical_packagesis uninitialized and this leads to a NULL pointer dereference in coretemp.
Quoted 1 line
I'll see if I can figure out a way to test that.
Ok, thanks. If you need some logs, or if I should test something just ask.
Quoted 3 lines
Does
947134d9b00f
("x86/smpboot: Do not usesmp_num_siblingsin__max_logical_packagescalculation") help?
No.
Re: [v6, 3/3] x86/smpboot: Fix __max_logical_packages estimate
Quoted 2 lines · Simon Gaiser
Prarit Bhargava:
Quoted 1 line · Simon Gaiser
<snip>
Quoted 11 lines
Quoted 7 lines
Quoted 5 lines
This breaks booting as Xen PV domain for me. The problem seems to be that
native_smp_cpus_done()is never called on a PV domain. So__max_logical_packagesis uninitialized and this leads to a NULL pointer dereference in coretemp.I'll see if I can figure out a way to test that.
Ok, thanks. If you need some logs, or if I should test something just ask.
Thanks, I may send you a patch off-list to test.
Quoted 6 lines
Quoted 3 lines
Does
947134d9b00f
("x86/smpboot: Do not usesmp_num_siblingsin__max_logical_packagescalculation") help?No.
Yeah, your description was absolutely correct. native_smp_cpus_done() is only called for HVM guests and not PV. I'm going to think about that and send you a patch in a bit.
P.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [v6,3/3] x86/smpboot: Fix __max_logical_packages estimate
In reply to Simon GaiserQuoted 2 lines · Simon Gaiser
Prarit Bhargava:
Quoted 1 line · Simon Gaiser
<snip>
Quoted 11 lines
Quoted 7 lines
Quoted 5 lines
This breaks booting as Xen PV domain for me. The problem seems to be that
native_smp_cpus_done()is never called on a PV domain. So__max_logical_packagesis uninitialized and this leads to a NULL pointer dereference in coretemp.I'll see if I can figure out a way to test that.
Ok, thanks. If you need some logs, or if I should test something just ask.
Thanks, I may send you a patch off-list to test.
Quoted 6 lines
Quoted 3 lines
Does
947134d9b00f
("x86/smpboot: Do not usesmp_num_siblingsin__max_logical_packagescalculation") help?No.
Yeah, your description was absolutely correct. native_smp_cpus_done() is only called for HVM guests and not PV. I'm going to think about that and send you a patch in a bit.
P.