Fix panic in logical packages calculation
Subsystems (2)
v4 cover Fix panic in logical packages calculation
A system booted with a small number of cores enabled per package panics because the estimate of __max_logical_packages is too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.
Andi noted that the logical_packages calculation wastes 128k on 64-bit x86 systems and posted patches to fix the allocation. His patches result in significant memory size savings, and I have put my fix on top of those patches.
Here's an example of the panic:
smpboot: Booting Node 1, Processors #1 OK
smpboot: Package 1 of CPU 1 exceeds BIOS package data 1.
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cpu/common.c:1087!
invalid opcode: 0000 [#1] SMP
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.0-rc2+ #4This was tested by dropping the number of active cores across all packages to reproduce the bug above. Additional testing included 2 socket and 4 socket systems and hotplugging entire sockets in different order.
Andi Kleen (2):
perf/x86/intel/uncore: Cache logical pkg id in uncore driver x86/topology: Avoid wasting 128k for package id array
Prarit Bhargava (1):
x86/smpboot: Fix __max_logical_packages estimate
arch/x86/events/intel/uncore.c | 1 + arch/x86/events/intel/uncore.h | 1 + arch/x86/events/intel/uncore_snbep.c | 2 +- arch/x86/include/asm/processor.h | 4 +- arch/x86/kernel/smpboot.c | 137 ++++++++++++----------------------- 5 files changed, 53 insertions(+), 92 deletions(-)
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
PATCH v4 1/3 perf/x86/intel/uncore: Cache logical pkg id in uncore driver
From: Andi Kleen <[email protected]>
The SNB-EP uncore driver is the only user of topology_phys_to_logical_pkg in a performance critical path. Change it query the logical pkg ID only once at initialization time and then cache it in box structure.
--- arch/x86/events/intel/uncore.c | 1 + arch/x86/events/intel/uncore.h | 1 + arch/x86/events/intel/uncore_snbep.c | 2 +- 3 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/events/intel/uncore.c +1 -0
@@ -947,6 +947,7 @@ static int uncore_pci_probe(struct pci_dev *pdev, const struct pci_device_id *idatomic_inc (&box->refcnt); box->pci_phys_id = phys_id; + box->logical_pkg_id =topology_phys_to_logical_pkg (box->pci_phys_id); box->pkgid = pkg; box->pci_dev = pdev; box->pmu = pmu;
arch/x86/events/intel/uncore.h +1 -0
@@ -99,6 +99,7 @@ struct intel_uncore_extra_reg {struct intel_uncore_box {int pci_phys_id; +int logical_pkg_id;int pkgid;int n_active;/* number of active events */ int n_events;
arch/x86/events/intel/uncore_snbep.c +1 -1
@@ -1056,7 +1056,7 @@ static void snbep_qpi_enable_event(struct intel_uncore_box *box, struct perf_eveif (reg1->idx !=EXTRA_REG_NONE ) {int idx = box->pmu->pmu_idx +SNBEP_PCI_QPI_PORT0_FILTER ; -int pkg =topology_phys_to_logical_pkg (box->pci_phys_id); +int pkg = box->logical_pkg_id;struct pci_dev *filter_pdev = uncore_extra_pci_dev[pkg].dev[idx];if (filter_pdev) {
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
Re: [PATCH 1/3 v4] perf/x86/intel/uncore: Cache logical pkg id in uncore driver
Quoted 5 lines · Prarit Bhargava
From: Andi Kleen <[email protected]>
The SNB-EP uncore driver is the only user of
topology_phys_to_logical_pkgin a performance critical path. Change it query the logical pkg ID only once at initialization time and then cache it in box structure.
That second sentence needs some care:
Avoid the lookup in the hotpath and cache the id at initialization time. This is also a prerequisite to change the underlying logical package management implementation w/o impacting this driver.
Other than that.
PATCH v4 2/3 x86/topology: Avoid wasting 128k for package id array
In reply to Prarit BhargavaFrom: Andi Kleen <[email protected]>
I was looking at large early boot allocations and noticed that since (1f12e32f x86/topology: Create logical package id) every 64bit system allocates a 128k array to convert logical package ids.
This happens because the array is sized to (MAX_LOCAL_APIC * u16) = 128k. This is a lot of waste especially for most systems which have one or two sockets.
Use a dynamically allocated array of size logical_packages to map the logical and physical packages.
[v2]: Decrease logical_packages when the last thread in a socket is removed.
[v3]: Add more logic to keep logical and physical package IDs
in synch.
[v4]: Keep logical mapping static by using hybrid approach of a small logical to physical array and keeping logical cpu information in cpu_data.
--- arch/x86/include/asm/processor.h | 4 +- arch/x86/kernel/smpboot.c | 86 +++++++++++++++++++--------------------- 2 files changed, 43 insertions(+), 47 deletions(-)
arch/x86/include/asm/processor.h +3 -1
@@ -124,8 +124,10 @@ struct cpuinfo_x86 {u16 booted_cores;/* Physical processor id: */ u16 phys_proc_id; -/* Logical processor id: */ +/* Logical processor (package) id: */ u16 logical_proc_id; +/* Physical package ID */ +u16 phys_pkg_id;/* Core id: */ u16 cpu_core_id;/* Index into per_cpu list: */
arch/x86/kernel/smpboot.c +40 -46
130 lines not on this page · raw message
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64
Re: [PATCH 2/3 v4] x86/topology: Avoid wasting 128k for package id array
arch/x86/include/asm/processor.h struct cpuinfo_x86 { · line 130
Quoted 13 lines · Prarit Bhargava
arch/x86/include/asm/processor.h +3
-1@@ -124,8 +124,10 @@ struct cpuinfo_x86 {u16 booted_cores;/* Physical processor id: */ u16 phys_proc_id; -/* Logical processor id: */ +/* Logical processor (package) id: */ u16 logical_proc_id; +/* Physical package ID */ +u16 phys_pkg_id;
How is this new field used aside of being written to and how is it different from phys_proc_id? AFAICT, it's the same as all callers to topology_update_package_map() are handing in cpu_data->phys_proc_id.
Quoted 16 lines
arch/x86/include/asm/processor.h +16
-0+/** +* topology_phys_to_logical_pkg - Map a physical package id to a logical +* +* Returns logical package id or -1 if not found +*/ +int topology_phys_to_logical_pkg (unsigned int phys_pkg) +{ +int log_pkg; + +for (log_pkg =0 ; log_pkg < logical_packages; log_pkg++) +if (logical_to_physical_pkg_map[log_pkg] == phys_pkg) +return log_pkg; + +return -1 ; +} +EXPORT_SYMBOL (topology_phys_to_logical_pkg);
....
arch/x86/include/asm/processor.h
Quoted 11 lines
arch/x86/include/asm/processor.h +9
-1+ +/* Allocate and copy a new array */ + ltp_pkg_map_new =kmalloc (logical_packages *sizeof (u16),GFP_KERNEL ); +BUG_ON (!ltp_pkg_map_new); +if (logical_to_physical_pkg_map) { +memcpy (ltp_pkg_map_new, logical_to_physical_pkg_map, + logical_packages *sizeof (u16)); +kfree (logical_to_physical_pkg_map); } - physical_to_logical_pkg[pkg] = new; + logical_to_physical_pkg_map = ltp_pkg_map_new;
This lacks serialization and is therefore broken against a concurrent topology_phys_to_logical_pkg() call for obvious reasons. The current user is probably safe, but this really needs to be fixed now.
Thanks,
tglx
Re: [PATCH 2/3 v4] x86/topology: Avoid wasting 128k for package id array
Quoted 10 lines · Thomas Gleixner
Quoted 9 lines · Prarit Bhargava
arch/x86/include/asm/processor.h +0
-1@@ -124,8 +124,10 @@ struct cpuinfo_x86 {u16 booted_cores;/* Physical processor id: */ u16 phys_proc_id; -/* Logical processor id: */
Quoted 9 lines
arch/x86/include/asm/processor.h struct cpuinfo_x86 { · line 130
Quoted 4 lines
arch/x86/include/asm/processor.h +3
-0+/* Logical processor (package) id: */ u16 logical_proc_id; +/* Physical package ID */ +u16 phys_pkg_id;How is this new field used aside of being written to and how is it different from
phys_proc_id? AFAICT, it's the same as all callers totopology_update_package_map()are handing incpu_data->phys_proc_id.
I've removed this in v5.
Quoted 19 lines
Quoted 16 lines
arch/x86/include/asm/processor.h +16
-0+/** +* topology_phys_to_logical_pkg - Map a physical package id to a logical +* +* Returns logical package id or -1 if not found +*/ +int topology_phys_to_logical_pkg (unsigned int phys_pkg) +{ +int log_pkg; + +for (log_pkg =0 ; log_pkg < logical_packages; log_pkg++) +if (logical_to_physical_pkg_map[log_pkg] == phys_pkg) +return log_pkg; + +return -1 ; +} +EXPORT_SYMBOL (topology_phys_to_logical_pkg);....
Quoted 15 lines
arch/x86/include/asm/processor.h
Quoted 11 lines
arch/x86/include/asm/processor.h +9
-1+ +/* Allocate and copy a new array */ + ltp_pkg_map_new =kmalloc (logical_packages *sizeof (u16),GFP_KERNEL ); +BUG_ON (!ltp_pkg_map_new); +if (logical_to_physical_pkg_map) { +memcpy (ltp_pkg_map_new, logical_to_physical_pkg_map, + logical_packages *sizeof (u16)); +kfree (logical_to_physical_pkg_map); } - physical_to_logical_pkg[pkg] = new; + logical_to_physical_pkg_map = ltp_pkg_map_new;This lacks serialization and is therefore broken against a concurrent
topology_phys_to_logical_pkg()call for obvious reasons. The current user is probably safe, but this really needs to be fixed now.
I'm using a spin_lock_irq in v5.
I am finishing testing on 4S & 8S systems now.
P.
Quoted 5 lines
Thanks,
tglx
PATCH v4 3/3 x86/smpboot: Fix __max_logical_packages estimate
In reply to Prarit BhargavaA system booted with a small number of cores enabled per package panics because the estimate of __max_logical_packages is too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package.
ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of __max_logical_packages is calculated as DIV_ROUND_UP(16 / 20) = 1 and not 4.
Here's an example of the panic:
smpboot: Booting Node 1, Processors #1 OK
smpboot: Package 1 of CPU 1 exceeds BIOS package data 1.
------------[ cut here ]------------
kernel BUG at arch/x86/kernel/cpu/common.c:1087!
invalid opcode: 0000 [#1] SMP
Modules linked in:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.0-rc2+ #4Calculate __max_logical_packages after the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages.
--- arch/x86/kernel/smpboot.c | 57 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 47 deletions(-)
arch/x86/kernel/smpboot.c +10 -47
@@ -307,12 +307,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)if (new >=0 ) goto found; -if (logical_packages >= __max_logical_packages) { -pr_warn ("Package %u of CPU %u exceeds BIOS package data %u.\n" , - logical_packages, cpu, __max_logical_packages); -return -ENOSPC ; - } - new = logical_packages++;/* Allocate and copy a new array */
@@ -335,46 +329,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu)return 0 ; } -static void __init smp_init_package_map (struct cpuinfo_x86 *c,unsigned int cpu) -{ -unsigned int ncpus; - -/* -* Today neither Intel nor AMD support heterogenous systems. That -* might change in the future.... -* -* While ideally we'd want '* smp_num_siblings' in the below @ncpus -* computation, this won't actually work since some Intel BIOSes -* report inconsistent HT data when they disable HT. -* -* In particular, they reduce the APIC-IDs to only include the cores, -* but leave the CPUID topology to say there are (2) siblings. -* This means we don't know how many threads there will be until -* after the APIC enumeration. -* -* By not including this we'll sometimes over-estimate the number of -* logical packages by the amount of !present siblings, but this is -* still better than MAX_LOCAL_APIC. -* -* We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited -* on the command line leading to a similar issue as the HT disable -* problem because the hyperthreads are usually enumerated after the -* primary cores. -*/ - ncpus = boot_cpu_data.x86_max_cores; -if (!ncpus) { -pr_warn ("x86_max_cores == zero !?!?" ); - ncpus =1 ; - } - - __max_logical_packages =DIV_ROUND_UP (total_cpus, ncpus); -pr_info ("Max logical packages: %u\n" , __max_logical_packages); - - logical_packages =0 ; - -topology_update_package_map (c->phys_proc_id, cpu); -} -void __init smp_store_boot_cpu_info (void ) {int id =0 ;/* CPU 0 */
@@ -382,7 +336,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; -smp_init_package_map (c, id); +topology_update_package_map (c->phys_proc_id, id); }/*
@@ -1382,7 +1336,16 @@ void __init native_smp_prepare_boot_cpu(void)void __init native_smp_cpus_done (unsigned int max_cpus) { +int ncpus; +pr_debug ("Boot done\n" ); +/* +* Today neither Intel nor AMD support heterogenous systems so +* extrapolate the boot cpu's data to all packages. +*/ + ncpus =cpu_data (0 ).booted_cores * smp_num_siblings; + __max_logical_packages =DIV_ROUND_UP (nr_cpu_ids, ncpus); +pr_info ("Max logical packages: %u\n" , __max_logical_packages);if (x86_has_numa_in_package)set_sched_topology (x86_numa_in_package_topology);
Signature · 1 line
2.15.0.rc0.39.g2f0e14e64