dump_stack: Allow runtime updates of the hardware description
Subsystems (3)
cover dump_stack: Allow runtime updates of the hardware description
When the kernel emits a stack trace, typically it includes a hardware description string, e.g.
Kernel panic - not syncing: sysrq triggered crash CPU: 6 PID: 46433 Comm: bash Tainted: G W 6.7.0-rc2+ #83
Quoted 1 line
Hardware name: IBM,9040-MR9 POWER9 (architected) 0x4e2102 0xf000005 of:IBM,FW950.01 (VM950_047) hv:phyp pSeries
Call Trace: dump_stack_lvl+0xc4/0x170 (unreliable) panic+0x39c/0x584 sysrq_handle_crash+0x80/0xe0 __handle_sysrq+0x208/0x4bc [...]
This string is a statically allocated buffer populated during boot by arch code calling dump_stack_set_arch_desc(). For most platforms this is sufficient.
But the string may become inaccurate on the IBM PowerVM platform due to live migration between machine models and firmware versions. Stack dumps emitted after a migration reflect the machine on which the kernel booted, not necessarily the machine on which it is currently running. This is potentially confusing for anyone investigating kernel issues on the platform.
To address this, this series introduces a new function that safely updates the hardware description string and updates the powerpc pseries platform code to call it after a migration. The series also includes changes addressing minor latent issues identified during the implementation.
Platforms which do not need the new functionality remain unchanged.
For this initial version at least, the powerpc/pseries part includes some "self-test" code that 1. verifies that reconstructing the hardware description string late in boot matches the one that was built earlier, and 2. fully exercises the update path before any migrations occur. This could be dropped or made configurable in the future.
---
Nathan Lynch (5):
dump_stack: Make arch description buffer __ro_after_init
dump_stack: Allow update of arch description string at runtime powerpc/prom: Add CPU info to hardware description string later powerpc/pseries: Prepare pseries_add_hw_description() for runtime use powerpc/pseries: Update hardware description string after migration
arch/powerpc/kernel/prom.c | 12 +++-- arch/powerpc/platforms/pseries/mobility.c | 5 ++ arch/powerpc/platforms/pseries/pseries.h | 1 + arch/powerpc/platforms/pseries/setup.c | 80 +++++++++++++++++++++++++++++-- include/linux/printk.h | 5 ++ lib/dump_stack.c | 57 ++++++++++++++++++++-- 6 files changed, 146 insertions(+), 14 deletions(-)
---
base-commit: 44a1aad2fe6c10bfe0589d8047057b10a4c18a19
change-id: 20240111-update-dump-stack-arch-str-7f0880d23f30
Best regards,
Signature · 1 line
Nathan Lynch <[email protected]>
RFC PATCH 1/5 dump_stack: Make arch description buffer __ro_after_init
The static hardware description buffer is populated by arch code during boot and should not change afterwards, so mark it __ro_after_init.
--- lib/dump_stack.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/dump_stack.c +2 -1
@@ -6,6 +6,7 @@#include <linux/kernel.h> #include <linux/buildid.h> +#include <linux/cache.h> #include <linux/export.h> #include <linux/sched.h> #include <linux/sched/debug.h>
@@ -15,7 +16,7 @@#include <linux/utsname.h> #include <linux/stop_machine.h> -static char dump_stack_arch_desc_str[128 ]; +static char dump_stack_arch_desc_str[128 ]__ro_after_init ;/** * dump_stack_set_arch_desc - set arch-specific str to show with task dumps
Signature · 1 line
2.43.0
RFC PATCH 2/5 dump_stack: Allow update of arch description string at runtime
In reply to Nathan LynchThe IBM PowerVM platform (targeted by powerpc/pseries) exposes the physical machine model and firmware version to partitions (guests), and this information is used to populate the arch description string, e.g.
IBM,8408-E8E POWER8E (raw) 0x4b0201 0xf000004 \
of:IBM,FW860.50 (SV860_146) hv:phyp pSeriesThe platform supports live migration of partitions between different machine models and firmware versions, so the arch description string set at boot can become inaccurate, potentially misleading anyone who's analyzing stack traces produced after a migration.
Introduce a RCU-guarded pointer to the current arch description string, initializing it to the static buffer populated at boot. Add to dump_stack_print_info() a RCU read-side critical section that accesses the buffer through this pointer. The majority of architectures which don't need to update the string after boot incur only an additional indirection.
As for platforms which do need that ability, they can use dump_stack_update_arch_desc(), which allocates and formats a new buffer, updates the pointer, and if appropriate frees the previous buffer.
--- include/linux/printk.h | 5 +++++ lib/dump_stack.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 55 insertions(+), 4 deletions(-)
include/linux/printk.h +5 -0
@@ -187,6 +187,7 @@ u32 log_buf_len_get(void);void log_buf_vmcoreinfo_setup (void );void __init setup_log_buf (int early); __printf(1, 2) void dump_stack_set_arch_desc(const char *fmt, ...); +__printf(1, 2) void dump_stack_update_arch_desc(const char *fmt, ...);void dump_stack_print_info (const char *log_lvl);void show_regs_print_info (const char *log_lvl); extern asmlinkage void dump_stack_lvl(const char *log_lvl) __cold;
@@ -253,6 +254,10 @@ static inline __printf(1, 2) void dump_stack_set_arch_desc(const char *fmt, ...) { } +static inline __printf(1, 2) void dump_stack_update_arch_desc(const char *fmt, ...) +{ +} +static inline void dump_stack_print_info (const char *log_lvl) { }
lib/dump_stack.c +50 -4
93 lines not on this page · raw message
Signature · 1 line
2.43.0
RFC PATCH 3/5 powerpc/prom: Add CPU info to hardware description string later
In reply to Nathan Lynchcur_cpu_spec->cpu_name is appended to ppc_hw_desc before cur_cpu_spec has taken on its final value. This is illustrated on pseries by comparing the CPU name as reported at boot ("POWER8E (raw)") to the contents of /proc/cpuinfo ("POWER8 (architected)"):
$ dmesg | grep Hardware
Hardware name: IBM,8408-E8E POWER8E (raw) 0x4b0201 0xf000004 \
of:IBM,FW860.50 (SV860_146) hv:phyp pSeries$ grep -m 1 ^cpu /proc/cpuinfo cpu : POWER8 (architected), altivec supported
Some 44x models would appear to be affected as well; see identical_pvr_fixup().
This results in incorrect CPU information in stack dumps -- ppc_hw_desc is an input to dump_stack_set_arch_desc().
Delay gathering the CPU name until after all potential calls to identify_cpu().
--- arch/powerpc/kernel/prom.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/powerpc/kernel/prom.c +8 -4
@@ -327,6 +327,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node, void *data) {const char *type =of_get_flat_dt_prop (node,"device_type" ,NULL ); +const __be32 *cpu_version =NULL ;const __be32 *prop;const __be32 *intserv;int i, nthreads;
@@ -398,7 +399,7 @@ static int __init early_init_dt_scan_cpus(unsigned long node, prop =of_get_flat_dt_prop (node,"cpu-version" ,NULL );if (prop && (be32_to_cpup (prop) &0xff000000 ) ==0x0f000000 ) {identify_cpu (0 ,be32_to_cpup (prop)); -seq_buf_printf (&ppc_hw_desc,"0x%04x " ,be32_to_cpup (prop)); + cpu_version = prop; }check_cpu_feature_properties (node);
@@ -409,6 +410,12 @@ static int __init early_init_dt_scan_cpus(unsigned long node, }identical_pvr_fixup (node); + +// We can now add the CPU name & PVR to the hardware description +seq_buf_printf (&ppc_hw_desc,"%s 0x%04lx " , cur_cpu_spec->cpu_name,mfspr (SPRN_PVR )); +if (cpu_version) +seq_buf_printf (&ppc_hw_desc,"0x%04x " ,be32_to_cpup (cpu_version)); +init_mmu_slb_size (node);#ifdef CONFIG_PPC64
@@ -846,9 +853,6 @@ void __init early_init_devtree(void *params)dt_cpu_ftrs_scan (); -// We can now add the CPU name & PVR to the hardware description -seq_buf_printf (&ppc_hw_desc,"%s 0x%04lx " , cur_cpu_spec->cpu_name,mfspr (SPRN_PVR )); -/* Retrieve CPU related informations from the flat tree * (altivec support, boot CPU ID, ...) */
Signature · 1 line
2.43.0
RFC PATCH 4/5 powerpc/pseries: Prepare pseries_add_hw_description() for runtime use
In reply to Nathan Lynchpseries_add_hw_description() will be used after boot to update the hardware description string emitted in stack dumps. Remove the __init and make it take a seq_buf * parameter instead of referencing ppc_hw_desc directly.
--- arch/powerpc/platforms/pseries/setup.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/powerpc/platforms/pseries/setup.c +5 -5
@@ -1007,7 +1007,7 @@ static void __init pSeries_cmo_feature_init(void)pr_debug (" <- fw_cmo_feature_init()\n" ); } -static void __init pseries_add_hw_description (void ) +static void pseries_add_hw_description (struct seq_buf *sb) {struct device_node *dn;const char *s;
@@ -1015,7 +1015,7 @@ static void __init pseries_add_hw_description(void) dn =of_find_node_by_path ("/openprom" );if (dn) {if (of_property_read_string (dn,"model" , &s) ==0 ) -seq_buf_printf (&ppc_hw_desc,"of:%s " , s); +seq_buf_printf (sb,"of:%s " , s);of_node_put (dn); }
@@ -1023,7 +1023,7 @@ static void __init pseries_add_hw_description(void) dn =of_find_node_by_path ("/hypervisor" );if (dn) {if (of_property_read_string (dn,"compatible" , &s) ==0 ) -seq_buf_printf (&ppc_hw_desc,"hv:%s " , s); +seq_buf_printf (sb,"hv:%s " , s);of_node_put (dn);return ;
@@ -1031,7 +1031,7 @@ static void __init pseries_add_hw_description(void)if (of_property_read_bool (of_root,"ibm,powervm-partition" ) ||of_property_read_bool (of_root,"ibm,fw-net-version" )) -seq_buf_printf (&ppc_hw_desc,"hv:phyp " ); +seq_buf_printf (sb,"hv:phyp " ); }/*
@@ -1041,7 +1041,7 @@ static void __init pseries_init(void) {pr_debug (" -> pseries_init()\n" ); -pseries_add_hw_description (); +pseries_add_hw_description (&ppc_hw_desc);#ifdef CONFIG_HVC_CONSOLE if (firmware_has_feature (FW_FEATURE_LPAR ))
Signature · 1 line
2.43.0
RFC PATCH 5/5 powerpc/pseries: Update hardware description string after migration
In reply to Nathan LynchIntroduce code that rebuilds the short hardware description printed by stack traces. This sort of duplicates some code from boot (prom.c mainly), but that code populates the string as early as possible using APIs that aren't available later. So sharing all the code between the boot and runtime versions isn't feasible.
To prevent "drift" between the boot and runtime versions, rebuild the description using the new runtime APIs in a late initcall and warn if it doesn't match the one built earlier. The initcall also invokes dump_stack_update_arch_desc() twice to fully exercise it before any partition migration occurs. These checks could be dropped or made configurable later.
Call pseries_update_hw_description() immediately after updating the device tree when resuming from a partition migration.
--- arch/powerpc/platforms/pseries/mobility.c | 5 +++ arch/powerpc/platforms/pseries/pseries.h | 1 + arch/powerpc/platforms/pseries/setup.c | 70 +++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+)
arch/powerpc/platforms/pseries/mobility.c +5 -0
@@ -378,6 +378,11 @@ void post_mobility_fixup(void) rc =pseries_devicetree_update (MIGRATION_SCOPE );if (rc)pr_err ("device tree update failed: %d\n" , rc); +/* +* Rebuild the hardware description printed in stack traces +* using the updated device tree. +*/ +pseries_update_hw_description ();cacheinfo_rebuild ();
arch/powerpc/platforms/pseries/pseries.h +1 -0
@@ -56,6 +56,7 @@ extern int dlpar_acquire_drc(u32 drc_index);extern int dlpar_release_drc (u32 drc_index);extern int dlpar_unisolate_drc (u32 drc_index);extern void post_mobility_fixup (void ); +void pseries_update_hw_description (void );void queue_hotplug_event (struct pseries_hp_errorlog *hp_errlog);int handle_dlpar_errorlog (struct pseries_hp_errorlog *hp_errlog);
arch/powerpc/platforms/pseries/setup.c +70 -0
76 lines not on this page · raw message
Signature · 1 line
2.43.0