powerpc/crash: Kernel handling of CPU and memory hotplug
Subsystems (8)
v18 cover powerpc/crash: Kernel handling of CPU and memory hotplug
Commit 247262756121 ("crash: add generic infrastructure for crash hotplug support") added a generic infrastructure that allows architectures to selectively update the kdump image component during CPU or memory add/remove events within the kernel itself.
This patch series adds crash hotplug handler for PowerPC and enable support to update the kdump image on CPU/Memory add/remove events.
Among the 6 patches in this series, the first two patches make changes to the generic crash hotplug handler to assist PowerPC in adding support for this feature. The last four patches add support for this feature.
The following section outlines the problem addressed by this patch series, along with the current solution, its shortcomings, and the proposed resolution.
Problem:
========
Due to CPU/Memory hotplug or online/offline events the elfcorehdr (which describes the CPUs and memory of the crashed kernel) and FDT (Flattened Device Tree) of kdump image becomes outdated. Consequently, attempting dump collection with an outdated elfcorehdr or FDT can lead to failed or inaccurate dump collection.
Going forward CPU hotplug or online/offline events are referred as CPU/Memory add/remove events.
Existing solution and its shortcoming:
======================================
The current solution to address the above issue involves monitoring the CPU/memory add/remove events in userspace using udev rules and whenever there are changes in CPU and memory resources, the entire kdump image is loaded again. The kdump image includes kernel, initrd, elfcorehdr, FDT, purgatory. Given that only elfcorehdr and FDT get outdated due to CPU/Memory add/remove events, reloading the entire kdump image is inefficient. More importantly, kdump remains inactive for a substantial amount of time until the kdump reload completes.
Proposed solution:
==================
Instead of initiating a full kdump image reload from userspace on CPU/Memory hotplug and online/offline events, the proposed solution aims to update only the necessary kdump image component within the kernel itself.
Git tree for testing:
=====================
https://github.com/sourabhjains/linux/tree/kdump-in-kernel-crash-update-v18
Above tree is rebased on top of powerpc/next branch.
To realize this feature, the kdump udev rule must be updated. On RHEL, add the following two lines at the top of the "/usr/lib/udev/rules.d/98-kexec.rules" file.
SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end" SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
With the above change to the kdump udev rule, kdump reload is avoided during CPU/Memory add/remove events if this feature is enabled in the kernel.
Note: only kexec_file_load syscall will work. For kexec_load minor changes are required in kexec tool.
Changelog:
----------
v18: [No functional changes]
- Update a comment in 2/6.
- Describe the clean-up done on x86 in patch description 2/6.
- Fix a minor typo in the patch description of 3/6.
v17: [https://lore.kernel.org/all/[email protected]/]
- Rebase the patch series on top linux-next tree and below patch series
https://lore.kernel.org/all/[email protected]/
- Split 0003 patch from v16 into two patches
1. Move get_crash_memory_ranges() along with other *_memory_ranges() functions to ranges.c and make them public.
2. Make update_cpus_node function public and take this function out of file_load_64.c
- Keep arch_crash_hotplug_support in crash.c instead of core_64.c [05/06]
- Use CONFIG_CRASH_MAX_MEMORY_RANGES to find extra elfcorehdr size [06/06]
v16: [https://lore.kernel.org/all/[email protected]/]
- Remove the unused #define crash_hotplug_cpu_support
and crash_hotplug_memory_support in arch/x86/include/asm/kexec.h.
- Document why two kexec flag bits are used in
`arch_crash_hotplug_memory_support` (x86).
- Use a switch case to handle different hotplug operations in arch_crash_handle_hotplug_event for PowerPC.
- Fix a typo in 4/5.
v15:
- Remove the patch that adds a new kexec flag for FDT update.
- Introduce a generic kexec flag bit to share hotplug support intent between the kexec tool and the kernel for the kexec_load syscall. (2/5)
- Introduce an architecture-specific handler to process the kexec flag for crash hotplug support. (2/5)
- Rename the @update_elfcorehdr member of the struct kimage to
@hotplug_support. (2/5)
- Use a common function to advertise hotplug support for both CPU
and Memory. (2/5)v14:
- Fix build warnings by including necessary header files
- Rebase to v6.7-rc5
v13:
- Fix a build warning, take ranges.c out of CONFIG_KEXEC_FILE
- Rebase to v6.7-rc4
v12:
- A patch to add new kexec flags to support this feature on kexec_load system call
- Change in the way this feature is advertise to userspace for both kexec_load syscall
- Rebase to v6.6-rc7
v11:
- Rebase to v6.4-rc6
- The patch that introduced CONFIG_CRASH_HOTPLUG for PowerPC has been removed. The config is now part of common configuration:
https://lore.kernel.org/all/[email protected]/
v10:
- Drop the patch that adds fdt_index attribute to struct kimage_arch Find the fdt segment index when needed.
- Added more details into commits messages.
- Rebased onto 6.3.0-rc5
v9:
- Removed patch to prepare elfcorehdr crash notes for possible CPUs. The patch is moved to generic patch series that introduces generic infrastructure for in kernel crash update.
- Removed patch to pass the hotplug action type to the arch crash hotplug handler function. The generic patch series has introduced the hotplug action type in kimage struct.
- Add detail commit message for better understanding.
v8:
- Restrict fdt_index initialization to machine_kexec_post_load it work for both kexec_load and kexec_file_load.[3/8] Laurent Dufour
- Updated the logic to find the number of offline core. [6/8]
- Changed the logic to find the elfcore program header to accommodate future memory ranges due memory hotplug events. [8/8]
v7
- added a new config to configure this feature
- pass hotplug action type to arch specific handler
v6
- Added crash memory hotplug support
v5:
- Replace COFNIG_CRASH_HOTPLUG with CONFIG_HOTPLUG_CPU.
- Move fdt segment identification for kexec_load case to load path instead of crash hotplug handler
- Keep new attribute defined under kimage_arch to track FDT segment under CONFIG_HOTPLUG_CPU config.
v4:
- Update the logic to find the additional space needed for hotadd CPUs post kexec load. Refer "[RFC v4 PATCH 4/5] powerpc/crash hp: add crash hotplug support for kexec_file_load" patch to know more about the change.
- Fix a couple of typo.
- Replace pr_err to pr_info_once to warn user about memory hotplug support.
- In crash hotplug handle exit the for loop if FDT segment is found.
v3
- Move fdt_index and fdt_index_vaild variables to kimage_arch struct.
- Rebase patche on top of
https://lore.kernel.org/lkml/[email protected]/
- Fixed warning reported by checpatch script
v2:
- Use generic hotplug handler introduced by
https://lore.kernel.org/lkml/[email protected]/
a significant change from v1.
Sourabh Jain (6):
crash: forward memory_notify arg to arch crash hotplug handler
crash: add a new kexec flag for hotplug support powerpc/kexec: move *_memory_ranges functions to ranges.c PowerPC/kexec: make the update_cpus_node() function public powerpc/crash: add crash CPU hotplug support powerpc/crash: add crash memory hotplug support
arch/powerpc/Kconfig | 4 + arch/powerpc/include/asm/kexec.h | 15 ++ arch/powerpc/include/asm/kexec_ranges.h | 20 +- arch/powerpc/kexec/Makefile | 4 +- arch/powerpc/kexec/core_64.c | 91 +++++++ arch/powerpc/kexec/crash.c | 196 +++++++++++++++ arch/powerpc/kexec/elf_64.c | 3 +- arch/powerpc/kexec/file_load_64.c | 314 +++--------------------- arch/powerpc/kexec/ranges.c | 312 ++++++++++++++++++++++- arch/x86/include/asm/kexec.h | 13 +- arch/x86/kernel/crash.c | 32 ++- drivers/base/cpu.c | 2 +- drivers/base/memory.c | 2 +- include/linux/crash_core.h | 15 +- include/linux/kexec.h | 11 +- include/uapi/linux/kexec.h | 1 + kernel/crash_core.c | 29 +-- kernel/kexec.c | 4 +- kernel/kexec_file.c | 5 + 19 files changed, 714 insertions(+), 359 deletions(-)
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v18 1/6 crash: forward memory_notify arg to arch crash hotplug handler
In the event of memory hotplug or online/offline events, the crash memory hotplug notifier crash_memhp_notifier() receives a memory_notify object but doesn't forward that object to the generic and architecture-specific crash hotplug handler.
The memory_notify object contains the starting PFN (Page Frame Number) and the number of pages in the hot-removed memory. This information is necessary for architectures like PowerPC to update/recreate the kdump image, specifically elfcorehdr.
So update the function signature of crash_handle_hotplug_event() and arch_crash_handle_hotplug_event() to accept the memory_notify object as an argument from crash memory hotplug notifier.
Since no such object is available in the case of CPU hotplug event, the crash CPU hotplug notifier crash_cpuhp_online() passes NULL to the crash hotplug handler.
---
* No chnages in v18
arch/x86/include/asm/kexec.h | 2 +- arch/x86/kernel/crash.c | 4 +++- include/linux/crash_core.h | 2 +- kernel/crash_core.c | 14 +++++++------- 4 files changed, 12 insertions(+), 10 deletions(-)
arch/x86/include/asm/kexec.h +1 -1
@@ -207,7 +207,7 @@ int arch_kimage_file_post_load_cleanup(struct kimage *image);extern void kdump_nmi_shootdown_cpus (void );#ifdef CONFIG_CRASH_HOTPLUG -void arch_crash_handle_hotplug_event (struct kimage *image); +void arch_crash_handle_hotplug_event (struct kimage *image,void *arg);#define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event#ifdef CONFIG_HOTPLUG_CPU
arch/x86/kernel/crash.c +3 -1
@@ -432,10 +432,12 @@ unsigned int arch_crash_get_elfcorehdr_size(void)/** * arch_crash_handle_hotplug_event() - Handle hotplug elfcorehdr changes * @image: a pointer to kexec_crash_image +* @arg: struct memory_notify handler for memory hotplug case and +* NULL for CPU hotplug case. * * Prepare the new elfcorehdr and replace the existing elfcorehdr. */ -void arch_crash_handle_hotplug_event (struct kimage *image) +void arch_crash_handle_hotplug_event (struct kimage *image,void *arg) {void *elfbuf =NULL , *old_elfcorehdr;unsigned long nr_mem_ranges;
include/linux/crash_core.h +1 -1
@@ -37,7 +37,7 @@ static inline void arch_kexec_unprotect_crashkres(void) { }#ifndef arch_crash_handle_hotplug_event -static inline void arch_crash_handle_hotplug_event (struct kimage *image) { } +static inline void arch_crash_handle_hotplug_event (struct kimage *image,void *arg) { }#endif int crash_check_update_elfcorehdr (void );
kernel/crash_core.c +7 -7
@@ -534,7 +534,7 @@ int crash_check_update_elfcorehdr(void)* list of segments it checks (since the elfcorehdr changes and thus * would require an update to purgatory itself to update the digest). */ -static void crash_handle_hotplug_event (unsigned int hp_action,unsigned int cpu) +static void crash_handle_hotplug_event (unsigned int hp_action,unsigned int cpu,void *arg) {struct kimage *image;
@@ -596,7 +596,7 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu) image->hp_action = hp_action;/* Now invoke arch-specific update handler */ -arch_crash_handle_hotplug_event (image); +arch_crash_handle_hotplug_event (image, arg);/* No longer handling a hotplug event */ image->hp_action =KEXEC_CRASH_HP_NONE ;
@@ -612,17 +612,17 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu)crash_hotplug_unlock (); } -static int crash_memhp_notifier (struct notifier_block *nb,unsigned long val,void *v) +static int crash_memhp_notifier (struct notifier_block *nb,unsigned long val,void *arg) {switch (val) {case MEM_ONLINE :crash_handle_hotplug_event (KEXEC_CRASH_HP_ADD_MEMORY , -KEXEC_CRASH_HP_INVALID_CPU ); +KEXEC_CRASH_HP_INVALID_CPU , arg);break ;case MEM_OFFLINE :crash_handle_hotplug_event (KEXEC_CRASH_HP_REMOVE_MEMORY , -KEXEC_CRASH_HP_INVALID_CPU ); +KEXEC_CRASH_HP_INVALID_CPU , arg);break ; }return NOTIFY_OK ;
@@ -635,13 +635,13 @@ static struct notifier_block crash_memhp_nb = {static int crash_cpuhp_online (unsigned int cpu) { -crash_handle_hotplug_event (KEXEC_CRASH_HP_ADD_CPU , cpu); +crash_handle_hotplug_event (KEXEC_CRASH_HP_ADD_CPU , cpu,NULL );return 0 ; }static int crash_cpuhp_offline (unsigned int cpu) { -crash_handle_hotplug_event (KEXEC_CRASH_HP_REMOVE_CPU , cpu); +crash_handle_hotplug_event (KEXEC_CRASH_HP_REMOVE_CPU , cpu,NULL );return 0 ; }
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v18 2/6 crash: add a new kexec flag for hotplug support
In reply to Sourabh JainCommit a72bbec70da2 ("crash: hotplug support for kexec_load()") introduced a new kexec flag, KEXEC_UPDATE_ELFCOREHDR. Kexec tool uses this flag to indicate to the kernel that it is safe to modify the elfcorehdr of the kdump image loaded using the kexec_load system call.
However, it is possible that architectures may need to update kexec segments other then elfcorehdr. For example, FDT (Flatten Device Tree) on PowerPC. Introducing a new kexec flag for every new kexec segment may not be a good solution. Hence, a generic kexec flag bit, KEXEC_CRASH_HOTPLUG_SUPPORT, is introduced to share the CPU/Memory hotplug support intent between the kexec tool and the kernel for the kexec_load system call.
Now we have two kexec flags that enables crash hotplug support for kexec_load system call. First is KEXEC_UPDATE_ELFCOREHDR (only used in x86), and second is KEXEC_CRASH_HOTPLUG_SUPPORT (for all architectures).
To simplify the process of finding and reporting the crash hotplug support the following changes are introduced.
1. Define arch specific function to process the kexec flags and determine crash hotplug support
2. Rename the @update_elfcorehdr member of struct kimage to @hotplug_support and populate it for both kexec_load and kexec_file_load syscalls, because architecture can update more than one kexec segment
3. Let generic function crash_check_hotplug_support report hotplug support for loaded kdump image based on value of @hotplug_support
To bring the x86 crash hotplug support in line with the above points, the following changes have been made:
- Introduce the arch_crash_hotplug_support function to process kexec flags and determine crash hotplug support
- Remove the arch_crash_hotplug_[cpu|memory]_support functions
---
Changes in v18:
* Describe x86 changes in commit message
* Update comment of crash_check_hotplug_support()
arch/x86/include/asm/kexec.h | 11 ++--------- arch/x86/kernel/crash.c | 28 +++++++++++++++++----------- drivers/base/cpu.c | 2 +- drivers/base/memory.c | 2 +- include/linux/crash_core.h | 13 ++++++------- include/linux/kexec.h | 11 +++++++---- include/uapi/linux/kexec.h | 1 + kernel/crash_core.c | 15 ++++++--------- kernel/kexec.c | 4 ++-- kernel/kexec_file.c | 5 +++++ 10 files changed, 48 insertions(+), 44 deletions(-)
arch/x86/include/asm/kexec.h +2 -9
@@ -210,15 +210,8 @@ extern void kdump_nmi_shootdown_cpus(void);void arch_crash_handle_hotplug_event (struct kimage *image,void *arg);#define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event -#ifdef CONFIG_HOTPLUG_CPU -int arch_crash_hotplug_cpu_support (void ); -#define crash_hotplug_cpu_support arch_crash_hotplug_cpu_support -#endif - -#ifdef CONFIG_MEMORY_HOTPLUG -int arch_crash_hotplug_memory_support (void ); -#define crash_hotplug_memory_support arch_crash_hotplug_memory_support -#endif +int arch_crash_hotplug_support (struct kimage *image,unsigned long kexec_flags); +#define arch_crash_hotplug_support arch_crash_hotplug_supportunsigned int arch_crash_get_elfcorehdr_size (void );#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
arch/x86/kernel/crash.c +17 -11
@@ -402,20 +402,26 @@ int crash_load_segments(struct kimage *image)#undef pr_fmt#define pr_fmt (fmt) "crash hp: " fmt -/* These functions provide the value for the sysfs crash_hotplug nodes */ -#ifdef CONFIG_HOTPLUG_CPU -int arch_crash_hotplug_cpu_support (void ) +int arch_crash_hotplug_support (struct kimage *image,unsigned long kexec_flags) { -return crash_check_update_elfcorehdr (); -} -#endif -#ifdef CONFIG_MEMORY_HOTPLUG -int arch_crash_hotplug_memory_support (void ) -{ -return crash_check_update_elfcorehdr (); -} +#ifdef CONFIG_KEXEC_FILE +if (image->file_mode) +return 1 ;#endif +/* +* Initially, crash hotplug support for kexec_load was added +* with the KEXEC_UPDATE_ELFCOREHDR flag. Later, this +* functionality was expanded to accommodate multiple kexec +* segment updates, leading to the introduction of the +* KEXEC_CRASH_HOTPLUG_SUPPORT kexec flag bit. Consequently, +* when the kexec tool sends either of these flags, it indicates +* that the required kexec segment (elfcorehdr) is excluded from +* the SHA calculation. +*/ +return (kexec_flags &KEXEC_UPDATE_ELFCOREHDR || + kexec_flags &KEXEC_CRASH_HOTPLUG_SUPPORT ); +}unsigned int arch_crash_get_elfcorehdr_size (void ) {
drivers/base/cpu.c +1 -1
@@ -306,7 +306,7 @@ static ssize_t crash_hotplug_show(struct device *dev,struct device_attribute *attr, char *buf) { - return sysfs_emit(buf, "%d\n", crash_hotplug_cpu_support()); + return sysfs_emit(buf, "%d\n", crash_check_hotplug_support()); }static DEVICE_ATTR_ADMIN_RO (crash_hotplug);#endif
drivers/base/memory.c +1 -1
@@ -535,7 +535,7 @@ static DEVICE_ATTR_RW(auto_online_blocks);static ssize_t crash_hotplug_show (struct device *dev,struct device_attribute *attr,char *buf) { -return sysfs_emit (buf,"%d\n" ,crash_hotplug_memory_support ()); +return sysfs_emit (buf,"%d\n" ,crash_check_hotplug_support ()); }static DEVICE_ATTR_RO (crash_hotplug);#endif
include/linux/crash_core.h +6 -7
@@ -40,14 +40,13 @@ static inline void arch_kexec_unprotect_crashkres(void) { }static inline void arch_crash_handle_hotplug_event (struct kimage *image,void *arg) { }#endif -int crash_check_update_elfcorehdr (void ); +int crash_check_hotplug_support (void ); -#ifndef crash_hotplug_cpu_support -static inline int crash_hotplug_cpu_support (void ) {return 0 ; } -#endif - -#ifndef crash_hotplug_memory_support -static inline int crash_hotplug_memory_support (void ) {return 0 ; } +#ifndef arch_crash_hotplug_support +static inline int arch_crash_hotplug_support (struct kimage *image,unsigned long kexec_flags) +{ +return 0 ; +}#endif #ifndef crash_get_elfcorehdr_size
include/linux/kexec.h +7 -4
@@ -319,8 +319,10 @@ struct kimage {/* If set, we are using file mode kexec syscall */ unsigned int file_mode:1;#ifdef CONFIG_CRASH_HOTPLUG -/* If set, allow changes to elfcorehdr of kexec_load'd image */ - unsigned int update_elfcorehdr:1; +/* If set, it is safe to update kexec segments that are +* excluded from SHA calculation. +*/ + unsigned int hotplug_support:1;#endif #ifdef ARCH_HAS_KIMAGE_ARCH
@@ -391,9 +393,10 @@ bool kexec_load_permitted(int kexec_image_type);/* List of defined/legal kexec flags */ #ifndef CONFIG_KEXEC_JUMP -#define KEXEC_FLAGS (KEXEC_ON_CRASH | KEXEC_UPDATE_ELFCOREHDR) +#define KEXEC_FLAGS (KEXEC_ON_CRASH | KEXEC_UPDATE_ELFCOREHDR | KEXEC_CRASH_HOTPLUG_SUPPORT)#else -#define KEXEC_FLAGS (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT | KEXEC_UPDATE_ELFCOREHDR) +#define KEXEC_FLAGS (KEXEC_ON_CRASH | KEXEC_PRESERVE_CONTEXT | KEXEC_UPDATE_ELFCOREHDR | \ + KEXEC_CRASH_HOTPLUG_SUPPORT)#endif /* List of defined/legal kexec file flags */
include/uapi/linux/kexec.h +1 -0
@@ -13,6 +13,7 @@#define KEXEC_ON_CRASH 0x00000001#define KEXEC_PRESERVE_CONTEXT 0x00000002#define KEXEC_UPDATE_ELFCOREHDR 0x00000004 +#define KEXEC_CRASH_HOTPLUG_SUPPORT 0x00000008#define KEXEC_ARCH_MASK 0xffff0000/*
kernel/crash_core.c +6 -9
@@ -493,10 +493,10 @@ static DEFINE_MUTEX(__crash_hotplug_lock);/* * This routine utilized when the crash_hotplug sysfs node is read. -* It reflects the kernel's ability/permission to update the crash -* elfcorehdr directly. +* It reflects the kernel's ability/permission to update the kdump +* image directly. */ -int crash_check_update_elfcorehdr (void ) +int crash_check_hotplug_support (void ) {int rc =0 ;
@@ -508,10 +508,7 @@ int crash_check_update_elfcorehdr(void)return 0 ; }if (kexec_crash_image) { -if (kexec_crash_image->file_mode) - rc =1 ; -else - rc = kexec_crash_image->update_elfcorehdr; + rc = kexec_crash_image->hotplug_support; }/* Release lock now that update complete */ kexec_unlock ();
@@ -552,8 +549,8 @@ static void crash_handle_hotplug_event(unsigned int hp_action, unsigned int cpu, image = kexec_crash_image; -/* Check that updating elfcorehdr is permitted */ -if (!(image->file_mode || image->update_elfcorehdr)) +/* Check that kexec segments update is permitted */ +if (!image->hotplug_support) goto out; if (hp_action == KEXEC_CRASH_HP_ADD_CPU ||
kernel/kexec.c +2 -2
@@ -135,8 +135,8 @@ static int do_kexec_load(unsigned long entry, unsigned long nr_segments, image->preserve_context =1 ;#ifdef CONFIG_CRASH_HOTPLUG -if (flags &KEXEC_UPDATE_ELFCOREHDR ) - image->update_elfcorehdr =1 ; +if ((flags &KEXEC_ON_CRASH ) &&arch_crash_hotplug_support (image, flags)) + image->hotplug_support =1 ;#endif ret =machine_kexec_prepare (image);
kernel/kexec_file.c +5 -0
@@ -376,6 +376,11 @@ SYSCALL_DEFINE5(kexec_file_load, int, kernel_fd, int, initrd_fd,if (ret) goto out; +#ifdef CONFIG_CRASH_HOTPLUG +if ((flags &KEXEC_FILE_ON_CRASH ) &&arch_crash_hotplug_support (image, flags)) + image->hotplug_support =1 ; +#endif + ret =machine_kexec_prepare (image);if (ret) goto out;
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v18 3/6 powerpc/kexec: move *_memory_ranges functions to ranges.c
In reply to Sourabh JainMove the following functions form kexec/{file_load_64.c => ranges.c} and make them public so that components other than KEXEC_FILE can also use these functions.
1. get_exclude_memory_ranges
2. get_reserved_memory_ranges
3. get_crash_memory_ranges
4. get_usable_memory_ranges
Later in the series get_crash_memory_ranges function is utilized for in-kernel updates to kdump image during CPU/Memory hotplug or online/offline events for both kexec_load and kexec_file_load syscalls.
Since the above functions are moved to ranges.c, some of the helper functions in ranges.c are no longer required to be public. Mark them as static and removed them from kexec_ranges.h header file.
Finally, remove the CONFIG_KEXEC_FILE build dependency for range.c because it is required for other config, such as CONFIG_CRASH_DUMP.
No functional changes are intended.
---
Chnages in v18:
* Fix a typo in the commit message
arch/powerpc/include/asm/kexec_ranges.h | 19 +- arch/powerpc/kexec/Makefile | 4 +- arch/powerpc/kexec/file_load_64.c | 190 -------------------- arch/powerpc/kexec/ranges.c | 227 +++++++++++++++++++++++- 4 files changed, 224 insertions(+), 216 deletions(-)
arch/powerpc/include/asm/kexec_ranges.h +4 -15
@@ -7,19 +7,8 @@void sort_memory_ranges (struct crash_mem *mrngs,bool merge);struct crash_mem *realloc_mem_ranges (struct crash_mem **mem_ranges);int add_mem_range (struct crash_mem **mem_ranges,u64 base,u64 size); -int add_tce_mem_ranges (struct crash_mem **mem_ranges); -int add_initrd_mem_range (struct crash_mem **mem_ranges); -#ifdef CONFIG_PPC_64S_HASH_MMU -int add_htab_mem_range (struct crash_mem **mem_ranges); -#else -static inline int add_htab_mem_range (struct crash_mem **mem_ranges) -{ -return 0 ; -} -#endif -int add_kernel_mem_range (struct crash_mem **mem_ranges); -int add_rtas_mem_range (struct crash_mem **mem_ranges); -int add_opal_mem_range (struct crash_mem **mem_ranges); -int add_reserved_mem_ranges (struct crash_mem **mem_ranges); - +int get_exclude_memory_ranges (struct crash_mem **mem_ranges); +int get_reserved_memory_ranges (struct crash_mem **mem_ranges); +int get_crash_memory_ranges (struct crash_mem **mem_ranges); +int get_usable_memory_ranges (struct crash_mem **mem_ranges);#endif /* _ASM_POWERPC_KEXEC_RANGES_H */
arch/powerpc/kexec/Makefile +2 -2
@@ -3,11 +3,11 @@ # Makefile for the linux kernel. # -obj-y += core.o core_$(BITS).o +obj-y += core.o core_$(BITS).o ranges.o obj-$(CONFIG_PPC32) += relocate_32.o -obj-$(CONFIG_KEXEC_FILE) += file_load.o ranges.o file_load_$(BITS).o elf_$(BITS).o +obj-$(CONFIG_KEXEC_FILE) += file_load.o file_load_$(BITS).o elf_$(BITS).o obj-$(CONFIG_VMCORE_INFO) += vmcore_info.o obj-$(CONFIG_CRASH_DUMP) += crash.o
arch/powerpc/kexec/file_load_64.c +0 -190
202 lines not on this page · raw message
arch/powerpc/kexec/ranges.c +218 -9
310 lines not on this page · raw message
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v18 4/6 PowerPC/kexec: make the update_cpus_node() function public
In reply to Sourabh JainMove the update_cpus_node() from kexec/{file_load_64.c => core_64.c} to allow other kexec components to use it.
Later in the series, this function is used for in-kernel updates to the kdump image during CPU/memory hotplug or online/offline events for both kexec_load and kexec_file_load syscalls.
No functional changes are intended.
---
* No changes in v18
arch/powerpc/include/asm/kexec.h | 4 ++ arch/powerpc/kexec/core_64.c | 91 +++++++++++++++++++++++++++++++ arch/powerpc/kexec/file_load_64.c | 87 ----------------------------- 3 files changed, 95 insertions(+), 87 deletions(-)
arch/powerpc/include/asm/kexec.h +4 -0
@@ -185,6 +185,10 @@ static inline void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *))#endif /* CONFIG_CRASH_DUMP */ +#if defined(CONFIG_KEXEC_FILE ) || defined(CONFIG_CRASH_DUMP ) +int update_cpus_node (void *fdt); +#endif +#ifdef CONFIG_PPC_BOOK3S_64 #include <asm/book3s/64/kexec.h> #endif
arch/powerpc/kexec/core_64.c +91 -0
106 lines not on this page · raw message
arch/powerpc/kexec/file_load_64.c +0 -87
93 lines not on this page · raw message
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v18 5/6 powerpc/crash: add crash CPU hotplug support
In reply to Sourabh JainDue to CPU/Memory hotplug or online/offline events, the elfcorehdr (which describes the CPUs and memory of the crashed kernel) and FDT (Flattened Device Tree) of kdump image becomes outdated. Consequently, attempting dump collection with an outdated elfcorehdr or FDT can lead to failed or inaccurate dump collection.
Going forward, CPU hotplug or online/offline events are referred as CPU/Memory add/remove events.
The current solution to address the above issue involves monitoring the CPU/Memory add/remove events in userspace using udev rules and whenever there are changes in CPU and memory resources, the entire kdump image is loaded again. The kdump image includes kernel, initrd, elfcorehdr, FDT, purgatory. Given that only elfcorehdr and FDT get outdated due to CPU/Memory add/remove events, reloading the entire kdump image is inefficient. More importantly, kdump remains inactive for a substantial amount of time until the kdump reload completes.
To address the aforementioned issue, commit 247262756121 ("crash: add generic infrastructure for crash hotplug support") added a generic infrastructure that allows architectures to selectively update the kdump image component during CPU or memory add/remove events within the kernel itself.
In the event of a CPU or memory add/remove events, the generic crash hotplug event handler, crash_handle_hotplug_event(), is triggered. It then acquires the necessary locks to update the kdump image and invokes the architecture-specific crash hotplug handler, arch_crash_handle_hotplug_event(), to update the required kdump image components.
This patch adds crash hotplug handler for PowerPC and enable support to update the kdump image on CPU add/remove events. Support for memory add/remove events is added in a subsequent patch with the title "powerpc: add crash memory hotplug support"
As mentioned earlier, only the elfcorehdr and FDT kdump image components need to be updated in the event of CPU or memory add/remove events. However, on PowerPC architecture crash hotplug handler only updates the FDT to enable crash hotplug support for CPU add/remove events. Here's why.
The elfcorehdr on PowerPC is built with possible CPUs, and thus, it does not need an update on CPU add/remove events. On the other hand, the FDT needs to be updated on CPU add events to include the newly added CPU. If the FDT is not updated and the kernel crashes on a newly added CPU, the kdump kernel will fail to boot due to the unavailability of the crashing CPU in the FDT. During the early boot, it is expected that the boot CPU must be a part of the FDT; otherwise, the kernel will raise a BUG and fail to boot. For more information, refer to commit 36ae37e3436b0 ("powerpc: Make boot_cpuid common between 32 and 64-bit"). Since it is okay to have an offline CPU in the kdump FDT, no action is taken in case of CPU removal.
There are two system calls, kexec_file_load and kexec_load, used to load the kdump image. Few changes have been made to ensure kernel can safely update the FDT of kdump image loaded using both system calls.
For kexec_file_load syscall the kdump image is prepared in kernel. So to support an increasing number of CPUs, the FDT is constructed with extra buffer space to ensure it can accommodate a possible number of CPU nodes. Additionally, a call to fdt_pack (which trims the unused space once the FDT is prepared) is avoided if this feature is enabled.
For the kexec_load syscall, the FDT is updated only if the KEXEC_CRASH_HOTPLUG_SUPPORT kexec flag is passed to the kernel by userspace (kexec tools). When userspace passes this flag to the kernel, it indicates that the FDT is built to accommodate possible CPUs, and the FDT segment is excluded from SHA calculation, making it safe to update.
The changes related to this feature are kept under the CRASH_HOTPLUG config, and it is enabled by default.
---
* No changes in v18
arch/powerpc/Kconfig | 4 ++ arch/powerpc/include/asm/kexec.h | 8 +++ arch/powerpc/kexec/crash.c | 103 ++++++++++++++++++++++++++++++ arch/powerpc/kexec/elf_64.c | 3 +- arch/powerpc/kexec/file_load_64.c | 17 +++++ 5 files changed, 134 insertions(+), 1 deletion(-)
arch/powerpc/Kconfig +4 -0
@@ -686,6 +686,10 @@ config ARCH_SELECTS_CRASH_DUMP depends on CRASH_DUMP select RELOCATABLE if PPC64 || 44x || PPC_85xx +config ARCH_SUPPORTS_CRASH_HOTPLUG + def_bool y + depends on PPC64 + config FA_DUMP bool "Firmware-assisted dump" depends on CRASH_DUMP && PPC64 && (PPC_RTAS || PPC_POWERNV)
arch/powerpc/include/asm/kexec.h +8 -0
@@ -135,6 +135,14 @@ static inline void crash_setup_regs(struct pt_regs *newregs,ppc_save_regs (newregs); } +#ifdef CONFIG_CRASH_HOTPLUG +void arch_crash_handle_hotplug_event (struct kimage *image,void *arg); +#define arch_crash_handle_hotplug_event arch_crash_handle_hotplug_event + +int arch_crash_hotplug_support (struct kimage *image,unsigned long kexec_flags); +#define arch_crash_hotplug_support arch_crash_hotplug_support +#endif /* CONFIG_CRASH_HOTPLUG */ +extern int crashing_cpu;extern void crash_send_ipi (void (*crash_ipi_callback)(struct pt_regs *));extern void crash_ipi_callback (struct pt_regs *regs);
arch/powerpc/kexec/crash.c +103 -0
112 lines not on this page · raw message
arch/powerpc/kexec/elf_64.c +2 -1
@@ -116,7 +116,8 @@ static void *elf64_load(struct kimage *image, char *kernel_buf,if (ret) goto out_free_fdt; -fdt_pack (fdt); +if (!IS_ENABLED (CONFIG_CRASH_HOTPLUG ) || image->type !=KEXEC_TYPE_CRASH ) +fdt_pack (fdt); kbuf.buffer = fdt; kbuf.bufsz = kbuf.memsz =fdt_totalsize (fdt);
arch/powerpc/kexec/file_load_64.c +17 -0
@@ -30,6 +30,7 @@#include <asm/iommu.h> #include <asm/prom.h> #include <asm/plpks.h> +#include <asm/cputhreads.h> struct umem_info {__be64 *buf;/* data buffer for usable-memory property */
@@ -789,6 +790,9 @@ static unsigned int kdump_extra_fdt_size_ppc64(struct kimage *image)unsigned int cpu_nodes, extra_size =0 ;struct device_node *dn;u64 usm_entries; +#ifdef CONFIG_CRASH_HOTPLUG +unsigned int possible_cpu_nodes; +#endif if (!IS_ENABLED (CONFIG_CRASH_DUMP ) || image->type !=KEXEC_TYPE_CRASH )return 0 ;
@@ -816,6 +820,19 @@ static unsigned int kdump_extra_fdt_size_ppc64(struct kimage *image)if (cpu_nodes > boot_cpu_node_count) extra_size += (cpu_nodes - boot_cpu_node_count) *cpu_node_size (); +#ifdef CONFIG_CRASH_HOTPLUG +/* +* Make sure enough space is reserved to accommodate possible CPU nodes +* in the crash FDT. This allows packing possible CPU nodes which are +* not yet present in the system without regenerating the entire FDT. +*/ +if (image->type ==KEXEC_TYPE_CRASH ) { + possible_cpu_nodes =num_possible_cpus () / threads_per_core; +if (possible_cpu_nodes > cpu_nodes) + extra_size += (possible_cpu_nodes - cpu_nodes) *cpu_node_size (); + } +#endif +return extra_size; }
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH v18 6/6 powerpc/crash: add crash memory hotplug support
In reply to Sourabh JainExtend the arch crash hotplug handler, as introduced by the patch title ("powerpc: add crash CPU hotplug support"), to also support memory add/remove events.
Elfcorehdr describes the memory of the crash kernel to capture the kernel; hence, it needs to be updated if memory resources change due to memory add/remove events. Therefore, arch_crash_handle_hotplug_event() is updated to recreate the elfcorehdr and replace it with the previous one on memory add/remove events.
The memblock list is used to prepare the elfcorehdr. In the case of memory hot remove, the memblock list is updated after the arch crash hotplug handler is triggered, as depicted in Figure 1. Thus, the hot-removed memory is explicitly removed from the crash memory ranges to ensure that the memory ranges added to elfcorehdr do not include the hot-removed memory.
Memory remove
|
v
Offline pages
|
v
Initiate memory notify call <----> crash hotplug handler
chain for MEM_OFFLINE event
|
vUpdate memblock list
Figure 1
There are two system calls, kexec_file_load and kexec_load, used to load the kdump image. A few changes have been made to ensure that the kernel can safely update the elfcorehdr component of the kdump image for both system calls.
For the kexec_file_load syscall, kdump image is prepared in the kernel. To support an increasing number of memory regions, the elfcorehdr is built with extra buffer space to ensure that it can accommodate additional memory ranges in future.
For the kexec_load syscall, the elfcorehdr is updated only if the KEXEC_CRASH_HOTPLUG_SUPPORT kexec flag is passed to the kernel by the kexec tool. Passing this flag to the kernel indicates that the elfcorehdr is built to accommodate additional memory ranges and the elfcorehdr segment is not considered for SHA calculation, making it safe to update.
The changes related to this feature are kept under the CRASH_HOTPLUG config, and it is enabled by default.
---
* No changes in v18.
arch/powerpc/include/asm/kexec.h | 3 + arch/powerpc/include/asm/kexec_ranges.h | 1 + arch/powerpc/kexec/crash.c | 95 ++++++++++++++++++++++++- arch/powerpc/kexec/file_load_64.c | 20 +++++- arch/powerpc/kexec/ranges.c | 85 ++++++++++++++++++++++ 5 files changed, 202 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/kexec.h +3 -0
@@ -141,6 +141,9 @@ void arch_crash_handle_hotplug_event(struct kimage *image, void *arg);int arch_crash_hotplug_support (struct kimage *image,unsigned long kexec_flags);#define arch_crash_hotplug_support arch_crash_hotplug_support + +unsigned int arch_crash_get_elfcorehdr_size (void ); +#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size#endif /* CONFIG_CRASH_HOTPLUG */ extern int crashing_cpu;
arch/powerpc/include/asm/kexec_ranges.h +1 -0
@@ -7,6 +7,7 @@void sort_memory_ranges (struct crash_mem *mrngs,bool merge);struct crash_mem *realloc_mem_ranges (struct crash_mem **mem_ranges);int add_mem_range (struct crash_mem **mem_ranges,u64 base,u64 size); +int remove_mem_range (struct crash_mem **mem_ranges,u64 base,u64 size);int get_exclude_memory_ranges (struct crash_mem **mem_ranges);int get_reserved_memory_ranges (struct crash_mem **mem_ranges);int get_crash_memory_ranges (struct crash_mem **mem_ranges);
arch/powerpc/kexec/crash.c +94 -1
125 lines not on this page · raw message
arch/powerpc/kexec/file_load_64.c +19 -1
@@ -595,6 +595,23 @@ static void update_backup_region_phdr(struct kimage *image, Elf64_Ehdr *ehdr) } } +static unsigned int kdump_extra_elfcorehdr_size (struct crash_mem *cmem) +{ +#if defined(CONFIG_CRASH_HOTPLUG ) && defined(CONFIG_MEMORY_HOTPLUG ) +unsigned int extra_sz =0 ; + +if (CONFIG_CRASH_MAX_MEMORY_RANGES > (unsigned int )PN_XNUM ) +pr_warn ("Number of Phdrs %u exceeds max\n" ,CONFIG_CRASH_MAX_MEMORY_RANGES ); +else if (cmem->nr_ranges >=CONFIG_CRASH_MAX_MEMORY_RANGES ) +pr_warn ("Configured crash mem ranges may not be enough\n" ); +else + extra_sz = (CONFIG_CRASH_MAX_MEMORY_RANGES - cmem->nr_ranges) *sizeof (Elf64_Phdr); + +return extra_sz; +#endif +return 0 ; +} +/** * load_elfcorehdr_segment - Setup crash memory ranges and initialize elfcorehdr * segment needed to load kdump kernel.
@@ -626,7 +643,8 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf) kbuf->buffer = headers; kbuf->mem =KEXEC_BUF_MEM_UNKNOWN ; - kbuf->bufsz = kbuf->memsz = headers_sz; + kbuf->bufsz = headers_sz; + kbuf->memsz = headers_sz +kdump_extra_elfcorehdr_size (cmem); kbuf->top_down = false; ret =kexec_add_buffer (kbuf);
arch/powerpc/kexec/ranges.c +85 -0
89 lines not on this page · raw message
Signature · 7 lines
2.43.0 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Michael Ellerman
AppliedQuoted 9 lines · Sourabh Jain
Commit 247262756121 ("crash: add generic infrastructure for crash hotplug support") added a generic infrastructure that allows architectures to selectively update the kdump image component during CPU or memory add/remove events within the kernel itself.
This patch series adds crash hotplug handler for PowerPC and enable support to update the kdump image on CPU/Memory add/remove events.
[...]
Applied to powerpc/topic/kdump-hotplug.
[1/6] crash: forward memory_notify arg to arch crash hotplug handler
https://git.kernel.org/powerpc/c/118005713e35a1893c6ee47ab2926cca277737de [2/6] crash: add a new kexec flag for hotplug support https://git.kernel.org/powerpc/c/79365026f86948b52c3cb7bf099dded92c559b4c [3/6] powerpc/kexec: move *_memory_ranges functions to ranges.c https://git.kernel.org/powerpc/c/f5f0da5a7b18fab383bac92044fd8f4f288c9d38 [4/6] PowerPC/kexec: make the update_cpus_node() function public https://git.kernel.org/powerpc/c/0857beff9c1ec8bb421a8b7a721da0f34cc886c0 [5/6] powerpc/crash: add crash CPU hotplug support https://git.kernel.org/powerpc/c/b741092d59761b98781fcb4f3f521312ed8d5006 [6/6] powerpc/crash: add crash memory hotplug support https://git.kernel.org/powerpc/c/849599b702ef8977fcd5b2f27c61ef773c42bb88
cheers
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 16 lines · Sourabh Jain
Commit 247262756121 ("crash: add generic infrastructure for crash hotplug support") added a generic infrastructure that allows architectures to selectively update the kdump image component during CPU or memory add/remove events within the kernel itself.
This patch series adds crash hotplug handler for PowerPC and enable support to update the kdump image on CPU/Memory add/remove events.
Among the 6 patches in this series, the first two patches make changes to the generic crash hotplug handler to assist PowerPC in adding support for this feature. The last four patches add support for this feature.
The following section outlines the problem addressed by this patch series, along with the current solution, its shortcomings, and the proposed resolution.
Quoted 7 lines
Problem:
========
Due to CPU/Memory hotplug or online/offline events the elfcorehdr (which describes the CPUs and memory of the crashed kernel) and FDT (Flattened Device Tree) of kdump image becomes outdated. Consequently, attempting dump collection with an outdated elfcorehdr or FDT can lead to failed or inaccurate dump collection.
Hi, Sourabh, are there any specific methods to reproduce the scenarios for this feature? I would like to port this feature to ARM64, but I don't know how to reproduce the issue.
Quoted 205 lines
Going forward CPU hotplug or online/offline events are referred as CPU/Memory add/remove events.
Existing solution and its shortcoming:
======================================
The current solution to address the above issue involves monitoring the CPU/memory add/remove events in userspace using udev rules and whenever there are changes in CPU and memory resources, the entire kdump image is loaded again. The kdump image includes kernel, initrd, elfcorehdr, FDT, purgatory. Given that only elfcorehdr and FDT get outdated due to CPU/Memory add/remove events, reloading the entire kdump image is inefficient. More importantly, kdump remains inactive for a substantial amount of time until the kdump reload completes.Proposed solution:
==================
Instead of initiating a full kdump image reload from userspace on CPU/Memory hotplug and online/offline events, the proposed solution aims to update only the necessary kdump image component within the kernel itself.Git tree for testing:
=====================
https://github.com/sourabhjains/linux/tree/kdump-in-kernel-crash-update-v18Above tree is rebased on top of powerpc/next branch.
To realize this feature, the kdump udev rule must be updated. On RHEL, add the following two lines at the top of the "
/usr/lib/udev/rules.d/98-kexec.rules" file.SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end" SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
With the above change to the kdump udev rule, kdump reload is avoided during CPU/Memory add/remove events if this feature is enabled in the kernel.
Note: only
kexec_file_loadsyscall will work. Forkexec_loadminor changes are required in kexec tool.Changelog:
----------
v18: [No functional changes]
- Update a comment in 2/6.
- Describe the clean-up done on x86 in patch description 2/6.
- Fix a minor typo in the patch description of 3/6.v17: [https://lore.kernel.org/all/[email protected]/]
- Rebase the patch series on top linux-next tree and below patch serieshttps://lore.kernel.org/all/[email protected]/- Split 0003 patch from v16 into two patches
1. Moveget_crash_memory_ranges()along with other*_memory_ranges()functions toranges.cand make them public.
2. Makeupdate_cpus_nodefunction public and take this function out offile_load_64.c
- Keeparch_crash_hotplug_supportincrash.cinstead ofcore_64.c[05/06]
- UseCONFIG_CRASH_MAX_MEMORY_RANGESto find extra elfcorehdr size [06/06]v16: [https://lore.kernel.org/all/[email protected]/]
- Remove the unused #definecrash_hotplug_cpu_support
andcrash_hotplug_memory_supportinarch/x86/include/asm/kexec.h.
- Document why two kexec flag bits are used in`arch_crash_hotplug_memory_support` (x86).- Use a switch case to handle different hotplug operations in
arch_crash_handle_hotplug_eventfor PowerPC.
- Fix a typo in 4/5.v15:
- Remove the patch that adds a new kexec flag for FDT update.
- Introduce a generic kexec flag bit to share hotplug support intent between the kexec tool and the kernel for thekexec_loadsyscall. (2/5)
- Introduce an architecture-specific handler to process the kexec flag for crash hotplug support. (2/5)
- Rename the @update_elfcorehdr member of the struct kimage to@hotplug_support. (2/5) - Use a common function to advertise hotplug support for both CPU and Memory. (2/5)v14:
- Fix build warnings by including necessary header files
- Rebase to v6.7-rc5v13:
- Fix a build warning, takeranges.cout ofCONFIG_KEXEC_FILE
- Rebase to v6.7-rc4v12:
- A patch to add new kexec flags to support this feature onkexec_loadsystem call
- Change in the way this feature is advertise to userspace for bothkexec_loadsyscall
- Rebase to v6.6-rc7v11:
- Rebase to v6.4-rc6
- The patch that introducedCONFIG_CRASH_HOTPLUGfor PowerPC has been removed. The config is now part of common configuration:
https://lore.kernel.org/all/[email protected]/v10:
- Drop the patch that addsfdt_indexattribute to structkimage_archFind the fdt segment index when needed.
- Added more details into commits messages.
- Rebased onto 6.3.0-rc5v9:
- Removed patch to prepare elfcorehdr crash notes for possible CPUs. The patch is moved to generic patch series that introduces generic infrastructure for in kernel crash update.
- Removed patch to pass the hotplug action type to the arch crash hotplug handler function. The generic patch series has introduced the hotplug action type in kimage struct.
- Add detail commit message for better understanding.v8:
- Restrictfdt_indexinitialization tomachine_kexec_post_loadit work for bothkexec_loadand kexec_file_load.[3/8] Laurent Dufour- Updated the logic to find the number of offline core. [6/8]
- Changed the logic to find the elfcore program header to accommodate future memory ranges due memory hotplug events. [8/8]
v7
- added a new config to configure this feature
- pass hotplug action type to arch specific handlerv6
- Added crash memory hotplug supportv5:
- ReplaceCOFNIG_CRASH_HOTPLUGwithCONFIG_HOTPLUG_CPU.
- Move fdt segment identification forkexec_loadcase to load path instead of crash hotplug handler
- Keep new attribute defined underkimage_archto track FDT segment underCONFIG_HOTPLUG_CPUconfig.v4:
- Update the logic to find the additional space needed for hotadd CPUs post kexec load. Refer "[RFC v4 PATCH 4/5] powerpc/crash hp: add crash hotplug support forkexec_file_load" patch to know more about the change.
- Fix a couple of typo.
- Replacepr_errtopr_info_onceto warn user about memory hotplug support.
- In crash hotplug handle exit the for loop if FDT segment is found.v3
- Movefdt_indexandfdt_index_vaildvariables tokimage_archstruct.
- Rebase patche on top ofhttps://lore.kernel.org/lkml/[email protected]/- Fixed warning reported by checpatch script
v2:
- Use generic hotplug handler introduced by
https://lore.kernel.org/lkml/[email protected]/
a significant change from v1.Cc Akhil Raj <[email protected]>Cc Andrew Morton <[email protected]>Cc Aneesh Kumar K.V <[email protected]>Cc Baoquan He <[email protected]>Cc Borislav Petkov (AMD) <[email protected]>Cc Boris Ostrovsky <[email protected]>Cc Christophe Leroy <[email protected]>Cc Dave Hansen <[email protected]>Cc Dave Young <[email protected]>Cc David Hildenbrand <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Hari Bathini <[email protected]>Cc Laurent Dufour <[email protected]>Cc Mahesh Salgaonkar <[email protected]>Cc Michael Ellerman <[email protected]>Cc Mimi Zohar <[email protected]>Cc Naveen N Rao <[email protected]>Cc Oscar Salvador <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Valentin Schneider <[email protected]>Cc Vivek Goyal <[email protected]>Sourabh Jain (6):
crash: forwardmemory_notifyarg to arch crash hotplug handler
crash: add a new kexec flag for hotplug support powerpc/kexec: move*_memory_rangesfunctions toranges.cPowerPC/kexec: make theupdate_cpus_node()function public powerpc/crash: add crash CPU hotplug support powerpc/crash: add crash memory hotplug supportarch/powerpc/Kconfig | 4 + arch/powerpc/include/asm/kexec.h | 15 ++ arch/powerpc/include/asm/kexec_ranges.h | 20 +- arch/powerpc/kexec/Makefile | 4 +- arch/powerpc/kexec/core_64.c | 91 +++++++ arch/powerpc/kexec/crash.c | 196 +++++++++++++++ arch/powerpc/kexec/elf_64.c | 3 +- arch/powerpc/kexec/file_load_64.c | 314 +++--------------------- arch/powerpc/kexec/ranges.c | 312 ++++++++++++++++++++++- arch/x86/include/asm/kexec.h | 13 +- arch/x86/kernel/crash.c | 32 ++- drivers/base/cpu.c | 2 +-
Quoted 9 lines
drivers/base/memory.c | 2 +- include/linux/crash_core.h | 15 +- include/linux/kexec.h | 11 +- include/uapi/linux/kexec.h | 1 + kernel/crash_core.c | 29 +-- kernel/kexec.c | 4 +- kernel/kexec_file.c | 5 + 19 files changed, 714 insertions(+), 359 deletions(-)
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Jinjie,
Quoted 25 lines · Jinjie Ruan
Quoted 16 lines · Sourabh Jain
Commit 247262756121 ("crash: add generic infrastructure for crash hotplug support") added a generic infrastructure that allows architectures to selectively update the kdump image component during CPU or memory add/remove events within the kernel itself.
This patch series adds crash hotplug handler for PowerPC and enable support to update the kdump image on CPU/Memory add/remove events.
Among the 6 patches in this series, the first two patches make changes to the generic crash hotplug handler to assist PowerPC in adding support for this feature. The last four patches add support for this feature.
The following section outlines the problem addressed by this patch series, along with the current solution, its shortcomings, and the proposed resolution.
Quoted 7 lines
Problem:
========
Due to CPU/Memory hotplug or online/offline events the elfcorehdr (which describes the CPUs and memory of the crashed kernel) and FDT (Flattened Device Tree) of kdump image becomes outdated. Consequently, attempting dump collection with an outdated elfcorehdr or FDT can lead to failed or inaccurate dump collection.
Quoted 3 lines
Hi, Sourabh, are there any specific methods to reproduce the scenarios for this feature? I would like to port this feature to ARM64, but I don't know how to reproduce the issue.
On PowerPC, this issue is reproducible if the kernel crashes after bulk CPU or memory hotplug operations. Try the same on ARM64; you might be able to reproduce the issue.
Note: I used to hotplug hundreds of LMBs (Logical Memory Blocks), each with a size of 256 MB, to easily reproduce this issue.
To hotplug CPU and memory, I used to use a PowerPC-specific tool. You may need to find one that is available for ARM64.
Along with fixing the problem mentioned above, this feature also significantly reduces the kdump service downtime. How?
1. Handle the kdump image update on CPU/Memory hotplug events in the kernel itself, without userspace intervention.
2. Only recreate/update the relevant kexec segment instead of reloading all segments.
Thanks,
Sourabh Jain
Quoted 205 lines
Quoted 205 lines
Going forward CPU hotplug or online/offline events are referred as CPU/Memory add/remove events.
Existing solution and its shortcoming:
======================================
The current solution to address the above issue involves monitoring the CPU/memory add/remove events in userspace using udev rules and whenever there are changes in CPU and memory resources, the entire kdump image is loaded again. The kdump image includes kernel, initrd, elfcorehdr, FDT, purgatory. Given that only elfcorehdr and FDT get outdated due to CPU/Memory add/remove events, reloading the entire kdump image is inefficient. More importantly, kdump remains inactive for a substantial amount of time until the kdump reload completes.Proposed solution:
==================
Instead of initiating a full kdump image reload from userspace on CPU/Memory hotplug and online/offline events, the proposed solution aims to update only the necessary kdump image component within the kernel itself.Git tree for testing:
=====================
https://github.com/sourabhjains/linux/tree/kdump-in-kernel-crash-update-v18Above tree is rebased on top of powerpc/next branch.
To realize this feature, the kdump udev rule must be updated. On RHEL, add the following two lines at the top of the "
/usr/lib/udev/rules.d/98-kexec.rules" file.SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end" SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
With the above change to the kdump udev rule, kdump reload is avoided during CPU/Memory add/remove events if this feature is enabled in the kernel.
Note: only
kexec_file_loadsyscall will work. Forkexec_loadminor changes are required in kexec tool.Changelog:
----------
v18: [No functional changes]
- Update a comment in 2/6.
- Describe the clean-up done on x86 in patch description 2/6.
- Fix a minor typo in the patch description of 3/6.v17: [https://lore.kernel.org/all/[email protected]/]
- Rebase the patch series on top linux-next tree and below patch serieshttps://lore.kernel.org/all/[email protected]/- Split 0003 patch from v16 into two patches
1. Moveget_crash_memory_ranges()along with other*_memory_ranges()functions toranges.cand make them public.
2. Makeupdate_cpus_nodefunction public and take this function out offile_load_64.c
- Keeparch_crash_hotplug_supportincrash.cinstead ofcore_64.c[05/06]
- UseCONFIG_CRASH_MAX_MEMORY_RANGESto find extra elfcorehdr size [06/06]v16: [https://lore.kernel.org/all/[email protected]/]
- Remove the unused #definecrash_hotplug_cpu_support
andcrash_hotplug_memory_supportinarch/x86/include/asm/kexec.h.
- Document why two kexec flag bits are used in`arch_crash_hotplug_memory_support` (x86).- Use a switch case to handle different hotplug operations in
arch_crash_handle_hotplug_eventfor PowerPC.
- Fix a typo in 4/5.v15:
- Remove the patch that adds a new kexec flag for FDT update.
- Introduce a generic kexec flag bit to share hotplug support intent between the kexec tool and the kernel for thekexec_loadsyscall. (2/5)
- Introduce an architecture-specific handler to process the kexec flag for crash hotplug support. (2/5)
- Rename the @update_elfcorehdr member of the struct kimage to@hotplug_support. (2/5) - Use a common function to advertise hotplug support for both CPU and Memory. (2/5)v14:
- Fix build warnings by including necessary header files
- Rebase to v6.7-rc5v13:
- Fix a build warning, takeranges.cout ofCONFIG_KEXEC_FILE
- Rebase to v6.7-rc4v12:
- A patch to add new kexec flags to support this feature onkexec_loadsystem call
- Change in the way this feature is advertise to userspace for bothkexec_loadsyscall
- Rebase to v6.6-rc7v11:
- Rebase to v6.4-rc6
- The patch that introducedCONFIG_CRASH_HOTPLUGfor PowerPC has been removed. The config is now part of common configuration:
https://lore.kernel.org/all/[email protected]/v10:
- Drop the patch that addsfdt_indexattribute to structkimage_archFind the fdt segment index when needed.
- Added more details into commits messages.
- Rebased onto 6.3.0-rc5v9:
- Removed patch to prepare elfcorehdr crash notes for possible CPUs. The patch is moved to generic patch series that introduces generic infrastructure for in kernel crash update.
- Removed patch to pass the hotplug action type to the arch crash hotplug handler function. The generic patch series has introduced the hotplug action type in kimage struct.
- Add detail commit message for better understanding.v8:
- Restrictfdt_indexinitialization tomachine_kexec_post_loadit work for bothkexec_loadand kexec_file_load.[3/8] Laurent Dufour- Updated the logic to find the number of offline core. [6/8]
- Changed the logic to find the elfcore program header to accommodate future memory ranges due memory hotplug events. [8/8]
v7
- added a new config to configure this feature
- pass hotplug action type to arch specific handlerv6
- Added crash memory hotplug supportv5:
- ReplaceCOFNIG_CRASH_HOTPLUGwithCONFIG_HOTPLUG_CPU.
- Move fdt segment identification forkexec_loadcase to load path instead of crash hotplug handler
- Keep new attribute defined underkimage_archto track FDT segment underCONFIG_HOTPLUG_CPUconfig.v4:
- Update the logic to find the additional space needed for hotadd CPUs post kexec load. Refer "[RFC v4 PATCH 4/5] powerpc/crash hp: add crash hotplug support forkexec_file_load" patch to know more about the change.
- Fix a couple of typo.
- Replacepr_errtopr_info_onceto warn user about memory hotplug support.
- In crash hotplug handle exit the for loop if FDT segment is found.v3
- Movefdt_indexandfdt_index_vaildvariables tokimage_archstruct.
- Rebase patche on top ofhttps://lore.kernel.org/lkml/[email protected]/- Fixed warning reported by checpatch script
v2:
- Use generic hotplug handler introduced by
https://lore.kernel.org/lkml/[email protected]/
a significant change from v1.Cc Akhil Raj <[email protected]>Cc Andrew Morton <[email protected]>Cc Aneesh Kumar K.V <[email protected]>Cc Baoquan He <[email protected]>Cc Borislav Petkov (AMD) <[email protected]>Cc Boris Ostrovsky <[email protected]>Cc Christophe Leroy <[email protected]>Cc Dave Hansen <[email protected]>Cc Dave Young <[email protected]>Cc David Hildenbrand <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Hari Bathini <[email protected]>Cc Laurent Dufour <[email protected]>Cc Mahesh Salgaonkar <[email protected]>Cc Michael Ellerman <[email protected]>Cc Mimi Zohar <[email protected]>Cc Naveen N Rao <[email protected]>Cc Oscar Salvador <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Valentin Schneider <[email protected]>Cc Vivek Goyal <[email protected]>Sourabh Jain (6):
crash: forwardmemory_notifyarg to arch crash hotplug handler
crash: add a new kexec flag for hotplug support powerpc/kexec: move*_memory_rangesfunctions toranges.cPowerPC/kexec: make theupdate_cpus_node()function public powerpc/crash: add crash CPU hotplug support powerpc/crash: add crash memory hotplug supportarch/powerpc/Kconfig | 4 + arch/powerpc/include/asm/kexec.h | 15 ++ arch/powerpc/include/asm/kexec_ranges.h | 20 +- arch/powerpc/kexec/Makefile | 4 +- arch/powerpc/kexec/core_64.c | 91 +++++++ arch/powerpc/kexec/crash.c | 196 +++++++++++++++ arch/powerpc/kexec/elf_64.c | 3 +- arch/powerpc/kexec/file_load_64.c | 314 +++--------------------- arch/powerpc/kexec/ranges.c | 312 ++++++++++++++++++++++- arch/x86/include/asm/kexec.h | 13 +- arch/x86/kernel/crash.c | 32 ++- drivers/base/cpu.c | 2 +- drivers/base/memory.c | 2 +-
Quoted 8 lines
Quoted 8 lines
include/linux/crash_core.h | 15 +- include/linux/kexec.h | 11 +- include/uapi/linux/kexec.h | 1 + kernel/crash_core.c | 29 +-- kernel/kexec.c | 4 +- kernel/kexec_file.c | 5 + 19 files changed, 714 insertions(+), 359 deletions(-)
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec