Skip to contentSkip to conversation

In kernel handling of CPU hotplug events for crash kernel

RFC PATCH Superseded Sourabh Jain 4 participants 19 messages Apr 11, 2022 – Apr 26, 2022

Subsystems (3)

v4 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v4 cover In kernel handling of CPU hotplug events for crash kernel

This patch series implements the crash hotplug handler on PowerPC introduced by https://lkml.org/lkml/2022/3/3/674 patch series.

The Problem:
============
Post hotplug/DLPAR events the capture kernel holds stale information about the system. Dump collection with stale capture kernel might end up in dump capture failure or an inaccurate dump collection.

Existing solution:
==================
The existing solution to keep the capture kernel up-to-date is observe the hotplug event via udev rule and trigger a full capture kernel reload post hotplug event.

Shortcomings:
------------------------------------------------
- Leaves a window where kernel crash might not lead to successful dump collection.
- Reloading all kexec components for each hotplug is inefficient. Since only one or two kexec components need to be updated due to hotplug event reloading all kexec component is redundant.
- udev rules are prone to races if hotplug events are frequent.

More about issues with an existing solution is posted here:
- https://lkml.org/lkml/2020/12/14/532
- https://lists.ozlabs.org/pipermail/linuxppc-dev/2022-February/240254.html

Proposed Solution:
==================
Instead of reloading all kexec segments on hotplug event, this patch series focuses on updating only the relevant kexec segment. Once the kexec segments are loaded in the kernel reserved area then an arch-specific hotplug handler will update the relevant kexec segment based on hotplug event type.

As mentioned above this patch series implemented a PowerPC crash hotplug handler for the CPU. The crash hotplug handler memory is in our TODO list.

A couple of minor changes are required to realize the benefit of the patch series:

- disalble the udev rule:

comment out the below line in kdump udev rule file:
RHEL: /usr/lib/udev/rules.d/98-kexec.rules
# SUBSYSTEM=="cpu", ACTION=="online", GOTO="kdump_reload_cpu"

- kexec tool needs to be updated with patch for kexec_load system call to work (not needed if -s option is used during kexec panic load):

---

kexec/arch/ppc64/kexec-elf-ppc64.c +31 -1
index 695b8b0..1dc6490 100644
@@ -45,6 +45,29 @@ uint64_t initrd_base, initrd_size;
 unsigned char reuse_initrd = 0;
 const char *ramdisk;
 
+#define MAX_CORE 256
+#define PER_CORE_NODE_SIZE 1500
+
+/**
+ * get_crash_fdt_mem_sz() - calcuate mem size for crash kernel FDT
+ * @fdt: pointer to crash kernel FDT
+ *
+ * Calculate the buffer space needed to add more CPU nodes in FDT after
+ * capture kenrel load due to hot add events.
+ *
+ * Some assumption are made to calculate the additional buffer size needed
+ * to accommodate future hot add CPUs to the crash FDT. The maximum core count
+ * in the system would not go beyond MAX_CORE and memory needed to store per core
+ * date in FDT is PER_CORE_NODE_SIZE.
+ *
+ * Certainly MAX_CORE count can be replaced with possible core count and
+ * PER_CORE_NODE_SIZE to some standard value instead of randomly observed
+ * core size value on Power9 LPAR.
+ */
+static unsigned int get_crash_fdt_mem_sz(void *fdt) {
+	return fdt_totalsize(fdt) + (PER_CORE_NODE_SIZE * MAX_CORE);
+}
+
 int elf_ppc64_probe(const char *buf, off_t len)
 {
 	struct mem_ehdr ehdr;
@@ -179,6 +202,7 @@ int elf_ppc64_load(int argc, char **argv, const char *buf, off_t len,
 	uint64_t max_addr, hole_addr;
 	char *seg_buf = NULL;
 	off_t seg_size = 0;
+	unsigned int mem_sz = 0;
 	struct mem_phdr *phdr;
 	size_t size;
 #ifdef NEED_RESERVE_DTB
@@ -329,7 +353,13 @@ int elf_ppc64_load(int argc, char **argv, const char *buf, off_t len,
 	if (result < 0)
 		return result;
 
-	my_dt_offset = add_buffer(info, seg_buf, seg_size, seg_size,
+	if (info->kexec_flags & KEXEC_ON_CRASH) {
+		mem_sz = get_crash_fdt_mem_sz((void *)seg_buf);
+		fdt_set_totalsize(seg_buf, mem_sz);
+		info->fdt_index = info->nr_segments;
+	}
+
+	my_dt_offset = add_buffer(info, seg_buf, seg_size, mem_sz,
 				0, 0, max_addr, -1);
 
 #ifdef NEED_RESERVE_DTB
kexec/kexec.c +3 -0
index f63b36b..846b1a8 100644
@@ -672,6 +672,9 @@ static void update_purgatory(struct kexec_info *info)
 		if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
 			continue;
 		}
+		if (info->fdt_index == i)
+			continue;
+
 		sha256_update(&ctx, info->segment[i].buf,
 			      info->segment[i].bufsz);
 		nullsz = info->segment[i].memsz - info->segment[i].bufsz;
kexec/kexec.h +4 -0
index 595dd68..0906a1b 100644
@@ -169,6 +169,7 @@ struct kexec_info {
 	int command_line_len;
 
 	int skip_checks;
+       // Given that we might need to update mutliple kexec segments
+       // then having array to keep indexes of all hotplug kexec segments
+       // will be helpful.
+	unsigned int fdt_index;

};

struct arch_map_entry {
---

---
Changelog:

v1 -> v2:
- Use generic hotplug handler introduced by https://lkml.org/lkml/2022/2/9/1406, a significant change from v1.

v2 -> v3
- Move fdt_index and fdt_index_vaild variables to kimage_arch struct.
- Rebase patche on top of https://lkml.org/lkml/2022/3/3/674 [v5]
- Fixed warning reported by checpatch script

v3 -> v4:
- Update the logic to find the additional space needed for hotadd CPUs post kexec load. Refer "[RFC v4 PATCH 4/5] powerpc/crash hp: add crash hotplug support for kexec_file_load" patch to know more about the change.
- Fix a couple of typo.
- Replace pr_err to pr_info_once to warn user about memory hotplug support.
- In crash hotplug handle exit the for loop if FDT segment is found.
---

Sourabh Jain (5):
powerpc/kexec: make update_cpus_node non-static powerpc/crash hp: introduce a new config option CRASH_HOTPLUG powrepc/crash hp: update kimage_arch struct powerpc/crash hp: add crash hotplug support for kexec_file_load powerpc/crash hp: add crash hotplug support for kexec_load

 arch/powerpc/Kconfig              |  11 +++
 arch/powerpc/include/asm/kexec.h  |   3 +
 arch/powerpc/kexec/core_64.c      | 154 ++++++++++++++++++++++++++++++
 arch/powerpc/kexec/elf_64.c       |  74 ++++++++++++++
 arch/powerpc/kexec/file_load_64.c |  87 -----------------
 5 files changed, 242 insertions(+), 87 deletions(-)
Signature · 1 line
2.35.1
#1 lore raw

RFC PATCH v4 1/5 powerpc/kexec: make update_cpus_node non-static

Make the update_cpus_node function non-static and export it for usage in other kexec components.

The update_cpus_node definition is moved to core_64.c so that it can be used with both kexec_load and kexec_file_load system calls.

---
 arch/powerpc/include/asm/kexec.h  |  1 +
 arch/powerpc/kexec/core_64.c      | 88 +++++++++++++++++++++++++++++++
 arch/powerpc/kexec/file_load_64.c | 87 ------------------------------
 3 files changed, 89 insertions(+), 87 deletions(-)
arch/powerpc/include/asm/kexec.h +1 -0
index 8ebdd23d987c..e1288826e22e 100644
@@ -127,6 +127,7 @@ unsigned int kexec_extra_fdt_size_ppc64(struct kimage *image);
 int setup_new_fdt_ppc64(const struct kimage *image, void *fdt,
 			unsigned long initrd_load_addr,
 			unsigned long initrd_len, const char *cmdline);
+int update_cpus_node(void *fdt);
 #endif /* CONFIG_PPC64 */
 
 #endif /* CONFIG_KEXEC_FILE */
arch/powerpc/kexec/core_64.c +88 -0

100 lines not on this page · raw message

arch/powerpc/kexec/file_load_64.c +0 -87

93 lines not on this page · raw message

Signature · 1 line
2.35.1
#2 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

In reply to Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

---
 arch/powerpc/Kconfig | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +11 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
 	  If unsure, say "y". Only special kernels like petitboot may
 	  need to say "N" here.
 
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.
+
 config PRESERVE_FA_DUMP
 	bool "Preserve Firmware-assisted dump"
 	depends on PPC64 && PPC_POWERNV && !FA_DUMP
Signature · 1 line
2.35.1
#3 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

Quoted 8 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

Quoted 26 lines
---
 arch/powerpc/Kconfig | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +11 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
 	  If unsure, say "y". Only special kernels like petitboot may
 	  need to say "N" here.
 
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.
+
 config PRESERVE_FA_DUMP
 	bool "Preserve Firmware-assisted dump"
 	depends on PPC64 && PPC_POWERNV && !FA_DUMP
#4 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

Quoted 10 lines · Laurent Dufour
Quoted 8 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

Thanks for the review.

- Sourabh Jain

Quoted 27 lines
Quoted 27 lines
---
  arch/powerpc/Kconfig | 11 +++++++++++
  1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +11 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
  	  If unsure, say "y". Only special kernels like petitboot may
  	  need to say "N" here.
 
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.
+
  config PRESERVE_FA_DUMP
  	bool "Preserve Firmware-assisted dump"
  	depends on PPC64 && PPC_POWERNV && !FA_DUMP
#5 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

In reply to Sourabh Jain
Quoted 20 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

---
 arch/powerpc/Kconfig | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +0 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
 	  If unsure, say "y". Only special kernels like petitboot may
 	  need to say "N" here.
 

arch/powerpc/Kconfig config FA_DUMP · line 635

Quoted 10 lines
arch/powerpc/Kconfig +10 -0
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.

Why would a user ever want to turn this off?

Seems to me we should just make it always behave this way, and not have a CONFIG option at all.

cheers

#6 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

Quoted 34 lines · Michael Ellerman
Quoted 20 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

---
  arch/powerpc/Kconfig | 11 +++++++++++
  1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +0 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
  	  If unsure, say "y". Only special kernels like petitboot may
  	  need to say "N" here.
 

arch/powerpc/Kconfig config FA_DUMP · line 635

Quoted 10 lines
arch/powerpc/Kconfig +10 -0
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.

Why would a user ever want to turn this off?

Quoted 2 lines

Seems to me we should just make it always behave this way, and not have a CONFIG option at all.

Sourabh,

Borislav Petkov also requested I remove the config option, which will be the case in upcoming v8.

Where I was using CONFIG_CRASH_HOTPLUG, I've replaced it with the CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG.

Eric

Quoted 3 lines

cheers

#7 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

Quoted 38 lines · Eric DeVolder
Quoted 34 lines · Michael Ellerman
Quoted 20 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

---
  arch/powerpc/Kconfig | 11 +++++++++++
  1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +0 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
  	  If unsure, say "y". Only special kernels like petitboot may
  	  need to say "N" here.
 

arch/powerpc/Kconfig config FA_DUMP · line 635

Quoted 10 lines
arch/powerpc/Kconfig +10 -0
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.

Why would a user ever want to turn this off?

Quoted 2 lines

Seems to me we should just make it always behave this way, and not have a CONFIG option at all.

Quoted 7 lines

Sourabh,

Borislav Petkov also requested I remove the config option, which will be the case in upcoming v8.

Where I was using CONFIG_CRASH_HOTPLUG, I've replaced it with the CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG.

If you're having to spell "CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG" in many places then you can still add CONFIG_CRASH_HOTPLUG to represent the sum of all the dependencies, just don't make it user-selectable.

cheers

#8 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

In reply to Eric DeVolder
Quoted 44 lines · Eric DeVolder
Quoted 38 lines · Michael Ellerman
Quoted 26 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for
both kexec_load and kexec_file_load system will be kept under this config.

---
? arch/powerpc/Kconfig | 11 +++++++++++
? 1 file changed, 11 insertions(+)

arch/powerpc/Kconfig +0 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP

??????? If unsure, say "y". Only special kernels like petitboot may ??????? need to say "N" here.
? +config CRASH_HOTPLUG
+??? bool "kernel updates of crash kexec segments"
+??? depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE +??? help
+????? An efficient way to keep the capture kernel up-to-date with CPU

Quoted 8 lines

+????? hotplug events. On CPU hotplug event the kexec segments of capture
+????? kernel becomes stale and need to be updated with latest CPU data.
+????? In this method the kernel performs minimal update to only relevant
+????? kexec segments on CPU hotplug event, instead of triggering full +????? capture kernel reload from userspace using udev rule.

Why would a user ever want to turn this off?

Quoted 2 lines

Seems to me we should just make it always behave this way, and not have a CONFIG option at all.

Quoted 8 lines

Sourabh,

Borislav Petkov also requested I remove the config option, which will be the
case in upcoming v8.

Where I was using CONFIG_CRASH_HOTPLUG, I've replaced it with the CONFIG_HOTPLUG_CPU || CONFIG_MEMORY_HOTPLUG.

Yes good idea. Even Michael suggested there is no need for new CONFIG option. I will also remove CRASH_HOTPLUG config in the next version.

Thanks,
Sourabh Jain

#9 lore raw

RFC PATCH v4 2/5 powerpc/crash hp: introduce a new config option CRASH_HOTPLUG

In reply to Michael Ellerman
Quoted 33 lines · Michael Ellerman
Quoted 20 lines · Sourabh Jain

The option CRASH_HOTPLUG enables, in kernel update to kexec segments on hotplug events.

All the updates needed on the capture kernel load path in the kernel for both kexec_load and kexec_file_load system will be kept under this config.

---
  arch/powerpc/Kconfig | 11 +++++++++++
  1 file changed, 11 insertions(+)
arch/powerpc/Kconfig +0 -0
index b779603978e1..777db33f75b5 100644
@@ -623,6 +623,17 @@ config FA_DUMP
  	  If unsure, say "y". Only special kernels like petitboot may
  	  need to say "N" here.
 

arch/powerpc/Kconfig config FA_DUMP · line 635

Quoted 10 lines
arch/powerpc/Kconfig +10 -0
+config CRASH_HOTPLUG
+	bool "kernel updates of crash kexec segments"
+	depends on CRASH_DUMP && (HOTPLUG_CPU) && KEXEC_FILE
+	help
+	  An efficient way to keep the capture kernel up-to-date with CPU
+	  hotplug events. On CPU hotplug event the kexec segments of capture
+	  kernel becomes stale and need to be updated with latest CPU data.
+	  In this method the kernel performs minimal update to only relevant
+	  kexec segments on CPU hotplug event, instead of triggering full
+	  capture kernel reload from userspace using udev rule.

Why would a user ever want to turn this off?

Quoted 2 lines

Seems to me we should just make it always behave this way, and not have a CONFIG option at all.

Yes, we don't need a new CONFIG option. Thanks for the suggestion.

Thanks,
Sourabh Jain

#10 lore raw

RFC PATCH v4 3/5 powrepc/crash hp: update kimage_arch struct

In reply to Sourabh Jain

Two new members fdt_index and fdt_index_valid are added in kimage_arch struct to track the FDT kexec segment. These new members of kimage_arch struct will help the crash hotplug handler to easily access the FDT segment from the kexec segment array. Otherwise, we have to loop through all kexec segments to find the FDT segments.

---
 arch/powerpc/include/asm/kexec.h | 2 ++
 1 file changed, 2 insertions(+)
arch/powerpc/include/asm/kexec.h +2 -0
index e1288826e22e..19c2cab6a880 100644
@@ -104,6 +104,8 @@ extern const struct kexec_file_ops kexec_elf64_ops;
 struct kimage_arch {
 	struct crash_mem *exclude_ranges;
 
+	int fdt_index;
+	bool fdt_index_valid;
 	unsigned long backup_start;
 	void *backup_buf;
 	void *fdt;
Signature · 1 line
2.35.1
#11 lore raw

RFC PATCH v4 3/5 powrepc/crash hp: update kimage_arch struct

Quoted 16 lines · Sourabh Jain

Two new members fdt_index and fdt_index_valid are added in kimage_arch struct to track the FDT kexec segment. These new members of kimage_arch struct will help the crash hotplug handler to easily access the FDT segment from the kexec segment array. Otherwise, we have to loop through all kexec segments to find the FDT segments.

---
 arch/powerpc/include/asm/kexec.h | 2 ++
 1 file changed, 2 insertions(+)
arch/powerpc/include/asm/kexec.h +0 -0
index e1288826e22e..19c2cab6a880 100644

arch/powerpc/include/asm/kexec.h extern const struct kexec_file_ops kexec_elf64_ops; · line 106

Quoted 3 lines
arch/powerpc/include/asm/kexec.h +0 -0
@@ -104,6 +104,8 @@ extern const struct kexec_file_ops kexec_elf64_ops;
 struct kimage_arch {
 	struct crash_mem *exclude_ranges;
 

#ifdef CONFIG_CRASH_HOTPLUG ?

arch/powerpc/include/asm/kexec.h extern const struct kexec_file_ops kexec_elf64_ops; · line 108

Quoted 2 lines
arch/powerpc/include/asm/kexec.h +2 -0
+	int fdt_index;
+	bool fdt_index_valid;

#endif

It seems that this is only used when CONFIG_CRASH_HOTPLUG is defined, isn't it?

Quoted 3 lines
arch/powerpc/include/asm/kexec.h +0 -0
 	unsigned long backup_start;
 	void *backup_buf;
 	void *fdt;
#12 lore raw

RFC PATCH v4 3/5 powrepc/crash hp: update kimage_arch struct

Quoted 21 lines · Laurent Dufour
Quoted 16 lines · Sourabh Jain

Two new members fdt_index and fdt_index_valid are added in kimage_arch struct to track the FDT kexec segment. These new members of kimage_arch struct will help the crash hotplug handler to easily access the FDT segment from the kexec segment array. Otherwise, we have to loop through all kexec segments to find the FDT segments.

---
  arch/powerpc/include/asm/kexec.h | 2 ++
  1 file changed, 2 insertions(+)
arch/powerpc/include/asm/kexec.h +0 -0
index e1288826e22e..19c2cab6a880 100644

arch/powerpc/include/asm/kexec.h extern const struct kexec_file_ops kexec_elf64_ops; · line 106

Quoted 3 lines
arch/powerpc/include/asm/kexec.h +0 -0
@@ -104,6 +104,8 @@ extern const struct kexec_file_ops kexec_elf64_ops;
  struct kimage_arch {
  	struct crash_mem *exclude_ranges;
 

#ifdef CONFIG_CRASH_HOTPLUG ?

Quoted 5 lines

arch/powerpc/include/asm/kexec.h extern const struct kexec_file_ops kexec_elf64_ops; · line 108

Quoted 2 lines
arch/powerpc/include/asm/kexec.h +2 -0
+	int fdt_index;
+	bool fdt_index_valid;

#endif

It seems that this is only used when CONFIG_CRASH_HOTPLUG is defined, isn't it?

Yes it should be kept under CONFIG_CRASH_HOTPLUG config.

Thanks,
Sourabh Jain

Quoted 4 lines
Quoted 4 lines
arch/powerpc/include/asm/kexec.h +0 -0
  	unsigned long backup_start;
  	void *backup_buf;
  	void *fdt;
#13 lore raw

RFC PATCH v4 4/5 powerpc/crash hp: add crash hotplug support for kexec_file_load

In reply to Sourabh Jain

Two major changes are done to enable the crash CPU hotplug handler. Firstly, updated the kexec load path to prepare kimage for hotplug changes, and secondly, implemented the crash hotplug handler.

On the kexec load path, the memsz allocation of the crash FDT segment is updated to ensure that it has sufficient buffer space to accommodate future hot add CPUs. Initialized the kimage members to track the kexec FDT segment.

The crash hotplug handler updates the cpus node of crash FDT. While we update crash FDT the kexec_crash_image is marked invalid and restored after FDT update to avoid race.

Since memory crash hotplug support is not there yet the crash hotplug the handler simply warns the user and returns.

---
 arch/powerpc/kexec/core_64.c | 46 ++++++++++++++++++++++
 arch/powerpc/kexec/elf_64.c  | 74 ++++++++++++++++++++++++++++++++++++
 2 files changed, 120 insertions(+)
arch/powerpc/kexec/core_64.c +46 -0
index 249d2632526d..62f77cc86407 100644
@@ -466,6 +466,52 @@ int update_cpus_node(void *fdt)
 	return ret;
 }
 
+#ifdef CONFIG_CRASH_HOTPLUG
+/**
+ * arch_crash_hotplug_handler() - Handle hotplug FDT changes
+ * @image: the active struct kimage
+ * @hp_action: the hot un/plug action being handled
+ * @a: first parameter dependent upon hp_action
+ * @b: first parameter dependent upon hp_action
+ *
+ * To accurately reflect CPU hot un/plug changes, the FDT
+ * must be updated with the new list of CPUs and memories.
+ */
+void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
+				unsigned long a, unsigned long b)
+{
+	void *fdt;
+
+	/* No action needed for CPU hot-unplug */
+	if (hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+		return;
+
+	/* crash update on memory hotplug is not support yet */
+	if (hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY || hp_action == KEXEC_CRASH_HP_ADD_MEMORY) {
+		pr_info_once("crash hp: crash update is not supported with memory hotplug\n");
+		return;
+	}
+
+	/* Must have valid FDT index */
+	if (!image->arch.fdt_index_valid) {
+		pr_err("crash hp: unable to locate FDT segment");
+		return;
+	}
+
+	fdt = __va((void *)image->segment[image->arch.fdt_index].mem);
+
+	/* Temporarily invalidate the crash image while it is replaced */
+	xchg(&kexec_crash_image, NULL);
+
+	/* update FDT to refelect changes to CPU resrouces */
+	if (update_cpus_node(fdt))
+		pr_err("crash hp: failed to update crash FDT");
+
+	/* The crash image is now valid once again */
+	xchg(&kexec_crash_image, image);
+}
+#endif /* CONFIG_CRASH_HOTPLUG */
+
 #ifdef CONFIG_PPC_64S_HASH_MMU
 /* Values we need to export to the second kernel via the device tree. */
 static unsigned long htab_base;
arch/powerpc/kexec/elf_64.c +74 -0

86 lines not on this page · raw message

Signature · 1 line
2.35.1
#14 lore raw

RFC PATCH v4 4/5 powerpc/crash hp: add crash hotplug support for kexec_file_load

Quoted 39 lines · Sourabh Jain

Two major changes are done to enable the crash CPU hotplug handler. Firstly, updated the kexec load path to prepare kimage for hotplug changes, and secondly, implemented the crash hotplug handler.

On the kexec load path, the memsz allocation of the crash FDT segment is updated to ensure that it has sufficient buffer space to accommodate future hot add CPUs. Initialized the kimage members to track the kexec FDT segment.

The crash hotplug handler updates the cpus node of crash FDT. While we update crash FDT the kexec_crash_image is marked invalid and restored after FDT update to avoid race.

Since memory crash hotplug support is not there yet the crash hotplug the handler simply warns the user and returns.

---
 arch/powerpc/kexec/core_64.c | 46 ++++++++++++++++++++++
 arch/powerpc/kexec/elf_64.c  | 74 ++++++++++++++++++++++++++++++++++++
 2 files changed, 120 insertions(+)
arch/powerpc/kexec/core_64.c +9 -0
index 249d2632526d..62f77cc86407 100644
@@ -466,6 +466,52 @@ int update_cpus_node(void *fdt)
 	return ret;
 }
 
+#ifdef CONFIG_CRASH_HOTPLUG
+/**
+ * arch_crash_hotplug_handler() - Handle hotplug FDT changes
+ * @image: the active struct kimage
+ * @hp_action: the hot un/plug action being handled
+ * @a: first parameter dependent upon hp_action
+ * @b: first parameter dependent upon hp_action
+ *
+ * To accurately reflect CPU hot un/plug changes, the FDT

arch/powerpc/kexec/core_64.c int update_cpus_node(void *fdt) · line 487

Quoted 10 lines
arch/powerpc/kexec/core_64.c +10 -0
+ * must be updated with the new list of CPUs and memories.
+ */
+void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
+				unsigned long a, unsigned long b)
+{
+	void *fdt;
+
+	/* No action needed for CPU hot-unplug */
+	if (hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+		return;

I should have report since in the previous version too, why nothing is done when CPU are removed?

Quoted 42 lines
arch/powerpc/kexec/core_64.c +27 -0
+
+	/* crash update on memory hotplug is not support yet */
+	if (hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY || hp_action == KEXEC_CRASH_HP_ADD_MEMORY) {
+		pr_info_once("crash hp: crash update is not supported with memory hotplug\n");
+		return;
+	}
+
+	/* Must have valid FDT index */
+	if (!image->arch.fdt_index_valid) {
+		pr_err("crash hp: unable to locate FDT segment");
+		return;
+	}
+
+	fdt = __va((void *)image->segment[image->arch.fdt_index].mem);
+
+	/* Temporarily invalidate the crash image while it is replaced */
+	xchg(&kexec_crash_image, NULL);
+
+	/* update FDT to refelect changes to CPU resrouces */
+	if (update_cpus_node(fdt))
+		pr_err("crash hp: failed to update crash FDT");
+
+	/* The crash image is now valid once again */
+	xchg(&kexec_crash_image, image);
+}
+#endif /* CONFIG_CRASH_HOTPLUG */
+
 #ifdef CONFIG_PPC_64S_HASH_MMU
 /* Values we need to export to the second kernel via the device tree. */
 static unsigned long htab_base;
arch/powerpc/kexec/elf_64.c +4 -0
index eeb258002d1e..9dc774548ce4 100644
@@ -24,6 +24,67 @@
 #include <linux/slab.h>
 #include <linux/types.h>
 
+#include <asm/kvm_book3s.h>
+#include <asm/kvm_ppc.h>
+
+#ifdef CONFIG_CRASH_HOTPLUG

arch/powerpc/kexec/elf_64.c line 40

Quoted 10 lines
arch/powerpc/kexec/elf_64.c +10 -0
+
+/**
+ * get_cpu_node_sz() - Calculate the space needed to store a CPU device type node
+ *                     in FDT. The calculation is done based on the existing CPU
+ *                     node in unflatten device tree. Loop through all the
+ *                     properties of the very first CPU type device node found in
+ *                     unflatten device tree and returns the sum of the property
+ *                     length and property string size of all properties of a CPU
+ *                     node.
+ */

I don't think this is following the indent rules.

Quoted 12 lines
arch/powerpc/kexec/elf_64.c +12 -0
+static int get_cpu_node_sz(void) {
+	struct device_node *dn = NULL;
+	struct property *pp;
+	int cpu_node_size = 0;
+
+	dn = of_find_node_by_type(NULL, "cpu");
+
+	if (!dn) {
+		pr_warn("Unable to locate cpu device_type node.\n");
+		goto out;
+	}
+

arch/powerpc/kexec/elf_64.c line 62

Quoted 10 lines
arch/powerpc/kexec/elf_64.c +10 -0
+	/* Every node in FDT starts with FDT_BEGIN_NODE and ends with
+	 * FDT_END_NODE that takes one byte each.
+	 */
+	cpu_node_size = 2;
+
+	for_each_property_of_node(dn, pp) {
+		/* For each property add two bytes extra. One for string null
+		 * character for property name and other for FDT property start
+		 * tag FDT_PROP.
+		 */

Coding style request to start with a blank comment line for multiple comment lines.

Quoted 48 lines
arch/powerpc/kexec/elf_64.c +38 -0
+		cpu_node_size = cpu_node_size + pp->length + strlen(pp->name) + 2;
+	}
+
+out:
+	return cpu_node_size;
+}
+
+/**
+ * get_crash_fdt_mem_sz() - calcuate mem size for crash kernel FDT
+ * @fdt: pointer to crash kernel FDT
+ *
+ * Calculate the buffer space needed to add more CPU nodes in crash FDT
+ * post capture kenrel load due to CPU hotplug events.
+ */
+static unsigned int get_crash_fdt_mem_sz(void *fdt)
+{
+	int fdt_cpu_nodes_sz, offline_cpu_cnt;
+
+	offline_cpu_cnt = (num_possible_cpus() - num_present_cpus()) / MAX_SMT_THREADS;
+	fdt_cpu_nodes_sz = get_cpu_node_sz() * offline_cpu_cnt;
+
+	return fdt_totalsize(fdt) + fdt_cpu_nodes_sz;
+}
+#endif
+
 static void *elf64_load(struct kimage *image, char *kernel_buf,
 			unsigned long kernel_len, char *initrd,
 			unsigned long initrd_len, char *cmdline,
@@ -123,6 +184,19 @@ static void *elf64_load(struct kimage *image, char *kernel_buf,
 	kbuf.buf_align = PAGE_SIZE;
 	kbuf.top_down = true;
 	kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+
+#ifdef CONFIG_CRASH_HOTPLUG
+	if (image->type == KEXEC_TYPE_CRASH) {
+		kbuf.memsz = get_crash_fdt_mem_sz(fdt);
+		fdt_set_totalsize(fdt, kbuf.memsz);
+		image->arch.fdt_index = image->nr_segments;
+		image->arch.fdt_index_valid = true;
+	} else
+#endif
+	{
+		kbuf.memsz = fdt_totalsize(fdt);
+	}
+
 	ret = kexec_add_buffer(&kbuf);
 	if (ret)
 		goto out_free_fdt;
#15 lore raw

RFC PATCH v4 4/5 powerpc/crash hp: add crash hotplug support for kexec_file_load

Quoted 44 lines · Laurent Dufour
Quoted 43 lines · Sourabh Jain

Two major changes are done to enable the crash CPU hotplug handler. Firstly, updated the kexec load path to prepare kimage for hotplug changes, and secondly, implemented the crash hotplug handler.

On the kexec load path, the memsz allocation of the crash FDT segment is updated to ensure that it has sufficient buffer space to accommodate future hot add CPUs. Initialized the kimage members to track the kexec FDT segment.

The crash hotplug handler updates the cpus node of crash FDT. While we update crash FDT the kexec_crash_image is marked invalid and restored after FDT update to avoid race.

Since memory crash hotplug support is not there yet the crash hotplug the handler simply warns the user and returns.

---
  arch/powerpc/kexec/core_64.c | 46 ++++++++++++++++++++++
  arch/powerpc/kexec/elf_64.c  | 74 ++++++++++++++++++++++++++++++++++++
  2 files changed, 120 insertions(+)
arch/powerpc/kexec/core_64.c +13 -0
index 249d2632526d..62f77cc86407 100644
@@ -466,6 +466,52 @@ int update_cpus_node(void *fdt)
  	return ret;
  }
 
+#ifdef CONFIG_CRASH_HOTPLUG
+/**
+ * arch_crash_hotplug_handler() - Handle hotplug FDT changes
+ * @image: the active struct kimage
+ * @hp_action: the hot un/plug action being handled
+ * @a: first parameter dependent upon hp_action
+ * @b: first parameter dependent upon hp_action
+ *
+ * To accurately reflect CPU hot un/plug changes, the FDT
+ * must be updated with the new list of CPUs and memories.
+ */
+void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
+				unsigned long a, unsigned long b)
Quoted 8 lines

arch/powerpc/kexec/core_64.c int update_cpus_node(void *fdt) · line 487

Quoted 6 lines
arch/powerpc/kexec/core_64.c +6 -0
+{
+	void *fdt;
+
+	/* No action needed for CPU hot-unplug */
+	if (hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+		return;

I should have report since in the previous version too, why nothing is done when CPU are removed?

Since CPU note addresses are already available for all possible CPUs (regardless they are online or not) the PHDR is allocated for all possible CPUs during elfcorehdr creation. At least for the kexec_load system call.

Now on the crash path, the crashing CPU initiates an IPI call to update the CPU data of all online CPUs and jumps to the purgatory. Hence no action is needed for the remove case.

With the above logic, we shouldn't be taking any action for the CPU add case too, right? But on PowerPC early boot path there is validation that checks the boot CPU is part of the? Flattened Device Tree (FDT) or not. If the boot CPU is not found in FDT the boot fails. Hence FDT needs to be updated for every new CPU added to the system but not needed when CPU is removed.

Thanks
Sourabh Jain

Quoted 128 lines
Quoted 43 lines
arch/powerpc/kexec/core_64.c +27 -0
+
+	/* crash update on memory hotplug is not support yet */
+	if (hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY || hp_action == KEXEC_CRASH_HP_ADD_MEMORY) {
+		pr_info_once("crash hp: crash update is not supported with memory hotplug\n");
+		return;
+	}
+
+	/* Must have valid FDT index */
+	if (!image->arch.fdt_index_valid) {
+		pr_err("crash hp: unable to locate FDT segment");
+		return;
+	}
+
+	fdt = __va((void *)image->segment[image->arch.fdt_index].mem);
+
+	/* Temporarily invalidate the crash image while it is replaced */
+	xchg(&kexec_crash_image, NULL);
+
+	/* update FDT to refelect changes to CPU resrouces */
+	if (update_cpus_node(fdt))
+		pr_err("crash hp: failed to update crash FDT");
+
+	/* The crash image is now valid once again */
+	xchg(&kexec_crash_image, image);
+}
+#endif /* CONFIG_CRASH_HOTPLUG */
+
  #ifdef CONFIG_PPC_64S_HASH_MMU
  /* Values we need to export to the second kernel via the device tree. */
  static unsigned long htab_base;
arch/powerpc/kexec/elf_64.c +4 -0
index eeb258002d1e..9dc774548ce4 100644
@@ -24,6 +24,67 @@
  #include <linux/slab.h>
  #include <linux/types.h>
 
+#include <asm/kvm_book3s.h>
+#include <asm/kvm_ppc.h>
+
+#ifdef CONFIG_CRASH_HOTPLUG

arch/powerpc/kexec/elf_64.c line 40

Quoted 10 lines
arch/powerpc/kexec/elf_64.c +10 -0
+
+/**
+ * get_cpu_node_sz() - Calculate the space needed to store a CPU device type node
+ *                     in FDT. The calculation is done based on the existing CPU
+ *                     node in unflatten device tree. Loop through all the
+ *                     properties of the very first CPU type device node found in
+ *                     unflatten device tree and returns the sum of the property
+ *                     length and property string size of all properties of a CPU
+ *                     node.
+ */

I don't think this is following the indent rules.

Quoted 12 lines
arch/powerpc/kexec/elf_64.c +12 -0
+static int get_cpu_node_sz(void) {
+	struct device_node *dn = NULL;
+	struct property *pp;
+	int cpu_node_size = 0;
+
+	dn = of_find_node_by_type(NULL, "cpu");
+
+	if (!dn) {
+		pr_warn("Unable to locate cpu device_type node.\n");
+		goto out;
+	}
+

arch/powerpc/kexec/elf_64.c line 62

Quoted 10 lines
arch/powerpc/kexec/elf_64.c +10 -0
+	/* Every node in FDT starts with FDT_BEGIN_NODE and ends with
+	 * FDT_END_NODE that takes one byte each.
+	 */
+	cpu_node_size = 2;
+
+	for_each_property_of_node(dn, pp) {
+		/* For each property add two bytes extra. One for string null
+		 * character for property name and other for FDT property start
+		 * tag FDT_PROP.
+		 */

Coding style request to start with a blank comment line for multiple comment lines.

Quoted 48 lines
arch/powerpc/kexec/elf_64.c +38 -0
+		cpu_node_size = cpu_node_size + pp->length + strlen(pp->name) + 2;
+	}
+
+out:
+	return cpu_node_size;
+}
+
+/**
+ * get_crash_fdt_mem_sz() - calcuate mem size for crash kernel FDT
+ * @fdt: pointer to crash kernel FDT
+ *
+ * Calculate the buffer space needed to add more CPU nodes in crash FDT
+ * post capture kenrel load due to CPU hotplug events.
+ */
+static unsigned int get_crash_fdt_mem_sz(void *fdt)
+{
+	int fdt_cpu_nodes_sz, offline_cpu_cnt;
+
+	offline_cpu_cnt = (num_possible_cpus() - num_present_cpus()) / MAX_SMT_THREADS;
+	fdt_cpu_nodes_sz = get_cpu_node_sz() * offline_cpu_cnt;
+
+	return fdt_totalsize(fdt) + fdt_cpu_nodes_sz;
+}
+#endif
+
  static void *elf64_load(struct kimage *image, char *kernel_buf,
  			unsigned long kernel_len, char *initrd,
  			unsigned long initrd_len, char *cmdline,
@@ -123,6 +184,19 @@ static void *elf64_load(struct kimage *image, char *kernel_buf,
  	kbuf.buf_align = PAGE_SIZE;
  	kbuf.top_down = true;
  	kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+
+#ifdef CONFIG_CRASH_HOTPLUG
+	if (image->type == KEXEC_TYPE_CRASH) {
+		kbuf.memsz = get_crash_fdt_mem_sz(fdt);
+		fdt_set_totalsize(fdt, kbuf.memsz);
+		image->arch.fdt_index = image->nr_segments;
+		image->arch.fdt_index_valid = true;
+	} else
+#endif
+	{
+		kbuf.memsz = fdt_totalsize(fdt);
+	}
+
  	ret = kexec_add_buffer(&kbuf);
  	if (ret)
  		goto out_free_fdt;
#16 lore raw

RFC PATCH v4 4/5 powerpc/crash hp: add crash hotplug support for kexec_file_load

Quoted 70 lines · Sourabh Jain
Quoted 43 lines · Laurent Dufour
Quoted 42 lines · Sourabh Jain

Two major changes are done to enable the crash CPU hotplug handler. Firstly, updated the kexec load path to prepare kimage for hotplug changes, and secondly, implemented the crash hotplug handler.

On the kexec load path, the memsz allocation of the crash FDT segment is updated to ensure that it has sufficient buffer space to accommodate future hot add CPUs. Initialized the kimage members to track the kexec FDT segment.

The crash hotplug handler updates the cpus node of crash FDT. While we update crash FDT the kexec_crash_image is marked invalid and restored after FDT update to avoid race.

Since memory crash hotplug support is not there yet the crash hotplug the handler simply warns the user and returns.

---
? arch/powerpc/kexec/core_64.c | 46 ++++++++++++++++++++++
? arch/powerpc/kexec/elf_64.c? | 74 ++++++++++++++++++++++++++++++++++++ ? 2 files changed, 120 insertions(+)

arch/powerpc/kexec/core_64.c +13 -0
index 249d2632526d..62f77cc86407 100644
@@ -466,6 +466,52 @@ int update_cpus_node(void *fdt)
 ????? return ret;
 ? }
+#ifdef CONFIG_CRASH_HOTPLUG
+/**
+ * arch_crash_hotplug_handler() - Handle hotplug FDT changes
+ * @image: the active struct kimage
+ * @hp_action: the hot un/plug action being handled
+ * @a: first parameter dependent upon hp_action
+ * @b: first parameter dependent upon hp_action
+ *
+ * To accurately reflect CPU hot un/plug changes, the FDT
+ * must be updated with the new list of CPUs and memories.
+ */
+void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
+??????????????? unsigned long a, unsigned long b)
Quoted 8 lines

arch/powerpc/kexec/core_64.c int update_cpus_node(void *fdt) · line 486

Quoted 6 lines
arch/powerpc/kexec/core_64.c +6 -0
+{
+??? void *fdt;
+
+??? /* No action needed for CPU hot-unplug */
+??? if (hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
+??????? return;

I should have report since in the previous version too, why nothing is done when CPU are removed?

Since CPU note addresses are already available for all possible CPUs (regardless they are online or not) the PHDR is allocated for all possible CPUs during elfcorehdr creation. At least for the kexec_load system call.

Now on the crash path, the crashing CPU initiates an IPI call to update the CPU data of all online CPUs and jumps to the purgatory. Hence no action is needed for the remove case.

With the above logic, we shouldn't be taking any action for the CPU add case too, right? But on PowerPC early boot path there is validation that checks the boot CPU is part of the? Flattened Device Tree (FDT) or not. If the boot CPU is not found in FDT the boot fails. Hence FDT needs to be updated for every new CPU added to the system but not needed when CPU is removed.

Quoted 10 lines

Thanks
Sourabh Jain

Quoted 7 lines
Quoted 7 lines
arch/powerpc/kexec/core_64.c +6 -0
+
+??? /* crash update on memory hotplug is not support yet */
+??? if (hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY || hp_action == KEXEC_CRASH_HP_ADD_MEMORY) {
+??????? pr_info_once("crash hp: crash update is not supported with memory hotplug\n");
+??????? return;
+??? }

Sourabh,

Baoquan's preference is to do away with the hp_action (and a and b) parameters to this function. As x86_64 has no need for them, I have no reason to argue for keeping them.

If you really need hp_action, then please respond to Baoquan's concern in the "[PATCH v7 4/8] crash: add generic infrastructure for crash hotplug support" thread.

Thanks,
eric

Quoted 120 lines
Quoted 120 lines
Quoted 35 lines
arch/powerpc/kexec/core_64.c +21 -0
+
+??? /* Must have valid FDT index */
+??? if (!image->arch.fdt_index_valid) {
+??????? pr_err("crash hp: unable to locate FDT segment");
+??????? return;
+??? }
+
+??? fdt = __va((void *)image->segment[image->arch.fdt_index].mem);
+
+??? /* Temporarily invalidate the crash image while it is replaced */
+??? xchg(&kexec_crash_image, NULL);
+
+??? /* update FDT to refelect changes to CPU resrouces */
+??? if (update_cpus_node(fdt))
+??????? pr_err("crash hp: failed to update crash FDT");
+
+??? /* The crash image is now valid once again */
+??? xchg(&kexec_crash_image, image);
+}
+#endif /* CONFIG_CRASH_HOTPLUG */
+

? #ifdef CONFIG_PPC_64S_HASH_MMU
? /* Values we need to export to the second kernel via the device tree. */ ? static unsigned long htab_base;

arch/powerpc/kexec/elf_64.c +0 -0
index eeb258002d1e..9dc774548ce4 100644
@@ -24,6 +24,67 @@

? #include <linux/slab.h>
? #include <linux/types.h>

arch/powerpc/kexec/elf_64.c +4 -0
+#include <asm/kvm_book3s.h>
+#include <asm/kvm_ppc.h>
+
+#ifdef CONFIG_CRASH_HOTPLUG

arch/powerpc/kexec/elf_64.c line 40

Quoted 10 lines
arch/powerpc/kexec/elf_64.c +10 -0
+
+/**
+ * get_cpu_node_sz() - Calculate the space needed to store a CPU device type node
+ *???????????????????? in FDT. The calculation is done based on the existing CPU
+ *???????????????????? node in unflatten device tree. Loop through all the
+ *???????????????????? properties of the very first CPU type device node found in
+ *???????????????????? unflatten device tree and returns the sum of the property
+ *???????????????????? length and property string size of all properties of a CPU
+ *???????????????????? node.
+ */

I don't think this is following the indent rules.

Quoted 12 lines
arch/powerpc/kexec/elf_64.c +12 -0
+static int get_cpu_node_sz(void) {
+??? struct device_node *dn = NULL;
+??? struct property *pp;
+??? int cpu_node_size = 0;
+
+??? dn = of_find_node_by_type(NULL, "cpu");
+
+??? if (!dn) {
+??????? pr_warn("Unable to locate cpu device_type node.\n");
+??????? goto out;
+??? }
+

arch/powerpc/kexec/elf_64.c

Quoted 10 lines
arch/powerpc/kexec/elf_64.c +10 -0
+??? /* Every node in FDT starts with FDT_BEGIN_NODE and ends with
+???? * FDT_END_NODE that takes one byte each.
+???? */
+??? cpu_node_size = 2;
+
+??? for_each_property_of_node(dn, pp) {
+??????? /* For each property add two bytes extra. One for string null
+???????? * character for property name and other for FDT property start
+???????? * tag FDT_PROP.
+???????? */

Coding style request to start with a blank comment line for multiple comment lines.

Quoted 48 lines

+??????? cpu_node_size = cpu_node_size + pp->length + strlen(pp->name) + 2; +??? }

arch/powerpc/kexec/elf_64.c +23 -0
+
+out:
+??? return cpu_node_size;
+}
+
+/**
+ * get_crash_fdt_mem_sz() - calcuate mem size for crash kernel FDT
+ * @fdt: pointer to crash kernel FDT
+ *
+ * Calculate the buffer space needed to add more CPU nodes in crash FDT
+ * post capture kenrel load due to CPU hotplug events.
+ */
+static unsigned int get_crash_fdt_mem_sz(void *fdt)
+{
+??? int fdt_cpu_nodes_sz, offline_cpu_cnt;
+
+??? offline_cpu_cnt = (num_possible_cpus() - num_present_cpus()) / MAX_SMT_THREADS;
+??? fdt_cpu_nodes_sz = get_cpu_node_sz() * offline_cpu_cnt;
+
+??? return fdt_totalsize(fdt) + fdt_cpu_nodes_sz;
+}
+#endif
+

? static void *elf64_load(struct kimage *image, char *kernel_buf, ????????????? unsigned long kernel_len, char *initrd, ????????????? unsigned long initrd_len, char *cmdline,

arch/powerpc/kexec/elf_64.c +13 -0
@@ -123,6 +184,19 @@ static void *elf64_load(struct kimage *image, char *kernel_buf,
 ????? kbuf.buf_align = PAGE_SIZE;
 ????? kbuf.top_down = true;
 ????? kbuf.mem = KEXEC_BUF_MEM_UNKNOWN;
+
+#ifdef CONFIG_CRASH_HOTPLUG
+??? if (image->type == KEXEC_TYPE_CRASH) {
+??????? kbuf.memsz = get_crash_fdt_mem_sz(fdt);
+??????? fdt_set_totalsize(fdt, kbuf.memsz);
+??????? image->arch.fdt_index = image->nr_segments;
+??????? image->arch.fdt_index_valid = true;
+??? } else
+#endif
+??? {
+??????? kbuf.memsz = fdt_totalsize(fdt);
+??? }
+
 ????? ret = kexec_add_buffer(&kbuf);
 ????? if (ret)
 ????????? goto out_free_fdt;
#17 lore raw

RFC PATCH v4 5/5 powerpc/crash hp: add crash hotplug support for kexec_load

In reply to Sourabh Jain

The kernel changes needed for crash hotplug support for kexec_load system calls are similar to kexec_file_load (which has already been implemented in earlier patches) except for finding the index of the FDT segment in the kexec segment array. Since the kexec segment array is prepared by the kexec tool in the userspace, the kernel is not aware of at which index FDT segment is present.

Now to enable crash hotplug support for the kexec_load case, the crash hotplug handler is updated to identify the index at which the FDT segment is present in the kexec segment array by comparing the first 32 bits of every kexec segment with the FDT magic number.

---
 arch/powerpc/kexec/core_64.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
arch/powerpc/kexec/core_64.c +21 -1
index 62f77cc86407..e3f224f8eb3a 100644
@@ -480,7 +480,9 @@ int update_cpus_node(void *fdt)
 void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
 				unsigned long a, unsigned long b)
 {
-	void *fdt;
+	void *fdt, *ptr;
+	unsigned int n;
+	unsigned long mem, memsz;
 
 	/* No action needed for CPU hot-unplug */
 	if (hp_action == KEXEC_CRASH_HP_REMOVE_CPU)
@@ -492,6 +494,24 @@ void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
 		return;
 	}
 
+	/* Sine kexec segments for kexec_load system call is prepred by
+	 * kexec tool in userspace we need loop through all the segments
+	 * to find out segment index corresponds FDT segment. In case of
+	 * kexec_file_load it is discovered during the load itself.
+	 */
+	if (!image->arch.fdt_index_valid) {
+		for (n = 0; n < image->nr_segments; n++) {
+			mem = image->segment[n].mem;
+			memsz = image->segment[n].memsz;
+			ptr = __va(mem);
+			if (ptr && fdt_magic(ptr) == FDT_MAGIC) {
+				image->arch.fdt_index = n;
+				image->arch.fdt_index_valid = true;
+				break;
+			}
+		}
+	}
+
 	/* Must have valid FDT index */
 	if (!image->arch.fdt_index_valid) {
 		pr_err("crash hp: unable to locate FDT segment");
Signature · 1 line
2.35.1
#18 lore raw

RFC PATCH v4 5/5 powerpc/crash hp: add crash hotplug support for kexec_load

Quoted 33 lines · Sourabh Jain

The kernel changes needed for crash hotplug support for kexec_load system calls are similar to kexec_file_load (which has already been implemented in earlier patches) except for finding the index of the FDT segment in the kexec segment array. Since the kexec segment array is prepared by the kexec tool in the userspace, the kernel is not aware of at which index FDT segment is present.

Now to enable crash hotplug support for the kexec_load case, the crash hotplug handler is updated to identify the index at which the FDT segment is present in the kexec segment array by comparing the first 32 bits of every kexec segment with the FDT magic number.

---
 arch/powerpc/kexec/core_64.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)
arch/powerpc/kexec/core_64.c +3 -1
index 62f77cc86407..e3f224f8eb3a 100644
@@ -480,7 +480,9 @@ int update_cpus_node(void *fdt)
 void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
 				unsigned long a, unsigned long b)
 {
-	void *fdt;
+	void *fdt, *ptr;
+	unsigned int n;
+	unsigned long mem, memsz;
 
 	/* No action needed for CPU hot-unplug */
 	if (hp_action == KEXEC_CRASH_HP_REMOVE_CPU)

arch/powerpc/kexec/core_64.c void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action, · line 497

Quoted 4 lines
arch/powerpc/kexec/core_64.c +1 -0
@@ -492,6 +494,24 @@ void arch_crash_hotplug_handler(struct kimage *image, unsigned int hp_action,
 		return;
 	}
 
+	/* Sine kexec segments for kexec_load system call is prepred by

Since

FWIW,

Quoted 20 lines
arch/powerpc/kexec/core_64.c +17 -0
+	 * kexec tool in userspace we need loop through all the segments
+	 * to find out segment index corresponds FDT segment. In case of
+	 * kexec_file_load it is discovered during the load itself.
+	 */
+	if (!image->arch.fdt_index_valid) {
+		for (n = 0; n < image->nr_segments; n++) {
+			mem = image->segment[n].mem;
+			memsz = image->segment[n].memsz;
+			ptr = __va(mem);
+			if (ptr && fdt_magic(ptr) == FDT_MAGIC) {
+				image->arch.fdt_index = n;
+				image->arch.fdt_index_valid = true;
+				break;
+			}
+		}
+	}
+
 	/* Must have valid FDT index */
 	if (!image->arch.fdt_index_valid) {
 		pr_err("crash hp: unable to locate FDT segment");