LoongArch: Add KHO support and selftests

From: George Guo <[email protected]>

KHO has to give the next kernel two addresses: the KHO FDT and the bootmem area list. arm64 and riscv put them under /chosen in the boot FDT. LoongArch systems booting with EFI and ACPI do not require a boot FDT, so KHO needs another way to pass this information.

Add a Linux EFI configuration table for them, under LINUX_EFI_KEXEC_HANDOVER_GUID. It points to a struct linux_efi_kho_data.

- On a cold boot, the LoongArch EFI stub installs the table, zeroed. Each kexec'd kernel is given the same EFI system table, so it finds the same table.

- In early boot, efi_config_parse_tables() reserves the table so neither the page allocator nor kexec segments use it, and calls kho_populate() if the table holds KHO state.

- Provide efi_kho_update() to update the table before kexec. Map the table during boot so updates do not require allocation or mapping at panic time.

- efi_kho_table_found() tells architecture code whether the table was found at boot, so it can turn KHO off when the table is missing.

Only the LoongArch stub installs the table, and only with CONFIG_KEXEC_HANDOVER. Elsewhere there is no table and this code does nothing.

Quoted 25 lines · George Guo

From: George Guo <[email protected]>

KHO has to give the next kernel two addresses: the KHO FDT and the bootmem area list. arm64 and riscv put them under /chosen in the boot FDT. LoongArch systems booting with EFI and ACPI do not require a boot FDT, so KHO needs another way to pass this information.

Add a Linux EFI configuration table for them, under LINUX_EFI_KEXEC_HANDOVER_GUID. It points to a struct linux_efi_kho_data.

- On a cold boot, the LoongArch EFI stub installs the table, zeroed. Each kexec'd kernel is given the same EFI system table, so it finds the same table.

- In early boot, efi_config_parse_tables() reserves the table so neither the page allocator nor kexec segments use it, and calls kho_populate() if the table holds KHO state.

- Provide efi_kho_update() to update the table before kexec. Map the table during boot so updates do not require allocation or mapping at panic time.

- efi_kho_table_found() tells architecture code whether the table was found at boot, so it can turn KHO off when the table is missing.

Quoted 4 lines

Only the LoongArch stub installs the table, and only with CONFIG_KEXEC_HANDOVER. Elsewhere there is no table and this code does nothing.

Can we handle this as an arch table then, and handle everything in the LoongArch arch code?

Can we handle this as an arch table then, and handle everything in the LoongArch arch code?

Hi Ard, Pratyush,

Quoted 2 lines · Ard Biesheuvel

Can we handle this as an arch table then, and handle everything in the LoongArch arch code?

Ard, LoongArch is currently the only user, so I understand the suggestion to keep the handling in the architecture code.

My preference is to keep the table definition and handling common, given the potential for x86 to use it as Pratyush described in his reply, to replace setup_data once x86 gains EFI support. The table contains no LoongArch-specific fields, and only the LoongArch stub installs it. On systems without the table, the common code does nothing.

Would that be acceptable, or would you prefer to keep the handling in LoongArch until a second architecture adopts it?

Thanks,
George

From: George Guo <[email protected]>

KHO has to give the next kernel two addresses: the KHO FDT and the bootmem area list. arm64 and riscv put them under /chosen in the boot FDT. LoongArch systems booting with EFI and ACPI do not require a boot FDT, so KHO needs another way to pass this information.

Add a Linux EFI configuration table for them, under LINUX_EFI_KEXEC_HANDOVER_GUID. It points to a struct linux_efi_kho_data.

- On a cold boot, the LoongArch EFI stub installs the table, zeroed. Each kexec'd kernel is given the same EFI system table, so it finds the same table.

- In early boot, efi_config_parse_tables() reserves the table so neither the page allocator nor kexec segments use it, and calls kho_populate() if the table holds KHO state.

- Provide efi_kho_update() to update the table before kexec. Map the table during boot so updates do not require allocation or mapping at panic time.

- efi_kho_table_found() tells architecture code whether the table was found at boot, so it can turn KHO off when the table is missing.

Only the LoongArch stub installs the table, and only with CONFIG_KEXEC_HANDOVER. Elsewhere there is no table and this code does nothing.

Signed-off-by George Guo <[email protected]>

---
 drivers/firmware/efi/efi.c               | 68 ++++++++++++++++++++++++
 drivers/firmware/efi/libstub/efi-stub.c  | 22 ++++++++
 drivers/firmware/efi/libstub/efistub.h   |  2 +
 drivers/firmware/efi/libstub/loongarch.c |  2 +
 include/linux/efi.h                      | 40 ++++++++++++++
 5 files changed, 134 insertions(+)

drivers/firmware/efi/efi.c +7 -0

index 0327a39d31fa..e409e4a57378 100644
@@ -24,6 +24,7 @@
 #include <linux/initrd.h>
 #include <linux/io.h>
 #include <linux/kexec.h>
+#include <linux/kexec_handover.h>
 #include <linux/platform_device.h>
 #include <linux/random.h>
 #include <linux/reboot.h>

@@ -62,6 +63,7 @@ unsigned long __ro_after_init efi_rng_seed = EFI_INVALID_TABLE_ADDR;
 static unsigned long __initdata mem_reserve = EFI_INVALID_TABLE_ADDR;
 static unsigned long __initdata rt_prop = EFI_INVALID_TABLE_ADDR;
 static unsigned long __initdata initrd = EFI_INVALID_TABLE_ADDR;
+static unsigned long efi_kho __initdata = EFI_INVALID_TABLE_ADDR;
 
 extern unsigned long primary_display_table;
 

@@ -629,6 +631,7 @@ static const efi_config_table_type_t common_tables[] __initconst = {
 	{EFI_TCG2_FINAL_EVENTS_TABLE_GUID,	&efi.tpm_final_log,	"TPMFinalLog"	},
 	{EFI_CC_FINAL_EVENTS_TABLE_GUID,	&efi.tpm_final_log,	"CCFinalLog"	},
 	{LINUX_EFI_MEMRESERVE_TABLE_GUID,	&mem_reserve,		"MEMRESERVE"	},
+	{LINUX_EFI_KEXEC_HANDOVER_GUID,		&efi_kho,		"KHO"		},
 	{LINUX_EFI_INITRD_MEDIA_GUID,		&initrd,		"INITRD"	},
 	{EFI_RT_PROPERTIES_TABLE_GUID,		&rt_prop,		"RTPROP"	},
 #ifdef CONFIG_OVMF_DEBUG_LOG

@@ -806,6 +809,23 @@ int __init efi_config_parse_tables(const efi_config_table_t *config_tables,
 		}
 	}
 
+	if (efi_kho != EFI_INVALID_TABLE_ADDR) {
+		struct linux_efi_kho_data *kho;
+
+		/* Keep the buddy allocator and kexec segments off the table. */

drivers/firmware/efi/efi.c +51 -0

+		}
+	}
+
 	if (rt_prop != EFI_INVALID_TABLE_ADDR) {
 		efi_rt_properties_table_t *tbl;
 

@@ -1171,6 +1191,54 @@ static int __init efi_memreserve_root_init(void)
 }
 early_initcall(efi_memreserve_root_init);
 
+static struct linux_efi_kho_data *efi_kho_table __ro_after_init;
+
+/* Did the previous kernel or the EFI stub pass a KHO table? */
+bool __init efi_kho_table_found(void)
+{
+	return efi_kho != EFI_INVALID_TABLE_ADDR;
+}
+
+static int __init efi_kho_table_init(void)
+{
+	if (efi_kho == EFI_INVALID_TABLE_ADDR)
+		return 0;
+
+	/* Map it now: the table is also written at panic time. */
+	efi_kho_table = memremap(efi_kho, sizeof(*efi_kho_table), MEMREMAP_WB);
+	if (WARN_ON_ONCE(!efi_kho_table))
+		efi_kho_table = ERR_PTR(-ENOMEM);
+
+	return 0;
+}
+early_initcall(efi_kho_table_init);
+
+/*
+ * Write the KHO addresses the next kernel should use, or zeroes
+ * if it should boot without KHO.
+ */
+int efi_kho_update(phys_addr_t fdt_addr, u64 fdt_size,
+		   phys_addr_t bootmem_addr, u64 bootmem_size)
+{
+	struct linux_efi_kho_data *kho = efi_kho_table;
+
+	/*
+	 * No table: either the architecture passes the KHO state some other
+	 * way, or it turned KHO off at boot because the table was missing.
+	 */
+	if (!kho)
+		return 0;
+	if (IS_ERR(kho))
+		return PTR_ERR(kho);
+
+	kho->fdt_addr		= fdt_addr;
+	kho->fdt_size		= fdt_size;
+	kho->bootmem_addr	= bootmem_addr;
+	kho->bootmem_size	= bootmem_size;
+
+	return 0;
+}
+
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,
 				  unsigned long code, void *unused)

drivers/firmware/efi/libstub/efi-stub.c +11 -0

index 42d6073bcd06..0fb0d4041d1c 100644
@@ -100,6 +100,28 @@ static void install_memreserve_table(void)
 		efi_err("Failed to install memreserve config table!\n");
 }
 
+void install_kho_table(void)
+{
+	struct linux_efi_kho_data *kho;
+	efi_guid_t kho_table_guid = LINUX_EFI_KEXEC_HANDOVER_GUID;
+	efi_status_t status;
+
+	if (!IS_ENABLED(CONFIG_KEXEC_HANDOVER))
+		return;
+
+	status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, sizeof(*kho), (void **)&kho);
+	if (status != EFI_SUCCESS) {

drivers/firmware/efi/libstub/efi-stub.c +2 -0

+}
+
 static u32 get_supported_rt_services(void)
 {

drivers/firmware/efi/libstub/efi-stub.c +0 -0

 	const efi_rt_properties_table_t *rt_prop_table;

drivers/firmware/efi/libstub/efistub.h +2 -0

index fd91fc15ec81..9e0034c1260a 100644
@@ -1057,6 +1057,8 @@ efi_status_t efi_exit_boot_services(void *handle, void *priv,
 efi_status_t efi_boot_kernel(void *handle, efi_loaded_image_t *image,
 			     unsigned long kernel_addr, char *cmdline_ptr);
 
+void install_kho_table(void);
+
 void *get_fdt(unsigned long *fdt_size);
 
 efi_status_t efi_alloc_virtmap(efi_memory_desc_t **virtmap,

drivers/firmware/efi/libstub/loongarch.c +2 -0

index 2b0c87dc9908..c2df097771e4 100644
@@ -74,6 +74,8 @@ efi_status_t efi_boot_kernel(void *handle, efi_loaded_image_t *image,
 	efi_status_t status;
 	u32 desc_ver;
 
+	install_kho_table();
+
 	status = efi_alloc_virtmap(&priv.runtime_map, &desc_size, &desc_ver);
 	if (status != EFI_SUCCESS) {
 		efi_err("Unable to retrieve UEFI memory map.\n");

include/linux/efi.h +3 -0

index aa15ff88539b..253e98bf9922 100644
@@ -422,6 +422,7 @@ void efi_native_runtime_setup(void);
 #define LINUX_EFI_COCO_SECRET_AREA_GUID		EFI_GUID(0xadf956ad, 0xe98c, 0x484c,  0xae, 0x11, 0xb5, 0x1c, 0x7d, 0x33, 0x64, 0x47)
 #define LINUX_EFI_BOOT_MEMMAP_GUID		EFI_GUID(0x800f683f, 0xd08b, 0x423a,  0xa2, 0x93, 0x96, 0x5c, 0x3c, 0x6f, 0xe2, 0xb4)
 #define LINUX_EFI_UNACCEPTED_MEM_TABLE_GUID	EFI_GUID(0xd5d1de3c, 0x105c, 0x44f9,  0x9e, 0xa9, 0xbc, 0xef, 0x98, 0x12, 0x00, 0x31)
+#define LINUX_EFI_KEXEC_HANDOVER_GUID		EFI_GUID(0xc941b6c7, 0x7b3f, 0x4af6,  0x9e, 0x50, 0xfc, 0xb3, 0xa8, 0x86, 0x8a, 0x17)
 
 #define RISCV_EFI_BOOT_PROTOCOL_GUID		EFI_GUID(0xccd15fec, 0x6f73, 0x4eec,  0x83, 0x95, 0x3e, 0x69, 0xe4, 0xb9, 0x40, 0xbf)
 

@@ -1273,6 +1274,45 @@ struct linux_efi_memreserve {
 
 void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size);
 
+/**
+ * struct linux_efi_kho_data - KHO state passed to the next kernel

include/linux/efi.h +27 -0

+ *
+ * The layout is fixed. Changing it requires a new GUID.
+ */
+struct linux_efi_kho_data {
+	u64 fdt_addr;
+	u64 fdt_size;
+	u64 bootmem_addr;
+	u64 bootmem_size;
+};
+
+#ifdef CONFIG_EFI
+bool efi_kho_table_found(void);
+int efi_kho_update(phys_addr_t fdt_addr, u64 fdt_size,
+		   phys_addr_t bootmem_addr, u64 bootmem_size);
+#else
+static inline bool efi_kho_table_found(void)
+{
+	return false;
+}
+
+static inline int efi_kho_update(phys_addr_t fdt_addr, u64 fdt_size,
+				 phys_addr_t bootmem_addr, u64 bootmem_size)
+{
+	return 0;
+}
+#endif
+
 /*
  * The LINUX_EFI_MOK_VARIABLE_TABLE_GUID config table can be provided
  * to the kernel by an EFI boot loader. The table contains a packed

From: George Guo <[email protected]>

On LoongArch, KHO state is passed to the next kernel only through an EFI configuration table. Only the EFI stub installs this table, during a cold boot.

A kernel entered through kexec does not run the EFI stub, so it cannot create the table if the previous kernel did not provide one. Without the table, it cannot pass KHO state to the next kernel on a subsequent kexec.

Add kho_disable() so architecture code can turn KHO off when the EFI configuration table is missing.

Signed-off-by George Guo <[email protected]>

---
 include/linux/kexec_handover.h     | 3 +++
 kernel/liveupdate/kexec_handover.c | 9 +++++++++
 2 files changed, 12 insertions(+)

From: George Guo <[email protected]>

LoongArch never calls memblock_allow_resize(), so the reserved region array cannot grow beyond INIT_MEMBLOCK_RESERVED_REGIONS entries (INIT_MEMBLOCK_REGIONS + NR_CPUS).

Quoted 5 lines · George Guo

From: George Guo <[email protected]>

LoongArch never calls memblock_allow_resize(), so the reserved region array cannot grow beyond INIT_MEMBLOCK_RESERVED_REGIONS entries (INIT_MEMBLOCK_REGIONS + NR_CPUS).

This might be something to fix independent of KHO.

Huacai,

What's the reason LoongArch never enables memblock resizing and why it defines INIT_MEMBLOCK_RESERVED_REGIONS to include NR_CPUS?

--
Sincerely yours,
Mike.

Hi, Mike,

Quoted 10 lines · Mike Rapoport

Quoted 5 lines · George Guo

From: George Guo <[email protected]>

LoongArch never calls memblock_allow_resize(), so the reserved region array cannot grow beyond INIT_MEMBLOCK_RESERVED_REGIONS entries (INIT_MEMBLOCK_REGIONS + NR_CPUS).

This might be something to fix independent of KHO.

Quoted 4 lines

Huacai,

What's the reason LoongArch never enables memblock resizing and why it defines INIT_MEMBLOCK_RESERVED_REGIONS to include NR_CPUS?

At first we didn't define INIT_MEMBLOCK_RESERVED_REGIONS, then we got boot failures on a 128-cores machine, so define it, and the value is similar to arm64.

The reason LoongArch never enables memblock resizing is because the mm code is derived from MIPS, and MIPS never enables it.

Huacai

Quoted 4 lines

--
Sincerely yours,
Mike.

Quoted 15 lines · Huacai Chen

Hi, Mike,

Quoted 12 lines · Mike Rapoport

Quoted 5 lines · George Guo

From: George Guo <[email protected]>

LoongArch never calls memblock_allow_resize(), so the reserved region array cannot grow beyond INIT_MEMBLOCK_RESERVED_REGIONS entries (INIT_MEMBLOCK_REGIONS + NR_CPUS).

This might be something to fix independent of KHO.

Huacai,

Quoted 5 lines

Quoted 2 lines

What's the reason LoongArch never enables memblock resizing and why it defines INIT_MEMBLOCK_RESERVED_REGIONS to include NR_CPUS?

At first we didn't define INIT_MEMBLOCK_RESERVED_REGIONS, then we got boot failures on a 128-cores machine, so define it, and the value is similar to arm64.

AFAIR, arm64 had to reserve some per cpu data before they could enable memblock resizing, so they had to make it static.

Quoted 3 lines

The reason LoongArch never enables memblock resizing is because the mm code is derived from MIPS, and MIPS never enables it.

If that's the only reason, you might want to revisit that :)

Quoted 7 lines

Huacai

Quoted 4 lines

--
Sincerely yours,
Mike.

--
Sincerely yours,
Mike.

Quoted 17 lines · Mike Rapoport

Quoted 15 lines · Huacai Chen

Hi, Mike,

Quoted 12 lines · Mike Rapoport

Quoted 5 lines · George Guo

From: George Guo <[email protected]>

LoongArch never calls memblock_allow_resize(), so the reserved region array cannot grow beyond INIT_MEMBLOCK_RESERVED_REGIONS entries (INIT_MEMBLOCK_REGIONS + NR_CPUS).

This might be something to fix independent of KHO.

Huacai,

Quoted 8 lines

Quoted 5 lines

Quoted 2 lines

What's the reason LoongArch never enables memblock resizing and why it defines INIT_MEMBLOCK_RESERVED_REGIONS to include NR_CPUS?

At first we didn't define INIT_MEMBLOCK_RESERVED_REGIONS, then we got boot failures on a 128-cores machine, so define it, and the value is similar to arm64.

AFAIR, arm64 had to reserve some per cpu data before they could enable memblock resizing, so they had to make it static.

LoongArch is similar, so below 64-cores machine has no problem.

Quoted 1 line

Quoted 4 lines

Quoted 2 lines

The reason LoongArch never enables memblock resizing is because the mm code is derived from MIPS, and MIPS never enables it.

If that's the only reason, you might want to revisit that :)

Yes, it seems we need memblock_allow_resize(), but maybe it is better to bring back paging_init()? Most other archs call memblock_allow_resize() in paging_init().

Huacai

Quoted 32 lines · Huacai Chen

Quoted 17 lines · Mike Rapoport

Quoted 15 lines · Huacai Chen

Hi, Mike,

Quoted 12 lines · Mike Rapoport

Quoted 5 lines · George Guo

From: George Guo <[email protected]>

LoongArch never calls memblock_allow_resize(), so the reserved region array cannot grow beyond INIT_MEMBLOCK_RESERVED_REGIONS entries (INIT_MEMBLOCK_REGIONS + NR_CPUS).

This might be something to fix independent of KHO.

Huacai,

Quoted 8 lines

Quoted 5 lines

Quoted 2 lines

What's the reason LoongArch never enables memblock resizing and why it defines INIT_MEMBLOCK_RESERVED_REGIONS to include NR_CPUS?

At first we didn't define INIT_MEMBLOCK_RESERVED_REGIONS, then we got boot failures on a 128-cores machine, so define it, and the value is similar to arm64.

AFAIR, arm64 had to reserve some per cpu data before they could enable memblock resizing, so they had to make it static.

LoongArch is similar, so below 64-cores machine has no problem.

Quoted 4 lines

Quoted 3 lines

The reason LoongArch never enables memblock resizing is because the mm code is derived from MIPS, and MIPS never enables it.

Quoted 4 lines

Quoted 1 line

If that's the only reason, you might want to revisit that :)

Yes, it seems we need memblock_allow_resize(), but maybe it is better to bring back paging_init()? Most other archs call memblock_allow_resize() in paging_init().

paging_init() is a historical relict that hugely misnamed everywhere :)

I think platform_init() should be fine.

As far as I can tell, by the time platform_init() calls init_numa_memory(), all memory reservations that must happen before any memblock allocations are done.

Quoted 1 line

Huacai

--
Sincerely yours,
Mike.