arm/efi: fix memblock reallocation crash due to persistent reservations

Bhupesh reports that having numerous memblock reservations at early boot may result in the following crash:

  Unable to handle kernel paging request at virtual address ffff80003ffe0000
  ...
  Call trace:
   __memcpy+0x110/0x180
   memblock_add_range+0x134/0x2e8
   memblock_reserve+0x70/0xb8
   memblock_alloc_base_nid+0x6c/0x88
   __memblock_alloc_base+0x3c/0x4c
   memblock_alloc_base+0x28/0x4c
   memblock_alloc+0x2c/0x38
   early_pgtable_alloc+0x20/0xb0
   paging_init+0x28/0x7f8

This is caused by the fact that we permit memblock resizing before the linear mapping is up, and so the memblock_reserved() array is moved into memory that is not mapped yet.

So let's ensure that this crash can no longer occur, by deferring to call to memblock_allow_resize() to after the linear mapping has been created.

Reported-by Bhupesh Sharma <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/mm/init.c | 2 --
 arch/arm64/mm/mmu.c  | 2 ++
 2 files changed, 2 insertions(+), 2 deletions(-)

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm/kernel/setup.c    | 1 +
 arch/arm64/kernel/setup.c  | 1 +
 drivers/firmware/efi/efi.c | 8 ++++++--
 include/linux/efi.h        | 7 +++++++
 4 files changed, 15 insertions(+), 2 deletions(-)

arch/arm/kernel/setup.c +1 -0

index ac7e08886863..e99f12eaf390 100644
@@ -1117,6 +1117,7 @@ void __init setup_arch(char **cmdline_p)
        early_ioremap_reset();
 
        paging_init(mdesc);
+       efi_apply_persistent_mem_reservations();
        request_standard_resources(mdesc);
 
        if (mdesc->restart)

arch/arm64/kernel/setup.c +1 -0

index 953e316521fc..f4fc1e0544b7 100644
@@ -313,6 +313,7 @@ void __init setup_arch(char **cmdline_p)
        arm64_memblock_init();
 
        paging_init();
+       efi_apply_persistent_mem_reservations();
 
        acpi_table_upgrade();
 

drivers/firmware/efi/efi.c +6 -2

index 249eb70691b0..55e4ea20bdc3 100644
@@ -592,7 +592,11 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz,
 
                early_memunmap(tbl, sizeof(*tbl));
        }
+       return 0;
+}
 
+int __init efi_apply_persistent_mem_reservations(void)
+{
        if (efi.mem_reserve != EFI_INVALID_TABLE_ADDR) {
                unsigned long prsv = efi.mem_reserve;
 

@@ -602,7 +606,7 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz,
                        /* reserve the entry itself */
                        memblock_reserve(prsv, sizeof(*rsv));
 
-                       rsv = early_memremap(prsv, sizeof(*rsv));
+                       rsv = memremap(prsv, sizeof(*rsv), MEMREMAP_WB);
                        if (rsv == NULL) {
                                pr_err("Could not map UEFI memreserve entry!\n");
                                return -ENOMEM;

@@ -612,7 +616,7 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz,
                                memblock_reserve(rsv->base, rsv->size);
 
                        prsv = rsv->next;
-                       early_memunmap(rsv, sizeof(*rsv));
+                       memunmap(rsv);
                }
        }
 

include/linux/efi.h +7 -1

index 845174e113ce..100ce4a4aff6 100644
@@ -1167,6 +1167,8 @@ static inline bool efi_enabled(int feature)
 extern void efi_reboot(enum reboot_mode reboot_mode, const char *__unused);
 
 extern bool efi_is_table_address(unsigned long phys_addr);
+
+extern int efi_apply_persistent_mem_reservations(void);
 #else
 static inline bool efi_enabled(int feature)
 {

@@ -1185,6 +1187,11 @@ static inline bool efi_is_table_address(unsigned long phys_addr)
 {
        return false;
 }
+
+static inline int efi_apply_persistent_mem_reservations(void)
+{
+       return 0;
+}
 #endif
 
 extern int efi_status_to_err(efi_status_t status);
--

2.19.1

Quoted 10 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Quoted 2 lines

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

Quoted 14 lines · Ard Biesheuvel

Quoted 12 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Quoted 4 lines

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

You're not going to get a quick answer to this, because it needs me to investigate what the effect of this change actually is by code review. I can't guarantee when I'll get around to that.

Quoted 15 lines · Russell King - ARM Linux

Quoted 14 lines · Ard Biesheuvel

Quoted 12 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Quoted 9 lines

Quoted 4 lines

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

You're not going to get a quick answer to this, because it needs me to investigate what the effect of this change actually is by code review. I can't guarantee when I'll get around to that.

Fair enough.

I will drop the ARM hunk for now then, and we'll fix ARM when you have more time.

Quoted 27 lines · Ard Biesheuvel

Quoted 24 lines · Russell King - ARM Linux

Quoted 11 lines · Ard Biesheuvel

Quoted 10 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Quoted 7 lines

Quoted 2 lines

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

You're not going to get a quick answer to this, because it needs me to investigate what the effect of this change actually is by code review. I can't guarantee when I'll get around to that.

Quoted 4 lines

Fair enough.

I will drop the ARM hunk for now then, and we'll fix ARM when you have more time.

I don't see how you can do that - you're dropping the processing of reserved areas out of the efi_config_parse_tables() path, so that won't happen any more on ARM if you don't apply the ARM hunk.

Quoted 28 lines · Russell King - ARM Linux

Quoted 27 lines · Ard Biesheuvel

Quoted 24 lines · Russell King - ARM Linux

Quoted 11 lines · Ard Biesheuvel

Quoted 10 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Quoted 7 lines

Quoted 2 lines

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

You're not going to get a quick answer to this, because it needs me to investigate what the effect of this change actually is by code review. I can't guarantee when I'll get around to that.

Quoted 8 lines

Quoted 4 lines

Fair enough.

I will drop the ARM hunk for now then, and we'll fix ARM when you have more time.

I don't see how you can do that - you're dropping the processing of reserved areas out of the efi_config_parse_tables() path, so that won't happen any more on ARM if you don't apply the ARM hunk.

The only reason for the whole persistent region thing is to support GICv3 redistributors memory tables across kexec, for those GIC implementations where LPIs cannot be disabled once they have been enabled.

There is no HW platform that combines 32bit cores and GICv3 (not to mention using EFI). The only existing "platform" is KVM/arm64, which doesn't suffer from the above problem (LPIs can be freely disabled in emulation). So the whole persistent region feature serves no purpose on 32bit ARM.

The only issue with dropping this hunk is that the memory reservation feature is enabled on 32bit ARM the first place, which can easily be dealt with something along those lines:

arch/arm64/Kconfig +1 -0

index 787d7850e064..8bb263c9b99a 100644
@@ -1290,6 +1290,7 @@ config EFI
 	select EFI_RUNTIME_WRAPPERS
 	select EFI_STUB
 	select EFI_ARMSTUB
+	select EFI_PERSISTENT_RESERVATIONS
 	default y
 	help
 	  This option provides support for runtime services provided

drivers/firmware/efi/Kconfig +3 -0

index 89110dfc7127..b728dab9e901 100644
@@ -198,3 +198,6 @@ config EFI_DEV_PATH_PARSER
 	bool
 	depends on ACPI
 	default n
+
+config EFI_PERSISTENT_RESERVATIONS
+       bool

drivers/firmware/efi/efi.c +7 -0

index 249eb70691b0..2a1903ab6d21 100644
@@ -962,6 +962,7 @@ bool efi_is_table_address(unsigned long phys_addr)
 	return false;
 }
 
+#ifdef CONFIG_EFI_PERSISTENT_RESERVATIONS
 static DEFINE_SPINLOCK(efi_mem_reserve_persistent_lock);
 
 int efi_mem_reserve_persistent(phys_addr_t addr, u64 size)

@@ -993,6 +994,12 @@ int efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 
 	return 0;
 }
+#else
+int efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
+{
+	return 0;
+}
+#endif
 
 #ifdef CONFIG_KEXEC
 static int update_efi_random_seed(struct notifier_block *nb,

Thanks,

	M.

Signature · 1 line

Jazz is not dead. It just smells funny...

Quoted 49 lines · Marc Zyngier

Quoted 28 lines · Russell King - ARM Linux

Quoted 27 lines · Ard Biesheuvel

Quoted 24 lines · Russell King - ARM Linux

Quoted 11 lines · Ard Biesheuvel

Quoted 10 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Quoted 7 lines

Quoted 2 lines

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

You're not going to get a quick answer to this, because it needs me to investigate what the effect of this change actually is by code review. I can't guarantee when I'll get around to that.

Quoted 8 lines

Quoted 4 lines

Fair enough.

I will drop the ARM hunk for now then, and we'll fix ARM when you have more time.

I don't see how you can do that - you're dropping the processing of reserved areas out of the efi_config_parse_tables() path, so that won't happen any more on ARM if you don't apply the ARM hunk.

The only reason for the whole persistent region thing is to support GICv3 redistributors memory tables across kexec, for those GIC implementations where LPIs cannot be disabled once they have been enabled.

There is no HW platform that combines 32bit cores and GICv3 (not to mention using EFI). The only existing "platform" is KVM/arm64, which doesn't suffer from the above problem (LPIs can be freely disabled in emulation). So the whole persistent region feature serves no purpose on 32bit ARM.

Quoted 3 lines

The only issue with dropping this hunk is that the memory reservation feature is enabled on 32bit ARM the first place, which can easily be dealt with something along those lines:

So, it was a waste of my time trying to understand what's going on here with Ard's patch because it shouldn't be present on 32-bit ARM... Grr.

Quoted 50 lines · Russell King - ARM Linux

Quoted 49 lines · Marc Zyngier

Quoted 28 lines · Russell King - ARM Linux

Quoted 27 lines · Ard Biesheuvel

Quoted 24 lines · Russell King - ARM Linux

Quoted 11 lines · Ard Biesheuvel

Quoted 10 lines · Ard Biesheuvel

The new memory EFI reservation feature we introduced to allow memory reservations to persist across kexec may trigger an unbounded number of calls to memblock_reserve(). The memblock subsystem can deal with this fine, but not before memblock resizing is enabled, which we can only do after paging_init(), when the memory we reallocate the array into is actually mapped.

So break out the memreserve table processing into a separate function and call if after paging_init() on both arm64 and ARM.

Quoted 7 lines

Quoted 2 lines

Cc Russell King <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

Russell,

If you are ok with this patch, may I have your ack please? I would like to send it out before the end of the week.

You're not going to get a quick answer to this, because it needs me to investigate what the effect of this change actually is by code review. I can't guarantee when I'll get around to that.

Quoted 8 lines

Quoted 4 lines

Fair enough.

I will drop the ARM hunk for now then, and we'll fix ARM when you have more time.

I don't see how you can do that - you're dropping the processing of reserved areas out of the efi_config_parse_tables() path, so that won't happen any more on ARM if you don't apply the ARM hunk.

The only reason for the whole persistent region thing is to support GICv3 redistributors memory tables across kexec, for those GIC implementations where LPIs cannot be disabled once they have been enabled.

There is no HW platform that combines 32bit cores and GICv3 (not to mention using EFI). The only existing "platform" is KVM/arm64, which doesn't suffer from the above problem (LPIs can be freely disabled in emulation). So the whole persistent region feature serves no purpose on 32bit ARM.

Quoted 7 lines

Quoted 3 lines

The only issue with dropping this hunk is that the memory reservation feature is enabled on 32bit ARM the first place, which can easily be dealt with something along those lines:

So, it was a waste of my time trying to understand what's going on here with Ard's patch because it shouldn't be present on 32-bit ARM... Grr.

Not really. I am trying really hard to do my part when it comes to keeping the maintenance level of the ARM architecture up. That means that all the improvements and enhancements that are implemented for EFI on arm64 are enabled on ARM as well, unless there are pressing reasons not to.

And even though this persistent reservation feature is only used by the GICv3 code at the moment, there may be other future uses as well, so I'd prefer not to special case ARM in this way.

This series addresses the kexec/kdump crash on arm64 system with many CPUs that was reported by Bhupesh.

Patches #1 and #2 fix the actual crash. Patches #3 and #4 optimize the EFI persistent memreserve infrastructure so that fewer memblock reservations are required.

Ard Biesheuvel (4):
arm64: memblock: don't permit memblock resizing until linear mapping
is up
efi/arm: defer persistent reservations until after paging_init()
efi: permit multiple entries in persistent memreserve data structure
efi: reduce the amount of memblock reservations for persistent
allocations

 arch/arm/kernel/setup.c                 |  1 +
 arch/arm64/kernel/setup.c               |  1 +
 arch/arm64/mm/init.c                    |  2 -
 arch/arm64/mm/mmu.c                     |  2 +
 drivers/firmware/efi/efi.c              | 59 ++++++++++++++------
 drivers/firmware/efi/libstub/arm-stub.c |  2 +-
 include/linux/efi.h                     | 23 +++++++-
 7 files changed, 68 insertions(+), 22 deletions(-)

Hi Ard,

Quoted 15 lines · Ard Biesheuvel

This series addresses the kexec/kdump crash on arm64 system with many CPUs that was reported by Bhupesh.

Patches #1 and #2 fix the actual crash. Patches #3 and #4 optimize the EFI persistent memreserve infrastructure so that fewer memblock reservations are required.

Ard Biesheuvel (4):
arm64: memblock: don't permit memblock resizing until linear mapping
is up
efi/arm: defer persistent reservations until after paging_init()
efi: permit multiple entries in persistent memreserve data structure
efi: reduce the amount of memblock reservations for persistent
allocations

Quoted 8 lines

 arch/arm/kernel/setup.c                 |  1 +
 arch/arm64/kernel/setup.c               |  1 +
 arch/arm64/mm/init.c                    |  2 -
 arch/arm64/mm/mmu.c                     |  2 +
 drivers/firmware/efi/efi.c              | 59 ++++++++++++++------
 drivers/firmware/efi/libstub/arm-stub.c |  2 +-
 include/linux/efi.h                     | 23 +++++++-
 7 files changed, 68 insertions(+), 22 deletions(-)

I've just given these patches a go a a TX2 box (one of the 224 CPU ones...), and kexec worked just fine (v4.20-rc1 vanilla didn't manage to kexec on this box).

There seem to be some additional userspace patches that are required for the ACPI tables not to be corrupted in the secondary kernel, but that's an orthogonal issue.

Feel free to add

Tested-by Marc Zyngier <[email protected]>

Quoted 37 lines · Marc Zyngier

Hi Ard,

Quoted 15 lines · Ard Biesheuvel

This series addresses the kexec/kdump crash on arm64 system with many CPUs that was reported by Bhupesh.

Patches #1 and #2 fix the actual crash. Patches #3 and #4 optimize the EFI persistent memreserve infrastructure so that fewer memblock reservations are required.

Ard Biesheuvel (4):
arm64: memblock: don't permit memblock resizing until linear mapping
is up
efi/arm: defer persistent reservations until after paging_init()
efi: permit multiple entries in persistent memreserve data structure
efi: reduce the amount of memblock reservations for persistent
allocations

Quoted 8 lines

 arch/arm/kernel/setup.c                 |  1 +
 arch/arm64/kernel/setup.c               |  1 +
 arch/arm64/mm/init.c                    |  2 -
 arch/arm64/mm/mmu.c                     |  2 +
 drivers/firmware/efi/efi.c              | 59 ++++++++++++++------
 drivers/firmware/efi/libstub/arm-stub.c |  2 +-
 include/linux/efi.h                     | 23 +++++++-
 7 files changed, 68 insertions(+), 22 deletions(-)

I've just given these patches a go a a TX2 box (one of the 224 CPU ones...), and kexec worked just fine (v4.20-rc1 vanilla didn't manage to kexec on this box).

There seem to be some additional userspace patches that are required for the ACPI tables not to be corrupted in the secondary kernel, but that's an orthogonal issue.

Feel free to add

Quoted 2 lines

Tested-by Marc Zyngier <[email protected]>

Thanks Marc.

Any thoughts on whether patches #3 and #4 should be included as fixes? I'm leaning towards yes.

This series addresses the kexec/kdump crash on arm64 system with many CPUs that was reported by Bhupesh.

Patches #1 and #2 fix the actual crash. Patches #3 and #4 optimize the EFI persistent memreserve infrastructure so that fewer memblock reservations are required.

Hi Ard,

Quoted 6 lines · Ard Biesheuvel

This series addresses the kexec/kdump crash on arm64 system with many CPUs that was reported by Bhupesh.

Patches #1 and #2 fix the actual crash. Patches #3 and #4 optimize the EFI persistent memreserve infrastructure so that fewer memblock reservations are required.

I acked the arm64 part and patches 3 and 4 look good afaict, so:

Acked-by Will Deacon <[email protected]>

for those as well.

The only thing I was wondering is whether or not exhausting the page-sized array in the first list entry is rare enough that we could just realloc the thing and copy instead of chaining together new pages. That said, without seeing the code it's hard to tell whether you save much complexity with such a scheme so I'll leave it up to you.

Quoted 12 lines · Will Deacon

Quoted 6 lines · Ard Biesheuvel

This series addresses the kexec/kdump crash on arm64 system with many CPUs that was reported by Bhupesh.

Patches #1 and #2 fix the actual crash. Patches #3 and #4 optimize the EFI persistent memreserve infrastructure so that fewer memblock reservations are required.

I acked the arm64 part and patches 3 and 4 look good afaict, so:

Acked-by Will Deacon <[email protected]>

Quoted 8 lines

for those as well.

The only thing I was wondering is whether or not exhausting the page-sized array in the first list entry is rare enough that we could just realloc the thing and copy instead of chaining together new pages. That said, without seeing the code it's hard to tell whether you save much complexity with such a scheme so I'll leave it up to you.

Well, the problem is that the page-sized array may have been allocated by a previous kernel, and so it may be memblock_reserve()d already, in which case reallocating does not actually free up the memory in a useful way.

Also, in the unlikely event that we race and allocate two new pages at the same time, the current scheme will not break (and we will ultimately fill up all the slots in both pages before allocating even more pages). This will become a lot trickier if we do reallocation.

So if the current approach looks correct to you, then I'd prefer to stick with it.

Do you agree with applying #3 and #4 as fixes?