Skip to contentSkip to conversation

efi/memreserve: register reservations as 'reserved' in /proc/iomem

PATCH In review Ard Biesheuvel 5 participants 6 messages Dec 4, 2019 – Dec 5, 2019

Subsystems (3)
EFI Live update kexec handover (KHO)
lore
Share & tools
lore

Ard Biesheuvel

author maintainer
#0 lore raw

PATCH 1/1 efi/memreserve: register reservations as 'reserved' in /proc/iomem

Memory regions that are reserved using efi_mem_reserve_persistent() are recorded in a special EFI config table which survives kexec, allowing the incoming kernel to honour them as well. However, such reservations are not visible in /proc/iomem, and so the kexec tools that load the incoming kernel and its initrd into memory may overwrite these reserved regions before the incoming kernel has a chance to reserve them from further use.

So add these reservations to /proc/iomem as they are created. Note that reservations that are inherited from a previous kernel are memblock_reserve()'d early on, so they are already visible in /proc/iomem.

Cc Masayoshi Mizuma <[email protected]>
Cc [email protected]
Cc [email protected]
Signed-off-by Ard Biesheuvel <[email protected]>
---
 drivers/firmware/efi/efi.c | 29 ++++++++++++++++++--
 1 file changed, 26 insertions(+), 3 deletions(-)
drivers/firmware/efi/efi.c +26 -3
index d101f072c8f8..fcd82dde23c8 100644
@@ -979,6 +979,24 @@ static int __init efi_memreserve_map_root(void)
 	return 0;
 }
 
+static int efi_mem_reserve_iomem(phys_addr_t addr, u64 size)
+{
+	struct resource *res, *parent;
+
+	res = kzalloc(sizeof(struct resource), GFP_ATOMIC);
+	if (!res)
+		return -ENOMEM;
+
+	res->name	= "reserved";
+	res->flags	= IORESOURCE_MEM;
+	res->start	= addr;
+	res->end	= addr + size - 1;
+
+	/* we expect a conflict with a 'System RAM' region */
+	parent = request_resource_conflict(&iomem_resource, res);
+	return parent ? request_resource(parent, res) : 0;
+}
+
 int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 {
 	struct linux_efi_memreserve *rsv;

@@ -1001,9 +1019,8 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 		if (index < rsv->size) {
 			rsv->entry[index].base = addr;
 			rsv->entry[index].size = size;
-
 			memunmap(rsv);
-			return 0;
+			return efi_mem_reserve_iomem(addr, size);
 		}
 		memunmap(rsv);
 	}

@@ -1013,6 +1030,12 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 	if (!rsv)
 		return -ENOMEM;
 
+	rc = efi_mem_reserve_iomem(__pa(rsv), SZ_4K);
+	if (rc) {
+		free_page(rsv);
+		return rc;
+	}
+
 	/*
 	 * The memremap() call above assumes that a linux_efi_memreserve entry
 	 * never crosses a page boundary, so let's ensure that this remains true

@@ -1029,7 +1052,7 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 	efi_memreserve_root->next = __pa(rsv);
 	spin_unlock(&efi_mem_reserve_persistent_lock);
 
-	return 0;
+	return efi_mem_reserve_iomem(addr, size);
 }
 
 static int __init efi_memreserve_root_init(void)
Signature · 1 line
2.17.1
#1 lore raw

Hello Ard,

Thank you for sending the patch, but unfortunately it doesn't work for the issue...

After applied your patch, the LPI tables are marked as reserved in /proc/iomem like as:

80300000-a1fdffff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table

However, kexec tries to allocate memory from System RAM, it doesn't care the reserved in System RAM.

Following example, kexec allocates memory 0x82ad0000-0x86640000 to locate the initrd, and LPI tables are also in the memory region, so LPI tables will be destroyed by kexec reboot.

# kexec -d -l /boot/vmlinuz-5.4.1+ --initrd=/boot/initramfs-5.4.1+.img
...
initrd: base 82ad0000, size 3b67c6fh (62291055)
...
segment[1].mem   = 0x82ad0000
segment[1].memsz = 0x3b70000   # 0x86640000 (== 0x82ad0000 + 0x3b70000)
...

I'm not sure why kexec doesn't care the reserved in System RAM, however, if the kexec behaivor is right, the LPI tables should not belong to System RAM.
Like as:

80300000-830effff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table
834a0000-a1fdffff : System RAM

I don't have ideas to separete LPI tables from System RAM... so I tried to add a new file to inform the LPI tables to userspace.

Thanks,
Masa

Quoted 88 lines · Ard Biesheuvel

Memory regions that are reserved using efi_mem_reserve_persistent() are recorded in a special EFI config table which survives kexec, allowing the incoming kernel to honour them as well. However, such reservations are not visible in /proc/iomem, and so the kexec tools that load the incoming kernel and its initrd into memory may overwrite these reserved regions before the incoming kernel has a chance to reserve them from further use.

So add these reservations to /proc/iomem as they are created. Note that reservations that are inherited from a previous kernel are memblock_reserve()'d early on, so they are already visible in /proc/iomem.

Cc Masayoshi Mizuma <[email protected]>
Cc [email protected]
Cc [email protected]
Signed-off-by Ard Biesheuvel <[email protected]>
---
 drivers/firmware/efi/efi.c | 29 ++++++++++++++++++--
 1 file changed, 26 insertions(+), 3 deletions(-)
drivers/firmware/efi/efi.c +26 -3
index d101f072c8f8..fcd82dde23c8 100644
@@ -979,6 +979,24 @@ static int __init efi_memreserve_map_root(void)
 	return 0;
 }
 
+static int efi_mem_reserve_iomem(phys_addr_t addr, u64 size)
+{
+	struct resource *res, *parent;
+
+	res = kzalloc(sizeof(struct resource), GFP_ATOMIC);
+	if (!res)
+		return -ENOMEM;
+
+	res->name	= "reserved";
+	res->flags	= IORESOURCE_MEM;
+	res->start	= addr;
+	res->end	= addr + size - 1;
+
+	/* we expect a conflict with a 'System RAM' region */
+	parent = request_resource_conflict(&iomem_resource, res);
+	return parent ? request_resource(parent, res) : 0;
+}
+
 int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 {
 	struct linux_efi_memreserve *rsv;

@@ -1001,9 +1019,8 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 		if (index < rsv->size) {
 			rsv->entry[index].base = addr;
 			rsv->entry[index].size = size;
-
 			memunmap(rsv);
-			return 0;
+			return efi_mem_reserve_iomem(addr, size);
 		}
 		memunmap(rsv);
 	}

@@ -1013,6 +1030,12 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 	if (!rsv)
 		return -ENOMEM;
 
+	rc = efi_mem_reserve_iomem(__pa(rsv), SZ_4K);
+	if (rc) {
+		free_page(rsv);
+		return rc;
+	}
+
 	/*
 	 * The memremap() call above assumes that a linux_efi_memreserve entry
 	 * never crosses a page boundary, so let's ensure that this remains true

@@ -1029,7 +1052,7 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
 	efi_memreserve_root->next = __pa(rsv);
 	spin_unlock(&efi_mem_reserve_persistent_lock);
 
-	return 0;
+	return efi_mem_reserve_iomem(addr, size);
 }
 
 static int __init efi_memreserve_root_init(void)
Signature · 1 line
2.17.1

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#2 lore raw

Hi Masa,

Quoted 13 lines · Masayoshi Mizuma

Thank you for sending the patch, but unfortunately it doesn't work for the issue...

After applied your patch, the LPI tables are marked as reserved in /proc/iomem like as:

80300000-a1fdffff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table

Quoted 4 lines

However, kexec tries to allocate memory from System RAM, it doesn't care the reserved in System RAM.

I'm not sure why kexec doesn't care the reserved in System RAM, however,

Hmm, we added these to fix a problem with the UEFI memory map, and more recently ACPI tables being overwritten by kexec.

Which version of kexec-tools are you using? Could you try:
https://git.linaro.org/people/takahiro.akashi/kexec-tools.git/commit/?h=arm64/resv_mem

Quoted 14 lines

if the kexec behaivor is right, the LPI tables should not belong to System RAM.

Like as:

80300000-830effff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table
834a0000-a1fdffff : System RAM

Quoted 2 lines

I don't have ideas to separete LPI tables from System RAM... so I tried to add a new file to inform the LPI tables to userspace.

This is how 'nomap' memory appears, we carve it out of System RAM. A side effect of this is kdump can't touch it, as you've told it this isn't memory.

As these tables are memory, mapped by the linear map, I think Ard's patch is the right thing to do ... I suspect your kexec-tools doesn't have those patches from Akashi to make it honour all second level entries.

Thanks,

James

Tested-by
#3 lore raw
Quoted 19 lines · James Morse

Hi Masa,

Quoted 15 lines · Masayoshi Mizuma

Thank you for sending the patch, but unfortunately it doesn't work for the issue...

After applied your patch, the LPI tables are marked as reserved in /proc/iomem like as:

80300000-a1fdffff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table

However, kexec tries to allocate memory from System RAM, it doesn't care the reserved in System RAM.

Quoted 7 lines
Quoted 1 line

I'm not sure why kexec doesn't care the reserved in System RAM, however,

Hmm, we added these to fix a problem with the UEFI memory map, and more recently ACPI tables being overwritten by kexec.

Which version of kexec-tools are you using? Could you try:
https://git.linaro.org/people/takahiro.akashi/kexec-tools.git/commit/?h=arm64/resv_mem

Thanks a lot! It worked and the issue is gone with Ard's patch and the linaro kexec (arm64/resv_mem branch).

Ard, please feel free to add:

Tested-by Masayoshi Mizuma <[email protected]>
Quoted 19 lines
Quoted 16 lines

if the kexec behaivor is right, the LPI tables should not belong to System RAM.

Like as:

80300000-830effff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table
834a0000-a1fdffff : System RAM

Quoted 2 lines

I don't have ideas to separete LPI tables from System RAM... so I tried to add a new file to inform the LPI tables to userspace.

Quoted 6 lines

This is how 'nomap' memory appears, we carve it out of System RAM. A side effect of this is kdump can't touch it, as you've told it this isn't memory.

As these tables are memory, mapped by the linear map, I think Ard's patch is the right thing to do ... I suspect your kexec-tools doesn't have those patches from Akashi to make it honour all second level entries.

I used the kexec on the top of master branch:
git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Should we use the linaro kexec for aarch64 machine? Or will the arm64/resv_mem branch be merged to the kexec on git.kernel.org...?

Thanks!
Masa

#4 lore raw

Hello Masa,

(+Cc Simon)

Quoted 29 lines · Masayoshi Mizuma
Quoted 19 lines · James Morse

Hi Masa,

Quoted 15 lines · Masayoshi Mizuma

Thank you for sending the patch, but unfortunately it doesn't work for the issue...

After applied your patch, the LPI tables are marked as reserved in /proc/iomem like as:

80300000-a1fdffff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table

However, kexec tries to allocate memory from System RAM, it doesn't care the reserved in System RAM.

Quoted 7 lines
Quoted 1 line

I'm not sure why kexec doesn't care the reserved in System RAM, however,

Hmm, we added these to fix a problem with the UEFI memory map, and more recently ACPI tables being overwritten by kexec.

Which version of kexec-tools are you using? Could you try:
https://git.linaro.org/people/takahiro.akashi/kexec-tools.git/commit/?h=arm64/resv_mem

Quoted 6 lines

Thanks a lot! It worked and the issue is gone with Ard's patch and the linaro kexec (arm64/resv_mem branch).

Ard, please feel free to add:

Tested-by Masayoshi Mizuma <[email protected]>

Same results at my side, so:

Tested-and-Reviewed-by Bhipesh Sharma <[email protected]>
Quoted 25 lines
Quoted 18 lines
Quoted 15 lines

if the kexec behaivor is right, the LPI tables should not belong to System RAM.

Like as:

80300000-830effff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table
834a0000-a1fdffff : System RAM

Quoted 2 lines

I don't have ideas to separete LPI tables from System RAM... so I tried to add a new file to inform the LPI tables to userspace.

Quoted 6 lines

This is how 'nomap' memory appears, we carve it out of System RAM. A side effect of this is kdump can't touch it, as you've told it this isn't memory.

As these tables are memory, mapped by the linear map, I think Ard's patch is the right thing to do ... I suspect your kexec-tools doesn't have those patches from Akashi to make it honour all second level entries.

Quoted 6 lines

I used the kexec on the top of master branch:
git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Should we use the linaro kexec for aarch64 machine? Or will the arm64/resv_mem branch be merged to the kexec on git.kernel.org...?

Glad that Ard's patch fixes the issue for you.
Regarding Akashi's patch, I think it was sent to upstream kexec-tools some time ago (see [0}) but seems not integrated in upstream kexec-tools (now I noticed my Tested-by email for the same got bounced off due to some gmail msmtp setting issues at my end - sorry for that). I have added Simon in Cc list.

Hi Simon,

Can you please help pick [0] in upstream kexec-tools with Tested-by from Masa and myself? Thanks a lot for your help.

[0]. http://lists.infradead.org/pipermail/kexec/2019-January/022201.html

Thanks,
Bhupesh

#5 lore raw
Quoted 38 lines · Bhupesh SHARMA

Hello Masa,

(+Cc Simon)

Quoted 32 lines · Masayoshi Mizuma
Quoted 19 lines · James Morse

Hi Masa,

Quoted 15 lines · Masayoshi Mizuma

Thank you for sending the patch, but unfortunately it doesn't work for the issue...

After applied your patch, the LPI tables are marked as reserved in /proc/iomem like as:

80300000-a1fdffff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table

However, kexec tries to allocate memory from System RAM, it doesn't care the reserved in System RAM.

Quoted 7 lines
Quoted 1 line

I'm not sure why kexec doesn't care the reserved in System RAM, however,

Hmm, we added these to fix a problem with the UEFI memory map, and more recently ACPI tables being overwritten by kexec.

Which version of kexec-tools are you using? Could you try:
https://git.linaro.org/people/takahiro.akashi/kexec-tools.git/commit/?h=arm64/resv_mem

Thanks a lot! It worked and the issue is gone with Ard's patch and the linaro kexec (arm64/resv_mem branch).

Quoted 7 lines
Quoted 3 lines

Ard, please feel free to add:

Tested-by Masayoshi Mizuma <[email protected]>

Same results at my side, so:

Tested-and-Reviewed-by Bhipesh Sharma <[email protected]>

Thank you all. I'll get this queued as a fix with cc:stable for v5.4

Quoted 48 lines
Quoted 25 lines
Quoted 18 lines
Quoted 15 lines

if the kexec behaivor is right, the LPI tables should not belong to System RAM.

Like as:

80300000-830effff : System RAM
80480000-8134ffff : Kernel code
81350000-817bffff : reserved
817c0000-82acffff : Kernel data
830f0000-830fffff : reserved # Property table
83480000-83480fff : reserved # Pending table
83490000-8349ffff : reserved # Pending table
834a0000-a1fdffff : System RAM

Quoted 2 lines

I don't have ideas to separete LPI tables from System RAM... so I tried to add a new file to inform the LPI tables to userspace.

Quoted 6 lines

This is how 'nomap' memory appears, we carve it out of System RAM. A side effect of this is kdump can't touch it, as you've told it this isn't memory.

As these tables are memory, mapped by the linear map, I think Ard's patch is the right thing to do ... I suspect your kexec-tools doesn't have those patches from Akashi to make it honour all second level entries.

Quoted 6 lines

I used the kexec on the top of master branch:
git://git.kernel.org/pub/scm/utils/kernel/kexec/kexec-tools.git

Should we use the linaro kexec for aarch64 machine? Or will the arm64/resv_mem branch be merged to the kexec on git.kernel.org...?

Glad that Ard's patch fixes the issue for you.
Regarding Akashi's patch, I think it was sent to upstream kexec-tools some time ago (see [0}) but seems not integrated in upstream kexec-tools (now I noticed my Tested-by email for the same got bounced off due to some gmail msmtp setting issues at my end - sorry for that). I have added Simon in Cc list.

Hi Simon,

Can you please help pick [0] in upstream kexec-tools with Tested-by from Masa and myself? Thanks a lot for your help.

[0]. http://lists.infradead.org/pipermail/kexec/2019-January/022201.html

Thanks,
Bhupesh