Skip to contentSkip to conversation

Support kdump for AMD secure memory encryption(SME)

PATCH Superseded Lianbo Jiang 4 participants 26 messages Jun 16, 2018 – Jun 22, 2018

Subsystems (8)
x86 IOMMU x86 MM Proc filesystem +4 more subsystems

v3 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v3 cover Support kdump for AMD secure memory encryption(SME)

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

When sme enabled on AMD server, we also need to support kdump. Because the memory is encrypted in the first kernel, we will remap the old memory encrypted to the second kernel(crash kernel), and sme is also enabled in the second kernel, otherwise the old memory encrypted can not be decrypted. Because simply changing the value of a C-bit on a page will not automatically encrypt the existing contents of a page, and any data in the page prior to the C-bit modification will become unintelligible. A page of memory that is marked encrypted will be automatically decrypted when read from DRAM and will be automatically encrypted when written to DRAM.

For the kdump, it is necessary to distinguish whether the memory is encrypted. Furthermore, we should also know which part of the memory is encrypted or decrypted. We will appropriately remap the memory according to the specific situation in order to tell cpu how to deal with the data(encrypted or decrypted). For example, when sme enabled, if the old memory is encrypted, we will remap the old memory in encrypted way, which will automatically decrypt the old memory encrypted when we read those data from the remapping address.

----------------------------------------------

| first-kernel | second-kernel | kdump support |
|      (mem_encrypt=on|off)    |   (yes|no)    |
|--------------+---------------+---------------|
|     on       |     on        |     yes       |
|     off      |     off       |     yes       |
|     on       |     off       |     no        |
|     off      |     on        |     no        |
|______________|_______________|_______________|

This patch is only for SME kdump, it is not support SEV kdump.

Test tools:
makedumpfile[v1.6.3]: https://github.com/LianboJ/makedumpfile
commit e1de103eca8f (A draft for kdump vmcore about AMD SME)
Author: Lianbo Jiang <[email protected]>
Date: Mon May 14 17:02:40 2018 +0800
Note: This patch can only dump vmcore in the case of SME enabled.

crash-7.2.1: https://github.com/crash-utility/crash.git
commit 1e1bd9c4c1be (Fix for the "bpf" command display on Linux 4.17-rc1)

Author: Dave Anderson <[email protected]>
Date:   Fri May 11 15:54:32 2018 -0400

Test environment:
HP ProLiant DL385Gen10 AMD EPYC 7251
8-Core Processor
32768 MB memory
600 GB disk space

Linux 4.17-rc7:
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
commit b04e217704b7 ("Linux 4.17-rc7")

Author: Linus Torvalds <[email protected]>
Date:   Sun May 27 13:01:47 2018 -0700

Reference:
AMD64 Architecture Programmer's Manual
https://support.amd.com/TechDocs/24593.pdf

Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. remove the '#ifdef' stuff throughout this patch.
3. put some logic into the early_memremap_pgprot_adjust() and clean the previous unnecessary changes, for example: arch/x86/include/asm/dmi.h, arch/x86/kernel/acpi/boot.c, drivers/acpi/tables.c.
4. add a new file and modify Makefile.
5. clean compile warning in copy_device_table() and some compile error.
6. split the original patch into four patches, it will be better for review.

Some known issues:
1. about SME
Upstream kernel doesn't work when we use kexec in the follow command. The system will hang.
(This issue doesn't matter with the kdump patch.)

Reproduce steps:

 # kexec -l /boot/vmlinuz-4.17.0-rc7+ --initrd=/boot/initramfs-4.17.0-rc7+.img --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=on rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug nokaslr"
 # kexec -e (or reboot)

The system will hang:

[ 1248.932239] kexec_core: Starting new kernel

early console in extract_kernel
input_data: 0x000000087e91c3b4
input_len: 0x000000000067fcbd
output: 0x000000087d400000
output_len: 0x0000000001b6fa90
kernel_total_size: 0x0000000001a9d000
trampoline_32bit: 0x0000000000099000

Decompressing Linux...
Parsing ELF... [-here the system will hang]

2. about SEV
Upstream kernel(Host OS) doesn't work in host side, some drivers about SEV always go wrong in host side. We can't boot SEV Guest OS to test kdump patch. Maybe it is more reasonable to improve SEV in another patch. When some drivers can work in host side and it can also boot Virtual Machine(SEV Guest OS), it will be suitable to fix SEV for kdump.

[  369.426131] INFO: task systemd-udevd:865 blocked for more than 120 seconds.
[  369.433177]       Not tainted 4.17.0-rc5+ #60
[  369.437585] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  369.445783] systemd-udevd   D    0   865    813 0x80000004
[  369.451323] Call Trace:
[  369.453815]  ? __schedule+0x290/0x870
[  369.457523]  schedule+0x32/0x80
[  369.460714]  __sev_do_cmd_locked+0x1f6/0x2a0 [ccp]
[  369.465556]  ? cleanup_uevent_env+0x10/0x10
[  369.470084]  ? remove_wait_queue+0x60/0x60
[  369.474219]  ? 0xffffffffc0247000
[  369.477572]  __sev_platform_init_locked+0x2b/0x70 [ccp]
[  369.482843]  sev_platform_init+0x1d/0x30 [ccp]
[  369.487333]  psp_pci_init+0x40/0xe0 [ccp]
[  369.491380]  ? 0xffffffffc0247000
[  369.494936]  sp_mod_init+0x18/0x1000 [ccp]
[  369.499071]  do_one_initcall+0x4e/0x1d4
[  369.502944]  ? _cond_resched+0x15/0x30
[  369.506728]  ? kmem_cache_alloc_trace+0xae/0x1d0
[  369.511386]  ? do_init_module+0x22/0x220
[  369.515345]  do_init_module+0x5a/0x220
[  369.519444]  load_module+0x21cb/0x2a50
[  369.523227]  ? m_show+0x1c0/0x1c0
[  369.526571]  ? security_capable+0x3f/0x60
[  369.530611]  __do_sys_finit_module+0x94/0xe0
[  369.534915]  do_syscall_64+0x5b/0x180
[  369.538607]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  369.543698] RIP: 0033:0x7f708e6311b9
[  369.547536] RSP: 002b:00007ffff9d32aa8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  369.555162] RAX: ffffffffffffffda RBX: 000055602a04c2d0 RCX: 00007f708e6311b9
[  369.562346] RDX: 0000000000000000 RSI: 00007f708ef52039 RDI: 0000000000000008
[  369.569801] RBP: 00007f708ef52039 R08: 0000000000000000 R09: 000055602a048b20
[  369.576988] R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000
[  369.584177] R13: 000055602a075260 R14: 0000000000020000 R15: 0000000000000000

Lianbo Jiang (4):
Add a function(ioremap_encrypted) for kdump when AMD sme enabled Allocate pages for kdump without encryption when SME is enabled Remap the device table of IOMMU in encrypted manner for kdump Help to dump the old memory encrypted into vmcore file

 arch/x86/include/asm/io.h            |  3 ++
 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/ioremap.c                | 28 +++++++++++++------
 drivers/iommu/amd_iommu_init.c       | 15 +++++++++-
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 kernel/kexec_core.c                  | 12 ++++++++
 8 files changed, 128 insertions(+), 15 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
Signature · 1 line
2.9.5
#1 lore raw

PATCH v3 1/4 Add a function(ioremap_encrypted) for kdump when AMD sme enabled

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

---
Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. put some logic into the early_memremap_pgprot_adjust() for early memremap.

 arch/x86/include/asm/io.h |  3 +++
 arch/x86/mm/ioremap.c     | 28 ++++++++++++++++++++--------
 2 files changed, 23 insertions(+), 8 deletions(-)
arch/x86/include/asm/io.h +3 -0
index f6e5b93..989d60b 100644
@@ -192,6 +192,9 @@ extern void __iomem *ioremap_cache(resource_size_t offset, unsigned long size);
 #define ioremap_cache ioremap_cache
 extern void __iomem *ioremap_prot(resource_size_t offset, unsigned long size, unsigned long prot_val);
 #define ioremap_prot ioremap_prot
+extern void __iomem *ioremap_encrypted(resource_size_t phys_addr,
+					unsigned long size);
+#define ioremap_encrypted ioremap_encrypted
 
 /**
  * ioremap     -   map bus memory into CPU space
arch/x86/mm/ioremap.c +20 -8
index c63a545..e365fc4 100644
@@ -24,6 +24,7 @@
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
 #include <asm/setup.h>
+#include <linux/crash_dump.h>
 
 #include "physaddr.h"
 
@@ -131,7 +132,8 @@ static void __ioremap_check_mem(resource_size_t addr, unsigned long size,
  * caller shouldn't need to know that small detail.
  */
 static void __iomem *__ioremap_caller(resource_size_t phys_addr,
-		unsigned long size, enum page_cache_mode pcm, void *caller)
+		unsigned long size, enum page_cache_mode pcm,
+		void *caller, bool encrypted)
 {
 	unsigned long offset, vaddr;
 	resource_size_t last_addr;
@@ -199,7 +201,7 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	 * resulting mapping.
 	 */
 	prot = PAGE_KERNEL_IO;
-	if (sev_active() && mem_flags.desc_other)
+	if ((sev_active() && mem_flags.desc_other) || encrypted)
 		prot = pgprot_encrypted(prot);
 
 	switch (pcm) {
@@ -291,7 +293,7 @@ void __iomem *ioremap_nocache(resource_size_t phys_addr, unsigned long size)
 	enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC_MINUS;
 
 	return __ioremap_caller(phys_addr, size, pcm,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_nocache);
 
@@ -324,7 +326,7 @@ void __iomem *ioremap_uc(resource_size_t phys_addr, unsigned long size)
 	enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC;
 
 	return __ioremap_caller(phys_addr, size, pcm,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL_GPL(ioremap_uc);
 
@@ -341,7 +343,7 @@ EXPORT_SYMBOL_GPL(ioremap_uc);
 void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC,
-					__builtin_return_address(0));
+					__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_wc);
 
@@ -358,14 +360,21 @@ EXPORT_SYMBOL(ioremap_wc);
 void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT,
-					__builtin_return_address(0));
+					__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_wt);
 
+void __iomem *ioremap_encrypted(resource_size_t phys_addr, unsigned long size)
+{
+	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
+				__builtin_return_address(0), true);
+}
+EXPORT_SYMBOL(ioremap_encrypted);
+
 void __iomem *ioremap_cache(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_cache);
 
@@ -374,7 +383,7 @@ void __iomem *ioremap_prot(resource_size_t phys_addr, unsigned long size,
 {
 	return __ioremap_caller(phys_addr, size,
 				pgprot2cachemode(__pgprot(prot_val)),
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_prot);
 
@@ -688,6 +697,9 @@ pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
 	if (encrypted_prot && memremap_should_map_decrypted(phys_addr, size))
 		encrypted_prot = false;
 
+	if (sme_active() && is_kdump_kernel())
+		encrypted_prot = false;
+
 	return encrypted_prot ? pgprot_encrypted(prot)
 			      : pgprot_decrypted(prot);
 }
Signature · 1 line
2.9.5
#2 lore raw

Re: [PATCH 1/4 V3] Add a function(ioremap_encrypted) for kdump when AMD sme enabled

Quoted 120 lines · Lianbo Jiang

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

---
Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. put some logic into the early_memremap_pgprot_adjust() for early memremap.

 arch/x86/include/asm/io.h |  3 +++
 arch/x86/mm/ioremap.c     | 28 ++++++++++++++++++++--------
 2 files changed, 23 insertions(+), 8 deletions(-)
arch/x86/include/asm/io.h +3 -0
index f6e5b93..989d60b 100644
@@ -192,6 +192,9 @@ extern void __iomem *ioremap_cache(resource_size_t offset, unsigned long size);
 #define ioremap_cache ioremap_cache
 extern void __iomem *ioremap_prot(resource_size_t offset, unsigned long size, unsigned long prot_val);
 #define ioremap_prot ioremap_prot
+extern void __iomem *ioremap_encrypted(resource_size_t phys_addr,
+					unsigned long size);
+#define ioremap_encrypted ioremap_encrypted
 
 /**
  * ioremap     -   map bus memory into CPU space
arch/x86/mm/ioremap.c +17 -8
index c63a545..e365fc4 100644
@@ -24,6 +24,7 @@
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
 #include <asm/setup.h>
+#include <linux/crash_dump.h>
 
 #include "physaddr.h"
 
@@ -131,7 +132,8 @@ static void __ioremap_check_mem(resource_size_t addr, unsigned long size,
  * caller shouldn't need to know that small detail.
  */
 static void __iomem *__ioremap_caller(resource_size_t phys_addr,
-		unsigned long size, enum page_cache_mode pcm, void *caller)
+		unsigned long size, enum page_cache_mode pcm,
+		void *caller, bool encrypted)
 {
 	unsigned long offset, vaddr;
 	resource_size_t last_addr;
@@ -199,7 +201,7 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	 * resulting mapping.
 	 */
 	prot = PAGE_KERNEL_IO;
-	if (sev_active() && mem_flags.desc_other)
+	if ((sev_active() && mem_flags.desc_other) || encrypted)
 		prot = pgprot_encrypted(prot);
 
 	switch (pcm) {
@@ -291,7 +293,7 @@ void __iomem *ioremap_nocache(resource_size_t phys_addr, unsigned long size)
 	enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC_MINUS;
 
 	return __ioremap_caller(phys_addr, size, pcm,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_nocache);
 
@@ -324,7 +326,7 @@ void __iomem *ioremap_uc(resource_size_t phys_addr, unsigned long size)
 	enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC;
 
 	return __ioremap_caller(phys_addr, size, pcm,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL_GPL(ioremap_uc);
 
@@ -341,7 +343,7 @@ EXPORT_SYMBOL_GPL(ioremap_uc);
 void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC,
-					__builtin_return_address(0));
+					__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_wc);
 
@@ -358,14 +360,21 @@ EXPORT_SYMBOL(ioremap_wc);
 void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT,
-					__builtin_return_address(0));
+					__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_wt);
 
+void __iomem *ioremap_encrypted(resource_size_t phys_addr, unsigned long size)
+{
+	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
+				__builtin_return_address(0), true);
+}
+EXPORT_SYMBOL(ioremap_encrypted);
+
 void __iomem *ioremap_cache(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_cache);
 
@@ -374,7 +383,7 @@ void __iomem *ioremap_prot(resource_size_t phys_addr, unsigned long size,
 {
 	return __ioremap_caller(phys_addr, size,
 				pgprot2cachemode(__pgprot(prot_val)),
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_prot);
 

arch/x86/mm/ioremap.c pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr, · line 702

Quoted 6 lines
arch/x86/mm/ioremap.c +3 -0
@@ -688,6 +697,9 @@ pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
 	if (encrypted_prot && memremap_should_map_decrypted(phys_addr, size))
 		encrypted_prot = false;
 
+	if (sme_active() && is_kdump_kernel())
+		encrypted_prot = false;
+

This change doesn't seem to go with the patch description. Maybe it should be a separate patch? Or provide more detail in the description for this change.

What is this change addressing? What early_memremap() calls were failing that required this? If this truly is required, you can probably move it up into the "if (sme_active())" block a few lines higher in the function.

Thanks,
Tom

Quoted 4 lines
arch/x86/mm/ioremap.c +0 -0
 	return encrypted_prot ? pgprot_encrypted(prot)
 			      : pgprot_decrypted(prot);
 }
author
#3 lore raw

Re: [PATCH 1/4 V3] Add a function(ioremap_encrypted) for kdump when AMD sme enabled

Quoted 132 lines · Tom Lendacky
Quoted 120 lines · Lianbo Jiang

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

---
Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. put some logic into the early_memremap_pgprot_adjust() for early memremap.

 arch/x86/include/asm/io.h |  3 +++
 arch/x86/mm/ioremap.c     | 28 ++++++++++++++++++++--------
 2 files changed, 23 insertions(+), 8 deletions(-)
arch/x86/include/asm/io.h +3 -0
index f6e5b93..989d60b 100644
@@ -192,6 +192,9 @@ extern void __iomem *ioremap_cache(resource_size_t offset, unsigned long size);
 #define ioremap_cache ioremap_cache
 extern void __iomem *ioremap_prot(resource_size_t offset, unsigned long size, unsigned long prot_val);
 #define ioremap_prot ioremap_prot
+extern void __iomem *ioremap_encrypted(resource_size_t phys_addr,
+					unsigned long size);
+#define ioremap_encrypted ioremap_encrypted
 
 /**
  * ioremap     -   map bus memory into CPU space
arch/x86/mm/ioremap.c +17 -8
index c63a545..e365fc4 100644
@@ -24,6 +24,7 @@
 #include <asm/pgalloc.h>
 #include <asm/pat.h>
 #include <asm/setup.h>
+#include <linux/crash_dump.h>
 
 #include "physaddr.h"
 
@@ -131,7 +132,8 @@ static void __ioremap_check_mem(resource_size_t addr, unsigned long size,
  * caller shouldn't need to know that small detail.
  */
 static void __iomem *__ioremap_caller(resource_size_t phys_addr,
-		unsigned long size, enum page_cache_mode pcm, void *caller)
+		unsigned long size, enum page_cache_mode pcm,
+		void *caller, bool encrypted)
 {
 	unsigned long offset, vaddr;
 	resource_size_t last_addr;
@@ -199,7 +201,7 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 	 * resulting mapping.
 	 */
 	prot = PAGE_KERNEL_IO;
-	if (sev_active() && mem_flags.desc_other)
+	if ((sev_active() && mem_flags.desc_other) || encrypted)
 		prot = pgprot_encrypted(prot);
 
 	switch (pcm) {
@@ -291,7 +293,7 @@ void __iomem *ioremap_nocache(resource_size_t phys_addr, unsigned long size)
 	enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC_MINUS;
 
 	return __ioremap_caller(phys_addr, size, pcm,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_nocache);
 
@@ -324,7 +326,7 @@ void __iomem *ioremap_uc(resource_size_t phys_addr, unsigned long size)
 	enum page_cache_mode pcm = _PAGE_CACHE_MODE_UC;
 
 	return __ioremap_caller(phys_addr, size, pcm,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL_GPL(ioremap_uc);
 
@@ -341,7 +343,7 @@ EXPORT_SYMBOL_GPL(ioremap_uc);
 void __iomem *ioremap_wc(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WC,
-					__builtin_return_address(0));
+					__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_wc);
 
@@ -358,14 +360,21 @@ EXPORT_SYMBOL(ioremap_wc);
 void __iomem *ioremap_wt(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WT,
-					__builtin_return_address(0));
+					__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_wt);
 
+void __iomem *ioremap_encrypted(resource_size_t phys_addr, unsigned long size)
+{
+	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
+				__builtin_return_address(0), true);
+}
+EXPORT_SYMBOL(ioremap_encrypted);
+
 void __iomem *ioremap_cache(resource_size_t phys_addr, unsigned long size)
 {
 	return __ioremap_caller(phys_addr, size, _PAGE_CACHE_MODE_WB,
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_cache);
 
@@ -374,7 +383,7 @@ void __iomem *ioremap_prot(resource_size_t phys_addr, unsigned long size,
 {
 	return __ioremap_caller(phys_addr, size,
 				pgprot2cachemode(__pgprot(prot_val)),
-				__builtin_return_address(0));
+				__builtin_return_address(0), false);
 }
 EXPORT_SYMBOL(ioremap_prot);
 

arch/x86/mm/ioremap.c pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr, · line 702

Quoted 6 lines
arch/x86/mm/ioremap.c +3 -0
@@ -688,6 +697,9 @@ pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
 	if (encrypted_prot && memremap_should_map_decrypted(phys_addr, size))
 		encrypted_prot = false;
 
+	if (sme_active() && is_kdump_kernel())
+		encrypted_prot = false;
+

This change doesn't seem to go with the patch description. Maybe it should be a separate patch? Or provide more detail in the description for this change.

Quoted 4 lines

What is this change addressing? What early_memremap() calls were failing that required this? If this truly is required, you can probably move it up into the "if (sme_active())" block a few lines higher in the function.

Thanks. About this issue, you could read the description of cover letter, we put some logic into the early_memremap_pgprot_adjust() in order to avoid the previous changes, such as arch/x86/include/asm/dmi.h, arch/x86/kernel/acpi/boot.c and drivers/acpi/tables.c. They will call early_memremap() to remap the memory, but we need to remap the memory without encryption.

Lianbo

Quoted 7 lines

Thanks,
Tom

Quoted 4 lines
arch/x86/mm/ioremap.c +0 -0
 	return encrypted_prot ? pgprot_encrypted(prot)
 			      : pgprot_decrypted(prot);
 }

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#4 lore raw

PATCH v3 2/4 Allocate pages for kdump without encryption when SME is enabled

In reply to Lianbo Jiang

When SME is enabled in the first kernel, we will allocate pages for kdump without encryption in order to be able to boot the second kernel in the same manner as kexec, which helps to keep the same code style.

---
 kernel/kexec_core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
kernel/kexec_core.c +12 -0
index 20fef1a..3c22a9b 100644
@@ -471,6 +471,16 @@ static struct page *kimage_alloc_crash_control_pages(struct kimage *image,
 		}
 	}
 
+	if (pages) {
+		unsigned int count, i;
+
+		pages->mapping = NULL;
+		set_page_private(pages, order);
+		count = 1 << order;
+		for (i = 0; i < count; i++)
+			SetPageReserved(pages + i);
+		arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
+	}
 	return pages;
 }
 
@@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result  = -ENOMEM;
 			goto out;
 		}
+		arch_kexec_post_alloc_pages(page_address(page), 1, 0);
 		ptr = kmap(page);
 		ptr += maddr & ~PAGE_MASK;
 		mchunk = min_t(size_t, mbytes,
@@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result = copy_from_user(ptr, buf, uchunk);
 		kexec_flush_icache_page(page);
 		kunmap(page);
+		arch_kexec_pre_free_pages(page_address(page), 1);
 		if (result) {
 			result = -EFAULT;
 			goto out;
Signature · 1 line
2.9.5
#5 lore raw

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

Quoted 16 lines · Lianbo Jiang

When SME is enabled in the first kernel, we will allocate pages for kdump without encryption in order to be able to boot the second kernel in the same manner as kexec, which helps to keep the same code style.

---
 kernel/kexec_core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
kernel/kexec_core.c +0 -0
index 20fef1a..3c22a9b 100644
@@ -471,6 +471,16 @@ static struct page *kimage_alloc_crash_control_pages(struct kimage *image,
 		}

kernel/kexec_core.c static struct page *kimage_alloc_crash_control_pages(struct kimage *image, · line 481

Quoted 10 lines
kernel/kexec_core.c +8 -0
 	}
 
+	if (pages) {
+		unsigned int count, i;
+
+		pages->mapping = NULL;
+		set_page_private(pages, order);
+		count = 1 << order;
+		for (i = 0; i < count; i++)
+			SetPageReserved(pages + i);

I guess you might imitate the kexec case, however kexec get pages from buddy. Crash pages are reserved in memblock, these codes might make no sense.

Quoted 29 lines
kernel/kexec_core.c +4 -0
+		arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
+	}
 	return pages;
 }
 
@@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result  = -ENOMEM;
 			goto out;
 		}
+		arch_kexec_post_alloc_pages(page_address(page), 1, 0);
 		ptr = kmap(page);
 		ptr += maddr & ~PAGE_MASK;
 		mchunk = min_t(size_t, mbytes,
@@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result = copy_from_user(ptr, buf, uchunk);
 		kexec_flush_icache_page(page);
 		kunmap(page);
+		arch_kexec_pre_free_pages(page_address(page), 1);
 		if (result) {
 			result = -EFAULT;
 			goto out;
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#6 lore raw

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

Quoted 19 lines · Baoquan He
Quoted 18 lines · Lianbo Jiang

When SME is enabled in the first kernel, we will allocate pages for kdump without encryption in order to be able to boot the second kernel in the same manner as kexec, which helps to keep the same code style.

---
 kernel/kexec_core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
kernel/kexec_core.c +0 -0
index 20fef1a..3c22a9b 100644
@@ -471,6 +471,16 @@ static struct page *kimage_alloc_crash_control_pages(struct kimage *image,
 		}
 	}
 
Quoted 12 lines

kernel/kexec_core.c static struct page *kimage_alloc_crash_control_pages(struct kimage *image, · line 481

Quoted 8 lines
kernel/kexec_core.c +8 -0
+	if (pages) {
+		unsigned int count, i;
+
+		pages->mapping = NULL;
+		set_page_private(pages, order);
+		count = 1 << order;
+		for (i = 0; i < count; i++)
+			SetPageReserved(pages + i);

I guess you might imitate the kexec case, however kexec get pages from buddy. Crash pages are reserved in memblock, these codes might make no sense.

Thanks for your comments.
We have changed the attribute of pages, so the original attribute of pages will be restored when they free.

Thanks.
Lianbo

Quoted 25 lines
Quoted 25 lines
kernel/kexec_core.c +4 -0
+		arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
+	}
 	return pages;
 }
 
@@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result  = -ENOMEM;
 			goto out;
 		}
+		arch_kexec_post_alloc_pages(page_address(page), 1, 0);
 		ptr = kmap(page);
 		ptr += maddr & ~PAGE_MASK;
 		mchunk = min_t(size_t, mbytes,
@@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result = copy_from_user(ptr, buf, uchunk);
 		kexec_flush_icache_page(page);
 		kunmap(page);
+		arch_kexec_pre_free_pages(page_address(page), 1);
 		if (result) {
 			result = -EFAULT;
 			goto out;
Signature · 1 line
2.9.5
Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#7 lore raw

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

Quoted 32 lines · lijiang
Quoted 19 lines · Baoquan He
Quoted 18 lines · Lianbo Jiang

When SME is enabled in the first kernel, we will allocate pages for kdump without encryption in order to be able to boot the second kernel in the same manner as kexec, which helps to keep the same code style.

---
 kernel/kexec_core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
kernel/kexec_core.c +0 -0
index 20fef1a..3c22a9b 100644
@@ -471,6 +471,16 @@ static struct page *kimage_alloc_crash_control_pages(struct kimage *image,
 		}
 	}
 
Quoted 12 lines

kernel/kexec_core.c static struct page *kimage_alloc_crash_control_pages(struct kimage *image, · line 481

Quoted 8 lines
kernel/kexec_core.c +8 -0
+	if (pages) {
+		unsigned int count, i;
+
+		pages->mapping = NULL;
+		set_page_private(pages, order);
+		count = 1 << order;
+		for (i = 0; i < count; i++)
+			SetPageReserved(pages + i);

I guess you might imitate the kexec case, however kexec get pages from buddy. Crash pages are reserved in memblock, these codes might make no sense.

Quoted 3 lines

Thanks for your comments.
We have changed the attribute of pages, so the original attribute of pages will be restored when they free.

Hmm, you can check what kimage_free() is doing, and where kimage->control_pages, dest_pages, unusable_pages is assigned. Do you know where these original attribute of pages comes from and they are used/needed in CRASH case, if you care about them?

Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
kernel/kexec_core.c +4 -0
+		arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
+	}
 	return pages;
 }
 
@@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result  = -ENOMEM;
 			goto out;
 		}
+		arch_kexec_post_alloc_pages(page_address(page), 1, 0);
 		ptr = kmap(page);
 		ptr += maddr & ~PAGE_MASK;
 		mchunk = min_t(size_t, mbytes,
@@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result = copy_from_user(ptr, buf, uchunk);
 		kexec_flush_icache_page(page);
 		kunmap(page);
+		arch_kexec_pre_free_pages(page_address(page), 1);
 		if (result) {
 			result = -EFAULT;
 			goto out;
Signature · 1 line
2.9.5
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#8 lore raw

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

Quoted 33 lines · Baoquan He
Quoted 32 lines · lijiang
Quoted 31 lines · Baoquan He
Quoted 16 lines · Lianbo Jiang

When SME is enabled in the first kernel, we will allocate pages for kdump without encryption in order to be able to boot the second kernel in the same manner as kexec, which helps to keep the same code style.

---
 kernel/kexec_core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
kernel/kexec_core.c +0 -0
index 20fef1a..3c22a9b 100644
@@ -471,6 +471,16 @@ static struct page *kimage_alloc_crash_control_pages(struct kimage *image,
 		}

kernel/kexec_core.c static struct page *kimage_alloc_crash_control_pages(struct kimage *image, · line 481

Quoted 10 lines
kernel/kexec_core.c +8 -0
 	}
 
+	if (pages) {
+		unsigned int count, i;
+
+		pages->mapping = NULL;
+		set_page_private(pages, order);
+		count = 1 << order;
+		for (i = 0; i < count; i++)
+			SetPageReserved(pages + i);

I guess you might imitate the kexec case, however kexec get pages from buddy. Crash pages are reserved in memblock, these codes might make no sense.

Quoted 9 lines
Quoted 3 lines

Thanks for your comments.
We have changed the attribute of pages, so the original attribute of pages will be restored when they free.

Hmm, you can check what kimage_free() is doing, and where kimage->control_pages, dest_pages, unusable_pages is assigned. Do you know where these original attribute of pages comes from and they are used/needed in CRASH case, if you care about them?

Originally, we want to have an opportunity to restore the previous attribute of pages, that should be more better if the pages are remembered in 'image->control_pages'. If we remove these codes, it is also harmless for kdump, but it will become strange, maybe someone could ask where to restore the previous attribute of pages.

Thanks.

Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
kernel/kexec_core.c +4 -0
+		arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
+	}
 	return pages;
 }
 
@@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result  = -ENOMEM;
 			goto out;
 		}
+		arch_kexec_post_alloc_pages(page_address(page), 1, 0);
 		ptr = kmap(page);
 		ptr += maddr & ~PAGE_MASK;
 		mchunk = min_t(size_t, mbytes,
@@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result = copy_from_user(ptr, buf, uchunk);
 		kexec_flush_icache_page(page);
 		kunmap(page);
+		arch_kexec_pre_free_pages(page_address(page), 1);
 		if (result) {
 			result = -EFAULT;
 			goto out;
Signature · 1 line
2.9.5
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#9 lore raw

Re: [PATCH 2/4 V3] Allocate pages for kdump without encryption when SME is enabled

Quoted 38 lines · lijiang
Quoted 37 lines · Baoquan He
Quoted 35 lines · lijiang
Quoted 19 lines · Baoquan He
Quoted 18 lines · Lianbo Jiang

When SME is enabled in the first kernel, we will allocate pages for kdump without encryption in order to be able to boot the second kernel in the same manner as kexec, which helps to keep the same code style.

---
 kernel/kexec_core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
kernel/kexec_core.c +0 -0
index 20fef1a..3c22a9b 100644
@@ -471,6 +471,16 @@ static struct page *kimage_alloc_crash_control_pages(struct kimage *image,
 		}
 	}
 
Quoted 12 lines

kernel/kexec_core.c static struct page *kimage_alloc_crash_control_pages(struct kimage *image, · line 481

Quoted 8 lines
kernel/kexec_core.c +8 -0
+	if (pages) {
+		unsigned int count, i;
+
+		pages->mapping = NULL;
+		set_page_private(pages, order);
+		count = 1 << order;
+		for (i = 0; i < count; i++)
+			SetPageReserved(pages + i);

I guess you might imitate the kexec case, however kexec get pages from buddy. Crash pages are reserved in memblock, these codes might make no sense.

Thanks for your comments.
We have changed the attribute of pages, so the original attribute of pages will be restored when they free.

Quoted 7 lines
Quoted 5 lines

Hmm, you can check what kimage_free() is doing, and where kimage->control_pages, dest_pages, unusable_pages is assigned. Do you know where these original attribute of pages comes from and they are used/needed in CRASH case, if you care about them?

Originally, we want to have an opportunity to restore the previous attribute of pages, that should be more better if the pages are remembered in 'image->control_pages'.

Again, please check who assigns value for 'image->control_pages'.

Quoted 39 lines

If we remove these codes, it is also harmless for kdump, but it will become strange, maybe someone could ask where to restore the previous attribute of pages.

Thanks.

Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
kernel/kexec_core.c +4 -0
+		arch_kexec_post_alloc_pages(page_address(pages), 1 << order, 0);
+	}
 	return pages;
 }
 
@@ -865,6 +875,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result  = -ENOMEM;
 			goto out;
 		}
+		arch_kexec_post_alloc_pages(page_address(page), 1, 0);
 		ptr = kmap(page);
 		ptr += maddr & ~PAGE_MASK;
 		mchunk = min_t(size_t, mbytes,
@@ -882,6 +893,7 @@ static int kimage_load_crash_segment(struct kimage *image,
 			result = copy_from_user(ptr, buf, uchunk);
 		kexec_flush_icache_page(page);
 		kunmap(page);
+		arch_kexec_pre_free_pages(page_address(page), 1);
 		if (result) {
 			result = -EFAULT;
 			goto out;
Signature · 1 line
2.9.5
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#10 lore raw

PATCH v3 3/4 Remap the device table of IOMMU in encrypted manner for kdump

In reply to Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +14 -1
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+
 	if (!old_devtb)
 		return false;
 
Signature · 1 line
2.9.5
#11 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

Quoted 21 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 900

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +9 -0
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+

You can probably just use "if (is_kdump_kernel())" here, since memory encryption is either on in both the first and second kernel or off in both the first and second kernel. At which point __sme_clr() will do the proper thing.

Actually, this needs to be done no matter what. When doing either the ioremap_encrypted() or the memremap(), the physical address should not include the encryption bit/mask.

Thanks,
Tom

Quoted 13 lines
drivers/iommu/amd_iommu_init.c +4 -1
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
 	if (!old_devtb)
 		return false;
 
author
#12 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

Quoted 38 lines · Tom Lendacky
Quoted 21 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 900

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +9 -0
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+

You can probably just use "if (is_kdump_kernel())" here, since memory encryption is either on in both the first and second kernel or off in both the first and second kernel. At which point __sme_clr() will do the proper thing.

Quoted 7 lines

Actually, this needs to be done no matter what. When doing either the ioremap_encrypted() or the memremap(), the physical address should not include the encryption bit/mask.

Thanks,
Tom

Thanks for your comments. If we don't remove the memory encryption mask, it will return false because the 'old_devtb_phys >= 0x100000000ULL' may become true.

Lianbo

Quoted 13 lines
Quoted 13 lines
drivers/iommu/amd_iommu_init.c +4 -1
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
 	if (!old_devtb)
 		return false;
 
#13 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

Quoted 43 lines · lijiang
Quoted 42 lines · Tom Lendacky
Quoted 21 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 900

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +9 -0
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+

You can probably just use "if (is_kdump_kernel())" here, since memory encryption is either on in both the first and second kernel or off in both the first and second kernel. At which point __sme_clr() will do the proper thing.

Actually, this needs to be done no matter what. When doing either the ioremap_encrypted() or the memremap(), the physical address should not include the encryption bit/mask.

Quoted 5 lines
Quoted 3 lines

Thanks,
Tom

Thanks for your comments. If we don't remove the memory encryption mask, it will return false because the 'old_devtb_phys >= 0x100000000ULL' may become true.

Lianbo, you may not get what Tom suggested. Tom means no matter what it is, encrypted or not in 1st kernel, we need get pure physicall address, and using below code is always right for both cases.

	if (is_kdump_kernel())
		old_devtb_phys = __sme_clr(old_devtb_phys);

And this is simpler. You even can add one line of code comment to say like "Physical address w/o encryption mask is needed here."

Quoted 12 lines

Lianbo

Quoted 10 lines
Quoted 10 lines
drivers/iommu/amd_iommu_init.c +4 -1
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
 	if (!old_devtb)
Quoted 8 lines
Quoted 3 lines

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 911

Quoted 3 lines
drivers/iommu/amd_iommu_init.c +0 -0
 		return false;
 

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#14 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

Quoted 50 lines · Baoquan He
Quoted 43 lines · lijiang
Quoted 42 lines · Tom Lendacky
Quoted 21 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 900

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +9 -0
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+

You can probably just use "if (is_kdump_kernel())" here, since memory encryption is either on in both the first and second kernel or off in both the first and second kernel. At which point __sme_clr() will do the proper thing.

Actually, this needs to be done no matter what. When doing either the ioremap_encrypted() or the memremap(), the physical address should not include the encryption bit/mask.

Quoted 5 lines
Quoted 3 lines

Thanks,
Tom

Thanks for your comments. If we don't remove the memory encryption mask, it will return false because the 'old_devtb_phys >= 0x100000000ULL' may become true.

Quoted 7 lines

Lianbo, you may not get what Tom suggested. Tom means no matter what it is, encrypted or not in 1st kernel, we need get pure physicall address, and using below code is always right for both cases.

	if (is_kdump_kernel())
		old_devtb_phys = __sme_clr(old_devtb_phys);

Thank you, Baoquan. I understood what Tom said. I just want to explain why removing the memory encryption mask is necessary before the 'old_devtb_phys >= 0x100000000ULL'.

Lianbo

Quoted 22 lines

And this is simpler. You even can add one line of code comment to say like "Physical address w/o encryption mask is needed here."

Quoted 20 lines

Lianbo

Quoted 5 lines
Quoted 5 lines
drivers/iommu/amd_iommu_init.c +0 -1
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
Quoted 8 lines

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 912

Quoted 8 lines
drivers/iommu/amd_iommu_init.c +4 -0
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
 	if (!old_devtb)
 		return false;
 

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#15 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

In reply to Baoquan He
Quoted 54 lines · Baoquan He
Quoted 43 lines · lijiang
Quoted 42 lines · Tom Lendacky
Quoted 21 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 900

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +9 -0
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+

You can probably just use "if (is_kdump_kernel())" here, since memory encryption is either on in both the first and second kernel or off in both the first and second kernel. At which point __sme_clr() will do the proper thing.

Actually, this needs to be done no matter what. When doing either the ioremap_encrypted() or the memremap(), the physical address should not include the encryption bit/mask.

Quoted 5 lines
Quoted 3 lines

Thanks,
Tom

Thanks for your comments. If we don't remove the memory encryption mask, it will return false because the 'old_devtb_phys >= 0x100000000ULL' may become true.

Lianbo, you may not get what Tom suggested. Tom means no matter what it is, encrypted or not in 1st kernel, we need get pure physicall address, and using below code is always right for both cases.

Quoted 5 lines
	if (is_kdump_kernel())
		old_devtb_phys = __sme_clr(old_devtb_phys);

And this is simpler. You even can add one line of code comment to say like "Physical address w/o encryption mask is needed here."

Even simpler, there's no need to even check for is_kdump_kernel(). The __sme_clr() should always be done if the physical address is going to be used for some form of io or memory remapping.

So you could just change the existing:

	old_devtb_phys = entry & PAGE_MASK;

to:

	old_devtb_phys = __sme_clr(entry) & PAGE_MASK;

Thanks,
Tom

Quoted 12 lines
Quoted 12 lines

Lianbo

Quoted 10 lines
Quoted 10 lines
drivers/iommu/amd_iommu_init.c +4 -1
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
 	if (!old_devtb)
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 911

Quoted 3 lines
drivers/iommu/amd_iommu_init.c +0 -0
 		return false;
 

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#16 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

Quoted 65 lines · Tom Lendacky
Quoted 54 lines · Baoquan He
Quoted 43 lines · lijiang
Quoted 42 lines · Tom Lendacky
Quoted 21 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}
 

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 900

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +9 -0
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);
+

You can probably just use "if (is_kdump_kernel())" here, since memory encryption is either on in both the first and second kernel or off in both the first and second kernel. At which point __sme_clr() will do the proper thing.

Actually, this needs to be done no matter what. When doing either the ioremap_encrypted() or the memremap(), the physical address should not include the encryption bit/mask.

Quoted 5 lines
Quoted 3 lines

Thanks,
Tom

Thanks for your comments. If we don't remove the memory encryption mask, it will return false because the 'old_devtb_phys >= 0x100000000ULL' may become true.

Lianbo, you may not get what Tom suggested. Tom means no matter what it is, encrypted or not in 1st kernel, we need get pure physicall address, and using below code is always right for both cases.

Quoted 5 lines
	if (is_kdump_kernel())
		old_devtb_phys = __sme_clr(old_devtb_phys);

And this is simpler. You even can add one line of code comment to say like "Physical address w/o encryption mask is needed here."

Even simpler, there's no need to even check for is_kdump_kernel(). The __sme_clr() should always be done if the physical address is going to be used for some form of io or memory remapping.

Quoted 7 lines

So you could just change the existing:

	old_devtb_phys = entry & PAGE_MASK;

to:

	old_devtb_phys = __sme_clr(entry) & PAGE_MASK;

Agree, this is even better.

Quoted 22 lines
Quoted 13 lines
Quoted 13 lines

Lianbo

Quoted 10 lines
Quoted 10 lines
drivers/iommu/amd_iommu_init.c +4 -1
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);> +
 	if (!old_devtb)
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 911

Quoted 3 lines
drivers/iommu/amd_iommu_init.c +0 -0
 		return false;
 

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#17 lore raw

Re: [PATCH 3/4 V3] Remap the device table of IOMMU in encrypted manner for kdump

In reply to Lianbo Jiang
Quoted 20 lines · Lianbo Jiang

In kdump mode, it will copy the device table of IOMMU from the old device table, which is encrypted when SME is enabled in the first kernel. So we must remap it in encrypted manner in order to be automatically decrypted when we read.

---
Some changes:
1. add some comments
2. clean compile warning.

 drivers/iommu/amd_iommu_init.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu_init.c +0 -0
index 904c575..a20af4c 100644
@@ -889,11 +889,24 @@ static bool copy_device_table(void)
 	}

drivers/iommu/amd_iommu_init.c static bool copy_device_table(void) · line 899

Quoted 10 lines
drivers/iommu/amd_iommu_init.c +8 -0
 
 	old_devtb_phys = entry & PAGE_MASK;
+
+	/*
+	 *  When sme enable in the first kernel, old_devtb_phys includes the
+	 *  memory encryption mask(sme_me_mask), we must remove the memory
+	 *  encryption mask to obtain the true physical address in kdump mode.
+	 */
+	if (mem_encrypt_active() && is_kdump_kernel())
+		old_devtb_phys = __sme_clr(old_devtb_phys);

This is my question asked in cover letter. If 1st kernel sme is off, 2nd kernel sme is on, you remap it with sme enabled or disabled on purpose, why does that not work? It's not concern to this patch.

Quoted 22 lines
drivers/iommu/amd_iommu_init.c +6 -1
+
 	if (old_devtb_phys >= 0x100000000ULL) {
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+	old_devtb = (mem_encrypt_active() && is_kdump_kernel())
+		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
+							dev_table_size)
+		    : memremap(old_devtb_phys, dev_table_size, MEMREMAP_WB);
+
 	if (!old_devtb)
 		return false;
 
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#18 lore raw

PATCH v3 4/4 Help to dump the old memory encrypted into vmcore file

In reply to Lianbo Jiang

In kdump mode, we need to dump the old memory into vmcore file, if SME is enabled in the first kernel, we must remap the old memory in encrypted manner, which will be automatically decrypted when we read from DRAM. It helps to parse the vmcore for some tools.

---
Some changes:
1. add a new file and modify Makefile.
2. remove some code in sev_active().

 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 4 files changed, 79 insertions(+), 6 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
arch/x86/kernel/Makefile +1 -0
index 02d6f5c..afb5bad 100644
@@ -96,6 +96,7 @@ obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
 obj-$(CONFIG_KEXEC_FILE)	+= kexec-bzimage64.o
 obj-$(CONFIG_CRASH_DUMP)	+= crash_dump_$(BITS).o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= crash_dump_encrypt.o
 obj-y				+= kprobes/
 obj-$(CONFIG_MODULES)		+= module.o
 obj-$(CONFIG_DOUBLEFAULT)	+= doublefault.o
arch/x86/kernel/crash_dump_encrypt.c new file +53 -0

53 lines not on this page · raw message

fs/proc/vmcore.c +14 -6
index a45f0af..5200266 100644
@@ -25,6 +25,8 @@
 #include <linux/uaccess.h>
 #include <asm/io.h>
 #include "internal.h"
+#include <linux/mem_encrypt.h>
+#include <asm/pgtable.h>
 
 /* List representing chunks of contiguous memory areas and their offsets in
  * vmcore file.
@@ -86,7 +88,8 @@ static int pfn_is_ram(unsigned long pfn)
 
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
-				u64 *ppos, int userbuf)
+				u64 *ppos, int userbuf,
+				bool encrypted)
 {
 	unsigned long pfn, offset;
 	size_t nr_bytes;
@@ -108,8 +111,11 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 		if (pfn_is_ram(pfn) == 0)
 			memset(buf, 0, nr_bytes);
 		else {
-			tmp = copy_oldmem_page(pfn, buf, nr_bytes,
-						offset, userbuf);
+			tmp = encrypted ? copy_oldmem_page_encrypted(pfn,
+					    buf, nr_bytes, offset, userbuf)
+					: copy_oldmem_page(pfn, buf, nr_bytes,
+							   offset, userbuf);
+
 			if (tmp < 0)
 				return tmp;
 		}
@@ -143,7 +149,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, false);
 }
 
 /*
@@ -151,7 +157,7 @@ ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
  */
 ssize_t __weak elfcorehdr_read_notes(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, sme_active());
 }
 
 /*
@@ -161,6 +167,7 @@ int __weak remap_oldmem_pfn_range(struct vm_area_struct *vma,
 				  unsigned long from, unsigned long pfn,
 				  unsigned long size, pgprot_t prot)
 {
+	prot = pgprot_encrypted(prot);
 	return remap_pfn_range(vma, from, pfn, size, prot);
 }
 
@@ -235,7 +242,8 @@ static ssize_t __read_vmcore(char *buffer, size_t buflen, loff_t *fpos,
 					    m->offset + m->size - *fpos,
 					    buflen);
 			start = m->paddr + *fpos - m->offset;
-			tmp = read_from_oldmem(buffer, tsz, &start, userbuf);
+			tmp = read_from_oldmem(buffer, tsz, &start, userbuf,
+						sme_active());
 			if (tmp < 0)
 				return tmp;
 			buflen -= tsz;
include/linux/crash_dump.h +11 -0
index f7ac2aa..f3414ff 100644
@@ -25,6 +25,17 @@ extern int remap_oldmem_pfn_range(struct vm_area_struct *vma,
 
 extern ssize_t copy_oldmem_page(unsigned long, char *, size_t,
 						unsigned long, int);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+extern ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					   size_t csize, unsigned long offset,
+					   int userbuf);
+#else
+static inline ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					size_t csize, unsigned long offset,
+					int userbuf) {
+	return csize;
+}
+#endif
 void vmcore_cleanup(void);
 
 /* Architecture code defines this if there are other possible ELF
Signature · 1 line
2.9.5
#19 lore raw

Re: [PATCH 4/4 V3] Help to dump the old memory encrypted into vmcore file

Quoted 127 lines · Lianbo Jiang

In kdump mode, we need to dump the old memory into vmcore file, if SME is enabled in the first kernel, we must remap the old memory in encrypted manner, which will be automatically decrypted when we read from DRAM. It helps to parse the vmcore for some tools.

---
Some changes:
1. add a new file and modify Makefile.
2. remove some code in sev_active().

 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 4 files changed, 79 insertions(+), 6 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
arch/x86/kernel/Makefile +1 -0
index 02d6f5c..afb5bad 100644
@@ -96,6 +96,7 @@ obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
 obj-$(CONFIG_KEXEC_FILE)	+= kexec-bzimage64.o
 obj-$(CONFIG_CRASH_DUMP)	+= crash_dump_$(BITS).o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= crash_dump_encrypt.o
 obj-y				+= kprobes/
 obj-$(CONFIG_MODULES)		+= module.o
 obj-$(CONFIG_DOUBLEFAULT)	+= doublefault.o
arch/x86/kernel/crash_dump_encrypt.c new file +53 -0
new file mode 100644
index 0000000..e44ef33
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ *	Memory preserving reboot related code.
+ *
+ *	Created by: Lianbo Jiang ([email protected])
+ *	Copyright (C) RedHat Corporation, 2018. All rights reserved
+ */
+
+#include <linux/errno.h>
+#include <linux/crash_dump.h>
+#include <linux/uaccess.h>
+#include <linux/io.h>
+
+/**
+ * copy_oldmem_page_encrypted - copy one page from "oldmem encrypted"
+ * @pfn: page frame number to be copied
+ * @buf: target memory address for the copy; this can be in kernel address
+ *	space or user address space (see @userbuf)
+ * @csize: number of bytes to copy
+ * @offset: offset in bytes into the page (based on pfn) to begin the copy
+ * @userbuf: if set, @buf is in user address space, use copy_to_user(),
+ *	otherwise @buf is in kernel address space, use memcpy().
+ *
+ * Copy a page from "oldmem encrypted". For this page, there is no pte
+ * mapped in the current kernel. We stitch up a pte, similar to
+ * kmap_atomic.
+ */
+
+ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+		size_t csize, unsigned long offset, int userbuf)
+{
+	void  *vaddr;
+
+	if (!csize)
+		return 0;
+
+	vaddr = (__force void *)ioremap_encrypted(pfn << PAGE_SHIFT,
+						  PAGE_SIZE);
+	if (!vaddr)
+		return -ENOMEM;
+
+	if (userbuf) {
+		if (copy_to_user((void __user *)buf, vaddr + offset, csize)) {
+			iounmap((void __iomem *)vaddr);
+			return -EFAULT;
+		}
+	} else
+		memcpy(buf, vaddr + offset, csize);
+
+	set_iounmap_nonlazy();
+	iounmap((void __iomem *)vaddr);
+	return csize;
+}
fs/proc/vmcore.c +9 -3
index a45f0af..5200266 100644
@@ -25,6 +25,8 @@
 #include <linux/uaccess.h>
 #include <asm/io.h>
 #include "internal.h"
+#include <linux/mem_encrypt.h>
+#include <asm/pgtable.h>
 
 /* List representing chunks of contiguous memory areas and their offsets in
  * vmcore file.
@@ -86,7 +88,8 @@ static int pfn_is_ram(unsigned long pfn)
 
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
-				u64 *ppos, int userbuf)
+				u64 *ppos, int userbuf,
+				bool encrypted)
 {
 	unsigned long pfn, offset;
 	size_t nr_bytes;
@@ -108,8 +111,11 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 		if (pfn_is_ram(pfn) == 0)
 			memset(buf, 0, nr_bytes);
 		else {
-			tmp = copy_oldmem_page(pfn, buf, nr_bytes,
-						offset, userbuf);
+			tmp = encrypted ? copy_oldmem_page_encrypted(pfn,
+					    buf, nr_bytes, offset, userbuf)
+					: copy_oldmem_page(pfn, buf, nr_bytes,
+							   offset, userbuf);
+
 			if (tmp < 0)
 				return tmp;
 		}

fs/proc/vmcore.c void __weak elfcorehdr_free(unsigned long long addr) · line 152

Quoted 5 lines
fs/proc/vmcore.c +1 -1
@@ -143,7 +149,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, false);

The elf header actually stays in kdump kernel reserved memory so it is not "oldmem", the original function is misleading and doing unnecessary things. But as for your patch maybe using it as is is good for the time being and add a code comment why the encrypted is "false".

	/* elfcorehdr stays in kdump kernel memory and it is not encrypted. */
	return read_from_oldmem(buf, count, ppos, 0, false);

I'm thinking to move the function to something like below, still not sure memremap works on every arches or not, still need more test

fs/proc/vmcore.c +19 -1
index cfb6674331fd..40c01cc42b38 100644
@@ -136,6 +136,24 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 	return read;
 }
 
+static ssize_t read_from_mem(char *buf, size_t count, u64 *ppos)
+{
+	resource_size_t offset = (resource_size_t)*ppos;
+	char *kbuf;
+
+	if (!count)
+		return 0;
+
+	kbuf = memremap(offset, count, MEMREMAP_WB);
+	if (!kbuf)
+		return 0;
+
+	memcpy(buf, kbuf, count);
+	memunmap(kbuf);
+
+	return count;
+}
+
 /*
  * Architectures may override this function to allocate ELF header in 2nd kernel
  */
@@ -155,7 +173,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_mem(buf, count, ppos);
 }
 
 /*
Quoted 36 lines
fs/proc/vmcore.c +4 -2
 }
 
 /*
@@ -151,7 +157,7 @@ ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
  */
 ssize_t __weak elfcorehdr_read_notes(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, sme_active());
 }
 
 /*
@@ -161,6 +167,7 @@ int __weak remap_oldmem_pfn_range(struct vm_area_struct *vma,
 				  unsigned long from, unsigned long pfn,
 				  unsigned long size, pgprot_t prot)
 {
+	prot = pgprot_encrypted(prot);
 	return remap_pfn_range(vma, from, pfn, size, prot);
 }
 
@@ -235,7 +242,8 @@ static ssize_t __read_vmcore(char *buffer, size_t buflen, loff_t *fpos,
 					    m->offset + m->size - *fpos,
 					    buflen);
 			start = m->paddr + *fpos - m->offset;
-			tmp = read_from_oldmem(buffer, tsz, &start, userbuf);
+			tmp = read_from_oldmem(buffer, tsz, &start, userbuf,
+						sme_active());
 			if (tmp < 0)
 				return tmp;
 			buflen -= tsz;
include/linux/crash_dump.h +0 -0
index f7ac2aa..f3414ff 100644
@@ -25,6 +25,17 @@ extern int remap_oldmem_pfn_range(struct vm_area_struct *vma,
 

include/linux/crash_dump.h extern int remap_oldmem_pfn_range(struct vm_area_struct *vma, · line 35

Quoted 10 lines
include/linux/crash_dump.h +8 -0
 extern ssize_t copy_oldmem_page(unsigned long, char *, size_t,
 						unsigned long, int);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+extern ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					   size_t csize, unsigned long offset,
+					   int userbuf);
+#else
+static inline ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					size_t csize, unsigned long offset,
+					int userbuf) {

Personally I prefer below because it is too long:

static inline
ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf, size_t csize,
unsigned long offset, int userbuf)

{
	return 0;
}

include/linux/crash_dump.h extern int remap_oldmem_pfn_range(struct vm_area_struct *vma, · line 36

Quoted 1 line
include/linux/crash_dump.h +1 -0
+	return csize;

As above it should be return 0;

Quoted 13 lines
include/linux/crash_dump.h +2 -0
+}
+#endif
 void vmcore_cleanup(void);
 
 /* Architecture code defines this if there are other possible ELF
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Thanks
Dave

author
#20 lore raw

Re: [PATCH 4/4 V3] Help to dump the old memory encrypted into vmcore file

Quoted 130 lines · Dave Young
Quoted 129 lines · Lianbo Jiang

In kdump mode, we need to dump the old memory into vmcore file, if SME is enabled in the first kernel, we must remap the old memory in encrypted manner, which will be automatically decrypted when we read from DRAM. It helps to parse the vmcore for some tools.

---
Some changes:
1. add a new file and modify Makefile.
2. remove some code in sev_active().

 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 4 files changed, 79 insertions(+), 6 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
arch/x86/kernel/Makefile +1 -0
index 02d6f5c..afb5bad 100644
@@ -96,6 +96,7 @@ obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
 obj-$(CONFIG_KEXEC_FILE)	+= kexec-bzimage64.o
 obj-$(CONFIG_CRASH_DUMP)	+= crash_dump_$(BITS).o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= crash_dump_encrypt.o
 obj-y				+= kprobes/
 obj-$(CONFIG_MODULES)		+= module.o
 obj-$(CONFIG_DOUBLEFAULT)	+= doublefault.o
arch/x86/kernel/crash_dump_encrypt.c new file +53 -0
new file mode 100644
index 0000000..e44ef33
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ *	Memory preserving reboot related code.
+ *
+ *	Created by: Lianbo Jiang ([email protected])
+ *	Copyright (C) RedHat Corporation, 2018. All rights reserved
+ */
+
+#include <linux/errno.h>
+#include <linux/crash_dump.h>
+#include <linux/uaccess.h>
+#include <linux/io.h>
+
+/**
+ * copy_oldmem_page_encrypted - copy one page from "oldmem encrypted"
+ * @pfn: page frame number to be copied
+ * @buf: target memory address for the copy; this can be in kernel address
+ *	space or user address space (see @userbuf)
+ * @csize: number of bytes to copy
+ * @offset: offset in bytes into the page (based on pfn) to begin the copy
+ * @userbuf: if set, @buf is in user address space, use copy_to_user(),
+ *	otherwise @buf is in kernel address space, use memcpy().
+ *
+ * Copy a page from "oldmem encrypted". For this page, there is no pte
+ * mapped in the current kernel. We stitch up a pte, similar to
+ * kmap_atomic.
+ */
+
+ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+		size_t csize, unsigned long offset, int userbuf)
+{
+	void  *vaddr;
+
+	if (!csize)
+		return 0;
+
+	vaddr = (__force void *)ioremap_encrypted(pfn << PAGE_SHIFT,
+						  PAGE_SIZE);
+	if (!vaddr)
+		return -ENOMEM;
+
+	if (userbuf) {
+		if (copy_to_user((void __user *)buf, vaddr + offset, csize)) {
+			iounmap((void __iomem *)vaddr);
+			return -EFAULT;
+		}
+	} else
+		memcpy(buf, vaddr + offset, csize);
+
+	set_iounmap_nonlazy();
+	iounmap((void __iomem *)vaddr);
+	return csize;
+}
fs/proc/vmcore.c +9 -3
index a45f0af..5200266 100644
@@ -25,6 +25,8 @@
 #include <linux/uaccess.h>
 #include <asm/io.h>
 #include "internal.h"
+#include <linux/mem_encrypt.h>
+#include <asm/pgtable.h>
 
 /* List representing chunks of contiguous memory areas and their offsets in
  * vmcore file.
@@ -86,7 +88,8 @@ static int pfn_is_ram(unsigned long pfn)
 
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
-				u64 *ppos, int userbuf)
+				u64 *ppos, int userbuf,
+				bool encrypted)
 {
 	unsigned long pfn, offset;
 	size_t nr_bytes;
@@ -108,8 +111,11 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 		if (pfn_is_ram(pfn) == 0)
 			memset(buf, 0, nr_bytes);
 		else {
-			tmp = copy_oldmem_page(pfn, buf, nr_bytes,
-						offset, userbuf);
+			tmp = encrypted ? copy_oldmem_page_encrypted(pfn,
+					    buf, nr_bytes, offset, userbuf)
+					: copy_oldmem_page(pfn, buf, nr_bytes,
+							   offset, userbuf);
+
 			if (tmp < 0)
 				return tmp;
 		}
@@ -143,7 +149,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
Quoted 9 lines

fs/proc/vmcore.c void __weak elfcorehdr_free(unsigned long long addr)

Quoted 3 lines
fs/proc/vmcore.c +1 -1
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, false);

The elf header actually stays in kdump kernel reserved memory so it is not "oldmem", the original function is misleading and doing unnecessary things. But as for your patch maybe using it as is is good for the time being and add a code comment why the encrypted is "false".

Thank you, Dave. It is a good idea to add some comments for the code.
I rechecked the code, the elf header should be still the old memory in the first kernel, but why is the old memory unencrypted? Because it copies the elf header from the memory encrypted(user space) to the memory unencrypted(kernel space) when SME is activated in the first kernel, this operation just leads to decryption.

Thanks.
Lianbo

Quoted 125 lines
	/* elfcorehdr stays in kdump kernel memory and it is not encrypted. */
	return read_from_oldmem(buf, count, ppos, 0, false);

I'm thinking to move the function to something like below, still not sure memremap works on every arches or not, still need more test

fs/proc/vmcore.c +19 -1
index cfb6674331fd..40c01cc42b38 100644
@@ -136,6 +136,24 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 	return read;
 }
 
+static ssize_t read_from_mem(char *buf, size_t count, u64 *ppos)
+{
+	resource_size_t offset = (resource_size_t)*ppos;
+	char *kbuf;
+
+	if (!count)
+		return 0;
+
+	kbuf = memremap(offset, count, MEMREMAP_WB);
+	if (!kbuf)
+		return 0;
+
+	memcpy(buf, kbuf, count);
+	memunmap(kbuf);
+
+	return count;
+}
+
 /*
  * Architectures may override this function to allocate ELF header in 2nd kernel
  */
@@ -155,7 +173,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_mem(buf, count, ppos);
 }
 
 /*
Quoted 38 lines
fs/proc/vmcore.c +4 -2
 }
 
 /*
@@ -151,7 +157,7 @@ ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
  */
 ssize_t __weak elfcorehdr_read_notes(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, sme_active());
 }
 
 /*
@@ -161,6 +167,7 @@ int __weak remap_oldmem_pfn_range(struct vm_area_struct *vma,
 				  unsigned long from, unsigned long pfn,
 				  unsigned long size, pgprot_t prot)
 {
+	prot = pgprot_encrypted(prot);
 	return remap_pfn_range(vma, from, pfn, size, prot);
 }
 
@@ -235,7 +242,8 @@ static ssize_t __read_vmcore(char *buffer, size_t buflen, loff_t *fpos,
 					    m->offset + m->size - *fpos,
 					    buflen);
 			start = m->paddr + *fpos - m->offset;
-			tmp = read_from_oldmem(buffer, tsz, &start, userbuf);
+			tmp = read_from_oldmem(buffer, tsz, &start, userbuf,
+						sme_active());
 			if (tmp < 0)
 				return tmp;
 			buflen -= tsz;
include/linux/crash_dump.h +0 -0
index f7ac2aa..f3414ff 100644
@@ -25,6 +25,17 @@ extern int remap_oldmem_pfn_range(struct vm_area_struct *vma,
 

include/linux/crash_dump.h extern int remap_oldmem_pfn_range(struct vm_area_struct *vma, · line 35

Quoted 10 lines
include/linux/crash_dump.h +8 -0
 extern ssize_t copy_oldmem_page(unsigned long, char *, size_t,
 						unsigned long, int);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+extern ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					   size_t csize, unsigned long offset,
+					   int userbuf);
+#else
+static inline ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					size_t csize, unsigned long offset,
+					int userbuf) {

Personally I prefer below because it is too long:

static inline
ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf, size_t csize,
unsigned long offset, int userbuf)

{
	return 0;
}

include/linux/crash_dump.h extern int remap_oldmem_pfn_range(struct vm_area_struct *vma, · line 36

Quoted 1 line
include/linux/crash_dump.h +1 -0
+	return csize;

As above it should be return 0;

Quoted 13 lines
include/linux/crash_dump.h +2 -0
+}
+#endif
 void vmcore_cleanup(void);
 
 /* Architecture code defines this if there are other possible ELF
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Thanks
Dave

author
#21 lore raw

Re: [PATCH 4/4 V3] Help to dump the old memory encrypted into vmcore file

Quoted 140 lines · lijiang
Quoted 130 lines · Dave Young
Quoted 129 lines · Lianbo Jiang

In kdump mode, we need to dump the old memory into vmcore file, if SME is enabled in the first kernel, we must remap the old memory in encrypted manner, which will be automatically decrypted when we read from DRAM. It helps to parse the vmcore for some tools.

---
Some changes:
1. add a new file and modify Makefile.
2. remove some code in sev_active().

 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 4 files changed, 79 insertions(+), 6 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
arch/x86/kernel/Makefile +1 -0
index 02d6f5c..afb5bad 100644
@@ -96,6 +96,7 @@ obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
 obj-$(CONFIG_KEXEC_FILE)	+= kexec-bzimage64.o
 obj-$(CONFIG_CRASH_DUMP)	+= crash_dump_$(BITS).o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= crash_dump_encrypt.o
 obj-y				+= kprobes/
 obj-$(CONFIG_MODULES)		+= module.o
 obj-$(CONFIG_DOUBLEFAULT)	+= doublefault.o
arch/x86/kernel/crash_dump_encrypt.c new file +53 -0
new file mode 100644
index 0000000..e44ef33
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ *	Memory preserving reboot related code.
+ *
+ *	Created by: Lianbo Jiang ([email protected])
+ *	Copyright (C) RedHat Corporation, 2018. All rights reserved
+ */
+
+#include <linux/errno.h>
+#include <linux/crash_dump.h>
+#include <linux/uaccess.h>
+#include <linux/io.h>
+
+/**
+ * copy_oldmem_page_encrypted - copy one page from "oldmem encrypted"
+ * @pfn: page frame number to be copied
+ * @buf: target memory address for the copy; this can be in kernel address
+ *	space or user address space (see @userbuf)
+ * @csize: number of bytes to copy
+ * @offset: offset in bytes into the page (based on pfn) to begin the copy
+ * @userbuf: if set, @buf is in user address space, use copy_to_user(),
+ *	otherwise @buf is in kernel address space, use memcpy().
+ *
+ * Copy a page from "oldmem encrypted". For this page, there is no pte
+ * mapped in the current kernel. We stitch up a pte, similar to
+ * kmap_atomic.
+ */
+
+ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+		size_t csize, unsigned long offset, int userbuf)
+{
+	void  *vaddr;
+
+	if (!csize)
+		return 0;
+
+	vaddr = (__force void *)ioremap_encrypted(pfn << PAGE_SHIFT,
+						  PAGE_SIZE);
+	if (!vaddr)
+		return -ENOMEM;
+
+	if (userbuf) {
+		if (copy_to_user((void __user *)buf, vaddr + offset, csize)) {
+			iounmap((void __iomem *)vaddr);
+			return -EFAULT;
+		}
+	} else
+		memcpy(buf, vaddr + offset, csize);
+
+	set_iounmap_nonlazy();
+	iounmap((void __iomem *)vaddr);
+	return csize;
+}
fs/proc/vmcore.c +9 -3
index a45f0af..5200266 100644
@@ -25,6 +25,8 @@
 #include <linux/uaccess.h>
 #include <asm/io.h>
 #include "internal.h"
+#include <linux/mem_encrypt.h>
+#include <asm/pgtable.h>
 
 /* List representing chunks of contiguous memory areas and their offsets in
  * vmcore file.
@@ -86,7 +88,8 @@ static int pfn_is_ram(unsigned long pfn)
 
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
-				u64 *ppos, int userbuf)
+				u64 *ppos, int userbuf,
+				bool encrypted)
 {
 	unsigned long pfn, offset;
 	size_t nr_bytes;
@@ -108,8 +111,11 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 		if (pfn_is_ram(pfn) == 0)
 			memset(buf, 0, nr_bytes);
 		else {
-			tmp = copy_oldmem_page(pfn, buf, nr_bytes,
-						offset, userbuf);
+			tmp = encrypted ? copy_oldmem_page_encrypted(pfn,
+					    buf, nr_bytes, offset, userbuf)
+					: copy_oldmem_page(pfn, buf, nr_bytes,
+							   offset, userbuf);
+
 			if (tmp < 0)
 				return tmp;
 		}
@@ -143,7 +149,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
Quoted 9 lines

fs/proc/vmcore.c void __weak elfcorehdr_free(unsigned long long addr)

Quoted 3 lines
fs/proc/vmcore.c +1 -1
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, false);

The elf header actually stays in kdump kernel reserved memory so it is not "oldmem", the original function is misleading and doing unnecessary things. But as for your patch maybe using it as is is good for the time being and add a code comment why the encrypted is "false".

Quoted 6 lines

Thank you, Dave. It is a good idea to add some comments for the code.
I rechecked the code, the elf header should be still the old memory in the first kernel, but why is the old memory unencrypted? Because it copies the elf header from the memory encrypted(user space) to the memory unencrypted(kernel space) when SME is activated in the first kernel, this operation just leads to decryption.

I just know your means, we allocated the memory unencrypted(kernel space) in reserved crash memory, from this point of view, it is not "oldmem". Thanks for your comments.

Thanks.
Lianbo

Quoted 110 lines

Thanks.
Lianbo

Quoted 108 lines
	/* elfcorehdr stays in kdump kernel memory and it is not encrypted. */
	return read_from_oldmem(buf, count, ppos, 0, false);

I'm thinking to move the function to something like below, still not sure memremap works on every arches or not, still need more test

fs/proc/vmcore.c +19 -1
index cfb6674331fd..40c01cc42b38 100644
@@ -136,6 +136,24 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 	return read;
 }
 
+static ssize_t read_from_mem(char *buf, size_t count, u64 *ppos)
+{
+	resource_size_t offset = (resource_size_t)*ppos;
+	char *kbuf;
+
+	if (!count)
+		return 0;
+
+	kbuf = memremap(offset, count, MEMREMAP_WB);
+	if (!kbuf)
+		return 0;
+
+	memcpy(buf, kbuf, count);
+	memunmap(kbuf);
+
+	return count;
+}
+
 /*
  * Architectures may override this function to allocate ELF header in 2nd kernel
  */
@@ -155,7 +173,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_mem(buf, count, ppos);
 }
 
 /*
Quoted 38 lines
fs/proc/vmcore.c +4 -2
 }
 
 /*
@@ -151,7 +157,7 @@ ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
  */
 ssize_t __weak elfcorehdr_read_notes(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, sme_active());
 }
 
 /*
@@ -161,6 +167,7 @@ int __weak remap_oldmem_pfn_range(struct vm_area_struct *vma,
 				  unsigned long from, unsigned long pfn,
 				  unsigned long size, pgprot_t prot)
 {
+	prot = pgprot_encrypted(prot);
 	return remap_pfn_range(vma, from, pfn, size, prot);
 }
 
@@ -235,7 +242,8 @@ static ssize_t __read_vmcore(char *buffer, size_t buflen, loff_t *fpos,
 					    m->offset + m->size - *fpos,
 					    buflen);
 			start = m->paddr + *fpos - m->offset;
-			tmp = read_from_oldmem(buffer, tsz, &start, userbuf);
+			tmp = read_from_oldmem(buffer, tsz, &start, userbuf,
+						sme_active());
 			if (tmp < 0)
 				return tmp;
 			buflen -= tsz;
include/linux/crash_dump.h +0 -0
index f7ac2aa..f3414ff 100644
@@ -25,6 +25,17 @@ extern int remap_oldmem_pfn_range(struct vm_area_struct *vma,
 

include/linux/crash_dump.h extern int remap_oldmem_pfn_range(struct vm_area_struct *vma, · line 35

Quoted 10 lines
include/linux/crash_dump.h +8 -0
 extern ssize_t copy_oldmem_page(unsigned long, char *, size_t,
 						unsigned long, int);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+extern ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					   size_t csize, unsigned long offset,
+					   int userbuf);
+#else
+static inline ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					size_t csize, unsigned long offset,
+					int userbuf) {

Personally I prefer below because it is too long:

static inline
ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf, size_t csize,
unsigned long offset, int userbuf)

{
	return 0;
}

include/linux/crash_dump.h extern int remap_oldmem_pfn_range(struct vm_area_struct *vma, · line 36

Quoted 1 line
include/linux/crash_dump.h +1 -0
+	return csize;

As above it should be return 0;

Great, Thank you, Dave.

Lianbo

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
include/linux/crash_dump.h +2 -0
+}
+#endif
 void vmcore_cleanup(void);
 
 /* Architecture code defines this if there are other possible ELF
Signature · 1 line
2.9.5
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Thanks
Dave

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#22 lore raw

Re: [PATCH 4/4 V3] Help to dump the old memory encrypted into vmcore file

In reply to Lianbo Jiang
Quoted 4 lines · Lianbo Jiang

In kdump mode, we need to dump the old memory into vmcore file, if SME is enabled in the first kernel, we must remap the old memory in encrypted manner, which will be automatically decrypted when we read from DRAM. It helps to parse the vmcore for some tools.

It implies 2nd kernel is sme enabled, right?

Then what if 1st kernel has no sme enabled?

Quoted 188 lines

---
Some changes:
1. add a new file and modify Makefile.
2. remove some code in sev_active().

 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 4 files changed, 79 insertions(+), 6 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
arch/x86/kernel/Makefile +1 -0
index 02d6f5c..afb5bad 100644
@@ -96,6 +96,7 @@ obj-$(CONFIG_KEXEC_CORE)	+= machine_kexec_$(BITS).o
 obj-$(CONFIG_KEXEC_CORE)	+= relocate_kernel_$(BITS).o crash.o
 obj-$(CONFIG_KEXEC_FILE)	+= kexec-bzimage64.o
 obj-$(CONFIG_CRASH_DUMP)	+= crash_dump_$(BITS).o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= crash_dump_encrypt.o
 obj-y				+= kprobes/
 obj-$(CONFIG_MODULES)		+= module.o
 obj-$(CONFIG_DOUBLEFAULT)	+= doublefault.o
arch/x86/kernel/crash_dump_encrypt.c new file +53 -0
new file mode 100644
index 0000000..e44ef33
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ *	Memory preserving reboot related code.
+ *
+ *	Created by: Lianbo Jiang ([email protected])
+ *	Copyright (C) RedHat Corporation, 2018. All rights reserved
+ */
+
+#include <linux/errno.h>
+#include <linux/crash_dump.h>
+#include <linux/uaccess.h>
+#include <linux/io.h>
+
+/**
+ * copy_oldmem_page_encrypted - copy one page from "oldmem encrypted"
+ * @pfn: page frame number to be copied
+ * @buf: target memory address for the copy; this can be in kernel address
+ *	space or user address space (see @userbuf)
+ * @csize: number of bytes to copy
+ * @offset: offset in bytes into the page (based on pfn) to begin the copy
+ * @userbuf: if set, @buf is in user address space, use copy_to_user(),
+ *	otherwise @buf is in kernel address space, use memcpy().
+ *
+ * Copy a page from "oldmem encrypted". For this page, there is no pte
+ * mapped in the current kernel. We stitch up a pte, similar to
+ * kmap_atomic.
+ */
+
+ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+		size_t csize, unsigned long offset, int userbuf)
+{
+	void  *vaddr;
+
+	if (!csize)
+		return 0;
+
+	vaddr = (__force void *)ioremap_encrypted(pfn << PAGE_SHIFT,
+						  PAGE_SIZE);
+	if (!vaddr)
+		return -ENOMEM;
+
+	if (userbuf) {
+		if (copy_to_user((void __user *)buf, vaddr + offset, csize)) {
+			iounmap((void __iomem *)vaddr);
+			return -EFAULT;
+		}
+	} else
+		memcpy(buf, vaddr + offset, csize);
+
+	set_iounmap_nonlazy();
+	iounmap((void __iomem *)vaddr);
+	return csize;
+}
fs/proc/vmcore.c +14 -6
index a45f0af..5200266 100644
@@ -25,6 +25,8 @@
 #include <linux/uaccess.h>
 #include <asm/io.h>
 #include "internal.h"
+#include <linux/mem_encrypt.h>
+#include <asm/pgtable.h>
 
 /* List representing chunks of contiguous memory areas and their offsets in
  * vmcore file.
@@ -86,7 +88,8 @@ static int pfn_is_ram(unsigned long pfn)
 
 /* Reads a page from the oldmem device from given offset. */
 static ssize_t read_from_oldmem(char *buf, size_t count,
-				u64 *ppos, int userbuf)
+				u64 *ppos, int userbuf,
+				bool encrypted)
 {
 	unsigned long pfn, offset;
 	size_t nr_bytes;
@@ -108,8 +111,11 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 		if (pfn_is_ram(pfn) == 0)
 			memset(buf, 0, nr_bytes);
 		else {
-			tmp = copy_oldmem_page(pfn, buf, nr_bytes,
-						offset, userbuf);
+			tmp = encrypted ? copy_oldmem_page_encrypted(pfn,
+					    buf, nr_bytes, offset, userbuf)
+					: copy_oldmem_page(pfn, buf, nr_bytes,
+							   offset, userbuf);
+
 			if (tmp < 0)
 				return tmp;
 		}
@@ -143,7 +149,7 @@ void __weak elfcorehdr_free(unsigned long long addr)
  */
 ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, false);
 }
 
 /*
@@ -151,7 +157,7 @@ ssize_t __weak elfcorehdr_read(char *buf, size_t count, u64 *ppos)
  */
 ssize_t __weak elfcorehdr_read_notes(char *buf, size_t count, u64 *ppos)
 {
-	return read_from_oldmem(buf, count, ppos, 0);
+	return read_from_oldmem(buf, count, ppos, 0, sme_active());
 }
 
 /*
@@ -161,6 +167,7 @@ int __weak remap_oldmem_pfn_range(struct vm_area_struct *vma,
 				  unsigned long from, unsigned long pfn,
 				  unsigned long size, pgprot_t prot)
 {
+	prot = pgprot_encrypted(prot);
 	return remap_pfn_range(vma, from, pfn, size, prot);
 }
 
@@ -235,7 +242,8 @@ static ssize_t __read_vmcore(char *buffer, size_t buflen, loff_t *fpos,
 					    m->offset + m->size - *fpos,
 					    buflen);
 			start = m->paddr + *fpos - m->offset;
-			tmp = read_from_oldmem(buffer, tsz, &start, userbuf);
+			tmp = read_from_oldmem(buffer, tsz, &start, userbuf,
+						sme_active());
 			if (tmp < 0)
 				return tmp;
 			buflen -= tsz;
include/linux/crash_dump.h +11 -0
index f7ac2aa..f3414ff 100644
@@ -25,6 +25,17 @@ extern int remap_oldmem_pfn_range(struct vm_area_struct *vma,
 
 extern ssize_t copy_oldmem_page(unsigned long, char *, size_t,
 						unsigned long, int);
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+extern ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					   size_t csize, unsigned long offset,
+					   int userbuf);
+#else
+static inline ssize_t copy_oldmem_page_encrypted(unsigned long pfn, char *buf,
+					size_t csize, unsigned long offset,
+					int userbuf) {
+	return csize;
+}
+#endif
 void vmcore_cleanup(void);
 
 /* Architecture code defines this if there are other possible ELF
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#23 lore raw
In reply to Lianbo Jiang
Quoted 31 lines · Lianbo Jiang

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

When sme enabled on AMD server, we also need to support kdump. Because the memory is encrypted in the first kernel, we will remap the old memory encrypted to the second kernel(crash kernel), and sme is also enabled in the second kernel, otherwise the old memory encrypted can not be decrypted. Because simply changing the value of a C-bit on a page will not automatically encrypt the existing contents of a page, and any data in the page prior to the C-bit modification will become unintelligible. A page of memory that is marked encrypted will be automatically decrypted when read from DRAM and will be automatically encrypted when written to DRAM.

For the kdump, it is necessary to distinguish whether the memory is encrypted. Furthermore, we should also know which part of the memory is encrypted or decrypted. We will appropriately remap the memory according to the specific situation in order to tell cpu how to deal with the data(encrypted or decrypted). For example, when sme enabled, if the old memory is encrypted, we will remap the old memory in encrypted way, which will automatically decrypt the old memory encrypted when we read those data from the remapping address.

----------------------------------------------

| first-kernel | second-kernel | kdump support |
|      (mem_encrypt=on|off)    |   (yes|no)    |
|--------------+---------------+---------------|
|     on       |     on        |     yes       |
|     off      |     off       |     yes       |
|     on       |     off       |     no        |
Quoted 1 line

| off | on | no |

It's not clear to me here. If 1st kernel sme is off, in 2nd kernel, when you remap the old memory with non-sme mode, why did it fail?

And please run scripts/get_maintainer.pl and add maintainers of component which is affected in patch to CC list.

Quoted 133 lines

|______________|_______________|_______________|

This patch is only for SME kdump, it is not support SEV kdump.

Test tools:
makedumpfile[v1.6.3]: https://github.com/LianboJ/makedumpfile
commit e1de103eca8f (A draft for kdump vmcore about AMD SME)
Author: Lianbo Jiang <[email protected]>
Date: Mon May 14 17:02:40 2018 +0800
Note: This patch can only dump vmcore in the case of SME enabled.

crash-7.2.1: https://github.com/crash-utility/crash.git
commit 1e1bd9c4c1be (Fix for the "bpf" command display on Linux 4.17-rc1)

Author: Dave Anderson <[email protected]>
Date:   Fri May 11 15:54:32 2018 -0400

Test environment:
HP ProLiant DL385Gen10 AMD EPYC 7251
8-Core Processor
32768 MB memory
600 GB disk space

Linux 4.17-rc7:
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
commit b04e217704b7 ("Linux 4.17-rc7")

Author: Linus Torvalds <torvalds-de/[email protected]>
Date:   Sun May 27 13:01:47 2018 -0700

Reference:
AMD64 Architecture Programmer's Manual
https://support.amd.com/TechDocs/24593.pdf

Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. remove the '#ifdef' stuff throughout this patch.
3. put some logic into the early_memremap_pgprot_adjust() and clean the previous unnecessary changes, for example: arch/x86/include/asm/dmi.h, arch/x86/kernel/acpi/boot.c, drivers/acpi/tables.c.
4. add a new file and modify Makefile.
5. clean compile warning in copy_device_table() and some compile error.
6. split the original patch into four patches, it will be better for review.

Some known issues:
1. about SME
Upstream kernel doesn't work when we use kexec in the follow command. The system will hang.
(This issue doesn't matter with the kdump patch.)

Reproduce steps:

 # kexec -l /boot/vmlinuz-4.17.0-rc7+ --initrd=/boot/initramfs-4.17.0-rc7+.img --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=on rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug nokaslr"
 # kexec -e (or reboot)

The system will hang:

[ 1248.932239] kexec_core: Starting new kernel

early console in extract_kernel
input_data: 0x000000087e91c3b4
input_len: 0x000000000067fcbd
output: 0x000000087d400000
output_len: 0x0000000001b6fa90
kernel_total_size: 0x0000000001a9d000
trampoline_32bit: 0x0000000000099000

Decompressing Linux...
Parsing ELF... [-here the system will hang]

2. about SEV
Upstream kernel(Host OS) doesn't work in host side, some drivers about SEV always go wrong in host side. We can't boot SEV Guest OS to test kdump patch. Maybe it is more reasonable to improve SEV in another patch. When some drivers can work in host side and it can also boot Virtual Machine(SEV Guest OS), it will be suitable to fix SEV for kdump.

[  369.426131] INFO: task systemd-udevd:865 blocked for more than 120 seconds.
[  369.433177]       Not tainted 4.17.0-rc5+ #60
[  369.437585] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  369.445783] systemd-udevd   D    0   865    813 0x80000004
[  369.451323] Call Trace:
[  369.453815]  ? __schedule+0x290/0x870
[  369.457523]  schedule+0x32/0x80
[  369.460714]  __sev_do_cmd_locked+0x1f6/0x2a0 [ccp]
[  369.465556]  ? cleanup_uevent_env+0x10/0x10
[  369.470084]  ? remove_wait_queue+0x60/0x60
[  369.474219]  ? 0xffffffffc0247000
[  369.477572]  __sev_platform_init_locked+0x2b/0x70 [ccp]
[  369.482843]  sev_platform_init+0x1d/0x30 [ccp]
[  369.487333]  psp_pci_init+0x40/0xe0 [ccp]
[  369.491380]  ? 0xffffffffc0247000
[  369.494936]  sp_mod_init+0x18/0x1000 [ccp]
[  369.499071]  do_one_initcall+0x4e/0x1d4
[  369.502944]  ? _cond_resched+0x15/0x30
[  369.506728]  ? kmem_cache_alloc_trace+0xae/0x1d0
[  369.511386]  ? do_init_module+0x22/0x220
[  369.515345]  do_init_module+0x5a/0x220
[  369.519444]  load_module+0x21cb/0x2a50
[  369.523227]  ? m_show+0x1c0/0x1c0
[  369.526571]  ? security_capable+0x3f/0x60
[  369.530611]  __do_sys_finit_module+0x94/0xe0
[  369.534915]  do_syscall_64+0x5b/0x180
[  369.538607]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  369.543698] RIP: 0033:0x7f708e6311b9
[  369.547536] RSP: 002b:00007ffff9d32aa8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  369.555162] RAX: ffffffffffffffda RBX: 000055602a04c2d0 RCX: 00007f708e6311b9
[  369.562346] RDX: 0000000000000000 RSI: 00007f708ef52039 RDI: 0000000000000008
[  369.569801] RBP: 00007f708ef52039 R08: 0000000000000000 R09: 000055602a048b20
[  369.576988] R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000
[  369.584177] R13: 000055602a075260 R14: 0000000000020000 R15: 0000000000000000

Lianbo Jiang (4):
Add a function(ioremap_encrypted) for kdump when AMD sme enabled Allocate pages for kdump without encryption when SME is enabled Remap the device table of IOMMU in encrypted manner for kdump Help to dump the old memory encrypted into vmcore file

 arch/x86/include/asm/io.h            |  3 ++
 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/ioremap.c                | 28 +++++++++++++------
 drivers/iommu/amd_iommu_init.c       | 15 +++++++++-
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 kernel/kexec_core.c                  | 12 ++++++++
 8 files changed, 128 insertions(+), 15 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#24 lore raw
Quoted 32 lines · Baoquan He
Quoted 29 lines · Lianbo Jiang

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

When sme enabled on AMD server, we also need to support kdump. Because the memory is encrypted in the first kernel, we will remap the old memory encrypted to the second kernel(crash kernel), and sme is also enabled in the second kernel, otherwise the old memory encrypted can not be decrypted. Because simply changing the value of a C-bit on a page will not automatically encrypt the existing contents of a page, and any data in the page prior to the C-bit modification will become unintelligible. A page of memory that is marked encrypted will be automatically decrypted when read from DRAM and will be automatically encrypted when written to DRAM.

For the kdump, it is necessary to distinguish whether the memory is encrypted. Furthermore, we should also know which part of the memory is encrypted or decrypted. We will appropriately remap the memory according to the specific situation in order to tell cpu how to deal with the data(encrypted or decrypted). For example, when sme enabled, if the old memory is encrypted, we will remap the old memory in encrypted way, which will automatically decrypt the old memory encrypted when we read those data from the remapping address.

----------------------------------------------

| first-kernel | second-kernel | kdump support |
|      (mem_encrypt=on|off)    |   (yes|no)    |
|--------------+---------------+---------------|
|     on       |     on        |     yes       |
|     off      |     off       |     yes       |
|     on       |     off       |     no        |
Quoted 5 lines
Quoted 1 line

| off | on | no |

It's not clear to me here. If 1st kernel sme is off, in 2nd kernel, when you remap the old memory with non-sme mode, why did it fail?

Thank you, Baoquan.
For kdump, there are two cases that doesn't need to support:

1. SME on(first kernel), but SME off(second kernel).
Because the old memory is encrypted, we can't decrypt the old memory if SME is off in the second kernel(in kdump mode).

2. SME off(first kernel), but SME on(second kernel)
Maybe this situation doesn't have significance in actual deployment, furthermore, it will also increase the complexity of the code. It's just for testing, maybe it is unnecessary to support it, because the old memory is unencrypted.

Thanks.
Lianbo

Quoted 2 lines

And please run scripts/get_maintainer.pl and add maintainers of component which is affected in patch to CC list.

Great! I forgot CC maintainers, thanks for your reminder.

Lianbo

Quoted 126 lines
Quoted 126 lines

|______________|_______________|_______________|

This patch is only for SME kdump, it is not support SEV kdump.

Test tools:
makedumpfile[v1.6.3]: https://github.com/LianboJ/makedumpfile
commit e1de103eca8f (A draft for kdump vmcore about AMD SME)
Author: Lianbo Jiang <[email protected]>
Date: Mon May 14 17:02:40 2018 +0800
Note: This patch can only dump vmcore in the case of SME enabled.

crash-7.2.1: https://github.com/crash-utility/crash.git
commit 1e1bd9c4c1be (Fix for the "bpf" command display on Linux 4.17-rc1)

Author: Dave Anderson <[email protected]>
Date:   Fri May 11 15:54:32 2018 -0400

Test environment:
HP ProLiant DL385Gen10 AMD EPYC 7251
8-Core Processor
32768 MB memory
600 GB disk space

Linux 4.17-rc7:
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
commit b04e217704b7 ("Linux 4.17-rc7")

Author: Linus Torvalds <[email protected]>
Date:   Sun May 27 13:01:47 2018 -0700

Reference:
AMD64 Architecture Programmer's Manual
https://support.amd.com/TechDocs/24593.pdf

Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. remove the '#ifdef' stuff throughout this patch.
3. put some logic into the early_memremap_pgprot_adjust() and clean the previous unnecessary changes, for example: arch/x86/include/asm/dmi.h, arch/x86/kernel/acpi/boot.c, drivers/acpi/tables.c.
4. add a new file and modify Makefile.
5. clean compile warning in copy_device_table() and some compile error.
6. split the original patch into four patches, it will be better for review.

Some known issues:
1. about SME
Upstream kernel doesn't work when we use kexec in the follow command. The system will hang.
(This issue doesn't matter with the kdump patch.)

Reproduce steps:

 # kexec -l /boot/vmlinuz-4.17.0-rc7+ --initrd=/boot/initramfs-4.17.0-rc7+.img --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=on rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug nokaslr"
 # kexec -e (or reboot)

The system will hang:

[ 1248.932239] kexec_core: Starting new kernel

early console in extract_kernel
input_data: 0x000000087e91c3b4
input_len: 0x000000000067fcbd
output: 0x000000087d400000
output_len: 0x0000000001b6fa90
kernel_total_size: 0x0000000001a9d000
trampoline_32bit: 0x0000000000099000

Decompressing Linux...
Parsing ELF... [-here the system will hang]

2. about SEV
Upstream kernel(Host OS) doesn't work in host side, some drivers about SEV always go wrong in host side. We can't boot SEV Guest OS to test kdump patch. Maybe it is more reasonable to improve SEV in another patch. When some drivers can work in host side and it can also boot Virtual Machine(SEV Guest OS), it will be suitable to fix SEV for kdump.

[  369.426131] INFO: task systemd-udevd:865 blocked for more than 120 seconds.
[  369.433177]       Not tainted 4.17.0-rc5+ #60
[  369.437585] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  369.445783] systemd-udevd   D    0   865    813 0x80000004
[  369.451323] Call Trace:
[  369.453815]  ? __schedule+0x290/0x870
[  369.457523]  schedule+0x32/0x80
[  369.460714]  __sev_do_cmd_locked+0x1f6/0x2a0 [ccp]
[  369.465556]  ? cleanup_uevent_env+0x10/0x10
[  369.470084]  ? remove_wait_queue+0x60/0x60
[  369.474219]  ? 0xffffffffc0247000
[  369.477572]  __sev_platform_init_locked+0x2b/0x70 [ccp]
[  369.482843]  sev_platform_init+0x1d/0x30 [ccp]
[  369.487333]  psp_pci_init+0x40/0xe0 [ccp]
[  369.491380]  ? 0xffffffffc0247000
[  369.494936]  sp_mod_init+0x18/0x1000 [ccp]
[  369.499071]  do_one_initcall+0x4e/0x1d4
[  369.502944]  ? _cond_resched+0x15/0x30
[  369.506728]  ? kmem_cache_alloc_trace+0xae/0x1d0
[  369.511386]  ? do_init_module+0x22/0x220
[  369.515345]  do_init_module+0x5a/0x220
[  369.519444]  load_module+0x21cb/0x2a50
[  369.523227]  ? m_show+0x1c0/0x1c0
[  369.526571]  ? security_capable+0x3f/0x60
[  369.530611]  __do_sys_finit_module+0x94/0xe0
[  369.534915]  do_syscall_64+0x5b/0x180
[  369.538607]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  369.543698] RIP: 0033:0x7f708e6311b9
[  369.547536] RSP: 002b:00007ffff9d32aa8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  369.555162] RAX: ffffffffffffffda RBX: 000055602a04c2d0 RCX: 00007f708e6311b9
[  369.562346] RDX: 0000000000000000 RSI: 00007f708ef52039 RDI: 0000000000000008
[  369.569801] RBP: 00007f708ef52039 R08: 0000000000000000 R09: 000055602a048b20
[  369.576988] R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000
[  369.584177] R13: 000055602a075260 R14: 0000000000020000 R15: 0000000000000000

Lianbo Jiang (4):
Add a function(ioremap_encrypted) for kdump when AMD sme enabled Allocate pages for kdump without encryption when SME is enabled Remap the device table of IOMMU in encrypted manner for kdump Help to dump the old memory encrypted into vmcore file

 arch/x86/include/asm/io.h            |  3 ++
 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/ioremap.c                | 28 +++++++++++++------
 drivers/iommu/amd_iommu_init.c       | 15 +++++++++-
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 kernel/kexec_core.c                  | 12 ++++++++
 8 files changed, 128 insertions(+), 15 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
Quoted 8 lines
Quoted 8 lines
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#25 lore raw
Quoted 41 lines · lijiang
Quoted 32 lines · Baoquan He
Quoted 29 lines · Lianbo Jiang

It is convenient to remap the old memory encrypted to the second kernel by calling ioremap_encrypted().

When sme enabled on AMD server, we also need to support kdump. Because the memory is encrypted in the first kernel, we will remap the old memory encrypted to the second kernel(crash kernel), and sme is also enabled in the second kernel, otherwise the old memory encrypted can not be decrypted. Because simply changing the value of a C-bit on a page will not automatically encrypt the existing contents of a page, and any data in the page prior to the C-bit modification will become unintelligible. A page of memory that is marked encrypted will be automatically decrypted when read from DRAM and will be automatically encrypted when written to DRAM.

For the kdump, it is necessary to distinguish whether the memory is encrypted. Furthermore, we should also know which part of the memory is encrypted or decrypted. We will appropriately remap the memory according to the specific situation in order to tell cpu how to deal with the data(encrypted or decrypted). For example, when sme enabled, if the old memory is encrypted, we will remap the old memory in encrypted way, which will automatically decrypt the old memory encrypted when we read those data from the remapping address.

----------------------------------------------

| first-kernel | second-kernel | kdump support |
|      (mem_encrypt=on|off)    |   (yes|no)    |
|--------------+---------------+---------------|
|     on       |     on        |     yes       |
|     off      |     off       |     yes       |
|     on       |     off       |     no        |
Quoted 5 lines
Quoted 1 line

| off | on | no |

It's not clear to me here. If 1st kernel sme is off, in 2nd kernel, when you remap the old memory with non-sme mode, why did it fail?

Thank you, Baoquan.
For kdump, there are two cases that doesn't need to support:

Quoted 8 lines

1. SME on(first kernel), but SME off(second kernel).
Because the old memory is encrypted, we can't decrypt the old memory if SME is off in the second kernel(in kdump mode).

2. SME off(first kernel), but SME on(second kernel)
Maybe this situation doesn't have significance in actual deployment, furthermore, it will also increase the complexity of the code. It's just for testing, maybe it is unnecessary to support it, because the old memory is unencrypted.

Hmm, sorry, I don't get why it is unnecessary because the old memory is unencrypted. We developers should cover all cases unless a certain case is no way to fix, or fixing it is not cost-effective. Otherwise there's no execuse for us to not fix. When QE try their best to cover all test cases, they don't consider its significance because even corner case need be swept out, that is their job. There isn't a rule for developers to decide if it's significant.

Or could you point out what complexity it will bring to defend your point?

Thanks
Baoquan

Quoted 139 lines
Quoted 2 lines

And please run scripts/get_maintainer.pl and add maintainers of component which is affected in patch to CC list.

Great! I forgot CC maintainers, thanks for your reminder.

Lianbo

Quoted 134 lines
Quoted 134 lines

|______________|_______________|_______________|

This patch is only for SME kdump, it is not support SEV kdump.

Test tools:
makedumpfile[v1.6.3]: https://github.com/LianboJ/makedumpfile
commit e1de103eca8f (A draft for kdump vmcore about AMD SME)
Author: Lianbo Jiang <[email protected]>
Date: Mon May 14 17:02:40 2018 +0800
Note: This patch can only dump vmcore in the case of SME enabled.

crash-7.2.1: https://github.com/crash-utility/crash.git
commit 1e1bd9c4c1be (Fix for the "bpf" command display on Linux 4.17-rc1)

Author: Dave Anderson <[email protected]>
Date:   Fri May 11 15:54:32 2018 -0400

Test environment:
HP ProLiant DL385Gen10 AMD EPYC 7251
8-Core Processor
32768 MB memory
600 GB disk space

Linux 4.17-rc7:
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
commit b04e217704b7 ("Linux 4.17-rc7")

Author: Linus Torvalds <[email protected]>
Date:   Sun May 27 13:01:47 2018 -0700

Reference:
AMD64 Architecture Programmer's Manual
https://support.amd.com/TechDocs/24593.pdf

Some changes:
1. remove the sme_active() check in __ioremap_caller().
2. remove the '#ifdef' stuff throughout this patch.
3. put some logic into the early_memremap_pgprot_adjust() and clean the previous unnecessary changes, for example: arch/x86/include/asm/dmi.h, arch/x86/kernel/acpi/boot.c, drivers/acpi/tables.c.
4. add a new file and modify Makefile.
5. clean compile warning in copy_device_table() and some compile error.
6. split the original patch into four patches, it will be better for review.

Some known issues:
1. about SME
Upstream kernel doesn't work when we use kexec in the follow command. The system will hang.
(This issue doesn't matter with the kdump patch.)

Reproduce steps:

 # kexec -l /boot/vmlinuz-4.17.0-rc7+ --initrd=/boot/initramfs-4.17.0-rc7+.img --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=on rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug nokaslr"
 # kexec -e (or reboot)

The system will hang:

[ 1248.932239] kexec_core: Starting new kernel

early console in extract_kernel
input_data: 0x000000087e91c3b4
input_len: 0x000000000067fcbd
output: 0x000000087d400000
output_len: 0x0000000001b6fa90
kernel_total_size: 0x0000000001a9d000
trampoline_32bit: 0x0000000000099000

Decompressing Linux...
Parsing ELF... [-here the system will hang]

2. about SEV
Upstream kernel(Host OS) doesn't work in host side, some drivers about SEV always go wrong in host side. We can't boot SEV Guest OS to test kdump patch. Maybe it is more reasonable to improve SEV in another patch. When some drivers can work in host side and it can also boot Virtual Machine(SEV Guest OS), it will be suitable to fix SEV for kdump.

[  369.426131] INFO: task systemd-udevd:865 blocked for more than 120 seconds.
[  369.433177]       Not tainted 4.17.0-rc5+ #60
[  369.437585] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  369.445783] systemd-udevd   D    0   865    813 0x80000004
[  369.451323] Call Trace:
[  369.453815]  ? __schedule+0x290/0x870
[  369.457523]  schedule+0x32/0x80
[  369.460714]  __sev_do_cmd_locked+0x1f6/0x2a0 [ccp]
[  369.465556]  ? cleanup_uevent_env+0x10/0x10
[  369.470084]  ? remove_wait_queue+0x60/0x60
[  369.474219]  ? 0xffffffffc0247000
[  369.477572]  __sev_platform_init_locked+0x2b/0x70 [ccp]
[  369.482843]  sev_platform_init+0x1d/0x30 [ccp]
[  369.487333]  psp_pci_init+0x40/0xe0 [ccp]
[  369.491380]  ? 0xffffffffc0247000
[  369.494936]  sp_mod_init+0x18/0x1000 [ccp]
[  369.499071]  do_one_initcall+0x4e/0x1d4
[  369.502944]  ? _cond_resched+0x15/0x30
[  369.506728]  ? kmem_cache_alloc_trace+0xae/0x1d0
[  369.511386]  ? do_init_module+0x22/0x220
[  369.515345]  do_init_module+0x5a/0x220
[  369.519444]  load_module+0x21cb/0x2a50
[  369.523227]  ? m_show+0x1c0/0x1c0
[  369.526571]  ? security_capable+0x3f/0x60
[  369.530611]  __do_sys_finit_module+0x94/0xe0
[  369.534915]  do_syscall_64+0x5b/0x180
[  369.538607]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  369.543698] RIP: 0033:0x7f708e6311b9
[  369.547536] RSP: 002b:00007ffff9d32aa8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[  369.555162] RAX: ffffffffffffffda RBX: 000055602a04c2d0 RCX: 00007f708e6311b9
[  369.562346] RDX: 0000000000000000 RSI: 00007f708ef52039 RDI: 0000000000000008
[  369.569801] RBP: 00007f708ef52039 R08: 0000000000000000 R09: 000055602a048b20
[  369.576988] R10: 0000000000000008 R11: 0000000000000246 R12: 0000000000000000
[  369.584177] R13: 000055602a075260 R14: 0000000000020000 R15: 0000000000000000

Lianbo Jiang (4):
Add a function(ioremap_encrypted) for kdump when AMD sme enabled Allocate pages for kdump without encryption when SME is enabled Remap the device table of IOMMU in encrypted manner for kdump Help to dump the old memory encrypted into vmcore file

 arch/x86/include/asm/io.h            |  3 ++
 arch/x86/kernel/Makefile             |  1 +
 arch/x86/kernel/crash_dump_encrypt.c | 53 ++++++++++++++++++++++++++++++++++++
 arch/x86/mm/ioremap.c                | 28 +++++++++++++------
 drivers/iommu/amd_iommu_init.c       | 15 +++++++++-
 fs/proc/vmcore.c                     | 20 ++++++++++----
 include/linux/crash_dump.h           | 11 ++++++++
 kernel/kexec_core.c                  | 12 ++++++++
 8 files changed, 128 insertions(+), 15 deletions(-)
 create mode 100644 arch/x86/kernel/crash_dump_encrypt.c
Signature · 7 lines
2.9.5


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu