Skip to contentSkip to conversation

x86/crash: Disable SGX in the kdump kernel

PATCH NAK Yuanhe Shu 3 participants 7 messages Nov 13

Subsystems (4)
lore
Share & tools
lore

Yuanhe Shu

author
#0 lore raw

PATCH v2 1/1 x86/crash: Disable SGX in the kdump kernel

The kdump capture kernel cannot use SGX: EPC memory is e820-reserved, so it is not part of the crash dump, and no enclave will ever run during dump capture. Yet sgx_init() runs unconditionally and pays the full EPC bookkeeping: sgx_setup_epc_section() allocates one struct sgx_epc_page (32 bytes) per 4K EPC page, i.e. EPC/128, plus a memremap of the whole EPC range.

The capture kernel has no memory other than the crashkernel reservation. On a VM with 15.5G EPC the bookkeeping alone costs 124 MB of vmalloc - nearly half of the stock crashkernel=256M - and the cost scales with EPC size, not RAM.

That is enough to break kdump outright: with the default 256M the capture kernel OOMs in early userspace (udevadm and journald are killed before the root device is enumerated) and no vmcore is produced, while 512M works. With this change the same 256M reservation captures a complete vmcore; SGX initializes as usual on normal boots.

Clear X86_FEATURE_SGX from an x86 early handler for elfcorehdr=, which only the capture kernel receives, so sgx_init() bails out on its existing capability check and no kdump knowledge leaks into the SGX code. Like nosgx, it has to be an early parameter: clearing a feature after alternatives are patched is too late.

---
v1: https://lore.kernel.org/r/[email protected]/

- Clear X86_FEATURE_SGX from an x86 elfcorehdr= early handler instead of checking is_kdump_kernel() in sgx_init() (Dave Hansen); retitled to x86/crash accordingly.

 arch/x86/kernel/crash_dump_64.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/kernel/crash_dump_64.c +14 -0
index 32d710f7eb84..4a275b969f3c 100644
@@ -12,6 +12,8 @@
 #include <linux/io.h>
 #include <linux/cc_platform.h>
 
+#include <asm/cpufeature.h>
+
 static ssize_t __copy_oldmem_page(struct iov_iter *iter, unsigned long pfn,
 				  size_t csize, unsigned long offset,
 				  bool encrypted)
@@ -62,3 +64,15 @@ ssize_t elfcorehdr_read(char *buf, size_t count, u64 *ppos)
 	return read_from_oldmem(&iter, count, ppos,
 				cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT));
 }
+
+/*
+ * elfcorehdr= is passed only to the capture kernel.  Turn off the
+ * features it has no use for.
+ */
+static int __init kdump_disable_features(char *arg)
+{
+	/* SGX metadata for a large EPC can exhaust crashkernel memory. */
+	setup_clear_cpu_cap(X86_FEATURE_SGX);
+	return 0;
+}
+early_param("elfcorehdr", kdump_disable_features);
Signature · 1 line
2.43.7
#1 lore raw
Sashiko has reviewed this patch and found no issues.
Quoted 24 lines

The kdump capture kernel cannot use SGX: EPC memory is e820-reserved, so it is not part of the crash dump, and no enclave will ever run during dump capture. Yet sgx_init() runs unconditionally and pays the full EPC bookkeeping: sgx_setup_epc_section() allocates one struct sgx_epc_page (32 bytes) per 4K EPC page, i.e. EPC/128, plus a memremap of the whole EPC range.

The capture kernel has no memory other than the crashkernel reservation. On a VM with 15.5G EPC the bookkeeping alone costs 124 MB of vmalloc - nearly half of the stock crashkernel=256M - and the cost scales with EPC size, not RAM.

That is enough to break kdump outright: with the default 256M the capture kernel OOMs in early userspace (udevadm and journald are killed before the root device is enumerated) and no vmcore is produced, while 512M works. With this change the same 256M reservation captures a complete vmcore; SGX initializes as usual on normal boots.

Clear X86_FEATURE_SGX from an x86 early handler for elfcorehdr=, which only the capture kernel receives, so sgx_init() bails out on its existing capability check and no kdump knowledge leaks into the SGX code. Like nosgx, it has to be an early parameter: clearing a feature after alternatives are patched is too late.

Quoted 3 lines

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
maintainer
#2 lore raw
In reply to Yuanhe Shu
Quoted 1 line · Yuanhe Shu

Clear X86_FEATURE_SGX from an x86 early handler for elfcorehdr=, which

First of all this cmdline param naming is maximally unintuitive and why does it even exist in the first place?

In case some user decides at kdump load time to disable sgx?

When do distros normally load kdump kernels? During boot?

So it is probably a script doing that.

So no user is even going to pay attention to this param or even know of its existence.

And how does an init script know that the kdump kernel should not dump EPC mem?

Do you catch my drift here?

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author
#3 lore raw
Quoted 2 lines · Borislav Petkov

First of all this cmdline param naming is maximally unintuitive and why does it even exist in the first place?

Thanks for the review.

elfcorehdr= is passed by kexec, so every capture kernel boots with it on the command line. We use it to detect the capture kernel, the same way is_kdump_kernel() does: that helper checks elfcorehdr_addr != ELFCORE_ADDR_MAX, and this parameter is what sets the address.

It also keeps the disable logic in kdump-specific code in the x86 path, as suggested in the v1 review.

We looked at two places: adding an is_kdump_kernel() check in setup_arch(), and this handler. We chose the latter to keep the disable logic in kdump code rather than the generic boot path. As an early parameter, it clears SGX before alternatives are patched.

--
Yuanhe

maintainer NAK
#4 lore raw
Quoted 15 lines · Yuanhe Shu
Quoted 2 lines · Borislav Petkov

First of all this cmdline param naming is maximally unintuitive and why does it even exist in the first place?

Thanks for the review.

elfcorehdr= is passed by kexec, so every capture kernel boots with it on the command line. We use it to detect the capture kernel, the same way is_kdump_kernel() does: that helper checks elfcorehdr_addr != ELFCORE_ADDR_MAX, and this parameter is what sets the address.

It also keeps the disable logic in kdump-specific code in the x86 path, as suggested in the v1 review.

Quoted 7 lines

We looked at two places: adding an is_kdump_kernel() check in setup_arch(), and this handler. We chose the latter to keep the disable logic in kdump code rather than the generic boot path. As an early parameter, it clears SGX before alternatives are patched.

--
Yuanhe

I don't think you even read my mail.

NAK.

Signature · 1 line
Small device. Typos and formatting crap
author
#5 lore raw
Quoted 1 line · Borislav Petkov

I don't think you even read my mail.

Sorry, I misunderstood your point - language and my reading comprehension got in the way.

Based on the v1 review and your feedback, my proposed approach is a small kdump_disable_features() in the x86 kdump code that checks is_kdump_kernel() and clears the cap, called from setup_arch() after parse_early_param(). This keeps the disable in x86 kdump code without a parameter hook. Does that work for you?

--
Yuanhe

maintainer
#6 lore raw
Quoted 8 lines · Yuanhe Shu

Sorry, I misunderstood your point - language and my reading comprehension got in the way.

Based on the v1 review and your feedback, my proposed approach is a small kdump_disable_features() in the x86 kdump code that checks is_kdump_kernel() and clears the cap, called from setup_arch() after parse_early_param(). This keeps the disable in x86 kdump code without a parameter hook. Does that work for you?

You should do what Dave told you:

https://lore.kernel.org/all/[email protected]

and no more.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette