Add restrictions for kexec/kdump jumping between 5-level and 4-level kernel
Subsystems (3)
cover Add restrictions for kexec/kdump jumping between 5-level and 4-level kernel
This is a RESEND post.
The original v1 post can be found here:
http://lkml.kernel.org/r/[email protected]
It's trying to fix several corner case issues for kexec/kdump when dynamic switching of paging mode is enabled in x86_64. Please click above link to check the details.
In v1, hpa raised concern that why the paging mode checking is not done before kexec jumping, the discussion can be found here:
http://lkml.kernel.org/r/[email protected]
As tglx said, it might be not doable for kdump since kdump kernel's reserved crashkernel region only owns a portion of memory, may be above 4G; and might be not safer to do paging mode checking and switching thing after crash.
So resend this patchset.
Baoquan He (3):
x86/boot: Add bit fields into xloadflags for 5-level kernel checking x86/kexec/64: Error out if try to jump to old 4-level kernel from
5-level kernel
x86/kdump/64: Change the upper limit of crashkernel reservation
arch/x86/boot/header.S | 12 +++++++++++- arch/x86/include/uapi/asm/bootparam.h | 2 ++ arch/x86/kernel/kexec-bzimage64.c | 5 +++++ arch/x86/kernel/setup.c | 18 ++++++++++++++---- 4 files changed, 32 insertions(+), 5 deletions(-)
Signature · 1 line
2.13.6
PATCH RESEND 1/3 x86/boot: Add bit fields into xloadflags for 5-level kernel checking
Add two bit fields XLF_5LEVEL and XLF_5LEVEL_ENABLED for 5-level kernel. Bit XLF_5LEVEL indicates if 5-level related code is contained in this kernel.
Bit XLF_5LEVEL_ENABLED indicates if CONFIG_X86_5LEVEL=y is set.
They are being used in later patch to check if kexec/kdump kernel is loaded in right place.
--- arch/x86/boot/header.S | 12 +++++++++++- arch/x86/include/uapi/asm/bootparam.h | 2 ++ 2 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/boot/header.S +11 -1
@@ -419,7 +419,17 @@ xloadflags: # define XLF4 0 #endif - .word XLF0 | XLF1 | XLF23 | XLF4 +#ifdef CONFIG_X86_64 +#ifdef CONFIG_X86_5LEVEL +#define XLF56 (XLF_5LEVEL|XLF_5LEVEL_ENABLED) +#else +#define XLF56 XLF_5LEVEL +#endif +#else +#define XLF56 0 +#endif + + .word XLF0 | XLF1 | XLF23 | XLF4 | XLF56 cmdline_size: .long COMMAND_LINE_SIZE-1 #length of the command line, #added with boot protocol
arch/x86/include/uapi/asm/bootparam.h +2 -0
@@ -29,6 +29,8 @@#define XLF_EFI_HANDOVER_32 (1<<2)#define XLF_EFI_HANDOVER_64 (1<<3)#define XLF_EFI_KEXEC (1<<4) +#define XLF_5LEVEL (1<<5) +#define XLF_5LEVEL_ENABLED (1<<6)#ifndef __ASSEMBLY__
Signature · 7 lines
2.13.6 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH RESEND 1/3] x86/boot: Add bit fields into xloadflags for 5-level kernel checking
Quoted 1 line · Baoquan He
Add two bit fields
XLF_5LEVELandXLF_5LEVEL_ENABLEDfor 5-level kernel.
These are not bit fields. These are simple bits.
Quoted 3 lines
Bit
XLF_5LEVELindicates if 5-level related code is contained in this kernel.
BitXLF_5LEVEL_ENABLEDindicates ifCONFIG_X86_5LEVEL=yis set.
I'm confused.
arch/x86/boot/header.S xloadflags: · line 432
Quoted 12 lines
arch/x86/boot/header.S +11
-1- .word XLF0 | XLF1 | XLF23 | XLF4 +#ifdef CONFIG_X86_64 +#ifdef CONFIG_X86_5LEVEL +#define XLF56 (XLF_5LEVEL|XLF_5LEVEL_ENABLED) +#else +#define XLF56 XLF_5LEVEL +#endif +#else +#define XLF56 0 +#endif + + .word XLF0 | XLF1 | XLF23 | XLF4 | XLF56
So this actually stores the bits, but looking at the following patch which fixes the real issue:
Review
Quoted 4 lines
(quoted diff) +4
-0+ if (!(header->xloadflags & XLF_5LEVEL) && pgtable_l5_enabled()) { + pr_err("Can not jump to old 4-level kernel from 5-level kernel.\n"); + return ret; + }
So what is XLF_5LEVEL_ENABLED used for and why does it exist at all?
Thanks,
tglx
Re: [PATCH RESEND 1/3] x86/boot: Add bit fields into xloadflags for 5-level kernel checking
Thanks for reviewing. I was in vacation, sorry for late reply.
Quoted 5 lines · Thomas Gleixner
Quoted 1 line · Baoquan He
Add two bit fields
XLF_5LEVELandXLF_5LEVEL_ENABLEDfor 5-level kernel.These are not bit fields. These are simple bits.
Indeed, they are only xloadflags bits, will change. Thanks.
Quoted 23 lines
Quoted 4 lines
Bit
XLF_5LEVELindicates if 5-level related code is contained in this kernel.
BitXLF_5LEVEL_ENABLEDindicates ifCONFIG_X86_5LEVEL=yis set.I'm confused.
arch/x86/boot/header.S xloadflags: · line 432
Quoted 12 lines
arch/x86/boot/header.S +11
-1- .word XLF0 | XLF1 | XLF23 | XLF4 +#ifdef CONFIG_X86_64 +#ifdef CONFIG_X86_5LEVEL +#define XLF56 (XLF_5LEVEL|XLF_5LEVEL_ENABLED) +#else +#define XLF56 XLF_5LEVEL +#endif +#else +#define XLF56 0 +#endif + + .word XLF0 | XLF1 | XLF23 | XLF4 | XLF56So this actually stores the bits, but looking at the following patch which fixes the real issue:
Quoted 6 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ if (!(header->xloadflags & XLF_5LEVEL) && pgtable_l5_enabled()) { + pr_err("Can not jump to old 4-level kernel from 5-level kernel.\n"); + return ret; + }So what is
XLF_5LEVEL_ENABLEDused for and why does it exist at all?
Yes, this is a little bit confusing. I explained it in the v1 cover letter:
http://lists.infradead.org/pipermail/kexec/2018-August/021419.html
As told at above, XLF_5LEVEL marks the new kernel containing 5level code, while XLF_5LEVEL_ENABLED marking the CONFIG_X86_5LEVEL option enabling. Hence if XLF_5LEVEL is set, XLF_5LEVEL_ENABLED not, means it's new kernel but can't be switched into 5-level.
For kexec_load and kexec_file_load, there's difference in loading behaviour. kexec_load will search available area top down to put kernel in system RAM, we need check if the kexec-ed kernel is in leve-5 paging mode, and limit the loading postion below 64 TB if not. But for kexec_file_load, it's searching area bottom up to put kernel, most of time area found below 4G. We don't have worry about the kexec_file_load interface which implements the loading functionality in kernel. That's why the XLF_5LEVEL_ENABLED bit is not used in this kernel patch set, I would like to post patch to kexec-tools for kexec_load after these patches have been accepted.
I ever tried to unify the behavious of these two interfaces on loading kernel, to make both kexec_load and kexec_file_load search and put kernel top to down, but that involves many lines of code change, seems people are worried about it and hesitated to offere ack, I just gave up. Please check below link:
https://lore.kernel.org/lkml/[email protected]/T/#u
Sorry for the inconvenience because of my missing explanation.
Thanks
Baoquan
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH v2 1/3 x86/boot: Add xloadflags bits for 5-level kernel checking
In reply to Baoquan HeAdd two bits XLF_5LEVEL and XLF_5LEVEL_ENABLED for 5-level kernel. Bit XLF_5LEVEL indicates if 5-level related code is contained in this kernel.
Bit XLF_5LEVEL_ENABLED indicates if CONFIG_X86_5LEVEL=y is set.
They are being used in later patch to check if kexec/kdump kernel is loaded in right place.
---v1->v2:
Change the wrong term in subject pointed out by tglx.
arch/x86/boot/header.S | 12 +++++++++++- arch/x86/include/uapi/asm/bootparam.h | 2 ++ 2 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/boot/header.S +11 -1
@@ -419,7 +419,17 @@ xloadflags: # define XLF4 0 #endif - .word XLF0 | XLF1 | XLF23 | XLF4 +#ifdef CONFIG_X86_64 +#ifdef CONFIG_X86_5LEVEL +#define XLF56 (XLF_5LEVEL|XLF_5LEVEL_ENABLED) +#else +#define XLF56 XLF_5LEVEL +#endif +#else +#define XLF56 0 +#endif + + .word XLF0 | XLF1 | XLF23 | XLF4 | XLF56 cmdline_size: .long COMMAND_LINE_SIZE-1 #length of the command line, #added with boot protocol
arch/x86/include/uapi/asm/bootparam.h +2 -0
@@ -29,6 +29,8 @@#define XLF_EFI_HANDOVER_32 (1<<2)#define XLF_EFI_HANDOVER_64 (1<<3)#define XLF_EFI_KEXEC (1<<4) +#define XLF_5LEVEL (1<<5) +#define XLF_5LEVEL_ENABLED (1<<6)#ifndef __ASSEMBLY__
Signature · 7 lines
2.17.2 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH RESEND 2/3 x86/kexec/64: Error out if try to jump to old 4-level kernel from 5-level kernel
In reply to Baoquan HeIn relocate_kernel() CR4.LA57 flag is set before kexec jumping if the kernel has 5-level paging enabled. Then in boot/compressed/head_64.S, it will check if the booting kernel is in 4-level or 5-level paging mode, and handle accordingly. However, the old kernel which doesn't contain the 5-level codes doesn't know how to cope with it, then #GP triggered.
Instead of triggering #GP during kexec kernel boot, error out during kexec loading if find out we are trying to jump to old 4-level kernel from 5-level kernel.
--- arch/x86/kernel/kexec-bzimage64.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/kernel/kexec-bzimage64.c +5 -0
@@ -316,6 +316,11 @@ static int bzImage64_probe(const char *buf, unsigned long len)return ret; } +if (!(header->xloadflags &XLF_5LEVEL ) &&pgtable_l5_enabled ()) { +pr_err ("Can not jump to old 4-level kernel from 5-level kernel.\n" ); +return ret; + } +/* I've got a bzImage */ pr_debug ("It's a relocatable bzImage64\n" ); ret =0 ;
Signature · 7 lines
2.13.6 _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH RESEND 3/3 x86/kdump/64: Change the upper limit of crashkernel reservation
In reply to Baoquan HeRestrict kdump to only reserve crashkernel below 64TB. Since the kdump jumping may be from 5-level to 4-level, and the kdump kernel is put above 64TB in 5-level kernel, then the jumping will fail. And the crashkernel reservation is done during the 1st kernel bootup, there's no way to detect the paging mode of kdump kernel at that time.
Hence change the upper limit of crashkernel reservation to 64TB on x86_64.
--- arch/x86/kernel/setup.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
arch/x86/kernel/setup.c +14 -4
@@ -451,16 +451,26 @@ static void __init memblock_x86_reserve_range_setup_data(void)#define CRASH_ALIGN (16 << 20)/* -* Keep the crash kernel below this limit. On 32 bits earlier kernels -* would limit the kernel to the low 512 MiB due to mapping restrictions. -* On 64bit, old kexec-tools need to under 896MiB. +* Keep the crash kernel below this limit. +* +* On 32 bits earlier kernels would limit the kernel to the low +* 512 MiB due to mapping restrictions. +* +* On 64bit, old kexec-tools need to be under 896MiB. The later +* supports to put kernel above 4G, up to system RAM top. Here +* kdump kernel need be restricted to be under 64TB, which is +* the upper limit of system RAM in 4-level paing mode. Since +* the kdump jumping could be from 5-level to 4-level, the jumping +* will fail if kernel is put above 64TB, and there's no way to +* detect the paging mode of the kernel which will be loaded for +* dumping during the 1st kernel bootup. */ #ifdef CONFIG_X86_32 # define CRASH_ADDR_LOW_MAX (512 << 20)# define CRASH_ADDR_HIGH_MAX (512 << 20)#else # define CRASH_ADDR_LOW_MAX (896UL << 20) -# define CRASH_ADDR_HIGH_MAX MAXMEM +# define CRASH_ADDR_HIGH_MAX (1ULL < 46)#endif static int __init reserve_crashkernel_low (void )
Signature · 1 line
2.13.6
Quoted 12 lines · Baoquan He
This is a RESEND post.
The original v1 post can be found here:
http://lkml.kernel.org/r/[email protected]It's trying to fix several corner case issues for kexec/kdump when dynamic switching of paging mode is enabled in
x86_64. Please click above link to check the details.In v1, hpa raised concern that why the paging mode checking is not done before kexec jumping, the discussion can be found here:
Quoted 8 lines
http://lkml.kernel.org/r/[email protected]
As tglx said, it might be not doable for kdump since kdump kernel's reserved crashkernel region only owns a portion of memory, may be above 4G; and might be not safer to do paging mode checking and switching thing after crash.
So resend this patchset.
Changes look good to me:
Signature · 6 lines
Kirill A. Shutemov _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec