x86/boot: Allow to perform randomization for uncompressed kernel image
Subsystems (2)
cover x86/boot: Allow to perform randomization for uncompressed kernel image
Hi all,
This RFC patch series introduces relocatable uncompressed kernel image, which is allowed to perform kerenl image virtual address randomization in 64-bit booting entry instead of decompression phase.
- Background
Currently, kernel image virtual address randomization is only performed during the decompression phase. However, in certain scenarios, such as secure container environments (e.g., Kata Containers), to speed up the boot process, the system may boot directly from an uncompressed kernel image. In such cases, virtual address randomization cannot be executed. Although the security enhancement provided by KASLR is limited, there is still a potential demand to allow uncompressed kernel images to perform virtual address randomization (for example, future support for x86 PIE).
- Approaches
Currently, the x86 kernel uses static compilation, but it retains relocation information through the '--emit-relocs' option, which is then simplified into a relocation table using 'relocs' tool. To enable virtual address randomization for uncompressed kernel images, relocation information is required, and there are several possible approaches:
1) Who will perform the randomization:
VMM: The VMM reads vmlinux.relocs after loading vmlinux to perform randomization. This would require additional modifications to the VMM, and vmlinux.relocs needs to be packaged when shipping.
Kernel: The kernel performs randomization itself at the kernel entry point, requiring no modifications to the VMM.
2) relocation information format:
vmlinux.relocs: It only contains the necessary relocation entries and is simplified, making it small enough. However, it is a format defined within the kernel that was previously used only internally and is not part of the ABI.
rela.* sections: It is the standard ELF ABI, but
it contains RIP-relative relocation entries, which are more common in kernel, causing the kernel image to be larger.
- Implementation
The final implementation of this plan extends the 'relocs' tool to allow the insertion of relocation information into a reserved section of the kernel (referencing the MIPS implementation). This enables the reading of that information and subsequent execution of relocations when booting directly from an uncompressed kernel. Currently, this implementation is only available for 64-bit and has been tested with both PVH entry booting and standard 64-bit Linux entry. And the default reserve size is 1MB for now, which is enough for defconfig.
- TODO
Clean up the decompression KASLR code to allow it to be shared with the booting phase.
Thanks!
Hou Wenlong (5):
x86/relocs: Cleanup cmdline options
x86/relocs: Insert relocations into input file
x86: Allow to build relocatable uncompressed kernel binary x86/boot: Perform virtual address relocation in kernel entry x86/boot: Use '.data.relocs' section for performing relocations during
decompression
arch/x86/Kconfig | 20 ++++++ arch/x86/Makefile.postlink | 33 +++++++++ arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/misc.c | 8 +++ arch/x86/boot/startup/Makefile | 1 + arch/x86/boot/startup/kaslr.c | 116 ++++++++++++++++++++++++++++++ arch/x86/include/asm/setup.h | 1 + arch/x86/kernel/head_64.S | 7 ++ arch/x86/kernel/vmlinux.lds.S | 20 ++++++ arch/x86/lib/cmdline.c | 6 ++ arch/x86/lib/kaslr.c | 5 ++ arch/x86/platform/pvh/head.S | 15 +++- arch/x86/tools/relocs.c | 64 ++++++++++++++--- arch/x86/tools/relocs.h | 15 ++-- arch/x86/tools/relocs_common.c | 24 ++++--- 15 files changed, 309 insertions(+), 32 deletions(-) create mode 100644 arch/x86/Makefile.postlink create mode 100644 arch/x86/boot/startup/kaslr.c
--
2.31.1
RFC PATCH 1/5 x86/relocs: Cleanup cmdline options
Move all cmdline options to global variables to make code cleaner and to facilitate the addition of new cmdline options.
--- arch/x86/tools/relocs.c | 12 +++++------- arch/x86/tools/relocs.h | 14 ++++++++------ arch/x86/tools/relocs_common.c | 13 +++++-------- 3 files changed, 18 insertions(+), 21 deletions(-)
arch/x86/tools/relocs.c +5 -7
@@ -127,7 +127,7 @@ static int is_reloc(enum symtype type, const char *sym_name)return sym_regex[type] && !regexec (&sym_regex_c[type], sym_name,0 ,NULL ,0 ); } -static void regex_init (int use_real_mode) +static void regex_init (void ) {char errbuf[128 ];int err;
@@ -950,7 +950,7 @@ static int write32_as_text(uint32_t v, FILE *f)return fprintf (f,"\t.long 0x%08" PRIx32"\n" , v) >0 ?0 :-1 ; } -static void emit_relocs (int as_text,int use_real_mode) +static void emit_relocs (void ) {int i;int (*write_reloc)(uint32_t ,FILE *) = write32;
@@ -1049,11 +1049,9 @@ static void print_reloc_info(void)# define process process_32#endif -void process (FILE *fp,int use_real_mode,int as_text, -int show_absolute_syms,int show_absolute_relocs, -int show_reloc_info) +void process (FILE *fp) { -regex_init (use_real_mode); +regex_init ();read_ehdr (fp);read_shdrs (fp);read_strtabs (fp);
@@ -1075,5 +1073,5 @@ void process(FILE *fp, int use_real_mode, int as_text,return ; } -emit_relocs (as_text, use_real_mode); +emit_relocs (); }
arch/x86/tools/relocs.h +8 -6
@@ -30,10 +30,12 @@ enum symtype {S_NSYMTYPES }; -void process_32 (FILE *fp,int use_real_mode,int as_text, -int show_absolute_syms,int show_absolute_relocs, -int show_reloc_info); -void process_64 (FILE *fp,int use_real_mode,int as_text, -int show_absolute_syms,int show_absolute_relocs, -int show_reloc_info); +extern int show_absolute_syms; +extern int show_absolute_relocs; +extern int show_reloc_info; +extern int as_text; +extern int use_real_mode; + +void process_32 (FILE *fp); +void process_64 (FILE *fp);#endif /* RELOCS_H */
arch/x86/tools/relocs_common.c +5 -8
@@ -1,6 +1,9 @@// SPDX-License-Identifier: GPL-2.0 #include "relocs.h" +int show_absolute_syms, show_absolute_relocs, show_reloc_info; +int as_text, use_real_mode; +void die (char *fmt, ...) {va_list ap;
@@ -18,8 +21,6 @@ static void usage(void)int main (int argc,char **argv) { -int show_absolute_syms, show_absolute_relocs, show_reloc_info; -int as_text, use_real_mode;const char *fname;FILE *fp;int i;
@@ -73,13 +74,9 @@ int main(int argc, char **argv) }rewind (fp);if (e_ident[EI_CLASS ] ==ELFCLASS64 ) -process_64 (fp, use_real_mode, as_text, - show_absolute_syms, show_absolute_relocs, - show_reloc_info); +process_64 (fp);else -process_32 (fp, use_real_mode, as_text, - show_absolute_syms, show_absolute_relocs, - show_reloc_info); +process_32 (fp);fclose (fp);return 0 ; }
Signature · 1 line
2.31.1
RFC PATCH 2/5 x86/relocs: Insert relocations into input file
In reply to Hou WenlongAdd a command line option to insert relocations into a reserved section named ".data.reloc" section of the input file. This allows for relocation of the uncompressed kernel and is same as the implementation in MIPS.
--- arch/x86/tools/relocs.c | 56 ++++++++++++++++++++++++++++++---- arch/x86/tools/relocs.h | 1 + arch/x86/tools/relocs_common.c | 13 ++++++-- 3 files changed, 61 insertions(+), 9 deletions(-)
arch/x86/tools/relocs.c +50 -6
@@ -735,6 +735,17 @@ static void walk_relocs(int (*process)(struct section *sec, Elf_Rel *rel, } } +static struct section *sec_lookup (const char *name) +{ +int i; + +for (i =0 ; i < shnum; i++) +if (!strcmp (sec_name (i), name)) +return &secs[i]; + +return NULL ; +} +#if ELF_BITS ==64 static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym,
@@ -950,7 +961,7 @@ static int write32_as_text(uint32_t v, FILE *f)return fprintf (f,"\t.long 0x%08" PRIx32"\n" , v) >0 ?0 :-1 ; } -static void emit_relocs (void ) +static void emit_relocs (FILE *f) {int i;int (*write_reloc)(uint32_t ,FILE *) = write32;
@@ -1001,21 +1012,54 @@ static void emit_relocs(void)for (i =0 ; i < relocs32.count; i++)write_reloc (relocs32.offset[i], stdout); }else { +FILE *outf = stdout; + +if (keep_relocs) { +struct section *sec_reloc; +uint32_t size_needed; +unsigned long offset; + + sec_reloc =sec_lookup (".data.reloc" ); +if (!sec_reloc) +die ("Could not find relocation data section\n" ); + + size_needed = (1 + relocs32.count) *sizeof (uint32_t ); +#if ELF_BITS ==64 + size_needed += (1 + relocs64.count) *sizeof (uint32_t ); +#endif +if (size_needed > sec_reloc->shdr.sh_size) +die ("Relocations overflow available space!\n" \ +"Please adjust CONFIG_RELOCATION_TABLE_SIZE " \ +"to at least 0x%08x\n" , (size_needed +0x1000 ) & ~0xFFF ); + +/* +* Place the relocations at the end of section to ensure +* compatibility with backward traversal during handling +* relocations. +*/ + offset = sec_reloc->shdr.sh_offset + sec_reloc->shdr.sh_size - + size_needed; +if (fseek (f, offset,SEEK_SET ) <0 ) +die ("Seek to %ld failed: %s\n" , offset,strerror (errno)); + + outf = f; + } +#if ELF_BITS ==64 /* Print a stop */ -write_reloc (0 , stdout); +write_reloc (0 , outf);/* Now print each relocation */ for (i =0 ; i < relocs64.count; i++) -write_reloc (relocs64.offset[i], stdout); +write_reloc (relocs64.offset[i], outf);#endif /* Print a stop */ -write_reloc (0 , stdout); +write_reloc (0 , outf);/* Now print each relocation */ for (i =0 ; i < relocs32.count; i++) -write_reloc (relocs32.offset[i], stdout); +write_reloc (relocs32.offset[i], outf); } }
@@ -1073,5 +1117,5 @@ void process(FILE *fp)return ; } -emit_relocs (); +emit_relocs (fp); }
arch/x86/tools/relocs.h +1 -0
@@ -35,6 +35,7 @@ extern int show_absolute_relocs;extern int show_reloc_info;extern int as_text;extern int use_real_mode; +extern int keep_relocs;void process_32 (FILE *fp);void process_64 (FILE *fp);
arch/x86/tools/relocs_common.c +10 -3
@@ -2,7 +2,7 @@#include "relocs.h" int show_absolute_syms, show_absolute_relocs, show_reloc_info; -int as_text, use_real_mode; +int as_text, use_real_mode, keep_relocs;void die (char *fmt, ...) {
@@ -15,7 +15,7 @@ void die(char *fmt, ...)static void usage (void ) { -die ("relocs [--abs-syms|--abs-relocs|--reloc-info|--text|--realmode]" \ +die ("relocs [--abs-syms|--abs-relocs|--reloc-info|--text|--realmode|--keep]" \" vmlinux\n" ); }
@@ -55,6 +55,10 @@ int main(int argc, char **argv) use_real_mode =1 ;continue ; } +if (strcmp (arg,"--keep" ) ==0 ) { + keep_relocs =1 ; +continue ; + } }else if (!fname) { fname = arg;
@@ -65,7 +69,10 @@ int main(int argc, char **argv)if (!fname) {usage (); } - fp =fopen (fname,"r" ); +if (keep_relocs) + fp =fopen (fname,"r+" ); +else + fp =fopen (fname,"r" );if (!fp) {die ("Cannot open %s: %s\n" , fname,strerror (errno)); }
Signature · 1 line
2.31.1
RFC PATCH 3/5 x86: Allow to build relocatable uncompressed kernel binary
In reply to Hou WenlongAdd support for building a relocatable uncompressed kernel binary, which contains the relocation table generated by the 'relocs' tool and can be used to perform KASLR directly when booting from it instead of a compressed kernel image. To simplify, follow the MIPS implementation to reserve a section to store the relocation table.
--- arch/x86/Kconfig | 20 ++++++++++++++++++++ arch/x86/Makefile.postlink | 33 +++++++++++++++++++++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 20 ++++++++++++++++++++ 3 files changed, 73 insertions(+) create mode 100644 arch/x86/Makefile.postlink
arch/x86/Kconfig +20 -0
@@ -2124,6 +2124,26 @@ config RANDOMIZE_BASE If unsure, say Y. +config RELOCATABLE_UNCOMPRESSED_KERNEL + bool "Build a relocatable uncompressed kernel" + depends on RANDOMIZE_BASE && X86_64 + help + A table of relocation data will be appended to the uncompressed + kernel binary and parsed at boot to do relocation. + +config RELOCATION_TABLE_SIZE + hex "Relocation table size" + depends on RELOCATABLE_UNCOMPRESSED_KERNEL + range 0x0 0x04000000 + default "0x00200000" + help + This option allows the amount of space reserved for the table to be + adjusted, although the default of 1Mb should be ok in most cases. + + The build will fail and a valid size suggested if this is too small. + + If unsure, leave at the default value. + # Relocation on x86 needs some additional build support config X86_NEED_RELOCS def_bool y
arch/x86/Makefile.postlink new file +33 -0
@@ -0,0 +1,33 @@ +# SPDX-License-Identifier: GPL-2.0 +# =========================================================================== +# Post-link x86 pass +# =========================================================================== +# +# 1. Insert relocations into vmlinux + +PHONY := __archpost +__archpost: + +-include include/config/auto.conf +include $(srctree)/scripts/Kbuild.include + +CMD_RELOCS = arch/x86/tools/relocs +quiet_cmd_relocs = RELOCS $@ + cmd_relocs = $(CMD_RELOCS) --keep $@ + +# `@true` prevents complaint when there is nothing to be done + +vmlinux vmlinux.unstripped: FORCE + @true +ifeq ($(CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL),y) + $(call if_changed,relocs) +endif + +clean: + @true + +PHONY += FORCE clean + +FORCE: + +.PHONY: $(PHONY)
arch/x86/kernel/vmlinux.lds.S +20 -0
@@ -289,6 +289,26 @@ SECTIONS } #endif +#ifdef CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL + . = ALIGN(4); + .data.reloc : AT(ADDR(.data.reloc) - LOAD_OFFSET) { + __relocation_start = .; + /* + * Space for relocation table + * This needs to be filled so that the + * relocs tool can overwrite the content. + * An invalid value is left at the start of the + * section to abort relocation if the table + * has not been filled in. + */ + LONG(0xFFFFFFFF); + FILL(0); + . += CONFIG_RELOCATION_TABLE_SIZE - 4; + __relocation_end = .; + __pi___relocation_end = .; + } +#endif + /* * struct alt_inst entries. From the header (alternative.h): * "Alternative instructions for different CPU types or capabilities"
Signature · 1 line
2.31.1
RFC PATCH 4/5 x86/boot: Perform virtual address relocation in kernel entry
In reply to Hou WenlongPerform virtual address relocation for the uncompressed kernel during booting, which is similar to the relocation during decompression.
--- arch/x86/boot/startup/Makefile | 1 + arch/x86/boot/startup/kaslr.c | 116 +++++++++++++++++++++++++++++++++ arch/x86/include/asm/setup.h | 1 + arch/x86/kernel/head_64.S | 7 ++ arch/x86/lib/cmdline.c | 6 ++ arch/x86/lib/kaslr.c | 5 ++ arch/x86/platform/pvh/head.S | 15 ++++- 7 files changed, 148 insertions(+), 3 deletions(-) create mode 100644 arch/x86/boot/startup/kaslr.c
arch/x86/boot/startup/Makefile +1 -0
@@ -20,6 +20,7 @@ KCOV_INSTRUMENT := n obj-$(CONFIG_X86_64) += gdt_idt.o map_kernel.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += sme.o sev-startup.o +obj-$(CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL) += kaslr.o pi-objs := $(patsubst %.o,$(obj)/%.o,$(obj-y)) lib-$(CONFIG_X86_64) += la57toggle.o
arch/x86/boot/startup/kaslr.c new file +116 -0
116 lines not on this page · raw message
arch/x86/include/asm/setup.h +1 -0
@@ -56,6 +56,7 @@ extern void startup_64_load_idt(void *vc_handler);extern void __pi_startup_64_load_idt (void *vc_handler);extern void early_setup_idt (void );extern void __init do_early_exception (struct pt_regs *regs,int trapnr); +extern void __init __relocate_kernel (unsigned long p2v_offset,struct boot_params *bp);#ifdef CONFIG_X86_INTEL_MID extern void x86_intel_mid_early_setup (void );
arch/x86/kernel/head_64.S +7 -0
@@ -97,6 +97,13 @@ SYM_CODE_START_NOALIGN(startup_64) /* Sanitize CPU configuration */ call verify_cpu +#ifdef CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL + leaq common_startup_64(%rip), %rdi + subq .Lcommon_startup_64(%rip), %rdi + movq %r15, %rsi + call __pi___relocate_kernel +#endif + /* * Derive the kernel's physical-to-virtual offset from the physical and * virtual addresses of common_startup_64().
arch/x86/lib/cmdline.c +6 -0
@@ -11,6 +11,10 @@#include <asm/cmdline.h> #include <asm/bug.h> +#ifndef CMDLINE_FUNC_PREFIX +#define CMDLINE_FUNC_PREFIX +#endif +static inline int myisspace (u8 c) {return c <=' ' ;/* Close enough approximation */
@@ -205,6 +209,7 @@ __cmdline_find_option(const char *cmdline, int max_cmdline_size,return len; } +CMDLINE_FUNC_PREFIX int cmdline_find_option_bool(const char *cmdline, const char *option) {int ret;
@@ -219,6 +224,7 @@ int cmdline_find_option_bool(const char *cmdline, const char *option)return ret; } +CMDLINE_FUNC_PREFIX int cmdline_find_option(const char *cmdline, const char *option, char *buffer,int bufsize) {
arch/x86/lib/kaslr.c +5 -0
@@ -13,6 +13,10 @@#include <asm/e820/api.h> #include <asm/shared/io.h> +#ifndef KASLR_FUNC_PREFIX +#define KASLR_FUNC_PREFIX +#endif +/* * When built for the regular kernel, several functions need to be stubbed out * or changed to their regular kernel equivalent.
@@ -46,6 +50,7 @@ static inline u16 i8254(void)return timer; } +KASLR_FUNC_PREFIX unsigned long kaslr_get_random_long (const char *purpose) {#ifdef CONFIG_X86_64
arch/x86/platform/pvh/head.S +12 -3
@@ -103,6 +103,17 @@ SYM_CODE_START(pvh_start_xen) btsl $_EFER_LME, %eax wrmsr + /* + * Fill the identity mapping entries instead of preconstructing them, + * as later relocations in __relocation_kernel() would modify them and + * break the mapping if they are prefilled, due to the generation of + * relocation entries. + */ + leal rva(pvh_init_top_pgt)(%ebp), %edi + addl $(pvh_level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC), (%edi) + leal rva(pvh_level3_ident_pgt)(%ebp), %edi + addl $(pvh_level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC), (%edi) + /* * Reuse the non-relocatable symbol emitted for the ELF note to * subtract the build time physical address of pvh_start_xen() from
@@ -254,7 +265,6 @@ SYM_DATA_END_LABEL(early_stack, SYM_L_LOCAL, early_stack_end) * startup_64 transitions to init_top_pgt. */ SYM_DATA_START_PAGE_ALIGNED(pvh_init_top_pgt) - .quad pvh_level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC .org pvh_init_top_pgt + L4_PAGE_OFFSET * 8, 0 .quad pvh_level3_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC .org pvh_init_top_pgt + L4_START_KERNEL * 8, 0
@@ -263,8 +273,7 @@ SYM_DATA_START_PAGE_ALIGNED(pvh_init_top_pgt) SYM_DATA_END(pvh_init_top_pgt) SYM_DATA_START_PAGE_ALIGNED(pvh_level3_ident_pgt) - .quad pvh_level2_ident_pgt - __START_KERNEL_map + _KERNPG_TABLE_NOENC - .fill 511, 8, 0 + .fill 512, 8, 0 SYM_DATA_END(pvh_level3_ident_pgt) SYM_DATA_START_PAGE_ALIGNED(pvh_level2_ident_pgt) /*
Signature · 1 line
2.31.1
RFC PATCH 5/5 x86/boot: Use '.data.relocs' section for performing relocations during decompression
In reply to Hou WenlongWhen CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL is enabled, all the relocations are already in the '.data.relocs' section of the kernel binary, so there is no need to generate 'vmlinux.relocs' again. Use it directly for performing relocations during decompression.
--- arch/x86/boot/compressed/Makefile | 6 ++++-- arch/x86/boot/compressed/misc.c | 8 ++++++++ 2 files changed, 12 insertions(+), 2 deletions(-)
arch/x86/boot/compressed/Makefile +4 -2
@@ -12,7 +12,7 @@ # vmlinux.bin is: # vmlinux stripped of debugging and comments # vmlinux.bin.all is: -# vmlinux.bin + vmlinux.relocs +# vmlinux.bin + vmlinux.relocs (CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL=n) # vmlinux.bin.(gz|bz2|lzma|...) is: # (see scripts/Makefile.lib size_append) # compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
@@ -76,7 +76,7 @@ LDFLAGS_vmlinux += -T hostprogs := mkpiggy HOST_EXTRACFLAGS += -I$(srctree)/tools/include -sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|_sinittext\|__inittext_end\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|_sinittext\|__inittext_end\|__bss_start\|_end\|__relocation_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' quiet_cmd_voffset = VOFFSET $@ cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@
@@ -132,7 +132,9 @@ $(obj)/vmlinux.relocs: vmlinux.unstripped FORCE $(call if_changed,relocs) vmlinux.bin.all-y := $(obj)/vmlinux.bin +ifndef CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs +endif $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE $(call if_changed,gzip)
arch/x86/boot/compressed/misc.c +8 -0
@@ -232,6 +232,14 @@ static void handle_relocations(void *output, unsigned long output_len, }debug_putstr ("Performing relocations... " ); +#ifdef CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL +/* +* If CONFIG_RELOCATABLE_UNCOMPRESSED_KERNEL is enabled, the relocations is in +* '.data.reloc' section of kernel binary. +*/ + output_len = VO___relocation_end - VO__text; +#endif +/* * Process relocations: 32 bit relocations first then 64 bit after. * Two sets of binary relocations are added to the end of the kernel
Signature · 1 line
2.31.1
H. Peter Anvin
reviewerQuoted 88 lines · Hou Wenlong
Hi all,
This RFC patch series introduces relocatable uncompressed kernel image, which is allowed to perform kerenl image virtual address randomization in 64-bit booting entry instead of decompression phase.
- Background
Currently, kernel image virtual address randomization is only performed during the decompression phase. However, in certain scenarios, such as secure container environments (e.g., Kata Containers), to speed up the boot process, the system may boot directly from an uncompressed kernel image. In such cases, virtual address randomization cannot be executed. Although the security enhancement provided by KASLR is limited, there is still a potential demand to allow uncompressed kernel images to perform virtual address randomization (for example, future support for x86 PIE).
- Approaches
Currently, the x86 kernel uses static compilation, but it retains relocation information through the '--emit-relocs' option, which is then simplified into a relocation table using 'relocs' tool. To enable virtual address randomization for uncompressed kernel images, relocation information is required, and there are several possible approaches:
1) Who will perform the randomization:
VMM: The VMM reads vmlinux.relocs after loading vmlinux to perform randomization. This would require additional modifications to the VMM, and vmlinux.relocs needs to be packaged when shipping.
Kernel: The kernel performs randomization itself at the kernel entry point, requiring no modifications to the VMM.
2) relocation information format:
vmlinux.relocs: It only contains the necessary relocation entries and is simplified, making it small enough. However, it is a format defined within the kernel that was previously used only internally and is not part of the ABI.
rela.* sections: It is the standard ELF ABI, but
it contains RIP-relative relocation entries, which are more common in kernel, causing the kernel image to be larger.- Implementation
The final implementation of this plan extends the 'relocs' tool to allow the insertion of relocation information into a reserved section of the kernel (referencing the MIPS implementation). This enables the reading of that information and subsequent execution of relocations when booting directly from an uncompressed kernel. Currently, this implementation is only available for 64-bit and has been tested with both PVH entry booting and standard 64-bit Linux entry. And the default reserve size is 1MB for now, which is enough for defconfig.
- TODO
Clean up the decompression KASLR code to allow it to be shared with the booting phase.
Thanks!
Hou Wenlong (5):
x86/relocs: Cleanup cmdline options
x86/relocs: Insert relocations into input file
x86: Allow to build relocatable uncompressed kernel binary x86/boot: Perform virtual address relocation in kernel entry x86/boot: Use '.data.relocs' section for performing relocations during
decompressionarch/x86/Kconfig | 20 ++++++ arch/x86/Makefile.postlink | 33 +++++++++ arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/misc.c | 8 +++ arch/x86/boot/startup/Makefile | 1 + arch/x86/boot/startup/kaslr.c | 116 ++++++++++++++++++++++++++++++ arch/x86/include/asm/setup.h | 1 + arch/x86/kernel/head_64.S | 7 ++ arch/x86/kernel/vmlinux.lds.S | 20 ++++++ arch/x86/lib/cmdline.c | 6 ++ arch/x86/lib/kaslr.c | 5 ++ arch/x86/platform/pvh/head.S | 15 +++- arch/x86/tools/relocs.c | 64 ++++++++++++++--- arch/x86/tools/relocs.h | 15 ++-- arch/x86/tools/relocs_common.c | 24 ++++--- 15 files changed, 309 insertions(+), 32 deletions(-)
Quoted 6 lines
create mode 100644
arch/x86/Makefile.postlink
create mode 100644arch/x86/boot/startup/kaslr.c--
2.31.1
Hi!
At a very quick glance this seems like a very reasonable thing to me, but since the intent is reduced boot latency (a very worthwhile goal!) do you perhaps have any measurements to show how much improvement we are talking about? That would be really useful.
Thanks!
Quoted 92 lines · H. Peter Anvin
Quoted 91 lines · Hou Wenlong
Hi all,
This RFC patch series introduces relocatable uncompressed kernel image, which is allowed to perform kerenl image virtual address randomization in 64-bit booting entry instead of decompression phase.
- Background
Currently, kernel image virtual address randomization is only performed during the decompression phase. However, in certain scenarios, such as secure container environments (e.g., Kata Containers), to speed up the boot process, the system may boot directly from an uncompressed kernel image. In such cases, virtual address randomization cannot be executed. Although the security enhancement provided by KASLR is limited, there is still a potential demand to allow uncompressed kernel images to perform virtual address randomization (for example, future support for x86 PIE).
- Approaches
Currently, the x86 kernel uses static compilation, but it retains relocation information through the '--emit-relocs' option, which is then simplified into a relocation table using 'relocs' tool. To enable virtual address randomization for uncompressed kernel images, relocation information is required, and there are several possible approaches:
1) Who will perform the randomization:
VMM: The VMM reads vmlinux.relocs after loading vmlinux to perform randomization. This would require additional modifications to the VMM, and vmlinux.relocs needs to be packaged when shipping.
Kernel: The kernel performs randomization itself at the kernel entry point, requiring no modifications to the VMM.
2) relocation information format:
vmlinux.relocs: It only contains the necessary relocation entries and is simplified, making it small enough. However, it is a format defined within the kernel that was previously used only internally and is not part of the ABI.
rela.* sections: It is the standard ELF ABI, but
it contains RIP-relative relocation entries, which are more common in kernel, causing the kernel image to be larger.- Implementation
The final implementation of this plan extends the 'relocs' tool to allow the insertion of relocation information into a reserved section of the kernel (referencing the MIPS implementation). This enables the reading of that information and subsequent execution of relocations when booting directly from an uncompressed kernel. Currently, this implementation is only available for 64-bit and has been tested with both PVH entry booting and standard 64-bit Linux entry. And the default reserve size is 1MB for now, which is enough for defconfig.
- TODO
Clean up the decompression KASLR code to allow it to be shared with the booting phase.
Thanks!
Hou Wenlong (5):
x86/relocs: Cleanup cmdline options
x86/relocs: Insert relocations into input file
x86: Allow to build relocatable uncompressed kernel binary x86/boot: Perform virtual address relocation in kernel entry x86/boot: Use '.data.relocs' section for performing relocations during
decompressionarch/x86/Kconfig | 20 ++++++ arch/x86/Makefile.postlink | 33 +++++++++ arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/misc.c | 8 +++ arch/x86/boot/startup/Makefile | 1 + arch/x86/boot/startup/kaslr.c | 116 ++++++++++++++++++++++++++++++ arch/x86/include/asm/setup.h | 1 + arch/x86/kernel/head_64.S | 7 ++ arch/x86/kernel/vmlinux.lds.S | 20 ++++++ arch/x86/lib/cmdline.c | 6 ++ arch/x86/lib/kaslr.c | 5 ++ arch/x86/platform/pvh/head.S | 15 +++- arch/x86/tools/relocs.c | 64 ++++++++++++++--- arch/x86/tools/relocs.h | 15 ++-- arch/x86/tools/relocs_common.c | 24 ++++--- 15 files changed, 309 insertions(+), 32 deletions(-)create mode 100644
arch/x86/Makefile.postlink
create mode 100644arch/x86/boot/startup/kaslr.c
Quoted 8 lines
Quoted 3 lines
--
2.31.1Hi!
At a very quick glance this seems like a very reasonable thing to me, but since the intent is reduced boot latency (a very worthwhile goal!) do you perhaps have any measurements to show how much improvement we are talking about? That would be really useful.
Hi!
Uh, sorry that it may not meet your needs. In fact, it will slow down when booting directly from an uncompressed kernel. The improvement described in the patchset compares booting directly from vmlinux versus booting from bzImage when we want to enable KASLR for guests in MicroVM scenarios. There is a similar idea in [0], where KASLR randomization is implemented on the VMM side. Now we want to implement it directly in the guest kernel to reduce modifications to the VMMs. There are some measurements in [0]; however, the comparison is between vmlinux and bzImage.
In my test environment, compared to the original direct kernel booting, it would add 2ms for my test configuration [1] based on the Kata Containers repository due to the self-relocation phase. Booting from bzImage does not affect the boot time, as it simply inserts 'vmlinux.relocs' into vmlinux, resulting in no change to the total size. The decompression time should also not be affected; I didn't notice any difference when measuring the decompression().
[0]: https://dl.acm.org/doi/epdf/10.1145/3492321.3519578
[1]: https://raw.githubusercontent.com/virt-pvm/misc/refs/heads/main/pvm-guest-6.12.33.config
Thanks!
Quoted 1 line
Thanks!
H. Peter Anvin
reviewerQuoted 122 lines · Hou Wenlong
Quoted 92 lines · H. Peter Anvin
Quoted 91 lines · Hou Wenlong
Hi all,
This RFC patch series introduces relocatable uncompressed kernel image, which is allowed to perform kerenl image virtual address randomization in 64-bit booting entry instead of decompression phase.
- Background
Currently, kernel image virtual address randomization is only performed during the decompression phase. However, in certain scenarios, such as secure container environments (e.g., Kata Containers), to speed up the boot process, the system may boot directly from an uncompressed kernel image. In such cases, virtual address randomization cannot be executed. Although the security enhancement provided by KASLR is limited, there is still a potential demand to allow uncompressed kernel images to perform virtual address randomization (for example, future support for x86 PIE).
- Approaches
Currently, the x86 kernel uses static compilation, but it retains relocation information through the '--emit-relocs' option, which is then simplified into a relocation table using 'relocs' tool. To enable virtual address randomization for uncompressed kernel images, relocation information is required, and there are several possible approaches:
1) Who will perform the randomization:
VMM: The VMM reads vmlinux.relocs after loading vmlinux to perform randomization. This would require additional modifications to the VMM, and vmlinux.relocs needs to be packaged when shipping.
Kernel: The kernel performs randomization itself at the kernel entry point, requiring no modifications to the VMM.
2) relocation information format:
vmlinux.relocs: It only contains the necessary relocation entries and is simplified, making it small enough. However, it is a format defined within the kernel that was previously used only internally and is not part of the ABI.
rela.* sections: It is the standard ELF ABI, but
it contains RIP-relative relocation entries, which are more common in kernel, causing the kernel image to be larger.- Implementation
The final implementation of this plan extends the 'relocs' tool to allow the insertion of relocation information into a reserved section of the kernel (referencing the MIPS implementation). This enables the reading of that information and subsequent execution of relocations when booting directly from an uncompressed kernel. Currently, this implementation is only available for 64-bit and has been tested with both PVH entry booting and standard 64-bit Linux entry. And the default reserve size is 1MB for now, which is enough for defconfig.
- TODO
Clean up the decompression KASLR code to allow it to be shared with the booting phase.
Thanks!
Hou Wenlong (5):
x86/relocs: Cleanup cmdline options
x86/relocs: Insert relocations into input file
x86: Allow to build relocatable uncompressed kernel binary x86/boot: Perform virtual address relocation in kernel entry x86/boot: Use '.data.relocs' section for performing relocations during
decompressionarch/x86/Kconfig | 20 ++++++ arch/x86/Makefile.postlink | 33 +++++++++ arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/misc.c | 8 +++ arch/x86/boot/startup/Makefile | 1 + arch/x86/boot/startup/kaslr.c | 116 ++++++++++++++++++++++++++++++ arch/x86/include/asm/setup.h | 1 + arch/x86/kernel/head_64.S | 7 ++ arch/x86/kernel/vmlinux.lds.S | 20 ++++++ arch/x86/lib/cmdline.c | 6 ++ arch/x86/lib/kaslr.c | 5 ++ arch/x86/platform/pvh/head.S | 15 +++- arch/x86/tools/relocs.c | 64 ++++++++++++++--- arch/x86/tools/relocs.h | 15 ++-- arch/x86/tools/relocs_common.c | 24 ++++--- 15 files changed, 309 insertions(+), 32 deletions(-)create mode 100644
arch/x86/Makefile.postlink
create mode 100644arch/x86/boot/startup/kaslr.cQuoted 8 lines
Quoted 3 lines
--
2.31.1Hi!
At a very quick glance this seems like a very reasonable thing to me, but since the intent is reduced boot latency (a very worthwhile goal!) do you perhaps have any measurements to show how much improvement we are talking about? That would be really useful.
Hi!
Uh, sorry that it may not meet your needs. In fact, it will slow down when booting directly from an uncompressed kernel. The improvement described in the patchset compares booting directly from vmlinux versus booting from bzImage when we want to enable KASLR for guests in MicroVM scenarios. There is a similar idea in [0], where KASLR randomization is implemented on the VMM side. Now we want to implement it directly in the guest kernel to reduce modifications to the VMMs. There are some measurements in [0]; however, the comparison is between vmlinux and bzImage.
In my test environment, compared to the original direct kernel booting, it would add 2ms for my test configuration [1] based on the Kata Containers repository due to the self-relocation phase. Booting from bzImage does not affect the boot time, as it simply inserts 'vmlinux.relocs' into vmlinux, resulting in no change to the total size. The decompression time should also not be affected; I didn't notice any difference when measuring the
decompression().
Quoted 6 lines
[0]: https://dl.acm.org/doi/epdf/10.1145/3492321.3519578
[1]: https://raw.githubusercontent.com/virt-pvm/misc/refs/heads/main/pvm-guest-6.12.33.configThanks!
Quoted 1 line
Thanks!
Didn't you say that that was the reason for this? I'm confused now.
Quoted 123 lines · H. Peter Anvin
Quoted 122 lines · Hou Wenlong
Quoted 92 lines · H. Peter Anvin
Quoted 91 lines · Hou Wenlong
Hi all,
This RFC patch series introduces relocatable uncompressed kernel image, which is allowed to perform kerenl image virtual address randomization in 64-bit booting entry instead of decompression phase.
- Background
Currently, kernel image virtual address randomization is only performed during the decompression phase. However, in certain scenarios, such as secure container environments (e.g., Kata Containers), to speed up the boot process, the system may boot directly from an uncompressed kernel image. In such cases, virtual address randomization cannot be executed. Although the security enhancement provided by KASLR is limited, there is still a potential demand to allow uncompressed kernel images to perform virtual address randomization (for example, future support for x86 PIE).
- Approaches
Currently, the x86 kernel uses static compilation, but it retains relocation information through the '--emit-relocs' option, which is then simplified into a relocation table using 'relocs' tool. To enable virtual address randomization for uncompressed kernel images, relocation information is required, and there are several possible approaches:
1) Who will perform the randomization:
VMM: The VMM reads vmlinux.relocs after loading vmlinux to perform randomization. This would require additional modifications to the VMM, and vmlinux.relocs needs to be packaged when shipping.
Kernel: The kernel performs randomization itself at the kernel entry point, requiring no modifications to the VMM.
2) relocation information format:
vmlinux.relocs: It only contains the necessary relocation entries and is simplified, making it small enough. However, it is a format defined within the kernel that was previously used only internally and is not part of the ABI.
rela.* sections: It is the standard ELF ABI, but
it contains RIP-relative relocation entries, which are more common in kernel, causing the kernel image to be larger.- Implementation
The final implementation of this plan extends the 'relocs' tool to allow the insertion of relocation information into a reserved section of the kernel (referencing the MIPS implementation). This enables the reading of that information and subsequent execution of relocations when booting directly from an uncompressed kernel. Currently, this implementation is only available for 64-bit and has been tested with both PVH entry booting and standard 64-bit Linux entry. And the default reserve size is 1MB for now, which is enough for defconfig.
- TODO
Clean up the decompression KASLR code to allow it to be shared with the booting phase.
Thanks!
Hou Wenlong (5):
x86/relocs: Cleanup cmdline options
x86/relocs: Insert relocations into input file
x86: Allow to build relocatable uncompressed kernel binary x86/boot: Perform virtual address relocation in kernel entry x86/boot: Use '.data.relocs' section for performing relocations during
decompressionarch/x86/Kconfig | 20 ++++++ arch/x86/Makefile.postlink | 33 +++++++++ arch/x86/boot/compressed/Makefile | 6 +- arch/x86/boot/compressed/misc.c | 8 +++ arch/x86/boot/startup/Makefile | 1 + arch/x86/boot/startup/kaslr.c | 116 ++++++++++++++++++++++++++++++ arch/x86/include/asm/setup.h | 1 + arch/x86/kernel/head_64.S | 7 ++ arch/x86/kernel/vmlinux.lds.S | 20 ++++++ arch/x86/lib/cmdline.c | 6 ++ arch/x86/lib/kaslr.c | 5 ++ arch/x86/platform/pvh/head.S | 15 +++- arch/x86/tools/relocs.c | 64 ++++++++++++++--- arch/x86/tools/relocs.h | 15 ++-- arch/x86/tools/relocs_common.c | 24 ++++--- 15 files changed, 309 insertions(+), 32 deletions(-)create mode 100644
arch/x86/Makefile.postlink
create mode 100644arch/x86/boot/startup/kaslr.cQuoted 8 lines
Quoted 3 lines
--
2.31.1Hi!
At a very quick glance this seems like a very reasonable thing to me, but since the intent is reduced boot latency (a very worthwhile goal!) do you perhaps have any measurements to show how much improvement we are talking about? That would be really useful.
Hi!
Uh, sorry that it may not meet your needs. In fact, it will slow down when booting directly from an uncompressed kernel. The improvement described in the patchset compares booting directly from vmlinux versus booting from bzImage when we want to enable KASLR for guests in MicroVM scenarios. There is a similar idea in [0], where KASLR randomization is implemented on the VMM side. Now we want to implement it directly in the guest kernel to reduce modifications to the VMMs. There are some measurements in [0]; however, the comparison is between vmlinux and bzImage.
In my test environment, compared to the original direct kernel booting, it would add 2ms for my test configuration [1] based on the Kata Containers repository due to the self-relocation phase. Booting from bzImage does not affect the boot time, as it simply inserts 'vmlinux.relocs' into vmlinux, resulting in no change to the total size. The decompression time should also not be affected; I didn't notice any difference when measuring the
decompression().
Quoted 8 lines
Quoted 6 lines
[0]: https://dl.acm.org/doi/epdf/10.1145/3492321.3519578
[1]: https://raw.githubusercontent.com/virt-pvm/misc/refs/heads/main/pvm-guest-6.12.33.configThanks!
Quoted 1 line
Thanks!
Didn't you say that that was the reason for this? I'm confused now.
Maybe my expression is not clear. :(
Let me reorganize my thoughts. If you want to enable KASLR for the guest, this patch makes booting faster, as the guest is now booting from vmlinux instead of bzImage. However, if you don’t need KASLR for the guest, you can continue booting from vmlinux, so the newly added relocation process may introduce a slight overhead (due to command line parsing), which is what I meant when I said it slows down the booting from vmlinux. I'm not sure if you need KASLR in your case.
Thanks!