Skip to contentSkip to conversation

x86: Refactor and consolidate startup code

PATCH Applied Ard Biesheuvel 6 participants 26 messages Apr 10, 2025 – Apr 12, 2025

Subsystems (3)

v4 of 5 Applied Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover x86: Refactor and consolidate startup code

From: Ard Biesheuvel <[email protected]>

!! NOTE: patches #7 - #10 depend on [0] !!

Reorganize C code that is used during early boot, either in the decompressor/EFI stub or the kernel proper, but before the kernel virtual mapping is up.

v4:
- drop patches that were queued up
- fix address space error in patch #1
- add patches for SEV-SNP boot code - these cannot be applied yet, but are included for completeness

v3:
- keep rip_rel_ptr() around in PIC code - sadly, it is still needed in some cases
- remove RIP_REL_REF() uses in separate patches
- keep __head annotations for now, they will all be removed later
- disable objtool validation for library objects (i.e., pieces that are not linked into vmlinux)

I will follow up with a series that gets rid of .head.text altogether, as it will no longer be needed at all once the startup code is checked for absolute relocations.

Cc Tom Lendacky <[email protected]>
Cc Dionna Amalie Glaze <[email protected]>
Cc Kevin Loughlin <[email protected]>

[0] https://lore.kernel.org/all/[email protected]/T/#u

Ard Biesheuvel (11):
x86/asm: Make rip_rel_ptr() usable from fPIC code x86/boot: Move the early GDT/IDT setup code into startup/ x86/boot: Move early kernel mapping code into startup/ x86/boot: Drop RIP_REL_REF() uses from early mapping code x86/boot: Move early SME init code into startup/ x86/boot: Drop RIP_REL_REF() uses from SME startup code x86/sev: Prepare for splitting off early SEV code x86/sev: Split off startup code from core code x86/boot: Move SEV startup code into startup/ x86/boot: Drop RIP_REL_REF() uses from early SEV code x86/asm: Retire RIP_REL_REF()

 arch/x86/boot/compressed/Makefile                          |    2 +-
 arch/x86/boot/compressed/sev.c                             |   17 +-
 arch/x86/boot/startup/Makefile                             |   16 +
 arch/x86/boot/startup/gdt_idt.c                            |   84 +
 arch/x86/boot/startup/map_kernel.c                         |  225 +++
 arch/x86/{coco/sev/shared.c => boot/startup/sev-shared.c}  |  375 +----
 arch/x86/boot/startup/sev-startup.c                        | 1395 ++++++++++++++++
 arch/x86/{mm/mem_encrypt_identity.c => boot/startup/sme.c} |   19 +-
 arch/x86/coco/sev/Makefile                                 |   19 -
 arch/x86/coco/sev/core.c                                   | 1726 ++++----------------
 arch/x86/include/asm/asm.h                                 |    5 -
 arch/x86/include/asm/coco.h                                |    2 +-
 arch/x86/include/asm/mem_encrypt.h                         |    2 +-
 arch/x86/include/asm/sev-internal.h                        |  112 ++
 arch/x86/include/asm/sev.h                                 |   37 +
 arch/x86/kernel/head64.c                                   |  285 +---
 arch/x86/mm/Makefile                                       |    6 -
 17 files changed, 2208 insertions(+), 2119 deletions(-)
 create mode 100644 arch/x86/boot/startup/gdt_idt.c
 create mode 100644 arch/x86/boot/startup/map_kernel.c
 rename arch/x86/{coco/sev/shared.c => boot/startup/sev-shared.c} (78%)
 create mode 100644 arch/x86/boot/startup/sev-startup.c
 rename arch/x86/{mm/mem_encrypt_identity.c => boot/startup/sme.c} (97%)
 create mode 100644 arch/x86/include/asm/sev-internal.h
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
#1 lore raw

PATCH v4 1/11 x86/asm: Make rip_rel_ptr() usable from fPIC code

From: Ard Biesheuvel <[email protected]>

RIP_REL_REF() is used in non-PIC C code that is called very early, before the kernel virtual mapping is up, which is the mapping that the linker expects. It is currently used in two different ways:
- to refer to the value of a global variable, including as an lvalue in assignments;
- to take the address of a global variable via the mapping that the code currently executes at.

The former case is only needed in non-PIC code, as PIC code will never use absolute symbol references when the address of the symbol is not being used. But taking the address of a variable in PIC code may still require extra care, as a stack allocated struct assignment may be emitted as a memcpy() from a statically allocated copy in .rodata.

For instance, this

  void startup_64_setup_gdt_idt(void)
  {
        struct desc_ptr startup_gdt_descr = {
                .address = (__force unsigned long)gdt_page.gdt,
                .size    = GDT_SIZE - 1,
        };

may result in an absolute symbol reference in PIC code, even though the struct is allocated on the stack and populated at runtime.

To address this case, make rip_rel_ptr() accessible in PIC code, and update any existing uses where the address of a global variable is taken using RIP_REL_REF.

Once all code of this nature has been moved into arch/x86/boot/startup and built with -fPIC, RIP_REL_REF() can be retired, and only rip_rel_ptr() will remain.

---
 arch/x86/coco/sev/core.c           |  2 +-
 arch/x86/coco/sev/shared.c         |  4 ++--
 arch/x86/include/asm/asm.h         |  2 +-
 arch/x86/kernel/head64.c           | 24 ++++++++++----------
 arch/x86/mm/mem_encrypt_identity.c |  6 ++---
 5 files changed, 19 insertions(+), 19 deletions(-)
arch/x86/coco/sev/core.c +1 -1
index b0c1a7a57497..832f7a7b10b2 100644
@@ -2400,7 +2400,7 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info)
 	 * kernel was loaded (physbase), so the get the CA address using
 	 * RIP-relative addressing.
 	 */
-	pa = (u64)&RIP_REL_REF(boot_svsm_ca_page);
+	pa = (u64)rip_rel_ptr(&boot_svsm_ca_page);
 
 	/*
 	 * Switch over to the boot SVSM CA while the current CA is still
arch/x86/coco/sev/shared.c +2 -2
index 2e4122f8aa6b..04982d356803 100644
@@ -475,7 +475,7 @@ static int sev_cpuid_hv(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid
  */
 static const struct snp_cpuid_table *snp_cpuid_get_table(void)
 {
-	return &RIP_REL_REF(cpuid_table_copy);
+	return rip_rel_ptr(&cpuid_table_copy);
 }
 
 /*
@@ -1681,7 +1681,7 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info)
 	 * routine is running identity mapped when called, both by the decompressor
 	 * code and the early kernel code.
 	 */
-	if (!rmpadjust((unsigned long)&RIP_REL_REF(boot_ghcb_page), RMP_PG_SIZE_4K, 1))
+	if (!rmpadjust((unsigned long)rip_rel_ptr(&boot_ghcb_page), RMP_PG_SIZE_4K, 1))
 		return false;
 
 	/*
arch/x86/include/asm/asm.h +1 -1
index cc2881576c2c..a9f07799e337 100644
@@ -114,13 +114,13 @@
 #endif
 
 #ifndef __ASSEMBLER__
-#ifndef __pic__
 static __always_inline __pure void *rip_rel_ptr(void *p)
 {
 	asm("leaq %c1(%%rip), %0" : "=r"(p) : "i"(p));
 
 	return p;
 }
+#ifndef __pic__
 #define RIP_REL_REF(var)	(*(typeof(&(var)))rip_rel_ptr(&(var)))
 #else
 #define RIP_REL_REF(var)	(var)
arch/x86/kernel/head64.c +12 -12
index fa9b6339975f..954d093f187b 100644
@@ -106,8 +106,8 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,
 	 * attribute.
 	 */
 	if (sme_get_me_mask()) {
-		paddr = (unsigned long)&RIP_REL_REF(__start_bss_decrypted);
-		paddr_end = (unsigned long)&RIP_REL_REF(__end_bss_decrypted);
+		paddr = (unsigned long)rip_rel_ptr(__start_bss_decrypted);
+		paddr_end = (unsigned long)rip_rel_ptr(__end_bss_decrypted);
 
 		for (; paddr < paddr_end; paddr += PMD_SIZE) {
 			/*
@@ -144,8 +144,8 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,
 unsigned long __head __startup_64(unsigned long p2v_offset,
 				  struct boot_params *bp)
 {
-	pmd_t (*early_pgts)[PTRS_PER_PMD] = RIP_REL_REF(early_dynamic_pgts);
-	unsigned long physaddr = (unsigned long)&RIP_REL_REF(_text);
+	pmd_t (*early_pgts)[PTRS_PER_PMD] = rip_rel_ptr(early_dynamic_pgts);
+	unsigned long physaddr = (unsigned long)rip_rel_ptr(_text);
 	unsigned long va_text, va_end;
 	unsigned long pgtable_flags;
 	unsigned long load_delta;
@@ -174,18 +174,18 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 		for (;;);
 
 	va_text = physaddr - p2v_offset;
-	va_end  = (unsigned long)&RIP_REL_REF(_end) - p2v_offset;
+	va_end  = (unsigned long)rip_rel_ptr(_end) - p2v_offset;
 
 	/* Include the SME encryption mask in the fixup value */
 	load_delta += sme_get_me_mask();
 
 	/* Fixup the physical addresses in the page table */
 
-	pgd = &RIP_REL_REF(early_top_pgt)->pgd;
+	pgd = rip_rel_ptr(early_top_pgt);
 	pgd[pgd_index(__START_KERNEL_map)] += load_delta;
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) {
-		p4d = (p4dval_t *)&RIP_REL_REF(level4_kernel_pgt);
+		p4d = (p4dval_t *)rip_rel_ptr(level4_kernel_pgt);
 		p4d[MAX_PTRS_PER_P4D - 1] += load_delta;
 
 		pgd[pgd_index(__START_KERNEL_map)] = (pgdval_t)p4d | _PAGE_TABLE;
@@ -258,7 +258,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 	 * error, causing the BIOS to halt the system.
 	 */
 
-	pmd = &RIP_REL_REF(level2_kernel_pgt)->pmd;
+	pmd = rip_rel_ptr(level2_kernel_pgt);
 
 	/* invalidate pages before the kernel image */
 	for (i = 0; i < pmd_index(va_text); i++)
@@ -531,7 +531,7 @@ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data;
 static void __head startup_64_load_idt(void *vc_handler)
 {
 	struct desc_ptr desc = {
-		.address = (unsigned long)&RIP_REL_REF(bringup_idt_table),
+		.address = (unsigned long)rip_rel_ptr(bringup_idt_table),
 		.size    = sizeof(bringup_idt_table) - 1,
 	};
 	struct idt_data data;
@@ -565,11 +565,11 @@ void early_setup_idt(void)
  */
 void __head startup_64_setup_gdt_idt(void)
 {
-	struct desc_struct *gdt = (void *)(__force unsigned long)gdt_page.gdt;
+	struct gdt_page *gp = rip_rel_ptr((void *)(__force unsigned long)&gdt_page);
 	void *handler = NULL;
 
 	struct desc_ptr startup_gdt_descr = {
-		.address = (unsigned long)&RIP_REL_REF(*gdt),
+		.address = (unsigned long)gp->gdt,
 		.size    = GDT_SIZE - 1,
 	};
 
@@ -582,7 +582,7 @@ void __head startup_64_setup_gdt_idt(void)
 		     "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory");
 
 	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))
-		handler = &RIP_REL_REF(vc_no_ghcb);
+		handler = rip_rel_ptr(vc_no_ghcb);
 
 	startup_64_load_idt(handler);
 }
arch/x86/mm/mem_encrypt_identity.c +3 -3
index 5eecdd92da10..e7fb3779b35f 100644
@@ -318,8 +318,8 @@ void __head sme_encrypt_kernel(struct boot_params *bp)
 	 *     memory from being cached.
 	 */
 
-	kernel_start = (unsigned long)RIP_REL_REF(_text);
-	kernel_end = ALIGN((unsigned long)RIP_REL_REF(_end), PMD_SIZE);
+	kernel_start = (unsigned long)rip_rel_ptr(_text);
+	kernel_end = ALIGN((unsigned long)rip_rel_ptr(_end), PMD_SIZE);
 	kernel_len = kernel_end - kernel_start;
 
 	initrd_start = 0;
@@ -345,7 +345,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp)
 	 *   pagetable structures for the encryption of the kernel
 	 *   pagetable structures for workarea (in case not currently mapped)
 	 */
-	execute_start = workarea_start = (unsigned long)RIP_REL_REF(sme_workarea);
+	execute_start = workarea_start = (unsigned long)rip_rel_ptr(sme_workarea);
 	execute_end = execute_start + (PAGE_SIZE * 2) + PMD_SIZE;
 	execute_len = execute_end - execute_start;
 
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#2 lore raw

[tip: x86/boot] x86/asm: Make rip_rel_ptr() usable from fPIC code

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     bcceba3c72c0cf06dfbae77f5aec70fb6187e8df
Gitweb:        https://git.kernel.org/tip/bcceba3c72c0cf06dfbae77f5aec70fb6187e8df
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:19 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:04 +02:00

x86/asm: Make rip_rel_ptr() usable from fPIC code

RIP_REL_REF() is used in non-PIC C code that is called very early, before the kernel virtual mapping is up, which is the mapping that the linker expects. It is currently used in two different ways:

- to refer to the value of a global variable, including as an lvalue in assignments;

- to take the address of a global variable via the mapping that the code currently executes at.

The former case is only needed in non-PIC code, as PIC code will never use absolute symbol references when the address of the symbol is not being used. But taking the address of a variable in PIC code may still require extra care, as a stack allocated struct assignment may be emitted as a memcpy() from a statically allocated copy in .rodata.

For instance, this

  void startup_64_setup_gdt_idt(void)
  {
        struct desc_ptr startup_gdt_descr = {
                .address = (__force unsigned long)gdt_page.gdt,
                .size    = GDT_SIZE - 1,
        };

may result in an absolute symbol reference in PIC code, even though the struct is allocated on the stack and populated at runtime.

To address this case, make rip_rel_ptr() accessible in PIC code, and update any existing uses where the address of a global variable is taken using RIP_REL_REF.

Once all code of this nature has been moved into arch/x86/boot/startup and built with -fPIC, RIP_REL_REF() can be retired, and only rip_rel_ptr() will remain.

---
 arch/x86/coco/sev/core.c           |  2 +-
 arch/x86/coco/sev/shared.c         |  4 ++--
 arch/x86/include/asm/asm.h         |  2 +-
 arch/x86/kernel/head64.c           | 24 ++++++++++++------------
 arch/x86/mm/mem_encrypt_identity.c |  6 +++---
 5 files changed, 19 insertions(+), 19 deletions(-)
arch/x86/coco/sev/core.c +1 -1
index b0c1a7a..832f7a7 100644
@@ -2400,7 +2400,7 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info)
 	 * kernel was loaded (physbase), so the get the CA address using
 	 * RIP-relative addressing.
 	 */
-	pa = (u64)&RIP_REL_REF(boot_svsm_ca_page);
+	pa = (u64)rip_rel_ptr(&boot_svsm_ca_page);
 
 	/*
 	 * Switch over to the boot SVSM CA while the current CA is still
arch/x86/coco/sev/shared.c +2 -2
index 2e4122f..04982d3 100644
@@ -475,7 +475,7 @@ static int sev_cpuid_hv(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid
  */
 static const struct snp_cpuid_table *snp_cpuid_get_table(void)
 {
-	return &RIP_REL_REF(cpuid_table_copy);
+	return rip_rel_ptr(&cpuid_table_copy);
 }
 
 /*
@@ -1681,7 +1681,7 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info)
 	 * routine is running identity mapped when called, both by the decompressor
 	 * code and the early kernel code.
 	 */
-	if (!rmpadjust((unsigned long)&RIP_REL_REF(boot_ghcb_page), RMP_PG_SIZE_4K, 1))
+	if (!rmpadjust((unsigned long)rip_rel_ptr(&boot_ghcb_page), RMP_PG_SIZE_4K, 1))
 		return false;
 
 	/*
arch/x86/include/asm/asm.h +1 -1
index cc28815..a9f0779 100644
@@ -114,13 +114,13 @@
 #endif
 
 #ifndef __ASSEMBLER__
-#ifndef __pic__
 static __always_inline __pure void *rip_rel_ptr(void *p)
 {
 	asm("leaq %c1(%%rip), %0" : "=r"(p) : "i"(p));
 
 	return p;
 }
+#ifndef __pic__
 #define RIP_REL_REF(var)	(*(typeof(&(var)))rip_rel_ptr(&(var)))
 #else
 #define RIP_REL_REF(var)	(var)
arch/x86/kernel/head64.c +12 -12
index fa9b633..954d093 100644
@@ -106,8 +106,8 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,
 	 * attribute.
 	 */
 	if (sme_get_me_mask()) {
-		paddr = (unsigned long)&RIP_REL_REF(__start_bss_decrypted);
-		paddr_end = (unsigned long)&RIP_REL_REF(__end_bss_decrypted);
+		paddr = (unsigned long)rip_rel_ptr(__start_bss_decrypted);
+		paddr_end = (unsigned long)rip_rel_ptr(__end_bss_decrypted);
 
 		for (; paddr < paddr_end; paddr += PMD_SIZE) {
 			/*
@@ -144,8 +144,8 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,
 unsigned long __head __startup_64(unsigned long p2v_offset,
 				  struct boot_params *bp)
 {
-	pmd_t (*early_pgts)[PTRS_PER_PMD] = RIP_REL_REF(early_dynamic_pgts);
-	unsigned long physaddr = (unsigned long)&RIP_REL_REF(_text);
+	pmd_t (*early_pgts)[PTRS_PER_PMD] = rip_rel_ptr(early_dynamic_pgts);
+	unsigned long physaddr = (unsigned long)rip_rel_ptr(_text);
 	unsigned long va_text, va_end;
 	unsigned long pgtable_flags;
 	unsigned long load_delta;
@@ -174,18 +174,18 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 		for (;;);
 
 	va_text = physaddr - p2v_offset;
-	va_end  = (unsigned long)&RIP_REL_REF(_end) - p2v_offset;
+	va_end  = (unsigned long)rip_rel_ptr(_end) - p2v_offset;
 
 	/* Include the SME encryption mask in the fixup value */
 	load_delta += sme_get_me_mask();
 
 	/* Fixup the physical addresses in the page table */
 
-	pgd = &RIP_REL_REF(early_top_pgt)->pgd;
+	pgd = rip_rel_ptr(early_top_pgt);
 	pgd[pgd_index(__START_KERNEL_map)] += load_delta;
 
 	if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) {
-		p4d = (p4dval_t *)&RIP_REL_REF(level4_kernel_pgt);
+		p4d = (p4dval_t *)rip_rel_ptr(level4_kernel_pgt);
 		p4d[MAX_PTRS_PER_P4D - 1] += load_delta;
 
 		pgd[pgd_index(__START_KERNEL_map)] = (pgdval_t)p4d | _PAGE_TABLE;
@@ -258,7 +258,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 	 * error, causing the BIOS to halt the system.
 	 */
 
-	pmd = &RIP_REL_REF(level2_kernel_pgt)->pmd;
+	pmd = rip_rel_ptr(level2_kernel_pgt);
 
 	/* invalidate pages before the kernel image */
 	for (i = 0; i < pmd_index(va_text); i++)
@@ -531,7 +531,7 @@ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data;
 static void __head startup_64_load_idt(void *vc_handler)
 {
 	struct desc_ptr desc = {
-		.address = (unsigned long)&RIP_REL_REF(bringup_idt_table),
+		.address = (unsigned long)rip_rel_ptr(bringup_idt_table),
 		.size    = sizeof(bringup_idt_table) - 1,
 	};
 	struct idt_data data;
@@ -565,11 +565,11 @@ void early_setup_idt(void)
  */
 void __head startup_64_setup_gdt_idt(void)
 {
-	struct desc_struct *gdt = (void *)(__force unsigned long)gdt_page.gdt;
+	struct gdt_page *gp = rip_rel_ptr((void *)(__force unsigned long)&gdt_page);
 	void *handler = NULL;
 
 	struct desc_ptr startup_gdt_descr = {
-		.address = (unsigned long)&RIP_REL_REF(*gdt),
+		.address = (unsigned long)gp->gdt,
 		.size    = GDT_SIZE - 1,
 	};
 
@@ -582,7 +582,7 @@ void __head startup_64_setup_gdt_idt(void)
 		     "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory");
 
 	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT))
-		handler = &RIP_REL_REF(vc_no_ghcb);
+		handler = rip_rel_ptr(vc_no_ghcb);
 
 	startup_64_load_idt(handler);
 }
arch/x86/mm/mem_encrypt_identity.c +3 -3
index 5eecdd9..e7fb377 100644
@@ -318,8 +318,8 @@ void __head sme_encrypt_kernel(struct boot_params *bp)
 	 *     memory from being cached.
 	 */
 
-	kernel_start = (unsigned long)RIP_REL_REF(_text);
-	kernel_end = ALIGN((unsigned long)RIP_REL_REF(_end), PMD_SIZE);
+	kernel_start = (unsigned long)rip_rel_ptr(_text);
+	kernel_end = ALIGN((unsigned long)rip_rel_ptr(_end), PMD_SIZE);
 	kernel_len = kernel_end - kernel_start;
 
 	initrd_start = 0;
@@ -345,7 +345,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp)
 	 *   pagetable structures for the encryption of the kernel
 	 *   pagetable structures for workarea (in case not currently mapped)
 	 */
-	execute_start = workarea_start = (unsigned long)RIP_REL_REF(sme_workarea);
+	execute_start = workarea_start = (unsigned long)rip_rel_ptr(sme_workarea);
 	execute_end = execute_start + (PAGE_SIZE * 2) + PMD_SIZE;
 	execute_len = execute_end - execute_start;
 
#3 lore raw

PATCH v4 2/11 x86/boot: Move the early GDT/IDT setup code into startup/

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Move the early GDT/IDT setup code that runs long before the kernel virtual mapping is up into arch/x86/boot/startup/, and build it in a way that ensures that the code tolerates being called from the 1:1 mapping of memory. The code itself is left unchanged by this patch.

Also tweak the sed symbol matching pattern in the decompressor to match on lower case 't' or 'b', as these will be emitted by Clang for symbols with hidden linkage.

---
 arch/x86/boot/compressed/Makefile |  2 +-
 arch/x86/boot/startup/Makefile    | 15 ++++
 arch/x86/boot/startup/gdt_idt.c   | 84 ++++++++++++++++++++
 arch/x86/kernel/head64.c          | 74 -----------------
 4 files changed, 100 insertions(+), 75 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
index 37b85ce9b2a3..0fcad7b7e007 100644
@@ -73,7 +73,7 @@ LDFLAGS_vmlinux += -T
 hostprogs	:= mkpiggy
 HOST_EXTRACFLAGS += -I$(srctree)/tools/include
 
-sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABCDGRSTVW] \(_text\|__start_rodata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p'
+sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p'
 
 quiet_cmd_voffset = VOFFSET $@
       cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@
arch/x86/boot/startup/Makefile +15 -0
index 8919a1cbcb5a..1beb5de30735 100644
@@ -1,6 +1,21 @@
 # SPDX-License-Identifier: GPL-2.0
 
 KBUILD_AFLAGS		+= -D__DISABLE_EXPORTS
+KBUILD_CFLAGS		+= -D__DISABLE_EXPORTS -mcmodel=small -fPIC \
+			   -Os -DDISABLE_BRANCH_PROFILING \
+			   $(DISABLE_STACKLEAK_PLUGIN) \
+			   -fno-stack-protector -D__NO_FORTIFY \
+			   -include $(srctree)/include/linux/hidden.h
+
+# disable ftrace hooks
+KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS))
+KASAN_SANITIZE	:= n
+KCSAN_SANITIZE	:= n
+KMSAN_SANITIZE	:= n
+UBSAN_SANITIZE	:= n
+KCOV_INSTRUMENT	:= n
+
+obj-$(CONFIG_X86_64)		+= gdt_idt.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/gdt_idt.c new file +84 -0

84 lines not on this page · raw message

arch/x86/kernel/head64.c +0 -74

77 lines not on this page · raw message

Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#4 lore raw

[tip: x86/boot] x86/boot: Move the early GDT/IDT setup code into startup/

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     4cecebf200efea0fc865b5656f6d12ead2eb5573
Gitweb:        https://git.kernel.org/tip/4cecebf200efea0fc865b5656f6d12ead2eb5573
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:20 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:04 +02:00

x86/boot: Move the early GDT/IDT setup code into startup/

Move the early GDT/IDT setup code that runs long before the kernel virtual mapping is up into arch/x86/boot/startup/, and build it in a way that ensures that the code tolerates being called from the 1:1 mapping of memory. The code itself is left unchanged by this patch.

Also tweak the sed symbol matching pattern in the decompressor to match on lower case 't' or 'b', as these will be emitted by Clang for symbols with hidden linkage.

---
 arch/x86/boot/compressed/Makefile |  2 +-
 arch/x86/boot/startup/Makefile    | 15 +++++-
 arch/x86/boot/startup/gdt_idt.c   | 84 ++++++++++++++++++++++++++++++-
 arch/x86/kernel/head64.c          | 74 +--------------------------
 4 files changed, 100 insertions(+), 75 deletions(-)
 create mode 100644 arch/x86/boot/startup/gdt_idt.c
arch/x86/boot/compressed/Makefile +1 -1
index 37b85ce..0fcad7b 100644
@@ -73,7 +73,7 @@ LDFLAGS_vmlinux += -T
 hostprogs	:= mkpiggy
 HOST_EXTRACFLAGS += -I$(srctree)/tools/include
 
-sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABCDGRSTVW] \(_text\|__start_rodata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p'
+sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p'
 
 quiet_cmd_voffset = VOFFSET $@
       cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@
arch/x86/boot/startup/Makefile +15 -0
index 8919a1c..1beb5de 100644
@@ -1,6 +1,21 @@
 # SPDX-License-Identifier: GPL-2.0
 
 KBUILD_AFLAGS		+= -D__DISABLE_EXPORTS
+KBUILD_CFLAGS		+= -D__DISABLE_EXPORTS -mcmodel=small -fPIC \
+			   -Os -DDISABLE_BRANCH_PROFILING \
+			   $(DISABLE_STACKLEAK_PLUGIN) \
+			   -fno-stack-protector -D__NO_FORTIFY \
+			   -include $(srctree)/include/linux/hidden.h
+
+# disable ftrace hooks
+KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS))
+KASAN_SANITIZE	:= n
+KCSAN_SANITIZE	:= n
+KMSAN_SANITIZE	:= n
+UBSAN_SANITIZE	:= n
+KCOV_INSTRUMENT	:= n
+
+obj-$(CONFIG_X86_64)		+= gdt_idt.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/gdt_idt.c new file +84 -0

84 lines not on this page · raw message

arch/x86/kernel/head64.c +0 -74

77 lines not on this page · raw message

#5 lore raw

PATCH v4 3/11 x86/boot: Move early kernel mapping code into startup/

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The startup code that constructs the kernel virtual mapping runs from the 1:1 mapping of memory itself, and therefore, cannot use absolute symbol references. Before making changes in subsequent patches, move this code into a separate source file under arch/x86/boot/startup/ where all such code will be kept from now on.

---
 arch/x86/boot/startup/Makefile     |   2 +-
 arch/x86/boot/startup/map_kernel.c | 224 ++++++++++++++++++++
 arch/x86/kernel/head64.c           | 211 +-----------------
 3 files changed, 226 insertions(+), 211 deletions(-)
arch/x86/boot/startup/Makefile +1 -1
index 1beb5de30735..10319aee666b 100644
@@ -15,7 +15,7 @@ KMSAN_SANITIZE	:= n
 UBSAN_SANITIZE	:= n
 KCOV_INSTRUMENT	:= n
 
-obj-$(CONFIG_X86_64)		+= gdt_idt.o
+obj-$(CONFIG_X86_64)		+= gdt_idt.o map_kernel.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/map_kernel.c new file +224 -0

224 lines not on this page · raw message

arch/x86/kernel/head64.c +1 -210

223 lines not on this page · raw message

Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#6 lore raw

[tip: x86/boot] x86/boot: Move early kernel mapping code into startup/

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     dbe0ad775cbbd614583409d96abbc8ea7edb5eb4
Gitweb:        https://git.kernel.org/tip/dbe0ad775cbbd614583409d96abbc8ea7edb5eb4
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:21 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:05 +02:00

x86/boot: Move early kernel mapping code into startup/

The startup code that constructs the kernel virtual mapping runs from the 1:1 mapping of memory itself, and therefore, cannot use absolute symbol references. Before making changes in subsequent patches, move this code into a separate source file under arch/x86/boot/startup/ where all such code will be kept from now on.

---
 arch/x86/boot/startup/Makefile     |   2 +-
 arch/x86/boot/startup/map_kernel.c | 224 ++++++++++++++++++++++++++++-
 arch/x86/kernel/head64.c           | 211 +--------------------------
 3 files changed, 226 insertions(+), 211 deletions(-)
 create mode 100644 arch/x86/boot/startup/map_kernel.c
arch/x86/boot/startup/Makefile +1 -1
index 1beb5de..10319ae 100644
@@ -15,7 +15,7 @@ KMSAN_SANITIZE	:= n
 UBSAN_SANITIZE	:= n
 KCOV_INSTRUMENT	:= n
 
-obj-$(CONFIG_X86_64)		+= gdt_idt.o
+obj-$(CONFIG_X86_64)		+= gdt_idt.o map_kernel.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/map_kernel.c new file +224 -0

224 lines not on this page · raw message

arch/x86/kernel/head64.c +1 -210

223 lines not on this page · raw message

#7 lore raw

PATCH v4 4/11 x86/boot: Drop RIP_REL_REF() uses from early mapping code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Now that __startup_64() is built using -fPIC, RIP_REL_REF() has become a NOP and can be removed. Only some occurrences of rip_rel_ptr() will remain, to explicitly take the address of certain global structures in the 1:1 mapping of memory.

While at it, update the code comment to describe why this is needed.

---
 arch/x86/boot/startup/map_kernel.c | 41 ++++++++++----------
 1 file changed, 21 insertions(+), 20 deletions(-)
arch/x86/boot/startup/map_kernel.c +21 -20
index 5f1b7e0ba26e..0eac3f17dbd3 100644
@@ -26,12 +26,12 @@ static inline bool check_la57_support(void)
 	if (!(native_read_cr4() & X86_CR4_LA57))
 		return false;
 
-	RIP_REL_REF(__pgtable_l5_enabled)	= 1;
-	RIP_REL_REF(pgdir_shift)		= 48;
-	RIP_REL_REF(ptrs_per_p4d)		= 512;
-	RIP_REL_REF(page_offset_base)		= __PAGE_OFFSET_BASE_L5;
-	RIP_REL_REF(vmalloc_base)		= __VMALLOC_BASE_L5;
-	RIP_REL_REF(vmemmap_base)		= __VMEMMAP_BASE_L5;
+	__pgtable_l5_enabled	= 1;
+	pgdir_shift		= 48;
+	ptrs_per_p4d		= 512;
+	page_offset_base	= __PAGE_OFFSET_BASE_L5;
+	vmalloc_base		= __VMALLOC_BASE_L5;
+	vmemmap_base		= __VMEMMAP_BASE_L5;
 
 	return true;
 }
@@ -81,12 +81,14 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,
 	return sme_get_me_mask();
 }
 
-/* Code in __startup_64() can be relocated during execution, but the compiler
- * doesn't have to generate PC-relative relocations when accessing globals from
- * that function. Clang actually does not generate them, which leads to
- * boot-time crashes. To work around this problem, every global pointer must
- * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determined
- * by subtracting p2v_offset from the RIP-relative address.
+/*
+ * This code is compiled using PIC codegen because it will execute from the
+ * early 1:1 mapping of memory, which deviates from the mapping expected by the
+ * linker. Due to this deviation, taking the address of a global variable will
+ * produce an ambiguous result when using the plain & operator.  Instead,
+ * rip_rel_ptr() must be used, which will return the RIP-relative address in
+ * the 1:1 mapping of memory. Kernel virtual addresses can be determined by
+ * subtracting p2v_offset from the RIP-relative address.
  */
 unsigned long __head __startup_64(unsigned long p2v_offset,
 				  struct boot_params *bp)
@@ -113,8 +115,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 	 * Compute the delta between the address I am compiled to run at
 	 * and the address I am actually running at.
 	 */
-	load_delta = __START_KERNEL_map + p2v_offset;
-	RIP_REL_REF(phys_base) = load_delta;
+	phys_base = load_delta = __START_KERNEL_map + p2v_offset;
 
 	/* Is the address not 2M aligned? */
 	if (load_delta & ~PMD_MASK)
@@ -138,11 +139,11 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 		pgd[pgd_index(__START_KERNEL_map)] = (pgdval_t)p4d | _PAGE_TABLE;
 	}
 
-	RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 2].pud += load_delta;
-	RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 1].pud += load_delta;
+	level3_kernel_pgt[PTRS_PER_PUD - 2].pud += load_delta;
+	level3_kernel_pgt[PTRS_PER_PUD - 1].pud += load_delta;
 
 	for (i = FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--)
-		RIP_REL_REF(level2_fixmap_pgt)[i].pmd += load_delta;
+		level2_fixmap_pgt[i].pmd += load_delta;
 
 	/*
 	 * Set up the identity mapping for the switchover.  These
@@ -153,12 +154,12 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 
 	pud = &early_pgts[0]->pmd;
 	pmd = &early_pgts[1]->pmd;
-	RIP_REL_REF(next_early_pgt) = 2;
+	next_early_pgt = 2;
 
 	pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask();
 
 	if (la57) {
-		p4d = &early_pgts[RIP_REL_REF(next_early_pgt)++]->pmd;
+		p4d = &early_pgts[next_early_pgt++]->pmd;
 
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
 		pgd[i + 0] = (pgdval_t)p4d + pgtable_flags;
@@ -179,7 +180,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 
 	pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL;
 	/* Filter out unsupported __PAGE_KERNEL_* bits: */
-	pmd_entry &= RIP_REL_REF(__supported_pte_mask);
+	pmd_entry &= __supported_pte_mask;
 	pmd_entry += sme_get_me_mask();
 	pmd_entry +=  physaddr;
 
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#8 lore raw

[tip: x86/boot] x86/boot: Drop RIP_REL_REF() uses from early mapping code

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     dafb26f4271b9cc9cad07d9abf3c71c492e14f4c
Gitweb:        https://git.kernel.org/tip/dafb26f4271b9cc9cad07d9abf3c71c492e14f4c
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:22 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:05 +02:00

x86/boot: Drop RIP_REL_REF() uses from early mapping code

Now that __startup_64() is built using -fPIC, RIP_REL_REF() has become a NOP and can be removed. Only some occurrences of rip_rel_ptr() will remain, to explicitly take the address of certain global structures in the 1:1 mapping of memory.

While at it, update the code comment to describe why this is needed.

---
 arch/x86/boot/startup/map_kernel.c | 41 ++++++++++++++---------------
 1 file changed, 21 insertions(+), 20 deletions(-)
arch/x86/boot/startup/map_kernel.c +21 -20
index 5f1b7e0..0eac3f1 100644
@@ -26,12 +26,12 @@ static inline bool check_la57_support(void)
 	if (!(native_read_cr4() & X86_CR4_LA57))
 		return false;
 
-	RIP_REL_REF(__pgtable_l5_enabled)	= 1;
-	RIP_REL_REF(pgdir_shift)		= 48;
-	RIP_REL_REF(ptrs_per_p4d)		= 512;
-	RIP_REL_REF(page_offset_base)		= __PAGE_OFFSET_BASE_L5;
-	RIP_REL_REF(vmalloc_base)		= __VMALLOC_BASE_L5;
-	RIP_REL_REF(vmemmap_base)		= __VMEMMAP_BASE_L5;
+	__pgtable_l5_enabled	= 1;
+	pgdir_shift		= 48;
+	ptrs_per_p4d		= 512;
+	page_offset_base	= __PAGE_OFFSET_BASE_L5;
+	vmalloc_base		= __VMALLOC_BASE_L5;
+	vmemmap_base		= __VMEMMAP_BASE_L5;
 
 	return true;
 }
@@ -81,12 +81,14 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,
 	return sme_get_me_mask();
 }
 
-/* Code in __startup_64() can be relocated during execution, but the compiler
- * doesn't have to generate PC-relative relocations when accessing globals from
- * that function. Clang actually does not generate them, which leads to
- * boot-time crashes. To work around this problem, every global pointer must
- * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determined
- * by subtracting p2v_offset from the RIP-relative address.
+/*
+ * This code is compiled using PIC codegen because it will execute from the
+ * early 1:1 mapping of memory, which deviates from the mapping expected by the
+ * linker. Due to this deviation, taking the address of a global variable will
+ * produce an ambiguous result when using the plain & operator.  Instead,
+ * rip_rel_ptr() must be used, which will return the RIP-relative address in
+ * the 1:1 mapping of memory. Kernel virtual addresses can be determined by
+ * subtracting p2v_offset from the RIP-relative address.
  */
 unsigned long __head __startup_64(unsigned long p2v_offset,
 				  struct boot_params *bp)
@@ -113,8 +115,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 	 * Compute the delta between the address I am compiled to run at
 	 * and the address I am actually running at.
 	 */
-	load_delta = __START_KERNEL_map + p2v_offset;
-	RIP_REL_REF(phys_base) = load_delta;
+	phys_base = load_delta = __START_KERNEL_map + p2v_offset;
 
 	/* Is the address not 2M aligned? */
 	if (load_delta & ~PMD_MASK)
@@ -138,11 +139,11 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 		pgd[pgd_index(__START_KERNEL_map)] = (pgdval_t)p4d | _PAGE_TABLE;
 	}
 
-	RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 2].pud += load_delta;
-	RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 1].pud += load_delta;
+	level3_kernel_pgt[PTRS_PER_PUD - 2].pud += load_delta;
+	level3_kernel_pgt[PTRS_PER_PUD - 1].pud += load_delta;
 
 	for (i = FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--)
-		RIP_REL_REF(level2_fixmap_pgt)[i].pmd += load_delta;
+		level2_fixmap_pgt[i].pmd += load_delta;
 
 	/*
 	 * Set up the identity mapping for the switchover.  These
@@ -153,12 +154,12 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 
 	pud = &early_pgts[0]->pmd;
 	pmd = &early_pgts[1]->pmd;
-	RIP_REL_REF(next_early_pgt) = 2;
+	next_early_pgt = 2;
 
 	pgtable_flags = _KERNPG_TABLE_NOENC + sme_get_me_mask();
 
 	if (la57) {
-		p4d = &early_pgts[RIP_REL_REF(next_early_pgt)++]->pmd;
+		p4d = &early_pgts[next_early_pgt++]->pmd;
 
 		i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD;
 		pgd[i + 0] = (pgdval_t)p4d + pgtable_flags;
@@ -179,7 +180,7 @@ unsigned long __head __startup_64(unsigned long p2v_offset,
 
 	pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL;
 	/* Filter out unsupported __PAGE_KERNEL_* bits: */
-	pmd_entry &= RIP_REL_REF(__supported_pte_mask);
+	pmd_entry &= __supported_pte_mask;
 	pmd_entry += sme_get_me_mask();
 	pmd_entry +=  physaddr;
 
#9 lore raw

PATCH v4 5/11 x86/boot: Move early SME init code into startup/

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Move the SME initialization code, which runs from the 1:1 mapping of memory as it operates on the kernel virtual mapping, into the new sub-directory arch/x86/boot/startup/ where all startup code will reside that needs to tolerate executing from the 1:1 mapping.

---
 arch/x86/boot/startup/Makefile                             | 1 +
 arch/x86/{mm/mem_encrypt_identity.c => boot/startup/sme.c} | 2 --
 arch/x86/mm/Makefile                                       | 6 ------
 3 files changed, 1 insertion(+), 8 deletions(-)
arch/x86/boot/startup/Makefile +1 -0
index 10319aee666b..ccdfc42a4d59 100644
@@ -16,6 +16,7 @@ UBSAN_SANITIZE	:= n
 KCOV_INSTRUMENT	:= n
 
 obj-$(CONFIG_X86_64)		+= gdt_idt.o map_kernel.o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= sme.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/sme.c renamed from arch/x86/mm/mem_encrypt_identity.c +0 -2
similarity index 99%
rename from arch/x86/mm/mem_encrypt_identity.c
rename to arch/x86/boot/startup/sme.c
index e7fb3779b35f..23d10cda5b58 100644
@@ -45,8 +45,6 @@
 #include <asm/coco.h>
 #include <asm/sev.h>
 
-#include "mm_internal.h"
-
 #define PGD_FLAGS		_KERNPG_TABLE_NOENC
 #define P4D_FLAGS		_KERNPG_TABLE_NOENC
 #define PUD_FLAGS		_KERNPG_TABLE_NOENC
arch/x86/mm/Makefile +0 -6
index 32035d5be5a0..3faa60f13a61 100644
@@ -3,12 +3,10 @@
 KCOV_INSTRUMENT_tlb.o			:= n
 KCOV_INSTRUMENT_mem_encrypt.o		:= n
 KCOV_INSTRUMENT_mem_encrypt_amd.o	:= n
-KCOV_INSTRUMENT_mem_encrypt_identity.o	:= n
 KCOV_INSTRUMENT_pgprot.o		:= n
 
 KASAN_SANITIZE_mem_encrypt.o		:= n
 KASAN_SANITIZE_mem_encrypt_amd.o	:= n
-KASAN_SANITIZE_mem_encrypt_identity.o	:= n
 KASAN_SANITIZE_pgprot.o		:= n
 
 # Disable KCSAN entirely, because otherwise we get warnings that some functions
@@ -16,12 +14,10 @@ KASAN_SANITIZE_pgprot.o		:= n
 KCSAN_SANITIZE := n
 # Avoid recursion by not calling KMSAN hooks for CEA code.
 KMSAN_SANITIZE_cpu_entry_area.o := n
-KMSAN_SANITIZE_mem_encrypt_identity.o := n
 
 ifdef CONFIG_FUNCTION_TRACER
 CFLAGS_REMOVE_mem_encrypt.o		= -pg
 CFLAGS_REMOVE_mem_encrypt_amd.o		= -pg
-CFLAGS_REMOVE_mem_encrypt_identity.o	= -pg
 CFLAGS_REMOVE_pgprot.o			= -pg
 endif
 
@@ -32,7 +28,6 @@ obj-y				+= pat/
 
 # Make sure __phys_addr has no stackprotector
 CFLAGS_physaddr.o		:= -fno-stack-protector
-CFLAGS_mem_encrypt_identity.o	:= -fno-stack-protector
 
 CFLAGS_fault.o := -I $(src)/../include/asm/trace
 
@@ -63,5 +58,4 @@ obj-$(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION)	+= pti.o
 obj-$(CONFIG_X86_MEM_ENCRYPT)	+= mem_encrypt.o
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_amd.o
 
-obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_identity.o
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#10 lore raw

[tip: x86/boot] x86/boot: Move early SME init code into startup/

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     7ae089ee75f3c77b94ff23204b41ea06da9f0193
Gitweb:        https://git.kernel.org/tip/7ae089ee75f3c77b94ff23204b41ea06da9f0193
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:23 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:05 +02:00

x86/boot: Move early SME init code into startup/

Move the SME initialization code, which runs from the 1:1 mapping of memory as it operates on the kernel virtual mapping, into the new sub-directory arch/x86/boot/startup/ where all startup code will reside that needs to tolerate executing from the 1:1 mapping.

---
 arch/x86/boot/startup/Makefile     |   1 +-
 arch/x86/boot/startup/sme.c        | 567 ++++++++++++++++++++++++++++-
 arch/x86/mm/Makefile               |   6 +-
 arch/x86/mm/mem_encrypt_identity.c | 569 +----------------------------
 4 files changed, 568 insertions(+), 575 deletions(-)
 create mode 100644 arch/x86/boot/startup/sme.c
 delete mode 100644 arch/x86/mm/mem_encrypt_identity.c
arch/x86/boot/startup/Makefile +1 -0
index 10319ae..ccdfc42 100644
@@ -16,6 +16,7 @@ UBSAN_SANITIZE	:= n
 KCOV_INSTRUMENT	:= n
 
 obj-$(CONFIG_X86_64)		+= gdt_idt.o map_kernel.o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= sme.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/sme.c new file +567 -0

567 lines not on this page · raw message

arch/x86/mm/Makefile +0 -6
index 32035d5..3faa60f 100644
@@ -3,12 +3,10 @@
 KCOV_INSTRUMENT_tlb.o			:= n
 KCOV_INSTRUMENT_mem_encrypt.o		:= n
 KCOV_INSTRUMENT_mem_encrypt_amd.o	:= n
-KCOV_INSTRUMENT_mem_encrypt_identity.o	:= n
 KCOV_INSTRUMENT_pgprot.o		:= n
 
 KASAN_SANITIZE_mem_encrypt.o		:= n
 KASAN_SANITIZE_mem_encrypt_amd.o	:= n
-KASAN_SANITIZE_mem_encrypt_identity.o	:= n
 KASAN_SANITIZE_pgprot.o		:= n
 
 # Disable KCSAN entirely, because otherwise we get warnings that some functions
@@ -16,12 +14,10 @@ KASAN_SANITIZE_pgprot.o		:= n
 KCSAN_SANITIZE := n
 # Avoid recursion by not calling KMSAN hooks for CEA code.
 KMSAN_SANITIZE_cpu_entry_area.o := n
-KMSAN_SANITIZE_mem_encrypt_identity.o := n
 
 ifdef CONFIG_FUNCTION_TRACER
 CFLAGS_REMOVE_mem_encrypt.o		= -pg
 CFLAGS_REMOVE_mem_encrypt_amd.o		= -pg
-CFLAGS_REMOVE_mem_encrypt_identity.o	= -pg
 CFLAGS_REMOVE_pgprot.o			= -pg
 endif
 
@@ -32,7 +28,6 @@ obj-y				+= pat/
 
 # Make sure __phys_addr has no stackprotector
 CFLAGS_physaddr.o		:= -fno-stack-protector
-CFLAGS_mem_encrypt_identity.o	:= -fno-stack-protector
 
 CFLAGS_fault.o := -I $(src)/../include/asm/trace
 
@@ -63,5 +58,4 @@ obj-$(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION)	+= pti.o
 obj-$(CONFIG_X86_MEM_ENCRYPT)	+= mem_encrypt.o
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_amd.o
 
-obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_identity.o
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_boot.o
arch/x86/mm/mem_encrypt_identity.c deleted +0 -569

569 lines not on this page · raw message

#11 lore raw

PATCH v4 6/11 x86/boot: Drop RIP_REL_REF() uses from SME startup code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

RIP_REL_REF() has no effect on code residing in arch/x86/boot/startup, as it is built with -fPIC. So remove any occurrences from the SME startup code.

Note the SME is the only caller of cc_set_mask() that requires this, so drop it from there as well.

---
 arch/x86/boot/startup/sme.c        | 11 +++++------
 arch/x86/include/asm/coco.h        |  2 +-
 arch/x86/include/asm/mem_encrypt.h |  2 +-
 3 files changed, 7 insertions(+), 8 deletions(-)
arch/x86/boot/startup/sme.c +5 -6
index 23d10cda5b58..5738b31c8e60 100644
@@ -297,8 +297,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp)
 	 * instrumentation or checking boot_cpu_data in the cc_platform_has()
 	 * function.
 	 */
-	if (!sme_get_me_mask() ||
-	    RIP_REL_REF(sev_status) & MSR_AMD64_SEV_ENABLED)
+	if (!sme_get_me_mask() || sev_status & MSR_AMD64_SEV_ENABLED)
 		return;
 
 	/*
@@ -524,7 +523,7 @@ void __head sme_enable(struct boot_params *bp)
 	me_mask = 1UL << (ebx & 0x3f);
 
 	/* Check the SEV MSR whether SEV or SME is enabled */
-	RIP_REL_REF(sev_status) = msr = __rdmsr(MSR_AMD64_SEV);
+	sev_status = msr = __rdmsr(MSR_AMD64_SEV);
 	feature_mask = (msr & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;
 
 	/*
@@ -560,8 +559,8 @@ void __head sme_enable(struct boot_params *bp)
 			return;
 	}
 
-	RIP_REL_REF(sme_me_mask) = me_mask;
-	RIP_REL_REF(physical_mask) &= ~me_mask;
-	RIP_REL_REF(cc_vendor) = CC_VENDOR_AMD;
+	sme_me_mask	= me_mask;
+	physical_mask	&= ~me_mask;
+	cc_vendor	= CC_VENDOR_AMD;
 	cc_set_mask(me_mask);
 }
arch/x86/include/asm/coco.h +1 -1
index e7225452963f..e1dbf8df1b69 100644
@@ -22,7 +22,7 @@ static inline u64 cc_get_mask(void)
 
 static inline void cc_set_mask(u64 mask)
 {
-	RIP_REL_REF(cc_mask) = mask;
+	cc_mask = mask;
 }
 
 u64 cc_mkenc(u64 val);
arch/x86/include/asm/mem_encrypt.h +1 -1
index 1530ee301dfe..ea6494628cb0 100644
@@ -61,7 +61,7 @@ void __init sev_es_init_vc_handling(void);
 
 static inline u64 sme_get_me_mask(void)
 {
-	return RIP_REL_REF(sme_me_mask);
+	return sme_me_mask;
 }
 
 #define __bss_decrypted __section(".bss..decrypted")
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#12 lore raw

[tip: x86/boot] x86/boot: Drop RIP_REL_REF() uses from SME startup code

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     bee174b27e54462ef18b38f8377d27ac0ad14350
Gitweb:        https://git.kernel.org/tip/bee174b27e54462ef18b38f8377d27ac0ad14350
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:24 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:05 +02:00

x86/boot: Drop RIP_REL_REF() uses from SME startup code

RIP_REL_REF() has no effect on code residing in arch/x86/boot/startup, as it is built with -fPIC. So remove any occurrences from the SME startup code.

Note the SME is the only caller of cc_set_mask() that requires this, so drop it from there as well.

---
 arch/x86/boot/startup/sme.c        | 11 +++++------
 arch/x86/include/asm/coco.h        |  2 +-
 arch/x86/include/asm/mem_encrypt.h |  2 +-
 3 files changed, 7 insertions(+), 8 deletions(-)
arch/x86/boot/startup/sme.c +5 -6
index 23d10cd..5738b31 100644
@@ -297,8 +297,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp)
 	 * instrumentation or checking boot_cpu_data in the cc_platform_has()
 	 * function.
 	 */
-	if (!sme_get_me_mask() ||
-	    RIP_REL_REF(sev_status) & MSR_AMD64_SEV_ENABLED)
+	if (!sme_get_me_mask() || sev_status & MSR_AMD64_SEV_ENABLED)
 		return;
 
 	/*
@@ -524,7 +523,7 @@ void __head sme_enable(struct boot_params *bp)
 	me_mask = 1UL << (ebx & 0x3f);
 
 	/* Check the SEV MSR whether SEV or SME is enabled */
-	RIP_REL_REF(sev_status) = msr = __rdmsr(MSR_AMD64_SEV);
+	sev_status = msr = __rdmsr(MSR_AMD64_SEV);
 	feature_mask = (msr & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;
 
 	/*
@@ -560,8 +559,8 @@ void __head sme_enable(struct boot_params *bp)
 			return;
 	}
 
-	RIP_REL_REF(sme_me_mask) = me_mask;
-	RIP_REL_REF(physical_mask) &= ~me_mask;
-	RIP_REL_REF(cc_vendor) = CC_VENDOR_AMD;
+	sme_me_mask	= me_mask;
+	physical_mask	&= ~me_mask;
+	cc_vendor	= CC_VENDOR_AMD;
 	cc_set_mask(me_mask);
 }
arch/x86/include/asm/coco.h +1 -1
index e722545..e1dbf8d 100644
@@ -22,7 +22,7 @@ static inline u64 cc_get_mask(void)
 
 static inline void cc_set_mask(u64 mask)
 {
-	RIP_REL_REF(cc_mask) = mask;
+	cc_mask = mask;
 }
 
 u64 cc_mkenc(u64 val);
arch/x86/include/asm/mem_encrypt.h +1 -1
index 1530ee3..ea64946 100644
@@ -61,7 +61,7 @@ void __init sev_es_init_vc_handling(void);
 
 static inline u64 sme_get_me_mask(void)
 {
-	return RIP_REL_REF(sme_me_mask);
+	return sme_me_mask;
 }
 
 #define __bss_decrypted __section(".bss..decrypted")
#13 lore raw

PATCH v4 7/11 x86/sev: Prepare for splitting off early SEV code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Prepare for splitting off parts of the SEV core.c source file into a file that carries code that must tolerate being called from the early 1:1 mapping. This will allow special build-time handling of thise code, to ensure that it gets generated in a way that is compatible with the early execution context.

So create a de-facto internal SEV API and put the definitions into sev-internal.h. No attempt is made to allow this header file to be included in arbitrary other sources - this is explicitly not the intent.

---
 arch/x86/boot/compressed/sev.c      |  15 ++-
 arch/x86/coco/sev/core.c            | 108 +++--------------
 arch/x86/coco/sev/shared.c          |  64 ++--------
 arch/x86/include/asm/sev-internal.h | 122 ++++++++++++++++++++
 arch/x86/include/asm/sev.h          |  37 ++++++
 5 files changed, 194 insertions(+), 152 deletions(-)
arch/x86/boot/compressed/sev.c +10 -5
index 89ba168f4f0f..478eca4f7180 100644
@@ -126,20 +126,25 @@ static bool fault_in_kernel_space(unsigned long address)
 #include "../../lib/inat.c"
 #include "../../lib/insn.c"
 
-/* Include code for early handlers */
-#include "../../coco/sev/shared.c"
+extern struct svsm_ca *boot_svsm_caa;
+extern u64 boot_svsm_caa_pa;
 
-static struct svsm_ca *svsm_get_caa(void)
+struct svsm_ca *svsm_get_caa(void)
 {
 	return boot_svsm_caa;
 }
 
-static u64 svsm_get_caa_pa(void)
+u64 svsm_get_caa_pa(void)
 {
 	return boot_svsm_caa_pa;
 }
 
-static int svsm_perform_call_protocol(struct svsm_call *call)
+int svsm_perform_call_protocol(struct svsm_call *call);
+
+/* Include code for early handlers */
+#include "../../coco/sev/shared.c"
+
+int svsm_perform_call_protocol(struct svsm_call *call)
 {
 	struct ghcb *ghcb;
 	int ret;
arch/x86/coco/sev/core.c +13 -95

172 lines not on this page · raw message

arch/x86/coco/sev/shared.c +12 -52

124 lines not on this page · raw message

arch/x86/include/asm/sev-internal.h new file +122 -0

122 lines not on this page · raw message

arch/x86/include/asm/sev.h +37 -0
index ba7999f66abe..a8661dfc9a9a 100644
@@ -15,6 +15,7 @@
 #include <asm/sev-common.h>
 #include <asm/coco.h>
 #include <asm/set_memory.h>
+#include <asm/svm.h>
 
 #define GHCB_PROTOCOL_MIN	1ULL
 #define GHCB_PROTOCOL_MAX	2ULL
@@ -83,6 +84,36 @@ extern void vc_no_ghcb(void);
 extern void vc_boot_ghcb(void);
 extern bool handle_vc_boot_ghcb(struct pt_regs *regs);
 
+/*
+ * Individual entries of the SNP CPUID table, as defined by the SNP
+ * Firmware ABI, Revision 0.9, Section 7.1, Table 14.
+ */
+struct snp_cpuid_fn {
+	u32 eax_in;
+	u32 ecx_in;
+	u64 xcr0_in;
+	u64 xss_in;
+	u32 eax;
+	u32 ebx;
+	u32 ecx;
+	u32 edx;
+	u64 __reserved;
+} __packed;
+
+/*
+ * SNP CPUID table, as defined by the SNP Firmware ABI, Revision 0.9,
+ * Section 8.14.2.6. Also noted there is the SNP firmware-enforced limit
+ * of 64 entries per CPUID table.
+ */
+#define SNP_CPUID_COUNT_MAX 64
+
+struct snp_cpuid_table {
+	u32 count;
+	u32 __reserved1;
+	u64 __reserved2;
+	struct snp_cpuid_fn fn[SNP_CPUID_COUNT_MAX];
+} __packed;
+
 /* PVALIDATE return codes */
 #define PVALIDATE_FAIL_SIZEMISMATCH	6
 
@@ -484,6 +515,12 @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 
+static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)
+{
+	ghcb->save.sw_exit_code = 0;
+	__builtin_memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap));
+}
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define snp_vmpl 0
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
merge bot Applied
#14 lore raw

[tip: x86/boot] x86/sev: Prepare for splitting off early SEV code

The following commit has been merged into the x86/boot branch of tip:

The following commit has been merged into the x86/boot branch of tip:

Commit-ID:     221df25fdf827b1fe5b904c6a396af06461a32f6
Gitweb:        https://git.kernel.org/tip/221df25fdf827b1fe5b904c6a396af06461a32f6
Author:        Ard Biesheuvel <[email protected]>
AuthorDate:    Thu, 10 Apr 2025 15:41:25 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Sat, 12 Apr 2025 11:13:05 +02:00

x86/sev: Prepare for splitting off early SEV code

Prepare for splitting off parts of the SEV core.c source file into a file that carries code that must tolerate being called from the early 1:1 mapping. This will allow special build-time handling of thise code, to ensure that it gets generated in a way that is compatible with the early execution context.

So create a de-facto internal SEV API and put the definitions into sev-internal.h. No attempt is made to allow this header file to be included in arbitrary other sources - this is explicitly not the intent.

---
 arch/x86/boot/compressed/sev.c      |  15 ++-
 arch/x86/coco/sev/core.c            | 108 ++----------------------
 arch/x86/coco/sev/shared.c          |  64 ++------------
 arch/x86/include/asm/sev-internal.h | 122 +++++++++++++++++++++++++++-
 arch/x86/include/asm/sev.h          |  37 ++++++++-
 5 files changed, 194 insertions(+), 152 deletions(-)
 create mode 100644 arch/x86/include/asm/sev-internal.h
arch/x86/boot/compressed/sev.c +10 -5
index bb55934..6eadd79 100644
@@ -126,20 +126,25 @@ static bool fault_in_kernel_space(unsigned long address)
 #include "../../lib/inat.c"
 #include "../../lib/insn.c"
 
-/* Include code for early handlers */
-#include "../../coco/sev/shared.c"
+extern struct svsm_ca *boot_svsm_caa;
+extern u64 boot_svsm_caa_pa;
 
-static struct svsm_ca *svsm_get_caa(void)
+struct svsm_ca *svsm_get_caa(void)
 {
 	return boot_svsm_caa;
 }
 
-static u64 svsm_get_caa_pa(void)
+u64 svsm_get_caa_pa(void)
 {
 	return boot_svsm_caa_pa;
 }
 
-static int svsm_perform_call_protocol(struct svsm_call *call)
+int svsm_perform_call_protocol(struct svsm_call *call);
+
+/* Include code for early handlers */
+#include "../../coco/sev/shared.c"
+
+int svsm_perform_call_protocol(struct svsm_call *call)
 {
 	struct ghcb *ghcb;
 	int ret;
arch/x86/coco/sev/core.c +13 -95

172 lines not on this page · raw message

arch/x86/coco/sev/shared.c +12 -52

124 lines not on this page · raw message

arch/x86/include/asm/sev-internal.h new file +122 -0

122 lines not on this page · raw message

arch/x86/include/asm/sev.h +37 -0
index ba7999f..a8661df 100644
@@ -15,6 +15,7 @@
 #include <asm/sev-common.h>
 #include <asm/coco.h>
 #include <asm/set_memory.h>
+#include <asm/svm.h>
 
 #define GHCB_PROTOCOL_MIN	1ULL
 #define GHCB_PROTOCOL_MAX	2ULL
@@ -83,6 +84,36 @@ extern void vc_no_ghcb(void);
 extern void vc_boot_ghcb(void);
 extern bool handle_vc_boot_ghcb(struct pt_regs *regs);
 
+/*
+ * Individual entries of the SNP CPUID table, as defined by the SNP
+ * Firmware ABI, Revision 0.9, Section 7.1, Table 14.
+ */
+struct snp_cpuid_fn {
+	u32 eax_in;
+	u32 ecx_in;
+	u64 xcr0_in;
+	u64 xss_in;
+	u32 eax;
+	u32 ebx;
+	u32 ecx;
+	u32 edx;
+	u64 __reserved;
+} __packed;
+
+/*
+ * SNP CPUID table, as defined by the SNP Firmware ABI, Revision 0.9,
+ * Section 8.14.2.6. Also noted there is the SNP firmware-enforced limit
+ * of 64 entries per CPUID table.
+ */
+#define SNP_CPUID_COUNT_MAX 64
+
+struct snp_cpuid_table {
+	u32 count;
+	u32 __reserved1;
+	u64 __reserved2;
+	struct snp_cpuid_fn fn[SNP_CPUID_COUNT_MAX];
+} __packed;
+
 /* PVALIDATE return codes */
 #define PVALIDATE_FAIL_SIZEMISMATCH	6
 
@@ -484,6 +515,12 @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 
+static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)
+{
+	ghcb->save.sw_exit_code = 0;
+	__builtin_memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap));
+}
+
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
 #define snp_vmpl 0
#15 lore raw

PATCH v4 8/11 x86/sev: Split off startup code from core code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Disentangle the SEV core code and the SEV code that is called during early boot. The latter piece will be moved into startup/ in a subsequent patch.

---
 arch/x86/boot/compressed/sev.c |    2 +
 arch/x86/coco/sev/Makefile     |   12 +-
 arch/x86/coco/sev/core.c       | 1574 ++++----------------
 arch/x86/coco/sev/shared.c     |  281 ----
 arch/x86/coco/sev/startup.c    | 1395 +++++++++++++++++
 5 files changed, 1658 insertions(+), 1606 deletions(-)
arch/x86/boot/compressed/sev.c +2 -0
index 478eca4f7180..714e30c66eae 100644
@@ -141,6 +141,8 @@ u64 svsm_get_caa_pa(void)
 
 int svsm_perform_call_protocol(struct svsm_call *call);
 
+u8 snp_vmpl;
+
 /* Include code for early handlers */
 #include "../../coco/sev/shared.c"
 
arch/x86/coco/sev/Makefile +6 -6
index dcb06dc8b5ae..7d7d2aee62f0 100644
@@ -1,18 +1,18 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y += core.o
+obj-y += core.o startup.o
 
 # jump tables are emitted using absolute references in non-PIC code
 # so they cannot be used in the early SEV startup code
-CFLAGS_core.o += -fno-jump-tables
+CFLAGS_startup.o += -fno-jump-tables
 
 ifdef CONFIG_FUNCTION_TRACER
-CFLAGS_REMOVE_core.o = -pg
+CFLAGS_REMOVE_startup.o = -pg
 endif
 
-KASAN_SANITIZE_core.o	:= n
-KMSAN_SANITIZE_core.o	:= n
-KCOV_INSTRUMENT_core.o	:= n
+KASAN_SANITIZE_startup.o	:= n
+KMSAN_SANITIZE_startup.o	:= n
+KCOV_INSTRUMENT_startup.o	:= n
 
 # With some compiler versions the generated code results in boot hangs, caused
 # by several compilation units. To be safe, disable all instrumentation.
arch/x86/coco/sev/core.c +255 -1319

1,712 lines not on this page · raw message

arch/x86/coco/sev/shared.c +0 -281

305 lines not on this page · raw message

arch/x86/coco/sev/startup.c new file +1395 -0

951 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.49.0.504.g3bcea36a83-goog

Part of the diff is not shown here. View the raw message.

#16 lore raw

Re: [PATCH v4 08/11] x86/sev: Split off startup code from core code

kernel test robot noticed the following build errors:

Hi Ard,

kernel test robot noticed the following build errors:

[auto build test ERROR on tip/master]
[also build test ERROR on next-20250411]
[cannot apply to tip/x86/core tip/x86/mm efi/next tip/auto-latest linus/master v6.15-rc1] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ard-Biesheuvel/x86-asm-Make-rip_rel_ptr-usable-from-fPIC-code/20250410-214836
base:   tip/master
patch link:    https://lore.kernel.org/r/20250410134117.3713574-21-ardb%2Bgit%40google.com
patch subject: [PATCH v4 08/11] x86/sev: Split off startup code from core code
config: x86_64-buildonly-randconfig-005-20250411 (https://download.01.org/0day-ci/archive/20250412/[email protected]/config)
compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250412/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> arch/x86/boot/compressed/sev.c:263:6: error: call to undeclared function 'vmgexit_psc'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     263 |         if (vmgexit_psc(boot_ghcb, desc))
         |             ^
>> arch/x86/boot/compressed/sev.c:266:2: error: call to undeclared function 'pvalidate_pages'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
     266 |         pvalidate_pages(desc);
         |         ^

arch/x86/boot/compressed/sev.c:266:2: note: did you mean 'pvalidate_4k_page'? arch/x86/boot/compressed/../../coco/sev/shared.c:1227:20: note: 'pvalidate_4k_page' declared here
1227 | static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,
| ^
2 errors generated.

vim +/vmgexit_psc +263 arch/x86/boot/compressed/sev.c

597cfe48212a3f arch/x86/boot/compressed/sev-es.c Joerg Roedel 2020-09-07  232  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  233  static phys_addr_t __snp_accept_memory(struct snp_psc_desc *desc,
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  234  				       phys_addr_t pa, phys_addr_t pa_end)
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  235  {
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  236  	struct psc_hdr *hdr;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  237  	struct psc_entry *e;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  238  	unsigned int i;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  239  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  240  	hdr = &desc->hdr;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  241  	memset(hdr, 0, sizeof(*hdr));
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  242  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  243  	e = desc->entries;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  244  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  245  	i = 0;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  246  	while (pa < pa_end && i < VMGEXIT_PSC_MAX_ENTRY) {
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  247  		hdr->end_entry = i;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  248  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  249  		e->gfn = pa >> PAGE_SHIFT;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  250  		e->operation = SNP_PAGE_STATE_PRIVATE;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  251  		if (IS_ALIGNED(pa, PMD_SIZE) && (pa_end - pa) >= PMD_SIZE) {
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  252  			e->pagesize = RMP_PG_SIZE_2M;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  253  			pa += PMD_SIZE;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  254  		} else {
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  255  			e->pagesize = RMP_PG_SIZE_4K;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  256  			pa += PAGE_SIZE;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  257  		}
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  258  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  259  		e++;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  260  		i++;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  261  	}
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  262  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06 @263  	if (vmgexit_psc(boot_ghcb, desc))
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  264  		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PSC);
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  265  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06 @266  	pvalidate_pages(desc);
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  267  
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  268  	return pa;
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  269  }
6c3211796326a9 arch/x86/boot/compressed/sev.c    Tom Lendacky 2023-06-06  270  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#17 lore raw

Re: [PATCH v4 08/11] x86/sev: Split off startup code from core code

In reply to Ard Biesheuvel
Quoted 6 lines · Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Disentangle the SEV core code and the SEV code that is called during early boot. The latter piece will be moved into startup/ in a subsequent patch.

Quoted 8 lines
---
 arch/x86/boot/compressed/sev.c |    2 +
 arch/x86/coco/sev/Makefile     |   12 +-
 arch/x86/coco/sev/core.c       | 1574 ++++----------------
 arch/x86/coco/sev/shared.c     |  281 ----
 arch/x86/coco/sev/startup.c    | 1395 +++++++++++++++++
 5 files changed, 1658 insertions(+), 1606 deletions(-)

x86-64 allmodconfig build failure:

arch/x86/boot/compressed/sev.c:263:13: error: implicit declaration of function ‘vmgexit_psc’ [-Wimplicit-function-declaration]
| ^~~~~~~~~~~
arch/x86/boot/compressed/sev.c:266:9: error: implicit declaration of function ‘pvalidate_pages’; did you mean ‘pvalidate_4k_page’? [-Wimplicit-function-declaration]

|         ^~~~~~~~~~~~~~~
|         pvalidate_4k_page

Thanks,

Ingo

#18 lore raw

Re: [PATCH v4 08/11] x86/sev: Split off startup code from core code

Quoted 18 lines · Ingo Molnar
Quoted 6 lines · Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Disentangle the SEV core code and the SEV code that is called during early boot. The latter piece will be moved into startup/ in a subsequent patch.

Quoted 8 lines
---
 arch/x86/boot/compressed/sev.c |    2 +
 arch/x86/coco/sev/Makefile     |   12 +-
 arch/x86/coco/sev/core.c       | 1574 ++++----------------
 arch/x86/coco/sev/shared.c     |  281 ----
 arch/x86/coco/sev/startup.c    | 1395 +++++++++++++++++
 5 files changed, 1658 insertions(+), 1606 deletions(-)
Quoted 7 lines

x86-64 allmodconfig build failure:

arch/x86/boot/compressed/sev.c:263:13: error: implicit declaration of function ‘vmgexit_psc’ [-Wimplicit-function-declaration]
| ^~~~~~~~~~~
arch/x86/boot/compressed/sev.c:266:9: error: implicit declaration of function ‘pvalidate_pages’; did you mean ‘pvalidate_4k_page’? [-Wimplicit-function-declaration]

|         ^~~~~~~~~~~~~~~
|         pvalidate_4k_page

Ignore that, I have now read the cover letter too, with the patch dependency mentioned there - as kindly pointed out by Ard in a private mail. :-)

Thanks,

Ingo

#19 lore raw

Re: [PATCH v4 08/11] x86/sev: Split off startup code from core code

Quoted 3 lines · Ingo Molnar

Ignore that, I have now read the cover letter too, with the patch dependency mentioned there - as kindly pointed out by Ard in a private mail. :-)

But there are other problems during the allmodconfig final link:

vmlinux.o: warning: objtool: __sev_es_nmi_complete+0x5a: call to __asan_memset() leaves .noinstr.text section
ld: error: unplaced orphan section .data.rel.local' from vmlinux.o'
make[2]: *** [scripts/Makefile.vmlinux:91: vmlinux.unstripped] Error 1

The objtool warning is caused by:

x86/sev: Split off startup code from core code

Tte link failure by:

x86/boot: Move SEV startup code into startup/

Thanks,

Ingo

maintainer
#20 lore raw

Re: [PATCH v4 08/11] x86/sev: Split off startup code from core code

Quoted 11 lines · Ingo Molnar
Quoted 3 lines · Ingo Molnar

Ignore that, I have now read the cover letter too, with the patch dependency mentioned there - as kindly pointed out by Ard in a private mail. :-)

But there are other problems during the allmodconfig final link:

vmlinux.o: warning: objtool: __sev_es_nmi_complete+0x5a: call to __asan_memset() leaves .noinstr.text section

This is an odd one, because noinstr functions should not be instrumented by kasan afaik.

Quoted 1 line

ld: error: unplaced orphan section .data.rel.local' from vmlinux.o'

This should have been included in 68f3ea7ee199ef77551e090dfef5a49046ea8443, the commit log has the details. TL;DR this is .rodata with relocatable quantities, which is not emitted into .rodata when using -fPIC.

Quoted 10 lines

make[2]: *** [scripts/Makefile.vmlinux:91: vmlinux.unstripped] Error 1

The objtool warning is caused by:

x86/sev: Split off startup code from core code

Tte link failure by:

x86/boot: Move SEV startup code into startup/

Thanks for the report - I'll fix these up in the next revision.

#21 lore raw

Re: [PATCH v4 08/11] x86/sev: Split off startup code from core code

Quoted 9 lines · Ard Biesheuvel
Quoted 8 lines · Ingo Molnar
Quoted 3 lines · Ingo Molnar

Ignore that, I have now read the cover letter too, with the patch dependency mentioned there - as kindly pointed out by Ard in a private mail. :-)

Quoted 6 lines
Quoted 3 lines

But there are other problems during the allmodconfig final link:

vmlinux.o: warning: objtool: __sev_es_nmi_complete+0x5a: call to __asan_memset() leaves .noinstr.text section

This is an odd one, because noinstr functions should not be instrumented by kasan afaik.

FWIW I'm not doing anything particularly weird on the build environment
side: GCC 14.2.0.

Thanks,

Ingo

#22 lore raw

PATCH v4 9/11 x86/boot: Move SEV startup code into startup/

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Move the SEV startup code into arch/x86/boot/startup/, where it will reside along with other code that executes extremely early, and therefore needs to be built in a special manner.

---
 arch/x86/boot/compressed/sev.c                              |  2 +-
 arch/x86/boot/startup/Makefile                              |  2 +-
 arch/x86/{coco/sev/shared.c => boot/startup/sev-shared.c}   |  0
 arch/x86/{coco/sev/startup.c => boot/startup/sev-startup.c} |  2 +-
 arch/x86/coco/sev/Makefile                                  | 21 +-------------------
 5 files changed, 4 insertions(+), 23 deletions(-)
arch/x86/boot/compressed/sev.c +1 -1
index 714e30c66eae..478c65149cf0 100644
@@ -144,7 +144,7 @@ int svsm_perform_call_protocol(struct svsm_call *call);
 u8 snp_vmpl;
 
 /* Include code for early handlers */
-#include "../../coco/sev/shared.c"
+#include "../../boot/startup/sev-shared.c"
 
 int svsm_perform_call_protocol(struct svsm_call *call)
 {
arch/x86/boot/startup/Makefile +1 -1
index ccdfc42a4d59..b56facb9091a 100644
@@ -16,7 +16,7 @@ UBSAN_SANITIZE	:= n
 KCOV_INSTRUMENT	:= n
 
 obj-$(CONFIG_X86_64)		+= gdt_idt.o map_kernel.o
-obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= sme.o
+obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= sme.o sev-startup.o
 
 lib-$(CONFIG_X86_64)		+= la57toggle.o
 lib-$(CONFIG_EFI_MIXED)		+= efi-mixed.o
arch/x86/boot/startup/sev-shared.c renamed from arch/x86/coco/sev/shared.c +0 -0
similarity index 100%
rename from arch/x86/coco/sev/shared.c
rename to arch/x86/boot/startup/sev-shared.c
arch/x86/boot/startup/sev-startup.c renamed from arch/x86/coco/sev/startup.c +1 -1
similarity index 99%
rename from arch/x86/coco/sev/startup.c
rename to arch/x86/boot/startup/sev-startup.c
index 9f5dc70cfb44..10b636009d1c 100644
@@ -422,7 +422,7 @@ static __always_inline void vc_forward_exception(struct es_em_ctxt *ctxt)
 }
 
 /* Include code shared with pre-decompression boot stage */
-#include "shared.c"
+#include "sev-shared.c"
 
 noinstr void __sev_put_ghcb(struct ghcb_state *state)
 {
arch/x86/coco/sev/Makefile +1 -20
index 7d7d2aee62f0..b89ba3fba343 100644
@@ -1,22 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 
-obj-y += core.o startup.o
-
-# jump tables are emitted using absolute references in non-PIC code
-# so they cannot be used in the early SEV startup code
-CFLAGS_startup.o += -fno-jump-tables
-
-ifdef CONFIG_FUNCTION_TRACER
-CFLAGS_REMOVE_startup.o = -pg
-endif
-
-KASAN_SANITIZE_startup.o	:= n
-KMSAN_SANITIZE_startup.o	:= n
-KCOV_INSTRUMENT_startup.o	:= n
-
-# With some compiler versions the generated code results in boot hangs, caused
-# by several compilation units. To be safe, disable all instrumentation.
-KCSAN_SANITIZE		:= n
-
-# Clang 14 and older may fail to respect __no_sanitize_undefined when inlining
-UBSAN_SANITIZE		:= n
+obj-y += core.o
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
#23 lore raw

PATCH v4 10/11 x86/boot: Drop RIP_REL_REF() uses from early SEV code

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Now that the early SEV code is built with -fPIC, RIP_REL_REF() has no effect and can be dropped.

---
 arch/x86/boot/startup/sev-shared.c  | 26 +++++++++-----------
 arch/x86/boot/startup/sev-startup.c | 16 ++++++------
 arch/x86/include/asm/sev-internal.h | 18 +++-----------
 3 files changed, 23 insertions(+), 37 deletions(-)
arch/x86/boot/startup/sev-shared.c +11 -15
index 815542295f16..173f3d1f777a 100644
@@ -299,7 +299,7 @@ static int svsm_perform_ghcb_protocol(struct ghcb *ghcb, struct svsm_call *call)
 	 * Fill in protocol and format specifiers. This can be called very early
 	 * in the boot, so use rip-relative references as needed.
 	 */
-	ghcb->protocol_version = RIP_REL_REF(ghcb_version);
+	ghcb->protocol_version = ghcb_version;
 	ghcb->ghcb_usage       = GHCB_DEFAULT_USAGE;
 
 	ghcb_set_sw_exit_code(ghcb, SVM_VMGEXIT_SNP_RUN_VMPL);
@@ -656,9 +656,9 @@ snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf)
 		leaf->eax = leaf->ebx = leaf->ecx = leaf->edx = 0;
 
 		/* Skip post-processing for out-of-range zero leafs. */
-		if (!(leaf->fn <= RIP_REL_REF(cpuid_std_range_max) ||
-		      (leaf->fn >= 0x40000000 && leaf->fn <= RIP_REL_REF(cpuid_hyp_range_max)) ||
-		      (leaf->fn >= 0x80000000 && leaf->fn <= RIP_REL_REF(cpuid_ext_range_max))))
+		if (!(leaf->fn <= cpuid_std_range_max ||
+		      (leaf->fn >= 0x40000000 && leaf->fn <= cpuid_hyp_range_max) ||
+		      (leaf->fn >= 0x80000000 && leaf->fn <= cpuid_ext_range_max)))
 			return 0;
 	}
 
@@ -1179,11 +1179,11 @@ static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info)
 		const struct snp_cpuid_fn *fn = &cpuid_table->fn[i];
 
 		if (fn->eax_in == 0x0)
-			RIP_REL_REF(cpuid_std_range_max) = fn->eax;
+			cpuid_std_range_max = fn->eax;
 		else if (fn->eax_in == 0x40000000)
-			RIP_REL_REF(cpuid_hyp_range_max) = fn->eax;
+			cpuid_hyp_range_max = fn->eax;
 		else if (fn->eax_in == 0x80000000)
-			RIP_REL_REF(cpuid_ext_range_max) = fn->eax;
+			cpuid_ext_range_max = fn->eax;
 	}
 }
 
@@ -1229,11 +1229,7 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,
 {
 	int ret;
 
-	/*
-	 * This can be called very early during boot, so use rIP-relative
-	 * references as needed.
-	 */
-	if (RIP_REL_REF(snp_vmpl)) {
+	if (snp_vmpl) {
 		svsm_pval_4k_page(paddr, validate);
 	} else {
 		ret = pvalidate(vaddr, RMP_PG_SIZE_4K, validate);
@@ -1377,7 +1373,7 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info)
 	if (!secrets_page->svsm_guest_vmpl)
 		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_SVSM_VMPL0);
 
-	RIP_REL_REF(snp_vmpl) = secrets_page->svsm_guest_vmpl;
+	snp_vmpl = secrets_page->svsm_guest_vmpl;
 
 	caa = secrets_page->svsm_caa;
 
@@ -1392,8 +1388,8 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info)
 	 * The CA is identity mapped when this routine is called, both by the
 	 * decompressor code and the early kernel code.
 	 */
-	RIP_REL_REF(boot_svsm_caa) = (struct svsm_ca *)caa;
-	RIP_REL_REF(boot_svsm_caa_pa) = caa;
+	boot_svsm_caa = (struct svsm_ca *)caa;
+	boot_svsm_caa_pa = caa;
 
 	/* Advertise the SVSM presence via CPUID. */
 	cpuid_table = (struct snp_cpuid_table *)snp_cpuid_get_table();
arch/x86/boot/startup/sev-startup.c +8 -8
index 10b636009d1c..e376a340b629 100644
@@ -467,10 +467,10 @@ int svsm_perform_call_protocol(struct svsm_call *call)
 	 * ghcbs_initialized is set, then it is late in the boot and no need
 	 * to worry about rip-relative references in called functions.
 	 */
-	if (RIP_REL_REF(sev_cfg).ghcbs_initialized)
+	if (sev_cfg.ghcbs_initialized)
 		ghcb = __sev_get_ghcb(&state);
-	else if (RIP_REL_REF(boot_ghcb))
-		ghcb = RIP_REL_REF(boot_ghcb);
+	else if (boot_ghcb)
+		ghcb = boot_ghcb;
 	else
 		ghcb = NULL;
 
@@ -479,7 +479,7 @@ int svsm_perform_call_protocol(struct svsm_call *call)
 			   : svsm_perform_msr_protocol(call);
 	} while (ret == -EAGAIN);
 
-	if (RIP_REL_REF(sev_cfg).ghcbs_initialized)
+	if (sev_cfg.ghcbs_initialized)
 		__sev_put_ghcb(&state);
 
 	native_local_irq_restore(flags);
@@ -542,7 +542,7 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long padd
 	 * This eliminates worries about jump tables or checking boot_cpu_data
 	 * in the cc_platform_has() function.
 	 */
-	if (!(RIP_REL_REF(sev_status) & MSR_AMD64_SEV_SNP_ENABLED))
+	if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED))
 		return;
 
 	 /*
@@ -561,7 +561,7 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr
 	 * This eliminates worries about jump tables or checking boot_cpu_data
 	 * in the cc_platform_has() function.
 	 */
-	if (!(RIP_REL_REF(sev_status) & MSR_AMD64_SEV_SNP_ENABLED))
+	if (!(sev_status & MSR_AMD64_SEV_SNP_ENABLED))
 		return;
 
 	 /* Ask hypervisor to mark the memory pages shared in the RMP table. */
@@ -1356,8 +1356,8 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info)
 	if (ret)
 		sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_SVSM_CA_REMAP_FAIL);
 
-	RIP_REL_REF(boot_svsm_caa) = (struct svsm_ca *)pa;
-	RIP_REL_REF(boot_svsm_caa_pa) = pa;
+	boot_svsm_caa = (struct svsm_ca *)pa;
+	boot_svsm_caa_pa = pa;
 }
 
 bool __head snp_init(struct boot_params *bp)
arch/x86/include/asm/sev-internal.h +4 -14
index 73cb774c3639..e54847a69107 100644
@@ -68,28 +68,18 @@ extern u64 boot_svsm_caa_pa;
 
 static __always_inline struct svsm_ca *svsm_get_caa(void)
 {
-	/*
-	 * Use rIP-relative references when called early in the boot. If
-	 * ->use_cas is set, then it is late in the boot and no need
-	 * to worry about rIP-relative references.
-	 */
-	if (RIP_REL_REF(sev_cfg).use_cas)
+	if (sev_cfg.use_cas)
 		return this_cpu_read(svsm_caa);
 	else
-		return RIP_REL_REF(boot_svsm_caa);
+		return boot_svsm_caa;
 }
 
 static __always_inline u64 svsm_get_caa_pa(void)
 {
-	/*
-	 * Use rIP-relative references when called early in the boot. If
-	 * ->use_cas is set, then it is late in the boot and no need
-	 * to worry about rIP-relative references.
-	 */
-	if (RIP_REL_REF(sev_cfg).use_cas)
+	if (sev_cfg.use_cas)
 		return this_cpu_read(svsm_caa_pa);
 	else
-		return RIP_REL_REF(boot_svsm_caa_pa);
+		return boot_svsm_caa_pa;
 }
 
 int svsm_perform_call_protocol(struct svsm_call *call);
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
#24 lore raw

PATCH v4 11/11 x86/asm: Retire RIP_REL_REF()

In reply to Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

Now that all users have been moved into startup/ where PIC codegen is used, RIP_REL_REF() is no longer needed. Remove it.

---
 arch/x86/include/asm/asm.h | 5 -----
 1 file changed, 5 deletions(-)
arch/x86/include/asm/asm.h +0 -5
index a9f07799e337..eef0771512de 100644
@@ -120,11 +120,6 @@ static __always_inline __pure void *rip_rel_ptr(void *p)
 
 	return p;
 }
-#ifndef __pic__
-#define RIP_REL_REF(var)	(*(typeof(&(var)))rip_rel_ptr(&(var)))
-#else
-#define RIP_REL_REF(var)	(var)
-#endif
 #endif
 
 /*
Signature · 1 line
2.49.0.504.g3bcea36a83-goog
maintainer
#25 lore raw
In reply to Ard Biesheuvel
Quoted 29 lines · Ard Biesheuvel

Ard Biesheuvel (11):
x86/asm: Make rip_rel_ptr() usable from fPIC code x86/boot: Move the early GDT/IDT setup code into startup/ x86/boot: Move early kernel mapping code into startup/ x86/boot: Drop RIP_REL_REF() uses from early mapping code x86/boot: Move early SME init code into startup/ x86/boot: Drop RIP_REL_REF() uses from SME startup code x86/sev: Prepare for splitting off early SEV code x86/sev: Split off startup code from core code x86/boot: Move SEV startup code into startup/ x86/boot: Drop RIP_REL_REF() uses from early SEV code x86/asm: Retire RIP_REL_REF()

 arch/x86/boot/compressed/Makefile                          |    2 +-
 arch/x86/boot/compressed/sev.c                             |   17 +-
 arch/x86/boot/startup/Makefile                             |   16 +
 arch/x86/boot/startup/gdt_idt.c                            |   84 +
 arch/x86/boot/startup/map_kernel.c                         |  225 +++
 arch/x86/{coco/sev/shared.c => boot/startup/sev-shared.c}  |  375 +----
 arch/x86/boot/startup/sev-startup.c                        | 1395 ++++++++++++++++
 arch/x86/{mm/mem_encrypt_identity.c => boot/startup/sme.c} |   19 +-
 arch/x86/coco/sev/Makefile                                 |   19 -
 arch/x86/coco/sev/core.c                                   | 1726 ++++----------------
 arch/x86/include/asm/asm.h                                 |    5 -
 arch/x86/include/asm/coco.h                                |    2 +-
 arch/x86/include/asm/mem_encrypt.h                         |    2 +-
 arch/x86/include/asm/sev-internal.h                        |  112 ++
 arch/x86/include/asm/sev.h                                 |   37 +
 arch/x86/kernel/head64.c                                   |  285 +---
Quoted 8 lines
 arch/x86/mm/Makefile                                       |    6 -
 17 files changed, 2208 insertions(+), 2119 deletions(-)
 create mode 100644 arch/x86/boot/startup/gdt_idt.c
 create mode 100644 arch/x86/boot/startup/map_kernel.c
 rename arch/x86/{coco/sev/shared.c => boot/startup/sev-shared.c} (78%)
 create mode 100644 arch/x86/boot/startup/sev-startup.c
 rename arch/x86/{mm/mem_encrypt_identity.c => boot/startup/sme.c} (97%)
 create mode 100644 arch/x86/include/asm/sev-internal.h

Looks sensible at a glance. The devil's in the detail with that stuff, ofc, so we will have to test it with as many toolchains and usage scenarios as possible.

Thx.