x86: strict separation of startup code
Subsystems (5)
v7 cover x86: strict separation of startup code
From: Ard Biesheuvel <[email protected]>
This series implements a strict separation between startup code and ordinary code, where startup code is built in a way that tolerates being invoked from the initial 1:1 mapping of memory.
The existing approach of emitting this code into .head.text and checking for absolute relocations in that section is not 100% safe, and produces diagnostics that are sometimes difficult to interpret. [0]
Instead, rely on symbol prefixes, similar to how this is implemented for the EFI stub and for the startup code in the arm64 port. This ensures that startup code can only call other startup code, unless a special symbol alias is emitted that exposes a non-startup routine to the startup code.
This is somewhat intrusive, as there are many data objects that are referenced both by startup code and by ordinary code, and an alias needs to be emitted for each of those. If startup code references anything that has not been made available to it explicitly, a build time link error will occur.
This ultimately allows the .head.text section to be dropped entirely, as it no longer has a special significance. Instead, code that only executes at boot is emitted into .init.text as it should.
The majority of changes is around early SEV code. The main issue is that its use of GHCB pages and SVSM calling areas in code that may run from both the 1:1 mapping and the kernel virtual mapping is problematic as it relies on __pa() to perform VA to PA translations, which are ambiguous in this context. Also, __pa() pulls in non-trivial instrumented code when CONFIG_DEBUG_VIRTUAL=y and so it is better to avoid VA to PA translations altogether in the startup code.
Changes since v6:
- Rebase onto latest tip/master which incorporates v6.17-rc1
- Use a descriptor struct instead of two additional function arguments in patch #8
- Clarify in commit log why the latent entropy GCC plugin needs to be disabled for startup code
- Apply Tom's Rb to patch #3
Changes since v5:
- Some cosmetic touchups to patch #1
- Fix error spotted by Tom in patch #2
- Rejig objtool 'noabs' feature using an arch hook to check for relocations that should be considered absolute, which includes all of R_X86_64_32, R_X86_64_32S and R_X86_64_64 on x86_64
Changes since v4:
- Incorporate feedback from Tom, and add a couple of RBs
- Drop patch that moved the MSR save/restore out of the early page state change helper - this is less efficient but likely negligible in practice
- Drop patch that unified the SEV-SNP hypervisor feature check, which was identified by Nikunj as the one breaking SEV-SNP boot.
Changes since RFT/v3:
- Rebase onto tip/master
- Incorporate Borislav's feedback on v3
- Switch to objtool to check for absolute references in startup code
- Remap inittext R-X when running on EFI implementations that require strict R-X/RW- separation
- Include a kbuild fix to incorporate arch/x86/boot/startup/ in the right manner
- For now, omit the LA57 changes that remove the problematic early 5-level paging checks. We can revisit this once there is agreement on the approach.
Changes since RFT/v2:
- Rebase onto tip/x86/boot and drop the patches from the previous revision that have been applied in the meantime.
- Omit the pgtable_l5_enabled() changes for now, and just expose PIC aliases for the variables in question - this can be sorted later.
- Don't use the boot SVSM calling area in snp_kexec_finish(), but pass down the correct per-CPU one to the early page state API.
- Rename arch/x86/coco/sev/sev-noinstr.o to arch/x86/coco/sev/noinstr.o
- Further reduce the amount of SEV code that needs to be constructed in a special way.
Change since RFC/v1:
- Include a major disentanglement/refactor of the SEV-SNP startup code, so that only code that really needs to run from the 1:1 mapping is included in the startup/ code
- Incorporate some early notes from Ingo
[0] https://lore.kernel.org/all/CAHk-=wj7k9nvJn6cpa3-5Ciwn2RGyE605BMkjWE4MqnvC9E92A@mail.gmail.com/
Ard Biesheuvel (22):
x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback x86/sev: Use MSR protocol for remapping SVSM calling area x86/sev: Use MSR protocol only for early SVSM PVALIDATE call x86/sev: Run RMPADJUST on SVSM calling area page to test VMPL x86/sev: Move GHCB page based HV communication out of startup code x86/sev: Avoid global variable to store virtual address of SVSM area x86/sev: Share implementation of MSR-based page state change x86/sev: Pass SVSM calling area down to early page state change API x86/sev: Use boot SVSM CA for all startup and init code x86/boot: Drop redundant RMPADJUST in SEV SVSM presence check x86/boot: Provide PIC aliases for 5-level paging related constants x86/sev: Provide PIC aliases for SEV related data objects x86/sev: Move __sev_[get|put]_ghcb() into separate noinstr object x86/sev: Export startup routines for later use
objtool: Add action to check for absence of absolute relocations x86/boot: Check startup code for absence of absolute relocations x86/boot: Revert "Reject absolute references in .head.text" x86/kbuild: Incorporate boot/startup/ via Kbuild makefile x86/boot: Create a confined code area for startup code efistub/x86: Remap inittext read-execute when needed x86/boot: Move startup code out of __head section x86/boot: Get rid of the .head.text section
arch/x86/Kbuild | 2 +
arch/x86/Makefile | 1 -
arch/x86/boot/compressed/Makefile | 2 +-
arch/x86/boot/compressed/misc.c | 2 +
arch/x86/boot/compressed/sev-handle-vc.c | 3 +
arch/x86/boot/compressed/sev.c | 120 ++------
arch/x86/boot/startup/Makefile | 22 ++
arch/x86/boot/startup/exports.h | 14 +
arch/x86/boot/startup/gdt_idt.c | 4 +-
arch/x86/boot/startup/map_kernel.c | 4 +-
arch/x86/boot/startup/sev-shared.c | 314 ++++++--------------
arch/x86/boot/startup/sev-startup.c | 205 ++-----------
arch/x86/boot/startup/sme.c | 27 +-
arch/x86/coco/sev/Makefile | 8 +-
arch/x86/coco/sev/core.c | 173 ++++++++---
arch/x86/coco/sev/{sev-nmi.c => noinstr.c} | 74 +++++
arch/x86/coco/sev/vc-handle.c | 2 +
arch/x86/coco/sev/vc-shared.c | 143 ++++++++-
arch/x86/include/asm/boot.h | 2 +
arch/x86/include/asm/init.h | 6 -
arch/x86/include/asm/setup.h | 1 +
arch/x86/include/asm/sev-internal.h | 26 +-
arch/x86/include/asm/sev.h | 23 +-
arch/x86/kernel/head64.c | 5 +-
arch/x86/kernel/head_32.S | 2 +-
arch/x86/kernel/head_64.S | 10 +-
arch/x86/kernel/vmlinux.lds.S | 9 +-
arch/x86/mm/mem_encrypt_amd.c | 6 -
arch/x86/mm/mem_encrypt_boot.S | 6 +-
arch/x86/platform/pvh/head.S | 2 +-
arch/x86/tools/relocs.c | 8 +-
drivers/firmware/efi/libstub/x86-stub.c | 4 +-
tools/objtool/arch/x86/decode.c | 12 +
tools/objtool/builtin-check.c | 2 +
tools/objtool/check.c | 47 ++-
tools/objtool/include/objtool/arch.h | 1 +
tools/objtool/include/objtool/builtin.h | 1 +
37 files changed, 671 insertions(+), 622 deletions(-)
create mode 100644 arch/x86/boot/startup/exports.h
rename arch/x86/coco/sev/{sev-nmi.c => noinstr.c} (61%)base-commit: cab48e960d35f35ff799f7aafce7a7bf7c56d8e3
Signature · 1 line
2.51.0.268.g9569e192d0-goog
PATCH v7 1/22 x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
From: Ard Biesheuvel <[email protected]>
There are two distinct callers of snp_cpuid(): one where the MSR protocol is always used, and one where the GHCB page based interface is always used.
The snp_cpuid() logic does not care about the distinction, which only matters at a lower level. But the fact that it supports both interfaces means that the GHCB page based logic is pulled into the early startup code where PA to VA conversions are problematic, given that it runs from the 1:1 mapping of memory.
So keep snp_cpuid() itself in the startup code, but factor out the hypervisor calls via a callback, so that the GHCB page handling can be moved out.
Code refactoring only - no functional change intended.
--- arch/x86/boot/startup/sev-shared.c | 59 ++++---------------- arch/x86/coco/sev/vc-shared.c | 49 +++++++++++++++- arch/x86/include/asm/sev.h | 3 +- 3 files changed, 61 insertions(+), 50 deletions(-)
arch/x86/boot/startup/sev-shared.c +11 -48
@@ -342,44 +342,7 @@ static int __sev_cpuid_hv_msr(struct cpuid_leaf *leaf)return ret; } -static int __sev_cpuid_hv_ghcb (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) -{ -u32 cr4 =native_read_cr4 (); -int ret; - -ghcb_set_rax (ghcb, leaf->fn); -ghcb_set_rcx (ghcb, leaf->subfn); - -if (cr4 &X86_CR4_OSXSAVE ) -/* Safe to read xcr0 */ -ghcb_set_xcr0 (ghcb,xgetbv (XCR_XFEATURE_ENABLED_MASK )); -else -/* xgetbv will cause #UD - use reset value for xcr0 */ -ghcb_set_xcr0 (ghcb,1 ); - - ret =sev_es_ghcb_hv_call (ghcb, ctxt,SVM_EXIT_CPUID ,0 ,0 ); -if (ret !=ES_OK ) -return ret; - -if (!(ghcb_rax_is_valid (ghcb) && -ghcb_rbx_is_valid (ghcb) && -ghcb_rcx_is_valid (ghcb) && -ghcb_rdx_is_valid (ghcb))) -return ES_VMM_ERROR ; - leaf->eax = ghcb->save.rax; - leaf->ebx = ghcb->save.rbx; - leaf->ecx = ghcb->save.rcx; - leaf->edx = ghcb->save.rdx; - -return ES_OK ; -} - -static int sev_cpuid_hv (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) -{ -return ghcb ?__sev_cpuid_hv_ghcb (ghcb, ctxt, leaf) - :__sev_cpuid_hv_msr (leaf); -}/* * This may be called early while still running on the initial identity
@@ -484,21 +447,21 @@ snp_cpuid_get_validated_func(struct cpuid_leaf *leaf)return false; } -static void snp_cpuid_hv (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) +static void snp_cpuid_hv_msr (void *ctx,struct cpuid_leaf *leaf) { -if (sev_cpuid_hv (ghcb, ctxt, leaf)) +if (__sev_cpuid_hv_msr (leaf))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_CPUID_HV ); } static int __head -snp_cpuid_postprocess (struct ghcb *ghcb,struct es_em_ctxt *ctxt, -struct cpuid_leaf *leaf) +snp_cpuid_postprocess (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf) {struct cpuid_leaf leaf_hv = *leaf;switch (leaf->fn) {case 0x1 : -snp_cpuid_hv (ghcb, ctxt, &leaf_hv); +cpuid_fn (ctx, &leaf_hv);/* initial APIC ID */ leaf->ebx = (leaf_hv.ebx &GENMASK (31 ,24 )) | (leaf->ebx &GENMASK (23 ,0 ));
@@ -517,7 +480,7 @@ snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt,break ;case 0xB : leaf_hv.subfn =0 ; -snp_cpuid_hv (ghcb, ctxt, &leaf_hv); +cpuid_fn (ctx, &leaf_hv);/* extended APIC ID */ leaf->edx = leaf_hv.edx;
@@ -565,7 +528,7 @@ snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt, }break ;case 0x8000001E : -snp_cpuid_hv (ghcb, ctxt, &leaf_hv); +cpuid_fn (ctx, &leaf_hv);/* extended APIC ID */ leaf->eax = leaf_hv.eax;
@@ -586,8 +549,8 @@ snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt,* Returns -EOPNOTSUPP if feature not enabled. Any other non-zero return value * should be treated as fatal by caller. */ -int __head -snp_cpuid (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) +int __head snp_cpuid(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();
@@ -621,7 +584,7 @@ snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf)return 0 ; } -return snp_cpuid_postprocess (ghcb, ctxt, leaf); +return snp_cpuid_postprocess (cpuid_fn, ctx, leaf); }/*
@@ -648,7 +611,7 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret =snp_cpuid (NULL ,NULL , &leaf); + ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf);if (!ret) goto cpuid_done;
arch/x86/coco/sev/vc-shared.c +48 -1
63 lines not on this page · raw message
arch/x86/include/asm/sev.h +2 -1
@@ -552,7 +552,8 @@ struct cpuid_leaf {u32 edx; }; -int snp_cpuid (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf); +int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf);void __noreturn sev_es_terminate (unsigned int set,unsigned int reason); enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
Signature · 1 line
2.51.0.268.g9569e192d0-goog
Borislav Petkov
maintainerRe: [PATCH v7 01/22] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
+ Joerg and Mike to doublecheck me.
Review void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) · line 617
Quoted 9 lines · Ard Biesheuvel
(quoted diff) +1
-1@@ -648,7 +611,7 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret = snp_cpuid(NULL, NULL, &leaf); + ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;
So this code becomes now:
---
ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf);
if (!ret)
goto cpuid_done;<--- tries to find the CPUID leaf in the CPUID table
<--- otherwise uses the MSR protocol to read CPUID from HV and massage it
if (ret != -EOPNOTSUPP)
goto fail;
if (__sev_cpuid_hv_msr(&leaf))
goto fail;<--- and now it tries to do the same - do CPUID over MSR protocol.
This flow made sense before your change because it'll try to use the GHCB protocol but you're zapping that now so, IOW, you can zap that second call too:
arch/x86/boot/startup/sev-shared.c +1 -8
@@ -612,16 +612,9 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.subfn = subfn; ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf); -if (!ret) - goto cpuid_done; - -if (ret != -EOPNOTSUPP ) - goto fail; - -if (__sev_cpuid_hv_msr (&leaf)) +if (ret && ret != -EOPNOTSUPP ) goto fail; -cpuid_done: regs->ax = leaf.eax; regs->bx = leaf.ebx; regs->cx = leaf.ecx;
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 01/22] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
Quoted 50 lines · Borislav Petkov
+ Joerg and Mike to doublecheck me.
Review void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) · line 617
Quoted 9 lines · Ard Biesheuvel
(quoted diff) +1
-1@@ -648,7 +611,7 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret = snp_cpuid(NULL, NULL, &leaf); + ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;So this code becomes now:
---
ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;<--- tries to find the CPUID leaf in the CPUID table
<--- otherwise uses the MSR protocol to read CPUID from HV and massage itif (ret != -EOPNOTSUPP) goto fail; if (__sev_cpuid_hv_msr(&leaf)) goto fail;<--- and now it tries to do the same - do CPUID over MSR protocol.
This flow made sense before your change because it'll try to use the GHCB protocol but you're zapping that now so, IOW, you can zap that second call too:
arch/x86/boot/startup/sev-shared.c +0
-4@@ -612,16 +612,9 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.subfn = subfn; ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf); -if (!ret) - goto cpuid_done; - -if (ret != -EOPNOTSUPP )
arch/x86/boot/startup/sev-shared.c void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) · line 620
Quoted 10 lines
arch/x86/boot/startup/sev-shared.c +1
-4- goto fail; - -if (__sev_cpuid_hv_msr (&leaf)) +if (ret && ret != -EOPNOTSUPP ) goto fail; -cpuid_done: regs->ax = leaf.eax; regs->bx = leaf.ebx; regs->cx = leaf.ecx;
This seems plausible but I'm not sure I understand 100% why this fallback logic was introduced in the first place, so I'll defer to the experts here.
Re: [PATCH v7 01/22] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
Quoted 56 lines · Ard Biesheuvel
Quoted 55 lines · Borislav Petkov
+ Joerg and Mike to doublecheck me.
Review void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) · line 617
Quoted 9 lines · Ard Biesheuvel
(quoted diff) +1
-1@@ -648,7 +611,7 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret = snp_cpuid(NULL, NULL, &leaf); + ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;So this code becomes now:
---
ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;<--- tries to find the CPUID leaf in the CPUID table
<--- otherwise uses the MSR protocol to read CPUID from HV and massage itif (ret != -EOPNOTSUPP) goto fail; if (__sev_cpuid_hv_msr(&leaf)) goto fail;<--- and now it tries to do the same - do CPUID over MSR protocol.
This flow made sense before your change because it'll try to use the GHCB protocol but you're zapping that now so, IOW, you can zap that second call too:
arch/x86/boot/startup/sev-shared.c +1
-7@@ -612,16 +612,9 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.subfn = subfn; ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf); -if (!ret) - goto cpuid_done; - -if (ret != -EOPNOTSUPP ) - goto fail; - -if (__sev_cpuid_hv_msr (&leaf)) +if (ret && ret != -EOPNOTSUPP ) goto fail;
Quoted 9 lines
arch/x86/boot/startup/sev-shared.c void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) · line 620
Quoted 5 lines
arch/x86/boot/startup/sev-shared.c +0
-1-cpuid_done: regs->ax = leaf.eax; regs->bx = leaf.ebx; regs->cx = leaf.ecx;This seems plausible but I'm not sure I understand 100% why this fallback logic was introduced in the first place, so I'll defer to the experts here.
The fallback logic is for supporting SNP guests (which will have a CPUID table) and SEV-ES guest (which won't have a CPUID table).
Thanks,
Tom
Re: [PATCH v7 01/22] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
In reply to Borislav PetkovQuoted 15 lines · Borislav Petkov
+ Joerg and Mike to doublecheck me.
Review void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) · line 617
Quoted 9 lines · Ard Biesheuvel
(quoted diff) +1
-1@@ -648,7 +611,7 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret = snp_cpuid(NULL, NULL, &leaf); + ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;So this code becomes now:
Quoted 7 lines
---
ret = snp_cpuid(snp_cpuid_hv_msr, NULL, &leaf); if (!ret) goto cpuid_done;<--- tries to find the CPUID leaf in the CPUID table
<--- otherwise uses the MSR protocol to read CPUID from HV and massage it
It only uses the MSR protocol for particular CPUID values in snp_cpuid_postprocess(). If the CPUID leaf isn't in the CPUID table, then it will set the CPUID values to all 0 and then call the post-processing routine which may or may not call the HV.
The second call to __sev_cpuid_hv_msr() only happens if there is no CPUID table - which will be the case for SEV-ES. So you can't remove the second call.
Thanks,
Tom
Quoted 37 lines
if (ret != -EOPNOTSUPP) goto fail; if (__sev_cpuid_hv_msr(&leaf)) goto fail;<--- and now it tries to do the same - do CPUID over MSR protocol.
This flow made sense before your change because it'll try to use the GHCB protocol but you're zapping that now so, IOW, you can zap that second call too:
arch/x86/boot/startup/sev-shared.c +1
-8@@ -612,16 +612,9 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.subfn = subfn; ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf); -if (!ret) - goto cpuid_done; - -if (ret != -EOPNOTSUPP ) - goto fail; - -if (__sev_cpuid_hv_msr (&leaf)) +if (ret && ret != -EOPNOTSUPP ) goto fail; -cpuid_done: regs->ax = leaf.eax; regs->bx = leaf.ebx; regs->cx = leaf.ecx;
Borislav Petkov
maintainerRe: [PATCH v7 01/22] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
Quoted 8 lines · Tom Lendacky
It only uses the MSR protocol for particular CPUID values in
snp_cpuid_postprocess(). If the CPUID leaf isn't in the CPUID table, then it will set the CPUID values to all 0 and then call the post-processing routine which may or may not call the HV.The second call to
__sev_cpuid_hv_msr()only happens if there is no CPUID table - which will be the case for SEV-ES. So you can't remove the second call.
This needs to be turned into a proper comment, at least, and stuck above it as the situation there is clear as mud. Especially after the dropping of the GHCB protocol call, which makes the confusion even more probable...
I'll do it tomorrow if you don't beat me to it today. :)
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH v7 01/22] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
Quoted 10 lines · Borislav Petkov
Quoted 8 lines · Tom Lendacky
It only uses the MSR protocol for particular CPUID values in
snp_cpuid_postprocess(). If the CPUID leaf isn't in the CPUID table, then it will set the CPUID values to all 0 and then call the post-processing routine which may or may not call the HV.The second call to
__sev_cpuid_hv_msr()only happens if there is no CPUID table - which will be the case for SEV-ES. So you can't remove the second call.
Quoted 5 lines
This needs to be turned into a proper comment, at least, and stuck above it as the situation there is clear as mud. Especially after the dropping of the GHCB protocol call, which makes the confusion even more probable...
I'll do it tomorrow if you don't beat me to it today. :)
Something like this?:
arch/x86/boot/startup/sev-shared.c +11 -0
@@ -458,6 +458,13 @@ void do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; +/* +* If SNP is active, then snp_cpuid() uses the CPUID table to obtain the +* CPUID values (with possible HV interaction during post-processing of +* the values). But if SNP is not active (no CPUID table present), then +* snp_cpuid() returns -EOPNOTSUPP so that an SEV-ES guest can call the +* HV to obtain the CPUID information. +*/ ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf);if (!ret) goto cpuid_done;
@@ -465,6 +472,10 @@ void do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)if (ret != -EOPNOTSUPP ) goto fail; +/* +* If we got here, we're an SEV-ES guest and need to invoke the HV for +* the CPUID data. +*/ if (__sev_cpuid_hv_msr (&leaf)) goto fail;
Quoted 1 line
[tip: x86/sev] x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
In reply to Ard BiesheuvelThe following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: e2e29752357f32feb69a68e9e6e7361405b3f289 Gitweb: https://git.kernel.org/tip/e2e29752357f32feb69a68e9e6e7361405b3f289 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:04 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Thu, 28 Aug 2025 16:52:05 +02:00
x86/sev: Separate MSR and GHCB based snp_cpuid() via a callback
There are two distinct callers of snp_cpuid(): the MSR protocol and the GHCB page based interface.
The snp_cpuid() logic does not care about the distinction, which only matters at a lower level. But the fact that it supports both interfaces means that the GHCB page based logic is pulled into the early startup code where PA to VA conversions are problematic, given that it runs from the 1:1 mapping of memory.
So keep snp_cpuid() itself in the startup code, but factor out the hypervisor calls via a callback, so that the GHCB page handling can be moved out.
Code refactoring only - no functional change intended.
--- arch/x86/boot/startup/sev-shared.c | 59 +++++------------------------ arch/x86/coco/sev/vc-shared.c | 49 +++++++++++++++++++++++- arch/x86/include/asm/sev.h | 3 +- 3 files changed, 61 insertions(+), 50 deletions(-)
arch/x86/boot/startup/sev-shared.c +11 -48
@@ -342,44 +342,7 @@ static int __sev_cpuid_hv_msr(struct cpuid_leaf *leaf)return ret; } -static int __sev_cpuid_hv_ghcb (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) -{ -u32 cr4 =native_read_cr4 (); -int ret; - -ghcb_set_rax (ghcb, leaf->fn); -ghcb_set_rcx (ghcb, leaf->subfn); - -if (cr4 &X86_CR4_OSXSAVE ) -/* Safe to read xcr0 */ -ghcb_set_xcr0 (ghcb,xgetbv (XCR_XFEATURE_ENABLED_MASK )); -else -/* xgetbv will cause #UD - use reset value for xcr0 */ -ghcb_set_xcr0 (ghcb,1 ); - - ret =sev_es_ghcb_hv_call (ghcb, ctxt,SVM_EXIT_CPUID ,0 ,0 ); -if (ret !=ES_OK ) -return ret; - -if (!(ghcb_rax_is_valid (ghcb) && -ghcb_rbx_is_valid (ghcb) && -ghcb_rcx_is_valid (ghcb) && -ghcb_rdx_is_valid (ghcb))) -return ES_VMM_ERROR ; - leaf->eax = ghcb->save.rax; - leaf->ebx = ghcb->save.rbx; - leaf->ecx = ghcb->save.rcx; - leaf->edx = ghcb->save.rdx; - -return ES_OK ; -} - -static int sev_cpuid_hv (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) -{ -return ghcb ?__sev_cpuid_hv_ghcb (ghcb, ctxt, leaf) - :__sev_cpuid_hv_msr (leaf); -}/* * This may be called early while still running on the initial identity
@@ -484,21 +447,21 @@ snp_cpuid_get_validated_func(struct cpuid_leaf *leaf)return false; } -static void snp_cpuid_hv (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) +static void snp_cpuid_hv_msr (void *ctx,struct cpuid_leaf *leaf) { -if (sev_cpuid_hv (ghcb, ctxt, leaf)) +if (__sev_cpuid_hv_msr (leaf))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_CPUID_HV ); } static int __head -snp_cpuid_postprocess (struct ghcb *ghcb,struct es_em_ctxt *ctxt, -struct cpuid_leaf *leaf) +snp_cpuid_postprocess (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf) {struct cpuid_leaf leaf_hv = *leaf;switch (leaf->fn) {case 0x1 : -snp_cpuid_hv (ghcb, ctxt, &leaf_hv); +cpuid_fn (ctx, &leaf_hv);/* initial APIC ID */ leaf->ebx = (leaf_hv.ebx &GENMASK (31 ,24 )) | (leaf->ebx &GENMASK (23 ,0 ));
@@ -517,7 +480,7 @@ snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt,break ;case 0xB : leaf_hv.subfn =0 ; -snp_cpuid_hv (ghcb, ctxt, &leaf_hv); +cpuid_fn (ctx, &leaf_hv);/* extended APIC ID */ leaf->edx = leaf_hv.edx;
@@ -565,7 +528,7 @@ snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt, }break ;case 0x8000001E : -snp_cpuid_hv (ghcb, ctxt, &leaf_hv); +cpuid_fn (ctx, &leaf_hv);/* extended APIC ID */ leaf->eax = leaf_hv.eax;
@@ -586,8 +549,8 @@ snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt,* Returns -EOPNOTSUPP if feature not enabled. Any other non-zero return value * should be treated as fatal by caller. */ -int __head -snp_cpuid (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf) +int __head snp_cpuid(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();
@@ -621,7 +584,7 @@ snp_cpuid(struct ghcb *ghcb, struct es_em_ctxt *ctxt, struct cpuid_leaf *leaf)return 0 ; } -return snp_cpuid_postprocess (ghcb, ctxt, leaf); +return snp_cpuid_postprocess (cpuid_fn, ctx, leaf); }/*
@@ -648,7 +611,7 @@ void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) leaf.fn = fn; leaf.subfn = subfn; - ret =snp_cpuid (NULL ,NULL , &leaf); + ret =snp_cpuid (snp_cpuid_hv_msr,NULL , &leaf);if (!ret) goto cpuid_done;
arch/x86/coco/sev/vc-shared.c +48 -1
63 lines not on this page · raw message
arch/x86/include/asm/sev.h +2 -1
@@ -552,7 +552,8 @@ struct cpuid_leaf {u32 edx; }; -int snp_cpuid (struct ghcb *ghcb,struct es_em_ctxt *ctxt,struct cpuid_leaf *leaf); +int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf);void __noreturn sev_es_terminate (unsigned int set,unsigned int reason); enum es_result sev_es_ghcb_hv_call(struct ghcb *ghcb,
PATCH v7 2/22 x86/sev: Use MSR protocol for remapping SVSM calling area
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
As the preceding code comment already indicates, remapping the SVSM calling area occurs long before the GHCB page is configured, and so calling svsm_perform_call_protocol() is guaranteed to result in a call to svsm_perform_msr_protocol().
So just call the latter directly. This allows most of the GHCB based API infrastructure to be moved out of the startup code in a subsequent patch.
--- arch/x86/boot/startup/sev-shared.c | 11 +++++++++++ arch/x86/boot/startup/sev-startup.c | 5 ++--- 2 files changed, 13 insertions(+), 3 deletions(-)
arch/x86/boot/startup/sev-shared.c +11 -0
@@ -724,6 +724,17 @@ static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info) } } +static int __head svsm_call_msr_protocol(struct svsm_call *call) +{ +int ret; + +do { + ret =svsm_perform_msr_protocol (call); + }while (ret == -EAGAIN ); + +return ret; +} + static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) {struct svsm_pvalidate_call *pc;
arch/x86/boot/startup/sev-startup.c +2 -3
@@ -295,7 +295,6 @@ static __head struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) static __head void svsm_setup(struct cc_blob_sev_info *cc_info) {struct svsm_call call = {}; -int ret;u64 pa;/*
@@ -325,8 +324,8 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info) call.caa =svsm_get_caa (); call.rax =SVSM_CORE_CALL (SVSM_CORE_REMAP_CA ); call.rcx = pa; - ret =svsm_perform_call_protocol (&call); -if (ret) + +if (svsm_call_msr_protocol (&call))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SVSM_CA_REMAP_FAIL ); boot_svsm_caa = (struct svsm_ca *)pa;
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Use MSR protocol for remapping SVSM calling area
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: c15a4705d59caeb44f4c373cf04e89041309e568 Gitweb: https://git.kernel.org/tip/c15a4705d59caeb44f4c373cf04e89041309e568 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:05 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sun, 31 Aug 2025 12:40:55 +02:00
x86/sev: Use MSR protocol for remapping SVSM calling area
As the preceding code comment already indicates, remapping the SVSM calling area occurs long before the GHCB page is configured, and so calling svsm_perform_call_protocol() is guaranteed to result in a call to svsm_perform_msr_protocol().
So just call the latter directly. This allows most of the GHCB based API infrastructure to be moved out of the startup code in a subsequent patch.
--- arch/x86/boot/startup/sev-shared.c | 11 +++++++++++ arch/x86/boot/startup/sev-startup.c | 5 ++--- 2 files changed, 13 insertions(+), 3 deletions(-)
arch/x86/boot/startup/sev-shared.c +11 -0
@@ -724,6 +724,17 @@ static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info) } } +static int __head svsm_call_msr_protocol(struct svsm_call *call) +{ +int ret; + +do { + ret =svsm_perform_msr_protocol (call); + }while (ret == -EAGAIN ); + +return ret; +} + static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) {struct svsm_pvalidate_call *pc;
arch/x86/boot/startup/sev-startup.c +2 -3
@@ -295,7 +295,6 @@ found_cc_info: static __head void svsm_setup(struct cc_blob_sev_info *cc_info) {struct svsm_call call = {}; -int ret;u64 pa;/*
@@ -325,8 +324,8 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info) call.caa =svsm_get_caa (); call.rax =SVSM_CORE_CALL (SVSM_CORE_REMAP_CA ); call.rcx = pa; - ret =svsm_perform_call_protocol (&call); -if (ret) + +if (svsm_call_msr_protocol (&call))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SVSM_CA_REMAP_FAIL ); boot_svsm_caa = (struct svsm_ca *)pa;
PATCH v7 3/22 x86/sev: Use MSR protocol only for early SVSM PVALIDATE call
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The early page state change API performs an SVSM call to PVALIDATE each page when running under a SVSM, and this involves either a GHCB page based call or a call based on the MSR protocol.
The GHCB page based variant involves VA to PA translation of the GHCB address, and this is best avoided in the startup code, where virtual addresses are ambiguous (1:1 or kernel virtual).
As this is the last remaining occurrence of svsm_perform_call_protocol() in the startup code, switch to the MSR protocol exclusively in this particular case, so that the GHCB based plumbing can be moved out of the startup code entirely in a subsequent patch.
--- arch/x86/boot/compressed/sev.c | 20 -------------------- arch/x86/boot/startup/sev-shared.c | 9 ++++++--- 2 files changed, 6 insertions(+), 23 deletions(-)
arch/x86/boot/compressed/sev.c +0 -20
@@ -50,31 +50,11 @@ u64 svsm_get_caa_pa(void)return boot_svsm_caa_pa; } -int svsm_perform_call_protocol (struct svsm_call *call); -u8 snp_vmpl;/* Include code for early handlers */ #include "../../boot/startup/sev-shared.c" -int svsm_perform_call_protocol (struct svsm_call *call) -{ -struct ghcb *ghcb; -int ret; - -if (boot_ghcb) - ghcb = boot_ghcb; -else - ghcb =NULL ; - -do { - ret = ghcb ?svsm_perform_ghcb_protocol (ghcb, call) - :svsm_perform_msr_protocol (call); - }while (ret == -EAGAIN ); - -return ret; -} -static bool sev_snp_enabled (void ) {return sev_status &MSR_AMD64_SEV_SNP_ENABLED ;
arch/x86/boot/startup/sev-shared.c +6 -3
@@ -741,7 +741,6 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate)struct svsm_call call = {};unsigned long flags;u64 pc_pa; -int ret;/* * This can be called very early in the boot, use native functions in
@@ -766,8 +765,12 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) call.rax =SVSM_CORE_CALL (SVSM_CORE_PVALIDATE ); call.rcx = pc_pa; - ret =svsm_perform_call_protocol (&call); -if (ret) +/* +* Use the MSR protocol exclusively, so that this code is usable in +* startup code where VA/PA translations of the GHCB page's address may +* be problematic. +*/ +if (svsm_call_msr_protocol (&call))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PVALIDATE );native_local_irq_restore (flags);
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Use MSR protocol only for early SVSM PVALIDATE call
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 7cb7b6de9cb90311a917d65c0228b6aa223dc456 Gitweb: https://git.kernel.org/tip/7cb7b6de9cb90311a917d65c0228b6aa223dc456 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:06 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sun, 31 Aug 2025 12:40:55 +02:00
x86/sev: Use MSR protocol only for early SVSM PVALIDATE call
The early page state change API performs an SVSM call to PVALIDATE each page when running under a SVSM, and this involves either a GHCB page based call or a call based on the MSR protocol.
The GHCB page based variant involves VA to PA translation of the GHCB address, and this is best avoided in the startup code, where virtual addresses are ambiguous (1:1 or kernel virtual).
As this is the last remaining occurrence of svsm_perform_call_protocol() in the startup code, switch to the MSR protocol exclusively in this particular case, so that the GHCB based plumbing can be moved out of the startup code entirely in a subsequent patch.
--- arch/x86/boot/compressed/sev.c | 20 -------------------- arch/x86/boot/startup/sev-shared.c | 9 ++++++--- 2 files changed, 6 insertions(+), 23 deletions(-)
arch/x86/boot/compressed/sev.c +0 -20
@@ -50,31 +50,11 @@ u64 svsm_get_caa_pa(void)return boot_svsm_caa_pa; } -int svsm_perform_call_protocol (struct svsm_call *call); -u8 snp_vmpl;/* Include code for early handlers */ #include "../../boot/startup/sev-shared.c" -int svsm_perform_call_protocol (struct svsm_call *call) -{ -struct ghcb *ghcb; -int ret; - -if (boot_ghcb) - ghcb = boot_ghcb; -else - ghcb =NULL ; - -do { - ret = ghcb ?svsm_perform_ghcb_protocol (ghcb, call) - :svsm_perform_msr_protocol (call); - }while (ret == -EAGAIN ); - -return ret; -} -static bool sev_snp_enabled (void ) {return sev_status &MSR_AMD64_SEV_SNP_ENABLED ;
arch/x86/boot/startup/sev-shared.c +6 -3
@@ -741,7 +741,6 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate)struct svsm_call call = {};unsigned long flags;u64 pc_pa; -int ret;/* * This can be called very early in the boot, use native functions in
@@ -766,8 +765,12 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) call.rax =SVSM_CORE_CALL (SVSM_CORE_PVALIDATE ); call.rcx = pc_pa; - ret =svsm_perform_call_protocol (&call); -if (ret) +/* +* Use the MSR protocol exclusively, so that this code is usable in +* startup code where VA/PA translations of the GHCB page's address may +* be problematic. +*/ +if (svsm_call_msr_protocol (&call))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PVALIDATE );native_local_irq_restore (flags);
PATCH v7 4/22 x86/sev: Run RMPADJUST on SVSM calling area page to test VMPL
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Determining the VMPL at which the kernel runs involves performing a RMPADJUST operation on an arbitrary page of memory, and observing whether it succeeds.
The use of boot_ghcb_page in the core kernel in this case is completely arbitrary, but results in the need to provide a PIC alias for it. So use boot_svsm_ca_page instead, which already needs this alias for other reasons.
--- arch/x86/boot/compressed/sev.c | 2 +- arch/x86/boot/startup/sev-shared.c | 5 +++-- arch/x86/boot/startup/sev-startup.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-)
arch/x86/boot/compressed/sev.c +1 -1
@@ -327,7 +327,7 @@ static bool early_snp_init(struct boot_params *bp)* running at VMPL0. The CA will be used to communicate with the * SVSM and request its services. */ -svsm_setup_ca (cc_info); +svsm_setup_ca (cc_info,rip_rel_ptr (&boot_ghcb_page));/* * Pass run-time kernel a pointer to CC info via boot_params so EFI
arch/x86/boot/startup/sev-shared.c +3 -2
@@ -801,7 +801,8 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,* Maintain the GPA of the SVSM Calling Area (CA) in order to utilize the SVSM * services needed when not running in VMPL0. */ -static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info) +static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info, +void *page) {struct snp_secrets_page *secrets_page;struct snp_cpuid_table *cpuid_table;
@@ -824,7 +825,7 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info)* routine is running identity mapped when called, both by the decompressor * code and the early kernel code. */ -if (!rmpadjust ((unsigned long )rip_rel_ptr (&boot_ghcb_page),RMP_PG_SIZE_4K ,1 )) +if (!rmpadjust ((unsigned long )page,RMP_PG_SIZE_4K ,1 ))return false;/*
arch/x86/boot/startup/sev-startup.c +1 -1
@@ -302,7 +302,7 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info)* running at VMPL0. The CA will be used to communicate with the * SVSM to perform the SVSM services. */ -if (!svsm_setup_ca (cc_info)) +if (!svsm_setup_ca (cc_info,rip_rel_ptr (&boot_svsm_ca_page)))return ;/*
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Run RMPADJUST on SVSM calling area page to test VMPL
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: e349241b97a8b1169a4e90375159df4d22061f9a Gitweb: https://git.kernel.org/tip/e349241b97a8b1169a4e90375159df4d22061f9a Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:07 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Sun, 31 Aug 2025 12:40:56 +02:00
x86/sev: Run RMPADJUST on SVSM calling area page to test VMPL
Determining the VMPL at which the kernel runs involves performing a RMPADJUST operation on an arbitrary page of memory, and observing whether it succeeds.
The use of boot_ghcb_page in the core kernel in this case is completely arbitrary, but results in the need to provide a PIC alias for it. So use boot_svsm_ca_page instead, which already needs this alias for other reasons.
--- arch/x86/boot/compressed/sev.c | 2 +- arch/x86/boot/startup/sev-shared.c | 5 +++-- arch/x86/boot/startup/sev-startup.c | 2 +- 3 files changed, 5 insertions(+), 4 deletions(-)
arch/x86/boot/compressed/sev.c +1 -1
@@ -327,7 +327,7 @@ static bool early_snp_init(struct boot_params *bp)* running at VMPL0. The CA will be used to communicate with the * SVSM and request its services. */ -svsm_setup_ca (cc_info); +svsm_setup_ca (cc_info,rip_rel_ptr (&boot_ghcb_page));/* * Pass run-time kernel a pointer to CC info via boot_params so EFI
arch/x86/boot/startup/sev-shared.c +3 -2
@@ -801,7 +801,8 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,* Maintain the GPA of the SVSM Calling Area (CA) in order to utilize the SVSM * services needed when not running in VMPL0. */ -static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info) +static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info, +void *page) {struct snp_secrets_page *secrets_page;struct snp_cpuid_table *cpuid_table;
@@ -824,7 +825,7 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info)* routine is running identity mapped when called, both by the decompressor * code and the early kernel code. */ -if (!rmpadjust ((unsigned long )rip_rel_ptr (&boot_ghcb_page),RMP_PG_SIZE_4K ,1 )) +if (!rmpadjust ((unsigned long )page,RMP_PG_SIZE_4K ,1 ))return false;/*
arch/x86/boot/startup/sev-startup.c +1 -1
@@ -302,7 +302,7 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info)* running at VMPL0. The CA will be used to communicate with the * SVSM to perform the SVSM services. */ -if (!svsm_setup_ca (cc_info)) +if (!svsm_setup_ca (cc_info,rip_rel_ptr (&boot_svsm_ca_page)))return ;/*
PATCH v7 5/22 x86/sev: Move GHCB page based HV communication out of startup code
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Both the decompressor and the core kernel implement an early #VC handler, which only deals with CPUID instructions, and full featured one, which can handle any #VC exception.
The former communicates with the hypervisor using the MSR based protocol, whereas the latter uses a shared GHCB page, which is configured a bit later during the boot, when the kernel runs from its ordinary virtual mapping, rather than the 1:1 mapping that the startup code uses.
Accessing this shared GHCB page from the core kernel's startup code is problematic, because it involves converting the GHCB address provided by the caller to a physical address. In the startup code, virtual to physical address translations are problematic, given that the virtual address might be a 1:1 mapped address, and such translations should therefore be avoided.
This means that exposing startup code dealing with the GHCB to callers that execute from the ordinary kernel virtual mapping should be avoided too. So move all GHCB page based communication out of the startup code, now that all communication occurring before the kernel virtual mapping is up relies on the MSR protocol only.
--- arch/x86/boot/compressed/sev-handle-vc.c | 3 + arch/x86/boot/startup/sev-shared.c | 143 +------------------- arch/x86/boot/startup/sev-startup.c | 42 ------ arch/x86/coco/sev/core.c | 76 +++++++++++ arch/x86/coco/sev/vc-handle.c | 2 + arch/x86/coco/sev/vc-shared.c | 94 +++++++++++++ arch/x86/include/asm/sev-internal.h | 7 +- arch/x86/include/asm/sev.h | 11 +- 8 files changed, 190 insertions(+), 188 deletions(-)
arch/x86/boot/compressed/sev-handle-vc.c +3 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #include "misc.h" +#include "error.h" #include "sev.h" #include <linux/kernel.h>
@@ -14,6 +15,8 @@#include <asm/fpu/xcr.h> #define __BOOT_COMPRESSED +#undef __init +#define __init/* Basic instruction decoding support needed */ #include "../../lib/inat.c"
arch/x86/boot/startup/sev-shared.c +4 -139
188 lines not on this page · raw message
arch/x86/boot/startup/sev-startup.c +0 -42
54 lines not on this page · raw message
arch/x86/coco/sev/core.c +76 -0
88 lines not on this page · raw message
arch/x86/coco/sev/vc-handle.c +2 -0
@@ -351,6 +351,8 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt, }#define sev_printk (fmt, ...) printk(fmt, ##__VA_ARGS__) +#define error (v) +#define has_cpuflag (f) boot_cpu_has(f)#include "vc-shared.c"
arch/x86/coco/sev/vc-shared.c +94 -0
103 lines not on this page · raw message
arch/x86/include/asm/sev-internal.h +2 -5
@@ -2,7 +2,6 @@#define DR7_RESET_VALUE 0x400 -extern struct ghcb boot_ghcb_page;extern u64 sev_hv_features;extern u64 sev_secrets_pa;
@@ -80,7 +79,8 @@ static __always_inline u64 svsm_get_caa_pa(void)return boot_svsm_caa_pa; } -int svsm_perform_call_protocol (struct svsm_call *call); +enum es_result verify_exception_info (struct ghcb *ghcb,struct es_em_ctxt *ctxt); +void vc_forward_exception (struct es_em_ctxt *ctxt);static inline u64 sev_es_rd_ghcb_msr (void ) {
@@ -97,9 +97,6 @@ static __always_inline void sev_es_wr_ghcb_msr(u64 val)native_wrmsr (MSR_AMD64_SEV_ES_GHCB , low, high); } -void snp_register_ghcb_early (unsigned long paddr); -bool sev_es_negotiate_protocol (void ); -bool sev_es_check_cpu_features (void );u64 get_hv_features (void );const struct snp_cpuid_table *snp_cpuid_get_table (void );
arch/x86/include/asm/sev.h +9 -2
@@ -503,6 +503,7 @@ static inline int pvalidate(unsigned long vaddr, bool rmp_psize, bool validate) }void setup_ghcb (void ); +void snp_register_ghcb_early (unsigned long paddr);void early_snp_set_memory_private (unsigned long vaddr,unsigned long paddr,unsigned long npages); void early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr,
@@ -540,8 +541,6 @@ static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)__builtin_memset (ghcb->save.valid_bitmap,0 ,sizeof (ghcb->save.valid_bitmap)); } -void vc_forward_exception (struct es_em_ctxt *ctxt); -/* I/O parameters for CPUID-related helpers */ struct cpuid_leaf {u32 fn;
@@ -552,15 +551,23 @@ struct cpuid_leaf {u32 edx; }; +int svsm_perform_msr_protocol (struct svsm_call *call);int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf),void *ctx,struct cpuid_leaf *leaf); +void svsm_issue_call (struct svsm_call *call,u8 *pending); +int svsm_process_result_codes (struct svsm_call *call); +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason);enum es_result sev_es_ghcb_hv_call (struct ghcb *ghcb,struct es_em_ctxt *ctxt,u64 exit_code,u64 exit_info_1,u64 exit_info_2); +bool sev_es_negotiate_protocol (void ); +bool sev_es_check_cpu_features (void ); + +extern u16 ghcb_version;extern struct ghcb *boot_ghcb;#else /* !CONFIG_AMD_MEM_ENCRYPT */
Signature · 1 line
2.51.0.268.g9569e192d0-goog
Borislav Petkov
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
arch/x86/boot/startup/sev-shared.c line 16
Quoted 9 lines · Ard Biesheuvel
arch/x86/boot/startup/sev-shared.c +0
-1@@ -13,12 +13,9 @@#ifndef __BOOT_COMPRESSED#define error (v) pr_err(v) -#define has_cpuflag (f) boot_cpu_has(f)
In file included from arch/x86/boot/startup/sev-startup.c:106:arch/x86/boot/startup/sev-shared.c: In function ‘pvalidate_4k_page’:arch/x86/boot/startup/sev-shared.c:661:26: error: implicit declaration of function ‘has_cpuflag’ [-Wimplicit-function-declaration]
661 | if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))
| ^~~~~~~~~~~
make[3]: *** [scripts/Makefile.build:287: arch/x86/boot/startup/sev-startup.o] Error 1 make[2]: *** [scripts/Makefile.build:556: arch/x86/boot/startup] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [/mnt/kernel/kernel/linux/Makefile:2011: .] Error 2
make: *** [Makefile:248: __sub-make] Error 2
We probably will have to use the CPUID MSR protocol thing here or so...
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 16 lines · Borislav Petkov
arch/x86/boot/startup/sev-shared.c line 16
Quoted 9 lines · Ard Biesheuvel
arch/x86/boot/startup/sev-shared.c +0
-1@@ -13,12 +13,9 @@#ifndef __BOOT_COMPRESSED#define error (v) pr_err(v) -#define has_cpuflag (f) boot_cpu_has(f)In file included from
arch/x86/boot/startup/sev-startup.c:106:arch/x86/boot/startup/sev-shared.c: In function ‘pvalidate_4k_page’:arch/x86/boot/startup/sev-shared.c:661:26: error: implicit declaration of function ‘has_cpuflag’ [-Wimplicit-function-declaration]
661 | if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))
Quoted 9 lines
| ^~~~~~~~~~~make[3]: *** [
scripts/Makefile.build:287:arch/x86/boot/startup/sev-startup.o] Error 1 make[2]: *** [scripts/Makefile.build:556:arch/x86/boot/startup] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [/mnt/kernel/kernel/linux/Makefile:2011: .] Error 2
make: *** [Makefile:248: __sub-make] Error 2We probably will have to use the CPUID MSR protocol thing here or so...
Huh I thought I had fixed that: we can just keep that has_cpuflag() definition if there is a need for it.
Ard Biesheuvel
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 25 lines · Ard Biesheuvel
Quoted 23 lines · Borislav Petkov
arch/x86/boot/startup/sev-shared.c line 16
Quoted 9 lines · Ard Biesheuvel
arch/x86/boot/startup/sev-shared.c +0
-1@@ -13,12 +13,9 @@#ifndef __BOOT_COMPRESSED#define error (v) pr_err(v) -#define has_cpuflag (f) boot_cpu_has(f)In file included from
arch/x86/boot/startup/sev-startup.c:106:arch/x86/boot/startup/sev-shared.c: In function ‘pvalidate_4k_page’:arch/x86/boot/startup/sev-shared.c:661:26: error: implicit declaration of function ‘has_cpuflag’ [-Wimplicit-function-declaration]
661 | if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))| ^~~~~~~~~~~make[3]: *** [
scripts/Makefile.build:287:arch/x86/boot/startup/sev-startup.o] Error 1 make[2]: *** [scripts/Makefile.build:556:arch/x86/boot/startup] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [/mnt/kernel/kernel/linux/Makefile:2011: .] Error 2
make: *** [Makefile:248: __sub-make] Error 2
Quoted 5 lines
Quoted 2 lines
We probably will have to use the CPUID MSR protocol thing here or so...
Huh I thought I had fixed that: we can just keep that
has_cpuflag()definition if there is a need for it.
OK it appears I've fixed it in the wrong place: the next patch adds back the definition of has_cpuflag() so I squashed that hunk into the wrong patch, it seems.
Borislav Petkov
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 3 lines · Ard Biesheuvel
OK it appears I've fixed it in the wrong place: the next patch adds back the definition of
has_cpuflag()so I squashed that hunk into the wrong patch, it seems.
The real question is - and I'm sceptical - whether the startup code runs too early for boot_cpu_has(). And how is the startup code going to call boot_cpu_has().
/me builds .s
Aha, so it gets converted into a boot_cpu_data access:
# arch/x86/boot/startup/sev-shared.c:662: if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))
testb %r13b, %r13b # validate
je .L46 #,
# ./arch/x86/include/asm/bitops.h:206: (addr[nr >> _BITOPS_LONG_SHIFT])) != 0;
movq 80+boot_cpu_data(%rip), %rax # MEM[(const volatile long unsigned int *)&boot_cpu_data + 80B], _15
# arch/x86/boot/startup/sev-shared.c:662: if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))But former question remains: AFAIK, you want to run the startup code waaay earlier, before we do identify_boot_cpu() which prepares boot_cpu_data, right?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 17 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel
OK it appears I've fixed it in the wrong place: the next patch adds back the definition of
has_cpuflag()so I squashed that hunk into the wrong patch, it seems.The real question is - and I'm sceptical - whether the startup code runs too early for
boot_cpu_has(). And how is the startup code going to callboot_cpu_has()./me builds .s
Aha, so it gets converted into a
boot_cpu_dataaccess:#
arch/x86/boot/startup/sev-shared.c:662: if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))
testb %r13b, %r13b # validate
je .L46 #,
Quoted 7 lines
# ./arch/x86/include/asm/bitops.h:206: (addr[nr >> _BITOPS_LONG_SHIFT])) != 0; movq 80+boot_cpu_data(%rip), %rax # MEM[(const volatile long unsigned int *)&boot_cpu_data + 80B], _15 # arch/x86/boot/startup/sev-shared.c:662: if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))But former question remains: AFAIK, you want to run the startup code waaay earlier, before we do
identify_boot_cpu()which preparesboot_cpu_data, right?
I suppose that in this particular case, things work out fine because calling sev_evict_cache() unnecessarily is harmless. But I agree that in general, relying on CPU flags in code that may be called this early is not great.
Perhaps this conditional should be moved into the caller instead (early_set_pages_state()), and early callers from inside the startup code should call sev_evict_cache() unconditionally?
Ard Biesheuvel
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 27 lines · Ard Biesheuvel
Quoted 17 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel
OK it appears I've fixed it in the wrong place: the next patch adds back the definition of
has_cpuflag()so I squashed that hunk into the wrong patch, it seems.The real question is - and I'm sceptical - whether the startup code runs too early for
boot_cpu_has(). And how is the startup code going to callboot_cpu_has()./me builds .s
Aha, so it gets converted into a
boot_cpu_dataaccess:#
arch/x86/boot/startup/sev-shared.c:662: if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))
testb %r13b, %r13b # validate
je .L46 #,Quoted 7 lines
# ./arch/x86/include/asm/bitops.h:206: (addr[nr >> _BITOPS_LONG_SHIFT])) != 0; movq 80+boot_cpu_data(%rip), %rax # MEM[(const volatile long unsigned int *)&boot_cpu_data + 80B], _15 # arch/x86/boot/startup/sev-shared.c:662: if (validate && !has_cpuflag(X86_FEATURE_COHERENCY_SFW_NO))But former question remains: AFAIK, you want to run the startup code waaay earlier, before we do
identify_boot_cpu()which preparesboot_cpu_data, right?
Quoted 8 lines
I suppose that in this particular case, things work out fine because calling
sev_evict_cache()unnecessarily is harmless. But I agree that in general, relying on CPU flags in code that may be called this early is not great.Perhaps this conditional should be moved into the caller instead (
early_set_pages_state()), and early callers from inside the startup code should callsev_evict_cache()unconditionally?
Alternatively, we might consider the below:
arch/x86/boot/compressed/sev.c +2 -0
@@ -342,6 +342,8 @@if (!(eax &BIT (1 )))return -ENODEV ; + sev_snp_needs_sfw = !(ebx &BIT (31 )); +return ebx &0x3f ; }
(quoted diff) +0 -0
b/arch/x86/boot/startup/sev-shared.c
index 8d2476e1ad3b..08cc1568d8af 100644
arch/x86/boot/startup/sev-shared.c +3 -1
@@ -31,6 +31,8 @@static u32 cpuid_hyp_range_max__ro_after_init ;static u32 cpuid_ext_range_max__ro_after_init ; +bool sev_snp_needs_sfw; +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason) {
@@ -639,7 +641,7 @@* If validating memory (making it private) and affected by the * cache-coherency vulnerability, perform the cache eviction mitigation. */ -if (validate && !has_cpuflag (X86_FEATURE_COHERENCY_SFW_NO )) +if (validate && sev_snp_needs_sfw)sev_evict_cache ((void *)vaddr,1 ); }
arch/x86/boot/startup/sme.c +1 -0
@@ -521,6 +521,7 @@return ; me_mask =1UL << (ebx &0x3f ); + sev_snp_needs_sfw = !(ebx &BIT (31 ));/* Check the SEV MSR whether SEV or SME is enabled */ sev_status = msr =native_rdmsrq (MSR_AMD64_SEV );
arch/x86/include/asm/sev.h +2 -0
@@ -570,6 +570,8 @@extern u16 ghcb_version;extern struct ghcb *boot_ghcb; +extern bool sev_snp_needs_sfw; +struct psc_desc {enum psc_op op;struct svsm_ca *ca;
Borislav Petkov
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 1 line · Ard Biesheuvel
Alternatively, we might consider the below:
This is the full diff ontop of this current patch - it removes the get_cpuflags() hunk too as it is not needed now. It builds fine and I'll test with it and if all looks good, merge it with your patch.
Thx.
---
arch/x86/boot/compressed/sev.c +2 -0
@@ -371,6 +371,8 @@ static int sev_check_cpu_support(void)if (!(eax &BIT (1 )))return -ENODEV ; + sev_snp_needs_sfw = !(ebx &BIT (31 )); +return ebx &0x3f ; }
arch/x86/boot/cpuflags.c +0 -13
@@ -106,18 +106,5 @@ void get_cpuflags(void)cpuid (0x80000001 , &ignored, &ignored, &cpu.flags[6 ], &cpu.flags[1 ]); } - -if (max_amd_level >=0x8000001f ) { -u32 ebx; - -/* -* The X86_FEATURE_COHERENCY_SFW_NO feature bit is in -* the virtualization flags entry (word 8) and set by -* scattered.c, so the bit needs to be explicitly set. -*/ -cpuid (0x8000001f , &ignored, &ebx, &ignored, &ignored); -if (ebx &BIT (31 )) -set_bit (X86_FEATURE_COHERENCY_SFW_NO , cpu.flags); - } } }
arch/x86/boot/startup/sev-shared.c +3 -1
@@ -51,6 +51,8 @@ static u32 cpuid_std_range_max __ro_after_init;static u32 cpuid_hyp_range_max__ro_after_init ;static u32 cpuid_ext_range_max__ro_after_init ; +bool sev_snp_needs_sfw; + void __head __noreturnsev_es_terminate (unsigned int set,unsigned int reason) {
@@ -658,7 +660,7 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,* If validating memory (making it private) and affected by the * cache-coherency vulnerability, perform the cache eviction mitigation. */ -if (validate && !has_cpuflag (X86_FEATURE_COHERENCY_SFW_NO )) +if (validate && sev_snp_needs_sfw)sev_evict_cache ((void *)vaddr,1 ); }
arch/x86/boot/startup/sme.c +1 -0
@@ -521,6 +521,7 @@ void __head sme_enable(struct boot_params *bp)return ; me_mask =1UL << (ebx &0x3f ); + sev_snp_needs_sfw = !(ebx &BIT (31 ));/* Check the SEV MSR whether SEV or SME is enabled */ sev_status = msr =native_rdmsrq (MSR_AMD64_SEV );
arch/x86/include/asm/sev.h +1 -0
@@ -569,6 +569,7 @@ bool sev_es_check_cpu_features(void);extern u16 ghcb_version;extern struct ghcb *boot_ghcb; +extern bool sev_snp_needs_sfw;#else /* !CONFIG_AMD_MEM_ENCRYPT */
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 8 lines · Borislav Petkov
Quoted 1 line · Ard Biesheuvel
Alternatively, we might consider the below:
This is the full diff ontop of this current patch - it removes the
get_cpuflags()hunk too as it is not needed now. It builds fine and I'll test with it and if all looks good, merge it with your patch.
...
Quoted 11 lines
arch/x86/boot/cpuflags.c +0
-3@@ -106,18 +106,5 @@ void get_cpuflags(void)cpuid (0x80000001 , &ignored, &ignored, &cpu.flags[6 ], &cpu.flags[1 ]); } - -if (max_amd_level >=0x8000001f ) { -u32 ebx;
arch/x86/boot/cpuflags.c void get_cpuflags(void) · line 121
Quoted 10 lines
arch/x86/boot/cpuflags.c +0
-10- -/* -* The X86_FEATURE_COHERENCY_SFW_NO feature bit is in -* the virtualization flags entry (word 8) and set by -* scattered.c, so the bit needs to be explicitly set. -*/ -cpuid (0x8000001f , &ignored, &ebx, &ignored, &ignored); -if (ebx &BIT (31 )) -set_bit (X86_FEATURE_COHERENCY_SFW_NO , cpu.flags); - }
Are you sure this can be removed? It is tested in other places too (arch/x86/coco/sev/core.c), and AIUI, it is a Linux defined CPU feature so it will never be set automatically.
Borislav Petkov
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 3 lines · Ard Biesheuvel
Are you sure this can be removed? It is tested in other places too (
arch/x86/coco/sev/core.c), and AIUI, it is a Linux defined CPU feature so it will never be set automatically.
It it set in scattered.c for the "late" code like sev/core.c:
arch/x86/kernel/cpu/scattered.c:51: { X86_FEATURE_COHERENCY_SFW_NO, CPUID_EBX, 31, 0x8000001f, 0 },
Tom added it to boot/cpuflags.c, in addition, for the early testing which we're doing differently now.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 05/22] x86/sev: Move GHCB page based HV communication out of startup code
Quoted 6 lines · Borislav Petkov
Quoted 3 lines · Ard Biesheuvel
Are you sure this can be removed? It is tested in other places too (
arch/x86/coco/sev/core.c), and AIUI, it is a Linux defined CPU feature so it will never be set automatically.
Quoted 7 lines
It it set in
scattered.cfor the "late" code likesev/core.c:
arch/x86/kernel/cpu/scattered.c:51: {X86_FEATURE_COHERENCY_SFW_NO,CPUID_EBX, 31,0x8000001f, 0 },Tom added it to
boot/cpuflags.c, in addition, for the early testing which we're doing differently now.
Ah ok, yes that makes sense.
[tip: x86/sev] x86/sev: Move GHCB page based HV communication out of startup code
In reply to Ard BiesheuvelThe following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 37dbd78f98a80e89b5413f4649d0fbd023d99b2f Gitweb: https://git.kernel.org/tip/37dbd78f98a80e89b5413f4649d0fbd023d99b2f Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:08 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:55:25 +02:00
x86/sev: Move GHCB page based HV communication out of startup code
Both the decompressor and the core kernel implement an early #VC handler, which only deals with CPUID instructions, and full featured one, which can handle any #VC exception.
The former communicates with the hypervisor using the MSR based protocol, whereas the latter uses a shared GHCB page, which is configured a bit later during the boot, when the kernel runs from its ordinary virtual mapping, rather than the 1:1 mapping that the startup code uses.
Accessing this shared GHCB page from the core kernel's startup code is problematic, because it involves converting the GHCB address provided by the caller to a physical address. In the startup code, virtual to physical address translations are problematic, given that the virtual address might be a 1:1 mapped address, and such translations should therefore be avoided.
This means that exposing startup code dealing with the GHCB to callers that execute from the ordinary kernel virtual mapping should be avoided too. So move all GHCB page based communication out of the startup code, now that all communication occurring before the kernel virtual mapping is up relies on the MSR protocol only.
As an exception, add a flag representing the need to apply the coherency fix in order to avoid exporting CPUID* helpers because of the code running too early for the *cpu_has* infrastructure.
--- arch/x86/boot/compressed/sev-handle-vc.c | 3 +- arch/x86/boot/compressed/sev.c | 2 +- arch/x86/boot/cpuflags.c | 13 +-- arch/x86/boot/startup/sev-shared.c | 145 +---------------------- arch/x86/boot/startup/sev-startup.c | 42 +------ arch/x86/boot/startup/sme.c | 1 +- arch/x86/coco/sev/core.c | 76 ++++++++++++- arch/x86/coco/sev/vc-handle.c | 2 +- arch/x86/coco/sev/vc-shared.c | 94 ++++++++++++++- arch/x86/include/asm/sev-internal.h | 7 +- arch/x86/include/asm/sev.h | 12 +- 11 files changed, 196 insertions(+), 201 deletions(-)
arch/x86/boot/compressed/sev-handle-vc.c +3 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #include "misc.h" +#include "error.h" #include "sev.h" #include <linux/kernel.h>
@@ -14,6 +15,8 @@#include <asm/fpu/xcr.h> #define __BOOT_COMPRESSED +#undef __init +#define __init/* Basic instruction decoding support needed */ #include "../../lib/inat.c"
arch/x86/boot/compressed/sev.c +2 -0
@@ -371,6 +371,8 @@ static int sev_check_cpu_support(void)if (!(eax &BIT (1 )))return -ENODEV ; + sev_snp_needs_sfw = !(ebx &BIT (31 )); +return ebx &0x3f ; }
arch/x86/boot/cpuflags.c +0 -13
@@ -106,18 +106,5 @@ void get_cpuflags(void)cpuid (0x80000001 , &ignored, &ignored, &cpu.flags[6 ], &cpu.flags[1 ]); } - -if (max_amd_level >=0x8000001f ) { -u32 ebx; - -/* -* The X86_FEATURE_COHERENCY_SFW_NO feature bit is in -* the virtualization flags entry (word 8) and set by -* scattered.c, so the bit needs to be explicitly set. -*/ -cpuid (0x8000001f , &ignored, &ebx, &ignored, &ignored); -if (ebx &BIT (31 )) -set_bit (X86_FEATURE_COHERENCY_SFW_NO , cpu.flags); - } } }
arch/x86/boot/startup/sev-shared.c +6 -139
196 lines not on this page · raw message
arch/x86/boot/startup/sev-startup.c +0 -42
54 lines not on this page · raw message
arch/x86/boot/startup/sme.c +1 -0
@@ -521,6 +521,7 @@ void __head sme_enable(struct boot_params *bp)return ; me_mask =1UL << (ebx &0x3f ); + sev_snp_needs_sfw = !(ebx &BIT (31 ));/* Check the SEV MSR whether SEV or SME is enabled */ sev_status = msr =native_rdmsrq (MSR_AMD64_SEV );
arch/x86/coco/sev/core.c +76 -0
88 lines not on this page · raw message
arch/x86/coco/sev/vc-handle.c +2 -0
@@ -351,6 +351,8 @@ fault: }#define sev_printk (fmt, ...) printk(fmt, ##__VA_ARGS__) +#define error (v) +#define has_cpuflag (f) boot_cpu_has(f)#include "vc-shared.c"
arch/x86/coco/sev/vc-shared.c +94 -0
103 lines not on this page · raw message
arch/x86/include/asm/sev-internal.h +2 -5
@@ -2,7 +2,6 @@#define DR7_RESET_VALUE 0x400 -extern struct ghcb boot_ghcb_page;extern u64 sev_hv_features;extern u64 sev_secrets_pa;
@@ -80,7 +79,8 @@ static __always_inline u64 svsm_get_caa_pa(void)return boot_svsm_caa_pa; } -int svsm_perform_call_protocol (struct svsm_call *call); +enum es_result verify_exception_info (struct ghcb *ghcb,struct es_em_ctxt *ctxt); +void vc_forward_exception (struct es_em_ctxt *ctxt);static inline u64 sev_es_rd_ghcb_msr (void ) {
@@ -97,9 +97,6 @@ static __always_inline void sev_es_wr_ghcb_msr(u64 val)native_wrmsr (MSR_AMD64_SEV_ES_GHCB , low, high); } -void snp_register_ghcb_early (unsigned long paddr); -bool sev_es_negotiate_protocol (void ); -bool sev_es_check_cpu_features (void );u64 get_hv_features (void );const struct snp_cpuid_table *snp_cpuid_get_table (void );
arch/x86/include/asm/sev.h +10 -2
@@ -503,6 +503,7 @@ static inline int pvalidate(unsigned long vaddr, bool rmp_psize, bool validate) }void setup_ghcb (void ); +void snp_register_ghcb_early (unsigned long paddr);void early_snp_set_memory_private (unsigned long vaddr,unsigned long paddr,unsigned long npages); void early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr,
@@ -540,8 +541,6 @@ static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)__builtin_memset (ghcb->save.valid_bitmap,0 ,sizeof (ghcb->save.valid_bitmap)); } -void vc_forward_exception (struct es_em_ctxt *ctxt); -/* I/O parameters for CPUID-related helpers */ struct cpuid_leaf {u32 fn;
@@ -552,16 +551,25 @@ struct cpuid_leaf {u32 edx; }; +int svsm_perform_msr_protocol (struct svsm_call *call);int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf),void *ctx,struct cpuid_leaf *leaf); +void svsm_issue_call (struct svsm_call *call,u8 *pending); +int svsm_process_result_codes (struct svsm_call *call); +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason);enum es_result sev_es_ghcb_hv_call (struct ghcb *ghcb,struct es_em_ctxt *ctxt,u64 exit_code,u64 exit_info_1,u64 exit_info_2); +bool sev_es_negotiate_protocol (void ); +bool sev_es_check_cpu_features (void ); + +extern u16 ghcb_version;extern struct ghcb *boot_ghcb; +extern bool sev_snp_needs_sfw;#else /* !CONFIG_AMD_MEM_ENCRYPT */
PATCH v7 6/22 x86/sev: Avoid global variable to store virtual address of SVSM area
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The boottime SVSM calling area is used both by the startup code running from a 1:1 mapping, and potentially later on running from the ordinary kernel mapping.
This SVSM calling area is statically allocated, and so its physical address doesn't change. However, its virtual address depends on the calling context (1:1 mapping or kernel virtual mapping), and even though the variable that holds the virtual address of this calling area gets updated from 1:1 address to kernel address during the boot, it is hard to reason about why this is guaranteed to be safe.
So instead, take the RIP-relative address of the boottime SVSM calling area whenever its virtual address is required, and only use a global variable for the physical address.
--- arch/x86/boot/compressed/sev.c | 5 ++--- arch/x86/boot/startup/sev-shared.c | 7 +------ arch/x86/boot/startup/sev-startup.c | 9 +++++---- arch/x86/coco/sev/core.c | 9 --------- arch/x86/include/asm/sev-internal.h | 3 +-- arch/x86/include/asm/sev.h | 2 -- arch/x86/mm/mem_encrypt_amd.c | 6 ------ 7 files changed, 9 insertions(+), 32 deletions(-)
arch/x86/boot/compressed/sev.c +2 -3
@@ -37,12 +37,12 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSED -extern struct svsm_ca *boot_svsm_caa;extern u64 boot_svsm_caa_pa;struct svsm_ca *svsm_get_caa (void ) { -return boot_svsm_caa; +/* The decompressor is mapped 1:1 so VA == PA */ +return (struct svsm_ca *)boot_svsm_caa_pa; }u64 svsm_get_caa_pa (void )
@@ -530,7 +530,6 @@ bool early_is_sevsnp_guest(void)/* Obtain the address of the calling area to use */ boot_rdmsr (MSR_SVSM_CAA , &m); - boot_svsm_caa = (void *)m.q; boot_svsm_caa_pa = m.q;/*
arch/x86/boot/startup/sev-shared.c +1 -6
@@ -13,6 +13,7 @@#ifndef __BOOT_COMPRESSED#define error (v) pr_err(v) +#define has_cpuflag (f) boot_cpu_has(f)#else #undef WARN #define WARN(condition, format...) (!!(condition))
@@ -26,7 +27,6 @@* early boot, both with identity mapped virtual addresses and proper kernel * virtual addresses. */ -struct svsm_ca *boot_svsm_caa__ro_after_init ;u64 boot_svsm_caa_pa__ro_after_init ;/*
@@ -718,11 +718,6 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info,if (caa & (PAGE_SIZE -1 ))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SVSM_CAA ); -/* -* The CA is identity mapped when this routine is called, both by the -* decompressor code and the early kernel code. -*/ - boot_svsm_caa = (struct svsm_ca *)caa; boot_svsm_caa_pa = caa;/* Advertise the SVSM presence via CPUID. */
arch/x86/boot/startup/sev-startup.c +5 -4
@@ -252,6 +252,7 @@ static __head struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) static __head void svsm_setup(struct cc_blob_sev_info *cc_info) { +struct snp_secrets_page *secrets = (void *)cc_info->secrets_phys;struct svsm_call call = {};u64 pa;
@@ -272,21 +273,21 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info) pa = (u64 )rip_rel_ptr (&boot_svsm_ca_page);/* -* Switch over to the boot SVSM CA while the current CA is still -* addressable. There is no GHCB at this point so use the MSR protocol. +* Switch over to the boot SVSM CA while the current CA is still 1:1 +* mapped and thus addressable with VA == PA. There is no GHCB at this +* point so use the MSR protocol. * * SVSM_CORE_REMAP_CA call: * RAX = 0 (Protocol=0, CallID=0) * RCX = New CA GPA */ - call.caa =svsm_get_caa (); + call.caa = (struct svsm_ca *)secrets->svsm_caa; call.rax =SVSM_CORE_CALL (SVSM_CORE_REMAP_CA ); call.rcx = pa;if (svsm_call_msr_protocol (&call))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SVSM_CA_REMAP_FAIL ); - boot_svsm_caa = (struct svsm_ca *)pa; boot_svsm_caa_pa = pa; }
arch/x86/coco/sev/core.c +0 -9
@@ -1666,15 +1666,6 @@ void sev_show_status(void)pr_cont ("\n" ); } -void __init snp_update_svsm_ca (void ) -{ -if (!snp_vmpl) -return ; - -/* Update the CAA to a proper kernel address */ - boot_svsm_caa = &boot_svsm_ca_page; -} -#ifdef CONFIG_SYSFS static ssize_t vmpl_show (struct kobject *kobj,struct kobj_attribute *attr,char *buf)
arch/x86/include/asm/sev-internal.h +1 -2
@@ -60,7 +60,6 @@ void early_set_pages_state(unsigned long vaddr, unsigned long paddr, DECLARE_PER_CPU(struct svsm_ca *, svsm_caa);DECLARE_PER_CPU (u64, svsm_caa_pa); -extern struct svsm_ca *boot_svsm_caa;extern u64 boot_svsm_caa_pa; static __always_inline struct svsm_ca *svsm_get_caa(void)
@@ -68,7 +67,7 @@ static __always_inline struct svsm_ca *svsm_get_caa(void)if (sev_cfg.use_cas)return this_cpu_read (svsm_caa);else -return boot_svsm_caa; +return rip_rel_ptr (&boot_svsm_ca_page); } static __always_inline u64 svsm_get_caa_pa(void)
arch/x86/include/asm/sev.h +0 -2
@@ -519,7 +519,6 @@ void snp_accept_memory(phys_addr_t start, phys_addr_t end);u64 snp_get_unsupported_features (u64 status);u64 sev_get_status (void );void sev_show_status (void ); -void snp_update_svsm_ca (void );int prepare_pte_enc (struct pte_enc_desc *d);void set_pte_enc_mask (pte_t *kpte,unsigned long pfn,pgprot_t new_prot);void snp_kexec_finish (void );
@@ -600,7 +599,6 @@ static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { }static inline u64 snp_get_unsupported_features (u64 status) {return 0 ; }static inline u64 sev_get_status (void ) {return 0 ; }static inline void sev_show_status (void ) { } -static inline void snp_update_svsm_ca (void ) { }static inline int prepare_pte_enc (struct pte_enc_desc *d) {return 0 ; }static inline void set_pte_enc_mask (pte_t *kpte,unsigned long pfn,pgprot_t new_prot) { }static inline void snp_kexec_finish (void ) { }
arch/x86/mm/mem_encrypt_amd.c +0 -6
@@ -536,12 +536,6 @@ void __init sme_early_init(void) x86_init.resources.dmi_setup = snp_dmi_setup; } -/* -* Switch the SVSM CA mapping (if active) from identity mapped to -* kernel mapped. -*/ -snp_update_svsm_ca (); -if (sev_status &MSR_AMD64_SNP_SECURE_TSC )setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); }
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Avoid global variable to store virtual address of SVSM area
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: a5f03880f06a6da6ea5f1d966fffffcb3fc65462 Gitweb: https://git.kernel.org/tip/a5f03880f06a6da6ea5f1d966fffffcb3fc65462 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:09 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:58:15 +02:00
x86/sev: Avoid global variable to store virtual address of SVSM area
The boottime SVSM calling area is used both by the startup code running from a 1:1 mapping, and potentially later on running from the ordinary kernel mapping.
This SVSM calling area is statically allocated, and so its physical address doesn't change. However, its virtual address depends on the calling context (1:1 mapping or kernel virtual mapping), and even though the variable that holds the virtual address of this calling area gets updated from 1:1 address to kernel address during the boot, it is hard to reason about why this is guaranteed to be safe.
So instead, take the RIP-relative address of the boottime SVSM calling area whenever its virtual address is required, and only use a global variable for the physical address.
--- arch/x86/boot/compressed/sev.c | 5 ++--- arch/x86/boot/startup/sev-shared.c | 7 +------ arch/x86/boot/startup/sev-startup.c | 9 +++++---- arch/x86/coco/sev/core.c | 9 --------- arch/x86/include/asm/sev-internal.h | 3 +-- arch/x86/include/asm/sev.h | 2 -- arch/x86/mm/mem_encrypt_amd.c | 6 ------ 7 files changed, 9 insertions(+), 32 deletions(-)
arch/x86/boot/compressed/sev.c +2 -3
@@ -37,12 +37,12 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSED -extern struct svsm_ca *boot_svsm_caa;extern u64 boot_svsm_caa_pa;struct svsm_ca *svsm_get_caa (void ) { -return boot_svsm_caa; +/* The decompressor is mapped 1:1 so VA == PA */ +return (struct svsm_ca *)boot_svsm_caa_pa; }u64 svsm_get_caa_pa (void )
@@ -532,7 +532,6 @@ bool early_is_sevsnp_guest(void)/* Obtain the address of the calling area to use */ boot_rdmsr (MSR_SVSM_CAA , &m); - boot_svsm_caa = (void *)m.q; boot_svsm_caa_pa = m.q;/*
arch/x86/boot/startup/sev-shared.c +1 -6
@@ -13,6 +13,7 @@#ifndef __BOOT_COMPRESSED#define error (v) pr_err(v) +#define has_cpuflag (f) boot_cpu_has(f)#else #undef WARN #define WARN(condition, format...) (!!(condition))
@@ -26,7 +27,6 @@* early boot, both with identity mapped virtual addresses and proper kernel * virtual addresses. */ -struct svsm_ca *boot_svsm_caa__ro_after_init ;u64 boot_svsm_caa_pa__ro_after_init ;/*
@@ -720,11 +720,6 @@ static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info,if (caa & (PAGE_SIZE -1 ))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SVSM_CAA ); -/* -* The CA is identity mapped when this routine is called, both by the -* decompressor code and the early kernel code. -*/ - boot_svsm_caa = (struct svsm_ca *)caa; boot_svsm_caa_pa = caa;/* Advertise the SVSM presence via CPUID. */
arch/x86/boot/startup/sev-startup.c +5 -4
@@ -252,6 +252,7 @@ found_cc_info: static __head void svsm_setup(struct cc_blob_sev_info *cc_info) { +struct snp_secrets_page *secrets = (void *)cc_info->secrets_phys;struct svsm_call call = {};u64 pa;
@@ -272,21 +273,21 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info) pa = (u64 )rip_rel_ptr (&boot_svsm_ca_page);/* -* Switch over to the boot SVSM CA while the current CA is still -* addressable. There is no GHCB at this point so use the MSR protocol. +* Switch over to the boot SVSM CA while the current CA is still 1:1 +* mapped and thus addressable with VA == PA. There is no GHCB at this +* point so use the MSR protocol. * * SVSM_CORE_REMAP_CA call: * RAX = 0 (Protocol=0, CallID=0) * RCX = New CA GPA */ - call.caa =svsm_get_caa (); + call.caa = (struct svsm_ca *)secrets->svsm_caa; call.rax =SVSM_CORE_CALL (SVSM_CORE_REMAP_CA ); call.rcx = pa;if (svsm_call_msr_protocol (&call))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SVSM_CA_REMAP_FAIL ); - boot_svsm_caa = (struct svsm_ca *)pa; boot_svsm_caa_pa = pa; }
arch/x86/coco/sev/core.c +0 -9
@@ -1666,15 +1666,6 @@ void sev_show_status(void)pr_cont ("\n" ); } -void __init snp_update_svsm_ca (void ) -{ -if (!snp_vmpl) -return ; - -/* Update the CAA to a proper kernel address */ - boot_svsm_caa = &boot_svsm_ca_page; -} -#ifdef CONFIG_SYSFS static ssize_t vmpl_show (struct kobject *kobj,struct kobj_attribute *attr,char *buf)
arch/x86/include/asm/sev-internal.h +1 -2
@@ -60,7 +60,6 @@ void early_set_pages_state(unsigned long vaddr, unsigned long paddr, DECLARE_PER_CPU(struct svsm_ca *, svsm_caa);DECLARE_PER_CPU (u64, svsm_caa_pa); -extern struct svsm_ca *boot_svsm_caa;extern u64 boot_svsm_caa_pa; static __always_inline struct svsm_ca *svsm_get_caa(void)
@@ -68,7 +67,7 @@ static __always_inline struct svsm_ca *svsm_get_caa(void)if (sev_cfg.use_cas)return this_cpu_read (svsm_caa);else -return boot_svsm_caa; +return rip_rel_ptr (&boot_svsm_ca_page); } static __always_inline u64 svsm_get_caa_pa(void)
arch/x86/include/asm/sev.h +0 -2
@@ -519,7 +519,6 @@ void snp_accept_memory(phys_addr_t start, phys_addr_t end);u64 snp_get_unsupported_features (u64 status);u64 sev_get_status (void );void sev_show_status (void ); -void snp_update_svsm_ca (void );int prepare_pte_enc (struct pte_enc_desc *d);void set_pte_enc_mask (pte_t *kpte,unsigned long pfn,pgprot_t new_prot);void snp_kexec_finish (void );
@@ -601,7 +600,6 @@ static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { }static inline u64 snp_get_unsupported_features (u64 status) {return 0 ; }static inline u64 sev_get_status (void ) {return 0 ; }static inline void sev_show_status (void ) { } -static inline void snp_update_svsm_ca (void ) { }static inline int prepare_pte_enc (struct pte_enc_desc *d) {return 0 ; }static inline void set_pte_enc_mask (pte_t *kpte,unsigned long pfn,pgprot_t new_prot) { }static inline void snp_kexec_finish (void ) { }
arch/x86/mm/mem_encrypt_amd.c +0 -6
@@ -536,12 +536,6 @@ void __init sme_early_init(void) x86_init.resources.dmi_setup = snp_dmi_setup; } -/* -* Switch the SVSM CA mapping (if active) from identity mapped to -* kernel mapped. -*/ -snp_update_svsm_ca (); -if (sev_status &MSR_AMD64_SNP_SECURE_TSC )setup_force_cpu_cap (X86_FEATURE_TSC_RELIABLE ); }
PATCH v7 7/22 x86/sev: Share implementation of MSR-based page state change
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Both the decompressor and the SEV startup code implement the exact same sequence for invoking the MSR based communication protocol to effectuate a page state change.
Before tweaking the internal APIs used in both versions, merge them and share them so those tweaks are only needed in a single place.
--- arch/x86/boot/compressed/sev.c | 40 ++------------------ arch/x86/boot/startup/sev-shared.c | 35 +++++++++++++++++ arch/x86/boot/startup/sev-startup.c | 29 +------------- 3 files changed, 39 insertions(+), 65 deletions(-)
arch/x86/boot/compressed/sev.c +3 -37
@@ -60,46 +60,12 @@ static bool sev_snp_enabled(void)return sev_status &MSR_AMD64_SEV_SNP_ENABLED ; } -static void __page_state_change (unsigned long paddr,enum psc_op op) -{ -u64 val, msr; - -/* -* If private -> shared then invalidate the page before requesting the -* state change in the RMP table. -*/ -if (op ==SNP_PAGE_STATE_SHARED ) -pvalidate_4k_page (paddr, paddr, false); - -/* Save the current GHCB MSR value */ - msr =sev_es_rd_ghcb_msr (); - -/* Issue VMGEXIT to change the page state in RMP table. */ -sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); -VMGEXIT (); - -/* Read the response of the VMGEXIT. */ - val =sev_es_rd_ghcb_msr (); -if ((GHCB_RESP_CODE (val) !=GHCB_MSR_PSC_RESP ) ||GHCB_MSR_PSC_RESP_VAL (val)) -sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PSC ); - -/* Restore the GHCB MSR value */ -sev_es_wr_ghcb_msr (msr); - -/* -* Now that page state is changed in the RMP table, validate it so that it is -* consistent with the RMP entry. -*/ -if (op ==SNP_PAGE_STATE_PRIVATE ) -pvalidate_4k_page (paddr, paddr, true); -} -void snp_set_page_private (unsigned long paddr) {if (!sev_snp_enabled ())return ; -__page_state_change (paddr,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (paddr, paddr,SNP_PAGE_STATE_PRIVATE ); }void snp_set_page_shared (unsigned long paddr)
@@ -107,7 +73,7 @@ void snp_set_page_shared(unsigned long paddr)if (!sev_snp_enabled ())return ; -__page_state_change (paddr,SNP_PAGE_STATE_SHARED ); +__page_state_change (paddr, paddr,SNP_PAGE_STATE_SHARED ); }bool early_setup_ghcb (void )
@@ -133,7 +99,7 @@ bool early_setup_ghcb(void)void snp_accept_memory (phys_addr_t start,phys_addr_t end) {for (phys_addr_t pa = start; pa < end; pa +=PAGE_SIZE ) -__page_state_change (pa,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (pa, pa,SNP_PAGE_STATE_PRIVATE ); }void sev_es_shutdown_ghcb (void )
arch/x86/boot/startup/sev-shared.c +35 -0
@@ -662,6 +662,41 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,sev_evict_cache ((void *)vaddr,1 ); } +static void __head __page_state_change(unsigned long vaddr, unsigned long paddr, +enum psc_op op) +{ +u64 val, msr; + +/* +* If private -> shared then invalidate the page before requesting the +* state change in the RMP table. +*/ +if (op ==SNP_PAGE_STATE_SHARED ) +pvalidate_4k_page (vaddr, paddr, false); + +/* Save the current GHCB MSR value */ + msr =sev_es_rd_ghcb_msr (); + +/* Issue VMGEXIT to change the page state in RMP table. */ +sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); +VMGEXIT (); + +/* Read the response of the VMGEXIT. */ + val =sev_es_rd_ghcb_msr (); +if ((GHCB_RESP_CODE (val) !=GHCB_MSR_PSC_RESP ) ||GHCB_MSR_PSC_RESP_VAL (val)) +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PSC ); + +/* Restore the GHCB MSR value */ +sev_es_wr_ghcb_msr (msr); + +/* +* Now that page state is changed in the RMP table, validate it so that it is +* consistent with the RMP entry. +*/ +if (op ==SNP_PAGE_STATE_PRIVATE ) +pvalidate_4k_page (vaddr, paddr, true); +} +/* * Maintain the GPA of the SVSM Calling Area (CA) in order to utilize the SVSM * services needed when not running in VMPL0.
arch/x86/boot/startup/sev-startup.c +1 -28
@@ -135,7 +135,6 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, unsigned long npages, enum psc_op op) {unsigned long paddr_end; -u64 val; vaddr = vaddr &PAGE_MASK ;
@@ -143,37 +142,11 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, paddr_end = paddr + (npages <<PAGE_SHIFT );while (paddr < paddr_end) { -/* Page validation must be rescinded before changing to shared */ -if (op ==SNP_PAGE_STATE_SHARED ) -pvalidate_4k_page (vaddr, paddr, false); - -/* -* Use the MSR protocol because this function can be called before -* the GHCB is established. -*/ -sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); -VMGEXIT (); - - val =sev_es_rd_ghcb_msr (); - -if (GHCB_RESP_CODE (val) !=GHCB_MSR_PSC_RESP ) - goto e_term; - -if (GHCB_MSR_PSC_RESP_VAL (val)) - goto e_term; - -/* Page validation must be performed after changing to private */ -if (op ==SNP_PAGE_STATE_PRIVATE ) -pvalidate_4k_page (vaddr, paddr, true); +__page_state_change (vaddr, paddr, op); vaddr +=PAGE_SIZE ; paddr +=PAGE_SIZE ; } - -return ; - -e_term: -sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PSC ); } void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr,
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Share implementation of MSR-based page state change
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: d5949ea50c5642ab7e3c4dd6020e23725c079b25 Gitweb: https://git.kernel.org/tip/d5949ea50c5642ab7e3c4dd6020e23725c079b25 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:10 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:58:19 +02:00
x86/sev: Share implementation of MSR-based page state change
Both the decompressor and the SEV startup code implement the exact same sequence for invoking the MSR based communication protocol to effectuate a page state change.
Before tweaking the internal APIs used in both versions, merge them and share them so those tweaks are only needed in a single place.
--- arch/x86/boot/compressed/sev.c | 40 ++-------------------------- arch/x86/boot/startup/sev-shared.c | 35 +++++++++++++++++++++++++- arch/x86/boot/startup/sev-startup.c | 29 +-------------------- 3 files changed, 39 insertions(+), 65 deletions(-)
arch/x86/boot/compressed/sev.c +3 -37
@@ -60,46 +60,12 @@ static bool sev_snp_enabled(void)return sev_status &MSR_AMD64_SEV_SNP_ENABLED ; } -static void __page_state_change (unsigned long paddr,enum psc_op op) -{ -u64 val, msr; - -/* -* If private -> shared then invalidate the page before requesting the -* state change in the RMP table. -*/ -if (op ==SNP_PAGE_STATE_SHARED ) -pvalidate_4k_page (paddr, paddr, false); - -/* Save the current GHCB MSR value */ - msr =sev_es_rd_ghcb_msr (); - -/* Issue VMGEXIT to change the page state in RMP table. */ -sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); -VMGEXIT (); - -/* Read the response of the VMGEXIT. */ - val =sev_es_rd_ghcb_msr (); -if ((GHCB_RESP_CODE (val) !=GHCB_MSR_PSC_RESP ) ||GHCB_MSR_PSC_RESP_VAL (val)) -sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PSC ); - -/* Restore the GHCB MSR value */ -sev_es_wr_ghcb_msr (msr); - -/* -* Now that page state is changed in the RMP table, validate it so that it is -* consistent with the RMP entry. -*/ -if (op ==SNP_PAGE_STATE_PRIVATE ) -pvalidate_4k_page (paddr, paddr, true); -} -void snp_set_page_private (unsigned long paddr) {if (!sev_snp_enabled ())return ; -__page_state_change (paddr,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (paddr, paddr,SNP_PAGE_STATE_PRIVATE ); }void snp_set_page_shared (unsigned long paddr)
@@ -107,7 +73,7 @@ void snp_set_page_shared(unsigned long paddr)if (!sev_snp_enabled ())return ; -__page_state_change (paddr,SNP_PAGE_STATE_SHARED ); +__page_state_change (paddr, paddr,SNP_PAGE_STATE_SHARED ); }bool early_setup_ghcb (void )
@@ -133,7 +99,7 @@ bool early_setup_ghcb(void)void snp_accept_memory (phys_addr_t start,phys_addr_t end) {for (phys_addr_t pa = start; pa < end; pa +=PAGE_SIZE ) -__page_state_change (pa,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (pa, pa,SNP_PAGE_STATE_PRIVATE ); }void sev_es_shutdown_ghcb (void )
arch/x86/boot/startup/sev-shared.c +35 -0
@@ -664,6 +664,41 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,sev_evict_cache ((void *)vaddr,1 ); } +static void __head __page_state_change(unsigned long vaddr, unsigned long paddr, +enum psc_op op) +{ +u64 val, msr; + +/* +* If private -> shared then invalidate the page before requesting the +* state change in the RMP table. +*/ +if (op ==SNP_PAGE_STATE_SHARED ) +pvalidate_4k_page (vaddr, paddr, false); + +/* Save the current GHCB MSR value */ + msr =sev_es_rd_ghcb_msr (); + +/* Issue VMGEXIT to change the page state in RMP table. */ +sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); +VMGEXIT (); + +/* Read the response of the VMGEXIT. */ + val =sev_es_rd_ghcb_msr (); +if ((GHCB_RESP_CODE (val) !=GHCB_MSR_PSC_RESP ) ||GHCB_MSR_PSC_RESP_VAL (val)) +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PSC ); + +/* Restore the GHCB MSR value */ +sev_es_wr_ghcb_msr (msr); + +/* +* Now that page state is changed in the RMP table, validate it so that it is +* consistent with the RMP entry. +*/ +if (op ==SNP_PAGE_STATE_PRIVATE ) +pvalidate_4k_page (vaddr, paddr, true); +} +/* * Maintain the GPA of the SVSM Calling Area (CA) in order to utilize the SVSM * services needed when not running in VMPL0.
arch/x86/boot/startup/sev-startup.c +1 -28
@@ -135,7 +135,6 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, unsigned long npages, enum psc_op op) {unsigned long paddr_end; -u64 val; vaddr = vaddr &PAGE_MASK ;
@@ -143,37 +142,11 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, paddr_end = paddr + (npages <<PAGE_SHIFT );while (paddr < paddr_end) { -/* Page validation must be rescinded before changing to shared */ -if (op ==SNP_PAGE_STATE_SHARED ) -pvalidate_4k_page (vaddr, paddr, false); - -/* -* Use the MSR protocol because this function can be called before -* the GHCB is established. -*/ -sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); -VMGEXIT (); - - val =sev_es_rd_ghcb_msr (); - -if (GHCB_RESP_CODE (val) !=GHCB_MSR_PSC_RESP ) - goto e_term; - -if (GHCB_MSR_PSC_RESP_VAL (val)) - goto e_term; - -/* Page validation must be performed after changing to private */ -if (op ==SNP_PAGE_STATE_PRIVATE ) -pvalidate_4k_page (vaddr, paddr, true); +__page_state_change (vaddr, paddr, op); vaddr +=PAGE_SIZE ; paddr +=PAGE_SIZE ; } - -return ; - -e_term: -sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_PSC ); } void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr,
PATCH v7 8/22 x86/sev: Pass SVSM calling area down to early page state change API
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The early page state change API is mostly only used very early, when only the boot time SVSM calling area is in use. However, this API is also called by the kexec finishing code, which runs very late, and potentially from a different CPU (which uses a different calling area).
To avoid pulling the per-CPU SVSM calling area pointers and related SEV state into the startup code, refactor the page state change API so the SVSM calling area virtual and physical addresses can be provided by the caller.
No functional change intended.
--- arch/x86/boot/compressed/sev.c | 24 +++++++++++++++++--- arch/x86/boot/startup/sev-shared.c | 23 ++++++++++--------- arch/x86/boot/startup/sev-startup.c | 16 +++++++++---- arch/x86/coco/sev/core.c | 7 ++++-- arch/x86/include/asm/sev-internal.h | 2 +- arch/x86/include/asm/sev.h | 6 +++++ 6 files changed, 57 insertions(+), 21 deletions(-)
arch/x86/boot/compressed/sev.c +21 -3
@@ -62,18 +62,30 @@ static bool sev_snp_enabled(void)void snp_set_page_private (unsigned long paddr) { +struct psc_desc d = { +SNP_PAGE_STATE_PRIVATE , + (struct svsm_ca *)boot_svsm_caa_pa, + boot_svsm_caa_pa + }; +if (!sev_snp_enabled ())return ; -__page_state_change (paddr, paddr,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (paddr, paddr, &d); }void snp_set_page_shared (unsigned long paddr) { +struct psc_desc d = { +SNP_PAGE_STATE_SHARED , + (struct svsm_ca *)boot_svsm_caa_pa, + boot_svsm_caa_pa + }; +if (!sev_snp_enabled ())return ; -__page_state_change (paddr, paddr,SNP_PAGE_STATE_SHARED ); +__page_state_change (paddr, paddr, &d); }bool early_setup_ghcb (void )
@@ -98,8 +110,14 @@ bool early_setup_ghcb(void)void snp_accept_memory (phys_addr_t start,phys_addr_t end) { +struct psc_desc d = { +SNP_PAGE_STATE_PRIVATE , + (struct svsm_ca *)boot_svsm_caa_pa, + boot_svsm_caa_pa + }; +for (phys_addr_t pa = start; pa < end; pa +=PAGE_SIZE ) -__page_state_change (pa, pa,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (pa, pa, &d); }void sev_es_shutdown_ghcb (void )
arch/x86/boot/startup/sev-shared.c +12 -11
@@ -600,7 +600,8 @@ static int __head svsm_call_msr_protocol(struct svsm_call *call)return ret; } -static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) +static void __head svsm_pval_4k_page(unsigned long paddr, bool validate, +struct svsm_ca *caa,u64 caa_pa) {struct svsm_pvalidate_call *pc;struct svsm_call call = {};
@@ -613,10 +614,10 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate)*/ flags =native_local_irq_save (); - call.caa =svsm_get_caa (); + call.caa = caa; pc = (struct svsm_pvalidate_call *)call.caa->svsm_buffer; - pc_pa =svsm_get_caa_pa () + offsetof(struct svsm_ca , svsm_buffer); + pc_pa = caa_pa + offsetof(struct svsm_ca , svsm_buffer); pc->num_entries =1 ; pc->cur_index =0 ;
@@ -642,12 +643,12 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) } static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, -bool validate) +bool validate,struct svsm_ca *caa,u64 caa_pa) {int ret;if (snp_vmpl) { -svsm_pval_4k_page (paddr, validate); +svsm_pval_4k_page (paddr, validate, caa, caa_pa); }else { ret =pvalidate (vaddr,RMP_PG_SIZE_4K , validate);if (ret)
@@ -663,7 +664,7 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, } static void __head __page_state_change(unsigned long vaddr, unsigned long paddr, -enum psc_op op) +const struct psc_desc *desc) {u64 val, msr;
@@ -671,14 +672,14 @@ static void __head __page_state_change(unsigned long vaddr, unsigned long paddr,* If private -> shared then invalidate the page before requesting the * state change in the RMP table. */ -if (op ==SNP_PAGE_STATE_SHARED ) -pvalidate_4k_page (vaddr, paddr, false); +if (desc->op ==SNP_PAGE_STATE_SHARED ) +pvalidate_4k_page (vaddr, paddr, false, desc->ca, desc->caa_pa);/* Save the current GHCB MSR value */ msr =sev_es_rd_ghcb_msr ();/* Issue VMGEXIT to change the page state in RMP table. */ -sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); +sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , desc->op));VMGEXIT ();/* Read the response of the VMGEXIT. */
@@ -693,8 +694,8 @@ static void __head __page_state_change(unsigned long vaddr, unsigned long paddr,* Now that page state is changed in the RMP table, validate it so that it is * consistent with the RMP entry. */ -if (op ==SNP_PAGE_STATE_PRIVATE ) -pvalidate_4k_page (vaddr, paddr, true); +if (desc->op ==SNP_PAGE_STATE_PRIVATE ) +pvalidate_4k_page (vaddr, paddr, true, desc->ca, desc->caa_pa); }/*
arch/x86/boot/startup/sev-startup.c +12 -4
@@ -132,7 +132,7 @@ noinstr void __sev_put_ghcb(struct ghcb_state *state) void __headearly_set_pages_state (unsigned long vaddr,unsigned long paddr, -unsigned long npages,enum psc_op op) +unsigned long npages,const struct psc_desc *desc) {unsigned long paddr_end;
@@ -142,7 +142,7 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, paddr_end = paddr + (npages <<PAGE_SHIFT );while (paddr < paddr_end) { -__page_state_change (vaddr, paddr, op); +__page_state_change (vaddr, paddr, desc); vaddr +=PAGE_SIZE ; paddr +=PAGE_SIZE ;
@@ -152,6 +152,10 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr,unsigned long npages) { +struct psc_desc d = { +SNP_PAGE_STATE_PRIVATE ,svsm_get_caa (),svsm_get_caa_pa () + }; +/* * This can be invoked in early boot while running identity mapped, so * use an open coded check for SNP instead of using cc_platform_has().
@@ -165,12 +169,16 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long padd* Ask the hypervisor to mark the memory pages as private in the RMP * table. */ -early_set_pages_state (vaddr, paddr, npages,SNP_PAGE_STATE_PRIVATE ); +early_set_pages_state (vaddr, paddr, npages, &d); } void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr,unsigned long npages) { +struct psc_desc d = { +SNP_PAGE_STATE_SHARED ,svsm_get_caa (),svsm_get_caa_pa () + }; +/* * This can be invoked in early boot while running identity mapped, so * use an open coded check for SNP instead of using cc_platform_has().
@@ -181,7 +189,7 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddrreturn ;/* Ask hypervisor to mark the memory pages shared in the RMP table. */ -early_set_pages_state (vaddr, paddr, npages,SNP_PAGE_STATE_SHARED ); +early_set_pages_state (vaddr, paddr, npages, &d); }/*
arch/x86/coco/sev/core.c +5 -2
@@ -607,8 +607,11 @@ static void set_pages_state(unsigned long vaddr, unsigned long npages, int op)unsigned long vaddr_end;/* Use the MSR protocol when a GHCB is not available. */ -if (!boot_ghcb) -return early_set_pages_state (vaddr,__pa (vaddr), npages, op); +if (!boot_ghcb) { +struct psc_desc d = { op,svsm_get_caa (),svsm_get_caa_pa () }; + +return early_set_pages_state (vaddr,__pa (vaddr), npages, &d); + } vaddr = vaddr &PAGE_MASK ; vaddr_end = vaddr + (npages <<PAGE_SHIFT );
arch/x86/include/asm/sev-internal.h +1 -1
@@ -55,7 +55,7 @@ DECLARE_PER_CPU(struct sev_es_runtime_data*, runtime_data); DECLARE_PER_CPU(struct sev_es_save_area *, sev_vmsa);void early_set_pages_state (unsigned long vaddr,unsigned long paddr, -unsigned long npages,enum psc_op op); +unsigned long npages,const struct psc_desc *desc); DECLARE_PER_CPU(struct svsm_ca *, svsm_caa);DECLARE_PER_CPU (u64, svsm_caa_pa);
arch/x86/include/asm/sev.h +6 -0
@@ -569,6 +569,12 @@ bool sev_es_check_cpu_features(void);extern u16 ghcb_version;extern struct ghcb *boot_ghcb; +struct psc_desc { +enum psc_op op; +struct svsm_ca *ca; +u64 caa_pa; +}; +#else /* !CONFIG_AMD_MEM_ENCRYPT */ #define snp_vmpl 0
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Pass SVSM calling area down to early page state change API
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 00d25566761746ba53934ad3a89ea79923a38d01 Gitweb: https://git.kernel.org/tip/00d25566761746ba53934ad3a89ea79923a38d01 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:11 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:58:22 +02:00
x86/sev: Pass SVSM calling area down to early page state change API
The early page state change API is mostly only used very early, when only the boot time SVSM calling area is in use. However, this API is also called by the kexec finishing code, which runs very late, and potentially from a different CPU (which uses a different calling area).
To avoid pulling the per-CPU SVSM calling area pointers and related SEV state into the startup code, refactor the page state change API so the SVSM calling area virtual and physical addresses can be provided by the caller.
No functional change intended.
--- arch/x86/boot/compressed/sev.c | 24 +++++++++++++++++++++--- arch/x86/boot/startup/sev-shared.c | 23 ++++++++++++----------- arch/x86/boot/startup/sev-startup.c | 16 ++++++++++++---- arch/x86/coco/sev/core.c | 7 +++++-- arch/x86/include/asm/sev-internal.h | 2 +- arch/x86/include/asm/sev.h | 6 ++++++ 6 files changed, 57 insertions(+), 21 deletions(-)
arch/x86/boot/compressed/sev.c +21 -3
@@ -62,18 +62,30 @@ static bool sev_snp_enabled(void)void snp_set_page_private (unsigned long paddr) { +struct psc_desc d = { +SNP_PAGE_STATE_PRIVATE , + (struct svsm_ca *)boot_svsm_caa_pa, + boot_svsm_caa_pa + }; +if (!sev_snp_enabled ())return ; -__page_state_change (paddr, paddr,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (paddr, paddr, &d); }void snp_set_page_shared (unsigned long paddr) { +struct psc_desc d = { +SNP_PAGE_STATE_SHARED , + (struct svsm_ca *)boot_svsm_caa_pa, + boot_svsm_caa_pa + }; +if (!sev_snp_enabled ())return ; -__page_state_change (paddr, paddr,SNP_PAGE_STATE_SHARED ); +__page_state_change (paddr, paddr, &d); }bool early_setup_ghcb (void )
@@ -98,8 +110,14 @@ bool early_setup_ghcb(void)void snp_accept_memory (phys_addr_t start,phys_addr_t end) { +struct psc_desc d = { +SNP_PAGE_STATE_PRIVATE , + (struct svsm_ca *)boot_svsm_caa_pa, + boot_svsm_caa_pa + }; +for (phys_addr_t pa = start; pa < end; pa +=PAGE_SIZE ) -__page_state_change (pa, pa,SNP_PAGE_STATE_PRIVATE ); +__page_state_change (pa, pa, &d); }void sev_es_shutdown_ghcb (void )
arch/x86/boot/startup/sev-shared.c +12 -11
@@ -602,7 +602,8 @@ static int __head svsm_call_msr_protocol(struct svsm_call *call)return ret; } -static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) +static void __head svsm_pval_4k_page(unsigned long paddr, bool validate, +struct svsm_ca *caa,u64 caa_pa) {struct svsm_pvalidate_call *pc;struct svsm_call call = {};
@@ -615,10 +616,10 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate)*/ flags =native_local_irq_save (); - call.caa =svsm_get_caa (); + call.caa = caa; pc = (struct svsm_pvalidate_call *)call.caa->svsm_buffer; - pc_pa =svsm_get_caa_pa () + offsetof(struct svsm_ca , svsm_buffer); + pc_pa = caa_pa + offsetof(struct svsm_ca , svsm_buffer); pc->num_entries =1 ; pc->cur_index =0 ;
@@ -644,12 +645,12 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) } static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, -bool validate) +bool validate,struct svsm_ca *caa,u64 caa_pa) {int ret;if (snp_vmpl) { -svsm_pval_4k_page (paddr, validate); +svsm_pval_4k_page (paddr, validate, caa, caa_pa); }else { ret =pvalidate (vaddr,RMP_PG_SIZE_4K , validate);if (ret)
@@ -665,7 +666,7 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, } static void __head __page_state_change(unsigned long vaddr, unsigned long paddr, -enum psc_op op) +const struct psc_desc *desc) {u64 val, msr;
@@ -673,14 +674,14 @@ static void __head __page_state_change(unsigned long vaddr, unsigned long paddr,* If private -> shared then invalidate the page before requesting the * state change in the RMP table. */ -if (op ==SNP_PAGE_STATE_SHARED ) -pvalidate_4k_page (vaddr, paddr, false); +if (desc->op ==SNP_PAGE_STATE_SHARED ) +pvalidate_4k_page (vaddr, paddr, false, desc->ca, desc->caa_pa);/* Save the current GHCB MSR value */ msr =sev_es_rd_ghcb_msr ();/* Issue VMGEXIT to change the page state in RMP table. */ -sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , op)); +sev_es_wr_ghcb_msr (GHCB_MSR_PSC_REQ_GFN (paddr >>PAGE_SHIFT , desc->op));VMGEXIT ();/* Read the response of the VMGEXIT. */
@@ -695,8 +696,8 @@ static void __head __page_state_change(unsigned long vaddr, unsigned long paddr,* Now that page state is changed in the RMP table, validate it so that it is * consistent with the RMP entry. */ -if (op ==SNP_PAGE_STATE_PRIVATE ) -pvalidate_4k_page (vaddr, paddr, true); +if (desc->op ==SNP_PAGE_STATE_PRIVATE ) +pvalidate_4k_page (vaddr, paddr, true, desc->ca, desc->caa_pa); }/*
arch/x86/boot/startup/sev-startup.c +12 -4
@@ -132,7 +132,7 @@ noinstr void __sev_put_ghcb(struct ghcb_state *state) void __headearly_set_pages_state (unsigned long vaddr,unsigned long paddr, -unsigned long npages,enum psc_op op) +unsigned long npages,const struct psc_desc *desc) {unsigned long paddr_end;
@@ -142,7 +142,7 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, paddr_end = paddr + (npages <<PAGE_SHIFT );while (paddr < paddr_end) { -__page_state_change (vaddr, paddr, op); +__page_state_change (vaddr, paddr, desc); vaddr +=PAGE_SIZE ; paddr +=PAGE_SIZE ;
@@ -152,6 +152,10 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr,unsigned long npages) { +struct psc_desc d = { +SNP_PAGE_STATE_PRIVATE ,svsm_get_caa (),svsm_get_caa_pa () + }; +/* * This can be invoked in early boot while running identity mapped, so * use an open coded check for SNP instead of using cc_platform_has().
@@ -165,12 +169,16 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long padd* Ask the hypervisor to mark the memory pages as private in the RMP * table. */ -early_set_pages_state (vaddr, paddr, npages,SNP_PAGE_STATE_PRIVATE ); +early_set_pages_state (vaddr, paddr, npages, &d); } void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr,unsigned long npages) { +struct psc_desc d = { +SNP_PAGE_STATE_SHARED ,svsm_get_caa (),svsm_get_caa_pa () + }; +/* * This can be invoked in early boot while running identity mapped, so * use an open coded check for SNP instead of using cc_platform_has().
@@ -181,7 +189,7 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddrreturn ;/* Ask hypervisor to mark the memory pages shared in the RMP table. */ -early_set_pages_state (vaddr, paddr, npages,SNP_PAGE_STATE_SHARED ); +early_set_pages_state (vaddr, paddr, npages, &d); }/*
arch/x86/coco/sev/core.c +5 -2
@@ -607,8 +607,11 @@ static void set_pages_state(unsigned long vaddr, unsigned long npages, int op)unsigned long vaddr_end;/* Use the MSR protocol when a GHCB is not available. */ -if (!boot_ghcb) -return early_set_pages_state (vaddr,__pa (vaddr), npages, op); +if (!boot_ghcb) { +struct psc_desc d = { op,svsm_get_caa (),svsm_get_caa_pa () }; + +return early_set_pages_state (vaddr,__pa (vaddr), npages, &d); + } vaddr = vaddr &PAGE_MASK ; vaddr_end = vaddr + (npages <<PAGE_SHIFT );
arch/x86/include/asm/sev-internal.h +1 -1
@@ -55,7 +55,7 @@ DECLARE_PER_CPU(struct sev_es_runtime_data*, runtime_data); DECLARE_PER_CPU(struct sev_es_save_area *, sev_vmsa);void early_set_pages_state (unsigned long vaddr,unsigned long paddr, -unsigned long npages,enum psc_op op); +unsigned long npages,const struct psc_desc *desc); DECLARE_PER_CPU(struct svsm_ca *, svsm_caa);DECLARE_PER_CPU (u64, svsm_caa_pa);
arch/x86/include/asm/sev.h +6 -0
@@ -570,6 +570,12 @@ extern u16 ghcb_version;extern struct ghcb *boot_ghcb;extern bool sev_snp_needs_sfw; +struct psc_desc { +enum psc_op op; +struct svsm_ca *ca; +u64 caa_pa; +}; +#else /* !CONFIG_AMD_MEM_ENCRYPT */ #define snp_vmpl 0
PATCH v7 9/22 x86/sev: Use boot SVSM CA for all startup and init code
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
To avoid having to reason about whether or not to use the per-CPU SVSM calling area when running startup and init code on the boot CPU, reuse the boot SVSM calling area as the per-CPU area for CPU #0.
This removes the need to make the per-CPU variables and associated state in sev_cfg accessible to the startup code once confined.
--- arch/x86/boot/compressed/sev.c | 13 ------ arch/x86/boot/startup/sev-startup.c | 11 ++--- arch/x86/coco/sev/core.c | 47 +++++++++----------- arch/x86/include/asm/sev-internal.h | 16 ------- 4 files changed, 28 insertions(+), 59 deletions(-)
arch/x86/boot/compressed/sev.c +0 -13
@@ -37,19 +37,6 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSED -extern u64 boot_svsm_caa_pa; - -struct svsm_ca *svsm_get_caa (void ) -{ -/* The decompressor is mapped 1:1 so VA == PA */ -return (struct svsm_ca *)boot_svsm_caa_pa; -} - -u64 svsm_get_caa_pa (void ) -{ -return boot_svsm_caa_pa; -} -u8 snp_vmpl;/* Include code for early handlers */
arch/x86/boot/startup/sev-startup.c +6 -5
@@ -50,9 +50,6 @@ u64 sev_secrets_pa __ro_after_init;/* For early boot SVSM communication */ struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; -DEFINE_PER_CPU(struct svsm_ca *, svsm_caa); -DEFINE_PER_CPU (u64, svsm_caa_pa); -/* * Nothing shall interrupt this code path while holding the per-CPU * GHCB. The backup GHCB is only for NMIs interrupting this path.
@@ -153,7 +150,9 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long padd unsigned long npages) {struct psc_desc d = { -SNP_PAGE_STATE_PRIVATE ,svsm_get_caa (),svsm_get_caa_pa () +SNP_PAGE_STATE_PRIVATE , +rip_rel_ptr (&boot_svsm_ca_page), + boot_svsm_caa_pa };/*
@@ -176,7 +175,9 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr unsigned long npages) {struct psc_desc d = { -SNP_PAGE_STATE_SHARED ,svsm_get_caa (),svsm_get_caa_pa () +SNP_PAGE_STATE_SHARED , +rip_rel_ptr (&boot_svsm_ca_page), + boot_svsm_caa_pa };/*
arch/x86/coco/sev/core.c +22 -25
67 lines not on this page · raw message
arch/x86/include/asm/sev-internal.h +0 -16
@@ -62,22 +62,6 @@ DECLARE_PER_CPU(u64, svsm_caa_pa);extern u64 boot_svsm_caa_pa; -static __always_inline struct svsm_ca *svsm_get_caa(void) -{ -if (sev_cfg.use_cas) -return this_cpu_read (svsm_caa); -else -return rip_rel_ptr (&boot_svsm_ca_page); -} - -static __always_inline u64 svsm_get_caa_pa(void) -{ -if (sev_cfg.use_cas) -return this_cpu_read (svsm_caa_pa); -else -return boot_svsm_caa_pa; -} -enum es_result verify_exception_info (struct ghcb *ghcb,struct es_em_ctxt *ctxt);void vc_forward_exception (struct es_em_ctxt *ctxt);
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Use boot SVSM CA for all startup and init code
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: c54604fb7f2522fec5b97e86103ec49e539e80fe Gitweb: https://git.kernel.org/tip/c54604fb7f2522fec5b97e86103ec49e539e80fe Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:12 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:58:26 +02:00
x86/sev: Use boot SVSM CA for all startup and init code
To avoid having to reason about whether or not to use the per-CPU SVSM calling area when running startup and init code on the boot CPU, reuse the boot SVSM calling area as the per-CPU area for the BSP.
Thus, remove the need to make the per-CPU variables and associated state in sev_cfg accessible to the startup code once confined.
[ bp: Massage commit message. ]
--- arch/x86/boot/compressed/sev.c | 13 +-------- arch/x86/boot/startup/sev-startup.c | 11 +++---- arch/x86/coco/sev/core.c | 47 +++++++++++++--------------- arch/x86/include/asm/sev-internal.h | 16 +---------- 4 files changed, 28 insertions(+), 59 deletions(-)
arch/x86/boot/compressed/sev.c +0 -13
@@ -37,19 +37,6 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSED -extern u64 boot_svsm_caa_pa; - -struct svsm_ca *svsm_get_caa (void ) -{ -/* The decompressor is mapped 1:1 so VA == PA */ -return (struct svsm_ca *)boot_svsm_caa_pa; -} - -u64 svsm_get_caa_pa (void ) -{ -return boot_svsm_caa_pa; -} -u8 snp_vmpl;/* Include code for early handlers */
arch/x86/boot/startup/sev-startup.c +6 -5
@@ -50,9 +50,6 @@ u64 sev_secrets_pa __ro_after_init;/* For early boot SVSM communication */ struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; -DEFINE_PER_CPU(struct svsm_ca *, svsm_caa); -DEFINE_PER_CPU (u64, svsm_caa_pa); -/* * Nothing shall interrupt this code path while holding the per-CPU * GHCB. The backup GHCB is only for NMIs interrupting this path.
@@ -153,7 +150,9 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long padd unsigned long npages) {struct psc_desc d = { -SNP_PAGE_STATE_PRIVATE ,svsm_get_caa (),svsm_get_caa_pa () +SNP_PAGE_STATE_PRIVATE , +rip_rel_ptr (&boot_svsm_ca_page), + boot_svsm_caa_pa };/*
@@ -176,7 +175,9 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr unsigned long npages) {struct psc_desc d = { -SNP_PAGE_STATE_SHARED ,svsm_get_caa (),svsm_get_caa_pa () +SNP_PAGE_STATE_SHARED , +rip_rel_ptr (&boot_svsm_ca_page), + boot_svsm_caa_pa };/*
arch/x86/coco/sev/core.c +22 -25
67 lines not on this page · raw message
arch/x86/include/asm/sev-internal.h +0 -16
@@ -62,22 +62,6 @@ DECLARE_PER_CPU(u64, svsm_caa_pa);extern u64 boot_svsm_caa_pa; -static __always_inline struct svsm_ca *svsm_get_caa(void) -{ -if (sev_cfg.use_cas) -return this_cpu_read (svsm_caa); -else -return rip_rel_ptr (&boot_svsm_ca_page); -} - -static __always_inline u64 svsm_get_caa_pa(void) -{ -if (sev_cfg.use_cas) -return this_cpu_read (svsm_caa_pa); -else -return boot_svsm_caa_pa; -} -enum es_result verify_exception_info (struct ghcb *ghcb,struct es_em_ctxt *ctxt);void vc_forward_exception (struct es_em_ctxt *ctxt);
PATCH v7 10/22 x86/boot: Drop redundant RMPADJUST in SEV SVSM presence check
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
snp_vmpl will be assigned a non-zero value when executing at a VMPL other than 0, and this is inferred from a call to RMPADJUST, which only works when running at VMPL0.
This means that testing snp_vmpl is sufficient, and there is no need to perform the same check again.
--- arch/x86/boot/compressed/sev.c | 20 +++----------------- 1 file changed, 3 insertions(+), 17 deletions(-)
arch/x86/boot/compressed/sev.c +3 -17
@@ -404,30 +404,16 @@ void sev_enable(struct boot_params *bp)*/ if (sev_status &MSR_AMD64_SEV_SNP_ENABLED ) {u64 hv_features; -int ret; hv_features =get_hv_features ();if (!(hv_features &GHCB_HV_FT_SNP ))sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED );/* -* Enforce running at VMPL0 or with an SVSM. -* -* Use RMPADJUST (see the rmpadjust() function for a description of -* what the instruction does) to update the VMPL1 permissions of a -* page. If the guest is running at VMPL0, this will succeed. If the -* guest is running at any other VMPL, this will fail. Linux SNP guests -* only ever run at a single VMPL level so permission mask changes of a -* lesser-privileged VMPL are a don't-care. +* Running at VMPL0 is required unless an SVSM is present and +* the hypervisor supports the required SVSM GHCB events. */ - ret =rmpadjust ((unsigned long )&boot_ghcb_page,RMP_PG_SIZE_4K ,1 ); - -/* -* Running at VMPL0 is not required if an SVSM is present and the hypervisor -* supports the required SVSM GHCB events. -*/ -if (ret && - !(snp_vmpl && (hv_features &GHCB_HV_FT_SNP_MULTI_VMPL ))) +if (snp_vmpl >0 && !(hv_features &GHCB_HV_FT_SNP_MULTI_VMPL ))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_NOT_VMPL0 ); }
Signature · 1 line
2.51.0.268.g9569e192d0-goog
Borislav Petkov
maintainerRe: [PATCH v7 10/22] x86/boot: Drop redundant RMPADJUST in SEV SVSM presence check
Review
Quoted 7 lines · Ard Biesheuvel
(quoted diff) +1
-6- /* - * Running at VMPL0 is not required if an SVSM is present and the hypervisor - * supports the required SVSM GHCB events. - */ - if (ret && - !(snp_vmpl && (hv_features & GHCB_HV_FT_SNP_MULTI_VMPL))) + if (snp_vmpl > 0 && !(hv_features & GHCB_HV_FT_SNP_MULTI_VMPL))
Yeah, it is unsigned:
arch/x86/boot/compressed/sev.c +1 -1
@@ -415,7 +415,7 @@ void sev_enable(struct boot_params *bp)* Running at VMPL0 is required unless an SVSM is present and * the hypervisor supports the required SVSM GHCB events. */ -if (snp_vmpl >0 && !(hv_features &GHCB_HV_FT_SNP_MULTI_VMPL )) +if (snp_vmpl && !(hv_features &GHCB_HV_FT_SNP_MULTI_VMPL ))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_NOT_VMPL0 ); }
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 10/22] x86/boot: Drop redundant RMPADJUST in SEV SVSM presence check
Quoted 17 lines · Borislav Petkov
Review
Quoted 7 lines · Ard Biesheuvel
(quoted diff) +1
-6- /* - * Running at VMPL0 is not required if an SVSM is present and the hypervisor - * supports the required SVSM GHCB events. - */ - if (ret && - !(snp_vmpl && (hv_features & GHCB_HV_FT_SNP_MULTI_VMPL))) + if (snp_vmpl > 0 && !(hv_features & GHCB_HV_FT_SNP_MULTI_VMPL))Yeah, it is unsigned:
arch/x86/boot/compressed/sev.c +0
-0
arch/x86/boot/compressed/sev.c void sev_enable(struct boot_params *bp) · line 421
Quoted 8 lines
arch/x86/boot/compressed/sev.c +1
-1@@ -415,7 +415,7 @@ void sev_enable(struct boot_params *bp)* Running at VMPL0 is required unless an SVSM is present and * the hypervisor supports the required SVSM GHCB events. */ -if (snp_vmpl >0 && !(hv_features &GHCB_HV_FT_SNP_MULTI_VMPL )) +if (snp_vmpl && !(hv_features &GHCB_HV_FT_SNP_MULTI_VMPL ))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_NOT_VMPL0 ); }
Why is that better?
[tip: x86/sev] x86/boot: Drop redundant RMPADJUST in SEV SVSM presence check
In reply to Ard BiesheuvelThe following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 68a501d7fd82454525797971c6a0005ceeb93153 Gitweb: https://git.kernel.org/tip/68a501d7fd82454525797971c6a0005ceeb93153 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:13 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:09 +02:00
x86/boot: Drop redundant RMPADJUST in SEV SVSM presence check
snp_vmpl will be assigned a non-zero value when executing at a VMPL other than 0, and this is inferred from a call to RMPADJUST, which only works when running at VMPL0.
This means that testing snp_vmpl is sufficient, and there is no need to perform the same check again.
--- arch/x86/boot/compressed/sev.c | 20 +++----------------- 1 file changed, 3 insertions(+), 17 deletions(-)
arch/x86/boot/compressed/sev.c +3 -17
@@ -406,30 +406,16 @@ void sev_enable(struct boot_params *bp)*/ if (sev_status &MSR_AMD64_SEV_SNP_ENABLED ) {u64 hv_features; -int ret; hv_features =get_hv_features ();if (!(hv_features &GHCB_HV_FT_SNP ))sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED );/* -* Enforce running at VMPL0 or with an SVSM. -* -* Use RMPADJUST (see the rmpadjust() function for a description of -* what the instruction does) to update the VMPL1 permissions of a -* page. If the guest is running at VMPL0, this will succeed. If the -* guest is running at any other VMPL, this will fail. Linux SNP guests -* only ever run at a single VMPL level so permission mask changes of a -* lesser-privileged VMPL are a don't-care. +* Running at VMPL0 is required unless an SVSM is present and +* the hypervisor supports the required SVSM GHCB events. */ - ret =rmpadjust ((unsigned long )&boot_ghcb_page,RMP_PG_SIZE_4K ,1 ); - -/* -* Running at VMPL0 is not required if an SVSM is present and the hypervisor -* supports the required SVSM GHCB events. -*/ -if (ret && - !(snp_vmpl && (hv_features &GHCB_HV_FT_SNP_MULTI_VMPL ))) +if (snp_vmpl && !(hv_features &GHCB_HV_FT_SNP_MULTI_VMPL ))sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_NOT_VMPL0 ); }
PATCH v7 11/22 x86/boot: Provide PIC aliases for 5-level paging related constants
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Provide PIC aliases for the global variables related to 5-level paging, so that the startup code can access them in order to populate the kernel page tables.
--- arch/x86/kernel/head64.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kernel/head64.c +3 -0
@@ -52,10 +52,13 @@ SYM_PIC_ALIAS(next_early_pgt);pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);unsigned int __pgtable_l5_enabled__ro_after_init ; +SYM_PIC_ALIAS (__pgtable_l5_enabled);unsigned int pgdir_shift__ro_after_init =39 ;EXPORT_SYMBOL (pgdir_shift); +SYM_PIC_ALIAS (pgdir_shift);unsigned int ptrs_per_p4d__ro_after_init =1 ;EXPORT_SYMBOL (ptrs_per_p4d); +SYM_PIC_ALIAS (ptrs_per_p4d);unsigned long page_offset_base__ro_after_init = __PAGE_OFFSET_BASE_L4;EXPORT_SYMBOL (page_offset_base);
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/boot: Provide PIC aliases for 5-level paging related constants
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: f27906b287403af53be26341cf86d73798f15fe8 Gitweb: https://git.kernel.org/tip/f27906b287403af53be26341cf86d73798f15fe8 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:14 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:40 +02:00
x86/boot: Provide PIC aliases for 5-level paging related constants
Provide PIC aliases for the global variables related to 5-level paging, so that the startup code can access them in order to populate the kernel page tables.
--- arch/x86/kernel/head64.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kernel/head64.c +3 -0
@@ -52,10 +52,13 @@ SYM_PIC_ALIAS(next_early_pgt);pmdval_t early_pmd_flags = __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_NX);unsigned int __pgtable_l5_enabled__ro_after_init ; +SYM_PIC_ALIAS (__pgtable_l5_enabled);unsigned int pgdir_shift__ro_after_init =39 ;EXPORT_SYMBOL (pgdir_shift); +SYM_PIC_ALIAS (pgdir_shift);unsigned int ptrs_per_p4d__ro_after_init =1 ;EXPORT_SYMBOL (ptrs_per_p4d); +SYM_PIC_ALIAS (ptrs_per_p4d);unsigned long page_offset_base__ro_after_init = __PAGE_OFFSET_BASE_L4;EXPORT_SYMBOL (page_offset_base);
PATCH v7 12/22 x86/sev: Provide PIC aliases for SEV related data objects
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Provide PIC aliases for data objects that are shared between the SEV startup code and the SEV code that executes later. This is needed so that the confined startup code is permitted to access them.
This requires some of these variables to be moved into a source file that is not part of the startup code, as the PIC alias is already implied, and exporting variables in the opposite direction is not supported.
Move ghcb_version as well, but don't provide a PIC alias as it is not actually needed.
--- arch/x86/boot/compressed/sev.c | 3 ++ arch/x86/boot/startup/sev-shared.c | 19 ----------- arch/x86/boot/startup/sev-startup.c | 9 ------ arch/x86/coco/sev/core.c | 34 ++++++++++++++++++++ 4 files changed, 37 insertions(+), 28 deletions(-)
arch/x86/boot/compressed/sev.c +3 -0
@@ -38,6 +38,9 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSEDu8 snp_vmpl; +u16 ghcb_version; + +u64 boot_svsm_caa_pa;/* Include code for early handlers */ #include "../../boot/startup/sev-shared.c"
arch/x86/boot/startup/sev-shared.c +0 -19
@@ -19,25 +19,6 @@ #define WARN(condition, format...) (!!(condition))#endif -/* -* SVSM related information: -* During boot, the page tables are set up as identity mapped and later -* changed to use kernel virtual addresses. Maintain separate virtual and -* physical addresses for the CAA to allow SVSM functions to be used during -* early boot, both with identity mapped virtual addresses and proper kernel -* virtual addresses. -*/ -u64 boot_svsm_caa_pa__ro_after_init ; - -/* -* Since feature negotiation related variables are set early in the boot -* process they must reside in the .data section so as not to be zeroed -* out when the .bss section is later cleared. -* -* GHCB protocol version negotiated with the hypervisor. -*/ -u16 ghcb_version__ro_after_init ; -/* Copy of the SNP firmware's CPUID page. */ static struct snp_cpuid_table cpuid_table_copy__ro_after_init ;
arch/x86/boot/startup/sev-startup.c +0 -9
@@ -41,15 +41,6 @@#include <asm/cpuid/api.h> #include <asm/cmdline.h> -/* Bitmap of SEV features supported by the hypervisor */ -u64 sev_hv_features__ro_after_init ; - -/* Secrets page physical address from the CC blob */ -u64 sev_secrets_pa__ro_after_init ; - -/* For early boot SVSM communication */ -struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; -/* * Nothing shall interrupt this code path while holding the per-CPU * GHCB. The backup GHCB is only for NMIs interrupting this path.
arch/x86/coco/sev/core.c +34 -0
@@ -46,6 +46,29 @@#include <asm/cmdline.h> #include <asm/msr.h> +/* Bitmap of SEV features supported by the hypervisor */ +u64 sev_hv_features__ro_after_init ; +SYM_PIC_ALIAS (sev_hv_features); + +/* Secrets page physical address from the CC blob */ +u64 sev_secrets_pa__ro_after_init ; +SYM_PIC_ALIAS (sev_secrets_pa); + +/* For early boot SVSM communication */ +struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; +SYM_PIC_ALIAS (boot_svsm_ca_page); + +/* +* SVSM related information: +* During boot, the page tables are set up as identity mapped and later +* changed to use kernel virtual addresses. Maintain separate virtual and +* physical addresses for the CAA to allow SVSM functions to be used during +* early boot, both with identity mapped virtual addresses and proper kernel +* virtual addresses. +*/ +u64 boot_svsm_caa_pa__ro_after_init ; +SYM_PIC_ALIAS (boot_svsm_caa_pa); + DEFINE_PER_CPU(struct svsm_ca *, svsm_caa);DEFINE_PER_CPU (u64, svsm_caa_pa);
@@ -119,6 +142,17 @@ DEFINE_PER_CPU(struct sev_es_save_area *, sev_vmsa);*/ u8 snp_vmpl__ro_after_init ;EXPORT_SYMBOL_GPL (snp_vmpl); +SYM_PIC_ALIAS (snp_vmpl); + +/* +* Since feature negotiation related variables are set early in the boot +* process they must reside in the .data section so as not to be zeroed +* out when the .bss section is later cleared. +* +* GHCB protocol version negotiated with the hypervisor. +*/ +u16 ghcb_version__ro_after_init ; +SYM_PIC_ALIAS (ghcb_version);/* For early boot hypervisor communication in SEV-ES enabled guests */ static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);
Signature · 1 line
2.51.0.268.g9569e192d0-goog
Borislav Petkov
maintainerRe: [PATCH v7 12/22] x86/sev: Provide PIC aliases for SEV related data objects
Quoted 6 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
Provide PIC aliases for data objects that are shared between the SEV startup code and the SEV code that executes later. This is needed so that the confined startup code is permitted to access them.
Quoted 7 lines
This requires some of these variables to be moved into a source file that is not part of the startup code, as the PIC alias is already implied, and exporting variables in the opposite direction is not supported.
Move
ghcb_versionas well, but don't provide a PIC alias as it is not actually needed.
I see
SYM_PIC_ALIAS(ghcb_version);
below.
Stale commit message?
Quoted 128 lines
Signed-off-by Ard Biesheuvel <[email protected]>--- arch/x86/boot/compressed/sev.c | 3 ++ arch/x86/boot/startup/sev-shared.c | 19 ----------- arch/x86/boot/startup/sev-startup.c | 9 ------ arch/x86/coco/sev/core.c | 34 ++++++++++++++++++++ 4 files changed, 37 insertions(+), 28 deletions(-)arch/x86/boot/compressed/sev.c +3
-0@@ -38,6 +38,9 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSEDu8 snp_vmpl; +u16 ghcb_version; + +u64 boot_svsm_caa_pa;/* Include code for early handlers */ #include "../../boot/startup/sev-shared.c" arch/x86/boot/startup/sev-shared.c +0
-19@@ -19,25 +19,6 @@ #define WARN(condition, format...) (!!(condition))#endif -/* -* SVSM related information: -* During boot, the page tables are set up as identity mapped and later -* changed to use kernel virtual addresses. Maintain separate virtual and -* physical addresses for the CAA to allow SVSM functions to be used during -* early boot, both with identity mapped virtual addresses and proper kernel -* virtual addresses. -*/ -u64 boot_svsm_caa_pa__ro_after_init ; - -/* -* Since feature negotiation related variables are set early in the boot -* process they must reside in the .data section so as not to be zeroed -* out when the .bss section is later cleared. -* -* GHCB protocol version negotiated with the hypervisor. -*/ -u16 ghcb_version__ro_after_init ; -/* Copy of the SNP firmware's CPUID page. */ static struct snp_cpuid_table cpuid_table_copy__ro_after_init ;arch/x86/boot/startup/sev-startup.c +0
-9@@ -41,15 +41,6 @@#include <asm/cpuid/api.h> #include <asm/cmdline.h> -/* Bitmap of SEV features supported by the hypervisor */ -u64 sev_hv_features__ro_after_init ; - -/* Secrets page physical address from the CC blob */ -u64 sev_secrets_pa__ro_after_init ; - -/* For early boot SVSM communication */ -struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; -/* * Nothing shall interrupt this code path while holding the per-CPU * GHCB. The backup GHCB is only for NMIs interrupting this path. arch/x86/coco/sev/core.c +34
-0@@ -46,6 +46,29 @@#include <asm/cmdline.h> #include <asm/msr.h> +/* Bitmap of SEV features supported by the hypervisor */ +u64 sev_hv_features__ro_after_init ; +SYM_PIC_ALIAS (sev_hv_features); + +/* Secrets page physical address from the CC blob */ +u64 sev_secrets_pa__ro_after_init ; +SYM_PIC_ALIAS (sev_secrets_pa); + +/* For early boot SVSM communication */ +struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; +SYM_PIC_ALIAS (boot_svsm_ca_page); + +/* +* SVSM related information: +* During boot, the page tables are set up as identity mapped and later +* changed to use kernel virtual addresses. Maintain separate virtual and +* physical addresses for the CAA to allow SVSM functions to be used during +* early boot, both with identity mapped virtual addresses and proper kernel +* virtual addresses. +*/ +u64 boot_svsm_caa_pa__ro_after_init ; +SYM_PIC_ALIAS (boot_svsm_caa_pa); + DEFINE_PER_CPU(struct svsm_ca *, svsm_caa);DEFINE_PER_CPU (u64, svsm_caa_pa);@@ -119,6 +142,17 @@ DEFINE_PER_CPU(struct sev_es_save_area *, sev_vmsa);*/ u8 snp_vmpl__ro_after_init ;EXPORT_SYMBOL_GPL (snp_vmpl); +SYM_PIC_ALIAS (snp_vmpl); + +/* +* Since feature negotiation related variables are set early in the boot +* process they must reside in the .data section so as not to be zeroed +* out when the .bss section is later cleared. +* +* GHCB protocol version negotiated with the hypervisor. +*/ +u16 ghcb_version__ro_after_init ; +SYM_PIC_ALIAS (ghcb_version);/* For early boot hypervisor communication in SEV-ES enabled guests */ static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);Signature · 1 line
2.51.0.268.g9569e192d0-goog
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerRe: [PATCH v7 12/22] x86/sev: Provide PIC aliases for SEV related data objects
Quoted 16 lines · Borislav Petkov
Quoted 6 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
Provide PIC aliases for data objects that are shared between the SEV startup code and the SEV code that executes later. This is needed so that the confined startup code is permitted to access them.
Quoted 7 lines
This requires some of these variables to be moved into a source file that is not part of the startup code, as the PIC alias is already implied, and exporting variables in the opposite direction is not supported.
Move
ghcb_versionas well, but don't provide a PIC alias as it is not actually needed.
Quoted 8 lines
I see
SYM_PIC_ALIAS(ghcb_version);below.
Stale commit message?
Yes, as a result of
https://lore.kernel.org/all/20250530165507.GBaDni69NqgOi4mFNn@fat_crate.local/
it was exported but I failed to update the commit log accordingly.
Borislav Petkov
maintainerRe: [PATCH v7 12/22] x86/sev: Provide PIC aliases for SEV related data objects
Quoted 1 line · Ard Biesheuvel
it was exported but I failed to update the commit log accordingly.
Ack, zapped commit message sentence locally.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette[tip: x86/sev] x86/sev: Provide PIC aliases for SEV related data objects
In reply to Ard BiesheuvelThe following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 9723dd0c705eb626bac2cd06b83a2c8514ed697a Gitweb: https://git.kernel.org/tip/9723dd0c705eb626bac2cd06b83a2c8514ed697a Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:15 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:43 +02:00
x86/sev: Provide PIC aliases for SEV related data objects
Provide PIC aliases for data objects that are shared between the SEV startup code and the SEV code that executes later. This is needed so that the confined startup code is permitted to access them.
This requires some of these variables to be moved into a source file that is not part of the startup code, as the PIC alias is already implied, and exporting variables in the opposite direction is not supported.
--- arch/x86/boot/compressed/sev.c | 3 ++- arch/x86/boot/startup/sev-shared.c | 19 +---------------- arch/x86/boot/startup/sev-startup.c | 9 +------- arch/x86/coco/sev/core.c | 34 ++++++++++++++++++++++++++++- 4 files changed, 37 insertions(+), 28 deletions(-)
arch/x86/boot/compressed/sev.c +3 -0
@@ -38,6 +38,9 @@ struct ghcb *boot_ghcb;#define __BOOT_COMPRESSEDu8 snp_vmpl; +u16 ghcb_version; + +u64 boot_svsm_caa_pa;/* Include code for early handlers */ #include "../../boot/startup/sev-shared.c"
arch/x86/boot/startup/sev-shared.c +0 -19
@@ -19,25 +19,6 @@ #define WARN(condition, format...) (!!(condition))#endif -/* -* SVSM related information: -* During boot, the page tables are set up as identity mapped and later -* changed to use kernel virtual addresses. Maintain separate virtual and -* physical addresses for the CAA to allow SVSM functions to be used during -* early boot, both with identity mapped virtual addresses and proper kernel -* virtual addresses. -*/ -u64 boot_svsm_caa_pa__ro_after_init ; - -/* -* Since feature negotiation related variables are set early in the boot -* process they must reside in the .data section so as not to be zeroed -* out when the .bss section is later cleared. -* -* GHCB protocol version negotiated with the hypervisor. -*/ -u16 ghcb_version__ro_after_init ; -/* Copy of the SNP firmware's CPUID page. */ static struct snp_cpuid_table cpuid_table_copy__ro_after_init ;
arch/x86/boot/startup/sev-startup.c +0 -9
@@ -41,15 +41,6 @@#include <asm/cpuid/api.h> #include <asm/cmdline.h> -/* Bitmap of SEV features supported by the hypervisor */ -u64 sev_hv_features__ro_after_init ; - -/* Secrets page physical address from the CC blob */ -u64 sev_secrets_pa__ro_after_init ; - -/* For early boot SVSM communication */ -struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; -/* * Nothing shall interrupt this code path while holding the per-CPU * GHCB. The backup GHCB is only for NMIs interrupting this path.
arch/x86/coco/sev/core.c +34 -0
@@ -46,6 +46,29 @@#include <asm/cmdline.h> #include <asm/msr.h> +/* Bitmap of SEV features supported by the hypervisor */ +u64 sev_hv_features__ro_after_init ; +SYM_PIC_ALIAS (sev_hv_features); + +/* Secrets page physical address from the CC blob */ +u64 sev_secrets_pa__ro_after_init ; +SYM_PIC_ALIAS (sev_secrets_pa); + +/* For early boot SVSM communication */ +struct svsm_ca boot_svsm_ca_page__aligned(PAGE_SIZE) ; +SYM_PIC_ALIAS (boot_svsm_ca_page); + +/* +* SVSM related information: +* During boot, the page tables are set up as identity mapped and later +* changed to use kernel virtual addresses. Maintain separate virtual and +* physical addresses for the CAA to allow SVSM functions to be used during +* early boot, both with identity mapped virtual addresses and proper kernel +* virtual addresses. +*/ +u64 boot_svsm_caa_pa__ro_after_init ; +SYM_PIC_ALIAS (boot_svsm_caa_pa); + DEFINE_PER_CPU(struct svsm_ca *, svsm_caa);DEFINE_PER_CPU (u64, svsm_caa_pa);
@@ -119,6 +142,17 @@ DEFINE_PER_CPU(struct sev_es_save_area *, sev_vmsa);*/ u8 snp_vmpl__ro_after_init ;EXPORT_SYMBOL_GPL (snp_vmpl); +SYM_PIC_ALIAS (snp_vmpl); + +/* +* Since feature negotiation related variables are set early in the boot +* process they must reside in the .data section so as not to be zeroed +* out when the .bss section is later cleared. +* +* GHCB protocol version negotiated with the hypervisor. +*/ +u16 ghcb_version__ro_after_init ; +SYM_PIC_ALIAS (ghcb_version);/* For early boot hypervisor communication in SEV-ES enabled guests */ static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);
PATCH v7 13/22 x86/sev: Move __sev_[get|put]_ghcb() into separate noinstr object
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Rename sev-nmi.c to noinstr.c, and move the get/put GHCB routines into it too, which are also annotated as 'noinstr' and suffer from the same problem as the NMI code, i.e., that GCC may ignore the __no_sanitize_address__ function attribute implied by 'noinstr' and insert KASAN instrumentation anyway.
---
arch/x86/boot/startup/sev-startup.c | 74 --------------------
arch/x86/coco/sev/Makefile | 8 +--
arch/x86/coco/sev/{sev-nmi.c => noinstr.c} | 74 ++++++++++++++++++++
3 files changed, 78 insertions(+), 78 deletions(-)arch/x86/boot/startup/sev-startup.c +0 -74
@@ -41,83 +41,9 @@#include <asm/cpuid/api.h> #include <asm/cmdline.h> -/* -* Nothing shall interrupt this code path while holding the per-CPU -* GHCB. The backup GHCB is only for NMIs interrupting this path. -* -* Callers must disable local interrupts around it. -*/ -noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) -{ -struct sev_es_runtime_data *data; -struct ghcb *ghcb; - -WARN_ON (!irqs_disabled ()); - - data =this_cpu_read (runtime_data); - ghcb = &data->ghcb_page; - -if (unlikely (data->ghcb_active)) { -/* GHCB is already in use - save its contents */ - -if (unlikely (data->backup_ghcb_active)) { -/* -* Backup-GHCB is also already in use. There is no way -* to continue here so just kill the machine. To make -* panic() work, mark GHCBs inactive so that messages -* can be printed out. -*/ - data->ghcb_active = false; - data->backup_ghcb_active = false; - -instrumentation_begin (); -panic ("Unable to handle #VC exception! GHCB and Backup GHCB are already in use" ); -instrumentation_end (); - } - -/* Mark backup_ghcb active before writing to it */ - data->backup_ghcb_active = true; - - state->ghcb = &data->backup_ghcb; - -/* Backup GHCB content */ - *state->ghcb = *ghcb; - }else { - state->ghcb =NULL ; - data->ghcb_active = true; - } - -return ghcb; -} -/* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" -noinstr void __sev_put_ghcb(struct ghcb_state *state) -{ -struct sev_es_runtime_data *data; -struct ghcb *ghcb; - -WARN_ON (!irqs_disabled ()); - - data =this_cpu_read (runtime_data); - ghcb = &data->ghcb_page; - -if (state->ghcb) { -/* Restore GHCB from Backup */ - *ghcb = *state->ghcb; - data->backup_ghcb_active = false; - state->ghcb =NULL ; - }else { -/* -* Invalidate the GHCB so a VMGEXIT instruction issued -* from userspace won't appear to be valid. -*/ -vc_ghcb_invalidate (ghcb); - data->ghcb_active = false; - } -} - void __headearly_set_pages_state (unsigned long vaddr,unsigned long paddr,unsigned long npages,const struct psc_desc *desc)
arch/x86/coco/sev/Makefile +4 -4
@@ -1,10 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 -obj-y += core.o sev-nmi.o vc-handle.o +obj-y += core.o noinstr.o vc-handle.o # Clang 14 and older may fail to respect __no_sanitize_undefined when inlining -UBSAN_SANITIZE_sev-nmi.o := n +UBSAN_SANITIZE_noinstr.o := n # GCC may fail to respect __no_sanitize_address or __no_kcsan when inlining -KASAN_SANITIZE_sev-nmi.o := n -KCSAN_SANITIZE_sev-nmi.o := n +KASAN_SANITIZE_noinstr.o := n +KCSAN_SANITIZE_noinstr.o := n
arch/x86/coco/sev/noinstr.c renamed from arch/x86/coco/sev/sev-nmi.c +74 -0
77 lines not on this page · raw message
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Move __sev_[get|put]_ghcb() into separate noinstr object
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: d4077e6ad35121b97f3233da5d60763de3d23df9 Gitweb: https://git.kernel.org/tip/d4077e6ad35121b97f3233da5d60763de3d23df9 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:16 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:46 +02:00
x86/sev: Move __sev_[get|put]_ghcb() into separate noinstr object
Rename sev-nmi.c to noinstr.c, and move the get/put GHCB routines into it too, which are also annotated as 'noinstr' and suffer from the same problem as the NMI code, i.e., that GCC may ignore the __no_sanitize_address__ function attribute implied by 'noinstr' and insert KASAN instrumentation anyway.
--- arch/x86/boot/startup/sev-startup.c | 74 +----------- arch/x86/coco/sev/Makefile | 8 +- arch/x86/coco/sev/noinstr.c | 182 +++++++++++++++++++++++++++- arch/x86/coco/sev/sev-nmi.c | 108 +---------------- 4 files changed, 186 insertions(+), 186 deletions(-) create mode 100644 arch/x86/coco/sev/noinstr.c delete mode 100644 arch/x86/coco/sev/sev-nmi.c
arch/x86/boot/startup/sev-startup.c +0 -74
@@ -41,83 +41,9 @@#include <asm/cpuid/api.h> #include <asm/cmdline.h> -/* -* Nothing shall interrupt this code path while holding the per-CPU -* GHCB. The backup GHCB is only for NMIs interrupting this path. -* -* Callers must disable local interrupts around it. -*/ -noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) -{ -struct sev_es_runtime_data *data; -struct ghcb *ghcb; - -WARN_ON (!irqs_disabled ()); - - data =this_cpu_read (runtime_data); - ghcb = &data->ghcb_page; - -if (unlikely (data->ghcb_active)) { -/* GHCB is already in use - save its contents */ - -if (unlikely (data->backup_ghcb_active)) { -/* -* Backup-GHCB is also already in use. There is no way -* to continue here so just kill the machine. To make -* panic() work, mark GHCBs inactive so that messages -* can be printed out. -*/ - data->ghcb_active = false; - data->backup_ghcb_active = false; - -instrumentation_begin (); -panic ("Unable to handle #VC exception! GHCB and Backup GHCB are already in use" ); -instrumentation_end (); - } - -/* Mark backup_ghcb active before writing to it */ - data->backup_ghcb_active = true; - - state->ghcb = &data->backup_ghcb; - -/* Backup GHCB content */ - *state->ghcb = *ghcb; - }else { - state->ghcb =NULL ; - data->ghcb_active = true; - } - -return ghcb; -} -/* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" -noinstr void __sev_put_ghcb(struct ghcb_state *state) -{ -struct sev_es_runtime_data *data; -struct ghcb *ghcb; - -WARN_ON (!irqs_disabled ()); - - data =this_cpu_read (runtime_data); - ghcb = &data->ghcb_page; - -if (state->ghcb) { -/* Restore GHCB from Backup */ - *ghcb = *state->ghcb; - data->backup_ghcb_active = false; - state->ghcb =NULL ; - }else { -/* -* Invalidate the GHCB so a VMGEXIT instruction issued -* from userspace won't appear to be valid. -*/ -vc_ghcb_invalidate (ghcb); - data->ghcb_active = false; - } -} - void __headearly_set_pages_state (unsigned long vaddr,unsigned long paddr,unsigned long npages,const struct psc_desc *desc)
arch/x86/coco/sev/Makefile +4 -4
@@ -1,10 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 -obj-y += core.o sev-nmi.o vc-handle.o +obj-y += core.o noinstr.o vc-handle.o # Clang 14 and older may fail to respect __no_sanitize_undefined when inlining -UBSAN_SANITIZE_sev-nmi.o := n +UBSAN_SANITIZE_noinstr.o := n # GCC may fail to respect __no_sanitize_address or __no_kcsan when inlining -KASAN_SANITIZE_sev-nmi.o := n -KCSAN_SANITIZE_sev-nmi.o := n +KASAN_SANITIZE_noinstr.o := n +KCSAN_SANITIZE_noinstr.o := n
arch/x86/coco/sev/noinstr.c new file +182 -0
182 lines not on this page · raw message
arch/x86/coco/sev/sev-nmi.c deleted +0 -108
108 lines not on this page · raw message
PATCH v7 14/22 x86/sev: Export startup routines for later use
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Create aliases that expose routines that are part of the startup code to other code in the core kernel, so that they can be called later as well.
--- arch/x86/boot/startup/exports.h | 14 ++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 2 ++ 2 files changed, 16 insertions(+)
arch/x86/boot/startup/exports.h new file +14 -0
@@ -0,0 +1,14 @@ + +/* +* The symbols below are functions that are implemented by the startup code, +* but called at runtime by the SEV code residing in the core kernel. +*/ +PROVIDE (early_set_pages_state = __pi_early_set_pages_state); +PROVIDE (early_snp_set_memory_private = __pi_early_snp_set_memory_private); +PROVIDE (early_snp_set_memory_shared = __pi_early_snp_set_memory_shared); +PROVIDE (get_hv_features = __pi_get_hv_features); +PROVIDE (sev_es_terminate = __pi_sev_es_terminate); +PROVIDE (snp_cpuid = __pi_snp_cpuid); +PROVIDE (snp_cpuid_get_table = __pi_snp_cpuid_get_table); +PROVIDE (svsm_issue_call = __pi_svsm_issue_call); +PROVIDE (svsm_process_result_codes = __pi_svsm_process_result_codes);
arch/x86/kernel/vmlinux.lds.S +2 -0
@@ -535,3 +535,5 @@ xen_elfnote_entry_value = xen_elfnote_phys32_entry_value = ABSOLUTE(xen_elfnote_phys32_entry) + ABSOLUTE(pvh_start_xen - LOAD_OFFSET); #endif + +#include "../boot/startup/exports.h"
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/sev: Export startup routines for later use
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 05ce314ba5155d57c86f8f276cb17f78ac5fb4f0 Gitweb: https://git.kernel.org/tip/05ce314ba5155d57c86f8f276cb17f78ac5fb4f0 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:17 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:49 +02:00
x86/sev: Export startup routines for later use
Create aliases that expose routines that are part of the startup code to other code in the core kernel, so that they can be called later as well.
--- arch/x86/boot/startup/exports.h | 14 ++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 2 ++ 2 files changed, 16 insertions(+) create mode 100644 arch/x86/boot/startup/exports.h
arch/x86/boot/startup/exports.h new file +14 -0
@@ -0,0 +1,14 @@ + +/* +* The symbols below are functions that are implemented by the startup code, +* but called at runtime by the SEV code residing in the core kernel. +*/ +PROVIDE (early_set_pages_state = __pi_early_set_pages_state); +PROVIDE (early_snp_set_memory_private = __pi_early_snp_set_memory_private); +PROVIDE (early_snp_set_memory_shared = __pi_early_snp_set_memory_shared); +PROVIDE (get_hv_features = __pi_get_hv_features); +PROVIDE (sev_es_terminate = __pi_sev_es_terminate); +PROVIDE (snp_cpuid = __pi_snp_cpuid); +PROVIDE (snp_cpuid_get_table = __pi_snp_cpuid_get_table); +PROVIDE (svsm_issue_call = __pi_svsm_issue_call); +PROVIDE (svsm_process_result_codes = __pi_svsm_process_result_codes);
arch/x86/kernel/vmlinux.lds.S +2 -0
@@ -535,3 +535,5 @@ xen_elfnote_entry_value = xen_elfnote_phys32_entry_value = ABSOLUTE(xen_elfnote_phys32_entry) + ABSOLUTE(pvh_start_xen - LOAD_OFFSET); #endif + +#include "../boot/startup/exports.h"
PATCH v7 15/22 objtool: Add action to check for absence of absolute relocations
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The x86 startup code must not use absolute references to code or data, as it executes before the kernel virtual mapping is up.
Add an action to objtool to check all allocatable sections (with the exception of __patchable_function_entries, which uses absolute references for nebulous reasons) and raise an error if any absolute references are found.
Note that debug sections typically contain lots of absolute references too, but those are not allocatable so they will be ignored.
--- tools/objtool/arch/x86/decode.c | 12 ++++++ tools/objtool/builtin-check.c | 2 + tools/objtool/check.c | 44 ++++++++++++++++++++ tools/objtool/include/objtool/arch.h | 1 + tools/objtool/include/objtool/builtin.h | 1 + 5 files changed, 60 insertions(+)
tools/objtool/arch/x86/decode.c +12 -0
@@ -880,3 +880,15 @@ unsigned int arch_reloc_size(struct reloc *reloc)return 8 ; } } + +bool arch_absolute_reloc (struct elf *elf,struct reloc *reloc) +{ +switch (reloc_type (reloc)) { +case R_X86_64_32 : +case R_X86_64_32S : +case R_X86_64_64 : +return true; +default : +return false; + } +}
tools/objtool/builtin-check.c +2 -0
@@ -87,6 +87,7 @@ static const struct option check_options[] = {OPT_BOOLEAN ('t' ,"static-call" , &opts.static_call,"annotate static calls" ),OPT_BOOLEAN ('u' ,"uaccess" , &opts.uaccess,"validate uaccess rules for SMAP" ),OPT_BOOLEAN (0 ,"cfi" , &opts.cfi,"annotate kernel control flow integrity (kCFI) function preambles" ), +OPT_BOOLEAN (0 ,"noabs" , &opts.noabs,"reject absolute references in allocatable sections" ),OPT_CALLBACK_OPTARG (0 ,"dump" ,NULL ,NULL ,"orc" ,"dump metadata" , parse_dump), OPT_GROUP("Options:"),
@@ -162,6 +163,7 @@ static bool opts_valid(void) opts.hack_noinstr || opts.ibt || opts.mcount || + opts.noabs || opts.noinstr || opts.orc || opts.retpoline ||
tools/objtool/check.c +44 -0
56 lines not on this page · raw message
tools/objtool/include/objtool/arch.h +1 -0
@@ -97,6 +97,7 @@ bool arch_is_embedded_insn(struct symbol *sym);int arch_rewrite_retpolines (struct objtool_file *file);bool arch_pc_relative_reloc (struct reloc *reloc); +bool arch_absolute_reloc (struct elf *elf,struct reloc *reloc);unsigned int arch_reloc_size (struct reloc *reloc);unsigned long arch_jump_table_sym_offset (struct reloc *reloc,struct reloc *table);
tools/objtool/include/objtool/builtin.h +1 -0
@@ -26,6 +26,7 @@ struct opts {bool uaccess;int prefix;bool cfi; +bool noabs;/* options: */ bool backtrace;
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] objtool: Add action to check for absence of absolute relocations
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 0d6e4563fc03d83f948e6a6f7963cc31a4c81914 Gitweb: https://git.kernel.org/tip/0d6e4563fc03d83f948e6a6f7963cc31a4c81914 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:18 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:51 +02:00
objtool: Add action to check for absence of absolute relocations
The x86 startup code must not use absolute references to code or data, as it executes before the kernel virtual mapping is up.
Add an action to objtool to check all allocatable sections (with the exception of __patchable_function_entries, which uses absolute references for nebulous reasons) and raise an error if any absolute references are found.
Note that debug sections typically contain lots of absolute references too, but those are not allocatable so they will be ignored.
--- tools/objtool/arch/x86/decode.c | 12 +++++++- tools/objtool/builtin-check.c | 2 +- tools/objtool/check.c | 44 ++++++++++++++++++++++++- tools/objtool/include/objtool/arch.h | 1 +- tools/objtool/include/objtool/builtin.h | 1 +- 5 files changed, 60 insertions(+)
tools/objtool/arch/x86/decode.c +12 -0
@@ -880,3 +880,15 @@ unsigned int arch_reloc_size(struct reloc *reloc)return 8 ; } } + +bool arch_absolute_reloc (struct elf *elf,struct reloc *reloc) +{ +switch (reloc_type (reloc)) { +case R_X86_64_32 : +case R_X86_64_32S : +case R_X86_64_64 : +return true; +default : +return false; + } +}
tools/objtool/builtin-check.c +2 -0
@@ -87,6 +87,7 @@ static const struct option check_options[] = {OPT_BOOLEAN ('t' ,"static-call" , &opts.static_call,"annotate static calls" ),OPT_BOOLEAN ('u' ,"uaccess" , &opts.uaccess,"validate uaccess rules for SMAP" ),OPT_BOOLEAN (0 ,"cfi" , &opts.cfi,"annotate kernel control flow integrity (kCFI) function preambles" ), +OPT_BOOLEAN (0 ,"noabs" , &opts.noabs,"reject absolute references in allocatable sections" ),OPT_CALLBACK_OPTARG (0 ,"dump" ,NULL ,NULL ,"orc" ,"dump metadata" , parse_dump), OPT_GROUP("Options:"),
@@ -162,6 +163,7 @@ static bool opts_valid(void) opts.hack_noinstr || opts.ibt || opts.mcount || + opts.noabs || opts.noinstr || opts.orc || opts.retpoline ||
tools/objtool/check.c +44 -0
56 lines not on this page · raw message
tools/objtool/include/objtool/arch.h +1 -0
@@ -97,6 +97,7 @@ bool arch_is_embedded_insn(struct symbol *sym);int arch_rewrite_retpolines (struct objtool_file *file);bool arch_pc_relative_reloc (struct reloc *reloc); +bool arch_absolute_reloc (struct elf *elf,struct reloc *reloc);unsigned int arch_reloc_size (struct reloc *reloc);unsigned long arch_jump_table_sym_offset (struct reloc *reloc,struct reloc *table);
tools/objtool/include/objtool/builtin.h +1 -0
@@ -26,6 +26,7 @@ struct opts {bool uaccess;int prefix;bool cfi; +bool noabs;/* options: */ bool backtrace;
Re: [PATCH v7 15/22] objtool: Add action to check for absence of absolute relocations
In reply to Ard BiesheuvelQuoted 32 lines · Ard Biesheuvel
tools/objtool/check.c +24
-0@@ -4686,6 +4686,47 @@ static void disas_warned_funcs(struct objtool_file *file)disas_funcs (funcs); } +__weak bool arch_absolute_reloc (struct elf *elf,struct reloc *reloc) +{ +unsigned int type =reloc_type (reloc); +size_t sz =elf_addr_size (elf); + +return (sz ==8 ) ? (type ==R_ABS64 ) : (type ==R_ABS32 ); +} + +static int check_abs_references (struct objtool_file *file) +{ +struct section *sec; +struct reloc *reloc; +int ret =0 ; + +for_each_sec (file, sec) { +/* absolute references in non-loadable sections are fine */ +if (!(sec->sh.sh_flags &SHF_ALLOC )) +continue ; + +/* section must have an associated .rela section */ +if (!sec->rsec) +continue ; + +/*
tools/objtool/check.c static void disas_warned_funcs(struct objtool_file *file) · line 4722
Quoted 10 lines
tools/objtool/check.c +10
-0+* Special case for compiler generated metadata that is not +* consumed until after boot. +*/ +if (!strcmp (sec->name,"__patchable_function_entries" )) +continue ; + +for_each_reloc (sec->rsec, reloc) { +if (arch_absolute_reloc (file->elf, reloc)) { +WARN ("section %s has absolute relocation at offset 0x%lx" , + sec->name,reloc_offset (reloc));
This is wrong for a 32-bit host:
In file included from check.c:16:check.c: In function ‘check_abs_references’:/home/andreas/src/linux/linux-6.18-rc1/tools/objtool/include/objtool/warn.h:47:3: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 7 has type ‘u64’ {aka ‘long long unsigned int’} [-Werror=format=]
47 | "%s%s%s: objtool" extra ": " format "\n", \
| ^~~~~~~~~~~~~~~~~
/home/andreas/src/linux/linux-6.18-rc1/tools/objtool/include/objtool/warn.h:54:2: note: in expansion of macro ‘___WARN’
54 | ___WARN(severity, "", format, ##__VA_ARGS__)
| ^~~~~~~/home/andreas/src/linux/linux-6.18-rc1/tools/objtool/include/objtool/warn.h:74:27: note: in expansion of macro ‘__WARN’
74 | #define WARN(format, ...) __WARN(WARN_STR, format, ##__VA_ARGS__)
| ^~~~~~
check.c:4713:5: note: in expansion of macro ‘WARN’
4713 | WARN("section %s has absolute relocation at offset 0x%lx",
| ^~~~cc1: all warnings being treated as errors
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
PATCH v7 16/22 x86/boot: Check startup code for absence of absolute relocations
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Invoke objtool on each startup code object individually to check for the absence of absolute relocations. This is needed because this code will be invoked from the 1:1 mapping of memory before those absolute virtual addresses (which are derived from the kernel virtual base address provided to the linker and possibly shifted at boot) are mapped.
Only objects built under arch/x86/boot/startup/ have this restriction, and once they have been incorporated into vmlinux.o, this distinction is difficult to make. So force the invocation of objtool for each object file individually, even if objtool is deferred to vmlinux.o for the rest of the build. In the latter case, only pass --noabs and nothing else; otherwise, append it to the existing objtool command line.
--- arch/x86/boot/startup/Makefile | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/boot/startup/Makefile +8 -0
@@ -19,6 +19,7 @@ KCOV_INSTRUMENT := n obj-$(CONFIG_X86_64) += gdt_idt.o map_kernel.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += sme.o sev-startup.o +pi-objs := $(patsubst %.o,$(obj)/%.o,$(obj-y)) lib-$(CONFIG_X86_64) += la57toggle.o lib-$(CONFIG_EFI_MIXED) += efi-mixed.o
@@ -28,3 +29,10 @@ lib-$(CONFIG_EFI_MIXED) += efi-mixed.o # to be linked into the decompressor or the EFI stub but not vmlinux # $(patsubst %.o,$(obj)/%.o,$(lib-y)): OBJECT_FILES_NON_STANDARD := y + +# +# Invoke objtool for each object individually to check for absolute +# relocations, even if other objtool actions are being deferred. +# +$(pi-objs): objtool-enabled = 1 +$(pi-objs): objtool-args = $(if $(delay-objtool),,$(objtool-args-y)) --noabs
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/boot: Check startup code for absence of absolute relocations
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 296650c8ac4f18e886dd2a606152c00adf527219 Gitweb: https://git.kernel.org/tip/296650c8ac4f18e886dd2a606152c00adf527219 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:19 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:54 +02:00
x86/boot: Check startup code for absence of absolute relocations
Invoke objtool on each startup code object individually to check for the absence of absolute relocations. This is needed because this code will be invoked from the 1:1 mapping of memory before those absolute virtual addresses (which are derived from the kernel virtual base address provided to the linker and possibly shifted at boot) are mapped.
Only objects built under arch/x86/boot/startup/ have this restriction, and once they have been incorporated into vmlinux.o, this distinction is difficult to make. So force the invocation of objtool for each object file individually, even if objtool is deferred to vmlinux.o for the rest of the build. In the latter case, only pass --noabs and nothing else; otherwise, append it to the existing objtool command line.
--- arch/x86/boot/startup/Makefile | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/boot/startup/Makefile +8 -0
@@ -19,6 +19,7 @@ KCOV_INSTRUMENT := n obj-$(CONFIG_X86_64) += gdt_idt.o map_kernel.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += sme.o sev-startup.o +pi-objs := $(patsubst %.o,$(obj)/%.o,$(obj-y)) lib-$(CONFIG_X86_64) += la57toggle.o lib-$(CONFIG_EFI_MIXED) += efi-mixed.o
@@ -28,3 +29,10 @@ lib-$(CONFIG_EFI_MIXED) += efi-mixed.o # to be linked into the decompressor or the EFI stub but not vmlinux # $(patsubst %.o,$(obj)/%.o,$(lib-y)): OBJECT_FILES_NON_STANDARD := y + +# +# Invoke objtool for each object individually to check for absolute +# relocations, even if other objtool actions are being deferred. +# +$(pi-objs): objtool-enabled = 1 +$(pi-objs): objtool-args = $(if $(delay-objtool),,$(objtool-args-y)) --noabs
PATCH v7 17/22 x86/boot: Revert "Reject absolute references in .head.text"
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
This reverts commit
faf0ed487415 ("x86/boot: Reject absolute references in .head.text")
The startup code is checked directly for the absence of absolute symbol references, so checking the .head.text section in the relocs tool is no longer needed.
--- arch/x86/tools/relocs.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
arch/x86/tools/relocs.c +1 -7
@@ -740,10 +740,10 @@ static void walk_relocs(int (*process)(struct section *sec, Elf_Rel *rel,static int do_reloc64 (struct section *sec,Elf_Rel *rel,ElfW (Sym) *sym,const char *symname) { -int headtext = !strcmp (sec_name (sec->shdr.sh_info),".head.text" );unsigned r_type =ELF64_R_TYPE (rel->r_info);ElfW (Addr) offset = rel->r_offset;int shn_abs = (sym->st_shndx ==SHN_ABS ) && !is_reloc (S_REL , symname); +if (sym->st_shndx ==SHN_UNDEF )return 0 ;
@@ -783,12 +783,6 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym,break ; } -if (headtext) { -die ("Absolute reference to symbol '%s' not permitted in .head.text\n" , - symname); -break ; - } -/* * Relocation offsets for 64 bit kernels are output * as 32 bits and sign extended back to 64 bits when
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/boot: Revert "Reject absolute references in .head.text"
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 2578560d2259735d7d51364e7991ea92d85fd56c Gitweb: https://git.kernel.org/tip/2578560d2259735d7d51364e7991ea92d85fd56c Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:20 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:56 +02:00
x86/boot: Revert "Reject absolute references in .head.text"
This reverts commit
faf0ed487415 ("x86/boot: Reject absolute references in .head.text")
The startup code is checked directly for the absence of absolute symbol references, so checking the .head.text section in the relocs tool is no longer needed.
--- arch/x86/tools/relocs.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
arch/x86/tools/relocs.c +1 -7
@@ -740,10 +740,10 @@ static void walk_relocs(int (*process)(struct section *sec, Elf_Rel *rel,static int do_reloc64 (struct section *sec,Elf_Rel *rel,ElfW (Sym) *sym,const char *symname) { -int headtext = !strcmp (sec_name (sec->shdr.sh_info),".head.text" );unsigned r_type =ELF64_R_TYPE (rel->r_info);ElfW (Addr) offset = rel->r_offset;int shn_abs = (sym->st_shndx ==SHN_ABS ) && !is_reloc (S_REL , symname); +if (sym->st_shndx ==SHN_UNDEF )return 0 ;
@@ -783,12 +783,6 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym,break ; } -if (headtext) { -die ("Absolute reference to symbol '%s' not permitted in .head.text\n" , - symname); -break ; - } -/* * Relocation offsets for 64 bit kernels are output * as 32 bits and sign extended back to 64 bits when
PATCH v7 18/22 x86/kbuild: Incorporate boot/startup/ via Kbuild makefile
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Using core-y is not the correct way to get kbuild to descend into arch/x86/boot/startup. For instance, building an individual object does not work as expected when the pattern rule is local to the Makefile
$ make arch/x86/boot/startup/map_kernel.pi.o
GEN Makefile
CALL /home/ardb/linux/scripts/checksyscalls.sh
DESCEND objtool
INSTALL libsubcmd_headers
make[3]: *** No rule to make target 'arch/x86/boot/startup/map_kernel.pi.o'. Stop.
make[2]: *** [/home/ardb/linux/scripts/Makefile.build:461: arch/x86] Error 2
make[1]: *** [/home/ardb/linux/Makefile:2011: .] Error 2
make: *** [/home/ardb/linux/Makefile:248: __sub-make] Error 2So use obj-y from arch.x86/Kbuild instead, which makes things work as expected.
--- arch/x86/Kbuild | 2 ++ arch/x86/Makefile | 1 - 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/Kbuild +2 -0
@@ -3,6 +3,8 @@ # Branch profiling isn't noinstr-safe. Disable it for arch/x86/* subdir-ccflags-$(CONFIG_TRACE_BRANCH_PROFILING) += -DDISABLE_BRANCH_PROFILING +obj-y += boot/startup/ + obj-$(CONFIG_ARCH_HAS_CC_PLATFORM) += coco/ obj-y += entry/
arch/x86/Makefile +0 -1
@@ -275,7 +275,6 @@ archprepare: $(cpufeaturemasks.hdr) ### # Kernel objects -core-y += arch/x86/boot/startup/ libs-y += arch/x86/lib/ # drivers-y are linked after core-y
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/kbuild: Incorporate boot/startup/ via Kbuild makefile
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 749627c3980e4421b709857e979e8aa16a4c7147 Gitweb: https://git.kernel.org/tip/749627c3980e4421b709857e979e8aa16a4c7147 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:21 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 17:59:59 +02:00
x86/kbuild: Incorporate boot/startup/ via Kbuild makefile
Using core-y is not the correct way to get kbuild to descend into arch/x86/boot/startup. For instance, building an individual object does not work as expected when the pattern rule is local to the Makefile
$ make arch/x86/boot/startup/map_kernel.pi.o
GEN Makefile
CALL /home/ardb/linux/scripts/checksyscalls.sh
DESCEND objtool
INSTALL libsubcmd_headers
make[3]: *** No rule to make target 'arch/x86/boot/startup/map_kernel.pi.o'. Stop.
make[2]: *** [/home/ardb/linux/scripts/Makefile.build:461: arch/x86] Error 2
make[1]: *** [/home/ardb/linux/Makefile:2011: .] Error 2
make: *** [/home/ardb/linux/Makefile:248: __sub-make] Error 2So use obj-y from arch.x86/Kbuild instead, which makes things work as expected.
--- arch/x86/Kbuild | 2 ++ arch/x86/Makefile | 1 - 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/Kbuild +2 -0
@@ -3,6 +3,8 @@ # Branch profiling isn't noinstr-safe. Disable it for arch/x86/* subdir-ccflags-$(CONFIG_TRACE_BRANCH_PROFILING) += -DDISABLE_BRANCH_PROFILING +obj-y += boot/startup/ + obj-$(CONFIG_ARCH_HAS_CC_PLATFORM) += coco/ obj-y += entry/
arch/x86/Makefile +0 -1
@@ -286,7 +286,6 @@ archprepare: $(cpufeaturemasks.hdr) ### # Kernel objects -core-y += arch/x86/boot/startup/ libs-y += arch/x86/lib/ # drivers-y are linked after core-y
PATCH v7 19/22 x86/boot: Create a confined code area for startup code
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
In order to be able to have tight control over which code may execute from the early 1:1 mapping of memory, but still link vmlinux as a single executable, prefix all symbol references in startup code with __pi_, and invoke it from outside using the __pi_ prefix.
Use objtool to check that no absolute symbol references are present in the startup code, as these cannot be used from code running from the 1:1 mapping.
Note that this also requires disabling the latent-entropy GCC plugin, as the global symbol references that it injects would require explicit exports, and given that the startup code rarely executes more than once, it is not a useful source of entropy anyway.
--- arch/x86/boot/startup/Makefile | 14 ++++++++++++++ arch/x86/boot/startup/sev-shared.c | 1 - arch/x86/boot/startup/sme.c | 1 - arch/x86/coco/sev/core.c | 2 +- arch/x86/include/asm/setup.h | 1 + arch/x86/include/asm/sev.h | 1 + arch/x86/kernel/head64.c | 2 +- arch/x86/kernel/head_64.S | 8 ++++---- arch/x86/mm/mem_encrypt_boot.S | 6 +++--- tools/objtool/check.c | 3 ++- 10 files changed, 27 insertions(+), 12 deletions(-)
arch/x86/boot/startup/Makefile +14 -0
@@ -4,6 +4,7 @@ KBUILD_AFLAGS += -D__DISABLE_EXPORTS KBUILD_CFLAGS += -D__DISABLE_EXPORTS -mcmodel=small -fPIC \ -Os -DDISABLE_BRANCH_PROFILING \ $(DISABLE_STACKLEAK_PLUGIN) \ + $(DISABLE_LATENT_ENTROPY_PLUGIN) \ -fno-stack-protector -D__NO_FORTIFY \ -fno-jump-tables \ -include $(srctree)/include/linux/hidden.h
@@ -36,3 +37,16 @@ $(patsubst %.o,$(obj)/%.o,$(lib-y)): OBJECT_FILES_NON_STANDARD := y # $(pi-objs): objtool-enabled = 1 $(pi-objs): objtool-args = $(if $(delay-objtool),,$(objtool-args-y)) --noabs + +# +# Confine the startup code by prefixing all symbols with __pi_ (for position +# independent). This ensures that startup code can only call other startup +# code, or code that has explicitly been made accessible to it via a symbol +# alias. +# +$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ +$(obj)/%.pi.o: $(obj)/%.o FORCE + $(call if_changed,objcopy) + +targets += $(obj-y) +obj-y := $(patsubst %.o,%.pi.o,$(obj-y))
arch/x86/boot/startup/sev-shared.c +0 -1
@@ -12,7 +12,6 @@#include <asm/setup_data.h> #ifndef __BOOT_COMPRESSED -#define error (v) pr_err(v)#define has_cpuflag (f) boot_cpu_has(f)#else #undef WARN
arch/x86/boot/startup/sme.c +0 -1
@@ -567,7 +567,6 @@ void __head sme_enable(struct boot_params *bp)#ifdef CONFIG_MITIGATION_PAGE_TABLE_ISOLATION /* Local version for startup code, which never operates on user page tables */ -__weak pgd_t __pti_set_user_pgtbl (pgd_t *pgdp,pgd_t pgd) {return pgd;
arch/x86/coco/sev/core.c +1 -1
@@ -272,7 +272,7 @@ static int svsm_perform_call_protocol(struct svsm_call *call)do { ret = ghcb ?svsm_perform_ghcb_protocol (ghcb, call) - :svsm_perform_msr_protocol (call); + :__pi_svsm_perform_msr_protocol (call); }while (ret == -EAGAIN );if (sev_cfg.ghcbs_initialized)
arch/x86/include/asm/setup.h +1 -0
@@ -53,6 +53,7 @@ extern void i386_reserve_resources(void);extern unsigned long __startup_64 (unsigned long p2v_offset,struct boot_params *bp);extern void startup_64_setup_gdt_idt (void );extern void startup_64_load_idt (void *vc_handler); +extern void __pi_startup_64_load_idt (void *vc_handler);extern void early_setup_idt (void );extern void __init do_early_exception (struct pt_regs *regs,int trapnr);
arch/x86/include/asm/sev.h +1 -0
@@ -551,6 +551,7 @@ struct cpuid_leaf { };int svsm_perform_msr_protocol (struct svsm_call *call); +int __pi_svsm_perform_msr_protocol (struct svsm_call *call);int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf),void *ctx,struct cpuid_leaf *leaf);
arch/x86/kernel/head64.c +1 -1
@@ -319,5 +319,5 @@ void early_setup_idt(void) handler = vc_boot_ghcb; } -startup_64_load_idt (handler); +__pi_startup_64_load_idt (handler); }
arch/x86/kernel/head_64.S +4 -4
@@ -71,7 +71,7 @@ SYM_CODE_START_NOALIGN(startup_64) xorl %edx, %edx wrmsr - call startup_64_setup_gdt_idt + call __pi_startup_64_setup_gdt_idt /* Now switch to __KERNEL_CS so IRET works reliably */ pushq $__KERNEL_CS
@@ -91,7 +91,7 @@ SYM_CODE_START_NOALIGN(startup_64) * subsequent code. Pass the boot_params pointer as the first argument. */ movq %r15, %rdi - call sme_enable + call __pi_sme_enable #endif /* Sanitize CPU configuration */
@@ -111,7 +111,7 @@ SYM_CODE_START_NOALIGN(startup_64) * programmed into CR3. */ movq %r15, %rsi - call __startup_64 + call __pi___startup_64 /* Form the CR3 value being sure to include the CR3 modifier */ leaq early_top_pgt(%rip), %rcx
@@ -562,7 +562,7 @@ SYM_CODE_START_NOALIGN(vc_no_ghcb) /* Call C handler */ movq %rsp, %rdi movq ORIG_RAX(%rsp), %rsi - call do_vc_no_ghcb + call __pi_do_vc_no_ghcb /* Unwind pt_regs */ POP_REGS
arch/x86/mm/mem_encrypt_boot.S +3 -3
@@ -16,7 +16,7 @@ .text .code64 -SYM_FUNC_START(sme_encrypt_execute) +SYM_FUNC_START(__pi_sme_encrypt_execute) /* * Entry parameters:
@@ -69,9 +69,9 @@ SYM_FUNC_START(sme_encrypt_execute) ANNOTATE_UNRET_SAFE ret int3 -SYM_FUNC_END(sme_encrypt_execute) +SYM_FUNC_END(__pi_sme_encrypt_execute) -SYM_FUNC_START(__enc_copy) +SYM_FUNC_START_LOCAL(__enc_copy) ANNOTATE_NOENDBR /* * Routine used to encrypt memory in place.
tools/objtool/check.c +2 -1
@@ -3575,7 +3575,8 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,if (func &&insn_func (insn) && func !=insn_func (insn)->pfunc) {/* Ignore KCFI type preambles, which always fall through */ if (!strncmp (func->name,"__cfi_" ,6 ) || - !strncmp (func->name,"__pfx_" ,6 )) + !strncmp (func->name,"__pfx_" ,6 ) || + !strncmp (func->name,"__pi___pfx_" ,11 ))return 0 ;if (file->ignore_unreachables)
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/boot: Create a confined code area for startup code
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: 7b38dec3c5af54665a4b29483aa02bd1c1e71cf1 Gitweb: https://git.kernel.org/tip/7b38dec3c5af54665a4b29483aa02bd1c1e71cf1 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:22 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 18:00:01 +02:00
x86/boot: Create a confined code area for startup code
In order to be able to have tight control over which code may execute from the early 1:1 mapping of memory, but still link vmlinux as a single executable, prefix all symbol references in startup code with __pi_, and invoke it from outside using the __pi_ prefix.
Use objtool to check that no absolute symbol references are present in the startup code, as these cannot be used from code running from the 1:1 mapping.
Note that this also requires disabling the latent-entropy GCC plugin, as the global symbol references that it injects would require explicit exports, and given that the startup code rarely executes more than once, it is not a useful source of entropy anyway.
--- arch/x86/boot/startup/Makefile | 14 ++++++++++++++ arch/x86/boot/startup/sev-shared.c | 1 - arch/x86/boot/startup/sme.c | 1 - arch/x86/coco/sev/core.c | 2 +- arch/x86/include/asm/setup.h | 1 + arch/x86/include/asm/sev.h | 1 + arch/x86/kernel/head64.c | 2 +- arch/x86/kernel/head_64.S | 8 ++++---- arch/x86/mm/mem_encrypt_boot.S | 6 +++--- tools/objtool/check.c | 3 ++- 10 files changed, 27 insertions(+), 12 deletions(-)
arch/x86/boot/startup/Makefile +14 -0
@@ -4,6 +4,7 @@ KBUILD_AFLAGS += -D__DISABLE_EXPORTS KBUILD_CFLAGS += -D__DISABLE_EXPORTS -mcmodel=small -fPIC \ -Os -DDISABLE_BRANCH_PROFILING \ $(DISABLE_STACKLEAK_PLUGIN) \ + $(DISABLE_LATENT_ENTROPY_PLUGIN) \ -fno-stack-protector -D__NO_FORTIFY \ -fno-jump-tables \ -include $(srctree)/include/linux/hidden.h
@@ -36,3 +37,16 @@ $(patsubst %.o,$(obj)/%.o,$(lib-y)): OBJECT_FILES_NON_STANDARD := y # $(pi-objs): objtool-enabled = 1 $(pi-objs): objtool-args = $(if $(delay-objtool),,$(objtool-args-y)) --noabs + +# +# Confine the startup code by prefixing all symbols with __pi_ (for position +# independent). This ensures that startup code can only call other startup +# code, or code that has explicitly been made accessible to it via a symbol +# alias. +# +$(obj)/%.pi.o: OBJCOPYFLAGS := --prefix-symbols=__pi_ +$(obj)/%.pi.o: $(obj)/%.o FORCE + $(call if_changed,objcopy) + +targets += $(obj-y) +obj-y := $(patsubst %.o,%.pi.o,$(obj-y))
arch/x86/boot/startup/sev-shared.c +0 -1
@@ -12,7 +12,6 @@#include <asm/setup_data.h> #ifndef __BOOT_COMPRESSED -#define error (v) pr_err(v)#define has_cpuflag (f) boot_cpu_has(f)#else #undef WARN
arch/x86/boot/startup/sme.c +0 -1
@@ -568,7 +568,6 @@ void __head sme_enable(struct boot_params *bp)#ifdef CONFIG_MITIGATION_PAGE_TABLE_ISOLATION /* Local version for startup code, which never operates on user page tables */ -__weak pgd_t __pti_set_user_pgtbl (pgd_t *pgdp,pgd_t pgd) {return pgd;
arch/x86/coco/sev/core.c +1 -1
@@ -272,7 +272,7 @@ static int svsm_perform_call_protocol(struct svsm_call *call)do { ret = ghcb ?svsm_perform_ghcb_protocol (ghcb, call) - :svsm_perform_msr_protocol (call); + :__pi_svsm_perform_msr_protocol (call); }while (ret == -EAGAIN );if (sev_cfg.ghcbs_initialized)
arch/x86/include/asm/setup.h +1 -0
@@ -53,6 +53,7 @@ extern void i386_reserve_resources(void);extern unsigned long __startup_64 (unsigned long p2v_offset,struct boot_params *bp);extern void startup_64_setup_gdt_idt (void );extern void startup_64_load_idt (void *vc_handler); +extern void __pi_startup_64_load_idt (void *vc_handler);extern void early_setup_idt (void );extern void __init do_early_exception (struct pt_regs *regs,int trapnr);
arch/x86/include/asm/sev.h +1 -0
@@ -551,6 +551,7 @@ struct cpuid_leaf { };int svsm_perform_msr_protocol (struct svsm_call *call); +int __pi_svsm_perform_msr_protocol (struct svsm_call *call);int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf),void *ctx,struct cpuid_leaf *leaf);
arch/x86/kernel/head64.c +1 -1
@@ -319,5 +319,5 @@ void early_setup_idt(void) handler = vc_boot_ghcb; } -startup_64_load_idt (handler); +__pi_startup_64_load_idt (handler); }
arch/x86/kernel/head_64.S +4 -4
@@ -71,7 +71,7 @@ SYM_CODE_START_NOALIGN(startup_64) xorl %edx, %edx wrmsr - call startup_64_setup_gdt_idt + call __pi_startup_64_setup_gdt_idt /* Now switch to __KERNEL_CS so IRET works reliably */ pushq $__KERNEL_CS
@@ -91,7 +91,7 @@ SYM_CODE_START_NOALIGN(startup_64) * subsequent code. Pass the boot_params pointer as the first argument. */ movq %r15, %rdi - call sme_enable + call __pi_sme_enable #endif /* Sanitize CPU configuration */
@@ -111,7 +111,7 @@ SYM_CODE_START_NOALIGN(startup_64) * programmed into CR3. */ movq %r15, %rsi - call __startup_64 + call __pi___startup_64 /* Form the CR3 value being sure to include the CR3 modifier */ leaq early_top_pgt(%rip), %rcx
@@ -562,7 +562,7 @@ SYM_CODE_START_NOALIGN(vc_no_ghcb) /* Call C handler */ movq %rsp, %rdi movq ORIG_RAX(%rsp), %rsi - call do_vc_no_ghcb + call __pi_do_vc_no_ghcb /* Unwind pt_regs */ POP_REGS
arch/x86/mm/mem_encrypt_boot.S +3 -3
@@ -16,7 +16,7 @@ .text .code64 -SYM_FUNC_START(sme_encrypt_execute) +SYM_FUNC_START(__pi_sme_encrypt_execute) /* * Entry parameters:
@@ -69,9 +69,9 @@ SYM_FUNC_START(sme_encrypt_execute) ANNOTATE_UNRET_SAFE ret int3 -SYM_FUNC_END(sme_encrypt_execute) +SYM_FUNC_END(__pi_sme_encrypt_execute) -SYM_FUNC_START(__enc_copy) +SYM_FUNC_START_LOCAL(__enc_copy) ANNOTATE_NOENDBR /* * Routine used to encrypt memory in place.
tools/objtool/check.c +2 -1
@@ -3564,7 +3564,8 @@ static int validate_branch(struct objtool_file *file, struct symbol *func,if (func &&insn_func (insn) && func !=insn_func (insn)->pfunc) {/* Ignore KCFI type preambles, which always fall through */ if (!strncmp (func->name,"__cfi_" ,6 ) || - !strncmp (func->name,"__pfx_" ,6 )) + !strncmp (func->name,"__pfx_" ,6 ) || + !strncmp (func->name,"__pi___pfx_" ,11 ))return 0 ;if (file->ignore_unreachables)
PATCH v7 20/22 efistub/x86: Remap inittext read-execute when needed
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Recent EFI x86 systems are more strict when it comes to mapping boot images, and require that mappings are either read-write or read-execute.
Now that the boot code is being cleaned up and refactored, most of it is being moved into .init.text [where it arguably belongs] but that implies that when booting on such strict EFI firmware, we need to take care to map .init.text (and the .altinstr_aux section that follows it) read-execute as well.
--- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/compressed/misc.c | 2 ++ arch/x86/include/asm/boot.h | 2 ++ arch/x86/kernel/vmlinux.lds.S | 2 ++ drivers/firmware/efi/libstub/x86-stub.c | 4 +++- 5 files changed, 10 insertions(+), 2 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
@@ -73,7 +73,7 @@ LDFLAGS_vmlinux += -T hostprogs := mkpiggy HOST_EXTRACFLAGS += -I$(srctree)/tools/include -sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|_sinittext\|__inittext_end\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' quiet_cmd_voffset = VOFFSET $@ cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@
arch/x86/boot/compressed/misc.c +2 -0
@@ -332,6 +332,8 @@ static size_t parse_elf(void *output) }const unsigned long kernel_text_size = VO___start_rodata - VO__text; +const unsigned long kernel_inittext_offset = VO__sinittext - VO__text; +const unsigned long kernel_inittext_size = VO___inittext_end - VO__sinittext;const unsigned long kernel_total_size = VO__end - VO__text;static u8 boot_heap[BOOT_HEAP_SIZE ]__aligned(4) ;
arch/x86/include/asm/boot.h +2 -0
@@ -82,6 +82,8 @@#ifndef __ASSEMBLER__extern unsigned int output_len;extern const unsigned long kernel_text_size; +extern const unsigned long kernel_inittext_offset; +extern const unsigned long kernel_inittext_size;extern const unsigned long kernel_total_size; unsigned long decompress_kernel(unsigned char *outbuf, unsigned long virt_addr,
arch/x86/kernel/vmlinux.lds.S +2 -0
@@ -227,6 +227,8 @@ SECTIONS */ .altinstr_aux : AT(ADDR(.altinstr_aux) - LOAD_OFFSET) { *(.altinstr_aux) + . = ALIGN(PAGE_SIZE); + __inittext_end = .; } INIT_DATA_SECTION(16)
drivers/firmware/efi/libstub/x86-stub.c +3 -1
@@ -788,7 +788,9 @@ static efi_status_t efi_decompress_kernel(unsigned long *kernel_entry, *kernel_entry = addr + entry; -return efi_adjust_memory_range_protection (addr, kernel_text_size); +return efi_adjust_memory_range_protection (addr, kernel_text_size) ?: +efi_adjust_memory_range_protection (addr + kernel_inittext_offset, + kernel_inittext_size); } static void __noreturn enter_kernel(unsigned long kernel_addr,
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] efistub/x86: Remap inittext read-execute when needed
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: e7b88bc0051c5062bdd73b58837cf277d0057358 Gitweb: https://git.kernel.org/tip/e7b88bc0051c5062bdd73b58837cf277d0057358 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:23 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 18:05:42 +02:00
efistub/x86: Remap inittext read-execute when needed
Recent EFI x86 systems are more strict when it comes to mapping boot images, and require that mappings are either read-write or read-execute.
Now that the boot code is being cleaned up and refactored, most of it is being moved into .init.text [where it arguably belongs] but that implies that when booting on such strict EFI firmware, we need to take care to map .init.text (and the .altinstr_aux section that follows it) read-execute as well.
--- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/compressed/misc.c | 2 ++ arch/x86/include/asm/boot.h | 2 ++ arch/x86/kernel/vmlinux.lds.S | 2 ++ drivers/firmware/efi/libstub/x86-stub.c | 4 +++- 5 files changed, 10 insertions(+), 2 deletions(-)
arch/x86/boot/compressed/Makefile +1 -1
@@ -73,7 +73,7 @@ LDFLAGS_vmlinux += -T hostprogs := mkpiggy HOST_EXTRACFLAGS += -I$(srctree)/tools/include -sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset := -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_rodata\|_sinittext\|__inittext_end\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' quiet_cmd_voffset = VOFFSET $@ cmd_voffset = $(NM) $< | sed -n $(sed-voffset) > $@
arch/x86/boot/compressed/misc.c +2 -0
@@ -332,6 +332,8 @@ static size_t parse_elf(void *output) }const unsigned long kernel_text_size = VO___start_rodata - VO__text; +const unsigned long kernel_inittext_offset = VO__sinittext - VO__text; +const unsigned long kernel_inittext_size = VO___inittext_end - VO__sinittext;const unsigned long kernel_total_size = VO__end - VO__text;static u8 boot_heap[BOOT_HEAP_SIZE ]__aligned(4) ;
arch/x86/include/asm/boot.h +2 -0
@@ -82,6 +82,8 @@#ifndef __ASSEMBLER__extern unsigned int output_len;extern const unsigned long kernel_text_size; +extern const unsigned long kernel_inittext_offset; +extern const unsigned long kernel_inittext_size;extern const unsigned long kernel_total_size; unsigned long decompress_kernel(unsigned char *outbuf, unsigned long virt_addr,
arch/x86/kernel/vmlinux.lds.S +2 -0
@@ -227,6 +227,8 @@ SECTIONS */ .altinstr_aux : AT(ADDR(.altinstr_aux) - LOAD_OFFSET) { *(.altinstr_aux) + . = ALIGN(PAGE_SIZE); + __inittext_end = .; } INIT_DATA_SECTION(16)
drivers/firmware/efi/libstub/x86-stub.c +3 -1
@@ -788,7 +788,9 @@ static efi_status_t efi_decompress_kernel(unsigned long *kernel_entry, *kernel_entry = addr + entry; -return efi_adjust_memory_range_protection (addr, kernel_text_size); +return efi_adjust_memory_range_protection (addr, kernel_text_size) ?: +efi_adjust_memory_range_protection (addr + kernel_inittext_offset, + kernel_inittext_size); } static void __noreturn enter_kernel(unsigned long kernel_addr,
PATCH v7 21/22 x86/boot: Move startup code out of __head section
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Move startup code out of the __head section, now that this no longer has a special significance. Move everything into .text or .init.text as appropriate, so that startup code is not kept around unnecessarily.
--- arch/x86/boot/compressed/sev.c | 3 -- arch/x86/boot/startup/gdt_idt.c | 4 +-- arch/x86/boot/startup/map_kernel.c | 4 +-- arch/x86/boot/startup/sev-shared.c | 36 ++++++++++---------- arch/x86/boot/startup/sev-startup.c | 14 ++++---- arch/x86/boot/startup/sme.c | 26 +++++++------- arch/x86/include/asm/init.h | 6 ---- arch/x86/kernel/head_32.S | 2 +- arch/x86/kernel/head_64.S | 2 +- arch/x86/platform/pvh/head.S | 2 +- 10 files changed, 45 insertions(+), 54 deletions(-)
arch/x86/boot/compressed/sev.c +0 -3
@@ -32,9 +32,6 @@ struct ghcb *boot_ghcb;#undef __init#define __init -#undef __head -#define __head -#define __BOOT_COMPRESSEDu8 snp_vmpl;
arch/x86/boot/startup/gdt_idt.c +2 -2
@@ -24,7 +24,7 @@ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data;/* This may run while still in the direct mapping */ -void __head startup_64_load_idt(void *vc_handler) +void startup_64_load_idt (void *vc_handler) {struct desc_ptr desc = { .address = (unsigned long )rip_rel_ptr (bringup_idt_table),
@@ -46,7 +46,7 @@ void __head startup_64_load_idt(void *vc_handler)/* * Setup boot CPU state needed before kernel switches to virtual addresses. */ -void __head startup_64_setup_gdt_idt(void) +void __init startup_64_setup_gdt_idt (void ) {struct gdt_page *gp =rip_rel_ptr ((void *)(__force unsigned long )&gdt_page);void *handler =NULL ;
arch/x86/boot/startup/map_kernel.c +2 -2
@@ -30,7 +30,7 @@ static inline bool check_la57_support(void)return true; } -static unsigned long __head sme_postprocess_startup (struct boot_params *bp, +static unsigned long __init sme_postprocess_startup (struct boot_params *bp,pmdval_t *pmd,unsigned long p2v_offset) {
@@ -84,7 +84,7 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,* the 1:1 mapping of memory. Kernel virtual addresses can be determined by * subtracting p2v_offset from the RIP-relative address. */ -unsigned long __head __startup_64 (unsigned long p2v_offset, +unsigned long __init __startup_64 (unsigned long p2v_offset,struct boot_params *bp) {pmd_t (*early_pgts)[PTRS_PER_PMD ] =rip_rel_ptr (early_dynamic_pgts);
arch/x86/boot/startup/sev-shared.c +18 -18
@@ -31,7 +31,7 @@ static u32 cpuid_std_range_max __ro_after_init;static u32 cpuid_hyp_range_max__ro_after_init ;static u32 cpuid_ext_range_max__ro_after_init ; -void __head __noreturn +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason) {u64 val =GHCB_MSR_TERM_REQ ;
@@ -50,7 +50,7 @@ sev_es_terminate(unsigned int set, unsigned int reason)/* * The hypervisor features are available from GHCB version 2 onward. */ -u64 get_hv_features (void ) +u64 __init get_hv_features (void ) {u64 val;
@@ -220,7 +220,7 @@ const struct snp_cpuid_table *snp_cpuid_get_table(void)* * Return: XSAVE area size on success, 0 otherwise. */ -static u32 __head snp_cpuid_calc_xsave_size(u64 xfeatures_en, bool compacted) +static u32 snp_cpuid_calc_xsave_size (u64 xfeatures_en,bool compacted) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();u64 xfeatures_found =0 ;
@@ -256,7 +256,7 @@ static u32 __head snp_cpuid_calc_xsave_size(u64 xfeatures_en, bool compacted)return xsave_size; } -static bool __head +static bool snp_cpuid_get_validated_func (struct cpuid_leaf *leaf) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();
@@ -298,7 +298,7 @@ static void snp_cpuid_hv_msr(void *ctx, struct cpuid_leaf *leaf)sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_CPUID_HV ); } -static int __head +static int snp_cpuid_postprocess (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf),void *ctx,struct cpuid_leaf *leaf) {
@@ -394,8 +394,8 @@ snp_cpuid_postprocess(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf),* Returns -EOPNOTSUPP if feature not enabled. Any other non-zero return value * should be treated as fatal by caller. */ -int __head snp_cpuid(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf), -void *ctx,struct cpuid_leaf *leaf) +int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();
@@ -437,7 +437,7 @@ int __head snp_cpuid(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf),* page yet, so it only supports the MSR based communication with the * hypervisor and only the CPUID exit-code. */ -void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) +void do_vc_no_ghcb (struct pt_regs *regs,unsigned long exit_code) {unsigned int subfn =lower_bits (regs->cx,32 );unsigned int fn =lower_bits (regs->ax,32 );
@@ -513,7 +513,7 @@ struct cc_setup_data {* Search for a Confidential Computing blob passed in as a setup_data entry * via the Linux Boot Protocol. */ -static __head +static __init struct cc_blob_sev_info *find_cc_blob_setup_data (struct boot_params *bp) {struct cc_setup_data *sd =NULL ;
@@ -541,7 +541,7 @@ struct cc_blob_sev_info *find_cc_blob_setup_data(struct boot_params *bp)* mapping needs to be updated in sync with all the changes to virtual memory * layout and related mapping facilities throughout the boot process. */ -static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info) +static void __init setup_cpuid_table (const struct cc_blob_sev_info *cc_info) {const struct snp_cpuid_table *cpuid_table_fw, *cpuid_table;int i;
@@ -569,7 +569,7 @@ static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info) } } -static int __head svsm_call_msr_protocol(struct svsm_call *call) +static int svsm_call_msr_protocol (struct svsm_call *call) {int ret;
@@ -580,8 +580,8 @@ static int __head svsm_call_msr_protocol(struct svsm_call *call)return ret; } -static void __head svsm_pval_4k_page(unsigned long paddr, bool validate, -struct svsm_ca *caa,u64 caa_pa) +static void svsm_pval_4k_page (unsigned long paddr,bool validate, +struct svsm_ca *caa,u64 caa_pa) {struct svsm_pvalidate_call *pc;struct svsm_call call = {};
@@ -622,8 +622,8 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate,native_local_irq_restore (flags); } -static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, -bool validate,struct svsm_ca *caa,u64 caa_pa) +static void pvalidate_4k_page (unsigned long vaddr,unsigned long paddr, +bool validate,struct svsm_ca *caa,u64 caa_pa) {int ret;
@@ -643,8 +643,8 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,sev_evict_cache ((void *)vaddr,1 ); } -static void __head __page_state_change(unsigned long vaddr, unsigned long paddr, -const struct psc_desc *desc) +static void __page_state_change (unsigned long vaddr,unsigned long paddr, +const struct psc_desc *desc) {u64 val, msr;
@@ -682,7 +682,7 @@ static void __head __page_state_change(unsigned long vaddr, unsigned long paddr,* Maintain the GPA of the SVSM Calling Area (CA) in order to utilize the SVSM * services needed when not running in VMPL0. */ -static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info, +static bool __init svsm_setup_ca (const struct cc_blob_sev_info *cc_info,void *page) {struct snp_secrets_page *secrets_page;
arch/x86/boot/startup/sev-startup.c +7 -7
@@ -44,7 +44,7 @@/* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" -void __head +void __init early_set_pages_state (unsigned long vaddr,unsigned long paddr,unsigned long npages,const struct psc_desc *desc) {
@@ -63,7 +63,7 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, } } -void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, +void __init early_snp_set_memory_private (unsigned long vaddr,unsigned long paddr,unsigned long npages) {struct psc_desc d = {
@@ -88,7 +88,7 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddearly_set_pages_state (vaddr, paddr, npages, &d); } -void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, +void __init early_snp_set_memory_shared (unsigned long vaddr,unsigned long paddr,unsigned long npages) {struct psc_desc d = {
@@ -123,7 +123,7 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr* * Scan for the blob in that order. */ -static __head struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) +static struct cc_blob_sev_info *__init find_cc_blob (struct boot_params *bp) {struct cc_blob_sev_info *cc_info;
@@ -149,7 +149,7 @@ static __head struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp)return cc_info; } -static __head void svsm_setup(struct cc_blob_sev_info *cc_info) +static void __init svsm_setup (struct cc_blob_sev_info *cc_info) {struct snp_secrets_page *secrets = (void *)cc_info->secrets_phys;struct svsm_call call = {};
@@ -190,7 +190,7 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info) boot_svsm_caa_pa = pa; } -bool __head snp_init(struct boot_params *bp) +bool __init snp_init (struct boot_params *bp) {struct cc_blob_sev_info *cc_info;
@@ -219,7 +219,7 @@ bool __head snp_init(struct boot_params *bp)return true; } -void __head __noreturn snp_abort(void) +void __init __noreturn snp_abort (void ) {sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED ); }
arch/x86/boot/startup/sme.c +13 -13
@@ -91,7 +91,7 @@ struct sme_populate_pgd_data {*/ static char sme_workarea[2 * PMD_SIZE] __section(".init.scratch"); -static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd) +static void __init sme_clear_pgd (struct sme_populate_pgd_data *ppd) {unsigned long pgd_start, pgd_end, pgd_size;pgd_t *pgd_p;
@@ -106,7 +106,7 @@ static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd)memset (pgd_p,0 , pgd_size); } -static pud_t __head *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) +static pud_t __init *sme_prepare_pgd (struct sme_populate_pgd_data *ppd) {pgd_t *pgd;p4d_t *p4d;
@@ -143,7 +143,7 @@ static pud_t __head *sme_prepare_pgd(struct sme_populate_pgd_data *ppd)return pud; } -static void __head sme_populate_pgd_large(struct sme_populate_pgd_data *ppd) +static void __init sme_populate_pgd_large (struct sme_populate_pgd_data *ppd) {pud_t *pud;pmd_t *pmd;
@@ -159,7 +159,7 @@ static void __head sme_populate_pgd_large(struct sme_populate_pgd_data *ppd)set_pmd (pmd,__pmd (ppd->paddr | ppd->pmd_flags)); } -static void __head sme_populate_pgd(struct sme_populate_pgd_data *ppd) +static void __init sme_populate_pgd (struct sme_populate_pgd_data *ppd) {pud_t *pud;pmd_t *pmd;
@@ -185,7 +185,7 @@ static void __head sme_populate_pgd(struct sme_populate_pgd_data *ppd)set_pte (pte,__pte (ppd->paddr | ppd->pte_flags)); } -static void __head __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) +static void __init __sme_map_range_pmd (struct sme_populate_pgd_data *ppd) {while (ppd->vaddr < ppd->vaddr_end) {sme_populate_pgd_large (ppd);
@@ -195,7 +195,7 @@ static void __head __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) } } -static void __head __sme_map_range_pte(struct sme_populate_pgd_data *ppd) +static void __init __sme_map_range_pte (struct sme_populate_pgd_data *ppd) {while (ppd->vaddr < ppd->vaddr_end) {sme_populate_pgd (ppd);
@@ -205,7 +205,7 @@ static void __head __sme_map_range_pte(struct sme_populate_pgd_data *ppd) } } -static void __head __sme_map_range(struct sme_populate_pgd_data *ppd, +static void __init __sme_map_range (struct sme_populate_pgd_data *ppd,pmdval_t pmd_flags,pteval_t pte_flags) {unsigned long vaddr_end;
@@ -229,22 +229,22 @@ static void __head __sme_map_range(struct sme_populate_pgd_data *ppd,__sme_map_range_pte (ppd); } -static void __head sme_map_range_encrypted(struct sme_populate_pgd_data *ppd) +static void __init sme_map_range_encrypted (struct sme_populate_pgd_data *ppd) {__sme_map_range (ppd,PMD_FLAGS_ENC ,PTE_FLAGS_ENC ); } -static void __head sme_map_range_decrypted(struct sme_populate_pgd_data *ppd) +static void __init sme_map_range_decrypted (struct sme_populate_pgd_data *ppd) {__sme_map_range (ppd,PMD_FLAGS_DEC ,PTE_FLAGS_DEC ); } -static void __head sme_map_range_decrypted_wp(struct sme_populate_pgd_data *ppd) +static void __init sme_map_range_decrypted_wp (struct sme_populate_pgd_data *ppd) {__sme_map_range (ppd,PMD_FLAGS_DEC_WP ,PTE_FLAGS_DEC_WP ); } -static unsigned long __head sme_pgtable_calc (unsigned long len) +static unsigned long __init sme_pgtable_calc (unsigned long len) {unsigned long entries =0 , tables =0 ;
@@ -281,7 +281,7 @@ static unsigned long __head sme_pgtable_calc(unsigned long len)return entries + tables; } -void __head sme_encrypt_kernel(struct boot_params *bp) +void __init sme_encrypt_kernel (struct boot_params *bp) {unsigned long workarea_start, workarea_end, workarea_len;unsigned long execute_start, execute_end, execute_len;
@@ -485,7 +485,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp)native_write_cr3 (__native_read_cr3 ()); } -void __head sme_enable(struct boot_params *bp) +void __init sme_enable (struct boot_params *bp) {unsigned int eax, ebx, ecx, edx;unsigned long feature_mask;
arch/x86/include/asm/init.h +0 -6
@@ -2,12 +2,6 @@ #ifndef _ASM_X86_INIT_H #define _ASM_X86_INIT_H -#if defined(CONFIG_CC_IS_CLANG) && CONFIG_CLANG_VERSION < 170000 -#define __head __section(".head.text") __no_sanitize_undefined __no_stack_protector -#else -#define __head __section(".head.text") __no_sanitize_undefined __no_kstack_erase -#endif - struct x86_mapping_info { void *(*alloc_pgt_page)(void *); /* allocate buf for page table */ void (*free_pgt_page)(void *, void *); /* free buf for page table */
arch/x86/kernel/head_32.S +1 -1
@@ -61,7 +61,7 @@ RESERVE_BRK(pagetables, INIT_MAP_SIZE) * any particular GDT layout, because we load our own as soon as we * can. */ -__HEAD + __INIT SYM_CODE_START(startup_32) movl pa(initial_stack),%ecx
arch/x86/kernel/head_64.S +1 -1
@@ -33,7 +33,7 @@ * because we need identity-mapped pages. */ - __HEAD + __INIT .code64 SYM_CODE_START_NOALIGN(startup_64) UNWIND_HINT_END_OF_STACK
arch/x86/platform/pvh/head.S +1 -1
@@ -24,7 +24,7 @@ #include <asm/nospec-branch.h> #include <xen/interface/elfnote.h> - __HEAD + __INIT /* * Entry point for PVH guests.
Signature · 1 line
2.51.0.268.g9569e192d0-goog
Ard Biesheuvel
maintainerRe: [PATCH v7 21/22] x86/boot: Move startup code out of __head section
Quoted 8 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
Move startup code out of the
__headsection, now that this no longer has a special significance. Move everything into .text or .init.text as appropriate, so that startup code is not kept around unnecessarily.Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 13 lines
--- arch/x86/boot/compressed/sev.c | 3 -- arch/x86/boot/startup/gdt_idt.c | 4 +-- arch/x86/boot/startup/map_kernel.c | 4 +-- arch/x86/boot/startup/sev-shared.c | 36 ++++++++++---------- arch/x86/boot/startup/sev-startup.c | 14 ++++---- arch/x86/boot/startup/sme.c | 26 +++++++------- arch/x86/include/asm/init.h | 6 ---- arch/x86/kernel/head_32.S | 2 +- arch/x86/kernel/head_64.S | 2 +- arch/x86/platform/pvh/head.S | 2 +- 10 files changed, 45 insertions(+), 54 deletions(-)
As reported here [0], this patch needs the next hunk folded in to avoid breaking CPU hotplug on i386
arch/x86/kernel/head_32.S +3 -0
@@ -136,6 +136,9 @@ SYM_CODE_END(startup_32) * If cpu hotplug is not supported then this code can go in init section * which will be freed later */ +#ifdef CONFIG_HOTPLUG_CPU + .text +#endif SYM_FUNC_START(startup_32_smp) cld movl $(__BOOT_DS),%eax
[tip: x86/sev] x86/boot: Move startup code out of __head section
In reply to Ard BiesheuvelThe following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: c5c30a37369313d1f8b84e96e6a4397b4e2b4eb8 Gitweb: https://git.kernel.org/tip/c5c30a37369313d1f8b84e96e6a4397b4e2b4eb8 Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:24 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 18:06:04 +02:00
x86/boot: Move startup code out of __head section
Move startup code out of the __head section, now that this no longer has a special significance. Move everything into .text or .init.text as appropriate, so that startup code is not kept around unnecessarily.
[ bp: Fold in hunk to fix 32-bit CPU hotplug:
--- arch/x86/boot/compressed/sev.c | 3 +-- arch/x86/boot/startup/gdt_idt.c | 4 +-- arch/x86/boot/startup/map_kernel.c | 4 +-- arch/x86/boot/startup/sev-shared.c | 36 ++++++++++++++-------------- arch/x86/boot/startup/sev-startup.c | 14 +++++------ arch/x86/boot/startup/sme.c | 26 ++++++++++---------- arch/x86/include/asm/init.h | 6 +----- arch/x86/kernel/head_32.S | 5 +++- arch/x86/kernel/head_64.S | 2 +- arch/x86/platform/pvh/head.S | 2 +- 10 files changed, 48 insertions(+), 54 deletions(-)
arch/x86/boot/compressed/sev.c +0 -3
@@ -32,9 +32,6 @@ struct ghcb *boot_ghcb;#undef __init#define __init -#undef __head -#define __head -#define __BOOT_COMPRESSEDu8 snp_vmpl;
arch/x86/boot/startup/gdt_idt.c +2 -2
@@ -24,7 +24,7 @@ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_data;/* This may run while still in the direct mapping */ -void __head startup_64_load_idt(void *vc_handler) +void startup_64_load_idt (void *vc_handler) {struct desc_ptr desc = { .address = (unsigned long )rip_rel_ptr (bringup_idt_table),
@@ -46,7 +46,7 @@ void __head startup_64_load_idt(void *vc_handler)/* * Setup boot CPU state needed before kernel switches to virtual addresses. */ -void __head startup_64_setup_gdt_idt(void) +void __init startup_64_setup_gdt_idt (void ) {struct gdt_page *gp =rip_rel_ptr ((void *)(__force unsigned long )&gdt_page);void *handler =NULL ;
arch/x86/boot/startup/map_kernel.c +2 -2
@@ -30,7 +30,7 @@ static inline bool check_la57_support(void)return true; } -static unsigned long __head sme_postprocess_startup (struct boot_params *bp, +static unsigned long __init sme_postprocess_startup (struct boot_params *bp,pmdval_t *pmd,unsigned long p2v_offset) {
@@ -84,7 +84,7 @@ static unsigned long __head sme_postprocess_startup(struct boot_params *bp,* the 1:1 mapping of memory. Kernel virtual addresses can be determined by * subtracting p2v_offset from the RIP-relative address. */ -unsigned long __head __startup_64 (unsigned long p2v_offset, +unsigned long __init __startup_64 (unsigned long p2v_offset,struct boot_params *bp) {pmd_t (*early_pgts)[PTRS_PER_PMD ] =rip_rel_ptr (early_dynamic_pgts);
arch/x86/boot/startup/sev-shared.c +18 -18
@@ -33,7 +33,7 @@ static u32 cpuid_ext_range_max __ro_after_init;bool sev_snp_needs_sfw; -void __head __noreturn +void __noreturn sev_es_terminate (unsigned int set,unsigned int reason) {u64 val =GHCB_MSR_TERM_REQ ;
@@ -52,7 +52,7 @@ sev_es_terminate(unsigned int set, unsigned int reason)/* * The hypervisor features are available from GHCB version 2 onward. */ -u64 get_hv_features (void ) +u64 __init get_hv_features (void ) {u64 val;
@@ -222,7 +222,7 @@ const struct snp_cpuid_table *snp_cpuid_get_table(void)* * Return: XSAVE area size on success, 0 otherwise. */ -static u32 __head snp_cpuid_calc_xsave_size(u64 xfeatures_en, bool compacted) +static u32 snp_cpuid_calc_xsave_size (u64 xfeatures_en,bool compacted) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();u64 xfeatures_found =0 ;
@@ -258,7 +258,7 @@ static u32 __head snp_cpuid_calc_xsave_size(u64 xfeatures_en, bool compacted)return xsave_size; } -static bool __head +static bool snp_cpuid_get_validated_func (struct cpuid_leaf *leaf) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();
@@ -300,7 +300,7 @@ static void snp_cpuid_hv_msr(void *ctx, struct cpuid_leaf *leaf)sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_CPUID_HV ); } -static int __head +static int snp_cpuid_postprocess (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf),void *ctx,struct cpuid_leaf *leaf) {
@@ -396,8 +396,8 @@ snp_cpuid_postprocess(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf),* Returns -EOPNOTSUPP if feature not enabled. Any other non-zero return value * should be treated as fatal by caller. */ -int __head snp_cpuid(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf), -void *ctx,struct cpuid_leaf *leaf) +int snp_cpuid (void (*cpuid_fn)(void *ctx,struct cpuid_leaf *leaf), +void *ctx,struct cpuid_leaf *leaf) {const struct snp_cpuid_table *cpuid_table =snp_cpuid_get_table ();
@@ -439,7 +439,7 @@ int __head snp_cpuid(void (*cpuid_fn)(void *ctx, struct cpuid_leaf *leaf),* page yet, so it only supports the MSR based communication with the * hypervisor and only the CPUID exit-code. */ -void __head do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code) +void do_vc_no_ghcb (struct pt_regs *regs,unsigned long exit_code) {unsigned int subfn =lower_bits (regs->cx,32 );unsigned int fn =lower_bits (regs->ax,32 );
@@ -515,7 +515,7 @@ struct cc_setup_data {* Search for a Confidential Computing blob passed in as a setup_data entry * via the Linux Boot Protocol. */ -static __head +static __init struct cc_blob_sev_info *find_cc_blob_setup_data (struct boot_params *bp) {struct cc_setup_data *sd =NULL ;
@@ -543,7 +543,7 @@ struct cc_blob_sev_info *find_cc_blob_setup_data(struct boot_params *bp)* mapping needs to be updated in sync with all the changes to virtual memory * layout and related mapping facilities throughout the boot process. */ -static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info) +static void __init setup_cpuid_table (const struct cc_blob_sev_info *cc_info) {const struct snp_cpuid_table *cpuid_table_fw, *cpuid_table;int i;
@@ -571,7 +571,7 @@ static void __head setup_cpuid_table(const struct cc_blob_sev_info *cc_info) } } -static int __head svsm_call_msr_protocol(struct svsm_call *call) +static int svsm_call_msr_protocol (struct svsm_call *call) {int ret;
@@ -582,8 +582,8 @@ static int __head svsm_call_msr_protocol(struct svsm_call *call)return ret; } -static void __head svsm_pval_4k_page(unsigned long paddr, bool validate, -struct svsm_ca *caa,u64 caa_pa) +static void svsm_pval_4k_page (unsigned long paddr,bool validate, +struct svsm_ca *caa,u64 caa_pa) {struct svsm_pvalidate_call *pc;struct svsm_call call = {};
@@ -624,8 +624,8 @@ static void __head svsm_pval_4k_page(unsigned long paddr, bool validate,native_local_irq_restore (flags); } -static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, -bool validate,struct svsm_ca *caa,u64 caa_pa) +static void pvalidate_4k_page (unsigned long vaddr,unsigned long paddr, +bool validate,struct svsm_ca *caa,u64 caa_pa) {int ret;
@@ -645,8 +645,8 @@ static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long paddr,sev_evict_cache ((void *)vaddr,1 ); } -static void __head __page_state_change(unsigned long vaddr, unsigned long paddr, -const struct psc_desc *desc) +static void __page_state_change (unsigned long vaddr,unsigned long paddr, +const struct psc_desc *desc) {u64 val, msr;
@@ -684,7 +684,7 @@ static void __head __page_state_change(unsigned long vaddr, unsigned long paddr,* Maintain the GPA of the SVSM Calling Area (CA) in order to utilize the SVSM * services needed when not running in VMPL0. */ -static bool __head svsm_setup_ca(const struct cc_blob_sev_info *cc_info, +static bool __init svsm_setup_ca (const struct cc_blob_sev_info *cc_info,void *page) {struct snp_secrets_page *secrets_page;
arch/x86/boot/startup/sev-startup.c +7 -7
@@ -44,7 +44,7 @@/* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" -void __head +void __init early_set_pages_state (unsigned long vaddr,unsigned long paddr,unsigned long npages,const struct psc_desc *desc) {
@@ -63,7 +63,7 @@ early_set_pages_state(unsigned long vaddr, unsigned long paddr, } } -void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddr, +void __init early_snp_set_memory_private (unsigned long vaddr,unsigned long paddr,unsigned long npages) {struct psc_desc d = {
@@ -88,7 +88,7 @@ void __head early_snp_set_memory_private(unsigned long vaddr, unsigned long paddearly_set_pages_state (vaddr, paddr, npages, &d); } -void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr, +void __init early_snp_set_memory_shared (unsigned long vaddr,unsigned long paddr,unsigned long npages) {struct psc_desc d = {
@@ -123,7 +123,7 @@ void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long paddr* * Scan for the blob in that order. */ -static __head struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) +static struct cc_blob_sev_info *__init find_cc_blob (struct boot_params *bp) {struct cc_blob_sev_info *cc_info;
@@ -149,7 +149,7 @@ found_cc_info:return cc_info; } -static __head void svsm_setup(struct cc_blob_sev_info *cc_info) +static void __init svsm_setup (struct cc_blob_sev_info *cc_info) {struct snp_secrets_page *secrets = (void *)cc_info->secrets_phys;struct svsm_call call = {};
@@ -190,7 +190,7 @@ static __head void svsm_setup(struct cc_blob_sev_info *cc_info) boot_svsm_caa_pa = pa; } -bool __head snp_init(struct boot_params *bp) +bool __init snp_init (struct boot_params *bp) {struct cc_blob_sev_info *cc_info;
@@ -219,7 +219,7 @@ bool __head snp_init(struct boot_params *bp)return true; } -void __head __noreturn snp_abort(void) +void __init __noreturn snp_abort (void ) {sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED ); }
arch/x86/boot/startup/sme.c +13 -13
@@ -91,7 +91,7 @@ struct sme_populate_pgd_data {*/ static char sme_workarea[2 * PMD_SIZE] __section(".init.scratch"); -static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd) +static void __init sme_clear_pgd (struct sme_populate_pgd_data *ppd) {unsigned long pgd_start, pgd_end, pgd_size;pgd_t *pgd_p;
@@ -106,7 +106,7 @@ static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd)memset (pgd_p,0 , pgd_size); } -static pud_t __head *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) +static pud_t __init *sme_prepare_pgd (struct sme_populate_pgd_data *ppd) {pgd_t *pgd;p4d_t *p4d;
@@ -143,7 +143,7 @@ static pud_t __head *sme_prepare_pgd(struct sme_populate_pgd_data *ppd)return pud; } -static void __head sme_populate_pgd_large(struct sme_populate_pgd_data *ppd) +static void __init sme_populate_pgd_large (struct sme_populate_pgd_data *ppd) {pud_t *pud;pmd_t *pmd;
@@ -159,7 +159,7 @@ static void __head sme_populate_pgd_large(struct sme_populate_pgd_data *ppd)set_pmd (pmd,__pmd (ppd->paddr | ppd->pmd_flags)); } -static void __head sme_populate_pgd(struct sme_populate_pgd_data *ppd) +static void __init sme_populate_pgd (struct sme_populate_pgd_data *ppd) {pud_t *pud;pmd_t *pmd;
@@ -185,7 +185,7 @@ static void __head sme_populate_pgd(struct sme_populate_pgd_data *ppd)set_pte (pte,__pte (ppd->paddr | ppd->pte_flags)); } -static void __head __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) +static void __init __sme_map_range_pmd (struct sme_populate_pgd_data *ppd) {while (ppd->vaddr < ppd->vaddr_end) {sme_populate_pgd_large (ppd);
@@ -195,7 +195,7 @@ static void __head __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) } } -static void __head __sme_map_range_pte(struct sme_populate_pgd_data *ppd) +static void __init __sme_map_range_pte (struct sme_populate_pgd_data *ppd) {while (ppd->vaddr < ppd->vaddr_end) {sme_populate_pgd (ppd);
@@ -205,7 +205,7 @@ static void __head __sme_map_range_pte(struct sme_populate_pgd_data *ppd) } } -static void __head __sme_map_range(struct sme_populate_pgd_data *ppd, +static void __init __sme_map_range (struct sme_populate_pgd_data *ppd,pmdval_t pmd_flags,pteval_t pte_flags) {unsigned long vaddr_end;
@@ -229,22 +229,22 @@ static void __head __sme_map_range(struct sme_populate_pgd_data *ppd,__sme_map_range_pte (ppd); } -static void __head sme_map_range_encrypted(struct sme_populate_pgd_data *ppd) +static void __init sme_map_range_encrypted (struct sme_populate_pgd_data *ppd) {__sme_map_range (ppd,PMD_FLAGS_ENC ,PTE_FLAGS_ENC ); } -static void __head sme_map_range_decrypted(struct sme_populate_pgd_data *ppd) +static void __init sme_map_range_decrypted (struct sme_populate_pgd_data *ppd) {__sme_map_range (ppd,PMD_FLAGS_DEC ,PTE_FLAGS_DEC ); } -static void __head sme_map_range_decrypted_wp(struct sme_populate_pgd_data *ppd) +static void __init sme_map_range_decrypted_wp (struct sme_populate_pgd_data *ppd) {__sme_map_range (ppd,PMD_FLAGS_DEC_WP ,PTE_FLAGS_DEC_WP ); } -static unsigned long __head sme_pgtable_calc (unsigned long len) +static unsigned long __init sme_pgtable_calc (unsigned long len) {unsigned long entries =0 , tables =0 ;
@@ -281,7 +281,7 @@ static unsigned long __head sme_pgtable_calc(unsigned long len)return entries + tables; } -void __head sme_encrypt_kernel(struct boot_params *bp) +void __init sme_encrypt_kernel (struct boot_params *bp) {unsigned long workarea_start, workarea_end, workarea_len;unsigned long execute_start, execute_end, execute_len;
@@ -485,7 +485,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp)native_write_cr3 (__native_read_cr3 ()); } -void __head sme_enable(struct boot_params *bp) +void __init sme_enable (struct boot_params *bp) {unsigned int eax, ebx, ecx, edx;unsigned long feature_mask;
arch/x86/include/asm/init.h +0 -6
@@ -2,12 +2,6 @@ #ifndef _ASM_X86_INIT_H #define _ASM_X86_INIT_H -#if defined(CONFIG_CC_IS_CLANG) && CONFIG_CLANG_VERSION < 170000 -#define __head __section(".head.text") __no_sanitize_undefined __no_stack_protector -#else -#define __head __section(".head.text") __no_sanitize_undefined __no_kstack_erase -#endif - struct x86_mapping_info { void *(*alloc_pgt_page)(void *); /* allocate buf for page table */ void (*free_pgt_page)(void *, void *); /* free buf for page table */
arch/x86/kernel/head_32.S +4 -1
@@ -61,7 +61,7 @@ RESERVE_BRK(pagetables, INIT_MAP_SIZE) * any particular GDT layout, because we load our own as soon as we * can. */ -__HEAD + __INIT SYM_CODE_START(startup_32) movl pa(initial_stack),%ecx
@@ -136,6 +136,9 @@ SYM_CODE_END(startup_32) * If cpu hotplug is not supported then this code can go in init section * which will be freed later */ +#ifdef CONFIG_HOTPLUG_CPU + .text +#endif SYM_FUNC_START(startup_32_smp) cld movl $(__BOOT_DS),%eax
arch/x86/kernel/head_64.S +1 -1
@@ -33,7 +33,7 @@ * because we need identity-mapped pages. */ - __HEAD + __INIT .code64 SYM_CODE_START_NOALIGN(startup_64) UNWIND_HINT_END_OF_STACK
arch/x86/platform/pvh/head.S +1 -1
@@ -24,7 +24,7 @@ #include <asm/nospec-branch.h> #include <xen/interface/elfnote.h> - __HEAD + __INIT /* * Entry point for PVH guests.
PATCH v7 22/22 x86/boot: Get rid of the .head.text section
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The .head.text section is now empty, so it can be dropped from the linker script.
--- arch/x86/kernel/vmlinux.lds.S | 5 ----- 1 file changed, 5 deletions(-)
arch/x86/kernel/vmlinux.lds.S +0 -5
@@ -160,11 +160,6 @@ SECTIONS } :text = 0xcccccccc - /* bootstrapping code */ - .head.text : AT(ADDR(.head.text) - LOAD_OFFSET) { - HEAD_TEXT - } :text = 0xcccccccc - /* End of text section, which should occupy whole number of pages */ _etext = .; . = ALIGN(PAGE_SIZE);
Signature · 1 line
2.51.0.268.g9569e192d0-goog
[tip: x86/sev] x86/boot: Get rid of the .head.text section
The following commit has been merged into the x86/sev branch of tip:
The following commit has been merged into the x86/sev branch of tip:
Commit-ID: ce39a6aa8802e718f9b68bf6892612e4fd7f9d2d Gitweb: https://git.kernel.org/tip/ce39a6aa8802e718f9b68bf6892612e4fd7f9d2d Author: Ard Biesheuvel <[email protected]> AuthorDate: Thu, 28 Aug 2025 12:22:25 +02:00 Committer: Borislav Petkov (AMD) <[email protected]> CommitterDate: Wed, 03 Sep 2025 18:06:32 +02:00
x86/boot: Get rid of the .head.text section
The .head.text section is now empty, so it can be dropped from the linker script.
--- arch/x86/kernel/vmlinux.lds.S | 5 ----- 1 file changed, 5 deletions(-)
arch/x86/kernel/vmlinux.lds.S +0 -5
@@ -160,11 +160,6 @@ SECTIONS } :text = 0xcccccccc - /* bootstrapping code */ - .head.text : AT(ADDR(.head.text) - LOAD_OFFSET) { - HEAD_TEXT - } :text = 0xcccccccc - /* End of text section, which should occupy whole number of pages */ _etext = .; . = ALIGN(PAGE_SIZE);
Borislav Petkov
maintainerQuoted 2 lines · Ard Biesheuvel
Changes since v6:
- Rebase onto latest tip/master which incorporates v6.17-rc1a
...
So, due to the interactions with the Secure AVIC stuff, I've been doing some patch tetris. Two patches: the first one goes ontop of x86/apic and the second one goes ontop of this set.
Will run some build tests with them first tho...
patch 1:
---
commit aa532319e46228422f7deb8d54853c4b218276f1 (HEAD -> refs/heads/tip-x86-apic)
Author: Borislav Petkov (AMD) <[email protected]> Date: Wed Sep 3 17:42:05 2025 +0200
WIP
arch/x86/coco/sev/core.c +2 -2
@@ -1129,7 +1129,7 @@ u64 savic_ghcb_msr_read(u32 reg)if (res !=ES_OK ) {pr_err ("Secure AVIC MSR (0x%llx) read returned error (%d)\n" , msr, res);/* MSR read failures are treated as fatal errors */ -snp_abort (); +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SAVIC_FAIL ); }__sev_put_ghcb (&state);
@@ -1159,7 +1159,7 @@ void savic_ghcb_msr_write(u32 reg, u64 value)if (res !=ES_OK ) {pr_err ("Secure AVIC MSR (0x%llx) write returned error (%d)\n" , msr, res);/* MSR writes should never fail. Any failure is fatal error for SNP guest */ -snp_abort (); +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SAVIC_FAIL ); }__sev_put_ghcb (&state);
arch/x86/include/asm/sev-common.h +1 -0
@@ -208,6 +208,7 @@ struct snp_psc_desc {#define GHCB_TERM_SVSM_CAA 9/* SVSM is present but CAA is not page aligned */ #define GHCB_TERM_SECURE_TSC 10/* Secure TSC initialization failed */ #define GHCB_TERM_SVSM_CA_REMAP_FAIL 11/* SVSM is present but CA could not be remapped */ +#define GHCB_TERM_SAVIC_FAIL 12/* Secure AVIC-specific failure */ #define GHCB_RESP_CODE (v) ((v) & GHCB_MSR_INFO_MASK)
arch/x86/kernel/apic/x2apic_savic.c +3 -3
@@ -363,7 +363,7 @@ static void savic_setup(void)*/ res =savic_register_gpa (gpa);if (res !=ES_OK ) -snp_abort (); +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SAVIC_FAIL );native_wrmsrq (MSR_AMD64_SAVIC_CONTROL , gpa |MSR_AMD64_SAVIC_EN |MSR_AMD64_SAVIC_ALLOWEDNMI );
@@ -376,13 +376,13 @@ static int savic_probe(void)if (!x2apic_mode) {pr_err ("Secure AVIC enabled in non x2APIC mode\n" ); -snp_abort (); +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SAVIC_FAIL );/* unreachable */ } savic_page = alloc_percpu(struct secure_avic_page);if (!savic_page) -snp_abort (); +sev_es_terminate (SEV_TERM_SET_LINUX ,GHCB_TERM_SAVIC_FAIL );;return 1 ; }
---
patch 2
---
commit 07d41a19c5a01506e1080e352c26c50c8dce6e6b (refs/remotes/ps2/tip-x86-sev, refs/remotes/ps2/HEAD)
Author: Borislav Petkov (AMD) <[email protected]> Date: Wed Sep 3 18:14:54 2025 +0200
WIP
arch/x86/boot/startup/sev-startup.c +1 -6
@@ -144,7 +144,7 @@ static struct cc_blob_sev_info *__init find_cc_blob(struct boot_params *bp) found_cc_info:if (cc_info->magic !=CC_BLOB_SEV_HDR_MAGIC ) -snp_abort (); +sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED );return cc_info; }
@@ -218,8 +218,3 @@ bool __init snp_init(struct boot_params *bp)return true; } - -void __init __noreturn snp_abort (void ) -{ -sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED ); -}
arch/x86/boot/startup/sme.c +1 -1
@@ -532,7 +532,7 @@ void __init sme_enable(struct boot_params *bp)* enablement abort the guest. */ if (snp_en ^ !!(msr &MSR_AMD64_SEV_SNP_ENABLED )) -snp_abort (); +sev_es_terminate (SEV_TERM_SET_GEN ,GHCB_SNP_UNSUPPORTED );/* Check if memory encryption is enabled */ if (feature_mask ==AMD_SME_BIT ) {
arch/x86/include/asm/sev.h +0 -2
@@ -512,7 +512,6 @@ void snp_set_memory_shared(unsigned long vaddr, unsigned long npages);void snp_set_memory_private (unsigned long vaddr,unsigned long npages);void snp_set_wakeup_secondary_cpu (void );bool snp_init (struct boot_params *bp); -void __noreturn snp_abort (void );void snp_dmi_setup (void );int snp_issue_svsm_attest_req (u64 call_id,struct svsm_call *call,struct svsm_attest_call *input);void snp_accept_memory (phys_addr_t start,phys_addr_t end);
@@ -597,7 +596,6 @@ static inline void snp_set_memory_shared(unsigned long vaddr, unsigned long npagstatic inline void snp_set_memory_private (unsigned long vaddr,unsigned long npages) { }static inline void snp_set_wakeup_secondary_cpu (void ) { }static inline bool snp_init (struct boot_params *bp) {return false; } -static inline void snp_abort (void ) { }static inline void snp_dmi_setup (void ) { }static inline int snp_issue_svsm_attest_req (u64 call_id,struct svsm_call *call,struct svsm_attest_call *input) {
tools/objtool/noreturns.h +0 -1
@@ -45,7 +45,6 @@ NORETURN(rewind_stack_and_make_dead) NORETURN(rust_begin_unwind) NORETURN(rust_helper_BUG) NORETURN(sev_es_terminate) -NORETURN(snp_abort) NORETURN(start_kernel) NORETURN(stop_this_cpu) NORETURN(usercopy_abort)
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteArd Biesheuvel
maintainerQuoted 11 lines · Borislav Petkov
Quoted 2 lines · Ard Biesheuvel
Changes since v6:
- Rebase onto latest tip/master which incorporates v6.17-rc1a...
So, due to the interactions with the Secure AVIC stuff, I've been doing some patch tetris. Two patches: the first one goes ontop of x86/apic and the second one goes ontop of this set.
Looks straight-forward enough ...