Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support
Subsystems (12)
v7 cover Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support
This patchset is based on top of the following patchset:
"[PATCH v10 0/9] KVM: mm: fd-based approach for supporting KVM"
https://lore.kernel.org/lkml/[email protected]/T/#me1dd3a4c295758b4e4ac8ff600f2db055bc5f987
and is also available at:
https://github.com/amdese/linux/commits/upmv10-host-snp-v7-rfc
== OVERVIEW ==
This version is being posted as an RFC due to fairly extensive changes relating to transitioning the SEV-SNP implementation to using restricted/private memslots (aka Unmapped Private Memory) to manage private guest pages instead of the legacy SEV memory registration ioctls.
Alongside that work we've also been investigating leveraging UPM to to implement lazy-pinning support for SEV guests, rather than the legacy SEV memory registration ioctls which rely on pinning everything in advance.
For both of these SEV and SEV-SNP use-cases we've needed to add a number of hooks in the restricted, so we thought it would be useful for this version at least to include both UPM-based SEV and SNP implementations so can see if these hooks might be needed for other archs/platforms and start consolidating around whether/how they should be defined for general usage. There are still some TODOs in this area, but we hope this implementation is complete enough to at least outline the required additions needed for using UPM for these use-cases.
Outside of UPM-related items, we've also included fairly extensive changes based on review feedback from v6 and would appreciate any feedback on those aspects as well.
== LAYOUT ==
PATCH 01-06: pre-patches that add the UPM hooks and KVM capability needed
to switch between UPM and legacy SEV memory registration.
PATCH 07-12: implement SEV lazy-pinning using UPM to manage private memory PATCH 13-32: general SNP detection/enablement for host and CCP driver PATCH 33-58: base KVM-specific support for running SNP guests PATCH 59-64: misc./documentation/IOMMU changes
== TESTING ==
Tested with the following QEMU tree, which is based on Chao Peng's UPM v10 QEMU tree:
https://github.com/mdroth/qemu/commits/upmv10-snpv3
SEV-SNP with UPM:
qemu-system-x86_64 -cpu EPYC-Milan-v2 \
-object memory-backend-memfd-private,id=ram1,size=1G,share=true \
-object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,upm-mode=on \
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
...SEV with UPM (requires patched OVMF[1]):
qemu-system-x86_64 -cpu EPYC-Milan-v2 \
-object memory-backend-memfd-private,id=ram1,size=1G,share=true \
-object sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,upm-mode=on \
-machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
...[1] https://github.com/mdroth/edk2/commits/upmv8-seves-v1
== BACKGROUND ==
This part of the Secure Encrypted Paging (SEV-SNP) series focuses on the changes required in a host OS for SEV-SNP support. The series builds upon SEV-SNP Guest Support now part of mainline.
This series provides the basic building blocks to support booting the SEV-SNP VMs, it does not cover all the security enhancement introduced by the SEV-SNP such as interrupt protection.
The CCP driver is enhanced to provide new APIs that use the SEV-SNP specific commands defined in the SEV-SNP firmware specification. The KVM driver uses those APIs to create and managed the SEV-SNP guests.
The GHCB specification version 2 introduces new set of NAE's that is used by the SEV-SNP guest to communicate with the hypervisor. The series provides support to handle the following new NAE events:
- Register GHCB GPA
- Page State Change Request
- Hypevisor feature
- Guest message request
The RMP check is enforced as soon as SEV-SNP is enabled. Not every memory access requires an RMP check. In particular, the read accesses from the hypervisor do not require RMP checks because the data confidentiality is already protected via memory encryption. When hardware encounters an RMP checks failure, it raises a page-fault exception. If RMP check failure is due to the page-size mismatch, then split the large page to resolve the fault.
The series does not provide support for the interrupt security and migration and those feature will be added after the base support.
Changes since v6:
* Added support for restrictedmem/UPM, and removed SEV-specific implementation of private memory management. As a result of this rework the following patches were no longer needed so were dropped:
- KVM: SVM: Mark the private vma unmergable for SEV-SNP guests
- KVM: SVM: Disallow registering memory range from HugeTLB for SNP guest
- KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX and SNP
- KVM: x86: Introduce kvm_mmu_get_tdp_walk() for SEV-SNP use
* Moved RMP table entry structure definition (struct rmpentry) to sev.c, to not expose this non-architectural definition to rest of the kernel and making the structure private to SNP code. Also made RMP table entry accessors to be inline functions and removed all accessors which are not called more than once. Added a new function rmptable_entry() to index into the RMP table and return RMP table entry.
* Moved RMPUPDATE, PSMASH helper function declerations to x86 arch specific include namespace from linux namespace. Added comments for these helper functions.
* Introduce set_memory_p() to provide a way to change atributes of a memory range to be marked as present and added to the kernel directmap, and invalidating/restoring pages from directmap are now done using set_memory_np() and set_memory_p().
* Added detailed comments around user RMP #PF fault handling and simplified computation of the faulting pfn for large-pages.
* Added support to return pfn from dump_pagetable() to do SEV-specific fault handling, this is added a pre-patch. This support is now used to dump RMP entry in case of RMP #PF in show_fault_oops().
* Added a new generic SNP command params structure sev_data_snp_addr, which is used for all SNP firmware API commands requiring a single physical address parameter.
* Added support for new SNP_INIT_EX command with support for HV-Fixed page range list.
* Added support for new SNP_SHUTDOWN_EX command which allows disabling enforcement of SNP in the IOMMU. Also DF_FLUSH is done at SNP shutdown if it indicates DF_FLUSH is required.
* Make sev_do_cmd() a generic API interface for the hypervisor to issue commands to manage an SEV and SNP guest. Also removed the API wrappers used by the hypervisor to manage an SEV-SNP guest. All these APIs now invoke sev_do_cmd() directly.
* Introduce snp leaked pages list. If pages are unsafe to be released back to the page-allocator as they can't be reclaimed or transitioned back to hypervisor/shared state are now added
to this internal leaked pages list to prevent fatal page faults when accessing these pages. The function snp_leak_pages() is renamed to snp_mark_pages_offline() and is an external function available to both CCP driver and the SNP hypervisor code. Removed call to memory_failure() when leaking/marking pages offline.
* Remove snp_set_rmp_state() multiplexor code and add new separate helpers such as rmp_mark_pages_firmware() & rmp_mark_pages_shared(). The callers now issue snp_reclaim_pages() directly when needed as done by __snp_free_firmware_pages() and unmap_firmware_writeable(). All callers of snp_set_rmp_state() modified to call helpers rmp_mark_pages_firmware() or rmp_mark_pages_shared() as required.
* Change snp_reclaim_pages() to take physical address as an argument and clear C-bit from this physical address argument internally.
* Output parameter sev_user_data_ext_snp_config in sev_ioctl_snp_get_config() is memset to zero to avoid kernel memory leaking.
* Prevent race between sev_ioctl_snp_set_config() and snp_guest_ext_guest_request() for sev->snp_certs_data by acquiring sev->snp_certs_lock mutex.
* Zeroed out struct sev_user_data_snp_config in sev_ioctl_snp_set_config() to prevent leaking uninitialized kernel memory.
* Optimized snp_safe_alloc_page() by avoiding multiple calls to pfn_to_page() and checking for a hugepage using pfn instead of expanding to full physical address.
* Invoke host_rmp_make_shared() with leak parameter set to true if VMSA page cannot be transitioned back to shared state.
* Fix snp_launch_finish() to always sent the ID_AUTH struct to the firmware. Use params.auth_key_en indicator to set
if the ID_AUTH struct contains an author key or not.
* Cleanup snp_context_create() and allocate certs_data in this function using kzalloc() to prevent giving the guest uninitialized kernel memory.
* Remove the check for guest supplied buffer greater than the data provided by the hypervisor in snp_handle_ext_guest_request().
* Add check in sev_snp_ap_create() if a malicious guest can RMPADJUST a large page into VMSA which will hit the SNP erratum where the CPU will incorrectly signal an RMP violation #PF if a hugepage collides with the RMP entry of VMSA page, reject the AP CREATE request if VMSA address from guest is 2M aligned.
* Make VMSAVE target area memory allocation SNP safe, implemented workaround for an SNP erratum where the CPU will incorrectly signal an RMP violation #PF if a hugepage (2mb or 1gb) collides with the RMP entry of the VMSAVE target page.
* Fix handle_split_page_fault() to work with memfd backed pages.
* Add KVM commands for per-VM instance certificates.
* Add IOMMU_SNP_SHUTDOWN support, this adds support for Host kexec support with SNP.
----------------------------------------------------------------
Ashish Kalra (6):
x86/mm/pat: Introduce set_memory_p
x86/fault: Return pfn from dump_pagetable() for SEV-specific fault handling.
crypto: ccp: Introduce snp leaked pages list
KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb
KVM: SVM: Make VMSAVE target area memory allocation SNP safe iommu/amd: Add IOMMU_SNP_SHUTDOWN support
Brijesh Singh (36):
x86/cpufeatures: Add SEV-SNP CPU feature
x86/sev: Add the host SEV-SNP initialization support
x86/sev: Add RMP entry lookup helpers
x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction x86/sev: Invalidate pages from the direct map when adding them to the RMP table x86/traps: Define RMP violation #PF error code
x86/fault: Add support to handle the RMP fault for user address x86/fault: Add support to dump RMP entry on fault
crypto:ccp: Define the SEV-SNP commands
crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP crypto:ccp: Provide API to issue SEV and SNP commands
crypto: ccp: Handle the legacy TMR allocation when SNP is enabled
crypto: ccp: Handle the legacy SEV command when SNP is enabled
crypto: ccp: Add the SNP_PLATFORM_STATUS command
crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command
crypto: ccp: Provide APIs to query extended attestation report
KVM: SVM: Provide the Hypervisor Feature support VMGEXIT
KVM: SVM: Make AVIC backing, VMSA and VMCB memory allocation SNP safe
KVM: SVM: Add initial SEV-SNP support
KVM: SVM: Add KVM_SNP_INIT command
KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command
KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command
KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command
KVM: X86: Keep the NPT and RMP page level in sync
KVM: x86: Define RMP page fault error bits for #NPF
KVM: SVM: Do not use long-lived GHCB map while setting scratch area
KVM: SVM: Remove the long-lived GHCB host map
KVM: SVM: Add support to handle GHCB GPA register VMGEXIT
KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT
KVM: SVM: Add support to handle Page State Change VMGEXIT
KVM: SVM: Introduce ops for the post gfn map and unmap
KVM: x86: Export the kvm_zap_gfn_range() for the SNP use
KVM: SVM: Add support to handle the RMP nested page fault
KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event
KVM: SVM: Add module parameter to enable the SEV-SNP
ccp: Add support to decrypt the page
Dionna Glaze (2):
x86/sev: Add KVM commands for instance certs x86/sev: Document KVM_SEV_SNP_{G,S}ET_CERTS
Hugh Dickins (1):
x86/fault: fix handle_split_page_fault() to work with memfd backed pages
Michael Roth (9):
KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
KVM: x86: Add 'fault_is_private' x86 op
KVM: x86: Add 'update_mem_attr' x86 op
KVM: x86: Add platform hooks for private memory invalidations
KVM: SEV: Implement .fault_is_private callback
KVM: SVM: Add KVM_EXIT_VMGEXIT
KVM: SVM: Add SNP-specific handling for memory attribute updates
KVM: x86/mmu: Generate KVM_EXIT_MEMORY_FAULT for implicit conversions for SNP
KVM: SEV: Handle restricted memory invalidations for SNP
Nikunj A Dadhania (5):
KVM: Fix memslot boundary condition for large page
KVM: SVM: Advertise private memory support to KVM
KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
KVM: Move kvm_for_each_memslot_in_hva_range() to be used in SVM
KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
Tom Lendacky (3):
KVM: SVM: Add support to handle AP reset MSR protocol
KVM: SVM: Use a VMSA physical address variable for populating VMCB
KVM: SVM: Support SEV-SNP AP Creation NAE event
Vishal Annapurve (2):
KVM: Add HVA range operator
KVM: SEV: Populate private memory fd during LAUNCH_UPDATE_DATA
Documentation/virt/coco/sev-guest.rst | 54 + .../virt/kvm/x86/amd-memory-encryption.rst | 146 ++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/kvm-x86-ops.h | 6 + arch/x86/include/asm/kvm_host.h | 23 + arch/x86/include/asm/msr-index.h | 11 +- arch/x86/include/asm/set_memory.h | 3 +- arch/x86/include/asm/sev-common.h | 28 + arch/x86/include/asm/sev.h | 28 + arch/x86/include/asm/svm.h | 6 + arch/x86/include/asm/trap_pf.h | 18 +- arch/x86/kernel/cpu/amd.c | 5 +- arch/x86/kernel/sev.c | 437 ++++ arch/x86/kvm/lapic.c | 5 +- arch/x86/kvm/mmu.h | 2 - arch/x86/kvm/mmu/mmu.c | 34 +- arch/x86/kvm/mmu/mmu_internal.h | 40 +- arch/x86/kvm/svm/sev.c | 2217 +++++++++++++++++--- arch/x86/kvm/svm/svm.c | 84 +- arch/x86/kvm/svm/svm.h | 75 +- arch/x86/kvm/trace.h | 34 + arch/x86/kvm/x86.c | 36 + arch/x86/mm/fault.c | 118 +- arch/x86/mm/pat/set_memory.c | 12 +- drivers/crypto/ccp/sev-dev.c | 1055 +++++++++- drivers/crypto/ccp/sev-dev.h | 18 + drivers/iommu/amd/init.c | 53 + include/linux/amd-iommu.h | 1 + include/linux/kvm_host.h | 15 + include/linux/mm.h | 3 +- include/linux/mm_types.h | 3 + include/linux/psp-sev.h | 352 +++- include/uapi/linux/kvm.h | 75 + include/uapi/linux/psp-sev.h | 60 + mm/memory.c | 15 + mm/restrictedmem.c | 16 + tools/arch/x86/include/asm/cpufeatures.h | 1 + virt/kvm/kvm_main.c | 87 +- 39 files changed, 4838 insertions(+), 347 deletions(-)
RFC PATCH v7 1/64 KVM: Fix memslot boundary condition for large page
From: Nikunj A Dadhania <[email protected]>
Aligned end boundary causes a kvm crash, handle the case.
--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kvm/mmu/mmu.c +3 -0
@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false); +if (gfn == last) + goto out; + gfn = last; gfn_end =min (last + pages, slot->base_gfn + slot->npages); mixed =mem_attrs_mixed (kvm, slot, level, attrs, gfn, gfn_end);
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 01/64] KVM: Fix memslot boundary condition for large page
Quoted 15 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Aligned end boundary causes a kvm crash, handle the case.
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0
arch/x86/kvm/mmu/mmu.c static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, · line 7163
Quoted 5 lines
arch/x86/kvm/mmu/mmu.c +2
-0@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false); +if (gfn == last) + goto out;
I'm guessing this was supposed to be "return;" here:
arch/x86/kvm/mmu/mmu.c: In function ‘kvm_update_lpage_private_shared_mixed’:arch/x86/kvm/mmu/mmu.c:7090:25: error: label ‘out’ used but not defined
7090 | goto out;
| ^~~~/me goes and digs deeper.
Aha, it was a "return" but you reordered the patches and the one adding the out label:
KVM: x86: Add 'update_mem_attr' x86 op
went further down and this became the first but it didn't have the label anymore.
Yeah, each patch needs to build successfully for bisection reasons, ofc.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 01/64] KVM: Fix memslot boundary condition for large page
Quoted 19 lines · Borislav Petkov
Quoted 18 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Aligned end boundary causes a kvm crash, handle the case.
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false);
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, · line 7163
Quoted 2 lines
arch/x86/kvm/mmu/mmu.c +2
-0+if (gfn == last) + goto out;I'm guessing this was supposed to be "return;" here:
If we finally need this, this should be "continue;", we can't skip the remaining huge page levels.
Thanks,
Chao
Quoted 25 lines
arch/x86/kvm/mmu/mmu.c: In function ‘kvm_update_lpage_private_shared_mixed’:arch/x86/kvm/mmu/mmu.c:7090:25: error: label ‘out’ used but not defined7090 | goto out; | ^~~~/me goes and digs deeper.
Aha, it was a "return" but you reordered the patches and the one adding the out label:
KVM: x86: Add '
update_mem_attr' x86 opwent further down and this became the first but it didn't have the label anymore.
Yeah, each patch needs to build successfully for bisection reasons, ofc.
Thx.
Signature · 4 lines
Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Re: [PATCH RFC v7 01/64] KVM: Fix memslot boundary condition for large page
In reply to Michael RothQuoted 4 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Aligned end boundary causes a kvm crash, handle the case.
Chao, are you aware of this issue already?
Quoted 23 lines
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kvm/mmu/mmu.c +3
-0@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false); +if (gfn == last) + goto out; + gfn = last; gfn_end =min (last + pages, slot->base_gfn + slot->npages); mixed =mem_attrs_mixed (kvm, slot, level, attrs, gfn, gfn_end);Signature · 1 line
2.25.1
BR, Jarkko
Re: [PATCH RFC v7 01/64] KVM: Fix memslot boundary condition for large page
Quoted 9 lines · Jarkko Sakkinen
Quoted 4 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Aligned end boundary causes a kvm crash, handle the case.
Chao, are you aware of this issue already?
Thanks Jarkko adding me. I'm not aware of there is a fix.
Quoted 15 lines
Quoted 15 lines
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false);
Quoted 3 lines
arch/x86/kvm/mmu/mmu.c static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, · line 7164
Quoted 3 lines
arch/x86/kvm/mmu/mmu.c +3
-0+if (gfn == last) + goto out; +
Nikunj or Michael, could you help me understand in which case it causes a KVM crash? To me, even the end is aligned to huge page boundary, but:
last = (end - 1) & mask;
so 'last' is the base address for the last effective huage page. Even when gfn == last, it should still a valid page and needs to be updated for mem_attrs, correct?
Thanks,
Chao
Quoted 9 lines
Quoted 6 lines
arch/x86/kvm/mmu/mmu.c +0
-0gfn = last; gfn_end =min (last + pages, slot->base_gfn + slot->npages); mixed =mem_attrs_mixed (kvm, slot, level, attrs, gfn, gfn_end);Signature · 1 line
2.25.1BR, Jarkko
Re: [PATCH RFC v7 01/64] KVM: Fix memslot boundary condition for large page
Quoted 4 lines · Chao Peng
Quoted 3 lines · Jarkko Sakkinen
Quoted 2 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Aligned end boundary causes a kvm crash, handle the case.
Chao, are you aware of this issue already?
Thanks Jarkko adding me. I'm not aware of there is a fix.
It was discussed here: https://lore.kernel.org/all/[email protected]/
I was hitting this with one of the selftests case.
Quoted 22 lines
Quoted 16 lines
Quoted 16 lines
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false);Quoted 3 lines
arch/x86/kvm/mmu/mmu.c static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, · line 7164
Quoted 3 lines
arch/x86/kvm/mmu/mmu.c +3
-0+if (gfn == last) + goto out; +Nikunj or Michael, could you help me understand in which case it causes a KVM crash? To me, even the end is aligned to huge page boundary, but:
Quoted 4 lines
last = (end - 1) & mask;so 'last' is the base address for the last effective huage page. Even when gfn == last, it should still a valid page and needs to be updated for
mem_attrs, correct?
Yes, that is correct with: last = (end - 1) & mask;
We can drop this patch from SNP series.
Regards
Nikunj
Re: [PATCH RFC v7 01/64] KVM: Fix memslot boundary condition for large page
Quoted 12 lines · Nikunj A. Dadhania
Quoted 9 lines · Chao Peng
Quoted 8 lines · Jarkko Sakkinen
Quoted 4 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Aligned end boundary causes a kvm crash, handle the case.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Chao, are you aware of this issue already?
Thanks Jarkko adding me. I'm not aware of there is a fix.
It was discussed here: https://lore.kernel.org/all/[email protected]/
I was hitting this with one of the selftests case.
Yeah, I remember that discussion. With the new UPM code, this bug should be fixed. If you still hit the issue please let me know.
Thanks,
Chao
Quoted 34 lines
Quoted 23 lines
Quoted 17 lines
Quoted 17 lines
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kvm/mmu/mmu.c +0
-0@@ -7159,6 +7159,9 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false);Quoted 3 lines
arch/x86/kvm/mmu/mmu.c static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, · line 7164
Quoted 3 lines
arch/x86/kvm/mmu/mmu.c +3
-0+if (gfn == last) + goto out; +Nikunj or Michael, could you help me understand in which case it causes a KVM crash? To me, even the end is aligned to huge page boundary, but:
Quoted 4 lines
last = (end - 1) & mask;so 'last' is the base address for the last effective huage page. Even when gfn == last, it should still a valid page and needs to be updated for
mem_attrs, correct?Yes, that is correct with: last = (end - 1) & mask;
We can drop this patch from SNP series.
Regards
Nikunj
RFC PATCH v7 2/64 KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
In reply to Michael RothThis mainly indicates to KVM that it should expect all private guest memory to be backed by private memslots. Ideally this would work similarly for others archs, give or take a few additional flags, but for now it's a simple boolean indicator for x86.
--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 10 ++++++++++ include/uapi/linux/kvm.h | 1 + 3 files changed, 14 insertions(+)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -1438,6 +1438,9 @@ struct kvm_arch {*/ #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1)struct kvm_mmu_memory_cache split_desc_cache; + +/* Use/enforce unmapped private memory. */ +bool upm_mode; };struct kvm_vm_stat {
arch/x86/kvm/x86.c +10 -0
@@ -4421,6 +4421,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_EXIT_HYPERCALL : r =KVM_EXIT_HYPERCALL_VALID_MASK ;break ; +#ifdef CONFIG_HAVE_KVM_MEMORY_ATTRIBUTES +case KVM_CAP_UNMAPPED_PRIVATE_MEM : + r =1 ; +break ; +#endif case KVM_CAP_SET_GUEST_DEBUG2 :return KVM_GUESTDBG_VALID_MASK ;#ifdef CONFIG_KVM_XEN
@@ -6382,6 +6387,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, }mutex_unlock (&kvm->lock);break ; +case KVM_CAP_UNMAPPED_PRIVATE_MEM : + kvm->arch.upm_mode = true; + r =0 ; +break ;default : r = -EINVAL ;break ;
@@ -12128,6 +12137,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.guest_can_read_msr_platform_info = true; kvm->arch.enable_pmu = enable_pmu; + kvm->arch.upm_mode = false;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);
include/uapi/linux/kvm.h +1 -0
@@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224#define KVM_CAP_MEMORY_ATTRIBUTES 225 +#define KVM_CAP_UNMAPPED_PRIVATE_MEM 240#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 02/64] KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
Quoted 4 lines · Michael Roth
This mainly indicates to KVM that it should expect all private guest memory to be backed by private memslots. Ideally this would work similarly for others archs, give or take a few additional flags, but for now it's a simple boolean indicator for x86.
...
include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1222
Quoted 9 lines
include/uapi/linux/kvm.h +1
-0@@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224#define KVM_CAP_MEMORY_ATTRIBUTES 225 +#define KVM_CAP_UNMAPPED_PRIVATE_MEM 240
Isn't this new cap supposed to be documented somewhere in Documentation/virt/kvm/api.rst ?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 02/64] KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
Quoted 8 lines · Borislav Petkov
Quoted 4 lines · Michael Roth
This mainly indicates to KVM that it should expect all private guest memory to be backed by private memslots. Ideally this would work similarly for others archs, give or take a few additional flags, but for now it's a simple boolean indicator for x86.
...
Quoted 12 lines
include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1222
Quoted 9 lines
include/uapi/linux/kvm.h +1
-0@@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224#define KVM_CAP_MEMORY_ATTRIBUTES 225 +#define KVM_CAP_UNMAPPED_PRIVATE_MEM 240Isn't this new cap supposed to be documented somewhere in
Documentation/virt/kvm/api.rst?
It should, but this is sort of a placeholder for now. Ideally we'd re-use the capabilities introduced by UPM patchset rather than introduce a new one. Originally the UPM patchset had a KVM_CAP_PRIVATE_MEM which we planned to use to switch between legacy SEV and UPM-based SEV (for lazy-pinning support) by making it writeable, but that was removed in v10 in favor of KVM_CAP_MEMORY_ATTRIBUTES, which is tied to the new KVM_GET_SUPPORTED_MEMORY_ATTRIBUTES/KVM_SET_MEMORY_ATTRIBUTES ioctls:
https://lore.kernel.org/lkml/CA+EHjTxXOdzcP25F57Mtmnb1NWyG5DcyqeDPqzjEOzRUrqH8FQ@mail.gmail.com/
It wasn't clear at the time if that was the right interface to use for this particular case, so we stuck with the more general 'use-upm/dont-use-upm' semantics originally provided by making KVM_CAP_UNMAPPED_PRIVATE_MEM/KVM_CAP_PRIVATE_MEM writeable.
But maybe it's okay to just make KVM_CAP_MEMORY_ATTRIBUTES writeable and require userspace to negotiate it rather than just tying it to CONFIG_HAVE_KVM_MEMORY_ATTRIBUTES. Or maybe introducing a new KVM_SET_SUPPORTED_MEMORY_ATTRIBUTES ioctl to pair with KVM_GET_SUPPORTED_MEMORY_ATTRIBUTES. It sort of makes sense, since userspace needs to be prepared to deal with KVM_EXIT_MEMORY_FAULTs relating to these attributes.
-Mike
Quoted 6 lines
Signature · 5 lines
-- Regards/Gruss, Boris. https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&data=05%7C01%7Cmichael.roth%40amd.com%7Cb019ddcb34a74fae1e3e08dae417c322%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638073087997837943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QRyV96y3drt%2BqwxfifWzJ6UF6te8NOKWAFuGAYOdKcg%3D&reserved=0
Borislav Petkov
maintainerRe: [PATCH RFC v7 02/64] KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
Quoted 7 lines · Michael Roth
But maybe it's okay to just make
KVM_CAP_MEMORY_ATTRIBUTESwriteable and require userspace to negotiate it rather than just tying it toCONFIG_HAVE_KVM_MEMORY_ATTRIBUTES. Or maybe introducing a newKVM_SET_SUPPORTED_MEMORY_ATTRIBUTESioctl to pair withKVM_GET_SUPPORTED_MEMORY_ATTRIBUTES. It sort of makes sense, since userspace needs to be prepared to deal withKVM_EXIT_MEMORY_FAULTsrelating to these attributes.
Makes sense.
AFAICT, ofc.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 02/64] KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
In reply to Michael RothQuoted 37 lines · Michael Roth
Quoted 8 lines · Borislav Petkov
Quoted 4 lines · Michael Roth
This mainly indicates to KVM that it should expect all private guest memory to be backed by private memslots. Ideally this would work similarly for others archs, give or take a few additional flags, but for now it's a simple boolean indicator for x86.
...
Quoted 12 lines
include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1222
Quoted 9 lines
include/uapi/linux/kvm.h +1
-0@@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224#define KVM_CAP_MEMORY_ATTRIBUTES 225 +#define KVM_CAP_UNMAPPED_PRIVATE_MEM 240Isn't this new cap supposed to be documented somewhere in
Documentation/virt/kvm/api.rst?It should, but this is sort of a placeholder for now. Ideally we'd re-use the capabilities introduced by UPM patchset rather than introduce a new one. Originally the UPM patchset had a
KVM_CAP_PRIVATE_MEMwhich we planned to use to switch between legacy SEV and UPM-based SEV (for lazy-pinning support) by making it writeable, but that was removed in v10 in favor ofKVM_CAP_MEMORY_ATTRIBUTES, which is tied to the new KVM_GET_SUPPORTED_MEMORY_ATTRIBUTES/KVM_SET_MEMORY_ATTRIBUTES ioctls:https://lore.kernel.org/lkml/CA+EHjTxXOdzcP25F57Mtmnb1NWyG5DcyqeDPqzjEOzRUrqH8FQ@mail.gmail.com/
It wasn't clear at the time if that was the right interface to use for this particular case, so we stuck with the more general 'use-upm/dont-use-upm' semantics originally provided by making KVM_CAP_UNMAPPED_PRIVATE_MEM/KVM_CAP_PRIVATE_MEM writeable.
Quoted 7 lines
But maybe it's okay to just make
KVM_CAP_MEMORY_ATTRIBUTESwriteable and require userspace to negotiate it rather than just tying it toCONFIG_HAVE_KVM_MEMORY_ATTRIBUTES. Or maybe introducing a newKVM_SET_SUPPORTED_MEMORY_ATTRIBUTESioctl to pair withKVM_GET_SUPPORTED_MEMORY_ATTRIBUTES. It sort of makes sense, since userspace needs to be prepared to deal withKVM_EXIT_MEMORY_FAULTsrelating to these attributes.
Doesn't upm patch set imply that user space should negotiate the memory attributes with the ioctl?
For me it looks like that the problem is introduced by conflicting usage pattern in the SNP code [*].
Perhaps sev_launch_update_gfn_handler() should not set memory attributes but instead expect user space to do it before the call?
[*] https://lore.kernel.org/all/[email protected]/
BR, Jarkko
Re: [PATCH RFC v7 02/64] KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
In reply to Michael RothQuoted 76 lines · Michael Roth
This mainly indicates to KVM that it should expect all private guest memory to be backed by private memslots. Ideally this would work similarly for others archs, give or take a few additional flags, but for now it's a simple boolean indicator for x86.
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 10 ++++++++++ include/uapi/linux/kvm.h | 1 + 3 files changed, 14 insertions(+)arch/x86/include/asm/kvm_host.h +3
-0@@ -1438,6 +1438,9 @@ struct kvm_arch {*/ #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1)struct kvm_mmu_memory_cache split_desc_cache; + +/* Use/enforce unmapped private memory. */ +bool upm_mode; };struct kvm_vm_stat {arch/x86/kvm/x86.c +10
-0@@ -4421,6 +4421,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_EXIT_HYPERCALL : r =KVM_EXIT_HYPERCALL_VALID_MASK ;break ; +#ifdef CONFIG_HAVE_KVM_MEMORY_ATTRIBUTES +case KVM_CAP_UNMAPPED_PRIVATE_MEM : + r =1 ; +break ; +#endif case KVM_CAP_SET_GUEST_DEBUG2 :return KVM_GUESTDBG_VALID_MASK ;#ifdef CONFIG_KVM_XEN @@ -6382,6 +6387,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, }mutex_unlock (&kvm->lock);break ; +case KVM_CAP_UNMAPPED_PRIVATE_MEM : + kvm->arch.upm_mode = true; + r =0 ; +break ;default : r = -EINVAL ;break ;@@ -12128,6 +12137,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.guest_can_read_msr_platform_info = true; kvm->arch.enable_pmu = enable_pmu; + kvm->arch.upm_mode = false;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);include/uapi/linux/kvm.h +1
-0@@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224#define KVM_CAP_MEMORY_ATTRIBUTES 225 +#define KVM_CAP_UNMAPPED_PRIVATE_MEM 240#ifdef KVM_CAP_IRQ_ROUTING Signature · 1 line
2.25.1
Why we want to carry non-UPM support still?
BR, Jarkko
Re: [PATCH RFC v7 02/64] KVM: x86: Add KVM_CAP_UNMAPPED_PRIVATE_MEMORY
Quoted 72 lines · Jarkko Sakkinen
Quoted 71 lines · Michael Roth
This mainly indicates to KVM that it should expect all private guest memory to be backed by private memslots. Ideally this would work similarly for others archs, give or take a few additional flags, but for now it's a simple boolean indicator for x86.
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 10 ++++++++++ include/uapi/linux/kvm.h | 1 + 3 files changed, 14 insertions(+)arch/x86/include/asm/kvm_host.h +3
-0@@ -1438,6 +1438,9 @@ struct kvm_arch {*/ #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1)struct kvm_mmu_memory_cache split_desc_cache; + +/* Use/enforce unmapped private memory. */ +bool upm_mode; };struct kvm_vm_stat {arch/x86/kvm/x86.c +10
-0@@ -4421,6 +4421,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_EXIT_HYPERCALL : r =KVM_EXIT_HYPERCALL_VALID_MASK ;break ; +#ifdef CONFIG_HAVE_KVM_MEMORY_ATTRIBUTES +case KVM_CAP_UNMAPPED_PRIVATE_MEM : + r =1 ; +break ; +#endif case KVM_CAP_SET_GUEST_DEBUG2 :return KVM_GUESTDBG_VALID_MASK ;#ifdef CONFIG_KVM_XEN @@ -6382,6 +6387,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, }mutex_unlock (&kvm->lock);break ; +case KVM_CAP_UNMAPPED_PRIVATE_MEM : + kvm->arch.upm_mode = true; + r =0 ; +break ;default : r = -EINVAL ;break ;@@ -12128,6 +12137,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.guest_can_read_msr_platform_info = true; kvm->arch.enable_pmu = enable_pmu; + kvm->arch.upm_mode = false;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);include/uapi/linux/kvm.h +1
-0@@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_DIRTY_LOG_RING_ACQ_REL 223#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 224#define KVM_CAP_MEMORY_ATTRIBUTES 225 +#define KVM_CAP_UNMAPPED_PRIVATE_MEM 240
Quoted 7 lines
Quoted 5 lines
include/uapi/linux/kvm.h +0
-0#ifdef KVM_CAP_IRQ_ROUTING Signature · 1 line
2.25.1Why we want to carry non-UPM support still?
For SNP, non-UPM support is no longer included in this patchset.
However, this patchset also adds support for UPM-based SEV (for lazy-pinning support). So we still need a way to let userspace switch between those 2 modes.
-Mike
Quoted 3 lines
BR, Jarkko
RFC PATCH v7 3/64 KVM: SVM: Advertise private memory support to KVM
In reply to Michael RothFrom: Nikunj A Dadhania <[email protected]>
KVM should use private memory for guests that have upm_mode flag set.
Add a kvm_x86_ops hook for determining UPM support that accounts for this situation by only enabling UPM test mode in the case of non-SEV guests.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/svm.c | 10 ++++++++++ arch/x86/kvm/x86.c | 8 ++++++++ 4 files changed, 20 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1635,6 +1635,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +int (*private_mem_enabled)(struct kvm *kvm);bool (*has_wbinvd_exit)(void );
arch/x86/kvm/svm/svm.c +10 -0
@@ -4694,6 +4694,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ; } +static int svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode ?1 :0 ; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ) ?1 :0 ; +} +static struct kvm_x86_ops svm_x86_ops__initdata = { .name ="kvm_amd" ,
@@ -4774,6 +4782,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_after_set_cpuid = svm_vcpu_after_set_cpuid, + .private_mem_enabled = svm_private_mem_enabled, + .has_wbinvd_exit = svm_has_wbinvd_exit, .get_l2_tsc_offset = svm_get_l2_tsc_offset,
arch/x86/kvm/x86.c +8 -0
@@ -12266,6 +12266,14 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, }EXPORT_SYMBOL_GPL (__x86_set_memory_region); +bool kvm_arch_has_private_mem (struct kvm *kvm) +{ +if (static_call (kvm_x86_private_mem_enabled)(kvm)) +return true; + +return false; +} +void kvm_arch_pre_destroy_vm (struct kvm *kvm) {kvm_mmu_pre_destroy_vm (kvm);
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 1 line · Michael Roth
+ bool (*private_mem_enabled)(struct kvm *kvm);
This looks like a function returning boolean to me. IOW, you can simplify this to:
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -129,6 +129,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1574,6 +1574,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +bool (*private_mem_enabled)(struct kvm *kvm);bool (*has_wbinvd_exit)(void );
arch/x86/kvm/svm/svm.c +10 -0
@@ -4680,6 +4680,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ; } +static bool svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ); +} +static struct kvm_x86_ops svm_x86_ops__initdata = { .name ="kvm_amd" ,
@@ -4760,6 +4768,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_after_set_cpuid = svm_vcpu_after_set_cpuid, + .private_mem_enabled = svm_private_mem_enabled, + .has_wbinvd_exit = svm_has_wbinvd_exit, .get_l2_tsc_offset = svm_get_l2_tsc_offset,
arch/x86/kvm/x86.c +5 -0
@@ -12556,6 +12556,11 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, }EXPORT_SYMBOL_GPL (__x86_set_memory_region); +bool kvm_arch_has_private_mem (struct kvm *kvm) +{ +return static_call (kvm_x86_private_mem_enabled)(kvm); +} +void kvm_arch_pre_destroy_vm (struct kvm *kvm) {kvm_mmu_pre_destroy_vm (kvm);
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 5 lines · Borislav Petkov
Quoted 1 line · Michael Roth
+ bool (*private_mem_enabled)(struct kvm *kvm);
This looks like a function returning boolean to me. IOW, you can simplify this to:
The semantics and existing uses of KVM_X86_OP_OPTIONAL_RET0() gave me the impression it needed to return an integer value, since by default if a platform doesn't implement the op it would "return 0", and so could still be called unconditionally.
Maybe that's not actually enforced, by it seems awkward to try to use a bool return instead. At least for KVM_X86_OP_OPTIONAL_RET0().
However, we could just use KVM_X86_OP() to declare it so we can cleanly use a function that returns bool, and then we just need to do:
bool kvm_arch_has_private_mem(struct kvm *kvm)
{
if (kvm_x86_ops.private_mem_enabled)
return static_call(kvm_x86_private_mem_enabled)(kvm);
}instead of relying on default return value. So I'll take that approach and adopt your other suggested changes.
...
On a separate topic though, at a high level, this hook is basically a way for platform-specific code to tell generic KVM code that private memslots are supported by overriding the kvm_arch_has_private_mem() weak reference. In this case the AMD platform is using using kvm->arch.upm_mode flag to convey that, which is in turn set by the KVM_CAP_UNMAPPED_PRIVATE_MEMORY introduced in this series.
But if, as I suggested in response to your PATCH 2 comments, we drop KVM_CAP_UNAMMPED_PRIVATE_MEMORY in favor of KVM_SET_SUPPORTED_MEMORY_ATTRIBUTES ioctl to enable "UPM mode" in SEV/SNP code, then we need to rethink things a bit, since KVM_SET_MEMORY_ATTRIBUTES in-part relies on kvm_arch_has_private_mem() to determine what flags are supported, whereas SEV/SNP code would be using what was set by KVM_SET_MEMORY_ATTRIBUTES to determine the return value in kvm_arch_has_private_mem().
So, for AMD, the return value of kvm_arch_has_private_mem() needs to rely on something else. Maybe the logic can just be:
bool svm_private_mem_enabled(struct kvm *kvm)
{
return sev_enabled(kvm) || sev_snp_enabled(kvm)
}(at least in the context of this patchset where UPM support is added for both SEV and SNP).
So I'll plan to make that change as well.
-Mike
Quoted 75 lines
arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -129,6 +129,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +1
-0@@ -1574,6 +1574,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +bool (*private_mem_enabled)(struct kvm *kvm);bool (*has_wbinvd_exit)(void );arch/x86/kvm/svm/svm.c +10
-0@@ -4680,6 +4680,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ; } +static bool svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ); +} +static struct kvm_x86_ops svm_x86_ops__initdata = { .name ="kvm_amd" ,@@ -4760,6 +4768,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_after_set_cpuid = svm_vcpu_after_set_cpuid, + .private_mem_enabled = svm_private_mem_enabled, + .has_wbinvd_exit = svm_has_wbinvd_exit, .get_l2_tsc_offset = svm_get_l2_tsc_offset,arch/x86/kvm/x86.c +5
-0@@ -12556,6 +12556,11 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, }EXPORT_SYMBOL_GPL (__x86_set_memory_region); +bool kvm_arch_has_private_mem (struct kvm *kvm) +{ +return static_call (kvm_x86_private_mem_enabled)(kvm); +} +void kvm_arch_pre_destroy_vm (struct kvm *kvm) {kvm_mmu_pre_destroy_vm (kvm);Signature · 5 lines
-- Regards/Gruss, Boris. https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&data=05%7C01%7Cmichael.roth%40amd.com%7C319e89ce555a46eace4d08dae506b51a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638074114318137471%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aG11K7va1BhemwlKCKKdcIXEwXGUzImYL%2BZ9%2FQ7XToI%3D&reserved=0
Borislav Petkov
maintainerRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 2 lines · Michael Roth
Maybe that's not actually enforced, by it seems awkward to try to use a bool return instead. At least for
KVM_X86_OP_OPTIONAL_RET0().
I don't see there being a problem/restriction for bool functions, see
5be2226f417d ("KVM: x86: allow defining return-0 static calls")
and __static_call_return0() returns a long which, if you wanna interpret as bool, works too as "false".
I still need to disassemble and single-step through a static_call to see what all that magic does in detail, to be sure.
Quoted 7 lines
However, we could just use
KVM_X86_OP()to declare it so we can cleanly use a function that returns bool, and then we just need to do:bool kvm_arch_has_private_mem(struct kvm *kvm) { if (kvm_x86_ops.private_mem_enabled) return static_call(kvm_x86_private_mem_enabled)(kvm);
That would be defeating the whole purpose of static calls, AFAICT, as you're testing the pointer. Might as well leave it be a normal function pointer then.
Quoted 16 lines
On a separate topic though, at a high level, this hook is basically a way for platform-specific code to tell generic KVM code that private memslots are supported by overriding the
kvm_arch_has_private_mem()weak reference. In this case the AMD platform is using usingkvm->arch.upm_modeflag to convey that, which is in turn set by theKVM_CAP_UNMAPPED_PRIVATE_MEMORYintroduced in this series.But if, as I suggested in response to your PATCH 2 comments, we drop
KVM_CAP_UNAMMPED_PRIVATE_MEMORYin favor ofKVM_SET_SUPPORTED_MEMORY_ATTRIBUTESioctl to enable "UPM mode" in SEV/SNP code, then we need to rethink things a bit, sinceKVM_SET_MEMORY_ATTRIBUTESin-part relies onkvm_arch_has_private_mem()to determine what flags are supported, whereas SEV/SNP code would be using what was set byKVM_SET_MEMORY_ATTRIBUTESto determine the return value inkvm_arch_has_private_mem().
Quoted 6 lines
So, for AMD, the return value of
kvm_arch_has_private_mem()needs to rely on something else. Maybe the logic can just be:bool svm_private_mem_enabled(struct kvm *kvm) { return sev_enabled(kvm) || sev_snp_enabled(kvm)
I haven't followed the whole discussion in detail but this means that SEV/SNP *means* UPM. I.e., no SEV/SNP without UPM, correct? I guess that's the final thing you guys decided to do ...
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 6 lines · Borislav Petkov
Quoted 2 lines · Michael Roth
Maybe that's not actually enforced, by it seems awkward to try to use a bool return instead. At least for
KVM_X86_OP_OPTIONAL_RET0().I don't see there being a problem/restriction for bool functions, see
Quoted 7 lines
5be2226f417d("KVM: x86: allow defining return-0 static calls")and
__static_call_return0()returns a long which, if you wanna interpret as bool, works too as "false".I still need to disassemble and single-step through a
static_callto see what all that magic does in detail, to be sure.
Hmm, yah, looking at that patch it seems pretty clear (at least at the time) that bool returns are expected to work fine and there are still existing cases that do things that way.
Quoted 11 lines
Quoted 8 lines
However, we could just use
KVM_X86_OP()to declare it so we can cleanly use a function that returns bool, and then we just need to do:bool kvm_arch_has_private_mem(struct kvm *kvm) { if (kvm_x86_ops.private_mem_enabled) return static_call(kvm_x86_private_mem_enabled)(kvm);That would be defeating the whole purpose of static calls, AFAICT, as you're testing the pointer. Might as well leave it be a normal function pointer then.
That's true, we should just use the function pointer for those cases. But given the above maybe KVM_X86_OP_OPTIONAL_RET0() is fine after all so we can continue using static_call().
Quoted 20 lines
Quoted 20 lines
On a separate topic though, at a high level, this hook is basically a way for platform-specific code to tell generic KVM code that private memslots are supported by overriding the
kvm_arch_has_private_mem()weak reference. In this case the AMD platform is using usingkvm->arch.upm_modeflag to convey that, which is in turn set by theKVM_CAP_UNMAPPED_PRIVATE_MEMORYintroduced in this series.But if, as I suggested in response to your PATCH 2 comments, we drop
KVM_CAP_UNAMMPED_PRIVATE_MEMORYin favor ofKVM_SET_SUPPORTED_MEMORY_ATTRIBUTESioctl to enable "UPM mode" in SEV/SNP code, then we need to rethink things a bit, sinceKVM_SET_MEMORY_ATTRIBUTESin-part relies onkvm_arch_has_private_mem()to determine what flags are supported, whereas SEV/SNP code would be using what was set byKVM_SET_MEMORY_ATTRIBUTESto determine the return value inkvm_arch_has_private_mem().So, for AMD, the return value of
kvm_arch_has_private_mem()needs to rely on something else. Maybe the logic can just be:
Quoted 7 lines
Quoted 3 lines
bool svm_private_mem_enabled(struct kvm *kvm) { return sev_enabled(kvm) || sev_snp_enabled(kvm)I haven't followed the whole discussion in detail but this means that SEV/SNP *means* UPM. I.e., no SEV/SNP without UPM, correct? I guess that's the final thing you guys decided to do ...
It's more about reporting whether UPM is *possible*. In the case of this patchset there is support for using UPM in conjunction with SEV or SEV-SNP, and that's all the above check is conveying. This is basically what KVM_SET_SUPPORTED_MEMORY_ATTRIBUTES ioctl will rely on to determine what attributes it reports to the guest as being supported, which is basically what tells userspace whether or not it can make use of UPM features.
In the case of SEV, it would still be up to userspace whether or not it actually wants to make use of UPM functionality like KVM_SET_MEMORY_ATTRIBUTES and private memslots. Otherwise, to maintain backward-compatibility, userspace can do things as it has always done and continue running SEV without relying on private memslots/KVM_SET_MEMORY_ATTRIBUTES or any of the new ioctls.
For SNP however it is required that userspace uses/implements UPM functionality.
-Mike
Quoted 8 lines
Thx.
Signature · 4 lines
Regards/Gruss, Boris. https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&data=05%7C01%7Cmichael.roth%40amd.com%7C4bfcf0c3d1e54046703008daef2e35d4%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085279100077750%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=oCjQx7LV1paTYdMA4JpFjIexf5UkiZEf2pYOWTvpkZ0%3D&reserved=0
Borislav Petkov
maintainerRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 8 lines · Michael Roth
In the case of SEV, it would still be up to userspace whether or not it actually wants to make use of UPM functionality like
KVM_SET_MEMORY_ATTRIBUTESand private memslots. Otherwise, to maintain backward-compatibility, userspace can do things as it has always done and continue running SEV without relying on private memslots/KVM_SET_MEMORY_ATTRIBUTES or any of the new ioctls.For SNP however it is required that userspace uses/implements UPM functionality.
Makes sense to me.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
In reply to Michael RothQuoted 15 lines · Michael Roth
Quoted 5 lines · Borislav Petkov
Quoted 1 line · Michael Roth
+ bool (*private_mem_enabled)(struct kvm *kvm);
This looks like a function returning boolean to me. IOW, you can simplify this to:
The semantics and existing uses of
KVM_X86_OP_OPTIONAL_RET0()gave me the impression it needed to return an integer value, since by default if a platform doesn't implement the op it would "return 0", and so could still be called unconditionally.Maybe that's not actually enforced, by it seems awkward to try to use a bool return instead. At least for
KVM_X86_OP_OPTIONAL_RET0().
Quoted 7 lines
However, we could just use
KVM_X86_OP()to declare it so we can cleanly use a function that returns bool, and then we just need to do:bool kvm_arch_has_private_mem(struct kvm *kvm) { if (kvm_x86_ops.private_mem_enabled) return static_call(kvm_x86_private_mem_enabled)(kvm);
I guess this is missing:
return false;
Quoted 14 lines
}instead of relying on default return value. So I'll take that approach and adopt your other suggested changes.
...
On a separate topic though, at a high level, this hook is basically a way for platform-specific code to tell generic KVM code that private memslots are supported by overriding the
kvm_arch_has_private_mem()weak reference. In this case the AMD platform is using usingkvm->arch.upm_modeflag to convey that, which is in turn set by theKVM_CAP_UNMAPPED_PRIVATE_MEMORYintroduced in this series.
Quoted 8 lines
But if, as I suggested in response to your PATCH 2 comments, we drop
KVM_CAP_UNAMMPED_PRIVATE_MEMORYin favor ofKVM_SET_SUPPORTED_MEMORY_ATTRIBUTESioctl to enable "UPM mode" in SEV/SNP code, then we need to rethink things a bit, sinceKVM_SET_MEMORY_ATTRIBUTESin-part relies onkvm_arch_has_private_mem()to determine what flags are supported, whereas SEV/SNP code would be using what was set byKVM_SET_MEMORY_ATTRIBUTESto determine the return value inkvm_arch_has_private_mem().
Does this mean that internal calls to kvm_vm_set_region_attr() will cease to exist, and it will rely for user space to use the ioctl properly instead?
Quoted 85 lines
So, for AMD, the return value of
kvm_arch_has_private_mem()needs to rely on something else. Maybe the logic can just be:bool svm_private_mem_enabled(struct kvm *kvm) { return sev_enabled(kvm) || sev_snp_enabled(kvm) }(at least in the context of this patchset where UPM support is added for both SEV and SNP).
So I'll plan to make that change as well.
-Mike
Quoted 70 lines
arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -129,6 +129,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +1
-0@@ -1574,6 +1574,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +bool (*private_mem_enabled)(struct kvm *kvm);bool (*has_wbinvd_exit)(void );arch/x86/kvm/svm/svm.c +10
-0@@ -4680,6 +4680,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ; } +static bool svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ); +} +static struct kvm_x86_ops svm_x86_ops__initdata = { .name ="kvm_amd" ,@@ -4760,6 +4768,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_after_set_cpuid = svm_vcpu_after_set_cpuid, + .private_mem_enabled = svm_private_mem_enabled, + .has_wbinvd_exit = svm_has_wbinvd_exit, .get_l2_tsc_offset = svm_get_l2_tsc_offset,arch/x86/kvm/x86.c +5
-0@@ -12556,6 +12556,11 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, }EXPORT_SYMBOL_GPL (__x86_set_memory_region); +bool kvm_arch_has_private_mem (struct kvm *kvm) +{ +return static_call (kvm_x86_private_mem_enabled)(kvm); +} +void kvm_arch_pre_destroy_vm (struct kvm *kvm) {kvm_mmu_pre_destroy_vm (kvm);
Quoted 5 lines
Quoted 5 lines
Signature · 4 lines
Regards/Gruss, Boris. https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&data=05%7C01%7Cmichael.roth%40amd.com%7C319e89ce555a46eace4d08dae506b51a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638074114318137471%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aG11K7va1BhemwlKCKKdcIXEwXGUzImYL%2BZ9%2FQ7XToI%3D&reserved=0
BR, Jarkko
Re: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 51 lines · Jarkko Sakkinen
Quoted 15 lines · Michael Roth
Quoted 5 lines · Borislav Petkov
Quoted 1 line · Michael Roth
+ bool (*private_mem_enabled)(struct kvm *kvm);
This looks like a function returning boolean to me. IOW, you can simplify this to:
The semantics and existing uses of
KVM_X86_OP_OPTIONAL_RET0()gave me the impression it needed to return an integer value, since by default if a platform doesn't implement the op it would "return 0", and so could still be called unconditionally.Maybe that's not actually enforced, by it seems awkward to try to use a bool return instead. At least for
KVM_X86_OP_OPTIONAL_RET0().Quoted 7 lines
However, we could just use
KVM_X86_OP()to declare it so we can cleanly use a function that returns bool, and then we just need to do:bool kvm_arch_has_private_mem(struct kvm *kvm) { if (kvm_x86_ops.private_mem_enabled) return static_call(kvm_x86_private_mem_enabled)(kvm);I guess this is missing:
return false;Quoted 14 lines
}instead of relying on default return value. So I'll take that approach and adopt your other suggested changes.
...
On a separate topic though, at a high level, this hook is basically a way for platform-specific code to tell generic KVM code that private memslots are supported by overriding the
kvm_arch_has_private_mem()weak reference. In this case the AMD platform is using usingkvm->arch.upm_modeflag to convey that, which is in turn set by theKVM_CAP_UNMAPPED_PRIVATE_MEMORYintroduced in this series.Quoted 8 lines
But if, as I suggested in response to your PATCH 2 comments, we drop
KVM_CAP_UNAMMPED_PRIVATE_MEMORYin favor ofKVM_SET_SUPPORTED_MEMORY_ATTRIBUTESioctl to enable "UPM mode" in SEV/SNP code, then we need to rethink things a bit, sinceKVM_SET_MEMORY_ATTRIBUTESin-part relies onkvm_arch_has_private_mem()to determine what flags are supported, whereas SEV/SNP code would be using what was set byKVM_SET_MEMORY_ATTRIBUTESto determine the return value inkvm_arch_has_private_mem().
Quoted 3 lines
Does this mean that internal calls to
kvm_vm_set_region_attr()will cease to exist, and it will rely for user space to use the ioctl properly instead?
Patches 1-3 are no longer needed and have been dropped for v8, instead "UPM mode" is set via KVM_VM_CREATE vm_type arg, and SEV/SNP can simply call kvm_arch_has_private_mem() to query whether userspace has enabled UPM mode or not.
But even still, we call kvm_vm_set_region_attr() in sev_launch_update_data() and snp_launch_update() after copying initial payload into private memory.
I don't think there's much worth in having userspace have to do it via KVM_SET_MEMORY_ATTRIBUTES afterward. It could be done that way I suppose, but generally RMP update from shared->private happens as part of KVM_SET_MEMORY_ATTRIBUTES, whereas in this case it would necessarily happen *after* the RMP updates, since SNP_LAUNCH_UPDATE expects the pages to be marked private beforehand.
Just seems like more corner cases to deal with and more boilerplate code for userspace, which already needed to operate under the assumption that pages will be private after SNP_LAUNCH_UPDATE, so seems to make sense to just have the memory attributes also updated accordingly.
-Mike
Quoted 93 lines
Quoted 86 lines
So, for AMD, the return value of
kvm_arch_has_private_mem()needs to rely on something else. Maybe the logic can just be:bool svm_private_mem_enabled(struct kvm *kvm) { return sev_enabled(kvm) || sev_snp_enabled(kvm) }(at least in the context of this patchset where UPM support is added for both SEV and SNP).
So I'll plan to make that change as well.
-Mike
Quoted 70 lines
arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -129,6 +129,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +1
-0@@ -1574,6 +1574,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +bool (*private_mem_enabled)(struct kvm *kvm);bool (*has_wbinvd_exit)(void );arch/x86/kvm/svm/svm.c +10
-0@@ -4680,6 +4680,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ; } +static bool svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ); +} +static struct kvm_x86_ops svm_x86_ops__initdata = { .name ="kvm_amd" ,@@ -4760,6 +4768,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_after_set_cpuid = svm_vcpu_after_set_cpuid, + .private_mem_enabled = svm_private_mem_enabled, + .has_wbinvd_exit = svm_has_wbinvd_exit, .get_l2_tsc_offset = svm_get_l2_tsc_offset,arch/x86/kvm/x86.c +5
-0@@ -12556,6 +12556,11 @@ void __user * __x86_set_memory_region(struct kvm *kvm, int id, gpa_t gpa, }EXPORT_SYMBOL_GPL (__x86_set_memory_region); +bool kvm_arch_has_private_mem (struct kvm *kvm) +{ +return static_call (kvm_x86_private_mem_enabled)(kvm); +} +void kvm_arch_pre_destroy_vm (struct kvm *kvm) {kvm_mmu_pre_destroy_vm (kvm);Quoted 5 lines
Quoted 5 lines
Signature · 4 lines
Regards/Gruss, Boris. https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&data=05%7C01%7Cmichael.roth%40amd.com%7C319e89ce555a46eace4d08dae506b51a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638074114318137471%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=aG11K7va1BhemwlKCKKdcIXEwXGUzImYL%2BZ9%2FQ7XToI%3D&reserved=0BR, Jarkko
Re: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
In reply to Michael RothQuoted 48 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
KVM should use private memory for guests that have
upm_modeflag set.Add a
kvm_x86_opshook for determining UPM support that accounts for this situation by only enabling UPM test mode in the case of non-SEV guests.Signed-off-by Nikunj A Dadhania <[email protected]>[mdr: add x86 hook for determining restricted/private memory support]Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/svm.c | 10 ++++++++++ arch/x86/kvm/x86.c | 8 ++++++++ 4 files changed, 20 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(private_mem_enabled); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +1
-0@@ -1635,6 +1635,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +int (*private_mem_enabled)(struct kvm *kvm);bool (*has_wbinvd_exit)(void );arch/x86/kvm/svm/svm.c +0
-0@@ -4694,6 +4694,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ;
arch/x86/kvm/svm/svm.c static int svm_vm_init(struct kvm *kvm) · line 4704
Quoted 10 lines
arch/x86/kvm/svm/svm.c +8
-0} +static int svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode ?1 :0 ; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ) ?1 :0 ; +} +
Is this new callback really needed? Shouldn't kvm->arch.upm_mode be sufficient enough to indicate whether the private memory will be used or not?
Probably the CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING is the concern here. But this Kconfig option is not even x86-specific, so shouldn't the handling of it be done in common code too?
For instance, can we explicitly set 'kvm->arch.upm_mode' to 'true' at some point of creating the VM if we see CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING is true?
[snip]
Sean Christopherson
maintainerRe: [PATCH RFC v7 03/64] KVM: SVM: Advertise private memory support to KVM
Quoted 9 lines · Kai
Quoted 8 lines · Michael Roth
arch/x86/kvm/svm/svm.c +0
-0@@ -4694,6 +4694,14 @@ static int svm_vm_init(struct kvm *kvm)return 0 ; }
Quoted 10 lines
arch/x86/kvm/svm/svm.c static int svm_vm_init(struct kvm *kvm) · line 4704
Quoted 8 lines
arch/x86/kvm/svm/svm.c +8
-0+static int svm_private_mem_enabled (struct kvm *kvm) +{ +if (sev_guest (kvm)) +return kvm->arch.upm_mode ?1 :0 ; + +return IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING ) ?1 :0 ; +} +Is this new callback really needed?
Probably not. For anything in this series that gets within spitting distance of CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, my recommendation is to make a mental note but otherwise ignore things like this for now. I suspect it will be much, much more efficient to sort all of this out when I smush UPM+SNP+TDX together in a few weeks.
RFC PATCH v7 4/64 KVM: x86: Add 'fault_is_private' x86 op
In reply to Michael RothThis callback is used by the KVM MMU to check whether a #NPF was or a private GPA or not.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 3 +-- arch/x86/kvm/mmu/mmu_internal.h | 40 +++++++++++++++++++++++++++--- 4 files changed, 39 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -132,6 +132,7 @@ KVM_X86_OP(complete_emulated_msr)KVM_X86_OP (vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled); +KVM_X86_OP_OPTIONAL_RET0 (fault_is_private);#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1636,6 +1636,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level);int (*private_mem_enabled)(struct kvm *kvm); +int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault);bool (*has_wbinvd_exit)(void );
arch/x86/kvm/mmu/mmu.c +1 -2
@@ -5646,8 +5646,7 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err }if (r ==RET_PF_INVALID ) { - r =kvm_mmu_do_page_fault (vcpu, cr2_or_gpa, -lower_32_bits (error_code), false); + r =kvm_mmu_do_page_fault (vcpu, cr2_or_gpa, error_code, false);if (KVM_BUG_ON (r ==RET_PF_INVALID , vcpu->kvm))return -EIO ; }
arch/x86/kvm/mmu/mmu_internal.h +36 -4
@@ -230,6 +230,38 @@ struct kvm_page_fault {int kvm_tdp_page_fault (struct kvm_vcpu *vcpu,struct kvm_page_fault *fault); +static bool kvm_mmu_fault_is_private (struct kvm *kvm,gpa_t gpa,u64 err) +{ +struct kvm_memory_slot *slot; +bool private_fault = false; +gfn_t gfn =gpa_to_gfn (gpa); + + slot =gfn_to_memslot (kvm, gfn); +if (!slot) { +pr_debug ("%s: no slot, GFN: 0x%llx\n" , __func__, gfn); + goto out; + } + +if (!kvm_slot_can_be_private (slot)) { +pr_debug ("%s: slot is not private, GFN: 0x%llx\n" , __func__, gfn); + goto out; + } + +if (static_call (kvm_x86_fault_is_private)(kvm, gpa, err, &private_fault) ==1 ) + goto out; + +/* +* Handling below is for UPM self-tests and guests that use +* slot->shared_bitmap for encrypted access tracking. +*/ +if (IS_ENABLED (CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING )) + private_fault =kvm_mem_is_private (kvm, gpa >>PAGE_SHIFT ); + +out: +pr_debug ("%s: GFN: 0x%llx, private: %d\n" , __func__, gfn, private_fault); +return private_fault; +} +/* * Return values of handle_mmio_page_fault(), mmu.page_fault(), fast_page_fault(), * and of course kvm_mmu_do_page_fault().
@@ -261,13 +293,13 @@ enum { };static inline int kvm_mmu_do_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa, -u32 err,bool prefetch) +u64 err,bool prefetch) {bool is_tdp =likely (vcpu->arch.mmu->page_fault == kvm_tdp_page_fault);struct kvm_page_fault fault = { .addr = cr2_or_gpa, - .error_code = err, + .error_code =lower_32_bits (err), .exec = err &PFERR_FETCH_MASK , .write = err &PFERR_WRITE_MASK , .present = err &PFERR_PRESENT_MASK ,
@@ -281,8 +313,8 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .max_level = KVM_MAX_HUGEPAGE_LEVEL, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, - .is_private = IS_ENABLED(CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING) && is_tdp && - kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), + .is_private = is_tdp && kvm_mmu_fault_is_private(vcpu->kvm, + cr2_or_gpa, err), };int r;
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 04/64] KVM: x86: Add 'fault_is_private' x86 op
Quoted 2 lines · Michael Roth
This callback is used by the KVM MMU to check whether a #NPF was or a private GPA or not.
s/or //
Quoted 26 lines
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 3 +-- arch/x86/kvm/mmu/mmu_internal.h | 40 +++++++++++++++++++++++++++--- 4 files changed, 39 insertions(+), 6 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -132,6 +132,7 @@ KVM_X86_OP(complete_emulated_msr)KVM_X86_OP (vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled); +KVM_X86_OP_OPTIONAL_RET0 (fault_is_private);#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1639
Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1
-0@@ -1636,6 +1636,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level);int (*private_mem_enabled)(struct kvm *kvm); +int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault);
bool
and then you don't need the silly "== 1" at the call site.
Quoted 2 lines
arch/x86/include/asm/kvm_host.h +0
-0bool (*has_wbinvd_exit)(void );
...
arch/x86/include/asm/kvm_host.h enum { · line 296
Quoted 6 lines
arch/x86/include/asm/kvm_host.h +1
-1@@ -261,13 +293,13 @@ enum { };static inline int kvm_mmu_do_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa, -u32 err,bool prefetch) +u64 err,bool prefetch)
The u32 -> u64 change of err could use a sentence or two of clarification in the commit message...
Quoted 24 lines
arch/x86/include/asm/kvm_host.h +3
-3{bool is_tdp =likely (vcpu->arch.mmu->page_fault == kvm_tdp_page_fault);struct kvm_page_fault fault = { .addr = cr2_or_gpa, - .error_code = err, + .error_code =lower_32_bits (err), .exec = err &PFERR_FETCH_MASK , .write = err &PFERR_WRITE_MASK , .present = err &PFERR_PRESENT_MASK ,@@ -281,8 +313,8 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .max_level = KVM_MAX_HUGEPAGE_LEVEL, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, - .is_private = IS_ENABLED(CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING) && is_tdp && - kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), + .is_private = is_tdp && kvm_mmu_fault_is_private(vcpu->kvm, + cr2_or_gpa, err), };int r;Signature · 1 line
2.25.1
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 04/64] KVM: x86: Add 'fault_is_private' x86 op
Quoted 39 lines · Borislav Petkov
Quoted 2 lines · Michael Roth
This callback is used by the KVM MMU to check whether a #NPF was or a private GPA or not.
s/or //
Quoted 26 lines
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 3 +-- arch/x86/kvm/mmu/mmu_internal.h | 40 +++++++++++++++++++++++++++--- 4 files changed, 39 insertions(+), 6 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -132,6 +132,7 @@ KVM_X86_OP(complete_emulated_msr)KVM_X86_OP (vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled); +KVM_X86_OP_OPTIONAL_RET0 (fault_is_private);#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +0
-0arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1639
Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1
-0@@ -1636,6 +1636,7 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level);int (*private_mem_enabled)(struct kvm *kvm); +int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault);bool
Quoted 1 line
and then you don't need the silly "== 1" at the call site.
Obviously I need to add some proper documentation for this, but a 1 return basically means 'private_fault' pass-by-ref arg has been set with the appropriate value, whereas 0 means "there's no platform-specific handling for this, so if you have some generic way to determine this then use that instead".
This is mainly to handle CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, which just parrots whatever kvm_mem_is_private() returns to support running KVM selftests without needed hardware/platform support. If we don't take care to skip this check where the above fault_is_private() hook returns 1, then it ends up breaking SNP in cases where the kernel has been compiled with CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, since SNP relies on the page fault flags to make this determination, not kvm_mem_is_private(), which normally only tracks the memory attributes set by userspace via KVM_SET_MEMORY_ATTRIBUTES ioctl.
Quoted 9 lines
Quoted 3 lines
arch/x86/include/asm/kvm_host.h +0
-0bool (*has_wbinvd_exit)(void );...
Quoted 3 lines
arch/x86/include/asm/kvm_host.h +0
-0@@ -261,13 +293,13 @@ enum { };
Quoted 6 lines
arch/x86/include/asm/kvm_host.h enum { · line 296
Quoted 3 lines
arch/x86/include/asm/kvm_host.h +1
-1static inline int kvm_mmu_do_page_fault (struct kvm_vcpu *vcpu,gpa_t cr2_or_gpa, -u32 err,bool prefetch) +u64 err,bool prefetch)The u32 -> u64 change of err could use a sentence or two of clarification in the commit message...
Will do.
-Mike
Quoted 31 lines
Quoted 25 lines
arch/x86/include/asm/kvm_host.h +3
-3{bool is_tdp =likely (vcpu->arch.mmu->page_fault == kvm_tdp_page_fault);struct kvm_page_fault fault = { .addr = cr2_or_gpa, - .error_code = err, + .error_code =lower_32_bits (err), .exec = err &PFERR_FETCH_MASK , .write = err &PFERR_WRITE_MASK , .present = err &PFERR_PRESENT_MASK ,@@ -281,8 +313,8 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .max_level = KVM_MAX_HUGEPAGE_LEVEL, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, - .is_private = IS_ENABLED(CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING) && is_tdp && - kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), + .is_private = is_tdp && kvm_mmu_fault_is_private(vcpu->kvm, + cr2_or_gpa, err), };int r;Signature · 1 line
2.25.1Signature · 4 lines
Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Borislav Petkov
maintainerRe: [PATCH RFC v7 04/64] KVM: x86: Add 'fault_is_private' x86 op
Quoted 5 lines · Michael Roth
Obviously I need to add some proper documentation for this, but a 1 return basically means '
private_fault' pass-by-ref arg has been set with the appropriate value, whereas 0 means "there's no platform-specific handling for this, so if you have some generic way to determine this then use that instead".
Still binary, tho, and can be bool, right?
I.e., you can just as well do:
if (static_call(kvm_x86_fault_is_private)(kvm, gpa, err, &private_fault))
goto out;
at the call site.
Quoted 9 lines
This is mainly to handle
CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, which just parrots whateverkvm_mem_is_private()returns to support running KVM selftests without needed hardware/platform support. If we don't take care to skip this check where the abovefault_is_private()hook returns 1, then it ends up breaking SNP in cases where the kernel has been compiled withCONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, since SNP relies on the page fault flags to make this determination, notkvm_mem_is_private(), which normally only tracks the memory attributes set by userspace viaKVM_SET_MEMORY_ATTRIBUTESioctl.
Some of that explanation belongs into the commit message, which is a bit lacking...
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteSean Christopherson
maintainerRe: [PATCH RFC v7 04/64] KVM: x86: Add 'fault_is_private' x86 op
Quoted 7 lines · Borislav Petkov
Quoted 5 lines · Michael Roth
Obviously I need to add some proper documentation for this, but a 1 return basically means '
private_fault' pass-by-ref arg has been set with the appropriate value, whereas 0 means "there's no platform-specific handling for this, so if you have some generic way to determine this then use that instead".
Quoted 8 lines
Still binary, tho, and can be bool, right?
I.e., you can just as well do:
if (static_call(kvm_x86_fault_is_private)(kvm, gpa, err,
&private_fault))
goto out;at the call site.
Ya. Don't spend too much time trying to make this look super pretty though, there are subtle bugs inherited from the base UPM series that need to be sorted out and will impact this code. E.g. invoking kvm_mem_is_private() outside of the protection of mmu_invalidate_seq means changes to the attributes may not be reflected in the page tables.
I'm also hoping we can avoid a callback entirely, though that may prove to be more pain than gain. I'm poking at the UPM and testing series right now, will circle back to this and TDX in a few weeks to see if there's a sane way to communicate shared vs. private without having to resort to a callback, and without having races between page faults, KVM_SET_MEMORY_ATTRIBUTES, and KVM_SET_USER_MEMORY_REGION2.
Quoted 10 lines
Quoted 9 lines
This is mainly to handle
CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, which just parrots whateverkvm_mem_is_private()returns to support running KVM selftests without needed hardware/platform support. If we don't take care to skip this check where the abovefault_is_private()hook returns 1, then it ends up breaking SNP in cases where the kernel has been compiled withCONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, since SNP relies on the page fault flags to make this determination, notkvm_mem_is_private(), which normally only tracks the memory attributes set by userspace viaKVM_SET_MEMORY_ATTRIBUTESioctl.
Quoted 2 lines
Some of that explanation belongs into the commit message, which is a bit lacking...
I'll circle back to this too when I give this series (and TDX) a proper look, there's got too be a better way to handle this.
Borislav Petkov
maintainerRe: [PATCH RFC v7 04/64] KVM: x86: Add 'fault_is_private' x86 op
Quoted 3 lines · Sean Christopherson
Ya. Don't spend too much time trying to make this look super pretty though, there are subtle bugs inherited from the base UPM series that need to be sorted out and will impact this code.
Yeah, I'm simply trying to find my way around the code and no better way than reviewing it. But thanks for the heads up.
Quoted 2 lines
I'll circle back to this too when I give this series (and TDX) a proper look, there's got too be a better way to handle this.
Good.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 04/64] KVM: x86: Add 'fault_is_private' x86 op
In reply to Sean ChristophersonQuoted 23 lines · Sean Christopherson
Quoted 7 lines · Borislav Petkov
Quoted 5 lines · Michael Roth
Obviously I need to add some proper documentation for this, but a 1 return basically means '
private_fault' pass-by-ref arg has been set with the appropriate value, whereas 0 means "there's no platform-specific handling for this, so if you have some generic way to determine this then use that instead".Quoted 8 lines
Still binary, tho, and can be bool, right?
I.e., you can just as well do:
if (static_call(kvm_x86_fault_is_private)(kvm, gpa, err,
&private_fault))
goto out;at the call site.
Ya. Don't spend too much time trying to make this look super pretty though, there are subtle bugs inherited from the base UPM series that need to be sorted out and will impact this code. E.g. invoking
kvm_mem_is_private()outside of the protection ofmmu_invalidate_seqmeans changes to the attributes may not be reflected in the page tables.
Quoted 5 lines
I'm also hoping we can avoid a callback entirely, though that may prove to be more pain than gain. I'm poking at the UPM and testing series right now, will circle back to this and TDX in a few weeks to see if there's a sane way to communicate shared vs. private without having to resort to a callback, and without having races between page faults,
KVM_SET_MEMORY_ATTRIBUTES, andKVM_SET_USER_MEMORY_REGION2.
Can circle back on this, but for v8 at least I've kept the callback, but simplified SVM implementation of it so that it's only needed for SNP. For protected-SEV it will fall through to the same generic handling used by UPM self-tests.
It seems like it's safe to have a callback of that sort here for TDX/SNP (or whatever we end up replacing the callback with), since the #NPF flags themselves won't change based on attribute updates, and the subsequent comparison to kvm_mem_is_private() will happen after mmu_invalidate_seq is logged.
But for protected-SEV and UPM selftests the initial kvm_mem_is_private() can become stale vs. the one in __kvm_faultin_pfn(), but it seems like ATM it would only lead to a spurious KVM_EXIT_MEMORY_FAULT, which SEV at least should treat at an implicit page-state change and be able to recover from. But yah, not ideal, and maybe for self-tests that makes it difficult to tell if things are working as expected or not.
Maybe we should just skip setting fault->is_private here in the non-TDX/non-SNP cases, and just have some other indicator so it's initialized/ignored in kvm_mem_is_private() later. I think some iterations of UPM did it this way prior to 'is_private' becoming const.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
This is mainly to handle
CONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, which just parrots whateverkvm_mem_is_private()returns to support running KVM selftests without needed hardware/platform support. If we don't take care to skip this check where the abovefault_is_private()hook returns 1, then it ends up breaking SNP in cases where the kernel has been compiled withCONFIG_HAVE_KVM_PRIVATE_MEM_TESTING, since SNP relies on the page fault flags to make this determination, notkvm_mem_is_private(), which normally only tracks the memory attributes set by userspace viaKVM_SET_MEMORY_ATTRIBUTESioctl.
Quoted 6 lines
Quoted 2 lines
Some of that explanation belongs into the commit message, which is a bit lacking...
I'll circle back to this too when I give this series (and TDX) a proper look, there's got too be a better way to handle this.
It seems like for SNP/TDX we just need to register the shared/encrypted bits with KVM MMU and let it handle checking the #NPF flags, but can iterate on that for the next spin when we have a better idea what it should look like.
-Mike
RFC PATCH v7 5/64 KVM: x86: Add 'update_mem_attr' x86 op
In reply to Michael RothThis callback will handle any platform-specific handling needed for converting pages between shared/private.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu/mmu.c | 10 ++++++++-- 3 files changed, 11 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -133,6 +133,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled);KVM_X86_OP_OPTIONAL_RET0 (fault_is_private); +KVM_X86_OP_OPTIONAL_RET0(update_mem_attr)#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1637,6 +1637,8 @@ struct kvm_x86_ops { int root_level);int (*private_mem_enabled)(struct kvm *kvm);int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault); +int (*update_mem_attr)(struct kvm_memory_slot *slot,unsigned int attr, +gfn_t start,gfn_t end);bool (*has_wbinvd_exit)(void );
arch/x86/kvm/mmu/mmu.c +8 -2
@@ -7130,7 +7130,7 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, {unsigned long pages, mask;gfn_t gfn, gfn_end, first, last; -int level; +int level, ret;bool mixed;/*
@@ -7153,7 +7153,7 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm,linfo_set_mixed (gfn, slot, level, mixed);if (first == last) -return ; + goto out;for (gfn = first + pages; gfn < last; gfn += pages)linfo_set_mixed (gfn, slot, level, false);
@@ -7166,6 +7166,12 @@ static void kvm_update_lpage_private_shared_mixed(struct kvm *kvm, mixed =mem_attrs_mixed (kvm, slot, level, attrs, gfn, gfn_end);linfo_set_mixed (gfn, slot, level, mixed); } + +out: + ret =static_call (kvm_x86_update_mem_attr)(slot, attrs, start, end); +if (ret) +pr_warn_ratelimited ("Failed to update GFN range 0x%llx-0x%llx with attributes 0x%lx. Ret: %d\n" , + start, end, attrs, ret); } void kvm_arch_set_memory_attributes(struct kvm *kvm,
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 05/64] KVM: x86: Add 'update_mem_attr' x86 op
Quoted 1 line · Michael Roth
This callback will handle any platform-specific handling needed for
s/handle/do/
Quoted 26 lines
converting pages between shared/private.
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu/mmu.c | 10 ++++++++-- 3 files changed, 11 insertions(+), 2 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -133,6 +133,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled);KVM_X86_OP_OPTIONAL_RET0 (fault_is_private); +KVM_X86_OP_OPTIONAL_RET0(update_mem_attr)#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1641
Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2
-0@@ -1637,6 +1637,8 @@ struct kvm_x86_ops { int root_level);int (*private_mem_enabled)(struct kvm *kvm);int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault); +int (*update_mem_attr)(struct kvm_memory_slot *slot,unsigned int attr, +gfn_t start,gfn_t end);
Does this one want to return bool too?
I guess I'll see later...
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRFC PATCH v7 6/64 KVM: x86: Add platform hooks for private memory invalidations
In reply to Michael RothIn some cases, like with SEV-SNP, guest memory needs to be updated in a platform-specific manner before it can be safely freed back to the host. Add hooks to wire up handling of this sort to the invalidation notifiers for restricted memory.
Also issue invalidations of all allocated pages during notifier unregistration so that the pages are not left in an unusable state when they eventually get freed back to the host upon FD release.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 5 +++++ include/linux/kvm_host.h | 2 ++ mm/restrictedmem.c | 16 ++++++++++++++++ virt/kvm/kvm_main.c | 5 +++++ 6 files changed, 30 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -134,6 +134,7 @@ KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled);KVM_X86_OP_OPTIONAL_RET0 (fault_is_private); KVM_X86_OP_OPTIONAL_RET0(update_mem_attr) +KVM_X86_OP_OPTIONAL(invalidate_restricted_mem)#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1639,6 +1639,7 @@ struct kvm_x86_ops {int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault);int (*update_mem_attr)(struct kvm_memory_slot *slot,unsigned int attr,gfn_t start,gfn_t end); +void (*invalidate_restricted_mem)(struct kvm_memory_slot *slot,gfn_t start,gfn_t end);bool (*has_wbinvd_exit)(void );
arch/x86/kvm/mmu/mmu.c +5 -0
@@ -7183,3 +7183,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,kvm_update_lpage_private_shared_mixed (kvm, slot, attrs, start, end); } + +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end) +{ +static_call_cond (kvm_x86_invalidate_restricted_mem)(slot, start, end); +}
include/linux/kvm_host.h +2 -0
@@ -2327,6 +2327,7 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,struct kvm_memory_slot *slot, unsigned long attrs, gfn_t start, gfn_t end); +#else static inline void kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot,
@@ -2366,6 +2367,7 @@ static inline int kvm_restricted_mem_get_pfn(struct kvm_memory_slot *slot, }void kvm_arch_memory_mce (struct kvm *kvm); +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end);#endif /* CONFIG_HAVE_KVM_RESTRICTED_MEM */ #endif
mm/restrictedmem.c +16 -0
@@ -54,6 +54,11 @@ static int restrictedmem_release(struct inode *inode, struct file *file) {struct restrictedmem_data *data = inode->i_mapping->private_data; +pr_debug ("%s: releasing memfd, invalidating page offsets 0x0-0x%llx\n" , + __func__, inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_start (data,0 , inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_end (data,0 , inode->i_size >>PAGE_SHIFT ); +fput (data->memfd);kfree (data);return 0 ;
@@ -258,6 +263,17 @@ void restrictedmem_unregister_notifier(struct file *file, struct restrictedmem_notifier *notifier) {struct restrictedmem_data *data = file->f_mapping->private_data; +struct inode *inode =file_inode (data->memfd); + +/* TODO: this will issue notifications to all registered notifiers, +* but it's only the one being unregistered that needs to process +* invalidations for any ranges still allocated at this point in +* time. For now this relies on KVM currently being the only notifier. +*/ +pr_debug ("%s: unregistering notifier, invalidating page offsets 0x0-0x%llx\n" , + __func__, inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_start (data,0 , inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_end (data,0 , inode->i_size >>PAGE_SHIFT );mutex_lock (&data->lock);list_del (¬ifier->list);
virt/kvm/kvm_main.c +5 -0
@@ -974,6 +974,9 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no &gfn_start, &gfn_end))return ; +pr_debug ("%s: start: 0x%lx, end: 0x%lx, roffset: 0x%llx, gfn_start: 0x%llx, gfn_end: 0x%llx\n" , + __func__, start, end, slot->restricted_offset, gfn_start, gfn_end); + gfn_range.start = gfn_start; gfn_range.end = gfn_end; gfn_range.slot = slot;
@@ -988,6 +991,8 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *noif (kvm_unmap_gfn_range (kvm, &gfn_range))kvm_flush_remote_tlbs (kvm); +kvm_arch_invalidate_restricted_mem (slot, gfn_start, gfn_end); +KVM_MMU_UNLOCK (kvm);srcu_read_unlock (&kvm->srcu, idx); }
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 06/64] KVM: x86: Add platform hooks for private memory invalidations
arch/x86/kvm/mmu/mmu.c void kvm_arch_set_memory_attributes(struct kvm *kvm, · line 7189
Quoted 12 lines · Michael Roth
arch/x86/kvm/mmu/mmu.c +4
-0@@ -7183,3 +7183,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,kvm_update_lpage_private_shared_mixed (kvm, slot, attrs, start, end); } + +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end) +{ +static_call_cond (kvm_x86_invalidate_restricted_mem)(slot, start, end);
Why _cond?
arch/x86/kvm/mmu/mmu.c void restrictedmem_unregister_notifier(struct file *file, · line 268
Quoted 7 lines
arch/x86/kvm/mmu/mmu.c +3
-0@@ -258,6 +263,17 @@ void restrictedmem_unregister_notifier(struct file *file, struct restrictedmem_notifier *notifier) {struct restrictedmem_data *data = file->f_mapping->private_data; +struct inode *inode =file_inode (data->memfd); + +/* TODO: this will issue notifications to all registered notifiers,
First of all:
verify_comment_style: WARNING: Multi-line comment needs to start text on the second line:
[+ /* TODO: this will issue notifications to all registered notifiers,]
Then, if you only want to run the callbacks for the one going away only, why don't you simply do:
mutex_lock(&data->lock);
notifier->ops->invalidate_start(notifier, 0, inode->i_size >> PAGE_SHIFT);
notifier->ops->invalidate_end(notifier, 0, inode->i_size >> PAGE_SHIFT);
list_del(¬ifier->list);
mutex_unlock(&data->lock);here?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 06/64] KVM: x86: Add platform hooks for private memory invalidations
Quoted 15 lines · Borislav Petkov
arch/x86/kvm/mmu/mmu.c void kvm_arch_set_memory_attributes(struct kvm *kvm, · line 7189
Quoted 12 lines · Michael Roth
arch/x86/kvm/mmu/mmu.c +4
-0@@ -7183,3 +7183,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,kvm_update_lpage_private_shared_mixed (kvm, slot, attrs, start, end); } + +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end) +{ +static_call_cond (kvm_x86_invalidate_restricted_mem)(slot, start, end);Why
_cond?
Since this hook is declared via KVM_X86_OP_OPTIONAL() (instead of KVM_X86_OP_OPTIONAL_RET0 like the previous hooks), the comment in kvm-x86-ops.h suggests this should be called via static_call_cond():
/*
* KVM_X86_OP() and KVM_X86_OP_OPTIONAL() are used to help generate
* both DECLARE/DEFINE_STATIC_CALL() invocations and
* "static_call_update()" calls.
*
* KVM_X86_OP_OPTIONAL() can be used for those functions that can have
* a NULL definition, for example if "static_call_cond()" will be used
* at the call sites. KVM_X86_OP_OPTIONAL_RET0() can be used likewise
* to make a definition optional, but in this case the default will
* be __static_call_return0.
*/
Quoted 17 lines
arch/x86/kvm/mmu/mmu.c void restrictedmem_unregister_notifier(struct file *file, · line 268
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +3
-0@@ -258,6 +263,17 @@ void restrictedmem_unregister_notifier(struct file *file, struct restrictedmem_notifier *notifier) {struct restrictedmem_data *data = file->f_mapping->private_data; +struct inode *inode =file_inode (data->memfd); + +/* TODO: this will issue notifications to all registered notifiers, First of all:
verify_comment_style: WARNING: Multi-line comment needs to start text on the second line:
[+ /* TODO: this will issue notifications to all registered notifiers,]Then, if you only want to run the callbacks for the one going away only, why don't you simply do:
Quoted 7 lines
mutex_lock(&data->lock); notifier->ops->invalidate_start(notifier, 0, inode->i_size >> PAGE_SHIFT); notifier->ops->invalidate_end(notifier, 0, inode->i_size >> PAGE_SHIFT); list_del(¬ifier->list); mutex_unlock(&data->lock);here?
That should do the trick. Thanks for the suggestion.
-Mike
Quoted 6 lines
Signature · 5 lines
-- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Re: [PATCH RFC v7 06/64] KVM: x86: Add platform hooks for private memory invalidations
In reply to Michael RothQuoted 125 lines · Michael Roth
In some cases, like with SEV-SNP, guest memory needs to be updated in a platform-specific manner before it can be safely freed back to the host. Add hooks to wire up handling of this sort to the invalidation notifiers for restricted memory.
Also issue invalidations of all allocated pages during notifier unregistration so that the pages are not left in an unusable state when they eventually get freed back to the host upon FD release.
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 5 +++++ include/linux/kvm_host.h | 2 ++ mm/restrictedmem.c | 16 ++++++++++++++++ virt/kvm/kvm_main.c | 5 +++++ 6 files changed, 30 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -134,6 +134,7 @@ KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled);KVM_X86_OP_OPTIONAL_RET0 (fault_is_private); KVM_X86_OP_OPTIONAL_RET0(update_mem_attr) +KVM_X86_OP_OPTIONAL(invalidate_restricted_mem)#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +1
-0@@ -1639,6 +1639,7 @@ struct kvm_x86_ops {int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault);int (*update_mem_attr)(struct kvm_memory_slot *slot,unsigned int attr,gfn_t start,gfn_t end); +void (*invalidate_restricted_mem)(struct kvm_memory_slot *slot,gfn_t start,gfn_t end);bool (*has_wbinvd_exit)(void );arch/x86/kvm/mmu/mmu.c +5
-0@@ -7183,3 +7183,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,kvm_update_lpage_private_shared_mixed (kvm, slot, attrs, start, end); } + +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end) +{ +static_call_cond (kvm_x86_invalidate_restricted_mem)(slot, start, end); +}include/linux/kvm_host.h +2
-0@@ -2327,6 +2327,7 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,struct kvm_memory_slot *slot, unsigned long attrs, gfn_t start, gfn_t end); +#else static inline void kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot,@@ -2366,6 +2367,7 @@ static inline int kvm_restricted_mem_get_pfn(struct kvm_memory_slot *slot, }void kvm_arch_memory_mce (struct kvm *kvm); +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end);#endif /* CONFIG_HAVE_KVM_RESTRICTED_MEM */ #endif mm/restrictedmem.c +16
-0@@ -54,6 +54,11 @@ static int restrictedmem_release(struct inode *inode, struct file *file) {struct restrictedmem_data *data = inode->i_mapping->private_data; +pr_debug ("%s: releasing memfd, invalidating page offsets 0x0-0x%llx\n" , + __func__, inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_start (data,0 , inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_end (data,0 , inode->i_size >>PAGE_SHIFT ); +fput (data->memfd);kfree (data);return 0 ;@@ -258,6 +263,17 @@ void restrictedmem_unregister_notifier(struct file *file, struct restrictedmem_notifier *notifier) {struct restrictedmem_data *data = file->f_mapping->private_data; +struct inode *inode =file_inode (data->memfd); + +/* TODO: this will issue notifications to all registered notifiers, +* but it's only the one being unregistered that needs to process +* invalidations for any ranges still allocated at this point in +* time. For now this relies on KVM currently being the only notifier. +*/ +pr_debug ("%s: unregistering notifier, invalidating page offsets 0x0-0x%llx\n" , + __func__, inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_start (data,0 , inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_end (data,0 , inode->i_size >>PAGE_SHIFT );mutex_lock (&data->lock);list_del (¬ifier->list);virt/kvm/kvm_main.c +3
-0@@ -974,6 +974,9 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no &gfn_start, &gfn_end))return ; +pr_debug ("%s: start: 0x%lx, end: 0x%lx, roffset: 0x%llx, gfn_start: 0x%llx, gfn_end: 0x%llx\n" , + __func__, start, end, slot->restricted_offset, gfn_start, gfn_end); + gfn_range.start = gfn_start; gfn_range.end = gfn_end; gfn_range.slot = slot;
virt/kvm/kvm_main.c static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no · line 994
Quoted 4 lines
virt/kvm/kvm_main.c +1
-0@@ -988,6 +991,8 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *noif (kvm_unmap_gfn_range (kvm, &gfn_range))kvm_flush_remote_tlbs (kvm); +kvm_arch_invalidate_restricted_mem (slot, gfn_start, gfn_end);
Calling kvm_arch_invalidate_restricted_mem while the KVM MMU lock is taken causes problems, because taking said lock disables preemption. Within kvm_arch_invalidate_restricted_mem a few calls down, eventually vm_unmap_aliases is called which tries to lock a mutex, which shouldn't happen with preemption disabled. This causes a "scheduling while atomic" bug:
[ 152.846596] BUG: scheduling while atomic: enarx/8302/0x00000002 [ 152.846599] Modules linked in: nf_conntrack_netlink(E) xfrm_user(E) xfrm_algo(E) xt_addrtype(E) br_netfilter(E) xt_CHECKSUM(E) xt_MASQUERADE(E) xt_conntrack(E) ipt_REJECT(E) nf_reject_ipv4(E) xt_tcpudp(E) nft_compat(E) nft_chain_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) nf_tables(E) libcrc32c(E) nfnetlink(E) bridge(E) stp(E) llc(E) bonding(E) intel_rapl_msr(E) intel_rapl_common(E) amd64_edac(E) edac_mce_amd(E) kvm_amd(E) tun(E) ipmi_ssif(E) rfkill(E) overlay(E) ghash_clmulni_intel(E) sha512_ssse3(E) sha512_generic(E) aesni_intel(E) libaes(E) crypto_simd(E) cryptd(E) rapl(E) wmi_bmof(E) binfmt_misc(E) kvm(E) irqbypass(E) nls_ascii(E) nls_cp437(E) vfat(E) fat(E) snd_usb_audio(E) snd_usbmidi_lib(E) snd_hwdep(E) mc(E) snd_pcm(E) snd_seq_midi(E) snd_seq_midi_event(E) snd_rawmidi(E) snd_seq(E) ast(E) snd_seq_device(E) drm_vram_helper(E) drm_ttm_helper(E) snd_timer(E) ttm(E) joydev(E) snd(E) ccp(E) drm_kms_helper(E) soundcore(E) sg(E) i2c_algo_bit(E) rng_core(E) [ 152.846629] k10temp(E) evdev(E) acpi_ipmi(E) ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) acpi_cpufreq(E) button(E) squashfs(E) loop(E) sch_fq_codel(E) msr(E) parport_pc(E) ppdev(E) lp(E) ramoops(E) parport(E) reed_solomon(E) fuse(E) drm(E) efi_pstore(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc32c_generic(E) crc16(E) mbcache(E) jbd2(E) rndis_host(E) cdc_ether(E) usbnet(E) mii(E) hid_generic(E) usbhid(E) hid(E) sd_mod(E) t10_pi(E) crc64_rocksoft(E) crc64(E) crc_t10dif(E) crct10dif_generic(E) crct10dif_pclmul(E) crct10dif_common(E) crc32_pclmul(E) crc32c_intel(E) ahci(E) libahci(E) xhci_pci(E) libata(E) bnxt_en(E) xhci_hcd(E) scsi_mod(E) ptp(E) scsi_common(E) pps_core(E) usbcore(E) i2c_piix4(E) usb_common(E) wmi(E) [ 152.846657] Preemption disabled at: [ 152.846657] [<ffffffffc146a09a>] kvm_restrictedmem_invalidate_begin+0xba/0x1c0 [kvm] [ 152.846688] CPU: 108 PID: 8302 Comm: enarx Tainted: G W E 6.1.0-rc4+ #30 [ 152.846690] Hardware name: Supermicro Super Server/H12SSL-NT, BIOS 2.4 04/14/2022 [ 152.846691] Call Trace: [ 152.846692] <TASK> [ 152.846694] dump_stack_lvl+0x49/0x63 [ 152.846695] ? kvm_restrictedmem_invalidate_begin+0xba/0x1c0 [kvm] [ 152.846723] dump_stack+0x10/0x16 [ 152.846725] __schedule_bug.cold+0x81/0x92 [ 152.846727] __schedule+0x809/0xa00 [ 152.846729] ? asm_sysvec_call_function+0x1b/0x20 [ 152.846731] schedule+0x6b/0xf0 [ 152.846733] schedule_preempt_disabled+0x18/0x30 [ 152.846735] __mutex_lock.constprop.0+0x723/0x750 [ 152.846738] ? smp_call_function_many_cond+0xc1/0x2e0 [ 152.846740] __mutex_lock_slowpath+0x13/0x20 [ 152.846742] mutex_lock+0x49/0x60 [ 152.846744] _vm_unmap_aliases+0x10e/0x160 [ 152.846746] vm_unmap_aliases+0x19/0x20 [ 152.846748] change_page_attr_set_clr+0xb7/0x1c0 [ 152.846751] set_memory_p+0x29/0x30 [ 152.846753] rmpupdate+0xd5/0x110 [ 152.846756] rmp_make_shared+0xb7/0xc0 [ 152.846758] snp_make_page_shared.constprop.0+0x4c/0x90 [kvm_amd] [ 152.846765] sev_invalidate_private_range+0x156/0x330 [kvm_amd] [ 152.846770] ? kvm_unmap_gfn_range+0xef/0x100 [kvm] [ 152.846801] kvm_arch_invalidate_restricted_mem+0xe/0x20 [kvm] [ 152.846829] kvm_restrictedmem_invalidate_begin+0x106/0x1c0 [kvm] [ 152.846856] restrictedmem_unregister_notifier+0x74/0x150 [ 152.846859] kvm_free_memslot+0x6b/0x80 [kvm] [ 152.846885] kvm_free_memslots.part.0+0x47/0x70 [kvm] [ 152.846911] kvm_destroy_vm+0x222/0x320 [kvm] [ 152.846937] kvm_put_kvm+0x2a/0x50 [kvm] [ 152.846964] kvm_vm_release+0x22/0x30 [kvm] [ 152.846990] __fput+0xa8/0x280 [ 152.846992] ____fput+0xe/0x20 [ 152.846994] task_work_run+0x61/0xb0 [ 152.846996] do_exit+0x362/0xb30 [ 152.846998] ? tomoyo_path_number_perm+0x6f/0x200 [ 152.847001] do_group_exit+0x38/0xa0 [ 152.847003] get_signal+0x999/0x9c0 [ 152.847005] arch_do_signal_or_restart+0x37/0x7e0 [ 152.847008] ? __might_fault+0x26/0x30 [ 152.847010] ? __rseq_handle_notify_resume+0xd5/0x4f0 [ 152.847013] exit_to_user_mode_prepare+0xd3/0x170 [ 152.847016] syscall_exit_to_user_mode+0x26/0x50 [ 152.847019] do_syscall_64+0x48/0x90 [ 152.847020] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 152.847022] RIP: 0033:0x7fa345f1aaff [ 152.847023] Code: Unable to access opcode bytes at 0x7fa345f1aad5. [ 152.847024] RSP: 002b:00007fff99d6c050 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 152.847026] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00007fa345f1aaff [ 152.847027] RDX: 00007fff99d6c188 RSI: 00000000c008aeba RDI: 0000000000000006 [ 152.847028] RBP: 00007fff99576000 R08: 0000000000000000 R09: 0000000000000000 [ 152.847029] R10: 0000000001680000 R11: 0000000000000246 R12: 00007fff99d752c0 [ 152.847030] R13: 00007fff99d75270 R14: 0000000000000000 R15: 00007fff99577000 [ 152.847032] </TASK>
This bug can be triggered by destroying multiple SNP VMs at the same time.
Quoted 8 lines
virt/kvm/kvm_main.c +1
-0+KVM_MMU_UNLOCK (kvm);srcu_read_unlock (&kvm->srcu, idx); }Signature · 1 line
2.25.1
Regards, Tom
Re: [PATCH RFC v7 06/64] KVM: x86: Add platform hooks for private memory invalidations
Quoted 199 lines · Tom Dohrmann
Quoted 125 lines · Michael Roth
In some cases, like with SEV-SNP, guest memory needs to be updated in a platform-specific manner before it can be safely freed back to the host. Add hooks to wire up handling of this sort to the invalidation notifiers for restricted memory.
Also issue invalidations of all allocated pages during notifier unregistration so that the pages are not left in an unusable state when they eventually get freed back to the host upon FD release.
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 5 +++++ include/linux/kvm_host.h | 2 ++ mm/restrictedmem.c | 16 ++++++++++++++++ virt/kvm/kvm_main.c | 5 +++++ 6 files changed, 30 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -134,6 +134,7 @@ KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);KVM_X86_OP_OPTIONAL_RET0 (private_mem_enabled);KVM_X86_OP_OPTIONAL_RET0 (fault_is_private); KVM_X86_OP_OPTIONAL_RET0(update_mem_attr) +KVM_X86_OP_OPTIONAL(invalidate_restricted_mem)#undef KVM_X86_OP#undef KVM_X86_OP_OPTIONALarch/x86/include/asm/kvm_host.h +1
-0@@ -1639,6 +1639,7 @@ struct kvm_x86_ops {int (*fault_is_private)(struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault);int (*update_mem_attr)(struct kvm_memory_slot *slot,unsigned int attr,gfn_t start,gfn_t end); +void (*invalidate_restricted_mem)(struct kvm_memory_slot *slot,gfn_t start,gfn_t end);bool (*has_wbinvd_exit)(void );arch/x86/kvm/mmu/mmu.c +5
-0@@ -7183,3 +7183,8 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,kvm_update_lpage_private_shared_mixed (kvm, slot, attrs, start, end); } + +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end) +{ +static_call_cond (kvm_x86_invalidate_restricted_mem)(slot, start, end); +}include/linux/kvm_host.h +2
-0@@ -2327,6 +2327,7 @@ void kvm_arch_set_memory_attributes(struct kvm *kvm,struct kvm_memory_slot *slot, unsigned long attrs, gfn_t start, gfn_t end); +#else static inline void kvm_arch_set_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot,@@ -2366,6 +2367,7 @@ static inline int kvm_restricted_mem_get_pfn(struct kvm_memory_slot *slot, }void kvm_arch_memory_mce (struct kvm *kvm); +void kvm_arch_invalidate_restricted_mem (struct kvm_memory_slot *slot,gfn_t start,gfn_t end);#endif /* CONFIG_HAVE_KVM_RESTRICTED_MEM */ #endif mm/restrictedmem.c +16
-0@@ -54,6 +54,11 @@ static int restrictedmem_release(struct inode *inode, struct file *file) {struct restrictedmem_data *data = inode->i_mapping->private_data; +pr_debug ("%s: releasing memfd, invalidating page offsets 0x0-0x%llx\n" , + __func__, inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_start (data,0 , inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_end (data,0 , inode->i_size >>PAGE_SHIFT ); +fput (data->memfd);kfree (data);return 0 ;@@ -258,6 +263,17 @@ void restrictedmem_unregister_notifier(struct file *file, struct restrictedmem_notifier *notifier) {struct restrictedmem_data *data = file->f_mapping->private_data; +struct inode *inode =file_inode (data->memfd); + +/* TODO: this will issue notifications to all registered notifiers, +* but it's only the one being unregistered that needs to process +* invalidations for any ranges still allocated at this point in +* time. For now this relies on KVM currently being the only notifier. +*/ +pr_debug ("%s: unregistering notifier, invalidating page offsets 0x0-0x%llx\n" , + __func__, inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_start (data,0 , inode->i_size >>PAGE_SHIFT ); +restrictedmem_invalidate_end (data,0 , inode->i_size >>PAGE_SHIFT );mutex_lock (&data->lock);list_del (¬ifier->list);virt/kvm/kvm_main.c +3
-0@@ -974,6 +974,9 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no &gfn_start, &gfn_end))return ; +pr_debug ("%s: start: 0x%lx, end: 0x%lx, roffset: 0x%llx, gfn_start: 0x%llx, gfn_end: 0x%llx\n" , + __func__, start, end, slot->restricted_offset, gfn_start, gfn_end); + gfn_range.start = gfn_start; gfn_range.end = gfn_end; gfn_range.slot = slot;virt/kvm/kvm_main.c static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *no · line 994
Quoted 4 lines
virt/kvm/kvm_main.c +1
-0@@ -988,6 +991,8 @@ static void kvm_restrictedmem_invalidate_begin(struct restrictedmem_notifier *noif (kvm_unmap_gfn_range (kvm, &gfn_range))kvm_flush_remote_tlbs (kvm); +kvm_arch_invalidate_restricted_mem (slot, gfn_start, gfn_end);Calling
kvm_arch_invalidate_restricted_memwhile the KVM MMU lock is taken causes problems, because taking said lock disables preemption. Withinkvm_arch_invalidate_restricted_mema few calls down, eventuallyvm_unmap_aliasesis called which tries to lock a mutex, which shouldn't happen with preemption disabled. This causes a "scheduling while atomic" bug:[ 152.846596] BUG: scheduling while atomic: enarx/8302/0x00000002 [ 152.846599] Modules linked in: nf_conntrack_netlink(E) xfrm_user(E) xfrm_algo(E) xt_addrtype(E) br_netfilter(E) xt_CHECKSUM(E) xt_MASQUERADE(E) xt_conntrack(E) ipt_REJECT(E) nf_reject_ipv4(E) xt_tcpudp(E) nft_compat(E) nft_chain_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) nf_tables(E) libcrc32c(E) nfnetlink(E) bridge(E) stp(E) llc(E) bonding(E) intel_rapl_msr(E) intel_rapl_common(E) amd64_edac(E) edac_mce_amd(E) kvm_amd(E) tun(E) ipmi_ssif(E) rfkill(E) overlay(E) ghash_clmulni_intel(E) sha512_ssse3(E) sha512_generic(E) aesni_intel(E) libaes(E) crypto_simd(E) cryptd(E) rapl(E) wmi_bmof(E) binfmt_misc(E) kvm(E) irqbypass(E) nls_ascii(E) nls_cp437(E) vfat(E) fat(E) snd_usb_audio(E) snd_usbmidi_lib(E) snd_hwdep(E) mc(E) snd_pcm(E) snd_seq_midi(E) snd_seq_midi_event(E) snd_rawmidi(E) snd_seq(E) ast(E) snd_seq_device(E) drm_vram_helper(E) drm_ttm_helper(E) snd_timer(E) ttm(E) joydev(E) snd(E) ccp(E) drm_kms_helper(E) soundcore(E) sg(E) i2c_algo_bit(E) rng_core(E) [ 152.846629] k10temp(E) evdev(E) acpi_ipmi(E) ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) acpi_cpufreq(E) button(E) squashfs(E) loop(E) sch_fq_codel(E) msr(E) parport_pc(E) ppdev(E) lp(E) ramoops(E) parport(E) reed_solomon(E) fuse(E) drm(E) efi_pstore(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc32c_generic(E) crc16(E) mbcache(E) jbd2(E) rndis_host(E) cdc_ether(E) usbnet(E) mii(E) hid_generic(E) usbhid(E) hid(E) sd_mod(E) t10_pi(E) crc64_rocksoft(E) crc64(E) crc_t10dif(E) crct10dif_generic(E) crct10dif_pclmul(E) crct10dif_common(E) crc32_pclmul(E) crc32c_intel(E) ahci(E) libahci(E) xhci_pci(E) libata(E) bnxt_en(E) xhci_hcd(E) scsi_mod(E) ptp(E) scsi_common(E) pps_core(E) usbcore(E) i2c_piix4(E) usb_common(E) wmi(E) [ 152.846657] Preemption disabled at: [ 152.846657] [<ffffffffc146a09a>] kvm_restrictedmem_invalidate_begin+0xba/0x1c0 [kvm] [ 152.846688] CPU: 108 PID: 8302 Comm: enarx Tainted: G W E 6.1.0-rc4+ #30 [ 152.846690] Hardware name: Supermicro Super Server/H12SSL-NT, BIOS 2.4 04/14/2022 [ 152.846691] Call Trace: [ 152.846692] <TASK> [ 152.846694] dump_stack_lvl+0x49/0x63 [ 152.846695] ? kvm_restrictedmem_invalidate_begin+0xba/0x1c0 [kvm] [ 152.846723] dump_stack+0x10/0x16 [ 152.846725] __schedule_bug.cold+0x81/0x92 [ 152.846727] __schedule+0x809/0xa00 [ 152.846729] ? asm_sysvec_call_function+0x1b/0x20 [ 152.846731] schedule+0x6b/0xf0 [ 152.846733] schedule_preempt_disabled+0x18/0x30 [ 152.846735] __mutex_lock.constprop.0+0x723/0x750 [ 152.846738] ? smp_call_function_many_cond+0xc1/0x2e0 [ 152.846740] __mutex_lock_slowpath+0x13/0x20 [ 152.846742] mutex_lock+0x49/0x60 [ 152.846744] _vm_unmap_aliases+0x10e/0x160 [ 152.846746] vm_unmap_aliases+0x19/0x20 [ 152.846748] change_page_attr_set_clr+0xb7/0x1c0 [ 152.846751] set_memory_p+0x29/0x30 [ 152.846753] rmpupdate+0xd5/0x110 [ 152.846756] rmp_make_shared+0xb7/0xc0 [ 152.846758] snp_make_page_shared.constprop.0+0x4c/0x90 [kvm_amd] [ 152.846765] sev_invalidate_private_range+0x156/0x330 [kvm_amd] [ 152.846770] ? kvm_unmap_gfn_range+0xef/0x100 [kvm] [ 152.846801] kvm_arch_invalidate_restricted_mem+0xe/0x20 [kvm] [ 152.846829] kvm_restrictedmem_invalidate_begin+0x106/0x1c0 [kvm] [ 152.846856] restrictedmem_unregister_notifier+0x74/0x150 [ 152.846859] kvm_free_memslot+0x6b/0x80 [kvm] [ 152.846885] kvm_free_memslots.part.0+0x47/0x70 [kvm] [ 152.846911] kvm_destroy_vm+0x222/0x320 [kvm] [ 152.846937] kvm_put_kvm+0x2a/0x50 [kvm] [ 152.846964] kvm_vm_release+0x22/0x30 [kvm] [ 152.846990] __fput+0xa8/0x280 [ 152.846992] ____fput+0xe/0x20 [ 152.846994] task_work_run+0x61/0xb0 [ 152.846996] do_exit+0x362/0xb30 [ 152.846998] ? tomoyo_path_number_perm+0x6f/0x200 [ 152.847001] do_group_exit+0x38/0xa0 [ 152.847003] get_signal+0x999/0x9c0 [ 152.847005] arch_do_signal_or_restart+0x37/0x7e0 [ 152.847008] ? __might_fault+0x26/0x30 [ 152.847010] ? __rseq_handle_notify_resume+0xd5/0x4f0 [ 152.847013] exit_to_user_mode_prepare+0xd3/0x170 [ 152.847016] syscall_exit_to_user_mode+0x26/0x50 [ 152.847019] do_syscall_64+0x48/0x90 [ 152.847020] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 152.847022] RIP: 0033:0x7fa345f1aaff [ 152.847023] Code: Unable to access opcode bytes at 0x7fa345f1aad5. [ 152.847024] RSP: 002b:00007fff99d6c050 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 [ 152.847026] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00007fa345f1aaff [ 152.847027] RDX: 00007fff99d6c188 RSI: 00000000c008aeba RDI: 0000000000000006 [ 152.847028] RBP: 00007fff99576000 R08: 0000000000000000 R09: 0000000000000000 [ 152.847029] R10: 0000000001680000 R11: 0000000000000246 R12: 00007fff99d752c0 [ 152.847030] R13: 00007fff99d75270 R14: 0000000000000000 R15: 00007fff99577000 [ 152.847032] </TASK>
Quoted 1 line
This bug can be triggered by destroying multiple SNP VMs at the same time.
I can also reproduce this one.
If I do "cargo test -- --test-threads=1", then this does not happen.
Even then I get this:
[ 232.054359] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 232.061466] ------------[ cut here ]------------ [ 232.061467] WARNING: CPU: 18 PID: 2436 at arch/x86/kvm/../../../virt/kvm/kvm_main.c:3665 mark_page_dirty_in_slot+0x99/0xd0 [ 232.061472] Modules linked in: af_packet irdma intel_rapl_msr i40e ib_uverbs ib_core dell_smbios wmi_bmof dell_wmi_descriptor evdev mac_hid dcdbas amd64_edac edac_mce_amd edac_core intel_rapl_common crc32_pclmul polyval_clmulni polyval_generic gf128mul ghash_clmulni_intel sha512_ssse3 sha512_generic aesni_intel libaes crypto_simd cryptd rapl deflate efi_pstore bonding tls cfg80211 rfkill ip6_tables xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp ip6t_rpfilter ipt_rpfilter xt_pkttype nft_compat nf_tables libcrc32c sch_fq_codel nfnetlink atkbd libps2 serio vivaldi_fmap loop tun tap macvlan bridge stp llc ipmi_ssif ipmi_watchdog dm_round_robin dm_multipath kvm_amd mgag200 drm_shmem_helper drm_kms_helper i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt ice ptp pps_core sp5100_tco watchdog k10temp i2c_piix4 ptdma virt_dma hed tpm_crb wmi acpi_ipmi ipmi_si ipmi_devintf ipmi_msghandler tpm_tis tpm_tis_core tpm acpi_power_meter nls_iso8859_1 nls_cp437 vfat fat [ 232.061525] tiny_power_button button ccp rng_core drm pstore fuse backlight i2c_core configfs efivarfs ip_tables x_tables autofs4 ext4 crc32c_generic crc16 mbcache jbd2 sd_mod ahci libahci xhci_pci xhci_pci_renesas libata xhci_hcd nvme nvme_core scsi_mod usbcore t10_pi crc32c_intel crc64_rocksoft crc64 crc_t10dif scsi_common crct10dif_generic crct10dif_pclmul crct10dif_common usb_common rtc_cmos dm_mod dax [ 232.061547] CPU: 18 PID: 2436 Comm: enarx Not tainted 6.1.0-rc4 #1-NixOS [ 232.061549] Hardware name: Dell Inc. PowerEdge R6515/068NXX, BIOS 2.6.6 01/13/2022 [ 232.061550] RIP: 0010:mark_page_dirty_in_slot+0x99/0xd0 [ 232.061552] Code: 83 04 01 00 00 4c 2b a3 b0 00 00 00 85 d2 74 27 c1 e6 10 5b 49 8d bd f8 19 00 00 5d 4c 89 e2 09 c6 41 5c 41 5d e9 07 aa 00 00 <0f> 0b 5b 5d 41 5c 41 5d c3 cc cc cc cc 48 8b 83 c0 00 00 00 49 63 [ 232.061553] RSP: 0018:ffffc169265978f0 EFLAGS: 00010246 [ 232.061555] RAX: 0000000080000000 RBX: ffffa0c045c49600 RCX: 0000000000000000 [ 232.061556] RDX: 0000000000000001 RSI: ffffffff92d076bd RDI: 00000000ffffffff [ 232.061557] RBP: ffffc1692177d000 R08: 0000000000000001 R09: 0000000000001000 [ 232.061557] R10: 0000000000000001 R11: 0000000000000001 R12: 00000000000ffe02 [ 232.061558] R13: 0000000000000000 R14: ffffc1692177d000 R15: 00007f6cacf89000 [ 232.061559] FS: 00007f6cacfc3540(0000) GS:ffffa0cfffa80000(0000) knlGS:0000000000000000 [ 232.061560] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 232.061561] CR2: 00007f6cacf89020 CR3: 000000208c9c6005 CR4: 0000000000770ee0 [ 232.061561] PKRU: 55555554 [ 232.061562] Call Trace: [ 232.061563] <TASK> [ 232.061566] __kvm_write_guest_page+0xac/0xf0 [ 232.061569] snp_launch_update_gfn_handler.cold+0x5e/0xfe [kvm_amd] [ 232.061578] kvm_vm_do_hva_range_op+0x142/0x1c0 [ 232.061579] ? sev_launch_update_gfn_handler+0x470/0x470 [kvm_amd] [ 232.061585] sev_mem_enc_ioctl+0x4dd/0x1270 [kvm_amd] [ 232.061591] ? sev_pin_memory+0x159/0x1a0 [kvm_amd] [ 232.061595] ? sev_mem_enc_register_region+0xe3/0x130 [kvm_amd] [ 232.061602] kvm_arch_vm_ioctl+0x6a6/0xc20 [ 232.061604] ? __blk_flush_plug+0x102/0x160 [ 232.061606] ? _raw_spin_unlock_irqrestore+0x23/0x40 [ 232.061609] ? get_page_from_freelist+0x1451/0x15a0 [ 232.061612] ? __mod_node_page_state+0x7c/0xb0 [ 232.061615] ? try_charge_memcg+0x466/0x800 [ 232.061618] ? __mod_node_page_state+0x7c/0xb0 [ 232.061619] ? __mod_memcg_lruvec_state+0x6e/0xd0 [ 232.061620] kvm_vm_ioctl+0x7ba/0x1290 [ 232.061622] ? folio_add_lru+0x6e/0xa0 [ 232.061624] ? _raw_spin_unlock+0x15/0x30 [ 232.061626] ? __handle_mm_fault+0xace/0xc70 [ 232.061629] ? handle_mm_fault+0xb2/0x2a0 [ 232.061630] __x64_sys_ioctl+0x8a/0xc0 [ 232.061634] do_syscall_64+0x3b/0x90 [ 232.061636] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 232.061638] RIP: 0033:0x7f6cad0c1e37 [ 232.061659] Code: ff ff 48 89 d8 5b 5d 41 5c c3 66 0f 1f 84 00 00 00 00 00 48 89 e8 48 f7 d8 48 39 c3 0f 92 c0 eb c9 66 90 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d b1 0f 0f 00 f7 d8 64 89 01 48 [ 232.061660] RSP: 002b:00007ffee9081668 EFLAGS: 00000216 ORIG_RAX: 0000000000000010 [ 232.061661] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f6cad0c1e37 [ 232.061662] RDX: 00007ffee9081748 RSI: 00000000c008aeba RDI: 0000000000000004 [ 232.061663] RBP: 00007ffee888e000 R08: 0000000000000006 R09: 0000000000000000 [ 232.061663] R10: 0000000000001000 R11: 0000000000000216 R12: 00007ffee908a260 [ 232.061664] R13: 00007ffee908a210 R14: 0000000000000000 R15: 00007ffee888f000 [ 232.061665] </TASK> [ 232.061666] ---[ end trace 0000000000000000 ]--- [ 232.759176] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 233.449477] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 235.284941] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 237.129469] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 237.978020] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 239.034002] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 239.598148] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 240.152122] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 240.718185] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 241.271235] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 241.836286] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 242.390345] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 242.962499] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 243.501430] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16 [ 244.093547] SEV-SNP launch update failed, ret: 0xfffffffb, fw_error: 0x16
There is no cumulative klog output, i.e. only first run of cargo test emits this. Also software runs without issues, including attestation and everything.
BR, Jarkko
RFC PATCH v7 7/64 KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
In reply to Michael RothFrom: Nikunj A Dadhania <[email protected]>
KVM_HC_MAP_GPA_RANGE hypercall is used by the SEV guest to notify a change in the page encryption status to the hypervisor.
The hypercall exits to userspace with KVM_EXIT_HYPERCALL exit code, currently this is used for explicit memory conversion between shared/private for memfd based private memory.
--- arch/x86/kvm/x86.c | 8 ++++++++ virt/kvm/kvm_main.c | 1 + 2 files changed, 9 insertions(+)
arch/x86/kvm/x86.c +8 -0
@@ -9649,6 +9649,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)break ;case KVM_HC_MAP_GPA_RANGE : {u64 gpa = a0, npages = a1, attrs = a2; +struct kvm_memory_slot *slot; ret = -KVM_ENOSYS ;if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE )))
@@ -9660,6 +9661,13 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)break ; } + slot =kvm_vcpu_gfn_to_memslot (vcpu,gpa_to_gfn (gpa)); +if (!vcpu->kvm->arch.upm_mode || + !kvm_slot_can_be_private (slot)) { + ret =0 ; +break ; + } + vcpu->run->exit_reason =KVM_EXIT_HYPERCALL ; vcpu->run->hypercall.nr =KVM_HC_MAP_GPA_RANGE ; vcpu->run->hypercall.args[0 ] = gpa;
virt/kvm/kvm_main.c +1 -0
@@ -2646,6 +2646,7 @@ struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfnreturn NULL ; } +EXPORT_SYMBOL_GPL (kvm_vcpu_gfn_to_memslot);bool kvm_is_visible_gfn (struct kvm *kvm,gfn_t gfn) {
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 07/64] KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
Quoted 8 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
KVM_HC_MAP_GPA_RANGEhypercall is used by the SEV guest to notify a change in the page encryption status to the hypervisor.The hypercall exits to userspace with
KVM_EXIT_HYPERCALLexit code, currently this is used for explicit memory conversion between shared/private for memfd based private memory.
So Tom and I spent a while to figure out what this is doing...
Please explain in more detail what that is. Like the hypercall gets ignored for memslots which cannot be private...?
And what's the story with supporting UPM with SEV{,-ES} guests?
In general, this text needs more background and why this is being done.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteSean Christopherson
maintainerRe: [PATCH RFC v7 07/64] KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
Quoted 6 lines · Borislav Petkov
Quoted 5 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
KVM_HC_MAP_GPA_RANGEhypercall is used by the SEV guest to notify a change in the page encryption status to the hypervisor.
Quoted 8 lines
Quoted 3 lines
The hypercall exits to userspace with
KVM_EXIT_HYPERCALLexit code, currently this is used for explicit memory conversion between shared/private for memfd based private memory.So Tom and I spent a while to figure out what this is doing...
Please explain in more detail what that is. Like the hypercall gets ignored for memslots which cannot be private...?
Don't bother, just drop the patch. It's perfectly legal for userspace to create the private memslot in response to a guest request.
Re: [PATCH RFC v7 07/64] KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
Quoted 7 lines · Sean Christopherson
Quoted 6 lines · Borislav Petkov
Quoted 5 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
KVM_HC_MAP_GPA_RANGEhypercall is used by the SEV guest to notify a change in the page encryption status to the hypervisor.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
The hypercall exits to userspace with
KVM_EXIT_HYPERCALLexit code, currently this is used for explicit memory conversion between shared/private for memfd based private memory.So Tom and I spent a while to figure out what this is doing...
Please explain in more detail what that is. Like the hypercall gets ignored for memslots which cannot be private...?
This was required when we were using per memslot bitmap for storing the private information, mem_attr_array is not dependent on memslot anymore.
Quoted 2 lines
Don't bother, just drop the patch.
Agree, we can drop this. I have tested SEV without this patch.
Quoted 1 line
It's perfectly legal for userspace to create the private memslot in response > to a guest request.
Sean, did not understand this part, how could a memslot be created on a guest request?
Regards
Nikunj
Sean Christopherson
maintainerRe: [PATCH RFC v7 07/64] KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
Quoted 5 lines · Nikunj A. Dadhania
Quoted 2 lines · Sean Christopherson
It's perfectly legal for userspace to create the private memslot in response to a guest request.
Sean, did not understand this part, how could a memslot be created on a guest request?
KVM_HC_MAP_GPA_RANGE gets routed to host userspace, at that point userspace can take any action it wants to satisfy the guest request. E.g. a userspace+guest setup could define memory as shared by default, and only create KVM_MEM_PRIVATE memslots for memory that the guest explicitly requests to be mapped private.
I don't anticipate any real world use cases actually doing something like that, but I also don't see any value in going out of our way to disallow it. Normally I like to be conservative when it comes to KVM's uAPI, e.g. allow the minimum needed to support known use cases, but restricting KVM_HC_MAP_GPA_RANGE doesn't actually achieve anything and just makes things more complex for KVM. E.g. the behavior is non-deterministic from KVM's perspective if a userspace memslots update is in-progress.
Re: [PATCH RFC v7 07/64] KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
In reply to Michael Rotharch/x86/kvm/x86.c int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
Quoted 21 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
KVM_HC_MAP_GPA_RANGEhypercall is used by the SEV guest to notify a change in the page encryption status to the hypervisor.The hypercall exits to userspace with
KVM_EXIT_HYPERCALLexit code, currently this is used for explicit memory conversion between shared/private for memfd based private memory.Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/x86.c | 8 ++++++++ virt/kvm/kvm_main.c | 1 + 2 files changed, 9 insertions(+)arch/x86/kvm/x86.c +0
-0@@ -9649,6 +9649,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
Couldn't find a better commit to comment on:
when the guest has the ptp-kvm module, it will issue a KVM_HC_CLOCK_PAIRING hypercall. This will pass sev_es_validate_vmgexit validation and end up in this function where kvm_pv_clock_pairing() is called, and that calls kvm_write_guest(). This results in a CPU soft-lockup, at least in my testing.
Are there any emulated hypercalls that make sense for snp guests? We should block at least the ones that definitely don't work.
Jeremi
Quoted 36 lines
arch/x86/kvm/x86.c +8
-0break ;case KVM_HC_MAP_GPA_RANGE : {u64 gpa = a0, npages = a1, attrs = a2; +struct kvm_memory_slot *slot; ret = -KVM_ENOSYS ;if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 <<KVM_HC_MAP_GPA_RANGE )))@@ -9660,6 +9661,13 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)break ; } + slot =kvm_vcpu_gfn_to_memslot (vcpu,gpa_to_gfn (gpa)); +if (!vcpu->kvm->arch.upm_mode || + !kvm_slot_can_be_private (slot)) { + ret =0 ; +break ; + } + vcpu->run->exit_reason =KVM_EXIT_HYPERCALL ; vcpu->run->hypercall.nr =KVM_HC_MAP_GPA_RANGE ; vcpu->run->hypercall.args[0 ] = gpa;virt/kvm/kvm_main.c +1
-0@@ -2646,6 +2646,7 @@ struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfnreturn NULL ; } +EXPORT_SYMBOL_GPL (kvm_vcpu_gfn_to_memslot);bool kvm_is_visible_gfn (struct kvm *kvm,gfn_t gfn) {Signature · 1 line
2.25.1
Re: [PATCH RFC v7 07/64] KVM: SEV: Handle KVM_HC_MAP_GPA_RANGE hypercall
Quoted 29 lines · Jeremi Piotrowski
arch/x86/kvm/x86.c int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
Quoted 21 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
KVM_HC_MAP_GPA_RANGEhypercall is used by the SEV guest to notify a change in the page encryption status to the hypervisor.The hypercall exits to userspace with
KVM_EXIT_HYPERCALLexit code, currently this is used for explicit memory conversion between shared/private for memfd based private memory.Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/x86.c | 8 ++++++++ virt/kvm/kvm_main.c | 1 + 2 files changed, 9 insertions(+)arch/x86/kvm/x86.c +0
-0@@ -9649,6 +9649,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)Couldn't find a better commit to comment on:
when the guest has the ptp-kvm module, it will issue aKVM_HC_CLOCK_PAIRINGhypercall. This will passsev_es_validate_vmgexitvalidation and end up in this function wherekvm_pv_clock_pairing()is called, and that callskvm_write_guest(). This results in a CPU soft-lockup, at least in my testing.
Quoted 4 lines
Are there any emulated hypercalls that make sense for snp guests? We should block at least the ones that definitely don't work.
Jeremi
So turns out the soft-lockup is a nested issue (details here for those
interested: [^1]), but the questions still stands, of whether we should block kvm_write_page (and similar) explicitly or rely on the rmp fault.
[^1]: https://github.com/jepio/linux/commit/6c3bdf552e93664ae172660e24ceceed60fd4df5
RFC PATCH v7 8/64 KVM: Move kvm_for_each_memslot_in_hva_range() to be used in SVM
In reply to Michael RothFrom: Nikunj A Dadhania <[email protected]>
Move the macro to kvm_host.h and make if visible for SVM to use.
No functional change intended.
--- include/linux/kvm_host.h | 6 ++++++ virt/kvm/kvm_main.c | 6 ------ 2 files changed, 6 insertions(+), 6 deletions(-)
include/linux/kvm_host.h +6 -0
@@ -1116,6 +1116,12 @@ static inline bool kvm_memslot_iter_is_valid(struct kvm_memslot_iter *iter, gfn_kvm_memslot_iter_is_valid (iter, end); \ kvm_memslot_iter_next(iter)) +/* Iterate over each memslot intersecting [start, last] (inclusive) range */ +#define kvm_for_each_memslot_in_hva_range (node, slots, start, last) \ + for (node = interval_tree_iter_first(&slots->hva_tree, start, last); \ + node; \ + node = interval_tree_iter_next(node, start, last)) +/* * KVM_SET_USER_MEMORY_REGION ioctl allows the following operations: * - create a new memory slot
virt/kvm/kvm_main.c +0 -6
@@ -623,12 +623,6 @@ static void kvm_null_fn(void) }#define IS_KVM_NULL_FN (fn) ((fn) == (void *)kvm_null_fn) -/* Iterate over each memslot intersecting [start, last] (inclusive) range */ -#define kvm_for_each_memslot_in_hva_range (node, slots, start, last) \ - for (node = interval_tree_iter_first(&slots->hva_tree, start, last); \ - node; \ - node = interval_tree_iter_next(node, start, last)) \ - static __always_inline int __kvm_handle_hva_range(struct kvm *kvm,const struct kvm_hva_range *range) {
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 08/64] KVM: Move kvm_for_each_memslot_in_hva_range() to be used in SVM
Quoted 3 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Move the macro to
kvm_host.hand make if visible for SVM to use.
s/if/it/
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRFC PATCH v7 9/64 KVM: Add HVA range operator
In reply to Michael RothFrom: Vishal Annapurve <[email protected]>
Introduce HVA range operator so that other KVM subsystems can operate on HVA range.
--- include/linux/kvm_host.h | 6 +++++ virt/kvm/kvm_main.c | 48 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+)
include/linux/kvm_host.h +6 -0
@@ -1407,6 +1407,12 @@ void kvm_mmu_invalidate_begin(struct kvm *kvm);void kvm_mmu_invalidate_range_add (struct kvm *kvm,gfn_t start,gfn_t end);void kvm_mmu_invalidate_end (struct kvm *kvm); +typedef int (*kvm_hva_range_op_t )(struct kvm *kvm, +struct kvm_gfn_range *range,void *data); + +int kvm_vm_do_hva_range_op (struct kvm *kvm,unsigned long hva_start, +unsigned long hva_end,kvm_hva_range_op_t handler,void *data); +long kvm_arch_dev_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg); long kvm_arch_vcpu_ioctl(struct file *filp,
virt/kvm/kvm_main.c +48 -0
54 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v7 10/64 KVM: SEV: Populate private memory fd during LAUNCH_UPDATE_DATA
In reply to Michael RothFrom: Vishal Annapurve <[email protected]>
This change adds handling of HVA ranges to copy contents to private memory while doing sev launch update data.
mem_attr array is updated during LAUNCH_UPDATE_DATA to ensure that encrypted memory is marked as private.
[mdr: use gfn_to_hva_memslot_prot() for shared GFN handler to deal with
read-only slots for ROMs]
--- arch/x86/kvm/svm/sev.c | 99 ++++++++++++++++++++++++++++++++++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 27 ++++++++--- 3 files changed, 111 insertions(+), 16 deletions(-)
arch/x86/kvm/svm/sev.c +90 -9
@@ -493,23 +493,26 @@ static unsigned long get_num_contig_pages(unsigned long idx,return pages; } -static int sev_launch_update_data (struct kvm *kvm,struct kvm_sev_cmd *argp) +static int sev_launch_update_shared_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) {unsigned long vaddr, vaddr_end, next_vaddr, npages, pages, size, i;struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; -struct kvm_sev_launch_update_data params;struct sev_data_launch_update_data data;struct page **inpages;int ret; -if (!sev_guest (kvm)) -return -ENOTTY ; - -if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) -return -EFAULT ; + vaddr =gfn_to_hva_memslot_prot (range->slot, range->start,NULL ); +pr_debug ("%s: shared GFN: %llx, slot.id: %d, slot.base_gfn: %llx, slot.userspace_addr: %lx, slot.flags: %x, vaddr: %lx\n" , + __func__, range->start, range->slot->id, range->slot->base_gfn, + range->slot->userspace_addr, range->slot->flags, vaddr); +if (kvm_is_error_hva (vaddr)) { +pr_err ("vaddr is erroneous 0x%lx\n" , vaddr); +return -EINVAL ; + } - vaddr = params.uaddr; - size = params.len; + size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size;/* Lock the user memory. */
@@ -561,6 +564,84 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_launch_update_priv_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) +{ +struct sev_data_launch_update_data data; +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; +gfn_t gfn; +kvm_pfn_t pfn; +struct kvm_memory_slot *memslot = range->slot; +int ret =0 ; + + data.reserved =0 ; + data.handle = sev->handle; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; +void *kvaddr; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + + kvaddr =pfn_to_kaddr (pfn); +if (!virt_addr_valid (kvaddr)) { +pr_err ("Invalid kvaddr 0x%llx\n" , (uint64_t )kvaddr); + ret = -EINVAL ; + goto e_ret; + } + + ret =kvm_read_guest_page (kvm, gfn, kvaddr,0 ,PAGE_SIZE ); +if (ret) { +pr_err ("guest read failed 0x%x\n" , ret); + goto e_ret; + } + +if (!this_cpu_has (X86_FEATURE_SME_COHERENT )) +clflush_cache_range (kvaddr,PAGE_SIZE ); + + data.len =PAGE_SIZE ; + data.address =__sme_set (pfn <<PAGE_SHIFT ); + ret =sev_issue_cmd (kvm,SEV_CMD_LAUNCH_UPDATE_DATA , &data, &argp->error); +if (ret) + goto e_ret; +kvm_release_pfn_clean (pfn); + } +kvm_vm_set_region_attr (kvm, range->start, range->end, + true/* priv_attr */ ); + +e_ret: +return ret; +} + +static int sev_launch_update_gfn_handler (struct kvm *kvm,struct kvm_gfn_range *range, +void *data) +{ +struct kvm_sev_cmd *argp = (struct kvm_sev_cmd *)data; + +if (kvm_slot_can_be_private (range->slot)) +return sev_launch_update_priv_gfn_handler (kvm, range, argp); + +return sev_launch_update_shared_gfn_handler (kvm, range, argp); +} + +static int sev_launch_update_data (struct kvm *kvm, +struct kvm_sev_cmd *argp) +{ +struct kvm_sev_launch_update_data params; + +if (!sev_guest (kvm)) +return -ENOTTY ; + +if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) +return -EFAULT ; + +return kvm_vm_do_hva_range_op (kvm, params.uaddr, params.uaddr + params.len, + sev_launch_update_gfn_handler, argp); +} +static int sev_es_sync_vmsa (struct vcpu_svm *svm) {struct sev_es_save_area *save = svm->sev_es.vmsa;
include/linux/kvm_host.h +1 -0
@@ -973,6 +973,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,void kvm_exit (void );void kvm_get_kvm (struct kvm *kvm); +int kvm_vm_set_region_attr (struct kvm *kvm,gfn_t start,gfn_t end,u64 attributes);bool kvm_get_kvm_safe (struct kvm *kvm);void kvm_put_kvm (struct kvm *kvm);bool file_is_kvm (struct file *file);
virt/kvm/kvm_main.c +20 -7
@@ -747,6 +747,7 @@ int kvm_vm_do_hva_range_op(struct kvm *kvm, unsigned long hva_start,return ret; } +EXPORT_SYMBOL_GPL (kvm_vm_do_hva_range_op); static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn, unsigned long start,
@@ -2595,12 +2596,28 @@ static void kvm_unmap_mem_range(struct kvm *kvm, gfn_t start, gfn_t end,kvm_flush_remote_tlbs (kvm); } +int kvm_vm_set_region_attr (struct kvm *kvm,gfn_t start,gfn_t end, +u64 attributes) +{ +gfn_t index; +void *entry; + + entry = attributes ?xa_mk_value (attributes) :NULL ; + +for (index = start; index < end; index++) +if (xa_err (xa_store (&kvm->mem_attr_array, index, entry, +GFP_KERNEL_ACCOUNT ))) +break ; + +return index; +} +EXPORT_SYMBOL_GPL (kvm_vm_set_region_attr); +static int kvm_vm_ioctl_set_mem_attributes (struct kvm *kvm,struct kvm_memory_attributes *attrs) {gfn_t start, end;unsigned long i; -void *entry;int idx;u64 supported_attrs =kvm_supported_mem_attributes (kvm);
@@ -2617,8 +2634,6 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, start = attrs->address >>PAGE_SHIFT ; end = (attrs->address + attrs->size -1 +PAGE_SIZE ) >>PAGE_SHIFT ; - entry = attrs->attributes ?xa_mk_value (attrs->attributes) :NULL ; -if (kvm_arch_has_private_mem (kvm)) {KVM_MMU_LOCK (kvm);kvm_mmu_invalidate_begin (kvm);
@@ -2627,10 +2642,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, }mutex_lock (&kvm->lock); -for (i = start; i < end; i++) -if (xa_err (xa_store (&kvm->mem_attr_array, i, entry, -GFP_KERNEL_ACCOUNT ))) -break ; + i =kvm_vm_set_region_attr (kvm, start, end, attrs->attributes);mutex_unlock (&kvm->lock);if (kvm_arch_has_private_mem (kvm)) {
@@ -2793,6 +2805,7 @@ unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot,return hva; } +EXPORT_SYMBOL_GPL (gfn_to_hva_memslot_prot);unsigned long gfn_to_hva_prot (struct kvm *kvm,gfn_t gfn,bool *writable) {
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 10/64] KVM: SEV: Populate private memory fd during LAUNCH_UPDATE_DATA
Quoted 3 lines · Michael Roth
From: Vishal Annapurve <[email protected]>
This change adds handling of HVA ranges to copy contents
s/This change adds handling of/Handle/
Quoted 15 lines
arch/x86/kvm/svm/sev.c +15
-0+static int sev_launch_update_priv_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) +{ +struct sev_data_launch_update_data data; +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; +gfn_t gfn; +kvm_pfn_t pfn; +struct kvm_memory_slot *memslot = range->slot; +int ret =0 ; + + data.reserved =0 ; + data.handle = sev->handle; + +for (gfn = range->start; gfn < range->end; gfn++) {
arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 591
Quoted 10 lines
arch/x86/kvm/svm/sev.c +10
-0+int order; +void *kvaddr; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + + kvaddr =pfn_to_kaddr (pfn); +if (!virt_addr_valid (kvaddr)) { +pr_err ("Invalid kvaddr 0x%llx\n" , (uint64_t )kvaddr);
Is that some debugging help leftover or what is that printk issued for?
arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 602
Quoted 11 lines
arch/x86/kvm/svm/sev.c +11
-0+ ret = -EINVAL ; + goto e_ret; + } + + ret =kvm_read_guest_page (kvm, gfn, kvaddr,0 ,PAGE_SIZE ); +if (ret) { +pr_err ("guest read failed 0x%x\n" , ret); + goto e_ret; + } + +if (!this_cpu_has (X86_FEATURE_SME_COHERENT ))
check_for_deprecated_apis: WARNING: arch/x86/kvm/svm/sev.c:602: Do not use this_cpu_has() - use cpu_feature_enabled() instead.
arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 613
Quoted 11 lines
arch/x86/kvm/svm/sev.c +11
-0+clflush_cache_range (kvaddr,PAGE_SIZE ); + + data.len =PAGE_SIZE ; + data.address =__sme_set (pfn <<PAGE_SHIFT ); + ret =sev_issue_cmd (kvm,SEV_CMD_LAUNCH_UPDATE_DATA , &data, &argp->error); +if (ret) + goto e_ret; +kvm_release_pfn_clean (pfn); + } +kvm_vm_set_region_attr (kvm, range->start, range->end, + true/* priv_attr */ );
No need to break that line.
Quoted 16 lines
arch/x86/kvm/svm/sev.c +16
-0+ +e_ret: +return ret; +} + +static int sev_launch_update_gfn_handler (struct kvm *kvm,struct kvm_gfn_range *range, +void *data) +{ +struct kvm_sev_cmd *argp = (struct kvm_sev_cmd *)data; + +if (kvm_slot_can_be_private (range->slot)) +return sev_launch_update_priv_gfn_handler (kvm, range, argp); + +return sev_launch_update_shared_gfn_handler (kvm, range, argp); +} +
arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 639
Quoted 10 lines
arch/x86/kvm/svm/sev.c +10
-0+static int sev_launch_update_data (struct kvm *kvm, +struct kvm_sev_cmd *argp) +{ +struct kvm_sev_launch_update_data params; + +if (!sev_guest (kvm)) +return -ENOTTY ; + +if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) +return -EFAULT ;
Not gonna check those user-supplied values for sanity?
Or is this check
if (WARN_ON_ONCE(hva_end <= hva_start))
return -EINVAL;in kvm_vm_do_hva_range_op() enough?
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 10/64] KVM: SEV: Populate private memory fd during LAUNCH_UPDATE_DATA
Quoted 26 lines · Borislav Petkov
Quoted 3 lines · Michael Roth
From: Vishal Annapurve <[email protected]>
This change adds handling of HVA ranges to copy contents
s/This change adds handling of/Handle/
Quoted 19 lines
arch/x86/kvm/svm/sev.c +19
-0+static int sev_launch_update_priv_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) +{ +struct sev_data_launch_update_data data; +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; +gfn_t gfn; +kvm_pfn_t pfn; +struct kvm_memory_slot *memslot = range->slot; +int ret =0 ; + + data.reserved =0 ; + data.handle = sev->handle; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; +void *kvaddr; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order);
Quoted 8 lines
arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 591
Quoted 6 lines
arch/x86/kvm/svm/sev.c +6
-0+if (ret) +return ret; + + kvaddr =pfn_to_kaddr (pfn); +if (!virt_addr_valid (kvaddr)) { +pr_err ("Invalid kvaddr 0x%llx\n" , (uint64_t )kvaddr);Is that some debugging help leftover or what is that printk issued for?
A mix of error-reporting and debugging I think. I think the error message isn't needed since the error value will get plumbed straight to userspace and anything beyond that is kernel debugging, so I added some context and switched this to pr_debug().
Quoted 57 lines
arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 602
Quoted 11 lines
arch/x86/kvm/svm/sev.c +11
-0+ ret = -EINVAL ; + goto e_ret; + } + + ret =kvm_read_guest_page (kvm, gfn, kvaddr,0 ,PAGE_SIZE ); +if (ret) { +pr_err ("guest read failed 0x%x\n" , ret); + goto e_ret; + } + +if (!this_cpu_has (X86_FEATURE_SME_COHERENT ))
check_for_deprecated_apis: WARNING:arch/x86/kvm/svm/sev.c:602: Do not usethis_cpu_has()- usecpu_feature_enabled()instead.arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 613
Quoted 11 lines
arch/x86/kvm/svm/sev.c +11
-0+clflush_cache_range (kvaddr,PAGE_SIZE ); + + data.len =PAGE_SIZE ; + data.address =__sme_set (pfn <<PAGE_SHIFT ); + ret =sev_issue_cmd (kvm,SEV_CMD_LAUNCH_UPDATE_DATA , &data, &argp->error); +if (ret) + goto e_ret; +kvm_release_pfn_clean (pfn); + } +kvm_vm_set_region_attr (kvm, range->start, range->end, + true/* priv_attr */ );No need to break that line.
Quoted 16 lines
arch/x86/kvm/svm/sev.c +16
-0+ +e_ret: +return ret; +} + +static int sev_launch_update_gfn_handler (struct kvm *kvm,struct kvm_gfn_range *range, +void *data) +{ +struct kvm_sev_cmd *argp = (struct kvm_sev_cmd *)data; + +if (kvm_slot_can_be_private (range->slot)) +return sev_launch_update_priv_gfn_handler (kvm, range, argp); + +return sev_launch_update_shared_gfn_handler (kvm, range, argp); +} +arch/x86/kvm/svm/sev.c static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 639
Quoted 10 lines
arch/x86/kvm/svm/sev.c +10
-0+static int sev_launch_update_data (struct kvm *kvm, +struct kvm_sev_cmd *argp) +{ +struct kvm_sev_launch_update_data params; + +if (!sev_guest (kvm)) +return -ENOTTY ; + +if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) +return -EFAULT ;Not gonna check those user-supplied values for sanity?
Quoted 6 lines
Or is this check
if (WARN_ON_ONCE(hva_end <= hva_start)) return -EINVAL;in
kvm_vm_do_hva_range_op()enough?
Only partially, but kvm_vm_do_hva_range_op() should sanitize the address range and only call sev_launch_update_gfn_handler() on valid GFNs within the range, so if userspace provides a bogus HVA range that doesn't actually correspond to a valid memslot it'll simply be treated as a no-op.
-Mike
Quoted 8 lines
Thx.
Signature · 4 lines
Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Re: [PATCH RFC v7 10/64] KVM: SEV: Populate private memory fd during LAUNCH_UPDATE_DATA
In reply to Michael RothQuoted 229 lines · Michael Roth
From: Vishal Annapurve <[email protected]>
This change adds handling of HVA ranges to copy contents to private memory while doing sev launch update data.
mem_attrarray is updated duringLAUNCH_UPDATE_DATAto ensure that encrypted memory is marked as private.Signed-off-by Vishal Annapurve <[email protected]>[mdr: use
gfn_to_hva_memslot_prot()for shared GFN handler to deal with
read-only slots for ROMs]Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/svm/sev.c | 99 ++++++++++++++++++++++++++++++++++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 27 ++++++++--- 3 files changed, 111 insertions(+), 16 deletions(-)arch/x86/kvm/svm/sev.c +90
-9@@ -493,23 +493,26 @@ static unsigned long get_num_contig_pages(unsigned long idx,return pages; } -static int sev_launch_update_data (struct kvm *kvm,struct kvm_sev_cmd *argp) +static int sev_launch_update_shared_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) {unsigned long vaddr, vaddr_end, next_vaddr, npages, pages, size, i;struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; -struct kvm_sev_launch_update_data params;struct sev_data_launch_update_data data;struct page **inpages;int ret; -if (!sev_guest (kvm)) -return -ENOTTY ; - -if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) -return -EFAULT ; + vaddr =gfn_to_hva_memslot_prot (range->slot, range->start,NULL ); +pr_debug ("%s: shared GFN: %llx, slot.id: %d, slot.base_gfn: %llx, slot.userspace_addr: %lx, slot.flags: %x, vaddr: %lx\n" , + __func__, range->start, range->slot->id, range->slot->base_gfn, + range->slot->userspace_addr, range->slot->flags, vaddr); +if (kvm_is_error_hva (vaddr)) { +pr_err ("vaddr is erroneous 0x%lx\n" , vaddr); +return -EINVAL ; + } - vaddr = params.uaddr; - size = params.len; + size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size;/* Lock the user memory. */ @@ -561,6 +564,84 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_launch_update_priv_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) +{ +struct sev_data_launch_update_data data; +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; +gfn_t gfn; +kvm_pfn_t pfn; +struct kvm_memory_slot *memslot = range->slot; +int ret =0 ; + + data.reserved =0 ; + data.handle = sev->handle; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; +void *kvaddr; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + + kvaddr =pfn_to_kaddr (pfn); +if (!virt_addr_valid (kvaddr)) { +pr_err ("Invalid kvaddr 0x%llx\n" , (uint64_t )kvaddr); + ret = -EINVAL ; + goto e_ret; + } + + ret =kvm_read_guest_page (kvm, gfn, kvaddr,0 ,PAGE_SIZE ); +if (ret) { +pr_err ("guest read failed 0x%x\n" , ret); + goto e_ret; + } + +if (!this_cpu_has (X86_FEATURE_SME_COHERENT )) +clflush_cache_range (kvaddr,PAGE_SIZE ); + + data.len =PAGE_SIZE ; + data.address =__sme_set (pfn <<PAGE_SHIFT ); + ret =sev_issue_cmd (kvm,SEV_CMD_LAUNCH_UPDATE_DATA , &data, &argp->error); +if (ret) + goto e_ret; +kvm_release_pfn_clean (pfn); + } +kvm_vm_set_region_attr (kvm, range->start, range->end, + true/* priv_attr */ ); + +e_ret: +return ret; +} + +static int sev_launch_update_gfn_handler (struct kvm *kvm,struct kvm_gfn_range *range, +void *data) +{ +struct kvm_sev_cmd *argp = (struct kvm_sev_cmd *)data; + +if (kvm_slot_can_be_private (range->slot)) +return sev_launch_update_priv_gfn_handler (kvm, range, argp); + +return sev_launch_update_shared_gfn_handler (kvm, range, argp); +} + +static int sev_launch_update_data (struct kvm *kvm, +struct kvm_sev_cmd *argp) +{ +struct kvm_sev_launch_update_data params; + +if (!sev_guest (kvm)) +return -ENOTTY ; + +if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) +return -EFAULT ; + +return kvm_vm_do_hva_range_op (kvm, params.uaddr, params.uaddr + params.len, + sev_launch_update_gfn_handler, argp); +} +static int sev_es_sync_vmsa (struct vcpu_svm *svm) {struct sev_es_save_area *save = svm->sev_es.vmsa;include/linux/kvm_host.h +1
-0@@ -973,6 +973,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,void kvm_exit (void );void kvm_get_kvm (struct kvm *kvm); +int kvm_vm_set_region_attr (struct kvm *kvm,gfn_t start,gfn_t end,u64 attributes);bool kvm_get_kvm_safe (struct kvm *kvm);void kvm_put_kvm (struct kvm *kvm);bool file_is_kvm (struct file *file);virt/kvm/kvm_main.c +20
-7@@ -747,6 +747,7 @@ int kvm_vm_do_hva_range_op(struct kvm *kvm, unsigned long hva_start,return ret; } +EXPORT_SYMBOL_GPL (kvm_vm_do_hva_range_op); static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn, unsigned long start,@@ -2595,12 +2596,28 @@ static void kvm_unmap_mem_range(struct kvm *kvm, gfn_t start, gfn_t end,kvm_flush_remote_tlbs (kvm); } +int kvm_vm_set_region_attr (struct kvm *kvm,gfn_t start,gfn_t end, +u64 attributes) +{ +gfn_t index; +void *entry; + + entry = attributes ?xa_mk_value (attributes) :NULL ; + +for (index = start; index < end; index++) +if (xa_err (xa_store (&kvm->mem_attr_array, index, entry, +GFP_KERNEL_ACCOUNT ))) +break ; + +return index; +} +EXPORT_SYMBOL_GPL (kvm_vm_set_region_attr); +static int kvm_vm_ioctl_set_mem_attributes (struct kvm *kvm,struct kvm_memory_attributes *attrs) {gfn_t start, end;unsigned long i; -void *entry;int idx;u64 supported_attrs =kvm_supported_mem_attributes (kvm);@@ -2617,8 +2634,6 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, start = attrs->address >>PAGE_SHIFT ; end = (attrs->address + attrs->size -1 +PAGE_SIZE ) >>PAGE_SHIFT ; - entry = attrs->attributes ?xa_mk_value (attrs->attributes) :NULL ; -if (kvm_arch_has_private_mem (kvm)) {KVM_MMU_LOCK (kvm);kvm_mmu_invalidate_begin (kvm);@@ -2627,10 +2642,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, }mutex_lock (&kvm->lock); -for (i = start; i < end; i++) -if (xa_err (xa_store (&kvm->mem_attr_array, i, entry, -GFP_KERNEL_ACCOUNT ))) -break ; + i =kvm_vm_set_region_attr (kvm, start, end, attrs->attributes);mutex_unlock (&kvm->lock);if (kvm_arch_has_private_mem (kvm)) {@@ -2793,6 +2805,7 @@ unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot,return hva; } +EXPORT_SYMBOL_GPL (gfn_to_hva_memslot_prot);unsigned long gfn_to_hva_prot (struct kvm *kvm,gfn_t gfn,bool *writable) {Signature · 1 line
2.25.1
Hmm.. but user space is still allowed to call KVM_SET_MEMORY_ATTRIBUTES with KVM_MEMORY_ATTRIBUTE_PRIVATE set? How do these behaviours complement each other?
SEV specific changes and kvm_vm_set_region_attr() definition should really be separate patches.
BR, Jarkko
Re: [PATCH RFC v7 10/64] KVM: SEV: Populate private memory fd during LAUNCH_UPDATE_DATA
Quoted 225 lines · Jarkko Sakkinen
Quoted 224 lines · Michael Roth
From: Vishal Annapurve <[email protected]>
This change adds handling of HVA ranges to copy contents to private memory while doing sev launch update data.
mem_attrarray is updated duringLAUNCH_UPDATE_DATAto ensure that encrypted memory is marked as private.Signed-off-by Vishal Annapurve <[email protected]>[mdr: use
gfn_to_hva_memslot_prot()for shared GFN handler to deal with
read-only slots for ROMs]Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/svm/sev.c | 99 ++++++++++++++++++++++++++++++++++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 27 ++++++++--- 3 files changed, 111 insertions(+), 16 deletions(-)arch/x86/kvm/svm/sev.c +90
-9@@ -493,23 +493,26 @@ static unsigned long get_num_contig_pages(unsigned long idx,return pages; } -static int sev_launch_update_data (struct kvm *kvm,struct kvm_sev_cmd *argp) +static int sev_launch_update_shared_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) {unsigned long vaddr, vaddr_end, next_vaddr, npages, pages, size, i;struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; -struct kvm_sev_launch_update_data params;struct sev_data_launch_update_data data;struct page **inpages;int ret; -if (!sev_guest (kvm)) -return -ENOTTY ; - -if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) -return -EFAULT ; + vaddr =gfn_to_hva_memslot_prot (range->slot, range->start,NULL ); +pr_debug ("%s: shared GFN: %llx, slot.id: %d, slot.base_gfn: %llx, slot.userspace_addr: %lx, slot.flags: %x, vaddr: %lx\n" , + __func__, range->start, range->slot->id, range->slot->base_gfn, + range->slot->userspace_addr, range->slot->flags, vaddr); +if (kvm_is_error_hva (vaddr)) { +pr_err ("vaddr is erroneous 0x%lx\n" , vaddr); +return -EINVAL ; + } - vaddr = params.uaddr; - size = params.len; + size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size;/* Lock the user memory. */ @@ -561,6 +564,84 @@ static int sev_launch_update_data(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_launch_update_priv_gfn_handler (struct kvm *kvm, +struct kvm_gfn_range *range, +struct kvm_sev_cmd *argp) +{ +struct sev_data_launch_update_data data; +struct kvm_sev_info *sev = &to_kvm_svm (kvm)->sev_info; +gfn_t gfn; +kvm_pfn_t pfn; +struct kvm_memory_slot *memslot = range->slot; +int ret =0 ; + + data.reserved =0 ; + data.handle = sev->handle; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; +void *kvaddr; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + + kvaddr =pfn_to_kaddr (pfn); +if (!virt_addr_valid (kvaddr)) { +pr_err ("Invalid kvaddr 0x%llx\n" , (uint64_t )kvaddr); + ret = -EINVAL ; + goto e_ret; + } + + ret =kvm_read_guest_page (kvm, gfn, kvaddr,0 ,PAGE_SIZE ); +if (ret) { +pr_err ("guest read failed 0x%x\n" , ret); + goto e_ret; + } + +if (!this_cpu_has (X86_FEATURE_SME_COHERENT )) +clflush_cache_range (kvaddr,PAGE_SIZE ); + + data.len =PAGE_SIZE ; + data.address =__sme_set (pfn <<PAGE_SHIFT ); + ret =sev_issue_cmd (kvm,SEV_CMD_LAUNCH_UPDATE_DATA , &data, &argp->error); +if (ret) + goto e_ret; +kvm_release_pfn_clean (pfn); + } +kvm_vm_set_region_attr (kvm, range->start, range->end, + true/* priv_attr */ ); + +e_ret: +return ret; +} + +static int sev_launch_update_gfn_handler (struct kvm *kvm,struct kvm_gfn_range *range, +void *data) +{ +struct kvm_sev_cmd *argp = (struct kvm_sev_cmd *)data; + +if (kvm_slot_can_be_private (range->slot)) +return sev_launch_update_priv_gfn_handler (kvm, range, argp); + +return sev_launch_update_shared_gfn_handler (kvm, range, argp); +} + +static int sev_launch_update_data (struct kvm *kvm, +struct kvm_sev_cmd *argp) +{ +struct kvm_sev_launch_update_data params; + +if (!sev_guest (kvm)) +return -ENOTTY ; + +if (copy_from_user (¶ms, (void __user *)(uintptr_t )argp->data,sizeof (params))) +return -EFAULT ; + +return kvm_vm_do_hva_range_op (kvm, params.uaddr, params.uaddr + params.len, + sev_launch_update_gfn_handler, argp); +} +static int sev_es_sync_vmsa (struct vcpu_svm *svm) {struct sev_es_save_area *save = svm->sev_es.vmsa;include/linux/kvm_host.h +1
-0@@ -973,6 +973,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,void kvm_exit (void );void kvm_get_kvm (struct kvm *kvm); +int kvm_vm_set_region_attr (struct kvm *kvm,gfn_t start,gfn_t end,u64 attributes);bool kvm_get_kvm_safe (struct kvm *kvm);void kvm_put_kvm (struct kvm *kvm);bool file_is_kvm (struct file *file);virt/kvm/kvm_main.c +20
-7@@ -747,6 +747,7 @@ int kvm_vm_do_hva_range_op(struct kvm *kvm, unsigned long hva_start,return ret; } +EXPORT_SYMBOL_GPL (kvm_vm_do_hva_range_op); static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn, unsigned long start,@@ -2595,12 +2596,28 @@ static void kvm_unmap_mem_range(struct kvm *kvm, gfn_t start, gfn_t end,kvm_flush_remote_tlbs (kvm); } +int kvm_vm_set_region_attr (struct kvm *kvm,gfn_t start,gfn_t end, +u64 attributes) +{ +gfn_t index; +void *entry; + + entry = attributes ?xa_mk_value (attributes) :NULL ; + +for (index = start; index < end; index++) +if (xa_err (xa_store (&kvm->mem_attr_array, index, entry, +GFP_KERNEL_ACCOUNT ))) +break ; + +return index; +} +EXPORT_SYMBOL_GPL (kvm_vm_set_region_attr); +static int kvm_vm_ioctl_set_mem_attributes (struct kvm *kvm,struct kvm_memory_attributes *attrs) {gfn_t start, end;unsigned long i; -void *entry;int idx;u64 supported_attrs =kvm_supported_mem_attributes (kvm);@@ -2617,8 +2634,6 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, start = attrs->address >>PAGE_SHIFT ; end = (attrs->address + attrs->size -1 +PAGE_SIZE ) >>PAGE_SHIFT ; - entry = attrs->attributes ?xa_mk_value (attrs->attributes) :NULL ; -if (kvm_arch_has_private_mem (kvm)) {KVM_MMU_LOCK (kvm);kvm_mmu_invalidate_begin (kvm);@@ -2627,10 +2642,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, }mutex_lock (&kvm->lock); -for (i = start; i < end; i++) -if (xa_err (xa_store (&kvm->mem_attr_array, i, entry, -GFP_KERNEL_ACCOUNT ))) -break ; + i =kvm_vm_set_region_attr (kvm, start, end, attrs->attributes);mutex_unlock (&kvm->lock);if (kvm_arch_has_private_mem (kvm)) {@@ -2793,6 +2805,7 @@ unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot,return hva; } +EXPORT_SYMBOL_GPL (gfn_to_hva_memslot_prot);
Quoted 9 lines
Quoted 5 lines
virt/kvm/kvm_main.c +0
-0unsigned long gfn_to_hva_prot (struct kvm *kvm,gfn_t gfn,bool *writable) {Signature · 1 line
2.25.1Hmm.. but user space is still allowed to call
KVM_SET_MEMORY_ATTRIBUTESwithKVM_MEMORY_ATTRIBUTE_PRIVATEset? How do these behaviours complement each other?
In the current series it's allowed. It reduces some headaches since an SNP guest can issue duplicate page-state changes without violating the spec, and it would complicate things further if kernel/userspace attempted to enforce something stricter.
Quoted 3 lines
SEV specific changes and
kvm_vm_set_region_attr()definition should really be separate patches.
Done.
-Mike
Quoted 2 lines
BR, Jarkko
RFC PATCH v7 11/64 KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
In reply to Michael RothFrom: Nikunj A Dadhania <[email protected]>
Pre-boot guest payload needs to be encrypted and VMM has copied it over to the private-fd. Add support to get the pfn from the memfile fd for encrypting the payload in-place.
--- arch/x86/kvm/svm/sev.c | 79 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 64 insertions(+), 15 deletions(-)
arch/x86/kvm/svm/sev.c +64 -15
@@ -107,6 +107,11 @@ static inline bool is_mirroring_enc_context(struct kvm *kvm)return !!to_kvm_svm (kvm)->sev_info.enc_context_owner; } +static bool kvm_is_upm_enabled (struct kvm *kvm) +{ +return kvm->arch.upm_mode; +} +/* Must be called with the sev_bitmap_lock held */ static bool __sev_recycle_asids (int min_asid,int max_asid) {
@@ -382,6 +387,38 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_get_memfile_pfn_handler (struct kvm *kvm,struct kvm_gfn_range *range,void *data) +{ +struct kvm_memory_slot *memslot = range->slot; +struct page **pages = data; +int ret =0 , i =0 ; +kvm_pfn_t pfn; +gfn_t gfn; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + +if (is_error_noslot_pfn (pfn)) +return -EFAULT ; + + pages[i++] =pfn_to_page (pfn); + } + +return ret; +} + +static int sev_get_memfile_pfn (struct kvm *kvm,unsigned long addr, +unsigned long size,unsigned long npages, +struct page **pages) +{ +return kvm_vm_do_hva_range_op (kvm, addr, size, + sev_get_memfile_pfn_handler, pages); +} +static struct page **sev_pin_memory (struct kvm *kvm,unsigned long uaddr,unsigned long ulen,unsigned long *n,int write)
@@ -424,16 +461,25 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,if (!pages)return ERR_PTR (-ENOMEM ); -/* Pin the user virtual address. */ - npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); -if (npinned != npages) { -pr_err ("SEV: Failure locking %lu pages.\n" , npages); - ret = -ENOMEM ; - goto err; +if (kvm_is_upm_enabled (kvm)) { +/* Get the PFN from memfile */ +if (sev_get_memfile_pfn (kvm, uaddr, ulen, npages, pages)) { +pr_err ("%s: ERROR: unable to find slot for uaddr %lx" , __func__, uaddr); + ret = -ENOMEM ; + goto err; + } + }else { +/* Pin the user virtual address. */ + npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); +if (npinned != npages) { +pr_err ("SEV: Failure locking %lu pages.\n" , npages); + ret = -ENOMEM ; + goto err; + } + sev->pages_locked = locked; } *n = npages; - sev->pages_locked = locked;return pages;
@@ -514,6 +560,7 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size; +WARN_ON (size <PAGE_SIZE );/* Lock the user memory. */ inpages =sev_pin_memory (kvm, vaddr, size, &npages,1 );
@@ -554,13 +601,16 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, } e_unpin: -/* content of memory is updated, mark pages dirty */ -for (i =0 ; i < npages; i++) { -set_page_dirty_lock (inpages[i]); -mark_page_accessed (inpages[i]); +if (!kvm_is_upm_enabled (kvm)) { +/* content of memory is updated, mark pages dirty */ +for (i =0 ; i < npages; i++) { +set_page_dirty_lock (inpages[i]); +mark_page_accessed (inpages[i]); + } +/* unlock the user pages */ +sev_unpin_memory (kvm, inpages, npages); } -/* unlock the user pages */ -sev_unpin_memory (kvm, inpages, npages); +return ret; }
@@ -609,9 +659,8 @@ static int sev_launch_update_priv_gfn_handler(struct kvm *kvm, goto e_ret;kvm_release_pfn_clean (pfn); } -kvm_vm_set_region_attr (kvm, range->start, range->end, - true/* priv_attr */ ); +kvm_vm_set_region_attr (kvm, range->start, range->end,KVM_MEMORY_ATTRIBUTE_PRIVATE ); e_ret:return ret; }
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 11/64] KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
Quoted 53 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Pre-boot guest payload needs to be encrypted and VMM has copied it over to the private-fd. Add support to get the pfn from the memfile fd for encrypting the payload in-place.
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/svm/sev.c | 79 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 64 insertions(+), 15 deletions(-)arch/x86/kvm/svm/sev.c +26
-0@@ -107,6 +107,11 @@ static inline bool is_mirroring_enc_context(struct kvm *kvm)return !!to_kvm_svm (kvm)->sev_info.enc_context_owner; } +static bool kvm_is_upm_enabled (struct kvm *kvm) +{ +return kvm->arch.upm_mode; +} +/* Must be called with the sev_bitmap_lock held */ static bool __sev_recycle_asids (int min_asid,int max_asid) {@@ -382,6 +387,38 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_get_memfile_pfn_handler (struct kvm *kvm,struct kvm_gfn_range *range,void *data) +{ +struct kvm_memory_slot *memslot = range->slot; +struct page **pages = data; +int ret =0 , i =0 ; +kvm_pfn_t pfn; +gfn_t gfn; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + +if (is_error_noslot_pfn (pfn)) +return -EFAULT ; + + pages[i++] =pfn_to_page (pfn); + } +
arch/x86/kvm/svm/sev.c static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 420
Quoted 10 lines
arch/x86/kvm/svm/sev.c +10
-0+return ret; +} + +static int sev_get_memfile_pfn (struct kvm *kvm,unsigned long addr, +unsigned long size,unsigned long npages, +struct page **pages) +{ +return kvm_vm_do_hva_range_op (kvm, addr, size, + sev_get_memfile_pfn_handler, pages); +}
The third argument for the kvm_vm_do_hva_range_op call should be addr+size; the function expects the end of the range not the size of the range.
Quoted 11 lines
arch/x86/kvm/svm/sev.c +1
-3+static struct page **sev_pin_memory (struct kvm *kvm,unsigned long uaddr,unsigned long ulen,unsigned long *n,int write)@@ -424,16 +461,25 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,if (!pages)return ERR_PTR (-ENOMEM ); -/* Pin the user virtual address. */ - npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); -if (npinned != npages) {
arch/x86/kvm/svm/sev.c static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, · line 470
Quoted 10 lines
arch/x86/kvm/svm/sev.c +7
-3-pr_err ("SEV: Failure locking %lu pages.\n" , npages); - ret = -ENOMEM ; - goto err; +if (kvm_is_upm_enabled (kvm)) { +/* Get the PFN from memfile */ +if (sev_get_memfile_pfn (kvm, uaddr, ulen, npages, pages)) { +pr_err ("%s: ERROR: unable to find slot for uaddr %lx" , __func__, uaddr); + ret = -ENOMEM ; + goto err; + }
This branch doesn't initialize npinned. If sev_get_memfile_pfn fails, the code following the err label passes the uninitialized value to unpin_user_pages.
Quoted 59 lines
arch/x86/kvm/svm/sev.c +20
-10+ }else { +/* Pin the user virtual address. */ + npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); +if (npinned != npages) { +pr_err ("SEV: Failure locking %lu pages.\n" , npages); + ret = -ENOMEM ; + goto err; + } + sev->pages_locked = locked; } *n = npages; - sev->pages_locked = locked;return pages;@@ -514,6 +560,7 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size; +WARN_ON (size <PAGE_SIZE );/* Lock the user memory. */ inpages =sev_pin_memory (kvm, vaddr, size, &npages,1 );@@ -554,13 +601,16 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, } e_unpin: -/* content of memory is updated, mark pages dirty */ -for (i =0 ; i < npages; i++) { -set_page_dirty_lock (inpages[i]); -mark_page_accessed (inpages[i]); +if (!kvm_is_upm_enabled (kvm)) { +/* content of memory is updated, mark pages dirty */ +for (i =0 ; i < npages; i++) { +set_page_dirty_lock (inpages[i]); +mark_page_accessed (inpages[i]); + } +/* unlock the user pages */ +sev_unpin_memory (kvm, inpages, npages); } -/* unlock the user pages */ -sev_unpin_memory (kvm, inpages, npages); +return ret; }@@ -609,9 +659,8 @@ static int sev_launch_update_priv_gfn_handler(struct kvm *kvm, goto e_ret;kvm_release_pfn_clean (pfn); } -kvm_vm_set_region_attr (kvm, range->start, range->end, - true/* priv_attr */ ); +kvm_vm_set_region_attr (kvm, range->start, range->end,KVM_MEMORY_ATTRIBUTE_PRIVATE ); e_ret:return ret; } --
Quoted 2 lines
2.25.1
Regards, Tom
Re: [PATCH RFC v7 11/64] KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
Quoted 59 lines · On 22/12/22 23:54, [email protected] wrote:
Quoted 58 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Pre-boot guest payload needs to be encrypted and VMM has copied it over to the private-fd. Add support to get the pfn from the memfile fd for encrypting the payload in-place.
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/svm/sev.c | 79 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 64 insertions(+), 15 deletions(-)arch/x86/kvm/svm/sev.c +31
-0@@ -107,6 +107,11 @@ static inline bool is_mirroring_enc_context(struct kvm *kvm)return !!to_kvm_svm (kvm)->sev_info.enc_context_owner; } +static bool kvm_is_upm_enabled (struct kvm *kvm) +{ +return kvm->arch.upm_mode; +} +/* Must be called with the sev_bitmap_lock held */ static bool __sev_recycle_asids (int min_asid,int max_asid) {@@ -382,6 +387,38 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_get_memfile_pfn_handler (struct kvm *kvm,struct kvm_gfn_range *range,void *data) +{ +struct kvm_memory_slot *memslot = range->slot; +struct page **pages = data; +int ret =0 , i =0 ; +kvm_pfn_t pfn; +gfn_t gfn; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + +if (is_error_noslot_pfn (pfn)) +return -EFAULT ; + + pages[i++] =pfn_to_page (pfn); + } + +return ret; +} + +static int sev_get_memfile_pfn(struct kvm *kvm, unsigned long addr, + unsigned long size, unsigned long npages,
Quoted 8 lines
arch/x86/kvm/svm/sev.c static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 420
Quoted 5 lines
arch/x86/kvm/svm/sev.c +5
-0+ struct page **pages) +{ +return kvm_vm_do_hva_range_op (kvm, addr, size, + sev_get_memfile_pfn_handler, pages); +}The third argument for the
kvm_vm_do_hva_range_opcall should be addr+size; the function expects the end of the range not the size of the range.
Good catch, will fix.
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/svm/sev.c +1
-0+static struct page **sev_pin_memory (struct kvm *kvm,unsigned long uaddr,unsigned long ulen,unsigned long *n,int write)@@ -424,16 +461,25 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,if (!pages)return ERR_PTR (-ENOMEM );
Quoted 16 lines
arch/x86/kvm/svm/sev.c static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr, · line 470
Quoted 13 lines
arch/x86/kvm/svm/sev.c +7
-6-/* Pin the user virtual address. */ - npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); -if (npinned != npages) { -pr_err ("SEV: Failure locking %lu pages.\n" , npages); - ret = -ENOMEM ; - goto err; +if (kvm_is_upm_enabled (kvm)) { +/* Get the PFN from memfile */ +if (sev_get_memfile_pfn (kvm, uaddr, ulen, npages, pages)) { +pr_err ("%s: ERROR: unable to find slot for uaddr %lx" , __func__, uaddr); + ret = -ENOMEM ; + goto err; + }This branch doesn't initialize npinned. If
sev_get_memfile_pfnfails, the code following the err label passes the uninitialized value tounpin_user_pages.
Sure, will fix.
Quoted 55 lines
Quoted 55 lines
arch/x86/kvm/svm/sev.c +19
-9+ }else { +/* Pin the user virtual address. */ + npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); +if (npinned != npages) { +pr_err ("SEV: Failure locking %lu pages.\n" , npages); + ret = -ENOMEM ; + goto err; + } + sev->pages_locked = locked; } *n = npages; - sev->pages_locked = locked;return pages;@@ -514,6 +560,7 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size; +WARN_ON (size <PAGE_SIZE );/* Lock the user memory. */ inpages =sev_pin_memory (kvm, vaddr, size, &npages,1 );@@ -554,13 +601,16 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, } e_unpin: -/* content of memory is updated, mark pages dirty */ -for (i =0 ; i < npages; i++) { -set_page_dirty_lock (inpages[i]); -mark_page_accessed (inpages[i]); +if (!kvm_is_upm_enabled (kvm)) { +/* content of memory is updated, mark pages dirty */ +for (i =0 ; i < npages; i++) { +set_page_dirty_lock (inpages[i]); +mark_page_accessed (inpages[i]); + } +/* unlock the user pages */ +sev_unpin_memory (kvm, inpages, npages); } -/* unlock the user pages */ -sev_unpin_memory (kvm, inpages, npages); +return ret; }@@ -609,9 +659,8 @@ static int sev_launch_update_priv_gfn_handler(struct kvm *kvm, goto e_ret;kvm_release_pfn_clean (pfn); } -kvm_vm_set_region_attr (kvm, range->start, range->end, - true/* priv_attr */ );
Quoted 9 lines
Quoted 7 lines
arch/x86/kvm/svm/sev.c +1
-1+kvm_vm_set_region_attr (kvm, range->start, range->end,KVM_MEMORY_ATTRIBUTE_PRIVATE ); e_ret:return ret; } --2.25.1
Regards, Tom
Thanks
Nikunj
Re: [PATCH RFC v7 11/64] KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
In reply to Michael RothQuoted 145 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Pre-boot guest payload needs to be encrypted and VMM has copied it over to the private-fd. Add support to get the pfn from the memfile fd for encrypting the payload in-place.
Signed-off-by Nikunj A Dadhania <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/kvm/svm/sev.c | 79 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 64 insertions(+), 15 deletions(-)arch/x86/kvm/svm/sev.c +64
-15@@ -107,6 +107,11 @@ static inline bool is_mirroring_enc_context(struct kvm *kvm)return !!to_kvm_svm (kvm)->sev_info.enc_context_owner; } +static bool kvm_is_upm_enabled (struct kvm *kvm) +{ +return kvm->arch.upm_mode; +} +/* Must be called with the sev_bitmap_lock held */ static bool __sev_recycle_asids (int min_asid,int max_asid) {@@ -382,6 +387,38 @@ static int sev_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)return ret; } +static int sev_get_memfile_pfn_handler (struct kvm *kvm,struct kvm_gfn_range *range,void *data) +{ +struct kvm_memory_slot *memslot = range->slot; +struct page **pages = data; +int ret =0 , i =0 ; +kvm_pfn_t pfn; +gfn_t gfn; + +for (gfn = range->start; gfn < range->end; gfn++) { +int order; + + ret =kvm_restricted_mem_get_pfn (memslot, gfn, &pfn, &order); +if (ret) +return ret; + +if (is_error_noslot_pfn (pfn)) +return -EFAULT ; + + pages[i++] =pfn_to_page (pfn); + } + +return ret; +} + +static int sev_get_memfile_pfn (struct kvm *kvm,unsigned long addr, +unsigned long size,unsigned long npages, +struct page **pages) +{ +return kvm_vm_do_hva_range_op (kvm, addr, size, + sev_get_memfile_pfn_handler, pages); +} +static struct page **sev_pin_memory (struct kvm *kvm,unsigned long uaddr,unsigned long ulen,unsigned long *n,int write)@@ -424,16 +461,25 @@ static struct page **sev_pin_memory(struct kvm *kvm, unsigned long uaddr,if (!pages)return ERR_PTR (-ENOMEM ); -/* Pin the user virtual address. */ - npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); -if (npinned != npages) { -pr_err ("SEV: Failure locking %lu pages.\n" , npages); - ret = -ENOMEM ; - goto err; +if (kvm_is_upm_enabled (kvm)) { +/* Get the PFN from memfile */ +if (sev_get_memfile_pfn (kvm, uaddr, ulen, npages, pages)) { +pr_err ("%s: ERROR: unable to find slot for uaddr %lx" , __func__, uaddr); + ret = -ENOMEM ; + goto err; + } + }else { +/* Pin the user virtual address. */ + npinned =pin_user_pages_fast (uaddr, npages, write ?FOLL_WRITE :0 , pages); +if (npinned != npages) { +pr_err ("SEV: Failure locking %lu pages.\n" , npages); + ret = -ENOMEM ; + goto err; + } + sev->pages_locked = locked; } *n = npages; - sev->pages_locked = locked;return pages;@@ -514,6 +560,7 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, size = (range->end - range->start) <<PAGE_SHIFT ; vaddr_end = vaddr + size; +WARN_ON (size <PAGE_SIZE );/* Lock the user memory. */ inpages =sev_pin_memory (kvm, vaddr, size, &npages,1 );@@ -554,13 +601,16 @@ static int sev_launch_update_shared_gfn_handler(struct kvm *kvm, } e_unpin: -/* content of memory is updated, mark pages dirty */ -for (i =0 ; i < npages; i++) { -set_page_dirty_lock (inpages[i]); -mark_page_accessed (inpages[i]); +if (!kvm_is_upm_enabled (kvm)) { +/* content of memory is updated, mark pages dirty */ +for (i =0 ; i < npages; i++) { +set_page_dirty_lock (inpages[i]); +mark_page_accessed (inpages[i]); + } +/* unlock the user pages */ +sev_unpin_memory (kvm, inpages, npages); } -/* unlock the user pages */ -sev_unpin_memory (kvm, inpages, npages); +return ret; }@@ -609,9 +659,8 @@ static int sev_launch_update_priv_gfn_handler(struct kvm *kvm, goto e_ret;kvm_release_pfn_clean (pfn); } -kvm_vm_set_region_attr (kvm, range->start, range->end, - true/* priv_attr */ ); +kvm_vm_set_region_attr (kvm, range->start, range->end,KVM_MEMORY_ATTRIBUTE_PRIVATE ); e_ret:return ret; }Signature · 1 line
2.25.1
kvm_vm_set_region_attr() should be fixed already in:
https://lore.kernel.org/all/[email protected]/
BR, Jarkko
Re: [PATCH RFC v7 11/64] KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
Quoted 11 lines · Jarkko Sakkinen
arch/x86/kvm/svm/sev.c static int sev_launch_update_priv_gfn_handler(struct kvm *kvm, · line 663
Quoted 10 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
arch/x86/kvm/svm/sev.c +1
-2@@ -609,9 +659,8 @@ static int sev_launch_update_priv_gfn_handler(struct kvm *kvm, goto e_ret;kvm_release_pfn_clean (pfn); } -kvm_vm_set_region_attr (kvm, range->start, range->end, - true/* priv_attr */ ); +kvm_vm_set_region_attr (kvm, range->start, range->end,KVM_MEMORY_ATTRIBUTE_PRIVATE );
As the memory attribute is no more a boolean in the UPM series, I had this change.
Quoted 9 lines
Quoted 6 lines
arch/x86/kvm/svm/sev.c +0
-0e_ret:return ret; }Signature · 1 line
2.25.1
kvm_vm_set_region_attr()should be fixed already in:Quoted 1 line
Will discuss with Mike and move this hunk to above patch.
Regards
Nikunj
Borislav Petkov
maintainerRe: [PATCH RFC v7 11/64] KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
In reply to Michael RothQuoted 3 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Pre-boot guest payload needs to be encrypted and VMM has copied it
"has to have copied it over" I presume?
Quoted 2 lines
over to the private-fd. Add support to get the pfn from the memfile fd for encrypting the payload in-place.
Why is that a good thing?
I guess with UPM you're supposed to get the PFN of that encrypted guest payload from that memslot.
IOW, such commit messages are too laconic for my taste and you could try to explain more why this is happening instead of me having to "reverse-deduce" what you're doing from the code...
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 11/64] KVM: SEV: Support private pages in LAUNCH_UPDATE_DATA
Quoted 6 lines · Borislav Petkov
Quoted 3 lines · Michael Roth
From: Nikunj A Dadhania <[email protected]>
Pre-boot guest payload needs to be encrypted and VMM has copied it
"has to have copied it over" I presume?
True, payload is being copied in patch 10/64 now.
Quoted 12 lines
Quoted 2 lines
over to the private-fd. Add support to get the pfn from the memfile fd for encrypting the payload in-place.
Why is that a good thing?
I guess with UPM you're supposed to get the PFN of that encrypted guest payload from that memslot.
IOW, such commit messages are too laconic for my taste and you could try to explain more why this is happening instead of me having to "reverse-deduce" what you're doing from the code...
I am updating the SEV related patches, will add more details in commit and send.
Regards
Nikunj
RFC PATCH v7 12/64 KVM: SEV: Implement .fault_is_private callback
In reply to Michael RothKVM MMU will use this to determine whether an #NPF should be serviced with restricted memory or not.
--- arch/x86/kvm/svm/sev.c | 23 +++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/svm/svm.h | 2 ++ 3 files changed, 27 insertions(+)
arch/x86/kvm/svm/sev.c +23 -0
@@ -3179,3 +3179,26 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)ghcb_set_sw_exit_info_2 (svm->sev_es.ghcb,1 ); } + +int sev_fault_is_private (struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault) +{ +gfn_t gfn =gpa_to_gfn (gpa); + +if (!kvm_is_upm_enabled (kvm) || !sev_guest (kvm)) + goto out_unhandled; + +/* +* For SEV, the hypervisor is not aware of implicit conversions in the +* guest, so it relies purely on explicit conversions via +* KVM_EXIT_HYPERCALL, so the resulting handling by userspace should +* update the backing memory source accordingly. Therefore, the backing +* source is the only indicator of whether the fault should be treated +* as private or not. +*/ + *private_fault =kvm_mem_is_private (kvm, gfn); + +return 1 ; + +out_unhandled: +return 0 ; +}
arch/x86/kvm/svm/svm.c +2 -0
@@ -4830,6 +4830,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons, + + .fault_is_private = sev_fault_is_private, };/*
arch/x86/kvm/svm/svm.h +2 -0
@@ -683,6 +683,8 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);void sev_es_prepare_switch_to_guest (struct sev_es_save_area *hostsa);void sev_es_unmap_ghcb (struct vcpu_svm *svm); +int sev_fault_is_private (struct kvm *kvm,gpa_t gpa,u64 error_code,bool *private_fault); +/* vmenter.S */ void __svm_sev_es_vcpu_run (struct vcpu_svm *svm,bool spec_ctrl_intercepted);
Signature · 1 line
2.25.1
RFC PATCH v7 13/64 x86/cpufeatures: Add SEV-SNP CPU feature
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cpu/amd.c | 5 +++-- tools/arch/x86/include/asm/cpufeatures.h | 1 + 3 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -420,6 +420,7 @@#define X86_FEATURE_SEV (19*32+ 1)/* AMD Secure Encrypted Virtualization */ #define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2)/* "" VM Page Flush MSR is supported */ #define X86_FEATURE_SEV_ES (19*32+ 3)/* AMD Secure Encrypted Virtualization - Encrypted State */ +#define X86_FEATURE_SEV_SNP (19*32+ 4)/* AMD Secure Encrypted Virtualization - Secure Nested Paging */ #define X86_FEATURE_V_TSC_AUX (19*32+ 9)/* "" Virtual TSC_AUX */ #define X86_FEATURE_SME_COHERENT (19*32+10)/* "" AMD hardware-enforced cache coherency */
arch/x86/kernel/cpu/amd.c +3 -2
@@ -558,8 +558,8 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)* SME feature (set in scattered.c). * If the kernel has not enabled SME via any means then * don't advertise the SME feature. -* For SEV: If BIOS has not enabled SEV then don't advertise the -* SEV and SEV_ES feature (set in scattered.c). +* For SEV: If BIOS has not enabled SEV then don't advertise SEV and +* any additional functionality based on it. * * In all cases, since support for SME and SEV requires long mode, * don't advertise the feature under CONFIG_X86_32.
@@ -594,6 +594,7 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c) clear_sev:setup_clear_cpu_cap (X86_FEATURE_SEV );setup_clear_cpu_cap (X86_FEATURE_SEV_ES ); +setup_clear_cpu_cap (X86_FEATURE_SEV_SNP ); } }
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -417,6 +417,7 @@#define X86_FEATURE_SEV (19*32+ 1)/* AMD Secure Encrypted Virtualization */ #define X86_FEATURE_VM_PAGE_FLUSH (19*32+ 2)/* "" VM Page Flush MSR is supported */ #define X86_FEATURE_SEV_ES (19*32+ 3)/* AMD Secure Encrypted Virtualization - Encrypted State */ +#define X86_FEATURE_SEV_SNP (19*32+ 4)/* AMD Secure Encrypted Virtualization - Secure Nested Paging */ #define X86_FEATURE_V_TSC_AUX (19*32+ 9)/* "" Virtual TSC_AUX */ #define X86_FEATURE_SME_COHERENT (19*32+10)/* "" AMD hardware-enforced cache coherency */
Signature · 1 line
2.25.1
Borislav Petkov
maintainerRe: [PATCH RFC v7 13/64] x86/cpufeatures: Add SEV-SNP CPU feature
Quoted 8 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.
That points to some review feedback I've given - dunno if it is relevant.
Quoted 2 lines
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>
I read this as Jarkko has handled this patch too. Is that the case?
Quoted 2 lines
Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>
Those last two are ok - you took ovef from Ashish.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 13/64] x86/cpufeatures: Add SEV-SNP CPU feature
Quoted 10 lines · Borislav Petkov
Quoted 8 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.
Quoted 7 lines
That points to some review feedback I've given - dunno if it is relevant.
Quoted 2 lines
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>I read this as Jarkko has handled this patch too. Is that the case?
Yes we shared some patches via an internal tree at some stages.
-Mike
Quoted 13 lines
Quoted 3 lines
Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>Those last two are ok - you took ovef from Ashish.
Thx.
Signature · 4 lines
Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Borislav Petkov
maintainerRe: [PATCH RFC v7 13/64] x86/cpufeatures: Add SEV-SNP CPU feature
Quoted 14 lines · Michael Roth
Quoted 13 lines · Borislav Petkov
Quoted 8 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.
That points to some review feedback I've given - dunno if it is relevant.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>I read this as Jarkko has handled this patch too. Is that the case?
Yes we shared some patches via an internal tree at some stages.
In the sense that, he took Brijesh's patch, then he did something with it(?) and then Ashish took it from him and then you took it from Ashish?
This is how I'm reading this SOB chain at least...
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH RFC v7 13/64] x86/cpufeatures: Add SEV-SNP CPU feature
Quoted 18 lines · Borislav Petkov
Quoted 17 lines · Michael Roth
Quoted 16 lines · Borislav Petkov
Quoted 8 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Add CPU feature detection for Secure Encrypted Virtualization with Secure Nested Paging. This feature adds a strong memory integrity protection to help prevent malicious hypervisor-based attacks like data replay, memory re-mapping, and more.
That points to some review feedback I've given - dunno if it is relevant.
Quoted 2 lines
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
I read this as Jarkko has handled this patch too. Is that the case?
Yes we shared some patches via an internal tree at some stages.
In the sense that, he took Brijesh's patch, then he did something with it(?) and then Ashish took it from him and then you took it from Ashish?
Yes, I think he rebased Ashish's tree on a newer tree and added his SoB on patches that required any conflict resolutions or changes on his end, so we kept those intact since then.
-Mike
Quoted 8 lines
This is how I'm reading this SOB chain at least...
Signature · 4 lines
Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Borislav Petkov
maintainerRe: [PATCH RFC v7 13/64] x86/cpufeatures: Add SEV-SNP CPU feature
Quoted 3 lines · Michael Roth
Yes, I think he rebased Ashish's tree on a newer tree and added his SoB on patches that required any conflict resolutions or changes on his end, so we kept those intact since then.
Ok, that makes sense.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRFC PATCH v7 14/64 x86/sev: Add the host SEV-SNP initialization support
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.
The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.
Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the RMP_BASE and RMP_END MSRs. If the RMP_BASE and RMP_END are not set then disable the SEV-SNP feature.
The SEV-SNP feature is enabled only after the RMP table is successfully initialized.
Also set SYSCFG.MFMD when enabling SNP as SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be se
RMP table entry format is non-architectural and it can vary by processor and is defined by the PPR. Restrict SNP support on the known CPU model and family for which the RMP table entry format is currently defined for.
Signed-off-b: Ashish Kalra <[email protected]>
--- arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/msr-index.h | 11 +- arch/x86/kernel/sev.c | 180 +++++++++++++++++++++++ 3 files changed, 197 insertions(+), 2 deletions(-)
arch/x86/include/asm/disabled-features.h +7 -1
@@ -87,6 +87,12 @@# define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31))#endif +#ifdef CONFIG_AMD_MEM_ENCRYPT +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif +/* * Make sure to add features to the correct mask */
@@ -110,7 +116,7 @@ DISABLE_ENQCMD)#define DISABLED_MASK17 0#define DISABLED_MASK18 0 -#define DISABLED_MASK19 0 +#define DISABLED_MASK19 (DISABLE_SEV_SNP)#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)#endif /* _ASM_X86_DISABLED_FEATURES_H */
arch/x86/include/asm/msr-index.h +10 -1
@@ -565,6 +565,8 @@#define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT)#define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)#define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +#define MSR_AMD64_RMP_BASE 0xc0010132 +#define MSR_AMD64_RMP_END 0xc0010133#define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f
@@ -649,7 +651,14 @@#define MSR_K8_TOP_MEM2 0xc001001d#define MSR_AMD64_SYSCFG 0xc0010010#define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT 23 -#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_SNP_EN_BIT 24 +#define MSR_AMD64_SYSCFG_SNP_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT) +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT 25 +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT) +#define MSR_AMD64_SYSCFG_MFDM_BIT 19 +#define MSR_AMD64_SYSCFG_MFDM BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT) +#define MSR_K8_INT_PENDING_MSG 0xc0010055/* C1E active bits in int pending message */ #define K8_INTP_C1E_ACTIVE_MASK 0x18000000
arch/x86/kernel/sev.c +180 -0
207 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 14/64] x86/sev: Add the host SEV-SNP initialization support
arch/x86/include/asm/disabled-features.h line 95
Quoted 6 lines · Michael Roth
arch/x86/include/asm/disabled-features.h +6
-0+#ifdef CONFIG_AMD_MEM_ENCRYPT +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif +
Would it make sense to split the SEV-* feature family into their own config flag(s) ?
I'm thinking in the context of SEV-SNP running on systems with Transparent SME enabled in the bios. In this case, enabling CONFIG_AMD_MEM_ENCRYPT will also enable SME in the kernel, which is a bit strange and not necessarily useful.
Commit 4e2c87949f2b ("crypto: ccp - When TSME and SME both detected notify user") highlights it.
--
Sabin.
Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
Kalra, Ashish
maintainerRe: [PATCH RFC v7 14/64] x86/sev: Add the host SEV-SNP initialization support
Quoted 19 lines · Sabin Rapan
arch/x86/include/asm/disabled-features.h line 95
Quoted 6 lines · Michael Roth
arch/x86/include/asm/disabled-features.h +6
-0+#ifdef CONFIG_AMD_MEM_ENCRYPT +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif +Would it make sense to split the SEV-* feature family into their own config flag(s) ?
I'm thinking in the context of SEV-SNP running on systems with Transparent SME enabled in the bios. In this case, enablingCONFIG_AMD_MEM_ENCRYPTwill also enable SME in the kernel, which is a bit strange and not necessarily useful.
Commit4e2c87949f2b("crypto: ccp - When TSME and SME both detected notify user") highlights it.
Yes, we plan to move the SNP host initialization stuff into a separate source file and under a different config flag such as CONFIG_KVM_AMD_SEV or something.
Thanks,
Ashish
Re: [PATCH RFC v7 14/64] x86/sev: Add the host SEV-SNP initialization support
In reply to Michael RothQuoted 199 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.
The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.
Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the
RMP_BASEandRMP_ENDMSRs. If theRMP_BASEandRMP_ENDare not set then disable the SEV-SNP feature.The SEV-SNP feature is enabled only after the RMP table is successfully initialized.
Also set SYSCFG.MFMD when enabling SNP as SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be se
RMP table entry format is non-architectural and it can vary by processor and is defined by the PPR. Restrict SNP support on the known CPU model and family for which the RMP table entry format is currently defined for.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-b: Ashish Kalra <[email protected]>
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/msr-index.h | 11 +- arch/x86/kernel/sev.c | 180 +++++++++++++++++++++++ 3 files changed, 197 insertions(+), 2 deletions(-)arch/x86/include/asm/disabled-features.h +7
-1@@ -87,6 +87,12 @@# define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31))#endif +#ifdef CONFIG_AMD_MEM_ENCRYPT +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif +/* * Make sure to add features to the correct mask */ @@ -110,7 +116,7 @@ DISABLE_ENQCMD)#define DISABLED_MASK17 0#define DISABLED_MASK18 0 -#define DISABLED_MASK19 0 +#define DISABLED_MASK19 (DISABLE_SEV_SNP)#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)#endif /* _ASM_X86_DISABLED_FEATURES_H */ arch/x86/include/asm/msr-index.h +10
-1@@ -565,6 +565,8 @@#define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT)#define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)#define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +#define MSR_AMD64_RMP_BASE 0xc0010132 +#define MSR_AMD64_RMP_END 0xc0010133#define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f@@ -649,7 +651,14 @@#define MSR_K8_TOP_MEM2 0xc001001d#define MSR_AMD64_SYSCFG 0xc0010010#define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT 23 -#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_SNP_EN_BIT 24 +#define MSR_AMD64_SYSCFG_SNP_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT) +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT 25 +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT) +#define MSR_AMD64_SYSCFG_MFDM_BIT 19 +#define MSR_AMD64_SYSCFG_MFDM BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT) +#define MSR_K8_INT_PENDING_MSG 0xc0010055/* C1E active bits in int pending message */ #define K8_INTP_C1E_ACTIVE_MASK 0x18000000arch/x86/kernel/sev.c +70
-0@@ -22,6 +22,9 @@#include <linux/efi.h> #include <linux/platform_device.h> #include <linux/io.h> +#include <linux/cpumask.h> +#include <linux/iommu.h> +#include <linux/amd-iommu.h> #include <asm/cpu_entry_area.h> #include <asm/stacktrace.h> @@ -38,6 +41,7 @@#include <asm/apic.h> #include <asm/cpuid.h> #include <asm/cmdline.h> +#include <asm/iommu.h> #define DR7_RESET_VALUE 0x400@@ -57,6 +61,12 @@#define AP_INIT_CR0_DEFAULT 0x60000010#define AP_INIT_MXCSR_DEFAULT 0x1f80 +/* +* The first 16KB from the RMP_BASE is used by the processor for the +* bookkeeping, the range needs to be added during the RMP entry lookup. +*/ +#define RMPTABLE_CPU_BOOKKEEPING_SZ 0x4000 +/* For early boot hypervisor communication in SEV-ES enabled guests */ static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);@@ -69,6 +79,9 @@ static struct ghcb *boot_ghcb __section(".data");/* Bitmap of SEV features supported by the hypervisor */ static u64 sev_hv_features__ro_after_init ; +static unsigned long rmptable_start__ro_after_init ; +static unsigned long rmptable_end__ro_after_init ; +/* #VC handler runtime per-CPU data */ struct sev_es_runtime_data {struct ghcb ghcb_page;@@ -2260,3 +2273,170 @@ static int __init snp_init_platform_device(void)return 0 ; }device_initcall (snp_init_platform_device); + +#undef pr_fmt +#define pr_fmt (fmt) "SEV-SNP: " fmt + +static int __mfd_enable (unsigned int cpu) +{ +u64 val; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return 0 ; + +rdmsrl (MSR_AMD64_SYSCFG , val); + + val |=MSR_AMD64_SYSCFG_MFDM ; + +wrmsrl (MSR_AMD64_SYSCFG , val); + +return 0 ; +} + +static __init void mfd_enable (void *arg) +{ +__mfd_enable (smp_processor_id ()); +} + +static int __snp_enable (unsigned int cpu) +{ +u64 val; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return 0 ; + +rdmsrl (MSR_AMD64_SYSCFG , val); + + val |=MSR_AMD64_SYSCFG_SNP_EN ; + val |=MSR_AMD64_SYSCFG_SNP_VMPL_EN ; + +wrmsrl (MSR_AMD64_SYSCFG , val); + +return 0 ; +} + +static __init void snp_enable (void *arg) +{ +__snp_enable (smp_processor_id ()); +} + +static bool get_rmptable_info (u64 *start,u64 *len) +{ +u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end; + +rdmsrl (MSR_AMD64_RMP_BASE , rmp_base); +rdmsrl (MSR_AMD64_RMP_END , rmp_end); + +if (!rmp_base || !rmp_end) { +pr_err ("Memory for the RMP table has not been reserved by BIOS\n" ); +return false;
arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2342
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+ } + + rmp_sz = rmp_end - rmp_base +1 ; + +/* +* Calculate the amount the memory that must be reserved by the BIOS to +* address the whole RAM. The reserved memory should also cover the +* RMP table itself. +*/ + calc_rmp_sz = (((rmp_sz >>PAGE_SHIFT ) +totalram_pages ()) <<4 ) +RMPTABLE_CPU_BOOKKEEPING_SZ ;
Since the rmptable is indexed by page number, I believe this check should be using max_pfn:
calc_rmp_sz = (max_pfn << 4) + RMPTABLE_CPU_BOOKKEEPING_SZ;
This accounts for holes/offsets in the memory map which lead to the top of memory having pfn > totalram_pages().
Quoted 103 lines
arch/x86/kernel/sev.c +100
-0+ +if (calc_rmp_sz > rmp_sz) { +pr_err ("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n" , + calc_rmp_sz, rmp_sz); +return false; + } + + *start = rmp_base; + *len = rmp_sz; + +pr_info ("RMP table physical address [0x%016llx - 0x%016llx]\n" , rmp_base, rmp_end); + +return true; +} + +static __init int __snp_rmptable_init (void ) +{ +u64 rmp_base, sz; +void *start; +u64 val; + +if (!get_rmptable_info (&rmp_base, &sz)) +return 1 ; + + start =memremap (rmp_base, sz,MEMREMAP_WB ); +if (!start) { +pr_err ("Failed to map RMP table addr 0x%llx size 0x%llx\n" , rmp_base, sz); +return 1 ; + } + +/* +* Check if SEV-SNP is already enabled, this can happen in case of +* kexec boot. +*/ +rdmsrl (MSR_AMD64_SYSCFG , val); +if (val &MSR_AMD64_SYSCFG_SNP_EN ) + goto skip_enable; + +/* Initialize the RMP table to zero */ +memset (start,0 , sz); + +/* Flush the caches to ensure that data is written before SNP is enabled. */ +wbinvd_on_all_cpus (); + +/* MFDM must be enabled on all the CPUs prior to enabling SNP. */ +on_each_cpu (mfd_enable,NULL ,1 ); + +/* Enable SNP on all CPUs. */ +on_each_cpu (snp_enable,NULL ,1 ); + +skip_enable: + rmptable_start = (unsigned long )start; + rmptable_end = rmptable_start + sz -1 ; + +return 0 ; +} + +static int __init snp_rmptable_init (void ) +{ +int family, model; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return 0 ; + + family = boot_cpu_data.x86; + model = boot_cpu_data.x86_model; + +/* +* RMP table entry format is not architectural and it can vary by processor and +* is defined by the per-processor PPR. Restrict SNP support on the known CPU +* model and family for which the RMP table entry format is currently defined for. +*/ +if (family !=0x19 || model >0xaf ) + goto nosnp; + +if (amd_iommu_snp_enable ()) + goto nosnp; + +if (__snp_rmptable_init ()) + goto nosnp; + +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/rmptable_init:online" , __snp_enable,NULL ); + +return 0 ; + +nosnp: +setup_clear_cpu_cap (X86_FEATURE_SEV_SNP ); +return -ENOSYS ; +} + +/* +* This must be called after the PCI subsystem. This is because amd_iommu_snp_enable() +* is called to ensure the IOMMU supports the SEV-SNP feature, which can only be +* called after subsys_initcall(). +* +* NOTE: IOMMU is enforced by SNP to ensure that hypervisor cannot program DMA +* directly into guest private memory. In case of SNP, the IOMMU ensures that +* the page(s) used for DMA are hypervisor owned. +*/ +fs_initcall (snp_rmptable_init);Signature · 1 line
2.25.1
Kalra, Ashish
maintainerRe: [PATCH RFC v7 14/64] x86/sev: Add the host SEV-SNP initialization support
Hello Jeremi,
Quoted 214 lines · Jeremi Piotrowski
Quoted 199 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.
The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.
Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the
RMP_BASEandRMP_ENDMSRs. If theRMP_BASEandRMP_ENDare not set then disable the SEV-SNP feature.The SEV-SNP feature is enabled only after the RMP table is successfully initialized.
Also set SYSCFG.MFMD when enabling SNP as SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be se
RMP table entry format is non-architectural and it can vary by processor and is defined by the PPR. Restrict SNP support on the known CPU model and family for which the RMP table entry format is currently defined for.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-b: Ashish Kalra <[email protected]>
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/msr-index.h | 11 +- arch/x86/kernel/sev.c | 180 +++++++++++++++++++++++ 3 files changed, 197 insertions(+), 2 deletions(-)arch/x86/include/asm/disabled-features.h +7
-1@@ -87,6 +87,12 @@# define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31))#endif +#ifdef CONFIG_AMD_MEM_ENCRYPT +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif +/* * Make sure to add features to the correct mask */ @@ -110,7 +116,7 @@ DISABLE_ENQCMD)#define DISABLED_MASK17 0#define DISABLED_MASK18 0 -#define DISABLED_MASK19 0 +#define DISABLED_MASK19 (DISABLE_SEV_SNP)#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)#endif /* _ASM_X86_DISABLED_FEATURES_H */ arch/x86/include/asm/msr-index.h +10
-1@@ -565,6 +565,8 @@#define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT)#define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT)#define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +#define MSR_AMD64_RMP_BASE 0xc0010132 +#define MSR_AMD64_RMP_END 0xc0010133#define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f@@ -649,7 +651,14 @@#define MSR_K8_TOP_MEM2 0xc001001d#define MSR_AMD64_SYSCFG 0xc0010010#define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT 23 -#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_SNP_EN_BIT 24 +#define MSR_AMD64_SYSCFG_SNP_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT) +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT 25 +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT) +#define MSR_AMD64_SYSCFG_MFDM_BIT 19 +#define MSR_AMD64_SYSCFG_MFDM BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT) +#define MSR_K8_INT_PENDING_MSG 0xc0010055/* C1E active bits in int pending message */ #define K8_INTP_C1E_ACTIVE_MASK 0x18000000arch/x86/kernel/sev.c +70
-0@@ -22,6 +22,9 @@#include <linux/efi.h> #include <linux/platform_device.h> #include <linux/io.h> +#include <linux/cpumask.h> +#include <linux/iommu.h> +#include <linux/amd-iommu.h> #include <asm/cpu_entry_area.h> #include <asm/stacktrace.h> @@ -38,6 +41,7 @@#include <asm/apic.h> #include <asm/cpuid.h> #include <asm/cmdline.h> +#include <asm/iommu.h> #define DR7_RESET_VALUE 0x400@@ -57,6 +61,12 @@#define AP_INIT_CR0_DEFAULT 0x60000010#define AP_INIT_MXCSR_DEFAULT 0x1f80 +/* +* The first 16KB from the RMP_BASE is used by the processor for the +* bookkeeping, the range needs to be added during the RMP entry lookup. +*/ +#define RMPTABLE_CPU_BOOKKEEPING_SZ 0x4000 +/* For early boot hypervisor communication in SEV-ES enabled guests */ static struct ghcb boot_ghcb_page __bss_decrypted __aligned(PAGE_SIZE);@@ -69,6 +79,9 @@ static struct ghcb *boot_ghcb __section(".data");/* Bitmap of SEV features supported by the hypervisor */ static u64 sev_hv_features__ro_after_init ; +static unsigned long rmptable_start__ro_after_init ; +static unsigned long rmptable_end__ro_after_init ; +/* #VC handler runtime per-CPU data */ struct sev_es_runtime_data {struct ghcb ghcb_page;@@ -2260,3 +2273,170 @@ static int __init snp_init_platform_device(void)return 0 ; }device_initcall (snp_init_platform_device); + +#undef pr_fmt +#define pr_fmt (fmt) "SEV-SNP: " fmt + +static int __mfd_enable (unsigned int cpu) +{ +u64 val; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return 0 ; + +rdmsrl (MSR_AMD64_SYSCFG , val); + + val |=MSR_AMD64_SYSCFG_MFDM ; + +wrmsrl (MSR_AMD64_SYSCFG , val); + +return 0 ; +} + +static __init void mfd_enable (void *arg) +{ +__mfd_enable (smp_processor_id ()); +} + +static int __snp_enable (unsigned int cpu) +{ +u64 val; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return 0 ; + +rdmsrl (MSR_AMD64_SYSCFG , val); + + val |=MSR_AMD64_SYSCFG_SNP_EN ; + val |=MSR_AMD64_SYSCFG_SNP_VMPL_EN ; + +wrmsrl (MSR_AMD64_SYSCFG , val); + +return 0 ; +} + +static __init void snp_enable (void *arg) +{ +__snp_enable (smp_processor_id ()); +} + +static bool get_rmptable_info (u64 *start,u64 *len) +{ +u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end; + +rdmsrl (MSR_AMD64_RMP_BASE , rmp_base); +rdmsrl (MSR_AMD64_RMP_END , rmp_end); + +if (!rmp_base || !rmp_end) { +pr_err ("Memory for the RMP table has not been reserved by BIOS\n" ); +return false;arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2342
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+ } + + rmp_sz = rmp_end - rmp_base +1 ; + +/* +* Calculate the amount the memory that must be reserved by the BIOS to +* address the whole RAM. The reserved memory should also cover the +* RMP table itself. +*/ + calc_rmp_sz = (((rmp_sz >>PAGE_SHIFT ) +totalram_pages ()) <<4 ) +RMPTABLE_CPU_BOOKKEEPING_SZ ;Since the rmptable is indexed by page number, I believe this check should be using
max_pfn:
Quoted 5 lines
calc_rmp_sz = (max_pfn << 4) + RMPTABLE_CPU_BOOKKEEPING_SZ;This accounts for holes/offsets in the memory map which lead to the top of memory having pfn >
totalram_pages().
I agree that this check should use max. addressable pfn to account for holes in the physical address map. The BIOS will probably also be computing RMP table size to cover the entire physical memory, which should be max. addressable PFN.
But, then we primarly need to check that all available RAM pages are covered by the RMP table, so the above check is sufficient for that, right ?
Also, i assume that max_pfn will take into account any hotplugged memory as i do know that totalram_pages() handles hotplugged memory.
Thanks,
Ashish
Quoted 15 lines
Quoted 15 lines
arch/x86/kernel/sev.c +15
-0+ +if (calc_rmp_sz > rmp_sz) { +pr_err ("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n" , + calc_rmp_sz, rmp_sz); +return false; + } + + *start = rmp_base; + *len = rmp_sz; + +pr_info ("RMP table physical address [0x%016llx - 0x%016llx]\n" , rmp_base, rmp_end); + +return true; +} +
Kalra, Ashish
maintainerRe: [PATCH RFC v7 14/64] x86/sev: Add the host SEV-SNP initialization support
Quoted 222 lines · Ashish
Hello Jeremi,
Quoted 213 lines · Jeremi Piotrowski
Quoted 202 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.
The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.
Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the
RMP_BASEandRMP_ENDMSRs. If theRMP_BASEandRMP_ENDare not set then disable the SEV-SNP feature.The SEV-SNP feature is enabled only after the RMP table is successfully initialized.
Also set SYSCFG.MFMD when enabling SNP as SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be se
RMP table entry format is non-architectural and it can vary by processor and is defined by the PPR. Restrict SNP support on the known CPU model and family for which the RMP table entry format is currently defined for.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-b: Ashish Kalra <[email protected]>
Signed-off-by Michael Roth <[email protected]>---
arch/x86/include/asm/disabled-features.h| 8 +- arch/x86/include/asm/msr-index.h | 11 +- arch/x86/kernel/sev.c | 180 +++++++++++++++++++++++3 files changed, 197 insertions(+), 2 deletions(-)arch/x86/include/asm/disabled-features.h +7
-1@@ -87,6 +87,12 @@ # define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31)) #endif +#ifdef CONFIG_AMD_MEM_ENCRYPT +# define DISABLE_SEV_SNP 0 +#else +# define DISABLE_SEV_SNP (1 << (X86_FEATURE_SEV_SNP & 31)) +#endif + /** Make sure to add features to the correct mask */ @@ -110,7 +116,7 @@ DISABLE_ENQCMD) #define DISABLED_MASK17 0 #define DISABLED_MASK18 0 -#define DISABLED_MASK19 0 +#define DISABLED_MASK19 (DISABLE_SEV_SNP) #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20) #endif /* _ASM_X86_DISABLED_FEATURES_H */arch/x86/include/asm/msr-index.h +7
-1@@ -565,6 +565,8 @@ #define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT) #define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT) #define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +#define MSR_AMD64_RMP_BASE 0xc0010132 +#define MSR_AMD64_RMP_END 0xc0010133 #define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f@@ -649,7 +651,14 @@ #define MSR_K8_TOP_MEM2 0xc001001d #define MSR_AMD64_SYSCFG 0xc0010010 #define MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT 23 -#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_MEM_ENCRYPT BIT_ULL(MSR_AMD64_SYSCFG_MEM_ENCRYPT_BIT) +#define MSR_AMD64_SYSCFG_SNP_EN_BIT 24 +#define MSR_AMD64_SYSCFG_SNP_EN BIT_ULL(MSR_AMD64_SYSCFG_SNP_EN_BIT) +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT 25 +#define MSR_AMD64_SYSCFG_SNP_VMPL_EN
BIT_ULL(MSR_AMD64_SYSCFG_SNP_VMPL_EN_BIT)arch/x86/include/asm/msr-index.h +2
-0+#define MSR_AMD64_SYSCFG_MFDM_BIT 19 +#define MSR_AMD64_SYSCFG_MFDM
BIT_ULL(MSR_AMD64_SYSCFG_MFDM_BIT)
+
#define MSR_K8_INT_PENDING_MSG0xc0010055/* C1E active bits in int pending message */ #define K8_INTP_C1E_ACTIVE_MASK0x18000000arch/x86/kernel/sev.c +7
-0@@ -22,6 +22,9 @@ #include <linux/efi.h> #include <linux/platform_device.h> #include <linux/io.h> +#include <linux/cpumask.h> +#include <linux/iommu.h> +#include <linux/amd-iommu.h> #include <asm/cpu_entry_area.h> #include <asm/stacktrace.h>@@ -38,6 +41,7 @@ #include <asm/apic.h> #include <asm/cpuid.h> #include <asm/cmdline.h> +#include <asm/iommu.h> #define DR7_RESET_VALUE 0x400@@ -57,6 +61,12 @@ #define AP_INIT_CR0_DEFAULT 0x60000010 #define AP_INIT_MXCSR_DEFAULT 0x1f80 +/* +* The first 16KB from the RMP_BASE is used by the processor for the +* bookkeeping, the range needs to be added during the RMP entry lookup.
+ */arch/x86/kernel/sev.c +2
-0+#define RMPTABLE_CPU_BOOKKEEPING_SZ 0x4000 +/* For early boot hypervisor communication in SEV-ES enabled guests */ static struct ghcb
boot_ghcb_page__bss_decrypted__aligned(PAGE_SIZE);arch/x86/kernel/sev.c +59
-0@@ -69,6 +79,9 @@ static struct ghcb *boot_ghcb __section(".data"); /* Bitmap of SEV features supported by the hypervisor */ static u64 sev_hv_features __ro_after_init; +static unsigned long rmptable_start__ro_after_init ; +static unsigned long rmptable_end__ro_after_init ; + /* #VC handler runtime per-CPU data */ struct sev_es_runtime_data { struct ghcb ghcb_page;@@ -2260,3 +2273,170 @@ static int __init snp_init_platform_device(void) return 0; } device_initcall(snp_init_platform_device); + +#undef pr_fmt +#define pr_fmt (fmt) "SEV-SNP: " fmt + +static int __mfd_enable (unsigned int cpu) +{ + u64 val; + + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) + return 0; + + rdmsrl(MSR_AMD64_SYSCFG, val); + + val |= MSR_AMD64_SYSCFG_MFDM; + + wrmsrl(MSR_AMD64_SYSCFG, val); + + return 0; +} + +static __init void mfd_enable (void *arg) +{ + __mfd_enable(smp_processor_id()); +} + +static int __snp_enable (unsigned int cpu) +{ + u64 val; + + if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP)) + return 0; + + rdmsrl(MSR_AMD64_SYSCFG, val); + + val |= MSR_AMD64_SYSCFG_SNP_EN; + val |= MSR_AMD64_SYSCFG_SNP_VMPL_EN; + + wrmsrl(MSR_AMD64_SYSCFG, val); + + return 0; +} + +static __init void snp_enable (void *arg) +{ + __snp_enable(smp_processor_id()); +} + +static bool get_rmptable_info (u64 *start,u64 *len) +{ + u64 calc_rmp_sz, rmp_sz, rmp_base, rmp_end; + + rdmsrl(MSR_AMD64_RMP_BASE, rmp_base); + rdmsrl(MSR_AMD64_RMP_END, rmp_end); + + if (!rmp_base || !rmp_end) { + pr_err("Memory for the RMP table has not been reserved byBIOS\n");
+ return false;
+ }arch/x86/kernel/sev.c +5
-0+ + rmp_sz = rmp_end - rmp_base + 1; + + /* +* Calculate the amount the memory that must be reserved by the Quoted 6 lines
BIOS to
+ * address the whole RAM. The reserved memory should also cover the + * RMP table itself.
+ */
+calc_rmp_sz= (((rmp_sz>>PAGE_SHIFT) +totalram_pages()) << 4)
+RMPTABLE_CPU_BOOKKEEPING_SZ;Since the rmptable is indexed by page number, I believe this check should be using
max_pfn:Quoted 5 lines
calc_rmp_sz= (max_pfn<< 4) +RMPTABLE_CPU_BOOKKEEPING_SZ;This accounts for holes/offsets in the memory map which lead to the top of memory having pfn >
totalram_pages().
Quoted 6 lines
I agree that this check should use max. addressable pfn to account for holes in the physical address map. The BIOS will probably also be computing RMP table size to cover the entire physical memory, which should be max. addressable PFN.
But, then we primarly need to check that all available RAM pages are covered by the RMP table, so the above check is sufficient for that, right ?
Also, i assume that
max_pfnwill take into account any hotplugged memory as i do know thattotalram_pages()handles hotplugged memory.
But essentially you are correct, as RMP table is indexed by PFN, we have to take into account these physical memory holes so that we can have entries for the max DRAM SPA, so i will fix this to use the max. addressable PFN, i.e., max_pfn.
Thanks,
Ashish
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
+
+ if (calc_rmp_sz>rmp_sz) {
+ pr_err("Memory reserved for the RMP table does not cover full system RAM (expected 0x%llx got 0x%llx)\n",
+calc_rmp_sz,rmp_sz);
+ return false;
+ }arch/x86/kernel/sev.c +5
-0+ + *start = rmp_base; + *len = rmp_sz; + + pr_info("RMP table physical address [0x%016llx - 0x%016llx]\n",
rmp_base,rmp_end);arch/x86/kernel/sev.c +4
-0+ + return true; +} +
Borislav Petkov
maintainerRe: [PATCH RFC v7 14/64] x86/sev: Add the host SEV-SNP initialization support
In reply to Michael RothQuoted 19 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The memory integrity guarantees of SEV-SNP are enforced through a new structure called the Reverse Map Table (RMP). The RMP is a single data structure shared across the system that contains one entry for every 4K page of DRAM that may be used by SEV-SNP VMs. The goal of RMP is to track the owner of each page of memory. Pages of memory can be owned by the hypervisor, owned by a specific VM or owned by the AMD-SP. See APM2 section 15.36.3 for more detail on RMP.
The RMP table is used to enforce access control to memory. The table itself is not directly writable by the software. New CPU instructions (RMPUPDATE, PVALIDATE, RMPADJUST) are used to manipulate the RMP entries.
Based on the platform configuration, the BIOS reserves the memory used for the RMP table. The start and end address of the RMP table must be queried by reading the
RMP_BASEandRMP_ENDMSRs. If theRMP_BASEandRMP_ENDare not set then disable the SEV-SNP feature.
Quoted 5 lines
The SEV-SNP feature is enabled only after the RMP table is successfully initialized.
Also set SYSCFG.MFMD when enabling SNP as SEV-SNP FW >= 1.51 requires that SYSCFG.MFMD must be se
set.
Quoted 7 lines
RMP table entry format is non-architectural and it can vary by processor and is defined by the PPR. Restrict SNP support on the known CPU model and family for which the RMP table entry format is currently defined for.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-b: Ashish Kalra <[email protected]>
^^
Somebody ate a 'y' here. :)
Quoted 6 lines
Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/msr-index.h | 11 +- arch/x86/kernel/sev.c | 180 +++++++++++++++++++++++ 3 files changed, 197 insertions(+), 2 deletions(-)
...
Quoted 1 line
+static
__initint__snp_rmptable_init(void)
Why is this one carved out of snp_rmptable_init() ?
Quoted 13 lines
arch/x86/kernel/sev.c +13
-0+{ +u64 rmp_base, sz; +void *start; +u64 val; + +if (!get_rmptable_info (&rmp_base, &sz)) +return 1 ; + + start =memremap (rmp_base, sz,MEMREMAP_WB ); +if (!start) { +pr_err ("Failed to map RMP table addr 0x%llx size 0x%llx\n" , rmp_base, sz); +return 1 ; + }
arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2381
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+ +/* +* Check if SEV-SNP is already enabled, this can happen in case of +* kexec boot. +*/ +rdmsrl (MSR_AMD64_SYSCFG , val); +if (val &MSR_AMD64_SYSCFG_SNP_EN ) + goto skip_enable; + +/* Initialize the RMP table to zero */
Useless comment.
arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2391
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+memset (start,0 , sz); + +/* Flush the caches to ensure that data is written before SNP is enabled. */ +wbinvd_on_all_cpus (); + +/* MFDM must be enabled on all the CPUs prior to enabling SNP. */ +on_each_cpu (mfd_enable,NULL ,1 ); + +/* Enable SNP on all CPUs. */ +on_each_cpu (snp_enable,NULL ,1 );
What happens if someone boots the machine with maxcpus=N, where N is less than all CPUs on the machine? The hotplug notifier should handle it but have you checked that it works fine?
Quoted 6 lines
arch/x86/kernel/sev.c +6
-0+skip_enable: + rmptable_start = (unsigned long )start; + rmptable_end = rmptable_start + sz -1 ; + +return 0 ; +}
arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2408
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+ +static int __init snp_rmptable_init (void ) +{ +int family, model; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return 0 ; + + family = boot_cpu_data.x86; + model = boot_cpu_data.x86_model;
Looks useless - just use boot_cpu_data directly below.
Quoted 13 lines
arch/x86/kernel/sev.c +13
-0+ +/* +* RMP table entry format is not architectural and it can vary by processor and +* is defined by the per-processor PPR. Restrict SNP support on the known CPU +* model and family for which the RMP table entry format is currently defined for. +*/ +if (family !=0x19 || model >0xaf ) + goto nosnp; + +if (amd_iommu_snp_enable ()) + goto nosnp; + +if (__snp_rmptable_init ())
arch/x86/kernel/sev.c static int __init snp_init_platform_device(void) · line 2431
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+ goto nosnp; + +cpuhp_setup_state (CPUHP_AP_ONLINE_DYN ,"x86/rmptable_init:online" , __snp_enable,NULL ); + +return 0 ; + +nosnp: +setup_clear_cpu_cap (X86_FEATURE_SEV_SNP ); +return -ENOSYS ; +}
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRFC PATCH v7 15/64 x86/sev: Add RMP entry lookup helpers
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
The snp_lookup_rmpentry() can be used by the host to read the RMP entry for a given page. The RMP entry format is documented in AMD PPR, see
https://bugzilla.kernel.org/attachment.cgi?id=296015.
--- arch/x86/include/asm/sev.h | 4 +- arch/x86/kernel/sev.c | 83 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 86 insertions(+), 1 deletion(-)
arch/x86/include/asm/sev.h +3 -1
@@ -83,7 +83,7 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);/* RMP page size */ #define RMP_PG_SIZE_4K 0 - +#define RMP_TO_X86_PG_LEVEL (level) (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M)#define RMPADJUST_VMSA_PAGE_BIT BIT(16)/* SNP Guest message request */
@@ -197,6 +197,7 @@ void snp_set_wakeup_secondary_cpu(void);bool snp_init (struct boot_params *bp);void __init __noreturn snp_abort (void );int snp_issue_guest_request (u64 exit_code,struct snp_req_data *input,unsigned long *fw_err); +int snp_lookup_rmpentry (u64 pfn,int *level);#else static inline void sev_es_ist_enter (struct pt_regs *regs) { }static inline void sev_es_ist_exit (void ) { }
@@ -221,6 +222,7 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *in {return -ENOTTY ; } +static inline int snp_lookup_rmpentry (u64 pfn,int *level) {return 0 ; }#endif #endif
arch/x86/kernel/sev.c +83 -0
97 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v7 16/64 x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.
The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.
Add the following external interface API functions:
int psmash(u64 pfn);
psmash is used to smash a 2MB aligned page into 4K pages while preserving the Validated bit in the RMP.
int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable); Used to assign a page to guest using the RMPUPDATE instruction.
int rmp_make_shared(u64 pfn, enum pg_level level);
Used to transition a page to hypervisor/shared state using the RMPUPDATE instruction.
--- arch/x86/include/asm/sev.h | 24 ++++++++++ arch/x86/kernel/sev.c | 95 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+)
arch/x86/include/asm/sev.h +24 -0
@@ -80,10 +80,15 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);/* Software defined (when rFlags.CF = 1) */ #define PVALIDATE_FAIL_NOUPDATE 255 +/* RMUPDATE detected 4K page and 2MB page overlap. */ +#define RMPUPDATE_FAIL_OVERLAP 7/* RMP page size */ #define RMP_PG_SIZE_4K 0 +#define RMP_PG_SIZE_2M 1#define RMP_TO_X86_PG_LEVEL (level) (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M) +#define X86_TO_RMP_PG_LEVEL (level) (((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M) +#define RMPADJUST_VMSA_PAGE_BIT BIT(16)/* SNP Guest message request */
@@ -133,6 +138,15 @@ struct snp_secrets_page_layout {u8 rsvd3[3840 ]; } __packed; +struct rmp_state { +u64 gpa; +u8 assigned; +u8 pagesize; +u8 immutable; +u8 rsvd; +u32 asid; +} __packed; +#ifdef CONFIG_AMD_MEM_ENCRYPT extern struct static_key_false sev_es_enable_key;extern void __sev_es_ist_enter (struct pt_regs *regs);
@@ -198,6 +212,9 @@ bool snp_init(struct boot_params *bp);void __init __noreturn snp_abort (void );int snp_issue_guest_request (u64 exit_code,struct snp_req_data *input,unsigned long *fw_err);int snp_lookup_rmpentry (u64 pfn,int *level); +int psmash (u64 pfn); +int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid,bool immutable); +int rmp_make_shared (u64 pfn,enum pg_level level);#else static inline void sev_es_ist_enter (struct pt_regs *regs) { }static inline void sev_es_ist_exit (void ) { }
@@ -223,6 +240,13 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *inreturn -ENOTTY ; }static inline int snp_lookup_rmpentry (u64 pfn,int *level) {return 0 ; } +static inline int psmash (u64 pfn) {return -ENXIO ; } +static inline int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid, +bool immutable) +{ +return -ENODEV ; +} +static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return -ENODEV ; }#endif #endif
arch/x86/kernel/sev.c +95 -0
98 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 16/64] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
Quoted 102 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.
The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.
Add the following external interface API functions:
int psmash(u64 pfn);
psmash is used to smash a 2MB aligned page into 4K pages while preserving the Validated bit in the RMP.int
rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable); Used to assign a page to guest using the RMPUPDATE instruction.int
rmp_make_shared(u64 pfn, enum pg_level level);
Used to transition a page to hypervisor/shared state using the RMPUPDATE instruction.Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>[mdr: add RMPUPDATE retry logic for transient FAIL_OVERLAP errors]Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/sev.h | 24 ++++++++++ arch/x86/kernel/sev.c | 95 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+)arch/x86/include/asm/sev.h +24
-0@@ -80,10 +80,15 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs);/* Software defined (when rFlags.CF = 1) */ #define PVALIDATE_FAIL_NOUPDATE 255 +/* RMUPDATE detected 4K page and 2MB page overlap. */ +#define RMPUPDATE_FAIL_OVERLAP 7/* RMP page size */ #define RMP_PG_SIZE_4K 0 +#define RMP_PG_SIZE_2M 1#define RMP_TO_X86_PG_LEVEL (level) (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M) +#define X86_TO_RMP_PG_LEVEL (level) (((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M) +#define RMPADJUST_VMSA_PAGE_BIT BIT(16)/* SNP Guest message request */ @@ -133,6 +138,15 @@ struct snp_secrets_page_layout {u8 rsvd3[3840 ]; } __packed; +struct rmp_state { +u64 gpa; +u8 assigned; +u8 pagesize; +u8 immutable; +u8 rsvd; +u32 asid; +} __packed; +#ifdef CONFIG_AMD_MEM_ENCRYPT extern struct static_key_false sev_es_enable_key;extern void __sev_es_ist_enter (struct pt_regs *regs);@@ -198,6 +212,9 @@ bool snp_init(struct boot_params *bp);void __init __noreturn snp_abort (void );int snp_issue_guest_request (u64 exit_code,struct snp_req_data *input,unsigned long *fw_err);int snp_lookup_rmpentry (u64 pfn,int *level); +int psmash (u64 pfn); +int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid,bool immutable); +int rmp_make_shared (u64 pfn,enum pg_level level);#else static inline void sev_es_ist_enter (struct pt_regs *regs) { }static inline void sev_es_ist_exit (void ) { }@@ -223,6 +240,13 @@ static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *inreturn -ENOTTY ; }static inline int snp_lookup_rmpentry (u64 pfn,int *level) {return 0 ; } +static inline int psmash (u64 pfn) {return -ENXIO ; } +static inline int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid, +bool immutable) +{ +return -ENODEV ; +} +static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return -ENODEV ; }#endif #endif arch/x86/kernel/sev.c +2
-0@@ -2523,3 +2523,98 @@ int snp_lookup_rmpentry(u64 pfn, int *level)return !!rmpentry_assigned (e); }EXPORT_SYMBOL_GPL (snp_lookup_rmpentry); + +/*
arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2537
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+* psmash is used to smash a 2MB aligned page into 4K +* pages while preserving the Validated bit in the RMP. +*/ +int psmash (u64 pfn) +{ +unsigned long paddr = pfn <<PAGE_SHIFT ; +int ret; + +if (!pfn_valid (pfn)) +return -EINVAL ;
We (and many other clouds) use a neat trick to reduce the number of struct pages Linux allocates for guest memory: In its simplest form, add mem= to the kernel cmdline and mmap() /dev/mem to access the reserved memory instead.
This means that the system covers more RAM than Linux contains, which means pfn_valid() is no longer a good indication whether a page is indeed valid. KVM handles this case fine, but this code does not.
Is there any particular reason why we need this check (and similar ones below and in other RMP related patches) in the first place? I would expect that PSMASH and friends return failure codes for invalid pfns.
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Kalra, Ashish
maintainerRe: [PATCH RFC v7 16/64] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
Quoted 124 lines · Alexander Graf
Quoted 108 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.
The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.
Add the following external interface API functions:
int psmash(u64 pfn);
psmash is used to smash a 2MB aligned page into 4K pages while preserving the Validated bit in the RMP.int
rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable); Used to assign a page to guest using the RMPUPDATE instruction.int
rmp_make_shared(u64 pfn, enum pg_level level);
Used to transition a page to hypervisor/shared state using the RMPUPDATE instruction.Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>[mdr: add RMPUPDATE retry logic for transient FAIL_OVERLAP errors]Signed-off-by Michael Roth <[email protected]>---
arch/x86/include/asm/sev.h| 24 ++++++++++ arch/x86/kernel/sev.c | 95 ++++++++++++++++++++++++++++++++++++++2 files changed, 119 insertions(+)arch/x86/include/asm/sev.h +5
-0@@ -80,10 +80,15 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); /* Software defined (when rFlags.CF = 1) */ #define PVALIDATE_FAIL_NOUPDATE 255 +/* RMUPDATE detected 4K page and 2MB page overlap. */ +#define RMPUPDATE_FAIL_OVERLAP 7 /* RMP page size */ #define RMP_PG_SIZE_4K 0 +#define RMP_PG_SIZE_2M 1 #define RMP_TO_X86_PG_LEVEL(level) (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M) +#define X86_TO_RMP_PG_LEVEL (level) (((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M) + #define RMPADJUST_VMSA_PAGE_BIT BIT(16)/* SNP Guest message request */
arch/x86/include/asm/sev.h +12
-0@@ -133,6 +138,15 @@ struct snp_secrets_page_layout { u8 rsvd3[3840]; } __packed; +struct rmp_state { + u64 gpa; + u8 assigned; + u8 pagesize; + u8 immutable; + u8 rsvd; + u32 asid; +} __packed; + #ifdef CONFIG_AMD_MEM_ENCRYPT extern struct static_key_false sev_es_enable_key; extern void __sev_es_ist_enter(struct pt_regs *regs);@@ -198,6 +212,9 @@ bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err); int snp_lookup_rmpentry(u64 pfn, int *level); +int psmash (u64 pfn); +int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid,bool immutable); +int rmp_make_shared (u64 pfn,enum pg_level level); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { }static inline void
sev_es_ist_exit(void){ }arch/x86/include/asm/sev.h +0
-0@@ -223,6 +240,13 @@ static inline int snp_issue_guest_request(u64
exit_code, structsnp_req_data*in
return-ENOTTY;
}
static inline intsnp_lookup_rmpentry(u64 pfn, int *level){ return 0; }arch/x86/include/asm/sev.h +2
-0+static inline int psmash (u64 pfn) {return -ENXIO ; } +static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_levellevel, int asid,
+ bool immutable)arch/x86/include/asm/sev.h +4
-0+{ + return -ENODEV; +} +static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return
-ENODEV; }
#endif#endif
arch/x86/kernel/sev.c +2
-0@@ -2523,3 +2523,98 @@ int snp_lookup_rmpentry(u64 pfn, int *level) return !!rmpentry_assigned(e); } EXPORT_SYMBOL_GPL(snp_lookup_rmpentry); + +/* arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2537
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+* psmash is used to smash a 2MB aligned page into 4K +* pages while preserving the Validated bit in the RMP. +*/ +int psmash (u64 pfn) +{ + unsigned long paddr = pfn << PAGE_SHIFT; + int ret; + + if (!pfn_valid(pfn)) + return -EINVAL;We (and many other clouds) use a neat trick to reduce the number of struct pages Linux allocates for guest memory: In its simplest form, add mem= to the kernel cmdline and
mmap()/dev/memto access the reserved memory instead.
Quoted 1 line
This means that the system covers more RAM than Linux contains, which means
pfn_valid()is no longer a good indication whether a page is indeed valid. KVM handles this case fine, but this code does not.
Hmm...but then is also using max_pfn reliable ?
Quoted 3 lines
Is there any particular reason why we need this check (and similar ones below and in other RMP related patches) in the first place. I would > expect that PSMASH and friends return failure codes for invalid pfns.
Yes, PSMASH does out of bounds check on the input SPA and additionally checks if SPA is 2M aligned, so guess we can rely on using PSMASH failing on invalid pfns.
Thanks,
Ashish
Re: [PATCH RFC v7 16/64] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
Quoted 135 lines · Ashish
Quoted 133 lines · Alexander Graf
Quoted 114 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.
The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.
Add the following external interface API functions:
int psmash(u64 pfn);
psmash is used to smash a 2MB aligned page into 4K pages while preserving the Validated bit in the RMP.int
rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
Used to assign a page to guest using the RMPUPDATE instruction.int
rmp_make_shared(u64 pfn, enum pg_level level); Used to transition a page to hypervisor/shared state using the RMPUPDATE instruction.Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>[mdr: add RMPUPDATE retry logic for transient FAIL_OVERLAP errors]Signed-off-by Michael Roth <[email protected]>---
arch/x86/include/asm/sev.h| 24 ++++++++++
arch/x86/kernel/sev.c | 95
++++++++++++++++++++++++++++++++++++++2 files changed, 119 insertions(+)arch/x86/include/asm/sev.h +5
-0@@ -80,10 +80,15 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); /* Software defined (when rFlags.CF = 1) */ #define PVALIDATE_FAIL_NOUPDATE 255 +/* RMUPDATE detected 4K page and 2MB page overlap. */ +#define RMPUPDATE_FAIL_OVERLAP 7 /* RMP page size */ #define RMP_PG_SIZE_4K 0 +#define RMP_PG_SIZE_2M 1 #define RMP_TO_X86_PG_LEVEL(level) (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M) +#define X86_TO_RMP_PG_LEVEL (level) (((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M) +#define RMPADJUST_VMSA_PAGE_BIT BIT(16)
/* SNP Guest message request */
arch/x86/include/asm/sev.h +9
-0@@ -133,6 +138,15 @@ struct snp_secrets_page_layout { u8 rsvd3[3840]; } __packed; +struct rmp_state { + u64 gpa; + u8 assigned; + u8 pagesize; + u8 immutable; + u8 rsvd; + u32 asid; +} __packed; + #ifdef CONFIG_AMD_MEM_ENCRYPT extern struct static_key_false sev_es_enable_key; extern void __sev_es_ist_enter(struct pt_regs *regs);@@ -198,6 +212,9 @@ bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void);int
snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err);
intsnp_lookup_rmpentry(u64 pfn, int *level);arch/x86/include/asm/sev.h +2
-0+int psmash (u64 pfn); +int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid,bool immutable);
+intrmp_make_shared(u64 pfn, enum pg_level level); #else
static inline voidsev_es_ist_enter(struct pt_regs *regs){ } static inline voidsev_es_ist_exit(void){ }arch/x86/include/asm/sev.h +0
-0@@ -223,6 +240,13 @@ static inline int snp_issue_guest_request(u64
exit_code, structsnp_req_data*in
return-ENOTTY;
}
static inline intsnp_lookup_rmpentry(u64 pfn, int *level){ return 0; }arch/x86/include/asm/sev.h +2
-0+static inline int psmash (u64 pfn) {return -ENXIO ; } +static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_levellevel, int asid,
+ bool immutable)arch/x86/include/asm/sev.h +4
-0+{ + return -ENODEV; +} +static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return
-ENODEV; }
#endif#endif
arch/x86/kernel/sev.c +2
-0@@ -2523,3 +2523,98 @@ int snp_lookup_rmpentry(u64 pfn, int *level) return !!rmpentry_assigned(e); } EXPORT_SYMBOL_GPL(snp_lookup_rmpentry); + +/* arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2537
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+* psmash is used to smash a 2MB aligned page into 4K +* pages while preserving the Validated bit in the RMP. +*/ +int psmash (u64 pfn) +{ + unsigned long paddr = pfn << PAGE_SHIFT; + int ret; + + if (!pfn_valid(pfn)) + return -EINVAL;We (and many other clouds) use a neat trick to reduce the number of struct pages Linux allocates for guest memory: In its simplest form, add mem= to the kernel cmdline and
mmap()/dev/memto access the reserved memory instead.
Quoted 5 lines
Quoted 3 lines
This means that the system covers more RAM than Linux contains, which means
pfn_valid()is no longer a good indication whether a page is indeed valid. KVM handles this case fine, but this code does not.Hmm...but then is also using
max_pfnreliable ?
I would expect it to not be reliable as it only looks at E820_TYPE_RAM, yes. Do you rely on max_pfn anywhere?
Quoted 9 lines
Quoted 5 lines
Is there any particular reason why we need this check (and similar ones
below and in other RMP related patches) in the first place. I would > expect that PSMASH and friends return failure codes for invalid pfns.Yes, PSMASH does out of bounds check on the input SPA and additionally checks if SPA is 2M aligned, so guess we can rely on using PSMASH failing on invalid pfns.
Perfect, please remove all the superfluous checks then. If you want to make our life easy, I'd recommend you try to try the patch set with mem= passed on the host and tell QEMU to mmap() /dev/mem for guest RAM. That way you should be able to find any other pitfalls :)
Alex
Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
Kalra, Ashish
maintainerRe: [PATCH RFC v7 16/64] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
Quoted 137 lines · Alexander Graf
Quoted 135 lines · Ashish
Quoted 133 lines · Alexander Graf
Quoted 114 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.
The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.
Add the following external interface API functions:
int psmash(u64 pfn);
psmash is used to smash a 2MB aligned page into 4K pages while preserving the Validated bit in the RMP.int
rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable);
Used to assign a page to guest using the RMPUPDATE instruction.int
rmp_make_shared(u64 pfn, enum pg_level level); Used to transition a page to hypervisor/shared state using the RMPUPDATE instruction.Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>[mdr: add RMPUPDATE retry logic for transient FAIL_OVERLAP errors]Signed-off-by Michael Roth <[email protected]>---
arch/x86/include/asm/sev.h| 24 ++++++++++
arch/x86/kernel/sev.c | 95
++++++++++++++++++++++++++++++++++++++2 files changed, 119 insertions(+)arch/x86/include/asm/sev.h +5
-0@@ -80,10 +80,15 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); /* Software defined (when rFlags.CF = 1) */ #define PVALIDATE_FAIL_NOUPDATE 255 +/* RMUPDATE detected 4K page and 2MB page overlap. */ +#define RMPUPDATE_FAIL_OVERLAP 7 /* RMP page size */ #define RMP_PG_SIZE_4K 0 +#define RMP_PG_SIZE_2M 1 #define RMP_TO_X86_PG_LEVEL(level) (((level) == RMP_PG_SIZE_4K) ? PG_LEVEL_4K : PG_LEVEL_2M) +#define X86_TO_RMP_PG_LEVEL (level) (((level) == PG_LEVEL_4K) ? RMP_PG_SIZE_4K : RMP_PG_SIZE_2M) +#define RMPADJUST_VMSA_PAGE_BIT BIT(16)
/* SNP Guest message request */
arch/x86/include/asm/sev.h +9
-0@@ -133,6 +138,15 @@ struct snp_secrets_page_layout { u8 rsvd3[3840]; } __packed; +struct rmp_state { + u64 gpa; + u8 assigned; + u8 pagesize; + u8 immutable; + u8 rsvd; + u32 asid; +} __packed; + #ifdef CONFIG_AMD_MEM_ENCRYPT extern struct static_key_false sev_es_enable_key; extern void __sev_es_ist_enter(struct pt_regs *regs);@@ -198,6 +212,9 @@ bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void);int
snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err);
intsnp_lookup_rmpentry(u64 pfn, int *level);arch/x86/include/asm/sev.h +2
-0+int psmash (u64 pfn); +int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid,bool immutable);
+intrmp_make_shared(u64 pfn, enum pg_level level); #else
static inline voidsev_es_ist_enter(struct pt_regs *regs){ } static inline voidsev_es_ist_exit(void){ }arch/x86/include/asm/sev.h +0
-0@@ -223,6 +240,13 @@ static inline int snp_issue_guest_request(u64
exit_code, structsnp_req_data*in
return-ENOTTY;
}
static inline intsnp_lookup_rmpentry(u64 pfn, int *level){ return 0; }arch/x86/include/asm/sev.h +2
-0+static inline int psmash (u64 pfn) {return -ENXIO ; } +static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_levellevel, int asid,
+ bool immutable)arch/x86/include/asm/sev.h +4
-0+{ + return -ENODEV; +} +static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return
-ENODEV; }
#endif#endif
arch/x86/kernel/sev.c +2
-0@@ -2523,3 +2523,98 @@ int snp_lookup_rmpentry(u64 pfn, int *level) return !!rmpentry_assigned(e); } EXPORT_SYMBOL_GPL(snp_lookup_rmpentry); + +/* arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2537
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+* psmash is used to smash a 2MB aligned page into 4K +* pages while preserving the Validated bit in the RMP. +*/ +int psmash (u64 pfn) +{ + unsigned long paddr = pfn << PAGE_SHIFT; + int ret; + + if (!pfn_valid(pfn)) + return -EINVAL;We (and many other clouds) use a neat trick to reduce the number of struct pages Linux allocates for guest memory: In its simplest form, add mem= to the kernel cmdline and
mmap()/dev/memto access the reserved memory instead.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
This means that the system covers more RAM than Linux contains, which means
pfn_valid()is no longer a good indication whether a page is indeed valid. KVM handles this case fine, but this code does not.Hmm...but then is also using
max_pfnreliable ?I would expect it to not be reliable as it only looks at
E820_TYPE_RAM, yes. Do you rely onmax_pfnanywhere?
We use it to check if the RMP table is covering the whole system RAM, to get the max. addressable PFN, which should be fine.
Thanks,
Ashish
Re: [PATCH RFC v7 16/64] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
In reply to Michael RothQuoted 32 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The hypervisor will use the instruction to add pages to the RMP table. See APM3 for details on the instruction operations.
The PSMASH instruction expands a 2MB RMP entry into a corresponding set of contiguous 4KB-Page RMP entries. The hypervisor will use this instruction to adjust the RMP entry without invalidating the previous RMP entry.
Add the following external interface API functions:
int psmash(u64 pfn);
psmash is used to smash a 2MB aligned page into 4K pages while preserving the Validated bit in the RMP.int
rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid, bool immutable); Used to assign a page to guest using the RMPUPDATE instruction.int
rmp_make_shared(u64 pfn, enum pg_level level);
Used to transition a page to hypervisor/shared state using the RMPUPDATE instruction.Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>[mdr: add RMPUPDATE retry logic for transient FAIL_OVERLAP errors]Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/sev.h | 24 ++++++++++ arch/x86/kernel/sev.c | 95 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 119 insertions(+)
...
Quoted 11 lines
arch/x86/kernel/sev.c +11
-0+ +static int rmpupdate (u64 pfn,struct rmp_state *val) +{ +unsigned long paddr = pfn <<PAGE_SHIFT ; +int retries =0 ; +int ret; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return -ENXIO ; + +retry:
arch/x86/kernel/sev.c int snp_lookup_rmpentry(u64 pfn, int *level) · line 2571
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+/* Binutils version 2.36 supports the RMPUPDATE mnemonic. */ + asmvolatile (".byte 0xF2, 0x0F, 0x01, 0xFE" + :"=a" (ret) + :"a" (paddr),"c" ((unsigned long )val) + :"memory" ,"cc" ); + +if (ret) { +if (!retries) { +pr_err ("RMPUPDATE failed, ret: %d, pfn: %llx, npages: %d, level: %d, retrying (max: %d)...\n" , + ret, pfn, npages, level,2 *num_present_cpus ());
This patch isn't bisectable - 'npages' isn't defined in this patch - it's defined later in Patch18
otherwise LGTM
Regards,
Liam
Quoted 11 lines
arch/x86/kernel/sev.c +11
-0+dump_stack (); + } + retries++; +if (retries <2 *num_present_cpus ()) + goto retry; + }else if (retries >0 ) { +pr_err ("RMPUPDATE for pfn %llx succeeded after %d retries\n" , pfn, retries); + } + +return ret; +}
RFC PATCH v7 17/64 x86/mm/pat: Introduce set_memory_p
In reply to Michael RothFrom: Ashish Kalra <[email protected]>
set_memory_p() provides a way to change atributes of a memory range to be marked as present.
--- arch/x86/include/asm/set_memory.h | 3 ++- arch/x86/mm/pat/set_memory.c | 12 ++++++------ 2 files changed, 8 insertions(+), 7 deletions(-)
arch/x86/include/asm/set_memory.h +2 -1
@@ -12,7 +12,7 @@* Cacheability : UnCached, WriteCombining, WriteThrough, WriteBack * Executability : eXecutable, NoteXecutable * Read/Write : ReadOnly, ReadWrite -* Presence : NotPresent +* Presence : NotPresent, Present * Encryption : Encrypted, Decrypted * * Within a category, the attributes are mutually exclusive.
@@ -44,6 +44,7 @@ int set_memory_uc(unsigned long addr, int numpages);int set_memory_wc (unsigned long addr,int numpages);int set_memory_wb (unsigned long addr,int numpages);int set_memory_np (unsigned long addr,int numpages); +int set_memory_p (unsigned long addr,int numpages);int set_memory_4k (unsigned long addr,int numpages);int set_memory_encrypted (unsigned long addr,int numpages);int set_memory_decrypted (unsigned long addr,int numpages);
arch/x86/mm/pat/set_memory.c +6 -6
@@ -1993,17 +1993,12 @@ int set_mce_nospec(unsigned long pfn) return rc; } -static int set_memory_p (unsigned long *addr,int numpages) -{ -return change_page_attr_set (addr, numpages,__pgprot (_PAGE_PRESENT),0 ); -} -/* Restore full speculative operation to the pfn. */ int clear_mce_nospec (unsigned long pfn) {unsigned long addr = (unsigned long )pfn_to_kaddr (pfn); -return set_memory_p (&addr,1 ); +return set_memory_p (addr,1 ); }EXPORT_SYMBOL_GPL (clear_mce_nospec);#endif /* CONFIG_X86_64 */
@@ -2039,6 +2034,11 @@ int set_memory_np(unsigned long addr, int numpages)return change_page_attr_clear (&addr, numpages,__pgprot (_PAGE_PRESENT),0 ); } +int set_memory_p (unsigned long addr,int numpages) +{ +return change_page_attr_set (&addr, numpages,__pgprot (_PAGE_PRESENT),0 ); +} +int set_memory_np_noalias (unsigned long addr,int numpages) {int cpa_flags =CPA_NO_CHECK_ALIAS ;
Signature · 1 line
2.25.1
RFC PATCH v7 18/64 x86/sev: Invalidate pages from the direct map when adding them to the RMP table
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
The integrity guarantee of SEV-SNP is enforced through the RMP table. The RMP is used with standard x86 and IOMMU page tables to enforce memory restrictions and page access rights. The RMP check is enforced as soon as SEV-SNP is enabled globally in the system. When hardware encounters an RMP-check failure, it raises a page-fault exception.
The rmp_make_private() and rmp_make_shared() helpers are used to add or remove the pages from the RMP table. Improve the rmp_make_private() to invalidate state so that pages cannot be used in the direct-map after they are added the RMP table, and restored to their default valid permission after the pages are removed from the RMP table.
--- arch/x86/kernel/sev.c | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-)
arch/x86/kernel/sev.c +37 -1
@@ -2549,15 +2549,40 @@ int psmash(u64 pfn) }EXPORT_SYMBOL_GPL (psmash); +static int restore_direct_map (u64 pfn,int npages) +{ +return set_memory_p ((unsigned long )pfn_to_kaddr (pfn), npages); +} + +static int invalidate_direct_map (unsigned long pfn,int npages) +{ +return set_memory_np ((unsigned long )pfn_to_kaddr (pfn), npages); +} +static int rmpupdate (u64 pfn,struct rmp_state *val) {unsigned long paddr = pfn <<PAGE_SHIFT ; +int ret, level, npages;int retries =0 ; -int ret;if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP ))return -ENXIO ; + level =RMP_TO_X86_PG_LEVEL (val->pagesize); + npages =page_level_size (level) /PAGE_SIZE ; + +/* +* If page is getting assigned in the RMP table then unmap it from the +* direct map. +*/ +if (val->assigned) { +if (invalidate_direct_map (pfn, npages)) { +pr_err ("Failed to unmap %d pages at pfn 0x%llx from the direct_map\n" , + npages, pfn); +return -EFAULT ; + } + } + retry:/* Binutils version 2.36 supports the RMPUPDATE mnemonic. */ asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE"
@@ -2578,6 +2603,17 @@ static int rmpupdate(u64 pfn, struct rmp_state *val)pr_err ("RMPUPDATE for pfn %llx succeeded after %d retries\n" , pfn, retries); } +/* +* Restore the direct map after the page is removed from the RMP table. +*/ +if (!ret && !val->assigned) { +if (restore_direct_map (pfn, npages)) { +pr_err ("Failed to map %d pages at pfn 0x%llx into the direct_map\n" , + npages, pfn); +return -EFAULT ; + } + } +return ret; }
Signature · 1 line
2.25.1
RFC PATCH v7 19/64 x86/traps: Define RMP violation #PF error code
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
Bit 31 in the page fault-error bit will be set when processor encounters an RMP violation.
While at it, use the BIT_ULL() macro.
Signed-off by: Ashish Kalra <[email protected]>
--- arch/x86/include/asm/trap_pf.h | 18 +++++++++++------- arch/x86/mm/fault.c | 1 + 2 files changed, 12 insertions(+), 7 deletions(-)
arch/x86/include/asm/trap_pf.h +11 -7
@@ -2,6 +2,8 @@#ifndef _ASM_X86_TRAP_PF_H#define _ASM_X86_TRAP_PF_H +#include <linux/bits.h> /* BIT() macro */ +/* * Page fault error code bits: *
@@ -12,15 +14,17 @@* bit 4 == 1: fault was an instruction fetch * bit 5 == 1: protection keys block access * bit 15 == 1: SGX MMU page-fault +* bit 31 == 1: fault was due to RMP violation */ enum x86_pf_error_code { -X86_PF_PROT =1 <<0 , -X86_PF_WRITE =1 <<1 , -X86_PF_USER =1 <<2 , -X86_PF_RSVD =1 <<3 , -X86_PF_INSTR =1 <<4 , -X86_PF_PK =1 <<5 , -X86_PF_SGX =1 <<15 , +X86_PF_PROT =BIT (0 ), +X86_PF_WRITE =BIT (1 ), +X86_PF_USER =BIT (2 ), +X86_PF_RSVD =BIT (3 ), +X86_PF_INSTR =BIT (4 ), +X86_PF_PK =BIT (5 ), +X86_PF_SGX =BIT (15 ), +X86_PF_RMP =BIT (31 ), };#endif /* _ASM_X86_TRAP_PF_H */
arch/x86/mm/fault.c +1 -0
@@ -567,6 +567,7 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad !(error_code &X86_PF_PROT ) ?"not-present page" : (error_code &X86_PF_RSVD ) ?"reserved bit violation" : (error_code &X86_PF_PK ) ?"protection keys violation" : + (error_code &X86_PF_RMP ) ?"RMP violation" : "permissions violation");if (!(error_code &X86_PF_USER ) &&user_mode (regs)) {
Signature · 1 line
2.25.1
RFC PATCH v7 20/64 x86/fault: Add support to handle the RMP fault for user address
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
When SEV-SNP is enabled globally, a write from the host goes through the RMP check. When the host writes to pages, hardware checks the following conditions at the end of page walk:
1. Assigned bit in the RMP table is zero (i.e page is shared).
2. If the page table entry that gives the sPA indicates that the target page size is a large page, then all RMP entries for the 4KB constituting pages of the target must have the assigned bit 0.
3. Immutable bit in the RMP table is not zero.
The hardware will raise page fault if one of the above conditions is not met. Try resolving the fault instead of taking fault again and again. If the host attempts to write to the guest private memory then send the SIGBUS signal to kill the process. If the page level between the host and RMP entry does not match, then split the address to keep the RMP and host page levels in sync.
--- arch/x86/mm/fault.c | 97 ++++++++++++++++++++++++++++++++++++++++ include/linux/mm.h | 3 +- include/linux/mm_types.h | 3 ++ mm/memory.c | 10 +++++ 4 files changed, 112 insertions(+), 1 deletion(-)
arch/x86/mm/fault.c +97 -0
@@ -33,6 +33,7 @@#include <asm/kvm_para.h> /* kvm_handle_async_pf */ #include <asm/vdso.h> /* fixup_vdso_exception() */ #include <asm/irq_stack.h> +#include <asm/sev.h> /* snp_lookup_rmpentry() */ #define CREATE_TRACE_POINTS #include <asm/trace/exceptions.h>
@@ -414,6 +415,7 @@ static void dump_pagetable(unsigned long address)pr_cont ("PTE %lx" ,pte_val (*pte)); out:pr_cont ("\n" ); +return ; bad:pr_info ("BAD\n" );
@@ -1240,6 +1242,90 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code, }NOKPROBE_SYMBOL (do_kern_addr_fault); +enum rmp_pf_ret { +RMP_PF_SPLIT =0 , +RMP_PF_RETRY =1 , +RMP_PF_UNMAP =2 , +}; + +/* +* The goal of RMP faulting routine is really to check whether the +* page that faulted should be accessible. That can be determined +* simply by looking at the RMP entry for the 4k address being accessed. +* If that entry has Assigned=1 then it's a bad address. It could be +* because the 2MB region was assigned as a large page, or it could be +* because the region is all 4k pages and that 4k was assigned. +* In either case, it's a bad access. +* There are basically two main possibilities: +* 1. The 2M entry has Assigned=1 and Page_Size=1. Then all 511 middle +* entries also have Assigned=1. This entire 2M region is a guest page. +* 2. The 2M entry has Assigned=0 and Page_Size=0. Then the 511 middle +* entries can be anything, this region consists of individual 4k assignments. +*/ +static int handle_user_rmp_page_fault (struct pt_regs *regs,unsigned long error_code, +unsigned long address) +{ +int rmp_level, level; +pgd_t *pgd; +pte_t *pte; +u64 pfn; + + pgd =__va (read_cr3_pa ()); + pgd +=pgd_index (address); + + pte =lookup_address_in_pgd (pgd, address, &level); + +/* +* It can happen if there was a race between an unmap event and +* the RMP fault delivery. +*/ +if (!pte || !pte_present (*pte)) +return RMP_PF_UNMAP ; + +/* +* RMP page fault handler follows this algorithm: +* 1. Compute the pfn for the 4kb page being accessed +* 2. Read that RMP entry -- If it is assigned then kill the process +* 3. Otherwise, check the level from the host page table +* If level=PG_LEVEL_4K then the page is already smashed +* so just retry the instruction +* 4. If level=PG_LEVEL_2M/1G, then the host page needs to be split +*/ + + pfn =pte_pfn (*pte); + +/* If its large page then calculte the fault pfn */ +if (level >PG_LEVEL_4K ) + pfn = pfn |PFN_DOWN (address & (page_level_size (level) -1 )); + +/* +* If its a guest private page, then the fault cannot be resolved. +* Send a SIGBUS to terminate the process. +* +* As documented in APM vol3 pseudo-code for RMPUPDATE, when the 2M range +* is covered by a valid (Assigned=1) 2M entry, the middle 511 4k entries +* also have Assigned=1. This means that if there is an access to a page +* which happens to lie within an Assigned 2M entry, the 4k RMP entry +* will also have Assigned=1. Therefore, the kernel should see that +* the page is not a valid page and the fault cannot be resolved. +*/ +if (snp_lookup_rmpentry (pfn, &rmp_level)) { +pr_info ("Fatal RMP page fault, terminating process, entry assigned for pfn 0x%llx\n" , + pfn); +do_sigbus (regs, error_code, address,VM_FAULT_SIGBUS ); +return RMP_PF_RETRY ; + } + +/* +* The backing page level is higher than the RMP page level, request +* to split the page. +*/ +if (level > rmp_level) +return RMP_PF_SPLIT ; + +return RMP_PF_RETRY ; +} +/* * Handle faults in the user portion of the address space. Nothing in here * should check X86_PF_USER without a specific justification: for almost
@@ -1337,6 +1423,17 @@ void do_user_addr_fault(struct pt_regs *regs,if (error_code &X86_PF_INSTR ) flags |=FAULT_FLAG_INSTRUCTION ; +/* +* If its an RMP violation, try resolving it. +*/ +if (error_code &X86_PF_RMP ) { +if (handle_user_rmp_page_fault (regs, error_code, address)) +return ; + +/* Ask to split the page */ + flags |=FAULT_FLAG_PAGE_SPLIT ; + } +#ifdef CONFIG_X86_64 /* * Faults in the vsyscall page might need emulation. The
include/linux/mm.h +2 -1
@@ -466,7 +466,8 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags) { FAULT_FLAG_USER, "USER" }, \ { FAULT_FLAG_REMOTE, "REMOTE" }, \ { FAULT_FLAG_INSTRUCTION, "INSTRUCTION" }, \ - { FAULT_FLAG_INTERRUPTIBLE, "INTERRUPTIBLE" } + { FAULT_FLAG_INTERRUPTIBLE, "INTERRUPTIBLE" }, \ + { FAULT_FLAG_PAGE_SPLIT, "PAGESPLIT" }/* * vm_fault is filled by the pagefault handler and passed to the vma's
include/linux/mm_types.h +3 -0
@@ -962,6 +962,8 @@ typedef struct {* mapped R/O. * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached. * We should only access orig_pte if this flag set. +* @FAULT_FLAG_PAGE_SPLIT: The fault was due page size mismatch, split the +* region to smaller page size and retry. * * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify * whether we would allow page faults to retry by specifying these two
@@ -999,6 +1001,7 @@ enum fault_flag {FAULT_FLAG_INTERRUPTIBLE =1 <<9 ,FAULT_FLAG_UNSHARE =1 <<10 ,FAULT_FLAG_ORIG_PTE_VALID =1 <<11 , + FAULT_FLAG_PAGE_SPLIT = 1 << 12, };typedef unsigned int __bitwise zap_flags_t ;
mm/memory.c +10 -0
@@ -4996,6 +4996,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)return 0 ; } +static int handle_split_page_fault (struct vm_fault *vmf) +{ +__split_huge_pmd (vmf->vma, vmf->pmd, vmf->address, false,NULL ); +return 0 ; +} +/* * By the time we get here, we already hold the mm semaphore *
@@ -5078,6 +5084,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,pmd_migration_entry_wait (mm, vmf.pmd);return 0 ; } + +if (flags &FAULT_FLAG_PAGE_SPLIT ) +return handle_split_page_fault (&vmf); +if (pmd_trans_huge (vmf.orig_pmd) ||pmd_devmap (vmf.orig_pmd)) {if (pmd_protnone (vmf.orig_pmd) &&vma_is_accessible (vma))return do_huge_pmd_numa_page (&vmf);
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 20/64] x86/fault: Add support to handle the RMP fault for user address
Quoted 6 lines · On Wed, 14 Dec 2022 13:40:12 -0600 Michael Roth <[email protected]> wrote:
From: Brijesh Singh <[email protected]>
When SEV-SNP is enabled globally, a write from the host goes through the RMP check. When the host writes to pages, hardware checks the following conditions at the end of page walk:
Quoted 6 lines
1. Assigned bit in the RMP table is zero (i.e page is shared).
2. If the page table entry that gives the sPA indicates that the target page size is a large page, then all RMP entries for the 4KB constituting pages of the target must have the assigned bit 0.
3. Immutable bit in the RMP table is not zero.
Just being curious. AMD APM table 15-37 "RMP Page Assignment Settings" shows Immuable bit is "don't care" when a page is owned by the hypervisor. The table 15-39 "RMP Memory Access Checks" shows the hardware will do "Hypervisor-owned" check for host data write and page table access. I suppose "Hypervisor-owned" check means HW will check if the RMP entry is configured according to the table 15-37 (Assign bit = 0, ASID = 0, Immutable = X)
None of them mentions that Immutable bit in the related RMP-entry should be 1 for hypervisor-owned page.
I can understand 1) 2). Can you explain more about 3)?
Quoted 212 lines
The hardware will raise page fault if one of the above conditions is not met. Try resolving the fault instead of taking fault again and again. If the host attempts to write to the guest private memory then send the SIGBUS signal to kill the process. If the page level between the host and RMP entry does not match, then split the address to keep the RMP and host page levels in sync.
Co-developed-by Jarkko Sakkinen <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>Co-developed-by Ashish Kalra <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/mm/fault.c | 97 ++++++++++++++++++++++++++++++++++++++++ include/linux/mm.h | 3 +- include/linux/mm_types.h | 3 ++ mm/memory.c | 10 +++++ 4 files changed, 112 insertions(+), 1 deletion(-)arch/x86/mm/fault.c +97
-0@@ -33,6 +33,7 @@#include <asm/kvm_para.h> /* kvm_handle_async_pf */ #include <asm/vdso.h> /* fixup_vdso_exception() */ #include <asm/irq_stack.h> +#include <asm/sev.h> /* snp_lookup_rmpentry() */ #define CREATE_TRACE_POINTS #include <asm/trace/exceptions.h> @@ -414,6 +415,7 @@ static void dump_pagetable(unsigned long address)pr_cont ("PTE %lx" ,pte_val (*pte)); out:pr_cont ("\n" ); +return ; bad:pr_info ("BAD\n" );@@ -1240,6 +1242,90 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code, }NOKPROBE_SYMBOL (do_kern_addr_fault); +enum rmp_pf_ret { +RMP_PF_SPLIT =0 , +RMP_PF_RETRY =1 , +RMP_PF_UNMAP =2 , +}; + +/* +* The goal of RMP faulting routine is really to check whether the +* page that faulted should be accessible. That can be determined +* simply by looking at the RMP entry for the 4k address being accessed. +* If that entry has Assigned=1 then it's a bad address. It could be +* because the 2MB region was assigned as a large page, or it could be +* because the region is all 4k pages and that 4k was assigned. +* In either case, it's a bad access. +* There are basically two main possibilities: +* 1. The 2M entry has Assigned=1 and Page_Size=1. Then all 511 middle +* entries also have Assigned=1. This entire 2M region is a guest page. +* 2. The 2M entry has Assigned=0 and Page_Size=0. Then the 511 middle +* entries can be anything, this region consists of individual 4k assignments. +*/ +static int handle_user_rmp_page_fault (struct pt_regs *regs,unsigned long error_code, +unsigned long address) +{ +int rmp_level, level; +pgd_t *pgd; +pte_t *pte; +u64 pfn; + + pgd =__va (read_cr3_pa ()); + pgd +=pgd_index (address); + + pte =lookup_address_in_pgd (pgd, address, &level); + +/* +* It can happen if there was a race between an unmap event and +* the RMP fault delivery. +*/ +if (!pte || !pte_present (*pte)) +return RMP_PF_UNMAP ; + +/* +* RMP page fault handler follows this algorithm: +* 1. Compute the pfn for the 4kb page being accessed +* 2. Read that RMP entry -- If it is assigned then kill the process +* 3. Otherwise, check the level from the host page table +* If level=PG_LEVEL_4K then the page is already smashed +* so just retry the instruction +* 4. If level=PG_LEVEL_2M/1G, then the host page needs to be split +*/ + + pfn =pte_pfn (*pte); + +/* If its large page then calculte the fault pfn */ +if (level >PG_LEVEL_4K ) + pfn = pfn |PFN_DOWN (address & (page_level_size (level) -1 )); + +/* +* If its a guest private page, then the fault cannot be resolved. +* Send a SIGBUS to terminate the process. +* +* As documented in APM vol3 pseudo-code for RMPUPDATE, when the 2M range +* is covered by a valid (Assigned=1) 2M entry, the middle 511 4k entries +* also have Assigned=1. This means that if there is an access to a page +* which happens to lie within an Assigned 2M entry, the 4k RMP entry +* will also have Assigned=1. Therefore, the kernel should see that +* the page is not a valid page and the fault cannot be resolved. +*/ +if (snp_lookup_rmpentry (pfn, &rmp_level)) { +pr_info ("Fatal RMP page fault, terminating process, entry assigned for pfn 0x%llx\n" , + pfn); +do_sigbus (regs, error_code, address,VM_FAULT_SIGBUS ); +return RMP_PF_RETRY ; + } + +/* +* The backing page level is higher than the RMP page level, request +* to split the page. +*/ +if (level > rmp_level) +return RMP_PF_SPLIT ; + +return RMP_PF_RETRY ; +} +/* * Handle faults in the user portion of the address space. Nothing in here * should check X86_PF_USER without a specific justification: for almost @@ -1337,6 +1423,17 @@ void do_user_addr_fault(struct pt_regs *regs,if (error_code &X86_PF_INSTR ) flags |=FAULT_FLAG_INSTRUCTION ; +/* +* If its an RMP violation, try resolving it. +*/ +if (error_code &X86_PF_RMP ) { +if (handle_user_rmp_page_fault (regs, error_code, address)) +return ; + +/* Ask to split the page */ + flags |=FAULT_FLAG_PAGE_SPLIT ; + } +#ifdef CONFIG_X86_64 /* * Faults in the vsyscall page might need emulation. The include/linux/mm.h +2
-1@@ -466,7 +466,8 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags) { FAULT_FLAG_USER, "USER" }, \ { FAULT_FLAG_REMOTE, "REMOTE" }, \ { FAULT_FLAG_INSTRUCTION, "INSTRUCTION" }, \ - { FAULT_FLAG_INTERRUPTIBLE, "INTERRUPTIBLE" } + { FAULT_FLAG_INTERRUPTIBLE, "INTERRUPTIBLE" }, \ + { FAULT_FLAG_PAGE_SPLIT, "PAGESPLIT" }/* * vm_fault is filled by the pagefault handler and passed to the vma's include/linux/mm_types.h +3
-0@@ -962,6 +962,8 @@ typedef struct {* mapped R/O. * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached. * We should only access orig_pte if this flag set. +* @FAULT_FLAG_PAGE_SPLIT: The fault was due page size mismatch, split the +* region to smaller page size and retry. * * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify * whether we would allow page faults to retry by specifying these two @@ -999,6 +1001,7 @@ enum fault_flag {FAULT_FLAG_INTERRUPTIBLE =1 <<9 ,FAULT_FLAG_UNSHARE =1 <<10 ,FAULT_FLAG_ORIG_PTE_VALID =1 <<11 , + FAULT_FLAG_PAGE_SPLIT = 1 << 12, };typedef unsigned int __bitwise zap_flags_t ;mm/memory.c +10
-0@@ -4996,6 +4996,12 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)return 0 ; } +static int handle_split_page_fault (struct vm_fault *vmf) +{ +__split_huge_pmd (vmf->vma, vmf->pmd, vmf->address, false,NULL ); +return 0 ; +} +/* * By the time we get here, we already hold the mm semaphore * @@ -5078,6 +5084,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,pmd_migration_entry_wait (mm, vmf.pmd);return 0 ; } + +if (flags &FAULT_FLAG_PAGE_SPLIT ) +return handle_split_page_fault (&vmf); +if (pmd_trans_huge (vmf.orig_pmd) ||pmd_devmap (vmf.orig_pmd)) {if (pmd_protnone (vmf.orig_pmd) &&vma_is_accessible (vma))return do_huge_pmd_numa_page (&vmf);
RFC PATCH v7 21/64 x86/fault: fix handle_split_page_fault() to work with memfd backed pages
In reply to Michael RothFrom: Hugh Dickins <[email protected]>
When the address is backed by a memfd, the code to split the page does nothing more than remove the PMD from the page tables. So immediately install a PTE to ensure that any other pages in that 2MB region are brought back as in 4K pages.
--- mm/memory.c | 5 +++++ 1 file changed, 5 insertions(+)
mm/memory.c +5 -0
@@ -4999,6 +4999,11 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)static int handle_split_page_fault (struct vm_fault *vmf) {__split_huge_pmd (vmf->vma, vmf->pmd, vmf->address, false,NULL ); +/* +* Install a PTE immediately to ensure that any other pages in +* this 2MB region are brought back in as 4K pages. +*/ +__pte_alloc (vmf->vma->vm_mm, vmf->pmd);return 0 ; }
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 21/64] x86/fault: fix handle_split_page_fault() to work with memfd backed pages
Quoted 11 lines · Michael Roth
From: Hugh Dickins <[email protected]>
When the address is backed by a memfd, the code to split the page does nothing more than remove the PMD from the page tables. So immediately install a PTE to ensure that any other pages in that 2MB region are brought back as in 4K pages.
Signed-off-by Hugh Dickins <[email protected]>Cc Hugh Dickins <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>
Hah, it's good to see this again, but it was "Suggested-by" me, not "Signed-off-by" me. And was a neat pragmatic one-liner workaround for the immediate problem we had, but came with caveats.
The problem is that we have one wind blowing in the split direction, and another wind (khugepaged) blowing in the collapse direction, and who wins for how long depends on factors I've not fully got to grips with (and is liable to differ between kernel releases).
Good and bad timing to see it. I was just yesterday reviewing a patch to the collapsing wind, which reminded me of an improvement yet to be made there, thinking I'd like to try it sometime; but recallng that someone somewhere relies on the splitting wind, and doesn't want the collapsing wind to blow any harder - now you remind me who!
Bad timing in that I don't have any quick answer on the right thing to do instead, and can't give it the thought it needs at the moment - perhaps others can chime in more usefully.
Hugh
p.s. I don't know where "handle_split_page_fault" comes in, but "x86/fault" in the subject looks wrong, since this appears to be in generic code; and "memfd" seems inappropriate too, but perhaps you have a situation where only memfds can reach handle_split_page_fault().
Quoted 22 lines
--- mm/memory.c | 5 +++++ 1 file changed, 5 insertions(+)mm/memory.c +5
-0@@ -4999,6 +4999,11 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)static int handle_split_page_fault (struct vm_fault *vmf) {__split_huge_pmd (vmf->vma, vmf->pmd, vmf->address, false,NULL ); +/* +* Install a PTE immediately to ensure that any other pages in +* this 2MB region are brought back in as 4K pages. +*/ +__pte_alloc (vmf->vma->vm_mm, vmf->pmd);return 0 ; }Signature · 1 line
2.25.1
RFC PATCH v7 22/64 x86/fault: Return pfn from dump_pagetable() for SEV-specific fault handling.
In reply to Michael RothFrom: Ashish Kalra <[email protected]>
Return pfn from dump_pagetable() to do SEV-specific fault handling. Used for handling SNP RMP page fault.
--- arch/x86/mm/fault.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
arch/x86/mm/fault.c +11 -4
@@ -311,7 +311,7 @@ static bool low_pfn(unsigned long pfn)return pfn < max_low_pfn; } -static void dump_pagetable (unsigned long address) +static unsigned long dump_pagetable (unsigned long address) {pgd_t *base =__va (read_cr3_pa ());pgd_t *pgd = &base[pgd_index (address)];
@@ -345,8 +345,10 @@ static void dump_pagetable(unsigned long address) pte =pte_offset_kernel (pmd, address);pr_cont ("*pte = %0*Lx " ,sizeof (*pte) *2 , (u64 )pte_val (*pte)); +return 0 ; out:pr_cont ("\n" ); + return 0; }#else /* CONFIG_X86_64: */
@@ -367,10 +369,11 @@ static int bad_address(void *p)return get_kernel_nofault (dummy, (unsigned long *)p); } -static void dump_pagetable (unsigned long address) +static unsigned long dump_pagetable (unsigned long address) {pgd_t *base =__va (read_cr3_pa ());pgd_t *pgd = base +pgd_index (address); +unsigned long pfn;p4d_t *p4d;pud_t *pud;pmd_t *pmd;
@@ -388,6 +391,7 @@ static void dump_pagetable(unsigned long address)if (bad_address (p4d)) goto bad; + pfn =p4d_pfn (*p4d);pr_cont ("P4D %lx " ,p4d_val (*p4d));if (!p4d_present (*p4d) ||p4d_large (*p4d)) goto out;
@@ -396,6 +400,7 @@ static void dump_pagetable(unsigned long address)if (bad_address (pud)) goto bad; + pfn =pud_pfn (*pud);pr_cont ("PUD %lx " ,pud_val (*pud));if (!pud_present (*pud) ||pud_large (*pud)) goto out;
@@ -404,6 +409,7 @@ static void dump_pagetable(unsigned long address)if (bad_address (pmd)) goto bad; + pfn =pmd_pfn (*pmd);pr_cont ("PMD %lx " ,pmd_val (*pmd));if (!pmd_present (*pmd) ||pmd_large (*pmd)) goto out;
@@ -412,13 +418,14 @@ static void dump_pagetable(unsigned long address)if (bad_address (pte)) goto bad; + pfn =pte_pfn (*pte);pr_cont ("PTE %lx" ,pte_val (*pte)); out:pr_cont ("\n" ); - -return ; +return pfn; bad:pr_info ("BAD\n" ); + return -1; }#endif /* CONFIG_X86_64 */
Signature · 1 line
2.25.1
RFC PATCH v7 23/64 x86/fault: Add support to dump RMP entry on fault
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
When SEV-SNP is enabled globally, a write from the host goes through the RMP check. If the hardware encounters the check failure, then it raises the #PF (with RMP set). Dump the RMP entry at the faulting pfn to help the debug.
--- arch/x86/include/asm/sev.h | 2 ++ arch/x86/kernel/sev.c | 43 ++++++++++++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 7 ++++++- 3 files changed, 51 insertions(+), 1 deletion(-)
arch/x86/include/asm/sev.h +2 -0
@@ -215,6 +215,7 @@ int snp_lookup_rmpentry(u64 pfn, int *level);int psmash (u64 pfn);int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid,bool immutable);int rmp_make_shared (u64 pfn,enum pg_level level); +void sev_dump_rmpentry (u64 pfn);#else static inline void sev_es_ist_enter (struct pt_regs *regs) { }static inline void sev_es_ist_exit (void ) { }
@@ -247,6 +248,7 @@ static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asreturn -ENODEV ; }static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return -ENODEV ; } +static inline void sev_dump_rmpentry (u64 pfn) {}#endif #endif
arch/x86/kernel/sev.c +43 -0
49 lines not on this page · raw message
arch/x86/mm/fault.c +6 -1
@@ -536,6 +536,8 @@ static void show_ldttss(const struct desc_ptr *gdt, const char *name, u16 index)static void show_fault_oops (struct pt_regs *regs,unsigned long error_code,unsigned long address) { +unsigned long pfn; +if (!oops_may_print ())return ;
@@ -608,7 +610,10 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long adshow_ldttss (&gdt,"TR" , tr); } -dump_pagetable (address); + pfn =dump_pagetable (address); + +if (error_code &X86_PF_RMP ) +sev_dump_rmpentry (pfn); }static noinline void
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 23/64] x86/fault: Add support to dump RMP entry on fault
Quoted 43 lines · Michael Roth
From: Brijesh Singh <[email protected]>
When SEV-SNP is enabled globally, a write from the host goes through the RMP check. If the hardware encounters the check failure, then it raises the #PF (with RMP set). Dump the RMP entry at the faulting pfn to help the debug.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/sev.h | 2 ++ arch/x86/kernel/sev.c | 43 ++++++++++++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 7 ++++++- 3 files changed, 51 insertions(+), 1 deletion(-)arch/x86/include/asm/sev.h +2
-0@@ -215,6 +215,7 @@ int snp_lookup_rmpentry(u64 pfn, int *level);int psmash (u64 pfn);int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid,bool immutable);int rmp_make_shared (u64 pfn,enum pg_level level); +void sev_dump_rmpentry (u64 pfn);#else static inline void sev_es_ist_enter (struct pt_regs *regs) { }static inline void sev_es_ist_exit (void ) { }@@ -247,6 +248,7 @@ static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asreturn -ENODEV ; }static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return -ENODEV ; } +static inline void sev_dump_rmpentry (u64 pfn) {}#endif #endif arch/x86/kernel/sev.c +0
-0@@ -2508,6 +2508,49 @@ static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level)return entry;
arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2518
Quoted 10 lines
arch/x86/kernel/sev.c +8
-0} +void sev_dump_rmpentry (u64 pfn) +{ +unsigned long pfn_end; +struct rmpentry *e; +int level; + + e =__snp_lookup_rmpentry (pfn, &level); +if (!e) {
if (IS_ERR(e)) {
Quoted 16 lines
arch/x86/kernel/sev.c +16
-0+pr_info ("failed to read RMP entry pfn 0x%llx\n" , pfn); +return ; + } + +if (rmpentry_assigned (e)) { +pr_info ("RMPEntry paddr 0x%llx [assigned=%d immutable=%d pagesize=%d gpa=0x%lx" +" asid=%d vmsa=%d validated=%d]\n" , pfn <<PAGE_SHIFT , +rmpentry_assigned (e), e->info.immutable,rmpentry_pagesize (e), + (unsigned long )e->info.gpa, e->info.asid, e->info.vmsa, + e->info.validated); +return ; + } + +/* +* If the RMP entry at the faulting pfn was not assigned, then not sure +* what caused the RMP violation. To get some useful debug information,
arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2544
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+* iterate through the entire 2MB region, and dump the RMP entries if +* one of the bit in the RMP entry is set. +*/ + pfn = pfn & ~(PTRS_PER_PMD -1 ); + pfn_end = pfn +PTRS_PER_PMD ; + +while (pfn < pfn_end) { + e =__snp_lookup_rmpentry (pfn, &level); +if (!e) +return ;
if (IS_ERR(e))
continue;
Quoted 40 lines
arch/x86/kernel/sev.c +9
-0+ +if (e->low || e->high) +pr_info ("RMPEntry paddr 0x%llx: [high=0x%016llx low=0x%016llx]\n" , + pfn <<PAGE_SHIFT , e->high, e->low); + pfn++; + } +} +EXPORT_SYMBOL_GPL (sev_dump_rmpentry); +/* * Return 1 if the RMP entry is assigned, 0 if it exists but is not assigned, * and -errno if there is no corresponding RMP entry. arch/x86/mm/fault.c +6
-2@@ -536,6 +536,8 @@ static void show_ldttss(const struct desc_ptr *gdt, const char *name, u16 index)static void show_fault_oops (struct pt_regs *regs,unsigned long error_code,unsigned long address) { +unsigned long pfn; +if (!oops_may_print ())return ;@@ -608,7 +610,10 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long adshow_ldttss (&gdt,"TR" , tr); } -dump_pagetable (address); + pfn =dump_pagetable (address); + +if (error_code &X86_PF_RMP ) +sev_dump_rmpentry (pfn); }static noinline void --2.25.1
Kalra, Ashish
maintainerRe: [PATCH RFC v7 23/64] x86/fault: Add support to dump RMP entry on fault
Quoted 46 lines · Alper Gun
Quoted 45 lines · Michael Roth
From: Brijesh Singh <[email protected]>
When SEV-SNP is enabled globally, a write from the host goes through the RMP check. If the hardware encounters the check failure, then it raises the #PF (with RMP set). Dump the RMP entry at the faulting pfn to help the debug.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- arch/x86/include/asm/sev.h | 2 ++ arch/x86/kernel/sev.c | 43 ++++++++++++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 7 ++++++- 3 files changed, 51 insertions(+), 1 deletion(-)arch/x86/include/asm/sev.h +2
-0@@ -215,6 +215,7 @@ int snp_lookup_rmpentry(u64 pfn, int *level);int psmash (u64 pfn);int rmp_make_private (u64 pfn,u64 gpa,enum pg_level level,int asid,bool immutable);int rmp_make_shared (u64 pfn,enum pg_level level); +void sev_dump_rmpentry (u64 pfn);#else static inline void sev_es_ist_enter (struct pt_regs *regs) { }static inline void sev_es_ist_exit (void ) { }@@ -247,6 +248,7 @@ static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asreturn -ENODEV ; }static inline int rmp_make_shared (u64 pfn,enum pg_level level) {return -ENODEV ; } +static inline void sev_dump_rmpentry (u64 pfn) {}#endif #endif arch/x86/kernel/sev.c +0
-0@@ -2508,6 +2508,49 @@ static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level)return entry; }
Quoted 10 lines
arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2518
Quoted 8 lines
arch/x86/kernel/sev.c +8
-0+void sev_dump_rmpentry (u64 pfn) +{ +unsigned long pfn_end; +struct rmpentry *e; +int level; + + e =__snp_lookup_rmpentry (pfn, &level); +if (!e) {if (
IS_ERR(e)) {
Yes, this needs to be fixed to IS_ERR(e)
Quoted 27 lines
Quoted 16 lines
arch/x86/kernel/sev.c +16
-0+pr_info ("failed to read RMP entry pfn 0x%llx\n" , pfn); +return ; + } + +if (rmpentry_assigned (e)) { +pr_info ("RMPEntry paddr 0x%llx [assigned=%d immutable=%d pagesize=%d gpa=0x%lx" +" asid=%d vmsa=%d validated=%d]\n" , pfn <<PAGE_SHIFT , +rmpentry_assigned (e), e->info.immutable,rmpentry_pagesize (e), + (unsigned long )e->info.gpa, e->info.asid, e->info.vmsa, + e->info.validated); +return ; + } + +/* +* If the RMP entry at the faulting pfn was not assigned, then not sure +* what caused the RMP violation. To get some useful debug information, arch/x86/kernel/sev.c static struct rmpentry *__snp_lookup_rmpentry(u64 pfn, int *level) · line 2544
Quoted 10 lines
arch/x86/kernel/sev.c +10
-0+* iterate through the entire 2MB region, and dump the RMP entries if +* one of the bit in the RMP entry is set. +*/ + pfn = pfn & ~(PTRS_PER_PMD -1 ); + pfn_end = pfn +PTRS_PER_PMD ; + +while (pfn < pfn_end) { + e =__snp_lookup_rmpentry (pfn, &level); +if (!e) +return ;if (
IS_ERR(e))
Quoted 1 line
continue;
Again, this is correct, but then it should be :
if (IS_ERR(e)) {
pfn++;
continue;
}Thanks,
Ashish
Quoted 40 lines
Quoted 40 lines
arch/x86/kernel/sev.c +9
-0+ +if (e->low || e->high) +pr_info ("RMPEntry paddr 0x%llx: [high=0x%016llx low=0x%016llx]\n" , + pfn <<PAGE_SHIFT , e->high, e->low); + pfn++; + } +} +EXPORT_SYMBOL_GPL (sev_dump_rmpentry); +/* * Return 1 if the RMP entry is assigned, 0 if it exists but is not assigned, * and -errno if there is no corresponding RMP entry. arch/x86/mm/fault.c +6
-2@@ -536,6 +536,8 @@ static void show_ldttss(const struct desc_ptr *gdt, const char *name, u16 index)static void show_fault_oops (struct pt_regs *regs,unsigned long error_code,unsigned long address) { +unsigned long pfn; +if (!oops_may_print ())return ;@@ -608,7 +610,10 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long adshow_ldttss (&gdt,"TR" , tr); } -dump_pagetable (address); + pfn =dump_pagetable (address); + +if (error_code &X86_PF_RMP ) +sev_dump_rmpentry (pfn); }static noinline void --2.25.1
RFC PATCH v7 24/64 crypto:ccp: Define the SEV-SNP commands
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
AMD introduced the next generation of SEV called SEV-SNP (Secure Nested Paging). SEV-SNP builds upon existing SEV and SEV-ES functionality while adding new hardware security protection.
Define the commands and structures used to communicate with the AMD-SP when creating and managing the SEV-SNP guests. The SEV-SNP firmware spec is available at developer.amd.com/sev.
--- drivers/crypto/ccp/sev-dev.c | 16 +++ include/linux/psp-sev.h | 247 +++++++++++++++++++++++++++++++++++ include/uapi/linux/psp-sev.h | 42 ++++++ 3 files changed, 305 insertions(+)
drivers/crypto/ccp/sev-dev.c +16 -0
@@ -126,6 +126,8 @@ static int sev_cmd_buffer_len(int cmd)switch (cmd) {case SEV_CMD_INIT :return sizeof (struct sev_data_init );case SEV_CMD_INIT_EX :return sizeof (struct sev_data_init_ex ); +case SEV_CMD_SNP_SHUTDOWN_EX :return sizeof (struct sev_data_snp_shutdown_ex ); +case SEV_CMD_SNP_INIT_EX :return sizeof (struct sev_data_snp_init_ex );case SEV_CMD_PLATFORM_STATUS :return sizeof (struct sev_user_data_status );case SEV_CMD_PEK_CSR :return sizeof (struct sev_data_pek_csr );case SEV_CMD_PEK_CERT_IMPORT :return sizeof (struct sev_data_pek_cert_import );
@@ -154,6 +156,20 @@ static int sev_cmd_buffer_len(int cmd)case SEV_CMD_GET_ID :return sizeof (struct sev_data_get_id );case SEV_CMD_ATTESTATION_REPORT :return sizeof (struct sev_data_attestation_report );case SEV_CMD_SEND_CANCEL :return sizeof (struct sev_data_send_cancel ); +case SEV_CMD_SNP_GCTX_CREATE :return sizeof (struct sev_data_snp_addr ); +case SEV_CMD_SNP_LAUNCH_START :return sizeof (struct sev_data_snp_launch_start ); +case SEV_CMD_SNP_LAUNCH_UPDATE :return sizeof (struct sev_data_snp_launch_update ); +case SEV_CMD_SNP_ACTIVATE :return sizeof (struct sev_data_snp_activate ); +case SEV_CMD_SNP_DECOMMISSION :return sizeof (struct sev_data_snp_addr ); +case SEV_CMD_SNP_PAGE_RECLAIM :return sizeof (struct sev_data_snp_page_reclaim ); +case SEV_CMD_SNP_GUEST_STATUS :return sizeof (struct sev_data_snp_guest_status ); +case SEV_CMD_SNP_LAUNCH_FINISH :return sizeof (struct sev_data_snp_launch_finish ); +case SEV_CMD_SNP_DBG_DECRYPT :return sizeof (struct sev_data_snp_dbg ); +case SEV_CMD_SNP_DBG_ENCRYPT :return sizeof (struct sev_data_snp_dbg ); +case SEV_CMD_SNP_PAGE_UNSMASH :return sizeof (struct sev_data_snp_page_unsmash ); +case SEV_CMD_SNP_PLATFORM_STATUS :return sizeof (struct sev_data_snp_addr ); +case SEV_CMD_SNP_GUEST_REQUEST :return sizeof (struct sev_data_snp_guest_request ); +case SEV_CMD_SNP_CONFIG :return sizeof (struct sev_user_data_snp_config );default :return 0 ; }
include/linux/psp-sev.h +247 -0
259 lines not on this page · raw message
include/uapi/linux/psp-sev.h +42 -0
54 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 24/64] crypto:ccp: Define the SEV-SNP commands
Quoted 5 lines · Michael Roth
From: Brijesh Singh <[email protected]>
include/uapi/linux/psp-sev.h +3
-0+/* +* struct sev_user_data_snp_config - system wide configuration value for SNP. +*
include/uapi/linux/psp-sev.h struct sev_user_data_get_id2 { · line 191
Quoted 10 lines
include/uapi/linux/psp-sev.h +10
-0+* @reported_tcb: The TCB version to report in the guest attestation report. +* @mask_chip_id: Indicates that the CHID_ID field in the attestation report +* will always be zero. +*/ +struct sev_user_data_snp_config { +__u64 reported_tcb;/* In */ +__u32 mask_chip_id;/* In */ +__u8 rsvd[52 ]; +} __packed; +
Based on table 45 section 8.6.1 in
https://www.amd.com/system/files/TechDocs/56860.pdf I think this should be
struct sev_user_data_snp_config {
__u64 reported_tcb; /* In */
__u32 mask_chip_id:1; /* In */
__u32 mask_chip_key:1; /* In */
__u32 rsvd:30;
__u8 rsvd1[52];
} __packed;--
Sabin.
Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.
RFC PATCH v7 25/64 crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
Before SNP VMs can be launched, the platform must be appropriately configured and initialized. Platform initialization is accomplished via the SNP_INIT command. Make sure to do a WBINVD and issue DF_FLUSH command to prepare for the first SNP guest launch after INIT.
During the execution of SNP_INIT command, the firmware configures and enables SNP security policy enforcement in many system components. Some system components write to regions of memory reserved by early x86 firmware (e.g. UEFI). Other system components write to regions provided by the operation system, hypervisor, or x86 firmware. Such system components can only write to HV-fixed pages or Default pages. They will error when attempting to write to other page states after SNP_INIT enables their SNP enforcement.
Starting in SNP firmware v1.52, the SNP_INIT_EX command takes a list of system physical address ranges to convert into the HV-fixed page states during the RMP initialization. If INIT_RMP is 1, hypervisors should provide all system physical address ranges that the hypervisor will never assign to a guest until the next RMP re-initialization. For instance, the memory that UEFI reserves should be included in the range list. This allows system components that occasionally write to memory (e.g. logging to UEFI reserved regions) to not fail due to RMP initialization and SNP enablement.
--- drivers/crypto/ccp/sev-dev.c | 225 +++++++++++++++++++++++++++++++++++ drivers/crypto/ccp/sev-dev.h | 2 + include/linux/psp-sev.h | 17 +++ 3 files changed, 244 insertions(+)
drivers/crypto/ccp/sev-dev.c +225 -0
@@ -26,6 +26,7 @@#include <linux/fs_struct.h> #include <asm/smp.h> +#include <asm/e820/types.h> #include "psp-dev.h" #include "sev-dev.h"
@@ -34,6 +35,10 @@#define SEV_FW_FILE "amd/sev.fw"#define SEV_FW_NAME_SIZE 64 +/* Minimum firmware version required for the SEV-SNP support */ +#define SNP_MIN_API_MAJOR 1 +#define SNP_MIN_API_MINOR 51 +static DEFINE_MUTEX (sev_cmd_mutex);static struct sev_misc_dev *misc_dev;
@@ -76,6 +81,13 @@ static void *sev_es_tmr;#define NV_LENGTH (32 * 1024)static void *sev_init_ex_buffer; +/* +* SEV_DATA_RANGE_LIST: +* Array containing range of pages that firmware transitions to HV-fixed +* page state. +*/ +struct sev_data_range_list *snp_range_list; +static inline bool sev_version_greater_or_equal (u8 maj,u8 min) {struct sev_device *sev = psp_master->sev_data;
@@ -830,6 +842,186 @@ static int sev_update_firmware(struct device *dev)return ret; } +static void snp_set_hsave_pa (void *arg) +{ +wrmsrl (MSR_VM_HSAVE_PA ,0 ); +} + +static int snp_filter_reserved_mem_regions (struct resource *rs,void *arg) +{ +struct sev_data_range_list *range_list = arg; +struct sev_data_range *range = &range_list->ranges[range_list->num_elements]; +size_t size; + +if ((range_list->num_elements *sizeof (struct sev_data_range ) + +sizeof (struct sev_data_range_list )) >PAGE_SIZE ) +return -E2BIG ; + +switch (rs->desc) { +case E820_TYPE_RESERVED : +case E820_TYPE_PMEM : +case E820_TYPE_ACPI : + range->base = rs->start &PAGE_MASK ; + size = (rs->end +1 ) - rs->start; + range->page_count = size >>PAGE_SHIFT ; + range_list->num_elements++; +break ; +default : +break ; + } + +return 0 ; +} + +static int __sev_snp_init_locked (int *error) +{ +struct psp_device *psp = psp_master; +struct sev_data_snp_init_ex data; +struct sev_device *sev; +int rc =0 ; + +if (!psp || !psp->sev_data) +return -ENODEV ; + + sev = psp->sev_data; + +if (sev->snp_initialized) +return 0 ; + +/* +* The SNP_INIT requires the MSR_VM_HSAVE_PA must be set to 0h +* across all cores. +*/ +on_each_cpu (snp_set_hsave_pa,NULL ,1 ); + +/* +* Starting in SNP firmware v1.52, the SNP_INIT_EX command takes a list of +* system physical address ranges to convert into the HV-fixed page states +* during the RMP initialization. For instance, the memory that UEFI +* reserves should be included in the range list. This allows system +* components that occasionally write to memory (e.g. logging to UEFI +* reserved regions) to not fail due to RMP initialization and SNP enablement. +*/ +if (sev_version_greater_or_equal (SNP_MIN_API_MAJOR ,52 )) { +/* +* Firmware checks that the pages containing the ranges enumerated +* in the RANGES structure are either in the Default page state or in the +* firmware page state. +*/ + snp_range_list =sev_fw_alloc (PAGE_SIZE ); +if (!snp_range_list) { +dev_err (sev->dev, +"SEV: SNP_INIT_EX range list memory allocation failed\n" ); +return -ENOMEM ; + } + +memset (snp_range_list,0 ,PAGE_SIZE ); + +/* +* Retrieve all reserved memory regions setup by UEFI from the e820 memory map +* to be setup as HV-fixed pages. +*/ + + rc =walk_iomem_res_desc (IORES_DESC_NONE ,IORESOURCE_MEM ,0 , ~0 , + snp_range_list, snp_filter_reserved_mem_regions); +if (rc) { +dev_err (sev->dev, +"SEV: SNP_INIT_EX walk_iomem_res_desc failed rc = %d\n" , rc); +return rc; + } + +memset (&data,0 ,sizeof (data)); + data.init_rmp =1 ; + data.list_paddr_en =1 ; + data.list_paddr =__pa (snp_range_list); + + rc =__sev_do_cmd_locked (SEV_CMD_SNP_INIT_EX , &data, error); +if (rc) +return rc; + }else { + rc =__sev_do_cmd_locked (SEV_CMD_SNP_INIT ,NULL , error); +if (rc) +return rc; + } + +/* Prepare for first SNP guest launch after INIT */ +wbinvd_on_all_cpus (); + rc =__sev_do_cmd_locked (SEV_CMD_SNP_DF_FLUSH ,NULL , error); +if (rc) +return rc; + + sev->snp_initialized = true; +dev_dbg (sev->dev,"SEV-SNP firmware initialized\n" ); + +return rc; +} + +int sev_snp_init (int *error,bool init_on_probe) +{ +int rc; + +if (!cpu_feature_enabled (X86_FEATURE_SEV_SNP )) +return -ENODEV ; + +if (init_on_probe && !psp_init_on_probe) +return 0 ; + +mutex_lock (&sev_cmd_mutex); + rc =__sev_snp_init_locked (error); +mutex_unlock (&sev_cmd_mutex); + +return rc; +} +EXPORT_SYMBOL_GPL (sev_snp_init); + +static int __sev_snp_shutdown_locked (int *error) +{ +struct sev_device *sev = psp_master->sev_data; +struct sev_data_snp_shutdown_ex data; +int ret; + +if (!sev->snp_initialized) +return 0 ; + +memset (&data,0 ,sizeof (data)); + data.length =sizeof (data); + data.iommu_snp_shutdown =1 ; + +wbinvd_on_all_cpus (); + +retry: + ret =__sev_do_cmd_locked (SEV_CMD_SNP_SHUTDOWN_EX , &data, error); +/* SHUTDOWN may require DF_FLUSH */ +if (*error ==SEV_RET_DFFLUSH_REQUIRED ) { + ret =__sev_do_cmd_locked (SEV_CMD_SNP_DF_FLUSH ,NULL ,NULL ); +if (ret) { +dev_err (sev->dev,"SEV-SNP DF_FLUSH failed\n" ); +return ret; + } + goto retry; + } +if (ret) { +dev_err (sev->dev,"SEV-SNP firmware shutdown failed\n" ); +return ret; + } + + sev->snp_initialized = false; +dev_dbg (sev->dev,"SEV-SNP firmware shutdown\n" ); + +return ret; +} + +static int sev_snp_shutdown (int *error) +{ +int rc; + +mutex_lock (&sev_cmd_mutex); + rc =__sev_snp_shutdown_locked (error); +mutex_unlock (&sev_cmd_mutex); + +return rc; +} +static int sev_ioctl_do_pek_import (struct sev_issue_cmd *argp,bool writable) {struct sev_device *sev = psp_master->sev_data;
@@ -1270,6 +1462,8 @@ int sev_dev_init(struct psp_device *psp)static void sev_firmware_shutdown (struct sev_device *sev) { +int error; +sev_platform_shutdown (NULL );if (sev_es_tmr) {
@@ -1286,6 +1480,14 @@ static void sev_firmware_shutdown(struct sev_device *sev) get_order(NV_LENGTH)); sev_init_ex_buffer =NULL ; } + +if (snp_range_list) { +free_pages ((unsigned long )snp_range_list, +get_order (PAGE_SIZE )); + snp_range_list =NULL ; + } + +sev_snp_shutdown (&error); }void sev_dev_destroy (struct psp_device *psp)
@@ -1341,6 +1543,26 @@ void sev_pci_init(void) } } +/* +* If boot CPU supports SNP, then first attempt to initialize +* the SNP firmware. +*/ +if (cpu_feature_enabled (X86_FEATURE_SEV_SNP )) { +if (!sev_version_greater_or_equal (SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR )) { +dev_err (sev->dev,"SEV-SNP support requires firmware version >= %d:%d\n" , +SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR ); + }else { + rc =sev_snp_init (&error, true); +if (rc) { +/* +* Don't abort the probe if SNP INIT failed, +* continue to initialize the legacy SEV firmware. +*/ +dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); + } + } + } +/* Obtain the TMR memory area for SEV-ES use */ sev_es_tmr =sev_fw_alloc (SEV_ES_TMR_SIZE );if (!sev_es_tmr)
@@ -1356,6 +1578,9 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ? +"-SNP" :"" , sev->api_major, sev->api_minor, sev->build); +return ; err:
drivers/crypto/ccp/sev-dev.h +2 -0
@@ -52,6 +52,8 @@ struct sev_device {u8 build;void *cmd_buf; + +bool snp_initialized; };int sev_dev_init (struct psp_device *psp);
include/linux/psp-sev.h +17 -0
@@ -794,6 +794,21 @@ struct sev_data_snp_shutdown_ex {*/ int sev_platform_init (int *error); +/** +* sev_snp_init - perform SEV SNP_INIT command +* +* @error: SEV command return code +* @init_on_probe: indicates if called during module probe/init +* +* Returns: +* 0 if the SEV successfully processed the command +* -%ENODEV if the SEV device is not available +* -%ENOTSUPP if the SEV does not support SEV +* -%ETIMEDOUT if the SEV command timed out +* -%EIO if the SEV returned a non-zero return code +*/ +int sev_snp_init (int *error,bool init_on_probe); +/** * sev_platform_status - perform SEV PLATFORM_STATUS command *
@@ -901,6 +916,8 @@ sev_platform_status(struct sev_user_data_status *status, int *error) { return -Estatic inline int sev_platform_init (int *error) {return -ENODEV ; } +static inline int sev_snp_init (int *error,bool init_on_probe) {return -ENODEV ; } +static inline int sev_guest_deactivate (struct sev_data_deactivate *data,int *error) {return -ENODEV ; }
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 25/64] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
Quoted 112 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Before SNP VMs can be launched, the platform must be appropriately configured and initialized. Platform initialization is accomplished via the
SNP_INITcommand. Make sure to do a WBINVD and issueDF_FLUSHcommand to prepare for the first SNP guest launch after INIT.During the execution of
SNP_INITcommand, the firmware configures and enables SNP security policy enforcement in many system components. Some system components write to regions of memory reserved by early x86 firmware (e.g. UEFI). Other system components write to regions provided by the operation system, hypervisor, or x86 firmware. Such system components can only write to HV-fixed pages or Default pages. They will error when attempting to write to other page states afterSNP_INITenables their SNP enforcement.Starting in SNP firmware v1.52, the
SNP_INIT_EXcommand takes a list of system physical address ranges to convert into the HV-fixed page states during the RMP initialization. IfINIT_RMPis 1, hypervisors should provide all system physical address ranges that the hypervisor will never assign to a guest until the next RMP re-initialization. For instance, the memory that UEFI reserves should be included in the range list. This allows system components that occasionally write to memory (e.g. logging to UEFI reserved regions) to not fail due to RMP initialization and SNP enablement.Co-developed-by Ashish Kalra <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 225 +++++++++++++++++++++++++++++++++++ drivers/crypto/ccp/sev-dev.h | 2 + include/linux/psp-sev.h | 17 +++ 3 files changed, 244 insertions(+)drivers/crypto/ccp/sev-dev.c +47
-0@@ -26,6 +26,7 @@#include <linux/fs_struct.h> #include <asm/smp.h> +#include <asm/e820/types.h> #include "psp-dev.h" #include "sev-dev.h" @@ -34,6 +35,10 @@#define SEV_FW_FILE "amd/sev.fw"#define SEV_FW_NAME_SIZE 64 +/* Minimum firmware version required for the SEV-SNP support */ +#define SNP_MIN_API_MAJOR 1 +#define SNP_MIN_API_MINOR 51 +static DEFINE_MUTEX (sev_cmd_mutex);static struct sev_misc_dev *misc_dev;@@ -76,6 +81,13 @@ static void *sev_es_tmr;#define NV_LENGTH (32 * 1024)static void *sev_init_ex_buffer; +/* +* SEV_DATA_RANGE_LIST: +* Array containing range of pages that firmware transitions to HV-fixed +* page state. +*/ +struct sev_data_range_list *snp_range_list; +static inline bool sev_version_greater_or_equal (u8 maj,u8 min) {struct sev_device *sev = psp_master->sev_data;@@ -830,6 +842,186 @@ static int sev_update_firmware(struct device *dev)return ret; } +static void snp_set_hsave_pa (void *arg) +{ +wrmsrl (MSR_VM_HSAVE_PA ,0 ); +} + +static int snp_filter_reserved_mem_regions (struct resource *rs,void *arg) +{ +struct sev_data_range_list *range_list = arg; +struct sev_data_range *range = &range_list->ranges[range_list->num_elements]; +size_t size; + +if ((range_list->num_elements *sizeof (struct sev_data_range ) + +sizeof (struct sev_data_range_list )) >PAGE_SIZE ) +return -E2BIG ; + +switch (rs->desc) { +case E820_TYPE_RESERVED : +case E820_TYPE_PMEM : +case E820_TYPE_ACPI : + range->base = rs->start &PAGE_MASK ; + size = (rs->end +1 ) - rs->start; + range->page_count = size >>PAGE_SHIFT ; + range_list->num_elements++; +break ; +default : +break ; + } + +return 0 ; +} + +static int __sev_snp_init_locked (int *error) +{ +struct psp_device *psp = psp_master; +struct sev_data_snp_init_ex data;
drivers/crypto/ccp/sev-dev.c static int sev_update_firmware(struct device *dev) · line 889
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+struct sev_device *sev; +int rc =0 ; + +if (!psp || !psp->sev_data) +return -ENODEV ; + + sev = psp->sev_data; + +if (sev->snp_initialized) +return 0 ;
Shouldn't this follow this check:
if (sev->state == SEV_STATE_INIT) {
/* debug printk about possible incorrect call order */
return -ENODEV;
}It is game over for SNP, if SEV_CMD_INIT{_EX} got first, which means that this should not proceed.
BR, Jarkko
Kalra, Ashish
maintainerRe: [PATCH RFC v7 25/64] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
Hello Jarkko,
Quoted 126 lines · Jarkko Sakkinen
Quoted 112 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Before SNP VMs can be launched, the platform must be appropriately configured and initialized. Platform initialization is accomplished via the
SNP_INITcommand. Make sure to do a WBINVD and issueDF_FLUSHcommand to prepare for the first SNP guest launch after INIT.During the execution of
SNP_INITcommand, the firmware configures and enables SNP security policy enforcement in many system components. Some system components write to regions of memory reserved by early x86 firmware (e.g. UEFI). Other system components write to regions provided by the operation system, hypervisor, or x86 firmware. Such system components can only write to HV-fixed pages or Default pages. They will error when attempting to write to other page states afterSNP_INITenables their SNP enforcement.Starting in SNP firmware v1.52, the
SNP_INIT_EXcommand takes a list of system physical address ranges to convert into the HV-fixed page states during the RMP initialization. IfINIT_RMPis 1, hypervisors should provide all system physical address ranges that the hypervisor will never assign to a guest until the next RMP re-initialization. For instance, the memory that UEFI reserves should be included in the range list. This allows system components that occasionally write to memory (e.g. logging to UEFI reserved regions) to not fail due to RMP initialization and SNP enablement.Co-developed-by Ashish Kalra <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 225 +++++++++++++++++++++++++++++++++++ drivers/crypto/ccp/sev-dev.h | 2 + include/linux/psp-sev.h | 17 +++ 3 files changed, 244 insertions(+)drivers/crypto/ccp/sev-dev.c +47
-0@@ -26,6 +26,7 @@#include <linux/fs_struct.h> #include <asm/smp.h> +#include <asm/e820/types.h> #include "psp-dev.h" #include "sev-dev.h" @@ -34,6 +35,10 @@#define SEV_FW_FILE "amd/sev.fw"#define SEV_FW_NAME_SIZE 64 +/* Minimum firmware version required for the SEV-SNP support */ +#define SNP_MIN_API_MAJOR 1 +#define SNP_MIN_API_MINOR 51 +static DEFINE_MUTEX (sev_cmd_mutex);static struct sev_misc_dev *misc_dev;@@ -76,6 +81,13 @@ static void *sev_es_tmr;#define NV_LENGTH (32 * 1024)static void *sev_init_ex_buffer; +/* +* SEV_DATA_RANGE_LIST: +* Array containing range of pages that firmware transitions to HV-fixed +* page state. +*/ +struct sev_data_range_list *snp_range_list; +static inline bool sev_version_greater_or_equal (u8 maj,u8 min) {struct sev_device *sev = psp_master->sev_data;@@ -830,6 +842,186 @@ static int sev_update_firmware(struct device *dev)return ret; } +static void snp_set_hsave_pa (void *arg) +{ +wrmsrl (MSR_VM_HSAVE_PA ,0 ); +} + +static int snp_filter_reserved_mem_regions (struct resource *rs,void *arg) +{ +struct sev_data_range_list *range_list = arg; +struct sev_data_range *range = &range_list->ranges[range_list->num_elements]; +size_t size; + +if ((range_list->num_elements *sizeof (struct sev_data_range ) + +sizeof (struct sev_data_range_list )) >PAGE_SIZE ) +return -E2BIG ; + +switch (rs->desc) { +case E820_TYPE_RESERVED : +case E820_TYPE_PMEM : +case E820_TYPE_ACPI : + range->base = rs->start &PAGE_MASK ; + size = (rs->end +1 ) - rs->start; + range->page_count = size >>PAGE_SHIFT ; + range_list->num_elements++; +break ; +default : +break ; + } + +return 0 ; +} + +static int __sev_snp_init_locked (int *error) +{ +struct psp_device *psp = psp_master; +struct sev_data_snp_init_ex data;drivers/crypto/ccp/sev-dev.c static int sev_update_firmware(struct device *dev) · line 889
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+struct sev_device *sev; +int rc =0 ; + +if (!psp || !psp->sev_data) +return -ENODEV ; + + sev = psp->sev_data; + +if (sev->snp_initialized) +return 0 ;Shouldn't this follow this check:
Quoted 7 lines
if (sev->state == SEV_STATE_INIT) { /* debug printk about possible incorrect call order */ return -ENODEV; }It is game over for SNP, if SEV_CMD_INIT{_EX} got first, which means that this should not proceed.
But, how will SEV_CMD_INIT_EX happen before as sev_pci_init() which is invoked during CCP module load/initialization, will first try to do sev_snp_init() if SNP is supported, before it invokes sev_platform_init() to do SEV firmware initialization ?
Thanks,
Ashish
Re: [PATCH RFC v7 25/64] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
Quoted 128 lines · Ashish
Hello Jarkko,
Quoted 125 lines · Jarkko Sakkinen
Quoted 106 lines · Michael Roth
From: Brijesh Singh <[email protected]>
Before SNP VMs can be launched, the platform must be appropriately configured and initialized. Platform initialization is accomplished via the
SNP_INITcommand. Make sure to do a WBINVD and issueDF_FLUSHcommand to prepare for the first SNP guest launch after INIT.During the execution of
SNP_INITcommand, the firmware configures and enables SNP security policy enforcement in many system components. Some system components write to regions of memory reserved by early x86 firmware (e.g. UEFI). Other system components write to regions provided by the operation system, hypervisor, or x86 firmware. Such system components can only write to HV-fixed pages or Default pages. They will error when attempting to write to other page states afterSNP_INITenables their SNP enforcement.Starting in SNP firmware v1.52, the
SNP_INIT_EXcommand takes a list of system physical address ranges to convert into the HV-fixed page states during the RMP initialization. IfINIT_RMPis 1, hypervisors should provide all system physical address ranges that the hypervisor will never assign to a guest until the next RMP re-initialization. For instance, the memory that UEFI reserves should be included in the range list. This allows system components that occasionally write to memory (e.g. logging to UEFI reserved regions) to not fail due to RMP initialization and SNP enablement.Co-developed-by Ashish Kalra <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 225 +++++++++++++++++++++++++++++++++++ drivers/crypto/ccp/sev-dev.h | 2 + include/linux/psp-sev.h | 17 +++ 3 files changed, 244 insertions(+)drivers/crypto/ccp/sev-dev.c +47
-0@@ -26,6 +26,7 @@#include <linux/fs_struct.h> #include <asm/smp.h> +#include <asm/e820/types.h> #include "psp-dev.h" #include "sev-dev.h" @@ -34,6 +35,10 @@#define SEV_FW_FILE "amd/sev.fw"#define SEV_FW_NAME_SIZE 64 +/* Minimum firmware version required for the SEV-SNP support */ +#define SNP_MIN_API_MAJOR 1 +#define SNP_MIN_API_MINOR 51 +static DEFINE_MUTEX (sev_cmd_mutex);static struct sev_misc_dev *misc_dev;@@ -76,6 +81,13 @@ static void *sev_es_tmr;#define NV_LENGTH (32 * 1024)static void *sev_init_ex_buffer; +/* +* SEV_DATA_RANGE_LIST: +* Array containing range of pages that firmware transitions to HV-fixed +* page state. +*/ +struct sev_data_range_list *snp_range_list; +static inline bool sev_version_greater_or_equal (u8 maj,u8 min) {struct sev_device *sev = psp_master->sev_data;@@ -830,6 +842,186 @@ static int sev_update_firmware(struct device *dev)return ret; } +static void snp_set_hsave_pa (void *arg) +{ +wrmsrl (MSR_VM_HSAVE_PA ,0 ); +} + +static int snp_filter_reserved_mem_regions (struct resource *rs,void *arg) +{ +struct sev_data_range_list *range_list = arg; +struct sev_data_range *range = &range_list->ranges[range_list->num_elements]; +size_t size; + +if ((range_list->num_elements *sizeof (struct sev_data_range ) + +sizeof (struct sev_data_range_list )) >PAGE_SIZE ) +return -E2BIG ; + +switch (rs->desc) { +case E820_TYPE_RESERVED : +case E820_TYPE_PMEM : +case E820_TYPE_ACPI : + range->base = rs->start &PAGE_MASK ; + size = (rs->end +1 ) - rs->start; + range->page_count = size >>PAGE_SHIFT ; + range_list->num_elements++; +break ; +default : +break ; + } + +return 0 ; +} + +static int __sev_snp_init_locked (int *error) +{ +struct psp_device *psp = psp_master; +struct sev_data_snp_init_ex data;drivers/crypto/ccp/sev-dev.c static int sev_update_firmware(struct device *dev) · line 888
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+struct sev_device *sev; +int rc =0 ; + +if (!psp || !psp->sev_data) +return -ENODEV ; + + sev = psp->sev_data; + +if (sev->snp_initialized) +return 0 ;Shouldn't this follow this check:
if (sev->state == SEV_STATE_INIT) { /* debug printk about possible incorrect call order */ return -ENODEV; }
Quoted 8 lines
Quoted 2 lines
It is game over for SNP, if SEV_CMD_INIT{_EX} got first, which means that this should not proceed.
But, how will
SEV_CMD_INIT_EXhappen before assev_pci_init()which is invoked during CCP module load/initialization, will first try to dosev_snp_init()if SNP is supported, before it invokessev_platform_init()to do SEV firmware initialization ?
Because the symbol is exported outside the driver to be called by other subsystems, you need to have a santiy check for the call order, as it is a hardware constraint. Otherwise, any unconsidered change in either side could unknowingily break the kernel.
Alternatively, you could choose not to export sev_snp_init(). It is supported by the fact that the call in sev_guest_init() is does nothing useful (for the reasons you already wrote).
BR, Jarkko
Re: [PATCH RFC v7 25/64] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
In reply to Michael RothQuoted 9 lines · Michael Roth
drivers/crypto/ccp/sev-dev.c +9
-0+/* +* If boot CPU supports SNP, then first attempt to initialize +* the SNP firmware. +*/ +if (cpu_feature_enabled (X86_FEATURE_SEV_SNP )) { +if (!sev_version_greater_or_equal (SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR )) { +dev_err (sev->dev,"SEV-SNP support requires firmware version >= %d:%d\n" , +SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR ); + }else {
drivers/crypto/ccp/sev-dev.c void sev_pci_init(void) · line 1564
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+ rc =sev_snp_init (&error, true); +if (rc) { +/* +* Don't abort the probe if SNP INIT failed, +* continue to initialize the legacy SEV firmware. +*/ +dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); + } + } + }
I think this is not right as there is a dep between sev init and this, and there is about a dozen of call sites already __sev_platform_init_locked().
Instead there should be __sev_snp_init_locked() that would be called as part of __sev_platform_init_locked() flow.
Also TMR allocation should be moved inside __sev_platform_init_locked, given that it needs to be marked into RMP after SNP init.
BR, Jarkko
Kalra, Ashish
maintainerRe: [PATCH RFC v7 25/64] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
Hello Jarkko,
Quoted 15 lines · Jarkko Sakkinen
Quoted 14 lines · Michael Roth
drivers/crypto/ccp/sev-dev.c +14
-0+/* +* If boot CPU supports SNP, then first attempt to initialize +* the SNP firmware. +*/ +if (cpu_feature_enabled (X86_FEATURE_SEV_SNP )) { +if (!sev_version_greater_or_equal (SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR )) { +dev_err (sev->dev,"SEV-SNP support requires firmware version >= %d:%d\n" , +SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR ); + }else { + rc =sev_snp_init (&error, true); +if (rc) { +/* +* Don't abort the probe if SNP INIT failed, +* continue to initialize the legacy SEV firmware.
Quoted 9 lines
drivers/crypto/ccp/sev-dev.c void sev_pci_init(void) · line 1564
Quoted 5 lines
drivers/crypto/ccp/sev-dev.c +5
-0+*/ +dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); + } + } + }I think this is not right as there is a dep between sev init and this, and there is about a dozen of call sites already
__sev_platform_init_locked().
sev_init ?
As this is invoked during CCP module load/initialization, shouldn't this get invoked before any other call sites invoking __sev_platform_init_locked() ?
Thanks,
Ashish
Quoted 8 lines
Instead there should be
__sev_snp_init_locked()that would be called as part of__sev_platform_init_locked()flow.Also TMR allocation should be moved inside
__sev_platform_init_locked, given that it needs to be marked into RMP after SNP init.BR, Jarkko
Re: [PATCH RFC v7 25/64] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
Quoted 28 lines · Ashish
Hello Jarkko,
Quoted 15 lines · Jarkko Sakkinen
Quoted 14 lines · Michael Roth
drivers/crypto/ccp/sev-dev.c +14
-0+/* +* If boot CPU supports SNP, then first attempt to initialize +* the SNP firmware. +*/ +if (cpu_feature_enabled (X86_FEATURE_SEV_SNP )) { +if (!sev_version_greater_or_equal (SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR )) { +dev_err (sev->dev,"SEV-SNP support requires firmware version >= %d:%d\n" , +SNP_MIN_API_MAJOR ,SNP_MIN_API_MINOR ); + }else { + rc =sev_snp_init (&error, true); +if (rc) { +/* +* Don't abort the probe if SNP INIT failed, +* continue to initialize the legacy SEV firmware. Quoted 9 lines
drivers/crypto/ccp/sev-dev.c void sev_pci_init(void) · line 1564
Quoted 5 lines
drivers/crypto/ccp/sev-dev.c +5
-0+*/ +dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); + } + } + }I think this is not right as there is a dep between sev init and this, and there is about a dozen of call sites already
__sev_platform_init_locked().
Quoted 4 lines
sev_init?As this is invoked during CCP module load/initialization, shouldn't this get invoked before any other call sites invoking
__sev_platform_init_locked()?
Then it should not be exported because this the only working call site.
However, the benefit of __sev_platform_init_locked() addressing SNP init is that psp_init_on_probe can also postpone SNP init without possibility to any side effects (other call sites than sev_guest_init()).
BR, Jarkko
RFC PATCH v7 26/64 crypto:ccp: Provide API to issue SEV and SNP commands
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
Make sev_do_cmd() a generic API interface for the hypervisor to issue commands to manage an SEV and SNP guest. The commands for SEV and SNP are defined in the SEV and SEV-SNP firmware specifications.
--- drivers/crypto/ccp/sev-dev.c | 3 ++- include/linux/psp-sev.h | 17 +++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
drivers/crypto/ccp/sev-dev.c +2 -1
@@ -415,7 +415,7 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)return ret; } -static int sev_do_cmd (int cmd,void *data,int *psp_ret) +int sev_do_cmd (int cmd,void *data,int *psp_ret) {int rc;
@@ -425,6 +425,7 @@ static int sev_do_cmd(int cmd, void *data, int *psp_ret)return rc; } +EXPORT_SYMBOL_GPL (sev_do_cmd);static int __sev_init_locked (int *error) {
include/linux/psp-sev.h +17 -0
@@ -907,6 +907,20 @@ int sev_guest_df_flush(int *error);*/ int sev_guest_decommission (struct sev_data_decommission *data,int *error); +/** +* sev_do_cmd - perform SEV command +* +* @error: SEV command return code +* +* Returns: +* 0 if the SEV successfully processed the command +* -%ENODEV if the SEV device is not available +* -%ENOTSUPP if the SEV does not support SEV +* -%ETIMEDOUT if the SEV command timed out +* -%EIO if the SEV returned a non-zero return code +*/ +int sev_do_cmd (int cmd,void *data,int *psp_ret); +void *psp_copy_user_blob (u64 uaddr,u32 len);#else /* !CONFIG_CRYPTO_DEV_SP_PSP */
@@ -924,6 +938,9 @@ sev_guest_deactivate(struct sev_data_deactivate *data, int *error) { return -ENOstatic inline int sev_guest_decommission (struct sev_data_decommission *data,int *error) {return -ENODEV ; } +static inline int +sev_do_cmd (int cmd,void *data,int *psp_ret) {return -ENODEV ; } +static inline int sev_guest_activate (struct sev_data_activate *data,int *error) {return -ENODEV ; }
Signature · 1 line
2.25.1
RFC PATCH v7 27/64 crypto: ccp: Introduce snp leaked pages list
In reply to Michael RothFrom: Ashish Kalra <[email protected]>
Pages are unsafe to be released back to the page-allocator, if they have been transitioned to firmware/guest state and can't be reclaimed or transitioned back to hypervisor/shared state. In this case add them to an internal leaked pages list to ensure that they are not freed or touched/accessed to cause fatal page faults.
--- drivers/crypto/ccp/sev-dev.c | 28 ++++++++++++++++++++++++++++ include/linux/psp-sev.h | 8 ++++++++ 2 files changed, 36 insertions(+)
drivers/crypto/ccp/sev-dev.c +28 -0
@@ -42,6 +42,12 @@static DEFINE_MUTEX (sev_cmd_mutex);static struct sev_misc_dev *misc_dev; +/* list of pages which are leaked and cannot be reclaimed */ +static LIST_HEAD (snp_leaked_pages_list); +static DEFINE_SPINLOCK (snp_leaked_pages_list_lock); + +static atomic_long_t snp_nr_leaked_pages =ATOMIC_LONG_INIT (0 ); +static int psp_cmd_timeout =100 ;module_param (psp_cmd_timeout, int,0644 );MODULE_PARM_DESC (psp_cmd_timeout," default timeout value, in seconds, for PSP commands" );
@@ -188,6 +194,28 @@ static int sev_cmd_buffer_len(int cmd)return 0 ; } +void snp_mark_pages_offline (unsigned long pfn,unsigned int npages) +{ +struct page *page =pfn_to_page (pfn); + +WARN (1 ,"psc failed, pfn 0x%lx pages %d (marked offline)\n" , pfn, npages); + +spin_lock (&snp_leaked_pages_list_lock); +while (npages--) { +/* +* Reuse the page's buddy list for chaining into the leaked +* pages list. This page should not be on a free list currently +* and is also unsafe to be added to a free list. +*/ +list_add_tail (&page->buddy_list, &snp_leaked_pages_list); +sev_dump_rmpentry (pfn); + pfn++; + } +spin_unlock (&snp_leaked_pages_list_lock); +atomic_long_inc (&snp_nr_leaked_pages); +} +EXPORT_SYMBOL_GPL (snp_mark_pages_offline); +static void *sev_fw_alloc (unsigned long len) {struct page *page;
include/linux/psp-sev.h +8 -0
@@ -923,6 +923,12 @@ int sev_do_cmd(int cmd, void *data, int *psp_ret);void *psp_copy_user_blob (u64 uaddr,u32 len); +/** +* sev_mark_pages_offline - insert non-reclaimed firmware/guest pages +* into a leaked pages list. +*/ +void snp_mark_pages_offline (unsigned long pfn,unsigned int npages); +#else /* !CONFIG_CRYPTO_DEV_SP_PSP */ static inline int
@@ -951,6 +957,8 @@ sev_issue_cmd_external_user(struct file *filep, unsigned int id, void *data, intstatic inline void *psp_copy_user_blob (u64 __user uaddr,u32 len) {return ERR_PTR (-EINVAL ); } +void snp_mark_pages_offline (unsigned long pfn,unsigned int npages) {} +#endif /* CONFIG_CRYPTO_DEV_SP_PSP */ #endif /* __PSP_SEV_H__ */
Signature · 1 line
2.25.1
RFC PATCH v7 28/64 crypto: ccp: Handle the legacy TMR allocation when SNP is enabled
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
The behavior and requirement for the SEV-legacy command is altered when the SNP firmware is in the INIT state. See SEV-SNP firmware specification for more details.
Allocate the Trusted Memory Region (TMR) as a 2mb sized/aligned region when SNP is enabled to satisfy new requirements for the SNP. Continue allocating a 1mb region for !SNP configuration.
While at it, provide API that can be used by others to allocate a page that can be used by the firmware. The immediate user for this API will be the KVM driver. The KVM driver to need to allocate a firmware context page during the guest creation. The context page need to be updated by the firmware. See the SEV-SNP specification for further details.
--- drivers/crypto/ccp/sev-dev.c | 148 +++++++++++++++++++++++++++++++++-- include/linux/psp-sev.h | 9 +++ 2 files changed, 149 insertions(+), 8 deletions(-)
drivers/crypto/ccp/sev-dev.c +140 -8
@@ -94,6 +94,13 @@ static void *sev_init_ex_buffer;*/ struct sev_data_range_list *snp_range_list; +/* When SEV-SNP is enabled the TMR needs to be 2MB aligned and 2MB size. */ +#define SEV_SNP_ES_TMR_SIZE (2 * 1024 * 1024) + +static size_t sev_es_tmr_size =SEV_ES_TMR_SIZE ; + +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret); +static inline bool sev_version_greater_or_equal (u8 maj,u8 min) {struct sev_device *sev = psp_master->sev_data;
@@ -216,11 +223,134 @@ void snp_mark_pages_offline(unsigned long pfn, unsigned int npages) }EXPORT_SYMBOL_GPL (snp_mark_pages_offline); +static int snp_reclaim_pages (unsigned long paddr,unsigned int npages,bool locked) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int ret, err, i, n =0 ; + +if (!pfn_valid (pfn)) { +pr_err ("%s: Invalid PFN %lx\n" , __func__, pfn); +return 0 ; + } + +for (i =0 ; i < npages; i++, pfn++, n++) { + paddr = pfn <<PAGE_SHIFT ; + +if (locked) + ret =__sev_do_cmd_locked (SEV_CMD_SNP_PAGE_RECLAIM , &paddr, &err); +else + ret =sev_do_cmd (SEV_CMD_SNP_PAGE_RECLAIM , &paddr, &err); + +if (ret) + goto cleanup; + + ret =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (ret) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to reclaim the page then page is no longer safe to +* be release back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); +return ret; +} + +static int rmp_mark_pages_firmware (unsigned long paddr,unsigned int npages,bool locked) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, n++, pfn++) { + rc =rmp_make_private (pfn,0 ,PG_LEVEL_4K ,0 , true); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* Try unrolling the firmware state changes by +* reclaiming the pages which were already changed to the +* firmware state. +*/ +snp_reclaim_pages (paddr, n, locked); + +return rc; +} + +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) +{ +unsigned long npages =1ul << order, paddr; +struct sev_device *sev; +struct page *page; + +if (!psp_master || !psp_master->sev_data) +return NULL ; + + page =alloc_pages (gfp_mask, order); +if (!page) +return NULL ; + +/* If SEV-SNP is initialized then add the page in RMP table. */ + sev = psp_master->sev_data; +if (!sev->snp_initialized) +return page; + + paddr =__pa ((unsigned long )page_address (page)); +if (rmp_mark_pages_firmware (paddr, npages, locked)) +return NULL ; + +return page; +} + +void *snp_alloc_firmware_page (gfp_t gfp_mask) +{ +struct page *page; + + page =__snp_alloc_firmware_pages (gfp_mask,0 , false); + +return page ?page_address (page) :NULL ; +} +EXPORT_SYMBOL_GPL (snp_alloc_firmware_page); + +static void __snp_free_firmware_pages (struct page *page,int order,bool locked) +{ +struct sev_device *sev = psp_master->sev_data; +unsigned long paddr, npages =1ul << order; + +if (!page) +return ; + + paddr =__pa ((unsigned long )page_address (page)); +if (sev->snp_initialized && +snp_reclaim_pages (paddr, npages, locked)) +return ; + +__free_pages (page, order); +} + +void snp_free_firmware_page (void *addr) +{ +if (!addr) +return ; + +__snp_free_firmware_pages (virt_to_page (addr),0 , false); +} +EXPORT_SYMBOL_GPL (snp_free_firmware_page); +static void *sev_fw_alloc (unsigned long len) {struct page *page; - page =alloc_pages (GFP_KERNEL ,get_order (len)); + page =__snp_alloc_firmware_pages (GFP_KERNEL ,get_order (len), false);if (!page)return NULL ;
@@ -468,7 +598,7 @@ static int __sev_init_locked(int *error) data.tmr_address =__pa (sev_es_tmr); data.flags |=SEV_INIT_FLAGS_SEV_ES ; - data.tmr_len =SEV_ES_TMR_SIZE ; + data.tmr_len = sev_es_tmr_size; }return __sev_do_cmd_locked (SEV_CMD_INIT , &data, error);
@@ -491,7 +621,7 @@ static int __sev_init_ex_locked(int *error) data.tmr_address =__pa (sev_es_tmr); data.flags |=SEV_INIT_FLAGS_SEV_ES ; - data.tmr_len =SEV_ES_TMR_SIZE ; + data.tmr_len = sev_es_tmr_size; }return __sev_do_cmd_locked (SEV_CMD_INIT_EX , &data, error);
@@ -982,6 +1112,8 @@ static int __sev_snp_init_locked(int *error) sev->snp_initialized = true;dev_dbg (sev->dev,"SEV-SNP firmware initialized\n" ); + sev_es_tmr_size =SEV_SNP_ES_TMR_SIZE ; +return rc; }
@@ -1499,8 +1631,9 @@ static void sev_firmware_shutdown(struct sev_device *sev)/* The TMR area was encrypted, flush it from the cache */ wbinvd_on_all_cpus (); -free_pages ((unsigned long )sev_es_tmr, -get_order (SEV_ES_TMR_SIZE )); +__snp_free_firmware_pages (virt_to_page (sev_es_tmr), +get_order (sev_es_tmr_size), + false); sev_es_tmr =NULL ; }
@@ -1511,8 +1644,7 @@ static void sev_firmware_shutdown(struct sev_device *sev) }if (snp_range_list) { -free_pages ((unsigned long )snp_range_list, -get_order (PAGE_SIZE )); +snp_free_firmware_page (snp_range_list); snp_range_list =NULL ; }
@@ -1593,7 +1725,7 @@ void sev_pci_init(void) }/* Obtain the TMR memory area for SEV-ES use */ - sev_es_tmr =sev_fw_alloc (SEV_ES_TMR_SIZE ); + sev_es_tmr =sev_fw_alloc (sev_es_tmr_size);if (!sev_es_tmr)dev_warn (sev->dev,"SEV: TMR allocation failed, SEV-ES support unavailable\n" );
include/linux/psp-sev.h +9 -0
@@ -922,6 +922,8 @@ int sev_guest_decommission(struct sev_data_decommission *data, int *error);int sev_do_cmd (int cmd,void *data,int *psp_ret);void *psp_copy_user_blob (u64 uaddr,u32 len); +void *snp_alloc_firmware_page (gfp_t mask); +void snp_free_firmware_page (void *addr);/** * sev_mark_pages_offline - insert non-reclaimed firmware/guest pages
@@ -959,6 +961,13 @@ static inline void *psp_copy_user_blob(u64 __user uaddr, u32 len) { return ERR_Pvoid snp_mark_pages_offline (unsigned long pfn,unsigned int npages) {} +static inline void *snp_alloc_firmware_page (gfp_t mask) +{ +return NULL ; +} + +static inline void snp_free_firmware_page (void *addr) { } +#endif /* CONFIG_CRYPTO_DEV_SP_PSP */ #endif /* __PSP_SEV_H__ */
Signature · 1 line
2.25.1
RFC PATCH v7 29/64 crypto: ccp: Handle the legacy SEV command when SNP is enabled
In reply to Michael RothFrom: Brijesh Singh <[email protected]>
The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..
A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:
1) system physical address points to a guest memory
2) system physical address points to a host memory
To handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.
For the case #2, use a bounce buffer to complete the request.
--- drivers/crypto/ccp/sev-dev.c | 370 ++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/sev-dev.h | 12 ++ 2 files changed, 372 insertions(+), 10 deletions(-)
drivers/crypto/ccp/sev-dev.c +360 -10
@@ -286,6 +286,30 @@ static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, booreturn rc; } +static int rmp_mark_pages_shared (unsigned long paddr,unsigned int npages) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, pfn++, n++) { + rc =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to change the page state to shared, then its not safe +* to release the page back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); + +return rc; +} +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) {unsigned long npages =1ul << order, paddr;
@@ -487,12 +511,295 @@ static int sev_write_init_ex_file_if_required(int cmd_id)return sev_write_init_ex_file (); } +static int alloc_snp_host_map (struct sev_device *sev) +{ +struct page *page; +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +memset (map,0 ,sizeof (*map)); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (SEV_FW_BLOB_MAX_SIZE )); +if (!page) +return -ENOMEM ; + + map->host =page_address (page); + } + +return 0 ; +} + +static void free_snp_host_map (struct sev_device *sev) +{ +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +if (map->host) { +__free_pages (virt_to_page (map->host),get_order (SEV_FW_BLOB_MAX_SIZE )); +memset (map,0 ,sizeof (*map)); + } + } +} + +static int map_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + + map->active = false; + +if (!paddr || !len) +return 0 ; + + map->paddr = *paddr; + map->len = len; + +/* If paddr points to a guest memory then change the page state to firmwware. */ +if (guest) { +if (rmp_mark_pages_firmware (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +if (!map->host) +return -ENOMEM ; + +/* Check if the pre-allocated buffer can be used to fullfil the request. */ +if (len >SEV_FW_BLOB_MAX_SIZE ) +return -EINVAL ; + +/* Transition the pre-allocated buffer to the firmware state. */ +if (rmp_mark_pages_firmware (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Set the paddr to use pre-allocated firmware buffer */ + *paddr =__psp_pa (map->host); + +done: + map->active = true; +return 0 ; +} + +static int unmap_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + +if (!map->active) +return 0 ; + +/* If paddr points to a guest memory then restore the page state to hypervisor. */ +if (guest) { +if (snp_reclaim_pages (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +/* +* Transition the pre-allocated buffer to hypervisor state before the access. +* +* This is because while changing the page state to firmware, the kernel unmaps +* the pages from the direct map, and to restore the direct map the pages must +* be transitioned back to the shared state. +*/ +if (snp_reclaim_pages (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Copy the response data firmware buffer to the callers buffer. */ +memcpy (__va (__sme_clr (map->paddr)), map->host,min_t (size_t, len, map->len)); + *paddr = map->paddr; + +done: + map->active = false; +return 0 ; +} + +static bool sev_legacy_cmd_buf_writable (int cmd) +{ +switch (cmd) { +case SEV_CMD_PLATFORM_STATUS : +case SEV_CMD_GUEST_STATUS : +case SEV_CMD_LAUNCH_START : +case SEV_CMD_RECEIVE_START : +case SEV_CMD_LAUNCH_MEASURE : +case SEV_CMD_SEND_START : +case SEV_CMD_SEND_UPDATE_DATA : +case SEV_CMD_SEND_UPDATE_VMSA : +case SEV_CMD_PEK_CSR : +case SEV_CMD_PDH_CERT_EXPORT : +case SEV_CMD_GET_ID : +case SEV_CMD_ATTESTATION_REPORT : +return true; +default : +return false; + } +} + +#define prep_buffer (name, addr, len, guest, map) \ + func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map) + +static int __snp_cmd_buf_copy (int cmd,void *cmd_buf,bool to_fw,int fw_err) +{ +int (*func)(u64 *paddr,u32 len,bool guest,struct snp_host_map *map); +struct sev_device *sev = psp_master->sev_data; +bool from_fw = !to_fw; + +/* +* After the command is completed, change the command buffer memory to +* hypervisor state. +* +* The immutable bit is automatically cleared by the firmware, so +* no not need to reclaim the page. +*/ +if (from_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_shared (__pa (cmd_buf),1 )) +return -EFAULT ; + +/* No need to go further if firmware failed to execute command. */ +if (fw_err) +return 0 ; + } + +if (to_fw) + func = map_firmware_writeable; +else + func = unmap_firmware_writeable; + +/* +* A command buffer may contains a system physical address. If the address +* points to a host memory then use an intermediate firmware page otherwise +* change the page state in the RMP table. +*/ +switch (cmd) { +case SEV_CMD_PDH_CERT_EXPORT : + if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address, + pdh_cert_len, false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address, + cert_chain_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_GET_ID : + if (prep_buffer(struct sev_data_get_id, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_PEK_CSR : + if (prep_buffer(struct sev_data_pek_csr, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_DATA : + if (prep_buffer(struct sev_data_launch_update_data, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_VMSA : + if (prep_buffer(struct sev_data_launch_update_vmsa, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_MEASURE : + if (prep_buffer(struct sev_data_launch_measure, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_SECRET : + if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_DECRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, false, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_ENCRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, true, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_ATTESTATION_REPORT : + if (prep_buffer(struct sev_data_attestation_report, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_START : + if (prep_buffer(struct sev_data_send_start, session_address, + session_len, false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_DATA : + if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_data, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_VMSA : + if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_vmsa, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_DATA : + if (prep_buffer(struct sev_data_receive_update_data, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_VMSA : + if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +default : +break ; + } + +/* The command buffer need to be in the firmware state. */ +if (to_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_firmware (__pa (cmd_buf),1 , true)) +return -EFAULT ; + } + +return 0 ; + +err: +return -EINVAL ; +} + +static inline bool need_firmware_copy (int cmd) +{ +struct sev_device *sev = psp_master->sev_data; + +/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */ +return ((cmd <SEV_CMD_SNP_INIT ) && sev->snp_initialized) ? true : false; +} + +static int snp_aware_copy_to_firmware (int cmd,void *data) +{ +return __snp_cmd_buf_copy (cmd, data, true,0 ); +} + +static int snp_aware_copy_from_firmware (int cmd,void *data,int fw_err) +{ +return __snp_cmd_buf_copy (cmd, data, false, fw_err); +} +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret) {struct psp_device *psp = psp_master;struct sev_device *sev;unsigned int phys_lsb, phys_msb;unsigned int reg, ret =0 ; +void *cmd_buf;int buf_len;if (!psp || !psp->sev_data)
@@ -512,12 +819,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)* work for some memory, e.g. vmalloc'd addresses, and @data may not be * physically contiguous. */ -if (data) -memcpy (sev->cmd_buf, data, buf_len); +if (data) { +if (sev->cmd_buf_active >2 ) +return -EBUSY ; + + cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf; + +memcpy (cmd_buf, data, buf_len); + sev->cmd_buf_active++; + +/* +* The behavior of the SEV-legacy commands is altered when the +* SNP firmware is in the INIT state. +*/ +if (need_firmware_copy (cmd) &&snp_aware_copy_to_firmware (cmd, sev->cmd_buf)) +return -EFAULT ; + }else { + cmd_buf = sev->cmd_buf; + }/* Get the physical address of the command buffer */ - phys_lsb = data ?lower_32_bits (__psp_pa (sev->cmd_buf)) :0 ; - phys_msb = data ?upper_32_bits (__psp_pa (sev->cmd_buf)) :0 ; + phys_lsb = data ?lower_32_bits (__psp_pa (cmd_buf)) :0 ; + phys_msb = data ?upper_32_bits (__psp_pa (cmd_buf)) :0 ;dev_dbg (sev->dev,"sev command id %#x buffer 0x%08x%08x timeout %us\n" , cmd, phys_msb, phys_lsb, psp_timeout);
@@ -560,15 +883,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) ret =sev_write_init_ex_file_if_required (cmd); } -print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, - buf_len, false); -/* * Copy potential output from the PSP back to data. Do this even on * failure in case the caller wants to glean something from the error. */ -if (data) -memcpy (data, sev->cmd_buf, buf_len); +if (data) { +/* +* Restore the page state after the command completes. +*/ +if (need_firmware_copy (cmd) && +snp_aware_copy_from_firmware (cmd, cmd_buf, ret)) +return -EFAULT ; + +memcpy (data, cmd_buf, buf_len); + sev->cmd_buf_active--; + } + +print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, + buf_len, false);return ret; }
@@ -1579,10 +1911,12 @@ int sev_dev_init(struct psp_device *psp)if (!sev) goto e_err; - sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,0 ); + sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,1 );if (!sev->cmd_buf) goto e_sev; + sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf +PAGE_SIZE ; + psp->sev_data = sev; sev->dev = dev;
@@ -1648,6 +1982,12 @@ static void sev_firmware_shutdown(struct sev_device *sev) snp_range_list =NULL ; } +/* +* The host map need to clear the immutable bit so it must be free'd before the +* SNP firmware shutdown. +*/ +free_snp_host_map (sev); +sev_snp_shutdown (&error); }
@@ -1722,6 +2062,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); } } + +/* +* Allocate the intermediate buffers used for the legacy command handling. +*/ +if (alloc_snp_host_map (sev)) { +dev_notice (sev->dev,"Failed to alloc host map (disabling legacy SEV)\n" ); + goto skip_legacy; + } }/* Obtain the TMR memory area for SEV-ES use */
@@ -1739,12 +2087,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +skip_legacy:dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ?"-SNP" :"" , sev->api_major, sev->api_minor, sev->build);return ; err: +free_snp_host_map (sev); psp_master->sev_data =NULL ; }
drivers/crypto/ccp/sev-dev.h +12 -0
@@ -29,11 +29,20 @@#define SEV_CMDRESP_CMD_SHIFT 16#define SEV_CMDRESP_IOC BIT(0) +#define MAX_SNP_HOST_MAP_BUFS 2 +struct sev_misc_dev {struct kref refcount;struct miscdevice misc; }; +struct snp_host_map { +u64 paddr; +u32 len; +void *host; +bool active; +}; +struct sev_device {struct device *dev;struct psp_device *psp;
@@ -52,8 +61,11 @@ struct sev_device {u8 build;void *cmd_buf; +void *cmd_buf_backup; +int cmd_buf_active;bool snp_initialized; +struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS ]; };int sev_dev_init (struct psp_device *psp);
Signature · 1 line
2.25.1
Re: [PATCH RFC v7 29/64] crypto: ccp: Handle the legacy SEV command when SNP is enabled
Quoted 204 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..
A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:
1) system physical address points to a guest memory
2) system physical address points to a host memoryTo handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.
For the case #2, use a bounce buffer to complete the request.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 370 ++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/sev-dev.h | 12 ++ 2 files changed, 372 insertions(+), 10 deletions(-)drivers/crypto/ccp/sev-dev.c +161
-0@@ -286,6 +286,30 @@ static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, booreturn rc; } +static int rmp_mark_pages_shared (unsigned long paddr,unsigned int npages) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, pfn++, n++) { + rc =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to change the page state to shared, then its not safe +* to release the page back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); + +return rc; +} +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) {unsigned long npages =1ul << order, paddr;@@ -487,12 +511,295 @@ static int sev_write_init_ex_file_if_required(int cmd_id)return sev_write_init_ex_file (); } +static int alloc_snp_host_map (struct sev_device *sev) +{ +struct page *page; +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +memset (map,0 ,sizeof (*map)); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (SEV_FW_BLOB_MAX_SIZE )); +if (!page) +return -ENOMEM ; + + map->host =page_address (page); + } + +return 0 ; +} + +static void free_snp_host_map (struct sev_device *sev) +{ +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +if (map->host) { +__free_pages (virt_to_page (map->host),get_order (SEV_FW_BLOB_MAX_SIZE )); +memset (map,0 ,sizeof (*map)); + } + } +} + +static int map_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + + map->active = false; + +if (!paddr || !len) +return 0 ; + + map->paddr = *paddr; + map->len = len; + +/* If paddr points to a guest memory then change the page state to firmwware. */ +if (guest) { +if (rmp_mark_pages_firmware (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +if (!map->host) +return -ENOMEM ; + +/* Check if the pre-allocated buffer can be used to fullfil the request. */ +if (len >SEV_FW_BLOB_MAX_SIZE ) +return -EINVAL ; + +/* Transition the pre-allocated buffer to the firmware state. */ +if (rmp_mark_pages_firmware (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Set the paddr to use pre-allocated firmware buffer */ + *paddr =__psp_pa (map->host); + +done: + map->active = true; +return 0 ; +} + +static int unmap_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + +if (!map->active) +return 0 ; + +/* If paddr points to a guest memory then restore the page state to hypervisor. */ +if (guest) { +if (snp_reclaim_pages (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +/* +* Transition the pre-allocated buffer to hypervisor state before the access. +* +* This is because while changing the page state to firmware, the kernel unmaps +* the pages from the direct map, and to restore the direct map the pages must +* be transitioned back to the shared state. +*/ +if (snp_reclaim_pages (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Copy the response data firmware buffer to the callers buffer. */ +memcpy (__va (__sme_clr (map->paddr)), map->host,min_t (size_t, len, map->len)); + *paddr = map->paddr; + +done: + map->active = false; +return 0 ; +} + +static bool sev_legacy_cmd_buf_writable (int cmd) +{ +switch (cmd) { +case SEV_CMD_PLATFORM_STATUS : +case SEV_CMD_GUEST_STATUS : +case SEV_CMD_LAUNCH_START : +case SEV_CMD_RECEIVE_START : +case SEV_CMD_LAUNCH_MEASURE : +case SEV_CMD_SEND_START : +case SEV_CMD_SEND_UPDATE_DATA : +case SEV_CMD_SEND_UPDATE_VMSA : +case SEV_CMD_PEK_CSR : +case SEV_CMD_PDH_CERT_EXPORT : +case SEV_CMD_GET_ID : +case SEV_CMD_ATTESTATION_REPORT : +return true; +default : +return false; + } +} + +#define prep_buffer (name, addr, len, guest, map) \ + func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map) + +static int __snp_cmd_buf_copy (int cmd,void *cmd_buf,bool to_fw,int fw_err) +{ +int (*func)(u64 *paddr,u32 len,bool guest,struct snp_host_map *map); +struct sev_device *sev = psp_master->sev_data; +bool from_fw = !to_fw; +
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 660
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+/* +* After the command is completed, change the command buffer memory to +* hypervisor state. +* +* The immutable bit is automatically cleared by the firmware, so +* no not need to reclaim the page. +*/ +if (from_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_shared (__pa (cmd_buf),1 )) +return -EFAULT ;
If we return here, we will skip calling unmap_firmware_writeable and we will leak some pages in firmware state.
Quoted 99 lines
drivers/crypto/ccp/sev-dev.c +99
-0+ +/* No need to go further if firmware failed to execute command. */ +if (fw_err) +return 0 ; + } + +if (to_fw) + func = map_firmware_writeable; +else + func = unmap_firmware_writeable; + +/* +* A command buffer may contains a system physical address. If the address +* points to a host memory then use an intermediate firmware page otherwise +* change the page state in the RMP table. +*/ +switch (cmd) { +case SEV_CMD_PDH_CERT_EXPORT : + if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address, + pdh_cert_len, false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address, + cert_chain_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_GET_ID : + if (prep_buffer(struct sev_data_get_id, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_PEK_CSR : + if (prep_buffer(struct sev_data_pek_csr, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_DATA : + if (prep_buffer(struct sev_data_launch_update_data, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_VMSA : + if (prep_buffer(struct sev_data_launch_update_vmsa, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_MEASURE : + if (prep_buffer(struct sev_data_launch_measure, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_SECRET : + if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_DECRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, false, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_ENCRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, true, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_ATTESTATION_REPORT : + if (prep_buffer(struct sev_data_attestation_report, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_START : + if (prep_buffer(struct sev_data_send_start, session_address, + session_len, false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_DATA : + if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_data, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_VMSA : + if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_vmsa, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_DATA : + if (prep_buffer(struct sev_data_receive_update_data, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_VMSA : + if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address, + guest_len, true, &sev->snp_host_map[0 ]))
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 769
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+ goto err; +break ; +default : +break ; + } + +/* The command buffer need to be in the firmware state. */ +if (to_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_firmware (__pa (cmd_buf),1 , true)) +return -EFAULT ;
This function moves two separate pages to firmware state. First calling map_firmware_writeable and second calling rmp_mark_pages_firmware for cmd_buf.
In case rmp_mark_pages_firmware fails for cmd_buf, the page which has already moved to firmware state in map_firmware_writeable should be reclaimed.
This is a problem especially if we leak a guest owned page in firmware state. Since this is used only by legacy SEV VMs, these leaked pages will never be reclaimed back when destroying these VMs.
Quoted 197 lines
drivers/crypto/ccp/sev-dev.c +80
-10+ } + +return 0 ; + +err: +return -EINVAL ; +} + +static inline bool need_firmware_copy (int cmd) +{ +struct sev_device *sev = psp_master->sev_data; + +/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */ +return ((cmd <SEV_CMD_SNP_INIT ) && sev->snp_initialized) ? true : false; +} + +static int snp_aware_copy_to_firmware (int cmd,void *data) +{ +return __snp_cmd_buf_copy (cmd, data, true,0 ); +} + +static int snp_aware_copy_from_firmware (int cmd,void *data,int fw_err) +{ +return __snp_cmd_buf_copy (cmd, data, false, fw_err); +} +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret) {struct psp_device *psp = psp_master;struct sev_device *sev;unsigned int phys_lsb, phys_msb;unsigned int reg, ret =0 ; +void *cmd_buf;int buf_len;if (!psp || !psp->sev_data)@@ -512,12 +819,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)* work for some memory, e.g. vmalloc'd addresses, and @data may not be * physically contiguous. */ -if (data) -memcpy (sev->cmd_buf, data, buf_len); +if (data) { +if (sev->cmd_buf_active >2 ) +return -EBUSY ; + + cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf; + +memcpy (cmd_buf, data, buf_len); + sev->cmd_buf_active++; + +/* +* The behavior of the SEV-legacy commands is altered when the +* SNP firmware is in the INIT state. +*/ +if (need_firmware_copy (cmd) &&snp_aware_copy_to_firmware (cmd, sev->cmd_buf)) +return -EFAULT ; + }else { + cmd_buf = sev->cmd_buf; + }/* Get the physical address of the command buffer */ - phys_lsb = data ?lower_32_bits (__psp_pa (sev->cmd_buf)) :0 ; - phys_msb = data ?upper_32_bits (__psp_pa (sev->cmd_buf)) :0 ; + phys_lsb = data ?lower_32_bits (__psp_pa (cmd_buf)) :0 ; + phys_msb = data ?upper_32_bits (__psp_pa (cmd_buf)) :0 ;dev_dbg (sev->dev,"sev command id %#x buffer 0x%08x%08x timeout %us\n" , cmd, phys_msb, phys_lsb, psp_timeout);@@ -560,15 +883,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) ret =sev_write_init_ex_file_if_required (cmd); } -print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, - buf_len, false); -/* * Copy potential output from the PSP back to data. Do this even on * failure in case the caller wants to glean something from the error. */ -if (data) -memcpy (data, sev->cmd_buf, buf_len); +if (data) { +/* +* Restore the page state after the command completes. +*/ +if (need_firmware_copy (cmd) && +snp_aware_copy_from_firmware (cmd, cmd_buf, ret)) +return -EFAULT ; + +memcpy (data, cmd_buf, buf_len); + sev->cmd_buf_active--; + } + +print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, + buf_len, false);return ret; }@@ -1579,10 +1911,12 @@ int sev_dev_init(struct psp_device *psp)if (!sev) goto e_err; - sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,0 ); + sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,1 );if (!sev->cmd_buf) goto e_sev; + sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf +PAGE_SIZE ; + psp->sev_data = sev; sev->dev = dev;@@ -1648,6 +1982,12 @@ static void sev_firmware_shutdown(struct sev_device *sev) snp_range_list =NULL ; } +/* +* The host map need to clear the immutable bit so it must be free'd before the +* SNP firmware shutdown. +*/ +free_snp_host_map (sev); +sev_snp_shutdown (&error); }@@ -1722,6 +2062,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); } } + +/* +* Allocate the intermediate buffers used for the legacy command handling. +*/ +if (alloc_snp_host_map (sev)) { +dev_notice (sev->dev,"Failed to alloc host map (disabling legacy SEV)\n" ); + goto skip_legacy; + } }/* Obtain the TMR memory area for SEV-ES use */ @@ -1739,12 +2087,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +skip_legacy:dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ?"-SNP" :"" , sev->api_major, sev->api_minor, sev->build);return ; err: +free_snp_host_map (sev); psp_master->sev_data =NULL ; }drivers/crypto/ccp/sev-dev.h +12
-1@@ -29,11 +29,20 @@#define SEV_CMDRESP_CMD_SHIFT 16#define SEV_CMDRESP_IOC BIT(0) +#define MAX_SNP_HOST_MAP_BUFS 2 +struct sev_misc_dev {struct kref refcount;struct miscdevice misc; }; +struct snp_host_map { +u64 paddr; +u32 len; +void *host; +bool active; +}; +struct sev_device {struct device *dev;struct psp_device *psp;@@ -52,8 +61,11 @@ struct sev_device {u8 build;void *cmd_buf; +void *cmd_buf_backup; +int cmd_buf_active;bool snp_initialized; +struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS ]; };int sev_dev_init (struct psp_device *psp); --2.25.1
Kalra, Ashish
maintainerRe: [PATCH RFC v7 29/64] crypto: ccp: Handle the legacy SEV command when SNP is enabled
Hello Alper,
Quoted 210 lines · Alper Gun
Quoted 209 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..
A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:
1) system physical address points to a guest memory
2) system physical address points to a host memoryTo handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.
For the case #2, use a bounce buffer to complete the request.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 370 ++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/sev-dev.h | 12 ++ 2 files changed, 372 insertions(+), 10 deletions(-)drivers/crypto/ccp/sev-dev.c +166
-0@@ -286,6 +286,30 @@ static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, booreturn rc; } +static int rmp_mark_pages_shared (unsigned long paddr,unsigned int npages) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, pfn++, n++) { + rc =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to change the page state to shared, then its not safe +* to release the page back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); + +return rc; +} +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) {unsigned long npages =1ul << order, paddr;@@ -487,12 +511,295 @@ static int sev_write_init_ex_file_if_required(int cmd_id)return sev_write_init_ex_file (); } +static int alloc_snp_host_map (struct sev_device *sev) +{ +struct page *page; +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +memset (map,0 ,sizeof (*map)); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (SEV_FW_BLOB_MAX_SIZE )); +if (!page) +return -ENOMEM ; + + map->host =page_address (page); + } + +return 0 ; +} + +static void free_snp_host_map (struct sev_device *sev) +{ +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +if (map->host) { +__free_pages (virt_to_page (map->host),get_order (SEV_FW_BLOB_MAX_SIZE )); +memset (map,0 ,sizeof (*map)); + } + } +} + +static int map_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + + map->active = false; + +if (!paddr || !len) +return 0 ; + + map->paddr = *paddr; + map->len = len; + +/* If paddr points to a guest memory then change the page state to firmwware. */ +if (guest) { +if (rmp_mark_pages_firmware (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +if (!map->host) +return -ENOMEM ; + +/* Check if the pre-allocated buffer can be used to fullfil the request. */ +if (len >SEV_FW_BLOB_MAX_SIZE ) +return -EINVAL ; + +/* Transition the pre-allocated buffer to the firmware state. */ +if (rmp_mark_pages_firmware (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Set the paddr to use pre-allocated firmware buffer */ + *paddr =__psp_pa (map->host); + +done: + map->active = true; +return 0 ; +} + +static int unmap_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + +if (!map->active) +return 0 ; + +/* If paddr points to a guest memory then restore the page state to hypervisor. */ +if (guest) { +if (snp_reclaim_pages (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +/* +* Transition the pre-allocated buffer to hypervisor state before the access. +* +* This is because while changing the page state to firmware, the kernel unmaps +* the pages from the direct map, and to restore the direct map the pages must +* be transitioned back to the shared state. +*/ +if (snp_reclaim_pages (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Copy the response data firmware buffer to the callers buffer. */ +memcpy (__va (__sme_clr (map->paddr)), map->host,min_t (size_t, len, map->len)); + *paddr = map->paddr; + +done: + map->active = false; +return 0 ; +} + +static bool sev_legacy_cmd_buf_writable (int cmd) +{ +switch (cmd) { +case SEV_CMD_PLATFORM_STATUS : +case SEV_CMD_GUEST_STATUS : +case SEV_CMD_LAUNCH_START : +case SEV_CMD_RECEIVE_START : +case SEV_CMD_LAUNCH_MEASURE : +case SEV_CMD_SEND_START : +case SEV_CMD_SEND_UPDATE_DATA : +case SEV_CMD_SEND_UPDATE_VMSA : +case SEV_CMD_PEK_CSR : +case SEV_CMD_PDH_CERT_EXPORT : +case SEV_CMD_GET_ID : +case SEV_CMD_ATTESTATION_REPORT : +return true; +default : +return false; + } +} + +#define prep_buffer (name, addr, len, guest, map) \ + func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map) + +static int __snp_cmd_buf_copy (int cmd,void *cmd_buf,bool to_fw,int fw_err) +{ +int (*func)(u64 *paddr,u32 len,bool guest,struct snp_host_map *map); +struct sev_device *sev = psp_master->sev_data; +bool from_fw = !to_fw; + +/* +* After the command is completed, change the command buffer memory to +* hypervisor state. +* +* The immutable bit is automatically cleared by the firmware, so
Quoted 8 lines
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 660
Quoted 5 lines
drivers/crypto/ccp/sev-dev.c +5
-0+* no not need to reclaim the page. +*/ +if (from_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_shared (__pa (cmd_buf),1 )) +return -EFAULT ;If we return here, we will skip calling
unmap_firmware_writeableand we will leak some pages in firmware state.
Do you mean those (guest) pages which were transitioned to firmware state as part of snp_aware_copy_to_firmware()->_snp_cmd_buf_copy()->map_firmware_writeable()?
Quoted 112 lines
Quoted 100 lines
drivers/crypto/ccp/sev-dev.c +99
-0+ +/* No need to go further if firmware failed to execute command. */ +if (fw_err) +return 0 ; + } + +if (to_fw) + func = map_firmware_writeable; +else + func = unmap_firmware_writeable; + +/* +* A command buffer may contains a system physical address. If the address +* points to a host memory then use an intermediate firmware page otherwise +* change the page state in the RMP table. +*/ +switch (cmd) { +case SEV_CMD_PDH_CERT_EXPORT : + if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address, + pdh_cert_len, false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address, + cert_chain_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_GET_ID : + if (prep_buffer(struct sev_data_get_id, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_PEK_CSR : + if (prep_buffer(struct sev_data_pek_csr, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_DATA : + if (prep_buffer(struct sev_data_launch_update_data, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_VMSA : + if (prep_buffer(struct sev_data_launch_update_vmsa, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_MEASURE : + if (prep_buffer(struct sev_data_launch_measure, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_SECRET : + if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_DECRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, false, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_ENCRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, true, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_ATTESTATION_REPORT : + if (prep_buffer(struct sev_data_attestation_report, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_START : + if (prep_buffer(struct sev_data_send_start, session_address, + session_len, false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_DATA : + if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_data, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_VMSA : + if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_vmsa, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_DATA : + if (prep_buffer(struct sev_data_receive_update_data, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_VMSA : + if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address, + guest_len, true, &sev->snp_host_map[0 ]))drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 769
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+ goto err; +break ; +default : +break ; + } + +/* The command buffer need to be in the firmware state. */ +if (to_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_firmware (__pa (cmd_buf),1 , true)) +return -EFAULT ;This function moves two separate pages to firmware state. First
Quoted 9 lines
calling
map_firmware_writeableand second callingrmp_mark_pages_firmwareforcmd_buf.
In casermp_mark_pages_firmwarefails forcmd_buf, the page which has already moved to firmware state inmap_firmware_writeableshould be reclaimed.
This is a problem especially if we leak a guest owned page in firmware state. Since this is used only by legacy SEV VMs, these leaked pages will never be reclaimed back when destroying these VMs.
Yes, this looks to be an inherent issue with the original patch, as you mentioned there are two pages - guest owned page and the HV cmd_buf, and failure to transition the cmd_buf back to HV/shared state has no corresponding recovery/reclaim for the transitioned guest page.
Thanks,
Ashish
Quoted 197 lines
Quoted 197 lines
drivers/crypto/ccp/sev-dev.c +80
-10+ } + +return 0 ; + +err: +return -EINVAL ; +} + +static inline bool need_firmware_copy (int cmd) +{ +struct sev_device *sev = psp_master->sev_data; + +/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */ +return ((cmd <SEV_CMD_SNP_INIT ) && sev->snp_initialized) ? true : false; +} + +static int snp_aware_copy_to_firmware (int cmd,void *data) +{ +return __snp_cmd_buf_copy (cmd, data, true,0 ); +} + +static int snp_aware_copy_from_firmware (int cmd,void *data,int fw_err) +{ +return __snp_cmd_buf_copy (cmd, data, false, fw_err); +} +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret) {struct psp_device *psp = psp_master;struct sev_device *sev;unsigned int phys_lsb, phys_msb;unsigned int reg, ret =0 ; +void *cmd_buf;int buf_len;if (!psp || !psp->sev_data)@@ -512,12 +819,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)* work for some memory, e.g. vmalloc'd addresses, and @data may not be * physically contiguous. */ -if (data) -memcpy (sev->cmd_buf, data, buf_len); +if (data) { +if (sev->cmd_buf_active >2 ) +return -EBUSY ; + + cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf; + +memcpy (cmd_buf, data, buf_len); + sev->cmd_buf_active++; + +/* +* The behavior of the SEV-legacy commands is altered when the +* SNP firmware is in the INIT state. +*/ +if (need_firmware_copy (cmd) &&snp_aware_copy_to_firmware (cmd, sev->cmd_buf)) +return -EFAULT ; + }else { + cmd_buf = sev->cmd_buf; + }/* Get the physical address of the command buffer */ - phys_lsb = data ?lower_32_bits (__psp_pa (sev->cmd_buf)) :0 ; - phys_msb = data ?upper_32_bits (__psp_pa (sev->cmd_buf)) :0 ; + phys_lsb = data ?lower_32_bits (__psp_pa (cmd_buf)) :0 ; + phys_msb = data ?upper_32_bits (__psp_pa (cmd_buf)) :0 ;dev_dbg (sev->dev,"sev command id %#x buffer 0x%08x%08x timeout %us\n" , cmd, phys_msb, phys_lsb, psp_timeout);@@ -560,15 +883,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) ret =sev_write_init_ex_file_if_required (cmd); } -print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, - buf_len, false); -/* * Copy potential output from the PSP back to data. Do this even on * failure in case the caller wants to glean something from the error. */ -if (data) -memcpy (data, sev->cmd_buf, buf_len); +if (data) { +/* +* Restore the page state after the command completes. +*/ +if (need_firmware_copy (cmd) && +snp_aware_copy_from_firmware (cmd, cmd_buf, ret)) +return -EFAULT ; + +memcpy (data, cmd_buf, buf_len); + sev->cmd_buf_active--; + } + +print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, + buf_len, false);return ret; }@@ -1579,10 +1911,12 @@ int sev_dev_init(struct psp_device *psp)if (!sev) goto e_err; - sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,0 ); + sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,1 );if (!sev->cmd_buf) goto e_sev; + sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf +PAGE_SIZE ; + psp->sev_data = sev; sev->dev = dev;@@ -1648,6 +1982,12 @@ static void sev_firmware_shutdown(struct sev_device *sev) snp_range_list =NULL ; } +/* +* The host map need to clear the immutable bit so it must be free'd before the +* SNP firmware shutdown. +*/ +free_snp_host_map (sev); +sev_snp_shutdown (&error); }@@ -1722,6 +2062,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); } } + +/* +* Allocate the intermediate buffers used for the legacy command handling. +*/ +if (alloc_snp_host_map (sev)) { +dev_notice (sev->dev,"Failed to alloc host map (disabling legacy SEV)\n" ); + goto skip_legacy; + } }/* Obtain the TMR memory area for SEV-ES use */ @@ -1739,12 +2087,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +skip_legacy:dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ?"-SNP" :"" , sev->api_major, sev->api_minor, sev->build);return ; err: +free_snp_host_map (sev); psp_master->sev_data =NULL ; }drivers/crypto/ccp/sev-dev.h +12
-1@@ -29,11 +29,20 @@#define SEV_CMDRESP_CMD_SHIFT 16#define SEV_CMDRESP_IOC BIT(0) +#define MAX_SNP_HOST_MAP_BUFS 2 +struct sev_misc_dev {struct kref refcount;struct miscdevice misc; }; +struct snp_host_map { +u64 paddr; +u32 len; +void *host; +bool active; +}; +struct sev_device {struct device *dev;struct psp_device *psp;@@ -52,8 +61,11 @@ struct sev_device {u8 build;void *cmd_buf; +void *cmd_buf_backup; +int cmd_buf_active;bool snp_initialized; +struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS ]; };int sev_dev_init (struct psp_device *psp); --2.25.1
Re: [PATCH RFC v7 29/64] crypto: ccp: Handle the legacy SEV command when SNP is enabled
Quoted 223 lines · Ashish
Hello Alper,
Quoted 210 lines · Alper Gun
Quoted 209 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..
A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:
1) system physical address points to a guest memory
2) system physical address points to a host memoryTo handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.
For the case #2, use a bounce buffer to complete the request.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 370 ++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/sev-dev.h | 12 ++ 2 files changed, 372 insertions(+), 10 deletions(-)drivers/crypto/ccp/sev-dev.c +166
-0@@ -286,6 +286,30 @@ static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, booreturn rc; } +static int rmp_mark_pages_shared (unsigned long paddr,unsigned int npages) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, pfn++, n++) { + rc =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to change the page state to shared, then its not safe +* to release the page back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); + +return rc; +} +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) {unsigned long npages =1ul << order, paddr;@@ -487,12 +511,295 @@ static int sev_write_init_ex_file_if_required(int cmd_id)return sev_write_init_ex_file (); } +static int alloc_snp_host_map (struct sev_device *sev) +{ +struct page *page; +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +memset (map,0 ,sizeof (*map)); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (SEV_FW_BLOB_MAX_SIZE )); +if (!page) +return -ENOMEM ; + + map->host =page_address (page); + } + +return 0 ; +} + +static void free_snp_host_map (struct sev_device *sev) +{ +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +if (map->host) { +__free_pages (virt_to_page (map->host),get_order (SEV_FW_BLOB_MAX_SIZE )); +memset (map,0 ,sizeof (*map)); + } + } +} + +static int map_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + + map->active = false; + +if (!paddr || !len) +return 0 ; + + map->paddr = *paddr; + map->len = len; + +/* If paddr points to a guest memory then change the page state to firmwware. */ +if (guest) { +if (rmp_mark_pages_firmware (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +if (!map->host) +return -ENOMEM ; + +/* Check if the pre-allocated buffer can be used to fullfil the request. */ +if (len >SEV_FW_BLOB_MAX_SIZE ) +return -EINVAL ; + +/* Transition the pre-allocated buffer to the firmware state. */ +if (rmp_mark_pages_firmware (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Set the paddr to use pre-allocated firmware buffer */ + *paddr =__psp_pa (map->host); + +done: + map->active = true; +return 0 ; +} + +static int unmap_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + +if (!map->active) +return 0 ; + +/* If paddr points to a guest memory then restore the page state to hypervisor. */ +if (guest) { +if (snp_reclaim_pages (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +/* +* Transition the pre-allocated buffer to hypervisor state before the access. +* +* This is because while changing the page state to firmware, the kernel unmaps +* the pages from the direct map, and to restore the direct map the pages must +* be transitioned back to the shared state. +*/ +if (snp_reclaim_pages (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Copy the response data firmware buffer to the callers buffer. */ +memcpy (__va (__sme_clr (map->paddr)), map->host,min_t (size_t, len, map->len)); + *paddr = map->paddr; + +done: + map->active = false; +return 0 ; +} + +static bool sev_legacy_cmd_buf_writable (int cmd) +{ +switch (cmd) { +case SEV_CMD_PLATFORM_STATUS : +case SEV_CMD_GUEST_STATUS : +case SEV_CMD_LAUNCH_START : +case SEV_CMD_RECEIVE_START : +case SEV_CMD_LAUNCH_MEASURE : +case SEV_CMD_SEND_START : +case SEV_CMD_SEND_UPDATE_DATA : +case SEV_CMD_SEND_UPDATE_VMSA : +case SEV_CMD_PEK_CSR : +case SEV_CMD_PDH_CERT_EXPORT : +case SEV_CMD_GET_ID : +case SEV_CMD_ATTESTATION_REPORT : +return true; +default : +return false; + } +} + +#define prep_buffer (name, addr, len, guest, map) \ + func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map) + +static int __snp_cmd_buf_copy (int cmd,void *cmd_buf,bool to_fw,int fw_err) +{ +int (*func)(u64 *paddr,u32 len,bool guest,struct snp_host_map *map); +struct sev_device *sev = psp_master->sev_data; +bool from_fw = !to_fw; + +/* +* After the command is completed, change the command buffer memory to +* hypervisor state. +* +* The immutable bit is automatically cleared by the firmware, so Quoted 8 lines
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 660
Quoted 5 lines
drivers/crypto/ccp/sev-dev.c +5
-0+* no not need to reclaim the page. +*/ +if (from_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_shared (__pa (cmd_buf),1 )) +return -EFAULT ;If we return here, we will skip calling
unmap_firmware_writeableand we will leak some pages in firmware state.
Quoted 3 lines
Do you mean those (guest) pages which were transitioned to firmware state as part of snp_aware_copy_to_firmware()->_snp_cmd_buf_copy()->map_firmware_writeable()?
yes, if we return here, these guest pages will be left in firmware state.
Quoted 6 lines
Quoted 6 lines
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 664
Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +4
-0+ +/* No need to go further if firmware failed to execute command. */ +if (fw_err) +return 0 ;
Same thing also here, we are possibly leaving guest pages in the firmware state.
Quoted 322 lines
Quoted 107 lines
Quoted 95 lines
drivers/crypto/ccp/sev-dev.c +95
-0+ } + +if (to_fw) + func = map_firmware_writeable; +else + func = unmap_firmware_writeable; + +/* +* A command buffer may contains a system physical address. If the address +* points to a host memory then use an intermediate firmware page otherwise +* change the page state in the RMP table. +*/ +switch (cmd) { +case SEV_CMD_PDH_CERT_EXPORT : + if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address, + pdh_cert_len, false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address, + cert_chain_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_GET_ID : + if (prep_buffer(struct sev_data_get_id, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_PEK_CSR : + if (prep_buffer(struct sev_data_pek_csr, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_DATA : + if (prep_buffer(struct sev_data_launch_update_data, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_VMSA : + if (prep_buffer(struct sev_data_launch_update_vmsa, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_MEASURE : + if (prep_buffer(struct sev_data_launch_measure, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_SECRET : + if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_DECRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, false, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_ENCRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, true, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_ATTESTATION_REPORT : + if (prep_buffer(struct sev_data_attestation_report, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_START : + if (prep_buffer(struct sev_data_send_start, session_address, + session_len, false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_DATA : + if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_data, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_VMSA : + if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_vmsa, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_DATA : + if (prep_buffer(struct sev_data_receive_update_data, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_VMSA : + if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address, + guest_len, true, &sev->snp_host_map[0 ]))drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 769
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+ goto err; +break ; +default : +break ; + } + +/* The command buffer need to be in the firmware state. */ +if (to_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_firmware (__pa (cmd_buf),1 , true)) +return -EFAULT ;This function moves two separate pages to firmware state. First
Quoted 9 lines
calling
map_firmware_writeableand second callingrmp_mark_pages_firmwareforcmd_buf.
In casermp_mark_pages_firmwarefails forcmd_buf, the page which has already moved to firmware state inmap_firmware_writeableshould be reclaimed.
This is a problem especially if we leak a guest owned page in firmware state. Since this is used only by legacy SEV VMs, these leaked pages will never be reclaimed back when destroying these VMs.Yes, this looks to be an inherent issue with the original patch, as you mentioned there are two pages - guest owned page and the HV
cmd_buf, and failure to transition thecmd_bufback to HV/shared state has no corresponding recovery/reclaim for the transitioned guest page.Thanks,
AshishQuoted 197 lines
Quoted 197 lines
drivers/crypto/ccp/sev-dev.c +80
-10+ } + +return 0 ; + +err: +return -EINVAL ; +} + +static inline bool need_firmware_copy (int cmd) +{ +struct sev_device *sev = psp_master->sev_data; + +/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */ +return ((cmd <SEV_CMD_SNP_INIT ) && sev->snp_initialized) ? true : false; +} + +static int snp_aware_copy_to_firmware (int cmd,void *data) +{ +return __snp_cmd_buf_copy (cmd, data, true,0 ); +} + +static int snp_aware_copy_from_firmware (int cmd,void *data,int fw_err) +{ +return __snp_cmd_buf_copy (cmd, data, false, fw_err); +} +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret) {struct psp_device *psp = psp_master;struct sev_device *sev;unsigned int phys_lsb, phys_msb;unsigned int reg, ret =0 ; +void *cmd_buf;int buf_len;if (!psp || !psp->sev_data)@@ -512,12 +819,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)* work for some memory, e.g. vmalloc'd addresses, and @data may not be * physically contiguous. */ -if (data) -memcpy (sev->cmd_buf, data, buf_len); +if (data) { +if (sev->cmd_buf_active >2 ) +return -EBUSY ; + + cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf; + +memcpy (cmd_buf, data, buf_len); + sev->cmd_buf_active++; + +/* +* The behavior of the SEV-legacy commands is altered when the +* SNP firmware is in the INIT state. +*/ +if (need_firmware_copy (cmd) &&snp_aware_copy_to_firmware (cmd, sev->cmd_buf)) +return -EFAULT ; + }else { + cmd_buf = sev->cmd_buf; + }/* Get the physical address of the command buffer */ - phys_lsb = data ?lower_32_bits (__psp_pa (sev->cmd_buf)) :0 ; - phys_msb = data ?upper_32_bits (__psp_pa (sev->cmd_buf)) :0 ; + phys_lsb = data ?lower_32_bits (__psp_pa (cmd_buf)) :0 ; + phys_msb = data ?upper_32_bits (__psp_pa (cmd_buf)) :0 ;dev_dbg (sev->dev,"sev command id %#x buffer 0x%08x%08x timeout %us\n" , cmd, phys_msb, phys_lsb, psp_timeout);@@ -560,15 +883,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) ret =sev_write_init_ex_file_if_required (cmd); } -print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, - buf_len, false); -/* * Copy potential output from the PSP back to data. Do this even on * failure in case the caller wants to glean something from the error. */ -if (data) -memcpy (data, sev->cmd_buf, buf_len); +if (data) { +/* +* Restore the page state after the command completes. +*/ +if (need_firmware_copy (cmd) && +snp_aware_copy_from_firmware (cmd, cmd_buf, ret)) +return -EFAULT ; + +memcpy (data, cmd_buf, buf_len); + sev->cmd_buf_active--; + } + +print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, + buf_len, false);return ret; }@@ -1579,10 +1911,12 @@ int sev_dev_init(struct psp_device *psp)if (!sev) goto e_err; - sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,0 ); + sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,1 );if (!sev->cmd_buf) goto e_sev; + sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf +PAGE_SIZE ; + psp->sev_data = sev; sev->dev = dev;@@ -1648,6 +1982,12 @@ static void sev_firmware_shutdown(struct sev_device *sev) snp_range_list =NULL ; } +/* +* The host map need to clear the immutable bit so it must be free'd before the +* SNP firmware shutdown. +*/ +free_snp_host_map (sev); +sev_snp_shutdown (&error); }@@ -1722,6 +2062,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); } } + +/* +* Allocate the intermediate buffers used for the legacy command handling. +*/ +if (alloc_snp_host_map (sev)) { +dev_notice (sev->dev,"Failed to alloc host map (disabling legacy SEV)\n" ); + goto skip_legacy; + } }/* Obtain the TMR memory area for SEV-ES use */ @@ -1739,12 +2087,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +skip_legacy:dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ?"-SNP" :"" , sev->api_major, sev->api_minor, sev->build);return ; err: +free_snp_host_map (sev); psp_master->sev_data =NULL ; }drivers/crypto/ccp/sev-dev.h +12
-1@@ -29,11 +29,20 @@#define SEV_CMDRESP_CMD_SHIFT 16#define SEV_CMDRESP_IOC BIT(0) +#define MAX_SNP_HOST_MAP_BUFS 2 +struct sev_misc_dev {struct kref refcount;struct miscdevice misc; }; +struct snp_host_map { +u64 paddr; +u32 len; +void *host; +bool active; +}; +struct sev_device {struct device *dev;struct psp_device *psp;@@ -52,8 +61,11 @@ struct sev_device {u8 build;void *cmd_buf; +void *cmd_buf_backup; +int cmd_buf_active;bool snp_initialized; +struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS ]; };int sev_dev_init (struct psp_device *psp); --2.25.1
Kalra, Ashish
maintainerRe: [PATCH RFC v7 29/64] crypto: ccp: Handle the legacy SEV command when SNP is enabled
Quoted 228 lines · Alper Gun
Quoted 223 lines · Ashish
Hello Alper,
Quoted 210 lines · Alper Gun
Quoted 209 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..
A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:
1) system physical address points to a guest memory
2) system physical address points to a host memoryTo handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.
For the case #2, use a bounce buffer to complete the request.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 370 ++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/sev-dev.h | 12 ++ 2 files changed, 372 insertions(+), 10 deletions(-)drivers/crypto/ccp/sev-dev.c +166
-0@@ -286,6 +286,30 @@ static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, booreturn rc; } +static int rmp_mark_pages_shared (unsigned long paddr,unsigned int npages) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, pfn++, n++) { + rc =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to change the page state to shared, then its not safe +* to release the page back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); + +return rc; +} +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) {unsigned long npages =1ul << order, paddr;@@ -487,12 +511,295 @@ static int sev_write_init_ex_file_if_required(int cmd_id)return sev_write_init_ex_file (); } +static int alloc_snp_host_map (struct sev_device *sev) +{ +struct page *page; +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +memset (map,0 ,sizeof (*map)); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (SEV_FW_BLOB_MAX_SIZE )); +if (!page) +return -ENOMEM ; + + map->host =page_address (page); + } + +return 0 ; +} + +static void free_snp_host_map (struct sev_device *sev) +{ +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +if (map->host) { +__free_pages (virt_to_page (map->host),get_order (SEV_FW_BLOB_MAX_SIZE )); +memset (map,0 ,sizeof (*map)); + } + } +} + +static int map_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + + map->active = false; + +if (!paddr || !len) +return 0 ; + + map->paddr = *paddr; + map->len = len; + +/* If paddr points to a guest memory then change the page state to firmwware. */ +if (guest) { +if (rmp_mark_pages_firmware (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +if (!map->host) +return -ENOMEM ; + +/* Check if the pre-allocated buffer can be used to fullfil the request. */ +if (len >SEV_FW_BLOB_MAX_SIZE ) +return -EINVAL ; + +/* Transition the pre-allocated buffer to the firmware state. */ +if (rmp_mark_pages_firmware (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Set the paddr to use pre-allocated firmware buffer */ + *paddr =__psp_pa (map->host); + +done: + map->active = true; +return 0 ; +} + +static int unmap_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + +if (!map->active) +return 0 ; + +/* If paddr points to a guest memory then restore the page state to hypervisor. */ +if (guest) { +if (snp_reclaim_pages (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +/* +* Transition the pre-allocated buffer to hypervisor state before the access. +* +* This is because while changing the page state to firmware, the kernel unmaps +* the pages from the direct map, and to restore the direct map the pages must +* be transitioned back to the shared state. +*/ +if (snp_reclaim_pages (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Copy the response data firmware buffer to the callers buffer. */ +memcpy (__va (__sme_clr (map->paddr)), map->host,min_t (size_t, len, map->len)); + *paddr = map->paddr; + +done: + map->active = false; +return 0 ; +} + +static bool sev_legacy_cmd_buf_writable (int cmd) +{ +switch (cmd) { +case SEV_CMD_PLATFORM_STATUS : +case SEV_CMD_GUEST_STATUS : +case SEV_CMD_LAUNCH_START : +case SEV_CMD_RECEIVE_START : +case SEV_CMD_LAUNCH_MEASURE : +case SEV_CMD_SEND_START : +case SEV_CMD_SEND_UPDATE_DATA : +case SEV_CMD_SEND_UPDATE_VMSA : +case SEV_CMD_PEK_CSR : +case SEV_CMD_PDH_CERT_EXPORT : +case SEV_CMD_GET_ID : +case SEV_CMD_ATTESTATION_REPORT : +return true; +default : +return false; + } +} + +#define prep_buffer (name, addr, len, guest, map) \ + func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map) + +static int __snp_cmd_buf_copy (int cmd,void *cmd_buf,bool to_fw,int fw_err) +{ +int (*func)(u64 *paddr,u32 len,bool guest,struct snp_host_map *map); +struct sev_device *sev = psp_master->sev_data; +bool from_fw = !to_fw; + +/* +* After the command is completed, change the command buffer memory to +* hypervisor state. +* +* The immutable bit is automatically cleared by the firmware, so Quoted 8 lines
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 660
Quoted 5 lines
drivers/crypto/ccp/sev-dev.c +5
-0+* no not need to reclaim the page. +*/ +if (from_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_shared (__pa (cmd_buf),1 )) +return -EFAULT ;If we return here, we will skip calling
unmap_firmware_writeableand we will leak some pages in firmware state.Quoted 3 lines
Do you mean those (guest) pages which were transitioned to firmware state as part of snp_aware_copy_to_firmware()->_snp_cmd_buf_copy()->map_firmware_writeable()?
Quoted 9 lines
yes, if we return here, these guest pages will be left in firmware state.
Quoted 6 lines
Quoted 6 lines
drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 664
Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +4
-0+ +/* No need to go further if firmware failed to execute command. */ +if (fw_err) +return 0 ;Same thing also here, we are possibly leaving guest pages in the firmware state.
Yes, i agree, both are inhererent bugs in the original patch and need to be fixed.
Thanks,
Ashish
Quoted 322 lines
Quoted 322 lines
Quoted 107 lines
Quoted 95 lines
drivers/crypto/ccp/sev-dev.c +95
-0+ } + +if (to_fw) + func = map_firmware_writeable; +else + func = unmap_firmware_writeable; + +/* +* A command buffer may contains a system physical address. If the address +* points to a host memory then use an intermediate firmware page otherwise +* change the page state in the RMP table. +*/ +switch (cmd) { +case SEV_CMD_PDH_CERT_EXPORT : + if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address, + pdh_cert_len, false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address, + cert_chain_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_GET_ID : + if (prep_buffer(struct sev_data_get_id, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_PEK_CSR : + if (prep_buffer(struct sev_data_pek_csr, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_DATA : + if (prep_buffer(struct sev_data_launch_update_data, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_VMSA : + if (prep_buffer(struct sev_data_launch_update_vmsa, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_MEASURE : + if (prep_buffer(struct sev_data_launch_measure, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_SECRET : + if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_DECRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, false, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_ENCRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, true, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_ATTESTATION_REPORT : + if (prep_buffer(struct sev_data_attestation_report, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_START : + if (prep_buffer(struct sev_data_send_start, session_address, + session_len, false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_DATA : + if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_data, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_VMSA : + if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_vmsa, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_DATA : + if (prep_buffer(struct sev_data_receive_update_data, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_VMSA : + if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address, + guest_len, true, &sev->snp_host_map[0 ]))drivers/crypto/ccp/sev-dev.c static int sev_write_init_ex_file_if_required(int cmd_id) · line 769
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+ goto err; +break ; +default : +break ; + } + +/* The command buffer need to be in the firmware state. */ +if (to_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_firmware (__pa (cmd_buf),1 , true)) +return -EFAULT ;This function moves two separate pages to firmware state. First
Quoted 9 lines
calling
map_firmware_writeableand second callingrmp_mark_pages_firmwareforcmd_buf.
In casermp_mark_pages_firmwarefails forcmd_buf, the page which has already moved to firmware state inmap_firmware_writeableshould be reclaimed.
This is a problem especially if we leak a guest owned page in firmware state. Since this is used only by legacy SEV VMs, these leaked pages will never be reclaimed back when destroying these VMs.Yes, this looks to be an inherent issue with the original patch, as you mentioned there are two pages - guest owned page and the HV
cmd_buf, and failure to transition thecmd_bufback to HV/shared state has no corresponding recovery/reclaim for the transitioned guest page.Thanks,
AshishQuoted 197 lines
Quoted 197 lines
drivers/crypto/ccp/sev-dev.c +80
-10+ } + +return 0 ; + +err: +return -EINVAL ; +} + +static inline bool need_firmware_copy (int cmd) +{ +struct sev_device *sev = psp_master->sev_data; + +/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */ +return ((cmd <SEV_CMD_SNP_INIT ) && sev->snp_initialized) ? true : false; +} + +static int snp_aware_copy_to_firmware (int cmd,void *data) +{ +return __snp_cmd_buf_copy (cmd, data, true,0 ); +} + +static int snp_aware_copy_from_firmware (int cmd,void *data,int fw_err) +{ +return __snp_cmd_buf_copy (cmd, data, false, fw_err); +} +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret) {struct psp_device *psp = psp_master;struct sev_device *sev;unsigned int phys_lsb, phys_msb;unsigned int reg, ret =0 ; +void *cmd_buf;int buf_len;if (!psp || !psp->sev_data)@@ -512,12 +819,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)* work for some memory, e.g. vmalloc'd addresses, and @data may not be * physically contiguous. */ -if (data) -memcpy (sev->cmd_buf, data, buf_len); +if (data) { +if (sev->cmd_buf_active >2 ) +return -EBUSY ; + + cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf; + +memcpy (cmd_buf, data, buf_len); + sev->cmd_buf_active++; + +/* +* The behavior of the SEV-legacy commands is altered when the +* SNP firmware is in the INIT state. +*/ +if (need_firmware_copy (cmd) &&snp_aware_copy_to_firmware (cmd, sev->cmd_buf)) +return -EFAULT ; + }else { + cmd_buf = sev->cmd_buf; + }/* Get the physical address of the command buffer */ - phys_lsb = data ?lower_32_bits (__psp_pa (sev->cmd_buf)) :0 ; - phys_msb = data ?upper_32_bits (__psp_pa (sev->cmd_buf)) :0 ; + phys_lsb = data ?lower_32_bits (__psp_pa (cmd_buf)) :0 ; + phys_msb = data ?upper_32_bits (__psp_pa (cmd_buf)) :0 ;dev_dbg (sev->dev,"sev command id %#x buffer 0x%08x%08x timeout %us\n" , cmd, phys_msb, phys_lsb, psp_timeout);@@ -560,15 +883,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) ret =sev_write_init_ex_file_if_required (cmd); } -print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, - buf_len, false); -/* * Copy potential output from the PSP back to data. Do this even on * failure in case the caller wants to glean something from the error. */ -if (data) -memcpy (data, sev->cmd_buf, buf_len); +if (data) { +/* +* Restore the page state after the command completes. +*/ +if (need_firmware_copy (cmd) && +snp_aware_copy_from_firmware (cmd, cmd_buf, ret)) +return -EFAULT ; + +memcpy (data, cmd_buf, buf_len); + sev->cmd_buf_active--; + } + +print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, + buf_len, false);return ret; }@@ -1579,10 +1911,12 @@ int sev_dev_init(struct psp_device *psp)if (!sev) goto e_err; - sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,0 ); + sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,1 );if (!sev->cmd_buf) goto e_sev; + sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf +PAGE_SIZE ; + psp->sev_data = sev; sev->dev = dev;@@ -1648,6 +1982,12 @@ static void sev_firmware_shutdown(struct sev_device *sev) snp_range_list =NULL ; } +/* +* The host map need to clear the immutable bit so it must be free'd before the +* SNP firmware shutdown. +*/ +free_snp_host_map (sev); +sev_snp_shutdown (&error); }@@ -1722,6 +2062,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); } } + +/* +* Allocate the intermediate buffers used for the legacy command handling. +*/ +if (alloc_snp_host_map (sev)) { +dev_notice (sev->dev,"Failed to alloc host map (disabling legacy SEV)\n" ); + goto skip_legacy; + } }/* Obtain the TMR memory area for SEV-ES use */ @@ -1739,12 +2087,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +skip_legacy:dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ?"-SNP" :"" , sev->api_major, sev->api_minor, sev->build);return ; err: +free_snp_host_map (sev); psp_master->sev_data =NULL ; }drivers/crypto/ccp/sev-dev.h +12
-1@@ -29,11 +29,20 @@#define SEV_CMDRESP_CMD_SHIFT 16#define SEV_CMDRESP_IOC BIT(0) +#define MAX_SNP_HOST_MAP_BUFS 2 +struct sev_misc_dev {struct kref refcount;struct miscdevice misc; }; +struct snp_host_map { +u64 paddr; +u32 len; +void *host; +bool active; +}; +struct sev_device {struct device *dev;struct psp_device *psp;@@ -52,8 +61,11 @@ struct sev_device {u8 build;void *cmd_buf; +void *cmd_buf_backup; +int cmd_buf_active;bool snp_initialized; +struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS ]; };int sev_dev_init (struct psp_device *psp); --2.25.1
Re: [PATCH RFC v7 29/64] crypto: ccp: Handle the legacy SEV command when SNP is enabled
In reply to Michael RothQuoted 369 lines · Michael Roth
From: Brijesh Singh <[email protected]>
The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..
A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:
1) system physical address points to a guest memory
2) system physical address points to a host memoryTo handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.
For the case #2, use a bounce buffer to complete the request.
Signed-off-by Brijesh Singh <[email protected]>Signed-off-by Ashish Kalra <[email protected]>Signed-off-by Michael Roth <[email protected]>--- drivers/crypto/ccp/sev-dev.c | 370 ++++++++++++++++++++++++++++++++++- drivers/crypto/ccp/sev-dev.h | 12 ++ 2 files changed, 372 insertions(+), 10 deletions(-)drivers/crypto/ccp/sev-dev.c +311
-2@@ -286,6 +286,30 @@ static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, booreturn rc; } +static int rmp_mark_pages_shared (unsigned long paddr,unsigned int npages) +{ +/* Cbit maybe set in the paddr */ +unsigned long pfn =__sme_clr (paddr) >>PAGE_SHIFT ; +int rc, n =0 , i; + +for (i =0 ; i < npages; i++, pfn++, n++) { + rc =rmp_make_shared (pfn,PG_LEVEL_4K ); +if (rc) + goto cleanup; + } + +return 0 ; + +cleanup: +/* +* If failed to change the page state to shared, then its not safe +* to release the page back to the system, leak it. +*/ +snp_mark_pages_offline (pfn, npages - n); + +return rc; +} +static struct page *__snp_alloc_firmware_pages (gfp_t gfp_mask,int order,bool locked) {unsigned long npages =1ul << order, paddr;@@ -487,12 +511,295 @@ static int sev_write_init_ex_file_if_required(int cmd_id)return sev_write_init_ex_file (); } +static int alloc_snp_host_map (struct sev_device *sev) +{ +struct page *page; +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +memset (map,0 ,sizeof (*map)); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,get_order (SEV_FW_BLOB_MAX_SIZE )); +if (!page) +return -ENOMEM ; + + map->host =page_address (page); + } + +return 0 ; +} + +static void free_snp_host_map (struct sev_device *sev) +{ +int i; + +for (i =0 ; i <MAX_SNP_HOST_MAP_BUFS ; i++) { +struct snp_host_map *map = &sev->snp_host_map[i]; + +if (map->host) { +__free_pages (virt_to_page (map->host),get_order (SEV_FW_BLOB_MAX_SIZE )); +memset (map,0 ,sizeof (*map)); + } + } +} + +static int map_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + + map->active = false; + +if (!paddr || !len) +return 0 ; + + map->paddr = *paddr; + map->len = len; + +/* If paddr points to a guest memory then change the page state to firmwware. */ +if (guest) { +if (rmp_mark_pages_firmware (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +if (!map->host) +return -ENOMEM ; + +/* Check if the pre-allocated buffer can be used to fullfil the request. */ +if (len >SEV_FW_BLOB_MAX_SIZE ) +return -EINVAL ; + +/* Transition the pre-allocated buffer to the firmware state. */ +if (rmp_mark_pages_firmware (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Set the paddr to use pre-allocated firmware buffer */ + *paddr =__psp_pa (map->host); + +done: + map->active = true; +return 0 ; +} + +static int unmap_firmware_writeable (u64 *paddr,u32 len,bool guest,struct snp_host_map *map) +{ +unsigned int npages =PAGE_ALIGN (len) >>PAGE_SHIFT ; + +if (!map->active) +return 0 ; + +/* If paddr points to a guest memory then restore the page state to hypervisor. */ +if (guest) { +if (snp_reclaim_pages (*paddr, npages, true)) +return -EFAULT ; + + goto done; + } + +/* +* Transition the pre-allocated buffer to hypervisor state before the access. +* +* This is because while changing the page state to firmware, the kernel unmaps +* the pages from the direct map, and to restore the direct map the pages must +* be transitioned back to the shared state. +*/ +if (snp_reclaim_pages (__pa (map->host), npages, true)) +return -EFAULT ; + +/* Copy the response data firmware buffer to the callers buffer. */ +memcpy (__va (__sme_clr (map->paddr)), map->host,min_t (size_t, len, map->len)); + *paddr = map->paddr; + +done: + map->active = false; +return 0 ; +} + +static bool sev_legacy_cmd_buf_writable (int cmd) +{ +switch (cmd) { +case SEV_CMD_PLATFORM_STATUS : +case SEV_CMD_GUEST_STATUS : +case SEV_CMD_LAUNCH_START : +case SEV_CMD_RECEIVE_START : +case SEV_CMD_LAUNCH_MEASURE : +case SEV_CMD_SEND_START : +case SEV_CMD_SEND_UPDATE_DATA : +case SEV_CMD_SEND_UPDATE_VMSA : +case SEV_CMD_PEK_CSR : +case SEV_CMD_PDH_CERT_EXPORT : +case SEV_CMD_GET_ID : +case SEV_CMD_ATTESTATION_REPORT : +return true; +default : +return false; + } +} + +#define prep_buffer (name, addr, len, guest, map) \ + func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map) + +static int __snp_cmd_buf_copy (int cmd,void *cmd_buf,bool to_fw,int fw_err) +{ +int (*func)(u64 *paddr,u32 len,bool guest,struct snp_host_map *map); +struct sev_device *sev = psp_master->sev_data; +bool from_fw = !to_fw; + +/* +* After the command is completed, change the command buffer memory to +* hypervisor state. +* +* The immutable bit is automatically cleared by the firmware, so +* no not need to reclaim the page. +*/ +if (from_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_shared (__pa (cmd_buf),1 )) +return -EFAULT ; + +/* No need to go further if firmware failed to execute command. */ +if (fw_err) +return 0 ; + } + +if (to_fw) + func = map_firmware_writeable; +else + func = unmap_firmware_writeable; + +/* +* A command buffer may contains a system physical address. If the address +* points to a host memory then use an intermediate firmware page otherwise +* change the page state in the RMP table. +*/ +switch (cmd) { +case SEV_CMD_PDH_CERT_EXPORT : + if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address, + pdh_cert_len, false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address, + cert_chain_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_GET_ID : + if (prep_buffer(struct sev_data_get_id, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_PEK_CSR : + if (prep_buffer(struct sev_data_pek_csr, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_DATA : + if (prep_buffer(struct sev_data_launch_update_data, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_VMSA : + if (prep_buffer(struct sev_data_launch_update_vmsa, address, len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_MEASURE : + if (prep_buffer(struct sev_data_launch_measure, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_LAUNCH_UPDATE_SECRET : + if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len, + true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_DECRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, false, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_DBG_ENCRYPT : + if (prep_buffer(struct sev_data_dbg, dst_addr, len, true, + &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_ATTESTATION_REPORT : + if (prep_buffer(struct sev_data_attestation_report, address, len, + false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_START : + if (prep_buffer(struct sev_data_send_start, session_address, + session_len, false, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_DATA : + if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_data, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_SEND_UPDATE_VMSA : + if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len, + false, &sev->snp_host_map[0 ])) + goto err; + if (prep_buffer(struct sev_data_send_update_vmsa, trans_address, + trans_len, false, &sev->snp_host_map[1 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_DATA : + if (prep_buffer(struct sev_data_receive_update_data, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +case SEV_CMD_RECEIVE_UPDATE_VMSA : + if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address, + guest_len, true, &sev->snp_host_map[0 ])) + goto err; +break ; +default : +break ; + } + +/* The command buffer need to be in the firmware state. */ +if (to_fw &&sev_legacy_cmd_buf_writable (cmd)) { +if (rmp_mark_pages_firmware (__pa (cmd_buf),1 , true)) +return -EFAULT ; + } + +return 0 ; + +err: +return -EINVAL ; +} + +static inline bool need_firmware_copy (int cmd) +{ +struct sev_device *sev = psp_master->sev_data; + +/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */ +return ((cmd <SEV_CMD_SNP_INIT ) && sev->snp_initialized) ? true : false; +} + +static int snp_aware_copy_to_firmware (int cmd,void *data) +{ +return __snp_cmd_buf_copy (cmd, data, true,0 ); +} + +static int snp_aware_copy_from_firmware (int cmd,void *data,int fw_err) +{ +return __snp_cmd_buf_copy (cmd, data, false, fw_err); +} +static int __sev_do_cmd_locked (int cmd,void *data,int *psp_ret) {struct psp_device *psp = psp_master;struct sev_device *sev;unsigned int phys_lsb, phys_msb;unsigned int reg, ret =0 ; +void *cmd_buf;int buf_len;if (!psp || !psp->sev_data)@@ -512,12 +819,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)* work for some memory, e.g. vmalloc'd addresses, and @data may not be * physically contiguous. */ -if (data) -memcpy (sev->cmd_buf, data, buf_len); +if (data) { +if (sev->cmd_buf_active >2 ) +return -EBUSY ; +
drivers/crypto/ccp/sev-dev.c static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) · line 835
Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10
-0+ cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf; + +memcpy (cmd_buf, data, buf_len); + sev->cmd_buf_active++; + +/* +* The behavior of the SEV-legacy commands is altered when the +* SNP firmware is in the INIT state. +*/ +if (need_firmware_copy (cmd) &&snp_aware_copy_to_firmware (cmd, sev->cmd_buf))
I believe this should be cmd_buf instead of sev->cmd_buf. snp_aware_copy_to_firmware(cmd, cmd_buf)
Quoted 140 lines
drivers/crypto/ccp/sev-dev.c +39
-8+return -EFAULT ; + }else { + cmd_buf = sev->cmd_buf; + }/* Get the physical address of the command buffer */ - phys_lsb = data ?lower_32_bits (__psp_pa (sev->cmd_buf)) :0 ; - phys_msb = data ?upper_32_bits (__psp_pa (sev->cmd_buf)) :0 ; + phys_lsb = data ?lower_32_bits (__psp_pa (cmd_buf)) :0 ; + phys_msb = data ?upper_32_bits (__psp_pa (cmd_buf)) :0 ;dev_dbg (sev->dev,"sev command id %#x buffer 0x%08x%08x timeout %us\n" , cmd, phys_msb, phys_lsb, psp_timeout);@@ -560,15 +883,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) ret =sev_write_init_ex_file_if_required (cmd); } -print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, - buf_len, false); -/* * Copy potential output from the PSP back to data. Do this even on * failure in case the caller wants to glean something from the error. */ -if (data) -memcpy (data, sev->cmd_buf, buf_len); +if (data) { +/* +* Restore the page state after the command completes. +*/ +if (need_firmware_copy (cmd) && +snp_aware_copy_from_firmware (cmd, cmd_buf, ret)) +return -EFAULT ; + +memcpy (data, cmd_buf, buf_len); + sev->cmd_buf_active--; + } + +print_hex_dump_debug ("(out): " ,DUMP_PREFIX_OFFSET ,16 ,2 , data, + buf_len, false);return ret; }@@ -1579,10 +1911,12 @@ int sev_dev_init(struct psp_device *psp)if (!sev) goto e_err; - sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,0 ); + sev->cmd_buf = (void *)devm_get_free_pages (dev,GFP_KERNEL ,1 );if (!sev->cmd_buf) goto e_sev; + sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf +PAGE_SIZE ; + psp->sev_data = sev; sev->dev = dev;@@ -1648,6 +1982,12 @@ static void sev_firmware_shutdown(struct sev_device *sev) snp_range_list =NULL ; } +/* +* The host map need to clear the immutable bit so it must be free'd before the +* SNP firmware shutdown. +*/ +free_snp_host_map (sev); +sev_snp_shutdown (&error); }@@ -1722,6 +2062,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV-SNP: failed to INIT error %#x\n" , error); } } + +/* +* Allocate the intermediate buffers used for the legacy command handling. +*/ +if (alloc_snp_host_map (sev)) { +dev_notice (sev->dev,"Failed to alloc host map (disabling legacy SEV)\n" ); + goto skip_legacy; + } }/* Obtain the TMR memory area for SEV-ES use */ @@ -1739,12 +2087,14 @@ void sev_pci_init(void)dev_err (sev->dev,"SEV: failed to INIT error %#x, rc %d\n" , error, rc); +skip_legacy:dev_info (sev->dev,"SEV%s API:%d.%d build:%d\n" , sev->snp_initialized ?"-SNP" :"" , sev->api_major, sev->api_minor, sev->build);return ; err: +free_snp_host_map (sev); psp_master->sev_data =NULL ; }drivers/crypto/ccp/sev-dev.h +12
-1@@ -29,11 +29,20 @@#define SEV_CMDRESP_CMD_SHIFT 16#define SEV_CMDRESP_IOC BIT(0) +#define MAX_SNP_HOST_MAP_BUFS 2 +struct sev_misc_dev {struct kref refcount;struct miscdevice misc; }; +struct snp_host_map { +u64 paddr; +u32 len; +void *host; +bool active; +}; +struct sev_device {struct device *dev;struct psp_device *psp;@@ -52,8 +61,11 @@ struct sev_device {u8 build;void *cmd_buf; +void *cmd_buf_backup; +int cmd_buf_active;bool snp_initialized; +struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS ]; };int sev_dev_init (struct psp_device *psp); --2.25.1