KVM: guest_memfd: RECLAIM+CONVERT cleanups
Subsystems (4)
Sean Christopherson
maintainerv6 cover KVM: guest_memfd: RECLAIM+CONVERT cleanups
These are the cleanups from the SNP VMSA series. Well, that's how they started out, now it's full into prep work for in-place conversion. Oh well.
Rework prepare()+invalidate() into make_private()+reclaim(), with direct line of sight to adding make_shared() for in-place conversion. Have the x86 backend use make_{private,shared} (no reclaim()) for SNP, as reclaim in SNP-land just means "make shared".
The goals are to eliminate the misleading use of the term "invalidate", drop the "prepare" terminology (which gets especially confusing with in-place conversion), and in general to prepare for the future. E.g. so that KVM ends up with something like:
if (!to_private) kvm_gmem_make_shared(inode, start, end);
instead of a funky:
if (!to_private) kvm_gmem_invalidate(inode, start, end);
and
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
if (kvm_gmem_is_private_mem(file_inode(file), index))
r = kvm_arch_gmem_make_private(kvm, gfn, *pfn,
(kvm_pfn_t)1 << *max_order);
#endif
instead of the current bare:
r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio);
We'll still need changes for things like TDX hugepage support, and obviously to add kvm_arch_gmem_make_shared() for in-place conversion, but I think/hope this sets the stage for SNP, TDX, and pKVM to build on the arch hooks without needing to make non-trivial changes to the core semantics.
v6:
- Provide separate make_{private,shared}() hooks instead of a single convert(), because forcing a single API just meant multiplexing on the front-end, and de-multiplexing on the back-end, along with weird sanity checks to enforce different requires for private vs. shared. [Ackerley]
- Kill off per-folio preparation entirely. [Sashiko, because Sashiko kept finding bugs in my attemps to refactor the code, so I went scorched earth].
- Pull in "Only "prepare" folios for private pages" from the in-place series to make it easier to kill of per-folio preparation.
- Collect some reviews (dropped many since things got reworked, at lot).
v5
- https://lore.kernel.org/all/[email protected]
- Drop the VMSA fixes (already applied).
- Keep a dedicated reclaim() flow. [Fuad, Ackerley]
- Pass a gfn, not a gpa, to convert() / reclaim(). [Ackerley]
- Collect reviews. [Ackerley]
- Align the gfn when folding __kvm_gmem_prepare_folio() into its caller. [Sashiko]
v4:
- https://lore.kernel.org/all/[email protected]
- Collect reviews. [Mike, Ackerley]
- Rework prepare()+invalidate() into a common convert() instead of simply renaming invalidate() to reclaim_memory(). [Ackerley]
- Fix several (mostly benign) issues in the "prepare" flow.
v3:
- https://lore.kernel.org/all/[email protected]
- Ensure disabling quirks while the VM is live won't result in KVM skipping the back-half of "zap all fast". [Sashiko]
- s/gmem_free_folio/gmem_reclaim_memory. [Ackerley]
- Use {READ,WRITE}_ONCE() when accessing the guest's VMSA GPA outside of the per-vCPU mutex, and comment. [Sashiko]
v2:
- https://lore.kernel.org/all/[email protected]
- Invalidate VMSAs if the memslot is DELETED or MOVED. [Sashiko]
- Limit stable@ patches without a Fixes to 6.12+.
v1: https://lore.kernel.org/all/[email protected]
Ackerley Tng (1):
KVM: guest_memfd: Only "prepare" folios for private pages
Sean Christopherson (8):
KVM: guest_memfd: Pass the number of pages instead of the end pfn into
.invalidate()
KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and
isolate it
KVM: x86: Rename kvm_x86_ops' gmem_invalidate() to gmem_make_shared()
KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare()
failure
KVM: guest_memfd: Add helpers to query SHARED vs. PRIVATE for a given
page
KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private()
/ CONVERT
KVM: guest_memfd: Explicitly pass number of pages to make_private()
hook
KVM: guest_memfd: Make private exactly what can be mapped on page
fault
arch/x86/include/asm/kvm-x86-ops.h | 8 ++-- arch/x86/include/asm/kvm_host.h | 9 ++-- arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/svm/sev.c | 35 +++++++-------- arch/x86/kvm/svm/svm.c | 4 +- arch/x86/kvm/svm/svm.h | 4 +- arch/x86/kvm/x86.c | 18 +++++--- include/linux/kvm_host.h | 10 +++-- virt/kvm/Kconfig | 6 ++- virt/kvm/guest_memfd.c | 68 +++++++----------------------- 10 files changed, 71 insertions(+), 94 deletions(-)
base-commit: 6bc96b971766fbbbbdd9fb2642cedacaf02da957
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Sean Christopherson
maintainerPATCH v6 1/9 KVM: guest_memfd: Pass the number of pages instead of the end pfn into .invalidate()
Pass the number of pages to "invalidate", i.e. reclaim, instead of the end pfn, as a first step towards aligning the function prototypes between the de facto "to private" and "to shared" arch hooks. Eventually, the goal is to end up with kvm_gmem_arch_make_{private,shared}(), and in both cases, providing the number of pages makes the call sites slightly nicer, and also avoids any confusion over whether the end pfn is inclusive or exclusive.
Opportunistically rename "start" to "pfn", again to align with the expected signature of make_private() (which needs to pass a starting gfn as well, at which point the "start" becomes noise).
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/svm/sev.c | 8 ++++---- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 6 +----- 6 files changed, 10 insertions(+), 14 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -1727,7 +1727,7 @@ struct kvm_x86_ops {#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif -void (*gmem_invalidate)(kvm_pfn_t start,kvm_pfn_t end); +void (*gmem_invalidate)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif
arch/x86/kvm/svm/sev.c +4 -4
@@ -5161,16 +5161,16 @@ int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order)return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) { -kvm_pfn_t pfn; +kvm_pfn_t end = pfn + nr_pages;if (!cc_platform_has (CC_ATTR_HOST_SEV_SNP ))return ; -pr_debug ("%s: PFN start 0x%llx PFN end 0x%llx\n" , __func__, start, end); +pr_debug ("%s: PFN start 0x%llx PFN end 0x%llx\n" , __func__, pfn, end); -for (pfn = start; pfn < end;) { +while (pfn < end) {bool use_2m_update = false;int rc, rmp_level;bool assigned;
arch/x86/kvm/svm/svm.h +1 -1
@@ -1010,7 +1010,7 @@ int sev_dev_get_attr(u32 group, u64 attr, u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code);int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +2 -2
@@ -10627,9 +10627,9 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) { -kvm_x86_call (gmem_invalidate)(start, end); +kvm_x86_call (gmem_invalidate)(pfn, nr_pages); }void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {
include/linux/kvm_host.h +1 -1
@@ -2614,7 +2614,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src,#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif
virt/kvm/guest_memfd.c +1 -5
@@ -530,11 +530,7 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE static void kvm_gmem_free_folio (struct folio *folio) { -struct page *page =folio_page (folio,0 ); -kvm_pfn_t pfn =page_to_pfn (page); -int order =folio_order (folio); - -kvm_arch_gmem_invalidate (pfn, pfn + (1ul << order)); +kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio)); }#endif
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 1/9] KVM: guest_memfd: Pass the number of pages instead of the end pfn into .invalidate()
Quoted 12 lines · Sean Christopherson
Pass the number of pages to "invalidate", i.e. reclaim, instead of the end pfn, as a first step towards aligning the function prototypes between the de facto "to private" and "to shared" arch hooks. Eventually, the goal is to end up with kvm_gmem_arch_make_{private,shared}(), and in both cases, providing the number of pages makes the call sites slightly nicer, and also avoids any confusion over whether the end pfn is inclusive or exclusive.
Opportunistically rename "start" to "pfn", again to align with the expected signature of
make_private()(which needs to pass a starting gfn as well, at which point the "start" becomes noise).
Quoted 5 lines
No functional change intended.
Cc Fuad Tabba <[email protected]>Cc Ackerley Tng <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Cheers,
/fuad
Quoted 109 lines
--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/svm/sev.c | 8 ++++---- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 6 +----- 6 files changed, 10 insertions(+), 14 deletions(-)arch/x86/include/asm/kvm_host.h +1
-1@@ -1727,7 +1727,7 @@ struct kvm_x86_ops {#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif -void (*gmem_invalidate)(kvm_pfn_t start,kvm_pfn_t end); +void (*gmem_invalidate)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif arch/x86/kvm/svm/sev.c +4
-4@@ -5161,16 +5161,16 @@ int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order)return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) { -kvm_pfn_t pfn; +kvm_pfn_t end = pfn + nr_pages;if (!cc_platform_has (CC_ATTR_HOST_SEV_SNP ))return ; -pr_debug ("%s: PFN start 0x%llx PFN end 0x%llx\n" , __func__, start, end); +pr_debug ("%s: PFN start 0x%llx PFN end 0x%llx\n" , __func__, pfn, end); -for (pfn = start; pfn < end;) { +while (pfn < end) {bool use_2m_update = false;int rc, rmp_level;bool assigned;arch/x86/kvm/svm/svm.h +1
-1@@ -1010,7 +1010,7 @@ int sev_dev_get_attr(u32 group, u64 attr, u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code);int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +2
-2@@ -10627,9 +10627,9 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) { -kvm_x86_call (gmem_invalidate)(start, end); +kvm_x86_call (gmem_invalidate)(pfn, nr_pages); }void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {include/linux/kvm_host.h +1
-1@@ -2614,7 +2614,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src,#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif virt/kvm/guest_memfd.c +1
-6@@ -530,11 +530,7 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE static void kvm_gmem_free_folio (struct folio *folio) { -struct page *page =folio_page (folio,0 ); -kvm_pfn_t pfn =page_to_pfn (page); -int order =folio_order (folio); - -kvm_arch_gmem_invalidate (pfn, pfn + (1ul << order)); +kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio)); }#endif --2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 1/9] KVM: guest_memfd: Pass the number of pages instead of the end pfn into .invalidate()
In reply to Sean ChristophersonQuoted 11 lines · Sean Christopherson
Pass the number of pages to "invalidate", i.e. reclaim, instead of the end pfn, as a first step towards aligning the function prototypes between the de facto "to private" and "to shared" arch hooks. Eventually, the goal is to end up with kvm_gmem_arch_make_{private,shared}(), and in both cases, providing the number of pages makes the call sites slightly nicer, and also avoids any confusion over whether the end pfn is inclusive or exclusive.
Opportunistically rename "start" to "pfn", again to align with the expected signature of
make_private()(which needs to pass a starting gfn as well, at which point the "start" becomes noise).
I like ranges being defined as "start" and "nr_pages" because it removes the ambiguity of whether "end" is inclusive or exclusive :)
Quoted 8 lines
[...snip...]
virt/kvm/guest_memfd.c +0
-0
virt/kvm/guest_memfd.c static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol · line 533
Quoted 9 lines
virt/kvm/guest_memfd.c +1
-5@@ -530,11 +530,7 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE static void kvm_gmem_free_folio (struct folio *folio) { -struct page *page =folio_page (folio,0 ); -kvm_pfn_t pfn =page_to_pfn (page); -int order =folio_order (folio); - -kvm_arch_gmem_invalidate (pfn, pfn + (1ul << order)); +kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio));
This could be just folio_pfn(folio) instead of having to pass the 0 index. The name of this function is free_folio() after all, so we'd be invalidating (or later, reclaiming) the entire folio.
Quoted 5 lines
virt/kvm/guest_memfd.c +0
-1}#endif --2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 1/9] KVM: guest_memfd: Pass the number of pages instead of the end pfn into .invalidate()
In reply to Sean ChristophersonQuoted 11 lines · Sean Christopherson
Pass the number of pages to "invalidate", i.e. reclaim, instead of the end pfn, as a first step towards aligning the function prototypes between the de facto "to private" and "to shared" arch hooks. Eventually, the goal is to end up with kvm_gmem_arch_make_{private,shared}(), and in both cases, providing the number of pages makes the call sites slightly nicer, and also avoids any confusion over whether the end pfn is inclusive or exclusive.
Opportunistically rename "start" to "pfn", again to align with the expected signature of
make_private()(which needs to pass a starting gfn as well, at which point the "start" becomes noise).
Quoted 5 lines
No functional change intended.
Cc Fuad Tabba <[email protected]>Cc Ackerley Tng <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v6 2/9 KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it
In reply to Sean ChristophersonRename guest_memfd's invalidate() hook to reclaim() and isolate it via its own RECLAIM Kconfig, as the hook is called when a folio is freed, which is far too late and lacks sufficient information for KVM to actually invalidate its usage of the memory. E.g. SNP uses the hook to convert memory back to SHARED so that it can be safely accessed by the host, there is no invalidation of guest mappings anywhere. Isolating the hook will also allow pKVM on arm64 to opt-in to reclaim() without also having to differentiate between reclaim and conversions to shared for active VMs.
Keep guest_memfd's trampoline, even though it would be trivial to wire up .free_folio() directly to an arch callback, to avoid bleeding guest_memfd internals into arch code (specifically, avoid referencing folios in arch code).
Leave the kvm_x86_ops hook as-is for the moment, as "reclaim" on SNP is the same as convert-to-shared, i.e. using a different name for the x86 hook will allow reusing it for in-place conversion.
--- arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/x86.c | 7 +++++-- include/linux/kvm_host.h | 5 ++++- virt/kvm/Kconfig | 4 ++++ virt/kvm/guest_memfd.c | 6 +++--- 5 files changed, 17 insertions(+), 6 deletions(-)
arch/x86/kvm/Kconfig +1 -0
@@ -161,6 +161,7 @@ config KVM_AMD_SEV select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATE help
arch/x86/kvm/x86.c +5 -2
@@ -10626,11 +10626,14 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord }#endif -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_x86_call (gmem_invalidate)(pfn, nr_pages); } +#endif + +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {kvm_x86_call (gmem_invalidate_range)(kvm, range);
include/linux/kvm_host.h +4 -1
@@ -2613,8 +2613,11 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, kvm_gmem_populate_cb post_populate, void *opaque);#endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages); +#endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif
virt/kvm/Kconfig +4 -0
@@ -111,6 +111,10 @@ config HAVE_KVM_ARCH_GMEM_PREPARE bool depends on KVM_GUEST_MEMFD +config HAVE_KVM_ARCH_GMEM_RECLAIM + bool + depends on KVM_GUEST_MEMFD + config HAVE_KVM_ARCH_GMEM_INVALIDATE bool depends on KVM_GUEST_MEMFD
virt/kvm/guest_memfd.c +3 -3
@@ -527,10 +527,10 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *folreturn MF_DELAYED ; } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM static void kvm_gmem_free_folio (struct folio *folio) { -kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio)); +kvm_arch_gmem_reclaim (folio_file_pfn (folio,0 ),folio_nr_pages (folio)); }#endif
@@ -538,7 +538,7 @@ static const struct address_space_operations kvm_gmem_aops = { .dirty_folio = noop_dirty_folio, .migrate_folio = kvm_gmem_migrate_folio, .error_remove_folio = kvm_gmem_error_folio, -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM .free_folio = kvm_gmem_free_folio,#endif };
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Sean Christopherson
maintainerPATCH v6 3/9 KVM: x86: Rename kvm_x86_ops' gmem_invalidate() to gmem_make_shared()
In reply to Sean ChristophersonRename kvm_x86_ops's gmem_invalidate() hook to gmem_make_shared(), as the hook doesn't invalidate anything, and so that KVM doesn't need to add yet another vendor callback to support "convert to shared" once in-place conversion comes along.
Opportunistically wrap the ops declarations with a GMEM_RECLAIM guard so that attempting to wire up a .gmem_make_shared() hook without selecting CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM will result in a build failure.
No functional change intended.
--- arch/x86/include/asm/kvm-x86-ops.h | 4 +++- arch/x86/include/asm/kvm_host.h | 4 +++- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 2 +- 6 files changed, 10 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +3 -1
@@ -149,7 +149,9 @@ KVM_X86_OP_OPTIONAL(alloc_apic_backing_page)#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE KVM_X86_OP_OPTIONAL_RET0(gmem_prepare)#endif -KVM_X86_OP_OPTIONAL(gmem_invalidate) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +KVM_X86_OP_OPTIONAL(gmem_make_shared) +#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE KVM_X86_OP_OPTIONAL(gmem_invalidate_range)#endif
arch/x86/include/asm/kvm_host.h +3 -1
@@ -1727,7 +1727,9 @@ struct kvm_x86_ops {#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif -void (*gmem_invalidate)(kvm_pfn_t pfn,kvm_pfn_t nr_pages); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages); +#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif
arch/x86/kvm/svm/sev.c +1 -1
@@ -5161,7 +5161,7 @@ int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order)return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) +void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_pfn_t end = pfn + nr_pages;
arch/x86/kvm/svm/svm.c +1 -1
@@ -5439,7 +5439,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_move_enc_context_from = sev_vm_move_enc_context_from, .gmem_prepare = sev_gmem_prepare, - .gmem_invalidate = sev_gmem_invalidate, + .gmem_make_shared = sev_gmem_make_shared, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,#endif
arch/x86/kvm/svm/svm.h +1 -1
@@ -1010,7 +1010,7 @@ int sev_dev_get_attr(u32 group, u64 attr, u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code);int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); -void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages); +void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +1 -1
@@ -10629,7 +10629,7 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages) { -kvm_x86_call (gmem_invalidate)(pfn, nr_pages); +kvm_x86_call (gmem_make_shared)(pfn, nr_pages); }#endif
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 3/9] KVM: x86: Rename kvm_x86_ops' gmem_invalidate() to gmem_make_shared()
Quoted 6 lines · Sean Christopherson
Rename kvm_x86_ops's
gmem_invalidate()hook togmem_make_shared(), as the hook doesn't invalidate anything, and so that KVM doesn't need to add yet another vendor callback to support "convert to shared" once in-place conversion comes along.
Quoted 7 lines
Opportunistically wrap the ops declarations with a
GMEM_RECLAIMguard so that attempting to wire up a .gmem_make_shared() hook without selectingCONFIG_HAVE_KVM_ARCH_GMEM_RECLAIMwill result in a build failure.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Cheers,
/fuad
Quoted 94 lines
--- arch/x86/include/asm/kvm-x86-ops.h | 4 +++- arch/x86/include/asm/kvm_host.h | 4 +++- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 2 +- 6 files changed, 10 insertions(+), 6 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +3
-1@@ -149,7 +149,9 @@ KVM_X86_OP_OPTIONAL(alloc_apic_backing_page)#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE KVM_X86_OP_OPTIONAL_RET0(gmem_prepare)#endif -KVM_X86_OP_OPTIONAL(gmem_invalidate) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +KVM_X86_OP_OPTIONAL(gmem_make_shared) +#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE KVM_X86_OP_OPTIONAL(gmem_invalidate_range)#endif arch/x86/include/asm/kvm_host.h +3
-1@@ -1727,7 +1727,9 @@ struct kvm_x86_ops {#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif -void (*gmem_invalidate)(kvm_pfn_t pfn,kvm_pfn_t nr_pages); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages); +#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif arch/x86/kvm/svm/sev.c +1
-1@@ -5161,7 +5161,7 @@ int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order)return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) +void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_pfn_t end = pfn + nr_pages;arch/x86/kvm/svm/svm.c +1
-1@@ -5439,7 +5439,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_move_enc_context_from = sev_vm_move_enc_context_from, .gmem_prepare = sev_gmem_prepare, - .gmem_invalidate = sev_gmem_invalidate, + .gmem_make_shared = sev_gmem_make_shared, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,#endif arch/x86/kvm/svm/svm.h +1
-1@@ -1010,7 +1010,7 @@ int sev_dev_get_attr(u32 group, u64 attr, u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code);int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); -void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages); +void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +1
-2@@ -10629,7 +10629,7 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages) { -kvm_x86_call (gmem_invalidate)(pfn, nr_pages); +kvm_x86_call (gmem_make_shared)(pfn, nr_pages); }#endif --2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 3/9] KVM: x86: Rename kvm_x86_ops' gmem_invalidate() to gmem_make_shared()
In reply to Sean ChristophersonQuoted 12 lines · Sean Christopherson
Rename kvm_x86_ops's
gmem_invalidate()hook togmem_make_shared(), as the hook doesn't invalidate anything, and so that KVM doesn't need to add yet another vendor callback to support "convert to shared" once in-place conversion comes along.Opportunistically wrap the ops declarations with a
GMEM_RECLAIMguard so that attempting to wire up a .gmem_make_shared() hook without selectingCONFIG_HAVE_KVM_ARCH_GMEM_RECLAIMwill result in a build failure.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 9 lines
--- arch/x86/include/asm/kvm-x86-ops.h | 4 +++- arch/x86/include/asm/kvm_host.h | 4 +++- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 2 +- 6 files changed, 10 insertions(+), 6 deletions(-)
This works well for the conversions series (will post soon, bit of testing to go).
arch/x86/include/asm/kvm-x86-ops.h KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) · line 152
Quoted 9 lines
arch/x86/include/asm/kvm-x86-ops.h +0
-1@@ -149,7 +149,9 @@ KVM_X86_OP_OPTIONAL(alloc_apic_backing_page)#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE KVM_X86_OP_OPTIONAL_RET0(gmem_prepare)#endif -KVM_X86_OP_OPTIONAL (gmem_invalidate)
I updated the next line (for conversions series) to
#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT) ||defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM)
Quoted 11 lines
arch/x86/include/asm/kvm-x86-ops.h +3
-0+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +KVM_X86_OP_OPTIONAL(gmem_make_shared) +#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE KVM_X86_OP_OPTIONAL(gmem_invalidate_range)#endif arch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_x86_ops { · line 1730
Quoted 4 lines
arch/x86/include/asm/kvm_host.h +0
-1@@ -1727,7 +1727,9 @@ struct kvm_x86_ops {#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif -void (*gmem_invalidate)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);
And this next line too
Quoted 24 lines
arch/x86/include/asm/kvm_host.h +3
-0+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages); +#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif arch/x86/kvm/svm/sev.c +1
-1@@ -5161,7 +5161,7 @@ int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order)return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages) +void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_pfn_t end = pfn + nr_pages;arch/x86/kvm/svm/svm.c +0
-0
arch/x86/kvm/svm/svm.c struct kvm_x86_ops svm_x86_ops __initdata = { · line 5442
Quoted 5 lines
arch/x86/kvm/svm/svm.c +1
-1@@ -5439,7 +5439,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_move_enc_context_from = sev_vm_move_enc_context_from, .gmem_prepare = sev_gmem_prepare, - .gmem_invalidate = sev_gmem_invalidate, + .gmem_make_shared = sev_gmem_make_shared,
And no change here, because this is gated on SNP-ness, and SNP is linked to the GMEM callback CONFIGs by Kconfig. :)
arch/x86/kvm/svm/svm.c struct kvm_x86_ops svm_x86_ops __initdata = {
Quoted 6 lines
arch/x86/kvm/svm/svm.c +0
-0.gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,#endif
Re: [PATCH v6 3/9] KVM: x86: Rename kvm_x86_ops' gmem_invalidate() to gmem_make_shared()
In reply to Sean ChristophersonQuoted 5 lines · Sean Christopherson
Rename kvm_x86_ops's
gmem_invalidate()hook togmem_make_shared(), as the hook doesn't invalidate anything, and so that KVM doesn't need to add yet another vendor callback to support "convert to shared" once in-place conversion comes along.
Quoted 7 lines
Opportunistically wrap the ops declarations with a
GMEM_RECLAIMguard so that attempting to wire up a .gmem_make_shared() hook without selectingCONFIG_HAVE_KVM_ARCH_GMEM_RECLAIMwill result in a build failure.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v6 4/9 KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare() failure
In reply to Sean ChristophersonDrop guest_memfd's ratelimited printk to log "preparation" failures, as KVM SNP already logs more precise messages in all error paths, and whether or not failure to convert the pfn to private is "unexpected", i.e. warrants logging, is firmly an architecture specific detail.
--- virt/kvm/guest_memfd.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
virt/kvm/guest_memfd.c +3 -7
@@ -66,15 +66,11 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slo#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -int rc =kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); -if (rc) { -pr_warn_ratelimited ("gmem: Failed to prepare folio for index %lx GFN %llx PFN %llx error %d.\n" , - index, gfn, pfn, rc); -return rc; - } -#endif +return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +#else return 0 ; +#endif }/*
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Sean Christopherson
maintainerPATCH v6 5/9 KVM: guest_memfd: Add helpers to query SHARED vs. PRIVATE for a given page
In reply to Sean ChristophersonAdd helpers to check if a given page in a guest_memfd instance is PRIVATE versus SHARED, and use the "is shared" helper instead of an open-coded equivalent in the user pagefault handler. In addition to the immediate usage, providing an "is private" helper will allow cleaning up the so called prepare() code, and eventually will be heavily used once in-place conversion support comes along.
No functional change intended.
--- virt/kvm/guest_memfd.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
virt/kvm/guest_memfd.c +11 -1
@@ -60,6 +60,16 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)return gfn - slot->base_gfn + slot->gmem.pgoff; } +static bool kvm_gmem_is_private_mem (struct inode *inode,pgoff_t index) +{ +return !(GMEM_I (inode)->flags &GUEST_MEMFD_FLAG_INIT_SHARED ); +} + +static bool kvm_gmem_is_shared_mem (struct inode *inode,pgoff_t index) +{ +return !kvm_gmem_is_private_mem (inode, index); +} +static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,pgoff_t index,struct folio *folio) {
@@ -397,7 +407,7 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)if (((loff_t )vmf->pgoff <<PAGE_SHIFT ) >=i_size_read (inode))return VM_FAULT_SIGBUS ; -if (!(GMEM_I (inode)->flags &GUEST_MEMFD_FLAG_INIT_SHARED )) +if (!kvm_gmem_is_shared_mem (inode, vmf->pgoff))return VM_FAULT_SIGBUS ; folio =kvm_gmem_get_folio (inode, vmf->pgoff);
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 5/9] KVM: guest_memfd: Add helpers to query SHARED vs. PRIVATE for a given page
Quoted 11 lines · Sean Christopherson
Add helpers to check if a given page in a
guest_memfdinstance is PRIVATE versus SHARED, and use the "is shared" helper instead of an open-coded equivalent in the user pagefault handler. In addition to the immediate usage, providing an "is private" helper will allow cleaning up the so calledprepare()code, and eventually will be heavily used once in-place conversion support comes along.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Cheers,
/fuad
Quoted 37 lines
--- virt/kvm/guest_memfd.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)virt/kvm/guest_memfd.c +11
-2@@ -60,6 +60,16 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)return gfn - slot->base_gfn + slot->gmem.pgoff; } +static bool kvm_gmem_is_private_mem (struct inode *inode,pgoff_t index) +{ +return !(GMEM_I (inode)->flags &GUEST_MEMFD_FLAG_INIT_SHARED ); +} + +static bool kvm_gmem_is_shared_mem (struct inode *inode,pgoff_t index) +{ +return !kvm_gmem_is_private_mem (inode, index); +} +static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,pgoff_t index,struct folio *folio) {@@ -397,7 +407,7 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)if (((loff_t )vmf->pgoff <<PAGE_SHIFT ) >=i_size_read (inode))return VM_FAULT_SIGBUS ; -if (!(GMEM_I (inode)->flags &GUEST_MEMFD_FLAG_INIT_SHARED )) +if (!kvm_gmem_is_shared_mem (inode, vmf->pgoff))return VM_FAULT_SIGBUS ; folio =kvm_gmem_get_folio (inode, vmf->pgoff); --2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 5/9] KVM: guest_memfd: Add helpers to query SHARED vs. PRIVATE for a given page
In reply to Sean ChristophersonQuoted 22 lines · Sean Christopherson
Add helpers to check if a given page in a
guest_memfdinstance is PRIVATE versus SHARED, and use the "is shared" helper instead of an open-coded equivalent in the user pagefault handler. In addition to the immediate usage, providing an "is private" helper will allow cleaning up the so calledprepare()code, and eventually will be heavily used once in-place conversion support comes along.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- virt/kvm/guest_memfd.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)virt/kvm/guest_memfd.c +0
-0@@ -60,6 +60,16 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)return gfn - slot->base_gfn + slot->gmem.pgoff; }
virt/kvm/guest_memfd.c static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn) · line 72
Quoted 10 lines
virt/kvm/guest_memfd.c +10
-0+static bool kvm_gmem_is_private_mem (struct inode *inode,pgoff_t index) +{ +return !(GMEM_I (inode)->flags &GUEST_MEMFD_FLAG_INIT_SHARED ); +} + +static bool kvm_gmem_is_shared_mem (struct inode *inode,pgoff_t index) +{ +return !kvm_gmem_is_private_mem (inode, index); +} +
Works well for the conversions series, thanks!
Quoted 14 lines
virt/kvm/guest_memfd.c +1
-2static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,pgoff_t index,struct folio *folio) {@@ -397,7 +407,7 @@ static vm_fault_t kvm_gmem_fault_user_mapping(struct vm_fault *vmf)if (((loff_t )vmf->pgoff <<PAGE_SHIFT ) >=i_size_read (inode))return VM_FAULT_SIGBUS ; -if (!(GMEM_I (inode)->flags &GUEST_MEMFD_FLAG_INIT_SHARED )) +if (!kvm_gmem_is_shared_mem (inode, vmf->pgoff))return VM_FAULT_SIGBUS ; folio =kvm_gmem_get_folio (inode, vmf->pgoff); --2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 5/9] KVM: guest_memfd: Add helpers to query SHARED vs. PRIVATE for a given page
In reply to Sean ChristophersonQuoted 10 lines · Sean Christopherson
Add helpers to check if a given page in a
guest_memfdinstance is PRIVATE versus SHARED, and use the "is shared" helper instead of an open-coded equivalent in the user pagefault handler. In addition to the immediate usage, providing an "is private" helper will allow cleaning up the so calledprepare()code, and eventually will be heavily used once in-place conversion support comes along.No functional change intended.
Signed-off-by Sean Christopherson<[email protected]>
Sean Christopherson
maintainerPATCH v6 6/9 KVM: guest_memfd: Only "prepare" folios for private pages
In reply to Sean ChristophersonFrom: Ackerley Tng <[email protected]>
When getting a guest_memfd pfn, prepare the folio, i.e. convert its pages to private, if and only if the page is actually private. The misnamed prepare() hook exists specifically to allow x86's SNP to assign pages to the owning VM in the RMP when mapping private memory into a guest.
Guarding the call will allow renaming the prepare() hook to better reflect its role, without creating a semantic mess, and will become a hard requirement once in-place conversion is supported, i.e. when CoCo VMs support SHARED guest_memfd pages.
For all intents, no functional change intended (the sole arch hook is a nop for SHARED memory).
--- virt/kvm/guest_memfd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
virt/kvm/guest_memfd.c +2 -1
@@ -814,7 +814,8 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); +if (kvm_gmem_is_private_mem (file_inode (file), index)) + r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio);folio_unlock (folio);
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 6/9] KVM: guest_memfd: Only "prepare" folios for private pages
Quoted 12 lines · Sean Christopherson
From: Ackerley Tng <[email protected]>
When getting a
guest_memfdpfn, prepare the folio, i.e. convert its pages to private, if and only if the page is actually private. The misnamedprepare()hook exists specifically to allow x86's SNP to assign pages to the owning VM in the RMP when mapping private memory into a guest.Guarding the call will allow renaming the
prepare()hook to better reflect its role, without creating a semantic mess, and will become a hard requirement once in-place conversion is supported, i.e. when CoCo VMs support SHAREDguest_memfdpages.
Quoted 7 lines
For all intents, no functional change intended (the sole arch hook is a nop for SHARED memory).
Suggested-by Michael Roth <[email protected]>Reviewed-by Fuad Tabba <[email protected]>[sean: rewrite changelog to fit the context]Signed-off-by Ackerley Tng <[email protected]>
The rewritten the changelog read better and clearer!
Quoted 18 lines
--- virt/kvm/guest_memfd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)virt/kvm/guest_memfd.c +2
-1@@ -814,7 +814,8 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); +if (kvm_gmem_is_private_mem (file_inode (file), index)) + r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio);folio_unlock (folio);
Sean Christopherson
maintainerPATCH v6 7/9 KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
In reply to Sean ChristophersonRework guest_memfd's prepare() hook into a more accurate make_private(), and rework its Kconfig from PREPARE to a more generic CONVERT. This will allow x86 to share (pun intended) a kvm_x86_ops.gmem_make_shared() hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.
No functional change intended.
--- arch/x86/include/asm/kvm-x86-ops.h | 4 ++-- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 6 +++--- include/linux/kvm_host.h | 5 +++-- virt/kvm/Kconfig | 2 +- virt/kvm/guest_memfd.c | 4 ++-- 10 files changed, 17 insertions(+), 16 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +2 -2
@@ -146,8 +146,8 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +KVM_X86_OP_OPTIONAL_RET0(gmem_make_private) #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM KVM_X86_OP_OPTIONAL(gmem_make_shared)
arch/x86/include/asm/kvm_host.h +2 -2
@@ -1724,8 +1724,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int (*gmem_make_private)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);
arch/x86/kvm/Kconfig +1 -1
@@ -160,7 +160,7 @@ config KVM_AMD_SEV depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES - select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATE
arch/x86/kvm/svm/sev.c +1 -1
@@ -5114,7 +5114,7 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)return false; } -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) +int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;
arch/x86/kvm/svm/svm.c +1 -1
@@ -5438,7 +5438,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_copy_enc_context_from = sev_vm_copy_enc_context_from, .vm_move_enc_context_from = sev_vm_move_enc_context_from, - .gmem_prepare = sev_gmem_prepare, + .gmem_make_private = sev_gmem_make_private, .gmem_make_shared = sev_gmem_make_shared, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,
arch/x86/kvm/svm/svm.h +1 -1
@@ -1009,7 +1009,7 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);
arch/x86/kvm/x86.c +3 -3
@@ -10619,10 +10619,10 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)return !kvm_arch_has_private_mem (kvm); } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) { -return kvm_x86_call (gmem_prepare)(kvm, pfn, gfn, max_order); +return kvm_x86_call (gmem_make_private)(kvm, pfn, gfn, max_order); }#endif
include/linux/kvm_host.h +3 -2
@@ -2579,8 +2579,9 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, }#endif /* CONFIG_KVM_GUEST_MEMFD */ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE
virt/kvm/Kconfig +1 -1
@@ -107,7 +107,7 @@ config KVM_GUEST_MEMFD select XARRAY_MULTI bool -config HAVE_KVM_ARCH_GMEM_PREPARE +config HAVE_KVM_ARCH_GMEM_CONVERT bool depends on KVM_GUEST_MEMFD
virt/kvm/guest_memfd.c +2 -2
@@ -73,11 +73,11 @@ static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,pgoff_t index,struct folio *folio) { -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_order (folio));#else return 0 ;#endif
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
Quoted 10 lines · Sean Christopherson
Rework guest_memfd's
prepare()hook into a more accuratemake_private(), and rework itsKconfigfrom PREPARE to a more generic CONVERT. This will allow x86 to share (pun intended) akvm_x86_ops.gmem_make_shared()hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Cheers,
/fuad
Quoted 163 lines
--- arch/x86/include/asm/kvm-x86-ops.h | 4 ++-- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 6 +++--- include/linux/kvm_host.h | 5 +++-- virt/kvm/Kconfig | 2 +- virt/kvm/guest_memfd.c | 4 ++-- 10 files changed, 17 insertions(+), 16 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +2
-2@@ -146,8 +146,8 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +KVM_X86_OP_OPTIONAL_RET0(gmem_make_private) #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM KVM_X86_OP_OPTIONAL(gmem_make_shared)arch/x86/include/asm/kvm_host.h +2
-2@@ -1724,8 +1724,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int (*gmem_make_private)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);arch/x86/kvm/Kconfig +1
-1@@ -160,7 +160,7 @@ config KVM_AMD_SEV depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES - select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATEarch/x86/kvm/svm/sev.c +1
-1@@ -5114,7 +5114,7 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)return false; } -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) +int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;arch/x86/kvm/svm/svm.c +1
-1@@ -5438,7 +5438,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_copy_enc_context_from = sev_vm_copy_enc_context_from, .vm_move_enc_context_from = sev_vm_move_enc_context_from, - .gmem_prepare = sev_gmem_prepare, + .gmem_make_private = sev_gmem_make_private, .gmem_make_shared = sev_gmem_make_shared, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,arch/x86/kvm/svm/svm.h +1
-1@@ -1009,7 +1009,7 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);arch/x86/kvm/x86.c +3
-3@@ -10619,10 +10619,10 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)return !kvm_arch_has_private_mem (kvm); } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) { -return kvm_x86_call (gmem_prepare)(kvm, pfn, gfn, max_order); +return kvm_x86_call (gmem_make_private)(kvm, pfn, gfn, max_order); }#endif include/linux/kvm_host.h +3
-2@@ -2579,8 +2579,9 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, }#endif /* CONFIG_KVM_GUEST_MEMFD */ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE virt/kvm/Kconfig +1
-1@@ -107,7 +107,7 @@ config KVM_GUEST_MEMFD select XARRAY_MULTI bool -config HAVE_KVM_ARCH_GMEM_PREPARE +config HAVE_KVM_ARCH_GMEM_CONVERT bool depends on KVM_GUEST_MEMFDvirt/kvm/guest_memfd.c +2
-3@@ -73,11 +73,11 @@ static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,pgoff_t index,struct folio *folio) { -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_order (folio));#else return 0 ;#endif --2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
In reply to Sean ChristophersonQuoted 9 lines · Sean Christopherson
Rework guest_memfd's
prepare()hook into a more accuratemake_private(), and rework itsKconfigfrom PREPARE to a more generic CONVERT. This will allow x86 to share (pun intended) akvm_x86_ops.gmem_make_shared()hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Quoted 3 lines
[...snip...]
Re: [PATCH v6 7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
In reply to Sean ChristophersonQuoted 7 lines · Sean Christopherson
Rework guest_memfd's
prepare()hook into a more accuratemake_private(), and rework itsKconfigfrom PREPARE to a more generic CONVERT.This will
allow x86 to share (pun intended) akvm_x86_ops.gmem_make_shared()hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.
Does it mean in the future we will introduce something like below?
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_order)
{
return kvm_x86_call(gmem_make_private)(kvm, pfn, gfn, max_order);
}+int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages) +{
+ return kvm_x86_call(gmem_make_private)(pfn, nr_pages);
+}
#endif
Quoted 163 lines
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 4 ++-- arch/x86/include/asm/kvm_host.h | 4 ++-- arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 6 +++--- include/linux/kvm_host.h | 5 +++-- virt/kvm/Kconfig | 2 +- virt/kvm/guest_memfd.c | 4 ++-- 10 files changed, 17 insertions(+), 16 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +2
-2@@ -146,8 +146,8 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +KVM_X86_OP_OPTIONAL_RET0(gmem_make_private) #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM KVM_X86_OP_OPTIONAL(gmem_make_shared)arch/x86/include/asm/kvm_host.h +2
-2@@ -1724,8 +1724,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int (*gmem_make_private)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);arch/x86/kvm/Kconfig +1
-1@@ -160,7 +160,7 @@ config KVM_AMD_SEV depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES - select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATEarch/x86/kvm/svm/sev.c +1
-1@@ -5114,7 +5114,7 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)return false; } -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) +int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;arch/x86/kvm/svm/svm.c +1
-1@@ -5438,7 +5438,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_copy_enc_context_from = sev_vm_copy_enc_context_from, .vm_move_enc_context_from = sev_vm_move_enc_context_from, - .gmem_prepare = sev_gmem_prepare, + .gmem_make_private = sev_gmem_make_private, .gmem_make_shared = sev_gmem_make_shared, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,arch/x86/kvm/svm/svm.h +1
-1@@ -1009,7 +1009,7 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order);void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);arch/x86/kvm/x86.c +3
-3@@ -10619,10 +10619,10 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)return !kvm_arch_has_private_mem (kvm); } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) { -return kvm_x86_call (gmem_prepare)(kvm, pfn, gfn, max_order); +return kvm_x86_call (gmem_make_private)(kvm, pfn, gfn, max_order); }#endif include/linux/kvm_host.h +3
-2@@ -2579,8 +2579,9 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, }#endif /* CONFIG_KVM_GUEST_MEMFD */ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE virt/kvm/Kconfig +1
-1@@ -107,7 +107,7 @@ config KVM_GUEST_MEMFD select XARRAY_MULTI bool -config HAVE_KVM_ARCH_GMEM_PREPARE +config HAVE_KVM_ARCH_GMEM_CONVERT bool depends on KVM_GUEST_MEMFDvirt/kvm/guest_memfd.c +2
-2@@ -73,11 +73,11 @@ static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,pgoff_t index,struct folio *folio) { -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_order (folio));#else return 0 ;#endif
Sean Christopherson
maintainerRe: [PATCH v6 7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
Quoted 18 lines · Xiaoyao Li
Quoted 7 lines · Sean Christopherson
Rework guest_memfd's
prepare()hook into a more accuratemake_private(), and rework itsKconfigfrom PREPARE to a more generic CONVERT.This will
allow x86 to share (pun intended) akvm_x86_ops.gmem_make_shared()hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.Does it mean in the future we will introduce something like below?
#ifdef
CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
intkvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_order){ return kvm_x86_call(gmem_make_private)(kvm, pfn, gfn, max_order); }
Review
Quoted 3 lines
(quoted diff) +3
-0+int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages) +{ + return kvm_x86_call(gmem_make_private)(pfn, nr_pages);
s/gmem_make_private/gmem_make_shared, but yeah, that's the plan for the in-place conversion series.
Re: [PATCH v6 7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
Quoted 19 lines · Sean Christopherson
Quoted 18 lines · Xiaoyao Li
Quoted 7 lines · Sean Christopherson
Rework guest_memfd's
prepare()hook into a more accuratemake_private(), and rework itsKconfigfrom PREPARE to a more generic CONVERT.This will
allow x86 to share (pun intended) akvm_x86_ops.gmem_make_shared()hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.Does it mean in the future we will introduce something like below?
#ifdef
CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
intkvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_order){ return kvm_x86_call(gmem_make_private)(kvm, pfn, gfn, max_order); }
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages) +{ + return kvm_x86_call(gmem_make_private)(pfn, nr_pages);s/gmem_make_private/gmem_make_shared, but yeah, that's the plan for the in-place conversion series.
Yup! Did this.
Should be able to send out v9 today, just need to run some tests...
Xiaoyao Li
Reviewed-byRe: [PATCH v6 7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT
In reply to Sean ChristophersonQuoted 19 lines · Sean Christopherson
Quoted 18 lines · Xiaoyao Li
Quoted 7 lines · Sean Christopherson
Rework guest_memfd's
prepare()hook into a more accuratemake_private(), and rework itsKconfigfrom PREPARE to a more generic CONVERT.This will
allow x86 to share (pun intended) akvm_x86_ops.gmem_make_shared()hook between the "convert to shared" and "reclaim" flows, which are one and the same for SNP.Does it mean in the future we will introduce something like below?
#ifdef
CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
intkvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_order){ return kvm_x86_call(gmem_make_private)(kvm, pfn, gfn, max_order); }
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages) +{ + return kvm_x86_call(gmem_make_private)(pfn, nr_pages);s/gmem_make_private/gmem_make_shared, but yeah, that's the plan for the in-place conversion series.
/faceplam my copy-paste mistake.
The patch looks good to me,
Sean Christopherson
maintainerPATCH v6 8/9 KVM: guest_memfd: Explicitly pass number of pages to make_private() hook
In reply to Sean ChristophersonTweak the guest_memfd make_private() hook to explicitly pass the number of pages to align with the signature of the make_shared() hook, and because the existing code is outright broken if a guest_memfd folio is comprised of more than one page (which can't happen, yet). The SNP code *tries* to create a corresponding huge entry, but if the RMP must use 4KiB entries for whatever reason, KVM will only convert the first pfn, and not the entire range of pfns that will be mapped into the guest.
Alternatively, @max_order could simply be repurposed as _the_ @order, but that will fall apart when in-place conversion comes along, at which point KVM will need to deal with conversions that aren't bound 1:1 to a folio. I.e. the number of pages to convert may not be exactly be a power-of-2 (and folios don't strictly guarantee power-of-2 pages anyways).
WARN in the SNP code if the number of pages to prepare is anything other than '1', i.e. if guest_memfd is trying to prepare/convert more than a single 4KiB page, as sev_gmem_prepare() doesn't actually handle conversion greater than order-0 folios.
Opportunistically swap the ordering of @pfn and @gfn params for kvm_x86_ops.gmem_make_private() to match kvm_arch_gmem_make_private().
--- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/svm/sev.c | 27 +++++++++++---------------- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 5 +++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 2 +- 6 files changed, 19 insertions(+), 22 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -1
@@ -1725,7 +1725,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT -int (*gmem_make_private)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int (*gmem_make_private)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);
arch/x86/kvm/svm/sev.c +11 -16
@@ -5090,15 +5090,7 @@ static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end)return true; } -static u8 max_level_for_order (int order) -{ -if (order >=KVM_HPAGE_GFN_SHIFT (PG_LEVEL_2M )) -return PG_LEVEL_2M ; - -return PG_LEVEL_4K ; -} - -static bool is_large_rmp_possible (struct kvm *kvm,kvm_pfn_t pfn,int order) +static bool is_large_rmp_possible (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_pfn_t pfn_aligned =ALIGN_DOWN (pfn,PTRS_PER_PMD );
@@ -5107,14 +5099,14 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)* PFN is currently shared, then the entire 2M-aligned range can be * set to private via a single 2M RMP entry. */ -if (max_level_for_order (order) >PG_LEVEL_4K && +if (nr_pages >=KVM_PAGES_PER_HPAGE (PG_LEVEL_2M ) &&is_pfn_range_shared (pfn_aligned, pfn_aligned +PTRS_PER_PMD ))return true;return false; } -int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) +int sev_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;
@@ -5125,6 +5117,9 @@ int sev_gmem_make_private(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_ordif (!sev_snp_guest (kvm))return 0 ; +if (WARN_ON_ONCE (nr_pages !=1 )) +return -EIO ; + rc =snp_lookup_rmpentry (pfn, &assigned, &level);if (rc) { pr_err_ratelimited("SEV: Failed to look up RMP entry: GFN %llx PFN %llx error %d\n",
@@ -5133,12 +5128,12 @@ int sev_gmem_make_private(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_ord }if (assigned) { -pr_debug ("%s: already assigned: gfn %llx pfn %llx max_order %d level %d\n" , - __func__, gfn, pfn, max_order, level); +pr_debug ("%s: already assigned: gfn %llx pfn %llx nr_pages %llx level %d\n" , + __func__, gfn, pfn, nr_pages, level);return 0 ; } -if (is_large_rmp_possible (kvm, pfn, max_order)) { +if (is_large_rmp_possible (pfn, nr_pages)) { level =PG_LEVEL_2M ; pfn_aligned =ALIGN_DOWN (pfn,PTRS_PER_PMD ); gfn_aligned =ALIGN_DOWN (gfn,PTRS_PER_PMD );
@@ -5155,8 +5150,8 @@ int sev_gmem_make_private(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_ordreturn -EINVAL ; } -pr_debug ("%s: updated: gfn %llx pfn %llx pfn_aligned %llx max_order %d level %d\n" , - __func__, gfn, pfn, pfn_aligned, max_order, level); +pr_debug ("%s: updated: gfn %llx pfn %llx pfn_aligned %llx nr_pages %llx level %d\n" , + __func__, gfn, pfn, pfn_aligned, nr_pages, level);return 0 ; }
arch/x86/kvm/svm/svm.h +1 -1
@@ -1009,7 +1009,7 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int sev_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);
arch/x86/kvm/x86.c +3 -2
@@ -10620,9 +10620,10 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm) }#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT -int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages) { -return kvm_x86_call (gmem_make_private)(kvm, pfn, gfn, max_order); +return kvm_x86_call (gmem_make_private)(kvm, gfn, pfn, nr_pages); }#endif
include/linux/kvm_host.h +1 -1
@@ -2581,7 +2581,7 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm,#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -int max_order); +kvm_pfn_t nr_pages);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE
virt/kvm/guest_memfd.c +1 -1
@@ -77,7 +77,7 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slokvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_order (folio)); +return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_nr_pages (folio));#else return 0 ;#endif
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 8/9] KVM: guest_memfd: Explicitly pass number of pages to make_private() hook
Quoted 69 lines · Sean Christopherson
Tweak the
guest_memfdmake_private()hook to explicitly pass the number of pages to align with the signature of themake_shared()hook, and because the existing code is outright broken if aguest_memfdfolio is comprised of more than one page (which can't happen, yet). The SNP code *tries* to create a corresponding huge entry, but if the RMP must use 4KiB entries for whatever reason, KVM will only convert the first pfn, and not the entire range of pfns that will be mapped into the guest.Alternatively, @max_order could simply be repurposed as _the_ @order, but that will fall apart when in-place conversion comes along, at which point KVM will need to deal with conversions that aren't bound 1:1 to a folio. I.e. the number of pages to convert may not be exactly be a power-of-2 (and folios don't strictly guarantee power-of-2 pages anyways).
WARN in the SNP code if the number of pages to prepare is anything other than '1', i.e. if
guest_memfdis trying to prepare/convert more than a single 4KiB page, assev_gmem_prepare()doesn't actually handle conversion greater than order-0 folios.Opportunistically swap the ordering of @pfn and @gfn params for
kvm_x86_ops.gmem_make_private()to matchkvm_arch_gmem_make_private().Fixes b85524314a3d ("KVM: guest_memfd: delay kvm_gmem_prepare_folio() until the memory is passed to the guest")Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/svm/sev.c | 27 +++++++++++---------------- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 5 +++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 2 +- 6 files changed, 19 insertions(+), 22 deletions(-)arch/x86/include/asm/kvm_host.h +2
-1@@ -1725,7 +1725,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT -int (*gmem_make_private)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int (*gmem_make_private)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);arch/x86/kvm/svm/sev.c +1
-9@@ -5090,15 +5090,7 @@ static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end)return true; } -static u8 max_level_for_order (int order) -{ -if (order >=KVM_HPAGE_GFN_SHIFT (PG_LEVEL_2M )) -return PG_LEVEL_2M ; - -return PG_LEVEL_4K ; -} - -static bool is_large_rmp_possible (struct kvm *kvm,kvm_pfn_t pfn,int order) +static bool is_large_rmp_possible (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_pfn_t pfn_aligned =ALIGN_DOWN (pfn,PTRS_PER_PMD );
arch/x86/kvm/svm/sev.c static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order) · line 5103
Quoted 6 lines
arch/x86/kvm/svm/sev.c +1
-1@@ -5107,14 +5099,14 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)* PFN is currently shared, then the entire 2M-aligned range can be * set to private via a single 2M RMP entry. */ -if (max_level_for_order (order) >PG_LEVEL_4K && +if (nr_pages >=KVM_PAGES_PER_HPAGE (PG_LEVEL_2M ) &&is_pfn_range_shared (pfn_aligned, pfn_aligned +PTRS_PER_PMD ))
This could actually just be return (everything in the if condition).
Thinking more about this, asking is_large_rmp_possible() of a pfn and nr_pages is perhaps the wrong question, since if nr_pages stretches into the next 2M range, then you might have a different answer for the start and end (and in-between) 2M ranges.
We can solve all these when we get to huge pages :)
arch/x86/kvm/svm/sev.c static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order) · line 5108
Quoted 5 lines
arch/x86/kvm/svm/sev.c +0
-0return true;return false; }
Quoted 3 lines
[...snip...]
Re: [PATCH v6 8/9] KVM: guest_memfd: Explicitly pass number of pages to make_private() hook
In reply to Sean ChristophersonQuoted 8 lines · Sean Christopherson
Tweak the
guest_memfdmake_private()hook to explicitly pass the number of pages to align with the signature of themake_shared()hook, and because the existing code is outright broken if aguest_memfdfolio is comprised of more than one page (which can't happen, yet). The SNP code *tries* to create a corresponding huge entry, but if the RMP must use 4KiB entries for whatever reason, KVM will only convert the first pfn, and not the entire range of pfns that will be mapped into the guest.
Quoted 4 lines
Alternatively, @max_order could simply be repurposed as _the_ @order, but that will fall apart when in-place conversion comes along, at which point KVM will need to deal with conversions that aren't bound 1:1 to a folio. I.e. the number of pages to convert may not be exactly be a power-of-2 (and
one redundant 'be'
Quoted 7 lines
folios don't strictly guarantee power-of-2 pages anyways).
WARN in the SNP code if the number of pages to prepare is anything other than '1', i.e. if
guest_memfdis trying to prepare/convert more than a single 4KiB page, assev_gmem_prepare()doesn't actually handle conversion greater than order-0 folios.
Quoted 5 lines
Opportunistically swap the ordering of @pfn and @gfn params for
kvm_x86_ops.gmem_make_private()to matchkvm_arch_gmem_make_private().Fixes b85524314a3d ("KVM: guest_memfd: delay kvm_gmem_prepare_folio() until the memory is passed to the guest")Signed-off-by Sean Christopherson <[email protected]>
Quoted 154 lines
--- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/svm/sev.c | 27 +++++++++++---------------- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/x86.c | 5 +++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 2 +- 6 files changed, 19 insertions(+), 22 deletions(-)arch/x86/include/asm/kvm_host.h +2
-1@@ -1725,7 +1725,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT -int (*gmem_make_private)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int (*gmem_make_private)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void (*gmem_make_shared)(kvm_pfn_t pfn,kvm_pfn_t nr_pages);arch/x86/kvm/svm/sev.c +11
-16@@ -5090,15 +5090,7 @@ static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end)return true; } -static u8 max_level_for_order (int order) -{ -if (order >=KVM_HPAGE_GFN_SHIFT (PG_LEVEL_2M )) -return PG_LEVEL_2M ; - -return PG_LEVEL_4K ; -} - -static bool is_large_rmp_possible (struct kvm *kvm,kvm_pfn_t pfn,int order) +static bool is_large_rmp_possible (kvm_pfn_t pfn,kvm_pfn_t nr_pages) {kvm_pfn_t pfn_aligned =ALIGN_DOWN (pfn,PTRS_PER_PMD );@@ -5107,14 +5099,14 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)* PFN is currently shared, then the entire 2M-aligned range can be * set to private via a single 2M RMP entry. */ -if (max_level_for_order (order) >PG_LEVEL_4K && +if (nr_pages >=KVM_PAGES_PER_HPAGE (PG_LEVEL_2M ) &&is_pfn_range_shared (pfn_aligned, pfn_aligned +PTRS_PER_PMD ))return true;return false; } -int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) +int sev_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;@@ -5125,6 +5117,9 @@ int sev_gmem_make_private(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_ordif (!sev_snp_guest (kvm))return 0 ; +if (WARN_ON_ONCE (nr_pages !=1 )) +return -EIO ; + rc =snp_lookup_rmpentry (pfn, &assigned, &level);if (rc) { pr_err_ratelimited("SEV: Failed to look up RMP entry: GFN %llx PFN %llx error %d\n",@@ -5133,12 +5128,12 @@ int sev_gmem_make_private(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_ord }if (assigned) { -pr_debug ("%s: already assigned: gfn %llx pfn %llx max_order %d level %d\n" , - __func__, gfn, pfn, max_order, level); +pr_debug ("%s: already assigned: gfn %llx pfn %llx nr_pages %llx level %d\n" , + __func__, gfn, pfn, nr_pages, level);return 0 ; } -if (is_large_rmp_possible (kvm, pfn, max_order)) { +if (is_large_rmp_possible (pfn, nr_pages)) { level =PG_LEVEL_2M ; pfn_aligned =ALIGN_DOWN (pfn,PTRS_PER_PMD ); gfn_aligned =ALIGN_DOWN (gfn,PTRS_PER_PMD );@@ -5155,8 +5150,8 @@ int sev_gmem_make_private(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_ordreturn -EINVAL ; } -pr_debug ("%s: updated: gfn %llx pfn %llx pfn_aligned %llx max_order %d level %d\n" , - __func__, gfn, pfn, pfn_aligned, max_order, level); +pr_debug ("%s: updated: gfn %llx pfn %llx pfn_aligned %llx nr_pages %llx level %d\n" , + __func__, gfn, pfn, pfn_aligned, nr_pages, level);return 0 ; }arch/x86/kvm/svm/svm.h +1
-1@@ -1009,7 +1009,7 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_make_private (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int sev_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_make_shared (kvm_pfn_t pfn,kvm_pfn_t nr_pages);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);arch/x86/kvm/x86.c +3
-2@@ -10620,9 +10620,10 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm) }#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT -int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) +int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages) { -return kvm_x86_call (gmem_make_private)(kvm, pfn, gfn, max_order); +return kvm_x86_call (gmem_make_private)(kvm, gfn, pfn, nr_pages); }#endif include/linux/kvm_host.h +1
-1@@ -2581,7 +2581,7 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm,#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -int max_order); +kvm_pfn_t nr_pages);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE virt/kvm/guest_memfd.c +1
-1@@ -77,7 +77,7 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slokvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_order (folio)); +return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_nr_pages (folio));#else return 0 ;#endif
Sean Christopherson
maintainerPATCH v6 9/9 KVM: guest_memfd: Make private exactly what can be mapped on page fault
In reply to Sean ChristophersonWhen calling into arch code to make the underlying memory private, i.e. to assign memory to the VM in SNP's RMP table, assign/convert *exactly* the range of memory that can be mapped into the guest for the current page fault, instead of aggressively converting/assigning the entire folio. For SNP, the mapping size in the stage-2 page tables (Nested Page Tables, NPT) must be at least the size of the corresponding RMP entry, e.g. assigning a 2MiB mapping in the RMP when it can only be mapped at 4KiB granualarity will ultimate result in another page fault (#NPF for SNP) to "smash" the RMP down to the correct mapping size.
Assigning the entire folio was necessary back when guest_memfd tracked preparedness, which was done on a per-folio basis. At the time, it made sense to do per-folio tracking/preparation, because tracking per-folio meant guest_memfd didn't need to add a separate data structure to track that information, and doing per-folio tracking only works if the entire folio is prepared (or not).
Now that guest_memfd no longer does preparation tracking (see commit 8622ef05709f ("KVM: guest_memfd: Remove preparation tracking")), in favor having SNP query the RMP, per-folio preparation, i.e. per-folio conversions to private, doesn't make any sense.
*If* SNP allowed the RMP size to be greater than the NPT size, then per-folio conversion could theoretically provide marginal value, as it would allow KVM to assign a hugepage in the RMP even if it can only be mapped into the NPT with a smaller page, e.g. because of memslot alignment. The documentation of that reasoning would be something like this:
/*
* If the memory is private from KVM's perspective, and hardware tracks
* VM-assigned private memory in a dedicated data structure, i.e. not
* in the stage-2 page tables, then call into arch code to assign the
* entire folio to the guest. Assigning the entire folio, e.g. instead
* of only the memory being mapped into the guest, allows KVM to assign
* an entire hugepage of memory in the out-of-band structure even if
* KVM can only map a smaller page size into the MMU, e.g. because the
* gmem hugepage is spread across multiple memslots.
*/
But even *if* a future SNP implementation supported that behavior, the value added would be dubious, as having a huge folio that is fully private, but can only be mapped at a smaller granularity, would be rare. E.g. maybe for memory at the top of lower DRAM that has holes for non-RAM assets?
So, convert/assign exactly what guest_memfd allows the caller to map to simplify the guest_memfd code and provide a (super) minor performance optimization for SNP. E.g. once hugepage support comes along, guest_memfd will only need a single flow to compute "how much memory can be assigned and at what size".
--- virt/kvm/guest_memfd.c | 53 ++++++------------------------------------ 1 file changed, 7 insertions(+), 46 deletions(-)
virt/kvm/guest_memfd.c +7 -46
@@ -70,50 +70,6 @@ static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)return !kvm_gmem_is_private_mem (inode, index); } -static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot, -pgoff_t index,struct folio *folio) -{ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT -kvm_pfn_t pfn =folio_file_pfn (folio, index); -gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; - -return kvm_arch_gmem_make_private (kvm, gfn, pfn,folio_nr_pages (folio)); -#else -return 0 ; -#endif -} - -/* -* Process @folio, which contains @gfn, so that the guest can use it. -* The folio must be locked and the gfn must be contained in @slot. -* On successful return the guest sees a zero page so as to avoid -* leaking host data and the up-to-date flag is set. -*/ -static int kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot, -gfn_t gfn,struct folio *folio) -{ -pgoff_t index; - -/* -* Preparing huge folios should always be safe, since it should -* be possible to split them later if needed. -* -* Right now the folio order is always going to be zero, but the -* code is ready for huge folios. The only assumption is that -* the base pgoff of memslots is naturally aligned with the -* requested page order, ensuring that huge folios can also use -* huge page table entries for GPA->HPA mapping. -* -* The order will be passed when creating the guest_memfd, and -* checked when creating memslots. -*/ -WARN_ON (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); - index =kvm_gmem_get_index (slot, gfn); - index =ALIGN_DOWN (index,folio_nr_pages (folio)); - -return __kvm_gmem_prepare_folio (kvm, slot, index, folio); -} -/* * Returns a locked folio on success. The caller is responsible for * setting the up-to-date flag before the memory is mapped into the guest.
@@ -799,7 +755,9 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, {pgoff_t index =kvm_gmem_get_index (slot, gfn);struct folio *folio; -int r =0 ; +int r =0 , __order; + + max_order = max_order ?: &__order;CLASS (gmem_get_file, file)(slot);if (!file)
@@ -814,8 +772,11 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT if (kvm_gmem_is_private_mem (file_inode (file), index)) - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); + r =kvm_arch_gmem_make_private (kvm, gfn, *pfn, + (kvm_pfn_t )1 << *max_order); +#endif folio_unlock (folio);
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Re: [PATCH v6 9/9] KVM: guest_memfd: Make private exactly what can be mapped on page fault
Quoted 15 lines · Sean Christopherson
[...snip...]
virt/kvm/guest_memfd.c +3
-1@@ -799,7 +755,9 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, {pgoff_t index =kvm_gmem_get_index (slot, gfn);struct folio *folio; -int r =0 ; +int r =0 , __order; + + max_order = max_order ?: &__order;CLASS (gmem_get_file, file)(slot);if (!file)
virt/kvm/guest_memfd.c int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, · line 778
Quoted 8 lines
virt/kvm/guest_memfd.c +3
-1@@ -814,8 +772,11 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT if (kvm_gmem_is_private_mem (file_inode (file), index)) - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); + r =kvm_arch_gmem_make_private (kvm, gfn, *pfn, + (kvm_pfn_t )1 << *max_order);
This is correct after __kvm_gmem_get_pfn() is called and errors are checked for, pfn and max_order must be valid. and max_order is always non-NULL after the statement with __order above.
Quoted 6 lines
virt/kvm/guest_memfd.c +1
-1+#endif folio_unlock (folio); --2.55.0.229.g6434b31f56-goog
Quoted 9 lines · Sean Christopherson
These are the cleanups from the SNP VMSA series. Well, that's how they started out, now it's full into prep work for in-place conversion. Oh well.
Rework prepare()+invalidate() into make_private()+reclaim(), with direct line of sight to adding
make_shared()for in-place conversion. Have the x86 backend use make_{private,shared} (noreclaim()) for SNP, as reclaim in SNP-land just means "make shared".[...]
Applied to kvm-x86 coco, thanks!
[1/9] KVM: guest_memfd: Pass the number of pages instead of the end pfn into .invalidate()
https://github.com/kvm-x86/linux/commit/0f0893ac5af3 [2/9] KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it https://github.com/kvm-x86/linux/commit/7b8529e70a11 [3/9] KVM: x86: Rename kvm_x86_ops' gmem_invalidate() to gmem_make_shared() https://github.com/kvm-x86/linux/commit/f3c073e25476 [4/9] KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare() failure https://github.com/kvm-x86/linux/commit/0f68fabe68fe [5/9] KVM: guest_memfd: Add helpers to query SHARED vs. PRIVATE for a given page https://github.com/kvm-x86/linux/commit/19e612cd59bf [6/9] KVM: guest_memfd: Only "prepare" folios for private pages https://github.com/kvm-x86/linux/commit/3df611b5a04e [7/9] KVM: guest_memfd: Rename prepare() hook and Kconfig to make_private() / CONVERT https://github.com/kvm-x86/linux/commit/2131c4f763d2 [8/9] KVM: guest_memfd: Explicitly pass number of pages to make_private() hook https://github.com/kvm-x86/linux/commit/fb50ca77b672 [9/9] KVM: guest_memfd: Make private exactly what can be mapped on page fault https://github.com/kvm-x86/linux/commit/2abcdf03fda1