KVM: guest_memfd: reclaim()/convert() cleanups
Subsystems (4)
Sean Christopherson
maintainerv5 cover KVM: guest_memfd: reclaim()/convert() cleanups
These are the cleanups from the SNP VMSA series. I kept it as v5 to maintain the paper trail.
Rework prepare()+invalidate() into convert()+reclaim(), and have the x86 backend use a single convert() flow for both (only SNP needs the hooks, and for SNP, reclaim just means "make shared"). The goals are to eliminate the misleading use of the term "invalidate", to drop a kvm_x86_ops callback, and to prepare for in-place conversion, e.g. so that KVM ends up with something like:
if (!to_private) kvm_gmem_make_shared(inode, start, end);
instead of a funky:
if (!to_private) kvm_gmem_invalidate(inode, start, end);
v5
- Drop the VMSA fixes (already applied).
- Keep a dedicated reclaim() flow. [Fuad, Ackerley]
- Pass a gfn, not a gpa, to convert() / reclaim(). [Ackerley]
- Collect reviews. [Ackerley]
- Align the gfn when folding __kvm_gmem_prepare_folio() into its caller. [Sashiko]
v4:
- https://lore.kernel.org/all/[email protected]
- Collect reviews. [Mike, Ackerley]
- Rework prepare()+invalidate() into a common convert() instead of simply renaming invalidate() to reclaim_memory(). [Ackerley]
- Fix several (mostly benign) issues in the "prepare" flow.
v3:
- https://lore.kernel.org/all/[email protected]
- Ensure disabling quirks while the VM is live won't result in KVM skipping the back-half of "zap all fast". [Sashiko]
- s/gmem_free_folio/gmem_reclaim_memory. [Ackerley]
- Use {READ,WRITE}_ONCE() when accessing the guest's VMSA GPA outside of the per-vCPU mutex, and comment. [Sashiko]
v2:
- https://lore.kernel.org/all/[email protected]
- Invalidate VMSAs if the memslot is DELETED or MOVED. [Sashiko]
- Limit stable@ patches without a Fixes to 6.12+.
v1: https://lore.kernel.org/all/[email protected]
Sean Christopherson (7):
KVM: guest_memfd: Plumb the number of pages and max order into
.invalidate()
KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and
isolate it
KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare()
failure
KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
KVM: guest_memfd: Explicitly pass number of pages to
kvm_arch_gmem_prepare()
KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into
.gmem_convert()
KVM: guest_memfd: Rework PREPARE config and hook into a more generic
CONVERT
arch/x86/include/asm/kvm-x86-ops.h | 5 ++-- arch/x86/include/asm/kvm_host.h | 6 ++-- arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/svm/sev.c | 23 +++++++++++++-- arch/x86/kvm/svm/svm.c | 3 +- arch/x86/kvm/svm/svm.h | 4 +-- arch/x86/kvm/x86.c | 18 +++++++----- include/linux/kvm_host.h | 10 +++++-- virt/kvm/Kconfig | 6 +++- virt/kvm/guest_memfd.c | 46 +++++++++++------------------- 10 files changed, 70 insertions(+), 54 deletions(-)
base-commit: 6bc96b971766fbbbbdd9fb2642cedacaf02da957
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Sean Christopherson
maintainerPATCH v5 1/7 KVM: guest_memfd: Plumb the number of pages and max order into .invalidate()
Plumb the number of pages to "invalidate", i.e. reclaim, along with the max order with which the pages can be mapped. This will allow x86 to combine its vendor-specific implementations for conversion and reclaim, a.k.a. preparation and invalidation, into a single hook.
No functional change intended.
--- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 7 ++----- 3 files changed, 5 insertions(+), 8 deletions(-)
arch/x86/kvm/x86.c +2 -2
@@ -10627,9 +10627,9 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -kvm_x86_call (gmem_invalidate)(start, end); +kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); }void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {
include/linux/kvm_host.h +1 -1
@@ -2614,7 +2614,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src,#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int order);void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif
virt/kvm/guest_memfd.c +2 -5
@@ -530,11 +530,8 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE static void kvm_gmem_free_folio (struct folio *folio) { -struct page *page =folio_page (folio,0 ); -kvm_pfn_t pfn =page_to_pfn (page); -int order =folio_order (folio); - -kvm_arch_gmem_invalidate (pfn, pfn + (1ul << order)); +kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio), +folio_order (folio)); }#endif
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 1/7] KVM: guest_memfd: Plumb the number of pages and max order into .invalidate()
virt/kvm/guest_memfd.c static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol · line 533
Quoted 12 lines · Sean Christopherson
[...snip...]
#ifdef
CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATEstatic voidkvm_gmem_free_folio(struct folio *folio){virt/kvm/guest_memfd.c +1
-5-struct page *page =folio_page (folio,0 ); -kvm_pfn_t pfn =page_to_pfn (page); -int order =folio_order (folio); - -kvm_arch_gmem_invalidate (pfn, pfn + (1ul << order)); + kvm_arch_gmem_invalidate(folio_file_pfn(folio, 0), folio_nr_pages(folio),
How about just folio_pfn(folio)? Since we're freeing the folio this will always be 0.
Quoted 4 lines
virt/kvm/guest_memfd.c +1
-0+ folio_order(folio)); }#endif
Re: [PATCH v5 1/7] KVM: guest_memfd: Plumb the number of pages and max order into .invalidate()
In reply to Sean ChristophersonQuoted 36 lines · Sean Christopherson
Plumb the number of pages to "invalidate", i.e. reclaim, along with the max order with which the pages can be mapped. This will allow x86 to combine its vendor-specific implementations for conversion and reclaim, a.k.a. preparation and invalidation, into a single hook.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 7 ++----- 3 files changed, 5 insertions(+), 8 deletions(-)arch/x86/kvm/x86.c +2
-2@@ -10627,9 +10627,9 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -kvm_x86_call (gmem_invalidate)(start, end); +kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); }void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {include/linux/kvm_host.h +0
-0
include/linux/kvm_host.h long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, · line 2617
Quoted 5 lines
include/linux/kvm_host.h +1
-1@@ -2614,7 +2614,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src,#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int order);
nit: the declaration names the last parameter "order" here, but the definition in arch/x86/kvm/x86.c calls it "max_order".
Cheers,
/fuad
Quoted 24 lines
include/linux/kvm_host.h +0
-0void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif virt/kvm/guest_memfd.c +2
-6@@ -530,11 +530,8 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *fol#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE static void kvm_gmem_free_folio (struct folio *folio) { -struct page *page =folio_page (folio,0 ); -kvm_pfn_t pfn =page_to_pfn (page); -int order =folio_order (folio); - -kvm_arch_gmem_invalidate (pfn, pfn + (1ul << order)); +kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio), +folio_order (folio)); }#endif --2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 1/7] KVM: guest_memfd: Plumb the number of pages and max order into .invalidate()
In reply to Sean ChristophersonQuoted 19 lines · Sean Christopherson
Plumb the number of pages to "invalidate", i.e. reclaim, along with the max order with which the pages can be mapped. This will allow x86 to combine its vendor-specific implementations for conversion and reclaim, a.k.a. preparation and invalidation, into a single hook.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 7 ++----- 3 files changed, 5 insertions(+), 8 deletions(-)arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord · line 10630
Quoted 5 lines
arch/x86/kvm/x86.c +1
-1@@ -10627,9 +10627,9 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order)
why use 'kvm_pfn_t' as the type for nr_pages instead of 'unsigned long'?
(I guess just because you kept the original type for @end)
Sean Christopherson
maintainerRe: [PATCH v5 1/7] KVM: guest_memfd: Plumb the number of pages and max order into .invalidate()
Quoted 27 lines · Xiaoyao Li
Quoted 19 lines · Sean Christopherson
Plumb the number of pages to "invalidate", i.e. reclaim, along with the max order with which the pages can be mapped. This will allow x86 to combine its vendor-specific implementations for conversion and reclaim, a.k.a. preparation and invalidation, into a single hook.
No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 +- virt/kvm/guest_memfd.c | 7 ++----- 3 files changed, 5 insertions(+), 8 deletions(-)arch/x86/kvm/x86.c +0
-0arch/x86/kvm/x86.c int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord · line 10629
Quoted 4 lines
arch/x86/kvm/x86.c +1
-1@@ -10627,9 +10627,9 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order)why use '
kvm_pfn_t' as the type fornr_pagesinstead of 'unsigned long'?
Quoted 1 line
(I guess just because you kept the original type for @end)
Because the number of pages, i.e. number of PFNs, need to be in the same "domain" as the staring PFN (and KVM's MMU needs to treat PFNs as u64s).
Sean Christopherson
maintainerPATCH v5 2/7 KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it
In reply to Sean ChristophersonRename guest_memfd's invalidate() hook to reclaim() and isolate it via its own RECLAIM Kconfig, as the hook is called when a folio is freed, which is far too late and lacks sufficient information for KVM to actually invalidate its usage of the memory. E.g. SNP uses the hook to convert memory back to SHARED so that it can be safely accessed by the host, there is no invalidation of guest mappings anywhere. Isolating the hook will also allow pKVM on arm64 to opt-in to reclaim() without also having to differentiate between reclaim and conversions to shared for active VMs.
Keep guest_memfd's trampoline, even though it would be trivial to wire up .free_folio() directly to an arch callback, to avoid bleeding guest_memfd internals into arch code (specifically, avoid referencing folios in arch code).
Leave the kvm_x86_ops hook as-is for the moment, as the hook can be dropped entirely ("reclaim" on SNP is the same as convert-to-shared).
--- arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/x86.c | 7 +++++-- include/linux/kvm_host.h | 5 ++++- virt/kvm/Kconfig | 4 ++++ virt/kvm/guest_memfd.c | 8 ++++---- 5 files changed, 18 insertions(+), 7 deletions(-)
arch/x86/kvm/Kconfig +1 -0
@@ -161,6 +161,7 @@ config KVM_AMD_SEV select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATE help
arch/x86/kvm/x86.c +5 -2
@@ -10626,11 +10626,14 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord }#endif -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) {kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); } +#endif + +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {kvm_x86_call (gmem_invalidate_range)(kvm, range);
include/linux/kvm_host.h +4 -1
@@ -2613,8 +2613,11 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, kvm_gmem_populate_cb post_populate, void *opaque);#endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int order); +#endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int order);void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif
virt/kvm/Kconfig +4 -0
@@ -111,6 +111,10 @@ config HAVE_KVM_ARCH_GMEM_PREPARE bool depends on KVM_GUEST_MEMFD +config HAVE_KVM_ARCH_GMEM_RECLAIM + bool + depends on KVM_GUEST_MEMFD + config HAVE_KVM_ARCH_GMEM_INVALIDATE bool depends on KVM_GUEST_MEMFD
virt/kvm/guest_memfd.c +4 -4
@@ -527,11 +527,11 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *folreturn MF_DELAYED ; } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM static void kvm_gmem_free_folio (struct folio *folio) { -kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio), -folio_order (folio)); +kvm_arch_gmem_reclaim (folio_file_pfn (folio,0 ),folio_nr_pages (folio), +folio_order (folio)); }#endif
@@ -539,7 +539,7 @@ static const struct address_space_operations kvm_gmem_aops = { .dirty_folio = noop_dirty_folio, .migrate_folio = kvm_gmem_migrate_folio, .error_remove_folio = kvm_gmem_error_folio, -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM .free_folio = kvm_gmem_free_folio,#endif };
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 2/7] KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it
Quoted 14 lines · Sean Christopherson
Rename guest_memfd's
invalidate()hook toreclaim()and isolate it via its own RECLAIMKconfig, as the hook is called when a folio is freed, which is far too late and lacks sufficient information for KVM to actually invalidate its usage of the memory. E.g. SNP uses the hook to convert memory back to SHARED so that it can be safely accessed by the host, there is no invalidation of guest mappings anywhere. Isolating the hook will also allow pKVM on arm64 to opt-in toreclaim()without also having to differentiate between reclaim and conversions to shared for active VMs.Keep guest_memfd's trampoline, even though it would be trivial to wire up .free_folio() directly to an arch callback, to avoid bleeding
guest_memfdinternals into arch code (specifically, avoid referencing folios in arch code).
Quoted 6 lines
Leave the
kvm_x86_opshook as-is for the moment, as the hook can be dropped entirely ("reclaim" on SNP is the same as convert-to-shared).[...snip...]
Re: [PATCH v5 2/7] KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it
In reply to Sean ChristophersonQuoted 15 lines · Sean Christopherson
Rename guest_memfd's
invalidate()hook toreclaim()and isolate it via its own RECLAIMKconfig, as the hook is called when a folio is freed, which is far too late and lacks sufficient information for KVM to actually invalidate its usage of the memory. E.g. SNP uses the hook to convert memory back to SHARED so that it can be safely accessed by the host, there is no invalidation of guest mappings anywhere. Isolating the hook will also allow pKVM on arm64 to opt-in toreclaim()without also having to differentiate between reclaim and conversions to shared for active VMs.Keep guest_memfd's trampoline, even though it would be trivial to wire up .free_folio() directly to an arch callback, to avoid bleeding
guest_memfdinternals into arch code (specifically, avoid referencing folios in arch code).
Quoted 6 lines
Leave the
kvm_x86_opshook as-is for the moment, as the hook can be dropped entirely ("reclaim" on SNP is the same as convert-to-shared).Cc Fuad Tabba <[email protected]>Cc Ackerley Tng <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Thanks Sean, this matches what pKVM needs: we'll select HAVE_KVM_ARCH_GMEM_RECLAIM and implement kvm_arch_gmem_reclaim() only, without the convert hook, so teardown (which for pKVM must scrub the page before it returns to the host) stays distinct from a private->shared conversion (which must preserve it).
Same nit as patch 1 (order/max_order).
Cheers,
/fuad
Quoted 104 lines
--- arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/x86.c | 7 +++++-- include/linux/kvm_host.h | 5 ++++- virt/kvm/Kconfig | 4 ++++ virt/kvm/guest_memfd.c | 8 ++++---- 5 files changed, 18 insertions(+), 7 deletions(-)arch/x86/kvm/Kconfig +1
-0@@ -161,6 +161,7 @@ config KVM_AMD_SEV select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATE helparch/x86/kvm/x86.c +5
-2@@ -10626,11 +10626,14 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord }#endif -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) {kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); } +#endif + +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {kvm_x86_call (gmem_invalidate_range)(kvm, range);include/linux/kvm_host.h +4
-1@@ -2613,8 +2613,11 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, kvm_gmem_populate_cb post_populate, void *opaque);#endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int order); +#endif +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int order);void kvm_arch_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);#endif virt/kvm/Kconfig +4
-0@@ -111,6 +111,10 @@ config HAVE_KVM_ARCH_GMEM_PREPARE bool depends on KVM_GUEST_MEMFD +config HAVE_KVM_ARCH_GMEM_RECLAIM + bool + depends on KVM_GUEST_MEMFD + config HAVE_KVM_ARCH_GMEM_INVALIDATE bool depends on KVM_GUEST_MEMFDvirt/kvm/guest_memfd.c +4
-5@@ -527,11 +527,11 @@ static int kvm_gmem_error_folio(struct address_space *mapping, struct folio *folreturn MF_DELAYED ; } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM static void kvm_gmem_free_folio (struct folio *folio) { -kvm_arch_gmem_invalidate (folio_file_pfn (folio,0 ),folio_nr_pages (folio), -folio_order (folio)); +kvm_arch_gmem_reclaim (folio_file_pfn (folio,0 ),folio_nr_pages (folio), +folio_order (folio)); }#endif @@ -539,7 +539,7 @@ static const struct address_space_operations kvm_gmem_aops = { .dirty_folio = noop_dirty_folio, .migrate_folio = kvm_gmem_migrate_folio, .error_remove_folio = kvm_gmem_error_folio, -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM .free_folio = kvm_gmem_free_folio,#endif }; --2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 2/7] KVM: guest_memfd: Rename invalidate() arch hook to reclaim() and isolate it
In reply to Sean ChristophersonQuoted 45 lines · Sean Christopherson
Rename guest_memfd's
invalidate()hook toreclaim()and isolate it via its own RECLAIMKconfig, as the hook is called when a folio is freed, which is far too late and lacks sufficient information for KVM to actually invalidate its usage of the memory. E.g. SNP uses the hook to convert memory back to SHARED so that it can be safely accessed by the host, there is no invalidation of guest mappings anywhere. Isolating the hook will also allow pKVM on arm64 to opt-in toreclaim()without also having to differentiate between reclaim and conversions to shared for active VMs.Keep guest_memfd's trampoline, even though it would be trivial to wire up .free_folio() directly to an arch callback, to avoid bleeding
guest_memfdinternals into arch code (specifically, avoid referencing folios in arch code).Leave the
kvm_x86_opshook as-is for the moment, as the hook can be dropped entirely ("reclaim" on SNP is the same as convert-to-shared).Cc Fuad Tabba <[email protected]>Cc Ackerley Tng <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/Kconfig | 1 + arch/x86/kvm/x86.c | 7 +++++-- include/linux/kvm_host.h | 5 ++++- virt/kvm/Kconfig | 4 ++++ virt/kvm/guest_memfd.c | 8 ++++---- 5 files changed, 18 insertions(+), 7 deletions(-)arch/x86/kvm/Kconfig +1
-0@@ -161,6 +161,7 @@ config KVM_AMD_SEV select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATE helparch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord · line 10630
Quoted 7 lines
arch/x86/kvm/x86.c +2
-2@@ -10626,11 +10626,14 @@ int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_ord }#endif -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE -void kvm_arch_gmem_invalidate (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM +void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order)
The kvm_pfn_t for nr_pages still catches my eye, but it's not the business of this patch.
Sean Christopherson
maintainerPATCH v5 3/7 KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare() failure
In reply to Sean ChristophersonDrop guest_memfd's ratelimited printk to log "preparation" failures, as KVM SNP alreads logs more precise messages in all error paths, and whether or not failure to convert the pfn to private is "unexpected", i.e. warrants logging, is firmly an architecture specific detail.
--- virt/kvm/guest_memfd.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
virt/kvm/guest_memfd.c +3 -7
@@ -66,15 +66,11 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slo#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -int rc =kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); -if (rc) { -pr_warn_ratelimited ("gmem: Failed to prepare folio for index %lx GFN %llx PFN %llx error %d.\n" , - index, gfn, pfn, rc); -return rc; - } -#endif +return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +#else return 0 ; +#endif }/*
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 3/7] KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare() failure
Quoted 3 lines · Sean Christopherson
Drop guest_memfd's ratelimited printk to log "preparation" failures, as KVM SNP alreads logs more precise messages in all error paths, and whether or
nit: already
Cheers,
/fuad
Quoted 35 lines
not failure to convert the pfn to private is "unexpected", i.e. warrants logging, is firmly an architecture specific detail.
Reviewed-by Ackerley Tng <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- virt/kvm/guest_memfd.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)virt/kvm/guest_memfd.c +3
-8@@ -66,15 +66,11 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slo#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -int rc =kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); -if (rc) { -pr_warn_ratelimited ("gmem: Failed to prepare folio for index %lx GFN %llx PFN %llx error %d.\n" , - index, gfn, pfn, rc); -return rc; - } -#endif +return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +#else return 0 ; +#endif }/* -- 2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 3/7] KVM: guest_memfd: Drop the redundant printk on arch gmem_prepare() failure
In reply to Sean ChristophersonQuoted 7 lines · Sean Christopherson
Drop guest_memfd's ratelimited printk to log "preparation" failures, as KVM SNP alreads logs more precise messages in all error paths, and whether or not failure to convert the pfn to private is "unexpected", i.e. warrants logging, is firmly an architecture specific detail.
Reviewed-by Ackerley Tng <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 27 lines
--- virt/kvm/guest_memfd.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)virt/kvm/guest_memfd.c +3
-7@@ -66,15 +66,11 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slo#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE kvm_pfn_t pfn =folio_file_pfn (folio, index);gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; -int rc =kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); -if (rc) { -pr_warn_ratelimited ("gmem: Failed to prepare folio for index %lx GFN %llx PFN %llx error %d.\n" , - index, gfn, pfn, rc); -return rc; - } -#endif +return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); +#else return 0 ; +#endif }/*
Sean Christopherson
maintainerPATCH v5 4/7 KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
In reply to Sean ChristophersonFold __kvm_gmem_prepare_folio() into kvm_gmem_prepare_folio() and extend the #ifdef guard to cover all of the kvm_gmem_prepare_folio(). Take care to align the gfn _before_ computing the index, otherwise KVM would end up with a mismatched gfn+pfn pair for folios larger than order-0 (this detail was subtly handled by recomputing gfn in the helper using the aligned pfn).
Providing a single-use, ~2 line tail-call helper adds no value. In fact, due to having to reconstitute the gfn, *and* effectively hiding the alignment adjustment for hugepages, the separate helper is very arguably a net negative.
Opportunistically convert the existing WARN in the prepare flow to a only fire once, e.g. so that a guest_memfd bug doesn't unintentionally Dos the kernel by spamming the log.
No functional change intended.
--- virt/kvm/guest_memfd.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-)
virt/kvm/guest_memfd.c +8 -16
@@ -60,19 +60,6 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)return gfn - slot->base_gfn + slot->gmem.pgoff; } -static int __kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot, -pgoff_t index,struct folio *folio) -{ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -kvm_pfn_t pfn =folio_file_pfn (folio, index); -gfn_t gfn = slot->base_gfn + index - slot->gmem.pgoff; - -return kvm_arch_gmem_prepare (kvm, gfn, pfn,folio_order (folio)); -#else -return 0 ; -#endif -} -/* * Process @folio, which contains @gfn, so that the guest can use it. * The folio must be locked and the gfn must be contained in @slot.
@@ -82,6 +69,7 @@ static int __kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slostatic int kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot,gfn_t gfn,struct folio *folio) { +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE pgoff_t index;/*
@@ -97,11 +85,15 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,* The order will be passed when creating the guest_memfd, and * checked when creating memslots. */ -WARN_ON (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); +WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); + gfn =ALIGN_DOWN (gfn,folio_nr_pages (folio)); index =kvm_gmem_get_index (slot, gfn); - index =ALIGN_DOWN (index,folio_nr_pages (folio)); -return __kvm_gmem_prepare_folio (kvm, slot, index, folio); +return kvm_arch_gmem_prepare (kvm, gfn,folio_file_pfn (folio, index), +folio_order (folio)); +#else +return 0 ; +#endif }/*
Signature · 1 line
2.55.0.141.g00534a21ce-goog
[email protected]
AI reviewRe: [PATCH v5 4/7] KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [Critical] Refactoring kvm_gmem_prepare_folio() and kvm_gmem_make_private() to align gfn before computing index produces a mismatched and unaligned pfn if the memslot's base_gfn is not aligned to the huge folio size. This causes the architecture-specific prepare function to write beyond the bounds of the folio, leading to host physical memory corruption.
Pre-existing issues:
- [High] In SEV-SNP, if large RMP entry preparation fails for a huge folio, KVM will only prepare the base 4K page, causing an incomplete initialization and an infinite page fault loop for subpage accesses. --
--- Patch [4]: [PATCH v5 4/7] KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller --- commit ef3a2643968d189a00f8b26552b9b814bf1aa548
Author: Sean Christopherson <[email protected]>
KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
This commit folds __kvm_gmem_prepare_folio() into its caller, removing the single-use tail-call helper. It also adjusts the alignment logic to align the gfn before computing the index, and converts a WARN to WARN_ON_ONCE.
Quoted 4 lines
virt/kvm/guest_memfd.c +0
-0
[ ... ]
virt/kvm/guest_memfd.c static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, · line 93
Quoted 13 lines
virt/kvm/guest_memfd.c +4
-3@@ -97,11 +85,15 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,* The order will be passed when creating the guest_memfd, and * checked when creating memslots. */ -WARN_ON (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); +WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); + gfn =ALIGN_DOWN (gfn,folio_nr_pages (folio)); index =kvm_gmem_get_index (slot, gfn); - index =ALIGN_DOWN (index,folio_nr_pages (folio)); -return __kvm_gmem_prepare_folio (kvm, slot, index, folio); +return kvm_arch_gmem_prepare (kvm, gfn,folio_file_pfn (folio, index), +folio_order (folio));
[Severity: Critical]
Does aligning gfn before computing index cause an unaligned index when slot->base_gfn is not huge-page aligned?
If a userspace process maps a guest_memfd using KVM_SET_USER_MEMORY_REGION2 with a base_gfn that is not aligned to the huge page size, kvm_gmem_get_index() could return an unaligned index.
Because folio_file_pfn() applies the index offset to the base PFN:
virt/kvm/guest_memfd.c:folio_file_pfn() {
return folio_pfn(folio) + (index & (folio_nr_pages(folio) - 1));
}an unaligned index causes it to return an offset PFN instead of the folio's base PFN.
Could this lead to host physical memory corruption? The architecture backend processes the full page count starting from the offset PFN, which would write beyond the allocated folio boundaries and overwrite unrelated host physical memory.
It looks like this same alignment pattern is also present in kvm_gmem_make_private().
[Severity: High]
This isn't a bug introduced by this patch, but could this generic function incorrectly assume a successful return means the entire huge folio was prepared?
Looking at sev_gmem_prepare() in arch/x86/kvm/svm/sev.c, if it falls back to PG_LEVEL_4K for a huge folio:
} else {
level = PG_LEVEL_4K;
pfn_aligned = pfn;
gfn_aligned = gfn;
}
rc = rmp_make_private(pfn_aligned, gfn_to_gpa(gfn_aligned), level, sev->asid, false);It appears rmp_make_private() is called only once for the base 4K page, without looping over the remaining pages in the huge folio.
Since kvm_arch_gmem_prepare() then returns success to this generic code, are the remaining 511 pages left uninitialized, potentially causing an infinite page fault loop for subsequent subpage accesses?
Quoted 4 lines
virt/kvm/guest_memfd.c +3
-0+#else +return 0 ; +#endif }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH v5 4/7] KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
In reply to Sean ChristophersonQuoted 5 lines · Sean Christopherson
[...snip...]
virt/kvm/guest_memfd.c +0
-0@@ -97,11 +85,15 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,* The order will be passed when creating the guest_memfd, and
virt/kvm/guest_memfd.c static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, · line 92
Quoted 10 lines
virt/kvm/guest_memfd.c +3
-3* checked when creating memslots. */ -WARN_ON (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); +WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); + gfn =ALIGN_DOWN (gfn,folio_nr_pages (folio)); index =kvm_gmem_get_index (slot, gfn); - index =ALIGN_DOWN (index,folio_nr_pages (folio)); -return __kvm_gmem_prepare_folio (kvm, slot, index, folio); + return kvm_arch_gmem_prepare(kvm, gfn, folio_file_pfn(folio, index),
Could this just be folio_pfn(folio) since this function is kvm_gmem_prepare_folio() and guest_memfd will always try to prepare the entire folio?
Quoted 5 lines
virt/kvm/guest_memfd.c +4
-0+ folio_order(folio)); +#else +return 0 ; +#endif }
Sean Christopherson
maintainerRe: [PATCH v5 4/7] KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
Quoted 15 lines · Ackerley Tng
Quoted 13 lines · Sean Christopherson
[...snip...]
virt/kvm/guest_memfd.c +2
-2@@ -97,11 +85,15 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,* The order will be passed when creating the guest_memfd, and * checked when creating memslots. */ -WARN_ON (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); +WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); + gfn =ALIGN_DOWN (gfn,folio_nr_pages (folio)); index =kvm_gmem_get_index (slot, gfn); - index =ALIGN_DOWN (index,folio_nr_pages (folio));
Quoted 6 lines
virt/kvm/guest_memfd.c static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, · line 92
Quoted 2 lines
virt/kvm/guest_memfd.c +1
-1-return __kvm_gmem_prepare_folio (kvm, slot, index, folio); + return kvm_arch_gmem_prepare(kvm, gfn, folio_file_pfn(folio, index),Could this just be
folio_pfn(folio)since this function iskvm_gmem_prepare_folio()andguest_memfdwill always try to prepare the entire folio?
No? Maybe? For this patch, I'm just trying to shuffle code araound. Even for this series, I'd prefer not to make any more semantic changes than are needed to get to a sane state, and to prepare for in-place conversion. If there's a need and/or a good reason to use folio_pfn(), by all means, send a patch.
Sean Christopherson
maintainerRe: [PATCH v5 4/7] KVM: guest_memfd: Fold __kvm_gmem_prepare_folio() into its sole caller
+Mike
Quoted 23 lines · Sean Christopherson
Quoted 15 lines · Ackerley Tng
Quoted 13 lines · Sean Christopherson
[...snip...]
virt/kvm/guest_memfd.c +2
-2@@ -97,11 +85,15 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,* The order will be passed when creating the guest_memfd, and * checked when creating memslots. */ -WARN_ON (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); +WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); + gfn =ALIGN_DOWN (gfn,folio_nr_pages (folio)); index =kvm_gmem_get_index (slot, gfn); - index =ALIGN_DOWN (index,folio_nr_pages (folio));Quoted 6 lines
virt/kvm/guest_memfd.c static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, · line 92
Quoted 2 lines
virt/kvm/guest_memfd.c +1
-1-return __kvm_gmem_prepare_folio (kvm, slot, index, folio); + return kvm_arch_gmem_prepare(kvm, gfn, folio_file_pfn(folio, index),Could this just be
folio_pfn(folio)since this function iskvm_gmem_prepare_folio()andguest_memfdwill always try to prepare the entire folio?
Quoted 4 lines
No? Maybe? For this patch, I'm just trying to shuffle code araound. Even for this series, I'd prefer not to make any more semantic changes than are needed to get to a sane state, and to prepare for in-place conversion. If there's a need and/or a good reason to use
folio_pfn(), by all means, send a patch.
I take that back. After far too much staring and digging into the history (and future) of this code, I finally see what you were getting at. Because KVM "prepares" entire folios, grabbing a specific page in the folio is nonsensical, just grab the base pfn and be done with it.
However, I'm not convinced to committing harder to per-folio "preparation" is the way to go. The pre-folio prepation was added when guest_memfd actually tracked preparedness. At the time, it made sense, because tracking per-folio meant we didn't need to add a separate data structure to track that information, and doing per-folio tracking only works if the entire folio is prepared (or not).
Now that we dropped that code in favor having SNP query the RMP, per-folio preparation, i.e. per-folio conversions to private, doesn't make as much sense. There is still technically an argument for per-folio conversion, *if* what I described below is even allowed by SNP.
If SNP allows the RMP size to be greater than the NPT size, then per-folio conversion would allow KVM to assign a hugepage in the RMP even if it can only be mapped into the NPT with a smaller page, e.g. because of memslot alignment. The documentation of that reasoning would be something like this:
/*
* If the memory is private from KVM's perspective, and hardware tracks
* VM-assigned private memory in a dedicated data structure, i.e. not
* in the stage-2 page tables, then call into arch code to assign the
* entire folio to the guest. Assigning the entire folio, e.g. instead
* of only the memory being mapped into the guest, allows KVM to assign
* an entire hugepage of memory in the out-of-band structure even if
* KVM can only map a smaller page size into the MMU, e.g. because the
* gmem hugepage is spread across multiple memslots.
*/
if (kvm_gmem_is_private_mem(file_inode(file), index))
r = kvm_gmem_make_private(kvm, slot, gfn, folio);
But it's not clear to me that SNP even supports mismatched page sizes, because several of the flows in the APM explicitly state that triggers an #NPF, and this comment in KVM's sev_handle_rmp_fault() also suggests that's not allow.
* 2) Guest access via NPT can trigger an #NPF if the NPT mapping is
* smaller than what is indicated by the 2MB RMP entry for the PFN
* that backs the GPA.
And even *if* that's allowed by SNP, I'm not at all convinced it's worth doing in KVM, because it should be a rare situation. E.g. maybe for memory at the top of TOLUD that has holes for non-RAM crud or something?
So instead of using folio_pfn(), I'm leaning strongly towards entirely dropping what is currently kvm_gmem_prepare_folio(), and instead converting exactly what is being mapped into the guest. Because I've messed up the gfn and index handling twice now, I'm still struggling to wrap my head around the math, *and* I really don't want to have two separate flows computing "how much memory can be assigned and at what size" once hugepage support comes along.
I.e. do this:
pgoff_t index = kvm_gmem_get_index(slot, gfn);
struct folio *folio;
int r = 0, __order;
max_order = max_order ?: &__order;
CLASS(gmem_get_file, file)(slot);
if (!file)
return -EFAULT;
folio = __kvm_gmem_get_pfn(file, slot, index, pfn, max_order);
if (IS_ERR(folio))
return PTR_ERR(folio);
if (!folio_test_uptodate(folio)) {
clear_highpage(folio_page(folio, 0));
folio_mark_uptodate(folio);
}
if (kvm_gmem_is_private_mem(file_inode(file), index))
r = kvm_arch_gmem_make_private(kvm, gfn, *pfn, *max_order);Am I missing something? Anyone feel strongly about keeping per-folio converions for when SNP supports hugepages?
Sean Christopherson
maintainerPATCH v5 5/7 KVM: guest_memfd: Explicitly pass number of pages to kvm_arch_gmem_prepare()
In reply to Sean ChristophersonTweak the guest_memfd prepare() hook to explicitly pass the number of pages to "prepare", so that the prepare() and invalidate() hooks can be combined into a single convert() hook, and because the existing code is outright broken if a guest_memfd folio is comprised of more than one page (which can't happen, yet). guest_memfd aligns the pfn down, and the SNP code *tries* to create a corresponding huge entry, but if the RMP must use 4KiB entries for whatever reason, KVM will only "prepare" the first pfn, and not the actual pfn that needs to be converted/prepared.
Alternatively, @max_order could simply be repurposed as _the_ @order, but that will fall apart when in-place conversion comes along, at which point KVM will need to deal with conversions that aren't strictly bound to a one folio.
WARN in the SNP code if the number of pages to prepare is anything other than '1', i.e. if guest_memfd is trying to prepare/convert more than a single 4KiB page, as sev_gmem_prepare() doesn't actually handle conversion greater than order-0 folios.
Opportunistically swap the ordering of @pfn and @gfn params for kvm_x86_ops.gmem_prepare() to match that of kvm_arch_gmem_prepare().
--- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/svm/sev.c | 6 +++++- arch/x86/kvm/svm/svm.h | 3 ++- arch/x86/kvm/x86.c | 5 +++-- include/linux/kvm_host.h | 3 ++- virt/kvm/guest_memfd.c | 7 ++++--- 6 files changed, 18 insertions(+), 9 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -1
@@ -1725,7 +1725,8 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int (*gmem_prepare)(struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int (*gmem_prepare)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order);#endif void (*gmem_invalidate)(kvm_pfn_t start,kvm_pfn_t end);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE
arch/x86/kvm/svm/sev.c +5 -1
@@ -5114,7 +5114,8 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)return false; } -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order) +int sev_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;
@@ -5125,6 +5126,9 @@ int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order)if (!sev_snp_guest (kvm))return 0 ; +if (WARN_ON_ONCE (nr_pages !=1 )) +return -EIO ; + rc =snp_lookup_rmpentry (pfn, &assigned, &level);if (rc) { pr_err_ratelimited("SEV: Failed to look up RMP entry: GFN %llx PFN %llx error %d\n",
arch/x86/kvm/svm/svm.h +2 -1
@@ -1009,7 +1009,8 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_prepare (struct kvm *kvm,kvm_pfn_t pfn,gfn_t gfn,int max_order); +int sev_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order);void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);
arch/x86/kvm/x86.c +3 -2
@@ -10620,9 +10620,10 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm) }#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order) +int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order) { -return kvm_x86_call (gmem_prepare)(kvm, pfn, gfn, max_order); +return kvm_x86_call (gmem_prepare)(kvm, gfn, pfn, nr_pages, max_order); }#endif
include/linux/kvm_host.h +2 -1
@@ -2580,7 +2580,8 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm,#endif /* CONFIG_KVM_GUEST_MEMFD */ #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,int max_order); +int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE
virt/kvm/guest_memfd.c +4 -3
@@ -70,6 +70,7 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, struct folio *folio) { #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE + unsigned long nr_pages = folio_nr_pages(folio);pgoff_t index;/*
@@ -85,12 +86,12 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,* The order will be passed when creating the guest_memfd, and * checked when creating memslots. */ -WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff,folio_nr_pages (folio))); - gfn =ALIGN_DOWN (gfn,folio_nr_pages (folio)); +WARN_ON_ONCE (!IS_ALIGNED (slot->gmem.pgoff, nr_pages)); + gfn =ALIGN_DOWN (gfn, nr_pages); index =kvm_gmem_get_index (slot, gfn);return kvm_arch_gmem_prepare (kvm, gfn,folio_file_pfn (folio, index), -folio_order (folio)); + nr_pages,folio_order (folio));#else return 0 ;#endif
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Sean Christopherson
maintainerPATCH v5 6/7 KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
In reply to Sean ChristophersonSmush x86's prepare() and invalidate() hooks into a common convert() flow, as they are effectively two sides of the same coin for SNP: they're invoked when private/shared memory is about to made accessible/visible to the guest or host. I.e. prepare() is really "make private", and invalidate() is really "make shared".
Using a single hook will yield more intuitive code when in-place conversion comes along.
For all intents and purposes, no functional change intended.
--- arch/x86/include/asm/kvm-x86-ops.h | 5 ++--- arch/x86/include/asm/kvm_host.h | 7 +++---- arch/x86/kvm/svm/sev.c | 21 ++++++++++++++++++--- arch/x86/kvm/svm/svm.c | 3 +-- arch/x86/kvm/svm/svm.h | 5 ++--- arch/x86/kvm/x86.c | 4 ++-- 6 files changed, 28 insertions(+), 17 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +2 -3
@@ -146,10 +146,9 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +KVM_X86_OP_OPTIONAL_RET0(gmem_convert)#endif -KVM_X86_OP_OPTIONAL(gmem_invalidate)#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE KVM_X86_OP_OPTIONAL(gmem_invalidate_range)#endif
arch/x86/include/asm/kvm_host.h +3 -4
@@ -1724,11 +1724,10 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int (*gmem_prepare)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order); +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif -void (*gmem_invalidate)(kvm_pfn_t start,kvm_pfn_t end);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif
arch/x86/kvm/svm/sev.c +18 -3
@@ -5114,8 +5114,8 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)return false; } -int sev_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order) +static int sev_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;
@@ -5165,8 +5165,9 @@ int sev_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +static void sev_gmem_make_shared (kvm_pfn_t start,kvm_pfn_t nr_pages) { +kvm_pfn_t end = start + nr_pages;kvm_pfn_t pfn;if (!cc_platform_has (CC_ATTR_HOST_SEV_SNP ))
@@ -5228,6 +5229,20 @@ void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end) } } +int sev_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int order,bool to_private) +{ +if (to_private) { +if (WARN_ON_ONCE (!kvm || gfn >> (kvm_host.maxphyaddr -PAGE_SHIFT ))) +return -EIO ; + +return sev_gmem_make_private (kvm, gfn, pfn, nr_pages, order); + } + +sev_gmem_make_shared (pfn, nr_pages); +return 0 ; +} +void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {struct kvm_vcpu *vcpu;
arch/x86/kvm/svm/svm.c +1 -2
@@ -5438,8 +5438,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_copy_enc_context_from = sev_vm_copy_enc_context_from, .vm_move_enc_context_from = sev_vm_move_enc_context_from, - .gmem_prepare = sev_gmem_prepare, - .gmem_invalidate = sev_gmem_invalidate, + .gmem_convert = sev_gmem_convert, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,#endif
arch/x86/kvm/svm/svm.h +2 -3
@@ -1009,9 +1009,8 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order); -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +int sev_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +2 -2
@@ -10623,14 +10623,14 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -return kvm_x86_call (gmem_prepare)(kvm, gfn, pfn, nr_pages, max_order); +return kvm_x86_call (gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true); }#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); +WARN_ON_ONCE (kvm_x86_call (gmem_convert)(NULL ,-1ull , pfn, nr_pages, max_order, false)); }#endif
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
arch/x86/kvm/svm/sev.c void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end) · line 5236
Quoted 5 lines · Sean Christopherson
arch/x86/kvm/svm/sev.c +5
-0+int sev_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int order,bool to_private) +{ +if (to_private) { +if (WARN_ON_ONCE (!kvm || gfn >> (kvm_host.maxphyaddr -PAGE_SHIFT )))
Omg, maxphyaddr is actually the number of bits in an address and not an address!
Thanks for explaining on v4.
Quoted 8 lines
arch/x86/kvm/svm/sev.c +8
-0+return -EIO ; + +return sev_gmem_make_private (kvm, gfn, pfn, nr_pages, order); + } + +sev_gmem_make_shared (pfn, nr_pages); +return 0 ; +}
arch/x86/kvm/svm/sev.c void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end)
Quoted 10 lines
arch/x86/kvm/svm/sev.c +2
-1+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); +WARN_ON_ONCE (kvm_x86_call (gmem_convert)(NULL ,-1ull , pfn, nr_pages, max_order, false));
This setup means later pKVM would have to define a .gmem_convert but actually only use it for reclaim, which seems kind of an odd definition. (We can fix this later.)
Quoted 5 lines
}
#endif--
2.55.0.141.g00534a21ce-goog
Sean Christopherson
maintainerRe: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
Quoted 10 lines · Ackerley Tng
arch/x86/kvm/x86.c bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm) · line 10633
Quoted 5 lines · Sean Christopherson
#ifdef
CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
voidkvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order){arch/x86/kvm/x86.c +1
-1-kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); +WARN_ON_ONCE (kvm_x86_call (gmem_convert)(NULL ,-1ull , pfn, nr_pages, max_order, false));This setup means later pKVM would have to define a .gmem_convert but actually only use it for reclaim, which seems kind of an odd definition. (We can fix this later.)
No, pKVM (on arm64) would just implement kvm_arch_gmem_reclaim(), and then do whatever it wants under the hood.
Re: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
Quoted 9 lines · Sean Christopherson
Quoted 7 lines · Ackerley Tng
Quoted 5 lines · Sean Christopherson
#ifdef
CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
voidkvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order){arch/x86/kvm/x86.c +1
-1-kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); +WARN_ON_ONCE (kvm_x86_call (gmem_convert)(NULL ,-1ull , pfn, nr_pages, max_order, false));
Quoted 6 lines
Quoted 3 lines
This setup means later pKVM would have to define a .gmem_convert but actually only use it for reclaim, which seems kind of an odd definition. (We can fix this later.)
No, pKVM (on arm64) would just implement
kvm_arch_gmem_reclaim(), and then do whatever it wants under the hood.
Right. pKVM would just select HAVE_KVM_ARCH_GMEM_RECLAIM and implement kvm_arch_gmem_reclaim() to scrub; the reclaim and convert hooks have independent Kconfigs.
Thank you Ackerley for checking, and thank you all for the respin.
Cheers,
/fuad
Re: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
In reply to Sean ChristophersonQuoted 7 lines · Sean Christopherson
Smush x86's
prepare()andinvalidate()hooks into a commonconvert()flow, as they are effectively two sides of the same coin for SNP: they're invoked when private/shared memory is about to made accessible/visible to the guest or host. I.e.prepare()is really "make private", andinvalidate()is really "make shared".
Quoted 6 lines
Using a single hook will yield more intuitive code when in-place conversion comes along.
For all intents and purposes, no functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Cheers,
/fuad
Quoted 146 lines
--- arch/x86/include/asm/kvm-x86-ops.h | 5 ++--- arch/x86/include/asm/kvm_host.h | 7 +++---- arch/x86/kvm/svm/sev.c | 21 ++++++++++++++++++--- arch/x86/kvm/svm/svm.c | 3 +-- arch/x86/kvm/svm/svm.h | 5 ++--- arch/x86/kvm/x86.c | 4 ++-- 6 files changed, 28 insertions(+), 17 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +2
-3@@ -146,10 +146,9 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -KVM_X86_OP_OPTIONAL_RET0(gmem_prepare) +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +KVM_X86_OP_OPTIONAL_RET0(gmem_convert)#endif -KVM_X86_OP_OPTIONAL(gmem_invalidate)#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE KVM_X86_OP_OPTIONAL(gmem_invalidate_range)#endif arch/x86/include/asm/kvm_host.h +3
-4@@ -1724,11 +1724,10 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int (*gmem_prepare)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order); +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif -void (*gmem_invalidate)(kvm_pfn_t start,kvm_pfn_t end);#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE void (*gmem_invalidate_range)(struct kvm *kvm,struct kvm_gfn_range *range);#endif arch/x86/kvm/svm/sev.c +18
-3@@ -5114,8 +5114,8 @@ static bool is_large_rmp_possible(struct kvm *kvm, kvm_pfn_t pfn, int order)return false; } -int sev_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order) +static int sev_gmem_make_private (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order) {struct kvm_sev_info *sev =to_kvm_sev_info (kvm);kvm_pfn_t pfn_aligned;@@ -5165,8 +5165,9 @@ int sev_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,return 0 ; } -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end) +static void sev_gmem_make_shared (kvm_pfn_t start,kvm_pfn_t nr_pages) { +kvm_pfn_t end = start + nr_pages;kvm_pfn_t pfn;if (!cc_platform_has (CC_ATTR_HOST_SEV_SNP ))@@ -5228,6 +5229,20 @@ void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end) } } +int sev_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int order,bool to_private) +{ +if (to_private) { +if (WARN_ON_ONCE (!kvm || gfn >> (kvm_host.maxphyaddr -PAGE_SHIFT ))) +return -EIO ; + +return sev_gmem_make_private (kvm, gfn, pfn, nr_pages, order); + } + +sev_gmem_make_shared (pfn, nr_pages); +return 0 ; +} +void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range) {struct kvm_vcpu *vcpu;arch/x86/kvm/svm/svm.c +1
-2@@ -5438,8 +5438,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = { .vm_copy_enc_context_from = sev_vm_copy_enc_context_from, .vm_move_enc_context_from = sev_vm_move_enc_context_from, - .gmem_prepare = sev_gmem_prepare, - .gmem_invalidate = sev_gmem_invalidate, + .gmem_convert = sev_gmem_convert, .gmem_invalidate_range = sev_gmem_invalidate_range, .gmem_max_mapping_level = sev_gmem_max_mapping_level,#endif arch/x86/kvm/svm/svm.h +2
-3@@ -1009,9 +1009,8 @@ int sev_cpu_init(struct svm_cpu_data *sd);int sev_dev_get_attr (u32 group,u64 attr,u64 *val);extern unsigned int max_sev_asid;void sev_handle_rmp_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u64 error_code); -int sev_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order); -void sev_gmem_invalidate (kvm_pfn_t start,kvm_pfn_t end); +int sev_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);void sev_gmem_invalidate_range (struct kvm *kvm,struct kvm_gfn_range *range);int sev_gmem_max_mapping_level (struct kvm *kvm,kvm_pfn_t pfn,bool is_private);struct vmcb_save_area *sev_decrypt_vmsa (struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +2
-3@@ -10623,14 +10623,14 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -return kvm_x86_call (gmem_prepare)(kvm, gfn, pfn, nr_pages, max_order); +return kvm_x86_call (gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true); }#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim (kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order) { -kvm_x86_call (gmem_invalidate)(pfn, pfn + nr_pages); +WARN_ON_ONCE (kvm_x86_call (gmem_convert)(NULL ,-1ull , pfn, nr_pages, max_order, false)); }#endif --2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
In reply to Sean ChristophersonQuoted 6 lines · Sean Christopherson
Smush x86's
prepare()andinvalidate()hooks into a commonconvert()flow, as they are effectively two sides of the same coin for SNP: they're invoked when private/shared memory is about to made accessible/visible to the guest or host. I.e.prepare()is really "make private", andinvalidate()is really "make shared".
Quoted 8 lines
Using a single hook will yield more intuitive code when in-place conversion comes along.
For all intents and purposes, no functional change intended.
[...snip...]
I just tried this out with in-place conversion and I'm stuck trying to make a call to kvm_arch_gmem_convert() in the conversions ioctl.
I think we wanted to have a single .gmem_convert callback rather than .gmem_prepare and .gmem_reclaim, but it turns out that it's not quite easy to just call convert, because I need to pass both gfn and pfn to .gmem_convert().
The conversion is called on the inode:
__kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
size_t nr_pages, uint64_t attrs, pgoff_t *err_index)
Then guest_memfd would call
kvm_gmem_convert(inode, start, end, to_private);
It is easy to get the kvm pointer for .gmem_convert(), but then to always pass something for gfn and pfn, I have to make sure that
+ the folio was allocated (to get a valid pfn)
+ some binding exists for the index (to get a valid gfn)
So for each folio that exists, I would check if there is a valid binding. If there is a valid binding, I compute the gfn, and if there's no valid binding I pass -1ull for gfn.
I think this is a little complicated and not actually all that intuitive...
Looking at KVM_AMD_SEV's Kconfig, it selects 4 gmem lifecycle callbacks:
config KVM_AMD_SEV
bool "AMD Secure Encrypted Virtualization (SEV) support" default y
depends on KVM_AMD && X86_64
depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select ARCH_HAS_CC_PLATFORM
select KVM_VM_MEMORY_ATTRIBUTES
select HAVE_KVM_ARCH_GMEM_CONVERT
select HAVE_KVM_ARCH_GMEM_RECLAIM
select HAVE_KVM_ARCH_GMEM_INVALIDATE
select HAVE_KVM_ARCH_GMEM_POPULATE
help
Provides support for launching encrypted VMs which use Secure Encrypted Virtualization (SEV), Secure Encrypted Virtualization with Encrypted State (SEV-ES), and Secure Encrypted Virtualization with Secure Nested Paging (SEV-SNP) technologies on AMD processors.
So actually smushing .gmem_prepare and .gmem_reclaim doesn't really save us CONFIG flags, and results in a kvm_gmem_convert() that needs to find the intersection of folio and binding existing just so the .gmem_convert call is valid for conversions in both directions.
If we set kvm_gmem_convert() up to do handle make_private() and make_shared() separately, then I don't think there's much value in collapsing .gmem_prepare() and .gmem_reclaim() just so we mux on the caller side and demux on the SNP side.
How about this:
1. Retain .gmem_reclaim() for "host reclaim" aka "return to host" aka "make shared" for x86.
+ Rename the make shared function to sev_gmem_reclaim
+ .gmem_reclaim = sev_gmem_reclaim
+ pKVM will defined kvm_arch_gmem_reclaim() to do what it needs
+ TDX and ARM CCA don't use this callback.
+ From __kvm_gmem_set_attributes(),
if (!to_private)
kvm_arch_gmem_reclaim(pfn, nr_pages)
+ From .free_folio() kvm_arch_gmem_reclaim(folio_pfn(folio), folio_nr_pages(folio))
2. Rename .gmem_prepare() to .gmem_grant(), or .gmem_assign(), something that is the opposite of "reclaim", that indicates the host granting/assigning/linking (other names welcome) the gfn <-> pfn
+ Only SNP will define this
+ .gmem_assign = sev_gmem_assign(kvm, gfn, pfn, nr_pages, order);
+ From kvm_gmem_get_pfn()
if (kvm_gmem_is_private_mem(inode, index))
kvm_arch_gmem_assign(kvm, gfn, pfn, nr_pages, order);
I think it is fair that "assign" or "grant" is only called for private pages, since only private pages need to be "given" away.
On the reclaim side, it's also fair that only shared pages need to be reclaimed.
And with this, we still have 4 CONFIGs just like above, except that it's now
select HAVE_KVM_ARCH_GMEM_ASSIGN
select HAVE_KVM_ARCH_GMEM_RECLAIM
select HAVE_KVM_ARCH_GMEM_INVALIDATE
select HAVE_KVM_ARCH_GMEM_POPULATE
And I guess it's more straightforward that one CONFIG selects one callback, avoiding having either of the CONFIGs select .gmem_convert():
Quoted 7 lines
arch/x86/include/asm/kvm_host.h +3
-0+#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif
Re: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
Quoted 31 lines · Ackerley Tng
Quoted 6 lines · Sean Christopherson
Smush x86's
prepare()andinvalidate()hooks into a commonconvert()flow, as they are effectively two sides of the same coin for SNP: they're invoked when private/shared memory is about to made accessible/visible to the guest or host. I.e.prepare()is really "make private", andinvalidate()is really "make shared".Quoted 8 lines
Using a single hook will yield more intuitive code when in-place conversion comes along.
For all intents and purposes, no functional change intended.
[...snip...]
I just tried this out with in-place conversion and I'm stuck trying to make a call to
kvm_arch_gmem_convert()in the conversions ioctl.I think we wanted to have a single .gmem_convert callback rather than .gmem_prepare and .gmem_reclaim, but it turns out that it's not quite easy to just call convert, because I need to pass both gfn and pfn to .gmem_convert().
The conversion is called on the inode:
__kvm_gmem_set_attributes(struct inode *inode,
pgoff_tstart,
size_tnr_pages,uint64_tattrs,pgoff_t*err_index)
Quoted 6 lines
Then
guest_memfdwould callkvm_gmem_convert(inode, start, end, to_private);It is easy to get the kvm pointer for .gmem_convert(), but then to always pass something for gfn and pfn, I have to make sure that
As I stated somewhere else, conversion to SHARED does NOT and *should* NOT require a valid GFN. Conceptually, a SHARED page can't be strictly associated with a single GFN.
Quoted 29 lines
+ the folio was allocated (to get a valid pfn)
+ some binding exists for the index (to get a valid gfn)So for each folio that exists, I would check if there is a valid binding. If there is a valid binding, I compute the gfn, and if there's no valid binding I pass -1ull for gfn.
I think this is a little complicated and not actually all that intuitive...
Looking at KVM_AMD_SEV's
Kconfig, it selects 4 gmem lifecycle callbacks:config
KVM_AMD_SEV
bool "AMD Secure Encrypted Virtualization (SEV) support" default y
depends onKVM_AMD&&X86_64
depends onCRYPTO_DEV_SP_PSP&& !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) selectARCH_HAS_CC_PLATFORM
selectKVM_VM_MEMORY_ATTRIBUTES
selectHAVE_KVM_ARCH_GMEM_CONVERT
selectHAVE_KVM_ARCH_GMEM_RECLAIM
selectHAVE_KVM_ARCH_GMEM_INVALIDATE
selectHAVE_KVM_ARCH_GMEM_POPULATE
help
Provides support for launching encrypted VMs which use Secure Encrypted Virtualization (SEV), Secure Encrypted Virtualization with Encrypted State (SEV-ES), and Secure Encrypted Virtualization with Secure Nested Paging (SEV-SNP) technologies on AMD processors.
Quoted 4 lines
So actually smushing .gmem_prepare and .gmem_reclaim doesn't really save us CONFIG flags, and results in a
kvm_gmem_convert()that needs to find the intersection of folio and binding existing just so the .gmem_convert call is valid for conversions in both directions.
And I'm saying don't do that for the to_shared case. If passing a NULL @kvm and -1ull for @gfn from guest_memfd is too ugly, I'd be a-ok with providing separate kvm_arch_gmem_make_{private,shared}() under GMEM_CONVERT.
I guess at that point I don't have a strong preference betwee having a single kvm_x86_ops hook versus also having kvm_x86_ops.gmem_make_{private,shared}().
I.e. Option A:
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
kvm_pfn_t nr_pages, int max_order)
{
return kvm_x86_call(gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true);
}
int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order,
bool to_private)
{
return kvm_x86_call(gmem_convert)(NULL, -1ull, pfn, nr_pages, max_order, false);
}
#endif
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
void kvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order)
{
WARN_ON_ONCE(kvm_x86_call(gmem_convert)(NULL, -1ull, pfn, nr_pages, max_order, false));
}
#endifOr Option B:
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
kvm_pfn_t nr_pages, int max_order)
{
return kvm_x86_call(gmem_make_private)(kvm, gfn, pfn, nr_pages, max_order);
}
int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order,
bool to_private)
{
kvm_x86_call(gmem_make_shared)(pfn, nr_pages, max_order);
return 0;
}
#endif
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM
void kvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order)
{
kvm_x86_call(gmem_make_shared)(pfn, nr_pages, max_order);
}
#endifTyping it out, option B does look prettier...
Quoted 15 lines
If we set
kvm_gmem_convert()up to do handlemake_private()andmake_shared()separately, then I don't think there's much value in collapsing .gmem_prepare() and .gmem_reclaim() just so we mux on the caller side and demux on the SNP side.How about this:
1. Retain .gmem_reclaim() for "host reclaim" aka "return to host" aka "make shared" for x86.
+ Rename the make shared function tosev_gmem_reclaim
+ .gmem_reclaim =sev_gmem_reclaim
+ pKVM will definedkvm_arch_gmem_reclaim()to do what it needs
+ TDX and ARM CCA don't use this callback.
+ From__kvm_gmem_set_attributes(),
Quoted 2 lines
if (!to_private)
kvm_arch_gmem_reclaim(pfn, nr_pages)
NAK, pKVM doesn't want to "reclaim" on conversions to SHARED. reclaim() and convert() are two similar but distinct operations. Irrespective of pKVM, IMO it's also very useful to differentiate between "this page is being shared with the host, but KVM still owns the page" and "this page is being freed back to the host". E.g. reclaim() *must* succeed or take evasive action, whereas convert() can gracefully return an error.
Quoted 37 lines
+ From .free_folio()
kvm_arch_gmem_reclaim(folio_pfn(folio), folio_nr_pages(folio))2. Rename .gmem_prepare() to .gmem_grant(), or .gmem_assign(), something that is the opposite of "reclaim", that indicates the host granting/assigning/linking (other names welcome) the gfn <-> pfn
+ Only SNP will define this
+ .gmem_assign =sev_gmem_assign(kvm, gfn, pfn, nr_pages, order);
+ Fromkvm_gmem_get_pfn()if (
kvm_gmem_is_private_mem(inode, index))
kvm_arch_gmem_assign(kvm, gfn, pfn, nr_pages, order);I think it is fair that "assign" or "grant" is only called for private pages, since only private pages need to be "given" away.
On the reclaim side, it's also fair that only shared pages need to be reclaimed.
And with this, we still have 4 CONFIGs just like above, except that it's now
select
HAVE_KVM_ARCH_GMEM_ASSIGN
selectHAVE_KVM_ARCH_GMEM_RECLAIM
selectHAVE_KVM_ARCH_GMEM_INVALIDATE
selectHAVE_KVM_ARCH_GMEM_POPULATEAnd I guess it's more straightforward that one CONFIG selects one callback, avoiding having either of the CONFIGs select .gmem_convert():
Quoted 7 lines
arch/x86/include/asm/kvm_host.h +3
-0+#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif
Re: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
Quoted 72 lines · Sean Christopherson
Quoted 31 lines · Ackerley Tng
Quoted 6 lines · Sean Christopherson
Smush x86's
prepare()andinvalidate()hooks into a commonconvert()flow, as they are effectively two sides of the same coin for SNP: they're invoked when private/shared memory is about to made accessible/visible to the guest or host. I.e.prepare()is really "make private", andinvalidate()is really "make shared".Quoted 8 lines
Using a single hook will yield more intuitive code when in-place conversion comes along.
For all intents and purposes, no functional change intended.
[...snip...]
I just tried this out with in-place conversion and I'm stuck trying to make a call to
kvm_arch_gmem_convert()in the conversions ioctl.I think we wanted to have a single .gmem_convert callback rather than .gmem_prepare and .gmem_reclaim, but it turns out that it's not quite easy to just call convert, because I need to pass both gfn and pfn to .gmem_convert().
The conversion is called on the inode:
__kvm_gmem_set_attributes(struct inode *inode,
pgoff_tstart,
size_tnr_pages,uint64_tattrs,pgoff_t*err_index)Quoted 6 lines
Then
guest_memfdwould callkvm_gmem_convert(inode, start, end, to_private);It is easy to get the kvm pointer for .gmem_convert(), but then to always pass something for gfn and pfn, I have to make sure that
As I stated somewhere else, conversion to SHARED does NOT and *should* NOT require a valid GFN. Conceptually, a SHARED page can't be strictly associated with a single GFN.
Quoted 29 lines
+ the folio was allocated (to get a valid pfn)
+ some binding exists for the index (to get a valid gfn)So for each folio that exists, I would check if there is a valid binding. If there is a valid binding, I compute the gfn, and if there's no valid binding I pass -1ull for gfn.
I think this is a little complicated and not actually all that intuitive...
Looking at KVM_AMD_SEV's
Kconfig, it selects 4 gmem lifecycle callbacks:config
KVM_AMD_SEV
bool "AMD Secure Encrypted Virtualization (SEV) support" default y
depends onKVM_AMD&&X86_64
depends onCRYPTO_DEV_SP_PSP&& !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) selectARCH_HAS_CC_PLATFORM
selectKVM_VM_MEMORY_ATTRIBUTES
selectHAVE_KVM_ARCH_GMEM_CONVERT
selectHAVE_KVM_ARCH_GMEM_RECLAIM
selectHAVE_KVM_ARCH_GMEM_INVALIDATE
selectHAVE_KVM_ARCH_GMEM_POPULATE
help
Provides support for launching encrypted VMs which use Secure Encrypted Virtualization (SEV), Secure Encrypted Virtualization with Encrypted State (SEV-ES), and Secure Encrypted Virtualization with Secure Nested Paging (SEV-SNP) technologies on AMD processors.
Quoted 7 lines
Quoted 4 lines
So actually smushing .gmem_prepare and .gmem_reclaim doesn't really save us CONFIG flags, and results in a
kvm_gmem_convert()that needs to find the intersection of folio and binding existing just so the .gmem_convert call is valid for conversions in both directions.And I'm saying don't do that for the
to_sharedcase. If passing a NULL @kvm and -1ull for @gfn fromguest_memfdis too ugly,
It is ugly, but I thought the point of providing a single .gmem_convert callback was so the conversion path could just call convert with kvm and gfn arguments that work for both directions.
I like Option B below better, which should also resolve my issue.
Quoted 72 lines
I'd be a-ok with providing
separate kvm_arch_gmem_make_{private,shared}() underGMEM_CONVERT.I guess at that point I don't have a strong preference betwee having a single
kvm_x86_opshook versus also having kvm_x86_ops.gmem_make_{private,shared}().I.e. Option A:
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { return kvm_x86_call(gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true); } int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order, bool to_private) { return kvm_x86_call(gmem_convert)(NULL, -1ull, pfn, nr_pages, max_order, false); } #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { WARN_ON_ONCE(kvm_x86_call(gmem_convert)(NULL, -1ull, pfn, nr_pages, max_order, false)); } #endifOr Option B:
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { return kvm_x86_call(gmem_make_private)(kvm, gfn, pfn, nr_pages, max_order); } int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order, bool to_private) { kvm_x86_call(gmem_make_shared)(pfn, nr_pages, max_order); return 0; } #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { kvm_x86_call(gmem_make_shared)(pfn, nr_pages, max_order); } #endifTyping it out, option B does look prettier...
Quoted 15 lines
If we set
kvm_gmem_convert()up to do handlemake_private()andmake_shared()separately, then I don't think there's much value in collapsing .gmem_prepare() and .gmem_reclaim() just so we mux on the caller side and demux on the SNP side.How about this:
1. Retain .gmem_reclaim() for "host reclaim" aka "return to host" aka "make shared" for x86.
+ Rename the make shared function tosev_gmem_reclaim
+ .gmem_reclaim =sev_gmem_reclaim
+ pKVM will definedkvm_arch_gmem_reclaim()to do what it needs
+ TDX and ARM CCA don't use this callback.
+ From__kvm_gmem_set_attributes(),Quoted 2 lines
if (!to_private)
kvm_arch_gmem_reclaim(pfn, nr_pages)
Quoted 7 lines
NAK, pKVM doesn't want to "reclaim" on conversions to SHARED.
reclaim()andconvert()are two similar but distinct operations. Irrespective of pKVM, IMO it's also very useful to differentiate between "this page is being shared with the host, but KVM still owns the page" and "this page is being freed back to the host". E.g.reclaim()*must* succeed or take evasive action, whereasconvert()can gracefully return an error.
Makes sense to distinguish reclaim and make_shared. Thanks!
Quoted 37 lines
Quoted 37 lines
+ From .free_folio()
kvm_arch_gmem_reclaim(folio_pfn(folio), folio_nr_pages(folio))2. Rename .gmem_prepare() to .gmem_grant(), or .gmem_assign(), something that is the opposite of "reclaim", that indicates the host granting/assigning/linking (other names welcome) the gfn <-> pfn
+ Only SNP will define this
+ .gmem_assign =sev_gmem_assign(kvm, gfn, pfn, nr_pages, order);
+ Fromkvm_gmem_get_pfn()if (
kvm_gmem_is_private_mem(inode, index))
kvm_arch_gmem_assign(kvm, gfn, pfn, nr_pages, order);I think it is fair that "assign" or "grant" is only called for private pages, since only private pages need to be "given" away.
On the reclaim side, it's also fair that only shared pages need to be reclaimed.
And with this, we still have 4 CONFIGs just like above, except that it's now
select
HAVE_KVM_ARCH_GMEM_ASSIGN
selectHAVE_KVM_ARCH_GMEM_RECLAIM
selectHAVE_KVM_ARCH_GMEM_INVALIDATE
selectHAVE_KVM_ARCH_GMEM_POPULATEAnd I guess it's more straightforward that one CONFIG selects one callback, avoiding having either of the CONFIGs select .gmem_convert():
Quoted 7 lines
arch/x86/include/asm/kvm_host.h +3
-0+#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif
Sean Christopherson
maintainerRe: [PATCH v5 6/7] KVM: x86: Combine .gmem_prepare()+.gmem_invalidate() into .gmem_convert()
Quoted 2 lines · Ackerley Tng
Sean Christopherson <[email protected]> writes:
I like Option B below better, which should also resolve my issue.
Roger that. The more I stare at this, the more I like Option B, too. I'll go with that.
Quoted 51 lines
Quoted 51 lines
I'd be a-ok with providing
separate kvm_arch_gmem_make_{private,shared}() underGMEM_CONVERT.I guess at that point I don't have a strong preference betwee having a single
kvm_x86_opshook versus also having kvm_x86_ops.gmem_make_{private,shared}().I.e. Option A:
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { return kvm_x86_call(gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true); } int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order, bool to_private) { return kvm_x86_call(gmem_convert)(NULL, -1ull, pfn, nr_pages, max_order, false); } #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { WARN_ON_ONCE(kvm_x86_call(gmem_convert)(NULL, -1ull, pfn, nr_pages, max_order, false)); } #endifOr Option B:
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT int kvm_arch_gmem_make_private(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { return kvm_x86_call(gmem_make_private)(kvm, gfn, pfn, nr_pages, max_order); } int kvm_arch_gmem_make_shared(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order, bool to_private) { kvm_x86_call(gmem_make_shared)(pfn, nr_pages, max_order); return 0; } #endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM void kvm_arch_gmem_reclaim(kvm_pfn_t pfn, kvm_pfn_t nr_pages, int max_order) { kvm_x86_call(gmem_make_shared)(pfn, nr_pages, max_order); } #endif
Sean Christopherson
maintainerPATCH v5 7/7 KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
In reply to Sean ChristophersonRework guest_memfd's "prepare" hook into a more generic "convert" flow in anticipation of supporting in-place conversion, at which point KVM will use the hook for both to-private and to-shared conversions, not just to "prepare" PRIVATE memory.
Opportunistically rename kvm_gmem_prepare_folio() to kvm_gmem_make_private() to better reflect its role.
No functional change intended.
--- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/x86.c | 8 ++++---- include/linux/kvm_host.h | 6 +++--- virt/kvm/Kconfig | 2 +- virt/kvm/guest_memfd.c | 12 ++++++------ 7 files changed, 17 insertions(+), 17 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -1
@@ -146,7 +146,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) KVM_X86_OP_OPTIONAL_RET0(gmem_convert)#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE
arch/x86/include/asm/kvm_host.h +1 -1
@@ -1724,7 +1724,7 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM )int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order,bool to_private);#endif
arch/x86/kvm/Kconfig +1 -1
@@ -160,7 +160,7 @@ config KVM_AMD_SEV depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES - select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATE
arch/x86/kvm/x86.c +4 -4
@@ -10619,11 +10619,11 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)return !kvm_arch_has_private_mem (kvm); } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private) { -return kvm_x86_call (gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true); +return kvm_x86_call (gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, to_private); }#endif
include/linux/kvm_host.h +3 -3
@@ -2579,9 +2579,9 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, }#endif /* CONFIG_KVM_GUEST_MEMFD */ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE
virt/kvm/Kconfig +1 -1
@@ -107,7 +107,7 @@ config KVM_GUEST_MEMFD select XARRAY_MULTI bool -config HAVE_KVM_ARCH_GMEM_PREPARE +config HAVE_KVM_ARCH_GMEM_CONVERT bool depends on KVM_GUEST_MEMFD
virt/kvm/guest_memfd.c +6 -6
@@ -66,10 +66,10 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)* On successful return the guest sees a zero page so as to avoid * leaking host data and the up-to-date flag is set. */ -static int kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot, -gfn_t gfn,struct folio *folio) +static int kvm_gmem_make_private (struct kvm *kvm,struct kvm_memory_slot *slot, +gfn_t gfn,struct folio *folio) { -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT unsigned long nr_pages =folio_nr_pages (folio);pgoff_t index;
@@ -90,8 +90,8 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, gfn =ALIGN_DOWN (gfn, nr_pages); index =kvm_gmem_get_index (slot, gfn); -return kvm_arch_gmem_prepare (kvm, gfn,folio_file_pfn (folio, index), - nr_pages,folio_order (folio)); +return kvm_arch_gmem_convert (kvm, gfn,folio_file_pfn (folio, index), + nr_pages,folio_order (folio), true);#else return 0 ;#endif
@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); + r =kvm_gmem_make_private (kvm, slot, gfn, folio);folio_unlock (folio);
Signature · 1 line
2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
Quoted 11 lines · Sean Christopherson
Rework guest_memfd's "prepare" hook into a more generic "convert" flow in anticipation of supporting in-place conversion, at which point KVM will use the hook for both to-private and to-shared conversions, not just to "prepare" PRIVATE memory.
Opportunistically rename
kvm_gmem_prepare_folio()tokvm_gmem_make_private()to better reflect its role.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Thanks!
Re: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
In reply to Sean ChristophersonQuoted 6 lines · Sean Christopherson
Rework guest_memfd's "prepare" hook into a more generic "convert" flow in anticipation of supporting in-place conversion, at which point KVM will use the hook for both to-private and to-shared conversions, not just to "prepare" PRIVATE memory.
Quoted 6 lines
Opportunistically rename
kvm_gmem_prepare_folio()tokvm_gmem_make_private()to better reflect its role.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Cheers,
/fuad
Quoted 140 lines
--- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/Kconfig | 2 +- arch/x86/kvm/x86.c | 8 ++++---- include/linux/kvm_host.h | 6 +++--- virt/kvm/Kconfig | 2 +- virt/kvm/guest_memfd.c | 12 ++++++------ 7 files changed, 17 insertions(+), 17 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +1
-1@@ -146,7 +146,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)KVM_X86_OP_OPTIONAL_RET0 (vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL(alloc_apic_backing_page) -#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) KVM_X86_OP_OPTIONAL_RET0(gmem_convert)#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE arch/x86/include/asm/kvm_host.h +1
-1@@ -1724,7 +1724,7 @@ struct kvm_x86_ops {gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu,gva_t gva,unsigned int flags);void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu); -#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM ) +#if defined(CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT ) || defined(CONFIG_HAVE_KVM_ARCH_GMEM_RECLAIM )int (*gmem_convert)(struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn,kvm_pfn_t nr_pages,int max_order,bool to_private);#endif arch/x86/kvm/Kconfig +1
-1@@ -160,7 +160,7 @@ config KVM_AMD_SEV depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) select ARCH_HAS_CC_PLATFORM select KVM_GENERIC_MEMORY_ATTRIBUTES - select HAVE_KVM_ARCH_GMEM_PREPARE + select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE select HAVE_KVM_ARCH_GMEM_POPULATEarch/x86/kvm/x86.c +4
-4@@ -10619,11 +10619,11 @@ bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)return !kvm_arch_has_private_mem (kvm); } -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order) +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private) { -return kvm_x86_call (gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, true); +return kvm_x86_call (gmem_convert)(kvm, gfn, pfn, nr_pages, max_order, to_private); }#endif include/linux/kvm_host.h +3
-3@@ -2579,9 +2579,9 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, }#endif /* CONFIG_KVM_GUEST_MEMFD */ -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE -int kvm_arch_gmem_prepare (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, -kvm_pfn_t nr_pages,int max_order); +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT +int kvm_arch_gmem_convert (struct kvm *kvm,gfn_t gfn,kvm_pfn_t pfn, +kvm_pfn_t nr_pages,int max_order,bool to_private);#endif #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_POPULATE virt/kvm/Kconfig +1
-1@@ -107,7 +107,7 @@ config KVM_GUEST_MEMFD select XARRAY_MULTI bool -config HAVE_KVM_ARCH_GMEM_PREPARE +config HAVE_KVM_ARCH_GMEM_CONVERT bool depends on KVM_GUEST_MEMFDvirt/kvm/guest_memfd.c +6
-7@@ -66,10 +66,10 @@ static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn)* On successful return the guest sees a zero page so as to avoid * leaking host data and the up-to-date flag is set. */ -static int kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot, -gfn_t gfn,struct folio *folio) +static int kvm_gmem_make_private (struct kvm *kvm,struct kvm_memory_slot *slot, +gfn_t gfn,struct folio *folio) { -#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT unsigned long nr_pages =folio_nr_pages (folio);pgoff_t index;@@ -90,8 +90,8 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, gfn =ALIGN_DOWN (gfn, nr_pages); index =kvm_gmem_get_index (slot, gfn); -return kvm_arch_gmem_prepare (kvm, gfn,folio_file_pfn (folio, index), - nr_pages,folio_order (folio)); +return kvm_arch_gmem_convert (kvm, gfn,folio_file_pfn (folio, index), + nr_pages,folio_order (folio), true);#else return 0 ;#endif @@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); + r =kvm_gmem_make_private (kvm, slot, gfn, folio);folio_unlock (folio); --2.55.0.141.g00534a21ce-goog
Re: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
In reply to Sean ChristophersonQuoted 8 lines · Sean Christopherson
Rework guest_memfd's "prepare" hook into a more generic "convert" flow in anticipation of supporting in-place conversion, at which point KVM will use the hook for both to-private and to-shared conversions, not just to "prepare" PRIVATE memory.
Opportunistically rename
kvm_gmem_prepare_folio()tokvm_gmem_make_private()to better reflect its role.
[...]
Quoted 12 lines
virt/kvm/guest_memfd.c +2
-2@@ -90,8 +90,8 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, gfn =ALIGN_DOWN (gfn, nr_pages); index =kvm_gmem_get_index (slot, gfn); -return kvm_arch_gmem_prepare (kvm, gfn,folio_file_pfn (folio, index), - nr_pages,folio_order (folio)); +return kvm_arch_gmem_convert (kvm, gfn,folio_file_pfn (folio, index), + nr_pages,folio_order (folio), true);#else return 0 ;#endif
virt/kvm/guest_memfd.c int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, · line 801
Quoted 5 lines
virt/kvm/guest_memfd.c +1
-1@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,folio_mark_uptodate (folio); } - r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio); + r =kvm_gmem_make_private (kvm, slot, gfn, folio);
I think this and above renaming don't make sense, because kvm_gmem_get_pfn() can be invoked for shared memory for non-Coco VMs when KVM_MEMSLOT_GMEM_ONLY is set.
Maybe this patch can be moved to gmem in-place series and after [1]?
Quoted 2 lines
virt/kvm/guest_memfd.c +0
-0folio_unlock (folio);
Re: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
Quoted 27 lines · Xiaoyao Li
Quoted 8 lines · Sean Christopherson
Rework guest_memfd's "prepare" hook into a more generic "convert" flow in anticipation of supporting in-place conversion, at which point KVM will use the hook for both to-private and to-shared conversions, not just to "prepare" PRIVATE memory.
Opportunistically rename
kvm_gmem_prepare_folio()tokvm_gmem_make_private()to better reflect its role.[...]
Quoted 15 lines
(quoted diff) +2
-2@@ -90,8 +90,8 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, gfn = ALIGN_DOWN(gfn, nr_pages); index = kvm_gmem_get_index(slot, gfn); - return kvm_arch_gmem_prepare(kvm, gfn, folio_file_pfn(folio, index), - nr_pages, folio_order(folio)); + return kvm_arch_gmem_convert(kvm, gfn, folio_file_pfn(folio, index), + nr_pages, folio_order(folio), true); #else return 0; #endif@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, folio_mark_uptodate(folio); }
Quoted 7 lines
Review int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, · line 801
Quoted 2 lines
(quoted diff) +1
-1- r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); + r = kvm_gmem_make_private(kvm, slot, gfn, folio);I think this and above renaming don't make sense, because
kvm_gmem_get_pfn()can be invoked for shared memory for non-Coco VMs whenKVM_MEMSLOT_GMEM_ONLYis set.
Makes sense.
Quoted 5 lines
Maybe this patch can be moved to gmem in-place series and after [1]?
How about doing everything else in this patch in this patch, other than this:
virt/kvm/guest_memfd.c +0 -0
@@ -66,8 +66,8 @@ static pgoff_t kvm_gmem_get_index(struct
kvm_memory_slot *slot, gfn_t gfn)
* On successful return the guest sees a zero page so as to avoid
* leaking host data and the up-to-date flag is set.
*/
-static int kvm_gmem_make_private(struct kvm *kvm, struct kvm_memory_slot *slot,
- gfn_t gfn, struct folio *folio)
+static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot,
+ gfn_t gfn, struct folio *folio)
{
#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
unsigned long nr_pages = folio_nr_pages(folio);
@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
folio_mark_uptodate(folio);
}
- r = kvm_gmem_make_private(kvm, slot, gfn, folio);
+ r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio);
folio_unlock(folio);
I'll move just this renaming to [1] like you suggested.
I think it's okay to continue to always call prepare_folio(), and within the prepare_folio() function, only do conversion when the CONVERT CONFIG is defined.
I'll also move "Opportunistically rename kvm_gmem_prepare_folio() to kvm_gmem_make_private() to better reflect its role." from Sean's commit message to [1].
Quoted 2 lines
Quoted 2 lines
(quoted diff) +0
-0folio_unlock(folio);
Re: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
Quoted 75 lines · Ackerley Tng
Quoted 27 lines · Xiaoyao Li
Quoted 8 lines · Sean Christopherson
Rework guest_memfd's "prepare" hook into a more generic "convert" flow in anticipation of supporting in-place conversion, at which point KVM will use the hook for both to-private and to-shared conversions, not just to "prepare" PRIVATE memory.
Opportunistically rename
kvm_gmem_prepare_folio()tokvm_gmem_make_private()to better reflect its role.[...]
Quoted 15 lines
(quoted diff) +2
-2@@ -90,8 +90,8 @@ static int kvm_gmem_prepare_folio(struct kvm *kvm, struct kvm_memory_slot *slot, gfn = ALIGN_DOWN(gfn, nr_pages); index = kvm_gmem_get_index(slot, gfn); - return kvm_arch_gmem_prepare(kvm, gfn, folio_file_pfn(folio, index), - nr_pages, folio_order(folio)); + return kvm_arch_gmem_convert(kvm, gfn, folio_file_pfn(folio, index), + nr_pages, folio_order(folio), true); #else return 0; #endif@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, folio_mark_uptodate(folio); }Quoted 7 lines
Review int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, · line 801
Quoted 2 lines
(quoted diff) +1
-1- r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); + r = kvm_gmem_make_private(kvm, slot, gfn, folio);I think this and above renaming don't make sense, because
kvm_gmem_get_pfn()can be invoked for shared memory for non-Coco VMs whenKVM_MEMSLOT_GMEM_ONLYis set.Makes sense.
Quoted 5 lines
Maybe this patch can be moved to gmem in-place series and after [1]?
How about doing everything else in this patch in this patch, other than this:
virt/kvm/guest_memfd.c +2
-2@@ -66,8 +66,8 @@ static pgoff_t kvm_gmem_get_index(structkvm_memory_slot *slot, gfn_t gfn) * On successful return the guest sees a zero page so as to avoid * leaking host data and the up-to-date flag is set. */ -static int kvm_gmem_make_private(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, struct folio *folio) +static int kvm_gmem_prepare_folio (struct kvm *kvm,struct kvm_memory_slot *slot, +gfn_t gfn,struct folio *folio) {#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT unsigned long nr_pages = folio_nr_pages(folio);virt/kvm/guest_memfd.c +1
-1@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, structkvm_memory_slot *slot,folio_mark_uptodate (folio ); } - r =kvm_gmem_make_private (kvm, slot, gfn, folio); + r =kvm_gmem_prepare_folio (kvm, slot, gfn, folio);folio_unlock (folio);
Quoted 5 lines
I'll move just this renaming to [1] like you suggested.
I think it's okay to continue to always call
prepare_folio(), and within theprepare_folio()function, only do conversion when the CONVERT CONFIG is defined.
I don't think so.
This patch not only renames kvm_arch_gmem_prepare() to kvm_arch_gmem_convert(), but also adds one more parameter
'bool to_private'
and hardcodes the new parameter to true. This mean the arch callback will convert the folio to private unconditionally in kvm_prepare_folio() when CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT is enabled.
Note the CONFIG is not a per-VM thing but a build time thing. The unconditionally-converting-to-private semantic can also be applied to gmem-only memslot for non-Coco VMs whenever the HAVE_KVM_ARCH_GMEM_CONVERT is enabled when building the kernel.
Though the code won't do anything for gmem-only memslot for non-Coco VMs, the literal semantic of the function and parameter is wrong.
Sean Christopherson
maintainerRe: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
Quoted 29 lines · Xiaoyao Li
Quoted 11 lines · Ackerley Tng
Xiaoyao Li <[email protected]> writes:
(quoted diff) +1
-1@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, folio_mark_uptodate(folio); } - r = kvm_gmem_make_private(kvm, slot, gfn, folio); + r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); folio_unlock(folio);Quoted 5 lines
I'll move just this renaming to [1] like you suggested.
I think it's okay to continue to always call
prepare_folio(), and within theprepare_folio()function, only do conversion when the CONVERT CONFIG is defined.I don't think so.
This patch not only renames
kvm_arch_gmem_prepare()tokvm_arch_gmem_convert(), but also adds one more parameter'bool
to_private'and hardcodes the new parameter to true. This mean the arch callback will convert the folio to private unconditionally in
kvm_prepare_folio()whenCONFIG_HAVE_KVM_ARCH_GMEM_CONVERTis enabled.
Quoted 7 lines
Note the CONFIG is not a per-VM thing but a build time thing. The unconditionally-converting-to-private semantic can also be applied to gmem-only memslot for non-Coco VMs whenever the
HAVE_KVM_ARCH_GMEM_CONVERTis enabled when building the kernel.Though the code won't do anything for gmem-only memslot for non-Coco VMs, the literal semantic of the function and parameter is wrong.
Agreed. How about I add a patch to guard the call with:
if (kvm_arch_has_private_mem(kvm) && !(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED)) r = kvm_gmem_make_private(kvm, slot, gfn, folio);
which is semantically correct pre-in-place conversion. And then when in-place conversion comes along, that will get switched to:
if (kvm_gmem_is_private_mem(inode, index)) r = kvm_gmem_make_private(kvm, slot, gfn, folio);
Which IMO yields a very clean and intuitive diff.
Actually, even better would be to insert a patch to provide kvm_gmem_is_private_mem() and kvm_gmem_is_shared_mem() as part of this prep, and pull in "KVM: guest_memfd:
Only prepare folios for private pages" with a massaged shortlog+changelog.
I.e. have this at the end of this prep work:
static bool kvm_gmem_is_private_mem(struct inode *inode, pgoff_t index)
{
return kvm_arch_has_private_mem(kvm) &&
!(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED);
}static bool kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index)
{
return !kvm_gmem_is_private_mem(inode, index);
}if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) return VM_FAULT_SIGBUS; if (kvm_gmem_is_private_mem(inode, index)) r = kvm_gmem_make_private(kvm, slot, gfn, folio);
And then "KVM: guest_memfd: Introduce per-gmem attributes, use to guard user mappings" isn't changing the semantics of the callers, it's only changing the internal plumbing for PRIVATE vs. SHARED.
Re: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
Quoted 47 lines · Sean Christopherson
Quoted 29 lines · Xiaoyao Li
Quoted 11 lines · Ackerley Tng
Xiaoyao Li <[email protected]> writes:
(quoted diff) +1
-1@@ -798,7 +798,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, folio_mark_uptodate(folio); } - r = kvm_gmem_make_private(kvm, slot, gfn, folio); + r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); folio_unlock(folio);Quoted 5 lines
I'll move just this renaming to [1] like you suggested.
I think it's okay to continue to always call
prepare_folio(), and within theprepare_folio()function, only do conversion when the CONVERT CONFIG is defined.I don't think so.
This patch not only renames
kvm_arch_gmem_prepare()tokvm_arch_gmem_convert(), but also adds one more parameter'bool
to_private'and hardcodes the new parameter to true. This mean the arch callback will convert the folio to private unconditionally in
kvm_prepare_folio()whenCONFIG_HAVE_KVM_ARCH_GMEM_CONVERTis enabled.Quoted 7 lines
Note the CONFIG is not a per-VM thing but a build time thing. The unconditionally-converting-to-private semantic can also be applied to gmem-only memslot for non-Coco VMs whenever the
HAVE_KVM_ARCH_GMEM_CONVERTis enabled when building the kernel.Though the code won't do anything for gmem-only memslot for non-Coco VMs, the literal semantic of the function and parameter is wrong.
Agreed. How about I add a patch to guard the call with:
if (kvm_arch_has_private_mem(kvm) && !(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED)) r = kvm_gmem_make_private(kvm, slot, gfn, folio);which is semantically correct pre-in-place conversion. And then when in-place conversion comes along, that will get switched to:
Quoted 8 lines
if (kvm_gmem_is_private_mem(inode, index)) r = kvm_gmem_make_private(kvm, slot, gfn, folio);Which IMO yields a very clean and intuitive diff.
Actually, even better would be to insert a patch to provide
kvm_gmem_is_private_mem()andkvm_gmem_is_shared_mem()as part of this prep, and pull in "KVM:guest_memfd:
Only prepare folios for private pages" with a massaged shortlog+changelog.
yeah. This looks good!
Quoted 7 lines
I.e. have this at the end of this prep work:
static bool
kvm_gmem_is_private_mem(struct inode *inode, pgoff_t index){ return kvm_arch_has_private_mem(kvm) && !(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED); }
I'm wondering if we really need to check kvm_arch_has_private_mem() here. It looks checking GUEST_MEMFD_FLAG_INIT_SHARED is sufficient.
Quoted 15 lines
static bool
kvm_gmem_is_shared_mem(struct inode *inode, pgoff_t index){ return !kvm_gmem_is_private_mem(inode, index); }if (!kvm_gmem_is_shared_mem(inode, vmf->pgoff)) return VM_FAULT_SIGBUS; if (kvm_gmem_is_private_mem(inode, index)) r = kvm_gmem_make_private(kvm, slot, gfn, folio);And then "KVM:
guest_memfd: Introduce per-gmem attributes, use to guard user mappings" isn't changing the semantics of the callers, it's only changing the internal plumbing for PRIVATE vs. SHARED.
Sean Christopherson
maintainerRe: [PATCH v5 7/7] KVM: guest_memfd: Rework PREPARE config and hook into a more generic CONVERT
Quoted 11 lines · Xiaoyao Li
Quoted 7 lines · Sean Christopherson
I.e. have this at the end of this prep work:
static bool
kvm_gmem_is_private_mem(struct inode *inode, pgoff_t index){ return kvm_arch_has_private_mem(kvm) && !(GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED); }I'm wondering if we really need to check
kvm_arch_has_private_mem()here. It looks checkingGUEST_MEMFD_FLAG_INIT_SHAREDis sufficient.
Ya, I came to the same conclusion and ended up dropping that line so that the patch would be a guaranteed "No functional change intended".