KVM: s390: More gmap and vsie fixes
Claudio Imbrenda
maintainercover KVM: s390: More gmap and vsie fixes
Another batch of fixups for gmap and vsie. Some minor fixes, some not-so-minor fixes that could have caused guest corruption under particular circumstances.
Claudio Imbrenda (7):
KVM: s390: Fix _gmap_crstep_xchg_atomic()
KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
KVM: s390: vsie: Fix rmap handling in _do_shadow_crste()
KVM: s390: Fix fault-in code
KVM: s390: Avoid potentially sleeping while atomic when zapping pages
KVM: s390: Lock pte when making page secure
KVM: s390: Prevent memslots outside the ASCE range
arch/s390/include/asm/gmap_helpers.h | 1 + arch/s390/kvm/faultin.c | 13 ++-- arch/s390/kvm/gaccess.c | 11 +-- arch/s390/kvm/gmap.h | 2 +- arch/s390/kvm/kvm-s390.c | 24 +++++- arch/s390/kvm/priv.c | 8 +- arch/s390/kvm/pv.c | 15 +++- arch/s390/mm/gmap_helpers.c | 111 ++++++++++++++++----------- 8 files changed, 121 insertions(+), 64 deletions(-)
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH 1/7 KVM: s390: Fix _gmap_crstep_xchg_atomic()
Return false and do not perform the operation if an unshadow event has been triggered.
The previous incorrect behaviour cleared the vsie_notif bit without returning false, which allowed shadow crstes to be installed without the vsie_notif bit.
--- arch/s390/kvm/gmap.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/s390/kvm/gmap.h +1 -1
@@ -273,11 +273,11 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, uniogmap_unmap_prefix (gmap, gfn, gfn + align); }if (crste_leaf (oldcrste) &&crste_needs_unshadow (oldcrste, newcrste)) { - newcrste.s.fc1.vsie_notif =0 ;if (needs_lock)gmap_handle_vsie_unshadow_event (gmap, gfn);else _gmap_handle_vsie_unshadow_event (gmap, gfn); +return false; }if (!oldcrste.s.fc1.d && newcrste.s.fc1.d && !newcrste.s.fc1.s)SetPageDirty (phys_to_page (crste_origin_large (newcrste)));
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v1 1/7] KVM: s390: Fix _gmap_crstep_xchg_atomic()
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Returning false without modifying memory breaks the compare-and-exchange contract, causing an…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Returning false without modifying memory breaks the compare-and-exchange contract, causing an infinite loop in retry loops.
- [High] Returning false on unshadow events bypasses the atomic exchange and triggers a KVM_BUG_ON during page unmapping, crashing the VM. --
commit 78e417232a11beab5e106b0965a752bf2abb9761
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Fix _gmap_crstep_xchg_atomic()
This commit modifies _gmap_crstep_xchg_atomic() to return false and skip the operation if an unshadow event is triggered. This prevents shadow crstes from being installed without the vsie_notif bit.
Quoted 5 lines
arch/s390/kvm/gmap.h +0
-0
arch/s390/kvm/gmap.h static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, unio · line 280
Quoted 9 lines
arch/s390/kvm/gmap.h +1
-1@@ -273,11 +273,11 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, uniogmap_unmap_prefix (gmap, gfn, gfn + align); }if (crste_leaf (oldcrste) &&crste_needs_unshadow (oldcrste, newcrste)) { - newcrste.s.fc1.vsie_notif =0 ;if (needs_lock)gmap_handle_vsie_unshadow_event (gmap, gfn);else _gmap_handle_vsie_unshadow_event (gmap, gfn); +return false;
[Severity: High]
Does returning false here break the compare-and-exchange contract?
Functions like _gmap_handle_minor_crste_fault() and gmap_sync_dirty_log() use a retry loop to atomically update page tables:
do {
old = READ_ONCE(*table);
...
} while (!gmap_crstep_xchg_atomic(...));By unconditionally returning false after handling an unshadow event, the actual dat_crstep_xchg_atomic() memory update is bypassed and the vsie_notif bit is never cleared from the CRSTE in memory.
Could this cause the retry loop to read the exact same unmodified CRSTE, trigger the unshadow path again, receive false again, and spin indefinitely without making forward progress? This might allow a guest to trigger a host CPU soft lockup.
[Severity: High]
Can returning false on unshadow events also trigger a KVM_BUG_ON() during page unmapping?
In _gmap_unmap_crste(), the code attempts to unmap a page by replacing its CRSTE with an empty entry _CRSTE_EMPTY(old.h.tt) which sets the invalid bit (h.i = 1).
If the page being unmapped is shadowed (old.s.fc1.vsie_notif == 1), crste_needs_unshadow() evaluates to true because newcrste.h.i is 1.
Would the modified _gmap_crstep_xchg_atomic() then unconditionally return false instead of performing the update, causing the enclosing KVM_BUG_ON(!gmap_crstep_xchg_atomic(...)) assertion to fail and crash the VM during a valid unmap operation?
Quoted 3 lines
arch/s390/kvm/gmap.h +0
-0}if (!oldcrste.s.fc1.d && newcrste.s.fc1.d && !newcrste.s.fc1.s)SetPageDirty (phys_to_page (crste_origin_large (newcrste)));
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Claudio Imbrenda
maintainerPATCH 2/7 KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
In reply to Claudio ImbrendaUntil now, gmap_helper_zap_one_page() was being called with the guest absolute address, but it expects a userspace virtual address.
This meant that in the best case the requested pages were not being discarded, and in the worst case that the wrong pages were being discarded.
Fix this by converting the guest absolute address to host virtual before passing it to gmap_helper_zap_one_page().
--- arch/s390/kvm/priv.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/s390/kvm/priv.c +6 -2
@@ -1188,6 +1188,7 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenunion crste *crstep;union pgste pgste;union pte *ptep; +hva_t hva;int i;lockdep_assert_held (&vcpu->kvm->mmu_lock);
@@ -1199,8 +1200,11 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenif (!ptep || ptep->s.pr)continue ; pgste =pgste_get_lock (ptep); -if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) -gmap_helper_zap_one_page (vcpu->kvm->mm, cbrl[i]); +if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) { + hva =gpa_to_hva (vcpu->kvm, cbrl[i]); +if (!kvm_is_error_hva (hva)) +gmap_helper_zap_one_page (vcpu->kvm->mm, hva); + }pgste_set_unlock (ptep, pgste); } }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v1 2/7] KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Sleeping in Atomic Context / Denial of Service --
commit 199d1c4ea9004cfaa67184f0dc232aee9284367d
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
This commit fixes a bug where gmap_helper_zap_one_page() was called with a guest absolute address instead of a host virtual address, causing pages to not be discarded or the wrong pages to be discarded. It converts the guest absolute address to a host virtual address before passing it to the helper.
Quoted 4 lines
arch/s390/kvm/priv.c +0
-0
[ ... ]
Quoted 9 lines
arch/s390/kvm/priv.c +1
-0@@ -1188,6 +1188,7 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenunion crste *crstep;union pgste pgste;union pte *ptep; +hva_t hva;int i;lockdep_assert_held (&vcpu->kvm->mmu_lock);
arch/s390/kvm/priv.c static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int len · line 1206
Quoted 9 lines
arch/s390/kvm/priv.c +4
-2@@ -1199,8 +1200,11 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenif (!ptep || ptep->s.pr)continue ; pgste =pgste_get_lock (ptep); -if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) -gmap_helper_zap_one_page (vcpu->kvm->mm, cbrl[i]); +if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) { + hva =gpa_to_hva (vcpu->kvm, cbrl[i]); +if (!kvm_is_error_hva (hva)) +gmap_helper_zap_one_page (vcpu->kvm->mm, hva);
[Severity: High]
This is a pre-existing issue and was not introduced by the patch under review, but does fixing the address resolution here unmask a sleeping-in-atomic bug?
Looking at _essa_clear_cbrl(), it executes while holding vcpu->kvm->mmu_lock and acquires a spinning bitlock via pgste_get_lock(ptep) right before calling gmap_helper_zap_one_page().
Inside gmap_helper_zap_one_page(), get_locked_pte() is called:
arch/s390/mm/gmap_helpers.c:gmap_helper_zap_one_page() {
...
ptep = get_locked_pte(mm, vmaddr, &ptl);
...
}get_locked_pte() attempts to walk the page table. If intermediate page tables are missing, it can perform sleepable memory allocations.
Sleeping while holding a spinlock or atomic bitlock causes a scheduling while atomic panic. Previously, passing a guest physical address instead of a host virtual address caused vma_lookup() in gmap_helper_zap_one_page() to return NULL before any sleeping operations occurred. With the address resolution fixed, can a guest trigger this path and cause a host denial of service?
I see this is addressed by a later commit in the series (KVM: s390:
Avoid potentially sleeping while atomic when zapping pages).
Quoted 4 lines
arch/s390/kvm/priv.c +1
-0+ }pgste_set_unlock (ptep, pgste); } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Claudio Imbrenda
maintainerPATCH 3/7 KVM: s390: vsie: Fix rmap handling in _do_shadow_crste()
In reply to Claudio ImbrendaFix _do_shadow_crste() to also apply a mask on the reverse address, to prevent spurious entries from being created, like already done in gmap_protect_rmap().
--- arch/s390/kvm/gaccess.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
arch/s390/kvm/gaccess.c +6 -5
@@ -1466,15 +1466,17 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, uni struct guest_fault *f, bool p) {union crste newcrste, oldcrste; -gfn_t gfn; +unsigned long mask; +gfn_t r_gfn;int rc;lockdep_assert_held (&sg->kvm->mmu_lock);lockdep_assert_held (&sg->parent->children_lock); - gfn = f->gfn & (is_pmd (*table) ? _SEGMENT_FR_MASK : _REGION3_FR_MASK); + mask =is_pmd (*table) ? _SEGMENT_FR_MASK : _REGION3_FR_MASK; + r_gfn =gpa_to_gfn (raddr) & mask;scoped_guard (spinlock, &sg->host_to_rmap_lock) - rc = gmap_insert_rmap(sg, gfn, gpa_to_gfn(raddr), host->h.tt); + rc = gmap_insert_rmap(sg, f->gfn & mask, r_gfn, host->h.tt);if (rc)return rc;
@@ -1497,8 +1499,7 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, unireturn -EAGAIN ; newcrste =_crste_fc1 (f->pfn, oldcrste.h.tt,0 , !p); - gfn =gpa_to_gfn (raddr); -while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, gfn, sg->asce)) +while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, r_gfn, sg->asce)) ;return 0 ; }
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH 4/7 KVM: s390: Fix fault-in code
In reply to Claudio ImbrendaFix the fault-in code so that it does not return success if a concurrent unmap event invalidated the fault-in process between the best-effort lockless check and the proper check with lock.
The new behaviour is to retry, like the best-effort lockless check already did.
This prevents the fault-in handler from returning success without having actually faulted in the requested page.
--- arch/s390/kvm/faultin.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
arch/s390/kvm/faultin.c +6 -7
@@ -36,7 +36,7 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fastruct kvm_s390_mmu_cache *mc =NULL ;struct kvm_memory_slot *slot;unsigned long inv_seq; -int foll, rc =0 ; +int foll, rc = -EAGAIN ; foll = f->write_attempt ?FOLL_WRITE :0 ; foll |= f->attempt_pfault ?FOLL_NOWAIT :0 ;
@@ -53,7 +53,7 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fareturn 0 ; } -while (1 ) { +while (rc == -EAGAIN ) { f->valid = false; inv_seq = kvm->mmu_invalidate_seq;/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
@@ -110,20 +110,19 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_faif (!mmu_invalidate_retry_gfn (kvm, inv_seq, f->gfn)) { f->valid = true; rc =gmap_link (mc, kvm->arch.gmap, f, slot); -kvm_release_faultin_page (kvm, f->page, !!rc, f->write_attempt); - f->page =NULL ; } +kvm_release_faultin_page (kvm, f->page, !!rc, f->write_attempt); } -kvm_release_faultin_page (kvm, f->page, true, false);if (rc == -ENOMEM ) { rc =kvm_s390_mmu_cache_topup (mc);if (rc)return rc; - }else if (rc != -EAGAIN ) { -return rc; + rc = -EAGAIN ; } } + +return rc; }int kvm_s390_get_guest_page (struct kvm *kvm,struct guest_fault *f,gfn_t gfn,bool w)
Signature · 1 line
2.54.0
Re: [PATCH v1 4/7] KVM: s390: Fix fault-in code
Quoted 10 lines · Claudio Imbrenda
Fix the fault-in code so that it does not return success if a concurrent unmap event invalidated the fault-in process between the best-effort lockless check and the proper check with lock.
The new behaviour is to retry, like the best-effort lockless check already did.
This prevents the fault-in handler from returning success without having actually faulted in the requested page.
One nit below.
Quoted 11 lines
Fixes e907ae530133 ("KVM: s390: Add helper functions for fault handling")Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kvm/faultin.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)arch/s390/kvm/faultin.c +0
-0
arch/s390/kvm/faultin.c int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fa · line 39
Quoted 5 lines
arch/s390/kvm/faultin.c +1
-1@@ -36,7 +36,7 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fastruct kvm_s390_mmu_cache *mc =NULL ;struct kvm_memory_slot *slot;unsigned long inv_seq; -int foll, rc =0 ; +int foll, rc = -EAGAIN ;
This is not in reverse Christmas tree anymore.
Split them into two lines.
...
Steffen
Claudio Imbrenda
maintainerPATCH 5/7 KVM: s390: Avoid potentially sleeping while atomic when zapping pages
In reply to Claudio ImbrendaFactor out try_get_locked_pte(), which behaves similarly to get_locked_pte(), but does not attempt to allocate missing tables and performs a spin_trylock() instead of blocking.
The new function is also exported, since it will be used in other patches.
If intermediate entries are missing, there can be no pte swap entry to free, so it's safe to ignore them.
This avoids potentially sleeping while atomic.
--- arch/s390/include/asm/gmap_helpers.h | 1 + arch/s390/mm/gmap_helpers.c | 111 ++++++++++++++++----------- 2 files changed, 68 insertions(+), 44 deletions(-)
arch/s390/include/asm/gmap_helpers.h +1 -0
@@ -12,5 +12,6 @@ void gmap_helper_zap_one_page(struct mm_struct *mm, unsigned long vmaddr);void gmap_helper_discard (struct mm_struct *mm,unsigned long vmaddr,unsigned long end);int gmap_helper_disable_cow_sharing (void );void gmap_helper_try_set_pte_unused (struct mm_struct *mm,unsigned long vmaddr); +pte_t *try_get_locked_pte (struct mm_struct *mm,unsigned long addr,spinlock_t **ptl);#endif /* _ASM_S390_GMAP_HELPERS_H */
arch/s390/mm/gmap_helpers.c +67 -44
145 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v1 5/7] KVM: s390: Avoid potentially sleeping while atomic when zapping pages
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] gmap_helper_zap_one_page silently skips discarding swapped pages when the page table lock is…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] gmap_helper_zap_one_page silently skips discarding swapped pages when the page table lock is contended, leaking memory and breaking guest discard semantics. --
commit cefa85bd369123cca902b4376ee79221b7451b20
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Avoid potentially sleeping while atomic when zapping pages
This commit factors out try_get_locked_pte() which performs a spin_trylock() and avoids blocking page table allocations. It modifies gmap page zapping functions to use this non-blocking helper.
Quoted 4 lines
arch/s390/mm/gmap_helpers.c +0
-0
[ ... ]
arch/s390/mm/gmap_helpers.c void gmap_helper_zap_one_page(struct mm_struct *mm, unsigned long vmaddr) · line 122
Quoted 8 lines
arch/s390/mm/gmap_helpers.c +1
-1@@ -57,7 +117,7 @@ void gmap_helper_zap_one_page(struct mm_struct *mm, unsigned long vmaddr)return ;/* Get pointer to the page table entry */ - ptep =get_locked_pte (mm, vmaddr, &ptl); + ptep =try_get_locked_pte (mm, vmaddr, &ptl);if (unlikely (!ptep))return ;
[Severity: Critical]
Does this silent return on lock contention create a guest data corruption issue or swap leak?
Because try_get_locked_pte() uses spin_trylock() and returns NULL when the page table lock is contended, this code will now silently skip discarding the swapped page if there is concurrent activity on the same PMD.
When a guest uses the ESSA instruction to discard a page, KVM marks it as logically zero. If the swap entry is never zapped due to lock contention, the guest might query the state and see it as zero, using it without initialization.
However, a subsequent access by the guest will cause the host page fault handler to fetch the old data from the swap device. KVM maps this non-zero data into the guest, completely violating the guarantee to supply a zeroed page.
Is there a fallback mechanism to handle the retry when try_get_locked_pte() fails due to contention rather than a missing page table?
Quoted 3 lines
arch/s390/mm/gmap_helpers.c +0
-0if (pte_swap (*ptep)) {arch/s390/mm/gmap_helpers.c +0
-0ptep_zap_softleaf_entry (mm,softleaf_from_pte (*ptep));pte_clear (mm, vmaddr, ptep);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Claudio Imbrenda
maintainerPATCH 6/7 KVM: s390: Lock pte when making page secure
In reply to Claudio ImbrendaMake sure _kvm_s390_pv_make_secure() takes the pte lock for the given address when attempting to make the page secure.
One of the steps in making the page secure is freezing the folio using folio_ref_freeze(), which temporarily sets the reference count to 0. Any attempt to get such a folio while frozen will fail and cause a warning to be printed.
Other users of folio_ref_freeze() make sure that the page is not mapped while it's being frozen, thus preventing gup functions from being able to access it. For _kvm_s390_pv_make_secure(), this is not possible, because the page needs to be mapped in order for the import to succeed.
By taking the pte lock, gup functions will be blocked until the import operation is done, thus avoiding the race.
--- arch/s390/kvm/pv.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
arch/s390/kvm/pv.c +12 -3
@@ -17,6 +17,7 @@#include <linux/pagewalk.h> #include <linux/sched/mm.h> #include <linux/mmu_notifier.h> +#include <asm/gmap_helpers.h> #include "kvm-s390.h" #include "dat.h" #include "gaccess.h"
@@ -73,6 +74,7 @@ static bool should_export_before_import(struct uv_cb_header *uvcb, struct mm_strstruct pv_make_secure {void *uvcb;struct folio *folio; +struct kvm *kvm;int rc;bool needs_export; };
@@ -103,17 +105,24 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f) {struct pv_make_secure *priv = f->priv;struct folio *folio; +spinlock_t *ptl;/* pte lock from try_get_locked_pte() */ +pte_t *ptep; folio =pfn_folio (f->pfn); priv->rc = -EAGAIN ; -if (folio_trylock (folio)) { +if (!folio_trylock (folio)) +return ; + + ptep =try_get_locked_pte (priv->kvm->mm,gfn_to_hva (priv->kvm, f->gfn), &ptl); +if (ptep) { priv->rc =__kvm_s390_pv_make_secure (f, folio);if (priv->rc == -E2BIG || priv->rc == -EBUSY ) { priv->folio = folio;folio_get (folio); } -folio_unlock (folio); +pte_unmap_unlock (ptep, ptl); } +folio_unlock (folio); }/**
@@ -127,7 +136,7 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f)*/ int kvm_s390_pv_make_secure (struct kvm *kvm,unsigned long gaddr,void *uvcb) { -struct pv_make_secure priv = { .uvcb = uvcb }; +struct pv_make_secure priv = { .uvcb = uvcb, .kvm = kvm, };struct guest_fault f = { .write_attempt = true, .gfn =gpa_to_gfn (gaddr),
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v1 6/7] KVM: s390: Lock pte when making page secure
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
New issues:
- [High] Lockless host page table walk without mmap_lock or rcu_read_lock can lead to a use-after-free.
- [High] Executing a polling hypercall with preemption disabled creates a soft lockup hazard.
- [High] Infinite loop (livelock) when guest memory is backed by Transparent Huge Pages (THP).
Pre-existing issues:
- [Critical] _kvm_s390_pv_make_secure blindly casts KVM-provided PFNs to a folio without validating whether the PFN corresponds to a refcounted struct page.
- [High] Holding a single PTE lock to prevent concurrent GUP during folio_ref_freeze() is insufficient if the folio is mapped by multiple PTEs. --
commit 1e05383f44b9d07a44357c266f26898f02b36e27
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Lock pte when making page secure
Make sure _kvm_s390_pv_make_secure() takes the pte lock for the given address when attempting to make the page secure.
One of the steps in making the page secure is freezing the folio using folio_ref_freeze(), which temporarily sets the reference count to 0. Any attempt to get such a folio while frozen will fail and cause a warning to be printed.
Other users of folio_ref_freeze() make sure that the page is not mapped while it's being frozen, thus preventing gup functions from being able to access it. For _kvm_s390_pv_make_secure(), this is not possible, because the page needs to be mapped in order for the import to succeed.
Quoted 2 lines
By taking the pte lock, gup functions will be blocked until the import operation is done, thus avoiding the race.
[Severity: High]
This isn't a bug introduced by this patch, but is holding a single PTE lock sufficient if the folio is mapped by multiple PTEs?
Userspace can create memory aliases (e.g., via mremap with MREMAP_DONTUNMAP or multiple mmaps of the same memfd/shmem). If a concurrent thread triggers GUP on an alias mapping, it would acquire a separate PTE lock.
Could that thread then observe the refcount of 0 temporarily set by KVM's folio_ref_freeze(), triggering the exact WARN_ON_ONCE this locking strategy aims to prevent?
Quoted 7 lines
Signed-off-by Claudio Imbrenda <[email protected]>Fixes e38c884df921 ("KVM: s390: Switch to new gmap")arch/s390/kvm/pv.c +0
-0
[ ... ]
arch/s390/kvm/pv.c static void _kvm_s390_pv_make_secure(struct guest_fault *f) · line 111
Quoted 8 lines
arch/s390/kvm/pv.c +2
-0@@ -103,17 +105,24 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f) {struct pv_make_secure *priv = f->priv;struct folio *folio; +spinlock_t *ptl;/* pte lock from try_get_locked_pte() */ +pte_t *ptep; folio =pfn_folio (f->pfn);
[Severity: Critical]
This is a pre-existing issue, but is it safe to unconditionally cast f->pfn to a folio here without checking if f->page is valid?
If a guest issues a secure conversion request on memory backed by un-refcounted device memory (VM_PFNMAP), __kvm_faultin_pfn() resolves the PFN but leaves f->page as NULL.
Could this cause folio_trylock(folio) below to access invalid memory and crash the kernel?
arch/s390/kvm/pv.c static void _kvm_s390_pv_make_secure(struct guest_fault *f) · line 116
Quoted 6 lines
arch/s390/kvm/pv.c +4
-1priv->rc = -EAGAIN ; -if (folio_trylock (folio)) { +if (!folio_trylock (folio)) +return ; + + ptep =try_get_locked_pte (priv->kvm->mm,gfn_to_hva (priv->kvm, f->gfn), &ptl);
[Severity: High]
Does this call to try_get_locked_pte() require mmap_lock or rcu_read_lock to be held?
_kvm_s390_pv_make_secure() is called from kvm_s390_faultin_gfn(), which is documented as running without the mm lock.
Could a concurrent munmap() free the page tables while they are being walked by try_get_locked_pte(), leading to a use-after-free?
arch/s390/kvm/pv.c static void _kvm_s390_pv_make_secure(struct guest_fault *f) · line 117
Quoted 1 line
arch/s390/kvm/pv.c +1
-0+if (ptep) {
[Severity: High]
What happens here if the guest memory is backed by Transparent Huge Pages (THP)?
try_get_locked_pte() returns NULL for large PMD-mapped pages. If it returns NULL, this block is skipped entirely, meaning -E2BIG is never returned and priv->folio is not set.
Will this bypass the fallback logic in kvm_s390_pv_make_secure() that calls s390_wiggle_split_folio(), causing a permanent livelock that repeatedly returns -EAGAIN?
arch/s390/kvm/pv.c static void _kvm_s390_pv_make_secure(struct guest_fault *f) · line 118
Quoted 1 line
arch/s390/kvm/pv.c +0
-0priv->rc =__kvm_s390_pv_make_secure (f, folio);
[Severity: High]
Could calling __kvm_s390_pv_make_secure() while holding the PTE spinlock create a soft lockup hazard?
If priv->needs_export is true, the code calls uv_convert_from_secure(), which eventually issues the Ultravisor hypercall via uv_call().
Does this hypercall execute an unbounded polling loop without cpu_relax() or cond_resched() while preemption is disabled by the pte lock?
Quoted 9 lines
arch/s390/kvm/pv.c +2
-1if (priv->rc == -E2BIG || priv->rc == -EBUSY ) { priv->folio = folio;folio_get (folio); } -folio_unlock (folio); +pte_unmap_unlock (ptep, ptl); } +folio_unlock (folio); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Claudio Imbrenda
maintainerPATCH 7/7 KVM: s390: Prevent memslots outside the ASCE range
In reply to Claudio ImbrendaWith KVM_S390_VM_MEM_LIMIT_SIZE, userspace can set the highest address allowed for the VM. Creating a memslot that lies over the maximum address does not make sense and is only a potential source of bugs.
Prevent creation of memslots over the maximum address, and prevent the maximum address from being reduced below the end of existing memslots.
--- arch/s390/kvm/kvm-s390.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)
arch/s390/kvm/kvm-s390.c +22 -2
@@ -1015,8 +1015,26 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *attreturn -EINVAL ; ret = -EBUSY ; -if (!kvm->created_vcpus) - ret =gmap_set_limit (kvm->arch.gmap,gpa_to_gfn (new_limit)); +if (!kvm->created_vcpus) { +struct kvm_memslots *slots; +struct kvm_memory_slot *ms; +int bkt; + + ret =0 ; +mutex_lock (&kvm->slots_arch_lock); + slots =kvm_memslots (kvm); +if (slots && !kvm_memslots_empty (slots)) { +kvm_for_each_memslot (ms, bkt, slots) { +if (gpa_to_gfn (new_limit) < ms->base_gfn + ms->npages) { + ret = -EBUSY ; +break ; + } + } + } +if (!ret) + ret =gmap_set_limit (kvm->arch.gmap,gpa_to_gfn (new_limit)); +mutex_unlock (&kvm->slots_arch_lock); + }VM_EVENT (kvm,3 ,"SET: max guest address: %lu" , new_limit);VM_EVENT (kvm,3 ,"New guest asce: 0x%p" , (void *)kvm->arch.gmap->asce.val);
@@ -5672,6 +5690,8 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,return -EINVAL ;if ((new->base_gfn + new->npages) *PAGE_SIZE > kvm->arch.mem_limit)return -EINVAL ; +if (!asce_contains_gfn (kvm->arch.gmap->asce, new->base_gfn + new->npages -1 )) +return -EINVAL ; }if (!kvm->arch.migration_mode)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v1 7/7] KVM: s390: Prevent memslots outside the ASCE range
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [High] Accessing kvm_memslots(kvm) without holding kvm->slots_lock or the KVM SRCU read lock triggers an unconditional lockdep warning.
- [Medium] Asymmetrical memory limit checks can cause false-positive -EBUSY errors when re-applying the exact same memory limit.
Pre-existing issues:
- [Critical] A data race on kvm->created_vcpus allows the memory limit to be modified concurrently with VCPU creation and execution, leading to hardware use-after-free or host panic. --
commit d947d07a1de4770a70d881cc020678eb270baac3
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Prevent memslots outside the ASCE range
This commit introduces checks to prevent creating memslots beyond the configured maximum guest address and prevents lowering the limit below the highest existing memslot.
arch/s390/kvm/kvm-s390.c static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att · line 1018
Quoted 11 lines
arch/s390/kvm/kvm-s390.c +1
-2@@ -1015,8 +1015,26 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *attreturn -EINVAL ; ret = -EBUSY ; -if (!kvm->created_vcpus) - ret =gmap_set_limit (kvm->arch.gmap,gpa_to_gfn (new_limit)); +if (!kvm->created_vcpus) {
[Severity: Critical]
This is a pre-existing issue, but is it safe to check kvm->created_vcpus locklessly here?
A concurrent KVM_CREATE_VCPU ioctl, which holds kvm->lock, could increment created_vcpus and allow the newly created VCPU to start running via KVM_RUN. If that happens, gmap_set_limit() might mutate the gmap->asce value concurrently with the VCPU running in SIE.
Because gmap_set_limit() can free top-level page tables, the running VCPU could read a torn asce.val or continue using the old ASCE while its tables are freed. Could this block be protected by kvm->lock, similar to how KVM_S390_VM_MEM_ENABLE_CMMA handles it?
arch/s390/kvm/kvm-s390.c static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att · line 1025
Quoted 7 lines
arch/s390/kvm/kvm-s390.c +7
-0+struct kvm_memslots *slots; +struct kvm_memory_slot *ms; +int bkt; + + ret =0 ; +mutex_lock (&kvm->slots_arch_lock); + slots =kvm_memslots (kvm);
[Severity: High]
Does this trigger a lockdep warning?
The kvm_memslots() accessor invokes srcu_dereference_check(), which expects either kvm->slots_lock or the KVM SRCU read lock to be held. Since only slots_arch_lock is acquired here, an unprivileged user calling KVM_S390_VM_MEM_LIMIT_SIZE might deterministically hit a lockdep assertion.
arch/s390/kvm/kvm-s390.c static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att · line 1028
Quoted 3 lines
arch/s390/kvm/kvm-s390.c +3
-0+if (slots && !kvm_memslots_empty (slots)) { +kvm_for_each_memslot (ms, bkt, slots) { +if (gpa_to_gfn (new_limit) < ms->base_gfn + ms->npages) {
[Severity: Medium]
Will this check cause false-positive -EBUSY errors if the exact same memory limit is re-applied?
KVM_SET_USER_MEMORY_REGION allows creating memslots up to the rounded hardware ASCE limit. If a user sets a limit and creates a memslot that gets rounded up beyond the requested limit (which succeeds), and then tries to re-apply the same limit, it appears this check would evaluate to true and fail.
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +9
-0+ ret = -EBUSY ; +break ; + } + } + } +if (!ret) + ret =gmap_set_limit (kvm->arch.gmap,gpa_to_gfn (new_limit)); +mutex_unlock (&kvm->slots_arch_lock); + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7