KVM: s390: More gmap and vsie fixes
Claudio Imbrenda
maintainerv2 cover KVM: s390: More gmap and vsie fixes
Another batch of fixups for gmap and vsie. Some minor fixes, some not-so-minor fixes that could have caused guest corruption under particular circumstances.
v1->v2:
* Reordered the patches
* _gmap_crstep_xchg_atomic() will now attempt to clear the vsie_notif bit when failing due to unshadowing; this prevents potential loops
* try_get_locked_pte() will now return -EAGAIN instead of NULL if the pte was reached but the lock was contended; this prevents potential loops
* _kvm_s390_pv_make_secure() will now attempt mmap_read_trylock(); this prevents try_get_locked_pte() from potentially racing
* Take kvm->slots_lock instead of kvm->slots_arch_lock in kvm_s390_set_mem_control() when handling KVM_S390_VM_MEM_LIMIT_SIZE, and also take kvm->lock.
* Minor cosmetic / style fixes.
Claudio Imbrenda (8):
KVM: s390: Fix _gmap_unmap_crste()
KVM: s390: Fix _gmap_crstep_xchg_atomic()
KVM: s390: Avoid potentially sleeping while atomic when zapping pages
KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
KVM: s390: vsie: Fix rmap handling in _do_shadow_crste()
KVM: s390: Fix fault-in code
KVM: s390: Lock pte when making page secure
KVM: s390: Prevent memslots outside the ASCE range
arch/s390/include/asm/gmap_helpers.h | 1 + arch/s390/kvm/faultin.c | 14 ++-- arch/s390/kvm/gaccess.c | 11 +-- arch/s390/kvm/gmap.c | 19 ++++- arch/s390/kvm/gmap.h | 3 + arch/s390/kvm/kvm-s390.c | 30 ++++++- arch/s390/kvm/priv.c | 8 +- arch/s390/kvm/pv.c | 21 ++++- arch/s390/mm/gmap_helpers.c | 117 ++++++++++++++++----------- 9 files changed, 157 insertions(+), 67 deletions(-)
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH v2 1/8 KVM: s390: Fix _gmap_unmap_crste()
In _gmap_unmap_crste(), the crste to be unmapped is zapped calling gmap_crstep_xchg_atomic() exactly once, and expecting it to succeed. This is a reasonable sanity check, since kvm->mmu_lock is being held in write mode, and thus no races should be possible.
An upcoming patch will change the behaviour of gmap_crstep_xchg_atomic() to return false and clear the vsie_notif bit if the operation triggers an unshadow operation. With the new behaviour, an unmap operation that triggers an unshadow would cause the VM to be killed.
Prepare for the change by checking if the vsie_notif bit was set in the old crste if gmap_crstep_xchg_atomic() fails the first time, and try a second time. The second time no failures are allowed.
--- arch/s390/kvm/gmap.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
arch/s390/kvm/gmap.c +16 -3
@@ -395,15 +395,28 @@ static long _gmap_unmap_crste(union crste *crstep, gfn_t gfn, gfn_t next, structstruct gmap_unmap_priv *priv = walk->priv;struct folio *folio =NULL ;union crste old = *crstep; +bool ok;if (!old.h.fc)return 0 ;if (old.s.fc1.pr &&test_bit (GMAP_FLAG_EXPORT_ON_UNMAP , &priv->gmap->flags)) folio =phys_to_folio (crste_origin_large (old)); -/* No races should happen because kvm->mmu_lock is held in write mode */ -KVM_BUG_ON (!gmap_crstep_xchg_atomic (priv->gmap, crstep, old,_CRSTE_EMPTY (old.h.tt), gfn), - priv->gmap->kvm); +/* +* No races should happen because kvm->mmu_lock is held in write mode, +* but the unmap operation could have triggered an unshadow, which +* causes gmap_crstep_xchg_atomic() to return false and clear the +* vsie_notif bit. Allow the operation to fail once, if the old crste +* had the vsie_notif bit set. A second failure is not allowed, for +* the reasons above. +*/ + ok =gmap_crstep_xchg_atomic (priv->gmap, crstep, old,_CRSTE_EMPTY (old.h.tt), gfn); +if (!ok) { +KVM_BUG_ON (!old.s.fc1.vsie_notif, priv->gmap->kvm); + old.s.fc1.vsie_notif =0 ; + ok =gmap_crstep_xchg_atomic (priv->gmap, crstep, old,_CRSTE_EMPTY (old.h.tt), gfn); +KVM_BUG_ON (!ok, priv->gmap->kvm); + }if (folio)uv_convert_from_secure_folio (folio);
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH v2 2/8 KVM: s390: Fix _gmap_crstep_xchg_atomic()
In reply to Claudio ImbrendaThe previous incorrect behaviour cleared the vsie_notif bit without returning false, which allowed shadow crstes to be installed without the vsie_notif bit.
Return false and do not perform the operation if an unshadow event has been triggered, but still attempt to clear the vsie_notif bit from the existing crste.
This will prevent the installation of shadow crstes without vsie_notif bit and will also prevent the caller from looping forever if it was not checking for the sg->invalidated flag.
--- arch/s390/kvm/gmap.h | 3 +++ 1 file changed, 3 insertions(+)
arch/s390/kvm/gmap.h +3 -0
@@ -273,11 +273,14 @@ static inline bool __must_check _gmap_crstep_xchg_atomic(struct gmap *gmap, uniogmap_unmap_prefix (gmap, gfn, gfn + align); }if (crste_leaf (oldcrste) &&crste_needs_unshadow (oldcrste, newcrste)) { + newcrste = oldcrste; newcrste.s.fc1.vsie_notif =0 ;if (needs_lock)gmap_handle_vsie_unshadow_event (gmap, gfn);else _gmap_handle_vsie_unshadow_event (gmap, gfn); +dat_crstep_xchg_atomic (crstep, oldcrste, newcrste, gfn, gmap->asce); +return false; }if (!oldcrste.s.fc1.d && newcrste.s.fc1.d && !newcrste.s.fc1.s)SetPageDirty (phys_to_page (crste_origin_large (newcrste)));
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH v2 3/8 KVM: s390: Avoid potentially sleeping while atomic when zapping pages
In reply to Claudio ImbrendaFactor out try_get_locked_pte(), which behaves similarly to get_locked_pte(), but does not attempt to allocate missing tables and performs a spin_trylock() instead of blocking.
The new function is also exported, since it will be used in other patches.
If intermediate entries are missing, there can be no pte swap entry to free, so it's safe to ignore them.
This avoids potentially sleeping while atomic.
--- arch/s390/include/asm/gmap_helpers.h | 1 + arch/s390/mm/gmap_helpers.c | 117 ++++++++++++++++----------- 2 files changed, 73 insertions(+), 45 deletions(-)
arch/s390/include/asm/gmap_helpers.h +1 -0
@@ -12,5 +12,6 @@ void gmap_helper_zap_one_page(struct mm_struct *mm, unsigned long vmaddr);void gmap_helper_discard (struct mm_struct *mm,unsigned long vmaddr,unsigned long end);int gmap_helper_disable_cow_sharing (void );void gmap_helper_try_set_pte_unused (struct mm_struct *mm,unsigned long vmaddr); +pte_t *try_get_locked_pte (struct mm_struct *mm,unsigned long addr,spinlock_t **ptl);#endif /* _ASM_S390_GMAP_HELPERS_H */
arch/s390/mm/gmap_helpers.c +72 -45
151 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 3/8] KVM: s390: Avoid potentially sleeping while atomic when zapping pages
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] ptep_zap_softleaf_entry unconditionally calls swap_put_entries_direct on non-present PTEs, which triggers warnings and fails when processing migration entries. --
commit 98730d8ed19b62efe3e43850095ef240bb137294
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Avoid potentially sleeping while atomic when zapping pages
This commit factors out try_get_locked_pte(), which performs a spin_trylock() instead of blocking and avoids allocating missing page tables. This new function is used to prevent sleeping while atomic when zapping pages.
arch/s390/mm/gmap_helpers.c static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry) · line 34
Quoted 6 lines
arch/s390/mm/gmap_helpers.c +0
-0@@ -34,6 +34,70 @@ static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry)swap_put_entries_direct (entry,1 );
[Severity: High]
This isn't a bug introduced by this patch, but does ptep_zap_softleaf_entry() incorrectly pass non-swap softleaf entries to generic swap reclaim code?
The function acts as a dispatch for non-present PTEs, handling both swap and migration entries, but unconditionally passes the entry to swap_put_entries_direct().
Migration entries use swap types greater than MAX_SWAPFILES. When swap_put_entries_direct() passes this to get_swap_device(), it will fail, printing a 'Bad swap file entry' error to the kernel log and hitting a WARN_ON_ONCE(!si).
Because this path is reachable by a guest issuing ESSA instructions on memory regions undergoing host-side migration, could a malicious guest intentionally trigger this path to spam the host's dmesg logs on every migrated page, creating a host-side denial of service?
Quoted 2 lines
arch/s390/mm/gmap_helpers.c +0
-0}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Claudio Imbrenda
maintainerPATCH v2 4/8 KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
In reply to Claudio ImbrendaUntil now, gmap_helper_zap_one_page() was being called with the guest absolute address, but it expects a userspace virtual address.
This meant that in the best case the requested pages were not being discarded, and in the worst case that the wrong pages were being discarded.
Fix this by converting the guest absolute address to host virtual before passing it to gmap_helper_zap_one_page().
--- arch/s390/kvm/priv.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/s390/kvm/priv.c +6 -2
@@ -1188,6 +1188,7 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenunion crste *crstep;union pgste pgste;union pte *ptep; +hva_t hva;int i;lockdep_assert_held (&vcpu->kvm->mmu_lock);
@@ -1199,8 +1200,11 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenif (!ptep || ptep->s.pr)continue ; pgste =pgste_get_lock (ptep); -if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) -gmap_helper_zap_one_page (vcpu->kvm->mm, cbrl[i]); +if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) { + hva =gpa_to_hva (vcpu->kvm, cbrl[i]); +if (!kvm_is_error_hva (hva)) +gmap_helper_zap_one_page (vcpu->kvm->mm, hva); + }pgste_set_unlock (ptep, pgste); } }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 4/8] KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] ptep_zap_softleaf_entry unconditionally calls swap_put_entries_direct for all non-present PTEs, including migration and device private entries, leading to WARN_ON_ONCE and potential reference leaks. --
commit 9ec1cdb739d15e125c9d1bb7c4db08fb59c07068
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Fix guest / virtual address confusion in _essa_clear_cbrl()
This patch fixes a bug where gmap_helper_zap_one_page was being called with a guest absolute address instead of the expected host virtual address. It resolves the issue by converting the address to a host virtual address before performing the zap.
Quoted 4 lines
arch/s390/kvm/priv.c +0
-0
[ ... ]
arch/s390/kvm/priv.c static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int len · line 1206
Quoted 10 lines
arch/s390/kvm/priv.c +4
-2@@ -1199,8 +1200,11 @@ static void _essa_clear_cbrl(struct kvm_vcpu *vcpu, unsigned long *cbrl, int lenif (!ptep || ptep->s.pr)continue ; pgste =pgste_get_lock (ptep); -if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) -gmap_helper_zap_one_page (vcpu->kvm->mm, cbrl[i]); +if (pgste.usage ==PGSTE_GPS_USAGE_UNUSED || pgste.zero) { + hva =gpa_to_hva (vcpu->kvm, cbrl[i]); +if (!kvm_is_error_hva (hva)) +gmap_helper_zap_one_page (vcpu->kvm->mm, hva);
^^^^^^^^^^^^^^^^^^^^^^^^
[Severity: High]
This is a pre-existing issue, but does fixing the address passed to gmap_helper_zap_one_page expose a bug when encountering non-present PTEs?
Looking at gmap_helper_zap_one_page in arch/s390/mm/gmap_helpers.c, it dispatches all pte_swap entries to ptep_zap_softleaf_entry:
if (pte_swap(*ptep)) {
ptep_zap_softleaf_entry(mm, softleaf_from_pte(*ptep));
pte_clear(mm, vmaddr, ptep);
}In ptep_zap_softleaf_entry, swap_put_entries_direct is called unconditionally. For non-genuine swap entries like migration or device private entries, get_swap_device returns NULL, which causes swap_put_entries_direct to hit a WARN_ON_ONCE and return early.
Furthermore, for device private entries, the required folio teardown sequence (normally performed by zap_nonpresent_ptes) is completely skipped. Since pte_clear is then called, could this lead to a permanent folio reference leak in addition to triggering a kernel warning?
Quoted 4 lines
arch/s390/kvm/priv.c +1
-0+ }pgste_set_unlock (ptep, pgste); } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Claudio Imbrenda
maintainerPATCH v2 5/8 KVM: s390: vsie: Fix rmap handling in _do_shadow_crste()
In reply to Claudio ImbrendaFix _do_shadow_crste() to also apply a mask on the reverse address, to prevent spurious entries from being created, like already done in gmap_protect_rmap().
--- arch/s390/kvm/gaccess.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
arch/s390/kvm/gaccess.c +6 -5
@@ -1466,15 +1466,17 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, uni struct guest_fault *f, bool p) {union crste newcrste, oldcrste; -gfn_t gfn; +unsigned long mask; +gfn_t r_gfn;int rc;lockdep_assert_held (&sg->kvm->mmu_lock);lockdep_assert_held (&sg->parent->children_lock); - gfn = f->gfn & (is_pmd (*table) ? _SEGMENT_FR_MASK : _REGION3_FR_MASK); + mask =is_pmd (*table) ? _SEGMENT_FR_MASK : _REGION3_FR_MASK; + r_gfn =gpa_to_gfn (raddr) & mask;scoped_guard (spinlock, &sg->host_to_rmap_lock) - rc = gmap_insert_rmap(sg, gfn, gpa_to_gfn(raddr), host->h.tt); + rc = gmap_insert_rmap(sg, f->gfn & mask, r_gfn, host->h.tt);if (rc)return rc;
@@ -1497,8 +1499,7 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, unireturn -EAGAIN ; newcrste =_crste_fc1 (f->pfn, oldcrste.h.tt,0 , !p); - gfn =gpa_to_gfn (raddr); -while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, gfn, sg->asce)) +while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, r_gfn, sg->asce)) ;return 0 ; }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 5/8] KVM: s390: vsie: Fix rmap handling in _do_shadow_crste()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] _do_shadow_crste() creates a corrupted Region-3 large page entry (PUD) due to _crste_fc1() hardcoding _SEGMENT_MASK, leading to hardware Translation-Specification exceptions. --
commit 249b86f32c315315b0e5e5c41fd24415c3c08a7f
Author: Claudio Imbrenda <[email protected]>
KVM: s390: vsie: Fix rmap handling in _do_shadow_crste()
Fix _do_shadow_crste() to also apply a mask on the reverse address, to prevent spurious entries from being created, like already done in gmap_protect_rmap().
Quoted 4 lines
arch/s390/kvm/gaccess.c +0
-0
[ ... ]
arch/s390/kvm/gaccess.c static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, uni · line 1501
Quoted 4 lines
arch/s390/kvm/gaccess.c +0
-0@@ -1497,8 +1499,7 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, unireturn -EAGAIN ; newcrste =_crste_fc1 (f->pfn, oldcrste.h.tt,0 , !p);
[Severity: Critical]
This is a pre-existing issue, but does this create a corrupted Region-3 large page entry when dealing with 2GB large pages?
Looking at _crste_fc1() in arch/s390/kvm/dat.h, it hardcodes _SEGMENT_MASK:
arch/s390/kvm/dat.h:_crste_fc1() {
union crste res = { .val = PFN_PHYS(pfn) & _SEGMENT_MASK };
...
}If oldcrste.h.tt is TABLE_TYPE_REGION3, f->pfn may have non-zero bits in the 20-30 range (which map to IBM bits 33-43) if the fault was not at the start of the 2GB page. Since _SEGMENT_MASK only masks out the lower 20 bits, these bits remain set.
Will writing this entry with bits 20-30 set (which should be strictly zero per architecture) cause the hardware to raise a Translation-Specification exception?
Quoted 6 lines
arch/s390/kvm/gaccess.c +1
-2- gfn =gpa_to_gfn (raddr); -while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, gfn, sg->asce)) +while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, r_gfn, sg->asce)) ;return 0 ; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Claudio Imbrenda
maintainerPATCH v2 6/8 KVM: s390: Fix fault-in code
In reply to Claudio ImbrendaFix the fault-in code so that it does not return success if a concurrent unmap event invalidated the fault-in process between the best-effort lockless check and the proper check with lock.
The new behaviour is to retry, like the best-effort lockless check already did.
This prevents the fault-in handler from returning success without having actually faulted in the requested page.
--- arch/s390/kvm/faultin.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
arch/s390/kvm/faultin.c +7 -7
@@ -36,7 +36,8 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fastruct kvm_s390_mmu_cache *mc =NULL ;struct kvm_memory_slot *slot;unsigned long inv_seq; -int foll, rc =0 ; +int rc = -EAGAIN ; +int foll; foll = f->write_attempt ?FOLL_WRITE :0 ; foll |= f->attempt_pfault ?FOLL_NOWAIT :0 ;
@@ -53,7 +54,7 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fareturn 0 ; } -while (1 ) { +while (rc == -EAGAIN ) { f->valid = false; inv_seq = kvm->mmu_invalidate_seq;/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */
@@ -110,20 +111,19 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_faif (!mmu_invalidate_retry_gfn (kvm, inv_seq, f->gfn)) { f->valid = true; rc =gmap_link (mc, kvm->arch.gmap, f, slot); -kvm_release_faultin_page (kvm, f->page, !!rc, f->write_attempt); - f->page =NULL ; } +kvm_release_faultin_page (kvm, f->page, !!rc, f->write_attempt); } -kvm_release_faultin_page (kvm, f->page, true, false);if (rc == -ENOMEM ) { rc =kvm_s390_mmu_cache_topup (mc);if (rc)return rc; - }else if (rc != -EAGAIN ) { -return rc; + rc = -EAGAIN ; } } + +return rc; }int kvm_s390_get_guest_page (struct kvm *kvm,struct guest_fault *f,gfn_t gfn,bool w)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 6/8] KVM: s390: Fix fault-in code
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Memory leak of the pinned host page reference when kvm_s390_new_mmu_cache() allocation fails. --
commit 132617ab1e43f506af43740ee664fcdba7a95ff5
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Fix fault-in code
This commit fixes the fault-in code so that it does not return success if a concurrent unmap event invalidated the fault-in process. It ensures the code retries properly instead of returning success without actually faulting in the requested page.
Quoted 4 lines
arch/s390/kvm/faultin.c +0
-0
[ ... ]
arch/s390/kvm/faultin.c int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fa · line 116
Quoted 9 lines
arch/s390/kvm/faultin.c +1
-2@@ -110,20 +111,19 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_faif (!mmu_invalidate_retry_gfn (kvm, inv_seq, f->gfn)) { f->valid = true; rc =gmap_link (mc, kvm->arch.gmap, f, slot); -kvm_release_faultin_page (kvm, f->page, !!rc, f->write_attempt); - f->page =NULL ; } +kvm_release_faultin_page (kvm, f->page, !!rc, f->write_attempt); }
[Severity: High]
This isn't a bug introduced by this patch, but I have a question about how f->page is handled when kvm_s390_new_mmu_cache() fails earlier in this loop.
Earlier in kvm_s390_faultin_gfn() in arch/s390/kvm/faultin.c, __kvm_faultin_pfn() resolves the PFN and pins the host page, storing it in f->page.
Then, if mc is NULL, the code attempts to allocate a new cache:
if (!mc) {
local_mc = kvm_s390_new_mmu_cache();
if (!local_mc)
return -ENOMEM;
mc = local_mc;
}When kvm_s390_new_mmu_cache() fails and returns -ENOMEM, it looks like the function returns directly without calling kvm_release_faultin_page() to drop the previously acquired reference in f->page.
Callers like kvm_s390_faultin_gfn_simple() and access_guest_page_with_key_gpa() declare struct guest_fault f locally on the stack and propagate the -ENOMEM error without inspecting or cleaning up f.page.
Can this permanently leak the pinned host page? Under memory pressure, could an unprivileged guest repeatedly trigger page faults to accumulate leaked page references and exhaust host memory?
Quoted 14 lines
arch/s390/kvm/faultin.c +3
-3-kvm_release_faultin_page (kvm, f->page, true, false);if (rc == -ENOMEM ) { rc =kvm_s390_mmu_cache_topup (mc);if (rc)return rc; - }else if (rc != -EAGAIN ) { -return rc; + rc = -EAGAIN ; } } + +return rc; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Claudio Imbrenda
maintainerPATCH v2 7/8 KVM: s390: Lock pte when making page secure
In reply to Claudio ImbrendaMake sure _kvm_s390_pv_make_secure() takes the pte lock for the given address when attempting to make the page secure.
One of the steps in making the page secure is freezing the folio using folio_ref_freeze(), which temporarily sets the reference count to 0. Any attempt to get such a folio while frozen will fail and cause a warning to be printed.
Other users of folio_ref_freeze() make sure that the page is not mapped while it's being frozen, thus preventing gup functions from being able to access it. For _kvm_s390_pv_make_secure(), this is not possible, because the page needs to be mapped in order for the import to succeed.
By taking the pte lock, gup functions will be blocked until the import operation is done, thus avoiding the race.
In theory this does not completely solve the issue: if a page is mapped through multiple mappings, locking one pte does not protect from calling gup on it through the other mapping. In practice this does not happen and it is a decent stopgap solution until a more correct solution is available.
--- arch/s390/kvm/pv.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
arch/s390/kvm/pv.c +20 -1
@@ -17,6 +17,7 @@#include <linux/pagewalk.h> #include <linux/sched/mm.h> #include <linux/mmu_notifier.h> +#include <asm/gmap_helpers.h> #include "kvm-s390.h" #include "dat.h" #include "gaccess.h"
@@ -73,6 +74,7 @@ static bool should_export_before_import(struct uv_cb_header *uvcb, struct mm_strstruct pv_make_secure {void *uvcb;struct folio *folio; +struct kvm *kvm;int rc;bool needs_export; };
@@ -103,9 +105,21 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f) {struct pv_make_secure *priv = f->priv;struct folio *folio; +spinlock_t *ptl;/* pte lock from try_get_locked_pte() */ +pte_t *ptep; folio =pfn_folio (f->pfn); priv->rc = -EAGAIN ; + +if (!mmap_read_trylock (priv->kvm->mm)) +return ; + + ptep =try_get_locked_pte (priv->kvm->mm,gfn_to_hva (priv->kvm, f->gfn), &ptl); +if (IS_ERR_VALUE (ptep)) { + priv->rc =PTR_ERR (ptep); + goto out; + } +if (folio_trylock (folio)) { priv->rc =__kvm_s390_pv_make_secure (f, folio);if (priv->rc == -E2BIG || priv->rc == -EBUSY ) {
@@ -114,6 +128,11 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f) }folio_unlock (folio); } + +if (ptep) +pte_unmap_unlock (ptep, ptl); +out: +mmap_read_unlock (priv->kvm->mm); }/**
@@ -127,7 +146,7 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f)*/ int kvm_s390_pv_make_secure (struct kvm *kvm,unsigned long gaddr,void *uvcb) { -struct pv_make_secure priv = { .uvcb = uvcb }; +struct pv_make_secure priv = { .uvcb = uvcb, .kvm = kvm, };struct guest_fault f = { .write_attempt = true, .gfn =gpa_to_gfn (gaddr),
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 7/8] KVM: s390: Lock pte when making page secure
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] Unconditional conversion of a raw PFN to a folio using pfn_folio() in _kvm_s390_pv_make_secure() leads to a host kernel panic when applied to device memory. --
commit e0fa339799f284890bdbcc3dd174cd7125deb24a
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Lock pte when making page secure
This commit ensures that _kvm_s390_pv_make_secure() takes the pte lock for the given address before freezing the folio during Secure Execution transitions. This prevents concurrent GUP access while the page is mapped.
Quoted 4 lines
arch/s390/kvm/pv.c +0
-0
[ ... ]
arch/s390/kvm/pv.c static void _kvm_s390_pv_make_secure(struct guest_fault *f) · line 111
Quoted 8 lines
arch/s390/kvm/pv.c +2
-0@@ -103,9 +105,21 @@ static void _kvm_s390_pv_make_secure(struct guest_fault *f) {struct pv_make_secure *priv = f->priv;struct folio *folio; +spinlock_t *ptl;/* pte lock from try_get_locked_pte() */ +pte_t *ptep; folio =pfn_folio (f->pfn);
[Severity: Critical]
This is a pre-existing issue, but can this unconditional conversion of a raw PFN to a folio cause a host kernel panic when a guest maps device memory?
If a malicious or misconfigured guest maps device memory (e.g., via VFIO PCI) into its address space and attempts to transition it to Secure Execution, KVM's fault-in logic will return a valid PFN but set f->page = NULL for raw device memory without a vmemmap.
If we unconditionally call pfn_folio(f->pfn) before checking f->page, pfn_to_page() yields an invalid pointer. The page_folio() wrapper then immediately dereferences this pointer to read the compound_head field.
Furthermore, even if the PFN is valid but points to a reserved page, the fallback logic in __kvm_s390_pv_make_secure() could attempt to increment the refcount of a reserved page:
__kvm_s390_pv_make_secure() {
...
if (!f->page)
folio_get(folio);
...
}Could this crash the kernel via VM_BUG_ON_FOLIO() on systems with CONFIG_DEBUG_VM enabled? Should the folio conversion be delayed until after verifying we have a normal refcounted memory page?
Quoted 12 lines
arch/s390/kvm/pv.c +10
-0priv->rc = -EAGAIN ; + +if (!mmap_read_trylock (priv->kvm->mm)) +return ; + + ptep =try_get_locked_pte (priv->kvm->mm,gfn_to_hva (priv->kvm, f->gfn), &ptl); +if (IS_ERR_VALUE (ptep)) { + priv->rc =PTR_ERR (ptep); + goto out; + } +if (folio_trylock (folio)) {
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Claudio Imbrenda
maintainerPATCH v2 8/8 KVM: s390: Prevent memslots outside the ASCE range
In reply to Claudio ImbrendaWith KVM_S390_VM_MEM_LIMIT_SIZE, userspace can set the highest address allowed for the VM. Creating a memslot that lies over the maximum address does not make sense and is only a potential source of bugs.
Prevent creation of memslots over the maximum address, and prevent the maximum address from being reduced below the end of existing memslots.
--- arch/s390/kvm/kvm-s390.c | 30 ++++++++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-)
arch/s390/kvm/kvm-s390.c +26 -4
@@ -999,7 +999,10 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *attbreak ; }case KVM_S390_VM_MEM_LIMIT_SIZE : { +struct kvm_memslots *slots; +struct kvm_memory_slot *ms;unsigned long new_limit; +int bkt;if (kvm_is_ucontrol (kvm))return -EINVAL ;
@@ -1007,6 +1010,8 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *attif (get_user (new_limit, (u64 __user *)attr->addr))return -EFAULT ; +guard (mutex)(&kvm->lock); +if (kvm->arch.mem_limit !=KVM_S390_NO_MEM_LIMIT && new_limit > kvm->arch.mem_limit)return -E2BIG ;
@@ -1014,12 +1019,27 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *attif (!new_limit)return -EINVAL ; - ret = -EBUSY ; -if (!kvm->created_vcpus) +if (kvm->created_vcpus) +return -EBUSY ; + + ret =0 ; + scoped_guard(mutex, &kvm->slots_lock) { + slots =kvm_memslots (kvm); +if (slots && !kvm_memslots_empty (slots)) { +kvm_for_each_memslot (ms, bkt, slots) { +if (gpa_to_gfn (new_limit) < ms->base_gfn + ms->npages) { + ret = -EBUSY ; +break ; + } + } + } + } +if (!ret) ret =gmap_set_limit (kvm->arch.gmap,gpa_to_gfn (new_limit)); +if (ret) +break ;VM_EVENT (kvm,3 ,"SET: max guest address: %lu" , new_limit); -VM_EVENT (kvm,3 ,"New guest asce: 0x%p" , - (void *)kvm->arch.gmap->asce.val); +VM_EVENT (kvm,3 ,"New guest asce: 0x%p" , (void *)kvm->arch.gmap->asce.val);break ; }default :
@@ -5672,6 +5692,8 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,return -EINVAL ;if ((new->base_gfn + new->npages) *PAGE_SIZE > kvm->arch.mem_limit)return -EINVAL ; +if (!asce_contains_gfn (kvm->arch.gmap->asce, new->base_gfn + new->npages -1 )) +return -EINVAL ; }if (!kvm->arch.migration_mode)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 8/8] KVM: s390: Prevent memslots outside the ASCE range
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [High] Dropping kvm->slots_lock before updating the ASCE limit creates a TOCTOU race and data race condition that allows memslots to bypass the new limit and risk reading torn values.
- [High] The exact user-provided memory limit is discarded and new memslots are checked against a coarse, rounded-up hardware ASCE boundary, allowing userspace to bypass the exact limit.
Pre-existing issues:
- [High] Setting a sub-page limit (e.g., 4095 bytes) silently removes the memory limit by expanding the VM address space to the maximum 16 Exabytes. --
commit afeca87ebcc558baee29b01c63a93364ae7ae27f
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Prevent memslots outside the ASCE range
With KVM_S390_VM_MEM_LIMIT_SIZE, userspace can set the highest address allowed for the VM. This patch prevents creation of memslots over the maximum address, and prevents the maximum address from being reduced below the end of existing memslots.
arch/s390/kvm/kvm-s390.c static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att · line 1020
Quoted 7 lines
arch/s390/kvm/kvm-s390.c +0
-0@@ -1014,12 +1019,27 @@ static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *attif (!new_limit)return -EINVAL ;
[Severity: High]
This is a pre-existing issue, but does this check fail to reject sub-page size limits that truncate to 0 GFNs?
If a sub-page limit (e.g., 4095) is passed, it passes this zero check but truncates to 0 when converted via gpa_to_gfn(new_limit) later in the function.
When gmap_set_limit() is called with a 0 limit, gmap_limit_to_type(0) returns TABLE_TYPE_REGION1, granting the maximum 16 EB limit. Could this allow an extremely restricted address limit to silently grant the maximum possible address space?
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +7
-2- ret = -EBUSY ; -if (!kvm->created_vcpus) +if (kvm->created_vcpus) +return -EBUSY ; + + ret =0 ; + scoped_guard(mutex, &kvm->slots_lock) { + slots =kvm_memslots (kvm); +if (slots && !kvm_memslots_empty (slots)) {
arch/s390/kvm/kvm-s390.c static int kvm_s390_set_mem_control(struct kvm *kvm, struct kvm_device_attr *att · line 1038
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +9
-0+kvm_for_each_memslot (ms, bkt, slots) { +if (gpa_to_gfn (new_limit) < ms->base_gfn + ms->npages) { + ret = -EBUSY ; +break ; + } + } + } + } +if (!ret) ret =gmap_set_limit (kvm->arch.gmap,gpa_to_gfn (new_limit));
[Severity: High]
Is there a Time-of-Check to Time-of-Use (TOCTOU) race here by dropping kvm->slots_lock before updating the ASCE limit?
By dropping slots_lock before calling gmap_set_limit(), the 64-bit gmap->asce value is modified non-atomically under mmu_lock.
Concurrently, KVM_SET_USER_MEMORY_REGION can take slots_lock and call kvm_arch_prepare_memory_region(), which reads gmap->asce locklessly:
kvm_arch_prepare_memory_region() {
...
if (!asce_contains_gfn(kvm->arch.gmap->asce, ...))
return -EINVAL;
}Could this allow kvm_arch_prepare_memory_region() to read the old, larger ASCE limit and validate a memslot that violates the newly requested limit? Additionally, is there a risk of reading a torn gmap->asce struct value during the non-atomic modification?
[ ... ]
arch/s390/kvm/kvm-s390.c int kvm_arch_prepare_memory_region(struct kvm *kvm, · line 5696
Quoted 6 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -5672,6 +5692,8 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,return -EINVAL ;if ((new->base_gfn + new->npages) *PAGE_SIZE > kvm->arch.mem_limit)return -EINVAL ; +if (!asce_contains_gfn (kvm->arch.gmap->asce, new->base_gfn + new->npages -1 )) +return -EINVAL ;
[Severity: High]
Does this allow userspace to bypass the exact requested memory limit?
Earlier in kvm_s390_set_mem_control(), the exact new_limit (e.g., 1 GB) is verified against existing memslots, but it is discarded and never stored in kvm->arch.mem_limit.
gmap_set_limit() then converts the exact limit to a coarse ASCE boundary, rounding it up to the next hardware limit (e.g., 2 GB).
Because this new check in kvm_arch_prepare_memory_region() only evaluates against the coarse ASCE boundary via asce_contains_gfn(), could userspace request a strict limit but silently allocate memslots up to the next hardware boundary?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8