KVM: s390: More memory management fixes
Claudio Imbrenda
maintainerv3 cover KVM: s390: More memory management fixes
This series fixes some bugs that have been recently introduced with the big gmap rewrite.
Most of the fixes are related to vSIE (nested guests), although some are more general. The last patch fixes an issue introduced last year.
v2->v3
* fix a typo in the code (missin negation in patch 2)
* fix pre-existing alignment issue in _gmap_crstep_xchg_atomic()
* fix nested guest memory shadowing
* properly handle another possible place in _do_shadow_crste() where an unshadowing event could be triggered
* fix _gaccess_do_shadow() to properly handle the case of real-address asces for the nested guest
* fix some typos
v1->v2
* propagate vsie notification bit when splitting a large page
* replace gmap_crstep_xchg() with gmap_crstep_xchg_atomic(); let the callers properly deal with races
* do not attempt to protect guest page table when the nested guest is running in a real address space
* fix return value of KVM_S390_VCPU_FAULT in case of error
Claudio Imbrenda (9):
KVM: s390: vsie: Fix dat_split_ste()
KVM: s390: Remove non-atomic dat_crstep_xchg()
KVM: s390: vsie: Fix check for pre-existing shadow mapping
KVM: s390: vsie: Fix nested guest memory shadowing
KVM: s390: Fix gmap_link()
KVM: s390: vsie: Fix refcount overflow for shadow gmaps
KVM: s390: vsie: Fix unshadowing while shadowing
KVM: s390: vsie: Fix guest page tables protection
KVM: s390: Fix KVM_S390_VCPU_FAULT ioctl
arch/s390/kvm/dat.c | 102 +++++---------------------- arch/s390/kvm/dat.h | 11 +-- arch/s390/kvm/gaccess.c | 58 ++++++++++----- arch/s390/kvm/gmap.c | 149 +++++++++++++++++++++++++++++---------- arch/s390/kvm/gmap.h | 26 +++---- arch/s390/kvm/kvm-s390.c | 18 ++++- arch/s390/kvm/vsie.c | 4 +- 7 files changed, 204 insertions(+), 164 deletions(-)
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 1/9 KVM: s390: vsie: Fix dat_split_ste()
If the guest misbehaves and puts the page tables for its nested guest inside the memory of the nested guest itself, and the guest and nested guest are being mapped with large pages, the shadow mapping will lose synchronization with the actual mapping, since this will cause the large page with the vsie notification bit to be split, but the vsie notification bit will not be propagated to the resulting small pages.
Fix this by propagating the vsie_notif bit from large pages to normal pages when splitting a large page.
--- arch/s390/kvm/dat.c | 1 + 1 file changed, 1 insertion(+)
arch/s390/kvm/dat.c +1 -0
@@ -292,6 +292,7 @@ static int dat_split_ste(struct kvm_s390_mmu_cache *mc, union pmd *pmdp, gfn_t g pt->ptes[i].val = init.val | i *PAGE_SIZE ;/* No need to take locks as the page table is not installed yet. */ pgste_init.prefix_notif = old.s.fc1.prefix_notif; + pgste_init.vsie_notif = old.s.fc1.vsie_notif; pgste_init.pcl = uses_skeys && init.h.i;dat_init_pgstes (pt, pgste_init.val); }else {
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 2/9 KVM: s390: Remove non-atomic dat_crstep_xchg()
In reply to Claudio ImbrendaIn practice dat_crstep_xchg() is racy and hard to use correctly. Simply remove it and replace its uses with dat_crstep_xchg_atomic().
This solves some actual races that lead to system hangs / crashes.
Opportunistically fix an alignment issue in _gmap_crstep_xchg_atomic().
--- arch/s390/kvm/dat.c | 53 ++++++++------------------ arch/s390/kvm/dat.h | 9 +++-- arch/s390/kvm/gaccess.c | 26 +++++++------ arch/s390/kvm/gmap.c | 82 ++++++++++++++++++++++++----------------- arch/s390/kvm/gmap.h | 26 +++++++------ 5 files changed, 98 insertions(+), 98 deletions(-)
arch/s390/kvm/dat.c +15 -38
@@ -134,32 +134,6 @@ int dat_set_asce_limit(struct kvm_s390_mmu_cache *mc, union asce *asce, int newtreturn 0 ; } -/** -* dat_crstep_xchg() - Exchange a gmap CRSTE with another. -* @crstep: Pointer to the CRST entry -* @new: Replacement entry. -* @gfn: The affected guest address. -* @asce: The ASCE of the address space. -* -* Context: This function is assumed to be called with kvm->mmu_lock held. -*/ -void dat_crstep_xchg (union crste *crstep,union crste new,gfn_t gfn,union asce asce) -{ -if (crstep->h.i) { -WRITE_ONCE (*crstep, new); -return ; - }else if (cpu_has_edat2 ()) { -crdte_crste (crstep, *crstep, new, gfn, asce); -return ; - } - -if (machine_has_tlb_guest ()) -idte_crste (crstep, gfn,IDTE_GUEST_ASCE , asce,IDTE_GLOBAL ); -else -idte_crste (crstep, gfn,0 ,NULL_ASCE ,IDTE_GLOBAL ); -WRITE_ONCE (*crstep, new); -} -/** * dat_crstep_xchg_atomic() - Atomically exchange a gmap CRSTE with another. * @crstep: Pointer to the CRST entry.
@@ -175,8 +149,8 @@ void dat_crstep_xchg(union crste *crstep, union crste new, gfn_t gfn, union asce* * Return: %true if the exchange was successful. */ -bool dat_crstep_xchg_atomic (union crste *crstep,union crste old,union crste new,gfn_t gfn, -union asce asce) +bool __must_check dat_crstep_xchg_atomic (union crste *crstep,union crste old,union crste new, +gfn_t gfn,union asce asce) {if (old.h.i)return arch_try_cmpxchg ((long *)crstep, &old.val, new.val);
@@ -894,7 +868,8 @@ static long _dat_slot_crste(union crste *crstep, gfn_t gfn, gfn_t next, struct d/* This table entry needs to be updated. */ if (walk->start <= gfn && walk->end >= next) { -dat_crstep_xchg_atomic (crstep, crste, new_crste, gfn, walk->asce); +if (!dat_crstep_xchg_atomic (crstep, crste, new_crste, gfn, walk->asce)) +return -EINVAL ;/* A lower level table was present, needs to be freed. */ if (!crste.h.fc && !crste.h.i) {if (is_pmd (crste))
@@ -1072,17 +1047,19 @@ int dat_link(struct kvm_s390_mmu_cache *mc, union asce asce, int level,static long dat_set_pn_crste (union crste *crstep,gfn_t gfn,gfn_t next,struct dat_walk *walk) { -union crste crste =READ_ONCE (*crstep); +union crste newcrste, oldcrste;int *n = walk->priv; -if (!crste.h.fc || crste.h.i || crste.h.p) -return 0 ; - - *n =2 ; -if (crste.s.fc1.prefix_notif) -return 0 ; - crste.s.fc1.prefix_notif =1 ; -dat_crstep_xchg (crstep, crste, gfn, walk->asce); +do { + oldcrste =READ_ONCE (*crstep); +if (!oldcrste.h.fc || oldcrste.h.i || oldcrste.h.p) +return 0 ; + *n =2 ; +if (oldcrste.s.fc1.prefix_notif) +return 0 ; + newcrste = oldcrste; + newcrste.s.fc1.prefix_notif =1 ; + }while (!dat_crstep_xchg_atomic (crstep, oldcrste, newcrste, gfn, walk->asce));return 0 ; }
arch/s390/kvm/dat.h +6 -3
@@ -938,11 +938,14 @@ static inline bool dat_pudp_xchg_atomic(union pud *pudp, union pud old, union pureturn dat_crstep_xchg_atomic (_CRSTEP (pudp),_CRSTE (old),_CRSTE (new), gfn, asce); } -static inline void dat_crstep_clear (union crste *crstep,gfn_t gfn,union asce asce) +static inline union crste dat_crstep_clear_atomic (union crste *crstep,gfn_t gfn,union asce asce) { -union crste newcrste =_CRSTE_EMPTY (crstep->h.tt); +union crste oldcrste, empty =_CRSTE_EMPTY (crstep->h.tt); -dat_crstep_xchg (crstep, newcrste, gfn, asce); +do { + oldcrste =READ_ONCE (*crstep); + }while (!dat_crstep_xchg_atomic (crstep, oldcrste, empty, gfn, asce)); +return oldcrste; }static inline int get_level (union crste *crstep,union pte *ptep)
arch/s390/kvm/gaccess.c +15 -11
@@ -1456,7 +1456,7 @@ static int _do_shadow_pte(struct gmap *sg, gpa_t raddr, union pte *ptep_h, unionstatic int _do_shadow_crste (struct gmap *sg,gpa_t raddr,union crste *host,union crste *table,struct guest_fault *f,bool p) { -union crste newcrste; +union crste newcrste, oldcrste;gfn_t gfn;int rc;
@@ -1469,16 +1469,20 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, uniif (rc)return rc; - newcrste =_crste_fc1 (f->pfn, host->h.tt, f->writable, !p); - newcrste.s.fc1.d |= host->s.fc1.d; - newcrste.s.fc1.sd |= host->s.fc1.sd; - newcrste.h.p &= host->h.p; - newcrste.s.fc1.vsie_notif =1 ; - newcrste.s.fc1.prefix_notif = host->s.fc1.prefix_notif; -_gmap_crstep_xchg (sg->parent, host, newcrste, f->gfn, false); - - newcrste =_crste_fc1 (f->pfn, host->h.tt,0 , !p); -dat_crstep_xchg (table, newcrste,gpa_to_gfn (raddr), sg->asce); +do { + oldcrste =READ_ONCE (*host); + newcrste =_crste_fc1 (f->pfn, oldcrste.h.tt, f->writable, !p); + newcrste.s.fc1.d |= oldcrste.s.fc1.d; + newcrste.s.fc1.sd |= oldcrste.s.fc1.sd; + newcrste.h.p &= oldcrste.h.p; + newcrste.s.fc1.vsie_notif =1 ; + newcrste.s.fc1.prefix_notif = oldcrste.s.fc1.prefix_notif; + }while (!_gmap_crstep_xchg_atomic (sg->parent, host, oldcrste, newcrste, f->gfn, false)); + + newcrste =_crste_fc1 (f->pfn, oldcrste.h.tt,0 , !p); + gfn =gpa_to_gfn (raddr); +while (!dat_crstep_xchg_atomic (table,READ_ONCE (*table), newcrste, gfn, sg->asce)) + ;return 0 ; }
arch/s390/kvm/gmap.c +48 -34
137 lines not on this page · raw message
arch/s390/kvm/gmap.h +14 -12
@@ -194,35 +194,37 @@ static inline union pgste gmap_ptep_xchg(struct gmap *gmap, union pte *ptep, unireturn _gmap_ptep_xchg (gmap, ptep, newpte, pgste, gfn, true); } -static inline void _gmap_crstep_xchg (struct gmap *gmap,union crste *crstep,union crste ne, -gfn_t gfn,bool needs_lock) +static inline bool __must_check _gmap_crstep_xchg_atomic (struct gmap *gmap,union crste *crstep, +union crste oldcrste,union crste newcrste, +gfn_t gfn,bool needs_lock) { -unsigned long align =8 + (is_pmd (*crstep) ?0 :11 ); +unsigned long align =is_pmd (*crstep) ? _PAGE_ENTRIES : _PAGE_ENTRIES * _CRST_ENTRIES;lockdep_assert_held (&gmap->kvm->mmu_lock);if (!needs_lock)lockdep_assert_held (&gmap->children_lock); gfn =ALIGN_DOWN (gfn, align); -if (crste_prefix (*crstep) && (ne.h.p || ne.h.i || !crste_prefix (ne))) { - ne.s.fc1.prefix_notif =0 ; +if (crste_prefix (oldcrste) && (newcrste.h.p || newcrste.h.i || !crste_prefix (newcrste))) { + newcrste.s.fc1.prefix_notif =0 ;gmap_unmap_prefix (gmap, gfn, gfn + align); } -if (crste_leaf (*crstep) && crstep->s.fc1.vsie_notif && - (ne.h.p || ne.h.i || !ne.s.fc1.vsie_notif)) { - ne.s.fc1.vsie_notif =0 ; +if (crste_leaf (oldcrste) && oldcrste.s.fc1.vsie_notif && + (newcrste.h.p || newcrste.h.i || !newcrste.s.fc1.vsie_notif)) { + newcrste.s.fc1.vsie_notif =0 ;if (needs_lock)gmap_handle_vsie_unshadow_event (gmap, gfn);else _gmap_handle_vsie_unshadow_event (gmap, gfn); } -dat_crstep_xchg (crstep, ne, gfn, gmap->asce); +return dat_crstep_xchg_atomic (crstep, oldcrste, newcrste, gfn, gmap->asce); } -static inline void gmap_crstep_xchg (struct gmap *gmap,union crste *crstep,union crste ne, -gfn_t gfn) +static inline bool __must_check gmap_crstep_xchg_atomic (struct gmap *gmap,union crste *crstep, +union crste oldcrste,union crste newcrste, +gfn_t gfn) { -return _gmap_crstep_xchg (gmap, crstep, ne, gfn, true); +return _gmap_crstep_xchg_atomic (gmap, crstep, oldcrste, newcrste, gfn, true); }/**
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 3/9 KVM: s390: vsie: Fix check for pre-existing shadow mapping
In reply to Claudio ImbrendaWhen shadowing a nested guest, a check is performed and no shadowing is attempted if the nested guest is already shadowed.
The existing check was incomplete; fix it by also checking whether the leaf DAT table entry in the existing shadow gmap has the same protection as the one specified in the guest DAT entry.
--- arch/s390/kvm/gaccess.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/s390/kvm/gaccess.c +3 -2
@@ -1506,8 +1506,9 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,if (rc)return rc; -/* A race occourred. The shadow mapping is already valid, nothing to do */ -if ((ptep && !ptep->h.i) || (!ptep &&crste_leaf (*table))) +/* A race occurred. The shadow mapping is already valid, nothing to do */ +if ((ptep && !ptep->h.i && ptep->h.p == w->p) || + (!ptep &&crste_leaf (*table) && !table->h.i && table->h.p == w->p))return 0 ; gl =get_level (table, ptep);
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 4/9 KVM: s390: vsie: Fix nested guest memory shadowing
In reply to Claudio ImbrendaFix _do_shadow_pte() and _do_shadow_crste() to properly mark the guest page as dirty if the shadow mapping is writeable but the guest mapping was write-protected and becomes writeable.
Fix _do_shadow_pte() to use the correct pointer (guest pte instead of nested guest) to set up the new pte.
--- arch/s390/kvm/gaccess.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
arch/s390/kvm/gaccess.c +7 -3
@@ -1437,9 +1437,11 @@ static int _do_shadow_pte(struct gmap *sg, gpa_t raddr, union pte *ptep_h, unionif (!pgste_get_trylock (ptep_h, &pgste))return -EAGAIN ; newpte =_pte (f->pfn, f->writable, !p,0 ); - newpte.s.d |= ptep->s.d; - newpte.s.sd |= ptep->s.sd; - newpte.h.p &= ptep->h.p; + newpte.s.d |= ptep_h->s.d; + newpte.s.sd |= ptep_h->s.sd; + newpte.h.p &= ptep_h->h.p; +if (!ptep_h->s.d && newpte.s.d) +SetPageDirty (pfn_to_page (newpte.h.pfra)); pgste =_gmap_ptep_xchg (sg->parent, ptep_h, newpte, pgste, f->gfn, false); pgste.vsie_notif =1 ;pgste_set_unlock (ptep_h, pgste);
@@ -1477,6 +1479,8 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, uni newcrste.h.p &= oldcrste.h.p; newcrste.s.fc1.vsie_notif =1 ; newcrste.s.fc1.prefix_notif = oldcrste.s.fc1.prefix_notif; +if (!oldcrste.s.fc1.d && newcrste.s.fc1.d) +SetPageDirty (phys_to_page (crste_origin_large (newcrste))); }while (!_gmap_crstep_xchg_atomic (sg->parent, host, oldcrste, newcrste, f->gfn, false)); newcrste =_crste_fc1 (f->pfn, oldcrste.h.tt,0 , !p);
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 5/9 KVM: s390: Fix gmap_link()
In reply to Claudio ImbrendaThe slow path of the fault handler ultimately called gmap_link(), which assumed the fault was a major fault, and blindly called dat_link().
In case of minor faults, things were not always handled properly; in particular the prefix and vsie marker bits were ignored.
Move dat_link() into gmap.c, renaming it accordingly. Once moved, the new _gmap_link() function will be able to correctly honour the prefix and vsie markers.
This will cause spurious unshadows in some uncommon cases.
--- arch/s390/kvm/dat.c | 48 ------------------------------------- arch/s390/kvm/dat.h | 2 -- arch/s390/kvm/gmap.c | 56 ++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 52 insertions(+), 54 deletions(-)
arch/s390/kvm/dat.c +0 -48
@@ -997,54 +997,6 @@ bool dat_test_age_gfn(union asce asce, gfn_t start, gfn_t end)return _dat_walk_gfn_range (start, end, asce, &test_age_ops,0 ,NULL ) >0 ; } -int dat_link (struct kvm_s390_mmu_cache *mc,union asce asce,int level, -bool uses_skeys,struct guest_fault *f) -{ -union crste oldval, newval; -union pte newpte, oldpte; -union pgste pgste; -int rc =0 ; - - rc =dat_entry_walk (mc, f->gfn, asce,DAT_WALK_ALLOC_CONTINUE , level, &f->crstep, &f->ptep); -if (rc == -EINVAL || rc == -ENOMEM ) -return rc; -if (rc) -return -EAGAIN ; - -if (WARN_ON_ONCE (unlikely (get_level (f->crstep, f->ptep) > level))) -return -EINVAL ; - -if (f->ptep) { - pgste =pgste_get_lock (f->ptep); - oldpte = *f->ptep; - newpte =_pte (f->pfn, f->writable, f->write_attempt | oldpte.s.d, !f->page); - newpte.s.sd = oldpte.s.sd; - oldpte.s.sd =0 ; -if (oldpte.val == _PTE_EMPTY.val || oldpte.h.pfra == f->pfn) { - pgste =__dat_ptep_xchg (f->ptep, pgste, newpte, f->gfn, asce, uses_skeys); -if (f->callback) - f->callback (f); - }else { - rc = -EAGAIN ; - } -pgste_set_unlock (f->ptep, pgste); - }else { - oldval =READ_ONCE (*f->crstep); - newval =_crste_fc1 (f->pfn, oldval.h.tt, f->writable, - f->write_attempt | oldval.s.fc1.d); - newval.s.fc1.sd = oldval.s.fc1.sd; -if (oldval.val !=_CRSTE_EMPTY (oldval.h.tt).val && -crste_origin_large (oldval) !=crste_origin_large (newval)) -return -EAGAIN ; -if (!dat_crstep_xchg_atomic (f->crstep, oldval, newval, f->gfn, asce)) -return -EAGAIN ; -if (f->callback) - f->callback (f); - } - -return rc; -} -static long dat_set_pn_crste (union crste *crstep,gfn_t gfn,gfn_t next,struct dat_walk *walk) {union crste newcrste, oldcrste;
arch/s390/kvm/dat.h +0 -2
@@ -540,8 +540,6 @@ int dat_set_slot(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t start, gf u16 type, u16 param);int dat_set_prefix_notif_bit (union asce asce,gfn_t gfn);bool dat_test_age_gfn (union asce asce,gfn_t start,gfn_t end); -int dat_link (struct kvm_s390_mmu_cache *mc,union asce asce,int level, -bool uses_skeys,struct guest_fault *f);int dat_perform_essa (union asce asce,gfn_t gfn,int orc,union essa_state *state,bool *dirty);long dat_reset_cmma (union asce asce,gfn_t start_gfn);
arch/s390/kvm/gmap.c +52 -4
71 lines not on this page · raw message
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 6/9 KVM: s390: vsie: Fix refcount overflow for shadow gmaps
In reply to Claudio ImbrendaIn most cases gmap_put() was not called when it should have.
Add the missing gmap_put() in vsie_run().
--- arch/s390/kvm/vsie.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/s390/kvm/vsie.c +3 -1
@@ -1328,7 +1328,7 @@ static void unregister_shadow_scb(struct kvm_vcpu *vcpu)static int vsie_run (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page) {struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s; -struct gmap *sg; +struct gmap *sg =NULL ;int rc =0 ;while (1 ) {
@@ -1368,6 +1368,8 @@ static int vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) sg =gmap_put (sg);cond_resched (); } +if (sg) + sg =gmap_put (sg);if (rc == -EFAULT ) {/*
Signature · 1 line
2.53.0
Claudio Imbrenda
maintainerPATCH v3 7/9 KVM: s390: vsie: Fix unshadowing while shadowing
In reply to Claudio ImbrendaIf shadowing causes the shadow gmap to get unshadowed, exit early to prevent an attempt to dereference the parent pointer, which at this point is NULL.
Opportunistically add some more checks to prevent NULL parents.
--- arch/s390/kvm/gaccess.c | 5 +++++ arch/s390/kvm/gmap.c | 11 ++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
arch/s390/kvm/gaccess.c +5 -0
@@ -1472,6 +1472,9 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, unireturn rc; do { + /* _gmap_crstep_xchg_atomic() could have unshadowed this shadow gmap */ + if (!sg->parent) + return -EAGAIN; oldcrste = READ_ONCE(*host); newcrste = _crste_fc1(f->pfn, oldcrste.h.tt, f->writable, !p); newcrste.s.fc1.d |= oldcrste.s.fc1.d;
@@ -1526,6 +1529,8 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg, entries[i - 1].pfn, i, entries[i - 1].writable);if (rc)return rc; +if (!sg->parent) +return -EAGAIN ; } rc = dat_entry_walk(NULL, entries[LEVEL_MEM].gfn, sg->parent->asce, DAT_WALK_LEAF,
arch/s390/kvm/gmap.c +10 -1
@@ -1163,6 +1163,7 @@ struct gmap_protect_asce_top_level {static inline int __gmap_protect_asce_top_level (struct kvm_s390_mmu_cache *mc,struct gmap *sg,struct gmap_protect_asce_top_level *context) { +struct gmap *parent;int rc, i;guard (write_lock)(&sg->kvm->mmu_lock);
@@ -1170,7 +1171,12 @@ static inline int __gmap_protect_asce_top_level(struct kvm_s390_mmu_cache *mc, sif (kvm_s390_array_needs_retry_safe (sg->kvm, context->seq, context->f))return -EAGAIN ; - scoped_guard(spinlock, &sg->parent->children_lock) { + parent =READ_ONCE (sg->parent); +if (!parent) +return -EAGAIN ; + scoped_guard(spinlock, &parent->children_lock) { +if (READ_ONCE (sg->parent) != parent) +return -EAGAIN ;for (i =0 ; i <CRST_TABLE_PAGES ; i++) {if (!context->f[i].valid)continue ;
@@ -1253,6 +1259,9 @@ struct gmap *gmap_create_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *parestruct gmap *sg, *new;int rc; +if (WARN_ON (!parent)) +return ERR_PTR (-EINVAL ); + scoped_guard(spinlock, &parent->children_lock) { sg =gmap_find_shadow (parent, asce, edat_level);if (sg) {
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH v3 7/9] KVM: s390: vsie: Fix unshadowing while shadowing
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] Missing unshadowing check in _do_shadow_pte and _do_shadow_crste leads to orphaned shadow mappings and use-after-free of host memory. --
--- Patch [7]: [PATCH v3 7/9] KVM: s390: vsie: Fix unshadowing while shadowing --- commit d5030d28123280edd38c0f9ad86e8a0833ab9f12
Author: Claudio Imbrenda <[email protected]>
KVM: s390: vsie: Fix unshadowing while shadowing
If shadowing causes the shadow gmap to get unshadowed, exit early to prevent an attempt to dereference the parent pointer, which at this point is NULL.
arch/s390/kvm/gaccess.c static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, uni · line 1479
Quoted 13 lines
arch/s390/kvm/gaccess.c +3
-0@@ -1472,6 +1472,9 @@ static int _do_shadow_crste(struct gmap *sg, gpa_t raddr, union crste *host, unireturn rc; do { + /* _gmap_crstep_xchg_atomic() could have unshadowed this shadow gmap */ + if (!sg->parent) + return -EAGAIN; oldcrste = READ_ONCE(*host); newcrste = _crste_fc1(f->pfn, oldcrste.h.tt, f->writable, !p);
[Severity: Critical]
This is a pre-existing issue, but since we are checking for unshadowing here, could _gmap_crstep_xchg_atomic() unshadow the shadow gmap on the exact iteration where it succeeds?
If _gmap_crstep_xchg_atomic() succeeds but unshadows sg, the loop exits and bypasses this check on the next iteration. It would then unconditionally install the shadow entry just below the loop:
_do_shadow_crste() {
...
} while (!_gmap_crstep_xchg_atomic(sg->parent, host, oldcrste, newcrste, f->gfn, false));
newcrste = _crste_fc1(f->pfn, oldcrste.h.tt, 0, !p);
gfn = gpa_to_gfn(raddr);
while (!dat_crstep_xchg_atomic(table, READ_ONCE(*table), newcrste, gfn, sg->asce))
;
...
}Could this leave the guest with orphaned shadow mappings? If the host later unmaps the page, it iterates over parent->children to invalidate shadow page tables. Since sg was removed during unshadowing, it would fail to find and invalidate it, which seems like it might allow a use-after-free of host memory.
Would we need a similar check after the loop exits here, and similarly in _do_shadow_pte() after _gmap_ptep_xchg() returns?
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Claudio Imbrenda
maintainerPATCH v3 8/9 KVM: s390: vsie: Fix guest page tables protection
In reply to Claudio ImbrendaWhen shadowing, the guest page tables are write-protected, in order to trap changes and properly unshadow the shadow mapping for the nested guest. Already shadowed levels are skipped, so that only the needed levels are write protected.
Currently the levels that get write protected are exactly one level too
deep: the last level (nested guest memory) gets protected in the wrong way, and will be protected again correctly a few lines afterwards; most importantly, the highest non-shadowed level does *not* get write protected.
Moreover, if the nested guest is running in a real address space, there are no DAT tables to shadow.
Write protect the correct levels, so that all the levels that need to be protected are protected, and avoid double protecting the last level; skip attempting to shadow the DAT tables when the nested guest is running in a real address space.
--- arch/s390/kvm/gaccess.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/s390/kvm/gaccess.c +10 -2
@@ -1518,6 +1518,13 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg, (!ptep && crste_leaf(*table) && !table->h.i && table->h.p == w->p))return 0 ; +/* In case of a real address space */ +if (w->level <=LEVEL_MEM ) { + l =TABLE_TYPE_PAGE_TABLE ; + hl =TABLE_TYPE_REGION1 ; + goto real_address_space; + } + gl =get_level (table, ptep);/*
@@ -1525,8 +1532,8 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,* only the page containing the entry, not the whole table. */ for (i = gl ; i >= w->level; i--) { - rc =gmap_protect_rmap (mc, sg, entries[i -1 ].gfn,gpa_to_gfn (saddr), - entries[i -1 ].pfn, i, entries[i -1 ].writable); + rc =gmap_protect_rmap (mc, sg, entries[i].gfn,gpa_to_gfn (saddr), + entries[i].pfn, i +1 , entries[i].writable);if (rc)return rc;if (!sg->parent)
@@ -1542,6 +1549,7 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,/* Get the smallest granularity */ l =min3 (gl, hl, w->level); +real_address_space: flags =DAT_WALK_SPLIT_ALLOC | (uses_skeys (sg->parent) ?DAT_WALK_USES_SKEYS :0 );/* If necessary, create the shadow mapping */ if (l < gl) {
Signature · 1 line
2.53.0
kernel test robot
build botRe: [PATCH v3 8/9] KVM: s390: vsie: Fix guest page tables protection
kernel test robot noticed the following build warnings:
Hi Claudio,
kernel test robot noticed the following build warnings:
[auto build test WARNING on kvm/queue]
[also build test WARNING on kvm/next linus/master v7.0-rc5 next-20260324] [cannot apply to kvms390/next kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Claudio-Imbrenda/KVM-s390-vsie-Fix-dat_split_ste/20260325-092851 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20260324174301.232921-9-imbrenda%40linux.ibm.com patch subject: [PATCH v3 8/9] KVM: s390: vsie: Fix guest page tables protection config: s390-defconfig (https://download.01.org/0day-ci/archive/20260325/[email protected]/config) compiler: clang version 23.0.0git (https://github.com/llvm/llvm-project 054e11d1a17e5ba88bb1a8ef32fad3346e80b186) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260325/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> arch/s390/kvm/gaccess.c:1522:6: warning: variable 'gl' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized]
1522 | if (w->level <= LEVEL_MEM) {
| ^~~~~~~~~~~~~~~~~~~~~
arch/s390/kvm/gaccess.c:1555:10: note: uninitialized use occurs here
1555 | if (l < gl) {
| ^~
arch/s390/kvm/gaccess.c:1522:2: note: remove the 'if' if its condition is always false
1522 | if (w->level <= LEVEL_MEM) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
1523 | l = TABLE_TYPE_PAGE_TABLE;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
1524 | hl = TABLE_TYPE_REGION1;
| ~~~~~~~~~~~~~~~~~~~~~~~~
1525 | goto real_address_space;
| ~~~~~~~~~~~~~~~~~~~~~~~~
1526 | }
| ~
arch/s390/kvm/gaccess.c:1500:22: note: initialize the variable 'gl' to silence this warning
1500 | int flags, i, hl, gl, l, rc;
| ^
| = 01 warning generated.
vim +1522 arch/s390/kvm/gaccess.c
1495
1496 static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,
1497 unsigned long saddr, struct pgtwalk *w)
1498 {
1499 struct guest_fault *entries;
1500 int flags, i, hl, gl, l, rc;
1501 union crste *table, *host;
1502 union pte *ptep, *ptep_h;
1503
1504 lockdep_assert_held(&sg->kvm->mmu_lock);
1505 lockdep_assert_held(&sg->parent->children_lock);
1506
1507 entries = get_entries(w);
1508 ptep_h = NULL;
1509 ptep = NULL;
1510
1511 rc = dat_entry_walk(NULL, gpa_to_gfn(saddr), sg->asce, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE,
1512 &table, &ptep);
1513 if (rc)
1514 return rc;
1515
1516 /* A race occurred. The shadow mapping is already valid, nothing to do */
1517 if ((ptep && !ptep->h.i && ptep->h.p == w->p) ||
1518 (!ptep && crste_leaf(*table) && !table->h.i && table->h.p == w->p))
1519 return 0;
1520
1521 /* In case of a real address space */
> 1522 if (w->level <= LEVEL_MEM) { 1523 l = TABLE_TYPE_PAGE_TABLE;
1524 hl = TABLE_TYPE_REGION1;
1525 goto real_address_space;
1526 }
1527
1528 gl = get_level(table, ptep);
1529
1530 /*
1531 * Skip levels that are already protected. For each level, protect
1532 * only the page containing the entry, not the whole table.
1533 */
1534 for (i = gl ; i >= w->level; i--) {
1535 rc = gmap_protect_rmap(mc, sg, entries[i].gfn, gpa_to_gfn(saddr),
1536 entries[i].pfn, i + 1, entries[i].writable);
1537 if (rc)
1538 return rc;
1539 if (!sg->parent)
1540 return -EAGAIN;
1541 }
1542
1543 rc = dat_entry_walk(NULL, entries[LEVEL_MEM].gfn, sg->parent->asce, DAT_WALK_LEAF,
1544 TABLE_TYPE_PAGE_TABLE, &host, &ptep_h);
1545 if (rc)
1546 return rc;
1547
1548 hl = get_level(host, ptep_h);
1549 /* Get the smallest granularity */
1550 l = min3(gl, hl, w->level);
1551
1552 real_address_space:
1553 flags = DAT_WALK_SPLIT_ALLOC | (uses_skeys(sg->parent) ? DAT_WALK_USES_SKEYS : 0);
1554 /* If necessary, create the shadow mapping */
1555 if (l < gl) {
1556 rc = dat_entry_walk(mc, gpa_to_gfn(saddr), sg->asce, flags, l, &table, &ptep);
1557 if (rc)
1558 return rc;
1559 }
1560 if (l < hl) {
1561 rc = dat_entry_walk(mc, entries[LEVEL_MEM].gfn, sg->parent->asce,
1562 flags, l, &host, &ptep_h);
1563 if (rc)
1564 return rc;
1565 }
1566
1567 if (KVM_BUG_ON(l > TABLE_TYPE_REGION3, sg->kvm))
1568 return -EFAULT;
1569 if (l == TABLE_TYPE_PAGE_TABLE)
1570 return _do_shadow_pte(sg, saddr, ptep_h, ptep, entries + LEVEL_MEM, w->p);
1571 return _do_shadow_crste(sg, saddr, host, table, entries + LEVEL_MEM, w->p);
1572 }
1573
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
[email protected]
AI reviewRe: [PATCH v3 8/9] KVM: s390: vsie: Fix guest page tables protection
In reply to Claudio ImbrendaThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] Uninitialized variable 'gl' used in condition, leading to skipped allocation and host kernel…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] Uninitialized variable 'gl' used in condition, leading to skipped allocation and host kernel NULL pointer dereference. --
--- Patch [8]: [PATCH v3 8/9] KVM: s390: vsie: Fix guest page tables protection --- commit ff58b599b812f86d2edff5eab1c005a2ef716cc7
Author: Claudio Imbrenda <[email protected]>
KVM: s390: vsie: Fix guest page tables protection
This patch adjusts the levels of guest page tables that are write-protected when shadowing, and avoids double protecting the last level. It also skips attempting to shadow the DAT tables when the nested guest is running in a real address space.
Quoted 16 lines
arch/s390/kvm/gaccess.c +7
-0@@ -1518,6 +1518,13 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg, (!ptep && crste_leaf(*table) && !table->h.i && table->h.p == w->p))return 0 ; +/* In case of a real address space */ +if (w->level <=LEVEL_MEM ) { + l =TABLE_TYPE_PAGE_TABLE ; + hl =TABLE_TYPE_REGION1 ; + goto real_address_space; + } + gl =get_level (table, ptep);
[ ... ]
arch/s390/kvm/gaccess.c static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg, · line 1555
Quoted 8 lines
arch/s390/kvm/gaccess.c +1
-0@@ -1542,6 +1549,7 @@ static int _gaccess_do_shadow(struct kvm_s390_mmu_cache *mc, struct gmap *sg,/* Get the smallest granularity */ l =min3 (gl, hl, w->level); +real_address_space: flags =DAT_WALK_SPLIT_ALLOC | (uses_skeys (sg->parent) ?DAT_WALK_USES_SKEYS :0 );/* If necessary, create the shadow mapping */ if (l < gl) {
^^
[Severity: Critical]
Does this leave the variable gl uninitialized when taking the goto?
When a nested guest is running in a real address space, the new goto statement bypasses the initialization of gl:
_gaccess_do_shadow() {
...
if (w->level <= LEVEL_MEM) {
l = TABLE_TYPE_PAGE_TABLE;
hl = TABLE_TYPE_REGION1;
goto real_address_space;
}
gl = get_level(table, ptep);
...
real_address_space:
...
if (l < gl) {If the uninitialized stack value for gl causes the if (l < gl) condition to evaluate to false, the dat_entry_walk() call is skipped.
Since ptep was initialized to NULL at the start of _gaccess_do_shadow() and the initial dat_entry_walk() may have stopped early without setting it, ptep can remain NULL.
Later in the function, _do_shadow_pte() is called with this NULL ptep, which could lead to a NULL pointer dereference in pgste_get_trylock() and a host kernel panic.
Is there a way to ensure gl is initialized before the jump, or provide a safe default when running in a real address space?
Quoted 4 lines
arch/s390/kvm/gaccess.c +0
-0rc =dat_entry_walk (mc,gpa_to_gfn (saddr), sg->asce, flags, l, &table, &ptep);if (rc)return rc; }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Claudio Imbrenda
maintainerPATCH v3 9/9 KVM: s390: Fix KVM_S390_VCPU_FAULT ioctl
In reply to Claudio ImbrendaA previous commit changed the behaviour of the KVM_S390_VCPU_FAULT ioctl. The current (wrong) implementation will trigger a guest addressing exception if the requested address lies outside of a memslot, unless the VM is UCONTROL.
Restore the previous behaviour by open coding the fault-in logic.
--- arch/s390/kvm/kvm-s390.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +15 -3
@@ -5520,9 +5520,21 @@ long kvm_arch_vcpu_ioctl(struct file *filp, }#endif case KVM_S390_VCPU_FAULT : { - idx =srcu_read_lock (&vcpu->kvm->srcu); - r =vcpu_dat_fault_handler (vcpu, arg,0 ); -srcu_read_unlock (&vcpu->kvm->srcu, idx); +gpa_t gaddr = arg; + + scoped_guard(srcu, &vcpu->kvm->srcu) { + r =vcpu_ucontrol_translate (vcpu, &gaddr); +if (r) +break ; + + r =kvm_s390_faultin_gfn_simple (vcpu,NULL ,gpa_to_gfn (gaddr), false); +if (r ==PGM_ADDRESSING ) + r = -EFAULT ; +if (r <=0 ) +break ; + r = -EIO ; +KVM_BUG_ON (r, vcpu->kvm); + }break ; }case KVM_ENABLE_CAP :
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH v3 9/9] KVM: s390: Fix KVM_S390_VCPU_FAULT ioctl
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
A previous commit changed the behaviour of the
KVM_S390_VCPU_FAULTioctl. The current (wrong) implementation will trigger a guest addressing exception if the requested address lies outside of a memslot, unless the VM is UCONTROL.Restore the previous behaviour by open coding the fault-in logic.
Fixes 3762e905ec2e ("KVM: s390: use __kvm_faultin_pfn()")Signed-off-by Claudio Imbrenda <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9