KVM: s390: some non-trivial fixes and cleanups for 6.14
Subsystems (6)
Claudio Imbrenda
maintainerCiao Paolo,
please pull the following changes:
- some selftest fixes
- move some kvm-related functions from mm into kvm
- remove all usage of page->index and page->lru from kvm
- fixes and cleanups for vsie
The following changes since commit 72deda0abee6e705ae71a93f69f55e33be5bca5c:
Merge tag 'soundwire-6.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/soundwire (2025-01-29 14:38:19 -0800)
are available in the Git repository at:
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git tags/kvm-s390-next-6.14-1
for you to fetch changes up to 8bc53739bc8f26c3418ca17a7dbe030bce22a60d:
KVM: s390: selftests: Streamline uc_skey test to issue iske after sske (2025-01-30 08:49:54 +0100)
----------------------------------------------------------------
- some selftest fixes
- move some kvm-related functions from mm into kvm
- remove all usage of page->index and page->lru from kvm
- fixes and cleanups for vsie
----------------------------------------------------------------
Christoph Schlameuss (1):
KVM: s390: selftests: Streamline uc_skey test to issue iske after sske
Claudio Imbrenda (14):
KVM: s390: wrapper for KVM_BUG
KVM: s390: fake memslot for ucontrol VMs
KVM: s390: selftests: fix ucontrol memory region test
KVM: s390: move pv gmap functions into kvm
KVM: s390: use __kvm_faultin_pfn()
KVM: s390: get rid of gmap_fault()
KVM: s390: get rid of gmap_translate()
KVM: s390: move some gmap shadowing functions away from mm/gmap.c
KVM: s390: stop using page->index for non-shadow gmaps
KVM: s390: stop using lists to keep track of used dat tables
KVM: s390: move gmap_shadow_pgt_lookup() into kvm
KVM: s390: remove useless page->index usage
KVM: s390: move PGSTE softbits
KVM: s390: remove the last user of page->index
David Hildenbrand (4):
KVM: s390: vsie: fix some corner-cases when grabbing vsie pages
KVM: s390: vsie: stop using page->index
KVM: s390: vsie: stop messing with page refcount
KVM: s390: vsie: stop using "struct page" for vsie page
Sean Christopherson (1):
KVM: Do not restrict the size of KVM-internal memory regions
Documentation/virt/kvm/api.rst | 2 +- arch/s390/include/asm/gmap.h | 20 +- arch/s390/include/asm/kvm_host.h | 6 +- arch/s390/include/asm/pgtable.h | 21 +- arch/s390/include/asm/uv.h | 6 +- arch/s390/kernel/uv.c | 292 +--------- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gaccess.c | 44 +- arch/s390/kvm/gmap-vsie.c | 142 +++++ arch/s390/kvm/gmap.c | 212 +++++++ arch/s390/kvm/gmap.h | 39 ++ arch/s390/kvm/intercept.c | 7 +- arch/s390/kvm/interrupt.c | 19 +- arch/s390/kvm/kvm-s390.c | 237 ++++++-- arch/s390/kvm/kvm-s390.h | 19 + arch/s390/kvm/pv.c | 21 + arch/s390/kvm/vsie.c | 106 ++-- arch/s390/mm/gmap.c | 681 +++++------------------ arch/s390/mm/pgalloc.c | 2 - tools/testing/selftests/kvm/s390/ucontrol_test.c | 32 +- virt/kvm/kvm_main.c | 10 +- 21 files changed, 990 insertions(+), 930 deletions(-) create mode 100644 arch/s390/kvm/gmap-vsie.c create mode 100644 arch/s390/kvm/gmap.c create mode 100644 arch/s390/kvm/gmap.h
Claudio Imbrenda
maintainer[GIT PULL v1 01/20] KVM: s390: vsie: fix some corner-cases when grabbing vsie pages
From: David Hildenbrand <[email protected]>
We try to reuse the same vsie page when re-executing the vsie with a given SCB address. The result is that we use the same shadow SCB -- residing in the vsie page -- and can avoid flushing the TLB when re-running the vsie on a CPU.
So, when we allocate a fresh vsie page, or when we reuse a vsie page for a different SCB address -- reusing the shadow SCB in different context -- we set ihcpu=0xffff to trigger the flush.
However, after we looked up the SCB address in the radix tree, but before we grabbed the vsie page by raising the refcount to 2, someone could reuse the vsie page for a different SCB address, adjusting page->index and the radix tree. In that case, we would be reusing the vsie page with a wrong page->index.
Another corner case is that we might set the SCB address for a vsie page, but fail the insertion into the radix tree. Whoever would reuse that page would remove the corresponding radix tree entry -- which might now be a valid entry pointing at another page, resulting in the wrong vsie page getting removed from the radix tree.
Let's handle such races better, by validating that the SCB address of a vsie page didn't change after we grabbed it (not reuse for a different SCB; the alternative would be performing another tree lookup), and by setting the SCB address to invalid until the insertion in the tree succeeded (SCB addresses are aligned to 512, so ULONG_MAX is invalid).
These scenarios are rare, the effects a bit unclear, and these issues were only found by code inspection. Let's CC stable to be safe.
Message-ID: <[email protected]>
--- arch/s390/kvm/vsie.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/vsie.c +19 -6
@@ -1362,8 +1362,14 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr) page =radix_tree_lookup (&kvm->arch.vsie.addr_to_page, addr >>9 );rcu_read_unlock ();if (page) { -if (page_ref_inc_return (page) ==2 ) -return page_to_virt (page); +if (page_ref_inc_return (page) ==2 ) { +if (page->index == addr) +return page_to_virt (page); +/* +* We raced with someone reusing + putting this vsie +* page before we grabbed it. +*/ + }page_ref_dec (page); }
@@ -1393,15 +1399,20 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr) kvm->arch.vsie.next++; kvm->arch.vsie.next %= nr_vcpus; } -radix_tree_delete (&kvm->arch.vsie.addr_to_page, page->index >>9 ); +if (page->index !=ULONG_MAX ) +radix_tree_delete (&kvm->arch.vsie.addr_to_page, + page->index >>9 ); } - page->index = addr; -/* double use of the same address */ +/* Mark it as invalid until it resides in the tree. */ + page->index =ULONG_MAX ; + +/* Double use of the same address or allocation failure. */ if (radix_tree_insert (&kvm->arch.vsie.addr_to_page, addr >>9 , page)) {page_ref_dec (page);mutex_unlock (&kvm->arch.vsie.mutex);return NULL ; } + page->index = addr;mutex_unlock (&kvm->arch.vsie.mutex); vsie_page =page_to_virt (page);
@@ -1496,7 +1507,9 @@ void kvm_s390_vsie_destroy(struct kvm *kvm) vsie_page =page_to_virt (page);release_gmap_shadow (vsie_page);/* free the radix tree entry */ -radix_tree_delete (&kvm->arch.vsie.addr_to_page, page->index >>9 ); +if (page->index !=ULONG_MAX ) +radix_tree_delete (&kvm->arch.vsie.addr_to_page, + page->index >>9 );__free_page (page); } kvm->arch.vsie.page_count =0 ;
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 03/20] KVM: s390: vsie: stop messing with page refcount
In reply to Claudio ImbrendaFrom: David Hildenbrand <[email protected]>
Let's stop messing with the page refcount, and use a flag that is set / cleared atomically to remember whether a vsie page is currently in use.
Note that we could use a page flag, or a lower bit of the scb_gpa. Let's keep it simple for now, we have sufficient space.
While at it, stop passing "struct kvm *" to put_vsie_page(), it's unused.
Message-ID: <[email protected]>
--- arch/s390/kvm/vsie.c | 46 +++++++++++++++++++++++++++++--------------- 1 file changed, 30 insertions(+), 16 deletions(-)
arch/s390/kvm/vsie.c +30 -16
@@ -23,6 +23,10 @@#include "kvm-s390.h" #include "gaccess.h" +enum vsie_page_flags { +VSIE_PAGE_IN_USE =0 , +}; +struct vsie_page {struct kvm_s390_sie_block scb_s;/* 0x0000 */ /*
@@ -52,7 +56,12 @@ struct vsie_page {* radix tree. */ gpa_t scb_gpa;/* 0x0258 */ -__u8 reserved[0x0700 -0x0260 ];/* 0x0260 */ +/* +* Flags: must be set/cleared atomically after the vsie page can be +* looked up by other CPUs. +*/ +unsigned long flags;/* 0x0260 */ +__u8 reserved[0x0700 -0x0268 ];/* 0x0268 */ struct kvm_s390_crypto_cb crycb;/* 0x0700 */ __u8 fac[S390_ARCH_FAC_LIST_SIZE_BYTE ];/* 0x0800 */ };
@@ -1351,6 +1360,20 @@ static int vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)return rc; } +/* Try getting a given vsie page, returning "true" on success. */ +static inline bool try_get_vsie_page (struct vsie_page *vsie_page) +{ +if (test_bit (VSIE_PAGE_IN_USE , &vsie_page->flags)) +return false; +return !test_and_set_bit (VSIE_PAGE_IN_USE , &vsie_page->flags); +} + +/* Put a vsie page acquired through get_vsie_page / try_get_vsie_page. */ +static void put_vsie_page (struct vsie_page *vsie_page) +{ +clear_bit (VSIE_PAGE_IN_USE , &vsie_page->flags); +} +/* * Get or create a vsie page for a scb address. *
@@ -1369,15 +1392,15 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)rcu_read_unlock ();if (page) { vsie_page =page_to_virt (page); -if (page_ref_inc_return (page) ==2 ) { +if (try_get_vsie_page (vsie_page)) {if (vsie_page->scb_gpa == addr)return vsie_page;/* * We raced with someone reusing + putting this vsie * page before we grabbed it. */ +put_vsie_page (vsie_page); } -page_ref_dec (page); }/*
@@ -1394,7 +1417,7 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)return ERR_PTR (-ENOMEM ); } vsie_page =page_to_virt (page); -page_ref_inc (page); +__set_bit (VSIE_PAGE_IN_USE , &vsie_page->flags); kvm->arch.vsie.pages[kvm->arch.vsie.page_count] = page; kvm->arch.vsie.page_count++; }else {
@@ -1402,9 +1425,8 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)while (true) { page = kvm->arch.vsie.pages[kvm->arch.vsie.next]; vsie_page =page_to_virt (page); -if (page_ref_inc_return (page) ==2 ) +if (try_get_vsie_page (vsie_page))break ; -page_ref_dec (page); kvm->arch.vsie.next++; kvm->arch.vsie.next %= nr_vcpus; }
@@ -1417,7 +1439,7 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)/* Double use of the same address or allocation failure. */ if (radix_tree_insert (&kvm->arch.vsie.addr_to_page, addr >>9 , page)) { -page_ref_dec (page); +put_vsie_page (vsie_page);mutex_unlock (&kvm->arch.vsie.mutex);return NULL ; }
@@ -1431,14 +1453,6 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)return vsie_page; } -/* put a vsie page acquired via get_vsie_page */ -static void put_vsie_page (struct kvm *kvm,struct vsie_page *vsie_page) -{ -struct page *page =pfn_to_page (__pa (vsie_page) >>PAGE_SHIFT ); - -page_ref_dec (page); -} -int kvm_s390_handle_vsie (struct kvm_vcpu *vcpu) {struct vsie_page *vsie_page;
@@ -1489,7 +1503,7 @@ int kvm_s390_handle_vsie(struct kvm_vcpu *vcpu) out_unpin_scb:unpin_scb (vcpu, vsie_page, scb_addr); out_put: -put_vsie_page (vcpu->kvm, vsie_page); +put_vsie_page (vsie_page);return rc <0 ? rc :0 ; }
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 06/20] KVM: s390: wrapper for KVM_BUG
In reply to Claudio ImbrendaWrap the call to KVM_BUG; this reduces code duplication and improves readability.
Message-ID: <[email protected]>
--- arch/s390/kvm/kvm-s390.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
arch/s390/kvm/kvm-s390.c +10 -9
@@ -4766,6 +4766,13 @@ static int vcpu_post_run_addressing_exception(struct kvm_vcpu *vcpu)return kvm_s390_inject_prog_irq (vcpu, &pgm_info); } +static void kvm_s390_assert_primary_as (struct kvm_vcpu *vcpu) +{ +KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, +"Unexpected program interrupt 0x%x, TEID 0x%016lx" , + current->thread.gmap_int_code, current->thread.gmap_teid.val); +} +static int vcpu_post_run_handle_fault (struct kvm_vcpu *vcpu) {unsigned int flags =0 ;
@@ -4781,9 +4788,7 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu) vcpu->stat.exit_null++;break ;case PGM_NON_SECURE_STORAGE_ACCESS : -KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, -"Unexpected program interrupt 0x%x, TEID 0x%016lx" , - current->thread.gmap_int_code, current->thread.gmap_teid.val); +kvm_s390_assert_primary_as (vcpu);/* * This is normal operation; a page belonging to a protected * guest has not been imported yet. Try to import the page into
@@ -4794,9 +4799,7 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu)break ;case PGM_SECURE_STORAGE_ACCESS :case PGM_SECURE_STORAGE_VIOLATION : -KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, -"Unexpected program interrupt 0x%x, TEID 0x%016lx" , - current->thread.gmap_int_code, current->thread.gmap_teid.val); +kvm_s390_assert_primary_as (vcpu);/* * This can happen after a reboot with asynchronous teardown; * the new guest (normal or protected) will run on top of the
@@ -4825,9 +4828,7 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu)case PGM_REGION_FIRST_TRANS :case PGM_REGION_SECOND_TRANS :case PGM_REGION_THIRD_TRANS : -KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, -"Unexpected program interrupt 0x%x, TEID 0x%016lx" , - current->thread.gmap_int_code, current->thread.gmap_teid.val); +kvm_s390_assert_primary_as (vcpu);if (vcpu->arch.gmap->pfault_enabled) { rc =gmap_fault (vcpu->arch.gmap, gaddr, flags |FAULT_FLAG_RETRY_NOWAIT );if (rc == -EFAULT )
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 09/20] KVM: s390: move pv gmap functions into kvm
In reply to Claudio ImbrendaMove gmap related functions from kernel/uv into kvm.
Create a new file to collect gmap-related functions.
Message-ID: <[email protected]>
--- arch/s390/include/asm/gmap.h | 1 + arch/s390/include/asm/uv.h | 6 +- arch/s390/kernel/uv.c | 292 ++++------------------------------- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gmap.c | 212 +++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 17 ++ arch/s390/kvm/intercept.c | 3 +- arch/s390/kvm/kvm-s390.c | 1 + arch/s390/kvm/pv.c | 21 +++ arch/s390/mm/gmap.c | 28 ++++ 10 files changed, 315 insertions(+), 268 deletions(-) create mode 100644 arch/s390/kvm/gmap.c create mode 100644 arch/s390/kvm/gmap.h
arch/s390/include/asm/gmap.h +1 -0
@@ -149,6 +149,7 @@ int s390_replace_asce(struct gmap *gmap);void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns);int __s390_uv_destroy_range (struct mm_struct *mm,unsigned long start,unsigned long end,bool interruptible); +int kvm_s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio,bool split);/** * s390_uv_destroy_range - Destroy a range of pages in the given mm.
arch/s390/include/asm/uv.h +3 -3
@@ -628,12 +628,12 @@ static inline int is_prot_virt_host(void) }int uv_pin_shared (unsigned long paddr); -int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); -int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);int uv_destroy_folio (struct folio *folio);int uv_destroy_pte (pte_t pte);int uv_convert_from_secure_pte (pte_t pte); -int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr); +int make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb); +int uv_convert_from_secure (unsigned long paddr); +int uv_convert_from_secure_folio (struct folio *folio);void setup_uv (void );
arch/s390/kernel/uv.c +29 -263
342 lines not on this page · raw message
arch/s390/kvm/Makefile +1 -1
@@ -8,7 +8,7 @@ include $(srctree)/virt/kvm/Makefile.kvm ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o -kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o +kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/gmap.c new file +212 -0
212 lines not on this page · raw message
arch/s390/kvm/gmap.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* KVM guest address space mapping code +* +* Copyright IBM Corp. 2007, 2016, 2025 +* Author(s): Martin Schwidefsky <[email protected]> +* Claudio Imbrenda <[email protected]> +*/ + +#ifndef ARCH_KVM_S390_GMAP_H +#define ARCH_KVM_S390_GMAP_H + +int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); +int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr); +int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr); + +#endif
arch/s390/kvm/intercept.c +2 -1
@@ -21,6 +21,7 @@#include "gaccess.h" #include "trace.h" #include "trace-s390.h" +#include "gmap.h" u8 kvm_s390_get_ilen (struct kvm_vcpu *vcpu) {
@@ -549,7 +550,7 @@ static int handle_pv_uvc(struct kvm_vcpu *vcpu)* If the unpin did not succeed, the guest will exit again for the UVC * and we will retry the unpin. */ -if (rc == -EINVAL ) +if (rc == -EINVAL || rc == -ENXIO )return 0 ;/* * If we got -EAGAIN here, we simply return it. It will eventually
arch/s390/kvm/kvm-s390.c +1 -0
@@ -50,6 +50,7 @@#include "kvm-s390.h" #include "gaccess.h" #include "pci.h" +#include "gmap.h" #define CREATE_TRACE_POINTS #include "trace.h"
arch/s390/kvm/pv.c +21 -0
@@ -17,6 +17,7 @@#include <linux/sched/mm.h> #include <linux/mmu_notifier.h> #include "kvm-s390.h" +#include "gmap.h" bool kvm_s390_pv_is_protected (struct kvm *kvm) {
@@ -638,10 +639,28 @@ static int unpack_one(struct kvm *kvm, unsigned long addr, u64 tweak, .tweak[1] = offset, };int ret =gmap_make_secure (kvm->arch.gmap, addr, &uvcb); +unsigned long vmaddr; +bool unlocked; *rc = uvcb.header.rc; *rrc = uvcb.header.rrc; +if (ret == -ENXIO ) { +mmap_read_lock (kvm->mm); + vmaddr =gfn_to_hva (kvm,gpa_to_gfn (addr)); +if (kvm_is_error_hva (vmaddr)) { + ret = -EFAULT ; + }else { + ret =fixup_user_fault (kvm->mm, vmaddr,FAULT_FLAG_WRITE , &unlocked); +if (!ret) + ret =__gmap_link (kvm->arch.gmap, addr, vmaddr); + } +mmap_read_unlock (kvm->mm); +if (!ret) +return -EAGAIN ; +return ret; + } +if (ret && ret != -EAGAIN )KVM_UV_EVENT (kvm,3 ,"PROTVIRT VM UNPACK: failed addr %llx with rc %x rrc %x" , uvcb.gaddr, *rc, *rrc);
@@ -660,6 +679,8 @@ int kvm_s390_pv_unpack(struct kvm *kvm, unsigned long addr, unsigned long size,KVM_UV_EVENT (kvm,3 ,"PROTVIRT VM UNPACK: start addr %lx size %lx" , addr, size); +guard (srcu)(&kvm->srcu); +while (offset < size) { ret =unpack_one (kvm, addr, tweak, offset, rc, rrc);if (ret == -EAGAIN ) {
arch/s390/mm/gmap.c +28 -0
@@ -3035,3 +3035,31 @@ int s390_replace_asce(struct gmap *gmap)return 0 ; }EXPORT_SYMBOL_GPL (s390_replace_asce); + +/** +* kvm_s390_wiggle_split_folio() - try to drain extra references to a folio and optionally split +* @mm: the mm containing the folio to work on +* @folio: the folio +* @split: whether to split a large folio +* +* Context: Must be called while holding an extra reference to the folio; +* the mm lock should not be held. +*/ +int kvm_s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio,bool split) +{ +int rc; + +lockdep_assert_not_held (&mm->mmap_lock); +folio_wait_writeback (folio); +lru_add_drain_all (); +if (split) { +folio_lock (folio); + rc =split_folio (folio); +folio_unlock (folio); + +if (rc != -EBUSY ) +return rc; + } +return -EAGAIN ; +} +EXPORT_SYMBOL_GPL (kvm_s390_wiggle_split_folio);
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 12/20] KVM: s390: get rid of gmap_translate()
In reply to Claudio ImbrendaAdd gpa_to_hva(), which uses memslots, and use it to replace all uses of gmap_translate().
Message-ID: <[email protected]>
--- arch/s390/include/asm/gmap.h | 1 - arch/s390/kvm/interrupt.c | 19 +++++++++++-------- arch/s390/kvm/kvm-s390.h | 9 +++++++++ arch/s390/mm/gmap.c | 20 -------------------- 4 files changed, 20 insertions(+), 29 deletions(-)
arch/s390/include/asm/gmap.h +0 -1
@@ -111,7 +111,6 @@ int gmap_map_segment(struct gmap *gmap, unsigned long from, unsigned long to, unsigned long len);int gmap_unmap_segment (struct gmap *gmap,unsigned long to,unsigned long len);unsigned long __gmap_translate (struct gmap *,unsigned long gaddr); -unsigned long gmap_translate (struct gmap *,unsigned long gaddr);int __gmap_link (struct gmap *gmap,unsigned long gaddr,unsigned long vmaddr);void gmap_discard (struct gmap *,unsigned long from,unsigned long to);void __gmap_zap (struct gmap *,unsigned long gaddr);
arch/s390/kvm/interrupt.c +11 -8
@@ -2893,7 +2893,8 @@ int kvm_set_routing_entry(struct kvm *kvm,struct kvm_kernel_irq_routing_entry *e, const struct kvm_irq_routing_entry *ue) { - u64 uaddr; + u64 uaddr_s, uaddr_i; +int idx;switch (ue->type) {/* we store the userspace addresses instead of the guest addresses */
@@ -2901,14 +2902,16 @@ int kvm_set_routing_entry(struct kvm *kvm,if (kvm_is_ucontrol (kvm))return -EINVAL ; e->set = set_adapter_int; - uaddr =gmap_translate (kvm->arch.gmap, ue->u.adapter.summary_addr); -if (uaddr == -EFAULT ) -return -EFAULT ; - e->adapter.summary_addr = uaddr; - uaddr =gmap_translate (kvm->arch.gmap, ue->u.adapter.ind_addr); -if (uaddr == -EFAULT ) + + idx =srcu_read_lock (&kvm->srcu); + uaddr_s =gpa_to_hva (kvm, ue->u.adapter.summary_addr); + uaddr_i =gpa_to_hva (kvm, ue->u.adapter.ind_addr); +srcu_read_unlock (&kvm->srcu, idx); + +if (kvm_is_error_hva (uaddr_s) ||kvm_is_error_hva (uaddr_i))return -EFAULT ; - e->adapter.ind_addr = uaddr; + e->adapter.summary_addr = uaddr_s; + e->adapter.ind_addr = uaddr_i; e->adapter.summary_offset = ue->u.adapter.summary_offset; e->adapter.ind_offset = ue->u.adapter.ind_offset; e->adapter.adapter_id = ue->u.adapter.adapter_id;
arch/s390/kvm/kvm-s390.h +9 -0
@@ -281,6 +281,15 @@ static inline u32 kvm_s390_get_gisa_desc(struct kvm *kvm)return gd; } +static inline hva_t gpa_to_hva (struct kvm *kvm,gpa_t gpa) +{ +hva_t hva =gfn_to_hva (kvm,gpa_to_gfn (gpa)); + +if (!kvm_is_error_hva (hva)) + hva |=offset_in_page (gpa); +return hva; +} +/* implemented in pv.c */ int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc);int kvm_s390_pv_create_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc);
arch/s390/mm/gmap.c +0 -20
@@ -463,26 +463,6 @@ unsigned long __gmap_translate(struct gmap *gmap, unsigned long gaddr) }EXPORT_SYMBOL_GPL (__gmap_translate); -/** -* gmap_translate - translate a guest address to a user space address -* @gmap: pointer to guest mapping meta data structure -* @gaddr: guest address -* -* Returns user space address which corresponds to the guest address or -* -EFAULT if no such mapping exists. -* This function does not establish potentially missing page table entries. -*/ -unsigned long gmap_translate (struct gmap *gmap,unsigned long gaddr) -{ -unsigned long rc; - -mmap_read_lock (gmap->mm); - rc =__gmap_translate (gmap, gaddr); -mmap_read_unlock (gmap->mm); -return rc; -} -EXPORT_SYMBOL_GPL (gmap_translate); -/** * gmap_unlink - disconnect a page table from the gmap shadow tables * @mm: pointer to the parent mm_struct
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 02/20] KVM: s390: vsie: stop using page->index
In reply to Claudio ImbrendaFrom: David Hildenbrand <[email protected]>
Let's stop using page->index, and instead use a field inside "struct vsie_page" to hold that value. We have plenty of space left in there.
This is one part of stopping using "struct page" when working with vsie pages. We place the "page_to_virt(page)" strategically, so the next cleanups requires less churn.
Message-ID: <[email protected]>
--- arch/s390/kvm/vsie.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-)
arch/s390/kvm/vsie.c +18 -10
@@ -46,7 +46,13 @@ struct vsie_page {gpa_t gvrd_gpa;/* 0x0240 */ gpa_t riccbd_gpa;/* 0x0248 */ gpa_t sdnx_gpa;/* 0x0250 */ -__u8 reserved[0x0700 -0x0258 ];/* 0x0258 */ +/* +* guest address of the original SCB. Remains set for free vsie +* pages, so we can properly look them up in our addr_to_page +* radix tree. +*/ +gpa_t scb_gpa;/* 0x0258 */ +__u8 reserved[0x0700 -0x0260 ];/* 0x0260 */ struct kvm_s390_crypto_cb crycb;/* 0x0700 */ __u8 fac[S390_ARCH_FAC_LIST_SIZE_BYTE ];/* 0x0800 */ };
@@ -1362,9 +1368,10 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr) page =radix_tree_lookup (&kvm->arch.vsie.addr_to_page, addr >>9 );rcu_read_unlock ();if (page) { + vsie_page =page_to_virt (page);if (page_ref_inc_return (page) ==2 ) { -if (page->index == addr) -return page_to_virt (page); +if (vsie_page->scb_gpa == addr) +return vsie_page;/* * We raced with someone reusing + putting this vsie * page before we grabbed it.
@@ -1386,6 +1393,7 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)mutex_unlock (&kvm->arch.vsie.mutex);return ERR_PTR (-ENOMEM ); } + vsie_page =page_to_virt (page);page_ref_inc (page); kvm->arch.vsie.pages[kvm->arch.vsie.page_count] = page; kvm->arch.vsie.page_count++;
@@ -1393,18 +1401,19 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)/* reuse an existing entry that belongs to nobody */ while (true) { page = kvm->arch.vsie.pages[kvm->arch.vsie.next]; + vsie_page =page_to_virt (page);if (page_ref_inc_return (page) ==2 )break ;page_ref_dec (page); kvm->arch.vsie.next++; kvm->arch.vsie.next %= nr_vcpus; } -if (page->index !=ULONG_MAX ) +if (vsie_page->scb_gpa !=ULONG_MAX )radix_tree_delete (&kvm->arch.vsie.addr_to_page, - page->index >>9 ); + vsie_page->scb_gpa >>9 ); }/* Mark it as invalid until it resides in the tree. */ - page->index =ULONG_MAX ; + vsie_page->scb_gpa =ULONG_MAX ;/* Double use of the same address or allocation failure. */ if (radix_tree_insert (&kvm->arch.vsie.addr_to_page, addr >>9 , page)) {
@@ -1412,10 +1421,9 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)mutex_unlock (&kvm->arch.vsie.mutex);return NULL ; } - page->index = addr; + vsie_page->scb_gpa = addr;mutex_unlock (&kvm->arch.vsie.mutex); - vsie_page =page_to_virt (page);memset (&vsie_page->scb_s,0 ,sizeof (struct kvm_s390_sie_block ));release_gmap_shadow (vsie_page); vsie_page->fault_addr =0 ;
@@ -1507,9 +1515,9 @@ void kvm_s390_vsie_destroy(struct kvm *kvm) vsie_page =page_to_virt (page);release_gmap_shadow (vsie_page);/* free the radix tree entry */ -if (page->index !=ULONG_MAX ) +if (vsie_page->scb_gpa !=ULONG_MAX )radix_tree_delete (&kvm->arch.vsie.addr_to_page, - page->index >>9 ); + vsie_page->scb_gpa >>9 );__free_page (page); } kvm->arch.vsie.page_count =0 ;
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 05/20] KVM: Do not restrict the size of KVM-internal memory regions
In reply to Claudio ImbrendaFrom: Sean Christopherson <[email protected]>
Exempt KVM-internal memslots from the KVM_MEM_MAX_NR_PAGES restriction, as the limit on the number of pages exists purely to play nice with dirty bitmap operations, which use 32-bit values to index the bitmaps, and dirty logging isn't supported for KVM-internal memslots.
Message-ID: <[email protected]>
--- virt/kvm/kvm_main.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +9 -1
@@ -1971,7 +1971,15 @@ static int kvm_set_memory_region(struct kvm *kvm,return -EINVAL ;if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr)return -EINVAL ; -if ((mem->memory_size >>PAGE_SHIFT ) >KVM_MEM_MAX_NR_PAGES ) + +/* +* The size of userspace-defined memory regions is restricted in order +* to play nice with dirty bitmap operations, which are indexed with an +* "unsigned int". KVM's internal memory regions don't support dirty +* logging, and so are exempt. +*/ +if (id <KVM_USER_MEM_SLOTS && + (mem->memory_size >>PAGE_SHIFT ) >KVM_MEM_MAX_NR_PAGES )return -EINVAL ; slots =__kvm_memslots (kvm, as_id);
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 07/20] KVM: s390: fake memslot for ucontrol VMs
In reply to Claudio ImbrendaCreate a fake memslot for ucontrol VMs. The fake memslot identity-maps userspace.
Now memslots will always be present, and ucontrol is not a special case anymore.
Message-ID: <[email protected]>
--- Documentation/virt/kvm/api.rst | 2 +- arch/s390/include/asm/kvm_host.h | 2 ++ arch/s390/kvm/kvm-s390.c | 17 ++++++++++++++++- arch/s390/kvm/kvm-s390.h | 2 ++ 4 files changed, 21 insertions(+), 2 deletions(-)
Documentation/virt/kvm/api.rst +1 -1
@@ -1419,7 +1419,7 @@ fetch) is injected in the guest. S390: ^^^^^ -Returns -EINVAL if the VM has the KVM_VM_S390_UCONTROL flag set. +Returns -EINVAL or -EEXIST if the VM has the KVM_VM_S390_UCONTROL flag set. Returns -EINVAL if called on a protected VM. 4.36 KVM_SET_TSS_ADDR
arch/s390/include/asm/kvm_host.h +2 -0
@@ -30,6 +30,8 @@#define KVM_S390_ESCA_CPU_SLOTS 248#define KVM_MAX_VCPUS 255 +#define KVM_INTERNAL_MEM_SLOTS 1 +/* * These seem to be used for allocating ->chip in the routing table, which we * don't use. 1 is as small as we can get to reduce the needed memory. If we
arch/s390/kvm/kvm-s390.c +16 -1
@@ -3428,8 +3428,20 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)VM_EVENT (kvm,3 ,"vm created with type %lu" , type);if (type &KVM_VM_S390_UCONTROL ) { +struct kvm_userspace_memory_region2 fake_memslot = { + .slot =KVM_S390_UCONTROL_MEMSLOT , + .guest_phys_addr =0 , + .userspace_addr =0 , + .memory_size =ALIGN_DOWN (TASK_SIZE , _SEGMENT_SIZE), + .flags =0 , + }; + kvm->arch.gmap =NULL ; kvm->arch.mem_limit =KVM_S390_NO_MEM_LIMIT ; +/* one flat fake memslot covering the whole address-space */ +mutex_lock (&kvm->slots_lock); +KVM_BUG_ON (kvm_set_internal_memslot (kvm, &fake_memslot), kvm); +mutex_unlock (&kvm->slots_lock); }else {if (sclp.hamax ==U64_MAX ) kvm->arch.mem_limit =TASK_SIZE_MAX ;
@@ -5854,7 +5866,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, {gpa_t size; -if (kvm_is_ucontrol (kvm)) +if (kvm_is_ucontrol (kvm) && new->id <KVM_USER_MEM_SLOTS )return -EINVAL ;/* When we are protected, we should not change the memory slots */
@@ -5906,6 +5918,9 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, {int rc =0 ; +if (kvm_is_ucontrol (kvm)) +return ; +switch (change) {case KVM_MR_DELETE : rc = gmap_unmap_segment(kvm->arch.gmap, old->base_gfn * PAGE_SIZE,
arch/s390/kvm/kvm-s390.h +2 -0
@@ -20,6 +20,8 @@#include <asm/processor.h> #include <asm/sclp.h> +#define KVM_S390_UCONTROL_MEMSLOT (KVM_USER_MEM_SLOTS + 0) +static inline void kvm_s390_fpu_store (struct kvm_run *run) {fpu_stfpc (&run->s.regs.fpc);
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 10/20] KVM: s390: use __kvm_faultin_pfn()
In reply to Claudio ImbrendaRefactor the existing page fault handling code to use __kvm_faultin_pfn().
This possible now that memslots are always present.
Message-ID: <[email protected]>
--- arch/s390/kvm/kvm-s390.c | 126 ++++++++++++++++++++++++++++++--------- arch/s390/kvm/kvm-s390.h | 6 ++ arch/s390/mm/gmap.c | 1 + 3 files changed, 106 insertions(+), 27 deletions(-)
arch/s390/kvm/kvm-s390.c +99 -27
@@ -4786,11 +4786,104 @@ static void kvm_s390_assert_primary_as(struct kvm_vcpu *vcpu) current->thread.gmap_int_code, current->thread.gmap_teid.val); } +/* +* __kvm_s390_handle_dat_fault() - handle a dat fault for the gmap of a vcpu +* @vcpu: the vCPU whose gmap is to be fixed up +* @gfn: the guest frame number used for memslots (including fake memslots) +* @gaddr: the gmap address, does not have to match @gfn for ucontrol gmaps +* @flags: FOLL_* flags +* +* Return: 0 on success, < 0 in case of error. +* Context: The mm lock must not be held before calling. May sleep. +*/ +int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags) +{ +struct kvm_memory_slot *slot; +unsigned int fault_flags; +bool writable, unlocked; +unsigned long vmaddr; +struct page *page; +kvm_pfn_t pfn; +int rc; + + slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn); +if (!slot || slot->flags &KVM_MEMSLOT_INVALID ) +return vcpu_post_run_addressing_exception (vcpu); + + fault_flags = flags &FOLL_WRITE ?FAULT_FLAG_WRITE :0 ; +if (vcpu->arch.gmap->pfault_enabled) + flags |=FOLL_NOWAIT ; + vmaddr =__gfn_to_hva_memslot (slot, gfn); + +try_again: + pfn =__kvm_faultin_pfn (slot, gfn, flags, &writable, &page); + +/* Access outside memory, inject addressing exception */ +if (is_noslot_pfn (pfn)) +return vcpu_post_run_addressing_exception (vcpu); +/* Signal pending: try again */ +if (pfn ==KVM_PFN_ERR_SIGPENDING ) +return -EAGAIN ; + +/* Needs I/O, try to setup async pfault (only possible with FOLL_NOWAIT) */ +if (pfn ==KVM_PFN_ERR_NEEDS_IO ) { +trace_kvm_s390_major_guest_pfault (vcpu); +if (kvm_arch_setup_async_pf (vcpu)) +return 0 ; + vcpu->stat.pfault_sync++; +/* Could not setup async pfault, try again synchronously */ + flags &= ~FOLL_NOWAIT ; + goto try_again; + } +/* Any other error */ +if (is_error_pfn (pfn)) +return -EFAULT ; + +/* Success */ +mmap_read_lock (vcpu->arch.gmap->mm); +/* Mark the userspace PTEs as young and/or dirty, to avoid page fault loops */ + rc =fixup_user_fault (vcpu->arch.gmap->mm, vmaddr, fault_flags, &unlocked); +if (!rc) + rc =__gmap_link (vcpu->arch.gmap, gaddr, vmaddr); + scoped_guard(spinlock, &vcpu->kvm->mmu_lock) { +kvm_release_faultin_page (vcpu->kvm, page, false, writable); + } +mmap_read_unlock (vcpu->arch.gmap->mm); +return rc; +} + +static int vcpu_dat_fault_handler (struct kvm_vcpu *vcpu,unsigned long gaddr,unsigned int flags) +{ +unsigned long gaddr_tmp; +gfn_t gfn; + + gfn =gpa_to_gfn (gaddr); +if (kvm_is_ucontrol (vcpu->kvm)) { +/* +* This translates the per-vCPU guest address into a +* fake guest address, which can then be used with the +* fake memslots that are identity mapping userspace. +* This allows ucontrol VMs to use the normal fault +* resolution path, like normal VMs. +*/ +mmap_read_lock (vcpu->arch.gmap->mm); + gaddr_tmp =__gmap_translate (vcpu->arch.gmap, gaddr); +mmap_read_unlock (vcpu->arch.gmap->mm); +if (gaddr_tmp == -EFAULT ) { + vcpu->run->exit_reason =KVM_EXIT_S390_UCONTROL ; + vcpu->run->s390_ucontrol.trans_exc_code = gaddr; + vcpu->run->s390_ucontrol.pgm_code =PGM_SEGMENT_TRANSLATION ; +return -EREMOTE ; + } + gfn =gpa_to_gfn (gaddr_tmp); + } +return __kvm_s390_handle_dat_fault (vcpu, gfn, gaddr, flags); +} +static int vcpu_post_run_handle_fault (struct kvm_vcpu *vcpu) {unsigned int flags =0 ;unsigned long gaddr; -int rc =0 ; gaddr = current->thread.gmap_teid.addr *PAGE_SIZE ;if (kvm_s390_cur_gmap_fault_is_write ())
@@ -4842,37 +4935,14 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu)case PGM_REGION_SECOND_TRANS :case PGM_REGION_THIRD_TRANS :kvm_s390_assert_primary_as (vcpu); -if (vcpu->arch.gmap->pfault_enabled) { - rc =gmap_fault (vcpu->arch.gmap, gaddr, flags |FAULT_FLAG_RETRY_NOWAIT ); -if (rc == -EFAULT ) -return vcpu_post_run_addressing_exception (vcpu); -if (rc == -EAGAIN ) { -trace_kvm_s390_major_guest_pfault (vcpu); -if (kvm_arch_setup_async_pf (vcpu)) -return 0 ; - vcpu->stat.pfault_sync++; - }else { -return rc; - } - } - rc =gmap_fault (vcpu->arch.gmap, gaddr, flags); -if (rc == -EFAULT ) { -if (kvm_is_ucontrol (vcpu->kvm)) { - vcpu->run->exit_reason =KVM_EXIT_S390_UCONTROL ; - vcpu->run->s390_ucontrol.trans_exc_code = gaddr; - vcpu->run->s390_ucontrol.pgm_code =0x10 ; -return -EREMOTE ; - } -return vcpu_post_run_addressing_exception (vcpu); - } -break ; +return vcpu_dat_fault_handler (vcpu, gaddr, flags);default :KVM_BUG (1 , vcpu->kvm,"Unexpected program interrupt 0x%x, TEID 0x%016lx" , current->thread.gmap_int_code, current->thread.gmap_teid.val);send_sig (SIGSEGV , current,0 );break ; } -return rc; +return 0 ; }static int vcpu_post_run (struct kvm_vcpu *vcpu,int exit_reason)
@@ -5751,7 +5821,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp, }#endif case KVM_S390_VCPU_FAULT : { - r =gmap_fault (vcpu->arch.gmap, arg,0 ); + idx =srcu_read_lock (&vcpu->kvm->srcu); + r =vcpu_dat_fault_handler (vcpu, arg,0 ); +srcu_read_unlock (&vcpu->kvm->srcu, idx);break ; }case KVM_ENABLE_CAP :
arch/s390/kvm/kvm-s390.h +6 -0
@@ -410,6 +410,12 @@ void kvm_s390_vcpu_unsetup_cmma(struct kvm_vcpu *vcpu);void kvm_s390_set_cpu_timer (struct kvm_vcpu *vcpu,__u64 cputm);__u64 kvm_s390_get_cpu_timer (struct kvm_vcpu *vcpu);int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc); +int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags); + +static inline int kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gpa_t gaddr,unsigned int flags) +{ +return __kvm_s390_handle_dat_fault (vcpu,gpa_to_gfn (gaddr), gaddr, flags); +}/* implemented in diag.c */ int kvm_s390_handle_diag (struct kvm_vcpu *vcpu);
arch/s390/mm/gmap.c +1 -0
@@ -605,6 +605,7 @@ int __gmap_link(struct gmap *gmap, unsigned long gaddr, unsigned long vmaddr)radix_tree_preload_end ();return rc; } +EXPORT_SYMBOL (__gmap_link);/** * fixup_user_fault_nowait - manually resolve a user page fault without waiting
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 11/20] KVM: s390: get rid of gmap_fault()
In reply to Claudio ImbrendaAll gmap page faults are already handled in kvm by the function kvm_s390_handle_dat_fault(); only few users of gmap_fault remained, all within kvm.
Convert those calls to use kvm_s390_handle_dat_fault() instead.
Remove gmap_fault() entirely since it has no more users.
Message-ID: <[email protected]>
--- arch/s390/include/asm/gmap.h | 1 - arch/s390/kvm/intercept.c | 4 +- arch/s390/mm/gmap.c | 124 ----------------------------------- 3 files changed, 2 insertions(+), 127 deletions(-)
arch/s390/include/asm/gmap.h +0 -1
@@ -113,7 +113,6 @@ int gmap_unmap_segment(struct gmap *gmap, unsigned long to, unsigned long len);unsigned long __gmap_translate (struct gmap *,unsigned long gaddr);unsigned long gmap_translate (struct gmap *,unsigned long gaddr);int __gmap_link (struct gmap *gmap,unsigned long gaddr,unsigned long vmaddr); -int gmap_fault (struct gmap *,unsigned long gaddr,unsigned int fault_flags);void gmap_discard (struct gmap *,unsigned long from,unsigned long to);void __gmap_zap (struct gmap *,unsigned long gaddr);void gmap_unlink (struct mm_struct *,unsigned long *table,unsigned long vmaddr);
arch/s390/kvm/intercept.c +2 -2
@@ -368,7 +368,7 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu) reg2, &srcaddr, GACC_FETCH, 0);if (rc)return kvm_s390_inject_prog_cond (vcpu, rc); - rc =gmap_fault (vcpu->arch.gmap, srcaddr,0 ); + rc =kvm_s390_handle_dat_fault (vcpu, srcaddr,0 );if (rc !=0 )return rc;
@@ -377,7 +377,7 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu) reg1, &dstaddr, GACC_STORE, 0);if (rc)return kvm_s390_inject_prog_cond (vcpu, rc); - rc =gmap_fault (vcpu->arch.gmap, dstaddr,FAULT_FLAG_WRITE ); + rc =kvm_s390_handle_dat_fault (vcpu, dstaddr,FOLL_WRITE );if (rc !=0 )return rc;
arch/s390/mm/gmap.c +0 -124
130 lines not on this page · raw message
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 14/20] KVM: s390: stop using page->index for non-shadow gmaps
In reply to Claudio ImbrendaThe host_to_guest radix tree will now map userspace addresses to guest addresses, instead of userspace addresses to segment tables.
When segment tables and page tables are needed, they are found using an additional gmap_table_walk().
This gets rid of all usage of page->index for non-shadow gmaps.
Message-ID: <[email protected]>
--- arch/s390/mm/gmap.c | 105 +++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 51 deletions(-)
arch/s390/mm/gmap.c +54 -51
@@ -24,8 +24,20 @@#include <asm/page.h> #include <asm/tlb.h> +/* +* The address is saved in a radix tree directly; NULL would be ambiguous, +* since 0 is a valid address, and NULL is returned when nothing was found. +* The lower bits are ignored by all users of the macro, so it can be used +* to distinguish a valid address 0 from a NULL. +*/ +#define VALID_GADDR_FLAG 1 +#define IS_GADDR_VALID (gaddr) ((gaddr) & VALID_GADDR_FLAG) +#define MAKE_VALID_GADDR (gaddr) (((gaddr) & HPAGE_MASK) | VALID_GADDR_FLAG) +#define GMAP_SHADOW_FAKE_TABLE 1ULL +static inline unsigned long *gmap_table_walk (struct gmap *gmap,unsigned long gaddr,int level); +static struct page *gmap_alloc_crst (void ) {struct page *page;
@@ -82,7 +94,6 @@ struct gmap *gmap_alloc(unsigned long limit) page =gmap_alloc_crst ();if (!page) goto out_free; - page->index =0 ;list_add (&page->lru, &gmap->crst_list); table =page_to_virt (page);crst_table_init (table, etype);
@@ -303,7 +314,6 @@ static int gmap_alloc_table(struct gmap *gmap, unsigned long *table,list_add (&page->lru, &gmap->crst_list); *table =__pa (new) | _REGION_ENTRY_LENGTH | (*table & _REGION_ENTRY_TYPE_MASK); - page->index = gaddr; page =NULL ; }spin_unlock (&gmap->guest_table_lock);
@@ -312,21 +322,23 @@ static int gmap_alloc_table(struct gmap *gmap, unsigned long *table,return 0 ; } -/** -* __gmap_segment_gaddr - find virtual address from segment pointer -* @entry: pointer to a segment table entry in the guest address space -* -* Returns the virtual address in the guest address space for the segment -*/ -static unsigned long __gmap_segment_gaddr (unsigned long *entry) +static unsigned long host_to_guest_lookup (struct gmap *gmap,unsigned long vmaddr) { -struct page *page; -unsigned long offset; +return (unsigned long )radix_tree_lookup (&gmap->host_to_guest, vmaddr >>PMD_SHIFT ); +} - offset = (unsigned long ) entry /sizeof (unsigned long ); - offset = (offset & (PTRS_PER_PMD -1 )) *PMD_SIZE ; - page =pmd_pgtable_page ((pmd_t *) entry); -return page->index + offset; +static unsigned long host_to_guest_delete (struct gmap *gmap,unsigned long vmaddr) +{ +return (unsigned long )radix_tree_delete (&gmap->host_to_guest, vmaddr >>PMD_SHIFT ); +} + +static pmd_t *host_to_guest_pmd_delete (struct gmap *gmap,unsigned long vmaddr, +unsigned long *gaddr) +{ + *gaddr =host_to_guest_delete (gmap, vmaddr); +if (IS_GADDR_VALID (*gaddr)) +return (pmd_t *)gmap_table_walk (gmap, *gaddr,1 ); +return NULL ; }/**
@@ -338,16 +350,19 @@ static unsigned long __gmap_segment_gaddr(unsigned long *entry)*/ static int __gmap_unlink_by_vmaddr (struct gmap *gmap,unsigned long vmaddr) { -unsigned long *entry; +unsigned long gaddr;int flush =0 ; +pmd_t *pmdp;BUG_ON (gmap_is_shadow (gmap));spin_lock (&gmap->guest_table_lock); - entry =radix_tree_delete (&gmap->host_to_guest, vmaddr >>PMD_SHIFT ); -if (entry) { - flush = (*entry != _SEGMENT_ENTRY_EMPTY); - *entry = _SEGMENT_ENTRY_EMPTY; + + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr); +if (pmdp) { + flush = (pmd_val (*pmdp) != _SEGMENT_ENTRY_EMPTY); + *pmdp =__pmd (_SEGMENT_ENTRY_EMPTY); } +spin_unlock (&gmap->guest_table_lock);return flush; }
@@ -564,7 +579,8 @@ int __gmap_link(struct gmap *gmap, unsigned long gaddr, unsigned long vmaddr)spin_lock (&gmap->guest_table_lock);if (*table == _SEGMENT_ENTRY_EMPTY) { rc =radix_tree_insert (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT , table); + vmaddr >>PMD_SHIFT , + (void *)MAKE_VALID_GADDR (gaddr));if (!rc) {if (pmd_leaf (*pmd)) { *table = (pmd_val(*pmd) &
@@ -1995,7 +2011,6 @@ void ptep_notify(struct mm_struct *mm, unsigned long vmaddr, pte_t *pte, unsigned long bits) {unsigned long offset, gaddr =0 ; -unsigned long *table;struct gmap *gmap, *sg, *next; offset = ((unsigned long ) pte) & (255 *sizeof (pte_t));
@@ -2003,12 +2018,9 @@ void ptep_notify(struct mm_struct *mm, unsigned long vmaddr,rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - table =radix_tree_lookup (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); -if (table) - gaddr =__gmap_segment_gaddr (table) + offset; + gaddr =host_to_guest_lookup (gmap, vmaddr) + offset;spin_unlock (&gmap->guest_table_lock); -if (!table) +if (!IS_GADDR_VALID (gaddr))continue ;if (!list_empty (&gmap->children) && (bits &PGSTE_VSIE_BIT )) {
@@ -2068,10 +2080,8 @@ static void gmap_pmdp_clear(struct mm_struct *mm, unsigned long vmaddr,rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - pmdp = (pmd_t *)radix_tree_delete (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr);if (pmdp) { - gaddr =__gmap_segment_gaddr ((unsigned long *)pmdp);pmdp_notify_gmap (gmap, pmdp, gaddr); WARN_ON(pmd_val(*pmdp) & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | _SEGMENT_ENTRY_GMAP_UC |
@@ -2115,28 +2125,25 @@ EXPORT_SYMBOL_GPL(gmap_pmdp_csp);*/ void gmap_pmdp_idte_local (struct mm_struct *mm,unsigned long vmaddr) { -unsigned long *entry, gaddr; +unsigned long gaddr;struct gmap *gmap;pmd_t *pmdp;rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - entry =radix_tree_delete (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); -if (entry) { - pmdp = (pmd_t *)entry; - gaddr =__gmap_segment_gaddr (entry); + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr); +if (pmdp) {pmdp_notify_gmap (gmap, pmdp, gaddr); -WARN_ON (*entry & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | - _SEGMENT_ENTRY_GMAP_UC | - _SEGMENT_ENTRY)); +WARN_ON (pmd_val (*pmdp) & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | + _SEGMENT_ENTRY_GMAP_UC | + _SEGMENT_ENTRY));if (MACHINE_HAS_TLB_GUEST )__pmdp_idte (gaddr, pmdp,IDTE_GUEST_ASCE , gmap->asce,IDTE_LOCAL );else if (MACHINE_HAS_IDTE )__pmdp_idte (gaddr, pmdp,0 ,0 ,IDTE_LOCAL ); - *entry = _SEGMENT_ENTRY_EMPTY; + *pmdp =__pmd (_SEGMENT_ENTRY_EMPTY); }spin_unlock (&gmap->guest_table_lock); }
@@ -2151,22 +2158,19 @@ EXPORT_SYMBOL_GPL(gmap_pmdp_idte_local);*/ void gmap_pmdp_idte_global (struct mm_struct *mm,unsigned long vmaddr) { -unsigned long *entry, gaddr; +unsigned long gaddr;struct gmap *gmap;pmd_t *pmdp;rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - entry =radix_tree_delete (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); -if (entry) { - pmdp = (pmd_t *)entry; - gaddr =__gmap_segment_gaddr (entry); + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr); +if (pmdp) {pmdp_notify_gmap (gmap, pmdp, gaddr); -WARN_ON (*entry & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | - _SEGMENT_ENTRY_GMAP_UC | - _SEGMENT_ENTRY)); +WARN_ON (pmd_val (*pmdp) & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | + _SEGMENT_ENTRY_GMAP_UC | + _SEGMENT_ENTRY));if (MACHINE_HAS_TLB_GUEST )__pmdp_idte (gaddr, pmdp,IDTE_GUEST_ASCE , gmap->asce,IDTE_GLOBAL );
@@ -2174,7 +2178,7 @@ void gmap_pmdp_idte_global(struct mm_struct *mm, unsigned long vmaddr)__pmdp_idte (gaddr, pmdp,0 ,0 ,IDTE_GLOBAL ); else__pmdp_csp (pmdp ); - *entry = _SEGMENT_ENTRY_EMPTY; + *pmdp =__pmd (_SEGMENT_ENTRY_EMPTY); }spin_unlock (&gmap->guest_table_lock); }
@@ -2690,7 +2694,6 @@ int s390_replace_asce(struct gmap *gmap) page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index =0 ; table =page_to_virt (page);memcpy (table, gmap->table,1UL << (CRST_ALLOC_ORDER +PAGE_SHIFT ));
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 16/20] KVM: s390: move gmap_shadow_pgt_lookup() into kvm
In reply to Claudio ImbrendaMove gmap_shadow_pgt_lookup() from mm/gmap.c into kvm/gaccess.c .
Message-ID: <[email protected]>
--- arch/s390/include/asm/gmap.h | 3 +-- arch/s390/kvm/gaccess.c | 42 +++++++++++++++++++++++++++++++- arch/s390/kvm/gmap.h | 2 ++ arch/s390/mm/gmap.c | 46 ++---------------------------------- 4 files changed, 46 insertions(+), 47 deletions(-)
arch/s390/include/asm/gmap.h +1 -2
@@ -125,8 +125,6 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, int fake);int gmap_shadow_pgt (struct gmap *sg,unsigned long saddr,unsigned long pgt,int fake); -int gmap_shadow_pgt_lookup (struct gmap *sg,unsigned long saddr, -unsigned long *pgt,int *dat_protection,int *fake);int gmap_shadow_page (struct gmap *sg,unsigned long saddr,pte_t pte);void gmap_register_pte_notifier (struct gmap_notifier *);
@@ -142,6 +140,7 @@ void s390_uv_destroy_pfns(unsigned long count, unsigned long *pfns);int __s390_uv_destroy_range (struct mm_struct *mm,unsigned long start,unsigned long end,bool interruptible);int kvm_s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio,bool split); +unsigned long *gmap_table_walk (struct gmap *gmap,unsigned long gaddr,int level);/** * s390_uv_destroy_range - Destroy a range of pages in the given mm.
arch/s390/kvm/gaccess.c +41 -1
66 lines not on this page · raw message
arch/s390/kvm/gmap.h +2 -0
@@ -10,6 +10,8 @@#ifndef ARCH_KVM_S390_GMAP_H #define ARCH_KVM_S390_GMAP_H +#define GMAP_SHADOW_FAKE_TABLE 1ULL +int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);
arch/s390/mm/gmap.c +2 -44
70 lines not on this page · raw message
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 18/20] KVM: s390: move PGSTE softbits
In reply to Claudio ImbrendaMove the softbits in the PGSTEs to the other usable area.
This leaves the 16-bit block of usable bits free, which will be used in the next patch for something else.
Message-ID: <[email protected]>
--- arch/s390/include/asm/pgtable.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/s390/include/asm/pgtable.h +3 -3
@@ -420,9 +420,9 @@ void setup_protection_map(void);#define PGSTE_HC_BIT 0x0020000000000000UL#define PGSTE_GR_BIT 0x0004000000000000UL#define PGSTE_GC_BIT 0x0002000000000000UL -#define PGSTE_UC_BIT 0x0000800000000000UL/* user dirty (migration) */ -#define PGSTE_IN_BIT 0x0000400000000000UL/* IPTE notify bit */ -#define PGSTE_VSIE_BIT 0x0000200000000000UL/* ref'd in a shadow table */ +#define PGSTE_UC_BIT 0x0000000000008000UL/* user dirty (migration) */ +#define PGSTE_IN_BIT 0x0000000000004000UL/* IPTE notify bit */ +#define PGSTE_VSIE_BIT 0x0000000000002000UL/* ref'd in a shadow table */ /* Guest Page State used for virtualization */ #define _PGSTE_GPS_ZERO 0x0000000080000000UL
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 04/20] KVM: s390: vsie: stop using "struct page" for vsie page
In reply to Claudio ImbrendaFrom: David Hildenbrand <[email protected]>
Now that we no longer use page->index and the page refcount explicitly, let's avoid messing with "struct page" completely.
Message-ID: <[email protected]>
--- arch/s390/include/asm/kvm_host.h | 4 +++- arch/s390/kvm/vsie.c | 31 ++++++++++++------------------- 2 files changed, 15 insertions(+), 20 deletions(-)
arch/s390/include/asm/kvm_host.h +3 -1
@@ -931,12 +931,14 @@ struct sie_page2 {u8 reserved928[0x1000 -0x928 ];/* 0x0928 */ }; +struct vsie_page ; +struct kvm_s390_vsie {struct mutex mutex;struct radix_tree_root addr_to_page;int page_count;int next; -struct page *pages[KVM_MAX_VCPUS ]; +struct vsie_page *pages[KVM_MAX_VCPUS ]; };struct kvm_s390_gisa_iam {
arch/s390/kvm/vsie.c +12 -19
@@ -599,7 +599,6 @@ void kvm_s390_vsie_gmap_notifier(struct gmap *gmap, unsigned long start,struct kvm *kvm = gmap->private;struct vsie_page *cur;unsigned long prefix; -struct page *page;int i;if (!gmap_is_shadow (gmap))
@@ -609,10 +608,9 @@ void kvm_s390_vsie_gmap_notifier(struct gmap *gmap, unsigned long start,* therefore we can safely reference them all the time. */ for (i =0 ; i < kvm->arch.vsie.page_count; i++) { - page =READ_ONCE (kvm->arch.vsie.pages[i]); -if (!page) + cur =READ_ONCE (kvm->arch.vsie.pages[i]); +if (!cur)continue ; - cur =page_to_virt (page);if (READ_ONCE (cur->gmap) != gmap)continue ; prefix = cur->scb_s.prefix <<GUEST_PREFIX_SHIFT ;
@@ -1384,14 +1382,12 @@ static void put_vsie_page(struct vsie_page *vsie_page)static struct vsie_page *get_vsie_page (struct kvm *kvm,unsigned long addr) {struct vsie_page *vsie_page; -struct page *page;int nr_vcpus;rcu_read_lock (); - page =radix_tree_lookup (&kvm->arch.vsie.addr_to_page, addr >>9 ); + vsie_page =radix_tree_lookup (&kvm->arch.vsie.addr_to_page, addr >>9 );rcu_read_unlock (); -if (page) { - vsie_page =page_to_virt (page); +if (vsie_page) {if (try_get_vsie_page (vsie_page)) {if (vsie_page->scb_gpa == addr)return vsie_page;
@@ -1411,20 +1407,18 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr)mutex_lock (&kvm->arch.vsie.mutex);if (kvm->arch.vsie.page_count < nr_vcpus) { - page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO |GFP_DMA ); -if (!page) { + vsie_page = (void *)__get_free_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO |GFP_DMA ); +if (!vsie_page) {mutex_unlock (&kvm->arch.vsie.mutex);return ERR_PTR (-ENOMEM ); } - vsie_page =page_to_virt (page);__set_bit (VSIE_PAGE_IN_USE , &vsie_page->flags); - kvm->arch.vsie.pages[kvm->arch.vsie.page_count] = page; + kvm->arch.vsie.pages[kvm->arch.vsie.page_count] = vsie_page; kvm->arch.vsie.page_count++; }else {/* reuse an existing entry that belongs to nobody */ while (true) { - page = kvm->arch.vsie.pages[kvm->arch.vsie.next]; - vsie_page =page_to_virt (page); + vsie_page = kvm->arch.vsie.pages[kvm->arch.vsie.next];if (try_get_vsie_page (vsie_page))break ; kvm->arch.vsie.next++;
@@ -1438,7 +1432,8 @@ static struct vsie_page *get_vsie_page(struct kvm *kvm, unsigned long addr) vsie_page->scb_gpa =ULONG_MAX ;/* Double use of the same address or allocation failure. */ -if (radix_tree_insert (&kvm->arch.vsie.addr_to_page, addr >>9 , page)) { +if (radix_tree_insert (&kvm->arch.vsie.addr_to_page, addr >>9 , + vsie_page)) {put_vsie_page (vsie_page);mutex_unlock (&kvm->arch.vsie.mutex);return NULL ;
@@ -1519,20 +1514,18 @@ void kvm_s390_vsie_init(struct kvm *kvm)void kvm_s390_vsie_destroy (struct kvm *kvm) {struct vsie_page *vsie_page; -struct page *page;int i;mutex_lock (&kvm->arch.vsie.mutex);for (i =0 ; i < kvm->arch.vsie.page_count; i++) { - page = kvm->arch.vsie.pages[i]; + vsie_page = kvm->arch.vsie.pages[i]; kvm->arch.vsie.pages[i] =NULL ; - vsie_page =page_to_virt (page);release_gmap_shadow (vsie_page);/* free the radix tree entry */ if (vsie_page->scb_gpa !=ULONG_MAX )radix_tree_delete (&kvm->arch.vsie.addr_to_page, vsie_page->scb_gpa >>9 ); -__free_page (page); +free_page ((unsigned long )vsie_page); } kvm->arch.vsie.page_count =0 ;mutex_unlock (&kvm->arch.vsie.mutex);
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 08/20] KVM: s390: selftests: fix ucontrol memory region test
In reply to Claudio ImbrendaWith the latest patch, attempting to create a memslot from userspace will result in an EEXIST error for UCONTROL VMs, instead of EINVAL, since the new memslot will collide with the internal memslot. There is no simple way to bring back the previous behaviour.
This is not a problem, but the test needs to be fixed accordingly.
Message-ID: <[email protected]>
--- tools/testing/selftests/kvm/s390/ucontrol_test.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/s390/ucontrol_test.c +6 -2
@@ -459,10 +459,14 @@ TEST_F(uc_kvm, uc_no_user_region) };ASSERT_EQ (-1 ,ioctl (self->vm_fd,KVM_SET_USER_MEMORY_REGION , ®ion)); -ASSERT_EQ (EINVAL , errno); + ASSERT_TRUE(errno == EEXIST || errno == EINVAL) +TH_LOG ("errno %s (%i) not expected for ioctl KVM_SET_USER_MEMORY_REGION" , +strerror (errno), errno);ASSERT_EQ (-1 ,ioctl (self->vm_fd,KVM_SET_USER_MEMORY_REGION2 , ®ion2)); -ASSERT_EQ (EINVAL , errno); + ASSERT_TRUE(errno == EEXIST || errno == EINVAL) +TH_LOG ("errno %s (%i) not expected for ioctl KVM_SET_USER_MEMORY_REGION2" , +strerror (errno), errno); }TEST_F (uc_kvm, uc_map_unmap)
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 13/20] KVM: s390: move some gmap shadowing functions away from mm/gmap.c
In reply to Claudio ImbrendaMove some gmap shadowing functions from mm/gmap.c to kvm/kvm-s390.c and the newly created kvm/gmap-vsie.c
This is a step toward removing gmap from mm.
Message-ID: <[email protected]>
--- arch/s390/include/asm/gmap.h | 9 +- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gmap-vsie.c | 142 +++++++++++++++++++++ arch/s390/kvm/gmap.h | 20 +++ arch/s390/kvm/kvm-s390.c | 74 ++++++++++- arch/s390/kvm/kvm-s390.h | 2 + arch/s390/kvm/vsie.c | 2 + arch/s390/mm/gmap.c | 238 +++++------------------------------ 8 files changed, 271 insertions(+), 218 deletions(-) create mode 100644 arch/s390/kvm/gmap-vsie.c
arch/s390/include/asm/gmap.h +4 -5
@@ -106,6 +106,8 @@ struct gmap *gmap_create(struct mm_struct *mm, unsigned long limit);void gmap_remove (struct gmap *gmap);struct gmap *gmap_get (struct gmap *gmap);void gmap_put (struct gmap *gmap); +void gmap_free (struct gmap *gmap); +struct gmap *gmap_alloc (unsigned long limit);int gmap_map_segment (struct gmap *gmap,unsigned long from,unsigned long to,unsigned long len);
@@ -118,9 +120,7 @@ void gmap_unlink(struct mm_struct *, unsigned long *table, unsigned long vmaddr)int gmap_read_table (struct gmap *gmap,unsigned long gaddr,unsigned long *val); -struct gmap *gmap_shadow (struct gmap *parent,unsigned long asce, -int edat_level); -int gmap_shadow_valid (struct gmap *sg,unsigned long asce,int edat_level); +void gmap_unshadow (struct gmap *sg);int gmap_shadow_r2t (struct gmap *sg,unsigned long saddr,unsigned long r2t,int fake); int gmap_shadow_r3t(struct gmap *sg, unsigned long saddr, unsigned long r3t,
@@ -136,8 +136,7 @@ int gmap_shadow_page(struct gmap *sg, unsigned long saddr, pte_t pte);void gmap_register_pte_notifier (struct gmap_notifier *);void gmap_unregister_pte_notifier (struct gmap_notifier *); -int gmap_mprotect_notify (struct gmap *,unsigned long start, -unsigned long len,int prot); +int gmap_protect_one (struct gmap *gmap,unsigned long gaddr,int prot,unsigned long bits);void gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr);
arch/s390/kvm/Makefile +1 -1
@@ -8,7 +8,7 @@ include $(srctree)/virt/kvm/Makefile.kvm ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o -kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o +kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o gmap-vsie.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/gmap-vsie.c new file +142 -0
142 lines not on this page · raw message
arch/s390/kvm/gmap.h +20 -0
@@ -13,5 +13,25 @@int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr); +struct gmap *gmap_shadow (struct gmap *parent,unsigned long asce,int edat_level); + +/** +* gmap_shadow_valid - check if a shadow guest address space matches the +* given properties and is still valid +* @sg: pointer to the shadow guest address space structure +* @asce: ASCE for which the shadow table is requested +* @edat_level: edat level to be used for the shadow translation +* +* Returns 1 if the gmap shadow is still valid and matches the given +* properties, the caller can continue using it. Returns 0 otherwise, the +* caller has to request a new shadow gmap in this case. +* +*/ +static inline int gmap_shadow_valid (struct gmap *sg,unsigned long asce,int edat_level) +{ +if (sg->removed) +return 0 ; +return sg->orig_asce == asce && sg->edat_level == edat_level; +}#endif
arch/s390/kvm/kvm-s390.c +71 -3
86 lines not on this page · raw message
arch/s390/kvm/kvm-s390.h +2 -0
@@ -420,6 +420,8 @@ void kvm_s390_set_cpu_timer(struct kvm_vcpu *vcpu, __u64 cputm);__u64 kvm_s390_get_cpu_timer (struct kvm_vcpu *vcpu);int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc);int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags); +int __kvm_s390_mprotect_many (struct gmap *gmap,gpa_t gpa,u8 npages,unsigned int prot, +unsigned long bits);static inline int kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gpa_t gaddr,unsigned int flags) {
arch/s390/kvm/vsie.c +2 -0
@@ -13,6 +13,7 @@#include <linux/bitmap.h> #include <linux/sched/signal.h> #include <linux/io.h> +#include <linux/mman.h> #include <asm/gmap.h> #include <asm/mmu_context.h>
@@ -22,6 +23,7 @@#include <asm/facility.h> #include "kvm-s390.h" #include "gaccess.h" +#include "gmap.h" enum vsie_page_flags {VSIE_PAGE_IN_USE =0 ,
arch/s390/mm/gmap.c +29 -209
291 lines not on this page · raw message
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 15/20] KVM: s390: stop using lists to keep track of used dat tables
In reply to Claudio ImbrendaUntil now, every dat table allocated to map a guest was put in a linked list. The page->lru field of struct page was used to keep track of which pages were being used, and when the gmap is torn down, the list was walked and all pages freed.
This patch gets rid of the usage of page->lru. Page tables are now freed by recursively walking the dat table tree.
Since s390_unlist_old_asce() becomes useless now, remove it.
Message-ID: <[email protected]>
--- arch/s390/include/asm/gmap.h | 5 -- arch/s390/mm/gmap.c | 102 ++++++++--------------------------- arch/s390/mm/pgalloc.c | 2 - 3 files changed, 23 insertions(+), 86 deletions(-)
arch/s390/include/asm/gmap.h +0 -5
@@ -23,7 +23,6 @@/** * struct gmap_struct - guest address space * @list: list head for the mm->context gmap list -* @crst_list: list of all crst tables used in the guest address space * @mm: pointer to the parent mm_struct * @guest_to_host: radix tree with guest to host address translation * @host_to_guest: radix tree with pointer to segment table entries
@@ -35,7 +34,6 @@* @guest_handle: protected virtual machine handle for the ultravisor * @host_to_rmap: radix tree with gmap_rmap lists * @children: list of shadow gmap structures -* @pt_list: list of all page tables used in the shadow guest address space * @shadow_lock: spinlock to protect the shadow gmap list * @parent: pointer to the parent gmap for shadow guest address spaces * @orig_asce: ASCE for which the shadow page table has been created
@@ -45,7 +43,6 @@*/ struct gmap {struct list_head list; -struct list_head crst_list;struct mm_struct *mm;struct radix_tree_root guest_to_host;struct radix_tree_root host_to_guest;
@@ -61,7 +58,6 @@ struct gmap {/* Additional data for shadow guest address spaces */ struct radix_tree_root host_to_rmap;struct list_head children; -struct list_head pt_list;spinlock_t shadow_lock;struct gmap *parent;unsigned long orig_asce;
@@ -141,7 +137,6 @@ int gmap_protect_one(struct gmap *gmap, unsigned long gaddr, int prot, unsignedvoid gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr);int s390_disable_cow_sharing (void ); -void s390_unlist_old_asce (struct gmap *gmap);int s390_replace_asce (struct gmap *gmap);void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns); int __s390_uv_destroy_range(struct mm_struct *mm, unsigned long start,
arch/s390/mm/gmap.c +23 -79
232 lines not on this page · raw message
arch/s390/mm/pgalloc.c +0 -2
@@ -176,8 +176,6 @@ unsigned long *page_table_alloc(struct mm_struct *mm) } table =ptdesc_to_virt (ptdesc);__arch_set_page_dat (table,1 ); -/* pt_list is used by gmap only */ -INIT_LIST_HEAD (&ptdesc->pt_list);memset64 ((u64 *)table, _PAGE_INVALID,PTRS_PER_PTE );memset64 ((u64 *)table +PTRS_PER_PTE ,0 ,PTRS_PER_PTE );return table;
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 17/20] KVM: s390: remove useless page->index usage
In reply to Claudio ImbrendaThe page->index field for VSIE dat tables is only used for segment tables.
Stop setting the field for all region tables.
Message-ID: <[email protected]>
--- arch/s390/mm/gmap.c | 9 --------- 1 file changed, 9 deletions(-)
arch/s390/mm/gmap.c +0 -9
@@ -1520,9 +1520,6 @@ int gmap_shadow_r2t(struct gmap *sg, unsigned long saddr, unsigned long r2t, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = r2t & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_r2t =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
@@ -1603,9 +1600,6 @@ int gmap_shadow_r3t(struct gmap *sg, unsigned long saddr, unsigned long r3t, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = r3t & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_r3t =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
@@ -1686,9 +1680,6 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = sgt & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_sgt =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 19/20] KVM: s390: remove the last user of page->index
In reply to Claudio ImbrendaShadow page tables use page->index to keep the g2 address of the guest page table being shadowed.
Instead of keeping the information in page->index, split the address and smear it over the 16-bit softbits areas of 4 PGSTEs.
This removes the last s390 user of page->index.
Message-ID: <[email protected]>
--- arch/s390/include/asm/pgtable.h | 15 +++++++++++++++ arch/s390/kvm/gaccess.c | 6 ++++-- arch/s390/mm/gmap.c | 22 ++++++++++++++++++++-- 3 files changed, 39 insertions(+), 4 deletions(-)
arch/s390/include/asm/pgtable.h +15 -0
@@ -420,6 +420,7 @@ void setup_protection_map(void);#define PGSTE_HC_BIT 0x0020000000000000UL#define PGSTE_GR_BIT 0x0004000000000000UL#define PGSTE_GC_BIT 0x0002000000000000UL +#define PGSTE_ST2_MASK 0x0000ffff00000000UL#define PGSTE_UC_BIT 0x0000000000008000UL/* user dirty (migration) */ #define PGSTE_IN_BIT 0x0000000000004000UL/* IPTE notify bit */ #define PGSTE_VSIE_BIT 0x0000000000002000UL/* ref'd in a shadow table */
@@ -2007,4 +2008,18 @@ extern void s390_reset_cmma(struct mm_struct *mm);#define pmd_pgtable (pmd) \ ((pgtable_t)__va(pmd_val(pmd) & -sizeof(pte_t)*PTRS_PER_PTE)) +static inline unsigned long gmap_pgste_get_pgt_addr (unsigned long *pgt) +{ +unsigned long *pgstes, res; + + pgstes = pgt + _PAGE_ENTRIES; + + res = (pgstes[0 ] &PGSTE_ST2_MASK ) <<16 ; + res |= pgstes[1 ] &PGSTE_ST2_MASK ; + res |= (pgstes[2 ] &PGSTE_ST2_MASK ) >>16 ; + res |= (pgstes[3 ] &PGSTE_ST2_MASK ) >>32 ; + +return res; +} +#endif /* _S390_PAGE_H */
arch/s390/kvm/gaccess.c +4 -2
@@ -1409,6 +1409,7 @@ static int kvm_s390_shadow_tables(struct gmap *sg, unsigned long saddr,static int shadow_pgt_lookup (struct gmap *sg,unsigned long saddr,unsigned long *pgt,int *dat_protection,int *fake) { +unsigned long pt_index;unsigned long *table;struct page *page;int rc;
@@ -1418,9 +1419,10 @@ static int shadow_pgt_lookup(struct gmap *sg, unsigned long saddr, unsigned longif (table && !(*table & _SEGMENT_ENTRY_INVALID)) {/* Shadow page tables are full pages (pte+pgste) */ page =pfn_to_page (*table >>PAGE_SHIFT ); - *pgt = page->index & ~GMAP_SHADOW_FAKE_TABLE ; + pt_index =gmap_pgste_get_pgt_addr (page_to_virt (page)); + *pgt = pt_index & ~GMAP_SHADOW_FAKE_TABLE ; *dat_protection = !!(*table & _SEGMENT_ENTRY_PROTECT); - *fake = !!(page->index &GMAP_SHADOW_FAKE_TABLE ); + *fake = !!(pt_index &GMAP_SHADOW_FAKE_TABLE ); rc =0 ; }else { rc = -EAGAIN ;
arch/s390/mm/gmap.c +20 -2
@@ -1733,6 +1733,23 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, }EXPORT_SYMBOL_GPL (gmap_shadow_sgt); +static void gmap_pgste_set_pgt_addr (struct ptdesc *ptdesc,unsigned long pgt_addr) +{ +unsigned long *pgstes =page_to_virt (ptdesc_page (ptdesc)); + + pgstes += _PAGE_ENTRIES; + + pgstes[0 ] &= ~PGSTE_ST2_MASK ; + pgstes[1 ] &= ~PGSTE_ST2_MASK ; + pgstes[2 ] &= ~PGSTE_ST2_MASK ; + pgstes[3 ] &= ~PGSTE_ST2_MASK ; + + pgstes[0 ] |= (pgt_addr >>16 ) &PGSTE_ST2_MASK ; + pgstes[1 ] |= pgt_addr &PGSTE_ST2_MASK ; + pgstes[2 ] |= (pgt_addr <<16 ) &PGSTE_ST2_MASK ; + pgstes[3 ] |= (pgt_addr <<32 ) &PGSTE_ST2_MASK ; +} +/** * gmap_shadow_pgt - instantiate a shadow page table * @sg: pointer to the shadow guest address space structure
@@ -1760,9 +1777,10 @@ int gmap_shadow_pgt(struct gmap *sg, unsigned long saddr, unsigned long pgt, ptdesc =page_table_alloc_pgste (sg->mm);if (!ptdesc)return -ENOMEM ; - ptdesc->pt_index = pgt & _SEGMENT_ENTRY_ORIGIN; + origin = pgt & _SEGMENT_ENTRY_ORIGIN;if (fake) - ptdesc->pt_index |=GMAP_SHADOW_FAKE_TABLE ; + origin |=GMAP_SHADOW_FAKE_TABLE ; +gmap_pgste_set_pgt_addr (ptdesc, origin); s_pgt =page_to_phys (ptdesc_page (ptdesc));/* Install shadow page table */ spin_lock (&sg->guest_table_lock);
Signature · 1 line
2.48.1
Claudio Imbrenda
maintainer[GIT PULL v1 20/20] KVM: s390: selftests: Streamline uc_skey test to issue iske after sske
In reply to Claudio ImbrendaFrom: Christoph Schlameuss <[email protected]>
In some rare situations a non default storage key is already set on the memory used by the test. Within normal VMs the key is reset / zapped when the memory is added to the VM. This is not the case for ucontrol VMs. With the initial iske check removed this test case can work in all situations. The function of the iske instruction is still validated by the remaining code.
Message-ID: <[email protected]>
--- .../selftests/kvm/s390/ucontrol_test.c | 24 +++++-------------- 1 file changed, 6 insertions(+), 18 deletions(-)
tools/testing/selftests/kvm/s390/ucontrol_test.c +6 -18
@@ -88,10 +88,6 @@ asm("test_skey_asm:\n"" ahi %r0,1\n" " st %r1,0(%r5,%r6)\n" -" iske %r1,%r6\n" -" ahi %r0,1\n" -" diag 0,0,0x44\n" -" sske %r1,%r6\n" " xgr %r1,%r1\n" " iske %r1,%r6\n"
@@ -600,7 +596,9 @@ TEST_F(uc_kvm, uc_skey)ASSERT_EQ (true,uc_handle_exit (self));ASSERT_EQ (1 , sync_regs->gprs[0 ]); -/* ISKE */ +/* SSKE + ISKE */ + sync_regs->gprs[1 ] = skeyvalue; + run->kvm_dirty_regs |=KVM_SYNC_GPRS ;ASSERT_EQ (0 ,uc_run_once (self));/*
@@ -612,21 +610,11 @@ TEST_F(uc_kvm, uc_skey)TEST_ASSERT_EQ (0 , sie_block->ictl & (ICTL_ISKE |ICTL_SSKE |ICTL_RRBE ));TEST_ASSERT_EQ (KVM_EXIT_S390_SIEIC , self->run->exit_reason);TEST_ASSERT_EQ (ICPT_INST , sie_block->icptcode); -TEST_REQUIRE (sie_block->ipa !=0xb229 ); +TEST_REQUIRE (sie_block->ipa !=0xb22b ); -/* ISKE contd. */ +/* SSKE + ISKE contd. */ ASSERT_EQ (false,uc_handle_exit (self));ASSERT_EQ (2 , sync_regs->gprs[0 ]); -/* assert initial skey (ACC = 0, R & C = 1) */ -ASSERT_EQ (0x06 , sync_regs->gprs[1 ]); -uc_assert_diag44 (self); - -/* SSKE + ISKE */ - sync_regs->gprs[1 ] = skeyvalue; - run->kvm_dirty_regs |=KVM_SYNC_GPRS ; -ASSERT_EQ (0 ,uc_run_once (self)); -ASSERT_EQ (false,uc_handle_exit (self)); -ASSERT_EQ (3 , sync_regs->gprs[0 ]);ASSERT_EQ (skeyvalue, sync_regs->gprs[1 ]);uc_assert_diag44 (self);
@@ -635,7 +623,7 @@ TEST_F(uc_kvm, uc_skey) run->kvm_dirty_regs |=KVM_SYNC_GPRS ;ASSERT_EQ (0 ,uc_run_once (self));ASSERT_EQ (false,uc_handle_exit (self)); -ASSERT_EQ (4 , sync_regs->gprs[0 ]); +ASSERT_EQ (3 , sync_regs->gprs[0 ]);/* assert R reset but rest of skey unchanged */ ASSERT_EQ (skeyvalue &0xfa , sync_regs->gprs[1 ]);ASSERT_EQ (0 , sync_regs->gprs[1 ] &0x04 );
Signature · 1 line
2.48.1