KVM: s390: Stop using page->index and other things
Claudio Imbrenda
maintainerv2 cover KVM: s390: Stop using page->index and other things
This patchseries starts moving some of the gmap logic into KVM itself, going towards the final goal of completely removing gmap from the non-kvm memory management code. Aside from just moving some code from mm/gmap into kvm, this series also starts using __kvm_faultin_pfn() to fault-in pages as needed.
But more importantly, this series removes almost all uses of page->index (and all uses of page->lru) from the s390 KVM code. The only remaining use is for the vsie pages, but that has already been taken care of by David in another series.
Unfortunately the mix of hastiness and holidays means that this series is a little bit all over the place, and not as complete as I would have liked to.
I'm posting it now so to try to speed up the removal of page->index, hopefully I will be able to post another short series before the upcoming merge window closes.
v1->v2:
* moved some code around between patches to improve readability and shuffle the order of some patches
* rebase on Sean's patchseries
* add Sean's patch to remove size limitations for internal memslots
* use Sean's new API for internal memslots to place one huge internal memslot instead of many 4T ones for UCONTROL guests
* create new kvm/gmap-vsie.c file for VSIE code, instead of dumping everything in the existing files, which are already too large
* improve comments and patch descriptions
* minor style and cosmetic fixes
Claudio Imbrenda (14):
KVM: s390: wrapper for KVM_BUG
KVM: s390: move pv gmap functions into kvm
KVM: s390: fake memslot for ucontrol VMs
KVM: s390: selftests: fix ucontrol memory region test
KVM: s390: use __kvm_faultin_pfn()
KVM: s390: get rid of gmap_fault()
KVM: s390: get rid of gmap_translate()
KVM: s390: move some gmap shadowing functions away from mm/gmap.c
KVM: s390: stop using page->index for non-shadow gmaps
KVM: s390: stop using lists to keep track of used dat tables
KVM: s390: move gmap_shadow_pgt_lookup() into kvm
KVM: s390: remove useless page->index usage
KVM: s390: move PGSTE softbits
KVM: s390: remove the last user of page->index
Sean Christopherson (1):
KVM: Do not restrict the size of KVM-internal memory regions
arch/s390/include/asm/gmap.h | 17 +- arch/s390/include/asm/kvm_host.h | 2 + arch/s390/include/asm/pgtable.h | 21 +- arch/s390/include/asm/uv.h | 7 +- arch/s390/kernel/uv.c | 293 ++------ arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gaccess.c | 42 ++ arch/s390/kvm/gmap-vsie.c | 142 ++++ arch/s390/kvm/gmap.c | 196 ++++++ arch/s390/kvm/gmap.h | 39 ++ arch/s390/kvm/intercept.c | 5 +- arch/s390/kvm/interrupt.c | 19 +- arch/s390/kvm/kvm-s390.c | 219 ++++-- arch/s390/kvm/kvm-s390.h | 19 + arch/s390/kvm/pv.c | 1 + arch/s390/kvm/vsie.c | 2 + arch/s390/mm/gmap.c | 638 +++--------------- .../selftests/kvm/s390x/ucontrol_test.c | 6 +- virt/kvm/kvm_main.c | 10 +- 19 files changed, 828 insertions(+), 852 deletions(-) create mode 100644 arch/s390/kvm/gmap-vsie.c create mode 100644 arch/s390/kvm/gmap.c create mode 100644 arch/s390/kvm/gmap.h
Signature · 1 line
2.47.1
Claudio Imbrenda
maintainerPATCH v2 1/15 KVM: Do not restrict the size of KVM-internal memory regions
From: Sean Christopherson <[email protected]>
Exempt KVM-internal memslots from the KVM_MEM_MAX_NR_PAGES restriction, as the limit on the number of pages exists purely to play nice with dirty bitmap operations, which use 32-bit values to index the bitmaps, and dirty logging isn't supported for KVM-internal memslots.
--- virt/kvm/kvm_main.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +9 -1
@@ -1966,7 +1966,15 @@ static int kvm_set_memory_region(struct kvm *kvm,return -EINVAL ;if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr)return -EINVAL ; -if ((mem->memory_size >>PAGE_SHIFT ) >KVM_MEM_MAX_NR_PAGES ) + +/* +* The size of userspace-defined memory regions is restricted in order +* to play nice with dirty bitmap operations, which are indexed with an +* "unsigned int". KVM's internal memory regions don't support dirty +* logging, and so are exempt. +*/ +if (id <KVM_USER_MEM_SLOTS && + (mem->memory_size >>PAGE_SHIFT ) >KVM_MEM_MAX_NR_PAGES )return -EINVAL ; slots =__kvm_memslots (kvm, as_id);
Signature · 1 line
2.47.1
Re: [PATCH v2 01/15] KVM: Do not restrict the size of KVM-internal memory regions
Quoted 9 lines · Claudio Imbrenda
From: Sean Christopherson <[email protected]>
Exempt KVM-internal memslots from the
KVM_MEM_MAX_NR_PAGESrestriction, as the limit on the number of pages exists purely to play nice with dirty bitmap operations, which use 32-bit values to index the bitmaps, and dirty logging isn't supported for KVM-internal memslots.
Quoted 25 lines
--- virt/kvm/kvm_main.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)virt/kvm/kvm_main.c +9
-1@@ -1966,7 +1966,15 @@ static int kvm_set_memory_region(struct kvm *kvm,return -EINVAL ;if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr)return -EINVAL ; -if ((mem->memory_size >>PAGE_SHIFT ) >KVM_MEM_MAX_NR_PAGES ) + +/* +* The size of userspace-defined memory regions is restricted in order +* to play nice with dirty bitmap operations, which are indexed with an +* "unsigned int". KVM's internal memory regions don't support dirty +* logging, and so are exempt. +*/ +if (id <KVM_USER_MEM_SLOTS && + (mem->memory_size >>PAGE_SHIFT ) >KVM_MEM_MAX_NR_PAGES )return -EINVAL ; slots =__kvm_memslots (kvm, as_id);
Claudio Imbrenda
maintainerPATCH v2 2/15 KVM: s390: wrapper for KVM_BUG
In reply to Claudio ImbrendaWrap the call to KVM_BUG; this reduces code duplication and improves readability.
--- arch/s390/kvm/kvm-s390.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
arch/s390/kvm/kvm-s390.c +10 -9
@@ -4766,6 +4766,13 @@ static int vcpu_post_run_addressing_exception(struct kvm_vcpu *vcpu)return kvm_s390_inject_prog_irq (vcpu, &pgm_info); } +static void kvm_s390_assert_primary_as (struct kvm_vcpu *vcpu) +{ +KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, +"Unexpected program interrupt 0x%x, TEID 0x%016lx" , + current->thread.gmap_int_code, current->thread.gmap_teid.val); +} +static int vcpu_post_run_handle_fault (struct kvm_vcpu *vcpu) {unsigned int flags =0 ;
@@ -4781,9 +4788,7 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu) vcpu->stat.exit_null++;break ;case PGM_NON_SECURE_STORAGE_ACCESS : -KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, -"Unexpected program interrupt 0x%x, TEID 0x%016lx" , - current->thread.gmap_int_code, current->thread.gmap_teid.val); +kvm_s390_assert_primary_as (vcpu);/* * This is normal operation; a page belonging to a protected * guest has not been imported yet. Try to import the page into
@@ -4794,9 +4799,7 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu)break ;case PGM_SECURE_STORAGE_ACCESS :case PGM_SECURE_STORAGE_VIOLATION : -KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, -"Unexpected program interrupt 0x%x, TEID 0x%016lx" , - current->thread.gmap_int_code, current->thread.gmap_teid.val); +kvm_s390_assert_primary_as (vcpu);/* * This can happen after a reboot with asynchronous teardown; * the new guest (normal or protected) will run on top of the
@@ -4825,9 +4828,7 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu)case PGM_REGION_FIRST_TRANS :case PGM_REGION_SECOND_TRANS :case PGM_REGION_THIRD_TRANS : -KVM_BUG (current->thread.gmap_teid.as !=PSW_BITS_AS_PRIMARY , vcpu->kvm, -"Unexpected program interrupt 0x%x, TEID 0x%016lx" , - current->thread.gmap_int_code, current->thread.gmap_teid.val); +kvm_s390_assert_primary_as (vcpu);if (vcpu->arch.gmap->pfault_enabled) { rc =gmap_fault (vcpu->arch.gmap, gaddr, flags |FAULT_FLAG_RETRY_NOWAIT );if (rc == -EFAULT )
Signature · 1 line
2.47.1
Re: [PATCH v2 02/15] KVM: s390: wrapper for KVM_BUG
Quoted 6 lines · Claudio Imbrenda
Wrap the call to
KVM_BUG; this reduces code duplication and improves readability.Signed-off-by Claudio Imbrenda <[email protected]>Reviewed-by Christian Borntraeger <[email protected]>Reviewed-by Christoph Schlameuss <[email protected]>
Claudio Imbrenda
maintainerPATCH v2 3/15 KVM: s390: move pv gmap functions into kvm
In reply to Claudio ImbrendaMove gmap related functions from kernel/uv into kvm.
Create a new file to collect gmap-related functions.
--- arch/s390/include/asm/uv.h | 7 +- arch/s390/kernel/uv.c | 293 ++++++------------------------------- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gmap.c | 196 +++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 17 +++ arch/s390/kvm/intercept.c | 1 + arch/s390/kvm/kvm-s390.c | 1 + arch/s390/kvm/pv.c | 1 + 8 files changed, 264 insertions(+), 254 deletions(-) create mode 100644 arch/s390/kvm/gmap.c create mode 100644 arch/s390/kvm/gmap.h
arch/s390/include/asm/uv.h +4 -3
@@ -628,12 +628,13 @@ static inline int is_prot_virt_host(void) }int uv_pin_shared (unsigned long paddr); -int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); -int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);int uv_destroy_folio (struct folio *folio);int uv_destroy_pte (pte_t pte);int uv_convert_from_secure_pte (pte_t pte); -int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr); +int uv_wiggle_folio (struct folio *folio,bool split); +int make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb); +int uv_convert_from_secure (unsigned long paddr); +int uv_convert_from_secure_folio (struct folio *folio);void setup_uv (void );
arch/s390/kernel/uv.c +43 -250
356 lines not on this page · raw message
arch/s390/kvm/Makefile +1 -1
@@ -8,7 +8,7 @@ include $(srctree)/virt/kvm/Makefile.kvm ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o -kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o +kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/gmap.c new file +196 -0
196 lines not on this page · raw message
arch/s390/kvm/gmap.h new file +17 -0
@@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* KVM guest address space mapping code +* +* Copyright IBM Corp. 2007, 2016, 2025 +* Author(s): Martin Schwidefsky <[email protected]> +* Claudio Imbrenda <[email protected]> +*/ + +#ifndef ARCH_KVM_S390_GMAP_H +#define ARCH_KVM_S390_GMAP_H + +int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); +int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr); +int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr); + +#endif
arch/s390/kvm/intercept.c +1 -0
@@ -21,6 +21,7 @@#include "gaccess.h" #include "trace.h" #include "trace-s390.h" +#include "gmap.h" u8 kvm_s390_get_ilen (struct kvm_vcpu *vcpu) {
arch/s390/kvm/kvm-s390.c +1 -0
@@ -50,6 +50,7 @@#include "kvm-s390.h" #include "gaccess.h" #include "pci.h" +#include "gmap.h" #define CREATE_TRACE_POINTS #include "trace.h"
arch/s390/kvm/pv.c +1 -0
@@ -17,6 +17,7 @@#include <linux/sched/mm.h> #include <linux/mmu_notifier.h> #include "kvm-s390.h" +#include "gmap.h" bool kvm_s390_pv_is_protected (struct kvm *kvm) {
Signature · 1 line
2.47.1
Janosch Frank
maintainerRe: [PATCH v2 03/15] KVM: s390: move pv gmap functions into kvm
Quoted 6 lines · Claudio Imbrenda
Move gmap related functions from kernel/uv into kvm.
Create a new file to collect gmap-related functions.
Signed-off-by Claudio Imbrenda <[email protected]>---
Thanks git, this is close to unreadable
Quoted 24 lines
/**
- *should_export_before_import- Determine whether an export is needed
- * before an import-like operation
- * @uvcb: the Ultravisor control block of the UVC to be performed
- * @mm: the mm of the process
- *
- * Returns whether an export is needed before every import-like operation.
- * This is needed for shared pages, which don't trigger a secure storage
- * exception when accessed from a different guest.
- *
- * Although considered as one, the Unpin Page UVC is not an actual import,
- * so it is not affected.
+ *uv_wiggle_folio()- try to drain extra references to a folio
+ * @folio: the folio
+ * @split: whether to split a large folio
*
- * No export is needed also when there is only one protected VM, because the
- * page cannot belong to the wrong VM in that case (there is no "other VM"
- * it can belong to).
- *
- * Return: true if an export is needed before every import, otherwise false.
+ * Context: Must be called while holding an extra reference to the folio;
+ * the mm lock should not be held.
*/
arch/s390/kernel/uv.c static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb) · line 289
Quoted 2 lines
arch/s390/kernel/uv.c +1
-1-static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) +int uv_wiggle_folio (struct folio *folio,bool split)
Should I expect a drop of references to also split THPs? Seems a bit odd to me but I do not know a lot about folios.
Quoted 13 lines
{arch/s390/kernel/uv.c +0
-12-/* -* The misc feature indicates, among other things, that importing a -* shared page from a different protected VM will automatically also -* transfer its ownership. -*/ -if (uv_has_feature (BIT_UV_FEAT_MISC )) -return false; -if (uvcb->cmd ==UVC_CMD_UNPIN_PAGE_SHARED ) -return false; -return atomic_read (&mm->context.protected_count) >1 ; -} -
[...]
Quoted 56 lines
arch/s390/kernel/uv.c +0
-54-/* -* Requests the Ultravisor to make a page accessible to a guest. -* If it's brought in the first time, it will be cleared. If -* it has been exported before, it will be decrypted and integrity -* checked. -*/ -int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb) -{ -struct vm_area_struct *vma; -bool drain_lru_called = false; -spinlock_t *ptelock; -unsigned long uaddr; -struct folio *folio; -pte_t *ptep;int rc; -again: - rc = -EFAULT ; -mmap_read_lock (gmap->mm); - - uaddr =__gmap_translate (gmap, gaddr); -if (IS_ERR_VALUE (uaddr)) - goto out; - vma =vma_lookup (gmap->mm, uaddr); -if (!vma) - goto out; -/* -* Secure pages cannot be huge and userspace should not combine both. -* In case userspace does it anyway this will result in an -EFAULT for -* the unpack. The guest is thus never reaching secure mode. If -* userspace is playing dirty tricky with mapping huge pages later -* on this will result in a segmentation fault. -*/ -if (is_vm_hugetlb_page (vma)) - goto out; - - rc = -ENXIO ; - ptep =get_locked_pte (gmap->mm, uaddr, &ptelock); -if (!ptep) - goto out; -if (pte_present (*ptep) && !(pte_val (*ptep) & _PAGE_INVALID) &&pte_write (*ptep)) { - folio =page_folio (pte_page (*ptep)); - rc = -EAGAIN ; -if (folio_test_large (folio)) { - rc = -E2BIG ; - }else if (folio_trylock (folio)) { -if (should_export_before_import (uvcb, gmap->mm)) -uv_convert_from_secure (PFN_PHYS (folio_pfn (folio))); - rc =make_folio_secure (folio, uvcb); -folio_unlock (folio); - } - -/* -* Once we drop the PTL, the folio may get unmapped and -* freed immediately. We need a temporary reference. -*/
arch/s390/kernel/uv.c static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb) · line 293
Quoted 10 lines
arch/s390/kernel/uv.c +1
-9-if (rc == -EAGAIN || rc == -E2BIG ) -folio_get (folio); - } -pte_unmap_unlock (ptep, ptelock); -out: -mmap_read_unlock (gmap->mm); - -switch (rc) { -case -E2BIG : +folio_wait_writeback (folio);
Add an assert_not_held for the mm mutex above "folio_wait_writeback(folio);"?
Quoted 219 lines
arch/s390/kernel/uv.c +8
-110+if (split) {folio_lock (folio); rc =split_folio (folio);folio_unlock (folio); -folio_put (folio); - -switch (rc) { -case 0 : -/* Splitting succeeded, try again immediately. */ - goto again; -case -EAGAIN : -/* Additional folio references. */ -if (drain_lru (&drain_lru_called)) - goto again; -return -EAGAIN ; -case -EBUSY : -/* Unexpected race. */ + +if (rc == -EBUSY )return -EAGAIN ; - } -WARN_ON_ONCE (1 ); -return -ENXIO ; -case -EAGAIN : -/* -* If we are here because the UVC returned busy or partial -* completion, this is just a useless check, but it is safe. -*/ -folio_wait_writeback (folio); -folio_put (folio); -return -EAGAIN ; -case -EBUSY : -/* Additional folio references. */ -if (drain_lru (&drain_lru_called)) - goto again; -return -EAGAIN ; -case -ENXIO : -if (gmap_fault (gmap, gaddr,FAULT_FLAG_WRITE )) -return -EFAULT ; -return -EAGAIN ; +if (rc != -EAGAIN ) +return rc; } -return rc; -} -EXPORT_SYMBOL_GPL (gmap_make_secure); - -int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr) -{ -struct uv_cb_cts uvcb = { - .header.cmd =UVC_CMD_CONV_TO_SEC_STOR , - .header.len =sizeof (uvcb), - .guest_handle = gmap->guest_handle, - .gaddr = gaddr, - }; - -return gmap_make_secure (gmap, gaddr, &uvcb); -} -EXPORT_SYMBOL_GPL (gmap_convert_to_secure); - -/** -* gmap_destroy_page - Destroy a guest page. -* @gmap: the gmap of the guest -* @gaddr: the guest address to destroy -* -* An attempt will be made to destroy the given guest page. If the attempt -* fails, an attempt is made to export the page. If both attempts fail, an -* appropriate error is returned. -*/ -int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr) -{ -struct vm_area_struct *vma; -struct folio_walk fw; -unsigned long uaddr; -struct folio *folio; -int rc; - - rc = -EFAULT ; -mmap_read_lock (gmap->mm); - - uaddr =__gmap_translate (gmap, gaddr); -if (IS_ERR_VALUE (uaddr)) - goto out; - vma =vma_lookup (gmap->mm, uaddr); -if (!vma) - goto out; -/* -* Huge pages should not be able to become secure -*/ -if (is_vm_hugetlb_page (vma)) - goto out; - - rc =0 ; - folio =folio_walk_start (&fw, vma, uaddr,0 ); -if (!folio) - goto out; -/* -* See gmap_make_secure(): large folios cannot be secure. Small -* folio implies FW_LEVEL_PTE. -*/ -if (folio_test_large (folio) || !pte_write (fw.pte)) - goto out_walk_end; - rc =uv_destroy_folio (folio); -/* -* Fault handlers can race; it is possible that two CPUs will fault -* on the same secure page. One CPU can destroy the page, reboot, -* re-enter secure mode and import it, while the second CPU was -* stuck at the beginning of the handler. At some point the second -* CPU will be able to progress, and it will not be able to destroy -* the page. In that case we do not want to terminate the process, -* we instead try to export the page. -*/ -if (rc) - rc =uv_convert_from_secure_folio (folio); -out_walk_end: -folio_walk_end (&fw, vma); -out: -mmap_read_unlock (gmap->mm); -return rc; +lru_add_drain_all (); +return -EAGAIN ; } -EXPORT_SYMBOL_GPL (gmap_destroy_page); +EXPORT_SYMBOL_GPL (uv_wiggle_folio);/* * To be called with the folio locked or with an extra reference! This will arch/s390/kvm/Makefile +1
-1@@ -8,7 +8,7 @@ include $(srctree)/virt/kvm/Makefile.kvm ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o -kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o +kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.oarch/s390/kvm/gmap.c new file +73
-0@@ -0,0 +1,196 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Guest memory management for KVM/s390 +* +* Copyright IBM Corp. 2008, 2020, 2024 +* +* Author(s): Claudio Imbrenda <[email protected]> +* Martin Schwidefsky <[email protected]> +* David Hildenbrand <[email protected]> +* Janosch Frank <[email protected]> +*/ + +#include <linux/compiler.h> +#include <linux/kvm.h> +#include <linux/kvm_host.h> +#include <linux/pgtable.h> +#include <linux/pagemap.h> + +#include <asm/lowcore.h> +#include <asm/gmap.h> +#include <asm/uv.h> + +#include "gmap.h" + +/** +* should_export_before_import - Determine whether an export is needed +* before an import-like operation +* @uvcb: the Ultravisor control block of the UVC to be performed +* @mm: the mm of the process +* +* Returns whether an export is needed before every import-like operation. +* This is needed for shared pages, which don't trigger a secure storage +* exception when accessed from a different guest. +* +* Although considered as one, the Unpin Page UVC is not an actual import, +* so it is not affected. +* +* No export is needed also when there is only one protected VM, because the +* page cannot belong to the wrong VM in that case (there is no "other VM" +* it can belong to). +* +* Return: true if an export is needed before every import, otherwise false. +*/ +static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) +{ +/* +* The misc feature indicates, among other things, that importing a +* shared page from a different protected VM will automatically also +* transfer its ownership. +*/ +if (uv_has_feature (BIT_UV_FEAT_MISC )) +return false; +if (uvcb->cmd ==UVC_CMD_UNPIN_PAGE_SHARED ) +return false; +return atomic_read (&mm->context.protected_count) >1 ; +} + +static int __gmap_make_secure (struct gmap *gmap,struct page *page,void *uvcb) +{ +struct folio *folio =page_folio (page); +int rc; + +/* +* Secure pages cannot be huge and userspace should not combine both. +* In case userspace does it anyway this will result in an -EFAULT for +* the unpack. The guest is thus never reaching secure mode. +* If userspace plays dirty tricks and decides to map huge pages at a +* later point in time, it will receive a segmentation fault or +* KVM_RUN will return -EFAULT. +*/ +if (folio_test_hugetlb (folio)) +return -EFAULT ; +if (folio_test_large (folio)) {
arch/s390/kvm/gmap.c line 83
Quoted 10 lines
arch/s390/kvm/gmap.c new file +10
-0+mmap_read_unlock (gmap->mm); + rc =uv_wiggle_folio (folio, true); +mmap_read_lock (gmap->mm); +if (rc) +return rc; + folio =page_folio (page); + } + + rc = -EAGAIN ; +if (folio_trylock (folio)) {
If you test for !folio_trylock() you could do an early return, no?
Quoted 23 lines
arch/s390/kvm/gmap.c new file +22
-0+if (should_export_before_import (uvcb, gmap->mm)) +uv_convert_from_secure (folio_to_phys (folio)); + rc =make_folio_secure (folio, uvcb); +folio_unlock (folio); + } + +/* +* Unlikely case: the page is not mapped anymore. Return success +* and let the proper fault handler fault in the page again. +*/ +if (rc == -ENXIO ) +return 0 ; +/* The folio has too many references, try to shake some off */ +if (rc == -EBUSY ) { +mmap_read_unlock (gmap->mm); +uv_wiggle_folio (folio, false); +mmap_read_lock (gmap->mm); +return -EAGAIN ; + } + +return rc; +}
Claudio Imbrenda
maintainerRe: [PATCH v2 03/15] KVM: s390: move pv gmap functions into kvm
Quoted 30 lines · On Fri, 17 Jan 2025 14:38:08 +0100 Janosch Frank <[email protected]> wrote:
Quoted 6 lines · Claudio Imbrenda
Move gmap related functions from kernel/uv into kvm.
Create a new file to collect gmap-related functions.
Signed-off-by Claudio Imbrenda <[email protected]>---
Thanks git, this is close to unreadable
Quoted 20 lines
/**
- *should_export_before_import- Determine whether an export is needed
- * before an import-like operation
- * @uvcb: the Ultravisor control block of the UVC to be performed
- * @mm: the mm of the process
- *
- * Returns whether an export is needed before every import-like operation.
- * This is needed for shared pages, which don't trigger a secure storage
- * exception when accessed from a different guest.
- *
- * Although considered as one, the Unpin Page UVC is not an actual import,
- * so it is not affected.
+ *uv_wiggle_folio()- try to drain extra references to a folio
+ * @folio: the folio
+ * @split: whether to split a large folio
*
- * No export is needed also when there is only one protected VM, because the
- * page cannot belong to the wrong VM in that case (there is no "other VM"
- * it can belong to).
- *
Quoted 8 lines
arch/s390/kernel/uv.c static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb) · line 289
Quoted 6 lines
- * Return: true if an export is needed before every import, otherwise false.
+ * Context: Must be called while holding an extra reference to the folio;
+ * the mm lock should not be held.
*/arch/s390/kernel/uv.c +1
-1-static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) +int uv_wiggle_folio (struct folio *folio,bool split)Should I expect a drop of references to also split THPs?
no, that's why we have the @split parameter
Quoted 79 lines
Seems a bit odd to me but I do not know a lot about folios.
Quoted 13 lines
{arch/s390/kernel/uv.c +0
-12-/* -* The misc feature indicates, among other things, that importing a -* shared page from a different protected VM will automatically also -* transfer its ownership. -*/ -if (uv_has_feature (BIT_UV_FEAT_MISC )) -return false; -if (uvcb->cmd ==UVC_CMD_UNPIN_PAGE_SHARED ) -return false; -return atomic_read (&mm->context.protected_count) >1 ; -} -[...]
Quoted 61 lines
arch/s390/kernel/uv.c +0
-59-/* -* Requests the Ultravisor to make a page accessible to a guest. -* If it's brought in the first time, it will be cleared. If -* it has been exported before, it will be decrypted and integrity -* checked. -*/ -int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb) -{ -struct vm_area_struct *vma; -bool drain_lru_called = false; -spinlock_t *ptelock; -unsigned long uaddr; -struct folio *folio; -pte_t *ptep;int rc; -again: - rc = -EFAULT ; -mmap_read_lock (gmap->mm); - - uaddr =__gmap_translate (gmap, gaddr); -if (IS_ERR_VALUE (uaddr)) - goto out; - vma =vma_lookup (gmap->mm, uaddr); -if (!vma) - goto out; -/* -* Secure pages cannot be huge and userspace should not combine both. -* In case userspace does it anyway this will result in an -EFAULT for -* the unpack. The guest is thus never reaching secure mode. If -* userspace is playing dirty tricky with mapping huge pages later -* on this will result in a segmentation fault. -*/ -if (is_vm_hugetlb_page (vma)) - goto out; - - rc = -ENXIO ; - ptep =get_locked_pte (gmap->mm, uaddr, &ptelock); -if (!ptep) - goto out; -if (pte_present (*ptep) && !(pte_val (*ptep) & _PAGE_INVALID) &&pte_write (*ptep)) { - folio =page_folio (pte_page (*ptep)); - rc = -EAGAIN ; -if (folio_test_large (folio)) { - rc = -E2BIG ; - }else if (folio_trylock (folio)) { -if (should_export_before_import (uvcb, gmap->mm)) -uv_convert_from_secure (PFN_PHYS (folio_pfn (folio))); - rc =make_folio_secure (folio, uvcb); -folio_unlock (folio); - } - -/* -* Once we drop the PTL, the folio may get unmapped and -* freed immediately. We need a temporary reference. -*/ -if (rc == -EAGAIN || rc == -E2BIG ) -folio_get (folio); - } -pte_unmap_unlock (ptep, ptelock); -out:
Quoted 8 lines
arch/s390/kernel/uv.c static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb) · line 293
Quoted 5 lines
arch/s390/kernel/uv.c +1
-4-mmap_read_unlock (gmap->mm); - -switch (rc) { -case -E2BIG : +folio_wait_writeback (folio);Add an
assert_not_heldfor the mm mutex above
"folio_wait_writeback(folio);"?
will do
[...]
Quoted 20 lines
Quoted 20 lines
arch/s390/kvm/gmap.c new file +20
-0+static int __gmap_make_secure (struct gmap *gmap,struct page *page,void *uvcb) +{ +struct folio *folio =page_folio (page); +int rc; + +/* +* Secure pages cannot be huge and userspace should not combine both. +* In case userspace does it anyway this will result in an -EFAULT for +* the unpack. The guest is thus never reaching secure mode. +* If userspace plays dirty tricks and decides to map huge pages at a +* later point in time, it will receive a segmentation fault or +* KVM_RUN will return -EFAULT. +*/ +if (folio_test_hugetlb (folio)) +return -EFAULT ; +if (folio_test_large (folio)) { +mmap_read_unlock (gmap->mm); + rc =uv_wiggle_folio (folio, true); +mmap_read_lock (gmap->mm); +if (rc)
Quoted 8 lines
arch/s390/kvm/gmap.c line 83
Quoted 6 lines
arch/s390/kvm/gmap.c new file +6
-0+return rc; + folio =page_folio (page); + } + + rc = -EAGAIN ; +if (folio_trylock (folio)) {If you test for !folio_trylock() you could do an early return, no?
true
Quoted 24 lines
Quoted 24 lines
arch/s390/kvm/gmap.c new file +22
-0+if (should_export_before_import (uvcb, gmap->mm)) +uv_convert_from_secure (folio_to_phys (folio)); + rc =make_folio_secure (folio, uvcb); +folio_unlock (folio); + } + +/* +* Unlikely case: the page is not mapped anymore. Return success +* and let the proper fault handler fault in the page again. +*/ +if (rc == -ENXIO ) +return 0 ; +/* The folio has too many references, try to shake some off */ +if (rc == -EBUSY ) { +mmap_read_unlock (gmap->mm); +uv_wiggle_folio (folio, false); +mmap_read_lock (gmap->mm); +return -EAGAIN ; + } + +return rc; +}
Janosch Frank
maintainerRe: [PATCH v2 03/15] KVM: s390: move pv gmap functions into kvm
In reply to Claudio ImbrendaQuoted 17 lines · Claudio Imbrenda
Move gmap related functions from kernel/uv into kvm.
Create a new file to collect gmap-related functions.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/include/asm/uv.h | 7 +- arch/s390/kernel/uv.c | 293 ++++++------------------------------- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gmap.c | 196 +++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 17 +++ arch/s390/kvm/intercept.c | 1 + arch/s390/kvm/kvm-s390.c | 1 + arch/s390/kvm/pv.c | 1 + 8 files changed, 264 insertions(+), 254 deletions(-)create mode 100644
arch/s390/kvm/gmap.c
create mode 100644arch/s390/kvm/gmap.h
[...]
arch/s390/kernel/uv.c static int expected_folio_refs(struct folio *folio) · line 239
Quoted 12 lines
arch/s390/kernel/uv.c +10
-1-static int make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb) +/** +* make_folio_secure() - make a folio secure +* @folio: the folio to make secure +* @uvcb: the uvcb that describes the UVC to be used +* +* The folio @folio will be made secure if possible, @uvcb will be passed +* as-is to the UVC. +* +* Return: 0 on success; +* -EBUSY if the folio is in writeback, has too many references, or is large;
I'd expect busy for writeback and too many references since someone is referencing or working with the page.
But EBUSY for large mappings?
Also, the large mapping doesn't just vanish by waiting, does it? You're actively splitting the mapping.
Quoted 30 lines
+ * -EAGAIN if the UVC needs to be attempted again; + * -ENXIO if the address is not mapped; + * -EINVAL if the UVC failed for other reasons.+ *
+ * Context: The caller must hold exactly one extra reference on the folio
+ * (it's the same logic assplit_folio())
+ */arch/s390/kernel/uv.c +6
-4+int make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb) {int expected, cc =0 ; +if (folio_test_large (folio)) +return -EBUSY ;if (folio_test_writeback (folio)) -return -EAGAIN ; - expected =expected_folio_refs (folio); +return -EBUSY ; + expected =expected_folio_refs (folio) +1 ;if (!folio_ref_freeze (folio, expected))return -EBUSY ;set_bit (PG_arch_1, &folio->flags);@@ -267,251 +276,35 @@ static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb)return -EAGAIN ;return uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; } +EXPORT_SYMBOL_GPL (make_folio_secure);/** -* should_export_before_import - Determine whether an export is needed -* before an import-like operation
arch/s390/kernel/uv.c static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb) · line 282
Quoted 10 lines
arch/s390/kernel/uv.c +1
-9-* @uvcb: the Ultravisor control block of the UVC to be performed -* @mm: the mm of the process -* -* Returns whether an export is needed before every import-like operation. -* This is needed for shared pages, which don't trigger a secure storage -* exception when accessed from a different guest. -* -* Although considered as one, the Unpin Page UVC is not an actual import, -* so it is not affected. +* uv_wiggle_folio() - try to drain extra references to a folio
How about adding the split into the name?uv_wiggle_split
Quoted 2 lines
arch/s390/kernel/uv.c +2
-0+* @folio: the folio +* @split: whether to split a large folio
[...]
Quoted 47 lines
arch/s390/kvm/gmap.c new file +47
-0+/** +* should_export_before_import - Determine whether an export is needed +* before an import-like operation +* @uvcb: the Ultravisor control block of the UVC to be performed +* @mm: the mm of the process +* +* Returns whether an export is needed before every import-like operation. +* This is needed for shared pages, which don't trigger a secure storage +* exception when accessed from a different guest. +* +* Although considered as one, the Unpin Page UVC is not an actual import, +* so it is not affected. +* +* No export is needed also when there is only one protected VM, because the +* page cannot belong to the wrong VM in that case (there is no "other VM" +* it can belong to). +* +* Return: true if an export is needed before every import, otherwise false. +*/ +static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) +{ +/* +* The misc feature indicates, among other things, that importing a +* shared page from a different protected VM will automatically also +* transfer its ownership. +*/ +if (uv_has_feature (BIT_UV_FEAT_MISC )) +return false; +if (uvcb->cmd ==UVC_CMD_UNPIN_PAGE_SHARED ) +return false; +return atomic_read (&mm->context.protected_count) >1 ; +} + +static int __gmap_make_secure (struct gmap *gmap,struct page *page,void *uvcb) +{ +struct folio *folio =page_folio (page); +int rc; + +/* +* Secure pages cannot be huge and userspace should not combine both. +* In case userspace does it anyway this will result in an -EFAULT for +* the unpack. The guest is thus never reaching secure mode. +* If userspace plays dirty tricks and decides to map huge pages at a +* later point in time, it will receive a segmentation fault or +* KVM_RUN will return -EFAULT. +*/ +if (folio_test_hugetlb (folio))
arch/s390/kvm/gmap.c line 81
Quoted 10 lines
arch/s390/kvm/gmap.c new file +10
-0+return -EFAULT ; +if (folio_test_large (folio)) { +mmap_read_unlock (gmap->mm); + rc =uv_wiggle_folio (folio, true); +mmap_read_lock (gmap->mm); +if (rc) +return rc; + folio =page_folio (page); + } +
Maybe add something like:
/*
* We can race with someone making the folio large again until
* we have locked the folio below.
*
* If we did, we will do the SIE entry/exit dance and end up
* here again.
*/
Quoted 24 lines
arch/s390/kvm/gmap.c new file +24
-0+ rc = -EAGAIN ; +if (folio_trylock (folio)) { +if (should_export_before_import (uvcb, gmap->mm)) +uv_convert_from_secure (folio_to_phys (folio)); + rc =make_folio_secure (folio, uvcb); +folio_unlock (folio); + } + +/* +* Unlikely case: the page is not mapped anymore. Return success +* and let the proper fault handler fault in the page again. +*/ +if (rc == -ENXIO ) +return 0 ; +/* The folio has too many references, try to shake some off */ +if (rc == -EBUSY ) { +mmap_read_unlock (gmap->mm); +uv_wiggle_folio (folio, false); +mmap_read_lock (gmap->mm); +return -EAGAIN ; + } + +return rc; +}
Claudio Imbrenda
maintainerPATCH v2 4/15 KVM: s390: fake memslot for ucontrol VMs
In reply to Claudio ImbrendaCreate a fake memslot for ucontrol VMs. The fake memslot identity-maps userspace.
Now memslots will always be present, and ucontrol is not a special case anymore.
--- arch/s390/include/asm/kvm_host.h | 2 ++ arch/s390/kvm/kvm-s390.c | 15 ++++++++++++++- arch/s390/kvm/kvm-s390.h | 2 ++ 3 files changed, 18 insertions(+), 1 deletion(-)
arch/s390/include/asm/kvm_host.h +2 -0
@@ -30,6 +30,8 @@#define KVM_S390_ESCA_CPU_SLOTS 248#define KVM_MAX_VCPUS 255 +#define KVM_INTERNAL_MEM_SLOTS 1 +/* * These seem to be used for allocating ->chip in the routing table, which we * don't use. 1 is as small as we can get to reduce the needed memory. If we
arch/s390/kvm/kvm-s390.c +14 -1
@@ -3429,8 +3429,18 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)VM_EVENT (kvm,3 ,"vm created with type %lu" , type);if (type &KVM_VM_S390_UCONTROL ) { +struct kvm_userspace_memory_region2 fake_memslot = { + .slot =KVM_S390_UCONTROL_MEMSLOT , + .guest_phys_addr =0 , + .userspace_addr =0 , + .memory_size =ALIGN_DOWN (TASK_SIZE , _SEGMENT_SIZE), + .flags =0 , + }; + kvm->arch.gmap =NULL ; kvm->arch.mem_limit =KVM_S390_NO_MEM_LIMIT ; +/* one flat fake memslot covering the whole address-space */ +KVM_BUG_ON (kvm_set_internal_memslot (kvm, &fake_memslot), kvm); }else {if (sclp.hamax ==U64_MAX ) kvm->arch.mem_limit =TASK_SIZE_MAX ;
@@ -5855,7 +5865,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, {gpa_t size; -if (kvm_is_ucontrol (kvm)) +if (kvm_is_ucontrol (kvm) && new->id <KVM_USER_MEM_SLOTS )return -EINVAL ;/* When we are protected, we should not change the memory slots */
@@ -5907,6 +5917,9 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, {int rc =0 ; +if (kvm_is_ucontrol (kvm)) +return ; +switch (change) {case KVM_MR_DELETE : rc = gmap_unmap_segment(kvm->arch.gmap, old->base_gfn * PAGE_SIZE,
arch/s390/kvm/kvm-s390.h +2 -0
@@ -20,6 +20,8 @@#include <asm/processor.h> #include <asm/sclp.h> +#define KVM_S390_UCONTROL_MEMSLOT (KVM_USER_MEM_SLOTS + 0) +static inline void kvm_s390_fpu_store (struct kvm_run *run) {fpu_stfpc (&run->s.regs.fpc);
Signature · 1 line
2.47.1
Janosch Frank
maintainerRe: [PATCH v2 04/15] KVM: s390: fake memslot for ucontrol VMs
Quoted 8 lines · Claudio Imbrenda
Create a fake memslot for ucontrol VMs. The fake memslot identity-maps userspace.
Now memslots will always be present, and ucontrol is not a special case anymore.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Claudio Imbrenda <[email protected]>
You'll need to update the Documentation because of the ucontrol EINVAL/EEXIST return change.
I don't like having EEXIST as a return value but I also don't see a lot of gains in changing common code just so we can return the correct rc.
Claudio Imbrenda
maintainerRe: [PATCH v2 04/15] KVM: s390: fake memslot for ucontrol VMs
Quoted 4 lines · On Thu, 16 Jan 2025 13:42:50 +0100 Janosch Frank <[email protected]> wrote:
Quoted 3 lines · Claudio Imbrenda
Create a fake memslot for ucontrol VMs. The fake memslot identity-maps userspace.
Quoted 8 lines
Quoted 5 lines
Now memslots will always be present, and ucontrol is not a special case anymore.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Claudio Imbrenda <[email protected]>You'll need to update the Documentation because of the ucontrol EINVAL/EEXIST return change.
ufff that fell off my brain cache; I'll fix it
Quoted 3 lines
I don't like having EEXIST as a return value but I also don't see a lot of gains in changing common code just so we can return the correct rc.
Claudio Imbrenda
maintainerPATCH v2 5/15 KVM: s390: selftests: fix ucontrol memory region test
In reply to Claudio ImbrendaWith the latest patch, attempting to create a memslot from userspace will result in an EEXIST error for UCONTROL VMs, instead of EINVAL, since the new memslot will collide with the internal memslot. There is no simple way to bring back the previous behaviour.
This is not a problem, but the test needs to be fixed accordingly.
--- tools/testing/selftests/kvm/s390x/ucontrol_test.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/s390x/ucontrol_test.c +4 -2
@@ -459,10 +459,12 @@ TEST_F(uc_kvm, uc_no_user_region) };ASSERT_EQ (-1 ,ioctl (self->vm_fd,KVM_SET_USER_MEMORY_REGION , ®ion)); -ASSERT_EQ (EINVAL , errno); +if (errno !=EEXIST ) +ASSERT_EQ (EINVAL , errno);ASSERT_EQ (-1 ,ioctl (self->vm_fd,KVM_SET_USER_MEMORY_REGION2 , ®ion2)); -ASSERT_EQ (EINVAL , errno); +if (errno !=EEXIST ) +ASSERT_EQ (EINVAL , errno); }TEST_F (uc_kvm, uc_map_unmap)
Signature · 1 line
2.47.1
Claudio Imbrenda
maintainerPATCH v2 6/15 KVM: s390: use __kvm_faultin_pfn()
In reply to Claudio ImbrendaRefactor the existing page fault handling code to use __kvm_faultin_pfn().
This possible now that memslots are always present.
--- arch/s390/kvm/kvm-s390.c | 122 ++++++++++++++++++++++++++++++--------- arch/s390/kvm/kvm-s390.h | 6 ++ arch/s390/mm/gmap.c | 1 + 3 files changed, 102 insertions(+), 27 deletions(-)
arch/s390/kvm/kvm-s390.c +95 -27
@@ -4784,11 +4784,102 @@ static void kvm_s390_assert_primary_as(struct kvm_vcpu *vcpu) current->thread.gmap_int_code, current->thread.gmap_teid.val); } +/* +* __kvm_s390_handle_dat_fault() - handle a dat fault for the gmap of a vcpu +* @vcpu: the vCPU whose gmap is to be fixed up +* @gfn: the guest frame number used for memslots (including fake memslots) +* @gaddr: the gmap address, does not have to match @gfn for ucontrol gmaps +* @flags: FOLL_* flags +* +* Return: 0 on success, < 0 in case or error. +* Context: The mm lock must not be held before calling. +*/ +int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags) +{ +struct kvm_memory_slot *slot; +unsigned int fault_flags; +bool writable, unlocked; +unsigned long vmaddr; +struct page *page; +kvm_pfn_t pfn; +int rc; + + slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn); +if (!slot || slot->flags &KVM_MEMSLOT_INVALID ) +return vcpu_post_run_addressing_exception (vcpu); + + fault_flags = flags &FOLL_WRITE ?FAULT_FLAG_WRITE :0 ; +if (vcpu->arch.gmap->pfault_enabled) + flags |=FOLL_NOWAIT ; + vmaddr =__gfn_to_hva_memslot (slot, gfn); + +try_again: + pfn =__kvm_faultin_pfn (slot, gfn, flags, &writable, &page); + +/* Access outside memory, inject addressing exception */ +if (is_noslot_pfn (pfn)) +return vcpu_post_run_addressing_exception (vcpu); +/* Signal pending: try again */ +if (pfn ==KVM_PFN_ERR_SIGPENDING ) +return -EAGAIN ; + +/* Needs I/O, try to setup async pfault (only possible with FOLL_NOWAIT) */ +if (pfn ==KVM_PFN_ERR_NEEDS_IO ) { +trace_kvm_s390_major_guest_pfault (vcpu); +if (kvm_arch_setup_async_pf (vcpu)) +return 0 ; + vcpu->stat.pfault_sync++; +/* Could not setup async pfault, try again synchronously */ + flags &= ~FOLL_NOWAIT ; + goto try_again; + } +/* Any other error */ +if (is_error_pfn (pfn)) +return -EFAULT ; + +/* Success */ +mmap_read_lock (vcpu->arch.gmap->mm); +/* Mark the userspace PTEs as young and/or dirty, to avoid page fault loops */ + rc =fixup_user_fault (vcpu->arch.gmap->mm, vmaddr, fault_flags, &unlocked); +if (!rc) + rc =__gmap_link (vcpu->arch.gmap, gaddr, vmaddr); +kvm_release_faultin_page (vcpu->kvm, page, false, writable); +mmap_read_unlock (vcpu->arch.gmap->mm); +return rc; +} + +static int vcpu_dat_fault_handler (struct kvm_vcpu *vcpu,unsigned long gaddr,unsigned int flags) +{ +unsigned long gaddr_tmp; +gfn_t gfn; + + gfn =gpa_to_gfn (gaddr); +if (kvm_is_ucontrol (vcpu->kvm)) { +/* +* This translates the per-vCPU guest address into a +* fake guest address, which can then be used with the +* fake memslots that are identity mapping userspace. +* This allows ucontrol VMs to use the normal fault +* resolution path, like normal VMs. +*/ +mmap_read_lock (vcpu->arch.gmap->mm); + gaddr_tmp =__gmap_translate (vcpu->arch.gmap, gaddr); +mmap_read_unlock (vcpu->arch.gmap->mm); +if (gaddr_tmp == -EFAULT ) { + vcpu->run->exit_reason =KVM_EXIT_S390_UCONTROL ; + vcpu->run->s390_ucontrol.trans_exc_code = gaddr; + vcpu->run->s390_ucontrol.pgm_code =PGM_SEGMENT_TRANSLATION ; +return -EREMOTE ; + } + gfn =gpa_to_gfn (gaddr_tmp); + } +return __kvm_s390_handle_dat_fault (vcpu, gfn, gaddr, flags); +} +static int vcpu_post_run_handle_fault (struct kvm_vcpu *vcpu) {unsigned int flags =0 ;unsigned long gaddr; -int rc =0 ; gaddr = current->thread.gmap_teid.addr *PAGE_SIZE ;if (kvm_s390_cur_gmap_fault_is_write ())
@@ -4840,37 +4931,14 @@ static int vcpu_post_run_handle_fault(struct kvm_vcpu *vcpu)case PGM_REGION_SECOND_TRANS :case PGM_REGION_THIRD_TRANS :kvm_s390_assert_primary_as (vcpu); -if (vcpu->arch.gmap->pfault_enabled) { - rc =gmap_fault (vcpu->arch.gmap, gaddr, flags |FAULT_FLAG_RETRY_NOWAIT ); -if (rc == -EFAULT ) -return vcpu_post_run_addressing_exception (vcpu); -if (rc == -EAGAIN ) { -trace_kvm_s390_major_guest_pfault (vcpu); -if (kvm_arch_setup_async_pf (vcpu)) -return 0 ; - vcpu->stat.pfault_sync++; - }else { -return rc; - } - } - rc =gmap_fault (vcpu->arch.gmap, gaddr, flags); -if (rc == -EFAULT ) { -if (kvm_is_ucontrol (vcpu->kvm)) { - vcpu->run->exit_reason =KVM_EXIT_S390_UCONTROL ; - vcpu->run->s390_ucontrol.trans_exc_code = gaddr; - vcpu->run->s390_ucontrol.pgm_code =0x10 ; -return -EREMOTE ; - } -return vcpu_post_run_addressing_exception (vcpu); - } -break ; +return vcpu_dat_fault_handler (vcpu, gaddr, flags);default :KVM_BUG (1 , vcpu->kvm,"Unexpected program interrupt 0x%x, TEID 0x%016lx" , current->thread.gmap_int_code, current->thread.gmap_teid.val);send_sig (SIGSEGV , current,0 );break ; } -return rc; +return 0 ; }static int vcpu_post_run (struct kvm_vcpu *vcpu,int exit_reason)
@@ -5749,7 +5817,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp, }#endif case KVM_S390_VCPU_FAULT : { - r =gmap_fault (vcpu->arch.gmap, arg,0 ); + r =vcpu_dat_fault_handler (vcpu, arg,0 );break ; }case KVM_ENABLE_CAP :
arch/s390/kvm/kvm-s390.h +6 -0
@@ -410,6 +410,12 @@ void kvm_s390_vcpu_unsetup_cmma(struct kvm_vcpu *vcpu);void kvm_s390_set_cpu_timer (struct kvm_vcpu *vcpu,__u64 cputm);__u64 kvm_s390_get_cpu_timer (struct kvm_vcpu *vcpu);int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc); +int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags); + +static inline int kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gpa_t gaddr,unsigned int flags) +{ +return __kvm_s390_handle_dat_fault (vcpu,gpa_to_gfn (gaddr), gaddr, flags); +}/* implemented in diag.c */ int kvm_s390_handle_diag (struct kvm_vcpu *vcpu);
arch/s390/mm/gmap.c +1 -0
@@ -605,6 +605,7 @@ int __gmap_link(struct gmap *gmap, unsigned long gaddr, unsigned long vmaddr)radix_tree_preload_end ();return rc; } +EXPORT_SYMBOL (__gmap_link);/** * fixup_user_fault_nowait - manually resolve a user page fault without waiting
Signature · 1 line
2.47.1
Re: [PATCH v2 06/15] KVM: s390: use __kvm_faultin_pfn()
Quoted 17 lines · Claudio Imbrenda
Refactor the existing page fault handling code to use
__kvm_faultin_pfn().This possible now that memslots are always present.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kvm/kvm-s390.c | 122 ++++++++++++++++++++++++++++++--------- arch/s390/kvm/kvm-s390.h | 6 ++ arch/s390/mm/gmap.c | 1 + 3 files changed, 102 insertions(+), 27 deletions(-)arch/s390/kvm/kvm-s390.c +0
-0@@ -4784,11 +4784,102 @@ static void kvm_s390_assert_primary_as(struct kvm_vcpu *vcpu) current->thread.gmap_int_code, current->thread.gmap_teid.val);
arch/s390/kvm/kvm-s390.c static void kvm_s390_assert_primary_as(struct kvm_vcpu *vcpu) · line 4794
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +8
-0} +/* +* __kvm_s390_handle_dat_fault() - handle a dat fault for the gmap of a vcpu +* @vcpu: the vCPU whose gmap is to be fixed up +* @gfn: the guest frame number used for memslots (including fake memslots) +* @gaddr: the gmap address, does not have to match @gfn for ucontrol gmaps +* @flags: FOLL_* flags +* +* Return: 0 on success, < 0 in case or error.
s/or/of/
With nit fixed:
Claudio Imbrenda
maintainerPATCH v2 7/15 KVM: s390: get rid of gmap_fault()
In reply to Claudio ImbrendaAll gmap page faults are already handled in kvm by the function kvm_s390_handle_dat_fault(); only few users of gmap_fault remained, all within kvm.
Convert those calls to use kvm_s390_handle_dat_fault() instead.
Remove gmap_fault() entirely since it has no more users.
--- arch/s390/include/asm/gmap.h | 1 - arch/s390/kvm/intercept.c | 4 +- arch/s390/mm/gmap.c | 124 ----------------------------------- 3 files changed, 2 insertions(+), 127 deletions(-)
arch/s390/include/asm/gmap.h +0 -1
@@ -113,7 +113,6 @@ int gmap_unmap_segment(struct gmap *gmap, unsigned long to, unsigned long len);unsigned long __gmap_translate (struct gmap *,unsigned long gaddr);unsigned long gmap_translate (struct gmap *,unsigned long gaddr);int __gmap_link (struct gmap *gmap,unsigned long gaddr,unsigned long vmaddr); -int gmap_fault (struct gmap *,unsigned long gaddr,unsigned int fault_flags);void gmap_discard (struct gmap *,unsigned long from,unsigned long to);void __gmap_zap (struct gmap *,unsigned long gaddr);void gmap_unlink (struct mm_struct *,unsigned long *table,unsigned long vmaddr);
arch/s390/kvm/intercept.c +2 -2
@@ -368,7 +368,7 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu) reg2, &srcaddr, GACC_FETCH, 0);if (rc)return kvm_s390_inject_prog_cond (vcpu, rc); - rc =gmap_fault (vcpu->arch.gmap, srcaddr,0 ); + rc =kvm_s390_handle_dat_fault (vcpu, srcaddr,0 );if (rc !=0 )return rc;
@@ -377,7 +377,7 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu) reg1, &dstaddr, GACC_STORE, 0);if (rc)return kvm_s390_inject_prog_cond (vcpu, rc); - rc =gmap_fault (vcpu->arch.gmap, dstaddr,FAULT_FLAG_WRITE ); + rc =kvm_s390_handle_dat_fault (vcpu, dstaddr,FOLL_WRITE );if (rc !=0 )return rc;
arch/s390/mm/gmap.c +0 -124
130 lines not on this page · raw message
Signature · 1 line
2.47.1
Re: [PATCH v2 07/15] KVM: s390: get rid of gmap_fault()
Quoted 9 lines · Claudio Imbrenda
All gmap page faults are already handled in kvm by the function
kvm_s390_handle_dat_fault(); only few users ofgmap_faultremained, all within kvm.Convert those calls to use
kvm_s390_handle_dat_fault()instead.Remove
gmap_fault()entirely since it has no more users.Signed-off-by Claudio Imbrenda <[email protected]>
Claudio Imbrenda
maintainerPATCH v2 8/15 KVM: s390: get rid of gmap_translate()
In reply to Claudio ImbrendaAdd gpa_to_hva(), which uses memslots, and use it to replace all uses of gmap_translate().
--- arch/s390/include/asm/gmap.h | 1 - arch/s390/kvm/interrupt.c | 19 +++++++++++-------- arch/s390/kvm/kvm-s390.h | 9 +++++++++ arch/s390/mm/gmap.c | 20 -------------------- 4 files changed, 20 insertions(+), 29 deletions(-)
arch/s390/include/asm/gmap.h +0 -1
@@ -111,7 +111,6 @@ int gmap_map_segment(struct gmap *gmap, unsigned long from, unsigned long to, unsigned long len);int gmap_unmap_segment (struct gmap *gmap,unsigned long to,unsigned long len);unsigned long __gmap_translate (struct gmap *,unsigned long gaddr); -unsigned long gmap_translate (struct gmap *,unsigned long gaddr);int __gmap_link (struct gmap *gmap,unsigned long gaddr,unsigned long vmaddr);void gmap_discard (struct gmap *,unsigned long from,unsigned long to);void __gmap_zap (struct gmap *,unsigned long gaddr);
arch/s390/kvm/interrupt.c +11 -8
@@ -2893,7 +2893,8 @@ int kvm_set_routing_entry(struct kvm *kvm,struct kvm_kernel_irq_routing_entry *e, const struct kvm_irq_routing_entry *ue) { - u64 uaddr; + u64 uaddr_s, uaddr_i; +int idx;switch (ue->type) {/* we store the userspace addresses instead of the guest addresses */
@@ -2901,14 +2902,16 @@ int kvm_set_routing_entry(struct kvm *kvm,if (kvm_is_ucontrol (kvm))return -EINVAL ; e->set = set_adapter_int; - uaddr =gmap_translate (kvm->arch.gmap, ue->u.adapter.summary_addr); -if (uaddr == -EFAULT ) -return -EFAULT ; - e->adapter.summary_addr = uaddr; - uaddr =gmap_translate (kvm->arch.gmap, ue->u.adapter.ind_addr); -if (uaddr == -EFAULT ) + + idx =srcu_read_lock (&kvm->srcu); + uaddr_s =gpa_to_hva (kvm, ue->u.adapter.summary_addr); + uaddr_i =gpa_to_hva (kvm, ue->u.adapter.ind_addr); +srcu_read_unlock (&kvm->srcu, idx); + +if (kvm_is_error_hva (uaddr_s) ||kvm_is_error_hva (uaddr_i))return -EFAULT ; - e->adapter.ind_addr = uaddr; + e->adapter.summary_addr = uaddr_s; + e->adapter.ind_addr = uaddr_i; e->adapter.summary_offset = ue->u.adapter.summary_offset; e->adapter.ind_offset = ue->u.adapter.ind_offset; e->adapter.adapter_id = ue->u.adapter.adapter_id;
arch/s390/kvm/kvm-s390.h +9 -0
@@ -281,6 +281,15 @@ static inline u32 kvm_s390_get_gisa_desc(struct kvm *kvm)return gd; } +static inline hva_t gpa_to_hva (struct kvm *kvm,gpa_t gpa) +{ +hva_t hva =gfn_to_hva (kvm,gpa_to_gfn (gpa)); + +if (!kvm_is_error_hva (hva)) + hva |=offset_in_page (gpa); +return hva; +} +/* implemented in pv.c */ int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc);int kvm_s390_pv_create_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc);
arch/s390/mm/gmap.c +0 -20
@@ -463,26 +463,6 @@ unsigned long __gmap_translate(struct gmap *gmap, unsigned long gaddr) }EXPORT_SYMBOL_GPL (__gmap_translate); -/** -* gmap_translate - translate a guest address to a user space address -* @gmap: pointer to guest mapping meta data structure -* @gaddr: guest address -* -* Returns user space address which corresponds to the guest address or -* -EFAULT if no such mapping exists. -* This function does not establish potentially missing page table entries. -*/ -unsigned long gmap_translate (struct gmap *gmap,unsigned long gaddr) -{ -unsigned long rc; - -mmap_read_lock (gmap->mm); - rc =__gmap_translate (gmap, gaddr); -mmap_read_unlock (gmap->mm); -return rc; -} -EXPORT_SYMBOL_GPL (gmap_translate); -/** * gmap_unlink - disconnect a page table from the gmap shadow tables * @mm: pointer to the parent mm_struct
Signature · 1 line
2.47.1
Re: [PATCH v2 08/15] KVM: s390: get rid of gmap_translate()
Quoted 11 lines · Claudio Imbrenda
Add
gpa_to_hva(), which uses memslots, and use it to replace all uses ofgmap_translate().Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/include/asm/gmap.h | 1 - arch/s390/kvm/interrupt.c | 19 +++++++++++-------- arch/s390/kvm/kvm-s390.h | 9 +++++++++ arch/s390/mm/gmap.c | 20 -------------------- 4 files changed, 20 insertions(+), 29 deletions(-)
Claudio Imbrenda
maintainerPATCH v2 9/15 KVM: s390: move some gmap shadowing functions away from mm/gmap.c
In reply to Claudio ImbrendaMove some gmap shadowing functions from mm/gmap.c to kvm/kvm-s390.c and the newly created kvm/gmap-vsie.c
This is a step toward removing gmap from mm.
--- arch/s390/include/asm/gmap.h | 9 +- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gmap-vsie.c | 142 +++++++++++++++++++++ arch/s390/kvm/gmap.h | 20 +++ arch/s390/kvm/kvm-s390.c | 62 +++++++++- arch/s390/kvm/kvm-s390.h | 2 + arch/s390/kvm/vsie.c | 2 + arch/s390/mm/gmap.c | 232 ++++------------------------------- 8 files changed, 254 insertions(+), 217 deletions(-) create mode 100644 arch/s390/kvm/gmap-vsie.c
arch/s390/include/asm/gmap.h +4 -5
@@ -106,6 +106,8 @@ struct gmap *gmap_create(struct mm_struct *mm, unsigned long limit);void gmap_remove (struct gmap *gmap);struct gmap *gmap_get (struct gmap *gmap);void gmap_put (struct gmap *gmap); +void gmap_free (struct gmap *gmap); +struct gmap *gmap_alloc (unsigned long limit);int gmap_map_segment (struct gmap *gmap,unsigned long from,unsigned long to,unsigned long len);
@@ -118,9 +120,7 @@ void gmap_unlink(struct mm_struct *, unsigned long *table, unsigned long vmaddr)int gmap_read_table (struct gmap *gmap,unsigned long gaddr,unsigned long *val); -struct gmap *gmap_shadow (struct gmap *parent,unsigned long asce, -int edat_level); -int gmap_shadow_valid (struct gmap *sg,unsigned long asce,int edat_level); +void gmap_unshadow (struct gmap *sg);int gmap_shadow_r2t (struct gmap *sg,unsigned long saddr,unsigned long r2t,int fake); int gmap_shadow_r3t(struct gmap *sg, unsigned long saddr, unsigned long r3t,
@@ -136,8 +136,7 @@ int gmap_shadow_page(struct gmap *sg, unsigned long saddr, pte_t pte);void gmap_register_pte_notifier (struct gmap_notifier *);void gmap_unregister_pte_notifier (struct gmap_notifier *); -int gmap_mprotect_notify (struct gmap *,unsigned long start, -unsigned long len,int prot); +int gmap_protect_one (struct gmap *gmap,unsigned long gaddr,int prot,unsigned long bits);void gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr);
arch/s390/kvm/Makefile +1 -1
@@ -8,7 +8,7 @@ include $(srctree)/virt/kvm/Makefile.kvm ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o -kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o +kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap.o gmap-vsie.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/gmap-vsie.c new file +142 -0
142 lines not on this page · raw message
arch/s390/kvm/gmap.h +20 -0
@@ -13,5 +13,25 @@int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr); +struct gmap *gmap_shadow (struct gmap *parent,unsigned long asce,int edat_level); + +/** +* gmap_shadow_valid - check if a shadow guest address space matches the +* given properties and is still valid +* @sg: pointer to the shadow guest address space structure +* @asce: ASCE for which the shadow table is requested +* @edat_level: edat level to be used for the shadow translation +* +* Returns 1 if the gmap shadow is still valid and matches the given +* properties, the caller can continue using it. Returns 0 otherwise, the +* caller has to request a new shadow gmap in this case. +* +*/ +static inline int gmap_shadow_valid (struct gmap *sg,unsigned long asce,int edat_level) +{ +if (sg->removed) +return 0 ; +return sg->orig_asce == asce && sg->edat_level == edat_level; +}#endif
arch/s390/kvm/kvm-s390.c +59 -3
74 lines not on this page · raw message
arch/s390/kvm/kvm-s390.h +2 -0
@@ -420,6 +420,8 @@ void kvm_s390_set_cpu_timer(struct kvm_vcpu *vcpu, __u64 cputm);__u64 kvm_s390_get_cpu_timer (struct kvm_vcpu *vcpu);int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc);int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags); +int __kvm_s390_mprotect_many (struct gmap *gmap,gpa_t gpa,u8 npages,unsigned int prot, +unsigned long bits);static inline int kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gpa_t gaddr,unsigned int flags) {
arch/s390/kvm/vsie.c +2 -0
@@ -13,6 +13,7 @@#include <linux/bitmap.h> #include <linux/sched/signal.h> #include <linux/io.h> +#include <linux/mman.h> #include <asm/gmap.h> #include <asm/mmu_context.h>
@@ -22,6 +23,7 @@#include <asm/facility.h> #include "kvm-s390.h" #include "gaccess.h" +#include "gmap.h" struct vsie_page {struct kvm_s390_sie_block scb_s;/* 0x0000 */
arch/s390/mm/gmap.c +24 -208
286 lines not on this page · raw message
Signature · 1 line
2.47.1
Janosch Frank
maintainerRe: [PATCH v2 09/15] KVM: s390: move some gmap shadowing functions away from mm/gmap.c
Quoted 6 lines · Claudio Imbrenda
Move some gmap shadowing functions from
mm/gmap.ctokvm/kvm-s390.cand the newly createdkvm/gmap-vsie.cThis is a step toward removing gmap from mm.
Signed-off-by Claudio Imbrenda <[email protected]>
[...]
Quoted 14 lines
arch/s390/mm/gmap.c +0
-14-/** -* gmap_mprotect_notify - change access rights for a range of ptes and -* call the notifier if any pte changes again -* @gmap: pointer to guest mapping meta data structure -* @gaddr: virtual address in the guest address space -* @len: size of area -* @prot: indicates access rights: PROT_NONE, PROT_READ or PROT_WRITE -* -* Returns 0 if for each page in the given range a gmap mapping exists, -* the new access rights could be set and the notifier could be armed. -* If the gmap mapping is missing for one or more pages -EFAULT is -* returned. If no memory could be allocated -ENOMEM is returned. -* This function establishes missing page table entries. -*/
arch/s390/mm/gmap.c static int gmap_protect_pte(struct gmap *gmap, unsigned long gaddr, · line 986
Quoted 10 lines
arch/s390/mm/gmap.c +6
-4-int gmap_mprotect_notify (struct gmap *gmap,unsigned long gaddr, -unsigned long len,int prot) -{ -int rc; +if (!pmd_leaf (*pmdp)) { + rc =gmap_protect_pte (gmap, gaddr, pmdp, prot, bits); + }else { + rc =gmap_protect_pmd (gmap, gaddr, pmdp, prot, bits); +if (!rc) + rc =1 ;
Instead of doing this, could you return PAGE_SIZE or HPAGE_SIZE if !rc?
That would also get rid of the ternary in the ALIGN() in __kvm_s390_mprotect_many() which is also a thorn in my side.
Claudio Imbrenda
maintainerPATCH v2 10/15 KVM: s390: stop using page->index for non-shadow gmaps
In reply to Claudio ImbrendaThe host_to_guest radix tree will now map userspace addresses to guest addresses, instead of userspace addresses to segment tables.
When segment tables and page tables are needed, they are found using an additional gmap_table_walk().
This gets rid of all usage of page->index for non-shadow gmaps.
--- arch/s390/mm/gmap.c | 96 +++++++++++++++++++++------------------------ 1 file changed, 45 insertions(+), 51 deletions(-)
arch/s390/mm/gmap.c +45 -51
@@ -24,8 +24,11 @@#include <asm/page.h> #include <asm/tlb.h> +#define GADDR_VALID (gaddr) ((gaddr) & 1)#define GMAP_SHADOW_FAKE_TABLE 1ULL +static inline unsigned long *gmap_table_walk (struct gmap *gmap,unsigned long gaddr,int level); +static struct page *gmap_alloc_crst (void ) {struct page *page;
@@ -82,7 +85,6 @@ struct gmap *gmap_alloc(unsigned long limit) page =gmap_alloc_crst ();if (!page) goto out_free; - page->index =0 ;list_add (&page->lru, &gmap->crst_list); table =page_to_virt (page);crst_table_init (table, etype);
@@ -303,7 +305,6 @@ static int gmap_alloc_table(struct gmap *gmap, unsigned long *table,list_add (&page->lru, &gmap->crst_list); *table =__pa (new) | _REGION_ENTRY_LENGTH | (*table & _REGION_ENTRY_TYPE_MASK); - page->index = gaddr; page =NULL ; }spin_unlock (&gmap->guest_table_lock);
@@ -312,21 +313,23 @@ static int gmap_alloc_table(struct gmap *gmap, unsigned long *table,return 0 ; } -/** -* __gmap_segment_gaddr - find virtual address from segment pointer -* @entry: pointer to a segment table entry in the guest address space -* -* Returns the virtual address in the guest address space for the segment -*/ -static unsigned long __gmap_segment_gaddr (unsigned long *entry) +static unsigned long host_to_guest_lookup (struct gmap *gmap,unsigned long vmaddr) { -struct page *page; -unsigned long offset; +return (unsigned long )radix_tree_lookup (&gmap->host_to_guest, vmaddr >>PMD_SHIFT ); +} - offset = (unsigned long ) entry /sizeof (unsigned long ); - offset = (offset & (PTRS_PER_PMD -1 )) *PMD_SIZE ; - page =pmd_pgtable_page ((pmd_t *) entry); -return page->index + offset; +static unsigned long host_to_guest_delete (struct gmap *gmap,unsigned long vmaddr) +{ +return (unsigned long )radix_tree_delete (&gmap->host_to_guest, vmaddr >>PMD_SHIFT ); +} + +static pmd_t *host_to_guest_pmd_delete (struct gmap *gmap,unsigned long vmaddr, +unsigned long *gaddr) +{ + *gaddr =host_to_guest_delete (gmap, vmaddr); +if (GADDR_VALID (*gaddr)) +return (pmd_t *)gmap_table_walk (gmap, *gaddr,1 ); +return NULL ; }/**
@@ -338,16 +341,19 @@ static unsigned long __gmap_segment_gaddr(unsigned long *entry)*/ static int __gmap_unlink_by_vmaddr (struct gmap *gmap,unsigned long vmaddr) { -unsigned long *entry; +unsigned long gaddr;int flush =0 ; +pmd_t *pmdp;BUG_ON (gmap_is_shadow (gmap));spin_lock (&gmap->guest_table_lock); - entry =radix_tree_delete (&gmap->host_to_guest, vmaddr >>PMD_SHIFT ); -if (entry) { - flush = (*entry != _SEGMENT_ENTRY_EMPTY); - *entry = _SEGMENT_ENTRY_EMPTY; + + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr); +if (pmdp) { + flush = (pmd_val (*pmdp) != _SEGMENT_ENTRY_EMPTY); + *pmdp =__pmd (_SEGMENT_ENTRY_EMPTY); } +spin_unlock (&gmap->guest_table_lock);return flush; }
@@ -564,7 +570,8 @@ int __gmap_link(struct gmap *gmap, unsigned long gaddr, unsigned long vmaddr)spin_lock (&gmap->guest_table_lock);if (*table == _SEGMENT_ENTRY_EMPTY) { rc =radix_tree_insert (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT , table); + vmaddr >>PMD_SHIFT , + (void *)((gaddr &HPAGE_MASK ) |1 ));if (!rc) {if (pmd_leaf (*pmd)) { *table = (pmd_val(*pmd) &
@@ -1991,7 +1998,6 @@ void ptep_notify(struct mm_struct *mm, unsigned long vmaddr, pte_t *pte, unsigned long bits) {unsigned long offset, gaddr =0 ; -unsigned long *table;struct gmap *gmap, *sg, *next; offset = ((unsigned long ) pte) & (255 *sizeof (pte_t));
@@ -1999,12 +2005,9 @@ void ptep_notify(struct mm_struct *mm, unsigned long vmaddr,rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - table =radix_tree_lookup (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); -if (table) - gaddr =__gmap_segment_gaddr (table) + offset; + gaddr =host_to_guest_lookup (gmap, vmaddr) + offset;spin_unlock (&gmap->guest_table_lock); -if (!table) +if (!GADDR_VALID (gaddr))continue ;if (!list_empty (&gmap->children) && (bits &PGSTE_VSIE_BIT )) {
@@ -2064,10 +2067,8 @@ static void gmap_pmdp_clear(struct mm_struct *mm, unsigned long vmaddr,rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - pmdp = (pmd_t *)radix_tree_delete (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr);if (pmdp) { - gaddr =__gmap_segment_gaddr ((unsigned long *)pmdp);pmdp_notify_gmap (gmap, pmdp, gaddr); WARN_ON(pmd_val(*pmdp) & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | _SEGMENT_ENTRY_GMAP_UC |
@@ -2111,28 +2112,25 @@ EXPORT_SYMBOL_GPL(gmap_pmdp_csp);*/ void gmap_pmdp_idte_local (struct mm_struct *mm,unsigned long vmaddr) { -unsigned long *entry, gaddr; +unsigned long gaddr;struct gmap *gmap;pmd_t *pmdp;rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - entry =radix_tree_delete (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); -if (entry) { - pmdp = (pmd_t *)entry; - gaddr =__gmap_segment_gaddr (entry); + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr); +if (pmdp) {pmdp_notify_gmap (gmap, pmdp, gaddr); -WARN_ON (*entry & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | - _SEGMENT_ENTRY_GMAP_UC | - _SEGMENT_ENTRY)); +WARN_ON (pmd_val (*pmdp) & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | + _SEGMENT_ENTRY_GMAP_UC | + _SEGMENT_ENTRY));if (MACHINE_HAS_TLB_GUEST )__pmdp_idte (gaddr, pmdp,IDTE_GUEST_ASCE , gmap->asce,IDTE_LOCAL );else if (MACHINE_HAS_IDTE )__pmdp_idte (gaddr, pmdp,0 ,0 ,IDTE_LOCAL ); - *entry = _SEGMENT_ENTRY_EMPTY; + *pmdp =__pmd (_SEGMENT_ENTRY_EMPTY); }spin_unlock (&gmap->guest_table_lock); }
@@ -2147,22 +2145,19 @@ EXPORT_SYMBOL_GPL(gmap_pmdp_idte_local);*/ void gmap_pmdp_idte_global (struct mm_struct *mm,unsigned long vmaddr) { -unsigned long *entry, gaddr; +unsigned long gaddr;struct gmap *gmap;pmd_t *pmdp;rcu_read_lock ();list_for_each_entry_rcu (gmap, &mm->context.gmap_list, list) {spin_lock (&gmap->guest_table_lock); - entry =radix_tree_delete (&gmap->host_to_guest, - vmaddr >>PMD_SHIFT ); -if (entry) { - pmdp = (pmd_t *)entry; - gaddr =__gmap_segment_gaddr (entry); + pmdp =host_to_guest_pmd_delete (gmap, vmaddr, &gaddr); +if (pmdp) {pmdp_notify_gmap (gmap, pmdp, gaddr); -WARN_ON (*entry & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | - _SEGMENT_ENTRY_GMAP_UC | - _SEGMENT_ENTRY)); +WARN_ON (pmd_val (*pmdp) & ~(_SEGMENT_ENTRY_HARDWARE_BITS_LARGE | + _SEGMENT_ENTRY_GMAP_UC | + _SEGMENT_ENTRY));if (MACHINE_HAS_TLB_GUEST )__pmdp_idte (gaddr, pmdp,IDTE_GUEST_ASCE , gmap->asce,IDTE_GLOBAL );
@@ -2170,7 +2165,7 @@ void gmap_pmdp_idte_global(struct mm_struct *mm, unsigned long vmaddr)__pmdp_idte (gaddr, pmdp,0 ,0 ,IDTE_GLOBAL ); else__pmdp_csp (pmdp ); - *entry = _SEGMENT_ENTRY_EMPTY; + *pmdp =__pmd (_SEGMENT_ENTRY_EMPTY); }spin_unlock (&gmap->guest_table_lock); }
@@ -2686,7 +2681,6 @@ int s390_replace_asce(struct gmap *gmap) page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index =0 ; table =page_to_virt (page);memcpy (table, gmap->table,1UL << (CRST_ALLOC_ORDER +PAGE_SHIFT ));
Signature · 1 line
2.47.1
Janosch Frank
maintainerRe: [PATCH v2 10/15] KVM: s390: stop using page->index for non-shadow gmaps
Quoted 18 lines · Claudio Imbrenda
The
host_to_guestradix tree will now map userspace addresses to guest addresses, instead of userspace addresses to segment tables.When segment tables and page tables are needed, they are found using an additional
gmap_table_walk().This gets rid of all usage of
page->indexfor non-shadow gmaps.Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/mm/gmap.c | 96 +++++++++++++++++++++------------------------ 1 file changed, 45 insertions(+), 51 deletions(-)arch/s390/mm/gmap.c +0
-0
arch/s390/mm/gmap.c line 27
Quoted 4 lines
arch/s390/mm/gmap.c +1
-0@@ -24,8 +24,11 @@#include <asm/page.h> #include <asm/tlb.h> +#define GADDR_VALID (gaddr) ((gaddr) & 1)
That's wild, please make the 1 a named constant.
Claudio Imbrenda
maintainerPATCH v2 11/15 KVM: s390: stop using lists to keep track of used dat tables
In reply to Claudio ImbrendaUntil now, every dat table allocated to map a guest was put in a linked list. The page->lru field of struct page was used to keep track of which pages were being used, and when the gmap is torn down, the list was walked and all pages freed.
This patch gets rid of the usage of page->lru. Page tables are now freed by recursively walking the dat table tree.
Since s390_unlist_old_asce() becomes useless now, remove it.
--- arch/s390/include/asm/gmap.h | 3 -- arch/s390/mm/gmap.c | 102 ++++++++--------------------------- 2 files changed, 23 insertions(+), 82 deletions(-)
arch/s390/include/asm/gmap.h +0 -3
@@ -45,7 +45,6 @@*/ struct gmap {struct list_head list; -struct list_head crst_list;struct mm_struct *mm;struct radix_tree_root guest_to_host;struct radix_tree_root host_to_guest;
@@ -61,7 +60,6 @@ struct gmap {/* Additional data for shadow guest address spaces */ struct radix_tree_root host_to_rmap;struct list_head children; -struct list_head pt_list;spinlock_t shadow_lock;struct gmap *parent;unsigned long orig_asce;
@@ -141,7 +139,6 @@ int gmap_protect_one(struct gmap *gmap, unsigned long gaddr, int prot, unsignedvoid gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr);int s390_disable_cow_sharing (void ); -void s390_unlist_old_asce (struct gmap *gmap);int s390_replace_asce (struct gmap *gmap);void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns); int __s390_uv_destroy_range(struct mm_struct *mm, unsigned long start,
arch/s390/mm/gmap.c +23 -79
232 lines not on this page · raw message
Signature · 1 line
2.47.1
Re: [PATCH v2 11/15] KVM: s390: stop using lists to keep track of used dat tables
Quoted 11 lines · Claudio Imbrenda
Until now, every dat table allocated to map a guest was put in a linked list. The
page->lrufield of struct page was used to keep track of which pages were being used, and when the gmap is torn down, the list was walked and all pages freed.This patch gets rid of the usage of
page->lru. Page tables are now freed by recursively walking the dat table tree.Since
s390_unlist_old_asce()becomes useless now, remove it.Signed-off-by Claudio Imbrenda <[email protected]>
Claudio Imbrenda
maintainerPATCH v2 12/15 KVM: s390: move gmap_shadow_pgt_lookup() into kvm
In reply to Claudio ImbrendaMove gmap_shadow_pgt_lookup() from mm/gmap.c into kvm/gaccess.c .
--- arch/s390/include/asm/gmap.h | 3 +-- arch/s390/kvm/gaccess.c | 40 +++++++++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 2 ++ arch/s390/mm/gmap.c | 46 ++---------------------------------- 4 files changed, 45 insertions(+), 46 deletions(-)
arch/s390/include/asm/gmap.h +1 -2
@@ -127,8 +127,6 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, int fake);int gmap_shadow_pgt (struct gmap *sg,unsigned long saddr,unsigned long pgt,int fake); -int gmap_shadow_pgt_lookup (struct gmap *sg,unsigned long saddr, -unsigned long *pgt,int *dat_protection,int *fake);int gmap_shadow_page (struct gmap *sg,unsigned long saddr,pte_t pte);void gmap_register_pte_notifier (struct gmap_notifier *);
@@ -143,6 +141,7 @@ int s390_replace_asce(struct gmap *gmap);void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns);int __s390_uv_destroy_range (struct mm_struct *mm,unsigned long start,unsigned long end,bool interruptible); +unsigned long *gmap_table_walk (struct gmap *gmap,unsigned long gaddr,int level);/** * s390_uv_destroy_range - Destroy a range of pages in the given mm.
arch/s390/kvm/gaccess.c +40 -0
@@ -16,6 +16,7 @@#include <asm/gmap.h> #include <asm/dat-bits.h> #include "kvm-s390.h" +#include "gmap.h" #include "gaccess.h" /*
@@ -1392,6 +1393,42 @@ static int kvm_s390_shadow_tables(struct gmap *sg, unsigned long saddr,return 0 ; } +/** +* gmap_shadow_pgt_lookup - find a shadow page table +* @sg: pointer to the shadow guest address space structure +* @saddr: the address in the shadow aguest address space +* @pgt: parent gmap address of the page table to get shadowed +* @dat_protection: if the pgtable is marked as protected by dat +* @fake: pgt references contiguous guest memory block, not a pgtable +* +* Returns 0 if the shadow page table was found and -EAGAIN if the page +* table was not found. +* +* Called with sg->mm->mmap_lock in read. +*/ +static int gmap_shadow_pgt_lookup (struct gmap *sg,unsigned long saddr,unsigned long *pgt, +int *dat_protection,int *fake) +{ +unsigned long *table; +struct page *page; +int rc; + +spin_lock (&sg->guest_table_lock); + table =gmap_table_walk (sg, saddr,1 );/* get segment pointer */ +if (table && !(*table & _SEGMENT_ENTRY_INVALID)) { +/* Shadow page tables are full pages (pte+pgste) */ + page =pfn_to_page (*table >>PAGE_SHIFT ); + *pgt = page->index & ~GMAP_SHADOW_FAKE_TABLE ; + *dat_protection = !!(*table & _SEGMENT_ENTRY_PROTECT); + *fake = !!(page->index &GMAP_SHADOW_FAKE_TABLE ); + rc =0 ; + }else { + rc = -EAGAIN ; + } +spin_unlock (&sg->guest_table_lock); +return rc; +} +/** * kvm_s390_shadow_fault - handle fault on a shadow page table * @vcpu: virtual cpu
@@ -1415,6 +1452,9 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg,int dat_protection, fake;int rc; +if (KVM_BUG_ON (!gmap_is_shadow (sg), vcpu->kvm)) +return -EFAULT ; +mmap_read_lock (sg->mm);/* * We don't want any guest-2 tables to change - so the parent
arch/s390/kvm/gmap.h +2 -0
@@ -10,6 +10,8 @@#ifndef ARCH_KVM_S390_GMAP_H #define ARCH_KVM_S390_GMAP_H +#define GMAP_SHADOW_FAKE_TABLE 1ULL +int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);
arch/s390/mm/gmap.c +2 -44
70 lines not on this page · raw message
Signature · 1 line
2.47.1
Re: [PATCH v2 12/15] KVM: s390: move gmap_shadow_pgt_lookup() into kvm
Quoted 2 lines · Claudio Imbrenda
Move
gmap_shadow_pgt_lookup()frommm/gmap.cintokvm/gaccess.c.
+1 for the replacement of BUG_ON with KVM_BUG_ON
Quoted 1 line
Signed-off-by Claudio Imbrenda <[email protected]>
Janosch Frank
maintainerRe: [PATCH v2 12/15] KVM: s390: move gmap_shadow_pgt_lookup() into kvm
In reply to Claudio ImbrendaQuoted 9 lines · Claudio Imbrenda
Move
gmap_shadow_pgt_lookup()frommm/gmap.cintokvm/gaccess.c.Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/include/asm/gmap.h | 3 +-- arch/s390/kvm/gaccess.c | 40 +++++++++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 2 ++ arch/s390/mm/gmap.c | 46 ++---------------------------------- 4 files changed, 45 insertions(+), 46 deletions(-)
[...]
Quoted 3 lines
+/**
+ *gmap_shadow_pgt_lookup- find a shadow page table
The other two VSIE functions in gaccess.c have the kvm_s390_shadow prefix but this one is only static anyway and hence we could just drop the gmap_ from the name.
Quoted 11 lines
+ * @sg: pointer to the shadow guest address space structure
+ * @saddr: the address in the shadow aguest address space
+ * @pgt: parent gmap address of the page table to get shadowed
+ * @dat_protection: if the pgtable is marked as protected by dat
+ * @fake: pgt references contiguous guest memory block, not a pgtable
+ *
+ * Returns 0 if the shadow page table was found and-EAGAINif the page
+ * table was not found.
+ *
+ * Called withsg->mm->mmap_lockin read.
+ */
arch/s390/kvm/gaccess.c static int kvm_s390_shadow_tables(struct gmap *sg, unsigned long saddr, · line 1417
Quoted 9 lines
arch/s390/kvm/gaccess.c +9
-0+static int gmap_shadow_pgt_lookup (struct gmap *sg,unsigned long saddr,unsigned long *pgt, +int *dat_protection,int *fake) +{ +unsigned long *table; +struct page *page; +int rc; + +spin_lock (&sg->guest_table_lock); + table =gmap_table_walk (sg, saddr,1 );/* get segment pointer */
I'd be happy if you could introduce an enum for the level argument in a future series.
Quoted 15 lines
arch/s390/kvm/gaccess.c +14
-0+if (table && !(*table & _SEGMENT_ENTRY_INVALID)) { +/* Shadow page tables are full pages (pte+pgste) */ + page =pfn_to_page (*table >>PAGE_SHIFT ); + *pgt = page->index & ~GMAP_SHADOW_FAKE_TABLE ; + *dat_protection = !!(*table & _SEGMENT_ENTRY_PROTECT); + *fake = !!(page->index &GMAP_SHADOW_FAKE_TABLE ); + rc =0 ; + }else { + rc = -EAGAIN ; + } +spin_unlock (&sg->guest_table_lock); +return rc; +} +
Claudio Imbrenda
maintainerRe: [PATCH v2 12/15] KVM: s390: move gmap_shadow_pgt_lookup() into kvm
Quoted 14 lines · On Mon, 20 Jan 2025 14:47:55 +0100 Janosch Frank <[email protected]> wrote:
Quoted 9 lines · Claudio Imbrenda
Move
gmap_shadow_pgt_lookup()frommm/gmap.cintokvm/gaccess.c.Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/include/asm/gmap.h | 3 +-- arch/s390/kvm/gaccess.c | 40 +++++++++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 2 ++ arch/s390/mm/gmap.c | 46 ++---------------------------------- 4 files changed, 45 insertions(+), 46 deletions(-)[...]
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
+/**
+ *gmap_shadow_pgt_lookup- find a shadow page tableThe other two VSIE functions in
gaccess.chave thekvm_s390_shadowprefix but this one is only static anyway and hence we could just drop thegmap_from the name.
fair enough, will fix
Quoted 16 lines
Quoted 16 lines
+ * @sg: pointer to the shadow guest address space structure
+ * @saddr: the address in the shadow aguest address space
+ * @pgt: parent gmap address of the page table to get shadowed
+ * @dat_protection: if the pgtable is marked as protected by dat
+ * @fake: pgt references contiguous guest memory block, not a pgtable
+ *
+ * Returns 0 if the shadow page table was found and-EAGAINif the page
+ * table was not found.
+ *
+ * Called withsg->mm->mmap_lockin read.
+ */arch/s390/kvm/gaccess.c +4
-0+static int gmap_shadow_pgt_lookup (struct gmap *sg,unsigned long saddr,unsigned long *pgt, +int *dat_protection,int *fake) +{ +unsigned long *table;
Quoted 8 lines
arch/s390/kvm/gaccess.c static int kvm_s390_shadow_tables(struct gmap *sg, unsigned long saddr, · line 1417
Quoted 5 lines
arch/s390/kvm/gaccess.c +5
-0+struct page *page; +int rc; + +spin_lock (&sg->guest_table_lock); + table =gmap_table_walk (sg, saddr,1 );/* get segment pointer */ I'd be happy if you could introduce an enum for the level argument in a future series.
in upcoming series, the gmap table walk will be completely rewritten, and yes, it will have enums
Quoted 16 lines
Quoted 16 lines
arch/s390/kvm/gaccess.c +14
-0+if (table && !(*table & _SEGMENT_ENTRY_INVALID)) { +/* Shadow page tables are full pages (pte+pgste) */ + page =pfn_to_page (*table >>PAGE_SHIFT ); + *pgt = page->index & ~GMAP_SHADOW_FAKE_TABLE ; + *dat_protection = !!(*table & _SEGMENT_ENTRY_PROTECT); + *fake = !!(page->index &GMAP_SHADOW_FAKE_TABLE ); + rc =0 ; + }else { + rc = -EAGAIN ; + } +spin_unlock (&sg->guest_table_lock); +return rc; +} +
Claudio Imbrenda
maintainerPATCH v2 13/15 KVM: s390: remove useless page->index usage
In reply to Claudio ImbrendaThe page->index field for VSIE dat tables is only used for segment tables.
Stop setting the field for all region tables.
--- arch/s390/mm/gmap.c | 9 --------- 1 file changed, 9 deletions(-)
arch/s390/mm/gmap.c +0 -9
@@ -1507,9 +1507,6 @@ int gmap_shadow_r2t(struct gmap *sg, unsigned long saddr, unsigned long r2t, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = r2t & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_r2t =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
@@ -1590,9 +1587,6 @@ int gmap_shadow_r3t(struct gmap *sg, unsigned long saddr, unsigned long r3t, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = r3t & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_r3t =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
@@ -1673,9 +1667,6 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = sgt & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_sgt =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
Signature · 1 line
2.47.1
Re: [PATCH v2 13/15] KVM: s390: remove useless page->index usage
Quoted 6 lines · Claudio Imbrenda
The
page->indexfield for VSIE dat tables is only used for segment tables.Stop setting the field for all region tables.
Signed-off-by Claudio Imbrenda <[email protected]>
Haha, I never noticed, good catch.
Quoted 38 lines
--- arch/s390/mm/gmap.c | 9 --------- 1 file changed, 9 deletions(-)arch/s390/mm/gmap.c +0
-9@@ -1507,9 +1507,6 @@ int gmap_shadow_r2t(struct gmap *sg, unsigned long saddr, unsigned long r2t, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = r2t & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_r2t =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);@@ -1590,9 +1587,6 @@ int gmap_shadow_r3t(struct gmap *sg, unsigned long saddr, unsigned long r3t, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = r3t & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_r3t =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);@@ -1673,9 +1667,6 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, page =gmap_alloc_crst ();if (!page)return -ENOMEM ; - page->index = sgt & _REGION_ENTRY_ORIGIN; -if (fake) - page->index |=GMAP_SHADOW_FAKE_TABLE ; s_sgt =page_to_phys (page);/* Install shadow region second table */ spin_lock (&sg->guest_table_lock);
Claudio Imbrenda
maintainerPATCH v2 14/15 KVM: s390: move PGSTE softbits
In reply to Claudio ImbrendaMove the softbits in the PGSTEs to the other usable area.
This leaves the 16-bit block of usable bits free, which will be used in the next patch for something else.
--- arch/s390/include/asm/pgtable.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/s390/include/asm/pgtable.h +3 -3
@@ -419,9 +419,9 @@ static inline int is_module_addr(void *addr)#define PGSTE_HC_BIT 0x0020000000000000UL#define PGSTE_GR_BIT 0x0004000000000000UL#define PGSTE_GC_BIT 0x0002000000000000UL -#define PGSTE_UC_BIT 0x0000800000000000UL/* user dirty (migration) */ -#define PGSTE_IN_BIT 0x0000400000000000UL/* IPTE notify bit */ -#define PGSTE_VSIE_BIT 0x0000200000000000UL/* ref'd in a shadow table */ +#define PGSTE_UC_BIT 0x0000000000008000UL/* user dirty (migration) */ +#define PGSTE_IN_BIT 0x0000000000004000UL/* IPTE notify bit */ +#define PGSTE_VSIE_BIT 0x0000000000002000UL/* ref'd in a shadow table */ /* Guest Page State used for virtualization */ #define _PGSTE_GPS_ZERO 0x0000000080000000UL
Signature · 1 line
2.47.1
Re: [PATCH v2 14/15] KVM: s390: move PGSTE softbits
Quoted 6 lines · Claudio Imbrenda
Move the softbits in the PGSTEs to the other usable area.
This leaves the 16-bit block of usable bits free, which will be used in the next patch for something else.
Signed-off-by Claudio Imbrenda <[email protected]>
Claudio Imbrenda
maintainerPATCH v2 15/15 KVM: s390: remove the last user of page->index
In reply to Claudio ImbrendaShadow page tables use page->index to keep the g2 address of the guest page table being shadowed.
Instead of keeping the information in page->index, split the address and smear it over the 16-bit softbits areas of 4 PGSTEs.
This removes the last s390 user of page->index.
--- arch/s390/include/asm/pgtable.h | 15 +++++++++++++++ arch/s390/kvm/gaccess.c | 6 ++++-- arch/s390/mm/gmap.c | 22 ++++++++++++++++++++-- 3 files changed, 39 insertions(+), 4 deletions(-)
arch/s390/include/asm/pgtable.h +15 -0
@@ -419,6 +419,7 @@ static inline int is_module_addr(void *addr)#define PGSTE_HC_BIT 0x0020000000000000UL#define PGSTE_GR_BIT 0x0004000000000000UL#define PGSTE_GC_BIT 0x0002000000000000UL +#define PGSTE_ST2_MASK 0x0000ffff00000000UL#define PGSTE_UC_BIT 0x0000000000008000UL/* user dirty (migration) */ #define PGSTE_IN_BIT 0x0000000000004000UL/* IPTE notify bit */ #define PGSTE_VSIE_BIT 0x0000000000002000UL/* ref'd in a shadow table */
@@ -2001,4 +2002,18 @@ extern void s390_reset_cmma(struct mm_struct *mm);#define pmd_pgtable (pmd) \ ((pgtable_t)__va(pmd_val(pmd) & -sizeof(pte_t)*PTRS_PER_PTE)) +static inline unsigned long gmap_pgste_get_index (unsigned long *pgt) +{ +unsigned long *pgstes, res; + + pgstes = pgt + _PAGE_ENTRIES; + + res = (pgstes[0 ] &PGSTE_ST2_MASK ) <<16 ; + res |= pgstes[1 ] &PGSTE_ST2_MASK ; + res |= (pgstes[2 ] &PGSTE_ST2_MASK ) >>16 ; + res |= (pgstes[3 ] &PGSTE_ST2_MASK ) >>32 ; + +return res; +} +#endif /* _S390_PAGE_H */
arch/s390/kvm/gaccess.c +4 -2
@@ -1409,6 +1409,7 @@ static int kvm_s390_shadow_tables(struct gmap *sg, unsigned long saddr,static int gmap_shadow_pgt_lookup (struct gmap *sg,unsigned long saddr,unsigned long *pgt,int *dat_protection,int *fake) { +unsigned long pt_index;unsigned long *table;struct page *page;int rc;
@@ -1418,9 +1419,10 @@ static int gmap_shadow_pgt_lookup(struct gmap *sg, unsigned long saddr, unsignedif (table && !(*table & _SEGMENT_ENTRY_INVALID)) {/* Shadow page tables are full pages (pte+pgste) */ page =pfn_to_page (*table >>PAGE_SHIFT ); - *pgt = page->index & ~GMAP_SHADOW_FAKE_TABLE ; + pt_index =gmap_pgste_get_index (page_to_virt (page)); + *pgt = pt_index & ~GMAP_SHADOW_FAKE_TABLE ; *dat_protection = !!(*table & _SEGMENT_ENTRY_PROTECT); - *fake = !!(page->index &GMAP_SHADOW_FAKE_TABLE ); + *fake = !!(pt_index &GMAP_SHADOW_FAKE_TABLE ); rc =0 ; }else { rc = -EAGAIN ;
arch/s390/mm/gmap.c +20 -2
@@ -1720,6 +1720,23 @@ int gmap_shadow_sgt(struct gmap *sg, unsigned long saddr, unsigned long sgt, }EXPORT_SYMBOL_GPL (gmap_shadow_sgt); +static void gmap_pgste_set_index (struct ptdesc *ptdesc,unsigned long pgt_addr) +{ +unsigned long *pgstes =page_to_virt (ptdesc_page (ptdesc)); + + pgstes += _PAGE_ENTRIES; + + pgstes[0 ] &= ~PGSTE_ST2_MASK ; + pgstes[1 ] &= ~PGSTE_ST2_MASK ; + pgstes[2 ] &= ~PGSTE_ST2_MASK ; + pgstes[3 ] &= ~PGSTE_ST2_MASK ; + + pgstes[0 ] |= (pgt_addr >>16 ) &PGSTE_ST2_MASK ; + pgstes[1 ] |= pgt_addr &PGSTE_ST2_MASK ; + pgstes[2 ] |= (pgt_addr <<16 ) &PGSTE_ST2_MASK ; + pgstes[3 ] |= (pgt_addr <<32 ) &PGSTE_ST2_MASK ; +} +/** * gmap_shadow_pgt - instantiate a shadow page table * @sg: pointer to the shadow guest address space structure
@@ -1747,9 +1764,10 @@ int gmap_shadow_pgt(struct gmap *sg, unsigned long saddr, unsigned long pgt, ptdesc =page_table_alloc_pgste (sg->mm);if (!ptdesc)return -ENOMEM ; - ptdesc->pt_index = pgt & _SEGMENT_ENTRY_ORIGIN; + origin = pgt & _SEGMENT_ENTRY_ORIGIN;if (fake) - ptdesc->pt_index |=GMAP_SHADOW_FAKE_TABLE ; + origin |=GMAP_SHADOW_FAKE_TABLE ; +gmap_pgste_set_index (ptdesc, origin); s_pgt =page_to_phys (ptdesc_page (ptdesc));/* Install shadow page table */ spin_lock (&sg->guest_table_lock);
Signature · 1 line
2.47.1
Re: [PATCH v2 15/15] KVM: s390: remove the last user of page->index
Quoted 9 lines · Claudio Imbrenda
Shadow page tables use
page->indexto keep the g2 address of the guest page table being shadowed.Instead of keeping the information in
page->index, split the address and smear it over the 16-bit softbits areas of 4 PGSTEs.This removes the last s390 user of
page->index.Signed-off-by Claudio Imbrenda <[email protected]>