KVM: s390: gmap rewrite, the real deal
Subsystems (7)
Claudio Imbrenda
maintainerv6 cover KVM: s390: gmap rewrite, the real deal
This series is the last big series of the gmap rewrite. It introduces the new code and actually uses it. The old code is then removed.
KVM on s390 will now use the mmu_notifier, like most other architectures. The gmap address space is now completely separate from userspace; no level of the page tables is shared between guest mapping and userspace.
One of the biggest advantages is that the page size of userspace is completely independent of the page size used by the guest. Userspace can mix normal pages, THPs, hugetlbfs, and more.
It's now possible to have nested guests and guests with huge pages running on the same host. In fact, it's possible to have a nested guest on a guest with huge pages. Transparent hugepages are also possible.
Patches 1 to 11 are mostly preparations; introducing some new bits and functions, and moving code around.
Patches 12 to 21 are the meat of the new gmap code; page table management functions and gmap management. This is the code that will be used to manage guest memory.
Patch 24 is unfortunately big; the existing code is converted to use the new gmap and all references to the old gmap are removed. This needs to be done all at once, unfortunately, hence the size of the patch.
Patch 25 and 26 remove all the now unused code.
Patch 27 and 28 allow for 1M pages to be used to back guests, and add some more functions that are useful for testing.
v5->v6:
* Added a mutex to avoid races during import-like operations in protected guests
* The functions uv_convert_from_secure_folio() and uv_destroy_folio() now work on the whole folio, and not just its first page
* Added warnings if uv_convert_from_secure_pte() fails in ptep_*()
* Changed the way shadow gmaps do reference counting
* Use a bitmap for gmap flags, instead of individual bool variables
* Rework the make_secure logic to work on gmap only, and not on userspace addresses
* Some refactorings
* Rebased onto 6.19-rc2
* Added more comments
* Minor style fixes
v4->v5:
* fix some locking issues
* fix comments and documentation
v3->v4:
* dat_link() can now return -ENOMEM when appropriate
* fixed a few vSIE races that led to use-after-free or deadlocks
* split part of the previous patch 23 and move it after patch 17, merge the rest of the patch into patch 19
* fix -ENOMEM handling in handle_pfmf() and handle_sske()
v2->v3:
* Add lots of small comments and cosmetic fixes
* Rename some functions to improve clarity
* Remove unused helper functions and macros
* Rename inline asm constraints labels to make them more understandable
* Refactor the code to pre-allocate the page tables (using custom caches) when sleeping is allowed, use the cached pages when holding spinlocks and handle gracefully allocation failures (i.e. retry instead of killing the guest)
* Refactor the code for fault handling; it's now in a separate file, and it takes a callback that can be optionally called when all the relevant locks are still held
* Use assembler mnemonics instead of manually specifying the opcode where appropriate
* Remove the LEVEL_* enum, and use TABLE_TYPE_* macros instead; introduce new TABLE_TYPE_PAGE_TABLE
* Remove usage of cpu_has_idte() since it is being removed from the kernel
* Improve storage key handling and PGSTE locking
* Introduce struct guest_fault to represent the state of a guest fault that is being resolved
* Minor CMMA fixes
Claudio Imbrenda (28):
KVM: s390: Refactor pgste lock and unlock functions
KVM: s390: add P bit in table entry bitfields, move union vaddress
s390: Make UV folio operations work on whole folio
s390: Move sske_frame() to a header
KVM: s390: Add gmap_helper_set_unused()
KVM: s390: Introduce import_lock
KVM: s390: Export two functions
s390/mm: Warn if uv_convert_from_secure_pte() fails
KVM: s390: vsie: Pass gmap explicitly as parameter
KVM: s390: Enable KVM_GENERIC_MMU_NOTIFIER
KVM: s390: Rename some functions in gaccess.c
KVM: s390: KVM-specific bitfields and helper functions
KVM: s390: KVM page table management functions: allocation
KVM: s390: KVM page table management functions: clear and replace
KVM: s390: KVM page table management functions: walks
KVM: s390: KVM page table management functions: storage keys
KVM: s390: KVM page table management functions: lifecycle management
KVM: s390: KVM page table management functions: CMMA
KVM: s390: New gmap code
KVM: s390: Add helper functions for fault handling
KVM: s390: Add some helper functions needed for vSIE
KVM: s390: Stop using CONFIG_PGSTE
KVM: s390: Storage key functions refactoring
KVM: s390: Switch to new gmap
KVM: s390: Remove gmap from s390/mm
KVM: S390: Remove PGSTE code from linux/s390 mm
KVM: s390: Enable 1M pages for gmap
KVM: s390: Storage key manipulation IOCTL
MAINTAINERS | 2 - arch/s390/Kconfig | 3 - arch/s390/include/asm/dat-bits.h | 32 +- arch/s390/include/asm/gmap.h | 174 -- arch/s390/include/asm/gmap_helpers.h | 1 + arch/s390/include/asm/hugetlb.h | 6 - arch/s390/include/asm/kvm_host.h | 7 + arch/s390/include/asm/mmu.h | 13 - arch/s390/include/asm/mmu_context.h | 6 +- arch/s390/include/asm/page.h | 4 - arch/s390/include/asm/pgalloc.h | 4 - arch/s390/include/asm/pgtable.h | 171 +- arch/s390/include/asm/tlb.h | 3 - arch/s390/include/asm/uaccess.h | 70 +- arch/s390/include/asm/uv.h | 3 +- arch/s390/kernel/uv.c | 142 +- arch/s390/kvm/Kconfig | 2 + arch/s390/kvm/Makefile | 3 +- arch/s390/kvm/dat.c | 1368 +++++++++++++++ arch/s390/kvm/dat.h | 965 ++++++++++ arch/s390/kvm/diag.c | 2 +- arch/s390/kvm/faultin.c | 148 ++ arch/s390/kvm/faultin.h | 92 + arch/s390/kvm/gaccess.c | 940 +++++----- arch/s390/kvm/gaccess.h | 20 +- arch/s390/kvm/gmap-vsie.c | 141 -- arch/s390/kvm/gmap.c | 1151 ++++++++++++ arch/s390/kvm/gmap.h | 240 +++ arch/s390/kvm/intercept.c | 15 +- arch/s390/kvm/interrupt.c | 6 +- arch/s390/kvm/kvm-s390.c | 939 ++++------ arch/s390/kvm/kvm-s390.h | 27 +- arch/s390/kvm/priv.c | 213 +-- arch/s390/kvm/pv.c | 177 +- arch/s390/kvm/vsie.c | 193 +- arch/s390/lib/uaccess.c | 184 +- arch/s390/mm/Makefile | 1 - arch/s390/mm/fault.c | 4 +- arch/s390/mm/gmap.c | 2436 -------------------------- arch/s390/mm/gmap_helpers.c | 96 +- arch/s390/mm/hugetlbpage.c | 24 - arch/s390/mm/page-states.c | 1 + arch/s390/mm/pageattr.c | 7 - arch/s390/mm/pgalloc.c | 24 - arch/s390/mm/pgtable.c | 814 +-------- include/uapi/linux/kvm.h | 10 + mm/khugepaged.c | 9 - 47 files changed, 5415 insertions(+), 5478 deletions(-) delete mode 100644 arch/s390/include/asm/gmap.h create mode 100644 arch/s390/kvm/dat.c create mode 100644 arch/s390/kvm/dat.h create mode 100644 arch/s390/kvm/faultin.c create mode 100644 arch/s390/kvm/faultin.h delete mode 100644 arch/s390/kvm/gmap-vsie.c create mode 100644 arch/s390/kvm/gmap.c create mode 100644 arch/s390/kvm/gmap.h delete mode 100644 arch/s390/mm/gmap.c
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 1/28 KVM: s390: Refactor pgste lock and unlock functions
Move the pgste lock and unlock functions back into mm/pgtable.c and duplicate them in mm/gmap_helpers.c to avoid function name collisions later on.
--- arch/s390/include/asm/pgtable.h | 22 ---------------------- arch/s390/mm/gmap_helpers.c | 23 ++++++++++++++++++++++- arch/s390/mm/pgtable.c | 23 ++++++++++++++++++++++- 3 files changed, 44 insertions(+), 24 deletions(-)
arch/s390/include/asm/pgtable.h +0 -22
@@ -2040,26 +2040,4 @@ static inline unsigned long gmap_pgste_get_pgt_addr(unsigned long *pgt) return res; } -static inline pgste_t pgste_get_lock (pte_t *ptep) -{ -unsigned long value =0 ; -#ifdef CONFIG_PGSTE -unsigned long *ptr = (unsigned long *)(ptep +PTRS_PER_PTE ); - -do { - value =__atomic64_or_barrier (PGSTE_PCL_BIT , ptr); - }while (value &PGSTE_PCL_BIT ); - value |=PGSTE_PCL_BIT ; -#endif -return __pgste (value); -} - -static inline void pgste_set_unlock (pte_t *ptep,pgste_t pgste) -{ -#ifdef CONFIG_PGSTE -barrier (); -WRITE_ONCE (*(unsigned long *)(ptep +PTRS_PER_PTE ),pgste_val (pgste) & ~PGSTE_PCL_BIT ); -#endif -} -#endif /* _S390_PAGE_H */
arch/s390/mm/gmap_helpers.c +22 -1
@@ -15,7 +15,6 @@#include <linux/pagewalk.h> #include <linux/ksm.h> #include <asm/gmap_helpers.h> -#include <asm/pgtable.h> /** * ptep_zap_softleaf_entry() - discard a software leaf entry.
@@ -35,6 +34,28 @@ static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry)free_swap_and_cache (entry); } +static inline pgste_t pgste_get_lock (pte_t *ptep) +{ +unsigned long value =0 ; +#ifdef CONFIG_PGSTE +unsigned long *ptr = (unsigned long *)(ptep +PTRS_PER_PTE ); + +do { + value =__atomic64_or_barrier (PGSTE_PCL_BIT , ptr); + }while (value &PGSTE_PCL_BIT ); + value |=PGSTE_PCL_BIT ; +#endif +return __pgste (value); +} + +static inline void pgste_set_unlock (pte_t *ptep,pgste_t pgste) +{ +#ifdef CONFIG_PGSTE +barrier (); +WRITE_ONCE (*(unsigned long *)(ptep +PTRS_PER_PTE ),pgste_val (pgste) & ~PGSTE_PCL_BIT ); +#endif +} +/** * gmap_helper_zap_one_page() - discard a page if it was swapped. * @mm: the mm
arch/s390/mm/pgtable.c +22 -1
@@ -24,7 +24,6 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include <asm/page-states.h> -#include <asm/pgtable.h> #include <asm/machine.h> pgprot_t pgprot_writecombine (pgprot_t prot)
@@ -116,6 +115,28 @@ static inline pte_t ptep_flush_lazy(struct mm_struct *mm,return old; } +static inline pgste_t pgste_get_lock (pte_t *ptep) +{ +unsigned long value =0 ; +#ifdef CONFIG_PGSTE +unsigned long *ptr = (unsigned long *)(ptep +PTRS_PER_PTE ); + +do { + value =__atomic64_or_barrier (PGSTE_PCL_BIT , ptr); + }while (value &PGSTE_PCL_BIT ); + value |=PGSTE_PCL_BIT ; +#endif +return __pgste (value); +} + +static inline void pgste_set_unlock (pte_t *ptep,pgste_t pgste) +{ +#ifdef CONFIG_PGSTE +barrier (); +WRITE_ONCE (*(unsigned long *)(ptep +PTRS_PER_PTE ),pgste_val (pgste) & ~PGSTE_PCL_BIT ); +#endif +} +static inline pgste_t pgste_get (pte_t *ptep) {unsigned long pgste =0 ;
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 2/28 KVM: s390: add P bit in table entry bitfields, move union vaddress
In reply to Claudio ImbrendaAdd P bit in hardware definition of region 3 and segment table entries.
Move union vaddress from kvm/gaccess.c to asm/dat_bits.h
--- arch/s390/include/asm/dat-bits.h | 32 ++++++++++++++++++++++++++++++-- arch/s390/kvm/gaccess.c | 26 -------------------------- 2 files changed, 30 insertions(+), 28 deletions(-)
arch/s390/include/asm/dat-bits.h +30 -2
@@ -9,6 +9,32 @@ #ifndef _S390_DAT_BITS_H #define _S390_DAT_BITS_H +/* + * vaddress union in order to easily decode a virtual address into its + * region first index, region second index etc. parts. + */ +union vaddress { + unsigned long addr; + struct { + unsigned long rfx : 11; + unsigned long rsx : 11; + unsigned long rtx : 11; + unsigned long sx : 11; + unsigned long px : 8; + unsigned long bx : 12; + }; + struct { + unsigned long rfx01 : 2; + unsigned long : 9; + unsigned long rsx01 : 2; + unsigned long : 9; + unsigned long rtx01 : 2; + unsigned long : 9; + unsigned long sx01 : 2; + unsigned long : 29; + }; +}; + union asce { unsigned long val; struct {
@@ -98,7 +124,8 @@ union region3_table_entry {struct { unsigned long : 53;unsigned long fc:1 ;/* Format-Control */ - unsigned long : 4; +unsigned long p :1 ;/* DAT-Protection Bit */ + unsigned long : 3;unsigned long i :1 ;/* Region-Invalid Bit */ unsigned long cr:1 ;/* Common-Region Bit */ unsigned long tt:2 ;/* Table-Type Bits */
@@ -140,7 +167,8 @@ union segment_table_entry {struct { unsigned long : 53;unsigned long fc:1 ;/* Format-Control */ - unsigned long : 4; +unsigned long p :1 ;/* DAT-Protection Bit */ + unsigned long : 3;unsigned long i :1 ;/* Segment-Invalid Bit */ unsigned long cs:1 ;/* Common-Segment Bit */ unsigned long tt:2 ;/* Table-Type Bits */
arch/s390/kvm/gaccess.c +0 -26
@@ -20,32 +20,6 @@#define GMAP_SHADOW_FAKE_TABLE 1ULL -/* -* vaddress union in order to easily decode a virtual address into its -* region first index, region second index etc. parts. -*/ -union vaddress { -unsigned long addr; -struct { -unsigned long rfx :11 ; -unsigned long rsx :11 ; -unsigned long rtx :11 ; -unsigned long sx :11 ; -unsigned long px :8 ; -unsigned long bx :12 ; - }; -struct { -unsigned long rfx01 :2 ; - unsigned long : 9; -unsigned long rsx01 :2 ; - unsigned long : 9; -unsigned long rtx01 :2 ; - unsigned long : 9; -unsigned long sx01 :2 ; - unsigned long : 29; - }; -}; -/* * raddress union which will contain the result (real or absolute address) * after a page table walk. The rfaa, sfaa and pfra members are used to
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 3/28 s390: Make UV folio operations work on whole folio
In reply to Claudio Imbrendauv_destroy_folio() and uv_convert_from_secure_folio() should work on all pages in the folio, not just the first one.
This was fine until now, but it will become a problem with upcoming patches.
--- arch/s390/kernel/uv.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-)
arch/s390/kernel/uv.c +12 -10
@@ -134,14 +134,15 @@ static int uv_destroy(unsigned long paddr)*/ int uv_destroy_folio (struct folio *folio) { +unsigned long i;int rc; -/* Large folios cannot be secure */ -if (unlikely (folio_test_large (folio))) -return 0 ; -folio_get (folio); - rc =uv_destroy (folio_to_phys (folio)); +for (i =0 ; i < (1 <<folio_order (folio)); i++) { + rc =uv_destroy (folio_to_phys (folio) + i *PAGE_SIZE ); +if (rc) +break ; + }if (!rc)clear_bit (PG_arch_1, &folio->flags.f);folio_put (folio);
@@ -183,14 +184,15 @@ EXPORT_SYMBOL_GPL(uv_convert_from_secure);*/ int uv_convert_from_secure_folio (struct folio *folio) { +unsigned long i;int rc; -/* Large folios cannot be secure */ -if (unlikely (folio_test_large (folio))) -return 0 ; -folio_get (folio); - rc =uv_convert_from_secure (folio_to_phys (folio)); +for (i =0 ; i < (1 <<folio_order (folio)); i++) { + rc =uv_convert_from_secure (folio_to_phys (folio) + i *PAGE_SIZE ); +if (rc) +break ; + }if (!rc)clear_bit (PG_arch_1, &folio->flags.f);folio_put (folio);
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 4/28 s390: Move sske_frame() to a header
In reply to Claudio ImbrendaMove the sske_frame() function to asm/pgtable.h, so it can be used in other modules too.
Opportunistically convert the .insn opcode specification to the appropriate mnemonic.
--- arch/s390/include/asm/pgtable.h | 7 +++++++ arch/s390/mm/pageattr.c | 7 ------- 2 files changed, 7 insertions(+), 7 deletions(-)
arch/s390/include/asm/pgtable.h +7 -0
@@ -1136,6 +1136,13 @@ static inline pte_t pte_mkhuge(pte_t pte) }#endif +static inline unsigned long sske_frame (unsigned long addr,unsigned char skey) +{ + asmvolatile ("sske %[skey],%[addr],1" + : [addr]"+a" (addr) : [skey]"d" (skey)); +return addr; +} +#define IPTE_GLOBAL 0#define IPTE_LOCAL 1
arch/s390/mm/pageattr.c +0 -7
@@ -16,13 +16,6 @@#include <asm/asm.h> #include <asm/set_memory.h> -static inline unsigned long sske_frame (unsigned long addr,unsigned char skey) -{ - asmvolatile (".insn rrf,0xb22b0000,%[skey],%[addr],1,0" - : [addr]"+a" (addr) : [skey]"d" (skey)); -return addr; -} -void __storage_key_init_range (unsigned long start,unsigned long end) {unsigned long boundary, size;
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 5/28 KVM: s390: Add gmap_helper_set_unused()
In reply to Claudio ImbrendaAdd gmap_helper_set_unused() to mark userspace ptes as unused.
Core mm code will use that information to discard unused pages instead of attempting to swap them.
--- arch/s390/include/asm/gmap_helpers.h | 1 + arch/s390/mm/gmap_helpers.c | 79 ++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+)
arch/s390/include/asm/gmap_helpers.h +1 -0
@@ -11,5 +11,6 @@void gmap_helper_zap_one_page (struct mm_struct *mm,unsigned long vmaddr);void gmap_helper_discard (struct mm_struct *mm,unsigned long vmaddr,unsigned long end);int gmap_helper_disable_cow_sharing (void ); +void gmap_helper_try_set_pte_unused (struct mm_struct *mm,unsigned long vmaddr);#endif /* _ASM_S390_GMAP_HELPERS_H */
arch/s390/mm/gmap_helpers.c +79 -0
85 lines not on this page · raw message
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 6/28 KVM: s390: Introduce import_lock
In reply to Claudio ImbrendaIntroduce import_lock to avoid future races when converting pages to secure.
--- arch/s390/include/asm/kvm_host.h | 2 ++ arch/s390/kvm/kvm-s390.c | 2 ++ 2 files changed, 4 insertions(+)
arch/s390/include/asm/kvm_host.h +2 -0
@@ -630,6 +630,8 @@ struct kvm_s390_pv {void *set_aside;struct list_head need_cleanup;struct mmu_notifier mmu_notifier; +/* Protects against concurrent import-like operations */ +struct mutex import_lock; };struct kvm_arch {
arch/s390/kvm/kvm-s390.c +2 -0
@@ -3330,6 +3330,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)char debug_name[16 ];int i, rc; +mutex_init (&kvm->arch.pv.import_lock); + rc = -EINVAL ;#ifdef CONFIG_KVM_S390_UCONTROL if (type & ~KVM_VM_S390_UCONTROL )
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 7/28 KVM: s390: Export two functions
In reply to Claudio ImbrendaExport __make_folio_secure() and s390_wiggle_split_folio(), as they will be needed to be used by KVM.
--- arch/s390/include/asm/uv.h | 2 ++ arch/s390/kernel/uv.c | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-)
arch/s390/include/asm/uv.h +2 -0
@@ -632,6 +632,8 @@ int uv_destroy_folio(struct folio *folio);int uv_destroy_pte (pte_t pte);int uv_convert_from_secure_pte (pte_t pte);int make_hva_secure (struct mm_struct *mm,unsigned long hva,struct uv_cb_header *uvcb); +int s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio); +int __make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb);int uv_convert_from_secure (unsigned long paddr);int uv_convert_from_secure_folio (struct folio *folio);
arch/s390/kernel/uv.c +4 -2
@@ -281,7 +281,7 @@ static int expected_folio_refs(struct folio *folio)* (it's the same logic as split_folio()), and the folio must be * locked. */ -static int __make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb) +int __make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb) {int expected, cc =0 ;
@@ -311,6 +311,7 @@ static int __make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb)return -EAGAIN ;return uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; } +EXPORT_SYMBOL (__make_folio_secure);static int make_folio_secure (struct mm_struct *mm,struct folio *folio,struct uv_cb_header *uvcb) {
@@ -339,7 +340,7 @@ static int make_folio_secure(struct mm_struct *mm, struct folio *folio, struct u* but another attempt can be made; * -EINVAL in case of other folio splitting errors. See split_folio(). */ -static int s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio) +int s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio) {int rc, tried_splits;
@@ -411,6 +412,7 @@ static int s390_wiggle_split_folio(struct mm_struct *mm, struct folio *folio) }return -EAGAIN ; } +EXPORT_SYMBOL_GPL (s390_wiggle_split_folio);int make_hva_secure (struct mm_struct *mm,unsigned long hva,struct uv_cb_header *uvcb) {
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 8/28 s390/mm: Warn if uv_convert_from_secure_pte() fails
In reply to Claudio ImbrendaIf uv_convert_from_secure_pte() fails, the page becomes unusable by the host. The failure can only occour in case of hardware malfunction or a serious KVM bug.
When the unusable page is reused, the system can have issues and hang.
Print a warning to aid debugging such unlikely scenarios.
--- arch/s390/include/asm/pgtable.h | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
arch/s390/include/asm/pgtable.h +5 -4
@@ -1239,7 +1239,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm, res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID));/* At this point the reference through the mapping is still present */ if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_from_secure_pte (res); +WARN_ON_ONCE (uv_convert_from_secure_pte (res));return res; }
@@ -1257,7 +1257,7 @@ static inline pte_t ptep_clear_flush(struct vm_area_struct *vma, res =ptep_xchg_direct (vma->vm_mm, addr, ptep,__pte (_PAGE_INVALID));/* At this point the reference through the mapping is still present */ if (mm_is_protected (vma->vm_mm) &&pte_present (res)) -uv_convert_from_secure_pte (res); +WARN_ON_ONCE (uv_convert_from_secure_pte (res));return res; }
@@ -1294,9 +1294,10 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,/* * If something went wrong and the page could not be destroyed, or * if this is not a mm teardown, the slower export is used as -* fallback instead. +* fallback instead. If even that fails, print a warning and leak +* the page, to avoid crashing the whole system. */ -uv_convert_from_secure_pte (res); +WARN_ON_ONCE (uv_convert_from_secure_pte (res));return res; }
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 9/28 KVM: s390: vsie: Pass gmap explicitly as parameter
In reply to Claudio ImbrendaPass the gmap explicitly as parameter, instead of just using vsie_page->gmap.
This will be used in upcoming patches.
--- arch/s390/kvm/vsie.c | 40 +++++++++++++++++++--------------------- 1 file changed, 19 insertions(+), 21 deletions(-)
arch/s390/kvm/vsie.c +19 -21
@@ -652,7 +652,7 @@ void kvm_s390_vsie_gmap_notifier(struct gmap *gmap, unsigned long start,* - -EAGAIN if the caller can retry immediately * - -ENOMEM if out of memory */ -static int map_prefix (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page) +static int map_prefix (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page,struct gmap *sg) {struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s;u64 prefix = scb_s->prefix <<GUEST_PREFIX_SHIFT ;
@@ -667,10 +667,9 @@ static int map_prefix(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)/* with mso/msl, the prefix lies at offset *mso* */ prefix += scb_s->mso; - rc =kvm_s390_shadow_fault (vcpu, vsie_page->gmap, prefix,NULL ); + rc =kvm_s390_shadow_fault (vcpu, sg, prefix,NULL );if (!rc && (scb_s->ecb &ECB_TE )) - rc =kvm_s390_shadow_fault (vcpu, vsie_page->gmap, - prefix +PAGE_SIZE ,NULL ); + rc =kvm_s390_shadow_fault (vcpu, sg, prefix +PAGE_SIZE ,NULL );/* * We don't have to mprotect, we will be called for all unshadows. * SIE will detect if protection applies and trigger a validity.
@@ -951,7 +950,7 @@ static int inject_fault(struct kvm_vcpu *vcpu, __u16 code, __u64 vaddr,* - > 0 if control has to be given to guest 2 * - < 0 if an error occurred */ -static int handle_fault (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page) +static int handle_fault (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page,struct gmap *sg) {int rc;
@@ -960,8 +959,7 @@ static int handle_fault(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)return inject_fault (vcpu,PGM_PROTECTION , current->thread.gmap_teid.addr *PAGE_SIZE ,1 ); - rc =kvm_s390_shadow_fault (vcpu, vsie_page->gmap, - current->thread.gmap_teid.addr *PAGE_SIZE ,NULL ); + rc =kvm_s390_shadow_fault (vcpu, sg, current->thread.gmap_teid.addr *PAGE_SIZE ,NULL );if (rc >0 ) { rc = inject_fault(vcpu, rc, current->thread.gmap_teid.addr * PAGE_SIZE,
@@ -978,12 +976,10 @@ static int handle_fault(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)* * Will ignore any errors. The next SIE fault will do proper fault handling. */ -static void handle_last_fault (struct kvm_vcpu *vcpu, -struct vsie_page *vsie_page) +static void handle_last_fault (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page,struct gmap *sg) {if (vsie_page->fault_addr) -kvm_s390_shadow_fault (vcpu, vsie_page->gmap, - vsie_page->fault_addr,NULL ); +kvm_s390_shadow_fault (vcpu, sg, vsie_page->fault_addr,NULL ); vsie_page->fault_addr =0 ; }
@@ -1065,7 +1061,7 @@ static u64 vsie_get_register(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page, } } -static int vsie_handle_mvpg (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page) +static int vsie_handle_mvpg (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page,struct gmap *sg) {struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s;unsigned long pei_dest, pei_src, src, dest, mask, prefix;
@@ -1083,8 +1079,8 @@ static int vsie_handle_mvpg(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) src =vsie_get_register (vcpu, vsie_page, scb_s->ipb >>16 ) & mask; src =_kvm_s390_real_to_abs (prefix, src) + scb_s->mso; - rc_dest =kvm_s390_shadow_fault (vcpu, vsie_page->gmap, dest, &pei_dest); - rc_src =kvm_s390_shadow_fault (vcpu, vsie_page->gmap, src, &pei_src); + rc_dest =kvm_s390_shadow_fault (vcpu, sg, dest, &pei_dest); + rc_src =kvm_s390_shadow_fault (vcpu, sg, src, &pei_src);/* * Either everything went well, or something non-critical went wrong * e.g. because of a race. In either case, simply retry.
@@ -1144,7 +1140,7 @@ static int vsie_handle_mvpg(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)* - > 0 if control has to be given to guest 2 * - < 0 if an error occurred */ -static int do_vsie_run (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page) +static int do_vsie_run (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page,struct gmap *sg) __releases(vcpu->kvm->srcu) __acquires(vcpu->kvm->srcu) {
@@ -1153,7 +1149,7 @@ static int do_vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)int guest_bp_isolation;int rc =0 ; -handle_last_fault (vcpu, vsie_page); +handle_last_fault (vcpu, vsie_page, sg);kvm_vcpu_srcu_read_unlock (vcpu);
@@ -1191,7 +1187,7 @@ static int do_vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) goto xfer_to_guest_mode_check; }guest_timing_enter_irqoff (); - rc =kvm_s390_enter_exit_sie (scb_s, vcpu->run->s.regs.gprs, vsie_page->gmap->asce); + rc =kvm_s390_enter_exit_sie (scb_s, vcpu->run->s.regs.gprs, sg->asce);guest_timing_exit_irqoff ();local_irq_enable (); }
@@ -1215,7 +1211,7 @@ static int do_vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)if (rc >0 ) rc =0 ;/* we could still have an icpt */ else if (current->thread.gmap_int_code) -return handle_fault (vcpu, vsie_page); +return handle_fault (vcpu, vsie_page, sg);switch (scb_s->icptcode) {case ICPT_INST :
@@ -1233,7 +1229,7 @@ static int do_vsie_run(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)break ;case ICPT_PARTEXEC :if (scb_s->ipa ==0xb254 ) - rc =vsie_handle_mvpg (vcpu, vsie_page); + rc =vsie_handle_mvpg (vcpu, vsie_page, sg);break ; }return rc;
@@ -1330,15 +1326,17 @@ static void unregister_shadow_scb(struct kvm_vcpu *vcpu)static int vsie_run (struct kvm_vcpu *vcpu,struct vsie_page *vsie_page) {struct kvm_s390_sie_block *scb_s = &vsie_page->scb_s; +struct gmap *sg;int rc =0 ;while (1 ) { rc =acquire_gmap_shadow (vcpu, vsie_page); + sg = vsie_page->gmap;if (!rc) - rc =map_prefix (vcpu, vsie_page); + rc =map_prefix (vcpu, vsie_page, sg);if (!rc) {update_intervention_requests (vsie_page); - rc =do_vsie_run (vcpu, vsie_page); + rc =do_vsie_run (vcpu, vsie_page, sg); }atomic_andnot (PROG_BLOCK_SIE , &scb_s->prog20);
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 10/28 KVM: s390: Enable KVM_GENERIC_MMU_NOTIFIER
In reply to Claudio ImbrendaEnable KVM_GENERIC_MMU_NOTIFIER, for now with empty placeholder callbacks.
Also enable KVM_MMU_LOCKLESS_AGING and define KVM_HAVE_MMU_RWLOCK.
--- arch/s390/include/asm/kvm_host.h | 1 + arch/s390/kvm/Kconfig | 2 ++ arch/s390/kvm/kvm-s390.c | 45 +++++++++++++++++++++++++++++++- 3 files changed, 47 insertions(+), 1 deletion(-)
arch/s390/include/asm/kvm_host.h +1 -0
@@ -27,6 +27,7 @@#include <asm/isc.h> #include <asm/guarded_storage.h> +#define KVM_HAVE_MMU_RWLOCK #define KVM_MAX_VCPUS 255#define KVM_INTERNAL_MEM_SLOTS 1
arch/s390/kvm/Kconfig +2 -0
@@ -30,6 +30,8 @@ config KVM select KVM_VFIO select MMU_NOTIFIER select VIRT_XFER_TO_GUEST_WORK + select KVM_GENERIC_MMU_NOTIFIER + select KVM_MMU_LOCKLESS_AGING help Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should work
arch/s390/kvm/kvm-s390.c +44 -1
57 lines not on this page · raw message
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 11/28 KVM: s390: Rename some functions in gaccess.c
In reply to Claudio ImbrendaRename some functions in gaccess.c to add a _gva or _gpa suffix to indicate whether the function accepts a virtual or a guest-absolute address.
This makes it easier to understand the code.
--- arch/s390/kvm/gaccess.c | 51 +++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 27 deletions(-)
arch/s390/kvm/gaccess.c +24 -27
@@ -397,7 +397,7 @@ static int deref_table(struct kvm *kvm, unsigned long gpa, unsigned long *val) }/** -* guest_translate - translate a guest virtual into a guest absolute address +* guest_translate_gva() - translate a guest virtual into a guest absolute address * @vcpu: virtual cpu * @gva: guest virtual address * @gpa: points to where guest physical (absolute) address should be stored
@@ -417,9 +417,9 @@ static int deref_table(struct kvm *kvm, unsigned long gpa, unsigned long *val)* the returned value is the program interruption code as defined * by the architecture */ -static unsigned long guest_translate (struct kvm_vcpu *vcpu,unsigned long gva, -unsigned long *gpa,const union asce asce, -enum gacc_mode mode,enum prot_type *prot) +static unsigned long guest_translate_gva (struct kvm_vcpu *vcpu,unsigned long gva, +unsigned long *gpa,const union asce asce, +enum gacc_mode mode,enum prot_type *prot) {union vaddress vaddr = {.addr = gva};union raddress raddr = {.addr = gva};
@@ -600,8 +600,8 @@ static int low_address_protection_enabled(struct kvm_vcpu *vcpu,return 1 ; } -static int vm_check_access_key (struct kvm *kvm,u8 access_key, -enum gacc_mode mode,gpa_t gpa) +static int vm_check_access_key_gpa (struct kvm *kvm,u8 access_key, +enum gacc_mode mode,gpa_t gpa) {u8 storage_key, access_control;bool fetch_protected;
@@ -663,9 +663,9 @@ static bool storage_prot_override_applies(u8 access_control)return access_control ==PAGE_SPO_ACC ; } -static int vcpu_check_access_key (struct kvm_vcpu *vcpu,u8 access_key, -enum gacc_mode mode,union asce asce,gpa_t gpa, -unsigned long ga,unsigned int len) +static int vcpu_check_access_key_gpa (struct kvm_vcpu *vcpu,u8 access_key, +enum gacc_mode mode,union asce asce,gpa_t gpa, +unsigned long ga,unsigned int len) {u8 storage_key, access_control;unsigned long hva;
@@ -757,7 +757,7 @@ static int guest_range_to_gpas(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar,return trans_exc (vcpu,PGM_PROTECTION , ga, ar, mode,PROT_TYPE_LA );if (psw_bits (*psw).dat) { - rc =guest_translate (vcpu, ga, &gpa, asce, mode, &prot); + rc =guest_translate_gva (vcpu, ga, &gpa, asce, mode, &prot);if (rc <0 )return rc; }else {
@@ -769,8 +769,7 @@ static int guest_range_to_gpas(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, }if (rc)return trans_exc (vcpu, rc, ga, ar, mode, prot); - rc =vcpu_check_access_key (vcpu, access_key, mode, asce, gpa, ga, - fragment_len); + rc =vcpu_check_access_key_gpa (vcpu, access_key, mode, asce, gpa, ga, fragment_len);if (rc)return trans_exc (vcpu, rc, ga, ar, mode,PROT_TYPE_KEYC );if (gpas)
@@ -782,8 +781,8 @@ static int guest_range_to_gpas(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar,return 0 ; } -static int access_guest_page (struct kvm *kvm,enum gacc_mode mode,gpa_t gpa, -void *data,unsigned int len) +static int access_guest_page_gpa (struct kvm *kvm,enum gacc_mode mode,gpa_t gpa, +void *data,unsigned int len) {const unsigned int offset =offset_in_page (gpa);const gfn_t gfn =gpa_to_gfn (gpa);
@@ -798,9 +797,8 @@ static int access_guest_page(struct kvm *kvm, enum gacc_mode mode, gpa_t gpa,return rc; } -static int -access_guest_page_with_key (struct kvm *kvm,enum gacc_mode mode,gpa_t gpa, -void *data,unsigned int len,u8 access_key) +static int access_guest_page_with_key_gpa (struct kvm *kvm,enum gacc_mode mode,gpa_t gpa, +void *data,unsigned int len,u8 access_key) {struct kvm_memory_slot *slot;bool writable;
@@ -808,7 +806,7 @@ access_guest_page_with_key(struct kvm *kvm, enum gacc_mode mode, gpa_t gpa,hva_t hva;int rc; - gfn = gpa >>PAGE_SHIFT ; + gfn =gpa_to_gfn (gpa); slot =gfn_to_memslot (kvm, gfn); hva =gfn_to_hva_memslot_prot (slot, gfn, &writable);
@@ -841,7 +839,7 @@ int access_guest_abs_with_key(struct kvm *kvm, gpa_t gpa, void *data,while (min (PAGE_SIZE - offset, len) >0 ) { fragment_len =min (PAGE_SIZE - offset, len); - rc =access_guest_page_with_key (kvm, mode, gpa, data, fragment_len, access_key); + rc =access_guest_page_with_key_gpa (kvm, mode, gpa, data, fragment_len, access_key);if (rc)return rc; offset =0 ;
@@ -901,15 +899,14 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar,for (idx =0 ; idx < nr_pages; idx++) { fragment_len =min (PAGE_SIZE -offset_in_page (gpas[idx]), len);if (try_fetch_prot_override &&fetch_prot_override_applies (ga, fragment_len)) { - rc =access_guest_page (vcpu->kvm, mode, gpas[idx], - data, fragment_len); + rc =access_guest_page_gpa (vcpu->kvm, mode, gpas[idx], data, fragment_len); }else { - rc =access_guest_page_with_key (vcpu->kvm, mode, gpas[idx], - data, fragment_len, access_key); + rc =access_guest_page_with_key_gpa (vcpu->kvm, mode, gpas[idx], + data, fragment_len, access_key); }if (rc ==PGM_PROTECTION && try_storage_prot_override) - rc =access_guest_page_with_key (vcpu->kvm, mode, gpas[idx], - data, fragment_len,PAGE_SPO_ACC ); + rc =access_guest_page_with_key_gpa (vcpu->kvm, mode, gpas[idx], + data, fragment_len,PAGE_SPO_ACC );if (rc)break ; len -= fragment_len;
@@ -943,7 +940,7 @@ int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,while (len && !rc) { gpa =kvm_s390_real_to_abs (vcpu, gra); fragment_len =min (PAGE_SIZE -offset_in_page (gpa), len); - rc =access_guest_page (vcpu->kvm, mode, gpa, data, fragment_len); + rc =access_guest_page_gpa (vcpu->kvm, mode, gpa, data, fragment_len); len -= fragment_len; gra += fragment_len; data += fragment_len;
@@ -1134,7 +1131,7 @@ int check_gpa_range(struct kvm *kvm, unsigned long gpa, unsigned long length,while (length && !rc) { fragment_len =min (PAGE_SIZE -offset_in_page (gpa), length); - rc =vm_check_access_key (kvm, access_key, mode, gpa); + rc =vm_check_access_key_gpa (kvm, access_key, mode, gpa); length -= fragment_len; gpa += fragment_len; }
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 12/28 KVM: s390: KVM-specific bitfields and helper functions
In reply to Claudio ImbrendaAdd KVM-s390 specific bitfields and helper functions to manipulate DAT tables.
--- arch/s390/kvm/dat.h | 720 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 720 insertions(+) create mode 100644 arch/s390/kvm/dat.h
arch/s390/kvm/dat.h new file +720 -0
@@ -0,0 +1,720 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* KVM guest address space mapping code +* +* Copyright IBM Corp. 2024, 2025 +* Author(s): Claudio Imbrenda <[email protected]> +*/ + +#ifndef __KVM_S390_DAT_H +#define __KVM_S390_DAT_H + +#include <linux/radix-tree.h> +#include <linux/refcount.h> +#include <linux/io.h> +#include <linux/kvm_types.h> +#include <linux/pgalloc.h> +#include <asm/tlbflush.h> +#include <asm/dat-bits.h> + +#define _ASCE (x) ((union asce) { .val = (x), }) +#define NULL_ASCE _ASCE(0) + +enum { + _DAT_TOKEN_NONE =0 , + _DAT_TOKEN_PIC, +}; + +#define _CRSTE_TOK (l, t, p) ((union crste) { \ + .tok.i = 1, \ + .tok.tt = (l), \ + .tok.type = (t), \ + .tok.par = (p) \ + }) +#define _CRSTE_PIC (l, p) _CRSTE_TOK(l, _DAT_TOKEN_PIC, p) + +#define _CRSTE_HOLE (l) _CRSTE_PIC(l, PGM_ADDRESSING) +#define _CRSTE_EMPTY (l) _CRSTE_TOK(l, _DAT_TOKEN_NONE, 0) + +#define _PMD_EMPTY _CRSTE_EMPTY(TABLE_TYPE_SEGMENT) + +#define _PTE_TOK (t, p) ((union pte) { .tok.i = 1, .tok.type = (t), .tok.par = (p) }) +#define _PTE_EMPTY _PTE_TOK(_DAT_TOKEN_NONE, 0) + +/* This fake table type is used for page table walks (both for normal page tables and vSIE) */ +#define TABLE_TYPE_PAGE_TABLE -1 + +enum dat_walk_flags { +DAT_WALK_CONTINUE =0x20 , +DAT_WALK_IGN_HOLES =0x10 , +DAT_WALK_SPLIT =0x08 , +DAT_WALK_ALLOC =0x04 , +DAT_WALK_ANY =0x02 , +DAT_WALK_LEAF =0x01 , +DAT_WALK_DEFAULT =0 +}; + +#define DAT_WALK_SPLIT_ALLOC (DAT_WALK_SPLIT | DAT_WALK_ALLOC) +#define DAT_WALK_ALLOC_CONTINUE (DAT_WALK_CONTINUE | DAT_WALK_ALLOC) +#define DAT_WALK_LEAF_ALLOC (DAT_WALK_LEAF | DAT_WALK_ALLOC) + +union pte { +unsigned long val; +union page_table_entry h; +struct { + unsigned long :56; /* Hardware bits */ +unsigned long u :1 ;/* Page unused */ +unsigned long s :1 ;/* Special */ +unsigned long w :1 ;/* Writable */ +unsigned long r :1 ;/* Readable */ +unsigned long d :1 ;/* Dirty */ +unsigned long y :1 ;/* Young */ +unsigned long sd:1 ;/* Soft dirty */ +unsigned long pr:1 ;/* Present */ + } s; +struct { +unsigned char hwbytes[7 ]; +unsigned char swbyte; + }; +union { +struct { +unsigned long type :16 ;/* Token type */ +unsigned long par :16 ;/* Token parameter */ + unsigned long :20; + unsigned long : 1; /* Must be 0 */ +unsigned long i :1 ;/* Must be 1 */ + unsigned long : 2; + unsigned long : 7; +unsigned long pr :1 ;/* Must be 0 */ + }; +struct { +unsigned long token:32 ;/* Token and parameter */ + unsigned long :32; + }; + } tok; +}; + +/* Soft dirty, needed as macro for atomic operations on ptes */ +#define _PAGE_SD 0x002 + +/* Needed as macro to perform atomic operations */ +#define PGSTE_CMMA_D_BIT 0x0000000000008000UL/* CMMA dirty soft-bit */ + +enum pgste_gps_usage { +PGSTE_GPS_USAGE_STABLE =0 , +PGSTE_GPS_USAGE_UNUSED , +PGSTE_GPS_USAGE_POT_VOLATILE , +PGSTE_GPS_USAGE_VOLATILE , +}; + +union pgste { +unsigned long val; +struct { +unsigned long acc :4 ; +unsigned long fp :1 ; + unsigned long : 3; +unsigned long pcl :1 ; +unsigned long hr :1 ; +unsigned long hc :1 ; + unsigned long : 2; +unsigned long gr :1 ; +unsigned long gc :1 ; + unsigned long : 1; + unsigned long :16; /* val16 */ +unsigned long zero :1 ; +unsigned long nodat :1 ; + unsigned long : 4; +unsigned long usage :2 ; + unsigned long : 8; +unsigned long cmma_d :1 ;/* Dirty flag for CMMA bits */ +unsigned long prefix_notif :1 ;/* Guest prefix invalidation notification */ +unsigned long vsie_notif :1 ;/* Referenced in a shadow table */ + unsigned long : 5; + unsigned long : 8; + }; +struct { +unsigned short hwbytes0; +unsigned short val16;/* used to store chunked values, see dat_{s,g}et_ptval() */ +unsigned short hwbytes4; +unsigned char flags;/* maps to the software bits */ +unsigned char hwbyte7; + } __packed; +}; + +union pmd { +unsigned long val; +union segment_table_entry h; +struct { +struct { + unsigned long :44; /* HW */ + unsigned long : 3; /* Unused */ + unsigned long : 1; /* HW */ +unsigned long w :1 ;/* Writable soft-bit */ +unsigned long r :1 ;/* Readable soft-bit */ +unsigned long d :1 ;/* Dirty */ +unsigned long y :1 ;/* Young */ +unsigned long prefix_notif :1 ;/* Guest prefix invalidation notification */ + unsigned long : 3; /* HW */ +unsigned long vsie_notif :1 ;/* Referenced in a shadow table */ + unsigned long : 1; /* Unused */ + unsigned long : 4; /* HW */ +unsigned long sd :1 ;/* Soft-Dirty */ +unsigned long pr :1 ;/* Present */ + } fc1; + } s; +}; + +union pud { +unsigned long val; +union region3_table_entry h; +struct { +struct { + unsigned long :33; /* HW */ + unsigned long :14; /* Unused */ + unsigned long : 1; /* HW */ +unsigned long w :1 ;/* Writable soft-bit */ +unsigned long r :1 ;/* Readable soft-bit */ +unsigned long d :1 ;/* Dirty */ +unsigned long y :1 ;/* Young */ +unsigned long prefix_notif :1 ;/* Guest prefix invalidation notification */ + unsigned long : 3; /* HW */ +unsigned long vsie_notif :1 ;/* Referenced in a shadow table */ + unsigned long : 1; /* Unused */ + unsigned long : 4; /* HW */ +unsigned long sd :1 ;/* Soft-Dirty */ +unsigned long pr :1 ;/* Present */ + } fc1; + } s; +}; + +union p4d { +unsigned long val; +union region2_table_entry h; +}; + +union pgd { +unsigned long val; +union region1_table_entry h; +}; + +union crste { +unsigned long val; +union { +struct { + unsigned long :52; + unsigned long : 1; +unsigned long fc:1 ; +unsigned long p :1 ; + unsigned long : 1; + unsigned long : 2; +unsigned long i :1 ; + unsigned long : 1; +unsigned long tt:2 ; + unsigned long : 2; + }; +struct { +unsigned long to:52 ; + unsigned long : 1; +unsigned long fc:1 ; +unsigned long p :1 ; + unsigned long : 1; +unsigned long tf:2 ; +unsigned long i :1 ; + unsigned long : 1; +unsigned long tt:2 ; +unsigned long tl:2 ; + } fc0; +struct { + unsigned long :47; +unsigned long av :1 ;/* ACCF-Validity Control */ +unsigned long acc:4 ;/* Access-Control Bits */ +unsigned long f :1 ;/* Fetch-Protection Bit */ +unsigned long fc :1 ;/* Format-Control */ +unsigned long p :1 ;/* DAT-Protection Bit */ +unsigned long iep:1 ;/* Instruction-Execution-Protection */ + unsigned long : 2; +unsigned long i :1 ;/* Segment-Invalid Bit */ +unsigned long cs :1 ;/* Common-Segment Bit */ +unsigned long tt :2 ;/* Table-Type Bits */ + unsigned long : 2; + } fc1; + } h; +struct { +struct { + unsigned long :47; + unsigned long : 1; /* HW (should be 0) */ +unsigned long w :1 ;/* Writable */ +unsigned long r :1 ;/* Readable */ +unsigned long d :1 ;/* Dirty */ +unsigned long y :1 ;/* Young */ +unsigned long prefix_notif :1 ;/* Guest prefix invalidation notification */ + unsigned long : 3; /* HW */ +unsigned long vsie_notif :1 ;/* Referenced in a shadow table */ + unsigned long : 1; + unsigned long : 4; /* HW */ +unsigned long sd :1 ;/* Soft-Dirty */ +unsigned long pr :1 ;/* Present */ + } fc1; + } s; +union { +struct { +unsigned long type :16 ;/* Token type */ +unsigned long par :16 ;/* Token parameter */ + unsigned long :26; +unsigned long i :1 ;/* Must be 1 */ + unsigned long : 1; +unsigned long tt :2 ; + unsigned long : 1; +unsigned long pr :1 ;/* Must be 0 */ + }; +struct { +unsigned long token:32 ;/* Token and parameter */ + unsigned long :32; + }; + } tok; +union pmd pmd; +union pud pud; +union p4d p4d; +union pgd pgd; +}; + +union skey { +unsigned char skey; +struct { +unsigned char acc :4 ; +unsigned char fp :1 ; +unsigned char r :1 ; +unsigned char c :1 ; +unsigned char zero:1 ; + }; +}; + +static_assert (sizeof (union pgste ) ==sizeof (unsigned long )); +static_assert (sizeof (union pte ) ==sizeof (unsigned long )); +static_assert (sizeof (union pmd ) ==sizeof (unsigned long )); +static_assert (sizeof (union pud ) ==sizeof (unsigned long )); +static_assert (sizeof (union p4d ) ==sizeof (unsigned long )); +static_assert (sizeof (union pgd ) ==sizeof (unsigned long )); +static_assert (sizeof (union crste ) ==sizeof (unsigned long )); +static_assert (sizeof (union skey ) ==sizeof (char )); + +struct segment_table { +union pmd pmds[_CRST_ENTRIES]; +}; + +struct region3_table { +union pud puds[_CRST_ENTRIES]; +}; + +struct region2_table { +union p4d p4ds[_CRST_ENTRIES]; +}; + +struct region1_table { +union pgd pgds[_CRST_ENTRIES]; +}; + +struct crst_table { +union { +union crste crstes[_CRST_ENTRIES]; +struct segment_table segment; +struct region3_table region3; +struct region2_table region2; +struct region1_table region1; + }; +}; + +struct page_table { +union pte ptes[_PAGE_ENTRIES]; +union pgste pgstes[_PAGE_ENTRIES]; +}; + +static_assert (sizeof (struct crst_table ) == _CRST_TABLE_SIZE); +static_assert (sizeof (struct page_table ) ==PAGE_SIZE ); + +/** +* _pte() - Useful constructor for union pte +* @pfn: the pfn this pte should point to. +* @writable: whether the pte should be writable. +* @dirty: whether the pte should be dirty. +* @special: whether the pte should be marked as special +* +* The pte is also marked as young and present. If the pte is marked as dirty, +* it gets marked as soft-dirty too. If the pte is not dirty, the hardware +* protect bit is set (independently of the write softbit); this way proper +* dirty tracking can be performed. +* +* Return: a union pte value. +*/ +static inline union pte _pte (kvm_pfn_t pfn,bool writable,bool dirty,bool special) +{ +union pte res = { .val =PFN_PHYS (pfn) }; + + res.h.p = !dirty; + res.s.y =1 ; + res.s.pr =1 ; + res.s.w = writable; + res.s.d = dirty; + res.s.sd = dirty; + res.s.s = special; +return res; +} + +static inline union crste _crste_fc0 (kvm_pfn_t pfn,int tt) +{ +union crste res = { .val =PFN_PHYS (pfn) }; + + res.h.tt = tt; + res.h.fc0.tl = _REGION_ENTRY_LENGTH; + res.h.fc0.tf =0 ; +return res; +} + +/** +* _crste() - Useful constructor for union crste with FC=1 +* @pfn: the pfn this pte should point to. +* @tt: the table type +* @writable: whether the pte should be writable. +* @dirty: whether the pte should be dirty. +* +* The crste is also marked as young and present. If the crste is marked as +* dirty, it gets marked as soft-dirty too. If the crste is not dirty, the +* hardware protect bit is set (independently of the write softbit); this way +* proper dirty tracking can be performed. +* +* Return: a union crste value. +*/ +static inline union crste _crste_fc1 (kvm_pfn_t pfn,int tt,bool writable,bool dirty) +{ +union crste res = { .val =PFN_PHYS (pfn) & _SEGMENT_MASK }; + + res.h.tt = tt; + res.h.p = !dirty; + res.h.fc =1 ; + res.s.fc1.y =1 ; + res.s.fc1.pr =1 ; + res.s.fc1.w = writable; + res.s.fc1.d = dirty; + res.s.fc1.sd = dirty; +return res; +} + +/** +* struct vsie_rmap - reverse mapping for shadow page table entries +* @next: pointer to next rmap in the list +* @r_gfn: virtual rmap address in the shadow guest address space +*/ +struct vsie_rmap { +struct vsie_rmap *next; +union { +unsigned long val; +struct { +long level:8 ; + unsigned long : 4; +unsigned long r_gfn:52 ; + }; + }; +}; + +static_assert (sizeof (struct vsie_rmap ) ==2 *sizeof (long )); + +static inline struct crst_table *crste_table_start (union crste *crstep) +{ +return (struct crst_table *)ALIGN_DOWN ((unsigned long )crstep, _CRST_TABLE_SIZE); +} + +static inline struct page_table *pte_table_start (union pte *ptep) +{ +return (struct page_table *)ALIGN_DOWN ((unsigned long )ptep, _PAGE_TABLE_SIZE); +} + +static inline bool crdte_crste (union crste *crstep,union crste old,union crste new,gfn_t gfn, +union asce asce) +{ +unsigned long dtt =0x10 | new.h.tt <<2 ; +void *table =crste_table_start (crstep); + +return crdte (old.val, new.val, table, dtt,gfn_to_gpa (gfn), asce.val); +} + +/** +* idte_crste() - invalidate a crste entry using idte +* @crstep: pointer to the crste to be invalidated +* @gfn: a gfn mapped by the crste +* @opt: options for the idte instruction +* @asce: the asce +* @local: whether the operation is cpu-local +*/ +static __always_inline void idte_crste(union crste *crstep, gfn_t gfn, unsigned long opt, +union asce asce,int local) +{ +unsigned long table_origin =__pa (crste_table_start (crstep)); +unsigned long gaddr =gfn_to_gpa (gfn) &HPAGE_MASK ; + +if (__builtin_constant_p (opt) && opt ==0 ) { +/* flush without guest asce */ + asmvolatile ("idte %[table_origin],0,%[gaddr],%[local]" + :"+m" (*crstep) + : [table_origin]"a" (table_origin), [gaddr]"a" (gaddr), + [local]"i" (local) + :"cc" ); + }else { +/* flush with guest asce */ + asmvolatile ("idte %[table_origin],%[asce],%[gaddr_opt],%[local]" + :"+m" (*crstep) + : [table_origin]"a" (table_origin), [gaddr_opt]"a" (gaddr | opt), + [asce]"a" (asce.val), [local]"i" (local) + :"cc" ); + } +} + +static inline void dat_init_pgstes (struct page_table *pt,unsigned long val) +{ +memset64 ((void *)pt->pgstes, val,PTRS_PER_PTE ); +} + +static inline void dat_init_page_table (struct page_table *pt,unsigned long ptes, +unsigned long pgstes) +{ +memset64 ((void *)pt->ptes, ptes,PTRS_PER_PTE ); +dat_init_pgstes (pt, pgstes); +} + +static inline gfn_t asce_end (union asce asce) +{ +return 1ULL << ((asce.dt +1 ) *11 + _SEGMENT_SHIFT -PAGE_SHIFT ); +} + +#define _CRSTE (x) ((union crste) { .val = _Generic((x), \ + union pgd : (x).val, \ + union p4d : (x).val, \ + union pud : (x).val, \ + union pmd : (x).val, \ + union crste : (x).val)}) + +#define _CRSTEP (x) ((union crste *)_Generic((*(x)), \ + union pgd : (x), \ + union p4d : (x), \ + union pud : (x), \ + union pmd : (x), \ + union crste : (x))) + +#define _CRSTP (x) ((struct crst_table *)_Generic((*(x)), \ + struct crst_table : (x), \ + struct segment_table : (x), \ + struct region3_table : (x), \ + struct region2_table : (x), \ + struct region1_table : (x))) + +static inline bool asce_contains_gfn (union asce asce,gfn_t gfn) +{ +return gfn <asce_end (asce); +} + +static inline bool is_pmd (union crste crste) +{ +return crste.h.tt ==TABLE_TYPE_SEGMENT ; +} + +static inline bool is_pud (union crste crste) +{ +return crste.h.tt ==TABLE_TYPE_REGION3 ; +} + +static inline bool is_p4d (union crste crste) +{ +return crste.h.tt ==TABLE_TYPE_REGION2 ; +} + +static inline bool is_pgd (union crste crste) +{ +return crste.h.tt ==TABLE_TYPE_REGION1 ; +} + +static inline phys_addr_t pmd_origin_large (union pmd pmd) +{ +return pmd.val & _SEGMENT_ENTRY_ORIGIN_LARGE; +} + +static inline phys_addr_t pud_origin_large (union pud pud) +{ +return pud.val & _REGION3_ENTRY_ORIGIN_LARGE; +} + +/** +* crste_origin_large() - Return the large frame origin of a large crste +* @crste: The crste whose origin is to be returned. Should be either a +* region-3 table entry or a segment table entry, in both cases with +* FC set to 1 (large pages). +* +* Return: The origin of the large frame pointed to by @crste, or -1 if the +* crste was not large (wrong table type, or FC==0) +*/ +static inline phys_addr_t crste_origin_large (union crste crste) +{ +if (unlikely (!crste.h.fc || crste.h.tt >TABLE_TYPE_REGION3 )) +return -1 ; +if (is_pmd (crste)) +return pmd_origin_large (crste.pmd); +return pud_origin_large (crste.pud); +} + +#define crste_origin (x) (_Generic((x), \ + union pmd : (x).val & _SEGMENT_ENTRY_ORIGIN, \ + union pud : (x).val & _REGION_ENTRY_ORIGIN, \ + union p4d : (x).val & _REGION_ENTRY_ORIGIN, \ + union pgd : (x).val & _REGION_ENTRY_ORIGIN)) + +static inline unsigned long pte_origin (union pte pte) +{ +return pte.val &PAGE_MASK ; +} + +static inline bool pmd_prefix (union pmd pmd) +{ +return pmd.h.fc && pmd.s.fc1.prefix_notif; +} + +static inline bool pud_prefix (union pud pud) +{ +return pud.h.fc && pud.s.fc1.prefix_notif; +} + +static inline bool crste_leaf (union crste crste) +{ +return (crste.h.tt <=TABLE_TYPE_REGION3 ) && crste.h.fc; +} + +static inline bool crste_prefix (union crste crste) +{ +return crste_leaf (crste) && crste.s.fc1.prefix_notif; +} + +static inline bool crste_dirty (union crste crste) +{ +return crste_leaf (crste) && crste.s.fc1.d; +} + +static inline union pgste *pgste_of (union pte *pte) +{ +return (union pgste *)(pte + _PAGE_ENTRIES); +} + +static inline bool pte_hole (union pte pte) +{ +return pte.h.i && !pte.tok.pr && pte.tok.type != _DAT_TOKEN_NONE; +} + +static inline bool _crste_hole (union crste crste) +{ +return crste.h.i && !crste.tok.pr && crste.tok.type != _DAT_TOKEN_NONE; +} + +#define crste_hole (x) _crste_hole(_CRSTE(x)) + +static inline bool _crste_none (union crste crste) +{ +return crste.h.i && !crste.tok.pr && crste.tok.type == _DAT_TOKEN_NONE; +} + +#define crste_none (x) _crste_none(_CRSTE(x)) + +static inline phys_addr_t large_pud_to_phys (union pud pud,gfn_t gfn) +{ +return pud_origin_large (pud) | (gfn_to_gpa (gfn) & ~_REGION3_MASK); +} + +static inline phys_addr_t large_pmd_to_phys (union pmd pmd,gfn_t gfn) +{ +return pmd_origin_large (pmd) | (gfn_to_gpa (gfn) & ~_SEGMENT_MASK); +} + +static inline phys_addr_t large_crste_to_phys (union crste crste,gfn_t gfn) +{ +if (unlikely (!crste.h.fc || crste.h.tt >TABLE_TYPE_REGION3 )) +return -1 ; +if (is_pmd (crste)) +return large_pmd_to_phys (crste.pmd, gfn); +return large_pud_to_phys (crste.pud, gfn); +} + +static inline bool cspg_crste (union crste *crstep,union crste old,union crste new) +{ +return cspg (&crstep->val, old.val, new.val); +} + +static inline struct page_table *dereference_pmd (union pmd pmd) +{ +return phys_to_virt (crste_origin (pmd)); +} + +static inline struct segment_table *dereference_pud (union pud pud) +{ +return phys_to_virt (crste_origin (pud)); +} + +static inline struct region3_table *dereference_p4d (union p4d p4d) +{ +return phys_to_virt (crste_origin (p4d)); +} + +static inline struct region2_table *dereference_pgd (union pgd pgd) +{ +return phys_to_virt (crste_origin (pgd)); +} + +static inline struct crst_table *_dereference_crste (union crste crste) +{ +if (unlikely (is_pmd (crste))) +return NULL ; +return phys_to_virt (crste_origin (crste.pud)); +} + +#define dereference_crste (x) (_Generic((x), \ + union pud : _dereference_crste(_CRSTE(x)), \ + union p4d : _dereference_crste(_CRSTE(x)), \ + union pgd : _dereference_crste(_CRSTE(x)), \ + union crste : _dereference_crste(_CRSTE(x)))) + +static inline struct crst_table *dereference_asce (union asce asce) +{ +return phys_to_virt (asce.val & _ASCE_ORIGIN); +} + +static inline void asce_flush_tlb (union asce asce) +{ +__tlb_flush_idte (asce.val); +} + +static inline bool pgste_get_trylock (union pte *ptep,union pgste *res) +{ +union pgste *pgstep =pgste_of (ptep); +union pgste old_pgste; + +if (READ_ONCE (pgstep->val) &PGSTE_PCL_BIT ) +return false; + old_pgste.val =__atomic64_or_barrier (PGSTE_PCL_BIT , &pgstep->val); +if (old_pgste.pcl) +return false; + old_pgste.pcl =1 ; + *res = old_pgste; +return true; +} + +static inline union pgste pgste_get_lock (union pte *ptep) +{ +union pgste res; + +while (!pgste_get_trylock (ptep, &res)) +cpu_relax (); +return res; +} + +static inline void pgste_set_unlock (union pte *ptep,union pgste pgste) +{ + pgste.pcl =0 ; +barrier (); +WRITE_ONCE (*pgste_of (ptep), pgste); +} + +#endif /* __KVM_S390_DAT_H */
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 13/28 KVM: s390: KVM page table management functions: allocation
In reply to Claudio ImbrendaAdd page table management functions to be used for KVM guest (gmap) page tables.
This patch adds the boilerplate and functions for the allocation and deallocation of DAT tables.
--- arch/s390/kvm/Makefile | 1 + arch/s390/kvm/dat.c | 103 +++++++++++++++++++++++++++++++++++++ arch/s390/kvm/dat.h | 77 +++++++++++++++++++++++++++ arch/s390/mm/page-states.c | 1 + 4 files changed, 182 insertions(+) create mode 100644 arch/s390/kvm/dat.c
arch/s390/kvm/Makefile +1 -0
@@ -9,6 +9,7 @@ ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap-vsie.o +kvm-y += dat.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/dat.c new file +103 -0
103 lines not on this page · raw message
arch/s390/kvm/dat.h +77 -0
87 lines not on this page · raw message
arch/s390/mm/page-states.c +1 -0
@@ -13,6 +13,7 @@#include <asm/page.h> int __bootdata_preserved (cmma_flag ); +EXPORT_SYMBOL (cmma_flag);void arch_free_page (struct page *page,int order) {
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 14/28 KVM: s390: KVM page table management functions: clear and replace
In reply to Claudio ImbrendaAdd page table management functions to be used for KVM guest (gmap) page tables.
This patch adds functions to clear, replace or exchange DAT table entries.
--- arch/s390/kvm/dat.c | 118 ++++++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/dat.h | 40 +++++++++++++++ 2 files changed, 158 insertions(+)
arch/s390/kvm/dat.c +118 -0
@@ -101,3 +101,121 @@ void dat_free_level(struct crst_table *table, bool owns_ptes) }dat_free_crst (table); } + +/** +* dat_crstep_xchg - exchange a gmap CRSTE with another +* @crstep: pointer to the CRST entry +* @new: replacement entry +* @gfn: the affected guest address +* @asce: the ASCE of the address space +* +* Context: This function is assumed to be called with kvm->mmu_lock held. +*/ +void dat_crstep_xchg (union crste *crstep,union crste new,gfn_t gfn,union asce asce) +{ +if (crstep->h.i) { +WRITE_ONCE (*crstep, new); +return ; + }else if (cpu_has_edat2 ()) { +crdte_crste (crstep, *crstep, new, gfn, asce); +return ; + } + +if (machine_has_tlb_guest ()) +idte_crste (crstep, gfn,IDTE_GUEST_ASCE , asce,IDTE_GLOBAL ); +else +idte_crste (crstep, gfn,0 ,NULL_ASCE ,IDTE_GLOBAL ); +WRITE_ONCE (*crstep, new); +} + +/** +* dat_crstep_xchg_atomic - atomically exchange a gmap CRSTE with another +* @crstep: pointer to the CRST entry +* @old: expected old value +* @new: replacement entry +* @gfn: the affected guest address +* @asce: the asce of the address space +* +* This function should only be called on invalid crstes, or on crstes with +* FC = 1, as that guarantees the presence of CSPG. +* +* This function is needed to atomically exchange a CRSTE that potentially +* maps a prefix area, without having to invalidate it inbetween. +* +* Context: This function is assumed to be called with kvm->mmu_lock held. +* +* Return: true if the exchange was successful. +*/ +bool dat_crstep_xchg_atomic (union crste *crstep,union crste old,union crste new,gfn_t gfn, +union asce asce) +{ +if (old.h.i) +return arch_try_cmpxchg ((long *)crstep, &old.val, new.val); +if (cpu_has_edat2 ()) +return crdte_crste (crstep, old, new, gfn, asce); +return cspg_crste (crstep, old, new); +} + +static void dat_set_storage_key_from_pgste (union pte pte,union pgste pgste) +{ +union skey nkey = { .acc = pgste.acc, .fp = pgste.fp }; + +page_set_storage_key (pte_origin (pte), nkey.skey,0 ); +} + +static void dat_move_storage_key (union pte old,union pte new) +{ +page_set_storage_key (pte_origin (new),page_get_storage_key (pte_origin (old)),1 ); +} + +static union pgste dat_save_storage_key_into_pgste (union pte pte,union pgste pgste) +{ +union skey skey; + + skey.skey =page_get_storage_key (pte_origin (pte)); + + pgste.acc = skey.acc; + pgste.fp = skey.fp; + pgste.gr |= skey.r; + pgste.gc |= skey.c; + +return pgste; +} + +union pgste __dat_ptep_xchg (union pte *ptep,union pgste pgste,union pte new,gfn_t gfn, +union asce asce,bool uses_skeys) +{ +union pte old =READ_ONCE (*ptep); + +/* Updating only the software bits while holding the pgste lock */ +if (!((ptep->val ^ new.val) & ~_PAGE_SW_BITS)) { +WRITE_ONCE (ptep->swbyte, new.swbyte); +return pgste; + } + +if (!old.h.i) { +unsigned long opts =IPTE_GUEST_ASCE | (pgste.nodat ?IPTE_NODAT :0 ); + +if (machine_has_tlb_guest ()) +__ptep_ipte (gfn_to_gpa (gfn), (void *)ptep, opts, asce.val,IPTE_GLOBAL ); +else +__ptep_ipte (gfn_to_gpa (gfn), (void *)ptep,0 ,0 ,IPTE_GLOBAL ); + } + +if (uses_skeys) { +if (old.h.i && !new.h.i) +/* Invalid to valid: restore storage keys from PGSTE */ +dat_set_storage_key_from_pgste (new, pgste); +else if (!old.h.i && new.h.i) +/* Valid to invalid: save storage keys to PGSTE */ + pgste =dat_save_storage_key_into_pgste (old, pgste); +else if (!old.h.i && !new.h.i) +/* Valid to valid: move storage keys */ +if (old.h.pfra != new.h.pfra) +dat_move_storage_key (old, new); +/* Invalid to invalid: nothing to do */ + } + +WRITE_ONCE (*ptep, new); +return pgste; +}
arch/s390/kvm/dat.h +40 -0
@@ -430,6 +430,12 @@ struct kvm_s390_mmu_cache {short int n_rmaps; }; +union pgste __must_check __dat_ptep_xchg (union pte *ptep,union pgste pgste,union pte new, +gfn_t gfn,union asce asce,bool uses_skeys); +bool dat_crstep_xchg_atomic (union crste *crstep,union crste old,union crste new,gfn_t gfn, +union asce asce); +void dat_crstep_xchg (union crste *crstep,union crste new,gfn_t gfn,union asce asce); +void dat_free_level (struct crst_table *table,bool owns_ptes);struct crst_table *dat_alloc_crst_sleepable (unsigned long init);
@@ -757,6 +763,21 @@ static inline void pgste_set_unlock(union pte *ptep, union pgste pgste)WRITE_ONCE (*pgste_of (ptep), pgste); } +static inline void dat_ptep_xchg (union pte *ptep,union pte new,gfn_t gfn,union asce asce, +bool has_skeys) +{ +union pgste pgste; + + pgste =pgste_get_lock (ptep); + pgste =__dat_ptep_xchg (ptep, pgste, new, gfn, asce, has_skeys); +pgste_set_unlock (ptep, pgste); +} + +static inline void dat_ptep_clear (union pte *ptep,gfn_t gfn,union asce asce,bool has_skeys) +{ +dat_ptep_xchg (ptep, _PTE_EMPTY, gfn, asce, has_skeys); +} +static inline void dat_free_pt (struct page_table *pt) {free_page ((unsigned long )pt);
@@ -794,4 +815,23 @@ static inline struct kvm_s390_mmu_cache *kvm_s390_new_mmu_cache(void)return NULL ; } +static inline bool dat_pmdp_xchg_atomic (union pmd *pmdp,union pmd old,union pmd new, +gfn_t gfn,union asce asce) +{ +return dat_crstep_xchg_atomic (_CRSTEP (pmdp),_CRSTE (old),_CRSTE (new), gfn, asce); +} + +static inline bool dat_pudp_xchg_atomic (union pud *pudp,union pud old,union pud new, +gfn_t gfn,union asce asce) +{ +return dat_crstep_xchg_atomic (_CRSTEP (pudp),_CRSTE (old),_CRSTE (new), gfn, asce); +} + +static inline void dat_crstep_clear (union crste *crstep,gfn_t gfn,union asce asce) +{ +union crste newcrste =_CRSTE_EMPTY (crstep->h.tt); + +dat_crstep_xchg (crstep, newcrste, gfn, asce); +} +#endif /* __KVM_S390_DAT_H */
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 15/28 KVM: s390: KVM page table management functions: walks
In reply to Claudio ImbrendaAdd page table management functions to be used for KVM guest (gmap) page tables.
This patch adds functions to walk to specific table entries, or to perform actions on a range of entries.
--- arch/s390/kvm/dat.c | 386 ++++++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/dat.h | 39 +++++ 2 files changed, 425 insertions(+)
arch/s390/kvm/dat.c +386 -0
@@ -219,3 +219,389 @@ union pgste __dat_ptep_xchg(union pte *ptep, union pgste pgste, union pte new, gWRITE_ONCE (*ptep, new);return pgste; } + +/* +* dat_split_ste - Split a segment table entry into page table entries +* +* Context: This function is assumed to be called with kvm->mmu_lock held. +* +* Return: 0 in case of success, -ENOMEM if running out of memory. +*/ +static int dat_split_ste (struct kvm_s390_mmu_cache *mc,union pmd *pmdp,gfn_t gfn, +union asce asce,bool uses_skeys) +{ +union pgste pgste_init; +struct page_table *pt; +union pmd new, old; +union pte init; +int i; + +BUG_ON (!mc); + old =READ_ONCE (*pmdp); + +/* Already split, nothing to do */ +if (!old.h.i && !old.h.fc) +return 0 ; + + pt =dat_alloc_pt_noinit (mc); +if (!pt) +return -ENOMEM ; + new.val =virt_to_phys (pt); + +while (old.h.i || old.h.fc) { + init.val =pmd_origin_large (old); + init.h.p = old.h.p; + init.h.i = old.h.i; + init.s.d = old.s.fc1.d; + init.s.w = old.s.fc1.w; + init.s.y = old.s.fc1.y; + init.s.sd = old.s.fc1.sd; + init.s.pr = old.s.fc1.pr; + pgste_init.val =0 ; +if (old.h.fc) { +for (i =0 ; i < _PAGE_ENTRIES; i++) + pt->ptes[i].val = init.val | i *PAGE_SIZE ; +/* no need to take locks as the page table is not installed yet */ + pgste_init.prefix_notif = old.s.fc1.prefix_notif; + pgste_init.pcl = uses_skeys && init.h.i; +dat_init_pgstes (pt, pgste_init.val); + }else { +dat_init_page_table (pt, init.val,0 ); + } + +if (dat_pmdp_xchg_atomic (pmdp, old, new, gfn, asce)) { +if (!pgste_init.pcl) +return 0 ; +for (i =0 ; i < _PAGE_ENTRIES; i++) { +union pgste pgste = pt->pgstes[i]; + + pgste =dat_save_storage_key_into_pgste (pt->ptes[i], pgste); +pgste_set_unlock (pt->ptes + i, pgste); + } +return 0 ; + } + old =READ_ONCE (*pmdp); + } + +dat_free_pt (pt); +return 0 ; +} + +/* +* dat_split_crste - Split a crste into smaller crstes +* +* Context: This function is assumed to be called with kvm->mmu_lock held. +* +* Return: 0 in case of success, -ENOMEM if running out of memory. +*/ +static int dat_split_crste (struct kvm_s390_mmu_cache *mc,union crste *crstep, +gfn_t gfn,union asce asce,bool uses_skeys) +{ +struct crst_table *table; +union crste old, new, init; +int i; + + old =READ_ONCE (*crstep); +if (is_pmd (old)) +return dat_split_ste (mc, &crstep->pmd, gfn, asce, uses_skeys); + +BUG_ON (!mc); + +/* Already split, nothing to do */ +if (!old.h.i && !old.h.fc) +return 0 ; + + table =dat_alloc_crst_noinit (mc); +if (!table) +return -ENOMEM ; + + new.val =virt_to_phys (table); + new.h.tt = old.h.tt; + new.h.fc0.tl = _REGION_ENTRY_LENGTH; + +while (old.h.i || old.h.fc) { + init = old; + init.h.tt--; +if (old.h.fc) { +for (i =0 ; i < _CRST_ENTRIES; i++) + table->crstes[i].val = init.val | i *HPAGE_SIZE ; + }else { +crst_table_init ((void *)table, init.val); + } +if (dat_crstep_xchg_atomic (crstep, old, new, gfn, asce)) +return 0 ; + old =READ_ONCE (*crstep); + } + +dat_free_crst (table); +return 0 ; +} + +/** +* dat_entry_walk() - walk the gmap page tables +* @mc: cache to use to allocate dat tables, if needed; can be NULL if neither +* @DAT_WALK_SPLIT or @DAT_WALK_ALLOC is specified. +* @gfn: guest frame +* @asce: the ASCE of the address space +* @flags: flags from WALK_* macros +* @walk_level: level to walk to, from LEVEL_* macros +* @last: will be filled the last visited non-pte DAT entry +* @ptepp: will be filled the last visited pte entry, if any, otherwise NULL +* +* Returns a table entry pointer for the given guest address and @level +* +* The @flags have the following meanings: +* * @DAT_WALK_IGN_HOLES: consider holes as normal table entries +* * @DAT_WALK_ALLOC: allocate new tables to reach the requested level, if needed +* * @DAT_WALK_SPLIT: split existing large pages to reach the requested level, if needed +* * @DAT_WALK_LEAF: return successfully whenever a large page is encountered +* * @DAT_WALK_ANY: return successfully even if the requested level could not be reached +* * @DAT_WALK_CONTINUE: walk to the requested level with the specified flags, and then try to +* continue walking to ptes with only DAT_WALK_ANY +* * @DAT_WALK_USES_SKEYS: storage keys are in use +* +* Context: called with kvm->mmu_lock held. +* +* Return: +* * PGM_ADDRESSING if the requested address lies outside memory +* * a PIC number if the requested address lies in a memory hole of type _DAT_TOKEN_PIC +* * -EFAULT if the requested address lies inside a memory hole of a different type +* * -EINVAL if the given ASCE is not compatible with the requested level +* * -EFBIG if the requested level could not be reached because a larger frame was found +* * -ENOENT if the requested level could not be reached for other reasons +* * -ENOMEM if running out of memory while allocating or splitting a table +*/ +int dat_entry_walk (struct kvm_s390_mmu_cache *mc,gfn_t gfn,union asce asce,int flags, +int walk_level,union crste **last,union pte **ptepp) +{ +union vaddress vaddr = { .addr =gfn_to_gpa (gfn) }; +bool continue_anyway = flags &DAT_WALK_CONTINUE ; +bool uses_skeys = flags &DAT_WALK_USES_SKEYS ; +bool ign_holes = flags &DAT_WALK_IGN_HOLES ; +bool allocate = flags &DAT_WALK_ALLOC ; +bool split = flags &DAT_WALK_SPLIT ; +bool leaf = flags &DAT_WALK_LEAF ; +bool any = flags &DAT_WALK_ANY ; +struct page_table *pgtable; +struct crst_table *table; +union crste entry; +int rc; + + *last =NULL ; + *ptepp =NULL ; +if (WARN_ON_ONCE (unlikely (!asce.val))) +return -EINVAL ; +if (WARN_ON_ONCE (unlikely (walk_level > asce.dt))) +return -EINVAL ; +if (!asce_contains_gfn (asce, gfn)) +return PGM_ADDRESSING ; + + table =dereference_asce (asce); +if (asce.dt >=ASCE_TYPE_REGION1 ) { + *last = table->crstes + vaddr.rfx; + entry =READ_ONCE (**last); +if (WARN_ON_ONCE (entry.h.tt !=TABLE_TYPE_REGION1 )) +return -EINVAL ; +if (crste_hole (entry) && !ign_holes) +return entry.tok.type == _DAT_TOKEN_PIC ? entry.tok.par : -EFAULT ; +if (walk_level ==TABLE_TYPE_REGION1 ) +return 0 ; +if (entry.pgd.h.i) { +if (!allocate) +return any ?0 : -ENOENT ; + rc =dat_split_crste (mc, *last, gfn, asce, uses_skeys); +if (rc) +return rc; + entry =READ_ONCE (**last); + } + table =dereference_crste (entry.pgd); + } + +if (asce.dt >=ASCE_TYPE_REGION2 ) { + *last = table->crstes + vaddr.rsx; + entry =READ_ONCE (**last); +if (WARN_ON_ONCE (entry.h.tt !=TABLE_TYPE_REGION2 )) +return -EINVAL ; +if (crste_hole (entry) && !ign_holes) +return entry.tok.type == _DAT_TOKEN_PIC ? entry.tok.par : -EFAULT ; +if (walk_level ==TABLE_TYPE_REGION2 ) +return 0 ; +if (entry.p4d.h.i) { +if (!allocate) +return any ?0 : -ENOENT ; + rc =dat_split_crste (mc, *last, gfn, asce, uses_skeys); +if (rc) +return rc; + entry =READ_ONCE (**last); + } + table =dereference_crste (entry.p4d); + } + +if (asce.dt >=ASCE_TYPE_REGION3 ) { + *last = table->crstes + vaddr.rtx; + entry =READ_ONCE (**last); +if (WARN_ON_ONCE (entry.h.tt !=TABLE_TYPE_REGION3 )) +return -EINVAL ; +if (crste_hole (entry) && !ign_holes) +return entry.tok.type == _DAT_TOKEN_PIC ? entry.tok.par : -EFAULT ; +if (walk_level ==TABLE_TYPE_REGION3 && + continue_anyway && !entry.pud.h.fc && !entry.h.i) { + walk_level =TABLE_TYPE_PAGE_TABLE ; + allocate = false; + } +if (walk_level ==TABLE_TYPE_REGION3 || ((leaf || any) && entry.pud.h.fc)) +return 0 ; +if (entry.pud.h.i && !entry.pud.h.fc) { +if (!allocate) +return any ?0 : -ENOENT ; + rc =dat_split_crste (mc, *last, gfn, asce, uses_skeys); +if (rc) +return rc; + entry =READ_ONCE (**last); + } +if (walk_level <=TABLE_TYPE_SEGMENT && entry.pud.h.fc) { +if (!split) +return -EFBIG ; + rc =dat_split_crste (mc, *last, gfn, asce, uses_skeys); +if (rc) +return rc; + entry =READ_ONCE (**last); + } + table =dereference_crste (entry.pud); + } + + *last = table->crstes + vaddr.sx; + entry =READ_ONCE (**last); +if (WARN_ON_ONCE (entry.h.tt !=TABLE_TYPE_SEGMENT )) +return -EINVAL ; +if (crste_hole (entry) && !ign_holes) +return entry.tok.type == _DAT_TOKEN_PIC ? entry.tok.par : -EFAULT ; +if (continue_anyway && !entry.pmd.h.fc && !entry.h.i) { + walk_level =TABLE_TYPE_PAGE_TABLE ; + allocate = false; + } +if (walk_level ==TABLE_TYPE_SEGMENT || ((leaf || any) && entry.pmd.h.fc)) +return 0 ; + +if (entry.pmd.h.i && !entry.pmd.h.fc) { +if (!allocate) +return any ?0 : -ENOENT ; + rc =dat_split_ste (mc, &(*last)->pmd, gfn, asce, uses_skeys); +if (rc) +return rc; + entry =READ_ONCE (**last); + } +if (walk_level <=TABLE_TYPE_PAGE_TABLE && entry.pmd.h.fc) { +if (!split) +return -EFBIG ; + rc =dat_split_ste (mc, &(*last)->pmd, gfn, asce, uses_skeys); +if (rc) +return rc; + entry =READ_ONCE (**last); + } + pgtable =dereference_pmd (entry.pmd); + *ptepp = pgtable->ptes + vaddr.px; +if (pte_hole (**ptepp) && !ign_holes) +return (*ptepp)->tok.type == _DAT_TOKEN_PIC ? (*ptepp)->tok.par : -EFAULT ; +return 0 ; +} + +static long dat_pte_walk_range (gfn_t gfn,gfn_t end,struct page_table *table,struct dat_walk *w) +{ +unsigned int idx = gfn & (_PAGE_ENTRIES -1 ); +long rc =0 ; + +for ( ; gfn < end; idx++, gfn++) { +if (pte_hole (READ_ONCE (table->ptes[idx]))) { +if (!(w->flags &DAT_WALK_IGN_HOLES )) +return -EFAULT ; +if (!(w->flags &DAT_WALK_ANY )) +continue ; + } + + rc = w->ops->pte_entry (table->ptes + idx, gfn, gfn +1 , w); +if (rc) +break ; + } +return rc; +} + +static long dat_crste_walk_range (gfn_t start,gfn_t end,struct crst_table *table, +struct dat_walk *walk) +{ +unsigned long idx, cur_shift, cur_size; +dat_walk_op the_op; +union crste crste; +gfn_t cur, next; +long rc =0 ; + + cur_shift =8 + table->crstes[0 ].h.tt *11 ; + idx = (start >> cur_shift) & (_CRST_ENTRIES -1 ); + cur_size =1UL << cur_shift; + +for (cur =ALIGN_DOWN (start, cur_size); cur < end; idx++, cur = next) { + next = cur + cur_size; + walk->last = table->crstes + idx; + crste =READ_ONCE (*walk->last); + +if (crste_hole (crste)) { +if (!(walk->flags &DAT_WALK_IGN_HOLES )) +return -EFAULT ; +if (!(walk->flags &DAT_WALK_ANY )) +continue ; + } + + the_op = walk->ops->crste_ops[crste.h.tt]; +if (the_op) { + rc =the_op (walk->last, cur, next, walk); + crste =READ_ONCE (*walk->last); + } +if (rc) +break ; +if (!crste.h.i && !crste.h.fc) { +if (!is_pmd (crste)) + rc =dat_crste_walk_range (max (start, cur),min (end, next), +_dereference_crste (crste), walk); +else if (walk->ops->pte_entry) + rc =dat_pte_walk_range (max (start, cur),min (end, next), +dereference_pmd (crste.pmd), walk); + } + } +return rc; +} + +/** +* _dat_walk_gfn_range() - walk DAT tables +* @start: the first guest page frame to walk +* @end: the guest page frame immediately after the last one to walk +* @asce: the ASCE of the guest mapping +* @ops: the gmap_walk_ops that will be used to perform the walk +* @flags: flags from WALK_* (currently only WALK_IGN_HOLES is supported) +* @priv: will be passed as-is to the callbacks +* +* Any callback returning non-zero causes the walk to stop immediately. +* +* Return: -EINVAL in case of error, -EFAULT if @start is too high for the given +* asce unless the DAT_WALK_IGN_HOLES flag is specified, otherwise it +* returns whatever the callbacks return. +*/ +long _dat_walk_gfn_range (gfn_t start,gfn_t end,union asce asce, +const struct dat_walk_ops *ops,int flags,void *priv) +{ +struct crst_table *table =dereference_asce (asce); +struct dat_walk walk = { + .ops = ops, + .asce = asce, + .priv = priv, + .flags = flags, + .start = start, + .end = end, + }; + +if (WARN_ON_ONCE (unlikely (!asce.val))) +return -EINVAL ; +if (!asce_contains_gfn (asce, start)) +return (flags &DAT_WALK_IGN_HOLES ) ?0 : -EFAULT ; + +return dat_crste_walk_range (start,min (end,asce_end (asce)), table, &walk); +}
arch/s390/kvm/dat.h +39 -0
@@ -45,6 +45,7 @@ enum {#define TABLE_TYPE_PAGE_TABLE -1enum dat_walk_flags { +DAT_WALK_USES_SKEYS =0x40 ,DAT_WALK_CONTINUE =0x20 ,DAT_WALK_IGN_HOLES =0x10 ,DAT_WALK_SPLIT =0x08 ,
@@ -332,6 +333,34 @@ struct page_table {static_assert (sizeof (struct crst_table ) == _CRST_TABLE_SIZE);static_assert (sizeof (struct page_table ) ==PAGE_SIZE ); +struct dat_walk ; + +typedef long (*dat_walk_op )(union crste *crste,gfn_t gfn,gfn_t next,struct dat_walk *w); + +struct dat_walk_ops { +union { +dat_walk_op crste_ops[4 ]; +struct { +dat_walk_op pmd_entry; +dat_walk_op pud_entry; +dat_walk_op p4d_entry; +dat_walk_op pgd_entry; + }; + }; +long (*pte_entry)(union pte *pte,gfn_t gfn,gfn_t next,struct dat_walk *w); +}; + +struct dat_walk { +const struct dat_walk_ops *ops; +union crste *last; +union pte *last_pte; +union asce asce; +gfn_t start; +gfn_t end; +int flags; +void *priv; +}; +/** * _pte() - Useful constructor for union pte * @pfn: the pfn this pte should point to.
@@ -436,6 +465,11 @@ bool dat_crstep_xchg_atomic(union crste *crstep, union crste old, union crste ne union asce asce);void dat_crstep_xchg (union crste *crstep,union crste new,gfn_t gfn,union asce asce); +long _dat_walk_gfn_range (gfn_t start,gfn_t end,union asce asce, +const struct dat_walk_ops *ops,int flags,void *priv); + +int dat_entry_walk (struct kvm_s390_mmu_cache *mc,gfn_t gfn,union asce asce,int flags, +int walk_level,union crste **last,union pte **ptepp);void dat_free_level (struct crst_table *table,bool owns_ptes);struct crst_table *dat_alloc_crst_sleepable (unsigned long init);
@@ -834,4 +868,9 @@ static inline void dat_crstep_clear(union crste *crstep, gfn_t gfn, union asce a dat_crstep_xchg(crstep, newcrste, gfn, asce); } +static inline int get_level (union crste *crstep,union pte *ptep) +{ +return ptep ?TABLE_TYPE_PAGE_TABLE : crstep->h.tt; +} +#endif /* __KVM_S390_DAT_H */
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 16/28 KVM: s390: KVM page table management functions: storage keys
In reply to Claudio ImbrendaAdd page table management functions to be used for KVM guest (gmap) page tables.
This patch adds functions related to storage key handling.
--- arch/s390/kvm/dat.c | 215 ++++++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/dat.h | 7 ++ 2 files changed, 222 insertions(+)
arch/s390/kvm/dat.c +215 -0
@@ -605,3 +605,218 @@ long _dat_walk_gfn_range(gfn_t start, gfn_t end, union asce asce,return dat_crste_walk_range (start,min (end,asce_end (asce)), table, &walk); } + +int dat_get_storage_key (union asce asce,gfn_t gfn,union skey *skey) +{ +union crste *crstep; +union pgste pgste; +union pte *ptep; +int rc; + + skey->skey =0 ; + rc =dat_entry_walk (NULL , gfn, asce,DAT_WALK_ANY ,TABLE_TYPE_PAGE_TABLE , &crstep, &ptep); +if (rc) +return rc; + +if (!ptep) { +union crste crste; + + crste =READ_ONCE (*crstep); +if (!crste.h.fc || !crste.s.fc1.pr) +return 0 ; + skey->skey =page_get_storage_key (large_crste_to_phys (crste, gfn)); +return 0 ; + } + pgste =pgste_get_lock (ptep); +if (ptep->h.i) { + skey->acc = pgste.acc; + skey->fp = pgste.fp; + }else { + skey->skey =page_get_storage_key (pte_origin (*ptep)); + } + skey->r |= pgste.gr; + skey->c |= pgste.gc; +pgste_set_unlock (ptep, pgste); +return 0 ; +} + +static void dat_update_ptep_sd (union pgste old,union pgste pgste,union pte *ptep) +{ +if (pgste.acc != old.acc || pgste.fp != old.fp || pgste.gr != old.gr || pgste.gc != old.gc) +__atomic64_or (_PAGE_SD, &ptep->val); +} + +int dat_set_storage_key (struct kvm_s390_mmu_cache *mc,union asce asce,gfn_t gfn, +union skey skey,bool nq) +{ +union pgste pgste, old; +union crste *crstep; +union pte *ptep; +int rc; + + rc =dat_entry_walk (mc, gfn, asce,DAT_WALK_LEAF_ALLOC ,TABLE_TYPE_PAGE_TABLE , + &crstep, &ptep); +if (rc) +return rc; + +if (!ptep) { +page_set_storage_key (large_crste_to_phys (*crstep, gfn), skey.skey, !nq); +return 0 ; + } + + old =pgste_get_lock (ptep); + pgste = old; + + pgste.acc = skey.acc; + pgste.fp = skey.fp; + pgste.gc = skey.c; + pgste.gr = skey.r; + +if (!ptep->h.i) { +union skey old_skey; + + old_skey.skey =page_get_storage_key (pte_origin (*ptep)); + pgste.hc |= old_skey.c; + pgste.hr |= old_skey.r; + skey.r =0 ; + skey.c =0 ; +page_set_storage_key (pte_origin (*ptep), skey.skey, !nq); + } + +dat_update_ptep_sd (old, pgste, ptep); +pgste_set_unlock (ptep, pgste); +return 0 ; +} + +static bool page_cond_set_storage_key (phys_addr_t paddr,union skey skey,union skey *oldkey, +bool nq,bool mr,bool mc) +{ + oldkey->skey =page_get_storage_key (paddr); +if (oldkey->acc == skey.acc && oldkey->fp == skey.fp && + (oldkey->r == skey.r || mr) && (oldkey->c == skey.c || mc)) +return false; +page_set_storage_key (paddr, skey.skey, !nq); +return true; +} + +int dat_cond_set_storage_key (struct kvm_s390_mmu_cache *mmc,union asce asce,gfn_t gfn, +union skey skey,union skey *oldkey,bool nq,bool mr,bool mc) +{ +union pgste pgste, old; +union crste *crstep; +union pte *ptep; +int rc; + + rc =dat_entry_walk (mmc, gfn, asce,DAT_WALK_LEAF_ALLOC ,TABLE_TYPE_PAGE_TABLE , + &crstep, &ptep); +if (rc) +return rc; + +if (!ptep) +return page_cond_set_storage_key (large_crste_to_phys (*crstep, gfn), skey, oldkey, + nq, mr, mc); + + old =pgste_get_lock (ptep); + pgste = old; + + rc =1 ; + pgste.acc = skey.acc; + pgste.fp = skey.fp; + pgste.gc = skey.c; + pgste.gr = skey.r; + +if (!ptep->h.i) { +union skey prev; + + rc =page_cond_set_storage_key (pte_origin (*ptep), skey, &prev, nq, mr, mc); + pgste.hc |= prev.c; + pgste.hr |= prev.r; +if (oldkey) + *oldkey = prev; + } + +dat_update_ptep_sd (old, pgste, ptep); +pgste_set_unlock (ptep, pgste); +return rc; +} + +int dat_reset_reference_bit (union asce asce,gfn_t gfn) +{ +union pgste pgste, old; +union crste *crstep; +union pte *ptep; +int rc; + + rc =dat_entry_walk (NULL , gfn, asce,DAT_WALK_ANY ,TABLE_TYPE_PAGE_TABLE , &crstep, &ptep); +if (rc) +return rc; + +if (!ptep) { +union crste crste =READ_ONCE (*crstep); + +if (!crste.h.fc || !crste.s.fc1.pr) +return 0 ; +return page_reset_referenced (large_crste_to_phys (*crstep, gfn)); + } + old =pgste_get_lock (ptep); + pgste = old; + +if (!ptep->h.i) { + rc =page_reset_referenced (pte_origin (*ptep)); + pgste.hr = rc >>1 ; + } + rc |= (pgste.gr <<1 ) | pgste.gc; + pgste.gr =0 ; + +dat_update_ptep_sd (old, pgste, ptep); +pgste_set_unlock (ptep, pgste); +return rc; +} + +static long dat_reset_skeys_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +union pgste pgste; + + pgste =pgste_get_lock (ptep); + pgste.acc =0 ; + pgste.fp =0 ; + pgste.gr =0 ; + pgste.gc =0 ; +if (ptep->s.pr) +page_set_storage_key (pte_origin (*ptep),PAGE_DEFAULT_KEY ,1 ); +pgste_set_unlock (ptep, pgste); + +if (need_resched ()) +return next; +return 0 ; +} + +static long dat_reset_skeys_crste (union crste *crstep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +phys_addr_t addr, end, origin =crste_origin_large (*crstep); + +if (!crstep->h.fc || !crstep->s.fc1.pr) +return 0 ; + + addr = ((max (gfn, walk->start) - gfn) <<PAGE_SHIFT ) + origin; + end = ((min (next, walk->end) - gfn) <<PAGE_SHIFT ) + origin; +while (ALIGN (addr +1 , _SEGMENT_SIZE) <= end) + addr =sske_frame (addr,PAGE_DEFAULT_KEY ); +for ( ; addr < end; addr +=PAGE_SIZE ) +page_set_storage_key (addr,PAGE_DEFAULT_KEY ,1 ); + +if (need_resched ()) +return next; +return 0 ; +} + +long dat_reset_skeys (union asce asce,gfn_t start) +{ +const struct dat_walk_ops ops = { + .pte_entry = dat_reset_skeys_pte, + .pmd_entry = dat_reset_skeys_crste, + .pud_entry = dat_reset_skeys_crste, + }; + +return _dat_walk_gfn_range (start,asce_end (asce), asce, &ops,DAT_WALK_IGN_HOLES ,NULL ); +}
arch/s390/kvm/dat.h +7 -0
@@ -472,6 +472,13 @@ int dat_entry_walk(struct kvm_s390_mmu_cache *mc, gfn_t gfn, union asce asce, in int walk_level, union crste **last, union pte **ptepp);void dat_free_level (struct crst_table *table,bool owns_ptes);struct crst_table *dat_alloc_crst_sleepable (unsigned long init); +int dat_get_storage_key (union asce asce,gfn_t gfn,union skey *skey); +int dat_set_storage_key (struct kvm_s390_mmu_cache *mc,union asce asce,gfn_t gfn, +union skey skey,bool nq); +int dat_cond_set_storage_key (struct kvm_s390_mmu_cache *mmc,union asce asce,gfn_t gfn, +union skey skey,union skey *oldkey,bool nq,bool mr,bool mc); +int dat_reset_reference_bit (union asce asce,gfn_t gfn); +long dat_reset_skeys (union asce asce,gfn_t start);int kvm_s390_mmu_cache_topup (struct kvm_s390_mmu_cache *mc);
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 17/28 KVM: s390: KVM page table management functions: lifecycle management
In reply to Claudio ImbrendaAdd page table management functions to be used for KVM guest (gmap) page tables.
This patch adds functions to handle memslot creation and destruction, additional per-pagetable data stored in the PGSTEs, mapping physical addresses into the gmap, and marking address ranges as prefix.
--- arch/s390/kvm/dat.c | 284 ++++++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/dat.h | 54 +++++++++ 2 files changed, 338 insertions(+)
arch/s390/kvm/dat.c +284 -0
@@ -102,6 +102,38 @@ void dat_free_level(struct crst_table *table, bool owns_ptes)dat_free_crst (table); } +int dat_set_asce_limit (struct kvm_s390_mmu_cache *mc,union asce *asce,int newtype) +{ +struct crst_table *table; +union crste crste; + +while (asce->dt > newtype) { + table =dereference_asce (*asce); + crste = table->crstes[0 ]; +if (crste.h.fc) +return 0 ; +if (!crste.h.i) { + asce->rsto = crste.h.fc0.to; +dat_free_crst (table); + }else { + crste.h.tt--; +crst_table_init ((void *)table, crste.val); + } + asce->dt--; + } +while (asce->dt < newtype) { + crste =_crste_fc0 (asce->rsto, asce->dt +1 ); + table =dat_alloc_crst_noinit (mc); +if (!table) +return -ENOMEM ; +crst_table_init ((void *)table,_CRSTE_HOLE (crste.h.tt).val); + table->crstes[0 ] = crste; + asce->rsto =__pa (table) >>PAGE_SHIFT ; + asce->dt++; + } +return 0 ; +} +/** * dat_crstep_xchg - exchange a gmap CRSTE with another * @crstep: pointer to the CRST entry
@@ -820,3 +852,255 @@ long dat_reset_skeys(union asce asce, gfn_t start)return _dat_walk_gfn_range (start,asce_end (asce), asce, &ops,DAT_WALK_IGN_HOLES ,NULL ); } + +struct slot_priv { +unsigned long token; +struct kvm_s390_mmu_cache *mc; +}; + +static long _dat_slot_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +struct slot_priv *p = walk->priv; +union crste dummy = { .val = p->token }; +union pte new_pte, pte =READ_ONCE (*ptep); + + new_pte =_PTE_TOK (dummy.tok.type, dummy.tok.par); + +/* Table entry already in the desired state */ +if (pte.val == new_pte.val) +return 0 ; + +dat_ptep_xchg (ptep, new_pte, gfn, walk->asce, false); +return 0 ; +} + +static long _dat_slot_crste (union crste *crstep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +union crste new_crste, crste =READ_ONCE (*crstep); +struct slot_priv *p = walk->priv; + + new_crste.val = p->token; + new_crste.h.tt = crste.h.tt; + +/* Table entry already in the desired state */ +if (crste.val == new_crste.val) +return 0 ; + +/* This table entry needs to be updated */ +if (walk->start <= gfn && walk->end >= next) { +dat_crstep_xchg_atomic (crstep, crste, new_crste, gfn, walk->asce); +/* A lower level table was present, needs to be freed */ +if (!crste.h.fc && !crste.h.i) +dat_free_level (dereference_crste (crste), true); +return 0 ; + } + +/* A lower level table is present, things will handled there */ +if (!crste.h.fc && !crste.h.i) +return 0 ; +/* Split (install a lower level table), and handle things there */ +return dat_split_crste (p->mc, crstep, gfn, walk->asce, false); +} + +static const struct dat_walk_ops dat_slot_ops = { + .pte_entry = _dat_slot_pte, + .crste_ops = { _dat_slot_crste, _dat_slot_crste, _dat_slot_crste, _dat_slot_crste, }, +}; + +int dat_set_slot (struct kvm_s390_mmu_cache *mc,union asce asce,gfn_t start,gfn_t end, +u16 type,u16 param) +{ +struct slot_priv priv = { + .token =_CRSTE_TOK (0 , type, param).val, + .mc = mc, + }; + +return _dat_walk_gfn_range (start, end, asce, &dat_slot_ops, +DAT_WALK_IGN_HOLES |DAT_WALK_ANY , &priv); +} + +static void pgste_set_unlock_multiple (union pte *first,int n,union pgste *pgstes) +{ +int i; + +for (i =0 ; i < n; i++) { +if (!pgstes[i].pcl) +break ; +pgste_set_unlock (first + i, pgstes[i]); + } +} + +static bool pgste_get_trylock_multiple (union pte *first,int n,union pgste *pgstes) +{ +int i; + +for (i =0 ; i < n; i++) { +if (!pgste_get_trylock (first + i, pgstes + i)) +break ; + } +if (i == n) +return true; +pgste_set_unlock_multiple (first, n, pgstes); +return false; +} + +unsigned long dat_get_ptval (struct page_table *table,struct ptval_param param) +{ +union pgste pgstes[4 ] = {}; +unsigned long res =0 ; +int i, n; + + n = param.len +1 ; + +while (!pgste_get_trylock_multiple (table->ptes + param.offset, n, pgstes)) +cpu_relax (); + +for (i =0 ; i < n; i++) + res = res <<16 | pgstes[i].val16; + +pgste_set_unlock_multiple (table->ptes + param.offset, n, pgstes); +return res; +} + +void dat_set_ptval (struct page_table *table,struct ptval_param param,unsigned long val) +{ +union pgste pgstes[4 ] = {}; +int i, n; + + n = param.len +1 ; + +while (!pgste_get_trylock_multiple (table->ptes + param.offset, n, pgstes)) +cpu_relax (); + +for (i = param.len; i >=0 ; i--) { + pgstes[i].val16 = val; + val = val >>16 ; + } + +pgste_set_unlock_multiple (table->ptes + param.offset, n, pgstes); +} + +static long _dat_test_young_pte (union pte *ptep,gfn_t start,gfn_t end,struct dat_walk *walk) +{ +return ptep->s.y; +} + +static long _dat_test_young_crste (union crste *crstep,gfn_t start,gfn_t end, +struct dat_walk *walk) +{ +return crstep->h.fc && crstep->s.fc1.y; +} + +static const struct dat_walk_ops test_age_ops = { + .pte_entry = _dat_test_young_pte, + .pmd_entry = _dat_test_young_crste, + .pud_entry = _dat_test_young_crste, +}; + +/** +* dat_test_age_gfn() - test young +* @asce: the ASCE whose address range is to be tested +* @start: the first guest frame of the range to check +* @end: the guest frame after the last in the range +* +* Context: called by KVM common code with the kvm mmu write lock held +* Return: 1 if any page in the given range is young, otherwise 0. +*/ +bool dat_test_age_gfn (union asce asce,gfn_t start,gfn_t end) +{ +return _dat_walk_gfn_range (start, end, asce, &test_age_ops,0 ,NULL ) >0 ; +} + +int dat_link (struct kvm_s390_mmu_cache *mc,union asce asce,int level, +bool uses_skeys,struct guest_fault *f) +{ +union crste oldval, newval; +union pte newpte, oldpte; +union pgste pgste; +int rc =0 ; + + rc =dat_entry_walk (mc, f->gfn, asce,DAT_WALK_ALLOC_CONTINUE , level, &f->crstep, &f->ptep); +if (rc == -EINVAL || rc == -ENOMEM ) +return rc; +if (rc) +return -EAGAIN ; + +if (WARN_ON_ONCE (unlikely (get_level (f->crstep, f->ptep) > level))) +return -EINVAL ; + +if (f->ptep) { + pgste =pgste_get_lock (f->ptep); + oldpte = *f->ptep; + newpte =_pte (f->pfn, f->writable, f->write_attempt | oldpte.s.d, !f->page); + newpte.s.sd = oldpte.s.sd; + oldpte.s.sd =0 ; +if (oldpte.val == _PTE_EMPTY.val || oldpte.h.pfra == f->pfn) { + pgste =__dat_ptep_xchg (f->ptep, pgste, newpte, f->gfn, asce, uses_skeys); +if (f->callback) + f->callback (f); + }else { + rc = -EAGAIN ; + } +pgste_set_unlock (f->ptep, pgste); + }else { + oldval =READ_ONCE (*f->crstep); + newval =_crste_fc1 (f->pfn, oldval.h.tt, f->writable, + f->write_attempt | oldval.s.fc1.d); + newval.s.fc1.sd = oldval.s.fc1.sd; +if (oldval.val !=_CRSTE_EMPTY (oldval.h.tt).val && +crste_origin_large (oldval) !=crste_origin_large (newval)) +return -EAGAIN ; +if (!dat_crstep_xchg_atomic (f->crstep, oldval, newval, f->gfn, asce)) +return -EAGAIN ; +if (f->callback) + f->callback (f); + } + +return rc; +} + +static long dat_set_pn_crste (union crste *crstep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +union crste crste =READ_ONCE (*crstep); +int *n = walk->priv; + +if (!crste.h.fc || crste.h.i || crste.h.p) +return 0 ; + + *n =2 ; +if (crste.s.fc1.prefix_notif) +return 0 ; + crste.s.fc1.prefix_notif =1 ; +dat_crstep_xchg (crstep, crste, gfn, walk->asce); +return 0 ; +} + +static long dat_set_pn_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +int *n = walk->priv; +union pgste pgste; + + pgste =pgste_get_lock (ptep); +if (!ptep->h.i && !ptep->h.p) { + pgste.prefix_notif =1 ; + *n +=1 ; + } +pgste_set_unlock (ptep, pgste); +return 0 ; +} + +int dat_set_prefix_notif_bit (union asce asce,gfn_t gfn) +{ +static const struct dat_walk_ops ops = { + .pte_entry = dat_set_pn_pte, + .pmd_entry = dat_set_pn_crste, + .pud_entry = dat_set_pn_crste, + }; + +int n =0 ; + +_dat_walk_gfn_range (gfn, gfn +2 , asce, &ops,DAT_WALK_IGN_HOLES , &n); +if (n !=2 ) +return -EAGAIN ; +return 0 ; +}
arch/s390/kvm/dat.h +54 -0
82 lines not on this page · raw message
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 18/28 KVM: s390: KVM page table management functions: CMMA
In reply to Claudio ImbrendaAdd page table management functions to be used for KVM guest (gmap) page tables.
This patch adds functions to handle CMMA and the ESSA instruction.
--- arch/s390/kvm/dat.c | 262 ++++++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/dat.h | 27 +++++ 2 files changed, 289 insertions(+)
arch/s390/kvm/dat.c +262 -0
@@ -1104,3 +1104,265 @@ int dat_set_prefix_notif_bit(union asce asce, gfn_t gfn)return -EAGAIN ;return 0 ; } + +/** +* dat_perform_essa() - perform ESSA actions on the PGSTE. +* @asce: the asce to operate on. +* @gfn: the guest page frame to operate on. +* @orc: the specific action to perform, see the ESSA_SET_* macros. +* @state: the storage attributes to be returned to the guest. +* @dirty: returns whether the function dirtied a previously clean entry. +* +* Context: Called with kvm->mmu_lock held. +* +* Return: +* * 1 if the page state has been altered and the page is to be added to the CBRL +* * 0 if the page state has been altered, but the page is not to be added to the CBRL +* * -1 if the page state has not been altered and the page is not to be added to the CBRL +*/ +int dat_perform_essa (union asce asce,gfn_t gfn,int orc,union essa_state *state,bool *dirty) +{ +union crste *crstep; +union pgste pgste; +union pte *ptep; +int res =0 ; + +if (dat_entry_walk (NULL , gfn, asce,0 ,TABLE_TYPE_PAGE_TABLE , &crstep, &ptep)) { + *state = (union essa_state ) { .exception =1 }; +return -1 ; + } + + pgste =pgste_get_lock (ptep); + + *state = (union essa_state ) { + .content = (ptep->h.i <<1 ) + (ptep->h.i && pgste.zero), + .nodat = pgste.nodat, + .usage = pgste.usage, + }; + +switch (orc) { +case ESSA_GET_STATE : + res =-1 ; +break ; +case ESSA_SET_STABLE : + pgste.usage =PGSTE_GPS_USAGE_STABLE ; + pgste.nodat =0 ; +break ; +case ESSA_SET_UNUSED : + pgste.usage =PGSTE_GPS_USAGE_UNUSED ; +if (ptep->h.i) + res =1 ; +break ; +case ESSA_SET_VOLATILE : + pgste.usage =PGSTE_GPS_USAGE_VOLATILE ; +if (ptep->h.i) + res =1 ; +break ; +case ESSA_SET_POT_VOLATILE : +if (!ptep->h.i) { + pgste.usage =PGSTE_GPS_USAGE_POT_VOLATILE ; + }else if (pgste.zero) { + pgste.usage =PGSTE_GPS_USAGE_VOLATILE ; + }else if (!pgste.gc) { + pgste.usage =PGSTE_GPS_USAGE_VOLATILE ; + res =1 ; + } +break ; +case ESSA_SET_STABLE_RESIDENT : + pgste.usage =PGSTE_GPS_USAGE_STABLE ; +/* +* Since the resident state can go away any time after this +* call, we will not make this page resident. We can revisit +* this decision if a guest will ever start using this. +*/ +break ; +case ESSA_SET_STABLE_IF_RESIDENT : +if (!ptep->h.i) + pgste.usage =PGSTE_GPS_USAGE_STABLE ; +break ; +case ESSA_SET_STABLE_NODAT : + pgste.usage =PGSTE_GPS_USAGE_STABLE ; + pgste.nodat =1 ; +break ; +default : +WARN_ONCE (1 ,"Invalid ORC!" ); + res =-1 ; +break ; + } +/* If we are discarding a page, set it to logical zero */ + pgste.zero = res ==1 ; +if (orc >0 ) { + *dirty = !pgste.cmma_d; + pgste.cmma_d =1 ; + } + +pgste_set_unlock (ptep, pgste); + +return res; +} + +static long dat_reset_cmma_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +union pgste pgste; + + pgste =pgste_get_lock (ptep); + pgste.usage =0 ; + pgste.nodat =0 ; + pgste.cmma_d =0 ; +pgste_set_unlock (ptep, pgste); +if (need_resched ()) +return next; +return 0 ; +} + +long dat_reset_cmma (union asce asce,gfn_t start) +{ +const struct dat_walk_ops dat_reset_cmma_ops = { + .pte_entry = dat_reset_cmma_pte, + }; + +return _dat_walk_gfn_range (start,asce_end (asce), asce, &dat_reset_cmma_ops, +DAT_WALK_IGN_HOLES ,NULL ); +} + +struct dat_get_cmma_state { +gfn_t start; +gfn_t end; +unsigned int count; +u8 *values; +atomic64_t *remaining; +}; + +static long __dat_peek_cmma_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +struct dat_get_cmma_state *state = walk->priv; +union pgste pgste; + + pgste =pgste_get_lock (ptep); + state->values[gfn - walk->start] = pgste.usage | (pgste.nodat <<6 ); +pgste_set_unlock (ptep, pgste); + state->end = next; + +return 0 ; +} + +static long __dat_peek_cmma_crste (union crste *crstep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +struct dat_get_cmma_state *state = walk->priv; + +if (crstep->h.i) + state->end =min (walk->end, next); +return 0 ; +} + +int dat_peek_cmma (gfn_t start,union asce asce,unsigned int *count,u8 *values) +{ +const struct dat_walk_ops ops = { + .pte_entry = __dat_peek_cmma_pte, + .pmd_entry = __dat_peek_cmma_crste, + .pud_entry = __dat_peek_cmma_crste, + .p4d_entry = __dat_peek_cmma_crste, + .pgd_entry = __dat_peek_cmma_crste, + }; +struct dat_get_cmma_state state = { .values = values, }; +int rc; + + rc =_dat_walk_gfn_range (start, start + *count, asce, &ops,DAT_WALK_DEFAULT , &state); + *count = state.end - start; +/* Return success if at least one value was saved, otherwise an error. */ +return (rc == -EFAULT && *count >0 ) ?0 : rc; +} + +static long __dat_get_cmma_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +struct dat_get_cmma_state *state = walk->priv; +union pgste pgste; + +if (state->start !=-1 ) { +if ((gfn - state->end) >KVM_S390_MAX_BIT_DISTANCE ) +return 1 ; +if (gfn - state->start >= state->count) +return 1 ; + } + +if (!READ_ONCE (*pgste_of (ptep)).cmma_d) +return 0 ; + + pgste =pgste_get_lock (ptep); +if (pgste.cmma_d) { +if (state->start ==-1 ) + state->start = gfn; + pgste.cmma_d =0 ; +atomic64_dec (state->remaining); + state->values[gfn - state->start] = pgste.usage | pgste.nodat <<6 ; + state->end = next; + } +pgste_set_unlock (ptep, pgste); +return 0 ; +} + +int dat_get_cmma (union asce asce,gfn_t *start,unsigned int *count,u8 *values,atomic64_t *rem) +{ +const struct dat_walk_ops ops = { .pte_entry = __dat_get_cmma_pte, }; +struct dat_get_cmma_state state = { + .remaining = rem, + .values = values, + .count = *count, + .start =-1 , + }; + +_dat_walk_gfn_range (*start,asce_end (asce), asce, &ops,DAT_WALK_IGN_HOLES , &state); + +if (state.start ==-1 ) { + *count =0 ; + }else { + *count = state.end - state.start; + *start = state.start; + } + +return 0 ; +} + +struct dat_set_cmma_state { +unsigned long mask; +const u8 *bits; +}; + +static long __dat_set_cmma_pte (union pte *ptep,gfn_t gfn,gfn_t next,struct dat_walk *walk) +{ +struct dat_set_cmma_state *state = walk->priv; +union pgste pgste, tmp; + + tmp.val = (state->bits[gfn - walk->start] <<24 ) & state->mask; + + pgste =pgste_get_lock (ptep); + pgste.usage = tmp.usage; + pgste.nodat = tmp.nodat; +pgste_set_unlock (ptep, pgste); + +return 0 ; +} + +/* +* This function sets the CMMA attributes for the given pages. If the input +* buffer has zero length, no action is taken, otherwise the attributes are +* set and the mm->context.uses_cmm flag is set. +*/ +int dat_set_cmma_bits (struct kvm_s390_mmu_cache *mc,union asce asce,gfn_t gfn, +unsigned long count,unsigned long mask,const uint8_t *bits) +{ +const struct dat_walk_ops ops = { .pte_entry = __dat_set_cmma_pte, }; +struct dat_set_cmma_state state = { .mask = mask, .bits = bits, }; +union crste *crstep; +union pte *ptep; +gfn_t cur; +int rc; + +for (cur =ALIGN_DOWN (gfn, _PAGE_ENTRIES); cur < gfn + count; cur += _PAGE_ENTRIES) { + rc =dat_entry_walk (mc, cur, asce,DAT_WALK_ALLOC ,TABLE_TYPE_PAGE_TABLE , + &crstep, &ptep); +if (rc) +return rc; + } +return _dat_walk_gfn_range (gfn, gfn + count, asce, &ops,DAT_WALK_IGN_HOLES , &state); +}
arch/s390/kvm/dat.h +27 -0
@@ -17,6 +17,15 @@#include <asm/tlbflush.h> #include <asm/dat-bits.h> +/* +* Base address and length must be sent at the start of each block, therefore +* it's cheaper to send some clean data, as long as it's less than the size of +* two longs. +*/ +#define KVM_S390_MAX_BIT_DISTANCE (2 * sizeof(void *)) +/* For consistency */ +#define KVM_S390_CMMA_SIZE_MAX ((u32)KVM_S390_SKEYS_MAX) +#define _ASCE (x) ((union asce) { .val = (x), })#define NULL_ASCE _ASCE(0)
@@ -433,6 +442,17 @@ static inline union crste _crste_fc1(kvm_pfn_t pfn, int tt, bool writable, boolreturn res; } +union essa_state { +unsigned char val; +struct { + unsigned char : 2; +unsigned char nodat :1 ; +unsigned char exception :1 ; +unsigned char usage :2 ; +unsigned char content :2 ; + }; +}; +/** * struct vsie_rmap - reverse mapping for shadow page table entries * @next: pointer to next rmap in the list
@@ -522,6 +542,13 @@ bool dat_test_age_gfn(union asce asce, gfn_t start, gfn_t end);int dat_link (struct kvm_s390_mmu_cache *mc,union asce asce,int level,bool uses_skeys,struct guest_fault *f); +int dat_perform_essa (union asce asce,gfn_t gfn,int orc,union essa_state *state,bool *dirty); +long dat_reset_cmma (union asce asce,gfn_t start_gfn); +int dat_peek_cmma (gfn_t start,union asce asce,unsigned int *count,u8 *values); +int dat_get_cmma (union asce asce,gfn_t *start,unsigned int *count,u8 *values,atomic64_t *rem); +int dat_set_cmma_bits (struct kvm_s390_mmu_cache *mc,union asce asce,gfn_t gfn, +unsigned long count,unsigned long mask,const uint8_t *bits); +int kvm_s390_mmu_cache_topup (struct kvm_s390_mmu_cache *mc);#define GFP_KVM_S390_MMU_CACHE (GFP_ATOMIC | __GFP_ACCOUNT | __GFP_NOWARN)
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 19/28 KVM: s390: New gmap code
In reply to Claudio ImbrendaNew gmap (guest map) code. This new gmap code will only be used by KVM.
This will replace the existing gmap.
--- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/gmap.c | 1083 ++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/gmap.h | 240 +++++++++ 3 files changed, 1324 insertions(+), 1 deletion(-) create mode 100644 arch/s390/kvm/gmap.c create mode 100644 arch/s390/kvm/gmap.h
arch/s390/kvm/Makefile +1 -1
@@ -9,7 +9,7 @@ ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap-vsie.o -kvm-y += dat.o +kvm-y += dat.o gmap.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/gmap.c new file +1083 -0
1,083 lines not on this page · raw message
arch/s390/kvm/gmap.h new file +240 -0
240 lines not on this page · raw message
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 20/28 KVM: s390: Add helper functions for fault handling
In reply to Claudio ImbrendaAdd some helper functions for handling multiple guest faults at the same time.
This will be needed for VSIE, where a nested guest access also needs to access all the page tables that map it.
--- arch/s390/include/asm/kvm_host.h | 1 + arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/faultin.c | 148 +++++++++++++++++++++++++++++++ arch/s390/kvm/faultin.h | 92 +++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 2 +- arch/s390/kvm/kvm-s390.h | 2 + 6 files changed, 245 insertions(+), 2 deletions(-) create mode 100644 arch/s390/kvm/faultin.c create mode 100644 arch/s390/kvm/faultin.h
arch/s390/include/asm/kvm_host.h +1 -0
@@ -442,6 +442,7 @@ struct kvm_vcpu_arch {bool acrs_loaded;struct kvm_s390_pv_vcpu pv;union diag318_info diag318_info; +void *mc;/* Placeholder */ };struct kvm_vm_stat {
arch/s390/kvm/Makefile +1 -1
@@ -9,7 +9,7 @@ ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap-vsie.o -kvm-y += dat.o gmap.o +kvm-y += dat.o gmap.o faultin.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o obj-$(CONFIG_KVM) += kvm.o
arch/s390/kvm/faultin.c new file +148 -0
148 lines not on this page · raw message
arch/s390/kvm/faultin.h new file +92 -0
92 lines not on this page · raw message
arch/s390/kvm/kvm-s390.c +1 -1
@@ -4637,7 +4637,7 @@ bool kvm_arch_can_dequeue_async_page_present(struct kvm_vcpu *vcpu)return true; } -static bool kvm_arch_setup_async_pf (struct kvm_vcpu *vcpu) +bool kvm_arch_setup_async_pf (struct kvm_vcpu *vcpu) {hva_t hva;struct kvm_arch_async_pf arch;
arch/s390/kvm/kvm-s390.h +2 -0
@@ -470,6 +470,8 @@ static inline int kvm_s390_handle_dat_fault(struct kvm_vcpu *vcpu, gpa_t gaddr,return __kvm_s390_handle_dat_fault (vcpu,gpa_to_gfn (gaddr), gaddr, flags); } +bool kvm_arch_setup_async_pf (struct kvm_vcpu *vcpu); +/* implemented in diag.c */ int kvm_s390_handle_diag (struct kvm_vcpu *vcpu);
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 21/28 KVM: s390: Add some helper functions needed for vSIE
In reply to Claudio ImbrendaImplement gmap_protect_asce_top_level(), which was a stub. This function was a stub due to cross dependencies with other patches.
--- arch/s390/kvm/gmap.c | 72 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 70 insertions(+), 2 deletions(-)
arch/s390/kvm/gmap.c +70 -2
@@ -22,6 +22,7 @@#include "dat.h" #include "gmap.h" #include "kvm-s390.h" +#include "faultin.h" static inline bool kvm_s390_is_in_sie (struct kvm_vcpu *vcpu) {
@@ -1009,10 +1010,77 @@ static struct gmap *gmap_find_shadow(struct gmap *parent, union asce asce, int ereturn NULL ; } +#define CRST_TABLE_PAGES (_CRST_TABLE_SIZE / PAGE_SIZE) +struct gmap_protect_asce_top_level { +unsigned long seq; +struct guest_fault f[CRST_TABLE_PAGES ]; +}; + +static inline int __gmap_protect_asce_top_level (struct kvm_s390_mmu_cache *mc,struct gmap *sg, +struct gmap_protect_asce_top_level *context) +{ +int rc, i; + +guard (write_lock)(&sg->kvm->mmu_lock); + +if (kvm_s390_array_needs_retry_safe (sg->kvm, context->seq, context->f)) +return -EAGAIN ; + + scoped_guard(spinlock, &sg->parent->children_lock) { +for (i =0 ; i <CRST_TABLE_PAGES ; i++) { + rc =gmap_protect_rmap (mc, sg, context->f[i].gfn,0 , context->f[i].pfn, +TABLE_TYPE_REGION1 +1 , context->f[i].writable); +if (rc) +return rc; + } +gmap_add_child (sg->parent, sg); + } + +kvm_s390_release_faultin_array (sg->kvm, context->f, false); +return 0 ; +} + +static inline int _gmap_protect_asce_top_level (struct kvm_s390_mmu_cache *mc,struct gmap *sg, +struct gmap_protect_asce_top_level *context) +{ +int rc; + +if (kvm_s390_array_needs_retry_unsafe (sg->kvm, context->seq, context->f)) +return -EAGAIN ; +do { + rc =kvm_s390_mmu_cache_topup (mc); +if (rc) +return rc; + rc =radix_tree_preload (GFP_KERNEL ); +if (rc) +return rc; + rc =__gmap_protect_asce_top_level (mc, sg, context); +radix_tree_preload_end (); + }while (rc == -ENOMEM ); + +return rc; +} +static int gmap_protect_asce_top_level (struct kvm_s390_mmu_cache *mc,struct gmap *sg) { -KVM_BUG_ON (1 , sg->kvm); -return -EINVAL ; +struct gmap_protect_asce_top_level context = {}; +union asce asce = sg->guest_asce; +int rc; + +KVM_BUG_ON (!is_shadow (sg), sg->kvm); + + context.seq = sg->kvm->mmu_invalidate_seq; +/* Pairs with the smp_wmb() in kvm_mmu_invalidate_end(). */ +smp_rmb (); + + rc =kvm_s390_get_guest_pages (sg->kvm, context.f, asce.rsto, asce.dt +1 , false); +if (rc >0 ) + rc = -EFAULT ; +if (!rc) + rc =_gmap_protect_asce_top_level (mc, sg, &context); +if (rc) +kvm_s390_release_faultin_array (sg->kvm, context.f, true); +return rc; }/**
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 22/28 KVM: s390: Stop using CONFIG_PGSTE
In reply to Claudio ImbrendaSwitch to using IS_ENABLED(CONFIG_KVM) instead of CONFIG_PGSTE, since the latter will be removed soon.
Many CONFIG_PGSTE are left behind, because they will be removed completely in upcoming patches. The ones replaced here are mostly the ones that will stay.
--- arch/s390/include/asm/mmu_context.h | 2 +- arch/s390/include/asm/pgtable.h | 4 ++-- arch/s390/mm/fault.c | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-)
arch/s390/include/asm/mmu_context.h +1 -1
@@ -29,7 +29,7 @@ static inline int init_new_context(struct task_struct *tsk,atomic_set (&mm->context.protected_count,0 ); mm->context.gmap_asce =0 ; mm->context.flush_mm =0 ; -#ifdef CONFIG_PGSTE +#if IS_ENABLED (CONFIG_KVM ) mm->context.has_pgste =0 ; mm->context.uses_skeys =0 ; mm->context.uses_cmm =0 ;
arch/s390/include/asm/pgtable.h +2 -2
@@ -577,7 +577,7 @@ static inline int mm_has_pgste(struct mm_struct *mm)static inline int mm_is_protected (struct mm_struct *mm) { -#ifdef CONFIG_PGSTE +#if IS_ENABLED (CONFIG_KVM )if (unlikely (atomic_read (&mm->context.protected_count)))return 1 ;#endif
@@ -632,7 +632,7 @@ static inline pud_t set_pud_bit(pud_t pud, pgprot_t prot)#define mm_forbids_zeropage mm_forbids_zeropagestatic inline int mm_forbids_zeropage (struct mm_struct *mm) { -#ifdef CONFIG_PGSTE +#if IS_ENABLED (CONFIG_KVM )if (!mm->context.allow_cow_sharing)return 1 ;#endif
arch/s390/mm/fault.c +2 -2
@@ -403,7 +403,7 @@ void do_dat_exception(struct pt_regs *regs) }NOKPROBE_SYMBOL (do_dat_exception); -#if IS_ENABLED (CONFIG_PGSTE ) +#if IS_ENABLED (CONFIG_KVM )void do_secure_storage_access (struct pt_regs *regs) {
@@ -470,4 +470,4 @@ void do_secure_storage_access(struct pt_regs *regs) }NOKPROBE_SYMBOL (do_secure_storage_access); -#endif /* CONFIG_PGSTE */ +#endif /* CONFIG_KVM */
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 23/28 KVM: s390: Storage key functions refactoring
In reply to Claudio ImbrendaRefactor some storage key functions to improve readability.
Introduce helper functions that will be used in the next patches.
--- arch/s390/kvm/gaccess.c | 36 +++++++++--------- arch/s390/kvm/gaccess.h | 4 +- arch/s390/kvm/kvm-s390.c | 80 +++++++++++++++------------------------- arch/s390/kvm/kvm-s390.h | 8 ++++ 4 files changed, 58 insertions(+), 70 deletions(-)
arch/s390/kvm/gaccess.c +18 -18
@@ -974,9 +974,8 @@ int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,* * -EAGAIN: transient failure (len 1 or 2) * * -EOPNOTSUPP: read-only memslot (should never occur) */ -int cmpxchg_guest_abs_with_key (struct kvm *kvm,gpa_t gpa,int len, -__uint128_t *old_addr,__uint128_t new, -u8 access_key,bool *success) +int cmpxchg_guest_abs_with_key (struct kvm *kvm,gpa_t gpa,int len,union kvm_s390_quad *old_addr, +union kvm_s390_quad new,u8 acc,bool *success) {gfn_t gfn =gpa_to_gfn (gpa);struct kvm_memory_slot *slot =gfn_to_memslot (kvm, gfn);
@@ -1008,41 +1007,42 @@ int cmpxchg_guest_abs_with_key(struct kvm *kvm, gpa_t gpa, int len,case 1 : {u8 old; - ret =cmpxchg_user_key ((u8 __user *)hva, &old, *old_addr, new, access_key); - *success = !ret && old == *old_addr; - *old_addr = old; + ret =cmpxchg_user_key ((u8 __user *)hva, &old, old_addr->one, new.one, acc); + *success = !ret && old == old_addr->one; + old_addr->one = old;break ; }case 2 : {u16 old; - ret =cmpxchg_user_key ((u16 __user *)hva, &old, *old_addr, new, access_key); - *success = !ret && old == *old_addr; - *old_addr = old; + ret =cmpxchg_user_key ((u16 __user *)hva, &old, old_addr->two, new.two, acc); + *success = !ret && old == old_addr->two; + old_addr->two = old;break ; }case 4 : {u32 old; - ret =cmpxchg_user_key ((u32 __user *)hva, &old, *old_addr, new, access_key); - *success = !ret && old == *old_addr; - *old_addr = old; + ret =cmpxchg_user_key ((u32 __user *)hva, &old, old_addr->four, new.four, acc); + *success = !ret && old == old_addr->four; + old_addr->four = old;break ; }case 8 : {u64 old; - ret =cmpxchg_user_key ((u64 __user *)hva, &old, *old_addr, new, access_key); - *success = !ret && old == *old_addr; - *old_addr = old; + ret =cmpxchg_user_key ((u64 __user *)hva, &old, old_addr->eight, new.eight, acc); + *success = !ret && old == old_addr->eight; + old_addr->eight = old;break ; }case 16 : {__uint128_t old; - ret =cmpxchg_user_key ((__uint128_t __user *)hva, &old, *old_addr, new, access_key); - *success = !ret && old == *old_addr; - *old_addr = old; + ret =cmpxchg_user_key ((__uint128_t __user *)hva, &old, old_addr->sixteen, + new.sixteen, acc); + *success = !ret && old == old_addr->sixteen; + old_addr->sixteen = old;break ; }default :
arch/s390/kvm/gaccess.h +2 -2
@@ -206,8 +206,8 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar,int access_guest_real (struct kvm_vcpu *vcpu,unsigned long gra,void *data,unsigned long len,enum gacc_mode mode); -int cmpxchg_guest_abs_with_key (struct kvm *kvm,gpa_t gpa,int len,__uint128_t *old, -__uint128_t new,u8 access_key,bool *success); +int cmpxchg_guest_abs_with_key (struct kvm *kvm,gpa_t gpa,int len,union kvm_s390_quad *old_addr, +union kvm_s390_quad new,u8 access_key,bool *success);/** * write_guest_with_key - copy data from kernel space to guest space
arch/s390/kvm/kvm-s390.c +30 -50
116 lines not on this page · raw message
arch/s390/kvm/kvm-s390.h +8 -0
@@ -22,6 +22,14 @@#define KVM_S390_UCONTROL_MEMSLOT (KVM_USER_MEM_SLOTS + 0) +union kvm_s390_quad { +__uint128_t sixteen; +unsigned long eight; +unsigned int four; +unsigned short two; +unsigned char one; +}; +static inline void kvm_s390_fpu_store (struct kvm_run *run) {fpu_stfpc (&run->s.regs.fpc);
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 24/28 KVM: s390: Switch to new gmap
In reply to Claudio ImbrendaSwitch KVM/s390 to use the new gmap code.
Remove includes to <gmap.h> and include "gmap.h" instead; fix all the existing users of the old gmap functions to use the new ones instead.
Fix guest storage key access functions to work with the new gmap.
--- arch/s390/Kconfig | 2 +- arch/s390/include/asm/kvm_host.h | 5 +- arch/s390/include/asm/mmu_context.h | 4 - arch/s390/include/asm/tlb.h | 3 - arch/s390/include/asm/uaccess.h | 70 +-- arch/s390/include/asm/uv.h | 1 - arch/s390/kernel/uv.c | 114 +--- arch/s390/kvm/Makefile | 2 +- arch/s390/kvm/diag.c | 2 +- arch/s390/kvm/gaccess.c | 869 +++++++++++++++++----------- arch/s390/kvm/gaccess.h | 18 +- arch/s390/kvm/gmap-vsie.c | 141 ----- arch/s390/kvm/intercept.c | 15 +- arch/s390/kvm/interrupt.c | 6 +- arch/s390/kvm/kvm-s390.c | 767 +++++++----------------- arch/s390/kvm/kvm-s390.h | 19 +- arch/s390/kvm/priv.c | 213 +++---- arch/s390/kvm/pv.c | 174 ++++-- arch/s390/kvm/vsie.c | 169 +++--- arch/s390/lib/uaccess.c | 184 +----- arch/s390/mm/gmap_helpers.c | 38 +- 21 files changed, 1106 insertions(+), 1710 deletions(-) delete mode 100644 arch/s390/kvm/gmap-vsie.c
arch/s390/Kconfig +1 -1
@@ -33,7 +33,7 @@ config GENERIC_LOCKBREAK def_bool y if PREEMPTION config PGSTE - def_bool y if KVM + def_bool n config AUDIT_ARCH def_bool y
arch/s390/include/asm/kvm_host.h +4 -1
@@ -442,7 +442,7 @@ struct kvm_vcpu_arch {bool acrs_loaded;struct kvm_s390_pv_vcpu pv;union diag318_info diag318_info; -void *mc;/* Placeholder */ +struct kvm_s390_mmu_cache *mc; };struct kvm_vm_stat {
@@ -636,6 +636,8 @@ struct kvm_s390_pv {struct mutex import_lock; }; +struct kvm_s390_mmu_cache ; +struct kvm_arch {struct esca_block *sca;debug_info_t *dbf;
@@ -675,6 +677,7 @@ struct kvm_arch {struct kvm_s390_pv pv;struct list_head kzdev_list;spinlock_t kzdev_list_lock; +struct kvm_s390_mmu_cache *mc; };#define KVM_HVA_ERR_BAD (-1UL)
arch/s390/include/asm/mmu_context.h +0 -4
@@ -30,11 +30,7 @@ static inline int init_new_context(struct task_struct *tsk, mm->context.gmap_asce =0 ; mm->context.flush_mm =0 ;#if IS_ENABLED (CONFIG_KVM ) - mm->context.has_pgste =0 ; - mm->context.uses_skeys =0 ; - mm->context.uses_cmm =0 ; mm->context.allow_cow_sharing =1 ; - mm->context.allow_gmap_hpage_1m =0 ;#endif switch (mm->context.asce_limit) {default :
arch/s390/include/asm/tlb.h +0 -3
@@ -36,7 +36,6 @@ static inline bool __tlb_remove_folio_pages(struct mmu_gather *tlb,#include <asm/tlbflush.h> #include <asm-generic/tlb.h> -#include <asm/gmap.h> /* * Release the page cache reference for a pte removed by
@@ -85,8 +84,6 @@ static inline void pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, tlb->mm->context.flush_mm =1 ; tlb->freed_tables =1 ; tlb->cleared_pmds =1 ; -if (mm_has_pgste (tlb->mm)) -gmap_unlink (tlb->mm, (unsigned long *)pte, address);tlb_remove_ptdesc (tlb,virt_to_ptdesc (pte)); }
arch/s390/include/asm/uaccess.h +10 -60
75 lines not on this page · raw message
arch/s390/include/asm/uv.h +0 -1
@@ -631,7 +631,6 @@ int uv_pin_shared(unsigned long paddr);int uv_destroy_folio (struct folio *folio);int uv_destroy_pte (pte_t pte);int uv_convert_from_secure_pte (pte_t pte); -int make_hva_secure (struct mm_struct *mm,unsigned long hva,struct uv_cb_header *uvcb);int s390_wiggle_split_folio (struct mm_struct *mm,struct folio *folio);int __make_folio_secure (struct folio *folio,struct uv_cb_header *uvcb);int uv_convert_from_secure (unsigned long paddr);
arch/s390/kernel/uv.c +7 -107
143 lines not on this page · raw message
arch/s390/kvm/Makefile +1 -1
@@ -8,7 +8,7 @@ include $(srctree)/virt/kvm/Makefile.kvm ccflags-y := -Ivirt/kvm -Iarch/s390/kvm kvm-y += kvm-s390.o intercept.o interrupt.o priv.o sigp.o -kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o gmap-vsie.o +kvm-y += diag.o gaccess.o guestdbg.o vsie.o pv.o kvm-y += dat.o gmap.o faultin.o kvm-$(CONFIG_VFIO_PCI_ZDEV_KVM) += pci.o
arch/s390/kvm/diag.c +1 -1
@@ -10,13 +10,13 @@#include <linux/kvm.h> #include <linux/kvm_host.h> -#include <asm/gmap.h> #include <asm/gmap_helpers.h> #include <asm/virtio-ccw.h> #include "kvm-s390.h" #include "trace.h" #include "trace-s390.h" #include "gaccess.h" +#include "gmap.h" static void do_discard_gfn_range (struct kvm_vcpu *vcpu,gfn_t gfn_start,gfn_t gfn_end) {
arch/s390/kvm/gaccess.c +518 -351
1,049 lines not on this page · raw message
arch/s390/kvm/gaccess.h +12 -6
@@ -206,7 +206,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar,int access_guest_real (struct kvm_vcpu *vcpu,unsigned long gra,void *data,unsigned long len,enum gacc_mode mode); -int cmpxchg_guest_abs_with_key (struct kvm *kvm,gpa_t gpa,int len,union kvm_s390_quad *old_addr, +int cmpxchg_guest_abs_with_key (struct kvm *kvm,gpa_t gpa,int len,union kvm_s390_quad *old,union kvm_s390_quad new,u8 access_key,bool *success);/**
@@ -450,11 +450,17 @@ void ipte_unlock(struct kvm *kvm);int ipte_lock_held (struct kvm *kvm);int kvm_s390_check_low_addr_prot_real (struct kvm_vcpu *vcpu,unsigned long gra); -/* MVPG PEI indication bits */ -#define PEI_DAT_PROT 2 -#define PEI_NOT_PTE 4 +union mvpg_pei { +unsigned long val; +struct { +unsigned long addr :61 ; +unsigned long not_pte :1 ; +unsigned long dat_prot:1 ; +unsigned long real :1 ; + }; +}; -int kvm_s390_shadow_fault (struct kvm_vcpu *vcpu,struct gmap *shadow, -unsigned long saddr,unsigned long *datptr); +int gaccess_shadow_fault (struct kvm_vcpu *vcpu,struct gmap *sg,gpa_t saddr, +union mvpg_pei *datptr,bool wr);#endif /* __KVM_S390_GACCESS_H */
arch/s390/kvm/gmap-vsie.c deleted +0 -141
141 lines not on this page · raw message
arch/s390/kvm/intercept.c +11 -4
@@ -21,6 +21,7 @@#include "gaccess.h" #include "trace.h" #include "trace-s390.h" +#include "faultin.h" u8 kvm_s390_get_ilen (struct kvm_vcpu *vcpu) {
@@ -367,8 +368,11 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu) reg2, &srcaddr, GACC_FETCH, 0);if (rc)return kvm_s390_inject_prog_cond (vcpu, rc); - rc =kvm_s390_handle_dat_fault (vcpu, srcaddr,0 ); -if (rc !=0 ) + +do { + rc =kvm_s390_faultin_gfn_simple (vcpu,NULL ,gpa_to_gfn (srcaddr), false); + }while (rc == -EAGAIN ); +if (rc)return rc;/* Ensure that the source is paged-in, no actual access -> no key checking */
@@ -376,8 +380,11 @@ static int handle_mvpg_pei(struct kvm_vcpu *vcpu) reg1, &dstaddr, GACC_STORE, 0);if (rc)return kvm_s390_inject_prog_cond (vcpu, rc); - rc =kvm_s390_handle_dat_fault (vcpu, dstaddr,FOLL_WRITE ); -if (rc !=0 ) + +do { + rc =kvm_s390_faultin_gfn_simple (vcpu,NULL ,gpa_to_gfn (dstaddr), true); + }while (rc == -EAGAIN ); +if (rc)return rc;kvm_s390_retry_instr (vcpu);
arch/s390/kvm/interrupt.c +3 -3
@@ -26,7 +26,6 @@#include <linux/uaccess.h> #include <asm/sclp.h> #include <asm/isc.h> -#include <asm/gmap.h> #include <asm/nmi.h> #include <asm/airq.h> #include <asm/tpi.h>
@@ -34,6 +33,7 @@#include "gaccess.h" #include "trace-s390.h" #include "pci.h" +#include "gmap.h" #define PFAULT_INIT 0x0600#define PFAULT_DONE 0x0680
@@ -2632,12 +2632,12 @@ static int flic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)case KVM_DEV_FLIC_APF_ENABLE :if (kvm_is_ucontrol (dev->kvm))return -EINVAL ; - dev->kvm->arch.gmap->pfault_enabled =1 ; +set_bit (GMAP_FLAG_PFAULT_ENABLED , &dev->kvm->arch.gmap->flags);break ;case KVM_DEV_FLIC_APF_DISABLE_WAIT :if (kvm_is_ucontrol (dev->kvm))return -EINVAL ; - dev->kvm->arch.gmap->pfault_enabled =0 ; +clear_bit (GMAP_FLAG_PFAULT_ENABLED , &dev->kvm->arch.gmap->flags);/* * Make sure no async faults are in transition when * clearing the queues. So we don't need to worry
arch/s390/kvm/kvm-s390.c +223 -544
1,198 lines not on this page · raw message
arch/s390/kvm/kvm-s390.h +4 -15
@@ -19,6 +19,8 @@#include <asm/facility.h> #include <asm/processor.h> #include <asm/sclp.h> +#include "dat.h" +#include "gmap.h" #define KVM_S390_UCONTROL_MEMSLOT (KVM_USER_MEM_SLOTS + 0)
@@ -114,9 +116,7 @@ static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)static inline int kvm_is_ucontrol (struct kvm *kvm) {#ifdef CONFIG_KVM_S390_UCONTROL -if (kvm->arch.gmap) -return 0 ; -return 1 ; +return test_bit (GMAP_FLAG_IS_UCONTROL , &kvm->arch.gmap->flags);#else return 0 ;#endif
@@ -440,14 +440,9 @@ int kvm_s390_skey_check_enable(struct kvm_vcpu *vcpu);/* implemented in vsie.c */ int kvm_s390_handle_vsie (struct kvm_vcpu *vcpu);void kvm_s390_vsie_kick (struct kvm_vcpu *vcpu); -void kvm_s390_vsie_gmap_notifier (struct gmap *gmap,unsigned long start, -unsigned long end); +void kvm_s390_vsie_gmap_notifier (struct gmap *gmap,gpa_t start,gpa_t end);void kvm_s390_vsie_init (struct kvm *kvm);void kvm_s390_vsie_destroy (struct kvm *kvm); -int gmap_shadow_valid (struct gmap *sg,unsigned long asce,int edat_level); - -/* implemented in gmap-vsie.c */ -struct gmap *gmap_shadow (struct gmap *parent,unsigned long asce,int edat_level);/* implemented in sigp.c */ int kvm_s390_handle_sigp (struct kvm_vcpu *vcpu);
@@ -469,15 +464,9 @@ void kvm_s390_vcpu_unsetup_cmma(struct kvm_vcpu *vcpu);void kvm_s390_set_cpu_timer (struct kvm_vcpu *vcpu,__u64 cputm);__u64 kvm_s390_get_cpu_timer (struct kvm_vcpu *vcpu);int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc); -int __kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gfn_t gfn,gpa_t gaddr,unsigned int flags);int __kvm_s390_mprotect_many (struct gmap *gmap,gpa_t gpa,u8 npages,unsigned int prot,unsigned long bits); -static inline int kvm_s390_handle_dat_fault (struct kvm_vcpu *vcpu,gpa_t gaddr,unsigned int flags) -{ -return __kvm_s390_handle_dat_fault (vcpu,gpa_to_gfn (gaddr), gaddr, flags); -} -bool kvm_arch_setup_async_pf (struct kvm_vcpu *vcpu);/* implemented in diag.c */
arch/s390/kvm/priv.c +78 -135
169 lines not on this page · raw message
arch/s390/kvm/pv.c +116 -58
174 lines not on this page · raw message
arch/s390/kvm/vsie.c +89 -80
169 lines not on this page · raw message
arch/s390/lib/uaccess.c +26 -158
184 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.52.0
Part of the diff is not shown here. View the raw message.
kernel test robot
build botRe: [PATCH v6 24/28] KVM: s390: Switch to new gmap
kernel test robot noticed the following build warnings:
Hi Claudio,
kernel test robot noticed the following build warnings:
[auto build test WARNING on s390/features]
[also build test WARNING on kvm/queue kvm/next linus/master v6.19-rc2 next-20251219] [cannot apply to kvms390/next kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Claudio-Imbrenda/KVM-s390-Refactor-pgste-lock-and-unlock-functions/20251223-012700 base: https://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git features patch link: https://lore.kernel.org/r/20251222165033.162329-25-imbrenda%40linux.ibm.com patch subject: [PATCH v6 24/28] KVM: s390: Switch to new gmap config: s390-defconfig (https://download.01.org/0day-ci/archive/20251224/[email protected]/config) compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 4ef602d446057dabf5f61fb221669ecbeda49279) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251224/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> Warning: arch/s390/kvm/gaccess.c:1052 function parameter 'size' not described in '__cmpxchg_with_key' >> Warning: arch/s390/kvm/gaccess.c:1587 expecting prototype for __kvm_s390_shadow_fault(). Prototype was for __gaccess_shadow_fault() instead >> Warning: arch/s390/kvm/gaccess.c:1052 function parameter 'size' not described in '__cmpxchg_with_key'
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Claudio Imbrenda
maintainerPATCH v6 25/28 KVM: s390: Remove gmap from s390/mm
In reply to Claudio ImbrendaRemove the now unused include/asm/gmap.h and mm/gmap.c files.
--- MAINTAINERS | 2 - arch/s390/include/asm/gmap.h | 174 --- arch/s390/include/asm/pgtable.h | 8 - arch/s390/mm/Makefile | 1 - arch/s390/mm/gmap.c | 2436 ------------------------------- arch/s390/mm/pgtable.c | 8 - 6 files changed, 2629 deletions(-) delete mode 100644 arch/s390/include/asm/gmap.h delete mode 100644 arch/s390/mm/gmap.c
MAINTAINERS +0 -2
@@ -13899,14 +13899,12 @@ L: [email protected] S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git F: Documentation/virt/kvm/s390* -F: arch/s390/include/asm/gmap.h F: arch/s390/include/asm/gmap_helpers.h F: arch/s390/include/asm/kvm* F: arch/s390/include/uapi/asm/kvm* F: arch/s390/include/uapi/asm/uvdevice.h F: arch/s390/kernel/uv.c F: arch/s390/kvm/ -F: arch/s390/mm/gmap.c F: arch/s390/mm/gmap_helpers.c F: drivers/s390/char/uvdevice.c F: tools/testing/selftests/drivers/s390x/uvdevice/
arch/s390/include/asm/gmap.h deleted +0 -174
174 lines not on this page · raw message
arch/s390/include/asm/pgtable.h +0 -8
@@ -1369,8 +1369,6 @@ static inline int ptep_set_access_flags(struct vm_area_struct *vma,void ptep_set_pte_at (struct mm_struct *mm,unsigned long addr,pte_t *ptep,pte_t entry);void ptep_set_notify (struct mm_struct *mm,unsigned long addr,pte_t *ptep); -void ptep_notify (struct mm_struct *mm,unsigned long addr, -pte_t *ptep,unsigned long bits);int ptep_force_prot (struct mm_struct *mm,unsigned long gaddr,pte_t *ptep,int prot,unsigned long bit); void ptep_zap_unused(struct mm_struct *mm, unsigned long addr,
@@ -1396,9 +1394,6 @@ int set_pgste_bits(struct mm_struct *mm, unsigned long addr,int get_pgste (struct mm_struct *mm,unsigned long hva,unsigned long *pgstep);int pgste_perform_essa (struct mm_struct *mm,unsigned long hva,int orc,unsigned long *oldpte,unsigned long *oldpgste); -void gmap_pmdp_invalidate (struct mm_struct *mm,unsigned long vmaddr); -void gmap_pmdp_idte_local (struct mm_struct *mm,unsigned long vmaddr); -void gmap_pmdp_idte_global (struct mm_struct *mm,unsigned long vmaddr);#define pgprot_writecombine pgprot_writecombinepgprot_t pgprot_writecombine (pgprot_t prot);
@@ -2023,9 +2018,6 @@ extern int __vmem_map_4k_page(unsigned long addr, unsigned long phys, pgprot_t pextern int vmem_map_4k_page (unsigned long addr,unsigned long phys,pgprot_t prot);extern void vmem_unmap_4k_page (unsigned long addr);extern pte_t *vmem_get_alloc_pte (unsigned long addr,bool alloc); -extern int s390_enable_sie (void ); -extern int s390_enable_skey (void ); -extern void s390_reset_cmma (struct mm_struct *mm);/* s390 has a private copy of get unmapped area to deal with cache synonyms */ #define HAVE_ARCH_UNMAPPED_AREA
arch/s390/mm/Makefile +0 -1
@@ -10,7 +10,6 @@ obj-$(CONFIG_CMM) += cmm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP) += dump_pagetables.o -obj-$(CONFIG_PGSTE) += gmap.o obj-$(CONFIG_PFAULT) += pfault.o obj-$(subst m,y,$(CONFIG_KVM)) += gmap_helpers.o
arch/s390/mm/gmap.c deleted +0 -2436
2,436 lines not on this page · raw message
arch/s390/mm/pgtable.c +0 -8
@@ -369,8 +369,6 @@ static inline void pmdp_idte_local(struct mm_struct *mm, mm->context.asce, IDTE_LOCAL); else __pmdp_idte(addr, pmdp, 0, 0, IDTE_LOCAL); -if (mm_has_pgste (mm) && mm->context.allow_gmap_hpage_1m) -gmap_pmdp_idte_local (mm, addr); } static inline void pmdp_idte_global(struct mm_struct *mm,
@@ -379,12 +377,8 @@ static inline void pmdp_idte_global(struct mm_struct *mm,if (machine_has_tlb_guest ()) {__pmdp_idte (addr, pmdp,IDTE_NODAT |IDTE_GUEST_ASCE , mm->context.asce,IDTE_GLOBAL ); -if (mm_has_pgste (mm) && mm->context.allow_gmap_hpage_1m) -gmap_pmdp_idte_global (mm, addr); }else {__pmdp_idte (addr, pmdp,0 ,0 ,IDTE_GLOBAL ); -if (mm_has_pgste (mm) && mm->context.allow_gmap_hpage_1m) -gmap_pmdp_idte_global (mm, addr); } }
@@ -419,8 +413,6 @@ static inline pmd_t pmdp_flush_lazy(struct mm_struct *mm, cpumask_of(smp_processor_id()))) { set_pmd(pmdp, set_pmd_bit(*pmdp, __pgprot(_SEGMENT_ENTRY_INVALID))); mm->context.flush_mm = 1; - if (mm_has_pgste(mm)) - gmap_pmdp_invalidate(mm, addr); } else {pmdp_idte_global (mm, addr, pmdp); }
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 26/28 KVM: S390: Remove PGSTE code from linux/s390 mm
In reply to Claudio ImbrendaRemove the PGSTE config option.
Remove all code from linux/s390 mm that involves PGSTEs.
--- arch/s390/Kconfig | 3 - arch/s390/include/asm/hugetlb.h | 6 - arch/s390/include/asm/mmu.h | 13 - arch/s390/include/asm/page.h | 4 - arch/s390/include/asm/pgalloc.h | 4 - arch/s390/include/asm/pgtable.h | 121 +---- arch/s390/kvm/dat.h | 1 + arch/s390/mm/hugetlbpage.c | 24 - arch/s390/mm/pgalloc.c | 24 - arch/s390/mm/pgtable.c | 827 +------------------------------- mm/khugepaged.c | 9 - 11 files changed, 15 insertions(+), 1021 deletions(-)
arch/s390/Kconfig +0 -3
@@ -32,9 +32,6 @@ config GENERIC_BUG_RELATIVE_POINTERS config GENERIC_LOCKBREAK def_bool y if PREEMPTION -config PGSTE - def_bool n - config AUDIT_ARCH def_bool y
arch/s390/include/asm/hugetlb.h +0 -6
@@ -37,12 +37,6 @@ static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,return __huge_ptep_get_and_clear (mm, addr, ptep); } -static inline void arch_clear_hugetlb_flags (struct folio *folio) -{ -clear_bit (PG_arch_1, &folio->flags.f); -} -#define arch_clear_hugetlb_flags arch_clear_hugetlb_flags -#define __HAVE_ARCH_HUGE_PTE_CLEARstatic inline void huge_pte_clear (struct mm_struct *mm,unsigned long addr,pte_t *ptep,unsigned long sz)
arch/s390/include/asm/mmu.h +0 -13
@@ -18,24 +18,11 @@ typedef struct {unsigned long vdso_base;/* The mmu context belongs to a secure guest. */ atomic_t protected_count; -/* -* The following bitfields need a down_write on the mm -* semaphore when they are written to. As they are only -* written once, they can be read without a lock. -*/ -/* The mmu context uses extended page tables. */ - unsigned int has_pgste:1; -/* The mmu context uses storage keys. */ - unsigned int uses_skeys:1; -/* The mmu context uses CMM. */ - unsigned int uses_cmm:1;/* * The mmu context allows COW-sharing of memory pages (KSM, zeropage). * Note that COW-sharing during fork() is currently always allowed. */ unsigned int allow_cow_sharing:1; -/* The gmaps associated with this context are allowed to use huge pages. */ - unsigned int allow_gmap_hpage_1m:1; } mm_context_t;#define INIT_MM_CONTEXT (name) \
arch/s390/include/asm/page.h +0 -4
@@ -78,7 +78,6 @@ static inline void copy_page(void *to, void *from)#ifdef STRICT_MM_TYPECHECKS typedef struct {unsigned long pgprot; }pgprot_t ; -typedef struct {unsigned long pgste; }pgste_t ;typedef struct {unsigned long pte; }pte_t ;typedef struct {unsigned long pmd; }pmd_t ;typedef struct {unsigned long pud; }pud_t ;
@@ -94,7 +93,6 @@ static __always_inline unsigned long name ## _val(name ## _t name) \#else /* STRICT_MM_TYPECHECKS */ typedef unsigned long pgprot_t ; -typedef unsigned long pgste_t ;typedef unsigned long pte_t ;typedef unsigned long pmd_t ;typedef unsigned long pud_t ;
@@ -110,7 +108,6 @@ static __always_inline unsigned long name ## _val(name ## _t name) \#endif /* STRICT_MM_TYPECHECKS */ DEFINE_PGVAL_FUNC(pgprot) -DEFINE_PGVAL_FUNC(pgste) DEFINE_PGVAL_FUNC(pte) DEFINE_PGVAL_FUNC(pmd) DEFINE_PGVAL_FUNC(pud)
@@ -120,7 +117,6 @@ DEFINE_PGVAL_FUNC(pgd)typedef pte_t *pgtable_t ;#define __pgprot (x) ((pgprot_t) { (x) } ) -#define __pgste (x) ((pgste_t) { (x) } )#define __pte (x) ((pte_t) { (x) } )#define __pmd (x) ((pmd_t) { (x) } )#define __pud (x) ((pud_t) { (x) } )
arch/s390/include/asm/pgalloc.h +0 -4
@@ -27,10 +27,6 @@ unsigned long *page_table_alloc_noprof(struct mm_struct *);#define page_table_alloc (...) alloc_hooks(page_table_alloc_noprof(__VA_ARGS__))void page_table_free (struct mm_struct *,unsigned long *); -struct ptdesc *page_table_alloc_pgste_noprof (struct mm_struct *mm); -#define page_table_alloc_pgste (...) alloc_hooks(page_table_alloc_pgste_noprof(__VA_ARGS__)) -void page_table_free_pgste (struct ptdesc *ptdesc); -static inline void crst_table_init (unsigned long *crst,unsigned long entry) {memset64 ((u64 *)crst, entry, _CRST_ENTRIES);
arch/s390/include/asm/pgtable.h +7 -114
167 lines not on this page · raw message
arch/s390/kvm/dat.h +1 -0
@@ -108,6 +108,7 @@ union pte {#define _PAGE_SD 0x002/* Needed as macro to perform atomic operations */ +#define PGSTE_PCL_BIT 0x0080000000000000UL/* PCL lock, HW bit */ #define PGSTE_CMMA_D_BIT 0x0000000000008000UL/* CMMA dirty soft-bit */ enum pgste_gps_usage {
arch/s390/mm/hugetlbpage.c +0 -24
@@ -135,29 +135,6 @@ static inline pte_t __rste_to_pte(unsigned long rste)return __pte (pteval); } -static void clear_huge_pte_skeys (struct mm_struct *mm,unsigned long rste) -{ -struct folio *folio; -unsigned long size, paddr; - -if (!mm_uses_skeys (mm) || - rste & _SEGMENT_ENTRY_INVALID) -return ; - -if ((rste & _REGION_ENTRY_TYPE_MASK) == _REGION_ENTRY_TYPE_R3) { - folio =page_folio (pud_page (__pud (rste))); - size =PUD_SIZE ; - paddr = rste &PUD_MASK ; - }else { - folio =page_folio (pmd_page (__pmd (rste))); - size =PMD_SIZE ; - paddr = rste &PMD_MASK ; - } - -if (!test_and_set_bit (PG_arch_1, &folio->flags.f)) -__storage_key_init_range (paddr, paddr + size); -} -void __set_huge_pte_at (struct mm_struct *mm,unsigned long addr,pte_t *ptep,pte_t pte) {
@@ -173,7 +150,6 @@ void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, }else if (likely (pte_present (pte))) rste |= _SEGMENT_ENTRY_LARGE; -clear_huge_pte_skeys (mm, rste);set_pte (ptep,__pte (rste)); }
arch/s390/mm/pgalloc.c +0 -24
@@ -114,30 +114,6 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)return -ENOMEM ; } -#ifdef CONFIG_PGSTE - -struct ptdesc *page_table_alloc_pgste_noprof (struct mm_struct *mm) -{ -struct ptdesc *ptdesc; -u64 *table; - - ptdesc =pagetable_alloc_noprof (GFP_KERNEL_ACCOUNT ,0 ); -if (ptdesc) { - table = (u64 *)ptdesc_address (ptdesc); -__arch_set_page_dat (table,1 ); -memset64 (table, _PAGE_INVALID,PTRS_PER_PTE ); -memset64 (table +PTRS_PER_PTE ,0 ,PTRS_PER_PTE ); - } -return ptdesc; -} - -void page_table_free_pgste (struct ptdesc *ptdesc) -{ -pagetable_free (ptdesc); -} - -#endif /* CONFIG_PGSTE */ -unsigned long *page_table_alloc_noprof (struct mm_struct *mm) {gfp_t gfp =GFP_KERNEL_ACCOUNT ;
arch/s390/mm/pgtable.c +7 -820
874 lines not on this page · raw message
mm/khugepaged.c +0 -9
@@ -343,15 +343,6 @@ int hugepage_madvise(struct vm_area_struct *vma, {switch (advice) {case MADV_HUGEPAGE : -#ifdef CONFIG_S390 -/* -* qemu blindly sets MADV_HUGEPAGE on all allocations, but s390 -* can't handle this properly after s390_enable_sie, so we simply -* ignore the madvise to prevent qemu from causing a SIGSEGV. -*/ -if (mm_has_pgste (vma->vm_mm)) -return 0 ; -#endif *vm_flags &= ~VM_NOHUGEPAGE ; *vm_flags |=VM_HUGEPAGE ;/*
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 27/28 KVM: s390: Enable 1M pages for gmap
In reply to Claudio ImbrendaWhile userspace is allowed to have pages of any size, the new gmap would always use 4k pages to back the guest.
Enable 1M pages for gmap.
This allows 1M pages to be used to back a guest when userspace is using 1M pages for the corresponding addresses (e.g. THP or hugetlbfs).
Remove the limitation that disallowed having nested guests and hugepages at the same time.
--- arch/s390/kvm/gmap.c | 2 +- arch/s390/kvm/kvm-s390.c | 6 +----- arch/s390/kvm/pv.c | 3 +++ 3 files changed, 5 insertions(+), 6 deletions(-)
arch/s390/kvm/gmap.c +1 -1
@@ -617,7 +617,7 @@ static inline bool gmap_2g_allowed(struct gmap *gmap, gfn_t gfn)static inline bool gmap_1m_allowed (struct gmap *gmap,gfn_t gfn) { -return false; +return test_bit (GMAP_FLAG_ALLOW_HPAGE_1M , &gmap->flags); }int gmap_link (struct kvm_s390_mmu_cache *mc,struct gmap *gmap,struct guest_fault *f)
arch/s390/kvm/kvm-s390.c +1 -5
@@ -851,6 +851,7 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) r = -EINVAL ; else { r =0 ; +set_bit (GMAP_FLAG_ALLOW_HPAGE_1M , &kvm->arch.gmap->flags);/* * We might have to create fake 4k page * tables. To avoid that the hardware works on
@@ -5739,11 +5740,6 @@ static int __init kvm_s390_init(void)return -ENODEV ; } -if (nested && hpage) { -pr_info ("A KVM host that supports nesting cannot back its KVM guests with huge pages\n" ); -return -EINVAL ; - } -for (i =0 ; i <16 ; i++) kvm_s390_fac_base[i] |= stfle_fac_list[i] &nonhyp_mask (i);
arch/s390/kvm/pv.c +3 -0
@@ -721,6 +721,9 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc) uvcb.flags.ap_allow_instr = kvm->arch.model.uv_feat_guest.ap; uvcb.flags.ap_instr_intr = kvm->arch.model.uv_feat_guest.ap_intr; +clear_bit (GMAP_FLAG_ALLOW_HPAGE_1M , &kvm->arch.gmap->flags); +gmap_split_huge_pages (kvm->arch.gmap); + cc =uv_call_sched (0 , (u64)&uvcb); *rc = uvcb.header.rc; *rrc = uvcb.header.rrc;
Signature · 1 line
2.52.0
Claudio Imbrenda
maintainerPATCH v6 28/28 KVM: s390: Storage key manipulation IOCTL
In reply to Claudio ImbrendaAdd a new IOCTL to allow userspace to manipulate storage keys directly.
This will make it easier to write selftests related to storage keys.
--- arch/s390/kvm/kvm-s390.c | 57 ++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 10 +++++++ 2 files changed, 67 insertions(+)
arch/s390/kvm/kvm-s390.c +57 -0
@@ -554,6 +554,37 @@ static void __kvm_s390_exit(void)debug_unregister (kvm_s390_dbf_uv); } +static int kvm_s390_keyop (struct kvm_s390_mmu_cache *mc,struct kvm *kvm,int op, +unsigned long addr,union skey skey) +{ +union asce asce = kvm->arch.gmap->asce; +gfn_t gfn =gpa_to_gfn (addr); +int r; + +guard (read_lock)(&kvm->mmu_lock); + +switch (op) { +case KVM_S390_KEYOP_SSKE : + r =dat_cond_set_storage_key (mc, asce, gfn, skey, &skey,0 ,0 ,0 ); +if (r >=0 ) +return skey.skey; +break ; +case KVM_S390_KEYOP_ISKE : + r =dat_get_storage_key (asce, gfn, &skey); +if (!r) +return skey.skey; +break ; +case KVM_S390_KEYOP_RRBE : + r =dat_reset_reference_bit (asce, gfn); +if (r >0 ) +return r <<1 ; +break ; +default : +return -EINVAL ; + } +return r; +} +/* Section: device related */ long kvm_arch_dev_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg)
@@ -2931,6 +2962,32 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = -EFAULT ;break ; } +case KVM_S390_KEYOP : { +struct kvm_s390_mmu_cache *mc; +struct kvm_s390_keyop kop; +union skey skey; + +if (copy_from_user (&kop, argp,sizeof (kop))) { + r = -EFAULT ; +break ; + } + skey.skey = kop.key; + + mc =kvm_s390_new_mmu_cache (); +if (!mc) +return -ENOMEM ; + + r =kvm_s390_keyop (mc, kvm, kop.operation, kop.user_addr, skey); +kvm_s390_free_mmu_cache (mc); +if (r <0 ) +break ; + + kop.key = r; + r =0 ; +if (copy_to_user (argp, &kop,sizeof (kop))) + r = -EFAULT ; +break ; + }case KVM_S390_ZPCI_OP : {struct kvm_s390_zpci_op args;
include/uapi/linux/kvm.h +10 -0
@@ -1219,6 +1219,15 @@ struct kvm_vfio_spapr_tce {__s32 tablefd; }; +#define KVM_S390_KEYOP_SSKE 0x01 +#define KVM_S390_KEYOP_ISKE 0x02 +#define KVM_S390_KEYOP_RRBE 0x03 +struct kvm_s390_keyop { +__u64 user_addr; +__u8 key; +__u8 operation; +}; +/* * KVM_CREATE_VCPU receives as a parameter the vcpu slot, and returns * a vcpu fd.
@@ -1238,6 +1247,7 @@ struct kvm_vfio_spapr_tce {#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x50, struct kvm_s390_ucas_mapping)#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x51, struct kvm_s390_ucas_mapping)#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x52, unsigned long) +#define KVM_S390_KEYOP _IOWR(KVMIO, 0x53, struct kvm_s390_keyop)/* Device model IOC */ #define KVM_CREATE_IRQCHIP _IO(KVMIO, 0x60)
Signature · 1 line
2.52.0
Re: [PATCH v6 28/28] KVM: s390: Storage key manipulation IOCTL
Quoted 5 lines · Claudio Imbrenda
Add a new IOCTL to allow userspace to manipulate storage keys directly.
This will make it easier to write selftests related to storage keys.
Signed-off-by Claudio Imbrenda <[email protected]>
Please add some user documentation for the new IOCTL.
[...]
include/uapi/linux/kvm.h struct kvm_vfio_spapr_tce { · line 1224
Quoted 11 lines
include/uapi/linux/kvm.h +3
-0@@ -1219,6 +1219,15 @@ struct kvm_vfio_spapr_tce {__s32 tablefd; }; +#define KVM_S390_KEYOP_SSKE 0x01 +#define KVM_S390_KEYOP_ISKE 0x02 +#define KVM_S390_KEYOP_RRBE 0x03
Just a nitpik, but why this order? In the arch the order is ISKE, SSKE, RRBE. Would it not be more logical to keep that order?
Quoted 17 lines
include/uapi/linux/kvm.h +7
-0+struct kvm_s390_keyop { +__u64 user_addr; +__u8 key; +__u8 operation; +}; +/* * KVM_CREATE_VCPU receives as a parameter the vcpu slot, and returns * a vcpu fd. @@ -1238,6 +1247,7 @@ struct kvm_vfio_spapr_tce {#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x50, struct kvm_s390_ucas_mapping)#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x51, struct kvm_s390_ucas_mapping)#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x52, unsigned long) +#define KVM_S390_KEYOP _IOWR(KVMIO, 0x53, struct kvm_s390_keyop)/* Device model IOC */ #define KVM_CREATE_IRQCHIP _IO(KVMIO, 0x60)
Claudio Imbrenda
maintainerRe: [PATCH v6 28/28] KVM: s390: Storage key manipulation IOCTL
Quoted 8 lines · On Wed, 14 Jan 2026 10:33:22 +0100 "Christoph Schlameuss"…
Quoted 5 lines · Claudio Imbrenda
Add a new IOCTL to allow userspace to manipulate storage keys directly.
This will make it easier to write selftests related to storage keys.
Signed-off-by Claudio Imbrenda <[email protected]>Please add some user documentation for the new IOCTL.
already done, it will be in the next iteration.
and also a capability for the new ioctl
Quoted 11 lines
[...]
Quoted 8 lines
include/uapi/linux/kvm.h +0
-0@@ -1219,6 +1219,15 @@ struct kvm_vfio_spapr_tce {__s32 tablefd; };
Quoted 6 lines
include/uapi/linux/kvm.h struct kvm_vfio_spapr_tce { · line 1224
Quoted 3 lines
include/uapi/linux/kvm.h +3
-0+#define KVM_S390_KEYOP_SSKE 0x01 +#define KVM_S390_KEYOP_ISKE 0x02 +#define KVM_S390_KEYOP_RRBE 0x03Just a nitpik, but why this order? In the arch the order is ISKE, SSKE, RRBE. Would it not be more logical to keep that order?
I don't know why I chose that order, I guess I can reorder it. It doesn't really make a difference, but I guess consistency is good
Quoted 19 lines
Quoted 18 lines
include/uapi/linux/kvm.h +7
-0+struct kvm_s390_keyop { +__u64 user_addr; +__u8 key; +__u8 operation; +}; +/* * KVM_CREATE_VCPU receives as a parameter the vcpu slot, and returns * a vcpu fd. @@ -1238,6 +1247,7 @@ struct kvm_vfio_spapr_tce {#define KVM_S390_UCAS_MAP _IOW(KVMIO, 0x50, struct kvm_s390_ucas_mapping)#define KVM_S390_UCAS_UNMAP _IOW(KVMIO, 0x51, struct kvm_s390_ucas_mapping)#define KVM_S390_VCPU_FAULT _IOW(KVMIO, 0x52, unsigned long) +#define KVM_S390_KEYOP _IOWR(KVMIO, 0x53, struct kvm_s390_keyop)/* Device model IOC */ #define KVM_CREATE_IRQCHIP _IO(KVMIO, 0x60)