Skip to content

KVM: s390: gmap rewrite, the real deal

Claudio Imbrenda kvm linux-s390 lkml 6 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest29 patches 1 replyIn review
  2. v628 patches 2 repliesSuperseded
  3. v523 patches 7 repliesSuperseded
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v423 patches0 repliesSuperseded
v323 patches18 repliesSuperseded
v220 patches54 repliesSuperseded

Changes in v7 from the cover letter

* Added selftests for the newly introduced KEYOP ioctl
* Fix the bugs found by the newly introduced selftests
* Fix a bug in the memslot removal code
* Fix a few bugs in vSIE shadowing/deshawoding code
* Fix deadlock between shadowing code and normal fault handling
* Added proper locking for memslot manipulation operations
* Added capability for the KEYOP ioctl, and API documentation
* Minor documentation fixes
* Fixed many style issues in comments
* Fixed regression with ucontrol VMs

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/29KVM: s390: Refactor pgste lock and unlock functionsarch/s390/include/asm/pgtable.h arch/s390/mm/gmap_helpers.c arch/s390/mm/pgtable.c+44 -240 replies
2/29KVM: s390: Add P bit in table entry bitfields, move union vaddressarch/s390/include/asm/dat-bits.h arch/s390/kvm/gaccess.c+30 -280 replies
3/29s390: Make UV folio operations work on whole folioarch/s390/kernel/uv.c+12 -100 replies
4/29s390: Move sske_frame() to a headerarch/s390/include/asm/pgtable.h arch/s390/mm/pageattr.c+7 -70 replies
5/29KVM: s390: Add gmap_helper_set_unused()arch/s390/include/asm/gmap_helpers.h arch/s390/mm/gmap_helpers.c+80 -00 replies
6/29KVM: s390: Introduce import_lockarch/s390/include/asm/kvm_host.h arch/s390/kvm/kvm-s390.c+4 -00 replies
7/29KVM: s390: Export two functionsarch/s390/include/asm/uv.h arch/s390/kernel/uv.c+6 -20 replies
8/29s390/mm: Warn if uv_convert_from_secure_pte() failsarch/s390/include/asm/pgtable.h+5 -40 replies
9/29KVM: s390: vsie: Pass gmap explicitly as parameterarch/s390/kvm/vsie.c+19 -210 replies
10/29KVM: s390: Enable KVM_GENERIC_MMU_NOTIFIERarch/s390/include/asm/kvm_host.h arch/s390/kvm/Kconfig arch/s390/kvm/kvm-s390.c+47 -10 replies
11/29KVM: s390: Rename some functions in gaccess.carch/s390/kvm/gaccess.c+24 -270 replies
12/29KVM: s390: KVM-specific bitfields and helper functionsarch/s390/kvm/dat.h+720 -00 replies
13/29KVM: s390: KVM page table management functions: allocationarch/s390/kvm/Makefile arch/s390/kvm/dat.c arch/s390/kvm/dat.h +1 more+182 -00 replies
14/29KVM: s390: KVM page table management functions: clear and replacearch/s390/kvm/dat.c arch/s390/kvm/dat.h+155 -00 replies
15/29KVM: s390: KVM page table management functions: walksarch/s390/kvm/dat.c arch/s390/kvm/dat.h+425 -00 replies
16/29KVM: s390: KVM page table management functions: storage keysarch/s390/kvm/dat.c arch/s390/kvm/dat.h+230 -00 replies
17/29KVM: s390: KVM page table management functions: lifecycle managementarch/s390/kvm/dat.c arch/s390/kvm/dat.h+348 -00 replies
18/29KVM: s390: KVM page table management functions: CMMAarch/s390/kvm/dat.c arch/s390/kvm/dat.h+302 -00 replies
19/29KVM: s390: New gmap codearch/s390/kvm/Makefile arch/s390/kvm/gmap.c arch/s390/kvm/gmap.h+1410 -10 replies
20/29KVM: s390: Add helper functions for fault handlingarch/s390/include/asm/kvm_host.h arch/s390/kvm/Makefile arch/s390/kvm/faultin.c +3 more+245 -20 replies
21/29KVM: s390: Add some helper functions needed for vSIEarch/s390/kvm/gmap.c+72 -20 replies
22/29KVM: s390: Stop using CONFIG_PGSTEarch/s390/include/asm/mmu_context.h arch/s390/include/asm/pgtable.h arch/s390/mm/fault.c+5 -50 replies
23/29KVM: s390: Storage key functions refactoringarch/s390/kvm/gaccess.c arch/s390/kvm/gaccess.h arch/s390/kvm/kvm-s390.c +1 more+59 -710 replies
24/29KVM: s390: Switch to new gmaparch/s390/Kconfig arch/s390/include/asm/kvm_host.h arch/s390/include/asm/mmu_context.h +18 more+1119 -17260 replies
25/29KVM: s390: Remove gmap from s390/mmMAINTAINERS arch/s390/include/asm/gmap.h arch/s390/include/asm/pgtable.h +3 more+0 -26290 replies
26/29KVM: S390: Remove PGSTE code from linux/s390 mmarch/s390/Kconfig arch/s390/include/asm/hugetlb.h arch/s390/include/asm/mmu.h +8 more+15 -10210 replies
27/29KVM: s390: Enable 1M pages for gmaparch/s390/kvm/gmap.c arch/s390/kvm/kvm-s390.c arch/s390/kvm/pv.c+5 -60 replies
28/29KVM: s390: Storage key manipulation IOCTLDocumentation/virt/kvm/api.rst arch/s390/kvm/kvm-s390.c include/uapi/linux/kvm.h+111 -00 replies
29/29KVM: s390: selftests: Add selftest for the KVM_S390_KEYOP ioctltools/testing/selftests/kvm/Makefile.kvm tools/testing/selftests/kvm/s390/keyop.c+300 -00 replies

Reviews trailers per patch

6 of 29 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchChristian BorntraegerSteffen EidenChristoph SchlameussNico BoehrNina Schoetterl-GlauschHeiko Carstens
SeriesA-b
1/29
2/29R-b R-b R-b
3/29
4/29R-b R-b R-b R-b
5/29A-b R-b T-b
6/29
7/29
8/29
9/29
10/29A-b R-b R-b
11/29R-b R-b
12/29
13/29
14/29
15/29
16/29
17/29
18/29
19/29
20/29
21/29
22/29R-b
23/29
24/29
25/29
26/29
27/29
28/29
29/29

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.

Read the v7 thread

Cover letter [PATCH v7 00/29] KVM: s390: gmap rewrite, the real deal

This series is the last big series of the gmap rewrite. It introduces the new code and actually uses it. The old code is then removed.

KVM on s390 will now use the mmu_notifier, like most other architectures. The gmap address space is now completely separate from userspace; no level of the page tables is shared between guest mapping and userspace.

One of the biggest advantages is that the page size of userspace is completely independent of the page size used by the guest. Userspace can mix normal pages, THPs, hugetlbfs, and more.

It's now possible to have nested guests and guests with huge pages running on the same host. In fact, it's possible to have a nested guest on a guest with huge pages. Transparent hugepages are also possible.

Patches 1 to 11 are mostly preparations; introducing some new bits and functions, and moving code around.

Patches 12 to 21 are the meat of the new gmap code; page table management functions and gmap management. This is the code that will be used to manage guest memory.

Patch 24 is unfortunately big; the existing code is converted to use the new gmap and all references to the old gmap are removed. This needs to be done all at once, unfortunately, hence the size of the patch.

Patch 25 and 26 remove all the now unused code.

Patch 27 and 28 allow for 1M pages to be used to back guests, and add some more functions that are useful for testing.

Patch 29 contains selftests for the storage key interface introduced in the patch 28

v6->v7:
* Added selftests for the newly introduced KEYOP ioctl
* Fix the bugs found by the newly introduced selftests
* Fix a bug in the memslot removal code
* Fix a few bugs in vSIE shadowing/deshawoding code
* Fix deadlock between shadowing code and normal fault handling
* Added proper locking for memslot manipulation operations
* Added capability for the KEYOP ioctl, and API documentation
* Minor documentation fixes
* Fixed many style issues in comments
* Fixed regression with ucontrol VMs

v5->v6:
* Added a mutex to avoid races during import-like operations in protected guests
* The functions uv_convert_from_secure_folio() and uv_destroy_folio() now work on the whole folio, and not just its first page
* Added warnings if uv_convert_from_secure_pte() fails in ptep_*()
* Changed the way shadow gmaps do reference counting
* Use a bitmap for gmap flags, instead of individual bool variables
* Rework the make_secure logic to work on gmap only, and not on userspace addresses
* Some refactorings
* Rebased onto 6.19-rc2
* Added more comments
* Minor style fixes

v4->v5:
* fix some locking issues
* fix comments and documentation

v3->v4:
* dat_link() can now return -ENOMEM when appropriate
* fixed a few vSIE races that led to use-after-free or deadlocks
* split part of the previous patch 23 and move it after patch 17, merge the rest of the patch into patch 19
* fix -ENOMEM handling in handle_pfmf() and handle_sske()

v2->v3:
* Add lots of small comments and cosmetic fixes
* Rename some functions to improve clarity
* Remove unused helper functions and macros
* Rename inline asm constraints labels to make them more understandable
* Refactor the code to pre-allocate the page tables (using custom caches) when sleeping is allowed, use the cached pages when holding spinlocks and handle gracefully allocation failures (i.e. retry instead of killing the guest)
* Refactor the code for fault handling; it's now in a separate file, and it takes a callback that can be optionally called when all the relevant locks are still held
* Use assembler mnemonics instead of manually specifying the opcode where appropriate
* Remove the LEVEL_* enum, and use TABLE_TYPE_* macros instead; introduce new TABLE_TYPE_PAGE_TABLE
* Remove usage of cpu_has_idte() since it is being removed from the kernel
* Improve storage key handling and PGSTE locking
* Introduce struct guest_fault to represent the state of a guest fault that is being resolved
* Minor CMMA fixes

Claudio Imbrenda (29):
KVM: s390: Refactor pgste lock and unlock functions
KVM: s390: Add P bit in table entry bitfields, move union vaddress
s390: Make UV folio operations work on whole folio
s390: Move sske_frame() to a header
KVM: s390: Add gmap_helper_set_unused()
KVM: s390: Introduce import_lock
KVM: s390: Export two functions
s390/mm: Warn if uv_convert_from_secure_pte() fails
KVM: s390: vsie: Pass gmap explicitly as parameter
KVM: s390: Enable KVM_GENERIC_MMU_NOTIFIER
KVM: s390: Rename some functions in gaccess.c
KVM: s390: KVM-specific bitfields and helper functions
KVM: s390: KVM page table management functions: allocation
KVM: s390: KVM page table management functions: clear and replace
KVM: s390: KVM page table management functions: walks
KVM: s390: KVM page table management functions: storage keys
KVM: s390: KVM page table management functions: lifecycle management
KVM: s390: KVM page table management functions: CMMA
KVM: s390: New gmap code
KVM: s390: Add helper functions for fault handling
KVM: s390: Add some helper functions needed for vSIE
KVM: s390: Stop using CONFIG_PGSTE
KVM: s390: Storage key functions refactoring
KVM: s390: Switch to new gmap
KVM: s390: Remove gmap from s390/mm
KVM: S390: Remove PGSTE code from linux/s390 mm
KVM: s390: Enable 1M pages for gmap
KVM: s390: Storage key manipulation IOCTL
KVM: s390: selftests: Add selftest for the KVM_S390_KEYOP ioctl

 Documentation/virt/kvm/api.rst           |   42 +
 MAINTAINERS                              |    2 -
 arch/s390/Kconfig                        |    3 -
 arch/s390/include/asm/dat-bits.h         |   32 +-
 arch/s390/include/asm/gmap.h             |  174 --
 arch/s390/include/asm/gmap_helpers.h     |    1 +
 arch/s390/include/asm/hugetlb.h          |    6 -
 arch/s390/include/asm/kvm_host.h         |    7 +
 arch/s390/include/asm/mmu.h              |   13 -
 arch/s390/include/asm/mmu_context.h      |    6 +-
 arch/s390/include/asm/page.h             |    4 -
 arch/s390/include/asm/pgalloc.h          |    4 -
 arch/s390/include/asm/pgtable.h          |  171 +-
 arch/s390/include/asm/tlb.h              |    3 -
 arch/s390/include/asm/uaccess.h          |   70 +-
 arch/s390/include/asm/uv.h               |    3 +-
 arch/s390/kernel/uv.c                    |  142 +-
 arch/s390/kvm/Kconfig                    |    2 +
 arch/s390/kvm/Makefile                   |    3 +-
 arch/s390/kvm/dat.c                      | 1391 ++++++++++++
 arch/s390/kvm/dat.h                      |  970 +++++++++
 arch/s390/kvm/diag.c                     |    2 +-
 arch/s390/kvm/faultin.c                  |  148 ++
 arch/s390/kvm/faultin.h                  |   92 +
 arch/s390/kvm/gaccess.c                  |  958 +++++----
 arch/s390/kvm/gaccess.h                  |   20 +-
 arch/s390/kvm/gmap-vsie.c                |  141 --
 arch/s390/kvm/gmap.c                     | 1235 +++++++++++
 arch/s390/kvm/gmap.h                     |  244 +++
 arch/s390/kvm/intercept.c                |   15 +-
 arch/s390/kvm/interrupt.c                |    6 +-
 arch/s390/kvm/kvm-s390.c                 |  954 ++++-----
 arch/s390/kvm/kvm-s390.h                 |   27 +-
 arch/s390/kvm/priv.c                     |  213 +-
 arch/s390/kvm/pv.c                       |  177 +-
 arch/s390/kvm/vsie.c                     |  192 +-
 arch/s390/lib/uaccess.c                  |  184 +-
 arch/s390/mm/Makefile                    |    1 -
 arch/s390/mm/fault.c                     |    4 +-
 arch/s390/mm/gmap.c                      | 2436 ----------------------
 arch/s390/mm/gmap_helpers.c              |   96 +-
 arch/s390/mm/hugetlbpage.c               |   24 -
 arch/s390/mm/page-states.c               |    1 +
 arch/s390/mm/pageattr.c                  |    7 -
 arch/s390/mm/pgalloc.c                   |   24 -
 arch/s390/mm/pgtable.c                   |  814 +-------
 include/uapi/linux/kvm.h                 |   11 +
 mm/khugepaged.c                          |    9 -
 tools/testing/selftests/kvm/Makefile.kvm |    1 +
 tools/testing/selftests/kvm/s390/keyop.c |  299 +++
 50 files changed, 5889 insertions(+), 5495 deletions(-)
 delete mode 100644 arch/s390/include/asm/gmap.h
 create mode 100644 arch/s390/kvm/dat.c
 create mode 100644 arch/s390/kvm/dat.h
 create mode 100644 arch/s390/kvm/faultin.c
 create mode 100644 arch/s390/kvm/faultin.h
 delete mode 100644 arch/s390/kvm/gmap-vsie.c
 create mode 100644 arch/s390/kvm/gmap.c
 create mode 100644 arch/s390/kvm/gmap.h
 delete mode 100644 arch/s390/mm/gmap.c
 create mode 100644 tools/testing/selftests/kvm/s390/keyop.c
Signature · 1 line
2.52.0