Skip to contentSkip to conversation

KVM: s390: Implement KVM_PRE_FAULT_MEMORY

PATCH In review Claudio Imbrenda 3 participants 9 messages May 27

Subsystems (5)
Documentation KVM Kselftest s390 +1 more subsystems
lore
Share & tools
lore

Claudio Imbrenda

author maintainer
#0 lore raw

v3 cover KVM: s390: Implement KVM_PRE_FAULT_MEMORY

Implement KVM_PRE_FAULT_MEMORY on s390, and update documentation and selftests accordingly.

Faulted-in pages will be marked as accessed, unlike x86, otherwise they will trigger a minor fault when accessed. Avoiding such faults is one of the points of KVM_PRE_FAULT_MEMORY.

This series depends on some kvm/s390 functions that are slightly buggy at the moment. Those functions are being fixed in a separate series. Even without those fixes, this series is not buggy, it's just a bit inefficient in some corner cases.

CCing x86 people due to changes in the documentation and the selftests just for completeness, since the selftest has been acked already.

v2->v3:
* Fix typos
* Correctly indicate page size in gmap_handle_minor_crste_fault() also in case nothing needed to be done.
* Pick up and acked-by

v1->v2:
* Add the crste_region3 flag in struct guest_fault to keep track of the page size used to solve a fault.
* Rewrite the s390 version of kvm_arch_vcpu_pre_fault_memory() so it looks and behaves similarly to the x86 version. No more loops, better handling of hugepage mappings, and more correct error handling.
* Remove the assertion vcpu->run->exit_reason == KVM_EXIT_IO from the selftest, as it is redundant and x86-specific.

Claudio Imbrenda (5):
KVM: s390: Track page size in struct guest_fault
KVM: s390: Implement KVM_PRE_FAULT_MEMORY
KVM: s390: Update KVM_PRE_FAULT_MEMORY API documentation
KVM: selftests: Fix pre_fault_memory_test to run on s390
KVM: selftests: Enable pre_fault_memory_test for s390

 Documentation/virt/kvm/api.rst                |  5 ++-
 arch/s390/kvm/Kconfig                         |  1 +
 arch/s390/kvm/dat.h                           |  1 +
 arch/s390/kvm/gmap.c                          |  2 +
 arch/s390/kvm/kvm-s390.c                      | 45 +++++++++++++++++++
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../selftests/kvm/pre_fault_memory_test.c     |  7 +--
 7 files changed, 54 insertions(+), 8 deletions(-)
Signature · 1 line
2.54.0
author maintainer
#1 lore raw

PATCH v3 1/5 KVM: s390: Track page size in struct guest_fault

Until now, the members of struct guest_fault are always accessed while holding the required locks, and thus the ptep and crstep pointers can be dereferenced safely.

There will be some new cases where callers of kvm_s390_faultin_gfn() need to know the size of the page used to solve the fault, at which point no locks are held anymore, and dereferencing the crstep field is not possible.

Introduce a new crste_region3 flag for struct guest_fault to indicate whether the crstep used to solve the fault was a region 3 entry with FC=1 (large pud).

This allows to disambiguate all three possible scenarios:
* If ptep is not NULL, the fault was solved with a pte.
* If ptep is NULL and crste_region3 is 0, a segment entry with FC=1 (large pmd) was used.
* If ptep is NULL and crste_region3 is 1, a region 3 entry with FC=1 (large pud) was used.

---
 arch/s390/kvm/dat.h  | 1 +
 arch/s390/kvm/gmap.c | 2 ++
 2 files changed, 3 insertions(+)
arch/s390/kvm/dat.h +1 -0
index 8f8278c44879..5d848e74e382 100644
@@ -500,6 +500,7 @@ struct guest_fault {
 	bool write_attempt;	/* Write access attempted */
 	bool attempt_pfault;	/* Attempt a pfault first */
 	bool valid;		/* This entry contains valid data */
+	bool crste_region3;     /* Whether crstep refers to a region3 entry */
 	void (*callback)(struct guest_fault *f);
 	void *priv;
 };
arch/s390/kvm/gmap.c +2 -0
index 3c26e35af0ef..fe138d17caaf 100644
@@ -531,6 +531,7 @@ static int gmap_handle_minor_crste_fault(struct gmap *gmap, struct guest_fault *
 	f->pfn = PHYS_PFN(large_crste_to_phys(oldcrste, f->gfn));
 	f->writable = oldcrste.s.fc1.w;
 
+	f->crste_region3 = is_pud(oldcrste);
 	/* Appropriate permissions already (race with another handler), nothing to do. */
 	if (!oldcrste.h.i && !(f->write_attempt && oldcrste.h.p))
 		return 0;
@@ -690,6 +691,7 @@ static int _gmap_link(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, int leve
 			if (oldval.val != _CRSTE_EMPTY(oldval.h.tt).val &&
 			    crste_origin_large(oldval) != crste_origin_large(newval))
 				return -EAGAIN;
+			f->crste_region3 = is_pud(newval);
 		} while (!gmap_crstep_xchg_atomic(gmap, f->crstep, oldval, newval, f->gfn));
 		if (f->callback)
 			f->callback(f);
Signature · 1 line
2.54.0
#2 lore raw

Re: [PATCH v3 1/5] KVM: s390: Track page size in struct guest_fault

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Critical] kvm_s390_faultin_gfn() silently returns false success when a concurrent MMU invalidation occurs, skipping the gmap_link() map operation and memory access callbacks. --

commit d74f84462345c6d6532552f9a640e4eaeed03bc3
Author: Claudio Imbrenda <[email protected]>

KVM: s390: Track page size in struct guest_fault

This patch adds a crste_region3 flag to struct guest_fault to indicate whether the crstep used to solve a fault is a region 3 entry with FC=1 (large pud). This allows callers of kvm_s390_faultin_gfn() to determine the page size used to solve the fault even when locks are no longer held.

arch/s390/kvm/dat.h struct guest_fault { · line 504

Quoted 9 lines
arch/s390/kvm/dat.h +1 -0
index 873e13ac5a27f..fad605305e05c 100644
@@ -501,6 +501,7 @@ struct guest_fault {
 	bool write_attempt;	/* Write access attempted */
 	bool attempt_pfault;	/* Attempt a pfault first */
 	bool valid;		/* This entry contains valid data */
+	bool crste_region3;     /* Whether crstep refers to a region3 entry */

[Severity: Critical]
This is a pre-existing issue, but does kvm_s390_faultin_gfn() silently return false success when a concurrent MMU invalidation occurs, skipping the gmap_link() map operation and memory access callbacks?

In kvm_s390_faultin_gfn(), the local variable rc is initialized to 0. Inside the while (1) loop, if mmu_invalidate_retry_gfn() inside the mmu_lock returns true, the code skips the gmap_link() call.

Because gmap_link() is skipped, rc is not updated and remains 0. The loop then evaluates "else if (rc != -EAGAIN)" and immediately returns 0 (success) instead of retrying:

arch/s390/kvm/faultin.c:kvm_s390_faultin_gfn() {
    ...
    int rc = 0;
    ...
    while (1) {
        ...
        scoped_guard(read_lock, &kvm->mmu_lock) {
            if (!mmu_invalidate_retry_gfn(kvm, inv_seq, f->gfn)) {
                f->valid = true;
                rc = gmap_link(mc, kvm->arch.gmap, f, slot);
                ...
            }
        }
        ...
        } else if (rc != -EAGAIN) {
            return rc;
        }
    }
}

When called via access_guest_page_with_key_gpa(), this false success causes the caller to return context.exception (implicitly 0). The actual memory access callback _access_guest_page_with_key_gpa() is never executed because it is invoked from within gmap_link().

For memory reads, the host kernel sees a successful return but the output buffer remains completely untouched, potentially leading to guest state corruption or information leaks.

Furthermore, for the KVM_PRE_FAULT_MEMORY API which relies on this function to populate f.ptep and the newly added f.crste_region3, if skipped, these fields retain their zero-initialized values, causing the ioctl to silently skip up to 1MB of memory while incorrectly reporting progress to userspace.

Quoted 3 lines
arch/s390/kvm/dat.h +0 -0
 	void (*callback)(struct guest_fault *f);
 	void *priv;
 };
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH v3 2/5 KVM: s390: Implement KVM_PRE_FAULT_MEMORY

In reply to Claudio Imbrenda

Implement and enable the KVM_PRE_FAULT_MEMORY ioctl for s390.

Faulted-in pages will be marked as accessed, unlike x86, otherwise they will trigger a minor fault when accessed. Avoiding such faults is one of the points of KVM_PRE_FAULT_MEMORY.

---
 arch/s390/kvm/Kconfig    |  1 +
 arch/s390/kvm/kvm-s390.c | 45 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
arch/s390/kvm/Kconfig +1 -0
index 5b835bc6a194..8d3ee17a1bcb 100644
@@ -30,6 +30,7 @@ config KVM
 	select KVM_VFIO
 	select VIRT_XFER_TO_GUEST_WORK
 	select KVM_MMU_LOCKLESS_AGING
+	select KVM_GENERIC_PRE_FAULT_MEMORY
 	help
 	  Support hosting paravirtualized guest machines using the SIE
 	  virtualization capability on the mainframe. This should work
arch/s390/kvm/kvm-s390.c +45 -0

57 lines not on this page · raw message

Signature · 1 line
2.54.0
#4 lore raw
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [Medium] The error check if (rc > 0) return -EIO; is unreachable and fails to handle page table walk exceptions.

Pre-existing issues:
- [High] kvm_s390_faultin_gfn() incorrectly returns success (0) when it races with an MMU notifier, leading to silent memory skips in KVM_PRE_FAULT_MEMORY and potential leaks of uninitialized host memory in access_guest_page_with_key_gpa().
- [High] KVM_PRE_FAULT_MEMORY can trigger a deterministic host kernel infinite loop if the requested GPA is within a KVM memslot but outside the gmap ASCE limit.
--

commit f2d2595771b18daab539c7480809ae63c3707ec4
Author: Claudio Imbrenda <[email protected]>

KVM: s390: Implement KVM_PRE_FAULT_MEMORY

Implement and enable the KVM_PRE_FAULT_MEMORY ioctl for s390.

Faulted-in pages will be marked as accessed, unlike x86, otherwise they will trigger a minor fault when accessed. Avoiding such faults is one of the points of KVM_PRE_FAULT_MEMORY.

Quoted 3 lines
arch/s390/kvm/kvm-s390.c +0 -0

[ ... ]

arch/s390/kvm/kvm-s390.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 5769

Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10 -0
+long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, struct kvm_pre_fault_memory *range)
+{
+	struct guest_fault f = { .gfn = gpa_to_gfn(range->gpa), };
+	gpa_t end;
+	int rc;
+
+	if (kvm_is_ucontrol(vcpu->kvm))
+		return -EINVAL;
+
+	rc = kvm_s390_faultin_gfn(vcpu, NULL, &f);

[Severity: High]
This is a pre-existing issue, but does kvm_s390_faultin_gfn() incorrectly return 0 when it races with an MMU notifier?

In kvm_s390_faultin_gfn(), rc is initialized to 0. Inside the retry loop, if mmu_invalidate_retry_gfn() evaluates to true due to a concurrent MMU invalidation, gmap_link() is skipped. But there is no else branch to set rc = -EAGAIN. rc remains 0, the loop terminates matching else if (rc != -EAGAIN), and the function incorrectly returns 0 (success) while dropping the page reference.

For KVM_PRE_FAULT_MEMORY, does this cause kvm_arch_vcpu_pre_fault_memory() to falsely assume a huge page was mapped (since f.ptep remains NULL) and silently skip up to 1MB of memory instead of retrying?

[Severity: High]
This is also a pre-existing issue, but can KVM_PRE_FAULT_MEMORY trigger a deterministic host kernel infinite loop if the requested GPA is within a KVM memslot but outside the gmap ASCE limit?

If userspace provides a GPA that falls within a valid KVM memslot but exceeds the bounds of the guest physical address space limit (gmap->asce), __kvm_faultin_pfn() successfully returns a PFN. When gmap_link() calls dat_entry_walk(), it detects the out-of-bounds access and returns PGM_ADDRESSING (5).

However, _gmap_link() blindly converts any error that is not -ENOMEM or -EINVAL to -EAGAIN. Does kvm_s390_faultin_gfn() receive -EAGAIN and continuously retry the exact same steps in its while(1) loop, causing an unkillable infinite loop on the host CPU since __kvm_faultin_pfn() does not yield or check for signals?

arch/s390/kvm/kvm-s390.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 5772

Quoted 3 lines
arch/s390/kvm/kvm-s390.c +3 -0
+	if (rc == PGM_ADDRESSING)
+		return -ENOENT;
+	if (rc > 0)

[Severity: Medium]
Is this error check unreachable?

kvm_s390_faultin_gfn() only ever returns > 0 for PGM_ADDRESSING, which is already explicitly caught and converted to -ENOENT right above this check. Any other PGM_ exception code from the page table walk (like PGM_PROTECTION or PGM_TRANSLATION_SPEC) is masked by _gmap_link() returning -EAGAIN, which then causes an infinite loop in kvm_s390_faultin_gfn().

Can rc > 0 ever evaluate to true here?

Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10 -0
+		return -EIO;
+	if (rc < 0)
+		return rc;
+
+	if (f.ptep)
+		return PAGE_SIZE;
+
+	end = ALIGN(range->gpa + PAGE_SIZE, f.crste_region3 ? _REGION3_SIZE : HPAGE_SIZE);
+	return min(range->size, end - range->gpa);
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author maintainer
#5 lore raw

PATCH v3 3/5 KVM: s390: Update KVM_PRE_FAULT_MEMORY API documentation

In reply to Claudio Imbrenda

Update the API documentation for KVM_PRE_FAULT_MEMORY to account for its s390 implementation.

---
 Documentation/virt/kvm/api.rst | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Documentation/virt/kvm/api.rst +3 -2
index 52bbbb553ce1..e7998feaa940 100644
@@ -6471,7 +6471,8 @@ Errors:
 
   ========== ===============================================================
   EINVAL     The specified `gpa` and `size` were invalid (e.g. not
-             page aligned, causes an overflow, or size is zero).
+             page aligned, causes an overflow, or size is zero), or the VM
+             is UCONTROL (s390).
   ENOENT     The specified `gpa` is outside defined memslots.
   EINTR      An unmasked signal is pending and no page was processed.
   EFAULT     The parameter address was invalid.
@@ -6494,7 +6495,7 @@ Errors:
 KVM_PRE_FAULT_MEMORY populates KVM's stage-2 page tables used to map memory
 for the current vCPU state.  KVM maps memory as if the vCPU generated a
 stage-2 read page fault, e.g. faults in memory as needed, but doesn't break
-CoW.  However, KVM does not mark any newly created stage-2 PTE as Accessed.
+CoW.  On x86, KVM does not mark any newly created stage-2 PTE as Accessed.
 
 In the case of confidential VM types where there is an initial set up of
 private guest memory before the guest is 'finalized'/measured, this ioctl
Signature · 1 line
2.54.0
author maintainer
#6 lore raw

PATCH v3 4/5 KVM: selftests: Fix pre_fault_memory_test to run on s390

In reply to Claudio Imbrenda

Add a missing #include <ucall_common.h> which is needed and otherwise not included on s390.

Remove the assertion vcpu->run->exit_reason == KVM_EXIT_IO since it is x86-specific and redundant anyway.

---
 tools/testing/selftests/kvm/pre_fault_memory_test.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
tools/testing/selftests/kvm/pre_fault_memory_test.c +1 -6
index fcb57fd034e6..a0fcae3cb7a8 100644
@@ -11,6 +11,7 @@
 #include <kvm_util.h>
 #include <processor.h>
 #include <pthread.h>
+#include <ucall_common.h>
 
 /* Arbitrarily chosen values */
 #define TEST_SIZE		(SZ_2M + PAGE_SIZE)
@@ -167,7 +168,6 @@ static void __test_pre_fault_memory(unsigned long vm_type, bool private)
 		.type = vm_type,
 	};
 	struct kvm_vcpu *vcpu;
-	struct kvm_run *run;
 	struct kvm_vm *vm;
 	struct ucall uc;
 
@@ -193,11 +193,6 @@ static void __test_pre_fault_memory(unsigned long vm_type, bool private)
 	vcpu_args_set(vcpu, 1, gva);
 	vcpu_run(vcpu);
 
-	run = vcpu->run;
-	TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
-		    "Wanted KVM_EXIT_IO, got exit reason: %u (%s)",
-		    run->exit_reason, exit_reason_str(run->exit_reason));
-
 	switch (get_ucall(vcpu, &uc)) {
 	case UCALL_ABORT:
 		REPORT_GUEST_ASSERT(uc);
Signature · 1 line
2.54.0
author maintainer
#7 lore raw

PATCH v3 5/5 KVM: selftests: Enable pre_fault_memory_test for s390

In reply to Claudio Imbrenda

Enable the pre_fault_memory_test to run on s390.

---
 tools/testing/selftests/kvm/Makefile.kvm | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 9118a5a51b89..fff939db89cd 100644
@@ -210,6 +210,7 @@ TEST_GEN_PROGS_s390 += s390/keyop
 TEST_GEN_PROGS_s390 += rseq_test
 TEST_GEN_PROGS_s390 += s390/irq_routing
 TEST_GEN_PROGS_s390 += mmu_stress_test
+TEST_GEN_PROGS_s390 += pre_fault_memory_test
 
 TEST_GEN_PROGS_riscv = $(TEST_GEN_PROGS_COMMON)
 TEST_GEN_PROGS_riscv += riscv/sbi_pmu_test
Signature · 1 line
2.54.0
Reviewed-by on the v3 series
#8 lore raw
In reply to Claudio Imbrenda
Quoted 3 lines · Claudio Imbrenda

Implement KVM_PRE_FAULT_MEMORY on s390, and update documentation and selftests accordingly.

...

Quoted 6 lines

Claudio Imbrenda (5):
KVM: s390: Track page size in struct guest_fault
KVM: s390: Implement KVM_PRE_FAULT_MEMORY
KVM: s390: Update KVM_PRE_FAULT_MEMORY API documentation
KVM: selftests: Fix pre_fault_memory_test to run on s390
KVM: selftests: Enable pre_fault_memory_test for s390

LGTM.

For the whole series: