KVM: s390: Implement KVM_PRE_FAULT_MEMORY
Subsystems (5)
Claudio Imbrenda
maintainerv3 cover KVM: s390: Implement KVM_PRE_FAULT_MEMORY
Implement KVM_PRE_FAULT_MEMORY on s390, and update documentation and selftests accordingly.
Faulted-in pages will be marked as accessed, unlike x86, otherwise they will trigger a minor fault when accessed. Avoiding such faults is one of the points of KVM_PRE_FAULT_MEMORY.
This series depends on some kvm/s390 functions that are slightly buggy at the moment. Those functions are being fixed in a separate series. Even without those fixes, this series is not buggy, it's just a bit inefficient in some corner cases.
CCing x86 people due to changes in the documentation and the selftests just for completeness, since the selftest has been acked already.
v2->v3:
* Fix typos
* Correctly indicate page size in gmap_handle_minor_crste_fault() also in case nothing needed to be done.
* Pick up and acked-by
v1->v2:
* Add the crste_region3 flag in struct guest_fault to keep track of the page size used to solve a fault.
* Rewrite the s390 version of kvm_arch_vcpu_pre_fault_memory() so it looks and behaves similarly to the x86 version. No more loops, better handling of hugepage mappings, and more correct error handling.
* Remove the assertion vcpu->run->exit_reason == KVM_EXIT_IO from the selftest, as it is redundant and x86-specific.
Claudio Imbrenda (5):
KVM: s390: Track page size in struct guest_fault
KVM: s390: Implement KVM_PRE_FAULT_MEMORY
KVM: s390: Update KVM_PRE_FAULT_MEMORY API documentation
KVM: selftests: Fix pre_fault_memory_test to run on s390
KVM: selftests: Enable pre_fault_memory_test for s390
Documentation/virt/kvm/api.rst | 5 ++- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/dat.h | 1 + arch/s390/kvm/gmap.c | 2 + arch/s390/kvm/kvm-s390.c | 45 +++++++++++++++++++ tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/pre_fault_memory_test.c | 7 +-- 7 files changed, 54 insertions(+), 8 deletions(-)
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH v3 1/5 KVM: s390: Track page size in struct guest_fault
Until now, the members of struct guest_fault are always accessed while holding the required locks, and thus the ptep and crstep pointers can be dereferenced safely.
There will be some new cases where callers of kvm_s390_faultin_gfn() need to know the size of the page used to solve the fault, at which point no locks are held anymore, and dereferencing the crstep field is not possible.
Introduce a new crste_region3 flag for struct guest_fault to indicate whether the crstep used to solve the fault was a region 3 entry with FC=1 (large pud).
This allows to disambiguate all three possible scenarios:
* If ptep is not NULL, the fault was solved with a pte.
* If ptep is NULL and crste_region3 is 0, a segment entry with FC=1 (large pmd) was used.
* If ptep is NULL and crste_region3 is 1, a region 3 entry with FC=1 (large pud) was used.
--- arch/s390/kvm/dat.h | 1 + arch/s390/kvm/gmap.c | 2 ++ 2 files changed, 3 insertions(+)
arch/s390/kvm/dat.h +1 -0
@@ -500,6 +500,7 @@ struct guest_fault {bool write_attempt;/* Write access attempted */ bool attempt_pfault;/* Attempt a pfault first */ bool valid;/* This entry contains valid data */ +bool crste_region3;/* Whether crstep refers to a region3 entry */ void (*callback)(struct guest_fault *f);void *priv; };
arch/s390/kvm/gmap.c +2 -0
@@ -531,6 +531,7 @@ static int gmap_handle_minor_crste_fault(struct gmap *gmap, struct guest_fault * f->pfn =PHYS_PFN (large_crste_to_phys (oldcrste, f->gfn)); f->writable = oldcrste.s.fc1.w; + f->crste_region3 =is_pud (oldcrste);/* Appropriate permissions already (race with another handler), nothing to do. */ if (!oldcrste.h.i && !(f->write_attempt && oldcrste.h.p))return 0 ;
@@ -690,6 +691,7 @@ static int _gmap_link(struct kvm_s390_mmu_cache *mc, struct gmap *gmap, int leveif (oldval.val !=_CRSTE_EMPTY (oldval.h.tt).val &&crste_origin_large (oldval) !=crste_origin_large (newval))return -EAGAIN ; + f->crste_region3 =is_pud (newval); }while (!gmap_crstep_xchg_atomic (gmap, f->crstep, oldval, newval, f->gfn));if (f->callback) f->callback (f);
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v3 1/5] KVM: s390: Track page size in struct guest_fault
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] kvm_s390_faultin_gfn() silently returns false success when a concurrent MMU invalidation occurs, skipping the gmap_link() map operation and memory access callbacks. --
commit d74f84462345c6d6532552f9a640e4eaeed03bc3
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Track page size in struct guest_fault
This patch adds a crste_region3 flag to struct guest_fault to indicate whether the crstep used to solve a fault is a region 3 entry with FC=1 (large pud). This allows callers of kvm_s390_faultin_gfn() to determine the page size used to solve the fault even when locks are no longer held.
arch/s390/kvm/dat.h struct guest_fault { · line 504
Quoted 9 lines
arch/s390/kvm/dat.h +1
-0@@ -501,6 +501,7 @@ struct guest_fault {bool write_attempt;/* Write access attempted */ bool attempt_pfault;/* Attempt a pfault first */ bool valid;/* This entry contains valid data */ +bool crste_region3;/* Whether crstep refers to a region3 entry */
[Severity: Critical]
This is a pre-existing issue, but does kvm_s390_faultin_gfn() silently return false success when a concurrent MMU invalidation occurs, skipping the gmap_link() map operation and memory access callbacks?
In kvm_s390_faultin_gfn(), the local variable rc is initialized to 0. Inside the while (1) loop, if mmu_invalidate_retry_gfn() inside the mmu_lock returns true, the code skips the gmap_link() call.
Because gmap_link() is skipped, rc is not updated and remains 0. The loop then evaluates "else if (rc != -EAGAIN)" and immediately returns 0 (success) instead of retrying:
arch/s390/kvm/faultin.c:kvm_s390_faultin_gfn() {
...
int rc = 0;
...
while (1) {
...
scoped_guard(read_lock, &kvm->mmu_lock) {
if (!mmu_invalidate_retry_gfn(kvm, inv_seq, f->gfn)) {
f->valid = true;
rc = gmap_link(mc, kvm->arch.gmap, f, slot);
...
}
}
...
} else if (rc != -EAGAIN) {
return rc;
}
}
}When called via access_guest_page_with_key_gpa(), this false success causes the caller to return context.exception (implicitly 0). The actual memory access callback _access_guest_page_with_key_gpa() is never executed because it is invoked from within gmap_link().
For memory reads, the host kernel sees a successful return but the output buffer remains completely untouched, potentially leading to guest state corruption or information leaks.
Furthermore, for the KVM_PRE_FAULT_MEMORY API which relies on this function to populate f.ptep and the newly added f.crste_region3, if skipped, these fields retain their zero-initialized values, causing the ioctl to silently skip up to 1MB of memory while incorrectly reporting progress to userspace.
Quoted 3 lines
arch/s390/kvm/dat.h +0
-0void (*callback)(struct guest_fault *f);void *priv; };
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Claudio Imbrenda
maintainerPATCH v3 2/5 KVM: s390: Implement KVM_PRE_FAULT_MEMORY
In reply to Claudio ImbrendaImplement and enable the KVM_PRE_FAULT_MEMORY ioctl for s390.
Faulted-in pages will be marked as accessed, unlike x86, otherwise they will trigger a minor fault when accessed. Avoiding such faults is one of the points of KVM_PRE_FAULT_MEMORY.
--- arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/kvm-s390.c | 45 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+)
arch/s390/kvm/Kconfig +1 -0
@@ -30,6 +30,7 @@ config KVM select KVM_VFIO select VIRT_XFER_TO_GUEST_WORK select KVM_MMU_LOCKLESS_AGING + select KVM_GENERIC_PRE_FAULT_MEMORY help Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should work
arch/s390/kvm/kvm-s390.c +45 -0
57 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [Medium] The error check if (rc > 0) return -EIO; is unreachable and fails to handle page table walk exceptions.
Pre-existing issues:
- [High] kvm_s390_faultin_gfn() incorrectly returns success (0) when it races with an MMU notifier, leading to silent memory skips in KVM_PRE_FAULT_MEMORY and potential leaks of uninitialized host memory in access_guest_page_with_key_gpa().
- [High] KVM_PRE_FAULT_MEMORY can trigger a deterministic host kernel infinite loop if the requested GPA is within a KVM memslot but outside the gmap ASCE limit.
--
commit f2d2595771b18daab539c7480809ae63c3707ec4
Author: Claudio Imbrenda <[email protected]>
KVM: s390: Implement KVM_PRE_FAULT_MEMORY
Implement and enable the KVM_PRE_FAULT_MEMORY ioctl for s390.
Faulted-in pages will be marked as accessed, unlike x86, otherwise they will trigger a minor fault when accessed. Avoiding such faults is one of the points of KVM_PRE_FAULT_MEMORY.
Quoted 3 lines
arch/s390/kvm/kvm-s390.c +0
-0
[ ... ]
arch/s390/kvm/kvm-s390.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 5769
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+long kvm_arch_vcpu_pre_fault_memory (struct kvm_vcpu *vcpu,struct kvm_pre_fault_memory *range) +{ +struct guest_fault f = { .gfn =gpa_to_gfn (range->gpa), }; +gpa_t end; +int rc; + +if (kvm_is_ucontrol (vcpu->kvm)) +return -EINVAL ; + + rc =kvm_s390_faultin_gfn (vcpu,NULL , &f);
[Severity: High]
This is a pre-existing issue, but does kvm_s390_faultin_gfn() incorrectly return 0 when it races with an MMU notifier?
In kvm_s390_faultin_gfn(), rc is initialized to 0. Inside the retry loop, if mmu_invalidate_retry_gfn() evaluates to true due to a concurrent MMU invalidation, gmap_link() is skipped. But there is no else branch to set rc = -EAGAIN. rc remains 0, the loop terminates matching else if (rc != -EAGAIN), and the function incorrectly returns 0 (success) while dropping the page reference.
For KVM_PRE_FAULT_MEMORY, does this cause kvm_arch_vcpu_pre_fault_memory() to falsely assume a huge page was mapped (since f.ptep remains NULL) and silently skip up to 1MB of memory instead of retrying?
[Severity: High]
This is also a pre-existing issue, but can KVM_PRE_FAULT_MEMORY trigger a deterministic host kernel infinite loop if the requested GPA is within a KVM memslot but outside the gmap ASCE limit?
If userspace provides a GPA that falls within a valid KVM memslot but exceeds the bounds of the guest physical address space limit (gmap->asce), __kvm_faultin_pfn() successfully returns a PFN. When gmap_link() calls dat_entry_walk(), it detects the out-of-bounds access and returns PGM_ADDRESSING (5).
However, _gmap_link() blindly converts any error that is not -ENOMEM or -EINVAL to -EAGAIN. Does kvm_s390_faultin_gfn() receive -EAGAIN and continuously retry the exact same steps in its while(1) loop, causing an unkillable infinite loop on the host CPU since __kvm_faultin_pfn() does not yield or check for signals?
arch/s390/kvm/kvm-s390.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 5772
Quoted 3 lines
arch/s390/kvm/kvm-s390.c +3
-0+if (rc ==PGM_ADDRESSING ) +return -ENOENT ; +if (rc >0 )
[Severity: Medium]
Is this error check unreachable?
kvm_s390_faultin_gfn() only ever returns > 0 for PGM_ADDRESSING, which is already explicitly caught and converted to -ENOENT right above this check. Any other PGM_ exception code from the page table walk (like PGM_PROTECTION or PGM_TRANSLATION_SPEC) is masked by _gmap_link() returning -EAGAIN, which then causes an infinite loop in kvm_s390_faultin_gfn().
Can rc > 0 ever evaluate to true here?
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+return -EIO ; +if (rc <0 ) +return rc; + +if (f.ptep) +return PAGE_SIZE ; + + end =ALIGN (range->gpa +PAGE_SIZE , f.crste_region3 ? _REGION3_SIZE :HPAGE_SIZE ); +return min (range->size, end - range->gpa); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Claudio Imbrenda
maintainerPATCH v3 3/5 KVM: s390: Update KVM_PRE_FAULT_MEMORY API documentation
In reply to Claudio ImbrendaUpdate the API documentation for KVM_PRE_FAULT_MEMORY to account for its s390 implementation.
--- Documentation/virt/kvm/api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Documentation/virt/kvm/api.rst +3 -2
@@ -6471,7 +6471,8 @@ Errors: ========== =============================================================== EINVAL The specified `gpa` and `size` were invalid (e.g. not - page aligned, causes an overflow, or size is zero). + page aligned, causes an overflow, or size is zero), or the VM + is UCONTROL (s390). ENOENT The specified `gpa` is outside defined memslots. EINTR An unmasked signal is pending and no page was processed. EFAULT The parameter address was invalid.
@@ -6494,7 +6495,7 @@ Errors: KVM_PRE_FAULT_MEMORY populates KVM's stage-2 page tables used to map memory for the current vCPU state. KVM maps memory as if the vCPU generated a stage-2 read page fault, e.g. faults in memory as needed, but doesn't break -CoW. However, KVM does not mark any newly created stage-2 PTE as Accessed. +CoW. On x86, KVM does not mark any newly created stage-2 PTE as Accessed. In the case of confidential VM types where there is an initial set up of private guest memory before the guest is 'finalized'/measured, this ioctl
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH v3 4/5 KVM: selftests: Fix pre_fault_memory_test to run on s390
In reply to Claudio ImbrendaAdd a missing #include <ucall_common.h> which is needed and otherwise not included on s390.
Remove the assertion vcpu->run->exit_reason == KVM_EXIT_IO since it is x86-specific and redundant anyway.
--- tools/testing/selftests/kvm/pre_fault_memory_test.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-)
tools/testing/selftests/kvm/pre_fault_memory_test.c +1 -6
@@ -11,6 +11,7 @@#include <kvm_util.h> #include <processor.h> #include <pthread.h> +#include <ucall_common.h> /* Arbitrarily chosen values */ #define TEST_SIZE (SZ_2M + PAGE_SIZE)
@@ -167,7 +168,6 @@ static void __test_pre_fault_memory(unsigned long vm_type, bool private) .type = vm_type, };struct kvm_vcpu *vcpu; -struct kvm_run *run;struct kvm_vm *vm;struct ucall uc;
@@ -193,11 +193,6 @@ static void __test_pre_fault_memory(unsigned long vm_type, bool private)vcpu_args_set (vcpu,1 , gva);vcpu_run (vcpu); - run = vcpu->run; -TEST_ASSERT (run->exit_reason ==KVM_EXIT_IO , -"Wanted KVM_EXIT_IO, got exit reason: %u (%s)" , - run->exit_reason,exit_reason_str (run->exit_reason)); -switch (get_ucall (vcpu, &uc)) {case UCALL_ABORT :REPORT_GUEST_ASSERT (uc);
Signature · 1 line
2.54.0
Claudio Imbrenda
maintainerPATCH v3 5/5 KVM: selftests: Enable pre_fault_memory_test for s390
In reply to Claudio ImbrendaEnable the pre_fault_memory_test to run on s390.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + 1 file changed, 1 insertion(+)
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -210,6 +210,7 @@ TEST_GEN_PROGS_s390 += s390/keyop TEST_GEN_PROGS_s390 += rseq_test TEST_GEN_PROGS_s390 += s390/irq_routing TEST_GEN_PROGS_s390 += mmu_stress_test +TEST_GEN_PROGS_s390 += pre_fault_memory_test TEST_GEN_PROGS_riscv = $(TEST_GEN_PROGS_COMMON) TEST_GEN_PROGS_riscv += riscv/sbi_pmu_test
Signature · 1 line
2.54.0
Quoted 3 lines · Claudio Imbrenda
Implement
KVM_PRE_FAULT_MEMORYon s390, and update documentation and selftests accordingly.
...
Quoted 6 lines
Claudio Imbrenda (5):
KVM: s390: Track page size in structguest_fault
KVM: s390: ImplementKVM_PRE_FAULT_MEMORY
KVM: s390: UpdateKVM_PRE_FAULT_MEMORYAPI documentation
KVM: selftests: Fixpre_fault_memory_testto run on s390
KVM: selftests: Enablepre_fault_memory_testfor s390
LGTM.
For the whole series: