KVM: riscv: Add KVM_PRE_FAULT_MEMORY support
Subsystems (4)
v2 cover KVM: riscv: Add KVM_PRE_FAULT_MEMORY support
This series adds RISC-V support for the generic KVM_PRE_FAULT_MEMORY vcpu ioctl and enables the existing generic KVM selftest for RISC-V.
KVM_PRE_FAULT_MEMORY lets userspace ask KVM to populate stage-2 mappings for a GPA range before running a vCPU that is expected to access that range. x86 already supports the ioctl; RISC-V currently falls back to normal first-touch G-stage faults.
Patch 1 implements the RISC-V arch hook using the existing G-stage map path. Patch 2 fixes the RISC-V selftest page-table walker for Sv57, which can be selected as the default guest mode. Patch 3 enables pre_fault_memory_test for RISC-V.
Tested with pre_fault_memory_test on a RISC-V KVM host.
Changes in v2:
- Drop the per-test guest_modes_append_default() call from pre_fault_memory_test.c. RISC-V already initializes supported and default guest modes from kvm_selftest_arch_init(), which is called by the common kvm_selftest_init() constructor, as suggested by Sean.
Jinyu Tang (3):
KVM: riscv: Implement KVM_PRE_FAULT_MEMORY
KVM: selftests: Add RISC-V Sv57 page table indexing
KVM: selftests: Enable pre_fault_memory_test for RISC-V
arch/riscv/kvm/Kconfig | 1 + arch/riscv/kvm/gstage.c | 3 ++ arch/riscv/kvm/mmu.c | 33 +++++++++++++++++++ arch/riscv/kvm/vm.c | 1 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/riscv/processor.h | 6 ++++ .../selftests/kvm/lib/riscv/processor.c | 2 ++ 7 files changed, 47 insertions(+)
Signature · 1 line
2.43.0
PATCH v2 1/3 KVM: riscv: Implement KVM_PRE_FAULT_MEMORY
The generic KVM_PRE_FAULT_MEMORY ioctl lets userspace populate KVM page tables before running a vCPU over a GPA range. x86 already supports the ioctl, but RISC-V does not expose the capability and has no arch hook.
Add the RISC-V arch hook and reuse the existing G-stage fault mapping path with a read access. Report progress using the G-stage mapping returned by the map path, so the ioctl can advance by the actual leaf size that covers the requested GPA.
Unlike x86, RISC-V's G-stage fault path does not return a detailed result such as RET_PF_FIXED or RET_PF_RETRY. Treat a successful call that does not return a visible G-stage mapping as no progress, instead of reporting that a page was prefaulted.
--- arch/riscv/kvm/Kconfig | 1 + arch/riscv/kvm/gstage.c | 3 +++ arch/riscv/kvm/mmu.c | 33 +++++++++++++++++++++++++++++++++ arch/riscv/kvm/vm.c | 1 + 4 files changed, 38 insertions(+)
arch/riscv/kvm/Kconfig +1 -0
@@ -28,6 +28,7 @@ config KVM select KVM_COMMON select KVM_GENERIC_DIRTYLOG_READ_PROTECT select KVM_GENERIC_HARDWARE_ENABLING + select KVM_GENERIC_PRE_FAULT_MEMORY select KVM_MMIO select VIRT_XFER_TO_GUEST_WORK select SCHED_INFO
arch/riscv/kvm/gstage.c +3 -0
@@ -280,6 +280,9 @@ int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage, out_map->level, true); }else if (ALIGN_DOWN (PFN_PHYS (pte_pfn (ptep_get (ptep))), page_size) == hpa) {kvm_riscv_gstage_update_pte_prot (gstage, ptep_level, gpa, ptep, prot); + out_map->addr =ALIGN_DOWN (gpa, page_size); + out_map->level = ptep_level; + out_map->pte =ptep_get (ptep);return 0 ; } }
arch/riscv/kvm/mmu.c +33 -0
@@ -748,6 +748,39 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,return ret; } +long kvm_arch_vcpu_pre_fault_memory (struct kvm_vcpu *vcpu, +struct kvm_pre_fault_memory *range) +{ +struct kvm_gstage_mapping out_map = {0 }; +struct kvm_memory_slot *memslot; +unsigned long map_size; +unsigned long hva; +gpa_t end; +gfn_t gfn; +int ret; + + gfn =gpa_to_gfn (range->gpa); + memslot =kvm_vcpu_gfn_to_memslot (vcpu, gfn); +if (!memslot) +return -ENOENT ; + + hva =gfn_to_hva_memslot_prot (memslot, gfn,NULL ); +if (kvm_is_error_hva (hva)) +return -ENOENT ; + + ret =kvm_riscv_mmu_map (vcpu, memslot, range->gpa, hva, false, &out_map); +if (ret) +return ret; + +if (!pte_val (out_map.pte)) +return -EAGAIN ; + + map_size =PAGE_SIZE << (out_map.level * kvm_riscv_gstage_index_bits); + end =ALIGN_DOWN (range->gpa, map_size) + map_size; + +return min_t (u64, range->size, end - range->gpa); +} +int kvm_riscv_mmu_alloc_pgd (struct kvm *kvm) {struct page *pgd_page;
arch/riscv/kvm/vm.c +1 -0
@@ -187,6 +187,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_MP_STATE :case KVM_CAP_IMMEDIATE_EXIT :case KVM_CAP_SET_GUEST_DEBUG : +case KVM_CAP_PRE_FAULT_MEMORY : r =1 ;break ;case KVM_CAP_NR_VCPUS :
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v2 1/3] KVM: riscv: Implement KVM_PRE_FAULT_MEMORY
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High] RISC-V KVM_PRE_FAULT_MEMORY leaks -EAGAIN to userspace on mmu_invalidate_retry or HWPOISON, which violates the ABI and differs from x86.
Pre-existing issues:
- [High] kvm_riscv_gstage_set_pte overwrites a non-leaf PTE with a leaf PTE without freeing the lower-level page table, causing a memory leak on THP collapse. --
--- Patch [1]: [PATCH v2 1/3] KVM: riscv: Implement KVM_PRE_FAULT_MEMORY --- commit 39a9f049be58ef7b5671f786fdc8299c1f217042
Author: Jinyu Tang <[email protected]>
KVM: riscv: Implement KVM_PRE_FAULT_MEMORY
The generic KVM_PRE_FAULT_MEMORY ioctl lets userspace populate KVM page tables before running a vCPU over a GPA range. This commit adds the RISC-V arch hook and reuses the existing G-stage fault mapping path.
Quoted 4 lines
arch/riscv/kvm/gstage.c +0
-0
[ ... ]
arch/riscv/kvm/gstage.c int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage, · line 288
Quoted 10 lines
arch/riscv/kvm/gstage.c +3
-0@@ -280,6 +280,9 @@ int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage, out_map->level, true); }else if (ALIGN_DOWN (PFN_PHYS (pte_pfn (ptep_get (ptep))), page_size) == hpa) {kvm_riscv_gstage_update_pte_prot (gstage, ptep_level, gpa, ptep, prot); + out_map->addr =ALIGN_DOWN (gpa, page_size); + out_map->level = ptep_level; + out_map->pte =ptep_get (ptep);return 0 ; } }
[Severity: High]
This isn't a bug introduced by this patch, but does kvm_riscv_gstage_set_pte() overwrite non-leaf PTEs with leaf PTEs without freeing the lower-level page table?
When a THP collapse occurs, KVM attempts to map the huge page at level 1. If the previous 4K leaf PTEs at level 0 were unmapped (e.g., via kvm_riscv_gstage_unmap_range()) but the level-0 page table was left allocated, kvm_riscv_gstage_get_leaf() returns false.
This function then falls through and calls kvm_riscv_gstage_set_pte() to install the level-1 leaf PTE. That appears to blindly overwrite the level-1 non-leaf PTE with a leaf PTE via set_pte(ptep, map->pte) without freeing the level-0 page table it pointed to.
Could this lead to a memory leak of level-0 page tables over time?
Quoted 4 lines
arch/riscv/kvm/mmu.c +0
-0
[ ... ]
Quoted 7 lines
arch/riscv/kvm/mmu.c +3
-0@@ -661,6 +661,39 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,return ret; } +long kvm_arch_vcpu_pre_fault_memory (struct kvm_vcpu *vcpu, +struct kvm_pre_fault_memory *range) +{
[ ... ]
arch/riscv/kvm/mmu.c int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot, · line 776
Quoted 6 lines
arch/riscv/kvm/mmu.c +6
-0+ ret =kvm_riscv_mmu_map (vcpu, memslot, range->gpa, hva, false, &out_map); +if (ret) +return ret; + +if (!pte_val (out_map.pte)) +return -EAGAIN ;
[Severity: High]
Does returning -EAGAIN here leak this internal error code to userspace?
When kvm_riscv_mmu_map() handles a concurrent host MMU invalidation (mmu_invalidate_retry() returns true) or a HWPOISONed page, it returns 0 but does not populate out_map.pte.
This arch hook then translates the internal !pte_val(out_map.pte) state into an -EAGAIN return value. Since the generic KVM ioctl loop bubbles this up, unprivileged userspace VMMs executing the KVM_PRE_FAULT_MEMORY ioctl concurrently with memory pressure or page migration will receive an undocumented -EAGAIN error, potentially causing them to fatally abort VM initialization.
Should this return 0 instead to indicate no progress, or is a different mechanism needed to avoid breaking the ABI?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
PATCH v2 2/3 KVM: selftests: Add RISC-V Sv57 page table indexing
In reply to Jinyu TangRISC-V selftests can create guests with five page-table levels, for example when the selected guest mode is Sv57. The RISC-V page-table walker only had index arrays for levels 0 through 3, so virt_arch_pg_map() indexed past the end of the arrays when level 4 was used.
Add the missing L4 index mask and shift so selftests can build guest page tables for Sv57 VMs.
--- tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++ tools/testing/selftests/kvm/lib/riscv/processor.c | 2 ++ 2 files changed, 5 insertions(+)
tools/testing/selftests/kvm/include/riscv/processor.h +3 -0
@@ -127,6 +127,9 @@ void vm_install_exception_handler(struct kvm_vm *vm, int vector, exception_handlvoid vm_install_interrupt_handler (struct kvm_vm *vm,exception_handler_fn handler); +/* L4 index Bit[56:48] */ +#define PGTBL_L4_INDEX_MASK 0x01FF000000000000ULL +#define PGTBL_L4_INDEX_SHIFT 48/* L3 index Bit[47:39] */ #define PGTBL_L3_INDEX_MASK 0x0000FF8000000000ULL#define PGTBL_L3_INDEX_SHIFT 39
tools/testing/selftests/kvm/lib/riscv/processor.c +2 -0
@@ -43,6 +43,7 @@ static u64 pte_index_mask[] = {PGTBL_L1_INDEX_MASK ,PGTBL_L2_INDEX_MASK ,PGTBL_L3_INDEX_MASK , + PGTBL_L4_INDEX_MASK, }; static u32 pte_index_shift[] = {
@@ -50,6 +51,7 @@ static u32 pte_index_shift[] = {PGTBL_L1_INDEX_SHIFT ,PGTBL_L2_INDEX_SHIFT ,PGTBL_L3_INDEX_SHIFT , + PGTBL_L4_INDEX_SHIFT, };static u64 pte_index (struct kvm_vm *vm,gva_t gva,int level)
Signature · 1 line
2.43.0
Re: [PATCH v2 2/3] KVM: selftests: Add RISC-V Sv57 page table indexing
Quoted 10 lines · Jinyu Tang
RISC-V selftests can create guests with five page-table levels, for example when the selected guest mode is Sv57. The RISC-V page-table walker only had index arrays for levels 0 through 3, so
virt_arch_pg_map()indexed past the end of the arrays when level 4 was used.Add the missing L4 index mask and shift so selftests can build guest page tables for Sv57 VMs.
Signed-off-by Jinyu Tang <[email protected]>
Thanks,
Nutty
Quoted 39 lines
--- tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++ tools/testing/selftests/kvm/lib/riscv/processor.c | 2 ++ 2 files changed, 5 insertions(+)tools/testing/selftests/kvm/include/riscv/processor.h +3
-0@@ -127,6 +127,9 @@ void vm_install_exception_handler(struct kvm_vm *vm, int vector, exception_handlvoid vm_install_interrupt_handler (struct kvm_vm *vm,exception_handler_fn handler); +/* L4 index Bit[56:48] */ +#define PGTBL_L4_INDEX_MASK 0x01FF000000000000ULL +#define PGTBL_L4_INDEX_SHIFT 48/* L3 index Bit[47:39] */ #define PGTBL_L3_INDEX_MASK 0x0000FF8000000000ULL#define PGTBL_L3_INDEX_SHIFT 39tools/testing/selftests/kvm/lib/riscv/processor.c +2
-0@@ -43,6 +43,7 @@ static u64 pte_index_mask[] = {PGTBL_L1_INDEX_MASK ,PGTBL_L2_INDEX_MASK ,PGTBL_L3_INDEX_MASK , + PGTBL_L4_INDEX_MASK, }; static u32 pte_index_shift[] = {@@ -50,6 +51,7 @@ static u32 pte_index_shift[] = {PGTBL_L1_INDEX_SHIFT ,PGTBL_L2_INDEX_SHIFT ,PGTBL_L3_INDEX_SHIFT , + PGTBL_L4_INDEX_SHIFT, };static u64 pte_index (struct kvm_vm *vm,gva_t gva,int level)
PATCH v2 3/3 KVM: selftests: Enable pre_fault_memory_test for RISC-V
In reply to Jinyu TangRISC-V now supports KVM_PRE_FAULT_MEMORY, so include the generic pre_fault_memory_test in the RISC-V KVM selftest build.
The test uses PAGE_SIZE from the architecture processor header. Define the normal 4K RISC-V selftest page size so the generic test can build for RISC-V.
---
v1 -> v2:
- Drop the per-test guest_modes_append_default() call from pre_fault_memory_test.c. RISC-V already initializes supported and default guest modes from kvm_selftest_arch_init(), which is called by the common kvm_selftest_init() constructor, as suggested by Sean.
tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++ 2 files changed, 4 insertions(+)
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -225,6 +225,7 @@ TEST_GEN_PROGS_riscv += coalesced_io_test TEST_GEN_PROGS_riscv += dirty_log_perf_test TEST_GEN_PROGS_riscv += get-reg-list TEST_GEN_PROGS_riscv += mmu_stress_test +TEST_GEN_PROGS_riscv += pre_fault_memory_test TEST_GEN_PROGS_riscv += rseq_test TEST_GEN_PROGS_riscv += steal_time
tools/testing/selftests/kvm/include/riscv/processor.h +3 -0
@@ -12,6 +12,9 @@#include <asm/vdso/processor.h> #include "kvm_util.h" +#define PAGE_SHIFT 12 +#define PAGE_SIZE BIT_ULL(PAGE_SHIFT) +#define INSN_OPCODE_MASK 0x007c#define INSN_OPCODE_SHIFT 2#define INSN_OPCODE_SYSTEM 28
Signature · 1 line
2.43.0
Re: [PATCH v2 3/3] KVM: selftests: Enable pre_fault_memory_test for RISC-V
Quoted 8 lines · Jinyu Tang
RISC-V now supports
KVM_PRE_FAULT_MEMORY, so include the genericpre_fault_memory_testin the RISC-V KVM selftest build.The test uses
PAGE_SIZEfrom the architecture processor header. Define the normal 4K RISC-V selftest page size so the generic test can build for RISC-V.Signed-off-by Jinyu Tang <[email protected]>
Thanks,
Nutty
Quoted 37 lines
---
v1 -> v2:
- Drop the per-testguest_modes_append_default()call frompre_fault_memory_test.c. RISC-V already initializes supported and default guest modes fromkvm_selftest_arch_init(), which is called by the commonkvm_selftest_init()constructor, as suggested by Sean.tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++ 2 files changed, 4 insertions(+)tools/testing/selftests/kvm/Makefile.kvm +1
-0@@ -225,6 +225,7 @@ TEST_GEN_PROGS_riscv += coalesced_io_test TEST_GEN_PROGS_riscv += dirty_log_perf_test TEST_GEN_PROGS_riscv += get-reg-list TEST_GEN_PROGS_riscv += mmu_stress_test +TEST_GEN_PROGS_riscv += pre_fault_memory_test TEST_GEN_PROGS_riscv += rseq_test TEST_GEN_PROGS_riscv += steal_timetools/testing/selftests/kvm/include/riscv/processor.h +3
-0@@ -12,6 +12,9 @@#include <asm/vdso/processor.h> #include "kvm_util.h" +#define PAGE_SHIFT 12 +#define PAGE_SIZE BIT_ULL(PAGE_SHIFT) +#define INSN_OPCODE_MASK 0x007c#define INSN_OPCODE_SHIFT 2#define INSN_OPCODE_SYSTEM 28