Skip to contentSkip to conversation

KVM: riscv: Add KVM_PRE_FAULT_MEMORY support

PATCH Superseded Jinyu Tang 3 participants 7 messages Aug 11 – Aug 14

Subsystems (4)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Jinyu Tang

author
#0 lore raw

v2 cover KVM: riscv: Add KVM_PRE_FAULT_MEMORY support

This series adds RISC-V support for the generic KVM_PRE_FAULT_MEMORY vcpu ioctl and enables the existing generic KVM selftest for RISC-V.

KVM_PRE_FAULT_MEMORY lets userspace ask KVM to populate stage-2 mappings for a GPA range before running a vCPU that is expected to access that range. x86 already supports the ioctl; RISC-V currently falls back to normal first-touch G-stage faults.

Patch 1 implements the RISC-V arch hook using the existing G-stage map path. Patch 2 fixes the RISC-V selftest page-table walker for Sv57, which can be selected as the default guest mode. Patch 3 enables pre_fault_memory_test for RISC-V.

Tested with pre_fault_memory_test on a RISC-V KVM host.

Changes in v2:
- Drop the per-test guest_modes_append_default() call from pre_fault_memory_test.c. RISC-V already initializes supported and default guest modes from kvm_selftest_arch_init(), which is called by the common kvm_selftest_init() constructor, as suggested by Sean.

Jinyu Tang (3):
KVM: riscv: Implement KVM_PRE_FAULT_MEMORY
KVM: selftests: Add RISC-V Sv57 page table indexing
KVM: selftests: Enable pre_fault_memory_test for RISC-V

 arch/riscv/kvm/Kconfig                        |  1 +
 arch/riscv/kvm/gstage.c                       |  3 ++
 arch/riscv/kvm/mmu.c                          | 33 +++++++++++++++++++
 arch/riscv/kvm/vm.c                           |  1 +
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../selftests/kvm/include/riscv/processor.h   |  6 ++++
 .../selftests/kvm/lib/riscv/processor.c       |  2 ++
 7 files changed, 47 insertions(+)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v2 1/3 KVM: riscv: Implement KVM_PRE_FAULT_MEMORY

The generic KVM_PRE_FAULT_MEMORY ioctl lets userspace populate KVM page tables before running a vCPU over a GPA range. x86 already supports the ioctl, but RISC-V does not expose the capability and has no arch hook.

Add the RISC-V arch hook and reuse the existing G-stage fault mapping path with a read access. Report progress using the G-stage mapping returned by the map path, so the ioctl can advance by the actual leaf size that covers the requested GPA.

Unlike x86, RISC-V's G-stage fault path does not return a detailed result such as RET_PF_FIXED or RET_PF_RETRY. Treat a successful call that does not return a visible G-stage mapping as no progress, instead of reporting that a page was prefaulted.

---
 arch/riscv/kvm/Kconfig  |  1 +
 arch/riscv/kvm/gstage.c |  3 +++
 arch/riscv/kvm/mmu.c    | 33 +++++++++++++++++++++++++++++++++
 arch/riscv/kvm/vm.c     |  1 +
 4 files changed, 38 insertions(+)
arch/riscv/kvm/Kconfig +1 -0
index ec2cee0a39e0..8ac209e8ac87 100644
@@ -28,6 +28,7 @@ config KVM
 	select KVM_COMMON
 	select KVM_GENERIC_DIRTYLOG_READ_PROTECT
 	select KVM_GENERIC_HARDWARE_ENABLING
+	select KVM_GENERIC_PRE_FAULT_MEMORY
 	select KVM_MMIO
 	select VIRT_XFER_TO_GUEST_WORK
 	select SCHED_INFO
arch/riscv/kvm/gstage.c +3 -0
index e5002cb9cbef..6bd8b8fd6ceb 100644
@@ -280,6 +280,9 @@ int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage,
 						    out_map->level, true);
 		} else if (ALIGN_DOWN(PFN_PHYS(pte_pfn(ptep_get(ptep))), page_size) == hpa) {
 			kvm_riscv_gstage_update_pte_prot(gstage, ptep_level, gpa, ptep, prot);
+			out_map->addr = ALIGN_DOWN(gpa, page_size);
+			out_map->level = ptep_level;
+			out_map->pte = ptep_get(ptep);
 			return 0;
 		}
 	}
arch/riscv/kvm/mmu.c +33 -0
index 6035b5ec9503..c96e21740c91 100644
@@ -748,6 +748,39 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
 	return ret;
 }
 
+long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,
+				    struct kvm_pre_fault_memory *range)
+{
+	struct kvm_gstage_mapping out_map = { 0 };
+	struct kvm_memory_slot *memslot;
+	unsigned long map_size;
+	unsigned long hva;
+	gpa_t end;
+	gfn_t gfn;
+	int ret;
+
+	gfn = gpa_to_gfn(range->gpa);
+	memslot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
+	if (!memslot)
+		return -ENOENT;
+
+	hva = gfn_to_hva_memslot_prot(memslot, gfn, NULL);
+	if (kvm_is_error_hva(hva))
+		return -ENOENT;
+
+	ret = kvm_riscv_mmu_map(vcpu, memslot, range->gpa, hva, false, &out_map);
+	if (ret)
+		return ret;
+
+	if (!pte_val(out_map.pte))
+		return -EAGAIN;
+
+	map_size = PAGE_SIZE << (out_map.level * kvm_riscv_gstage_index_bits);
+	end = ALIGN_DOWN(range->gpa, map_size) + map_size;
+
+	return min_t(u64, range->size, end - range->gpa);
+}
+
 int kvm_riscv_mmu_alloc_pgd(struct kvm *kvm)
 {
 	struct page *pgd_page;
arch/riscv/kvm/vm.c +1 -0
index a9f083feeb76..58500a19b33b 100644
@@ -187,6 +187,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_MP_STATE:
 	case KVM_CAP_IMMEDIATE_EXIT:
 	case KVM_CAP_SET_GUEST_DEBUG:
+	case KVM_CAP_PRE_FAULT_MEMORY:
 		r = 1;
 		break;
 	case KVM_CAP_NR_VCPUS:
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH v2 1/3] KVM: riscv: Implement KVM_PRE_FAULT_MEMORY

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] RISC-V KVM_PRE_FAULT_MEMORY leaks -EAGAIN to userspace on mmu_invalidate_retry or HWPOISON, which violates the ABI and differs from x86.

Pre-existing issues:
- [High] kvm_riscv_gstage_set_pte overwrites a non-leaf PTE with a leaf PTE without freeing the lower-level page table, causing a memory leak on THP collapse. --

--- Patch [1]: [PATCH v2 1/3] KVM: riscv: Implement KVM_PRE_FAULT_MEMORY --- commit 39a9f049be58ef7b5671f786fdc8299c1f217042
Author: Jinyu Tang <[email protected]>

KVM: riscv: Implement KVM_PRE_FAULT_MEMORY

The generic KVM_PRE_FAULT_MEMORY ioctl lets userspace populate KVM page tables before running a vCPU over a GPA range. This commit adds the RISC-V arch hook and reuses the existing G-stage fault mapping path.

Quoted 4 lines
arch/riscv/kvm/gstage.c +0 -0
index b0474fcf065aa..ebd1e8a4b9e38 100644

[ ... ]

arch/riscv/kvm/gstage.c int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage, · line 288

Quoted 10 lines
arch/riscv/kvm/gstage.c +3 -0
@@ -280,6 +280,9 @@ int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage,
 						    out_map->level, true);
 		} else if (ALIGN_DOWN(PFN_PHYS(pte_pfn(ptep_get(ptep))), page_size) == hpa) {
 			kvm_riscv_gstage_update_pte_prot(gstage, ptep_level, gpa, ptep, prot);
+			out_map->addr = ALIGN_DOWN(gpa, page_size);
+			out_map->level = ptep_level;
+			out_map->pte = ptep_get(ptep);
 			return 0;
 		}
 	}

[Severity: High]
This isn't a bug introduced by this patch, but does kvm_riscv_gstage_set_pte() overwrite non-leaf PTEs with leaf PTEs without freeing the lower-level page table?

When a THP collapse occurs, KVM attempts to map the huge page at level 1. If the previous 4K leaf PTEs at level 0 were unmapped (e.g., via kvm_riscv_gstage_unmap_range()) but the level-0 page table was left allocated, kvm_riscv_gstage_get_leaf() returns false.

This function then falls through and calls kvm_riscv_gstage_set_pte() to install the level-1 leaf PTE. That appears to blindly overwrite the level-1 non-leaf PTE with a leaf PTE via set_pte(ptep, map->pte) without freeing the level-0 page table it pointed to.

Could this lead to a memory leak of level-0 page tables over time?

Quoted 4 lines
arch/riscv/kvm/mmu.c +0 -0
index 8a0aa5e0e216e..926b53723e31c 100644

[ ... ]

Quoted 7 lines
arch/riscv/kvm/mmu.c +3 -0
@@ -661,6 +661,39 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
 	return ret;
 }
 
+long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,
+				    struct kvm_pre_fault_memory *range)
+{

[ ... ]

arch/riscv/kvm/mmu.c int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot, · line 776

Quoted 6 lines
arch/riscv/kvm/mmu.c +6 -0
+	ret = kvm_riscv_mmu_map(vcpu, memslot, range->gpa, hva, false, &out_map);
+	if (ret)
+		return ret;
+
+	if (!pte_val(out_map.pte))
+		return -EAGAIN;

[Severity: High]
Does returning -EAGAIN here leak this internal error code to userspace?

When kvm_riscv_mmu_map() handles a concurrent host MMU invalidation (mmu_invalidate_retry() returns true) or a HWPOISONed page, it returns 0 but does not populate out_map.pte.

This arch hook then translates the internal !pte_val(out_map.pte) state into an -EAGAIN return value. Since the generic KVM ioctl loop bubbles this up, unprivileged userspace VMMs executing the KVM_PRE_FAULT_MEMORY ioctl concurrently with memory pressure or page migration will receive an undocumented -EAGAIN error, potentially causing them to fatally abort VM initialization.

Should this return 0 instead to indicate no progress, or is a different mechanism needed to avoid breaking the ABI?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author
#3 lore raw

PATCH v2 2/3 KVM: selftests: Add RISC-V Sv57 page table indexing

In reply to Jinyu Tang

RISC-V selftests can create guests with five page-table levels, for example when the selected guest mode is Sv57. The RISC-V page-table walker only had index arrays for levels 0 through 3, so virt_arch_pg_map() indexed past the end of the arrays when level 4 was used.

Add the missing L4 index mask and shift so selftests can build guest page tables for Sv57 VMs.

---
 tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++
 tools/testing/selftests/kvm/lib/riscv/processor.c     | 2 ++
 2 files changed, 5 insertions(+)
tools/testing/selftests/kvm/include/riscv/processor.h +3 -0
index e3acf2ae9881..abde3c71c891 100644
@@ -127,6 +127,9 @@ void vm_install_exception_handler(struct kvm_vm *vm, int vector, exception_handl
 
 void vm_install_interrupt_handler(struct kvm_vm *vm, exception_handler_fn handler);
 
+/* L4 index Bit[56:48] */
+#define PGTBL_L4_INDEX_MASK			0x01FF000000000000ULL
+#define PGTBL_L4_INDEX_SHIFT			48
 /* L3 index Bit[47:39] */
 #define PGTBL_L3_INDEX_MASK			0x0000FF8000000000ULL
 #define PGTBL_L3_INDEX_SHIFT			39
tools/testing/selftests/kvm/lib/riscv/processor.c +2 -0
index ded5429f3448..b4d41a407553 100644
@@ -43,6 +43,7 @@ static u64 pte_index_mask[] = {
 	PGTBL_L1_INDEX_MASK,
 	PGTBL_L2_INDEX_MASK,
 	PGTBL_L3_INDEX_MASK,
+	PGTBL_L4_INDEX_MASK,
 };
 
 static u32 pte_index_shift[] = {
@@ -50,6 +51,7 @@ static u32 pte_index_shift[] = {
 	PGTBL_L1_INDEX_SHIFT,
 	PGTBL_L2_INDEX_SHIFT,
 	PGTBL_L3_INDEX_SHIFT,
+	PGTBL_L4_INDEX_SHIFT,
 };
 
 static u64 pte_index(struct kvm_vm *vm, gva_t gva, int level)
Signature · 1 line
2.43.0
Reviewed-by on v2 2/3
#4 lore raw

Re: [PATCH v2 2/3] KVM: selftests: Add RISC-V Sv57 page table indexing

Quoted 10 lines · Jinyu Tang

RISC-V selftests can create guests with five page-table levels, for example when the selected guest mode is Sv57. The RISC-V page-table walker only had index arrays for levels 0 through 3, so virt_arch_pg_map() indexed past the end of the arrays when level 4 was used.

Add the missing L4 index mask and shift so selftests can build guest page tables for Sv57 VMs.

Thanks,
Nutty

Quoted 39 lines
---
  tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++
  tools/testing/selftests/kvm/lib/riscv/processor.c     | 2 ++
  2 files changed, 5 insertions(+)
tools/testing/selftests/kvm/include/riscv/processor.h +3 -0
index e3acf2ae9881..abde3c71c891 100644
@@ -127,6 +127,9 @@ void vm_install_exception_handler(struct kvm_vm *vm, int vector, exception_handl
 
  void vm_install_interrupt_handler(struct kvm_vm *vm, exception_handler_fn handler);
 
+/* L4 index Bit[56:48] */
+#define PGTBL_L4_INDEX_MASK			0x01FF000000000000ULL
+#define PGTBL_L4_INDEX_SHIFT			48
  /* L3 index Bit[47:39] */
  #define PGTBL_L3_INDEX_MASK			0x0000FF8000000000ULL
  #define PGTBL_L3_INDEX_SHIFT			39
tools/testing/selftests/kvm/lib/riscv/processor.c +2 -0
index ded5429f3448..b4d41a407553 100644
@@ -43,6 +43,7 @@ static u64 pte_index_mask[] = {
  	PGTBL_L1_INDEX_MASK,
  	PGTBL_L2_INDEX_MASK,
  	PGTBL_L3_INDEX_MASK,
+	PGTBL_L4_INDEX_MASK,
  };
 
  static u32 pte_index_shift[] = {
@@ -50,6 +51,7 @@ static u32 pte_index_shift[] = {
  	PGTBL_L1_INDEX_SHIFT,
  	PGTBL_L2_INDEX_SHIFT,
  	PGTBL_L3_INDEX_SHIFT,
+	PGTBL_L4_INDEX_SHIFT,
  };
 
  static u64 pte_index(struct kvm_vm *vm, gva_t gva, int level)
author
#5 lore raw

PATCH v2 3/3 KVM: selftests: Enable pre_fault_memory_test for RISC-V

In reply to Jinyu Tang

RISC-V now supports KVM_PRE_FAULT_MEMORY, so include the generic pre_fault_memory_test in the RISC-V KVM selftest build.

The test uses PAGE_SIZE from the architecture processor header. Define the normal 4K RISC-V selftest page size so the generic test can build for RISC-V.

---
v1 -> v2:
- Drop the per-test guest_modes_append_default() call from pre_fault_memory_test.c. RISC-V already initializes supported and default guest modes from kvm_selftest_arch_init(), which is called by the common kvm_selftest_init() constructor, as suggested by Sean.

 tools/testing/selftests/kvm/Makefile.kvm              | 1 +
 tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++
 2 files changed, 4 insertions(+)
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 6fc34e9bf8e1..ac64ac92fd4b 100644
@@ -225,6 +225,7 @@ TEST_GEN_PROGS_riscv += coalesced_io_test
 TEST_GEN_PROGS_riscv += dirty_log_perf_test
 TEST_GEN_PROGS_riscv += get-reg-list
 TEST_GEN_PROGS_riscv += mmu_stress_test
+TEST_GEN_PROGS_riscv += pre_fault_memory_test
 TEST_GEN_PROGS_riscv += rseq_test
 TEST_GEN_PROGS_riscv += steal_time
 
tools/testing/selftests/kvm/include/riscv/processor.h +3 -0
index abde3c71c891..70487c8ed155 100644
@@ -12,6 +12,9 @@
 #include <asm/vdso/processor.h>
 #include "kvm_util.h"
 
+#define PAGE_SHIFT		12
+#define PAGE_SIZE		BIT_ULL(PAGE_SHIFT)
+
 #define INSN_OPCODE_MASK	0x007c
 #define INSN_OPCODE_SHIFT	2
 #define INSN_OPCODE_SYSTEM	28
Signature · 1 line
2.43.0
Reviewed-by on v2 3/3
#6 lore raw

Re: [PATCH v2 3/3] KVM: selftests: Enable pre_fault_memory_test for RISC-V

Quoted 8 lines · Jinyu Tang

RISC-V now supports KVM_PRE_FAULT_MEMORY, so include the generic pre_fault_memory_test in the RISC-V KVM selftest build.

The test uses PAGE_SIZE from the architecture processor header. Define the normal 4K RISC-V selftest page size so the generic test can build for RISC-V.

Thanks,
Nutty

Quoted 37 lines

---
v1 -> v2:
- Drop the per-test guest_modes_append_default() call from pre_fault_memory_test.c. RISC-V already initializes supported and default guest modes from kvm_selftest_arch_init(), which is called by the common kvm_selftest_init() constructor, as suggested by Sean.

  tools/testing/selftests/kvm/Makefile.kvm              | 1 +
  tools/testing/selftests/kvm/include/riscv/processor.h | 3 +++
  2 files changed, 4 insertions(+)
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 6fc34e9bf8e1..ac64ac92fd4b 100644
@@ -225,6 +225,7 @@ TEST_GEN_PROGS_riscv += coalesced_io_test
  TEST_GEN_PROGS_riscv += dirty_log_perf_test
  TEST_GEN_PROGS_riscv += get-reg-list
  TEST_GEN_PROGS_riscv += mmu_stress_test
+TEST_GEN_PROGS_riscv += pre_fault_memory_test
  TEST_GEN_PROGS_riscv += rseq_test
  TEST_GEN_PROGS_riscv += steal_time
 
tools/testing/selftests/kvm/include/riscv/processor.h +3 -0
index abde3c71c891..70487c8ed155 100644
@@ -12,6 +12,9 @@
  #include <asm/vdso/processor.h>
  #include "kvm_util.h"
 
+#define PAGE_SHIFT		12
+#define PAGE_SIZE		BIT_ULL(PAGE_SHIFT)
+
  #define INSN_OPCODE_MASK	0x007c
  #define INSN_OPCODE_SHIFT	2
  #define INSN_OPCODE_SYSTEM	28