Skip to contentSkip to conversation

RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping

PATCH Superseded Yuhang.Chen 3 participants 3 messages Aug 4 – Aug 5

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping

Add a kernel command-line option, kvm-riscv.wfi_trap_policy=trap|auto, that controls whether a WFI executed by a VS-mode guest traps into KVM (HS-mode) or executes natively.

HSTATUS.VTW governs VS-mode WFI: when set, the WFI traps into KVM, which blocks the vCPU through kvm_vcpu_halt() and releases the CPU to other runnable tasks; when clear, the guest runs WFI natively. Because RISC-V WFI is only a hint (it may be a no-op on some implementations), the policy is re-evaluated each time a vCPU is loaded rather than fixed once at reset.

trap : always trap VS-mode WFI into KVM (HSTATUS.VTW=1). This is the
default and preserves the previous unconditional behavior.

auto : clear HSTATUS.VTW so the guest runs WFI natively only when the
vCPU is the sole runnable task on the current CPU; otherwise keep trapping. When the vCPU is alone, skipping the virtual-instruction exit cannot starve another task, and on hardware that honors WFI the hart blocks until a VS-mode interrupt. As soon as another task becomes runnable, the policy traps again so that KVM blocks the vCPU and yields the CPU to it. The vCPU therefore never monopolizes the CPU the way an unconditional native WFI would: it either blocks through kvm_vcpu_halt(), or runs WFI natively only when no other task needs the CPU.

Measured on QEMU TCG (-smp 1, -cpu max): wfi_exit_stat delta and guest wake count over a 3 s window. "busy" adds a CPU-bound competitor that shares the vCPU's CPU so that single_task_running() reports false:

  policy  busy  exits  wakes   cpu%  note
  ------  ----  -----  -----   ----  ------------------------
  trap    off    286    285    6.5   default; no regression
  trap    on     291    290  101.0   trap is unconditional
  auto    off      4    287    5.0   sole task: native WFI
  auto    on     285    284  101.5   competitor -> traps

With "auto", WFI exits drop to ~0 when the vCPU is the only runnable task, and rise back to the trap level as soon as a competitor appears, which is the desired dynamic behavior. Host CPU stays low in the sole-task case; the ~101% in the busy cases is the forked competitor, not the vCPU.

Assisted-by YuanSheng:deepseek-v4-pro

---
Changes in v2:

- Drop the "notrap" mode, which cleared HSTATUS.VTW unconditionally and so never trapped: the vCPU never reached kvm_vcpu_halt() and could stay busy even while idle.
- Add the "auto" mode, which clears HSTATUS.VTW only when the vCPU is the sole runnable task (single_task_running()) and otherwise traps. The policy is applied dynamically from kvm_arch_vcpu_load() instead of once at reset, so a vCPU that stops being the sole runnable task switches back to trapping.
- Update the kernel-parameters.txt entry for trap/auto.

v1: https://lore.kernel.org/all/[email protected]/

---
 .../admin-guide/kernel-parameters.txt         | 16 +++++
 arch/riscv/kvm/vcpu.c                         | 59 +++++++++++++++++++
 2 files changed, 75 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +16 -0
index b5493a7f8f22..29fc824b5ce2 100644
@@ -3254,6 +3254,22 @@ Kernel parameters
 
 			notrap: clear WFI instruction trap
 
+	kvm-riscv.wfi_trap_policy=
+			[KVM,RISCV] Control when to set the WFI instruction
+			trap (HSTATUS.VTW) for KVM VMs. The policy is
+			re-evaluated each time a vCPU is loaded, not only at
+			reset, since RISC-V WFI is only a hint.
+
+			trap: always trap VS-mode WFI into KVM (HSTATUS.VTW=1)
+
+			auto: trap unless the vCPU is the only runnable task on
+				the current CPU, in which case clear the trap
+				(HSTATUS.VTW=0) and let the guest execute WFI
+				natively
+
+			Defaults to trap, preserving the previous unconditional
+			behavior.
+
 	kvm_cma_resv_ratio=n [PPC,EARLY]
 			Reserves given percentage from system memory area for
 			contiguous memory allocation for KVM hash pagetable
arch/riscv/kvm/vcpu.c +59 -0

85 lines not on this page · raw message

Signature · 1 line
2.34.1
#1 lore raw
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The dynamic WFI trap policy check is performed only in `kvm_arch_vcpu_load()`, leading to a TOCTOU…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The dynamic WFI trap policy check is performed only in kvm_arch_vcpu_load(), leading to a TOCTOU race that causes CPU starvation for newly awakened tasks.
- [Low] Using early_param in a tristate KVM module breaks the build when KVM is compiled as a loadable module (CONFIG_KVM=m).
--

--- Patch [1]: [PATCH v2] RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping --- commit 74512223938eb34f8681ae4dfb342e0d3e2fc417
Author: Yuhang.Chen <[email protected]>
Subject: RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping

This patch adds a kernel command-line option to control whether a WFI executed by a VS-mode guest traps into KVM or executes natively. The policy can be 'trap' or 'auto', with 'auto' dynamically allowing native WFI only when the vCPU is the sole runnable task.

Quoted 12 lines
arch/riscv/kvm/vcpu.c +4 -0
index 977e36ab83d3f..9a4a7d0729d37 100644
@@ -26,6 +28,59 @@
 
 static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_former_vcpu);
 
+/*
+ * WFI trap policy for VS-mode guests, controllable through the
+ * kvm-riscv.wfi_trap_policy= kernel command-line option.
+ */

[ ... ]

Quoted 8 lines
arch/riscv/kvm/vcpu.c +8 -0
+static int __init early_kvm_riscv_wfi_trap_policy_cfg(char *arg)
+{
+	if (!arg)
+		return -EINVAL;
+
+	if (strcmp(arg, "trap") == 0) {
+		kvm_riscv_wfi_trap_policy = KVM_RISCV_WFI_TRAP;
+		return 0;

arch/riscv/kvm/vcpu.c line 60

Quoted 10 lines
arch/riscv/kvm/vcpu.c +10 -0
+	}
+
+	if (strcmp(arg, "auto") == 0) {
+		kvm_riscv_wfi_trap_policy = KVM_RISCV_WFI_AUTO;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);

[Severity: Low]
Does using early_param() break the build when KVM is compiled as a loadable module (CONFIG_KVM=m)?

The early_param macro defined in include/linux/init.h is strictly wrapped in an #ifndef MODULE block, leaving it undefined for loadable modules.

Since early_kvm_riscv_wfi_trap_policy_cfg() relies on this macro, would compiling with CONFIG_KVM=m result in implicit declaration or undefined macro errors?

[ ... ]

arch/riscv/kvm/vcpu.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 700

Quoted 9 lines
arch/riscv/kvm/vcpu.c +3 -0
@@ -637,6 +693,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 
 	kvm_make_request(KVM_REQ_STEAL_UPDATE, vcpu);
 
+	/* Re-evaluate the WFI trap policy for this vCPU. */
+	kvm_riscv_vcpu_update_wfi_trap(vcpu);
+
 	vcpu->cpu = cpu;
 }

[Severity: High]
Is there a time-of-check to time-of-use race here that could cause CPU starvation for newly awakened tasks?

The policy is evaluated in kvm_arch_vcpu_load(), which clears HSTATUS.VTW when the vCPU is the sole runnable task.

If an interrupt wakes another task on the same CPU while the vCPU is in the guest, but preemption is not immediately forced (e.g., the vCPU still has CFS timeslice remaining), KVM handles the exit and re-enters the guest loop in kvm_arch_vcpu_ioctl_run() without calling kvm_arch_vcpu_load().

Could the guest then execute WFI natively because HSTATUS.VTW remains stale and cleared, halting the physical CPU and starving the newly woken task until the next hardware interrupt?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#2 lore raw
In reply to Yuhang.Chen
kernel test robot noticed the following build errors:

Hi Yuhang.Chen,

kernel test robot noticed the following build errors:

[auto build test ERROR on kvm/queue]
[also build test ERROR on kvm/next mst-vhost/linux-next linus/master v7.2-rc6 next-20260804] [cannot apply to kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Yuhang-Chen/RISC-V-KVM-Add-kvm-riscv-wfi_trap_policy-to-control-VS-mode-WFI-trapping/20260805-030043
base:   https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue
patch link:    https://lore.kernel.org/r/20260804024707.2400404-1-yhchen312%40gmail.com
patch subject: [PATCH v2] RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping
config: riscv-randconfig-001-20260805 (https://download.01.org/0day-ci/archive/20260805/[email protected]/config)
compiler: riscv64-linux-gcc (GCC) 8.5.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260805/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All error/warnings (new ones prefixed by >>):

>> arch/riscv/kvm/vcpu.c:60:13: error: expected declaration specifiers or '...' before string constant
    early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
>> arch/riscv/kvm/vcpu.c:60:42: error: expected declaration specifiers or '...' before 'early_kvm_riscv_wfi_trap_policy_cfg'
    early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);
                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> arch/riscv/kvm/vcpu.c:43:19: warning: 'early_kvm_riscv_wfi_trap_policy_cfg' defined but not used [-Wunused-function]

static int __init early_kvm_riscv_wfi_trap_policy_cfg(char *arg)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +60 arch/riscv/kvm/vcpu.c

    39	
    40	static enum kvm_riscv_wfi_trap_policy kvm_riscv_wfi_trap_policy __read_mostly =
    41		KVM_RISCV_WFI_TRAP;
    42	
  > 43	static int __init early_kvm_riscv_wfi_trap_policy_cfg(char *arg)
    44	{
    45		if (!arg)
    46			return -EINVAL;
    47	
    48		if (strcmp(arg, "trap") == 0) {
    49			kvm_riscv_wfi_trap_policy = KVM_RISCV_WFI_TRAP;
    50			return 0;
    51		}
    52	
    53		if (strcmp(arg, "auto") == 0) {
    54			kvm_riscv_wfi_trap_policy = KVM_RISCV_WFI_AUTO;
    55			return 0;
    56		}
    57	
    58		return -EINVAL;
    59	}
  > 60	early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);
    61	

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki