RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping
Subsystems (4)
PATCH v2 1/1 RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping
Add a kernel command-line option, kvm-riscv.wfi_trap_policy=trap|auto, that controls whether a WFI executed by a VS-mode guest traps into KVM (HS-mode) or executes natively.
HSTATUS.VTW governs VS-mode WFI: when set, the WFI traps into KVM, which blocks the vCPU through kvm_vcpu_halt() and releases the CPU to other runnable tasks; when clear, the guest runs WFI natively. Because RISC-V WFI is only a hint (it may be a no-op on some implementations), the policy is re-evaluated each time a vCPU is loaded rather than fixed once at reset.
trap : always trap VS-mode WFI into KVM (HSTATUS.VTW=1). This is the
default and preserves the previous unconditional behavior.
auto : clear HSTATUS.VTW so the guest runs WFI natively only when the
vCPU is the sole runnable task on the current CPU; otherwise keep trapping. When the vCPU is alone, skipping the virtual-instruction exit cannot starve another task, and on hardware that honors WFI the hart blocks until a VS-mode interrupt. As soon as another task becomes runnable, the policy traps again so that KVM blocks the vCPU and yields the CPU to it. The vCPU therefore never monopolizes the CPU the way an unconditional native WFI would: it either blocks through kvm_vcpu_halt(), or runs WFI natively only when no other task needs the CPU.
Measured on QEMU TCG (-smp 1, -cpu max): wfi_exit_stat delta and guest wake count over a 3 s window. "busy" adds a CPU-bound competitor that shares the vCPU's CPU so that single_task_running() reports false:
policy busy exits wakes cpu% note ------ ---- ----- ----- ---- ------------------------ trap off 286 285 6.5 default; no regression trap on 291 290 101.0 trap is unconditional auto off 4 287 5.0 sole task: native WFI auto on 285 284 101.5 competitor -> traps
With "auto", WFI exits drop to ~0 when the vCPU is the only runnable task, and rise back to the trap level as soon as a competitor appears, which is the desired dynamic behavior. Host CPU stays low in the sole-task case; the ~101% in the busy cases is the forked competitor, not the vCPU.
---
Changes in v2:
- Drop the "notrap" mode, which cleared HSTATUS.VTW unconditionally and so never trapped: the vCPU never reached kvm_vcpu_halt() and could stay busy even while idle.
- Add the "auto" mode, which clears HSTATUS.VTW only when the vCPU is the sole runnable task (single_task_running()) and otherwise traps. The policy is applied dynamically from kvm_arch_vcpu_load() instead of once at reset, so a vCPU that stops being the sole runnable task switches back to trapping.
- Update the kernel-parameters.txt entry for trap/auto.
v1: https://lore.kernel.org/all/[email protected]/
--- .../admin-guide/kernel-parameters.txt | 16 +++++ arch/riscv/kvm/vcpu.c | 59 +++++++++++++++++++ 2 files changed, 75 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +16 -0
@@ -3254,6 +3254,22 @@ Kernel parameters notrap: clear WFI instruction trap + kvm-riscv.wfi_trap_policy= + [KVM,RISCV] Control when to set the WFI instruction + trap (HSTATUS.VTW) for KVM VMs. The policy is + re-evaluated each time a vCPU is loaded, not only at + reset, since RISC-V WFI is only a hint. + + trap: always trap VS-mode WFI into KVM (HSTATUS.VTW=1) + + auto: trap unless the vCPU is the only runnable task on + the current CPU, in which case clear the trap + (HSTATUS.VTW=0) and let the guest execute WFI + natively + + Defaults to trap, preserving the previous unconditional + behavior. + kvm_cma_resv_ratio=n [PPC,EARLY] Reserves given percentage from system memory area for contiguous memory allocation for KVM hash pagetable
arch/riscv/kvm/vcpu.c +59 -0
85 lines not on this page · raw message
Signature · 1 line
2.34.1
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The dynamic WFI trap policy check is performed only in `kvm_arch_vcpu_load()`, leading to a TOCTOU…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The dynamic WFI trap policy check is performed only in kvm_arch_vcpu_load(), leading to a TOCTOU race that causes CPU starvation for newly awakened tasks.
- [Low] Using early_param in a tristate KVM module breaks the build when KVM is compiled as a loadable module (CONFIG_KVM=m).
--
--- Patch [1]: [PATCH v2] RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping --- commit 74512223938eb34f8681ae4dfb342e0d3e2fc417
Author: Yuhang.Chen <[email protected]>
Subject: RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping
This patch adds a kernel command-line option to control whether a WFI executed by a VS-mode guest traps into KVM or executes natively. The policy can be 'trap' or 'auto', with 'auto' dynamically allowing native WFI only when the vCPU is the sole runnable task.
Quoted 12 lines
arch/riscv/kvm/vcpu.c +4
-0@@ -26,6 +28,59 @@ static DEFINE_PER_CPU(struct kvm_vcpu *, kvm_former_vcpu); +/* +* WFI trap policy for VS-mode guests, controllable through the +* kvm-riscv.wfi_trap_policy= kernel command-line option. +*/
[ ... ]
Quoted 8 lines
arch/riscv/kvm/vcpu.c +8
-0+static int __init early_kvm_riscv_wfi_trap_policy_cfg (char *arg) +{ +if (!arg) +return -EINVAL ; + +if (strcmp (arg,"trap" ) ==0 ) { + kvm_riscv_wfi_trap_policy =KVM_RISCV_WFI_TRAP ; +return 0 ;
arch/riscv/kvm/vcpu.c line 60
Quoted 10 lines
arch/riscv/kvm/vcpu.c +10
-0+ } + +if (strcmp (arg,"auto" ) ==0 ) { + kvm_riscv_wfi_trap_policy =KVM_RISCV_WFI_AUTO ; +return 0 ; + } + +return -EINVAL ; +} +early_param ("kvm-riscv.wfi_trap_policy" , early_kvm_riscv_wfi_trap_policy_cfg);
[Severity: Low]
Does using early_param() break the build when KVM is compiled as a loadable module (CONFIG_KVM=m)?
The early_param macro defined in include/linux/init.h is strictly wrapped in an #ifndef MODULE block, leaving it undefined for loadable modules.
Since early_kvm_riscv_wfi_trap_policy_cfg() relies on this macro, would compiling with CONFIG_KVM=m result in implicit declaration or undefined macro errors?
[ ... ]
arch/riscv/kvm/vcpu.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 700
Quoted 9 lines
arch/riscv/kvm/vcpu.c +3
-0@@ -637,6 +693,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)kvm_make_request (KVM_REQ_STEAL_UPDATE , vcpu); +/* Re-evaluate the WFI trap policy for this vCPU. */ +kvm_riscv_vcpu_update_wfi_trap (vcpu); + vcpu->cpu = cpu; }
[Severity: High]
Is there a time-of-check to time-of-use race here that could cause CPU starvation for newly awakened tasks?
The policy is evaluated in kvm_arch_vcpu_load(), which clears HSTATUS.VTW when the vCPU is the sole runnable task.
If an interrupt wakes another task on the same CPU while the vCPU is in the guest, but preemption is not immediately forced (e.g., the vCPU still has CFS timeslice remaining), KVM handles the exit and re-enters the guest loop in kvm_arch_vcpu_ioctl_run() without calling kvm_arch_vcpu_load().
Could the guest then execute WFI natively because HSTATUS.VTW remains stale and cleared, halting the physical CPU and starving the newly woken task until the next hardware interrupt?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Yuhang.Chen,
kernel test robot noticed the following build errors:
[auto build test ERROR on kvm/queue]
[also build test ERROR on kvm/next mst-vhost/linux-next linus/master v7.2-rc6 next-20260804] [cannot apply to kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Yuhang-Chen/RISC-V-KVM-Add-kvm-riscv-wfi_trap_policy-to-control-VS-mode-WFI-trapping/20260805-030043 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20260804024707.2400404-1-yhchen312%40gmail.com patch subject: [PATCH v2] RISC-V: KVM: Add kvm-riscv.wfi_trap_policy to control VS-mode WFI trapping config: riscv-randconfig-001-20260805 (https://download.01.org/0day-ci/archive/20260805/[email protected]/config) compiler: riscv64-linux-gcc (GCC) 8.5.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260805/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All error/warnings (new ones prefixed by >>):
>> arch/riscv/kvm/vcpu.c:60:13: error: expected declaration specifiers or '...' before string constant
early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);
^~~~~~~~~~~~~~~~~~~~~~~~~~~
>> arch/riscv/kvm/vcpu.c:60:42: error: expected declaration specifiers or '...' before 'early_kvm_riscv_wfi_trap_policy_cfg'
early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> arch/riscv/kvm/vcpu.c:43:19: warning: 'early_kvm_riscv_wfi_trap_policy_cfg' defined but not used [-Wunused-function] static int __init early_kvm_riscv_wfi_trap_policy_cfg(char *arg)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vim +60 arch/riscv/kvm/vcpu.c
39
40 static enum kvm_riscv_wfi_trap_policy kvm_riscv_wfi_trap_policy __read_mostly =
41 KVM_RISCV_WFI_TRAP;
42
> 43 static int __init early_kvm_riscv_wfi_trap_policy_cfg(char *arg)
44 {
45 if (!arg)
46 return -EINVAL;
47
48 if (strcmp(arg, "trap") == 0) {
49 kvm_riscv_wfi_trap_policy = KVM_RISCV_WFI_TRAP;
50 return 0;
51 }
52
53 if (strcmp(arg, "auto") == 0) {
54 kvm_riscv_wfi_trap_policy = KVM_RISCV_WFI_AUTO;
55 return 0;
56 }
57
58 return -EINVAL;
59 }
> 60 early_param("kvm-riscv.wfi_trap_policy", early_kvm_riscv_wfi_trap_policy_cfg);
61 --
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki