KVM: arm64: pKVM vCPU state management at EL2
Fuad Tabba
reviewerv7 cover KVM: arm64: pKVM vCPU state management at EL2
Hi folks,
Changes since v6 [1]:
- Rebased onto v7.2-rc5.
- Expanded the comment and commit message in patch 8 explaining why PC and PSTATE are copied back on every exit. No functional change.
- Collected Steffen's Reviewed-by on patch 3 (thanks!).
Building on Will's pKVM infrastructure series [2], this series reworks how pKVM moves vCPU state between the host and EL2, and stops copying a non-protected guest's state on every world switch.
EL2 gains proper primitives for the state it transfers: vCPU lookup helpers, and VGIC flush/sync that reduces how much host state EL2 dereferences. The series also moves some preparatory code (such as sys reg access and PSCI helpers) to shared headers and HYP, and implements lazy copying of a non-protected guest's register state back to the host until the host actually needs it, instead of on every exit.
This is the first of two series moving pKVM vCPU state management to EL2. The follow-up completes the job for protected VMs: state isolation, PSCI handling at EL2, and the resulting API behaviour.
The series is structured as follows:
01-04: Preparatory refactoring (MPIDR, sys reg access, vCPU reset, PSCI
helpers) to shared headers and HYP.
05: Host and hypervisor vCPU lookup primitives. 06-07: VGIC: reduce EL2's exposure to host state, add flush/sync primitives. 08: Lazy state sync for non-protected guests.
Based on v7.2-rc5 (f5098b6bae761).
Cheers,
/fuad
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
Fuad Tabba (5):
KVM: arm64: Extract MPIDR computation into a shared header
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
KVM: arm64: Factor out reusable vCPU reset helpers
KVM: arm64: Move PSCI helper functions to a shared header
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
Marc Zyngier (3):
KVM: arm64: Add host and hypervisor vCPU lookup primitives
KVM: arm64: Minimise EL2's exposure of host VGIC state during world
switch
KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
arch/arm64/include/asm/kvm_arm.h | 12 ++ arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/include/asm/kvm_emulate.h | 63 +++++++ arch/arm64/include/asm/kvm_host.h | 2 + arch/arm64/kvm/arm.c | 7 + arch/arm64/kvm/handle_exit.c | 23 +++ arch/arm64/kvm/hyp/exception.c | 34 +--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 261 +++++++++++++++++++++++---- arch/arm64/kvm/psci.c | 30 +-- arch/arm64/kvm/reset.c | 60 +----- arch/arm64/kvm/sys_regs.c | 14 +- arch/arm64/kvm/sys_regs.h | 19 ++ include/kvm/arm_psci.h | 27 +++ 13 files changed, 393 insertions(+), 160 deletions(-)
base-commit: f5098b6bae761e346ebcd9da7f95622c04733cff
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 1/8 KVM: arm64: Extract MPIDR computation into a shared header
Extract the vCPU MPIDR computation embedded in reset_mpidr() into a kvm_calculate_mpidr() inline in sys_regs.h, so it can be computed without duplicating the logic. A follow-up series reuses it to reset protected vCPUs at EL2.
No functional change intended.
--- arch/arm64/kvm/sys_regs.c | 14 +------------- arch/arm64/kvm/sys_regs.h | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+), 13 deletions(-)
arch/arm64/kvm/sys_regs.c +1 -13
@@ -976,21 +976,9 @@ static u64 reset_actlr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r)static u64 reset_mpidr (struct kvm_vcpu *vcpu,const struct sys_reg_desc *r) { -u64 mpidr; +u64 mpidr =kvm_calculate_mpidr (vcpu); -/* -* Map the vcpu_id into the first three affinity level fields of -* the MPIDR. We limit the number of VCPUs in level 0 due to a -* limitation to 16 CPUs in that level in the ICC_SGIxR registers -* of the GICv3 to be able to address each CPU directly when -* sending IPIs. -*/ - mpidr = (vcpu->vcpu_id &0x0f ) <<MPIDR_LEVEL_SHIFT (0 ); - mpidr |= ((vcpu->vcpu_id >>4 ) &0xff ) <<MPIDR_LEVEL_SHIFT (1 ); - mpidr |= ((vcpu->vcpu_id >>12 ) &0xff ) <<MPIDR_LEVEL_SHIFT (2 ); - mpidr |= (1ULL <<31 );vcpu_write_sys_reg (vcpu, mpidr,MPIDR_EL1 ); -return mpidr; }
arch/arm64/kvm/sys_regs.h +19 -0
@@ -222,6 +222,25 @@ find_reg(const struct sys_reg_params *params, const struct sys_reg_desc table[],return __inline_bsearch ((void *)pval, table, num,sizeof (table[0 ]), match_sys_reg); } +static inline u64 kvm_calculate_mpidr (const struct kvm_vcpu *vcpu) +{ +u64 mpidr; + +/* +* Map the vcpu_id into the first three affinity level fields of +* the MPIDR. We limit the number of VCPUs in level 0 due to a +* limitation to 16 CPUs in that level in the ICC_SGIxR registers +* of the GICv3 to be able to address each CPU directly when +* sending IPIs. +*/ + mpidr = (vcpu->vcpu_id &0x0f ) <<MPIDR_LEVEL_SHIFT (0 ); + mpidr |= ((vcpu->vcpu_id >>4 ) &0xff ) <<MPIDR_LEVEL_SHIFT (1 ); + mpidr |= ((vcpu->vcpu_id >>12 ) &0xff ) <<MPIDR_LEVEL_SHIFT (2 ); + mpidr |= (1ULL <<31 ); + +return mpidr; +} +const struct sys_reg_desc *get_reg_by_id (u64 id,const struct sys_reg_desc table[],unsigned int num);
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 2/8 KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
In reply to Fuad TabbaThe vcpu_{read,write}_sys_reg() accessors abstract where a system register lives, but their VHE implementation cannot be linked into the nVHE hypervisor, so exception.c open-codes has_vhe() wrappers instead.
Redirect the accessors to the raw context accessors for nVHE hyp builds, where registers are always in memory, and drop the local wrappers. This lets hyp code reuse helpers built on the canonical accessors.
No functional change intended.
--- arch/arm64/include/asm/kvm_emulate.h | 6 +++++ arch/arm64/kvm/hyp/exception.c | 34 ++++++++-------------------- 2 files changed, 15 insertions(+), 25 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +6 -0
@@ -506,6 +506,12 @@ static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)return __vcpu_sys_reg (vcpu,MPIDR_EL1 ) &MPIDR_HWID_BITMASK ; } +/* In nVHE hyp code, registers are always in memory: use the raw accessors. */ +#if defined(__KVM_NVHE_HYPERVISOR__) +#define vcpu_read_sys_reg (v, r) __vcpu_sys_reg(v, r) +#define vcpu_write_sys_reg (v, x, r) __vcpu_assign_sys_reg(v, r, x) +#endif +static inline void kvm_vcpu_set_be (struct kvm_vcpu *vcpu) {if (vcpu_mode_is_32bit (vcpu)) {
arch/arm64/kvm/hyp/exception.c +9 -25
@@ -20,22 +20,6 @@#error Hypervisor code only!#endif -static inline u64 __vcpu_read_sys_reg (const struct kvm_vcpu *vcpu,int reg) -{ -if (has_vhe ()) -return vcpu_read_sys_reg (vcpu, reg); - -return __vcpu_sys_reg (vcpu, reg); -} - -static inline void __vcpu_write_sys_reg (struct kvm_vcpu *vcpu,u64 val,int reg) -{ -if (has_vhe ()) -vcpu_write_sys_reg (vcpu, val, reg); -else -__vcpu_assign_sys_reg (vcpu, reg, val); -} -static void __vcpu_write_spsr (struct kvm_vcpu *vcpu,unsigned long target_mode,u64 val) {
@@ -101,14 +85,14 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,switch (target_mode) {case PSR_MODE_EL1h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL1 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL1 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 );break ;case PSR_MODE_EL2h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL2 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL2 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 );break ;default :/* Don't do that */
@@ -185,7 +169,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,*/ static unsigned long get_except32_cpsr (struct kvm_vcpu *vcpu,u32 mode) { -u32 sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); +u32 sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 );unsigned long old, new; old = *vcpu_cpsr (vcpu);
@@ -281,7 +265,7 @@ static void enter_exception32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset) {unsigned long spsr = *vcpu_cpsr (vcpu);bool is_thumb = (spsr &PSR_AA32_T_BIT ); -u32 sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); +u32 sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 );u32 return_address; *vcpu_cpsr (vcpu) =get_except32_cpsr (vcpu, mode);
@@ -305,7 +289,7 @@ static void enter_exception32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)if (sctlr & (1 <<13 )) vect_offset +=0xffff0000 ;else /* always have security exceptions */ - vect_offset +=__vcpu_read_sys_reg (vcpu,VBAR_EL1 ); + vect_offset +=vcpu_read_sys_reg (vcpu,VBAR_EL1 ); *vcpu_pc (vcpu) = vect_offset; }
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 3/8 KVM: arm64: Factor out reusable vCPU reset helpers
In reply to Fuad TabbaPull the reusable pieces out of kvm_reset_vcpu(): expose the reset PSTATE values in kvm_arm.h, and split the core register reset and the PSCI-driven reset into kvm_reset_vcpu_core() and kvm_reset_vcpu_psci(). A follow-up series reuses these to reset protected vCPUs at EL2.
No functional change intended.
--- arch/arm64/include/asm/kvm_arm.h | 12 ++++++ arch/arm64/include/asm/kvm_emulate.h | 57 ++++++++++++++++++++++++++ arch/arm64/kvm/reset.c | 60 ++-------------------------- 3 files changed, 72 insertions(+), 57 deletions(-)
arch/arm64/include/asm/kvm_arm.h +12 -0
@@ -348,4 +348,16 @@ { PSR_AA32_MODE_UND, "32-bit UND" }, \ { PSR_AA32_MODE_SYS, "32-bit SYS" } +/* +* ARMv8 Reset Values +*/ +#define VCPU_RESET_PSTATE_EL1 (PSR_MODE_EL1h | PSR_A_BIT | PSR_I_BIT | \ + PSR_F_BIT | PSR_D_BIT) + +#define VCPU_RESET_PSTATE_EL2 (PSR_MODE_EL2h | PSR_A_BIT | PSR_I_BIT | \ + PSR_F_BIT | PSR_D_BIT) + +#define VCPU_RESET_PSTATE_SVC (PSR_AA32_MODE_SVC | PSR_AA32_A_BIT | \ + PSR_AA32_I_BIT | PSR_AA32_F_BIT) +#endif /* __ARM64_KVM_ARM_H__ */
arch/arm64/include/asm/kvm_emulate.h +57 -0
61 lines not on this page · raw message
arch/arm64/kvm/reset.c +3 -57
85 lines not on this page · raw message
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 4/8 KVM: arm64: Move PSCI helper functions to a shared header
In reply to Fuad TabbaMove kvm_psci_valid_affinity() and kvm_psci_narrow_to_32bit() from psci.c to include/kvm/arm_psci.h, and move psci_affinity_mask() there too, renaming it kvm_psci_affinity_mask() now that it is no longer file-local. A follow-up series handles some protected-guest PSCI calls at EL2 using these helpers.
No functional change intended.
--- arch/arm64/kvm/psci.c | 30 +----------------------------- include/kvm/arm_psci.h | 27 +++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 29 deletions(-)
arch/arm64/kvm/psci.c +1 -29
@@ -21,16 +21,6 @@* as described in ARM document number ARM DEN 0022A. */ -#define AFFINITY_MASK (level) ~((0x1UL << ((level) * MPIDR_LEVEL_BITS)) - 1) - -static unsigned long psci_affinity_mask (unsigned long affinity_level) -{ -if (affinity_level <=3 ) -return MPIDR_HWID_BITMASK &AFFINITY_MASK (affinity_level); - -return 0 ; -} -static unsigned long kvm_psci_vcpu_suspend (struct kvm_vcpu *vcpu) {/*
@@ -51,12 +41,6 @@ static unsigned long kvm_psci_vcpu_suspend(struct kvm_vcpu *vcpu)return PSCI_RET_SUCCESS ; } -static inline bool kvm_psci_valid_affinity (struct kvm_vcpu *vcpu, -unsigned long affinity) -{ -return !(affinity & ~MPIDR_HWID_BITMASK ); -} -static unsigned long kvm_psci_vcpu_on (struct kvm_vcpu *source_vcpu) {struct vcpu_reset_state *reset_state;
@@ -135,7 +119,7 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu)return PSCI_RET_INVALID_PARAMS ;/* Determine target affinity mask */ - target_affinity_mask =psci_affinity_mask (lowest_affinity_level); + target_affinity_mask =kvm_psci_affinity_mask (lowest_affinity_level);if (!target_affinity_mask)return PSCI_RET_INVALID_PARAMS ;
@@ -220,18 +204,6 @@ static void kvm_psci_system_suspend(struct kvm_vcpu *vcpu) run->exit_reason =KVM_EXIT_SYSTEM_EVENT ; } -static void kvm_psci_narrow_to_32bit (struct kvm_vcpu *vcpu) -{ -int i; - -/* -* Zero the input registers' upper 32 bits. They will be fully -* zeroed on exit, so we're fine changing them in place. -*/ -for (i =1 ; i <4 ; i++) -vcpu_set_reg (vcpu, i,lower_32_bits (vcpu_get_reg (vcpu, i))); -} -static unsigned long kvm_psci_check_allowed_function (struct kvm_vcpu *vcpu,u32 fn) {/*
include/kvm/arm_psci.h +27 -0
@@ -38,6 +38,33 @@ static inline int kvm_psci_version(struct kvm_vcpu *vcpu)return KVM_ARM_PSCI_0_1 ; } +/* Narrow the PSCI register arguments (r1 to r3) to 32 bits. */ +static inline void kvm_psci_narrow_to_32bit (struct kvm_vcpu *vcpu) +{ +int i; + +/* +* Zero the input registers' upper 32 bits. They will be fully +* zeroed on exit, so we're fine changing them in place. +*/ +for (i =1 ; i <4 ; i++) +vcpu_set_reg (vcpu, i,lower_32_bits (vcpu_get_reg (vcpu, i))); +} + +static inline bool kvm_psci_valid_affinity (struct kvm_vcpu *vcpu, +unsigned long affinity) +{ +return !(affinity & ~MPIDR_HWID_BITMASK ); +} + +static inline unsigned long kvm_psci_affinity_mask (unsigned long affinity_level) +{ +if (affinity_level <=3 ) +return MPIDR_HWID_BITMASK & + ~((0x1UL << (affinity_level *MPIDR_LEVEL_BITS )) -1 ); + +return 0 ; +}int kvm_psci_call (struct kvm_vcpu *vcpu);
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 5/8 KVM: arm64: Add host and hypervisor vCPU lookup primitives
In reply to Fuad TabbaFrom: Marc Zyngier <[email protected]>
The nVHE hypervisor repeatedly resolves a host vCPU into the EL2 address space and validates that the loaded hyp vCPU matches it, with that logic open-coded in each handler.
Add __get_host_hyp_vcpus() and the get_host_hyp_vcpus() macro, which translate the host vCPU into the hypervisor's address space and, when pKVM is enabled, also return the loaded hyp vCPU if it matches. If pKVM is enabled but the loaded hyp vCPU does not correspond to the requested host vCPU, both the host and hyp vCPU are returned as NULL. Convert handle___kvm_vcpu_run() to use it.
No functional change intended.
--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 52 ++++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 14 deletions(-)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +38 -14
@@ -214,14 +214,45 @@ static void handle___pkvm_vcpu_put(struct kvm_cpu_context *host_ctxt)pkvm_put_hyp_vcpu (hyp_vcpu); } -static void handle___kvm_vcpu_run (struct kvm_cpu_context *host_ctxt) +static struct kvm_vcpu *__get_host_hyp_vcpus (struct kvm_vcpu *arg, +struct pkvm_hyp_vcpu **hyp_vcpup) { - DECLARE_REG(struct kvm_vcpu *, host_vcpu, host_ctxt, 1); -int ret; +struct kvm_vcpu *host_vcpu =kern_hyp_va (arg); +struct pkvm_hyp_vcpu *hyp_vcpu =NULL ;if (unlikely (is_protected_kvm_enabled ())) { -struct pkvm_hyp_vcpu *hyp_vcpu =pkvm_get_loaded_hyp_vcpu (); + hyp_vcpu =pkvm_get_loaded_hyp_vcpu (); +if (!hyp_vcpu || hyp_vcpu->host_vcpu != host_vcpu) { + hyp_vcpu =NULL ; + host_vcpu =NULL ; + } + } + + *hyp_vcpup = hyp_vcpu; +return host_vcpu; +} + +#define get_host_hyp_vcpus (ctxt, regnr, hyp_vcpup) \ + ({ \ + DECLARE_REG(struct kvm_vcpu *, __vcpu, ctxt, regnr); \ + __get_host_hyp_vcpus(__vcpu, hyp_vcpup); \ + }) + +static void handle___kvm_vcpu_run (struct kvm_cpu_context *host_ctxt) +{ +struct pkvm_hyp_vcpu *hyp_vcpu; +struct kvm_vcpu *host_vcpu; +int ret; + + host_vcpu =get_host_hyp_vcpus (host_ctxt,1 , &hyp_vcpu); + +if (!host_vcpu) { + ret = -EINVAL ; + goto out; + } + +if (unlikely (hyp_vcpu)) {/* * KVM (and pKVM) doesn't support SME guests for now, and * ensures that SME features aren't enabled in pstate when
@@ -233,23 +264,16 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt) goto out; } -if (!hyp_vcpu) { - ret = -EINVAL ; - goto out; - } -flush_hyp_vcpu (hyp_vcpu); ret =__kvm_vcpu_run (&hyp_vcpu->vcpu);sync_hyp_vcpu (hyp_vcpu); }else { -struct kvm_vcpu *vcpu =kern_hyp_va (host_vcpu); -/* The host is fully trusted, run its vCPU directly. */ -fpsimd_lazy_switch_to_guest (vcpu); - ret =__kvm_vcpu_run (vcpu); -fpsimd_lazy_switch_to_host (vcpu); +fpsimd_lazy_switch_to_guest (host_vcpu); + ret =__kvm_vcpu_run (host_vcpu); +fpsimd_lazy_switch_to_host (host_vcpu); } out:cpu_reg (host_ctxt,1 ) = ret;
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 6/8 KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
In reply to Fuad TabbaFrom: Marc Zyngier <[email protected]>
The host passes a vgic_v3_cpu_if pointer to the __vgic_v3_save_aprs and __vgic_v3_restore_vmcr_aprs hypercalls, which EL2 dereferences wholesale. That exposes the host's full VGIC emulation state to the hypervisor, against pKVM's isolation goals.
Recover the host vCPU from the supplied cpu_if via container_of() and copy only vgic_vmcr and the active priority registers between EL2's hyp-side state and the host vCPU, so EL2 no longer dereferences the host's vgic_v3_cpu_if directly.
--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 67 ++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 4 deletions(-)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +63 -4
@@ -7,6 +7,8 @@#include <hyp/adjust_pc.h> #include <hyp/switch.h> +#include <linux/irqchip/arm-gic-v3.h> +#include <asm/pgtable-types.h> #include <asm/kvm_asm.h> #include <asm/kvm_emulate.h>
@@ -239,6 +241,16 @@ static struct kvm_vcpu *__get_host_hyp_vcpus(struct kvm_vcpu *arg,__get_host_hyp_vcpus (__vcpu, hyp_vcpup); \ }) +#define get_host_hyp_vcpus_from_vgic_v3_cpu_if (ctxt, regnr, hyp_vcpup) \ + ({ \ + DECLARE_REG(struct vgic_v3_cpu_if *, cif, ctxt, regnr);\ + struct kvm_vcpu *__vcpu = container_of(cif, \ + struct kvm_vcpu, \ + arch.vgic_cpu.vgic_v3); \ + \ + __get_host_hyp_vcpus(__vcpu, hyp_vcpup); \ + }) +static void handle___kvm_vcpu_run (struct kvm_cpu_context *host_ctxt) {struct pkvm_hyp_vcpu *hyp_vcpu;
@@ -508,16 +520,63 @@ static void handle___vgic_v3_init_lrs(struct kvm_cpu_context *host_ctxt)static void handle___vgic_v3_save_aprs (struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(struct vgic_v3_cpu_if *, cpu_if, host_ctxt, 1); +struct pkvm_hyp_vcpu *hyp_vcpu; +struct kvm_vcpu *host_vcpu; -__vgic_v3_save_aprs (kern_hyp_va (cpu_if)); + host_vcpu =get_host_hyp_vcpus_from_vgic_v3_cpu_if (host_ctxt,1 , + &hyp_vcpu); +if (!host_vcpu) +return ; + +if (unlikely (hyp_vcpu)) { +struct vgic_v3_cpu_if *hyp_cpu_if, *host_cpu_if; +int i; + + hyp_cpu_if = &hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3; +__vgic_v3_save_aprs (hyp_cpu_if); + + host_cpu_if = &host_vcpu->arch.vgic_cpu.vgic_v3; + host_cpu_if->vgic_vmcr = hyp_cpu_if->vgic_vmcr; +for (i =0 ; i <ARRAY_SIZE (host_cpu_if->vgic_ap0r); i++) { + host_cpu_if->vgic_ap0r[i] = hyp_cpu_if->vgic_ap0r[i]; + host_cpu_if->vgic_ap1r[i] = hyp_cpu_if->vgic_ap1r[i]; + } + }else { +__vgic_v3_save_aprs (&host_vcpu->arch.vgic_cpu.vgic_v3); + } }static void handle___vgic_v3_restore_vmcr_aprs (struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(struct vgic_v3_cpu_if *, cpu_if, host_ctxt, 1); +struct pkvm_hyp_vcpu *hyp_vcpu; +struct kvm_vcpu *host_vcpu; -__vgic_v3_restore_vmcr_aprs (kern_hyp_va (cpu_if)); + host_vcpu =get_host_hyp_vcpus_from_vgic_v3_cpu_if (host_ctxt,1 , + &hyp_vcpu); +if (!host_vcpu) +return ; + +if (unlikely (hyp_vcpu)) { +struct vgic_v3_cpu_if *hyp_cpu_if, *host_cpu_if; +int i; + + hyp_cpu_if = &hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3; + host_cpu_if = &host_vcpu->arch.vgic_cpu.vgic_v3; + + hyp_cpu_if->vgic_vmcr = host_cpu_if->vgic_vmcr; +/* Should be a one-off */ + hyp_cpu_if->vgic_sre = (ICC_SRE_EL1_DIB | +ICC_SRE_EL1_DFB | +ICC_SRE_EL1_SRE ); +for (i =0 ; i <ARRAY_SIZE (host_cpu_if->vgic_ap0r); i++) { + hyp_cpu_if->vgic_ap0r[i] = host_cpu_if->vgic_ap0r[i]; + hyp_cpu_if->vgic_ap1r[i] = host_cpu_if->vgic_ap1r[i]; + } + +__vgic_v3_restore_vmcr_aprs (hyp_cpu_if); + }else { +__vgic_v3_restore_vmcr_aprs (&host_vcpu->arch.vgic_cpu.vgic_v3); + } }static void handle___pkvm_init (struct kvm_cpu_context *host_ctxt)
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 7/8 KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
In reply to Fuad TabbaFrom: Marc Zyngier <[email protected]>
pKVM performs its own world switch for protected VMs but has no primitives to move the per-vCPU VGIC state between the host and hypervisor vCPU contexts.
Add flush_hyp_vgic_state() and sync_hyp_vgic_state(). Flush copies vgic_hcr, the in-use list registers and used_lrs from the host into the hyp vCPU and pins vgic_sre to a fixed value; sync copies vgic_hcr, vgic_vmcr and the in-use list registers back. The active priority registers are handled separately by the save/restore-aprs path.
Bound used_lrs by hyp_gicv3_nr_lr, the cached implemented-LR count, instead of reading ICH_VTR_EL2 on each entry. That clamps the host-supplied value and avoids a per-entry sysreg read that is costly under NV.
--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 55 ++++++++++++++++++++++-------- 1 file changed, 41 insertions(+), 14 deletions(-)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +41 -14
@@ -104,6 +104,45 @@ static void fpsimd_sve_sync(struct kvm_vcpu *vcpu) *host_data_ptr (fp_owner) =FP_STATE_HOST_OWNED ; } +static void flush_hyp_vgic_state (struct pkvm_hyp_vcpu *hyp_vcpu) +{ +struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu; +struct vgic_v3_cpu_if *host_cpu_if, *hyp_cpu_if; +unsigned int used_lrs, i; + + host_cpu_if = &host_vcpu->arch.vgic_cpu.vgic_v3; + hyp_cpu_if = &hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3; + + used_lrs = host_cpu_if->used_lrs; + used_lrs =min (used_lrs, hyp_gicv3_nr_lr); + + hyp_cpu_if->vgic_hcr = host_cpu_if->vgic_hcr; +/* Should be a one-off */ + hyp_cpu_if->vgic_sre = (ICC_SRE_EL1_DIB | +ICC_SRE_EL1_DFB | +ICC_SRE_EL1_SRE ); + hyp_cpu_if->used_lrs = used_lrs; + +for (i =0 ; i < used_lrs; i++) + hyp_cpu_if->vgic_lr[i] = host_cpu_if->vgic_lr[i]; +} + +static void sync_hyp_vgic_state (struct pkvm_hyp_vcpu *hyp_vcpu) +{ +struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu; +struct vgic_v3_cpu_if *host_cpu_if, *hyp_cpu_if; +unsigned int i; + + host_cpu_if = &host_vcpu->arch.vgic_cpu.vgic_v3; + hyp_cpu_if = &hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3; + + host_cpu_if->vgic_hcr = hyp_cpu_if->vgic_hcr; + host_cpu_if->vgic_vmcr = hyp_cpu_if->vgic_vmcr; + +for (i =0 ; i < hyp_cpu_if->used_lrs; i++) + host_cpu_if->vgic_lr[i] = hyp_cpu_if->vgic_lr[i]; +} +static void flush_debug_state (struct pkvm_hyp_vcpu *hyp_vcpu) {struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu;
@@ -152,13 +191,7 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.vsesr_el2 = host_vcpu->arch.vsesr_el2; - hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3; - -/* Bound used_lrs by the number of implemented list registers. */ - hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs = - min_t(unsigned int, - hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs, - hyp_gicv3_nr_lr); +flush_hyp_vgic_state (hyp_vcpu); hyp_vcpu->vcpu.arch.pid = host_vcpu->arch.pid; }
@@ -166,9 +199,6 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)static void sync_hyp_vcpu (struct pkvm_hyp_vcpu *hyp_vcpu) {struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu; -struct vgic_v3_cpu_if *hyp_cpu_if = &hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3; -struct vgic_v3_cpu_if *host_cpu_if = &host_vcpu->arch.vgic_cpu.vgic_v3; -unsigned int i;fpsimd_sve_sync (&hyp_vcpu->vcpu);sync_debug_state (hyp_vcpu);
@@ -181,10 +211,7 @@ static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) host_vcpu->arch.iflags = hyp_vcpu->vcpu.arch.iflags; - host_cpu_if->vgic_hcr = hyp_cpu_if->vgic_hcr; - host_cpu_if->vgic_vmcr = hyp_cpu_if->vgic_vmcr; -for (i =0 ; i < hyp_cpu_if->used_lrs; ++i) - host_cpu_if->vgic_lr[i] = hyp_cpu_if->vgic_lr[i]; +sync_hyp_vgic_state (hyp_vcpu); }static void handle___pkvm_vcpu_load (struct kvm_cpu_context *host_ctxt)
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH v7 8/8 KVM: arm64: Implement lazy vCPU state sync for non-protected guests
In reply to Fuad TabbapKVM copies a non-protected guest's register context between the host and the hypervisor on every world switch, even when the host never inspects it. Defer the copy: on entry, flush the host context into the hyp vCPU only when the host marked it dirty (PKVM_HOST_STATE_DIRTY); on exit, leave it in the hyp vCPU and copy it back only when the host needs it, via a __pkvm_vcpu_sync_state hypercall or at vcpu put. A protected guest's context is copied as before, since lazy sync only helps where the host is trusted to see the guest's registers.
PC and PSTATE are the exception, copied back on every exit: the kvm_exit tracepoint reports the guest's real exit PC, and vcpu_mode_is_bad_32bit(), the SError-masking checks and the host software-step state machine all read PSTATE before the next on-demand sync. Without it an ARM_EXCEPTION_IRQ exit syncs nothing, so flush_debug_state() pushes a stale PSTATE.SS back on the next entry and single-stepping runs an extra instruction before the step is reported.
The host needs the full context when it is about to read it (trap handling) or write it (the SError injection that writes ESR_EL1). Sync both from handle_exit_early(), which runs non-preemptible so the loaded hyp vCPU is stable without a preempt guard.
--- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/include/asm/kvm_host.h | 2 + arch/arm64/kvm/arm.c | 7 +++ arch/arm64/kvm/handle_exit.c | 23 ++++++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 89 ++++++++++++++++++++++++++++-- 5 files changed, 117 insertions(+), 5 deletions(-)
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -113,6 +113,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_finalize_teardown_vm, __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_load, __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_put, + __KVM_HOST_SMCCC_FUNC___pkvm_vcpu_sync_state, __KVM_HOST_SMCCC_FUNC___pkvm_tlb_flush_vmid,MARKER (__KVM_HOST_SMCCC_FUNC_MAX)
arch/arm64/include/asm/kvm_host.h +2 -0
@@ -1051,6 +1051,8 @@ struct kvm_vcpu_arch {#define INCREMENT_PC __vcpu_single_flag(iflags, BIT(1))/* Target EL/MODE (not a single flag, but let's abuse the macro) */ #define EXCEPT_MASK __vcpu_single_flag(iflags, GENMASK(3, 1)) +/* Host-set: the hyp flushes the non-protected vCPU state in on entry */ +#define PKVM_HOST_STATE_DIRTY __vcpu_single_flag(iflags, BIT(4))/* Helpers to encode exceptions with minimum fuss */ #define __EXCEPT_MASK_VAL unpack_vcpu_flag(EXCEPT_MASK)
arch/arm64/kvm/arm.c +7 -0
@@ -735,6 +735,10 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)if (is_protected_kvm_enabled ()) {kvm_call_hyp (__vgic_v3_save_aprs, &vcpu->arch.vgic_cpu.vgic_v3);kvm_call_hyp_nvhe (__pkvm_vcpu_put); + +/* __pkvm_vcpu_put implies a sync of the state */ +if (!kvm_vm_is_protected (vcpu->kvm)) +vcpu_set_flag (vcpu,PKVM_HOST_STATE_DIRTY ); }kvm_vcpu_put_debug (vcpu);
@@ -966,6 +970,9 @@ int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)return ret;if (is_protected_kvm_enabled ()) { +/* Start with the vcpu in a dirty state */ +if (!kvm_vm_is_protected (vcpu->kvm)) +vcpu_set_flag (vcpu,PKVM_HOST_STATE_DIRTY ); ret =pkvm_create_hyp_vm (kvm);if (ret)return ret;
arch/arm64/kvm/handle_exit.c +23 -0
@@ -486,9 +486,32 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) } } +static void handle_exit_pkvm_state (struct kvm_vcpu *vcpu,int exception_index) +{ +int exception_code =ARM_EXCEPTION_CODE (exception_index); + +if (!is_protected_kvm_enabled () ||kvm_vm_is_protected (vcpu->kvm)) +return ; + +/* +* Sync the context back when the host will read (trap) or write +* (SError) it. Preempt-off here, so the loaded hyp vCPU is stable. +*/ +if (exception_code ==ARM_EXCEPTION_TRAP || + exception_code ==ARM_EXCEPTION_EL1_SERROR || +ARM_SERROR_PENDING (exception_index)) { +kvm_call_hyp_nvhe (__pkvm_vcpu_sync_state); +vcpu_set_flag (vcpu,PKVM_HOST_STATE_DIRTY ); + }else { +vcpu_clear_flag (vcpu,PKVM_HOST_STATE_DIRTY ); + } +} +/* For exit types that need handling before we can be preempted */ void handle_exit_early (struct kvm_vcpu *vcpu,int exception_index) { +handle_exit_pkvm_state (vcpu, exception_index); +if (ARM_SERROR_PENDING (exception_index)) {if (this_cpu_has_cap (ARM64_HAS_RAS_EXTN )) {u64 disr =kvm_vcpu_get_disr (vcpu);
arch/arm64/kvm/hyp/nvhe/hyp-main.c +84 -5
120 lines not on this page · raw message
Signature · 1 line
2.39.5
[email protected]
AI reviewRe: [PATCH v7 4/8] KVM: arm64: Move PSCI helper functions to a shared header
In reply to Fuad TabbaThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Missing header dependencies in include/kvm/arm_psci.h --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Missing header dependencies in include/kvm/arm_psci.h
--
--- Patch [4]: [PATCH v7 4/8] KVM: arm64: Move PSCI helper functions to a shared header --- commit 4d648c3ac7ab8549b7dcf7d52de4756db87e329c
Author: Fuad Tabba <[email protected]>
KVM: arm64: Move PSCI helper functions to a shared header
This commit moves several PSCI helper functions from psci.c to the shared include/kvm/arm_psci.h header. This makes them available for upcoming protected-guest PSCI calls at EL2.
Quoted 4 lines
include/kvm/arm_psci.h +0
-0
[ ... ]
include/kvm/arm_psci.h static inline int kvm_psci_version(struct kvm_vcpu *vcpu) · line 52
Quoted 12 lines
include/kvm/arm_psci.h +12
-0+/* Narrow the PSCI register arguments (r1 to r3) to 32 bits. */ +static inline void kvm_psci_narrow_to_32bit (struct kvm_vcpu *vcpu) +{ +int i; + +/* +* Zero the input registers' upper 32 bits. They will be fully +* zeroed on exit, so we're fine changing them in place. +*/ +for (i =1 ; i <4 ; i++) +vcpu_set_reg (vcpu, i,lower_32_bits (vcpu_get_reg (vcpu, i))); +}
[Severity: Low]
Does this header need to include additional dependencies to remain self-contained?
This inline function uses vcpu_set_reg() and vcpu_get_reg(), which are defined in <asm/kvm_emulate.h>, as well as lower_32_bits() from <linux/wordpart.h>. If a new C file includes <kvm/arm_psci.h> without first including those dependencies, it looks like it could trigger a build failure.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH v7 2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
In reply to Fuad TabbaThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Pre-existing bug: enter_exception64 reads stale SCTLR_EL1 during early trap handling in nVHE/pKVM. --
--- Patch [2]: [PATCH v7 2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code --- commit e62bbea08d3143161297d701776a47e619a76db1
Author: Fuad Tabba <[email protected]>
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
This commit replaces the locally open-coded __vcpu_read_sys_reg and __vcpu_write_sys_reg wrappers in exception.c with the canonical vcpu_{read,write}_sys_reg accessors. For nVHE builds, it maps these accessors directly to the raw in-memory context accessors, allowing HYP code to reuse helpers without function duplication.
Quoted 4 lines
arch/arm64/kvm/hyp/exception.c +0
-0
[ ... ]
arch/arm64/kvm/hyp/exception.c static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode, · line 89
Quoted 9 lines
arch/arm64/kvm/hyp/exception.c +2
-3@@ -101,14 +85,14 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,switch (target_mode) {case PSR_MODE_EL1h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL1 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL1 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 );
[Severity: High]
This is a pre-existing issue, but does enter_exception64() read a stale SCTLR_EL1 value from the in-memory context here?
When a protected VM triggers an early EL2 trap, the hypervisor handles it via the kvm_handle_pvm_sysreg() -> inject_undef64() -> inject_sync64() call chain. At that point, the guest's system registers aren't fully saved to the in-memory vcpu->arch.ctxt.
While inject_sync64() explicitly synchronizes VBAR_EL1 from hardware to memory:
arch/arm64/kvm/hyp/nvhe/sys_regs.c:inject_sync64() {
...
__vcpu_assign_sys_reg(vcpu, VBAR_EL1, read_sysreg_el1(SYS_VBAR));
...
}It doesn't appear to synchronize SCTLR_EL1 before triggering exception injection.
Since nVHE builds map vcpu_read_sys_reg() directly to the memory context, enter_exception64() seems to use a stale SCTLR_EL1 value to calculate the PSR_PAN_BIT and PSR_SSBS_BIT for the exception's PSTATE.
This corrupted PSTATE is then written directly to the hardware SYS_SPSR_EL2 by inject_sync64(), potentially poisoning the guest's state upon ERET.
Should SCTLR_EL1 also be synchronized from hardware before injection?
Quoted 11 lines
arch/arm64/kvm/hyp/exception.c +4
-3+vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 );break ;case PSR_MODE_EL2h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL2 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL2 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 );break ;default :
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Fuad Tabba
reviewerRe: [PATCH v7 4/8] KVM: arm64: Move PSCI helper functions to a shared header
In reply to [email protected]Quoted 3 lines · On Wed, 29 Jul 2026 at 14:31, <[email protected]> wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Missing header dependencies ininclude/kvm/arm_psci.h
nope:
https://lore.kernel.org/all/CA+EHjTwpB2fmoYGfkP5RVZcB4sgbVPQ0=QR5dyUJnpQ_VQa+pA@mail.gmail.com/
/fuad
Quoted 43 lines
--
--- Patch [4]: [PATCH v7 4/8] KVM: arm64: Move PSCI helper functions to a shared header --- commit
4d648c3ac7ab8549b7dcf7d52de4756db87e329c
Author: Fuad Tabba <[email protected]>KVM: arm64: Move PSCI helper functions to a shared header
This commit moves several PSCI helper functions from
psci.cto the sharedinclude/kvm/arm_psci.hheader. This makes them available for upcoming protected-guest PSCI calls at EL2.Quoted 4 lines
include/kvm/arm_psci.h +0
-0[ ... ]
include/kvm/arm_psci.h static inline int kvm_psci_version(struct kvm_vcpu *vcpu) · line 52
Quoted 12 lines
include/kvm/arm_psci.h +12
-0+/* Narrow the PSCI register arguments (r1 to r3) to 32 bits. */ +static inline void kvm_psci_narrow_to_32bit (struct kvm_vcpu *vcpu) +{ +int i; + +/* +* Zero the input registers' upper 32 bits. They will be fully +* zeroed on exit, so we're fine changing them in place. +*/ +for (i =1 ; i <4 ; i++) +vcpu_set_reg (vcpu, i,lower_32_bits (vcpu_get_reg (vcpu, i))); +}[Severity: Low]
Does this header need to include additional dependencies to remain self-contained?This inline function uses
vcpu_set_reg()andvcpu_get_reg(), which are defined in <asm/kvm_emulate.h>, as well aslower_32_bits()from <linux/wordpart.h>. If a new C file includes <kvm/arm_psci.h> without first including those dependencies, it looks like it could trigger a build failure.--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH v7 8/8] KVM: arm64: Implement lazy vCPU state sync for non-protected guests
In reply to Fuad TabbaThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Missing HCR_EL2 state synchronization breaks virtual SError injection, causing an infinite…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Missing HCR_EL2 state synchronization breaks virtual SError injection, causing an infinite exception loop.
- [High] Lazy state sync fails to flush context after KVM_REQ_VCPU_RESET, causing vCPU resets (e.g., PSCI CPU_ON) to be ignored. --
--- Patch [8]: [PATCH v7 8/8] KVM: arm64: Implement lazy vCPU state sync for non-protected guests --- commit 80df5c4ca2cca0a1726813fa129b353353482892
Author: Fuad Tabba <[email protected]>
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
This commit optimizes world switches for non-protected guests by deferring the copy of the vCPU register context between the host and hypervisor. State is now only synced on entry if marked dirty, and on exit only when explicitly needed by the host.
Quoted 15 lines
arch/arm64/kvm/handle_exit.c +8
-0@@ -486,9 +486,32 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) } } +static void handle_exit_pkvm_state (struct kvm_vcpu *vcpu,int exception_index) +{ +int exception_code =ARM_EXCEPTION_CODE (exception_index); + +if (!is_protected_kvm_enabled () ||kvm_vm_is_protected (vcpu->kvm)) +return ; + +/*
arch/arm64/kvm/handle_exit.c int handle_exit(struct kvm_vcpu *vcpu, int exception_index) · line 506
Quoted 10 lines
arch/arm64/kvm/handle_exit.c +10
-0+* Sync the context back when the host will read (trap) or write +* (SError) it. Preempt-off here, so the loaded hyp vCPU is stable. +*/ +if (exception_code ==ARM_EXCEPTION_TRAP || + exception_code ==ARM_EXCEPTION_EL1_SERROR || +ARM_SERROR_PENDING (exception_index)) { +kvm_call_hyp_nvhe (__pkvm_vcpu_sync_state); +vcpu_set_flag (vcpu,PKVM_HOST_STATE_DIRTY ); + }else { +vcpu_clear_flag (vcpu,PKVM_HOST_STATE_DIRTY );
[Severity: High]
Is it possible for KVM_REQ_VCPU_RESET updates to be ignored due to this cleared flag?
If a secondary vCPU is powered on (for example, via PSCI CPU_ON), it receives KVM_REQ_VCPU_RESET and gets kicked, leading to an exit with an IRQ. Since that exit takes this else branch and clears the dirty flag, and the reset request is processed afterward in kvm_vcpu_check_requests() which modifies the host-side vCPU context, wouldn't flush_hyp_vcpu() skip syncing this updated reset state because the flag remains cleared?
This could result in the target vCPU resuming with its pre-IRQ stale context, eventually overwriting the new reset state on the next exit.
Quoted 3 lines
arch/arm64/kvm/handle_exit.c +3
-0+ } +} +
[ ... ]
arch/arm64/kvm/hyp/nvhe/hyp-main.c static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 208
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +0
-3@@ -203,9 +255,16 @@ static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)fpsimd_sve_sync (&hyp_vcpu->vcpu);sync_debug_state (hyp_vcpu); - host_vcpu->arch.ctxt = hyp_vcpu->vcpu.arch.ctxt; - - host_vcpu->arch.hcr_el2 = hyp_vcpu->vcpu.arch.hcr_el2;
[Severity: High]
Does removing this HCR_EL2 assignment break virtual SError injection and cause an infinite exception loop?
When the host injects a virtual SError into a non-protected guest under pKVM by setting HCR_VSE in the host-side vcpu->arch.hcr_el2, the hardware automatically clears the VSE bit in the physical HCR_EL2 register once the guest takes the exception.
Without syncing this hardware-cleared state back to the host here, the host's HCR_VSE bit remains set. On the next entry, flush_hyp_vcpu() will repeatedly re-inject the stale HCR_VSE bit from the host into the hyp vCPU, locking up the guest.
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +10
-0+if (pkvm_hyp_vcpu_is_protected (hyp_vcpu)) { + host_vcpu->arch.ctxt = hyp_vcpu->vcpu.arch.ctxt; + }else { +/* +* PC feeds trace_kvm_exit(), PSTATE.SS the host software-step +* machine, and both run before the next on-demand ctxt sync. +*/ + host_vcpu->arch.ctxt.regs.pc = hyp_vcpu->vcpu.arch.ctxt.regs.pc; + host_vcpu->arch.ctxt.regs.pstate = hyp_vcpu->vcpu.arch.ctxt.regs.pstate; + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
[email protected]
AI reviewRe: [PATCH v7 6/8] KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
In reply to Fuad TabbaThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [Critical] The patch fixes an arbitrary host pointer dereference in GICv3 hypercalls but completely misses the identical vulnerabilities in GICv5 hypercalls (handle___vgic_v5_save_apr and handle___vgic_v5_restore_vmcr_apr).
- [Critical] Unvalidated host pointer dereference in handle___kvm_adjust_pc allows arbitrary memory modification in EL2.
--
--- Patch [6]: [PATCH v7 6/8] KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch --- commit def324be61872b6056efce4e3eb9ff57eb9a9e10
Author: Marc Zyngier <[email protected]>
Subject: KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
This commit restricts EL2's exposure to host VGIC state by recovering the host vCPU securely rather than directly dereferencing the host-provided vgic_v3_cpu_if pointer in __vgic_v3_save_aprs and __vgic_v3_restore_vmcr_aprs. It introduces a helper to fetch the host vCPU using container_of() and copies only the necessary state.
Quoted 4 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +0
-0
[ ... ]
arch/arm64/kvm/hyp/nvhe/hyp-main.c static void handle___vgic_v3_save_aprs(struct kvm_cpu_context *host_ctxt) · line 525
Quoted 9 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +4
-2@@ -508,16 +520,63 @@ static void handle___vgic_v3_save_aprs(struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(struct vgic_v3_cpu_if *, cpu_if, host_ctxt, 1); +struct pkvm_hyp_vcpu *hyp_vcpu; +struct kvm_vcpu *host_vcpu; -__vgic_v3_save_aprs (kern_hyp_va (cpu_if)); + host_vcpu =get_host_hyp_vcpus_from_vgic_v3_cpu_if (host_ctxt,1 , + &hyp_vcpu);
[ ... ]
[Severity: Critical]
This is a pre-existing issue, but does the same regression exist for the GICv5 hypercalls?
The patch replaces the direct kern_hyp_va() dereference with a secure lookup for GICv3. However, it appears the same pattern remains in handle___vgic_v5_save_apr() and handle___vgic_v5_restore_vmcr_apr():
handle___vgic_v5_save_apr()
__vgic_v5_save_apr(kern_hyp_va(cpu_if));
Can a compromised host pass an arbitrary physical address here, causing __vgic_v5_save_apr() to write the hardware value of SYS_ICH_APR_EL2 to that chosen address, breaking EL2 isolation?
[Severity: Critical]
This is also a pre-existing issue, but does handle___kvm_adjust_pc() have a similar unvalidated host pointer dereference?
In arch/arm64/kvm/hyp/nvhe/hyp-main.c:
static void handle___kvm_adjust_pc(struct kvm_cpu_context *host_ctxt)
{
DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1);
__kvm_adjust_pc(kern_hyp_va(vcpu));
}If the host kernel passes a crafted vcpu pointer, could it point to an arbitrary memory region where the iflags offset has the INCREMENT_PC bit set?
This would cause the hypervisor to execute:
arch/arm64/kvm/hyp/exception.c:__kvm_adjust_pc() {
...
} else if (vcpu_get_flag(vcpu, INCREMENT_PC)) {
kvm_skip_instr(vcpu);
...
}Could this allow an attacker to increment arbitrary EL2 memory by adding 4 to the value at the regs.pc offset?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Fuad Tabba
reviewerRe: [PATCH v7 2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
In reply to [email protected]Quoted 5 lines · On Wed, 29 Jul 2026 at 14:34, <[email protected]> wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Pre-existing bug:enter_exception64reads staleSCTLR_EL1during early trap handling in nVHE/pKVM.
Already fixed:
https://lore.kernel.org/all/[email protected]/.
/fuad
Quoted 74 lines
--
--- Patch [2]: [PATCH v7 2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code --- commit
e62bbea08d3143161297d701776a47e619a76db1
Author: Fuad Tabba <[email protected]>KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
This commit replaces the locally open-coded
__vcpu_read_sys_regand__vcpu_write_sys_regwrappers inexception.cwith the canonical vcpu_{read,write}_sys_reg accessors. For nVHE builds, it maps these accessors directly to the raw in-memory context accessors, allowing HYP code to reuse helpers without function duplication.Quoted 4 lines
arch/arm64/kvm/hyp/exception.c +0
-0[ ... ]
arch/arm64/kvm/hyp/exception.c static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode, · line 89
Quoted 9 lines
arch/arm64/kvm/hyp/exception.c +2
-3@@ -101,14 +85,14 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,switch (target_mode) {case PSR_MODE_EL1h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL1 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL1 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 );[Severity: High]
This is a pre-existing issue, but doesenter_exception64()read a staleSCTLR_EL1value from the in-memory context here?When a protected VM triggers an early EL2 trap, the hypervisor handles it via the
kvm_handle_pvm_sysreg()->inject_undef64()->inject_sync64()call chain. At that point, the guest's system registers aren't fully saved to the in-memoryvcpu->arch.ctxt.While
inject_sync64()explicitly synchronizesVBAR_EL1from hardware to memory:arch/arm64/kvm/hyp/nvhe/sys_regs.c:inject_sync64() { ... __vcpu_assign_sys_reg(vcpu, VBAR_EL1, read_sysreg_el1(SYS_VBAR)); ... }It doesn't appear to synchronize
SCTLR_EL1before triggering exception injection.Since nVHE builds map
vcpu_read_sys_reg()directly to the memory context,enter_exception64()seems to use a staleSCTLR_EL1value to calculate thePSR_PAN_BITandPSR_SSBS_BITfor the exception's PSTATE.This corrupted PSTATE is then written directly to the hardware
SYS_SPSR_EL2byinject_sync64(), potentially poisoning the guest's state upon ERET.Should
SCTLR_EL1also be synchronized from hardware before injection?arch/arm64/kvm/hyp/exception.c static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode, · line 97
Quoted 11 lines
arch/arm64/kvm/hyp/exception.c +4
-3+vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 );break ;case PSR_MODE_EL2h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL2 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL2 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 );break ;default :--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Fuad Tabba
reviewerRe: [PATCH v7 6/8] KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
In reply to [email protected]Quoted 6 lines · On Wed, 29 Jul 2026 at 14:39, <[email protected]> wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [Critical] The patch fixes an arbitrary host pointer dereference in GICv3 hypercalls but completely misses the identical vulnerabilities in GICv5 hypercalls (handle___vgic_v5_save_aprandhandle___vgic_v5_restore_vmcr_apr).
- [Critical] Unvalidated host pointer dereference inhandle___kvm_adjust_pcallows arbitrary memory modification in EL2.
https://lore.kernel.org/all/CA+EHjTyGULmVCgyoya3bXG4gRj0OYFE1gnJLhNE6kvCrZFtXyQ@mail.gmail.com/
/fuad
Quoted 77 lines
--
--- Patch [6]: [PATCH v7 6/8] KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch --- commit
def324be61872b6056efce4e3eb9ff57eb9a9e10
Author: Marc Zyngier <[email protected]>
Subject: KVM: arm64: Minimise EL2's exposure of host VGIC state during world switchThis commit restricts EL2's exposure to host VGIC state by recovering the host vCPU securely rather than directly dereferencing the host-provided
vgic_v3_cpu_ifpointer in__vgic_v3_save_aprsand__vgic_v3_restore_vmcr_aprs. It introduces a helper to fetch the host vCPU usingcontainer_of()and copies only the necessary state.Quoted 4 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +0
-0[ ... ]
arch/arm64/kvm/hyp/nvhe/hyp-main.c static void handle___vgic_v3_save_aprs(struct kvm_cpu_context *host_ctxt) · line 525
Quoted 9 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +4
-2@@ -508,16 +520,63 @@ static void handle___vgic_v3_save_aprs(struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(struct vgic_v3_cpu_if *, cpu_if, host_ctxt, 1); +struct pkvm_hyp_vcpu *hyp_vcpu; +struct kvm_vcpu *host_vcpu; -__vgic_v3_save_aprs (kern_hyp_va (cpu_if)); + host_vcpu =get_host_hyp_vcpus_from_vgic_v3_cpu_if (host_ctxt,1 , + &hyp_vcpu);[ ... ]
[Severity: Critical]
This is a pre-existing issue, but does the same regression exist for the GICv5 hypercalls?The patch replaces the direct
kern_hyp_va()dereference with a secure lookup for GICv3. However, it appears the same pattern remains inhandle___vgic_v5_save_apr()andhandle___vgic_v5_restore_vmcr_apr():
handle___vgic_v5_save_apr()__vgic_v5_save_apr(kern_hyp_va(cpu_if));Can a compromised host pass an arbitrary physical address here, causing
__vgic_v5_save_apr()to write the hardware value ofSYS_ICH_APR_EL2to that chosen address, breaking EL2 isolation?[Severity: Critical]
This is also a pre-existing issue, but doeshandle___kvm_adjust_pc()have a similar unvalidated host pointer dereference?In
arch/arm64/kvm/hyp/nvhe/hyp-main.c:static void
handle___kvm_adjust_pc(struct kvm_cpu_context *host_ctxt){ DECLARE_REG(struct kvm_vcpu *, vcpu, host_ctxt, 1); __kvm_adjust_pc(kern_hyp_va(vcpu)); }If the host kernel passes a crafted vcpu pointer, could it point to an arbitrary memory region where the iflags offset has the
INCREMENT_PCbit set?This would cause the hypervisor to execute:
arch/arm64/kvm/hyp/exception.c:__kvm_adjust_pc() { ... } else if (vcpu_get_flag(vcpu, INCREMENT_PC)) { kvm_skip_instr(vcpu); ... }Could this allow an attacker to increment arbitrary EL2 memory by adding 4 to the value at the regs.pc offset?
--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Fuad Tabba
reviewerRe: [PATCH v7 8/8] KVM: arm64: Implement lazy vCPU state sync for non-protected guests
In reply to [email protected]Quoted 4 lines · On Wed, 29 Jul 2026 at 14:36, <[email protected]> wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] MissingHCR_EL2state synchronization breaks virtual SError injection, causing an infinite exception loop.
- [High] Lazy state sync fails to flush context afterKVM_REQ_VCPU_RESET, causing vCPU resets (e.g., PSCICPU_ON) to be ignored.
I think this covers it:
https://lore.kernel.org/all/CA+EHjTxiyUSJqRcadSCOmR+tELSsBiGaTYH9xxXh-g-xyuHZdA@mail.gmail.com/
/fuad
Quoted 97 lines
--
--- Patch [8]: [PATCH v7 8/8] KVM: arm64: Implement lazy vCPU state sync for non-protected guests --- commit
80df5c4ca2cca0a1726813fa129b353353482892
Author: Fuad Tabba <[email protected]>KVM: arm64: Implement lazy vCPU state sync for non-protected guests
This commit optimizes world switches for non-protected guests by deferring the copy of the vCPU register context between the host and hypervisor. State is now only synced on entry if marked dirty, and on exit only when explicitly needed by the host.
Quoted 15 lines
arch/arm64/kvm/handle_exit.c +8
-0@@ -486,9 +486,32 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) } } +static void handle_exit_pkvm_state (struct kvm_vcpu *vcpu,int exception_index) +{ +int exception_code =ARM_EXCEPTION_CODE (exception_index); + +if (!is_protected_kvm_enabled () ||kvm_vm_is_protected (vcpu->kvm)) +return ; + +/* arch/arm64/kvm/handle_exit.c int handle_exit(struct kvm_vcpu *vcpu, int exception_index) · line 506
Quoted 10 lines
arch/arm64/kvm/handle_exit.c +10
-0+* Sync the context back when the host will read (trap) or write +* (SError) it. Preempt-off here, so the loaded hyp vCPU is stable. +*/ +if (exception_code ==ARM_EXCEPTION_TRAP || + exception_code ==ARM_EXCEPTION_EL1_SERROR || +ARM_SERROR_PENDING (exception_index)) { +kvm_call_hyp_nvhe (__pkvm_vcpu_sync_state); +vcpu_set_flag (vcpu,PKVM_HOST_STATE_DIRTY ); + }else { +vcpu_clear_flag (vcpu,PKVM_HOST_STATE_DIRTY );[Severity: High]
Is it possible forKVM_REQ_VCPU_RESETupdates to be ignored due to this cleared flag?If a secondary vCPU is powered on (for example, via PSCI
CPU_ON), it receivesKVM_REQ_VCPU_RESETand gets kicked, leading to an exit with an IRQ. Since that exit takes this else branch and clears the dirty flag, and the reset request is processed afterward inkvm_vcpu_check_requests()which modifies the host-side vCPU context, wouldn'tflush_hyp_vcpu()skip syncing this updated reset state because the flag remains cleared?This could result in the target vCPU resuming with its pre-IRQ stale context, eventually overwriting the new reset state on the next exit.
Quoted 3 lines
arch/arm64/kvm/handle_exit.c +3
-0+ } +} +[ ... ]
arch/arm64/kvm/hyp/nvhe/hyp-main.c static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 208
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +0
-3@@ -203,9 +255,16 @@ static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)fpsimd_sve_sync (&hyp_vcpu->vcpu);sync_debug_state (hyp_vcpu); - host_vcpu->arch.ctxt = hyp_vcpu->vcpu.arch.ctxt; - - host_vcpu->arch.hcr_el2 = hyp_vcpu->vcpu.arch.hcr_el2;[Severity: High]
Does removing thisHCR_EL2assignment break virtual SError injection and cause an infinite exception loop?When the host injects a virtual SError into a non-protected guest under pKVM by setting
HCR_VSEin the host-sidevcpu->arch.hcr_el2, the hardware automatically clears the VSE bit in the physicalHCR_EL2register once the guest takes the exception.Without syncing this hardware-cleared state back to the host here, the host's
HCR_VSEbit remains set. On the next entry,flush_hyp_vcpu()will repeatedly re-inject the staleHCR_VSEbit from the host into the hyp vCPU, locking up the guest.arch/arm64/kvm/hyp/nvhe/hyp-main.c static void sync_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 267
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +10
-0+if (pkvm_hyp_vcpu_is_protected (hyp_vcpu)) { + host_vcpu->arch.ctxt = hyp_vcpu->vcpu.arch.ctxt; + }else { +/* +* PC feeds trace_kvm_exit(), PSTATE.SS the host software-step +* machine, and both run before the next on-demand ctxt sync. +*/ + host_vcpu->arch.ctxt.regs.pc = hyp_vcpu->vcpu.arch.ctxt.regs.pc; + host_vcpu->arch.ctxt.regs.pstate = hyp_vcpu->vcpu.arch.ctxt.regs.pstate; + }--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Joey Gouly
reviewerRe: [PATCH v7 2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
In reply to Fuad TabbaQuoted 22 lines · Fuad Tabba
The vcpu_{read,write}_sys_reg() accessors abstract where a system register lives, but their VHE implementation cannot be linked into the nVHE hypervisor, so
exception.copen-codeshas_vhe()wrappers instead.Redirect the accessors to the raw context accessors for nVHE hyp builds, where registers are always in memory, and drop the local wrappers. This lets hyp code reuse helpers built on the canonical accessors.
No functional change intended.
Suggested-by Marc Zyngier <[email protected]>Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 6 +++++ arch/arm64/kvm/hyp/exception.c | 34 ++++++++-------------------- 2 files changed, 15 insertions(+), 25 deletions(-)arch/arm64/include/asm/kvm_emulate.h +0
-0
arch/arm64/include/asm/kvm_emulate.h static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu) · line 514
Quoted 9 lines
arch/arm64/include/asm/kvm_emulate.h +6
-0@@ -506,6 +506,12 @@ static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu)return __vcpu_sys_reg (vcpu,MPIDR_EL1 ) &MPIDR_HWID_BITMASK ; } +/* In nVHE hyp code, registers are always in memory: use the raw accessors. */ +#if defined(__KVM_NVHE_HYPERVISOR__) +#define vcpu_read_sys_reg (v, r) __vcpu_sys_reg(v, r) +#define vcpu_write_sys_reg (v, x, r) __vcpu_assign_sys_reg(v, r, x) +#endif +
Does this make sense in arch/arm64/include/asm/kvm_host.h instead? Next to the definitions of __vcpu_sys_reg() etc.
Thanks,
Joey
Quoted 81 lines
arch/arm64/include/asm/kvm_emulate.h +0
-0static inline void kvm_vcpu_set_be (struct kvm_vcpu *vcpu) {if (vcpu_mode_is_32bit (vcpu)) {arch/arm64/kvm/hyp/exception.c +9
-25@@ -20,22 +20,6 @@#error Hypervisor code only!#endif -static inline u64 __vcpu_read_sys_reg (const struct kvm_vcpu *vcpu,int reg) -{ -if (has_vhe ()) -return vcpu_read_sys_reg (vcpu, reg); - -return __vcpu_sys_reg (vcpu, reg); -} - -static inline void __vcpu_write_sys_reg (struct kvm_vcpu *vcpu,u64 val,int reg) -{ -if (has_vhe ()) -vcpu_write_sys_reg (vcpu, val, reg); -else -__vcpu_assign_sys_reg (vcpu, reg, val); -} -static void __vcpu_write_spsr (struct kvm_vcpu *vcpu,unsigned long target_mode,u64 val) {@@ -101,14 +85,14 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,switch (target_mode) {case PSR_MODE_EL1h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL1 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL1 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL1 );break ;case PSR_MODE_EL2h: - vbar =__vcpu_read_sys_reg (vcpu,VBAR_EL2 ); - sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); -__vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 ); + vbar =vcpu_read_sys_reg (vcpu,VBAR_EL2 ); + sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL2 ); +vcpu_write_sys_reg (vcpu, *vcpu_pc (vcpu),ELR_EL2 );break ;default :/* Don't do that */ @@ -185,7 +169,7 @@ static void enter_exception64(struct kvm_vcpu *vcpu, unsigned long target_mode,*/ static unsigned long get_except32_cpsr (struct kvm_vcpu *vcpu,u32 mode) { -u32 sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); +u32 sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 );unsigned long old, new; old = *vcpu_cpsr (vcpu);@@ -281,7 +265,7 @@ static void enter_exception32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset) {unsigned long spsr = *vcpu_cpsr (vcpu);bool is_thumb = (spsr &PSR_AA32_T_BIT ); -u32 sctlr =__vcpu_read_sys_reg (vcpu,SCTLR_EL1 ); +u32 sctlr =vcpu_read_sys_reg (vcpu,SCTLR_EL1 );u32 return_address; *vcpu_cpsr (vcpu) =get_except32_cpsr (vcpu, mode);@@ -305,7 +289,7 @@ static void enter_exception32(struct kvm_vcpu *vcpu, u32 mode, u32 vect_offset)if (sctlr & (1 <<13 )) vect_offset +=0xffff0000 ;else /* always have security exceptions */ - vect_offset +=__vcpu_read_sys_reg (vcpu,VBAR_EL1 ); + vect_offset +=vcpu_read_sys_reg (vcpu,VBAR_EL1 ); *vcpu_pc (vcpu) = vect_offset; }Signature · 1 line
2.39.5
Fuad Tabba
reviewerRe: [PATCH v7 2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
Hi Joey,
On Wed, 29 Jul 2026 at 14:52, Joey Gouly <[email protected]> wrote:
...
Quoted 9 lines
arch/arm64/include/asm/kvm_emulate.h static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu) · line 514
Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +6
-0+/* In nVHE hyp code, registers are always in memory: use the raw accessors. */ +#if defined(__KVM_NVHE_HYPERVISOR__) +#define vcpu_read_sys_reg (v, r) __vcpu_sys_reg(v, r) +#define vcpu_write_sys_reg (v, x, r) __vcpu_assign_sys_reg(v, r, x) +#endif +Does this make sense in
arch/arm64/include/asm/kvm_host.hinstead? Next to the definitions of__vcpu_sys_reg()etc.
It could go in kvm_host.h, but not next to __vcpu_sys_reg() as
suggested: the redirect has to come after the vcpu_{read,write}_sys_reg() prototypes it overrides, or the nVHE preprocessor expands the prototypes themselves. A placement that builds has to sit below those prototypes, not next to the raw accessors, so it wouldn't group with them the way you're suggesting anyway.
It's currently in kvm_emulate.h, after those prototypes, which is where Marc suggested putting it when he moved the patch off the earlier rename [1]. I'd prefer to keep it there.
Cheers,
/fuad
Quoted 11 lines · Fuad Tabba
Changes since v6 [1]:
- Rebased onto v7.2-rc5.
- Expanded the comment and commit message in patch 8 explaining why PC and PSTATE are copied back on every exit. No functional change.
- Collected Steffen's Reviewed-by on patch 3 (thanks!).Building on Will's pKVM infrastructure series [2], this series reworks how pKVM moves vCPU state between the host and EL2, and stops copying a non-protected guest's state on every world switch.
[...]
Applied to next, thanks!
[1/8] KVM: arm64: Extract MPIDR computation into a shared header
https://git.kernel.org/kvmarm/kvmarm/c/c0646b964733 [2/8] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code https://git.kernel.org/kvmarm/kvmarm/c/f88099ad9fc2 [3/8] KVM: arm64: Factor out reusable vCPU reset helpers https://git.kernel.org/kvmarm/kvmarm/c/29b2e2ab7fa7 [4/8] KVM: arm64: Move PSCI helper functions to a shared header https://git.kernel.org/kvmarm/kvmarm/c/86db2bd7f5f0 [5/8] KVM: arm64: Add host and hypervisor vCPU lookup primitives https://git.kernel.org/kvmarm/kvmarm/c/e9f88aa0b16c [6/8] KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch https://git.kernel.org/kvmarm/kvmarm/c/79c3122a1e14 [7/8] KVM: arm64: Add primitives to flush/sync the VGIC state at EL2 https://git.kernel.org/kvmarm/kvmarm/c/42fe26543e66 [8/8] KVM: arm64: Implement lazy vCPU state sync for non-protected guests https://git.kernel.org/kvmarm/kvmarm/c/734dc8c01c83
--
Best,
Oliver