KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3
Marc Zyngier
maintainerv3 cover KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3
This is the thid version of a series adding support for two extensions targeting Nested Virt on arm64:
- FEAT_NV2p1 is effectively a bug fix for two registers (CNTHCTL_EL2 and CPTR_EL2) that are missing stateful bits when accessed from EL1 in a NV configuration. When this is present, the hypervisor can avoid a bunch of traps.
- FEAT_NV3 is much more ambitious, and changes the way ERET behaves in a NV environment. By moving EL1 accesses to HCR_EL2 from memory (via VNCR) to a dedicated register (NVHCR_EL2), the HW can detect whether the guest is performing an ERET for itself (NVHCR_EL2.TGE==1) or to its own guest (NVHCR_EL2.TGE==0). In the former case, ERET is done directly, and no trap occurs. Similar optimisations are available for a class of TLBI instructions.
The whole thing has been tested on an FVP model, and shown measurable improvements for an L1 guest (about 1.5% fewer instructions).
Given that this isn't very convincing on its own, I have built an approximate emulation of FEAT_NV3 that L1 (and deeper levels) can use on actual production hardware. For these deeper levels, the numbers are in the double digit of percentage point reduction, see below for details.
Does it make NV better? Yes!
Does it make NV good? Get real!
Anyway, patches on top of -rc4.
* Side notes about performance evaluation with NV3 emulation:
If you are interested in the NV3 emulation, I have a branch [0] that contains it all. Note that the improvement only kicks while running an L2 guest running at EL2 -- L1 will not see any improvement. But I get close to 40% improvements on L2, and 60% on L3 for the only workload that really matters (Debian Installer).
If all your nested guests are running this hack, then you can pass "kvm-arm.mode=nested,nvtge" on the host command line, and all the nested guests will opt in the accelerated behaviour.
If you like to live dangerously (or have a machine that does not have FEAT_HCX), you can instead pass 'kvm-arm.mode=nested,nvtge=force', and ERET will get the acceleration without even guest buy-in.
* From v2 [2]:
- Improved comments on the subtleties of trapping ERET in nested contexts
- Rebased on top of v7.2-rc4
- Collected RBs from Yao Yuan, with thanks
* From v1 [1]:
- Expanded generation of RESx and UNKN symbols to both Mapping and Fields qualifiers
- Dropped HFGIRT_EL2.ERET checks on ERET fast-path
- Fixed the NVHCR_EL2 trap handling
- Added FORCE_RESx() to HCRX_EL2 constraints
- Fixed ID_AA64MMFR4_EL1 sanitisation
- Added NV3 acceleration for TLBIOS
- Collected RBs from Joey, with thanks
[0] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv3
[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/[email protected]
Marc Zyngier (28):
arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release
KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
KVM: arm64: Drop __HCRX_EL2_* masks
KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation
KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register
KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast
path
arm64: Add ARM64_HAS_NV2P1 capability
KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
KVM: arm64: Expose FEAT_NV2p1 to NV guests
arm64: Add FEAT_NV2p1 detection
arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
arm64: Add ARM64_HAS_NV3 capability
KVM: arm64: Split NV-specific exit fixups from the non-NV handling
KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
KVM: arm64: Add kvm_has_nv{2,3}() predicates
KVM: arm64: Make HCR_EL2 a non-VNCR register
KVM: arm64: Add sanitisation for NVHCR_EL2
KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
KVM: arm64: Add routing for NVHCR_EL2 trap
KVM: arm64: Add NVHCR_EL2 context switching
KVM: arm64: Engage NV3 ERET trap elision
KVM: arm64: Engage NV3 TLBI trap elision
KVM: arm64: Add FEAT_NV3 detection
KVM: arm64: Expose FEAT_NV3 to guests
arm64: Add override for ID_AA64MMFR4_EL1.NV_frac
arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/include/asm/kvm_arm.h | 15 ------ arch/arm64/include/asm/kvm_emulate.h | 45 ++++++++++++++++- arch/arm64/include/asm/kvm_host.h | 3 +- arch/arm64/include/asm/vncr_mapping.h | 2 +- arch/arm64/kernel/cpufeature.c | 18 ++++++- arch/arm64/kernel/image-vars.h | 1 + arch/arm64/kernel/pi/idreg-override.c | 10 ++++ arch/arm64/kvm/arch_timer.c | 10 +++- arch/arm64/kvm/config.c | 27 +++++++++- arch/arm64/kvm/emulate-nested.c | 18 +++++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 27 ++++++++-- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++ arch/arm64/kvm/hyp/vhe/switch.c | 52 +++++++++++++++----- arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 21 +++++--- arch/arm64/kvm/nested.c | 14 +++++- arch/arm64/kvm/sys_regs.c | 57 ++++++++++++++++++++-- arch/arm64/tools/cpucaps | 2 + arch/arm64/tools/gen-sysreg.awk | 8 +-- arch/arm64/tools/sysreg | 44 +++++++++++++++-- 20 files changed, 321 insertions(+), 65 deletions(-)
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 1/28 arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
The sysreg file is using the Mapping or Fields qualifiers to indicate that a given encoding is only a mapping to a particular register (or an instance of a more generic register definition).
As a result, we don't output any definition, and instead expect the canonical definitions to be used.
This works rather well for individual fields, but creates problems for macros that refer to more generic classes of bits such as RESx.
Relax the above rule by emitting the RESx and UNKN values for Mapping and Fields qualifiers as well.
--- arch/arm64/tools/gen-sysreg.awk | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/tools/gen-sysreg.awk +4 -4
@@ -228,7 +228,7 @@ $1 == "EndSysreg" && block_current() == "Sysreg" { } # Currently this is effectivey a comment, in future we may want to emit -# defines for the fields. +# defines for the fields. "Mapping" does emit the RESx/UNKN definitions. ($1 == "Fields" || $1 == "Mapping") && block_current() == "Sysreg" { expect_fields(2)
@@ -239,9 +239,9 @@ $1 == "EndSysreg" && block_current() == "Sysreg" { print "" next_bit = -1 - res0 = null - res1 = null - unkn = null + res0 = $2 "_RES0" + res1 = $2 "_RES1" + unkn = $2 "_UNKN" next }
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The updated comment explicitly claims '"Mapping" does emit the RESx/UNKN definitions', which…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The updated comment explicitly claims '"Mapping" does emit the RESx/UNKN definitions', which misleadingly omits the fact that the 'Fields' qualifier also emits these definitions. --
--- Patch [1]: [PATCH v3 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions --- commit ed28be9bab17b360a9e12769aa91d7ac6b649b8d
Author: Marc Zyngier <[email protected]>
arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
This commit relaxes the rule that skips emitting definitions for Mapping and Fields qualifiers. It updates gen-sysreg.awk to emit RESx and UNKN values for both of these qualifiers.
arch/arm64/tools/gen-sysreg.awk $1 == "EndSysreg" && block_current() == "Sysreg" { · line 231
Quoted 10 lines
arch/arm64/tools/gen-sysreg.awk +1
-1@@ -228,7 +228,7 @@ $1 == "EndSysreg" && block_current() == "Sysreg" { } # Currently this is effectivey a comment, in future we may want to emit -# defines for the fields. +# defines for the fields. "Mapping" does emit the RESx/UNKN definitions.
[Severity: Low]
This isn't a bug, but does this comment incorrectly imply that only "Mapping" emits the definitions?
The condition on the next line applies identically to both "Fields" and "Mapping", and the commit message notes that both qualifiers are being updated to emit the RESx/UNKN values.
Quoted 2 lines
arch/arm64/tools/gen-sysreg.awk +0
-0($1 == "Fields" || $1 == "Mapping") && block_current() == "Sysreg" { expect_fields(2)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Marc Zyngier
maintainerRe: [PATCH v3 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
Quoted 30 lines · On Wed, 22 Jul 2026 08:55:53 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The updated comment explicitly claims '"Mapping" does emit the RESx/UNKN definitions', which misleadingly omits the fact that the 'Fields' qualifier also emits these definitions. ----- Patch [1]: [PATCH v3 01/28] arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions --- commit
ed28be9bab17b360a9e12769aa91d7ac6b649b8d
Author: Marc Zyngier <[email protected]>arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
This commit relaxes the rule that skips emitting definitions for Mapping and Fields qualifiers. It updates gen-sysreg.awk to emit RESx and UNKN values for both of these qualifiers.
arch/arm64/tools/gen-sysreg.awk $1 == "EndSysreg" && block_current() == "Sysreg" { · line 231
Quoted 10 lines
arch/arm64/tools/gen-sysreg.awk +1
-1@@ -228,7 +228,7 @@ $1 == "EndSysreg" && block_current() == "Sysreg" { } # Currently this is effectivey a comment, in future we may want to emit -# defines for the fields. +# defines for the fields. "Mapping" does emit the RESx/UNKN definitions.[Severity: Low]
This isn't a bug, but does this comment incorrectly imply that only "Mapping" emits the definitions?
Quoted 3 lines
The condition on the next line applies identically to both "Fields" and "Mapping", and the commit message notes that both qualifiers are being updated to emit the RESx/UNKN values.
Fixed.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH v3 2/28 arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release
In reply to Marc ZyngierID_AA64MMFR4_EL1 has gained a few fields and enum values in the past few months, so resync its definition with the 2026-03 JSON release.
--- arch/arm64/tools/sysreg | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-)
arch/arm64/tools/sysreg +28 -4
@@ -2386,17 +2386,40 @@ EndEnum EndSysreg Sysreg ID_AA64MMFR4_EL1 3 0 0 7 4 -Res0 63:48 -UnsignedEnum 47:44 SRMASK +UnsignedEnum 63:60 MTEFGT + 0b0000 NI + 0b0001 IMP +EndEnum +UnsignedEnum 59:56 SCRX + 0b0000 NI + 0b0001 IMP +EndEnum +UnsignedEnum 55:52 TEV + 0b0000 NI + 0b0001 IMP +EndEnum +UnsignedEnum 51:48 TPS + 0b0000 VAL_0000 + 0b0001 VAL_0001 + 0b0010 VAL_0010 +EndEnum +UnsignedEnum 47:44 SRMASK + 0b0000 NI + 0b0001 IMP + 0b0010 SRMASK2 +EndEnum +UnsignedEnum 43:40 TLBID 0b0000 NI 0b0001 IMP EndEnum -Res0 43:40 UnsignedEnum 39:36 E3DSE 0b0000 NI 0b0001 IMP EndEnum -Res0 35:32 +UnsignedEnum 35:32 EAESR + 0b0000 NI + 0b0001 IMP +EndEnum UnsignedEnum 31:28 RMEGDI 0b0000 NI 0b0001 IMP
@@ -2410,6 +2433,7 @@ UnsignedEnum 23:20 NV_frac 0b0000 NV_NV2 0b0001 NV2_ONLY 0b0010 NV2P1 + 0b0011 NV3 EndEnum UnsignedEnum 19:16 FGWTE3 0b0000 NI
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 3/28 KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
In reply to Marc ZyngierThe way we merge the guest-provided HCRX_EL2 value with the host's is bonkers. We try to make it look like the FGT registers by using positive and negative polarities for traps, but most of these bits are not strictly about trapping, as they actively change the way some architectural state is managed.
It would be far better to deal with these bits like we do for HCR_EL2, by enumerating the list of bits we don't allow the guest to override. This is simplified by the fact that HCRX_EL2 only affects EL1, and not EL2.
Re-jig the HCRX_EL2 handling with a macro that list the bits excluded from the merge (TMEA, PTTWI, EnIDCP128).
--- arch/arm64/kvm/hyp/include/hyp/switch.h | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +20 -2
@@ -325,6 +325,24 @@ static inline void __deactivate_traps_mpam(void)write_sysreg_s (MPAMHCR_HOST_FLAGS ,SYS_MPAMHCR_EL2 ); } +/* +* Just like for HCR_EL2, we can't let the guest mess with some of the +* basics we rely on in HCRX_EL2. However, the major difference is that +* HCRX_EL2 only affects EL1, and never EL2 (sudden outburst of sanity, I +* guess). So it is always the guest inflicting it on its own guestx. +* +* Things we don't want to let the guest control are: +* +* - TMEA: That's for us to decide how an SEA is routed, not the guest. +* +* - PTTWI: Similarly, it is for us to decide whether Reduced Coherency for +* the PTW is a thing. It really isn't. +* +* - EnIDCP128: We don't allow IMPDEF sysregs -- full stop. +*/ +#define NV_HCRX_GUEST_EXCLUDE (HCRX_EL2_TMEA | HCRX_EL2_PTTWI | \ + HCRX_EL2_EnIDCP128) +static inline void __activate_traps_common (struct kvm_vcpu *vcpu) {struct kvm_cpu_context *hctxt =host_data_ptr (host_ctxt);
@@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)u64 hcrx = vcpu->arch.hcrx_el2;if (is_nested_ctxt (vcpu)) {u64 val =__vcpu_sys_reg (vcpu,HCRX_EL2 ); - hcrx |= val & __HCRX_EL2_MASK; - hcrx &= ~(~val & __HCRX_EL2_nMASK); + hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE ); + hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE ); }ctxt_sys_reg (hctxt,HCRX_EL2 ) =read_sysreg_s (SYS_HCRX_EL2 );
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 4/28 KVM: arm64: Drop __HCRX_EL2_* masks
In reply to Marc ZyngierThe __HCRX_EL2_* masks are a leftover from a time where we didn't have much sanitisation for the system registers. Since we are now in a better place, rely on the existing checks to detect unhandled bits in HCRX_EL2.
--- arch/arm64/include/asm/kvm_arm.h | 15 --------------- arch/arm64/kvm/config.c | 5 +++-- arch/arm64/kvm/emulate-nested.c | 5 ----- 3 files changed, 3 insertions(+), 22 deletions(-)
arch/arm64/include/asm/kvm_arm.h +0 -15
@@ -287,21 +287,6 @@ GENMASK(19, 18) | \ GENMASK(15, 0)) -/* -* Polarity masks for HCRX_EL2, limited to the bits that we know about -* at this point in time. It doesn't mean that we actually *handle* -* them, but that at least those that are not advertised to a guest -* will be RES0 for that guest. -*/ -#define __HCRX_EL2_MASK (BIT_ULL(6)) -#define __HCRX_EL2_nMASK (GENMASK_ULL(24, 14) | \ - GENMASK_ULL(11, 7) | \ - GENMASK_ULL(5, 0)) -#define __HCRX_EL2_RES0 ~(__HCRX_EL2_nMASK | __HCRX_EL2_MASK) -#define __HCRX_EL2_RES1 ~(__HCRX_EL2_nMASK | \ - __HCRX_EL2_MASK | \ - __HCRX_EL2_RES0) -/* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */ #define HPFAR_MASK (~UL(0xf))/*
arch/arm64/kvm/config.c +3 -2
@@ -930,10 +930,12 @@ static const struct reg_bits_to_feat_map hcrx_feat_map[] = {NEEDS_FEAT (HCRX_EL2_EnASR,FEAT_LS64_V ),NEEDS_FEAT (HCRX_EL2_EnALS,FEAT_LS64 ),NEEDS_FEAT (HCRX_EL2_EnAS0,FEAT_LS64_ACCDATA ), +FORCE_RES0 (HCRX_EL2_RES0 ), + FORCE_RES1(HCRX_EL2_RES1), }; -static const DECLARE_FEAT_MAP(hcrx_desc, __HCRX_EL2, +static const DECLARE_FEAT_MAP(hcrx_desc, HCRX_EL2, hcrx_feat_map, FEAT_HCX); static const struct reg_bits_to_feat_map hcr_feat_map[] = {
@@ -1579,7 +1581,6 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)break ;case HCRX_EL2 : resx =compute_reg_resx_bits (kvm, &hcrx_desc,0 ,0 ); - resx.res1 |= __HCRX_EL2_RES1;break ;case HCR_EL2 : resx =compute_reg_resx_bits (kvm, &hcr_desc,0 ,0 );
arch/arm64/kvm/emulate-nested.c +0 -5
@@ -2320,7 +2320,6 @@ int __init populate_nv_trap_config(void)BUILD_BUG_ON (__NR_CGT_GROUP_IDS__ >BIT (TC_CGT_BITS ));BUILD_BUG_ON (__NR_FGT_GROUP_IDS__ >BIT (TC_FGT_BITS ));BUILD_BUG_ON (__NR_FG_FILTER_IDS__ >BIT (TC_FGF_BITS )); -BUILD_BUG_ON (__HCRX_EL2_MASK & __HCRX_EL2_nMASK);for (int i =0 ; i <ARRAY_SIZE (encoding_to_cgt); i++) {const struct encoding_to_trap_config *cgt = &encoding_to_cgt[i];
@@ -2346,10 +2345,6 @@ int __init populate_nv_trap_config(void) } } -if (__HCRX_EL2_RES0 !=HCRX_EL2_RES0 ) -kvm_info ("Sanitised HCR_EL2_RES0 = %016llx, expecting %016llx\n" , - __HCRX_EL2_RES0,HCRX_EL2_RES0 ); -kvm_info ("nv: %ld coarse grained trap handlers\n" ,ARRAY_SIZE (encoding_to_cgt));
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 5/28 KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation
In reply to Marc ZyngierHCRX_EL2.SRMASKEn is a new bit enabling FEAT_SRMASK for a guest. We don't plan to support it any time soon, but it doesn't hurt to actively document it, specially as we are going to add more bits we actually care about.
--- arch/arm64/kvm/config.c | 1 + arch/arm64/tools/sysreg | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
arch/arm64/kvm/config.c +1 -0
@@ -904,6 +904,7 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,static const struct reg_bits_to_feat_map hcrx_feat_map[] = { +NEEDS_FEAT (HCRX_EL2_SRMASKEn,FEAT_SRMASK ),NEEDS_FEAT (HCRX_EL2_PACMEn, feat_pauth_lr),NEEDS_FEAT (HCRX_EL2_EnFPM,FEAT_FPMR ),NEEDS_FEAT (HCRX_EL2_GCSEn,FEAT_GCS ),
arch/arm64/tools/sysreg +3 -1
@@ -4545,7 +4545,9 @@ Fields ZCR_ELx EndSysreg Sysreg HCRX_EL2 3 4 1 2 2 -Res0 63:25 +Res0 63:27 +Field 26 SRMASKEn +Res0 25 Field 24 PACMEn Field 23 EnFPM Field 22 GCSEn
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 6/28 KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register
In reply to Marc ZyngierIt may not be obvious unless you look at it closely, but CPTR_EL2 is treated very differently from other registers. It is one the registers that, despite looking very similar between EL1 and EL2 when E2H==1, have RES0 bits that get in the way.
Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL, just like CNTHCTL_EL2 (and for the same reasons). This makes it possible to use vcpu_read_sys_reg() with it, and will be necessary once we support FEAT_NV2P1.
--- arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/kvm/sys_regs.c | 20 ++++++++++++++++++-- 2 files changed, 19 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +1 -1
@@ -617,7 +617,7 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)*/ static inline u64 vcpu_sanitised_cptr_el2 (const struct kvm_vcpu *vcpu) { -u64 cptr =__vcpu_sys_reg (vcpu,CPTR_EL2 ); +u64 cptr =vcpu_read_sys_reg (vcpu,CPTR_EL2 );if (!vcpu_el2_e2h_is_set (vcpu)) cptr =translate_cptr_el2_to_cpacr_el1 (cptr);
arch/arm64/kvm/sys_regs.c +18 -2
@@ -183,8 +183,6 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,switch (reg) {MAPPED_EL2_SYSREG (SCTLR_EL2 ,SCTLR_EL1 , translate_sctlr_el2_to_sctlr_el1 ); -MAPPED_EL2_SYSREG (CPTR_EL2 ,CPACR_EL1 , - translate_cptr_el2_to_cpacr_el1 );MAPPED_EL2_SYSREG (TTBR0_EL2 ,TTBR0_EL1 , translate_ttbr0_el2_to_ttbr0_el1 );MAPPED_EL2_SYSREG (TTBR1_EL2 ,TTBR1_EL1 ,NULL );
@@ -210,6 +208,19 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg, loc->loc = ((is_hyp_ctxt (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) ?SR_LOC_SPECIAL :SR_LOC_MEMORY );break ; +case CPTR_EL2 : +/* +* CPTR_EL2 is just as special, and needs a certain amount +* of handholding. It always lives in memory, due to being +* heavily trapped thanks to CPACR_EL1.TCPAC being RES0. +* FEAT_NV2p1 fixes this. +*/ +locate_mapped_el2_register (vcpu,CPTR_EL2 ,CPACR_EL1 , + translate_cptr_el2_to_cpacr_el1, + loc); +if (is_hyp_ctxt (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) + loc->loc =SR_LOC_SPECIAL ; +break ;default : loc->loc =locate_direct_register (vcpu, reg); }
@@ -314,6 +325,8 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) val &=CNTKCTL_VALID_BITS ; val |=__vcpu_sys_reg (vcpu, reg) & ~CNTKCTL_VALID_BITS ;return val; +case CPTR_EL2 : +return __vcpu_sys_reg (vcpu, reg);default :WARN_ON_ONCE (1 ); }
@@ -359,6 +372,9 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, enum vcpu_sysreg reg)*/ write_sysreg_el1 (val,SYS_CNTKCTL );break ; +case CPTR_EL2 : +write_sysreg_el1 (val,SYS_CPACR ); +break ;default :WARN_ON_ONCE (1 ); }
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 7/28 KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path
In reply to Marc ZyngierWe currently avoid using the ERET fast path if the guest has HCR_EL2.NV set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.
Similarly, evaluating HFGITR_EL2.ERET makes little sense, as this only applies to the nested context, while the ERET fast-path is purely for the benefit of L1.
Replace these bizarre checks with is_nested_ctxt() which makes a lot more sense: if we are running an L2, the ERET trap must go to L1.
--- arch/arm64/kvm/hyp/vhe/switch.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +7 -5
@@ -344,13 +344,15 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)* if this is a VHE guest hypervisor returning to its own * userspace, or the hypervisor performing a local exception * return. No need to save/restore registers, no need to -* switch S2 MMU. Just do the canonical ERET. +* switch S2 MMU. Just do the canonical ERET unless we are in +* nested context. * -* Unless the trap has to be forwarded further down the line, -* of course... +* Note that this is made possible because KVM itself never traps +* ERET when running an L2. The consequence is that any ERET trap is +* the result of HCR_EL2 or HFGITR_EL2 programming by L1 for its own +* guest, and the exception must be forwarded to L1. */ -if ((__vcpu_sys_reg (vcpu,HCR_EL2 ) &HCR_NV ) || - (__vcpu_sys_reg (vcpu,HFGITR_EL2 ) &HFGITR_EL2_ERET )) +if (is_nested_ctxt (vcpu))return false; spsr =read_sysreg_el1 (SYS_SPSR );
Signature · 1 line
2.47.3
Re: [PATCH v3 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path
Quoted 24 lines · Marc Zyngier
We currently avoid using the ERET fast path if the guest has HCR_EL2.NV set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.
Similarly, evaluating HFGITR_EL2.ERET makes little sense, as this only applies to the nested context, while the ERET fast-path is purely for the benefit of L1.
Replace these bizarre checks with
is_nested_ctxt()which makes a lot more sense: if we are running an L2, the ERET trap must go to L1.Fixes dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/vhe/switch.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)arch/arm64/kvm/hyp/vhe/switch.c +0
-0@@ -344,13 +344,15 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)* if this is a VHE guest hypervisor returning to its own * userspace, or the hypervisor performing a local exception * return. No need to save/restore registers, no need to
arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 353
Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +6
-3-* switch S2 MMU. Just do the canonical ERET. +* switch S2 MMU. Just do the canonical ERET unless we are in +* nested context. * -* Unless the trap has to be forwarded further down the line, -* of course... +* Note that this is made possible because KVM itself never traps +* ERET when running an L2. The consequence is that any ERET trap is +* the result of HCR_EL2 or HFGITR_EL2 programming by L1 for its own +* guest, and the exception must be forwarded to L1.
Yes, right.
Quoted 9 lines
arch/arm64/kvm/hyp/vhe/switch.c +1
-3*/ -if ((__vcpu_sys_reg (vcpu,HCR_EL2 ) &HCR_NV ) || - (__vcpu_sys_reg (vcpu,HFGITR_EL2 ) &HFGITR_EL2_ERET )) +if (is_nested_ctxt (vcpu))return false; spsr =read_sysreg_el1 (SYS_SPSR ); --2.47.3
Marc Zyngier
maintainerPATCH v3 8/28 arm64: Add ARM64_HAS_NV2P1 capability
In reply to Marc ZyngierAs we're about to deal with FEAT_NV2P1, add a new capability that will be used to key any support for it.
--- arch/arm64/tools/cpucaps | 1 + 1 file changed, 1 insertion(+)
arch/arm64/tools/cpucaps +1 -0
@@ -51,6 +51,7 @@ HAS_LS64_V HAS_LSUI HAS_MOPS HAS_NESTED_VIRT +HAS_NV2P1 HAS_BBML2_NOABORT HAS_PAN HAS_PMUV3
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 9/28 KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
In reply to Marc ZyngierWith FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.
Prevent such trapping and context switch CPACTR_EL1 in NV contexts when NV2P1 is present.
--- arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++-- arch/arm64/kvm/hyp/vhe/switch.c | 3 +++ arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 8 +++++--- arch/arm64/kvm/sys_regs.c | 5 ++++- 4 files changed, 15 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +3 -2
@@ -108,9 +108,10 @@ static inline void __activate_cptr_traps_vhe(struct kvm_vcpu *vcpu)* The architecture is a bit crap (what a surprise): an EL2 guest * writing to CPTR_EL2 via CPACR_EL1 can't set any of TCPAC or TTA, * as they are RES0 in the guest's view. To work around it, trap the -* sucker using the very same bit it can't set... +* sucker using the very same bit it can't set. FEAT_NV2p1 fixes it. */ -if (vcpu_el2_e2h_is_set (vcpu) &&is_hyp_ctxt (vcpu)) +if (!cpus_have_final_cap (ARM64_HAS_NV2P1 ) && +vcpu_el2_e2h_is_set (vcpu) &&is_hyp_ctxt (vcpu)) val |=CPTR_EL2_TCPAC ;/*
arch/arm64/kvm/hyp/vhe/switch.c +3 -0
@@ -443,6 +443,9 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code)u64 esr =kvm_vcpu_get_esr (vcpu);int rt; +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +return false; +if (!is_hyp_ctxt (vcpu) ||esr_sys64_to_sysreg (esr) !=SYS_CPACR_EL1 )return false;
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +5 -3
@@ -42,10 +42,12 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)u64 val;/* -* We don't save CPTR_EL2, as accesses to CPACR_EL1 -* are always trapped, ensuring that the in-memory -* copy is always up-to-date. A small blessing... +* Without FEAT_NV2p1, we don't save CPTR_EL2, as accesses +* to CPACR_EL1 are always trapped, ensuring that the +* in-memory copy is always up-to-date. A small blessing... */ +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +__vcpu_assign_sys_reg (vcpu,CPTR_EL2 ,read_sysreg_el1 (SYS_CPACR ));__vcpu_assign_sys_reg (vcpu,SCTLR_EL2 ,read_sysreg_el1 (SYS_SCTLR ));__vcpu_assign_sys_reg (vcpu,TTBR0_EL2 ,read_sysreg_el1 (SYS_TTBR0 ));__vcpu_assign_sys_reg (vcpu,TTBR1_EL2 ,read_sysreg_el1 (SYS_TTBR1 ));
arch/arm64/kvm/sys_regs.c +4 -1
@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) val |=__vcpu_sys_reg (vcpu, reg) & ~CNTKCTL_VALID_BITS ;return val;case CPTR_EL2 : -return __vcpu_sys_reg (vcpu, reg); +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +return read_sysreg_el1 (SYS_CPACR ); +else +return __vcpu_sys_reg (vcpu, reg);default :WARN_ON_ONCE (1 ); }
Signature · 1 line
2.47.3
Re: [PATCH v3 09/28] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
Quoted 71 lines · Marc Zyngier
With
FEAT_NV2P1, it is no longer necessary to trapCPTR_EL2accesses viaCPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.Prevent such trapping and context switch
CPACTR_EL1in NV contexts when NV2P1 is present.Reviewed-by Joey Gouly <[email protected]>Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++-- arch/arm64/kvm/hyp/vhe/switch.c | 3 +++ arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 8 +++++--- arch/arm64/kvm/sys_regs.c | 5 ++++- 4 files changed, 15 insertions(+), 6 deletions(-)arch/arm64/kvm/hyp/include/hyp/switch.h +3
-2@@ -108,9 +108,10 @@ static inline void __activate_cptr_traps_vhe(struct kvm_vcpu *vcpu)* The architecture is a bit crap (what a surprise): an EL2 guest * writing to CPTR_EL2 via CPACR_EL1 can't set any of TCPAC or TTA, * as they are RES0 in the guest's view. To work around it, trap the -* sucker using the very same bit it can't set... +* sucker using the very same bit it can't set. FEAT_NV2p1 fixes it. */ -if (vcpu_el2_e2h_is_set (vcpu) &&is_hyp_ctxt (vcpu)) +if (!cpus_have_final_cap (ARM64_HAS_NV2P1 ) && +vcpu_el2_e2h_is_set (vcpu) &&is_hyp_ctxt (vcpu)) val |=CPTR_EL2_TCPAC ;/* arch/arm64/kvm/hyp/vhe/switch.c +3
-0@@ -443,6 +443,9 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code)u64 esr =kvm_vcpu_get_esr (vcpu);int rt; +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +return false; +if (!is_hyp_ctxt (vcpu) ||esr_sys64_to_sysreg (esr) !=SYS_CPACR_EL1 )return false;arch/arm64/kvm/hyp/vhe/sysreg-sr.c +5
-3@@ -42,10 +42,12 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)u64 val;/* -* We don't save CPTR_EL2, as accesses to CPACR_EL1 -* are always trapped, ensuring that the in-memory -* copy is always up-to-date. A small blessing... +* Without FEAT_NV2p1, we don't save CPTR_EL2, as accesses +* to CPACR_EL1 are always trapped, ensuring that the +* in-memory copy is always up-to-date. A small blessing... */ +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +__vcpu_assign_sys_reg (vcpu,CPTR_EL2 ,read_sysreg_el1 (SYS_CPACR ));__vcpu_assign_sys_reg (vcpu,SCTLR_EL2 ,read_sysreg_el1 (SYS_SCTLR ));__vcpu_assign_sys_reg (vcpu,TTBR0_EL2 ,read_sysreg_el1 (SYS_TTBR0 ));__vcpu_assign_sys_reg (vcpu,TTBR1_EL2 ,read_sysreg_el1 (SYS_TTBR1 ));arch/arm64/kvm/sys_regs.c +0
-0
arch/arm64/kvm/sys_regs.c u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) · line 332
Quoted 8 lines
arch/arm64/kvm/sys_regs.c +4
-1@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) val |=__vcpu_sys_reg (vcpu, reg) & ~CNTKCTL_VALID_BITS ;return val;case CPTR_EL2 : -return __vcpu_sys_reg (vcpu, reg); +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +return read_sysreg_el1 (SYS_CPACR ); +else +return __vcpu_sys_reg (vcpu, reg);
Quoted 5 lines
arch/arm64/kvm/sys_regs.c +0
-1default :WARN_ON_ONCE (1 ); } --2.47.3
Marc Zyngier
maintainerPATCH v3 10/28 KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
In reply to Marc ZyngierWith NV2p1, it is no longer necessary to use the split approach where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1, and we can treat the CNTKCTL_EL1 accessor as if it was "normal".
Key the special casing on FEAT_NV2P1 not being implemented.
--- arch/arm64/kvm/arch_timer.c | 10 ++++++++-- arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 13 ++++++++++--- arch/arm64/kvm/sys_regs.c | 6 ++++-- 3 files changed, 22 insertions(+), 7 deletions(-)
arch/arm64/kvm/arch_timer.c +8 -2
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)assign_clear_set_bit (tvt02,CNTHCTL_EL1NVVCT , clr, set);assign_clear_set_bit (tpt02,CNTHCTL_EL1NVPCT , clr, set); -/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */ -sysreg_clear_set (cnthctl_el2, clr, set); +/* +* This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless +* we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1. +*/ +if (!cpus_have_final_cap (ARM64_HAS_NV2P1 )) +sysreg_clear_set (cnthctl_el2, clr, set); +else +sysreg_clear_set (cntkctl_el1, clr, set); }void kvm_timer_vcpu_load (struct kvm_vcpu *vcpu)
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +10 -3
@@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)* The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where * the interesting CNTHCTL_EL2 bits live. So preserve these * bits when reading back the guest-visible value. +* +* While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV +* even more broken than it already was with NV2. */ val =read_sysreg_el1 (SYS_CNTKCTL ); - val &=CNTKCTL_VALID_BITS ; - __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS); - __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val); +if (!cpus_have_final_cap (ARM64_HAS_NV2P1 )) { + val &=CNTKCTL_VALID_BITS ; + __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS); + __vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val); + }else { +__vcpu_assign_sys_reg (vcpu,CNTHCTL_EL2 , val); + } }__vcpu_assign_sys_reg (vcpu,SP_EL2 ,read_sysreg (sp_el1));
arch/arm64/kvm/sys_regs.c +4 -2
@@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)switch (reg) {case CNTHCTL_EL2 : val =read_sysreg_el1 (SYS_CNTKCTL ); - val &=CNTKCTL_VALID_BITS ; - val |=__vcpu_sys_reg (vcpu, reg) & ~CNTKCTL_VALID_BITS ; +if (!cpus_have_final_cap (ARM64_HAS_NV2P1 )) { + val &=CNTKCTL_VALID_BITS ; + val |=__vcpu_sys_reg (vcpu, reg) & ~CNTKCTL_VALID_BITS ; + }return val;case CPTR_EL2 :if (cpus_have_final_cap (ARM64_HAS_NV2P1 ))
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 11/28 KVM: arm64: Expose FEAT_NV2p1 to NV guests
In reply to Marc ZyngierSince NV2p1 is reducing the number of traps, it is valuable to expose it to NV guests. Do so.
--- arch/arm64/kvm/nested.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/arm64/kvm/nested.c +6 -2
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)* You get EITHER * * - FEAT_VHE without FEAT_E2H0 -* - FEAT_NV limited to FEAT_NV2 +* - FEAT_NV limited to FEAT_NV2(p1) * - HCR_EL2.NV1 being RES0 * * OR
@@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)if (test_bit (KVM_ARM_VCPU_HAS_EL2_E2H0 , kvm->arch.vcpu_features)) { val =0 ; }else { - val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); + val &= ID_AA64MMFR4_EL1_NV_frac; +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) + val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ); +else + val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); val |=SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 ,E2H0 ,NI_NV1 ); }break ;
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The ARM64_HAS_NV2P1 CPU feature capability is checked dynamically to enable FEAT_NV2p1 logic in…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The ARM64_HAS_NV2P1 CPU feature capability is checked dynamically to enable FEAT_NV2p1 logic in KVM, but the capability is never registered in the kernel's CPU feature framework, rendering the new feature entirely dead code. --
--- Patch [11]: [PATCH v3 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests --- commit 2bb48d2c7459805b9d877b250f95afb9c6786a43
Author: Marc Zyngier <[email protected]>
KVM: arm64: Expose FEAT_NV2p1 to NV guests
This commit exposes the FEAT_NV2p1 feature to nested virtualization guests by updating the ID_AA64MMFR4_EL1 register limits when the hardware capability is present. This intends to reduce the number of traps for NV guests.
Quoted 4 lines
arch/arm64/kvm/nested.c +0
-0
[ ... ]
arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1742
Quoted 7 lines
arch/arm64/kvm/nested.c +2
-1@@ -1738,7 +1738,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)if (test_bit (KVM_ARM_VCPU_HAS_EL2_E2H0 , kvm->arch.vcpu_features)) { val =0 ; }else { - val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); + val &= ID_AA64MMFR4_EL1_NV_frac; +if (cpus_have_final_cap (ARM64_HAS_NV2P1 ))
[Severity: Medium]
Is ARM64_HAS_NV2P1 registered in the kernel's CPU feature framework?
It appears that the ARM64_HAS_NV2P1 capability is missing from the arm64_features array in arch/arm64/kernel/cpufeature.c. Because it is not mapped to a hardware ID register field, this capability check in limit_nv_id_reg() will always evaluate to false.
Does this render the newly added FEAT_NV2p1 support unreachable?
Quoted 6 lines
arch/arm64/kvm/nested.c +3
-0+ val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ); + else + val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); val |=SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 ,E2H0 ,NI_NV1 ); }break ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Marc Zyngier
maintainerRe: [PATCH v3 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests
Quoted 31 lines · On Wed, 22 Jul 2026 09:32:52 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] TheARM64_HAS_NV2P1CPU feature capability is checked dynamically to enableFEAT_NV2p1logic in KVM, but the capability is never registered in the kernel's CPU feature framework, rendering the new feature entirely dead code. ----- Patch [11]: [PATCH v3 11/28] KVM: arm64: Expose
FEAT_NV2p1to NV guests --- commit2bb48d2c7459805b9d877b250f95afb9c6786a43
Author: Marc Zyngier <[email protected]>KVM: arm64: Expose
FEAT_NV2p1to NV guestsThis commit exposes the
FEAT_NV2p1feature to nested virtualization guests by updating theID_AA64MMFR4_EL1register limits when the hardware capability is present. This intends to reduce the number of traps for NV guests.Quoted 4 lines
arch/arm64/kvm/nested.c +0
-0[ ... ]
arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1742
Quoted 7 lines
arch/arm64/kvm/nested.c +2
-1@@ -1738,7 +1738,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)if (test_bit (KVM_ARM_VCPU_HAS_EL2_E2H0 , kvm->arch.vcpu_features)) { val =0 ; }else { - val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); + val &= ID_AA64MMFR4_EL1_NV_frac; +if (cpus_have_final_cap (ARM64_HAS_NV2P1 ))[Severity: Medium]
IsARM64_HAS_NV2P1registered in the kernel's CPU feature framework?
Quoted 6 lines
It appears that the
ARM64_HAS_NV2P1capability is missing from thearm64_featuresarray inarch/arm64/kernel/cpufeature.c. Because it is not mapped to a hardware ID register field, this capability check inlimit_nv_id_reg()will always evaluate to false.Does this render the newly added
FEAT_NV2p1support unreachable?
See follow-up patch.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 11/28] KVM: arm64: Expose FEAT_NV2p1 to NV guests
In reply to Marc ZyngierQuoted 2 lines · Marc Zyngier
Since NV2p1 is reducing the number of traps, it is valuable to expose it to NV guests. Do so.
Quoted 34 lines
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/nested.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/arm64/kvm/nested.c +6
-3@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)* You get EITHER * * - FEAT_VHE without FEAT_E2H0 -* - FEAT_NV limited to FEAT_NV2 +* - FEAT_NV limited to FEAT_NV2(p1) * - HCR_EL2.NV1 being RES0 * * OR @@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)if (test_bit (KVM_ARM_VCPU_HAS_EL2_E2H0 , kvm->arch.vcpu_features)) { val =0 ; }else { - val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); + val &= ID_AA64MMFR4_EL1_NV_frac; +if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) + val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ); +else + val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); val |=SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 ,E2H0 ,NI_NV1 ); }break ; --2.47.3
Marc Zyngier
maintainerPATCH v3 12/28 arm64: Add FEAT_NV2p1 detection
In reply to Marc ZyngierAdd the necessary NV2p1 probing to the cpufeature infrastructure.
--- arch/arm64/kernel/cpufeature.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kernel/cpufeature.c +7 -0
@@ -2620,6 +2620,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { {/* Sentinel */ } }, }, + { + .desc = "FEAT_NV2p1", + .capability = ARM64_HAS_NV2P1, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) + }, { .capability =ARM64_HAS_32BIT_EL0_DO_NOT_USE , .type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 12/28] arm64: Add FEAT_NV2p1 detection
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Missing isb() immediately after writing to control-plane system registers. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Missing isb() immediately after writing to control-plane system registers. --
--- Patch [12]: [PATCH v3 12/28] arm64: Add FEAT_NV2p1 detection --- commit 5488b94ad37f884fedfaca076980f395c30a72bf
Author: Marc Zyngier <[email protected]>
arm64: Add FEAT_NV2p1 detection
This commit adds the necessary NV2p1 probing to the cpufeature infrastructure.
Quoted 4 lines
arch/arm64/kernel/cpufeature.c +0
-0
[ ... ]
arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2629
Quoted 11 lines
arch/arm64/kernel/cpufeature.c +7
-0@@ -2620,6 +2620,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { {/* Sentinel */ } }, }, + { + .desc = "FEAT_NV2p1", + .capability = ARM64_HAS_NV2P1, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) + },
[Severity: High]
Are we missing an isb() after writing to control-plane system registers when this capability is enabled?
Looking at arch/arm64/kvm/arch_timer.c in timer_set_traps(), there is a missing isb() after the system register writes:
arch/arm64/kvm/arch_timer.c:timer_set_traps() {
...
if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
sysreg_clear_set(cnthctl_el2, clr, set);
else
sysreg_clear_set(cntkctl_el1, clr, set);
}The sysreg_clear_set macro evaluates to a conditional write but lacks an immediate isb(), and the function returns shortly after.
Similarly, in arch/arm64/kvm/sys_regs.c, are we missing synchronization before the break?
arch/arm64/kvm/sys_regs.c:vcpu_write_sys_reg() {
...
case CNTHCTL_EL2:
write_sysreg_el1(val, SYS_CNTKCTL);
break;
case CPTR_EL2:
write_sysreg_el1(val, SYS_CPACR);
break;
...
}The writes to SYS_CPACR and SYS_CNTKCTL are followed by a break and a subsequent function return, rather than an immediate isb().
Without immediate synchronization, the CPU might operate under a stale configuration for several cycles, which could cause unexpected traps or break memory safety semantics.
Quoted 3 lines
arch/arm64/kernel/cpufeature.c +0
-0{ .capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE, .type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
Marc Zyngier
maintainerRe: [PATCH v3 12/28] arm64: Add FEAT_NV2p1 detection
Quoted 3 lines · On Wed, 22 Jul 2026 09:13:05 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Missingisb()immediately after writing to control-plane system registers.
https://lore.kernel.org/all/[email protected]/
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 12/28] arm64: Add FEAT_NV2p1 detection
In reply to Marc ZyngierQuoted 12 lines · Marc Zyngier
Add the necessary NV2p1 probing to the cpufeature infrastructure.
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kernel/cpufeature.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kernel/cpufeature.c +0
-0
arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2629
Quoted 10 lines
arch/arm64/kernel/cpufeature.c +7
-0@@ -2620,6 +2620,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { {/* Sentinel */ } }, }, + { + .desc = "FEAT_NV2p1", + .capability = ARM64_HAS_NV2P1, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) + },
Quoted 5 lines
arch/arm64/kernel/cpufeature.c +0
-1{ .capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE, .type = ARM64_CPUCAP_SYSTEM_FEATURE, --2.47.3
Marc Zyngier
maintainerPATCH v3 13/28 arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
In reply to Marc ZyngierFEAT_NV3 introduces a new register that contains the HCR_EL2 value exposed to a NV guest. As such, it has the exact same layout as HCR_EL2.
Describe NVHCR_EL2 as a mapping to HCR_EL2.
--- arch/arm64/tools/sysreg | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/tools/sysreg +3 -0
@@ -4266,6 +4266,9 @@ Field 1 E2TRE Field 0 E0HTRE EndSysreg +Sysreg NVHCR_EL2 3 4 1 5 0 +Mapping HCR_EL2 +EndSysreg Sysreg HDFGRTR2_EL2 3 4 3 1 0 Res0 63:25
Signature · 1 line
2.47.3
Re: [PATCH v3 13/28] arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
Quoted 16 lines · Marc Zyngier
FEAT_NV3introduces a new register that contains theHCR_EL2value exposed to a NV guest. As such, it has the exact same layout asHCR_EL2.Describe
NVHCR_EL2as a mapping toHCR_EL2.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/tools/sysreg | 3 +++ 1 file changed, 3 insertions(+)arch/arm64/tools/sysreg +0
-0
arch/arm64/tools/sysreg Field 1 E2TRE · line 4269
Quoted 4 lines
arch/arm64/tools/sysreg +1
-0@@ -4266,6 +4266,9 @@ Field 1 E2TRE Field 0 E0HTRE EndSysreg +Sysreg NVHCR_EL2 3 4 1 5 0
3 4 1 5 0, right.
Quoted 7 lines
arch/arm64/tools/sysreg +2
-1+Mapping HCR_EL2 +EndSysreg Sysreg HDFGRTR2_EL2 3 4 3 1 0 Res0 63:25 --2.47.3
Marc Zyngier
maintainerPATCH v3 14/28 arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
In reply to Marc ZyngierFEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them in the sysreg file.
--- arch/arm64/tools/sysreg | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
arch/arm64/tools/sysreg +6 -1
@@ -4548,7 +4548,12 @@ Fields ZCR_ELx EndSysreg Sysreg HCRX_EL2 3 4 1 2 2 -Res0 63:27 +Res0 63:35 +Field 34 NVnTTLBOS +Field 33 NVnTTLBIS +Field 32 NVnTTLB +Res0 31:28 +Field 27 NVTGE Field 26 SRMASKEn Res0 25 Field 24 PACMEn
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] A `.sysreg` declarative change alters the generated `HCRX_EL2_RES0` mask by allocating new fields…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] A .sysreg declarative change alters the generated HCRX_EL2_RES0 mask by allocating new fields, but KVM's feature maps were not updated, leading to semantic drift. --
--- Patch [14]: [PATCH v3 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3 --- commit 60811bd1f3efd334ec0429ff21afed15626dfb76
Author: Marc Zyngier <[email protected]>
arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
FEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them in the sysreg file.
Quoted 5 lines
arch/arm64/tools/sysreg +0
-0
arch/arm64/tools/sysreg Fields ZCR_ELx · line 4556
Quoted 10 lines
arch/arm64/tools/sysreg +6
-1@@ -4548,7 +4548,12 @@ Fields ZCR_ELx EndSysreg Sysreg HCRX_EL2 3 4 1 2 2 -Res0 63:27 +Res0 63:35 +Field 34 NVnTTLBOS +Field 33 NVnTTLBIS +Field 32 NVnTTLB +Res0 31:28 +Field 27 NVTGE
[Severity: High]
Will this change to the declarative definition of HCRX_EL2 inadvertently break KVM's guest register sanitization?
By changing bits 27, 32, 33, and 34 from Res0 to Field, this silently removes them from the auto-generated HCRX_EL2_RES0 mask.
However, KVM's hcrx_feat_map in arch/arm64/kvm/config.c relies on that mask and was not updated to explicitly restrict these newly defined FEAT_NV3 fields:
arch/arm64/kvm/config.c:hcrx_feat_map[] = {
NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
...
};Because these fields are now unmapped, check_feat_map() strictly enforces the initialization check and will fail unconditionally on host boot:
arch/arm64/kvm/config.c:check_feat_map() {
...
if (mask != ~resx)
kvm_err("Undefined %s behaviour, bits %016llx\n",
str, mask ^ ~resx);
}More importantly, KVM uses these feature maps in compute_reg_resx_bits() to calculate the valid bits a guest can modify. Without the fields present in HCRX_EL2_RES0 or mapped in hcrx_feat_map, KVM fails to mark these bits as RES0 for guests.
Could this allow an untrusted L1 nested guest to write to these unsanitized EL2-level nested virtualization control bits, propagating to physical hardware when context-switching?
Quoted 3 lines
arch/arm64/tools/sysreg +0
-0Field 26 SRMASKEn Res0 25 Field 24 PACMEn
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Marc Zyngier
maintainerRe: [PATCH v3 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
Quoted 5 lines · On Wed, 22 Jul 2026 09:11:54 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] A.sysregdeclarative change alters the generatedHCRX_EL2_RES0mask by allocating new fields, but KVM's feature
maps were not updated, leading to semantic drift.
https://lore.kernel.org/all/[email protected]/
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH v3 15/28 arm64: Add ARM64_HAS_NV3 capability
In reply to Marc ZyngierAs a bunch of KVM code is going to depend on FEAT_NV3 being detected on the host, add a new capability that will describe it.
--- arch/arm64/tools/cpucaps | 1 + 1 file changed, 1 insertion(+)
arch/arm64/tools/cpucaps +1 -0
@@ -52,6 +52,7 @@ HAS_LSUI HAS_MOPS HAS_NESTED_VIRT HAS_NV2P1 +HAS_NV3 HAS_BBML2_NOABORT HAS_PAN HAS_PMUV3
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 16/28 KVM: arm64: Split NV-specific exit fixups from the non-NV handling
In reply to Marc ZyngierIn order to facilitate further changes, move the NV handling of early fixups in its own helper. This also makes the code slightly simpler to parse.
--- arch/arm64/kvm/hyp/vhe/switch.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +11 -7
@@ -539,18 +539,15 @@ static const exit_handler_fn hyp_exit_handlers[] = { [0x3F] = kvm_hyp_handle_impdef, }; -static inline bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +static void fixup_nv_guest_exit (struct kvm_vcpu *vcpu) { -synchronize_vcpu_pstate (vcpu); -/* * If we were in HYP context on entry, adjust the PSTATE view * so that the usual helpers work correctly. This enforces our * invariant that the guest's HYP context status is preserved * across a run. */ -if (vcpu_has_nv (vcpu) && -unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) { +if (unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) {u64 mode = *vcpu_cpsr (vcpu) & (PSR_MODE_MASK |PSR_MODE32_BIT );switch (mode) {
@@ -567,8 +564,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) }/* Apply extreme paranoia! */ -BUG_ON (vcpu_has_nv (vcpu) && - !!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +BUG_ON (!!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +} + +static bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +{ +synchronize_vcpu_pstate (vcpu); + +if (vcpu_has_nv (vcpu)) +fixup_nv_guest_exit (vcpu);return __fixup_guest_exit (vcpu, exit_code, hyp_exit_handlers); }
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 16/28] KVM: arm64: Split NV-specific exit fixups from the non-NV handling
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Removing `inline` from `fixup_guest_exit()` and breaking out `fixup_nv_guest_exit()` exposes them…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Removing inline from fixup_guest_exit() and breaking out fixup_nv_guest_exit() exposes them to kprobes, which will cause a fatal host panic if probed. --
--- Patch [16]: [PATCH v3 16/28] KVM: arm64: Split NV-specific exit fixups from the non-NV handling --- commit 54c98942c16b88b656ffdb68540aac18e6fe4d3f
Author: Marc Zyngier <[email protected]>
KVM: arm64: Split NV-specific exit fixups from the non-NV handling
This commit moves the nested virtualization (NV) handling of early fixups into its own helper function. This simplifies parsing and prepares the code for further changes.
arch/arm64/kvm/hyp/vhe/switch.c static const exit_handler_fn hyp_exit_handlers[] = { · line 542
Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +1
-1@@ -539,18 +539,15 @@ static const exit_handler_fn hyp_exit_handlers[] = { [0x3F] = kvm_hyp_handle_impdef, }; -static inline bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +static void fixup_nv_guest_exit (struct kvm_vcpu *vcpu)
[Severity: High]
Does extracting this function without a NOKPROBE_SYMBOL() annotation expose it to kprobes?
Since this executes on the VM exit path while the KVM hypervisor vectors are still loaded, placing a kprobe here would trigger a BRK instruction that traps to __kvm_hyp_vector. This cannot handle kprobes and would unconditionally call hyp_panic(), crashing the host.
Should we add NOKPROBE_SYMBOL(fixup_nv_guest_exit) to prevent this?
Quoted 12 lines
arch/arm64/kvm/hyp/vhe/switch.c +1
-4{ -synchronize_vcpu_pstate (vcpu); -/* * If we were in HYP context on entry, adjust the PSTATE view * so that the usual helpers work correctly. This enforces our * invariant that the guest's HYP context status is preserved * across a run. */ -if (vcpu_has_nv (vcpu) && -unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) { +if (unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) {
[ ... ]
arch/arm64/kvm/hyp/vhe/switch.c static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · line 570
Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +4
-2@@ -567,8 +564,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) }/* Apply extreme paranoia! */ -BUG_ON (vcpu_has_nv (vcpu) && - !!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +BUG_ON (!!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +} + +static bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code)
[Severity: High]
By removing the inline keyword here, doesn't this also expose fixup_guest_exit() to kprobes if the compiler chooses not to inline it?
Should this function also receive a NOKPROBE_SYMBOL() annotation to avoid the same host panic hazard?
Quoted 8 lines
arch/arm64/kvm/hyp/vhe/switch.c +5
-0+{ +synchronize_vcpu_pstate (vcpu); + +if (vcpu_has_nv (vcpu)) +fixup_nv_guest_exit (vcpu);return __fixup_guest_exit (vcpu, exit_code, hyp_exit_handlers); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Marc Zyngier
maintainerRe: [PATCH v3 16/28] KVM: arm64: Split NV-specific exit fixups from the non-NV handling
Quoted 29 lines · On Wed, 22 Jul 2026 09:23:03 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Removinginlinefromfixup_guest_exit()and breaking outfixup_nv_guest_exit()exposes them to kprobes, which will cause a fatal host panic if probed. ----- Patch [16]: [PATCH v3 16/28] KVM: arm64: Split NV-specific exit fixups from the non-NV handling --- commit
54c98942c16b88b656ffdb68540aac18e6fe4d3f
Author: Marc Zyngier <[email protected]>
KVM: arm64: Split NV-specific exit fixups from the non-NV handlingThis commit moves the nested virtualization (NV) handling of early fixups into its own helper function. This simplifies parsing and prepares the code for further changes.
arch/arm64/kvm/hyp/vhe/switch.c static const exit_handler_fn hyp_exit_handlers[] = { · line 542
Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +1
-1@@ -539,18 +539,15 @@ static const exit_handler_fn hyp_exit_handlers[] = { [0x3F] = kvm_hyp_handle_impdef, }; -static inline bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +static void fixup_nv_guest_exit (struct kvm_vcpu *vcpu)[Severity: High]
Does extracting this function without aNOKPROBE_SYMBOL()annotation expose it to kprobes?
Quoted 6 lines
Since this executes on the VM exit path while the KVM hypervisor vectors are still loaded, placing a kprobe here would trigger a BRK instruction that traps to
__kvm_hyp_vector. This cannot handle kprobes and would unconditionally callhyp_panic(), crashing the host.Should we add
NOKPROBE_SYMBOL(fixup_nv_guest_exit)to prevent this?
Maybe, but this is out of scope for this series.
Quoted 19 lines
Quoted 13 lines
arch/arm64/kvm/hyp/vhe/switch.c +1
-4{ -synchronize_vcpu_pstate (vcpu); -/* * If we were in HYP context on entry, adjust the PSTATE view * so that the usual helpers work correctly. This enforces our * invariant that the guest's HYP context status is preserved * across a run. */ -if (vcpu_has_nv (vcpu) && -unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) { +if (unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) {[ ... ]
Quoted 3 lines
arch/arm64/kvm/hyp/vhe/switch.c +0
-0@@ -567,8 +564,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) }
Quoted 11 lines
arch/arm64/kvm/hyp/vhe/switch.c static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · line 570
Quoted 7 lines
arch/arm64/kvm/hyp/vhe/switch.c +4
-2/* Apply extreme paranoia! */ -BUG_ON (vcpu_has_nv (vcpu) && - !!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +BUG_ON (!!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +} + +static bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code)[Severity: High]
By removing the inline keyword here, doesn't this also exposefixup_guest_exit()to kprobes if the compiler chooses not to inline it?
Marking something inline doesn't guarantee inlining either.
Quoted 3 lines
Should this function also receive a
NOKPROBE_SYMBOL()annotation to avoid the same host panic hazard?
See above.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 16/28] KVM: arm64: Split NV-specific exit fixups from the non-NV handling
In reply to Marc ZyngierQuoted 3 lines · Marc Zyngier
In order to facilitate further changes, move the NV handling of early fixups in its own helper. This also makes the code slightly simpler to parse.
Quoted 51 lines
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/vhe/switch.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)arch/arm64/kvm/hyp/vhe/switch.c +11
-8@@ -539,18 +539,15 @@ static const exit_handler_fn hyp_exit_handlers[] = { [0x3F] = kvm_hyp_handle_impdef, }; -static inline bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +static void fixup_nv_guest_exit (struct kvm_vcpu *vcpu) { -synchronize_vcpu_pstate (vcpu); -/* * If we were in HYP context on entry, adjust the PSTATE view * so that the usual helpers work correctly. This enforces our * invariant that the guest's HYP context status is preserved * across a run. */ -if (vcpu_has_nv (vcpu) && -unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) { +if (unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) {u64 mode = *vcpu_cpsr (vcpu) & (PSR_MODE_MASK |PSR_MODE32_BIT );switch (mode) {@@ -567,8 +564,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) }/* Apply extreme paranoia! */ -BUG_ON (vcpu_has_nv (vcpu) && - !!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +BUG_ON (!!host_data_test_flag (VCPU_IN_HYP_CONTEXT ) !=is_hyp_ctxt (vcpu)); +} + +static bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +{ +synchronize_vcpu_pstate (vcpu); + +if (vcpu_has_nv (vcpu)) +fixup_nv_guest_exit (vcpu);return __fixup_guest_exit (vcpu, exit_code, hyp_exit_handlers); } --2.47.3
Marc Zyngier
maintainerPATCH v3 17/28 KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
In reply to Marc ZyngierExpose the FEAT_NV3 control bits to the sanitisation code so that KVM stops moaning about the unattributed bits.
--- arch/arm64/kvm/config.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/arm64/kvm/config.c +6 -0
@@ -225,6 +225,7 @@ struct reg_feat_map_desc {#define FEAT_HCX ID_AA64MMFR1_EL1, HCX, IMP#define FEAT_S2PIE ID_AA64MMFR3_EL1, S2PIE, IMP#define FEAT_GCIE ID_AA64PFR2_EL1, GCIE, IMP +#define FEAT_NV3 ID_AA64MMFR4_EL1, NV_frac, NV3static bool not_feat_aa64el3 (struct kvm *kvm) {
@@ -904,6 +905,11 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,static const struct reg_bits_to_feat_map hcrx_feat_map[] = { +NEEDS_FEAT (HCRX_EL2_NVTGE | + HCRX_EL2_NVnTTLB | + HCRX_EL2_NVnTTLBIS | + HCRX_EL2_NVnTTLBOS, +FEAT_NV3 ),NEEDS_FEAT (HCRX_EL2_SRMASKEn,FEAT_SRMASK ),NEEDS_FEAT (HCRX_EL2_PACMEn, feat_pauth_lr),NEEDS_FEAT (HCRX_EL2_EnFPM,FEAT_FPMR ),
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 18/28 KVM: arm64: Add kvm_has_nv{2,3}() predicates
In reply to Marc ZyngierAdd a new set of predicates indicating whether VM is capable of NV2, NV3, and is in a nested NV3 context.
This is going to become useful as we start dealing with a mix of behaviours (NV2, NV3, NV2 on NV3...).
--- arch/arm64/include/asm/kvm_emulate.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +20 -0
@@ -266,6 +266,26 @@ static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu) (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); } +static inline bool kvm_has_nv2 (struct kvm *kvm) +{ +return (cpus_have_final_cap (ARM64_HAS_NESTED_VIRT ) && +kvm_has_feat (kvm,ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY )); +} + +static inline bool kvm_has_nv3 (struct kvm *kvm) +{ +return (cpus_have_final_cap (ARM64_HAS_NESTED_VIRT ) && +cpus_have_final_cap (ARM64_HAS_NV3 ) && +kvm_has_feat (kvm,ID_AA64MMFR4_EL1 , NV_frac,NV3 )); +} + +static inline bool is_nested_nv3_ctxt (struct kvm_vcpu *vcpu) +{ +return (has_vhe () &&kvm_has_nv3 (vcpu->kvm) &&is_nested_ctxt (vcpu) && + (__vcpu_sys_reg (vcpu,HCR_EL2 ) &HCR_EL2_NV ) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_NVTGE )); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
Signature · 1 line
2.47.3
Re: [PATCH v3 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates
Quoted 19 lines · Marc Zyngier
Add a new set of predicates indicating whether VM is capable of NV2, NV3, and is in a nested NV3 context.
This is going to become useful as we start dealing with a mix of behaviours (NV2, NV3, NV2 on NV3...).
Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)arch/arm64/include/asm/kvm_emulate.h +0
-0@@ -266,6 +266,26 @@ static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu) (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_TMEA ); }
arch/arm64/include/asm/kvm_emulate.h static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu) · line 278
Quoted 10 lines
arch/arm64/include/asm/kvm_emulate.h +10
-0+static inline bool kvm_has_nv2 (struct kvm *kvm) +{ +return (cpus_have_final_cap (ARM64_HAS_NESTED_VIRT ) && +kvm_has_feat (kvm,ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY )); +} + +static inline bool kvm_has_nv3 (struct kvm *kvm) +{ + return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) && + cpus_have_final_cap(ARM64_HAS_NV3) &&
Minor: only check cpus_have_final_cap(ARM64_HAS_NV3) is enough ?
Quoted 15 lines
arch/arm64/include/asm/kvm_emulate.h +10
-1+ kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3)); +} + +static inline bool is_nested_nv3_ctxt (struct kvm_vcpu *vcpu) +{ +return (has_vhe () &&kvm_has_nv3 (vcpu->kvm) &&is_nested_ctxt (vcpu) && + (__vcpu_sys_reg (vcpu,HCR_EL2 ) &HCR_EL2_NV ) && + (__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_NVTGE )); +} +/* * The layout of SPSR for an AArch32 state is different when observed from an * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32 -- 2.47.3
Marc Zyngier
maintainerPATCH v3 19/28 KVM: arm64: Make HCR_EL2 a non-VNCR register
In reply to Marc ZyngierFEAT_NV3 makes a fundamental change to the architecture, by moving guest-initiated HCR_EL2 accesses to the NVHCR_EL2 register. As the names suggests, this is HCR_EL2 for a NV guest.
But where do NVHCR_EL2 accesses from a guest go? The are redirected to the VNCR page, right where HCR_EL2 is stored in the NV2 case. Does it hurt? Good. There's more coming.
The challenge here is to make KVM work seamlessly, without rewriting everything. Which implies that things such as __vcpu_sys_reg(HCR_EL2) must work, no matter the underlying NV implementation.
A simple way to deal with it is to move HCR_EL2's canonical storage outside of VNCR for the vast majority of the KVM code, and only have a copy at entry/exit times. Given that we don't really support NV3 yet, this is pretty simple.
In the process, advertise NVHCR_EL2 as the register that now holds offset 0x78 in the VNCR page.
--- arch/arm64/include/asm/kvm_host.h | 3 ++- arch/arm64/include/asm/vncr_mapping.h | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 9 +++++++++ 3 files changed, 12 insertions(+), 2 deletions(-)
arch/arm64/include/asm/kvm_host.h +2 -1
@@ -543,6 +543,7 @@ enum vcpu_sysreg {MDCR_EL2 ,/* Monitor Debug Configuration Register (EL2) */ CNTHCTL_EL2 ,/* Counter-timer Hypervisor Control register */ ZCR_EL2 ,/* SVE Control Register (EL2) */ +HCR_EL2 ,/* Hypervisor Control Register */ /* Any VNCR-capable reg goes after this point */ MARKER(__VNCR_START__),
@@ -571,7 +572,7 @@ enum vcpu_sysreg {VNCR (TFSR_EL1 ),/* Tag Fault Status Register (EL1) */ VNCR (VPIDR_EL2 ),/* Virtualization Processor ID Register */ VNCR (VMPIDR_EL2 ),/* Virtualization Multiprocessor ID Register */ -VNCR (HCR_EL2 ),/* Hypervisor Configuration Register */ +VNCR (NVHCR_EL2 ),/* NV Hypervisor Configuration Register */ VNCR (HSTR_EL2 ),/* Hypervisor System Trap Register */ VNCR (VTTBR_EL2 ),/* Virtualization Translation Table Base Register */ VNCR(VTCR_EL2), /* Virtualization Translation Control Register */
arch/arm64/include/asm/vncr_mapping.h +1 -1
@@ -11,7 +11,7 @@#define VNCR_VTCR_EL2 0x040#define VNCR_VMPIDR_EL2 0x050#define VNCR_CNTVOFF_EL2 0x060 -#define VNCR_HCR_EL2 0x078 +#define VNCR_NVHCR_EL2 0x078#define VNCR_HSTR_EL2 0x080#define VNCR_VPIDR_EL2 0x088#define VNCR_TPIDR_EL2 0x090
arch/arm64/kvm/hyp/vhe/switch.c +9 -0
@@ -70,6 +70,9 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)if (!vcpu_el2_e2h_is_set (vcpu)) hcr |=HCR_NV1 ; +/* Publish the guest's view of HCR_EL2 to the HW */ +__vcpu_assign_sys_reg (vcpu,NVHCR_EL2 ,__vcpu_sys_reg (vcpu,HCR_EL2 )); +/* * Nothing in HCR_EL2 should impact running in hypervisor * context, apart from bits we have defined as RESx (E2H,
@@ -549,6 +552,7 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)*/ if (unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) {u64 mode = *vcpu_cpsr (vcpu) & (PSR_MODE_MASK |PSR_MODE32_BIT ); +u64 hcr;switch (mode) {case PSR_MODE_EL1t:
@@ -561,6 +565,11 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu) *vcpu_cpsr (vcpu) &= ~(PSR_MODE_MASK |PSR_MODE32_BIT ); *vcpu_cpsr (vcpu) |= mode; + +/* Publish the latest HCR_EL2 to the emulation */ + hcr =__vcpu_sys_reg (vcpu,NVHCR_EL2 ); + +__vcpu_assign_sys_reg (vcpu,HCR_EL2 , hcr); }/* Apply extreme paranoia! */
Signature · 1 line
2.47.3
Re: [PATCH v3 19/28] KVM: arm64: Make HCR_EL2 a non-VNCR register
Quoted 85 lines · Marc Zyngier
FEAT_NV3makes a fundamental change to the architecture, by moving guest-initiatedHCR_EL2accesses to theNVHCR_EL2register. As the names suggests, this isHCR_EL2for a NV guest.But where do
NVHCR_EL2accesses from a guest go? The are redirected to the VNCR page, right whereHCR_EL2is stored in the NV2 case. Does it hurt? Good. There's more coming.The challenge here is to make KVM work seamlessly, without rewriting everything. Which implies that things such as
__vcpu_sys_reg(HCR_EL2)must work, no matter the underlying NV implementation.A simple way to deal with it is to move HCR_EL2's canonical storage outside of VNCR for the vast majority of the KVM code, and only have a copy at entry/exit times. Given that we don't really support NV3 yet, this is pretty simple.
In the process, advertise
NVHCR_EL2as the register that now holds offset0x78in the VNCR page.Reviewed-by Joey Gouly <[email protected]>Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 3 ++- arch/arm64/include/asm/vncr_mapping.h | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 9 +++++++++ 3 files changed, 12 insertions(+), 2 deletions(-)arch/arm64/include/asm/kvm_host.h +2
-1@@ -543,6 +543,7 @@ enum vcpu_sysreg {MDCR_EL2 ,/* Monitor Debug Configuration Register (EL2) */ CNTHCTL_EL2 ,/* Counter-timer Hypervisor Control register */ ZCR_EL2 ,/* SVE Control Register (EL2) */ +HCR_EL2 ,/* Hypervisor Control Register */ /* Any VNCR-capable reg goes after this point */ MARKER(__VNCR_START__),@@ -571,7 +572,7 @@ enum vcpu_sysreg {VNCR (TFSR_EL1 ),/* Tag Fault Status Register (EL1) */ VNCR (VPIDR_EL2 ),/* Virtualization Processor ID Register */ VNCR (VMPIDR_EL2 ),/* Virtualization Multiprocessor ID Register */ -VNCR (HCR_EL2 ),/* Hypervisor Configuration Register */ +VNCR (NVHCR_EL2 ),/* NV Hypervisor Configuration Register */ VNCR (HSTR_EL2 ),/* Hypervisor System Trap Register */ VNCR (VTTBR_EL2 ),/* Virtualization Translation Table Base Register */ VNCR(VTCR_EL2), /* Virtualization Translation Control Register */arch/arm64/include/asm/vncr_mapping.h +1
-1@@ -11,7 +11,7 @@#define VNCR_VTCR_EL2 0x040#define VNCR_VMPIDR_EL2 0x050#define VNCR_CNTVOFF_EL2 0x060 -#define VNCR_HCR_EL2 0x078 +#define VNCR_NVHCR_EL2 0x078#define VNCR_HSTR_EL2 0x080#define VNCR_VPIDR_EL2 0x088#define VNCR_TPIDR_EL2 0x090arch/arm64/kvm/hyp/vhe/switch.c +4
-0@@ -70,6 +70,9 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)if (!vcpu_el2_e2h_is_set (vcpu)) hcr |=HCR_NV1 ; +/* Publish the guest's view of HCR_EL2 to the HW */ +__vcpu_assign_sys_reg (vcpu,NVHCR_EL2 ,__vcpu_sys_reg (vcpu,HCR_EL2 )); +/* * Nothing in HCR_EL2 should impact running in hypervisor * context, apart from bits we have defined as RESx (E2H, @@ -549,6 +552,7 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)*/ if (unlikely (host_data_test_flag (VCPU_IN_HYP_CONTEXT ))) {u64 mode = *vcpu_cpsr (vcpu) & (PSR_MODE_MASK |PSR_MODE32_BIT ); +u64 hcr;switch (mode) {case PSR_MODE_EL1t:
arch/arm64/kvm/hyp/vhe/switch.c static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu) · line 572
Quoted 8 lines
arch/arm64/kvm/hyp/vhe/switch.c +5
-0@@ -561,6 +565,11 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu) *vcpu_cpsr (vcpu) &= ~(PSR_MODE_MASK |PSR_MODE32_BIT ); *vcpu_cpsr (vcpu) |= mode; + +/* Publish the latest HCR_EL2 to the emulation */ + hcr =__vcpu_sys_reg (vcpu,NVHCR_EL2 ); + +__vcpu_assign_sys_reg (vcpu,HCR_EL2 , hcr);
Quoted 5 lines
arch/arm64/kvm/hyp/vhe/switch.c +0
-1}/* Apply extreme paranoia! */ --2.47.3
Marc Zyngier
maintainerPATCH v3 20/28 KVM: arm64: Add sanitisation for NVHCR_EL2
In reply to Marc ZyngierJust like any other VNCR-based register, NVHCR_EL2 requires some level of sanitisation. Being specified as a live copy of HCR_EL2, it adopts the exact same format, but depends on FEAT_NV3 instead.
A subtle aspect is that we only want to apply the sanitisation if FEAT_NV3 is actually present, as the VNCR location is otherwise used to back accesses to HCR_EL2.
--- arch/arm64/kvm/config.c | 15 +++++++++++++++ arch/arm64/kvm/nested.c | 4 ++++ 2 files changed, 19 insertions(+)
arch/arm64/kvm/config.c +15 -0
@@ -1019,6 +1019,9 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = { static const DECLARE_FEAT_MAP(hcr_desc, HCR_EL2, hcr_feat_map, FEAT_AA64EL2); +static const DECLARE_FEAT_MAP(nvhcr_desc, NVHCR_EL2, + hcr_feat_map, FEAT_NV3); +static const struct reg_bits_to_feat_map sctlr2_feat_map[] = { NEEDS_FEAT(SCTLR2_EL1_NMEA | SCTLR2_EL1_EASE,
@@ -1393,6 +1396,7 @@ void __init check_feature_map(void)check_reg_desc (&hdfgwtr2_desc);check_reg_desc (&hcrx_desc);check_reg_desc (&hcr_desc); +check_reg_desc (&nvhcr_desc);check_reg_desc (&sctlr2_desc);check_reg_desc (&tcr2_el2_desc);check_reg_desc (&sctlr_el1_desc);
@@ -1592,6 +1596,17 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)case HCR_EL2 : resx =compute_reg_resx_bits (kvm, &hcr_desc,0 ,0 );break ; +case NVHCR_EL2 : +/* +* Only apply sanitisation if we do have FEAT_NV3. +* Otherwise, the register aliases with HCR_EL2 in VNCR, +* and we're better off relying on data transfers between +* NVHCR_EL2 and HCR_EL2 to sanitise things. +*/ + resx = (kvm_has_nv3 (kvm) ? +compute_reg_resx_bits (kvm, &nvhcr_desc,0 ,0 ) : + (typeof (resx )){}); +break ;case SCTLR2_EL1 :case SCTLR2_EL2 : resx =compute_reg_resx_bits (kvm, &sctlr2_desc,0 ,0 );
arch/arm64/kvm/nested.c +4 -0
@@ -1830,6 +1830,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) resx =get_reg_fixed_bits (kvm,HCR_EL2 );set_sysreg_masks (kvm,HCR_EL2 , resx); +/* NVHCR_EL2 */ + resx =get_reg_fixed_bits (kvm,NVHCR_EL2 ); +set_sysreg_masks (kvm,NVHCR_EL2 , resx); +/* HCRX_EL2 */ resx =get_reg_fixed_bits (kvm,HCRX_EL2 );set_sysreg_masks (kvm,HCRX_EL2 , resx);
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 20/28] KVM: arm64: Add sanitisation for NVHCR_EL2
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Subsystem rule violation: Numeric range loop over sparse enum vcpu_sysreg --
--- Patch [20]: [PATCH v3 20/28] KVM: arm64: Add sanitisation for NVHCR_EL2 --- commit b143c0689f4a9db3345ab50e6a712d0698ebaa75
Author: Marc Zyngier <[email protected]>
KVM: arm64: Add sanitisation for NVHCR_EL2
This commit adds sanitisation support for the NVHCR_EL2 register. It mimics the format of HCR_EL2 but conditionally depends on FEAT_NV3 being present. If FEAT_NV3 is absent, it relies on standard VNCR data transfers.
[ ... ]
Quoted 5 lines
arch/arm64/kvm/nested.c +0
-0
arch/arm64/kvm/nested.c int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) · line 1837
Quoted 10 lines
arch/arm64/kvm/nested.c +4
-0@@ -1828,6 +1828,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) resx =get_reg_fixed_bits (kvm,HCR_EL2 );set_sysreg_masks (kvm,HCR_EL2 , resx); +/* NVHCR_EL2 */ + resx =get_reg_fixed_bits (kvm,NVHCR_EL2 ); +set_sysreg_masks (kvm,NVHCR_EL2 , resx); +/* HCRX_EL2 */ resx =get_reg_fixed_bits (kvm,HCRX_EL2 );set_sysreg_masks (kvm,HCRX_EL2 , resx);
[Severity: High]
This is a pre-existing issue, but later in kvm_init_nv_sysregs() there is a loop over enum vcpu_sysreg:
for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) __vcpu_rmw_sys_reg(vcpu, sr, |=, 0);
Does this numeric range loop iterate over unintended registers?
The KVM ARM64 subsystem guidelines state that enum vcpu_sysreg is sparsely numbered by VNCR byte offsets. Using a < comparison and incrementing over a numeric range touches register indices that happen to fall within the byte span.
Could this silently drop or corrupt guest state by applying operations to unrelated registers?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=20
Marc Zyngier
maintainerRe: [PATCH v3 20/28] KVM: arm64: Add sanitisation for NVHCR_EL2
Quoted 45 lines · On Wed, 22 Jul 2026 09:38:03 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Subsystem rule violation: Numeric range loop over sparseenum vcpu_sysreg----- Patch [20]: [PATCH v3 20/28] KVM: arm64: Add sanitisation for
NVHCR_EL2--- commitb143c0689f4a9db3345ab50e6a712d0698ebaa75
Author: Marc Zyngier <[email protected]>KVM: arm64: Add sanitisation for
NVHCR_EL2This commit adds sanitisation support for the
NVHCR_EL2register. It mimics the format ofHCR_EL2but conditionally depends onFEAT_NV3being present. IfFEAT_NV3is absent, it relies on standard VNCR data transfers.[ ... ]
Quoted 5 lines
arch/arm64/kvm/nested.c +0
-0arch/arm64/kvm/nested.c int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) · line 1837
Quoted 10 lines
arch/arm64/kvm/nested.c +4
-0@@ -1828,6 +1828,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) resx =get_reg_fixed_bits (kvm,HCR_EL2 );set_sysreg_masks (kvm,HCR_EL2 , resx); +/* NVHCR_EL2 */ + resx =get_reg_fixed_bits (kvm,NVHCR_EL2 ); +set_sysreg_masks (kvm,NVHCR_EL2 , resx); +/* HCRX_EL2 */ resx =get_reg_fixed_bits (kvm,HCRX_EL2 );set_sysreg_masks (kvm,HCRX_EL2 , resx);[Severity: High]
This is a pre-existing issue, but later inkvm_init_nv_sysregs()there is a loop over enumvcpu_sysreg:for (enum vcpu_sysreg sr = __SANITISED_REG_START__; sr < NR_SYS_REGS; sr++) __vcpu_rmw_sys_reg(vcpu, sr, |=, 0);Does this numeric range loop iterate over unintended registers?
Quoted 7 lines
The KVM ARM64 subsystem guidelines state that enum
vcpu_sysregis sparsely numbered by VNCR byte offsets. Using a < comparison and incrementing over a numeric range touches register indices that happen to fall within the byte span.Could this silently drop or corrupt guest state by applying operations to unrelated registers?
These are unallocated VNCR entries. we can do whatever we want with them, including scribbling Pi to the nth digit if there is enough space.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 20/28] KVM: arm64: Add sanitisation for NVHCR_EL2
In reply to Marc ZyngierQuoted 41 lines · Marc Zyngier
Just like any other VNCR-based register,
NVHCR_EL2requires some level of sanitisation. Being specified as a live copy ofHCR_EL2, it adopts the exact same format, but depends onFEAT_NV3instead.A subtle aspect is that we only want to apply the sanitisation if
FEAT_NV3is actually present, as the VNCR location is otherwise used to back accesses toHCR_EL2.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/config.c | 15 +++++++++++++++ arch/arm64/kvm/nested.c | 4 ++++ 2 files changed, 19 insertions(+)arch/arm64/kvm/config.c +5
-0@@ -1019,6 +1019,9 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = { static const DECLARE_FEAT_MAP(hcr_desc, HCR_EL2, hcr_feat_map, FEAT_AA64EL2); +static const DECLARE_FEAT_MAP(nvhcr_desc, NVHCR_EL2, + hcr_feat_map, FEAT_NV3); +static const struct reg_bits_to_feat_map sctlr2_feat_map[] = { NEEDS_FEAT(SCTLR2_EL1_NMEA | SCTLR2_EL1_EASE,@@ -1393,6 +1396,7 @@ void __init check_feature_map(void)check_reg_desc (&hdfgwtr2_desc);check_reg_desc (&hcrx_desc);check_reg_desc (&hcr_desc); +check_reg_desc (&nvhcr_desc);check_reg_desc (&sctlr2_desc);check_reg_desc (&tcr2_el2_desc);check_reg_desc (&sctlr_el1_desc);@@ -1592,6 +1596,17 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)case HCR_EL2 : resx =compute_reg_resx_bits (kvm, &hcr_desc,0 ,0 );break ; +case NVHCR_EL2 :
arch/arm64/kvm/config.c struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg) · line 1609
Quoted 10 lines
arch/arm64/kvm/config.c +10
-0+/* +* Only apply sanitisation if we do have FEAT_NV3. +* Otherwise, the register aliases with HCR_EL2 in VNCR, +* and we're better off relying on data transfers between +* NVHCR_EL2 and HCR_EL2 to sanitise things. +*/ + resx = (kvm_has_nv3 (kvm) ? +compute_reg_resx_bits (kvm, &nvhcr_desc,0 ,0 ) : + (typeof (resx )){}); +break ;
Quoted 20 lines
arch/arm64/kvm/config.c +0
-0case SCTLR2_EL1 :case SCTLR2_EL2 : resx =compute_reg_resx_bits (kvm, &sctlr2_desc,0 ,0 );arch/arm64/kvm/nested.c +4
-1@@ -1830,6 +1830,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu) resx =get_reg_fixed_bits (kvm,HCR_EL2 );set_sysreg_masks (kvm,HCR_EL2 , resx); +/* NVHCR_EL2 */ + resx =get_reg_fixed_bits (kvm,NVHCR_EL2 ); +set_sysreg_masks (kvm,NVHCR_EL2 , resx); +/* HCRX_EL2 */ resx =get_reg_fixed_bits (kvm,HCRX_EL2 );set_sysreg_masks (kvm,HCRX_EL2 , resx); --2.47.3
Marc Zyngier
maintainerPATCH v3 21/28 KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
In reply to Marc ZyngierExpose NVHCR_EL2 to userspace, and treat the direct access as UNDEF, as that would only outline a bug in our exception routing.
The generic accessors are also updated to deal with the relatively uncommon location of that register.
--- arch/arm64/kvm/sys_regs.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
arch/arm64/kvm/sys_regs.c +28 -0
@@ -221,6 +221,20 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,if (is_hyp_ctxt (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) loc->loc =SR_LOC_SPECIAL ;break ; +case NVHCR_EL2 : +/* +* Yes, NVHCR_EL2 maps to itself when loaded in nested +* context. If you feel like the architecture is double +* backing on itself upside down, you're not alone. +*/ +WARN_ON_ONCE (!kvm_has_nv3 (vcpu->kvm)); +if (is_hyp_ctxt (vcpu)) { + loc->loc =SR_LOC_MEMORY ; + }else { + loc->loc =SR_LOC_LOADED |SR_LOC_MAPPED ; + loc->map_reg =NVHCR_EL2 ; + } +break ;default : loc->loc =locate_direct_register (vcpu, reg); }
@@ -260,6 +274,7 @@ static u64 read_sr_from_cpu(enum vcpu_sysreg reg)case DACR32_EL2 : val =read_sysreg_s (SYS_DACR32_EL2 );break ;case IFSR32_EL2 : val =read_sysreg_s (SYS_IFSR32_EL2 );break ;case DBGVCR32_EL2 : val =read_sysreg_s (SYS_DBGVCR32_EL2 );break ; +case NVHCR_EL2 : val =read_sysreg_s (SYS_NVHCR_EL2 );break ;default :WARN_ON_ONCE (1 ); }
@@ -298,6 +313,7 @@ static void write_sr_to_cpu(enum vcpu_sysreg reg, u64 val)case DACR32_EL2 :write_sysreg_s (val,SYS_DACR32_EL2 );break ;case IFSR32_EL2 :write_sysreg_s (val,SYS_IFSR32_EL2 );break ;case DBGVCR32_EL2 :write_sysreg_s (val,SYS_DBGVCR32_EL2 );break ; +case NVHCR_EL2 :write_sysreg_s (val,SYS_NVHCR_EL2 );break ;default :WARN_ON_ONCE (1 ); } }
@@ -2861,6 +2877,16 @@ static unsigned int vncr_el2_visibility(const struct kvm_vcpu *vcpu,return REG_HIDDEN ; } +static unsigned int nvhcr_el2_visibility (const struct kvm_vcpu *vcpu, +const struct sys_reg_desc *rd) +{ +if (el2_visibility (vcpu, rd) ==0 && +kvm_has_feat (vcpu->kvm,ID_AA64MMFR4_EL1 , NV_frac,NV3 )) +return 0 ; + +return REG_HIDDEN ; +} +static unsigned int sctlr2_visibility (const struct kvm_vcpu *vcpu,const struct sys_reg_desc *rd) {
@@ -3774,6 +3800,8 @@ static const struct sys_reg_desc sys_reg_descs[] = { sve_el2_visibility),EL2_REG_VNCR (HCRX_EL2 , reset_val,0 ), +EL2_REG_FILTERED (NVHCR_EL2 , undef_access, reset_val,0 , + nvhcr_el2_visibility),EL2_REG (TTBR0_EL2 , access_rw, reset_val,0 ), EL2_REG(TTBR1_EL2, access_rw, reset_val, 0),
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 22/28 KVM: arm64: Add routing for NVHCR_EL2 trap
In reply to Marc ZyngierNVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0 and HCR_EL2.NV==1. Describe this in the exception routing tables.
--- arch/arm64/kvm/emulate-nested.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/arm64/kvm/emulate-nested.c +13 -0
@@ -136,6 +136,8 @@ enum cgt_group_id {CGT_CPTR_TTA ,CGT_MDCR_HPMN , + CGT_HCR_NV_HCRX_nNVTGE, +/* Must be last */ __NR_CGT_GROUP_IDS__ };
@@ -588,6 +590,15 @@ static enum trap_behaviour check_mdcr_hpmn(struct kvm_vcpu *vcpu)return BEHAVE_HANDLE_LOCALLY ; } +static enum trap_behaviour check_hcr_nv_hcrx_nnvtge (struct kvm_vcpu *vcpu) +{ +if ((__vcpu_sys_reg (vcpu,HCR_EL2 ) &HCR_EL2_NV ) && + !(__vcpu_sys_reg (vcpu,HCRX_EL2 ) &HCRX_EL2_NVTGE )) +return BEHAVE_FORWARD_RW ; + +return BEHAVE_HANDLE_LOCALLY ; +} +#define CCC (id, fn) \ [id - __COMPLEX_CONDITIONS__] = fn
@@ -598,6 +609,7 @@ static const complex_condition_check ccc[] = {CCC (CGT_CNTHCTL_EL1NVVCT , check_cnthctl_el1nvvct),CCC (CGT_CPTR_TTA , check_cptr_tta),CCC (CGT_MDCR_HPMN , check_mdcr_hpmn), + CCC(CGT_HCR_NV_HCRX_nNVTGE, check_hcr_nv_hcrx_nnvtge), };/*
@@ -853,6 +865,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = {SR_TRAP (SYS_SCTLR2_EL2 ,CGT_HCR_NV ),SR_RANGE_TRAP (SYS_HCR_EL2 ,SYS_HCRX_EL2 ,CGT_HCR_NV ), +SR_TRAP (SYS_NVHCR_EL2 , CGT_HCR_NV_HCRX_nNVTGE),SR_TRAP (SYS_SMPRIMAP_EL2 ,CGT_HCR_NV ),SR_TRAP (SYS_SMCR_EL2 ,CGT_HCR_NV ), SR_RANGE_TRAP(SYS_TTBR0_EL2,
Signature · 1 line
2.47.3
Marc Zyngier
maintainerPATCH v3 23/28 KVM: arm64: Add NVHCR_EL2 context switching
In reply to Marc ZyngierSince NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it must be dealt with in some particular way:
- for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly reflects HCR_EL2 as read and written by the guest itself. It must therefore be eagerly synced back with the emulation code which only knows about HCR_EL2. This is unconditional if NV3 is available on the host.
- For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we just context switch it like any other EL1 register. Yes, EL1, as that's where this thing runs from the PoV of L1. This is conditioned on the guest using NV3.
--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 10 ++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-)
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +11 -0
@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)if (ctxt_has_sctlr2 (ctxt))ctxt_sys_reg (ctxt,SCTLR2_EL1 ) =read_sysreg_el1 (SYS_SCTLR2 ); + +/* Retrieve L2's HCR_EL2, and save it for future use */ +if (is_nested_nv3_ctxt (ctxt_to_vcpu (ctxt))) +ctxt_sys_reg (ctxt,NVHCR_EL2 ) =read_sysreg_s (SYS_NVHCR_EL2 ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt)
@@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,if (ctxt_has_sctlr2 (ctxt))write_sysreg_el1 (ctxt_sys_reg (ctxt,SCTLR2_EL1 ),SYS_SCTLR2 ); + +/* +* Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3. +* Otherwise, the data is already in place in the L1's own VNCR. +*/ +if (is_nested_nv3_ctxt (ctxt_to_vcpu (ctxt))) +write_sysreg_s (ctxt_sys_reg (ctxt,NVHCR_EL2 ),SYS_NVHCR_EL2 ); }/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
arch/arm64/kvm/hyp/vhe/switch.c +8 -2
@@ -71,7 +71,10 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu) hcr |=HCR_NV1 ;/* Publish the guest's view of HCR_EL2 to the HW */ -__vcpu_assign_sys_reg (vcpu,NVHCR_EL2 ,__vcpu_sys_reg (vcpu,HCR_EL2 )); +if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_el2_e2h_is_set (vcpu)) +write_sysreg_s (__vcpu_sys_reg (vcpu,HCR_EL2 ),SYS_NVHCR_EL2 ); +else +__vcpu_assign_sys_reg (vcpu,NVHCR_EL2 ,__vcpu_sys_reg (vcpu,HCR_EL2 ));/* * Nothing in HCR_EL2 should impact running in hypervisor
@@ -567,7 +570,10 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu) *vcpu_cpsr (vcpu) |= mode;/* Publish the latest HCR_EL2 to the emulation */ - hcr =__vcpu_sys_reg (vcpu,NVHCR_EL2 ); + hcr = (cpus_have_final_cap (ARM64_HAS_NV3 ) && +vcpu_el2_e2h_is_set (vcpu)) ? +read_sysreg_s (SYS_NVHCR_EL2 ) : +__vcpu_sys_reg (vcpu,NVHCR_EL2 );__vcpu_assign_sys_reg (vcpu,HCR_EL2 , hcr); }
Signature · 1 line
2.47.3
Re: [PATCH v3 23/28] KVM: arm64: Add NVHCR_EL2 context switching
Quoted 67 lines · Marc Zyngier
Since
NVHCR_EL2represents theHCR_EL2state of the EL1 guest, it must be dealt with in some particular way:- for a guest in hyp context (an L1 by definition),
NVHCR_EL2directly reflectsHCR_EL2as read and written by the guest itself. It must therefore be eagerly synced back with the emulation code which only knows aboutHCR_EL2. This is unconditional if NV3 is available on the host.- For an L2 guest,
NVHCR_EL2is controlled by the L1 guest, and we just context switch it like any other EL1 register. Yes, EL1, as that's where this thing runs from the PoV of L1. This is conditioned on the guest using NV3.Reviewed-by Joey Gouly <[email protected]>Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 10 ++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-)arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +11
-0@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)if (ctxt_has_sctlr2 (ctxt))ctxt_sys_reg (ctxt,SCTLR2_EL1 ) =read_sysreg_el1 (SYS_SCTLR2 ); + +/* Retrieve L2's HCR_EL2, and save it for future use */ +if (is_nested_nv3_ctxt (ctxt_to_vcpu (ctxt))) +ctxt_sys_reg (ctxt,NVHCR_EL2 ) =read_sysreg_s (SYS_NVHCR_EL2 ); }static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt)@@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,if (ctxt_has_sctlr2 (ctxt))write_sysreg_el1 (ctxt_sys_reg (ctxt,SCTLR2_EL1 ),SYS_SCTLR2 ); + +/* +* Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3. +* Otherwise, the data is already in place in the L1's own VNCR. +*/ +if (is_nested_nv3_ctxt (ctxt_to_vcpu (ctxt))) +write_sysreg_s (ctxt_sys_reg (ctxt,NVHCR_EL2 ),SYS_NVHCR_EL2 ); }/* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */ arch/arm64/kvm/hyp/vhe/switch.c +4
-1@@ -71,7 +71,10 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu) hcr |=HCR_NV1 ;/* Publish the guest's view of HCR_EL2 to the HW */ -__vcpu_assign_sys_reg (vcpu,NVHCR_EL2 ,__vcpu_sys_reg (vcpu,HCR_EL2 )); +if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_el2_e2h_is_set (vcpu)) +write_sysreg_s (__vcpu_sys_reg (vcpu,HCR_EL2 ),SYS_NVHCR_EL2 ); +else +__vcpu_assign_sys_reg (vcpu,NVHCR_EL2 ,__vcpu_sys_reg (vcpu,HCR_EL2 ));/* * Nothing in HCR_EL2 should impact running in hypervisor
arch/arm64/kvm/hyp/vhe/switch.c static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu) · line 576
Quoted 8 lines
arch/arm64/kvm/hyp/vhe/switch.c +4
-1@@ -567,7 +570,10 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu) *vcpu_cpsr (vcpu) |= mode;/* Publish the latest HCR_EL2 to the emulation */ - hcr =__vcpu_sys_reg (vcpu,NVHCR_EL2 ); + hcr = (cpus_have_final_cap (ARM64_HAS_NV3 ) && +vcpu_el2_e2h_is_set (vcpu)) ? +read_sysreg_s (SYS_NVHCR_EL2 ) : +__vcpu_sys_reg (vcpu,NVHCR_EL2 );
Quoted 5 lines
arch/arm64/kvm/hyp/vhe/switch.c +0
-1__vcpu_assign_sys_reg (vcpu,HCR_EL2 , hcr); } --2.47.3
Marc Zyngier
maintainerPATCH v3 24/28 KVM: arm64: Engage NV3 ERET trap elision
In reply to Marc ZyngierWhen running on NV3 HW, always engage ERET trap elision when running the L1 context, as there is no benefit in not doing so.
An L1 can itself engage trap elision by setting its own view of HCRX_EL2.NVTGE==1, which will subsequently be honored.
--- arch/arm64/include/asm/kvm_emulate.h | 10 ++++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 4 ++++ 2 files changed, 14 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +10 -0
@@ -706,6 +706,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)if (kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_V )) vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR; + +/* +* NV3 is a host-specific extension, and we always use it +* when present and that the guest uses NV. It may be be +* hidden from the guest though. +*/ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) && +vcpu_has_nv (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) { + vcpu->arch.hcrx_el2 |=HCRX_EL2_NVTGE ; + } } }#endif /* __ARM64_KVM_EMULATE_H__ */
arch/arm64/kvm/hyp/vhe/switch.c +4 -0
@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)u64 esr =kvm_vcpu_get_esr (vcpu);u64 spsr, elr, mode; +/* With NV3, the fast path is handled in HW */ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_el2_e2h_is_set (vcpu)) +return false; +/* * Going through the whole put/load motions is a waste of time * if this is a VHE guest hypervisor returning to its own
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 24/28] KVM: arm64: Engage NV3 ERET trap elision
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] NV3 ERET trap elision (HCRX_EL2_NVTGE) is evaluated statically at vCPU initialization, failing to…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] NV3 ERET trap elision (HCRX_EL2_NVTGE) is evaluated statically at vCPU initialization, failing to reflect dynamic changes to HCR_EL2.E2H by the guest, resulting in a severe performance regression.
- [Low] Typographical error in the newly added comment for NV3.
--
--- Patch [24]: [PATCH v3 24/28] KVM: arm64: Engage NV3 ERET trap elision --- commit c8d6c5654353c481c8b8e9e0a6741ae11cc31c69
Author: Marc Zyngier <[email protected]>
KVM: arm64: Engage NV3 ERET trap elision
This patch enables ERET trap elision for L1 contexts on NV3 hardware by setting HCRX_EL2.NVTGE, and adjusts the EL2 fast-path to bounce back to EL1 when NV3 and E2H are active, assuming the hardware handles the fast path.
Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +0
-0
[ ... ]
arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 712
Quoted 8 lines
arch/arm64/include/asm/kvm_emulate.h +4
-0@@ -706,6 +706,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)if (kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_V )) vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR; + +/* +* NV3 is a host-specific extension, and we always use it +* when present and that the guest uses NV. It may be be
[Severity: Low]
This isn't a bug, but there is a typo (be be) in this comment.
arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 718
Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +6
-0+* hidden from the guest though. +*/ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) && +vcpu_has_nv (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) { + vcpu->arch.hcrx_el2 |=HCRX_EL2_NVTGE ; + }
[Severity: High]
Does evaluating the dynamic guest state vcpu_el2_e2h_is_set() here work correctly given this populates the static vcpu->arch.hcrx_el2 cache?
If the guest hypervisor (L1) initializes and dynamically enables HCR_EL2.E2H after KVM has already initialized the vCPU via kvm_calculate_traps(), this HCRX_EL2_NVTGE bit won't be set because vcpu_set_hcrx() is not refreshed dynamically.
This causes the hardware to fail to engage trap elision, forcing the ERET instruction to trap to EL2 where it encounters the new check below.
arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 351
Quoted 12 lines
arch/arm64/kvm/hyp/vhe/switch.c +4
-0@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)u64 esr =kvm_vcpu_get_esr (vcpu);u64 spsr, elr, mode; +/* With NV3, the fast path is handled in HW */ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_el2_e2h_is_set (vcpu)) +return false; +
[Severity: High]
Will this unconditionally abort the fast path and force a severe performance regression if the hardware trap elision wasn't engaged?
Because the HCRX_EL2_NVTGE bit was evaluated statically in vcpu_set_hcrx() and missed the guest's dynamic update, the ERET traps into this function.
kvm_hyp_handle_eret() evaluates the dynamic E2H state, correctly observes that it is now set, and returns false under the incorrect assumption that the hardware handled the fast path. This bounces the trap to the slow host EL1 path, forcing a full, expensive context switch to emulate every L1 ERET.
Quoted 3 lines
arch/arm64/kvm/hyp/vhe/switch.c +0
-0/* * Going through the whole put/load motions is a waste of time * if this is a VHE guest hypervisor returning to its own
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=24
Marc Zyngier
maintainerRe: [PATCH v3 24/28] KVM: arm64: Engage NV3 ERET trap elision
Quoted 23 lines · On Wed, 22 Jul 2026 09:50:34 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] NV3 ERET trap elision (HCRX_EL2_NVTGE) is evaluated statically at vCPU initialization, failing to reflect dynamic changes to HCR_EL2.E2H by the guest, resulting in a severe performance regression.
- [Low] Typographical error in the newly added comment for NV3.
----- Patch [24]: [PATCH v3 24/28] KVM: arm64: Engage NV3 ERET trap elision --- commit
c8d6c5654353c481c8b8e9e0a6741ae11cc31c69
Author: Marc Zyngier <[email protected]>KVM: arm64: Engage NV3 ERET trap elision
This patch enables ERET trap elision for L1 contexts on NV3 hardware by setting HCRX_EL2.NVTGE, and adjusts the EL2 fast-path to bounce back to EL1 when NV3 and E2H are active, assuming the hardware handles the fast path.
Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +0
-0[ ... ]
Quoted 2 lines
arch/arm64/include/asm/kvm_emulate.h +0
-0@@ -706,6 +706,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
Quoted 9 lines
arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 712
Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +4
-0if (kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_V )) vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR; + +/* +* NV3 is a host-specific extension, and we always use it +* when present and that the guest uses NV. It may be be [Severity: Low]
This isn't a bug, but there is a typo (be be) in this comment.
Fixed.
Quoted 12 lines
arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 718
Quoted 7 lines
arch/arm64/include/asm/kvm_emulate.h +6
-0+* hidden from the guest though. +*/ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) && +vcpu_has_nv (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) { + vcpu->arch.hcrx_el2 |=HCRX_EL2_NVTGE ; + }[Severity: High]
Does evaluating the dynamic guest statevcpu_el2_e2h_is_set()here work correctly given this populates the staticvcpu->arch.hcrx_el2cache?
Quoted 4 lines
If the guest hypervisor (L1) initializes and dynamically enables HCR_EL2.E2H after KVM has already initialized the vCPU via
kvm_calculate_traps(), thisHCRX_EL2_NVTGEbit won't be set becausevcpu_set_hcrx()is not refreshed dynamically.
vcpu_el2_e2h_is_set() is a fixed value. Always. It cannot change.
Quoted 12 lines
This causes the hardware to fail to engage trap elision, forcing the ERET instruction to trap to EL2 where it encounters the new check below.
Quoted 8 lines
arch/arm64/kvm/hyp/vhe/switch.c +0
-0@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)u64 esr =kvm_vcpu_get_esr (vcpu);u64 spsr, elr, mode;
Quoted 8 lines
arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 351
Quoted 4 lines
arch/arm64/kvm/hyp/vhe/switch.c +4
-0+/* With NV3, the fast path is handled in HW */ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_el2_e2h_is_set (vcpu)) +return false; +[Severity: High]
Will this unconditionally abort the fast path and force a severe performance regression if the hardware trap elision wasn't engaged?
As the comment says: we never get there if we're not *already* on the slow path. That's the spec. Read the spec.
Quoted 8 lines
Because the
HCRX_EL2_NVTGEbit was evaluated statically invcpu_set_hcrx()and missed the guest's dynamic update, the ERET traps into this function.
kvm_hyp_handle_eret()evaluates the dynamic E2H state, correctly observes that it is now set, and returns false under the incorrect assumption that the hardware handled the fast path. This bounces the trap to the slow host EL1 path, forcing a full, expensive context switch to emulate every L1 ERET.
Yet another Artificial Idiot nonsense.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 24/28] KVM: arm64: Engage NV3 ERET trap elision
In reply to Marc ZyngierQuoted 39 lines · Marc Zyngier
When running on NV3 HW, always engage ERET trap elision when running the L1 context, as there is no benefit in not doing so.
An L1 can itself engage trap elision by setting its own view of HCRX_EL2.NVTGE==1, which will subsequently be honored.
Reviewed-by Joey Gouly <[email protected]>Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/include/asm/kvm_emulate.h | 10 ++++++++++ arch/arm64/kvm/hyp/vhe/switch.c | 4 ++++ 2 files changed, 14 insertions(+)arch/arm64/include/asm/kvm_emulate.h +10
-0@@ -706,6 +706,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)if (kvm_has_feat (kvm,ID_AA64ISAR1_EL1 ,LS64 ,LS64_V )) vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR; + +/* +* NV3 is a host-specific extension, and we always use it +* when present and that the guest uses NV. It may be be +* hidden from the guest though. +*/ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) && +vcpu_has_nv (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) { + vcpu->arch.hcrx_el2 |=HCRX_EL2_NVTGE ; + } } }#endif /* __ARM64_KVM_EMULATE_H__ */ arch/arm64/kvm/hyp/vhe/switch.c +0
-0
arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 351
Quoted 7 lines
arch/arm64/kvm/hyp/vhe/switch.c +4
-0@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)u64 esr =kvm_vcpu_get_esr (vcpu);u64 spsr, elr, mode; +/* With NV3, the fast path is handled in HW */ +if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_el2_e2h_is_set (vcpu)) +return false; +
Quoted 5 lines
arch/arm64/kvm/hyp/vhe/switch.c +0
-1/* * Going through the whole put/load motions is a waste of time * if this is a VHE guest hypervisor returning to its own -- 2.47.3
Marc Zyngier
maintainerPATCH v3 25/28 KVM: arm64: Engage NV3 TLBI trap elision
In reply to Marc ZyngierSimilarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs that only affects the guest's S1 translation. Enable this, with the express condition that the guest isn't NV2 aware, as we otherwise need to trap these TLBIs to deal with VNCR mappings.
--- arch/arm64/include/asm/kvm_emulate.h | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +13 -0
@@ -715,6 +715,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)if (cpus_have_final_cap (ARM64_HAS_NV3 ) &&vcpu_has_nv (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) { vcpu->arch.hcrx_el2 |=HCRX_EL2_NVTGE ; + +/* +* If the guest is NV2-capable, then we need to see +* all the TLBIs, as configured in HCR_EL2. +* Otherwise, relax the TLBI traps to only TGE=0. +*/ +if (!kvm_has_nv2 (vcpu->kvm)) { + vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB | + HCRX_EL2_NVnTTLBIS); + +if (kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) + vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;; + } } } }
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 25/28] KVM: arm64: Engage NV3 TLBI trap elision
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Stray double semicolon creates an empty statement. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Stray double semicolon creates an empty statement.
--
--- Patch [25]: [PATCH v3 25/28] KVM: arm64: Engage NV3 TLBI trap elision --- commit dabbb189ece66c0128f9610303e005d9bdd94252
Author: Marc Zyngier <[email protected]>
KVM: arm64: Engage NV3 TLBI trap elision
This commit enables NV3 TLBI trap elision for guests that are not NV2 aware, relaxing the TLBI traps when only the guest's S1 translation is affected.
Quoted 3 lines
arch/arm64/include/asm/kvm_emulate.h +0
-0
[ ... ]
Quoted 1 line
arch/arm64/include/asm/kvm_emulate.h +0
-0@@ -715,6 +715,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
[ ... ]
arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 729
Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +6
-0+if (!kvm_has_nv2 (vcpu->kvm)) { + vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB | + HCRX_EL2_NVnTTLBIS); + +if (kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) + vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;;
[Severity: Low]
This isn't a bug, but there's a stray double semicolon at the end of this statement.
Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +1
-0+ } } } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=25
Marc Zyngier
maintainerRe: [PATCH v3 25/28] KVM: arm64: Engage NV3 TLBI trap elision
Quoted 20 lines · On Wed, 22 Jul 2026 09:57:45 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Stray double semicolon creates an empty statement.
----- Patch [25]: [PATCH v3 25/28] KVM: arm64: Engage NV3 TLBI trap elision --- commit
dabbb189ece66c0128f9610303e005d9bdd94252
Author: Marc Zyngier <[email protected]>KVM: arm64: Engage NV3 TLBI trap elision
This commit enables NV3 TLBI trap elision for guests that are not NV2 aware, relaxing the TLBI traps when only the guest's S1 translation is affected.
Quoted 3 lines
arch/arm64/include/asm/kvm_emulate.h +0
-0[ ... ]
Quoted 1 line
arch/arm64/include/asm/kvm_emulate.h +0
-0@@ -715,6 +715,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)[ ... ]
Quoted 10 lines
arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 729
Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +6
-0+if (!kvm_has_nv2 (vcpu->kvm)) { + vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB | + HCRX_EL2_NVnTTLBIS); + +if (kvm_has_feat (kvm,ID_AA64ISAR0_EL1 ,TLB ,OS )) + vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;;[Severity: Low]
This isn't a bug, but there's a stray double semicolon at the end of this statement.
I'll fix that.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerPATCH v3 26/28 KVM: arm64: Add FEAT_NV3 detection
In reply to Marc ZyngierNow that everything is in place to engage the FEAT_NV3 fast-path, add the detection code to cpufeature.c.
--- arch/arm64/kernel/cpufeature.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/arm64/kernel/cpufeature.c +7 -0
@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature,ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) }, + { + .desc ="FEAT_NV3" , + .capability = ARM64_HAS_NV3, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV3 ) + }, { .capability =ARM64_HAS_32BIT_EL0_DO_NOT_USE , .type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 26/28] KVM: arm64: Add FEAT_NV3 detection
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM's FEAT_NV3 fast-path evaluates to false for all guests, causing stale SYS_NVHCR_EL2 state…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM's FEAT_NV3 fast-path evaluates to false for all guests, causing stale SYS_NVHCR_EL2 state during L2 nested entry.
- [High] Writes to SYS_NVHCR_EL2 lack required immediate Context Synchronization Event (ISB).
--
--- Patch [26]: [PATCH v3 26/28] KVM: arm64: Add FEAT_NV3 detection --- commit e5daceb215a787136ab61f9dafcd9ffe40f4346c
Author: Marc Zyngier <[email protected]>
KVM: arm64: Add FEAT_NV3 detection
This commit adds the FEAT_NV3 detection code to cpufeature.c by introducing the ARM64_HAS_NV3 capability. It engages the FEAT_NV3 fast-path in KVM now that the infrastructure is in place.
arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2632
Quoted 11 lines
arch/arm64/kernel/cpufeature.c +3
-0@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature,ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) }, + { + .desc ="FEAT_NV3" , + .capability = ARM64_HAS_NV3,
[Severity: High]
Could engaging this capability break nested virtualization on NV3 hardware?
By enabling ARM64_HAS_NV3, the fast-path checks now depend on kvm_has_nv3(), which appears to require the guest to advertise NV3:
arch/arm64/include/asm/kvm_emulate.h:kvm_has_nv3() {
return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
cpus_have_final_cap(ARM64_HAS_NV3) &&
kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
}Since NV3 is explicitly hidden from guests (ID_AA64MMFR4_EL1.NV_frac is capped at NV2P1), won't kvm_has_feat() always evaluate to false?
This would cause is_nested_nv3_ctxt() to always return false, skipping the SYS_NVHCR_EL2 restore in L2 nested entry:
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h:__sysreg_restore_el1_state() {
...
if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
...
}Will the physical CPU execute the L2 guest using an uninitialized or stale SYS_NVHCR_EL2 value?
[Severity: High]
Are there missing context synchronization events in this new fast-path?
When ARM64_HAS_NV3 is enabled, the code writes to the control-plane system register SYS_NVHCR_EL2 without an immediate isb():
arch/arm64/kvm/hyp/vhe/switch.c:__compute_hcr() {
if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
else
__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
/* No ISB here */
guest_hcr = kvm_vcpu_apply_reg_masks(vcpu, HCR_EL2, 0);
...
}It also happens during state restore before returning:
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h:__sysreg_restore_el1_state() {
...
if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
}/* No ISB before return */
Since SYS_NVHCR_EL2 is a control-plane register, doesn't it require an isb() as the very next instruction to prevent the CPU from executing subsequent instructions with undefined pipeline state?
Quoted 7 lines
arch/arm64/kernel/cpufeature.c +4
-0+ .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV3 ) + }, { .capability =ARM64_HAS_32BIT_EL0_DO_NOT_USE , .type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=26
Marc Zyngier
maintainerRe: [PATCH v3 26/28] KVM: arm64: Add FEAT_NV3 detection
Quoted 34 lines · On Wed, 22 Jul 2026 09:53:31 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM'sFEAT_NV3fast-path evaluates to false for all guests, causing staleSYS_NVHCR_EL2state during L2 nested entry.
- [High] Writes toSYS_NVHCR_EL2lack required immediate Context Synchronization Event (ISB).
----- Patch [26]: [PATCH v3 26/28] KVM: arm64: Add
FEAT_NV3detection --- commite5daceb215a787136ab61f9dafcd9ffe40f4346c
Author: Marc Zyngier <[email protected]>KVM: arm64: Add
FEAT_NV3detectionThis commit adds the
FEAT_NV3detection code tocpufeature.cby introducing theARM64_HAS_NV3capability. It engages theFEAT_NV3fast-path in KVM now that the infrastructure is in place.arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2632
Quoted 11 lines
arch/arm64/kernel/cpufeature.c +3
-0@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature,ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) }, + { + .desc ="FEAT_NV3" , + .capability = ARM64_HAS_NV3,[Severity: High]
Could engaging this capability break nested virtualization on NV3 hardware?By enabling
ARM64_HAS_NV3, the fast-path checks now depend onkvm_has_nv3(), which appears to require the guest to advertise NV3:
Quoted 8 lines
arch/arm64/include/asm/kvm_emulate.h:kvm_has_nv3() { return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) && cpus_have_final_cap(ARM64_HAS_NV3) && kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3)); }Since NV3 is explicitly hidden from guests (ID_AA64MMFR4_EL1.NV_frac is capped at NV2P1), won't
kvm_has_feat()always evaluate to false?
Yes. The enablement is in the following patch.
Quoted 11 lines
This would cause
is_nested_nv3_ctxt()to always return false, skipping theSYS_NVHCR_EL2restore in L2 nested entry:arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h:__sysreg_restore_el1_state() { ... if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt))) write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2); ... }
Quoted 2 lines
Will the physical CPU execute the L2 guest using an uninitialized or stale
SYS_NVHCR_EL2value?
No. This simply results in the NV2 behaviour.
Quoted 25 lines
[Severity: High]
Are there missing context synchronization events in this new fast-path?When
ARM64_HAS_NV3is enabled, the code writes to the control-plane system registerSYS_NVHCR_EL2without an immediateisb():arch/arm64/kvm/hyp/vhe/switch.c:__compute_hcr() { if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu)) write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2); else __vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2)); /* No ISB here */ guest_hcr = kvm_vcpu_apply_reg_masks(vcpu, HCR_EL2, 0); ... }It also happens during state restore before returning:
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h:__sysreg_restore_el1_state() { ... if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt))) write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2); }
Quoted 5 lines
/* No ISB before return */
Since
SYS_NVHCR_EL2is a control-plane register, doesn't it require anisb()as the very next instruction to prevent the CPU from executing subsequent instructions with undefined pipeline state?
Nonsense.
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 26/28] KVM: arm64: Add FEAT_NV3 detection
In reply to Marc ZyngierQuoted 13 lines · Marc Zyngier
Now that everything is in place to engage the
FEAT_NV3fast-path, add the detection code tocpufeature.c.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kernel/cpufeature.c | 7 +++++++ 1 file changed, 7 insertions(+)arch/arm64/kernel/cpufeature.c +0
-0
arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2636
Quoted 10 lines
arch/arm64/kernel/cpufeature.c +7
-0@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_cpuid_feature,ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ) }, + { + .desc ="FEAT_NV3" , + .capability = ARM64_HAS_NV3, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_cpuid_feature, +ARM64_CPUID_FIELDS (ID_AA64MMFR4_EL1 , NV_frac,NV3 ) + },
Quoted 5 lines
arch/arm64/kernel/cpufeature.c +0
-1{ .capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE, .type = ARM64_CPUCAP_SYSTEM_FEATURE, --2.47.3
Marc Zyngier
maintainerPATCH v3 27/28 KVM: arm64: Expose FEAT_NV3 to guests
In reply to Marc ZyngierFurther enable FEAT_NV3 by making it visible to NV guests.
--- arch/arm64/kvm/nested.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm64/kvm/nested.c +4 -2
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)* You get EITHER * * - FEAT_VHE without FEAT_E2H0 -* - FEAT_NV limited to FEAT_NV2(p1) +* - FEAT_NV limited to FEAT_NV2(p1)/NV3 * - HCR_EL2.NV1 being RES0 * * OR
@@ -1741,7 +1741,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) val =0 ; }else { val &= ID_AA64MMFR4_EL1_NV_frac; -if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +if (cpus_have_final_cap (ARM64_HAS_NV3 )) + val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV3 ); +else if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV2P1 );else val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY );
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH v3 27/28] KVM: arm64: Expose FEAT_NV3 to guests
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The new system register NVHCR_EL2, which is exposed to userspace when FEAT_NV3 is enabled, is…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The new system register NVHCR_EL2, which is exposed to userspace when FEAT_NV3 is enabled, is not registered in the get-reg-list selftest. --
--- Patch [27]: [PATCH v3 27/28] KVM: arm64: Expose FEAT_NV3 to guests --- commit 736b25683999a8114fb8bc21450f4b90835ba889
Author: Marc Zyngier <[email protected]>
KVM: arm64: Expose FEAT_NV3 to guests
This commit further enables FEAT_NV3 by making it visible to NV guests.
arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1743
Quoted 10 lines
arch/arm64/kvm/nested.c +2
-1@@ -1739,7 +1739,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) val =0 ; }else { val &= ID_AA64MMFR4_EL1_NV_frac; -if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +if (cpus_have_final_cap (ARM64_HAS_NV3 )) + val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV3 );
[Severity: Medium]
By exposing FEAT_NV3 here, will new system registers like NVHCR_EL2 become visible to userspace on NV3-capable hardware? If so, does the get-reg-list selftest in tools/testing/selftests/kvm/arm64/get-reg-list.c need to be updated (e.g., adding them to el2_regs or using REG_FEAT) to prevent the test suite from failing when it discovers unaccounted registers?
Quoted 4 lines
arch/arm64/kvm/nested.c +1
-0+else if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV2P1 );else val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY );
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=27
Marc Zyngier
maintainerRe: [PATCH v3 27/28] KVM: arm64: Expose FEAT_NV3 to guests
Quoted 22 lines · On Wed, 22 Jul 2026 10:06:17 +0100, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The new system registerNVHCR_EL2, which is exposed to userspace whenFEAT_NV3is enabled, is not registered in the get-reg-list selftest. ----- Patch [27]: [PATCH v3 27/28] KVM: arm64: Expose
FEAT_NV3to guests --- commit736b25683999a8114fb8bc21450f4b90835ba889
Author: Marc Zyngier <[email protected]>KVM: arm64: Expose
FEAT_NV3to guestsThis commit further enables
FEAT_NV3by making it visible to NV guests.Quoted 9 lines
arch/arm64/kvm/nested.c +1
-1@@ -1739,7 +1739,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) val =0 ; }else { val &= ID_AA64MMFR4_EL1_NV_frac; -if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +if (cpus_have_final_cap (ARM64_HAS_NV3 ))
Quoted 8 lines
arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1745
Quoted 1 line
arch/arm64/kvm/nested.c +1
-0+ val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV3 );[Severity: Medium]
By exposingFEAT_NV3here, will new system registers likeNVHCR_EL2become visible to userspace on NV3-capable hardware? If so, does the get-reg-list selftest intools/testing/selftests/kvm/arm64/get-reg-list.cneed to be updated (e.g., adding them toel2_regsor usingREG_FEAT) to prevent the test suite from failing when it discovers unaccounted registers?
Indeed. And this only proves that this test is utterly pointless. It should compute the visibility of registers on its own, from first principles, instead of relying on someone updating it for one reason or another...
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v3 27/28] KVM: arm64: Expose FEAT_NV3 to guests
In reply to Marc ZyngierQuoted 21 lines · Marc Zyngier
Further enable
FEAT_NV3by making it visible to NV guests.Signed-off-by Marc Zyngier <[email protected]>--- arch/arm64/kvm/nested.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)arch/arm64/kvm/nested.c +1
-1@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)* You get EITHER * * - FEAT_VHE without FEAT_E2H0 -* - FEAT_NV limited to FEAT_NV2(p1) +* - FEAT_NV limited to FEAT_NV2(p1)/NV3 * - HCR_EL2.NV1 being RES0 * * OR
arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1746
Quoted 7 lines
arch/arm64/kvm/nested.c +3
-1@@ -1741,7 +1741,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) val =0 ; }else { val &= ID_AA64MMFR4_EL1_NV_frac; -if (cpus_have_final_cap (ARM64_HAS_NV2P1 )) +if (cpus_have_final_cap (ARM64_HAS_NV3 )) + val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV3 ); +else if (cpus_have_final_cap (ARM64_HAS_NV2P1 ))
Quoted 5 lines
arch/arm64/kvm/nested.c +0
-1val =ID_REG_LIMIT_FIELD_ENUM (val,ID_AA64MMFR4_EL1 , NV_frac,NV2P1 ); else val =SYS_FIELD_PREP_ENUM (ID_AA64MMFR4_EL1 , NV_frac,NV2_ONLY ); --2.47.3
Marc Zyngier
maintainerPATCH v3 28/28 arm64: Add override for ID_AA64MMFR4_EL1.NV_frac
In reply to Marc ZyngierIn a very unsurprising turn of events, there is a large class of firmware that is totally unable to deal with FEAT_NV3, and doesn't set the required SCR2_EL3.NV3En bit, leading to an UNDEF exception or an unhandled trap to EL3, depending on the implementation.
Allow the unfortunate user to override ID_AA64MMFR4_EL1.NV_frac and get a working system. Hopefully firmware will be fixed before actually HW ships, but I have been there before... :-/
--- arch/arm64/include/asm/cpufeature.h | 1 + arch/arm64/kernel/cpufeature.c | 4 +++- arch/arm64/kernel/image-vars.h | 1 + arch/arm64/kernel/pi/idreg-override.c | 10 ++++++++++ 4 files changed, 15 insertions(+), 1 deletion(-)
arch/arm64/include/asm/cpufeature.h +1 -0
@@ -968,6 +968,7 @@ struct arm64_ftr_reg *get_arm64_ftr_reg(u32 sys_id);extern struct arm64_ftr_override id_aa64mmfr0_override;extern struct arm64_ftr_override id_aa64mmfr1_override;extern struct arm64_ftr_override id_aa64mmfr2_override; +extern struct arm64_ftr_override id_aa64mmfr4_override;extern struct arm64_ftr_override id_aa64pfr0_override;extern struct arm64_ftr_override id_aa64pfr1_override;extern struct arm64_ftr_override id_aa64zfr0_override;
arch/arm64/kernel/cpufeature.c +3 -1
@@ -785,6 +785,7 @@ static const struct arm64_ftr_bits ftr_raz[] = {struct arm64_ftr_override __read_mostly id_aa64mmfr0_override;struct arm64_ftr_override __read_mostly id_aa64mmfr1_override;struct arm64_ftr_override __read_mostly id_aa64mmfr2_override; +struct arm64_ftr_override __read_mostly id_aa64mmfr4_override;struct arm64_ftr_override __read_mostly id_aa64pfr0_override;struct arm64_ftr_override __read_mostly id_aa64pfr1_override;struct arm64_ftr_override __read_mostly id_aa64zfr0_override;
@@ -858,7 +859,8 @@ static const struct __ftr_reg_entry {ARM64_FTR_REG_OVERRIDE (SYS_ID_AA64MMFR2_EL1 , ftr_id_aa64mmfr2, &id_aa64mmfr2_override),ARM64_FTR_REG (SYS_ID_AA64MMFR3_EL1 , ftr_id_aa64mmfr3), -ARM64_FTR_REG (SYS_ID_AA64MMFR4_EL1 , ftr_id_aa64mmfr4), +ARM64_FTR_REG_OVERRIDE (SYS_ID_AA64MMFR4_EL1 , ftr_id_aa64mmfr4, + &id_aa64mmfr4_override),/* Op1 = 0, CRn = 10, CRm = 4 */ ARM64_FTR_REG(SYS_MPAMIDR_EL1, ftr_mpamidr),
arch/arm64/kernel/image-vars.h +1 -0
@@ -51,6 +51,7 @@ PI_EXPORT_SYM(id_aa64isar2_override);PI_EXPORT_SYM (id_aa64mmfr0_override);PI_EXPORT_SYM (id_aa64mmfr1_override);PI_EXPORT_SYM (id_aa64mmfr2_override); +PI_EXPORT_SYM (id_aa64mmfr4_override);PI_EXPORT_SYM (id_aa64pfr0_override);PI_EXPORT_SYM (id_aa64pfr1_override);PI_EXPORT_SYM (id_aa64smfr0_override);
arch/arm64/kernel/pi/idreg-override.c +10 -0
@@ -106,6 +106,15 @@ static const struct ftr_set_desc mmfr2 __prel64_initconst = { }, }; +static const struct ftr_set_desc mmfr4 __prel64_initconst = { + .name ="id_aa64mmfr4" , + .override = &id_aa64mmfr4_override, + .fields = { +FIELD ("nv_frac" , ID_AA64MMFR4_EL1_NV_frac_SHIFT,NULL ), + {} + }, +}; +static bool __init pfr0_sve_filter (u64 val) {/*
@@ -220,6 +229,7 @@ PREL64(const struct ftr_set_desc, reg) regs[] __prel64_initconst = { { &mmfr0 }, { &mmfr1 }, { &mmfr2 }, + { &mmfr4 }, { &pfr0 }, { &pfr1 }, { &isar1 },
Signature · 1 line
2.47.3