Skip to contentSkip to conversation

KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3

PATCH Superseded Marc Zyngier 6 participants 57 messages Jul 14 – Jul 17

Subsystems (3)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

v2 cover KVM: arm64: Add support for FEAT_NV2p1 and FEAT_NV3

This is the second version of a series adding support for two extensions targeting Nested Virt on arm64:

- FEAT_NV2p1 is effectively a bug fix for two registers (CNTHCTL_EL2 and CPTR_EL2) that are missing stateful bits when accessed from EL1 in a NV configuration. When this is present, the hypervisor can avoid a bunch of traps.

- FEAT_NV3 is much more ambitious, and changes the way ERET behaves in a NV environment. By moving EL1 accesses to HCR_EL2 from memory (via VNCR) to a dedicated register (NVHCR_EL2), the HW can detect whether the guest is performing an ERET for itself (NVHCR_EL2.TGE==1) or to its own guest (NVHCR_EL2.TGE==0). In the former case, ERET is done directly, and no trap occurs. Similar optimisations are available for a class of TLBI instructions.

The whole thing has been tested on an FVP model, and shown measurable improvements for an L1 guest (about 1.5% fewer instructions).

Given that this isn't very convincing on its own, I have built an approximate emulation of FEAT_NV3 that L1 (and deeper levels) can use on actual production hardware. For these deeper levels, the numbers are in the double digit of percentage point reduction, see below for details.

Does it make NV better? Yes!
Does it make NV good? Get real!

Anyway, patches on top of -rc3 plus the current state of kvmarm/fixes.

* Side notes about performance evaluation with NV3 emulation:

If you are interested in the NV3 emulation, I have a branch [0] that contains it all. Note that the improvement only kicks while running an L2 guest running at EL2 -- L1 will not see any improvement. But I get close to 40% improvements on L2, and 60% on L3 for the only workload that really matters (Debian Installer).

If all your nested guests are running this hack, then you can pass "kvm-arm.mode=nested,nvtge" on the host command line, and all the nested guests will opt in the accelerated behaviour.

If you like to live dangerously (or have a machine that does not have FEAT_HCX), you can instead pass 'kvm-arm.mode=nested,nvtge=force', and ERET will get the acceleration without even guest buy-in.

* From v1 [1]:

- Expanded generation of RESx and UNKN symbols to both Mapping and Fields qualifiers

- Dropped HFGIRT_EL2.ERET checks on ERET fast-path

- Fixed the NVHCR_EL2 trap handling

- Added FORCE_RESx() to HCRX_EL2 constraints

- Fixed ID_AA64MMFR4_EL1 sanitisation

- Added NV3 acceleration for TLBIOS

- Collected RBs from Joey, with thanks

[0] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/nv3
[1] https://lore.kernel.org/r/[email protected]

Marc Zyngier (28):
arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions
arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release
KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
KVM: arm64: Drop __HCRX_EL2_* masks
KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation
KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register
KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast
path
arm64: Add ARM64_HAS_NV2P1 capability
KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
KVM: arm64: Expose FEAT_NV2p1 to NV guests
arm64: Add FEAT_NV2p1 detection
arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
arm64: Add ARM64_HAS_NV3 capability
KVM: arm64: Split NV-specific exit fixups from the non-NV handling
KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
KVM: arm64: Add kvm_has_nv{2,3}() predicates
KVM: arm64: Make HCR_EL2 a non-VNCR register
KVM: arm64: Add sanitisation for NVHCR_EL2
KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
KVM: arm64: Add routing for NVHCR_EL2 trap
KVM: arm64: Add NVHCR_EL2 context switching
KVM: arm64: Engage NV3 ERET trap elision
KVM: arm64: Engage NV3 TLBI trap elision
KVM: arm64: Add FEAT_NV3 detection
KVM: arm64: Expose FEAT_NV3 to guests
arm64: Add override for ID_AA64MMFR4_EL1.NV_frac

 arch/arm64/include/asm/cpufeature.h        |  1 +
 arch/arm64/include/asm/kvm_arm.h           | 15 ------
 arch/arm64/include/asm/kvm_emulate.h       | 45 ++++++++++++++++-
 arch/arm64/include/asm/kvm_host.h          |  3 +-
 arch/arm64/include/asm/vncr_mapping.h      |  2 +-
 arch/arm64/kernel/cpufeature.c             | 18 ++++++-
 arch/arm64/kernel/image-vars.h             |  1 +
 arch/arm64/kernel/pi/idreg-override.c      | 10 ++++
 arch/arm64/kvm/arch_timer.c                | 10 +++-
 arch/arm64/kvm/config.c                    | 27 +++++++++-
 arch/arm64/kvm/emulate-nested.c            | 18 +++++--
 arch/arm64/kvm/hyp/include/hyp/switch.h    | 27 ++++++++--
 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++
 arch/arm64/kvm/hyp/vhe/switch.c            | 49 ++++++++++++++-----
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c         | 21 +++++---
 arch/arm64/kvm/nested.c                    | 14 +++++-
 arch/arm64/kvm/sys_regs.c                  | 57 ++++++++++++++++++++--
 arch/arm64/tools/cpucaps                   |  2 +
 arch/arm64/tools/gen-sysreg.awk            |  8 +--
 arch/arm64/tools/sysreg                    | 42 ++++++++++++++--
 20 files changed, 316 insertions(+), 65 deletions(-)
Signature · 1 line
2.47.3
author maintainer
#1 lore raw

PATCH v2 1/28 arm64: sysreg: Emit RESx/UNKN values for Mapping/Fields definitions

The sysreg file is using the Mapping or Fields qualifiers to indicate that a given encoding is only a mapping to a particular register (or an instance of a more generic register definition).

As a result, we don't output any definition, and instead expect the canonical definitions to be used.

This works rather well for individual fields, but creates problems for macros that refer to more generic classes of bits such as RESx.

Relax the above rule by emitting the RESx and UNKN values for Mapping and Fields qualifiers as well.

---
 arch/arm64/tools/gen-sysreg.awk | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/tools/gen-sysreg.awk +4 -4
index 86860ab672dc7..e099f7d3b2833 100755
@@ -228,7 +228,7 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
 }
 
 # Currently this is effectivey a comment, in future we may want to emit
-# defines for the fields.
+# defines for the fields. "Mapping" does emit the RESx/UNKN definitions.
 ($1 == "Fields" || $1 == "Mapping") && block_current() == "Sysreg" {
 	expect_fields(2)
 
@@ -239,9 +239,9 @@ $1 == "EndSysreg" && block_current() == "Sysreg" {
 	print ""
 
 	next_bit = -1
-	res0 = null
-	res1 = null
-	unkn = null
+	res0 = $2 "_RES0"
+	res1 = $2 "_RES1"
+	unkn = $2 "_UNKN"
 
 	next
 }
Signature · 1 line
2.47.3
author maintainer
#2 lore raw

PATCH v2 2/28 arm64: Update ID_AA64MMFR4_EL1 description to 2026-03 JSON release

In reply to Marc Zyngier

ID_AA64MMFR4_EL1 has gained a few fields and enum values in the past few months, so resync its definition with the 2026-03 JSON release.

---
 arch/arm64/tools/sysreg | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
arch/arm64/tools/sysreg +27 -3
index bc1788b1662b7..32e2f9856768b 100644
@@ -2386,17 +2386,40 @@ EndEnum
 EndSysreg
 
 Sysreg	ID_AA64MMFR4_EL1	3	0	0	7	4
-Res0	63:48
+UnsignedEnum	63:60	MTEFGT
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	59:56	SCRX
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	55:52	TEV
+	0b0000	NI
+	0b0001	IMP
+EndEnum
+UnsignedEnum	51:48	TPS
+	0b0000	VAL_0000
+	0b0001	VAL_0001
+	0b0010	VAL_0010
+EndEnum
 UnsignedEnum	47:44	SRMASK
 	0b0000	NI
 	0b0001	IMP
+	0b0010	SRMASK2
+EndEnum
+UnsignedEnum	43:40	TLBID
+	0b0000	NI
+	0b0001	IMP
 EndEnum
-Res0	43:40
 UnsignedEnum	39:36	E3DSE
 	0b0000	NI
 	0b0001	IMP
 EndEnum
-Res0	35:32
+UnsignedEnum	35:32	EAESR
+	0b0000	NI
+	0b0001	IMP
+EndEnum
 UnsignedEnum	31:28	RMEGDI
 	0b0000	NI
 	0b0001	IMP
@@ -2410,6 +2433,7 @@ UnsignedEnum	23:20	NV_frac
 	0b0000	NV_NV2
 	0b0001	NV2_ONLY
 	0b0010	NV2P1
+	0b0011	NV3
 EndEnum
 UnsignedEnum	19:16	FGWTE3
 	0b0000	NI
Signature · 1 line
2.47.3
author maintainer
#3 lore raw

PATCH v2 3/28 KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE

In reply to Marc Zyngier

The way we merge the guest-provided HCRX_EL2 value with the host's is bonkers. We try to make it look like the FGT registers by using positive and negative polarities for traps, but most of these bits are not strictly about trapping, as they actively change the way some architectural state is managed.

It would be far better to deal with these bits like we do for HCR_EL2, by enumerating the list of bits we don't allow the guest to override. This is simplified by the fact that HCRX_EL2 only affects EL1, and not EL2.

Re-jig the HCRX_EL2 handling with a macro that list the bits excluded from the merge (TMEA, PTTWI, EnIDCP128).

---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +20 -2
index 4bf624a49591d..8e5f492f39086 100644
@@ -325,6 +325,24 @@ static inline void __deactivate_traps_mpam(void)
 		write_sysreg_s(MPAMHCR_HOST_FLAGS, SYS_MPAMHCR_EL2);
 }
 
+/*
+ * Just like for HCR_EL2, we can't let the guest mess with some of the
+ * basics we rely on in HCRX_EL2. However, the major difference is that
+ * HCRX_EL2 only affects EL1, and never EL2 (sudden outburst of sanity, I
+ * guess). So it is always the guest inflicting it on its own guestx.
+ *
+ * Things we don't want to let the guest control are:
+ *
+ * - TMEA: That's for us to decide how an SEA is routed, not the guest.
+ *
+ * - PTTWI: Similarly, it is for us to decide whether Reduced Coherency for
+ *   the PTW is a thing. It really isn't.
+ *
+ * - EnIDCP128: We don't allow IMPDEF sysregs -- full stop.
+ */
+#define NV_HCRX_GUEST_EXCLUDE	(HCRX_EL2_TMEA	    | HCRX_EL2_PTTWI | \
+				 HCRX_EL2_EnIDCP128)
+
 static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);
@@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 		u64 hcrx = vcpu->arch.hcrx_el2;
 		if (is_nested_ctxt(vcpu)) {
 			u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
-			hcrx |= val & __HCRX_EL2_MASK;
-			hcrx &= ~(~val & __HCRX_EL2_nMASK);
+			hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
+			hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
 		}
 
 		ctxt_sys_reg(hctxt, HCRX_EL2) = read_sysreg_s(SYS_HCRX_EL2);
Signature · 1 line
2.47.3
Reviewed-by on v2 3/28
#4 lore raw

Re: [PATCH v2 03/28] KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE

Quoted 49 lines · Marc Zyngier

The way we merge the guest-provided HCRX_EL2 value with the host's is bonkers. We try to make it look like the FGT registers by using positive and negative polarities for traps, but most of these bits are not strictly about trapping, as they actively change the way some architectural state is managed.

It would be far better to deal with these bits like we do for HCR_EL2, by enumerating the list of bits we don't allow the guest to override. This is simplified by the fact that HCRX_EL2 only affects EL1, and not EL2.

Re-jig the HCRX_EL2 handling with a macro that list the bits excluded from the merge (TMEA, PTTWI, EnIDCP128).

---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +18 -0
index 4bf624a49591d..8e5f492f39086 100644
@@ -325,6 +325,24 @@ static inline void __deactivate_traps_mpam(void)
 		write_sysreg_s(MPAMHCR_HOST_FLAGS, SYS_MPAMHCR_EL2);
 }
 
+/*
+ * Just like for HCR_EL2, we can't let the guest mess with some of the
+ * basics we rely on in HCRX_EL2. However, the major difference is that
+ * HCRX_EL2 only affects EL1, and never EL2 (sudden outburst of sanity, I
+ * guess). So it is always the guest inflicting it on its own guestx.
+ *
+ * Things we don't want to let the guest control are:
+ *
+ * - TMEA: That's for us to decide how an SEA is routed, not the guest.
+ *
+ * - PTTWI: Similarly, it is for us to decide whether Reduced Coherency for
+ *   the PTW is a thing. It really isn't.
+ *
+ * - EnIDCP128: We don't allow IMPDEF sysregs -- full stop.
+ */
+#define NV_HCRX_GUEST_EXCLUDE	(HCRX_EL2_TMEA	    | HCRX_EL2_PTTWI | \
+				 HCRX_EL2_EnIDCP128)
+
 static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpu_context *hctxt = host_data_ptr(host_ctxt);

arch/arm64/kvm/hyp/include/hyp/switch.h static inline void __activate_traps_common(struct kvm_vcpu *vcpu) · line 372

Quoted 7 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +2 -2
@@ -350,8 +368,8 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu)
 		u64 hcrx = vcpu->arch.hcrx_el2;
 		if (is_nested_ctxt(vcpu)) {
 			u64 val = __vcpu_sys_reg(vcpu, HCRX_EL2);
-			hcrx |= val & __HCRX_EL2_MASK;
-			hcrx &= ~(~val & __HCRX_EL2_nMASK);
+			hcrx |= (val & ~NV_HCRX_GUEST_EXCLUDE);
+			hcrx &= ~(~val & ~NV_HCRX_GUEST_EXCLUDE);
Quoted 6 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +0 -1
 		}
 
 		ctxt_sys_reg(hctxt, HCRX_EL2) = read_sysreg_s(SYS_HCRX_EL2);
--

2.47.3

author maintainer
#5 lore raw

PATCH v2 4/28 KVM: arm64: Drop __HCRX_EL2_* masks

In reply to Marc Zyngier

The __HCRX_EL2_* masks are a leftover from a time where we didn't have much sanitisation for the system registers. Since we are now in a better place, rely on the existing checks to detect unhandled bits in HCRX_EL2.

---
 arch/arm64/include/asm/kvm_arm.h | 15 ---------------
 arch/arm64/kvm/config.c          |  5 +++--
 arch/arm64/kvm/emulate-nested.c  |  5 -----
 3 files changed, 3 insertions(+), 22 deletions(-)
arch/arm64/include/asm/kvm_arm.h +0 -15
index 3f9233b5a1308..f6cd851047947 100644
@@ -287,21 +287,6 @@
 				 GENMASK(19, 18) |	\
 				 GENMASK(15, 0))
 
-/*
- * Polarity masks for HCRX_EL2, limited to the bits that we know about
- * at this point in time. It doesn't mean that we actually *handle*
- * them, but that at least those that are not advertised to a guest
- * will be RES0 for that guest.
- */
-#define __HCRX_EL2_MASK		(BIT_ULL(6))
-#define __HCRX_EL2_nMASK	(GENMASK_ULL(24, 14) | \
-				 GENMASK_ULL(11, 7)  | \
-				 GENMASK_ULL(5, 0))
-#define __HCRX_EL2_RES0		~(__HCRX_EL2_nMASK | __HCRX_EL2_MASK)
-#define __HCRX_EL2_RES1		~(__HCRX_EL2_nMASK | \
-				  __HCRX_EL2_MASK  | \
-				  __HCRX_EL2_RES0)
-
 /* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */
 #define HPFAR_MASK	(~UL(0xf))
 /*
arch/arm64/kvm/config.c +3 -2
index 0622162b089e5..96ddcefc71eb5 100644
@@ -930,10 +930,12 @@ static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
 	NEEDS_FEAT(HCRX_EL2_EnASR, FEAT_LS64_V),
 	NEEDS_FEAT(HCRX_EL2_EnALS, FEAT_LS64),
 	NEEDS_FEAT(HCRX_EL2_EnAS0, FEAT_LS64_ACCDATA),
+	FORCE_RES0(HCRX_EL2_RES0),
+	FORCE_RES1(HCRX_EL2_RES1),
 };
 
 
-static const DECLARE_FEAT_MAP(hcrx_desc, __HCRX_EL2,
+static const DECLARE_FEAT_MAP(hcrx_desc, HCRX_EL2,
 			      hcrx_feat_map, FEAT_HCX);
 
 static const struct reg_bits_to_feat_map hcr_feat_map[] = {
@@ -1579,7 +1581,6 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
 		break;
 	case HCRX_EL2:
 		resx = compute_reg_resx_bits(kvm, &hcrx_desc, 0, 0);
-		resx.res1 |= __HCRX_EL2_RES1;
 		break;
 	case HCR_EL2:
 		resx = compute_reg_resx_bits(kvm, &hcr_desc, 0, 0);
arch/arm64/kvm/emulate-nested.c +0 -5
index 3c82f392845d1..b7f3d86a94031 100644
@@ -2320,7 +2320,6 @@ int __init populate_nv_trap_config(void)
 	BUILD_BUG_ON(__NR_CGT_GROUP_IDS__ > BIT(TC_CGT_BITS));
 	BUILD_BUG_ON(__NR_FGT_GROUP_IDS__ > BIT(TC_FGT_BITS));
 	BUILD_BUG_ON(__NR_FG_FILTER_IDS__ > BIT(TC_FGF_BITS));
-	BUILD_BUG_ON(__HCRX_EL2_MASK & __HCRX_EL2_nMASK);
 
 	for (int i = 0; i < ARRAY_SIZE(encoding_to_cgt); i++) {
 		const struct encoding_to_trap_config *cgt = &encoding_to_cgt[i];
@@ -2346,10 +2345,6 @@ int __init populate_nv_trap_config(void)
 		}
 	}
 
-	if (__HCRX_EL2_RES0 != HCRX_EL2_RES0)
-		kvm_info("Sanitised HCR_EL2_RES0 = %016llx, expecting %016llx\n",
-			 __HCRX_EL2_RES0, HCRX_EL2_RES0);
-
 	kvm_info("nv: %ld coarse grained trap handlers\n",
 		 ARRAY_SIZE(encoding_to_cgt));
 
Signature · 1 line
2.47.3
author maintainer
#6 lore raw

PATCH v2 5/28 KVM: arm64: Plumb HCRX_EL2.SRMASKEn in HCRX_EL2 sanitisation

In reply to Marc Zyngier

HCRX_EL2.SRMASKEn is a new bit enabling FEAT_SRMASK for a guest. We don't plan to support it any time soon, but it doesn't hurt to actively document it, specially as we are going to add more bits we actually care about.

---
 arch/arm64/kvm/config.c | 1 +
 arch/arm64/tools/sysreg | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)
arch/arm64/kvm/config.c +1 -0
index 96ddcefc71eb5..c61d62b796023 100644
@@ -904,6 +904,7 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
 
 
 static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
+	NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
 	NEEDS_FEAT(HCRX_EL2_PACMEn, feat_pauth_lr),
 	NEEDS_FEAT(HCRX_EL2_EnFPM, FEAT_FPMR),
 	NEEDS_FEAT(HCRX_EL2_GCSEn, FEAT_GCS),
arch/arm64/tools/sysreg +3 -1
index 32e2f9856768b..c6e8117a6f9cd 100644
@@ -4545,7 +4545,9 @@ Fields	ZCR_ELx
 EndSysreg
 
 Sysreg	HCRX_EL2	3	4	1	2	2
-Res0	63:25
+Res0	63:27
+Field	26	SRMASKEn
+Res0	25
 Field	24	PACMEn
 Field	23	EnFPM
 Field	22	GCSEn
Signature · 1 line
2.47.3
author maintainer
#7 lore raw

PATCH v2 6/28 KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register

In reply to Marc Zyngier

It may not be obvious unless you look at it closely, but CPTR_EL2 is treated very differently from other registers. It is one the registers that, despite looking very similar between EL1 and EL2 when E2H==1, have RES0 bits that get in the way.

Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL, just like CNTHCTL_EL2 (and for the same reasons). This makes it possible to use vcpu_read_sys_reg() with it, and will be necessary once we support FEAT_NV2P1.

---
 arch/arm64/include/asm/kvm_emulate.h |  2 +-
 arch/arm64/kvm/sys_regs.c            | 20 ++++++++++++++++++--
 2 files changed, 19 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +1 -1
index 5bf3d7e1d92c7..9831166695186 100644
@@ -617,7 +617,7 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
  */
 static inline u64 vcpu_sanitised_cptr_el2(const struct kvm_vcpu *vcpu)
 {
-	u64 cptr = __vcpu_sys_reg(vcpu, CPTR_EL2);
+	u64 cptr = vcpu_read_sys_reg(vcpu, CPTR_EL2);
 
 	if (!vcpu_el2_e2h_is_set(vcpu))
 		cptr = translate_cptr_el2_to_cpacr_el1(cptr);
arch/arm64/kvm/sys_regs.c +18 -2
index 5d5c579d45790..6b47d936efb32 100644
@@ -183,8 +183,6 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 	switch (reg) {
 		MAPPED_EL2_SYSREG(SCTLR_EL2,   SCTLR_EL1,
 				  translate_sctlr_el2_to_sctlr_el1	     );
-		MAPPED_EL2_SYSREG(CPTR_EL2,    CPACR_EL1,
-				  translate_cptr_el2_to_cpacr_el1	     );
 		MAPPED_EL2_SYSREG(TTBR0_EL2,   TTBR0_EL1,
 				  translate_ttbr0_el2_to_ttbr0_el1	     );
 		MAPPED_EL2_SYSREG(TTBR1_EL2,   TTBR1_EL1,   NULL	     );
@@ -210,6 +208,19 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 		loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
 			    SR_LOC_SPECIAL : SR_LOC_MEMORY);
 		break;
+	case CPTR_EL2:
+		/*
+		 * CPTR_EL2 is just as special, and needs a certain amount
+		 * of handholding. It always lives in memory, due to being
+		 * heavily trapped thanks to CPACR_EL1.TCPAC being RES0.
+		 * FEAT_NV2p1 fixes this.
+		 */
+		locate_mapped_el2_register(vcpu, CPTR_EL2, CPACR_EL1,
+					   translate_cptr_el2_to_cpacr_el1,
+					   loc);
+		if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+			loc->loc = SR_LOC_SPECIAL;
+		break;
 	default:
 		loc->loc = locate_direct_register(vcpu, reg);
 	}
@@ -314,6 +325,8 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val &= CNTKCTL_VALID_BITS;
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
+		case CPTR_EL2:
+			return __vcpu_sys_reg(vcpu, reg);
 		default:
 			WARN_ON_ONCE(1);
 		}
@@ -359,6 +372,9 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, enum vcpu_sysreg reg)
 			 */
 			write_sysreg_el1(val, SYS_CNTKCTL);
 			break;
+		case CPTR_EL2:
+			write_sysreg_el1(val, SYS_CPACR);
+			break;
 		default:
 			WARN_ON_ONCE(1);
 		}
Signature · 1 line
2.47.3
Reviewed-by on v2 6/28
#8 lore raw

Re: [PATCH v2 06/28] KVM: arm64: Classify CPTR_EL2 as a SR_LOC_SPECIAL register

Quoted 73 lines · Marc Zyngier

It may not be obvious unless you look at it closely, but CPTR_EL2 is treated very differently from other registers. It is one the registers that, despite looking very similar between EL1 and EL2 when E2H==1, have RES0 bits that get in the way.

Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL, just like CNTHCTL_EL2 (and for the same reasons). This makes it possible to use vcpu_read_sys_reg() with it, and will be necessary once we support FEAT_NV2P1.

---
 arch/arm64/include/asm/kvm_emulate.h |  2 +-
 arch/arm64/kvm/sys_regs.c            | 20 ++++++++++++++++++--
 2 files changed, 19 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +1 -1
index 5bf3d7e1d92c7..9831166695186 100644
@@ -617,7 +617,7 @@ static __always_inline void kvm_incr_pc(struct kvm_vcpu *vcpu)
  */
 static inline u64 vcpu_sanitised_cptr_el2(const struct kvm_vcpu *vcpu)
 {
-	u64 cptr = __vcpu_sys_reg(vcpu, CPTR_EL2);
+	u64 cptr = vcpu_read_sys_reg(vcpu, CPTR_EL2);
 
 	if (!vcpu_el2_e2h_is_set(vcpu))
 		cptr = translate_cptr_el2_to_cpacr_el1(cptr);
arch/arm64/kvm/sys_regs.c +15 -2
index 5d5c579d45790..6b47d936efb32 100644
@@ -183,8 +183,6 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 	switch (reg) {
 		MAPPED_EL2_SYSREG(SCTLR_EL2,   SCTLR_EL1,
 				  translate_sctlr_el2_to_sctlr_el1	     );
-		MAPPED_EL2_SYSREG(CPTR_EL2,    CPACR_EL1,
-				  translate_cptr_el2_to_cpacr_el1	     );
 		MAPPED_EL2_SYSREG(TTBR0_EL2,   TTBR0_EL1,
 				  translate_ttbr0_el2_to_ttbr0_el1	     );
 		MAPPED_EL2_SYSREG(TTBR1_EL2,   TTBR1_EL1,   NULL	     );
@@ -210,6 +208,19 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 		loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
 			    SR_LOC_SPECIAL : SR_LOC_MEMORY);
 		break;
+	case CPTR_EL2:
+		/*
+		 * CPTR_EL2 is just as special, and needs a certain amount
+		 * of handholding. It always lives in memory, due to being
+		 * heavily trapped thanks to CPACR_EL1.TCPAC being RES0.
+		 * FEAT_NV2p1 fixes this.
+		 */
+		locate_mapped_el2_register(vcpu, CPTR_EL2, CPACR_EL1,
+					   translate_cptr_el2_to_cpacr_el1,
+					   loc);
+		if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
+			loc->loc = SR_LOC_SPECIAL;
+		break;
 	default:
 		loc->loc = locate_direct_register(vcpu, reg);
 	}
@@ -314,6 +325,8 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val &= CNTKCTL_VALID_BITS;
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
+		case CPTR_EL2:
+			return __vcpu_sys_reg(vcpu, reg);
 		default:
 			WARN_ON_ONCE(1);
 		}

arch/arm64/kvm/sys_regs.c void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, enum vcpu_sysreg reg) · line 377

Quoted 6 lines
arch/arm64/kvm/sys_regs.c +3 -0
@@ -359,6 +372,9 @@ void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, enum vcpu_sysreg reg)
 			 */
 			write_sysreg_el1(val, SYS_CNTKCTL);
 			break;
+		case CPTR_EL2:
+			write_sysreg_el1(val, SYS_CPACR);
+			break;
Quoted 6 lines
arch/arm64/kvm/sys_regs.c +0 -1
 		default:
 			WARN_ON_ONCE(1);
 		}
--

2.47.3

author maintainer
#9 lore raw

PATCH v2 7/28 KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path

In reply to Marc Zyngier

We currently avoid using the ERET fast path if the guest has HCR_EL2.NV set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.

Similarly, evaluating HFGITR_EL2.ERET makes little sense, as this only applies to the nested context, while the ERET fast-path is purely for the benefit of L1.

Replace these bizarre checks with is_nested_ctxt() which makes a lot more sense: if we are running an L2, the ERET trap must go to L1.

Fixes dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
---
 arch/arm64/kvm/hyp/vhe/switch.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +3 -6
index bbe9cebd3d9d5..a83be345f8450 100644
@@ -344,13 +344,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * if this is a VHE guest hypervisor returning to its own
 	 * userspace, or the hypervisor performing a local exception
 	 * return. No need to save/restore registers, no need to
-	 * switch S2 MMU. Just do the canonical ERET.
-	 *
-	 * Unless the trap has to be forwarded further down the line,
-	 * of course...
+	 * switch S2 MMU. Just do the canonical ERET unlesss we are in
+	 * nested context.
 	 */
-	if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV) ||
-	    (__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET))
+	if (is_nested_ctxt(vcpu))
 		return false;
 
 	spsr = read_sysreg_el1(SYS_SPSR);
Signature · 1 line
2.47.3
maintainer
#10 lore raw

Re: [PATCH v2 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path

Hey,

Quoted 21 lines · Marc Zyngier

We currently avoid using the ERET fast path if the guest has HCR_EL2.NV set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.

Similarly, evaluating HFGITR_EL2.ERET makes little sense, as this only applies to the nested context, while the ERET fast-path is purely for the benefit of L1.

Replace these bizarre checks with is_nested_ctxt() which makes a lot more sense: if we are running an L2, the ERET trap must go to L1.

Fixes dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
---
 arch/arm64/kvm/hyp/vhe/switch.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +0 -0
index bbe9cebd3d9d5..a83be345f8450 100644

arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 349

Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +2 -4
@@ -344,13 +344,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * if this is a VHE guest hypervisor returning to its own
 	 * userspace, or the hypervisor performing a local exception
 	 * return. No need to save/restore registers, no need to
-	 * switch S2 MMU. Just do the canonical ERET.
-	 *
-	 * Unless the trap has to be forwarded further down the line,
-	 * of course...
+	 * switch S2 MMU. Just do the canonical ERET unlesss we are in
+	 * nested context.
 	 */

typo: unless

I feel like it wouldn't hurt to add a bit more detail to the comment here, that L0 KVM doesn't use ERET traps in the nested context. Otherwise it isn't clear why these always belong to the L1.

Thanks,
Oliver

Quoted 9 lines
arch/arm64/kvm/hyp/vhe/switch.c +1 -2
-	if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_NV) ||
-	    (__vcpu_sys_reg(vcpu, HFGITR_EL2) & HFGITR_EL2_ERET))
+	if (is_nested_ctxt(vcpu))
 		return false;
 
 	spsr = read_sysreg_el1(SYS_SPSR);
Signature · 1 line
2.47.3
author maintainer
#11 lore raw

Re: [PATCH v2 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path

Quoted 38 lines · Oliver Upton

Hey,

Quoted 21 lines · Marc Zyngier

We currently avoid using the ERET fast path if the guest has HCR_EL2.NV set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.

Similarly, evaluating HFGITR_EL2.ERET makes little sense, as this only applies to the nested context, while the ERET fast-path is purely for the benefit of L1.

Replace these bizarre checks with is_nested_ctxt() which makes a lot more sense: if we are running an L2, the ERET trap must go to L1.

Fixes dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
---
 arch/arm64/kvm/hyp/vhe/switch.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +0 -0
index bbe9cebd3d9d5..a83be345f8450 100644

arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 349

Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +2 -4
@@ -344,13 +344,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * if this is a VHE guest hypervisor returning to its own
 	 * userspace, or the hypervisor performing a local exception
 	 * return. No need to save/restore registers, no need to
-	 * switch S2 MMU. Just do the canonical ERET.
-	 *
-	 * Unless the trap has to be forwarded further down the line,
-	 * of course...
+	 * switch S2 MMU. Just do the canonical ERET unlesss we are in
+	 * nested context.
 	 */

typo: unless

Quoted 3 lines

I feel like it wouldn't hurt to add a bit more detail to the comment here, that L0 KVM doesn't use ERET traps in the nested context. Otherwise it isn't clear why these always belong to the L1.

We already have a tiny bit of information in kvm_handle_eret(), but that's obviously a bit far down the line. Something like this?

arch/arm64/kvm/hyp/vhe/switch.c +6 -1
index a83be345f8450..684cebf23aa0e 100644
@@ -344,8 +344,13 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * if this is a VHE guest hypervisor returning to its own
 	 * userspace, or the hypervisor performing a local exception
 	 * return. No need to save/restore registers, no need to
-	 * switch S2 MMU. Just do the canonical ERET unlesss we are in
+	 * switch S2 MMU. Just do the canonical ERET unless we are in
 	 * nested context.
+	 *
+	 * Note that this is made possible because KVM itself never traps
+	 * ERET when running an L2. The consequence is that any ERET trap is
+	 * the result of HCR_EL2 or HFGITR_EL2 programming by L1 for its own
+	 * guest, and the exception must be forwarded to L1.
 	 */
 	if (is_nested_ctxt(vcpu))
 		return false;

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#12 lore raw

Re: [PATCH v2 07/28] KVM: arm64: Don't evaluate HCR_EL2.NV nor HFGITR_EL2.ERET on ERET fast path

Quoted 56 lines · Marc Zyngier
Quoted 38 lines · Oliver Upton

Hey,

Quoted 21 lines · Marc Zyngier

We currently avoid using the ERET fast path if the guest has HCR_EL2.NV set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.

Similarly, evaluating HFGITR_EL2.ERET makes little sense, as this only applies to the nested context, while the ERET fast-path is purely for the benefit of L1.

Replace these bizarre checks with is_nested_ctxt() which makes a lot more sense: if we are running an L2, the ERET trap must go to L1.

Fixes dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
---
 arch/arm64/kvm/hyp/vhe/switch.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +0 -0
index bbe9cebd3d9d5..a83be345f8450 100644

arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 349

Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +2 -4
@@ -344,13 +344,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * if this is a VHE guest hypervisor returning to its own
 	 * userspace, or the hypervisor performing a local exception
 	 * return. No need to save/restore registers, no need to
-	 * switch S2 MMU. Just do the canonical ERET.
-	 *
-	 * Unless the trap has to be forwarded further down the line,
-	 * of course...
+	 * switch S2 MMU. Just do the canonical ERET unlesss we are in
+	 * nested context.
 	 */

typo: unless

Quoted 3 lines

I feel like it wouldn't hurt to add a bit more detail to the comment here, that L0 KVM doesn't use ERET traps in the nested context. Otherwise it isn't clear why these always belong to the L1.

We already have a tiny bit of information in kvm_handle_eret(), but that's obviously a bit far down the line. Something like this?

arch/arm64/kvm/hyp/vhe/switch.c +0 -1
index a83be345f8450..684cebf23aa0e 100644
@@ -344,8 +344,13 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	 * if this is a VHE guest hypervisor returning to its own
 	 * userspace, or the hypervisor performing a local exception
 	 * return. No need to save/restore registers, no need to
-	 * switch S2 MMU. Just do the canonical ERET unlesss we are in

arch/arm64/kvm/hyp/vhe/switch.c static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) · line 356

Quoted 10 lines
arch/arm64/kvm/hyp/vhe/switch.c +6 -0
+	 * switch S2 MMU. Just do the canonical ERET unless we are in
 	 * nested context.
+	 *
+	 * Note that this is made possible because KVM itself never traps
+	 * ERET when running an L2. The consequence is that any ERET trap is
+	 * the result of HCR_EL2 or HFGITR_EL2 programming by L1 for its own
+	 * guest, and the exception must be forwarded to L1.
 	 */
 	if (is_nested_ctxt(vcpu))
 		return false;

Works for me!

Thanks,
Oliver

author maintainer
#13 lore raw

PATCH v2 8/28 arm64: Add ARM64_HAS_NV2P1 capability

In reply to Marc Zyngier

As we're about to deal with FEAT_NV2P1, add a new capability that will be used to key any support for it.

---
 arch/arm64/tools/cpucaps | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/tools/cpucaps +1 -0
index 9b85a84f6fd49..242dc211d8efa 100644
@@ -51,6 +51,7 @@ HAS_LS64_V
 HAS_LSUI
 HAS_MOPS
 HAS_NESTED_VIRT
+HAS_NV2P1
 HAS_BBML2_NOABORT
 HAS_PAN
 HAS_PMUV3
Signature · 1 line
2.47.3
author maintainer
#14 lore raw

PATCH v2 9/28 KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present

In reply to Marc Zyngier

With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.

Prevent such trapping and context switch CPACTR_EL1 in NV contexts when NV2P1 is present.

---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 5 +++--
 arch/arm64/kvm/hyp/vhe/switch.c         | 3 +++
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c      | 8 +++++---
 arch/arm64/kvm/sys_regs.c               | 5 ++++-
 4 files changed, 15 insertions(+), 6 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +3 -2
index 8e5f492f39086..7b27296c94607 100644
@@ -108,9 +108,10 @@ static inline void __activate_cptr_traps_vhe(struct kvm_vcpu *vcpu)
 	 * The architecture is a bit crap (what a surprise): an EL2 guest
 	 * writing to CPTR_EL2 via CPACR_EL1 can't set any of TCPAC or TTA,
 	 * as they are RES0 in the guest's view. To work around it, trap the
-	 * sucker using the very same bit it can't set...
+	 * sucker using the very same bit it can't set. FEAT_NV2p1 fixes it.
 	 */
-	if (vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu))
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1) &&
+	    vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu))
 		val |= CPTR_EL2_TCPAC;
 
 	/*
arch/arm64/kvm/hyp/vhe/switch.c +3 -0
index a83be345f8450..8460a8a6745b3 100644
@@ -438,6 +438,9 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code)
 	u64 esr = kvm_vcpu_get_esr(vcpu);
 	int rt;
 
+	if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+		return false;
+
 	if (!is_hyp_ctxt(vcpu) || esr_sys64_to_sysreg(esr) != SYS_CPACR_EL1)
 		return false;
 
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +5 -3
index be685b63e8cf2..6f0f046e4ca4e 100644
@@ -42,10 +42,12 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
 		u64 val;
 
 		/*
-		 * We don't save CPTR_EL2, as accesses to CPACR_EL1
-		 * are always trapped, ensuring that the in-memory
-		 * copy is always up-to-date. A small blessing...
+		 * Without FEAT_NV2p1, we don't save CPTR_EL2, as accesses
+		 * to CPACR_EL1 are always trapped, ensuring that the
+		 * in-memory copy is always up-to-date. A small blessing...
 		 */
+		if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+			__vcpu_assign_sys_reg(vcpu, CPTR_EL2, read_sysreg_el1(SYS_CPACR));
 		__vcpu_assign_sys_reg(vcpu, SCTLR_EL2,	 read_sysreg_el1(SYS_SCTLR));
 		__vcpu_assign_sys_reg(vcpu, TTBR0_EL2,	 read_sysreg_el1(SYS_TTBR0));
 		__vcpu_assign_sys_reg(vcpu, TTBR1_EL2,	 read_sysreg_el1(SYS_TTBR1));
arch/arm64/kvm/sys_regs.c +4 -1
index 6b47d936efb32..1dfc1f88bec82 100644
@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
 		case CPTR_EL2:
-			return __vcpu_sys_reg(vcpu, reg);
+			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+				return read_sysreg_el1(SYS_CPACR);
+			else
+				return __vcpu_sys_reg(vcpu, reg);
 		default:
 			WARN_ON_ONCE(1);
 		}
Signature · 1 line
2.47.3
#15 lore raw

Re: [PATCH v2 09/28] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Nested trap configuration for CPTR_EL2 is evaluated using a stale memory backing store under…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Nested trap configuration for CPTR_EL2 is evaluated using a stale memory backing store under FEAT_NV2P1, breaking nested virtualization isolation. --

--- Patch [9]: [PATCH v2 09/28] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present --- commit c67129a02beeb5d291611085a5d2f889ac9d97e8
Author: Marc Zyngier <[email protected]>

KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present

This patch relaxes the handling of CPTR_EL2 accesses via CPACR_EL1 when FEAT_NV2P1 is present, stopping the trap of CPACR_EL1. It modifies the read path to use the hardware SYS_CPACR_EL1 register instead of the memory backing store for CPTR_EL2.

[ ... ]

arch/arm64/kvm/sys_regs.c u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) · line 332

Quoted 13 lines
arch/arm64/kvm/sys_regs.c +4 -1
index 6b47d936efb32..1dfc1f88bec82 100644
@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
 		case CPTR_EL2:
-			return __vcpu_sys_reg(vcpu, reg);
+			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+				return read_sysreg_el1(SYS_CPACR);
+			else
+				return __vcpu_sys_reg(vcpu, reg);

[Severity: High]
Since this changes CPTR_EL2 to be read directly from the hardware register SYS_CPACR under FEAT_NV2P1 (as the memory backing store can be stale without trapping), will direct readers of the backing store now read stale values?

For example, in arch/arm64/kvm/emulate-nested.c:get_behaviour():

    val = __vcpu_sys_reg(vcpu, tb->index);

And in arch/arm64/kvm/emulate-nested.c:check_cptr_tta():

    u64 val = __vcpu_sys_reg(vcpu, CPTR_EL2);

Because the L1 guest writes to CPACR_EL1 natively without trapping, the backing store is not synchronously updated. If an L2 guest traps to host KVM, wouldn't these functions evaluate trap configurations using the stale memory backing store, potentially bypassing the L1 hypervisor's trap configurations?

Quoted 3 lines
arch/arm64/kvm/sys_regs.c +0 -0
 		default:
 			WARN_ON_ONCE(1);
 		}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
author maintainer
#16 lore raw

Re: [PATCH v2 09/28] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present

Quoted 37 lines · On Tue, 14 Jul 2026 11:12:36 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Nested trap configuration for CPTR_EL2 is evaluated using a stale memory backing store under FEAT_NV2P1, breaking nested virtualization isolation. --

--- Patch [9]: [PATCH v2 09/28] KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present --- commit c67129a02beeb5d291611085a5d2f889ac9d97e8
Author: Marc Zyngier <[email protected]>

KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present

This patch relaxes the handling of CPTR_EL2 accesses via CPACR_EL1 when FEAT_NV2P1 is present, stopping the trap of CPACR_EL1. It modifies the read path to use the hardware SYS_CPACR_EL1 register instead of the memory backing store for CPTR_EL2.

[ ... ]

arch/arm64/kvm/sys_regs.c u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) · line 332

Quoted 13 lines
arch/arm64/kvm/sys_regs.c +4 -1
index 6b47d936efb32..1dfc1f88bec82 100644
@@ -326,7 +326,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
 			return val;
 		case CPTR_EL2:
-			return __vcpu_sys_reg(vcpu, reg);
+			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+				return read_sysreg_el1(SYS_CPACR);
+			else
+				return __vcpu_sys_reg(vcpu, reg);

[Severity: High]
Since this changes CPTR_EL2 to be read directly from the hardware register SYS_CPACR under FEAT_NV2P1 (as the memory backing store can be stale without trapping), will direct readers of the backing store now read stale values?

Quoted 3 lines

For example, in arch/arm64/kvm/emulate-nested.c:get_behaviour():

    val = __vcpu_sys_reg(vcpu, tb->index);

get_behaviour() is only called when a trap from L2 needs to be routed to L1. Therefore, the L1 registers cannot be resident at this point, and looking for them in memory is the right thing to do.

Quoted 10 lines

And in arch/arm64/kvm/emulate-nested.c:check_cptr_tta():

    u64 val = __vcpu_sys_reg(vcpu, CPTR_EL2);

Because the L1 guest writes to CPACR_EL1 natively without trapping, the backing store is not synchronously updated. If an L2 guest traps to host KVM, wouldn't these functions evaluate trap configurations using the stale memory backing store, potentially bypassing the L1 hypervisor's trap configurations?

If running an L2, L1's view of CPTR_EL2 (which it accessed as CPACR_EL1) cannot be resident. Where would L2's accesses to CPACR_EL1 go otherwise?

You don't understand NV.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#17 lore raw

PATCH v2 10/28 KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

In reply to Marc Zyngier

With NV2p1, it is no longer necessary to use the split approach where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1, and we can treat the CNTKCTL_EL1 accessor as if it was "normal".

Key the special casing on FEAT_NV2P1 not being implemented.

---
 arch/arm64/kvm/arch_timer.c        | 10 ++++++++--
 arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 13 ++++++++++---
 arch/arm64/kvm/sys_regs.c          |  6 ++++--
 3 files changed, 22 insertions(+), 7 deletions(-)
arch/arm64/kvm/arch_timer.c +8 -2
index 4155fe89b58a1..db60facad9f3c 100644
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
 	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
 
-	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
-	sysreg_clear_set(cnthctl_el2, clr, set);
+	/*
+	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+	 */
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+		sysreg_clear_set(cnthctl_el2, clr, set);
+	else
+		sysreg_clear_set(cntkctl_el1, clr, set);
 }
 
 void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +10 -3
index 6f0f046e4ca4e..0c4ef1ce32ae7 100644
@@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
 		 * The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where
 		 * the interesting CNTHCTL_EL2 bits live. So preserve these
 		 * bits when reading back the guest-visible value.
+		 *
+		 * While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV
+		 * even more broken than it already was with NV2.
 		 */
 		val = read_sysreg_el1(SYS_CNTKCTL);
-		val &= CNTKCTL_VALID_BITS;
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+			val &= CNTKCTL_VALID_BITS;
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		} else {
+			__vcpu_assign_sys_reg(vcpu, CNTHCTL_EL2, val);
+		}
 	}
 
 	__vcpu_assign_sys_reg(vcpu, SP_EL2,	 read_sysreg(sp_el1));
arch/arm64/kvm/sys_regs.c +4 -2
index 1dfc1f88bec82..9439c5b2b1fe8 100644
@@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 		switch (reg) {
 		case CNTHCTL_EL2:
 			val = read_sysreg_el1(SYS_CNTKCTL);
-			val &= CNTKCTL_VALID_BITS;
-			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+				val &= CNTKCTL_VALID_BITS;
+				val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			}
 			return val;
 		case CPTR_EL2:
 			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
Signature · 1 line
2.47.3
#18 lore raw

Re: [PATCH v2 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Removing the split-state preservation for `CNTHCTL_EL2` on FEAT_NV2p1 systems permanently corrupts…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Removing the split-state preservation for CNTHCTL_EL2 on FEAT_NV2p1 systems permanently corrupts the guest's in-memory ECV bit and leaks host-specific trap overrides to userspace. --

--- Patch [10]: [PATCH v2 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present --- commit e272e43dc6ce85d3337902d78430aad6bf5b83fc
Author: Marc Zyngier <[email protected]>

KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

With NV2p1, it is no longer necessary to use the split approach where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1, and we can treat the CNTKCTL_EL1 accessor as if it was "normal".

Quoted 4 lines
arch/arm64/kvm/arch_timer.c +0 -0
index 4155fe89b58a1..db60facad9f3c 100644

[ ... ]

Quoted 15 lines
arch/arm64/kvm/arch_timer.c +8 -2
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
 	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
 
-	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
-	sysreg_clear_set(cnthctl_el2, clr, set);
+	/*
+	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+	 */
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+		sysreg_clear_set(cnthctl_el2, clr, set);
+	else
+		sysreg_clear_set(cntkctl_el1, clr, set);
 }

[ ... ]

Quoted 4 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +0 -0
index 6f0f046e4ca4e..0c4ef1ce32ae7 100644

[ ... ]

Quoted 9 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +3 -0
@@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
 		 * The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where
 		 * the interesting CNTHCTL_EL2 bits live. So preserve these
 		 * bits when reading back the guest-visible value.
+		 *
+		 * While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV
+		 * even more broken than it already was with NV2.
 		 */
 		val = read_sysreg_el1(SYS_CNTKCTL);

arch/arm64/kvm/hyp/vhe/sysreg-sr.c static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu) · line 83

Quoted 10 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +7 -3
-		val &= CNTKCTL_VALID_BITS;
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+			val &= CNTKCTL_VALID_BITS;
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		} else {
+			__vcpu_assign_sys_reg(vcpu, CNTHCTL_EL2, val);
+		}

[Severity: High]
Does removing the split-state preservation for CNTHCTL_EL2 unconditionally overwrite the guest's in-memory register state?

Since timer_set_traps() applies host-specific overrides to CNTKCTL_EL1 on the host side, calling __vcpu_assign_sys_reg() here appears to blindly copy those host-specific traps directly into the guest's architectural state on exit.

Furthermore, as the comment above notes that NV2p1 makes the ECV bit even more broken, won't this assignment corrupt the guest's ECV state as well?

Quoted 3 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +0 -0
 	}
 
 	__vcpu_assign_sys_reg(vcpu, SP_EL2,	 read_sysreg(sp_el1));

[ ... ]

Quoted 4 lines
arch/arm64/kvm/sys_regs.c +0 -0
index 1dfc1f88bec82..9439c5b2b1fe8 100644

[ ... ]

arch/arm64/kvm/sys_regs.c u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) · line 329

Quoted 11 lines
arch/arm64/kvm/sys_regs.c +4 -2
@@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 		switch (reg) {
 		case CNTHCTL_EL2:
 			val = read_sysreg_el1(SYS_CNTKCTL);
-			val &= CNTKCTL_VALID_BITS;
-			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+				val &= CNTKCTL_VALID_BITS;
+				val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			}
 			return val;

[Severity: High]
Does this directly return the host-polluted physical register to userspace during KVM_GET_ONE_REG?

Because the host-specific overrides from timer_set_traps() are present in CNTKCTL_EL1, skipping the bitmasking on ARM64_HAS_NV2P1 systems seems to leak these hypervisor-specific configurations to userspace. This could break live migration by restoring an altered execution environment.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
author maintainer
#19 lore raw

Re: [PATCH v2 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

Quoted 62 lines · On Tue, 14 Jul 2026 10:53:49 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Removing the split-state preservation for CNTHCTL_EL2 on FEAT_NV2p1 systems permanently corrupts the guest's in-memory ECV bit and leaks host-specific trap overrides to userspace. --

--- Patch [10]: [PATCH v2 10/28] KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present --- commit e272e43dc6ce85d3337902d78430aad6bf5b83fc
Author: Marc Zyngier <[email protected]>

KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

With NV2p1, it is no longer necessary to use the split approach where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1, and we can treat the CNTKCTL_EL1 accessor as if it was "normal".

Quoted 4 lines
arch/arm64/kvm/arch_timer.c +0 -0
index 4155fe89b58a1..db60facad9f3c 100644

[ ... ]

Quoted 15 lines
arch/arm64/kvm/arch_timer.c +8 -2
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
 	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
 
-	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
-	sysreg_clear_set(cnthctl_el2, clr, set);
+	/*
+	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+	 */
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+		sysreg_clear_set(cnthctl_el2, clr, set);
+	else
+		sysreg_clear_set(cntkctl_el1, clr, set);
 }

[ ... ]

Quoted 4 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +0 -0
index 6f0f046e4ca4e..0c4ef1ce32ae7 100644

[ ... ]

Quoted 15 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +6 -3
@@ -69,11 +69,18 @@ static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu)
 		 * The EL1 view of CNTKCTL_EL1 has a bunch of RES0 bits where
 		 * the interesting CNTHCTL_EL2 bits live. So preserve these
 		 * bits when reading back the guest-visible value.
+		 *
+		 * While NV2p1 fixes some of that, it makes CNTHCTL_EL2.ECV
+		 * even more broken than it already was with NV2.
 		 */
 		val = read_sysreg_el1(SYS_CNTKCTL);
-		val &= CNTKCTL_VALID_BITS;
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
-		__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+			val &= CNTKCTL_VALID_BITS;
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, &=, ~CNTKCTL_VALID_BITS);
Quoted 8 lines

arch/arm64/kvm/hyp/vhe/sysreg-sr.c static void __sysreg_save_vel2_state(struct kvm_vcpu *vcpu) · line 83

Quoted 4 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +4 -0
+			__vcpu_rmw_sys_reg(vcpu, CNTHCTL_EL2, |=, val);
+		} else {
+			__vcpu_assign_sys_reg(vcpu, CNTHCTL_EL2, val);
+		}

[Severity: High]
Does removing the split-state preservation for CNTHCTL_EL2 unconditionally overwrite the guest's in-memory register state?

How?

Quoted 4 lines

Since timer_set_traps() applies host-specific overrides to CNTKCTL_EL1 on the host side, calling __vcpu_assign_sys_reg() here appears to blindly copy those host-specific traps directly into the guest's architectural state on exit.

No. CNTKCTL_EL1 is not CNTKCTL_EL12. Not only you don't understand NV, you don't even understand VHE.

Quoted 33 lines

Furthermore, as the comment above notes that NV2p1 makes the ECV bit even more broken, won't this assignment corrupt the guest's ECV state as well?

Quoted 3 lines
arch/arm64/kvm/hyp/vhe/sysreg-sr.c +0 -0
 	}
 
 	__vcpu_assign_sys_reg(vcpu, SP_EL2,	 read_sysreg(sp_el1));

[ ... ]

Quoted 4 lines
arch/arm64/kvm/sys_regs.c +0 -0
index 1dfc1f88bec82..9439c5b2b1fe8 100644

[ ... ]

arch/arm64/kvm/sys_regs.c u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) · line 329

Quoted 11 lines
arch/arm64/kvm/sys_regs.c +4 -2
@@ -322,8 +322,10 @@ u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg)
 		switch (reg) {
 		case CNTHCTL_EL2:
 			val = read_sysreg_el1(SYS_CNTKCTL);
-			val &= CNTKCTL_VALID_BITS;
-			val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			if (!cpus_have_final_cap(ARM64_HAS_NV2P1)) {
+				val &= CNTKCTL_VALID_BITS;
+				val |= __vcpu_sys_reg(vcpu, reg) & ~CNTKCTL_VALID_BITS;
+			}
 			return val;

[Severity: High]
Does this directly return the host-polluted physical register to userspace during KVM_GET_ONE_REG?

Quoted 4 lines

Because the host-specific overrides from timer_set_traps() are present in CNTKCTL_EL1, skipping the bitmasking on ARM64_HAS_NV2P1 systems seems to leak these hypervisor-specific configurations to userspace. This could break live migration by restoring an altered execution environment.

CNTKCTL_EL1 != CNTKCTL_EL12.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#20 lore raw

PATCH v2 11/28 KVM: arm64: Expose FEAT_NV2p1 to NV guests

In reply to Marc Zyngier

Since NV2p1 is reducing the number of traps, it is valuable to expose it to NV guests. Do so.

---
 arch/arm64/kvm/nested.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/arm64/kvm/nested.c +6 -2
index dfb96edbdc43c..35a1c42ad3f84 100644
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		 * You get EITHER
 		 *
 		 * - FEAT_VHE without FEAT_E2H0
-		 * - FEAT_NV limited to FEAT_NV2
+		 * - FEAT_NV limited to FEAT_NV2(p1)
 		 * - HCR_EL2.NV1 being RES0
 		 *
 		 * OR
@@ -1740,7 +1740,11 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features)) {
 			val = 0;
 		} else {
-			val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
+			val &= ID_AA64MMFR4_EL1_NV_frac;
+			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
+			else
+				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
 			val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI_NV1);
 		}
 		break;
Signature · 1 line
2.47.3
author maintainer
#21 lore raw

PATCH v2 12/28 arm64: Add FEAT_NV2p1 detection

In reply to Marc Zyngier

Add the necessary NV2p1 probing to the cpufeature infrastructure.

---
 arch/arm64/kernel/cpufeature.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kernel/cpufeature.c +7 -0
index 9a22df0c5120f..c9c124b0ccc8e 100644
@@ -2620,6 +2620,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 			{ /* Sentinel */ }
 		},
 	},
+	{
+		.desc = "FEAT_NV2p1",
+		.capability = ARM64_HAS_NV2P1,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.matches = has_cpuid_feature,
+		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
+	},
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
2.47.3
#22 lore raw

Re: [PATCH v2 12/28] arm64: Add FEAT_NV2p1 detection

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Missing immediate Context Synchronization Event (isb) after writing to control-plane system…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Missing immediate Context Synchronization Event (isb) after writing to control-plane system registers cnthctl_el2 and cntkctl_el1. --

--- Patch [12]: [PATCH v2 12/28] arm64: Add FEAT_NV2p1 detection --- commit e272e43dc6ce85d3337902d78430aad6bf5b83fc
Author: Marc Zyngier <[email protected]>

KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

This commit relaxes the handling of CNTHCTL_EL2 when FEAT_NV2p1 is present. It removes the need for a split approach to access bits and instead relies on the normal CNTKCTL_EL1 accessor.

Quoted 8 lines
arch/arm64/kvm/arch_timer.c +0 -0
index 4155fe89b58a1..db60facad9f3c 100644
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
 	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
 

arch/arm64/kvm/arch_timer.c static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map) · line 886

Quoted 10 lines
arch/arm64/kvm/arch_timer.c +8 -2
-	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
-	sysreg_clear_set(cnthctl_el2, clr, set);
+	/*
+	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+	 */
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+		sysreg_clear_set(cnthctl_el2, clr, set);
+	else
+		sysreg_clear_set(cntkctl_el1, clr, set);

[Severity: Medium]
Does this code need an immediate context synchronization event after writing to these control-plane system registers?

Modifying cntkctl_el1 (and cnthctl_el2) via sysreg_clear_set() expands to a conditional write_sysreg() without an immediate isb() barrier. The function returns immediately after, which may allow intervening instructions to execute with architecturally undefined pipeline state before a context synchronization event occurs.

This path is reachable during normal VM execution when loading vCPU timer state. While the cnthctl_el2 write is pre-existing, the cntkctl_el1 write was newly introduced here, which could introduce a regression.

Quoted 3 lines
arch/arm64/kvm/arch_timer.c +0 -0
 }
 
 void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
author maintainer
#23 lore raw

Re: [PATCH v2 12/28] arm64: Add FEAT_NV2p1 detection

+ Fuad.

Quoted 23 lines · On Tue, 14 Jul 2026 10:46:08 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Missing immediate Context Synchronization Event (isb) after writing to control-plane system registers cnthctl_el2 and cntkctl_el1. --

--- Patch [12]: [PATCH v2 12/28] arm64: Add FEAT_NV2p1 detection --- commit e272e43dc6ce85d3337902d78430aad6bf5b83fc
Author: Marc Zyngier <[email protected]>

KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present

This commit relaxes the handling of CNTHCTL_EL2 when FEAT_NV2p1 is present. It removes the need for a split approach to access bits and instead relies on the normal CNTKCTL_EL1 accessor.

Quoted 8 lines
arch/arm64/kvm/arch_timer.c +0 -0
index 4155fe89b58a1..db60facad9f3c 100644
@@ -876,8 +876,14 @@ static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map)
 	assign_clear_set_bit(tvt02, CNTHCTL_EL1NVVCT, clr, set);
 	assign_clear_set_bit(tpt02, CNTHCTL_EL1NVPCT, clr, set);
 
Quoted 14 lines

arch/arm64/kvm/arch_timer.c static void timer_set_traps(struct kvm_vcpu *vcpu, struct timer_map *map) · line 886

Quoted 10 lines
arch/arm64/kvm/arch_timer.c +8 -2
-	/* This only happens on VHE, so use the CNTHCTL_EL2 accessor. */
-	sysreg_clear_set(cnthctl_el2, clr, set);
+	/*
+	 * This only happens on VHE, so use the CNTHCTL_EL2 accessor, unless
+	 * we are sure CNTKCTL_EL1 is completely stateful with FEAT_NV2p1.
+	 */
+	if (!cpus_have_final_cap(ARM64_HAS_NV2P1))
+		sysreg_clear_set(cnthctl_el2, clr, set);
+	else
+		sysreg_clear_set(cntkctl_el1, clr, set);

[Severity: Medium]
Does this code need an immediate context synchronization event after writing to these control-plane system registers?

No.

Quoted 10 lines

Modifying cntkctl_el1 (and cnthctl_el2) via sysreg_clear_set() expands to a conditional write_sysreg() without an immediate isb() barrier. The function returns immediately after, which may allow intervening instructions to execute with architecturally undefined pipeline state before a context synchronization event occurs.

This path is reachable during normal VM execution when loading vCPU timer state. While the cnthctl_el2 write is pre-existing, the cntkctl_el1 write was newly introduced here, which could introduce a regression.

No.

Fuad, can we please either teach Sashiko about out of context register updates not immediate requiring synchronisation? The constant insistence on having ISBs all over the map is pushing people in doing the wrong thing.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
reviewer
#24 lore raw

Re: [PATCH v2 12/28] arm64: Add FEAT_NV2p1 detection

Hi Marc,

<snip>

Quoted 4 lines

Fuad, can we please either teach Sashiko about out of context register updates not immediate requiring synchronisation? The constant insistence on having ISBs all over the map is pushing people in doing the wrong thing.

Done:
https://github.com/masoncl/review-prompts/pull/87

Cheers,
/fuad

Quoted 7 lines

Thanks,

        M.

--
Without deviation from the norm, progress is not possible.

author maintainer
#25 lore raw

PATCH v2 13/28 arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2

In reply to Marc Zyngier

FEAT_NV3 introduces a new register that contains the HCR_EL2 value exposed to a NV guest. As such, it has the exact same layout as HCR_EL2.

Describe NVHCR_EL2 as a mapping to HCR_EL2.

---
 arch/arm64/tools/sysreg | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/tools/sysreg +3 -0
index c6e8117a6f9cd..31e4ea455a9ce 100644
@@ -4266,6 +4266,9 @@ Field	1	E2TRE
 Field	0	E0HTRE
 EndSysreg
 
+Sysreg	NVHCR_EL2	3	4	1	5	0
+Mapping	HCR_EL2
+EndSysreg
 
 Sysreg HDFGRTR2_EL2	3	4	3	1	0
 Res0	63:25
Signature · 1 line
2.47.3
author maintainer
#26 lore raw

PATCH v2 14/28 arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3

In reply to Marc Zyngier

FEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them in the sysreg file.

---
 arch/arm64/tools/sysreg | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
arch/arm64/tools/sysreg +6 -1
index 31e4ea455a9ce..afe9337851a2e 100644
@@ -4548,7 +4548,12 @@ Fields	ZCR_ELx
 EndSysreg
 
 Sysreg	HCRX_EL2	3	4	1	2	2
-Res0	63:27
+Res0	63:35
+Field	34	NVnTTLBOS
+Field	33	NVnTTLBIS
+Field	32	NVnTTLB
+Res0	31:28
+Field	27	NVTGE
 Field	26	SRMASKEn
 Res0	25
 Field	24	PACMEn
Signature · 1 line
2.47.3
#27 lore raw

Re: [PATCH v2 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Adding FEAT_NV3 fields to HCRX_EL2 in the sysreg file silently alters the auto-generated…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Adding FEAT_NV3 fields to HCRX_EL2 in the sysreg file silently alters the auto-generated HCRX_EL2_RES0 mask. Because KVM's manually maintained feature maps were not updated to account for this change, it triggers a boot-time verification failure and allows guests to unconditionally set these unsupported bits, breaching hypervisor isolation. --

--- Patch [14]: [PATCH v2 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3 --- commit c8a8a5e81c8baa4b59a436c9bd034245cdeb5fa4
Author: Marc Zyngier <[email protected]>

arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3

This patch introduces 4 new HCRX_EL2 control bits related to FEAT_NV3 and describes them in the sysreg file.

Quoted 5 lines
arch/arm64/tools/sysreg +0 -0
index 42839d2b15ae4..67d013b376a3c 100644

arch/arm64/tools/sysreg Fields ZCR_ELx · line 4556

Quoted 10 lines
arch/arm64/tools/sysreg +6 -1
@@ -4548,7 +4548,12 @@ Fields	ZCR_ELx
 EndSysreg
 
 Sysreg	HCRX_EL2	3	4	1	2	2
-Res0	63:27
+Res0	63:35
+Field	34	NVnTTLBOS
+Field	33	NVnTTLBIS
+Field	32	NVnTTLB
+Res0	31:28
+Field	27	NVTGE

[Severity: High]
Does this change require a corresponding update to KVM's feature map?

By defining these new fields here, the auto-generated HCRX_EL2_RES0 mask is shrunk. Because KVM's manually maintained feature map consumes this mask without adding NEEDS_FEAT rules for the new bits:

arch/arm64/kvm/config.c:hcrx_feat_map[] = {
        ...
        FORCE_RES0(HCRX_EL2_RES0),
        FORCE_RES1(HCRX_EL2_RES1),
};

Will this trigger a deterministic verification failure during host boot? When check_feat_map() validates the map coverage, it checks the mask:

arch/arm64/kvm/config.c:check_feat_map() {
        ...
        if (mask != ~resx)
                kvm_err("Undefined %s behaviour, bits %016llx\n",
                        str, mask ^ ~resx);
        ...
}

Furthermore, without these bits being tracked, could a nested guest be permitted to unconditionally modify these FEAT_NV3 configuration bits via MSR HCRX_EL2, escaping the hypervisor's RES0 masking protections?

Quoted 3 lines
arch/arm64/tools/sysreg +0 -0
 Field	26	SRMASKEn
 Res0	25
 Field	24	PACMEn
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
author maintainer
#28 lore raw

Re: [PATCH v2 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3

Quoted 46 lines · On Tue, 14 Jul 2026 10:45:36 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Adding FEAT_NV3 fields to HCRX_EL2 in the sysreg file silently alters the auto-generated HCRX_EL2_RES0 mask. Because KVM's manually maintained feature maps were not updated to account for this change, it triggers a boot-time verification failure and allows guests to unconditionally set these unsupported bits, breaching hypervisor isolation. --

--- Patch [14]: [PATCH v2 14/28] arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3 --- commit c8a8a5e81c8baa4b59a436c9bd034245cdeb5fa4
Author: Marc Zyngier <[email protected]>

arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3

This patch introduces 4 new HCRX_EL2 control bits related to FEAT_NV3 and describes them in the sysreg file.

Quoted 5 lines
arch/arm64/tools/sysreg +0 -0
index 42839d2b15ae4..67d013b376a3c 100644

arch/arm64/tools/sysreg Fields ZCR_ELx · line 4556

Quoted 10 lines
arch/arm64/tools/sysreg +6 -1
@@ -4548,7 +4548,12 @@ Fields	ZCR_ELx
 EndSysreg
 
 Sysreg	HCRX_EL2	3	4	1	2	2
-Res0	63:27
+Res0	63:35
+Field	34	NVnTTLBOS
+Field	33	NVnTTLBIS
+Field	32	NVnTTLB
+Res0	31:28
+Field	27	NVTGE

[Severity: High]
Does this change require a corresponding update to KVM's feature map?

By defining these new fields here, the auto-generated HCRX_EL2_RES0 mask is shrunk. Because KVM's manually maintained feature map consumes this mask without adding NEEDS_FEAT rules for the new bits:

arch/arm64/kvm/config.c:hcrx_feat_map[] = {
        ...
        FORCE_RES0(HCRX_EL2_RES0),
        FORCE_RES1(HCRX_EL2_RES1),
};

Will this trigger a deterministic verification failure during host boot? When check_feat_map() validates the map coverage, it checks the mask:

Quoted 7 lines
arch/arm64/kvm/config.c:check_feat_map() {
        ...
        if (mask != ~resx)
                kvm_err("Undefined %s behaviour, bits %016llx\n",
                        str, mask ^ ~resx);
        ...
}

Yes, but that's done on purpose. We never update the two in lockstep, which is why we have this warning. See the following patches.

Quoted 4 lines

Furthermore, without these bits being tracked, could a nested guest be permitted to unconditionally modify these FEAT_NV3 configuration bits via MSR HCRX_EL2, escaping the hypervisor's RES0 masking protections?

An untracked bit defaults to being RES0. You should try reading the code.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#29 lore raw

PATCH v2 15/28 arm64: Add ARM64_HAS_NV3 capability

In reply to Marc Zyngier

As a bunch of KVM code is going to depend on FEAT_NV3 being detected on the host, add a new capability that will describe it.

---
 arch/arm64/tools/cpucaps | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/tools/cpucaps +1 -0
index 242dc211d8efa..7e0414509e89a 100644
@@ -52,6 +52,7 @@ HAS_LSUI
 HAS_MOPS
 HAS_NESTED_VIRT
 HAS_NV2P1
+HAS_NV3
 HAS_BBML2_NOABORT
 HAS_PAN
 HAS_PMUV3
Signature · 1 line
2.47.3
#30 lore raw

Re: [PATCH v2 15/28] arm64: Add ARM64_HAS_NV3 capability

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The newly added ARM64_HAS_NV3 capability lacks the required hardware detection logic, rendering…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added ARM64_HAS_NV3 capability lacks the required hardware detection logic, rendering it non-functional. --

--- Patch [15]: [PATCH v2 15/28] arm64: Add ARM64_HAS_NV3 capability --- commit 99ad2a0b713bc6373d924541643c7d6b05f45704
Author: Marc Zyngier <[email protected]>

arm64: Add ARM64_HAS_NV3 capability

This commit adds the ARM64_HAS_NV3 capability ID to the cpucaps list. It is intended to describe host support for FEAT_NV3 so that upcoming KVM code can depend on it being detected.

arch/arm64/tools/cpucaps HAS_LSUI · line 55

Quoted 9 lines
arch/arm64/tools/cpucaps +1 -0
index 242dc211d8efa..7e0414509e89a 100644
@@ -52,6 +52,7 @@ HAS_LSUI
 HAS_MOPS
 HAS_NESTED_VIRT
 HAS_NV2P1
+HAS_NV3

[Severity: Medium]
Does this capability require a corresponding detection entry in the arm64_features array located in arch/arm64/kernel/cpufeature.c?

Without an entry in the arm64_features array to perform the hardware evaluation, the pointer in cpucap_ptrs for ARM64_HAS_NV3 remains NULL. Could this cause any subsequent code checking this capability to always receive false, even on capable hardware?

Quoted 3 lines
arch/arm64/tools/cpucaps +0 -0
 HAS_BBML2_NOABORT
 HAS_PAN
 HAS_PMUV3
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
author maintainer
#31 lore raw

Re: [PATCH v2 15/28] arm64: Add ARM64_HAS_NV3 capability

Quoted 29 lines · On Tue, 14 Jul 2026 10:40:00 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The newly added ARM64_HAS_NV3 capability lacks the required hardware detection logic, rendering it non-functional. --

--- Patch [15]: [PATCH v2 15/28] arm64: Add ARM64_HAS_NV3 capability --- commit 99ad2a0b713bc6373d924541643c7d6b05f45704
Author: Marc Zyngier <[email protected]>

arm64: Add ARM64_HAS_NV3 capability

This commit adds the ARM64_HAS_NV3 capability ID to the cpucaps list. It is intended to describe host support for FEAT_NV3 so that upcoming KVM code can depend on it being detected.

arch/arm64/tools/cpucaps HAS_LSUI · line 55

Quoted 9 lines
arch/arm64/tools/cpucaps +1 -0
index 242dc211d8efa..7e0414509e89a 100644
@@ -52,6 +52,7 @@ HAS_LSUI
 HAS_MOPS
 HAS_NESTED_VIRT
 HAS_NV2P1
+HAS_NV3

[Severity: Medium]
Does this capability require a corresponding detection entry in the arm64_features array located in arch/arm64/kernel/cpufeature.c?

Quoted 4 lines

Without an entry in the arm64_features array to perform the hardware evaluation, the pointer in cpucap_ptrs for ARM64_HAS_NV3 remains NULL. Could this cause any subsequent code checking this capability to always receive false, even on capable hardware?

Yes, on purpose. See the followup patch adding the detection code once everything is in place.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#32 lore raw

PATCH v2 16/28 KVM: arm64: Split NV-specific exit fixups from the non-NV handling

In reply to Marc Zyngier

In order to facilitate further changes, move the NV handling of early fixups in its own helper. This also makes the code slightly simpler to parse.

---
 arch/arm64/kvm/hyp/vhe/switch.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
arch/arm64/kvm/hyp/vhe/switch.c +11 -7
index 8460a8a6745b3..e26cc575e60e1 100644
@@ -534,18 +534,15 @@ static const exit_handler_fn hyp_exit_handlers[] = {
 	[0x3F]				= kvm_hyp_handle_impdef,
 };
 
-static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 {
-	synchronize_vcpu_pstate(vcpu);
-
 	/*
 	 * If we were in HYP context on entry, adjust the PSTATE view
 	 * so that the usual helpers work correctly. This enforces our
 	 * invariant that the guest's HYP context status is preserved
 	 * across a run.
 	 */
-	if (vcpu_has_nv(vcpu) &&
-	    unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
+	if (unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
 		u64 mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
 
 		switch (mode) {
@@ -562,8 +559,15 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
 	}
 
 	/* Apply extreme paranoia! */
-	BUG_ON(vcpu_has_nv(vcpu) &&
-	       !!host_data_test_flag(VCPU_IN_HYP_CONTEXT) != is_hyp_ctxt(vcpu));
+	BUG_ON(!!host_data_test_flag(VCPU_IN_HYP_CONTEXT) != is_hyp_ctxt(vcpu));
+}
+
+static bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)
+{
+	synchronize_vcpu_pstate(vcpu);
+
+	if (vcpu_has_nv(vcpu))
+		fixup_nv_guest_exit(vcpu);
 
 	return __fixup_guest_exit(vcpu, exit_code, hyp_exit_handlers);
 }
Signature · 1 line
2.47.3
author maintainer
#33 lore raw

PATCH v2 17/28 KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation

In reply to Marc Zyngier

Expose the FEAT_NV3 control bits to the sanitisation code so that KVM stops moaning about the unattributed bits.

---
 arch/arm64/kvm/config.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/arm64/kvm/config.c +6 -0
index c61d62b796023..467f57eb8e6bb 100644
@@ -225,6 +225,7 @@ struct reg_feat_map_desc {
 #define FEAT_HCX		ID_AA64MMFR1_EL1, HCX, IMP
 #define FEAT_S2PIE		ID_AA64MMFR3_EL1, S2PIE, IMP
 #define FEAT_GCIE		ID_AA64PFR2_EL1, GCIE, IMP
+#define FEAT_NV3		ID_AA64MMFR4_EL1, NV_frac, NV3
 
 static bool not_feat_aa64el3(struct kvm *kvm)
 {
@@ -904,6 +905,11 @@ static const DECLARE_FEAT_MAP_FGT(hdfgwtr2_desc, hdfgwtr2_masks,
 
 
 static const struct reg_bits_to_feat_map hcrx_feat_map[] = {
+	NEEDS_FEAT(HCRX_EL2_NVTGE		|
+		   HCRX_EL2_NVnTTLB		|
+		   HCRX_EL2_NVnTTLBIS		|
+		   HCRX_EL2_NVnTTLBOS,
+		   FEAT_NV3),
 	NEEDS_FEAT(HCRX_EL2_SRMASKEn, FEAT_SRMASK),
 	NEEDS_FEAT(HCRX_EL2_PACMEn, feat_pauth_lr),
 	NEEDS_FEAT(HCRX_EL2_EnFPM, FEAT_FPMR),
Signature · 1 line
2.47.3
author maintainer
#34 lore raw

PATCH v2 18/28 KVM: arm64: Add kvm_has_nv{2,3}() predicates

In reply to Marc Zyngier

Add a new set of predicates indicating whether VM is capable of NV2, NV3, and is in a nested NV3 context.

This is going to become useful as we start dealing with a mix of behaviours (NV2, NV3, NV2 on NV3...).

---
 arch/arm64/include/asm/kvm_emulate.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +20 -0
index 9831166695186..c562d8171d5e1 100644
@@ -266,6 +266,26 @@ static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu)
 	       (__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_TMEA);
 }
 
+static inline bool kvm_has_nv2(struct kvm *kvm)
+{
+	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY));
+}
+
+static inline bool kvm_has_nv3(struct kvm *kvm)
+{
+	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+		cpus_have_final_cap(ARM64_HAS_NV3) &&
+		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
+}
+
+static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)
+{
+	return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
+		(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
+		(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
+}
+
 /*
  * The layout of SPSR for an AArch32 state is different when observed from an
  * AArch64 SPSR_ELx or an AArch32 SPSR_*. This function generates the AArch32
Signature · 1 line
2.47.3
#35 lore raw

Re: [PATCH v2 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] kvm_has_nv3() will always return false because ARM64_HAS_NV3 is never populated in the…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] kvm_has_nv3() will always return false because ARM64_HAS_NV3 is never populated in the capabilities bitmap. --

--- Patch [18]: [PATCH v2 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates --- commit e61ff01470e9cbe730c6702e605c3c17d525ca4e
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add kvm_has_nv{2,3}() predicates

This commit adds a new set of predicates indicating whether a VM is capable of NV2, NV3, and is in a nested NV3 context.

Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +0 -0
index 9831166695186..c562d8171d5e1 100644

[ ... ]

arch/arm64/include/asm/kvm_emulate.h static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu) · line 278

Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +4 -0
+static inline bool kvm_has_nv3(struct kvm *kvm)
+{
+	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+		cpus_have_final_cap(ARM64_HAS_NV3) &&

[Severity: Medium]
Will kvm_has_nv3() always return false here?

It appears the ARM64_HAS_NV3 capability was added to the cpucaps tools, but the corresponding hardware detection entry is missing from the arm64_features array in arch/arm64/kernel/cpufeature.c. Since it is never populated, cpus_have_final_cap(ARM64_HAS_NV3) will unconditionally evaluate to false, turning any code relying on this predicate into dead code.

Quoted 2 lines
arch/arm64/include/asm/kvm_emulate.h +2 -0
+		kvm_has_feat(kvm, ID_AA64MMFR4_EL1, NV_frac, NV3));
+}

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=18
author maintainer
#36 lore raw

Re: [PATCH v2 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates

Quoted 29 lines · On Tue, 14 Jul 2026 10:48:03 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] kvm_has_nv3() will always return false because ARM64_HAS_NV3 is never populated in the capabilities bitmap. --

--- Patch [18]: [PATCH v2 18/28] KVM: arm64: Add kvm_has_nv{2,3}() predicates --- commit e61ff01470e9cbe730c6702e605c3c17d525ca4e
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add kvm_has_nv{2,3}() predicates

This commit adds a new set of predicates indicating whether a VM is capable of NV2, NV3, and is in a nested NV3 context.

Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +0 -0
index 9831166695186..c562d8171d5e1 100644

[ ... ]

arch/arm64/include/asm/kvm_emulate.h static inline bool vserror_state_is_nested(struct kvm_vcpu *vcpu) · line 278

Quoted 4 lines
arch/arm64/include/asm/kvm_emulate.h +4 -0
+static inline bool kvm_has_nv3(struct kvm *kvm)
+{
+	return (cpus_have_final_cap(ARM64_HAS_NESTED_VIRT) &&
+		cpus_have_final_cap(ARM64_HAS_NV3) &&

[Severity: Medium]
Will kvm_has_nv3() always return false here?

Quoted 5 lines

It appears the ARM64_HAS_NV3 capability was added to the cpucaps tools, but the corresponding hardware detection entry is missing from the arm64_features array in arch/arm64/kernel/cpufeature.c. Since it is never populated, cpus_have_final_cap(ARM64_HAS_NV3) will unconditionally evaluate to false, turning any code relying on this predicate into dead code.

Patch #26 is your friend.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#37 lore raw

PATCH v2 19/28 KVM: arm64: Make HCR_EL2 a non-VNCR register

In reply to Marc Zyngier

FEAT_NV3 makes a fundamental change to the architecture, by moving guest-initiated HCR_EL2 accesses to the NVHCR_EL2 register. As the names suggests, this is HCR_EL2 for a NV guest.

But where do NVHCR_EL2 accesses from a guest go? The are redirected to the VNCR page, right where HCR_EL2 is stored in the NV2 case. Does it hurt? Good. There's more coming.

The challenge here is to make KVM work seamlessly, without rewriting everything. Which implies that things such as __vcpu_sys_reg(HCR_EL2) must work, no matter the underlying NV implementation.

A simple way to deal with it is to move HCR_EL2's canonical storage outside of VNCR for the vast majority of the KVM code, and only have a copy at entry/exit times. Given that we don't really support NV3 yet, this is pretty simple.

In the process, advertise NVHCR_EL2 as the register that now holds offset 0x78 in the VNCR page.

---
 arch/arm64/include/asm/kvm_host.h     | 3 ++-
 arch/arm64/include/asm/vncr_mapping.h | 2 +-
 arch/arm64/kvm/hyp/vhe/switch.c       | 9 +++++++++
 3 files changed, 12 insertions(+), 2 deletions(-)
arch/arm64/include/asm/kvm_host.h +2 -1
index bae2c4f92ef5c..2648c8a717ba0 100644
@@ -543,6 +543,7 @@ enum vcpu_sysreg {
 	MDCR_EL2,	/* Monitor Debug Configuration Register (EL2) */
 	CNTHCTL_EL2,	/* Counter-timer Hypervisor Control register */
 	ZCR_EL2,	/* SVE Control Register (EL2) */
+	HCR_EL2,	/* Hypervisor Control Register */
 
 	/* Any VNCR-capable reg goes after this point */
 	MARKER(__VNCR_START__),
@@ -571,7 +572,7 @@ enum vcpu_sysreg {
 	VNCR(TFSR_EL1),	/* Tag Fault Status Register (EL1) */
 	VNCR(VPIDR_EL2),/* Virtualization Processor ID Register */
 	VNCR(VMPIDR_EL2),/* Virtualization Multiprocessor ID Register */
-	VNCR(HCR_EL2),	/* Hypervisor Configuration Register */
+	VNCR(NVHCR_EL2),/* NV Hypervisor Configuration Register */
 	VNCR(HSTR_EL2),	/* Hypervisor System Trap Register */
 	VNCR(VTTBR_EL2),/* Virtualization Translation Table Base Register */
 	VNCR(VTCR_EL2),	/* Virtualization Translation Control Register */
arch/arm64/include/asm/vncr_mapping.h +1 -1
index 14366d35ce82f..9e8a49fa8b638 100644
@@ -11,7 +11,7 @@
 #define VNCR_VTCR_EL2           0x040
 #define VNCR_VMPIDR_EL2         0x050
 #define VNCR_CNTVOFF_EL2        0x060
-#define VNCR_HCR_EL2            0x078
+#define VNCR_NVHCR_EL2          0x078
 #define VNCR_HSTR_EL2           0x080
 #define VNCR_VPIDR_EL2          0x088
 #define VNCR_TPIDR_EL2          0x090
arch/arm64/kvm/hyp/vhe/switch.c +9 -0
index e26cc575e60e1..ee92f6ad91c2b 100644
@@ -70,6 +70,9 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
 		if (!vcpu_el2_e2h_is_set(vcpu))
 			hcr |= HCR_NV1;
 
+		/* Publish the guest's view of HCR_EL2 to the HW */
+		__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+
 		/*
 		 * Nothing in HCR_EL2 should impact running in hypervisor
 		 * context, apart from bits we have defined as RESx (E2H,
@@ -544,6 +547,7 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 	 */
 	if (unlikely(host_data_test_flag(VCPU_IN_HYP_CONTEXT))) {
 		u64 mode = *vcpu_cpsr(vcpu) & (PSR_MODE_MASK | PSR_MODE32_BIT);
+		u64 hcr;
 
 		switch (mode) {
 		case PSR_MODE_EL1t:
@@ -556,6 +560,11 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 
 		*vcpu_cpsr(vcpu) &= ~(PSR_MODE_MASK | PSR_MODE32_BIT);
 		*vcpu_cpsr(vcpu) |= mode;
+
+		/* Publish the latest HCR_EL2 to the emulation */
+		hcr = __vcpu_sys_reg(vcpu, NVHCR_EL2);
+
+		__vcpu_assign_sys_reg(vcpu, HCR_EL2, hcr);
 	}
 
 	/* Apply extreme paranoia! */
Signature · 1 line
2.47.3
author maintainer
#38 lore raw

PATCH v2 20/28 KVM: arm64: Add sanitisation for NVHCR_EL2

In reply to Marc Zyngier

Just like any other VNCR-based register, NVHCR_EL2 requires some level of sanitisation. Being specified as a live copy of HCR_EL2, it adopts the exact same format, but depends on FEAT_NV3 instead.

A subtle aspect is that we only want to apply the sanitisation if FEAT_NV3 is actually present, as the VNCR location is otherwise used to back accesses to HCR_EL2.

---
 arch/arm64/kvm/config.c | 15 +++++++++++++++
 arch/arm64/kvm/nested.c |  4 ++++
 2 files changed, 19 insertions(+)
arch/arm64/kvm/config.c +15 -0
index 467f57eb8e6bb..1053676551aff 100644
@@ -1019,6 +1019,9 @@ static const struct reg_bits_to_feat_map hcr_feat_map[] = {
 static const DECLARE_FEAT_MAP(hcr_desc, HCR_EL2,
 			      hcr_feat_map, FEAT_AA64EL2);
 
+static const DECLARE_FEAT_MAP(nvhcr_desc, NVHCR_EL2,
+			      hcr_feat_map, FEAT_NV3);
+
 static const struct reg_bits_to_feat_map sctlr2_feat_map[] = {
 	NEEDS_FEAT(SCTLR2_EL1_NMEA	|
 		   SCTLR2_EL1_EASE,
@@ -1393,6 +1396,7 @@ void __init check_feature_map(void)
 	check_reg_desc(&hdfgwtr2_desc);
 	check_reg_desc(&hcrx_desc);
 	check_reg_desc(&hcr_desc);
+	check_reg_desc(&nvhcr_desc);
 	check_reg_desc(&sctlr2_desc);
 	check_reg_desc(&tcr2_el2_desc);
 	check_reg_desc(&sctlr_el1_desc);
@@ -1592,6 +1596,17 @@ struct resx get_reg_fixed_bits(struct kvm *kvm, enum vcpu_sysreg reg)
 	case HCR_EL2:
 		resx = compute_reg_resx_bits(kvm, &hcr_desc, 0, 0);
 		break;
+	case NVHCR_EL2:
+		/*
+		 * Only apply sanitisation if we do have FEAT_NV3.
+		 * Otherwise, the register aliases with HCR_EL2 in VNCR,
+		 * and we're better off relying on data transfers between
+		 * NVHCR_EL2 and HCR_EL2 to sanitise things.
+		 */
+		resx = (kvm_has_nv3(kvm) ?
+			compute_reg_resx_bits(kvm, &nvhcr_desc, 0, 0) :
+			(typeof(resx)){});
+		break;
 	case SCTLR2_EL1:
 	case SCTLR2_EL2:
 		resx = compute_reg_resx_bits(kvm, &sctlr2_desc, 0, 0);
arch/arm64/kvm/nested.c +4 -0
index 35a1c42ad3f84..f646894fec9a9 100644
@@ -1830,6 +1830,10 @@ int kvm_init_nv_sysregs(struct kvm_vcpu *vcpu)
 	resx = get_reg_fixed_bits(kvm, HCR_EL2);
 	set_sysreg_masks(kvm, HCR_EL2, resx);
 
+	/* NVHCR_EL2 */
+	resx = get_reg_fixed_bits(kvm, NVHCR_EL2);
+	set_sysreg_masks(kvm, NVHCR_EL2, resx);
+
 	/* HCRX_EL2 */
 	resx = get_reg_fixed_bits(kvm, HCRX_EL2);
 	set_sysreg_masks(kvm, HCRX_EL2, resx);
Signature · 1 line
2.47.3
author maintainer
#39 lore raw

PATCH v2 21/28 KVM: arm64: Add NVHCR_EL2 handling to the sysreg array

In reply to Marc Zyngier

Expose NVHCR_EL2 to userspace, and treat the direct access as UNDEF, as that would only outline a bug in our exception routing.

The generic accessors are also updated to deal with the relatively uncommon location of that register.

---
 arch/arm64/kvm/sys_regs.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
arch/arm64/kvm/sys_regs.c +28 -0
index 9439c5b2b1fe8..0aeb2e736fde3 100644
@@ -221,6 +221,20 @@ static void locate_register(const struct kvm_vcpu *vcpu, enum vcpu_sysreg reg,
 		if (is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu))
 			loc->loc = SR_LOC_SPECIAL;
 		break;
+	case NVHCR_EL2:
+		/*
+		 * Yes, NVHCR_EL2 maps to itself when loaded in nested
+		 * context. If you feel like the architecture is double
+		 * backing on itself upside down, you're not alone.
+		 */
+		WARN_ON_ONCE(!kvm_has_nv3(vcpu->kvm));
+		if (is_hyp_ctxt(vcpu)) {
+			loc->loc = SR_LOC_MEMORY;
+		} else {
+			loc->loc = SR_LOC_LOADED | SR_LOC_MAPPED;
+			loc->map_reg = NVHCR_EL2;
+		}
+		break;
 	default:
 		loc->loc = locate_direct_register(vcpu, reg);
 	}
@@ -260,6 +274,7 @@ static u64 read_sr_from_cpu(enum vcpu_sysreg reg)
 	case DACR32_EL2:	val = read_sysreg_s(SYS_DACR32_EL2);	break;
 	case IFSR32_EL2:	val = read_sysreg_s(SYS_IFSR32_EL2);	break;
 	case DBGVCR32_EL2:	val = read_sysreg_s(SYS_DBGVCR32_EL2);	break;
+	case NVHCR_EL2:		val = read_sysreg_s(SYS_NVHCR_EL2);	break;
 	default:		WARN_ON_ONCE(1);
 	}
 
@@ -298,6 +313,7 @@ static void write_sr_to_cpu(enum vcpu_sysreg reg, u64 val)
 	case DACR32_EL2:	write_sysreg_s(val, SYS_DACR32_EL2);	break;
 	case IFSR32_EL2:	write_sysreg_s(val, SYS_IFSR32_EL2);	break;
 	case DBGVCR32_EL2:	write_sysreg_s(val, SYS_DBGVCR32_EL2);	break;
+	case NVHCR_EL2:		write_sysreg_s(val, SYS_NVHCR_EL2);	break;
 	default:		WARN_ON_ONCE(1);
 	}
 }
@@ -2861,6 +2877,16 @@ static unsigned int vncr_el2_visibility(const struct kvm_vcpu *vcpu,
 	return REG_HIDDEN;
 }
 
+static unsigned int nvhcr_el2_visibility(const struct kvm_vcpu *vcpu,
+					const struct sys_reg_desc *rd)
+{
+	if (el2_visibility(vcpu, rd) == 0 &&
+	    kvm_has_feat(vcpu->kvm, ID_AA64MMFR4_EL1, NV_frac, NV3))
+		return 0;
+
+	return REG_HIDDEN;
+}
+
 static unsigned int sctlr2_visibility(const struct kvm_vcpu *vcpu,
 				      const struct sys_reg_desc *rd)
 {
@@ -3774,6 +3800,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 			 sve_el2_visibility),
 
 	EL2_REG_VNCR(HCRX_EL2, reset_val, 0),
+	EL2_REG_FILTERED(NVHCR_EL2, undef_access, reset_val, 0,
+			 nvhcr_el2_visibility),
 
 	EL2_REG(TTBR0_EL2, access_rw, reset_val, 0),
 	EL2_REG(TTBR1_EL2, access_rw, reset_val, 0),
Signature · 1 line
2.47.3
#40 lore raw

Re: [PATCH v2 21/28] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Userspace writes to NVHCR_EL2 via KVM_SET_ONE_REG are silently discarded upon L1 guest entry. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Userspace writes to NVHCR_EL2 via KVM_SET_ONE_REG are silently discarded upon L1 guest entry. --

--- Patch [21]: [PATCH v2 21/28] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array --- commit 40ce0e0627839603da739968567e9f3998f1ed15
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add NVHCR_EL2 handling to the sysreg array

Expose NVHCR_EL2 to userspace, and treat the direct access as UNDEF, as that would only outline a bug in our exception routing.

Quoted 4 lines
arch/arm64/kvm/sys_regs.c +0 -0
index 9439c5b2b1fe8..0aeb2e736fde3 100644

[ ... ]

arch/arm64/kvm/sys_regs.c static const struct sys_reg_desc sys_reg_descs[] = { · line 3804

Quoted 6 lines
arch/arm64/kvm/sys_regs.c +2 -0
@@ -3774,6 +3800,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 			 sve_el2_visibility),
 
 	EL2_REG_VNCR(HCRX_EL2, reset_val, 0),
+	EL2_REG_FILTERED(NVHCR_EL2, undef_access, reset_val, 0,
+			 nvhcr_el2_visibility),

[Severity: Medium]
Will userspace writes to NVHCR_EL2 be silently discarded upon L1 guest entry?

Because NVHCR_EL2 is exposed without a custom .set_user handler, writes via KVM_SET_ONE_REG will update vcpu->arch.ctxt.sys_regs[NVHCR_EL2] using the default behavior. However, upon running the VCPU in hyp_ctxt (L1 hypervisor), __compute_hcr() unconditionally executes:

  __vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));

This seems to clobber the userspace write, causing UAPI state loss and a KVM ABI violation where KVM_SET_ONE_REG writes are expected to be preserved.

Does this require a custom .set_user handler to maintain alias synchronization between NVHCR_EL2 and HCR_EL2?

Quoted 3 lines
arch/arm64/kvm/sys_regs.c +0 -0
 	EL2_REG(TTBR0_EL2, access_rw, reset_val, 0),
 	EL2_REG(TTBR1_EL2, access_rw, reset_val, 0),
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=21
author maintainer
#41 lore raw

Re: [PATCH v2 21/28] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array

Quoted 24 lines · On Tue, 14 Jul 2026 11:24:21 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Userspace writes to NVHCR_EL2 via KVM_SET_ONE_REG are silently discarded upon L1 guest entry. --

--- Patch [21]: [PATCH v2 21/28] KVM: arm64: Add NVHCR_EL2 handling to the sysreg array --- commit 40ce0e0627839603da739968567e9f3998f1ed15
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add NVHCR_EL2 handling to the sysreg array

Expose NVHCR_EL2 to userspace, and treat the direct access as UNDEF, as that would only outline a bug in our exception routing.

Quoted 4 lines
arch/arm64/kvm/sys_regs.c +0 -0
index 9439c5b2b1fe8..0aeb2e736fde3 100644

[ ... ]

Quoted 3 lines
arch/arm64/kvm/sys_regs.c +0 -0
@@ -3774,6 +3800,8 @@ static const struct sys_reg_desc sys_reg_descs[] = {
 			 sve_el2_visibility),
 
Quoted 6 lines

arch/arm64/kvm/sys_regs.c static const struct sys_reg_desc sys_reg_descs[] = { · line 3804

Quoted 3 lines
arch/arm64/kvm/sys_regs.c +2 -0
 	EL2_REG_VNCR(HCRX_EL2, reset_val, 0),
+	EL2_REG_FILTERED(NVHCR_EL2, undef_access, reset_val, 0,
+			 nvhcr_el2_visibility),

[Severity: Medium]
Will userspace writes to NVHCR_EL2 be silently discarded upon L1 guest entry?

Short answer: no. Longer answer follows.

Quoted 6 lines

Because NVHCR_EL2 is exposed without a custom .set_user handler, writes via KVM_SET_ONE_REG will update vcpu->arch.ctxt.sys_regs[NVHCR_EL2] using the default behavior. However, upon running the VCPU in hyp_ctxt (L1 hypervisor), __compute_hcr() unconditionally executes:

  __vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));

If userspace can write to NVHCR_EL2, that's because the guest has NV3 exposed to it. And if so, then the host also has NV3 and is using it. Then the code looks like this:

	/* Publish the guest's view of HCR_EL2 to the HW */
	if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
		write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
	else
		__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));

and the value of HCR_EL2 is written into the NVHCR_EL2 *register*.

Quoted 3 lines

This seems to clobber the userspace write, causing UAPI state loss and a KVM ABI violation where KVM_SET_ONE_REG writes are expected to be preserved.

See above?

Quoted 3 lines

Does this require a custom .set_user handler to maintain alias synchronization between NVHCR_EL2 and HCR_EL2?

No.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#42 lore raw

PATCH v2 22/28 KVM: arm64: Add routing for NVHCR_EL2 trap

In reply to Marc Zyngier

NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0 and HCR_EL2.NV==1. Describe this in the exception routing tables.

---
 arch/arm64/kvm/emulate-nested.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/arm64/kvm/emulate-nested.c +13 -0
index b7f3d86a94031..625604019fb32 100644
@@ -136,6 +136,8 @@ enum cgt_group_id {
 	CGT_CPTR_TTA,
 	CGT_MDCR_HPMN,
 
+	CGT_HCR_NV_HCRX_nNVTGE,
+
 	/* Must be last */
 	__NR_CGT_GROUP_IDS__
 };
@@ -588,6 +590,15 @@ static enum trap_behaviour check_mdcr_hpmn(struct kvm_vcpu *vcpu)
 	return BEHAVE_HANDLE_LOCALLY;
 }
 
+static enum trap_behaviour check_hcr_nv_hcrx_nnvtge(struct kvm_vcpu *vcpu)
+{
+	if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
+	    !(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE))
+		return BEHAVE_FORWARD_RW;
+
+	return BEHAVE_HANDLE_LOCALLY;
+}
+
 #define CCC(id, fn)				\
 	[id - __COMPLEX_CONDITIONS__] = fn
 
@@ -598,6 +609,7 @@ static const complex_condition_check ccc[] = {
 	CCC(CGT_CNTHCTL_EL1NVVCT, check_cnthctl_el1nvvct),
 	CCC(CGT_CPTR_TTA, check_cptr_tta),
 	CCC(CGT_MDCR_HPMN, check_mdcr_hpmn),
+	CCC(CGT_HCR_NV_HCRX_nNVTGE, check_hcr_nv_hcrx_nnvtge),
 };
 
 /*
@@ -853,6 +865,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = {
 	SR_TRAP(SYS_SCTLR2_EL2,		CGT_HCR_NV),
 	SR_RANGE_TRAP(SYS_HCR_EL2,
 		      SYS_HCRX_EL2,	CGT_HCR_NV),
+	SR_TRAP(SYS_NVHCR_EL2,		CGT_HCR_NV_HCRX_nNVTGE),
 	SR_TRAP(SYS_SMPRIMAP_EL2,	CGT_HCR_NV),
 	SR_TRAP(SYS_SMCR_EL2,		CGT_HCR_NV),
 	SR_RANGE_TRAP(SYS_TTBR0_EL2,
Signature · 1 line
2.47.3
author maintainer
#43 lore raw

PATCH v2 23/28 KVM: arm64: Add NVHCR_EL2 context switching

In reply to Marc Zyngier

Since NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it must be dealt with in some particular way:

- for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly reflects HCR_EL2 as read and written by the guest itself. It must therefore be eagerly synced back with the emulation code which only knows about HCR_EL2. This is unconditional if NV3 is available on the host.

- For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we just context switch it like any other EL1 register. Yes, EL1, as that's where this thing runs from the PoV of L1. This is conditioned on the guest using NV3.

---
 arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 11 +++++++++++
 arch/arm64/kvm/hyp/vhe/switch.c            | 10 ++++++++--
 2 files changed, 19 insertions(+), 2 deletions(-)
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +11 -0
index a17cbe7582de9..c382848d31947 100644
@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
 
 	if (ctxt_has_sctlr2(ctxt))
 		ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
+
+	/* Retrieve L2's HCR_EL2, and save it for future use */
+	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+		ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);
 }
 
 static inline void __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt)
@@ -285,6 +289,13 @@ static inline void __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt,
 
 	if (ctxt_has_sctlr2(ctxt))
 		write_sysreg_el1(ctxt_sys_reg(ctxt, SCTLR2_EL1), SYS_SCTLR2);
+
+	/*
+	 * Publish the L2 view of HCR_EL2 to the HW if L1 is using NV3.
+	 * Otherwise, the data is already in place in the L1's own VNCR.
+	 */
+	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+		write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
 }
 
 /* Read the VCPU state's PSTATE, but translate (v)EL2 to EL1. */
arch/arm64/kvm/hyp/vhe/switch.c +8 -2
index ee92f6ad91c2b..e823046d28796 100644
@@ -71,7 +71,10 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
 			hcr |= HCR_NV1;
 
 		/* Publish the guest's view of HCR_EL2 to the HW */
-		__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+		if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+			write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
+		else
+			__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
 
 		/*
 		 * Nothing in HCR_EL2 should impact running in hypervisor
@@ -562,7 +565,10 @@ static void fixup_nv_guest_exit(struct kvm_vcpu *vcpu)
 		*vcpu_cpsr(vcpu) |= mode;
 
 		/* Publish the latest HCR_EL2 to the emulation */
-		hcr = __vcpu_sys_reg(vcpu, NVHCR_EL2);
+		hcr = (cpus_have_final_cap(ARM64_HAS_NV3) &&
+		       vcpu_el2_e2h_is_set(vcpu)) ?
+			read_sysreg_s(SYS_NVHCR_EL2) :
+			__vcpu_sys_reg(vcpu, NVHCR_EL2);
 
 		__vcpu_assign_sys_reg(vcpu, HCR_EL2, hcr);
 	}
Signature · 1 line
2.47.3
#44 lore raw

Re: [PATCH v2 23/28] KVM: arm64: Add NVHCR_EL2 context switching

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] A guest can leak the SYS_NVHCR_EL2 hardware register state of a previous vCPU by configuring E2H=0…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] A guest can leak the SYS_NVHCR_EL2 hardware register state of a previous vCPU by configuring E2H=0 and NVTGE=1. --

--- Patch [23]: [PATCH v2 23/28] KVM: arm64: Add NVHCR_EL2 context switching --- commit d9946edce781237514b1a1fe42315834ea75838a
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add NVHCR_EL2 context switching

This commit introduces context switching for the NVHCR_EL2 register to support nested virtualization. For an L1 guest in hyp context, the register is eagerly synced with HCR_EL2 emulation. For an L2 guest, it is context switched as an EL1 register if NV3 is in use.

arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) · line 178

Quoted 12 lines
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +4 -0
index a17cbe7582de9..c382848d31947 100644
@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
 
 	if (ctxt_has_sctlr2(ctxt))
 		ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
+
+	/* Retrieve L2's HCR_EL2, and save it for future use */
+	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+		ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);

[Severity: High]
Does this create an asymmetry with how __compute_hcr() initializes the register?

In __compute_hcr(), the write to SYS_NVHCR_EL2 is gated by an E2H check:

	if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
		write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);

However, this save path relies on is_nested_nv3_ctxt(), which checks NV and NVTGE but does not verify E2H:

static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)

{
	return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
		(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
		(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
}

If an L1 guest sets HCR_EL2.E2H = 0, HCR_EL2.NV = 1, and HCRX_EL2.NVTGE = 1, could __compute_hcr() skip writing SYS_NVHCR_EL2 on L1 entry, leaving the hardware register uninitialized (holding a previous vCPU's state)?

When transitioning to an L2 guest and later preempting, would this save path evaluate is_nested_nv3_ctxt() as true? Could this cause it to read the uninitialized SYS_NVHCR_EL2 register and leak the previous vCPU's hardware state into the guest's context?

Quoted 1 line
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +0 -0
 }

[ ... ]

Quoted 16 lines
arch/arm64/kvm/hyp/vhe/switch.c +4 -1
index ee92f6ad91c2b..e823046d28796 100644
@@ -71,7 +71,10 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu)
 			hcr |= HCR_NV1;
 
 		/* Publish the guest's view of HCR_EL2 to the HW */
-		__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
+		if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+			write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);
+		else
+			__vcpu_assign_sys_reg(vcpu, NVHCR_EL2, __vcpu_sys_reg(vcpu, HCR_EL2));
 
 		/*
 		 * Nothing in HCR_EL2 should impact running in hypervisor
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=23
author maintainer
#45 lore raw

Re: [PATCH v2 23/28] KVM: arm64: Add NVHCR_EL2 context switching

Quoted 33 lines · On Tue, 14 Jul 2026 11:03:21 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] A guest can leak the SYS_NVHCR_EL2 hardware register state of a previous vCPU by configuring E2H=0 and NVTGE=1. --

--- Patch [23]: [PATCH v2 23/28] KVM: arm64: Add NVHCR_EL2 context switching --- commit d9946edce781237514b1a1fe42315834ea75838a
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add NVHCR_EL2 context switching

This commit introduces context switching for the NVHCR_EL2 register to support nested virtualization. For an L1 guest in hyp context, the register is eagerly synced with HCR_EL2 emulation. For an L2 guest, it is context switched as an EL1 register if NV3 is in use.

arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) · line 178

Quoted 12 lines
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h +4 -0
index a17cbe7582de9..c382848d31947 100644
@@ -172,6 +172,10 @@ static inline void __sysreg_save_el1_state(struct kvm_cpu_context *ctxt)
 
 	if (ctxt_has_sctlr2(ctxt))
 		ctxt_sys_reg(ctxt, SCTLR2_EL1) = read_sysreg_el1(SYS_SCTLR2);
+
+	/* Retrieve L2's HCR_EL2, and save it for future use */
+	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
+		ctxt_sys_reg(ctxt, NVHCR_EL2) = read_sysreg_s(SYS_NVHCR_EL2);

[Severity: High]
Does this create an asymmetry with how __compute_hcr() initializes the register?

Quoted 5 lines

In __compute_hcr(), the write to SYS_NVHCR_EL2 is gated by an E2H check:

	if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
		write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);

This is a host setting.

Quoted 9 lines

However, this save path relies on is_nested_nv3_ctxt(), which checks NV and NVTGE but does not verify E2H:

static inline bool is_nested_nv3_ctxt(struct kvm_vcpu *vcpu)

{
	return (has_vhe() && kvm_has_nv3(vcpu->kvm) && is_nested_ctxt(vcpu) &&
		(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_EL2_NV) &&
		(__vcpu_sys_reg(vcpu, HCRX_EL2) & HCRX_EL2_NVTGE));
}

This is a guest setting.

Quoted 4 lines

If an L1 guest sets HCR_EL2.E2H = 0, HCR_EL2.NV = 1, and HCRX_EL2.NVTGE = 1, could __compute_hcr() skip writing SYS_NVHCR_EL2 on L1 entry, leaving the hardware register uninitialized (holding a previous vCPU's state)?

host != guest.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#46 lore raw

PATCH v2 24/28 KVM: arm64: Engage NV3 ERET trap elision

In reply to Marc Zyngier

When running on NV3 HW, always engage ERET trap elision when running the L1 context, as there is no benefit in not doing so.

An L1 can itself engage trap elision by setting its own view of HCRX_EL2.NVTGE==1, which will subsequently be honored.

---
 arch/arm64/include/asm/kvm_emulate.h | 10 ++++++++++
 arch/arm64/kvm/hyp/vhe/switch.c      |  4 ++++
 2 files changed, 14 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +10 -0
index c562d8171d5e1..b32870a5e1236 100644
@@ -706,6 +706,16 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
 
 		if (kvm_has_feat(kvm, ID_AA64ISAR1_EL1, LS64, LS64_V))
 			vcpu->arch.hcrx_el2 |= HCRX_EL2_EnASR;
+
+		/*
+		 * NV3 is a host-specific extension, and we always use it
+		 * when present and that the guest uses NV. It may be be
+		 * hidden from the guest though.
+		 */
+		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
+		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
+			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+		}
 	}
 }
 #endif /* __ARM64_KVM_EMULATE_H__ */
arch/arm64/kvm/hyp/vhe/switch.c +4 -0
index e823046d28796..8a39e72d8386b 100644
@@ -345,6 +345,10 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	u64 esr = kvm_vcpu_get_esr(vcpu);
 	u64 spsr, elr, mode;
 
+	/* With NV3, the fast path is handled in HW */
+	if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
+		return false;
+
 	/*
 	 * Going through the whole put/load motions is a waste of time
 	 * if this is a VHE guest hypervisor returning to its own
Signature · 1 line
2.47.3
author maintainer
#47 lore raw

PATCH v2 25/28 KVM: arm64: Engage NV3 TLBI trap elision

In reply to Marc Zyngier

Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs that only affects the guest's S1 translation. Enable this, with the express condition that the guest isn't NV2 aware, as we otherwise need to trap these TLBIs to deal with VNCR mappings.

---
 arch/arm64/include/asm/kvm_emulate.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +13 -0
index b32870a5e1236..d17b161d4ba3a 100644
@@ -715,6 +715,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
 		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
 		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
 			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+
+			/*
+			 * If the guest is NV2-capable, then we need to see
+			 * all the TLBIs, as configured in HCR_EL2.
+			 * Otherwise, relax the TLBI traps to only TGE=0.
+			 */
+			if (!kvm_has_nv2(vcpu->kvm)) {
+				vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB   |
+							HCRX_EL2_NVnTTLBIS);
+
+				if (kvm_has_feat(kvm, ID_AA64ISAR0_EL1, TLB, OS))
+					vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;;
+			}
 		}
 	}
 }
Signature · 1 line
2.47.3
#48 lore raw

Re: [PATCH v2 25/28] KVM: arm64: Engage NV3 TLBI trap elision

Quoted 21 lines · Marc Zyngier

Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs that only affects the guest's S1 translation. Enable this, with the express condition that the guest isn't NV2 aware, as we otherwise need to trap these TLBIs to deal with VNCR mappings.

---
 arch/arm64/include/asm/kvm_emulate.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +3 -0
index b32870a5e1236..d17b161d4ba3a 100644
@@ -715,6 +715,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
 		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
 		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
 			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+
+			/*
+			 * If the guest is NV2-capable, then we need to see

arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 730

Quoted 10 lines
arch/arm64/include/asm/kvm_emulate.h +10 -0
+			 * all the TLBIs, as configured in HCR_EL2.
+			 * Otherwise, relax the TLBI traps to only TGE=0.
+			 */
+			if (!kvm_has_nv2(vcpu->kvm)) {
+				vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB   |
+							HCRX_EL2_NVnTTLBIS);
+
+				if (kvm_has_feat(kvm, ID_AA64ISAR0_EL1, TLB, OS))
+					vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;;
+			}

I also suspected if this is dead code, but Sashiko had beat me to it :P So from your response in v1, is this code here just for completeness?

It's amusing that we're not able to take advantage of this feature at all!

Thanks,
Wei-Lin Chang

Quoted 6 lines
arch/arm64/include/asm/kvm_emulate.h +0 -0
 		}
 	}
 }
Signature · 1 line
2.47.3
author maintainer
#49 lore raw

Re: [PATCH v2 25/28] KVM: arm64: Engage NV3 TLBI trap elision

Quoted 28 lines · Wei-Lin Chang
Quoted 26 lines · Marc Zyngier

Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs that only affects the guest's S1 translation. Enable this, with the express condition that the guest isn't NV2 aware, as we otherwise need to trap these TLBIs to deal with VNCR mappings.

---
 arch/arm64/include/asm/kvm_emulate.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +8 -0
index b32870a5e1236..d17b161d4ba3a 100644
@@ -715,6 +715,19 @@ static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu)
 		if (cpus_have_final_cap(ARM64_HAS_NV3) &&
 		    vcpu_has_nv(vcpu) && vcpu_el2_e2h_is_set(vcpu)) {
 			vcpu->arch.hcrx_el2 |= HCRX_EL2_NVTGE;
+
+			/*
+			 * If the guest is NV2-capable, then we need to see
+			 * all the TLBIs, as configured in HCR_EL2.
+			 * Otherwise, relax the TLBI traps to only TGE=0.
+			 */
+			if (!kvm_has_nv2(vcpu->kvm)) {
+				vcpu->arch.hcrx_el2 |= (HCRX_EL2_NVnTTLB   |
Quoted 8 lines

arch/arm64/include/asm/kvm_emulate.h static inline void vcpu_set_hcrx(struct kvm_vcpu *vcpu) · line 730

Quoted 5 lines
arch/arm64/include/asm/kvm_emulate.h +5 -0
+							HCRX_EL2_NVnTTLBIS);
+
+				if (kvm_has_feat(kvm, ID_AA64ISAR0_EL1, TLB, OS))
+					vcpu->arch.hcrx_el2 |= HCRX_EL2_NVnTTLBOS;;
+			}

I also suspected if this is dead code, but Sashiko had beat me to it :P So from your response in v1, is this code here just for completeness?

No, this is functional. It is just that we don't have any way to expose to a nested guest something that would be in the outer shareable domain, and not in the inner one.

Quoted 2 lines

It's amusing that we're not able to take advantage of this feature at all!

We can. It is just that there is nothing out there that will exercise it.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#50 lore raw

PATCH v2 26/28 KVM: arm64: Add FEAT_NV3 detection

In reply to Marc Zyngier

Now that everything is in place to engage the FEAT_NV3 fast-path, add the detection code to cpufeature.c.

---
 arch/arm64/kernel/cpufeature.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/arm64/kernel/cpufeature.c +7 -0
index c9c124b0ccc8e..6ae1c816e2010 100644
@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.matches = has_cpuid_feature,
 		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
 	},
+	{
+		.desc = "FEAT_NV3",
+		.capability = ARM64_HAS_NV3,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.matches = has_cpuid_feature,
+		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
+	},
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
2.47.3
#51 lore raw

Re: [PATCH v2 26/28] KVM: arm64: Add FEAT_NV3 detection

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Writes to control-plane system registers (SYS_NVHCR_EL2 and SYS_VNCR_EL2) are missing the architecturally required isb() synchronization barrier. --

--- Patch [26]: [PATCH v2 26/28] KVM: arm64: Add FEAT_NV3 detection --- commit 1e08d22547f3313a77b075620f905436357c4b40
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add FEAT_NV3 detection

Now that everything is in place to engage the FEAT_NV3 fast-path, add the detection code to cpufeature.c.

Quoted 5 lines
arch/arm64/kernel/cpufeature.c +0 -0
index c9c124b0ccc8e..6ae1c816e2010 100644

arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2636

Quoted 10 lines
arch/arm64/kernel/cpufeature.c +7 -0
@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.matches = has_cpuid_feature,
 		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
 	},
+	{
+		.desc = "FEAT_NV3",
+		.capability = ARM64_HAS_NV3,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.matches = has_cpuid_feature,
+		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
+	},

[Severity: High]
This isn't a bug introduced by this patch, but enabling FEAT_NV3 here exposes pre-existing paths where control-plane system registers are written without the architecturally required immediate isb() synchronization barrier.

For example, in arch/arm64/kvm/hyp/vhe/switch.c:__compute_hcr():

	if (cpus_have_final_cap(ARM64_HAS_NV3) && vcpu_el2_e2h_is_set(vcpu))
		write_sysreg_s(__vcpu_sys_reg(vcpu, HCR_EL2), SYS_NVHCR_EL2);

and a bit later in the same function:

		write_sysreg_s(vcpu->arch.ctxt.vncr_array, SYS_VNCR_EL2);
	} else {

Also in arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h:__sysreg_restore_el2_return_state():

	if (is_nested_nv3_ctxt(ctxt_to_vcpu(ctxt)))
		write_sysreg_s(ctxt_sys_reg(ctxt, NVHCR_EL2), SYS_NVHCR_EL2);
}

None of these writes are immediately followed by an isb(). Since they are control-plane system registers, execution continues to evaluate masks or return from the function under an architecturally undefined pipeline state.

Should there be an isb() immediately following these writes to SYS_NVHCR_EL2 and SYS_VNCR_EL2?

Quoted 3 lines
arch/arm64/kernel/cpufeature.c +0 -0
 	{
 		.capability = ARM64_HAS_32BIT_EL0_DO_NOT_USE,
 		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=26
author maintainer
#52 lore raw

Re: [PATCH v2 26/28] KVM: arm64: Add FEAT_NV3 detection

Quoted 28 lines · On Tue, 14 Jul 2026 11:07:03 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Writes to control-plane system registers (SYS_NVHCR_EL2 and SYS_VNCR_EL2) are missing the architecturally required isb() synchronization barrier. --

--- Patch [26]: [PATCH v2 26/28] KVM: arm64: Add FEAT_NV3 detection --- commit 1e08d22547f3313a77b075620f905436357c4b40
Author: Marc Zyngier <[email protected]>

KVM: arm64: Add FEAT_NV3 detection

Now that everything is in place to engage the FEAT_NV3 fast-path, add the detection code to cpufeature.c.

Quoted 12 lines
arch/arm64/kernel/cpufeature.c +4 -0
index c9c124b0ccc8e..6ae1c816e2010 100644
@@ -2627,6 +2627,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.matches = has_cpuid_feature,
 		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV2P1)
 	},
+	{
+		.desc = "FEAT_NV3",
+		.capability = ARM64_HAS_NV3,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
Quoted 8 lines

arch/arm64/kernel/cpufeature.c static const struct arm64_cpu_capabilities arm64_features[] = { · line 2636

Quoted 3 lines
arch/arm64/kernel/cpufeature.c +3 -0
+		.matches = has_cpuid_feature,
+		ARM64_CPUID_FIELDS(ID_AA64MMFR4_EL1, NV_frac, NV3)
+	},

[Severity: High]
This isn't a bug introduced by this patch, but enabling FEAT_NV3 here exposes pre-existing paths where control-plane system registers are written without the architecturally required immediate isb() synchronization barrier.

Plonk.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#53 lore raw

PATCH v2 27/28 KVM: arm64: Expose FEAT_NV3 to guests

In reply to Marc Zyngier

Further enable FEAT_NV3 by making it visible to NV guests.

---
 arch/arm64/kvm/nested.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
arch/arm64/kvm/nested.c +4 -2
index f646894fec9a9..70d605e60cfb3 100644
@@ -1728,7 +1728,7 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 		 * You get EITHER
 		 *
 		 * - FEAT_VHE without FEAT_E2H0
-		 * - FEAT_NV limited to FEAT_NV2(p1)
+		 * - FEAT_NV limited to FEAT_NV2(p1)/NV3
 		 * - HCR_EL2.NV1 being RES0
 		 *
 		 * OR
@@ -1741,7 +1741,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 			val = 0;
 		} else {
 			val &= ID_AA64MMFR4_EL1_NV_frac;
-			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+			if (cpus_have_final_cap(ARM64_HAS_NV3))
+				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV3);
+			else if (cpus_have_final_cap(ARM64_HAS_NV2P1))
 				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
 			else
 				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);
Signature · 1 line
2.47.3
#54 lore raw

Re: [PATCH v2 27/28] KVM: arm64: Expose FEAT_NV3 to guests

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Host nested virtualization capabilities (ID_AA64MMFR4_EL1.NV_frac) are leaked to non-NV guests, breaking live migration across heterogeneous hosts. --

--- Patch [27]: [PATCH v2 27/28] KVM: arm64: Expose FEAT_NV3 to guests --- commit 508d53194bf7ddb4e61f279e50216aa88dd973ad
Author: Marc Zyngier <[email protected]>

KVM: arm64: Expose FEAT_NV3 to guests

Further enable FEAT_NV3 by making it visible to NV guests.

Quoted 4 lines
arch/arm64/kvm/nested.c +0 -0
index f646894fec9a9..70d605e60cfb3 100644

[ ... ]

arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1749

Quoted 11 lines
arch/arm64/kvm/nested.c +3 -1
@@ -1741,7 +1741,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 			val = 0;
 		} else {
 			val &= ID_AA64MMFR4_EL1_NV_frac;
-			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+			if (cpus_have_final_cap(ARM64_HAS_NV3))
+				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV3);
+			else if (cpus_have_final_cap(ARM64_HAS_NV2P1))
 				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
 			else
 				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);

[Severity: High]
This isn't a bug introduced by this patch, but while looking at NV feature exposure here, could ID_AA64MMFR4_EL1.NV_frac leak into non-NV guests?

Looking at __kvm_read_sanitised_id_reg() in arch/arm64/kvm/sys_regs.c, it seems SYS_ID_AA64MMFR4_EL1 is not explicitly masked out for non-NV guests in the switch statement:

	case SYS_ID_MMFR4_EL1:
		val &= ~ID_MMFR4_EL1_CCIDX;
		break;
	}

If a non-NV guest is created on an NV3-capable host, will the host's raw NV_frac leak into the VM's ID_AA64MMFR4_EL1 register?

If this VM is later migrated to a host with only NV2P1, could userspace's attempt to restore the capability via KVM_SET_ONE_REG fail because the destination host's arm64_check_features() rejects the higher NV_frac value?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=27
author maintainer
#55 lore raw

Re: [PATCH v2 27/28] KVM: arm64: Expose FEAT_NV3 to guests

Quoted 45 lines · On Tue, 14 Jul 2026 11:15:59 +0100, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Host nested virtualization capabilities (ID_AA64MMFR4_EL1.NV_frac) are leaked to non-NV guests, breaking live migration across heterogeneous hosts. --

--- Patch [27]: [PATCH v2 27/28] KVM: arm64: Expose FEAT_NV3 to guests --- commit 508d53194bf7ddb4e61f279e50216aa88dd973ad
Author: Marc Zyngier <[email protected]>

KVM: arm64: Expose FEAT_NV3 to guests

Further enable FEAT_NV3 by making it visible to NV guests.

Quoted 4 lines
arch/arm64/kvm/nested.c +0 -0
index f646894fec9a9..70d605e60cfb3 100644

[ ... ]

arch/arm64/kvm/nested.c u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · line 1749

Quoted 11 lines
arch/arm64/kvm/nested.c +3 -1
@@ -1741,7 +1741,9 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val)
 			val = 0;
 		} else {
 			val &= ID_AA64MMFR4_EL1_NV_frac;
-			if (cpus_have_final_cap(ARM64_HAS_NV2P1))
+			if (cpus_have_final_cap(ARM64_HAS_NV3))
+				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV3);
+			else if (cpus_have_final_cap(ARM64_HAS_NV2P1))
 				val = ID_REG_LIMIT_FIELD_ENUM(val, ID_AA64MMFR4_EL1, NV_frac, NV2P1);
 			else
 				val = SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, NV_frac, NV2_ONLY);

[Severity: High]
This isn't a bug introduced by this patch, but while looking at NV feature exposure here, could ID_AA64MMFR4_EL1.NV_frac leak into non-NV guests?

Looking at __kvm_read_sanitised_id_reg() in arch/arm64/kvm/sys_regs.c, it seems SYS_ID_AA64MMFR4_EL1 is not explicitly masked out for non-NV guests in the switch statement:

	case SYS_ID_MMFR4_EL1:
		val &= ~ID_MMFR4_EL1_CCIDX;
		break;
	}
Quoted 6 lines

If a non-NV guest is created on an NV3-capable host, will the host's raw NV_frac leak into the VM's ID_AA64MMFR4_EL1 register?

If this VM is later migrated to a host with only NV2P1, could userspace's attempt to restore the capability via KVM_SET_ONE_REG fail because the destination host's arm64_check_features() rejects the higher NV_frac value?

Yeah, we should ignore this field being written back for a non-NV guest. It won't be evaluated anyway in a non-NV context.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#56 lore raw

PATCH v2 28/28 arm64: Add override for ID_AA64MMFR4_EL1.NV_frac

In reply to Marc Zyngier

In a very unsurprising turn of events, there is a large class of firmware that is totally unable to deal with FEAT_NV3, and doesn't set the required SCR2_EL3.NV3En bit, leading to an UNDEF exception or an unhandled trap to EL3, depending on the implementation.

Allow the unfortunate user to override ID_AA64MMFR4_EL1.NV_frac and get a working system. Hopefully firmware will be fixed before actually HW ships, but I have been there before... :-/

---
 arch/arm64/include/asm/cpufeature.h   |  1 +
 arch/arm64/kernel/cpufeature.c        |  4 +++-
 arch/arm64/kernel/image-vars.h        |  1 +
 arch/arm64/kernel/pi/idreg-override.c | 10 ++++++++++
 4 files changed, 15 insertions(+), 1 deletion(-)
arch/arm64/include/asm/cpufeature.h +1 -0
index a57870fa96db5..a42683af79fb5 100644
@@ -968,6 +968,7 @@ struct arm64_ftr_reg *get_arm64_ftr_reg(u32 sys_id);
 extern struct arm64_ftr_override id_aa64mmfr0_override;
 extern struct arm64_ftr_override id_aa64mmfr1_override;
 extern struct arm64_ftr_override id_aa64mmfr2_override;
+extern struct arm64_ftr_override id_aa64mmfr4_override;
 extern struct arm64_ftr_override id_aa64pfr0_override;
 extern struct arm64_ftr_override id_aa64pfr1_override;
 extern struct arm64_ftr_override id_aa64zfr0_override;
arch/arm64/kernel/cpufeature.c +3 -1
index 6ae1c816e2010..14fbfa8e6b7b5 100644
@@ -785,6 +785,7 @@ static const struct arm64_ftr_bits ftr_raz[] = {
 struct arm64_ftr_override __read_mostly id_aa64mmfr0_override;
 struct arm64_ftr_override __read_mostly id_aa64mmfr1_override;
 struct arm64_ftr_override __read_mostly id_aa64mmfr2_override;
+struct arm64_ftr_override __read_mostly id_aa64mmfr4_override;
 struct arm64_ftr_override __read_mostly id_aa64pfr0_override;
 struct arm64_ftr_override __read_mostly id_aa64pfr1_override;
 struct arm64_ftr_override __read_mostly id_aa64zfr0_override;
@@ -858,7 +859,8 @@ static const struct __ftr_reg_entry {
 	ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2,
 			       &id_aa64mmfr2_override),
 	ARM64_FTR_REG(SYS_ID_AA64MMFR3_EL1, ftr_id_aa64mmfr3),
-	ARM64_FTR_REG(SYS_ID_AA64MMFR4_EL1, ftr_id_aa64mmfr4),
+	ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR4_EL1, ftr_id_aa64mmfr4,
+			       &id_aa64mmfr4_override),
 
 	/* Op1 = 0, CRn = 10, CRm = 4 */
 	ARM64_FTR_REG(SYS_MPAMIDR_EL1, ftr_mpamidr),
arch/arm64/kernel/image-vars.h +1 -0
index d4c7d45ae6bc8..d15c2cb1b0f28 100644
@@ -51,6 +51,7 @@ PI_EXPORT_SYM(id_aa64isar2_override);
 PI_EXPORT_SYM(id_aa64mmfr0_override);
 PI_EXPORT_SYM(id_aa64mmfr1_override);
 PI_EXPORT_SYM(id_aa64mmfr2_override);
+PI_EXPORT_SYM(id_aa64mmfr4_override);
 PI_EXPORT_SYM(id_aa64pfr0_override);
 PI_EXPORT_SYM(id_aa64pfr1_override);
 PI_EXPORT_SYM(id_aa64smfr0_override);
arch/arm64/kernel/pi/idreg-override.c +10 -0
index bc57b290e5e7b..4e47616bcac23 100644
@@ -106,6 +106,15 @@ static const struct ftr_set_desc mmfr2 __prel64_initconst = {
 	},
 };
 
+static const struct ftr_set_desc mmfr4 __prel64_initconst = {
+	.name		= "id_aa64mmfr4",
+	.override	= &id_aa64mmfr4_override,
+	.fields		= {
+		FIELD("nv_frac", ID_AA64MMFR4_EL1_NV_frac_SHIFT, NULL),
+		{}
+	},
+};
+
 static bool __init pfr0_sve_filter(u64 val)
 {
 	/*
@@ -220,6 +229,7 @@ PREL64(const struct ftr_set_desc, reg) regs[] __prel64_initconst = {
 	{ &mmfr0	},
 	{ &mmfr1	},
 	{ &mmfr2	},
+	{ &mmfr4	},
 	{ &pfr0 	},
 	{ &pfr1 	},
 	{ &isar1	},
Signature · 1 line
2.47.3