Skip to contentSkip to conversation

KVM: arm64: AT + SR accessor fixes

PATCH In review Marc Zyngier 2 participants 7 messages Aug 9, 2025 – Aug 15, 2025

Subsystems (3)
lore
Share & tools
lore
author maintainer
#2 lore raw

PATCH 2/2 KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors

In reply to Marc Zyngier

Volodymyr reports (again!) that under some circumstances (E2H==0, walking S1 PTs), PAR_EL1 doesn't report the value of the latest walk in the CPU register, but that instead the value is written to the backing store.

Further investigation indicates that the root cause of this is that a group of registers (PAR_EL1, TPIDR*_EL{0,1}) should always be considered as "on CPU", as they are not remapped between EL1 and EL2. We fail to treat them accordingly, and end-up considering that the register (PAR_EL1 in this example) should be written to memory instead of in the register.

While it would be possible to quickly work around it, it is obvious that the way we track these things at the moment is pretty horrible, and could do with some improvement.

Revamp the whole thing by:

- defining a location for a register (memory, cpu), potentially depending on the state of the vcpu

- define a transformation for this register (mapped register, potential translation, special register needing some particular attention)

- convey this information in a structure that can be easily passed around

As a result, the accessors themselves become much simpler, as the state is explicit instead of being driven by hard-to-understand conventions.

We get rid of the "pure EL2 register" notion, which wasn't very useful, and add sanitisation of the values by applying the RESx masks as required, something that was missing so far.

And of course, we add the missing registers to the list, with the indication that they are always loaded.

Fixes fedc612314acf ("KVM: arm64: nv: Handle virtual EL2 registers in vcpu_read/write_sys_reg()")
---
 arch/arm64/include/asm/kvm_host.h |   4 +-
 arch/arm64/kvm/sys_regs.c         | 243 +++++++++++++++---------------
 2 files changed, 127 insertions(+), 120 deletions(-)
arch/arm64/include/asm/kvm_host.h +2 -2
index 2f2394cce24ed..a9661d35bfd02 100644
@@ -1160,8 +1160,8 @@ u64 kvm_vcpu_apply_reg_masks(const struct kvm_vcpu *, enum vcpu_sysreg, u64);
 		__v;							\
 	})
 
-u64 vcpu_read_sys_reg(const struct kvm_vcpu *vcpu, int reg);
-void vcpu_write_sys_reg(struct kvm_vcpu *vcpu, u64 val, int reg);
+u64 vcpu_read_sys_reg(const struct kvm_vcpu *, enum vcpu_sysreg);
+void vcpu_write_sys_reg(struct kvm_vcpu *, u64, enum vcpu_sysreg);
 
 static inline bool __vcpu_read_sys_reg_from_cpu(int reg, u64 *val)
 {
arch/arm64/kvm/sys_regs.c +125 -118

293 lines not on this page · raw message

Signature · 1 line
2.39.2
author maintainer
#1 lore raw

PATCH 1/2 KVM: arm64: nv: Fix ATS12 handling of single-stage translation

In reply to Marc Zyngier

Volodymyr reports that using a Xen DomU as a nested guest (where HCR_EL2.E2H == 0), ATS12 results in a translation that stops at the L2's S1, which isn't something you'd normally expects.

Comparing the code against the spec proves to be illuminating, and suggests that the author of such code must have been tired, cross-eyed, drunk, or maybe all of the above.

The gist of it is that, apart from HCR_EL2.VM or HCR_EL2.DC being 0, only the use of the EL2&0 translation regime limits the walk to S1 only, and that we must finish the S2 walk in any other case. Which solves the above issue, as E2H==0 indicates that ATS12 walks the EL1&0 translation regime.

Explicitly checking for EL2&0 fixes this.

Fixes be04cebf3e788 ("KVM: arm64: nv: Add emulation of AT S12E{0,1}{R,W}")
---
 arch/arm64/kvm/at.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm64/kvm/at.c +3 -3
index 0e56105339493..d71ca4ddc9d1e 100644
@@ -1420,10 +1420,10 @@ void __kvm_at_s12(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
 		return;
 
 	/*
-	 * If we only have a single stage of translation (E2H=0 or
-	 * TGE=1), exit early. Same thing if {VM,DC}=={0,0}.
+	 * If we only have a single stage of translation (EL2&0), exit
+	 * early. Same thing if {VM,DC}=={0,0}.
 	 */
-	if (!vcpu_el2_e2h_is_set(vcpu) || vcpu_el2_tge_is_set(vcpu) ||
+	if (compute_translation_regime(vcpu, op) == TR_EL20 ||
 	    !(vcpu_read_sys_reg(vcpu, HCR_EL2) & (HCR_VM | HCR_DC)))
 		return;
 
Signature · 1 line
2.39.2

Marc Zyngier

author maintainer
#0 lore raw

cover KVM: arm64: AT + SR accessor fixes

Volodymyr having reported[1] a couple of rather interesting bugs while running Xen under KVM, here's a couple of patches to plug these issues:

- a fix for ATS12 stopping the walk at S1 under the wrong conditions

- a much larger fix for the vcpu_{read,write}_sys_reg() accessors, fixing the fate of TPIDR*_EL{0,1} and PAR_EL1, and overall becoming much easier to work with

The latter is a pretty large change, but is worth it IMO as it makes everything much more straightforward.

Volodymyr, I'd very much welcome your feedback on those, as despite my best effort, I didn't manage to even boot Debian's packaging of Xen (Grub just refuses to run *anything* after Xen being installed -- I guess it's not tested at all).

[1] https://lore.kernel.org/r/[email protected]

Marc Zyngier (2):
KVM: arm64: nv: Fix ATS12 handling of single-stage translation
KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors

 arch/arm64/include/asm/kvm_host.h |   4 +-
 arch/arm64/kvm/at.c               |   6 +-
 arch/arm64/kvm/sys_regs.c         | 243 +++++++++++++++---------------
 3 files changed, 130 insertions(+), 123 deletions(-)
Signature · 1 line
2.39.2
#3 lore raw

Re: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors

In reply to Marc Zyngier
Quoted 18 lines · Marc Zyngier
arch/arm64/kvm/sys_regs.c +12 -1
@@ -144,125 +156,120 @@ static bool get_el2_to_el1_mapping(unsigned int reg,
 		MAPPED_EL2_SYSREG(ZCR_EL2,     ZCR_EL1,     NULL	     );
 		MAPPED_EL2_SYSREG(CONTEXTIDR_EL2, CONTEXTIDR_EL1, NULL	     );
 		MAPPED_EL2_SYSREG(SCTLR2_EL2,  SCTLR2_EL1,  NULL	     );
+	case CNTHCTL_EL2:
+		/* CNTHCTL_EL2 is super special, until we support NV2.1 */
+		loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
+			    SR_LOC_SPECIAL : SR_LOC_MEMORY);
+		break;
+	case TPIDR_EL0:
+	case TPIDRRO_EL0:
+	case TPIDR_EL1:
+	case PAR_EL1:
+		/* These registers are always loaded, no matter what */
+		loc->loc = SR_LOC_LOADED;
+		break;
 	default:
-		return false;

arch/arm64/kvm/sys_regs.c static bool get_el2_to_el1_mapping(unsigned int reg, · line 181

Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10 -0
+		/*
+		 * Non-mapped EL2 registers are by definition in memory, but
+		 * we don't need to distinguish them here, as the CPU
+		 * register accessors will bail out and we'll end-up using
+		 * the backing store.
+		 *
+		 * EL1 registers are, however, only loaded if we're
+		 * not in hypervisor context.
+		 */
+		loc->loc = is_hyp_ctxt(vcpu) ? SR_LOC_MEMORY : SR_LOC_LOADED;

Hmm... I get the feeling that this flow is becoming even more subtle. There's some implicit coupling between this switch statement and the __vcpu_{read,write}_sys_reg_from_cpu() which feels like it could be error prone. Especially since we're gonna lose the WARN() that would inform us if an on-CPU register was actually redirected to memory.

I'm wondering if we need some macro hell containing the block of registers we handle on-CPU and expand that can be expanded into this triage switch case as well as the sysreg accessor.

What you have definitely seems correct, though. I'll twiddle a bit and see if I come up with something, although I imagine what you have is what we'll use in the end anyway.

Thanks,
Oliver

author maintainer
#4 lore raw

Re: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors

Hey Oliver,

Thanks for looking into this.

Quoted 28 lines · Oliver Upton
Quoted 26 lines · Marc Zyngier
arch/arm64/kvm/sys_regs.c +20 -1
@@ -144,125 +156,120 @@ static bool get_el2_to_el1_mapping(unsigned int reg,
 		MAPPED_EL2_SYSREG(ZCR_EL2,     ZCR_EL1,     NULL	     );
 		MAPPED_EL2_SYSREG(CONTEXTIDR_EL2, CONTEXTIDR_EL1, NULL	     );
 		MAPPED_EL2_SYSREG(SCTLR2_EL2,  SCTLR2_EL1,  NULL	     );
+	case CNTHCTL_EL2:
+		/* CNTHCTL_EL2 is super special, until we support NV2.1 */
+		loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
+			    SR_LOC_SPECIAL : SR_LOC_MEMORY);
+		break;
+	case TPIDR_EL0:
+	case TPIDRRO_EL0:
+	case TPIDR_EL1:
+	case PAR_EL1:
+		/* These registers are always loaded, no matter what */
+		loc->loc = SR_LOC_LOADED;
+		break;
 	default:
-		return false;
+		/*
+		 * Non-mapped EL2 registers are by definition in memory, but
+		 * we don't need to distinguish them here, as the CPU
+		 * register accessors will bail out and we'll end-up using
+		 * the backing store.
+		 *
+		 * EL1 registers are, however, only loaded if we're
+		 * not in hypervisor context.
Quoted 8 lines

arch/arm64/kvm/sys_regs.c static bool get_el2_to_el1_mapping(unsigned int reg, · line 181

Quoted 2 lines
arch/arm64/kvm/sys_regs.c +2 -0
+		 */
+		loc->loc = is_hyp_ctxt(vcpu) ? SR_LOC_MEMORY : SR_LOC_LOADED;

Hmm... I get the feeling that this flow is becoming even more subtle. There's some implicit coupling between this switch statement and the __vcpu_{read,write}_sys_reg_from_cpu() which feels like it could be error prone. Especially since we're gonna lose the WARN() that would inform us if an on-CPU register was actually redirected to memory.

This implicit behaviour was already present, and nobody noticed it. See how the FGT2 registers are currently missing from the list of "pure" registers. We didn't see the problem because the fallback saves us. This is what decided me to throw away the "pure" annotation and lump both non-remapped EL2 and EL1 registers together.

Quoted 3 lines

I'm wondering if we need some macro hell containing the block of registers we handle on-CPU and expand that can be expanded into this triage switch case as well as the sysreg accessor.

That's an interesting approach. A bit tricky, because we do rely on heavy inlining and constant propagation in the CPU accessors, but maybe there's a way to deal with that...

Quoted 3 lines

What you have definitely seems correct, though. I'll twiddle a bit and see if I come up with something, although I imagine what you have is what we'll use in the end anyway.

I'll have a look in parallel and post my findings, if any.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author maintainer
#5 lore raw

Re: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors

In reply to Oliver Upton
Quoted 37 lines · Oliver Upton
Quoted 18 lines · Marc Zyngier
arch/arm64/kvm/sys_regs.c +12 -1
@@ -144,125 +156,120 @@ static bool get_el2_to_el1_mapping(unsigned int reg,
 		MAPPED_EL2_SYSREG(ZCR_EL2,     ZCR_EL1,     NULL	     );
 		MAPPED_EL2_SYSREG(CONTEXTIDR_EL2, CONTEXTIDR_EL1, NULL	     );
 		MAPPED_EL2_SYSREG(SCTLR2_EL2,  SCTLR2_EL1,  NULL	     );
+	case CNTHCTL_EL2:
+		/* CNTHCTL_EL2 is super special, until we support NV2.1 */
+		loc->loc = ((is_hyp_ctxt(vcpu) && vcpu_el2_e2h_is_set(vcpu)) ?
+			    SR_LOC_SPECIAL : SR_LOC_MEMORY);
+		break;
+	case TPIDR_EL0:
+	case TPIDRRO_EL0:
+	case TPIDR_EL1:
+	case PAR_EL1:
+		/* These registers are always loaded, no matter what */
+		loc->loc = SR_LOC_LOADED;
+		break;
 	default:
-		return false;

arch/arm64/kvm/sys_regs.c static bool get_el2_to_el1_mapping(unsigned int reg, · line 181

Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10 -0
+		/*
+		 * Non-mapped EL2 registers are by definition in memory, but
+		 * we don't need to distinguish them here, as the CPU
+		 * register accessors will bail out and we'll end-up using
+		 * the backing store.
+		 *
+		 * EL1 registers are, however, only loaded if we're
+		 * not in hypervisor context.
+		 */
+		loc->loc = is_hyp_ctxt(vcpu) ? SR_LOC_MEMORY : SR_LOC_LOADED;

Hmm... I get the feeling that this flow is becoming even more subtle. There's some implicit coupling between this switch statement and the __vcpu_{read,write}_sys_reg_from_cpu() which feels like it could be error prone. Especially since we're gonna lose the WARN() that would inform us if an on-CPU register was actually redirected to memory.

Quoted 7 lines

I'm wondering if we need some macro hell containing the block of registers we handle on-CPU and expand that can be expanded into this triage switch case as well as the sysreg accessor.

What you have definitely seems correct, though. I'll twiddle a bit and see if I come up with something, although I imagine what you have is what we'll use in the end anyway.

My current conclusion is that a macro hack is not really practical, if only because we end-up here from out-of-line C code, and that at this stage we've lost all symbolic information.

We *could* take the nuclear option of re-modelling the sysreg enum as a bunch of #define, similar to the way we deal with vcpu flags, and have accessors for the various bits of information, but that comes with two different problems:

- we don't have a good way to iterate over symbolic registers

- we need to repaint a large portion of the code base

Given that, I've taken another approach, which is to move all these things close together (no more inlining), and add enough WARN_ON()s that you really have to try and game the code to miss something and not get caught. In the process, I found a couple of extra stragglers that are always loaded when running a 32bit guest (the *32_EL2 registers...).

I've pushed the current state on my kvm-arm64/at-fixes-6.17 branch, and I'll try to repost patches over the weekend.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#6 lore raw

Re: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors

Quoted 20 lines · Marc Zyngier

My current conclusion is that a macro hack is not really practical, if only because we end-up here from out-of-line C code, and that at this stage we've lost all symbolic information.

We *could* take the nuclear option of re-modelling the sysreg enum as a bunch of #define, similar to the way we deal with vcpu flags, and have accessors for the various bits of information, but that comes with two different problems:

- we don't have a good way to iterate over symbolic registers

- we need to repaint a large portion of the code base

Given that, I've taken another approach, which is to move all these things close together (no more inlining), and add enough WARN_ON()s that you really have to try and game the code to miss something and not get caught. In the process, I found a couple of extra stragglers that are always loaded when running a 32bit guest (the *32_EL2 registers...).

Quoted 2 lines

I've pushed the current state on my kvm-arm64/at-fixes-6.17 branch, and I'll try to repost patches over the weekend.

Thanks! I've taken a glance at the branch and LGTM. Just wanted to make sure we have sufficient idiotproofing for the next time I misuse this stuff ;-)

Best,
Oliver