KVM: arm64: AT + SR accessor fixes
Marc Zyngier
maintainerPATCH 2/2 KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors
In reply to Marc ZyngierVolodymyr reports (again!) that under some circumstances (E2H==0, walking S1 PTs), PAR_EL1 doesn't report the value of the latest walk in the CPU register, but that instead the value is written to the backing store.
Further investigation indicates that the root cause of this is that a group of registers (PAR_EL1, TPIDR*_EL{0,1}) should always be considered as "on CPU", as they are not remapped between EL1 and EL2. We fail to treat them accordingly, and end-up considering that the register (PAR_EL1 in this example) should be written to memory instead of in the register.
While it would be possible to quickly work around it, it is obvious that the way we track these things at the moment is pretty horrible, and could do with some improvement.
Revamp the whole thing by:
- defining a location for a register (memory, cpu), potentially depending on the state of the vcpu
- define a transformation for this register (mapped register, potential translation, special register needing some particular attention)
- convey this information in a structure that can be easily passed around
As a result, the accessors themselves become much simpler, as the state is explicit instead of being driven by hard-to-understand conventions.
We get rid of the "pure EL2 register" notion, which wasn't very useful, and add sanitisation of the values by applying the RESx masks as required, something that was missing so far.
And of course, we add the missing registers to the list, with the indication that they are always loaded.
--- arch/arm64/include/asm/kvm_host.h | 4 +- arch/arm64/kvm/sys_regs.c | 243 +++++++++++++++--------------- 2 files changed, 127 insertions(+), 120 deletions(-)
arch/arm64/include/asm/kvm_host.h +2 -2
@@ -1160,8 +1160,8 @@ u64 kvm_vcpu_apply_reg_masks(const struct kvm_vcpu *, enum vcpu_sysreg, u64); __v; \ }) -u64 vcpu_read_sys_reg (const struct kvm_vcpu *vcpu,int reg); -void vcpu_write_sys_reg (struct kvm_vcpu *vcpu,u64 val,int reg); +u64 vcpu_read_sys_reg (const struct kvm_vcpu *,enum vcpu_sysreg ); +void vcpu_write_sys_reg (struct kvm_vcpu *,u64 ,enum vcpu_sysreg );static inline bool __vcpu_read_sys_reg_from_cpu (int reg,u64 *val) {
arch/arm64/kvm/sys_regs.c +125 -118
293 lines not on this page · raw message
Signature · 1 line
2.39.2
Marc Zyngier
maintainerPATCH 1/2 KVM: arm64: nv: Fix ATS12 handling of single-stage translation
In reply to Marc ZyngierVolodymyr reports that using a Xen DomU as a nested guest (where HCR_EL2.E2H == 0), ATS12 results in a translation that stops at the L2's S1, which isn't something you'd normally expects.
Comparing the code against the spec proves to be illuminating, and suggests that the author of such code must have been tired, cross-eyed, drunk, or maybe all of the above.
The gist of it is that, apart from HCR_EL2.VM or HCR_EL2.DC being 0, only the use of the EL2&0 translation regime limits the walk to S1 only, and that we must finish the S2 walk in any other case. Which solves the above issue, as E2H==0 indicates that ATS12 walks the EL1&0 translation regime.
Explicitly checking for EL2&0 fixes this.
--- arch/arm64/kvm/at.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm64/kvm/at.c +3 -3
@@ -1420,10 +1420,10 @@ void __kvm_at_s12(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)return ;/* -* If we only have a single stage of translation (E2H=0 or -* TGE=1), exit early. Same thing if {VM,DC}=={0,0}. +* If we only have a single stage of translation (EL2&0), exit +* early. Same thing if {VM,DC}=={0,0}. */ -if (!vcpu_el2_e2h_is_set (vcpu) ||vcpu_el2_tge_is_set (vcpu) || +if (compute_translation_regime (vcpu, op) ==TR_EL20 || !(vcpu_read_sys_reg (vcpu,HCR_EL2 ) & (HCR_VM |HCR_DC )))return ;
Signature · 1 line
2.39.2
Marc Zyngier
maintainercover KVM: arm64: AT + SR accessor fixes
Volodymyr having reported[1] a couple of rather interesting bugs while running Xen under KVM, here's a couple of patches to plug these issues:
- a fix for ATS12 stopping the walk at S1 under the wrong conditions
- a much larger fix for the vcpu_{read,write}_sys_reg() accessors, fixing the fate of TPIDR*_EL{0,1} and PAR_EL1, and overall becoming much easier to work with
The latter is a pretty large change, but is worth it IMO as it makes everything much more straightforward.
Volodymyr, I'd very much welcome your feedback on those, as despite my best effort, I didn't manage to even boot Debian's packaging of Xen (Grub just refuses to run *anything* after Xen being installed -- I guess it's not tested at all).
[1] https://lore.kernel.org/r/[email protected]
Marc Zyngier (2):
KVM: arm64: nv: Fix ATS12 handling of single-stage translation
KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors
arch/arm64/include/asm/kvm_host.h | 4 +- arch/arm64/kvm/at.c | 6 +- arch/arm64/kvm/sys_regs.c | 243 +++++++++++++++--------------- 3 files changed, 130 insertions(+), 123 deletions(-)
Signature · 1 line
2.39.2
Re: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors
In reply to Marc ZyngierQuoted 18 lines · Marc Zyngier
arch/arm64/kvm/sys_regs.c +12
-1@@ -144,125 +156,120 @@ static bool get_el2_to_el1_mapping(unsigned int reg,MAPPED_EL2_SYSREG (ZCR_EL2 ,ZCR_EL1 ,NULL );MAPPED_EL2_SYSREG (CONTEXTIDR_EL2 ,CONTEXTIDR_EL1 ,NULL );MAPPED_EL2_SYSREG (SCTLR2_EL2 ,SCTLR2_EL1 ,NULL ); +case CNTHCTL_EL2 : +/* CNTHCTL_EL2 is super special, until we support NV2.1 */ + loc->loc = ((is_hyp_ctxt (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) ? +SR_LOC_SPECIAL :SR_LOC_MEMORY ); +break ; +case TPIDR_EL0 : +case TPIDRRO_EL0 : +case TPIDR_EL1 : +case PAR_EL1 : +/* These registers are always loaded, no matter what */ + loc->loc =SR_LOC_LOADED ; +break ;default : -return false;
arch/arm64/kvm/sys_regs.c static bool get_el2_to_el1_mapping(unsigned int reg, · line 181
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10
-0+/* +* Non-mapped EL2 registers are by definition in memory, but +* we don't need to distinguish them here, as the CPU +* register accessors will bail out and we'll end-up using +* the backing store. +* +* EL1 registers are, however, only loaded if we're +* not in hypervisor context. +*/ + loc->loc =is_hyp_ctxt (vcpu) ?SR_LOC_MEMORY :SR_LOC_LOADED ;
Hmm... I get the feeling that this flow is becoming even more subtle. There's some implicit coupling between this switch statement and the __vcpu_{read,write}_sys_reg_from_cpu() which feels like it could be error prone. Especially since we're gonna lose the WARN() that would inform us if an on-CPU register was actually redirected to memory.
I'm wondering if we need some macro hell containing the block of registers we handle on-CPU and expand that can be expanded into this triage switch case as well as the sysreg accessor.
What you have definitely seems correct, though. I'll twiddle a bit and see if I come up with something, although I imagine what you have is what we'll use in the end anyway.
Thanks,
Oliver
Marc Zyngier
maintainerRe: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors
Hey Oliver,
Thanks for looking into this.
Quoted 28 lines · Oliver Upton
Quoted 26 lines · Marc Zyngier
arch/arm64/kvm/sys_regs.c +20
-1@@ -144,125 +156,120 @@ static bool get_el2_to_el1_mapping(unsigned int reg,MAPPED_EL2_SYSREG (ZCR_EL2 ,ZCR_EL1 ,NULL );MAPPED_EL2_SYSREG (CONTEXTIDR_EL2 ,CONTEXTIDR_EL1 ,NULL );MAPPED_EL2_SYSREG (SCTLR2_EL2 ,SCTLR2_EL1 ,NULL ); +case CNTHCTL_EL2 : +/* CNTHCTL_EL2 is super special, until we support NV2.1 */ + loc->loc = ((is_hyp_ctxt (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) ? +SR_LOC_SPECIAL :SR_LOC_MEMORY ); +break ; +case TPIDR_EL0 : +case TPIDRRO_EL0 : +case TPIDR_EL1 : +case PAR_EL1 : +/* These registers are always loaded, no matter what */ + loc->loc =SR_LOC_LOADED ; +break ;default : -return false; +/* +* Non-mapped EL2 registers are by definition in memory, but +* we don't need to distinguish them here, as the CPU +* register accessors will bail out and we'll end-up using +* the backing store. +* +* EL1 registers are, however, only loaded if we're +* not in hypervisor context.
Quoted 8 lines
arch/arm64/kvm/sys_regs.c static bool get_el2_to_el1_mapping(unsigned int reg, · line 181
Quoted 2 lines
arch/arm64/kvm/sys_regs.c +2
-0+*/ + loc->loc =is_hyp_ctxt (vcpu) ?SR_LOC_MEMORY :SR_LOC_LOADED ;Hmm... I get the feeling that this flow is becoming even more subtle. There's some implicit coupling between this switch statement and the __vcpu_{read,write}_sys_reg_from_cpu() which feels like it could be error prone. Especially since we're gonna lose the
WARN()that would inform us if an on-CPU register was actually redirected to memory.
This implicit behaviour was already present, and nobody noticed it. See how the FGT2 registers are currently missing from the list of "pure" registers. We didn't see the problem because the fallback saves us. This is what decided me to throw away the "pure" annotation and lump both non-remapped EL2 and EL1 registers together.
Quoted 3 lines
I'm wondering if we need some macro hell containing the block of registers we handle on-CPU and expand that can be expanded into this triage switch case as well as the sysreg accessor.
That's an interesting approach. A bit tricky, because we do rely on heavy inlining and constant propagation in the CPU accessors, but maybe there's a way to deal with that...
Quoted 3 lines
What you have definitely seems correct, though. I'll twiddle a bit and see if I come up with something, although I imagine what you have is what we'll use in the end anyway.
I'll have a look in parallel and post my findings, if any.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerRe: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors
In reply to Oliver UptonQuoted 37 lines · Oliver Upton
Quoted 18 lines · Marc Zyngier
arch/arm64/kvm/sys_regs.c +12
-1@@ -144,125 +156,120 @@ static bool get_el2_to_el1_mapping(unsigned int reg,MAPPED_EL2_SYSREG (ZCR_EL2 ,ZCR_EL1 ,NULL );MAPPED_EL2_SYSREG (CONTEXTIDR_EL2 ,CONTEXTIDR_EL1 ,NULL );MAPPED_EL2_SYSREG (SCTLR2_EL2 ,SCTLR2_EL1 ,NULL ); +case CNTHCTL_EL2 : +/* CNTHCTL_EL2 is super special, until we support NV2.1 */ + loc->loc = ((is_hyp_ctxt (vcpu) &&vcpu_el2_e2h_is_set (vcpu)) ? +SR_LOC_SPECIAL :SR_LOC_MEMORY ); +break ; +case TPIDR_EL0 : +case TPIDRRO_EL0 : +case TPIDR_EL1 : +case PAR_EL1 : +/* These registers are always loaded, no matter what */ + loc->loc =SR_LOC_LOADED ; +break ;default : -return false;arch/arm64/kvm/sys_regs.c static bool get_el2_to_el1_mapping(unsigned int reg, · line 181
Quoted 10 lines
arch/arm64/kvm/sys_regs.c +10
-0+/* +* Non-mapped EL2 registers are by definition in memory, but +* we don't need to distinguish them here, as the CPU +* register accessors will bail out and we'll end-up using +* the backing store. +* +* EL1 registers are, however, only loaded if we're +* not in hypervisor context. +*/ + loc->loc =is_hyp_ctxt (vcpu) ?SR_LOC_MEMORY :SR_LOC_LOADED ;Hmm... I get the feeling that this flow is becoming even more subtle. There's some implicit coupling between this switch statement and the __vcpu_{read,write}_sys_reg_from_cpu() which feels like it could be error prone. Especially since we're gonna lose the
WARN()that would inform us if an on-CPU register was actually redirected to memory.
Quoted 7 lines
I'm wondering if we need some macro hell containing the block of registers we handle on-CPU and expand that can be expanded into this triage switch case as well as the sysreg accessor.
What you have definitely seems correct, though. I'll twiddle a bit and see if I come up with something, although I imagine what you have is what we'll use in the end anyway.
My current conclusion is that a macro hack is not really practical, if only because we end-up here from out-of-line C code, and that at this stage we've lost all symbolic information.
We *could* take the nuclear option of re-modelling the sysreg enum as a bunch of #define, similar to the way we deal with vcpu flags, and have accessors for the various bits of information, but that comes with two different problems:
- we don't have a good way to iterate over symbolic registers
- we need to repaint a large portion of the code base
Given that, I've taken another approach, which is to move all these things close together (no more inlining), and add enough WARN_ON()s that you really have to try and game the code to miss something and not get caught. In the process, I found a couple of extra stragglers that are always loaded when running a 32bit guest (the *32_EL2 registers...).
I've pushed the current state on my kvm-arm64/at-fixes-6.17 branch, and I'll try to repost patches over the weekend.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH 2/2] KVM: arm64: Fix vcpu_{read,write}_sys_reg() accessors
Quoted 20 lines · Marc Zyngier
My current conclusion is that a macro hack is not really practical, if only because we end-up here from out-of-line C code, and that at this stage we've lost all symbolic information.
We *could* take the nuclear option of re-modelling the sysreg enum as a bunch of #define, similar to the way we deal with vcpu flags, and have accessors for the various bits of information, but that comes with two different problems:
- we don't have a good way to iterate over symbolic registers
- we need to repaint a large portion of the code base
Given that, I've taken another approach, which is to move all these things close together (no more inlining), and add enough WARN_ON()s that you really have to try and game the code to miss something and not get caught. In the process, I found a couple of extra stragglers that are always loaded when running a 32bit guest (the *32_EL2 registers...).
Quoted 2 lines
I've pushed the current state on my kvm-arm64/at-fixes-6.17 branch, and I'll try to repost patches over the weekend.
Thanks! I've taken a glance at the branch and LGTM. Just wanted to make sure we have sufficient idiotproofing for the next time I misuse this stuff ;-)
Best,
Oliver