Fix setting SVE and SME traps in (h)VHE
cover Fix setting SVE and SME traps in (h)VHE
The (re)setting and disabling of SVE/SME trap handling (mostly) done for the hVHE work [*] misses a couple of cases.
This patch series ensures that these traps are disabled on setup and reset. Moreover, it makes the code consistent in using CPACR_EL1 or CPTR_EL2, depending on the mode.
Based on Linux 6.5-rc2.
Cheers,
/fuad
[*] https://lore.kernel.org/all/[email protected]/
Fuad Tabba (5):
KVM: arm64: Use the appropriate feature trap register for SVE at EL2
setup
KVM: arm64: Disable SME Traps for (h)VHE at setup
KVM: arm64: Use the appropriate feature trap register when activating
traps
KVM: arm64: Fix resetting SVE trap values on reset for hVHE
KVM: arm64: Fix resetting SME trap values on reset for (h)VHE
arch/arm64/include/asm/el2_setup.h | 27 +++++++++++++++++++++------ arch/arm64/include/asm/kvm_emulate.h | 8 ++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++- 3 files changed, 34 insertions(+), 7 deletions(-)
base-commit: fdf0eaf11452d72945af31804e2a1048ee1b574c
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
PATCH 1/5 KVM: arm64: Use the appropriate feature trap register for SVE at EL2 setup
Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e., CPTR_EL2 or CPACR_EL1, when setting up SVE feature traps.
--- arch/arm64/include/asm/el2_setup.h | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
arch/arm64/include/asm/el2_setup.h +9 -6
@@ -143,11 +143,12 @@ and x1, x1, #HCR_E2H cbz x1, .LnVHE_\@ mov x0, #(CPACR_EL1_FPEN_EL1EN | CPACR_EL1_FPEN_EL0EN) - b .Lset_cptr_\@ + msr cpacr_el1, x0 + b .Lskip_set_cptr_\@ .LnVHE_\@: mov x0, #0x33ff -.Lset_cptr_\@: msr cptr_el2, x0 // Disable copro. traps to EL2 +.Lskip_set_cptr_\@: .endm/* Disable any fine grained traps */
@@ -268,19 +269,21 @@ check_override id_aa64pfr0, ID_AA64PFR0_EL1_SVE_SHIFT, .Linit_sve_\@, .Lskip_sve_\@, x1, x2 .Linit_sve_\@: /* SVE register access */ - mrs x0, cptr_el2 // Disable SVE traps mrs x1, hcr_el2 and x1, x1, #HCR_E2H cbz x1, .Lcptr_nvhe_\@ -// VHE case +// (h)VHE case + mrs x0, cpacr_el1 // Disable SVE traps orr x0, x0, #(CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN) - b .Lset_cptr_\@ + msr cpacr_el1, x0 + b .Lskip_set_cptr_\@ .Lcptr_nvhe_\@: // nVHE case + mrs x0, cptr_el2 // Disable SVE traps bic x0, x0, #CPTR_EL2_TZ -.Lset_cptr_\@: msr cptr_el2, x0 +.Lskip_set_cptr_\@: isb mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector msr_s SYS_ZCR_EL2, x1 // length for EL1.
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
PATCH 2/5 KVM: arm64: Disable SME Traps for (h)VHE at setup
In reply to Fuad TabbaEnsure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.
--- arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/arm64/include/asm/el2_setup.h +12 -0
@@ -292,9 +292,21 @@ check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2 .Linit_sme_\@: /* SME register access and priority mapping */ + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .Lcptr_nvhe_sme_\@ + +// (h)VHE case + mrs x0, cpacr_el1 // Disable SME traps + orr x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN) + msr cpacr_el1, x0 + b .Lskip_set_cptr_sme_\@ + +.Lcptr_nvhe_sme_\@: // nVHE case mrs x0, cptr_el2 // Disable SME traps bic x0, x0, #CPTR_EL2_TSM msr cptr_el2, x0 +.Lskip_set_cptr_sme_\@: isb mrs x1, sctlr_el2
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
Re: [PATCH v1 2/5] KVM: arm64: Disable SME Traps for (h)VHE at setup
Quoted 13 lines · Fuad Tabba
Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.
Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/arm64/include/asm/el2_setup.h +0
-0
arch/arm64/include/asm/el2_setup.h line 297
Quoted 6 lines
arch/arm64/include/asm/el2_setup.h +3
-0@@ -292,9 +292,21 @@ check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2 .Linit_sme_\@: /* SME register access and priority mapping */ + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .Lcptr_nvhe_sme_\@
Hmm.. Our test for hVHE is repeated enough times, it might make sense for readability to wrap it up in a macro:
.macro check_hvhe pass, fail, tmp
mrs \tmp, hcr_el2 and \tmp, \tmp, #HCR_E2H cbnz \tmp, \pass b \fail
.endm
So long as we tolerate the additional word of bloat for the extra branch :)
Quoted 17 lines
arch/arm64/include/asm/el2_setup.h +8
-0+// (h)VHE case +mrs x0, cpacr_el1// Disable SME traps + orr x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN) + msr cpacr_el1, x0 + b .Lskip_set_cptr_sme_\@ + +.Lcptr_nvhe_sme_\@: // nVHE case mrs x0, cptr_el2 // Disable SME traps bic x0, x0, #CPTR_EL2_TSM msr cptr_el2, x0 +.Lskip_set_cptr_sme_\@: isb mrs x1, sctlr_el2Signature · 1 line
2.41.0.255.g8b1d071c50-goog
Signature · 2 lines
Thanks, Oliver
Re: [PATCH v1 2/5] KVM: arm64: Disable SME Traps for (h)VHE at setup
Hi Oliver,
Quoted 26 lines · Oliver Upton
Quoted 13 lines · Fuad Tabba
Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.
Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/arm64/include/asm/el2_setup.h +0
-0arch/arm64/include/asm/el2_setup.h line 297
Quoted 6 lines
arch/arm64/include/asm/el2_setup.h +3
-0@@ -292,9 +292,21 @@ check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2 .Linit_sme_\@: /* SME register access and priority mapping */ + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .Lcptr_nvhe_sme_\@Hmm.. Our test for hVHE is repeated enough times, it might make sense for readability to wrap it up in a macro:
.macro
check_hvhepass, fail, tmp
Quoted 8 lines
mrs \tmp, hcr_el2 and \tmp, \tmp, #HCR_E2H cbnz \tmp, \pass b \fail.endm
So long as we tolerate the additional word of bloat for the extra branch :)
I did consider that, but went for less bloat. I'll wrap it in a macro for the respin though, and leave it up to the maintainers to decide which one to pick :D
Cheers,
/fuad
Quoted 22 lines
Quoted 16 lines
arch/arm64/include/asm/el2_setup.h +8
-1+// (h)VHE case +mrs x0, cpacr_el1// Disable SME traps + orr x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN) + msr cpacr_el1, x0 + b .Lskip_set_cptr_sme_\@ + +.Lcptr_nvhe_sme_\@: // nVHE case mrs x0, cptr_el2 // Disable SME traps bic x0, x0, #CPTR_EL2_TSM msr cptr_el2, x0 +.Lskip_set_cptr_sme_\@: isb mrs x1, sctlr_el2 --Quoted 2 lines
2.41.0.255.g8b1d071c50-goog
--
Thanks,
Oliver
Re: [PATCH v1 2/5] KVM: arm64: Disable SME Traps for (h)VHE at setup
Hi Oliver,
Quoted 37 lines · Fuad Tabba
Hi Oliver,
Quoted 32 lines · Oliver Upton
Quoted 13 lines · Fuad Tabba
Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.
Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/arm64/include/asm/el2_setup.h +0
-0arch/arm64/include/asm/el2_setup.h line 297
Quoted 6 lines
arch/arm64/include/asm/el2_setup.h +3
-0@@ -292,9 +292,21 @@ check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2 .Linit_sme_\@: /* SME register access and priority mapping */ + mrs x1, hcr_el2 + and x1, x1, #HCR_E2H + cbz x1, .Lcptr_nvhe_sme_\@Hmm.. Our test for hVHE is repeated enough times, it might make sense for readability to wrap it up in a macro:
.macro
check_hvhepass, fail, tmpmrs \tmp, hcr_el2 and \tmp, \tmp, #HCR_E2H cbnz \tmp, \pass b \fail.endm
Quoted 6 lines
Quoted 2 lines
So long as we tolerate the additional word of bloat for the extra branch :)
I did consider that, but went for less bloat. I'll wrap it in a macro for the respin though, and leave it up to the maintainers to decide which one to pick :D
Actually, with this macro I get the same desired effect without affecting the emitted code:
/* Check if running in host at EL2 mode, i.e., (h)VHE. Jumps to fail if not. */ .macro __check_e2h fail, tmp
mrs \tmp, hcr_el2
and \tmp, \tmp, #HCR_E2H
cbz \tmp, \fail
.endm
Unless someone disagrees, I will use this for the respin.
Cheers,
/fuad
Quoted 26 lines
Cheers,
/fuadQuoted 22 lines
Quoted 16 lines
arch/arm64/include/asm/el2_setup.h +8
-1+// (h)VHE case +mrs x0, cpacr_el1// Disable SME traps + orr x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN) + msr cpacr_el1, x0 + b .Lskip_set_cptr_sme_\@ + +.Lcptr_nvhe_sme_\@: // nVHE case mrs x0, cptr_el2 // Disable SME traps bic x0, x0, #CPTR_EL2_TSM msr cptr_el2, x0 +.Lskip_set_cptr_sme_\@: isb mrs x1, sctlr_el2 --Quoted 2 lines
2.41.0.255.g8b1d071c50-goog
--
Thanks,
Oliver
PATCH 3/5 KVM: arm64: Use the appropriate feature trap register when activating traps
In reply to Fuad TabbaUse the architectural feature trap/control register that corresponds to the current KVM mode, i.e., CPTR_EL2 or CPACR_EL1, when activating traps.
--- arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/switch.c +5 -1
@@ -63,7 +63,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)__activate_traps_fpsimd32 (vcpu); } -write_sysreg (val, cptr_el2); +if (has_hvhe ()) +write_sysreg (val, cpacr_el1); +else +write_sysreg (val, cptr_el2); +write_sysreg (__this_cpu_read (kvm_hyp_vector), vbar_el2);if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) {
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
Re: [PATCH v1 3/5] KVM: arm64: Use the appropriate feature trap register when activating traps
Quoted 14 lines · Fuad Tabba
Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e.,
CPTR_EL2orCPACR_EL1, when activating traps.Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)arch/arm64/kvm/hyp/nvhe/switch.c +0
-0
arch/arm64/kvm/hyp/nvhe/switch.c static void __activate_traps(struct kvm_vcpu *vcpu) · line 70
Quoted 9 lines
arch/arm64/kvm/hyp/nvhe/switch.c +5
-1@@ -63,7 +63,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)__activate_traps_fpsimd32 (vcpu); } -write_sysreg (val, cptr_el2); +if (has_hvhe ()) +write_sysreg (val, cpacr_el1); +else +write_sysreg (val, cptr_el2); +
Does it make sense to have an accessor that's shared with kvm_reset_cptr_el2()? Due to the lack of synchronization between the aliases we really want to avoid inconsistently using one or the other.
Signature · 2 lines
Thanks, Oliver
Re: [PATCH v1 3/5] KVM: arm64: Use the appropriate feature trap register when activating traps
Hi Oliver,
Quoted 19 lines · Oliver Upton
Quoted 17 lines · Fuad Tabba
Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e.,
CPTR_EL2orCPACR_EL1, when activating traps.Signed-off-by Fuad Tabba <[email protected]>--- arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)arch/arm64/kvm/hyp/nvhe/switch.c +0
-0@@ -63,7 +63,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)__activate_traps_fpsimd32 (vcpu); }
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/switch.c static void __activate_traps(struct kvm_vcpu *vcpu) · line 70
Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/switch.c +5
-1-write_sysreg (val, cptr_el2); +if (has_hvhe ()) +write_sysreg (val, cpacr_el1); +else +write_sysreg (val, cptr_el2); +Does it make sense to have an accessor that's shared with
kvm_reset_cptr_el2()? Due to the lack of synchronization between the aliases we really want to avoid inconsistently using one or the other.
I think that's a good idea. I'll add one when I respin.
Cheers,
/fuad
Quoted 3 lines
--
Thanks,
Oliver
PATCH 4/5 KVM: arm64: Fix resetting SVE trap values on reset for hVHE
In reply to Fuad TabbaEnsure that SVE traps are disabled for hVHE, if the FPSIMD state isn't owned by the guest, when getting the reset value for the architectural feature control register.
--- arch/arm64/include/asm/kvm_emulate.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +4 -0
@@ -580,6 +580,10 @@ static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu) CPACR_EL1_ZEN_EL1EN); }else if (has_hvhe ()) { val = (CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN ); + +if (vcpu_has_sve (vcpu) && + (vcpu->arch.fp_state !=FP_STATE_GUEST_OWNED )) + val |=CPACR_EL1_ZEN_EL1EN |CPACR_EL1_ZEN_EL0EN ; }else { val =CPTR_NVHE_EL2_RES1 ;
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
PATCH 5/5 KVM: arm64: Fix resetting SME trap values on reset for (h)VHE
In reply to Fuad TabbaEnsure that SME traps are disabled for (h)VHE when getting the reset value for the architectural feature control register.
--- arch/arm64/include/asm/kvm_emulate.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +4 -0
@@ -578,12 +578,16 @@ static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu)if (has_vhe ()) { val = (CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN |CPACR_EL1_ZEN_EL1EN ); +if (cpus_have_final_cap (ARM64_SME )) + val |=CPACR_EL1_SMEN_EL1EN ; }else if (has_hvhe ()) { val = (CPACR_EL1_FPEN_EL0EN |CPACR_EL1_FPEN_EL1EN );if (vcpu_has_sve (vcpu) && (vcpu->arch.fp_state !=FP_STATE_GUEST_OWNED )) val |=CPACR_EL1_ZEN_EL1EN |CPACR_EL1_ZEN_EL0EN ; +if (cpus_have_final_cap (ARM64_SME )) + val |=CPACR_EL1_SMEN_EL1EN |CPACR_EL1_SMEN_EL0EN ; }else { val =CPTR_NVHE_EL2_RES1 ;
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
Marc Zyngier
maintainerHey Fuad,
Many thanks for looking into this, much appreciated. A passing comment below.
Quoted 7 lines · Fuad Tabba
The (re)setting and disabling of SVE/SME trap handling (mostly) done for the hVHE work [*] misses a couple of cases.
This patch series ensures that these traps are disabled on setup and reset. Moreover, it makes the code consistent in using
CPACR_EL1orCPTR_EL2, depending on the mode.
To provide context, this last point isn't really a correctness issue. It only ensures that:
- we don't need to issue a synchronisation when alternating one or the other accessor,
- when running hVHE under NV, we don't trap unnecessarily on accessing CPTR_EL2, while CPACR_EL1 can be used directly without any trap.
I'll have a look at the actual patches shortly.
Thanks again,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Hi Marc,
Quoted 16 lines · Marc Zyngier
Hey Fuad,
Many thanks for looking into this, much appreciated. A passing comment below.
Quoted 7 lines · Fuad Tabba
The (re)setting and disabling of SVE/SME trap handling (mostly) done for the hVHE work [*] misses a couple of cases.
This patch series ensures that these traps are disabled on setup and reset. Moreover, it makes the code consistent in using
CPACR_EL1orCPTR_EL2, depending on the mode.
Quoted 8 lines
To provide context, this last point isn't really a correctness issue. It only ensures that:
- we don't need to issue a synchronisation when alternating one or the other accessor,
- when running hVHE under NV, we don't trap unnecessarily on accessing
CPTR_EL2, whileCPACR_EL1can be used directly without any trap.
Thanks for that. I'll include this in the cover letter when I respin to give more context.
Cheers,
/fuad
Quoted 9 lines
I'll have a look at the actual patches shortly.
Thanks again,
M.--
Without deviation from the norm, progress is not possible.