Skip to contentSkip to conversation

Fix setting SVE and SME traps in (h)VHE

PATCH Superseded Fuad Tabba 3 participants 13 messages Jul 19, 2023 – Jul 20, 2023

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Fuad Tabba

author
#0 lore raw

cover Fix setting SVE and SME traps in (h)VHE

The (re)setting and disabling of SVE/SME trap handling (mostly) done for the hVHE work [*] misses a couple of cases.

This patch series ensures that these traps are disabled on setup and reset. Moreover, it makes the code consistent in using CPACR_EL1 or CPTR_EL2, depending on the mode.

Based on Linux 6.5-rc2.

Cheers,
/fuad

[*] https://lore.kernel.org/all/[email protected]/

Fuad Tabba (5):
KVM: arm64: Use the appropriate feature trap register for SVE at EL2
setup
KVM: arm64: Disable SME Traps for (h)VHE at setup
KVM: arm64: Use the appropriate feature trap register when activating
traps
KVM: arm64: Fix resetting SVE trap values on reset for hVHE
KVM: arm64: Fix resetting SME trap values on reset for (h)VHE

 arch/arm64/include/asm/el2_setup.h   | 27 +++++++++++++++++++++------
 arch/arm64/include/asm/kvm_emulate.h |  8 ++++++++
 arch/arm64/kvm/hyp/nvhe/switch.c     |  6 +++++-
 3 files changed, 34 insertions(+), 7 deletions(-)

base-commit: fdf0eaf11452d72945af31804e2a1048ee1b574c

Signature · 1 line
2.41.0.255.g8b1d071c50-goog
author
#1 lore raw

PATCH 1/5 KVM: arm64: Use the appropriate feature trap register for SVE at EL2 setup

Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e., CPTR_EL2 or CPACR_EL1, when setting up SVE feature traps.

---
 arch/arm64/include/asm/el2_setup.h | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
arch/arm64/include/asm/el2_setup.h +9 -6
index 8e5ffb58f83e..c106b31d776c 100644
@@ -143,11 +143,12 @@
 	and	x1, x1, #HCR_E2H
 	cbz	x1, .LnVHE_\@
 	mov	x0, #(CPACR_EL1_FPEN_EL1EN | CPACR_EL1_FPEN_EL0EN)
-	b	.Lset_cptr_\@
+	msr	cpacr_el1, x0
+	b	.Lskip_set_cptr_\@
 .LnVHE_\@:
 	mov	x0, #0x33ff
-.Lset_cptr_\@:
 	msr	cptr_el2, x0			// Disable copro. traps to EL2
+.Lskip_set_cptr_\@:
 .endm
 
 /* Disable any fine grained traps */
@@ -268,19 +269,21 @@
 	check_override id_aa64pfr0, ID_AA64PFR0_EL1_SVE_SHIFT, .Linit_sve_\@, .Lskip_sve_\@, x1, x2
 
 .Linit_sve_\@:	/* SVE register access */
-	mrs	x0, cptr_el2			// Disable SVE traps
 	mrs	x1, hcr_el2
 	and	x1, x1, #HCR_E2H
 	cbz	x1, .Lcptr_nvhe_\@
 
-	// VHE case
+	// (h)VHE case
+	mrs	x0, cpacr_el1			// Disable SVE traps
 	orr	x0, x0, #(CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
-	b	.Lset_cptr_\@
+	msr	cpacr_el1, x0
+	b	.Lskip_set_cptr_\@
 
 .Lcptr_nvhe_\@: // nVHE case
+	mrs	x0, cptr_el2			// Disable SVE traps
 	bic	x0, x0, #CPTR_EL2_TZ
-.Lset_cptr_\@:
 	msr	cptr_el2, x0
+.Lskip_set_cptr_\@:
 	isb
 	mov	x1, #ZCR_ELx_LEN_MASK		// SVE: Enable full vector
 	msr_s	SYS_ZCR_EL2, x1			// length for EL1.
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
author
#2 lore raw

PATCH 2/5 KVM: arm64: Disable SME Traps for (h)VHE at setup

In reply to Fuad Tabba

Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.

---
 arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/arm64/include/asm/el2_setup.h +12 -0
index c106b31d776c..7001b6db3ccc 100644
@@ -292,9 +292,21 @@
 	check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2
 
 .Linit_sme_\@:	/* SME register access and priority mapping */
+	mrs	x1, hcr_el2
+	and	x1, x1, #HCR_E2H
+	cbz	x1, .Lcptr_nvhe_sme_\@
+
+	// (h)VHE case
+	mrs	x0, cpacr_el1			// Disable SME traps
+	orr	x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN)
+	msr	cpacr_el1, x0
+	b	.Lskip_set_cptr_sme_\@
+
+.Lcptr_nvhe_sme_\@: // nVHE case
 	mrs	x0, cptr_el2			// Disable SME traps
 	bic	x0, x0, #CPTR_EL2_TSM
 	msr	cptr_el2, x0
+.Lskip_set_cptr_sme_\@:
 	isb
 
 	mrs	x1, sctlr_el2
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
#3 lore raw

Re: [PATCH v1 2/5] KVM: arm64: Disable SME Traps for (h)VHE at setup

Quoted 13 lines · Fuad Tabba

Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.

---
 arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/arm64/include/asm/el2_setup.h +0 -0
index c106b31d776c..7001b6db3ccc 100644

arch/arm64/include/asm/el2_setup.h line 297

Quoted 6 lines
arch/arm64/include/asm/el2_setup.h +3 -0
@@ -292,9 +292,21 @@
 	check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2
 
 .Linit_sme_\@:	/* SME register access and priority mapping */
+	mrs	x1, hcr_el2
+	and	x1, x1, #HCR_E2H
+	cbz	x1, .Lcptr_nvhe_sme_\@

Hmm.. Our test for hVHE is repeated enough times, it might make sense for readability to wrap it up in a macro:

.macro check_hvhe pass, fail, tmp

	mrs	\tmp, hcr_el2
	and	\tmp, \tmp, #HCR_E2H
	cbnz	\tmp, \pass
	b	\fail

.endm

So long as we tolerate the additional word of bloat for the extra branch :)

Quoted 17 lines
arch/arm64/include/asm/el2_setup.h +8 -0
+	// (h)VHE case
+	mrs	x0, cpacr_el1			// Disable SME traps
+	orr	x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN)
+	msr	cpacr_el1, x0
+	b	.Lskip_set_cptr_sme_\@
+
+.Lcptr_nvhe_sme_\@: // nVHE case
 	mrs	x0, cptr_el2			// Disable SME traps
 	bic	x0, x0, #CPTR_EL2_TSM
 	msr	cptr_el2, x0
+.Lskip_set_cptr_sme_\@:
 	isb
 
 	mrs	x1, sctlr_el2
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
Signature · 2 lines
Thanks,
Oliver
author
#4 lore raw

Re: [PATCH v1 2/5] KVM: arm64: Disable SME Traps for (h)VHE at setup

Hi Oliver,

Quoted 26 lines · Oliver Upton
Quoted 13 lines · Fuad Tabba

Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.

---
 arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/arm64/include/asm/el2_setup.h +0 -0
index c106b31d776c..7001b6db3ccc 100644

arch/arm64/include/asm/el2_setup.h line 297

Quoted 6 lines
arch/arm64/include/asm/el2_setup.h +3 -0
@@ -292,9 +292,21 @@
      check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2
 
 .Linit_sme_\@:       /* SME register access and priority mapping */
+     mrs     x1, hcr_el2
+     and     x1, x1, #HCR_E2H
+     cbz     x1, .Lcptr_nvhe_sme_\@

Hmm.. Our test for hVHE is repeated enough times, it might make sense for readability to wrap it up in a macro:

.macro check_hvhe pass, fail, tmp

Quoted 8 lines
        mrs     \tmp, hcr_el2
        and     \tmp, \tmp, #HCR_E2H
        cbnz    \tmp, \pass
        b       \fail

.endm

So long as we tolerate the additional word of bloat for the extra branch :)

I did consider that, but went for less bloat. I'll wrap it in a macro for the respin though, and leave it up to the maintainers to decide which one to pick :D

Cheers,
/fuad

Quoted 22 lines
Quoted 16 lines
arch/arm64/include/asm/el2_setup.h +8 -1
+     // (h)VHE case
+     mrs     x0, cpacr_el1                   // Disable SME traps
+     orr     x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN)
+     msr     cpacr_el1, x0
+     b       .Lskip_set_cptr_sme_\@
+
+.Lcptr_nvhe_sme_\@: // nVHE case
      mrs     x0, cptr_el2                    // Disable SME traps
      bic     x0, x0, #CPTR_EL2_TSM
      msr     cptr_el2, x0
+.Lskip_set_cptr_sme_\@:
      isb
 
      mrs     x1, sctlr_el2
--
Quoted 2 lines

2.41.0.255.g8b1d071c50-goog

--
Thanks,
Oliver

author
#5 lore raw

Re: [PATCH v1 2/5] KVM: arm64: Disable SME Traps for (h)VHE at setup

Hi Oliver,

Quoted 37 lines · Fuad Tabba

Hi Oliver,

Quoted 32 lines · Oliver Upton
Quoted 13 lines · Fuad Tabba

Ensure that SME traps are disabled for (h)VHE when setting up EL2, as they are for nVHE.

---
 arch/arm64/include/asm/el2_setup.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/arm64/include/asm/el2_setup.h +0 -0
index c106b31d776c..7001b6db3ccc 100644

arch/arm64/include/asm/el2_setup.h line 297

Quoted 6 lines
arch/arm64/include/asm/el2_setup.h +3 -0
@@ -292,9 +292,21 @@
      check_override id_aa64pfr1, ID_AA64PFR1_EL1_SME_SHIFT, .Linit_sme_\@, .Lskip_sme_\@, x1, x2
 
 .Linit_sme_\@:       /* SME register access and priority mapping */
+     mrs     x1, hcr_el2
+     and     x1, x1, #HCR_E2H
+     cbz     x1, .Lcptr_nvhe_sme_\@

Hmm.. Our test for hVHE is repeated enough times, it might make sense for readability to wrap it up in a macro:

.macro check_hvhe pass, fail, tmp

        mrs     \tmp, hcr_el2
        and     \tmp, \tmp, #HCR_E2H
        cbnz    \tmp, \pass
        b       \fail

.endm

Quoted 6 lines
Quoted 2 lines

So long as we tolerate the additional word of bloat for the extra branch :)

I did consider that, but went for less bloat. I'll wrap it in a macro for the respin though, and leave it up to the maintainers to decide which one to pick :D

Actually, with this macro I get the same desired effect without affecting the emitted code:

/* Check if running in host at EL2 mode, i.e., (h)VHE. Jumps to fail if not. */ .macro __check_e2h fail, tmp
mrs \tmp, hcr_el2
and \tmp, \tmp, #HCR_E2H
cbz \tmp, \fail
.endm

Unless someone disagrees, I will use this for the respin.

Cheers,
/fuad

Quoted 26 lines

Cheers,
/fuad

Quoted 22 lines
Quoted 16 lines
arch/arm64/include/asm/el2_setup.h +8 -1
+     // (h)VHE case
+     mrs     x0, cpacr_el1                   // Disable SME traps
+     orr     x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN)
+     msr     cpacr_el1, x0
+     b       .Lskip_set_cptr_sme_\@
+
+.Lcptr_nvhe_sme_\@: // nVHE case
      mrs     x0, cptr_el2                    // Disable SME traps
      bic     x0, x0, #CPTR_EL2_TSM
      msr     cptr_el2, x0
+.Lskip_set_cptr_sme_\@:
      isb
 
      mrs     x1, sctlr_el2
--
Quoted 2 lines

2.41.0.255.g8b1d071c50-goog

--
Thanks,
Oliver

author
#6 lore raw

PATCH 3/5 KVM: arm64: Use the appropriate feature trap register when activating traps

In reply to Fuad Tabba

Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e., CPTR_EL2 or CPACR_EL1, when activating traps.

---
 arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/switch.c +5 -1
index 0a6271052def..e5ea517bab9c 100644
@@ -63,7 +63,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
 		__activate_traps_fpsimd32(vcpu);
 	}
 
-	write_sysreg(val, cptr_el2);
+	if (has_hvhe())
+		write_sysreg(val, cpacr_el1);
+	else
+		write_sysreg(val, cptr_el2);
+
 	write_sysreg(__this_cpu_read(kvm_hyp_vector), vbar_el2);
 
 	if (cpus_have_final_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
#7 lore raw

Re: [PATCH v1 3/5] KVM: arm64: Use the appropriate feature trap register when activating traps

Quoted 14 lines · Fuad Tabba

Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e., CPTR_EL2 or CPACR_EL1, when activating traps.

---
 arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/switch.c +0 -0
index 0a6271052def..e5ea517bab9c 100644

arch/arm64/kvm/hyp/nvhe/switch.c static void __activate_traps(struct kvm_vcpu *vcpu) · line 70

Quoted 9 lines
arch/arm64/kvm/hyp/nvhe/switch.c +5 -1
@@ -63,7 +63,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
 		__activate_traps_fpsimd32(vcpu);
 	}
 
-	write_sysreg(val, cptr_el2);
+	if (has_hvhe())
+		write_sysreg(val, cpacr_el1);
+	else
+		write_sysreg(val, cptr_el2);
+

Does it make sense to have an accessor that's shared with kvm_reset_cptr_el2()? Due to the lack of synchronization between the aliases we really want to avoid inconsistently using one or the other.

Signature · 2 lines
Thanks,
Oliver
author
#8 lore raw

Re: [PATCH v1 3/5] KVM: arm64: Use the appropriate feature trap register when activating traps

Hi Oliver,

Quoted 19 lines · Oliver Upton
Quoted 17 lines · Fuad Tabba

Use the architectural feature trap/control register that corresponds to the current KVM mode, i.e., CPTR_EL2 or CPACR_EL1, when activating traps.

---
 arch/arm64/kvm/hyp/nvhe/switch.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/switch.c +0 -0
index 0a6271052def..e5ea517bab9c 100644
@@ -63,7 +63,11 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
              __activate_traps_fpsimd32(vcpu);
      }
 
Quoted 10 lines

arch/arm64/kvm/hyp/nvhe/switch.c static void __activate_traps(struct kvm_vcpu *vcpu) · line 70

Quoted 6 lines
arch/arm64/kvm/hyp/nvhe/switch.c +5 -1
-     write_sysreg(val, cptr_el2);
+     if (has_hvhe())
+             write_sysreg(val, cpacr_el1);
+     else
+             write_sysreg(val, cptr_el2);
+

Does it make sense to have an accessor that's shared with kvm_reset_cptr_el2()? Due to the lack of synchronization between the aliases we really want to avoid inconsistently using one or the other.

I think that's a good idea. I'll add one when I respin.

Cheers,
/fuad

Quoted 3 lines

--
Thanks,
Oliver

author
#9 lore raw

PATCH 4/5 KVM: arm64: Fix resetting SVE trap values on reset for hVHE

In reply to Fuad Tabba

Ensure that SVE traps are disabled for hVHE, if the FPSIMD state isn't owned by the guest, when getting the reset value for the architectural feature control register.

Fixes 75c76ab5a641 ("KVM: arm64: Rework CPTR_EL2 programming for HVHE configuration")
---
 arch/arm64/include/asm/kvm_emulate.h | 4 ++++
 1 file changed, 4 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +4 -0
index efc0b45d79c3..997c0ae72204 100644
@@ -580,6 +580,10 @@ static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu)
 		       CPACR_EL1_ZEN_EL1EN);
 	} else if (has_hvhe()) {
 		val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN);
+
+		if (vcpu_has_sve(vcpu) &&
+		    (vcpu->arch.fp_state != FP_STATE_GUEST_OWNED))
+			val |= CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN;
 	} else {
 		val = CPTR_NVHE_EL2_RES1;
 
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
author
#10 lore raw

PATCH 5/5 KVM: arm64: Fix resetting SME trap values on reset for (h)VHE

In reply to Fuad Tabba

Ensure that SME traps are disabled for (h)VHE when getting the reset value for the architectural feature control register.

Fixes 75c76ab5a641 ("KVM: arm64: Rework CPTR_EL2 programming for HVHE configuration")
---
 arch/arm64/include/asm/kvm_emulate.h | 4 ++++
 1 file changed, 4 insertions(+)
arch/arm64/include/asm/kvm_emulate.h +4 -0
index 997c0ae72204..c37584f77f4f 100644
@@ -578,12 +578,16 @@ static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu)
 	if (has_vhe()) {
 		val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN |
 		       CPACR_EL1_ZEN_EL1EN);
+		if (cpus_have_final_cap(ARM64_SME))
+			val |= CPACR_EL1_SMEN_EL1EN;
 	} else if (has_hvhe()) {
 		val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN);
 
 		if (vcpu_has_sve(vcpu) &&
 		    (vcpu->arch.fp_state != FP_STATE_GUEST_OWNED))
 			val |= CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN;
+		if (cpus_have_final_cap(ARM64_SME))
+			val |= CPACR_EL1_SMEN_EL1EN | CPACR_EL1_SMEN_EL0EN;
 	} else {
 		val = CPTR_NVHE_EL2_RES1;
 
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
maintainer
#11 lore raw
In reply to Fuad Tabba

Hey Fuad,

Many thanks for looking into this, much appreciated. A passing comment below.

Quoted 7 lines · Fuad Tabba

The (re)setting and disabling of SVE/SME trap handling (mostly) done for the hVHE work [*] misses a couple of cases.

This patch series ensures that these traps are disabled on setup and reset. Moreover, it makes the code consistent in using CPACR_EL1 or CPTR_EL2, depending on the mode.

To provide context, this last point isn't really a correctness issue. It only ensures that:

- we don't need to issue a synchronisation when alternating one or the other accessor,

- when running hVHE under NV, we don't trap unnecessarily on accessing CPTR_EL2, while CPACR_EL1 can be used directly without any trap.

I'll have a look at the actual patches shortly.

Thanks again,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author
#12 lore raw

Hi Marc,

Quoted 16 lines · Marc Zyngier

Hey Fuad,

Many thanks for looking into this, much appreciated. A passing comment below.

Quoted 7 lines · Fuad Tabba

The (re)setting and disabling of SVE/SME trap handling (mostly) done for the hVHE work [*] misses a couple of cases.

This patch series ensures that these traps are disabled on setup and reset. Moreover, it makes the code consistent in using CPACR_EL1 or CPTR_EL2, depending on the mode.

Quoted 8 lines

To provide context, this last point isn't really a correctness issue. It only ensures that:

- we don't need to issue a synchronisation when alternating one or the other accessor,

- when running hVHE under NV, we don't trap unnecessarily on accessing CPTR_EL2, while CPACR_EL1 can be used directly without any trap.

Thanks for that. I'll include this in the cover letter when I respin to give more context.

Cheers,
/fuad

Quoted 9 lines

I'll have a look at the actual patches shortly.

Thanks again,

        M.

--
Without deviation from the norm, progress is not possible.