KVM: arm64: Rework FPSIMD/SVE tracking

PATCH Superseded Marc Zyngier 3 participants 15 messages Oct 28, 2021 – Nov 23, 2021

Subsystems (3)

KVM arm64 KVM arm64

lore

Share & tools

lore

Marc Zyngier

author maintainer

2021-10-28 11:16 UTC #0 lore raw

v2 cover KVM: arm64: Rework FPSIMD/SVE tracking

This is v2 of this series aiming at simplifying the FP handling.

It recently became apparent that we are mapping each vcpu thread's thread_info structure at EL2 for the sole purpose of checking on the TIF_FOREIGN_FPSTATE flag.

Given that this looks like a slightly over-engineered way of sharing a single bit of information, let's move to a slightly more obvious implementation by maintaining a vcpu-private shadow flag that represents the same state.

In the same vein, it appears that the code that deals with saving the host SVE state when used by the guest can never run, and that's by construction. This is actually a good thing, because it be guaranteed to explode on nVHE. Let's get rid of it.

I also take this opportunity to add what looks like a missing, and nonetheless crucial piece of information to the FPSIMD code regarding the way KVM (ab)uses the TIF_FOREIGN_FPSTATE.

Lightly tested on an A53 box with a bunch of paranoia instances running in both host and guests, and more heavily on a FVP to check the SVE behaviour (using the sve-test selftest running in both host and guest at the same time).

* From v1 [1]:
- New patch getting rid of the host SVE save code
- Reworded the documentation update patch

[1] https://lore.kernel.org/r/[email protected]

Marc Zyngier (5):
KVM: arm64: Reorder vcpu flag definitions
KVM: arm64: Get rid of host SVE tracking/saving
KVM: arm64: Introduce flag shadowing TIF_FOREIGN_FPSTATE
KVM: arm64: Stop mapping current thread_info at EL2 arm64/fpsimd: Document the use of TIF_FOREIGN_FPSTATE by KVM

 arch/arm64/include/asm/kvm_host.h       | 29 ++++++++++---------
 arch/arm64/kernel/fpsimd.c              |  6 +++-
 arch/arm64/kvm/arm.c                    |  1 +
 arch/arm64/kvm/fpsimd.c                 | 37 ++++++++++---------------
 arch/arm64/kvm/hyp/include/hyp/switch.h | 30 +++-----------------
 arch/arm64/kvm/hyp/nvhe/switch.c        |  1 -
 arch/arm64/kvm/hyp/vhe/switch.c         |  1 -
 7 files changed, 38 insertions(+), 67 deletions(-)

Signature · 1 line

2.30.2

Marc Zyngier

author maintainer

2021-10-28 11:16 UTC #1 lore raw

PATCH v2 1/5 KVM: arm64: Reorder vcpu flag definitions

The vcpu arch flags are in an interesting, semi random order. As I have made the mistake of reusing a flag once, let's rework this in an order that I find a bit less confusing.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

arch/arm64/include/asm/kvm_host.h +12 ~~-12~~

index f8be56d5342b..1363c1ff66fb 100644
@@ -416,14 +416,12 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_VCPU_SVE_FINALIZED	(1 << 6) /* SVE config completed */
 #define KVM_ARM64_GUEST_HAS_PTRAUTH	(1 << 7) /* PTRAUTH exposed to guest */
 #define KVM_ARM64_PENDING_EXCEPTION	(1 << 8) /* Exception pending */
+/*
+ * Overlaps with KVM_ARM64_EXCEPT_MASK on purpose so that it can't be
+ * set together with an exception...
+ */
+#define KVM_ARM64_INCREMENT_PC		(1 << 9) /* Increment PC */
 #define KVM_ARM64_EXCEPT_MASK		(7 << 9) /* Target EL/MODE */
-#define KVM_ARM64_DEBUG_STATE_SAVE_SPE	(1 << 12) /* Save SPE context if active  */
-#define KVM_ARM64_DEBUG_STATE_SAVE_TRBE	(1 << 13) /* Save TRBE context if active  */
-
-#define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
-				 KVM_GUESTDBG_USE_SW_BP | \
-				 KVM_GUESTDBG_USE_HW | \
-				 KVM_GUESTDBG_SINGLESTEP)
 /*
  * When KVM_ARM64_PENDING_EXCEPTION is set, KVM_ARM64_EXCEPT_MASK can
  * take the following values:

@@ -441,11 +439,13 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_EXCEPT_AA64_EL1	(0 << 11)
 #define KVM_ARM64_EXCEPT_AA64_EL2	(1 << 11)
 
-/*
- * Overlaps with KVM_ARM64_EXCEPT_MASK on purpose so that it can't be
- * set together with an exception...
- */
-#define KVM_ARM64_INCREMENT_PC		(1 << 9) /* Increment PC */
+#define KVM_ARM64_DEBUG_STATE_SAVE_SPE	(1 << 12) /* Save SPE context if active  */
+#define KVM_ARM64_DEBUG_STATE_SAVE_TRBE	(1 << 13) /* Save TRBE context if active  */
+
+#define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
+				 KVM_GUESTDBG_USE_SW_BP | \
+				 KVM_GUESTDBG_USE_HW | \
+				 KVM_GUESTDBG_SINGLESTEP)
 
 #define vcpu_has_sve(vcpu) (system_supports_sve() &&			\
 			    ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE))

Signature · 1 line

2.30.2

Marc Zyngier

author maintainer

2021-10-28 11:16 UTC #2 lore raw

PATCH v2 2/5 KVM: arm64: Get rid of host SVE tracking/saving

In reply to Marc Zyngier

The SVE host tracking in KVM is pretty involved. It relies on a set of flags tracking the ownership of the SVE register, as well as that of the EL0 access.

It is also pretty scary: __hyp_sve_save_host() computes a thread_struct pointer and obtains a sve_state which gets directly accessed without further ado, even on nVHE. How can this even work?

The answer to that is that it doesn't, and that this is mostly dead code. Closer examination shows that on executing a syscall, userspace loses its SVE state entirely. This is part of the ABI. Another thing to notice is that although the kernel provides helpers such as kernel_neon_begin()/end(), they only deal with the FP/NEON state, and not SVE.

Given that you can only execute a guest as the result of a syscall, and that the kernel cannot use SVE by itself, it becomes pretty obvious that there is never any host SVE state to save, and that this code is only there to increase confusion.

Get rid of the TIF_SVE tracking and host save infrastructure altogether.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h       |  1 -
 arch/arm64/kvm/fpsimd.c                 | 17 +++++-----------
 arch/arm64/kvm/hyp/include/hyp/switch.h | 27 +++----------------------
 3 files changed, 8 insertions(+), 37 deletions(-)

arch/arm64/include/asm/kvm_host.h +0 -1

index 1363c1ff66fb..e24d960244d9 100644
@@ -410,7 +410,6 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_DEBUG_DIRTY		(1 << 0)
 #define KVM_ARM64_FP_ENABLED		(1 << 1) /* guest FP regs loaded */
 #define KVM_ARM64_FP_HOST		(1 << 2) /* host FP regs loaded */
-#define KVM_ARM64_HOST_SVE_IN_USE	(1 << 3) /* backup for host TIF_SVE */
 #define KVM_ARM64_HOST_SVE_ENABLED	(1 << 4) /* SVE enabled for EL0 */
 #define KVM_ARM64_GUEST_HAS_SVE		(1 << 5) /* SVE exposed to guest */
 #define KVM_ARM64_VCPU_SVE_FINALIZED	(1 << 6) /* SVE config completed */

arch/arm64/kvm/fpsimd.c +5 ~~-12~~

index 5621020b28de..38ca332c10fe 100644
@@ -73,15 +73,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 {
 	BUG_ON(!current->mm);
+	BUG_ON(test_thread_flag(TIF_SVE));
 
-	vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
-			      KVM_ARM64_HOST_SVE_IN_USE |
-			      KVM_ARM64_HOST_SVE_ENABLED);
+	vcpu->arch.flags &= ~KVM_ARM64_FP_ENABLED;
 	vcpu->arch.flags |= KVM_ARM64_FP_HOST;
 
-	if (test_thread_flag(TIF_SVE))
-		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_IN_USE;
-
 	if (read_sysreg(cpacr_el1) & CPACR_EL1_ZEN_EL0EN)
 		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_ENABLED;
 }

@@ -115,13 +111,11 @@ void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 {
 	unsigned long flags;
-	bool host_has_sve = system_supports_sve();
-	bool guest_has_sve = vcpu_has_sve(vcpu);
 
 	local_irq_save(flags);
 
 	if (vcpu->arch.flags & KVM_ARM64_FP_ENABLED) {
-		if (guest_has_sve) {
+		if (vcpu_has_sve(vcpu)) {
 			__vcpu_sys_reg(vcpu, ZCR_EL1) = read_sysreg_el1(SYS_ZCR);
 
 			/* Restore the VL that was saved when bound to the CPU */

@@ -131,7 +125,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 		}
 
 		fpsimd_save_and_flush_cpu_state();
-	} else if (has_vhe() && host_has_sve) {
+	} else if (has_vhe() && system_supports_sve()) {
 		/*
 		 * The FPSIMD/SVE state in the CPU has not been touched, and we
 		 * have SVE (and VHE): CPACR_EL1 (alias CPTR_EL2) has been

@@ -145,8 +139,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 			sysreg_clear_set(CPACR_EL1, CPACR_EL1_ZEN_EL0EN, 0);
 	}
 
-	update_thread_flag(TIF_SVE,
-			   vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE);
+	update_thread_flag(TIF_SVE, 0);
 
 	local_irq_restore(flags);
 }

arch/arm64/kvm/hyp/include/hyp/switch.h +3 ~~-24~~

index a0e78a6027be..722dfde7f1aa 100644
@@ -207,16 +207,6 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
 	return __get_fault_info(esr, &vcpu->arch.fault);
 }
 
-static inline void __hyp_sve_save_host(struct kvm_vcpu *vcpu)
-{
-	struct thread_struct *thread;
-
-	thread = container_of(vcpu->arch.host_fpsimd_state, struct thread_struct,
-			      uw.fpsimd_state);
-
-	__sve_save_state(sve_pffr(thread), &vcpu->arch.host_fpsimd_state->fpsr);
-}
-
 static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 {
 	sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2);

@@ -228,21 +218,14 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 /* Check for an FPSIMD/SVE trap and handle as appropriate */
 static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
 {
-	bool sve_guest, sve_host;
+	bool sve_guest;
 	u8 esr_ec;
 	u64 reg;
 
 	if (!system_supports_fpsimd())
 		return false;
 
-	if (system_supports_sve()) {
-		sve_guest = vcpu_has_sve(vcpu);
-		sve_host = vcpu->arch.flags & KVM_ARM64_HOST_SVE_IN_USE;
-	} else {
-		sve_guest = false;
-		sve_host = false;
-	}
-
+	sve_guest = vcpu_has_sve(vcpu);
 	esr_ec = kvm_vcpu_trap_get_class(vcpu);
 	if (esr_ec != ESR_ELx_EC_FP_ASIMD &&
 	    esr_ec != ESR_ELx_EC_SVE)

@@ -269,11 +252,7 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)
 	isb();
 
 	if (vcpu->arch.flags & KVM_ARM64_FP_HOST) {
-		if (sve_host)
-			__hyp_sve_save_host(vcpu);
-		else
-			__fpsimd_save_state(vcpu->arch.host_fpsimd_state);
-
+		__fpsimd_save_state(vcpu->arch.host_fpsimd_state);
 		vcpu->arch.flags &= ~KVM_ARM64_FP_HOST;
 	}

Signature · 1 line

2.30.2

Mark Brown

Reviewed-by on v2 2/5

2021-10-28 13:02 UTC #3 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Quoted 15 lines · Marc Zyngier

The SVE host tracking in KVM is pretty involved. It relies on a set of flags tracking the ownership of the SVE register, as well as that of the EL0 access.
It is also pretty scary: __hyp_sve_save_host() computes a thread_struct pointer and obtains a sve_state which gets directly accessed without further ado, even on nVHE. How can this even work?
The answer to that is that it doesn't, and that this is mostly dead code. Closer examination shows that on executing a syscall, userspace loses its SVE state entirely. This is part of the ABI. Another thing to notice is that although the kernel provides helpers such as kernel_neon_begin()/end(), they only deal with the FP/NEON state, and not SVE.

Quoted 4 lines

Given that you can only execute a guest as the result of a syscall, and that the kernel cannot use SVE by itself, it becomes pretty obvious that there is never any host SVE state to save, and that this code is only there to increase confusion.

Ah, this explains a lot and does in fact make life a lot easier, though we're going to get some of the fun back for SME since the ABI does not invalidate ZA on syscall. That said there we have a register we can check to see if the state is live rather than having to track what's going on with TIF. I've also currently got changes in the SME patch set which do mean that we won't clear TIF_SVE on syscall entry while SME is active, however I can rework that to fit in with this change easily enough which given the simplifications introduced seems like it is clearly the right thing to do so:

Reviewed-by Mark Brown <[email protected]>

Zenghui Yu

reviewer

2021-11-10 13:19 UTC #4 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

In reply to Marc Zyngier

Hi Marc,

Quoted 31 lines · Marc Zyngier

The SVE host tracking in KVM is pretty involved. It relies on a set of flags tracking the ownership of the SVE register, as well as that of the EL0 access.
It is also pretty scary: __hyp_sve_save_host() computes a thread_struct pointer and obtains a sve_state which gets directly accessed without further ado, even on nVHE. How can this even work?
The answer to that is that it doesn't, and that this is mostly dead code. Closer examination shows that on executing a syscall, userspace loses its SVE state entirely. This is part of the ABI. Another thing to notice is that although the kernel provides helpers such as kernel_neon_begin()/end(), they only deal with the FP/NEON state, and not SVE.
Given that you can only execute a guest as the result of a syscall, and that the kernel cannot use SVE by itself, it becomes pretty obvious that there is never any host SVE state to save, and that this code is only there to increase confusion.
Get rid of the TIF_SVE tracking and host save infrastructure altogether.
Signed-off-by Marc Zyngier <[email protected]>
arch/arm64/kvm/fpsimd.c +0 -0
index 5621020b28de..38ca332c10fe 100644
@@ -73,15 +73,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 {
 	BUG_ON(!current->mm);

arch/arm64/kvm/fpsimd.c int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) · line 83

Quoted 10 lines

arch/arm64/kvm/fpsimd.c +2 -5

+	BUG_ON(test_thread_flag(TIF_SVE));
 
-	vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
-			      KVM_ARM64_HOST_SVE_IN_USE |
-			      KVM_ARM64_HOST_SVE_ENABLED);
+	vcpu->arch.flags &= ~KVM_ARM64_FP_ENABLED;
 	vcpu->arch.flags |= KVM_ARM64_FP_HOST;
 
-	if (test_thread_flag(TIF_SVE))
-		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_IN_USE;

The comment about TIF_SVE on top of kvm_arch_vcpu_load_fp() becomes obsolete now. Maybe worth removing it?

| *
| * TIF_SVE is backed up here, since it may get clobbered with guest state.
| * This flag is restored by kvm_arch_vcpu_put_fp(vcpu).

Quoted 7 lines

arch/arm64/kvm/hyp/include/hyp/switch.h +0 -0

index a0e78a6027be..722dfde7f1aa 100644
@@ -207,16 +207,6 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
 	return __get_fault_info(esr, &vcpu->arch.fault);
 }

arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool __populate_fault_info(struct kvm_vcpu *vcpu) · line 218

Quoted 10 lines

arch/arm64/kvm/hyp/include/hyp/switch.h +0 -9

 
-static inline void __hyp_sve_save_host(struct kvm_vcpu *vcpu)
-{
-	struct thread_struct *thread;
-
-	thread = container_of(vcpu->arch.host_fpsimd_state, struct thread_struct,
-			      uw.fpsimd_state);
-
-	__sve_save_state(sve_pffr(thread), &vcpu->arch.host_fpsimd_state->fpsr);
-}

Nit: This removes the only user of __sve_save_state() helper. Should we still keep it in fpsimd.S?

Thanks,
Zenghui

Marc Zyngier

author maintainer

2021-11-22 15:57 UTC #5 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Hi Zenghui,

Quoted 47 lines · Zenghui Yu

Hi Marc,
Quoted 29 lines · Marc Zyngier
The SVE host tracking in KVM is pretty involved. It relies on a set of flags tracking the ownership of the SVE register, as well as that of the EL0 access.
It is also pretty scary: __hyp_sve_save_host() computes a thread_struct pointer and obtains a sve_state which gets directly accessed without further ado, even on nVHE. How can this even work?
The answer to that is that it doesn't, and that this is mostly dead code. Closer examination shows that on executing a syscall, userspace loses its SVE state entirely. This is part of the ABI. Another thing to notice is that although the kernel provides helpers such as kernel_neon_begin()/end(), they only deal with the FP/NEON state, and not SVE.
Given that you can only execute a guest as the result of a syscall, and that the kernel cannot use SVE by itself, it becomes pretty obvious that there is never any host SVE state to save, and that this code is only there to increase confusion.
Get rid of the TIF_SVE tracking and host save infrastructure altogether.
Signed-off-by Marc Zyngier <[email protected]>
arch/arm64/kvm/fpsimd.c +0 -0
index 5621020b28de..38ca332c10fe 100644
@@ -73,15 +73,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
arch/arm64/kvm/fpsimd.c int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) · line 81
Quoted 10 lines
arch/arm64/kvm/fpsimd.c +2 -3
 {
 	BUG_ON(!current->mm);
+	BUG_ON(test_thread_flag(TIF_SVE));
 -	vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
-			      KVM_ARM64_HOST_SVE_IN_USE |
-			      KVM_ARM64_HOST_SVE_ENABLED);
+	vcpu->arch.flags &= ~KVM_ARM64_FP_ENABLED;
 	vcpu->arch.flags |= KVM_ARM64_FP_HOST;
 -	if (test_thread_flag(TIF_SVE))
-		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_IN_USE;
The comment about TIF_SVE on top of kvm_arch_vcpu_load_fp() becomes obsolete now. Maybe worth removing it?

Quoted 3 lines

| *
| * TIF_SVE is backed up here, since it may get clobbered with guest state.
| * This flag is restored by kvm_arch_vcpu_put_fp(vcpu).

Indeed. Now gone.

Quoted 12 lines

arch/arm64/kvm/hyp/include/hyp/switch.h +0 -3

index a0e78a6027be..722dfde7f1aa 100644
@@ -207,16 +207,6 @@ static inline bool __populate_fault_info(struct kvm_vcpu *vcpu)
 	return __get_fault_info(esr, &vcpu->arch.fault);
 }
 -static inline void __hyp_sve_save_host(struct kvm_vcpu *vcpu)
-{
-	struct thread_struct *thread;
-

Quoted 8 lines

arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool __populate_fault_info(struct kvm_vcpu *vcpu) · line 218
Quoted 5 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +0 -5
-	thread = container_of(vcpu->arch.host_fpsimd_state, struct thread_struct,
-			      uw.fpsimd_state);
-
-	__sve_save_state(sve_pffr(thread), &vcpu->arch.host_fpsimd_state->fpsr);
-}
Nit: This removes the only user of __sve_save_state() helper. Should we still keep it in fpsimd.S?

I was in two minds about that, as I'd like to eventually be able to use SVE for protected guests, where the hypervisor itself has to be in charge of the FP/SVE save-restore.

But that's probably several months away, and I can always revert a deletion patch if I need to, so let's get rid of it now.

Thanks for the suggestions.

M.

Signature · 5 lines

Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Mark Brown

2021-11-22 17:58 UTC #6 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Quoted 11 lines · Marc Zyngier

Quoted 2 lines · Zenghui Yu
Nit: This removes the only user of __sve_save_state() helper. Should we still keep it in fpsimd.S?
I was in two minds about that, as I'd like to eventually be able to use SVE for protected guests, where the hypervisor itself has to be in charge of the FP/SVE save-restore.
But that's probably several months away, and I can always revert a deletion patch if I need to, so let's get rid of it now.

While we're on the subject of potential future work we might in future want to not disable SVE on every syscall if (as seems likely) it turns out that that's more performant for small vector lengths which would mean some minor reshuffling here to do something like convert the saved state to FPSIMD and drop TIF_SVE in _vcpu_load_fp(). As with using SVE in protected guests that can just be done when needed though.

Marc Zyngier

author maintainer

2021-11-22 18:10 UTC #7 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Quoted 11 lines · Mark Brown

Quoted 8 lines · Marc Zyngier
Quoted 2 lines · Zenghui Yu
Nit: This removes the only user of __sve_save_state() helper. Should we still keep it in fpsimd.S?
I was in two minds about that, as I'd like to eventually be able to use SVE for protected guests, where the hypervisor itself has to be in charge of the FP/SVE save-restore.

Quoted 6 lines

Quoted 2 lines
But that's probably several months away, and I can always revert a deletion patch if I need to, so let's get rid of it now.
While we're on the subject of potential future work we might in future want to not disable SVE on every syscall if (as seems likely) it turns out that that's more performant for small vector lengths

[...]

How are you going to retrofit that into userspace? This would be an ABI change, and I'm not sure how you'd want to deal with that transition...

M.

Signature · 5 lines

Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Mark Brown

2021-11-22 18:30 UTC #8 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Quoted 9 lines · Marc Zyngier

Quoted 3 lines · Mark Brown
While we're on the subject of potential future work we might in future want to not disable SVE on every syscall if (as seems likely) it turns out that that's more performant for small vector lengths
How are you going to retrofit that into userspace? This would be an ABI change, and I'm not sure how you'd want to deal with that transition...

We don't need to change the ABI, the ABI just says we zero the registers that aren't shared with FPSIMD. Instead of doing that on taking a SVE access trap to reenable SVE after having disabled TIF_SVE we could do that during the syscall, userspace can't tell the difference other than via the different formats we use to report the SVE register set via ptrace if it single steps over a syscall. Even then I'm struggling to think of a scenario where userspace would be relying on that.

You could also implement a similar optimisation by forcing on TIF_SVE whenever we return to userspace but that would create a cost for userspace tasks that don't use SVE on SVE capable hardware so doesn't seem as good. In any case it's not an issue for now since anything here will need benchmarking on a reasonable range of hardware.

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Marc Zyngier

author maintainer

2021-11-23 10:11 UTC #9 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Quoted 12 lines · Mark Brown

Quoted 9 lines · Marc Zyngier
Quoted 3 lines · Mark Brown
While we're on the subject of potential future work we might in future want to not disable SVE on every syscall if (as seems likely) it turns out that that's more performant for small vector lengths
How are you going to retrofit that into userspace? This would be an ABI change, and I'm not sure how you'd want to deal with that transition...

Quoted 7 lines

We don't need to change the ABI, the ABI just says we zero the registers that aren't shared with FPSIMD. Instead of doing that on taking a SVE access trap to reenable SVE after having disabled TIF_SVE we could do that during the syscall, userspace can't tell the difference other than via the different formats we use to report the SVE register set via ptrace if it single steps over a syscall. Even then I'm struggling to think of a scenario where userspace would be relying on that.

That's not the point I'm trying to make.

Userspace expects to have lost SVE information over a syscall (even if the VL is 128, it expects to have lost P0..P15 and FFR). How do you plan to tell userspace that this behaviour has changed?

M.

Signature · 1 line

Without deviation from the norm, progress is not possible.

Mark Brown

2021-11-23 12:33 UTC #10 lore raw

Re: [PATCH v2 2/5] KVM: arm64: Get rid of host SVE tracking/saving

Quoted 11 lines · Marc Zyngier

Quoted 3 lines · Mark Brown
We don't need to change the ABI, the ABI just says we zero the registers that aren't shared with FPSIMD. Instead of doing that on taking a SVE access trap to reenable SVE after having disabled TIF_SVE we could do
That's not the point I'm trying to make.
Userspace expects to have lost SVE information over a syscall (even if the VL is 128, it expects to have lost P0..P15 and FFR). How do you plan to tell userspace that this behaviour has changed?

My point is that this doesn't need to change. Userspace can't tell if we zeroed the non-shared state on syscall or on some later access trap.

Marc Zyngier

author maintainer

2021-10-28 11:16 UTC #11 lore raw

PATCH v2 3/5 KVM: arm64: Introduce flag shadowing TIF_FOREIGN_FPSTATE

In reply to Marc Zyngier

We currently have to maintain a mapping the thread_info structure at EL2 in order to be able to check the TIF_FOREIGN_FPSTATE flag.

In order to eventually get rid of this, start with a vcpu flag that shadows the thread flag on each entry into the hypervisor.

Reviewed-by Mark Brown <[email protected]>

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h       | 2 ++
 arch/arm64/kvm/arm.c                    | 1 +
 arch/arm64/kvm/fpsimd.c                 | 8 ++++++++
 arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +-
 4 files changed, 12 insertions(+), 1 deletion(-)

arch/arm64/include/asm/kvm_host.h +2 -0

index e24d960244d9..0db523e447a5 100644
@@ -440,6 +440,7 @@ struct kvm_vcpu_arch {
 
 #define KVM_ARM64_DEBUG_STATE_SAVE_SPE	(1 << 12) /* Save SPE context if active  */
 #define KVM_ARM64_DEBUG_STATE_SAVE_TRBE	(1 << 13) /* Save TRBE context if active  */
+#define KVM_ARM64_FP_FOREIGN_FPSTATE	(1 << 14)
 
 #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
 				 KVM_GUESTDBG_USE_SW_BP | \

@@ -735,6 +736,7 @@ long kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm,
 /* Guest/host FPSIMD coordination helpers */
 int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu);
+void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu);
 void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu);

arch/arm64/kvm/arm.c +1 -0

index fe102cd2e518..4afb84e994e2 100644
@@ -840,6 +840,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 		}
 
 		kvm_arm_setup_debug(vcpu);
+		kvm_arch_vcpu_ctxflush_fp(vcpu);
 
 		/**************************************************************
 		 * Enter the guest

arch/arm64/kvm/fpsimd.c +8 -0

index 38ca332c10fe..f464db2e9e47 100644
@@ -82,6 +82,14 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 		vcpu->arch.flags |= KVM_ARM64_HOST_SVE_ENABLED;
 }
 
+void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu)
+{
+	if (test_thread_flag(TIF_FOREIGN_FPSTATE))
+		vcpu->arch.flags |= KVM_ARM64_FP_FOREIGN_FPSTATE;
+	else
+		vcpu->arch.flags &= ~KVM_ARM64_FP_FOREIGN_FPSTATE;
+}
+
 /*
  * If the guest FPSIMD state was loaded, update the host's context
  * tracking data mark the CPU FPSIMD regs as dirty and belonging to vcpu

arch/arm64/kvm/hyp/include/hyp/switch.h +1 -1

index 722dfde7f1aa..d49b1b3725f5 100644
@@ -44,7 +44,7 @@ static inline bool update_fp_enabled(struct kvm_vcpu *vcpu)
 	 * trap the accesses.
 	 */
 	if (!system_supports_fpsimd() ||
-	    vcpu->arch.host_thread_info->flags & _TIF_FOREIGN_FPSTATE)
+	    vcpu->arch.flags & KVM_ARM64_FP_FOREIGN_FPSTATE)
 		vcpu->arch.flags &= ~(KVM_ARM64_FP_ENABLED |
 				      KVM_ARM64_FP_HOST);

Signature · 1 line

2.30.2

Marc Zyngier

author maintainer

2021-10-28 11:16 UTC #12 lore raw

PATCH v2 4/5 KVM: arm64: Stop mapping current thread_info at EL2

In reply to Marc Zyngier

Now that we can track an equivalent of TIF_FOREIGN_FPSTATE, drop the mapping of current's thread_info at EL2.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/include/asm/kvm_host.h       |  2 --
 arch/arm64/kvm/fpsimd.c                 | 12 +-----------
 arch/arm64/kvm/hyp/include/hyp/switch.h |  1 -
 arch/arm64/kvm/hyp/nvhe/switch.c        |  1 -
 arch/arm64/kvm/hyp/vhe/switch.c         |  1 -
 5 files changed, 1 insertion(+), 16 deletions(-)

arch/arm64/include/asm/kvm_host.h +0 -2

index 0db523e447a5..da4a82d8ce5d 100644
@@ -26,7 +26,6 @@
 #include <asm/fpsimd.h>
 #include <asm/kvm.h>
 #include <asm/kvm_asm.h>
-#include <asm/thread_info.h>
 
 #define __KVM_HAVE_ARCH_INTC_INITIALIZED

@@ -320,7 +319,6 @@ struct kvm_vcpu_arch {
 	struct kvm_guest_debug_arch vcpu_debug_state;
 	struct kvm_guest_debug_arch external_debug_state;
 
-	struct thread_info *host_thread_info;	/* hyp VA */
 	struct user_fpsimd_state *host_fpsimd_state;	/* hyp VA */
 
 	struct {

arch/arm64/kvm/fpsimd.c +1 ~~-11~~

index f464db2e9e47..f54d6a1eaff6 100644
@@ -7,7 +7,6 @@
  */
 #include <linux/irqflags.h>
 #include <linux/sched.h>
-#include <linux/thread_info.h>
 #include <linux/kvm_host.h>
 #include <asm/fpsimd.h>
 #include <asm/kvm_asm.h>

@@ -28,17 +27,9 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 {
 	int ret;
 
-	struct thread_info *ti = &current->thread_info;
 	struct user_fpsimd_state *fpsimd = &current->thread.uw.fpsimd_state;
 
-	/*
-	 * Make sure the host task thread flags and fpsimd state are
-	 * visible to hyp:
-	 */
-	ret = create_hyp_mappings(ti, ti + 1, PAGE_HYP);
-	if (ret)
-		goto error;
-
+	/* Make sure the host task fpsimd state is visible to hyp: */
 	ret = create_hyp_mappings(fpsimd, fpsimd + 1, PAGE_HYP);
 	if (ret)
 		goto error;

@@ -54,7 +45,6 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu)
 			goto error;
 	}
 
-	vcpu->arch.host_thread_info = kern_hyp_va(ti);
 	vcpu->arch.host_fpsimd_state = kern_hyp_va(fpsimd);
 error:
 	return ret;

arch/arm64/kvm/hyp/include/hyp/switch.h +0 -1

index d49b1b3725f5..d2d7b72de84a 100644
@@ -28,7 +28,6 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
-#include <asm/thread_info.h>
 
 extern struct exception_table_entry __start___kvm_ex_table;
 extern struct exception_table_entry __stop___kvm_ex_table;

arch/arm64/kvm/hyp/nvhe/switch.c +0 -1

index a34b01cc8ab9..012890c2af1b 100644
@@ -25,7 +25,6 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
-#include <asm/thread_info.h>
 
 #include <nvhe/mem_protect.h>

arch/arm64/kvm/hyp/vhe/switch.c +0 -1

index ded2c66675f0..c7fc04b11673 100644
@@ -24,7 +24,6 @@
 #include <asm/fpsimd.h>
 #include <asm/debug-monitors.h>
 #include <asm/processor.h>
-#include <asm/thread_info.h>
 
 /* VHE specific context */
 DEFINE_PER_CPU(struct kvm_host_data, kvm_host_data);

Signature · 1 line

2.30.2

Marc Zyngier

author maintainer

2021-10-28 11:16 UTC #13 lore raw

PATCH v2 5/5 arm64/fpsimd: Document the use of TIF_FOREIGN_FPSTATE by KVM

In reply to Marc Zyngier

The bit of documentation that talks about TIF_FOREIGN_FPSTATE does not mention the ungodly tricks that KVM plays with this flag.

Try and document this for the posterity.

Signed-off-by Marc Zyngier <[email protected]>

---
 arch/arm64/kernel/fpsimd.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

arch/arm64/kernel/fpsimd.c +5 -1

index ff4962750b3d..1fbd6ba7dbac 100644
@@ -78,7 +78,11 @@
  * indicate whether or not the userland FPSIMD state of the current task is
  * present in the registers. The flag is set unless the FPSIMD registers of this
  * CPU currently contain the most recent userland FPSIMD state of the current
- * task.
+ * task. If the task is behaving as a VMM, then this is will be managed by
+ * KVM which will clear it to indicate that the vcpu FPSIMD state is currently
+ * loaded on the CPU, allowing the state to be saved if a FPSIMD-aware
+ * softirq kicks in. Upon vcpu_put(), KVM will save the vcpu FP state and
+ * flag the register state as invalid.
  *
  * In order to allow softirq handlers to use FPSIMD, kernel_neon_begin() may
  * save the task's FPSIMD context back to task_struct from softirq context.

Signature · 1 line

2.30.2

Mark Brown

2021-10-28 13:14 UTC #14 lore raw

Re: [PATCH v2 5/5] arm64/fpsimd: Document the use of TIF_FOREIGN_FPSTATE by KVM

Quoted 4 lines · Marc Zyngier

The bit of documentation that talks about TIF_FOREIGN_FPSTATE does not mention the ungodly tricks that KVM plays with this flag.
Try and document this for the posterity.

Reviwed-by Mark Brown <[email protected]>