Skip to contentSkip to conversation

KVM: arm64: Fix handling of host fpsimd/sve state in protected mode

PATCH Applied Fuad Tabba 3 participants 24 messages Jun 3, 2024 – Jun 4, 2024

Subsystems (2)
lore
Share & tools
lore

Fuad Tabba

author
#0 lore raw

v4 cover KVM: arm64: Fix handling of host fpsimd/sve state in protected mode

Changes since v3 [1]:
- Rebased on Linux 6.10-rc2 (c3f38fa61af7)
- Dropped v3 patches 8--11 (Oliver)
- Removed unnecessary isb()s (Oliver)
- Formatting/comments (Mark)
- Fix __sve_save_state()/__sve_restore_state() prototypes (Mark)
- Save/restore ffr with the sve state
- Added a patch that checks at hyp that SME features aren't enabled on guest entry, to ensure it's not in streaming mode

With the KVM host data rework [2], handling of fpsimd and sve state in protected mode is done at hyp. For protected VMs, we don't want to leak any guest state to the host, including whether a guest has used fpsimd/sve.

To complete the work started with the host data rework, in regards to protected mode, ensure that the host's fpsimd context and its sve context are restored on guest exit, since the rework has hidden the fpsimd/sve state from the host.

This patch series eagerly restores the host fpsimd/sve state on guest exit when running in protected mode, which happens only if the guest has used fpsimd/sve. This means that the saving of the state is lazy, similar to the behavior of KVM in other modes, but the restoration of the host state is eager.

This series is based on Linux 6.10-rc2 (c3f38fa61af7).

Tested on qemu, with the kernel sve stress tests.

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/

Fuad Tabba (9):
KVM: arm64: Reintroduce __sve_save_state
KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state
KVM: arm64: Abstract set/clear of CPTR_EL2 bits behind helper
KVM: arm64: Specialize handling of host fpsimd state on trap
KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM
KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM
KVM: arm64: Consolidate initializing the host data's fpsimd_state/sve
in pKVM
KVM: arm64: Refactor CPACR trap bit setting/clearing to use ELx format
KVM: arm64: Ensure that SME controls are disabled in protected mode

 arch/arm64/include/asm/el2_setup.h      |  6 +-
 arch/arm64/include/asm/kvm_arm.h        |  6 ++
 arch/arm64/include/asm/kvm_emulate.h    | 71 +++++++++++++++++++--
 arch/arm64/include/asm/kvm_host.h       | 25 +++++++-
 arch/arm64/include/asm/kvm_hyp.h        |  4 +-
 arch/arm64/include/asm/kvm_pkvm.h       |  9 +++
 arch/arm64/kvm/arm.c                    | 76 ++++++++++++++++++++++
 arch/arm64/kvm/fpsimd.c                 | 11 +++-
 arch/arm64/kvm/hyp/fpsimd.S             |  6 ++
 arch/arm64/kvm/hyp/include/hyp/switch.h | 36 ++++++-----
 arch/arm64/kvm/hyp/include/nvhe/pkvm.h  |  1 -
 arch/arm64/kvm/hyp/nvhe/hyp-main.c      | 84 ++++++++++++++++++++++---
 arch/arm64/kvm/hyp/nvhe/pkvm.c          | 17 ++---
 arch/arm64/kvm/hyp/nvhe/setup.c         | 25 +++++++-
 arch/arm64/kvm/hyp/nvhe/switch.c        | 24 ++++++-
 arch/arm64/kvm/hyp/vhe/switch.c         | 12 ++--
 arch/arm64/kvm/reset.c                  |  3 +
 17 files changed, 358 insertions(+), 58 deletions(-)

base-commit: c3f38fa61af77b49866b006939479069cd451173

Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
author
#1 lore raw

PATCH v4 1/9 KVM: arm64: Reintroduce __sve_save_state

Now that the hypervisor is handling the host sve state in protected mode, it needs to be able to save it.

This reverts commit e66425fc9ba3 ("KVM: arm64: Remove unused __sve_save_state").

---
 arch/arm64/include/asm/kvm_hyp.h | 1 +
 arch/arm64/kvm/hyp/fpsimd.S      | 6 ++++++
 2 files changed, 7 insertions(+)
arch/arm64/include/asm/kvm_hyp.h +1 -0
index 3e80464f8953..2ab23589339a 100644
@@ -111,6 +111,7 @@ void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu);
 
 void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
 void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
+void __sve_save_state(void *sve_pffr, u32 *fpsr);
 void __sve_restore_state(void *sve_pffr, u32 *fpsr);
 
 u64 __guest_enter(struct kvm_vcpu *vcpu);
arch/arm64/kvm/hyp/fpsimd.S +6 -0
index 61e6f3ba7b7d..e950875e31ce 100644
@@ -25,3 +25,9 @@ SYM_FUNC_START(__sve_restore_state)
 	sve_load 0, x1, x2, 3
 	ret
 SYM_FUNC_END(__sve_restore_state)
+
+SYM_FUNC_START(__sve_save_state)
+	mov	x2, #1
+	sve_save 0, x1, x2, 3
+	ret
+SYM_FUNC_END(__sve_save_state)
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
#2 lore raw

Re: [PATCH v4 1/9] KVM: arm64: Reintroduce __sve_save_state

arch/arm64/kvm/hyp/fpsimd.S SYM_FUNC_START(__sve_restore_state) · line 33

Quoted 7 lines · Fuad Tabba
arch/arm64/kvm/hyp/fpsimd.S +6 -0
+void __sve_save_state(void *sve_pffr, u32 *fpsr);
 
+SYM_FUNC_START(__sve_save_state)
+	mov	x2, #1
+	sve_save 0, x1, x2, 3
+	ret
+SYM_FUNC_END(__sve_save_state)

This is still trying to use x2 as an argument here?

author
#3 lore raw

Re: [PATCH v4 1/9] KVM: arm64: Reintroduce __sve_save_state

Hi Mark,

Quoted 5 lines · Mark Brown
Quoted 1 line · Fuad Tabba

+void __sve_save_state(void *sve_pffr, u32 *fpsr);

Quoted 7 lines

arch/arm64/kvm/hyp/fpsimd.S SYM_FUNC_START(__sve_restore_state) · line 33

Quoted 5 lines
arch/arm64/kvm/hyp/fpsimd.S +5 -0
+SYM_FUNC_START(__sve_save_state)
+     mov     x2, #1
+     sve_save 0, x1, x2, 3
+     ret
+SYM_FUNC_END(__sve_save_state)

This is still trying to use x2 as an argument here?

Since __sve_restore_state also has the same problem, I fixed both in the following patch:

KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state
https://lore.kernel.org/all/[email protected]/

Cheers,
/fuad

#4 lore raw

Re: [PATCH v4 1/9] KVM: arm64: Reintroduce __sve_save_state

Quoted 10 lines · Fuad Tabba
Quoted 3 lines · Mark Brown

On Mon, Jun 03, 2024 at 01:28:43PM +0100, Fuad Tabba wrote:

This is still trying to use x2 as an argument here?

Since __sve_restore_state also has the same problem, I fixed both in the following patch:

KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state
https://lore.kernel.org/all/[email protected]/

Ah, I see - it'd have been good to note that in the changelog.

author
#5 lore raw

PATCH v4 2/9 KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state

In reply to Fuad Tabba

Since the prototypes for __sve_save_state/__sve_restore_state at hyp were added, the underlying macro has acquired a third parameter for saving/restoring ffr.

Fix the prototypes to account for the third parameter, and restore the ffr for the guest since it is saved.

---
 arch/arm64/include/asm/kvm_hyp.h        | 4 ++--
 arch/arm64/kvm/hyp/include/hyp/switch.h | 3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_hyp.h +2 -2
index 2ab23589339a..686cce7e4e96 100644
@@ -111,8 +111,8 @@ void __debug_restore_host_buffers_nvhe(struct kvm_vcpu *vcpu);
 
 void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
 void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
-void __sve_save_state(void *sve_pffr, u32 *fpsr);
-void __sve_restore_state(void *sve_pffr, u32 *fpsr);
+void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr);
+void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr);
 
 u64 __guest_enter(struct kvm_vcpu *vcpu);
 
arch/arm64/kvm/hyp/include/hyp/switch.h +2 -1
index a92566f36022..d58933ae8fd5 100644
@@ -316,7 +316,8 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 {
 	sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2);
 	__sve_restore_state(vcpu_sve_pffr(vcpu),
-			    &vcpu->arch.ctxt.fp_regs.fpsr);
+			    &vcpu->arch.ctxt.fp_regs.fpsr,
+			    true);
 	write_sysreg_el1(__vcpu_sys_reg(vcpu, ZCR_EL1), SYS_ZCR);
 }
 
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
Reviewed-by on v4 2/9
#6 lore raw

Re: [PATCH v4 2/9] KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state

Quoted 6 lines · Fuad Tabba

Since the prototypes for __sve_save_state/__sve_restore_state at hyp were added, the underlying macro has acquired a third parameter for saving/restoring ffr.

Fix the prototypes to account for the third parameter, and restore the ffr for the guest since it is saved.

The change to fix restore should be sent as a bug fix.

author
#7 lore raw

PATCH v4 3/9 KVM: arm64: Abstract set/clear of CPTR_EL2 bits behind helper

In reply to Fuad Tabba

The same traps controlled by CPTR_EL2 or CPACR_EL1 need to be toggled in different parts of the code, but the exact bits and their polarity differ between these two formats and the mode (vhe/nvhe/hvhe).

To reduce the amount of duplicated code and the chance of getting the wrong bit/polarity or missing a field, abstract the set/clear of CPTR_EL2 bits behind a helper.

Since (h)VHE is the way of the future, use the CPACR_EL1 format, which is a subset of the VHE CPTR_EL2, as a reference.

No functional change intended.

---
 arch/arm64/include/asm/kvm_arm.h        |  6 +++
 arch/arm64/include/asm/kvm_emulate.h    | 62 +++++++++++++++++++++++++
 arch/arm64/kvm/hyp/include/hyp/switch.h | 18 ++-----
 arch/arm64/kvm/hyp/nvhe/hyp-main.c      |  6 +--
 4 files changed, 73 insertions(+), 19 deletions(-)
arch/arm64/include/asm/kvm_arm.h +6 -0
index e01bb5ca13b7..b2adc2c6c82a 100644
@@ -305,6 +305,12 @@
 				 GENMASK(19, 14) |	\
 				 BIT(11))
 
+#define CPTR_VHE_EL2_RES0	(GENMASK(63, 32) |	\
+				 GENMASK(27, 26) |	\
+				 GENMASK(23, 22) |	\
+				 GENMASK(19, 18) |	\
+				 GENMASK(15, 0))
+
 /* Hyp Debug Configuration Register bits */
 #define MDCR_EL2_E2TB_MASK	(UL(0x3))
 #define MDCR_EL2_E2TB_SHIFT	(UL(24))
arch/arm64/include/asm/kvm_emulate.h +62 -0

68 lines not on this page · raw message

arch/arm64/kvm/hyp/include/hyp/switch.h +4 -14
index d58933ae8fd5..055d2ca7264e 100644
@@ -331,7 +331,6 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code)
 {
 	bool sve_guest;
 	u8 esr_ec;
-	u64 reg;
 
 	if (!system_supports_fpsimd())
 		return false;
@@ -354,19 +353,10 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code)
 	/* Valid trap.  Switch the context: */
 
 	/* First disable enough traps to allow us to update the registers */
-	if (has_vhe() || has_hvhe()) {
-		reg = CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN;
-		if (sve_guest)
-			reg |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN;
-
-		sysreg_clear_set(cpacr_el1, 0, reg);
-	} else {
-		reg = CPTR_EL2_TFP;
-		if (sve_guest)
-			reg |= CPTR_EL2_TZ;
-
-		sysreg_clear_set(cptr_el2, reg, 0);
-	}
+	if (sve_guest)
+		cpacr_clear_set(0, CPACR_ELx_FPEN | CPACR_ELx_ZEN);
+	else
+		cpacr_clear_set(0, CPACR_ELx_FPEN);
 	isb();
 
 	/* Write out the host state if it's in the registers */
arch/arm64/kvm/hyp/nvhe/hyp-main.c +1 -5
index d5c48dc98f67..f71394d0e32a 100644
@@ -405,11 +405,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)
 		handle_host_smc(host_ctxt);
 		break;
 	case ESR_ELx_EC_SVE:
-		if (has_hvhe())
-			sysreg_clear_set(cpacr_el1, 0, (CPACR_EL1_ZEN_EL1EN |
-							CPACR_EL1_ZEN_EL0EN));
-		else
-			sysreg_clear_set(cptr_el2, CPTR_EL2_TZ, 0);
+		cpacr_clear_set(0, CPACR_ELx_ZEN);
 		isb();
 		sve_cond_update_zcr_vq(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2);
 		break;
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
author
#8 lore raw

PATCH v4 4/9 KVM: arm64: Specialize handling of host fpsimd state on trap

In reply to Fuad Tabba

In subsequent patches, n/vhe will diverge on saving the host fpsimd/sve state when taking a guest fpsimd/sve trap. Add a specialized helper to handle it.

No functional change intended.

---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 4 +++-
 arch/arm64/kvm/hyp/nvhe/switch.c        | 5 +++++
 arch/arm64/kvm/hyp/vhe/switch.c         | 5 +++++
 3 files changed, 13 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +3 -1
index 055d2ca7264e..9b904c858df0 100644
@@ -321,6 +321,8 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 	write_sysreg_el1(__vcpu_sys_reg(vcpu, ZCR_EL1), SYS_ZCR);
 }
 
+static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu);
+
 /*
  * We trap the first access to the FP/SIMD to save the host context and
  * restore the guest context lazily.
@@ -361,7 +363,7 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code)
 
 	/* Write out the host state if it's in the registers */
 	if (host_owns_fp_regs())
-		__fpsimd_save_state(*host_data_ptr(fpsimd_state));
+		kvm_hyp_save_fpsimd_host(vcpu);
 
 	/* Restore the guest state */
 	if (sve_guest)
arch/arm64/kvm/hyp/nvhe/switch.c +5 -0
index 6758cd905570..019f863922fa 100644
@@ -182,6 +182,11 @@ static bool kvm_handle_pvm_sys64(struct kvm_vcpu *vcpu, u64 *exit_code)
 		kvm_handle_pvm_sysreg(vcpu, exit_code));
 }
 
+static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu)
+{
+	__fpsimd_save_state(*host_data_ptr(fpsimd_state));
+}
+
 static const exit_handler_fn hyp_exit_handlers[] = {
 	[0 ... ESR_ELx_EC_MAX]		= NULL,
 	[ESR_ELx_EC_CP15_32]		= kvm_hyp_handle_cp15_32,
arch/arm64/kvm/hyp/vhe/switch.c +5 -0
index d7af5f46f22a..20073579e9f5 100644
@@ -262,6 +262,11 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code)
 	return true;
 }
 
+static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu)
+{
+	__fpsimd_save_state(*host_data_ptr(fpsimd_state));
+}
+
 static const exit_handler_fn hyp_exit_handlers[] = {
 	[0 ... ESR_ELx_EC_MAX]		= NULL,
 	[ESR_ELx_EC_CP15_32]		= kvm_hyp_handle_cp15_32,
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
author
#9 lore raw

PATCH v4 5/9 KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM

In reply to Fuad Tabba

Protected mode needs to maintain (save/restore) the host's sve state, rather than relying on the host kernel to do that. This is to avoid leaking information to the host about guests and the type of operations they are performing.

As a first step towards that, allocate memory mapped at hyp, per cpu, for the host sve state. The following patch will use this memory to save/restore the host state.

---
 arch/arm64/include/asm/kvm_host.h | 17 ++++++++
 arch/arm64/include/asm/kvm_hyp.h  |  1 +
 arch/arm64/include/asm/kvm_pkvm.h |  9 ++++
 arch/arm64/kvm/arm.c              | 68 +++++++++++++++++++++++++++++++
 arch/arm64/kvm/hyp/nvhe/pkvm.c    |  2 +
 arch/arm64/kvm/hyp/nvhe/setup.c   | 24 +++++++++++
 arch/arm64/kvm/reset.c            |  3 ++
 7 files changed, 124 insertions(+)
arch/arm64/include/asm/kvm_host.h +17 -0
index 8170c04fde91..90df7ccec5f4 100644
@@ -76,6 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; };
 DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use);
 
 extern unsigned int __ro_after_init kvm_sve_max_vl;
+extern unsigned int __ro_after_init kvm_host_sve_max_vl;
 int __init kvm_arm_init_sve(void);
 
 u32 __attribute_const__ kvm_target_cpu(void);
@@ -521,6 +522,20 @@ struct kvm_cpu_context {
 	u64 *vncr_array;
 };
 
+struct cpu_sve_state {
+	__u64 zcr_el1;
+
+	/*
+	 * Ordering is important since __sve_save_state/__sve_restore_state
+	 * relies on it.
+	 */
+	__u32 fpsr;
+	__u32 fpcr;
+
+	/* Must be SVE_VQ_BYTES (128 bit) aligned. */
+	__u8 sve_regs[];
+};
+
 /*
  * This structure is instantiated on a per-CPU basis, and contains
  * data that is:
@@ -534,7 +549,9 @@ struct kvm_cpu_context {
  */
 struct kvm_host_data {
 	struct kvm_cpu_context host_ctxt;
+
 	struct user_fpsimd_state *fpsimd_state;	/* hyp VA */
+	struct cpu_sve_state *sve_state;	/* hyp VA */
 
 	/* Ownership of the FP regs */
 	enum {
arch/arm64/include/asm/kvm_hyp.h +1 -0
index 686cce7e4e96..b05bceca3385 100644
@@ -143,5 +143,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val);
 
 extern unsigned long kvm_nvhe_sym(__icache_flags);
 extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits);
+extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl);
 
 #endif /* __ARM64_KVM_HYP_H__ */
arch/arm64/include/asm/kvm_pkvm.h +9 -0
index ad9cfb5c1ff4..cd56acd9a842 100644
@@ -128,4 +128,13 @@ static inline unsigned long hyp_ffa_proxy_pages(void)
 	return (2 * KVM_FFA_MBOX_NR_PAGES) + DIV_ROUND_UP(desc_max, PAGE_SIZE);
 }
 
+static inline size_t pkvm_host_sve_state_size(void)
+{
+	if (!system_supports_sve())
+		return 0;
+
+	return size_add(sizeof(struct cpu_sve_state),
+			SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl)));
+}
+
 #endif	/* __ARM64_KVM_PKVM_H__ */
arch/arm64/kvm/arm.c +68 -0

104 lines not on this page · raw message

arch/arm64/kvm/hyp/nvhe/pkvm.c +2 -0
index 16aa4875ddb8..25e9a94f6d76 100644
@@ -18,6 +18,8 @@ unsigned long __icache_flags;
 /* Used by kvm_get_vttbr(). */
 unsigned int kvm_arm_vmid_bits;
 
+unsigned int kvm_host_sve_max_vl;
+
 /*
  * Set trap register values based on features in ID_AA64PFR0.
  */
arch/arm64/kvm/hyp/nvhe/setup.c +24 -0
index 859f22f754d3..3fae42479598 100644
@@ -67,6 +67,28 @@ static int divide_memory_pool(void *virt, unsigned long size)
 	return 0;
 }
 
+static int pkvm_create_host_sve_mappings(void)
+{
+	void *start, *end;
+	int ret, i;
+
+	if (!system_supports_sve())
+		return 0;
+
+	for (i = 0; i < hyp_nr_cpus; i++) {
+		struct kvm_host_data *host_data = per_cpu_ptr(&kvm_host_data, i);
+		struct cpu_sve_state *sve_state = host_data->sve_state;
+
+		start = kern_hyp_va(sve_state);
+		end = start + PAGE_ALIGN(pkvm_host_sve_state_size());
+		ret = pkvm_create_mappings(start, end, PAGE_HYP);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
 static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size,
 				 unsigned long *per_cpu_base,
 				 u32 hyp_va_bits)
@@ -125,6 +147,8 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size,
 			return ret;
 	}
 
+	pkvm_create_host_sve_mappings();
+
 	/*
 	 * Map the host sections RO in the hypervisor, but transfer the
 	 * ownership from the host to the hypervisor itself to make sure they
arch/arm64/kvm/reset.c +3 -0
index 1b7b58cb121f..3fc8ca164dbe 100644
@@ -32,6 +32,7 @@
 
 /* Maximum phys_shift supported for any VM on this host */
 static u32 __ro_after_init kvm_ipa_limit;
+unsigned int __ro_after_init kvm_host_sve_max_vl;
 
 /*
  * ARMv8 Reset Values
@@ -51,6 +52,8 @@ int __init kvm_arm_init_sve(void)
 {
 	if (system_supports_sve()) {
 		kvm_sve_max_vl = sve_max_virtualisable_vl();
+		kvm_host_sve_max_vl = sve_max_vl();
+		kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl;
 
 		/*
 		 * The get_sve_reg()/set_sve_reg() ioctl interface will need
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
#10 lore raw

Re: [PATCH v4 5/9] KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM

arch/arm64/kvm/reset.c int __init kvm_arm_init_sve(void) · line 56

Quoted 5 lines · Fuad Tabba

{

arch/arm64/kvm/reset.c +2 -0
 	if (system_supports_sve()) {
 		kvm_sve_max_vl = sve_max_virtualisable_vl();
+		kvm_host_sve_max_vl = sve_max_vl();
+		kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl;

As discussed on the prior version this either needs to be the maximum available physical VL on any individual PE or the hypervisor needs to use this VL explicitly when saving/restoring host state.

author
#11 lore raw

Re: [PATCH v4 5/9] KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM

Hi Mark,

Quoted 9 lines · Mark Brown

arch/arm64/kvm/reset.c int __init kvm_arm_init_sve(void) · line 56

Quoted 5 lines · Fuad Tabba
 {
      if (system_supports_sve()) {
              kvm_sve_max_vl = sve_max_virtualisable_vl();
arch/arm64/kvm/reset.c +2 -0
+             kvm_host_sve_max_vl = sve_max_vl();
+             kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl;
Quoted 3 lines

As discussed on the prior version this either needs to be the maximum available physical VL on any individual PE or the hypervisor needs to use this VL explicitly when saving/restoring host state.

Sorry it took me a while, but I understand what the issue is now. I'll send a patch to fix this.

Thanks for being patient with me,
/fuad

author
#12 lore raw

PATCH v4 6/9 KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM

In reply to Fuad Tabba

When running in protected mode we don't want to leak protected guest state to the host, including whether a guest has used fpsimd/sve. Therefore, eagerly restore the host state on guest exit when running in protected mode, which happens only if the guest has used fpsimd/sve.

---
 arch/arm64/kvm/hyp/include/hyp/switch.h | 13 ++++-
 arch/arm64/kvm/hyp/nvhe/hyp-main.c      | 67 +++++++++++++++++++++++--
 arch/arm64/kvm/hyp/nvhe/pkvm.c          |  2 +
 arch/arm64/kvm/hyp/nvhe/switch.c        | 16 +++++-
 4 files changed, 93 insertions(+), 5 deletions(-)
arch/arm64/kvm/hyp/include/hyp/switch.h +12 -1
index 9b904c858df0..0c4de44534b7 100644
@@ -321,6 +321,17 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu)
 	write_sysreg_el1(__vcpu_sys_reg(vcpu, ZCR_EL1), SYS_ZCR);
 }
 
+static inline void __hyp_sve_save_host(void)
+{
+	struct cpu_sve_state *sve_state = *host_data_ptr(sve_state);
+
+	sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR);
+	write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2);
+	__sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl),
+			 &sve_state->fpsr,
+			 true);
+}
+
 static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu);
 
 /*
@@ -355,7 +366,7 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code)
 	/* Valid trap.  Switch the context: */
 
 	/* First disable enough traps to allow us to update the registers */
-	if (sve_guest)
+	if (sve_guest || (is_protected_kvm_enabled() && system_supports_sve()))
 		cpacr_clear_set(0, CPACR_ELx_FPEN | CPACR_ELx_ZEN);
 	else
 		cpacr_clear_set(0, CPACR_ELx_FPEN);
arch/arm64/kvm/hyp/nvhe/hyp-main.c +64 -3

94 lines not on this page · raw message

arch/arm64/kvm/hyp/nvhe/pkvm.c +2 -0
index 25e9a94f6d76..feb27b4ce459 100644
@@ -588,6 +588,8 @@ int __pkvm_init_vcpu(pkvm_handle_t handle, struct kvm_vcpu *host_vcpu,
 	if (ret)
 		unmap_donated_memory(hyp_vcpu, sizeof(*hyp_vcpu));
 
+	hyp_vcpu->vcpu.arch.cptr_el2 = kvm_get_reset_cptr_el2(&hyp_vcpu->vcpu);
+
 	return ret;
 }
 
arch/arm64/kvm/hyp/nvhe/switch.c +15 -1
index 019f863922fa..bef74de7065b 100644
@@ -184,7 +184,21 @@ static bool kvm_handle_pvm_sys64(struct kvm_vcpu *vcpu, u64 *exit_code)
 
 static void kvm_hyp_save_fpsimd_host(struct kvm_vcpu *vcpu)
 {
-	__fpsimd_save_state(*host_data_ptr(fpsimd_state));
+	/*
+	 * Non-protected kvm relies on the host restoring its sve state.
+	 * Protected kvm restores the host's sve state as not to reveal that
+	 * fpsimd was used by a guest nor leak upper sve bits.
+	 */
+	if (unlikely(is_protected_kvm_enabled() && system_supports_sve())) {
+		__hyp_sve_save_host();
+
+		/* Re-enable SVE traps if not supported for the guest vcpu. */
+		if (!vcpu_has_sve(vcpu))
+			cpacr_clear_set(CPACR_ELx_ZEN, 0);
+
+	} else {
+		__fpsimd_save_state(*host_data_ptr(fpsimd_state));
+	}
 }
 
 static const exit_handler_fn hyp_exit_handlers[] = {
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
#13 lore raw

Re: [PATCH v4 6/9] KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM

arch/arm64/kvm/hyp/nvhe/hyp-main.c DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); · line 61

Quoted 4 lines · Fuad Tabba
arch/arm64/kvm/hyp/nvhe/hyp-main.c +4 -0
+static void fpsimd_sve_flush(void)
+{
+	*host_data_ptr(fp_owner) = FP_STATE_HOST_OWNED;
+}

My previous comments about this being confusing still stand.

author
#14 lore raw

Re: [PATCH v4 6/9] KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM

Hi Mark,

Quoted 9 lines · Mark Brown

arch/arm64/kvm/hyp/nvhe/hyp-main.c DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); · line 61

Quoted 4 lines · Fuad Tabba
arch/arm64/kvm/hyp/nvhe/hyp-main.c +4 -0
+static void fpsimd_sve_flush(void)
+{
+     *host_data_ptr(fp_owner) = FP_STATE_HOST_OWNED;
+}

My previous comments about this being confusing still stand.

Sorry, I missed this in my reply to v3.

This follows the convention for save/flush in hyp-main.c. Since the act of flushing the fpsimd/sve state is lazy, i.e., only takes place if the guest were to use fpsimd/sve, then the only thing that we need to do to flush is to mark the state as owned by the host.

You suggested inlining this, but since this is static, I think the compiler would do that. Even though it's only one line, it maintains symmetry with fpsimd_sve_sync().

Cheers,
/fuad

#15 lore raw

Re: [PATCH v4 6/9] KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM

Quoted 10 lines · Fuad Tabba
Quoted 8 lines · Mark Brown

arch/arm64/kvm/hyp/nvhe/hyp-main.c DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); · line 61

Quoted 4 lines · Fuad Tabba
arch/arm64/kvm/hyp/nvhe/hyp-main.c +4 -0
+static void fpsimd_sve_flush(void)
+{
+     *host_data_ptr(fp_owner) = FP_STATE_HOST_OWNED;
+}

My previous comments about this being confusing still stand.

Quoted 6 lines

Sorry, I missed this in my reply to v3.

This follows the convention for save/flush in hyp-main.c. Since the act of flushing the fpsimd/sve state is lazy, i.e., only takes place if the guest were to use fpsimd/sve, then the only thing that we need to do to flush is to mark the state as owned by the host.

I think this needs a comment mentioning what's going on here.

Quoted 3 lines

You suggested inlining this, but since this is static, I think the compiler would do that. Even though it's only one line, it maintains symmetry with fpsimd_sve_sync().

The reason I was suggesting inlining was that it removes the need to name the function.

maintainer
#16 lore raw

Re: [PATCH v4 6/9] KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM

Quoted 20 lines · Mark Brown

[1 <text/plain; utf-8 (quoted-printable)>]

Quoted 10 lines · Fuad Tabba
Quoted 8 lines · Mark Brown

arch/arm64/kvm/hyp/nvhe/hyp-main.c DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); · line 61

Quoted 4 lines · Fuad Tabba
arch/arm64/kvm/hyp/nvhe/hyp-main.c +4 -0
+static void fpsimd_sve_flush(void)
+{
+     *host_data_ptr(fp_owner) = FP_STATE_HOST_OWNED;
+}

My previous comments about this being confusing still stand.

Quoted 6 lines

Sorry, I missed this in my reply to v3.

This follows the convention for save/flush in hyp-main.c. Since the act of flushing the fpsimd/sve state is lazy, i.e., only takes place if the guest were to use fpsimd/sve, then the only thing that we need to do to flush is to mark the state as owned by the host.

Quoted 8 lines

I think this needs a comment mentioning what's going on here.

Quoted 3 lines

You suggested inlining this, but since this is static, I think the compiler would do that. Even though it's only one line, it maintains symmetry with fpsimd_sve_sync().

The reason I was suggesting inlining was that it removes the need to name the function.

You're missing the point.

The name is important, and the current name is correct. We use *flush (resp. *sync) for operations that need to happen before (resp. after) the entry into the guest. This is consistent with the rest of the code base for most of the subsystems KVM/arm64 deals with.

So the current form of this function, as a standalone function with its current name, stays.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#17 lore raw

Re: [PATCH v4 6/9] KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM

Quoted 11 lines · Marc Zyngier
Quoted 8 lines · Mark Brown
Quoted 3 lines · Fuad Tabba

You suggested inlining this, but since this is static, I think the compiler would do that. Even though it's only one line, it maintains symmetry with fpsimd_sve_sync().

The reason I was suggesting inlining was that it removes the need to name the function.

You're missing the point.

I think not.

Quoted 4 lines

The name is important, and the current name is correct. We use *flush (resp. *sync) for operations that need to happen before (resp. after) the entry into the guest. This is consistent with the rest of the code base for most of the subsystems KVM/arm64 deals with.

Hence my suggestion to add documentation to the function to avoid the risk of surprises, the above text explains the motivation for the suggestion that Faud is responding to. Either approach would work.

author
#18 lore raw

PATCH v4 7/9 KVM: arm64: Consolidate initializing the host data's fpsimd_state/sve in pKVM

In reply to Fuad Tabba

Now that we have introduced finalize_init_hyp_mode(), lets consolidate the initializing of the host_data fpsimd_state and sve state.

---
 arch/arm64/include/asm/kvm_host.h      | 10 ++++++++--
 arch/arm64/kvm/arm.c                   | 20 ++++++++++++++------
 arch/arm64/kvm/hyp/include/nvhe/pkvm.h |  1 -
 arch/arm64/kvm/hyp/nvhe/pkvm.c         | 11 -----------
 arch/arm64/kvm/hyp/nvhe/setup.c        |  1 -
 5 files changed, 22 insertions(+), 21 deletions(-)
arch/arm64/include/asm/kvm_host.h +8 -2
index 90df7ccec5f4..36b8e97bf49e 100644
@@ -550,8 +550,14 @@ struct cpu_sve_state {
 struct kvm_host_data {
 	struct kvm_cpu_context host_ctxt;
 
-	struct user_fpsimd_state *fpsimd_state;	/* hyp VA */
-	struct cpu_sve_state *sve_state;	/* hyp VA */
+	/*
+	 * All pointers in this union are hyp VA.
+	 * sve_state is only used in pKVM and if system_supports_sve().
+	 */
+	union {
+		struct user_fpsimd_state *fpsimd_state;
+		struct cpu_sve_state *sve_state;
+	};
 
 	/* Ownership of the FP regs */
 	enum {
arch/arm64/kvm/arm.c +14 -6
index 1acf7415e831..59716789fe0f 100644
@@ -2444,14 +2444,22 @@ static void finalize_init_hyp_mode(void)
 {
 	int cpu;
 
-	if (!is_protected_kvm_enabled() || !system_supports_sve())
-		return;
+	if (system_supports_sve() && is_protected_kvm_enabled()) {
+		for_each_possible_cpu(cpu) {
+			struct cpu_sve_state *sve_state;
 
-	for_each_possible_cpu(cpu) {
-		struct cpu_sve_state *sve_state;
+			sve_state = per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state;
+			per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state =
+				kern_hyp_va(sve_state);
+		}
+	} else {
+		for_each_possible_cpu(cpu) {
+			struct user_fpsimd_state *fpsimd_state;
 
-		sve_state = per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state;
-		per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state = kern_hyp_va(sve_state);
+			fpsimd_state = &per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->host_ctxt.fp_regs;
+			per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->fpsimd_state =
+				kern_hyp_va(fpsimd_state);
+		}
 	}
 }
 
arch/arm64/kvm/hyp/include/nvhe/pkvm.h +0 -1
index 22f374e9f532..24a9a8330d19 100644
@@ -59,7 +59,6 @@ static inline bool pkvm_hyp_vcpu_is_protected(struct pkvm_hyp_vcpu *hyp_vcpu)
 }
 
 void pkvm_hyp_vm_table_init(void *tbl);
-void pkvm_host_fpsimd_state_init(void);
 
 int __pkvm_init_vm(struct kvm *host_kvm, unsigned long vm_hva,
 		   unsigned long pgd_hva);
arch/arm64/kvm/hyp/nvhe/pkvm.c +0 -11
index feb27b4ce459..ea67fcbf8376 100644
@@ -249,17 +249,6 @@ void pkvm_hyp_vm_table_init(void *tbl)
 	vm_table = tbl;
 }
 
-void pkvm_host_fpsimd_state_init(void)
-{
-	unsigned long i;
-
-	for (i = 0; i < hyp_nr_cpus; i++) {
-		struct kvm_host_data *host_data = per_cpu_ptr(&kvm_host_data, i);
-
-		host_data->fpsimd_state = &host_data->host_ctxt.fp_regs;
-	}
-}
-
 /*
  * Return the hyp vm structure corresponding to the handle.
  */
arch/arm64/kvm/hyp/nvhe/setup.c +0 -1
index 3fae42479598..f4350ba07b0b 100644
@@ -324,7 +324,6 @@ void __noreturn __pkvm_init_finalise(void)
 		goto out;
 
 	pkvm_hyp_vm_table_init(vm_table_base);
-	pkvm_host_fpsimd_state_init();
 out:
 	/*
 	 * We tail-called to here from handle___pkvm_init() and will not return,
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
Reviewed-by on v4 7/9
#19 lore raw

Re: [PATCH v4 7/9] KVM: arm64: Consolidate initializing the host data's fpsimd_state/sve in pKVM

Quoted 3 lines · Fuad Tabba

Now that we have introduced finalize_init_hyp_mode(), lets consolidate the initializing of the host_data fpsimd_state and sve state.

author
#20 lore raw

PATCH v4 8/9 KVM: arm64: Refactor CPACR trap bit setting/clearing to use ELx format

In reply to Fuad Tabba

When setting/clearing CPACR bits for EL0 and EL1, use the ELx format of the bits, which covers both. This makes the code clearer, and reduces the chances of accidentally missing a bit.

No functional change intended.

---
 arch/arm64/include/asm/el2_setup.h   | 6 +++---
 arch/arm64/include/asm/kvm_emulate.h | 9 ++++-----
 arch/arm64/kvm/fpsimd.c              | 4 +---
 arch/arm64/kvm/hyp/nvhe/pkvm.c       | 2 +-
 arch/arm64/kvm/hyp/nvhe/switch.c     | 5 ++---
 arch/arm64/kvm/hyp/vhe/switch.c      | 7 +++----
 6 files changed, 14 insertions(+), 19 deletions(-)
arch/arm64/include/asm/el2_setup.h +3 -3
index e4546b29dd0c..fd87c4b8f984 100644
@@ -146,7 +146,7 @@
 /* Coprocessor traps */
 .macro __init_el2_cptr
 	__check_hvhe .LnVHE_\@, x1
-	mov	x0, #(CPACR_EL1_FPEN_EL1EN | CPACR_EL1_FPEN_EL0EN)
+	mov	x0, #CPACR_ELx_FPEN
 	msr	cpacr_el1, x0
 	b	.Lskip_set_cptr_\@
 .LnVHE_\@:
@@ -277,7 +277,7 @@
 
 	// (h)VHE case
 	mrs	x0, cpacr_el1			// Disable SVE traps
-	orr	x0, x0, #(CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
+	orr	x0, x0, #CPACR_ELx_ZEN
 	msr	cpacr_el1, x0
 	b	.Lskip_set_cptr_\@
 
@@ -298,7 +298,7 @@
 
 	// (h)VHE case
 	mrs	x0, cpacr_el1			// Disable SME traps
-	orr	x0, x0, #(CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN)
+	orr	x0, x0, #CPACR_ELx_SMEN
 	msr	cpacr_el1, x0
 	b	.Lskip_set_cptr_sme_\@
 
arch/arm64/include/asm/kvm_emulate.h +4 -5
index 2d7a0bdf9d03..21650e7924d4 100644
@@ -632,17 +632,16 @@ static __always_inline u64 kvm_get_reset_cptr_el2(struct kvm_vcpu *vcpu)
 	u64 val;
 
 	if (has_vhe()) {
-		val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN |
-		       CPACR_EL1_ZEN_EL1EN);
+		val = (CPACR_ELx_FPEN | CPACR_EL1_ZEN_EL1EN);
 		if (cpus_have_final_cap(ARM64_SME))
 			val |= CPACR_EL1_SMEN_EL1EN;
 	} else if (has_hvhe()) {
-		val = (CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN);
+		val = CPACR_ELx_FPEN;
 
 		if (!vcpu_has_sve(vcpu) || !guest_owns_fp_regs())
-			val |= CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN;
+			val |= CPACR_ELx_ZEN;
 		if (cpus_have_final_cap(ARM64_SME))
-			val |= CPACR_EL1_SMEN_EL1EN | CPACR_EL1_SMEN_EL0EN;
+			val |= CPACR_ELx_SMEN;
 	} else {
 		val = CPTR_NVHE_EL2_RES1;
 
arch/arm64/kvm/fpsimd.c +1 -3
index 1807d3a79a8a..eb21f29d91fc 100644
@@ -161,9 +161,7 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu)
 	if (has_vhe() && system_supports_sme()) {
 		/* Also restore EL0 state seen on entry */
 		if (vcpu_get_flag(vcpu, HOST_SME_ENABLED))
-			sysreg_clear_set(CPACR_EL1, 0,
-					 CPACR_EL1_SMEN_EL0EN |
-					 CPACR_EL1_SMEN_EL1EN);
+			sysreg_clear_set(CPACR_EL1, 0, CPACR_ELx_SMEN);
 		else
 			sysreg_clear_set(CPACR_EL1,
 					 CPACR_EL1_SMEN_EL0EN,
arch/arm64/kvm/hyp/nvhe/pkvm.c +1 -1
index ea67fcbf8376..95cf18574251 100644
@@ -65,7 +65,7 @@ static void pvm_init_traps_aa64pfr0(struct kvm_vcpu *vcpu)
 	/* Trap SVE */
 	if (!FIELD_GET(ARM64_FEATURE_MASK(ID_AA64PFR0_EL1_SVE), feature_ids)) {
 		if (has_hvhe())
-			cptr_clear |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN;
+			cptr_clear |= CPACR_ELx_ZEN;
 		else
 			cptr_set |= CPTR_EL2_TZ;
 	}
arch/arm64/kvm/hyp/nvhe/switch.c +2 -3
index bef74de7065b..6af179c6356d 100644
@@ -48,15 +48,14 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
 	val |= has_hvhe() ? CPACR_EL1_TTA : CPTR_EL2_TTA;
 	if (cpus_have_final_cap(ARM64_SME)) {
 		if (has_hvhe())
-			val &= ~(CPACR_EL1_SMEN_EL1EN | CPACR_EL1_SMEN_EL0EN);
+			val &= ~CPACR_ELx_SMEN;
 		else
 			val |= CPTR_EL2_TSM;
 	}
 
 	if (!guest_owns_fp_regs()) {
 		if (has_hvhe())
-			val &= ~(CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN |
-				 CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN);
+			val &= ~(CPACR_ELx_FPEN | CPACR_ELx_ZEN);
 		else
 			val |= CPTR_EL2_TFP | CPTR_EL2_TZ;
 
arch/arm64/kvm/hyp/vhe/switch.c +3 -4
index 20073579e9f5..8fbb6a2e0559 100644
@@ -93,8 +93,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
 
 	val = read_sysreg(cpacr_el1);
 	val |= CPACR_ELx_TTA;
-	val &= ~(CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN |
-		 CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN);
+	val &= ~(CPACR_ELx_ZEN | CPACR_ELx_SMEN);
 
 	/*
 	 * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to
@@ -109,9 +108,9 @@ static void __activate_traps(struct kvm_vcpu *vcpu)
 
 	if (guest_owns_fp_regs()) {
 		if (vcpu_has_sve(vcpu))
-			val |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN;
+			val |= CPACR_ELx_ZEN;
 	} else {
-		val &= ~(CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN);
+		val &= ~CPACR_ELx_FPEN;
 		__activate_traps_fpsimd32(vcpu);
 	}
 
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
author
#21 lore raw

PATCH v4 9/9 KVM: arm64: Ensure that SME controls are disabled in protected mode

In reply to Fuad Tabba

KVM (and pKVM) do not support SME guests. Therefore KVM ensures that the host's SME state is flushed and that SME controls for enabling access to ZA storage and for streaming are disabled.

pKVM needs to protect against a buggy/malicious host. Ensure that it wouldn't run a guest when protected mode is enabled should any of the SME controls be enabled.

---
 arch/arm64/kvm/fpsimd.c            |  7 +++++++
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +++++++++++
 2 files changed, 18 insertions(+)
arch/arm64/kvm/fpsimd.c +7 -0
index eb21f29d91fc..521b32868d0d 100644
@@ -90,6 +90,13 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 			fpsimd_save_and_flush_cpu_state();
 		}
 	}
+
+	/*
+	 * If normal guests gain SME support, maintain this behavior for pKVM
+	 * guests, which don't support SME.
+	 */
+	WARN_ON(is_protected_kvm_enabled() && system_supports_sme() &&
+		read_sysreg_s(SYS_SVCR));
 }
 
 /*
arch/arm64/kvm/hyp/nvhe/hyp-main.c +11 -0
index bd93b8a9e172..f43d845f3c4e 100644
@@ -140,6 +140,17 @@ static void handle___kvm_vcpu_run(struct kvm_cpu_context *host_ctxt)
 		struct pkvm_hyp_vcpu *hyp_vcpu;
 		struct kvm *host_kvm;
 
+		/*
+		 * KVM (and pKVM) doesn't support SME guests for now, and
+		 * ensures that SME features aren't enabled in pstate when
+		 * loading a vcpu. Therefore, if SME features enabled the host
+		 * is misbehaving.
+		 */
+		if (unlikely(system_supports_sme() && read_sysreg_s(SYS_SVCR))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
 		host_kvm = kern_hyp_va(host_vcpu->kvm);
 		hyp_vcpu = pkvm_load_hyp_vcpu(host_kvm->arch.pkvm.handle,
 					      host_vcpu->vcpu_idx);
Signature · 1 line
2.45.1.288.g0e0cd299f1-goog
#22 lore raw

Re: [PATCH v4 9/9] KVM: arm64: Ensure that SME controls are disabled in protected mode

arch/arm64/kvm/fpsimd.c void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu) · line 100

Quoted 13 lines · Fuad Tabba

+++ b/arch/arm64/kvm/fpsimd.c

arch/arm64/kvm/fpsimd.c +7 -0
@@ -90,6 +90,13 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu)
 			fpsimd_save_and_flush_cpu_state();
 		}
 	}
+
+	/*
+	 * If normal guests gain SME support, maintain this behavior for pKVM
+	 * guests, which don't support SME.
+	 */
+	WARN_ON(is_protected_kvm_enabled() && system_supports_sme() &&
+		read_sysreg_s(SYS_SVCR));
 }

The comment doesn't line up clearly with the check here, we're checking if the hypervisor supports protected guests not if the current guest is protected. At this point we're dealing with the host state rather than the guest state so the guest being protected or not doesn't matter when we check SVCR. I'm not sure this assert really makes sense, or perhaps it should be hoisted into the above system_support_sme() section?

maintainer Applied
#23 lore raw
In reply to Fuad Tabba
Quoted 11 lines · Fuad Tabba

Changes since v3 [1]:
- Rebased on Linux 6.10-rc2 (c3f38fa61af7)
- Dropped v3 patches 8--11 (Oliver)
- Removed unnecessary isb()s (Oliver)
- Formatting/comments (Mark)
- Fix __sve_save_state()/__sve_restore_state() prototypes (Mark)
- Save/restore ffr with the sve state
- Added a patch that checks at hyp that SME features aren't enabled on guest entry, to ensure it's not in streaming mode

[...]

Applied to fixes, thanks!

[1/9] KVM: arm64: Reintroduce __sve_save_state

      commit: 87bb39ed40bdf1596b8820e800226e24eb642677
[2/9] KVM: arm64: Fix prototype for __sve_save_state/__sve_restore_state
      commit: 45f4ea9bcfe909b3461059990b1e232e55dde809
[3/9] KVM: arm64: Abstract set/clear of CPTR_EL2 bits behind helper
      commit: 6d8fb3cbf7e06431a607c30c1bc4cd53a62c220a
[4/9] KVM: arm64: Specialize handling of host fpsimd state on trap
      commit: e511e08a9f496948b13aac50610f2d17335f56c3
[5/9] KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM
      commit: 66d5b53e20a6e00b7ce3b652a3e2db967f7b33d0
[6/9] KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM
      commit: b5b9955617bc0b41546f2fa7c3dbcc048b43dc82
[7/9] KVM: arm64: Consolidate initializing the host data's fpsimd_state/sve in pKVM
      commit: 1696fc2174dbab12228ea9ec4c213d6aeea348f8
[8/9] KVM: arm64: Refactor CPACR trap bit setting/clearing to use ELx format
      commit: a69283ae1db8dd416870d931caa9e2d3d2c1cd8b
[9/9] KVM: arm64: Ensure that SME controls are disabled in protected mode
      commit: afb91f5f8ad7af172d993a34fde1947892408f53

Cheers,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.