Skip to contentSkip to conversation

Risc-V Kvm Smstateen

PATCH Superseded Mayuresh Chitale 4 participants 13 messages Jul 24, 2023 – Jul 25, 2023

Subsystems (3)
Device tree bindings RISC-V KVM RISC-V

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Mayuresh Chitale

author
#0 lore raw

v3 cover Risc-V Kvm Smstateen

This series adds support to detect the Smstateen extension for both, the host and the guest vcpu. It also adds senvcfg and sstateen0 to the ONE_REG interface and the vcpu context save/restore.

Changes in v3:
- Move DT bindings change to a separate patch

Changes in v2:
- Add smstaeen description in riscv/extensions.yaml
- Avoid line wrap at 80 chars

Mayuresh Chitale (7):
RISC-V: Detect Smstateen extension
dt-bindings: riscv: Add smstateen entry
RISC-V: KVM: Add kvm_vcpu_config
RISC-V: KVM: Enable Smstateen accesses
RISCV: KVM: Add senvcfg context save/restore
RISCV: KVM: Add sstateen0 context save/restore
RISCV: KVM: Add sstateen0 to ONE_REG

 .../devicetree/bindings/riscv/extensions.yaml |   6 +
 arch/riscv/include/asm/csr.h                  |  18 +++
 arch/riscv/include/asm/hwcap.h                |   1 +
 arch/riscv/include/asm/kvm_host.h             |  18 +++
 arch/riscv/include/uapi/asm/kvm.h             |  11 ++
 arch/riscv/kernel/cpu.c                       |   1 +
 arch/riscv/kernel/cpufeature.c                |   1 +
 arch/riscv/kvm/vcpu.c                         | 111 ++++++++++++++++--
 8 files changed, 154 insertions(+), 13 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v3 1/7 RISC-V: Detect Smstateen extension

Extend the ISA string parsing to detect the Smstateen extension

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 arch/riscv/include/asm/hwcap.h | 1 +
 arch/riscv/kernel/cpu.c        | 1 +
 arch/riscv/kernel/cpufeature.c | 1 +
 3 files changed, 3 insertions(+)
arch/riscv/include/asm/hwcap.h +1 -0
index f041bfa7f6a0..fad1fd1fcd05 100644
@@ -53,6 +53,7 @@
 #define RISCV_ISA_EXT_ZICSR		40
 #define RISCV_ISA_EXT_ZIFENCEI		41
 #define RISCV_ISA_EXT_ZIHPM		42
+#define RISCV_ISA_EXT_SMSTATEEN		43
 
 #define RISCV_ISA_EXT_MAX		64
 #define RISCV_ISA_EXT_NAME_LEN_MAX	32
arch/riscv/kernel/cpu.c +1 -0
index a2fc952318e9..fb0df651bc48 100644
@@ -217,6 +217,7 @@ static struct riscv_isa_ext_data isa_ext_arr[] = {
 	__RISCV_ISA_EXT_DATA(zbb, RISCV_ISA_EXT_ZBB),
 	__RISCV_ISA_EXT_DATA(zbs, RISCV_ISA_EXT_ZBS),
 	__RISCV_ISA_EXT_DATA(smaia, RISCV_ISA_EXT_SMAIA),
+	__RISCV_ISA_EXT_DATA(smstateen, RISCV_ISA_EXT_SMSTATEEN),
 	__RISCV_ISA_EXT_DATA(ssaia, RISCV_ISA_EXT_SSAIA),
 	__RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF),
 	__RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC),
arch/riscv/kernel/cpufeature.c +1 -0
index a8f66c015229..c3742a765f8b 100644
@@ -301,6 +301,7 @@ void __init riscv_fill_hwcap(void)
 			} else {
 				/* sorted alphabetically */
 				SET_ISA_EXT_MAP("smaia", RISCV_ISA_EXT_SMAIA);
+				SET_ISA_EXT_MAP("smstateen", RISCV_ISA_EXT_SMSTATEEN);
 				SET_ISA_EXT_MAP("ssaia", RISCV_ISA_EXT_SSAIA);
 				SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF);
 				SET_ISA_EXT_MAP("sstc", RISCV_ISA_EXT_SSTC);
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH v3 2/7 dt-bindings: riscv: Add smstateen entry

In reply to Mayuresh Chitale

Add an entry for the Smstateen extension to the riscv,isa-extensions property.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 Documentation/devicetree/bindings/riscv/extensions.yaml | 6 ++++++
 1 file changed, 6 insertions(+)
Documentation/devicetree/bindings/riscv/extensions.yaml +6 -0
index cc1f546fdbdc..36ff6749fbba 100644
@@ -128,6 +128,12 @@ properties:
             changes to interrupts as frozen at commit ccbddab ("Merge pull
             request #42 from riscv/jhauser-2023-RC4") of riscv-aia.
 
+        - const: smstateen
+          description: |
+            The standard Smstateen extension for controlling access to CSRs
+            added by other RISC-V extensions in H/S/VS/U/VU modes and as
+            ratified at commit a28bfae (Ratified (#7)) of riscv-state-enable.
+
         - const: ssaia
           description: |
             The standard Ssaia supervisor-level extension for the advanced
Signature · 1 line
2.34.1
#3 lore raw

PATCH v3 2/7 dt-bindings: riscv: Add smstateen entry

Quoted 5 lines · Mayuresh Chitale

Add an entry for the Smstateen extension to the riscv,isa-extensions property.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
Reviewed-by Conor Dooley <[email protected]>

Thanks,
Conor.

Quoted 24 lines
---
 Documentation/devicetree/bindings/riscv/extensions.yaml | 6 ++++++
 1 file changed, 6 insertions(+)
Documentation/devicetree/bindings/riscv/extensions.yaml +6 -0
index cc1f546fdbdc..36ff6749fbba 100644
@@ -128,6 +128,12 @@ properties:
             changes to interrupts as frozen at commit ccbddab ("Merge pull
             request #42 from riscv/jhauser-2023-RC4") of riscv-aia.
 
+        - const: smstateen
+          description: |
+            The standard Smstateen extension for controlling access to CSRs
+            added by other RISC-V extensions in H/S/VS/U/VU modes and as
+            ratified at commit a28bfae (Ratified (#7)) of riscv-state-enable.
+
         - const: ssaia
           description: |
             The standard Ssaia supervisor-level extension for the advanced
Signature · 1 line
2.34.1

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kvm-riscv/attachments/20230724/57ad6f6d/attachment-0001.sig>

author
#4 lore raw

PATCH v3 3/7 RISC-V: KVM: Add kvm_vcpu_config

In reply to Mayuresh Chitale

Add a placeholder for all registers such as henvcfg, hstateen etc which have 'static' configurations depending on extensions supported by the guest. The values are derived once and are then subsequently written to the corresponding CSRs while switching to the vcpu.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 arch/riscv/include/asm/kvm_host.h |  7 +++++++
 arch/riscv/kvm/vcpu.c             | 27 ++++++++++++++-------------
 2 files changed, 21 insertions(+), 13 deletions(-)
arch/riscv/include/asm/kvm_host.h +7 -0
index 2d8ee53b66c7..c0c50b4b3394 100644
@@ -164,6 +164,10 @@ struct kvm_vcpu_csr {
 	unsigned long scounteren;
 };
 
+struct kvm_vcpu_config {
+	u64 henvcfg;
+};
+
 struct kvm_vcpu_arch {
 	/* VCPU ran at least once */
 	bool ran_atleast_once;

@@ -244,6 +248,9 @@ struct kvm_vcpu_arch {
 
 	/* Performance monitoring context */
 	struct kvm_pmu pmu_context;
+
+	/* 'static' configurations which are set only once */
+	struct kvm_vcpu_config cfg;
 };
 
 static inline void kvm_arch_sync_events(struct kvm *kvm) {}
arch/riscv/kvm/vcpu.c +14 -13
index d12ef99901fc..e01f47bb636f 100644
@@ -980,31 +980,28 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
 	return -EINVAL;
 }
 
-static void kvm_riscv_vcpu_update_config(const unsigned long *isa)
+static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
 {
-	u64 henvcfg = 0;
+	const unsigned long *isa = vcpu->arch.isa;
+	struct kvm_vcpu_config *cfg = &vcpu->arch.cfg;
 
 	if (riscv_isa_extension_available(isa, SVPBMT))
-		henvcfg |= ENVCFG_PBMTE;
+		cfg->henvcfg |= ENVCFG_PBMTE;
 
 	if (riscv_isa_extension_available(isa, SSTC))
-		henvcfg |= ENVCFG_STCE;
+		cfg->henvcfg |= ENVCFG_STCE;
 
 	if (riscv_isa_extension_available(isa, ZICBOM))
-		henvcfg |= (ENVCFG_CBIE | ENVCFG_CBCFE);
+		cfg->henvcfg |= (ENVCFG_CBIE | ENVCFG_CBCFE);
 
 	if (riscv_isa_extension_available(isa, ZICBOZ))
-		henvcfg |= ENVCFG_CBZE;
-
-	csr_write(CSR_HENVCFG, henvcfg);
-#ifdef CONFIG_32BIT
-	csr_write(CSR_HENVCFGH, henvcfg >> 32);
-#endif
+		cfg->henvcfg |= ENVCFG_CBZE;
 }
 
 void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 {
 	struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+	struct kvm_vcpu_config *cfg = &vcpu->arch.cfg;
 
 	csr_write(CSR_VSSTATUS, csr->vsstatus);
 	csr_write(CSR_VSIE, csr->vsie);

@@ -1015,8 +1012,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	csr_write(CSR_VSTVAL, csr->vstval);
 	csr_write(CSR_HVIP, csr->hvip);
 	csr_write(CSR_VSATP, csr->vsatp);
-
-	kvm_riscv_vcpu_update_config(vcpu->arch.isa);
+	csr_write(CSR_HENVCFG, cfg->henvcfg);
+	if (IS_ENABLED(CONFIG_32BIT))
+		csr_write(CSR_HENVCFGH, cfg->henvcfg >> 32);
 
 	kvm_riscv_gstage_update_hgatp(vcpu);
 

@@ -1136,6 +1134,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 	struct kvm_cpu_trap trap;
 	struct kvm_run *run = vcpu->run;
 
+	if (!vcpu->arch.ran_atleast_once)
+		kvm_riscv_vcpu_setup_config(vcpu);
+
 	/* Mark this VCPU ran at least once */
 	vcpu->arch.ran_atleast_once = true;
Signature · 1 line
2.34.1
author
#5 lore raw

PATCH v3 4/7 RISC-V: KVM: Enable Smstateen accesses

In reply to Mayuresh Chitale

Configure hstateen0 register so that the AIA state and envcfg are accessible to the vcpus. This includes registers such as siselect, sireg, siph, sieh and all the IMISC registers.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 arch/riscv/include/asm/csr.h      | 16 ++++++++++++++++
 arch/riscv/include/asm/kvm_host.h |  1 +
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 16 ++++++++++++++++
 4 files changed, 34 insertions(+)
arch/riscv/include/asm/csr.h +16 -0
index 7bac43a3176e..38730677dcd5 100644
@@ -201,6 +201,18 @@
 #define ENVCFG_CBIE_INV			_AC(0x3, UL)
 #define ENVCFG_FIOM			_AC(0x1, UL)
 
+/* Smstateen bits */
+#define SMSTATEEN0_AIA_IMSIC_SHIFT	58
+#define SMSTATEEN0_AIA_IMSIC		(_ULL(1) << SMSTATEEN0_AIA_IMSIC_SHIFT)
+#define SMSTATEEN0_AIA_SHIFT		59
+#define SMSTATEEN0_AIA			(_ULL(1) << SMSTATEEN0_AIA_SHIFT)
+#define SMSTATEEN0_AIA_ISEL_SHIFT	60
+#define SMSTATEEN0_AIA_ISEL		(_ULL(1) << SMSTATEEN0_AIA_ISEL_SHIFT)
+#define SMSTATEEN0_HSENVCFG_SHIFT	62
+#define SMSTATEEN0_HSENVCFG		(_ULL(1) << SMSTATEEN0_HSENVCFG_SHIFT)
+#define SMSTATEEN0_SSTATEEN0_SHIFT	63
+#define SMSTATEEN0_SSTATEEN0		(_ULL(1) << SMSTATEEN0_SSTATEEN0_SHIFT)
+
 /* symbolic CSR names: */
 #define CSR_CYCLE		0xc00
 #define CSR_TIME		0xc01

@@ -347,6 +359,10 @@
 #define CSR_VSIEH		0x214
 #define CSR_VSIPH		0x254
 
+/* Hypervisor stateen CSRs */
+#define CSR_HSTATEEN0		0x60c
+#define CSR_HSTATEEN0H		0x61c
+
 #define CSR_MSTATUS		0x300
 #define CSR_MISA		0x301
 #define CSR_MIDELEG		0x303
arch/riscv/include/asm/kvm_host.h +1 -0
index c0c50b4b3394..ee55e5fc8b84 100644
@@ -166,6 +166,7 @@ struct kvm_vcpu_csr {
 
 struct kvm_vcpu_config {
 	u64 henvcfg;
+	u64 hstateen0;
 };
 
 struct kvm_vcpu_arch {
arch/riscv/include/uapi/asm/kvm.h +1 -0
index 930fdc4101cd..7bc1634b0a89 100644
@@ -124,6 +124,7 @@ enum KVM_RISCV_ISA_EXT_ID {
 	KVM_RISCV_ISA_EXT_SSAIA,
 	KVM_RISCV_ISA_EXT_V,
 	KVM_RISCV_ISA_EXT_SVNAPOT,
+	KVM_RISCV_ISA_EXT_SMSTATEEN,
 	KVM_RISCV_ISA_EXT_MAX,
 };
arch/riscv/kvm/vcpu.c +16 -0
index e01f47bb636f..d3166b676430 100644
@@ -70,6 +70,7 @@ static const unsigned long kvm_isa_ext_arr[] = {
 	KVM_ISA_EXT_ARR(ZIHINTPAUSE),
 	KVM_ISA_EXT_ARR(ZICBOM),
 	KVM_ISA_EXT_ARR(ZICBOZ),
+	KVM_ISA_EXT_ARR(SMSTATEEN),
 };
 
 static unsigned long kvm_riscv_vcpu_base2isa_ext(unsigned long base_ext)

@@ -996,6 +997,16 @@ static void kvm_riscv_vcpu_setup_config(struct kvm_vcpu *vcpu)
 
 	if (riscv_isa_extension_available(isa, ZICBOZ))
 		cfg->henvcfg |= ENVCFG_CBZE;
+
+	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN)) {
+		cfg->hstateen0 |= SMSTATEEN0_HSENVCFG;
+		if (riscv_isa_extension_available(isa, SSAIA))
+			cfg->hstateen0 |= SMSTATEEN0_AIA_IMSIC |
+					  SMSTATEEN0_AIA |
+					  SMSTATEEN0_AIA_ISEL;
+		if (riscv_isa_extension_available(isa, SMSTATEEN))
+			cfg->hstateen0 |= SMSTATEEN0_SSTATEEN0;
+	}
 }
 
 void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)

@@ -1015,6 +1026,11 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 	csr_write(CSR_HENVCFG, cfg->henvcfg);
 	if (IS_ENABLED(CONFIG_32BIT))
 		csr_write(CSR_HENVCFGH, cfg->henvcfg >> 32);
+	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN)) {
+		csr_write(CSR_HSTATEEN0, cfg->hstateen0);
+		if (IS_ENABLED(CONFIG_32BIT))
+			csr_write(CSR_HSTATEEN0H, cfg->hstateen0 >> 32);
+	}
 
 	kvm_riscv_gstage_update_hgatp(vcpu);
Signature · 1 line
2.34.1
author
#6 lore raw

PATCH v3 5/7 RISCV: KVM: Add senvcfg context save/restore

In reply to Mayuresh Chitale

Add senvcfg context save/restore for guest VCPUs and also add it to the ONE_REG interface to allow its access from user space.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 arch/riscv/include/asm/csr.h      |  1 +
 arch/riscv/include/asm/kvm_host.h |  2 ++
 arch/riscv/include/uapi/asm/kvm.h |  1 +
 arch/riscv/kvm/vcpu.c             | 16 ++++++++++++++++
 4 files changed, 20 insertions(+)
arch/riscv/include/asm/csr.h +1 -0
index 38730677dcd5..b52270278733 100644
@@ -285,6 +285,7 @@
 #define CSR_SIE			0x104
 #define CSR_STVEC		0x105
 #define CSR_SCOUNTEREN		0x106
+#define CSR_SENVCFG		0x10a
 #define CSR_SSCRATCH		0x140
 #define CSR_SEPC		0x141
 #define CSR_SCAUSE		0x142
arch/riscv/include/asm/kvm_host.h +2 -0
index ee55e5fc8b84..c3cc0cb39cf8 100644
@@ -162,6 +162,7 @@ struct kvm_vcpu_csr {
 	unsigned long hvip;
 	unsigned long vsatp;
 	unsigned long scounteren;
+	unsigned long senvcfg;
 };
 
 struct kvm_vcpu_config {

@@ -188,6 +189,7 @@ struct kvm_vcpu_arch {
 	unsigned long host_sscratch;
 	unsigned long host_stvec;
 	unsigned long host_scounteren;
+	unsigned long host_senvcfg;
 
 	/* CPU context of Host */
 	struct kvm_cpu_context host_context;
arch/riscv/include/uapi/asm/kvm.h +1 -0
index 7bc1634b0a89..74c7f42de29d 100644
@@ -79,6 +79,7 @@ struct kvm_riscv_csr {
 	unsigned long sip;
 	unsigned long satp;
 	unsigned long scounteren;
+	unsigned long senvcfg;
 };
 
 /* AIA CSR registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
arch/riscv/kvm/vcpu.c +16 -0
index d3166b676430..5764d22efa29 100644
@@ -1129,6 +1129,20 @@ static void kvm_riscv_update_hvip(struct kvm_vcpu *vcpu)
 	kvm_riscv_vcpu_aia_update_hvip(vcpu);
 }
 
+static __always_inline void kvm_riscv_vcpu_swap_in_guest_state(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+
+	vcpu->arch.host_senvcfg = csr_swap(CSR_SENVCFG, csr->senvcfg);
+}
+
+static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+
+	csr->senvcfg = csr_swap(CSR_SENVCFG, vcpu->arch.host_senvcfg);
+}
+
 /*
  * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while
  * the vCPU is running.

@@ -1138,10 +1152,12 @@ static void kvm_riscv_update_hvip(struct kvm_vcpu *vcpu)
  */
 static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu)
 {
+	kvm_riscv_vcpu_swap_in_guest_state(vcpu);
 	guest_state_enter_irqoff();
 	__kvm_riscv_switch_to(&vcpu->arch);
 	vcpu->arch.last_exit_cpu = vcpu->cpu;
 	guest_state_exit_irqoff();
+	kvm_riscv_vcpu_swap_in_host_state(vcpu);
 }
 
 int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
Signature · 1 line
2.34.1
author
#7 lore raw

PATCH v3 6/7 RISCV: KVM: Add sstateen0 context save/restore

In reply to Mayuresh Chitale

Define sstateen0 and add sstateen0 save/restore for guest VCPUs.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 arch/riscv/include/asm/csr.h      |  1 +
 arch/riscv/include/asm/kvm_host.h |  8 ++++++++
 arch/riscv/kvm/vcpu.c             | 12 ++++++++++++
 3 files changed, 21 insertions(+)
arch/riscv/include/asm/csr.h +1 -0
index b52270278733..5168f37d8e75 100644
@@ -286,6 +286,7 @@
 #define CSR_STVEC		0x105
 #define CSR_SCOUNTEREN		0x106
 #define CSR_SENVCFG		0x10a
+#define CSR_SSTATEEN0		0x10c
 #define CSR_SSCRATCH		0x140
 #define CSR_SEPC		0x141
 #define CSR_SCAUSE		0x142
arch/riscv/include/asm/kvm_host.h +8 -0
index c3cc0cb39cf8..c9837772b109 100644
@@ -170,6 +170,10 @@ struct kvm_vcpu_config {
 	u64 hstateen0;
 };
 
+struct kvm_vcpu_smstateen_csr {
+	unsigned long sstateen0;
+};
+
 struct kvm_vcpu_arch {
 	/* VCPU ran at least once */
 	bool ran_atleast_once;

@@ -190,6 +194,7 @@ struct kvm_vcpu_arch {
 	unsigned long host_stvec;
 	unsigned long host_scounteren;
 	unsigned long host_senvcfg;
+	unsigned long host_sstateen0;
 
 	/* CPU context of Host */
 	struct kvm_cpu_context host_context;

@@ -200,6 +205,9 @@ struct kvm_vcpu_arch {
 	/* CPU CSR context of Guest VCPU */
 	struct kvm_vcpu_csr guest_csr;
 
+	/* CPU Smstateen CSR context of Guest VCPU */
+	struct kvm_vcpu_smstateen_csr smstateen_csr;
+
 	/* CPU context upon Guest VCPU reset */
 	struct kvm_cpu_context guest_reset_context;
arch/riscv/kvm/vcpu.c +12 -0
index 5764d22efa29..1f8e8b5f659b 100644
@@ -1131,16 +1131,28 @@ static void kvm_riscv_update_hvip(struct kvm_vcpu *vcpu)
 
 static __always_inline void kvm_riscv_vcpu_swap_in_guest_state(struct kvm_vcpu *vcpu)
 {
+	struct kvm_vcpu_smstateen_csr *smcsr = &vcpu->arch.smstateen_csr;
 	struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+	struct kvm_vcpu_config *cfg = &vcpu->arch.cfg;
 
 	vcpu->arch.host_senvcfg = csr_swap(CSR_SENVCFG, csr->senvcfg);
+	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN) &&
+	    (cfg->hstateen0 & SMSTATEEN0_SSTATEEN0))
+		vcpu->arch.host_sstateen0 = csr_swap(CSR_SSTATEEN0,
+						     smcsr->sstateen0);
 }
 
 static __always_inline void kvm_riscv_vcpu_swap_in_host_state(struct kvm_vcpu *vcpu)
 {
+	struct kvm_vcpu_smstateen_csr *smcsr = &vcpu->arch.smstateen_csr;
 	struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+	struct kvm_vcpu_config *cfg = &vcpu->arch.cfg;
 
 	csr->senvcfg = csr_swap(CSR_SENVCFG, vcpu->arch.host_senvcfg);
+	if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN) &&
+	    (cfg->hstateen0 & SMSTATEEN0_SSTATEEN0))
+		smcsr->sstateen0 = csr_swap(CSR_SSTATEEN0,
+					    vcpu->arch.host_sstateen0);
 }
 
 /*
Signature · 1 line
2.34.1
author
#8 lore raw

PATCH v3 7/7 RISCV: KVM: Add sstateen0 to ONE_REG

In reply to Mayuresh Chitale

Add support for sstateen0 CSR to the ONE_REG interface to allow its access from user space.

Signed-off-by Mayuresh Chitale <[email protected]>
Reviewed-by Andrew Jones <[email protected]>
---
 arch/riscv/include/uapi/asm/kvm.h |  9 +++++++
 arch/riscv/kvm/vcpu.c             | 40 +++++++++++++++++++++++++++++++
 2 files changed, 49 insertions(+)
arch/riscv/include/uapi/asm/kvm.h +9 -0
index 74c7f42de29d..bdddfb20299a 100644
@@ -93,6 +93,11 @@ struct kvm_riscv_aia_csr {
 	unsigned long iprio2h;
 };
 
+/* Smstateen CSR for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_smstateen_csr {
+	unsigned long sstateen0;
+};
+
 /* TIMER registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
 struct kvm_riscv_timer {
 	__u64 frequency;

@@ -173,10 +178,14 @@ enum KVM_RISCV_SBI_EXT_ID {
 #define KVM_REG_RISCV_CSR		(0x03 << KVM_REG_RISCV_TYPE_SHIFT)
 #define KVM_REG_RISCV_CSR_GENERAL	(0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT)
 #define KVM_REG_RISCV_CSR_AIA		(0x1 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_CSR_SMSTATEEN	(0x2 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+
 #define KVM_REG_RISCV_CSR_REG(name)	\
 		(offsetof(struct kvm_riscv_csr, name) / sizeof(unsigned long))
 #define KVM_REG_RISCV_CSR_AIA_REG(name)	\
 	(offsetof(struct kvm_riscv_aia_csr, name) / sizeof(unsigned long))
+#define KVM_REG_RISCV_CSR_SMSTATEEN_REG(name)	\
+	(offsetof(struct kvm_riscv_smstateen_csr, name) / sizeof(unsigned long))
 
 /* Timer registers are mapped as type 4 */
 #define KVM_REG_RISCV_TIMER		(0x04 << KVM_REG_RISCV_TYPE_SHIFT)
arch/riscv/kvm/vcpu.c +40 -0
index 1f8e8b5f659b..630669513bd7 100644
@@ -507,6 +507,34 @@ static int kvm_riscv_vcpu_general_get_csr(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static inline int kvm_riscv_vcpu_smstateen_set_csr(struct kvm_vcpu *vcpu,
+						   unsigned long reg_num,
+						   unsigned long reg_val)
+{
+	struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr;
+
+	if (reg_num >= sizeof(struct kvm_riscv_smstateen_csr) /
+	    sizeof(unsigned long))
+		return -EINVAL;
+
+	((unsigned long *)csr)[reg_num] = reg_val;
+	return 0;
+}
+
+static int kvm_riscv_vcpu_smstateen_get_csr(struct kvm_vcpu *vcpu,
+					    unsigned long reg_num,
+					    unsigned long *out_val)
+{
+	struct kvm_vcpu_smstateen_csr *csr = &vcpu->arch.smstateen_csr;
+
+	if (reg_num >= sizeof(struct kvm_riscv_smstateen_csr) /
+	    sizeof(unsigned long))
+		return -EINVAL;
+
+	*out_val = ((unsigned long *)csr)[reg_num];
+	return 0;
+}
+
 static inline int kvm_riscv_vcpu_general_set_csr(struct kvm_vcpu *vcpu,
 						 unsigned long reg_num,
 						 unsigned long reg_val)

@@ -552,6 +580,12 @@ static int kvm_riscv_vcpu_get_reg_csr(struct kvm_vcpu *vcpu,
 	case KVM_REG_RISCV_CSR_AIA:
 		rc = kvm_riscv_vcpu_aia_get_csr(vcpu, reg_num, &reg_val);
 		break;
+	case KVM_REG_RISCV_CSR_SMSTATEEN:
+		rc = -EINVAL;
+		if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN))
+			rc = kvm_riscv_vcpu_smstateen_get_csr(vcpu, reg_num,
+							      &reg_val);
+		break;
 	default:
 		rc = -EINVAL;
 		break;

@@ -591,6 +625,12 @@ static int kvm_riscv_vcpu_set_reg_csr(struct kvm_vcpu *vcpu,
 	case KVM_REG_RISCV_CSR_AIA:
 		rc = kvm_riscv_vcpu_aia_set_csr(vcpu, reg_num, reg_val);
 		break;
+	case KVM_REG_RISCV_CSR_SMSTATEEN:
+		rc = -EINVAL;
+		if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SMSTATEEN))
+			rc = kvm_riscv_vcpu_smstateen_set_csr(vcpu, reg_num,
+							      reg_val);
+		break;
 	default:
 		rc = -EINVAL;
 		break;
Signature · 1 line
2.34.1
#9 lore raw

v3 cover Risc-V Kvm Smstateen

In reply to Mayuresh Chitale

Hey Mayuresh,

Quoted 3 lines · Mayuresh Chitale

This series adds support to detect the Smstateen extension for both, the host and the guest vcpu. It also adds senvcfg and sstateen0 to the ONE_REG interface and the vcpu context save/restore.

There's not really an explanation in this series of where Smstateen is needed, or why it is only implemented for KVM. The spec mentions that this also applies to separate user threads, as well as to guests running in a hypervisor. As your first patch will lead to smstateen being set in /proc/cpuinfo, it could reasonably be assumed that the kernel itself supports the extension. Why does only KVM, and not the kernel, require support for smstateen?

Thanks,
Conor.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kvm-riscv/attachments/20230724/412e56ab/attachment.sig>

#10 lore raw

v3 cover Risc-V Kvm Smstateen

Quoted 8 lines · On Mon, Jul 24, 2023 at 10:08?PM Conor Dooley <[email protected]> wrote:

Hey Mayuresh,

Quoted 3 lines · Mayuresh Chitale

This series adds support to detect the Smstateen extension for both, the host and the guest vcpu. It also adds senvcfg and sstateen0 to the ONE_REG interface and the vcpu context save/restore.

Quoted 7 lines

There's not really an explanation in this series of where Smstateen is needed, or why it is only implemented for KVM. The spec mentions that this also applies to separate user threads, as well as to guests running in a hypervisor. As your first patch will lead to smstateen being set in /proc/cpuinfo, it could reasonably be assumed that the kernel itself supports the extension. Why does only KVM, and not the kernel, require support for smstateen?

Here's the motivation behind Smstateen from the spec:
"The implementation of optional RISC-V extensions has the potential to open covert channels between separate user threads, or between separate guest OSes running under a hypervisor. The problem occurs when an extension adds processor state---usually explicit registers, but possibly other forms of state---that the main OS or hypervisor is unaware of (and hence won?t context-switch) but that can be modified/written by one user thread or guest OS and perceived/examined/read by another."

Based on the above, Ssaia extension related CSRs need to be explicitly enabled for HS-mode by M-mode (which OpenSBI already does) and for VS-mode by HS-mode (which this series adds for KVM RISC-V).

Currently, there are no new extensions addings CSRs for user-space so RISC-V kernel does not need to set up sstateenX CSRs for processes or tasks but in the future RISC-V kernel might touch sstateenX CSRs.

Regards,
Anup

#11 lore raw

v3 cover Risc-V Kvm Smstateen

Hey Anup,

Quoted 14 lines · Anup Patel
Quoted 5 lines · On Mon, Jul 24, 2023 at 10:08?PM Conor Dooley <[email protected]> wrote:
Quoted 3 lines · Mayuresh Chitale

This series adds support to detect the Smstateen extension for both, the host and the guest vcpu. It also adds senvcfg and sstateen0 to the ONE_REG interface and the vcpu context save/restore.

Quoted 7 lines

There's not really an explanation in this series of where Smstateen is needed, or why it is only implemented for KVM. The spec mentions that this also applies to separate user threads, as well as to guests running in a hypervisor. As your first patch will lead to smstateen being set in /proc/cpuinfo, it could reasonably be assumed that the kernel itself supports the extension. Why does only KVM, and not the kernel, require support for smstateen?

Quoted 8 lines

Here's the motivation behind Smstateen from the spec:
"The implementation of optional RISC-V extensions has the potential to open covert channels between separate user threads, or between separate guest OSes running under a hypervisor. The problem occurs when an extension adds processor state---usually explicit registers, but possibly other forms of state---that the main OS or hypervisor is unaware of (and hence won?t context-switch) but that can be modified/written by one user thread or guest OS and perceived/examined/read by another."

This much I gathered from my (brief) reading of the spec.

Quoted 3 lines

Based on the above, Ssaia extension related CSRs need to be explicitly enabled for HS-mode by M-mode (which OpenSBI already does) and for VS-mode by HS-mode (which this series adds for KVM RISC-V).

Ah right. Reading back through the patches, in [4/7] I see "Configure hstateen0 register so that the AIA state and envcfg are accessible to the vcpus". I would ask that, at least, [1/7] is updated to provide this motivation & the rationale for why only KVM needs to care. The motivation for the work should appear in the patchset somewhere, and probably in the cover too.

Quoted 3 lines

Currently, there are no new extensions addings CSRs for user-space so RISC-V kernel does not need to set up sstateenX CSRs for processes or tasks but in the future RISC-V kernel might touch sstateenX CSRs.

Right, that is what I figured was going on, ignoring it for now, in the hopes that we remember to deal with it before some userspace visible side channel shows up.

Dumb question maybe, but I find this to be quite -ENOPARSE:

Quoted 5 lines

Bit 0 of these registers is not custom state itself; it is a standard field of a standard CSR, either mstateen0, hstateen0, or sstateen0. The requirements that non-standard extensions must meet to be conforming are not relaxed due solely to changes in the value of this bit. In particular, if software sets this bit but does not execute any custom instructions or access any custom state, the software must continue to execute as specified by all relevant RISC-V standards, or the hardware is not standard-conforming.

Does this mean that bit 0 of the CSRs mentioned in the quote controls all non-standard extensions at the respective privilege level? If so, does that not make the "ignore that we will now report the presence of this extension" approach flimsier, since we have little visibility into what may exist on that front?

Granted, it is not as if delaying this patchset would benefit anyone in that regard, since those attempting to exploit the side channel know that the side channel exists, whether the kernel reports having sstateen or not. This probably just boils down to /proc/cpuinfo being a terrible interface for detecting extension support in the kernel.
I've got some other comments about it that came up on IRC yesterday, so I'll go complain about it elsewhere :)

Thanks,
Conor.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/kvm-riscv/attachments/20230725/61a278c4/attachment.sig>

#12 lore raw

v3 cover Risc-V Kvm Smstateen

Quoted 33 lines · On Tue, Jul 25, 2023 at 3:37?PM Conor Dooley <[email protected]> wrote:

Hey Anup,

Quoted 14 lines · Anup Patel
Quoted 5 lines · On Mon, Jul 24, 2023 at 10:08?PM Conor Dooley <[email protected]> wrote:
Quoted 3 lines · Mayuresh Chitale

This series adds support to detect the Smstateen extension for both, the host and the guest vcpu. It also adds senvcfg and sstateen0 to the ONE_REG interface and the vcpu context save/restore.

Quoted 7 lines

There's not really an explanation in this series of where Smstateen is needed, or why it is only implemented for KVM. The spec mentions that this also applies to separate user threads, as well as to guests running in a hypervisor. As your first patch will lead to smstateen being set in /proc/cpuinfo, it could reasonably be assumed that the kernel itself supports the extension. Why does only KVM, and not the kernel, require support for smstateen?

Quoted 8 lines

Here's the motivation behind Smstateen from the spec:
"The implementation of optional RISC-V extensions has the potential to open covert channels between separate user threads, or between separate guest OSes running under a hypervisor. The problem occurs when an extension adds processor state---usually explicit registers, but possibly other forms of state---that the main OS or hypervisor is unaware of (and hence won?t context-switch) but that can be modified/written by one user thread or guest OS and perceived/examined/read by another."

This much I gathered from my (brief) reading of the spec.

Quoted 3 lines

Based on the above, Ssaia extension related CSRs need to be explicitly enabled for HS-mode by M-mode (which OpenSBI already does) and for VS-mode by HS-mode (which this series adds for KVM RISC-V).

Quoted 6 lines

Ah right. Reading back through the patches, in [4/7] I see "Configure hstateen0 register so that the AIA state and envcfg are accessible to the vcpus". I would ask that, at least, [1/7] is updated to provide this motivation & the rationale for why only KVM needs to care. The motivation for the work should appear in the patchset somewhere, and probably in the cover too.

Yes, more details can be added to the commit description of PATCH1 and cover letter.

Quoted 15 lines
Quoted 4 lines

Currently, there are no new extensions addings CSRs for user-space so RISC-V kernel does not need to set up sstateenX CSRs for processes or tasks but in the future RISC-V kernel might touch sstateenX CSRs.

Right, that is what I figured was going on, ignoring it for now, in the hopes that we remember to deal with it before some userspace visible side channel shows up.

Dumb question maybe, but I find this to be quite -ENOPARSE:

Quoted 5 lines

Bit 0 of these registers is not custom state itself; it is a standard field of a standard CSR, either mstateen0, hstateen0, or sstateen0. The requirements that non-standard extensions must meet to be conforming are not relaxed due solely to changes in the value of this bit. In particular, if software sets this bit but does not execute any custom instructions or access any custom state, the software must continue to execute as specified by all relevant RISC-V standards, or the hardware is not standard-conforming.

Quoted 5 lines

Does this mean that bit 0 of the CSRs mentioned in the quote controls all non-standard extensions at the respective privilege level? If so, does that not make the "ignore that we will now report the presence of this extension" approach flimsier, since we have little visibility into what may exist on that front?

Yes, bit0 of the stateen CSRs controls the access to custom registers added by custom extensions. Of course, not all custom extensions would add custom registers.

Higher privilege levels (such as hypervisors) should not enable bit0 by default. It should be enabled only if higher privilege levels knows what it is and how to context-switch the custom registers.

Quoted 11 lines

Granted, it is not as if delaying this patchset would benefit anyone in that regard, since those attempting to exploit the side channel know that the side channel exists, whether the kernel reports having sstateen or not. This probably just boils down to /proc/cpuinfo being a terrible interface for detecting extension support in the kernel.
I've got some other comments about it that came up on IRC yesterday, so I'll go complain about it elsewhere :)

Thanks,
Conor.

Regards,
Anup