Skip to contentSkip to conversation

KVM: arm64: Sanitise host vCPU fields copied in flush_hyp_vcpu()

PATCH Superseded Hyunwoo Kim 3 participants 7 messages Jun 4 – Jun 6

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover KVM: arm64: Sanitise host vCPU fields copied in flush_hyp_vcpu()

flush_hyp_vcpu() copies the host vCPU context and vGIC state into the hyp's private vCPU on every run. This series sanitises two fields that it currently copies verbatim (host -> EL2): __hyp_running_vcpu is cleared in the guest context, and used_lrs is bounded by the number of implemented list registers.

v1 -> v2: split into two patches, one per field, per review.

Hyunwoo Kim (2):
KVM: arm64: Clear __hyp_running_vcpu when flushing the pKVM hyp vCPU
KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/2 KVM: arm64: Clear __hyp_running_vcpu when flushing the pKVM hyp vCPU

flush_hyp_vcpu() copies the host vCPU context into the hyp's private vCPU on every run. ctxt_to_vcpu() expects a guest context to have a NULL __hyp_running_vcpu, which is only ever set on the host context, so that it resolves the vCPU via container_of(). While this is generally the case, flush_hyp_vcpu() copies the context verbatim and does not enforce this, so a value provided by the host is dereferenced at EL2 (host -> EL2).

Fix by clearing __hyp_running_vcpu after the copy.

Fixes be66e67f1750 ("KVM: arm64: Use the pKVM hyp vCPU structure in handle___kvm_vcpu_run()")
---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +3 -0
index 06db299c37a89..02c5d6e5abcbf 100644
@@ -128,6 +128,9 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.ctxt	= host_vcpu->arch.ctxt;
 
+	/* __hyp_running_vcpu must be NULL in a guest context. */
+	hyp_vcpu->vcpu.arch.ctxt.__hyp_running_vcpu = NULL;
+
 	hyp_vcpu->vcpu.arch.mdcr_el2	= host_vcpu->arch.mdcr_el2;
 	hyp_vcpu->vcpu.arch.hcr_el2 &= ~(HCR_TWI | HCR_TWE);
 	hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) &
Signature · 1 line
2.43.0
#2 lore raw

PATCH v2 2/2 KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

In reply to Hyunwoo Kim

flush_hyp_vcpu() copies the host vGIC state into the hyp's private vCPU on every run. The vGIC list register save and restore use used_lrs as their loop bound and expect it to stay within the number of implemented list registers. While this is generally the case, flush_hyp_vcpu() copies vgic_v3 verbatim and does not enforce this, so a value provided by the host is used at EL2 to index vgic_lr[] and access ICH_LR<n>_EL2 (host -> EL2).

Fix by clamping used_lrs to the number of implemented list registers after the copy, as the trusted path already does in vgic_flush_lr_state().

Fixes be66e67f1750 ("KVM: arm64: Use the pKVM hyp vCPU structure in handle___kvm_vcpu_run()")
---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +8 -0
index 02c5d6e5abcbf..cd807fdb11ba8 100644
@@ -7,6 +7,7 @@
 #include <hyp/adjust_pc.h>
 #include <hyp/switch.h>
 
+#include <asm/arch_gicv3.h>
 #include <asm/pgtable-types.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>
@@ -142,6 +143,13 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
 
+	/* Bound used_lrs by the number of implemented list registers. */
+	if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))
+		hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+			min_t(unsigned int,
+			      hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+			      (read_gicreg(ICH_VTR_EL2) & 0xf) + 1);
+
 	hyp_vcpu->vcpu.arch.pid = host_vcpu->arch.pid;
 }
 
Signature · 1 line
2.43.0
maintainer
#3 lore raw

Re: [PATCH v2 2/2] KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

Quoted 32 lines · Hyunwoo Kim

flush_hyp_vcpu() copies the host vGIC state into the hyp's private vCPU on every run. The vGIC list register save and restore use used_lrs as their loop bound and expect it to stay within the number of implemented list registers. While this is generally the case, flush_hyp_vcpu() copies vgic_v3 verbatim and does not enforce this, so a value provided by the host is used at EL2 to index vgic_lr[] and access ICH_LR<n>_EL2 (host -> EL2).

Fix by clamping used_lrs to the number of implemented list registers after the copy, as the trusted path already does in vgic_flush_lr_state().

Fixes be66e67f1750 ("KVM: arm64: Use the pKVM hyp vCPU structure in handle___kvm_vcpu_run()")
---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +1 -0
index 02c5d6e5abcbf..cd807fdb11ba8 100644
@@ -7,6 +7,7 @@
 #include <hyp/adjust_pc.h>
 #include <hyp/switch.h>
 
+#include <asm/arch_gicv3.h>
 #include <asm/pgtable-types.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>

arch/arm64/kvm/hyp/nvhe/hyp-main.c static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 147

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +2 -0
@@ -142,6 +143,13 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
 
+	/* Bound used_lrs by the number of implemented list registers. */
+	if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))

There is no pKVM support without a GICv3 CPU interface, and absolutely everything already assumes it. Why do we need this extra check?

arch/arm64/kvm/hyp/nvhe/hyp-main.c static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 152

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +5 -0
+		hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+			min_t(unsigned int,
+			      hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+			      (read_gicreg(ICH_VTR_EL2) & 0xf) + 1);
+

Reading ICH_VTR_EL2 on each entry is going to cause some really heavy trapping under NV, and we should avoid this.

kvm_vgic_global_state.nr_lr contains this information, and it should only be a matter of replicating it (or compute it once) at init time.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#4 lore raw

Re: [PATCH v2 2/2] KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

Quoted 52 lines · Marc Zyngier
Quoted 32 lines · Hyunwoo Kim

flush_hyp_vcpu() copies the host vGIC state into the hyp's private vCPU on every run. The vGIC list register save and restore use used_lrs as their loop bound and expect it to stay within the number of implemented list registers. While this is generally the case, flush_hyp_vcpu() copies vgic_v3 verbatim and does not enforce this, so a value provided by the host is used at EL2 to index vgic_lr[] and access ICH_LR<n>_EL2 (host -> EL2).

Fix by clamping used_lrs to the number of implemented list registers after the copy, as the trusted path already does in vgic_flush_lr_state().

Fixes be66e67f1750 ("KVM: arm64: Use the pKVM hyp vCPU structure in handle___kvm_vcpu_run()")
---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +1 -0
index 02c5d6e5abcbf..cd807fdb11ba8 100644
@@ -7,6 +7,7 @@
 #include <hyp/adjust_pc.h>
 #include <hyp/switch.h>
 
+#include <asm/arch_gicv3.h>
 #include <asm/pgtable-types.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>

arch/arm64/kvm/hyp/nvhe/hyp-main.c static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 147

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +2 -0
@@ -142,6 +143,13 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
 
+	/* Bound used_lrs by the number of implemented list registers. */
+	if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))

There is no pKVM support without a GICv3 CPU interface, and absolutely everything already assumes it. Why do we need this extra check?

arch/arm64/kvm/hyp/nvhe/hyp-main.c static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 152

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +5 -0
+		hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+			min_t(unsigned int,
+			      hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+			      (read_gicreg(ICH_VTR_EL2) & 0xf) + 1);
+

Reading ICH_VTR_EL2 on each entry is going to cause some really heavy trapping under NV, and we should avoid this.

Quoted 2 lines

kvm_vgic_global_state.nr_lr contains this information, and it should only be a matter of replicating it (or compute it once) at init time.

Does this approach look reasonable to you?

---

arch/arm64/include/asm/kvm_hyp.h +1 -0
index 8d06b62e7188..25199769a1d6 100644
@@ -157,5 +157,6 @@ extern unsigned long kvm_nvhe_sym(__icache_flags);
 extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits);
 extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl);
 extern unsigned long kvm_nvhe_sym(hyp_nr_cpus);
+extern unsigned int kvm_nvhe_sym(hyp_vgic_nr_lr);
 
 #endif /* __ARM64_KVM_HYP_H__ */
arch/arm64/kvm/arm.c +1 -0
index 9453321ef8c6..891fe2c7b854 100644
@@ -2426,6 +2426,7 @@ static int __init init_subsystems(void)
 	switch (err) {
 	case 0:
 		vgic_present = true;
+		kvm_nvhe_sym(hyp_vgic_nr_lr) = kvm_vgic_global_state.nr_lr;
 		break;
 	case -ENODEV:
 	case -ENXIO:
arch/arm64/kvm/hyp/nvhe/hyp-main.c +9 -0
index 06db299c37a8..8bb9362bc284 100644
@@ -128,6 +128,9 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.ctxt	= host_vcpu->arch.ctxt;
 
+	/* A guest context must keep a NULL __hyp_running_vcpu. */
+	hyp_vcpu->vcpu.arch.ctxt.__hyp_running_vcpu = NULL;
+
 	hyp_vcpu->vcpu.arch.mdcr_el2	= host_vcpu->arch.mdcr_el2;
 	hyp_vcpu->vcpu.arch.hcr_el2 &= ~(HCR_TWI | HCR_TWE);
 	hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) &
@@ -139,6 +142,12 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
 
+	/* Bound used_lrs by the number of implemented list registers. */
+	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+		min_t(unsigned int,
+		      hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+		      hyp_vgic_nr_lr);
+
 	hyp_vcpu->vcpu.arch.pid = host_vcpu->arch.pid;
 }
 
arch/arm64/kvm/hyp/nvhe/setup.c +1 -0
index d461981616d9..ebc6b4afc336 100644
@@ -20,6 +20,7 @@
 #include <nvhe/trap_handler.h>
 
 unsigned long hyp_nr_cpus;
+unsigned int hyp_vgic_nr_lr;
 
 #define hyp_percpu_size ((unsigned long)__per_cpu_end - \
 			 (unsigned long)__per_cpu_start)
Quoted 7 lines

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.

Best regards,
Hyunwoo Kim

maintainer
#5 lore raw

Re: [PATCH v2 2/2] KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

Quoted 77 lines · Hyunwoo Kim
Quoted 52 lines · Marc Zyngier
Quoted 32 lines · Hyunwoo Kim

flush_hyp_vcpu() copies the host vGIC state into the hyp's private vCPU on every run. The vGIC list register save and restore use used_lrs as their loop bound and expect it to stay within the number of implemented list registers. While this is generally the case, flush_hyp_vcpu() copies vgic_v3 verbatim and does not enforce this, so a value provided by the host is used at EL2 to index vgic_lr[] and access ICH_LR<n>_EL2 (host -> EL2).

Fix by clamping used_lrs to the number of implemented list registers after the copy, as the trusted path already does in vgic_flush_lr_state().

Fixes be66e67f1750 ("KVM: arm64: Use the pKVM hyp vCPU structure in handle___kvm_vcpu_run()")
---
 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +1 -0
index 02c5d6e5abcbf..cd807fdb11ba8 100644
@@ -7,6 +7,7 @@
 #include <hyp/adjust_pc.h>
 #include <hyp/switch.h>
 
+#include <asm/arch_gicv3.h>
 #include <asm/pgtable-types.h>
 #include <asm/kvm_asm.h>
 #include <asm/kvm_emulate.h>

arch/arm64/kvm/hyp/nvhe/hyp-main.c static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 147

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +2 -0
@@ -142,6 +143,13 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
 
+	/* Bound used_lrs by the number of implemented list registers. */
+	if (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif))

There is no pKVM support without a GICv3 CPU interface, and absolutely everything already assumes it. Why do we need this extra check?

arch/arm64/kvm/hyp/nvhe/hyp-main.c static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) · line 150

Quoted 5 lines
arch/arm64/kvm/hyp/nvhe/hyp-main.c +5 -0
+		hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+			min_t(unsigned int,
+			      hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+			      (read_gicreg(ICH_VTR_EL2) & 0xf) + 1);
+

Reading ICH_VTR_EL2 on each entry is going to cause some really heavy trapping under NV, and we should avoid this.

Quoted 2 lines

kvm_vgic_global_state.nr_lr contains this information, and it should only be a matter of replicating it (or compute it once) at init time.

Does this approach look reasonable to you?

---

arch/arm64/include/asm/kvm_hyp.h +1 -0
index 8d06b62e7188..25199769a1d6 100644
@@ -157,5 +157,6 @@ extern unsigned long kvm_nvhe_sym(__icache_flags);
 extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits);
 extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl);
 extern unsigned long kvm_nvhe_sym(hyp_nr_cpus);
+extern unsigned int kvm_nvhe_sym(hyp_vgic_nr_lr);
 
 #endif /* __ARM64_KVM_HYP_H__ */
arch/arm64/kvm/arm.c +0 -0
index 9453321ef8c6..891fe2c7b854 100644

arch/arm64/kvm/arm.c static int __init init_subsystems(void) · line 2429

Quoted 4 lines
arch/arm64/kvm/arm.c +1 -0
@@ -2426,6 +2426,7 @@ static int __init init_subsystems(void)
 	switch (err) {
 	case 0:
 		vgic_present = true;
+		kvm_nvhe_sym(hyp_vgic_nr_lr) = kvm_vgic_global_state.nr_lr;

You probably want to guard this on kvm_vgic_global_state.gicv3_cpuif to be sure this stays to 0 when running with a GICv2 implementation.

And maybe rename it to hyp_gicv3_nr_lr to avoid any confusion.

Quoted 35 lines
arch/arm64/kvm/arm.c +0 -0
 		break;
 	case -ENODEV:
 	case -ENXIO:
arch/arm64/kvm/hyp/nvhe/hyp-main.c +9 -0
index 06db299c37a8..8bb9362bc284 100644
@@ -128,6 +128,9 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.ctxt	= host_vcpu->arch.ctxt;
 
+	/* A guest context must keep a NULL __hyp_running_vcpu. */
+	hyp_vcpu->vcpu.arch.ctxt.__hyp_running_vcpu = NULL;
+
 	hyp_vcpu->vcpu.arch.mdcr_el2	= host_vcpu->arch.mdcr_el2;
 	hyp_vcpu->vcpu.arch.hcr_el2 &= ~(HCR_TWI | HCR_TWE);
 	hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) &
@@ -139,6 +142,12 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
 
 	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3 = host_vcpu->arch.vgic_cpu.vgic_v3;
 
+	/* Bound used_lrs by the number of implemented list registers. */
+	hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs =
+		min_t(unsigned int,
+		      hyp_vcpu->vcpu.arch.vgic_cpu.vgic_v3.used_lrs,
+		      hyp_vgic_nr_lr);
+
 	hyp_vcpu->vcpu.arch.pid = host_vcpu->arch.pid;
 }
 
arch/arm64/kvm/hyp/nvhe/setup.c +0 -0
index d461981616d9..ebc6b4afc336 100644

arch/arm64/kvm/hyp/nvhe/setup.c line 23

Quoted 4 lines
arch/arm64/kvm/hyp/nvhe/setup.c +1 -0
@@ -20,6 +20,7 @@
 #include <nvhe/trap_handler.h>
 
 unsigned long hyp_nr_cpus;
+unsigned int hyp_vgic_nr_lr;

You could probably move this close to the place where it is used instead of an unrelated file.

Other than that, and with the vcpu-related hunk in its own patch, this looks reasonable.

Thanks,

	M.
Signature · 1 line
Jazz isn't dead. It just smells funny.
Reviewed-by Tested-by on the v2 series
#6 lore raw
In reply to Hyunwoo Kim
Quoted 8 lines · Hyunwoo Kim

flush_hyp_vcpu() copies the host vCPU context and vGIC state into the hyp's private vCPU on every run. This series sanitises two fields that it currently copies verbatim (host -> EL2): __hyp_running_vcpu is cleared in the guest context, and used_lrs is bounded by the number of implemented list registers.

v1 -> v2: split into two patches, one per field, per review.

For the series:

Cheers,
/fuad

Quoted 11 lines

Hyunwoo Kim (2):
KVM: arm64: Clear __hyp_running_vcpu when flushing the pKVM hyp vCPU
KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU

 arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

--
2.43.0