Skip to contentSkip to conversation

KVM: x86: Plug an intra-guest Spectre v2 hole

PATCH In review Sean Christopherson 2 participants 5 messages Jan 28 – Jan 30

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

cover KVM: x86: Plug an intra-guest Spectre v2 hole

Fix a bug where KVM neglects to emit IBPB when a vCPU is migrated *back* to a previous pCPU, in which case a misbehaving task B in the guest can unduly infuence e.g. the BTB of task A in the guest (running in the same vCPU).

FWIW, I've had the first patch sitting around for several months as a micro-optimization, but was never posted it as I wasn't 100% confident my analysis was correct. But it makes plugging this hole much easier, so here it is... Reviews most definitely welcome.

Sean Christopherson (2):
KVM: x86: Defer IBPBs for vCPU and nested transitions until core run
loop
KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest

 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/x86.c              | 20 +++++++++++++++++++-
 arch/x86/kvm/x86.h              |  2 +-
 3 files changed, 21 insertions(+), 2 deletions(-)

base-commit: e81f7c908e1664233974b9f20beead78cde6343a

Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#1 lore raw

PATCH 1/2 KVM: x86: Defer IBPBs for vCPU and nested transitions until core run loop

When emitting an Indirect Branch Prediction Barrier to isolate different guest security domains (different vCPUs or L1 vs. L2 in the same vCPU), defer the IBPB until VM-Enter is imminent to avoid redundant and/or unnecessary IBPBs. E.g. if a vCPU is loaded on a CPU without ever doing VM-Enter, then _KVM_ isn't responsible for doing an IBPB as KVM's job is purely to mitigate guests<=>guest attacks; guest=>host attacks are covered by IBRS.

---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/x86.c              | 7 ++++++-
 arch/x86/kvm/x86.h              | 2 +-
 3 files changed, 8 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index e441f270f354..76bbc80a2d1d 100644
@@ -826,6 +826,7 @@ struct kvm_vcpu_arch {
 	u64 smbase;
 	u64 smi_count;
 	bool at_instruction_boundary;
+	bool need_ibpb;
 	bool tpr_access_reporting;
 	bool xfd_no_write_intercept;
 	u64 microcode_version;
arch/x86/kvm/x86.c +6 -1
index 8acfdfc583a1..e5ae655702b4 100644
@@ -5187,7 +5187,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		 * is handled on the nested VM-Exit path.
 		 */
 		if (static_branch_likely(&switch_vcpu_ibpb))
-			indirect_branch_prediction_barrier();
+			vcpu->arch.need_ibpb = true;
 		per_cpu(last_vcpu, cpu) = vcpu;
 	}
 
@@ -11315,6 +11315,11 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 		kvm_make_request(KVM_REQ_EVENT, vcpu);
 	}
 
+	if (unlikely(vcpu->arch.need_ibpb)) {
+		indirect_branch_prediction_barrier();
+		vcpu->arch.need_ibpb = false;
+	}
+
 	fpregs_assert_state_consistent();
 	if (test_thread_flag(TIF_NEED_FPU_LOAD))
 		switch_fpu_return();
arch/x86/kvm/x86.h +1 -1
index 70e81f008030..6708142d051d 100644
@@ -169,7 +169,7 @@ static inline void kvm_nested_vmexit_handle_ibrs(struct kvm_vcpu *vcpu)
 
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) ||
 	    guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBRS))
-		indirect_branch_prediction_barrier();
+		vcpu->arch.need_ibpb = true;
 }
 
 /*
Signature · 1 line
2.52.0.457.g6b5491de43-goog
#2 lore raw

Re: [PATCH 1/2] KVM: x86: Defer IBPBs for vCPU and nested transitions until core run loop

Quoted 24 lines · Sean Christopherson

When emitting an Indirect Branch Prediction Barrier to isolate different guest security domains (different vCPUs or L1 vs. L2 in the same vCPU), defer the IBPB until VM-Enter is imminent to avoid redundant and/or unnecessary IBPBs. E.g. if a vCPU is loaded on a CPU without ever doing VM-Enter, then _KVM_ isn't responsible for doing an IBPB as KVM's job is purely to mitigate guests<=>guest attacks; guest=>host attacks are covered by IBRS.

---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/x86.c              | 7 ++++++-
 arch/x86/kvm/x86.h              | 2 +-
 3 files changed, 8 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index e441f270f354..76bbc80a2d1d 100644

arch/x86/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 829

Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1 -0
@@ -826,6 +826,7 @@ struct kvm_vcpu_arch {
 	u64 smbase;
 	u64 smi_count;
 	bool at_instruction_boundary;
+	bool need_ibpb;

We have IBPB_ON_VMEXIT, so this is a bit confusing, the reader could assume this is an optimization for IBPB_ON_VMEXIT.

Maybe need_ibpb_on_vmenter or need_ibpb_on_run?

arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5190

Quoted 13 lines
arch/x86/include/asm/kvm_host.h +0 -0
 	bool tpr_access_reporting;
 	bool xfd_no_write_intercept;
 	u64 microcode_version;
arch/x86/kvm/x86.c +1 -1
index 8acfdfc583a1..e5ae655702b4 100644
@@ -5187,7 +5187,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		 * is handled on the nested VM-Exit path.
 		 */
 		if (static_branch_likely(&switch_vcpu_ibpb))
-			indirect_branch_prediction_barrier();
+			vcpu->arch.need_ibpb = true;

This means that if we run vCPU A on a pCPU, then load vCPU B without running it, we won't do an IBPB. At least not until vCPU B (or any vCPU) actually runs on the pCPU.

My question would be, is it possible for training done by vCPU A to lead KVM into leaking some of vCPU B's state after loading it, even without actually running vCPU B?

Basically this scenario (all on the same pCPU):

1. Malicious vCPU A runs and injects branch predictor entries.

2. KVM loads vCPU B to perform some action without actually running vCPU B. The training from (1) leaks some of vCPU B's state into the microarchitectural state.

3. Malicious vCPU A runs again and extracts the leaked data.

Quoted 20 lines
arch/x86/kvm/x86.c +5 -0
 		per_cpu(last_vcpu, cpu) = vcpu;
 	}
 
@@ -11315,6 +11315,11 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 		kvm_make_request(KVM_REQ_EVENT, vcpu);
 	}
 
+	if (unlikely(vcpu->arch.need_ibpb)) {
+		indirect_branch_prediction_barrier();
+		vcpu->arch.need_ibpb = false;
+	}
+
 	fpregs_assert_state_consistent();
 	if (test_thread_flag(TIF_NEED_FPU_LOAD))
 		switch_fpu_return();
arch/x86/kvm/x86.h +0 -0
index 70e81f008030..6708142d051d 100644

arch/x86/kvm/x86.h static inline void kvm_nested_vmexit_handle_ibrs(struct kvm_vcpu *vcpu) · line 172

Quoted 5 lines
arch/x86/kvm/x86.h +1 -1
@@ -169,7 +169,7 @@ static inline void kvm_nested_vmexit_handle_ibrs(struct kvm_vcpu *vcpu)
 
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) ||
 	    guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBRS))
-		indirect_branch_prediction_barrier();
+		vcpu->arch.need_ibpb = true;

I think the same question more-or-less applies here, could an L2 guest lead KVM to leak some of L1's state before L1 is actually run? Although in this case it could be harder for any leaked state to survive KVM running L1 and then going back to L2.

Quoted 6 lines
arch/x86/kvm/x86.h +0 -0
 }
 
 /*
Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#3 lore raw

PATCH 2/2 KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest

In reply to Sean Christopherson

Emit an Indirect Branch Prediction Barrier if a vCPU is migrated to a different pCPU and IBPB support is advertised to the guest, to ensure any IBPBs performed by the guest are effective across pCPUs. Ideally, KVM would only emit IBPB if the guest performed an IBPB since the vCPU last ran on the "new" pCPU, but pCPU migration is a relatively rare/slow path, and so the cost of tracking which pCPUs a vCPUs has run on, let alone intercepting PRED_CMD writes, outweighs the potential benefits of avoiding IBPBs on pCPU migration.

E.g. if a single vCPU is bouncing between pCPUs A and B, and the guest is doing IBPBs on context switches to mitigate cross-task attacks, then the following scenario can occur and needs to be mitigated by KVM:

1. vCPU starts on pCPU A. It runs a userspace task (task #1) which installs various branch predictions into pCPU A's BTB.
2. The vCPU is migrated to pCPU B.
3. The guest switches to userspace task #2 and emits an IBPB, on pCPU B.
4. The vCPU is migrated back to pCPU A. Userspace task (task #2) in the guest now consumes the potentially dangerous branch predictions installed in step 1 from task #1.

---
 arch/x86/kvm/x86.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kvm/x86.c +13 -0
index e5ae655702b4..9d1641c2d83c 100644
@@ -5201,6 +5201,19 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
 	}
 
+	/*
+	 * If the vCPU is migrated to a different pCPU than the one on which
+	 * the vCPU last ran, and IBPB is advertised to the vCPU, then flush
+	 * indirect branch predictors before the next VM-Enter to ensure the
+	 * vCPU doesn't consume prediction information from a previous run on
+	 * the "new" pCPU.
+	 */
+	if (unlikely(vcpu->arch.last_vmentry_cpu != cpu &&
+		     vcpu->arch.last_vmentry_cpu >= 0) &&
+	    (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) ||
+	     guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBPB)))
+		vcpu->arch.need_ibpb = true;
+
 	if (unlikely(vcpu->cpu != cpu) || kvm_check_tsc_unstable()) {
 		s64 tsc_delta = !vcpu->arch.last_host_tsc ? 0 :
 				rdtsc() - vcpu->arch.last_host_tsc;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
#4 lore raw

Re: [PATCH 2/2] KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest

Quoted 38 lines · Sean Christopherson

Emit an Indirect Branch Prediction Barrier if a vCPU is migrated to a different pCPU and IBPB support is advertised to the guest, to ensure any IBPBs performed by the guest are effective across pCPUs. Ideally, KVM would only emit IBPB if the guest performed an IBPB since the vCPU last ran on the "new" pCPU, but pCPU migration is a relatively rare/slow path, and so the cost of tracking which pCPUs a vCPUs has run on, let alone intercepting PRED_CMD writes, outweighs the potential benefits of avoiding IBPBs on pCPU migration.

E.g. if a single vCPU is bouncing between pCPUs A and B, and the guest is doing IBPBs on context switches to mitigate cross-task attacks, then the following scenario can occur and needs to be mitigated by KVM:

1. vCPU starts on pCPU A. It runs a userspace task (task #1) which installs various branch predictions into pCPU A's BTB.
2. The vCPU is migrated to pCPU B.
3. The guest switches to userspace task #2 and emits an IBPB, on pCPU B.
4. The vCPU is migrated back to pCPU A. Userspace task (task #2) in the guest now consumes the potentially dangerous branch predictions installed in step 1 from task #1.

---
 arch/x86/kvm/x86.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kvm/x86.c +2 -0
index e5ae655702b4..9d1641c2d83c 100644
@@ -5201,6 +5201,19 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
 		kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
 	}
 
+	/*
+	 * If the vCPU is migrated to a different pCPU than the one on which

arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5215

Quoted 10 lines
arch/x86/kvm/x86.c +10 -0
+	 * the vCPU last ran, and IBPB is advertised to the vCPU, then flush
+	 * indirect branch predictors before the next VM-Enter to ensure the
+	 * vCPU doesn't consume prediction information from a previous run on
+	 * the "new" pCPU.
+	 */
+	if (unlikely(vcpu->arch.last_vmentry_cpu != cpu &&
+		     vcpu->arch.last_vmentry_cpu >= 0) &&
+	    (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) ||
+	     guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBPB)))
+		vcpu->arch.need_ibpb = true;

Disregarding deferring the IBPB, the logic looks sound. Should this be moved right after the other IBPB condition above though?

Quoted 7 lines
arch/x86/kvm/x86.c +1 -0
+
 	if (unlikely(vcpu->cpu != cpu) || kvm_check_tsc_unstable()) {
 		s64 tsc_delta = !vcpu->arch.last_host_tsc ? 0 :
 				rdtsc() - vcpu->arch.last_host_tsc;
Signature · 1 line
2.52.0.457.g6b5491de43-goog