KVM: x86: Plug an intra-guest Spectre v2 hole
Sean Christopherson
maintainercover KVM: x86: Plug an intra-guest Spectre v2 hole
Fix a bug where KVM neglects to emit IBPB when a vCPU is migrated *back* to a previous pCPU, in which case a misbehaving task B in the guest can unduly infuence e.g. the BTB of task A in the guest (running in the same vCPU).
FWIW, I've had the first patch sitting around for several months as a micro-optimization, but was never posted it as I wasn't 100% confident my analysis was correct. But it makes plugging this hole much easier, so here it is... Reviews most definitely welcome.
Sean Christopherson (2):
KVM: x86: Defer IBPBs for vCPU and nested transitions until core run
loop
KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 20 +++++++++++++++++++- arch/x86/kvm/x86.h | 2 +- 3 files changed, 21 insertions(+), 2 deletions(-)
base-commit: e81f7c908e1664233974b9f20beead78cde6343a
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Sean Christopherson
maintainerPATCH 1/2 KVM: x86: Defer IBPBs for vCPU and nested transitions until core run loop
When emitting an Indirect Branch Prediction Barrier to isolate different guest security domains (different vCPUs or L1 vs. L2 in the same vCPU), defer the IBPB until VM-Enter is imminent to avoid redundant and/or unnecessary IBPBs. E.g. if a vCPU is loaded on a CPU without ever doing VM-Enter, then _KVM_ isn't responsible for doing an IBPB as KVM's job is purely to mitigate guests<=>guest attacks; guest=>host attacks are covered by IBRS.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 7 ++++++- arch/x86/kvm/x86.h | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -826,6 +826,7 @@ struct kvm_vcpu_arch {u64 smbase;u64 smi_count;bool at_instruction_boundary; +bool need_ibpb;bool tpr_access_reporting;bool xfd_no_write_intercept;u64 microcode_version;
arch/x86/kvm/x86.c +6 -1
@@ -5187,7 +5187,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)* is handled on the nested VM-Exit path. */ if (static_branch_likely (&switch_vcpu_ibpb)) -indirect_branch_prediction_barrier (); + vcpu->arch.need_ibpb = true;per_cpu (last_vcpu, cpu) = vcpu; }
@@ -11315,6 +11315,11 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_make_request (KVM_REQ_EVENT , vcpu); } +if (unlikely (vcpu->arch.need_ibpb)) { +indirect_branch_prediction_barrier (); + vcpu->arch.need_ibpb = false; + } +fpregs_assert_state_consistent ();if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return ();
arch/x86/kvm/x86.h +1 -1
@@ -169,7 +169,7 @@ static inline void kvm_nested_vmexit_handle_ibrs(struct kvm_vcpu *vcpu)if (guest_cpu_cap_has (vcpu,X86_FEATURE_SPEC_CTRL ) ||guest_cpu_cap_has (vcpu,X86_FEATURE_AMD_IBRS )) -indirect_branch_prediction_barrier (); + vcpu->arch.need_ibpb = true; }/*
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH 1/2] KVM: x86: Defer IBPBs for vCPU and nested transitions until core run loop
Quoted 24 lines · Sean Christopherson
When emitting an Indirect Branch Prediction Barrier to isolate different guest security domains (different vCPUs or L1 vs. L2 in the same vCPU), defer the IBPB until VM-Enter is imminent to avoid redundant and/or unnecessary IBPBs. E.g. if a vCPU is loaded on a CPU without ever doing VM-Enter, then _KVM_ isn't responsible for doing an IBPB as KVM's job is purely to mitigate guests<=>guest attacks; guest=>host attacks are covered by IBRS.
Cc Yosry Ahmed <[email protected]>Cc Jim Mattson <[email protected]>Cc David Kaplan <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 7 ++++++- arch/x86/kvm/x86.h | 2 +- 3 files changed, 8 insertions(+), 2 deletions(-)arch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 829
Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1
-0@@ -826,6 +826,7 @@ struct kvm_vcpu_arch {u64 smbase;u64 smi_count;bool at_instruction_boundary; +bool need_ibpb;
We have IBPB_ON_VMEXIT, so this is a bit confusing, the reader could assume this is an optimization for IBPB_ON_VMEXIT.
Maybe need_ibpb_on_vmenter or need_ibpb_on_run?
arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5190
Quoted 13 lines
arch/x86/include/asm/kvm_host.h +0
-0bool tpr_access_reporting;bool xfd_no_write_intercept;u64 microcode_version;arch/x86/kvm/x86.c +1
-1@@ -5187,7 +5187,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)* is handled on the nested VM-Exit path. */ if (static_branch_likely (&switch_vcpu_ibpb)) -indirect_branch_prediction_barrier (); + vcpu->arch.need_ibpb = true;
This means that if we run vCPU A on a pCPU, then load vCPU B without running it, we won't do an IBPB. At least not until vCPU B (or any vCPU) actually runs on the pCPU.
My question would be, is it possible for training done by vCPU A to lead KVM into leaking some of vCPU B's state after loading it, even without actually running vCPU B?
Basically this scenario (all on the same pCPU):
1. Malicious vCPU A runs and injects branch predictor entries.
2. KVM loads vCPU B to perform some action without actually running vCPU B. The training from (1) leaks some of vCPU B's state into the microarchitectural state.
3. Malicious vCPU A runs again and extracts the leaked data.
Quoted 20 lines
arch/x86/kvm/x86.c +5
-0per_cpu (last_vcpu, cpu) = vcpu; }@@ -11315,6 +11315,11 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_make_request (KVM_REQ_EVENT , vcpu); } +if (unlikely (vcpu->arch.need_ibpb)) { +indirect_branch_prediction_barrier (); + vcpu->arch.need_ibpb = false; + } +fpregs_assert_state_consistent ();if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return ();arch/x86/kvm/x86.h +0
-0
arch/x86/kvm/x86.h static inline void kvm_nested_vmexit_handle_ibrs(struct kvm_vcpu *vcpu) · line 172
Quoted 5 lines
arch/x86/kvm/x86.h +1
-1@@ -169,7 +169,7 @@ static inline void kvm_nested_vmexit_handle_ibrs(struct kvm_vcpu *vcpu)if (guest_cpu_cap_has (vcpu,X86_FEATURE_SPEC_CTRL ) ||guest_cpu_cap_has (vcpu,X86_FEATURE_AMD_IBRS )) -indirect_branch_prediction_barrier (); + vcpu->arch.need_ibpb = true;
I think the same question more-or-less applies here, could an L2 guest lead KVM to leak some of L1's state before L1 is actually run? Although in this case it could be harder for any leaked state to survive KVM running L1 and then going back to L2.
Quoted 6 lines
arch/x86/kvm/x86.h +0
-0}/* Signature · 1 line
2.52.0.457.g6b5491de43-goog
Sean Christopherson
maintainerPATCH 2/2 KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest
In reply to Sean ChristophersonEmit an Indirect Branch Prediction Barrier if a vCPU is migrated to a different pCPU and IBPB support is advertised to the guest, to ensure any IBPBs performed by the guest are effective across pCPUs. Ideally, KVM would only emit IBPB if the guest performed an IBPB since the vCPU last ran on the "new" pCPU, but pCPU migration is a relatively rare/slow path, and so the cost of tracking which pCPUs a vCPUs has run on, let alone intercepting PRED_CMD writes, outweighs the potential benefits of avoiding IBPBs on pCPU migration.
E.g. if a single vCPU is bouncing between pCPUs A and B, and the guest is doing IBPBs on context switches to mitigate cross-task attacks, then the following scenario can occur and needs to be mitigated by KVM:
1. vCPU starts on pCPU A. It runs a userspace task (task #1) which installs various branch predictions into pCPU A's BTB.
2. The vCPU is migrated to pCPU B.
3. The guest switches to userspace task #2 and emits an IBPB, on pCPU B.
4. The vCPU is migrated back to pCPU A. Userspace task (task #2) in the guest now consumes the potentially dangerous branch predictions installed in step 1 from task #1.
--- arch/x86/kvm/x86.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/kvm/x86.c +13 -0
@@ -5201,6 +5201,19 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)kvm_make_request (KVM_REQ_CLOCK_UPDATE , vcpu); } +/* +* If the vCPU is migrated to a different pCPU than the one on which +* the vCPU last ran, and IBPB is advertised to the vCPU, then flush +* indirect branch predictors before the next VM-Enter to ensure the +* vCPU doesn't consume prediction information from a previous run on +* the "new" pCPU. +*/ +if (unlikely (vcpu->arch.last_vmentry_cpu != cpu && + vcpu->arch.last_vmentry_cpu >=0 ) && + (guest_cpu_cap_has (vcpu,X86_FEATURE_SPEC_CTRL ) || +guest_cpu_cap_has (vcpu,X86_FEATURE_AMD_IBPB ))) + vcpu->arch.need_ibpb = true; +if (unlikely (vcpu->cpu != cpu) ||kvm_check_tsc_unstable ()) {s64 tsc_delta = !vcpu->arch.last_host_tsc ?0 :rdtsc () - vcpu->arch.last_host_tsc;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Re: [PATCH 2/2] KVM: x86: Emit IBPB on pCPU migration if IBPB is advertised to guest
Quoted 38 lines · Sean Christopherson
Emit an Indirect Branch Prediction Barrier if a vCPU is migrated to a different pCPU and IBPB support is advertised to the guest, to ensure any IBPBs performed by the guest are effective across pCPUs. Ideally, KVM would only emit IBPB if the guest performed an IBPB since the vCPU last ran on the "new" pCPU, but pCPU migration is a relatively rare/slow path, and so the cost of tracking which pCPUs a vCPUs has run on, let alone intercepting
PRED_CMDwrites, outweighs the potential benefits of avoiding IBPBs on pCPU migration.E.g. if a single vCPU is bouncing between pCPUs A and B, and the guest is doing IBPBs on context switches to mitigate cross-task attacks, then the following scenario can occur and needs to be mitigated by KVM:
1. vCPU starts on pCPU A. It runs a userspace task (task #1) which installs various branch predictions into pCPU A's BTB.
2. The vCPU is migrated to pCPU B.
3. The guest switches to userspace task #2 and emits an IBPB, on pCPU B.
4. The vCPU is migrated back to pCPU A. Userspace task (task #2) in the guest now consumes the potentially dangerous branch predictions installed in step 1 from task #1.Reported-by David Kaplan <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)arch/x86/kvm/x86.c +2
-0@@ -5201,6 +5201,19 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)kvm_make_request (KVM_REQ_CLOCK_UPDATE , vcpu); } +/* +* If the vCPU is migrated to a different pCPU than the one on which
arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5215
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+* the vCPU last ran, and IBPB is advertised to the vCPU, then flush +* indirect branch predictors before the next VM-Enter to ensure the +* vCPU doesn't consume prediction information from a previous run on +* the "new" pCPU. +*/ +if (unlikely (vcpu->arch.last_vmentry_cpu != cpu && + vcpu->arch.last_vmentry_cpu >=0 ) && + (guest_cpu_cap_has (vcpu,X86_FEATURE_SPEC_CTRL ) || +guest_cpu_cap_has (vcpu,X86_FEATURE_AMD_IBPB ))) + vcpu->arch.need_ibpb = true;
Disregarding deferring the IBPB, the logic looks sound. Should this be moved right after the other IBPB condition above though?
Quoted 7 lines
arch/x86/kvm/x86.c +1
-0+if (unlikely (vcpu->cpu != cpu) ||kvm_check_tsc_unstable ()) {s64 tsc_delta = !vcpu->arch.last_host_tsc ?0 :rdtsc () - vcpu->arch.last_host_tsc;Signature · 1 line
2.52.0.457.g6b5491de43-goog