Skip to contentSkip to conversation

KVM: x86: hyper-v: Bound the bank index in hv_is_vp_in_sparse_set()

PATCH Applied Hyunwoo Kim 3 participants 4 messages Jun 6 – Jun 9

Subsystems (5)
Stable branch x86 KVM KVM x86 +1 more subsystems
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: x86: hyper-v: Bound the bank index in hv_is_vp_in_sparse_set()

hv_is_vp_in_sparse_set() uses valid_bit_nr, i.e. vp_id divided by HV_VCPUS_PER_SPARSE_BANK, as the test_bit() index into valid_bank_mask. valid_bank_mask is a single u64 and a sparse vCPU set holds at most HV_MAX_SPARSE_VCPU_BANKS banks, so valid_bit_nr must be less than HV_MAX_SPARSE_VCPU_BANKS.

The caller in kvm_hv_send_ipi_to_many() passes kvm_hv_get_vpindex(), which is below KVM_MAX_VCPUS and therefore always within that bound. The L2 direct flush branch in kvm_hv_flush_tlb(), however, passes hv_v->nested.vp_id, copied verbatim from the enlightened VMCS without any bounds check, so valid_bit_nr can reach HV_MAX_SPARSE_VCPU_BANKS or more and test_bit() then reads beyond valid_bank_mask.

Return false before the test_bit() when valid_bit_nr is not below HV_MAX_SPARSE_VCPU_BANKS, since such a VP cannot be present in the set.

Fixes c58a318f6090 ("KVM: x86: hyper-v: L2 TLB flush")
---
 arch/x86/kvm/hyperv.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/hyperv.c +4 -0
index 4438ecac9a89..d8782cb7ba02 100644
@@ -1839,6 +1839,10 @@ static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba
 	int valid_bit_nr = vp_id / HV_VCPUS_PER_SPARSE_BANK;
 	unsigned long sbank;
 
+	/* A bank index beyond the mask can't be set, the VP isn't in the set. */
+	if (valid_bit_nr >= HV_MAX_SPARSE_VCPU_BANKS)
+		return false;
+
 	if (!test_bit(valid_bit_nr, (unsigned long *)&valid_bank_mask))
 		return false;
 
Signature · 1 line
2.43.0
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 30 lines · Hyunwoo Kim

hv_is_vp_in_sparse_set() uses valid_bit_nr, i.e. vp_id divided by HV_VCPUS_PER_SPARSE_BANK, as the test_bit() index into valid_bank_mask. valid_bank_mask is a single u64 and a sparse vCPU set holds at most HV_MAX_SPARSE_VCPU_BANKS banks, so valid_bit_nr must be less than HV_MAX_SPARSE_VCPU_BANKS.

The caller in kvm_hv_send_ipi_to_many() passes kvm_hv_get_vpindex(), which is below KVM_MAX_VCPUS and therefore always within that bound. The L2 direct flush branch in kvm_hv_flush_tlb(), however, passes hv_v->nested.vp_id, copied verbatim from the enlightened VMCS without any bounds check, so valid_bit_nr can reach HV_MAX_SPARSE_VCPU_BANKS or more and test_bit() then reads beyond valid_bank_mask.

Return false before the test_bit() when valid_bit_nr is not below HV_MAX_SPARSE_VCPU_BANKS, since such a VP cannot be present in the set.

Fixes c58a318f6090 ("KVM: x86: hyper-v: L2 TLB flush")
---
 arch/x86/kvm/hyperv.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/hyperv.c +0 -0
index 4438ecac9a89..d8782cb7ba02 100644

arch/x86/kvm/hyperv.c static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba · line 1847

Quoted 9 lines
arch/x86/kvm/hyperv.c +4 -0
@@ -1839,6 +1839,10 @@ static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba
 	int valid_bit_nr = vp_id / HV_VCPUS_PER_SPARSE_BANK;
 	unsigned long sbank;
 
+	/* A bank index beyond the mask can't be set, the VP isn't in the set. */
+	if (valid_bit_nr >= HV_MAX_SPARSE_VCPU_BANKS)
+		return false;
+
 	if (!test_bit(valid_bit_nr, (unsigned long *)&valid_bank_mask))
 		return false;

I think the concern is valid, so

what I'm not sure about if we should also deliberately crash the VM which does such a hypercall. This way it would be easier to find buggy L1s but given that they are most likely Windows, we need to do some tests to see if this is not actually happening today (e.g. Hyper-V usign VP_ID or '-1' for something). Let's have this as a future TODO item.

Signature · 1 line
Vitaly
#2 lore raw
Quoted 35 lines · Vitaly Kuznetsov
Quoted 33 lines · Hyunwoo Kim

hv_is_vp_in_sparse_set() uses valid_bit_nr, i.e. vp_id divided by HV_VCPUS_PER_SPARSE_BANK, as the test_bit() index into valid_bank_mask. valid_bank_mask is a single u64 and a sparse vCPU set holds at most HV_MAX_SPARSE_VCPU_BANKS banks, so valid_bit_nr must be less than HV_MAX_SPARSE_VCPU_BANKS.

The caller in kvm_hv_send_ipi_to_many() passes kvm_hv_get_vpindex(), which is below KVM_MAX_VCPUS and therefore always within that bound. The L2 direct flush branch in kvm_hv_flush_tlb(), however, passes hv_v->nested.vp_id, copied verbatim from the enlightened VMCS without any bounds check, so valid_bit_nr can reach HV_MAX_SPARSE_VCPU_BANKS or more and test_bit() then reads beyond valid_bank_mask.

Return false before the test_bit() when valid_bit_nr is not below HV_MAX_SPARSE_VCPU_BANKS, since such a VP cannot be present in the set.

Fixes c58a318f6090 ("KVM: x86: hyper-v: L2 TLB flush")
---
 arch/x86/kvm/hyperv.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/hyperv.c +0 -0
index 4438ecac9a89..d8782cb7ba02 100644
@@ -1839,6 +1839,10 @@ static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba
 	int valid_bit_nr = vp_id / HV_VCPUS_PER_SPARSE_BANK;
 	unsigned long sbank;
 
Quoted 8 lines

arch/x86/kvm/hyperv.c static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 sparse_ba · line 1847

Quoted 6 lines
arch/x86/kvm/hyperv.c +4 -0
+	/* A bank index beyond the mask can't be set, the VP isn't in the set. */
+	if (valid_bit_nr >= HV_MAX_SPARSE_VCPU_BANKS)
+		return false;
+
 	if (!test_bit(valid_bit_nr, (unsigned long *)&valid_bank_mask))
 		return false;

I think the concern is valid, so

Yeah, easy to trigger with KASAN and:

testing/selftests/kvm/x86/hyperv_evmcs.c +5 -1
index c7fa114aee20..0cf5f891a20d 100644
@@ -59,6 +59,10 @@ void l2_guest_code(void)
        vmcall();
        rdmsr_from_l2(MSR_GS_BASE); /* intercepted */
 
+       asm volatile ("movq %0, %%xmm0" :: "r"(-1ull));
+       hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0,
+                        HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES);
+
        /* L2 TLB flush tests */
        hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0,
                         HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS);
@@ -117,7 +121,7 @@ void guest_code(struct vmx_pages *vmx_pages, struct hyperv_test_pages *hv_pages,
        current_evmcs->partition_assist_page = hv_pages->partition_assist_gpa;
        current_evmcs->hv_enlightenments_control.nested_flush_hypercall = 1;
        current_evmcs->hv_vm_id = 1;
-       current_evmcs->hv_vp_id = 1;
+       current_evmcs->hv_vp_id = -1;
        current_vp_assist->nested_control.features.directhypercall = 1;
        *(u32 *)(hv_pages->partition_assist) = 0;
 
Quoted 7 lines

what I'm not sure about if we should also deliberately crash the VM which does such a hypercall. This way it would be easier to find buggy L1s but given that they are most likely Windows, we need to do some tests to see if this is not actually happening today (e.g. Hyper-V usign VP_ID or '-1' for something). Let's have this as a future TODO item.

+1

maintainer Applied
#3 lore raw
In reply to Hyunwoo Kim
Quoted 14 lines · Hyunwoo Kim

hv_is_vp_in_sparse_set() uses valid_bit_nr, i.e. vp_id divided by HV_VCPUS_PER_SPARSE_BANK, as the test_bit() index into valid_bank_mask. valid_bank_mask is a single u64 and a sparse vCPU set holds at most HV_MAX_SPARSE_VCPU_BANKS banks, so valid_bit_nr must be less than HV_MAX_SPARSE_VCPU_BANKS.

The caller in kvm_hv_send_ipi_to_many() passes kvm_hv_get_vpindex(), which is below KVM_MAX_VCPUS and therefore always within that bound. The L2 direct flush branch in kvm_hv_flush_tlb(), however, passes hv_v->nested.vp_id, copied verbatim from the enlightened VMCS without any bounds check, so valid_bit_nr can reach HV_MAX_SPARSE_VCPU_BANKS or more and test_bit() then reads beyond valid_bank_mask.

Quoted 1 line

[...]

Applied to kvm-x86 misc, with a heavily massaged changelog and a KASAN splat. I also added a

  BUILD_BUG_ON(BITS_PER_TYPE(valid_bank_mask) != HV_MAX_SPARSE_VCPU_BANKS);

to ensure we don't re-introduce the bug if HV_MAX_SPARSE_VCPU_BANKS ever grows beyond 64.

Thanks!

[1/1] KVM: x86: hyper-v: Bound the bank index in hv_is_vp_in_sparse_set()

      https://github.com/kvm-x86/linux/commit/4721f8160f17

--
https://github.com/kvm-x86/linux/tree/next