Skip to contentSkip to conversation

Documentation: KVM: Formalizing taking vcpu->mutex *outside* of kvm->slots_lock

PATCH Applied Sean Christopherson 1 participant 2 messages Mar 2 – Mar 5

Subsystems (2)
lore
Share & tools
lore

Sean Christopherson

author reviewer
#0 lore raw

PATCH v2 1/1 Documentation: KVM: Formalizing taking vcpu->mutex *outside* of kvm->slots_lock

Explicitly document the ordering of vcpu->mutex being taken *outside* of kvm->slots_lock. While somewhat unintuitive since vCPUs conceptually have narrower scope than VMs, the scope of the owning object (vCPU versus VM) doesn't automatically carry over to the lock. In this case, vcpu->mutex has far broader scope than kvm->slots_lock. As Paolo put it, it's a "don't worry about multiple ioctls at the same time" mutex that's intended to be taken at the outer edges of KVM.

More importantly, arm64 and x86 have gained flows that take kvm->slots_lock inside of vcpu->mutex. x86's kvm_inhibit_apic_access_page() is particularly nasty, as slots_lock is taken quite deep within KVM_RUN, i.e. simply swapping the ordering isn't an option.

Commit to the vcpu->mutex => kvm->slots_lock ordering, as vcpu->mutex really is intended to be a "top-level" lock, whereas kvm->slots_lock is "just" a helper lock.

Opportunistically document that vcpu->mutex is also taken outside of slots_arch_lock, e.g. when allocating shadow roots on x86 (which is the entire reason slots_arch_lock exists, as shadow roots must be allocated while holding kvm->srcu)

kvm_mmu_new_pgd()
|
-> kvm_mmu_reload()

     |
     -> kvm_mmu_load()
        |
        -> mmu_alloc_shadow_roots()
           |
           -> mmu_first_shadow_root_alloc()

but also when manipulating memslots in vCPU context, e.g. when inhibiting the APIC-access page via the aforementioned kvm_inhibit_apic_access_page()

kvm_inhibit_apic_access_page()
|
-> __x86_set_memory_region()

     |
     -> kvm_set_internal_memslot()
        |
        -> kvm_set_memory_region()
           |
           -> kvm_set_memslot()

---

v2:
- Document that vcpu->mutex is taken outside of slots_arch_lock. [Paolo]
- Rework changelog to not claim that taking vcpu->mutex outside of slots_lock is "arguably wrong". [Paolo]
- Provide sample call chains for slots_arch_lock. [Paolo]

v1:
- https://lore.kernel.org/all/[email protected]
- https://lore.kernel.org/all/[email protected]

 Documentation/virt/kvm/locking.rst | 2 ++
 1 file changed, 2 insertions(+)
Documentation/virt/kvm/locking.rst +2 -0
index ae8bce7fecbe..662231e958a0 100644
@@ -17,6 +17,8 @@ The acquisition orders for mutexes are as follows:
 
 - kvm->lock is taken outside kvm->slots_lock and kvm->irq_lock
 
+- vcpu->mutex is taken outside kvm->slots_lock and kvm->slots_arch_lock
+
 - kvm->slots_lock is taken outside kvm->irq_lock, though acquiring
   them together is quite rare.
 

base-commit: b1195183ed42f1522fae3fe44ebee3af437aa000

Signature · 1 line
2.53.0.473.g4a7958ca14-goog
author reviewer Applied
#1 lore raw
Quoted 9 lines · Sean Christopherson

Explicitly document the ordering of vcpu->mutex being taken *outside* of kvm->slots_lock. While somewhat unintuitive since vCPUs conceptually have narrower scope than VMs, the scope of the owning object (vCPU versus VM) doesn't automatically carry over to the lock. In this case, vcpu->mutex has far broader scope than kvm->slots_lock. As Paolo put it, it's a "don't worry about multiple ioctls at the same time" mutex that's intended to be taken at the outer edges of KVM.

[...]

Applied to kvm-x86 generic.

[1/1] Documentation: KVM: Formalizing taking vcpu->mutex *outside* of kvm->slots_lock

      https://github.com/kvm-x86/linux/commit/f8211e95dfda

--
https://github.com/kvm-x86/linux/tree/next