KVM: x86: TLB flushing fixes and enhancements
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest37 patches 45 repliesIn review
- v232 patches 12 repliesSuperseded
Changes in v3 from the cover letter
- Fix freeing of roots during INVVPID, I botched things when tweaking Junaid's original patch.
- Move "last vpid02" logic into nested VMX flushing helper. [Paolo]
- Split "skip tlb flush" and "skip mmu sync" logic during fast roots switch. [Paolo]
- Unconditionally skip tlb flush during fast roots switch on nested VMX transitions, i.e. let nested_vmx_transition_tlb_flush() do the work. This avoids flushing when EPT=0 and VPID=1. [Paolo]
- Do more cr3->pgd conversions in related code that drove me bonkers when trying to figure out what was broken with VPID. I think this knocks off the last code that uses "cr3" for variables/functions that work with cr3 or eptp.
Patches in v3 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/37 | KVM: VMX: Flush all EPTP/VPID contexts on remote TLB flusharch/ | +27 | 4 replies | |
| 2/37 | KVM: nVMX: Validate the EPTP when emulating INVEPT(EXTENT_CONTEXT)arch/ | +5 | 3 replies | |
| 3/37 | KVM: nVMX: Invalidate all EPTP contexts when emulating INVEPT for L1arch/ | +4 | 9 replies | |
| 4/37 | KVM: nVMX: Invalidate all roots when emulating INVVPID without EPTarch/ | +14 | 5 replies | |
| 5/37 | KVM: x86: Export kvm_propagate_fault() (as kvm_inject_emulated_page_fault)arch/ arch/ | +8 | 3 replies | |
| 6/37 | KVM: x86: Consolidate logic for injecting page faults to L1arch/ arch/ arch/ +1 more | +10 | 1 reply | |
| 7/37 | KVM: x86: Sync SPTEs when injecting page/EPT fault into L1arch/ arch/ arch/ | +51 | 0 replies | |
| 8/37 | KVM: VMX: Skip global INVVPID fallback if vpid==0 in vpid_sync_context()arch/ | +1 | 1 reply | |
| 9/37 | KVM: VMX: Use vpid_sync_context() directly when possiblearch/ | +6 | 0 replies | |
| 10/37 | KVM: VMX: Move vpid_sync_vcpu_addr() down a few linesarch/ | +13 | 0 replies | |
| 11/37 | KVM: VMX: Handle INVVPID fallback logic in vpid_sync_vcpu_addr()arch/ arch/ | +6 | 0 replies | |
| 12/37 | KVM: VMX: Drop redundant capability checks in low level INVVPID helpersarch/ arch/ | +2 | 0 replies | |
| 13/37 | KVM: nVMX: Use vpid_sync_vcpu_addr() to emulate INVVPID with addressarch/ | +1 | 0 replies | |
| 14/37 | KVM: x86: Move "flush guest's TLB" logic to separate kvm_x86_ops hookarch/ arch/ arch/ +1 more | +26 | 5 replies | |
| 15/37 | KVM: VMX: Clean up vmx_flush_tlb_gva()arch/ | +3 | 0 replies | |
| 16/37 | KVM: x86: Drop @invalidate_gpa param from kvm_x86_ops' tlb_flush()arch/ arch/ arch/ +3 more | +24 | 1 reply | |
| 17/37 | KVM: SVM: Wire up ->tlb_flush_guest() directly to svm_flush_tlb()arch/ | +1 | 1 reply | |
| 18/37 | KVM: VMX: Move vmx_flush_tlb() to vmx.carch/ arch/ | +25 | 1 reply | |
| 19/37 | KVM: nVMX: Move nested_get_vpid02() to vmx/nested.harch/ arch/ | +7 | 1 reply | |
| 20/37 | KVM: VMX: Introduce vmx_flush_tlb_current()arch/ | +18 | 0 replies | |
| 21/37 | KVM: SVM: Document the ASID logic in svm_flush_tlb()arch/ | +7 | 0 replies | |
| 22/37 | KVM: x86: Rename ->tlb_flush() to ->tlb_flush_all()arch/ arch/ arch/ +2 more | +13 | 0 replies | |
| 23/37 | KVM: nVMX: Add helper to handle TLB flushes on nested VM-Enter/VM-Exitarch/ | +44 | 6 replies | |
| 24/37 | KVM: x86: Introduce KVM_REQ_TLB_FLUSH_CURRENT to flush current ASIDarch/ arch/ arch/ +3 more | +31 | 0 replies | |
| 25/37 | KVM: x86/mmu: Use KVM_REQ_TLB_FLUSH_CURRENT for MMU specific flushesarch/ | +6 | 0 replies | |
| 26/37 | KVM: nVMX: Selectively use TLB_FLUSH_CURRENT for nested VM-Enter/VM-Exitarch/ | +5 | 0 replies | |
| 27/37 | KVM: nVMX: Reload APIC access page on nested VM-Exit only if necessaryarch/ arch/ arch/ | +12 | 0 replies | |
| 28/37 | KVM: VMX: Retrieve APIC access page HPA only when necessaryarch/ arch/ arch/ | +16 | 0 replies | |
| 29/37 | KVM: VMX: Don't reload APIC access page if its control is disabledarch/ | +13 | 0 replies | |
| 30/37 | KVM: x86/mmu: Move fast_cr3_switch() side effects to __kvm_mmu_new_cr3()arch/ | +31 | 0 replies | |
| 31/37 | KVM: x86/mmu: Add separate override for MMU sync during fast CR3 switcharch/ arch/ arch/ +1 more | +11 | 1 reply | |
| 32/37 | KVM: x86/mmu: Add module param to force TLB flush on root reusearch/ | +5 | 0 replies | |
| 33/37 | KVM: nVMX: Skip MMU sync on nested VMX transition when possiblearch/ arch/ | +44 | 1 reply | |
| 34/37 | KVM: nVMX: Don't flush TLB on nested VMX transitionarch/ arch/ | +5 | 2 replies | |
| 35/37 | KVM: nVMX: Free only the affected contexts when emulating INVEPTarch/ | +21 | 0 replies | |
| 36/37 | KVM: x86: Replace "cr3" with "pgd" in "new cr3/pgd" related codearch/ arch/ arch/ +2 more | +39 | 0 replies | |
| 37/37 | KVM: VMX: Clean cr3/pgd handling in vmx_load_mmu_pgd()arch/ | +4 | 0 replies |
Reviews trailers per patch
16 of 37 patches have a Reviewed-by or Acked-by.
| Patch | Liran Alon | Vitaly Kuznetsov | Paolo Bonzini | Miaohe Lin |
|---|---|---|---|---|
| 1/37 | R-b | |||
| 2/37 | R-b | |||
| 3/37 | R-b | |||
| 4/37 | R-b | |||
| 5/37 | R-b | |||
| 6/37 | ||||
| 7/37 | ||||
| 8/37 | R-b | |||
| 9/37 | R-b | R-b | R-b | |
| 10/37 | R-b | R-b | R-b | |
| 11/37 | R-b | R-b | R-b | |
| 12/37 | ||||
| 13/37 | R-b | R-b | ||
| 14/37 | R-b | |||
| 15/37 | R-b | R-b | ||
| 16/37 | R-b | |||
| 17/37 | R-b | |||
| 18/37 | R-b | |||
| 19/37 | R-b | |||
| 20/37 | ||||
| 21/37 | ||||
| 22/37 | ||||
| 23/37 | ||||
| 24/37 | ||||
| 25/37 | ||||
| 26/37 | ||||
| 27/37 | ||||
| 28/37 | ||||
| 29/37 | ||||
| 30/37 | ||||
| 31/37 | ||||
| 32/37 | ||||
| 33/37 | ||||
| 34/37 | ||||
| 35/37 | ||||
| 36/37 | ||||
| 37/37 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.
Cover letter [PATCH v3 00/37] KVM: x86: TLB flushing fixes and enhancements
VMX TLB flushing cleanup series to fix a variety of bugs, and to avoid unnecessary TLB flushes on nested VMX transitions.
1) Nested VMX doesn't properly flush all ASIDs/contexts on system events, e.g. on mmu_notifier invalidate all contexts for L1 *and* L2 need to be invalidated, but KVM generally only flushes L1 or L2 (or just L1).
2) #1 is largely benign because nested VMX always flushes the new context on nested VM-Entry/VM-Exit.
High level overview:
a) Fix the main TLB flushing bug with a big hammer.
b) Fix a few other flushing related bugs.
c) Clean up vmx_tlb_flush(), i.e. what was v1 of this series.
d) Reintroduce current-ASID/context flushing to regain some of the precision that got blasted away by the big hammer in #1.
e) Fix random code paths that unnecessarily trigger TLB flushes on nested VMX transitions.
f) Stop flushing on every nested VMX transition.
g) Extra cleanup.
v3:
- Fix freeing of roots during INVVPID, I botched things when tweaking Junaid's original patch.
- Move "last vpid02" logic into nested VMX flushing helper. [Paolo]
- Split "skip tlb flush" and "skip mmu sync" logic during fast roots switch. [Paolo]
- Unconditionally skip tlb flush during fast roots switch on nested VMX transitions, i.e. let nested_vmx_transition_tlb_flush() do the work. This avoids flushing when EPT=0 and VPID=1. [Paolo]
- Do more cr3->pgd conversions in related code that drove me bonkers when trying to figure out what was broken with VPID. I think this knocks off the last code that uses "cr3" for variables/functions that work with cr3 or eptp.
v2:
- Basically a new series.
v1:
- https://patchwork.kernel.org/cover/11394987/
Junaid Shahid (2):
KVM: nVMX: Invalidate all roots when emulating INVVPID without EPT
KVM: x86: Sync SPTEs when injecting page/EPT fault into L1
Sean Christopherson (35):
KVM: VMX: Flush all EPTP/VPID contexts on remote TLB flush
KVM: nVMX: Validate the EPTP when emulating INVEPT(EXTENT_CONTEXT)
KVM: nVMX: Invalidate all EPTP contexts when emulating INVEPT for L1
KVM: x86: Export kvm_propagate_fault() (as
kvm_inject_emulated_page_fault)
KVM: x86: Consolidate logic for injecting page faults to L1
KVM: VMX: Skip global INVVPID fallback if vpid==0 in
vpid_sync_context()
KVM: VMX: Use vpid_sync_context() directly when possible
KVM: VMX: Move vpid_sync_vcpu_addr() down a few lines
KVM: VMX: Handle INVVPID fallback logic in vpid_sync_vcpu_addr()
KVM: VMX: Drop redundant capability checks in low level INVVPID
helpers
KVM: nVMX: Use vpid_sync_vcpu_addr() to emulate INVVPID with address
KVM: x86: Move "flush guest's TLB" logic to separate kvm_x86_ops hook
KVM: VMX: Clean up vmx_flush_tlb_gva()
KVM: x86: Drop @invalidate_gpa param from kvm_x86_ops' tlb_flush()
KVM: SVM: Wire up ->tlb_flush_guest() directly to svm_flush_tlb()
KVM: VMX: Move vmx_flush_tlb() to vmx.c
KVM: nVMX: Move nested_get_vpid02() to vmx/nested.h
KVM: VMX: Introduce vmx_flush_tlb_current()
KVM: SVM: Document the ASID logic in svm_flush_tlb()
KVM: x86: Rename ->tlb_flush() to ->tlb_flush_all()
KVM: nVMX: Add helper to handle TLB flushes on nested VM-Enter/VM-Exit
KVM: x86: Introduce KVM_REQ_TLB_FLUSH_CURRENT to flush current ASID
KVM: x86/mmu: Use KVM_REQ_TLB_FLUSH_CURRENT for MMU specific flushes
KVM: nVMX: Selectively use TLB_FLUSH_CURRENT for nested
VM-Enter/VM-Exit
KVM: nVMX: Reload APIC access page on nested VM-Exit only if necessary
KVM: VMX: Retrieve APIC access page HPA only when necessary
KVM: VMX: Don't reload APIC access page if its control is disabled
KVM: x86/mmu: Move fast_cr3_switch() side effects to
__kvm_mmu_new_cr3()
KVM: x86/mmu: Add separate override for MMU sync during fast CR3
switch
KVM: x86/mmu: Add module param to force TLB flush on root reuse
KVM: nVMX: Skip MMU sync on nested VMX transition when possible
KVM: nVMX: Don't flush TLB on nested VMX transition
KVM: nVMX: Free only the affected contexts when emulating INVEPT
KVM: x86: Replace "cr3" with "pgd" in "new cr3/pgd" related code
KVM: VMX: Clean cr3/pgd handling in vmx_load_mmu_pgd()
arch/x86/include/asm/kvm_host.h | 25 +++- arch/x86/kvm/mmu/mmu.c | 145 +++++++++---------- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/svm.c | 19 ++- arch/x86/kvm/vmx/nested.c | 249 ++++++++++++++++++++++---------- arch/x86/kvm/vmx/nested.h | 7 + arch/x86/kvm/vmx/ops.h | 32 ++-- arch/x86/kvm/vmx/vmx.c | 119 ++++++++++++--- arch/x86/kvm/vmx/vmx.h | 19 +-- arch/x86/kvm/x86.c | 67 ++++++--- arch/x86/kvm/x86.h | 6 + 11 files changed, 448 insertions(+), 242 deletions(-)
Signature · 1 line
2.24.1