KVM: nVMX: Mark APIC page dirty on VM-Exit
Sean Christopherson
maintainerv3 cover KVM: nVMX: Mark APIC page dirty on VM-Exit
Extended version of Fred's patch to mark the APIC access page dirty on VM-Exit (KVM already marks it dirty when it's unmapped).
v3:
- Fix a benign memslots bug in __kvm_vcpu_map().
- Mark vmcs12 pages dirty if and only if they're mapped (out-of-band).
- Don't mark the APIC access page dirty when deliver nested posted IRQ.
v2: https://lore.kernel.org/all/[email protected]
Fred Griffoul (1):
KVM: nVMX: Mark APIC access page dirty when syncing vmcs12 pages
Sean Christopherson (4):
KVM: Use vCPU specific memslots in __kvm_vcpu_map()
KVM: x86: Mark vmcs12 pages as dirty if and only if they're mapped
KVM: nVMX: Precisely mark vAPIC and PID maps dirty when delivering
nested PI
KVM: VMX: Move nested_mark_vmcs12_pages_dirty() to vmx.c, and rename
arch/x86/kvm/vmx/nested.c | 25 ++----------------------- arch/x86/kvm/vmx/vmx.c | 11 ++++++++++- include/linux/kvm_host.h | 9 ++++++++- virt/kvm/kvm_main.c | 2 +- 4 files changed, 21 insertions(+), 26 deletions(-)
base-commit: 115d5de2eef32ac5cd488404b44b38789362dbe6
Signature · 1 line
2.52.0.rc2.455.g230fcf2819-goog
Sean Christopherson
maintainerPATCH v3 1/5 KVM: Use vCPU specific memslots in __kvm_vcpu_map()
When establishing a "host access map", lookup the gfn in the vCPU specific memslots, as the intent is that the mapping will be established for the current vCPU context. Specifically, using __kvm_vcpu_map() in x86's SMM context should create mappings based on the SMM memslots, not the non-SMM memslots.
Luckily, the bug is benign as x86 is the only architecture with multiple memslot address spaces, and all of x86's usage is limited to non-SMM. The calls in (or reachable by) {svm,vmx}_enter_smm() are made before enter_smm() sets HF_SMM_MASK, and the calls in {svm,vmx}_leave_smm() are made after emulator_leave_smm() clears HF_SMM_MASK.
Note, kvm_vcpu_unmap() uses the vCPU specific memslots, only the map() side of things is broken.
--- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
virt/kvm/kvm_main.c +1 -1
@@ -3118,7 +3118,7 @@ int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, bool writable) {struct kvm_follow_pfn kfp = { - .slot =gfn_to_memslot (vcpu->kvm, gfn), + .slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn), .gfn = gfn, .flags = writable ?FOLL_WRITE :0 , .refcounted_page = &map->pinned_page,
Signature · 1 line
2.52.0.rc2.455.g230fcf2819-goog
Sean Christopherson
maintainerPATCH v3 2/5 KVM: x86: Mark vmcs12 pages as dirty if and only if they're mapped
In reply to Sean ChristophersonMark vmcs12 pages as dirty (in KVM's dirty log bitmap) if and only if the page is mapped, i.e. if the page is actually "active" in vmcs02. For some pages, KVM simply disables the associated VMCS control if the vmcs12 page is unreachable, i.e. it's possible for nested VM-Enter to succeed with a "bad" vmcs12 page.
--- arch/x86/kvm/vmx/nested.c | 15 +++------------ include/linux/kvm_host.h | 9 ++++++++- 2 files changed, 11 insertions(+), 13 deletions(-)
arch/x86/kvm/vmx/nested.c +3 -12
@@ -3983,23 +3983,14 @@ static void vmcs12_save_pending_event(struct kvm_vcpu *vcpu,void nested_mark_vmcs12_pages_dirty (struct kvm_vcpu *vcpu) { -struct vmcs12 *vmcs12 =get_vmcs12 (vcpu); -gfn_t gfn; +struct vcpu_vmx *vmx =to_vmx (vcpu);/* * Don't need to mark the APIC access page dirty; it is never * written to by the CPU during APIC virtualization. */ - -if (nested_cpu_has (vmcs12,CPU_BASED_TPR_SHADOW )) { - gfn = vmcs12->virtual_apic_page_addr >>PAGE_SHIFT ; -kvm_vcpu_mark_page_dirty (vcpu, gfn); - } - -if (nested_cpu_has_posted_intr (vmcs12)) { - gfn = vmcs12->posted_intr_desc_addr >>PAGE_SHIFT ; -kvm_vcpu_mark_page_dirty (vcpu, gfn); - } +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map); +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map); }static int vmx_complete_nested_posted_interrupt (struct kvm_vcpu *vcpu)
include/linux/kvm_host.h +8 -1
@@ -1381,6 +1381,7 @@ bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn);unsigned long kvm_host_page_size (struct kvm_vcpu *vcpu,gfn_t gfn);void mark_page_dirty_in_slot (struct kvm *kvm,const struct kvm_memory_slot *memslot,gfn_t gfn);void mark_page_dirty (struct kvm *kvm,gfn_t gfn); +void kvm_vcpu_mark_page_dirty (struct kvm_vcpu *vcpu,gfn_t gfn);int __kvm_vcpu_map (struct kvm_vcpu *vcpu,gpa_t gpa,struct kvm_host_map *map,bool writable);
@@ -1398,6 +1399,13 @@ static inline int kvm_vcpu_map_readonly(struct kvm_vcpu *vcpu, gpa_t gpa,return __kvm_vcpu_map (vcpu, gpa, map, false); } +static inline void kvm_vcpu_map_mark_dirty (struct kvm_vcpu *vcpu, +struct kvm_host_map *map) +{ +if (kvm_vcpu_mapped (map)) +kvm_vcpu_mark_page_dirty (vcpu, map->gfn); +} +unsigned long kvm_vcpu_gfn_to_hva (struct kvm_vcpu *vcpu,gfn_t gfn);unsigned long kvm_vcpu_gfn_to_hva_prot (struct kvm_vcpu *vcpu,gfn_t gfn,bool *writable); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, int offset,
@@ -1410,7 +1418,6 @@ int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, const void *data int offset, int len);int kvm_vcpu_write_guest (struct kvm_vcpu *vcpu,gpa_t gpa,const void *data,unsigned long len); -void kvm_vcpu_mark_page_dirty (struct kvm_vcpu *vcpu,gfn_t gfn);/** * kvm_gpc_init - initialize gfn_to_pfn_cache.
Signature · 1 line
2.52.0.rc2.455.g230fcf2819-goog
Sean Christopherson
maintainerPATCH v3 3/5 KVM: nVMX: Precisely mark vAPIC and PID maps dirty when delivering nested PI
In reply to Sean ChristophersonExplicitly mark the vmcs12 vAPIC and PI descriptor pages as dirty when delivering a nested posted interrupt instead of marking all vmcs12 pages as dirty. This will allow marking the APIC access page (and any future
vmcs12 pages) as dirty in nested_mark_vmcs12_pages_dirty() without over-
dirtying in the nested PI case. Manually marking the vAPIC and PID pages as dirty also makes the flow a bit more self-documenting, e.g. it's not obvious at first glance that vmx->nested.pi_desc is actually a host kernel mapping of a vmcs12 page.
No functional change intended.
--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -4027,7 +4027,8 @@ static int vmx_complete_nested_posted_interrupt(struct kvm_vcpu *vcpu) } } -nested_mark_vmcs12_pages_dirty (vcpu); +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map); +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map);return 0 ; mmio_needed:
Signature · 1 line
2.52.0.rc2.455.g230fcf2819-goog
Sean Christopherson
maintainerPATCH v3 4/5 KVM: VMX: Move nested_mark_vmcs12_pages_dirty() to vmx.c, and rename
In reply to Sean ChristophersonMove nested_mark_vmcs12_pages_dirty() to vmx.c now that it's only used in the VM-Exit path, and add "all" to its name to document that its purpose is to mark all (mapped-out-of-band) vmcs12 pages as dirty.
No functional change intended.
--- arch/x86/kvm/vmx/nested.c | 13 ------------- arch/x86/kvm/vmx/vmx.c | 14 +++++++++++++- 2 files changed, 13 insertions(+), 14 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -13
@@ -3980,19 +3980,6 @@ static void vmcs12_save_pending_event(struct kvm_vcpu *vcpu, } } - -void nested_mark_vmcs12_pages_dirty (struct kvm_vcpu *vcpu) -{ -struct vcpu_vmx *vmx =to_vmx (vcpu); - -/* -* Don't need to mark the APIC access page dirty; it is never -* written to by the CPU during APIC virtualization. -*/ -kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map); -kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map); -} -static int vmx_complete_nested_posted_interrupt (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
arch/x86/kvm/vmx/vmx.c +13 -1
@@ -6378,6 +6378,18 @@ static void vmx_flush_pml_buffer(struct kvm_vcpu *vcpu)vmcs_write16 (GUEST_PML_INDEX ,PML_HEAD_INDEX ); } +static void nested_vmx_mark_all_vmcs12_pages_dirty (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +/* +* Don't need to mark the APIC access page dirty; it is never +* written to by the CPU during APIC virtualization. +*/ +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map); +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map); +} +static void vmx_dump_sel (char *name,uint32_t sel) { pr_err("%s sel=0x%04x, attr=0x%05x, limit=0x%08x, base=0x%016lx\n",
@@ -6655,7 +6667,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)* Mark them dirty on every exit from L2 to prevent them from * getting out of sync with dirty tracking. */ -nested_mark_vmcs12_pages_dirty (vcpu); +nested_vmx_mark_all_vmcs12_pages_dirty (vcpu);/* * Synthesize a triple fault if L2 state is invalid. In normal
Signature · 1 line
2.52.0.rc2.455.g230fcf2819-goog
Re: [PATCH v3 4/5] KVM: VMX: Move nested_mark_vmcs12_pages_dirty() to vmx.c, and rename
Quoted 17 lines · Sean Christopherson
Move
nested_mark_vmcs12_pages_dirty()tovmx.cnow that it's only used in the VM-Exit path, and add "all" to its name to document that its purpose is to mark all (mapped-out-of-band) vmcs12 pages as dirty.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/vmx/nested.c | 13 ------------- arch/x86/kvm/vmx/vmx.c | 14 +++++++++++++- 2 files changed, 13 insertions(+), 14 deletions(-)arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c static void vmcs12_save_pending_event(struct kvm_vcpu *vcpu, · line 3984
Quoted 5 lines
arch/x86/kvm/vmx/nested.c +0
-2@@ -3980,19 +3980,6 @@ static void vmcs12_save_pending_event(struct kvm_vcpu *vcpu, } } - -void nested_mark_vmcs12_pages_dirty (struct kvm_vcpu *vcpu)
The declaration in the header file is left over.
Since the patch series has been merged in the kvm-x86 next branch, I sent a patch to remove it.
Quoted 46 lines
arch/x86/kvm/vmx/nested.c +0
-11-{ -struct vcpu_vmx *vmx =to_vmx (vcpu); - -/* -* Don't need to mark the APIC access page dirty; it is never -* written to by the CPU during APIC virtualization. -*/ -kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map); -kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map); -} -static int vmx_complete_nested_posted_interrupt (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);arch/x86/kvm/vmx/vmx.c +13
-1@@ -6378,6 +6378,18 @@ static void vmx_flush_pml_buffer(struct kvm_vcpu *vcpu)vmcs_write16 (GUEST_PML_INDEX ,PML_HEAD_INDEX ); } +static void nested_vmx_mark_all_vmcs12_pages_dirty (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +/* +* Don't need to mark the APIC access page dirty; it is never +* written to by the CPU during APIC virtualization. +*/ +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map); +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map); +} +static void vmx_dump_sel (char *name,uint32_t sel) { pr_err("%s sel=0x%04x, attr=0x%05x, limit=0x%08x, base=0x%016lx\n",@@ -6655,7 +6667,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)* Mark them dirty on every exit from L2 to prevent them from * getting out of sync with dirty tracking. */ -nested_mark_vmcs12_pages_dirty (vcpu); +nested_vmx_mark_all_vmcs12_pages_dirty (vcpu);/* * Synthesize a triple fault if L2 state is invalid. In normal
Sean Christopherson
maintainerPATCH v3 5/5 KVM: nVMX: Mark APIC access page dirty when syncing vmcs12 pages
In reply to Sean ChristophersonFrom: Fred Griffoul <[email protected]>
For consistency with commit 7afe79f5734a ("KVM: nVMX: Mark vmcs12's APIC access page dirty when unmapping"), which marks the page dirty during unmap operations, also mark it dirty during vmcs12 page synchronization.
--- arch/x86/kvm/vmx/vmx.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
arch/x86/kvm/vmx/vmx.c +1 -4
@@ -6382,10 +6382,7 @@ static void nested_vmx_mark_all_vmcs12_pages_dirty(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -/* -* Don't need to mark the APIC access page dirty; it is never -* written to by the CPU during APIC virtualization. -*/ +kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.apic_access_page_map);kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.virtual_apic_map);kvm_vcpu_map_mark_dirty (vcpu, &vmx->nested.pi_desc_map); }
Signature · 1 line
2.52.0.rc2.455.g230fcf2819-goog
Quoted 9 lines · Sean Christopherson
Extended version of Fred's patch to mark the APIC access page dirty on VM-Exit (KVM already marks it dirty when it's unmapped).
v3:
- Fix a benign memslots bug in__kvm_vcpu_map().
- Mark vmcs12 pages dirty if and only if they're mapped (out-of-band).
- Don't mark the APIC access page dirty when deliver nested posted IRQ.[...]
Applied to kvm-x86 apic, thanks!
[1/5] KVM: Use vCPU specific memslots in __kvm_vcpu_map()
https://github.com/kvm-x86/linux/commit/44da6629d282 [2/5] KVM: x86: Mark vmcs12 pages as dirty if and only if they're mapped https://github.com/kvm-x86/linux/commit/70b02809ded9 [3/5] KVM: nVMX: Precisely mark vAPIC and PID maps dirty when delivering nested PI https://github.com/kvm-x86/linux/commit/f74bb1d2eda1 [4/5] KVM: VMX: Move nested_mark_vmcs12_pages_dirty() to vmx.c, and rename https://github.com/kvm-x86/linux/commit/57dfa61f6248 [5/5] KVM: nVMX: Mark APIC access page dirty when syncing vmcs12 pages https://github.com/kvm-x86/linux/commit/c9d7134679eb