Extend KVM trace_kvm_nested_vmrun() to support VMX
v4 cover Extend KVM trace_kvm_nested_vmrun() to support VMX
This patch set update trace_kvm_nested_vmrun() to support VMX. In addition, add the guest_pgd field to enrich the trace information.
v3 -> v4:
- Rebase to tip of kvm/queue.
v2 -> v3:
- Split the insertion of guest pgd field into a separate patch [seanjc].
- Update field names as suggested [seanjc].
v2:
- https://lore.kernel.org/lkml/[email protected]/T/
v1 link:
- https://lore.kernel.org/lkml/[email protected]/T/
David Matlack (1):
KVM: nVMX: Add tracepoint for nested vmenter
Mingwei Zhang (2):
KVM: x86: Update trace function for nested VM entry to support VMX
KVM: x86: Print guest pgd in kvm_nested_vmenter()
arch/x86/kvm/svm/nested.c | 8 ++++++-- arch/x86/kvm/trace.h | 33 +++++++++++++++++++++++---------- arch/x86/kvm/vmx/nested.c | 10 ++++++++++ arch/x86/kvm/x86.c | 2 +- 4 files changed, 40 insertions(+), 13 deletions(-)
base-commit: 372d07084593dc7a399bf9bee815711b1fb1bcf2
Signature · 1 line
2.37.2.672.g94769d06f0-goog
PATCH v4 1/3 KVM: x86: Update trace function for nested VM entry to support VMX
Update trace function for nested VM entry to support VMX. Existing trace function only supports nested VMX and the information printed out is AMD specific.
So, update trace_kvm_nested_vmrun() to trace_kvm_nested_vmenter(), since 'vmenter' is generic. Add a new field 'isa' to recognize Intel and AMD; Update the output to print out VMX/SVM related naming respectively, eg., vmcb vs. vmcs; npt vs. ept.
Opportunistically update the call site of trace_kvm_nested_vmenter() to make one line per parameter.
--- arch/x86/kvm/svm/nested.c | 6 ++++-- arch/x86/kvm/trace.h | 28 ++++++++++++++++++---------- arch/x86/kvm/x86.c | 2 +- 3 files changed, 23 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/nested.c +4 -2
@@ -781,11 +781,13 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa,struct vcpu_svm *svm =to_svm (vcpu);int ret; -trace_kvm_nested_vmrun (svm->vmcb->save.rip, vmcb12_gpa, +trace_kvm_nested_vmenter (svm->vmcb->save.rip, + vmcb12_gpa, vmcb12->save.rip, vmcb12->control.int_ctl, vmcb12->control.event_inj, - vmcb12->control.nested_ctl); + vmcb12->control.nested_ctl, +KVM_ISA_SVM ); trace_kvm_nested_intercepts(vmcb12->control.intercepts[INTERCEPT_CR] & 0xffff, vmcb12->control.intercepts[INTERCEPT_CR] >> 16,
arch/x86/kvm/trace.h +18 -10
@@ -589,10 +589,11 @@ TRACE_EVENT(kvm_pv_eoi,/* * Tracepoint for nested VMRUN */ -TRACE_EVENT(kvm_nested_vmrun, +TRACE_EVENT(kvm_nested_vmenter, TP_PROTO(__u64 rip, __u64 vmcb, __u64 nested_rip, __u32 int_ctl, - __u32 event_inj, bool npt), - TP_ARGS(rip, vmcb, nested_rip, int_ctl, event_inj, npt), + __u32 event_inj, bool tdp_enabled, __u32 isa), + TP_ARGS(rip, vmcb, nested_rip, int_ctl, event_inj, tdp_enabled, + isa), TP_STRUCT__entry( __field( __u64, rip )
@@ -600,7 +601,8 @@ TRACE_EVENT(kvm_nested_vmrun, __field( __u64, nested_rip ) __field( __u32, int_ctl ) __field( __u32, event_inj ) -__field ( bool, npt ) + __field( bool, tdp_enabled ) +__field ( __u32, isa ) ), TP_fast_assign(
@@ -609,14 +611,20 @@ TRACE_EVENT(kvm_nested_vmrun, __entry->nested_rip = nested_rip; __entry->int_ctl = int_ctl; __entry->event_inj = event_inj; - __entry->npt = npt; + __entry->tdp_enabled = tdp_enabled; + __entry->isa = isa; ), -TP_printk ("rip: 0x%016llx vmcb: 0x%016llx nrip: 0x%016llx int_ctl: 0x%08x " -"event_inj: 0x%08x npt: %s" , - __entry->rip, __entry->vmcb, __entry->nested_rip, - __entry->int_ctl, __entry->event_inj, - __entry->npt ?"on" :"off" ) +TP_printk ("rip: 0x%016llx %s: 0x%016llx nested_rip: 0x%016llx " +"int_ctl: 0x%08x event_inj: 0x%08x nested_%s: %s" , + __entry->rip, + __entry->isa ==KVM_ISA_VMX ?"vmcs" :"vmcb" , + __entry->vmcb, + __entry->nested_rip, + __entry->int_ctl, + __entry->event_inj, + __entry->isa ==KVM_ISA_VMX ?"ept" :"npt" , + __entry->tdp_enabled ?"on" :"off" ) ); TRACE_EVENT(kvm_nested_intercepts,
arch/x86/kvm/x86.c +1 -1
@@ -13375,7 +13375,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_page_fault);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_msr);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_cr); -EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmrun); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmenter);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmexit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_vmexit_inject);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_intr_vmexit);
Signature · 1 line
2.37.2.672.g94769d06f0-goog
Sean Christopherson
maintainerRe: [PATCH v4 1/3] KVM: x86: Update trace function for nested VM entry to support VMX
Quoted 26 lines · Mingwei Zhang
Update trace function for nested VM entry to support VMX. Existing trace function only supports nested VMX and the information printed out is AMD specific.
So, update
trace_kvm_nested_vmrun()totrace_kvm_nested_vmenter(), since 'vmenter' is generic. Add a new field 'isa' to recognize Intel and AMD; Update the output to print out VMX/SVM related naming respectively, eg., vmcb vs. vmcs; npt vs. ept.Opportunistically update the call site of
trace_kvm_nested_vmenter()to make one line per parameter.Signed-off-by Mingwei Zhang <[email protected]>--- arch/x86/kvm/svm/nested.c | 6 ++++-- arch/x86/kvm/trace.h | 28 ++++++++++++++++++---------- arch/x86/kvm/x86.c | 2 +- 3 files changed, 23 insertions(+), 13 deletions(-)arch/x86/kvm/svm/nested.c +0
-0@@ -781,11 +781,13 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa,struct vcpu_svm *svm =to_svm (vcpu);int ret;
arch/x86/kvm/svm/nested.c int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, · line 790
Quoted 10 lines
arch/x86/kvm/svm/nested.c +4
-2-trace_kvm_nested_vmrun (svm->vmcb->save.rip, vmcb12_gpa, +trace_kvm_nested_vmenter (svm->vmcb->save.rip, + vmcb12_gpa, vmcb12->save.rip, vmcb12->control.int_ctl, vmcb12->control.event_inj, - vmcb12->control.nested_ctl); + vmcb12->control.nested_ctl, +KVM_ISA_SVM );
Align indentation.
Quoted 9 lines
arch/x86/kvm/svm/nested.c +0
-0trace_kvm_nested_intercepts(vmcb12->control.intercepts[INTERCEPT_CR] & 0xffff, vmcb12->control.intercepts[INTERCEPT_CR] >> 16,arch/x86/kvm/trace.h +0
-0@@ -589,10 +589,11 @@ TRACE_EVENT(kvm_pv_eoi,/*
arch/x86/kvm/trace.h TRACE_EVENT(kvm_pv_eoi, · line 596
Quoted 10 lines
arch/x86/kvm/trace.h +4
-3* Tracepoint for nested VMRUN */ -TRACE_EVENT(kvm_nested_vmrun, +TRACE_EVENT(kvm_nested_vmenter, TP_PROTO(__u64 rip, __u64 vmcb, __u64 nested_rip, __u32 int_ctl, - __u32 event_inj, bool npt), - TP_ARGS(rip, vmcb, nested_rip, int_ctl, event_inj, npt), + __u32 event_inj, bool tdp_enabled, __u32 isa), + TP_ARGS(rip, vmcb, nested_rip, int_ctl, event_inj, tdp_enabled, + isa),
Let this poke out, "isa" feels quite lonely here. It's a moot point when patch 3 comes along, mostly a "for future reference" thing.
Quoted 26 lines
arch/x86/kvm/trace.h +4
-7TP_STRUCT__entry ( __field( __u64, rip )@@ -600,7 +601,8 @@ TRACE_EVENT(kvm_nested_vmrun, __field( __u64, nested_rip ) __field( __u32, int_ctl ) __field( __u32, event_inj ) -__field ( bool, npt ) + __field( bool, tdp_enabled ) +__field ( __u32, isa ) ), TP_fast_assign(@@ -609,14 +611,20 @@ TRACE_EVENT(kvm_nested_vmrun, __entry->nested_rip = nested_rip; __entry->int_ctl = int_ctl; __entry->event_inj = event_inj; - __entry->npt = npt; + __entry->tdp_enabled = tdp_enabled; + __entry->isa = isa; ), -TP_printk ("rip: 0x%016llx vmcb: 0x%016llx nrip: 0x%016llx int_ctl: 0x%08x " -"event_inj: 0x%08x npt: %s" , - __entry->rip, __entry->vmcb, __entry->nested_rip, - __entry->int_ctl, __entry->event_inj, - __entry->npt ?"on" :"off" )
arch/x86/kvm/trace.h TRACE_EVENT(kvm_nested_vmrun, · line 627
Quoted 10 lines
arch/x86/kvm/trace.h +10
-0+TP_printk ("rip: 0x%016llx %s: 0x%016llx nested_rip: 0x%016llx " +"int_ctl: 0x%08x event_inj: 0x%08x nested_%s: %s" , + __entry->rip, + __entry->isa ==KVM_ISA_VMX ?"vmcs" :"vmcb" , + __entry->vmcb, + __entry->nested_rip, + __entry->int_ctl, + __entry->event_inj, + __entry->isa ==KVM_ISA_VMX ?"ept" :"npt" , + __entry->tdp_enabled ?"on" :"off" )
Align indentation.
PATCH v4 2/3 KVM: nVMX: Add tracepoint for nested vmenter
In reply to Mingwei ZhangFrom: David Matlack <[email protected]>
Call trace_kvm_nested_vmenter() during nested VMLAUNCH/VMRESUME to bring parity with nSVM's usage of the tracepoint during nested VMRUN.
Attempt to use analagous VMCS fields to the VMCB fields that are reported in the SVM case:
"int_ctl": 32-bit field of the VMCB that the CPU uses to deliver virtual interrupts. The analagous VMCS field is the 16-bit "guest interrupt status".
"event_inj": 32-bit field of VMCB that is used to inject events (exceptions and interrupts) into the guest. The analagous VMCS field is the "VM-entry interruption-information field".
"npt_enabled": 1 when the VCPU has enabled nested paging. The analagous VMCS field is the enable-EPT execution control.
"npt_addr": 64-bit field when the VCPU has enabled nested paging. The analagous VMCS field is the ept_pointer.
--- arch/x86/kvm/vmx/nested.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/kvm/vmx/nested.c +8 -0
@@ -3364,6 +3364,14 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, };u32 failed_index; +trace_kvm_nested_vmenter (kvm_rip_read (vcpu), + vmx->nested.current_vmptr, + vmcs12->guest_rip, + vmcs12->guest_intr_status, + vmcs12->vm_entry_intr_info_field, + vmcs12->secondary_vm_exec_control &SECONDARY_EXEC_ENABLE_EPT , +KVM_ISA_VMX ); +kvm_service_local_tlb_flush_requests (vcpu); evaluate_pending_interrupts = exec_controls_get(vmx) &
Signature · 1 line
2.37.2.672.g94769d06f0-goog
PATCH v4 3/3 KVM: x86: Print guest pgd in kvm_nested_vmenter()
In reply to Mingwei ZhangPrint guest pgd in kvm_nested_vmenter() to enrich the information for tracing. When tdp is enabled, print the value of tdp page table (EPT/NPT); when tdp is disabled, print the value of non-nested CR3.
--- arch/x86/kvm/svm/nested.c | 2 ++ arch/x86/kvm/trace.h | 13 +++++++++---- arch/x86/kvm/vmx/nested.c | 2 ++ 3 files changed, 13 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/nested.c +2 -0
@@ -787,6 +787,8 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, vmcb12->control.int_ctl, vmcb12->control.event_inj, vmcb12->control.nested_ctl, + vmcb12->control.nested_cr3, + vmcb12->save.cr3, KVM_ISA_SVM); trace_kvm_nested_intercepts(vmcb12->control.intercepts[INTERCEPT_CR] & 0xffff,
arch/x86/kvm/trace.h +9 -4
@@ -591,9 +591,10 @@ TRACE_EVENT(kvm_pv_eoi,*/ TRACE_EVENT(kvm_nested_vmenter, TP_PROTO(__u64 rip, __u64 vmcb, __u64 nested_rip, __u32 int_ctl, - __u32 event_inj, bool tdp_enabled, __u32 isa), + __u32 event_inj, bool tdp_enabled, __u64 guest_tdp, + __u64 guest_cr3, __u32 isa), TP_ARGS(rip, vmcb, nested_rip, int_ctl, event_inj, tdp_enabled, - isa), + guest_tdp, guest_cr3, isa), TP_STRUCT__entry( __field( __u64, rip )
@@ -602,6 +603,7 @@ TRACE_EVENT(kvm_nested_vmenter, __field( __u32, int_ctl ) __field( __u32, event_inj ) __field( bool, tdp_enabled ) + __field( __u64, guest_pgd )__field ( __u32, isa ) ),
@@ -612,11 +614,13 @@ TRACE_EVENT(kvm_nested_vmenter, __entry->int_ctl = int_ctl; __entry->event_inj = event_inj; __entry->tdp_enabled = tdp_enabled; + __entry->guest_pgd = tdp_enabled ? guest_tdp : guest_cr3; __entry->isa = isa; ), TP_printk("rip: 0x%016llx %s: 0x%016llx nested_rip: 0x%016llx " - "int_ctl: 0x%08x event_inj: 0x%08x nested_%s: %s", + "int_ctl: 0x%08x event_inj: 0x%08x nested_%s: %s, " + "guest_pgd: 0x%016llx", __entry->rip, __entry->isa == KVM_ISA_VMX ? "vmcs" : "vmcb", __entry->vmcb,
@@ -624,7 +628,8 @@ TRACE_EVENT(kvm_nested_vmenter, __entry->int_ctl, __entry->event_inj, __entry->isa ==KVM_ISA_VMX ?"ept" :"npt" , - __entry->tdp_enabled ? "on" : "off") + __entry->tdp_enabled ?"on" :"off" , + __entry->guest_pgd) ); TRACE_EVENT(kvm_nested_intercepts,
arch/x86/kvm/vmx/nested.c +2 -0
@@ -3370,6 +3370,8 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, vmcs12->guest_intr_status, vmcs12->vm_entry_intr_info_field, vmcs12->secondary_vm_exec_control &SECONDARY_EXEC_ENABLE_EPT , + vmcs12->ept_pointer, + vmcs12->guest_cr3, KVM_ISA_VMX);kvm_service_local_tlb_flush_requests (vcpu);
Signature · 1 line
2.37.2.672.g94769d06f0-goog
Sean Christopherson
maintainerRe: [PATCH v4 3/3] KVM: x86: Print guest pgd in kvm_nested_vmenter()
arch/x86/kvm/trace.h TRACE_EVENT(kvm_pv_eoi, · line 594
Quoted 11 lines · Mingwei Zhang
---
arch/x86/kvm/trace.h +1
-1@@ -591,9 +591,10 @@ TRACE_EVENT(kvm_pv_eoi,*/ TRACE_EVENT(kvm_nested_vmenter, TP_PROTO(__u64 rip, __u64 vmcb, __u64 nested_rip, __u32 int_ctl, - __u32 event_inj, bool tdp_enabled, __u32 isa), + __u32 event_inj, bool tdp_enabled, __u64 guest_tdp,
s/guest_tdp_pgd to differentiate it from "tdp_enabled"
arch/x86/kvm/trace.h TRACE_EVENT(kvm_nested_vmenter, · line 623
Quoted 4 lines
arch/x86/kvm/trace.h +2
-1TP_printk("rip: 0x%016llx %s: 0x%016llx nested_rip: 0x%016llx " - "int_ctl: 0x%08x event_inj: 0x%08x nested_%s: %s", + "int_ctl: 0x%08x event_inj: 0x%08x nested_%s: %s, " + "guest_pgd: 0x%016llx",
It's a little gross, but this can spit out nested_eptp vs. nested_cr3 vs. guest_cr3.
arch/x86/kvm/trace.h TRACE_EVENT(kvm_nested_vmenter, · line 631
Quoted 10 lines
arch/x86/kvm/trace.h +1
-1__entry->rip, __entry->isa ==KVM_ISA_VMX ?"vmcs" :"vmcb" , __entry->vmcb,@@ -624,7 +628,8 @@ TRACE_EVENT(kvm_nested_vmenter, __entry->int_ctl, __entry->event_inj, __entry->isa ==KVM_ISA_VMX ?"ept" :"npt" , - __entry->tdp_enabled ? "on" : "off") + __entry->tdp_enabled ? "on" : "off",
To keep things aligned, and because "on nested_eptp" reads as a combined snippet
event_inj: 0x00000000 nested_ept: off guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept: off guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept: on nested_eptp: 0x0000000007ec501e
event_inj: 0x00000000 nested_ept: on nested_eptp: 0x0000000007ec501e
what about teaking the format so that the output looks like this?
event_inj: 0x00000000 nested_ept=n guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept=n guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept=y nested_eptp: 0x0000000007ec501e
event_inj: 0x00000000 nested_ept=y nested_eptp: 0x0000000007ec501e
Deviating from the ": " style bothers me, but I find this difficult to read. Again, letters delimited by whitespace get visually clumped together.
event_inj: 0x00000000 nested_ept: n guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept: n guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept: y nested_eptp: 0x0000000007ec501e
event_inj: 0x00000000 nested_ept: y nested_eptp: 0x0000000007ec501e
And this looks like a typo
event_inj: 0x00000000 nested_ept:n guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept:n guest_cr3: 0x0000000001007000
event_inj: 0x00000000 nested_ept:y nested_eptp: 0x0000000007ec501e
event_inj: 0x00000000 nested_ept:y nested_eptp: 0x0000000007ec501e
We could leave off the nested_{ept/npt} entirely and leave the differentation to the nested_eptp vs. nested_cr3 vs. guest_cr3, but I don't think that's worth shaving a few chars.
Quoted 2 lines · Mingwei Zhang
This patch set update
trace_kvm_nested_vmrun()to support VMX. In addition, add theguest_pgdfield to enrich the trace information.
...
Quoted 7 lines
David Matlack (1):
KVM: nVMX: Add tracepoint for nested vmenterMingwei Zhang (2):
KVM: x86: Update trace function for nested VM entry to support VMX
KVM: x86: Print guest pgd inkvm_nested_vmenter()
Quoted 5 lines
arch/x86/kvm/svm/nested.c | 8 ++++++-- arch/x86/kvm/trace.h | 33 +++++++++++++++++++++++---------- arch/x86/kvm/vmx/nested.c | 10 ++++++++++ arch/x86/kvm/x86.c | 2 +- 4 files changed, 40 insertions(+), 13 deletions(-)
Some nits that I fixed up when applying (will respond to individual patches).
Pushed to branch for_paolo/6.1 at:
https://github.com/sean-jc/linux.git
Unless you hear otherwise, it will make its way the official KVM tree "soon".
Note, the commit IDs are not guaranteed to be stable.