KVM: X86: Some tracepoint enhancements
v2 cover KVM: X86: Some tracepoint enhancements
v2:
- fix commit messages, change format of ple window tracepoint [Sean]
- rebase [Wanpeng]
Each small patch explains itself. I noticed them when I'm tracing some IRQ paths and I found them helpful at least to me.
Please have a look. Thanks,
Peter Xu (3):
KVM: X86: Trace vcpu_id for vmexit
KVM: X86: Remove tailing newline for tracepoints
KVM: X86: Tune PLE Window tracepoint
arch/x86/kvm/svm.c | 16 ++++++++-------- arch/x86/kvm/trace.h | 30 ++++++++++++------------------ arch/x86/kvm/vmx/vmx.c | 14 ++++++++------ arch/x86/kvm/x86.c | 2 +- 4 files changed, 29 insertions(+), 33 deletions(-)
Signature · 1 line
2.21.0
PATCH v2 1/3 KVM: X86: Trace vcpu_id for vmexit
Tracing the ID helps to pair vmenters and vmexits for guests with multiple vCPUs.
--- arch/x86/kvm/trace.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/kvm/trace.h +4 -1
@@ -232,17 +232,20 @@ TRACE_EVENT(kvm_exit, __field( u32, isa ) __field( u64, info1 ) __field( u64, info2 ) +__field ( int, vcpu_id ) ),TP_fast_assign ( __entry->exit_reason = exit_reason; __entry->guest_rip =kvm_rip_read (vcpu); __entry->isa = isa; + __entry->vcpu_id = vcpu->vcpu_id; kvm_x86_ops->get_exit_info (vcpu, &__entry->info1, &__entry->info2); ), - TP_printk("reason %s rip 0x%lx info %llx %llx", + TP_printk("vcpu %d reason %s rip 0x%lx info %llx %llx", + __entry->vcpu_id, (__entry->isa == KVM_ISA_VMX) ? __print_symbolic(__entry->exit_reason, VMX_EXIT_REASONS) : __print_symbolic(__entry->exit_reason, SVM_EXIT_REASONS),
Signature · 1 line
2.21.0
Re: [PATCH v2 1/3] KVM: X86: Trace vcpu_id for vmexit
Quoted 14 lines · Peter Xu
Tracing the ID helps to pair vmenters and vmexits for guests with multiple vCPUs.
Reviewed-by Krish Sadhukhan <[email protected]>Signed-off-by Peter Xu <[email protected]>--- arch/x86/kvm/trace.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)arch/x86/kvm/trace.h +0
-0
arch/x86/kvm/trace.h TRACE_EVENT(kvm_exit, · line 235
Quoted 4 lines
arch/x86/kvm/trace.h +1
-0@@ -232,17 +232,20 @@ TRACE_EVENT(kvm_exit, __field( u32, isa ) __field( u64, info1 ) __field( u64, info2 ) +__field ( int, vcpu_id )
It doesn't actually affect anything, but vcpu_id is stored and printed as an 'unsigned int' everywhere else in the trace code. Stylistically I like that approach even though struct kvm_vcpu holds it as a signed int.
Quoted 20 lines
arch/x86/kvm/trace.h +3
-1), TP_fast_assign( __entry->exit_reason = exit_reason; __entry->guest_rip =kvm_rip_read (vcpu); __entry->isa = isa; + __entry->vcpu_id = vcpu->vcpu_id; kvm_x86_ops->get_exit_info (vcpu, &__entry->info1, &__entry->info2); ), - TP_printk("reason %s rip 0x%lx info %llx %llx", + TP_printk("vcpu %d reason %s rip 0x%lx info %llx %llx", + __entry->vcpu_id, (__entry->isa == KVM_ISA_VMX) ? __print_symbolic(__entry->exit_reason, VMX_EXIT_REASONS) : __print_symbolic(__entry->exit_reason, SVM_EXIT_REASONS),Signature · 1 line
2.21.0
Re: [PATCH v2 1/3] KVM: X86: Trace vcpu_id for vmexit
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Peter Xu
Tracing the ID helps to pair vmenters and vmexits for guests with multiple vCPUs.
Reviewed-by Krish Sadhukhan <[email protected]>Signed-off-by Peter Xu <[email protected]>--- arch/x86/kvm/trace.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Quoted 13 lines
arch/x86/kvm/trace.h TRACE_EVENT(kvm_exit, · line 235
Quoted 9 lines
arch/x86/kvm/trace.h +1
-0@@ -232,17 +232,20 @@ TRACE_EVENT(kvm_exit, __field( u32, isa ) __field( u64, info1 ) __field( u64, info2 ) +__field ( int, vcpu_id )It doesn't actually affect anything, but
vcpu_idis stored and printed as an 'unsigned int' everywhere else in the trace code. Stylistically I like that approach even though structkvm_vcpuholds it as a signed int.
True. I can switch to unsigned int to get aligned with the rest if there's other comment to address. Though from codebase-wise I would even prefer signed because it gives us a chance to set an invalid vcpu id (-1) where necessary, or notice something severly wrong when <-1. After all it should far cover our usage (IIUC max vcpu id should be 512 cross archs).
Thanks,
Signature · 1 line
Peter Xu
Re: [PATCH v2 1/3] KVM: X86: Trace vcpu_id for vmexit
Quoted 25 lines · Peter Xu
Quoted 10 lines · Sean Christopherson
Quoted 9 lines · Peter Xu
Tracing the ID helps to pair vmenters and vmexits for guests with multiple vCPUs.
Reviewed-by Krish Sadhukhan <[email protected]>Signed-off-by Peter Xu <[email protected]>--- arch/x86/kvm/trace.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)Quoted 13 lines
arch/x86/kvm/trace.h TRACE_EVENT(kvm_exit, · line 235
Quoted 9 lines
arch/x86/kvm/trace.h +1
-0@@ -232,17 +232,20 @@ TRACE_EVENT(kvm_exit, __field( u32, isa ) __field( u64, info1 ) __field( u64, info2 ) +__field ( int, vcpu_id )It doesn't actually affect anything, but
vcpu_idis stored and printed as an 'unsigned int' everywhere else in the trace code. Stylistically I like that approach even though structkvm_vcpuholds it as a signed int.
Quoted 4 lines
True. I can switch to unsigned int to get aligned with the rest if there's other comment to address. Though from codebase-wise I would even prefer signed because it gives us a chance to set an invalid vcpu id (-1) where necessary, or notice something severly wrong when <-1.
I agree that a signed int makes sense for flows like kvm_get_vcpu_by_id(), where the incoming id isn't sanitized. But for struct kvm_vcpu, a negative vcpu_id is simply impossible, e.g. vcpu_id is set to a postive (and capped) value as part of the core vcpu initialization and is never changed. I prefer storing an unsigned val in the tracing as it communicates that vcpu_id is always valid. I suspect struct kvm_vcpu uses a signed int purely to avoid having to constantly cast it to an int.
Quoted 2 lines
After all it should far cover our usage (IIUC max vcpu id should be 512 cross archs).
PATCH v2 2/3 KVM: X86: Remove tailing newline for tracepoints
In reply to Peter XuIt's done by TP_printk() already.
--- arch/x86/kvm/trace.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/trace.h +2 -2
@@ -1323,7 +1323,7 @@ TRACE_EVENT(kvm_avic_incomplete_ipi, __entry->index = index; ), -TP_printk ("vcpu=%u, icrh:icrl=%#010x:%08x, id=%u, index=%u\n" , +TP_printk ("vcpu=%u, icrh:icrl=%#010x:%08x, id=%u, index=%u" , __entry->vcpu, __entry->icrh, __entry->icrl, __entry->id, __entry->index) );
@@ -1348,7 +1348,7 @@ TRACE_EVENT(kvm_avic_unaccelerated_access, __entry->vec = vec; ), - TP_printk("vcpu=%u, offset=%#x(%s), %s, %s, vec=%#x\n", + TP_printk("vcpu=%u, offset=%#x(%s), %s, %s, vec=%#x", __entry->vcpu, __entry->offset, __print_symbolic(__entry->offset, kvm_trace_symbol_apic),
Signature · 1 line
2.21.0
Re: [PATCH v2 2/3] KVM: X86: Remove tailing newline for tracepoints
Quoted 5 lines · Peter Xu
It's done by
TP_printk()already.Reviewed-by Krish Sadhukhan <[email protected]>Signed-off-by Peter Xu <[email protected]>---
PATCH v2 3/3 KVM: X86: Tune PLE Window tracepoint
In reply to Peter XuThe PLE window tracepoint triggers even if the window is not changed, and the wording can be a bit confusing too. One example line:
kvm_ple_window: vcpu 0: ple_window 4096 (shrink 4096)
It easily let people think of "the window now is 4096 which is shrinked", but the truth is the value actually didn't change (4096).
Let's only dump this message if the value really changed, and we make the message even simpler like:
kvm_ple_window: vcpu 4 old 4096 new 8192 (growed)
--- arch/x86/kvm/svm.c | 16 ++++++++-------- arch/x86/kvm/trace.h | 21 ++++++--------------- arch/x86/kvm/vmx/vmx.c | 14 ++++++++------ arch/x86/kvm/x86.c | 2 +- 4 files changed, 23 insertions(+), 30 deletions(-)
arch/x86/kvm/svm.c +8 -8
@@ -1269,11 +1269,11 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) pause_filter_count_grow, pause_filter_count_max); -if (control->pause_filter_count != old) +if (control->pause_filter_count != old) {mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - -trace_kvm_ple_window_grow (vcpu->vcpu_id, - control->pause_filter_count, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + control->pause_filter_count, old); + } }static void shrink_ple_window (struct kvm_vcpu *vcpu)
@@ -1287,11 +1287,11 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) pause_filter_count, pause_filter_count_shrink, pause_filter_count); -if (control->pause_filter_count != old) +if (control->pause_filter_count != old) {mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - -trace_kvm_ple_window_shrink (vcpu->vcpu_id, - control->pause_filter_count, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + control->pause_filter_count, old); + } }static __init int svm_hardware_setup (void )
arch/x86/kvm/trace.h +6 -15
@@ -890,36 +890,27 @@ TRACE_EVENT(kvm_pml_full,TP_printk ("vcpu %d: PML full" , __entry->vcpu_id) ); -TRACE_EVENT (kvm_ple_window, - TP_PROTO(bool grow, unsigned int vcpu_id, int new, int old), -TP_ARGS (grow, vcpu_id, new, old), +TRACE_EVENT (kvm_ple_window_update, + TP_PROTO(unsigned int vcpu_id, int new, int old), +TP_ARGS (vcpu_id, new, old),TP_STRUCT__entry ( - __field( bool, grow ) __field( unsigned int, vcpu_id ) __field( int, new )__field ( int, old ) ),TP_fast_assign ( - __entry->grow = grow; __entry->vcpu_id = vcpu_id; __entry->new = new; __entry->old = old; ), -TP_printk ("vcpu %u: ple_window %d (%s %d)" , - __entry->vcpu_id, - __entry->new, - __entry->grow ?"grow" :"shrink" , - __entry->old) +TP_printk ("vcpu %u old %d new %d (%s)" , + __entry->vcpu_id, __entry->old, __entry->new, + __entry->old < __entry->new ?"growed" :"shrinked" ) ); -#define trace_kvm_ple_window_grow (vcpu_id, new, old) \ - trace_kvm_ple_window(true, vcpu_id, new, old) -#define trace_kvm_ple_window_shrink (vcpu_id, new, old) \ - trace_kvm_ple_window(false, vcpu_id, new, old) - TRACE_EVENT(kvm_pvclock_update, TP_PROTO(unsigned int vcpu_id, struct pvclock_vcpu_time_info *pvclock), TP_ARGS(vcpu_id, pvclock),
arch/x86/kvm/vmx/vmx.c +8 -6
@@ -5233,10 +5233,11 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) ple_window_grow, ple_window_max); -if (vmx->ple_window != old) +if (vmx->ple_window != old) { vmx->ple_window_dirty = true; - -trace_kvm_ple_window_grow (vcpu->vcpu_id, vmx->ple_window, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + vmx->ple_window, old); + } }static void shrink_ple_window (struct kvm_vcpu *vcpu)
@@ -5248,10 +5249,11 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) ple_window_shrink, ple_window); -if (vmx->ple_window != old) +if (vmx->ple_window != old) { vmx->ple_window_dirty = true; - -trace_kvm_ple_window_shrink (vcpu->vcpu_id, vmx->ple_window, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + vmx->ple_window, old); + } }/*
arch/x86/kvm/x86.c +1 -1
@@ -10082,7 +10082,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_skinit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_intercepts);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_write_tsc_offset); -EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_ple_window); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_ple_window_update);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_pml_full);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_pi_irte_update);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_unaccelerated_access);
Signature · 1 line
2.21.0
Re: [PATCH v2 3/3] KVM: X86: Tune PLE Window tracepoint
Quoted 67 lines · Peter Xu
The PLE window tracepoint triggers even if the window is not changed, and the wording can be a bit confusing too. One example line:
kvm_ple_window: vcpu 0:ple_window4096 (shrink 4096)It easily let people think of "the window now is 4096 which is shrinked", but the truth is the value actually didn't change (4096).
Let's only dump this message if the value really changed, and we make the message even simpler like:
kvm_ple_window: vcpu 4 old 4096 new 8192 (growed)Signed-off-by Peter Xu <[email protected]>--- arch/x86/kvm/svm.c | 16 ++++++++-------- arch/x86/kvm/trace.h | 21 ++++++--------------- arch/x86/kvm/vmx/vmx.c | 14 ++++++++------ arch/x86/kvm/x86.c | 2 +- 4 files changed, 23 insertions(+), 30 deletions(-)arch/x86/kvm/svm.c +8
-8@@ -1269,11 +1269,11 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) pause_filter_count_grow, pause_filter_count_max); -if (control->pause_filter_count != old) +if (control->pause_filter_count != old) {mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - -trace_kvm_ple_window_grow (vcpu->vcpu_id, - control->pause_filter_count, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + control->pause_filter_count, old); + } }static void shrink_ple_window (struct kvm_vcpu *vcpu)@@ -1287,11 +1287,11 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) pause_filter_count, pause_filter_count_shrink, pause_filter_count); -if (control->pause_filter_count != old) +if (control->pause_filter_count != old) {mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - -trace_kvm_ple_window_shrink (vcpu->vcpu_id, - control->pause_filter_count, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + control->pause_filter_count, old); + } }static __init int svm_hardware_setup (void )arch/x86/kvm/trace.h +0
-2@@ -890,36 +890,27 @@ TRACE_EVENT(kvm_pml_full,TP_printk ("vcpu %d: PML full" , __entry->vcpu_id) ); -TRACE_EVENT(kvm_ple_window, - TP_PROTO(bool grow, unsigned int vcpu_id, int new, int old),
arch/x86/kvm/trace.h TRACE_EVENT(kvm_pml_full, · line 900
Quoted 10 lines
arch/x86/kvm/trace.h +3
-2- TP_ARGS(grow, vcpu_id, new, old), +TRACE_EVENT(kvm_ple_window_update, + TP_PROTO(unsigned int vcpu_id, int new, int old), + TP_ARGS(vcpu_id, new, old), TP_STRUCT__entry( -__field (bool ,grow ) __field( unsigned int, vcpu_id ) __field( int, new ) __field( int, old )
Not your code, but these should really be 'unsigned int', especially now that they are directly compared when printing "growed" versus "shrinked". For SVM it doesn't matter since the underlying hardware fields are only 16 bits, but on VMX they're 32 bits, e.g. theoretically userspace could set ple_window and ple_window_max to a negative value.
The ple_window variable in struct vcpu_vmx and local snapshots of the field should also be updated, but that can be done separately.
Quoted 77 lines
arch/x86/kvm/trace.h +3
-11), TP_fast_assign( - __entry->grow = grow; __entry->vcpu_id = vcpu_id; __entry->new = new; __entry->old = old; ), -TP_printk ("vcpu %u: ple_window %d (%s %d)" , - __entry->vcpu_id, - __entry->new, - __entry->grow ?"grow" :"shrink" , - __entry->old) +TP_printk ("vcpu %u old %d new %d (%s)" , + __entry->vcpu_id, __entry->old, __entry->new, + __entry->old < __entry->new ?"growed" :"shrinked" ) ); -#define trace_kvm_ple_window_grow (vcpu_id, new, old) \ - trace_kvm_ple_window(true, vcpu_id, new, old) -#define trace_kvm_ple_window_shrink (vcpu_id, new, old) \ - trace_kvm_ple_window(false, vcpu_id, new, old) - TRACE_EVENT(kvm_pvclock_update, TP_PROTO(unsigned int vcpu_id, struct pvclock_vcpu_time_info *pvclock), TP_ARGS(vcpu_id, pvclock),arch/x86/kvm/vmx/vmx.c +8
-6@@ -5233,10 +5233,11 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) ple_window_grow, ple_window_max); -if (vmx->ple_window != old) +if (vmx->ple_window != old) { vmx->ple_window_dirty = true; - -trace_kvm_ple_window_grow (vcpu->vcpu_id, vmx->ple_window, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + vmx->ple_window, old); + } }static void shrink_ple_window (struct kvm_vcpu *vcpu)@@ -5248,10 +5249,11 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) ple_window_shrink, ple_window); -if (vmx->ple_window != old) +if (vmx->ple_window != old) { vmx->ple_window_dirty = true; - -trace_kvm_ple_window_shrink (vcpu->vcpu_id, vmx->ple_window, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + vmx->ple_window, old); + } }/* arch/x86/kvm/x86.c +1
-1@@ -10082,7 +10082,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_invlpga);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_skinit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_nested_intercepts);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_write_tsc_offset); -EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_ple_window); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_ple_window_update);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_pml_full);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_pi_irte_update);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_unaccelerated_access);Signature · 1 line
2.21.0
Re: [PATCH v2 3/3] KVM: X86: Tune PLE Window tracepoint
Quoted 85 lines · Sean Christopherson
Quoted 67 lines · Peter Xu
The PLE window tracepoint triggers even if the window is not changed, and the wording can be a bit confusing too. One example line:
kvm_ple_window: vcpu 0:ple_window4096 (shrink 4096)It easily let people think of "the window now is 4096 which is shrinked", but the truth is the value actually didn't change (4096).
Let's only dump this message if the value really changed, and we make the message even simpler like:
kvm_ple_window: vcpu 4 old 4096 new 8192 (growed)Signed-off-by Peter Xu <[email protected]>--- arch/x86/kvm/svm.c | 16 ++++++++-------- arch/x86/kvm/trace.h | 21 ++++++--------------- arch/x86/kvm/vmx/vmx.c | 14 ++++++++------ arch/x86/kvm/x86.c | 2 +- 4 files changed, 23 insertions(+), 30 deletions(-)arch/x86/kvm/svm.c +8
-8@@ -1269,11 +1269,11 @@ static void grow_ple_window(struct kvm_vcpu *vcpu) pause_filter_count_grow, pause_filter_count_max); -if (control->pause_filter_count != old) +if (control->pause_filter_count != old) {mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - -trace_kvm_ple_window_grow (vcpu->vcpu_id, - control->pause_filter_count, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + control->pause_filter_count, old); + } }static void shrink_ple_window (struct kvm_vcpu *vcpu)@@ -1287,11 +1287,11 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu) pause_filter_count, pause_filter_count_shrink, pause_filter_count); -if (control->pause_filter_count != old) +if (control->pause_filter_count != old) {mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - -trace_kvm_ple_window_shrink (vcpu->vcpu_id, - control->pause_filter_count, old); +trace_kvm_ple_window_update (vcpu->vcpu_id, + control->pause_filter_count, old); + } }static __init int svm_hardware_setup (void )arch/x86/kvm/trace.h +0
-2@@ -890,36 +890,27 @@ TRACE_EVENT(kvm_pml_full,TP_printk ("vcpu %d: PML full" , __entry->vcpu_id) ); -TRACE_EVENT(kvm_ple_window, - TP_PROTO(bool grow, unsigned int vcpu_id, int new, int old),arch/x86/kvm/trace.h TRACE_EVENT(kvm_pml_full, · line 900
Quoted 10 lines
arch/x86/kvm/trace.h +3
-2- TP_ARGS(grow, vcpu_id, new, old), +TRACE_EVENT(kvm_ple_window_update, + TP_PROTO(unsigned int vcpu_id, int new, int old), + TP_ARGS(vcpu_id, new, old), TP_STRUCT__entry( -__field (bool ,grow ) __field( unsigned int, vcpu_id ) __field( int, new ) __field( int, old )Not your code, but these should really be 'unsigned int', especially now that they are directly compared when printing "growed" versus "shrinked". For SVM it doesn't matter since the underlying hardware fields are only 16 bits, but on VMX they're 32 bits, e.g. theoretically userspace could set
ple_windowandple_window_maxto a negative value.
Quoted 2 lines
The
ple_windowvariable in structvcpu_vmxand local snapshots of the field should also be updated, but that can be done separately.
Indeed. Let me add a separated patch. Thanks,
Signature · 1 line
Peter Xu