kvm: Improving directed yield in PLE handler
v2 cover kvm: Improving directed yield in PLE handler
Currently Pause Looop Exit (PLE) handler is doing directed yield to a random VCPU on PL exit. Though we already have filtering while choosing the candidate to yield_to, we can do better.
Problem is, for large vcpu guests, we have more probability of yielding to a bad vcpu. We are not able to prevent directed yield to same guy who has done PL exit recently, who perhaps spins again and wastes CPU.
Fix that by keeping track of who has done PL exit. So The Algorithm in series give chance to a VCPU which has:
(a) Not done PLE exit at all (probably he is preempted lock-holder)
(b) VCPU skipped in last iteration because it did PL exit, and probably has become eligible now (next eligible lock holder)
Future enhancemnets:
(1) Currently we have a boolean to decide on eligibility of vcpu. It would be nice if I get feedback on guest (>32 vcpu) whether we can improve better with integer counter. (with counter = say f(log n )).
(2) We have not considered system load during iteration of vcpu. With that information we can limit the scan and also decide whether schedule() is better. [ I am able to use #kicked vcpus to decide on this But may be there are better ideas like information from global loadavg.]
(3) We can exploit this further with PV patches since it also knows about next eligible lock-holder.
Changes since V1:
- Add more documentation for structure and algorithm and Rename plo ==> ple (Rik).
- change dy_eligible initial value to false. (otherwise very first directed yield will not be skipped. (Nikunj)
- fixup signoff/from issue
Summary: There is a very good improvement for moderate / no overcommit scenario
for kvm based guest on PLE machine (which is difficult ;) ).
kernbench sysbench ebizzy
1x 28 % -0.04 % 105 %
2x 7 % 0.83 % 26 %
---
Link for V1: (It also has result)
https://lkml.org/lkml/2012/7/9/32
Raghavendra K T (2):
kvm vcpu: Note down pause loop exit
kvm PLE handler: Choose better candidate for directed yield
arch/s390/include/asm/kvm_host.h | 5 +++++ arch/x86/include/asm/kvm_host.h | 13 ++++++++++++- arch/x86/kvm/svm.c | 1 + arch/x86/kvm/vmx.c | 1 + arch/x86/kvm/x86.c | 34 +++++++++++++++++++++++++++++++++- virt/kvm/kvm_main.c | 3 +++ 6 files changed, 55 insertions(+), 2 deletions(-)
RFC PATCH v2 1/2 kvm vcpu: Note down pause loop exit
From: Raghavendra K T <[email protected]>
Noting pause loop exited vcpu helps in filtering right candidate to yield. Yielding to same vcpu may result in more wastage of cpu.
---
arch/x86/include/asm/kvm_host.h | 11 +++++++++++ arch/x86/kvm/svm.c | 1 + arch/x86/kvm/vmx.c | 1 + arch/x86/kvm/x86.c | 4 +++- 4 files changed, 16 insertions(+), 1 deletions(-)
arch/x86/include/asm/kvm_host.h +11 -0
@@ -484,6 +484,17 @@ struct kvm_vcpu_arch {u64 length;u64 status; } osvw; + +/* +* Pause loop exit optimization +* pause_loop_exited: set when a vcpu does a pause loop exit. +* dy_eligible: indicates whether vcpu is eligible for directed yield. +*/ +struct { +bool pause_loop_exited; +bool dy_eligible; + } ple; + };struct kvm_lpage_info {
arch/x86/kvm/svm.c +1 -0
@@ -3264,6 +3264,7 @@ static int interrupt_window_interception(struct vcpu_svm *svm)static int pause_interception (struct vcpu_svm *svm) { + svm->vcpu.arch.ple.pause_loop_exited = true;kvm_vcpu_on_spin (&(svm->vcpu));return 1 ; }
arch/x86/kvm/vmx.c +1 -0
@@ -4945,6 +4945,7 @@ out:static int handle_pause (struct kvm_vcpu *vcpu) {skip_emulated_instruction (vcpu); + vcpu->arch.ple.pause_loop_exited = true;kvm_vcpu_on_spin (vcpu);return 1 ;
arch/x86/kvm/x86.c +3 -1
@@ -5331,7 +5331,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (req_immediate_exit)smp_send_reschedule (vcpu->cpu); - + vcpu->arch.ple.pause_loop_exited = false;kvm_guest_enter ();if (unlikely (vcpu->arch.switch_db_regs)) {
@@ -6168,6 +6168,8 @@ int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)BUG_ON (vcpu->kvm ==NULL ); kvm = vcpu->kvm; + vcpu->arch.ple.pause_loop_exited = false; + vcpu->arch.ple.dy_eligible = false; vcpu->arch.emulate_ctxt.ops = &emulate_ops;if (!irqchip_in_kernel (kvm) ||kvm_vcpu_is_bsp (vcpu)) vcpu->arch.mp_state =KVM_MP_STATE_RUNNABLE ;
Re: [PATCH RFC V2 1/2] kvm vcpu: Note down pause loop exit
Quoted 6 lines · Raghavendra K T
From: Raghavendra K T<[email protected]>
Noting pause loop exited vcpu helps in filtering right candidate to yield. Yielding to same vcpu may result in more wastage of cpu.
Signed-off-by Raghavendra K T<[email protected]>
Signature · 1 line
All rights reversed
RFC PATCH v2 2/2 kvm PLE handler: Choose better candidate for directed yield
In reply to Raghavendra K TFrom: Raghavendra K T <[email protected]>
Currently PLE handler can repeatedly do a directed yield to same vcpu that has recently done PL exit. This can degrade the performance.
Try to yield to most eligible guy instead by alternate yielding. Precisely, give chance to a VCPU which has:
(a) Not done PLE exit at all (probably he is preempted lock-holder)
(b) VCPU skipped in last iteration because it did PL exit, and probably has become eligible now (next eligible lock holder)
---
arch/s390/include/asm/kvm_host.h | 5 +++++ arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/x86.c | 30 ++++++++++++++++++++++++++++++ virt/kvm/kvm_main.c | 3 +++ 4 files changed, 39 insertions(+), 1 deletions(-)
arch/s390/include/asm/kvm_host.h +5 -0
@@ -256,5 +256,10 @@ struct kvm_arch{ struct gmap *gmap; }; +static inline bool kvm_arch_vcpu_check_and_update_eligible (struct kvm_vcpu *v) +{ +return true; +} +extern int sie64a (struct kvm_s390_sie_block *,u64 *);#endif
arch/x86/include/asm/kvm_host.h +1 -1
@@ -966,7 +966,7 @@ extern bool kvm_find_async_pf_gfn(struct kvm_vcpu *vcpu, gfn_t gfn);void kvm_complete_insn_gp (struct kvm_vcpu *vcpu,int err);int kvm_is_in_guest (void ); - +bool kvm_arch_vcpu_check_and_update_eligible (struct kvm_vcpu *vcpu);void kvm_pmu_init (struct kvm_vcpu *vcpu);void kvm_pmu_destroy (struct kvm_vcpu *vcpu);void kvm_pmu_reset (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +30 -0
@@ -6623,6 +6623,36 @@ bool kvm_arch_can_inject_async_page_present(struct kvm_vcpu *vcpu) kvm_x86_ops->interrupt_allowed (vcpu); } +/* +* Helper that checks whether a VCPU is eligible for directed yield. +* Most eligible candidate to yield is decided by following heuristics: +* +* (a) VCPU which has not done PL exit recently (preempted lock holder), +* indicated by @pause_loop_exited. Cleared just before guest_enter() +* +* (b) VCPU which has done PL exit but did not get chance last time. +* (mostly it has become eligible now since we have probably +* yielded to lockholder in last iteration. This is done by toggling +* @dy_eligible each time a VCPU checked for eligibility.) +* +* Yielding to a recently PL exited VCPU before yielding to preempted +* lock-holder results in uneligible VCPU burning CPU. Giving priority +* for a potential lock-holder increases lock progress chance. +*/ +bool kvm_arch_vcpu_check_and_update_eligible (struct kvm_vcpu *vcpu) +{ +bool eligible; + + eligible = !vcpu->arch.ple.pause_loop_exited || + (vcpu->arch.ple.pause_loop_exited && + vcpu->arch.ple.dy_eligible); + +if (vcpu->arch.ple.pause_loop_exited) + vcpu->arch.ple.dy_eligible = !vcpu->arch.ple.dy_eligible; + +return eligible; +} +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_inj_virq);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_page_fault);
virt/kvm/kvm_main.c +3 -0
@@ -1595,6 +1595,9 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)continue ;if (waitqueue_active (&vcpu->wq))continue ; +if (!kvm_arch_vcpu_check_and_update_eligible (vcpu)) { +continue ; + }if (kvm_vcpu_yield_to (vcpu)) { kvm->last_boosted_vcpu = i; yielded =1 ;
Re: [PATCH RFC V2 2/2] kvm PLE handler: Choose better candidate for directed yield
Quoted 5 lines · Raghavendra K T
From: Raghavendra K T<[email protected]>
Currently PLE handler can repeatedly do a directed yield to same vcpu that has recently done PL exit. This can degrade the performance.
Quoted 7 lines
Try to yield to most eligible guy instead by alternate yielding. Precisely, give chance to a VCPU which has:
(a) Not done PLE exit at all (probably he is preempted lock-holder)
(b) VCPU skipped in last iteration because it did PL exit, and probably has become eligible now (next eligible lock holder)Signed-off-by Raghavendra K T<[email protected]>
Signature · 1 line
All rights reversed
Re: [PATCH RFC V2 2/2] kvm PLE handler: Choose better candidate for directed yield
In reply to Raghavendra K Tvirt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1600
Quoted 9 lines · Raghavendra K T
virt/kvm/kvm_main.c +3
-0@@ -1595,6 +1595,9 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)continue ;if (waitqueue_active (&vcpu->wq))continue ; +if (!kvm_arch_vcpu_check_and_update_eligible (vcpu)) { +continue ; + }
Saw this fly by for a second time and now it annoyed me enough to mention it: those curly braces are superfluous and should be dropped.
Thanks,
Ingo
Re: [PATCH RFC V2 2/2] kvm PLE handler: Choose better candidate for directed yield
Quoted 17 lines · Ingo Molnar
virt/kvm/kvm_main.c void kvm_vcpu_on_spin(struct kvm_vcpu *me) · line 1600
Quoted 9 lines · Raghavendra K T
virt/kvm/kvm_main.c +3
-0@@ -1595,6 +1595,9 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me)continue ;if (waitqueue_active (&vcpu->wq))continue ; +if (!kvm_arch_vcpu_check_and_update_eligible (vcpu)) { +continue ; + }Saw this fly by for a second time and now it annoyed me enough to mention it: those curly braces are superfluous and should be dropped.
Oops! Sorry.will remove {} and resend with Rik's reviewed-by. TBH, block got simplified after iterations (local), this one leftout. my mistake.