Skip to contentSkip to conversation

KVM:VMX: Add support for Pause-Loop Exiting

PATCH Superseded Zhai, Edwin 2 participants 2 messages Jul 8, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM:VMX: Add support for Pause-Loop Exiting

KVM:VMX: Add support for Pause-Loop Exiting

New NHM processors will support Pause-Loop Exiting by adding 2 VM-execution control fields:
PLE_Gap - upper bound on the amount of time between two successive
executions of PAUSE in a loop.
PLE_Window - upper bound on the amount of time a guest is allowed to execute in
a PAUSE loop

If the time, between this execution of PAUSE and previous one, exceeds the PLE_Gap, processor consider this PAUSE belongs to a new loop. Otherwise, processor determins the the total execution time of this loop(since 1st PAUSE in this loop), and triggers a VM exit if total time exceeds the PLE_Window.
* Refer SDM volume 3b section 21.6.13 & 22.1.3.

Pause-Loop Exiting can be used to detect Lock-Holder Preemption, where one VP is sched-out after hold a spinlock, then other VPs for same lock are sched-in to waste the CPU time.

Our tests indicate that most spinlocks are held for less than 2^12 cycles. Performance tests show that with 2X LP over-commitment we can get +2% perf improvement for kernel build(Even more perf gain with more LPs).

----
include/asm/vmx.h | 4 ++++
kvm/vmx.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++-

2 files changed, 54 insertions(+), 1 deletion(-)
arch/x86/include/asm/vmx.h +4 -0
Index: linux-2.6/arch/x86/include/asm/vmx.h
===================================================================
@@ -56,6 +56,7 @@
 #define SECONDARY_EXEC_ENABLE_VPID              0x00000020
 #define SECONDARY_EXEC_WBINVD_EXITING		0x00000040
 #define SECONDARY_EXEC_UNRESTRICTED_GUEST	0x00000080
+#define SECONDARY_EXEC_PAUSE_LOOP_EXITING	0x00000400
 
 
 #define PIN_BASED_EXT_INTR_MASK                 0x00000001
@@ -144,6 +145,8 @@ enum vmcs_field {
 	VM_ENTRY_INSTRUCTION_LEN        = 0x0000401a,
 	TPR_THRESHOLD                   = 0x0000401c,
 	SECONDARY_VM_EXEC_CONTROL       = 0x0000401e,
+	PLE_GAP                         = 0x00004020,
+	PLE_WINDOW                      = 0x00004022,
 	VM_INSTRUCTION_ERROR            = 0x00004400,
 	VM_EXIT_REASON                  = 0x00004402,
 	VM_EXIT_INTR_INFO               = 0x00004404,
@@ -248,6 +251,7 @@ enum vmcs_field {
 #define EXIT_REASON_MSR_READ            31
 #define EXIT_REASON_MSR_WRITE           32
 #define EXIT_REASON_MWAIT_INSTRUCTION   36
+#define EXIT_REASON_PAUSE_INSTRUCTION   40
 #define EXIT_REASON_MCE_DURING_VMENTRY	 41
 #define EXIT_REASON_TPR_BELOW_THRESHOLD 43
 #define EXIT_REASON_APIC_ACCESS         44
arch/x86/kvm/vmx.c +50 -1

96 lines not on this page · raw message

#1 lore raw
Quoted 6 lines · Edwin

/*
+ * Indicate a busy-waiting vcpu in spinlock. We do not enable the PAUSE
+ * exiting, so only get here on cpu with PAUSE-Loop-Exiting.
+ */

arch/x86/kvm/vmx.c static void handle_invalid_guest_state(s · line 3400

Quoted 8 lines
arch/x86/kvm/vmx.c +8 -0
+static int handle_pause(struct kvm_vcpu *vcpu,
+				struct kvm_run *kvm_run)
+{
+	skip_emulated_instruction(vcpu);
+	yield();
+	return 1;
+}
+

yield() does nothing in Linux. Please look at http://lkml.org/lkml/2009/5/11/294 for what needs to be added to the scheduler (that message is part of a thread discussing the same feature for svm).

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.