Skip to contentSkip to conversation

Support Pause Filter in AMD processors

PATCH In review Mark Langsdorf 3 participants 4 messages Oct 2, 2009 – Oct 12, 2009

Subsystems (3)
x86 KVM KVM x86
lore
Share & tools
lore

Mark Langsdorf

author
#0 lore raw

PATCH 1/1 Support Pause Filter in AMD processors

Quoted 1 line

From 66741f741da741e58e8162ef7809dd7d6f8e01cf Mon Sep 17 00:00:00 2001

From: Mark Langsdorf <[email protected]>
Date: Fri, 2 Oct 2009 10:32:33 -0500
Subject: [PATCH] Support Pause Filter in AMD processors

New AMD processors (Family 0x10 models 8+) support the Pause Filter Feature. This feature creates a new field in the VMCB called Pause Filter Count. If Pause Filter Count is greater than 0 and intercepting PAUSEs is enabled, the processor will increment an internal counter when a PAUSE instruction occurs instead of intercepting. When the internal counter reaches the Pause Filter Count value, a PAUSE intercept will occur.

This feature can be used to detect contended spinlocks, especially when the lock holding VCPU is not scheduled. Rescheduling another VCPU prevents the VCPU seeking the lock from wasting its quantum by spinning idly.

Experimental results show that most spinlocks are held for less than 1000 PAUSE cycles or more than a few thousand. Default the Pause Filter Counter to 3000 to detect the contended spinlocks.

Processor support for this feature is indicated by a CPUID bit.

On a 24 core system running 4 guests each with 16 VCPUs, this patch improved overall performance of each guest's 32 job kernbench by approximately 3-5% when combined with a scheduler algorithm thati caused the VCPU to sleep for a brief period. Further performance improvement may be possible with a more sophisticated yield algorithm.

This patch depends on the changes to the kvm code from KVM:VMX: Add support for Pause Loop Exiting
http://www.mail-archive.com/[email protected]/msg23089.html

-Mark Langsdorf
Operating System Research Center
AMD

---
 arch/x86/include/asm/svm.h |    3 ++-
 arch/x86/kvm/svm.c         |   16 ++++++++++++++++
 2 files changed, 18 insertions(+), 1 deletions(-)
arch/x86/include/asm/svm.h +2 -1
index 85574b7..1fecb7e 100644
@@ -57,7 +57,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 	u16 intercept_dr_write;
 	u32 intercept_exceptions;
 	u64 intercept;
-	u8 reserved_1[44];
+	u8 reserved_1[42];
+	u16 pause_filter_count;
 	u64 iopm_base_pa;
 	u64 msrpm_base_pa;
 	u64 tsc_offset;
arch/x86/kvm/svm.c +16 -0
index 9a4daca..d5d2e03 100644
@@ -46,6 +46,7 @@ MODULE_LICENSE("GPL");
 #define SVM_FEATURE_NPT  (1 << 0)
 #define SVM_FEATURE_LBRV (1 << 1)
 #define SVM_FEATURE_SVML (1 << 2)
+#define SVM_FEATURE_PAUSE_FILTER (1 << 10)
 
 #define NESTED_EXIT_HOST	0	/* Exit handled on host level */
 #define NESTED_EXIT_DONE	1	/* Exit caused nested vmexit  */

@@ -659,6 +660,11 @@ static void init_vmcb(struct vcpu_svm *svm)
 	svm->nested.vmcb = 0;
 	svm->vcpu.arch.hflags = 0;
 
+	if (svm_has(SVM_FEATURE_PAUSE_FILTER)) {
+		control->pause_filter_count = 3000;
+		control->intercept |= (1ULL << INTERCEPT_PAUSE);
+	}
+
 	enable_gif(svm);
 }
 

@@ -2270,6 +2276,15 @@ static int interrupt_window_interception(struct vcpu_svm *svm)
 	return 1;
 }
 
+static int pause_interception(struct vcpu_svm *svm)
+{
+	static int pause_count = 0;
+
+	kvm_vcpu_on_spin(&(svm->vcpu));
+printk(KERN_ERR "MJLL pause intercepted %d\n", ++pause_count);
+	return 1;
+}
+
 static int (*svm_exit_handlers[])(struct vcpu_svm *svm) = {
 	[SVM_EXIT_READ_CR0]           		= emulate_on_interception,
 	[SVM_EXIT_READ_CR3]           		= emulate_on_interception,

@@ -2305,6 +2320,7 @@ static int (*svm_exit_handlers[])(struct vcpu_svm *svm) = {
 	[SVM_EXIT_CPUID]			= cpuid_interception,
 	[SVM_EXIT_IRET]                         = iret_interception,
 	[SVM_EXIT_INVD]                         = emulate_on_interception,
+	[SVM_EXIT_PAUSE]			= pause_interception,
 	[SVM_EXIT_HLT]				= halt_interception,
 	[SVM_EXIT_INVLPG]			= invlpg_interception,
 	[SVM_EXIT_INVLPGA]			= invlpga_interception,
Signature · 1 line
1.6.0.2
#1 lore raw

arch/x86/kvm/svm.c static int interrupt_window_interception(struct vcpu_svm *svm) · line 2284

Quoted 6 lines · Mark Langsdorf
arch/x86/kvm/svm.c +6 -0
+static int pause_interception(struct vcpu_svm *svm)
+{
+	static int pause_count = 0;
+
+	kvm_vcpu_on_spin(&(svm->vcpu));
+printk(KERN_ERR "MJLL pause intercepted %d\n", ++pause_count);

Debugging leftover?

Quoted 2 lines
arch/x86/kvm/svm.c +2 -0
+	return 1;
+}
author
#2 lore raw

PATCH 1/1 Support Pause Filter in AMD processors

In reply to Mark Langsdorf
Quoted 1 line

From 8ec340648103510095ff339b914706c81e9d815d Mon Sep 17 00:00:00 2001

From: Mark Langsdorf <[email protected]>
Date: Wed, 9 Sep 2009 22:12:51 -0500
Subject: [PATCH] [PATCH] Support Pause Filter in AMD processors

New AMD processors (Family 0x10 models 8+) support the Pause Filter Feature. This feature creates a new field in the VMCB called Pause Filter Count. If Pause Filter Count is greater than 0 and intercepting PAUSEs is enabled, the processor will increment an internal counter when a PAUSE instruction occurs instead of intercepting. When the internal counter reaches the Pause Filter Count value, a PAUSE intercept will occur.

This feature can be used to detect contended spinlocks, especially when the lock holding VCPU is not scheduled. Rescheduling another VCPU prevents the VCPU seeking the lock from wasting its quantum by spinning idly.

Experimental results show that most spinlocks are held for less than 1000 PAUSE cycles or more than a few thousand. Default the Pause Filter Counter to 3000 to detect the contended spinlocks.

Processor support for this feature is indicated by a CPUID bit.

On a 24 core system running 4 guests each with 16 VCPUs, this patch improved overall performance of each guest's 32 job kernbench by approximately 3-5% when combined with a scheduler algorithm thati caused the VCPU to sleep for a brief period. Further performance improvement may be possible with a more sophisticated yield algorithm.

This patch depends on the changes to the kvm code from "KVM:VMX: Add support for Pause Loop Exiting"
http://www.mail-archive.com/[email protected]/msg23089.html

-Mark Langsdorf
Operating System Research Center
AMD

Signed-of-by Mark Langsdorf <[email protected]>
---
 arch/x86/include/asm/svm.h |    3 ++-
 arch/x86/kvm/svm.c         |   13 +++++++++++++
 2 files changed, 15 insertions(+), 1 deletions(-)
arch/x86/include/asm/svm.h +2 -1
index 85574b7..1fecb7e 100644
@@ -57,7 +57,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 	u16 intercept_dr_write;
 	u32 intercept_exceptions;
 	u64 intercept;
-	u8 reserved_1[44];
+	u8 reserved_1[42];
+	u16 pause_filter_count;
 	u64 iopm_base_pa;
 	u64 msrpm_base_pa;
 	u64 tsc_offset;
arch/x86/kvm/svm.c +13 -0
index 279a2ae..a07f969 100644
@@ -46,6 +46,7 @@ MODULE_LICENSE("GPL");
 #define SVM_FEATURE_NPT  (1 << 0)
 #define SVM_FEATURE_LBRV (1 << 1)
 #define SVM_FEATURE_SVML (1 << 2)
+#define SVM_FEATURE_PAUSE_FILTER (1 << 10)
 
 #define NESTED_EXIT_HOST	0	/* Exit handled on host level */
 #define NESTED_EXIT_DONE	1	/* Exit caused nested vmexit  */

@@ -660,6 +661,11 @@ static void init_vmcb(struct vcpu_svm *svm)
 	svm->nested.vmcb = 0;
 	svm->vcpu.arch.hflags = 0;
 
+	if (svm_has(SVM_FEATURE_PAUSE_FILTER)) {
+		control->pause_filter_count = 3000;
+		control->intercept |= (1ULL << INTERCEPT_PAUSE);
+	}
+
 	enable_gif(svm);
 }
 

@@ -2261,6 +2267,12 @@ static int interrupt_window_interception(struct vcpu_svm *svm)
 	return 1;
 }
 
+static int pause_interception(struct vcpu_svm *svm)
+{
+	kvm_vcpu_on_spin(&(svm->vcpu));
+	return 1;
+}
+
 static int (*svm_exit_handlers[])(struct vcpu_svm *svm) = {
 	[SVM_EXIT_READ_CR0]           		= emulate_on_interception,
 	[SVM_EXIT_READ_CR3]           		= emulate_on_interception,

@@ -2296,6 +2308,7 @@ static int (*svm_exit_handlers[])(struct vcpu_svm *svm) = {
 	[SVM_EXIT_CPUID]			= cpuid_interception,
 	[SVM_EXIT_IRET]                         = iret_interception,
 	[SVM_EXIT_INVD]                         = emulate_on_interception,
+	[SVM_EXIT_PAUSE]			= pause_interception,
 	[SVM_EXIT_HLT]				= halt_interception,
 	[SVM_EXIT_INVLPG]			= invlpg_interception,
 	[SVM_EXIT_INVLPGA]			= invlpga_interception,
Signature · 1 line
1.6.0.2
Applied
#3 lore raw
Quoted 37 lines · Mark Langsdorf
Quoted 1 line

From 8ec340648103510095ff339b914706c81e9d815d Mon Sep 17 00:00:00 2001

From: Mark Langsdorf <[email protected]>
Date: Wed, 9 Sep 2009 22:12:51 -0500
Subject: [PATCH] [PATCH] Support Pause Filter in AMD processors

New AMD processors (Family 0x10 models 8+) support the Pause Filter Feature. This feature creates a new field in the VMCB called Pause Filter Count. If Pause Filter Count is greater than 0 and intercepting PAUSEs is enabled, the processor will increment an internal counter when a PAUSE instruction occurs instead of intercepting. When the internal counter reaches the Pause Filter Count value, a PAUSE intercept will occur.

This feature can be used to detect contended spinlocks, especially when the lock holding VCPU is not scheduled. Rescheduling another VCPU prevents the VCPU seeking the lock from wasting its quantum by spinning idly.

Experimental results show that most spinlocks are held for less than 1000 PAUSE cycles or more than a few thousand. Default the Pause Filter Counter to 3000 to detect the contended spinlocks.

Processor support for this feature is indicated by a CPUID bit.

On a 24 core system running 4 guests each with 16 VCPUs, this patch improved overall performance of each guest's 32 job kernbench by approximately 3-5% when combined with a scheduler algorithm thati caused the VCPU to sleep for a brief period. Further performance improvement may be possible with a more sophisticated yield algorithm.

This patch depends on the changes to the kvm code from "KVM:VMX: Add support for Pause Loop Exiting"
http://www.mail-archive.com/[email protected]/msg23089.html

Quoted 5 lines

-Mark Langsdorf
Operating System Research Center
AMD

Signed-of-by Mark Langsdorf <[email protected]>

Applied, thanks.