Skip to contentSkip to conversation

x86, mwaitt: introduce AMD mwaitt support

PATCH Superseded Huang Rui 3 participants 11 messages Jun 15, 2015 x86

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Huang Rui

author
#0 lore raw

v5 cover x86, mwaitt: introduce AMD mwaitt support

Hi,

This patch set introduces a new instruction support on AMD Carrizo (Family 15h, Model 60h-6fh). It adds mwaitx delay function with a configurable timer.

Andy and Boris provide a suggestion which use mwaitx on delay method.

Some discussions of the background, please see:
http://marc.info/?l=linux-kernel&m=143202042530498&w=2
http://marc.info/?l=linux-kernel&m=143161327003541&w=2
http://marc.info/?l=linux-kernel&m=143222815331016&w=2

They are rebased on tip/master.

Changes from v1 -> v2
- Remove mwaitx idle implementation since some disputes without power improvement.
- Add a patch which implement another use case on delay.
- Introduce a kernel parameter (delay) to make delay method configurable.

Changes from v2 -> v3
- Add compared data on commit message
- Remove kernel parameter
- Add hint to avoid to access deep state in future
- Update mwaitx delay method as Petter's suggestion

Changes from v3 -> v4
- Put the MONITORX/MWAITX description into comments

Changes from v4 -> v5
- Remove mwaitx function
- Use mwaitx_delay at init_amd
- Use cpu_tts as montioring address scope

I already do some testing with mwaitx on udelay.

Test scenario:

glb_loops = usec_to_tsc(delay_usec)
rdtsc -> read TSC counters as start
mwaitx_delay(glb_loops)
rdtsc -> read TSC counters as end
diff = end - start

Compared the real TSC counts (diff), that means the loops of counter goes. And glb_loops is the input value of the EBX, that is the expect loops which user configures. Below is 10000 us of mwaitx delay, we could see about 1200 (diff - glb_loops) delayed with mwaitx.

[ 2369.008651] start=4401974718758, end=4401992576888, diff=17858130, glb_loops=17856939

Thanks,
Rui

Huang Rui (2):
x86, mwaitt: add monitorx and mwaitx instruction x86, mwaitt: introduce mwaix delay with a configurable timer

 arch/x86/include/asm/cpufeature.h |  1 +
 arch/x86/include/asm/delay.h      |  1 +
 arch/x86/include/asm/mwait.h      | 43 +++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/cpu/amd.c         |  4 ++++
 arch/x86/lib/delay.c              | 45 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 94 insertions(+)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH v5 1/2 x86, mwaitt: add monitorx and mwaitx instruction

On AMD Carrizo processors (Family 15h, Model 60h-6fh), there is a new feature called MWAITT (Mwait with a timer) as an extension of Monitor/Mwait.

MWAITT, another name is MWAITX (MWAIT with extensions), has a configurable timer that causes MWAITX to exit on expiration.

Compared with MONITOR/MWAIT, there are minor differences in opcode and input parameters.

MWAITX ECX[1]: enable timer if set
MWAITX EBX[31:0]: max wait time expressed in SW P0 clocks

MWAIT MWAITX
opcode 0f 01 c9 | 0f 01 fb
ECX[0] value of RFLAGS.IF seen by instruction
ECX[1] unused/#GP if set | enable timer if set
ECX[31:2] unused/#GP if set
EAX unused (reserve for hint)
EBX[31:0] unused | max wait time (loops)

MONITOR MONITORX
opcode 0f 01 c8 | 0f 01 fa
EAX (logical) address to monitor
ECX #GP if not zero

The software P0 frequency is the same as the TSC frequency.

Max timeout = EBX/(TSC frequency)

---
 arch/x86/include/asm/cpufeature.h |  1 +
 arch/x86/include/asm/mwait.h      | 40 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)
arch/x86/include/asm/cpufeature.h +1 -0
index 3d6606f..3ef1f6e 100644
@@ -176,6 +176,7 @@
 #define X86_FEATURE_PERFCTR_NB  ( 6*32+24) /* NB performance counter extensions */
 #define X86_FEATURE_BPEXT	(6*32+26) /* data breakpoint extension */
 #define X86_FEATURE_PERFCTR_L2	( 6*32+28) /* L2 performance counter extensions */
+#define X86_FEATURE_MWAITT	( 6*32+29) /* Mwait extension (MonitorX/MwaitX) */
 
 /*
  * Auxiliary flags: Linux defined - For features scattered in various
arch/x86/include/asm/mwait.h +40 -0
index 653dfa7..1fbc89d 100644
@@ -23,6 +23,14 @@ static inline void __monitor(const void *eax, unsigned long ecx,
 		     :: "a" (eax), "c" (ecx), "d"(edx));
 }
 
+static inline void __monitorx(const void *eax, unsigned long ecx,
+			      unsigned long edx)
+{
+	/* "monitorx %eax, %ecx, %edx;" */
+	asm volatile(".byte 0x0f, 0x01, 0xfa;"
+		     :: "a" (eax), "c" (ecx), "d"(edx));
+}
+
 static inline void __mwait(unsigned long eax, unsigned long ecx)
 {
 	/* "mwait %eax, %ecx;" */
@@ -30,6 +38,38 @@ static inline void __mwait(unsigned long eax, unsigned long ecx)
 		     :: "a" (eax), "c" (ecx));
 }
 
+/*
+ * MWAITT allows for both a timer value to get you out of the MWAIT as
+ * well as the normal exit conditions.
+ *
+ * MWAITX ECX[1]: enable timer if set
+ * MWAITX EBX[31:0]: max wait time expressed in SW P0 clocks
+ *
+ * Below is the compared data between MWAIT and MWAITX on AMD
+ * processors:
+ *                 MWAIT                           MWAITX
+ * opcode          0f 01 c9           |            0f 01 fb
+ * ECX[0]                  value of RFLAGS.IF seen by instruction
+ * ECX[1]          unused/#GP if set  |            enable timer if set
+ * ECX[31:2]                     unused/#GP if set
+ * EAX                           unused (reserve for hint)
+ * EBX[31:0]       unused             |            max wait time (loops)
+ *
+ *                 MONITOR                         MONITORX
+ * opcode          0f 01 c8           |            0f 01 fa
+ * EAX                     (logical) address to monitor
+ * ECX                     #GP if not zero
+ *
+ * The software P0 frequency is the same as the TSC frequency.
+ */
+static inline void __mwaitx(unsigned long eax, unsigned long ebx,
+			    unsigned long ecx)
+{
+	/* "mwaitx %eax, %ebx, %ecx;" */
+	asm volatile(".byte 0x0f, 0x01, 0xfb;"
+		     :: "a" (eax), "b" (ebx), "c" (ecx));
+}
+
 static inline void __sti_mwait(unsigned long eax, unsigned long ecx)
 {
 	trace_hardirqs_on();
Signature · 1 line
1.9.1
#2 lore raw

Re: [PATCH v5 1/2] x86, mwaitt: add monitorx and mwaitx instruction

Quoted 9 lines · Huang Rui

+/*
+ * MWAITT allows for both a timer value to get you out of the MWAIT as
+ * well as the normal exit conditions.
+ *
+ * MWAITX ECX[1]: enable timer if set
+ * MWAITX EBX[31:0]: max wait time expressed in SW P0 clocks
+ *
+ * Below is the compared data between MWAIT and MWAITX on AMD
+ * processors:

Quoted 6 lines
+ *                 MWAIT                           MWAITX
+ * opcode          0f 01 c9           |            0f 01 fb

+ * ECX[0] value of RFLAGS.IF seen by instruction
+ * ECX[1] unused/#GP if set | enable timer if set
+ * ECX[31:2] unused/#GP if set
+ * EAX unused (reserve for hint)

Seeing how you're stuffing a !0 value in here in the next patch, the above comment seems slightly incorrect, no?

Quoted 23 lines

+ * EBX[31:0] unused | max wait time (loops)
+ *

+ *                 MONITOR                         MONITORX
+ * opcode          0f 01 c8           |            0f 01 fa

+ * EAX (logical) address to monitor
+ * ECX #GP if not zero
+ *
+ * The software P0 frequency is the same as the TSC frequency.
+ */

arch/x86/include/asm/mwait.h +8 -0
+static inline void __mwaitx(unsigned long eax, unsigned long ebx,
+			    unsigned long ecx)
+{
+	/* "mwaitx %eax, %ebx, %ecx;" */
+	asm volatile(".byte 0x0f, 0x01, 0xfb;"
+		     :: "a" (eax), "b" (ebx), "c" (ecx));
+}
+
 static inline void __sti_mwait(unsigned long eax, unsigned long ecx)
 {
 	trace_hardirqs_on();
Signature · 1 line
1.9.1
author
#3 lore raw

Re: [PATCH v5 1/2] x86, mwaitt: add monitorx and mwaitx instruction

Quoted 17 lines · Peter Zijlstra
Quoted 9 lines · Huang Rui

+/*
+ * MWAITT allows for both a timer value to get you out of the MWAIT as
+ * well as the normal exit conditions.
+ *
+ * MWAITX ECX[1]: enable timer if set
+ * MWAITX EBX[31:0]: max wait time expressed in SW P0 clocks
+ *
+ * Below is the compared data between MWAIT and MWAITX on AMD
+ * processors:

Quoted 6 lines
+ *                 MWAIT                           MWAITX
+ * opcode          0f 01 c9           |            0f 01 fb

+ * ECX[0] value of RFLAGS.IF seen by instruction
+ * ECX[1] unused/#GP if set | enable timer if set
+ * ECX[31:2] unused/#GP if set
+ * EAX unused (reserve for hint)

Quoted 3 lines

Seeing how you're stuffing a !0 value in here in the next patch, the above comment seems slightly incorrect, no?

That's because current processor doesn't support to go C1 state. The next patch use the hint is to avoid core go to C1 state in future processor.

Thanks,
Rui

author
#4 lore raw

PATCH v5 2/2 x86, mwaitt: introduce mwaix delay with a configurable timer

In reply to Huang Rui

MWAITX can enable a timer and a corresponding timer value specified in SW P0 clocks. The SW P0 frequency is the same with TSC. The timer provides an upper bound on how long the instruction waits before exiting.

The implementation of delay function in kernel can lerverage the timer of MWAITX. This patch provides a new method (delay_mwaitx) to measure delay time.

---
 arch/x86/include/asm/delay.h |  1 +
 arch/x86/include/asm/mwait.h |  3 +++
 arch/x86/kernel/cpu/amd.c    |  4 ++++
 arch/x86/lib/delay.c         | 45 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 53 insertions(+)
arch/x86/include/asm/delay.h +1 -0
index 9b3b4f2..36a760b 100644
@@ -4,5 +4,6 @@
 #include <asm-generic/delay.h>
 
 void use_tsc_delay(void);
+void use_mwaitx_delay(void);
 
 #endif /* _ASM_X86_DELAY_H */
arch/x86/include/asm/mwait.h +3 -0
index 1fbc89d..47f3540 100644
@@ -14,6 +14,9 @@
 #define CPUID5_ECX_INTERRUPT_BREAK	0x2
 
 #define MWAIT_ECX_INTERRUPT_BREAK	0x1
+#define MWAITX_ECX_TIMER_ENABLE		BIT(1)
+#define MWAITX_MAX_LOOPS		((u32)-1)
+#define MWAITX_DISABLE_CSTATES		0xf
 
 static inline void __monitor(const void *eax, unsigned long ecx,
 			     unsigned long edx)
arch/x86/kernel/cpu/amd.c +4 -0
index 5bd3a99..1f0a8e2 100644
@@ -11,6 +11,7 @@
 #include <asm/cpu.h>
 #include <asm/smp.h>
 #include <asm/pci-direct.h>
+#include <asm/delay.h>
 
 #ifdef CONFIG_X86_64
 # include <asm/mmconfig.h>
@@ -661,6 +662,9 @@ static void init_amd(struct cpuinfo_x86 *c)
 
 	early_init_amd(c);
 
+	if (static_cpu_has_safe(X86_FEATURE_MWAITT))
+		use_mwaitx_delay();
+
 	/*
 	 * Bit 31 in normal CPUID used for nonstandard 3DNow ID;
 	 * 3DNow is IDd by bit 31 in extended CPUID (1*32+31) anyway
arch/x86/lib/delay.c +45 -0

63 lines not on this page · raw message

Signature · 1 line
1.9.1
#5 lore raw

Re: [PATCH v5 2/2] x86, mwaitt: introduce mwaix delay with a configurable timer

Quoted 5 lines · Huang Rui
arch/x86/include/asm/mwait.h +0 -0
index 1fbc89d..47f3540 100644

arch/x86/include/asm/mwait.h line 22

Quoted 9 lines
arch/x86/include/asm/mwait.h +3 -0
@@ -14,6 +14,9 @@
 #define CPUID5_ECX_INTERRUPT_BREAK	0x2
 
 #define MWAIT_ECX_INTERRUPT_BREAK	0x1
+#define MWAITX_ECX_TIMER_ENABLE		BIT(1)
+#define MWAITX_MAX_LOOPS		((u32)-1)
+#define MWAITX_DISABLE_CSTATES		0xf
 
 static inline void __monitor(const void *eax, unsigned long ecx,
 			     unsigned long edx)

Should this hunk not be part of the previous patch?

Quoted 28 lines

/*
+ * On AMD platforms mwaitx has a configurable 32-bit timer, that counts
+ * with TSC frequency. And the input value is the loop of the counter, it
+ * will exit with the timer expired.
+ */

arch/x86/include/asm/mwait.h +23 -0
+static void delay_mwaitx(unsigned long __loops)
+{
+	u32 end, start, delay, loops = __loops;
+
+	rdtsc_barrier();
+	rdtscl(start);
+
+	for (;;) {
+		delay = min(MWAITX_MAX_LOOPS, loops);
+
+		/*
+		 * Use cpu_tss as a cacheline-aligned, seldomly
+		 * accessed per-cpu variable as the monitor target.
+		 */
+		__monitorx(this_cpu_ptr(&cpu_tss), 0, 0);
+		/*
+		 * AMD, like Intel, supports the EAX hint and EAX=0xf
+		 * means, do not enter any deep C-state and we use it
+		 * here in delay() to minimize wakeup latency.
+		 */
+		__mwaitx(MWAITX_DISABLE_CSTATES, delay, MWAITX_ECX_TIMER_ENABLE);
+
+		rdtsc_barrier();

arch/x86/include/asm/mwait.h

Quoted 10 lines
arch/x86/include/asm/mwait.h +10 -0
+		rdtscl(end);
+
+		if (loops <= end - start)
+			break;
+
+		loops -= end - start;
+
+		start = end;
+	}
+}

OK, so what is not explained is how this delay is 'better' than the TSC delay loop we currently have.

Seeing how we disable C states, its unlikely to use less energy, so what exactly is its benefit, other than using fancy new instructions?

#6 lore raw

Re: [PATCH v5 2/2] x86, mwaitt: introduce mwaix delay with a configurable timer

Quoted 2 lines · Peter Zijlstra

Seeing how we disable C states, its unlikely to use less energy, so what exactly is its benefit, other than using fancy new instructions?

If the "disabling of C states" turns into a "go into a C-state which has small wakeup latency" in the future, would be cool. Because we have a bunch of places where we do delay.

Rui, what happens if you enter C1 instead? Any improvements or is the wakeup latency too high?

Thanks.

Signature · 5 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
author
#7 lore raw

Re: [PATCH v5 2/2] x86, mwaitt: introduce mwaix delay with a configurable timer

Quoted 11 lines · Borislav Petkov
Quoted 2 lines · Peter Zijlstra

Seeing how we disable C states, its unlikely to use less energy, so what exactly is its benefit, other than using fancy new instructions?

If the "disabling of C states" turns into a "go into a C-state which has small wakeup latency" in the future, would be cool. Because we have a bunch of places where we do delay.

Rui, what happens if you enter C1 instead? Any improvements or is the wakeup latency too high?

Hmm, the current processor cannot enter C1 with MWAITX, so I don't confirm if it would have higher wakeup latency on C1 in future.

Thanks,
Rui

#8 lore raw

Re: [PATCH v5 2/2] x86, mwaitt: introduce mwaix delay with a configurable timer

Quoted 2 lines · Huang Rui

Hmm, the current processor cannot enter C1 with MWAITX, so I don't confirm if it would have higher wakeup latency on C1 in future.

I remember you saying MWAITX enters currently something between C1 and C0. How does that behave wrt latency and power savings?

Signature · 5 lines
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
author
#9 lore raw

Re: [PATCH v5 2/2] x86, mwaitt: introduce mwaix delay with a configurable timer

Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Huang Rui

Hmm, the current processor cannot enter C1 with MWAITX, so I don't confirm if it would have higher wakeup latency on C1 in future.

I remember you saying MWAITX enters currently something between C1 and C0. How does that behave wrt latency and power savings?

Yes, right. The power consumption is less than C0 (C0 > MWAITX > C1). The waiting exit speed is faster than HTL.

Thanks,
Rui

author
#10 lore raw

Re: [PATCH v5 2/2] x86, mwaitt: introduce mwaix delay with a configurable timer

In reply to Peter Zijlstra
Quoted 13 lines · Peter Zijlstra
Quoted 12 lines · Huang Rui
arch/x86/include/asm/mwait.h +3 -0
index 1fbc89d..47f3540 100644
@@ -14,6 +14,9 @@
 #define CPUID5_ECX_INTERRUPT_BREAK	0x2
 
 #define MWAIT_ECX_INTERRUPT_BREAK	0x1
+#define MWAITX_ECX_TIMER_ENABLE		BIT(1)
+#define MWAITX_MAX_LOOPS		((u32)-1)
+#define MWAITX_DISABLE_CSTATES		0xf
 
Quoted 5 lines

arch/x86/include/asm/mwait.h line 22

Quoted 2 lines
arch/x86/include/asm/mwait.h +0 -0
 static inline void __monitor(const void *eax, unsigned long ecx,
 			     unsigned long edx)

Should this hunk not be part of the previous patch?

This definitions are used to implement delay function, so I put it at this patch. :)

Thanks,
Rui