Skip to contentSkip to conversation

extend alternative instruction framework to support more than one alternative

PATCH In review Joerg Roedel 2 participants 7 messages Feb 19, 2007 – Feb 20, 2007

lore
Share & tools
lore
#0 lore raw

cover extend alternative instruction framework to support more than one alternative

Hi,

this small series of patches extend the alternative instructions framework on i386 and x86_64 architectures to support more than one alternative per instruction. The last patch in this series includes a first use of this extension to optimize the get_cycles_sync() function.

The extension works as follows:

The alt_instr struct is extended with 2 more fields:

	u8  used;
	u8  instr_cpuid;

which are both set to 0 for all existing entries. If an existing entry should have more than one alternative instruction the further alternatives are encoded in a following alt_instr structure. In this case the 'use' variable is either 1 oder 2 in that following structure. This encodes the number of further alternative instructions in that struct. The first following instruction is pointed to by the *instr pointer in the struct. Its matching cpuid field is 'instr_cpuid'. The third alternative is encoded like the first one in the original entry using the *replacement pointer and the cpuid field. These following structs could be cascaded to support an arbitrary number of alternative instructions. When applying the alternative instructions the sequence of alternatives is scanned and the first matching entry wins. The patches should apply cleanly to recent Linus git kernels.

Regards,
Joerg

Signature · 3 lines
Joerg Roedel
Operating System Research Center
AMD Saxony LLC & Co. KG
#6 lore raw

PATCH 3/3 optimize get_cycles_sync for Linux as KVM guest

From: Joerg Roedel <[email protected]>

This patch modifies the get_cycles_sync() function on i386 and x86_64 to use the RDTSCP (if it is available) instruction to synchronize with the CPU core and not CPUID. This is especially usefull when running Linux as a KVM guest because CPUID is intercepted and will cause a VMEXIT there.

Signature · 3 lines
Joerg Roedel
Operating System Research Center
AMD Saxony LLC & Co. KG
include/asm-i386/cpufeature.h +1 -0
index 3f92b94..7275e41 100644
@@ -49,6 +49,7 @@
 #define X86_FEATURE_MP		(1*32+19) /* MP Capable. */
 #define X86_FEATURE_NX		(1*32+20) /* Execute Disable */
 #define X86_FEATURE_MMXEXT	(1*32+22) /* AMD MMX extensions */
+#define X86_FEATURE_RDTSCP      (1*32+27) /* RDTSCP */
 #define X86_FEATURE_LM		(1*32+29) /* Long Mode (x86-64) */
 #define X86_FEATURE_3DNOWEXT	(1*32+30) /* AMD 3DNow! extensions */
 #define X86_FEATURE_3DNOW	(1*32+31) /* 3DNow! */
include/asm-x86_64/tsc.h +11 -5
index 9a0a368..7db952d 100644
@@ -34,22 +34,28 @@ static inline cycles_t get_cycles(void)
 /* Like get_cycles, but make sure the CPU is synchronized. */
 static __always_inline cycles_t get_cycles_sync(void)
 {
-	unsigned long long ret;
-#ifdef X86_FEATURE_SYNC_RDTSC
+	unsigned int a, d;
 	unsigned eax;
 
+#ifdef X86_FEATURE_SYNC_RDTSC
 	/*
 	 * Don't do an additional sync on CPUs where we know
 	 * RDTSC is already synchronous:
 	 */
 	alternative_io("cpuid", ASM_NOP2, X86_FEATURE_SYNC_RDTSC,
 			  "=a" (eax), "0" (1) : "ebx","ecx","edx","memory");
+	/* We use RDTSCP if it is available, no extra CPUID required then */
+	alternative_add_one(ASM_NOP2, X86_FEATURE_RDTSCP);
 #else
-	sync_core();
+	/* no CPUID required if we use RDTSCP */
+	alternative_io("cpuid", ASM_NOP2, X86_FEATURE_RDTSCP,
+			  "=a" (eax), "0" (1) : "ebx","ecx","edx","memory");
 #endif
-	rdtscll(ret);
 
-	return ret;
+	alternative_io("rdtsc\n" ASM_NOP1, "rdtscp", X86_FEATURE_RDTSCP,
+			ALTERNATIVE_OUTPUT2("=a" (a), "=d" (d)), "0" (1) : "ecx","memory");
+
+	return ((unsigned long long)a) | ((unsigned long long)d) << 32;
 }
 
 extern void tsc_init(void);
#2 lore raw

Re: [discuss] [PATCH 1/3] x86_64: additions to the i386 alternative extensions to support x86_64 architecture

In reply to Joerg Roedel

[Patch 2/3] of course.

Signature · 3 lines
Joerg Roedel
Operating System Research Center
AMD Saxony LLC & Co. KG
#5 lore raw

PATCH 1/3 i386: extend alternative instruction framwork

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

This patch extends the alternative instruction framework to support an arbitrary number of alternatives on the i386 architecture.

Signature · 3 lines
Joerg Roedel
Operating System Research Center
AMD Saxony LLC & Co. KG
arch/i386/kernel/alternative.c +28 -8
index 9eca21b..2dedf4b 100644
@@ -149,19 +149,38 @@ extern u8 __smp_alt_begin[], __smp_alt_end[];
    self modifying code. This implies that assymetric systems where
    APs have less capabilities than the boot processor are not handled.
    Tough. Make sure you disable such features by hand. */
-
 void apply_alternatives(struct alt_instr *start, struct alt_instr *end)
 {
 	struct alt_instr *a;
-	u8 *instr;
+	u8 *instr = NULL, *new_instr = NULL;
+	u8 instr_len = 0, new_instr_len = 0;
 	int diff;
 
 	DPRINTK("%s: alt table %p -> %p\n", __FUNCTION__, start, end);
 	for (a = start; a < end; a++) {
-		BUG_ON(a->replacementlen > a->instrlen);
-		if (!boot_cpu_has(a->cpuid))
+
+		if (a->used == 0) {
+			instr = a->instr;
+			instr_len = a->instrlen;
+			new_instr =
+				boot_cpu_has(a->cpuid) ? a->replacement : NULL;
+			new_instr_len =
+				boot_cpu_has(a->cpuid) ? a->replacementlen : 0;
+		} else if (a->used >= 1 && boot_cpu_has(a->instr_cpuid)) {
+			BUG_ON(instr == NULL);
+			new_instr = a->instr;
+			new_instr_len = a->instrlen;
+		} else if (a->used == 2 && boot_cpu_has(a->cpuid)) {
+			BUG_ON(instr == NULL);
+			new_instr = a->replacement;
+			new_instr_len = a->replacementlen;
+		}
+
+		if (instr == NULL || new_instr == NULL)
 			continue;
-		instr = a->instr;
+
+		BUG_ON(new_instr_len > instr_len);
+
 #ifdef CONFIG_X86_64
 		/* vsyscall code is not mapped yet. resolve it manually. */
 		if (instr >= (u8 *)VSYSCALL_START && instr < (u8*)VSYSCALL_END) {
@@ -170,9 +189,10 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end)
 				__FUNCTION__, a->instr, instr);
 		}
 #endif
-		memcpy(instr, a->replacement, a->replacementlen);
-		diff = a->instrlen - a->replacementlen;
-		nop_out(instr + a->replacementlen, diff);
+		memcpy(instr, new_instr, new_instr_len);
+		diff = instr_len - new_instr_len;
+		nop_out(instr + new_instr_len, diff);
+		instr = new_instr = NULL;
 	}
 }
 
include/asm-i386/alternative.h +85 -2

123 lines not on this page · raw message

#1 lore raw

PATCH 1/3 x86_64: additions to the i386 alternative extensions to support x86_64 architecture

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

This patch adds the necessary changes to extend the i386 alternative instruction framework extension on the x86_64 architecture.

Signature · 3 lines
Joerg Roedel
Operating System Research Center
AMD Saxony LLC & Co. KG
arch/x86_64/lib/clear_page.S +2 -0
index 9a10a78..4e34c12 100644
@@ -53,6 +53,8 @@ ENDPROC(clear_page)
 	.align 8
 	.quad clear_page
 	.quad 1b
+	.byte 0x00
+	.byte 0x00
 	.byte X86_FEATURE_REP_GOOD
 	.byte .Lclear_page_end - clear_page
 	.byte 2b - 1b
arch/x86_64/lib/copy_page.S +2 -0
index 727a5d4..2aa51c2 100644
@@ -113,6 +113,8 @@ ENDPROC(copy_page)
 	.align 8
 	.quad copy_page
 	.quad 1b
+	.byte 0x00
+	.byte 0x00
 	.byte X86_FEATURE_REP_GOOD
 	.byte .Lcopy_page_end - copy_page
 	.byte 2b - 1b
arch/x86_64/lib/copy_user.S +2 -0
index 70bebd3..2dd39d1 100644
@@ -27,6 +27,8 @@
 	.align 8
 	.quad  0b
 	.quad  2b
+	.byte  0x00
+	.byte  0x00
 	.byte  \feature		     /* when feature is set */
 	.byte  5
 	.byte  5
arch/x86_64/lib/memcpy.S +2 -0
index 0ea0ddc..ee42a08 100644
@@ -123,6 +123,8 @@ ENDPROC(__memcpy)
 	.align 8
 	.quad memcpy
 	.quad 1b
+	.byte 0x00
+	.byte 0x00
 	.byte X86_FEATURE_REP_GOOD
 	.byte .Lfinal - memcpy
 	.byte 2b - 1b
arch/x86_64/lib/memset.S +2 -0
index 2c59481..814e091 100644
@@ -127,6 +127,8 @@ ENDPROC(__memset)
 	.align 8
 	.quad memset
 	.quad 1b
+	.byte 0x00
+	.byte 0x00
 	.byte X86_FEATURE_REP_GOOD
 	.byte .Lfinal - memset
 	.byte 2b - 1b
include/asm-x86_64/alternative.h +71 -2

115 lines not on this page · raw message

#3 lore raw

Re: [discuss] [PATCH 1/3] x86_64: additions to the i386 alternative extensions to support x86_64 architecture

Quoted 4 lines · Joerg Roedel

From: Joerg Roedel <[email protected]>

This patch adds the necessary changes to extend the i386 alternative instruction framework extension on the x86_64 architecture.

Looks complicated and somewhat fragile.

I think I would prefer it if you changed the pad field to a "bit not set" cpuid field. Then at least 2 alternatives + a nothing alternative could be described too.

-Andi

#4 lore raw

Re: [discuss] [PATCH 1/3] x86_64: additions to the i386 alternative extensions to support x86_64 architecture

Quoted 7 lines · Andi Kleen
Quoted 4 lines · Joerg Roedel

From: Joerg Roedel <[email protected]>

This patch adds the necessary changes to extend the i386 alternative instruction framework extension on the x86_64 architecture.

Looks complicated and somewhat fragile.

The tests on i386 and x86_64 just work fine :-)

Quoted 3 lines

I think I would prefer it if you changed the pad field to a "bit not set" cpuid field. Then at least 2 alternatives + a nothing alternative could be described too.

I'm not sure I fully understand what you mean. Do you mean to add a further alternative instruction to the struct for supporting 2 alternative instructions?

Joerg

Signature · 3 lines
Joerg Roedel
Operating System Research Center
AMD Saxony LLC & Co. KG