extend alternative instruction framework to support more than one alternative
cover extend alternative instruction framework to support more than one alternative
Hi,
this small series of patches extend the alternative instructions framework on i386 and x86_64 architectures to support more than one alternative per instruction. The last patch in this series includes a first use of this extension to optimize the get_cycles_sync() function.
The extension works as follows:
The alt_instr struct is extended with 2 more fields:
u8 used; u8 instr_cpuid;
which are both set to 0 for all existing entries. If an existing entry should have more than one alternative instruction the further alternatives are encoded in a following alt_instr structure. In this case the 'use' variable is either 1 oder 2 in that following structure. This encodes the number of further alternative instructions in that struct. The first following instruction is pointed to by the *instr pointer in the struct. Its matching cpuid field is 'instr_cpuid'. The third alternative is encoded like the first one in the original entry using the *replacement pointer and the cpuid field. These following structs could be cascaded to support an arbitrary number of alternative instructions. When applying the alternative instructions the sequence of alternatives is scanned and the first matching entry wins. The patches should apply cleanly to recent Linus git kernels.
Regards,
Joerg
Signature · 3 lines
Joerg Roedel Operating System Research Center AMD Saxony LLC & Co. KG
PATCH 3/3 optimize get_cycles_sync for Linux as KVM guest
From: Joerg Roedel <[email protected]>
This patch modifies the get_cycles_sync() function on i386 and x86_64 to use the RDTSCP (if it is available) instruction to synchronize with the CPU core and not CPUID. This is especially usefull when running Linux as a KVM guest because CPUID is intercepted and will cause a VMEXIT there.
Signature · 3 lines
Joerg Roedel Operating System Research Center AMD Saxony LLC & Co. KG
include/asm-i386/cpufeature.h +1 -0
@@ -49,6 +49,7 @@#define X86_FEATURE_MP (1*32+19)/* MP Capable. */ #define X86_FEATURE_NX (1*32+20)/* Execute Disable */ #define X86_FEATURE_MMXEXT (1*32+22)/* AMD MMX extensions */ +#define X86_FEATURE_RDTSCP (1*32+27)/* RDTSCP */ #define X86_FEATURE_LM (1*32+29)/* Long Mode (x86-64) */ #define X86_FEATURE_3DNOWEXT (1*32+30)/* AMD 3DNow! extensions */ #define X86_FEATURE_3DNOW (1*32+31)/* 3DNow! */
include/asm-x86_64/tsc.h +11 -5
@@ -34,22 +34,28 @@ static inline cycles_t get_cycles(void)/* Like get_cycles, but make sure the CPU is synchronized. */ static __always_inline cycles_t get_cycles_sync(void) { -unsigned long long ret; -#ifdef X86_FEATURE_SYNC_RDTSC +unsigned int a, d;unsigned eax; +#ifdef X86_FEATURE_SYNC_RDTSC /* * Don't do an additional sync on CPUs where we know * RDTSC is already synchronous: */ alternative_io ("cpuid" ,ASM_NOP2 ,X86_FEATURE_SYNC_RDTSC , "=a" (eax), "0" (1) : "ebx","ecx","edx","memory"); +/* We use RDTSCP if it is available, no extra CPUID required then */ +alternative_add_one (ASM_NOP2 ,X86_FEATURE_RDTSCP );#else -sync_core (); +/* no CPUID required if we use RDTSCP */ +alternative_io ("cpuid" ,ASM_NOP2 ,X86_FEATURE_RDTSCP , + "=a" (eax), "0" (1) : "ebx","ecx","edx","memory");#endif -rdtscll (ret); -return ret; +alternative_io ("rdtsc\n" ASM_NOP1 ,"rdtscp" ,X86_FEATURE_RDTSCP , + ALTERNATIVE_OUTPUT2("=a" (a), "=d" (d)), "0" (1) : "ecx","memory"); + +return ((unsigned long long )a) | ((unsigned long long )d) <<32 ; }extern void tsc_init (void );
Re: [discuss] [PATCH 1/3] x86_64: additions to the i386 alternative extensions to support x86_64 architecture
In reply to Joerg Roedel[Patch 2/3] of course.
Signature · 3 lines
Joerg Roedel Operating System Research Center AMD Saxony LLC & Co. KG
PATCH 1/3 i386: extend alternative instruction framwork
In reply to Joerg RoedelFrom: Joerg Roedel <[email protected]>
This patch extends the alternative instruction framework to support an arbitrary number of alternatives on the i386 architecture.
Signature · 3 lines
Joerg Roedel Operating System Research Center AMD Saxony LLC & Co. KG
arch/i386/kernel/alternative.c +28 -8
@@ -149,19 +149,38 @@ extern u8 __smp_alt_begin[], __smp_alt_end[];self modifying code. This implies that assymetric systems where APs have less capabilities than the boot processor are not handled. Tough. Make sure you disable such features by hand. */ -void apply_alternatives (struct alt_instr *start,struct alt_instr *end) {struct alt_instr *a; -u8 *instr; +u8 *instr =NULL , *new_instr =NULL ; +u8 instr_len =0 , new_instr_len =0 ;int diff;DPRINTK ("%s: alt table %p -> %p\n" , __FUNCTION__, start, end);for (a = start; a < end; a++) { -BUG_ON (a->replacementlen > a->instrlen); -if (!boot_cpu_has (a->cpuid)) + +if (a->used ==0 ) { + instr = a->instr; + instr_len = a->instrlen; + new_instr = +boot_cpu_has (a->cpuid) ? a->replacement :NULL ; + new_instr_len = +boot_cpu_has (a->cpuid) ? a->replacementlen :0 ; + }else if (a->used >=1 &&boot_cpu_has (a->instr_cpuid)) { +BUG_ON (instr ==NULL ); + new_instr = a->instr; + new_instr_len = a->instrlen; + }else if (a->used ==2 &&boot_cpu_has (a->cpuid)) { +BUG_ON (instr ==NULL ); + new_instr = a->replacement; + new_instr_len = a->replacementlen; + } + +if (instr ==NULL || new_instr ==NULL )continue ; - instr = a->instr; + +BUG_ON (new_instr_len > instr_len); +#ifdef CONFIG_X86_64 /* vsyscall code is not mapped yet. resolve it manually. */ if (instr >= (u8 *)VSYSCALL_START && instr < (u8 *)VSYSCALL_END ) {
@@ -170,9 +189,10 @@ void apply_alternatives(struct alt_instr *start, struct alt_instr *end) __FUNCTION__, a->instr, instr); }#endif -memcpy (instr, a->replacement, a->replacementlen); - diff = a->instrlen - a->replacementlen; -nop_out (instr + a->replacementlen, diff); +memcpy (instr, new_instr, new_instr_len); + diff = instr_len - new_instr_len; +nop_out (instr + new_instr_len, diff); + instr = new_instr =NULL ; } }
include/asm-i386/alternative.h +85 -2
123 lines not on this page · raw message
PATCH 1/3 x86_64: additions to the i386 alternative extensions to support x86_64 architecture
In reply to Joerg RoedelFrom: Joerg Roedel <[email protected]>
This patch adds the necessary changes to extend the i386 alternative instruction framework extension on the x86_64 architecture.
Signature · 3 lines
Joerg Roedel Operating System Research Center AMD Saxony LLC & Co. KG
arch/x86_64/lib/clear_page.S +2 -0
@@ -53,6 +53,8 @@ ENDPROC(clear_page) .align 8 .quad clear_page .quad 1b + .byte 0x00 + .byte 0x00 .byte X86_FEATURE_REP_GOOD .byte .Lclear_page_end - clear_page .byte 2b - 1b
arch/x86_64/lib/copy_page.S +2 -0
@@ -113,6 +113,8 @@ ENDPROC(copy_page) .align 8 .quad copy_page .quad 1b + .byte 0x00 + .byte 0x00 .byte X86_FEATURE_REP_GOOD .byte .Lcopy_page_end - copy_page .byte 2b - 1b
arch/x86_64/lib/copy_user.S +2 -0
@@ -27,6 +27,8 @@ .align 8 .quad 0b .quad 2b + .byte 0x00 + .byte 0x00 .byte \feature /* when feature is set */ .byte 5 .byte 5
arch/x86_64/lib/memcpy.S +2 -0
@@ -123,6 +123,8 @@ ENDPROC(__memcpy) .align 8 .quad memcpy .quad 1b + .byte 0x00 + .byte 0x00 .byte X86_FEATURE_REP_GOOD .byte .Lfinal - memcpy .byte 2b - 1b
arch/x86_64/lib/memset.S +2 -0
@@ -127,6 +127,8 @@ ENDPROC(__memset) .align 8 .quad memset .quad 1b + .byte 0x00 + .byte 0x00 .byte X86_FEATURE_REP_GOOD .byte .Lfinal - memset .byte 2b - 1b
include/asm-x86_64/alternative.h +71 -2
115 lines not on this page · raw message
Re: [discuss] [PATCH 1/3] x86_64: additions to the i386 alternative extensions to support x86_64 architecture
Quoted 4 lines · Joerg Roedel
From: Joerg Roedel <[email protected]>
This patch adds the necessary changes to extend the i386 alternative instruction framework extension on the
x86_64architecture.
Looks complicated and somewhat fragile.
I think I would prefer it if you changed the pad field to a "bit not set" cpuid field. Then at least 2 alternatives + a nothing alternative could be described too.
-Andi
Re: [discuss] [PATCH 1/3] x86_64: additions to the i386 alternative extensions to support x86_64 architecture
Quoted 7 lines · Andi Kleen
Quoted 4 lines · Joerg Roedel
From: Joerg Roedel <[email protected]>
This patch adds the necessary changes to extend the i386 alternative instruction framework extension on the
x86_64architecture.Looks complicated and somewhat fragile.
The tests on i386 and x86_64 just work fine :-)
Quoted 3 lines
I think I would prefer it if you changed the pad field to a "bit not set" cpuid field. Then at least 2 alternatives + a nothing alternative could be described too.
I'm not sure I fully understand what you mean. Do you mean to add a further alternative instruction to the struct for supporting 2 alternative instructions?
Joerg
Signature · 3 lines
Joerg Roedel Operating System Research Center AMD Saxony LLC & Co. KG