Instruction counting instrumentation for ARM, 2nd version
PATCH 1/1 Instruction counting instrumentation for ARM, 2nd version
Hi,
Attached is the second version of Timo Töyry's instruction counting patch for arm-linux-user targets. Applies to QEMU 0.10.4. To try this, just add -instrcount to your regular qemu-arm command line. See [1] for description on how this is implemented.
The patch should now be complete for ARMv6, VFP and Thumb. However, Thumb2, NEON, and more advanced VFP instrumentation is still missing due to lacking public documentation from ARM. Measured overhead for instruction counting using this approach is between 5-35%, and overhead when the patch is applied but instruction counting disabled is very close to 0%. See below for details.
Instruction counting can be used to choosing the most appropriate CPU variant for an embedded application. It can also be used as a part of a more detailed performance analysis, as instruction mix reflects the performance of the code.
There are also other ways to implement instruction counting (see responses to our previous patch [1]). As to our approach, I believe this is the cleanest way to implement this specific instrumentation. However, from the general point of view, I'm not so sure. I'd like to ask for more comments and options.
Two main reasons for us for to construct this patch: First, we needed the functionality. Second, we're considering using QEMU as a platform for other instrumentation work too, so we thought to start with something simple enough.
What we would really like to see is a more general instrumentation framework, not just instruction counting and not just for ARM targets. The reason for this is that many measurements are so much easier to do in emulated/simulated environment than in real HW (e.g. instruction counting) and doing them in QEMU is way faster than using an ISA interpreter such as Valgrind. Some other instrumentation examples that come to my mind are cache usage efficiency analysis, branch profiling, and naturally instruction and memory access tracing.
A specific bonus of such framework would be that it could be used for the profiling part of possible future optimizations in code translation. Consider IA-32 EL (Intel's x86 emulator for Itanium).
Anyways, I'd like to know if there's a general interest in this kind of framework, and if so, how we should proceed to get it implemented into mainline.
Regards,
Sami Kiminki
Embedded Software Group / Helsinki University of Technology
References:
[1] http://lists.gnu.org/archive/html/qemu-devel/2009-05/msg00922.html
Overhead measurements
=====================
(Core 2 Q6600, QEMU compiled with gcc 4.3.3 and benchmarks with gcc 4.3.2 (arm1176jzf-s), average of 5 runs)
Vanilla QEMU:
- dhrystone: 1.709 s
- dhrystone thumb: 2.185 s
- mibench/jpeg/cjpeg: 0.453 s
- mibench/mad: 0.822 s
Patched QEMU, instruction counting disabled:
- dhrystone: 1.700 s (-0.5%)
- dhrystone thumb: 2.172 s (-0.6%)
- mibench/jpeg/cjpeg: 0.456 s (+0.7%)
- mibench/mad: 0.820 s (-0.2%)
Patched QEMU, instruction counting enabled:
- dhrystone: 1.882 s (+10%)
- dhrystone thumb: 2.300 s (+5%)
- mibench/jpeg/cjpeg: 0.513 s (+13%)
- mibench/mad: 1.078 s (+31%)
Makefile +4 -0
@@ -179,6 +179,10 @@ libqemu_common.a: $(OBJS) # USER_OBJS is code used by qemu userspace emulation USER_OBJS=cutils.o cache-utils.o +# instruction count instrumentation +USER_OBJS+=instrumentation.o + + libqemu_user.a: $(USER_OBJS) ######################################################################
Makefile.target +3 -1
@@ -208,8 +208,10 @@ ifeq ($(findstring s390, $(TARGET_ARCH) LIBOBJS+=s390-dis.o endif -# libqemu +# instrumentation support +LIBOBJS+=instrumentation.o +# libqemu libqemu.a: $(LIBOBJS) translate.o: translate.c cpu.h
Only in qemu-0.10.4-instrcount: instrumentation.c
Only in qemu-0.10.4-instrcount: instrumentation.h
linux-user/main.c +13 -2
@@ -35,6 +35,8 @@#include "envlist.h" +#include "instrumentation.h" +#define DEBUG_LOGFILE "/tmp/qemu.log"char *exec_path;
@@ -2199,6 +2201,12 @@ static void usage(void) "-p pagesize set the host page size to 'pagesize'\n" "-strace log system calls\n" "\n" +#ifdef TARGET_ARM + "Other:\n" + "-instrcount Count instructions\n" + "-instrcountlog=qemu_log Output counters to qemu-log file\n" + "\n" +#endif"Environment variables:\n" "QEMU_STRACE Print system calls and arguments similar to the\n" " 'strace' program. Enable by setting to any value.\n"
@@ -2341,8 +2349,11 @@ int main(int argc, char **argv, char **e (void )envlist_unsetenv (envlist,"LD_PRELOAD" ); }else if (!strcmp (r,"strace" )) { do_strace =1 ; - }else - { + }else if (!strcmp (r,"instrcount" )) { + instrumentation_count_instructions =1 ; + }else if (!strcmp (r,"instrcountlog=qemu_log" )) { + instrumentation_count_instructions_log =1 ; + }else {usage (); } }
linux-user/syscall.c +66 -0
90 lines not on this page · raw message
target-arm/cpu.h +344 -0
360 lines not on this page · raw message
target-arm/translate.c +745 -36
1,923 lines not on this page · raw message
Quoted 8 lines · Sami Kiminki
What we would really like to see is a more general instrumentation framework, not just instruction counting and not just for ARM targets. The reason for this is that many measurements are so much easier to do in emulated/simulated environment than in real HW (e.g. instruction counting) and doing them in QEMU is way faster than using an ISA interpreter such as Valgrind. Some other instrumentation examples that come to my mind are cache usage efficiency analysis, branch profiling, and naturally instruction and memory access tracing.
Valgrind isn't an interpreter. It's a JIT compiler like QEMU's TCG, so are you sure these things are way faster in QEMU?
What might work better would be to share some code from Valgrind/Cachegrind.
-- Jamie
Quoted 10 lines · Jamie Lokier
Quoted 8 lines · Sami Kiminki
What we would really like to see is a more general instrumentation framework, not just instruction counting and not just for ARM targets. The reason for this is that many measurements are so much easier to do in emulated/simulated environment than in real HW (e.g. instruction counting) and doing them in QEMU is way faster than using an ISA interpreter such as Valgrind. Some other instrumentation examples that come to my mind are cache usage efficiency analysis, branch profiling, and naturally instruction and memory access tracing.
Quoted 2 lines
Valgrind isn't an interpreter. It's a JIT compiler like QEMU's TCG, so are you sure these things are way faster in QEMU?
No, not anymore. Thanks for correcting ;)
However, what would still speak in favor of having some instrumentation support in QEMU, is measuring things at system-level. I assume this is pretty high in embedded system developers' wish-list.
Quoted 1 line
What might work better would be to share some code from Valgrind/Cachegrind.
Obviously.
- Sami
On Fri, Jun 12, 2009 at 12:56 PM, Sami Kiminki<[email protected]> wrote:
[...]
Quoted 6 lines
There are also other ways to implement instruction counting (see responses to our previous patch [1]). As to our approach, I believe this is the cleanest way to implement this specific instrumentation. However, from the general point of view, I'm not so sure. I'd like to ask for more comments and options.
Apart from the fact your patch doesn't contain instrumentation.[ch] it is bogus in several places and shows why I think the approach is probably not the best.
E.g.:
@@ -6300,6 +6809,7 @@ static void disas_arm_insn(CPUState * en
if (insn & (1 << 20)) {
gen_helper_mark_exclusive(cpu_env, cpu_T[1]);
switch (op1) {
+ instr_count_inc(ARM_INSTRUCTION_LDREX);
case 0: /* ldrex */
tmp = gen_ld32(addr, IS_USER(s));
break;This is misplaced.
- if (op1 & 2)
+ if (op1 & 2) {
gen_helper_double_saturate(tmp2, tmp2);- if (op1 & 1)
+ if (op1 & 1) instr_count_inc(ARM_INSTRUCTION_QDSUB);
+ else instr_count_inc(ARM_INSTRUCTION_QDADD);
+ }
+ if (op1 & 1) {
gen_helper_sub_saturate(tmp, tmp, tmp2);
- else
+ instr_count_inc(ARM_INSTRUCTION_QSUB);
+ }
+ else {
gen_helper_add_saturate(tmp, tmp, tmp2);+ instr_count_inc(ARM_INSTRUCTION_QADD);
+ }
Here your two last instr_count_inc will overwrite the first two ones. So you don't distinguish between QDADD/QDSUB and QADD/QSUB.
I also would like to see a more general instrumentation framework but your approach is too intrusive and so it's too easy to introduce hard to detect bugs.
Cheers,
Laurent
Re: [Qemu-devel] [PATCH] Instruction counting instrumentation for ARM, 2nd version & 2nd try
In reply to a message not in the archiveApparently I succeeded in forgetting a switch when diffing and 2 files were missing. Attached is a new attempt.
- Sami
Quoted 7 lines · Laurent Desnogues
Hi Sami,
instruction.[ch] are not in your patch it seems :)
Cheers,
Laurent
Makefile +4 -0
@@ -179,6 +179,10 @@ libqemu_common.a: $(OBJS) # USER_OBJS is code used by qemu userspace emulation USER_OBJS=cutils.o cache-utils.o +# instruction count instrumentation +USER_OBJS+=instrumentation.o + + libqemu_user.a: $(USER_OBJS) ######################################################################
Makefile.target +3 -1
@@ -208,8 +208,10 @@ ifeq ($(findstring s390, $(TARGET_ARCH) LIBOBJS+=s390-dis.o endif -# libqemu +# instrumentation support +LIBOBJS+=instrumentation.o +# libqemu libqemu.a: $(LIBOBJS) translate.o: translate.c cpu.h
instrumentation.c +37 -0
@@ -0,0 +1,37 @@ +/* +* instrument.c +* +* Created on: May 14, 2009 +* Author: ttoyry +*/ + +#include "instrumentation.h" +unsigned int instrumentation_count_instructions =0 ; +unsigned int instrumentation_count_instructions_log =0 ; + +/* +static instr_counter_offsets instr_offsets; + +static inline void instr_count_inc_init(uint32_t offset, int instr) +{ +if (!instrumentation_count_instructions) return; +instr_offsets.cpustate_offset = offset; +TCGv tmp = new_tmp(); +tcg_gen_ld_i32(tmp, cpu_env, instr_offsets.cpustate_offset + sizeof(uint32_t) * instr); +instr_offsets.tcg_offset[0] = gen_opparam_ptr - 1; +tcg_gen_addi_i32(tmp, tmp, 1); +tcg_gen_st_i32(tmp, cpu_env, instr_offsets.cpustate_offset + sizeof(uint32_t) * instr); +instr_offsets.tcg_offset[1] = gen_opparam_ptr - 1; +dead_tmp(tmp); +} + +Increment instruction counter +static inline void instr_count_inc(int instr) +{ +if (!instrumentation_count_instructions) return; +*(instr_offsets.tcg_offset[0]) = instr_offsets.cpustate_offset + sizeof(uint32_t) * instr; +*(instr_offsets.tcg_offset[1]) = instr_offsets.cpustate_offset + sizeof(uint32_t) * instr; +} +*/ + +
instrumentation.h +47 -0
47 lines not on this page · raw message
linux-user/main.c +13 -2
@@ -35,6 +35,8 @@#include "envlist.h" +#include "instrumentation.h" +#define DEBUG_LOGFILE "/tmp/qemu.log"char *exec_path;
@@ -2199,6 +2201,12 @@ static void usage(void) "-p pagesize set the host page size to 'pagesize'\n" "-strace log system calls\n" "\n" +#ifdef TARGET_ARM + "Other:\n" + "-instrcount Count instructions\n" + "-instrcountlog=qemu_log Output counters to qemu-log file\n" + "\n" +#endif"Environment variables:\n" "QEMU_STRACE Print system calls and arguments similar to the\n" " 'strace' program. Enable by setting to any value.\n"
@@ -2341,8 +2349,11 @@ int main(int argc, char **argv, char **e (void )envlist_unsetenv (envlist,"LD_PRELOAD" ); }else if (!strcmp (r,"strace" )) { do_strace =1 ; - }else - { + }else if (!strcmp (r,"instrcount" )) { + instrumentation_count_instructions =1 ; + }else if (!strcmp (r,"instrcountlog=qemu_log" )) { + instrumentation_count_instructions_log =1 ; + }else {usage (); } }
linux-user/syscall.c +66 -0
90 lines not on this page · raw message
target-arm/cpu.h +344 -0
360 lines not on this page · raw message
target-arm/translate.c +745 -36
1,923 lines not on this page · raw message
PATCH 1/1 Instruction counting instrumentation for ARM v3
In reply to Laurent DesnoguesHi,
Attached is a version of Timo Töyry's patch which should fix the issues pointed out below. I'm not expecting this to get committed, but thought to share it anyway, should anyone have use for it.
- Sami
Quoted 47 lines · Laurent Desnogues
On Fri, Jun 12, 2009 at 12:56 PM, Sami Kiminki<[email protected]> wrote:
[...]Quoted 6 lines
There are also other ways to implement instruction counting (see responses to our previous patch [1]). As to our approach, I believe this is the cleanest way to implement this specific instrumentation. However, from the general point of view, I'm not so sure. I'd like to ask for more comments and options.
Apart from the fact your patch doesn't contain instrumentation.[ch] it is bogus in several places and shows why I think the approach is probably not the best.
E.g.:
(quoted diff) +1
-0@@ -6300,6 +6809,7 @@ static void disas_arm_insn(CPUState * en if (insn & (1 << 20)) { gen_helper_mark_exclusive(cpu_env, cpu_T[1]); switch (op1) { + instr_count_inc(ARM_INSTRUCTION_LDREX); case 0: /* ldrex */ tmp = gen_ld32(addr, IS_USER(s)); break;This is misplaced.
(quoted diff) +10
-3- if (op1 & 2) + if (op1 & 2) { gen_helper_double_saturate(tmp2, tmp2); - if (op1 & 1) + if (op1 & 1) instr_count_inc(ARM_INSTRUCTION_QDSUB); + else instr_count_inc(ARM_INSTRUCTION_QDADD); + } + if (op1 & 1) { gen_helper_sub_saturate(tmp, tmp, tmp2); - else + instr_count_inc(ARM_INSTRUCTION_QSUB); + } + else { gen_helper_add_saturate(tmp, tmp, tmp2); + instr_count_inc(ARM_INSTRUCTION_QADD); + }Here your two last
instr_count_incwill overwrite the first two ones. So you don't distinguish between QDADD/QDSUB and QADD/QSUB.
Quoted 7 lines
I also would like to see a more general instrumentation framework but your approach is too intrusive and so it's too easy to introduce hard to detect bugs.
Cheers,
Laurent
Makefile +4 -0
@@ -179,6 +179,10 @@ libqemu_common.a: $(OBJS) # USER_OBJS is code used by qemu userspace emulation USER_OBJS=cutils.o cache-utils.o +# instruction count instrumentation +USER_OBJS+=instrumentation.o + + libqemu_user.a: $(USER_OBJS) ######################################################################
Makefile.target +3 -1
@@ -208,8 +208,10 @@ ifeq ($(findstring s390, $(TARGET_ARCH) LIBOBJS+=s390-dis.o endif -# libqemu +# instrumentation support +LIBOBJS+=instrumentation.o +# libqemu libqemu.a: $(LIBOBJS) translate.o: translate.c cpu.h
instrumentation.c +37 -0
@@ -0,0 +1,37 @@ +/* +* instrument.c +* +* Created on: May 14, 2009 +* Author: ttoyry +*/ + +#include "instrumentation.h" +unsigned int instrumentation_count_instructions =0 ; +unsigned int instrumentation_count_instructions_log =0 ; + +/* +static instr_counter_offsets instr_offsets; + +static inline void instr_count_inc_init(uint32_t offset, int instr) +{ +if (!instrumentation_count_instructions) return; +instr_offsets.cpustate_offset = offset; +TCGv tmp = new_tmp(); +tcg_gen_ld_i32(tmp, cpu_env, instr_offsets.cpustate_offset + sizeof(uint32_t) * instr); +instr_offsets.tcg_offset[0] = gen_opparam_ptr - 1; +tcg_gen_addi_i32(tmp, tmp, 1); +tcg_gen_st_i32(tmp, cpu_env, instr_offsets.cpustate_offset + sizeof(uint32_t) * instr); +instr_offsets.tcg_offset[1] = gen_opparam_ptr - 1; +dead_tmp(tmp); +} + +Increment instruction counter +static inline void instr_count_inc(int instr) +{ +if (!instrumentation_count_instructions) return; +*(instr_offsets.tcg_offset[0]) = instr_offsets.cpustate_offset + sizeof(uint32_t) * instr; +*(instr_offsets.tcg_offset[1]) = instr_offsets.cpustate_offset + sizeof(uint32_t) * instr; +} +*/ + +
instrumentation.h +47 -0
47 lines not on this page · raw message
linux-user/main.c +13 -2
@@ -35,6 +35,8 @@#include "envlist.h" +#include "instrumentation.h" +#define DEBUG_LOGFILE "/tmp/qemu.log"char *exec_path;
@@ -2199,6 +2201,12 @@ static void usage(void) "-p pagesize set the host page size to 'pagesize'\n" "-strace log system calls\n" "\n" +#ifdef TARGET_ARM + "Other:\n" + "-instrcount Count instructions\n" + "-instrcountlog=qemu_log Output counters to qemu-log file\n" + "\n" +#endif"Environment variables:\n" "QEMU_STRACE Print system calls and arguments similar to the\n" " 'strace' program. Enable by setting to any value.\n"
@@ -2341,8 +2349,11 @@ int main(int argc, char **argv, char **e (void )envlist_unsetenv (envlist,"LD_PRELOAD" ); }else if (!strcmp (r,"strace" )) { do_strace =1 ; - }else - { + }else if (!strcmp (r,"instrcount" )) { + instrumentation_count_instructions =1 ; + }else if (!strcmp (r,"instrcountlog=qemu_log" )) { + instrumentation_count_instructions_log =1 ; + }else {usage (); } }
linux-user/syscall.c +66 -0
90 lines not on this page · raw message
target-arm/cpu.h +350 -0
366 lines not on this page · raw message
target-arm/translate.c +765 -36
1,967 lines not on this page · raw message