XTF: add micro benchmarks
cover XTF: add micro benchmarks
Dear all,
I added a benchmark category to XTF, and added functions to measure time in the guest. Finally, I added a first micro benchmark that measures the time to call a specified hypercall, and print the average time the hypercall takes.
The added category should be useful to implement further micro benchmarks. I already implemented a few more that I will publish once the environment is agreed on.
Best,
Norbert
Norbert Manthey (4):
categories: add benchmark
time: add stubs
time: provide measurement template
perf: measure MMUEXT_MARK_SUPER test
build/common.mk | 2 +- build/files.mk | 1 + common/time.c | 279 ++++++++++++++++++ include/xtf/time.h | 81 +++++ tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile | 9 + tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c | 80 +++++ xtf-runner | 2 +- 7 files changed, 452 insertions(+), 2 deletions(-) create mode 100644 common/time.c create mode 100644 include/xtf/time.h create mode 100644 tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile create mode 100644 tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c
Signature · 20 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/4 categories: add benchmark
As XTF allows to write tests that interact with the hypervisor, we would like to use this capability to implement micro benchmarks, so that we can measure the performance impact of modifications to the hypervisor.
This change introduces a category benchmark, which can be used as container for tests of this kind.
--- build/common.mk | 2 +- xtf-runner | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
build/common.mk +1 -1
@@ -1,4 +1,4 @@ -ALL_CATEGORIES := special functional xsa utility in-development +ALL_CATEGORIES := special functional xsa utility in-development benchmark ALL_ENVIRONMENTS := pv64 pv32pae hvm64 hvm32pae hvm32pse hvm32
xtf-runner +1 -1
@@ -43,7 +43,7 @@ def exit_code(state): # All test categories default_categories = set(("functional", "xsa")) -non_default_categories = set(("special", "utility", "in-development")) +non_default_categories = set(("special", "utility", "in-development", "benchmark")) all_categories = default_categories | non_default_categories # All test environments
Signature · 20 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/4 time: add stubs
In reply to Norbert MantheyTo measure how long a certain interaction takes, we need time primitives. This commit introduces these primitives, so that future tests can use the gettimeofday function to retrieve the current time.
--- build/files.mk | 1 + common/time.c | 203 +++++++++++++++++++++++++++++++++++++++++++++ include/xtf/time.h | 66 +++++++++++++++ 3 files changed, 270 insertions(+) create mode 100644 common/time.c create mode 100644 include/xtf/time.h
build/files.mk +1 -0
@@ -16,6 +16,7 @@ obj-perarch += $(ROOT)/common/libc/vsnprintf.o obj-perarch += $(ROOT)/common/report.o obj-perarch += $(ROOT)/common/setup.o obj-perarch += $(ROOT)/common/xenbus.o +obj-perarch += $(ROOT)/common/time.o obj-perenv += $(ROOT)/arch/x86/decode.o obj-perenv += $(ROOT)/arch/x86/desc.o
common/time.c new file +203 -0
203 lines not on this page · raw message
include/xtf/time.h new file +66 -0
66 lines not on this page · raw message
Signature · 20 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/4 time: provide measurement template
In reply to Norbert MantheyThe added function measure_performance allows to measure the run time of a function, by computing the average time it takes to call that function a given number of retries. The measured total time is returned in nano seconds. Furthermore, the value is printed via printk in a fixed format, to allow processing the output further.
This format is, where average-time provides ns with ps granularity:
perf test_name <average-time> ns
--- common/time.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++ include/xtf/time.h | 15 +++++++++ 2 files changed, 91 insertions(+)
common/time.c +76 -0
@@ -192,6 +192,82 @@ void msleep(uint64_t t)mspin_sleep (t); } +unsigned long cycles2nsec (uint64_t cycles) +{ +return scale_delta (cycles, + shared_info.vcpu_info[0 ].time.tsc_to_system_mul, + shared_info.vcpu_info[0 ].time.tsc_shift); +} + +unsigned long measure_performance (const char * test_name, +const char * function_name, +function_under_test_t call, +unsigned long measure_seconds, +unsigned long callibration_calls, +int print_times) +{ +unsigned long start, end;// time stamps for before and after the execution +unsigned long calibration_ns, total_measured_ns, avg_ns, avg_ps_fraction; +unsigned long counter; +unsigned long measure_calls; +int rc =0 ; + + +/* Calibrate by measuring time for given amount of callibration calls*/ +printk ("Calibrate %s by calling %lu times\n" , function_name, + callibration_calls); + start =rdtscp (); +for (counter =0 ; counter < callibration_calls; ++ counter) + { + rc =call (); + } + end =rdtscp (); + +/* Calculate the total number in nano seconds */ + calibration_ns =cycles2nsec (end - start); + +/* Calculate number of calls for about measure_seconds based on +(callibration_calls / calibration_ns) equals +(measure_calls / (measure_seconds * 1000 * 1000 * 1000)) */ + measure_calls = ((measure_seconds *1000UL *1000UL *1000UL ) + / calibration_ns) * callibration_calls; + +printk ("Measure %s by calling %lu times\n" , function_name, measure_calls); + +/* Perform all calls, measure start and end time */ + start =rdtscp (); +for (counter =0 ; counter < measure_calls; ++ counter) + { + rc =call (); + } + end =rdtscp (); + +/* Calculate the total number in nano seconds */ + total_measured_ns =cycles2nsec (end - start); + avg_ns = total_measured_ns / measure_calls; + avg_ps_fraction = (total_measured_ns / (measure_calls/1000 )) %1000 ; + +if (print_times) + { +/* Show the result of the last query */ +printk ("%s's last return value: %d\n" , function_name, rc); + +/* Print average time and total time */ +printk ("Avg %s call time: avg: %ld.%s%ld ns total: %ld ns\n" , + function_name, avg_ns, + avg_ps_fraction <10 ?"00" : (avg_ps_fraction <100 ?"0" :"" ), + avg_ps_fraction, total_measured_ns); + +/* Print performance value */ +printk ("perf %s %ld.%s%ld ns\n" , test_name, avg_ns, + avg_ps_fraction <10 ?"00" : (avg_ps_fraction <100 ?"0" :"" ), + avg_ps_fraction); + } + +/* Return average run time in pico seconds */ +return avg_ns *1000 + avg_ps_fraction; +} +/* * Local variables: * mode: C
include/xtf/time.h +15 -0
@@ -53,6 +53,21 @@ int gettimeofday(struct timeval *tp);/* This returns the current epoch time */ #define NOW () current_time() +unsigned long cycles2nsec (uint64_t cycles); + +/* Signature of a function to be called for measurement */ +typedef int (*function_under_test_t )(void ); + +/* Measure the time it takes to call the passed function. Measure for a given +amount of time after calibrating for a given amount of calls. Returns the +average run time of the measure call in pico seconds. */ +unsigned long measure_performance (const char * test_name, +const char * function_name, +function_under_test_t call, +unsigned long measure_seconds, +unsigned long callibration_calls, +int print_times); +#endif /* XTF_TIME_H */ /*
Signature · 20 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 4/4 perf: measure MMUEXT_MARK_SUPER test
In reply to Norbert MantheyA first simple test is to call a hypercall in a tight loop. To measure implementation aspects of the hypervisor, we picked a hypercall that is not implemented and hence results in a no-op, namely the hypercall mmuext_op with the command MMUEXT_MARK_SUPER.
The test calibrates the execution time for 1000 calls to the hypercall, and next calculates the number of calls to take about 5 minutes.
--- tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile | 9 +++ tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c | 80 +++++++++++++++++++ 2 files changed, 89 insertions(+) create mode 100644 tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile create mode 100644 tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c
tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile new file +9 -0
@@ -0,0 +1,9 @@ +include $(ROOT)/build/common.mk + +NAME := perf-PV-MMUEXT_MARK_SUPER-noop +CATEGORY := benchmark +TEST-ENVS := pv64 + +obj-perenv += main.o + +include $(ROOT)/build/gen.mk
tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c new file +80 -0
80 lines not on this page · raw message
Signature · 20 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel