Skip to contentSkip to conversation

XTF perf - Micro Benchmarks

PATCH New Norbert Manthey 1 participant 5 messages Feb 11, 2019

lore
Share & tools
lore
#0 lore raw

Dear all,

I added a perf category to XTF, and added functions to measure time in the guest. Finally, I added a first micro benchmark that measures the time to call a specified hypercall, and print the average time the hypercall takes.

The added category should be useful to implement further micro benchmarks.

Best,
Norbert

Amazon Development Center Germany GmbH Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#1 lore raw

PATCH 1/4 categories: add perf

As XTF allows to write tests that interact with the hypervisor, we would like to use this capability to implement micro benchmarks, so that we can measure the performance impact of modifications to the hypervisor.

---
 build/common.mk | 2 +-
 xtf-runner      | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
build/common.mk +1 -1
@@ -1,4 +1,4 @@
-ALL_CATEGORIES     := special functional xsa utility in-development
+ALL_CATEGORIES     := special functional xsa utility in-development perf
 
 ALL_ENVIRONMENTS   := pv64 pv32pae hvm64 hvm32pae hvm32pse hvm32
 
xtf-runner +1 -1
@@ -37,7 +37,7 @@ def exit_code(state):
 
 # All test categories
 default_categories     = set(("functional", "xsa"))
-non_default_categories = set(("special", "utility", "in-development"))
+non_default_categories = set(("special", "utility", "in-development", "perf"))
 all_categories         = default_categories | non_default_categories
 
 # All test environments
Signature · 18 lines
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B



_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

PATCH 2/4 time: add stubs

In reply to Norbert Manthey

To measure how long a certain interaction takes, we need time primitives. This commit introduces these primitives, so that future tests can use the gettimeofday function to retrieve the current time.

---
 build/files.mk     |   1 +
 common/time.c      | 203 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 include/xtf/time.h |  66 +++++++++++++++++
 3 files changed, 270 insertions(+)
 create mode 100644 common/time.c
 create mode 100644 include/xtf/time.h
build/files.mk +1 -0
@@ -16,6 +16,7 @@ obj-perarch += $(ROOT)/common/libc/vsnprintf.o
 obj-perarch += $(ROOT)/common/report.o
 obj-perarch += $(ROOT)/common/setup.o
 obj-perarch += $(ROOT)/common/xenbus.o
+obj-perarch += $(ROOT)/common/time.o
 
 obj-perenv += $(ROOT)/arch/x86/decode.o
 obj-perenv += $(ROOT)/arch/x86/desc.o
common/time.c new file +203 -0

203 lines not on this page · raw message

include/xtf/time.h new file +66 -0

66 lines not on this page · raw message

Signature · 18 lines
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B



_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH 3/4 time: provide measurement template

In reply to Norbert Manthey

The added function measure_performance allows to measure the run time of a function, by computing the average time it takes to call that function a given number of retries. The measured total time is returned in nano seconds. Furthermore, the value is printed via printk in a fixed format, to allow processing the output further.

This format is, where average-time provides ns with ps granularity:

perf test_name <average-time> ns

---
 common/time.c      | 36 ++++++++++++++++++++++++++++++++++++
 include/xtf/time.h |  8 +++++++-
 2 files changed, 43 insertions(+), 1 deletion(-)
common/time.c +36 -0
@@ -192,6 +192,42 @@ void msleep(uint64_t t)
     mspin_sleep(t);
 }
 
+long measure_performance(const char* test_name, const char* function_name,
+                         unsigned long retries, to_be_measured call)
+{
+    struct timeval start, end;
+    int rc = 0;
+
+    printk("Start calling %s %lu times\n", function_name, retries);
+
+    /* Perform all calls, measure start and end time */
+    gettimeofday(&start);
+    for(unsigned long counter = 0; counter < retries; ++ counter)
+    {
+        rc = call();
+    }
+    gettimeofday(&end);
+
+    /* Calculate the total number in nano seconds */
+    long total_ns = (end.sec - start.sec)*1000000000 + (end.nsec - start.nsec);
+    long avg_ns = total_ns / retries;
+    long avg_ps = (total_ns / (retries/1000)) % 1000;
+
+    /* Show the result of the last query */
+    printk("%s last result: %d\n", function_name, rc);
+
+    /* Print average time and total time */
+    printk("Avg %s call time: avg: %ld.%s%ld ns total: %ld ns\n",
+           function_name, avg_ns,
+           avg_ps < 10 ? "0" : (avg_ps < 100 ? "0" : ""), avg_ps, total_ns);
+
+    /* Print performance value */
+    printk("perf %s %ld.%s%ld ns\n", test_name, avg_ns,
+           avg_ps < 10 ? "0" : (avg_ps < 100 ? "0" : ""), avg_ps);
+
+    return total_ns;
+}
+
 /*
  * Local variables:
  * mode: C
include/xtf/time.h +7 -1
@@ -49,10 +49,16 @@ void msleep(uint64_t f);
 
 int gettimeofday(struct timeval *tp);
 
-
 /* This returns the current epoch time */
 #define NOW() current_time()
 
+/* Signature of a function to be called for measurement */
+typedef int (*to_be_measured)(void);
+
+/* Measure the time it takes to call the passed function RETRIES times */
+long measure_performance(const char* test_name, const char* function_name,
+                         unsigned long retries, to_be_measured call);
+
 #endif /* XTF_TIME_H */
 
 /*
Signature · 18 lines
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B



_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

PATCH 4/4 perf: measure MMUEXT_MARK_SUPER test

In reply to Norbert Manthey

A first simple test is to call a hypercall in a tight loop. To measure implementation aspects of the hypervisor, we picked a hypercall that is not implemented and hence results in a no-op, namely the hypercall mmuext_op with the command MMUEXT_MARK_SUPER.

---
 tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile |  9 ++++
 tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c   | 66 +++++++++++++++++++++++++++
 2 files changed, 75 insertions(+)
 create mode 100644 tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile
 create mode 100644 tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c
tests/perf-PV-MMUEXT_MARK_SUPER-noop/Makefile new file +9 -0
new file mode 100644
@@ -0,0 +1,9 @@
+include $(ROOT)/build/common.mk
+
+NAME      := perf-PV-MMUEXT_MARK_SUPER-noop
+CATEGORY  := perf
+TEST-ENVS := pv64
+
+obj-perenv += main.o
+
+include $(ROOT)/build/gen.mk
tests/perf-PV-MMUEXT_MARK_SUPER-noop/main.c new file +66 -0

66 lines not on this page · raw message

Signature · 18 lines
2.7.4




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrer: Christian Schlaeger, Ralf Herbrich
Ust-ID: DE 289 237 879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B



_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel