Skip to contentSkip to conversation

Running benchmarks via OSSTest

RFC PATCH In review Dario Faggioli 3 participants 21 messages Jun 26, 2014 – Jul 16, 2014

lore
Share & tools
lore
#0 lore raw

cover Running benchmarks via OSSTest

Hello Everyone,

This is something I've been working in the latest few days, and it's about making it possible to run benchmarks via OSSTest. I've (of course) tested it in standalone mode, but nothing forbids to integrate it in our push gate, as we were saying at the hackathon. Of course, there are a lot of things to be decided. Where to run benchmarks, which benchmarks, how frequently, what to do with the results, etc, but it's a start! :-P

So, in standalone mode, after applying the patches, you can do the following, and you'll have a PV guest running unixbench for a while, and the results fetched for you on the OSSTest standalone controller stash area:

# Retrieve the benchmark archive
./mg-unixbench-download
# Reset, create the benchmarking flight, install the host and build Xen and Linux ./standalone-reset -t bench
./standalone run-job -R -h ghoul3 build-amd64 # consider --lvextendmax=XXGB, if necessary
./standalone run-job -h ghoul3 build-amd64-pvops
# Or set-paths --if build happened already-- like this:
#./standalone set-paths build-amd64
#./standalone set-paths build-amd64-pvops
# Install Xen on the host
./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-xen-install ./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-host-reboot # Install (and start) the guest
./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-debian-install ./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-debian-fixup debian ./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-guest-start debian # Prepare the benchmarking environment in the guest
./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-unixbench-prep debian # Run the benchmark in the guest
./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-unixbench-run debian # Retrieve the benchmark results:
./standalone run-test -h ghoul3 bench-unixbench-amd64-amd64-xl-credit ts-unixbench-reslts debian

A few jobs are being created. In case you want to use one, instead of invoking all the ts-s, run the following:

# Retrieve the benchmark archive
./mg-unixbench-download
# Reset, create the benchmarking flight, install the host and build Xen and Linux ./standalone-reset -t bench
./standalone run-job -R -h ghoul3 build-amd64 # consider --lvextendmax=XXGB, if necessary
./standalone run-job -h ghoul3 build-amd64-pvops
# Run the benchmarking job:
./standalone run-job -h ghoul3 bench-unixbench-amd64-amd64-xl-credit2

Almost all the loginc to use an HVM guest are there, a just need a few fixups in ts-debian-hvm-install that I'll submit as a separate patch.

I tested both the ways in standalone mode, and it seems to be working here.

And that's it... So, what do you think about it? IanC? IanJ?

I now it's still a sort of a prototype, but I'd love to hear your thoughts before proceeding any further.

One thing about the last two patches. They are there to show how the code will look like if we take a slightly different path than what's outlined in the first part of the series. The idea is, in fact, to have the mg-xxx-download and the ts-xxx-{prep,run,reslts} scripts to be more generic, accept one (or more) benchmark name(s) as parameter and prepare the target (host or guest) for them. That would make the single scripts slightly more complex, but will avoid having to have 4 scripts (downloading, preparing, running and fetching results) for each new benchmark we want to introduce support for.

Personally, I think I prefer them to be generic, i.e., something similar to what they look like after all the series is applied, as compared to how they look only until patch 7. This is exacly an example of something I'd like to hear your thoughts about before going forward. :-)

The series is also available here:

git://xenbits.xen.org/people/dariof/osstest.git benchmarking-with-osstest-RFC

Regards,
Dario
---
Dario Faggioli (9):
mg-unixbench-download: new script for downloading the unixbench archive
ts-unixbench-prep: prep the environment for running unixbench
ts-unixbench-run: kick off the benchmark on the target
ts-unixbench-reslts: for retrieving the results
sg-run-job: new recipe for a unixbench job
make-bench-flight: to create a benchmarking flight
make-flight: introduce a new -t option
mg-unixbench-download: make it generic
ts-unixbench-prep: make it generic

 ap-common             |    8 ++++
 make-bench-flight     |   92 ++++++++++++++++++++++++++++++++++++++++++++++
 mg-unixbench-download |   49 ++++++++++++++++++++++++
 sg-run-job            |   28 ++++++++++++++
 standalone-reset      |    9 +++-
 ts-unixbench-prep     |   99 +++++++++++++++++++++++++++++++++++++++++++++++++
 ts-unixbench-reslts   |   56 ++++++++++++++++++++++++++++
 ts-unixbench-run      |   45 ++++++++++++++++++++++
 8 files changed, 383 insertions(+), 3 deletions(-)
 create mode 100755 make-bench-flight
 create mode 100755 mg-unixbench-download
 create mode 100755 ts-unixbench-prep
 create mode 100755 ts-unixbench-reslts
 create mode 100755 ts-unixbench-run

--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-------------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

#1 lore raw

RFC PATCH 1/9 mg-unixbench-download: new script for downloading the unixbench archive

The script fetches it, and saves it in c{Images}/benchs.

Default values for URL and actual filename are in ap-common, and can be overridden as usual.

XXX: this may or may not be something suitable to do in a
function in cri-common, called by ap-fetch-version. It's not a git tree, and it's not something that changes much so I think it's fine like it is in this patch. If it is not, advise otherwise and I'll cope.

---
 ap-common             |    4 ++++
 mg-unixbench-download |   34 ++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)
 create mode 100755 mg-unixbench-download
ap-common +4 -0
index ea4b04c..000ca69 100644
@@ -136,3 +136,7 @@ info_linux_tree () {
 
 	return 0
 }
+
+# UnixBench. By default we try to grab version 5.1.3
+: ${REMOTEFILE_UNIXBENCH:=http://byte-unixbench.googlecode.com/files/unixbench-5.1.3.tgz}
+: ${LOCALFILE_UNIXBENCH:=unixbench.tgz}
mg-unixbench-download new file +34 -0
new file mode 100755
index 0000000..891b0b1
@@ -0,0 +1,34 @@
+#!/bin/bash
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+set -e
+
+if [ -f standalone.config ] ; then
+    . standalone.config
+fi
+
+. cri-getconfig
+. ap-common
+
+fail () { echo >&2 "$0: $1"; exit 1; }
+
+images=`getconfig Images`;
+dstdir="${images}/benchs"
+mkdir -p $dstdir
+
+wget ${REMOTEFILE_UNIXBENCH} -O ${dstdir}/${LOCALFILE_UNIXBENCH} || \
+    fail "failed downloading the benchmark"
#2 lore raw

Re: [PATCH RFC 1/9] mg-unixbench-download: new script for downloading the unixbench archive

Quoted 4 lines · Dario Faggioli

The script fetches it, and saves it in c{Images}/benchs.

Default values for URL and actual filename are in ap-common, and can be overridden as usual.

ap-* are to do with the pushing machinery, so I'm not sure this is the right place for it.

I think it would be OK to hardcode the URL into the script and put the version number in the config file (e.g. production-config). Perhaps with a default (in the script).

Quoted 5 lines

XXX: this may or may not be something suitable to do in a
function in cri-common, called by ap-fetch-version. It's not a git tree, and it's not something that changes much so I think it's fine like it is in this patch. If it is not, advise otherwise and I'll cope.

The usual approach would be to have a setup script (as you have done here) to populate an output directory rather than to run the dl on every run (as you propose in this comment, I think).

All ones for Ian J though really.

Ian

#3 lore raw

Re: [PATCH RFC 1/9] mg-unixbench-download: new script for downloading the unixbench archive

Quoted 9 lines · Ian Campbell
Quoted 4 lines · Dario Faggioli

The script fetches it, and saves it in c{Images}/benchs.

Default values for URL and actual filename are in ap-common, and can be overridden as usual.

ap-* are to do with the pushing machinery, so I'm not sure this is the right place for it.

Right. Sorry then, I must have missed this.

Quoted 4 lines

I think it would be OK to hardcode the URL into the script and put the version number in the config file (e.g. production-config). Perhaps with a default (in the script).

I'll do it this way.

Quoted 10 lines
Quoted 5 lines

XXX: this may or may not be something suitable to do in a
function in cri-common, called by ap-fetch-version. It's not a git tree, and it's not something that changes much so I think it's fine like it is in this patch. If it is not, advise otherwise and I'll cope.

The usual approach would be to have a setup script (as you have done here) to populate an output directory rather than to run the dl on every run (as you propose in this comment, I think).

Great, less (re)work! :-)

Thanks and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#4 lore raw

RFC PATCH 2/9 ts-unixbench-prep: prep the environment for running unixbench

In reply to Dario Faggioli

by installing some dependencies, shipping the archive, untaring and building the sources.

This accepts two parametrs, in the form 'host=somehost someguest', as most of the ts-guest-xxx scripts. If only the first one is provided, it must be 'host=somehost', and the script will prep the host.

XXX: I see it useful that this works for the host too, as
at some point we may want to run benchmarks on dom0 or even bare metal.

---
 ts-unixbench-prep |   79 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 ts-unixbench-prep
ts-unixbench-prep new file +79 -0
new file mode 100755
index 0000000..607d1a8
@@ -0,0 +1,79 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::TestSupport;
+use File::Basename;
+
+use feature 'switch';
+
+tsreadconfig();
+
+# what we expect as argument list is:
+#  host=<somehost> [<someguest>]
+our $ho= selecthost($ARGV[0]); shift @ARGV;
+our $gho= $ho;
+if ( (0+@ARGV) == 1) {
+  $gho= selectguest($ARGV[0], $ho);
+  my $err= guest_check_ip($gho);
+  die "$err $gho->{Name}" if defined $err;
+}
+
+logm("Prep the environment for running unixbench");
+
+# packages needed to build and run
+target_install_packages_norec($gho, qw(build-essential libx11-dev
+                                       libgl1-mesa-dev libxext-dev));
+
+# Ship the benchmark to the target machine. By default, we expect to find
+# UnixBench 5.1.3, stored in $c{Images}/benchs/unixbench.tgz.
+# To use something different, define r{'unixbench_file'}.
+#
+# We also assume we're dealing with a tarball, and we try to figure out
+# whether or not it is compressed, and if yes, how. Check is rather basic,
+# relying only on the file extension.
+#
+my $unixbench_file= (defined($r{'unixbench_file'})) ? $r{'unixbench_file'} :
+    "$c{Images}/benchs/unixbench.tgz";
+target_putfile_root($gho, 60, "$unixbench_file", "/root");
+
+my @exts= qw(.tar .tgz .tbz2. .bz2 .gzip);
+my $extract_cmd= 'xf';
+my ($name, $dir, $ext) = fileparse($unixbench_file, @exts);
+
+given ($ext) {
+  $extract_cmd= 'xf' when ('.tar');
+  $extract_cmd= 'zxf' when ['.tgz', '.gzip'];
+  $extract_cmd= 'jxf' when  ['.tbz2', '.bz2'];
+  default { die "unrecognised file extension $ext" }
+}
+
+# Prepare the target, by installing dependencies, and build the benchmark
+target_install_packages_norec($gho, qw(build-essential libx11-dev
+                                       libgl1-mesa-dev libxext-dev
+                                       x11-apps));
+target_cmd_root($gho, <<END, 200);
+        set -ex
+        rm -rf /root/unixbench/
+        mkdir /root/unixbench
+        tar $extract_cmd /root/$name$ext -C /root/unixbench --strip-components=1
+        cd /root/unixbench
+        sed -e "s/^# GRAPHIC_TESTS =/GRAPHIC_TESTS =/" -i Makefile
+        make
+END
#5 lore raw

Re: [PATCH RFC 2/9] ts-unixbench-prep: prep the environment for running unixbench

Quoted 1 line · Dario Faggioli

by installing some dependencies,

this isn't currently possible due to the (possibly futuristic) potential to share build hosts. ts-xen-build-prep is special in this regard (AIUI). I don't see you installing anything which would be so intolerable to install on every machine.

I have an old patch "apt: lock osstest's usages of apt-get against each other" which is supposed to help with this, but I need to revisit and fix some comments from Ian I think.

Quoted 2 lines

shipping the archive, untaring
and building the sources.

ts-unixbench-build would be the more conventional name I think.

Quoted 8 lines

This accepts two parametrs, in the form 'host=somehost someguest', as most of the ts-guest-xxx scripts. If only the first one is provided, it must be 'host=somehost', and the script will prep the host.

XXX: I see it useful that this works for the host too, as
at some point we may want to run benchmarks on dom0 or even bare metal.

Absolutely. I realise now that the comments about apt install above is only really relevant to the host case.

Quoted 12 lines
---
 ts-unixbench-prep |   79 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 ts-unixbench-prep
ts-unixbench-prep new file +0 -0
new file mode 100755
index 0000000..607d1a8

ts-unixbench-prep line 3

Quoted 3 lines
ts-unixbench-prep new file +3 -0
@@ -0,0 +1,79 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.

Out of date.

ts-unixbench-prep line 30

Quoted 5 lines
ts-unixbench-prep new file +5 -0
+tsreadconfig();
+
+# what we expect as argument list is:
+#  host=<somehost> [<someguest>]
+our $ho= selecthost($ARGV[0]); shift @ARGV;

IIRC Ian just made select* consume ARGV, so the shift wouldn't be needed any more.

ts-unixbench-prep line 32

Quoted 2 lines
ts-unixbench-prep new file +2 -0
+our $gho= $ho;
+if ( (0+@ARGV) == 1) {

Is the 0+ needed, I thought arrays in scalar context evaluated to their length already without forcing it like that.

ts-unixbench-prep line 64

Quoted 5 lines
ts-unixbench-prep new file +5 -0
+given ($ext) {
+  $extract_cmd= 'xf' when ('.tar');
+  $extract_cmd= 'zxf' when ['.tgz', '.gzip'];
+  $extract_cmd= 'jxf' when  ['.tbz2', '.bz2'];
+  default { die "unrecognised file extension $ext" }

I think "xaf" will automatically handle different compression algs. I'm just not quite sure what it does if the tarball is uncompressed.

ts-unixbench-prep line 73

Quoted 9 lines
ts-unixbench-prep new file +9 -0
+}
+
+# Prepare the target, by installing dependencies, and build the benchmark
+target_install_packages_norec($gho, qw(build-essential libx11-dev
+                                       libgl1-mesa-dev libxext-dev
+                                       x11-apps));
+target_cmd_root($gho, <<END, 200);
+        set -ex
+        rm -rf /root/unixbench/

Ideally you would use the osstest user. I'm not sure if that is set up for guests though, might be host only.

ts-unixbench-prep line 79

Quoted 6 lines
ts-unixbench-prep new file +6 -0
+        mkdir /root/unixbench
+        tar $extract_cmd /root/$name$ext -C /root/unixbench --strip-components=1
+        cd /root/unixbench
+        sed -e "s/^# GRAPHIC_TESTS =/GRAPHIC_TESTS =/" -i Makefile
+        make
+END

Ian.

#6 lore raw

Re: [PATCH RFC 2/9] ts-unixbench-prep: prep the environment for running unixbench

Ian Campbell writes ("Re: [PATCH RFC 2/9] ts-unixbench-prep: prep the environment for running unixbench"):

Quoted 7 lines · Dario Faggioli
Quoted 1 line

by installing some dependencies,

this isn't currently possible due to the (possibly futuristic) potential to share build hosts. ts-xen-build-prep is special in this regard (AIUI). I don't see you installing anything which would be so intolerable to install on every machine.

We do actually share build hosts right now. I think you meant share test hosts.

Quoted 3 lines

I have an old patch "apt: lock osstest's usages of apt-get against each other" which is supposed to help with this, but I need to revisit and fix some comments from Ian I think.

That's what I remember.

I'll try to look at the rest of Darios series RSN.

Ian.

#7 lore raw

Re: [PATCH RFC 2/9] ts-unixbench-prep: prep the environment for running unixbench

In reply to Ian Campbell
Quoted 12 lines · Ian Campbell
Quoted 1 line · Dario Faggioli

by installing some dependencies,

this isn't currently possible due to the (possibly futuristic) potential to share build hosts. ts-xen-build-prep is special in this regard (AIUI). I don't see you installing anything which would be so intolerable to install on every machine.

I have an old patch "apt: lock osstest's usages of apt-get against each other" which is supposed to help with this, but I need to revisit and fix some comments from Ian I think.

Mm... I'd be fine with installing everything on every test host. However, as we grow the number of supported benchmark, dependencies will also grow. E.g., if we want to run SpecJBB, we'll need JVM stuff. If we want to run even a subset of Phoronix test suite, we'll need a lot of stuff, I think... Is this still acceptable?

Quoted 5 lines
Quoted 2 lines

shipping the archive, untaring
and building the sources.

ts-unixbench-build would be the more conventional name I think.

Indeed. Fact is, there are benchmarks that require building (e.g., this one), and benchmark that does not (e.g., kernbench).

I thought it would be good to have all the prepping script to have a similar name (if not to include all the prep work in just one ts-bench-prep that takes the benchmark name as a parameter! Look at the very last patch of the series).

That's the reason for the 'prep' instead than 'build'. What do you think?

Quoted 5 lines
Quoted 5 lines

This accepts two parametrs, in the form 'host=somehost someguest', as most of the ts-guest-xxx scripts. If only the first one is provided, it must be 'host=somehost', and the script will prep the host.

Quoted 7 lines
Quoted 3 lines

XXX: I see it useful that this works for the host too, as
at some point we may want to run benchmarks on dom0 or even bare metal.

Absolutely. I realise now that the comments about apt install above is only really relevant to the host case.

Which would make the 'path' for running benches on host and guest a bit different, but that's certainly fine. If we're installing *everything* on hosts, I've got no problem making this prepping scripts available for guests only.

Thanks and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#8 lore raw

Re: [PATCH RFC 2/9] ts-unixbench-prep: prep the environment for running unixbench

In reply to Ian Campbell
Quoted 5 lines · Ian Campbell
Quoted 5 lines
---
 ts-unixbench-prep |   79 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 ts-unixbench-prep
Quoted 12 lines

ts-unixbench-prep line 3

Quoted 9 lines
ts-unixbench-prep new file +3 -0
new file mode 100755
index 0000000..607d1a8
@@ -0,0 +1,79 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.

Out of date.

Oops. :-)

Quoted 9 lines

ts-unixbench-prep line 30

Quoted 5 lines
ts-unixbench-prep new file +5 -0
+tsreadconfig();
+
+# what we expect as argument list is:
+#  host=<somehost> [<someguest>]
+our $ho= selecthost($ARGV[0]); shift @ARGV;

IIRC Ian just made select* consume ARGV, so the shift wouldn't be needed any more.

Yeah, I saw it. I'll build on top of that as far as it will hit the repo.

Quoted 6 lines

ts-unixbench-prep line 32

Quoted 2 lines
ts-unixbench-prep new file +2 -0
+our $gho= $ho;
+if ( (0+@ARGV) == 1) {

Is the 0+ needed, I thought arrays in scalar context evaluated to their length already without forcing it like that.

Ehm, my Perl-fu is still very poor. ISTR trying and having some issues, but yeah, I'll give it another shoot at doing things properly. Sorry :-)

Quoted 9 lines

ts-unixbench-prep line 64

Quoted 5 lines
ts-unixbench-prep new file +5 -0
+given ($ext) {
+  $extract_cmd= 'xf' when ('.tar');
+  $extract_cmd= 'zxf' when ['.tgz', '.gzip'];
+  $extract_cmd= 'jxf' when  ['.tbz2', '.bz2'];
+  default { die "unrecognised file extension $ext" }

I think "xaf" will automatically handle different compression algs. I'm just not quite sure what it does if the tarball is uncompressed.

Oh, I did not know that, I'll give it a try. About the uncompressed tarball case, I don't think we should worry much about being too flexible here. Perhaps what I was trying to do above is already too much. I mean, we expect the file to have a specific name, and to be in a specific place already, we can well ask for it to be compressed, and bail if it is not. What do you think?

Quoted 4 lines
Quoted 4 lines
ts-unixbench-prep new file +4 -0
+}
+
+# Prepare the target, by installing dependencies, and build the benchmark
+target_install_packages_norec($gho, qw(build-essential libx11-dev
Quoted 9 lines

ts-unixbench-prep line 73

Quoted 5 lines
ts-unixbench-prep new file +5 -0
+                                       libgl1-mesa-dev libxext-dev
+                                       x11-apps));
+target_cmd_root($gho, <<END, 200);
+        set -ex
+        rm -rf /root/unixbench/

Ideally you would use the osstest user. I'm not sure if that is set up for guests though, might be host only.

I don't recall that either, and I'll check. There are benchmarks, though, that wants to run as root... I guess that is not a big deal, at least for guests, is it? For hosts, I see it could be a bigger issue...

Thanks again and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#9 lore raw

RFC PATCH 3/9 ts-unixbench-run: kick off the benchmark on the target

In reply to Dario Faggioli

by simply running it, as it requires.

There is a runvar called 'unixbench_params', for specifying the arguments.

---
 ts-unixbench-run |   45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)
 create mode 100755 ts-unixbench-run
ts-unixbench-run new file +45 -0
new file mode 100755
index 0000000..dc81a73
@@ -0,0 +1,45 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+# what we expect as argument list is:
+#  host=<somehost> [<someguest>]
+our $ho= selecthost($ARGV[0]); shift @ARGV;
+our $gho= $ho;
+if ( (0+@ARGV) == 1) {
+  $gho= selectguest($ARGV[0], $ho);
+  my $err= guest_check_ip($gho);
+  die "$err $gho->{Name}" if defined $err;
+}
+
+my $args= $r{ 'unixbench_params' };
+
+logm("Run unixbench with following parameters: $args");
+
+# XXX Arbitrary timeout...
+target_cmd_root($gho, <<END, 20000);
+        set -ex
+        cd /root/unixbench/
+        rm -rf results/*
+        time ./Run $args
+END
#10 lore raw

RFC PATCH 4/9 ts-unixbench-reslts: for retrieving the results

In reply to Dario Faggioli

and store them in $c{Stash}/$flight/$job/results.

XXX: - I think $c{Stash}/<flight>/<job>/results is a fine
place (very similar to what build jobs do, IMO). If a different location should be used, please advise.
- This is, for now, only fetching the results. Of course we'd like to do something with them (printing, plotting, storing for long term usage or comparison, etc.). Let's have a discussion about this and I'll implementi it too

---
 ts-unixbench-reslts |   56 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)
 create mode 100755 ts-unixbench-reslts
ts-unixbench-reslts new file +56 -0
new file mode 100755
index 0000000..37bcfc6
@@ -0,0 +1,56 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use DBI;
+use IO::File;
+use POSIX;
+use Osstest::TestSupport;
+use File::Basename;
+
+tsreadconfig();
+
+# what we expect as argument list is:
+#  host=<somehost> [<someguest>]
+our $ho= selecthost($ARGV[0]); shift @ARGV;
+our $gho= $ho;
+if ( (0+@ARGV) == 1) {
+  $gho= selectguest($ARGV[0], $ho);
+  my $err= guest_check_ip($gho);
+  die "$err $gho->{Name}" if defined $err;
+}
+
+my $lresults_dir= "$stash/results";
+my $rresults_dir= "/root/unixbench/results";
+ensuredir($lresults_dir);
+
+my $resultspat= "$rresults_dir/$gho->{Guest}*-01*";
+
+my $rresfiles_str= target_cmd_output_root($gho, <<END);
+    chmod a+r $resultspat 2>&1 ||:
+    echo $resultspat
+END
+
+my @rresfiles= split(' ', $rresfiles_str);
+
+foreach (@rresfiles) {
+  my ($name, $dir, $ext) = fileparse($_, qr/\.[^.]*/);
+  target_getfile_root($gho, 60, "$_", "$lresults_dir/$gho->{Name}-unixbench$ext");
+}
+
+logm("results captured to $lresults_dir");
#11 lore raw

Re: [PATCH RFC 4/9] ts-unixbench-reslts: for retrieving the results

ts-unixbench-reslts line 53

Quoted 3 lines · Dario Faggioli
ts-unixbench-reslts new file +3 -0
+foreach (@rresfiles) {
+  my ($name, $dir, $ext) = fileparse($_, qr/\.[^.]*/);
+  target_getfile_root($gho, 60, "$_", "$lresults_dir/$gho->{Name}-unixbench$ext");

There's a general need for a target_getfile_(root_)stash I think, as an analogue to e.g. target_putfilecontents_stash.

Quoted 4 lines
ts-unixbench-reslts new file +3 -0
+}
+
+logm("results captured to $lresults_dir");
#12 lore raw

Re: [PATCH RFC 4/9] ts-unixbench-reslts: for retrieving the results

Quoted 9 lines · Ian Campbell

ts-unixbench-reslts line 53

Quoted 3 lines · Dario Faggioli
ts-unixbench-reslts new file +3 -0
+foreach (@rresfiles) {
+  my ($name, $dir, $ext) = fileparse($_, qr/\.[^.]*/);
+  target_getfile_root($gho, 60, "$_", "$lresults_dir/$gho->{Name}-unixbench$ext");

There's a general need for a target_getfile_(root_)stash I think, as an analogue to e.g. target_putfilecontents_stash.

I certainly can look into implementing that one.

Thanks and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#13 lore raw

RFC PATCH 5/9 sg-run-job: new recipe for a unixbench job

In reply to Dario Faggioli

Installs and run unixbench inside a PV and an HVM guest.

XXX - The HVM case needs that a few fixup are performed
after ts-debian-hvm-install. I'll do that as a separate and independent patch.
- This is mostly a proof of concept. I think the recipes it defines are actually useful, but of course we can discuss of what is the best course of actions, depending how we want to achieve.

---
 sg-run-job |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
sg-run-job +28 -0
index e903869..36b6870 100755
@@ -317,6 +317,34 @@ proc run-job/test-rumpuserxen {} {
     run-ts . =   ts-guest-destroy          + host $g
 }
 
+#-------- benchmarks --------
+
+proc bench-unixbench-guest {g} {
+    run-ts . = ts-unixbench-prep   + host $g
+    run-ts . = ts-unixbench-run    + host $g
+    run-ts . = ts-unixbench-reslts + host $g
+    run-ts . = ts-guest-stop       + host $g
+}
+
+proc bench-unixbench-pv {} {
+    run-ts . = ts-debian-install + host
+    run-ts . = ts-debian-fixup   + host debian
+    run-ts . = ts-guest-start    + host debian
+    bench-unixbench-guest debian
+}
+
+proc bench-unixbench-hvm {} {
+    run-ts . = ts-debian-hvm-install
+    bench-unixbench-guest debianhvm
+}
+
+proc need-hosts/bench-unixbench {} { return host }
+proc run-job/bench-unixbench {} {
+    bench-unixbench-pv
+    #bench-unixbench-hvm TODO: a few fixup needed, will submit as
+    #                          separate patches.
+}
+
 #---------- builds ----------
 
 proc need-hosts/build {} { return BUILD }
#14 lore raw

RFC PATCH 6/9 make-bench-flight: to create a benchmarking flight

In reply to Dario Faggioli

instead of polluting make-flight.

Two bench-unixbench based jobs are created, one using credit, the other credit2.

XXX - Wheter or not to create a new script was discussed
briefly with IanC (the last time I was in Cambridge).
- As for the previous patch, this is mostly a proof of concept... We can do everything here, let's discuss about it!

---
 make-bench-flight |   92 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)
 create mode 100755 make-bench-flight
make-bench-flight new file +92 -0
new file mode 100755
index 0000000..4dd69cc
@@ -0,0 +1,92 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. ap-common
+. cri-common
+. mfi-common
+
+defsuite=`getconfig DebianSuite`
+defguestsuite=`getconfig GuestDebianSuite`
+
+if [ x$buildflight = x ]; then
+
+  if [ "x$BUILD_LVEXTEND_MAX" != x ]; then
+     BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
+  fi
+
+  create_build_jobs
+
+else
+
+  bfi=$buildflight.
+
+fi
+
+job_create_test_filter_callback () {
+    :
+}
+
+test_matrix_branch_filter_callback () {
+    :
+}
+
+do_unixbench_tests () {
+  if [ $xenarch != amd64 -o $dom0arch != amd64 ]; then
+    return
+  fi
+
+  for sched in credit credit2; do
+    job_create_test bench-unixbench-$xenarch$kern-$dom0arch-xl-${sched}        \
+      bench-unixbench xl $xenarch $dom0arch guests_vcpus=4                     \
+      xen_boot_append="sched=${sched}" unixbench_params='-i 5 -c 1 -c 2 -c 4'  \
+      $debian_runvars bios=seabios debianhvm_image=debian-7.2.0-amd64-CD-1.iso \
+      all_hostflags=$most_hostflags
+  done
+}
+
+test_matrix_do_one () {
+  case ${xenarch} in
+  amd64) domUarches="amd64";;
+  i386)  domUarches="";;
+  armhf) domUarches="armhf";;
+  esac
+
+  for domU in $domUarches ; do
+      do_unixbench_tests
+  done
+}
+
+test_matrix_iterate
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
#15 lore raw

RFC PATCH 7/9 make-flight: introduce a new -t option

In reply to Dario Faggioli

to make it possible to call the new make-bench-flight, and generate the benchmarking jobs. It's best combined with -f, so that one can create a benchmarking flight with all the benchmarking jobs.

It is generic, so, when passing '-t sometype', a script called make-sometype-flight is what will be invoked.

XXX Just FTR, this was also part of that conversation
with IanC.

---
 standalone-reset |    9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
standalone-reset +6 -3
index 8555039..f041e6d 100755
@@ -23,14 +23,17 @@ usage(){
 usage: ./standalone-reset [<options>] [<branch> [<xenbranch> [<buildflight>]]]
  branch and xenbranch default, separately, to xen-unstable
 options:
- -f<flight>     generate flight "flight", default is "standalone"
+ -f <flight>      generate flight "flight", default is "standalone"
+ -t <flight_type> generate a different type of flight (it calls
+                  make-<flight_type>-flight)
 END
 }
 
 flight="standalone"
-while getopts "f:" opt; do
+while getopts "f:t:" opt; do
     case "$opt" in
         f) flight=${OPTARG};;
+        t) flight_type="-${OPTARG}";;
         *) usage; exit 1;;
     esac
 done
@@ -140,7 +143,7 @@ fi
 export BUILD_LVEXTEND_MAX
 
 OSSTEST_FLIGHT=$flight \
-./make-flight "$branch" "$xenbranch" play $buildflight >/dev/null
+./make${flight_type}-flight "$branch" "$xenbranch" play $buildflight >/dev/null
 
 #---------- done ----------
#16 lore raw

Re: [PATCH RFC 7/9] make-flight: introduce a new -t option

Quoted 10 lines · Dario Faggioli

to make it possible to call the new make-bench-flight, and generate the benchmarking jobs. It's best combined with -f, so that one can create a benchmarking flight with all the benchmarking jobs.

It is generic, so, when passing '-t sometype', a script called make-sometype-flight is what will be invoked.

XXX Just FTR, this was also part of that conversation
with IanC.

WFM. Perhaps you'd like to plumb this into "standalone make-flight" too?

Ian.

#17 lore raw

Re: [PATCH RFC 7/9] make-flight: introduce a new -t option

Quoted 6 lines · Ian Campbell
Quoted 5 lines · Dario Faggioli

to make it possible to call the new make-bench-flight, and generate the benchmarking jobs. It's best combined with -f, so that one can create a benchmarking flight with all the benchmarking jobs.

Quoted 8 lines
Quoted 5 lines

It is generic, so, when passing '-t sometype', a script called make-sometype-flight is what will be invoked.

XXX Just FTR, this was also part of that conversation
with IanC.

WFM. Perhaps you'd like to plumb this into "standalone make-flight" too?

Sure!

Thanks and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#18 lore raw

RFC PATCH 8/9 mg-unixbench-download: make it generic

In reply to Dario Faggioli

so that it can handle the retrieval of different benchmarks. The name of the benchmark to fetch should be among the supported one, and is passed as an argument.

XXX This is a stab. As detailed in the cover letter, I
just wanted to give an idea of how things would look like, if we take this path. I personally prefer it like this, but let's talk about it.

Of course, if we decide to go this way, the name will be changed from mg-unixbench-download to mg-bench-download, and I'll structure the file like this from the beginning.

---
 ap-common             |    4 ++++
 mg-unixbench-download |   19 +++++++++++++++++--
 2 files changed, 21 insertions(+), 2 deletions(-)
ap-common +4 -0
index 000ca69..ff0f205 100644
@@ -140,3 +140,7 @@ info_linux_tree () {
 # UnixBench. By default we try to grab version 5.1.3
 : ${REMOTEFILE_UNIXBENCH:=http://byte-unixbench.googlecode.com/files/unixbench-5.1.3.tgz}
 : ${LOCALFILE_UNIXBENCH:=unixbench.tgz}
+
+# hackbench. By default, directly from mingo page!
+: ${REMOTEFILE_HACKBENCH:=http://people.redhat.com/mingo/cfs-scheduler/tools/hackbench.c}
+: ${LOCALFILE_HACKBENCH:=hackbench.c}
mg-unixbench-download +17 -2
index 891b0b1..c4d5ab3 100755
@@ -30,5 +30,20 @@ images=`getconfig Images`;
 dstdir="${images}/benchs"
 mkdir -p $dstdir
 
-wget ${REMOTEFILE_UNIXBENCH} -O ${dstdir}/${LOCALFILE_UNIXBENCH} || \
-    fail "failed downloading the benchmark"
+for bench in "$@"; do
+  case $bench in
+    unixbench)
+      bench_url=${REMOTEFILE_UNIXBENCH}
+      bench_dst=${dstdir}/${LOCALFILE_UNIXBENCH}
+      ;;
+    hackbench)
+      bench_url=${REMOTEFILE_HACKBENCH}
+      bench_dst=${dstdir}/${LOCALFILE_HACKBENCH}
+      ;;
+    *)
+      fail "unknown benchmark $bench"
+      ;;
+    esac
+
+    wget ${bench_url} -O ${bench_dst} || fail "failed downloading ${bench}"
+done
#19 lore raw

RFC PATCH 9/9 ts-unixbench-prep: make it generic

In reply to Dario Faggioli

to allow dealing with preparation of multiple benchmarks.

XXX As for the previous patch, this is only to show how
the code will look if we go this way.

---
 ts-unixbench-prep |   94 ++++++++++++++++++++++++++++++++---------------------
 1 file changed, 57 insertions(+), 37 deletions(-)
ts-unixbench-prep +57 -37
index 607d1a8..f0258ab 100755
@@ -26,7 +26,10 @@ use feature 'switch';
 tsreadconfig();
 
 # what we expect as argument list is:
-#  host=<somehost> [<someguest>]
+#  host=<somehost> [<someguest>] <benchname>
+@ARGV= reverse @ARGV;
+our $benchname= $ARGV[0]; shift @ARGV;
+@ARGV= reverse @ARGV;
 our $ho= selecthost($ARGV[0]); shift @ARGV;
 our $gho= $ho;
 if ( (0+@ARGV) == 1) {
@@ -35,45 +38,62 @@ if ( (0+@ARGV) == 1) {
   die "$err $gho->{Name}" if defined $err;
 }
 
-logm("Prep the environment for running unixbench");
+# In general, we know what benchmarks comes in archives. We assume them to be
+# tarballs, and we try to figure out whether or not it is compressed, and if
+# yes, how. Check is rather basic, relying only on the file extension.
+sub archive_extract_cmd($) {
+  my $benchfile_path= @_;
+  my @exts= qw(.tar .tgz .tbz2. .bz2 .gzip);
+  my ($name, $dir, $ext) = fileparse($benchfile_path, @exts);
+  my $cmd;
 
-# packages needed to build and run
-target_install_packages_norec($gho, qw(build-essential libx11-dev
-                                       libgl1-mesa-dev libxext-dev));
+  given ($ext) {
+    $cmd= 'xf' when ('.tar');
+    $cmd= 'zxf' when ['.tgz', '.gzip'];
+    $cmd= 'jxf' when  ['.tbz2', '.bz2'];
+    default { die "unrecognised file extension $ext" }
+  }
+  return $cmd
+}
 
-# Ship the benchmark to the target machine. By default, we expect to find
-# UnixBench 5.1.3, stored in $c{Images}/benchs/unixbench.tgz.
-# To use something different, define r{'unixbench_file'}.
-#
-# We also assume we're dealing with a tarball, and we try to figure out
-# whether or not it is compressed, and if yes, how. Check is rather basic,
-# relying only on the file extension.
-#
-my $unixbench_file= (defined($r{'unixbench_file'})) ? $r{'unixbench_file'} :
-    "$c{Images}/benchs/unixbench.tgz";
-target_putfile_root($gho, 60, "$unixbench_file", "/root");
+# Ship the benchmark to the target machine and prepare the environment
+logm("Prep the environment for running $benchname");
 
-my @exts= qw(.tar .tgz .tbz2. .bz2 .gzip);
-my $extract_cmd= 'xf';
-my ($name, $dir, $ext) = fileparse($unixbench_file, @exts);
+if ($benchname eq "unixbench") {
+  # By default, we expect to find UnixBench 5.1.3, stored in
+  # $c{Images}/benchs/unixbench.tgz. To use something different,
+  # define r{'unixbench_file'}.
+  my $unixbench_file= (defined($r{'unixbench_file'})) ? $r{'unixbench_file'} :
+      "$c{Images}/benchs/unixbench.tgz";
+  target_putfile_root($gho, 60, "$unixbench_file", "/root");
 
-given ($ext) {
-  $extract_cmd= 'xf' when ('.tar');
-  $extract_cmd= 'zxf' when ['.tgz', '.gzip'];
-  $extract_cmd= 'jxf' when  ['.tbz2', '.bz2'];
-  default { die "unrecognised file extension $ext" }
-}
+  my $extract_cmd= archive_extract_cmd($unixbench_file);
+
+  # Packages needed to build and run
+  target_install_packages_norec($gho, qw(build-essential libx11-dev
+                                         libgl1-mesa-dev libxext-dev));
 
-# Prepare the target, by installing dependencies, and build the benchmark
-target_install_packages_norec($gho, qw(build-essential libx11-dev
-                                       libgl1-mesa-dev libxext-dev
-                                       x11-apps));
-target_cmd_root($gho, <<END, 200);
-        set -ex
-        rm -rf /root/unixbench/
-        mkdir /root/unixbench
-        tar $extract_cmd /root/$name$ext -C /root/unixbench --strip-components=1
-        cd /root/unixbench
-        sed -e "s/^# GRAPHIC_TESTS =/GRAPHIC_TESTS =/" -i Makefile
-        make
+  target_cmd_root($gho, <<END, 200);
+          set -ex
+          rm -rf /root/unixbench/
+          mkdir /root/unixbench
+          tar $extract_cmd /root/$name$ext -C /root/unixbench --strip-components=1
+          cd /root/unixbench
+          sed -e "s/^# GRAPHIC_TESTS =/GRAPHIC_TESTS =/" -i Makefile
+          make
 END
+} elsif ($benchname eq "hackbench") {
+  # In case of hackbench, it is just a .c file.
+  my $hackbench_file= (defined($r{'hackbench_file'})) ? $r{'hackbench_file'} :
+      "$c{Images}/benchs/hackbench.c";
+  target_putfile_root($gho, 60, "$hackbench_file", "/root");
+
+  # Packages needed to build and run
+  target_install_packages_norec($gho, qw(build-essential));
+
+  target_cmd_root($gho, <<END, 200);
+          set -ex
+          rm -f /root/hackbench
+          gcc -g -Wall -O2 -o /root/hackbench /root/hackbench.c -lpthread
+END
+}
#20 lore raw

Re: [PATCH RFC 9/9] ts-unixbench-prep: make it generic

Quoted 5 lines · Dario Faggioli

to allow dealing with preparation of multiple benchmarks.

XXX As for the previous patch, this is only to show how
the code will look if we go this way.

Not only it's a stub, it also has a bug in it! :-P

Attached a correct version (and I also updated the branch so that, if you clone it, everything should work fine).

git://xenbits.xen.org/people/dariof/osstest.git benchmarking-with-osstest-RFC

Hopefully, this does not affect too much the review of the series, given the nature of this very patch.

In summary, if you want to try and see how the thing works, either apply only patches 1-7, or, if you want to apply 9 as well, use the version attached to this email. Or just clone the git repo (it has the fixed version patch 9/9).

Sorry and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

commit d61f362e934421a74b2b7238035af3fce6ac307b

Author: Dario Faggioli <[email protected]>
Date:   Wed Jun 25 16:55:56 2014 +0200

ts-unixbench-prep: make it generic

to allow dealing with preparation of multiple benchmarks.

XXX As for the previous patch, this is only to show how
the code will look if we go this way.

ts-unixbench-prep +57 -37
index 607d1a8..55816b7 100755
@@ -26,7 +26,10 @@ use feature 'switch';
 tsreadconfig();
 
 # what we expect as argument list is:
-#  host=<somehost> [<someguest>]
+#  host=<somehost> [<someguest>] <benchname>
+@ARGV= reverse @ARGV;
+our $benchname= $ARGV[0]; shift @ARGV;
+@ARGV= reverse @ARGV;
 our $ho= selecthost($ARGV[0]); shift @ARGV;
 our $gho= $ho;
 if ( (0+@ARGV) == 1) {
@@ -35,45 +38,62 @@ if ( (0+@ARGV) == 1) {
   die "$err $gho->{Name}" if defined $err;
 }
 
-logm("Prep the environment for running unixbench");
+# In general, we know what benchmarks comes in archives. We assume them to be
+# tarballs, and we try to figure out whether or not it is compressed, and if
+# yes, how. Check is rather basic, relying only on the file extension.
+sub archive_extract_cmd($) {
+  my $benchfile_path= "@_";
+  my @exts= qw(.tar .tgz .tbz2. .bz2 .gzip);
+  my ($name, $dir, $ext) = fileparse($benchfile_path, @exts);
+  my $cmd;
 
-# packages needed to build and run
-target_install_packages_norec($gho, qw(build-essential libx11-dev
-                                       libgl1-mesa-dev libxext-dev));
+  given ($ext) {
+    $cmd= 'xf' when ('.tar');
+    $cmd= 'zxf' when ['.tgz', '.gzip'];
+    $cmd= 'jxf' when  ['.tbz2', '.bz2'];
+    default { die "unrecognised file extension $ext" }
+  }
+  return ($cmd, $name, $ext);
+}
 
-# Ship the benchmark to the target machine. By default, we expect to find
-# UnixBench 5.1.3, stored in $c{Images}/benchs/unixbench.tgz.
-# To use something different, define r{'unixbench_file'}.
-#
-# We also assume we're dealing with a tarball, and we try to figure out
-# whether or not it is compressed, and if yes, how. Check is rather basic,
-# relying only on the file extension.
-#
-my $unixbench_file= (defined($r{'unixbench_file'})) ? $r{'unixbench_file'} :
-    "$c{Images}/benchs/unixbench.tgz";
-target_putfile_root($gho, 60, "$unixbench_file", "/root");
+# Ship the benchmark to the target machine and prepare the environment
+logm("Prep the environment for running $benchname");
 
-my @exts= qw(.tar .tgz .tbz2. .bz2 .gzip);
-my $extract_cmd= 'xf';
-my ($name, $dir, $ext) = fileparse($unixbench_file, @exts);
+if ($benchname eq "unixbench") {
+  # By default, we expect to find UnixBench 5.1.3, stored in
+  # $c{Images}/benchs/unixbench.tgz. To use something different,
+  # define r{'unixbench_file'}.
+  my $unixbench_file= (defined($r{'unixbench_file'})) ? $r{'unixbench_file'} :
+      "$c{Images}/benchs/unixbench.tgz";
+  target_putfile_root($gho, 60, "$unixbench_file", "/root");
 
-given ($ext) {
-  $extract_cmd= 'xf' when ('.tar');
-  $extract_cmd= 'zxf' when ['.tgz', '.gzip'];
-  $extract_cmd= 'jxf' when  ['.tbz2', '.bz2'];
-  default { die "unrecognised file extension $ext" }
-}
+  my ($extract_cmd, $name, $ext)= archive_extract_cmd($unixbench_file);
+
+  # Packages needed to build and run
+  target_install_packages_norec($gho, qw(build-essential libx11-dev
+                                         libgl1-mesa-dev libxext-dev));
 
-# Prepare the target, by installing dependencies, and build the benchmark
-target_install_packages_norec($gho, qw(build-essential libx11-dev
-                                       libgl1-mesa-dev libxext-dev
-                                       x11-apps));
-target_cmd_root($gho, <<END, 200);
-        set -ex
-        rm -rf /root/unixbench/
-        mkdir /root/unixbench
-        tar $extract_cmd /root/$name$ext -C /root/unixbench --strip-components=1
-        cd /root/unixbench
-        sed -e "s/^# GRAPHIC_TESTS =/GRAPHIC_TESTS =/" -i Makefile
-        make
+  target_cmd_root($gho, <<END, 200);
+          set -ex
+          rm -rf /root/unixbench/
+          mkdir /root/unixbench
+          tar $extract_cmd /root/$name$ext -C /root/unixbench --strip-components=1
+          cd /root/unixbench
+          sed -e "s/^# GRAPHIC_TESTS =/GRAPHIC_TESTS =/" -i Makefile
+          make
 END
+} elsif ($benchname eq "hackbench") {
+  # In case of hackbench, it is just a .c file.
+  my $hackbench_file= (defined($r{'hackbench_file'})) ? $r{'hackbench_file'} :
+      "$c{Images}/benchs/hackbench.c";
+  target_putfile_root($gho, 60, "$hackbench_file", "/root");
+
+  # Packages needed to build and run
+  target_install_packages_norec($gho, qw(build-essential));
+
+  target_cmd_root($gho, <<END, 200);
+          set -ex
+          rm -f /root/hackbench
+          gcc -g -Wall -O2 -o /root/hackbench /root/hackbench.c -lpthread
+END
+}

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel