Skip to contentSkip to conversation

0/3] Series short description

RFC PATCH In review Dario Faggioli 2 participants 8 messages Dec 5, 2013 – Dec 6, 2013

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 0/3] Series short description

OSSTest: allow for handling multiple guests at the same time

via the ts-debian-install, ts-debian-fixup and
ts-guest-{start,stop,destroy} test scripts.

The idea is to enable something like that:

 $ OSSTEST_JOB=test-amd64-amd64-xl
 $ export OSSTEST_JOB
 $ ./ts-debian-install host=tg03 debian1 debian2 debian3
 $ ./ts-debian-fixup host=tg03 debian1 debian2 debian3
 $ ./ts-guest-start host=tg03 debian1 debian2 debian3
 $ ./ts-guest-stop host=tg03 debian1 debian2
 $ ./ts-guest-destroy host=tg03 debian3

In fact, when wanting to use OSSTest for some fairly complex test or benchamrk that involves more than one or two VMs, I think it could be convinient to install, start, stop, etc., them like this, rather than having to call each script for all the domains.

It should also save some time, when it comes, for example, to starting domains, as the operation is first requsted for all of them, and only after that we start checking wether they're up (the idea being that when we start checking, after having started the last one, the first one have probably finished booting).

It's going to be mostly useful in standalone mode, but I guess it could allow to write more compact recipes too.

I tested this both as reported above and by issueing a full `./sg-run-job test-amd64-amd64-xl'.

I can go ahead and do the same for other ts-* scripts but:
1) I don't think is that much relevant/useful to do something like this in, for instance, ts-guest-migrate or ts-guest-saverestore
2) given my limited perl skills, I though I better ask for some early feedback. :-)

Any comments are welcome.

Regards,
Dario

---

Dario Faggioli (3):
ts-debian-install: support for installing multiple guests
ts-debian-fixup: support multiple guests ts-guest-start, -stop, -destroy: support multiple guests

 ts-debian-fixup   |   30 ++++++++++++++++++---------
 ts-debian-install |   59 +++++++++++++++++++++++++++++++----------------------
 ts-guest-destroy  |   28 +++++++++++++++++++------
 ts-guest-start    |   35 ++++++++++++++++++++++---------
 ts-guest-stop     |   30 ++++++++++++++++++++-------
 5 files changed, 123 insertions(+), 59 deletions(-)
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
#1 lore raw

RFC PATCH 1/1 1/3] ts-debian-install: support for installing multiple guests

so that now, for example, this is possible:

$ OSSTEST_JOB=test-amd64-amd64-xl
$ export OSSTEST_JOB
$ ./ts-debian-install host=tg03 debian1 debian2

This assumes that, if an host is to be explicitly specified, that happens via the first argument and in the form 'host=somehost', as it typically is the case for standalone mode.

If no first argument in 'host=' form is found, we assume no explicit host designation at all, as it typicall is the case for non-standalone mode, and we consider all the parameters a list of guest names. We do this also if the first parametr is just 'host', as this seem to be another way to ask to just use the host from the runvars (typical in non-standalone, but functional in both modes).

---
 ts-debian-install |   59 +++++++++++++++++++++++++++++++----------------------
 1 file changed, 35 insertions(+), 24 deletions(-)
ts-debian-install +35 -24
index 519edc9..a9a89dd 100755
@@ -22,9 +22,14 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($whhost,$gn) = @ARGV;
-$whhost ||= 'host';
-$gn ||= 'debian';
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+
+our (@guests) = @ARGV;
+$guests[0] = 'debian' unless defined(@guests);
 
 our $ho= selecthost($whhost);
 
@@ -32,27 +37,30 @@ our $ram_mb=    512;
 our $swap_mb=  1000;
 our $disk_mb= 10000;
 
-our $guesthost= "$gn.guest.osstest";
-our $gho;
+our $guesthost;
+our %gho;
+
+sub prep ($) {
+    my ($gn) = @_;
 
-sub prep () {
     target_install_packages_norec($ho, qw(lvm2 xen-tools));
 
-    $gho= prepareguest($ho, $gn, $guesthost, 22,
-                       $swap_mb + $disk_mb + 2,
-                       40);
-    target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+    $gho{$gn}= prepareguest($ho, $gn, $guesthost, 22,
+                            $swap_mb + $disk_mb + 2,
+                            40);
+    target_cmd_root($ho, "umount $gho{$gn}->{Lvdev} ||:");
 }
 
-sub ginstall () {
-    my $arch= $r{"$gho->{Guest}_arch"};
+sub ginstall ($) {
+    my ($gn) = @_;
+    my $arch= $r{"$gho{$gn}->{Guest}_arch"};
     my $archarg= defined($arch) ? "--arch $arch" : '';
-    my $gsuite= guest_var($gho,'suite',$c{GuestDebianSuite});
+    my $gsuite= guest_var($gho{$gn},'suite',$c{GuestDebianSuite});
 
-    my $kernpath = guest_var($gho,'kernel_path',$r{xen_kernel_path});
-    my $initrd = guest_var($gho,'initrd_path',$r{xen_initrd_path});
+    my $kernpath = guest_var($gho{$gn},'kernel_path',$r{xen_kernel_path});
+    my $initrd = guest_var($gho{$gn},'initrd_path',$r{xen_initrd_path});
     if (!$kernpath) {
-	my $kernver= guest_var($gho,'kernel_ver',$r{xen_kernel_ver});
+	my $kernver= guest_var($gho{$gn},'kernel_ver',$r{xen_kernel_ver});
 	$kernver ||= target_cmd_output($ho, 'uname -r');
 	$kernpath = "/boot/vmlinuz-$kernver";
 	$initrd ||= "/boot/initrd.img-$kernver";
@@ -65,21 +73,24 @@ sub ginstall () {
     
     target_cmd_root($ho, <<END, 2000);
         xen-create-image \\
-            --dhcp --mac $gho->{Ether} \\
+            --dhcp --mac $gho{$gn}->{Ether} \\
             --memory ${ram_mb}Mb --swap ${swap_mb}Mb \\
             --dist $gsuite \\
             --mirror http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} \\
-            --hostname $gho->{Name} \\
-            --lvm $gho->{Vg} --force \\
+            --hostname $gho{$gn}->{Name} \\
+            --lvm $gho{$gn}->{Vg} --force \\
             --kernel $kernpath \\
             --genpass 0 --password xenroot \\
             $initrd_opt \\
             $archarg
 END
-    my $cfg_xend= "/etc/xen/$gho->{Name}.cfg";
-    store_runvar("$gho->{Guest}_cfgpath", $cfg_xend);
-    store_runvar("$gho->{Guest}_swap_lv", "$gho->{Name}-swap");
+    my $cfg_xend= "/etc/xen/$gho{$gn}->{Name}.cfg";
+    store_runvar("$gho{$gn}->{Guest}_cfgpath", $cfg_xend);
+    store_runvar("$gho{$gn}->{Guest}_swap_lv", "$gho{$gn}->{Name}-swap");
 }
 
-prep();
-ginstall();
+foreach my $g (@guests) {
+    $guesthost = "$g.guest.osstest";
+    prep($g);
+    ginstall($g);
+}
#2 lore raw

RFC PATCH 1/1 2/3] ts-debian-fixup: support multiple guests

In reply to Dario Faggioli

so that now, for example, this is possible:

$ OSSTEST_JOB=test-amd64-amd64-xl
$ export OSSTEST_JOB
$ ./ts-debian-install host=tg03 debian1 debian2
$ ./ts-debian-fixup host=tg03 debian1 debian2

This again assumes that either the host is explicitly specified via a 'host=somehost' first argument, or the argument are all guest names.

---
 ts-debian-fixup |   30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)
ts-debian-fixup +20 -10
index f001418..e281ad7 100755
@@ -22,7 +22,14 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,$gho);
 
 our ($cfgfile,$cfgstash,$cfg);
 
@@ -163,12 +170,15 @@ sub writecfg () {
     target_putfile_root($ho,10, $cfgstash, $cfgfile);
 }
 
-savecfg();
-ether();
-target_kernkind_check($gho);
-access();
-console();
-filesystems();
-otherfixupcfg();
-writecfg();
-unmount();
+foreach my $g (@guests) {
+    ($ho,$gho) = ts_get_host_guest($whhost, $g);
+    savecfg();
+    ether();
+    target_kernkind_check($gho);
+    access();
+    console();
+    filesystems();
+    otherfixupcfg();
+    writecfg();
+    unmount();
+}
#3 lore raw

RFC PATCH 1/1 3/3] ts-guest-start, -stop, -destroy: support multiple guests

In reply to Dario Faggioli

so that now, for example, this is possible:

$ OSSTEST_JOB=test-amd64-amd64-xl
$ export OSSTEST_JOB
$ ./ts-debian-install host=tg03 debian1 debian2 debian3
$ ./ts-debian-fixup host=tg03 debian1 debian2 debian3
$ ./ts-guest-start host=tg03 debian1 debian2 debian3
$ ./ts-guest-stop host=tg03 debian1 debian2
$ ./ts-guest-destroy host=tg03 debian3

This again assumes that either the host is explicitly specified via a 'host=somehost' first argument, or the arguments are all guest names.

---
 ts-guest-destroy |   28 +++++++++++++++++++++-------
 ts-guest-start   |   35 +++++++++++++++++++++++++----------
 ts-guest-stop    |   30 ++++++++++++++++++++++--------
 3 files changed, 68 insertions(+), 25 deletions(-)
ts-guest-destroy +21 -7
index 738650a..794f38d 100755
@@ -22,13 +22,27 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,%gho);
 
-sub destroy () {
-    guest_destroy($ho, $gho);
-    guest_checkrunning($ho, $gho) and die $gho->{Name};
+sub destroy ($) {
+    my ($gn) = @_;
+    guest_destroy($ho, $gho{$gn});
+    guest_checkrunning($ho, $gho{$gn}) and die $gho{$gn}->{Name};
 }
 
-guest_await_dhcp_tcp($gho, 5);
-destroy();
-target_ping_check_down($gho);
+# Let's first destroy all domains and then check they're gone
+foreach my $g (@guests) {
+    ($ho,$gho{$g}) = ts_get_host_guest($whhost, $g);
+    guest_await_dhcp_tcp($gho{$g}, 5);
+    destroy($g);
+}
+foreach my $g (@guests) {
+    target_ping_check_down($gho{$g});
+}
ts-guest-start +25 -10
index 057afe6..410f5fd 100755
@@ -22,20 +22,35 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,%gho);
 
-sub start () {
-    guest_umount_lv($ho, $gho);
+sub start ($) {
+    my ($gn) = @_;
+    guest_umount_lv($ho, $gho{$gn});
     my $cmd= toolstack()->{Command}." create ".
-        $r{ $gho->{Guest}.'_'. toolstack()->{CfgPathVar} };
+        $r{ $gho{$gn}->{Guest}.'_'. toolstack()->{CfgPathVar} };
     target_cmd_root($ho, $cmd, 30);
 }
 
-sub checkstart () {
-    guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+sub checkstart ($) {
+    my ($gn) = @_;
+    guest_checkrunning($ho, $gho{$gn}) or die "$gho{$gn}->{Name} not running";
 }
 
-start();
-checkstart();
-guest_await($gho, target_var($gho,'boot_timeout'));
-guest_check_up($gho);
+# Let's first ask all domains to start and then check they're all actually up
+foreach my $g (@guests) {
+    ($ho,$gho{$g}) = ts_get_host_guest($whhost, $g);
+    start($g);
+    checkstart($g);
+}
+foreach my $g (@guests) {
+    guest_await($gho{$g}, target_var($gho{$g},'boot_timeout'));
+    guest_check_up($gho{$g});
+}
ts-guest-stop +22 -8
index cc7db4c..a4fddf4 100755
@@ -22,17 +22,31 @@ use Osstest::TestSupport;
 
 tsreadconfig();
 
-our ($ho,$gho) = ts_get_host_guest(@ARGV);
+our $whhost = 'host';
+if (@ARGV && ($ARGV[0] =~ m/^host=.*$/ || $ARGV[0] eq 'host')) {
+    $whhost = $ARGV[0];
+    shift @ARGV;
+}
+our (@guests) = @ARGV;
+
+our ($ho,%gho);
 
-sub stop () {
-    guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+sub stop ($) {
+    my ($gn) = @_;
+    guest_checkrunning($ho, $gho{$gn}) or die "$gho{$gn}->{Name} not running";
     target_cmd_root($ho,
 		    toolstack()->{Command}
 		    ." shutdown -w "
-		    .$gho->{Name}, 200);
-    guest_checkrunning($ho, $gho) and die $gho->{Name};
+		    .$gho{$gn}->{Name}, 200);
+    guest_checkrunning($ho, $gho{$gn}) and die $gho{$gn}->{Name};
 }
 
-guest_await_dhcp_tcp($gho, 5);
-stop();
-target_ping_check_down($gho);
+# Let's first ask all domains to stop and then check they're down
+foreach my $g (@guests) {
+    ($ho,$gho{$g}) = ts_get_host_guest($whhost, $g);
+    guest_await_dhcp_tcp($gho{$g}, 5);
+    stop($g);
+}
+foreach my $g (@guests) {
+    target_ping_check_down($gho{$g});
+}
#4 lore raw
In reply to Dario Faggioli

Dario Faggioli writes ("[OSSTEST PATCH [RFC] 0/3] Series short description"):

Quoted 10 lines

OSSTest: allow for handling multiple guests at the same time

via the ts-debian-install, ts-debian-fixup and
ts-guest-{start,stop,destroy} test scripts.

The idea is to enable something like that:

 $ OSSTEST_JOB=test-amd64-amd64-xl
 $ export OSSTEST_JOB
 $ ./ts-debian-install host=tg03 debian1 debian2 debian3

I don't much like this approach, I'm afraid.

* It's more serialised that necessary
* The iteration is distributed over bunches of scripts
* You can't set up heterogeonous systems
* It's nearly-duplicating functionality from sg-run-job
* The logfiles from the different guest operations are mixed up
* The multiple guests are conflated into a single cell in the test results matrix

sg-run-job already knows how to parallelise operations on multiple hosts. It wouldn't be too hard to make it capable of doing these kind of stepwise operations on multiple guests.

Quoted 1 line

I can go ahead and do the same for other ts-* scripts but:

Thanks for asking!

Quoted 2 lines

2) given my limited perl skills, I though I better ask for some early feedback. :-)

How's your Tcl ? :-)

Ian.

#5 lore raw
Quoted 6 lines · Ian Jackson

Dario Faggioli writes ("[OSSTEST PATCH [RFC] 0/3] Series short description"):

Quoted 5 lines

OSSTest: allow for handling multiple guests at the same time

via the ts-debian-install, ts-debian-fixup and
ts-guest-{start,stop,destroy} test scripts.

Quoted 8 lines
Quoted 5 lines

The idea is to enable something like that:

 $ OSSTEST_JOB=test-amd64-amd64-xl
 $ export OSSTEST_JOB
 $ ./ts-debian-install host=tg03 debian1 debian2 debian3

I don't much like this approach, I'm afraid.

Fair enough, I'm happy I sent this out pretty early then! :-)

Just one thing.

Quoted 3 lines

* It's more serialised that necessary
* The iteration is distributed over bunches of scripts
* You can't set up heterogeonous systems

What do you mean with this?

Quoted 9 lines

* It's nearly-duplicating functionality from sg-run-job
* The logfiles from the different guest operations are mixed up
* The multiple guests are conflated into a single cell in the test results matrix

sg-run-job already knows how to parallelise operations on multiple hosts. It wouldn't be too hard to make it capable of doing these kind of stepwise operations on multiple guests.

Ok. So, given that ...

Quoted 5 lines
Quoted 2 lines

2) given my limited perl skills, I though I better ask for some early feedback. :-)

How's your Tcl ? :-)

... I knew a language called like that existed, but that is mostly it, and while I try to understand that fancy syntax, any pointer to what I should look at to understand how it "already knows how to parallelise operations on multiple hosts" and hence how I'd be able to "make it capable of doing these kind of stepwise operations on multiple guests"?

I found a 'proc per-host-ts'. Is that a good starting point?

Thanks and Regards,
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#6 lore raw

Dario Faggioli writes ("Re: [Xen-devel] [OSSTEST PATCH [RFC] 0/3] Series short description"):

Quoted 4 lines · Ian Jackson
Quoted 1 line

* You can't set up heterogeonous systems

What do you mean with this?

I mean, you can't easily set up a mixed setup with a number of different guest OS's, because each script only makes a particular kind of guest. If you wanted to do that you'd have to do it in sg-run-job - and if you were going to write that then it could take care of the whole problem.

Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

2) given my limited perl skills, I though I better ask for some early feedback. :-)

How's your Tcl ? :-)

... I knew a language called like that existed, but that is mostly it, and while I try to understand that fancy syntax, any pointer to what I should look at to understand how it "already knows how to parallelise operations on multiple hosts" and hence how I'd be able to "make it capable of doing these kind of stepwise operations on multiple guests"?

Tcl itself is just a programming language. It happens to have very good support for writing event-continuation-passing-style programs, with little code. (I also really like it.)

In sg-run-job the core of this is done with the spawn-ts and reap-ts procedures. Mostly in sg-run-job, these are combined into run-ts, which spawns and then immediately reaps.

But there is an existing proc per-host-ts which interprets its arguments as a specification of a set of hosts, and spawns the script on each host and then reaps them again, so that the same script runs in parallel on all the hosts.

This is used in the main run-job procedure for host installation, and actually takes parallel effect in the paired migration tests. But it could be refactored to be slightly more general and then we could have a similar per-guest-ts procedure.

The tcl.tk website has comprehensive language documentation of course, but I would be happy to help/review/etc. Do you like learning new languages ... ?

Ian.

#7 lore raw
Quoted 11 lines · Ian Jackson

Dario Faggioli writes ("Re: [Xen-devel] [OSSTEST PATCH [RFC] 0/3] Series short description"):

Quoted 5 lines

... I knew a language called like that existed, but that is mostly it, and while I try to understand that fancy syntax, any pointer to what I should look at to understand how it "already knows how to parallelise operations on multiple hosts" and hence how I'd be able to "make it capable of doing these kind of stepwise operations on multiple guests"?

Tcl itself is just a programming language. It happens to have very good support for writing event-continuation-passing-style programs, with little code.

Yes, that I knew. I've seen it being used for event based simulation, but I never got to that.

Quoted 2 lines

(I also really like it.)

That I suspected. :-P

Quoted 4 lines

In sg-run-job the core of this is done with the spawn-ts and reap-ts procedures. Mostly in sg-run-job, these are combined into run-ts, which spawns and then immediately reaps.

Ok.

Quoted 5 lines

But there is an existing proc per-host-ts which interprets its arguments as a specification of a set of hosts, and spawns the script on each host and then reaps them again, so that the same script runs in parallel on all the hosts.

Right, as I said, I felt like that could be the spot. I'll look into it and try to understand what goes on there (and the how to extend that as per my needs).

Quoted 4 lines

The tcl.tk website has comprehensive language documentation of course, but I would be happy to help/review/etc. Do you like learning new languages ... ?

I certainly do! :-)

Thanks for the explanation and the pointers.
Dario

Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel