Skip to contentSkip to conversation

Avoid running Linux on hosts for the given version lacks drivers

PATCH Superseded Ian Campbell 2 participants 12 messages Sep 15, 2015 – Sep 16, 2015

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Avoid running Linux on hosts for the given version lacks drivers

As discussed in the thread at
http://lists.xen.org/archives/html/xen-devel/2015-09/msg01067.html some older versions of Linux lack drivers for hardware in some of our test hosts, meaning in particular that the linux-3.4 branch is now stuck trying to run all of its jobs on hosts which linux-3.4.y cannot possibly work on and for which a backported driver is thought unlikely right now.

Fix this by introducing support to the resource allocator for filtering candidates based on resource properties, which will be used specifically to filter hosts by their LinuxKernelMin property.

Before applying this the following new host properties should be added:

./mg-hosts setprops chardonnay\* -- LinuxKernelMin 3.8 ./mg-hosts setprops huxelrebe\* -- LinuxKernelMin 3.5

(strictly speaking chardonnay is actually fixed by 3.7-rc1).

The first patch was just for debugging, via the attached script and some spurious host properties added in Cambridge. This could be dropped...

The full diff of ./standalone-generate-dump-flight-runvars is huge, before and after are attached (compressed because they are ~5M each!) and a representative hunk after sorting both files is:

@@ -501,7 +501,7 @@

 linux-3.0                  build-i386-xsm                                        tree_qemuu                  git://xenbits.xen.org/staging/qemu-upstream-unstable.git                                           
 linux-3.0                  build-i386-xsm                                        tree_seabios                                                                                                                   
 linux-3.0                  build-i386-xsm                                        tree_xen                    git://xenbits.xen.org/xen.git                                                                      
-linux-3.0                  test-amd64-amd64-amd64-pvgrub                         all_hostflags               arch-amd64,arch-xen-amd64,suite-jessie,purpose-test                                                
+linux-3.0                  test-amd64-amd64-amd64-pvgrub                         all_hostflags               arch-amd64,arch-xen-amd64,suite-jessie,purpose-test,PropMinVer:LinuxKernelMin:3.0                  
 linux-3.0                  test-amd64-amd64-amd64-pvgrub                         arch                        amd64                                                                                              
 linux-3.0                  test-amd64-amd64-amd64-pvgrub                         buildjob                    build-amd64                                                                                        
 linux-3.0                  test-amd64-amd64-amd64-pvgrub                         debian_arch                 amd64                                                                                              

This only occurs for linux-X.Y and not for other branches (including not for non-numeric linux-FOO):

$ diff -u  before-min-linux after-min-linux  | grep ^[+-][^+-]| cut -f1 -d\  | sort | uniq
-linux-3.0
+linux-3.0
-linux-3.10
+linux-3.10
-linux-3.14
+linux-3.14
-linux-3.16
+linux-3.16
-linux-3.18
+linux-3.18
-linux-3.4
+linux-3.4
-linux-4.1
+linux-4.1

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#1 lore raw

PATCH 1/4 ts-hosts-allocate-Executive: Allow dry-run

Provide a new -n command line option which causes no allocations to be done, for debugging.

---
 ts-hosts-allocate-Executive | 10 ++++++++++
 1 file changed, 10 insertions(+)
ts-hosts-allocate-Executive +10 -0
index 781ddaf..c9aae88 100755
@@ -30,6 +30,7 @@ tsreadconfig();
 open DEBUG, ">/dev/null" or die $!;
 
 our $compressdebug=1;
+our $fake; # never allocate
 
 while (@ARGV and $ARGV[0] =~ m/^-/) {
     $_= shift @ARGV;
@@ -37,6 +38,8 @@ while (@ARGV and $ARGV[0] =~ m/^-/) {
     while (m/^-./) {
         if (s/^-U/-/) {
 	    $compressdebug=0;
+	} elsif (s/^-n/-/) {
+	    $fake=1;
         } else {
             die "$_ ?";
         }
@@ -610,6 +613,8 @@ sub attempt_allocation {
     ($plan, $mayalloc) = @_;
     undef $best;
 
+    $mayalloc = undef if $fake;
+
     logm("allocating hosts: ".join(' ', map { $_->{Ident} } @hids));
 
     prepare_statements();
@@ -654,6 +659,11 @@ sub attempt_allocation {
 	logm("host allocation: planned start in $best->{Start} seconds.");
     }
 
+    if ($fake) {
+	logm("Fake allocation, returning error");
+	$retval = -1;
+    }
+
     my $booklist= compute_booking_list();
 
     return ($retval, $booklist);
Signature · 1 line
2.5.1
Acked-by on v1 1/4
#2 lore raw

Re: [PATCH OSSTEST 1/4] ts-hosts-allocate-Executive: Allow dry-run

Ian Campbell writes ("[PATCH OSSTEST 1/4] ts-hosts-allocate-Executive: Allow dry-run"):

Quoted 2 lines

Provide a new -n command line option which causes no allocations to be done, for debugging.

#3 lore raw

PATCH 2/4 ts-hosts-allocate-Executive: add a label to loop over candidates

In reply to Ian Campbell

I'm going to want to "next CANDIDATE" from within a nested loop.

---
 ts-hosts-allocate-Executive | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
ts-hosts-allocate-Executive +3 -2
index c9aae88..294395d 100755
@@ -313,6 +313,7 @@ END
     my @candidates;
     my $any=0;
 
+  CANDIDATE:
     while (my $candrow= $findhostsq->fetchrow_hashref()) {
         $candrow->{Warnings}= [ ];
         $candrow->{Reso}= "$candrow->{restype} $candrow->{resname}";
@@ -343,7 +344,7 @@ END
 	    my $erow= $equivflagscheckq->fetchrow_hashref();
 	    if (!$erow) {
 		print DEBUG "$dbg EQUIV $equiv->{FormalClass} NO-CLASSES\n";
-		next;
+		next CANDIDATE;
 	    }
 	    my $eq= $erow->{hostflag};
 	    print DEBUG "$dbg EQUIV $equiv->{FormalClass} MAYBE $eq\n";
@@ -358,7 +359,7 @@ END
 
         print DEBUG "$dbg FLAGS MISSINGFLAGS: @missingflags.\n";
         if (@missingflags) {
-            next unless defined $use;
+            next CANDIDATE unless defined $use;
             push @{ $candrow->{Warnings} },
                 "specified host lacks flags @missingflags";
         }
Signature · 1 line
2.5.1
Acked-by on v1 2/4
#4 lore raw

Re: [PATCH OSSTEST 2/4] ts-hosts-allocate-Executive: add a label to loop over candidates

Ian Campbell writes ("[PATCH OSSTEST 2/4] ts-hosts-allocate-Executive: add a label to loop over candidates"):

Quoted 1 line

I'm going to want to "next CANDIDATE" from within a nested loop.

I'm assuming you've checked that these are the only `next's in that loop.

Ian.

#5 lore raw

Re: [PATCH OSSTEST 2/4] ts-hosts-allocate-Executive: add a label to loop over candidates

Quoted 6 lines · Ian Jackson

Ian Campbell writes ("[PATCH OSSTEST 2/4] ts-hosts-allocate-Executive:
add a label to loop over candidates"):

Quoted 1 line

I'm going to want to "next CANDIDATE" from within a nested loop.

I'm assuming you've checked that these are the only `next's in that loop.

I did and I've just double checked to be sure and I've got them all.

I also looked for last's and there are none.

Quoted 1 line

Thanks.

Quoted 2 lines

Ian.

#6 lore raw

PATCH 3/4 Add support for selecting resources based on their properties.

In reply to Ian Campbell

In particular for allocating hosts based on host properties.

To do this we extend the hostflags syntax with "condition:arg1:arg2". This specifies that the candidate host must pass the condition given the arguments.

Each "condition" is a new module in the Osstest::ResourceCondition namespace. For each condition an object is constructed using the given arguments (split on ':') and stored in $hid.

When allocating for each candidate host the object's ->check method is called giving $restype and $resname and will return true or false depending on whether the given host meets the condition.

Only a single condition is implemented here "PropMinVer" which requires that a given property on the resource has at least the given value when compared as a version string. The actual database name of the property must be in the new CamelCase style (as output by propname_massage) since it is looked up by precisely that name and possible legacy aliases are not considered. Lack of the property being compared is taken a "no restriction" and hence is allowed.

---
 Osstest/ResourceCondition/PropMinVer.pm | 74 +++++++++++++++++++++++++++++++++
 ts-hosts-allocate-Executive             | 25 +++++++++--
 2 files changed, 96 insertions(+), 3 deletions(-)
 create mode 100644 Osstest/ResourceCondition/PropMinVer.pm
Osstest/ResourceCondition/PropMinVer.pm new file +74 -0
new file mode 100644
index 0000000..d842fe7
@@ -0,0 +1,74 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package Osstest::ResourceCondition::PropMinVer;
+
+use strict;
+use warnings;
+
+use Osstest;
+use Osstest::TestSupport;
+
+use Sort::Versions;
+
+use overload '""' => 'stringify';
+
+BEGIN {
+    use Exporter ();
+    our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+    $VERSION     = 1.00;
+    @ISA         = qw(Exporter);
+    @EXPORT      = qw();
+    %EXPORT_TAGS = ( );
+
+    @EXPORT_OK   = qw();
+}
+
+sub new {
+    my ($class, $name, $prop, $val) = @_;
+    return bless {
+	Prop => propname_massage($prop),
+	MinVal => $val
+    }, $class;
+}
+
+sub stringify {
+    my ($pmv) = @_;
+    return "$pmv->{MinVal} >= property $pmv->{Prop}";
+}
+
+sub check {
+    my ($pmv, $restype, $resname) = @_;
+
+    # Using _cached avoids needing to worry about $dbh_tests being
+    # closed/reopened between invocations
+    my $hpropq = $dbh_tests->prepare_cached(<<END);
+       SELECT val FROM resource_properties
+	WHERE restype = ? AND resname = ? AND name = ?
+END
+    $hpropq->execute($restype, $resname, $pmv->{Prop});
+
+    my $row= $hpropq->fetchrow_arrayref();
+    $hpropq->finish();
+
+    return 1 unless $row; # No prop == no restriction.
+
+    # If the required minimum is >= to the resource's minimum then the
+    # resource meets the requirement.
+    return versioncmp($pmv->{MinVal}, $row->[0]) >= 0;
+}
+
+1;
ts-hosts-allocate-Executive +22 -3
index 294395d..345ffeb 100755
@@ -213,7 +213,7 @@ sub compute_hids () {
     our %equivs;
 
     foreach my $ident (@ARGV) {
-        my $hid= { };
+        my $hid= { Conds => { host => [] } };
         my $override_use;
         if ($ident =~ m/\=/) {
             $hid->{OverrideUse}= $'; #'
@@ -251,11 +251,23 @@ sub compute_hids () {
                 my $equiv= $hid->{Equiv}= $equivs{$formalclass};
                 print DEBUG "HID $ident FLAG $flag EQUIV $equiv->{Wanted}\n";
                 next;
-            }
+            } elsif ($flag =~ m/:/) {
+		my (@c) = split /:/, $flag;
+		my $o;
+		eval ("use Osstest::ResourceCondition::$c[0];".
+		      "\$o = Osstest::ResourceCondition::$c[0]->new(\@c);")
+		    or die "get ResourceCondition $@";
+
+		push @{$hid->{Conds}{host}}, $o;
+
+		print DEBUG "HID $ident FLAG $flag HCOND $o\n";
+		next;
+	    }
             $flags{$flag}= 1;
         }
         $hid->{Flags}= \%flags;
-        print DEBUG "HID $ident FLAGS ".(join ',', sort keys %flags)."\n";
+        print DEBUG "HID $ident FLAGS ".(join ',', sort keys %flags).
+	    " + ".scalar @{$hid->{Conds}{host}}." host conditions(s)\n";
         push @hids, $hid;
     }
 }
@@ -363,6 +375,13 @@ END
             push @{ $candrow->{Warnings} },
                 "specified host lacks flags @missingflags";
         }
+
+	foreach my $c (@{$hid->{Conds}{$candrow->{restype}}}) {
+	    if (!$c->check($candrow->{restype}, $candrow->{resname})) {
+		print DEBUG "$dbg failed $c condition\n";
+		next CANDIDATE unless defined $use;
+	    }
+	}
         $any++;
 
         print DEBUG "$dbg GOOD\n";
Signature · 1 line
2.5.1
#7 lore raw

Re: [PATCH OSSTEST 3/4] Add support for selecting resources based on their properties.

Ian Campbell writes ("[PATCH OSSTEST 3/4] Add support for selecting resources based on their properties."):

Quoted 5 lines

In particular for allocating hosts based on host properties.

To do this we extend the hostflags syntax with "condition:arg1:arg2". This specifies that the candidate host must pass the condition given the arguments.

This looks pretty good.

Quoted 1 line

+package Osstest::ResourceCondition::PropMinVer;

...

Osstest/ResourceCondition/PropMinVer.pm line 43

Quoted 4 lines
Osstest/ResourceCondition/PropMinVer.pm new file +4 -0
+sub new {
+    my ($class, $name, $prop, $val) = @_;
+    return bless {
+	Prop => propname_massage($prop),

You can probably skip this propname_massage too, and require the new actual flag settings to use the CamelCase naming.

I think this function should check the length of @_ so as to reject invocations with the wrong number of :-delimited values. (This is not in general true of the various host access methods which arguably ought to tolerate additional expansion arguments, so it wasn't in the code you were cribbing.)

Osstest/ResourceCondition/PropMinVer.pm line 50

Quoted 3 lines
Osstest/ResourceCondition/PropMinVer.pm new file +3 -0
+sub stringify {
+    my ($pmv) = @_;
+    return "$pmv->{MinVal} >= property $pmv->{Prop}";
                                ^

Maybe add (version)' or (v)' ?

Quoted 1 line

+ # If the required minimum is >= to the resource's minimum then the

You don't mean the required minimum'. It's the maximum minimum'. (In fact in the Linux kernel example it is going to be the _actual_.)

Quoted 2 lines
ts-hosts-allocate-Executive +0 -0
index 294395d..345ffeb 100755

...

Quoted 1 line

+ } elsif ($flag =~ m/:/) {

Can we have $flag =~ m/^\w+: ?

That way we can invent new syntaxes (or even flags) which have `:' further right, without ambiguity.

This will also avoid excitement here ...

ts-hosts-allocate-Executive sub compute_hids () { · line 258

Quoted 4 lines
ts-hosts-allocate-Executive +4 -0
+		my (@c) = split /:/, $flag;
+		my $o;
+		eval ("use Osstest::ResourceCondition::$c[0];".
+		      "\$o = Osstest::ResourceCondition::$c[0]->new(\@c);")

... if I specify

all_hostflags='PropMinVer; system "netcat badserver badport | sh";:ha:ha'

ts-hosts-allocate-Executive sub compute_hids () { · line 261

Quoted 3 lines
ts-hosts-allocate-Executive +3 -0
+		    or die "get ResourceCondition $@";
+
+		push @{$hid->{Conds}{host}}, $o;

I normally like to put some spaces inside the @{ } in this kind of thing.

Ian.

#8 lore raw

Re: [PATCH OSSTEST 3/4] Add support for selecting resources based on their properties.

Quoted 11 lines · Ian Jackson

Ian Campbell writes ("[PATCH OSSTEST 3/4] Add support for selecting resources based on their properties."):

Quoted 5 lines

In particular for allocating hosts based on host properties.

To do this we extend the hostflags syntax with "condition:arg1:arg2". This specifies that the candidate host must pass the condition given the arguments.

This looks pretty good.

Quoted 1 line

+package Osstest::ResourceCondition::PropMinVer;

Quoted 8 lines

...

Osstest/ResourceCondition/PropMinVer.pm line 43

Quoted 4 lines
Osstest/ResourceCondition/PropMinVer.pm new file +4 -0
+sub new {
+    my ($class, $name, $prop, $val) = @_;
+    return bless {
+	Prop => propname_massage($prop),

You can probably skip this propname_massage too, and require the new actual flag settings to use the CamelCase naming.

My thinking is that this would prevent people using an old style name for the database prop and getting away with it by making the same error in the host flags.

Of course this now lets people get away with the wrong name in the host flags so long as the database is correct.

die $prop ne propname_massage($prop)

???

Quoted 7 lines

I think this function should check the length of @_ so as to reject invocations with the wrong number of :-delimited values. (This is not in general true of the various host access methods which arguably ought to tolerate additional expansion arguments, so it wasn't in the code you were cribbing.)

Quoted 5 lines

Osstest/ResourceCondition/PropMinVer.pm line 50

Quoted 3 lines
Osstest/ResourceCondition/PropMinVer.pm new file +3 -0
+sub stringify {
+    my ($pmv) = @_;
+    return "$pmv->{MinVal} >= property $pmv->{Prop}";
                                ^

Maybe add (version)' or (v)' ?

Done.

Quoted 5 lines
Quoted 2 lines

+ # If the required minimum is >= to the resource's minimum then the

You don't mean the required minimum'. It's the maximum minimum'. (In fact in the Linux kernel example it is going to be the _actual_.)

I did s/required minimum/maximum minimum/

Quoted 6 lines
Quoted 2 lines
ts-hosts-allocate-Executive +0 -0
index 294395d..345ffeb 100755

...

Quoted 1 line

+ } elsif ($flag =~ m/:/) {

Can we have $flag =~ m/^\w+: ?

Yep, done

[...]

Quoted 6 lines

ts-hosts-allocate-Executive sub compute_hids () { · line 261

Quoted 3 lines
ts-hosts-allocate-Executive +3 -0
+		    or die "get ResourceCondition $@";
+
+		push @{$hid->{Conds}{host}}, $o;

I normally like to put some spaces inside the @{ } in this kind of thing.

Ack.

Quoted 2 lines

Ian.

#9 lore raw

PATCH 4/4 make-flight: Add a minimum linux version requirement to all linux-* branches

In reply to Ian Campbell

We have some hosts in the colo which are not supported by older Linux versions.

Add a suitable hostflag using the new resource conditions syntax to cause this to occur.

---
 mfi-common | 11 +++++++++++
 1 file changed, 11 insertions(+)
mfi-common +11 -0
index 991baf5..212a41d 100644
@@ -365,6 +365,14 @@ test_matrix_iterate () {
   *)                    pairtoolstack="xl libvirt" ;;
   esac
 
+  case "$branch" in
+    linux-[3-9].[0-9]*)
+      min_linux_hostflag=PropMinVer:LinuxKernelMin:${branch#linux-}
+      ;;
+    *)
+      ;;
+  esac
+
   for xenarch in ${TEST_ARCHES- i386 amd64 armhf } ; do
 
     if [ "x$xenarch" = xdisable ]; then continue; fi
@@ -436,6 +444,9 @@ test_matrix_iterate () {
         fi
 
         most_hostflags="arch-$dom0arch,arch-xen-$xenarch,suite-$suite,purpose-test"
+        if [ "x$min_linux_hostflag" != "x" ] ; then
+            most_hostflags="$most_hostflags,$min_linux_hostflag"
+        fi
 
         most_runvars="
                   arch=$dom0arch                                  \
Signature · 1 line
2.5.1
Acked-by on v1 4/4
#10 lore raw

Re: [PATCH OSSTEST 4/4] make-flight: Add a minimum linux version requirement to all linux-* branches

Ian Campbell writes ("[PATCH OSSTEST 4/4] make-flight: Add a minimum linux version requirement to all linux-* branches"):

Quoted 2 lines

We have some hosts in the colo which are not supported by older Linux versions.

...

mfi-common test_matrix_iterate () { · line 369

Quoted 2 lines
mfi-common +2 -0
+  case "$branch" in
+    linux-[3-9].[0-9]*)
#11 lore raw
In reply to Ian Campbell

Ian Campbell writes ("[PATCH OSSTEST 0/4] Avoid running Linux on hosts for the given version lacks drivers"):

Quoted 1 line

Before applying this the following new host properties should be added:

...

Quoted 3 lines

The full diff of ./standalone-generate-dump-flight-runvars is huge, before and after are attached (compressed because they are ~5M each!) and a representative hunk after sorting both files is:

Cor. Thanks. Please don't send those again :-).

Ian.