Configure the Calxeda fabric on host boot
PATCH 1/1 Configure the Calxeda fabric on host boot
The fabric on the Calxeda midway boxes (marilith-* in osstest) does not learn mac address (at least not with the firmware we have, and with Calxeda folding this seems unlikely to get fixed). This means that guests do not get network connectivity unless their mac addresses explicitly registered with the fabric.
Registrations can be done with the bridge(8) tool from the iproute2 package which unfortunately is only present in Jessie+ and not in Wheezy. So I have done my own backport and placed it in $images/wheezy-iproute2 and arranged for it to be installed along with the transitional iproute package (from the same source) which is needed to satisfy various dependencies.
The registrations are ephemeral and need to be renewed on each reboot, so add the necessary commands to rc.local during ts-xen-install.
This required leaking a certain amount of the implementation of select_ether. Unless we want to do the bodge on every ts-guest-satart, reboot, migrate etc then this seems to be the best way.
In my testing I've set NRCXFabricMACs to 8. test-armhf-armhf-xl now gets past ts-guest-start and as far as the migration tests.
--- Osstest/CXFabric.pm | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++ Osstest/TestSupport.pm | 21 ++++++++---- ts-xen-install | 3 ++ 3 files changed, 103 insertions(+), 7 deletions(-) create mode 100644 Osstest/CXFabric.pm
Osstest/CXFabric.pm new file +86 -0
@@ -0,0 +1,86 @@ +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2014 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +package Osstest::CXFabric; + +use strict; +use warnings; + +use Osstest; +use Osstest::TestSupport; + +BEGIN { + use Exporter (); + our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); + $VERSION = 1.00; + @ISA = qw(Exporter); + @EXPORT = qw( + setup_cxfabric + ); + %EXPORT_TAGS = ( ); + + @EXPORT_OK = qw(); +} + +sub setup_cxfabric($) +{ + my ($ho) = @_; + + my $nr = get_host_property($ho, 'NRCXFabricMACs', 0); + return unless $nr; + + die unless $nr < 2**16; + + my $prefix = ether_prefix($ho); + logm("Registering $nr MAC addresses with CX fabric using prefix $prefix"); + + if ( $ho->{Suite} =~ m/wheezy/ ) + { + # iproute2 is not in Wheezy nor wheezy-backports. Use our own backport. + my $images = "$c{Images}/wheezy-iproute2"; + my $ver = '3.12.0-1~xen70+1'; + + target_putfile_root($ho, 10, "$images/iproute_${ver}_all.deb", "iproute_${ver}_all.deb"); + target_putfile_root($ho, 10, "$images/iproute2_${ver}_armhf.deb", "iproute2_${ver}_armhf.deb"); + + target_cmd_root($ho, <<END); +dpkg -i iproute_${ver}_all.deb iproute2_${ver}_armhf.deb +END + } else { + target_install_packages($ho, qw(iproute2)); + } + + my $banner = '# osstest: register potential guest MACs with CX fabric'; + my $rclocal = "$banner\n"; + # Osstest::TestSupport::select_ether allocates sequentially from $prefix:00:01 + my $i = 0; + while ( $i++ < $nr ) { + $rclocal .= sprintf("bridge fdb add $prefix:%02x:%02x dev eth0\n", + $i >> 8, $i & 0xff); + } + + target_editfile_root($ho, '/etc/rc.local', sub { + my $had_banner = 0; + while (<::EI>) { + $had_banner = 1 if m/^$banner$/; + print ::EO $rclocal if m/^exit 0$/ && !$had_banner; + print ::EO; + } + }); + +} + +1;
Osstest/TestSupport.pm +14 -7
@@ -97,6 +97,8 @@ BEGIN { await_webspace_fetch_byleaf create_webfile file_link_contents get_timeout setup_pxeboot setup_pxeboot_local host_pxefile + + ether_prefix ); %EXPORT_TAGS = ( );
@@ -1245,6 +1247,17 @@ sub target_choose_vg ($$) { return $bestvg; } +sub ether_prefix($) { + my ($ho) = @_; + my $prefix = get_host_property($ho, 'gen-ether-prefix-base'); + $prefix =~ m/^(\w+:\w+):(\w+):(\w+)$/ or die "$prefix ?"; + my $lhs = $1; + my $pv = (hex($2)<<8) | (hex($3)); + $pv ^= $mjobdb->gen_ether_offset($ho,$flight); + $prefix = sprintf "%s:%02x:%02x", $lhs, ($pv>>8)&0xff, $pv&0xff; + return $prefix; +} + sub select_ether ($$) { my ($ho,$vn) = @_; # must be run outside transaction
@@ -1252,13 +1265,7 @@ sub select_ether ($$) { return $ether if defined $ether; db_retry($flight,'running', $dbh_tests,[qw(flights)], sub { - my $prefix = get_host_property($ho, 'gen-ether-prefix-base'); - $prefix =~ m/^(\w+:\w+):(\w+):(\w+)$/ or die "$prefix ?"; - my $lhs = $1; - my $pv = (hex($2)<<8) | (hex($3)); - $pv ^= $mjobdb->gen_ether_offset($ho,$flight); - $prefix = sprintf "%s:%02x:%02x", $lhs, ($pv>>8)&0xff, $pv&0xff; - + my $prefix = ether_prefix($ho); my $glob_ether = $mjobdb->jobdb_db_glob('*_ether'); my $previous= $dbh_tests->selectrow_array(<<END, {}, $flight);
ts-xen-install +3 -0
@@ -22,6 +22,7 @@ use File::Path; use POSIX; use Osstest::Debian; use Osstest::TestSupport; +use Osstest::CXFabric; my $checkmode= 0;
@@ -122,6 +123,8 @@ sub adjustconfig () { }); target_cmd_root($ho, 'mkdir -p /var/log/xen/console'); + + setup_cxfabric($ho); } sub setupboot () {
Signature · 1 line
1.8.5.2
Ian Campbell writes ("[PATCH OSSTEST] Configure the Calxeda fabric on host boot"):
Quoted 5 lines
The fabric on the Calxeda midway boxes (marilith-* in osstest) does not learn mac address (at least not with the firmware we have, and with Calxeda folding this seems unlikely to get fixed). This means that guests do not get network connectivity unless their mac addresses explicitly registered with the fabric.
This looks mostly fine (well, FSVO "fine"!) to me, but:
Quoted 1 line
+ my $nr =
get_host_property($ho, 'NRCXFabricMACs', 0);
This shouldn't be a host property, but a global config option (or even hardcoded). There should be a host flag for the bodge, probably "need-calxeda-bridge-fixup" or something.
And you could wrap these lines, while you're at it:
Osstest/CXFabric.pm line 57
Quoted 2 lines
Osstest/CXFabric.pm new file +2
-0+ target_putfile_root($ho, 10, "$images/iproute_${ver}_all.deb", "iproute_${ver}_all.deb"); + target_putfile_root($ho, 10, "$images/iproute2_${ver}_armhf.deb", "iproute2_${ver}_armhf.deb");
Or replace this extensive code with something more condensed, e.g.:
+ my @debs = map { "iproute2_${ver}_${_}.deb" } qw(all armhf);
+ target_putfile_root($ho, 10, "$images/$_", $_) foreach @debs;
+ target_cmd_root($ho, "dpkg -i @debs");Ian.
Quoted 7 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST] Configure the Calxeda fabric on host boot"):
Quoted 5 lines
The fabric on the Calxeda midway boxes (marilith-* in osstest) does not learn mac address (at least not with the firmware we have, and with Calxeda folding this seems unlikely to get fixed). This means that guests do not get network connectivity unless their mac addresses explicitly registered with the fabric.
Quoted 7 lines
This looks mostly fine (well, FSVO "fine"!) to me, but:
Quoted 1 line
+ my $nr =
get_host_property($ho, 'NRCXFabricMACs', 0);This shouldn't be a host property, but a global config option (or even hardcoded). There should be a host flag for the bodge, probably "need-calxeda-bridge-fixup" or something.
equiv-marilith isn't suitable I take it?
Quoted 4 lines
And you could wrap these lines, while you're at it:
Quoted 2 lines
Osstest/CXFabric.pm new file +2
-0+ target_putfile_root($ho, 10, "$images/iproute_${ver}_all.deb", "iproute_${ver}_all.deb"); + target_putfile_root($ho, 10, "$images/iproute2_${ver}_armhf.deb", "iproute2_${ver}_armhf.deb");
Sure.
Quoted 3 lines
Or replace this extensive code with something more condensed, e.g.:
+ my @debs = map { "iproute2_${ver}_${_}.deb" } qw(all armhf);
This misses the iproute vs iproute2 distinction in the names, but I'll see if I can get something like this to work.
Quoted 4 lines
+
target_putfile_root($ho, 10, "$images/$_", $_)foreach @debs;
+target_cmd_root($ho, "dpkg -i @debs");Ian.
Ian Campbell writes ("Re: [PATCH OSSTEST] Configure the Calxeda fabric on host boot"):
Quoted 6 lines · Ian Jackson
Quoted 3 lines
This shouldn't be a host property, but a global config option (or even hardcoded). There should be a host flag for the bodge, probably "need-calxeda-bridge-fixup" or something.
equiv-marilith isn't suitable I take it?
Wow, that's _really_ bodgy.
Quoted 5 lines
Quoted 4 lines
And you could wrap these lines, while you're at it:
Quoted 2 lines
Osstest/CXFabric.pm new file +2
-0+ target_putfile_root($ho, 10, "$images/iproute_${ver}_all.deb", "iproute_${ver}_all.deb"); + target_putfile_root($ho, 10, "$images/iproute2_${ver}_armhf.deb", "iproute2_${ver}_armhf.deb");
Quoted 8 lines
Sure.
Quoted 3 lines
Or replace this extensive code with something more condensed, e.g.:
+ my @debs = map { "iproute2_${ver}_${_}.deb" } qw(all armhf);
This misses the iproute vs iproute2 distinction in the names, but I'll see if I can get something like this to work.
Oh so it does. Then you need to say
+ my @debs = "iproute_${ver}_all.deb", "iproute2_${ver}_armhf.deb";
but you can at least spell that out only once rather than three times as you do...
Ian.
(trying to turn everything into Common Lisp)
Quoted 9 lines · Ian Jackson
Ian Campbell writes ("Re: [PATCH OSSTEST] Configure the Calxeda fabric on host boot"):
Quoted 6 lines · Ian Jackson
Quoted 3 lines
This shouldn't be a host property, but a global config option (or even hardcoded). There should be a host flag for the bodge, probably "need-calxeda-bridge-fixup" or something.
equiv-marilith isn't suitable I take it?
Wow, that's _really_ bodgy.
I'll take that as a no ;-)
Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
And you could wrap these lines, while you're at it:
Quoted 2 lines
Osstest/CXFabric.pm new file +2
-0+ target_putfile_root($ho, 10, "$images/iproute_${ver}_all.deb", "iproute_${ver}_all.deb"); + target_putfile_root($ho, 10, "$images/iproute2_${ver}_armhf.deb", "iproute2_${ver}_armhf.deb");Sure.
Quoted 3 lines
Or replace this extensive code with something more condensed, e.g.:
+ my @debs = map { "iproute2_${ver}_${_}.deb" } qw(all armhf);
Quoted 7 lines
Quoted 2 lines
This misses the iproute vs iproute2 distinction in the names, but I'll see if I can get something like this to work.
Oh so it does. Then you need to say
+ my @debs = "iproute_${ver}_all.deb", "iproute2_${ver}_armhf.deb";
but you can at least spell that out only once rather than three times as you do...
Ack
Quoted 2 lines
Ian.
(trying to turn everything into Common Lisp)
(Ack)
Ian.