Skip to contentSkip to conversation

support for ARM32 arndale and cubietruck platforms

PATCH Superseded Ian Campbell 2 participants 22 messages Nov 20, 2014 – Nov 25, 2014

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover support for ARM32 arndale and cubietruck platforms

I'm preparing a bunch of each of these boards to be racked for inclusion in osstest. This series adds the necessary osstest support:

* Support for platforms which require us to supply a Device Tree
* Removing various hardcoded midway assumptions
* Some new features needed to workaround h/w quirks (of arndale in particular)
* Support for the power relay switch board which these are attached to
* Some new convenience features

I have run a full (including host reinstall) build-* job on each platform.

I have successfully run test-armhf-armhf-xl on cubietruck and this time around on arndale too.

For reference the relevant bits of my ~/.xen-osstest/config are below.

Since v2 I've addressed review and figured out a working arndale kernel config.

Quite a bit of this is now acked and I think the rest is in pretty good shape. The major blocker right now is that rerunning mg-debian-installer-update pulls in a newer kernel from backports.org which doesn't boot on the existing midway platform. I'm investigating that at the moment.

Despite that in theory most of this could go in now, since the midway kernel issue is already there if we rerun mg-debian-installer-update and most of the rest is benign until osstest's db learns about the new systems. "make-flight: Run a basic test on each arm platform" could be omitted for now. Or we could just wait with the whole lot.

Ian.

#HostProp_metrocentre_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 1 HostProp_westfield_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 2 HostProp_lakeside_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 3 HostProp_bluewater_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 4

HostProp_braque_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 5 HostProp_picaso_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 6 HostProp_metzinger_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 7 HostProp_gleizes_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 8

HostGroupProp_arndale_LinuxSerialConsole ttySAC2 HostGroupProp_arndale_Build_Make_Flags -j4 HostGroupProp_arndale_XenSerialConsole dtuart HostGroupProp_arndale_XenDTUARTPath /serial@12C20000 HostGroupProp_arndale_Interface_Force eth0 HostGroupProp_arndale_ExtraInitramfsModules clk-s2mps11 s5m8767 i2c-s3c2410 phy-exynos5250-sata HostGroupProp_arndale_Rootdelay 3
HostGroupProp_arndale_UBootScriptEarlyCommands setenv xen_addr_r 0x41000000

HostGroupFlags_arndale suite-wheezy,equiv-arndale,need-kernel-deb-armmp,no-di-kernel,force-mac-address,need-uboot-bootscr

HostGroup_metrocentre arndale HostProp_metrocentre_Fqdn metrocentre.uk.xensource.com HostProp_metrocentre_Ether 9e:04:00:59:64:5a

HostGroup_westfield arndale HostProp_westfield_Fqdn westfield.uk.xensource.com HostProp_westfield_Ether a6:46:13:77:e8:2f

HostGroup_lakeside arndale
HostProp_lakeside_Fqdn lakeside.uk.xensource.com
HostProp_lakeside_Ether e2:e7:75:3f:df:4c

HostGroup_bluewater arndale HostProp_bluewater_Fqdn bluewater.uk.xensource.com HostProp_bluewater_Ether f2:dc:22:d7:e9:e9

HostGroupProp_cubietruck_LinuxSerialConsole ttyS0
HostGroupProp_cubietruck_Build_Make_Flags -j4
HostGroupProp_cubietruck_XenSerialConsole dtuart
HostGroupProp_cubietruck_XenDTUARTPath /soc@01c00000/serial@01c28000 HostGroupProp_cubietruck_UBootScriptEarlyCommands setenv xen_addr_r 0x41000000 HostGroupFlags_cubietruck suite-wheezy,equiv-cubietruck,need-kernel-deb-armmp,no-di-kernel,need-uboot-bootscr

HostGroup_braque cubietruck
HostProp_braque_Fqdn braque.uk.xensource.com

HostGroup_picaso cubietruck
HostProp_picaso_Fqdn picaso.uk.xensource.com

HostGroup_metzinger cubietruck
HostProp_metzinger metzinger.uk.xensource.com

HostGroup_gleizes cubietruck
HostProp_gleizes_Fqdn gleizes.uk.xensource.com

#1 lore raw

PATCH v3 1/15 standalone: Introduce "HostGroups" for use in OSSTEST_CONFIG

This saves repeating identical HostProp and HostFlags for sets of identical machines. e.g.

    HostGroupProp_cubietruck_LinuxSerialConsole ttyS0
    HostGroupProp_cubietruck_Build_Make_Flags -j12
    HostGroupProp_cubietruck_XenSerialConsole dtuart
    HostGroupProp_cubietruck_XenDTUARTPath /soc@01c00000/serial@01c28000
    HostGroupFlags_cubietruck suite-wheezy,equiv-cubietruck,need-kernel-deb-armmp,no-di-kernel,need-uboot-bootscr

    HostGroup_braque cubietruck
    HostProp_braque_Fqdn braque.uk.xensource.com

    HostGroup_picaso cubietruck
    HostProp_picaso_Fqdn picaso.uk.xensource.com

    HostGroup_metzinger cubietruck
    HostProp_metzinger metzinger.uk.xensource.com

    HostGroup_gleizes cubietruck
    HostProp_gleizes_Fqdn gleizes.uk.xensource.com

---
v3: Simplify lookup of HostGroup_FOO
v2: Set HostGroup props after DB
Clarify docs

---
 Osstest/HostDB/Static.pm |  2 ++
 Osstest/TestSupport.pm   | 17 ++++++++++++++++-
 README                   | 22 ++++++++++++++++++++++
 3 files changed, 40 insertions(+), 1 deletion(-)
Osstest/HostDB/Static.pm +2 -0
index d0c13a1..ad18395 100644
@@ -58,6 +58,8 @@ sub get_flags ($$) { #method
     };
 
     $process->('HostFlags');
+    $process->("HostGroupFlags_$ho->{Properties}{HostGroup}")
+	if $ho->{Properties}{HostGroup};
     $process->("HostFlags_$ho->{Name}");
 
     return $flags;
Osstest/TestSupport.pm +16 -1
index 46b6720..a3b6936 100644
@@ -767,7 +767,13 @@ sub selecthost ($) {
 	$ho->{Properties}{$pn} = $val;
     };
 
-    # First, we use the config file's general properites as defaults
+    # First, set the prop group if any.
+    if ( $c{"HostGroup_${name}"} ) {
+	$setprop->("HostGroup", $c{"HostGroup_${name}"});
+	logm("Host $name is in HostGroup $ho->{Properties}{HostGroup}");
+    }
+
+    # Next, we use the config file's general properites as defaults
     foreach my $k (keys %c) {
 	next unless $k =~ m/^HostProp_([A-Z].*)$/;
 	$setprop->($1, $c{$k});
@@ -776,6 +782,15 @@ sub selecthost ($) {
     # Then we read in the HostDB's properties
     $mhostdb->get_properties($name, $ho->{Properties});
 
+    # Next, we set any HostGroup based properties
+    if ( $ho->{Properties}{HostGroup} ) {
+	foreach my $k (keys %c) {
+	    next unless $k =~ m/^HostGroupProp_([-a-z0-9]+)_(.*)$/;
+	    next unless $1 eq $ho->{Properties}{HostGroup};
+	    $setprop->($2, $c{$k});
+	}
+    }
+
     # Finally, we override any host-specific properties from the config
     foreach my $k (keys %c) {
 	next unless $k =~ m/^HostProp_([-a-z0-9]+)_(.*)$/;
README +22 -0
index 9a85549..1226369 100644
@@ -333,6 +333,28 @@ HostProp_<testbox>_TftpScope
    Defines the Tftp scope (i.e. subnet) where this host resides. See
    "TftpFoo_<scope> and TftpFoo" below.
 
+HostFlags_<testbox>
+   Defines a set of flags for the host. Flags is a list separated by
+   whitespace, comma or semi-colon. A flag can be unset by prepending
+   a !. Only used in standalone mode.
+
+HostGroup_<testbox> <group>
+   Defines a group of similar hosts of which <testbox> is a
+   member. This can then be used with HostGroupProp and HostGroupFlags
+
+HostGroupProps_<group>_<prop>
+   Equivalent to writing HostProp_<testbox>_<prop> for every testbox
+   which declares HostGroup_<testbox>_<group>. Allows setting a set of
+   common properties for a group of similar machines. These settings
+   take precedence over the database provided settings, but are
+   themselves overridden by host-specific properties.
+
+HostGroupFlags_<group>
+   Equivalent to writing HostFlags_<testbox> for every testbox which
+   declares HostGroup_<testbox>_<group>. Allows setting a set of
+   common flags for a group of similar machines. These flags are
+   merged with the host specific flags. Only used in standalone mode.
+
 DebianPreseed
    Text to add to the debian-installer preseed file.  Optional
    but you will need to set some NTP servers here if your firewall
Signature · 1 line
2.1.1
Acked-by on v3 1/15
#2 lore raw

Re: [PATCH OSSTEST v3 01/15] standalone: Introduce "HostGroups" for use in OSSTEST_CONFIG

Ian Campbell writes ("[PATCH OSSTEST v3 01/15] standalone: Introduce "HostGroups" for use in OSSTEST_CONFIG"):

Quoted 2 lines

This saves repeating identical HostProp and HostFlags for sets of identical machines. e.g.

#3 lore raw

PATCH v3 2/15 ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}

In reply to Ian Campbell

This makes the kernel's .config available in /proc/config.gz and embeds a copy which can be extracted with linux/scripts/extract-ikconfig (which I've not tried, but have no reason to doubt).

Having this around can be handy with an older osstest installed test box and to confirm you've booted the kernel you think you have when you are messing with .config options.

---
 ts-kernel-build | 3 +++
 1 file changed, 3 insertions(+)
ts-kernel-build +3 -0
index 3b48920..f02205f 100755
@@ -436,6 +436,9 @@ setopt CONFIG_MIGRATION n
 
 setopt CONFIG_ACPI_HOTPLUG_MEMORY n
 
+setopt CONFIG_IKCONFIG y
+setopt CONFIG_IKCONFIG_PROC y
+
 # Should all be set one way or another in defconfig but aren't
 setopt CONFIG_NUMA n
 setopt CONFIG_X86_VSMP n
Signature · 1 line
2.1.1
Acked-by on v3 2/15
#4 lore raw

Re: [PATCH OSSTEST v3 02/15] ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}

Ian Campbell writes ("[PATCH OSSTEST v3 02/15] ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}"):

Quoted 1 line

This makes the kernel's .config available in /proc/config.gz and embeds a co\

py

Quoted 2 lines

which can be extracted with linux/scripts/extract-ikconfig (which I've not tried, but have no reason to doubt).

(But, can you make your editor produce commit messages which are less than 70-75 columns wide, so they don't get wrap damage when I reply?)

Thanks,
Ian.

#5 lore raw

PATCH v3 3/15 Add simple helper to update DI for all architectures.

In reply to Ian Campbell

Uses DebianNonfreeFirmware, even (especially) for production, so move the README stanza out of standalone only section. The current default matches what is in the current production versions of DI.

---
 README                         | 16 ++++++++--------
 mg-debian-installer-update-all | 31 +++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+), 8 deletions(-)
 create mode 100755 mg-debian-installer-update-all
README +8 -8
index 1226369..3fe5ecc 100644
@@ -360,14 +360,6 @@ DebianPreseed
    but you will need to set some NTP servers here if your firewall
    doesn't permit NTP to Debian's pool.ntp.org servers.
 
-DebianNonfreeFirmware
-  List of debs of non-free firmware to include in the massaged
-  debian-installer.  You will need this if you want to use network
-  card which requires non-free firmware.  The default is just
-  "firmware-bnx2'.  If your host operating system doesn't have
-  grep-dctrl (for example because it's not Debian) then you must set
-  this to the empty string, by writing  DebianNonfreeFirmware=''
-
 ========================================
 
 Config settings relevant only to standalone mode
@@ -420,6 +412,14 @@ GuestDebianSuite   defaults to DebianSuite
 
 DebianPreseed      added to existing preseed file
 
+DebianNonfreeFirmware
+  List of debs of non-free firmware to include in the massaged
+  debian-installer.  You will need this if you want to use network
+  card which requires non-free firmware.  The default is just
+  "firmware-bnx2'.  If your host operating system doesn't have
+  grep-dctrl (for example because it's not Debian) then you must set
+  this to the empty string, by writing  DebianNonfreeFirmware=''
+
 TftpFoo_<scope> and TftpFoo
 
     Describes various properties relating to Tftp in a given <scope>,
mg-debian-installer-update-all new file +31 -0
new file mode 100755
index 0000000..eca4a5f
@@ -0,0 +1,31 @@
+#!/bin/bash
+# usage
+#   ./mg-debian-installer-update-all
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+set -e
+
+. cri-getconfig
+
+suite=`getconfig DebianSuite`
+fws=`getconfig DebianNonfreeFirmware`
+arches="armhf amd64 i386"
+
+for arch in $arches ; do
+    ./mg-debian-installer-update $suite $arch $fws
+done
Signature · 1 line
2.1.1
#6 lore raw

PATCH v3 4/15 make-flight: Run a basic test on each arm platform

In reply to Ian Campbell

Unlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate platform-$platform host flag.

The existing test-ARCH-ARCH-xl test is retained as a floating test, while a new variant is added for each distinct platform present in the hostdb which is tied to that platform type. The intention is that only arm platforms will have platforms at first, although perhaps platform-intel and platform-amd could be added in the future too.

There are currently no platform-* flags in the hostdb, so tested with s/platform-/equiv-/ and:

  ( set -ex ;
    source ./cri-getplatforms ;
    blessing=real ;
    export OSSTEST_CONFIG=production-config ;
    for p in '' `getplatforms "armhf"` ; do
      set +x ;
      echo PLATFORM: $p ;
    done

)
which prints:
PLATFORM:
PLATFORM: marilith
and with s/armhf/amd64/:
PLATFORM:
PLATFORM: rackservers-s40670
PLATFORM: r310-moth
PLATFORM: rackservers-s40680
PLATFORM: dell-r310
PLATFORM: rackservers-s40679
PLATFORM: rackservers-s40663
PLATFORM: rackservers-q21011

Also tested in standalone mode with a ~/.xen-osstest/config containing:

PlatformsArmhf midway cubietruck arndale

v3:

    Split /\s+/

---
v2:
Use platform-* prop not equiv-*
Select platforms from host db

---
 Osstest/HostDB/Executive.pm | 28 ++++++++++++++++++++++++++++
 Osstest/HostDB/Static.pm    |  9 +++++++++
 README                      |  3 +++
 cri-getplatforms            | 25 +++++++++++++++++++++++++
 make-flight                 |  9 +++++++--
 5 files changed, 72 insertions(+), 2 deletions(-)
 create mode 100755 cri-getplatforms
Osstest/HostDB/Executive.pm +28 -0
index b2c8dc9..6257829 100644
@@ -67,6 +67,34 @@ END
     return $flags;
 }
 
+sub get_arch_platforms ($$) {
+    my ($hd, $blessing, $arch) = @_;
+
+    my @plats = ( );
+    my $platsq = $dbh_tests->prepare(<<END);
+SELECT DISTINCT hostflag
+           FROM hostflags h0
+   WHERE EXISTS (
+       SELECT *
+         FROM hostflags h1, hostflags h2
+        WHERE h0.hostname = h1.hostname AND h1.hostname = h2.hostname
+          AND h1.hostflag = ?
+          AND h2.hostflag = ?
+   )
+   AND hostflag like 'platform-%';
+END
+
+    $platsq->execute("blessed-$blessing", "arch-$arch");
+
+    while (my ($plat) = $platsq->fetchrow_array()) {
+	$plat =~ s/^platform-//g or die;
+	push @plats, $plat;
+    }
+
+    $platsq->finish();
+    return @plats;
+}
+
 sub default_methods ($$) {
     my ($hd, $ho) = @_;
 
Osstest/HostDB/Static.pm +9 -0
index ad18395..60f5d3c 100644
@@ -65,6 +65,15 @@ sub get_flags ($$) { #method
     return $flags;
 }
 
+sub get_arch_platforms ($$) {
+    my ($hd, $blessing, $arch) = @_;
+
+    my $prop = "Platforms".ucfirst($arch);
+
+    return split /\s+/, $c{$prop} if $c{$prop};
+    return () unless $c{$prop};
+}
+
 sub default_methods ($$) { #method
     my ($hd, $ho) = @_;
 
README +3 -0
index 3fe5ecc..35532b7 100644
@@ -395,6 +395,9 @@ The keys in ~/.ssh/id_{rsa,dsa}.pub and ~/.ssh/authorized_keys
 
 TestHostKeypairPath
 
+Platforms<Arch>
+   List of platforms (i.e. distinct host types) to run a basic test on.
+
 HostProp_GenEtherPrefixBase 5e:36:0e:f5
 # 	                               :00:01 guest number in job appended
 #            in standalone jobdb, ^^^^^ xor'd with low 16 bits of your uid
cri-getplatforms new file +25 -0
new file mode 100755
index 0000000..067730c
@@ -0,0 +1,25 @@
+# -*- bash -*-
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2014 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+getplatforms () {
+        perl -e '
+                use Osstest;
+                csreadconfig();
+                print join " ", $mhostdb->get_arch_platforms("'$blessing'", "'$1'") or die $!;
+        '
+}
make-flight +7 -2
index 9963a46..0814eba 100755
@@ -27,6 +27,7 @@ buildflight=$4
 flight=`./cs-flight-create $blessing $branch`
 
 . cri-common
+. cri-getplatforms
 . ap-common
 . mfi-common
 
@@ -284,10 +285,14 @@ do_passthrough_tests () {
 test_matrix_do_one () {
 
   # Basic PV Linux test with xl
+  for platform in '' `getplatforms $xenarch` ; do
+    suffix=${platform:+-$platform}
+    hostflags=${most_hostflags}${platform:+,platform-$platform}
 
-  job_create_test test-$xenarch$kern-$dom0arch-xl test-debian xl \
+    job_create_test test-$xenarch$kern-$dom0arch-xl$suffix test-debian xl \
             $xenarch $dom0arch                                   \
-            $debian_runvars all_hostflags=$most_hostflags
+            $debian_runvars all_hostflags=$hostflags
+  done
 
   job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \
             $xenarch $dom0arch                                       \
Signature · 1 line
2.1.1
Acked-by on v3 4/15
#7 lore raw

Re: [PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform

Ian Campbell writes ("[PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform"):

Quoted 4 lines

Unlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate platform-$platform host flag.

Commit message rewrapped so I can quote it.

I have just noticed, looking at some of your actual patches in osstest.git, that `git log' has wrap damage in an 80-column xterm.

Would it be too much to ask you to go through this series and rewrap the commit messages ?

Aside from that,

Ian.

#8 lore raw

Re: [PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform

Quoted 6 lines · Ian Jackson

Ian Campbell writes ("[PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform"):

Quoted 4 lines

Unlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate platform-$platform host flag.

Quoted 7 lines

Commit message rewrapped so I can quote it.

I have just noticed, looking at some of your actual patches in osstest.git, that `git log' has wrap damage in an 80-column xterm.

Would it be too much to ask you to go through this series and rewrap the commit messages ?

I'd previously experimented with ":set wm=10" in my .vimrc but I disabled it because it caused all sorts of oddities. Googling it again it seems like I may actually have wanted ":set tw=70" so I've added that, lets see how I get on with it.

I've reflowed the commit messages for the patches in my branch with that ready for v4.

Quoted 4 lines

Aside from that,

Thanks.

Ian

#9 lore raw

PATCH v3 5/15 Osstest/PDU: Add eth008.pm method to control the ARM rack PDU

In reply to Ian Campbell

This controls the eth008 relay board: http://www.robot-electronics.co.uk/htm/eth008tech.htm

Due to the use of the CGI interface this requires firmware version 4+.

---
v3: Use LWP::UserAgent
v2: Pass username and password via a netrc file.

---
 Osstest/PDU/eth008.pm | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)
 create mode 100644 Osstest/PDU/eth008.pm
Osstest/PDU/eth008.pm new file +65 -0
new file mode 100644
index 0000000..fc5fa96
@@ -0,0 +1,65 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2014 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package Osstest::PDU::eth008;
+
+use strict;
+use warnings;
+
+use Osstest;
+use Osstest::TestSupport;
+use LWP::UserAgent;
+
+BEGIN {
+    use Exporter ();
+    our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+    $VERSION     = 1.00;
+    @ISA         = qw(Exporter);
+    @EXPORT      = qw();
+    %EXPORT_TAGS = ( );
+
+    @EXPORT_OK   = qw();
+}
+
+sub new {
+    my ($class, $ho, $methname, $pdu, $user, $pass, $port, @opts) = @_;
+    return bless { Host => $ho,
+		   PDU => $pdu,
+		   User => $user,
+		   Pass => $pass,
+		   Port => $port,
+		   Opts => \@opts }, $class;
+}
+
+sub pdu_power_state {
+    my ($mo, $on) = @_;
+    my $op= $on ? "DOA" : "DOI"; # Digital Output (In)Active
+
+    # Use the CGI interface since it is less prone to being firewalled
+    # off, unlike the standard interface on port 17494. This is only
+    # available from firmware v4 onwards.
+
+    my $ua = LWP::UserAgent->new;
+
+    $ua->credentials("$mo->{PDU}:80", "Protected", $mo->{User}, $mo->{Pass});
+
+    my $resp = $ua->get("http://$mo->{PDU}/io.cgi?$op$mo->{Port}=0");
+
+    die "failed" unless $resp->is_success;
+    logm($resp->decoded_content);
+}
+
+1;
Signature · 1 line
2.1.1
#10 lore raw

PATCH v3 6/15 Osstest/Debian: Refactor code to set bootargs in u-boot script

In reply to Ian Campbell
---
 Osstest/Debian.pm | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
Osstest/Debian.pm +9 -3
index c8db601..33a4ca4 100644
@@ -660,9 +660,15 @@ END
     }
 
     if ( $ho->{Flags}{'need-uboot-bootscr'} ) {
+	my @bootargs;
+
 	my $root=target_guest_lv_name($ho,"root");
-	my $console = get_host_native_linux_console($ho);
-	my $consolecmd = "console=$console" unless $console eq "NONE";
+	my $console=get_host_native_linux_console($ho);
+
+	push @bootargs, "root=$root";
+	push @bootargs, "console=$console" unless $console eq "NONE";
+
+	my $bootargs = join ' ', @bootargs;
 
 	preseed_hook_command($ho, 'late_command', $sfx, <<END);
 #!/bin/sh
@@ -674,7 +680,7 @@ kernel=`readlink \$r/vmlinuz | sed -e 's|boot/||'`
 initrd=`readlink \$r/initrd.img | sed -e 's|boot/||'`
 
 cat >\$r/boot/boot <<EOF
-setenv bootargs $consolecmd root=$root
+setenv bootargs $bootargs
 mw.l 800000 0 10000
 scsi scan
 ext2load scsi 0 \\\${kernel_addr_r} \$kernel
Signature · 1 line
2.1.1
#11 lore raw

PATCH v3 7/15 Osstest/Debian: Install dtbs into target filesystem in /boot/dtbs

In reply to Ian Campbell

This is done whenever dtbs.tar.gz exists. mg-debian-installer-update produces the necessary inputs on the relevant platform (armhf).

---
v3: More careful check/stat for existence of dtbs.tar.gz
v2: Install dtbs iff dtbs.tar.gz exists.

---
 Osstest/Debian.pm | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)
Osstest/Debian.pm +25 -3
index 33a4ca4..1b4d7a7 100644
@@ -20,6 +20,8 @@ package Osstest::Debian;
 use strict;
 use warnings;
 
+use POSIX;
+
 use IO::File;
 use File::Copy;
 
@@ -514,6 +516,9 @@ sub preseed_create ($$;@) {
     my $disk= $xopts{DiskDevice} || '/dev/sda';
     my $suite= $xopts{Suite} || $c{DebianSuite};
 
+    my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.
+	$c{TftpDiVersion}.'-'.$ho->{Suite};
+
     my $hostsq= $dbh_tests->prepare(<<END);
         SELECT val FROM runvars
          WHERE flight=? AND name LIKE '%host'
@@ -627,12 +632,29 @@ $overlays
 echo latecmd done.
 END
 
+    my $dtbs = "$d_i/dtbs.tar.gz";
+    if (!stat $dtbs) {
+        $!==&ENOENT or die "dtbs $!";
+    } elsif (-e _) {
+	my $durl = create_webfile($ho, "dtbs", sub {
+	    copy("$d_i/dtbs.tar.gz", $_[0])
+		or die "Copy dtbs failed: $!";
+	});
+	preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/target
+
+wget -O \$r/tmp/dtbs.tar.gz $durl
+
+in-target tar -C /boot -xaf /tmp/dtbs.tar.gz
+END
+    }
+
     foreach my $kp (keys %{ $ho->{Flags} }) {
 	$kp =~ s/need-kernel-deb-// or next;
 
-	my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.
-	    $c{TftpDiVersion}.'-'.$ho->{Suite};
-
 	my $kurl = create_webfile($ho, "kernel", sub {
 	    copy("$d_i/$kp.deb", $_[0])
 		or die "Copy kernel failed: $!";
Signature · 1 line
2.1.1
#12 lore raw

PATCH v3 8/15 Osstest/Debian: Support for loading an FDT from u-boot script

In reply to Ian Campbell

The currently supported platform provides an FDT preloaded at 0x1000. Replace this with ${fdt_addr} (which the current platform exposes) and for platforms which do not provide an fdt arrange to load the relevant file as named in the ${fdtfile} (which is conventionally provided by u-boot for platforms which need this).

Drop some random memory clearing rune, I've no idea what the intended purpose was, 0x800000 doesn't correspond to any $foo_addr_r on the midway systems for example.

Also get rid of the scsi scan which must necessarily have already happened (since the boot.scr itselfs lives on a scsi drive).

Lastly, add some echos to show progress through the script, as an aid to debugging.

---
v2: Refactor uboot dtb loading scriptlet, which previously appeared twice

---
 Osstest/Debian.pm | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)
Osstest/Debian.pm +19 -6
index 1b4d7a7..9530aa4 100644
@@ -114,6 +114,15 @@ sub bl_getmenu_open ($$$) {
     return $f;
 }
 
+sub uboot_scr_load_dtb () {
+    return <<'END';
+if test -z "\${fdt_addr}" && test -n "\${fdtfile}" ; then
+    echo Loading dtbs/\${fdtfile}
+    ext2load scsi 0 \${fdt_addr_r} dtbs/\${fdtfile}
+    setenv fdt_addr \${fdt_addr_r}
+fi
+END
+}
 sub setupboot_uboot ($$$) {
     my ($ho,$want_kernver,$xenhopt,$xenkopt) = @_;
     my $bl= { };
@@ -132,6 +141,8 @@ sub setupboot_uboot ($$$) {
 
 	my $early_commands = get_host_property($ho, 'UBootScriptEarlyCommands', '');
 
+	my $load_dtb = uboot_scr_load_dtb();
+
 	target_cmd_root($ho, <<END);
 if test ! -f /boot/$kern ; then
     exit 1
@@ -143,9 +154,7 @@ cp -n /boot/boot.scr /boot/boot.scr.bak
 xen=`readlink /boot/$xen`
 
 cat >/boot/boot <<EOF
-
-mw.l 800000 0 10000
-scsi scan
+${load_dtb}
 
 fdt addr \\\${fdt_addr}
 fdt resize
@@ -692,6 +701,8 @@ END
 
 	my $bootargs = join ' ', @bootargs;
 
+	my $load_dtb = uboot_scr_load_dtb();
+
 	preseed_hook_command($ho, 'late_command', $sfx, <<END);
 #!/bin/sh
 set -ex
@@ -703,11 +714,13 @@ initrd=`readlink \$r/initrd.img | sed -e 's|boot/||'`
 
 cat >\$r/boot/boot <<EOF
 setenv bootargs $bootargs
-mw.l 800000 0 10000
-scsi scan
+${load_dtb}
+echo Loading \$kernel
 ext2load scsi 0 \\\${kernel_addr_r} \$kernel
+echo Loading \$initrd
 ext2load scsi 0 \\\${ramdisk_addr_r} \$initrd
-bootz \\\${kernel_addr_r} \\\${ramdisk_addr_r}:\\\${filesize} 0x1000
+echo Booting
+bootz \\\${kernel_addr_r} \\\${ramdisk_addr_r}:\\\${filesize} \\\${fdt_addr}
 EOF
 
 in-target mkimage -A arm -T script -d /boot/boot /boot/boot.scr
Signature · 1 line
2.1.1
#13 lore raw

PATCH v3 9/15 Osstest/Debian: Add support for "ExtraInitramfsModules" host property

In reply to Ian Campbell

The arndale platform needs a bunch of clk, phy and regulator stuff in order to access its root filesystem. However mkinitramfs is not (currently?) able to figure this out and therefore doesn't include them in the initrd. See
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762042

Add a new host prop which can list these required additional module and arranges for a suitable /etc/initramfs-tools/modules to be created on install.

Using the new HostGroupProp syntax the required modules are:

HostGroupProp_arndale_ExtraInitramfsModules clk-s2mps11 s5m8767 i2c-s3c2410 phy-exynos5250-sata

---
v2: References to bugs.

---
 Osstest/Debian.pm | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
Osstest/Debian.pm +17 -0
index 9530aa4..8b70442 100644
@@ -727,6 +727,23 @@ in-target mkimage -A arm -T script -d /boot/boot /boot/boot.scr
 END
     }
 
+    my $modules = get_host_property($ho, "ExtraInitramfsModules", "NONE");
+    if ( $modules ne "NONE" )
+    {
+	# This is currently the best available way to add modules to
+	# the installed initramfs. See
+	# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764805
+        preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+for i in $modules ; do
+    echo \$i >> /target/etc/initramfs-tools/modules
+done
+in-target update-initramfs -u -k all
+END
+    }
+
     my @extra_packages = ();
     push(@extra_packages, "u-boot-tools") if $ho->{Flags}{'need-uboot-bootscr'};
 
Signature · 1 line
2.1.1
#14 lore raw

PATCH v3 10/15 Osstest/Debian: support adding a rootdelay property to bootargs

In reply to Ian Campbell

arndale appears to be quite slow (and asynchronous) at finding it's scsi controller. rootdelay=3 seems to do the trick.

---
 Osstest/Debian.pm | 2 ++
 1 file changed, 2 insertions(+)
Osstest/Debian.pm +2 -0
index 8b70442..bdbb55d 100644
@@ -694,9 +694,11 @@ END
 	my @bootargs;
 
 	my $root=target_guest_lv_name($ho,"root");
+	my $rootdelay=get_host_property($ho, "rootdelay");
 	my $console=get_host_native_linux_console($ho);
 
 	push @bootargs, "root=$root";
+	push @bootargs, "rootdelay=$rootdelay" if $rootdelay;
 	push @bootargs, "console=$console" unless $console eq "NONE";
 
 	my $bootargs = join ' ', @bootargs;
Signature · 1 line
2.1.1
#15 lore raw

PATCH v3 11/15 Osstest/Debian: Remove hardcoded midway specific addresses from boot.scr

In reply to Ian Campbell

This depends on a change to the hostdb to add "0x01000000" as the value of a new UBootSetXenAddrR property of the midway machines.

Most platforms will need something similar. For both cubietruck and arndale 0x41000000.

---
v3: Move the property to a specific one like I was supposed to last time
v2: s/go along with/depends on/

---
 Osstest/Debian.pm | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
Osstest/Debian.pm +5 -4
index bdbb55d..0f92661 100644
@@ -140,9 +140,13 @@ sub setupboot_uboot ($$$) {
 	logm("Linux options: $xenkopt");
 
 	my $early_commands = get_host_property($ho, 'UBootScriptEarlyCommands', '');
+	my $xen_addr_r = get_host_property($ho, 'UBootSetXenAddrR', undef);
 
 	my $load_dtb = uboot_scr_load_dtb();
 
+	my $set_xen_addr_r =
+	    $xen_addr_r ? "setenv xen_addr_r $xen_addr_r" : "";
+
 	target_cmd_root($ho, <<END);
 if test ! -f /boot/$kern ; then
     exit 1
@@ -160,14 +164,11 @@ fdt addr \\\${fdt_addr}
 fdt resize
 
 ${early_commands}
+${set_xen_addr_r}
 
 fdt set /chosen \\\#address-cells <1>
 fdt set /chosen \\\#size-cells <1>
 
-setenv xen_addr_r 0x01000000
-#   kernel_addr_r=0x02000000
-#  ramdisk_addr_r=0x04000000
-
 ext2load scsi 0 \\\${xen_addr_r} \$xen
 setenv bootargs "$xenhopt"
 echo Loaded \$xen to \\\${xen_addr_r} (\\\${filesize})
Signature · 1 line
2.1.1
#16 lore raw

PATCH v3 12/15 Osstest/Debian: Add "clk_ignore_unused" to default command line

In reply to Ian Campbell

dom0 is not aware that some clocks are actually in use (e.g. by the hypervisor), so this stops the kernel from messing with (specifically, disabling) those clocks. It's harmless even when not needed.

Really there ought to be some interface to communicate this from Xen to dom0, or some other mechanism to gate things. See http://bugs.xenproject.org/xen/bug/45

---
v3: More verbiage about why this is done
v2: New patch, previously incorrectly included in "Osstest/Debian: Workaround
oddities in the u-boot script parser."

---
 Osstest/Debian.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Osstest/Debian.pm +2 -1
index 0f92661..3e56c1f 100644
@@ -178,7 +178,8 @@ ext2load scsi 0 \\\${kernel_addr_r} $kern
 fdt mknod /chosen module\@0
 fdt set /chosen/module\@0 compatible "xen,linux-zimage" "xen,multiboot-module"
 fdt set /chosen/module\@0 reg <\\\${kernel_addr_r} \\\${filesize}>
-fdt set /chosen/module\@0 bootargs "$xenkopt ro root=$root"
+# clk_ignore_unused is due to http://bugs.xenproject.org/xen/bug/45
+fdt set /chosen/module\@0 bootargs "$xenkopt ro root=$root clk_ignore_unused"
 echo Loaded $kern to \\\${kernel_addr_r} (\\\${filesize})
 echo command line: $xenkopt ro root=$root
 
Signature · 1 line
2.1.1
#17 lore raw

PATCH v3 13/15 Osstest/Debian: Add 0x prefix to $filesize

In reply to Ian Campbell

$filesize is an unprefixed hex number, but fdt set requires the 0x to interpret it properly. See http://lists.denx.de/pipermail/u-boot/2014-October/193622.html

---
v3: Reference ML thread.
v2: New patch, previously included in "Osstest/Debian: Workaround oddities in
the u-boot script parser.". Note that the oddities with requiring a space before the ">" is no longer reproducible with the latest u-boot.

---
 Osstest/Debian.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Osstest/Debian.pm +2 -2
index 3e56c1f..82669a5 100644
@@ -177,7 +177,7 @@ echo command line: \\\${bootargs}
 ext2load scsi 0 \\\${kernel_addr_r} $kern
 fdt mknod /chosen module\@0
 fdt set /chosen/module\@0 compatible "xen,linux-zimage" "xen,multiboot-module"
-fdt set /chosen/module\@0 reg <\\\${kernel_addr_r} \\\${filesize}>
+fdt set /chosen/module\@0 reg <\\\${kernel_addr_r} 0x\\\${filesize}>
 # clk_ignore_unused is due to http://bugs.xenproject.org/xen/bug/45
 fdt set /chosen/module\@0 bootargs "$xenkopt ro root=$root clk_ignore_unused"
 echo Loaded $kern to \\\${kernel_addr_r} (\\\${filesize})
@@ -186,7 +186,7 @@ echo command line: $xenkopt ro root=$root
 ext2load scsi 0 \\\${ramdisk_addr_r} $initrd
 fdt mknod /chosen module\@1
 fdt set /chosen/module\@1 compatible "xen,linux-initrd" "xen,multiboot-module"
-fdt set /chosen/module\@1 reg <\\\${ramdisk_addr_r} \\\${filesize}>
+fdt set /chosen/module\@1 reg <\\\${ramdisk_addr_r} 0x\\\${filesize}>
 echo Loaded $initrd to \\\${ramdisk_addr_r} (\\\${filesize})
 
 fdt print /chosen
Signature · 1 line
2.1.1
#18 lore raw

PATCH v3 14/15 ts-kernel-build: Adjust kernel .config to work on the arndale boards.

In reply to Ian Campbell

Various drivers are missing from multi_v7_defconfig in v3.16, also some drivers which don't play nice are enabled by default, so remove them.

---
v3: This is a more extensive version of "ts-kernel-build: Enable CONFIG_PHY_EXYNOS5250_SATA". Ian acked that but this does a lot more so I've dropped it.

---
 ts-kernel-build | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
ts-kernel-build +35 -0
index f02205f..b2f19cf 100755
@@ -171,6 +171,41 @@ setopt CONFIG_BLK_DEV_NBD y
 # At least with Linux 3.4.77 on wheezy, the nbd module is
 # not loaded automatically.
 
+# Enabling Exynos4 forces wierd CONFIG_HZ==200, and we don't
+# support that platform anyway.
+setopt CONFIG_ARCH_EXYNOS4 n
+
+# Having these on breaks USB
+setopt CONFIG_SAMSUNG_USB2PHY n
+setopt CONFIG_SAMSUNG_USB3PHY n
+
+# These cause i2c bus timeout errors on boot.
+# https://groups.google.com/a/chromium.org/forum/#!topic/chromium-os-reviews/f1DW9NcSPVU?
+# http://patchwork.ozlabs.org/patch/337812/
+setopt CONFIG_SENSORS_LM90 n
+setopt CONFIG_ICS932S401 n
+
+# Enable some additional drivers for Arndale.
+setopt CONFIG_PHY_EXYNOS5250_SATA m
+setopt CONFIG_USB_EHCI_EXYNOS m
+setopt CONFIG_USB_OHCI_EXYNOS m
+setopt CONFIG_USB_HSIC_USB3503 m
+setopt CONFIG_USB_DWC3 m
+setopt CONFIG_USB_DWC3_HOST y
+setopt CONFIG_USB_DWC3_EXYNOS m
+setopt CONFIG_USB_DWC3_PCI n
+setopt CONFIG_PHY_SAMSUNG_USB2 m
+setopt CONFIG_PHY_EXYNOS5250_USB2 y
+setopt CONFIG_PHY_EXYNOS5_USBDRD m
+setopt CONFIG_RTC_DRV_S5M y
+setopt CONFIG_COMMON_CLK_S2MPS11 m
+setopt CONFIG_I2C_S3C2410 y
+setopt CONFIG_MMC_DW m
+setopt CONFIG_MMC_DW_EXYNOS m
+setopt CONFIG_REGULATOR_S5M8767 m
+
+####
+
 END
 
 our $config_features= <<END;
Signature · 1 line
2.1.1
#19 lore raw

PATCH v3 15/15 Debian: Create boot.scr with a suffix and copy to boot.scr

In reply to Ian Campbell

This ensures that we always have a boot script to boot at least the native Debian kernel and Xen available, even after multiple iterations of installation, which is handy when debugging a system.

---
 Osstest/Debian.pm | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
Osstest/Debian.pm +8 -6
index 82669a5..18586fc 100644
@@ -152,12 +152,12 @@ if test ! -f /boot/$kern ; then
     exit 1
 fi
 # Save a copy of the original
-cp -n /boot/boot /boot/boot.bak
-cp -n /boot/boot.scr /boot/boot.scr.bak
+cp -n /boot/boot.xen /boot/boot.xen.bak
+cp -n /boot/boot.scr.xen /boot/boot.scr.xen.bak
 
 xen=`readlink /boot/$xen`
 
-cat >/boot/boot <<EOF
+cat >/boot/boot.xen <<EOF
 ${load_dtb}
 
 fdt addr \\\${fdt_addr}
@@ -194,7 +194,8 @@ fdt print /chosen
 echo Booting \\\${xen_addr_r} - \\\${fdt_addr}
 bootz \\\${xen_addr_r} - \\\${fdt_addr}
 EOF
-mkimage -A arm -T script -d /boot/boot /boot/boot.scr
+mkimage -A arm -T script -d /boot/boot.xen /boot/boot.scr.xen
+cp /boot/boot.scr.xen /boot/boot.scr
 END
     };
 
@@ -716,7 +717,7 @@ r=/target #/
 kernel=`readlink \$r/vmlinuz | sed -e 's|boot/||'`
 initrd=`readlink \$r/initrd.img | sed -e 's|boot/||'`
 
-cat >\$r/boot/boot <<EOF
+cat >\$r/boot/boot.deb <<EOF
 setenv bootargs $bootargs
 ${load_dtb}
 echo Loading \$kernel
@@ -727,7 +728,8 @@ echo Booting
 bootz \\\${kernel_addr_r} \\\${ramdisk_addr_r}:\\\${filesize} \\\${fdt_addr}
 EOF
 
-in-target mkimage -A arm -T script -d /boot/boot /boot/boot.scr
+in-target mkimage -A arm -T script -d /boot/boot.deb /boot/boot.scr.deb
+in-target cp /boot/boot.scr.deb /boot/boot.scr
 END
     }
 
Signature · 1 line
2.1.1
#20 lore raw
In reply to Ian Campbell
Quoted 1 line · Ian Campbell

This series

Which I fat fingered and sent to "xendevel" instead of xen-devel@lists. I've just resent with the right things. Sorry for the noise.

Ian.

#21 lore raw
In reply to Ian Campbell
Quoted 4 lines · Ian Campbell

The major blocker right now is that rerunning mg-debian-installer-update pulls in a newer kernel from backports.org which doesn't boot on the existing midway platform. I'm investigating that at the moment.

This investigation resulted in an upstream patch:
http://thread.gmane.org/gmane.linux.drivers.devicetree/100386
http://thread.gmane.org/gmane.linux.drivers.devicetree/100420

which I hope will go upstream shortly.

The underlying issue was a bogus size field in the header of the DT which is burnt into these systems. When booting via the u-boot command line this can be worked around with:
fdt addr $fdt_addr
fdt resize
which walks the FDT, recalculates the real size and updates the header.

Unfortunately there is no opportunity to do this when booting via PXE, as we do for host install.

Given the lack of support from the, now-defunct, manufacture of these systems I'm a bit reluctant to go poking around in the firmware to fixup the embedded FDT in case I brick it.

I think the fix will go upstream shortly, then I can add it to the Debian kernel, get it uploaded, wait for it to propagate into Jessie, get it uploaded to bpo.

In principal I could build us a custom kernel deb to use here (it's reasonably easy for me), but would you be happy with that? Perhaps mg-update-debian-installer should be modified to look in some locally constructed repo instead of at bpo? I'm wary of doing this, since it just makes it harder for someone else to replicate things, plus it just seems dodgy...

Ian.