support for ARM32 arndale and cubietruck platforms
v3 cover support for ARM32 arndale and cubietruck platforms
I'm preparing a bunch of each of these boards to be racked for inclusion in osstest. This series adds the necessary osstest support:
* Support for platforms which require us to supply a Device Tree
* Removing various hardcoded midway assumptions
* Some new features needed to workaround h/w quirks (of arndale in particular)
* Support for the power relay switch board which these are attached to
* Some new convenience features
I have run a full (including host reinstall) build-* job on each platform.
I have successfully run test-armhf-armhf-xl on cubietruck and this time around on arndale too.
For reference the relevant bits of my ~/.xen-osstest/config are below.
Since v2 I've addressed review and figured out a working arndale kernel config.
Quite a bit of this is now acked and I think the rest is in pretty good shape. The major blocker right now is that rerunning mg-debian-installer-update pulls in a newer kernel from backports.org which doesn't boot on the existing midway platform. I'm investigating that at the moment.
Despite that in theory most of this could go in now, since the midway kernel issue is already there if we rerun mg-debian-installer-update and most of the rest is benign until osstest's db learns about the new systems. "make-flight: Run a basic test on each arm platform" could be omitted for now. Or we could just wait with the whole lot.
Ian.
#HostProp_metrocentre_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 1 HostProp_westfield_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 2 HostProp_lakeside_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 3 HostProp_bluewater_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 4
HostProp_braque_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 5 HostProp_picaso_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 6 HostProp_metzinger_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 7 HostProp_gleizes_PowerMethod eth008 arm-pdu-01.uk.xensource.com admin password 8
HostGroupProp_arndale_LinuxSerialConsole ttySAC2 HostGroupProp_arndale_Build_Make_Flags -j4 HostGroupProp_arndale_XenSerialConsole dtuart HostGroupProp_arndale_XenDTUARTPath /serial@12C20000 HostGroupProp_arndale_Interface_Force eth0 HostGroupProp_arndale_ExtraInitramfsModules clk-s2mps11 s5m8767 i2c-s3c2410 phy-exynos5250-sata HostGroupProp_arndale_Rootdelay 3HostGroupProp_arndale_UBootScriptEarlyCommands setenv xen_addr_r 0x41000000
HostGroupFlags_arndale suite-wheezy,equiv-arndale,need-kernel-deb-armmp,no-di-kernel,force-mac-address,need-uboot-bootscr
HostGroup_metrocentre arndale HostProp_metrocentre_Fqdn metrocentre.uk.xensource.com HostProp_metrocentre_Ether 9e:04:00:59:64:5a
HostGroup_westfield arndale HostProp_westfield_Fqdn westfield.uk.xensource.com HostProp_westfield_Ether a6:46:13:77:e8:2f
HostGroup_lakeside arndaleHostProp_lakeside_Fqdn lakeside.uk.xensource.comHostProp_lakeside_Ether e2:e7:75:3f:df:4c
HostGroup_bluewater arndale HostProp_bluewater_Fqdn bluewater.uk.xensource.com HostProp_bluewater_Ether f2:dc:22:d7:e9:e9
HostGroupProp_cubietruck_LinuxSerialConsole ttyS0HostGroupProp_cubietruck_Build_Make_Flags -j4HostGroupProp_cubietruck_XenSerialConsole dtuartHostGroupProp_cubietruck_XenDTUARTPath /soc@01c00000/serial@01c28000 HostGroupProp_cubietruck_UBootScriptEarlyCommands setenv xen_addr_r 0x41000000 HostGroupFlags_cubietruck suite-wheezy,equiv-cubietruck,need-kernel-deb-armmp,no-di-kernel,need-uboot-bootscr
HostGroup_braque cubietruckHostProp_braque_Fqdn braque.uk.xensource.com
HostGroup_picaso cubietruckHostProp_picaso_Fqdn picaso.uk.xensource.com
HostGroup_metzinger cubietruckHostProp_metzinger metzinger.uk.xensource.com
HostGroup_gleizes cubietruckHostProp_gleizes_Fqdn gleizes.uk.xensource.com
PATCH v3 1/15 standalone: Introduce "HostGroups" for use in OSSTEST_CONFIG
This saves repeating identical HostProp and HostFlags for sets of identical machines. e.g.
HostGroupProp_cubietruck_LinuxSerialConsole ttyS0
HostGroupProp_cubietruck_Build_Make_Flags -j12
HostGroupProp_cubietruck_XenSerialConsole dtuart
HostGroupProp_cubietruck_XenDTUARTPath /soc@01c00000/serial@01c28000
HostGroupFlags_cubietruck suite-wheezy,equiv-cubietruck,need-kernel-deb-armmp,no-di-kernel,need-uboot-bootscr
HostGroup_braque cubietruck
HostProp_braque_Fqdn braque.uk.xensource.com
HostGroup_picaso cubietruck
HostProp_picaso_Fqdn picaso.uk.xensource.com
HostGroup_metzinger cubietruck
HostProp_metzinger metzinger.uk.xensource.com
HostGroup_gleizes cubietruck
HostProp_gleizes_Fqdn gleizes.uk.xensource.com---
v3: Simplify lookup of HostGroup_FOO
v2: Set HostGroup props after DB
Clarify docs
--- Osstest/HostDB/Static.pm | 2 ++ Osstest/TestSupport.pm | 17 ++++++++++++++++- README | 22 ++++++++++++++++++++++ 3 files changed, 40 insertions(+), 1 deletion(-)
Osstest/HostDB/Static.pm +2 -0
@@ -58,6 +58,8 @@ sub get_flags ($$) { #method }; $process->('HostFlags'); + $process->("HostGroupFlags_$ho->{Properties}{HostGroup}") + if $ho->{Properties}{HostGroup}; $process->("HostFlags_$ho->{Name}"); return $flags;
Osstest/TestSupport.pm +16 -1
@@ -767,7 +767,13 @@ sub selecthost ($) { $ho->{Properties}{$pn} = $val; }; - # First, we use the config file's general properites as defaults + # First, set the prop group if any. + if ( $c{"HostGroup_${name}"} ) { + $setprop->("HostGroup", $c{"HostGroup_${name}"}); + logm("Host $name is in HostGroup $ho->{Properties}{HostGroup}"); + } + + # Next, we use the config file's general properites as defaults foreach my $k (keys %c) { next unless $k =~ m/^HostProp_([A-Z].*)$/; $setprop->($1, $c{$k});
@@ -776,6 +782,15 @@ sub selecthost ($) { # Then we read in the HostDB's properties $mhostdb->get_properties($name, $ho->{Properties}); + # Next, we set any HostGroup based properties + if ( $ho->{Properties}{HostGroup} ) { + foreach my $k (keys %c) { + next unless $k =~ m/^HostGroupProp_([-a-z0-9]+)_(.*)$/; + next unless $1 eq $ho->{Properties}{HostGroup}; + $setprop->($2, $c{$k}); + } + } + # Finally, we override any host-specific properties from the config foreach my $k (keys %c) { next unless $k =~ m/^HostProp_([-a-z0-9]+)_(.*)$/;
README +22 -0
@@ -333,6 +333,28 @@ HostProp_<testbox>_TftpScope Defines the Tftp scope (i.e. subnet) where this host resides. See "TftpFoo_<scope> and TftpFoo" below. +HostFlags_<testbox> + Defines a set of flags for the host. Flags is a list separated by + whitespace, comma or semi-colon. A flag can be unset by prepending + a !. Only used in standalone mode. + +HostGroup_<testbox> <group> + Defines a group of similar hosts of which <testbox> is a + member. This can then be used with HostGroupProp and HostGroupFlags + +HostGroupProps_<group>_<prop> + Equivalent to writing HostProp_<testbox>_<prop> for every testbox + which declares HostGroup_<testbox>_<group>. Allows setting a set of + common properties for a group of similar machines. These settings + take precedence over the database provided settings, but are + themselves overridden by host-specific properties. + +HostGroupFlags_<group> + Equivalent to writing HostFlags_<testbox> for every testbox which + declares HostGroup_<testbox>_<group>. Allows setting a set of + common flags for a group of similar machines. These flags are + merged with the host specific flags. Only used in standalone mode. + DebianPreseed Text to add to the debian-installer preseed file. Optional but you will need to set some NTP servers here if your firewall
Signature · 1 line
2.1.1
Re: [PATCH OSSTEST v3 01/15] standalone: Introduce "HostGroups" for use in OSSTEST_CONFIG
Ian Campbell writes ("[PATCH OSSTEST v3 01/15] standalone: Introduce "HostGroups" for use in OSSTEST_CONFIG"):
Quoted 2 lines
This saves repeating identical HostProp and HostFlags for sets of identical machines. e.g.
PATCH v3 2/15 ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}
In reply to Ian CampbellThis makes the kernel's .config available in /proc/config.gz and embeds a copy which can be extracted with linux/scripts/extract-ikconfig (which I've not tried, but have no reason to doubt).
Having this around can be handy with an older osstest installed test box and to confirm you've booted the kernel you think you have when you are messing with .config options.
--- ts-kernel-build | 3 +++ 1 file changed, 3 insertions(+)
ts-kernel-build +3 -0
@@ -436,6 +436,9 @@ setopt CONFIG_MIGRATION n setopt CONFIG_ACPI_HOTPLUG_MEMORY n +setopt CONFIG_IKCONFIG y +setopt CONFIG_IKCONFIG_PROC y + # Should all be set one way or another in defconfig but aren't setopt CONFIG_NUMA n setopt CONFIG_X86_VSMP n
Signature · 1 line
2.1.1
Re: [PATCH OSSTEST v3 02/15] ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}
Ian Campbell writes ("[PATCH OSSTEST v3 02/15] ts-kernel-build: enable CONFIG_IKCONFIG{_PROC}"):
Quoted 1 line
This makes the kernel's .config available in
/proc/config.gzand embeds a co\
py
Quoted 2 lines
which can be extracted with linux/scripts/extract-ikconfig (which I've not tried, but have no reason to doubt).
(But, can you make your editor produce commit messages which are less than 70-75 columns wide, so they don't get wrap damage when I reply?)
Thanks,
Ian.
PATCH v3 3/15 Add simple helper to update DI for all architectures.
In reply to Ian CampbellUses DebianNonfreeFirmware, even (especially) for production, so move the README stanza out of standalone only section. The current default matches what is in the current production versions of DI.
--- README | 16 ++++++++-------- mg-debian-installer-update-all | 31 +++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 8 deletions(-) create mode 100755 mg-debian-installer-update-all
README +8 -8
@@ -360,14 +360,6 @@ DebianPreseed but you will need to set some NTP servers here if your firewall doesn't permit NTP to Debian's pool.ntp.org servers. -DebianNonfreeFirmware - List of debs of non-free firmware to include in the massaged - debian-installer. You will need this if you want to use network - card which requires non-free firmware. The default is just - "firmware-bnx2'. If your host operating system doesn't have - grep-dctrl (for example because it's not Debian) then you must set - this to the empty string, by writing DebianNonfreeFirmware='' - ======================================== Config settings relevant only to standalone mode
@@ -420,6 +412,14 @@ GuestDebianSuite defaults to DebianSuite DebianPreseed added to existing preseed file +DebianNonfreeFirmware + List of debs of non-free firmware to include in the massaged + debian-installer. You will need this if you want to use network + card which requires non-free firmware. The default is just + "firmware-bnx2'. If your host operating system doesn't have + grep-dctrl (for example because it's not Debian) then you must set + this to the empty string, by writing DebianNonfreeFirmware='' + TftpFoo_<scope> and TftpFoo Describes various properties relating to Tftp in a given <scope>,
mg-debian-installer-update-all new file +31 -0
@@ -0,0 +1,31 @@ +#!/bin/bash +# usage +# ./mg-debian-installer-update-all + +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2015 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +set -e + +. cri-getconfig + +suite=`getconfig DebianSuite` +fws=`getconfig DebianNonfreeFirmware` +arches="armhf amd64 i386" + +for arch in $arches ; do + ./mg-debian-installer-update $suite $arch $fws +done
Signature · 1 line
2.1.1
PATCH v3 4/15 make-flight: Run a basic test on each arm platform
In reply to Ian CampbellUnlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate platform-$platform host flag.
The existing test-ARCH-ARCH-xl test is retained as a floating test, while a new variant is added for each distinct platform present in the hostdb which is tied to that platform type. The intention is that only arm platforms will have platforms at first, although perhaps platform-intel and platform-amd could be added in the future too.
There are currently no platform-* flags in the hostdb, so tested with s/platform-/equiv-/ and:
( set -ex ;
source ./cri-getplatforms ;
blessing=real ;
export OSSTEST_CONFIG=production-config ;
for p in '' `getplatforms "armhf"` ; do
set +x ;
echo PLATFORM: $p ;
done )
which prints:
PLATFORM:
PLATFORM: marilith
and with s/armhf/amd64/:
PLATFORM:
PLATFORM: rackservers-s40670
PLATFORM: r310-moth
PLATFORM: rackservers-s40680
PLATFORM: dell-r310
PLATFORM: rackservers-s40679
PLATFORM: rackservers-s40663
PLATFORM: rackservers-q21011
Also tested in standalone mode with a ~/.xen-osstest/config containing:
PlatformsArmhf midway cubietruck arndale
v3:
Split /\s+/
---
v2:
Use platform-* prop not equiv-*
Select platforms from host db
--- Osstest/HostDB/Executive.pm | 28 ++++++++++++++++++++++++++++ Osstest/HostDB/Static.pm | 9 +++++++++ README | 3 +++ cri-getplatforms | 25 +++++++++++++++++++++++++ make-flight | 9 +++++++-- 5 files changed, 72 insertions(+), 2 deletions(-) create mode 100755 cri-getplatforms
Osstest/HostDB/Executive.pm +28 -0
@@ -67,6 +67,34 @@ END return $flags; } +sub get_arch_platforms ($$) { + my ($hd, $blessing, $arch) = @_; + + my @plats = ( ); + my $platsq = $dbh_tests->prepare(<<END); +SELECT DISTINCT hostflag + FROM hostflags h0 + WHERE EXISTS ( + SELECT * + FROM hostflags h1, hostflags h2 + WHERE h0.hostname = h1.hostname AND h1.hostname = h2.hostname + AND h1.hostflag = ? + AND h2.hostflag = ? + ) + AND hostflag like 'platform-%'; +END + + $platsq->execute("blessed-$blessing", "arch-$arch"); + + while (my ($plat) = $platsq->fetchrow_array()) { + $plat =~ s/^platform-//g or die; + push @plats, $plat; + } + + $platsq->finish(); + return @plats; +} + sub default_methods ($$) { my ($hd, $ho) = @_;
Osstest/HostDB/Static.pm +9 -0
@@ -65,6 +65,15 @@ sub get_flags ($$) { #method return $flags; } +sub get_arch_platforms ($$) { + my ($hd, $blessing, $arch) = @_; + + my $prop = "Platforms".ucfirst($arch); + + return split /\s+/, $c{$prop} if $c{$prop}; + return () unless $c{$prop}; +} + sub default_methods ($$) { #method my ($hd, $ho) = @_;
README +3 -0
@@ -395,6 +395,9 @@ The keys in ~/.ssh/id_{rsa,dsa}.pub and ~/.ssh/authorized_keys TestHostKeypairPath +Platforms<Arch> + List of platforms (i.e. distinct host types) to run a basic test on. + HostProp_GenEtherPrefixBase 5e:36:0e:f5 # :00:01 guest number in job appended # in standalone jobdb, ^^^^^ xor'd with low 16 bits of your uid
cri-getplatforms new file +25 -0
@@ -0,0 +1,25 @@ +# -*- bash -*- + +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2014 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +getplatforms () { + perl -e ' + use Osstest; + csreadconfig(); + print join " ", $mhostdb->get_arch_platforms("'$blessing'", "'$1'") or die $!; + ' +}
make-flight +7 -2
@@ -27,6 +27,7 @@ buildflight=$4 flight=`./cs-flight-create $blessing $branch` . cri-common +. cri-getplatforms . ap-common . mfi-common
@@ -284,10 +285,14 @@ do_passthrough_tests () { test_matrix_do_one () { # Basic PV Linux test with xl + for platform in '' `getplatforms $xenarch` ; do + suffix=${platform:+-$platform} + hostflags=${most_hostflags}${platform:+,platform-$platform} - job_create_test test-$xenarch$kern-$dom0arch-xl test-debian xl \ + job_create_test test-$xenarch$kern-$dom0arch-xl$suffix test-debian xl \ $xenarch $dom0arch \ - $debian_runvars all_hostflags=$most_hostflags + $debian_runvars all_hostflags=$hostflags + done job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \ $xenarch $dom0arch \
Signature · 1 line
2.1.1
Re: [PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform
Ian Campbell writes ("[PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform"):
Quoted 4 lines
Unlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate platform-$platform host flag.
Commit message rewrapped so I can quote it.
I have just noticed, looking at some of your actual patches in osstest.git, that `git log' has wrap damage in an 80-column xterm.
Would it be too much to ask you to go through this series and rewrap the commit messages ?
Aside from that,
Ian.
Re: [PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform
Quoted 6 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST v3 04/15] make-flight: Run a basic test on each arm platform"):
Quoted 4 lines
Unlike x86 there is enough variation in the ARM platforms that it is worth having a basic test on each as part of a standard run. This relies on each host having an appropriate platform-$platform host flag.
Quoted 7 lines
Commit message rewrapped so I can quote it.
I have just noticed, looking at some of your actual patches in osstest.git, that `git log' has wrap damage in an 80-column xterm.
Would it be too much to ask you to go through this series and rewrap the commit messages ?
I'd previously experimented with ":set wm=10" in my .vimrc but I disabled it because it caused all sorts of oddities. Googling it again it seems like I may actually have wanted ":set tw=70" so I've added that, lets see how I get on with it.
I've reflowed the commit messages for the patches in my branch with that ready for v4.
Quoted 4 lines
Aside from that,
Acked-by Ian Jackson <[email protected]>
Thanks.
Ian
PATCH v3 5/15 Osstest/PDU: Add eth008.pm method to control the ARM rack PDU
In reply to Ian CampbellThis controls the eth008 relay board: http://www.robot-electronics.co.uk/htm/eth008tech.htm
Due to the use of the CGI interface this requires firmware version 4+.
---
v3: Use LWP::UserAgent
v2: Pass username and password via a netrc file.
--- Osstest/PDU/eth008.pm | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 Osstest/PDU/eth008.pm
Osstest/PDU/eth008.pm new file +65 -0
@@ -0,0 +1,65 @@ +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2014 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +package Osstest::PDU::eth008; + +use strict; +use warnings; + +use Osstest; +use Osstest::TestSupport; +use LWP::UserAgent; + +BEGIN { + use Exporter (); + our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS); + $VERSION = 1.00; + @ISA = qw(Exporter); + @EXPORT = qw(); + %EXPORT_TAGS = ( ); + + @EXPORT_OK = qw(); +} + +sub new { + my ($class, $ho, $methname, $pdu, $user, $pass, $port, @opts) = @_; + return bless { Host => $ho, + PDU => $pdu, + User => $user, + Pass => $pass, + Port => $port, + Opts => \@opts }, $class; +} + +sub pdu_power_state { + my ($mo, $on) = @_; + my $op= $on ? "DOA" : "DOI"; # Digital Output (In)Active + + # Use the CGI interface since it is less prone to being firewalled + # off, unlike the standard interface on port 17494. This is only + # available from firmware v4 onwards. + + my $ua = LWP::UserAgent->new; + + $ua->credentials("$mo->{PDU}:80", "Protected", $mo->{User}, $mo->{Pass}); + + my $resp = $ua->get("http://$mo->{PDU}/io.cgi?$op$mo->{Port}=0"); + + die "failed" unless $resp->is_success; + logm($resp->decoded_content); +} + +1;
Signature · 1 line
2.1.1
PATCH v3 6/15 Osstest/Debian: Refactor code to set bootargs in u-boot script
In reply to Ian Campbell--- Osstest/Debian.pm | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
Osstest/Debian.pm +9 -3
@@ -660,9 +660,15 @@ END } if ( $ho->{Flags}{'need-uboot-bootscr'} ) { + my @bootargs; + my $root=target_guest_lv_name($ho,"root"); - my $console = get_host_native_linux_console($ho); - my $consolecmd = "console=$console" unless $console eq "NONE"; + my $console=get_host_native_linux_console($ho); + + push @bootargs, "root=$root"; + push @bootargs, "console=$console" unless $console eq "NONE"; + + my $bootargs = join ' ', @bootargs; preseed_hook_command($ho, 'late_command', $sfx, <<END); #!/bin/sh
@@ -674,7 +680,7 @@ kernel=`readlink \$r/vmlinuz | sed -e 's|boot/||'` initrd=`readlink \$r/initrd.img | sed -e 's|boot/||'` cat >\$r/boot/boot <<EOF -setenv bootargs $consolecmd root=$root +setenv bootargs $bootargs mw.l 800000 0 10000 scsi scan ext2load scsi 0 \\\${kernel_addr_r} \$kernel
Signature · 1 line
2.1.1
PATCH v3 7/15 Osstest/Debian: Install dtbs into target filesystem in /boot/dtbs
In reply to Ian CampbellThis is done whenever dtbs.tar.gz exists. mg-debian-installer-update produces the necessary inputs on the relevant platform (armhf).
---
v3: More careful check/stat for existence of dtbs.tar.gz
v2: Install dtbs iff dtbs.tar.gz exists.
--- Osstest/Debian.pm | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-)
Osstest/Debian.pm +25 -3
@@ -20,6 +20,8 @@ package Osstest::Debian; use strict; use warnings; +use POSIX; + use IO::File; use File::Copy;
@@ -514,6 +516,9 @@ sub preseed_create ($$;@) { my $disk= $xopts{DiskDevice} || '/dev/sda'; my $suite= $xopts{Suite} || $c{DebianSuite}; + my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'. + $c{TftpDiVersion}.'-'.$ho->{Suite}; + my $hostsq= $dbh_tests->prepare(<<END); SELECT val FROM runvars WHERE flight=? AND name LIKE '%host'
@@ -627,12 +632,29 @@ $overlays echo latecmd done. END + my $dtbs = "$d_i/dtbs.tar.gz"; + if (!stat $dtbs) { + $!==&ENOENT or die "dtbs $!"; + } elsif (-e _) { + my $durl = create_webfile($ho, "dtbs", sub { + copy("$d_i/dtbs.tar.gz", $_[0]) + or die "Copy dtbs failed: $!"; + }); + preseed_hook_command($ho, 'late_command', $sfx, <<END); +#!/bin/sh +set -ex + +r=/target + +wget -O \$r/tmp/dtbs.tar.gz $durl + +in-target tar -C /boot -xaf /tmp/dtbs.tar.gz +END + } + foreach my $kp (keys %{ $ho->{Flags} }) { $kp =~ s/need-kernel-deb-// or next; - my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'. - $c{TftpDiVersion}.'-'.$ho->{Suite}; - my $kurl = create_webfile($ho, "kernel", sub { copy("$d_i/$kp.deb", $_[0]) or die "Copy kernel failed: $!";
Signature · 1 line
2.1.1
PATCH v3 8/15 Osstest/Debian: Support for loading an FDT from u-boot script
In reply to Ian CampbellThe currently supported platform provides an FDT preloaded at 0x1000. Replace this with ${fdt_addr} (which the current platform exposes) and for platforms which do not provide an fdt arrange to load the relevant file as named in the ${fdtfile} (which is conventionally provided by u-boot for platforms which need this).
Drop some random memory clearing rune, I've no idea what the intended purpose was, 0x800000 doesn't correspond to any $foo_addr_r on the midway systems for example.
Also get rid of the scsi scan which must necessarily have already happened (since the boot.scr itselfs lives on a scsi drive).
Lastly, add some echos to show progress through the script, as an aid to debugging.
---
v2: Refactor uboot dtb loading scriptlet, which previously appeared twice
--- Osstest/Debian.pm | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-)
Osstest/Debian.pm +19 -6
@@ -114,6 +114,15 @@ sub bl_getmenu_open ($$$) { return $f; } +sub uboot_scr_load_dtb () { + return <<'END'; +if test -z "\${fdt_addr}" && test -n "\${fdtfile}" ; then + echo Loading dtbs/\${fdtfile} + ext2load scsi 0 \${fdt_addr_r} dtbs/\${fdtfile} + setenv fdt_addr \${fdt_addr_r} +fi +END +} sub setupboot_uboot ($$$) { my ($ho,$want_kernver,$xenhopt,$xenkopt) = @_; my $bl= { };
@@ -132,6 +141,8 @@ sub setupboot_uboot ($$$) { my $early_commands = get_host_property($ho, 'UBootScriptEarlyCommands', ''); + my $load_dtb = uboot_scr_load_dtb(); + target_cmd_root($ho, <<END); if test ! -f /boot/$kern ; then exit 1
@@ -143,9 +154,7 @@ cp -n /boot/boot.scr /boot/boot.scr.bak xen=`readlink /boot/$xen` cat >/boot/boot <<EOF - -mw.l 800000 0 10000 -scsi scan +${load_dtb} fdt addr \\\${fdt_addr} fdt resize
@@ -692,6 +701,8 @@ END my $bootargs = join ' ', @bootargs; + my $load_dtb = uboot_scr_load_dtb(); + preseed_hook_command($ho, 'late_command', $sfx, <<END); #!/bin/sh set -ex
@@ -703,11 +714,13 @@ initrd=`readlink \$r/initrd.img | sed -e 's|boot/||'` cat >\$r/boot/boot <<EOF setenv bootargs $bootargs -mw.l 800000 0 10000 -scsi scan +${load_dtb} +echo Loading \$kernel ext2load scsi 0 \\\${kernel_addr_r} \$kernel +echo Loading \$initrd ext2load scsi 0 \\\${ramdisk_addr_r} \$initrd -bootz \\\${kernel_addr_r} \\\${ramdisk_addr_r}:\\\${filesize} 0x1000 +echo Booting +bootz \\\${kernel_addr_r} \\\${ramdisk_addr_r}:\\\${filesize} \\\${fdt_addr} EOF in-target mkimage -A arm -T script -d /boot/boot /boot/boot.scr
Signature · 1 line
2.1.1
PATCH v3 9/15 Osstest/Debian: Add support for "ExtraInitramfsModules" host property
In reply to Ian CampbellThe arndale platform needs a bunch of clk, phy and regulator stuff in order to access its root filesystem. However mkinitramfs is not (currently?) able to figure this out and therefore doesn't include them in the initrd. See
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762042
Add a new host prop which can list these required additional module and arranges for a suitable /etc/initramfs-tools/modules to be created on install.
Using the new HostGroupProp syntax the required modules are:
HostGroupProp_arndale_ExtraInitramfsModules clk-s2mps11 s5m8767 i2c-s3c2410 phy-exynos5250-sata
---
v2: References to bugs.
--- Osstest/Debian.pm | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
Osstest/Debian.pm +17 -0
@@ -727,6 +727,23 @@ in-target mkimage -A arm -T script -d /boot/boot /boot/boot.scr END } + my $modules = get_host_property($ho, "ExtraInitramfsModules", "NONE"); + if ( $modules ne "NONE" ) + { + # This is currently the best available way to add modules to + # the installed initramfs. See + # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764805 + preseed_hook_command($ho, 'late_command', $sfx, <<END); +#!/bin/sh +set -ex + +for i in $modules ; do + echo \$i >> /target/etc/initramfs-tools/modules +done +in-target update-initramfs -u -k all +END + } + my @extra_packages = (); push(@extra_packages, "u-boot-tools") if $ho->{Flags}{'need-uboot-bootscr'};
Signature · 1 line
2.1.1
PATCH v3 10/15 Osstest/Debian: support adding a rootdelay property to bootargs
In reply to Ian Campbellarndale appears to be quite slow (and asynchronous) at finding it's scsi controller. rootdelay=3 seems to do the trick.
--- Osstest/Debian.pm | 2 ++ 1 file changed, 2 insertions(+)
Osstest/Debian.pm +2 -0
@@ -694,9 +694,11 @@ END my @bootargs; my $root=target_guest_lv_name($ho,"root"); + my $rootdelay=get_host_property($ho, "rootdelay"); my $console=get_host_native_linux_console($ho); push @bootargs, "root=$root"; + push @bootargs, "rootdelay=$rootdelay" if $rootdelay; push @bootargs, "console=$console" unless $console eq "NONE"; my $bootargs = join ' ', @bootargs;
Signature · 1 line
2.1.1
PATCH v3 11/15 Osstest/Debian: Remove hardcoded midway specific addresses from boot.scr
In reply to Ian CampbellThis depends on a change to the hostdb to add "0x01000000" as the value of a new UBootSetXenAddrR property of the midway machines.
Most platforms will need something similar. For both cubietruck and arndale 0x41000000.
---
v3: Move the property to a specific one like I was supposed to last time
v2: s/go along with/depends on/
--- Osstest/Debian.pm | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
Osstest/Debian.pm +5 -4
@@ -140,9 +140,13 @@ sub setupboot_uboot ($$$) { logm("Linux options: $xenkopt"); my $early_commands = get_host_property($ho, 'UBootScriptEarlyCommands', ''); + my $xen_addr_r = get_host_property($ho, 'UBootSetXenAddrR', undef); my $load_dtb = uboot_scr_load_dtb(); + my $set_xen_addr_r = + $xen_addr_r ? "setenv xen_addr_r $xen_addr_r" : ""; + target_cmd_root($ho, <<END); if test ! -f /boot/$kern ; then exit 1
@@ -160,14 +164,11 @@ fdt addr \\\${fdt_addr} fdt resize ${early_commands} +${set_xen_addr_r} fdt set /chosen \\\#address-cells <1> fdt set /chosen \\\#size-cells <1> -setenv xen_addr_r 0x01000000 -# kernel_addr_r=0x02000000 -# ramdisk_addr_r=0x04000000 - ext2load scsi 0 \\\${xen_addr_r} \$xen setenv bootargs "$xenhopt" echo Loaded \$xen to \\\${xen_addr_r} (\\\${filesize})
Signature · 1 line
2.1.1
PATCH v3 12/15 Osstest/Debian: Add "clk_ignore_unused" to default command line
In reply to Ian Campbelldom0 is not aware that some clocks are actually in use (e.g. by the hypervisor), so this stops the kernel from messing with (specifically, disabling) those clocks. It's harmless even when not needed.
Really there ought to be some interface to communicate this from Xen to dom0, or some other mechanism to gate things. See http://bugs.xenproject.org/xen/bug/45
---
v3: More verbiage about why this is done
v2: New patch, previously incorrectly included in "Osstest/Debian: Workaround
oddities in the u-boot script parser."
--- Osstest/Debian.pm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Osstest/Debian.pm +2 -1
@@ -178,7 +178,8 @@ ext2load scsi 0 \\\${kernel_addr_r} $kern fdt mknod /chosen module\@0 fdt set /chosen/module\@0 compatible "xen,linux-zimage" "xen,multiboot-module" fdt set /chosen/module\@0 reg <\\\${kernel_addr_r} \\\${filesize}> -fdt set /chosen/module\@0 bootargs "$xenkopt ro root=$root" +# clk_ignore_unused is due to http://bugs.xenproject.org/xen/bug/45 +fdt set /chosen/module\@0 bootargs "$xenkopt ro root=$root clk_ignore_unused" echo Loaded $kern to \\\${kernel_addr_r} (\\\${filesize}) echo command line: $xenkopt ro root=$root
Signature · 1 line
2.1.1
PATCH v3 13/15 Osstest/Debian: Add 0x prefix to $filesize
In reply to Ian Campbell$filesize is an unprefixed hex number, but fdt set requires the 0x to interpret it properly. See http://lists.denx.de/pipermail/u-boot/2014-October/193622.html
---
v3: Reference ML thread.
v2: New patch, previously included in "Osstest/Debian: Workaround oddities in
the u-boot script parser.". Note that the oddities with requiring a space before the ">" is no longer reproducible with the latest u-boot.
--- Osstest/Debian.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Osstest/Debian.pm +2 -2
@@ -177,7 +177,7 @@ echo command line: \\\${bootargs} ext2load scsi 0 \\\${kernel_addr_r} $kern fdt mknod /chosen module\@0 fdt set /chosen/module\@0 compatible "xen,linux-zimage" "xen,multiboot-module" -fdt set /chosen/module\@0 reg <\\\${kernel_addr_r} \\\${filesize}> +fdt set /chosen/module\@0 reg <\\\${kernel_addr_r} 0x\\\${filesize}> # clk_ignore_unused is due to http://bugs.xenproject.org/xen/bug/45 fdt set /chosen/module\@0 bootargs "$xenkopt ro root=$root clk_ignore_unused" echo Loaded $kern to \\\${kernel_addr_r} (\\\${filesize})
@@ -186,7 +186,7 @@ echo command line: $xenkopt ro root=$root ext2load scsi 0 \\\${ramdisk_addr_r} $initrd fdt mknod /chosen module\@1 fdt set /chosen/module\@1 compatible "xen,linux-initrd" "xen,multiboot-module" -fdt set /chosen/module\@1 reg <\\\${ramdisk_addr_r} \\\${filesize}> +fdt set /chosen/module\@1 reg <\\\${ramdisk_addr_r} 0x\\\${filesize}> echo Loaded $initrd to \\\${ramdisk_addr_r} (\\\${filesize}) fdt print /chosen
Signature · 1 line
2.1.1
PATCH v3 14/15 ts-kernel-build: Adjust kernel .config to work on the arndale boards.
In reply to Ian CampbellVarious drivers are missing from multi_v7_defconfig in v3.16, also some drivers which don't play nice are enabled by default, so remove them.
---
v3: This is a more extensive version of "ts-kernel-build: Enable CONFIG_PHY_EXYNOS5250_SATA". Ian acked that but this does a lot more so I've dropped it.
--- ts-kernel-build | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
ts-kernel-build +35 -0
@@ -171,6 +171,41 @@ setopt CONFIG_BLK_DEV_NBD y # At least with Linux 3.4.77 on wheezy, the nbd module is # not loaded automatically. +# Enabling Exynos4 forces wierd CONFIG_HZ==200, and we don't +# support that platform anyway. +setopt CONFIG_ARCH_EXYNOS4 n + +# Having these on breaks USB +setopt CONFIG_SAMSUNG_USB2PHY n +setopt CONFIG_SAMSUNG_USB3PHY n + +# These cause i2c bus timeout errors on boot. +# https://groups.google.com/a/chromium.org/forum/#!topic/chromium-os-reviews/f1DW9NcSPVU? +# http://patchwork.ozlabs.org/patch/337812/ +setopt CONFIG_SENSORS_LM90 n +setopt CONFIG_ICS932S401 n + +# Enable some additional drivers for Arndale. +setopt CONFIG_PHY_EXYNOS5250_SATA m +setopt CONFIG_USB_EHCI_EXYNOS m +setopt CONFIG_USB_OHCI_EXYNOS m +setopt CONFIG_USB_HSIC_USB3503 m +setopt CONFIG_USB_DWC3 m +setopt CONFIG_USB_DWC3_HOST y +setopt CONFIG_USB_DWC3_EXYNOS m +setopt CONFIG_USB_DWC3_PCI n +setopt CONFIG_PHY_SAMSUNG_USB2 m +setopt CONFIG_PHY_EXYNOS5250_USB2 y +setopt CONFIG_PHY_EXYNOS5_USBDRD m +setopt CONFIG_RTC_DRV_S5M y +setopt CONFIG_COMMON_CLK_S2MPS11 m +setopt CONFIG_I2C_S3C2410 y +setopt CONFIG_MMC_DW m +setopt CONFIG_MMC_DW_EXYNOS m +setopt CONFIG_REGULATOR_S5M8767 m + +#### + END our $config_features= <<END;
Signature · 1 line
2.1.1
PATCH v3 15/15 Debian: Create boot.scr with a suffix and copy to boot.scr
In reply to Ian CampbellThis ensures that we always have a boot script to boot at least the native Debian kernel and Xen available, even after multiple iterations of installation, which is handy when debugging a system.
--- Osstest/Debian.pm | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
Osstest/Debian.pm +8 -6
@@ -152,12 +152,12 @@ if test ! -f /boot/$kern ; then exit 1 fi # Save a copy of the original -cp -n /boot/boot /boot/boot.bak -cp -n /boot/boot.scr /boot/boot.scr.bak +cp -n /boot/boot.xen /boot/boot.xen.bak +cp -n /boot/boot.scr.xen /boot/boot.scr.xen.bak xen=`readlink /boot/$xen` -cat >/boot/boot <<EOF +cat >/boot/boot.xen <<EOF ${load_dtb} fdt addr \\\${fdt_addr}
@@ -194,7 +194,8 @@ fdt print /chosen echo Booting \\\${xen_addr_r} - \\\${fdt_addr} bootz \\\${xen_addr_r} - \\\${fdt_addr} EOF -mkimage -A arm -T script -d /boot/boot /boot/boot.scr +mkimage -A arm -T script -d /boot/boot.xen /boot/boot.scr.xen +cp /boot/boot.scr.xen /boot/boot.scr END };
@@ -716,7 +717,7 @@ r=/target #/ kernel=`readlink \$r/vmlinuz | sed -e 's|boot/||'` initrd=`readlink \$r/initrd.img | sed -e 's|boot/||'` -cat >\$r/boot/boot <<EOF +cat >\$r/boot/boot.deb <<EOF setenv bootargs $bootargs ${load_dtb} echo Loading \$kernel
@@ -727,7 +728,8 @@ echo Booting bootz \\\${kernel_addr_r} \\\${ramdisk_addr_r}:\\\${filesize} \\\${fdt_addr} EOF -in-target mkimage -A arm -T script -d /boot/boot /boot/boot.scr +in-target mkimage -A arm -T script -d /boot/boot.deb /boot/boot.scr.deb +in-target cp /boot/boot.scr.deb /boot/boot.scr END }
Signature · 1 line
2.1.1
Quoted 1 line · Ian Campbell
This series
Which I fat fingered and sent to "xendevel" instead of xen-devel@lists. I've just resent with the right things. Sorry for the noise.
Ian.
Quoted 4 lines · Ian Campbell
The major blocker right now is that rerunning mg-debian-installer-update pulls in a newer kernel from backports.org which doesn't boot on the existing midway platform. I'm investigating that at the moment.
This investigation resulted in an upstream patch:
http://thread.gmane.org/gmane.linux.drivers.devicetree/100386
http://thread.gmane.org/gmane.linux.drivers.devicetree/100420
which I hope will go upstream shortly.
The underlying issue was a bogus size field in the header of the DT which is burnt into these systems. When booting via the u-boot command line this can be worked around with:
fdt addr $fdt_addr
fdt resize
which walks the FDT, recalculates the real size and updates the header.
Unfortunately there is no opportunity to do this when booting via PXE, as we do for host install.
Given the lack of support from the, now-defunct, manufacture of these systems I'm a bit reluctant to go poking around in the firmware to fixup the embedded FDT in case I brick it.
I think the fix will go upstream shortly, then I can add it to the Debian kernel, get it uploaded, wait for it to propagate into Jessie, get it uploaded to bpo.
In principal I could build us a custom kernel deb to use here (it's reasonably easy for me), but would you be happy with that? Perhaps mg-update-debian-installer should be modified to look in some locally constructed repo instead of at bpo? I'm wary of doing this, since it just makes it harder for someone else to replicate things, plus it just seems dodgy...
Ian.