Master image generation and testing
cover Master image generation and testing
Hello,
This is what I currently use for doing automated deployment for a genericx86-64 image. More details on the commit messages. There is nothing particular to genericx86-64,
it should work on any EFI-enabled device with gummiboot as bootloader.
Also see YB#1596 and it's sub-bug #5614 for a bit of background. Any comments, questions, most welcome.
Cheers,
Stefan
--
Note that this is actually v3.
v1 and v2 [1] were sent on the poky mailing list, as everything was added to meta-yocto-bsp layer. Nothing much changed since v2 except moving the files around.
Changes in this version include:
- rebase on top of meta
- add to the master image only the required utilities instead of an entire packagegroup
- use TEST_POWERCONTROL_EXTRA_ARGS with TEST_POWERCONTROL_CMD
- split the second patch in the earlier series in two
[1] https://lists.yoctoproject.org/pipermail/poky/2014-March/009631.html
Please review the following changes for suitability for inclusion. If you have any objections or suggestions for improvement, please respond to the patches. If you agree with the changes, please provide your Acked-by.
The following changes since commit 49aad7da07e187f206e963001844605731b01247:
bitbake: runqueue: Remove use of waitpid on worker processes (2014-03-19 17:46:33 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib stefans/masterimage
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/masterimage
Stefan Stanacar (4):
initrdscripts: copy of the default install scripts
initrdscripts: add install scripts with a second rootfs
recipes-extended: add master image for testing purposes lib/oeqa: add a test target controller for EFI targets
meta/lib/oeqa/controllers/masterimage.py | 124 ++++++++++++ meta/lib/oeqa/runtime/ssh.py | 2 + .../initrdscripts/files/init-install-efi-testfs.sh | 199 +++++++++++++++++++ .../initrdscripts/files/init-install-testfs.sh | 211 +++++++++++++++++++++ .../initramfs-live-install-efi-testfs_1.0.bb | 20 ++ .../initramfs-live-install-testfs_1.0.bb | 20 ++ .../images/core-image-testmaster-initramfs.bb | 20 ++ .../images/core-image-testmaster.bb | 13 ++ 8 files changed, 609 insertions(+) create mode 100644 meta/lib/oeqa/controllers/masterimage.py create mode 100644 meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh create mode 100644 meta/recipes-core/initrdscripts/files/init-install-testfs.sh create mode 100644 meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb create mode 100644 meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb create mode 100644 meta/recipes-extended/images/core-image-testmaster-initramfs.bb create mode 100644 meta/recipes-extended/images/core-image-testmaster.bb
Signature · 1 line
1.8.5.3
RFC PATCH 1/4 initrdscripts: copy of the default install scripts
These are the install files for a live image, an exact copy of init-install.sh and init-install-efi.sh from core. There are renamed here with -testfs. This is just a dummy commit, just to make it easier to see the diff in the next commit.
--- .../initrdscripts/files/init-install-efi-testfs.sh | 197 +++++++++++++++++++ .../initrdscripts/files/init-install-testfs.sh | 208 +++++++++++++++++++++ 2 files changed, 405 insertions(+) create mode 100644 meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh create mode 100644 meta/recipes-core/initrdscripts/files/init-install-testfs.sh
meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh new file +197 -0
@@ -0,0 +1,197 @@ +#!/bin/sh -e +# +# Copyright (c) 2012, Intel Corporation. +# All rights reserved. +# +# install.sh [device_name] [rootfs_name] +# + +PATH=/sbin:/bin:/usr/sbin:/usr/bin + +# We need 20 Mb for the boot partition +boot_size=20 + +# 5% for swap +swap_ratio=5 + +found="no" + +echo "Searching for a hard drive..." +for device in 'hda' 'hdb' 'sda' 'sdb' 'mmcblk0' 'mmcblk1' +do + if [ -e /sys/block/${device}/removable ]; then + if [ "$(cat /sys/block/${device}/removable)" = "0" ]; then + found="yes" + + while true; do + # Try sleeping here to avoid getting kernel messages + # obscuring/confusing user + sleep 5 + echo "Found drive at /dev/${device}. Do you want to install this image there ? [y/n]" + read answer + if [ "$answer" = "y" ] ; then + break + fi + + if [ "$answer" = "n" ] ; then + found=no + break + fi + + echo "Please answer y or n" + done + fi + fi + + if [ "$found" = "yes" ]; then + break; + fi + +done + +if [ "$found" = "no" ]; then + exit 1 +fi + +echo "Installing image on /dev/${device}" + +# +# The udev automounter can cause pain here, kill it +# +rm -f /etc/udev/rules.d/automount.rules +rm -f /etc/udev/scripts/mount* + +# +# Unmount anything the automounter had mounted +# +umount /dev/${device}* 2> /dev/null || /bin/true + +mkdir -p /tmp +cat /proc/mounts > /etc/mtab + +disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d" " -f 3 | sed -e "s/MB//") + +swap_size=$((disk_size*swap_ratio/100)) +rootfs_size=$((disk_size-boot_size-swap_size)) + +rootfs_start=$((boot_size)) +rootfs_end=$((rootfs_start+rootfs_size)) +swap_start=$((rootfs_end)) + +# MMC devices are special in a couple of ways +# 1) they use a partition prefix character 'p' +# 2) they are detected asynchronously (need rootwait) +rootwait="" +part_prefix="" +if [ ! "${device#mmcblk}" = "${device}" ]; then + part_prefix="p" + rootwait="rootwait" +fi +bootfs=/dev/${device}${part_prefix}1 +rootfs=/dev/${device}${part_prefix}2 +swap=/dev/${device}${part_prefix}3 + +echo "*****************" +echo "Boot partition size: $boot_size MB ($bootfs)" +echo "Rootfs partition size: $rootfs_size MB ($rootfs)" +echo "Swap partition size: $swap_size MB ($swap)" +echo "*****************" +echo "Deleting partition table on /dev/${device} ..." +dd if=/dev/zero of=/dev/${device} bs=512 count=2 + +echo "Creating new partition table on /dev/${device} ..." +parted /dev/${device} mklabel gpt + +echo "Creating boot partition on $bootfs" +parted /dev/${device} mkpart primary 0% $boot_size +parted /dev/${device} set 1 boot on + +echo "Creating rootfs partition on $rootfs" +parted /dev/${device} mkpart primary $rootfs_start $rootfs_end + +echo "Creating swap partition on $swap" +parted /dev/${device} mkpart primary $swap_start 100% + +parted /dev/${device} print + +echo "Formatting $bootfs to vfat..." +mkfs.vfat $bootfs + +echo "Formatting $rootfs to ext3..." +mkfs.ext3 $rootfs + +echo "Formatting swap partition...($swap)" +mkswap $swap + +mkdir /ssd +mkdir /rootmnt +mkdir /bootmnt + +mount $rootfs /ssd +mount -o rw,loop,noatime,nodiratime /media/$1/$2 /rootmnt + +echo "Copying rootfs files..." +cp -a /rootmnt/* /ssd + +if [ -d /ssd/etc/ ] ; then + echo "$swap swap swap defaults 0 0" >> /ssd/etc/fstab + + # We dont want udev to mount our root device while we're booting... + if [ -d /ssd/etc/udev/ ] ; then + echo "/dev/${device}" >> /ssd/etc/udev/mount.blacklist + fi +fi + +umount /ssd +umount /rootmnt + +echo "Preparing boot partition..." +mount $bootfs /ssd + +EFIDIR="/ssd/EFI/BOOT" +mkdir -p $EFIDIR +cp /media/$1/vmlinuz /ssd +# Copy the efi loader +cp /media/$1/EFI/BOOT/*.efi $EFIDIR + +if [ -f /media/$1/EFI/BOOT/grub.cfg ]; then + GRUBCFG="$EFIDIR/grub.cfg" + cp /media/$1/EFI/BOOT/grub.cfg $GRUBCFG + # Update grub config for the installed image + # Delete the install entry + sed -i "/menuentry 'install'/,/^}/d" $GRUBCFG + # Delete the initrd lines + sed -i "/initrd /d" $GRUBCFG + # Delete any LABEL= strings + sed -i "s/ LABEL=[^ ]*/ /" $GRUBCFG + # Delete any root= strings + sed -i "s/ root=[^ ]*/ /" $GRUBCFG + # Add the root= and other standard boot options + sed -i "s@linux /vmlinuz *@linux /vmlinuz root=$rootfs rw $rootwait quiet @" $GRUBCFG +fi + +if [ -d /media/$1/loader ]; then + GUMMIBOOT_CFGS="/ssd/loader/entries/*.conf" + # copy config files for gummiboot + cp -dr /media/$1/loader /ssd + # delete the install entry + rm -f /ssd/loader/entries/install.conf + # delete the initrd lines + sed -i "/initrd /d" $GUMMIBOOT_CFGS + # delete any LABEL= strings + sed -i "s/ LABEL=[^ ]*/ /" $GUMMIBOOT_CFGS + # delete any root= strings + sed -i "s/ root=[^ ]*/ /" $GUMMIBOOT_CFGS + # add the root= and other standard boot options + sed -i "s@options *@options root=$rootfs rw $rootwait quiet @" $GUMMIBOOT_CFGS +fi + +umount /ssd +sync + +echo "Remove your installation media, and press ENTER" + +read enter + +echo "Rebooting..." +reboot -f
meta/recipes-core/initrdscripts/files/init-install-testfs.sh new file +208 -0
208 lines not on this page · raw message
Signature · 1 line
1.8.5.3
RFC PATCH 2/4 initrdscripts: add install scripts with a second rootfs
In reply to Stefan StanacarThe install scripts are similar to the default ones, but:
- custom partitioning, replaces the swap partiton with a second root filesystem
- adds labels to the partitions
- preconfigures a boot loader entry for the second
Other arches not using installers would simply deploy the image as they do, they just need to follow the same schema.
--- .../initrdscripts/files/init-install-efi-testfs.sh | 36 ++++++++++++---------- .../initrdscripts/files/init-install-testfs.sh | 33 +++++++++++--------- .../initramfs-live-install-efi-testfs_1.0.bb | 20 ++++++++++++ .../initramfs-live-install-testfs_1.0.bb | 20 ++++++++++++ 4 files changed, 77 insertions(+), 32 deletions(-) create mode 100644 meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb create mode 100644 meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb
meta/recipes-core/initrdscripts/files/init-install-efi-testfs.sh +19 -17
@@ -8,11 +8,11 @@ PATH=/sbin:/bin:/usr/sbin:/usr/bin -# We need 20 Mb for the boot partition -boot_size=20 +# We need 200 Mb for the boot partition +boot_size=200 -# 5% for swap -swap_ratio=5 +# 50% for the second rootfs +testfs_ratio=50 found="no"
@@ -71,12 +71,12 @@ cat /proc/mounts > /etc/mtab disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d" " -f 3 | sed -e "s/MB//") -swap_size=$((disk_size*swap_ratio/100)) -rootfs_size=$((disk_size-boot_size-swap_size)) +testfs_size=$((disk_size*testfs_ratio/100)) +rootfs_size=$((disk_size-boot_size-testfs_size)) rootfs_start=$((boot_size)) rootfs_end=$((rootfs_start+rootfs_size)) -swap_start=$((rootfs_end)) +testfs_start=$((rootfs_end)) # MMC devices are special in a couple of ways # 1) they use a partition prefix character 'p'
@@ -89,12 +89,12 @@ if [ ! "${device#mmcblk}" = "${device}" ]; then fi bootfs=/dev/${device}${part_prefix}1 rootfs=/dev/${device}${part_prefix}2 -swap=/dev/${device}${part_prefix}3 +testfs=/dev/${device}${part_prefix}3 echo "*****************" echo "Boot partition size: $boot_size MB ($bootfs)" echo "Rootfs partition size: $rootfs_size MB ($rootfs)" -echo "Swap partition size: $swap_size MB ($swap)" +echo "Testfs partition size: $testfs_size MB ($testfs)" echo "*****************" echo "Deleting partition table on /dev/${device} ..." dd if=/dev/zero of=/dev/${device} bs=512 count=2
@@ -109,19 +109,19 @@ parted /dev/${device} set 1 boot on echo "Creating rootfs partition on $rootfs" parted /dev/${device} mkpart primary $rootfs_start $rootfs_end -echo "Creating swap partition on $swap" -parted /dev/${device} mkpart primary $swap_start 100% +echo "Creating testfs partition on $testfs" +parted /dev/${device} mkpart primary $testfs_start 100% parted /dev/${device} print echo "Formatting $bootfs to vfat..." -mkfs.vfat $bootfs +mkfs.vfat -n "boot" $bootfs echo "Formatting $rootfs to ext3..." -mkfs.ext3 $rootfs +mkfs.ext3 -L "platform" $rootfs -echo "Formatting swap partition...($swap)" -mkswap $swap +echo "Formatting $testfs to ext3..." +mkfs.ext3 -L "testrootfs" $testfs mkdir /ssd mkdir /rootmnt
@@ -133,9 +133,9 @@ mount -o rw,loop,noatime,nodiratime /media/$1/$2 /rootmnt echo "Copying rootfs files..." cp -a /rootmnt/* /ssd -if [ -d /ssd/etc/ ] ; then - echo "$swap swap swap defaults 0 0" >> /ssd/etc/fstab +touch /ssd/etc/masterimage +if [ -d /ssd/etc/ ] ; then # We dont want udev to mount our root device while we're booting... if [ -d /ssd/etc/udev/ ] ; then echo "/dev/${device}" >> /ssd/etc/udev/mount.blacklist
@@ -184,6 +184,8 @@ if [ -d /media/$1/loader ]; then sed -i "s/ root=[^ ]*/ /" $GUMMIBOOT_CFGS # add the root= and other standard boot options sed -i "s@options *@options root=$rootfs rw $rootwait quiet @" $GUMMIBOOT_CFGS + # Add the test label + echo -ne "title test\nlinux /test-kernel\noptions root=$testfs rw $rootwait quiet\n" > /ssd/loader/entries/test.conf fi umount /ssd
meta/recipes-core/initrdscripts/files/init-install-testfs.sh +18 -15
@@ -8,10 +8,10 @@ PATH=/sbin:/bin:/usr/sbin:/usr/bin # We need 20 Mb for the boot partition -boot_size=20 +boot_size=200 -# 5% for the swap -swap_ratio=5 +# 50% for the the test partition +testfs_ratio=50 # Get a list of hard drives hdnamelist=""
@@ -94,12 +94,12 @@ cat /proc/mounts > /etc/mtab disk_size=$(parted /dev/${device} unit mb print | grep Disk | cut -d" " -f 3 | sed -e "s/MB//") -swap_size=$((disk_size*swap_ratio/100)) -rootfs_size=$((disk_size-boot_size-swap_size)) +testfs_size=$((disk_size*testfs_ratio/100)) +rootfs_size=$((disk_size-boot_size-testfs_size)) rootfs_start=$((boot_size)) rootfs_end=$((rootfs_start+rootfs_size)) -swap_start=$((rootfs_end)) +testfs_start=$((rootfs_end)) # MMC devices are special in a couple of ways # 1) they use a partition prefix character 'p'
@@ -112,12 +112,12 @@ if [ ! "${device#mmcblk}" = "${device}" ]; then fi bootfs=/dev/${device}${part_prefix}1 rootfs=/dev/${device}${part_prefix}2 -swap=/dev/${device}${part_prefix}3 +testfs=/dev/${device}${part_prefix}3 echo "*****************" echo "Boot partition size: $boot_size MB ($bootfs)" echo "Rootfs partition size: $rootfs_size MB ($rootfs)" -echo "Swap partition size: $swap_size MB ($swap)" +echo "Testfs partition size: $testfs_size MB ($testfs)" echo "*****************" echo "Deleting partition table on /dev/${device} ..." dd if=/dev/zero of=/dev/${device} bs=512 count=2
@@ -131,19 +131,19 @@ parted /dev/${device} mkpart primary 0% $boot_size echo "Creating rootfs partition on $rootfs" parted /dev/${device} mkpart primary $rootfs_start $rootfs_end -echo "Creating swap partition on $swap" -parted /dev/${device} mkpart primary $swap_start 100% +echo "Creating testfs partition on $testfs" +parted /dev/${device} mkpart primary $testfs_start 100% parted /dev/${device} print echo "Formatting $bootfs to ext3..." -mkfs.ext3 $bootfs +mkfs.ext3 -L "boot" $bootfs echo "Formatting $rootfs to ext3..." -mkfs.ext3 $rootfs +mkfs.ext3 -L "platform" $rootfs -echo "Formatting swap partition...($swap)" -mkswap $swap +echo "Formatting testfs partition...($testfs)" +mkfs.ext3 -L "testrootfs" $testfs mkdir /tgt_root mkdir /src_root
@@ -155,7 +155,6 @@ mount -o rw,loop,noatime,nodiratime /media/$1/$2 /src_root echo "Copying rootfs files..." cp -a /src_root/* /tgt_root if [ -d /tgt_root/etc/ ] ; then - echo "$swap swap swap defaults 0 0" >> /tgt_root/etc/fstab echo "$bootfs /boot ext3 defaults 1 2" >> /tgt_root/etc/fstab # We dont want udev to mount our root device while we're booting... if [ -d /tgt_root/etc/udev/ ] ; then
@@ -179,6 +178,10 @@ if [ -f /etc/grub.d/40_custom ] ; then sed -i "s/__CONSOLE__/$5/g" $GRUBCFG sed -i "/#/d" $GRUBCFG sed -i "/exec tail/d" $GRUBCFG + + # Add the test label + echo -ne "\nmenuentry 'test' {\nlinux /test-kernel root=$testfs rw $rootwait quiet\n}\n" >> $GRUBCFG + chmod 0444 $GRUBCFG fi grub-install /dev/${device}
meta/recipes-core/initrdscripts/initramfs-live-install-efi-testfs_1.0.bb new file +20 -0
@@ -0,0 +1,20 @@ +SUMMARY = "Live image install script for grub-efi with a second rootfs/kernel option" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" +SRC_URI = "file://init-install-efi-testfs.sh" + +RDEPENDS_${PN} = "parted e2fsprogs-mke2fs dosfstools" + +do_install() { + install -m 0755 ${WORKDIR}/init-install-efi-testfs.sh ${D}/install-efi.sh +} + +# While this package maybe an allarch due to it being a +# simple script, reality is that it is Host specific based +# on the COMPATIBLE_HOST below, which needs to take precedence +#inherit allarch +INHIBIT_DEFAULT_DEPS = "1" + +FILES_${PN} = " /install-efi.sh " + +COMPATIBLE_HOST = "(i.86|x86_64).*-linux"
meta/recipes-core/initrdscripts/initramfs-live-install-testfs_1.0.bb new file +20 -0
@@ -0,0 +1,20 @@ +SUMMARY = "Live image install script for grub with a second rootfs" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" +SRC_URI = "file://init-install-testfs.sh" + +RDEPENDS_${PN} = "grub parted e2fsprogs-mke2fs" + +do_install() { + install -m 0755 ${WORKDIR}/init-install-testfs.sh ${D}/install.sh +} + +# While this package maybe an allarch due to it being a +# simple script, reality is that it is Host specific based +# on the COMPATIBLE_HOST below, which needs to take precedence +#inherit allarch +INHIBIT_DEFAULT_DEPS = "1" + +FILES_${PN} = " /install.sh " + +COMPATIBLE_HOST = "(i.86|x86_64).*-linux"
Signature · 1 line
1.8.5.3
RFC PATCH 3/4 recipes-extended: add master image for testing purposes
In reply to Stefan StanacarAdd a custom recipe with custom initramfs and installers, that makes it easy to deploy a master image for testing purposes.
We need a master image running on the target hardware, that should be a known good build, with a set of utilities installed so that we use it to deploy the images under test.
This core-image-testmaster recipe isn't a requirement per se, any image can
be used as long as the required conditions are met. The test code assumes:
- that the device has a second rootfs labeled as testrootfs
- it has a properly configured bootloader entry for the second kernel and rootfs
- the master image has a /etc/masterimage file so it can differentiate between master
and test images
- the master image has tar, mount, bash (basically the normal linux utilities not the busybox
ones)
[YOCTO #5614]
--- .../images/core-image-testmaster-initramfs.bb | 20 ++++++++++++++++++++ .../recipes-extended/images/core-image-testmaster.bb | 13 +++++++++++++ 2 files changed, 33 insertions(+) create mode 100644 meta/recipes-extended/images/core-image-testmaster-initramfs.bb create mode 100644 meta/recipes-extended/images/core-image-testmaster.bb
meta/recipes-extended/images/core-image-testmaster-initramfs.bb new file +20 -0
@@ -0,0 +1,20 @@ +DESCRIPTION = "Small image capable of booting a device. The kernel includes \ +the Minimal RAM-based Initial Root Filesystem (initramfs), which finds the \ +first 'init' program more efficiently." + +PACKAGE_INSTALL = "initramfs-live-boot initramfs-live-install-testfs initramfs-live-install-efi-testfs busybox udev base-passwd ${ROOTFS_BOOTSTRAP_INSTALL}" + +# Do not pollute the initrd image with rootfs features +IMAGE_FEATURES = "" + +export IMAGE_BASENAME = "core-image-testmaster-initramfs" +IMAGE_LINGUAS = "" + +LICENSE = "MIT" + +IMAGE_FSTYPES = "${INITRAMFS_FSTYPES}" +inherit core-image + +IMAGE_ROOTFS_SIZE = "8192" + +BAD_RECOMMENDATIONS += "busybox-syslog"
meta/recipes-extended/images/core-image-testmaster.bb new file +13 -0
@@ -0,0 +1,13 @@ +DESCRIPTION = "A master image to be deployed on a target useful for testing other images" + +IMAGE_FEATURES += "ssh-server-openssh package-management" + +inherit core-image + +IMAGE_INSTALL += "\ + bash coreutils util-linux tar gzip kmod \ + python-modules python-misc \ + e2fsprogs e2fsprogs-mke2fs parted \ + " +INITRD_IMAGE = "core-image-testmaster-initramfs" +
Signature · 1 line
1.8.5.3
RFC PATCH 4/4 lib/oeqa: add a test target controller for EFI targets
In reply to Stefan StanacarThe purpose of this module is to deploy a test image on a EFI-enabled hardware and run our runtime tests.
A bit of background:
- testimage.bbclass uses the concept of TEST_TARGET which is a class name
that is responsible for target deploying. A layer can provide
it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget (ssh into an already up and running machine and run tests), the default one being qemu.
- basically testimage does something like:
- target.deploy()
- target.start()
- runTests()
- target.stop()
This module assumes a running EFI (and gummiboot as bootloader) machine with core-image-testmaster installed (or similar).
In order to use this Master Image mechanism there are some hard requirements:
- it only works for EFI-enabled hardware with EFI_PROVIDER = "gummiboot"
- your hardware under test has to be in DHCP-enabled network that gives it the same IP for each reboot
One time setup (master image):
- build core-image-testmaster with EFI_PROVIDER = "gummiboot"
- TEMP: for now you'll need SRCREV_meta_pn-linux-yocto = "${AUTOREV}" in local.conf so that some kernel configs required by gummiboot get enabled
- install the image on the target
Test image setup:
- build your test image, e.g core-image-sato as you usually do, but with these in local.conf:
IMAGE_FSTYPES += "tar.gz"
(similar to above you need SRCREV_meta_pn-linux-yocto = "${AUTOREV}")
- Now run the tests:
INHERIT += "testimage"
TEST_TARGET = "GenericEfi"
TEST_TARGET_IP = "192.168.2.3"
bitbake core-image-sato -c testimage
Other notes:
- TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling. The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like TEST_POWERCONTROL_CMD="/home/stefans/powercontrol.exp test 192.168.2.1 nuc1" in local.conf.
Basically my expect script does: 'ssh [email protected] and runs a python script there that controls power for a label called nuc1'.
- if no command is defined it would use classic reboot. This is fine as long as the machine
actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where some manual interaction is okay from time to time.
[YOCTO #5614]
--- meta/lib/oeqa/controllers/masterimage.py | 124 +++++++++++++++++++++++++++++++ meta/lib/oeqa/runtime/ssh.py | 2 + 2 files changed, 126 insertions(+) create mode 100644 meta/lib/oeqa/controllers/masterimage.py
meta/lib/oeqa/controllers/masterimage.py new file +124 -0
@@ -0,0 +1,124 @@ +import os +import bb +import traceback +import time + +import oeqa.targetcontrol +import oeqa.utils.sshcontrol as sshcontrol +import oeqa.utils.commands as commands + +class GenericEfi(oeqa.targetcontrol.SimpleRemoteTarget): + + def __init__(self, d): + # let our base class do the ip thing + super(GenericEfi, self).__init__(d) + + # test rootfs + kernel + self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz') + self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE")) + if not os.path.isfile(self.rootfs): + # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be + # the same as the config with which the image was build, ie + # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz" + # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage + bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \ + \nExpected path: %s" % self.rootfs) + if not os.path.isfile(self.kernel): + bb.fatal("No kernel found. Expected path: %s" % self.kernel) + + # if the user knows what he's doing, then by all means... + # test-rootfs.tar.gz and test-kernel are hardcoded names in other places + # they really have to be used like that in commands though + cmds = d.getVar("TEST_DEPLOY_CMDS", True) + if cmds: + self.deploy_cmds = cmds.split("\n") + else: + self.deploy_cmds = [ + 'mount -L boot /boot', + 'mkdir -p /mnt/testrootfs', + 'mount -L testrootfs /mnt/testrootfs', + 'modprobe efivarfs', + 'mount -t efivarfs efivarfs /sys/firmware/efi/efivars', + 'cp ~/test-kernel /boot', + 'rm -rf /mnt/testrootfs/*', + 'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs', + r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" > /sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f' + ] + + # master ssh connection + self.master = None + + # this is the name of the command that controls the power for a board + # e.g: TEST_POWERCONTROL_CMD = "/home/user/myscripts/powercontrol.py ${MACHINE} what-ever-other-args-the-script-wants" + # the command should take as the last argument "off" and "on" and "cycle" (off, on) + self.powercontrol_cmd = d.getVar("TEST_POWERCONTROL_CMD", True) or None + self.powercontrol_args = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or "" + self.origenv = os.environ + if self.powercontrol_cmd: + if self.powercontrol_args: + self.powercontrol_cmd = "%s %s" % (self.powercontrol_cmd, self.powercontrol_args) + # the external script for controlling power might use ssh + # ssh + keys means we need the original user env + bborigenv = d.getVar("BB_ORIGENV", False) or {} + for key in bborigenv: + val = bborigenv.getVar(key, True) + if val is not None: + self.origenv[key] = str(val) + self.power_ctl("on") + + def power_ctl(self, msg): + if self.powercontrol_cmd: + cmd = "%s %s" % (self.powercontrol_cmd, msg) + commands.runCmd(cmd, preexec_fn=os.setsid, env=self.origenv) + + def power_cycle(self, conn): + if self.powercontrol_cmd: + # be nice, don't just cut power + conn.run("shutdown -h now") + time.sleep(10) + self.power_ctl("cycle") + else: + status, output = conn.run("reboot") + if status != 0: + bb.error("Failed rebooting target and no power control command defined. You need to manually reset the device.\n%s" % output) + + def deploy(self): + bb.plain("%s - deploying image on target" % self.pn) + # base class just sets the ssh log file for us + super(GenericEfi, self).deploy() + self.master = sshcontrol.SSHControl(ip=self.ip, logfile=self.sshlog, timeout=600, port=self.port) + try: + self._deploy() + except Exception as e: + bb.fatal("Failed deploying test image: %s" % e) + + def _deploy(self): + # make sure we are in the right image + status, output = self.master.run("cat /etc/masterimage") + if status != 0: + raise Exception("No ssh connectivity or target isn't running a master image.\n%s" % output) + + # make sure these aren't mounted + self.master.run("umount /boot; umount /mnt/testrootfs; umount /sys/firmware/efi/efivars;") + + # from now on, every deploy cmd should return 0 + self.master.ignore_status = False + self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz") + self.master.copy_to(self.kernel, "~/test-kernel") + for cmd in self.deploy_cmds: + self.master.run(cmd) + + + def start(self, params=None): + bb.plain("%s - boot test image on target" % self.pn) + self.power_cycle(self.master) + # assuming the reboot worked, we need to wait a bit + # there are better ways than a timeout but this should work for my purpose for now + time.sleep(120) + # we are live, set the ssh object for the target/test image + self.connection = sshcontrol.SSHControl(self.ip, logfile=self.sshlog, port=self.port) + bb.plain("%s - start running tests" % self.pn) + + def stop(self): + bb.plain("%s - reboot/powercycle target" % self.pn) + self.power_cycle(self.connection)
meta/lib/oeqa/runtime/ssh.py +2 -0
@@ -14,3 +14,5 @@ class SshTest(oeRuntimeTest): def test_ssh(self): (status, output) = self.target.run('uname -a') self.assertEqual(status, 0, msg="SSH Test failed: %s" % output) + (status, output) = self.target.run('cat /etc/masterimage') + self.assertEqual(status, 1, msg="This isn't the right image - /etc/masterimage shouldn't be here %s" % output)
Signature · 1 line
1.8.5.3
Re: [RFC PATCH 4/4] lib/oeqa: add a test target controller for EFI targets
Quoted 44 lines · Stefan Stanacar
The purpose of this module is to deploy a test image on a EFI-enabled hardware and run our runtime tests.
A bit of background:
- testimage.bbclass uses the concept ofTEST_TARGETwhich is a class name
that is responsible for target deploying. A layer can provide
it's ownTEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget (ssh into an already up and running machine and run tests), the default one being qemu.
- basically testimage does something like:
-target.deploy()
-target.start()
-runTests()
-target.stop()This module assumes a running EFI (and gummiboot as bootloader) machine with core-image-testmaster installed (or similar).
In order to use this Master Image mechanism there are some hard requirements:
- it only works for EFI-enabled hardware withEFI_PROVIDER= "gummiboot"
- your hardware under test has to be in DHCP-enabled network that gives it the same IP for each rebootOne time setup (master image):
- build core-image-testmaster withEFI_PROVIDER= "gummiboot"
- TEMP: for now you'll need SRCREV_meta_pn-linux-yocto = "${AUTOREV}" in local.conf so that some kernel configs required by gummiboot get enabled
- install the image on the targetTest image setup:
- build your test image, e.g core-image-sato as you usually do, but with these in local.conf:IMAGE_FSTYPES += "tar.gz"(similar to above you need SRCREV_meta_pn-linux-yocto = "${AUTOREV}")
- Now run the tests:
INHERIT += "testimage"
TEST_TARGET= "GenericEfi"
TEST_TARGET_IP= "192.168.2.3"
bitbake core-image-sato -c testimageOther notes:
-TEST_POWERCONTROL_CMD(togheter withTEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling. The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like TEST_POWERCONTROL_CMD="/home/stefans/powercontrol.exp test 192.168.2.1 nuc1" in local.conf.
Basically my expect script does: 'ssh [email protected] and runs a python script there that controls power for a label called nuc1'.
- if no command is defined it would use classic reboot. This is fine as long as the machine
actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where some manual interaction is okay from time to time.
Quoted 3 lines
[YOCTO #5614]
Signed-off-by Stefan Stanacar <[email protected]>
Nice.
Quoted 27 lines
--- meta/lib/oeqa/controllers/masterimage.py | 124 +++++++++++++++++++++++++++++++ meta/lib/oeqa/runtime/ssh.py | 2 + 2 files changed, 126 insertions(+) create mode 100644 meta/lib/oeqa/controllers/masterimage.pymeta/lib/oeqa/controllers/masterimage.py new file +15
-0@@ -0,0 +1,124 @@ +import os +import bb +import traceback +import time + +import oeqa.targetcontrol +import oeqa.utils.sshcontrol as sshcontrol +import oeqa.utils.commands as commands + +class GenericEfi(oeqa.targetcontrol.SimpleRemoteTarget): + + def __init__(self, d): + # let our base class do the ip thing + super(GenericEfi, self).__init__(d) +
meta/lib/oeqa/controllers/masterimage.py line 25
Quoted 10 lines
meta/lib/oeqa/controllers/masterimage.py new file +10
-0+ # test rootfs + kernel + self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz') + self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE")) + if not os.path.isfile(self.rootfs): + # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be + # the same as the config with which the image was build, ie + # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz" + # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage + bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \ + \nExpected path: %s" % self.rootfs)
Couldn't testimage class add it?
meta/lib/oeqa/controllers/masterimage.py line 34
Quoted 9 lines
meta/lib/oeqa/controllers/masterimage.py new file +9
-0+ if not os.path.isfile(self.kernel): + bb.fatal("No kernel found. Expected path: %s" % self.kernel) + + # if the user knows what he's doing, then by all means... + # test-rootfs.tar.gz and test-kernel are hardcoded names in other places + # they really have to be used like that in commands though + cmds = d.getVar("TEST_DEPLOY_CMDS", True) + if cmds: + self.deploy_cmds = cmds.split("\n")
Good.
meta/lib/oeqa/controllers/masterimage.py line 44
Quoted 10 lines
meta/lib/oeqa/controllers/masterimage.py new file +10
-0+ else: + self.deploy_cmds = [ + 'mount -L boot /boot', + 'mkdir -p /mnt/testrootfs', + 'mount -L testrootfs /mnt/testrootfs', + 'modprobe efivarfs', + 'mount -t efivarfs efivarfs /sys/firmware/efi/efivars', + 'cp ~/test-kernel /boot', + 'rm -rf /mnt/testrootfs/*', + 'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',
How target see those files? NFS?
meta/lib/oeqa/controllers/masterimage.py line 45
Quoted 1 line
meta/lib/oeqa/controllers/masterimage.py new file +1
-0+ r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" > /sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'
Can you elaborate this? Seems too 'black magic' for me.
Quoted 97 lines
meta/lib/oeqa/controllers/masterimage.py new file +79
-0+ ] + + # master ssh connection + self.master = None + + # this is the name of the command that controls the power for a board + # e.g: TEST_POWERCONTROL_CMD = "/home/user/myscripts/powercontrol.py ${MACHINE} what-ever-other-args-the-script-wants" + # the command should take as the last argument "off" and "on" and "cycle" (off, on) + self.powercontrol_cmd = d.getVar("TEST_POWERCONTROL_CMD", True) or None + self.powercontrol_args = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or "" + self.origenv = os.environ + if self.powercontrol_cmd: + if self.powercontrol_args: + self.powercontrol_cmd = "%s %s" % (self.powercontrol_cmd, self.powercontrol_args) + # the external script for controlling power might use ssh + # ssh + keys means we need the original user env + bborigenv = d.getVar("BB_ORIGENV", False) or {} + for key in bborigenv: + val = bborigenv.getVar(key, True) + if val is not None: + self.origenv[key] = str(val) + self.power_ctl("on") + + def power_ctl(self, msg): + if self.powercontrol_cmd: + cmd = "%s %s" % (self.powercontrol_cmd, msg) + commands.runCmd(cmd, preexec_fn=os.setsid, env=self.origenv) + + def power_cycle(self, conn): + if self.powercontrol_cmd: + # be nice, don't just cut power + conn.run("shutdown -h now") + time.sleep(10) + self.power_ctl("cycle") + else: + status, output = conn.run("reboot") + if status != 0: + bb.error("Failed rebooting target and no power control command defined. You need to manually reset the device.\n%s" % output) + + def deploy(self): + bb.plain("%s - deploying image on target" % self.pn) + # base class just sets the ssh log file for us + super(GenericEfi, self).deploy() + self.master = sshcontrol.SSHControl(ip=self.ip, logfile=self.sshlog, timeout=600, port=self.port) + try: + self._deploy() + except Exception as e: + bb.fatal("Failed deploying test image: %s" % e) + + def _deploy(self): + # make sure we are in the right image + status, output = self.master.run("cat /etc/masterimage") + if status != 0: + raise Exception("No ssh connectivity or target isn't running a master image.\n%s" % output) + + # make sure these aren't mounted + self.master.run("umount /boot; umount /mnt/testrootfs; umount /sys/firmware/efi/efivars;") + + # from now on, every deploy cmd should return 0 + self.master.ignore_status = False + self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz") + self.master.copy_to(self.kernel, "~/test-kernel") + for cmd in self.deploy_cmds: + self.master.run(cmd) + + + def start(self, params=None): + bb.plain("%s - boot test image on target" % self.pn) + self.power_cycle(self.master) + # assuming the reboot worked, we need to wait a bit + # there are better ways than a timeout but this should work for my purpose for now + time.sleep(120) + # we are live, set the ssh object for the target/test image + self.connection = sshcontrol.SSHControl(self.ip, logfile=self.sshlog, port=self.port) + bb.plain("%s - start running tests" % self.pn) + + def stop(self): + bb.plain("%s - reboot/powercycle target" % self.pn) + self.power_cycle(self.connection)meta/lib/oeqa/runtime/ssh.py +2
-1@@ -14,3 +14,5 @@ class SshTest(oeRuntimeTest): def test_ssh(self): (status, output) = self.target.run('uname -a') self.assertEqual(status, 0, msg="SSH Test failed: %s" % output) + (status, output) = self.target.run('cat /etc/masterimage') + self.assertEqual(status, 1, msg="This isn't the right image - /etc/masterimage shouldn't be here %s" % output) --1.8.5.3
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
Re: [RFC PATCH 4/4] lib/oeqa: add a test target controller for EFI targets
Hi Otavio,
Quoted 14 lines · Otavio Salvador
meta/lib/oeqa/controllers/masterimage.py line 25
Quoted 10 lines · Stefan Stanacar
meta/lib/oeqa/controllers/masterimage.py new file +10
-0+ # test rootfs + kernel + self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz') + self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE")) + if not os.path.isfile(self.rootfs): + # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be + # the same as the config with which the image was build, ie + # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz" + # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage + bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \ + \nExpected path: %s" % self.rootfs)
Quoted 2 lines
Couldn't testimage class add it?
Nope... testimage task can be triggered in two ways: from testimage.bbclass or testimage-auto.bbclass, that means:
- manually: you have already built your image, then add INHERIT +=
"testimage" in local.conf and run bitbake <image> -c testimage (the AB does it this way). The testimage task does not have a depends on do_rootfs, so it won't create a tar.gz rootfs if IMAGE_FSTYPES wasn't added to local.conf at build time.
- automatically: you add TEST_IMAGE = "1" in local.conf before bitbake
<image>. That's a hidden inherit in image.bbclass that inherits testimage-auto that does depend on rootfs.
We want to support both cases... So checking that tar.gz is IMAGE_FSTYPES doesn't mean that the image actually was built with that. Also, tar.gz might not be there although the image was built with that (The AB has two different build steps for build and running tests, and in between the config can be overwritten).
Quoted 23 lines
meta/lib/oeqa/controllers/masterimage.py line 34
Quoted 9 lines
meta/lib/oeqa/controllers/masterimage.py new file +9
-0+ if not os.path.isfile(self.kernel): + bb.fatal("No kernel found. Expected path: %s" % self.kernel) + + # if the user knows what he's doing, then by all means... + # test-rootfs.tar.gz and test-kernel are hardcoded names in other places + # they really have to be used like that in commands though + cmds = d.getVar("TEST_DEPLOY_CMDS", True) + if cmds: + self.deploy_cmds = cmds.split("\n")Good.
meta/lib/oeqa/controllers/masterimage.py line 44
Quoted 10 lines
meta/lib/oeqa/controllers/masterimage.py new file +10
-0+ else: + self.deploy_cmds = [ + 'mount -L boot /boot', + 'mkdir -p /mnt/testrootfs', + 'mount -L testrootfs /mnt/testrootfs', + 'modprobe efivarfs', + 'mount -t efivarfs efivarfs /sys/firmware/efi/efivars', + 'cp ~/test-kernel /boot', + 'rm -rf /mnt/testrootfs/*', + 'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',
Quoted 1 line
How target see those files? NFS?
Nope, see below in _deploy.
Those files are scp'ed from the build machine to the target.
self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz") self.master.copy_to(self.kernel, "~/test-kernel") for cmd in self.deploy_cmds:
self.master.run(cmd)
Quoted 5 lines
Quoted 2 lines
+ r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" >
/sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'Can you elaborate this? Seems too 'black magic' for me.
Okay so one of the reasons this GenericEfi target requires that the master image deployed uses gummiboot is because of that. gummiboot actually reads and obeys system EFI variables. [1] One of those variables is LoaderEntryOneShot which tells it to boot a label (kernel + rootfs) once and only once so this how we know that we always get back to the master/good image.
[1] "Some EFI variables control the loader or exported the loaders state to the started operating system. The vendor UUID 4a67b082-0a4c-41cf-b6c7-440b29bb8c4f and the variable names are supposed to be shared across all loaders implementations which follow this scheme of configuration: "
http://freedesktop.org/wiki/Software/gummiboot/
Accessing the EFI vars subsystem is done in two ways:
- old sysfs-efivars interface (CONFIG_EFI_VARS), populated
at /sys/firmware/efi/vars, which has some limitations (basically you need another tool to read/write)
- new efivarfs interface (CONFIG_EFIVAR_FS), typically mounted like
this: mount -t efivarfs efivarfs /sys/firmware/efi/efivars It was added in 3.8 intended as a replacement for the sysfs-efivars interface, has no maximum per-variable size limitation and supports UEFI Secure Boot variables.
You can read more about that at:
http://uefidk.intel.com/blog/accessing-uefi-variables-linux#sthash.2wI28Mkn.dpuf
The first four bytes are EFI_VARIABLE_NON_VOLATILE, EFI_VARIABLE_BOOTSERVICE_ACCESS and EFI_VARIABLE_RUNTIME_ACCESS bits set in the attribute bitmask, and the rest is just the output from:
[stefans@firebird ~]$ echo -en "test" | iconv -f ascii -t utf-16le | hexdump -C
00000000 74 00 65 00 73 00 74 00 |t.e.s.t.|
00000008
[stefans@firebird ~]$
The gummiboot loader config that gets installed with the master image has two labels: boot and test and this is how we tell it that for the next boot only we want the test label.
The simplest tool to read EFI vars I've found is efivar. I've used it when debugging and playing with this stuff.
There is a recipe here:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=stefans/wip2&id=3df22b8ab36553052381458314b43b76cd1d3dff
I haven't submitted it because I don't know how many people need it.
HTH,
Cheers,
Stefan
Quoted 100 lines
Quoted 97 lines
meta/lib/oeqa/controllers/masterimage.py new file +79
-0+ ] + + # master ssh connection + self.master = None + + # this is the name of the command that controls the power for a board + # e.g: TEST_POWERCONTROL_CMD = "/home/user/myscripts/powercontrol.py ${MACHINE} what-ever-other-args-the-script-wants" + # the command should take as the last argument "off" and "on" and "cycle" (off, on) + self.powercontrol_cmd = d.getVar("TEST_POWERCONTROL_CMD", True) or None + self.powercontrol_args = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or "" + self.origenv = os.environ + if self.powercontrol_cmd: + if self.powercontrol_args: + self.powercontrol_cmd = "%s %s" % (self.powercontrol_cmd, self.powercontrol_args) + # the external script for controlling power might use ssh + # ssh + keys means we need the original user env + bborigenv = d.getVar("BB_ORIGENV", False) or {} + for key in bborigenv: + val = bborigenv.getVar(key, True) + if val is not None: + self.origenv[key] = str(val) + self.power_ctl("on") + + def power_ctl(self, msg): + if self.powercontrol_cmd: + cmd = "%s %s" % (self.powercontrol_cmd, msg) + commands.runCmd(cmd, preexec_fn=os.setsid, env=self.origenv) + + def power_cycle(self, conn): + if self.powercontrol_cmd: + # be nice, don't just cut power + conn.run("shutdown -h now") + time.sleep(10) + self.power_ctl("cycle") + else: + status, output = conn.run("reboot") + if status != 0: + bb.error("Failed rebooting target and no power control command defined. You need to manually reset the device.\n%s" % output) + + def deploy(self): + bb.plain("%s - deploying image on target" % self.pn) + # base class just sets the ssh log file for us + super(GenericEfi, self).deploy() + self.master = sshcontrol.SSHControl(ip=self.ip, logfile=self.sshlog, timeout=600, port=self.port) + try: + self._deploy() + except Exception as e: + bb.fatal("Failed deploying test image: %s" % e) + + def _deploy(self): + # make sure we are in the right image + status, output = self.master.run("cat /etc/masterimage") + if status != 0: + raise Exception("No ssh connectivity or target isn't running a master image.\n%s" % output) + + # make sure these aren't mounted + self.master.run("umount /boot; umount /mnt/testrootfs; umount /sys/firmware/efi/efivars;") + + # from now on, every deploy cmd should return 0 + self.master.ignore_status = False + self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz") + self.master.copy_to(self.kernel, "~/test-kernel") + for cmd in self.deploy_cmds: + self.master.run(cmd) + + + def start(self, params=None): + bb.plain("%s - boot test image on target" % self.pn) + self.power_cycle(self.master) + # assuming the reboot worked, we need to wait a bit + # there are better ways than a timeout but this should work for my purpose for now + time.sleep(120) + # we are live, set the ssh object for the target/test image + self.connection = sshcontrol.SSHControl(self.ip, logfile=self.sshlog, port=self.port) + bb.plain("%s - start running tests" % self.pn) + + def stop(self): + bb.plain("%s - reboot/powercycle target" % self.pn) + self.power_cycle(self.connection)meta/lib/oeqa/runtime/ssh.py +2
-1@@ -14,3 +14,5 @@ class SshTest(oeRuntimeTest): def test_ssh(self): (status, output) = self.target.run('uname -a') self.assertEqual(status, 0, msg="SSH Test failed: %s" % output) + (status, output) = self.target.run('cat /etc/masterimage') + self.assertEqual(status, 1, msg="This isn't the right image - /etc/masterimage shouldn't be here %s" % output) --1.8.5.3
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [RFC PATCH 4/4] lib/oeqa: add a test target controller for EFI targets
Quoted 29 lines · StefanX
Quoted 14 lines · Otavio Salvador
meta/lib/oeqa/controllers/masterimage.py line 25
Quoted 10 lines · Stefan Stanacar
meta/lib/oeqa/controllers/masterimage.py new file +10
-0+ # test rootfs + kernel + self.rootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.tar.gz') + self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE")) + if not os.path.isfile(self.rootfs): + # we could've checked that IMAGE_FSTYPES contains tar.gz but the config for running testimage might not be + # the same as the config with which the image was build, ie + # you bitbake core-image-sato with IMAGE_FSTYPES += "tar.gz" + # and your autobuilder overwrites the config, adds the test bits and runs bitbake core-image-sato -c testimage + bb.fatal("No rootfs found. Did you build the image ?\nIf yes, did you build it with IMAGE_FSTYPES += \"tar.gz\" ? \ + \nExpected path: %s" % self.rootfs)Quoted 2 lines
Couldn't testimage class add it?
Nope... testimage task can be triggered in two ways: from testimage.bbclass or testimage-auto.bbclass, that means:
- manually: you have already built your image, then add INHERIT +=
"testimage" in local.conf and run bitbake <image> -c testimage (the AB does it this way). The testimage task does not have a depends ondo_rootfs, so it won't create a tar.gz rootfs ifIMAGE_FSTYPESwasn't added to local.conf at build time.
- automatically: you addTEST_IMAGE= "1" in local.conf before bitbake
<image>. That's a hidden inherit in image.bbclass that inherits testimage-auto that does depend on rootfs.
Quoted 5 lines
We want to support both cases... So checking that tar.gz is
IMAGE_FSTYPESdoesn't mean that the image actually was built with that. Also, tar.gz might not be there although the image was built with that (The AB has two different build steps for build and running tests, and in between the config can be overwritten).
Ok.
Quoted 25 lines
Quoted 23 lines
meta/lib/oeqa/controllers/masterimage.py line 34
Quoted 9 lines
meta/lib/oeqa/controllers/masterimage.py new file +9
-0+ if not os.path.isfile(self.kernel): + bb.fatal("No kernel found. Expected path: %s" % self.kernel) + + # if the user knows what he's doing, then by all means... + # test-rootfs.tar.gz and test-kernel are hardcoded names in other places + # they really have to be used like that in commands though + cmds = d.getVar("TEST_DEPLOY_CMDS", True) + if cmds: + self.deploy_cmds = cmds.split("\n")Good.
meta/lib/oeqa/controllers/masterimage.py line 44
Quoted 10 lines
meta/lib/oeqa/controllers/masterimage.py new file +10
-0+ else: + self.deploy_cmds = [ + 'mount -L boot /boot', + 'mkdir -p /mnt/testrootfs', + 'mount -L testrootfs /mnt/testrootfs', + 'modprobe efivarfs', + 'mount -t efivarfs efivarfs /sys/firmware/efi/efivars', + 'cp ~/test-kernel /boot', + 'rm -rf /mnt/testrootfs/*', + 'tar xzvf ~/test-rootfs.tar.gz -C /mnt/testrootfs',Quoted 1 line
How target see those files? NFS?
Quoted 7 lines
Nope, see below in
_deploy.
Those files are scp'ed from the build machine to the target.
self.master.copy_to(self.rootfs, "~/test-rootfs.tar.gz")self.master.copy_to(self.kernel, "~/test-kernel")for cmd in self.deploy_cmds:
self.master.run(cmd)
What happens if the image is bigger than RAM size?
Quoted 11 lines
Quoted 4 lines
Quoted 1 line
+ r'printf "\x07\x00\x00\x00\x74\x00\x65\x00\x73\x00\x74\x00\x00\x00" >
/sys/firmware/efi/efivars/LoaderEntryOneShot-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f'Can you elaborate this? Seems too 'black magic' for me.
Okay so one of the reasons this GenericEfi target requires that the master image deployed uses gummiboot is because of that. gummiboot actually reads and obeys system EFI variables. [1] One of those variables is LoaderEntryOneShot which tells it to boot a label (kernel + rootfs) once and only once so this how we know that we always get back to the master/good image.
So it is misnamed. GummiBootEfi would be clearer.
Quoted 20 lines
[1] "Some EFI variables control the loader or exported the loaders state to the started operating system. The vendor UUID 4a67b082-0a4c-41cf-b6c7-440b29bb8c4f and the variable names are supposed to be shared across all loaders implementations which follow this scheme of configuration: "
http://freedesktop.org/wiki/Software/gummiboot/Accessing the EFI vars subsystem is done in two ways:
- old sysfs-efivars interface (CONFIG_EFI_VARS), populated
at/sys/firmware/efi/vars, which has some limitations (basically you need another tool to read/write)
- new efivarfs interface (CONFIG_EFIVAR_FS), typically mounted like
this: mount -t efivarfs efivarfs/sys/firmware/efi/efivarsIt was added in 3.8 intended as a replacement for the sysfs-efivars interface, has no maximum per-variable size limitation and supports UEFI Secure Boot variables.
You can read more about that at:
http://uefidk.intel.com/blog/accessing-uefi-variables-linux#sthash.2wI28Mkn.dpuf
Quoted 8 lines
The first four bytes are
EFI_VARIABLE_NON_VOLATILE,EFI_VARIABLE_BOOTSERVICE_ACCESSandEFI_VARIABLE_RUNTIME_ACCESSbits set in the attribute bitmask, and the rest is just the output from:
[stefans@firebird ~]$ echo -en "test" | iconv -f ascii -t utf-16le | hexdump -C
00000000 74 00 65 00 73 00 74 00 |t.e.s.t.|
00000008
[stefans@firebird ~]$
So please make those constants in the code and proper documented.
Quoted 8 lines
The gummiboot loader config that gets installed with the master image has two labels: boot and test and this is how we tell it that for the next boot only we want the test label.
The simplest tool to read EFI vars I've found is efivar. I've used it when debugging and playing with this stuff.
There is a recipe here:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=stefans/wip2&id=3df22b8ab36553052381458314b43b76cd1d3dff
I think it would be nice for debugging so in worse case send t to meta-oe
:-)
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750