Support for wic images in runqemu and oe-selftest
cover Support for wic images in runqemu and oe-selftest
Hi,
This patchset contains set of fixes for wic, runqemu and oe-selftest to support booting and testing wic images in qemu.
This work is based on Cristian's work from poky-contrib:ciorga/wic-testing Thank you Cristian!
The patchset also includes bugfix for wic --use-uuid option and a bit of code cleanup for wic tests.
[YOCTO #8691]
[YOCTO #8498]
[YOCTO #8499]
The following changes since commit 1ba62f9e53f5e2d0da9fe6ee52007562deb77f32:
pth: Remove dead code (2016-02-01 17:16:05 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/oe-core/qemuRunner-wic-8498
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/oe-core/qemuRunner-wic-8498
Ed Bartosh (15):
runqemu: add support for wic images
runqemu: don't use virtio for wic images
runqemu: don't set KERNEL for wic images
runqemu: support path/to/<image>-<machine>.wic
oe-selftest: remove unused parameter
qemurunner: add parameter to method 'start'
oeqa/targetcontrol: make ssh control optional
oeqa/targetcontrol: support wic image type
wic-image-minimal: change IMAGE_FSTYPES
wic-image-minimal: updated .wks to boot by qemu
oe-selftest: add new wic testcase
wic: fix processing of --use-uuid
wic-image-minimal: use uuid for root partition
oe-selftest: pylinted wic tests
oe-selftest: don't use specific tasks
.../recipes-test/images/wic-image-minimal.bb | 2 +- .../recipes-test/images/wic-image-minimal.wks | 10 ++--- meta/lib/oeqa/selftest/devtool.py | 2 +- meta/lib/oeqa/selftest/imagefeatures.py | 6 +-- meta/lib/oeqa/selftest/wic.py | 32 +++++++++------ meta/lib/oeqa/targetcontrol.py | 20 +++++----- meta/lib/oeqa/utils/commands.py | 4 +- meta/lib/oeqa/utils/qemurunner.py | 45 +++++++++++----------- scripts/lib/wic/partition.py | 3 ++ scripts/runqemu | 15 +++++--- scripts/runqemu-internal | 21 +++++----- 11 files changed, 90 insertions(+), 70 deletions(-)
--
Regards,
Ed
PATCH 1/15 runqemu: add support for wic images
Quemu should be able to run wic images this way:
runqemu <machine> <image recipe> wic
Tested with 'runqemu qemux86-64 wic-image-minimal wic'
--- scripts/runqemu | 3 ++- scripts/runqemu-internal | 16 ++++++++-------- 2 files changed, 10 insertions(+), 9 deletions(-)
scripts/runqemu +2 -1
@@ -40,6 +40,7 @@ usage() { echo "Examples:" echo " $MYNAME qemuarm" echo " $MYNAME qemux86-64 core-image-sato ext4" + echo " $MYNAME qemux86-64 wic-image-minimal wic" echo " $MYNAME path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial" echo " $MYNAME qemux86 ramfs" echo " $MYNAME qemux86 iso"
@@ -116,7 +117,7 @@ while true; do [ -z "$MACHINE" -o "$MACHINE" = "$arg" ] && MACHINE=$arg || \ error "conflicting MACHINE types [$MACHINE] and [$arg]" ;; - "ext2" | "ext3" | "ext4" | "jffs2" | "nfs" | "btrfs" | "hddimg" | "hdddirect" ) + "ext2" | "ext3" | "ext4" | "jffs2" | "nfs" | "btrfs" | "hddimg" | "hdddirect" | "wic" ) [ -z "$FSTYPE" -o "$FSTYPE" = "$arg" ] && FSTYPE=$arg || \ error "conflicting FSTYPE types [$FSTYPE] and [$arg]" ;;
scripts/runqemu-internal +8 -8
@@ -349,7 +349,7 @@ if [ "$MACHINE" = "qemuarm" -o "$MACHINE" = "qemuarmv6" -o "$MACHINE" = "qemuarm export QEMU_AUDIO_DRV="none" QEMU_UI_OPTIONS="$QEMU_UI_OPTIONS" # QEMU_UI_OPTIONS="$QEMU_UI_OPTIONS -force-pointer" - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then KERNCMDLINE="root=$DROOT rw console=ttyAMA0,115200 console=tty $KERNEL_NETWORK_CMD mem=$QEMU_MEMORY highres=off" QEMUOPTIONS="$QEMU_NETWORK_CMD -M ${MACHINE_SUBTYPE} $ROOTFS_OPTIONS -no-reboot $QEMU_UI_OPTIONS" fi
@@ -383,7 +383,7 @@ if [ "$MACHINE" = "qemuarm64" ]; then else QEMU_UI_OPTIONS="" fi - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then KERNCMDLINE="root=$DROOT rw console=ttyAMA0,38400 mem=$QEMU_MEMORY highres=off $KERNEL_NETWORK_CMD" # qemu-system-aarch64 only support '-machine virt -cpu cortex-a57' for now QEMUOPTIONS="$QEMU_NETWORK_CMD -machine virt -cpu cortex-a57 $ROOTFS_OPTIONS $QEMU_UI_OPTIONS"
@@ -411,7 +411,7 @@ if [ "$MACHINE" = "qemux86" ]; then else QEMU_UI_OPTIONS="$QEMU_UI_OPTIONS -vga vmware" fi - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then KERNCMDLINE="vga=0 uvesafb.mode_option=640x480-32 root=$DROOT rw mem=$QEMU_MEMORY $KERNEL_NETWORK_CMD" QEMUOPTIONS="$QEMU_NETWORK_CMD -cpu $CPU_SUBTYPE $ROOTFS_OPTIONS $QEMU_UI_OPTIONS" fi
@@ -449,7 +449,7 @@ if [ "$MACHINE" = "qemux86-64" ]; then else QEMU_UI_OPTIONS="$QEMU_UI_OPTIONS -vga vmware" fi - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then KERNCMDLINE="vga=0 uvesafb.mode_option=640x480-32 root=$DROOT rw mem=$QEMU_MEMORY $KERNEL_NETWORK_CMD" QEMUOPTIONS="$QEMU_NETWORK_CMD -cpu $CPU_SUBTYPE $ROOTFS_OPTIONS $QEMU_UI_OPTIONS" fi
@@ -495,7 +495,7 @@ if [ "$MACHINE" = "qemumips" -o "$MACHINE" = "qemumipsel" -o "$MACHINE" = "qemum esac MACHINE_SUBTYPE=malta QEMU_UI_OPTIONS="-vga cirrus $QEMU_UI_OPTIONS" - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then #KERNCMDLINE="root=/dev/hda console=ttyS0 console=tty0 $KERNEL_NETWORK_CMD mem=$QEMU_MEMORY" KERNCMDLINE="root=$DROOT rw console=ttyS0 console=tty $KERNEL_NETWORK_CMD mem=$QEMU_MEMORY" QEMUOPTIONS="$QEMU_NETWORK_CMD -M $MACHINE_SUBTYPE $ROOTFS_OPTIONS -no-reboot $QEMU_UI_OPTIONS"
@@ -521,7 +521,7 @@ if [ "$MACHINE" = "qemuppc" ]; then else QEMU_NETWORK_CMD="-net nic,model=pcnet $QEMU_TAP_CMD" fi - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then KERNCMDLINE="root=$DROOT rw console=ttyS0 console=tty $KERNEL_NETWORK_CMD mem=$QEMU_MEMORY" QEMUOPTIONS="$QEMU_NETWORK_CMD -cpu $CPU_SUBTYPE -M $MACHINE_SUBTYPE $ROOTFS_OPTIONS -no-reboot $QEMU_UI_OPTIONS" fi
@@ -540,7 +540,7 @@ if [ "$MACHINE" = "qemush4" ]; then QEMU=qemu-system-sh4 MACHINE_SUBTYPE=r2d QEMU_UI_OPTIONS="$QEMU_UI_OPTIONS" - if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" ]; then + if [ "${FSTYPE:0:3}" = "ext" -o "$FSTYPE" = "btrfs" -o "$FSTYPE" = "wic" ]; then #KERNCMDLINE="root=/dev/hda console=ttyS0 console=tty0 $KERNEL_NETWORK_CMD mem=$QEMU_MEMORY" KERNCMDLINE="root=/dev/hda rw console=ttySC1 noiotrap earlyprintk=sh-sci.1 console=tty $KERNEL_NETWORK_CMD mem=$QEMU_MEMORY" QEMUOPTIONS="$QEMU_NETWORK_CMD -M $MACHINE_SUBTYPE -hda $ROOTFS -no-reboot $QEMU_UI_OPTIONS -monitor null -serial vc -serial stdio"
@@ -726,7 +726,7 @@ echo "Running $QEMU..." if [ "$FSTYPE" = "vmdk" -o "$FSTYPE" = "hddimg" -o "$FSTYPE" = "hdddirect" ]; then echo $QEMUBIN $VM $QEMUOPTIONS $SERIALOPTS -no-reboot $SCRIPT_QEMU_OPT $SCRIPT_QEMU_EXTRA_OPT LD_PRELOAD="$GL_LD_PRELOAD" $QEMUBIN $VM $QEMUOPTIONS $SERIALOPTS -no-reboot $SCRIPT_QEMU_OPT $SCRIPT_QEMU_EXTRA_OPT -elif [ "$FSTYPE" = "iso" ]; then +elif [ "$FSTYPE" = "iso" -o "$FSTYPE" = "wic" ]; then echo $QEMUBIN $QEMUOPTIONS $SERIALOPTS -no-reboot $SCRIPT_QEMU_OPT $SCRIPT_QEMU_EXTRA_OPT LD_PRELOAD="$GL_LD_PRELOAD" $QEMUBIN $QEMUOPTIONS $SERIALOPTS -no-reboot $SCRIPT_QEMU_OPT $SCRIPT_QEMU_EXTRA_OPT else
Signature · 1 line
2.1.4
PATCH 2/15 runqemu: don't use virtio for wic images
In reply to Ed Bartoshqemu hands with 'waiting for root device' message during the boot of wic images if '-drive if=virtio' option is used.
Removed if=virtio from qemu command line for wic images. This should fix above mentioned boot issue.
--- scripts/runqemu-internal | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/runqemu-internal +2 -1
@@ -265,7 +265,8 @@ else QEMU_NETWORK_CMD="-net nic,model=virtio $QEMU_TAP_CMD" fi DROOT="/dev/vda" - ROOTFS_OPTIONS="-drive file=$ROOTFS,if=virtio,format=raw" + ROOTFS_OPTIONS="-drive file=$ROOTFS,format=raw" + [ "$FSTYPE" != "wic" ] && ROOTFS_OPTIONS="$ROOTFS_OPTIONS,if=virtio" KERNCMDLINE="mem=$QEMU_MEMORY" QEMU_UI_OPTIONS="-show-cursor -usb -usbdevice tablet"
Signature · 1 line
2.1.4
Re: [wic][PATCH 02/15] runqemu: don't use virtio for wic images
Quoted 5 lines · Ed Bartosh
qemu hands with 'waiting for root device' message during the boot of wic images if '-drive if=virtio' option is used.
Removed if=virtio from qemu command line for wic images. This should fix above mentioned boot issue.
Does this not change the command for other image types too? What side effects might that have?
Cheers,
Richard
Re: [wic][PATCH 02/15] runqemu: don't use virtio for wic images
Quoted 8 lines · Richard Purdie
Quoted 5 lines · Ed Bartosh
qemu hands with 'waiting for root device' message during the boot of wic images if '-drive if=virtio' option is used.
Removed if=virtio from qemu command line for wic images. This should fix above mentioned boot issue.
Does this not change the command for other image types too?
No, it doesn't. This patch changes qemu command line only for wic images.
--
Regards,
Ed
Re: [wic][PATCH 02/15] runqemu: don't use virtio for wic images
Quoted 6 lines · Ed Bartosh
Quoted 4 lines
Quoted 2 lines
Removed if=virtio from qemu command line for wic images. This should fix above mentioned boot issue.
Does this not change the command for other image types too?
No, it doesn't. This patch changes qemu command line only for wic images.
Any idea why? virtio is better, surely?
Ross
Re: [wic][PATCH 02/15] runqemu: don't use virtio for wic images
In reply to Ed BartoshQuoted 11 lines · Ed Bartosh
Quoted 8 lines · Richard Purdie
Quoted 5 lines · Ed Bartosh
qemu hands with 'waiting for root device' message during the boot of wic images if '-drive if=virtio' option is used.
Removed if=virtio from qemu command line for wic images. This should fix above mentioned boot issue.
Does this not change the command for other image types too?
No, it doesn't. This patch changes qemu command line only for wic images.
Sorry, somehow I didn't see the second line in that diff which adds it back.
I do worry about creating too many codepaths in this code though. Any idea why virtio doesn't work?
Cheers,
Richard
Re: [wic][PATCH 02/15] runqemu: don't use virtio for wic images
Quoted 10 lines · Richard Purdie
Quoted 9 lines · Ed Bartosh
Quoted 8 lines · Richard Purdie
Quoted 5 lines · Ed Bartosh
qemu hands with 'waiting for root device' message during the boot of wic images if '-drive if=virtio' option is used.
Removed if=virtio from qemu command line for wic images. This should fix above mentioned boot issue.
Does this not change the command for other image types too?
Quoted 8 lines
Quoted 2 lines
No, it doesn't. This patch changes qemu command line only for wic images.
Sorry, somehow I didn't see the second line in that diff which adds it back.
I do worry about creating too many codepaths in this code though. Any idea why virtio doesn't work?
I'll look at it and update patchset. Thanks for review!
--
Regards,
Ed
PATCH 3/15 runqemu: don't set KERNEL for wic images
In reply to Ed BartoshWic images should be boot as is, without pointing qemu to the kernel binary. Current code doesn't use kernel, but sets KERNEL variable and shows kernel path in the console output. This can confuse users.
Changed runqemu and runqemu-internal code to avoid setting KERNEL variable and show kernel path.
--- scripts/runqemu | 2 +- scripts/runqemu-internal | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
scripts/runqemu +1 -1
@@ -454,7 +454,7 @@ if [ -e "$ROOTFS" -a -z "$FSTYPE" ]; then fi fi -if [ -z "$KERNEL" -a "x$FSTYPE" != "xvmdk" -a "x$FSTYPE" != "xhddimg" -a "x$FSTYPE" != "xhdddirect" ]; then +if [ -z "$KERNEL" -a "x$FSTYPE" != "xvmdk" -a "x$FSTYPE" != "xhddimg" -a "x$FSTYPE" != "xhdddirect" -a "x$FSTYPE" != "xwic" ]; then setup_path_vars 1 eval kernel_file=\$${machine2}_DEFAULT_KERNEL KERNEL=$DEPLOY_DIR_IMAGE/$kernel_file
scripts/runqemu-internal +1 -1
@@ -302,7 +302,7 @@ case "$MACHINE" in ;; esac -if [ ! -f "$KERNEL" -a "x$FSTYPE" != "xvmdk" -a "x$FSTYPE" != "xhddimg" -a "x$FSTYPE" != "xhdddirect" ]; then +if [ ! -f "$KERNEL" -a "x$FSTYPE" != "xvmdk" -a "x$FSTYPE" != "xhddimg" -a "x$FSTYPE" != "xhdddirect" -a "x$FSTYPE" != "xwic" ]; then echo "Error: Kernel image file $KERNEL doesn't exist" cleanup return 1
Signature · 1 line
2.1.4
PATCH 4/15 runqemu: support path/to/<image>-<machine>.wic
In reply to Ed BartoshSupported providing wic image path to runqemu:
runquemu path/to/<image>-<machine>.wic
[YOCTO #8691]
--- scripts/runqemu | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
scripts/runqemu +6 -4
@@ -47,6 +47,7 @@ usage() { echo " $MYNAME qemux86 qemuparams=\"-m 256\"" echo " $MYNAME qemux86 bootparams=\"psplash=false\"" echo " $MYNAME path/to/<image>-<machine>.vmdk" + echo " $MYNAME path/to/<image>-<machine>.wic" exit 1 }
@@ -88,7 +89,7 @@ process_filename() { [ -z "$KERNEL" ] && KERNEL=$filename || \ error "conflicting KERNEL args [$KERNEL] and [$filename]" ;; - /ext[234]/|/jffs2/|/btrfs/) + /ext[234]/|/jffs2/|/btrfs/|/wic/) # A file ending in a supportted fs type is a rootfs image if [ -z "$FSTYPE" -o "$FSTYPE" = "$EXT" ]; then FSTYPE=$EXT
@@ -235,7 +236,7 @@ elif [ ! -w /dev/net/tun ] ; then fi # Report errors for missing combinations of options -if [ -z "$MACHINE" -a -z "$KERNEL" -a -z "$VM" ]; then +if [ -z "$MACHINE" -a -z "$KERNEL" -a -z "$VM" -a "$FSTYPE" != "wic" ]; then error "you must specify at least a MACHINE, VM, or KERNEL argument" fi if [ "$FSTYPE" = "nfs" -a -z "$ROOTFS" ]; then
@@ -243,8 +244,9 @@ if [ "$FSTYPE" = "nfs" -a -z "$ROOTFS" ]; then fi if [ -z "$MACHINE" ]; then - if [ "x$FSTYPE" = "xvmdk" ] || [ "x$FSTYPE" = "xhddimg" ] || [ "x$FSTYPE" = "xhdddirect" ]; then - MACHINE=`basename $VM | sed -n 's/.*\(qemux86-64\|qemux86\|qemuarm64\|qemuarm\|qemumips64\|qemumips\|qemuppc\|qemush4\).*/\1/p'` + if [ "x$FSTYPE" = "xvmdk" ] || [ "x$FSTYPE" = "xhddimg" ] || [ "x$FSTYPE" = "xhdddirect" ] || [ "x$FSTYPE" = "xwic" ]; then + [ "x$FSTYPE" = "xwic" ] && filename=$ROOTFS || filename=$VM + MACHINE=`basename $filename | sed -n 's/.*\(qemux86-64\|qemux86\|qemuarm64\|qemuarm\|qemumips64\|qemumips\|qemuppc\|qemush4\).*/\1/p'` if [ -z "$MACHINE" ]; then error "Unable to set MACHINE from image filename [$VM]" fi
Signature · 1 line
2.1.4
PATCH 5/15 oe-selftest: remove unused parameter
In reply to Ed BartoshFrom: Ed Bartosh <[email protected]>
Removed unused parameter 'test' from runqemu function.
--- meta/lib/oeqa/selftest/devtool.py | 2 +- meta/lib/oeqa/selftest/imagefeatures.py | 6 +++--- meta/lib/oeqa/utils/commands.py | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-)
meta/lib/oeqa/selftest/devtool.py +1 -1
@@ -987,7 +987,7 @@ class DevtoolTests(DevtoolBase): result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe) self.assertIn(' %s' % testfile, result.output) # Boot the image - with runqemu(testimage, self) as qemu: + with runqemu(testimage) as qemu: # Now really test deploy-target result = runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, qemu.ip)) # Run a test command to see if it was installed properly
meta/lib/oeqa/selftest/imagefeatures.py +3 -3
@@ -31,7 +31,7 @@ class ImageFeatures(oeSelfTest): # Build a core-image-minimal bitbake('core-image-minimal') - with runqemu("core-image-minimal", self) as qemu: + with runqemu("core-image-minimal") as qemu: # Attempt to ssh with each user into qemu with empty password for user in [self.root_user, self.test_user]: ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
@@ -58,7 +58,7 @@ class ImageFeatures(oeSelfTest): # Build a core-image-minimal bitbake('core-image-minimal') - with runqemu("core-image-minimal", self) as qemu: + with runqemu("core-image-minimal") as qemu: # Attempt to ssh with each user into qemu with empty password for user in [self.root_user, self.test_user]: ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
@@ -112,7 +112,7 @@ class ImageFeatures(oeSelfTest): self.fail('No rpm package found in image') # Now do a couple of runtime tests - with runqemu("core-image-minimal", self) as qemu: + with runqemu("core-image-minimal") as qemu: command = "rpm --version" status, output = qemu.run(command) self.assertEqual(0, status, 'Failed to run command "%s": %s' % (command, output))
meta/lib/oeqa/utils/commands.py +1 -1
@@ -177,7 +177,7 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec= @contextlib.contextmanager -def runqemu(pn, test): +def runqemu(pn): import bb.tinfoil import bb.build
Signature · 1 line
2.1.4
PATCH 6/15 qemurunner: add parameter to method 'start'
In reply to Ed BartoshQemuRunner requires pair of ip addresses provided through kernel commandline for method 'start' to work. These ip addresses are used to connect to the image using ssh and run tests there. However, this functionality should not be mandatory as testing doesn't always require ssh connection. Some tests can be run using serial console.
Added new parameter 'get_ip' to QemuRunner.start to make it possible to skip getting pair of ip addresses from kernel command line. This should allow oe-selftest to test images without modifying kernel command line.
[YOCTO #8498]
--- meta/lib/oeqa/utils/qemurunner.py | 45 ++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 22 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +23 -22
@@ -91,7 +91,7 @@ class QemuRunner: self._dump_host() raise SystemExit - def start(self, qemuparams = None): + def start(self, qemuparams = None, get_ip = True): if self.display: os.environ["DISPLAY"] = self.display # Set this flag so that Qemu doesn't do any grabs as SDL grabs
@@ -178,27 +178,28 @@ class QemuRunner: if self.is_alive(): logger.info("qemu started - qemu procces pid is %s" % self.qemupid) - cmdline = '' - with open('/proc/%s/cmdline' % self.qemupid) as p: - cmdline = p.read() - # It is needed to sanitize the data received - # because is possible to have control characters - cmdline = re_control_char.sub('', cmdline) - try: - ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) - if not ips or len(ips) != 3: - raise ValueError - else: - self.ip = ips[0] - self.server_ip = ips[1] - except IndexError, ValueError: - logger.info("Couldn't get ip from qemu process arguments! Here is the qemu command line used:\n%s\nand output from runqemu:\n%s" % (cmdline, self.getOutput(output))) - self._dump_host() - self.stop() - return False - logger.info("qemu cmdline used:\n{}".format(cmdline)) - logger.info("Target IP: %s" % self.ip) - logger.info("Server IP: %s" % self.server_ip) + if get_ip: + cmdline = '' + with open('/proc/%s/cmdline' % self.qemupid) as p: + cmdline = p.read() + # It is needed to sanitize the data received + # because is possible to have control characters + cmdline = re_control_char.sub('', cmdline) + try: + ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) + if not ips or len(ips) != 3: + raise ValueError + else: + self.ip = ips[0] + self.server_ip = ips[1] + except IndexError, ValueError: + logger.info("Couldn't get ip from qemu process arguments! Here is the qemu command line used:\n%s\nand output from runqemu:\n%s" % (cmdline, self.getOutput(output))) + self._dump_host() + self.stop() + return False + logger.info("qemu cmdline used:\n{}".format(cmdline)) + logger.info("Target IP: %s" % self.ip) + logger.info("Server IP: %s" % self.server_ip) self.thread = LoggingThread(self.log, threadsock, logger) self.thread.start()
Signature · 1 line
2.1.4
PATCH 7/15 oeqa/targetcontrol: make ssh control optional
In reply to Ed BartoshAdded new parameter 'ssh' to targetcontrol 'start' method to be able to test images without running ssh server.
[YOCTO #8498]
--- meta/lib/oeqa/targetcontrol.py | 18 ++++++++++-------- meta/lib/oeqa/utils/commands.py | 4 ++-- 2 files changed, 12 insertions(+), 10 deletions(-)
meta/lib/oeqa/targetcontrol.py +10 -8
@@ -68,7 +68,7 @@ class BaseTarget(object): bb.note("SSH log file: %s" % self.sshlog) @abstractmethod - def start(self, params=None): + def start(self, params=None, ssh=True): pass @abstractmethod
@@ -176,11 +176,12 @@ class QemuTarget(BaseTarget): bb.note("Qemu log file: %s" % self.qemulog) super(QemuTarget, self).deploy() - def start(self, params=None): - if self.runner.start(params): - self.ip = self.runner.ip - self.server_ip = self.runner.server_ip - self.connection = SSHControl(ip=self.ip, logfile=self.sshlog) + def start(self, params=None, ssh=True): + if self.runner.start(params, get_ip=ssh): + if ssh: + self.ip = self.runner.ip + self.server_ip = self.runner.server_ip + self.connection = SSHControl(ip=self.ip, logfile=self.sshlog) else: self.stop() if os.path.exists(self.qemulog):
@@ -231,8 +232,9 @@ class SimpleRemoteTarget(BaseTarget): def deploy(self): super(SimpleRemoteTarget, self).deploy() - def start(self, params=None): - self.connection = SSHControl(self.ip, logfile=self.sshlog, port=self.port) + def start(self, params=None, ssh=True): + if ssh: + self.connection = SSHControl(self.ip, logfile=self.sshlog, port=self.port) def stop(self): self.connection = None
meta/lib/oeqa/utils/commands.py +2 -2
@@ -177,7 +177,7 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec= @contextlib.contextmanager -def runqemu(pn): +def runqemu(pn, ssh=True): import bb.tinfoil import bb.build
@@ -211,7 +211,7 @@ def runqemu(pn): try: qemu.deploy() try: - qemu.start() + qemu.start(ssh=ssh) except bb.build.FuncFailed: raise Exception('Failed to start QEMU - see the logs in %s' % logdir)
Signature · 1 line
2.1.4
PATCH 8/15 oeqa/targetcontrol: support wic image type
In reply to Ed BartoshAdded 'wic' to the list image types supported by targetcontrol.
This is a preparation for booting and testing wic images with oe-selftest.
[YOCTO #8498]
--- meta/lib/oeqa/targetcontrol.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/targetcontrol.py +1 -1
@@ -113,7 +113,7 @@ class BaseTarget(object): class QemuTarget(BaseTarget): - supported_image_fstypes = ['ext3', 'ext4', 'cpio.gz'] + supported_image_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic'] def __init__(self, d):
Signature · 1 line
2.1.4
PATCH 9/15 wic-image-minimal: change IMAGE_FSTYPES
In reply to Ed BartoshReplaced wic.bz2 -> wic in IMAGE_FSTYPES.
wic-image-minimal is going to be booted in qemu and tested. As runqemu doesn't support compressed images this recipe needs to produce raw wic image.
--- meta-selftest/recipes-test/images/wic-image-minimal.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta-selftest/recipes-test/images/wic-image-minimal.bb +1 -1
@@ -4,7 +4,7 @@ SRC_URI = "file://${FILE_DIRNAME}/${BPN}.wks" IMAGE_INSTALL = "packagegroup-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP}" -IMAGE_FSTYPES = "wic.bz2" +IMAGE_FSTYPES = "wic" RM_OLD_IMAGE = "1" DEPENDS = "syslinux syslinux-native parted-native dosfstools-native mtools-native"
Signature · 1 line
2.1.4
PATCH 10/15 wic-image-minimal: updated .wks to boot by qemu
In reply to Ed BartoshWith below list of changes wic-image-minimal can be boot by qemu:
- Changed device name sda -> hda.
- Added root=/dev/hda2 to the kernel command line.
- Changed mount point for core partition to /mnt to have it mounted on boot time.
--- meta-selftest/recipes-test/images/wic-image-minimal.wks | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
meta-selftest/recipes-test/images/wic-image-minimal.wks +5 -5
@@ -2,9 +2,9 @@ # long-description: This image contains boot partition and 3 rootfs partitions # created from core-image-minimal and wic-image-minimal image recipes. -part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 -part /core --source rootfs --rootfs-dir=core-image-minimal --ondisk sda --fstype=ext4 --label core --align 1024 -part /backup --source rootfs --rootfs-dir=wic-image-minimal --ondisk sda --fstype=ext4 --label backup --align 1024 +part /boot --source bootimg-pcbios --ondisk hda --label boot --active --align 1024 +part / --source rootfs --ondisk hda --fstype=ext4 --label platform --align 1024 +part /mnt --source rootfs --rootfs-dir=core-image-minimal --ondisk hda --fstype=ext4 --label core --align 1024 +part backup --source rootfs --rootfs-dir=wic-image-minimal --ondisk hda --fstype=ext4 --label backup --align 1024 -bootloader --timeout=0 --append="rootwait console=tty0" +bootloader --timeout=0 --append="rootwait console=tty0 root=/dev/hda2"
Signature · 1 line
2.1.4
PATCH 11/15 oe-selftest: add new wic testcase
In reply to Ed BartoshAdded test_qemu testcase to boot wic-image-minimal and test that 2 partitions mentioned in .wks are mounted.
[YOCTO #8499]
--- meta/lib/oeqa/selftest/wic.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/wic.py +11 -1
@@ -30,7 +30,7 @@ from glob import glob from shutil import rmtree from oeqa.selftest.base import oeSelfTest -from oeqa.utils.commands import runCmd, bitbake, get_bb_var +from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu from oeqa.utils.decorators import testcase
@@ -264,3 +264,13 @@ class Wic(oeSelfTest): self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image))) + + def test_qemu(self): + """Test wic-image-minimal under qemu""" + self.assertEqual(0, bitbake('wic-image-minimal').status) + + with runqemu('wic-image-minimal', ssh=False) as qemu: + command = "mount |grep '^/dev/' | cut -f1,3 -d ' '" + status, output = qemu.run_serial(command) + self.assertEqual(1, status, 'Failed to run command "%s": %s' % (command, output)) + self.assertEqual(output, '/dev/root /\r\n/dev/hda3 /mnt')
Signature · 1 line
2.1.4
PATCH 12/15 wic: fix processing of --use-uuid
In reply to Ed BartoshProcessing of this option was lost during recent change of wks parsting. It was discovered during the work on booting wic images under qemu. Now, when -use-uuid is fixed it's possible to specify root partition by partition uuid. This will be done in the following commit.
--- scripts/lib/wic/partition.py | 3 +++ 1 file changed, 3 insertions(+)
scripts/lib/wic/partition.py +3 -0
@@ -57,7 +57,10 @@ class Partition(object): self.size = args.size self.source = args.source self.sourceparams = args.sourceparams + self.use_uuid = args.use_uuid self.uuid = args.uuid + if args.use_uuid and not self.uuid: + self.uuid = str(uuid.uuid4()) self.lineno = lineno self.source_file = ""
Signature · 1 line
2.1.4
PATCH 13/15 wic-image-minimal: use uuid for root partition
In reply to Ed BartoshUsed --uuid option to generate PARTUUID for root partion and gpt partition type. As this image is used in runtime testing under qemu it makes sense to ensure that this way of specifying root partition works.
--- meta-selftest/recipes-test/images/wic-image-minimal.wks | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta-selftest/recipes-test/images/wic-image-minimal.wks +2 -2
@@ -3,8 +3,8 @@ # created from core-image-minimal and wic-image-minimal image recipes. part /boot --source bootimg-pcbios --ondisk hda --label boot --active --align 1024 -part / --source rootfs --ondisk hda --fstype=ext4 --label platform --align 1024 +part / --source rootfs --ondisk hda --fstype=ext4 --label platform --align 1024 --use-uuid part /mnt --source rootfs --rootfs-dir=core-image-minimal --ondisk hda --fstype=ext4 --label core --align 1024 part backup --source rootfs --rootfs-dir=wic-image-minimal --ondisk hda --fstype=ext4 --label backup --align 1024 -bootloader --timeout=0 --append="rootwait console=tty0 root=/dev/hda2" +bootloader --ptable gpt --timeout=0 --append="rootwait console=tty0"
Signature · 1 line
2.1.4
PATCH 14/15 oe-selftest: pylinted wic tests
In reply to Ed BartoshFixed the following pylint warnings:
C: 45, 0: Line too long (91/80) (line-too-long) C:105, 0: Wrong continued indentation.
ignore_status=True).status)
^ | (bad-continuation)
W: 83, 8: Redefining built-in 'vars' (redefined-builtin) W:175, 8: Redefining built-in 'vars' (redefined-builtin)
W: 27, 0: Unused import sys (unused-import)
--- meta/lib/oeqa/selftest/wic.py | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
meta/lib/oeqa/selftest/wic.py +9 -9
@@ -24,7 +24,6 @@ """Test cases for wic.""" import os -import sys from glob import glob from shutil import rmtree
@@ -42,7 +41,8 @@ class Wic(oeSelfTest): def setUpLocal(self): """This code is executed before each test method.""" - self.write_config('IMAGE_FSTYPES += " hddimg"\nMACHINE_FEATURES_append = " efi"\n') + self.write_config('IMAGE_FSTYPES += " hddimg"\n' + 'MACHINE_FEATURES_append = " efi"\n') # Do this here instead of in setUpClass as the base setUp does some # clean up which can result in the native tools built earlier in
@@ -80,14 +80,14 @@ class Wic(oeSelfTest): @testcase(1212) def test_build_artifacts(self): """Test wic create directdisk providing all artifacts.""" - vars = dict((var.lower(), get_bb_var(var, 'core-image-minimal')) \ + bbvars = dict((var.lower(), get_bb_var(var, 'core-image-minimal')) \ for var in ('STAGING_DATADIR', 'DEPLOY_DIR_IMAGE', 'STAGING_DIR_NATIVE', 'IMAGE_ROOTFS')) status = runCmd("wic create directdisk " "-b %(staging_datadir)s " "-k %(deploy_dir_image)s " "-n %(staging_dir_native)s " - "-r %(image_rootfs)s" % vars).status + "-r %(image_rootfs)s" % bbvars).status self.assertEqual(0, status) self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
@@ -102,7 +102,7 @@ class Wic(oeSelfTest): def test_unsupported_subcommand(self): """Test unsupported subcommand""" self.assertEqual(1, runCmd('wic unsupported', - ignore_status=True).status) + ignore_status=True).status) @testcase(1214) def test_no_command(self):
@@ -172,20 +172,20 @@ class Wic(oeSelfTest): @testcase(1269) def test_rootfs_artifacts(self): """Test usage of rootfs plugin with rootfs paths""" - vars = dict((var.lower(), get_bb_var(var, 'core-image-minimal')) \ + bbvars = dict((var.lower(), get_bb_var(var, 'core-image-minimal')) \ for var in ('STAGING_DATADIR', 'DEPLOY_DIR_IMAGE', 'STAGING_DIR_NATIVE', 'IMAGE_ROOTFS')) - vars['wks'] = "directdisk-multi-rootfs" + bbvars['wks'] = "directdisk-multi-rootfs" status = runCmd("wic create %(wks)s " "-b %(staging_datadir)s " "-k %(deploy_dir_image)s " "-n %(staging_dir_native)s " "--rootfs-dir rootfs1=%(image_rootfs)s " "--rootfs-dir rootfs2=%(image_rootfs)s" \ - % vars).status + % bbvars).status self.assertEqual(0, status) self.assertEqual(1, len(glob(self.resultdir + \ - "%(wks)s-*.direct" % vars))) + "%(wks)s-*.direct" % bbvars))) @testcase(1346) def test_iso_image(self):
Signature · 1 line
2.1.4
PATCH 15/15 oe-selftest: don't use specific tasks
In reply to Ed BartoshImage should be build usual way, i.e. using bitbake <image>. Specifying do_mage_complete and do_rootfs_wicenv tasks should not be needed anymore as those tasks should be run by bitbake for every image.
Removed specifying do_mage_complete and do_rootfs_wicenv tasks from bitbake calls.
--- meta/lib/oeqa/selftest/wic.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/selftest/wic.py +1 -1
@@ -50,7 +50,7 @@ class Wic(oeSelfTest): if not Wic.image_is_ready: bitbake('syslinux syslinux-native parted-native gptfdisk-native ' 'dosfstools-native mtools-native') - bitbake('core-image-minimal:do_image_complete core-image-minimal:do_rootfs_wicenv') + bitbake('core-image-minimal') Wic.image_is_ready = True rmtree(self.resultdir, ignore_errors=True)
Signature · 1 line
2.1.4