Properly fix oe-selftest QEMU usage + related fixes
cover Properly fix oe-selftest QEMU usage + related fixes
The tests in oeqa/selftest/imagefeatures have had a long list of issues that we've worked around; this patchset implements the ability to run QEMU from oe-selftest tests properly and attempts to fix the remaining known issues with the tests. At the same time we can make the devtool test that uses QEMU use the new runqemu() logic, so do that as well.
When we did do the proper runqemu() implementation however, I uncovered a number of other problems to do with error handling, process termination and logging. This patchset should address those as well.
Thanks to Richard for the initial set of fixes which I built upon here.
The following changes since commit 03d01393d14b7b20dcb40ff89b1628883fd3b545:
toaster.bbclass: Fix ValueError (2015-07-27 12:26:57 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/oe-selftest-qemu-fixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/oe-selftest-qemu-fixes
Paul Eggleton (9):
oe-selftest: add scripts/lib and bitbake/lib to path oeqa/targetcontrol: create test directory before copying rootfs image oeqa/utils/qemurunner: fix logging
oeqa/utils/qemurunner: avoid blocking on stty when running under
oe-selftest
oeqa/utils/qemurunner: fix error handling if runqemu exits with an
error
oeqa/targetcontrol: write QemuRunner log output to a file oeqa/selftest/imagefeatures: fix RPM4 test oeqa/selftest/imagefeatures: remove gummiboot tests oeqa/selftest/devtool: use new runqemu function
Richard Purdie (2):
oeqa/selftest/imagefeatures: Use QemuTarget code oeqa/selftest/imagefeatures: fix SSH without password tests
meta/lib/oeqa/selftest/devtool.py | 72 +++++----- meta/lib/oeqa/selftest/imagefeatures.py | 247 +++++++++----------------------- meta/lib/oeqa/targetcontrol.py | 12 ++ meta/lib/oeqa/utils/commands.py | 49 +++++++ meta/lib/oeqa/utils/qemurunner.py | 70 +++++---- scripts/oe-selftest | 5 +- 6 files changed, 207 insertions(+), 248 deletions(-)
Signature · 1 line
2.1.0
PATCH 1/11 oe-selftest: add scripts/lib and bitbake/lib to path
In particular, this allows us to use code from bitbake's bb module (such as tinfoil).
--- scripts/oe-selftest | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/oe-selftest +4 -1
@@ -31,7 +31,10 @@ import unittest import logging import argparse -sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'meta/lib'))) +sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/lib') +import scriptpath +scriptpath.add_bitbake_lib_path() +scriptpath.add_oe_lib_path() import oeqa.selftest import oeqa.utils.ftools as ftools
Signature · 1 line
2.1.0
PATCH 2/11 oeqa/targetcontrol: create test directory before copying rootfs image
In reply to Paul EggletonThe test directory might not exist at this point so just go ahead and create it.
--- meta/lib/oeqa/targetcontrol.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/oeqa/targetcontrol.py +1 -0
@@ -143,6 +143,7 @@ class QemuTarget(BaseTarget): def deploy(self): try: + bb.utils.mkdirhier(self.testdir) shutil.copyfile(self.origrootfs, self.rootfs) except Exception as e: bb.fatal("Error copying rootfs: %s" % e)
Signature · 1 line
2.1.0
PATCH 3/11 oeqa/utils/qemurunner: fix logging
In reply to Paul EggletonOE-Core commit 519e381278d40bdac79add340e4c0460a9f97e17 unfortunately broke logging in two different ways:
1) it prevented logging to the task log from working within bitbake -c testimage. This is due to the logger object being set up too early which interferes with BitBake's own logging. If we prefix the name with "BitBake." everything works (and we don't need to set the logging level).
2) Additionally because it called the log functions on the logging module and not the logger object it set up, this caused the oe-selftest logging to start printing everything from that point forward.
Fix these two issues and return us to the desired behaviour for do_testimage.
--- meta/lib/oeqa/utils/qemurunner.py | 49 +++++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 25 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +24 -25
@@ -14,8 +14,7 @@ import socket import select import logging -logger = logging.getLogger("QemuRunner") -logger.setLevel(logging.DEBUG - 2) +logger = logging.getLogger("BitBake.QemuRunner") class QemuRunner:
@@ -51,11 +50,11 @@ class QemuRunner: self.server_socket.bind(("127.0.0.1",0)) self.server_socket.listen(2) self.serverport = self.server_socket.getsockname()[1] - logging.info("Created listening socket for qemu serial console on: 127.0.0.1:%s" % self.serverport) + logger.info("Created listening socket for qemu serial console on: 127.0.0.1:%s" % self.serverport) return True except socket.error, msg: self.server_socket.close() - logging.error("Failed to create listening socket: %s" % msg[1]) + logger.error("Failed to create listening socket: %s" % msg[1]) return False
@@ -68,18 +67,18 @@ class QemuRunner: if self.display: os.environ["DISPLAY"] = self.display else: - logging.error("To start qemu I need a X desktop, please set DISPLAY correctly (e.g. DISPLAY=:1)") + logger.error("To start qemu I need a X desktop, please set DISPLAY correctly (e.g. DISPLAY=:1)") return False if not os.path.exists(self.rootfs): - logging.error("Invalid rootfs %s" % self.rootfs) + logger.error("Invalid rootfs %s" % self.rootfs) return False if not os.path.exists(self.tmpdir): - logging.error("Invalid TMPDIR path %s" % self.tmpdir) + logger.error("Invalid TMPDIR path %s" % self.tmpdir) return False else: os.environ["OE_TMPDIR"] = self.tmpdir if not os.path.exists(self.deploy_dir_image): - logging.error("Invalid DEPLOY_DIR_IMAGE path %s" % self.deploy_dir_image) + logger.error("Invalid DEPLOY_DIR_IMAGE path %s" % self.deploy_dir_image) return False else: os.environ["DEPLOY_DIR_IMAGE"] = self.deploy_dir_image
@@ -97,28 +96,28 @@ class QemuRunner: launch_cmd = 'runqemu %s %s %s' % (self.machine, self.rootfs, self.qemuparams) self.runqemu = subprocess.Popen(launch_cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,preexec_fn=os.setpgrp) - logging.info("runqemu started, pid is %s" % self.runqemu.pid) - logging.info("waiting at most %s seconds for qemu pid" % self.runqemutime) + logger.info("runqemu started, pid is %s" % self.runqemu.pid) + logger.info("waiting at most %s seconds for qemu pid" % self.runqemutime) endtime = time.time() + self.runqemutime while not self.is_alive() and time.time() < endtime: time.sleep(1) if self.is_alive(): - logging.info("qemu started - qemu procces pid is %s" % self.qemupid) + logger.info("qemu started - qemu procces pid is %s" % self.qemupid) cmdline = '' with open('/proc/%s/cmdline' % self.qemupid) as p: cmdline = p.read() ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) if not ips or len(ips) != 3: - logging.info("Couldn't get ip from qemu process arguments! Here is the qemu command line used: %s" % cmdline) + logger.info("Couldn't get ip from qemu process arguments! Here is the qemu command line used: %s" % cmdline) self.stop() return False else: self.ip = ips[0] self.server_ip = ips[1] - logging.info("Target IP: %s" % self.ip) - logging.info("Server IP: %s" % self.server_ip) - logging.info("Waiting at most %d seconds for login banner" % self.boottime) + logger.info("Target IP: %s" % self.ip) + logger.info("Server IP: %s" % self.server_ip) + logger.info("Waiting at most %d seconds for login banner" % self.boottime) endtime = time.time() + self.boottime socklist = [self.server_socket] reachedlogin = False
@@ -131,7 +130,7 @@ class QemuRunner: self.qemusock.setblocking(0) socklist.append(self.qemusock) socklist.remove(self.server_socket) - logging.info("Connection from %s:%s" % addr) + logger.info("Connection from %s:%s" % addr) else: data = sock.recv(1024) if data:
@@ -140,24 +139,24 @@ class QemuRunner: if re.search(".* login:", self.bootlog): stopread = True reachedlogin = True - logging.info("Reached login banner") + logger.info("Reached login banner") else: socklist.remove(sock) sock.close() stopread = True if not reachedlogin: - logging.info("Target didn't reached login boot in %d seconds" % self.boottime) + logger.info("Target didn't reached login boot in %d seconds" % self.boottime) lines = "\n".join(self.bootlog.splitlines()[-5:]) - logging.info("Last 5 lines of text:\n%s" % lines) - logging.info("Check full boot log: %s" % self.logfile) + logger.info("Last 5 lines of text:\n%s" % lines) + logger.info("Check full boot log: %s" % self.logfile) self.stop() return False else: - logging.info("Qemu pid didn't appeared in %s seconds" % self.runqemutime) + logger.info("Qemu pid didn't appeared in %s seconds" % self.runqemutime) output = self.runqemu.stdout self.stop() - logging.info("Output from runqemu:\n%s" % output.read()) + logger.info("Output from runqemu:\n%s" % output.read()) return False return self.is_alive()
@@ -165,13 +164,13 @@ class QemuRunner: def stop(self): if self.runqemu: - logging.info("Sending SIGTERM to runqemu") + logger.info("Sending SIGTERM to runqemu") os.killpg(self.runqemu.pid, signal.SIGTERM) endtime = time.time() + self.runqemutime while self.runqemu.poll() is None and time.time() < endtime: time.sleep(1) if self.runqemu.poll() is None: - logging.info("Sending SIGKILL to runqemu") + logger.info("Sending SIGKILL to runqemu") os.killpg(self.runqemu.pid, signal.SIGKILL) self.runqemu = None if self.server_socket:
@@ -181,7 +180,7 @@ class QemuRunner: self.ip = None def restart(self, qemuparams = None): - logging.info("Restarting qemu process") + logger.info("Restarting qemu process") if self.runqemu.poll() is None: self.stop() if self.start(qemuparams):
Signature · 1 line
2.1.0
PATCH 4/11 oeqa/utils/qemurunner: avoid blocking on stty when running under oe-selftest
In reply to Paul Eggletonrunqemu-internal runs stty to return the terminal to its previous state in case QEMU hasn't done that properly (which it at least used to do when it crashed). For some reason I have yet to determine, stty blocks (on tcsetattr() according to gdb) when run within QemuRunner() under oe-selftest, with the result that we always wait until the timeout and then we kill the script, which adds an extra delay after QEMU is stopped. Naturally you would assume that this is something to do with the nature of the terminal under which it is being run; however no amount of playing around with stdin/stdout/stderr seemed to fix the issue, apart from passing in subprocess.PIPE as stdin which makes stty error out with "stty: standard input: Inappropriate ioctl for device". I was also unable to come up with a reliable test for the terminal which we could use inside runqemu-internal to avoid calling stty. For now, go with the stdin=subprocess.PIPE workaround to at least avoid the delay with minimal ill effect.
--- meta/lib/oeqa/utils/qemurunner.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/qemurunner.py +5 -1
@@ -94,7 +94,11 @@ class QemuRunner: self.qemuparams = self.qemuparams[:-1] + " " + qemuparams + " " + '\"' launch_cmd = 'runqemu %s %s %s' % (self.machine, self.rootfs, self.qemuparams) - self.runqemu = subprocess.Popen(launch_cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,preexec_fn=os.setpgrp) + # FIXME: We pass in stdin=subprocess.PIPE here to work around stty + # blocking at the end of the runqemu script when using this within + # oe-selftest (this makes stty error out immediately). There ought + # to be a proper fix but this will suffice for now. + self.runqemu = subprocess.Popen(launch_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, preexec_fn=os.setpgrp) logger.info("runqemu started, pid is %s" % self.runqemu.pid) logger.info("waiting at most %s seconds for qemu pid" % self.runqemutime)
Signature · 1 line
2.1.0
PATCH 5/11 oeqa/utils/qemurunner: fix error handling if runqemu exits with an error
In reply to Paul Eggleton* Don't wait for QEMU to start if it's never going to (because runqemu exited with an error)
* Don't error out if killing the process fails with "no such process" (we don't care if it's already dead)
--- meta/lib/oeqa/utils/qemurunner.py | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/qemurunner.py +14 -1
@@ -12,6 +12,7 @@ import signal import re import socket import select +import errno import logging logger = logging.getLogger("BitBake.QemuRunner")
@@ -104,6 +105,14 @@ class QemuRunner: logger.info("waiting at most %s seconds for qemu pid" % self.runqemutime) endtime = time.time() + self.runqemutime while not self.is_alive() and time.time() < endtime: + if self.runqemu.poll(): + if self.runqemu.returncode: + # No point waiting any longer + logger.info('runqemu exited with code %d' % self.runqemu.returncode) + output = self.runqemu.stdout + self.stop() + logger.info("Output from runqemu:\n%s" % output.read()) + return False time.sleep(1) if self.is_alive():
@@ -169,7 +178,11 @@ class QemuRunner: if self.runqemu: logger.info("Sending SIGTERM to runqemu") - os.killpg(self.runqemu.pid, signal.SIGTERM) + try: + os.killpg(self.runqemu.pid, signal.SIGTERM) + except OSError as e: + if e.errno != errno.ESRCH: + raise endtime = time.time() + self.runqemutime while self.runqemu.poll() is None and time.time() < endtime: time.sleep(1)
Signature · 1 line
2.1.0
PATCH 6/11 oeqa/targetcontrol: write QemuRunner log output to a file
In reply to Paul EggletonIf we use this outside of testimage we don't have a task log; so let's just explicitly write the log output to a file all the time so it's always there to look at (particularly useful when runqemu exits immediately with an error.)
--- meta/lib/oeqa/targetcontrol.py | 11 +++++++++++ 1 file changed, 11 insertions(+)
meta/lib/oeqa/targetcontrol.py +11 -0
@@ -10,6 +10,7 @@ import subprocess import bb import traceback import sys +import logging from oeqa.utils.sshcontrol import SSHControl from oeqa.utils.qemurunner import QemuRunner from oeqa.utils.qemutinyrunner import QemuTinyRunner
@@ -123,6 +124,16 @@ class QemuTarget(BaseTarget): self.rootfs = os.path.join(self.testdir, d.getVar("IMAGE_LINK_NAME", True) + '-testimage.' + self.image_fstype) self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE", False) + '-' + d.getVar('MACHINE', False) + '.bin') + # Log QemuRunner log output to a file + import oe.path + bb.utils.mkdirhier(self.testdir) + self.qemurunnerlog = os.path.join(self.testdir, 'qemurunner_log.%s' % self.datetime) + logger = logging.getLogger('BitBake.QemuRunner') + loggerhandler = logging.FileHandler(self.qemurunnerlog) + loggerhandler.setFormatter(logging.Formatter("%(levelname)s: %(message)s")) + logger.addHandler(loggerhandler) + oe.path.symlink(os.path.basename(self.qemurunnerlog), os.path.join(self.testdir, 'qemurunner_log'), force=True) + if d.getVar("DISTRO", True) == "poky-tiny": self.runner = QemuTinyRunner(machine=d.getVar("MACHINE", True), rootfs=self.rootfs,
Signature · 1 line
2.1.0
PATCH 7/11 oeqa/selftest/imagefeatures: Use QemuTarget code
In reply to Paul EggletonFrom: Richard Purdie <[email protected]>
Create a runqemu function which uses the QemuTarget() code from oeqa.targetcontrol to setup the QEMU instance, with all of the added robustness that that gives us. To do this, a datastore is needed for the recipe in question (core-image-minimal) so we do the work needed to set this up. We then use this runqemu function within the imagefeatures tests instead of a hand-rolled implementation.
We can then use SSHControl to run the SSH tests rather than rolling our own code to do that as an added bonus.
Fixed and extended by Paul Eggleton <[email protected]>.
Part of the fix for [YOCTO #7994].
--- meta/lib/oeqa/selftest/imagefeatures.py | 96 +++++++-------------------------- meta/lib/oeqa/utils/commands.py | 49 +++++++++++++++++ 2 files changed, 67 insertions(+), 78 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +18 -78
@@ -1,20 +1,18 @@ from oeqa.selftest.base import oeSelfTest -from oeqa.utils.commands import runCmd, bitbake, get_bb_var +from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu from oeqa.utils.decorators import testcase -import pexpect +from oeqa.utils.sshcontrol import SSHControl from os.path import isfile -from os import system, killpg +from os import system import glob -import signal - +import os +import sys +import logging class ImageFeatures(oeSelfTest): test_user = 'tester' root_user = 'root' - prompt = r'qemux86:\S+[$#]\s+' - ssh_cmd = "ssh {} -l {} -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" - get_ip_patt = r'\s+ip=(?P<qemu_ip>(\d+.){3}\d+)::' @testcase(1107) def test_non_root_user_can_connect_via_ssh_without_password(self):
@@ -37,41 +35,12 @@ class ImageFeatures(oeSelfTest): # Build a core-image-minimal bitbake('core-image-minimal') - # Boot qemu image - proc_qemu = pexpect.spawn('runqemu qemux86 nographic') - try: - proc_qemu.expect(self.get_ip_patt, timeout=100) - qemu_ip = proc_qemu.match.group('qemu_ip') - proc_qemu.expect('qemux86 login:', timeout=100) - except Exception as e: - try: - killpg(proc_qemu.pid, signal.SIGTERM) - except: - pass - self.fail('Failed to start qemu: %s' % e) - - # Attempt to ssh with each user into qemu with empty password - for user in [self.root_user, self.test_user]: - proc_ssh = pexpect.spawn(self.ssh_cmd.format(qemu_ip, user)) - index = proc_ssh.expect([self.prompt, pexpect.TIMEOUT, pexpect.EOF]) - if index == 0: - # user successfully logged in with empty password - pass - elif index == 1: - killpg(proc_qemu.pid, signal.SIGTERM) - proc_ssh.terminate() - self.fail('Failed to ssh with {} user into qemu (timeout).'.format(user)) - else: - killpg(proc_qemu.pid, signal.SIGTERM) - proc_ssh.terminate() - self.fail('Failed to ssh with {} user into qemu (eof).'.format(user)) - proc_ssh.terminate() - - # Cleanup - try: - killpg(proc_qemu.pid, signal.SIGTERM) - except: - pass + with runqemu("core-image-minimal", self) as qemu: + # Attempt to ssh with each user into qemu with empty password + for user in [self.root_user, self.test_user]: + ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user) + status, output = ssh.run("true") + self.assertEqual(status, 0, 'ssh to user %s failed with %s' % (user, output)) @testcase(1115) def test_all_users_can_connect_via_ssh_without_password(self):
@@ -82,7 +51,6 @@ class ImageFeatures(oeSelfTest): Author: Ionut Chisanovici <[email protected]> AutomatedBy: Daniel Istrate <[email protected]> """ - features = 'EXTRA_IMAGE_FEATURES += "ssh-server-openssh allow-empty-password"\n' features += 'INHERIT += "extrausers"\n' features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
@@ -93,41 +61,13 @@ class ImageFeatures(oeSelfTest): # Build a core-image-minimal bitbake('core-image-minimal') - # Boot qemu image - proc_qemu = pexpect.spawn('runqemu qemux86 nographic') - try: - proc_qemu.expect(self.get_ip_patt, timeout=100) - qemu_ip = proc_qemu.match.group('qemu_ip') - proc_qemu.expect('qemux86 login:', timeout=100) - except Exception as e: - try: - killpg(proc_qemu.pid, signal.SIGTERM) - except: - pass - self.fail('Failed to start qemu: %s' % e) - - # Attempt to ssh with each user into qemu with empty password - for user in [self.root_user, self.test_user]: - proc_ssh = pexpect.spawn(self.ssh_cmd.format(qemu_ip, user)) - index = proc_ssh.expect([self.prompt, pexpect.TIMEOUT, pexpect.EOF]) - if index == 0: - # user successfully logged in with empty password - pass - elif index == 1: - killpg(proc_qemu.pid, signal.SIGTERM) - proc_ssh.terminate() - self.fail('Failed to ssh with {} user into qemu (timeout).'.format(user)) - else: - killpg(proc_qemu.pid, signal.SIGTERM) - proc_ssh.terminate() - self.fail('Failed to ssh with {} user into qemu (eof).'.format(user)) - proc_ssh.terminate() + with runqemu("core-image-minimal", self) as qemu: + # Attempt to ssh with each user into qemu with empty password + for user in [self.root_user, self.test_user]: + ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user) + status, output = ssh.run("true") + self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output) - # Cleanup - try: - killpg(proc_qemu.pid, signal.SIGTERM) - except: - pass @testcase(1114) def test_rpm_version_4_support_on_image(self):
meta/lib/oeqa/utils/commands.py +49 -0
58 lines not on this page · raw message
Signature · 1 line
2.1.0
PATCH 8/11 oeqa/selftest/imagefeatures: fix SSH without password tests
In reply to Paul EggletonFrom: Richard Purdie <[email protected]>
* We need to set EXTRA_IMAGE_FEATURES outright or existing values will affect the test
* For test case 1107 we need "empty-root-password" to match the behaviour described in the test case
* For test case 1115 we shouldn't be able to connect as root with the features we are setting
Test cases 1107 and 1115 have been updated in Testopia to match.
--- meta/lib/oeqa/selftest/imagefeatures.py | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +9 -4
@@ -25,7 +25,7 @@ class ImageFeatures(oeSelfTest): AutomatedBy: Daniel Istrate <[email protected]> """ - features = 'EXTRA_IMAGE_FEATURES += "ssh-server-openssh empty-root-password"\n' + features = 'EXTRA_IMAGE_FEATURES = "ssh-server-openssh empty-root-password allow-empty-password"\n' features += 'INHERIT += "extrausers"\n' features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
@@ -46,12 +46,14 @@ class ImageFeatures(oeSelfTest): def test_all_users_can_connect_via_ssh_without_password(self): """ Summary: Check if all users can connect via ssh without password - Expected: 1. Connection to the image via ssh using root or tester user without providing a password should be allowed. + Expected: 1. Connection to the image via ssh using root user without providing a password should NOT be allowed. + 2. Connection to the image via ssh using tester user without providing a password should be allowed. Product: oe-core Author: Ionut Chisanovici <[email protected]> AutomatedBy: Daniel Istrate <[email protected]> """ - features = 'EXTRA_IMAGE_FEATURES += "ssh-server-openssh allow-empty-password"\n' + + features = 'EXTRA_IMAGE_FEATURES = "ssh-server-openssh allow-empty-password"\n' features += 'INHERIT += "extrausers"\n' features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
@@ -66,7 +68,10 @@ class ImageFeatures(oeSelfTest): for user in [self.root_user, self.test_user]: ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user) status, output = ssh.run("true") - self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output) + if user == 'root': + self.assertNotEqual(status, 0, 'ssh to user root was allowed when it should not have been') + else: + self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output) @testcase(1114)
Signature · 1 line
2.1.0
PATCH 9/11 oeqa/selftest/imagefeatures: fix RPM4 test
In reply to Paul Eggleton* Use our new runqemu function
* Don't hard-code the RPM4 version
* Double-check the native version is RPM4
* Check that an rpm 4.x package is in the image manifest (this isn't strictly necessary, but "belt-and-braces" and it serves as an example of how to do that)
* Check that the database is working on the target
* Ensure the image actually has openssh in it so we can connect to it
Initial runqemu adaptation by Richard Purdie <[email protected]>
Part of the fix for [YOCTO #7994].
--- meta/lib/oeqa/selftest/imagefeatures.py | 75 +++++++++++++++++++-------------- 1 file changed, 44 insertions(+), 31 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +44 -31
@@ -78,16 +78,17 @@ class ImageFeatures(oeSelfTest): def test_rpm_version_4_support_on_image(self): """ Summary: Check rpm version 4 support on image - Expected: Rpm version must be 4.11.2 + Expected: Rpm version must be 4.x Product: oe-core Author: Ionut Chisanovici <[email protected]> AutomatedBy: Daniel Istrate <[email protected]> """ - rpm_version = '4.11.2' - features = 'IMAGE_INSTALL_append = " rpm"\n' - features += 'PREFERRED_VERSION_rpm = "{}"\n'.format(rpm_version) - features += 'PREFERRED_VERSION_rpm-native = "{}"\n'.format(rpm_version) + features = 'PREFERRED_VERSION_rpm = "4.%"\n' + features += 'PREFERRED_VERSION_rpm-native = "4.%"\n' + # Use openssh in IMAGE_INSTALL instead of ssh-server-openssh in EXTRA_IMAGE_FEATURES as a workaround for bug 8047 + features += 'IMAGE_INSTALL_append = " openssh"\n' + features += 'EXTRA_IMAGE_FEATURES = "empty-root-password allow-empty-password package-management"\n' features += 'RPMROOTFSDEPENDS_remove = "rpmresolve-native:do_populate_sysroot"' # Append 'features' to local.conf
@@ -96,32 +97,44 @@ class ImageFeatures(oeSelfTest): # Build a core-image-minimal bitbake('core-image-minimal') - # Boot qemu image & get rpm version - proc_qemu = pexpect.spawn('runqemu qemux86 nographic') - try: - proc_qemu.expect('qemux86 login:', timeout=100) - proc_qemu.sendline(self.root_user) - proc_qemu.expect(self.prompt) - proc_qemu.sendline('rpm --version') - proc_qemu.expect(self.prompt) - except Exception as e: - try: - killpg(proc_qemu.pid, signal.SIGTERM) - except: - pass - self.fail('Failed to start qemu: %s' % e) - - found_rpm_version = proc_qemu.before - - # Make sure the retrieved rpm version is the expected one - self.assertIn(rpm_version, found_rpm_version, - 'RPM version is not {}, found instead {}.'.format(rpm_version, found_rpm_version)) - - # Cleanup (close qemu) - try: - killpg(proc_qemu.pid, signal.SIGTERM) - except: - pass + # Check the native version of rpm is correct + native_bindir = get_bb_var('STAGING_BINDIR_NATIVE') + result = runCmd(os.path.join(native_bindir, 'rpm') + ' --version') + self.assertIn('version 4.', result.output) + + # Check manifest for the rpm package + deploydir = get_bb_var('DEPLOY_DIR_IMAGE') + imgname = get_bb_var('IMAGE_LINK_NAME', 'core-image-minimal') + with open(os.path.join(deploydir, imgname) + '.manifest', 'r') as f: + for line in f: + splitline = line.split() + if len(splitline) > 2: + rpm_version = splitline[2] + if splitline[0] == 'rpm': + if not rpm_version.startswith('4.'): + self.fail('rpm version %s found in image, expected 4.x' % rpm_version) + break + else: + self.fail('No rpm package found in image') + + # Now do a couple of runtime tests + with runqemu("core-image-minimal", self) as qemu: + command = "rpm --version" + status, output = qemu.run(command) + self.assertEqual(0, status, 'Failed to run command "%s": %s' % (command, output)) + found_rpm_version = output.strip() + + # Make sure the retrieved rpm version is the expected one + if rpm_version not in found_rpm_version: + self.fail('RPM version is not {}, found instead {}.'.format(rpm_version, found_rpm_version)) + + # Test that the rpm database is there and working + command = "rpm -qa" + status, output = qemu.run(command) + self.assertEqual(0, status, 'Failed to run command "%s": %s' % (command, output)) + self.assertIn('packagegroup-core-boot', output) + self.assertIn('busybox', output) + @testcase(1116) def test_clutter_image_can_be_built(self):
Signature · 1 line
2.1.0
PATCH 10/11 oeqa/selftest/imagefeatures: remove gummiboot tests
In reply to Paul EggletonThese tests were _deleting_ meta-intel if it happened to appear under COREBASE, which could have been catastrophic if there was any work in progress in that directory. It turns out we don't even need meta-intel, but we do need a machine that's set up appropriately (e.g. genericx86-64). Tests that involve layers outside of OE-Core don't really belong in OE-Core, and genericx86-64 is in meta-yocto-bsp; however we will soon have the capability to have selftest tests in other layers, so remove this here so we can add a fixed version in meta-yocto-bsp after that happens.
--- meta/lib/oeqa/selftest/imagefeatures.py | 75 --------------------------------- 1 file changed, 75 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +0 -75
@@ -2,9 +2,6 @@ from oeqa.selftest.base import oeSelfTest from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu from oeqa.utils.decorators import testcase from oeqa.utils.sshcontrol import SSHControl -from os.path import isfile -from os import system -import glob import os import sys import logging
@@ -169,75 +166,3 @@ class ImageFeatures(oeSelfTest): # Build a core-image-weston bitbake('core-image-weston') - -class Gummiboot(oeSelfTest): - - meta_intel_dir = '' - - def setUpLocal(self): - """ - Common setup for test cases: 1101, 1103 - """ - - self.meta_intel_dir = get_bb_var('COREBASE') + '/meta-intel' - meta_nuc_dir = self.meta_intel_dir + '/meta-nuc' - meta_intel_repo = 'http://git.yoctoproject.org/git/meta-intel' - - # Delete meta_intel_dir - system('rm -rf ' + self.meta_intel_dir) - - # Delete meta-intel dir even if the setUp fails - self.add_command_to_tearDown('rm -rf ' + self.meta_intel_dir) - - # Clone meta-intel - runCmd('git clone ' + meta_intel_repo + ' ' + self.meta_intel_dir) - - # Add meta-intel and meta-nuc layers in conf/bblayers.conf - features = 'BBLAYERS += "' + self.meta_intel_dir + ' ' + meta_nuc_dir + '"' - self.append_bblayers_config(features) - - # Set EFI_PROVIDER = "gummiboot" and MACHINE = "nuc" in conf/local.conf - features = 'EFI_PROVIDER = "gummiboot"\n' - features += 'MACHINE = "nuc"' - self.append_config(features) - - # Run "bitbake syslinux syslinux-native parted-native dosfstools-native mtools-native core-image-minimal " - # to build a nuc/efi gummiboot image - - bitbake('syslinux syslinux-native parted-native dosfstools-native mtools-native core-image-minimal') - - @testcase(1101) - def test_efi_gummiboot_images_can_be_build(self): - """ - Summary: Check if efi/gummiboot images can be buit - Expected: 1. File gummibootx64.efi should be available in build/tmp/deploy/images/nuc - 2. Efi/gummiboot images can be built - Product: oe-core - Author: Ionut Chisanovici <[email protected]> - AutomatedBy: Daniel Istrate <[email protected]> - """ - - look_for_file = 'gummibootx64.efi' - file_location = get_bb_var('COREBASE') + '/build/tmp/deploy/images/nuc/' - - found = isfile(file_location + look_for_file) - self.assertTrue(found, 'File {} not found under {}.'.format(look_for_file, file_location)) - - @testcase(1103) - def test_wic_command_can_create_efi_gummiboot_installation_images(self): - """ - Summary: Check that wic command can create efi/gummiboot installation images - Expected: A .direct file in folder /var/tmp/wic/ must be created. - Product: oe-core - Author: Ionut Chisanovici <[email protected]> - AutomatedBy: Daniel Istrate <[email protected]> - """ - - # Create efi/gummiboot installation images - wic_create_cmd = 'wic create mkgummidisk -e core-image-minimal' - runCmd(wic_create_cmd) - - # Verify that a .direct file was created - direct_file = '/var/tmp/wic/build/*.direct' - ret = glob.glob(direct_file) - self.assertEqual(1, len(ret), 'Failed to find the .direct file')
Signature · 1 line
2.1.0
PATCH 11/11 oeqa/selftest/devtool: use new runqemu function
In reply to Paul EggletonUse the common code we now have to run QEMU instead of running it ourselves, avoiding reliance on the machine showing up at 192.168.7.2. This also makes a copy of the image rather than using -snapshot so if we need to inspect the image after a failure, we can.
Fixes [YOCTO #7928].
--- meta/lib/oeqa/selftest/devtool.py | 72 ++++++++++++++++++--------------------- 1 file changed, 34 insertions(+), 38 deletions(-)
meta/lib/oeqa/selftest/devtool.py +34 -38
@@ -8,7 +8,7 @@ import glob import oeqa.utils.ftools as ftools from oeqa.selftest.base import oeSelfTest -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer +from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer, runqemu from oeqa.utils.decorators import testcase class DevtoolBase(oeSelfTest):
@@ -799,12 +799,10 @@ class DevtoolTests(DevtoolBase): self.skipTest('No tap devices found - you must set up tap devices with scripts/runqemu-gen-tapdevs before running this test') workspacedir = os.path.join(self.builddir, 'workspace') self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be run with a workspace directory under the build directory') - import pexpect # Definitions testrecipe = 'mdadm' testfile = '/sbin/mdadm' testimage = 'oe-selftest-image' - testhost = '192.168.7.2' testcommand = '/sbin/mdadm --help' # Build an image to run bitbake("%s qemu-native qemu-helper-native" % testimage)
@@ -821,44 +819,42 @@ class DevtoolTests(DevtoolBase): self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe) result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir)) # Test that deploy-target at this point fails (properly) - result = runCmd('devtool deploy-target -n %s root@%s' % (testrecipe, testhost), ignore_status=True) + result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe, ignore_status=True) self.assertNotEqual(result.output, 0, 'devtool deploy-target should have failed, output: %s' % result.output) self.assertNotIn(result.output, 'Traceback', 'devtool deploy-target should have failed with a proper error not a traceback, output: %s' % result.output) result = runCmd('devtool build %s' % testrecipe) # First try a dry-run of deploy-target - result = runCmd('devtool deploy-target -n %s root@%s' % (testrecipe, testhost)) + result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe) self.assertIn(' %s' % testfile, result.output) # Boot the image - console = pexpect.spawn('runqemu %s %s qemuparams="-snapshot" nographic' % (machine, testimage)) - console.expect("login:", timeout=120) - # Now really test deploy-target - result = runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, testhost)) - # Run a test command to see if it was installed properly - sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' - result = runCmd('ssh %s root@%s %s' % (sshargs, testhost, testcommand)) - # Check if it deployed all of the files with the right ownership/perms - # First look on the host - need to do this under pseudo to get the correct ownership/perms - installdir = get_bb_var('D', testrecipe) - fakerootenv = get_bb_var('FAKEROOTENV', testrecipe) - fakerootcmd = get_bb_var('FAKEROOTCMD', testrecipe) - result = runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir) - filelist1 = self._process_ls_output(result.output) - - # Now look on the target - tempdir2 = tempfile.mkdtemp(prefix='devtoolqa') - self.track_for_cleanup(tempdir2) - tmpfilelist = os.path.join(tempdir2, 'files.txt') - with open(tmpfilelist, 'w') as f: - for line in filelist1: - splitline = line.split() - f.write(splitline[-1] + '\n') - result = runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, testhost)) - filelist2 = self._process_ls_output(result.output) - filelist1.sort(key=lambda item: item.split()[-1]) - filelist2.sort(key=lambda item: item.split()[-1]) - self.assertEqual(filelist1, filelist2) - # Test undeploy-target - result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, testhost)) - result = runCmd('ssh %s root@%s %s' % (sshargs, testhost, testcommand), ignore_status=True) - self.assertNotEqual(result, 0, 'undeploy-target did not remove command as it should have') - console.close() + with runqemu(testimage, self) as qemu: + # Now really test deploy-target + result = runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, qemu.ip)) + # Run a test command to see if it was installed properly + sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' + result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand)) + # Check if it deployed all of the files with the right ownership/perms + # First look on the host - need to do this under pseudo to get the correct ownership/perms + installdir = get_bb_var('D', testrecipe) + fakerootenv = get_bb_var('FAKEROOTENV', testrecipe) + fakerootcmd = get_bb_var('FAKEROOTCMD', testrecipe) + result = runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir) + filelist1 = self._process_ls_output(result.output) + + # Now look on the target + tempdir2 = tempfile.mkdtemp(prefix='devtoolqa') + self.track_for_cleanup(tempdir2) + tmpfilelist = os.path.join(tempdir2, 'files.txt') + with open(tmpfilelist, 'w') as f: + for line in filelist1: + splitline = line.split() + f.write(splitline[-1] + '\n') + result = runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, qemu.ip)) + filelist2 = self._process_ls_output(result.output) + filelist1.sort(key=lambda item: item.split()[-1]) + filelist2.sort(key=lambda item: item.split()[-1]) + self.assertEqual(filelist1, filelist2) + # Test undeploy-target + result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip)) + result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand), ignore_status=True) + self.assertNotEqual(result, 0, 'undeploy-target did not remove command as it should have')
Signature · 1 line
2.1.0