Skip to contentSkip to conversation

Properly fix oe-selftest QEMU usage + related fixes

PATCH New Paul Eggleton 1 participant 12 messages Jul 27, 2015

lore
Share & tools
lore
#0 lore raw

cover Properly fix oe-selftest QEMU usage + related fixes

The tests in oeqa/selftest/imagefeatures have had a long list of issues that we've worked around; this patchset implements the ability to run QEMU from oe-selftest tests properly and attempts to fix the remaining known issues with the tests. At the same time we can make the devtool test that uses QEMU use the new runqemu() logic, so do that as well.

When we did do the proper runqemu() implementation however, I uncovered a number of other problems to do with error handling, process termination and logging. This patchset should address those as well.

Thanks to Richard for the initial set of fixes which I built upon here.

The following changes since commit 03d01393d14b7b20dcb40ff89b1628883fd3b545:

toaster.bbclass: Fix ValueError (2015-07-27 12:26:57 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/oe-selftest-qemu-fixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/oe-selftest-qemu-fixes

Paul Eggleton (9):
oe-selftest: add scripts/lib and bitbake/lib to path oeqa/targetcontrol: create test directory before copying rootfs image oeqa/utils/qemurunner: fix logging
oeqa/utils/qemurunner: avoid blocking on stty when running under
oe-selftest
oeqa/utils/qemurunner: fix error handling if runqemu exits with an
error
oeqa/targetcontrol: write QemuRunner log output to a file oeqa/selftest/imagefeatures: fix RPM4 test oeqa/selftest/imagefeatures: remove gummiboot tests oeqa/selftest/devtool: use new runqemu function

Richard Purdie (2):
oeqa/selftest/imagefeatures: Use QemuTarget code oeqa/selftest/imagefeatures: fix SSH without password tests

 meta/lib/oeqa/selftest/devtool.py       |  72 +++++-----
 meta/lib/oeqa/selftest/imagefeatures.py | 247 +++++++++-----------------------
 meta/lib/oeqa/targetcontrol.py          |  12 ++
 meta/lib/oeqa/utils/commands.py         |  49 +++++++
 meta/lib/oeqa/utils/qemurunner.py       |  70 +++++----
 scripts/oe-selftest                     |   5 +-
 6 files changed, 207 insertions(+), 248 deletions(-)
Signature · 1 line
2.1.0
#1 lore raw

PATCH 1/11 oe-selftest: add scripts/lib and bitbake/lib to path

In particular, this allows us to use code from bitbake's bb module (such as tinfoil).

---
 scripts/oe-selftest | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/oe-selftest +4 -1
index fa6245a..6db6e23 100755
@@ -31,7 +31,10 @@ import unittest
 import logging
 import argparse
 
-sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'meta/lib')))
+sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/lib')
+import scriptpath
+scriptpath.add_bitbake_lib_path()
+scriptpath.add_oe_lib_path()
 
 import oeqa.selftest
 import oeqa.utils.ftools as ftools
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/11 oeqa/targetcontrol: create test directory before copying rootfs image

In reply to Paul Eggleton

The test directory might not exist at this point so just go ahead and create it.

---
 meta/lib/oeqa/targetcontrol.py | 1 +
 1 file changed, 1 insertion(+)
meta/lib/oeqa/targetcontrol.py +1 -0
index 60b09b2..beacdaf 100644
@@ -143,6 +143,7 @@ class QemuTarget(BaseTarget):
 
     def deploy(self):
         try:
+            bb.utils.mkdirhier(self.testdir)
             shutil.copyfile(self.origrootfs, self.rootfs)
         except Exception as e:
             bb.fatal("Error copying rootfs: %s" % e)
Signature · 1 line
2.1.0
#3 lore raw

PATCH 3/11 oeqa/utils/qemurunner: fix logging

In reply to Paul Eggleton

OE-Core commit 519e381278d40bdac79add340e4c0460a9f97e17 unfortunately broke logging in two different ways:

1) it prevented logging to the task log from working within bitbake -c testimage. This is due to the logger object being set up too early which interferes with BitBake's own logging. If we prefix the name with "BitBake." everything works (and we don't need to set the logging level).

2) Additionally because it called the log functions on the logging module and not the logger object it set up, this caused the oe-selftest logging to start printing everything from that point forward.

Fix these two issues and return us to the desired behaviour for do_testimage.

---
 meta/lib/oeqa/utils/qemurunner.py | 49 +++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 25 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +24 -25
index 59c9934..4de3c64 100644
@@ -14,8 +14,7 @@ import socket
 import select
 
 import logging
-logger = logging.getLogger("QemuRunner")
-logger.setLevel(logging.DEBUG - 2)
+logger = logging.getLogger("BitBake.QemuRunner")
 
 class QemuRunner:
 
@@ -51,11 +50,11 @@ class QemuRunner:
             self.server_socket.bind(("127.0.0.1",0))
             self.server_socket.listen(2)
             self.serverport = self.server_socket.getsockname()[1]
-            logging.info("Created listening socket for qemu serial console on: 127.0.0.1:%s" % self.serverport)
+            logger.info("Created listening socket for qemu serial console on: 127.0.0.1:%s" % self.serverport)
             return True
         except socket.error, msg:
             self.server_socket.close()
-            logging.error("Failed to create listening socket: %s" % msg[1])
+            logger.error("Failed to create listening socket: %s" % msg[1])
             return False
 
 
@@ -68,18 +67,18 @@ class QemuRunner:
         if self.display:
             os.environ["DISPLAY"] = self.display
         else:
-            logging.error("To start qemu I need a X desktop, please set DISPLAY correctly (e.g. DISPLAY=:1)")
+            logger.error("To start qemu I need a X desktop, please set DISPLAY correctly (e.g. DISPLAY=:1)")
             return False
         if not os.path.exists(self.rootfs):
-            logging.error("Invalid rootfs %s" % self.rootfs)
+            logger.error("Invalid rootfs %s" % self.rootfs)
             return False
         if not os.path.exists(self.tmpdir):
-            logging.error("Invalid TMPDIR path %s" % self.tmpdir)
+            logger.error("Invalid TMPDIR path %s" % self.tmpdir)
             return False
         else:
             os.environ["OE_TMPDIR"] = self.tmpdir
         if not os.path.exists(self.deploy_dir_image):
-            logging.error("Invalid DEPLOY_DIR_IMAGE path %s" % self.deploy_dir_image)
+            logger.error("Invalid DEPLOY_DIR_IMAGE path %s" % self.deploy_dir_image)
             return False
         else:
             os.environ["DEPLOY_DIR_IMAGE"] = self.deploy_dir_image
@@ -97,28 +96,28 @@ class QemuRunner:
         launch_cmd = 'runqemu %s %s %s' % (self.machine, self.rootfs, self.qemuparams)
         self.runqemu = subprocess.Popen(launch_cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,preexec_fn=os.setpgrp)
 
-        logging.info("runqemu started, pid is %s" % self.runqemu.pid)
-        logging.info("waiting at most %s seconds for qemu pid" % self.runqemutime)
+        logger.info("runqemu started, pid is %s" % self.runqemu.pid)
+        logger.info("waiting at most %s seconds for qemu pid" % self.runqemutime)
         endtime = time.time() + self.runqemutime
         while not self.is_alive() and time.time() < endtime:
             time.sleep(1)
 
         if self.is_alive():
-            logging.info("qemu started - qemu procces pid is %s" % self.qemupid)
+            logger.info("qemu started - qemu procces pid is %s" % self.qemupid)
             cmdline = ''
             with open('/proc/%s/cmdline' % self.qemupid) as p:
                 cmdline = p.read()
             ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1])
             if not ips or len(ips) != 3:
-                logging.info("Couldn't get ip from qemu process arguments! Here is the qemu command line used: %s" % cmdline)
+                logger.info("Couldn't get ip from qemu process arguments! Here is the qemu command line used: %s" % cmdline)
                 self.stop()
                 return False
             else:
                 self.ip = ips[0]
                 self.server_ip = ips[1]
-            logging.info("Target IP: %s" % self.ip)
-            logging.info("Server IP: %s" % self.server_ip)
-            logging.info("Waiting at most %d seconds for login banner" % self.boottime)
+            logger.info("Target IP: %s" % self.ip)
+            logger.info("Server IP: %s" % self.server_ip)
+            logger.info("Waiting at most %d seconds for login banner" % self.boottime)
             endtime = time.time() + self.boottime
             socklist = [self.server_socket]
             reachedlogin = False
@@ -131,7 +130,7 @@ class QemuRunner:
                         self.qemusock.setblocking(0)
                         socklist.append(self.qemusock)
                         socklist.remove(self.server_socket)
-                        logging.info("Connection from %s:%s" % addr)
+                        logger.info("Connection from %s:%s" % addr)
                     else:
                         data = sock.recv(1024)
                         if data:
@@ -140,24 +139,24 @@ class QemuRunner:
                             if re.search(".* login:", self.bootlog):
                                 stopread = True
                                 reachedlogin = True
-                                logging.info("Reached login banner")
+                                logger.info("Reached login banner")
                         else:
                             socklist.remove(sock)
                             sock.close()
                             stopread = True
 
             if not reachedlogin:
-                logging.info("Target didn't reached login boot in %d seconds" % self.boottime)
+                logger.info("Target didn't reached login boot in %d seconds" % self.boottime)
                 lines = "\n".join(self.bootlog.splitlines()[-5:])
-                logging.info("Last 5 lines of text:\n%s" % lines)
-                logging.info("Check full boot log: %s" % self.logfile)
+                logger.info("Last 5 lines of text:\n%s" % lines)
+                logger.info("Check full boot log: %s" % self.logfile)
                 self.stop()
                 return False
         else:
-            logging.info("Qemu pid didn't appeared in %s seconds" % self.runqemutime)
+            logger.info("Qemu pid didn't appeared in %s seconds" % self.runqemutime)
             output = self.runqemu.stdout
             self.stop()
-            logging.info("Output from runqemu:\n%s" % output.read())
+            logger.info("Output from runqemu:\n%s" % output.read())
             return False
 
         return self.is_alive()
@@ -165,13 +164,13 @@ class QemuRunner:
     def stop(self):
 
         if self.runqemu:
-            logging.info("Sending SIGTERM to runqemu")
+            logger.info("Sending SIGTERM to runqemu")
             os.killpg(self.runqemu.pid, signal.SIGTERM)
             endtime = time.time() + self.runqemutime
             while self.runqemu.poll() is None and time.time() < endtime:
                 time.sleep(1)
             if self.runqemu.poll() is None:
-                logging.info("Sending SIGKILL to runqemu")
+                logger.info("Sending SIGKILL to runqemu")
                 os.killpg(self.runqemu.pid, signal.SIGKILL)
             self.runqemu = None
         if self.server_socket:
@@ -181,7 +180,7 @@ class QemuRunner:
         self.ip = None
 
     def restart(self, qemuparams = None):
-        logging.info("Restarting qemu process")
+        logger.info("Restarting qemu process")
         if self.runqemu.poll() is None:
             self.stop()
         if self.start(qemuparams):
Signature · 1 line
2.1.0
#4 lore raw

PATCH 4/11 oeqa/utils/qemurunner: avoid blocking on stty when running under oe-selftest

In reply to Paul Eggleton

runqemu-internal runs stty to return the terminal to its previous state in case QEMU hasn't done that properly (which it at least used to do when it crashed). For some reason I have yet to determine, stty blocks (on tcsetattr() according to gdb) when run within QemuRunner() under oe-selftest, with the result that we always wait until the timeout and then we kill the script, which adds an extra delay after QEMU is stopped. Naturally you would assume that this is something to do with the nature of the terminal under which it is being run; however no amount of playing around with stdin/stdout/stderr seemed to fix the issue, apart from passing in subprocess.PIPE as stdin which makes stty error out with "stty: standard input: Inappropriate ioctl for device". I was also unable to come up with a reliable test for the terminal which we could use inside runqemu-internal to avoid calling stty. For now, go with the stdin=subprocess.PIPE workaround to at least avoid the delay with minimal ill effect.

---
 meta/lib/oeqa/utils/qemurunner.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/qemurunner.py +5 -1
index 4de3c64..11186d6 100644
@@ -94,7 +94,11 @@ class QemuRunner:
             self.qemuparams = self.qemuparams[:-1] + " " + qemuparams + " " + '\"'
 
         launch_cmd = 'runqemu %s %s %s' % (self.machine, self.rootfs, self.qemuparams)
-        self.runqemu = subprocess.Popen(launch_cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,preexec_fn=os.setpgrp)
+        # FIXME: We pass in stdin=subprocess.PIPE here to work around stty
+        # blocking at the end of the runqemu script when using this within
+        # oe-selftest (this makes stty error out immediately). There ought
+        # to be a proper fix but this will suffice for now.
+        self.runqemu = subprocess.Popen(launch_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE, preexec_fn=os.setpgrp)
 
         logger.info("runqemu started, pid is %s" % self.runqemu.pid)
         logger.info("waiting at most %s seconds for qemu pid" % self.runqemutime)
Signature · 1 line
2.1.0
#5 lore raw

PATCH 5/11 oeqa/utils/qemurunner: fix error handling if runqemu exits with an error

In reply to Paul Eggleton

* Don't wait for QEMU to start if it's never going to (because runqemu exited with an error)
* Don't error out if killing the process fails with "no such process" (we don't care if it's already dead)

---
 meta/lib/oeqa/utils/qemurunner.py | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
meta/lib/oeqa/utils/qemurunner.py +14 -1
index 11186d6..71da21c 100644
@@ -12,6 +12,7 @@ import signal
 import re
 import socket
 import select
+import errno
 
 import logging
 logger = logging.getLogger("BitBake.QemuRunner")
@@ -104,6 +105,14 @@ class QemuRunner:
         logger.info("waiting at most %s seconds for qemu pid" % self.runqemutime)
         endtime = time.time() + self.runqemutime
         while not self.is_alive() and time.time() < endtime:
+            if self.runqemu.poll():
+                if self.runqemu.returncode:
+                    # No point waiting any longer
+                    logger.info('runqemu exited with code %d' % self.runqemu.returncode)
+                    output = self.runqemu.stdout
+                    self.stop()
+                    logger.info("Output from runqemu:\n%s" % output.read())
+                    return False
             time.sleep(1)
 
         if self.is_alive():
@@ -169,7 +178,11 @@ class QemuRunner:
 
         if self.runqemu:
             logger.info("Sending SIGTERM to runqemu")
-            os.killpg(self.runqemu.pid, signal.SIGTERM)
+            try:
+                os.killpg(self.runqemu.pid, signal.SIGTERM)
+            except OSError as e:
+                if e.errno != errno.ESRCH:
+                    raise
             endtime = time.time() + self.runqemutime
             while self.runqemu.poll() is None and time.time() < endtime:
                 time.sleep(1)
Signature · 1 line
2.1.0
#6 lore raw

PATCH 6/11 oeqa/targetcontrol: write QemuRunner log output to a file

In reply to Paul Eggleton

If we use this outside of testimage we don't have a task log; so let's just explicitly write the log output to a file all the time so it's always there to look at (particularly useful when runqemu exits immediately with an error.)

---
 meta/lib/oeqa/targetcontrol.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
meta/lib/oeqa/targetcontrol.py +11 -0
index beacdaf..138e6178 100644
@@ -10,6 +10,7 @@ import subprocess
 import bb
 import traceback
 import sys
+import logging
 from oeqa.utils.sshcontrol import SSHControl
 from oeqa.utils.qemurunner import QemuRunner
 from oeqa.utils.qemutinyrunner import QemuTinyRunner
@@ -123,6 +124,16 @@ class QemuTarget(BaseTarget):
         self.rootfs = os.path.join(self.testdir, d.getVar("IMAGE_LINK_NAME", True) + '-testimage.' + self.image_fstype)
         self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE", False) + '-' + d.getVar('MACHINE', False) + '.bin')
 
+        # Log QemuRunner log output to a file
+        import oe.path
+        bb.utils.mkdirhier(self.testdir)
+        self.qemurunnerlog = os.path.join(self.testdir, 'qemurunner_log.%s' % self.datetime)
+        logger = logging.getLogger('BitBake.QemuRunner')
+        loggerhandler = logging.FileHandler(self.qemurunnerlog)
+        loggerhandler.setFormatter(logging.Formatter("%(levelname)s: %(message)s"))
+        logger.addHandler(loggerhandler)
+        oe.path.symlink(os.path.basename(self.qemurunnerlog), os.path.join(self.testdir, 'qemurunner_log'), force=True)
+
         if d.getVar("DISTRO", True) == "poky-tiny":
             self.runner = QemuTinyRunner(machine=d.getVar("MACHINE", True),
                             rootfs=self.rootfs,
Signature · 1 line
2.1.0
#7 lore raw

PATCH 7/11 oeqa/selftest/imagefeatures: Use QemuTarget code

In reply to Paul Eggleton

From: Richard Purdie <[email protected]>

Create a runqemu function which uses the QemuTarget() code from oeqa.targetcontrol to setup the QEMU instance, with all of the added robustness that that gives us. To do this, a datastore is needed for the recipe in question (core-image-minimal) so we do the work needed to set this up. We then use this runqemu function within the imagefeatures tests instead of a hand-rolled implementation.

We can then use SSHControl to run the SSH tests rather than rolling our own code to do that as an added bonus.

Fixed and extended by Paul Eggleton <[email protected]>.

Part of the fix for [YOCTO #7994].

---
 meta/lib/oeqa/selftest/imagefeatures.py | 96 +++++++--------------------------
 meta/lib/oeqa/utils/commands.py         | 49 +++++++++++++++++
 2 files changed, 67 insertions(+), 78 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +18 -78
index 1795b7b..d48435f 100644
@@ -1,20 +1,18 @@
 from oeqa.selftest.base import oeSelfTest
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
 from oeqa.utils.decorators import testcase
-import pexpect
+from oeqa.utils.sshcontrol import SSHControl
 from os.path import isfile
-from os import system, killpg
+from os import system
 import glob
-import signal
-
+import os
+import sys
+import logging
 
 class ImageFeatures(oeSelfTest):
 
     test_user = 'tester'
     root_user = 'root'
-    prompt = r'qemux86:\S+[$#]\s+'
-    ssh_cmd = "ssh {} -l {} -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
-    get_ip_patt = r'\s+ip=(?P<qemu_ip>(\d+.){3}\d+)::'
 
     @testcase(1107)
     def test_non_root_user_can_connect_via_ssh_without_password(self):
@@ -37,41 +35,12 @@ class ImageFeatures(oeSelfTest):
         # Build a core-image-minimal
         bitbake('core-image-minimal')
 
-        # Boot qemu image
-        proc_qemu = pexpect.spawn('runqemu qemux86 nographic')
-        try:
-            proc_qemu.expect(self.get_ip_patt, timeout=100)
-            qemu_ip = proc_qemu.match.group('qemu_ip')
-            proc_qemu.expect('qemux86 login:', timeout=100)
-        except Exception as e:
-            try:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-            except:
-                pass
-            self.fail('Failed to start qemu: %s' % e)
-
-        # Attempt to ssh with each user into qemu with empty password
-        for user in [self.root_user, self.test_user]:
-            proc_ssh = pexpect.spawn(self.ssh_cmd.format(qemu_ip, user))
-            index = proc_ssh.expect([self.prompt, pexpect.TIMEOUT, pexpect.EOF])
-            if index == 0:
-                # user successfully logged in with empty password
-                pass
-            elif index == 1:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-                proc_ssh.terminate()
-                self.fail('Failed to ssh with {} user into qemu (timeout).'.format(user))
-            else:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-                proc_ssh.terminate()
-                self.fail('Failed to ssh with {} user into qemu (eof).'.format(user))
-            proc_ssh.terminate()
-
-        # Cleanup
-        try:
-            killpg(proc_qemu.pid, signal.SIGTERM)
-        except:
-            pass
+        with runqemu("core-image-minimal", self) as qemu:
+            # Attempt to ssh with each user into qemu with empty password
+            for user in [self.root_user, self.test_user]:
+                ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
+                status, output = ssh.run("true")
+                self.assertEqual(status, 0, 'ssh to user %s failed with %s' % (user, output))
 
     @testcase(1115)
     def test_all_users_can_connect_via_ssh_without_password(self):
@@ -82,7 +51,6 @@ class ImageFeatures(oeSelfTest):
         Author:      Ionut Chisanovici <[email protected]>
         AutomatedBy: Daniel Istrate <[email protected]>
         """
-
         features = 'EXTRA_IMAGE_FEATURES += "ssh-server-openssh allow-empty-password"\n'
         features += 'INHERIT += "extrausers"\n'
         features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
@@ -93,41 +61,13 @@ class ImageFeatures(oeSelfTest):
         # Build a core-image-minimal
         bitbake('core-image-minimal')
 
-        # Boot qemu image
-        proc_qemu = pexpect.spawn('runqemu qemux86 nographic')
-        try:
-            proc_qemu.expect(self.get_ip_patt, timeout=100)
-            qemu_ip = proc_qemu.match.group('qemu_ip')
-            proc_qemu.expect('qemux86 login:', timeout=100)
-        except Exception as e:
-            try:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-            except:
-                pass
-            self.fail('Failed to start qemu: %s' % e)
-
-        # Attempt to ssh with each user into qemu with empty password
-        for user in [self.root_user, self.test_user]:
-            proc_ssh = pexpect.spawn(self.ssh_cmd.format(qemu_ip, user))
-            index = proc_ssh.expect([self.prompt, pexpect.TIMEOUT, pexpect.EOF])
-            if index == 0:
-                # user successfully logged in with empty password
-                pass
-            elif index == 1:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-                proc_ssh.terminate()
-                self.fail('Failed to ssh with {} user into qemu (timeout).'.format(user))
-            else:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-                proc_ssh.terminate()
-                self.fail('Failed to ssh with {} user into qemu (eof).'.format(user))
-            proc_ssh.terminate()
+        with runqemu("core-image-minimal", self) as qemu:
+            # Attempt to ssh with each user into qemu with empty password
+            for user in [self.root_user, self.test_user]:
+                ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
+                status, output = ssh.run("true")
+                self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output)
 
-        # Cleanup
-        try:
-            killpg(proc_qemu.pid, signal.SIGTERM)
-        except:
-            pass
 
     @testcase(1114)
     def test_rpm_version_4_support_on_image(self):
meta/lib/oeqa/utils/commands.py +49 -0

58 lines not on this page · raw message

Signature · 1 line
2.1.0
#8 lore raw

PATCH 8/11 oeqa/selftest/imagefeatures: fix SSH without password tests

In reply to Paul Eggleton

From: Richard Purdie <[email protected]>

* We need to set EXTRA_IMAGE_FEATURES outright or existing values will affect the test
* For test case 1107 we need "empty-root-password" to match the behaviour described in the test case
* For test case 1115 we shouldn't be able to connect as root with the features we are setting

Test cases 1107 and 1115 have been updated in Testopia to match.

---
 meta/lib/oeqa/selftest/imagefeatures.py | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +9 -4
index d48435f..20cc58d 100644
@@ -25,7 +25,7 @@ class ImageFeatures(oeSelfTest):
         AutomatedBy: Daniel Istrate <[email protected]>
         """
 
-        features = 'EXTRA_IMAGE_FEATURES += "ssh-server-openssh empty-root-password"\n'
+        features = 'EXTRA_IMAGE_FEATURES = "ssh-server-openssh empty-root-password allow-empty-password"\n'
         features += 'INHERIT += "extrausers"\n'
         features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
 
@@ -46,12 +46,14 @@ class ImageFeatures(oeSelfTest):
     def test_all_users_can_connect_via_ssh_without_password(self):
         """
         Summary:     Check if all users can connect via ssh without password
-        Expected:    1. Connection to the image via ssh using root or tester user without providing a password should be allowed.
+        Expected: 1. Connection to the image via ssh using root user without providing a password should NOT be allowed.
+                  2. Connection to the image via ssh using tester user without providing a password should be allowed.
         Product:     oe-core
         Author:      Ionut Chisanovici <[email protected]>
         AutomatedBy: Daniel Istrate <[email protected]>
         """
-        features = 'EXTRA_IMAGE_FEATURES += "ssh-server-openssh allow-empty-password"\n'
+
+        features = 'EXTRA_IMAGE_FEATURES = "ssh-server-openssh allow-empty-password"\n'
         features += 'INHERIT += "extrausers"\n'
         features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
 
@@ -66,7 +68,10 @@ class ImageFeatures(oeSelfTest):
             for user in [self.root_user, self.test_user]:
                 ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
                 status, output = ssh.run("true")
-                self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output)
+                if user == 'root':
+                    self.assertNotEqual(status, 0, 'ssh to user root was allowed when it should not have been')
+                else:
+                    self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output)
 
 
     @testcase(1114)
Signature · 1 line
2.1.0
#9 lore raw

PATCH 9/11 oeqa/selftest/imagefeatures: fix RPM4 test

In reply to Paul Eggleton

* Use our new runqemu function
* Don't hard-code the RPM4 version
* Double-check the native version is RPM4
* Check that an rpm 4.x package is in the image manifest (this isn't strictly necessary, but "belt-and-braces" and it serves as an example of how to do that)
* Check that the database is working on the target
* Ensure the image actually has openssh in it so we can connect to it

Initial runqemu adaptation by Richard Purdie <[email protected]>

Part of the fix for [YOCTO #7994].

---
 meta/lib/oeqa/selftest/imagefeatures.py | 75 +++++++++++++++++++--------------
 1 file changed, 44 insertions(+), 31 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +44 -31
index 20cc58d..a370d1e 100644
@@ -78,16 +78,17 @@ class ImageFeatures(oeSelfTest):
     def test_rpm_version_4_support_on_image(self):
         """
         Summary:     Check rpm version 4 support on image
-        Expected:    Rpm version must be 4.11.2
+        Expected:    Rpm version must be 4.x
         Product:     oe-core
         Author:      Ionut Chisanovici <[email protected]>
         AutomatedBy: Daniel Istrate <[email protected]>
         """
 
-        rpm_version = '4.11.2'
-        features = 'IMAGE_INSTALL_append = " rpm"\n'
-        features += 'PREFERRED_VERSION_rpm = "{}"\n'.format(rpm_version)
-        features += 'PREFERRED_VERSION_rpm-native = "{}"\n'.format(rpm_version)
+        features = 'PREFERRED_VERSION_rpm = "4.%"\n'
+        features += 'PREFERRED_VERSION_rpm-native = "4.%"\n'
+        # Use openssh in IMAGE_INSTALL instead of ssh-server-openssh in EXTRA_IMAGE_FEATURES as a workaround for bug 8047
+        features += 'IMAGE_INSTALL_append = " openssh"\n'
+        features += 'EXTRA_IMAGE_FEATURES = "empty-root-password allow-empty-password package-management"\n'
         features += 'RPMROOTFSDEPENDS_remove = "rpmresolve-native:do_populate_sysroot"'
 
         # Append 'features' to local.conf
@@ -96,32 +97,44 @@ class ImageFeatures(oeSelfTest):
         # Build a core-image-minimal
         bitbake('core-image-minimal')
 
-        # Boot qemu image & get rpm version
-        proc_qemu = pexpect.spawn('runqemu qemux86 nographic')
-        try:
-            proc_qemu.expect('qemux86 login:', timeout=100)
-            proc_qemu.sendline(self.root_user)
-            proc_qemu.expect(self.prompt)
-            proc_qemu.sendline('rpm --version')
-            proc_qemu.expect(self.prompt)
-        except Exception as e:
-            try:
-                killpg(proc_qemu.pid, signal.SIGTERM)
-            except:
-                pass
-            self.fail('Failed to start qemu: %s' % e)
-
-        found_rpm_version = proc_qemu.before
-
-        # Make sure the retrieved rpm version is the expected one
-        self.assertIn(rpm_version, found_rpm_version,
-                      'RPM version is not {}, found instead {}.'.format(rpm_version, found_rpm_version))
-
-        # Cleanup (close qemu)
-        try:
-            killpg(proc_qemu.pid, signal.SIGTERM)
-        except:
-            pass
+        # Check the native version of rpm is correct
+        native_bindir = get_bb_var('STAGING_BINDIR_NATIVE')
+        result = runCmd(os.path.join(native_bindir, 'rpm') + ' --version')
+        self.assertIn('version 4.', result.output)
+
+        # Check manifest for the rpm package
+        deploydir = get_bb_var('DEPLOY_DIR_IMAGE')
+        imgname = get_bb_var('IMAGE_LINK_NAME', 'core-image-minimal')
+        with open(os.path.join(deploydir, imgname) + '.manifest', 'r') as f:
+            for line in f:
+                splitline = line.split()
+                if len(splitline) > 2:
+                    rpm_version = splitline[2]
+                    if splitline[0] == 'rpm':
+                        if not rpm_version.startswith('4.'):
+                            self.fail('rpm version %s found in image, expected 4.x' % rpm_version)
+                        break
+            else:
+                self.fail('No rpm package found in image')
+
+        # Now do a couple of runtime tests
+        with runqemu("core-image-minimal", self) as qemu:
+            command = "rpm --version"
+            status, output = qemu.run(command)
+            self.assertEqual(0, status, 'Failed to run command "%s": %s' % (command, output))
+            found_rpm_version = output.strip()
+
+            # Make sure the retrieved rpm version is the expected one
+            if rpm_version not in found_rpm_version:
+                self.fail('RPM version is not {}, found instead {}.'.format(rpm_version, found_rpm_version))
+
+            # Test that the rpm database is there and working
+            command = "rpm -qa"
+            status, output = qemu.run(command)
+            self.assertEqual(0, status, 'Failed to run command "%s": %s' % (command, output))
+            self.assertIn('packagegroup-core-boot', output)
+            self.assertIn('busybox', output)
+
 
     @testcase(1116)
     def test_clutter_image_can_be_built(self):
Signature · 1 line
2.1.0
#10 lore raw

PATCH 10/11 oeqa/selftest/imagefeatures: remove gummiboot tests

In reply to Paul Eggleton

These tests were _deleting_ meta-intel if it happened to appear under COREBASE, which could have been catastrophic if there was any work in progress in that directory. It turns out we don't even need meta-intel, but we do need a machine that's set up appropriately (e.g. genericx86-64). Tests that involve layers outside of OE-Core don't really belong in OE-Core, and genericx86-64 is in meta-yocto-bsp; however we will soon have the capability to have selftest tests in other layers, so remove this here so we can add a fixed version in meta-yocto-bsp after that happens.

---
 meta/lib/oeqa/selftest/imagefeatures.py | 75 ---------------------------------
 1 file changed, 75 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +0 -75
index a370d1e..fcffc42 100644
@@ -2,9 +2,6 @@ from oeqa.selftest.base import oeSelfTest
 from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
 from oeqa.utils.decorators import testcase
 from oeqa.utils.sshcontrol import SSHControl
-from os.path import isfile
-from os import system
-import glob
 import os
 import sys
 import logging
@@ -169,75 +166,3 @@ class ImageFeatures(oeSelfTest):
         # Build a core-image-weston
         bitbake('core-image-weston')
 
-
-class Gummiboot(oeSelfTest):
-
-    meta_intel_dir = ''
-
-    def setUpLocal(self):
-        """
-        Common setup for test cases: 1101, 1103
-        """
-
-        self.meta_intel_dir = get_bb_var('COREBASE') + '/meta-intel'
-        meta_nuc_dir = self.meta_intel_dir + '/meta-nuc'
-        meta_intel_repo = 'http://git.yoctoproject.org/git/meta-intel'
-
-        # Delete meta_intel_dir
-        system('rm -rf ' + self.meta_intel_dir)
-
-        # Delete meta-intel dir even if the setUp fails
-        self.add_command_to_tearDown('rm -rf ' + self.meta_intel_dir)
-
-        # Clone meta-intel
-        runCmd('git clone ' + meta_intel_repo + ' ' + self.meta_intel_dir)
-
-        # Add meta-intel and meta-nuc layers in conf/bblayers.conf
-        features = 'BBLAYERS += "' + self.meta_intel_dir + ' ' + meta_nuc_dir + '"'
-        self.append_bblayers_config(features)
-
-        # Set EFI_PROVIDER = "gummiboot" and MACHINE = "nuc" in conf/local.conf
-        features = 'EFI_PROVIDER = "gummiboot"\n'
-        features += 'MACHINE = "nuc"'
-        self.append_config(features)
-
-        # Run "bitbake syslinux syslinux-native parted-native dosfstools-native mtools-native core-image-minimal "
-        # to build a nuc/efi gummiboot image
-
-        bitbake('syslinux syslinux-native parted-native dosfstools-native mtools-native core-image-minimal')
-
-    @testcase(1101)
-    def test_efi_gummiboot_images_can_be_build(self):
-        """
-        Summary:     Check if efi/gummiboot images can be buit
-        Expected:    1. File gummibootx64.efi should be available in build/tmp/deploy/images/nuc
-                     2. Efi/gummiboot images can be built
-        Product:     oe-core
-        Author:      Ionut Chisanovici <[email protected]>
-        AutomatedBy: Daniel Istrate <[email protected]>
-        """
-
-        look_for_file = 'gummibootx64.efi'
-        file_location = get_bb_var('COREBASE') + '/build/tmp/deploy/images/nuc/'
-
-        found = isfile(file_location + look_for_file)
-        self.assertTrue(found, 'File {} not found under {}.'.format(look_for_file, file_location))
-
-    @testcase(1103)
-    def test_wic_command_can_create_efi_gummiboot_installation_images(self):
-        """
-        Summary:     Check that wic command can create efi/gummiboot installation images
-        Expected:    A .direct file in folder /var/tmp/wic/ must be created.
-        Product:     oe-core
-        Author:      Ionut Chisanovici <[email protected]>
-        AutomatedBy: Daniel Istrate <[email protected]>
-        """
-
-        # Create efi/gummiboot installation images
-        wic_create_cmd = 'wic create mkgummidisk -e core-image-minimal'
-        runCmd(wic_create_cmd)
-
-        # Verify that a .direct file was created
-        direct_file = '/var/tmp/wic/build/*.direct'
-        ret = glob.glob(direct_file)
-        self.assertEqual(1, len(ret), 'Failed to find the .direct file')
Signature · 1 line
2.1.0
#11 lore raw

PATCH 11/11 oeqa/selftest/devtool: use new runqemu function

In reply to Paul Eggleton

Use the common code we now have to run QEMU instead of running it ourselves, avoiding reliance on the machine showing up at 192.168.7.2. This also makes a copy of the image rather than using -snapshot so if we need to inspect the image after a failure, we can.

Fixes [YOCTO #7928].

---
 meta/lib/oeqa/selftest/devtool.py | 72 ++++++++++++++++++---------------------
 1 file changed, 34 insertions(+), 38 deletions(-)
meta/lib/oeqa/selftest/devtool.py +34 -38
index 21cd7f5..c833e54 100644
@@ -8,7 +8,7 @@ import glob
 
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.base import oeSelfTest
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer, runqemu
 from oeqa.utils.decorators import testcase
 
 class DevtoolBase(oeSelfTest):
@@ -799,12 +799,10 @@ class DevtoolTests(DevtoolBase):
             self.skipTest('No tap devices found - you must set up tap devices with scripts/runqemu-gen-tapdevs before running this test')
         workspacedir = os.path.join(self.builddir, 'workspace')
         self.assertTrue(not os.path.exists(workspacedir), 'This test cannot be run with a workspace directory under the build directory')
-        import pexpect
         # Definitions
         testrecipe = 'mdadm'
         testfile = '/sbin/mdadm'
         testimage = 'oe-selftest-image'
-        testhost = '192.168.7.2'
         testcommand = '/sbin/mdadm --help'
         # Build an image to run
         bitbake("%s qemu-native qemu-helper-native" % testimage)
@@ -821,44 +819,42 @@ class DevtoolTests(DevtoolBase):
         self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
         result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         # Test that deploy-target at this point fails (properly)
-        result = runCmd('devtool deploy-target -n %s root@%s' % (testrecipe, testhost), ignore_status=True)
+        result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe, ignore_status=True)
         self.assertNotEqual(result.output, 0, 'devtool deploy-target should have failed, output: %s' % result.output)
         self.assertNotIn(result.output, 'Traceback', 'devtool deploy-target should have failed with a proper error not a traceback, output: %s' % result.output)
         result = runCmd('devtool build %s' % testrecipe)
         # First try a dry-run of deploy-target
-        result = runCmd('devtool deploy-target -n %s root@%s' % (testrecipe, testhost))
+        result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe)
         self.assertIn('  %s' % testfile, result.output)
         # Boot the image
-        console = pexpect.spawn('runqemu %s %s qemuparams="-snapshot" nographic' % (machine, testimage))
-        console.expect("login:", timeout=120)
-        # Now really test deploy-target
-        result = runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, testhost))
-        # Run a test command to see if it was installed properly
-        sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
-        result = runCmd('ssh %s root@%s %s' % (sshargs, testhost, testcommand))
-        # Check if it deployed all of the files with the right ownership/perms
-        # First look on the host - need to do this under pseudo to get the correct ownership/perms
-        installdir = get_bb_var('D', testrecipe)
-        fakerootenv = get_bb_var('FAKEROOTENV', testrecipe)
-        fakerootcmd = get_bb_var('FAKEROOTCMD', testrecipe)
-        result = runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir)
-        filelist1 = self._process_ls_output(result.output)
-
-        # Now look on the target
-        tempdir2 = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir2)
-        tmpfilelist = os.path.join(tempdir2, 'files.txt')
-        with open(tmpfilelist, 'w') as f:
-            for line in filelist1:
-                splitline = line.split()
-                f.write(splitline[-1] + '\n')
-        result = runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, testhost))
-        filelist2 = self._process_ls_output(result.output)
-        filelist1.sort(key=lambda item: item.split()[-1])
-        filelist2.sort(key=lambda item: item.split()[-1])
-        self.assertEqual(filelist1, filelist2)
-        # Test undeploy-target
-        result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, testhost))
-        result = runCmd('ssh %s root@%s %s' % (sshargs, testhost, testcommand), ignore_status=True)
-        self.assertNotEqual(result, 0, 'undeploy-target did not remove command as it should have')
-        console.close()
+        with runqemu(testimage, self) as qemu:
+            # Now really test deploy-target
+            result = runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, qemu.ip))
+            # Run a test command to see if it was installed properly
+            sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
+            result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand))
+            # Check if it deployed all of the files with the right ownership/perms
+            # First look on the host - need to do this under pseudo to get the correct ownership/perms
+            installdir = get_bb_var('D', testrecipe)
+            fakerootenv = get_bb_var('FAKEROOTENV', testrecipe)
+            fakerootcmd = get_bb_var('FAKEROOTCMD', testrecipe)
+            result = runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir)
+            filelist1 = self._process_ls_output(result.output)
+
+            # Now look on the target
+            tempdir2 = tempfile.mkdtemp(prefix='devtoolqa')
+            self.track_for_cleanup(tempdir2)
+            tmpfilelist = os.path.join(tempdir2, 'files.txt')
+            with open(tmpfilelist, 'w') as f:
+                for line in filelist1:
+                    splitline = line.split()
+                    f.write(splitline[-1] + '\n')
+            result = runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, qemu.ip))
+            filelist2 = self._process_ls_output(result.output)
+            filelist1.sort(key=lambda item: item.split()[-1])
+            filelist2.sort(key=lambda item: item.split()[-1])
+            self.assertEqual(filelist1, filelist2)
+            # Test undeploy-target
+            result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip))
+            result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand), ignore_status=True)
+            self.assertNotEqual(result, 0, 'undeploy-target did not remove command as it should have')
Signature · 1 line
2.1.0