Some new runtime tests or fixes
cover Some new runtime tests or fixes
Hello,
This set includes new runtime tests and required changes for them.
Not all of them will get run by default, for example all the project build tests on target (iptables, cvs and sudoku) need to be enabled manually. Same applies to tests which require special options (skeleton, logrotate, pam). A new patch for AutoBuilder is needed to add a new target for these. The new smart tests are the most important change here, (and those will get run by default on all targets on AB).
Cheers,
Stefan
The following changes since commit 199b1a8c7d7109d9b59d67bf4dcfeb03be07587a:
mkfontscale: This no longer needs a full libx11, xproto suffices (2013-08-22 18:30:06 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib stefans/stage-oeqa1
for you to fetch changes up to 6d7bb8409eb99b415d2500c5146a6b0c7eac2e7f:
lib/oeqa: add a restart method for base class and use it for build tests (2013-08-23 15:14:43 +0300)
----------------------------------------------------------------
Alexandru Georgescu (1):
lib/oeqa/runtime: add basic test for x32 images
Alexandru Palalau (4):
lib/oeqa/runtime: add new logrotate test
lib/oeqa/runtime: add new skeletoninit test
lib/oeqa/runtime: add new PAM support test
lib/oeqa/runtime: add new scp test
Cornel Stoicescu (2):
lib/oeqa/runtime: add test for perl
lib/oeqa/runtime: add a test for ldd
Mihai Lindner (4):
lib/oeqa/utils: new file: httpserver.py useful for serving files over HTTP to the target lib/oeqa/utils: qemurunner: save host IP address
oeqa/utils/decorators: return the decorated method
lib/oeqa/runtime: smart: add new smart tests
Mihai Prica (4):
lib/oeqa/utils: targetbuild: Add helper class for building packages on target lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target lib/oeqa/runtime: add adjust date and time test lib/oeqa/runtime: add vncserver for target test
Stefan Stanacar (6):
classess/testimage: change default test suites oeqa/utils/targetbuild: change download to use bitbake's fetcher lib/oeqa/runtime: rpm: add install and erase tests lib/oeqa: change behaviour for unskippable tests lib/oeqa/utils: qemurunner: improve kill and restart lib/oeqa: add a restart method for base class and use it for build tests
meta/classes/testimage.bbclass | 4 +- meta/lib/oeqa/oetest.py | 11 +++- meta/lib/oeqa/runtime/buildcvs.py | 32 +++++++++++ meta/lib/oeqa/runtime/buildiptables.py | 32 +++++++++++ meta/lib/oeqa/runtime/buildsudoku.py | 29 ++++++++++ meta/lib/oeqa/runtime/date.py | 27 +++++++++ meta/lib/oeqa/runtime/files/test.pl | 2 + meta/lib/oeqa/runtime/ldd.py | 19 ++++++ meta/lib/oeqa/runtime/logrotate.py | 27 +++++++++ meta/lib/oeqa/runtime/pam.py | 24 ++++++++ meta/lib/oeqa/runtime/perl.py | 28 +++++++++ meta/lib/oeqa/runtime/rpm.py | 30 +++++++++- meta/lib/oeqa/runtime/scp.py | 22 +++++++ meta/lib/oeqa/runtime/smart.py | 102 +++++++++++++++++++++++++++++---- meta/lib/oeqa/runtime/vnc.py | 22 +++++++ meta/lib/oeqa/runtime/x32lib.py | 17 ++++++ meta/lib/oeqa/skeletoninit.py | 28 +++++++++ meta/lib/oeqa/utils/decorators.py | 6 +- meta/lib/oeqa/utils/httpserver.py | 32 +++++++++++ meta/lib/oeqa/utils/qemurunner.py | 39 ++++++++----- meta/lib/oeqa/utils/targetbuild.py | 63 ++++++++++++++++++++ 21 files changed, 561 insertions(+), 35 deletions(-) create mode 100644 meta/lib/oeqa/runtime/buildcvs.py create mode 100644 meta/lib/oeqa/runtime/buildiptables.py create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py create mode 100644 meta/lib/oeqa/runtime/date.py create mode 100644 meta/lib/oeqa/runtime/files/test.pl create mode 100644 meta/lib/oeqa/runtime/ldd.py create mode 100644 meta/lib/oeqa/runtime/logrotate.py create mode 100644 meta/lib/oeqa/runtime/pam.py create mode 100644 meta/lib/oeqa/runtime/perl.py create mode 100644 meta/lib/oeqa/runtime/scp.py create mode 100644 meta/lib/oeqa/runtime/vnc.py create mode 100644 meta/lib/oeqa/runtime/x32lib.py create mode 100644 meta/lib/oeqa/skeletoninit.py create mode 100644 meta/lib/oeqa/utils/httpserver.py create mode 100644 meta/lib/oeqa/utils/targetbuild.py
Alexandru Georgescu (1):
lib/oeqa/runtime: add basic test for x32 images
Alexandru Palalau (4):
lib/oeqa/runtime: add new logrotate test
lib/oeqa/runtime: add new skeletoninit test
lib/oeqa/runtime: add new PAM support test
lib/oeqa/runtime: add new scp test
Cornel Stoicescu (2):
lib/oeqa/runtime: add test for perl
lib/oeqa/runtime: add a test for ldd
Mihai Lindner (4):
lib/oeqa/utils: new file: httpserver.py useful for serving files over
HTTP to the target
lib/oeqa/utils: qemurunner: save host IP address oeqa/utils/decorators: return the decorated method lib/oeqa/runtime: smart: add new smart tests
Mihai Prica (4):
lib/oeqa/utils: targetbuild: Add helper class for building packages on
target
lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on
target
lib/oeqa/runtime: add adjust date and time test lib/oeqa/runtime: add vncserver for target test
Stefan Stanacar (6):
classess/testimage: change default test suites oeqa/utils/targetbuild: change download to use bitbake's fetcher lib/oeqa/runtime: rpm: add install and erase tests lib/oeqa: change behaviour for unskippable tests lib/oeqa/utils: qemurunner: improve kill and restart lib/oeqa: add a restart method for base class and use it for build
tests
meta/classes/testimage.bbclass | 4 +- meta/lib/oeqa/oetest.py | 11 +++- meta/lib/oeqa/runtime/buildcvs.py | 32 +++++++++++ meta/lib/oeqa/runtime/buildiptables.py | 32 +++++++++++ meta/lib/oeqa/runtime/buildsudoku.py | 29 ++++++++++ meta/lib/oeqa/runtime/date.py | 27 +++++++++ meta/lib/oeqa/runtime/files/test.pl | 2 + meta/lib/oeqa/runtime/ldd.py | 19 ++++++ meta/lib/oeqa/runtime/logrotate.py | 27 +++++++++ meta/lib/oeqa/runtime/pam.py | 24 ++++++++ meta/lib/oeqa/runtime/perl.py | 28 +++++++++ meta/lib/oeqa/runtime/rpm.py | 30 +++++++++- meta/lib/oeqa/runtime/scp.py | 22 +++++++ meta/lib/oeqa/runtime/smart.py | 102 +++++++++++++++++++++++++++++---- meta/lib/oeqa/runtime/vnc.py | 22 +++++++ meta/lib/oeqa/runtime/x32lib.py | 17 ++++++ meta/lib/oeqa/skeletoninit.py | 28 +++++++++ meta/lib/oeqa/utils/decorators.py | 6 +- meta/lib/oeqa/utils/httpserver.py | 32 +++++++++++ meta/lib/oeqa/utils/qemurunner.py | 39 ++++++++----- meta/lib/oeqa/utils/targetbuild.py | 63 ++++++++++++++++++++ 21 files changed, 561 insertions(+), 35 deletions(-) create mode 100644 meta/lib/oeqa/runtime/buildcvs.py create mode 100644 meta/lib/oeqa/runtime/buildiptables.py create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py create mode 100644 meta/lib/oeqa/runtime/date.py create mode 100644 meta/lib/oeqa/runtime/files/test.pl create mode 100644 meta/lib/oeqa/runtime/ldd.py create mode 100644 meta/lib/oeqa/runtime/logrotate.py create mode 100644 meta/lib/oeqa/runtime/pam.py create mode 100644 meta/lib/oeqa/runtime/perl.py create mode 100644 meta/lib/oeqa/runtime/scp.py create mode 100644 meta/lib/oeqa/runtime/vnc.py create mode 100644 meta/lib/oeqa/runtime/x32lib.py create mode 100644 meta/lib/oeqa/skeletoninit.py create mode 100644 meta/lib/oeqa/utils/httpserver.py create mode 100644 meta/lib/oeqa/utils/targetbuild.py
Signature · 1 line
1.8.3.1
PATCH 1/21 lib/oeqa/runtime: add basic test for x32 images
From: Alexandru Georgescu <[email protected]>
Checks that an x86-64-x32 image has the right binaries.
--- meta/lib/oeqa/runtime/x32lib.py | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 meta/lib/oeqa/runtime/x32lib.py
meta/lib/oeqa/runtime/x32lib.py new file +17 -0
@@ -0,0 +1,17 @@ +import unittest +from oeqa.oetest import oeRuntimeTest, skipModule +from oeqa.utils.decorators import * + +def setUpModule(): + #check if DEFAULTTUNE is set and it's value is: x86-64-x32 + defaulttune = oeRuntimeTest.tc.d.getVar("DEFAULTTUNE", True) + if "x86-64-x32" not in defaulttune: + skipModule("DEFAULTTUNE is not set to x86-64-x32") + +class X32libTest(oeRuntimeTest): + + @skipUnlessPassed("test_ssh") + def test_x32_file(self): + status1 = self.target.run("readelf -h /bin/ls | grep Class | grep ELF32")[0] + status2 = self.target.run("readelf -h /bin/ls | grep Machine | grep X86-64")[0] + self.assertTrue(status1 == 0 and status2 == 0, msg="/bin/ls isn't an X86-64 ELF32 binary. readelf says: %s" % self.target.run("readelf -h /bin/ls")[1])
Signature · 1 line
1.8.3.1
PATCH 2/21 lib/oeqa/runtime: add test for perl
In reply to Stefan StanacarFrom: Cornel Stoicescu <[email protected]>
This test runs a perl script on the target and checks the output.
--- meta/lib/oeqa/runtime/files/test.pl | 2 ++ meta/lib/oeqa/runtime/perl.py | 28 ++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 meta/lib/oeqa/runtime/files/test.pl create mode 100644 meta/lib/oeqa/runtime/perl.py
meta/lib/oeqa/runtime/files/test.pl new file +2 -0
@@ -0,0 +1,2 @@ +$a = 9.01e+21 - 9.01e+21 + 0.01; +print ("the value of a is ", $a, "\n");
meta/lib/oeqa/runtime/perl.py new file +28 -0
@@ -0,0 +1,28 @@ +import unittest +import os +from oeqa.oetest import oeRuntimeTest, skipModule +from oeqa.utils.decorators import * + +def setUpModule(): + if not oeRuntimeTest.hasPackage("perl"): + skipModule("No perl package in the image") + + +class PerlTest(oeRuntimeTest): + + @classmethod + def setUpClass(self): + oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.pl"), "/tmp/test.pl") + + def test_perl_exists(self): + (status, output) = self.target.run('which perl') + self.assertEqual(status, 0, msg="Perl binary not in PATH or not on target.") + + def test_perl_works(self): + (status, output) = self.target.run('perl /tmp/test.pl') + self.assertEqual(status, 0, msg="Exit status was not 0. Output: %s" % output) + self.assertEqual(output, "the value of a is 0.01", msg="Incorrect output: %s" % output) + + @classmethod + def tearDownClass(self): + oeRuntimeTest.tc.target.run("rm /tmp/test.pl")
Signature · 1 line
1.8.3.1
PATCH 3/21 lib/oeqa/runtime: add a test for ldd
In reply to Stefan StanacarFrom: Cornel Stoicescu <[email protected]>
This test checks that at least one path in RTLDLIST exists.
--- meta/lib/oeqa/runtime/ldd.py | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 meta/lib/oeqa/runtime/ldd.py
meta/lib/oeqa/runtime/ldd.py new file +19 -0
@@ -0,0 +1,19 @@ +import unittest +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * + +def setUpModule(): + if not oeRuntimeTest.hasFeature("tools-sdk"): + skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES") + +class LddTest(oeRuntimeTest): + + @skipUnlessPassed('test_ssh') + def test_ldd_exists(self): + (status, output) = self.target.run('which ldd') + self.assertEqual(status, 0, msg = "ldd does not exist in PATH !") + + @skipUnlessPassed('test_ldd_exists') + def test_ldd(self): + (status, output) = self.target.run('for i in $(which ldd | xargs cat | grep "^RTLDLIST"|cut -d\'=\' -f2|tr -d \'"\'); do test -f $i && echo $i && break; done') + self.assertEqual(status, 0, msg = "ldd path not correct or RTLDLIST files don't exist. ")
Signature · 1 line
1.8.3.1
PATCH 4/21 lib/oeqa/runtime: add new logrotate test
In reply to Stefan StanacarFrom: Alexandru Palalau <[email protected]>
New logrotate test which verifies the log directory change in logrotate.conf. Needs an image with logrotate installed.
--- meta/lib/oeqa/runtime/logrotate.py | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 meta/lib/oeqa/runtime/logrotate.py
meta/lib/oeqa/runtime/logrotate.py new file +27 -0
@@ -0,0 +1,27 @@ +# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=289 testcase +# Note that the image under test must have logrotate installed + +import unittest +from oeqa.oetest import oeRuntimeTest, skipModule +from oeqa.utils.decorators import * + +def setUpModule(): + if not oeRuntimeTest.hasPackage("logrotate"): + skipModule("No logrotate package in image") + + +class LogrotateTest(oeRuntimeTest): + + @skipUnlessPassed("test_ssh") + def test_1_logrotate_setup(self): + (status, output) = self.target.run('mkdir /home/root/logrotate_dir') + self.assertEqual(status, 0, msg = "Could not create logrotate_dir. Output: %s" % output) + (status, output) = self.target.run("sed -i 's#wtmp {#wtmp {\\n olddir /home/root/logrotate_dir#' /etc/logrotate.conf") + self.assertEqual(status, 0, msg = "Could not write to logrotate.conf file. Status and output: %s and %s)" % (status, output)) + + @skipUnlessPassed("test_1_logrotate_setup") + def test_2_logrotate(self): + (status, output) = self.target.run('logrotate -f /etc/logrotate.conf') + self.assertEqual(status, 0, msg = "logrotate service could not be reloaded. Status and output: %s and %s" % (status, output)) + output = self.target.run('ls -la /home/root/logrotate_dir/ | wc -l')[1] + self.assertTrue(int(output)>=3, msg = "new logfile could not be created. List of files within log directory: %s" %(self.target.run('ls -la /home/root/logrotate_dir')[1]))
Signature · 1 line
1.8.3.1
PATCH 5/21 lib/oeqa/runtime: add new skeletoninit test
In reply to Stefan StanacarFrom: Alexandru Palalau <[email protected]>
New test which verifies the usage of skeleton init script available with meta-skeleton layer
--- meta/lib/oeqa/skeletoninit.py | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 meta/lib/oeqa/skeletoninit.py
meta/lib/oeqa/skeletoninit.py new file +28 -0
@@ -0,0 +1,28 @@ +# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=284 testcase +# Note that the image under test must have meta-skeleton layer in bblayers and IMAGE_INSTALL_append = " service" in local.conf + +import unittest +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * + +def setUpModule(): + if not oeRuntimeTest.hasPackage("service"): + skipModule("No service package in image") + + +class SkeletonBasicTest(oeRuntimeTest): + + @skipUnlessPassed('test_ssh') + @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"), "Not appropiate for systemd image") + def test_skeleton_availability(self): + (status, output) = self.target.run('ls /etc/init.d/skeleton') + self.assertEqual(status, 0, msg = "skeleton init script not found. Output:\n%s " % output) + (status, output) = self.target.run('ls /usr/sbin/skeleton-test') + self.assertEqual(status, 0, msg = "skeleton-test not found. Output:\n%s" % output) + + @skipUnlessPassed('test_skeleton_availability') + @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"), "Not appropiate for systemd image") + def test_skeleton_script(self): + output1 = self.target.run("/etc/init.d/skeleton start")[1] + (status, output2) = self.target.run(oeRuntimeTest.pscmd + ' | grep [s]keleton-test') + self.assertEqual(status, 0, msg = "Skeleton script could not be started:\n%s\n%s" % (output1, output2))
Signature · 1 line
1.8.3.1
PATCH 6/21 lib/oeqa/runtime: add new PAM support test
In reply to Stefan StanacarFrom: Alexandru Palalau <[email protected]>
New test which verifies some usual commands functionality with PAM support
--- meta/lib/oeqa/runtime/pam.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 meta/lib/oeqa/runtime/pam.py
meta/lib/oeqa/runtime/pam.py new file +24 -0
@@ -0,0 +1,24 @@ +# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=287 testcase +# Note that the image under test must have "pam" in DISTRO_FEATURES + +import unittest +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * + +def setUpModule(): + if not oeRuntimeTest.hasFeature("pam"): + skipModule("target doesn't have 'pam' in DISTRO_FEATURES") + + +class PamBasicTest(oeRuntimeTest): + + @skipUnlessPassed('test_ssh') + def test_pam(self): + (status, output) = self.target.run('login --help') + self.assertEqual(status, 1, msg = "login command does not work as expected. Status and output:%s and %s" %(status, output)) + (status, output) = self.target.run('passwd --help') + self.assertEqual(status, 6, msg = "passwd command does not work as expected. Status and output:%s and %s" %(status, output)) + (status, output) = self.target.run('su --help') + self.assertEqual(status, 2, msg = "su command does not work as expected. Status and output:%s and %s" %(status, output)) + (status, output) = self.target.run('useradd --help') + self.assertEqual(status, 2, msg = "useradd command does not work as expected. Status and output:%s and %s" %(status, output))
Signature · 1 line
1.8.3.1
PATCH 7/21 lib/oeqa/runtime: add new scp test
In reply to Stefan StanacarFrom: Alexandru Palalau <[email protected]>
Copies a 5MB to target using scp, more of an
network test than a scp one.
--- meta/lib/oeqa/runtime/scp.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 meta/lib/oeqa/runtime/scp.py
meta/lib/oeqa/runtime/scp.py new file +22 -0
@@ -0,0 +1,22 @@ +import subprocess +import unittest +import os +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * + +def setUpModule(): + if not (oeRuntimeTest.hasPackage("dropbear") or oeRuntimeTest.hasPackage("openssh-sshd")): + skipModule("No ssh package in image") + + +class ScpTest(oeRuntimeTest): + + def setUp(self): + subprocess.check_call("dd if=/dev/zero of=%s bs=512k count=10" % os.path.join(oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR", True), 'test_scp_file'), shell=True) + + @skipUnlessPassed('test_ssh') + def test_scp(self): + (status, output) = self.target.copy_to(os.path.join(oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR", True), 'test_scp_file'), '/tmp/test_scp_file') + self.assertEqual(status, 0, msg = "File could not be copied. Output: %s" % output) + (status, output) = self.target.run("ls -la /tmp/test_scp_file") + self.assertEqual(status, 0, msg = "SCP test failed")
Signature · 1 line
1.8.3.1
PATCH 8/21 lib/oeqa/utils: new file: httpserver.py useful for serving files over HTTP to the target
In reply to Stefan StanacarFrom: Mihai Lindner <[email protected]>
It can be used by smart repo/channel tests to serve deploy_dir.
--- meta/lib/oeqa/utils/httpserver.py | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 meta/lib/oeqa/utils/httpserver.py
meta/lib/oeqa/utils/httpserver.py new file +32 -0
@@ -0,0 +1,32 @@ +import SimpleHTTPServer +import multiprocessing +import os + +class HTTPServer(SimpleHTTPServer.BaseHTTPServer.HTTPServer): + + def server_start(self, root_dir): + os.chdir(root_dir) + self.serve_forever() + +class HTTPRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler): + + def log_message(self, format_str, *args): + pass + +class HTTPService(object): + + def __init__(self, root_dir): + self.root_dir = root_dir + self.port = 0 + + def start(self): + self.server = HTTPServer(('', self.port), HTTPRequestHandler) + if self.port == 0: + self.port = self.server.server_port + self.process = multiprocessing.Process(target=self.server.server_start, args=[self.root_dir]) + self.process.start() + + def stop(self): + self.server.server_close() + self.process.terminate() + self.process.join()
Signature · 1 line
1.8.3.1
PATCH 9/21 lib/oeqa/utils: qemurunner: save host IP address
In reply to Stefan StanacarFrom: Mihai Lindner <[email protected]>
Save host IP address to host_ip.
Read /proc/PID/cmdline on host to look for IPs of target and host; instead of running 'ps'.
Also removed some extra empty lines from file.
--- meta/lib/oeqa/utils/qemurunner.py | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +3 -8
@@ -54,7 +54,6 @@ class QemuRunner: def launch(self, qemuparams = None): - if self.display: os.environ["DISPLAY"] = self.display else:
@@ -84,12 +83,11 @@ class QemuRunner: if self.is_alive(): bb.note("qemu started - qemu procces pid is %s" % self.qemupid) - pscmd = 'ps -p %s -fww | grep -o "192\.168\.7\.[0-9]*::" | awk -F":" \'{print $1}\'' % self.qemupid - self.ip = subprocess.Popen(pscmd,shell=True,stdout=subprocess.PIPE).communicate()[0].strip() + cmdline = open('/proc/%s/cmdline' % self.qemupid).read() + self.ip, _, self.host_ip = cmdline.split('ip=')[1].split(' ')[0].split(':')[0:3] if not re.search("^((?:[0-9]{1,3}\.){3}[0-9]{1,3})$", self.ip): bb.note("Couldn't get ip from qemu process arguments, I got '%s'" % self.ip) - bb.note("Here is the ps output:\n%s" % \ - subprocess.Popen("ps -p %s -fww" % self.qemupid,shell=True,stdout=subprocess.PIPE).communicate()[0]) + bb.note("Here is the ps output:\n%s" % cmdline) self.kill() return False bb.note("IP found: %s" % self.ip)
@@ -122,7 +120,6 @@ class QemuRunner: sock.close() stopread = True - if not reachedlogin: bb.note("Target didn't reached login boot in %d seconds" % self.boottime) lines = "\n".join(self.bootlog.splitlines()[-5:])
@@ -139,7 +136,6 @@ class QemuRunner: return self.is_alive() - def kill(self): if self.server_socket: self.server_socket.close()
@@ -207,4 +203,3 @@ class QemuRunner: basecmd = os.path.basename(basecmd) if "qemu-system" in basecmd and "-serial tcp" in commands[p]: return [int(p),commands[p]] -
Signature · 1 line
1.8.3.1
PATCH 10/21 oeqa/utils/decorators: return the decorated method
In reply to Stefan StanacarFrom: Mihai Lindner <[email protected]>
Decorators should return whatever the decorated methods return.
--- meta/lib/oeqa/utils/decorators.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/utils/decorators.py +3 -3
@@ -17,7 +17,7 @@ class skipIfFailure(object): def wrapped_f(*args): if self.testcase in (oeRuntimeTest.testFailures or oeRuntimeTest.testErrors): raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase) - f(*args) + return f(*args) wrapped_f.__name__ = f.__name__ return wrapped_f
@@ -30,7 +30,7 @@ class skipIfSkipped(object): def wrapped_f(*args): if self.testcase in oeRuntimeTest.testSkipped: raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase) - f(*args) + return f(*args) wrapped_f.__name__ = f.__name__ return wrapped_f
@@ -45,6 +45,6 @@ class skipUnlessPassed(object): self.testcase in oeRuntimeTest.testFailures or \ self.testcase in oeRuntimeTest.testErrors: raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase) - f(*args) + return f(*args) wrapped_f.__name__ = f.__name__ return wrapped_f
Signature · 1 line
1.8.3.1
PATCH 11/21 lib/oeqa/runtime: smart: add new smart tests
In reply to Stefan StanacarFrom: Mihai Lindner <[email protected]>
Add class to be inherited by smart tests, along with more basic tests and tests using a rpm repository.
--- meta/lib/oeqa/runtime/smart.py | 102 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 91 insertions(+), 11 deletions(-)
meta/lib/oeqa/runtime/smart.py +91 -11
@@ -1,6 +1,7 @@ import unittest from oeqa.oetest import oeRuntimeTest from oeqa.utils.decorators import * +from oeqa.utils.httpserver import HTTPService def setUpModule(): if not oeRuntimeTest.hasFeature("package-management"):
@@ -8,21 +9,100 @@ def setUpModule(): if not oeRuntimeTest.hasPackage("smart"): skipModule("Image doesn't have smart installed") -class SmartHelpTest(oeRuntimeTest): +class SmartTest(oeRuntimeTest): + + longMessage = True + + @skipUnlessPassed('test_smart_help') + def smart(self, command, expected = 0): + command = 'smart %s' % command + status, output = self.target.run(command) + message = os.linesep.join([command, output]) + self.assertEqual(status, expected, message) + return output + +class SmartBasicTest(SmartTest): @skipUnlessPassed('test_ssh') def test_smart_help(self): - status = self.target.run('smart --help')[0] - self.assertEqual(status, 0) + self.smart('--help') -class SmartQueryTest(oeRuntimeTest): + def test_smart_version(self): + self.smart('--version') + + def test_smart_info(self): + self.smart('info python-smartpm') - @skipUnlessPassed('test_smart_help') def test_smart_query(self): - (status, output) = self.target.run('smart query rpm') - self.assertEqual(status, 0, msg="smart query failed, output: %s" % output) + self.smart('query python-smartpm') - @skipUnlessPassed('test_smart_query') - def test_smart_info(self): - (status, output) = self.target.run('smart info rpm') - self.assertEqual(status, 0, msg="smart info rpm failed, output: %s" % output) + def test_smart_search(self): + self.smart('search python-smartpm') + + def test_smart_stats(self): + self.smart('stats') + +class SmartRepoTest(SmartTest): + + @classmethod + def setUpClass(self): + self.repo_server = HTTPService(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True)) + self.repo_server.start() + + @classmethod + def tearDownClass(self): + self.repo_server.stop() + + def test_smart_channel(self): + self.smart('channel', 1) + + def test_smart_channel_add(self): + image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE', True) + deploy_url = 'http://%s:%s/%s' %(self.tc.qemu.host_ip, self.repo_server.port, image_pkgtype) + for arch in os.listdir('%s/%s' % (self.repo_server.root_dir, image_pkgtype)): + self.smart('channel -y --add {a} type=rpm-md baseurl={u}/{a}'.format(a=arch, u=deploy_url)) + self.smart('update') + + def test_smart_channel_help(self): + self.smart('channel --help') + + def test_smart_channel_list(self): + self.smart('channel --list') + + def test_smart_channel_show(self): + self.smart('channel --show') + + def test_smart_channel_rpmsys(self): + self.smart('channel --show rpmsys') + self.smart('channel --disable rpmsys') + self.smart('channel --enable rpmsys') + + @skipUnlessPassed('test_smart_channel_add') + def test_smart_install(self): + self.smart('remove -y psplash-default') + self.smart('install -y psplash-default') + + @skipUnlessPassed('test_smart_install') + def test_smart_install_dependency(self): + self.smart('remove -y psplash') + self.smart('install -y psplash-default') + + @skipUnlessPassed('test_smart_channel_add') + def test_smart_install_from_disk(self): + self.smart('remove -y psplash-default') + self.smart('download psplash-default') + self.smart('install -y ./psplash-default*') + + @skipUnlessPassed('test_smart_channel_add') + def test_smart_install_from_http(self): + url = 'http://' + output = self.smart('download --urls psplash-default') + for line in output.splitlines(): + if line.startswith(url): + url = line + self.smart('remove -y psplash-default') + self.smart('install -y %s' % url) + + @skipUnlessPassed('test_smart_install') + def test_smart_reinstall(self): + self.smart('reinstall -y psplash-default')
Signature · 1 line
1.8.3.1
PATCH 12/21 lib/oeqa/utils: targetbuild: Add helper class for building packages on target
In reply to Stefan StanacarFrom: Mihai Prica <[email protected]>
This class can be used for test cases that configure and build packages on target.
--- meta/lib/oeqa/utils/targetbuild.py | 66 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 meta/lib/oeqa/utils/targetbuild.py
meta/lib/oeqa/utils/targetbuild.py new file +66 -0
@@ -0,0 +1,66 @@ +# Copyright (C) 2013 Intel Corporation +# +# Released under the MIT license (see COPYING.MIT) + +# Provides a class for automating build tests for projects + +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * +import bb.process +import os +import re + + +class TargetBuildProject(): + + def __init__(self, target, uri, foldername=None): + self.target = target + self.uri = uri + self.targetdir = "/home/root/" + + if not foldername: + self.archive = os.path.basename(uri) + self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive) + else: + self.fname = foldername + + def download_archive(self): + wgetcmd = oeRuntimeTest.tc.d.getVar('FETCHCMD_wget', True).split() + self.testdir = oeRuntimeTest.tc.d.getVar('TEST_LOG_DIR', True) + + try: + output = bb.process.run(wgetcmd + ['-P', self.testdir, self.uri])[0] + except bb.process.CmdError: + raise Exception("Failed to download archive, output: %s" % output) + + (status, output) = oeRuntimeTest.tc.target.copy_to( + os.path.join(self.testdir, self.archive), + self.targetdir) + if status != 0: + raise Exception("Failed to copy archive to target, output: %s" % output) + + (status, output) = oeRuntimeTest.tc.target.run( + 'tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir)) + if status != 0: + raise Exception("Failed to extract archive, output: %s" % output) + + #Change targetdir to project folder + self.targetdir = self.targetdir + self.fname + + # The timeout parameter of target.run is set to 0 to make the ssh command + # run with no timeout. + def run_configure(self): + return self.target.run('cd %s; ./configure' % self.targetdir, 0)[0] + + def run_make(self): + return self.target.run('cd %s; make' % self.targetdir, 0)[0] + + def run_install(self): + return self.target.run('cd %s; make install' % self.targetdir, 0)[0] + + def clean(self): + self.target.run('rm -r %s*' % self.targetdir) + try: + bb.process.run(['rm', '-rf', os.path.join(self.testdir, self.archive)]) + except bb.process.CmdError: + bb.note("Failed to remove archive")
Signature · 1 line
1.8.3.1
PATCH 13/21 lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target
In reply to Stefan StanacarFrom: Mihai Prica <[email protected]>
Downloads iptables/cvs/sudoku-savant sources and builds them on target.
--- meta/lib/oeqa/runtime/buildcvs.py | 30 ++++++++++++++++++++++++++++++ meta/lib/oeqa/runtime/buildiptables.py | 30 ++++++++++++++++++++++++++++++ meta/lib/oeqa/runtime/buildsudoku.py | 27 +++++++++++++++++++++++++++ 3 files changed, 87 insertions(+) create mode 100644 meta/lib/oeqa/runtime/buildcvs.py create mode 100644 meta/lib/oeqa/runtime/buildiptables.py create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py
meta/lib/oeqa/runtime/buildcvs.py new file +30 -0
@@ -0,0 +1,30 @@ +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * +from oeqa.utils.targetbuild import TargetBuildProject + +def setUpModule(): + if not oeRuntimeTest.hasFeature("tools-sdk"): + skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES") + +class BuildCvsTest(oeRuntimeTest): + + @classmethod + def setUpClass(self): + self.project = TargetBuildProject(oeRuntimeTest.tc.target, + "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2") + self.project.download_archive() + + @skipUnlessPassed("test_ssh") + def test_cvs(self): + self.assertEqual(self.project.run_configure(), 0, + msg="Running configure failed") + + self.assertEqual(self.project.run_make(), 0, + msg="Running make failed") + + self.assertEqual(self.project.run_install(), 0, + msg="Running make install failed") + + @classmethod + def tearDownClass(self): + self.project.clean()
meta/lib/oeqa/runtime/buildiptables.py new file +30 -0
@@ -0,0 +1,30 @@ +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * +from oeqa.utils.targetbuild import TargetBuildProject + +def setUpModule(): + if not oeRuntimeTest.hasFeature("tools-sdk"): + skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES") + +class BuildIptablesTest(oeRuntimeTest): + + @classmethod + def setUpClass(self): + self.project = TargetBuildProject(oeRuntimeTest.tc.target, + "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2") + self.project.download_archive() + + @skipUnlessPassed("test_ssh") + def test_iptables(self): + self.assertEqual(self.project.run_configure(), 0, + msg="Running configure failed") + + self.assertEqual(self.project.run_make(), 0, + msg="Running make failed") + + self.assertEqual(self.project.run_install(), 0, + msg="Running make install failed") + + @classmethod + def tearDownClass(self): + self.project.clean()
meta/lib/oeqa/runtime/buildsudoku.py new file +27 -0
@@ -0,0 +1,27 @@ +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * +from oeqa.utils.targetbuild import TargetBuildProject + +def setUpModule(): + if not oeRuntimeTest.hasFeature("tools-sdk"): + skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES") + +class SudokuTest(oeRuntimeTest): + + @classmethod + def setUpClass(self): + self.project = TargetBuildProject(oeRuntimeTest.tc.target, + "http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2") + self.project.download_archive() + + @skipUnlessPassed("test_ssh") + def test_sudoku(self): + self.assertEqual(self.project.run_configure(), 0, + msg="Running configure failed") + + self.assertEqual(self.project.run_make(), 0, + msg="Running make failed") + + @classmethod + def tearDownClass(self): + self.project.clean()
Signature · 1 line
1.8.3.1
PATCH 14/21 lib/oeqa/runtime: add adjust date and time test
In reply to Stefan StanacarFrom: Mihai Prica <[email protected]>
--- meta/lib/oeqa/runtime/date.py | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 meta/lib/oeqa/runtime/date.py
meta/lib/oeqa/runtime/date.py new file +27 -0
@@ -0,0 +1,27 @@ +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * +import re + +def setUpModule(): + skipModuleUnless(oeRuntimeTest.tc.target.run('which date')[0] == 0, "No date in image or no connection") + +class DateTest(oeRuntimeTest): + + def setUp(self): + (status, output) = self.target.run('date +"%Y-%m-%d %T"') + self.assertEqual(status, 0, msg="Failed to get initial date, output: %s" % output) + self.oldDate = output + + @skipUnlessPassed("test_ssh") + def test_date(self): + sampleDate = '"2016-08-09 10:00:00"' + (status, output) = self.target.run("date -s %s" % sampleDate) + self.assertEqual(status, 0, msg="Date set failed, output: %s" % output) + + (status, output) = self.target.run("date -R") + p = re.match('Tue, 09 Aug 2016 10:00:.. \+0000', output) + self.assertTrue(p, msg="The date was not set correctly, output: %s" % output) + + def tearDown(self): + (status, output) = self.target.run('date -s "%s"' % self.oldDate) + self.assertEqual(status, 0, msg="Failed to reset date, output: %s" % output)
Signature · 1 line
1.8.3.1
PATCH 15/21 lib/oeqa/runtime: add vncserver for target test
In reply to Stefan StanacarFrom: Mihai Prica <[email protected]>
--- meta/lib/oeqa/runtime/vnc.py | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 meta/lib/oeqa/runtime/vnc.py
meta/lib/oeqa/runtime/vnc.py new file +22 -0
@@ -0,0 +1,22 @@ +from oeqa.oetest import oeRuntimeTest +from oeqa.utils.decorators import * +import re + +def setUpModule(): + skipModuleUnless(oeRuntimeTest.tc.target.run('which x11vnc')[0] == 0, "No x11vnc in image") + +class VNCTest(oeRuntimeTest): + + @skipUnlessPassed('test_ssh') + def test_vnc(self): + (status, output) = self.target.run('x11vnc -display :0.0 -bg -q') + self.assertEqual(status, 0, msg="x11vnc server failed to start: %s" % output) + port = re.search('PORT=[0-9]*', output) + self.assertTrue(port, msg="Listening port not specified in command output: %s" %output) + + (status, output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -i [x]11vnc') + self.assertEqual(status, 0, msg="x11vnc process not running") + + vncport = port.group(0).split('=')[1] + (status, output) = self.target.run('netstat -atun | grep :%s | grep LISTEN' % vncport) + self.assertEqual(status, 0, msg="x11vnc server not running on port %s" % vncport)
Signature · 1 line
1.8.3.1
PATCH 16/21 classess/testimage: change default test suites
In reply to Stefan StanacarSome new tests were added, safe to have them in the defaults for sato-sdk and sato. Not all of the new tests are here though, either because they aren't applicable to default images or take too long. (like build iptables/cvs/sudoky ones, they can be enabled
in local.conf and a special target on AB setups.). Also reorder them a bit.
--- meta/classes/testimage.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes/testimage.bbclass +2 -2
@@ -26,8 +26,8 @@ TEST_LOG_DIR ?= "${WORKDIR}/testimage" DEFAULT_TEST_SUITES = "ping auto" DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping" -DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh connman df rpm smart xorg syslog dmesg" -DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh connman df rpm smart gcc xorg syslog dmesg" +DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg" +DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp vnc date perl ldd gcc rpm smart dmesg" TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
Signature · 1 line
1.8.3.1
PATCH 17/21 oeqa/utils/targetbuild: change download to use bitbake's fetcher
In reply to Stefan StanacarUse bb.fetcher2 instead of running our own wget command (helps with proxy too). Also no need to use the class attribute target, use self.target, as the tests pass that to the class.
Also, we shouldn't clean the archive, now that it gets to DL_DIR.
--- meta/lib/oeqa/utils/targetbuild.py | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-)
meta/lib/oeqa/utils/targetbuild.py +15 -18
@@ -5,8 +5,8 @@ # Provides a class for automating build tests for projects from oeqa.oetest import oeRuntimeTest -from oeqa.utils.decorators import * -import bb.process +import bb.fetch2 +import bb.data import os import re
@@ -18,6 +18,9 @@ class TargetBuildProject(): self.uri = uri self.targetdir = "/home/root/" + self.localdata = bb.data.createCopy(oeRuntimeTest.tc.d) + bb.data.update_data(self.localdata) + if not foldername: self.archive = os.path.basename(uri) self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
@@ -25,22 +28,20 @@ class TargetBuildProject(): self.fname = foldername def download_archive(self): - wgetcmd = oeRuntimeTest.tc.d.getVar('FETCHCMD_wget', True).split() - self.testdir = oeRuntimeTest.tc.d.getVar('TEST_LOG_DIR', True) try: - output = bb.process.run(wgetcmd + ['-P', self.testdir, self.uri])[0] - except bb.process.CmdError: - raise Exception("Failed to download archive, output: %s" % output) - - (status, output) = oeRuntimeTest.tc.target.copy_to( - os.path.join(self.testdir, self.archive), - self.targetdir) + self.localdata.delVar("BB_STRICT_CHECKSUM") + fetcher = bb.fetch2.Fetch([self.uri], self.localdata) + fetcher.download() + self.localarchive = fetcher.localpath(self.uri) + except bb.fetch2.BBFetchException: + raise Exception("Failed to download archive: %s" % self.uri) + + (status, output) = self.target.copy_to(self.localarchive, self.targetdir) if status != 0: raise Exception("Failed to copy archive to target, output: %s" % output) - (status, output) = oeRuntimeTest.tc.target.run( - 'tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir)) + (status, output) = self.target.run('tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir)) if status != 0: raise Exception("Failed to extract archive, output: %s" % output)
@@ -59,8 +60,4 @@ class TargetBuildProject(): return self.target.run('cd %s; make install' % self.targetdir, 0)[0] def clean(self): - self.target.run('rm -r %s*' % self.targetdir) - try: - bb.process.run(['rm', '-rf', os.path.join(self.testdir, self.archive)]) - except bb.process.CmdError: - bb.note("Failed to remove archive") + self.target.run('rm -rf %s' % self.targetdir)
Signature · 1 line
1.8.3.1
Re: [PATCH 17/21] oeqa/utils/targetbuild: change download to use bitbake's fetcher
Hi Stefan,
Quoted 78 lines · On Friday 23 August 2013 18:30:57 Stefan Stanacar wrote:
Use bb.fetcher2 instead of running our own wget command (helps with proxy too). Also no need to use the class attribute target, use self.target, as the tests pass that to the class.
Also, we shouldn't clean the archive, now that it gets toDL_DIR.Signed-off-by Stefan Stanacar <[email protected]>--- meta/lib/oeqa/utils/targetbuild.py | 33 +++++++++++++++------------------ 1 file changed, 15 insertions(+), 18 deletions(-)(quoted diff) +0
-0
b/meta/lib/oeqa/utils/targetbuild.pyindex 7555add..9b2cf53 100644meta/lib/oeqa/utils/targetbuild.py +5
-5@@ -5,8 +5,8 @@ # Provides a class for automating build tests for projects from oeqa.oetest import oeRuntimeTest -from oeqa.utils.decorators import * -import bb.process +import bb.fetch2 +import bb.data import os import re@@ -18,6 +18,9 @@ class TargetBuildProject(): self.uri = uri self.targetdir = "/home/root/" + self.localdata = bb.data.createCopy(oeRuntimeTest.tc.d) + bb.data.update_data(self.localdata) + if not foldername: self.archive = os.path.basename(uri) self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)@@ -25,22 +28,20 @@ class TargetBuildProject(): self.fname = foldername def download_archive(self): - wgetcmd = oeRuntimeTest.tc.d.getVar('FETCHCMD_wget', True).split() - self.testdir = oeRuntimeTest.tc.d.getVar('TEST_LOG_DIR', True) try: - output = bb.process.run(wgetcmd + ['-P', self.testdir,self.uri])[0] - except bb.process.CmdError:
- raise Exception("Failed to download archive, output: %s" %
output) -meta/lib/oeqa/utils/targetbuild.py +8
-3- (status, output) = oeRuntimeTest.tc.target.copy_to( - os.path.join(self.testdir, self.archive), - self.targetdir) + self.localdata.delVar("BB_STRICT_CHECKSUM") + fetcher = bb.fetch2.Fetch([self.uri], self.localdata) + fetcher.download() + self.localarchive = fetcher.localpath(self.uri) + except bb.fetch2.BBFetchException: + raise Exception("Failed to download archive: %s" % self.uri) + + (status, output) = self.target.copy_to(self.localarchive,self.targetdir) if status != 0:
raise Exception("Failed to copy archive to target, output: %s"
% output)meta/lib/oeqa/utils/targetbuild.py +0
-2- (status, output) = oeRuntimeTest.tc.target.run( - 'tar xf %s%s -C %s' % (self.targetdir, self.archive,self.targetdir)) + (status, output) = self.target.run('tar xf %s%s
-C %s' % (self.targetdir, self.archive, self.targetdir)) if status != 0:
raise Exception("Failed to extract archive, output: %s" %
output)meta/lib/oeqa/utils/targetbuild.py +0
-0@@ -59,8 +60,4 @@ class TargetBuildProject(): return self.target.run('cd %s; make install' % self.targetdir,0)[0]
def clean(self):
meta/lib/oeqa/utils/targetbuild.py +0
-3- self.target.run('rm -r %s*' % self.targetdir) - try: - bb.process.run(['rm', '-rf', os.path.join(self.testdir,self.archive)]) - except bb.process.CmdError:
meta/lib/oeqa/utils/targetbuild.py class TargetBuildProject(): · line 63
Quoted 2 lines
meta/lib/oeqa/utils/targetbuild.py +1
-1- bb.note("Failed to remove archive") + self.target.run('rm -rf %s' % self.targetdir)
Could you please squash this into the commit that adds this module?
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
PATCH 18/21 lib/oeqa/runtime: rpm: add install and erase tests
In reply to Stefan StanacarCopies to target rpm-doc file from deploy_dir and tries to install and then remove that package. rpm-doc was chosen because it's small, it only adds a few files to target, and it's almost always found in deploy_dir for images with package-management/rpm.
--- meta/lib/oeqa/runtime/rpm.py | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-)
meta/lib/oeqa/runtime/rpm.py +27 -3
@@ -1,6 +1,8 @@ import unittest +import os from oeqa.oetest import oeRuntimeTest, skipModule from oeqa.utils.decorators import * +import oe.packagedata def setUpModule(): if not oeRuntimeTest.hasFeature("package-management"):
@@ -9,17 +11,39 @@ def setUpModule(): skipModule("rpm module skipped: target doesn't have rpm as primary package manager") -class RpmHelpTest(oeRuntimeTest): +class RpmBasicTest(oeRuntimeTest): @skipUnlessPassed('test_ssh') def test_rpm_help(self): (status, output) = self.target.run('rpm --help') self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output)) -class RpmQueryTest(oeRuntimeTest): - @skipUnlessPassed('test_rpm_help') def test_rpm_query(self): (status, output) = self.target.run('rpm -q rpm') self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output)) +class RpmInstallRemoveTest(oeRuntimeTest): + + @classmethod + def setUpClass(self): + deploydir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True), "rpm", oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True)) + pkgdata = oe.packagedata.read_subpkgdata("rpm-doc", oeRuntimeTest.tc.d) + # pick rpm-doc as a test file to get installed, because it's small and it will always be built for standard targets + testrpmfile = "rpm-doc-%s-%s.%s.rpm" % (pkgdata["PKGV"], pkgdata["PKGR"], oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True)) + oeRuntimeTest.tc.target.copy_to(os.path.join(deploydir,testrpmfile), "/tmp/rpm-doc.rpm") + + @skipUnlessPassed('test_rpm_help') + def test_rpm_install(self): + (status, output) = self.target.run('rpm -ivh /tmp/rpm-doc.rpm') + self.assertEqual(status, 0, msg="Failed to install rpm-doc package: %s" % output) + + @skipUnlessPassed('test_rpm_install') + def test_rpm_remove(self): + (status,output) = self.target.run('rpm -e rpm-doc') + self.assertEqual(status, 0, msg="Failed to remove rpm-doc package: %s" % output) + + @classmethod + def tearDownClass(self): + oeRuntimeTest.tc.target.run('rm -f /tmp/rpm-doc.rpm') +
Signature · 1 line
1.8.3.1
PATCH 19/21 lib/oeqa: change behaviour for unskippable tests
In reply to Stefan StanacarWhen a test module wants to be skipped because it doesn't apply to the image but it was nevertheless a required
test (one in TEST_SUITES), we issued an warning that it
was a required test and went on with running the module. Usually all tests in the module failed (e.g gcc tests on a non-sdk image), but this allowed us to know that something went wrong with the image (some package/feature didn't make it).
However, instead of just issuing an warning and running the tests it's better to throw an exception. The traceback will tell us what's wrong, and we don't run every single test method.
Output will look like this:
--snip--
| NOTE: Test modules ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
| NOTE: Found 5 tests
| test_ping (oeqa.runtime.ping.PingTest) ... ok
| test_ssh (oeqa.runtime.ssh.SshTest) ... ok
| ERROR
|
| ======================================================================
| ERROR: setUpModule (oeqa.runtime.gcc)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
| File "/mnt/back/yocto/poky/meta/lib/oeqa/runtime/gcc.py", line 8, in setUpModule
| skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
| File "/mnt/back/yocto/poky/meta/lib/oeqa/oetest.py", line 108, in skipModule
| "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
| Exception:
| Test gcc wants to be skipped.
| Reason is: Image doesn't have tools-sdk in IMAGE_FEATURES
| Test was required in TEST_SUITES, so either the condition for skipping is wrong
| or the image really doesn't have the requred feature/package when it should.
|
| ----------------------------------------------------------------------
| Ran 2 tests in 1.036s
|
| FAILED (errors=1)
| NOTE: Sending SIGTERM to runqemu--snip--
--- meta/lib/oeqa/oetest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/lib/oeqa/oetest.py +3 -1
@@ -96,7 +96,9 @@ def skipModule(reason, pos=2): if modname not in oeRuntimeTest.tc.testsrequired: raise unittest.SkipTest("%s: %s" % (modname, reason)) else: - bb.warn("Test %s is required, not skipping" % modname) + raise Exception("\nTest %s wants to be skipped.\nReason is: %s" \ + "\nTest was required in TEST_SUITES, so either the condition for skipping is wrong" \ + "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason)) def skipModuleIf(cond, reason):
Signature · 1 line
1.8.3.1
PATCH 20/21 lib/oeqa/utils: qemurunner: improve kill and restart
In reply to Stefan StanacarTweak QemuRunner so we can actually restart the qemu target in a test (if we want more memory for example).
--- meta/lib/oeqa/utils/qemurunner.py | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +21 -7
@@ -32,6 +32,10 @@ class QemuRunner: self.boottime = boottime self.runqemutime = runqemutime + self.create_socket() + + def create_socket(self): + self.bootlog = '' self.qemusock = None
@@ -137,21 +141,31 @@ class QemuRunner: return self.is_alive() def kill(self): + + if self.runqemu: + bb.note("Sending SIGTERM to runqemu") + os.kill(-self.runqemu.pid,signal.SIGTERM) + endtime = time.time() + self.runqemutime + while self.runqemu.poll() is None and time.time() < endtime: + time.sleep(1) + if self.runqemu.poll() is None: + bb.note("Sending SIGKILL to runqemu") + os.kill(-self.runqemu.pid,signal.SIGKILL) + self.runqemu = None if self.server_socket: self.server_socket.close() self.server_socket = None - if self.runqemu.pid: - os.kill(-self.runqemu.pid,signal.SIGTERM) - os.kill(-self.runqemu.pid,signal.SIGKILL) - self.runqemu.pid = None self.qemupid = None self.ip = None def restart(self, qemuparams = None): - if self.is_alive(): + bb.note("Restarting qemu process") + if self.runqemu.poll() is None: self.kill() - bb.note("Qemu Restart required...") - return self.launch(qemuparams) + self.create_socket() + if self.launch(qemuparams): + return True + return False def is_alive(self): qemu_child = self.find_child(str(self.runqemu.pid))
Signature · 1 line
1.8.3.1
PATCH 21/21 lib/oeqa: add a restart method for base class and use it for build tests
In reply to Stefan StanacarThis can be used by tests to restart the target (useful for passing extra qemuparams, like more RAM)
Build projects like cvs/iptables can take 15min+ on qemu targets with the default RAM, and sudoku fails with out of memory, so use this there.
--- meta/lib/oeqa/oetest.py | 7 +++++++ meta/lib/oeqa/runtime/buildcvs.py | 2 ++ meta/lib/oeqa/runtime/buildiptables.py | 2 ++ meta/lib/oeqa/runtime/buildsudoku.py | 2 ++ 4 files changed, 13 insertions(+)
meta/lib/oeqa/oetest.py +7 -0
@@ -81,6 +81,13 @@ class oeRuntimeTest(unittest.TestCase): else: return False + @classmethod + def restartTarget(self,params=None): + + if oeRuntimeTest.tc.qemu.restart(params): + oeRuntimeTest.tc.target.host = oeRuntimeTest.tc.qemu.ip + else: + raise Exception("Restarting target failed") def getmodule(pos=2):
meta/lib/oeqa/runtime/buildcvs.py +2 -0
@@ -10,6 +10,7 @@ class BuildCvsTest(oeRuntimeTest): @classmethod def setUpClass(self): + self.restartTarget("-m 512") self.project = TargetBuildProject(oeRuntimeTest.tc.target, "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2") self.project.download_archive()
@@ -28,3 +29,4 @@ class BuildCvsTest(oeRuntimeTest): @classmethod def tearDownClass(self): self.project.clean() + self.restartTarget()
meta/lib/oeqa/runtime/buildiptables.py +2 -0
@@ -10,6 +10,7 @@ class BuildIptablesTest(oeRuntimeTest): @classmethod def setUpClass(self): + self.restartTarget("-m 512") self.project = TargetBuildProject(oeRuntimeTest.tc.target, "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2") self.project.download_archive()
@@ -28,3 +29,4 @@ class BuildIptablesTest(oeRuntimeTest): @classmethod def tearDownClass(self): self.project.clean() + self.restartTarget()
meta/lib/oeqa/runtime/buildsudoku.py +2 -0
@@ -10,6 +10,7 @@ class SudokuTest(oeRuntimeTest): @classmethod def setUpClass(self): + self.restartTarget("-m 512") self.project = TargetBuildProject(oeRuntimeTest.tc.target, "http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2") self.project.download_archive()
@@ -25,3 +26,4 @@ class SudokuTest(oeRuntimeTest): @classmethod def tearDownClass(self): self.project.clean() + self.restartTarget()
Signature · 1 line
1.8.3.1
Re: [PATCH 21/21] lib/oeqa: add a restart method for base class and use it for build tests
Quoted 83 lines · On Friday 23 August 2013 18:31:01 Stefan Stanacar wrote:
This can be used by tests to restart the target (useful for passing extra qemuparams, like more RAM)
Build projects like cvs/iptables can take 15min+ on qemu targets with the default RAM, and sudoku fails with out of memory, so use this there.
Signed-off-by Stefan Stanacar <[email protected]>--- meta/lib/oeqa/oetest.py | 7 +++++++ meta/lib/oeqa/runtime/buildcvs.py | 2 ++ meta/lib/oeqa/runtime/buildiptables.py | 2 ++ meta/lib/oeqa/runtime/buildsudoku.py | 2 ++ 4 files changed, 13 insertions(+)meta/lib/oeqa/oetest.py +7
-0@@ -81,6 +81,13 @@ class oeRuntimeTest(unittest.TestCase): else: return False + @classmethod + def restartTarget(self,params=None): + + if oeRuntimeTest.tc.qemu.restart(params): + oeRuntimeTest.tc.target.host = oeRuntimeTest.tc.qemu.ip + else: + raise Exception("Restarting target failed") def getmodule(pos=2):(quoted diff) +0
-0
b/meta/lib/oeqa/runtime/buildcvs.pyindex 9bf764d..f024dfa 100644meta/lib/oeqa/runtime/buildcvs.py +1
-0@@ -10,6 +10,7 @@ class BuildCvsTest(oeRuntimeTest): @classmethod def setUpClass(self): + self.restartTarget("-m 512") self.project = TargetBuildProject(oeRuntimeTest.tc.target,"http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2" )
self.project.download_archive()meta/lib/oeqa/runtime/buildcvs.py +1
-0@@ -28,3 +29,4 @@ class BuildCvsTest(oeRuntimeTest): @classmethod def tearDownClass(self): self.project.clean() + self.restartTarget()(quoted diff) +0
-0
b/meta/lib/oeqa/runtime/buildiptables.pyindex 50faf5d..88ece3b 100644meta/lib/oeqa/runtime/buildiptables.py +1
-0@@ -10,6 +10,7 @@ class BuildIptablesTest(oeRuntimeTest): @classmethod def setUpClass(self): + self.restartTarget("-m 512") self.project = TargetBuildProject(oeRuntimeTest.tc.target, "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2")
self.project.download_archive()meta/lib/oeqa/runtime/buildiptables.py +1
-0@@ -28,3 +29,4 @@ class BuildIptablesTest(oeRuntimeTest): @classmethod def tearDownClass(self): self.project.clean() + self.restartTarget()(quoted diff) +0
-0
b/meta/lib/oeqa/runtime/buildsudoku.pyindex 61dc1ff..0a7306d 100644meta/lib/oeqa/runtime/buildsudoku.py +1
-0@@ -10,6 +10,7 @@ class SudokuTest(oeRuntimeTest): @classmethod def setUpClass(self): + self.restartTarget("-m 512") self.project = TargetBuildProject(oeRuntimeTest.tc.target,"http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudok u-savant-1.3/sudoku-savant-1.3.tar.bz2")
self.project.download_archive()
meta/lib/oeqa/runtime/buildsudoku.py class SudokuTest(oeRuntimeTest): · line 29
Quoted 4 lines
meta/lib/oeqa/runtime/buildsudoku.py +1
-0@@ -25,3 +26,4 @@ class SudokuTest(oeRuntimeTest): @classmethod def tearDownClass(self): self.project.clean() + self.restartTarget()
Could you please you move the function addition earlier in the series and merge the test changes into the earlier commits that added them?
Thanks,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre