Skip to contentSkip to conversation

Some new runtime tests or fixes

PATCH Superseded Stefan Stanacar 2 participants 24 messages Aug 23, 2013

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Some new runtime tests or fixes

Hello,

This set includes new runtime tests and required changes for them.

Not all of them will get run by default, for example all the project build tests on target (iptables, cvs and sudoku) need to be enabled manually. Same applies to tests which require special options (skeleton, logrotate, pam). A new patch for AutoBuilder is needed to add a new target for these. The new smart tests are the most important change here, (and those will get run by default on all targets on AB).

Cheers,
Stefan

The following changes since commit 199b1a8c7d7109d9b59d67bf4dcfeb03be07587a:

mkfontscale: This no longer needs a full libx11, xproto suffices (2013-08-22 18:30:06 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib stefans/stage-oeqa1

for you to fetch changes up to 6d7bb8409eb99b415d2500c5146a6b0c7eac2e7f:

lib/oeqa: add a restart method for base class and use it for build tests (2013-08-23 15:14:43 +0300)

----------------------------------------------------------------
Alexandru Georgescu (1):
lib/oeqa/runtime: add basic test for x32 images

Alexandru Palalau (4):
lib/oeqa/runtime: add new logrotate test
lib/oeqa/runtime: add new skeletoninit test
lib/oeqa/runtime: add new PAM support test
lib/oeqa/runtime: add new scp test

Cornel Stoicescu (2):
lib/oeqa/runtime: add test for perl
lib/oeqa/runtime: add a test for ldd

Mihai Lindner (4):
lib/oeqa/utils: new file: httpserver.py useful for serving files over HTTP to the target lib/oeqa/utils: qemurunner: save host IP address
oeqa/utils/decorators: return the decorated method
lib/oeqa/runtime: smart: add new smart tests

Mihai Prica (4):
lib/oeqa/utils: targetbuild: Add helper class for building packages on target lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target lib/oeqa/runtime: add adjust date and time test lib/oeqa/runtime: add vncserver for target test

Stefan Stanacar (6):
classess/testimage: change default test suites oeqa/utils/targetbuild: change download to use bitbake's fetcher lib/oeqa/runtime: rpm: add install and erase tests lib/oeqa: change behaviour for unskippable tests lib/oeqa/utils: qemurunner: improve kill and restart lib/oeqa: add a restart method for base class and use it for build tests

 meta/classes/testimage.bbclass         |   4 +-
 meta/lib/oeqa/oetest.py                |  11 +++-
 meta/lib/oeqa/runtime/buildcvs.py      |  32 +++++++++++
 meta/lib/oeqa/runtime/buildiptables.py |  32 +++++++++++
 meta/lib/oeqa/runtime/buildsudoku.py   |  29 ++++++++++
 meta/lib/oeqa/runtime/date.py          |  27 +++++++++
 meta/lib/oeqa/runtime/files/test.pl    |   2 +
 meta/lib/oeqa/runtime/ldd.py           |  19 ++++++
 meta/lib/oeqa/runtime/logrotate.py     |  27 +++++++++
 meta/lib/oeqa/runtime/pam.py           |  24 ++++++++
 meta/lib/oeqa/runtime/perl.py          |  28 +++++++++
 meta/lib/oeqa/runtime/rpm.py           |  30 +++++++++-
 meta/lib/oeqa/runtime/scp.py           |  22 +++++++
 meta/lib/oeqa/runtime/smart.py         | 102 +++++++++++++++++++++++++++++----
 meta/lib/oeqa/runtime/vnc.py           |  22 +++++++
 meta/lib/oeqa/runtime/x32lib.py        |  17 ++++++
 meta/lib/oeqa/skeletoninit.py          |  28 +++++++++
 meta/lib/oeqa/utils/decorators.py      |   6 +-
 meta/lib/oeqa/utils/httpserver.py      |  32 +++++++++++
 meta/lib/oeqa/utils/qemurunner.py      |  39 ++++++++-----
 meta/lib/oeqa/utils/targetbuild.py     |  63 ++++++++++++++++++++
 21 files changed, 561 insertions(+), 35 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/buildcvs.py
 create mode 100644 meta/lib/oeqa/runtime/buildiptables.py
 create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py
 create mode 100644 meta/lib/oeqa/runtime/date.py
 create mode 100644 meta/lib/oeqa/runtime/files/test.pl
 create mode 100644 meta/lib/oeqa/runtime/ldd.py
 create mode 100644 meta/lib/oeqa/runtime/logrotate.py
 create mode 100644 meta/lib/oeqa/runtime/pam.py
 create mode 100644 meta/lib/oeqa/runtime/perl.py
 create mode 100644 meta/lib/oeqa/runtime/scp.py
 create mode 100644 meta/lib/oeqa/runtime/vnc.py
 create mode 100644 meta/lib/oeqa/runtime/x32lib.py
 create mode 100644 meta/lib/oeqa/skeletoninit.py
 create mode 100644 meta/lib/oeqa/utils/httpserver.py
 create mode 100644 meta/lib/oeqa/utils/targetbuild.py

Alexandru Georgescu (1):
lib/oeqa/runtime: add basic test for x32 images

Alexandru Palalau (4):
lib/oeqa/runtime: add new logrotate test
lib/oeqa/runtime: add new skeletoninit test
lib/oeqa/runtime: add new PAM support test
lib/oeqa/runtime: add new scp test

Cornel Stoicescu (2):
lib/oeqa/runtime: add test for perl
lib/oeqa/runtime: add a test for ldd

Mihai Lindner (4):
lib/oeqa/utils: new file: httpserver.py useful for serving files over
HTTP to the target
lib/oeqa/utils: qemurunner: save host IP address oeqa/utils/decorators: return the decorated method lib/oeqa/runtime: smart: add new smart tests

Mihai Prica (4):
lib/oeqa/utils: targetbuild: Add helper class for building packages on
target
lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on
target
lib/oeqa/runtime: add adjust date and time test lib/oeqa/runtime: add vncserver for target test

Stefan Stanacar (6):
classess/testimage: change default test suites oeqa/utils/targetbuild: change download to use bitbake's fetcher lib/oeqa/runtime: rpm: add install and erase tests lib/oeqa: change behaviour for unskippable tests lib/oeqa/utils: qemurunner: improve kill and restart lib/oeqa: add a restart method for base class and use it for build
tests

 meta/classes/testimage.bbclass         |   4 +-
 meta/lib/oeqa/oetest.py                |  11 +++-
 meta/lib/oeqa/runtime/buildcvs.py      |  32 +++++++++++
 meta/lib/oeqa/runtime/buildiptables.py |  32 +++++++++++
 meta/lib/oeqa/runtime/buildsudoku.py   |  29 ++++++++++
 meta/lib/oeqa/runtime/date.py          |  27 +++++++++
 meta/lib/oeqa/runtime/files/test.pl    |   2 +
 meta/lib/oeqa/runtime/ldd.py           |  19 ++++++
 meta/lib/oeqa/runtime/logrotate.py     |  27 +++++++++
 meta/lib/oeqa/runtime/pam.py           |  24 ++++++++
 meta/lib/oeqa/runtime/perl.py          |  28 +++++++++
 meta/lib/oeqa/runtime/rpm.py           |  30 +++++++++-
 meta/lib/oeqa/runtime/scp.py           |  22 +++++++
 meta/lib/oeqa/runtime/smart.py         | 102 +++++++++++++++++++++++++++++----
 meta/lib/oeqa/runtime/vnc.py           |  22 +++++++
 meta/lib/oeqa/runtime/x32lib.py        |  17 ++++++
 meta/lib/oeqa/skeletoninit.py          |  28 +++++++++
 meta/lib/oeqa/utils/decorators.py      |   6 +-
 meta/lib/oeqa/utils/httpserver.py      |  32 +++++++++++
 meta/lib/oeqa/utils/qemurunner.py      |  39 ++++++++-----
 meta/lib/oeqa/utils/targetbuild.py     |  63 ++++++++++++++++++++
 21 files changed, 561 insertions(+), 35 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/buildcvs.py
 create mode 100644 meta/lib/oeqa/runtime/buildiptables.py
 create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py
 create mode 100644 meta/lib/oeqa/runtime/date.py
 create mode 100644 meta/lib/oeqa/runtime/files/test.pl
 create mode 100644 meta/lib/oeqa/runtime/ldd.py
 create mode 100644 meta/lib/oeqa/runtime/logrotate.py
 create mode 100644 meta/lib/oeqa/runtime/pam.py
 create mode 100644 meta/lib/oeqa/runtime/perl.py
 create mode 100644 meta/lib/oeqa/runtime/scp.py
 create mode 100644 meta/lib/oeqa/runtime/vnc.py
 create mode 100644 meta/lib/oeqa/runtime/x32lib.py
 create mode 100644 meta/lib/oeqa/skeletoninit.py
 create mode 100644 meta/lib/oeqa/utils/httpserver.py
 create mode 100644 meta/lib/oeqa/utils/targetbuild.py
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/21 lib/oeqa/runtime: add basic test for x32 images

From: Alexandru Georgescu <[email protected]>

Checks that an x86-64-x32 image has the right binaries.

---
 meta/lib/oeqa/runtime/x32lib.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/x32lib.py
meta/lib/oeqa/runtime/x32lib.py new file +17 -0
new file mode 100644
index 0000000..6bad201
@@ -0,0 +1,17 @@
+import unittest
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+        #check if DEFAULTTUNE is set and it's value is: x86-64-x32
+        defaulttune = oeRuntimeTest.tc.d.getVar("DEFAULTTUNE", True)
+        if "x86-64-x32" not in defaulttune:
+            skipModule("DEFAULTTUNE is not set to x86-64-x32")
+
+class X32libTest(oeRuntimeTest):
+
+    @skipUnlessPassed("test_ssh")
+    def test_x32_file(self):
+        status1 = self.target.run("readelf -h /bin/ls | grep Class | grep ELF32")[0]
+        status2 = self.target.run("readelf -h /bin/ls | grep Machine | grep X86-64")[0]
+        self.assertTrue(status1 == 0 and status2 == 0, msg="/bin/ls isn't an X86-64 ELF32 binary. readelf says: %s" % self.target.run("readelf -h /bin/ls")[1])
Signature · 1 line
1.8.3.1
#2 lore raw

PATCH 2/21 lib/oeqa/runtime: add test for perl

In reply to Stefan Stanacar

From: Cornel Stoicescu <[email protected]>

This test runs a perl script on the target and checks the output.

---
 meta/lib/oeqa/runtime/files/test.pl |  2 ++
 meta/lib/oeqa/runtime/perl.py       | 28 ++++++++++++++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/files/test.pl
 create mode 100644 meta/lib/oeqa/runtime/perl.py
meta/lib/oeqa/runtime/files/test.pl new file +2 -0
new file mode 100644
index 0000000..689c8f1
@@ -0,0 +1,2 @@
+$a = 9.01e+21 - 9.01e+21 + 0.01;
+print ("the value of a is ", $a, "\n");
meta/lib/oeqa/runtime/perl.py new file +28 -0
new file mode 100644
index 0000000..c9bb684
@@ -0,0 +1,28 @@
+import unittest
+import os
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("perl"):
+        skipModule("No perl package in the image")
+
+
+class PerlTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.pl"), "/tmp/test.pl")
+
+    def test_perl_exists(self):
+        (status, output) = self.target.run('which perl')
+        self.assertEqual(status, 0, msg="Perl binary not in PATH or not on target.")
+
+    def test_perl_works(self):
+        (status, output) = self.target.run('perl /tmp/test.pl')
+        self.assertEqual(status, 0, msg="Exit status was not 0. Output: %s" % output)
+        self.assertEqual(output, "the value of a is 0.01", msg="Incorrect output: %s" % output)
+
+    @classmethod
+    def tearDownClass(self):
+        oeRuntimeTest.tc.target.run("rm /tmp/test.pl")
Signature · 1 line
1.8.3.1
#3 lore raw

PATCH 3/21 lib/oeqa/runtime: add a test for ldd

In reply to Stefan Stanacar

From: Cornel Stoicescu <[email protected]>

This test checks that at least one path in RTLDLIST exists.

---
 meta/lib/oeqa/runtime/ldd.py | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/ldd.py
meta/lib/oeqa/runtime/ldd.py new file +19 -0
new file mode 100644
index 0000000..577c5e2
@@ -0,0 +1,19 @@
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class LddTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    def test_ldd_exists(self):
+        (status, output) = self.target.run('which ldd')
+        self.assertEqual(status, 0, msg = "ldd does not exist in PATH !")
+
+    @skipUnlessPassed('test_ldd_exists')
+    def test_ldd(self):
+        (status, output) = self.target.run('for i in $(which ldd | xargs cat | grep "^RTLDLIST"|cut -d\'=\' -f2|tr -d \'"\'); do test -f $i && echo $i && break; done')
+        self.assertEqual(status, 0, msg = "ldd path not correct or RTLDLIST files don't exist. ")
Signature · 1 line
1.8.3.1
#4 lore raw

PATCH 4/21 lib/oeqa/runtime: add new logrotate test

In reply to Stefan Stanacar

From: Alexandru Palalau <[email protected]>

New logrotate test which verifies the log directory change in logrotate.conf. Needs an image with logrotate installed.

---
 meta/lib/oeqa/runtime/logrotate.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/logrotate.py
meta/lib/oeqa/runtime/logrotate.py new file +27 -0
new file mode 100644
index 0000000..80489a3
@@ -0,0 +1,27 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=289 testcase
+# Note that the image under test must have logrotate installed
+
+import unittest
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("logrotate"):
+        skipModule("No logrotate package in image")
+
+
+class LogrotateTest(oeRuntimeTest):
+
+    @skipUnlessPassed("test_ssh")
+    def test_1_logrotate_setup(self):
+        (status, output) = self.target.run('mkdir /home/root/logrotate_dir')
+        self.assertEqual(status, 0, msg = "Could not create logrotate_dir. Output: %s" % output)
+        (status, output) = self.target.run("sed -i 's#wtmp {#wtmp {\\n    olddir /home/root/logrotate_dir#' /etc/logrotate.conf")
+        self.assertEqual(status, 0, msg = "Could not write to logrotate.conf file. Status and output: %s and %s)" % (status, output))
+
+    @skipUnlessPassed("test_1_logrotate_setup")
+    def test_2_logrotate(self):
+        (status, output) = self.target.run('logrotate -f /etc/logrotate.conf')
+        self.assertEqual(status, 0, msg = "logrotate service could not be reloaded. Status and output: %s and %s" % (status, output))
+        output = self.target.run('ls -la /home/root/logrotate_dir/ | wc -l')[1]
+        self.assertTrue(int(output)>=3, msg = "new logfile could not be created. List of files within log directory: %s" %(self.target.run('ls -la /home/root/logrotate_dir')[1]))
Signature · 1 line
1.8.3.1
#5 lore raw

PATCH 5/21 lib/oeqa/runtime: add new skeletoninit test

In reply to Stefan Stanacar

From: Alexandru Palalau <[email protected]>

New test which verifies the usage of skeleton init script available with meta-skeleton layer

---
 meta/lib/oeqa/skeletoninit.py | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100644 meta/lib/oeqa/skeletoninit.py
meta/lib/oeqa/skeletoninit.py new file +28 -0
new file mode 100644
index 0000000..557e715
@@ -0,0 +1,28 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=284 testcase
+# Note that the image under test must have meta-skeleton layer in bblayers and IMAGE_INSTALL_append = " service" in local.conf
+
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("service"):
+        skipModule("No service package in image")
+
+
+class SkeletonBasicTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"), "Not appropiate for systemd image")
+    def test_skeleton_availability(self):
+        (status, output) = self.target.run('ls /etc/init.d/skeleton')
+        self.assertEqual(status, 0, msg = "skeleton init script not found. Output:\n%s " % output)
+        (status, output) =  self.target.run('ls /usr/sbin/skeleton-test')
+        self.assertEqual(status, 0, msg = "skeleton-test not found. Output:\n%s" % output)
+
+    @skipUnlessPassed('test_skeleton_availability')
+    @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"), "Not appropiate for systemd image")
+    def test_skeleton_script(self):
+        output1 = self.target.run("/etc/init.d/skeleton start")[1]
+        (status, output2) = self.target.run(oeRuntimeTest.pscmd + ' | grep [s]keleton-test')
+        self.assertEqual(status, 0, msg = "Skeleton script could not be started:\n%s\n%s" % (output1, output2))
Signature · 1 line
1.8.3.1
#6 lore raw

PATCH 6/21 lib/oeqa/runtime: add new PAM support test

In reply to Stefan Stanacar

From: Alexandru Palalau <[email protected]>

New test which verifies some usual commands functionality with PAM support

---
 meta/lib/oeqa/runtime/pam.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/pam.py
meta/lib/oeqa/runtime/pam.py new file +24 -0
new file mode 100644
index 0000000..52e1eb8
@@ -0,0 +1,24 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=287 testcase
+# Note that the image under test must have "pam" in DISTRO_FEATURES
+
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("pam"):
+        skipModule("target doesn't have 'pam' in DISTRO_FEATURES")
+
+
+class PamBasicTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    def test_pam(self):
+        (status, output) = self.target.run('login --help')
+        self.assertEqual(status, 1, msg = "login command does not work as expected. Status and output:%s and %s" %(status, output))
+        (status, output) = self.target.run('passwd --help')
+        self.assertEqual(status, 6, msg = "passwd command does not work as expected. Status and output:%s and %s" %(status, output))
+        (status, output) = self.target.run('su --help')
+        self.assertEqual(status, 2, msg = "su command does not work as expected. Status and output:%s and %s" %(status, output))
+        (status, output) = self.target.run('useradd --help')
+        self.assertEqual(status, 2, msg = "useradd command does not work as expected. Status and output:%s and %s" %(status, output))
Signature · 1 line
1.8.3.1
#7 lore raw

PATCH 7/21 lib/oeqa/runtime: add new scp test

In reply to Stefan Stanacar

From: Alexandru Palalau <[email protected]>

Copies a 5MB to target using scp, more of an
network test than a scp one.

---
 meta/lib/oeqa/runtime/scp.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/scp.py
meta/lib/oeqa/runtime/scp.py new file +22 -0
new file mode 100644
index 0000000..b914802
@@ -0,0 +1,22 @@
+import subprocess
+import unittest
+import os
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not (oeRuntimeTest.hasPackage("dropbear") or oeRuntimeTest.hasPackage("openssh-sshd")):
+        skipModule("No ssh package in image")
+
+
+class ScpTest(oeRuntimeTest):
+
+    def setUp(self):
+         subprocess.check_call("dd if=/dev/zero of=%s bs=512k count=10" % os.path.join(oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR", True), 'test_scp_file'), shell=True)
+
+    @skipUnlessPassed('test_ssh')
+    def test_scp(self):
+        (status, output) = self.target.copy_to(os.path.join(oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR", True), 'test_scp_file'), '/tmp/test_scp_file')
+        self.assertEqual(status, 0, msg = "File could not be copied. Output: %s" % output)
+        (status, output) = self.target.run("ls -la /tmp/test_scp_file")
+        self.assertEqual(status, 0, msg = "SCP test failed")
Signature · 1 line
1.8.3.1
#8 lore raw

PATCH 8/21 lib/oeqa/utils: new file: httpserver.py useful for serving files over HTTP to the target

In reply to Stefan Stanacar

From: Mihai Lindner <[email protected]>

It can be used by smart repo/channel tests to serve deploy_dir.

---
 meta/lib/oeqa/utils/httpserver.py | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/httpserver.py
meta/lib/oeqa/utils/httpserver.py new file +32 -0
new file mode 100644
index 0000000..d4b6154
@@ -0,0 +1,32 @@
+import SimpleHTTPServer
+import multiprocessing
+import os
+
+class HTTPServer(SimpleHTTPServer.BaseHTTPServer.HTTPServer):
+
+    def server_start(self, root_dir):
+        os.chdir(root_dir)
+        self.serve_forever()
+
+class HTTPRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
+
+    def log_message(self, format_str, *args):
+        pass
+
+class HTTPService(object):
+
+    def __init__(self, root_dir):
+        self.root_dir = root_dir
+        self.port = 0
+
+    def start(self):
+        self.server = HTTPServer(('', self.port), HTTPRequestHandler)
+        if self.port == 0:
+            self.port = self.server.server_port
+        self.process = multiprocessing.Process(target=self.server.server_start, args=[self.root_dir])
+        self.process.start()
+
+    def stop(self):
+        self.server.server_close()
+        self.process.terminate()
+        self.process.join()
Signature · 1 line
1.8.3.1
#9 lore raw

PATCH 9/21 lib/oeqa/utils: qemurunner: save host IP address

In reply to Stefan Stanacar

From: Mihai Lindner <[email protected]>

Save host IP address to host_ip.
Read /proc/PID/cmdline on host to look for IPs of target and host; instead of running 'ps'.
Also removed some extra empty lines from file.

---
 meta/lib/oeqa/utils/qemurunner.py | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +3 -8
index 20bb1e5..9ae618f 100644
@@ -54,7 +54,6 @@ class QemuRunner:
 
     def launch(self, qemuparams = None):
 
-
         if self.display:
             os.environ["DISPLAY"] = self.display
         else:
@@ -84,12 +83,11 @@ class QemuRunner:
 
         if self.is_alive():
             bb.note("qemu started - qemu procces pid is %s" % self.qemupid)
-            pscmd = 'ps -p %s -fww | grep -o "192\.168\.7\.[0-9]*::" | awk -F":" \'{print $1}\'' % self.qemupid
-            self.ip = subprocess.Popen(pscmd,shell=True,stdout=subprocess.PIPE).communicate()[0].strip()
+            cmdline = open('/proc/%s/cmdline' % self.qemupid).read()
+            self.ip, _, self.host_ip = cmdline.split('ip=')[1].split(' ')[0].split(':')[0:3]
             if not re.search("^((?:[0-9]{1,3}\.){3}[0-9]{1,3})$", self.ip):
                 bb.note("Couldn't get ip from qemu process arguments, I got '%s'" % self.ip)
-                bb.note("Here is the ps output:\n%s" % \
-                        subprocess.Popen("ps -p %s -fww" % self.qemupid,shell=True,stdout=subprocess.PIPE).communicate()[0])
+                bb.note("Here is the ps output:\n%s" % cmdline)
                 self.kill()
                 return False
             bb.note("IP found: %s" % self.ip)
@@ -122,7 +120,6 @@ class QemuRunner:
                             sock.close()
                             stopread = True
 
-
             if not reachedlogin:
                 bb.note("Target didn't reached login boot in %d seconds" % self.boottime)
                 lines = "\n".join(self.bootlog.splitlines()[-5:])
@@ -139,7 +136,6 @@ class QemuRunner:
 
         return self.is_alive()
 
-
     def kill(self):
         if self.server_socket:
             self.server_socket.close()
@@ -207,4 +203,3 @@ class QemuRunner:
             basecmd = os.path.basename(basecmd)
             if "qemu-system" in basecmd and "-serial tcp" in commands[p]:
                 return [int(p),commands[p]]
-
Signature · 1 line
1.8.3.1
#10 lore raw

PATCH 10/21 oeqa/utils/decorators: return the decorated method

In reply to Stefan Stanacar

From: Mihai Lindner <[email protected]>

Decorators should return whatever the decorated methods return.

---
 meta/lib/oeqa/utils/decorators.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/lib/oeqa/utils/decorators.py +3 -3
index e0ca6fd..33fed5a 100644
@@ -17,7 +17,7 @@ class skipIfFailure(object):
         def wrapped_f(*args):
             if self.testcase in (oeRuntimeTest.testFailures or oeRuntimeTest.testErrors):
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
-            f(*args)
+            return f(*args)
         wrapped_f.__name__ = f.__name__
         return wrapped_f
 
@@ -30,7 +30,7 @@ class skipIfSkipped(object):
         def wrapped_f(*args):
             if self.testcase in oeRuntimeTest.testSkipped:
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
-            f(*args)
+            return f(*args)
         wrapped_f.__name__ = f.__name__
         return wrapped_f
 
@@ -45,6 +45,6 @@ class skipUnlessPassed(object):
                     self.testcase in  oeRuntimeTest.testFailures or \
                     self.testcase in oeRuntimeTest.testErrors:
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
-            f(*args)
+            return f(*args)
         wrapped_f.__name__ = f.__name__
         return wrapped_f
Signature · 1 line
1.8.3.1
#11 lore raw

PATCH 11/21 lib/oeqa/runtime: smart: add new smart tests

In reply to Stefan Stanacar

From: Mihai Lindner <[email protected]>

Add class to be inherited by smart tests, along with more basic tests and tests using a rpm repository.

---
 meta/lib/oeqa/runtime/smart.py | 102 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 91 insertions(+), 11 deletions(-)
meta/lib/oeqa/runtime/smart.py +91 -11
index 0b03a30..4ea2699 100644
@@ -1,6 +1,7 @@
 import unittest
 from oeqa.oetest import oeRuntimeTest
 from oeqa.utils.decorators import *
+from oeqa.utils.httpserver import HTTPService
 
 def setUpModule():
     if not oeRuntimeTest.hasFeature("package-management"):
@@ -8,21 +9,100 @@ def setUpModule():
     if not oeRuntimeTest.hasPackage("smart"):
         skipModule("Image doesn't have smart installed")
 
-class SmartHelpTest(oeRuntimeTest):
+class SmartTest(oeRuntimeTest):
+
+    longMessage = True
+
+    @skipUnlessPassed('test_smart_help')
+    def smart(self, command, expected = 0):
+        command = 'smart %s' % command
+        status, output = self.target.run(command)
+        message = os.linesep.join([command, output])
+        self.assertEqual(status, expected, message)
+        return output
+
+class SmartBasicTest(SmartTest):
 
     @skipUnlessPassed('test_ssh')
     def test_smart_help(self):
-        status = self.target.run('smart --help')[0]
-        self.assertEqual(status, 0)
+        self.smart('--help')
 
-class SmartQueryTest(oeRuntimeTest):
+    def test_smart_version(self):
+        self.smart('--version')
+
+    def test_smart_info(self):
+        self.smart('info python-smartpm')
 
-    @skipUnlessPassed('test_smart_help')
     def test_smart_query(self):
-        (status, output) = self.target.run('smart query rpm')
-        self.assertEqual(status, 0, msg="smart query failed, output: %s" % output)
+        self.smart('query python-smartpm')
 
-    @skipUnlessPassed('test_smart_query')
-    def test_smart_info(self):
-        (status, output) = self.target.run('smart info rpm')
-        self.assertEqual(status, 0, msg="smart info rpm failed, output: %s" % output)
+    def test_smart_search(self):
+        self.smart('search python-smartpm')
+
+    def test_smart_stats(self):
+        self.smart('stats')
+
+class SmartRepoTest(SmartTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.repo_server = HTTPService(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True))
+        self.repo_server.start()
+
+    @classmethod
+    def tearDownClass(self):
+        self.repo_server.stop()
+
+    def test_smart_channel(self):
+        self.smart('channel', 1)
+
+    def test_smart_channel_add(self):
+        image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE', True)
+        deploy_url = 'http://%s:%s/%s' %(self.tc.qemu.host_ip, self.repo_server.port, image_pkgtype)
+        for arch in os.listdir('%s/%s' % (self.repo_server.root_dir, image_pkgtype)):
+            self.smart('channel -y --add {a} type=rpm-md baseurl={u}/{a}'.format(a=arch, u=deploy_url))
+        self.smart('update')
+
+    def test_smart_channel_help(self):
+        self.smart('channel --help')
+
+    def test_smart_channel_list(self):
+        self.smart('channel --list')
+
+    def test_smart_channel_show(self):
+        self.smart('channel --show')
+
+    def test_smart_channel_rpmsys(self):
+        self.smart('channel --show rpmsys')
+        self.smart('channel --disable rpmsys')
+        self.smart('channel --enable rpmsys')
+
+    @skipUnlessPassed('test_smart_channel_add')
+    def test_smart_install(self):
+        self.smart('remove -y psplash-default')
+        self.smart('install -y psplash-default')
+
+    @skipUnlessPassed('test_smart_install')
+    def test_smart_install_dependency(self):
+        self.smart('remove -y psplash')
+        self.smart('install -y psplash-default')
+
+    @skipUnlessPassed('test_smart_channel_add')
+    def test_smart_install_from_disk(self):
+        self.smart('remove -y psplash-default')
+        self.smart('download psplash-default')
+        self.smart('install -y ./psplash-default*')
+
+    @skipUnlessPassed('test_smart_channel_add')
+    def test_smart_install_from_http(self):
+        url = 'http://'
+        output = self.smart('download --urls psplash-default')
+        for line in output.splitlines():
+            if line.startswith(url):
+                url = line
+        self.smart('remove -y psplash-default')
+        self.smart('install -y %s' % url)
+
+    @skipUnlessPassed('test_smart_install')
+    def test_smart_reinstall(self):
+        self.smart('reinstall -y psplash-default')
Signature · 1 line
1.8.3.1
#12 lore raw

PATCH 12/21 lib/oeqa/utils: targetbuild: Add helper class for building packages on target

In reply to Stefan Stanacar

From: Mihai Prica <[email protected]>

This class can be used for test cases that configure and build packages on target.

---
 meta/lib/oeqa/utils/targetbuild.py | 66 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/targetbuild.py
meta/lib/oeqa/utils/targetbuild.py new file +66 -0
new file mode 100644
index 0000000..7555add
@@ -0,0 +1,66 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Provides a class for automating build tests for projects
+
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+import bb.process
+import os
+import re
+
+
+class TargetBuildProject():
+
+    def __init__(self, target, uri, foldername=None):
+        self.target = target
+        self.uri = uri
+        self.targetdir = "/home/root/"
+
+        if not foldername:
+            self.archive = os.path.basename(uri)
+            self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
+        else:
+            self.fname = foldername
+
+    def download_archive(self):
+        wgetcmd = oeRuntimeTest.tc.d.getVar('FETCHCMD_wget', True).split()
+        self.testdir = oeRuntimeTest.tc.d.getVar('TEST_LOG_DIR', True)
+
+        try:
+            output = bb.process.run(wgetcmd + ['-P', self.testdir, self.uri])[0]
+        except bb.process.CmdError:
+            raise Exception("Failed to download archive, output: %s" % output)
+
+        (status, output) = oeRuntimeTest.tc.target.copy_to(
+            os.path.join(self.testdir, self.archive),
+            self.targetdir)
+        if status != 0:
+            raise Exception("Failed to copy archive to target, output: %s" % output)
+
+        (status, output) = oeRuntimeTest.tc.target.run(
+            'tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir))
+        if status != 0:
+            raise Exception("Failed to extract archive, output: %s" % output)
+
+        #Change targetdir to project folder
+        self.targetdir = self.targetdir + self.fname
+
+    # The timeout parameter of target.run is set to 0 to make the ssh command
+    # run with no timeout.
+    def run_configure(self):
+        return self.target.run('cd %s; ./configure' % self.targetdir, 0)[0]
+
+    def run_make(self):
+        return self.target.run('cd %s; make' % self.targetdir, 0)[0]
+
+    def run_install(self):
+        return self.target.run('cd %s; make install' % self.targetdir, 0)[0]
+
+    def clean(self):
+        self.target.run('rm -r %s*' % self.targetdir)
+        try:
+            bb.process.run(['rm', '-rf', os.path.join(self.testdir, self.archive)])
+        except bb.process.CmdError:
+            bb.note("Failed to remove archive")
Signature · 1 line
1.8.3.1
#13 lore raw

PATCH 13/21 lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target

In reply to Stefan Stanacar

From: Mihai Prica <[email protected]>

Downloads iptables/cvs/sudoku-savant sources and builds them on target.

---
 meta/lib/oeqa/runtime/buildcvs.py      | 30 ++++++++++++++++++++++++++++++
 meta/lib/oeqa/runtime/buildiptables.py | 30 ++++++++++++++++++++++++++++++
 meta/lib/oeqa/runtime/buildsudoku.py   | 27 +++++++++++++++++++++++++++
 3 files changed, 87 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/buildcvs.py
 create mode 100644 meta/lib/oeqa/runtime/buildiptables.py
 create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py
meta/lib/oeqa/runtime/buildcvs.py new file +30 -0
new file mode 100644
index 0000000..9bf764d
@@ -0,0 +1,30 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+from oeqa.utils.targetbuild import TargetBuildProject
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class BuildCvsTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+                        "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2")
+        self.project.download_archive()
+
+    @skipUnlessPassed("test_ssh")
+    def test_cvs(self):
+        self.assertEqual(self.project.run_configure(), 0,
+                        msg="Running configure failed")
+
+        self.assertEqual(self.project.run_make(), 0,
+                        msg="Running make failed")
+
+        self.assertEqual(self.project.run_install(), 0,
+                        msg="Running make install failed")
+
+    @classmethod
+    def tearDownClass(self):
+        self.project.clean()
meta/lib/oeqa/runtime/buildiptables.py new file +30 -0
new file mode 100644
index 0000000..50faf5d
@@ -0,0 +1,30 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+from oeqa.utils.targetbuild import TargetBuildProject
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class BuildIptablesTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+                        "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2")
+        self.project.download_archive()
+
+    @skipUnlessPassed("test_ssh")
+    def test_iptables(self):
+        self.assertEqual(self.project.run_configure(), 0,
+                        msg="Running configure failed")
+
+        self.assertEqual(self.project.run_make(), 0,
+                        msg="Running make failed")
+
+        self.assertEqual(self.project.run_install(), 0,
+                        msg="Running make install failed")
+
+    @classmethod
+    def tearDownClass(self):
+        self.project.clean()
meta/lib/oeqa/runtime/buildsudoku.py new file +27 -0
new file mode 100644
index 0000000..61dc1ff
@@ -0,0 +1,27 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+from oeqa.utils.targetbuild import TargetBuildProject
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class SudokuTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+                        "http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2")
+        self.project.download_archive()
+
+    @skipUnlessPassed("test_ssh")
+    def test_sudoku(self):
+        self.assertEqual(self.project.run_configure(), 0,
+                        msg="Running configure failed")
+
+        self.assertEqual(self.project.run_make(), 0,
+                        msg="Running make failed")
+
+    @classmethod
+    def tearDownClass(self):
+        self.project.clean()
Signature · 1 line
1.8.3.1
#14 lore raw

PATCH 14/21 lib/oeqa/runtime: add adjust date and time test

In reply to Stefan Stanacar

From: Mihai Prica <[email protected]>

---
 meta/lib/oeqa/runtime/date.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/date.py
meta/lib/oeqa/runtime/date.py new file +27 -0
new file mode 100644
index 0000000..d6c04cb
@@ -0,0 +1,27 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+import re
+
+def setUpModule():
+    skipModuleUnless(oeRuntimeTest.tc.target.run('which date')[0] == 0, "No date in image or no connection")
+
+class DateTest(oeRuntimeTest):
+
+    def setUp(self):
+        (status, output) = self.target.run('date +"%Y-%m-%d %T"')
+        self.assertEqual(status, 0, msg="Failed to get initial date, output: %s" % output)
+        self.oldDate = output
+
+    @skipUnlessPassed("test_ssh")
+    def test_date(self):
+        sampleDate = '"2016-08-09 10:00:00"'
+        (status, output) = self.target.run("date -s %s" % sampleDate)
+        self.assertEqual(status, 0, msg="Date set failed, output: %s" % output)
+
+        (status, output) = self.target.run("date -R")
+        p = re.match('Tue, 09 Aug 2016 10:00:.. \+0000', output)
+        self.assertTrue(p, msg="The date was not set correctly, output: %s" % output)
+
+    def tearDown(self):
+        (status, output) = self.target.run('date -s "%s"' % self.oldDate)
+        self.assertEqual(status, 0, msg="Failed to reset date, output: %s" % output)
Signature · 1 line
1.8.3.1
#15 lore raw

PATCH 15/21 lib/oeqa/runtime: add vncserver for target test

In reply to Stefan Stanacar

From: Mihai Prica <[email protected]>

---
 meta/lib/oeqa/runtime/vnc.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/vnc.py
meta/lib/oeqa/runtime/vnc.py new file +22 -0
new file mode 100644
index 0000000..9476184
@@ -0,0 +1,22 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+import re
+
+def setUpModule():
+    skipModuleUnless(oeRuntimeTest.tc.target.run('which x11vnc')[0] == 0, "No x11vnc in image")
+
+class VNCTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    def test_vnc(self):
+        (status, output) = self.target.run('x11vnc -display :0.0 -bg -q')
+        self.assertEqual(status, 0, msg="x11vnc server failed to start: %s" % output)
+        port = re.search('PORT=[0-9]*', output)
+        self.assertTrue(port, msg="Listening port not specified in command output: %s" %output)
+
+        (status, output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -i [x]11vnc')
+        self.assertEqual(status, 0, msg="x11vnc process not running")
+
+        vncport = port.group(0).split('=')[1]
+        (status, output) = self.target.run('netstat -atun | grep :%s | grep LISTEN' % vncport)
+        self.assertEqual(status, 0, msg="x11vnc server not running on port %s" % vncport)
Signature · 1 line
1.8.3.1
#16 lore raw

PATCH 16/21 classess/testimage: change default test suites

In reply to Stefan Stanacar

Some new tests were added, safe to have them in the defaults for sato-sdk and sato. Not all of the new tests are here though, either because they aren't applicable to default images or take too long. (like build iptables/cvs/sudoky ones, they can be enabled
in local.conf and a special target on AB setups.). Also reorder them a bit.

---
 meta/classes/testimage.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes/testimage.bbclass +2 -2
index 86121e4..2bebbd8 100644
@@ -26,8 +26,8 @@ TEST_LOG_DIR ?= "${WORKDIR}/testimage"
 
 DEFAULT_TEST_SUITES = "ping auto"
 DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
-DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh connman df rpm smart xorg syslog dmesg"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh connman df rpm smart gcc xorg syslog dmesg"
+DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp vnc date perl ldd gcc rpm smart dmesg"
 
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
Signature · 1 line
1.8.3.1
#17 lore raw

PATCH 17/21 oeqa/utils/targetbuild: change download to use bitbake's fetcher

In reply to Stefan Stanacar

Use bb.fetcher2 instead of running our own wget command (helps with proxy too). Also no need to use the class attribute target, use self.target, as the tests pass that to the class.
Also, we shouldn't clean the archive, now that it gets to DL_DIR.

---
 meta/lib/oeqa/utils/targetbuild.py | 33 +++++++++++++++------------------
 1 file changed, 15 insertions(+), 18 deletions(-)
meta/lib/oeqa/utils/targetbuild.py +15 -18
index 7555add..9b2cf53 100644
@@ -5,8 +5,8 @@
 # Provides a class for automating build tests for projects
 
 from oeqa.oetest import oeRuntimeTest
-from oeqa.utils.decorators import *
-import bb.process
+import bb.fetch2
+import bb.data
 import os
 import re
 
@@ -18,6 +18,9 @@ class TargetBuildProject():
         self.uri = uri
         self.targetdir = "/home/root/"
 
+        self.localdata = bb.data.createCopy(oeRuntimeTest.tc.d)
+        bb.data.update_data(self.localdata)
+
         if not foldername:
             self.archive = os.path.basename(uri)
             self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
@@ -25,22 +28,20 @@ class TargetBuildProject():
             self.fname = foldername
 
     def download_archive(self):
-        wgetcmd = oeRuntimeTest.tc.d.getVar('FETCHCMD_wget', True).split()
-        self.testdir = oeRuntimeTest.tc.d.getVar('TEST_LOG_DIR', True)
 
         try:
-            output = bb.process.run(wgetcmd + ['-P', self.testdir, self.uri])[0]
-        except bb.process.CmdError:
-            raise Exception("Failed to download archive, output: %s" % output)
-
-        (status, output) = oeRuntimeTest.tc.target.copy_to(
-            os.path.join(self.testdir, self.archive),
-            self.targetdir)
+            self.localdata.delVar("BB_STRICT_CHECKSUM")
+            fetcher = bb.fetch2.Fetch([self.uri], self.localdata)
+            fetcher.download()
+            self.localarchive = fetcher.localpath(self.uri)
+        except bb.fetch2.BBFetchException:
+            raise Exception("Failed to download archive: %s" % self.uri)
+
+        (status, output) = self.target.copy_to(self.localarchive, self.targetdir)
         if status != 0:
             raise Exception("Failed to copy archive to target, output: %s" % output)
 
-        (status, output) = oeRuntimeTest.tc.target.run(
-            'tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir))
+        (status, output) = self.target.run('tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir))
         if status != 0:
             raise Exception("Failed to extract archive, output: %s" % output)
 
@@ -59,8 +60,4 @@ class TargetBuildProject():
         return self.target.run('cd %s; make install' % self.targetdir, 0)[0]
 
     def clean(self):
-        self.target.run('rm -r %s*' % self.targetdir)
-        try:
-            bb.process.run(['rm', '-rf', os.path.join(self.testdir, self.archive)])
-        except bb.process.CmdError:
-            bb.note("Failed to remove archive")
+        self.target.run('rm -rf %s' % self.targetdir)
Signature · 1 line
1.8.3.1
#18 lore raw

Re: [PATCH 17/21] oeqa/utils/targetbuild: change download to use bitbake's fetcher

Hi Stefan,

Quoted 78 lines · On Friday 23 August 2013 18:30:57 Stefan Stanacar wrote:

Use bb.fetcher2 instead of running our own wget command (helps with proxy too). Also no need to use the class attribute target, use self.target, as the tests pass that to the class.
Also, we shouldn't clean the archive, now that it gets to DL_DIR.

---
 meta/lib/oeqa/utils/targetbuild.py | 33 +++++++++++++++------------------
 1 file changed, 15 insertions(+), 18 deletions(-)
(quoted diff) +0 -0

b/meta/lib/oeqa/utils/targetbuild.py index 7555add..9b2cf53 100644

meta/lib/oeqa/utils/targetbuild.py +5 -5
@@ -5,8 +5,8 @@
 # Provides a class for automating build tests for projects
 
 from oeqa.oetest import oeRuntimeTest
-from oeqa.utils.decorators import *
-import bb.process
+import bb.fetch2
+import bb.data
 import os
 import re
 
@@ -18,6 +18,9 @@ class TargetBuildProject():
         self.uri = uri
         self.targetdir = "/home/root/"
 
+        self.localdata = bb.data.createCopy(oeRuntimeTest.tc.d)
+        bb.data.update_data(self.localdata)
+
         if not foldername:
             self.archive = os.path.basename(uri)
             self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
@@ -25,22 +28,20 @@ class TargetBuildProject():
             self.fname = foldername
 
     def download_archive(self):
-        wgetcmd = oeRuntimeTest.tc.d.getVar('FETCHCMD_wget', True).split()
-        self.testdir = oeRuntimeTest.tc.d.getVar('TEST_LOG_DIR', True)
 
         try:
-            output = bb.process.run(wgetcmd + ['-P', self.testdir,

self.uri])[0] - except bb.process.CmdError:
- raise Exception("Failed to download archive, output: %s" %
output) -

meta/lib/oeqa/utils/targetbuild.py +8 -3
-        (status, output) = oeRuntimeTest.tc.target.copy_to(
-            os.path.join(self.testdir, self.archive),
-            self.targetdir)
+            self.localdata.delVar("BB_STRICT_CHECKSUM")
+            fetcher = bb.fetch2.Fetch([self.uri], self.localdata)
+            fetcher.download()
+            self.localarchive = fetcher.localpath(self.uri)
+        except bb.fetch2.BBFetchException:
+            raise Exception("Failed to download archive: %s" % self.uri)
+
+        (status, output) = self.target.copy_to(self.localarchive,

self.targetdir) if status != 0:
raise Exception("Failed to copy archive to target, output: %s"
% output)

meta/lib/oeqa/utils/targetbuild.py +0 -2
-        (status, output) = oeRuntimeTest.tc.target.run(
-            'tar xf %s%s -C %s' % (self.targetdir, self.archive,

self.targetdir)) + (status, output) = self.target.run('tar xf %s%s
-C %s' % (self.targetdir, self.archive, self.targetdir)) if status != 0:
raise Exception("Failed to extract archive, output: %s" %
output)

meta/lib/oeqa/utils/targetbuild.py +0 -0
@@ -59,8 +60,4 @@ class TargetBuildProject():
         return self.target.run('cd %s; make install' % self.targetdir,

0)[0]

def clean(self):

meta/lib/oeqa/utils/targetbuild.py +0 -3
-        self.target.run('rm -r %s*' % self.targetdir)
-        try:
-            bb.process.run(['rm', '-rf', os.path.join(self.testdir,

self.archive)]) - except bb.process.CmdError:

meta/lib/oeqa/utils/targetbuild.py class TargetBuildProject(): · line 63

Quoted 2 lines
meta/lib/oeqa/utils/targetbuild.py +1 -1
-            bb.note("Failed to remove archive")
+        self.target.run('rm -rf %s' % self.targetdir)

Could you please squash this into the commit that adds this module?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#19 lore raw

PATCH 18/21 lib/oeqa/runtime: rpm: add install and erase tests

In reply to Stefan Stanacar

Copies to target rpm-doc file from deploy_dir and tries to install and then remove that package. rpm-doc was chosen because it's small, it only adds a few files to target, and it's almost always found in deploy_dir for images with package-management/rpm.

---
 meta/lib/oeqa/runtime/rpm.py | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
meta/lib/oeqa/runtime/rpm.py +27 -3
index 57101b0..154cad5 100644
@@ -1,6 +1,8 @@
 import unittest
+import os
 from oeqa.oetest import oeRuntimeTest, skipModule
 from oeqa.utils.decorators import *
+import oe.packagedata
 
 def setUpModule():
     if not oeRuntimeTest.hasFeature("package-management"):
@@ -9,17 +11,39 @@ def setUpModule():
             skipModule("rpm module skipped: target doesn't have rpm as primary package manager")
 
 
-class RpmHelpTest(oeRuntimeTest):
+class RpmBasicTest(oeRuntimeTest):
 
     @skipUnlessPassed('test_ssh')
     def test_rpm_help(self):
         (status, output) = self.target.run('rpm --help')
         self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
 
-class RpmQueryTest(oeRuntimeTest):
-
     @skipUnlessPassed('test_rpm_help')
     def test_rpm_query(self):
         (status, output) = self.target.run('rpm -q rpm')
         self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
 
+class RpmInstallRemoveTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        deploydir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True), "rpm", oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True))
+        pkgdata = oe.packagedata.read_subpkgdata("rpm-doc", oeRuntimeTest.tc.d)
+        # pick rpm-doc as a test file to get installed, because it's small and it will always be built for standard targets
+        testrpmfile = "rpm-doc-%s-%s.%s.rpm" % (pkgdata["PKGV"], pkgdata["PKGR"], oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True))
+        oeRuntimeTest.tc.target.copy_to(os.path.join(deploydir,testrpmfile), "/tmp/rpm-doc.rpm")
+
+    @skipUnlessPassed('test_rpm_help')
+    def test_rpm_install(self):
+        (status, output) = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
+        self.assertEqual(status, 0, msg="Failed to install rpm-doc package: %s" % output)
+
+    @skipUnlessPassed('test_rpm_install')
+    def test_rpm_remove(self):
+        (status,output) = self.target.run('rpm -e rpm-doc')
+        self.assertEqual(status, 0, msg="Failed to remove rpm-doc package: %s" % output)
+
+    @classmethod
+    def tearDownClass(self):
+        oeRuntimeTest.tc.target.run('rm -f /tmp/rpm-doc.rpm')
+
Signature · 1 line
1.8.3.1
#20 lore raw

PATCH 19/21 lib/oeqa: change behaviour for unskippable tests

In reply to Stefan Stanacar

When a test module wants to be skipped because it doesn't apply to the image but it was nevertheless a required
test (one in TEST_SUITES), we issued an warning that it
was a required test and went on with running the module. Usually all tests in the module failed (e.g gcc tests on a non-sdk image), but this allowed us to know that something went wrong with the image (some package/feature didn't make it).

However, instead of just issuing an warning and running the tests it's better to throw an exception. The traceback will tell us what's wrong, and we don't run every single test method.
Output will look like this:
--snip--

| NOTE: Test modules  ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
| NOTE: Found 5 tests
| test_ping (oeqa.runtime.ping.PingTest) ... ok
| test_ssh (oeqa.runtime.ssh.SshTest) ... ok
| ERROR
|
| ======================================================================
| ERROR: setUpModule (oeqa.runtime.gcc)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/runtime/gcc.py", line 8, in setUpModule
|     skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/oetest.py", line 108, in skipModule
|     "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
| Exception:
| Test gcc wants to be skipped.
| Reason is: Image doesn't have tools-sdk in IMAGE_FEATURES
| Test was required in TEST_SUITES, so either the condition for skipping is wrong
| or the image really doesn't have the requred feature/package when it should.
|
| ----------------------------------------------------------------------
| Ran 2 tests in 1.036s
|
| FAILED (errors=1)
| NOTE: Sending SIGTERM to runqemu

--snip--

---
 meta/lib/oeqa/oetest.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/lib/oeqa/oetest.py +3 -1
index 7f6baa4..e694c0b 100644
@@ -96,7 +96,9 @@ def skipModule(reason, pos=2):
     if modname not in oeRuntimeTest.tc.testsrequired:
         raise unittest.SkipTest("%s: %s" % (modname, reason))
     else:
-        bb.warn("Test %s is required, not skipping" % modname)
+        raise Exception("\nTest %s wants to be skipped.\nReason is: %s" \
+                "\nTest was required in TEST_SUITES, so either the condition for skipping is wrong" \
+                "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
 
 def skipModuleIf(cond, reason):
 
Signature · 1 line
1.8.3.1
#21 lore raw

PATCH 20/21 lib/oeqa/utils: qemurunner: improve kill and restart

In reply to Stefan Stanacar

Tweak QemuRunner so we can actually restart the qemu target in a test (if we want more memory for example).

---
 meta/lib/oeqa/utils/qemurunner.py | 28 +++++++++++++++++++++-------
 1 file changed, 21 insertions(+), 7 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +21 -7
index 9ae618f..6ee5b85 100644
@@ -32,6 +32,10 @@ class QemuRunner:
         self.boottime = boottime
         self.runqemutime = runqemutime
 
+        self.create_socket()
+
+    def create_socket(self):
+
         self.bootlog = ''
         self.qemusock = None
 
@@ -137,21 +141,31 @@ class QemuRunner:
         return self.is_alive()
 
     def kill(self):
+
+        if self.runqemu:
+            bb.note("Sending SIGTERM to runqemu")
+            os.kill(-self.runqemu.pid,signal.SIGTERM)
+            endtime = time.time() + self.runqemutime
+            while self.runqemu.poll() is None and time.time() < endtime:
+                time.sleep(1)
+            if self.runqemu.poll() is None:
+                bb.note("Sending SIGKILL to runqemu")
+                os.kill(-self.runqemu.pid,signal.SIGKILL)
+            self.runqemu = None
         if self.server_socket:
             self.server_socket.close()
             self.server_socket = None
-        if self.runqemu.pid:
-            os.kill(-self.runqemu.pid,signal.SIGTERM)
-            os.kill(-self.runqemu.pid,signal.SIGKILL)
-            self.runqemu.pid = None
         self.qemupid = None
         self.ip = None
 
     def restart(self, qemuparams = None):
-        if self.is_alive():
+        bb.note("Restarting qemu process")
+        if self.runqemu.poll() is None:
             self.kill()
-        bb.note("Qemu Restart required...")
-        return self.launch(qemuparams)
+        self.create_socket()
+        if self.launch(qemuparams):
+            return True
+        return False
 
     def is_alive(self):
         qemu_child = self.find_child(str(self.runqemu.pid))
Signature · 1 line
1.8.3.1
#22 lore raw

PATCH 21/21 lib/oeqa: add a restart method for base class and use it for build tests

In reply to Stefan Stanacar

This can be used by tests to restart the target (useful for passing extra qemuparams, like more RAM)

Build projects like cvs/iptables can take 15min+ on qemu targets with the default RAM, and sudoku fails with out of memory, so use this there.

---
 meta/lib/oeqa/oetest.py                | 7 +++++++
 meta/lib/oeqa/runtime/buildcvs.py      | 2 ++
 meta/lib/oeqa/runtime/buildiptables.py | 2 ++
 meta/lib/oeqa/runtime/buildsudoku.py   | 2 ++
 4 files changed, 13 insertions(+)
meta/lib/oeqa/oetest.py +7 -0
index e694c0b..c9dc5dc 100644
@@ -81,6 +81,13 @@ class oeRuntimeTest(unittest.TestCase):
         else:
             return False
 
+    @classmethod
+    def restartTarget(self,params=None):
+
+        if oeRuntimeTest.tc.qemu.restart(params):
+            oeRuntimeTest.tc.target.host = oeRuntimeTest.tc.qemu.ip
+        else:
+            raise Exception("Restarting target failed")
 
 
 def getmodule(pos=2):
meta/lib/oeqa/runtime/buildcvs.py +2 -0
index 9bf764d..f024dfa 100644
@@ -10,6 +10,7 @@ class BuildCvsTest(oeRuntimeTest):
 
     @classmethod
     def setUpClass(self):
+        self.restartTarget("-m 512")
         self.project = TargetBuildProject(oeRuntimeTest.tc.target,
                         "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2")
         self.project.download_archive()
@@ -28,3 +29,4 @@ class BuildCvsTest(oeRuntimeTest):
     @classmethod
     def tearDownClass(self):
         self.project.clean()
+        self.restartTarget()
meta/lib/oeqa/runtime/buildiptables.py +2 -0
index 50faf5d..88ece3b 100644
@@ -10,6 +10,7 @@ class BuildIptablesTest(oeRuntimeTest):
 
     @classmethod
     def setUpClass(self):
+        self.restartTarget("-m 512")
         self.project = TargetBuildProject(oeRuntimeTest.tc.target,
                         "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2")
         self.project.download_archive()
@@ -28,3 +29,4 @@ class BuildIptablesTest(oeRuntimeTest):
     @classmethod
     def tearDownClass(self):
         self.project.clean()
+        self.restartTarget()
meta/lib/oeqa/runtime/buildsudoku.py +2 -0
index 61dc1ff..0a7306d 100644
@@ -10,6 +10,7 @@ class SudokuTest(oeRuntimeTest):
 
     @classmethod
     def setUpClass(self):
+        self.restartTarget("-m 512")
         self.project = TargetBuildProject(oeRuntimeTest.tc.target,
                         "http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2")
         self.project.download_archive()
@@ -25,3 +26,4 @@ class SudokuTest(oeRuntimeTest):
     @classmethod
     def tearDownClass(self):
         self.project.clean()
+        self.restartTarget()
Signature · 1 line
1.8.3.1
#23 lore raw

Re: [PATCH 21/21] lib/oeqa: add a restart method for base class and use it for build tests

Quoted 83 lines · On Friday 23 August 2013 18:31:01 Stefan Stanacar wrote:

This can be used by tests to restart the target (useful for passing extra qemuparams, like more RAM)

Build projects like cvs/iptables can take 15min+ on qemu targets with the default RAM, and sudoku fails with out of memory, so use this there.

---
 meta/lib/oeqa/oetest.py                | 7 +++++++
 meta/lib/oeqa/runtime/buildcvs.py      | 2 ++
 meta/lib/oeqa/runtime/buildiptables.py | 2 ++
 meta/lib/oeqa/runtime/buildsudoku.py   | 2 ++
 4 files changed, 13 insertions(+)
meta/lib/oeqa/oetest.py +7 -0
index e694c0b..c9dc5dc 100644
@@ -81,6 +81,13 @@ class oeRuntimeTest(unittest.TestCase):
         else:
             return False
 
+    @classmethod
+    def restartTarget(self,params=None):
+
+        if oeRuntimeTest.tc.qemu.restart(params):
+            oeRuntimeTest.tc.target.host = oeRuntimeTest.tc.qemu.ip
+        else:
+            raise Exception("Restarting target failed")
 
 
 def getmodule(pos=2):
(quoted diff) +0 -0

b/meta/lib/oeqa/runtime/buildcvs.py index 9bf764d..f024dfa 100644

meta/lib/oeqa/runtime/buildcvs.py +1 -0
@@ -10,6 +10,7 @@ class BuildCvsTest(oeRuntimeTest):
 
     @classmethod
     def setUpClass(self):
+        self.restartTarget("-m 512")
         self.project = TargetBuildProject(oeRuntimeTest.tc.target,
 

"http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2" ) self.project.download_archive()

meta/lib/oeqa/runtime/buildcvs.py +1 -0
@@ -28,3 +29,4 @@ class BuildCvsTest(oeRuntimeTest):
     @classmethod
     def tearDownClass(self):
         self.project.clean()
+        self.restartTarget()
(quoted diff) +0 -0

b/meta/lib/oeqa/runtime/buildiptables.py index 50faf5d..88ece3b 100644

meta/lib/oeqa/runtime/buildiptables.py +1 -0
@@ -10,6 +10,7 @@ class BuildIptablesTest(oeRuntimeTest):
 
     @classmethod
     def setUpClass(self):
+        self.restartTarget("-m 512")
         self.project = TargetBuildProject(oeRuntimeTest.tc.target,
 
 "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2")

self.project.download_archive()

meta/lib/oeqa/runtime/buildiptables.py +1 -0
@@ -28,3 +29,4 @@ class BuildIptablesTest(oeRuntimeTest):
     @classmethod
     def tearDownClass(self):
         self.project.clean()
+        self.restartTarget()
(quoted diff) +0 -0

b/meta/lib/oeqa/runtime/buildsudoku.py index 61dc1ff..0a7306d 100644

meta/lib/oeqa/runtime/buildsudoku.py +1 -0
@@ -10,6 +10,7 @@ class SudokuTest(oeRuntimeTest):
 
     @classmethod
     def setUpClass(self):
+        self.restartTarget("-m 512")
         self.project = TargetBuildProject(oeRuntimeTest.tc.target,
 

"http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudok u-savant-1.3/sudoku-savant-1.3.tar.bz2") self.project.download_archive()

meta/lib/oeqa/runtime/buildsudoku.py class SudokuTest(oeRuntimeTest): · line 29

Quoted 4 lines
meta/lib/oeqa/runtime/buildsudoku.py +1 -0
@@ -25,3 +26,4 @@ class SudokuTest(oeRuntimeTest):
     @classmethod
     def tearDownClass(self):
         self.project.clean()
+        self.restartTarget()

Could you please you move the function addition earlier in the series and merge the test changes into the earlier commits that added them?

Thanks,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre