Skip to contentSkip to conversation

oeqa core and oe-selftest threaded enablement

PATCH Superseded Aníbal Limón 4 participants 52 messages Jul 11, 2017 – Jul 17, 2017

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover oeqa core and oe-selftest threaded enablement

This series is to enable oe-selftest threaded runs along some fixes,

* Implementation of main/end thread usage in oeqa threaded
* Adaptation of oe-selftest cases to be able to run in threaded env, ** Usage of own build directory by Test class.
** Mark the test modules that are enabled to run into a thread. ** Split some test modules because aren't support run into a thread
due to bitbake/tinfoil constraints.

The oe-selftest script now has an cmdline option (-t) to enable threaded runs, by default is set to 1, this needs to be set manually because depending on HW resources available can cause ran out of cpu/memory.

By default the oe-selftest cases runs on the main thread and uses the main build directory for compatibility purposes.

The following changes since commit 854c8c2e4c24ea3ddfec6e5b5f6477f0620510e4:

oeqa/tinfoil: Improve test_wait_event for race issues (2017-07-08 13:34:46 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib alimon/oe_selftest_threaded
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/oe_selftest_threaded

Aníbal Limón (30):
oeqa/core/loader: Switch method definition for _make_failed_test oeqa/selftest/{context,case}: Handle KeyboardInterrupt/SIGINT and
SIGTERM
selftest/cases/package: Call parent setUpClass method bb/tinfoil: run_command handle busy status in bitbake server oe/copy_buildsystem: check_sstate_task_list also pop BBPATH from env oeqa/core/threaded: Enable support to use the main thread oeqa/core/threaded: Add support to run into a thread at end of
execution
oeqa/core/threaded: logSummary add skipped tests info oeqa/core/tests: Update test_loader threaded to cover main thread
usage
oeqa/selftest/{case,context}: Add builddir by test class and context oeqa/selftest/case: Add wrappers to utils.commands modules oeqa/selftest/case: Creates meta-selftest layer per class oeqa/selftest/case: tearDown extra commands print what actually fails oeqa/selftest/case: Support bitbake memres mode in per build directory oeqa/selftest/cases: Use testlayer_path instead of call
get_test_layer()
oeqa/selftest/cases: Use builddir from class instead of get from
environment
oeqa/selftest/cases: Use wrapper methods from OESelfTestCase class oeqa/selftest/cases: tinfoil to run in the main thread oeqa/selftest/cases: imagefeatures enable threaded runs oeqa/selftest/cases: runqemu enable thraded runs oeqa/selftest/cases: runtime enable threaded runs oeqa/selftest/cases: eSDK enable threaded runs oeqa/selftest/cases: devtool enable threaded runs oeqa/selftest/cases: recipetool enable for threaded runs oeqa/selftest/cases: Move devtool deploy test case to own module selftest/cases/devtool{,end}: Move update/finish_modify tests to its
own module
seltest/cases/devtool: Build dbus on test_devtool_add_git_local
argparse_oe: Add int_positive type
oeqa/selftest/context: Enable support for threaded runs oeqa/selftest/cases: systemd_boot enable threaded runs

 bitbake/lib/bb/tinfoil.py                          |   23 +-
 .../lib/oeqa/selftest/cases/systemd_boot.py        |   26 +-
 meta/lib/oe/copy_buildsystem.py                    |    1 +
 meta/lib/oeqa/core/loader.py                       |    4 +-
 meta/lib/oeqa/core/tests/test_loader.py            |   17 +-
 meta/lib/oeqa/core/threaded.py                     |  169 +++-
 meta/lib/oeqa/selftest/case.py                     |  251 ++++-
 .../lib/oeqa/selftest/cases/_sstatetests_noauto.py |   23 +-
 meta/lib/oeqa/selftest/cases/archiver.py           |   21 +-
 meta/lib/oeqa/selftest/cases/bblayers.py           |   45 +-
 meta/lib/oeqa/selftest/cases/bbtests.py            |  103 +-
 meta/lib/oeqa/selftest/cases/buildhistory.py       |    9 +-
 meta/lib/oeqa/selftest/cases/buildoptions.py       |   51 +-
 meta/lib/oeqa/selftest/cases/containerimage.py     |    9 +-
 meta/lib/oeqa/selftest/cases/devtool.py            | 1014 +++++---------------
 meta/lib/oeqa/selftest/cases/devtool_deploy.py     |   93 ++
 meta/lib/oeqa/selftest/cases/devtool_end.py        |  506 ++++++++++
 meta/lib/oeqa/selftest/cases/distrodata.py         |    7 +-
 meta/lib/oeqa/selftest/cases/eSDK.py               |   61 +-
 meta/lib/oeqa/selftest/cases/image_typedep.py      |    8 +-
 meta/lib/oeqa/selftest/cases/imagefeatures.py      |   78 +-
 meta/lib/oeqa/selftest/cases/imagefeatures_boot.py |   63 ++
 meta/lib/oeqa/selftest/cases/layerappend.py        |   22 +-
 meta/lib/oeqa/selftest/cases/liboe.py              |   13 +-
 meta/lib/oeqa/selftest/cases/lic_checksum.py       |    8 +-
 meta/lib/oeqa/selftest/cases/manifest.py           |   13 +-
 meta/lib/oeqa/selftest/cases/oelib/buildhistory.py |    7 +-
 meta/lib/oeqa/selftest/cases/oescripts.py          |    5 +-
 meta/lib/oeqa/selftest/cases/package.py            |   12 +-
 meta/lib/oeqa/selftest/cases/pkgdata.py            |   73 +-
 meta/lib/oeqa/selftest/cases/prservice.py          |   19 +-
 meta/lib/oeqa/selftest/cases/recipetool.py         |  105 +-
 meta/lib/oeqa/selftest/cases/runqemu.py            |    7 +-
 meta/lib/oeqa/selftest/cases/runtime_test.py       |  225 +----
 .../lib/oeqa/selftest/cases/runtime_test_export.py |  104 ++
 .../oeqa/selftest/cases/runtime_test_postinsts.py  |  114 +++
 meta/lib/oeqa/selftest/cases/signing.py            |   40 +-
 meta/lib/oeqa/selftest/cases/sstate.py             |    5 +-
 meta/lib/oeqa/selftest/cases/sstatetests.py        |   51 +-
 meta/lib/oeqa/selftest/cases/tinfoil.py            |    1 -
 meta/lib/oeqa/selftest/context.py                  |  137 ++-
 scripts/lib/argparse_oe.py                         |    7 +
 42 files changed, 1997 insertions(+), 1553 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/devtool_deploy.py
 create mode 100644 meta/lib/oeqa/selftest/cases/devtool_end.py
 create mode 100644 meta/lib/oeqa/selftest/cases/imagefeatures_boot.py
 create mode 100644 meta/lib/oeqa/selftest/cases/runtime_test_export.py
 create mode 100644 meta/lib/oeqa/selftest/cases/runtime_test_postinsts.py
Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/30 oeqa/core/loader: Switch method definition for _make_failed_test

This was a mistake of me to define wrong what methods needs to be defined by certain python version.

See rev d8380d098a290510b442a7abd2dd5a50cabf5844.

---
 meta/lib/oeqa/core/loader.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/lib/oeqa/core/loader.py +2 -2
index 80e3d2800cd..3306e85d219 100644
@@ -13,7 +13,7 @@ from oeqa.core.decorator import decoratorClasses, OETestDecorator, \
         OETestFilter, OETestDiscover
 
 if sys.version_info >= (3,4,4):
-    def _make_failed_test(classname, methodname, exception, suiteClass):
+    def _make_failed_test(classname, exception, suiteClass):
         """
             When loading tests, the unittest framework stores any exceptions and
             displays them only when the 'run' method is called.
@@ -23,7 +23,7 @@ if sys.version_info >= (3,4,4):
         """
         raise exception
 else:
-    def _make_failed_test(classname, exception, suiteClass):
+    def _make_failed_test(classname, methodname, exception, suiteClass):
         raise exception
 unittest.loader._make_failed_test = _make_failed_test
 
Signature · 1 line
2.11.0
#7 lore raw

PATCH 2/30 oeqa/selftest/{context, case}: Handle KeyboardInterrupt/SIGINT and SIGTERM

In reply to Aníbal Limón

In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration.

[YOCTO #11650]

---
 meta/lib/oeqa/selftest/case.py    |  36 +++++++------
 meta/lib/oeqa/selftest/context.py | 107 +++++++++++++++++++++++++++-----------
 2 files changed, 97 insertions(+), 46 deletions(-)
meta/lib/oeqa/selftest/case.py +21 -15
index 31a11fddda9..871009c568b 100644
@@ -13,28 +13,34 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.core.case import OETestCase
 
 class OESelftestTestCase(OETestCase):
-    builddir = os.environ.get("BUILDDIR") or ""
-    localconf_path = os.path.join(builddir, "conf/local.conf")
-    localconf_backup = os.path.join(builddir, "conf/local.bk")
-    testinc_path = os.path.join(builddir, "conf/selftest.inc")
-    local_bblayers_path = os.path.join(builddir, "conf/bblayers.conf")
-    local_bblayers_backup = os.path.join(builddir, "conf/bblayers.bk")
-    testinc_bblayers_path = os.path.join(builddir, "conf/bblayers.inc")
-    machineinc_path = os.path.join(builddir, "conf/machine.inc")
-
     def __init__(self, methodName="runTest"):
         self._extra_tear_down_commands = []
-        self._track_for_cleanup = [
-            self.testinc_path, self.testinc_bblayers_path,
-            self.machineinc_path, self.localconf_backup,
-            self.local_bblayers_backup]
-
         super(OESelftestTestCase, self).__init__(methodName)
 
     @classmethod
     def setUpClass(cls):
         super(OESelftestTestCase, cls).setUpClass()
-        cls.testlayer_path = cls.tc.testlayer_path
+
+        cls.testlayer_path = cls.tc.config_paths['testlayer_path']
+        cls.builddir = cls.tc.config_paths['builddir']
+
+        cls.localconf_path = cls.tc.config_paths['localconf']
+        cls.localconf_backup = cls.tc.config_paths['localconf_class_backup']
+        cls.local_bblayers_path = cls.tc.config_paths['bblayers']
+        cls.local_bblayers_backup = cls.tc.config_paths['bblayers_class_backup']
+
+        cls.testinc_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/selftest.inc")
+        cls.testinc_bblayers_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/bblayers.inc")
+        cls.machineinc_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/machine.inc")
+
+        cls._track_for_cleanup = [
+            cls.testinc_path, cls.testinc_bblayers_path,
+            cls.machineinc_path, cls.localconf_backup,
+            cls.local_bblayers_backup]
+
         cls.add_include()
 
     @classmethod
meta/lib/oeqa/selftest/context.py +76 -31

150 lines not on this page · raw message

Signature · 1 line
2.11.0
#10 lore raw

PATCH 3/30 selftest/cases/package: Call parent setUpClass method

In reply to Aníbal Limón

Since config paths are now passed in Test context the setUpClass method is expected to be call.

---
 meta/lib/oeqa/selftest/cases/package.py | 2 ++
 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/cases/package.py +2 -0
index 5b9a6d15851..b07d6ed0abe 100644
@@ -17,6 +17,8 @@ class VersionOrdering(OESelftestTestCase):
 
     @classmethod
     def setUpClass(cls):
+        super(VersionOrdering, cls).setUpClass()
+
         # Build the tools we need and populate a sysroot
         bitbake("dpkg-native opkg-native rpm-native python3-native")
         bitbake("build-sysroots -c build_native_sysroot")
Signature · 1 line
2.11.0
#14 lore raw

PATCH 4/30 bb/tinfoil: run_command handle busy status in bitbake server

In reply to Aníbal Limón

When tinfoil request a command to bitbake is handled in async manner [1], sometimes is this ends on return a Busy status.

This is a workaround a needs to be fixed in proper manner inside bitbake code.

For example when running clientComplete and buildFile is on progress,

ERROR: Function failed: base_do_unpack
Traceback (most recent call last):
File "/home/alimon/repos/poky/scripts/lib/devtool/standard.py", line
797, in modify
initial_rev = _extract_source(srctree, args.keep_temp, args.branch,
False, rd, tinfoil)
File "/home/alimon/repos/poky/scripts/lib/devtool/standard.py", line
562, in _extract_source
runtask(fn, 'unpack')
File "/home/alimon/repos/poky/scripts/lib/devtool/standard.py", line
552, in runtask
raise DevtoolError('Task do_%s failed' % task)
devtool.DevtoolError: Task do_unpack failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/alimon/repos/poky/scripts/devtool", line 351, in <module>
ret = main()
File "/home/alimon/repos/poky/scripts/devtool", line 338, in main
ret = args.func(args, config, basepath, workspace)
File "/home/alimon/repos/poky/scripts/lib/devtool/standard.py", line
864, in modify
tinfoil.shutdown()
File "/home/alimon/repos/poky/bitbake/lib/bb/tinfoil.py", line 427, in
shutdown
self.run_command('clientComplete')
File "/home/alimon/repos/poky/bitbake/lib/bb/tinfoil.py", line 320, in
run_command
raise TinfoilCommandFailed(result[1])
bb.tinfoil.TinfoilCommandFailed: Busy (buildFile in progress)

---
 bitbake/lib/bb/tinfoil.py | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)
bitbake/lib/bb/tinfoil.py +19 -4
index fb0da622433..f3a1563e8f3 100644
@@ -21,6 +21,7 @@ import os
 import sys
 import atexit
 import re
+import time
 from collections import OrderedDict, defaultdict
 
 import bb.cache
@@ -298,7 +299,7 @@ class Tinfoil:
         config_params = TinfoilConfigParameters(config_only=False)
         self.run_actions(config_params)
 
-    def run_command(self, command, *params):
+    def run_command(self, command, *params, ntries=0):
         """
         Run a command on the server (as implemented in bb.command).
         Note that there are two types of command - synchronous and
@@ -315,9 +316,23 @@ class Tinfoil:
         commandline = [command]
         if params:
             commandline.extend(params)
-        result = self.server_connection.connection.runCommand(commandline)
-        if result[1]:
+
+        # XXX: Tinfoil commands are run by Cooker in async mode so gives
+        # some time to get done.
+        result = None
+        while True:
+            result = self.server_connection.connection.runCommand(commandline)
+            if not result[1]:
+                break
+            elif ntries == 0:
+                break
+            elif 'Busy' in result[1]:
+                ntries = ntries - 1
+                time.sleep(1)
+                continue
+
             raise TinfoilCommandFailed(result[1])
+
         return result[0]
 
     def set_event_mask(self, eventlist):
@@ -424,7 +439,7 @@ class Tinfoil:
 
     def shutdown(self):
         if self.server_connection:
-            self.run_command('clientComplete')
+            self.run_command('clientComplete', ntries=1)
             _server_connections.remove(self.server_connection)
             bb.event.ui_queue = []
             self.server_connection.terminate()
Signature · 1 line
2.11.0
#15 lore raw

PATCH 5/30 oe/copy_buildsystem: check_sstate_task_list also pop BBPATH from env

In reply to Aníbal Limón

The BBPATH environment could be set and can make a failure when try to build an extensible sdk because it will look the bitbake.lock file in the original build folder.

Example:

$ export BBPATH=`pwd`
$ bitbake core-image-minimal -c populate_sdk_ext

ERROR: bitbake failed:
ERROR: Only one copy of bitbake should be run against a build directory
ERROR: core-image-minimal-1.0-r0 do_populate_sdk_ext: Function failed:
copy_buildsystem

---
 meta/lib/oe/copy_buildsystem.py | 1 +
 1 file changed, 1 insertion(+)
meta/lib/oe/copy_buildsystem.py +1 -0
index a3729041835..dd506a39e6c 100644
@@ -239,6 +239,7 @@ def check_sstate_task_list(d, targets, filteroutfile, cmdprefix='', cwd=None, lo
     cmd = "%sBB_SETSCENE_ENFORCE=1 PSEUDO_DISABLED=1 oe-check-sstate %s -s -o %s %s" % (cmdprefix, targets, filteroutfile, logparam)
     env = dict(d.getVar('BB_ORIGENV', False))
     env.pop('BUILDDIR', '')
+    env.pop('BBPATH', '')
     pathitems = env['PATH'].split(':')
     env['PATH'] = ':'.join([item for item in pathitems if not item.endswith('/bitbake/bin')])
     bb.process.run(cmd, stderr=subprocess.STDOUT, env=env, cwd=cwd, executable='/bin/bash')
Signature · 1 line
2.11.0
#16 lore raw

PATCH 6/30 oeqa/core/threaded: Enable support to use the main thread

In reply to Aníbal Limón

Some test cases needs to be executed by the main thread for several resons, this implmentation enables usage of the main thread to execute suites.

The rules are if some test case request by test class attr _main_thread it will be executed, if no tests are scheduled to be executed into the main thread the algorithm with take the first suite in the pool, finallay this will avoid thread usage if only one suite needs to be executed.

---
 meta/lib/oeqa/core/threaded.py | 104 ++++++++++++++++++++++++++++++++---------
 1 file changed, 83 insertions(+), 21 deletions(-)
meta/lib/oeqa/core/threaded.py +83 -21
index 2cafe03a212..34217f1a8b8 100644
@@ -27,9 +27,42 @@ class OETestLoaderThreaded(OETestLoader):
             self.process_num = min(multiprocessing.cpu_count(),
                     len(suite._tests))
 
-        suites = []
-        for _ in range(self.process_num):
-            suites.append(self.suiteClass())
+        suites = {}
+        suites['main'] = self.suiteClass()
+        suites['pool'] = []
+        for _ in range(self.process_num - 1):
+            suites['pool'].append(self.suiteClass())
+
+        def _add_to_main_thread(main_suite, case, depends):
+            """
+                Some test cases needs to be run into the main
+                thread for several resons.
+
+                A test case that needs to run in the main thread
+                can be for specific set via test class _main_thread
+                attr or because is on the same module or for a dependency
+                reason.
+            """
+
+            if hasattr(case.__class__, '_main_thread') and \
+                    case.__class__._main_thread or \
+                    self.process_num == 1:
+                main_suite.addTest(case)
+                return True
+
+            for c in main_suite._tests:
+                if case.__module__ == c.__module__:
+                    main_suite.addTest(case)
+                    return True
+
+            if case.id() in depends:
+                case_depends = depends[case.id()]
+                for c in main_suite._tests:
+                    if c.id() in case_depends:
+                        main_suite.addTest(case)
+                        return True
+
+            return False
 
         def _search_for_module_idx(suites, case):
             """
@@ -37,8 +70,7 @@ class OETestLoaderThreaded(OETestLoader):
                 in the same thread because PyUnit keeps track
                 of setUp{Module, Class,} and tearDown{Module, Class,}.
             """
-
-            for idx in range(self.process_num):
+            for idx in range(self.process_num - 1):
                 suite = suites[idx]
                 for c in suite._tests:
                     if case.__module__ == c.__module__:
@@ -53,7 +85,7 @@ class OETestLoaderThreaded(OETestLoader):
                 of dependant test to figure out if skip or not.
             """
 
-            for idx in range(self.process_num):
+            for idx in range(self.process_num - 1):
                 suite = suites[idx]
 
                 for case in suite._tests:
@@ -62,6 +94,11 @@ class OETestLoaderThreaded(OETestLoader):
             return -1
 
         def _get_best_idx(suites):
+            """
+                The best index is selected to the suite that has
+                minor test cases to run.
+            """
+
             sizes = [len(suite._tests) for suite in suites]
             return sizes.index(min(sizes))
 
@@ -71,27 +108,35 @@ class OETestLoaderThreaded(OETestLoader):
                 if isinstance(case, TestSuite):
                     _fill_suites(case)
                 else:
-                    idx = _search_for_module_idx(suites, case)
-
                     depends = {}
                     if 'depends' in self.tc._registry:
                         depends = self.tc._registry['depends']
 
+                    if _add_to_main_thread(suites['main'], case, depends):
+                        continue
+
+                    # Get the best index in the suite pool to add the case
+                    idx = _search_for_module_idx(suites['pool'], case)
                     if idx == -1 and case.id() in depends:
                         case_depends = depends[case.id()] 
-                        idx = _search_for_depend_idx(suites, case_depends)
-
+                        idx = _search_for_depend_idx(suites['pool'], case_depends)
                     if idx == -1:
-                        idx = _get_best_idx(suites)
+                        idx = _get_best_idx(suites['pool'])
+                    suites['pool'][idx].addTest(case)
 
-                    suites[idx].addTest(case)
         _fill_suites(suite)
 
-        suites_tmp = suites
-        suites = []
+        # clean suites in pool without test cases
+        suites_tmp = suites['pool']
+        suites['pool'] = []
         for suite in suites_tmp:
             if len(suite._tests) > 0:
-                suites.append(suite)
+                suites['pool'].append(suite)
+
+        # if the main suite doesn't have test cases
+        # use the first element of the suites pool
+        if not len(suites['main']._tests):
+            suites['main'] = suites['pool'].pop(0)
 
         return suites
 
@@ -251,16 +296,33 @@ class OETestRunnerThreaded(OETestRunner):
     def run(self, suites):
         result = OETestResultThreaded(self.tc)
 
-        pool = _ThreadedPool(len(suites), len(suites), stream=self.stream,
-                result=result)
-        for s in suites:
-            pool.add_task(super(OETestRunnerThreaded, self).run, s)
-        pool.start()
-        pool.wait_completion()
+        pool = None
+        if suites['pool']:
+            thread_no = len(suites['pool'])
+            pool = _ThreadedPool(thread_no, thread_no, stream=self.stream,
+                    result=result)
+            for s in suites['pool']:
+                pool.add_task(super(OETestRunnerThreaded, self).run, s)
+            pool.start()
+
+        run_start_time = time.time()
+        rc = super(OETestRunnerThreaded, self).run(suites['main'])
+        run_end_time = time.time()
+        result.addResult(rc, run_start_time, run_end_time)
+        self.stream.finish()
+
+        if pool:
+            pool.wait_completion()
         result._fill_tc_results()
 
         return result
 
+    def list_tests(self, suite, display_type):
+        suite['pool'].insert(0, suite['main'])
+
+        return super(OETestRunnerThreaded, self).list_tests(
+                suite['pool'], display_type)
+
 class OETestContextThreaded(OETestContext):
     loaderClass = OETestLoaderThreaded
     runnerClass = OETestRunnerThreaded
Signature · 1 line
2.11.0
#17 lore raw

PATCH 7/30 oeqa/core/threaded: Add support to run into a thread at end of execution

In reply to Aníbal Limón

Some test cases aren't allowed to run into a multi-thread environment so add the posibility to run those tests at end of execution.

---
 meta/lib/oeqa/core/threaded.py | 102 +++++++++++++++++++++++++++++++----------
 1 file changed, 78 insertions(+), 24 deletions(-)
meta/lib/oeqa/core/threaded.py +78 -24
index 34217f1a8b8..a7dc0aed401 100644
@@ -30,18 +30,35 @@ class OETestLoaderThreaded(OETestLoader):
         suites = {}
         suites['main'] = self.suiteClass()
         suites['pool'] = []
+        suites['end'] = self.suiteClass()
         for _ in range(self.process_num - 1):
             suites['pool'].append(self.suiteClass())
 
+        def _add_by_module_or_dep(suite, case, depends):
+            """
+                A test case that needs to run into the same thread
+                because is on the same module or for dependency
+                reasons.
+            """
+
+            for c in suite._tests:
+                if case.__module__ == c.__module__:
+                    suite.addTest(case)
+                    return True
+
+            if case.id() in depends:
+                case_depends = depends[case.id()]
+                for c in suite._tests:
+                    if c.id() in case_depends:
+                        suite.addTest(case)
+                        return True
+
+            return False
+
         def _add_to_main_thread(main_suite, case, depends):
             """
                 Some test cases needs to be run into the main
-                thread for several resons.
-
-                A test case that needs to run in the main thread
-                can be for specific set via test class _main_thread
-                attr or because is on the same module or for a dependency
-                reason.
+                thread by request.
             """
 
             if hasattr(case.__class__, '_main_thread') and \
@@ -50,19 +67,20 @@ class OETestLoaderThreaded(OETestLoader):
                 main_suite.addTest(case)
                 return True
 
-            for c in main_suite._tests:
-                if case.__module__ == c.__module__:
-                    main_suite.addTest(case)
-                    return True
+            return _add_by_module_or_dep(main_suite, case, depends)
 
-            if case.id() in depends:
-                case_depends = depends[case.id()]
-                for c in main_suite._tests:
-                    if c.id() in case_depends:
-                        main_suite.addTest(case)
-                        return True
+        def _add_to_end_thread(end_suite, case, depends):
+            """
+                Some test cases needs to be run into at end of
+                execution into the main by request.
+            """
+            if hasattr(case.__class__, '_end_thread') and \
+                    case.__class__._end_thread or \
+                    self.process_num == 1:
+                end_suite.addTest(case)
+                return True
 
-            return False
+            return _add_by_module_or_dep(end_suite, case, depends)
 
         def _search_for_module_idx(suites, case):
             """
@@ -112,6 +130,9 @@ class OETestLoaderThreaded(OETestLoader):
                     if 'depends' in self.tc._registry:
                         depends = self.tc._registry['depends']
 
+                    if _add_to_end_thread(suites['end'], case, depends):
+                        continue
+
                     if _add_to_main_thread(suites['main'], case, depends):
                         continue
 
@@ -135,7 +156,7 @@ class OETestLoaderThreaded(OETestLoader):
 
         # if the main suite doesn't have test cases
         # use the first element of the suites pool
-        if not len(suites['main']._tests):
+        if not len(suites['main']._tests) and len(suites['pool']):
             suites['main'] = suites['pool'].pop(0)
 
         return suites
@@ -268,6 +289,12 @@ class _ThreadedPool:
         self.tasks = queue.Queue(num_tasks)
         self.workers = []
 
+        self.stream = stream
+        self.result = result
+
+        self.end_task = None
+        self.end_worker = None
+
         for _ in range(num_workers):
             worker = _Worker(self.tasks, result, stream)
             self.workers.append(worker)
@@ -280,12 +307,25 @@ class _ThreadedPool:
         """Add a task to the queue"""
         self.tasks.put((func, args, kargs))
 
+    def add_end_task(self, func, *args, **kwargs):
+        """Add a task to be executed at end"""
+
+        self.end_task = queue.Queue(1)
+        self.end_task.put((func, args, kwargs))
+        self.end_worker = _Worker(self.end_task, self.result,
+                self.stream)
+
     def wait_completion(self):
         """Wait for completion of all the tasks in the queue"""
         self.tasks.join()
         for worker in self.workers:
             worker.join()
 
+        if self.end_task:
+            self.end_worker.start()
+            self.end_task.join()
+            self.end_worker.join()
+
 class OETestRunnerThreaded(OETestRunner):
     streamLoggerClass = OEStreamLoggerThreaded
 
@@ -293,32 +333,46 @@ class OETestRunnerThreaded(OETestRunner):
         super(OETestRunnerThreaded, self).__init__(tc, *args, **kwargs)
         self.resultclass = OETestResultThreadedInternal # XXX: XML reporting overrides at __init__
 
+    def _run_main_thread(self, suite, result):
+        if len(suite._tests):
+            run_start_time = time.time()
+            rc = super(OETestRunnerThreaded, self).run(suite)
+            run_end_time = time.time()
+            result.addResult(rc, run_start_time, run_end_time)
+            self.stream.finish()
+
     def run(self, suites):
         result = OETestResultThreaded(self.tc)
 
         pool = None
+
         if suites['pool']:
             thread_no = len(suites['pool'])
             pool = _ThreadedPool(thread_no, thread_no, stream=self.stream,
                     result=result)
             for s in suites['pool']:
                 pool.add_task(super(OETestRunnerThreaded, self).run, s)
-            pool.start()
 
-        run_start_time = time.time()
-        rc = super(OETestRunnerThreaded, self).run(suites['main'])
-        run_end_time = time.time()
-        result.addResult(rc, run_start_time, run_end_time)
-        self.stream.finish()
+        if len(suites['end']._tests):
+            if not pool:
+                pool = _ThreadedPool(0, 0, stream=self.stream,
+                            result=result)
+            pool.add_end_task(super(OETestRunnerThreaded, self).run,
+                    suites['end'])
 
         if pool:
+            pool.start()
+        self._run_main_thread(suites['main'], result)
+        if pool:
             pool.wait_completion()
+
         result._fill_tc_results()
 
         return result
 
     def list_tests(self, suite, display_type):
         suite['pool'].insert(0, suite['main'])
+        suite['pool'].append(suite['end'])
 
         return super(OETestRunnerThreaded, self).list_tests(
                 suite['pool'], display_type)
Signature · 1 line
2.11.0
#20 lore raw

PATCH 8/30 oeqa/core/threaded: logSummary add skipped tests info

In reply to Aníbal Limón
---
 meta/lib/oeqa/core/threaded.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
meta/lib/oeqa/core/threaded.py +11 -0
index a7dc0aed401..022c9ac59a2 100644
@@ -234,6 +234,14 @@ class OETestResultThreaded(object):
                     self._results[tid]['result'].wasSuccessful()
         return wasSuccessful
 
+    def getSkippedTests(self):
+        skipped = 0
+
+        for tid in self._results.keys():
+            skipped = skipped + len(self.tc._results[tid]['skipped'])
+
+        return skipped
+
     def stop(self):
         for tid in self._results.keys():
             self._results[tid]['result'].stop()
@@ -248,6 +256,9 @@ class OETestResultThreaded(object):
             msg = "%s - OK - All required tests passed" % component
         else:
             msg = "%s - FAIL - Required tests failed" % component
+        skipped = self.getSkippedTests()
+        if skipped:
+            msg += " (skipped=%d)" % skipped
         self.tc.logger.info(msg)
 
     def logDetails(self):
Signature · 1 line
2.11.0
#21 lore raw

PATCH 9/30 oeqa/core/tests: Update test_loader threaded to cover main thread usage

In reply to Aníbal Limón
---
 meta/lib/oeqa/core/tests/test_loader.py | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
meta/lib/oeqa/core/tests/test_loader.py +10 -7
index e0d917d317d..c4d5eeb462b 100755
@@ -88,25 +88,28 @@ class TestLoader(TestBase):
         self.cases_path = [os.path.join(self.cases_path, 'loader', 'threaded')]
 
         tc = self._testLoaderThreaded()
-        self.assertEqual(len(tc.suites), 3, "Expected to be 3 suites")
+        self.assertTrue(len(tc.suites['main']._tests),
+                "Expected to have tests in main suite")
+        self.assertEqual(len(tc.suites['pool']), 2,
+                "Expected to be 2 suites in pool")
 
         case_ids = ['threaded.ThreadedTest.test_threaded_no_depends',
                 'threaded.ThreadedTest2.test_threaded_same_module',
                 'threaded_depends.ThreadedTest3.test_threaded_depends']
-        for case in tc.suites[0]._tests:
+        for case in tc.suites['main']._tests:
             self.assertEqual(case.id(),
-                    case_ids[tc.suites[0]._tests.index(case)])
+                    case_ids[tc.suites['main']._tests.index(case)])
 
         case_ids = ['threaded_alone.ThreadedTestAlone.test_threaded_alone']
-        for case in tc.suites[1]._tests:
+        for case in tc.suites['pool'][0]._tests:
             self.assertEqual(case.id(),
-                    case_ids[tc.suites[1]._tests.index(case)])
+                    case_ids[tc.suites['pool'][0]._tests.index(case)])
 
         case_ids = ['threaded_module.ThreadedTestModule.test_threaded_module',
                 'threaded_module.ThreadedTestModule2.test_threaded_module2']
-        for case in tc.suites[2]._tests:
+        for case in tc.suites['pool'][1]._tests:
             self.assertEqual(case.id(),
-                    case_ids[tc.suites[2]._tests.index(case)])
+                    case_ids[tc.suites['pool'][1]._tests.index(case)])
 
         self.cases_path = cases_path
 
Signature · 1 line
2.11.0
#22 lore raw

PATCH 10/30 oeqa/selftest/{case, context}: Add builddir by test class and context

In reply to Aníbal Limón

The build directory by Test class will enable to use several instances of bitbake in parallel to enable oe-selftest threaded runs.

[YOCTO #11429]

---
 meta/lib/oeqa/selftest/case.py    | 97 ++++++++++++++++++++++++++++-----------
 meta/lib/oeqa/selftest/context.py |  4 ++
 2 files changed, 74 insertions(+), 27 deletions(-)
meta/lib/oeqa/selftest/case.py +70 -27
index 871009c568b..dd24e366abd 100644
@@ -13,28 +13,73 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.core.case import OETestCase
 
 class OESelftestTestCase(OETestCase):
+    _use_own_builddir = False
+    _main_thread = True
+    _end_thread = False
+
     def __init__(self, methodName="runTest"):
         self._extra_tear_down_commands = []
         super(OESelftestTestCase, self).__init__(methodName)
 
     @classmethod
+    def _setUpBuildDir(cls):
+        if cls._use_own_builddir:
+            cls.builddir = os.path.join(cls.tc.config_paths['base_builddir'],
+                    cls.__module__, cls.__name__)
+
+            cls.localconf_path = os.path.join(cls.builddir, "conf/local.conf")
+            cls.localconf_backup = os.path.join(cls.builddir,
+                    "conf/local.conf.bk")
+            cls.local_bblayers_path = os.path.join(cls.builddir,
+                    "conf/bblayers.conf")
+            cls.local_bblayers_backup = os.path.join(cls.builddir,
+                    "conf/bblayers.conf.bk")
+        else:
+            cls.builddir = cls.tc.config_paths['builddir']
+            cls.localconf_path = cls.tc.config_paths['localconf']
+            cls.localconf_backup = cls.tc.config_paths['localconf_class_backup']
+            cls.local_bblayers_path = cls.tc.config_paths['bblayers']
+            cls.local_bblayers_backup = \
+                    cls.tc.config_paths['bblayers_class_backup']
+
+        cls.testinc_path = os.path.join(cls.builddir, "conf/selftest.inc")
+        cls.testinc_bblayers_path = os.path.join(cls.builddir,
+                "conf/bblayers.inc")
+        cls.machineinc_path = os.path.join(cls.builddir, "conf/machine.inc")
+
+        # creates a custom build directory for every test class
+        if not os.path.exists(cls.builddir):
+            os.makedirs(cls.builddir)
+
+            builddir_conf = os.path.join(cls.builddir, 'conf')
+            origdir_conf = os.path.join(cls.tc.config_paths['builddir'], 'conf')
+            shutil.copytree(origdir_conf, builddir_conf)
+
+            ftools.append_file(cls.localconf_path, "# added by oe-selftest base class")
+
+            # shares original sstate_dir across build directories to speed up
+            sstate_line = "SSTATE_DIR=\"%s\"" % cls.td['SSTATE_DIR']
+            ftools.append_file(cls.localconf_path, sstate_line)
+
+            # shares original dl_dir across build directories to avoid additional
+            # network usage
+            dldir_line = "DL_DIR=\"%s\"" % cls.td['DL_DIR']
+            ftools.append_file(cls.localconf_path, dldir_line)
+
+            # use the same value of threads for BB_NUMBER_THREADS/PARALLEL_MAKE
+            # to avoid ran out resources (cpu/memory)
+            if hasattr(cls.tc.loader, 'process_num'):
+                ftools.append_file(cls.localconf_path, "BB_NUMBER_THREADS?=\"%d\"" %
+                        cls.tc.loader.process_num)
+                ftools.append_file(cls.localconf_path, "PARALLEL_MAKE?=\"-j %d\"" %
+                        cls.tc.loader.process_num)
+
+    @classmethod
     def setUpClass(cls):
         super(OESelftestTestCase, cls).setUpClass()
 
         cls.testlayer_path = cls.tc.config_paths['testlayer_path']
-        cls.builddir = cls.tc.config_paths['builddir']
-
-        cls.localconf_path = cls.tc.config_paths['localconf']
-        cls.localconf_backup = cls.tc.config_paths['localconf_class_backup']
-        cls.local_bblayers_path = cls.tc.config_paths['bblayers']
-        cls.local_bblayers_backup = cls.tc.config_paths['bblayers_class_backup']
-
-        cls.testinc_path = os.path.join(cls.tc.config_paths['builddir'],
-                "conf/selftest.inc")
-        cls.testinc_bblayers_path = os.path.join(cls.tc.config_paths['builddir'],
-                "conf/bblayers.inc")
-        cls.machineinc_path = os.path.join(cls.tc.config_paths['builddir'],
-                "conf/machine.inc")
+        cls._setUpBuildDir()
 
         cls._track_for_cleanup = [
             cls.testinc_path, cls.testinc_bblayers_path,
@@ -52,35 +97,31 @@ class OESelftestTestCase(OETestCase):
     @classmethod
     def add_include(cls):
         if "#include added by oe-selftest" \
-            not in ftools.read_file(os.path.join(cls.builddir, "conf/local.conf")):
-                cls.logger.info("Adding: \"include selftest.inc\" in %s" % os.path.join(cls.builddir, "conf/local.conf"))
-                ftools.append_file(os.path.join(cls.builddir, "conf/local.conf"), \
+            not in ftools.read_file(cls.localconf_path):
+                ftools.append_file(cls.localconf_path, \
                         "\n#include added by oe-selftest\ninclude machine.inc\ninclude selftest.inc")
 
         if "#include added by oe-selftest" \
-            not in ftools.read_file(os.path.join(cls.builddir, "conf/bblayers.conf")):
-                cls.logger.info("Adding: \"include bblayers.inc\" in bblayers.conf")
-                ftools.append_file(os.path.join(cls.builddir, "conf/bblayers.conf"), \
+            not in ftools.read_file(cls.local_bblayers_path):
+                ftools.append_file(cls.local_bblayers_path, \
                         "\n#include added by oe-selftest\ninclude bblayers.inc")
 
     @classmethod
     def remove_include(cls):
         if "#include added by oe-selftest.py" \
-            in ftools.read_file(os.path.join(cls.builddir, "conf/local.conf")):
-                cls.logger.info("Removing the include from local.conf")
-                ftools.remove_from_file(os.path.join(cls.builddir, "conf/local.conf"), \
+            in ftools.read_file(cls.localconf_path):
+                ftools.remove_from_file(cls.localconf_path, \
                         "\n#include added by oe-selftest.py\ninclude machine.inc\ninclude selftest.inc")
 
         if "#include added by oe-selftest.py" \
-            in ftools.read_file(os.path.join(cls.builddir, "conf/bblayers.conf")):
-                cls.logger.info("Removing the include from bblayers.conf")
-                ftools.remove_from_file(os.path.join(cls.builddir, "conf/bblayers.conf"), \
+            in ftools.read_file(cls.local_bblayers_path):
+                ftools.remove_from_file(cls.local_bblayers_path, \
                         "\n#include added by oe-selftest.py\ninclude bblayers.inc")
 
     @classmethod
     def remove_inc_files(cls):
         try:
-            os.remove(os.path.join(cls.builddir, "conf/selftest.inc"))
+            os.remove(cls.testinc_path)
             for root, _, files in os.walk(cls.testlayer_path):
                 for f in files:
                     if f == 'test_recipe.inc':
@@ -96,7 +137,7 @@ class OESelftestTestCase(OETestCase):
 
     def setUp(self):
         super(OESelftestTestCase, self).setUp()
-        os.chdir(self.builddir)
+
         # Check if local.conf or bblayers.conf files backup exists
         # from a previous failed test and restore them
         if os.path.isfile(self.localconf_backup) or os.path.isfile(
@@ -239,6 +280,7 @@ to ensure accurate results.")
         except OSError as e:
             if e.errno != errno.ENOENT:
                 raise
+
     def write_bblayers_config(self, data):
         """Write to <builddir>/conf/bblayers.inc"""
         self.logger.debug("Writing to: %s\n%s\n" % (self.testinc_bblayers_path, data))
@@ -269,4 +311,5 @@ to ensure accurate results.")
     def assertNotExists(self, expr, msg=None):
         if os.path.exists(expr):
             msg = self._formatMessage(msg, "%s exists when it should not" % safe_repr(expr))
+
             raise self.failureException(msg)
meta/lib/oeqa/selftest/context.py +4 -0
index 4575a0537fb..697ea0b4933 100644
@@ -8,6 +8,7 @@ import sys
 import imp
 import signal
 from shutil import copyfile
+import tempfile
 from random import choice
 
 import oeqa
@@ -133,6 +134,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         copyfile(self.tc_kwargs['init']['config_paths']['bblayers'], 
                 self.tc_kwargs['init']['config_paths']['bblayers_backup'])
 
+        self.tc_kwargs['init']['config_paths']['base_builddir'] = \
+                tempfile.mkdtemp(prefix='build-selftest-', dir=builddir)
+
     def _pre_run(self):
         def _check_required_env_variables(vars):
             for var in vars:
Signature · 1 line
2.11.0
#23 lore raw

PATCH 11/30 oeqa/selftest/case: Add wrappers to utils.commands modules

In reply to Aníbal Limón

This methods are wrappers to use builddir by Test class.

---
 meta/lib/oeqa/selftest/case.py | 96 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 94 insertions(+), 2 deletions(-)
meta/lib/oeqa/selftest/case.py +94 -2
index dd24e366abd..3998aeac5c4 100644
@@ -6,10 +6,11 @@ import os
 import shutil
 import glob
 import errno
+import re
 from unittest.util import safe_repr
 
 import oeqa.utils.ftools as ftools
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+from oeqa.utils.commands import runCmd, bitbake, get_bb_env, get_bb_var, get_bb_vars
 from oeqa.core.case import OETestCase
 
 class OESelftestTestCase(OETestCase):
@@ -197,7 +198,7 @@ to ensure accurate results.")
         if self._extra_tear_down_commands:
             failed_extra_commands = []
             for command in self._extra_tear_down_commands:
-                result = runCmd(command, ignore_status=True)
+                result = self.runCmd(command, ignore_status=True)
                 if not result.status ==  0:
                     failed_extra_commands.append(command)
             if failed_extra_commands:
@@ -313,3 +314,94 @@ to ensure accurate results.")
             msg = self._formatMessage(msg, "%s exists when it should not" % safe_repr(expr))
 
             raise self.failureException(msg)
+
+    # utils commands to run on it's on builddir
+    @classmethod 
+    def _env_own_builddir(cls, **kwargs):
+        env = None
+
+        if 'env' in kwargs:
+            env = kwargs['env']
+
+            if not 'BUILDDIR' in env:
+                env['BUILDDIR'] = cls.builddir
+            if not 'BBPATH' in env:
+                env['BBPATH'] = cls.builddir
+
+        else:
+            env = os.environ.copy()
+            env['BUILDDIR'] = cls.builddir
+            env['BBPATH'] = cls.builddir
+
+        kwargs['env'] = env
+
+        # XXX: tinfoil doesn't honor BBPATH bblayers and tinfoil test
+        # modules uses it
+        if not 'cwd' in kwargs:
+            kwargs['cwd'] = cls.builddir
+
+        # XXX: uncomment for debugging purposes
+        #kwargs['output_log'] = cls.logger
+
+        return kwargs
+
+    @classmethod
+    def runCmd(cls, *args, **kwargs):
+        kwargs = cls._env_own_builddir(**kwargs)
+        return runCmd(*args, **kwargs)
+
+    @classmethod
+    def bitbake(cls, *args, **kwargs):
+        kwargs = cls._env_own_builddir(**kwargs)
+        return bitbake(*args, **kwargs)
+
+    @classmethod
+    def get_bb_env(cls, target=None, postconfig=None):
+        if target:
+            return cls.bitbake("-e %s" % target, postconfig=postconfig).output
+        else:
+            return cls.bitbake("-e", postconfig=postconfig).output
+
+    @classmethod
+    def get_bb_vars(cls, variables=None, target=None, postconfig=None):
+        """Get values of multiple bitbake variables"""
+        bbenv = cls.get_bb_env(target, postconfig=postconfig)
+
+        if variables is not None:
+            variables = variables.copy()
+        var_re = re.compile(r'^(export )?(?P<var>\w+(_.*)?)="(?P<value>.*)"$')
+        unset_re = re.compile(r'^unset (?P<var>\w+)$')
+        lastline = None
+        values = {}
+        for line in bbenv.splitlines():
+            match = var_re.match(line)
+            val = None
+            if match:
+                val = match.group('value')
+            else:
+                match = unset_re.match(line)
+                if match:
+                    # Handle [unexport] variables
+                    if lastline.startswith('#   "'):
+                        val = lastline.split('"')[1]
+            if val:
+                var = match.group('var')
+                if variables is None:
+                    values[var] = val
+                else:
+                    if var in variables:
+                        values[var] = val
+                        variables.remove(var)
+                    # Stop after all required variables have been found
+                    if not variables:
+                        break
+            lastline = line
+        if variables:
+            # Fill in missing values
+            for var in variables:
+                values[var] = None
+        return values
+    
+    @classmethod 
+    def get_bb_var(cls, var, target=None, postconfig=None):
+        return cls.get_bb_vars([var], target, postconfig)[var]
Signature · 1 line
2.11.0
#24 lore raw

PATCH 12/30 oeqa/selftest/case: Creates meta-selftest layer per class

In reply to Aníbal Limón

The meta-selftest layer is used by test cases to modify meta data but in a threaded environment two test cases can modify the meta data causing errors because the signatures will change.

---
 meta/lib/oeqa/selftest/case.py | 34 +++++++++++++++++++++++++++++++---
 1 file changed, 31 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/case.py +31 -3
index 3998aeac5c4..bbdce4cf9e8 100644
@@ -7,6 +7,7 @@ import shutil
 import glob
 import errno
 import re
+import subprocess
 from unittest.util import safe_repr
 
 import oeqa.utils.ftools as ftools
@@ -24,6 +25,8 @@ class OESelftestTestCase(OETestCase):
 
     @classmethod
     def _setUpBuildDir(cls):
+        cls.orig_testlayer_path = cls.tc.config_paths['testlayer_path']
+
         if cls._use_own_builddir:
             cls.builddir = os.path.join(cls.tc.config_paths['base_builddir'],
                     cls.__module__, cls.__name__)
@@ -35,14 +38,23 @@ class OESelftestTestCase(OETestCase):
                     "conf/bblayers.conf")
             cls.local_bblayers_backup = os.path.join(cls.builddir,
                     "conf/bblayers.conf.bk")
+
+            cls.base_testlayer_path = os.path.join(cls.builddir,
+                    'layers')
+            cls.testlayer_path = os.path.join(cls.base_testlayer_path,
+                    os.path.basename(cls.orig_testlayer_path))
         else:
             cls.builddir = cls.tc.config_paths['builddir']
+
             cls.localconf_path = cls.tc.config_paths['localconf']
             cls.localconf_backup = cls.tc.config_paths['localconf_class_backup']
             cls.local_bblayers_path = cls.tc.config_paths['bblayers']
             cls.local_bblayers_backup = \
                     cls.tc.config_paths['bblayers_class_backup']
 
+            cls.base_testlayer_path = os.path.dirname(cls.orig_testlayer_path)
+            cls.testlayer_path = cls.orig_testlayer_path
+
         cls.testinc_path = os.path.join(cls.builddir, "conf/selftest.inc")
         cls.testinc_bblayers_path = os.path.join(cls.builddir,
                 "conf/bblayers.inc")
@@ -53,8 +65,11 @@ class OESelftestTestCase(OETestCase):
             os.makedirs(cls.builddir)
 
             builddir_conf = os.path.join(cls.builddir, 'conf')
-            origdir_conf = os.path.join(cls.tc.config_paths['builddir'], 'conf')
-            shutil.copytree(origdir_conf, builddir_conf)
+            os.makedirs(builddir_conf)
+            shutil.copyfile(cls.tc.config_paths['localconf_backup'],
+                    os.path.join(builddir_conf, 'local.conf'))
+            shutil.copyfile(cls.tc.config_paths['bblayers_backup'],
+                    os.path.join(builddir_conf, 'bblayers.conf'))
 
             ftools.append_file(cls.localconf_path, "# added by oe-selftest base class")
 
@@ -75,11 +90,24 @@ class OESelftestTestCase(OETestCase):
                 ftools.append_file(cls.localconf_path, "PARALLEL_MAKE?=\"-j %d\"" %
                         cls.tc.loader.process_num)
 
+            # copy meta-selftest per class to avoid races when changing meta-data
+            # and init git repository because some tests review the repo status
+            os.makedirs(cls.base_testlayer_path)
+            shutil.copytree(cls.orig_testlayer_path, cls.testlayer_path)
+            cls.runCmd("git init; git add *; git commit -a -m 'initial'",
+                            cwd=cls.testlayer_path)
+
+            # XXX: sometimes meta-selftest isn't on bblayers at first backup
+            try:
+                cls.runCmd("bitbake-layers remove-layer %s" % cls.orig_testlayer_path)
+            except:
+                pass
+            cls.runCmd("bitbake-layers add-layer %s" % cls.testlayer_path)
+
     @classmethod
     def setUpClass(cls):
         super(OESelftestTestCase, cls).setUpClass()
 
-        cls.testlayer_path = cls.tc.config_paths['testlayer_path']
         cls._setUpBuildDir()
 
         cls._track_for_cleanup = [
Signature · 1 line
2.11.0
#25 lore raw

PATCH 13/30 oeqa/selftest/case: tearDown extra commands print what actually fails

In reply to Aníbal Limón

Its better to have the output to see what actually fails in a command that is aim to execute at end of a test case.

---
 meta/lib/oeqa/selftest/case.py | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/case.py +9 -4
index bbdce4cf9e8..c6f2d184ea3 100644
@@ -224,13 +224,18 @@ to ensure accurate results.")
 
     def tearDown(self):
         if self._extra_tear_down_commands:
-            failed_extra_commands = []
+            failed_extra_commands = {}
             for command in self._extra_tear_down_commands:
                 result = self.runCmd(command, ignore_status=True)
-                if not result.status ==  0:
-                    failed_extra_commands.append(command)
+                if not result.status == 0:
+                    failed_extra_commands[command] = result
             if failed_extra_commands:
-                self.logger.warning("tearDown commands have failed: %s" % ', '.join(map(str, failed_extra_commands)))
+                self.logger.warning("%s: tearDown commands have failed" % \
+                        self.id())
+                for cmd in failed_extra_commands:
+                    result = failed_extra_commands[cmd]
+                    self.logger.warning("%s: %s\n%s" % (self.id(), cmd,
+                        result.output))
                 self.logger.debug("Trying to move on.")
             self._extra_tear_down_commands = []
 
Signature · 1 line
2.11.0
#26 lore raw

PATCH 14/30 oeqa/selftest/case: Support bitbake memres mode in per build directory

In reply to Aníbal Limón

If BBSERVER is set on the environment the bitbake is set to be used as a memres, so starts an bitbake server per TestClass.

---
 meta/lib/oeqa/selftest/case.py | 9 +++++++++
 1 file changed, 9 insertions(+)
meta/lib/oeqa/selftest/case.py +9 -0
index c6f2d184ea3..d98a3760415 100644
@@ -97,6 +97,12 @@ class OESelftestTestCase(OETestCase):
             cls.runCmd("git init; git add *; git commit -a -m 'initial'",
                             cwd=cls.testlayer_path)
 
+            if cls._use_own_builddir and 'BBSERVER' in os.environ:
+                env = os.environ.copy()
+                del env['BBSERVER']
+                result = cls.runCmd('bitbake --server-only -t xmlrpc -B localhost:-1',
+                        env=env)
+
             # XXX: sometimes meta-selftest isn't on bblayers at first backup
             try:
                 cls.runCmd("bitbake-layers remove-layer %s" % cls.orig_testlayer_path)
@@ -119,6 +125,9 @@ class OESelftestTestCase(OETestCase):
 
     @classmethod
     def tearDownClass(cls):
+        if cls._use_own_builddir and 'BBSERVER' in os.environ:
+            cls.runCmd('bitbake --kill-server')
+
         cls.remove_include()
         cls.remove_inc_files()
         super(OESelftestTestCase, cls).tearDownClass()
Signature · 1 line
2.11.0
#27 lore raw

PATCH 15/30 oeqa/selftest/cases: Use testlayer_path instead of call get_test_layer()

In reply to Aníbal Limón

The testlayer_path is set at init of selftest so isn't need to call every time get_test_layer to get it.

---
 meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py | 2 +-
 meta/lib/oeqa/selftest/cases/devtool.py             | 4 ++--
 meta/lib/oeqa/selftest/cases/oescripts.py           | 2 +-
 meta/lib/oeqa/selftest/cases/sstate.py              | 2 +-
 meta/lib/oeqa/selftest/cases/sstatetests.py         | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)
meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py +1 -1
index 0e5896234c3..b42aa3638d5 100644
@@ -2,7 +2,7 @@ import os
 import shutil
 
 import oeqa.utils.ftools as ftools
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_test_layer
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.selftest.cases.sstate import SStateBase
 
 
meta/lib/oeqa/selftest/cases/devtool.py +2 -2
index 88d69724f93..28c84679a17 100644
@@ -8,7 +8,7 @@ import fnmatch
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer
-from oeqa.utils.commands import get_bb_vars, runqemu, get_test_layer
+from oeqa.utils.commands import get_bb_vars, runqemu
 from oeqa.core.decorator.oeid import OETestID
 
 class DevtoolBase(OESelftestTestCase):
@@ -1530,7 +1530,7 @@ class DevtoolTests(DevtoolBase):
         # Ensure the recipe is where we think it should be (so that cleanup doesn't trash things)
         self.assertIn('/meta/', recipedir)
         relpth = os.path.relpath(recipedir, os.path.join(get_bb_var('COREBASE'), 'meta'))
-        appenddir = os.path.join(get_test_layer(), relpth)
+        appenddir = os.path.join(self.testlayer_path, relpth)
         self.track_for_cleanup(appenddir)
         # Try finish to the original layer
         self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
meta/lib/oeqa/selftest/cases/oescripts.py +1 -1
index 1ee753763ec..f7fe200cfac 100644
@@ -1,6 +1,6 @@
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.selftest.cases.buildhistory import BuildhistoryBase
-from oeqa.utils.commands import Command, runCmd, bitbake, get_bb_var, get_test_layer
+from oeqa.utils.commands import Command, runCmd, bitbake, get_bb_var
 from oeqa.core.decorator.oeid import OETestID
 
 class BuildhistoryDiffTests(BuildhistoryBase):
meta/lib/oeqa/selftest/cases/sstate.py +1 -1
index bc2fdbd8ccb..b8c2880ad06 100644
@@ -6,7 +6,7 @@ import shutil
 
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_vars, get_test_layer
+from oeqa.utils.commands import runCmd, bitbake, get_bb_vars
 
 
 class SStateBase(OESelftestTestCase):
meta/lib/oeqa/selftest/cases/sstatetests.py +1 -1
index 07a206824aa..4617d16d212 100644
@@ -4,7 +4,7 @@ import glob
 import subprocess
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_test_layer
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.selftest.cases.sstate import SStateBase
 from oeqa.core.decorator.oeid import OETestID
 
Signature · 1 line
2.11.0
#28 lore raw

PATCH 16/30 oeqa/selftest/cases: Use builddir from class instead of get from environment

In reply to Aníbal Limón

Now the build directory is setup by Test class, so the builddir attr points to the actual BUILDDIR instead of get from environment.

---
 meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py | 7 +++----
 meta/lib/oeqa/selftest/cases/bbtests.py             | 4 ++--
 meta/lib/oeqa/selftest/cases/eSDK.py                | 2 +-
 meta/lib/oeqa/selftest/cases/signing.py             | 2 +-
 4 files changed, 7 insertions(+), 8 deletions(-)
meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py +3 -4
index b42aa3638d5..98b8b60f51a 100644
@@ -11,7 +11,6 @@ class RebuildFromSState(SStateBase):
     @classmethod
     def setUpClass(self):
         super(RebuildFromSState, self).setUpClass()
-        self.builddir = os.path.join(os.environ.get('BUILDDIR'))
 
     def get_dep_targets(self, primary_targets):
         found_targets = []
@@ -24,16 +23,16 @@ class RebuildFromSState(SStateBase):
         os.mkdir(builddir)
         self.track_for_cleanup(builddir)
         os.mkdir(os.path.join(builddir, 'conf'))
-        shutil.copyfile(os.path.join(os.environ.get('BUILDDIR'), 'conf/local.conf'), os.path.join(builddir, 'conf/local.conf'))
+        shutil.copyfile(self.localconf_path, os.path.join(builddir, 'conf/local.conf'))
         config = {}
         config['default_sstate_dir'] = "SSTATE_DIR ?= \"${TOPDIR}/sstate-cache\""
         config['null_sstate_mirrors'] = "SSTATE_MIRRORS = \"\""
         config['default_tmp_dir'] = "TMPDIR = \"${TOPDIR}/tmp\""
         for key in config:
             ftools.append_file(os.path.join(builddir, 'conf/selftest.inc'), config[key])
-        shutil.copyfile(os.path.join(os.environ.get('BUILDDIR'), 'conf/bblayers.conf'), os.path.join(builddir, 'conf/bblayers.conf'))
+        shutil.copyfile(self.local_bblayers_path, os.path.join(builddir, 'conf/bblayers.conf'))
         try:
-            shutil.copyfile(os.path.join(os.environ.get('BUILDDIR'), 'conf/auto.conf'), os.path.join(builddir, 'conf/auto.conf'))
+            shutil.copyfile(self.autoconf_path, os.path.join(builddir, 'conf/auto.conf'))
         except:
             pass
 
meta/lib/oeqa/selftest/cases/bbtests.py +2 -2
index 4c82049032b..df11a6bc6d0 100644
@@ -22,8 +22,8 @@ class BitbakeTests(OESelftestTestCase):
     @OETestID(790)
     def test_run_bitbake_from_dir_2(self):
         my_env = os.environ.copy()
-        my_env['BBPATH'] = my_env['BUILDDIR']
-        os.chdir(os.path.dirname(os.environ['BUILDDIR']))
+        my_env['BBPATH'] = self.builddir
+        os.chdir(os.path.dirname(self.builddir))
         self.assertEqual(bitbake('-e', env=my_env).status, 0, msg = "bitbake couldn't run from builddir")
 
     @OETestID(806)
meta/lib/oeqa/selftest/cases/eSDK.py +1 -1
index f36c3ccd3b4..60f4e239ab0 100644
@@ -49,7 +49,7 @@ class oeSDKExtSelfTest(OESelftestTestCase):
 
     @staticmethod
     def update_configuration(cls, image, tmpdir_eSDKQA, env_eSDK, ext_sdk_path):
-        sstate_dir = os.path.join(os.environ['BUILDDIR'], 'sstate-cache')
+        sstate_dir = os.path.join(cls.builddir, 'sstate-cache')
 
         oeSDKExtSelfTest.generate_eSDK(cls.image)
 
meta/lib/oeqa/selftest/cases/signing.py +1 -1
index edb5f653f20..6ef8d8eb5d0 100644
@@ -105,7 +105,7 @@ class Signing(OESelftestTestCase):
 
         test_recipe = 'ed'
 
-        builddir = os.environ.get('BUILDDIR')
+        builddir = self.builddir
         sstatedir = os.path.join(builddir, 'test-sstate')
 
         self.add_command_to_tearDown('bitbake -c clean %s' % test_recipe)
Signature · 1 line
2.11.0
#29 lore raw

PATCH 17/30 oeqa/selftest/cases: Use wrapper methods from OESelfTestCase class

In reply to Aníbal Limón

In order to support threaded runs in oe-selftest cases, there is a need to use wrapper methods that takes into account the current builddir by Test class.

---
 .../lib/oeqa/selftest/cases/_sstatetests_noauto.py |  16 ++--
 meta/lib/oeqa/selftest/cases/archiver.py           |  21 +++--
 meta/lib/oeqa/selftest/cases/bblayers.py           |  45 ++++-----
 meta/lib/oeqa/selftest/cases/bbtests.py            | 101 +++++++++++----------
 meta/lib/oeqa/selftest/cases/buildhistory.py       |   9 +-
 meta/lib/oeqa/selftest/cases/buildoptions.py       |  51 ++++++-----
 meta/lib/oeqa/selftest/cases/containerimage.py     |   9 +-
 meta/lib/oeqa/selftest/cases/distrodata.py         |   7 +-
 meta/lib/oeqa/selftest/cases/image_typedep.py      |   8 +-
 meta/lib/oeqa/selftest/cases/layerappend.py        |  22 ++---
 meta/lib/oeqa/selftest/cases/liboe.py              |  13 +--
 meta/lib/oeqa/selftest/cases/lic_checksum.py       |   8 +-
 meta/lib/oeqa/selftest/cases/manifest.py           |  13 +--
 meta/lib/oeqa/selftest/cases/oelib/buildhistory.py |   7 +-
 meta/lib/oeqa/selftest/cases/oescripts.py          |   5 +-
 meta/lib/oeqa/selftest/cases/package.py            |  10 +-
 meta/lib/oeqa/selftest/cases/pkgdata.py            |  73 +++++++--------
 meta/lib/oeqa/selftest/cases/prservice.py          |  19 ++--
 meta/lib/oeqa/selftest/cases/signing.py            |  38 ++++----
 meta/lib/oeqa/selftest/cases/sstate.py             |   5 +-
 meta/lib/oeqa/selftest/cases/sstatetests.py        |  51 ++++++-----
 21 files changed, 273 insertions(+), 258 deletions(-)
meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py +6 -10
index 98b8b60f51a..08e71f33526 100644
@@ -2,19 +2,15 @@ import os
 import shutil
 
 import oeqa.utils.ftools as ftools
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.selftest.cases.sstate import SStateBase
 
-
 class RebuildFromSState(SStateBase):
-
-    @classmethod
-    def setUpClass(self):
-        super(RebuildFromSState, self).setUpClass()
+    _use_own_builddir = True
+    _main_thread = False
 
     def get_dep_targets(self, primary_targets):
         found_targets = []
-        bitbake("-g " + ' '.join(map(str, primary_targets)))
+        self.bitbake("-g " + ' '.join(map(str, primary_targets)))
         with open(os.path.join(self.builddir, 'pn-buildlist'), 'r') as pnfile:
             found_targets = pnfile.read().splitlines()
         return found_targets
@@ -59,7 +55,7 @@ class RebuildFromSState(SStateBase):
             rebuild_targets = primary_targets
 
         self.configure_builddir(buildA)
-        runCmd((". %s/oe-init-build-env %s && " % (get_bb_var('COREBASE'), buildA)) + 'bitbake  ' + ' '.join(map(str, primary_targets)), shell=True, executable='/bin/bash')
+        self.runCmd((". %s/oe-init-build-env %s && " % (self.get_bb_var('COREBASE'), buildA)) + 'bitbake  ' + ' '.join(map(str, primary_targets)), shell=True, executable='/bin/bash')
         self.hardlink_tree(os.path.join(buildA, 'sstate-cache'), os.path.join(self.builddir, 'sstate-cache-buildA'))
         shutil.rmtree(buildA)
 
@@ -69,13 +65,13 @@ class RebuildFromSState(SStateBase):
             self.configure_builddir(buildB)
             self.hardlink_tree(os.path.join(self.builddir, 'sstate-cache-buildA'), os.path.join(buildB, 'sstate-cache'))
 
-            result_cleansstate = runCmd((". %s/oe-init-build-env %s && " % (get_bb_var('COREBASE'), buildB)) + 'bitbake -ccleansstate ' + target, ignore_status=True, shell=True, executable='/bin/bash')
+            result_cleansstate = self.runCmd((". %s/oe-init-build-env %s && " % (self.get_bb_var('COREBASE'), buildB)) + 'bitbake -ccleansstate ' + target, ignore_status=True, shell=True, executable='/bin/bash')
             if not result_cleansstate.status == 0:
                 failed_cleansstate.append(target)
                 shutil.rmtree(buildB)
                 continue
 
-            result_build = runCmd((". %s/oe-init-build-env %s && " % (get_bb_var('COREBASE'), buildB)) + 'bitbake ' + target, ignore_status=True, shell=True, executable='/bin/bash')
+            result_build = self.runCmd((". %s/oe-init-build-env %s && " % (self.get_bb_var('COREBASE'), buildB)) + 'bitbake ' + target, ignore_status=True, shell=True, executable='/bin/bash')
             if not result_build.status == 0:
                 failed_rebuild.append(target)
 
meta/lib/oeqa/selftest/cases/archiver.py +11 -10
index 72026d573cc..347d0a8a5f5 100644
@@ -1,10 +1,11 @@
 import os
 import glob
-from oeqa.utils.commands import bitbake, get_bb_vars
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.core.decorator.oeid import OETestID
 
 class Archiver(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @OETestID(1345)
     def test_archiver_allows_to_filter_on_recipe_name(self):
@@ -26,10 +27,10 @@ class Archiver(OESelftestTestCase):
         features += 'COPYLEFT_PN_EXCLUDE = "%s"\n' % exclude_recipe
         self.write_config(features)
 
-        bitbake('-c clean %s %s' % (include_recipe, exclude_recipe))
-        bitbake("-c deploy_archives %s %s" % (include_recipe, exclude_recipe))
+        self.bitbake('-c clean %s %s' % (include_recipe, exclude_recipe))
+        self.bitbake("-c deploy_archives %s %s" % (include_recipe, exclude_recipe))
 
-        bb_vars = get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS'])
+        bb_vars = self.get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS'])
         src_path = os.path.join(bb_vars['DEPLOY_DIR_SRC'], bb_vars['TARGET_SYS'])
 
         # Check that include_recipe was included
@@ -58,10 +59,10 @@ class Archiver(OESelftestTestCase):
         features += 'COPYLEFT_RECIPE_TYPES = "target"\n'
         self.write_config(features)
 
-        bitbake('-c clean %s %s' % (target_recipe, native_recipe))
-        bitbake("%s -c deploy_archives %s" % (target_recipe, native_recipe))
+        self.bitbake('-c clean %s %s' % (target_recipe, native_recipe))
+        self.bitbake("%s -c deploy_archives %s" % (target_recipe, native_recipe))
 
-        bb_vars = get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS', 'BUILD_SYS'])
+        bb_vars = self.get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS', 'BUILD_SYS'])
         src_path_target = os.path.join(bb_vars['DEPLOY_DIR_SRC'], bb_vars['TARGET_SYS'])
         src_path_native = os.path.join(bb_vars['DEPLOY_DIR_SRC'], bb_vars['BUILD_SYS'])
 
@@ -95,10 +96,10 @@ class Archiver(OESelftestTestCase):
         features += 'COPYLEFT_PN_EXCLUDE = "%s"\n' % target_recipes[1]
         self.write_config(features)
 
-        bitbake('-c clean %s %s' % (' '.join(target_recipes), ' '.join(native_recipes)))
-        bitbake('-c deploy_archives %s %s' % (' '.join(target_recipes), ' '.join(native_recipes)))
+        self.bitbake('-c clean %s %s' % (' '.join(target_recipes), ' '.join(native_recipes)))
+        self.bitbake('-c deploy_archives %s %s' % (' '.join(target_recipes), ' '.join(native_recipes)))
 
-        bb_vars = get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS', 'BUILD_SYS'])
+        bb_vars = self.get_bb_vars(['DEPLOY_DIR_SRC', 'TARGET_SYS', 'BUILD_SYS'])
         src_path_target = os.path.join(bb_vars['DEPLOY_DIR_SRC'], bb_vars['TARGET_SYS'])
         src_path_native = os.path.join(bb_vars['DEPLOY_DIR_SRC'], bb_vars['BUILD_SYS'])
 
meta/lib/oeqa/selftest/cases/bblayers.py +23 -22

109 lines not on this page · raw message

meta/lib/oeqa/selftest/cases/bbtests.py +53 -48

271 lines not on this page · raw message

meta/lib/oeqa/selftest/cases/buildhistory.py +3 -6
index 06792d9146d..ed0965bf261 100644
@@ -3,13 +3,10 @@ import re
 import datetime
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, get_bb_vars
-
 
 class BuildhistoryBase(OESelftestTestCase):
-
     def config_buildhistory(self, tmp_bh_location=False):
-        bb_vars = get_bb_vars(['USER_CLASSES', 'INHERIT'])
+        bb_vars = self.get_bb_vars(['USER_CLASSES', 'INHERIT'])
         if (not 'buildhistory' in bb_vars['USER_CLASSES']) and (not 'buildhistory' in bb_vars['INHERIT']):
             add_buildhistory_config = 'INHERIT += "buildhistory"\nBUILDHISTORY_COMMIT = "1"'
             self.append_config(add_buildhistory_config)
@@ -30,8 +27,8 @@ class BuildhistoryBase(OESelftestTestCase):
 
         self.append_config(global_config)
         self.append_recipeinc(target, target_config)
-        bitbake("-cclean %s" % target)
-        result = bitbake(target, ignore_status=True)
+        self.bitbake("-cclean %s" % target)
+        result = self.bitbake(target, ignore_status=True)
         self.remove_config(global_config)
         self.remove_recipeinc(target, target_config)
 
meta/lib/oeqa/selftest/cases/buildoptions.py +26 -25

148 lines not on this page · raw message

meta/lib/oeqa/selftest/cases/containerimage.py +5 -4
index 99a5cc9e575..4220d80ac4a 100644
@@ -1,7 +1,6 @@
 import os
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, get_bb_vars, runCmd
 from oeqa.core.decorator.oeid import OETestID
 
 # This test builds an image with using the "container" IMAGE_FSTYPE, and
@@ -18,6 +17,8 @@ from oeqa.core.decorator.oeid import OETestID
 # default other than what is in ROOTFS_BOOTSTRAP_INSTALL.
 #
 class ContainerImageTests(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     # Verify that when specifying a IMAGE_TYPEDEP_ of the form "foo.bar" that
     # the conversion type bar gets added as a dep as well
@@ -41,7 +42,7 @@ PACKAGE_CLASSES = "package_ipk"
 IMAGE_FEATURES = ""
 """)
 
-        bbvars = get_bb_vars(['bindir', 'sysconfdir', 'localstatedir',
+        bbvars = self.get_bb_vars(['bindir', 'sysconfdir', 'localstatedir',
                               'DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'],
                               target='container-test-image')
         expected_files = [
@@ -75,11 +76,11 @@ IMAGE_FEATURES = ""
         expected_files = sorted(expected_files)
 
         # Build the image of course
-        bitbake('container-test-image')
+        self.bitbake('container-test-image')
 
         image = os.path.join(bbvars['DEPLOY_DIR_IMAGE'],
                              bbvars['IMAGE_LINK_NAME'] + '.tar.bz2')
 
         # Ensure the files in the image are what we expect
-        result = runCmd("tar tf {} | sort".format(image), shell=True)
+        result = self.runCmd("tar tf {} | sort".format(image), shell=True)
         self.assertEqual(result.output.split('\n'), expected_files)
meta/lib/oeqa/selftest/cases/distrodata.py +4 -3
index d5d286d5cc3..86c99a71b80 100644
@@ -1,9 +1,10 @@
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
 from oeqa.utils.decorators import testcase
 from oeqa.utils.ftools import write_file
 
 class Distrodata(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @classmethod
     def setUpClass(cls):
@@ -21,8 +22,8 @@ class Distrodata(OESelftestTestCase):
         feature += 'LICENSE_FLAGS_WHITELIST += " commercial"\n'
 
         self.write_config(feature)
-        bitbake('-c checkpkg world')
-        checkpkg_result = open(os.path.join(get_bb_var("LOG_DIR"), "checkpkg.csv")).readlines()[1:]
+        self.bitbake('-c checkpkg world')
+        checkpkg_result = open(os.path.join(self.get_bb_var("LOG_DIR"), "checkpkg.csv")).readlines()[1:]
         exceptions = [exc.strip() for exc in open(self.exceptions_path).readlines()]
         failed_upstream_checks = [pkg_data[0] for pkg_data in [pkg_line.split('\t') for pkg_line in checkpkg_result] if pkg_data[11] == '']
         regressed_failures = set(failed_upstream_checks) - set(exceptions)
meta/lib/oeqa/selftest/cases/image_typedep.py +5 -3
index e6788853a36..647725760a1 100644
@@ -1,10 +1,11 @@
 import os
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake
 from oeqa.core.decorator.oeid import OETestID
 
 class ImageTypeDepTests(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     # Verify that when specifying a IMAGE_TYPEDEP_ of the form "foo.bar" that
     # the conversion type bar gets added as a dep as well
@@ -27,8 +28,9 @@ inherit image
 """)
         # First get the dependency that should exist for bz2, it will look
         # like CONVERSION_DEPENDS_bz2="somedep"
-        result = bitbake('-e emptytest')
+        result = self.bitbake('-e emptytest')
 
+        dep = ''
         for line in result.output.split('\n'):
             if line.startswith('CONVERSION_DEPENDS_bz2'):
                 dep = line.split('=')[1].strip('"')
@@ -36,7 +38,7 @@ inherit image
 
         # Now get the dependency task list and check for the expected task
         # dependency
-        bitbake('-g emptytest')
+        self.bitbake('-g emptytest')
 
         taskdependsfile = os.path.join(self.builddir, 'task-depends.dot')
         dep =  dep + ".do_populate_sysroot"
meta/lib/oeqa/selftest/cases/layerappend.py +11 -11
index 95621163090..518e31def6f 100644
@@ -1,11 +1,13 @@
 import os
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 import oeqa.utils.ftools as ftools
 from oeqa.core.decorator.oeid import OETestID
 
 class LayerAppendTests(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
+
     layerconf = """
 # We have a conf and classes directory, append to BBPATH
 BBPATH .= ":${LAYERDIR}"
@@ -51,10 +53,8 @@ SRC_URI_append += "file://appendtest.txt"
 
     @OETestID(1196)
     def test_layer_appends(self):
-        corebase = get_bb_var("COREBASE")
-
         for l in ["0", "1", "2"]:
-            layer = os.path.join(corebase, "meta-layertest" + l)
+            layer = os.path.join(self.builddir, "meta-layertest" + l)
             self.assertFalse(os.path.exists(layer))
             os.mkdir(layer)
             os.mkdir(layer + "/conf")
@@ -78,18 +78,18 @@ SRC_URI_append += "file://appendtest.txt"
                     f.write("Layer 2 test")
             self.track_for_cleanup(layer)
 
-        self.layerappend = "BBLAYERS += \"{0}/meta-layertest0 {0}/meta-layertest1 {0}/meta-layertest2\"".format(corebase)
+        self.layerappend = "BBLAYERS += \"{0}/meta-layertest0 {0}/meta-layertest1 {0}/meta-layertest2\"".format(self.builddir)
         ftools.append_file(self.builddir + "/conf/bblayers.conf", self.layerappend)
-        stagingdir = get_bb_var("SYSROOT_DESTDIR", "layerappendtest")
-        bitbake("layerappendtest")
+        stagingdir = self.get_bb_var("SYSROOT_DESTDIR", "layerappendtest")
+        self.bitbake("layerappendtest")
         data = ftools.read_file(stagingdir + "/appendtest.txt")
         self.assertEqual(data, "Layer 2 test")
-        os.remove(corebase + "/meta-layertest2/recipes-test/layerappendtest/appendtest.txt")
-        bitbake("layerappendtest")
+        os.remove(self.builddir + "/meta-layertest2/recipes-test/layerappendtest/appendtest.txt")
+        self.bitbake("layerappendtest")
         data = ftools.read_file(stagingdir + "/appendtest.txt")
         self.assertEqual(data, "Layer 1 test")
-        with open(corebase + "/meta-layertest2/recipes-test/layerappendtest/appendtest.txt", "w") as f:
+        with open(self.builddir + "/meta-layertest2/recipes-test/layerappendtest/appendtest.txt", "w") as f:
             f.write("Layer 2 test")
-        bitbake("layerappendtest")
+        self.bitbake("layerappendtest")
         data = ftools.read_file(stagingdir + "/appendtest.txt")
         self.assertEqual(data, "Layer 2 test")
meta/lib/oeqa/selftest/cases/liboe.py +7 -6
index e84609246a7..cfe15866419 100644
@@ -1,15 +1,16 @@
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.core.decorator.oeid import OETestID
-from oeqa.utils.commands import get_bb_var, get_bb_vars, bitbake, runCmd
 import oe.path
 import os
 
 class LibOE(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @classmethod
     def setUpClass(cls):
         super(LibOE, cls).setUpClass()
-        cls.tmp_dir = get_bb_var('TMPDIR')
+        cls.tmp_dir = cls.get_bb_var('TMPDIR')
 
     @OETestID(1635)
     def test_copy_tree_special(self):
@@ -54,20 +55,20 @@ class LibOE(OESelftestTestCase):
         testfilename = 'testxattr'
 
         # ensure we have setfattr available
-        bitbake("attr-native")
+        self.bitbake("attr-native")
 
-        bb_vars = get_bb_vars(['SYSROOT_DESTDIR', 'bindir'], 'attr-native')
+        bb_vars = self.get_bb_vars(['SYSROOT_DESTDIR', 'bindir'], 'attr-native')
         destdir = bb_vars['SYSROOT_DESTDIR']
         bindir = bb_vars['bindir']
         bindir = destdir + bindir
 
         # create a file with xattr and copy it
         open(oe.path.join(src, testfilename), 'w+b').close()
-        runCmd('%s/setfattr -n user.oetest -v "testing liboe" %s' % (bindir, oe.path.join(src, testfilename)))
+        self.runCmd('%s/setfattr -n user.oetest -v "testing liboe" %s' % (bindir, oe.path.join(src, testfilename)))
         oe.path.copytree(src, dst)
 
         # ensure file in dest has user.oetest xattr
-        result = runCmd('%s/getfattr -n user.oetest %s' % (bindir, oe.path.join(dst, testfilename)))
+        result = self.runCmd('%s/getfattr -n user.oetest %s' % (bindir, oe.path.join(dst, testfilename)))
         self.assertIn('user.oetest="testing liboe"', result.output, 'Extended attribute not sert in dst')
 
         oe.path.remove(testloc)
meta/lib/oeqa/selftest/cases/lic_checksum.py +4 -4
index 37407157c1e..6b4eb32e07c 100644
@@ -2,11 +2,11 @@ import os
 import tempfile
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake
-from oeqa.utils import CommandError
 from oeqa.core.decorator.oeid import OETestID
 
 class LicenseTests(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     # Verify that changing a license file that has an absolute path causes
     # the license qa to fail due to a mismatched md5sum.
@@ -24,12 +24,12 @@ INHIBIT_DEFAULT_DEPS = "1"
 LIC_FILES_CHKSUM = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 """ % (lic_path, lic_path))
-        result = bitbake(bitbake_cmd)
+        result = self.bitbake(bitbake_cmd)
 
         with open(lic_path, "w") as f:
             f.write("data")
 
         self.write_config("INHERIT_remove = \"report-error\"")
-        result = bitbake(bitbake_cmd, ignore_status=True)
+        result = self.bitbake(bitbake_cmd, ignore_status=True)
         if error_msg not in result.output:
             raise AssertionError(result.output)
meta/lib/oeqa/selftest/cases/manifest.py +7 -6
index 146071934dd..534a9075902 100644
@@ -1,7 +1,6 @@
 import os
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import get_bb_var, get_bb_vars, bitbake
 from oeqa.core.decorator.oeid import OETestID
 
 class ManifestEntry:
@@ -12,6 +11,8 @@ class ManifestEntry:
 
 class VerifyManifest(OESelftestTestCase):
     '''Tests for the manifest files and contents of an image'''
+    _use_own_builddir = True
+    _main_thread = False
 
     @classmethod
     def check_manifest_entries(self, manifest, path):
@@ -37,7 +38,7 @@ class VerifyManifest(OESelftestTestCase):
     @classmethod
     def get_dir_from_bb_var(self, bb_var, target = None):
         target == self.buildtarget if target == None else target
-        directory = get_bb_var(bb_var, target);
+        directory = self.get_bb_var(bb_var, target);
         if not directory or not os.path.isdir(directory):
             self.logger.debug("{}: {} points to {} when target = {}"\
                     .format(self.classname, bb_var, directory, target))
@@ -53,7 +54,7 @@ class VerifyManifest(OESelftestTestCase):
 
         self.logger.info("{}: doing bitbake {} as a prerequisite of the test"\
                 .format(self.classname, self.buildtarget))
-        if bitbake(self.buildtarget).status:
+        if self.bitbake(self.buildtarget).status:
             self.logger.debug("{} Failed to setup {}"\
                     .format(self.classname, self.buildtarget))
             self.skipTest("{}: Cannot setup testing scenario"\
@@ -69,7 +70,7 @@ class VerifyManifest(OESelftestTestCase):
         bbargs = sdktask + ' ' + self.buildtarget
         self.logger.debug("{}: doing bitbake {} as a prerequisite of the test"\
                 .format(self.classname, bbargs))
-        if bitbake(bbargs).status:
+        if self.bitbake(bbargs).status:
             self.logger.debug("{} Failed to bitbake {}"\
                     .format(self.classname, bbargs))
             self.skipTest("{}: Cannot setup testing scenario"\
@@ -84,7 +85,7 @@ class VerifyManifest(OESelftestTestCase):
         try:
             mdir = self.get_dir_from_bb_var('SDK_DEPLOY', self.buildtarget)
             for k in d_target.keys():
-                bb_vars = get_bb_vars(['SDK_NAME', 'SDK_VERSION'], self.buildtarget)
+                bb_vars = self.get_bb_vars(['SDK_NAME', 'SDK_VERSION'], self.buildtarget)
                 mfilename[k] = "{}-toolchain-{}.{}.manifest".format(
                         bb_vars['SDK_NAME'],
                         bb_vars['SDK_VERSION'],
@@ -134,7 +135,7 @@ class VerifyManifest(OESelftestTestCase):
         try:
             mdir = self.get_dir_from_bb_var('DEPLOY_DIR_IMAGE',
                                                 self.buildtarget)
-            mfilename = get_bb_var("IMAGE_LINK_NAME", self.buildtarget)\
+            mfilename = self.get_bb_var("IMAGE_LINK_NAME", self.buildtarget)\
                     + ".manifest"
             mpath = os.path.join(mdir, mfilename)
             if not os.path.isfile(mpath): raise IOError
meta/lib/oeqa/selftest/cases/oelib/buildhistory.py +5 -2
index 08675fd8208..b77aa228a02 100644
@@ -1,15 +1,18 @@
 import os
 from oeqa.selftest.case import OESelftestTestCase
 import tempfile
-from oeqa.utils.commands import get_bb_var
 from oeqa.core.decorator.oeid import OETestID
+from git import Repo
+from oe.buildhistory_analysis import blob_to_dict, compare_dict_blobs
 
 class TestBlobParsing(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     def setUp(self):
         import time
         self.repo_path = tempfile.mkdtemp(prefix='selftest-buildhistory',
-            dir=get_bb_var('TOPDIR'))
+            dir=self.get_bb_var('TOPDIR'))
 
         try:
             from git import Repo
meta/lib/oeqa/selftest/cases/oescripts.py +3 -2
index f7fe200cfac..8d6aa04d3e0 100644
@@ -1,15 +1,16 @@
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.selftest.cases.buildhistory import BuildhistoryBase
-from oeqa.utils.commands import Command, runCmd, bitbake, get_bb_var
 from oeqa.core.decorator.oeid import OETestID
 
 class BuildhistoryDiffTests(BuildhistoryBase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @OETestID(295)
     def test_buildhistory_diff(self):
         target = 'xcursor-transparent-theme'
         self.run_buildhistory_operation(target, target_config="PR = \"r1\"", change_bh_location=True)
         self.run_buildhistory_operation(target, target_config="PR = \"r0\"", change_bh_location=False, expect_error=True)
-        result = runCmd("buildhistory-diff -p %s" % get_bb_var('BUILDHISTORY_DIR'))
+        result = self.runCmd("buildhistory-diff -p %s" % self.get_bb_var('BUILDHISTORY_DIR'))
         expected_output = 'PR changed from "r1" to "r0"'
         self.assertTrue(expected_output in result.output, msg="Did not find expected output: %s" % result.output)
meta/lib/oeqa/selftest/cases/package.py +6 -4
index b07d6ed0abe..ea9dec09e71 100644
@@ -1,10 +1,12 @@
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.core.decorator.oeid import OETestID
-from oeqa.utils.commands import bitbake, get_bb_vars
 import subprocess, os
 import oe.path
 
 class VersionOrdering(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
+
     # version1, version2, sort order
     tests = (
         ("1.0", "1.0", 0),
@@ -20,11 +22,11 @@ class VersionOrdering(OESelftestTestCase):
         super(VersionOrdering, cls).setUpClass()
 
         # Build the tools we need and populate a sysroot
-        bitbake("dpkg-native opkg-native rpm-native python3-native")
-        bitbake("build-sysroots -c build_native_sysroot")
+        cls.bitbake("dpkg-native opkg-native rpm-native python3-native")
+        cls.bitbake("build-sysroots -c build_native_sysroot")
 
         # Get the paths so we can point into the sysroot correctly
-        vars = get_bb_vars(["STAGING_DIR", "BUILD_ARCH", "bindir_native", "libdir_native"])
+        vars = cls.get_bb_vars(["STAGING_DIR", "BUILD_ARCH", "bindir_native", "libdir_native"])
         cls.staging = oe.path.join(vars["STAGING_DIR"], vars["BUILD_ARCH"])
         cls.bindir = oe.path.join(cls.staging, vars["bindir_native"])
         cls.libdir = oe.path.join(cls.staging, vars["libdir_native"])
meta/lib/oeqa/selftest/cases/pkgdata.py +37 -36

210 lines not on this page · raw message

meta/lib/oeqa/selftest/cases/prservice.py +10 -9
index ed36f0fed76..67ef931927b 100644
@@ -5,16 +5,17 @@ import datetime
 
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.core.decorator.oeid import OETestID
 from oeqa.utils.network import get_free_port
 
 class BitbakePrTests(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @classmethod
     def setUpClass(cls):
         super(BitbakePrTests, cls).setUpClass()
-        cls.pkgdata_dir = get_bb_var('PKGDATA_DIR')
+        cls.pkgdata_dir = cls.get_bb_var('PKGDATA_DIR')
 
     def get_pr_version(self, package_name):
         package_data_file = os.path.join(self.pkgdata_dir, 'runtime', package_name)
@@ -24,7 +25,7 @@ class BitbakePrTests(OESelftestTestCase):
         return int(find_pr.group(1))
 
     def get_task_stamp(self, package_name, recipe_task):
-        stampdata = get_bb_var('STAMP', target=package_name).split('/')
+        stampdata = self.get_bb_var('STAMP', target=package_name).split('/')
         prefix = stampdata[-1]
         package_stamps_path = "/".join(stampdata[:-1])
         stamps = []
@@ -39,7 +40,7 @@ class BitbakePrTests(OESelftestTestCase):
     def increment_package_pr(self, package_name):
         inc_data = "do_package_append() {\n    bb.build.exec_func('do_test_prserv', d)\n}\ndo_test_prserv() {\necho \"The current date is: %s\"\n}" % datetime.datetime.now()
         self.write_recipeinc(package_name, inc_data)
-        res = bitbake(package_name, ignore_status=True)
+        res = self.bitbake(package_name, ignore_status=True)
         self.delete_recipeinc(package_name)
         self.assertEqual(res.status, 0, msg=res.output)
         self.assertTrue("NOTE: Started PRServer with DBfile" in res.output, msg=res.output)
@@ -71,15 +72,15 @@ class BitbakePrTests(OESelftestTestCase):
         pr_1 = self.get_pr_version(package_name)
 
         exported_db_path = os.path.join(self.builddir, 'export.inc')
-        export_result = runCmd("bitbake-prserv-tool export %s" % exported_db_path, ignore_status=True)
+        export_result = self.runCmd("bitbake-prserv-tool export %s" % exported_db_path, ignore_status=True)
         self.assertEqual(export_result.status, 0, msg="PR Service database export failed: %s" % export_result.output)
 
         if replace_current_db:
-            current_db_path = os.path.join(get_bb_var('PERSISTENT_DIR'), 'prserv.sqlite3')
+            current_db_path = os.path.join(self.get_bb_var('PERSISTENT_DIR'), 'prserv.sqlite3')
             self.assertTrue(os.path.exists(current_db_path), msg="Path to current PR Service database is invalid: %s" % current_db_path)
             os.remove(current_db_path)
 
-        import_result = runCmd("bitbake-prserv-tool import %s" % exported_db_path, ignore_status=True)
+        import_result = self.runCmd("bitbake-prserv-tool import %s" % exported_db_path, ignore_status=True)
         os.remove(exported_db_path)
         self.assertEqual(import_result.status, 0, msg="PR Service database import failed: %s" % import_result.output)
 
@@ -124,8 +125,8 @@ class BitbakePrTests(OESelftestTestCase):
     def test_stopping_prservice_message(self):
         port = get_free_port()
 
-        runCmd('bitbake-prserv --host localhost --port %s --loglevel=DEBUG --start' % port)
-        ret = runCmd('bitbake-prserv --host localhost --port %s --loglevel=DEBUG --stop' % port)
+        self.runCmd('bitbake-prserv --host localhost --port %s --loglevel=DEBUG --start' % port)
+        ret = self.runCmd('bitbake-prserv --host localhost --port %s --loglevel=DEBUG --stop' % port)
 
         self.assertEqual(ret.status, 0)
 
meta/lib/oeqa/selftest/cases/signing.py +19 -19
index 6ef8d8eb5d0..e4a4514a026 100644
@@ -1,5 +1,4 @@
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
 import os
 import glob
 import re
@@ -8,8 +7,9 @@ import tempfile
 from oeqa.core.decorator.oeid import OETestID
 from oeqa.utils.ftools import write_file
 
-
 class Signing(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     gpg_dir = ""
     pub_key_path = ""
@@ -28,7 +28,7 @@ class Signing(OESelftestTestCase):
         cls.pub_key_path = os.path.join(cls.testlayer_path, 'files', 'signing', "key.pub")
         cls.secret_key_path = os.path.join(cls.testlayer_path, 'files', 'signing', "key.secret")
 
-        runCmd('gpg --batch --homedir %s --import %s %s' % (cls.gpg_dir, cls.pub_key_path, cls.secret_key_path))
+        cls.runCmd('gpg --batch --homedir %s --import %s %s' % (cls.gpg_dir, cls.pub_key_path, cls.secret_key_path))
 
     @OETestID(1362)
     def test_signing_packages(self):
@@ -43,7 +43,7 @@ class Signing(OESelftestTestCase):
         """
         import oe.packagedata
 
-        package_classes = get_bb_var('PACKAGE_CLASSES')
+        package_classes = self.get_bb_var('PACKAGE_CLASSES')
         if 'package_rpm' not in package_classes:
             self.skipTest('This test requires RPM Packaging.')
 
@@ -56,13 +56,13 @@ class Signing(OESelftestTestCase):
 
         self.write_config(feature)
 
-        bitbake('-c clean %s' % test_recipe)
-        bitbake('-f -c package_write_rpm %s' % test_recipe)
+        self.bitbake('-c clean %s' % test_recipe)
+        self.bitbake('-f -c package_write_rpm %s' % test_recipe)
 
         self.add_command_to_tearDown('bitbake -c clean %s' % test_recipe)
 
         needed_vars = ['PKGDATA_DIR', 'DEPLOY_DIR_RPM', 'PACKAGE_ARCH', 'STAGING_BINDIR_NATIVE']
-        bb_vars = get_bb_vars(needed_vars, test_recipe)
+        bb_vars = self.get_bb_vars(needed_vars, test_recipe)
         pkgdatadir = bb_vars['PKGDATA_DIR']
         pkgdata = oe.packagedata.read_pkgdatafile(pkgdatadir + "/runtime/ed")
         if 'PKGE' in pkgdata:
@@ -78,10 +78,10 @@ class Signing(OESelftestTestCase):
         # Use a temporary rpmdb
         rpmdb = tempfile.mkdtemp(prefix='oeqa-rpmdb')
 
-        runCmd('%s/rpmkeys --define "_dbpath %s" --import %s' %
+        self.runCmd('%s/rpmkeys --define "_dbpath %s" --import %s' %
                (staging_bindir_native, rpmdb, self.pub_key_path))
 
-        ret = runCmd('%s/rpmkeys --define "_dbpath %s" --checksig %s' %
+        ret = self.runCmd('%s/rpmkeys --define "_dbpath %s" --checksig %s' %
                      (staging_bindir_native, rpmdb, pkg_deploy))
         # tmp/deploy/rpm/i586/ed-1.9-r0.i586.rpm: rsa sha1 md5 OK
         self.assertIn('rsa sha1 (md5) pgp md5 OK', ret.output, 'Package signed incorrectly.')
@@ -89,8 +89,8 @@ class Signing(OESelftestTestCase):
 
         #Check that an image can be built from signed packages
         self.add_command_to_tearDown('bitbake -c clean core-image-minimal')
-        bitbake('-c clean core-image-minimal')
-        bitbake('core-image-minimal')
+        self.bitbake('-c clean core-image-minimal')
+        self.bitbake('core-image-minimal')
 
 
     @OETestID(1382)
@@ -121,8 +121,8 @@ class Signing(OESelftestTestCase):
 
         self.write_config(feature)
 
-        bitbake('-c clean %s' % test_recipe)
-        bitbake(test_recipe)
+        self.bitbake('-c clean %s' % test_recipe)
+        self.bitbake(test_recipe)
 
         recipe_sig = glob.glob(sstatedir + '/*/*:ed:*_package.tgz.sig')
         recipe_tgz = glob.glob(sstatedir + '/*/*:ed:*_package.tgz')
@@ -130,7 +130,7 @@ class Signing(OESelftestTestCase):
         self.assertEqual(len(recipe_sig), 1, 'Failed to find .sig file.')
         self.assertEqual(len(recipe_tgz), 1, 'Failed to find .tgz file.')
 
-        ret = runCmd('gpg --homedir %s --verify %s %s' % (self.gpg_dir, recipe_sig[0], recipe_tgz[0]))
+        ret = self.runCmd('gpg --homedir %s --verify %s %s' % (self.gpg_dir, recipe_sig[0], recipe_tgz[0]))
         # gpg: Signature made Thu 22 Oct 2015 01:45:09 PM EEST using RSA key ID 61EEFB30
         # gpg: Good signature from "testuser (nocomment) <[email protected]>"
         self.assertIn('gpg: Good signature from', ret.output, 'Package signed incorrectly.')
@@ -153,19 +153,19 @@ class LockedSignatures(OESelftestTestCase):
 
         self.add_command_to_tearDown('rm -f %s' % os.path.join(self.builddir, locked_sigs_file))
 
-        bitbake(test_recipe)
+        self.bitbake(test_recipe)
         # Generate locked sigs include file
-        bitbake('-S none %s' % test_recipe)
+        self.bitbake('-S none %s' % test_recipe)
 
         feature = 'require %s\n' % locked_sigs_file
         feature += 'SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "warn"\n'
         self.write_config(feature)
 
         # Build a locked recipe
-        bitbake(test_recipe)
+        self.bitbake(test_recipe)
 
         # Make a change that should cause the locked task signature to change
-        recipe_append_file = test_recipe + '_' + get_bb_var('PV', test_recipe) + '.bbappend'
+        recipe_append_file = test_recipe + '_' + self.get_bb_var('PV', test_recipe) + '.bbappend'
         recipe_append_path = os.path.join(self.testlayer_path, 'recipes-test', test_recipe, recipe_append_file)
         feature = 'SUMMARY += "test locked signature"\n'
 
@@ -175,7 +175,7 @@ class LockedSignatures(OESelftestTestCase):
         self.add_command_to_tearDown('rm -rf %s' % os.path.join(self.testlayer_path, 'recipes-test', test_recipe))
 
         # Build the recipe again
-        ret = bitbake(test_recipe)
+        ret = self.bitbake(test_recipe)
 
         # Verify you get the warning and that the real task *isn't* run (i.e. the locked signature has worked)
         patt = r'WARNING: The %s:do_package sig is computed to be \S+, but the sig is locked to \S+ in SIGGEN_LOCKEDSIGS\S+' % test_recipe
meta/lib/oeqa/selftest/cases/sstate.py +2 -3
index b8c2880ad06..e29787f6e11 100644
@@ -6,7 +6,6 @@ import shutil
 
 import oeqa.utils.ftools as ftools
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_vars
 
 
 class SStateBase(OESelftestTestCase):
@@ -16,7 +15,7 @@ class SStateBase(OESelftestTestCase):
         self.temp_sstate_location = None
         needed_vars = ['SSTATE_DIR', 'NATIVELSBSTRING', 'TCLIBC', 'TUNE_ARCH',
                        'TOPDIR', 'TARGET_VENDOR', 'TARGET_OS']
-        bb_vars = get_bb_vars(needed_vars)
+        bb_vars = self.get_bb_vars(needed_vars)
         self.sstate_path = bb_vars['SSTATE_DIR']
         self.hostdistro = bb_vars['NATIVELSBSTRING']
         self.tclibc = bb_vars['TCLIBC']
@@ -35,7 +34,7 @@ class SStateBase(OESelftestTestCase):
             config_temp_sstate = "SSTATE_DIR = \"%s\"" % temp_sstate_path
             self.append_config(config_temp_sstate)
             self.track_for_cleanup(temp_sstate_path)
-        bb_vars = get_bb_vars(['SSTATE_DIR', 'NATIVELSBSTRING'])
+        bb_vars = self.get_bb_vars(['SSTATE_DIR', 'NATIVELSBSTRING'])
         self.sstate_path = bb_vars['SSTATE_DIR']
         self.hostdistro = bb_vars['NATIVELSBSTRING']
         self.distro_specific_sstate = os.path.join(self.sstate_path, self.hostdistro)
meta/lib/oeqa/selftest/cases/sstatetests.py +26 -25

166 lines not on this page · raw message

Signature · 1 line
2.11.0
#30 lore raw

PATCH 18/30 oeqa/selftest/cases: tinfoil to run in the main thread

In reply to Aníbal Limón

The tinfoil tests needs to initialize bitbake internals and wasn't designed to be in a thread environment causing problems when try for example set signal handlers in a thread different than the main one.

To workaround this execute tinfoil tests in the main thread and don't use custom builddir because isn't make sense too.

---
 meta/lib/oeqa/selftest/cases/tinfoil.py | 1 -
 1 file changed, 1 deletion(-)
meta/lib/oeqa/selftest/cases/tinfoil.py +0 -1
index aa1af7e0423..7248b755812 100644
@@ -9,7 +9,6 @@ from oeqa.core.decorator.oeid import OETestID
 
 class TinfoilTests(OESelftestTestCase):
     """ Basic tests for the tinfoil API """
-
     @OETestID(1568)
     def test_getvar(self):
         with bb.tinfoil.Tinfoil() as tinfoil:
Signature · 1 line
2.11.0
#33 lore raw

PATCH 19/30 oeqa/selftest/cases: imagefeatures enable threaded runs

In reply to Aníbal Limón

- Change to use wrapper methos from OESelfTestCase.
- Move tests that use runqemu to its own module because it needs tinfoil and run in the main thread.

---
 meta/lib/oeqa/selftest/cases/imagefeatures.py      | 78 +++-------------------
 meta/lib/oeqa/selftest/cases/imagefeatures_boot.py | 63 +++++++++++++++++
 2 files changed, 73 insertions(+), 68 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/imagefeatures_boot.py
meta/lib/oeqa/selftest/cases/imagefeatures.py +10 -68
index e6652ec7aba..325be934419 100644
@@ -1,69 +1,11 @@
+import os
+
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
 from oeqa.core.decorator.oeid import OETestID
-from oeqa.utils.sshcontrol import SSHControl
-import os
 
 class ImageFeatures(OESelftestTestCase):
-
-    test_user = 'tester'
-    root_user = 'root'
-
-    @OETestID(1107)
-    def test_non_root_user_can_connect_via_ssh_without_password(self):
-        """
-        Summary: Check if non root user can connect via ssh without password
-        Expected: 1. Connection to the image via ssh using root user without providing a password should be allowed.
-                  2. Connection to the image via ssh using tester user without providing a password should be allowed.
-        Product: oe-core
-        Author: Ionut Chisanovici <[email protected]>
-        AutomatedBy: Daniel Istrate <[email protected]>
-        """
-
-        features = 'EXTRA_IMAGE_FEATURES = "ssh-server-openssh empty-root-password allow-empty-password"\n'
-        features += 'INHERIT += "extrausers"\n'
-        features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
-        self.write_config(features)
-
-        # Build a core-image-minimal
-        bitbake('core-image-minimal')
-
-        with runqemu("core-image-minimal") as qemu:
-            # Attempt to ssh with each user into qemu with empty password
-            for user in [self.root_user, self.test_user]:
-                ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
-                status, output = ssh.run("true")
-                self.assertEqual(status, 0, 'ssh to user %s failed with %s' % (user, output))
-
-    @OETestID(1115)
-    def test_all_users_can_connect_via_ssh_without_password(self):
-        """
-        Summary:     Check if all users can connect via ssh without password
-        Expected: 1. Connection to the image via ssh using root user without providing a password should NOT be allowed.
-                  2. Connection to the image via ssh using tester user without providing a password should be allowed.
-        Product:     oe-core
-        Author:      Ionut Chisanovici <[email protected]>
-        AutomatedBy: Daniel Istrate <[email protected]>
-        """
-
-        features = 'EXTRA_IMAGE_FEATURES = "ssh-server-openssh allow-empty-password"\n'
-        features += 'INHERIT += "extrausers"\n'
-        features += 'EXTRA_USERS_PARAMS = "useradd -p \'\' {}; usermod -s /bin/sh {};"'.format(self.test_user, self.test_user)
-        self.write_config(features)
-
-        # Build a core-image-minimal
-        bitbake('core-image-minimal')
-
-        with runqemu("core-image-minimal") as qemu:
-            # Attempt to ssh with each user into qemu with empty password
-            for user in [self.root_user, self.test_user]:
-                ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user=user)
-                status, output = ssh.run("true")
-                if user == 'root':
-                    self.assertNotEqual(status, 0, 'ssh to user root was allowed when it should not have been')
-                else:
-                    self.assertEqual(status, 0, 'ssh to user tester failed with %s' % output)
-
+    _use_own_builddir = True
+    _main_thread = False
 
     @OETestID(1116)
     def test_clutter_image_can_be_built(self):
@@ -76,7 +18,7 @@ class ImageFeatures(OESelftestTestCase):
         """
 
         # Build a core-image-clutter
-        bitbake('core-image-clutter')
+        self.bitbake('core-image-clutter')
 
     @OETestID(1117)
     def test_wayland_support_in_image(self):
@@ -89,12 +31,12 @@ class ImageFeatures(OESelftestTestCase):
         AutomatedBy: Daniel Istrate <[email protected]>
         """
 
-        distro_features = get_bb_var('DISTRO_FEATURES')
+        distro_features = self.get_bb_var('DISTRO_FEATURES')
         if not ('opengl' in distro_features and 'wayland' in distro_features):
             self.skipTest('neither opengl nor wayland present on DISTRO_FEATURES so core-image-weston cannot be built')
 
         # Build a core-image-weston
-        bitbake('core-image-weston')
+        self.bitbake('core-image-weston')
 
     @OETestID(1497)
     def test_bmap(self):
@@ -110,10 +52,10 @@ class ImageFeatures(OESelftestTestCase):
         self.write_config(features)
 
         image_name = 'core-image-minimal'
-        bitbake(image_name)
+        self.bitbake(image_name)
 
-        deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
-        link_name = get_bb_var('IMAGE_LINK_NAME', image_name)
+        deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
+        link_name = self.get_bb_var('IMAGE_LINK_NAME', image_name)
         image_path = os.path.join(deploy_dir_image, "%s.ext4" % link_name)
         bmap_path = "%s.bmap" % image_path
 
meta/lib/oeqa/selftest/cases/imagefeatures_boot.py new file +63 -0

63 lines not on this page · raw message

Signature · 1 line
2.11.0
#34 lore raw

PATCH 20/30 oeqa/selftest/cases: runqemu enable thraded runs

In reply to Aníbal Limón

- Update to use wrappers {bitbake,get_bb_var} from OESelfTestCase class.
- Run into the main thread because it needs tinfoil to run.

---
 meta/lib/oeqa/selftest/cases/runqemu.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/cases/runqemu.py +3 -4
index 4050a4123ba..e30cb24046f 100644
@@ -6,12 +6,11 @@ import re
 import logging
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, runqemu, get_bb_var
+from oeqa.utils.commands import runqemu
 from oeqa.core.decorator.oeid import OETestID
 
 class RunqemuTests(OESelftestTestCase):
     """Runqemu test class"""
-
     image_is_ready = False
     deploy_dir_image = ''
 
@@ -37,8 +36,8 @@ SYSLINUX_TIMEOUT = "10"
         )
 
         if not RunqemuTests.image_is_ready:
-            RunqemuTests.deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
-            bitbake(self.recipe)
+            RunqemuTests.deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
+            self.bitbake(self.recipe)
             RunqemuTests.image_is_ready = True
 
     @OETestID(2001)
Signature · 1 line
2.11.0
#39 lore raw

PATCH 21/30 oeqa/selftest/cases: runtime enable threaded runs

In reply to Aníbal Limón

- Use wrappers from OESelfTestCase for runCmd, bitbake, etc.
- Split into tree modules because runtime_test_export and runtime_test_postinsts uses runqemu/tinfoil and needs to be executed into the main thread.

---
 meta/lib/oeqa/selftest/cases/runtime_test.py       | 225 +--------------------
 .../lib/oeqa/selftest/cases/runtime_test_export.py | 104 ++++++++++
 .../oeqa/selftest/cases/runtime_test_postinsts.py  | 114 +++++++++++
 3 files changed, 226 insertions(+), 217 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/runtime_test_export.py
 create mode 100644 meta/lib/oeqa/selftest/cases/runtime_test_postinsts.py
meta/lib/oeqa/selftest/cases/runtime_test.py +8 -217
index 2a70ae15b81..e45410e0e56 100644
@@ -1,110 +1,9 @@
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars, runqemu
 from oeqa.core.decorator.oeid import OETestID
-import os
-import re
-
-class TestExport(OESelftestTestCase):
-
-    @classmethod
-    def tearDownClass(cls):
-        runCmd("rm -rf /tmp/sdk")
-        super(TestExport, cls).tearDownClass()
-
-    @OETestID(1499)
-    def test_testexport_basic(self):
-        """
-        Summary: Check basic testexport functionality with only ping test enabled.
-        Expected: 1. testexport directory must be created.
-                  2. runexported.py must run without any error/exception.
-                  3. ping test must succeed.
-        Product: oe-core
-        Author: Mariano Lopez <[email protected]>
-        """
-
-        features = 'INHERIT += "testexport"\n'
-        # These aren't the actual IP addresses but testexport class needs something defined
-        features += 'TEST_SERVER_IP = "192.168.7.1"\n'
-        features += 'TEST_TARGET_IP = "192.168.7.1"\n'
-        features += 'TEST_SUITES = "ping"\n'
-        self.write_config(features)
-
-        # Build tesexport for core-image-minimal
-        bitbake('core-image-minimal')
-        bitbake('-c testexport core-image-minimal')
-
-        testexport_dir = get_bb_var('TEST_EXPORT_DIR', 'core-image-minimal')
-
-        # Verify if TEST_EXPORT_DIR was created
-        isdir = os.path.isdir(testexport_dir)
-        self.assertEqual(True, isdir, 'Failed to create testexport dir: %s' % testexport_dir)
-
-        with runqemu('core-image-minimal') as qemu:
-            # Attempt to run runexported.py to perform ping test
-            test_path = os.path.join(testexport_dir, "oe-test")
-            data_file = os.path.join(testexport_dir, 'data', 'testdata.json')
-            manifest = os.path.join(testexport_dir, 'data', 'manifest')
-            cmd = ("%s runtime --test-data-file %s --packages-manifest %s "
-                   "--target-ip %s --server-ip %s --quiet"
-                  % (test_path, data_file, manifest, qemu.ip, qemu.server_ip))
-            result = runCmd(cmd)
-            # Verify ping test was succesful
-            self.assertEqual(0, result.status, 'oe-test runtime returned a non 0 status')
-
-    @OETestID(1641)
-    def test_testexport_sdk(self):
-        """
-        Summary: Check sdk functionality for testexport.
-        Expected: 1. testexport directory must be created.
-                  2. SDK tarball must exists.
-                  3. Uncompressing of tarball must succeed.
-                  4. Check if the SDK directory is added to PATH.
-                  5. Run tar from the SDK directory.
-        Product: oe-core
-        Author: Mariano Lopez <[email protected]>
-        """
-
-        features = 'INHERIT += "testexport"\n'
-        # These aren't the actual IP addresses but testexport class needs something defined
-        features += 'TEST_SERVER_IP = "192.168.7.1"\n'
-        features += 'TEST_TARGET_IP = "192.168.7.1"\n'
-        features += 'TEST_SUITES = "ping"\n'
-        features += 'TEST_EXPORT_SDK_ENABLED = "1"\n'
-        features += 'TEST_EXPORT_SDK_PACKAGES = "nativesdk-tar"\n'
-        self.write_config(features)
-
-        # Build tesexport for core-image-minimal
-        bitbake('core-image-minimal')
-        bitbake('-c testexport core-image-minimal')
-
-        needed_vars = ['TEST_EXPORT_DIR', 'TEST_EXPORT_SDK_DIR', 'TEST_EXPORT_SDK_NAME']
-        bb_vars = get_bb_vars(needed_vars, 'core-image-minimal')
-        testexport_dir = bb_vars['TEST_EXPORT_DIR']
-        sdk_dir = bb_vars['TEST_EXPORT_SDK_DIR']
-        sdk_name = bb_vars['TEST_EXPORT_SDK_NAME']
-
-        # Check for SDK
-        tarball_name = "%s.sh" % sdk_name
-        tarball_path = os.path.join(testexport_dir, sdk_dir, tarball_name)
-        msg = "Couldn't find SDK tarball: %s" % tarball_path
-        self.assertEqual(os.path.isfile(tarball_path), True, msg)
-
-        # Extract SDK and run tar from SDK
-        result = runCmd("%s -y -d /tmp/sdk" % tarball_path)
-        self.assertEqual(0, result.status, "Couldn't extract SDK")
-
-        env_script = result.output.split()[-1]
-        result = runCmd(". %s; which tar" % env_script, shell=True)
-        self.assertEqual(0, result.status, "Couldn't setup SDK environment")
-        is_sdk_tar = True if "/tmp/sdk" in result.output else False
-        self.assertTrue(is_sdk_tar, "Couldn't setup SDK environment")
-
-        tar_sdk = result.output
-        result = runCmd("%s --version" % tar_sdk)
-        self.assertEqual(0, result.status, "Couldn't run tar from SDK")
-
 
 class TestImage(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @OETestID(1644)
     def test_testimage_install(self):
@@ -115,7 +14,7 @@ class TestImage(OESelftestTestCase):
         Product: oe-core
         Author: Mariano Lopez <[email protected]>
         """
-        if get_bb_var('DISTRO') == 'poky-tiny':
+        if self.get_bb_var('DISTRO') == 'poky-tiny':
             self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
 
         features = 'INHERIT += "testimage"\n'
@@ -123,8 +22,8 @@ class TestImage(OESelftestTestCase):
         self.write_config(features)
 
         # Build core-image-sato and testimage
-        bitbake('core-image-full-cmdline socat')
-        bitbake('-c testimage core-image-full-cmdline')
+        self.bitbake('core-image-full-cmdline socat')
+        self.bitbake('-c testimage core-image-full-cmdline')
 
     @OETestID(1883)
     def test_testimage_dnf(self):
@@ -134,7 +33,7 @@ class TestImage(OESelftestTestCase):
         Product: oe-core
         Author: Alexander Kanavin <[email protected]>
         """
-        if get_bb_var('DISTRO') == 'poky-tiny':
+        if self.get_bb_var('DISTRO') == 'poky-tiny':
             self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
 
         features = 'INHERIT += "testimage"\n'
@@ -147,113 +46,5 @@ class TestImage(OESelftestTestCase):
         self.write_config(features)
 
         # Build core-image-sato and testimage
-        bitbake('core-image-full-cmdline socat')
-        bitbake('-c testimage core-image-full-cmdline')
-
-class Postinst(OESelftestTestCase):
-    @OETestID(1540)
-    def test_verify_postinst(self):
-        """
-        Summary: The purpose of this test is to verify the execution order of postinst Bugzilla ID: [5319]
-        Expected :
-        1. Compile a minimal image.
-        2. The compiled image will add the created layer with the recipes postinst[ abdpt]
-        3. Run qemux86
-        4. Validate the task execution order
-        Author: Francisco Pedraza <[email protected]>
-        """
-        features = 'INHERIT += "testimage"\n'
-        features += 'CORE_IMAGE_EXTRA_INSTALL += "postinst-at-rootfs \
-postinst-delayed-a \
-postinst-delayed-b \
-postinst-delayed-d \
-postinst-delayed-p \
-postinst-delayed-t \
-"\n'
-        self.write_config(features)
-
-        bitbake('core-image-minimal -f ')
-
-        postinst_list = ['100-postinst-at-rootfs',
-                         '101-postinst-delayed-a',
-                         '102-postinst-delayed-b',
-                         '103-postinst-delayed-d',
-                         '104-postinst-delayed-p',
-                         '105-postinst-delayed-t']
-        path_workdir = get_bb_var('WORKDIR','core-image-minimal')
-        workspacedir = 'testimage/qemu_boot_log'
-        workspacedir = os.path.join(path_workdir, workspacedir)
-        rexp = re.compile("^Running postinst .*/(?P<postinst>.*)\.\.\.$")
-        with runqemu('core-image-minimal') as qemu:
-            with open(workspacedir) as f:
-                found = False
-                idx = 0
-                for line in f.readlines():
-                    line = line.strip().replace("^M","")
-                    if not line: # To avoid empty lines
-                        continue
-                    m = rexp.search(line)
-                    if m:
-                        self.assertEqual(postinst_list[idx], m.group('postinst'), "Fail")
-                        idx = idx+1
-                        found = True
-                    elif found:
-                        self.assertEqual(idx, len(postinst_list), "Not found all postinsts")
-                        break
-
-    @OETestID(1545)
-    def test_postinst_rootfs_and_boot(self):
-        """
-        Summary:        The purpose of this test case is to verify Post-installation
-                        scripts are called when rootfs is created and also test
-                        that script can be delayed to run at first boot.
-        Dependencies:   NA
-        Steps:          1. Add proper configuration to local.conf file
-                        2. Build a "core-image-minimal" image
-                        3. Verify that file created by postinst_rootfs recipe is
-                           present on rootfs dir.
-                        4. Boot the image created on qemu and verify that the file
-                           created by postinst_boot recipe is present on image.
-        Expected:       The files are successfully created during rootfs and boot
-                        time for 3 different package managers: rpm,ipk,deb and
-                        for initialization managers: sysvinit and systemd.
-
-        """
-        file_rootfs_name = "this-was-created-at-rootfstime"
-        fileboot_name = "this-was-created-at-first-boot"
-        rootfs_pkg = 'postinst-at-rootfs'
-        boot_pkg = 'postinst-delayed-a'
-        #Step 1
-        common_features = 'MACHINE = "qemux86"\n'
-        common_features += 'CORE_IMAGE_EXTRA_INSTALL += "%s %s "\n'% (rootfs_pkg, boot_pkg)
-        common_features += 'IMAGE_FEATURES += "ssh-server-openssh"\n'
-        for init_manager in ("sysvinit", "systemd"):
-            #for sysvinit no extra configuration is needed,
-            features = ''
-            if (init_manager is "systemd"):
-                features += 'DISTRO_FEATURES_append = " systemd"\n'
-                features += 'VIRTUAL-RUNTIME_init_manager = "systemd"\n'
-                features += 'DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"\n'
-                features += 'VIRTUAL-RUNTIME_initscripts = ""\n'
-            for classes in ("package_rpm package_deb package_ipk",
-                            "package_deb package_rpm package_ipk",
-                            "package_ipk package_deb package_rpm"):
-                features += 'PACKAGE_CLASSES = "%s"\n' % classes
-                self.write_config(common_features + features)
-
-                #Step 2
-                bitbake('core-image-minimal')
-
-                #Step 3
-                file_rootfs_created = os.path.join(get_bb_var('IMAGE_ROOTFS',"core-image-minimal"),
-                                                   file_rootfs_name)
-                found = os.path.isfile(file_rootfs_created)
-                self.assertTrue(found, "File %s was not created at rootfs time by %s" % \
-                                (file_rootfs_name, rootfs_pkg))
-
-                #Step 4
-                testcommand = 'ls /etc/'+fileboot_name
-                with runqemu('core-image-minimal') as qemu:
-                    sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
-                    result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand))
-                    self.assertEqual(result.status, 0, 'File %s was not created at firts boot'% fileboot_name)
+        self.bitbake('core-image-full-cmdline socat')
+        self.bitbake('-c testimage core-image-full-cmdline')
meta/lib/oeqa/selftest/cases/runtime_test_export.py new file +104 -0

104 lines not on this page · raw message

meta/lib/oeqa/selftest/cases/runtime_test_postinsts.py new file +114 -0

114 lines not on this page · raw message

Signature · 1 line
2.11.0
#40 lore raw

PATCH 22/30 oeqa/selftest/cases: eSDK enable threaded runs

In reply to Aníbal Limón

- Change some staticmethods to classmethods to be able access wrappers from OESelfTestCase.
- Remove unused method update_configuration.

---
 meta/lib/oeqa/selftest/cases/eSDK.py | 61 +++++++++++-------------------------
 1 file changed, 19 insertions(+), 42 deletions(-)
meta/lib/oeqa/selftest/cases/eSDK.py +19 -42
index 60f4e239ab0..618f93c53c1 100644
@@ -4,7 +4,6 @@ import os
 import glob
 from oeqa.core.decorator.oeid import OETestID
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
 
 class oeSDKExtSelfTest(OESelftestTestCase):
     """
@@ -12,16 +11,15 @@ class oeSDKExtSelfTest(OESelftestTestCase):
     # This code is planned to be part of the automation for eSDK containig
     # Install libraries and headers, image generation binary feeds, sdk-update.
     """
-
-    @staticmethod
-    def get_esdk_environment(env_eSDK, tmpdir_eSDKQA):
+    @classmethod
+    def get_esdk_environment(cls, env_eSDK, tmpdir_eSDKQA):
         # XXX: at this time use the first env need to investigate
         # what environment load oe-selftest, i586, x86_64
         pattern = os.path.join(tmpdir_eSDKQA, 'environment-setup-*')
         return glob.glob(pattern)[0]
 
-    @staticmethod
-    def run_esdk_cmd(env_eSDK, tmpdir_eSDKQA, cmd, postconfig=None, **options):
+    @classmethod
+    def run_esdk_cmd(cls, env_eSDK, tmpdir_eSDKQA, cmd, postconfig=None, **options):
         if postconfig:
             esdk_conf_file = os.path.join(tmpdir_eSDKQA, 'conf', 'local.conf')
             with open(esdk_conf_file, 'a+') as f:
@@ -31,57 +29,37 @@ class oeSDKExtSelfTest(OESelftestTestCase):
         if not 'shell' in options:
             options['shell'] = True
 
-        runCmd("cd %s; . %s; %s" % (tmpdir_eSDKQA, env_eSDK, cmd), **options)
+        cls.runCmd("cd %s; . %s; %s" % (tmpdir_eSDKQA, env_eSDK, cmd), **options)
 
-    @staticmethod
-    def generate_eSDK(image):
+    @classmethod
+    def generate_eSDK(cls, image):
         pn_task = '%s -c populate_sdk_ext' % image
-        bitbake(pn_task)
+        cls.bitbake(pn_task)
 
-    @staticmethod
-    def get_eSDK_toolchain(image):
+    @classmethod
+    def get_eSDK_toolchain(cls, image):
         pn_task = '%s -c populate_sdk_ext' % image
 
-        bb_vars = get_bb_vars(['SDK_DEPLOY', 'TOOLCHAINEXT_OUTPUTNAME'], pn_task)
+        bb_vars = cls.get_bb_vars(['SDK_DEPLOY', 'TOOLCHAINEXT_OUTPUTNAME'], pn_task)
         sdk_deploy = bb_vars['SDK_DEPLOY']
         toolchain_name = bb_vars['TOOLCHAINEXT_OUTPUTNAME']
         return os.path.join(sdk_deploy, toolchain_name + '.sh')
 
-    @staticmethod
-    def update_configuration(cls, image, tmpdir_eSDKQA, env_eSDK, ext_sdk_path):
-        sstate_dir = os.path.join(cls.builddir, 'sstate-cache')
-
-        oeSDKExtSelfTest.generate_eSDK(cls.image)
-
-        cls.ext_sdk_path = oeSDKExtSelfTest.get_eSDK_toolchain(cls.image)
-        runCmd("%s -y -d \"%s\"" % (cls.ext_sdk_path, cls.tmpdir_eSDKQA))
-
-        cls.env_eSDK = oeSDKExtSelfTest.get_esdk_environment('', cls.tmpdir_eSDKQA)
-
-        sstate_config="""
-SDK_LOCAL_CONF_WHITELIST = "SSTATE_MIRRORS"
-SSTATE_MIRRORS =  "file://.* file://%s/PATH"
-CORE_IMAGE_EXTRA_INSTALL = "perl"
-        """ % sstate_dir
-
-        with open(os.path.join(cls.tmpdir_eSDKQA, 'conf', 'local.conf'), 'a+') as f:
-            f.write(sstate_config)
-
     @classmethod
     def setUpClass(cls):
         super(oeSDKExtSelfTest, cls).setUpClass()
         cls.tmpdir_eSDKQA = tempfile.mkdtemp(prefix='eSDKQA')
 
-        sstate_dir = get_bb_var('SSTATE_DIR')
+        sstate_dir = cls.get_bb_var('SSTATE_DIR')
 
         cls.image = 'core-image-minimal'
-        oeSDKExtSelfTest.generate_eSDK(cls.image)
+        cls.generate_eSDK(cls.image)
 
         # Install eSDK
-        cls.ext_sdk_path = oeSDKExtSelfTest.get_eSDK_toolchain(cls.image)
-        runCmd("%s -y -d \"%s\"" % (cls.ext_sdk_path, cls.tmpdir_eSDKQA))
+        cls.ext_sdk_path = cls.get_eSDK_toolchain(cls.image)
+        cls.runCmd("%s -y -d \"%s\"" % (cls.ext_sdk_path, cls.tmpdir_eSDKQA))
 
-        cls.env_eSDK = oeSDKExtSelfTest.get_esdk_environment('', cls.tmpdir_eSDKQA)
+        cls.env_eSDK = cls.get_esdk_environment('', cls.tmpdir_eSDKQA)
 
         # Configure eSDK to use sstate mirror from poky
         sstate_config="""
@@ -99,13 +77,12 @@ SSTATE_MIRRORS =  "file://.* file://%s/PATH"
     @OETestID(1602)
     def test_install_libraries_headers(self):
         pn_sstate = 'bc'
-        bitbake(pn_sstate)
+        self.bitbake(pn_sstate)
         cmd = "devtool sdk-install %s " % pn_sstate
-        oeSDKExtSelfTest.run_esdk_cmd(self.env_eSDK, self.tmpdir_eSDKQA, cmd)
+        self.run_esdk_cmd(self.env_eSDK, self.tmpdir_eSDKQA, cmd)
 
     @OETestID(1603)
     def test_image_generation_binary_feeds(self):
         image = 'core-image-minimal'
         cmd = "devtool build-image %s" % image
-        oeSDKExtSelfTest.run_esdk_cmd(self.env_eSDK, self.tmpdir_eSDKQA, cmd)
-
+        self.run_esdk_cmd(self.env_eSDK, self.tmpdir_eSDKQA, cmd)
Signature · 1 line
2.11.0
#41 lore raw

PATCH 23/30 oeqa/selftest/cases: devtool enable threaded runs

In reply to Aníbal Limón

- Add new class DevtoolCommon to split devtool suite into other modules.
- Change to use wrappers from OESelfTest class (runCmd, bitbake).
- When remove workspacedir using bitbake-layers use full path because now has it own builddir.
- Add a build of xz-native because some tests needs it to decompress tarballs and is supposed to be provided, see bug [YOCTO #11474].

---
 meta/lib/oeqa/selftest/cases/devtool.py | 644 ++++++++++++++++----------------
 1 file changed, 325 insertions(+), 319 deletions(-)
meta/lib/oeqa/selftest/cases/devtool.py +325 -319
index 28c84679a17..c66e77a9ace 100644
@@ -6,13 +6,12 @@ import glob
 import fnmatch
 
 import oeqa.utils.ftools as ftools
+from oeqa.utils.commands import create_temp_layer, runqemu
+
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer
-from oeqa.utils.commands import get_bb_vars, runqemu
 from oeqa.core.decorator.oeid import OETestID
 
 class DevtoolBase(OESelftestTestCase):
-
     def _test_recipe_contents(self, recipefile, checkvars, checkinherits):
         with open(recipefile, 'r') as f:
             invar = None
@@ -62,7 +61,7 @@ class DevtoolBase(OESelftestTestCase):
             self.assertIn(inherit, inherits, 'Missing inherit of %s' % inherit)
 
     def _check_bbappend(self, testrecipe, recipefile, appenddir):
-        result = runCmd('bitbake-layers show-appends', cwd=self.builddir)
+        result = self.runCmd('bitbake-layers show-appends', cwd=self.builddir)
         resultlines = result.output.splitlines()
         inrecipe = False
         bbappends = []
@@ -88,7 +87,7 @@ class DevtoolBase(OESelftestTestCase):
         create_temp_layer(templayerdir, templayername, priority, recipepathspec)
         if addlayer:
             self.add_command_to_tearDown('bitbake-layers remove-layer %s || true' % templayerdir)
-            result = runCmd('bitbake-layers add-layer %s' % templayerdir, cwd=self.builddir)
+            result = self.runCmd('bitbake-layers add-layer %s' % templayerdir, cwd=self.builddir)
 
     def _process_ls_output(self, output):
         """
@@ -114,47 +113,54 @@ class DevtoolBase(OESelftestTestCase):
         return filelist
 
 
-class DevtoolTests(DevtoolBase):
-
+class DevtoolCommon(DevtoolBase):
     @classmethod
     def setUpClass(cls):
-        super(DevtoolTests, cls).setUpClass()
-        bb_vars = get_bb_vars(['TOPDIR', 'SSTATE_DIR'])
+        super(DevtoolCommon, cls).setUpClass()
+        bb_vars = cls.get_bb_vars(['TOPDIR', 'SSTATE_DIR'])
         cls.original_sstate = bb_vars['SSTATE_DIR']
         cls.devtool_sstate = os.path.join(bb_vars['TOPDIR'], 'sstate_devtool')
         cls.sstate_conf  = 'SSTATE_DIR = "%s"\n' % cls.devtool_sstate
         cls.sstate_conf += ('SSTATE_MIRRORS += "file://.* file:///%s/PATH"\n'
                             % cls.original_sstate)
 
+        # XXX: some test cases needs xz-native to unpack like mdadm recipes
+        # and it is supposed to be provided
+        cls.bitbake("xz-native")
+
     @classmethod
     def tearDownClass(cls):
         cls.logger.debug('Deleting devtool sstate cache on %s' % cls.devtool_sstate)
-        runCmd('rm -rf %s' % cls.devtool_sstate)
-        super(DevtoolTests, cls).tearDownClass()
+        cls.runCmd('rm -rf %s' % cls.devtool_sstate)
+        super(DevtoolCommon, cls).tearDownClass()
 
     def setUp(self):
         """Test case setup function"""
-        super(DevtoolTests, self).setUp()
+        super(DevtoolCommon, self).setUp()
         self.workspacedir = os.path.join(self.builddir, 'workspace')
         self.assertTrue(not os.path.exists(self.workspacedir),
                         'This test cannot be run with a workspace directory '
                         'under the build directory')
         self.append_config(self.sstate_conf)
 
+class DevtoolTests(DevtoolCommon):
+    _use_own_builddir = True
+    _main_thread = False
+
     def _check_src_repo(self, repo_dir):
         """Check srctree git repository"""
         self.assertTrue(os.path.isdir(os.path.join(repo_dir, '.git')),
                         'git repository for external source tree not found')
-        result = runCmd('git status --porcelain', cwd=repo_dir)
+        result = self.runCmd('git status --porcelain', cwd=repo_dir)
         self.assertEqual(result.output.strip(), "",
                          'Created git repo is not clean')
-        result = runCmd('git symbolic-ref HEAD', cwd=repo_dir)
+        result = self.runCmd('git symbolic-ref HEAD', cwd=repo_dir)
         self.assertEqual(result.output.strip(), "refs/heads/devtool",
                          'Wrong branch in git repo')
 
     def _check_repo_status(self, repo_dir, expected_status):
         """Check the worktree status of a repository"""
-        result = runCmd('git status . --porcelain',
+        result = self.runCmd('git status . --porcelain',
                         cwd=repo_dir)
         for line in result.output.splitlines():
             for ind, (f_status, fn_re) in enumerate(expected_status):
@@ -171,21 +177,21 @@ class DevtoolTests(DevtoolBase):
     @OETestID(1158)
     def test_create_workspace(self):
         # Check preconditions
-        result = runCmd('bitbake-layers show-layers')
+        result = self.runCmd('bitbake-layers show-layers')
         self.assertTrue('/workspace' not in result.output, 'This test cannot be run with a workspace layer in bblayers.conf')
         # Try creating a workspace layer with a specific path
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
-        result = runCmd('devtool create-workspace %s' % tempdir)
+        result = self.runCmd('devtool create-workspace %s' % tempdir)
         self.assertTrue(os.path.isfile(os.path.join(tempdir, 'conf', 'layer.conf')), msg = "No workspace created. devtool output: %s " % result.output)
-        result = runCmd('bitbake-layers show-layers')
+        result = self.runCmd('bitbake-layers show-layers')
         self.assertIn(tempdir, result.output)
         # Try creating a workspace layer with the default path
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool create-workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool create-workspace')
         self.assertTrue(os.path.isfile(os.path.join(self.workspacedir, 'conf', 'layer.conf')), msg = "No workspace created. devtool output: %s " % result.output)
-        result = runCmd('bitbake-layers show-layers')
+        result = self.runCmd('bitbake-layers show-layers')
         self.assertNotIn(tempdir, result.output)
         self.assertIn(self.workspacedir, result.output)
 
@@ -195,25 +201,25 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         url = 'http://www.ivarch.com/programs/sources/pv-1.5.3.tar.bz2'
-        result = runCmd('wget %s' % url, cwd=tempdir)
-        result = runCmd('tar xfv pv-1.5.3.tar.bz2', cwd=tempdir)
+        result = self.runCmd('wget %s' % url, cwd=tempdir)
+        result = self.runCmd('tar xfv pv-1.5.3.tar.bz2', cwd=tempdir)
         srcdir = os.path.join(tempdir, 'pv-1.5.3')
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure')), 'Unable to find configure script in source directory')
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
         self.add_command_to_tearDown('bitbake -c cleansstate pv')
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool add pv %s' % srcdir)
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool add pv %s' % srcdir)
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn('pv', result.output)
         self.assertIn(srcdir, result.output)
         # Clean up anything in the workdir/sysroot/sstate cache (have to do this *after* devtool add since the recipe only exists then)
-        bitbake('pv -c cleansstate')
+        self.bitbake('pv -c cleansstate')
         # Test devtool build
-        result = runCmd('devtool build pv')
-        bb_vars = get_bb_vars(['D', 'bindir'], 'pv')
+        result = self.runCmd('devtool build pv')
+        bb_vars = self.get_bb_vars(['D', 'bindir'], 'pv')
         installdir = bb_vars['D']
         self.assertTrue(installdir, 'Could not query installdir variable')
         bindir = bb_vars['bindir']
@@ -233,22 +239,22 @@ class DevtoolTests(DevtoolBase):
         url = 'https://git.yoctoproject.org/git/dbus-wait'
         # Force fetching to "noname" subdir so we verify we're picking up the name from autoconf
         # instead of the directory name
-        result = runCmd('git clone %s noname' % url, cwd=tempdir)
+        result = self.runCmd('git clone %s noname' % url, cwd=tempdir)
         srcdir = os.path.join(tempdir, 'noname')
-        result = runCmd('git reset --hard %s' % srcrev, cwd=srcdir)
+        result = self.runCmd('git reset --hard %s' % srcrev, cwd=srcdir)
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure.ac')), 'Unable to find configure script in source directory')
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # Don't specify a name since we should be able to auto-detect it
-        result = runCmd('devtool add %s' % srcdir)
+        result = self.runCmd('devtool add %s' % srcdir)
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
         # Check the recipe name is correct
-        recipefile = get_bb_var('FILE', pn)
+        recipefile = self.get_bb_var('FILE', pn)
         self.assertIn('%s_git.bb' % pn, recipefile, 'Recipe file incorrectly named')
         self.assertIn(recipefile, result.output)
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(pn, result.output)
         self.assertIn(srcdir, result.output)
         self.assertIn(recipefile, result.output)
@@ -269,42 +275,42 @@ class DevtoolTests(DevtoolBase):
         self.track_for_cleanup(tempdir)
         version = '1.1'
         url = 'https://www.intra2net.com/en/developer/libftdi/download/libftdi1-%s.tar.bz2' % version
-        result = runCmd('wget %s' % url, cwd=tempdir)
-        result = runCmd('tar xfv libftdi1-%s.tar.bz2' % version, cwd=tempdir)
+        result = self.runCmd('wget %s' % url, cwd=tempdir)
+        result = self.runCmd('tar xfv libftdi1-%s.tar.bz2' % version, cwd=tempdir)
         srcdir = os.path.join(tempdir, 'libftdi1-%s' % version)
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'CMakeLists.txt')), 'Unable to find CMakeLists.txt in source directory')
         # Test devtool add (and use -V so we test that too)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool add libftdi %s -V %s' % (srcdir, version))
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool add libftdi %s -V %s' % (srcdir, version))
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn('libftdi', result.output)
         self.assertIn(srcdir, result.output)
         # Clean up anything in the workdir/sysroot/sstate cache (have to do this *after* devtool add since the recipe only exists then)
-        bitbake('libftdi -c cleansstate')
+        self.bitbake('libftdi -c cleansstate')
         # libftdi's python/CMakeLists.txt is a bit broken, so let's just disable it
         # There's also the matter of it installing cmake files to a path we don't
         # normally cover, which triggers the installed-vs-shipped QA test we have
         # within do_package
         recipefile = '%s/recipes/libftdi/libftdi_%s.bb' % (self.workspacedir, version)
-        result = runCmd('recipetool setvar %s EXTRA_OECMAKE -- \'-DPYTHON_BINDINGS=OFF -DLIBFTDI_CMAKE_CONFIG_DIR=${datadir}/cmake/Modules\'' % recipefile)
+        result = self.runCmd('recipetool setvar %s EXTRA_OECMAKE -- \'-DPYTHON_BINDINGS=OFF -DLIBFTDI_CMAKE_CONFIG_DIR=${datadir}/cmake/Modules\'' % recipefile)
         with open(recipefile, 'a') as f:
             f.write('\nFILES_${PN}-dev += "${datadir}/cmake/Modules"\n')
             # We don't have the ability to pick up this dependency automatically yet...
             f.write('\nDEPENDS += "libusb1"\n')
             f.write('\nTESTLIBOUTPUT = "${COMPONENTS_DIR}/${TUNE_PKGARCH}/${PN}/${libdir}"\n')
         # Test devtool build
-        result = runCmd('devtool build libftdi')
-        bb_vars = get_bb_vars(['TESTLIBOUTPUT', 'STAMP'], 'libftdi')
+        result = self.runCmd('devtool build libftdi')
+        bb_vars = self.get_bb_vars(['TESTLIBOUTPUT', 'STAMP'], 'libftdi')
         staging_libdir = bb_vars['TESTLIBOUTPUT']
         self.assertTrue(staging_libdir, 'Could not query TESTLIBOUTPUT variable')
         self.assertTrue(os.path.isfile(os.path.join(staging_libdir, 'libftdi1.so.2.1.0')), "libftdi binary not found in STAGING_LIBDIR. Output of devtool build libftdi %s" % result.output)
         # Test devtool reset
         stampprefix = bb_vars['STAMP']
-        result = runCmd('devtool reset libftdi')
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool reset libftdi')
+        result = self.runCmd('devtool status')
         self.assertNotIn('libftdi', result.output)
         self.assertTrue(stampprefix, 'Unable to get STAMP value for recipe libftdi')
         matches = glob.glob(stampprefix + '*')
@@ -323,34 +329,34 @@ class DevtoolTests(DevtoolBase):
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
         self.add_command_to_tearDown('bitbake -c cleansstate %s' % testrecipe)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool add %s %s -f %s' % (testrecipe, srcdir, url))
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool add %s %s -f %s' % (testrecipe, srcdir, url))
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created. %s' % result.output)
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'setup.py')), 'Unable to find setup.py in source directory')
         self.assertTrue(os.path.isdir(os.path.join(srcdir, '.git')), 'git repository for external source tree was not created')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(srcdir, result.output)
         # Check recipe
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         self.assertIn('%s_%s.bb' % (testrecipe, testver), recipefile, 'Recipe file incorrectly named')
         checkvars = {}
         checkvars['S'] = '${WORKDIR}/MarkupSafe-${PV}'
         checkvars['SRC_URI'] = url.replace(testver, '${PV}')
         self._test_recipe_contents(recipefile, checkvars, [])
         # Try with version specified
-        result = runCmd('devtool reset -n %s' % testrecipe)
+        result = self.runCmd('devtool reset -n %s' % testrecipe)
         shutil.rmtree(srcdir)
         fakever = '1.9'
-        result = runCmd('devtool add %s %s -f %s -V %s' % (testrecipe, srcdir, url, fakever))
+        result = self.runCmd('devtool add %s %s -f %s -V %s' % (testrecipe, srcdir, url, fakever))
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'setup.py')), 'Unable to find setup.py in source directory')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(srcdir, result.output)
         # Check recipe
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         self.assertIn('%s_%s.bb' % (testrecipe, fakever), recipefile, 'Recipe file incorrectly named')
         checkvars = {}
         checkvars['S'] = '${WORKDIR}/MarkupSafe-%s' % testver
@@ -368,16 +374,16 @@ class DevtoolTests(DevtoolBase):
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
         self.add_command_to_tearDown('bitbake -c cleansstate %s' % testrecipe)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool add %s %s -a -f %s' % (testrecipe, srcdir, url))
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool add %s %s -a -f %s' % (testrecipe, srcdir, url))
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created: %s' % result.output)
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'imraa', 'imraa.c')), 'Unable to find imraa/imraa.c in source directory')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(srcdir, result.output)
         # Check recipe
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         self.assertIn('_git.bb', recipefile, 'Recipe file incorrectly named')
         checkvars = {}
         checkvars['S'] = '${WORKDIR}/git'
@@ -386,17 +392,17 @@ class DevtoolTests(DevtoolBase):
         checkvars['SRCREV'] = '${AUTOREV}'
         self._test_recipe_contents(recipefile, checkvars, [])
         # Try with revision and version specified
-        result = runCmd('devtool reset -n %s' % testrecipe)
+        result = self.runCmd('devtool reset -n %s' % testrecipe)
         shutil.rmtree(srcdir)
         url_rev = '%s;rev=%s' % (url, checkrev)
-        result = runCmd('devtool add %s %s -f "%s" -V 1.5' % (testrecipe, srcdir, url_rev))
+        result = self.runCmd('devtool add %s %s -f "%s" -V 1.5' % (testrecipe, srcdir, url_rev))
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'imraa', 'imraa.c')), 'Unable to find imraa/imraa.c in source directory')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(srcdir, result.output)
         # Check recipe
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         self.assertIn('_git.bb', recipefile, 'Recipe file incorrectly named')
         checkvars = {}
         checkvars['S'] = '${WORKDIR}/git'
@@ -416,17 +422,17 @@ class DevtoolTests(DevtoolBase):
         srcdir = os.path.join(self.workspacedir, 'sources', testrecipe)
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool add %s' % url)
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool add %s' % url)
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created. %s' % result.output)
         self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure')), 'Unable to find configure script in source directory')
         self.assertTrue(os.path.isdir(os.path.join(srcdir, '.git')), 'git repository for external source tree was not created')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(srcdir, result.output)
         # Check recipe
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         self.assertIn('%s_%s.bb' % (testrecipe, testver), recipefile, 'Recipe file incorrectly named')
         checkvars = {}
         checkvars['S'] = None
@@ -440,21 +446,21 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean mdadm')
-        result = runCmd('devtool modify mdadm -x %s' % tempdir)
+        result = self.runCmd('devtool modify mdadm -x %s' % tempdir)
         self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
         matches = glob.glob(os.path.join(self.workspacedir, 'appends', 'mdadm_*.bbappend'))
         self.assertTrue(matches, 'bbappend not created %s' % result.output)
 
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn('mdadm', result.output)
         self.assertIn(tempdir, result.output)
         self._check_src_repo(tempdir)
 
-        bitbake('mdadm -C unpack')
+        self.bitbake('mdadm -C unpack')
 
         def check_line(checkfile, expected, message, present=True):
             # Check for $expected, on a line on its own, in checkfile.
@@ -465,7 +471,7 @@ class DevtoolTests(DevtoolBase):
                     self.assertNotIn(expected + '\n', f, message)
 
         modfile = os.path.join(tempdir, 'mdadm.8.in')
-        bb_vars = get_bb_vars(['PKGD', 'mandir'], 'mdadm')
+        bb_vars = self.get_bb_vars(['PKGD', 'mandir'], 'mdadm')
         pkgd = bb_vars['PKGD']
         self.assertTrue(pkgd, 'Could not query PKGD variable')
         mandir = bb_vars['mandir']
@@ -475,20 +481,20 @@ class DevtoolTests(DevtoolBase):
         check_line(modfile, 'Linux Software RAID', 'Could not find initial string')
         check_line(modfile, 'antique pin sardine', 'Unexpectedly found replacement string', present=False)
 
-        result = runCmd("sed -i 's!^Linux Software RAID$!antique pin sardine!' %s" % modfile)
+        result = self.runCmd("sed -i 's!^Linux Software RAID$!antique pin sardine!' %s" % modfile)
         check_line(modfile, 'antique pin sardine', 'mdadm.8.in file not modified (sed failed)')
 
-        bitbake('mdadm -c package')
+        self.bitbake('mdadm -c package')
         check_line(manfile, 'antique pin sardine', 'man file not modified. man searched file path: %s' % manfile)
 
-        result = runCmd('git checkout -- %s' % modfile, cwd=tempdir)
+        result = self.runCmd('git checkout -- %s' % modfile, cwd=tempdir)
         check_line(modfile, 'Linux Software RAID', 'man .in file not restored (git failed)')
 
-        bitbake('mdadm -c package')
+        self.bitbake('mdadm -c package')
         check_line(manfile, 'Linux Software RAID', 'man file not updated. man searched file path: %s' % manfile)
 
-        result = runCmd('devtool reset mdadm')
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool reset mdadm')
+        result = self.runCmd('devtool status')
         self.assertNotIn('mdadm', result.output)
 
     @OETestID(1620)
@@ -501,7 +507,7 @@ class DevtoolTests(DevtoolBase):
             self.assertNotExists(f)
 
         # Clean up anything in the workdir/sysroot/sstate cache
-        bitbake('mdadm m4 -c cleansstate')
+        self.bitbake('mdadm m4 -c cleansstate')
         # Try modifying a recipe
         tempdir_mdadm = tempfile.mkdtemp(prefix='devtoolqa')
         tempdir_m4 = tempfile.mkdtemp(prefix='devtoolqa')
@@ -510,27 +516,27 @@ class DevtoolTests(DevtoolBase):
         self.track_for_cleanup(tempdir_m4)
         self.track_for_cleanup(builddir_m4)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean mdadm m4')
         self.write_recipeinc('m4', 'EXTERNALSRC_BUILD = "%s"\ndo_clean() {\n\t:\n}\n' % builddir_m4)
         try:
-            runCmd('devtool modify mdadm -x %s' % tempdir_mdadm)
-            runCmd('devtool modify m4 -x %s' % tempdir_m4)
+            self.runCmd('devtool modify mdadm -x %s' % tempdir_mdadm)
+            self.runCmd('devtool modify m4 -x %s' % tempdir_m4)
             assertNoFile(tempdir_mdadm, 'mdadm')
             assertNoFile(builddir_m4, 'src/m4')
-            result = bitbake('m4 -e')
-            result = bitbake('mdadm m4 -c compile')
+            result = self.bitbake('m4 -e')
+            result = self.bitbake('mdadm m4 -c compile')
             self.assertEqual(result.status, 0)
             assertFile(tempdir_mdadm, 'mdadm')
             assertFile(builddir_m4, 'src/m4')
             # Check that buildclean task exists and does call make clean
-            bitbake('mdadm m4 -c buildclean')
+            self.bitbake('mdadm m4 -c buildclean')
             assertNoFile(tempdir_mdadm, 'mdadm')
             assertNoFile(builddir_m4, 'src/m4')
-            bitbake('mdadm m4 -c compile')
+            self.bitbake('mdadm m4 -c compile')
             assertFile(tempdir_mdadm, 'mdadm')
             assertFile(builddir_m4, 'src/m4')
-            bitbake('mdadm m4 -c clean')
+            self.bitbake('mdadm m4 -c clean')
             # Check that buildclean task is run before clean for B == S
             assertNoFile(tempdir_mdadm, 'mdadm')
             # Check that buildclean task is not run before clean for B != S
@@ -544,11 +550,11 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
 
         testrecipes = 'perf kernel-devsrc package-index core-image-minimal meta-toolchain packagegroup-core-sdk meta-ide-support'.split()
         # Find actual name of gcc-source since it now includes the version - crude, but good enough for this purpose
-        result = runCmd('bitbake-layers show-recipes gcc-source*')
+        result = self.runCmd('bitbake-layers show-recipes gcc-source*')
         for line in result.output.splitlines():
             # just match those lines that contain a real target
             m = re.match('(?P<recipe>^[a-zA-Z0-9.-]+)(?P<colon>:$)', line)
@@ -556,14 +562,14 @@ class DevtoolTests(DevtoolBase):
                 testrecipes.append(m.group('recipe'))
         for testrecipe in testrecipes:
             # Check it's a valid recipe
-            bitbake('%s -e' % testrecipe)
+            self.bitbake('%s -e' % testrecipe)
             # devtool extract should fail
-            result = runCmd('devtool extract %s %s' % (testrecipe, os.path.join(tempdir, testrecipe)), ignore_status=True)
+            result = self.runCmd('devtool extract %s %s' % (testrecipe, os.path.join(tempdir, testrecipe)), ignore_status=True)
             self.assertNotEqual(result.status, 0, 'devtool extract on %s should have failed. devtool output: %s' % (testrecipe, result.output))
             self.assertNotIn('Fetching ', result.output, 'devtool extract on %s should have errored out before trying to fetch' % testrecipe)
             self.assertIn('ERROR: ', result.output, 'devtool extract on %s should have given an ERROR' % testrecipe)
             # devtool modify should fail
-            result = runCmd('devtool modify %s -x %s' % (testrecipe, os.path.join(tempdir, testrecipe)), ignore_status=True)
+            result = self.runCmd('devtool modify %s -x %s' % (testrecipe, os.path.join(tempdir, testrecipe)), ignore_status=True)
             self.assertNotEqual(result.status, 0, 'devtool modify on %s should have failed. devtool output: %s' %  (testrecipe, result.output))
             self.assertIn('ERROR: ', result.output, 'devtool modify on %s should have given an ERROR' % testrecipe)
 
@@ -575,22 +581,22 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
 
         bbclassextended = False
         inheritnative = False
         testrecipes = 'mtools-native apt-native desktop-file-utils-native'.split()
         for testrecipe in testrecipes:
-            checkextend = 'native' in (get_bb_var('BBCLASSEXTEND', testrecipe) or '').split()
+            checkextend = 'native' in (self.get_bb_var('BBCLASSEXTEND', testrecipe) or '').split()
             if not bbclassextended:
                 bbclassextended = checkextend
             if not inheritnative:
                 inheritnative = not checkextend
-            result = runCmd('devtool modify %s -x %s' % (testrecipe, os.path.join(tempdir, testrecipe)))
+            result = self.runCmd('devtool modify %s -x %s' % (testrecipe, os.path.join(tempdir, testrecipe)))
             self.assertNotIn('ERROR: ', result.output, 'ERROR in devtool modify output: %s' % result.output)
-            result = runCmd('devtool build %s' % testrecipe)
+            result = self.runCmd('devtool build %s' % testrecipe)
             self.assertNotIn('ERROR: ', result.output, 'ERROR in devtool build output: %s' % result.output)
-            result = runCmd('devtool reset %s' % testrecipe)
+            result = self.runCmd('devtool reset %s' % testrecipe)
             self.assertNotIn('ERROR: ', result.output, 'ERROR in devtool reset output: %s' % result.output)
 
         self.assertTrue(bbclassextended, 'None of these recipes are BBCLASSEXTENDed to native - need to adjust testrecipes list: %s' % ', '.join(testrecipes))
@@ -601,35 +607,35 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_modify_git(self):
         # Check preconditions
         testrecipe = 'mkelfimage'
-        src_uri = get_bb_var('SRC_URI', testrecipe)
+        src_uri = self.get_bb_var('SRC_URI', testrecipe)
         self.assertIn('git://', src_uri, 'This test expects the %s recipe to be a git recipe' % testrecipe)
         # Clean up anything in the workdir/sysroot/sstate cache
-        bitbake('%s -c cleansstate' % testrecipe)
+        self.bitbake('%s -c cleansstate' % testrecipe)
         # Try modifying a recipe
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created. devtool output: %s' % result.output)
         matches = glob.glob(os.path.join(self.workspacedir, 'appends', 'mkelfimage_*.bbappend'))
         self.assertTrue(matches, 'bbappend not created')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(tempdir, result.output)
         # Check git repo
         self._check_src_repo(tempdir)
         # Try building
-        bitbake(testrecipe)
+        self.bitbake(testrecipe)
 
     @OETestID(1167)
     def test_devtool_modify_localfiles(self):
         # Check preconditions
         testrecipe = 'lighttpd'
-        src_uri = (get_bb_var('SRC_URI', testrecipe) or '').split()
+        src_uri = (self.get_bb_var('SRC_URI', testrecipe) or '').split()
         foundlocal = False
         for item in src_uri:
             if item.startswith('file://') and '.patch' not in item:
@@ -637,24 +643,24 @@ class DevtoolTests(DevtoolBase):
                 break
         self.assertTrue(foundlocal, 'This test expects the %s recipe to fetch local files and it seems that it no longer does' % testrecipe)
         # Clean up anything in the workdir/sysroot/sstate cache
-        bitbake('%s -c cleansstate' % testrecipe)
+        self.bitbake('%s -c cleansstate' % testrecipe)
         # Try modifying a recipe
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         self.assertExists(os.path.join(tempdir, 'configure.ac'), 'Extracted source could not be found')
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
         matches = glob.glob(os.path.join(self.workspacedir, 'appends', '%s_*.bbappend' % testrecipe))
         self.assertTrue(matches, 'bbappend not created')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(testrecipe, result.output)
         self.assertIn(tempdir, result.output)
         # Try building
-        bitbake(testrecipe)
+        self.bitbake(testrecipe)
 
     @OETestID(1378)
     def test_devtool_modify_virtual(self):
@@ -664,14 +670,14 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool modify %s -x %s' % (virtrecipe, tempdir))
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool modify %s -x %s' % (virtrecipe, tempdir))
         self.assertExists(os.path.join(tempdir, 'Makefile.am'), 'Extracted source could not be found')
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
         matches = glob.glob(os.path.join(self.workspacedir, 'appends', '%s_*.bbappend' % realrecipe))
         self.assertTrue(matches, 'bbappend not created %s' % result.output)
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertNotIn(virtrecipe, result.output)
         self.assertIn(realrecipe, result.output)
         # Check git repo
@@ -683,7 +689,7 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe(self):
         # Check preconditions
         testrecipe = 'minicom'
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         self.assertNotIn('git://', src_uri, 'This test expects the %s recipe to NOT be a git recipe' % testrecipe)
@@ -692,21 +698,21 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
         # We don't use -x here so that we test the behaviour of devtool modify without it
-        result = runCmd('devtool modify %s %s' % (testrecipe, tempdir))
+        result = self.runCmd('devtool modify %s %s' % (testrecipe, tempdir))
         # Check git repo
         self._check_src_repo(tempdir)
         # Add a couple of commits
         # FIXME: this only tests adding, need to also test update and remove
-        result = runCmd('echo "Additional line" >> README', cwd=tempdir)
-        result = runCmd('git commit -a -m "Change the README"', cwd=tempdir)
-        result = runCmd('echo "A new file" > devtool-new-file', cwd=tempdir)
-        result = runCmd('git add devtool-new-file', cwd=tempdir)
-        result = runCmd('git commit -m "Add a new file"', cwd=tempdir)
+        result = self.runCmd('echo "Additional line" >> README', cwd=tempdir)
+        result = self.runCmd('git commit -a -m "Change the README"', cwd=tempdir)
+        result = self.runCmd('echo "A new file" > devtool-new-file', cwd=tempdir)
+        result = self.runCmd('git add devtool-new-file', cwd=tempdir)
+        result = self.runCmd('git commit -m "Add a new file"', cwd=tempdir)
         self.add_command_to_tearDown('cd %s; rm %s/*.patch; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = runCmd('devtool update-recipe %s' % testrecipe)
+        result = self.runCmd('devtool update-recipe %s' % testrecipe)
         expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
                            ('??', '.*/0001-Change-the-README.patch$'),
                            ('??', '.*/0002-Add-a-new-file.patch$')]
@@ -716,7 +722,7 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_git(self):
         # Check preconditions
         testrecipe = 'mtd-utils'
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         self.assertIn('git://', src_uri, 'This test expects the %s recipe to be a git recipe' % testrecipe)
@@ -730,25 +736,25 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         # Check git repo
         self._check_src_repo(tempdir)
         # Add a couple of commits
         # FIXME: this only tests adding, need to also test update and remove
-        result = runCmd('echo "# Additional line" >> Makefile.am', cwd=tempdir)
-        result = runCmd('git commit -a -m "Change the Makefile"', cwd=tempdir)
-        result = runCmd('echo "A new file" > devtool-new-file', cwd=tempdir)
-        result = runCmd('git add devtool-new-file', cwd=tempdir)
-        result = runCmd('git commit -m "Add a new file"', cwd=tempdir)
+        result = self.runCmd('echo "# Additional line" >> Makefile.am', cwd=tempdir)
+        result = self.runCmd('git commit -a -m "Change the Makefile"', cwd=tempdir)
+        result = self.runCmd('echo "A new file" > devtool-new-file', cwd=tempdir)
+        result = self.runCmd('git add devtool-new-file', cwd=tempdir)
+        result = self.runCmd('git commit -m "Add a new file"', cwd=tempdir)
         self.add_command_to_tearDown('cd %s; rm -rf %s; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = runCmd('devtool update-recipe -m srcrev %s' % testrecipe)
+        result = self.runCmd('devtool update-recipe -m srcrev %s' % testrecipe)
         expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile))] + \
                           [(' D', '.*/%s$' % patch) for patch in patches]
         self._check_repo_status(os.path.dirname(recipefile), expected_status)
 
-        result = runCmd('git diff %s' % os.path.basename(recipefile), cwd=os.path.dirname(recipefile))
+        result = self.runCmd('git diff %s' % os.path.basename(recipefile), cwd=os.path.dirname(recipefile))
         addlines = ['SRCREV = ".*"', 'SRC_URI = "git://git.infradead.org/mtd-utils.git"']
         srcurilines = src_uri.split()
         srcurilines[0] = 'SRC_URI = "' + srcurilines[0]
@@ -772,9 +778,9 @@ class DevtoolTests(DevtoolBase):
                         break
                 self.assertTrue(matched, 'Unexpected diff remove line: %s' % line)
         # Now try with auto mode
-        runCmd('cd %s; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, os.path.basename(recipefile)))
-        result = runCmd('devtool update-recipe %s' % testrecipe)
-        result = runCmd('git rev-parse --show-toplevel', cwd=os.path.dirname(recipefile))
+        self.runCmd('cd %s; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, os.path.basename(recipefile)))
+        result = self.runCmd('devtool update-recipe %s' % testrecipe)
+        result = self.runCmd('git rev-parse --show-toplevel', cwd=os.path.dirname(recipefile))
         topleveldir = result.output.strip()
         relpatchpath = os.path.join(os.path.relpath(os.path.dirname(recipefile), topleveldir), testrecipe)
         expected_status = [(' M', os.path.relpath(recipefile, topleveldir)),
@@ -786,7 +792,7 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_append(self):
         # Check preconditions
         testrecipe = 'mdadm'
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         self.assertNotIn('git://', src_uri, 'This test expects the %s recipe to NOT be a git recipe' % testrecipe)
@@ -797,19 +803,19 @@ class DevtoolTests(DevtoolBase):
         templayerdir = os.path.join(tempdir, 'layer')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempsrcdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempsrcdir))
         # Check git repo
         self._check_src_repo(tempsrcdir)
         # Add a commit
-        result = runCmd("sed 's!\\(#define VERSION\\W*\"[^\"]*\\)\"!\\1-custom\"!' -i ReadMe.c", cwd=tempsrcdir)
-        result = runCmd('git commit -a -m "Add our custom version"', cwd=tempsrcdir)
+        result = self.runCmd("sed 's!\\(#define VERSION\\W*\"[^\"]*\\)\"!\\1-custom\"!' -i ReadMe.c", cwd=tempsrcdir)
+        result = self.runCmd('git commit -a -m "Add our custom version"', cwd=tempsrcdir)
         self.add_command_to_tearDown('cd %s; rm -f %s/*.patch; git checkout .' % (os.path.dirname(recipefile), testrecipe))
         # Create a temporary layer and add it to bblayers.conf
         self._create_temp_layer(templayerdir, True, 'selftestupdaterecipe')
         # Create the bbappend
-        result = runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
         self.assertNotIn('WARNING:', result.output)
         # Check recipe is still clean
         self._check_repo_status(os.path.dirname(recipefile), [])
@@ -829,12 +835,12 @@ class DevtoolTests(DevtoolBase):
             self.assertEqual(expectedlines, f.readlines())
 
         # Check we can run it again and bbappend isn't modified
-        result = runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
         with open(bbappendfile, 'r') as f:
             self.assertEqual(expectedlines, f.readlines())
         # Drop new commit and check patch gets deleted
-        result = runCmd('git reset HEAD^', cwd=tempsrcdir)
-        result = runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('git reset HEAD^', cwd=tempsrcdir)
+        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
         self.assertNotExists(patchfile, 'Patch file not deleted')
         expectedlines2 = ['FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n',
                          '\n']
@@ -842,9 +848,9 @@ class DevtoolTests(DevtoolBase):
             self.assertEqual(expectedlines2, f.readlines())
         # Put commit back and check we can run it if layer isn't in bblayers.conf
         os.remove(bbappendfile)
-        result = runCmd('git commit -a -m "Add our custom version"', cwd=tempsrcdir)
-        result = runCmd('bitbake-layers remove-layer %s' % templayerdir, cwd=self.builddir)
-        result = runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('git commit -a -m "Add our custom version"', cwd=tempsrcdir)
+        result = self.runCmd('bitbake-layers remove-layer %s' % templayerdir, cwd=self.builddir)
+        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
         self.assertIn('WARNING: Specified layer is not currently enabled in bblayers.conf', result.output)
         self.assertExists(patchfile, 'Patch file not created (with disabled layer)')
         with open(bbappendfile, 'r') as f:
@@ -855,7 +861,7 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_append_git(self):
         # Check preconditions
         testrecipe = 'mtd-utils'
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         self.assertIn('git://', src_uri, 'This test expects the %s recipe to be a git recipe' % testrecipe)
@@ -870,14 +876,14 @@ class DevtoolTests(DevtoolBase):
         templayerdir = os.path.join(tempdir, 'layer')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempsrcdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempsrcdir))
         # Check git repo
         self._check_src_repo(tempsrcdir)
         # Add a commit
-        result = runCmd('echo "# Additional line" >> Makefile.am', cwd=tempsrcdir)
-        result = runCmd('git commit -a -m "Change the Makefile"', cwd=tempsrcdir)
+        result = self.runCmd('echo "# Additional line" >> Makefile.am', cwd=tempsrcdir)
+        result = self.runCmd('git commit -a -m "Change the Makefile"', cwd=tempsrcdir)
         self.add_command_to_tearDown('cd %s; rm -f %s/*.patch; git checkout .' % (os.path.dirname(recipefile), testrecipe))
         # Create a temporary layer
         os.makedirs(os.path.join(templayerdir, 'conf'))
@@ -889,9 +895,9 @@ class DevtoolTests(DevtoolBase):
             f.write('BBFILE_PRIORITY_oeselftesttemplayer = "999"\n')
             f.write('BBFILE_PATTERN_IGNORE_EMPTY_oeselftesttemplayer = "1"\n')
         self.add_command_to_tearDown('bitbake-layers remove-layer %s || true' % templayerdir)
-        result = runCmd('bitbake-layers add-layer %s' % templayerdir, cwd=self.builddir)
+        result = self.runCmd('bitbake-layers add-layer %s' % templayerdir, cwd=self.builddir)
         # Create the bbappend
-        result = runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
         self.assertNotIn('WARNING:', result.output)
         # Check recipe is still clean
         self._check_repo_status(os.path.dirname(recipefile), [])
@@ -902,7 +908,7 @@ class DevtoolTests(DevtoolBase):
         self.assertNotExists(os.path.join(appenddir, testrecipe), 'Patch directory should not be created')
 
         # Check bbappend contents
-        result = runCmd('git rev-parse HEAD', cwd=tempsrcdir)
+        result = self.runCmd('git rev-parse HEAD', cwd=tempsrcdir)
         expectedlines = set(['SRCREV = "%s"\n' % result.output,
                              '\n',
                              'SRC_URI = "%s"\n' % git_uri,
@@ -911,14 +917,14 @@ class DevtoolTests(DevtoolBase):
             self.assertEqual(expectedlines, set(f.readlines()))
 
         # Check we can run it again and bbappend isn't modified
-        result = runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
         with open(bbappendfile, 'r') as f:
             self.assertEqual(expectedlines, set(f.readlines()))
         # Drop new commit and check SRCREV changes
-        result = runCmd('git reset HEAD^', cwd=tempsrcdir)
-        result = runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('git reset HEAD^', cwd=tempsrcdir)
+        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
         self.assertNotExists(os.path.join(appenddir, testrecipe), 'Patch directory should not be created')
-        result = runCmd('git rev-parse HEAD', cwd=tempsrcdir)
+        result = self.runCmd('git rev-parse HEAD', cwd=tempsrcdir)
         expectedlines = set(['SRCREV = "%s"\n' % result.output,
                              '\n',
                              'SRC_URI = "%s"\n' % git_uri,
@@ -927,12 +933,12 @@ class DevtoolTests(DevtoolBase):
             self.assertEqual(expectedlines, set(f.readlines()))
         # Put commit back and check we can run it if layer isn't in bblayers.conf
         os.remove(bbappendfile)
-        result = runCmd('git commit -a -m "Change the Makefile"', cwd=tempsrcdir)
-        result = runCmd('bitbake-layers remove-layer %s' % templayerdir, cwd=self.builddir)
-        result = runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
+        result = self.runCmd('git commit -a -m "Change the Makefile"', cwd=tempsrcdir)
+        result = self.runCmd('bitbake-layers remove-layer %s' % templayerdir, cwd=self.builddir)
+        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
         self.assertIn('WARNING: Specified layer is not currently enabled in bblayers.conf', result.output)
         self.assertNotExists(os.path.join(appenddir, testrecipe), 'Patch directory should not be created')
-        result = runCmd('git rev-parse HEAD', cwd=tempsrcdir)
+        result = self.runCmd('git rev-parse HEAD', cwd=tempsrcdir)
         expectedlines = set(['SRCREV = "%s"\n' % result.output,
                              '\n',
                              'SRC_URI = "%s"\n' % git_uri,
@@ -945,28 +951,28 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_local_files(self):
         """Check that local source files are copied over instead of patched"""
         testrecipe = 'makedevs'
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         # Setup srctree for modifying the recipe
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be
         # building it)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         # Check git repo
         self._check_src_repo(tempdir)
         # Try building just to ensure we haven't broken that
-        bitbake("%s" % testrecipe)
+        self.bitbake("%s" % testrecipe)
         # Edit / commit local source
-        runCmd('echo "/* Foobar */" >> oe-local-files/makedevs.c', cwd=tempdir)
-        runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
-        runCmd('echo "Bar" > new-file', cwd=tempdir)
-        runCmd('git add new-file', cwd=tempdir)
-        runCmd('git commit -m "Add new file"', cwd=tempdir)
+        self.runCmd('echo "/* Foobar */" >> oe-local-files/makedevs.c', cwd=tempdir)
+        self.runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
+        self.runCmd('echo "Bar" > new-file', cwd=tempdir)
+        self.runCmd('git add new-file', cwd=tempdir)
+        self.runCmd('git commit -m "Add new file"', cwd=tempdir)
         self.add_command_to_tearDown('cd %s; git clean -fd .; git checkout .' %
                                      os.path.dirname(recipefile))
-        runCmd('devtool update-recipe %s' % testrecipe)
+        self.runCmd('devtool update-recipe %s' % testrecipe)
         expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
                            (' M', '.*/makedevs/makedevs.c$'),
                            ('??', '.*/makedevs/new-local$'),
@@ -977,36 +983,36 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_local_files_2(self):
         """Check local source files support when oe-local-files is in Git"""
         testrecipe = 'lzo'
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         # Setup srctree for modifying the recipe
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         # Check git repo
         self._check_src_repo(tempdir)
         # Add oe-local-files to Git
-        runCmd('rm oe-local-files/.gitignore', cwd=tempdir)
-        runCmd('git add oe-local-files', cwd=tempdir)
-        runCmd('git commit -m "Add local sources"', cwd=tempdir)
+        self.runCmd('rm oe-local-files/.gitignore', cwd=tempdir)
+        self.runCmd('git add oe-local-files', cwd=tempdir)
+        self.runCmd('git commit -m "Add local sources"', cwd=tempdir)
         # Edit / commit local sources
-        runCmd('echo "# Foobar" >> oe-local-files/acinclude.m4', cwd=tempdir)
-        runCmd('git commit -am "Edit existing file"', cwd=tempdir)
-        runCmd('git rm oe-local-files/run-ptest', cwd=tempdir)
-        runCmd('git commit -m"Remove file"', cwd=tempdir)
-        runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
-        runCmd('git add oe-local-files/new-local', cwd=tempdir)
-        runCmd('git commit -m "Add new local file"', cwd=tempdir)
-        runCmd('echo "Gar" > new-file', cwd=tempdir)
-        runCmd('git add new-file', cwd=tempdir)
-        runCmd('git commit -m "Add new file"', cwd=tempdir)
+        self.runCmd('echo "# Foobar" >> oe-local-files/acinclude.m4', cwd=tempdir)
+        self.runCmd('git commit -am "Edit existing file"', cwd=tempdir)
+        self.runCmd('git rm oe-local-files/run-ptest', cwd=tempdir)
+        self.runCmd('git commit -m"Remove file"', cwd=tempdir)
+        self.runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
+        self.runCmd('git add oe-local-files/new-local', cwd=tempdir)
+        self.runCmd('git commit -m "Add new local file"', cwd=tempdir)
+        self.runCmd('echo "Gar" > new-file', cwd=tempdir)
+        self.runCmd('git add new-file', cwd=tempdir)
+        self.runCmd('git commit -m "Add new file"', cwd=tempdir)
         self.add_command_to_tearDown('cd %s; git clean -fd .; git checkout .' %
                                      os.path.dirname(recipefile))
         # Checkout unmodified file to working copy -> devtool should still pick
         # the modified version from HEAD
-        runCmd('git checkout HEAD^ -- oe-local-files/acinclude.m4', cwd=tempdir)
-        runCmd('devtool update-recipe %s' % testrecipe)
+        self.runCmd('git checkout HEAD^ -- oe-local-files/acinclude.m4', cwd=tempdir)
+        self.runCmd('devtool update-recipe %s' % testrecipe)
         expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
                            (' M', '.*/acinclude.m4$'),
                            (' D', '.*/run-ptest$'),
@@ -1018,19 +1024,19 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_local_files_3(self):
         # First, modify the recipe
         testrecipe = 'devtool-test-localonly'
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = runCmd('devtool modify %s' % testrecipe)
+        result = self.runCmd('devtool modify %s' % testrecipe)
         # Modify one file
-        runCmd('echo "Another line" >> file2', cwd=os.path.join(self.workspacedir, 'sources', testrecipe, 'oe-local-files'))
+        self.runCmd('echo "Another line" >> file2', cwd=os.path.join(self.workspacedir, 'sources', testrecipe, 'oe-local-files'))
         self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = runCmd('devtool update-recipe %s' % testrecipe)
+        result = self.runCmd('devtool update-recipe %s' % testrecipe)
         expected_status = [(' M', '.*/%s/file2$' % testrecipe)]
         self._check_repo_status(os.path.dirname(recipefile), expected_status)
 
@@ -1038,27 +1044,27 @@ class DevtoolTests(DevtoolBase):
     def test_devtool_update_recipe_local_patch_gz(self):
         # First, modify the recipe
         testrecipe = 'devtool-test-patch-gz'
-        if get_bb_var('DISTRO') == 'poky-tiny':
+        if self.get_bb_var('DISTRO') == 'poky-tiny':
             self.skipTest("The DISTRO 'poky-tiny' does not provide the dependencies needed by %s" % testrecipe)
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = runCmd('devtool modify %s' % testrecipe)
+        result = self.runCmd('devtool modify %s' % testrecipe)
         # Modify one file
         srctree = os.path.join(self.workspacedir, 'sources', testrecipe)
-        runCmd('echo "Another line" >> README', cwd=srctree)
-        runCmd('git commit -a --amend --no-edit', cwd=srctree)
+        self.runCmd('echo "Another line" >> README', cwd=srctree)
+        self.runCmd('git commit -a --amend --no-edit', cwd=srctree)
         self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = runCmd('devtool update-recipe %s' % testrecipe)
+        result = self.runCmd('devtool update-recipe %s' % testrecipe)
         expected_status = [(' M', '.*/%s/readme.patch.gz$' % testrecipe)]
         self._check_repo_status(os.path.dirname(recipefile), expected_status)
         patch_gz = os.path.join(os.path.dirname(recipefile), testrecipe, 'readme.patch.gz')
-        result = runCmd('file %s' % patch_gz)
+        result = self.runCmd('file %s' % patch_gz)
         if 'gzip compressed data' not in result.output:
             self.fail('New patch file is not gzipped - file reports:\n%s' % result.output)
 
@@ -1069,15 +1075,15 @@ class DevtoolTests(DevtoolBase):
         # was also in SRC_URI
         # First, modify the recipe
         testrecipe = 'devtool-test-subdir'
-        bb_vars = get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
+        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
         recipefile = bb_vars['FILE']
         src_uri = bb_vars['SRC_URI']
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = runCmd('devtool modify %s' % testrecipe)
+        result = self.runCmd('devtool modify %s' % testrecipe)
         testfile = os.path.join(self.workspacedir, 'sources', testrecipe, 'testfile')
         self.assertExists(testfile, 'Extracted source could not be found')
         with open(testfile, 'r') as f:
@@ -1085,7 +1091,7 @@ class DevtoolTests(DevtoolBase):
         self.assertEqual(contents, 'Modified version', 'File has apparently not been overwritten as it should have been')
         # Test devtool update-recipe without modifying any files
         self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = runCmd('devtool update-recipe %s' % testrecipe)
+        result = self.runCmd('devtool update-recipe %s' % testrecipe)
         expected_status = []
         self._check_repo_status(os.path.dirname(recipefile), expected_status)
 
@@ -1094,7 +1100,7 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         # Try devtool extract
         self.track_for_cleanup(tempdir)
-        result = runCmd('devtool extract matchbox-terminal %s' % tempdir)
+        result = self.runCmd('devtool extract matchbox-terminal %s' % tempdir)
         self.assertExists(os.path.join(tempdir, 'Makefile.am'), 'Extracted source could not be found')
         # devtool extract shouldn't create the workspace
         self.assertNotExists(self.workspacedir)
@@ -1105,7 +1111,7 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         # Try devtool extract
         self.track_for_cleanup(tempdir)
-        result = runCmd('devtool extract virtual/make %s' % tempdir)
+        result = self.runCmd('devtool extract virtual/make %s' % tempdir)
         self.assertExists(os.path.join(tempdir, 'Makefile.am'), 'Extracted source could not be found')
         # devtool extract shouldn't create the workspace
         self.assertNotExists(self.workspacedir)
@@ -1116,21 +1122,21 @@ class DevtoolTests(DevtoolBase):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         testrecipe1 = 'mdadm'
         testrecipe2 = 'cronie'
-        result = runCmd('devtool modify -x %s %s' % (testrecipe1, os.path.join(tempdir, testrecipe1)))
-        result = runCmd('devtool modify -x %s %s' % (testrecipe2, os.path.join(tempdir, testrecipe2)))
-        result = runCmd('devtool build %s' % testrecipe1)
-        result = runCmd('devtool build %s' % testrecipe2)
-        stampprefix1 = get_bb_var('STAMP', testrecipe1)
+        result = self.runCmd('devtool modify -x %s %s' % (testrecipe1, os.path.join(tempdir, testrecipe1)))
+        result = self.runCmd('devtool modify -x %s %s' % (testrecipe2, os.path.join(tempdir, testrecipe2)))
+        result = self.runCmd('devtool build %s' % testrecipe1)
+        result = self.runCmd('devtool build %s' % testrecipe2)
+        stampprefix1 = self.get_bb_var('STAMP', testrecipe1)
         self.assertTrue(stampprefix1, 'Unable to get STAMP value for recipe %s' % testrecipe1)
-        stampprefix2 = get_bb_var('STAMP', testrecipe2)
+        stampprefix2 = self.get_bb_var('STAMP', testrecipe2)
         self.assertTrue(stampprefix2, 'Unable to get STAMP value for recipe %s' % testrecipe2)
-        result = runCmd('devtool reset -a')
+        result = self.runCmd('devtool reset -a')
         self.assertIn(testrecipe1, result.output)
         self.assertIn(testrecipe2, result.output)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertNotIn(testrecipe1, result.output)
         self.assertNotIn(testrecipe2, result.output)
         matches1 = glob.glob(stampprefix1 + '*')
@@ -1147,14 +1153,14 @@ class DevtoolTests(DevtoolBase):
         # really this has to be done as an oe-selftest test.
         #
         # Check preconditions
-        machine = get_bb_var('MACHINE')
+        machine = self.get_bb_var('MACHINE')
         if not machine.startswith('qemu'):
             self.skipTest('This test only works with qemu machines')
         if not os.path.exists('/etc/runqemu-nosudo'):
             self.skipTest('You must set up tap devices with scripts/runqemu-gen-tapdevs before running this test')
-        result = runCmd('PATH="$PATH:/sbin:/usr/sbin" ip tuntap show', ignore_status=True)
+        result = self.runCmd('PATH="$PATH:/sbin:/usr/sbin" ip tuntap show', ignore_status=True)
         if result.status != 0:
-            result = runCmd('PATH="$PATH:/sbin:/usr/sbin" ifconfig -a', ignore_status=True)
+            result = self.runCmd('PATH="$PATH:/sbin:/usr/sbin" ifconfig -a', ignore_status=True)
             if result.status != 0:
                 self.skipTest('Failed to determine if tap devices exist with ifconfig or ip: %s' % result.output)
         for line in result.output.splitlines():
@@ -1169,41 +1175,41 @@ class DevtoolTests(DevtoolBase):
         testimage = 'oe-selftest-image'
         testcommand = '/sbin/mdadm --help'
         # Build an image to run
-        bitbake("%s qemu-native qemu-helper-native" % testimage)
-        deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
+        self.bitbake("%s qemu-native qemu-helper-native" % testimage)
+        deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
         self.add_command_to_tearDown('bitbake -c clean %s' % testimage)
         self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, testimage))
         # Clean recipe so the first deploy will fail
-        bitbake("%s -c clean" % testrecipe)
+        self.bitbake("%s -c clean" % testrecipe)
         # Try devtool modify
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
-        result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
         # Test that deploy-target at this point fails (properly)
-        result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe, ignore_status=True)
+        result = self.runCmd('devtool deploy-target -n %s root@localhost' % testrecipe, ignore_status=True)
         self.assertNotEqual(result.output, 0, 'devtool deploy-target should have failed, output: %s' % result.output)
         self.assertNotIn(result.output, 'Traceback', 'devtool deploy-target should have failed with a proper error not a traceback, output: %s' % result.output)
-        result = runCmd('devtool build %s' % testrecipe)
+        result = self.runCmd('devtool build %s' % testrecipe)
         # First try a dry-run of deploy-target
-        result = runCmd('devtool deploy-target -n %s root@localhost' % testrecipe)
+        result = self.runCmd('devtool deploy-target -n %s root@localhost' % testrecipe)
         self.assertIn('  %s' % testfile, result.output)
         # Boot the image
         with runqemu(testimage) as qemu:
             # Now really test deploy-target
-            result = runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, qemu.ip))
+            result = self.runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, qemu.ip))
             # Run a test command to see if it was installed properly
             sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
-            result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand))
+            result = self.runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand))
             # Check if it deployed all of the files with the right ownership/perms
             # First look on the host - need to do this under pseudo to get the correct ownership/perms
-            bb_vars = get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD'], testrecipe)
+            bb_vars = self.get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD'], testrecipe)
             installdir = bb_vars['D']
             fakerootenv = bb_vars['FAKEROOTENV']
             fakerootcmd = bb_vars['FAKEROOTCMD']
-            result = runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir)
+            result = self.runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir)
             filelist1 = self._process_ls_output(result.output)
 
             # Now look on the target
@@ -1214,14 +1220,14 @@ class DevtoolTests(DevtoolBase):
                 for line in filelist1:
                     splitline = line.split()
                     f.write(splitline[-1] + '\n')
-            result = runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, qemu.ip))
+            result = self.runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, qemu.ip))
             filelist2 = self._process_ls_output(result.output)
             filelist1.sort(key=lambda item: item.split()[-1])
             filelist2.sort(key=lambda item: item.split()[-1])
             self.assertEqual(filelist1, filelist2)
             # Test undeploy-target
-            result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip))
-            result = runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand), ignore_status=True)
+            result = self.runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip))
+            result = self.runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand), ignore_status=True)
             self.assertNotEqual(result, 0, 'undeploy-target did not remove command as it should have')
 
     @OETestID(1366)
@@ -1231,22 +1237,22 @@ class DevtoolTests(DevtoolBase):
         self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
         image = 'core-image-minimal'
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean %s' % image)
-        bitbake('%s -c clean' % image)
+        self.bitbake('%s -c clean' % image)
         # Add target and native recipes to workspace
         recipes = ['mdadm', 'parted-native']
         for recipe in recipes:
             tempdir = tempfile.mkdtemp(prefix='devtoolqa')
             self.track_for_cleanup(tempdir)
             self.add_command_to_tearDown('bitbake -c clean %s' % recipe)
-            runCmd('devtool modify %s -x %s' % (recipe, tempdir))
+            self.runCmd('devtool modify %s -x %s' % (recipe, tempdir))
         # Try to build image
-        result = runCmd('devtool build-image %s' % image)
+        result = self.runCmd('devtool build-image %s' % image)
         self.assertNotEqual(result, 0, 'devtool build-image failed')
         # Check if image contains expected packages
-        deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
-        image_link_name = get_bb_var('IMAGE_LINK_NAME', image)
+        deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
+        image_link_name = self.get_bb_var('IMAGE_LINK_NAME', image)
         reqpkgs = [item for item in recipes if not item.endswith('-native')]
         with open(os.path.join(deploy_dir_image, image_link_name + '.manifest'), 'r') as f:
             for line in f:
@@ -1263,23 +1269,23 @@ class DevtoolTests(DevtoolBase):
         # Check preconditions
         self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # Check parameters
-        result = runCmd('devtool upgrade -h')
+        result = self.runCmd('devtool upgrade -h')
         for param in 'recipename srctree --version -V --branch -b --keep-temp --no-patch'.split():
             self.assertIn(param, result.output)
         # For the moment, we are using a real recipe.
         recipe = 'devtool-upgrade-test1'
         version = '1.6.0'
-        oldrecipefile = get_bb_var('FILE', recipe)
+        oldrecipefile = self.get_bb_var('FILE', recipe)
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         # Check that recipe is not already under devtool control
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output)
         # Check upgrade. Code does not check if new PV is older or newer that current PV, so, it may be that
         # we are downgrading instead of upgrading.
-        result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version))
+        result = self.runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, version))
         # Check if srctree at least is populated
         self.assertTrue(len(os.listdir(tempdir)) > 0, 'srctree (%s) should be populated with new (%s) source code' % (tempdir, version))
         # Check new recipe subdirectory is present
@@ -1288,7 +1294,7 @@ class DevtoolTests(DevtoolBase):
         newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, '%s_%s.bb' % (recipe, version))
         self.assertExists(newrecipefile, 'Recipe file should exist after upgrade')
         # Check devtool status and make sure recipe is present
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(recipe, result.output)
         self.assertIn(tempdir, result.output)
         # Check recipe got changed as expected
@@ -1298,8 +1304,8 @@ class DevtoolTests(DevtoolBase):
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
         # Check devtool reset recipe
-        result = runCmd('devtool reset %s -n' % recipe)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool reset %s -n' % recipe)
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output)
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting')
 
@@ -1308,24 +1314,24 @@ class DevtoolTests(DevtoolBase):
         # Check preconditions
         self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         recipe = 'devtool-upgrade-test2'
         commit = '6cc6077a36fe2648a5f993fe7c16c9632f946517'
-        oldrecipefile = get_bb_var('FILE', recipe)
+        oldrecipefile = self.get_bb_var('FILE', recipe)
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         # Check that recipe is not already under devtool control
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output)
         # Check upgrade
-        result = runCmd('devtool upgrade %s %s -S %s' % (recipe, tempdir, commit))
+        result = self.runCmd('devtool upgrade %s %s -S %s' % (recipe, tempdir, commit))
         # Check if srctree at least is populated
         self.assertTrue(len(os.listdir(tempdir)) > 0, 'srctree (%s) should be populated with new (%s) source code' % (tempdir, commit))
         # Check new recipe file is present
         newrecipefile = os.path.join(self.workspacedir, 'recipes', recipe, os.path.basename(oldrecipefile))
         self.assertExists(newrecipefile, 'Recipe file should exist after upgrade')
         # Check devtool status and make sure recipe is present
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(recipe, result.output)
         self.assertIn(tempdir, result.output)
         # Check recipe got changed as expected
@@ -1335,8 +1341,8 @@ class DevtoolTests(DevtoolBase):
             newlines = f.readlines()
         self.assertEqual(desiredlines, newlines)
         # Check devtool reset recipe
-        result = runCmd('devtool reset %s -n' % recipe)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool reset %s -n' % recipe)
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output)
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after resetting')
 
@@ -1347,10 +1353,10 @@ class DevtoolTests(DevtoolBase):
         This test executes the selftest-reverse command from meta-selftest."""
 
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
 
         s = "Microsoft Made No Profit From Anyone's Zunes Yo"
-        result = runCmd("devtool --quiet selftest-reverse \"%s\"" % s)
+        result = self.runCmd("devtool --quiet selftest-reverse \"%s\"" % s)
         self.assertEqual(result.output, s[::-1])
 
     def _copy_file_with_cleanup(self, srcfile, basedstdir, *paths):
@@ -1371,12 +1377,12 @@ class DevtoolTests(DevtoolBase):
         """Test that devtool loads only the first found plugin in BBPATH."""
 
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
 
-        devtool = runCmd("which devtool")
-        fromname = runCmd("devtool --quiet pluginfile")
+        devtool = self.runCmd("which devtool")
+        fromname = self.runCmd("devtool --quiet pluginfile")
         srcfile = fromname.output
-        bbpath = get_bb_var('BBPATH')
+        bbpath = self.get_bb_var('BBPATH')
         searchpath = bbpath.split(':') + [os.path.dirname(devtool.output)]
         plugincontent = []
         with open(srcfile) as fh:
@@ -1385,12 +1391,12 @@ class DevtoolTests(DevtoolBase):
             self.assertIn('meta-selftest', srcfile, 'wrong bbpath plugin found')
             for path in searchpath:
                 self._copy_file_with_cleanup(srcfile, path, 'lib', 'devtool')
-            result = runCmd("devtool --quiet count")
+            result = self.runCmd("devtool --quiet count")
             self.assertEqual(result.output, '1')
-            result = runCmd("devtool --quiet multiloaded")
+            result = self.runCmd("devtool --quiet multiloaded")
             self.assertEqual(result.output, "no")
             for path in searchpath:
-                result = runCmd("devtool --quiet bbdir")
+                result = self.runCmd("devtool --quiet bbdir")
                 self.assertEqual(result.output, path)
                 os.unlink(os.path.join(result.output, 'lib', 'devtool', 'bbpath.py'))
         finally:
@@ -1401,32 +1407,32 @@ class DevtoolTests(DevtoolBase):
         # Check preconditions
         self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         # Use a "real" recipe from meta-selftest
         recipe = 'devtool-upgrade-test1'
         oldversion = '1.5.3'
         newversion = '1.6.0'
-        oldrecipefile = get_bb_var('FILE', recipe)
+        oldrecipefile = self.get_bb_var('FILE', recipe)
         recipedir = os.path.dirname(oldrecipefile)
-        result = runCmd('git status --porcelain .', cwd=recipedir)
+        result = self.runCmd('git status --porcelain .', cwd=recipedir)
         if result.output.strip():
             self.fail('Recipe directory for %s contains uncommitted changes' % recipe)
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         # Check that recipe is not already under devtool control
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output)
         # Do the upgrade
-        result = runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, newversion))
+        result = self.runCmd('devtool upgrade %s %s -V %s' % (recipe, tempdir, newversion))
         # Check devtool status and make sure recipe is present
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(recipe, result.output)
         self.assertIn(tempdir, result.output)
         # Make a change to the source
-        result = runCmd('sed -i \'/^#include "pv.h"/a \\/* Here is a new comment *\\/\' src/pv/number.c', cwd=tempdir)
-        result = runCmd('git status --porcelain', cwd=tempdir)
+        result = self.runCmd('sed -i \'/^#include "pv.h"/a \\/* Here is a new comment *\\/\' src/pv/number.c', cwd=tempdir)
+        result = self.runCmd('git status --porcelain', cwd=tempdir)
         self.assertIn('M src/pv/number.c', result.output)
-        result = runCmd('git commit src/pv/number.c -m "Add a comment to the code"', cwd=tempdir)
+        result = self.runCmd('git commit src/pv/number.c -m "Add a comment to the code"', cwd=tempdir)
         # Check if patch is there
         recipedir = os.path.dirname(oldrecipefile)
         olddir = os.path.join(recipedir, recipe + '-' + oldversion)
@@ -1441,8 +1447,8 @@ class DevtoolTests(DevtoolBase):
         self.assertIn('/meta-selftest/', recipedir)
         # Try finish to the original layer
         self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
-        result = runCmd('devtool finish %s meta-selftest' % recipe)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool finish %s meta-selftest' % recipe)
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output, 'Recipe should have been reset by finish but wasn\'t')
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after finish')
         self.assertNotExists(oldrecipefile, 'Old recipe file should have been deleted but wasn\'t')
@@ -1461,12 +1467,12 @@ class DevtoolTests(DevtoolBase):
         # Try finish to a different layer - should create a bbappend
         # This cleanup isn't strictly necessary but do it anyway just in case it goes wrong and writes to here
         self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
-        oe_core_dir = os.path.join(get_bb_var('COREBASE'), 'meta')
+        oe_core_dir = os.path.join(self.get_bb_var('COREBASE'), 'meta')
         newrecipedir = os.path.join(oe_core_dir, 'recipes-test', 'devtool')
         newrecipefile = os.path.join(newrecipedir, '%s_%s.bb' % (recipe, newversion))
         self.track_for_cleanup(newrecipedir)
-        result = runCmd('devtool finish %s oe-core' % recipe)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool finish %s oe-core' % recipe)
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output, 'Recipe should have been reset by finish but wasn\'t')
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after finish')
         self.assertExists(oldrecipefile, 'Old recipe file should not have been deleted')
@@ -1482,25 +1488,25 @@ class DevtoolTests(DevtoolBase):
         # Try modifying a recipe
         self.track_for_cleanup(self.workspacedir)
         recipe = 'mdadm'
-        oldrecipefile = get_bb_var('FILE', recipe)
+        oldrecipefile = self.get_bb_var('FILE', recipe)
         recipedir = os.path.dirname(oldrecipefile)
-        result = runCmd('git status --porcelain .', cwd=recipedir)
+        result = self.runCmd('git status --porcelain .', cwd=recipedir)
         if result.output.strip():
             self.fail('Recipe directory for %s contains uncommitted changes' % recipe)
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
-        result = runCmd('devtool modify %s %s' % (recipe, tempdir))
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        result = self.runCmd('devtool modify %s %s' % (recipe, tempdir))
         self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
         # Test devtool status
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool status')
         self.assertIn(recipe, result.output)
         self.assertIn(tempdir, result.output)
         # Make a change to the source
-        result = runCmd('sed -i \'/^#include "mdadm.h"/a \\/* Here is a new comment *\\/\' maps.c', cwd=tempdir)
-        result = runCmd('git status --porcelain', cwd=tempdir)
+        result = self.runCmd('sed -i \'/^#include "mdadm.h"/a \\/* Here is a new comment *\\/\' maps.c', cwd=tempdir)
+        result = self.runCmd('git status --porcelain', cwd=tempdir)
         self.assertIn('M maps.c', result.output)
-        result = runCmd('git commit maps.c -m "Add a comment to the code"', cwd=tempdir)
+        result = self.runCmd('git commit maps.c -m "Add a comment to the code"', cwd=tempdir)
         for entry in os.listdir(recipedir):
             filesdir = os.path.join(recipedir, entry)
             if os.path.isdir(filesdir):
@@ -1516,8 +1522,8 @@ class DevtoolTests(DevtoolBase):
         self.assertIn('/meta/', recipedir)
         # Try finish to the original layer
         self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
-        result = runCmd('devtool finish %s meta' % recipe)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool finish %s meta' % recipe)
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output, 'Recipe should have been reset by finish but wasn\'t')
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after finish')
         expected_status = [(' M', '.*/%s$' % os.path.basename(oldrecipefile)),
@@ -1529,16 +1535,16 @@ class DevtoolTests(DevtoolBase):
         recipe, oldrecipefile, recipedir, filesdir = self._setup_test_devtool_finish_modify()
         # Ensure the recipe is where we think it should be (so that cleanup doesn't trash things)
         self.assertIn('/meta/', recipedir)
-        relpth = os.path.relpath(recipedir, os.path.join(get_bb_var('COREBASE'), 'meta'))
+        relpth = os.path.relpath(recipedir, os.path.join(self.get_bb_var('COREBASE'), 'meta'))
         appenddir = os.path.join(self.testlayer_path, relpth)
         self.track_for_cleanup(appenddir)
         # Try finish to the original layer
         self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
-        result = runCmd('devtool finish %s meta-selftest' % recipe)
-        result = runCmd('devtool status')
+        result = self.runCmd('devtool finish %s meta-selftest' % recipe)
+        result = self.runCmd('devtool status')
         self.assertNotIn(recipe, result.output, 'Recipe should have been reset by finish but wasn\'t')
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after finish')
-        result = runCmd('git status --porcelain .', cwd=recipedir)
+        result = self.runCmd('git status --porcelain .', cwd=recipedir)
         if result.output.strip():
             self.fail('Recipe directory for %s contains the following unexpected changes after finish:\n%s' % (recipe, result.output.strip()))
         recipefn = os.path.splitext(os.path.basename(oldrecipefile))[0]
@@ -1562,7 +1568,7 @@ class DevtoolTests(DevtoolBase):
         # Check preconditions
         self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
 
         # First run devtool add
         # We already have this recipe in OE-Core, but that doesn't matter
@@ -1571,7 +1577,7 @@ class DevtoolTests(DevtoolBase):
         recipefile = os.path.join(self.workspacedir, 'recipes', recipename, '%s_%s.bb' % (recipename, recipever))
         url = 'http://downloads.yoctoproject.org/mirror/sources/i2c-tools-%s.tar.bz2' % recipever
         def add_recipe():
-            result = runCmd('devtool add %s' % url)
+            result = self.runCmd('devtool add %s' % url)
             self.assertExists(recipefile, 'Expected recipe file not created')
             self.assertExists(os.path.join(self.workspacedir, 'sources', recipename), 'Source directory not created')
             checkvars = {}
@@ -1583,7 +1589,7 @@ class DevtoolTests(DevtoolBase):
         newrecipename = 'mynewrecipe'
         newrecipever = '456'
         newrecipefile = os.path.join(self.workspacedir, 'recipes', newrecipename, '%s_%s.bb' % (newrecipename, newrecipever))
-        result = runCmd('devtool rename %s %s -V %s' % (recipename, newrecipename, newrecipever))
+        result = self.runCmd('devtool rename %s %s -V %s' % (recipename, newrecipename, newrecipever))
         self.assertExists(newrecipefile, 'Recipe file not renamed')
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipename), 'Old recipe directory still exists')
         newsrctree = os.path.join(self.workspacedir, 'sources', newrecipename)
@@ -1593,11 +1599,11 @@ class DevtoolTests(DevtoolBase):
         checkvars['SRC_URI'] = url
         self._test_recipe_contents(newrecipefile, checkvars, [])
         # Try again - change just name this time
-        result = runCmd('devtool reset -n %s' % newrecipename)
+        result = self.runCmd('devtool reset -n %s' % newrecipename)
         shutil.rmtree(newsrctree)
         add_recipe()
         newrecipefile = os.path.join(self.workspacedir, 'recipes', newrecipename, '%s_%s.bb' % (newrecipename, recipever))
-        result = runCmd('devtool rename %s %s' % (recipename, newrecipename))
+        result = self.runCmd('devtool rename %s %s' % (recipename, newrecipename))
         self.assertExists(newrecipefile, 'Recipe file not renamed')
         self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipename), 'Old recipe directory still exists')
         self.assertExists(os.path.join(self.workspacedir, 'sources', newrecipename), 'Source directory not renamed')
@@ -1606,11 +1612,11 @@ class DevtoolTests(DevtoolBase):
         checkvars['SRC_URI'] = url.replace(recipever, '${PV}')
         self._test_recipe_contents(newrecipefile, checkvars, [])
         # Try again - change just version this time
-        result = runCmd('devtool reset -n %s' % newrecipename)
+        result = self.runCmd('devtool reset -n %s' % newrecipename)
         shutil.rmtree(newsrctree)
         add_recipe()
         newrecipefile = os.path.join(self.workspacedir, 'recipes', recipename, '%s_%s.bb' % (recipename, newrecipever))
-        result = runCmd('devtool rename %s -V %s' % (recipename, newrecipever))
+        result = self.runCmd('devtool rename %s -V %s' % (recipename, newrecipever))
         self.assertExists(newrecipefile, 'Recipe file not renamed')
         self.assertExists(os.path.join(self.workspacedir, 'sources', recipename), 'Source directory no longer exists')
         checkvars = {}
@@ -1641,59 +1647,59 @@ class DevtoolTests(DevtoolBase):
                          and modification to the source and configurations are reflected
                          when building the kernel.
          """
-        kernel_provider = get_bb_var('PREFERRED_PROVIDER_virtual/kernel')
+        kernel_provider = self.get_bb_var('PREFERRED_PROVIDER_virtual/kernel')
         # Clean up the enviroment
-        bitbake('%s -c clean' % kernel_provider)
+        self.bitbake('%s -c clean' % kernel_provider)
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
         self.track_for_cleanup(tempdir)
         self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
         self.add_command_to_tearDown('bitbake -c clean %s' % kernel_provider)
         #Step 1
         #Here is just generated the config file instead of all the kernel to optimize the
         #time of executing this test case.
-        bitbake('%s -c configure' % kernel_provider)
-        bbconfig = os.path.join(get_bb_var('B', kernel_provider),'.config')
-        buildir= get_bb_var('TOPDIR')
+        self.bitbake('%s -c configure' % kernel_provider)
+        bbconfig = os.path.join(self.get_bb_var('B', kernel_provider),'.config')
+        buildir= self.get_bb_var('TOPDIR')
         #Step 2
-        runCmd('cp %s %s' % (bbconfig, buildir))
+        self.runCmd('cp %s %s' % (bbconfig, buildir))
         self.assertExists(os.path.join(buildir, '.config'), 'Could not copy .config file from kernel')
 
         tmpconfig = os.path.join(buildir, '.config')
         #Step 3
-        bitbake('%s -c clean' % kernel_provider)
+        self.bitbake('%s -c clean' % kernel_provider)
         #Step 4.1
-        runCmd('devtool modify virtual/kernel -x %s' % tempdir)
+        self.runCmd('devtool modify virtual/kernel -x %s' % tempdir)
         self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
         #Step 4.2
         configfile = os.path.join(tempdir,'.config')
-        diff = runCmd('diff %s %s' % (tmpconfig, configfile))
+        diff = self.runCmd('diff %s %s' % (tmpconfig, configfile))
         self.assertEqual(0,diff.status,'Kernel .config file is not the same using bitbake and devtool')
         #Step 4.3
         #NOTE: virtual/kernel is mapped to kernel_provider
-        result = runCmd('devtool build %s' % kernel_provider)
+        result = self.runCmd('devtool build %s' % kernel_provider)
         self.assertEqual(0,result.status,'Cannot build kernel using `devtool build`')
-        kernelfile = os.path.join(get_bb_var('KBUILD_OUTPUT', kernel_provider), 'vmlinux')
+        kernelfile = os.path.join(self.get_bb_var('KBUILD_OUTPUT', kernel_provider), 'vmlinux')
         self.assertExists(kernelfile, 'Kernel was not build correctly')
 
         #Modify the kernel source
         modfile = os.path.join(tempdir,'arch/x86/boot/header.S')
         modstring = "Use a boot loader. Devtool testing."
-        modapplied = runCmd("sed -i 's/Use a boot loader./%s/' %s" % (modstring, modfile))
+        modapplied = self.runCmd("sed -i 's/Use a boot loader./%s/' %s" % (modstring, modfile))
         self.assertEqual(0,modapplied.status,'Modification to %s on kernel source failed' % modfile)
         #Modify the configuration
         codeconfigfile = os.path.join(tempdir,'.config.new')
         modconfopt = "CONFIG_SG_POOL=n"
-        modconf = runCmd("sed -i 's/CONFIG_SG_POOL=y/%s/' %s" % (modconfopt, codeconfigfile))
+        modconf = self.runCmd("sed -i 's/CONFIG_SG_POOL=y/%s/' %s" % (modconfopt, codeconfigfile))
         self.assertEqual(0,modconf.status,'Modification to %s failed' % codeconfigfile)
         #Build again kernel with devtool
-        rebuild = runCmd('devtool build %s' % kernel_provider)
+        rebuild = self.runCmd('devtool build %s' % kernel_provider)
         self.assertEqual(0,rebuild.status,'Fail to build kernel after modification of source and config')
         #Step 4.4
-        bzimagename = 'bzImage-' + get_bb_var('KERNEL_VERSION_NAME', kernel_provider)
-        bzimagefile = os.path.join(get_bb_var('D', kernel_provider),'boot', bzimagename)
-        checkmodcode = runCmd("grep '%s' %s" % (modstring, bzimagefile))
+        bzimagename = 'bzImage-' + self.get_bb_var('KERNEL_VERSION_NAME', kernel_provider)
+        bzimagefile = os.path.join(self.get_bb_var('D', kernel_provider),'boot', bzimagename)
+        checkmodcode = self.runCmd("grep '%s' %s" % (modstring, bzimagefile))
         self.assertEqual(0,checkmodcode.status,'Modification on kernel source failed')
         #Step 4.5
-        checkmodconfg = runCmd("grep %s %s" % (modconfopt, codeconfigfile))
+        checkmodconfg = self.runCmd("grep %s %s" % (modconfopt, codeconfigfile))
         self.assertEqual(0,checkmodconfg.status,'Modification to configuration file failed')
Signature · 1 line
2.11.0
#42 lore raw

PATCH 24/30 oeqa/selftest/cases: recipetool enable for threaded runs

In reply to Aníbal Limón

- Change to use wrappers from OESelfTestCase.
- Move templayer dir creation to RecipetoolBase class because now every class has its own build folder.

---
 meta/lib/oeqa/selftest/cases/recipetool.py | 105 +++++++++++++++--------------
 1 file changed, 54 insertions(+), 51 deletions(-)
meta/lib/oeqa/selftest/cases/recipetool.py +54 -51
index bdd405f1cc0..59544336578 100644
@@ -3,30 +3,30 @@ import shutil
 import tempfile
 import urllib.parse
 
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var
-from oeqa.utils.commands import get_bb_vars, create_temp_layer
-from oeqa.core.decorator.oeid import OETestID
 from oeqa.selftest.cases import devtool
+from oeqa.utils.commands import create_temp_layer
+from oeqa.core.decorator.oeid import OETestID
 
-templayerdir = None
-
-def setUpModule():
-    global templayerdir
-    templayerdir = tempfile.mkdtemp(prefix='recipetoolqa')
-    create_temp_layer(templayerdir, 'selftestrecipetool')
-    runCmd('bitbake-layers add-layer %s' % templayerdir)
-
+class RecipetoolBase(devtool.DevtoolBase):
+    @classmethod
+    def setUpClass(cls):
+        super(RecipetoolBase, cls).setUpClass()
 
-def tearDownModule():
-    runCmd('bitbake-layers remove-layer %s' % templayerdir, ignore_status=True)
-    runCmd('rm -rf %s' % templayerdir)
+        cls.templayerdir = tempfile.mkdtemp(prefix='recipetoolqa')
+        create_temp_layer(cls.templayerdir, 'selftestrecipetool')
+        cls.runCmd('bitbake-layers add-layer %s' % cls.templayerdir)
 
+    @classmethod
+    def tearDownClass(cls):
+        super(RecipetoolBase, cls).tearDownClass()
 
-class RecipetoolBase(devtool.DevtoolBase):
+        cls.runCmd('bitbake-layers remove-layer %s' % cls.templayerdir,
+                ignore_status=True)
+        cls.runCmd('rm -rf %s' % cls.templayerdir)
 
     def setUpLocal(self):
         super(RecipetoolBase, self).setUpLocal()
-        self.templayerdir = templayerdir
+
         self.tempdir = tempfile.mkdtemp(prefix='recipetoolqa')
         self.track_for_cleanup(self.tempdir)
         self.testfile = os.path.join(self.tempdir, 'testfile')
@@ -34,15 +34,15 @@ class RecipetoolBase(devtool.DevtoolBase):
             f.write('Test file\n')
 
     def tearDownLocal(self):
-        runCmd('rm -rf %s/recipes-*' % self.templayerdir)
+        self.runCmd('rm -rf %s/recipes-*' % self.templayerdir)
         super(RecipetoolBase, self).tearDownLocal()
 
     def _try_recipetool_appendcmd(self, cmd, testrecipe, expectedfiles, expectedlines=None):
-        result = runCmd(cmd)
+        result = self.runCmd(cmd)
         self.assertNotIn('Traceback', result.output)
 
         # Check the bbappend was created and applies properly
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         bbappendfile = self._check_bbappend(testrecipe, recipefile, self.templayerdir)
 
         # Check the bbappend contents
@@ -66,14 +66,16 @@ class RecipetoolBase(devtool.DevtoolBase):
 
 
 class RecipetoolTests(RecipetoolBase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @classmethod
     def setUpClass(cls):
         super(RecipetoolTests, cls).setUpClass()
         # Ensure we have the right data in shlibs/pkgdata
         cls.logger.info('Running bitbake to generate pkgdata')
-        bitbake('-c packagedata base-files coreutils busybox selftest-recipetool-appendfile')
-        bb_vars = get_bb_vars(['COREBASE', 'BBPATH'])
+        cls.bitbake('-c packagedata base-files coreutils busybox selftest-recipetool-appendfile')
+        bb_vars = cls.get_bb_vars(['COREBASE', 'BBPATH'])
         cls.corebase = bb_vars['COREBASE']
         cls.bbpath = bb_vars['BBPATH']
 
@@ -83,7 +85,7 @@ class RecipetoolTests(RecipetoolBase):
 
     def _try_recipetool_appendfile_fail(self, destfile, newfile, checkerror):
         cmd = 'recipetool appendfile %s %s %s' % (self.templayerdir, destfile, newfile)
-        result = runCmd(cmd, ignore_status=True)
+        result = self.runCmd(cmd, ignore_status=True)
         self.assertNotEqual(result.status, 0, 'Command "%s" should have failed but didn\'t' % cmd)
         self.assertNotIn('Traceback', result.output)
         for errorstr in checkerror:
@@ -125,7 +127,7 @@ class RecipetoolTests(RecipetoolBase):
         bbappendfile, _ = self._try_recipetool_appendfile('coreutils', '/bin/ls', self.testfile, '-r coreutils', expectedlines, [testfile2name])
         # But file should have
         copiedfile = os.path.join(os.path.dirname(bbappendfile), 'coreutils', testfile2name)
-        result = runCmd('diff -q %s %s' % (testfile2, copiedfile), ignore_status=True)
+        result = self.runCmd('diff -q %s %s' % (testfile2, copiedfile), ignore_status=True)
         self.assertNotEqual(result.status, 0, 'New file should have been copied but was not %s' % result.output)
 
     @OETestID(1178)
@@ -133,7 +135,7 @@ class RecipetoolTests(RecipetoolBase):
         # Try appending a binary file
         # /bin/ls can be a symlink to /usr/bin/ls
         ls = os.path.realpath("/bin/ls")
-        result = runCmd('recipetool appendfile %s /bin/ls %s -r coreutils' % (self.templayerdir, ls))
+        result = self.runCmd('recipetool appendfile %s /bin/ls %s -r coreutils' % (self.templayerdir, ls))
         self.assertIn('WARNING: ', result.output)
         self.assertIn('is a binary', result.output)
 
@@ -326,7 +328,7 @@ class RecipetoolTests(RecipetoolBase):
         # Try creating a bbappend in a layer that's not in bblayers.conf and has a different structure
         exttemplayerdir = os.path.join(self.tempdir, 'extlayer')
         self._create_temp_layer(exttemplayerdir, False, 'oeselftestextlayer', recipepathspec='metadata/recipes/recipes-*/*')
-        result = runCmd('recipetool appendfile %s /usr/share/selftest-replaceme-orig %s' % (exttemplayerdir, self.testfile))
+        result = self.runCmd('recipetool appendfile %s /usr/share/selftest-replaceme-orig %s' % (exttemplayerdir, self.testfile))
         self.assertNotIn('Traceback', result.output)
         createdfiles = []
         for root, _, files in os.walk(exttemplayerdir):
@@ -341,7 +343,7 @@ class RecipetoolTests(RecipetoolBase):
     def test_recipetool_appendfile_wildcard(self):
 
         def try_appendfile_wc(options):
-            result = runCmd('recipetool appendfile %s /etc/profile %s %s' % (self.templayerdir, self.testfile, options))
+            result = self.runCmd('recipetool appendfile %s /etc/profile %s %s' % (self.templayerdir, self.testfile, options))
             self.assertNotIn('Traceback', result.output)
             bbappendfile = None
             for root, _, files in os.walk(self.templayerdir):
@@ -351,11 +353,11 @@ class RecipetoolTests(RecipetoolBase):
                         break
             if not bbappendfile:
                 self.fail('No bbappend file created')
-            runCmd('rm -rf %s/recipes-*' % self.templayerdir)
+            self.runCmd('rm -rf %s/recipes-*' % self.templayerdir)
             return bbappendfile
 
         # Check without wildcard option
-        recipefn = os.path.basename(get_bb_var('FILE', 'base-files'))
+        recipefn = os.path.basename(self.get_bb_var('FILE', 'base-files'))
         filename = try_appendfile_wc('')
         self.assertEqual(filename, recipefn.replace('.bb', '.bbappend'))
         # Now check with wildcard option
@@ -369,7 +371,7 @@ class RecipetoolTests(RecipetoolBase):
         os.makedirs(tempsrc)
         recipefile = os.path.join(self.tempdir, 'logrotate_3.8.7.bb')
         srcuri = 'https://github.com/logrotate/logrotate/archive/r3-8-7.tar.gz'
-        result = runCmd('recipetool create -o %s %s -x %s' % (recipefile, srcuri, tempsrc))
+        result = self.runCmd('recipetool create -o %s %s -x %s' % (recipefile, srcuri, tempsrc))
         self.assertTrue(os.path.isfile(recipefile))
         checkvars = {}
         checkvars['LICENSE'] = 'GPLv2'
@@ -381,16 +383,16 @@ class RecipetoolTests(RecipetoolBase):
 
     @OETestID(1194)
     def test_recipetool_create_git(self):
-        if 'x11' not in get_bb_var('DISTRO_FEATURES'):
+        if 'x11' not in self.get_bb_var('DISTRO_FEATURES'):
             self.skipTest('Test requires x11 as distro feature')
         # Ensure we have the right data in shlibs/pkgdata
-        bitbake('libpng pango libx11 libxext jpeg libcheck')
+        self.bitbake('libpng pango libx11 libxext jpeg libcheck')
         # Try adding a recipe
         tempsrc = os.path.join(self.tempdir, 'srctree')
         os.makedirs(tempsrc)
         recipefile = os.path.join(self.tempdir, 'libmatchbox.bb')
         srcuri = 'git://git.yoctoproject.org/libmatchbox'
-        result = runCmd(['recipetool', 'create', '-o', recipefile, srcuri + ";rev=9f7cf8895ae2d39c465c04cc78e918c157420269", '-x', tempsrc])
+        result = self.runCmd(['recipetool', 'create', '-o', recipefile, srcuri + ";rev=9f7cf8895ae2d39c465c04cc78e918c157420269", '-x', tempsrc])
         self.assertTrue(os.path.isfile(recipefile), 'recipetool did not create recipe file; output:\n%s' % result.output)
         checkvars = {}
         checkvars['LICENSE'] = 'LGPLv2.1'
@@ -409,7 +411,7 @@ class RecipetoolTests(RecipetoolBase):
         os.makedirs(temprecipe)
         pv = '1.7.3.0'
         srcuri = 'http://www.dest-unreach.org/socat/download/socat-%s.tar.bz2' % pv
-        result = runCmd('recipetool create %s -o %s' % (srcuri, temprecipe))
+        result = self.runCmd('recipetool create %s -o %s' % (srcuri, temprecipe))
         dirlist = os.listdir(temprecipe)
         if len(dirlist) > 1:
             self.fail('recipetool created more than just one file; output:\n%s\ndirlist:\n%s' % (result.output, str(dirlist)))
@@ -432,7 +434,7 @@ class RecipetoolTests(RecipetoolBase):
         os.makedirs(temprecipe)
         recipefile = os.path.join(temprecipe, 'navit_0.5.0.bb')
         srcuri = 'http://downloads.sourceforge.net/project/navit/v0.5.0/navit-0.5.0.tar.gz'
-        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        result = self.runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
         self.assertTrue(os.path.isfile(recipefile))
         checkvars = {}
         checkvars['LICENSE'] = set(['Unknown', 'GPLv2', 'LGPLv2'])
@@ -450,7 +452,7 @@ class RecipetoolTests(RecipetoolBase):
         os.makedirs(temprecipe)
         recipefile = os.path.join(temprecipe, 'meson_git.bb')
         srcuri = 'https://github.com/mesonbuild/meson;rev=0.32.0'
-        result = runCmd(['recipetool', 'create', '-o', temprecipe, srcuri])
+        result = self.runCmd(['recipetool', 'create', '-o', temprecipe, srcuri])
         self.assertTrue(os.path.isfile(recipefile))
         checkvars = {}
         checkvars['LICENSE'] = set(['Apache-2.0'])
@@ -466,7 +468,7 @@ class RecipetoolTests(RecipetoolBase):
         pv = '0.32.0'
         recipefile = os.path.join(temprecipe, 'meson_%s.bb' % pv)
         srcuri = 'https://github.com/mesonbuild/meson/releases/download/%s/meson-%s.tar.gz' % (pv, pv)
-        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        result = self.runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
         self.assertTrue(os.path.isfile(recipefile))
         checkvars = {}
         checkvars['LICENSE'] = set(['Apache-2.0'])
@@ -481,7 +483,7 @@ class RecipetoolTests(RecipetoolBase):
         os.makedirs(temprecipe)
         recipefile = os.path.join(temprecipe, 'matchbox-terminal_git.bb')
         srcuri = 'http://git.yoctoproject.org/git/matchbox-terminal'
-        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        result = self.runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
         self.assertTrue(os.path.isfile(recipefile))
         checkvars = {}
         checkvars['LICENSE'] = set(['GPLv2'])
@@ -506,8 +508,8 @@ class RecipetoolTests(RecipetoolBase):
     def test_recipetool_load_plugin(self):
         """Test that recipetool loads only the first found plugin in BBPATH."""
 
-        recipetool = runCmd("which recipetool")
-        fromname = runCmd("recipetool --quiet pluginfile")
+        recipetool = self.runCmd("which recipetool")
+        fromname = self.runCmd("recipetool --quiet pluginfile")
         srcfile = fromname.output
         searchpath = self.bbpath.split(':') + [os.path.dirname(recipetool.output)]
         plugincontent = []
@@ -517,12 +519,12 @@ class RecipetoolTests(RecipetoolBase):
             self.assertIn('meta-selftest', srcfile, 'wrong bbpath plugin found')
             for path in searchpath:
                 self._copy_file_with_cleanup(srcfile, path, 'lib', 'recipetool')
-            result = runCmd("recipetool --quiet count")
+            result = self.runCmd("recipetool --quiet count")
             self.assertEqual(result.output, '1')
-            result = runCmd("recipetool --quiet multiloaded")
+            result = self.runCmd("recipetool --quiet multiloaded")
             self.assertEqual(result.output, "no")
             for path in searchpath:
-                result = runCmd("recipetool --quiet bbdir")
+                result = self.runCmd("recipetool --quiet bbdir")
                 self.assertEqual(result.output, path)
                 os.unlink(os.path.join(result.output, 'lib', 'recipetool', 'bbpath.py'))
         finally:
@@ -548,16 +550,15 @@ class RecipetoolAppendsrcBase(RecipetoolBase):
 
     def _try_recipetool_appendsrcfile_fail(self, testrecipe, newfile, destfile, checkerror):
         cmd = 'recipetool appendsrcfile %s %s %s %s' % (self.templayerdir, testrecipe, newfile, destfile or '')
-        result = runCmd(cmd, ignore_status=True)
+        result = self.runCmd(cmd, ignore_status=True)
         self.assertNotEqual(result.status, 0, 'Command "%s" should have failed but didn\'t' % cmd)
         self.assertNotIn('Traceback', result.output)
         for errorstr in checkerror:
             self.assertIn(errorstr, result.output)
 
-    @staticmethod
-    def _get_first_file_uri(recipe):
+    def _get_first_file_uri(self, recipe):
         '''Return the first file:// in SRC_URI for the specified recipe.'''
-        src_uri = get_bb_var('SRC_URI', recipe).split()
+        src_uri = self.get_bb_var('SRC_URI', recipe).split()
         for uri in src_uri:
             p = urllib.parse.urlparse(uri)
             if p.scheme == 'file':
@@ -605,7 +606,7 @@ class RecipetoolAppendsrcBase(RecipetoolBase):
 
         self._try_recipetool_appendsrcfiles(testrecipe, newfiles, expectedfiles=expectedfiles, destdir=destdir, options=options)
 
-        bb_vars = get_bb_vars(['SRC_URI', 'FILE', 'FILESEXTRAPATHS'], testrecipe)
+        bb_vars = self.get_bb_vars(['SRC_URI', 'FILE', 'FILESEXTRAPATHS'], testrecipe)
         src_uri = bb_vars['SRC_URI'].split()
         for f in expectedfiles:
             if destdir:
@@ -623,6 +624,8 @@ class RecipetoolAppendsrcBase(RecipetoolBase):
 
 
 class RecipetoolAppendsrcTests(RecipetoolAppendsrcBase):
+    _use_own_builddir = True
+    _main_thread = False
 
     @OETestID(1273)
     def test_recipetool_appendsrcfile_basic(self):
@@ -632,7 +635,7 @@ class RecipetoolAppendsrcTests(RecipetoolAppendsrcBase):
     def test_recipetool_appendsrcfile_basic_wildcard(self):
         testrecipe = 'base-files'
         self._test_appendsrcfile(testrecipe, 'a-file', options='-w')
-        recipefile = get_bb_var('FILE', testrecipe)
+        recipefile = self.get_bb_var('FILE', testrecipe)
         bbappendfile = self._check_bbappend(testrecipe, recipefile, self.templayerdir)
         self.assertEqual(os.path.basename(bbappendfile), '%s_%%.bbappend' % testrecipe)
 
@@ -647,7 +650,7 @@ class RecipetoolAppendsrcTests(RecipetoolAppendsrcBase):
     @OETestID(1280)
     def test_recipetool_appendsrcfile_srcdir_basic(self):
         testrecipe = 'bash'
-        bb_vars = get_bb_vars(['S', 'WORKDIR'], testrecipe)
+        bb_vars = self.get_bb_vars(['S', 'WORKDIR'], testrecipe)
         srcdir = bb_vars['S']
         workdir = bb_vars['WORKDIR']
         subdir = os.path.relpath(srcdir, workdir)
@@ -674,13 +677,13 @@ class RecipetoolAppendsrcTests(RecipetoolAppendsrcBase):
     def test_recipetool_appendsrcfile_replace_file_srcdir(self):
         testrecipe = 'bash'
         filepath = 'Makefile.in'
-        bb_vars = get_bb_vars(['S', 'WORKDIR'], testrecipe)
+        bb_vars = self.get_bb_vars(['S', 'WORKDIR'], testrecipe)
         srcdir = bb_vars['S']
         workdir = bb_vars['WORKDIR']
         subdir = os.path.relpath(srcdir, workdir)
 
         self._test_appendsrcfile(testrecipe, filepath, srcdir=subdir)
-        bitbake('%s:do_unpack' % testrecipe)
+        self.bitbake('%s:do_unpack' % testrecipe)
         self.assertEqual(open(self.testfile, 'r').read(), open(os.path.join(srcdir, filepath), 'r').read())
 
     @OETestID(1278)
Signature · 1 line
2.11.0
#43 lore raw

PATCH 25/30 oeqa/selftest/cases: Move devtool deploy test case to own module

In reply to Aníbal Limón

The devtool deploy test case uses runqemu that uses tinfoil so tinfoil requires to run on the main thread.

---
 meta/lib/oeqa/selftest/cases/devtool.py        | 91 ++-----------------------
 meta/lib/oeqa/selftest/cases/devtool_deploy.py | 93 ++++++++++++++++++++++++++
 2 files changed, 97 insertions(+), 87 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/devtool_deploy.py
meta/lib/oeqa/selftest/cases/devtool.py +4 -87
index c66e77a9ace..3d8d246bb7c 100644
@@ -6,7 +6,7 @@ import glob
 import fnmatch
 
 import oeqa.utils.ftools as ftools
-from oeqa.utils.commands import create_temp_layer, runqemu
+from oeqa.utils.commands import create_temp_layer
 
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.core.decorator.oeid import OETestID
@@ -114,6 +114,9 @@ class DevtoolBase(OESelftestTestCase):
 
 
 class DevtoolCommon(DevtoolBase):
+    _use_own_builddir = True
+    _main_thread = False
+
     @classmethod
     def setUpClass(cls):
         super(DevtoolCommon, cls).setUpClass()
@@ -1144,92 +1147,6 @@ class DevtoolTests(DevtoolCommon):
         matches2 = glob.glob(stampprefix2 + '*')
         self.assertFalse(matches2, 'Stamp files exist for recipe %s that should have been cleaned' % testrecipe2)
 
-    @OETestID(1272)
-    def test_devtool_deploy_target(self):
-        # NOTE: Whilst this test would seemingly be better placed as a runtime test,
-        # unfortunately the runtime tests run under bitbake and you can't run
-        # devtool within bitbake (since devtool needs to run bitbake itself).
-        # Additionally we are testing build-time functionality as well, so
-        # really this has to be done as an oe-selftest test.
-        #
-        # Check preconditions
-        machine = self.get_bb_var('MACHINE')
-        if not machine.startswith('qemu'):
-            self.skipTest('This test only works with qemu machines')
-        if not os.path.exists('/etc/runqemu-nosudo'):
-            self.skipTest('You must set up tap devices with scripts/runqemu-gen-tapdevs before running this test')
-        result = self.runCmd('PATH="$PATH:/sbin:/usr/sbin" ip tuntap show', ignore_status=True)
-        if result.status != 0:
-            result = self.runCmd('PATH="$PATH:/sbin:/usr/sbin" ifconfig -a', ignore_status=True)
-            if result.status != 0:
-                self.skipTest('Failed to determine if tap devices exist with ifconfig or ip: %s' % result.output)
-        for line in result.output.splitlines():
-            if line.startswith('tap'):
-                break
-        else:
-            self.skipTest('No tap devices found - you must set up tap devices with scripts/runqemu-gen-tapdevs before running this test')
-        self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
-        # Definitions
-        testrecipe = 'mdadm'
-        testfile = '/sbin/mdadm'
-        testimage = 'oe-selftest-image'
-        testcommand = '/sbin/mdadm --help'
-        # Build an image to run
-        self.bitbake("%s qemu-native qemu-helper-native" % testimage)
-        deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
-        self.add_command_to_tearDown('bitbake -c clean %s' % testimage)
-        self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, testimage))
-        # Clean recipe so the first deploy will fail
-        self.bitbake("%s -c clean" % testrecipe)
-        # Try devtool modify
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
-        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
-        # Test that deploy-target at this point fails (properly)
-        result = self.runCmd('devtool deploy-target -n %s root@localhost' % testrecipe, ignore_status=True)
-        self.assertNotEqual(result.output, 0, 'devtool deploy-target should have failed, output: %s' % result.output)
-        self.assertNotIn(result.output, 'Traceback', 'devtool deploy-target should have failed with a proper error not a traceback, output: %s' % result.output)
-        result = self.runCmd('devtool build %s' % testrecipe)
-        # First try a dry-run of deploy-target
-        result = self.runCmd('devtool deploy-target -n %s root@localhost' % testrecipe)
-        self.assertIn('  %s' % testfile, result.output)
-        # Boot the image
-        with runqemu(testimage) as qemu:
-            # Now really test deploy-target
-            result = self.runCmd('devtool deploy-target -c %s root@%s' % (testrecipe, qemu.ip))
-            # Run a test command to see if it was installed properly
-            sshargs = '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
-            result = self.runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand))
-            # Check if it deployed all of the files with the right ownership/perms
-            # First look on the host - need to do this under pseudo to get the correct ownership/perms
-            bb_vars = self.get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD'], testrecipe)
-            installdir = bb_vars['D']
-            fakerootenv = bb_vars['FAKEROOTENV']
-            fakerootcmd = bb_vars['FAKEROOTCMD']
-            result = self.runCmd('%s %s find . -type f -exec ls -l {} \;' % (fakerootenv, fakerootcmd), cwd=installdir)
-            filelist1 = self._process_ls_output(result.output)
-
-            # Now look on the target
-            tempdir2 = tempfile.mkdtemp(prefix='devtoolqa')
-            self.track_for_cleanup(tempdir2)
-            tmpfilelist = os.path.join(tempdir2, 'files.txt')
-            with open(tmpfilelist, 'w') as f:
-                for line in filelist1:
-                    splitline = line.split()
-                    f.write(splitline[-1] + '\n')
-            result = self.runCmd('cat %s | ssh -q %s root@%s \'xargs ls -l\'' % (tmpfilelist, sshargs, qemu.ip))
-            filelist2 = self._process_ls_output(result.output)
-            filelist1.sort(key=lambda item: item.split()[-1])
-            filelist2.sort(key=lambda item: item.split()[-1])
-            self.assertEqual(filelist1, filelist2)
-            # Test undeploy-target
-            result = self.runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip))
-            result = self.runCmd('ssh %s root@%s %s' % (sshargs, qemu.ip, testcommand), ignore_status=True)
-            self.assertNotEqual(result, 0, 'undeploy-target did not remove command as it should have')
-
     @OETestID(1366)
     def test_devtool_build_image(self):
         """Test devtool build-image plugin"""
meta/lib/oeqa/selftest/cases/devtool_deploy.py new file +93 -0

93 lines not on this page · raw message

Signature · 1 line
2.11.0
#44 lore raw

PATCH 26/30 selftest/cases/devtool{, end}: Move update/finish_modify tests to its own module

In reply to Aníbal Limón

This devtool tests are set to run at end because made changes into the poky repository causing problems (non deteministic meta data) in other execution threads.

---
 meta/lib/oeqa/selftest/cases/devtool.py     | 545 ++--------------------------
 meta/lib/oeqa/selftest/cases/devtool_end.py | 506 ++++++++++++++++++++++++++
 2 files changed, 532 insertions(+), 519 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/devtool_end.py
meta/lib/oeqa/selftest/cases/devtool.py +26 -519
index 3d8d246bb7c..44ba361f16c 100644
@@ -112,6 +112,32 @@ class DevtoolBase(OESelftestTestCase):
             filelist.append(' '.join(splitline))
         return filelist
 
+    def _check_src_repo(self, repo_dir):
+        """Check srctree git repository"""
+        self.assertTrue(os.path.isdir(os.path.join(repo_dir, '.git')),
+                        'git repository for external source tree not found')
+        result = self.runCmd('git status --porcelain', cwd=repo_dir)
+        self.assertEqual(result.output.strip(), "",
+                         'Created git repo is not clean')
+        result = self.runCmd('git symbolic-ref HEAD', cwd=repo_dir)
+        self.assertEqual(result.output.strip(), "refs/heads/devtool",
+                         'Wrong branch in git repo')
+
+    def _check_repo_status(self, repo_dir, expected_status):
+        """Check the worktree status of a repository"""
+        result = self.runCmd('git status . --porcelain',
+                        cwd=repo_dir)
+        for line in result.output.splitlines():
+            for ind, (f_status, fn_re) in enumerate(expected_status):
+                if re.match(fn_re, line[3:]):
+                    if f_status != line[:2]:
+                        self.fail('Unexpected status in line: %s' % line)
+                    expected_status.pop(ind)
+                    break
+            else:
+                self.fail('Unexpected modified file in line: %s' % line)
+        if expected_status:
+            self.fail('Missing file changes: %s' % expected_status)
 
 class DevtoolCommon(DevtoolBase):
     _use_own_builddir = True
@@ -150,33 +176,6 @@ class DevtoolTests(DevtoolCommon):
     _use_own_builddir = True
     _main_thread = False
 
-    def _check_src_repo(self, repo_dir):
-        """Check srctree git repository"""
-        self.assertTrue(os.path.isdir(os.path.join(repo_dir, '.git')),
-                        'git repository for external source tree not found')
-        result = self.runCmd('git status --porcelain', cwd=repo_dir)
-        self.assertEqual(result.output.strip(), "",
-                         'Created git repo is not clean')
-        result = self.runCmd('git symbolic-ref HEAD', cwd=repo_dir)
-        self.assertEqual(result.output.strip(), "refs/heads/devtool",
-                         'Wrong branch in git repo')
-
-    def _check_repo_status(self, repo_dir, expected_status):
-        """Check the worktree status of a repository"""
-        result = self.runCmd('git status . --porcelain',
-                        cwd=repo_dir)
-        for line in result.output.splitlines():
-            for ind, (f_status, fn_re) in enumerate(expected_status):
-                if re.match(fn_re, line[3:]):
-                    if f_status != line[:2]:
-                        self.fail('Unexpected status in line: %s' % line)
-                    expected_status.pop(ind)
-                    break
-            else:
-                self.fail('Unexpected modified file in line: %s' % line)
-        if expected_status:
-            self.fail('Missing file changes: %s' % expected_status)
-
     @OETestID(1158)
     def test_create_workspace(self):
         # Check preconditions
@@ -687,417 +686,6 @@ class DevtoolTests(DevtoolCommon):
         self._check_src_repo(tempdir)
         # This is probably sufficient
 
-
-    @OETestID(1169)
-    def test_devtool_update_recipe(self):
-        # Check preconditions
-        testrecipe = 'minicom'
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        self.assertNotIn('git://', src_uri, 'This test expects the %s recipe to NOT be a git recipe' % testrecipe)
-        self._check_repo_status(os.path.dirname(recipefile), [])
-        # First, modify a recipe
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        # We don't use -x here so that we test the behaviour of devtool modify without it
-        result = self.runCmd('devtool modify %s %s' % (testrecipe, tempdir))
-        # Check git repo
-        self._check_src_repo(tempdir)
-        # Add a couple of commits
-        # FIXME: this only tests adding, need to also test update and remove
-        result = self.runCmd('echo "Additional line" >> README', cwd=tempdir)
-        result = self.runCmd('git commit -a -m "Change the README"', cwd=tempdir)
-        result = self.runCmd('echo "A new file" > devtool-new-file', cwd=tempdir)
-        result = self.runCmd('git add devtool-new-file', cwd=tempdir)
-        result = self.runCmd('git commit -m "Add a new file"', cwd=tempdir)
-        self.add_command_to_tearDown('cd %s; rm %s/*.patch; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = self.runCmd('devtool update-recipe %s' % testrecipe)
-        expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
-                           ('??', '.*/0001-Change-the-README.patch$'),
-                           ('??', '.*/0002-Add-a-new-file.patch$')]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
-    @OETestID(1172)
-    def test_devtool_update_recipe_git(self):
-        # Check preconditions
-        testrecipe = 'mtd-utils'
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        self.assertIn('git://', src_uri, 'This test expects the %s recipe to be a git recipe' % testrecipe)
-        patches = []
-        for entry in src_uri.split():
-            if entry.startswith('file://') and entry.endswith('.patch'):
-                patches.append(entry[7:].split(';')[0])
-        self.assertGreater(len(patches), 0, 'The %s recipe does not appear to contain any patches, so this test will not be effective' % testrecipe)
-        self._check_repo_status(os.path.dirname(recipefile), [])
-        # First, modify a recipe
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
-        # Check git repo
-        self._check_src_repo(tempdir)
-        # Add a couple of commits
-        # FIXME: this only tests adding, need to also test update and remove
-        result = self.runCmd('echo "# Additional line" >> Makefile.am', cwd=tempdir)
-        result = self.runCmd('git commit -a -m "Change the Makefile"', cwd=tempdir)
-        result = self.runCmd('echo "A new file" > devtool-new-file', cwd=tempdir)
-        result = self.runCmd('git add devtool-new-file', cwd=tempdir)
-        result = self.runCmd('git commit -m "Add a new file"', cwd=tempdir)
-        self.add_command_to_tearDown('cd %s; rm -rf %s; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = self.runCmd('devtool update-recipe -m srcrev %s' % testrecipe)
-        expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile))] + \
-                          [(' D', '.*/%s$' % patch) for patch in patches]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
-        result = self.runCmd('git diff %s' % os.path.basename(recipefile), cwd=os.path.dirname(recipefile))
-        addlines = ['SRCREV = ".*"', 'SRC_URI = "git://git.infradead.org/mtd-utils.git"']
-        srcurilines = src_uri.split()
-        srcurilines[0] = 'SRC_URI = "' + srcurilines[0]
-        srcurilines.append('"')
-        removelines = ['SRCREV = ".*"'] + srcurilines
-        for line in result.output.splitlines():
-            if line.startswith('+++') or line.startswith('---'):
-                continue
-            elif line.startswith('+'):
-                matched = False
-                for item in addlines:
-                    if re.match(item, line[1:].strip()):
-                        matched = True
-                        break
-                self.assertTrue(matched, 'Unexpected diff add line: %s' % line)
-            elif line.startswith('-'):
-                matched = False
-                for item in removelines:
-                    if re.match(item, line[1:].strip()):
-                        matched = True
-                        break
-                self.assertTrue(matched, 'Unexpected diff remove line: %s' % line)
-        # Now try with auto mode
-        self.runCmd('cd %s; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, os.path.basename(recipefile)))
-        result = self.runCmd('devtool update-recipe %s' % testrecipe)
-        result = self.runCmd('git rev-parse --show-toplevel', cwd=os.path.dirname(recipefile))
-        topleveldir = result.output.strip()
-        relpatchpath = os.path.join(os.path.relpath(os.path.dirname(recipefile), topleveldir), testrecipe)
-        expected_status = [(' M', os.path.relpath(recipefile, topleveldir)),
-                           ('??', '%s/0001-Change-the-Makefile.patch' % relpatchpath),
-                           ('??', '%s/0002-Add-a-new-file.patch' % relpatchpath)]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
-    @OETestID(1170)
-    def test_devtool_update_recipe_append(self):
-        # Check preconditions
-        testrecipe = 'mdadm'
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        self.assertNotIn('git://', src_uri, 'This test expects the %s recipe to NOT be a git recipe' % testrecipe)
-        self._check_repo_status(os.path.dirname(recipefile), [])
-        # First, modify a recipe
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        tempsrcdir = os.path.join(tempdir, 'source')
-        templayerdir = os.path.join(tempdir, 'layer')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempsrcdir))
-        # Check git repo
-        self._check_src_repo(tempsrcdir)
-        # Add a commit
-        result = self.runCmd("sed 's!\\(#define VERSION\\W*\"[^\"]*\\)\"!\\1-custom\"!' -i ReadMe.c", cwd=tempsrcdir)
-        result = self.runCmd('git commit -a -m "Add our custom version"', cwd=tempsrcdir)
-        self.add_command_to_tearDown('cd %s; rm -f %s/*.patch; git checkout .' % (os.path.dirname(recipefile), testrecipe))
-        # Create a temporary layer and add it to bblayers.conf
-        self._create_temp_layer(templayerdir, True, 'selftestupdaterecipe')
-        # Create the bbappend
-        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
-        self.assertNotIn('WARNING:', result.output)
-        # Check recipe is still clean
-        self._check_repo_status(os.path.dirname(recipefile), [])
-        # Check bbappend was created
-        splitpath = os.path.dirname(recipefile).split(os.sep)
-        appenddir = os.path.join(templayerdir, splitpath[-2], splitpath[-1])
-        bbappendfile = self._check_bbappend(testrecipe, recipefile, appenddir)
-        patchfile = os.path.join(appenddir, testrecipe, '0001-Add-our-custom-version.patch')
-        self.assertExists(patchfile, 'Patch file not created')
-
-        # Check bbappend contents
-        expectedlines = ['FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n',
-                         '\n',
-                         'SRC_URI += "file://0001-Add-our-custom-version.patch"\n',
-                         '\n']
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, f.readlines())
-
-        # Check we can run it again and bbappend isn't modified
-        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, f.readlines())
-        # Drop new commit and check patch gets deleted
-        result = self.runCmd('git reset HEAD^', cwd=tempsrcdir)
-        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
-        self.assertNotExists(patchfile, 'Patch file not deleted')
-        expectedlines2 = ['FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"\n',
-                         '\n']
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines2, f.readlines())
-        # Put commit back and check we can run it if layer isn't in bblayers.conf
-        os.remove(bbappendfile)
-        result = self.runCmd('git commit -a -m "Add our custom version"', cwd=tempsrcdir)
-        result = self.runCmd('bitbake-layers remove-layer %s' % templayerdir, cwd=self.builddir)
-        result = self.runCmd('devtool update-recipe %s -a %s' % (testrecipe, templayerdir))
-        self.assertIn('WARNING: Specified layer is not currently enabled in bblayers.conf', result.output)
-        self.assertExists(patchfile, 'Patch file not created (with disabled layer)')
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, f.readlines())
-        # Deleting isn't expected to work under these circumstances
-
-    @OETestID(1171)
-    def test_devtool_update_recipe_append_git(self):
-        # Check preconditions
-        testrecipe = 'mtd-utils'
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        self.assertIn('git://', src_uri, 'This test expects the %s recipe to be a git recipe' % testrecipe)
-        for entry in src_uri.split():
-            if entry.startswith('git://'):
-                git_uri = entry
-                break
-        self._check_repo_status(os.path.dirname(recipefile), [])
-        # First, modify a recipe
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        tempsrcdir = os.path.join(tempdir, 'source')
-        templayerdir = os.path.join(tempdir, 'layer')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempsrcdir))
-        # Check git repo
-        self._check_src_repo(tempsrcdir)
-        # Add a commit
-        result = self.runCmd('echo "# Additional line" >> Makefile.am', cwd=tempsrcdir)
-        result = self.runCmd('git commit -a -m "Change the Makefile"', cwd=tempsrcdir)
-        self.add_command_to_tearDown('cd %s; rm -f %s/*.patch; git checkout .' % (os.path.dirname(recipefile), testrecipe))
-        # Create a temporary layer
-        os.makedirs(os.path.join(templayerdir, 'conf'))
-        with open(os.path.join(templayerdir, 'conf', 'layer.conf'), 'w') as f:
-            f.write('BBPATH .= ":${LAYERDIR}"\n')
-            f.write('BBFILES += "${LAYERDIR}/recipes-*/*/*.bbappend"\n')
-            f.write('BBFILE_COLLECTIONS += "oeselftesttemplayer"\n')
-            f.write('BBFILE_PATTERN_oeselftesttemplayer = "^${LAYERDIR}/"\n')
-            f.write('BBFILE_PRIORITY_oeselftesttemplayer = "999"\n')
-            f.write('BBFILE_PATTERN_IGNORE_EMPTY_oeselftesttemplayer = "1"\n')
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s || true' % templayerdir)
-        result = self.runCmd('bitbake-layers add-layer %s' % templayerdir, cwd=self.builddir)
-        # Create the bbappend
-        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
-        self.assertNotIn('WARNING:', result.output)
-        # Check recipe is still clean
-        self._check_repo_status(os.path.dirname(recipefile), [])
-        # Check bbappend was created
-        splitpath = os.path.dirname(recipefile).split(os.sep)
-        appenddir = os.path.join(templayerdir, splitpath[-2], splitpath[-1])
-        bbappendfile = self._check_bbappend(testrecipe, recipefile, appenddir)
-        self.assertNotExists(os.path.join(appenddir, testrecipe), 'Patch directory should not be created')
-
-        # Check bbappend contents
-        result = self.runCmd('git rev-parse HEAD', cwd=tempsrcdir)
-        expectedlines = set(['SRCREV = "%s"\n' % result.output,
-                             '\n',
-                             'SRC_URI = "%s"\n' % git_uri,
-                             '\n'])
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, set(f.readlines()))
-
-        # Check we can run it again and bbappend isn't modified
-        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, set(f.readlines()))
-        # Drop new commit and check SRCREV changes
-        result = self.runCmd('git reset HEAD^', cwd=tempsrcdir)
-        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
-        self.assertNotExists(os.path.join(appenddir, testrecipe), 'Patch directory should not be created')
-        result = self.runCmd('git rev-parse HEAD', cwd=tempsrcdir)
-        expectedlines = set(['SRCREV = "%s"\n' % result.output,
-                             '\n',
-                             'SRC_URI = "%s"\n' % git_uri,
-                             '\n'])
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, set(f.readlines()))
-        # Put commit back and check we can run it if layer isn't in bblayers.conf
-        os.remove(bbappendfile)
-        result = self.runCmd('git commit -a -m "Change the Makefile"', cwd=tempsrcdir)
-        result = self.runCmd('bitbake-layers remove-layer %s' % templayerdir, cwd=self.builddir)
-        result = self.runCmd('devtool update-recipe -m srcrev %s -a %s' % (testrecipe, templayerdir))
-        self.assertIn('WARNING: Specified layer is not currently enabled in bblayers.conf', result.output)
-        self.assertNotExists(os.path.join(appenddir, testrecipe), 'Patch directory should not be created')
-        result = self.runCmd('git rev-parse HEAD', cwd=tempsrcdir)
-        expectedlines = set(['SRCREV = "%s"\n' % result.output,
-                             '\n',
-                             'SRC_URI = "%s"\n' % git_uri,
-                             '\n'])
-        with open(bbappendfile, 'r') as f:
-            self.assertEqual(expectedlines, set(f.readlines()))
-        # Deleting isn't expected to work under these circumstances
-
-    @OETestID(1370)
-    def test_devtool_update_recipe_local_files(self):
-        """Check that local source files are copied over instead of patched"""
-        testrecipe = 'makedevs'
-        recipefile = self.get_bb_var('FILE', testrecipe)
-        # Setup srctree for modifying the recipe
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be
-        # building it)
-        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
-        # Check git repo
-        self._check_src_repo(tempdir)
-        # Try building just to ensure we haven't broken that
-        self.bitbake("%s" % testrecipe)
-        # Edit / commit local source
-        self.runCmd('echo "/* Foobar */" >> oe-local-files/makedevs.c', cwd=tempdir)
-        self.runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
-        self.runCmd('echo "Bar" > new-file', cwd=tempdir)
-        self.runCmd('git add new-file', cwd=tempdir)
-        self.runCmd('git commit -m "Add new file"', cwd=tempdir)
-        self.add_command_to_tearDown('cd %s; git clean -fd .; git checkout .' %
-                                     os.path.dirname(recipefile))
-        self.runCmd('devtool update-recipe %s' % testrecipe)
-        expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
-                           (' M', '.*/makedevs/makedevs.c$'),
-                           ('??', '.*/makedevs/new-local$'),
-                           ('??', '.*/makedevs/0001-Add-new-file.patch$')]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
-    @OETestID(1371)
-    def test_devtool_update_recipe_local_files_2(self):
-        """Check local source files support when oe-local-files is in Git"""
-        testrecipe = 'lzo'
-        recipefile = self.get_bb_var('FILE', testrecipe)
-        # Setup srctree for modifying the recipe
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        result = self.runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
-        # Check git repo
-        self._check_src_repo(tempdir)
-        # Add oe-local-files to Git
-        self.runCmd('rm oe-local-files/.gitignore', cwd=tempdir)
-        self.runCmd('git add oe-local-files', cwd=tempdir)
-        self.runCmd('git commit -m "Add local sources"', cwd=tempdir)
-        # Edit / commit local sources
-        self.runCmd('echo "# Foobar" >> oe-local-files/acinclude.m4', cwd=tempdir)
-        self.runCmd('git commit -am "Edit existing file"', cwd=tempdir)
-        self.runCmd('git rm oe-local-files/run-ptest', cwd=tempdir)
-        self.runCmd('git commit -m"Remove file"', cwd=tempdir)
-        self.runCmd('echo "Foo" > oe-local-files/new-local', cwd=tempdir)
-        self.runCmd('git add oe-local-files/new-local', cwd=tempdir)
-        self.runCmd('git commit -m "Add new local file"', cwd=tempdir)
-        self.runCmd('echo "Gar" > new-file', cwd=tempdir)
-        self.runCmd('git add new-file', cwd=tempdir)
-        self.runCmd('git commit -m "Add new file"', cwd=tempdir)
-        self.add_command_to_tearDown('cd %s; git clean -fd .; git checkout .' %
-                                     os.path.dirname(recipefile))
-        # Checkout unmodified file to working copy -> devtool should still pick
-        # the modified version from HEAD
-        self.runCmd('git checkout HEAD^ -- oe-local-files/acinclude.m4', cwd=tempdir)
-        self.runCmd('devtool update-recipe %s' % testrecipe)
-        expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
-                           (' M', '.*/acinclude.m4$'),
-                           (' D', '.*/run-ptest$'),
-                           ('??', '.*/new-local$'),
-                           ('??', '.*/0001-Add-new-file.patch$')]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
-    @OETestID(1627)
-    def test_devtool_update_recipe_local_files_3(self):
-        # First, modify the recipe
-        testrecipe = 'devtool-test-localonly'
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = self.runCmd('devtool modify %s' % testrecipe)
-        # Modify one file
-        self.runCmd('echo "Another line" >> file2', cwd=os.path.join(self.workspacedir, 'sources', testrecipe, 'oe-local-files'))
-        self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = self.runCmd('devtool update-recipe %s' % testrecipe)
-        expected_status = [(' M', '.*/%s/file2$' % testrecipe)]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
-    @OETestID(1629)
-    def test_devtool_update_recipe_local_patch_gz(self):
-        # First, modify the recipe
-        testrecipe = 'devtool-test-patch-gz'
-        if self.get_bb_var('DISTRO') == 'poky-tiny':
-            self.skipTest("The DISTRO 'poky-tiny' does not provide the dependencies needed by %s" % testrecipe)
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = self.runCmd('devtool modify %s' % testrecipe)
-        # Modify one file
-        srctree = os.path.join(self.workspacedir, 'sources', testrecipe)
-        self.runCmd('echo "Another line" >> README', cwd=srctree)
-        self.runCmd('git commit -a --amend --no-edit', cwd=srctree)
-        self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = self.runCmd('devtool update-recipe %s' % testrecipe)
-        expected_status = [(' M', '.*/%s/readme.patch.gz$' % testrecipe)]
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-        patch_gz = os.path.join(os.path.dirname(recipefile), testrecipe, 'readme.patch.gz')
-        result = self.runCmd('file %s' % patch_gz)
-        if 'gzip compressed data' not in result.output:
-            self.fail('New patch file is not gzipped - file reports:\n%s' % result.output)
-
-    @OETestID(1628)
-    def test_devtool_update_recipe_local_files_subdir(self):
-        # Try devtool extract on a recipe that has a file with subdir= set in
-        # SRC_URI such that it overwrites a file that was in an archive that
-        # was also in SRC_URI
-        # First, modify the recipe
-        testrecipe = 'devtool-test-subdir'
-        bb_vars = self.get_bb_vars(['FILE', 'SRC_URI'], testrecipe)
-        recipefile = bb_vars['FILE']
-        src_uri = bb_vars['SRC_URI']
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.track_for_cleanup(self.workspacedir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        # (don't bother with cleaning the recipe on teardown, we won't be building it)
-        result = self.runCmd('devtool modify %s' % testrecipe)
-        testfile = os.path.join(self.workspacedir, 'sources', testrecipe, 'testfile')
-        self.assertExists(testfile, 'Extracted source could not be found')
-        with open(testfile, 'r') as f:
-            contents = f.read().rstrip()
-        self.assertEqual(contents, 'Modified version', 'File has apparently not been overwritten as it should have been')
-        # Test devtool update-recipe without modifying any files
-        self.add_command_to_tearDown('cd %s; rm %s/*; git checkout %s %s' % (os.path.dirname(recipefile), testrecipe, testrecipe, os.path.basename(recipefile)))
-        result = self.runCmd('devtool update-recipe %s' % testrecipe)
-        expected_status = []
-        self._check_repo_status(os.path.dirname(recipefile), expected_status)
-
     @OETestID(1163)
     def test_devtool_extract(self):
         tempdir = tempfile.mkdtemp(prefix='devtoolqa')
@@ -1399,87 +987,6 @@ class DevtoolTests(DevtoolCommon):
         self.assertExists(os.path.join(newdir, patchfn), 'Patch file should have been copied into new directory but wasn\'t')
         self.assertExists(os.path.join(newdir, '0002-Add-a-comment-to-the-code.patch'), 'New patch file should have been created but wasn\'t')
 
-    def _setup_test_devtool_finish_modify(self):
-        # Check preconditions
-        self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
-        # Try modifying a recipe
-        self.track_for_cleanup(self.workspacedir)
-        recipe = 'mdadm'
-        oldrecipefile = self.get_bb_var('FILE', recipe)
-        recipedir = os.path.dirname(oldrecipefile)
-        result = self.runCmd('git status --porcelain .', cwd=recipedir)
-        if result.output.strip():
-            self.fail('Recipe directory for %s contains uncommitted changes' % recipe)
-        tempdir = tempfile.mkdtemp(prefix='devtoolqa')
-        self.track_for_cleanup(tempdir)
-        self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
-        result = self.runCmd('devtool modify %s %s' % (recipe, tempdir))
-        self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
-        # Test devtool status
-        result = self.runCmd('devtool status')
-        self.assertIn(recipe, result.output)
-        self.assertIn(tempdir, result.output)
-        # Make a change to the source
-        result = self.runCmd('sed -i \'/^#include "mdadm.h"/a \\/* Here is a new comment *\\/\' maps.c', cwd=tempdir)
-        result = self.runCmd('git status --porcelain', cwd=tempdir)
-        self.assertIn('M maps.c', result.output)
-        result = self.runCmd('git commit maps.c -m "Add a comment to the code"', cwd=tempdir)
-        for entry in os.listdir(recipedir):
-            filesdir = os.path.join(recipedir, entry)
-            if os.path.isdir(filesdir):
-                break
-        else:
-            self.fail('Unable to find recipe files directory for %s' % recipe)
-        return recipe, oldrecipefile, recipedir, filesdir
-
-    @OETestID(1621)
-    def test_devtool_finish_modify_origlayer(self):
-        recipe, oldrecipefile, recipedir, filesdir = self._setup_test_devtool_finish_modify()
-        # Ensure the recipe is where we think it should be (so that cleanup doesn't trash things)
-        self.assertIn('/meta/', recipedir)
-        # Try finish to the original layer
-        self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
-        result = self.runCmd('devtool finish %s meta' % recipe)
-        result = self.runCmd('devtool status')
-        self.assertNotIn(recipe, result.output, 'Recipe should have been reset by finish but wasn\'t')
-        self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after finish')
-        expected_status = [(' M', '.*/%s$' % os.path.basename(oldrecipefile)),
-                           ('??', '.*/.*-Add-a-comment-to-the-code.patch$')]
-        self._check_repo_status(recipedir, expected_status)
-
-    @OETestID(1622)
-    def test_devtool_finish_modify_otherlayer(self):
-        recipe, oldrecipefile, recipedir, filesdir = self._setup_test_devtool_finish_modify()
-        # Ensure the recipe is where we think it should be (so that cleanup doesn't trash things)
-        self.assertIn('/meta/', recipedir)
-        relpth = os.path.relpath(recipedir, os.path.join(self.get_bb_var('COREBASE'), 'meta'))
-        appenddir = os.path.join(self.testlayer_path, relpth)
-        self.track_for_cleanup(appenddir)
-        # Try finish to the original layer
-        self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
-        result = self.runCmd('devtool finish %s meta-selftest' % recipe)
-        result = self.runCmd('devtool status')
-        self.assertNotIn(recipe, result.output, 'Recipe should have been reset by finish but wasn\'t')
-        self.assertNotExists(os.path.join(self.workspacedir, 'recipes', recipe), 'Recipe directory should not exist after finish')
-        result = self.runCmd('git status --porcelain .', cwd=recipedir)
-        if result.output.strip():
-            self.fail('Recipe directory for %s contains the following unexpected changes after finish:\n%s' % (recipe, result.output.strip()))
-        recipefn = os.path.splitext(os.path.basename(oldrecipefile))[0]
-        recipefn = recipefn.split('_')[0] + '_%'
-        appendfile = os.path.join(appenddir, recipefn + '.bbappend')
-        self.assertExists(appendfile, 'bbappend %s should have been created but wasn\'t' % appendfile)
-        newdir = os.path.join(appenddir, recipe)
-        files = os.listdir(newdir)
-        foundpatch = None
-        for fn in files:
-            if fnmatch.fnmatch(fn, '*-Add-a-comment-to-the-code.patch'):
-                foundpatch = fn
-        if not foundpatch:
-            self.fail('No patch file created next to bbappend')
-        files.remove(foundpatch)
-        if files:
-            self.fail('Unexpected file(s) copied next to bbappend: %s' % ', '.join(files))
-
     @OETestID(1626)
     def test_devtool_rename(self):
         # Check preconditions
meta/lib/oeqa/selftest/cases/devtool_end.py new file +506 -0

506 lines not on this page · raw message

Signature · 1 line
2.11.0
#45 lore raw

PATCH 27/30 seltest/cases/devtool: Build dbus on test_devtool_add_git_local

In reply to Aníbal Limón

The dbus-wait recipe has the dependency of dbus, due to now we have build folder per test class the dependency needs to be build before run devtool add because without it the DEPENDS field is unset.

The devtool/recipetool uses previously build recipes to figure out the dependencies.

---
 meta/lib/oeqa/selftest/cases/devtool.py | 2 ++
 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/cases/devtool.py +2 -0
index 44ba361f16c..c5ca95022bb 100644
@@ -248,6 +248,8 @@ class DevtoolTests(DevtoolCommon):
         # Test devtool add
         self.track_for_cleanup(self.workspacedir)
         self.add_command_to_tearDown('bitbake-layers remove-layer %s' % self.workspacedir)
+        # dbus needs to be built because is a dependency of dbus-wait
+        self.bitbake('dbus')
         # Don't specify a name since we should be able to auto-detect it
         result = self.runCmd('devtool add %s' % srcdir)
         self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created')
Signature · 1 line
2.11.0
#46 lore raw

PATCH 28/30 argparse_oe: Add int_positive type

In reply to Aníbal Limón

Sometimes only expect positive values from cmdline so it's better to filter at parsing cmdline step instead of validate later.

---
 scripts/lib/argparse_oe.py | 7 +++++++
 1 file changed, 7 insertions(+)
scripts/lib/argparse_oe.py +7 -0
index bf6eb17197b..9bdfc1ceca2 100644
@@ -167,3 +167,10 @@ class OeHelpFormatter(argparse.HelpFormatter):
             return '\n'.join(lines)
         else:
             return super(OeHelpFormatter, self)._format_action(action)
+
+def int_positive(value):
+    ivalue = int(value)
+    if ivalue <= 0:
+        raise argparse.ArgumentTypeError(
+                "%s is not a positive int value" % value)
+    return ivalue
Signature · 1 line
2.11.0
#49 lore raw

PATCH 29/30 oeqa/selftest/context: Enable support for threaded runs

In reply to Aníbal Limón

Add an option to specify how many threads will be used for execution, default to 1.

If the thread_num are greater than 1 the OESelftestContextThreaded will be used, this is due to compatibility reasons.

---
 meta/lib/oeqa/selftest/context.py | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/context.py +22 -4
index 697ea0b4933..e54c871596d 100644
@@ -11,11 +11,12 @@ from shutil import copyfile
 import tempfile
 from random import choice
 
-import oeqa
+from argparse_oe import int_positive
 
-from oeqa.core.context import OETestContext, OETestContextExecutor
+import oeqa
+from oeqa.core.context import OETestContext, OETestContextExecutor 
 from oeqa.core.exception import OEQAPreRun
-
+from oeqa.core.threaded import OETestContextThreaded
 from oeqa.utils.commands import runCmd, get_bb_vars, get_test_layer
 
 class OESelftestTestContext(OETestContext):
@@ -38,14 +39,19 @@ class OESelftestTestContext(OETestContext):
     def listTests(self, display_type, machine=None):
         return super(OESelftestTestContext, self).listTests(display_type)
 
+class OESelftestTestContextThreaded(OESelftestTestContext, OETestContextThreaded):
+    pass
+
 class OESelftestTestContextExecutor(OETestContextExecutor):
-    _context_class = OESelftestTestContext
+    _context_class = OESelftestTestContextThreaded
     _script_executor = 'oe-selftest'
 
     name = 'oe-selftest'
     help = 'oe-selftest test component'
     description = 'Executes selftest tests'
 
+    DEFAULT_THREADS = 1
+
     def register_commands(self, logger, parser):
         group = parser.add_mutually_exclusive_group(required=True)
 
@@ -66,6 +72,11 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
                 action="store_true", default=False,
                 help='List all available tests.')
 
+        parser.add_argument('-t', '--thread-num', required=False, action='store',
+                dest="thread_num", default=self.DEFAULT_THREADS, type=int_positive,
+                help='Number of threads to use for execute selftests,'\
+                       ' default: %d' % self.DEFAULT_THREADS)
+
         parser.add_argument('--machine', required=False, choices=['random', 'all'],
                             help='Run tests on different machines (random/all).')
         
@@ -137,6 +148,8 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         self.tc_kwargs['init']['config_paths']['base_builddir'] = \
                 tempfile.mkdtemp(prefix='build-selftest-', dir=builddir)
 
+        self.tc_kwargs['load']['process_num'] = args.thread_num
+
     def _pre_run(self):
         def _check_required_env_variables(vars):
             for var in vars:
@@ -199,6 +212,11 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
         self.module_paths = self._get_cases_paths(
                 self.tc_kwargs['init']['td']['BBPATH'].split(':'))
 
+        if self.tc_kwargs['load']['process_num'] == 1:
+            self._context_class = OESelftestTestContext
+            # OESelftestTestContext class doesn't expect process_num
+            del self.tc_kwargs['load']['process_num']
+
         self.tc = self._context_class(**self.tc_kwargs['init'])
         self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
 
Signature · 1 line
2.11.0
#50 lore raw

PATCH 30/30 oeqa/selftest/cases: systemd_boot enable threaded runs

In reply to Aníbal Limón
---
 .../lib/oeqa/selftest/cases/systemd_boot.py        | 26 ++++++++++++----------
 1 file changed, 14 insertions(+), 12 deletions(-)
meta-yocto-bsp/lib/oeqa/selftest/cases/systemd_boot.py +14 -12
index dd5eeec1633..0bd52dc6b85 100644
@@ -3,9 +3,11 @@ import os
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.core.decorator.oeid import OETestID
 from oeqa.core.decorator.depends import OETestDepends
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
 
 class Systemdboot(OESelftestTestCase):
+    _use_own_builddir = True
+    _main_thread = False
+
     def _common_setup(self):
         """
         Common setup for test cases: 1445, 1528
@@ -22,7 +24,7 @@ class Systemdboot(OESelftestTestCase):
         """
 
         # Build a genericx86-64/efi systemdboot image
-        bitbake('mtools-native core-image-minimal')
+        self.bitbake('mtools-native core-image-minimal')
 
 
     @OETestID(1445)
@@ -38,14 +40,14 @@ class Systemdboot(OESelftestTestCase):
 
         # We'd use DEPLOY_DIR_IMAGE here, except that we need its value for
         # MACHINE="genericx86-64 which is probably not the one configured
-        systemdbootfile = os.path.join(get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64', 'systemd-bootx64.efi')
+        systemdbootfile = os.path.join(self.get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64', 'systemd-bootx64.efi')
 
         self._common_setup()
 
         # Ensure we're actually testing that this gets built and not that
         # it was around from an earlier build
-        bitbake('-c cleansstate systemd-boot')
-        runCmd('rm -f %s' % systemdbootfile)
+        self.bitbake('-c cleansstate systemd-boot')
+        self.runCmd('rm -f %s' % systemdbootfile)
 
         self._common_build()
 
@@ -71,20 +73,20 @@ class Systemdboot(OESelftestTestCase):
         AutomatedBy:  Jose Perez Carranza <jose.perez.carranza at linux-intel.com>
         """
 
-        systemdbootfile = os.path.join(get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64',
+        systemdbootfile = os.path.join(self.get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64',
                                            'systemd-bootx64.efi')
-        systemdbootimage = os.path.join(get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64',
+        systemdbootimage = os.path.join(self.get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64',
                                                 'core-image-minimal-genericx86-64.hddimg')
-        imagebootfile = os.path.join(get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64',
+        imagebootfile = os.path.join(self.get_bb_var('DEPLOY_DIR'), 'images', 'genericx86-64',
                                                             'bootx64.efi')
-        mcopynative = os.path.join(get_bb_var('STAGING_BINDIR_NATIVE'), 'mcopy')
+        mcopynative = os.path.join(self.get_bb_var('STAGING_BINDIR_NATIVE'), 'mcopy')
 
         #Clean environment before start the test
         if os.path.isfile(imagebootfile):
-            runCmd('rm -f %s' % imagebootfile)
+            self.runCmd('rm -f %s' % imagebootfile)
 
             #Step 1
-            runCmd('%s -i %s ::EFI/BOOT/bootx64.efi %s' % (mcopynative ,systemdbootimage,
+            self.runCmd('%s -i %s ::EFI/BOOT/bootx64.efi %s' % (mcopynative ,systemdbootimage,
                                                            imagebootfile))
 
             #Step 2
@@ -93,6 +95,6 @@ class Systemdboot(OESelftestTestCase):
                             % imagebootfile)
 
             #Step 3
-            result = runCmd('md5sum %s %s' % (systemdbootfile, imagebootfile))
+            result = self.runCmd('md5sum %s %s' % (systemdbootfile, imagebootfile))
             self.assertEqual(result.output.split()[0], result.output.split()[2],
                              '%s was not correclty generated' % imagebootfile)
Signature · 1 line
2.11.0
#51 lore raw

✗ patchtest: failure for oeqa core and oe-selftest threaded enablement

In reply to Aníbal Limón

== Series Details ==

Series: oeqa core and oe-selftest threaded enablement
Revision: 1
URL : https://patchwork.openembedded.org/series/7694/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at b1c4661742)

* Issue Series sent to the wrong mailing list or some patches from the series correspond to different mailing lists [test_target_mailing_list]

  Suggested fix    Send the series again to the correct mailing list (ML)
  Suggested ML     [email protected] [http://git.yoctoproject.org/cgit/cgit.cgi/poky/]
  Patch's path:    meta-yocto-bsp/lib/oeqa/selftest/cases/systemd_boot.py

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
#11 lore raw

Re: [PATCH 03/30] selftest/cases/package: Call parent setUpClass method

In reply to Aníbal Limón

meta/lib/oeqa/selftest/cases/package.py class VersionOrdering(OESelftestTestCase): · line 21

Quoted 3 lines · Aníbal Limón

def setUpClass(cls):

meta/lib/oeqa/selftest/cases/package.py +2 -0
+        super(VersionOrdering, cls).setUpClass()
+

In Python3 one can simply use super().setUpClass().

Not sure whether that's worth a V2 by itself. There's also plenty of other code outside of this patch which could be simplified like this.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#31 lore raw

Re: [PATCH 18/30] oeqa/selftest/cases: tinfoil to run in the main thread

In reply to Aníbal Limón
Quoted 19 lines · Aníbal Limón

The tinfoil tests needs to initialize bitbake internals and wasn't designed to be in a thread environment causing problems when try for example set signal handlers in a thread different than the main one.

To workaround this execute tinfoil tests in the main thread and don't use custom builddir because isn't make sense too.

---
 meta/lib/oeqa/selftest/cases/tinfoil.py | 1 -
 1 file changed, 1 deletion(-)
meta/lib/oeqa/selftest/cases/tinfoil.py +0 -0
index aa1af7e0423..7248b755812 100644

meta/lib/oeqa/selftest/cases/tinfoil.py from oeqa.core.decorator.oeid import OETestID · line 14

Quoted 7 lines
meta/lib/oeqa/selftest/cases/tinfoil.py +0 -1
@@ -9,7 +9,6 @@ from oeqa.core.decorator.oeid import OETestID
 
 class TinfoilTests(OESelftestTestCase):
     """ Basic tests for the tinfoil API """
-
     @OETestID(1568)
     def test_getvar(self):
         with bb.tinfoil.Tinfoil() as tinfoil:

There's just a line removal in this patch. That's probably not what was intended?

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#35 lore raw

Re: [PATCH 20/30] oeqa/selftest/cases: runqemu enable thraded runs

In reply to Aníbal Limón

s/thraded/threaded/ in the subject. There are more spelling mistakes elsewhere ("wrapper methos"). I don't know how important that is.

Quoted 1 line · Aníbal Limón

- Update to use wrappers {bitbake,get_bb_var} from OESelfTestCase class.

s/OESelfTestCase/OESelftestTestCase/

Sorry to be pedantic, but I wanted to look up what these wrappers do and couldn't even find the class ;-}

I've found them in the "oeqa/selftest/case: Add wrappers to utils.commands modules" patch. I'm a bit worried that this entire patch series is introducing concepts and methods without any documentation or explanations why things are done this way. I suspect it will make it very hard to write selftests correctly.

For example, this patch and others like it seem fairly arbitrary. It doesn't explain why self.bitbake() is better than bitbake(). If in some future patch or test someone were to use bitbake() when they should have used self.bitbake() it's not going to be obvious either whether that is correct.

Perhaps all OE tests should have these wrappers and only OESelftestTestCase does something special with them? Then we can gradually replace the direct calls to oeqa.utils.commands completely.

Add also a wrapper for runqemu and we can get rid of "from oeqa.utils.commands import" completely.

Quoted 36 lines

- Run into the main thread because it needs tinfoil to run.

---
 meta/lib/oeqa/selftest/cases/runqemu.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/cases/runqemu.py +3 -4
index 4050a4123ba..e30cb24046f 100644
@@ -6,12 +6,11 @@ import re
 import logging
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, runqemu, get_bb_var
+from oeqa.utils.commands import runqemu
 from oeqa.core.decorator.oeid import OETestID
 
 class RunqemuTests(OESelftestTestCase):
     """Runqemu test class"""
-
     image_is_ready = False
     deploy_dir_image = ''
 
@@ -37,8 +36,8 @@ SYSLINUX_TIMEOUT = "10"
         )
 
         if not RunqemuTests.image_is_ready:
-            RunqemuTests.deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
-            bitbake(self.recipe)
+            RunqemuTests.deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
+            self.bitbake(self.recipe)
             RunqemuTests.image_is_ready = True
 
     @OETestID(2001)
Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#8 lore raw

Re: [PATCH 02/30] oeqa/selftest/{context, case}: Handle KeyboardInterrupt/SIGINT and SIGTERM

In reply to Aníbal Limón

I saw a similar patch in the past. Is this a new revision?

Leo

Quoted 225 lines · Aníbal Limón

In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration.

[YOCTO #11650]

---
 meta/lib/oeqa/selftest/case.py    |  36 +++++++------
 meta/lib/oeqa/selftest/context.py | 107 +++++++++++++++++++++++++++-----------
 2 files changed, 97 insertions(+), 46 deletions(-)
meta/lib/oeqa/selftest/case.py +21 -15
index 31a11fddda9..871009c568b 100644
@@ -13,28 +13,34 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.core.case import OETestCase
 
 class OESelftestTestCase(OETestCase):
-    builddir = os.environ.get("BUILDDIR") or ""
-    localconf_path = os.path.join(builddir, "conf/local.conf")
-    localconf_backup = os.path.join(builddir, "conf/local.bk")
-    testinc_path = os.path.join(builddir, "conf/selftest.inc")
-    local_bblayers_path = os.path.join(builddir, "conf/bblayers.conf")
-    local_bblayers_backup = os.path.join(builddir, "conf/bblayers.bk")
-    testinc_bblayers_path = os.path.join(builddir, "conf/bblayers.inc")
-    machineinc_path = os.path.join(builddir, "conf/machine.inc")
-
     def __init__(self, methodName="runTest"):
         self._extra_tear_down_commands = []
-        self._track_for_cleanup = [
-            self.testinc_path, self.testinc_bblayers_path,
-            self.machineinc_path, self.localconf_backup,
-            self.local_bblayers_backup]
-
         super(OESelftestTestCase, self).__init__(methodName)
 
     @classmethod
     def setUpClass(cls):
         super(OESelftestTestCase, cls).setUpClass()
-        cls.testlayer_path = cls.tc.testlayer_path
+
+        cls.testlayer_path = cls.tc.config_paths['testlayer_path']
+        cls.builddir = cls.tc.config_paths['builddir']
+
+        cls.localconf_path = cls.tc.config_paths['localconf']
+        cls.localconf_backup = cls.tc.config_paths['localconf_class_backup']
+        cls.local_bblayers_path = cls.tc.config_paths['bblayers']
+        cls.local_bblayers_backup = cls.tc.config_paths['bblayers_class_backup']
+
+        cls.testinc_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/selftest.inc")
+        cls.testinc_bblayers_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/bblayers.inc")
+        cls.machineinc_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/machine.inc")
+
+        cls._track_for_cleanup = [
+            cls.testinc_path, cls.testinc_bblayers_path,
+            cls.machineinc_path, cls.localconf_backup,
+            cls.local_bblayers_backup]
+
         cls.add_include()
 
     @classmethod
meta/lib/oeqa/selftest/context.py +76 -31
index ca87398224c..4575a0537fb 100644
@@ -6,6 +6,8 @@ import time
 import glob
 import sys
 import imp
+import signal
+from shutil import copyfile
 from random import choice
 
 import oeqa
@@ -16,13 +18,12 @@ from oeqa.core.exception import OEQAPreRun
 from oeqa.utils.commands import runCmd, get_bb_vars, get_test_layer
 
 class OESelftestTestContext(OETestContext):
-    def __init__(self, td=None, logger=None, machines=None, testlayer_path=None):
+    def __init__(self, td=None, logger=None, machines=None, config_paths=None):
         super(OESelftestTestContext, self).__init__(td, logger)
 
         self.machines = machines
         self.custommachine = None
-
-        self.testlayer_path = testlayer_path
+        self.config_paths = config_paths
 
     def runTests(self, machine=None):
         if machine:
@@ -108,7 +109,29 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 
         self.tc_kwargs['init']['td'] = get_bb_vars()
         self.tc_kwargs['init']['machines'] = self._get_available_machines()
-        self.tc_kwargs['init']['testlayer_path'] = get_test_layer()
+
+        builddir = os.environ.get("BUILDDIR")
+        self.tc_kwargs['init']['config_paths'] = {}
+        self.tc_kwargs['init']['config_paths']['testlayer_path'] = \
+                get_test_layer()
+        self.tc_kwargs['init']['config_paths']['builddir'] = builddir
+        self.tc_kwargs['init']['config_paths']['localconf'] = \
+                os.path.join(builddir, "conf/local.conf")
+        self.tc_kwargs['init']['config_paths']['localconf_backup'] = \
+                os.path.join(builddir, "conf/local.conf.orig")
+        self.tc_kwargs['init']['config_paths']['localconf_class_backup'] = \
+                os.path.join(builddir, "conf/local.conf.bk")
+        self.tc_kwargs['init']['config_paths']['bblayers'] = \
+                os.path.join(builddir, "conf/bblayers.conf")
+        self.tc_kwargs['init']['config_paths']['bblayers_backup'] = \
+                os.path.join(builddir, "conf/bblayers.conf.orig")
+        self.tc_kwargs['init']['config_paths']['bblayers_class_backup'] = \
+                os.path.join(builddir, "conf/bblayers.conf.bk")
+
+        copyfile(self.tc_kwargs['init']['config_paths']['localconf'],
+                self.tc_kwargs['init']['config_paths']['localconf_backup'])
+        copyfile(self.tc_kwargs['init']['config_paths']['bblayers'], 
+                self.tc_kwargs['init']['config_paths']['bblayers_backup'])
 
     def _pre_run(self):
         def _check_required_env_variables(vars):
@@ -131,7 +154,7 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
                     runCmd("bitbake-layers add-layer %s" %meta_selftestdir)
                     # reload data is needed because a meta-selftest layer was add
                     self.tc.td = get_bb_vars()
-                    self.tc.testlayer_path = get_test_layer()
+                    self.tc.config_paths['testlayer_path'] = get_test_layer()
                 else:
                     self.tc.logger.error("could not locate meta-selftest in:\n%s" % meta_selftestdir)
                     raise OEQAPreRun
@@ -184,41 +207,63 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
             rc.logSummary(self.name)
 
         return rc
+
+    def _signal_clean_handler(self, signum, frame):
+        sys.exit(1)
 
     def run(self, logger, args):
         self._process_args(logger, args)
-        rc = None
 
-        if args.machine:
-            logger.info('Custom machine mode enabled. MACHINE set to %s' %
-                    args.machine)
+        signal.signal(signal.SIGTERM, self._signal_clean_handler)
 
-            if args.machine == 'all':
-                results = []
-                for m in self.tc_kwargs['init']['machines']:
-                    self.tc_kwargs['run']['machine'] = m
-                    results.append(self._internal_run(logger, args))
+        rc = None
+        try:
+            if args.machine:
+                logger.info('Custom machine mode enabled. MACHINE set to %s' %
+                        args.machine)
+
+                if args.machine == 'all':
+                    results = []
+                    for m in self.tc_kwargs['init']['machines']:
+                        self.tc_kwargs['run']['machine'] = m
+                        results.append(self._internal_run(logger, args))
+
+                        # XXX: the oe-selftest script only needs to know if one
+                        # machine run fails
+                        for r in results:
+                            rc = r
+                            if not r.wasSuccessful():
+                                break
 
-                    # XXX: the oe-selftest script only needs to know if one
-                    # machine run fails
-                    for r in results:
-                        rc = r
-                        if not r.wasSuccessful():
-                            break
+                else:
+                    self.tc_kwargs['run']['machine'] = args.machine
+                    return self._internal_run(logger, args)
 
             else:
                 self.tc_kwargs['run']['machine'] = args.machine
-                return self._internal_run(logger, args)
-
-        else:
-            self.tc_kwargs['run']['machine'] = args.machine
-            rc = self._internal_run(logger, args)
-
-        output_link = os.path.join(os.path.dirname(args.output_log),
-                "%s-results.log" % self.name)
-        if os.path.exists(output_link):
-            os.remove(output_link)
-        os.symlink(args.output_log, output_link)
+                rc = self._internal_run(logger, args)
+        finally:
+            config_paths = self.tc_kwargs['init']['config_paths']
+            if os.path.exists(config_paths['localconf_backup']):
+                copyfile(config_paths['localconf_backup'],
+                        config_paths['localconf'])
+                os.remove(config_paths['localconf_backup'])
+
+            if os.path.exists(config_paths['bblayers_backup']):
+                copyfile(config_paths['bblayers_backup'], 
+                        config_paths['bblayers'])
+                os.remove(config_paths['bblayers_backup'])
+
+            if os.path.exists(config_paths['localconf_class_backup']):
+                os.remove(config_paths['localconf_class_backup'])
+            if os.path.exists(config_paths['bblayers_class_backup']):
+                os.remove(config_paths['bblayers_class_backup'])
+
+            output_link = os.path.join(os.path.dirname(args.output_log),
+                    "%s-results.log" % self.name)
+            if os.path.exists(output_link):
+                os.remove(output_link)
+            os.symlink(args.output_log, output_link)
 
         return rc
 
#18 lore raw

Re: [PATCH 07/30] oeqa/core/threaded: Add support to run into a thread at end of execution

In reply to Aníbal Limón
Quoted 3 lines · Aníbal Limón

Some test cases aren't allowed to run into a multi-thread environment so add the posibility to run those tests at end of execution.

Hi Anibal,

which is the reason these need to run at the end?

Quoted 194 lines
---
 meta/lib/oeqa/core/threaded.py | 102 +++++++++++++++++++++++++++++++----------
 1 file changed, 78 insertions(+), 24 deletions(-)
meta/lib/oeqa/core/threaded.py +78 -24
index 34217f1a8b8..a7dc0aed401 100644
@@ -30,18 +30,35 @@ class OETestLoaderThreaded(OETestLoader):
         suites = {}
         suites['main'] = self.suiteClass()
         suites['pool'] = []
+        suites['end'] = self.suiteClass()
         for _ in range(self.process_num - 1):
             suites['pool'].append(self.suiteClass())
 
+        def _add_by_module_or_dep(suite, case, depends):
+            """
+                A test case that needs to run into the same thread
+                because is on the same module or for dependency
+                reasons.
+            """
+
+            for c in suite._tests:
+                if case.__module__ == c.__module__:
+                    suite.addTest(case)
+                    return True
+
+            if case.id() in depends:
+                case_depends = depends[case.id()]
+                for c in suite._tests:
+                    if c.id() in case_depends:
+                        suite.addTest(case)
+                        return True
+
+            return False
+
         def _add_to_main_thread(main_suite, case, depends):
             """
                 Some test cases needs to be run into the main
-                thread for several resons.
-
-                A test case that needs to run in the main thread
-                can be for specific set via test class _main_thread
-                attr or because is on the same module or for a dependency
-                reason.
+                thread by request.
             """
 
             if hasattr(case.__class__, '_main_thread') and \
@@ -50,19 +67,20 @@ class OETestLoaderThreaded(OETestLoader):
                 main_suite.addTest(case)
                 return True
 
-            for c in main_suite._tests:
-                if case.__module__ == c.__module__:
-                    main_suite.addTest(case)
-                    return True
+            return _add_by_module_or_dep(main_suite, case, depends)
 
-            if case.id() in depends:
-                case_depends = depends[case.id()]
-                for c in main_suite._tests:
-                    if c.id() in case_depends:
-                        main_suite.addTest(case)
-                        return True
+        def _add_to_end_thread(end_suite, case, depends):
+            """
+                Some test cases needs to be run into at end of
+                execution into the main by request.
+            """
+            if hasattr(case.__class__, '_end_thread') and \
+                    case.__class__._end_thread or \
+                    self.process_num == 1:
+                end_suite.addTest(case)
+                return True
 
-            return False
+            return _add_by_module_or_dep(end_suite, case, depends)
 
         def _search_for_module_idx(suites, case):
             """
@@ -112,6 +130,9 @@ class OETestLoaderThreaded(OETestLoader):
                     if 'depends' in self.tc._registry:
                         depends = self.tc._registry['depends']
 
+                    if _add_to_end_thread(suites['end'], case, depends):
+                        continue
+
                     if _add_to_main_thread(suites['main'], case, depends):
                         continue
 
@@ -135,7 +156,7 @@ class OETestLoaderThreaded(OETestLoader):
 
         # if the main suite doesn't have test cases
         # use the first element of the suites pool
-        if not len(suites['main']._tests):
+        if not len(suites['main']._tests) and len(suites['pool']):
             suites['main'] = suites['pool'].pop(0)
 
         return suites
@@ -268,6 +289,12 @@ class _ThreadedPool:
         self.tasks = queue.Queue(num_tasks)
         self.workers = []
 
+        self.stream = stream
+        self.result = result
+
+        self.end_task = None
+        self.end_worker = None
+
         for _ in range(num_workers):
             worker = _Worker(self.tasks, result, stream)
             self.workers.append(worker)
@@ -280,12 +307,25 @@ class _ThreadedPool:
         """Add a task to the queue"""
         self.tasks.put((func, args, kargs))
 
+    def add_end_task(self, func, *args, **kwargs):
+        """Add a task to be executed at end"""
+
+        self.end_task = queue.Queue(1)
+        self.end_task.put((func, args, kwargs))
+        self.end_worker = _Worker(self.end_task, self.result,
+                self.stream)
+
     def wait_completion(self):
         """Wait for completion of all the tasks in the queue"""
         self.tasks.join()
         for worker in self.workers:
             worker.join()
 
+        if self.end_task:
+            self.end_worker.start()
+            self.end_task.join()
+            self.end_worker.join()
+
 class OETestRunnerThreaded(OETestRunner):
     streamLoggerClass = OEStreamLoggerThreaded
 
@@ -293,32 +333,46 @@ class OETestRunnerThreaded(OETestRunner):
         super(OETestRunnerThreaded, self).__init__(tc, *args, **kwargs)
         self.resultclass = OETestResultThreadedInternal # XXX: XML reporting overrides at __init__
 
+    def _run_main_thread(self, suite, result):
+        if len(suite._tests):
+            run_start_time = time.time()
+            rc = super(OETestRunnerThreaded, self).run(suite)
+            run_end_time = time.time()
+            result.addResult(rc, run_start_time, run_end_time)
+            self.stream.finish()
+
     def run(self, suites):
         result = OETestResultThreaded(self.tc)
 
         pool = None
+
         if suites['pool']:
             thread_no = len(suites['pool'])
             pool = _ThreadedPool(thread_no, thread_no, stream=self.stream,
                     result=result)
             for s in suites['pool']:
                 pool.add_task(super(OETestRunnerThreaded, self).run, s)
-            pool.start()
 
-        run_start_time = time.time()
-        rc = super(OETestRunnerThreaded, self).run(suites['main'])
-        run_end_time = time.time()
-        result.addResult(rc, run_start_time, run_end_time)
-        self.stream.finish()
+        if len(suites['end']._tests):
+            if not pool:
+                pool = _ThreadedPool(0, 0, stream=self.stream,
+                            result=result)
+            pool.add_end_task(super(OETestRunnerThreaded, self).run,
+                    suites['end'])
 
         if pool:
+            pool.start()
+        self._run_main_thread(suites['main'], result)
+        if pool:
             pool.wait_completion()
+
         result._fill_tc_results()
 
         return result
 
     def list_tests(self, suite, display_type):
         suite['pool'].insert(0, suite['main'])
+        suite['pool'].append(suite['end'])
 
         return super(OETestRunnerThreaded, self).list_tests(
                 suite['pool'], display_type)
#47 lore raw

Re: [PATCH 28/30] argparse_oe: Add int_positive type

In reply to Aníbal Limón
Quoted 13 lines · Aníbal Limón

Sometimes only expect positive values from cmdline so it's better to filter at parsing cmdline step instead of validate later.

---
 scripts/lib/argparse_oe.py | 7 +++++++
 1 file changed, 7 insertions(+)
scripts/lib/argparse_oe.py +0 -0
index bf6eb17197b..9bdfc1ceca2 100644

scripts/lib/argparse_oe.py class OeHelpFormatter(argparse.HelpFormatter): · line 170

Quoted 4 lines
scripts/lib/argparse_oe.py +1 -0
@@ -167,3 +167,10 @@ class OeHelpFormatter(argparse.HelpFormatter):
             return '\n'.join(lines)
         else:
             return super(OeHelpFormatter, self)._format_action(action)
+

when adding an argument into a script, one can define the type expected so and the library itself does what you are doing for free, so not sure if this is needed.

Quoted 6 lines
scripts/lib/argparse_oe.py +6 -0
+def int_positive(value):
+    ivalue = int(value)
+    if ivalue <= 0:
+        raise argparse.ArgumentTypeError(
+                "%s is not a positive int value" % value)
+    return ivalue
#9 lore raw

Re: [PATCH 02/30] oeqa/selftest/{context, case}: Handle KeyboardInterrupt/SIGINT and SIGTERM

In reply to Leonardo Sandoval

Yes is the same,

Anibal

Quoted 233 lines · Leonardo Sandoval

I saw a similar patch in the past. Is this a new revision?

Leo

Quoted 225 lines · Aníbal Limón

In order to avoid corrupt local.conf and bblayers.conf adds signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block to restore previously backuped configuration.

[YOCTO #11650]

---
 meta/lib/oeqa/selftest/case.py    |  36 +++++++------
 meta/lib/oeqa/selftest/context.py | 107 +++++++++++++++++++++++++++-----------
 2 files changed, 97 insertions(+), 46 deletions(-)
meta/lib/oeqa/selftest/case.py +21 -15
index 31a11fddda9..871009c568b 100644
@@ -13,28 +13,34 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var
 from oeqa.core.case import OETestCase
 
 class OESelftestTestCase(OETestCase):
-    builddir = os.environ.get("BUILDDIR") or ""
-    localconf_path = os.path.join(builddir, "conf/local.conf")
-    localconf_backup = os.path.join(builddir, "conf/local.bk")
-    testinc_path = os.path.join(builddir, "conf/selftest.inc")
-    local_bblayers_path = os.path.join(builddir, "conf/bblayers.conf")
-    local_bblayers_backup = os.path.join(builddir, "conf/bblayers.bk")
-    testinc_bblayers_path = os.path.join(builddir, "conf/bblayers.inc")
-    machineinc_path = os.path.join(builddir, "conf/machine.inc")
-
     def __init__(self, methodName="runTest"):
         self._extra_tear_down_commands = []
-        self._track_for_cleanup = [
-            self.testinc_path, self.testinc_bblayers_path,
-            self.machineinc_path, self.localconf_backup,
-            self.local_bblayers_backup]
-
         super(OESelftestTestCase, self).__init__(methodName)
 
     @classmethod
     def setUpClass(cls):
         super(OESelftestTestCase, cls).setUpClass()
-        cls.testlayer_path = cls.tc.testlayer_path
+
+        cls.testlayer_path = cls.tc.config_paths['testlayer_path']
+        cls.builddir = cls.tc.config_paths['builddir']
+
+        cls.localconf_path = cls.tc.config_paths['localconf']
+        cls.localconf_backup = cls.tc.config_paths['localconf_class_backup']
+        cls.local_bblayers_path = cls.tc.config_paths['bblayers']
+        cls.local_bblayers_backup = cls.tc.config_paths['bblayers_class_backup']
+
+        cls.testinc_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/selftest.inc")
+        cls.testinc_bblayers_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/bblayers.inc")
+        cls.machineinc_path = os.path.join(cls.tc.config_paths['builddir'],
+                "conf/machine.inc")
+
+        cls._track_for_cleanup = [
+            cls.testinc_path, cls.testinc_bblayers_path,
+            cls.machineinc_path, cls.localconf_backup,
+            cls.local_bblayers_backup]
+
         cls.add_include()
 
     @classmethod
meta/lib/oeqa/selftest/context.py +76 -31
index ca87398224c..4575a0537fb 100644
@@ -6,6 +6,8 @@ import time
 import glob
 import sys
 import imp
+import signal
+from shutil import copyfile
 from random import choice
 
 import oeqa
@@ -16,13 +18,12 @@ from oeqa.core.exception import OEQAPreRun
 from oeqa.utils.commands import runCmd, get_bb_vars, get_test_layer
 
 class OESelftestTestContext(OETestContext):
-    def __init__(self, td=None, logger=None, machines=None, testlayer_path=None):
+    def __init__(self, td=None, logger=None, machines=None, config_paths=None):
         super(OESelftestTestContext, self).__init__(td, logger)
 
         self.machines = machines
         self.custommachine = None
-
-        self.testlayer_path = testlayer_path
+        self.config_paths = config_paths
 
     def runTests(self, machine=None):
         if machine:
@@ -108,7 +109,29 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
 
         self.tc_kwargs['init']['td'] = get_bb_vars()
         self.tc_kwargs['init']['machines'] = self._get_available_machines()
-        self.tc_kwargs['init']['testlayer_path'] = get_test_layer()
+
+        builddir = os.environ.get("BUILDDIR")
+        self.tc_kwargs['init']['config_paths'] = {}
+        self.tc_kwargs['init']['config_paths']['testlayer_path'] = \
+                get_test_layer()
+        self.tc_kwargs['init']['config_paths']['builddir'] = builddir
+        self.tc_kwargs['init']['config_paths']['localconf'] = \
+                os.path.join(builddir, "conf/local.conf")
+        self.tc_kwargs['init']['config_paths']['localconf_backup'] = \
+                os.path.join(builddir, "conf/local.conf.orig")
+        self.tc_kwargs['init']['config_paths']['localconf_class_backup'] = \
+                os.path.join(builddir, "conf/local.conf.bk")
+        self.tc_kwargs['init']['config_paths']['bblayers'] = \
+                os.path.join(builddir, "conf/bblayers.conf")
+        self.tc_kwargs['init']['config_paths']['bblayers_backup'] = \
+                os.path.join(builddir, "conf/bblayers.conf.orig")
+        self.tc_kwargs['init']['config_paths']['bblayers_class_backup'] = \
+                os.path.join(builddir, "conf/bblayers.conf.bk")
+
+        copyfile(self.tc_kwargs['init']['config_paths']['localconf'],
+                self.tc_kwargs['init']['config_paths']['localconf_backup'])
+        copyfile(self.tc_kwargs['init']['config_paths']['bblayers'], 
+                self.tc_kwargs['init']['config_paths']['bblayers_backup'])
 
     def _pre_run(self):
         def _check_required_env_variables(vars):
@@ -131,7 +154,7 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
                     runCmd("bitbake-layers add-layer %s" %meta_selftestdir)
                     # reload data is needed because a meta-selftest layer was add
                     self.tc.td = get_bb_vars()
-                    self.tc.testlayer_path = get_test_layer()
+                    self.tc.config_paths['testlayer_path'] = get_test_layer()
                 else:
                     self.tc.logger.error("could not locate meta-selftest in:\n%s" % meta_selftestdir)
                     raise OEQAPreRun
@@ -184,41 +207,63 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
             rc.logSummary(self.name)
 
         return rc
+
+    def _signal_clean_handler(self, signum, frame):
+        sys.exit(1)
 
     def run(self, logger, args):
         self._process_args(logger, args)
-        rc = None
 
-        if args.machine:
-            logger.info('Custom machine mode enabled. MACHINE set to %s' %
-                    args.machine)
+        signal.signal(signal.SIGTERM, self._signal_clean_handler)
 
-            if args.machine == 'all':
-                results = []
-                for m in self.tc_kwargs['init']['machines']:
-                    self.tc_kwargs['run']['machine'] = m
-                    results.append(self._internal_run(logger, args))
+        rc = None
+        try:
+            if args.machine:
+                logger.info('Custom machine mode enabled. MACHINE set to %s' %
+                        args.machine)
+
+                if args.machine == 'all':
+                    results = []
+                    for m in self.tc_kwargs['init']['machines']:
+                        self.tc_kwargs['run']['machine'] = m
+                        results.append(self._internal_run(logger, args))
+
+                        # XXX: the oe-selftest script only needs to know if one
+                        # machine run fails
+                        for r in results:
+                            rc = r
+                            if not r.wasSuccessful():
+                                break
 
-                    # XXX: the oe-selftest script only needs to know if one
-                    # machine run fails
-                    for r in results:
-                        rc = r
-                        if not r.wasSuccessful():
-                            break
+                else:
+                    self.tc_kwargs['run']['machine'] = args.machine
+                    return self._internal_run(logger, args)
 
             else:
                 self.tc_kwargs['run']['machine'] = args.machine
-                return self._internal_run(logger, args)
-
-        else:
-            self.tc_kwargs['run']['machine'] = args.machine
-            rc = self._internal_run(logger, args)
-
-        output_link = os.path.join(os.path.dirname(args.output_log),
-                "%s-results.log" % self.name)
-        if os.path.exists(output_link):
-            os.remove(output_link)
-        os.symlink(args.output_log, output_link)
+                rc = self._internal_run(logger, args)
+        finally:
+            config_paths = self.tc_kwargs['init']['config_paths']
+            if os.path.exists(config_paths['localconf_backup']):
+                copyfile(config_paths['localconf_backup'],
+                        config_paths['localconf'])
+                os.remove(config_paths['localconf_backup'])
+
+            if os.path.exists(config_paths['bblayers_backup']):
+                copyfile(config_paths['bblayers_backup'], 
+                        config_paths['bblayers'])
+                os.remove(config_paths['bblayers_backup'])
+
+            if os.path.exists(config_paths['localconf_class_backup']):
+                os.remove(config_paths['localconf_class_backup'])
+            if os.path.exists(config_paths['bblayers_class_backup']):
+                os.remove(config_paths['bblayers_class_backup'])
+
+            output_link = os.path.join(os.path.dirname(args.output_log),
+                    "%s-results.log" % self.name)
+            if os.path.exists(output_link):
+                os.remove(output_link)
+            os.symlink(args.output_log, output_link)
 
         return rc
 
#19 lore raw

Re: [PATCH 07/30] oeqa/core/threaded: Add support to run into a thread at end of execution

In reply to Leonardo Sandoval
Quoted 9 lines · Leonardo Sandoval
Quoted 3 lines · Aníbal Limón

Some test cases aren't allowed to run into a multi-thread environment so add the posibility to run those tests at end of execution.

Hi Anibal,

which is the reason these need to run at the end?

Some devtool tests made modifications to the meta directory causing deterministic meta-data bitbake errors in other threads, see the patch devtool_end in this series.

Anibal

Quoted 198 lines
Quoted 196 lines
---
 meta/lib/oeqa/core/threaded.py | 102 +++++++++++++++++++++++++++++++----------
 1 file changed, 78 insertions(+), 24 deletions(-)
meta/lib/oeqa/core/threaded.py +78 -24
index 34217f1a8b8..a7dc0aed401 100644
@@ -30,18 +30,35 @@ class OETestLoaderThreaded(OETestLoader):
         suites = {}
         suites['main'] = self.suiteClass()
         suites['pool'] = []
+        suites['end'] = self.suiteClass()
         for _ in range(self.process_num - 1):
             suites['pool'].append(self.suiteClass())
 
+        def _add_by_module_or_dep(suite, case, depends):
+            """
+                A test case that needs to run into the same thread
+                because is on the same module or for dependency
+                reasons.
+            """
+
+            for c in suite._tests:
+                if case.__module__ == c.__module__:
+                    suite.addTest(case)
+                    return True
+
+            if case.id() in depends:
+                case_depends = depends[case.id()]
+                for c in suite._tests:
+                    if c.id() in case_depends:
+                        suite.addTest(case)
+                        return True
+
+            return False
+
         def _add_to_main_thread(main_suite, case, depends):
             """
                 Some test cases needs to be run into the main
-                thread for several resons.
-
-                A test case that needs to run in the main thread
-                can be for specific set via test class _main_thread
-                attr or because is on the same module or for a dependency
-                reason.
+                thread by request.
             """
 
             if hasattr(case.__class__, '_main_thread') and \
@@ -50,19 +67,20 @@ class OETestLoaderThreaded(OETestLoader):
                 main_suite.addTest(case)
                 return True
 
-            for c in main_suite._tests:
-                if case.__module__ == c.__module__:
-                    main_suite.addTest(case)
-                    return True
+            return _add_by_module_or_dep(main_suite, case, depends)
 
-            if case.id() in depends:
-                case_depends = depends[case.id()]
-                for c in main_suite._tests:
-                    if c.id() in case_depends:
-                        main_suite.addTest(case)
-                        return True
+        def _add_to_end_thread(end_suite, case, depends):
+            """
+                Some test cases needs to be run into at end of
+                execution into the main by request.
+            """
+            if hasattr(case.__class__, '_end_thread') and \
+                    case.__class__._end_thread or \
+                    self.process_num == 1:
+                end_suite.addTest(case)
+                return True
 
-            return False
+            return _add_by_module_or_dep(end_suite, case, depends)
 
         def _search_for_module_idx(suites, case):
             """
@@ -112,6 +130,9 @@ class OETestLoaderThreaded(OETestLoader):
                     if 'depends' in self.tc._registry:
                         depends = self.tc._registry['depends']
 
+                    if _add_to_end_thread(suites['end'], case, depends):
+                        continue
+
                     if _add_to_main_thread(suites['main'], case, depends):
                         continue
 
@@ -135,7 +156,7 @@ class OETestLoaderThreaded(OETestLoader):
 
         # if the main suite doesn't have test cases
         # use the first element of the suites pool
-        if not len(suites['main']._tests):
+        if not len(suites['main']._tests) and len(suites['pool']):
             suites['main'] = suites['pool'].pop(0)
 
         return suites
@@ -268,6 +289,12 @@ class _ThreadedPool:
         self.tasks = queue.Queue(num_tasks)
         self.workers = []
 
+        self.stream = stream
+        self.result = result
+
+        self.end_task = None
+        self.end_worker = None
+
         for _ in range(num_workers):
             worker = _Worker(self.tasks, result, stream)
             self.workers.append(worker)
@@ -280,12 +307,25 @@ class _ThreadedPool:
         """Add a task to the queue"""
         self.tasks.put((func, args, kargs))
 
+    def add_end_task(self, func, *args, **kwargs):
+        """Add a task to be executed at end"""
+
+        self.end_task = queue.Queue(1)
+        self.end_task.put((func, args, kwargs))
+        self.end_worker = _Worker(self.end_task, self.result,
+                self.stream)
+
     def wait_completion(self):
         """Wait for completion of all the tasks in the queue"""
         self.tasks.join()
         for worker in self.workers:
             worker.join()
 
+        if self.end_task:
+            self.end_worker.start()
+            self.end_task.join()
+            self.end_worker.join()
+
 class OETestRunnerThreaded(OETestRunner):
     streamLoggerClass = OEStreamLoggerThreaded
 
@@ -293,32 +333,46 @@ class OETestRunnerThreaded(OETestRunner):
         super(OETestRunnerThreaded, self).__init__(tc, *args, **kwargs)
         self.resultclass = OETestResultThreadedInternal # XXX: XML reporting overrides at __init__
 
+    def _run_main_thread(self, suite, result):
+        if len(suite._tests):
+            run_start_time = time.time()
+            rc = super(OETestRunnerThreaded, self).run(suite)
+            run_end_time = time.time()
+            result.addResult(rc, run_start_time, run_end_time)
+            self.stream.finish()
+
     def run(self, suites):
         result = OETestResultThreaded(self.tc)
 
         pool = None
+
         if suites['pool']:
             thread_no = len(suites['pool'])
             pool = _ThreadedPool(thread_no, thread_no, stream=self.stream,
                     result=result)
             for s in suites['pool']:
                 pool.add_task(super(OETestRunnerThreaded, self).run, s)
-            pool.start()
 
-        run_start_time = time.time()
-        rc = super(OETestRunnerThreaded, self).run(suites['main'])
-        run_end_time = time.time()
-        result.addResult(rc, run_start_time, run_end_time)
-        self.stream.finish()
+        if len(suites['end']._tests):
+            if not pool:
+                pool = _ThreadedPool(0, 0, stream=self.stream,
+                            result=result)
+            pool.add_end_task(super(OETestRunnerThreaded, self).run,
+                    suites['end'])
 
         if pool:
+            pool.start()
+        self._run_main_thread(suites['main'], result)
+        if pool:
             pool.wait_completion()
+
         result._fill_tc_results()
 
         return result
 
     def list_tests(self, suite, display_type):
         suite['pool'].insert(0, suite['main'])
+        suite['pool'].append(suite['end'])
 
         return super(OETestRunnerThreaded, self).list_tests(
                 suite['pool'], display_type)
#32 lore raw

Re: [PATCH 18/30] oeqa/selftest/cases: tinfoil to run in the main thread

In reply to Patrick Ohly
Quoted 21 lines · Patrick Ohly
Quoted 20 lines · Aníbal Limón

The tinfoil tests needs to initialize bitbake internals and wasn't designed to be in a thread environment causing problems when try for example set signal handlers in a thread different than the main one.

To workaround this execute tinfoil tests in the main thread and don't use custom builddir because isn't make sense too.

---
 meta/lib/oeqa/selftest/cases/tinfoil.py | 1 -
 1 file changed, 1 deletion(-)
meta/lib/oeqa/selftest/cases/tinfoil.py +0 -0
index aa1af7e0423..7248b755812 100644
@@ -9,7 +9,6 @@ from oeqa.core.decorator.oeid import OETestID
 
Quoted 9 lines

meta/lib/oeqa/selftest/cases/tinfoil.py from oeqa.core.decorator.oeid import OETestID · line 14

Quoted 6 lines
meta/lib/oeqa/selftest/cases/tinfoil.py +0 -1
 class TinfoilTests(OESelftestTestCase):
     """ Basic tests for the tinfoil API """
-
     @OETestID(1568)
     def test_getvar(self):
         with bb.tinfoil.Tinfoil() as tinfoil:

There's just a line removal in this patch. That's probably not what was intended?

Yes, thanks for pointing it, i changed the default behaviour to the old one it means run by default into the main thread and uses the same build dir, so this patch needs to be removed.

Cheers,
Anibal

Quoted 1 line
#48 lore raw

Re: [PATCH 28/30] argparse_oe: Add int_positive type

In reply to Leonardo Sandoval
Quoted 9 lines · Leonardo Sandoval
Quoted 8 lines · Aníbal Limón

Sometimes only expect positive values from cmdline so it's better to filter at parsing cmdline step instead of validate later.

---
 scripts/lib/argparse_oe.py | 7 +++++++
 1 file changed, 7 insertions(+)
Quoted 14 lines

scripts/lib/argparse_oe.py class OeHelpFormatter(argparse.HelpFormatter): · line 170

Quoted 9 lines
scripts/lib/argparse_oe.py +1 -0
index bf6eb17197b..9bdfc1ceca2 100644
@@ -167,3 +167,10 @@ class OeHelpFormatter(argparse.HelpFormatter):
             return '\n'.join(lines)
         else:
             return super(OeHelpFormatter, self)._format_action(action)
+

when adding an argument into a script, one can define the type expected so and the library itself does what you are doing for free, so not sure if this is needed.

The argparse only has int type so i need to ensure that only positive numbers are allowed.

Anibal

Quoted 11 lines
Quoted 9 lines
scripts/lib/argparse_oe.py +6 -0
+def int_positive(value):
+    ivalue = int(value)
+    if ivalue <= 0:
+        raise argparse.ArgumentTypeError(
+                "%s is not a positive int value" % value)
+    return ivalue
#36 lore raw

Re: [PATCH 20/30] oeqa/selftest/cases: runqemu enable thraded runs

In reply to Patrick Ohly
Quoted 10 lines · Patrick Ohly

s/thraded/threaded/ in the subject. There are more spelling mistakes elsewhere ("wrapper methos"). I don't know how important that is.

Quoted 1 line · Aníbal Limón

- Update to use wrappers {bitbake,get_bb_var} from OESelfTestCase class.

s/OESelfTestCase/OESelftestTestCase/

Sorry to be pedantic, but I wanted to look up what these wrappers do and couldn't even find the class ;-}

Yes np,

Quoted 13 lines

I've found them in the "oeqa/selftest/case: Add wrappers to utils.commands modules" patch. I'm a bit worried that this entire patch series is introducing concepts and methods without any documentation or explanations why things are done this way. I suspect it will make it very hard to write selftests correctly.

For example, this patch and others like it seem fairly arbitrary. It doesn't explain why self.bitbake() is better than bitbake(). If in some future patch or test someone were to use bitbake() when they should have used self.bitbake() it's not going to be obvious either whether that is correct.

Quoted 3 lines

Perhaps all OE tests should have these wrappers and only OESelftestTestCase does something special with them? Then we can gradually replace the direct calls to oeqa.utils.commands completely.

You are right, the reason for add the wrappers is to be able to run bitbake, runCmd, get_bb_var{,s}, in certain build directory.

For compatibility reasons the base class OESelftestTestCase is set to use the main thread and the original builddir, and that's the reason to don't delete the oeqa.utils.commands.

I guess the best way to address this is to include a README explaining this situation into the meta/lib/oeqa/selftest folder.

Quoted 3 lines

Add also a wrapper for runqemu and we can get rid of "from oeqa.utils.commands import" completely.

Yes we can after the refkit and other selftest are adapted.

Cheers,
Anibal

Quoted 38 lines
Quoted 37 lines

- Run into the main thread because it needs tinfoil to run.

---
 meta/lib/oeqa/selftest/cases/runqemu.py | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
meta/lib/oeqa/selftest/cases/runqemu.py +3 -4
index 4050a4123ba..e30cb24046f 100644
@@ -6,12 +6,11 @@ import re
 import logging
 
 from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import bitbake, runqemu, get_bb_var
+from oeqa.utils.commands import runqemu
 from oeqa.core.decorator.oeid import OETestID
 
 class RunqemuTests(OESelftestTestCase):
     """Runqemu test class"""
-
     image_is_ready = False
     deploy_dir_image = ''
 
@@ -37,8 +36,8 @@ SYSLINUX_TIMEOUT = "10"
         )
 
         if not RunqemuTests.image_is_ready:
-            RunqemuTests.deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
-            bitbake(self.recipe)
+            RunqemuTests.deploy_dir_image = self.get_bb_var('DEPLOY_DIR_IMAGE')
+            self.bitbake(self.recipe)
             RunqemuTests.image_is_ready = True
 
     @OETestID(2001)
#12 lore raw

Re: [PATCH 03/30] selftest/cases/package: Call parent setUpClass method

In reply to Patrick Ohly
Quoted 9 lines · Patrick Ohly

meta/lib/oeqa/selftest/cases/package.py class VersionOrdering(OESelftestTestCase): · line 21

Quoted 3 lines · Aníbal Limón

def setUpClass(cls):

meta/lib/oeqa/selftest/cases/package.py +2 -0
+        super(VersionOrdering, cls).setUpClass()
+

In Python3 one can simply use super().setUpClass().

Not sure whether that's worth a V2 by itself. There's also plenty of other code outside of this patch which could be simplified like this.

Yes may be is better to send a patch changing all the cases to the new py3 form.

Anibal

Quoted 1 line
#13 lore raw

Re: [PATCH 03/30] selftest/cases/package: Call parent setUpClass method

Quoted 7 lines · Aníbal Limón
Quoted 5 lines · Patrick Ohly
Quoted 3 lines · Aníbal Limón

def setUpClass(cls):

meta/lib/oeqa/selftest/cases/package.py +2 -0
+        super(VersionOrdering, cls).setUpClass()
+
Quoted 7 lines
Quoted 4 lines

In Python3 one can simply use super().setUpClass().

Not sure whether that's worth a V2 by itself. There's also plenty of other code outside of this patch which could be simplified like this.

Yes may be is better to send a patch changing all the cases to the new py3 form.

Definitely not something for this patch series. However, I think new code should avoid the more complex form whenever possible. So if you do a V2, then I'd prefer to have super() in your patch.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#37 lore raw

Re: [PATCH 20/30] oeqa/selftest/cases: runqemu enable thraded runs

In reply to Aníbal Limón
Quoted 4 lines · Aníbal Limón
Quoted 2 lines

Add also a wrapper for runqemu and we can get rid of "from oeqa.utils.commands import" completely.

Yes we can after the refkit and other selftest are adapted.

My proposal wasn't to remove oeqa.utils.commands, only the "import" statements for it in selftest test cases. There might be legitimate uses for oeqa.utils.commands outside of selftests.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#2 lore raw

Re: [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test

In reply to Aníbal Limón
Quoted 4 lines · Aníbal Limón

This was a mistake of me to define wrong what methods needs to be defined by certain python version.

See rev d8380d098a290510b442a7abd2dd5a50cabf5844.

This will fix this error that we see in Refkit with current OE-core master, right?

00:07:10.313 Traceback (most recent call last):
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 70, in <module>
00:07:10.313 ret = main()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 57, in main
00:07:10.313 results = args.func(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 215, in run
00:07:10.313 rc = self._internal_run(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 176, in _internal_run
00:07:10.313 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", line 51, in loadTests
00:07:10.313 self.suites = self.loader.discover()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", line 286, in discover
00:07:10.313 pattern='*.py', top_level_dir=path)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover
00:07:10.313 tests = list(self._find_tests(start_dir, pattern))
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in _find_tests
00:07:10.313 yield _make_failed_import_test(name, self.suiteClass)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in _make_failed_import_test
00:07:10.313 return _make_failed_test(name, ImportError(message), suiteClass)
00:07:10.313 TypeError: _make_failed_test() missing 1 required positional argument: 'suiteClass'

Can this particular patch please be merged into OE-core master independently from the patch series? It's not really related to it anyway.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#3 lore raw

Re: [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test

Quoted 30 lines · Patrick Ohly
Quoted 4 lines · Aníbal Limón

This was a mistake of me to define wrong what methods needs to be defined by certain python version.

See rev d8380d098a290510b442a7abd2dd5a50cabf5844.

This will fix this error that we see in Refkit with current OE-core master, right?

00:07:10.313 Traceback (most recent call last):
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 70, in <module>
00:07:10.313 ret = main()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 57, in main
00:07:10.313 results = args.func(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 215, in run
00:07:10.313 rc = self._internal_run(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 176, in _internal_run
00:07:10.313 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", line 51, in loadTests
00:07:10.313 self.suites = self.loader.discover()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", line 286, in discover
00:07:10.313 pattern='*.py', top_level_dir=path)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover
00:07:10.313 tests = list(self._find_tests(start_dir, pattern))
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in _find_tests
00:07:10.313 yield _make_failed_import_test(name, self.suiteClass)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in _make_failed_import_test
00:07:10.313 return _make_failed_test(name, ImportError(message), suiteClass)
00:07:10.313 TypeError: _make_failed_test() missing 1 required positional argument: 'suiteClass'

Quoted 3 lines

Can this particular patch please be merged into OE-core master independently from the patch series? It's not really related to it anyway.

Yes this will fix that error, showing the error into the case.

By default unittest module shows this kind of error at run step so it's better for us to notice when a test case have syntax or modules missing errors at load step.

Cheers,
Anibal

Quoted 1 line
#38 lore raw

Re: [PATCH 20/30] oeqa/selftest/cases: runqemu enable thraded runs

In reply to Patrick Ohly
Quoted 9 lines · Patrick Ohly
Quoted 4 lines · Aníbal Limón
Quoted 2 lines

Add also a wrapper for runqemu and we can get rid of "from oeqa.utils.commands import" completely.

Yes we can after the refkit and other selftest are adapted.

My proposal wasn't to remove oeqa.utils.commands, only the "import" statements for it in selftest test cases. There might be legitimate uses for oeqa.utils.commands outside of selftests.

Agree,
Anibal

Quoted 1 line
#4 lore raw

Re: [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test

In reply to Aníbal Limón
Quoted 32 lines · Aníbal Limón
Quoted 30 lines · Patrick Ohly
Quoted 4 lines · Aníbal Limón

This was a mistake of me to define wrong what methods needs to be defined by certain python version.

See rev d8380d098a290510b442a7abd2dd5a50cabf5844.

This will fix this error that we see in Refkit with current OE-core master, right?

00:07:10.313 Traceback (most recent call last):
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 70, in <module>
00:07:10.313 ret = main()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 57, in main
00:07:10.313 results = args.func(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 215, in run
00:07:10.313 rc = self._internal_run(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 176, in _internal_run
00:07:10.313 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", line 51, in loadTests
00:07:10.313 self.suites = self.loader.discover()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", line 286, in discover
00:07:10.313 pattern='*.py', top_level_dir=path)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover
00:07:10.313 tests = list(self._find_tests(start_dir, pattern))
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in _find_tests
00:07:10.313 yield _make_failed_import_test(name, self.suiteClass)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in _make_failed_import_test
00:07:10.313 return _make_failed_test(name, ImportError(message), suiteClass)
00:07:10.313 TypeError: _make_failed_test() missing 1 required positional argument: 'suiteClass'

Quoted 5 lines
Quoted 3 lines

Can this particular patch please be merged into OE-core master independently from the patch series? It's not really related to it anyway.

Yes this will fix that error, showing the error into the case.

The fix doesn't work for me:

File "/usr/lib/python3.5/unittest/loader.py", line 41, in _make_failed_import_test
return _make_failed_test(name, ImportError(message), suiteClass, message)
TypeError: _make_failed_test() takes 3 positional arguments but 4 were given

You need to mimic the exact same signature as in Python itself, and Python 3.5 seems to be different again:

/usr/lib/python3.5/unittest/loader.py:def _make_failed_test(methodname, exception, suiteClass, message):

You can probably get away with

if sys.version_info >= (3,4,4):
def _make_failed_test(classname, exception, suiteClass, message=None):

The actual problem in refkit is that we are hitting this _make_failed_test() at all. After fixing meta/lib/oeqa/core/loader.py, I am getting a more useful exception and know what to do about the actual problem.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#5 lore raw

Re: [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test

Quoted 41 lines · Patrick Ohly
Quoted 32 lines · Aníbal Limón
Quoted 30 lines · Patrick Ohly
Quoted 4 lines · Aníbal Limón

This was a mistake of me to define wrong what methods needs to be defined by certain python version.

See rev d8380d098a290510b442a7abd2dd5a50cabf5844.

This will fix this error that we see in Refkit with current OE-core master, right?

00:07:10.313 Traceback (most recent call last):
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 70, in <module>
00:07:10.313 ret = main()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 57, in main
00:07:10.313 results = args.func(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 215, in run
00:07:10.313 rc = self._internal_run(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 176, in _internal_run
00:07:10.313 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", line 51, in loadTests
00:07:10.313 self.suites = self.loader.discover()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", line 286, in discover
00:07:10.313 pattern='*.py', top_level_dir=path)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover
00:07:10.313 tests = list(self._find_tests(start_dir, pattern))
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in _find_tests
00:07:10.313 yield _make_failed_import_test(name, self.suiteClass)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in _make_failed_import_test
00:07:10.313 return _make_failed_test(name, ImportError(message), suiteClass)
00:07:10.313 TypeError: _make_failed_test() missing 1 required positional argument: 'suiteClass'

Quoted 5 lines
Quoted 3 lines

Can this particular patch please be merged into OE-core master independently from the patch series? It's not really related to it anyway.

Yes this will fix that error, showing the error into the case.

The fix doesn't work for me:

Quoted 8 lines

File "/usr/lib/python3.5/unittest/loader.py", line 41, in _make_failed_import_test
return _make_failed_test(name, ImportError(message), suiteClass, message)
TypeError: _make_failed_test() takes 3 positional arguments but 4 were given

You need to mimic the exact same signature as in Python itself, and Python 3.5 seems to be different again:

/usr/lib/python3.5/unittest/loader.py:def _make_failed_test(methodname, exception, suiteClass, message):

This is strange, i tested on a python 3.5.x and python 3.4.3 versions. I don't know why isn't working on your machine.

Cheers,
Anibal

Quoted 11 lines

You can probably get away with

if sys.version_info >= (3,4,4):
def _make_failed_test(classname, exception, suiteClass, message=None):

The actual problem in refkit is that we are hitting this _make_failed_test() at all. After fixing meta/lib/oeqa/core/loader.py, I am getting a more useful exception and know what to do about the actual problem.

#6 lore raw

Re: [PATCH 01/30] oeqa/core/loader: Switch method definition for _make_failed_test

Quoted 47 lines · Aníbal Limón
Quoted 45 lines · Patrick Ohly
Quoted 32 lines · Aníbal Limón
Quoted 30 lines · Patrick Ohly
Quoted 4 lines · Aníbal Limón

This was a mistake of me to define wrong what methods needs to be defined by certain python version.

See rev d8380d098a290510b442a7abd2dd5a50cabf5844.

This will fix this error that we see in Refkit with current OE-core master, right?

00:07:10.313 Traceback (most recent call last):
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 70, in <module>
00:07:10.313 ret = main()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/scripts/oe-selftest", line 57, in main
00:07:10.313 results = args.func(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 215, in run
00:07:10.313 rc = self._internal_run(logger, args)
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/selftest/context.py", line 176, in _internal_run
00:07:10.313 self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/context.py", line 51, in loadTests
00:07:10.313 self.suites = self.loader.discover()
00:07:10.313 File "/srv/jenkins/workspace/ci-2017-07-14_01-48-19-build-2315/openembedded-core/meta/lib/oeqa/core/loader.py", line 286, in discover
00:07:10.313 pattern='*.py', top_level_dir=path)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover
00:07:10.313 tests = list(self._find_tests(start_dir, pattern))
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 327, in _find_tests
00:07:10.313 yield _make_failed_import_test(name, self.suiteClass)
00:07:10.313 File "/usr/lib64/python3.4/unittest/loader.py", line 39, in _make_failed_import_test
00:07:10.313 return _make_failed_test(name, ImportError(message), suiteClass)
00:07:10.313 TypeError: _make_failed_test() missing 1 required positional argument: 'suiteClass'

Quoted 5 lines
Quoted 3 lines

Can this particular patch please be merged into OE-core master independently from the patch series? It's not really related to it anyway.

Yes this will fix that error, showing the error into the case.

The fix doesn't work for me:

File "/usr/lib/python3.5/unittest/loader.py", line 41, in _make_failed_import_test
return _make_failed_test(name, ImportError(message), suiteClass, message)
TypeError: _make_failed_test() takes 3 positional arguments but 4 were given

Quoted 7 lines
Quoted 4 lines

You need to mimic the exact same signature as in Python itself, and Python 3.5 seems to be different again:

/usr/lib/python3.5/unittest/loader.py:def _make_failed_test(methodname, exception, suiteClass, message):

This is strange, i tested on a python 3.5.x and python 3.4.3 versions. I don't know why isn't working on your machine.

Have you tested with a test case that fails?

python3.5 = 3.5.3 definitely has four parameters, as quoted above.

I don't know which version has "def _make_failed_test(classname, exception, suiteClass)".

The actual error in refkit is:

ImportError: Failed to import test module: refkit_ostree
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/loader.py", line 428, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name

    __import__(name)
  File "/fast/work/intel-iot-refkit-pr/meta-refkit-core/lib/oeqa/selftest/cases/refkit_ostree.py", line 12, in <module>
    class RefkitOSTreeUpdateBase(SystemUpdateBase):
  File "/fast/work/intel-iot-refkit-pr/meta-refkit-core/lib/oeqa/selftest/cases/refkit_ostree.py", line 35, in RefkitOSTreeUpdateBase
    'socat-native')

File "/fast/work/intel-iot-refkit-pr/openembedded-core/meta/lib/oeqa/utils/commands.py", line 227, in get_bb_vars
bbenv = get_bb_env(target, postconfig=postconfig)
File "/fast/work/intel-iot-refkit-pr/openembedded-core/meta/lib/oeqa/utils/commands.py", line 221, in get_bb_env
return bitbake("-e %s" % target, postconfig=postconfig).output
File "/fast/work/intel-iot-refkit-pr/openembedded-core/meta/lib/oeqa/utils/commands.py", line 213, in bitbake
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
File "/fast/work/intel-iot-refkit-pr/openembedded-core/meta/lib/oeqa/utils/commands.py", line 191, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake -e socat-native' returned non-zero exit status 1:
Loading cache...done.
Loaded 3495 entries from dependency cache.
Parsing recipes...done.
ERROR: No recipes available for:
/fast/work/intel-iot-refkit-pr/openembedded-core/../meta-refkit-core/bbappends/meta-security/recipes-tpm/swtpm/swtpm-wrappers-native.bbappend /fast/work/intel-iot-refkit-pr/openembedded-core/../meta-refkit-core/bbappends/meta-security/recipes-tpm/trousers/trousers_%.bbappend
Parsing of 2606 .bb files complete (2602 cached, 4 parsed). 3499 targets, 415 skipped, 0 masked, 0 errors.

Summary: There was 1 ERROR message shown, returning a non-zero exit code.

RefkitOSTreeUpdateBase calls get_bb_vars() as part of the class definition. That's the part that is failing.

The failure is caused by meta-security re-structuring the layers. Will be easy enough to fix in meta-refkit-core, now that I know what it is.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.