oe-selftest QA test fixes
cover oe-selftest QA test fixes
Some fixes for issues in the QA tests; some of these came up on the autobuilder, others were stumbled over while I was fixing the other issues.
The following changes since commit 353e70f3964eb6531ecb3dc69cbd827053d8f463:
init-ifupdown: Add missing script for qemuarm64 (2014-12-23 10:12:47 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/qafixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/qafixes
Paul Eggleton (7):
oeqa/utils: replace some tabs with spaces
oeqa/utils: fix testcase decorator to allow calling tests individually oeqa/selftest: fix test_event_handler for changes in bitbake output oeqa/selftest: skip test_incremental_image_generation if not using rpm oeqa/selftest: improve failure messages for devtool tests oeqa/selftest: populate pkgdata/shlibs in test_recipetool_create_git oeqa/selftest: fix test_force_task so it doesn't taint the entire build
meta/lib/oeqa/selftest/bbtests.py | 3 +- meta/lib/oeqa/selftest/buildoptions.py | 3 ++ meta/lib/oeqa/selftest/devtool.py | 22 +++++---- meta/lib/oeqa/utils/commands.py | 10 ++-- meta/lib/oeqa/utils/decorators.py | 85 +++++++++++++++++----------------- 5 files changed, 65 insertions(+), 58 deletions(-)
Signature · 1 line
1.9.3
PATCH 1/7 oeqa/utils: replace some tabs with spaces
Python code should use spaces only for indentation.
--- meta/lib/oeqa/utils/commands.py | 10 ++--- meta/lib/oeqa/utils/decorators.py | 84 +++++++++++++++++++-------------------- 2 files changed, 47 insertions(+), 47 deletions(-)
meta/lib/oeqa/utils/commands.py +5 -5
@@ -110,11 +110,11 @@ def runCmd(command, ignore_status=False, timeout=None, assert_error=True, **opti def bitbake(command, ignore_status=False, timeout=None, postconfig=None, **options): if postconfig: - postconfig_file = os.path.join(os.environ.get('BUILDDIR'), 'oeqa-post.conf') - ftools.write_file(postconfig_file, postconfig) - extra_args = "-R %s" % postconfig_file + postconfig_file = os.path.join(os.environ.get('BUILDDIR'), 'oeqa-post.conf') + ftools.write_file(postconfig_file, postconfig) + extra_args = "-R %s" % postconfig_file else: - extra_args = "" + extra_args = "" if isinstance(command, basestring): cmd = "bitbake " + extra_args + " " + command
@@ -122,7 +122,7 @@ def bitbake(command, ignore_status=False, timeout=None, postconfig=None, **optio cmd = [ "bitbake" ] + [a for a in (command + extra_args.split(" ")) if a not in [""]] try: - return runCmd(cmd, ignore_status, timeout, **options) + return runCmd(cmd, ignore_status, timeout, **options) finally: if postconfig: os.remove(postconfig_file)
meta/lib/oeqa/utils/decorators.py +42 -42
107 lines not on this page · raw message
Signature · 1 line
1.9.3
PATCH 2/7 oeqa/utils: fix testcase decorator to allow calling tests individually
In reply to Paul EggletonWithout this, running tests individually failed with a traceback:
$ oe-selftest --run-test
buildoptions.ImageOptionsTests.test_incremental_image_generation
2014-12-23 14:40:37,636 - selftest - INFO - Checking that everything is in order before running the tests
2014-12-23 14:40:38,408 - selftest - INFO - Running bitbake -p
2014-12-23 14:40:40,235 - selftest - INFO - Loading tests from:
oeqa.selftest.buildoptions.ImageOptionsTests.test_incremental_image_generation
Traceback (most recent call last):
File "/home/user/poky/scripts/oe-selftest", line 179, in <module>
ret = main()
File "/home/user/poky/scripts/oe-selftest", line 164, in main
suite.addTests(loader.loadTestsFromName(test))
File "/usr/lib64/python2.7/unittest/loader.py", line 91, in loadTestsFromName
module = __import__('.'.join(parts_copy))
File "/home/user/poky/meta/lib/oeqa/selftest/buildoptions.py", line 12, in <module>
class ImageOptionsTests(oeSelfTest):
File "/home/user/poky/meta/lib/oeqa/selftest/buildoptions.py", line 14, in ImageOptionsTests
@testcase(761)
NameError: global name 'func' is not defined--- meta/lib/oeqa/utils/decorators.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/oeqa/utils/decorators.py +1 -0
@@ -89,6 +89,7 @@ class testcase(object): def wrapped_f(*args): return func(*args) wrapped_f.test_case = self.test_case + wrapped_f.__name__ = func.__name__ return wrapped_f class NoParsingFilter(logging.Filter):
Signature · 1 line
1.9.3
PATCH 3/7 oeqa/selftest: fix test_event_handler for changes in bitbake output
In reply to Paul Eggleton--- meta/lib/oeqa/selftest/bbtests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/selftest/bbtests.py +1 -1
@@ -27,7 +27,7 @@ class BitbakeTests(oeSelfTest): def test_event_handler(self): self.write_config("INHERIT += \"test_events\"") result = bitbake('m4-native') - find_build_started = re.search("NOTE: Test for bb\.event\.BuildStarted(\n.*)*NOTE: Preparing runqueue", result.output) + find_build_started = re.search("NOTE: Test for bb\.event\.BuildStarted(\n.*)*NOTE: Preparing RunQueue", result.output) find_build_completed = re.search("Tasks Summary:.*(\n.*)*NOTE: Test for bb\.event\.BuildCompleted", result.output) self.assertTrue(find_build_started, msg = "Match failed in:\n%s" % result.output) self.assertTrue(find_build_completed, msg = "Match failed in:\n%s" % result.output)
Signature · 1 line
1.9.3
PATCH 4/7 oeqa/selftest: skip test_incremental_image_generation if not using rpm
In reply to Paul EggletonWe have just implemented incremental ipk image generation, but at the moment this test doesn't support that, so skip it if not using rpm.
--- meta/lib/oeqa/selftest/buildoptions.py | 3 +++ 1 file changed, 3 insertions(+)
meta/lib/oeqa/selftest/buildoptions.py +3 -0
@@ -13,6 +13,9 @@ class ImageOptionsTests(oeSelfTest): @testcase(761) def test_incremental_image_generation(self): + image_pkgtype = get_bb_var("IMAGE_PKGTYPE") + if image_pkgtype != 'rpm': + self.skipTest('Not using RPM as main package format') bitbake("-c cleanall core-image-minimal") self.write_config('INC_RPM_IMAGE_GEN = "1"') self.append_config('IMAGE_FEATURES += "ssh-server-openssh"')
Signature · 1 line
1.9.3
PATCH 5/7 oeqa/selftest: improve failure messages for devtool tests
In reply to Paul EggletonassertTrue prints "False is not True" if it fails, which is pretty much useless. Use a more appropriate assertion test where practical and add a message where it isn't.
--- meta/lib/oeqa/selftest/devtool.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
meta/lib/oeqa/selftest/devtool.py +10 -10
@@ -25,15 +25,15 @@ class DevtoolTests(oeSelfTest): result = runCmd('devtool create-workspace %s' % tempdir) self.assertTrue(os.path.isfile(os.path.join(tempdir, 'conf', 'layer.conf'))) result = runCmd('bitbake-layers show-layers') - self.assertTrue(tempdir in result.output) + self.assertIn(tempdir, result.output) # Try creating a workspace layer with the default path self.track_for_cleanup(workspacedir) self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') result = runCmd('devtool create-workspace') self.assertTrue(os.path.isfile(os.path.join(workspacedir, 'conf', 'layer.conf'))) result = runCmd('bitbake-layers show-layers') - self.assertTrue(tempdir not in result.output) - self.assertTrue(workspacedir in result.output) + self.assertNotIn(tempdir, result.output) + self.assertIn(workspacedir, result.output) def test_recipetool_create(self): # Try adding a recipe
@@ -74,7 +74,7 @@ class DevtoolTests(oeSelfTest): recipefile = os.path.join(tempdir, 'libmatchbox.bb') srcuri = 'git://git.yoctoproject.org/libmatchbox' result = runCmd('recipetool create -o %s %s -x %s' % (recipefile, srcuri, tempsrc)) - self.assertTrue(os.path.isfile(recipefile)) + self.assertTrue(os.path.isfile(recipefile), 'recipetool did not create recipe file; output:\n%s' % result.output) checkvars = {} checkvars['LICENSE'] = 'LGPLv2.1' checkvars['LIC_FILES_CHKSUM'] = 'file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34'
@@ -111,7 +111,7 @@ class DevtoolTests(oeSelfTest): result = runCmd('wget %s' % url, cwd=tempdir) result = runCmd('tar xfv pv-1.5.3.tar.bz2', cwd=tempdir) srcdir = os.path.join(tempdir, 'pv-1.5.3') - self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure'))) + self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure')), 'Unable to find configure script in source directory') # Test devtool add self.track_for_cleanup(workspacedir) self.add_command_to_tearDown('bitbake -c cleansstate pv')
@@ -127,9 +127,9 @@ class DevtoolTests(oeSelfTest): # Test devtool build result = runCmd('devtool build pv') installdir = get_bb_var('D', 'pv') - self.assertTrue(installdir) + self.assertTrue(installdir, 'Could not query installdir variable') bindir = get_bb_var('bindir', 'pv') - self.assertTrue(bindir) + self.assertTrue(bindir, 'Could not query bindir variable') if bindir[0] == '/': bindir = bindir[1:] self.assertTrue(os.path.isfile(os.path.join(installdir, bindir, 'pv')), 'pv binary not found in D')
@@ -167,9 +167,9 @@ class DevtoolTests(oeSelfTest): result = runCmd("sed -i 's!^\.TH.*!.TH MDADM 8 \"\" v9.999-custom!' %s" % os.path.join(tempdir, 'mdadm.8.in')) bitbake('mdadm -c package') pkgd = get_bb_var('PKGD', 'mdadm') - self.assertTrue(pkgd) + self.assertTrue(pkgd, 'Could not query PKGD variable') mandir = get_bb_var('mandir', 'mdadm') - self.assertTrue(mandir) + self.assertTrue(mandir, 'Could not query mandir variable') if mandir[0] == '/': mandir = mandir[1:] with open(os.path.join(pkgd, mandir, 'man8', 'mdadm.8'), 'r') as f:
@@ -223,7 +223,7 @@ class DevtoolTests(oeSelfTest): elif re.search('minicom_[^_]*.bb$', line): self.assertEqual(line[:3], ' M ', 'Unexpected status in line: %s' % line) else: - self.assertTrue(False, 'Unexpected modified file in status: %s' % line) + raise AssertionError('Unexpected modified file in status: %s' % line) def test_devtool_extract(self): # Check preconditions
Signature · 1 line
1.9.3
PATCH 6/7 oeqa/selftest: populate pkgdata/shlibs in test_recipetool_create_git
In reply to Paul EggletonBuild some recipes so that we have the needed data in the sysroot for recipetool to refer to when scanning for dependencies.
--- meta/lib/oeqa/selftest/devtool.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/devtool.py +2 -0
@@ -66,6 +66,8 @@ class DevtoolTests(oeSelfTest): self.assertEqual(checkvars, {}, 'Some variables not found') def test_recipetool_create_git(self): + # Ensure we have the right data in shlibs/pkgdata + bitbake('libpng pango libx11 libxext') # Try adding a recipe tempdir = tempfile.mkdtemp(prefix='devtoolqa') self.track_for_cleanup(tempdir)
Signature · 1 line
1.9.3
PATCH 7/7 oeqa/selftest: fix test_force_task so it doesn't taint the entire build
In reply to Paul EggletonUsing -f sets a taint on the specified task, and m4-native is in the dependency chain for just about everything, which means that everything gets forced to rebuild. We don't need the influence of this test to extend outside of the test itself, so ensure that the taint gets cleared at the end of the test.
--- meta/lib/oeqa/selftest/bbtests.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/oeqa/selftest/bbtests.py +1 -0
@@ -71,6 +71,7 @@ class BitbakeTests(oeSelfTest): @testcase(163) def test_force_task(self): bitbake('m4-native') + self.add_command_to_tearDown('bitbake -c clean m4-native') result = bitbake('-C compile m4-native') look_for_tasks = ['do_compile', 'do_install', 'do_populate_sysroot'] for task in look_for_tasks:
Signature · 1 line
1.9.3
Re: [PATCH 7/7] oeqa/selftest: fix test_force_task so it doesn't taint the entire build
Quoted 1 line · On Tuesday 23 December 2014 16:08:48 Paul Eggleton wrote:
Using -f sets a taint on the specified task
Hmm, that would be -C in this case, not -f. I've adjusted the commit message on the branch.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre