Support supported distros on selftest
cover Support supported distros on selftest
From: Leonardo Sandoval <[email protected]>
Originally, the test defined inside oeqa/selftest were defined for the 'poky' distro. However, there are more supported distros, including nodistro, poky-lsb and poky-tiny. These series takes into consideration the distribution, deciding either run or skip certain tests.
[YOCTO #8525]
The following changes since commit 4a7eb4b92f32c17abd1111246b1acb9ad3daa355:
build-appliance-image: Update to master head revision (2016-10-11 08:51:36 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib lsandov1/oe-selftest-non-poky-distros
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lsandov1/oe-selftest-non-poky-distros
Leonardo Sandoval (9):
selftest: set correct linux-yocto recipe when distro is poky-tiny
selftest: skip two bbtest tests depending on distro
selftest: skip test_read_only_image on poky-tiny distro
selftest: skip some devtool tests when distro is poky-tiny due to
missing libx11
selftest: skip those imagefeatures tests not relevant for certain
distros
selftest: skip test_recipetool_create_git on poky-tiny distro
selftest: skip runtime-test's test_testimage_install on poky-tiny
distro
selftest: skip sstatetests methods considering distro and its features
selftest: skip wic test methods for poky-tiny distro
meta/lib/oeqa/selftest/base.py | 1 + meta/lib/oeqa/selftest/bblayers.py | 6 ++-- meta/lib/oeqa/selftest/bbtests.py | 7 ++++ meta/lib/oeqa/selftest/buildoptions.py | 2 ++ meta/lib/oeqa/selftest/devtool.py | 7 +++- meta/lib/oeqa/selftest/imagefeatures.py | 18 +++++++++-- meta/lib/oeqa/selftest/recipetool.py | 2 ++ meta/lib/oeqa/selftest/runtime-test.py | 2 ++ meta/lib/oeqa/selftest/sstate.py | 10 +++--- meta/lib/oeqa/selftest/sstatetests.py | 57 ++++++++++++++++++++++++++++----- meta/lib/oeqa/selftest/wic.py | 9 ++++++ 11 files changed, 102 insertions(+), 19 deletions(-)
Signature · 1 line
2.1.4
PATCH 1/9 selftest: set correct linux-yocto recipe when distro is poky-tiny
From: Leonardo Sandoval <[email protected]>
The poky-tiny distro points to linux-yocto-tiny instead of linux-yocto, so setting the correct name depending on the distro.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/base.py | 1 + meta/lib/oeqa/selftest/bblayers.py | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-)
meta/lib/oeqa/selftest/base.py +1 -0
@@ -41,6 +41,7 @@ class oeSelfTest(unittest.TestCase): self.testinc_path, self.testinc_bblayers_path, self.machineinc_path, self.localconf_backup, self.local_bblayers_backup] + self.distro = get_bb_var('DISTRO') super(oeSelfTest, self).__init__(methodName) def setUp(self):
meta/lib/oeqa/selftest/bblayers.py +4 -2
@@ -68,6 +68,7 @@ class BitbakeLayers(oeSelfTest): @testcase(1384) def test_bitbakelayers_showrecipes(self): + distro = get_bb_var('DISTRO') result = runCmd('bitbake-layers show-recipes') self.assertIn('aspell:', result.output) self.assertIn('mtd-utils:', result.output)
@@ -77,11 +78,12 @@ class BitbakeLayers(oeSelfTest): self.assertIn('mtd-utils:', result.output) self.assertNotIn('aspell:', result.output) result = runCmd('bitbake-layers show-recipes -i kernel') - self.assertIn('linux-yocto:', result.output) + linux_yocto = 'linux-yocto-tiny' if distro == 'poky-tiny' else 'linux-yocto' + self.assertIn(linux_yocto, result.output) self.assertNotIn('mtd-utils:', result.output) result = runCmd('bitbake-layers show-recipes -i image') self.assertIn('core-image-minimal', result.output) - self.assertNotIn('linux-yocto:', result.output) + self.assertNotIn(linux_yocto, result.output) self.assertNotIn('mtd-utils:', result.output) result = runCmd('bitbake-layers show-recipes -i cmake,pkgconfig') self.assertIn('libproxy:', result.output)
Signature · 1 line
2.1.4
Re: [PATCH 1/9] selftest: set correct linux-yocto recipe when distro is poky-tiny
Quoted 6 lines · On 14 October 2016 at 17:40, <[email protected]> wrote:
result = runCmd('bitbake-layers show-recipes -i kernel')
(quoted diff) +1
-1- self.assertIn('linux-yocto:', result.output) + linux_yocto = 'linux-yocto-tiny' if distro == 'poky-tiny' else'linux-yocto'
+self.assertIn(linux_yocto, result.output)
If we want this test to be distro- and machine-agnostic then we should handle other BSPs that don't use linux-yocto. As this is just a test for bitbake-layers, I'd say we can just delete the "inherits kernel" block.
Ross
Re: [PATCH 1/9] selftest: set correct linux-yocto recipe when distro is poky-tiny
Quoted 12 lines · Ross
On 14 October 2016 at 17:40, <[email protected] <mailto:[email protected]>> wrote:
result = runCmd('bitbake-layers show-recipes -i kernel')
-self.assertIn('linux-yocto:', result.output)
+linux_yocto= 'linux-yocto-tiny' if distro == 'poky-tiny'
else 'linux-yocto'
+self.assertIn(linux_yocto, result.output)If we want this test to be distro- and machine-agnostic then we should handle other BSPs that don't use linux-yocto. As this is just a test for bitbake-layers, I'd say we can just delete the "inherits kernel" block.
Agreed. Removed all the kernel checks, so sending a v2 today.
Quoted 1 line
Ross
PATCH 2/9 selftest: skip two bbtest tests depending on distro
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
There are two bbtests that just makes sense on certain distros:test_bitbake_g cannot be done on poky-tiny and test_non_gplv3 just makes sense on poky and poky-tiny. Skip these when necessary under the latter conditions.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/bbtests.py | 7 +++++++ 1 file changed, 7 insertions(+)
meta/lib/oeqa/selftest/bbtests.py +7 -0
@@ -114,6 +114,9 @@ class BitbakeTests(oeSelfTest): @testcase(167) def test_bitbake_g(self): + if self.distro == 'poky-tiny': + self.skipTest('core-image-full-cmdline is not buildable with poky-tiny') + result = bitbake('-g core-image-full-cmdline') for f in ['pn-buildlist', 'pn-depends.dot', 'package-depends.dot', 'task-depends.dot']: self.addCleanup(os.remove, f)
@@ -229,6 +232,10 @@ INHERIT_remove = \"report-error\" @testcase(1119) def test_non_gplv3(self): + supported_distros = ['poky', 'poky-tiny'] + if not self.distro in supported_distros: + self.skipTest('Test considers only %s distros' % ','.join(supported_distros)) + data = 'INCOMPATIBLE_LICENSE = "GPLv3"' conf = os.path.join(self.builddir, 'conf/local.conf') ftools.append_file(conf ,data)
Signature · 1 line
2.1.4
Re: [PATCH 2/9] selftest: skip two bbtest tests depending on distro
Quoted 21 lines · On Fri, 2016-10-14 at 11:40 -0500, [email protected]…
From: Leonardo Sandoval <[email protected]>
There are two bbtests that just makes sense on certain distros:
test_bitbake_gcannot be done on poky-tiny andtest_non_gplv3just makes sense on poky and poky-tiny. Skip these when necessary under the latter conditions.[YOCTO #8525]
Signed-off-by Leonardo Sandoval <[email protected]tel.com>
---
meta/lib/oeqa/selftest/bbtests.py | 7 +++++++1 file changed, 7 insertions(+)meta/lib/oeqa/selftest/bbtests.py +0
-0@@ -114,6 +114,9 @@ class BitbakeTests(oeSelfTest):
Quoted 7 lines
@testcase(167)
deftest_bitbake_g(self):
+ if self.distro == 'poky-tiny':
+ self.skipTest('core-image-full-cmdline is not buildable with poky-tiny')
+
This test is still valid for poky-tiny, it's just that the image the test builds isn't buildable with poky-tiny, right?
I think it would be better to change the test to a different, smaller, image that can also be built for poky-tiny (core-image-minimal?). We'd also need to change the target the test checks for the presence of in pn-buildlist.
Regards,
Joshua
Quoted 19 lines
result = bitbake('-g core-image-full-cmdline') for f in ['pn-buildlist', 'pn-depends.dot', 'package- depends.dot', 'task-depends.dot']:
self.addCleanup(os.remove, f)meta/lib/oeqa/selftest/bbtests.py +0
-0@@ -229,6 +232,10 @@ INHERIT_remove = \"report-error\"
@testcase(1119)
deftest_non_gplv3(self):
+ supported_distros = ['poky', 'poky-tiny']
+ if not self.distro insupported_distros:
+ self.skipTest('Test considers only %s distros' % ','.join(supported_distros))
+
data = 'INCOMPATIBLE_LICENSE= "GPLv3"' conf =os.path.join(self.builddir, 'conf/local.conf')ftools.append_file(conf ,data)
--
2.1.4
Re: [PATCH 2/9] selftest: skip two bbtest tests depending on distro
Quoted 33 lines · Joshua Lock
Quoted 21 lines · On Fri, 2016-10-14 at 11:40 -0500, [email protected]…
From: Leonardo Sandoval <[email protected]>
There are two bbtests that just makes sense on certain distros:
test_bitbake_gcannot be done on poky-tiny andtest_non_gplv3just makes sense on poky and poky-tiny. Skip these when necessary under the latter conditions.[YOCTO #8525]
Signed-off-by Leonardo Sandoval <[email protected]tel.com>
--- meta/lib/oeqa/selftest/bbtests.py | 7 +++++++ 1 file changed, 7 insertions(+)meta/lib/oeqa/selftest/bbtests.py +0
-0meta/lib/oeqa/selftest/bbtests.py class BitbakeTests(oeSelfTest): · line 120
Quoted 7 lines
meta/lib/oeqa/selftest/bbtests.py +3
-0@@ -114,6 +114,9 @@ class BitbakeTests(oeSelfTest): @testcase(167) def test_bitbake_g(self): + if self.distro == 'poky-tiny': + self.skipTest('core-image-full-cmdline is not buildable with poky-tiny') +This test is still valid for poky-tiny, it's just that the image the test builds isn't buildable with poky-tiny, right?
Quoted 4 lines
I think it would be better to change the test to a different, smaller, image that can also be built for poky-tiny (core-image-minimal?). We'd also need to change the target the test checks for the presence of in pn-buildlist.
Good idea. Which image and pn-buildlist package check do you suggest?
Quoted 23 lines
Regards,
Joshua
Quoted 19 lines
meta/lib/oeqa/selftest/bbtests.py +0
-0result = bitbake('-g core-image-full-cmdline') for f in ['pn-buildlist', 'pn-depends.dot', 'package-depends.dot', 'task-depends.dot']:
self.addCleanup(os.remove, f)meta/lib/oeqa/selftest/bbtests.py +4
-0@@ -229,6 +232,10 @@ INHERIT_remove = \"report-error\" @testcase(1119) def test_non_gplv3(self): + supported_distros = ['poky', 'poky-tiny'] + if not self.distro in supported_distros: + self.skipTest('Test considers only %s distros' % ','.join(supported_distros)) + data = 'INCOMPATIBLE_LICENSE = "GPLv3"' conf = os.path.join(self.builddir, 'conf/local.conf')
ftools.append_file(conf ,data)Signature · 1 line
2.1.4
Re: [PATCH 2/9] selftest: skip two bbtest tests depending on distro
Quoted 41 lines · Leonardo Sandoval
Quoted 39 lines · Joshua Lock
Quoted 24 lines · On Fri, 2016-10-14 at 11:40 -0500, [email protected]…
From: Leonardo Sandoval <[email protected] om>
There are two bbtests that just makes sense on certain distros:
test_bitbake_gcannot be done on poky-tiny andtest_non_gplv3just makes sense on poky and poky-tiny. Skip these when necessary under the latter conditions.[YOCTO #8525]
Signed-off-by Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.in
tel.com>
---
meta/lib/oeqa/selftest/bbtests.py| 7 +++++++1 file changed, 7 insertions(+)meta/lib/oeqa/selftest/bbtests.py +0
-0@@ -114,6 +114,9 @@ class BitbakeTests(oeSelfTest):Quoted 8 lines
@testcase(167)
deftest_bitbake_g(self):
+ if self.distro == 'poky-tiny':
+ self.skipTest('core-image-full-cmdline is not buildable
with poky-tiny')
+This test is still valid for poky-tiny, it's just that the image the
test builds isn't buildable with poky-tiny, right?
Quoted 7 lines
Quoted 6 lines
I think it would be better to change the test to a different, smaller,
image that can also be built for poky-tiny (core-image-minimal?). We'd also need to change the target the test checks for the presence of
in pn-buildlist.Good idea. Which image and pn-buildlist package check do you suggest?
It doesn't really matter and will depend on which image you decide to build for the test.
The goal of the test appears to be to validate the output of bitbake -e. Hopefully we can find an image and recipe combination which is buildable across distros but I don't know what that would be off the top of my head.
Regards,
Joshua
Quoted 26 lines
Quoted 25 lines
Regards,
Joshua
Quoted 20 lines
result = bitbake('-g core-image-full-cmdline') for f in ['pn-buildlist', 'pn-depends.dot', 'package- depends.dot', 'task-depends.dot']:
self.addCleanup(os.remove, f)meta/lib/oeqa/selftest/bbtests.py +0
-0@@ -229,6 +232,10 @@ INHERIT_remove = \"report-error\"
@testcase(1119)
deftest_non_gplv3(self):
+ supported_distros = ['poky', 'poky-tiny']
+ if not self.distro insupported_distros:
+ self.skipTest('Test considers only %s distros' % ','.join(supported_distros))
+
data = 'INCOMPATIBLE_LICENSE= "GPLv3"' conf =os.path.join(self.builddir, 'conf/local.conf')ftools.append_file(conf ,data)
--
2.1.4
PATCH 3/9 selftest: skip test_read_only_image on poky-tiny distro
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
The test test_read_only_image builds core-image-sato which is not buildable in poky-tiny so skip it if this is the case.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/buildoptions.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/buildoptions.py +2 -0
@@ -60,6 +60,8 @@ class ImageOptionsTests(oeSelfTest): @testcase(1435) def test_read_only_image(self): self.write_config('IMAGE_FEATURES += "read-only-rootfs"') + if self.distro == 'poky-tiny': + self.skipTest('core-image-sato is not buildable with poky-tiny') bitbake("core-image-sato") # do_image will fail if there are any pending postinsts
Signature · 1 line
2.1.4
Re: [PATCH 3/9] selftest: skip test_read_only_image on poky-tiny distro
Quoted 4 lines · On 14 October 2016 at 17:40, <[email protected]> wrote:
(quoted diff) +2
-0+ if self.distro == 'poky-tiny': + self.skipTest('core-image-sato is not buildable withpoky-tiny')
DISTRO_FEATURE check - sato needs opengl x11 so this is the actual blocker.
For a read_only_image test, switching the image to core-image-base or similar would exercise the code just as well without needing as much building or assumptions on DISTRO/MACHINE.
Ross
Re: [PATCH 3/9] selftest: skip test_read_only_image on poky-tiny distro
Quoted 12 lines · Ross
On 14 October 2016 at 17:40, <[email protected] <mailto:[email protected]>> wrote:
+ if self.distro == 'poky-tiny':
+ self.skipTest('core-image-sato is not buildable with
poky-tiny')
DISTRO_FEATUREcheck - sato needs opengl x11 so this is the actual blocker.For a
read_only_imagetest, switching the image to core-image-base or similar would exercise the code just as well without needing as much building or assumptions on DISTRO/MACHINE.
I see. So instead of using the distro value, I will use the 'libx11 in DISTRO_FEATURE' check. Sending V2 today.
Quoted 1 line
Ross
PATCH 4/9 selftest: skip some devtool tests when distro is poky-tiny due to missing libx11
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
When distro is poky-tiny, there is no recipe providing libx11, thus skip relevant tests.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/devtool.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/devtool.py +6 -1
@@ -337,7 +337,8 @@ class DevtoolTests(DevtoolBase): @testcase(1161) def test_devtool_add_fetch_git(self): - # Fetch source + if self.distro == 'poky-tiny': + self.skipTest('libmatchbox2 is not buildable with poky-tiny because it requires a libx11 provider') tempdir = tempfile.mkdtemp(prefix='devtoolqa') self.track_for_cleanup(tempdir) url = 'git://git.yoctoproject.org/libmatchbox'
@@ -577,6 +578,8 @@ class DevtoolTests(DevtoolBase): @testcase(1378) def test_devtool_modify_virtual(self): + if self.distro == 'poky-tiny': + self.skipTest('Test not possible with poky-tiny because it requires a libx11 provider') # Try modifying a virtual recipe virtrecipe = 'virtual/libx11' realrecipe = 'libx11'
@@ -941,6 +944,8 @@ class DevtoolTests(DevtoolBase): @testcase(1379) def test_devtool_extract_virtual(self): + if self.distro == 'poky-tiny': + self.skipTest('Test not possible with poky-tiny because it requires a libx11 provider') tempdir = tempfile.mkdtemp(prefix='devtoolqa') # Try devtool extract self.track_for_cleanup(tempdir)
Signature · 1 line
2.1.4
Re: [PATCH 4/9] selftest: skip some devtool tests when distro is poky-tiny due to missing libx11
Quoted 9 lines · On 14 October 2016 at 17:40, <[email protected]> wrote:
def
test_devtool_add_fetch_git(self):meta/lib/oeqa/selftest/devtool.py +2
-1- # Fetch source + if self.distro == 'poky-tiny': + self.skipTest('libmatchbox2 is not buildable with poky-tinybecause it requires a libx11 provider')
tempdir =tempfile.mkdtemp(prefix='devtoolqa')self.track_for_cleanup(tempdir)
url = 'git://git.yoctoproject.org/libmatchbox'
Changing the test to use a recipe that doesn't require a distro feature that many distros disable seems like a more sustainable fix.
Ross
Re: [PATCH 4/9] selftest: skip some devtool tests when distro is poky-tiny due to missing libx11
Quoted 14 lines · Ross
On 14 October 2016 at 17:40, <[email protected] <mailto:[email protected]>> wrote:
def
test_devtool_add_fetch_git(self):
- # Fetch source
+ if self.distro == 'poky-tiny':
+ self.skipTest('libmatchbox2 is not buildable with
poky-tiny because it requires a libx11 provider')
tempdir =tempfile.mkdtemp(prefix='devtoolqa')self.track_for_cleanup(tempdir)
url = 'git://git.yoctoproject.org/libmatchbox
<http://git.yoctoproject.org/libmatchbox>'
Quoted 2 lines
Changing the test to use a recipe that doesn't require a distro feature that many distros disable seems like a more sustainable fix.
Which recipe do you think would be more generic?
Quoted 1 line
Ross
Re: [PATCH 4/9] selftest: skip some devtool tests when distro is poky-tiny due to missing libx11
Quoted 15 lines · Leonardo Sandoval
On 10/17/2016 06:42 AM, Burton, Ross
wrote:Quoted 4 lines
Quoted 1 line
Quoted 1 line
On 14 October 2016 at 17:40, <leonardo.sandoval.gonzalez@
linux.intel.com>
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines · wrote:
deftest_devtool_add_fetch_git(self):- # Fetch source
+ if self.distro == 'poky-tiny':
+ self.skipTest('libmatchbox2 is not
Quoted 1 line
Quoted 1 line
Quoted 1 line
buildable with poky-tiny because it requires a
libx11
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
provider')
Quoted 1 line
Quoted 1 line
Quoted 1 line
tempdir =
tempfile.mkdtemp(prefix='devtoolqa')
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
self.track_for_cleanup(tempdir)
Quoted 1 line
Quoted 1 line
Quoted 1 line
url =
'git://git.yoctoproject.org/libmatchbox'
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Changing the test to use a recipe that doesn't require a
Quoted 1 line
Quoted 1 line
distro feature that many distros disable seems like a
more
Quoted 9 lines
Quoted 8 lines
sustainable fix.
Which recipe do you think would be more generic?
Something in the minimal image which doesn't rely on DISTRO_FEATURES which aren't in the default distro.
Thanks,
Joshua
PATCH 5/9 selftest: skip those imagefeatures tests not relevant for certain distros
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
core-image-clutter|weston can only be built on poky and poky-lsb, so skip these tests in case other distro is defined. In the other hand, poky-tiny does not support ext4 filesystem types, so skip test_bmap which includes these image features.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/imagefeatures.py | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
meta/lib/oeqa/selftest/imagefeatures.py +15 -3
@@ -78,7 +78,11 @@ class ImageFeatures(oeSelfTest): """ # Build a core-image-clutter - bitbake('core-image-clutter') + image = 'core-image-clutter' + supported_distros = ['poky', 'poky-lsb'] + if not self.distro in supported_distros: + self.skipTest('Test is consider for distros (%s) supporting %s' % (','.join(supported_distros), image)) + bitbake(image) @testcase(1117) def test_wayland_support_in_image(self):
@@ -96,7 +100,11 @@ class ImageFeatures(oeSelfTest): self.write_config(features) # Build a core-image-weston - bitbake('core-image-weston') + image = 'core-image-weston' + supported_distros = ['poky', 'poky-lsb'] + if not self.distro in supported_distros: + self.skipTest('Test is only considered for distros (%s) supporting %s' % (','.join(supported_distros), image)) + bitbake(image) def test_bmap(self): """
@@ -107,7 +115,11 @@ class ImageFeatures(oeSelfTest): Author: Ed Bartosh <[email protected]> """ - features = 'IMAGE_FSTYPES += " ext4 ext4.bmap"' + ext4_features = "ext4 ext4.bmap" + features = 'IMAGE_FSTYPES += " %s"' % ext4_features + if self.distro == 'poky-tiny': + self.skipTest('Features (%s) not supported for poky-tiny' % ext4_features) + self.write_config(features) image_name = 'core-image-minimal'
Signature · 1 line
2.1.4
Re: [PATCH 5/9] selftest: skip those imagefeatures tests not relevant for certain distros
Quoted 9 lines · On 14 October 2016 at 17:40, <[email protected]> wrote:
# Build a core-image-clutter
meta/lib/oeqa/selftest/imagefeatures.py +4
-1- bitbake('core-image-clutter') + image = 'core-image-clutter' + supported_distros = ['poky', 'poky-lsb'] + if not self.distro in supported_distros: + self.skipTest('Test is consider for distros (%s) supporting%s' % (','.join(supported_distros), image))
+ bitbake(image)
Wouldn't it be better to check DISTRO_FEATURES instead - i.e. core-image-weston needs opengl and wayland DISTRO_FEATURES to be present.
Ross
PATCH 6/9 selftest: skip test_recipetool_create_git on poky-tiny distro
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
The test defines several bitbake targets, including libx11, this one in particular cannot be provided with a poky-tiny distro so skip it if this is case.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/recipetool.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/recipetool.py +2 -0
@@ -382,6 +382,8 @@ class RecipetoolTests(RecipetoolBase): @testcase(1194) def test_recipetool_create_git(self): + if self.distro == 'poky-tiny': + self.skipTest('Test not possible with poky-tiny because it requires a libx11 provider') # Ensure we have the right data in shlibs/pkgdata bitbake('libpng pango libx11 libxext jpeg libcheck') # Try adding a recipe
Signature · 1 line
2.1.4
Re: [PATCH 6/9] selftest: skip test_recipetool_create_git on poky-tiny distro
Quoted 4 lines · On 14 October 2016 at 17:40, <[email protected]> wrote:
(quoted diff) +2
-0+ if self.distro == 'poky-tiny': + self.skipTest('Test not possible with poky-tiny because itrequires a libx11 provider')
Same as the other one - change the test to use a recipe that doesn't need a DISTRO_FEATURE that is often disabled.
Ross
PATCH 7/9 selftest: skip runtime-test's test_testimage_install on poky-tiny distro
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Distro poky-tiny cannot build the core-image-full-cmdline image, so skip the test if this is the case.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/runtime-test.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/runtime-test.py +2 -0
@@ -94,6 +94,8 @@ class TestImage(oeSelfTest): Product: oe-core Author: Mariano Lopez <[email protected]> """ + if self.distro == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') features = 'INHERIT += "testimage"\n' features += 'TEST_SUITES = "ping ssh selftest"\n'
Signature · 1 line
2.1.4
Re: [PATCH 7/9] selftest: skip runtime-test's test_testimage_install on poky-tiny distro
Quoted 7 lines · On Fri, 2016-10-14 at 11:40 -0500, [email protected]…
From: Leonardo Sandoval <[email protected]>
Distro poky-tiny cannot build the core-image-full-cmdline image, so skip the
test if this is the case.[YOCTO #8525]
Wouldn't it be better to use a different image, most likely with extra packages installed, and maybe a different list of TEST_SUITES for poky- tiny so that we're still testing this functionality on poky-tiny?
Thanks,
Joshua
Quoted 25 lines
Signed-off-by Leonardo Sandoval <[email protected]tel.com>
---
meta/lib/oeqa/selftest/runtime-test.py | 2 ++1 file changed, 2 insertions(+)meta/lib/oeqa/selftest/runtime-test.py +0
-0@@ -94,6 +94,8 @@ class TestImage(oeSelfTest):Product: oe-core
Author: Mariano Lopez <[email protected]> """
+ if self.distro == 'poky-tiny':
+ self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
features = 'INHERIT += "testimage"\n'
features += 'TEST_SUITES= "ping ssh selftest"\n' --
2.1.4
Re: [PATCH 7/9] selftest: skip runtime-test's test_testimage_install on poky-tiny distro
Quoted 4 lines · Joshua Lock
Quoted 2 lines · On Fri, 2016-10-14 at 11:40 -0500, [email protected]…
From: Leonardo Sandoval <[email protected]>
Quoted 8 lines
Quoted 5 lines
Distro poky-tiny cannot build the core-image-full-cmdline image, so skip the
test if this is the case.[YOCTO #8525]
Wouldn't it be better to use a different image, most likely with extra packages installed, and maybe a different list of
TEST_SUITESfor poky- tiny so that we're still testing this functionality on poky-tiny?
Yes, that is a good idea. Which image and the accompanied tests do you suggest?
Quoted 28 lines
Thanks,
Joshua
Quoted 24 lines
Signed-off-by Leonardo Sandoval <[email protected]tel.com>
--- meta/lib/oeqa/selftest/runtime-test.py | 2 ++ 1 file changed, 2 insertions(+)meta/lib/oeqa/selftest/runtime-test.py +2
-0@@ -94,6 +94,8 @@ class TestImage(oeSelfTest): Product: oe-core Author: Mariano Lopez <[email protected]> """ + if self.distro == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') features = 'INHERIT += "testimage"\n'features += 'TEST_SUITES = "ping ssh selftest"\n'Signature · 1 line
2.1.4
Re: [PATCH 7/9] selftest: skip runtime-test's test_testimage_install on poky-tiny distro
Quoted 21 lines · Leonardo Sandoval
Quoted 12 lines · Joshua Lock
Quoted 9 lines · On Fri, 2016-10-14 at 11:40 -0500, [email protected]…
From: Leonardo Sandoval <[email protected] om>
Distro poky-tiny cannot build the core-image-full-cmdline image, so
skip the
test if this is the case.Quoted 6 lines
Quoted 1 line
[YOCTO #8525]
Wouldn't it be better to use a different image, most likely with extra
packages installed, and maybe a different list ofTEST_SUITESfor poky-
tiny so that we're still testing this functionality on poky-tiny?
Quoted 3 lines
Yes, that is a good idea. Which image and the accompanied tests do you
suggest?
I'd be inclined to suggest core-image-minimal with extra packages installed to enable the same TEST_SUITES, but I can't recall exactly what's being tested here. Would that combination make sense?
Thanks,
Joshua
Quoted 34 lines
Quoted 33 lines
Thanks,
Joshua
Quoted 27 lines
Signed-off-by Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.in
tel.com>
---
meta/lib/oeqa/selftest/runtime-test.py| 2 ++1 file changed, 2 insertions(+)meta/lib/oeqa/selftest/runtime-test.py +0
-0@@ -94,6 +94,8 @@ class TestImage(oeSelfTest):Product: oe-core
Author: Mariano Lopez <[email protected]> """
+ if self.distro == 'poky-tiny':
+ self.skipTest('core-image-full-cmdline not buildable for
poky-tiny')
features = 'INHERIT += "testimage"\n' features += 'TEST_SUITES= "ping ssh selftest"\n' --
2.1.4
PATCH 8/9 selftest: skip sstatetests methods considering distro and its features
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Some tests builds images that poky-tiny cannot build so skip those. Also, bitbake world fails if distro does not contain opengl, so skip those tests if opengl is not present. One other change was made into the sstate
module: rename self.distro to a better name (self.hostdistro)
[YOCTO #8525]
--- meta/lib/oeqa/selftest/sstate.py | 10 +++--- meta/lib/oeqa/selftest/sstatetests.py | 57 ++++++++++++++++++++++++++++++----- 2 files changed, 54 insertions(+), 13 deletions(-)
meta/lib/oeqa/selftest/sstate.py +5 -5
@@ -14,8 +14,8 @@ class SStateBase(oeSelfTest): def setUpLocal(self): self.temp_sstate_location = None self.sstate_path = get_bb_var('SSTATE_DIR') - self.distro = get_bb_var('NATIVELSBSTRING') - self.distro_specific_sstate = os.path.join(self.sstate_path, self.distro) + self.hostdistro = get_bb_var('NATIVELSBSTRING') + self.distro_specific_sstate = os.path.join(self.sstate_path, self.hostdistro) # Creates a special sstate configuration with the option to add sstate mirrors def config_sstate(self, temp_sstate_location=False, add_local_mirrors=[]):
@@ -27,8 +27,8 @@ class SStateBase(oeSelfTest): self.append_config(config_temp_sstate) self.track_for_cleanup(temp_sstate_path) self.sstate_path = get_bb_var('SSTATE_DIR') - self.distro = get_bb_var('NATIVELSBSTRING') - self.distro_specific_sstate = os.path.join(self.sstate_path, self.distro) + self.hostdistro = get_bb_var('NATIVELSBSTRING') + self.distro_specific_sstate = os.path.join(self.sstate_path, self.hostdistro) if add_local_mirrors: config_set_sstate_if_not_set = 'SSTATE_MIRRORS ?= ""'
@@ -42,7 +42,7 @@ class SStateBase(oeSelfTest): def search_sstate(self, filename_regex, distro_specific=True, distro_nonspecific=True): result = [] for root, dirs, files in os.walk(self.sstate_path): - if distro_specific and re.search("%s/[a-z0-9]{2}$" % self.distro, root): + if distro_specific and re.search("%s/[a-z0-9]{2}$" % self.hostdistro, root): for f in files: if re.search(filename_regex, f): result.append(f)
meta/lib/oeqa/selftest/sstatetests.py +49 -8
122 lines not on this page · raw message
Signature · 1 line
2.1.4
PATCH 9/9 selftest: skip wic test methods for poky-tiny distro
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
The poky-tiny distro does not supported the intended images to be created with wic, so skp these tests.
[YOCTO #8525]
--- meta/lib/oeqa/selftest/wic.py | 9 +++++++++ 1 file changed, 9 insertions(+)
meta/lib/oeqa/selftest/wic.py +9 -0
@@ -190,6 +190,8 @@ class Wic(oeSelfTest): @testcase(1346) def test_iso_image(self): """Test creation of hybrid iso image with legacy and EFI boot""" + if self.distro == 'poky-tiny': + self.skipTest('wic image (mkhybridiso) not supported for poky-tiny distro') self.assertEqual(0, runCmd("wic create mkhybridiso " "--image-name core-image-minimal").status) self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.direct")))
@@ -246,6 +248,9 @@ class Wic(oeSelfTest): def test_mkgummidisk(self): """Test creation of mkgummidisk image""" image = "mkgummidisk" + if self.distro == 'poky-tiny': + self.skipTest('wic image name (%s) not supported for poky-tiny distro' % image) + self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
@@ -254,6 +259,8 @@ class Wic(oeSelfTest): def test_mkefidisk(self): """Test creation of mkefidisk image""" image = "mkefidisk" + if self.distro == 'poky-tiny': + self.skipTest('wic image (%s) not supported for poky-tiny distro' % image) self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
@@ -288,6 +295,8 @@ class Wic(oeSelfTest): def test_systemd_bootdisk(self): """Test creation of systemd-bootdisk image""" image = "systemd-bootdisk" + if self.distro == 'poky-tiny': + self.skipTest('wic image (%s) not supported for poky-tiny distro' % image) self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
Signature · 1 line
2.1.4
v2 cover Skip selftests depending on distro and its features
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Originally, the selftests were defined for the 'poky' distro. However, there are more supported distros, including nodistro, poky-lsb and poky-tiny. These series takes into consideration the distro name and its features to skip certain tests.
[YOCTO #8525]
The following changes since commit 12a0ee049e453b6d0d2ce2f3fa981d1b6e02bd78:
dev-manual: Added note about RPM not dealing with post-install (2016-11-23 11:10:35 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib lsandov1/oe-selftest-distro-agnostic
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lsandov1/oe-selftest-distro-agnostic
Leonardo Sandoval (13):
selftest: base: new object member to store the DISTRO value
selftest: bbtests: use minimal image so all distros can execute it
selftest: bbtests: run non-gplv3 test only on relevant distros
selftest: buildoptions: skip read-only-image test depending on distro
selftest: buildoptions: skip test in case features are missing
selftest: devtool: use mraa instead of libmatchbox to lessen distro
requirements
selftest: imagefeatures: skip tests in case distro feature is missing
selftest: imagefeatures: skip bmap test in case of poky-tiny
selftest: recipetool: use mraa instead of libmatchbox to lessen distro
requirements
selftest: runtime-test: skip image-install test for poky-tiny
selftest: sstatetests: skip methods in case of poky-tiny or opengl is
missing
selftest: wic: skip tests for poky-tiny distro policy
selftest: bblayers: remove linux kernel checks for show-recipes check
meta/lib/oeqa/selftest/base.py | 1 + meta/lib/oeqa/selftest/bblayers.py | 6 +--- meta/lib/oeqa/selftest/bbtests.py | 8 +++-- meta/lib/oeqa/selftest/buildoptions.py | 3 ++ meta/lib/oeqa/selftest/devtool.py | 17 ++++++---- meta/lib/oeqa/selftest/imagefeatures.py | 9 +++++- meta/lib/oeqa/selftest/recipetool.py | 27 ++++++++++------ meta/lib/oeqa/selftest/runtime-test.py | 2 ++ meta/lib/oeqa/selftest/sstate.py | 10 +++--- meta/lib/oeqa/selftest/sstatetests.py | 57 ++++++++++++++++++++++++++++----- meta/lib/oeqa/selftest/wic.py | 9 ++++++ 11 files changed, 112 insertions(+), 37 deletions(-)
Signature · 1 line
2.1.4
PATCH v2 1/13 selftest: base: new object member to store the DISTRO value
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Instead of quering it multiple times, query once and use it on test method skip checks. Also, rename current distro sstate object member to a more meaninful name.
--- meta/lib/oeqa/selftest/base.py | 1 + meta/lib/oeqa/selftest/sstate.py | 10 +++++----- 2 files changed, 6 insertions(+), 5 deletions(-)
meta/lib/oeqa/selftest/base.py +1 -0
@@ -41,6 +41,7 @@ class oeSelfTest(unittest.TestCase): self.testinc_path, self.testinc_bblayers_path, self.machineinc_path, self.localconf_backup, self.local_bblayers_backup] + self.distro = get_bb_var('DISTRO') super(oeSelfTest, self).__init__(methodName) def setUp(self):
meta/lib/oeqa/selftest/sstate.py +5 -5
@@ -14,8 +14,8 @@ class SStateBase(oeSelfTest): def setUpLocal(self): self.temp_sstate_location = None self.sstate_path = get_bb_var('SSTATE_DIR') - self.distro = get_bb_var('NATIVELSBSTRING') - self.distro_specific_sstate = os.path.join(self.sstate_path, self.distro) + self.hostdistro = get_bb_var('NATIVELSBSTRING') + self.distro_specific_sstate = os.path.join(self.sstate_path, self.hostdistro) # Creates a special sstate configuration with the option to add sstate mirrors def config_sstate(self, temp_sstate_location=False, add_local_mirrors=[]):
@@ -27,8 +27,8 @@ class SStateBase(oeSelfTest): self.append_config(config_temp_sstate) self.track_for_cleanup(temp_sstate_path) self.sstate_path = get_bb_var('SSTATE_DIR') - self.distro = get_bb_var('NATIVELSBSTRING') - self.distro_specific_sstate = os.path.join(self.sstate_path, self.distro) + self.hostdistro = get_bb_var('NATIVELSBSTRING') + self.distro_specific_sstate = os.path.join(self.sstate_path, self.hostdistro) if add_local_mirrors: config_set_sstate_if_not_set = 'SSTATE_MIRRORS ?= ""'
@@ -42,7 +42,7 @@ class SStateBase(oeSelfTest): def search_sstate(self, filename_regex, distro_specific=True, distro_nonspecific=True): result = [] for root, dirs, files in os.walk(self.sstate_path): - if distro_specific and re.search("%s/[a-z0-9]{2}$" % self.distro, root): + if distro_specific and re.search("%s/[a-z0-9]{2}$" % self.hostdistro, root): for f in files: if re.search(filename_regex, f): result.append(f)
Signature · 1 line
2.1.4
PATCH v2 2/13 selftest: bbtests: use minimal image so all distros can execute it
From: Leonardo Sandoval <[email protected]>
poky-tiny distro cannot build full-cmdline image, so use an image (core-image-minimal) that can be built in all distros.
--- meta/lib/oeqa/selftest/bbtests.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/lib/oeqa/selftest/bbtests.py +2 -2
@@ -114,11 +114,11 @@ class BitbakeTests(oeSelfTest): @testcase(167) def test_bitbake_g(self): - result = bitbake('-g core-image-full-cmdline') + result = bitbake('-g core-image-minimal') for f in ['pn-buildlist', 'pn-depends.dot', 'package-depends.dot', 'task-depends.dot']: self.addCleanup(os.remove, f) self.assertTrue('NOTE: PN build list saved to \'pn-buildlist\'' in result.output, msg = "No dependency \"pn-buildlist\" file was generated for the given task target. bitbake output: %s" % result.output) - self.assertTrue('openssh' in ftools.read_file(os.path.join(self.builddir, 'pn-buildlist')), msg = "No \"openssh\" dependency found in pn-buildlist file.") + self.assertTrue('busybox' in ftools.read_file(os.path.join(self.builddir, 'pn-buildlist')), msg = "No \"busybox\" dependency found in pn-buildlist file.") @testcase(899) def test_image_manifest(self):
Signature · 1 line
2.1.4
PATCH v2 3/13 selftest: bbtests: run non-gplv3 test only on relevant distros
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
The non-gplv3 test makes sense just for poky and poky-tiny distros which include only non-gplv3 packages, thus skip other distros.
--- meta/lib/oeqa/selftest/bbtests.py | 4 ++++ 1 file changed, 4 insertions(+)
meta/lib/oeqa/selftest/bbtests.py +4 -0
@@ -229,6 +229,10 @@ INHERIT_remove = \"report-error\" @testcase(1119) def test_non_gplv3(self): + supported_distros = ['poky', 'poky-tiny'] + if self.distro not in supported_distros: + self.skipTest('Test considers only %s distros' % ','.join(supported_distros)) + data = 'INCOMPATIBLE_LICENSE = "GPLv3"' conf = os.path.join(self.builddir, 'conf/local.conf') ftools.append_file(conf ,data)
Signature · 1 line
2.1.4
PATCH v2 4/13 selftest: buildoptions: skip read-only-image test depending on distro
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Poky-tiny cannot build core-image-sato, so skip test (read-only-image) in this case.
--- meta/lib/oeqa/selftest/buildoptions.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/buildoptions.py +2 -0
@@ -43,6 +43,8 @@ class ImageOptionsTests(oeSelfTest): @testcase(1435) def test_read_only_image(self): + if self.distro == 'poky-tiny': + self.skipTest('core-image-sato is not buildable with poky-tiny') self.write_config('IMAGE_FEATURES += "read-only-rootfs"') bitbake("core-image-sato") # do_image will fail if there are any pending postinsts
Signature · 1 line
2.1.4
PATCH v2 5/13 selftest: buildoptions: skip test in case features are missing
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
The sato image needs opengl and x11 as distro features, so skip test if this is not the case.
--- meta/lib/oeqa/selftest/buildoptions.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
meta/lib/oeqa/selftest/buildoptions.py +3 -2
@@ -43,8 +43,9 @@ class ImageOptionsTests(oeSelfTest): @testcase(1435) def test_read_only_image(self): - if self.distro == 'poky-tiny': - self.skipTest('core-image-sato is not buildable with poky-tiny') + distro_features = get_bb_var('DISTRO_FEATURES') + if not ('x11' in distro_features and 'opengl' in distro_features): + self.skipTest('core-image-sato requires x11 and opengl in distro features') self.write_config('IMAGE_FEATURES += "read-only-rootfs"') bitbake("core-image-sato") # do_image will fail if there are any pending postinsts
Signature · 1 line
2.1.4
PATCH v2 6/13 selftest: devtool: use mraa instead of libmatchbox to lessen distro requirements
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
libmatchbox requires x11 distro feature to be present, so use another recipe (mraa) with no extra requirements, allowing test execution of all supported distros.
--- meta/lib/oeqa/selftest/devtool.py | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
meta/lib/oeqa/selftest/devtool.py +10 -7
@@ -340,12 +340,11 @@ class DevtoolTests(DevtoolBase): @testcase(1161) def test_devtool_add_fetch_git(self): - # Fetch source tempdir = tempfile.mkdtemp(prefix='devtoolqa') self.track_for_cleanup(tempdir) - url = 'git://git.yoctoproject.org/libmatchbox' - checkrev = '462f0652055d89c648ddd54fd7b03f175c2c6973' - testrecipe = 'libmatchbox2' + url = 'gitsm://git.yoctoproject.org/mraa' + checkrev = 'ae127b19a50aa54255e4330ccfdd9a5d058e581d' + testrecipe = 'mraa' srcdir = os.path.join(tempdir, testrecipe) # Test devtool add self.track_for_cleanup(self.workspacedir)
@@ -353,7 +352,7 @@ class DevtoolTests(DevtoolBase): self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') result = runCmd('devtool add %s %s -a -f %s' % (testrecipe, srcdir, url)) self.assertTrue(os.path.exists(os.path.join(self.workspacedir, 'conf', 'layer.conf')), 'Workspace directory not created: %s' % result.output) - self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure.ac')), 'Unable to find configure.ac in source directory') + self.assertTrue(os.path.isfile(os.path.join(srcdir, 'imraa', 'imraa.c')), 'Unable to find configure.ac in source directory') # Test devtool status result = runCmd('devtool status') self.assertIn(testrecipe, result.output)
@@ -363,7 +362,7 @@ class DevtoolTests(DevtoolBase): self.assertIn('_git.bb', recipefile, 'Recipe file incorrectly named') checkvars = {} checkvars['S'] = '${WORKDIR}/git' - checkvars['PV'] = '1.12+git${SRCPV}' + checkvars['PV'] = '1.0+git${SRCPV}' checkvars['SRC_URI'] = url checkvars['SRCREV'] = '${AUTOREV}' self._test_recipe_contents(recipefile, checkvars, [])
@@ -372,7 +371,7 @@ class DevtoolTests(DevtoolBase): shutil.rmtree(srcdir) url_rev = '%s;rev=%s' % (url, checkrev) result = runCmd('devtool add %s %s -f "%s" -V 1.5' % (testrecipe, srcdir, url_rev)) - self.assertTrue(os.path.isfile(os.path.join(srcdir, 'configure.ac')), 'Unable to find configure.ac in source directory') + self.assertTrue(os.path.isfile(os.path.join(srcdir, 'imraa', 'imraa.c')), 'Unable to find imraa/imraa.c in source directory') # Test devtool status result = runCmd('devtool status') self.assertIn(testrecipe, result.output)
@@ -580,6 +579,8 @@ class DevtoolTests(DevtoolBase): @testcase(1378) def test_devtool_modify_virtual(self): + if self.distro == 'poky-tiny': + self.skipTest('Test not possible with poky-tiny because it requires a libx11 provider') # Try modifying a virtual recipe virtrecipe = 'virtual/libx11' realrecipe = 'libx11'
@@ -1013,6 +1014,8 @@ class DevtoolTests(DevtoolBase): @testcase(1379) def test_devtool_extract_virtual(self): + if self.distro == 'poky-tiny': + self.skipTest('Test not possible with poky-tiny because it requires a libx11 provider') tempdir = tempfile.mkdtemp(prefix='devtoolqa') # Try devtool extract self.track_for_cleanup(tempdir)
Signature · 1 line
2.1.4
PATCH v2 7/13 selftest: imagefeatures: skip tests in case distro feature is missing
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
core-image-clutter and core-image-weston, both required opengl in distro features, skip relevant tests if this is not the case.
--- meta/lib/oeqa/selftest/imagefeatures.py | 4 ++++ 1 file changed, 4 insertions(+)
meta/lib/oeqa/selftest/imagefeatures.py +4 -0
@@ -78,6 +78,8 @@ class ImageFeatures(oeSelfTest): """ # Build a core-image-clutter + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('opengl not present on DISTRO_FEATURES so core-image-clutter cannot be built') bitbake('core-image-clutter') @testcase(1117)
@@ -96,6 +98,8 @@ class ImageFeatures(oeSelfTest): self.write_config(features) # Build a core-image-weston + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('opengl not present on DISTRO_FEATURES so core-image-weston cannot be built') bitbake('core-image-weston') def test_bmap(self):
Signature · 1 line
2.1.4
Re: [PATCH v2 07/13] selftest: imagefeatures: skip tests in case distro feature is missing
Quoted 17 lines · On 24 November 2016 at 20:58, <[email protected]>…
From: Leonardo Sandoval <[email protected]>
core-image-clutter and core-image-weston, both required opengl in distro features, skip relevant tests if this is not the case.
Signed-off-by Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
--- meta/lib/oeqa/selftest/imagefeatures.py | 4 ++++ 1 file changed, 4 insertions(+)meta/lib/oeqa/selftest/imagefeatures.py +0
-0
meta/lib/oeqa/selftest/imagefeatures.py class ImageFeatures(oeSelfTest): · line 85
Quoted 8 lines
meta/lib/oeqa/selftest/imagefeatures.py +2
-0@@ -78,6 +78,8 @@ class ImageFeatures(oeSelfTest): """ # Build a core-image-clutter + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('opengl not present on DISTRO_FEATURES so core-image-clutter cannot be built') bitbake('core-image-clutter')
I don't see the point of this test. Lets save five minutes by just deleting it.
meta/lib/oeqa/selftest/imagefeatures.py class ImageFeatures(oeSelfTest): · line 104
Quoted 9 lines
@testcase(1117)meta/lib/oeqa/selftest/imagefeatures.py +2
-0@@ -96,6 +98,8 @@ class ImageFeatures(oeSelfTest): self.write_config(features) # Build a core-image-weston + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('opengl not present on DISTRO_FEATURES so core-image-weston cannot be built') bitbake('core-image-weston')
The full test is:
features = 'DISTRO_FEATURES_append = " wayland"\n' features += 'CORE_IMAGE_EXTRA_INSTALL += "wayland weston"' self.write_config(features)
# Build a core-image-weston
bitbake('core-image-weston')
Adding wayland and weston to IMAGE_INSTALL is pointless as thats what the image does. Adding wayland is madness if the distro doesn't support it.
Can you rewrite this test so it's basically just "if DISTRO_FEATURES contains opengl and wayland, build core-image-weston".
Ross
PATCH v2 8/13 selftest: imagefeatures: skip bmap test in case of poky-tiny
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Poky-tiny distro cannot execute bmap test because ext4 filesystem is used, so skip test in this is the case.
--- meta/lib/oeqa/selftest/imagefeatures.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/imagefeatures.py +4 -1
@@ -111,7 +111,10 @@ class ImageFeatures(oeSelfTest): Author: Ed Bartosh <[email protected]> """ - features = 'IMAGE_FSTYPES += " ext4 ext4.bmap"' + ext4_features = "ext4 ext4.bmap" + features = 'IMAGE_FSTYPES += " %s"' % ext4_features + if self.distro == 'poky-tiny': + self.skipTest('Features (%s) not supported for poky-tiny' % ext4_features) self.write_config(features) image_name = 'core-image-minimal'
Signature · 1 line
2.1.4
PATCH v2 9/13 selftest: recipetool: use mraa instead of libmatchbox to lessen distro requirements
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
libmatchbox requires x11 distro feature to be present, so use another recipe (mraa) with no extra requirements, allowing test execution of all supported distros.
--- meta/lib/oeqa/selftest/recipetool.py | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-)
meta/lib/oeqa/selftest/recipetool.py +18 -9
@@ -383,22 +383,31 @@ class RecipetoolTests(RecipetoolBase): @testcase(1194) def test_recipetool_create_git(self): # Ensure we have the right data in shlibs/pkgdata - bitbake('libpng pango libx11 libxext jpeg libcheck') + bitbake('swig-native') # Try adding a recipe tempsrc = os.path.join(self.tempdir, 'srctree') os.makedirs(tempsrc) - recipefile = os.path.join(self.tempdir, 'libmatchbox.bb') - srcuri = 'git://git.yoctoproject.org/libmatchbox' - result = runCmd(['recipetool', 'create', '-o', recipefile, srcuri + ";rev=9f7cf8895ae2d39c465c04cc78e918c157420269", '-x', tempsrc]) + recipefile = os.path.join(self.tempdir, 'mraa.bb') + srcuri = 'git://git.yoctoproject.org/mraa' + result = runCmd(['recipetool', 'create', '-o', recipefile, srcuri + ";rev=a2277e9f8ca0b4562a42ce3476adf8be71886571", '-x', tempsrc]) + # MRAA has MIT license (http://git.yoctoproject.org/cgit/cgit.cgi/mraa/tree/COPYING), set it instead of "MIT & Unknown & LGPLv2.1" found by the tool + result = runCmd(['sed', '-i', '-e', 's/^LICENSE =.*/LICENCE = "MIT"/', recipefile]) self.assertTrue(os.path.isfile(recipefile), 'recipetool did not create recipe file; output:\n%s' % result.output) checkvars = {} - checkvars['LICENSE'] = 'LGPLv2.1' - checkvars['LIC_FILES_CHKSUM'] = 'file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34' + checkvars['LICENSE'] = 'MIT' + checkvars['LIC_FILES_CHKSUM'] = 'file://COPYING;md5=66493d54e65bfc12c7983ff2e884f37f \ + file://cmake/modules/LICENSE_1_0.txt;md5=e4224ccaecb14d942c71d31bef20d78c \ + file://cmake/modules/LICENSE.LGPL;md5=fbc093901857fcd118f065f900982c24 \ + file://cmake/modules/Copyright.txt;md5=3ba5a6c34481652ce573e5c4e1d707e4 \ + file://cmake/modules/COPYING-CMAKE-SCRIPTS;md5=2d149a0f4588c9f5e60c729e786dad45 \ + file://src/i2c/LICENSE;md5=48c7b6208b3868cd901b3248010b4d5d \ + file://doxygen2jsdoc/COPYING;md5=e8db6501ed294e65418a933925d12058 \ + file://doxygen2jsdoc/generators/yuidoc/tmpl/LICENSE;md5=7bd082d54c3a2328a1844a1d8b797f77' checkvars['S'] = '${WORKDIR}/git' - checkvars['PV'] = '1.11+git${SRCPV}' + checkvars['PV'] = '1.0+git${SRCPV}' checkvars['SRC_URI'] = srcuri - checkvars['DEPENDS'] = set(['libcheck', 'libjpeg-turbo', 'libpng', 'libx11', 'libxext', 'pango']) - inherits = ['autotools', 'pkgconfig'] + checkvars['DEPENDS'] = set(['swig-native']) + inherits = ['cmake', 'pythonnative', 'python-dir'] self._test_recipe_contents(recipefile, checkvars, inherits) @testcase(1392)
Signature · 1 line
2.1.4
PATCH v2 10/13 selftest: runtime-test: skip image-install test for poky-tiny
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
poky-tiny cannot build full-cmdline image, so skip this test in this case.
--- meta/lib/oeqa/selftest/runtime-test.py | 2 ++ 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/runtime-test.py +2 -0
@@ -94,6 +94,8 @@ class TestImage(oeSelfTest): Product: oe-core Author: Mariano Lopez <[email protected]> """ + if self.distro == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') features = 'INHERIT += "testimage"\n' features += 'TEST_SUITES = "ping ssh selftest"\n'
Signature · 1 line
2.1.4
PATCH v2 11/13 selftest: sstatetests: skip methods in case of poky-tiny or opengl is missing
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Some tests defines images that poky-tiny cannot build so skip them. Also, bitbake world fails if distro does not contain opengl, so skip also those.
--- meta/lib/oeqa/selftest/sstatetests.py | 57 ++++++++++++++++++++++++++++++----- 1 file changed, 49 insertions(+), 8 deletions(-)
meta/lib/oeqa/selftest/sstatetests.py +49 -8
@@ -42,20 +42,36 @@ class SStateTests(SStateBase): @testcase(975) def test_sstate_creation_distro_specific_pass(self): targetarch = get_bb_var('TUNE_ARCH') - self.run_test_sstate_creation(['binutils-cross-'+ targetarch, 'binutils-native'], distro_specific=True, distro_nonspecific=False, temp_sstate_location=True) + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['binutils-cross-'+ targetarch, 'binutils-native'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_sstate_creation(targets, distro_specific=True, distro_nonspecific=False, temp_sstate_location=True) @testcase(1374) def test_sstate_creation_distro_specific_fail(self): targetarch = get_bb_var('TUNE_ARCH') - self.run_test_sstate_creation(['binutils-cross-'+ targetarch, 'binutils-native'], distro_specific=False, distro_nonspecific=True, temp_sstate_location=True, should_pass=False) + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['binutils-cross-'+ targetarch, 'binutils-native'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_sstate_creation(targets, distro_specific=False, distro_nonspecific=True, temp_sstate_location=True, should_pass=False) @testcase(976) def test_sstate_creation_distro_nonspecific_pass(self): - self.run_test_sstate_creation(['glibc-initial'], distro_specific=False, distro_nonspecific=True, temp_sstate_location=True) + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['glibc-initial'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_sstate_creation(targets, distro_specific=False, distro_nonspecific=True, temp_sstate_location=True) @testcase(1375) def test_sstate_creation_distro_nonspecific_fail(self): - self.run_test_sstate_creation(['glibc-initial'], distro_specific=True, distro_nonspecific=False, temp_sstate_location=True, should_pass=False) + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['glibc-initial'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_sstate_creation(targets, distro_specific=True, distro_nonspecific=False, temp_sstate_location=True, should_pass=False) # Test the sstate files deletion part of the do_cleansstate task
@@ -78,17 +94,28 @@ class SStateTests(SStateBase): @testcase(977) def test_cleansstate_task_distro_specific_nonspecific(self): targetarch = get_bb_var('TUNE_ARCH') - self.run_test_cleansstate_task(['binutils-cross-' + targetarch, 'binutils-native', 'glibc-initial'], distro_specific=True, distro_nonspecific=True, temp_sstate_location=True) + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['binutils-cross-' + targetarch, 'binutils-native', 'glibc-initial'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_cleansstate_task(targets, distro_specific=True, distro_nonspecific=True, temp_sstate_location=True) @testcase(1376) def test_cleansstate_task_distro_nonspecific(self): - self.run_test_cleansstate_task(['glibc-initial'], distro_specific=False, distro_nonspecific=True, temp_sstate_location=True) + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['glibc-initial'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_cleansstate_task(targets, distro_specific=False, distro_nonspecific=True, temp_sstate_location=True) @testcase(1377) def test_cleansstate_task_distro_specific(self): targetarch = get_bb_var('TUNE_ARCH') - self.run_test_cleansstate_task(['binutils-cross-'+ targetarch, 'binutils-native', 'glibc-initial'], distro_specific=True, distro_nonspecific=False, temp_sstate_location=True) - + # Execute the test in all distros expect poky-tiny where glibc-initial is not provided + targets = ['binutils-cross-'+ targetarch, 'binutils-native', 'glibc-initial'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the following targets %s' % (self.distro, ','.join(targets))) + self.run_test_cleansstate_task(targets, distro_specific=True, distro_nonspecific=False, temp_sstate_location=True) # Test rebuilding of distro-specific sstate files def run_test_rebuild_distro_specific_sstate(self, targets, temp_sstate_location=True):
@@ -228,6 +255,8 @@ class SStateTests(SStateBase): build machines and running a builds, override the variables calling uname() manually and check using bitbake -S. """ + if self.distro == 'poky-tiny': + self.skipTest('core-image-sato not buildable for poky-tiny') topdir = get_bb_var('TOPDIR') targetvendor = get_bb_var('TARGET_VENDOR')
@@ -276,6 +305,8 @@ PACKAGE_CLASSES = "package_rpm package_ipk package_deb" detected. Rather than requiring two different build machines and running builds, override the variables manually and check using bitbake -S. """ + if self.distro == 'poky-tiny': + self.skipTest('core-image-sato not buildable for poky-tiny') topdir = get_bb_var('TOPDIR') self.write_config("""
@@ -310,6 +341,8 @@ NATIVELSBSTRING = \"DistroB\" Also, rather than duplicate the test, check nativesdk stamps are the same between the two MACHINE values. """ + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('Distro %s does not include opengl feature, required to build intended (bitbake) targets' % self.distro) configA = """ TMPDIR = \"${TOPDIR}/tmp-sstatesamehash\"
@@ -328,6 +361,8 @@ MACHINE = \"qemuarm\" Also, rather than duplicate the test, check nativesdk stamps are the same between the two MACHINE values. """ + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('Distro %s does not include opengl feature, required to build intended (bitbake) targets' % self.distro) configA = """ TMPDIR = \"${TOPDIR}/tmp-sstatesamehash\"
@@ -387,6 +422,9 @@ MULTILIBS = \"\" qemux86copy machine to test this. Also include multilibs in the test. """ + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('Distro %s does not include opengl feature, required to build intended (bitbake) targets' % self.distro) + topdir = get_bb_var('TOPDIR') targetos = get_bb_var('TARGET_OS') targetvendor = get_bb_var('TARGET_VENDOR')
@@ -433,6 +471,9 @@ DEFAULTTUNE_virtclass-multilib-lib32 = "x86" classes inherits should be the same. """ + if 'opengl' not in get_bb_var('DISTRO_FEATURES'): + self.skipTest('Distro %s does not include opengl feature, required to build intended (bitbake) targets' % self.distro) + topdir = get_bb_var('TOPDIR') targetvendor = get_bb_var('TARGET_VENDOR') self.write_config("""
Signature · 1 line
2.1.4
Re: [PATCH v2 11/13] selftest: sstatetests: skip methods in case of poky-tiny or opengl is missing
Quoted 17 lines · On 24 November 2016 at 20:58, <[email protected]>…
meta/lib/oeqa/selftest/sstatetests.py +1
-1@@ -42,20 +42,36 @@ class SStateTests(SStateBase): @testcase(975) def test_sstate_creation_distro_specific_pass(self): targetarch = get_bb_var('TUNE_ARCH') - self.run_test_sstate_creation(['binutils-cross-'+ targetarch, 'binutils-native'], distro_specific=True, distro_nonspecific=False, temp_sstate_location=True) + # Execute the test in all distros expect poky-tiny whereglibc-initial is not provided
meta/lib/oeqa/selftest/sstatetests.py +3
-0+ targets = ['binutils-cross-'+ targetarch, 'binutils-native'] + if self.distro == 'poky-tiny': + self.skipTest('Distro %s does not support building the
Quoted 4 lines
following targets %s' % (self.distro, ','.join(targets)))
+ self.run_test_sstate_creation(targets, distro_specific=True,
distro_nonspecific=False, temp_sstate_location=True)
Sounds like it would be easier to just remove glibc-initial from that list, and either remove it entirely or replace it with a virtual name that both glibc and musl provide.
Quoted 11 lines
meta/lib/oeqa/selftest/sstatetests.py +2
-0@@ -228,6 +255,8 @@ class SStateTests(SStateBase): build machines and running a builds, override the variables calling uname() manually and check using bitbake -S. """ + if self.distro == 'poky-tiny': + self.skipTest('core-image-sato not buildable for poky-tiny') topdir = get_bb_var('TOPDIR')targetvendor =
get_bb_var('TARGET_VENDOR')
For all of these tests that just do bitbake -S core-image-sato, if world works then that is more flexible and more comprehensive.
Ross
Re: [PATCH v2 11/13] selftest: sstatetests: skip methods in case of poky-tiny or opengl is missing
Quoted 25 lines · Ross
On 24 November 2016 at 20:58, <[email protected] <mailto:[email protected]>> wrote:
diff --git
a/meta/lib/oeqa/selftest/sstatetests.pyb/meta/lib/oeqa/selftest/sstatetests.py
index 6642539..8ea3932 100644
---a/meta/lib/oeqa/selftest/sstatetests.py
+++b/meta/lib/oeqa/selftest/sstatetests.py
@@ -42,20 +42,36 @@ class SStateTests(SStateBase):
@testcase(975)
deftest_sstate_creation_distro_specific_pass(self):
targetarch =get_bb_var('TUNE_ARCH')
- self.run_test_sstate_creation(['binutils-cross-'+
targetarch, 'binutils-native'], distro_specific=True, distro_nonspecific=False, temp_sstate_location=True)
+ # Execute the test in all distros expect poky-tiny where
glibc-initial is not provided
+ targets = ['binutils-cross-'+ targetarch, 'binutils-native']
+ if self.distro == 'poky-tiny':
+ self.skipTest('Distro %s does not support building
the following targets %s' % (self.distro, ','.join(targets)))
+ self.run_test_sstate_creation(targets,
distro_specific=True, distro_nonspecific=False, temp_sstate_location=True)
Quoted 1 line
Sounds like it would be easier to just remove glibc-initial from that list, and either remove it entirely or replace it with a virtual name that both glibc and musl provide.
Ok
Quoted 10 lines
@@ -228,6 +255,8 @@ class SStateTests(SStateBase):
build machines and running a builds, override the
variables callinguname()
manually and check using bitbake -S. """
+ if self.distro == 'poky-tiny':
+ self.skipTest('core-image-sato not buildable for
poky-tiny')
Quoted 6 lines
topdir =
get_bb_var('TOPDIR')targetvendor =get_bb_var('TARGET_VENDOR')For all of these tests that just do bitbake -S core-image-sato, if world works then that is more flexible and more comprehensive.
Agree. Sending v3 soon.
Quoted 1 line
Ross
PATCH v2 12/13 selftest: wic: skip tests for poky-tiny distro policy
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Poky-tiny just supports cpio.gz as filesystem type so wic tests must be skipped.
--- meta/lib/oeqa/selftest/wic.py | 9 +++++++++ 1 file changed, 9 insertions(+)
meta/lib/oeqa/selftest/wic.py +9 -0
@@ -190,6 +190,8 @@ class Wic(oeSelfTest): @testcase(1346) def test_iso_image(self): """Test creation of hybrid iso image with legacy and EFI boot""" + if self.distro == 'poky-tiny': + self.skipTest('wic image (mkhybridiso) not supported for poky-tiny distro') self.assertEqual(0, runCmd("wic create mkhybridiso " "--image-name core-image-minimal").status) self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.direct")))
@@ -246,6 +248,9 @@ class Wic(oeSelfTest): def test_mkgummidisk(self): """Test creation of mkgummidisk image""" image = "mkgummidisk" + if self.distro == 'poky-tiny': + self.skipTest('wic image name (%s) not supported for poky-tiny distro' % image) + self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
@@ -254,6 +259,8 @@ class Wic(oeSelfTest): def test_mkefidisk(self): """Test creation of mkefidisk image""" image = "mkefidisk" + if self.distro == 'poky-tiny': + self.skipTest('wic image (%s) not supported for poky-tiny distro' % image) self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
@@ -288,6 +295,8 @@ class Wic(oeSelfTest): def test_systemd_bootdisk(self): """Test creation of systemd-bootdisk image""" image = "systemd-bootdisk" + if self.distro == 'poky-tiny': + self.skipTest('wic image (%s) not supported for poky-tiny distro' % image) self.assertEqual(0, runCmd("wic create %s -e core-image-minimal" \ % image).status) self.assertEqual(1, len(glob(self.resultdir + "%s-*direct" % image)))
Signature · 1 line
2.1.4
PATCH v2 13/13 selftest: bblayers: remove linux kernel checks for show-recipes check
In reply to [email protected]From: Leonardo Sandoval <[email protected]>
Preferred kernel recipes depends on the distro, so remove the kernel checks to avoid failures on non-poky distros and make the test distro agnostic.
--- meta/lib/oeqa/selftest/bblayers.py | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
meta/lib/oeqa/selftest/bblayers.py +1 -5
@@ -68,20 +68,16 @@ class BitbakeLayers(oeSelfTest): @testcase(1384) def test_bitbakelayers_showrecipes(self): + distro = get_bb_var('DISTRO') result = runCmd('bitbake-layers show-recipes') self.assertIn('aspell:', result.output) self.assertIn('mtd-utils:', result.output) - self.assertIn('linux-yocto:', result.output) self.assertIn('core-image-minimal:', result.output) result = runCmd('bitbake-layers show-recipes mtd-utils') self.assertIn('mtd-utils:', result.output) self.assertNotIn('aspell:', result.output) - result = runCmd('bitbake-layers show-recipes -i kernel') - self.assertIn('linux-yocto:', result.output) - self.assertNotIn('mtd-utils:', result.output) result = runCmd('bitbake-layers show-recipes -i image') self.assertIn('core-image-minimal', result.output) - self.assertNotIn('linux-yocto:', result.output) self.assertNotIn('mtd-utils:', result.output) result = runCmd('bitbake-layers show-recipes -i cmake,pkgconfig') self.assertIn('libproxy:', result.output)
Signature · 1 line
2.1.4