Skip to contentSkip to conversation

Script and layer for running tests

PATCH In review Stefan Stanacar 3 participants 9 messages Nov 27, 2013 – Nov 28, 2013

lore
Share & tools
lore
#0 lore raw

cover Script and layer for running tests

Hello,

This series adds an oe-selftest script, some modules and a new layer meta-selftest. which are meant to help in writing tests (using python unittest) for various bitbake tools/scripts as well as simple output checks or do
complete builds with different options (with the emphasis that everything checked is done outside of bitbake context, just like a human would do.) For more details, please see YOCTO #4740.

Cheers,
Stefan

Changes since the RFC:
- more tests added
- tests are now in meta/lib/oeqa/selftest instead of scripts/lib/selftest/tests
- small changes to some of the files in the meta-selftest layer
- more cleanups done by default in the base class

The following changes since commit 32adaac34a614d106e6dd3e9f1130f4e94ff39ae:

libpng: set reasonable SUMMARY (2013-11-27 11:51:25 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib stefans/selftest
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/selftest

Alexandru Palalau (1):
lib/oeqa/selftest: buildoptions.py: add simple image build tests

Corneliu Stoicescu (2):
meta-selftest: create a new test layer to be used by oe-selftest
script
lib/oeqa/selftest: add test modules for expected bitbake output and
bitbake-layers

Stefan Stanacar (1):
scripts/oe-selftest: script to run builds as unittest against bitbake
or various scripts

 .gitignore                                         |   1 +
 meta-selftest/COPYING.MIT                          |  17 +++
 meta-selftest/README                               |   3 +
 meta-selftest/classes/test_events.bbclass          |  16 +++
 meta-selftest/conf/layer.conf                      |  10 ++
 .../recipes-test/aspell/aspell_0.0.0.1.bb          |  28 ++++
 .../recipes-test/aspell/aspell_0.60.6.1.bbappend   |   2 +
 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend   |   2 +
 .../recipes-test/man/man/man-1.5h1-make.patch      |  16 +++
 meta-selftest/recipes-test/man/man_1.6g.bbappend   |   2 +
 .../xcursor-transparent-theme_0.1.1.bbappend       |   2 +
 meta/lib/oeqa/selftest/__init__.py                 |   2 +
 meta/lib/oeqa/selftest/base.py                     |  98 ++++++++++++++
 meta/lib/oeqa/selftest/bblayers.py                 |  37 ++++++
 meta/lib/oeqa/selftest/bbtests.py                  |  97 ++++++++++++++
 meta/lib/oeqa/selftest/buildoptions.py             |  86 ++++++++++++
 meta/lib/oeqa/utils/commands.py                    | 137 +++++++++++++++++++
 meta/lib/oeqa/utils/ftools.py                      |  27 ++++
 scripts/oe-selftest                                | 148 +++++++++++++++++++++
 19 files changed, 731 insertions(+)
 create mode 100644 meta-selftest/COPYING.MIT
 create mode 100644 meta-selftest/README
 create mode 100644 meta-selftest/classes/test_events.bbclass
 create mode 100644 meta-selftest/conf/layer.conf
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend
 create mode 100644 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend
 create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch
 create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend
 create mode 100644 meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend
 create mode 100644 meta/lib/oeqa/selftest/__init__.py
 create mode 100644 meta/lib/oeqa/selftest/base.py
 create mode 100644 meta/lib/oeqa/selftest/bblayers.py
 create mode 100644 meta/lib/oeqa/selftest/bbtests.py
 create mode 100644 meta/lib/oeqa/selftest/buildoptions.py
 create mode 100644 meta/lib/oeqa/utils/commands.py
 create mode 100644 meta/lib/oeqa/utils/ftools.py
 create mode 100755 scripts/oe-selftest
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/4 scripts/oe-selftest: script to run builds as unittest against bitbake or various scripts

The purpose of oe-selftest is to run unittest modules added from meta/lib/oeqa/selftest, which are tests against bitbake tools.

Right now the script it's useful for simple tests like:
- "bitbake --someoption, change some metadata, bitbake X, check something" type scenarios (PR service, error output, etc)
- or "bitbake-layers <...>" type scripts and yocto-bsp tools.

This commit also adds some helper modules that the tests will use and a base class. Also, most of the tests will have a dependency on a meta-selftest layer
which contains specially modified recipes/bbappends/include files for the purpose of the tests. The tests themselves will usually write to ".inc" files from the layer or in conf/selftest.inc (which is added as an include in local.conf at the start and removed at the end)

It's a simple matter or sourcing the enviroment, adding the meta-selftest layer to bblayers.conf and running: oe-selftest to get some results. It would finish faster if at least a core-image-minimal was built before.

[ YOCTO #4740 ]

---
 meta/lib/oeqa/selftest/__init__.py |   2 +
 meta/lib/oeqa/selftest/base.py     |  98 ++++++++++++++++++++++++
 meta/lib/oeqa/utils/commands.py    | 137 ++++++++++++++++++++++++++++++++++
 meta/lib/oeqa/utils/ftools.py      |  27 +++++++
 scripts/oe-selftest                | 148 +++++++++++++++++++++++++++++++++++++
 5 files changed, 412 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/__init__.py
 create mode 100644 meta/lib/oeqa/selftest/base.py
 create mode 100644 meta/lib/oeqa/utils/commands.py
 create mode 100644 meta/lib/oeqa/utils/ftools.py
 create mode 100755 scripts/oe-selftest
meta/lib/oeqa/selftest/__init__.py new file +2 -0
new file mode 100644
index 0000000..3ad9513
@@ -0,0 +1,2 @@
+from pkgutil import extend_path
+__path__ = extend_path(__path__, __name__)
meta/lib/oeqa/selftest/base.py new file +98 -0

98 lines not on this page · raw message

meta/lib/oeqa/utils/commands.py new file +137 -0

137 lines not on this page · raw message

meta/lib/oeqa/utils/ftools.py new file +27 -0
new file mode 100644
index 0000000..64ebe3d
@@ -0,0 +1,27 @@
+import os
+import re
+
+def write_file(path, data):
+    wdata = data.rstrip() + "\n"
+    with open(path, "w") as f:
+        f.write(wdata)
+
+def append_file(path, data):
+    wdata = data.rstrip() + "\n"
+    with open(path, "a") as f:
+            f.write(wdata)
+
+def read_file(path):
+    data = None
+    with open(path) as f:
+        data = f.read()
+    return data
+
+def remove_from_file(path, data):
+    lines = read_file(path).splitlines()
+    rmdata = data.strip().splitlines()
+    for l in rmdata:
+        for c in range(0, lines.count(l)):
+            i = lines.index(l)
+            del(lines[i])
+    write_file(path, "\n".join(lines))
scripts/oe-selftest new file +148 -0

148 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#2 lore raw

PATCH 2/4 meta-selftest: create a new test layer to be used by oe-selftest script

In reply to Stefan Stanacar

From: Corneliu Stoicescu <[email protected]>

Everything in this layer is meant to be used by tests called by scripts/oe-selftest. These are helper recipes/appends to test various bitbake options or scripts.
Currently most of these files here only have "include test_recipe.inc" which is the file tests will actually use.

---
 .gitignore                                         |  1 +
 meta-selftest/COPYING.MIT                          | 17 +++++++++++++
 meta-selftest/README                               |  3 +++
 meta-selftest/classes/test_events.bbclass          | 16 +++++++++++++
 meta-selftest/conf/layer.conf                      | 10 ++++++++
 .../recipes-test/aspell/aspell_0.0.0.1.bb          | 28 ++++++++++++++++++++++
 .../recipes-test/aspell/aspell_0.60.6.1.bbappend   |  2 ++
 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend   |  2 ++
 .../recipes-test/man/man/man-1.5h1-make.patch      | 16 +++++++++++++
 meta-selftest/recipes-test/man/man_1.6g.bbappend   |  2 ++
 .../xcursor-transparent-theme_0.1.1.bbappend       |  2 ++
 11 files changed, 99 insertions(+)
 create mode 100644 meta-selftest/COPYING.MIT
 create mode 100644 meta-selftest/README
 create mode 100644 meta-selftest/classes/test_events.bbclass
 create mode 100644 meta-selftest/conf/layer.conf
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb
 create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend
 create mode 100644 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend
 create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch
 create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend
 create mode 100644 meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend
.gitignore +1 -0
index 0171597..b6755fe 100644
@@ -21,3 +21,4 @@ bitbake/doc/manual/pdf/
 bitbake/doc/manual/txt/
 bitbake/doc/manual/xhtml/
 pull-*/
+!meta-selftest
meta-selftest/COPYING.MIT new file +17 -0
new file mode 100644
index 0000000..89de354
@@ -0,0 +1,17 @@
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
meta-selftest/README new file +3 -0
new file mode 100644
index 0000000..11a6fee
@@ -0,0 +1,3 @@
+This layer is intended as test layer, used by scripts/oe-selftest
+and it's probably a mistake to include it in your builds (unless you
+want to run the script).
meta-selftest/classes/test_events.bbclass new file +16 -0
new file mode 100644
index 0000000..35324eb
@@ -0,0 +1,16 @@
+python test1_eventhandler() {
+    bb.note("Test for bb.event.BuildStarted")
+}
+python test2_eventhandler() {
+    bb.note("Test for bb.event.BuildCompleted")
+}
+python test3_eventhandler() {
+    bb.note("Test for bb.event.InvalidEvent")
+}
+
+addhandler test1_eventhandler
+test1_eventhandler[eventmask] = "bb.event.BuildStarted"
+addhandler test2_eventhandler
+test2_eventhandler[eventmask] = "bb.event.BuildCompleted"
+addhandler test3_eventhandler
+test3_eventhandler[eventmask] = "bb.event.InvalidEvent"
meta-selftest/conf/layer.conf new file +10 -0
new file mode 100644
index 0000000..a847b78
@@ -0,0 +1,10 @@
+# We have a conf and classes directory, add to BBPATH
+BBPATH .= ":${LAYERDIR}"
+
+# We have recipes-* directories, add to BBFILES
+BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
+	${LAYERDIR}/recipes-*/*/*.bbappend"
+
+BBFILE_COLLECTIONS += "selftest"
+BBFILE_PATTERN_selftest = "^${LAYERDIR}/"
+BBFILE_PRIORITY_selftest = "5"
meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb new file +28 -0
new file mode 100644
index 0000000..427726a
@@ -0,0 +1,28 @@
+# This recipe is a copy from the oe-core one.
+# It has a lower and invalid version number in order not to be accidentally used by bitbake.
+# It is used for tests that require overlayed recipe files.
+
+DESCRIPTION = "GNU Aspell spell-checker"
+SECTION = "console/utils"
+LICENSE="LGPLv2 | LGPLv2.1"
+LIC_FILES_CHKSUM = "file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34"
+PR = "r1"
+
+SRC_URI = "${GNU_MIRROR}/aspell/aspell-${PV}.tar.gz"
+SRC_URI[md5sum] = "e66a9c9af6a60dc46134fdacf6ce97d7"
+SRC_URI[sha256sum] = "f52583a83a63633701c5f71db3dc40aab87b7f76b29723aeb27941eff42df6e1"
+
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[curses] = "--enable-curses,--disable-curses,ncurses"
+
+PACKAGES += "libaspell libpspell libpspell-dev aspell-utils"
+
+FILES_${PN}-dbg += "${libdir}/aspell-0.60/.debu*"
+FILES_libaspell = "${libdir}/libaspell.so.* ${libdir}/aspell*"
+FILES_aspell-utils = "${bindir}/word-list-compress ${bindir}/aspell-import ${bindir}/run-with-aspell ${bindir}/pre*"
+FILES_${PN} = "${bindir}/aspell"
+FILES_libpspell = "${libdir}/libpspell.so.*"
+FILES_libpspell-dev = "${libdir}/libpspell* ${bindir}/pspell-config ${includedir}/pspell"
+
+ARM_INSTRUCTION_SET = "arm"
+inherit autotools gettext
meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend new file +2 -0
new file mode 100644
index 0000000..2057209
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
meta-selftest/recipes-test/m4/m4_1.4.17.bbappend new file +2 -0
new file mode 100644
index 0000000..2057209
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
meta-selftest/recipes-test/man/man/man-1.5h1-make.patch new file +16 -0
new file mode 100644
index 0000000..a0d59b7
@@ -0,0 +1,16 @@
+Test patch here!
+This is invalid patch used by tests in scripts/lib/selftests.
+
+--- man-1.5g/man/Makefile.in.mike	Fri Apr  9 13:35:54 1999
++++ man-1.5g/man/Makefile.in	Fri Apr  9 13:36:45 1999
+@@ -1,8 +1,8 @@
+ #MAKE THIS PATCH INVALID
+ MAN1 = man whatis apropos
+-MAN5 = man.conf
++MAN5 = man.config
+ MAN8 = makewhatis
+-ALL = man.1 whatis.1 apropos.1 man.conf.5
++ALL = man.1 whatis.1 apropos.1 man.config.5
+ MAYBE8 = makewhatis
+ 
+ .SUFFIXES: .man .1 .5 .8
meta-selftest/recipes-test/man/man_1.6g.bbappend new file +2 -0
new file mode 100644
index 0000000..2057209
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend new file +2 -0
new file mode 100644
index 0000000..2057209
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
Signature · 1 line
1.8.3.1
#3 lore raw

PATCH 3/4 lib/oeqa/selftest: buildoptions.py: add simple image build tests

In reply to Stefan Stanacar

From: Alexandru Palalau <[email protected]>

Build images and tests different build options like RM_OLD_IMAGE and for WARN_QA/ERROR_QA behaviour.

---
 meta/lib/oeqa/selftest/buildoptions.py | 86 ++++++++++++++++++++++++++++++++++
 1 file changed, 86 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/buildoptions.py
meta/lib/oeqa/selftest/buildoptions.py new file +86 -0
new file mode 100644
index 0000000..f99dda7
@@ -0,0 +1,86 @@
+import unittest
+import os
+import logging
+import re
+
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var
+import oeqa.utils.ftools as ftools
+
+class ImageOptionsTests(oeSelfTest):
+
+    def test_incremental_image_generation(self):
+        bitbake("-c cleanall core-image-minimal")
+        self.write_config('INC_RPM_IMAGE_GEN = "1"')
+        self.append_config('IMAGE_FEATURES += "ssh-server-openssh"')
+        bitbake("core-image-minimal")
+        res = runCmd("grep 'Installing openssh-sshd' %s" % (os.path.join(get_bb_var("WORKDIR", "core-image-minimal"), "temp/log.do_rootfs")), ignore_status=True)
+        self.remove_config('IMAGE_FEATURES += "ssh-server-openssh"')
+        self.assertEqual(0, res.status, msg="No match for openssh-sshd in log.do_rootfs")
+        bitbake("core-image-minimal")
+        res = runCmd("grep 'Removing openssh-sshd' %s" %(os.path.join(get_bb_var("WORKDIR", "core-image-minimal"), "temp/log.do_rootfs")),ignore_status=True)
+        self.assertEqual(0, res.status, msg="openssh-sshd was not removed from image")
+
+    def test_rm_old_image(self):
+        bitbake("core-image-minimal")
+        deploydir = get_bb_var("DEPLOY_DIR_IMAGE", target="core-image-minimal")
+        imagename = get_bb_var("IMAGE_LINK_NAME", target="core-image-minimal")
+        oldimgpath = os.path.realpath(os.path.join(deploydir, imagename + ".ext3"))
+        self.append_config("RM_OLD_IMAGE = \"1\"")
+        bitbake("core-image-minimal")
+        self.assertFalse(os.path.exists(oldimgpath), msg="Old image path still exists: %s" % oldimgpath)
+
+    def test_ccache_tool(self):
+        bitbake("ccache-native")
+        self.assertTrue(os.path.isfile(os.path.join(get_bb_var('STAGING_BINDIR_NATIVE', 'ccache-native'), "ccache")))
+        self.write_config('INHERIT += "ccache"')
+        bitbake("m4 -c cleansstate")
+        bitbake("m4 -c compile")
+        res = runCmd("grep ccache %s" % (os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile")), ignore_status=True)
+        self.assertEqual(0, res.status, msg="No match for ccache in m4 log.do_compile")
+        bitbake("ccache-native -ccleansstate")
+
+
+class DiskMonTest(oeSelfTest):
+
+    def test_stoptask_behavior(self):
+        result = runCmd("df -k %s" % os.getcwd())
+        size = result.output.split("\n")[1].split()[3]
+        self.append_config('BB_DISKMON_DIRS = "STOPTASKS,${TMPDIR},%sK,4510K"' % size)
+        res = bitbake("core-image-minimal", ignore_status = True)
+        self.assertTrue('ERROR: No new tasks can be executed since the disk space monitor action is "STOPTASKS"!' in res.output)
+        self.assertEqual(res.status, 1)
+        self.append_config('BB_DISKMON_DIRS = "ABORT,${TMPDIR},%sK,4510K"' % size)
+        res = bitbake("core-image-minimal", ignore_status = True)
+        self.assertTrue('ERROR: Immediately abort since the disk space monitor action is "ABORT"!' in res.output)
+        self.assertEqual(res.status, 1)
+        self.append_config('BB_DISKMON_DIRS = "WARN,${TMPDIR},%sK,4510K"' % size)
+        res = bitbake("core-image-minimal")
+        self.assertTrue('WARNING: The free space' in res.output)
+
+class SanityOptionsTest(oeSelfTest):
+
+    def test_options_warnqa_errorqa_switch(self):
+        bitbake("xcursor-transparent-theme -ccleansstate")
+
+        if "packages-list" not in get_bb_var("ERROR_QA"):
+            self.write_config("ERROR_QA_append = \" packages-list\"")
+
+        self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += \"${PN}-dbg\"')
+        res = bitbake("xcursor-transparent-theme", ignore_status=True)
+        self.delete_recipeinc('xcursor-transparent-theme')
+        self.assertTrue("ERROR: QA Issue: xcursor-transparent-theme-dbg is listed in PACKAGES multiple times, this leads to packaging errors." in res.output)
+        self.assertEqual(res.status, 1)
+        self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += \"${PN}-dbg\"')
+        self.append_config('ERROR_QA_remove = "packages-list"')
+        self.append_config('WARN_QA_append = " packages-list"')
+        bitbake("xcursor-transparent-theme")
+        bitbake("xcursor-transparent-theme -ccleansstate")
+        self.delete_recipeinc('xcursor-transparent-theme')
+
+    def test_sanity_userspace_dependency(self):
+        self.append_config('WARN_QA_append = " unsafe-references-in-binaries unsafe-references-in-scripts"')
+        bitbake("-ccleansstate gzip nfs-utils")
+        res = bitbake("gzip nfs-utils")
+        self.assertTrue("WARNING: QA Issue: gzip" in res.output)
+        self.assertTrue("WARNING: QA Issue: nfs-utils" in res.output)
Signature · 1 line
1.8.3.1
#4 lore raw

PATCH 4/4 lib/oeqa/selftest: add test modules for expected bitbake output and bitbake-layers

In reply to Stefan Stanacar

From: Corneliu Stoicescu <[email protected]>

Tests for bitbake-layers and expected output for some bitbake options.

---
 meta/lib/oeqa/selftest/bblayers.py | 37 +++++++++++++++
 meta/lib/oeqa/selftest/bbtests.py  | 97 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 134 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/bblayers.py
 create mode 100644 meta/lib/oeqa/selftest/bbtests.py
meta/lib/oeqa/selftest/bblayers.py new file +37 -0
new file mode 100644
index 0000000..52aa4f8
@@ -0,0 +1,37 @@
+import unittest
+import os
+import logging
+import re
+import shutil
+
+import oeqa.utils.ftools as ftools
+from oeqa.selftest.base import oeSelfTest
+from oeqa.utils.commands import runCmd
+
+class BitbakeLayers(oeSelfTest):
+
+    def test_bitbakelayers_showcrossdepends(self):
+        result = runCmd('bitbake-layers show-cross-depends')
+        self.assertTrue('aspell' in result.output)
+
+    def test_bitbakelayers_showlayers(self):
+        result = runCmd('bitbake-layers show_layers')
+        self.assertTrue('meta-selftest' in result.output)
+
+    def test_bitbakelayers_showappends(self):
+        result = runCmd('bitbake-layers show_appends')
+        self.assertTrue('xcursor-transparent-theme_0.1.1.bbappend' in result.output, msg='xcursor-transparent-theme_0.1.1.bbappend file was not recognised')
+
+    def test_bitbakelayers_showoverlayed(self):
+        result = runCmd('bitbake-layers show_overlayed')
+        self.assertTrue('aspell' in result.output, msg='xcursor-transparent-theme_0.1.1.bbappend file was not recognised')
+
+    def test_bitbakelayers_flatten(self):
+        self.assertFalse(os.path.isdir(os.path.join(self.builddir, 'test')))
+        result = runCmd('bitbake-layers flatten test')
+        bb_file = os.path.join(self.builddir, 'test/recipes-graphics/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bb')
+        self.assertTrue(os.path.isfile(bb_file))
+        contents = ftools.read_file(bb_file)
+        find_in_contents = re.search("##### bbappended from meta-selftest #####\n(.*\n)*include test_recipe.inc", contents)
+        shutil.rmtree(os.path.join(self.builddir, 'test'))
+        self.assertTrue(find_in_contents)
meta/lib/oeqa/selftest/bbtests.py new file +97 -0

97 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#5 lore raw
In reply to Stefan Stanacar
Quoted 11 lines · Stefan Stanacar

This series adds an oe-selftest script, some modules and a new layer meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbake tools/scripts as well as simple output checks or do complete builds with different options (with the emphasis that everything checked
is done outside of bitbake context, just like a human would do.) For more details,
please see YOCTO #4740.

Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#6 lore raw

Hi Chris,

Quoted 13 lines · On Wednesday 27 November 2013 11:42:06 Chris Larson wrote:
Quoted 10 lines · Stefan Stanacar

This series adds an oe-selftest script, some modules and a new layer meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbake tools/scripts as well as simple output checks or do complete builds with different options (with the emphasis that everything checked
is done outside of bitbake context, just like a human would do.) For more details,
please see YOCTO #4740.

Quoted 2 lines

Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.

It really needs to stay in the same place as the tests that rely on it; they can't work without it. It might be suggested that the tests should go elsewhere with them; but I don't think that would make sense - we want people to be able to run these tests easily if they make changes to OE-Core to check if they have caused any regressions.

If you have a look at the layer, there's not a lot in it, and I wouldn't expect a lot to be added in future. One possible change would be to just have the layer created on the fly in some temporary directory by the tests themselves, rather than having it in the repository; on the other hand it does make it a bit more difficult to look into outside of the test environment.

Thoughts?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#7 lore raw
Quoted 28 lines · Paul Eggleton
Quoted 8 lines · On Wednesday 27 November 2013 11:42:06 Chris Larson wrote:
Quoted 6 lines · Stefan Stanacar

This series adds an oe-selftest script, some modules and a new layer meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbake tools/scripts as well as simple output checks or do complete builds with different options (with the emphasis that

everything

Quoted 2 lines
Quoted 2 lines

checked
is done outside of bitbake context, just like a human would do.) For

more

Quoted 5 lines
Quoted 2 lines

details,
please see YOCTO #4740.

Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.

It really needs to stay in the same place as the tests that rely on it; they
can't work without it. It might be suggested that the tests should go elsewhere with them; but I don't think that would make sense - we want people
to be able to run these tests easily if they make changes to OE-Core to check
if they have caused any regressions.

Quoted 7 lines

If you have a look at the layer, there's not a lot in it, and I wouldn't expect a lot to be added in future. One possible change would be to just have
the layer created on the fly in some temporary directory by the tests themselves, rather than having it in the repository; on the other hand it does
make it a bit more difficult to look into outside of the test environment.

I understand what you’re saying, but if this script is supposed to be generally useful for OE/Yocto, not just Poky, then the poky repository shouldn’t be its definitive location. Further, patches against oe-core should be submitted against the oe-core repository, not the poky repository, and anything poky specific should go to the poky list.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#8 lore raw
Quoted 19 lines · On Thursday 28 November 2013 13:29:32 Chris Larson wrote:
Quoted 10 lines · Paul Eggleton
Quoted 2 lines · On Wednesday 27 November 2013 11:42:06 Chris Larson wrote:

Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.

It really needs to stay in the same place as the tests that rely on it; they can't work without it. It might be suggested that the tests should go elsewhere with them; but I don't think that would make sense - we want people to be able to run these tests easily if they make changes to OE- Core to check if they have caused any regressions.

Quoted 6 lines

If you have a look at the layer, there's not a lot in it, and I wouldn't expect a lot to be added in future. One possible change would be to just have the layer created on the fly in some temporary directory by the tests themselves, rather than having it in the repository; on the other hand it does make it a bit more difficult to look into outside of the test environment.

Quoted 3 lines

I understand what you’re saying, but if this script is supposed to be generally useful for OE/Yocto, not just Poky, then the poky repository shouldn’t be its definitive location.

Poky isn't its definitive location, it has been sent as a pull request to the OE-Core mailing list to be applied on top of OE-Core.

Quoted 3 lines

Further, patches against oe-core should be submitted against the oe-core repository, not the poky repository, and anything poky specific should go to the poky list.

Point taken, but there isn't actually anything poky-specific in this series. Can we please concentrate on the content of the patches?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre