Script and layer for running tests
PATCH 3/4 lib/oeqa/selftest: buildoptions.py: add simple image build tests
In reply to Stefan StanacarFrom: Alexandru Palalau <[email protected]>
Build images and tests different build options like RM_OLD_IMAGE and for WARN_QA/ERROR_QA behaviour.
--- meta/lib/oeqa/selftest/buildoptions.py | 86 ++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) create mode 100644 meta/lib/oeqa/selftest/buildoptions.py
meta/lib/oeqa/selftest/buildoptions.py new file +86 -0
@@ -0,0 +1,86 @@ +import unittest +import os +import logging +import re + +from oeqa.selftest.base import oeSelfTest +from oeqa.utils.commands import runCmd, bitbake, get_bb_var +import oeqa.utils.ftools as ftools + +class ImageOptionsTests(oeSelfTest): + + def test_incremental_image_generation(self): + bitbake("-c cleanall core-image-minimal") + self.write_config('INC_RPM_IMAGE_GEN = "1"') + self.append_config('IMAGE_FEATURES += "ssh-server-openssh"') + bitbake("core-image-minimal") + res = runCmd("grep 'Installing openssh-sshd' %s" % (os.path.join(get_bb_var("WORKDIR", "core-image-minimal"), "temp/log.do_rootfs")), ignore_status=True) + self.remove_config('IMAGE_FEATURES += "ssh-server-openssh"') + self.assertEqual(0, res.status, msg="No match for openssh-sshd in log.do_rootfs") + bitbake("core-image-minimal") + res = runCmd("grep 'Removing openssh-sshd' %s" %(os.path.join(get_bb_var("WORKDIR", "core-image-minimal"), "temp/log.do_rootfs")),ignore_status=True) + self.assertEqual(0, res.status, msg="openssh-sshd was not removed from image") + + def test_rm_old_image(self): + bitbake("core-image-minimal") + deploydir = get_bb_var("DEPLOY_DIR_IMAGE", target="core-image-minimal") + imagename = get_bb_var("IMAGE_LINK_NAME", target="core-image-minimal") + oldimgpath = os.path.realpath(os.path.join(deploydir, imagename + ".ext3")) + self.append_config("RM_OLD_IMAGE = \"1\"") + bitbake("core-image-minimal") + self.assertFalse(os.path.exists(oldimgpath), msg="Old image path still exists: %s" % oldimgpath) + + def test_ccache_tool(self): + bitbake("ccache-native") + self.assertTrue(os.path.isfile(os.path.join(get_bb_var('STAGING_BINDIR_NATIVE', 'ccache-native'), "ccache"))) + self.write_config('INHERIT += "ccache"') + bitbake("m4 -c cleansstate") + bitbake("m4 -c compile") + res = runCmd("grep ccache %s" % (os.path.join(get_bb_var("WORKDIR","m4"),"temp/log.do_compile")), ignore_status=True) + self.assertEqual(0, res.status, msg="No match for ccache in m4 log.do_compile") + bitbake("ccache-native -ccleansstate") + + +class DiskMonTest(oeSelfTest): + + def test_stoptask_behavior(self): + result = runCmd("df -k %s" % os.getcwd()) + size = result.output.split("\n")[1].split()[3] + self.append_config('BB_DISKMON_DIRS = "STOPTASKS,${TMPDIR},%sK,4510K"' % size) + res = bitbake("core-image-minimal", ignore_status = True) + self.assertTrue('ERROR: No new tasks can be executed since the disk space monitor action is "STOPTASKS"!' in res.output) + self.assertEqual(res.status, 1) + self.append_config('BB_DISKMON_DIRS = "ABORT,${TMPDIR},%sK,4510K"' % size) + res = bitbake("core-image-minimal", ignore_status = True) + self.assertTrue('ERROR: Immediately abort since the disk space monitor action is "ABORT"!' in res.output) + self.assertEqual(res.status, 1) + self.append_config('BB_DISKMON_DIRS = "WARN,${TMPDIR},%sK,4510K"' % size) + res = bitbake("core-image-minimal") + self.assertTrue('WARNING: The free space' in res.output) + +class SanityOptionsTest(oeSelfTest): + + def test_options_warnqa_errorqa_switch(self): + bitbake("xcursor-transparent-theme -ccleansstate") + + if "packages-list" not in get_bb_var("ERROR_QA"): + self.write_config("ERROR_QA_append = \" packages-list\"") + + self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += \"${PN}-dbg\"') + res = bitbake("xcursor-transparent-theme", ignore_status=True) + self.delete_recipeinc('xcursor-transparent-theme') + self.assertTrue("ERROR: QA Issue: xcursor-transparent-theme-dbg is listed in PACKAGES multiple times, this leads to packaging errors." in res.output) + self.assertEqual(res.status, 1) + self.write_recipeinc('xcursor-transparent-theme', 'PACKAGES += \"${PN}-dbg\"') + self.append_config('ERROR_QA_remove = "packages-list"') + self.append_config('WARN_QA_append = " packages-list"') + bitbake("xcursor-transparent-theme") + bitbake("xcursor-transparent-theme -ccleansstate") + self.delete_recipeinc('xcursor-transparent-theme') + + def test_sanity_userspace_dependency(self): + self.append_config('WARN_QA_append = " unsafe-references-in-binaries unsafe-references-in-scripts"') + bitbake("-ccleansstate gzip nfs-utils") + res = bitbake("gzip nfs-utils") + self.assertTrue("WARNING: QA Issue: gzip" in res.output) + self.assertTrue("WARNING: QA Issue: nfs-utils" in res.output)
Signature · 1 line
1.8.3.1
cover Script and layer for running tests
Hello,
This series adds an oe-selftest script, some modules and a new layer meta-selftest. which are meant to help in writing tests (using python unittest) for various bitbake tools/scripts as well as simple output checks or do
complete builds with different options (with the emphasis that everything checked is done outside of bitbake context, just like a human would do.) For more details, please see YOCTO #4740.
Cheers,
Stefan
Changes since the RFC:
- more tests added
- tests are now in meta/lib/oeqa/selftest instead of scripts/lib/selftest/tests
- small changes to some of the files in the meta-selftest layer
- more cleanups done by default in the base class
The following changes since commit 32adaac34a614d106e6dd3e9f1130f4e94ff39ae:
libpng: set reasonable SUMMARY (2013-11-27 11:51:25 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib stefans/selftest
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/selftest
Alexandru Palalau (1):
lib/oeqa/selftest: buildoptions.py: add simple image build tests
Corneliu Stoicescu (2):
meta-selftest: create a new test layer to be used by oe-selftest
script
lib/oeqa/selftest: add test modules for expected bitbake output and
bitbake-layers
Stefan Stanacar (1):
scripts/oe-selftest: script to run builds as unittest against bitbake
or various scripts
.gitignore | 1 + meta-selftest/COPYING.MIT | 17 +++ meta-selftest/README | 3 + meta-selftest/classes/test_events.bbclass | 16 +++ meta-selftest/conf/layer.conf | 10 ++ .../recipes-test/aspell/aspell_0.0.0.1.bb | 28 ++++ .../recipes-test/aspell/aspell_0.60.6.1.bbappend | 2 + meta-selftest/recipes-test/m4/m4_1.4.17.bbappend | 2 + .../recipes-test/man/man/man-1.5h1-make.patch | 16 +++ meta-selftest/recipes-test/man/man_1.6g.bbappend | 2 + .../xcursor-transparent-theme_0.1.1.bbappend | 2 + meta/lib/oeqa/selftest/__init__.py | 2 + meta/lib/oeqa/selftest/base.py | 98 ++++++++++++++ meta/lib/oeqa/selftest/bblayers.py | 37 ++++++ meta/lib/oeqa/selftest/bbtests.py | 97 ++++++++++++++ meta/lib/oeqa/selftest/buildoptions.py | 86 ++++++++++++ meta/lib/oeqa/utils/commands.py | 137 +++++++++++++++++++ meta/lib/oeqa/utils/ftools.py | 27 ++++ scripts/oe-selftest | 148 +++++++++++++++++++++ 19 files changed, 731 insertions(+) create mode 100644 meta-selftest/COPYING.MIT create mode 100644 meta-selftest/README create mode 100644 meta-selftest/classes/test_events.bbclass create mode 100644 meta-selftest/conf/layer.conf create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend create mode 100644 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend create mode 100644 meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend create mode 100644 meta/lib/oeqa/selftest/__init__.py create mode 100644 meta/lib/oeqa/selftest/base.py create mode 100644 meta/lib/oeqa/selftest/bblayers.py create mode 100644 meta/lib/oeqa/selftest/bbtests.py create mode 100644 meta/lib/oeqa/selftest/buildoptions.py create mode 100644 meta/lib/oeqa/utils/commands.py create mode 100644 meta/lib/oeqa/utils/ftools.py create mode 100755 scripts/oe-selftest
Signature · 1 line
1.8.3.1
PATCH 1/4 scripts/oe-selftest: script to run builds as unittest against bitbake or various scripts
The purpose of oe-selftest is to run unittest modules added from meta/lib/oeqa/selftest, which are tests against bitbake tools.
Right now the script it's useful for simple tests like:
- "bitbake --someoption, change some metadata, bitbake X, check something" type scenarios (PR service, error output, etc)
- or "bitbake-layers <...>" type scripts and yocto-bsp tools.
This commit also adds some helper modules that the tests will use and a base class. Also, most of the tests will have a dependency on a meta-selftest layer
which contains specially modified recipes/bbappends/include files for the purpose of the tests. The tests themselves will usually write to ".inc" files from the layer or in conf/selftest.inc (which is added as an include in local.conf at the start and removed at the end)
It's a simple matter or sourcing the enviroment, adding the meta-selftest layer to bblayers.conf and running: oe-selftest to get some results. It would finish faster if at least a core-image-minimal was built before.
[ YOCTO #4740 ]
--- meta/lib/oeqa/selftest/__init__.py | 2 + meta/lib/oeqa/selftest/base.py | 98 ++++++++++++++++++++++++ meta/lib/oeqa/utils/commands.py | 137 ++++++++++++++++++++++++++++++++++ meta/lib/oeqa/utils/ftools.py | 27 +++++++ scripts/oe-selftest | 148 +++++++++++++++++++++++++++++++++++++ 5 files changed, 412 insertions(+) create mode 100644 meta/lib/oeqa/selftest/__init__.py create mode 100644 meta/lib/oeqa/selftest/base.py create mode 100644 meta/lib/oeqa/utils/commands.py create mode 100644 meta/lib/oeqa/utils/ftools.py create mode 100755 scripts/oe-selftest
meta/lib/oeqa/selftest/__init__.py new file +2 -0
@@ -0,0 +1,2 @@ +from pkgutil import extend_path +__path__ = extend_path(__path__, __name__)
meta/lib/oeqa/selftest/base.py new file +98 -0
98 lines not on this page · raw message
meta/lib/oeqa/utils/commands.py new file +137 -0
137 lines not on this page · raw message
meta/lib/oeqa/utils/ftools.py new file +27 -0
@@ -0,0 +1,27 @@ +import os +import re + +def write_file(path, data): + wdata = data.rstrip() + "\n" + with open(path, "w") as f: + f.write(wdata) + +def append_file(path, data): + wdata = data.rstrip() + "\n" + with open(path, "a") as f: + f.write(wdata) + +def read_file(path): + data = None + with open(path) as f: + data = f.read() + return data + +def remove_from_file(path, data): + lines = read_file(path).splitlines() + rmdata = data.strip().splitlines() + for l in rmdata: + for c in range(0, lines.count(l)): + i = lines.index(l) + del(lines[i]) + write_file(path, "\n".join(lines))
scripts/oe-selftest new file +148 -0
148 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH 2/4 meta-selftest: create a new test layer to be used by oe-selftest script
In reply to Stefan StanacarFrom: Corneliu Stoicescu <[email protected]>
Everything in this layer is meant to be used by tests called by scripts/oe-selftest. These are helper recipes/appends to test various bitbake options or scripts.
Currently most of these files here only have "include test_recipe.inc" which is the file tests will actually use.
--- .gitignore | 1 + meta-selftest/COPYING.MIT | 17 +++++++++++++ meta-selftest/README | 3 +++ meta-selftest/classes/test_events.bbclass | 16 +++++++++++++ meta-selftest/conf/layer.conf | 10 ++++++++ .../recipes-test/aspell/aspell_0.0.0.1.bb | 28 ++++++++++++++++++++++ .../recipes-test/aspell/aspell_0.60.6.1.bbappend | 2 ++ meta-selftest/recipes-test/m4/m4_1.4.17.bbappend | 2 ++ .../recipes-test/man/man/man-1.5h1-make.patch | 16 +++++++++++++ meta-selftest/recipes-test/man/man_1.6g.bbappend | 2 ++ .../xcursor-transparent-theme_0.1.1.bbappend | 2 ++ 11 files changed, 99 insertions(+) create mode 100644 meta-selftest/COPYING.MIT create mode 100644 meta-selftest/README create mode 100644 meta-selftest/classes/test_events.bbclass create mode 100644 meta-selftest/conf/layer.conf create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb create mode 100644 meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend create mode 100644 meta-selftest/recipes-test/m4/m4_1.4.17.bbappend create mode 100644 meta-selftest/recipes-test/man/man/man-1.5h1-make.patch create mode 100644 meta-selftest/recipes-test/man/man_1.6g.bbappend create mode 100644 meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend
.gitignore +1 -0
@@ -21,3 +21,4 @@ bitbake/doc/manual/pdf/ bitbake/doc/manual/txt/ bitbake/doc/manual/xhtml/ pull-*/ +!meta-selftest
meta-selftest/COPYING.MIT new file +17 -0
@@ -0,0 +1,17 @@ +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE.
meta-selftest/README new file +3 -0
@@ -0,0 +1,3 @@ +This layer is intended as test layer, used by scripts/oe-selftest +and it's probably a mistake to include it in your builds (unless you +want to run the script).
meta-selftest/classes/test_events.bbclass new file +16 -0
@@ -0,0 +1,16 @@ +python test1_eventhandler() { + bb.note("Test for bb.event.BuildStarted") +} +python test2_eventhandler() { + bb.note("Test for bb.event.BuildCompleted") +} +python test3_eventhandler() { + bb.note("Test for bb.event.InvalidEvent") +} + +addhandler test1_eventhandler +test1_eventhandler[eventmask] = "bb.event.BuildStarted" +addhandler test2_eventhandler +test2_eventhandler[eventmask] = "bb.event.BuildCompleted" +addhandler test3_eventhandler +test3_eventhandler[eventmask] = "bb.event.InvalidEvent"
meta-selftest/conf/layer.conf new file +10 -0
@@ -0,0 +1,10 @@ +# We have a conf and classes directory, add to BBPATH +BBPATH .= ":${LAYERDIR}" + +# We have recipes-* directories, add to BBFILES +BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ + ${LAYERDIR}/recipes-*/*/*.bbappend" + +BBFILE_COLLECTIONS += "selftest" +BBFILE_PATTERN_selftest = "^${LAYERDIR}/" +BBFILE_PRIORITY_selftest = "5"
meta-selftest/recipes-test/aspell/aspell_0.0.0.1.bb new file +28 -0
@@ -0,0 +1,28 @@ +# This recipe is a copy from the oe-core one. +# It has a lower and invalid version number in order not to be accidentally used by bitbake. +# It is used for tests that require overlayed recipe files. + +DESCRIPTION = "GNU Aspell spell-checker" +SECTION = "console/utils" +LICENSE="LGPLv2 | LGPLv2.1" +LIC_FILES_CHKSUM = "file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34" +PR = "r1" + +SRC_URI = "${GNU_MIRROR}/aspell/aspell-${PV}.tar.gz" +SRC_URI[md5sum] = "e66a9c9af6a60dc46134fdacf6ce97d7" +SRC_URI[sha256sum] = "f52583a83a63633701c5f71db3dc40aab87b7f76b29723aeb27941eff42df6e1" + +PACKAGECONFIG ??= "" +PACKAGECONFIG[curses] = "--enable-curses,--disable-curses,ncurses" + +PACKAGES += "libaspell libpspell libpspell-dev aspell-utils" + +FILES_${PN}-dbg += "${libdir}/aspell-0.60/.debu*" +FILES_libaspell = "${libdir}/libaspell.so.* ${libdir}/aspell*" +FILES_aspell-utils = "${bindir}/word-list-compress ${bindir}/aspell-import ${bindir}/run-with-aspell ${bindir}/pre*" +FILES_${PN} = "${bindir}/aspell" +FILES_libpspell = "${libdir}/libpspell.so.*" +FILES_libpspell-dev = "${libdir}/libpspell* ${bindir}/pspell-config ${includedir}/pspell" + +ARM_INSTRUCTION_SET = "arm" +inherit autotools gettext
meta-selftest/recipes-test/aspell/aspell_0.60.6.1.bbappend new file +2 -0
@@ -0,0 +1,2 @@ +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests. +include test_recipe.inc
meta-selftest/recipes-test/m4/m4_1.4.17.bbappend new file +2 -0
@@ -0,0 +1,2 @@ +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests. +include test_recipe.inc
meta-selftest/recipes-test/man/man/man-1.5h1-make.patch new file +16 -0
@@ -0,0 +1,16 @@ +Test patch here! +This is invalid patch used by tests in scripts/lib/selftests. + +--- man-1.5g/man/Makefile.in.mike Fri Apr 9 13:35:54 1999 ++++ man-1.5g/man/Makefile.in Fri Apr 9 13:36:45 1999 +@@ -1,8 +1,8 @@ + #MAKE THIS PATCH INVALID + MAN1 = man whatis apropos +-MAN5 = man.conf ++MAN5 = man.config + MAN8 = makewhatis +-ALL = man.1 whatis.1 apropos.1 man.conf.5 ++ALL = man.1 whatis.1 apropos.1 man.config.5 + MAYBE8 = makewhatis + + .SUFFIXES: .man .1 .5 .8
meta-selftest/recipes-test/man/man_1.6g.bbappend new file +2 -0
@@ -0,0 +1,2 @@ +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests. +include test_recipe.inc
meta-selftest/recipes-test/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bbappend new file +2 -0
@@ -0,0 +1,2 @@ +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests. +include test_recipe.inc
Signature · 1 line
1.8.3.1
PATCH 4/4 lib/oeqa/selftest: add test modules for expected bitbake output and bitbake-layers
In reply to Stefan StanacarFrom: Corneliu Stoicescu <[email protected]>
Tests for bitbake-layers and expected output for some bitbake options.
--- meta/lib/oeqa/selftest/bblayers.py | 37 +++++++++++++++ meta/lib/oeqa/selftest/bbtests.py | 97 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 134 insertions(+) create mode 100644 meta/lib/oeqa/selftest/bblayers.py create mode 100644 meta/lib/oeqa/selftest/bbtests.py
meta/lib/oeqa/selftest/bblayers.py new file +37 -0
@@ -0,0 +1,37 @@ +import unittest +import os +import logging +import re +import shutil + +import oeqa.utils.ftools as ftools +from oeqa.selftest.base import oeSelfTest +from oeqa.utils.commands import runCmd + +class BitbakeLayers(oeSelfTest): + + def test_bitbakelayers_showcrossdepends(self): + result = runCmd('bitbake-layers show-cross-depends') + self.assertTrue('aspell' in result.output) + + def test_bitbakelayers_showlayers(self): + result = runCmd('bitbake-layers show_layers') + self.assertTrue('meta-selftest' in result.output) + + def test_bitbakelayers_showappends(self): + result = runCmd('bitbake-layers show_appends') + self.assertTrue('xcursor-transparent-theme_0.1.1.bbappend' in result.output, msg='xcursor-transparent-theme_0.1.1.bbappend file was not recognised') + + def test_bitbakelayers_showoverlayed(self): + result = runCmd('bitbake-layers show_overlayed') + self.assertTrue('aspell' in result.output, msg='xcursor-transparent-theme_0.1.1.bbappend file was not recognised') + + def test_bitbakelayers_flatten(self): + self.assertFalse(os.path.isdir(os.path.join(self.builddir, 'test'))) + result = runCmd('bitbake-layers flatten test') + bb_file = os.path.join(self.builddir, 'test/recipes-graphics/xcursor-transparent-theme/xcursor-transparent-theme_0.1.1.bb') + self.assertTrue(os.path.isfile(bb_file)) + contents = ftools.read_file(bb_file) + find_in_contents = re.search("##### bbappended from meta-selftest #####\n(.*\n)*include test_recipe.inc", contents) + shutil.rmtree(os.path.join(self.builddir, 'test')) + self.assertTrue(find_in_contents)
meta/lib/oeqa/selftest/bbtests.py new file +97 -0
97 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Quoted 11 lines · Stefan Stanacar
This series adds an oe-selftest script, some modules and a new layer meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbaketools/scriptsas well as simple output checks or do complete builds with different options (with the emphasis that everything checked
is done outside of bitbake context, just like a human would do.) For more details,
please see YOCTO #4740.
Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Hi Chris,
Quoted 13 lines · On Wednesday 27 November 2013 11:42:06 Chris Larson wrote:
Quoted 10 lines · Stefan Stanacar
This series adds an oe-selftest script, some modules and a new layer meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbaketools/scriptsas well as simple output checks or do complete builds with different options (with the emphasis that everything checked
is done outside of bitbake context, just like a human would do.) For more details,
please see YOCTO #4740.
Quoted 2 lines
Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.
It really needs to stay in the same place as the tests that rely on it; they can't work without it. It might be suggested that the tests should go elsewhere with them; but I don't think that would make sense - we want people to be able to run these tests easily if they make changes to OE-Core to check if they have caused any regressions.
If you have a look at the layer, there's not a lot in it, and I wouldn't expect a lot to be added in future. One possible change would be to just have the layer created on the fly in some temporary directory by the tests themselves, rather than having it in the repository; on the other hand it does make it a bit more difficult to look into outside of the test environment.
Thoughts?
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Quoted 28 lines · Paul Eggleton
Quoted 8 lines · On Wednesday 27 November 2013 11:42:06 Chris Larson wrote:
Quoted 6 lines · Stefan Stanacar
This series adds an oe-selftest script, some modules and a new layer meta-selftest.
which are meant to help in writing tests (using python unittest) for various
bitbaketools/scriptsas well as simple output checks or do complete builds with different options (with the emphasis thateverything
Quoted 2 lines
Quoted 2 lines
checked
is done outside of bitbake context, just like a human would do.) Formore
Quoted 5 lines
Quoted 2 lines
details,
please see YOCTO #4740.Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.
It really needs to stay in the same place as the tests that rely on it; they
can't work without it. It might be suggested that the tests should go elsewhere with them; but I don't think that would make sense - we want people
to be able to run these tests easily if they make changes to OE-Core to check
if they have caused any regressions.
Quoted 7 lines
If you have a look at the layer, there's not a lot in it, and I wouldn't expect a lot to be added in future. One possible change would be to just have
the layer created on the fly in some temporary directory by the tests themselves, rather than having it in the repository; on the other hand it does
make it a bit more difficult to look into outside of the test environment.
I understand what you’re saying, but if this script is supposed to be generally useful for OE/Yocto, not just Poky, then the poky repository shouldn’t be its definitive location. Further, patches against oe-core should be submitted against the oe-core repository, not the poky repository, and anything poky specific should go to the poky list.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Quoted 19 lines · On Thursday 28 November 2013 13:29:32 Chris Larson wrote:
Quoted 10 lines · Paul Eggleton
Quoted 2 lines · On Wednesday 27 November 2013 11:42:06 Chris Larson wrote:
Will this have its own repository like all the other poky integrated layers? It really should, in my opinion.
It really needs to stay in the same place as the tests that rely on it; they can't work without it. It might be suggested that the tests should go elsewhere with them; but I don't think that would make sense - we want people to be able to run these tests easily if they make changes to OE- Core to check if they have caused any regressions.
Quoted 6 lines
If you have a look at the layer, there's not a lot in it, and I wouldn't expect a lot to be added in future. One possible change would be to just have the layer created on the fly in some temporary directory by the tests themselves, rather than having it in the repository; on the other hand it does make it a bit more difficult to look into outside of the test environment.
Quoted 3 lines
I understand what you’re saying, but if this script is supposed to be generally useful for OE/Yocto, not just Poky, then the poky repository shouldn’t be its definitive location.
Poky isn't its definitive location, it has been sent as a pull request to the OE-Core mailing list to be applied on top of OE-Core.
Quoted 3 lines
Further, patches against oe-core should be submitted against the oe-core repository, not the poky repository, and anything poky specific should go to the poky list.
Point taken, but there isn't actually anything poky-specific in this series. Can we please concentrate on the content of the patches?
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre