selftest/runtime-test.py: Add test for import test from other layers
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 selftest/runtime-test.py: Add test for import test from other layers
From: Mariano Lopez <[email protected]>
There are some features in testimage/testexport that are not tested; this might lead to break some of these features without notice.
This adds a new test in order to test two features of testimage:
- Import test from other layers.
- Install/Unistall in the DUT without a package manager.
[YOCTO #9764]
[YOCTO #9766]
---
meta-selftest/lib/oeqa/runtime/selftest.json | 6 +++++
meta-selftest/lib/oeqa/runtime/selftest.py | 31 ++++++++++++++++++++++
.../selftest/{testexport.py => runtime-test.py} | 19 +++++++++++++
3 files changed, 56 insertions(+)
create mode 100644 meta-selftest/lib/oeqa/runtime/selftest.json
create mode 100644 meta-selftest/lib/oeqa/runtime/selftest.py
rename meta/lib/oeqa/selftest/{testexport.py => runtime-test.py} (76%)meta-selftest/lib/oeqa/runtime/selftest.json new file +6 -0
@@ -0,0 +1,6 @@ +{ + "test_install_package": { + "pkg": "socat", + "rm": true + } +}
meta-selftest/lib/oeqa/runtime/selftest.py new file +31 -0
@@ -0,0 +1,31 @@ +from oeqa.oetest import oeRuntimeTest, skipModule +from oeqa.utils.decorators import * + +class Selftest(oeRuntimeTest): + + @skipUnlessPassed("test_ssh") + def test_install_package(self): + """ + Summary: Check basic package installation functionality. + Expected: 1. Before the test socat must be installed using scp. + 2. After the test socat must be unistalled using ssh. + This can't be checked in this test. + Product: oe-core + Author: Mariano Lopez <[email protected]> + """ + + (status, output) = self.target.run("socat -V") + self.assertEqual(status, 0, msg="socat is not installed") + + @skipUnlessPassed("test_install_package") + def test_verify_unistall(self): + """ + Summary: Check basic package installation functionality. + Expected: 1. test_install_package must unistall socat. + This test is just to verify that. + Product: oe-core + Author: Mariano Lopez <[email protected]> + """ + + (status, output) = self.target.run("socat -V") + self.assertNotEqual(status, 0, msg="socat is still installed")
meta/lib/oeqa/selftest/runtime-test.py renamed from meta/lib/oeqa/selftest/testexport.py +19 -0
@@ -42,3 +42,22 @@ class TestExport(oeSelfTest): # Verify ping test was succesful failure = True if 'FAIL' in result.output else False self.assertNotEqual(True, failure, 'ping test failed') + + +class TestImage(oeSelfTest): + + def test_testimage_install(self): + """ + Summary: Check install packages functionality for testimage/testexport. + Expected: 1. Import tests from a directory other than meta. + 2. Check install/unistall of socat. + Product: oe-core + Author: Mariano Lopez <[email protected]> + """ + + features = 'INHERIT += "testexport"\n' + features = 'TEST_SUITES = "ping ssh selftest"\n' + + # Build core-image-sato and testimage + bitbake('core-image-sato socat') + bitbake('-c testimage core-image-sato')
Signature · 1 line
2.6.6
meta/lib/oeqa/selftest/runtime-test.py class TestExport(oeSelfTest): · line 63
Quoted 7 lines · On 13 June 2016 at 14:32, <[email protected]> wrote:
meta/lib/oeqa/selftest/runtime-test.py renamed from meta/lib/oeqa/selftest/testexport.py +6
-0+ features = 'INHERIT += "testexport"\n' + features = 'TEST_SUITES = "ping ssh selftest"\n' + + # Build core-image-sato and testimage + bitbake('core-image-sato socat') + bitbake('-c testimage core-image-sato')
If we're just running ping/ssh/selftest, can the image be something a lot smaller such as core-image-base?
Ross
Quoted 11 lines · Ross
On 13 June 2016 at 14:32, <[email protected] <mailto:[email protected]>> wrote:
+ features = 'INHERIT += "testexport"\n'
+ features = 'TEST_SUITES= "ping ssh selftest"\n'
+
+ # Build core-image-sato and testimage
+ bitbake('core-image-sato socat')
+ bitbake('-c testimage core-image-sato')
Quoted 3 lines
If we're just running ping/ssh/selftest, can the image be something a lot smaller such as core-image-base?
Ross
Unfortunately, base or minimal don't have ssh installed, and ssh it is required for the runtime tests. I could add the ssh to DISTRO_FEATURES but personally I think that is not clean. What do you think?
Mariano
Quoted 18 lines · Mariano Lopez
On 06/14/2016 04:26 AM, Burton, Ross wrote:
meta/lib/oeqa/selftest/runtime-test.py class TestExport(oeSelfTest): · line 63
Quoted 7 lines · On 13 June 2016 at 14:32, <[email protected]> wrote:
meta/lib/oeqa/selftest/runtime-test.py renamed from meta/lib/oeqa/selftest/testexport.py +6
-0+ features = 'INHERIT += "testexport"\n' + features = 'TEST_SUITES = "ping ssh selftest"\n' + + # Build core-image-sato and testimage + bitbake('core-image-sato socat') + bitbake('-c testimage core-image-sato')If we're just running ping/ssh/selftest, can the image be something a lot smaller such as core-image-base?
Quoted 7 lines
Ross
Unfortunately, base or minimal don't have ssh installed, and ssh it is required for the runtime tests.
I could add the ssh toDISTRO_FEATURESbut personally I think that is not clean. What do you think?
Can ssh be added to images automatically if testing is enabled ?
Quoted 9 lines
Mariano
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 5 lines · Khem Raj
Quoted 3 lines · Mariano Lopez
If we're just running ping/ssh/selftest, can the image be something a lot smaller such as core-image-base?
Quoted 8 lines
Quoted 7 lines
Ross
Unfortunately, base or minimal don't have ssh installed, and ssh it is required for the runtime tests.
I could add the ssh toDISTRO_FEATURESbut personally I think that is not clean. What do you think?Can ssh be added to images automatically if testing is enabled ?
IMO that is not a good practice, because the image being tested is polluted with packages not meant to be there.
Mariano
Quoted 18 lines · Mariano Lopez
Quoted 15 lines · Khem Raj
Quoted 11 lines · Mariano Lopez
If we're just running ping/ssh/selftest, can the image be something a lot smaller such as core-image-base?
Ross
Unfortunately, base or minimal don't have ssh installed, and ssh it is required for the runtime tests.
I could add the ssh toDISTRO_FEATURESbut personally I think that is not clean. What do you think?
Quoted 5 lines
Quoted 1 line
Can ssh be added to images automatically if testing is enabled ?
IMO that is not a good practice, because the image being tested is polluted with packages not meant to be there.
I thought you mentioned that ssh is required for testing, I dont see that as polluting.
and its only added to image with have ptest turned on.
Quoted 2 lines
Mariano
Quoted 16 lines · Khem Raj
Quoted 14 lines · Mariano Lopez
Quoted 12 lines · Khem Raj
Quoted 10 lines · Mariano Lopez
If we're just running ping/ssh/selftest, can the image be something a lot smaller such as core-image-base?
Ross
Unfortunately, base or minimal don't have ssh installed, and ssh it is required for the runtime tests.
I could add the ssh toDISTRO_FEATURESbut personally I think that is not clean. What do you think?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Can ssh be added to images automatically if testing is enabled ?
IMO that is not a good practice, because the image being tested is polluted with packages not meant to be there.
I thought you mentioned that ssh is required for testing, I dont see that as polluting.
and its only added to image with have ptest turned on.
This is for testimage/testexport and there is a TEST_SUITE specific for minimal image.
If the resource consumption seems to be the problem; adding ssh to minimal or base would trigger the build of a new rootfs/image; in the other hand is very likely core-image-sato is already built when running this test.
Quoted 2 lines · On 13 June 2016 at 14:32, <[email protected]> wrote:
+ features = 'INHERIT += "testexport"\n'
You mean testimage here as that class isn't inherited by default.
Quoted 2 lines
+ features = '
TEST_SUITES= "ping ssh selftest"\n'
This overwrites the first assignment to features, and neither is actually written using self.write_config().
meta/lib/oeqa/selftest/runtime-test.py class TestExport(oeSelfTest): · line 63
Quoted 4 lines
meta/lib/oeqa/selftest/runtime-test.py renamed from meta/lib/oeqa/selftest/testexport.py +3
-0+ # Build core-image-sato and testimage + bitbake('core-image-sato socat') + bitbake('-c testimage core-image-sato')
I personally believe that core-image-sato is massive overkill for selftest and core-image-full-cmdline is just as comprehensive (has ssh, for example) without building all of Sato.
Ross
Quoted 11 lines · Ross
On 13 June 2016 at 14:32, <[email protected] <mailto:[email protected]>> wrote:
+ features = 'INHERIT += "testexport"\n'
You mean testimage here as that class isn't inherited by default.
+ features = '
TEST_SUITES= "ping ssh selftest"\n'
Quoted 8 lines
This overwrites the first assignment to features, and neither is actually written using
self.write_config().+ # Build core-image-sato and testimage
+ bitbake('core-image-sato socat')
+ bitbake('-c testimage core-image-sato')I personally believe that core-image-sato is massive overkill for selftest and core-image-full-cmdline is just as comprehensive (has ssh, for example) without building all of Sato.
I don't know what happened when I made this commit and submitted the patch, this patch is just wrong. I swear I tested it locally, perhaps I messed up the branches or something like that. I'll work on a v2 of this.
Mariano