Skip to content

OEQA Framework Refactor & Improvements

Aníbal Limón openembedded-core 3 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest30 patches 0 repliesNew
  2. v232 patches 3 repliesSuperseded
  3. v132 patches 1 replySuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/30oeqa/core: Add base OEQA frameworkmeta/lib/oeqa/core/__init__.py meta/lib/oeqa/core/case.py meta/lib/oeqa/core/exception.py +1 more+136 -00 replies
2/30oeqa/core: Add utils module for OEQA frameworkmeta/lib/oeqa/core/utils/__init__.py meta/lib/oeqa/core/utils/misc.py meta/lib/oeqa/core/utils/path.py +1 more+127 -00 replies
3/30oeqa/core: Add loader, context and decorator modulesmeta/lib/oeqa/core/context.py meta/lib/oeqa/core/decorator/__init__.py meta/lib/oeqa/core/loader.py+446 -00 replies
4/30oeqa/core/decorator: Add support for OETestDependsmeta/lib/oeqa/core/decorator/depends.py+94 -00 replies
5/30oeqa/core/decorator: Add support for OETestID and OETestTagmeta/lib/oeqa/core/decorator/oeid.py meta/lib/oeqa/core/decorator/oetag.py+47 -00 replies
6/30oeqa/core/decorator: Add support for OETimeout decoratormeta/lib/oeqa/core/decorator/oetimeout.py+25 -00 replies
7/30oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVarmeta/lib/oeqa/core/decorator/data.py+36 -00 replies
8/30oeqa/core: Add tests for the OEQA frameworkmeta/lib/oeqa/core/tests/__init__.py meta/lib/oeqa/core/tests/cases/data.py meta/lib/oeqa/core/tests/cases/depends.py +10 more+481 -00 replies
9/30scripts/oe-test: Add new oe-test scriptscripts/oe-test+98 -00 replies
10/30oeqa/core/context: Add support of OETestContextExecutormeta/lib/oeqa/core/context.py+85 -00 replies
11/30oeqa/core/cases: Add example test casesmeta/lib/oeqa/core/cases/__init__.py meta/lib/oeqa/core/cases/example/data.json meta/lib/oeqa/core/cases/example/test_basic.py+21 -00 replies
12/30oeqa/core: Add READMEmeta/lib/oeqa/core/README+38 -00 replies
13/30oe/data: Add export2json functionmeta/lib/oe/data.py+28 -00 replies
14/30classes/rootfs-postcommands: Add write_image_test_datameta/classes/rootfs-postcommands.bbclass+18 -00 replies
15/30classes/populate_sdk_base: Add write_sdk_test_data to postprocessmeta/classes/populate_sdk_base.bbclass+8 -10 replies
16/30oeqa: Move common files to oeqa/files instead of runtime onlymeta/lib/oeqa/files/test.c meta/lib/oeqa/files/test.cpp meta/lib/oeqa/files/test.pl +1 more+0 -00 replies
17/30oeqa/sdk: Move test cases inside cases directorymeta/lib/oeqa/sdk/cases/__init__.py meta/lib/oeqa/sdk/cases/buildcvs.py meta/lib/oeqa/sdk/cases/buildgalculator.py +4 more+0 -00 replies
18/30oeqa/{runtime, sdk}/files: Move testsdkmakefile from runtime to sdk modulemeta/lib/oeqa/sdk/files/testsdkmakefile+0 -00 replies
19/30oeqa/sdk: Add case and context modules for the SDK componentmeta/lib/oeqa/oetest.py meta/lib/oeqa/sdk/__init__.py meta/lib/oeqa/sdk/case.py +1 more+145 -510 replies
20/30classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContextmeta/classes/testsdk.bbclass+55 -120 replies
21/30oeqa/utils: Move targetbuild to buildproject modulemeta/lib/oeqa/runtime/utils/__init__.py meta/lib/oeqa/runtime/utils/targetbuildproject.py meta/lib/oeqa/sdk/utils/__init__.py +2 more+79 -670 replies
22/30oeqa/utils: {Target, SDK, }BuildProject remove dependency of bbmeta/lib/oeqa/runtime/utils/targetbuildproject.py meta/lib/oeqa/sdk/utils/sdkbuildproject.py meta/lib/oeqa/utils/buildproject.py+17 -340 replies
23/30oeqa/sdk/cases: Migrate tests to the new OEQA frameworkmeta/lib/oeqa/sdk/cases/__init__.py meta/lib/oeqa/sdk/cases/buildcvs.py meta/lib/oeqa/sdk/cases/buildgalculator.py +4 more+84 -690 replies
24/30classes/testsdk: Remove the need of TEST_LOG_DIR variablemeta/classes/testsdk.bbclass meta/lib/oeqa/sdk/cases/buildcvs.py meta/lib/oeqa/sdk/cases/buildgalculator.py +1 more+6 -120 replies
25/30oeqa/sdkext: Move test cases inside cases directorymeta/lib/oeqa/sdkext/__init__.py meta/lib/oeqa/sdkext/cases/devtool.py meta/lib/oeqa/sdkext/cases/sdk_update.py+0 -30 replies
26/30oeqa/sdkext: Adds case and context modules.meta/classes/testsdk.bbclass meta/lib/oeqa/oetest.py meta/lib/oeqa/sdkext/case.py +1 more+88 -930 replies
27/30classes/testsdk: Migrate to use the new OESDKExtTestContextmeta/classes/testsdk.bbclass+5 -50 replies
28/30oeqa/sdkext/cases: Migrate test case to new OEQA frameworkmeta/lib/oeqa/sdk/cases/buildgalculator.py meta/lib/oeqa/sdk/cases/gcc.py meta/lib/oeqa/sdkext/cases/devtool.py +1 more+38 -320 replies
29/30oeqa/runtime: Fix TargetBuildProject instancesmeta/classes/testimage.bbclass meta/lib/oeqa/runtime/buildcvs.py meta/lib/oeqa/runtime/buildgalculator.py +1 more+19 -100 replies
30/30oeqa: Fix files handling on runtime tests.meta/classes/testexport.bbclass meta/lib/oeqa/oetest.py meta/lib/oeqa/runtime/gcc.py +2 more+10 -40 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCHv3 00/30] OEQA Framework Refactor & Improvements

This patchset is related to OEQA Framework for details read the RFC send to the Openembedded architecture ML.

http://lists.openembedded.org/pipermail/openembedded-architecture/2016-December/000351.html

Changes in v2: Fixes sdk extensible test case sdk_update.
Changes in v3: Squashed two commits related to d -> td and remove_safe function.

The following changes since commit 931ce04b0884ebe73158251d0d9a461214dfe1aa:

bitbake: toaster: Reference pip3 not pip (2016-12-08 15:52:08 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib alimon/oeqa_sdk_migration
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/oeqa_sdk_migration

Aníbal Limón (26):
oeqa/core: Add base OEQA framework
oeqa/core: Add loader, context and decorator modules oeqa/core/decorator: Add support for OETestDepends oeqa/core/decorator: Add support for OETestDataDepends and
skipIfDataVar
scripts/oe-test: Add new oe-test script
oeqa/core/context: Add support of OETestContextExecutor oeqa/core/cases: Add example test cases
oeqa/core: Add README
oe/data: Add export2json function classes/rootfs-postcommands: Add write_image_test_data classes/populate_sdk_base: Add write_sdk_test_data to postprocess
oeqa: Move common files to oeqa/files instead of runtime only oeqa/sdk: Move test cases inside cases directory oeqa/{runtime,sdk}/files: Move testsdkmakefile from runtime to sdk
module
oeqa/sdk: Add case and context modules for the SDK component classes/testsdk: Migrates testsdk.bbclass to use new OESDKTestContext oeqa/utils: Move targetbuild to buildproject module oeqa/utils: {Target,SDK,}BuildProject remove dependency of bb oeqa/sdk/cases: Migrate tests to the new OEQA framework classes/testsdk: Remove the need of TEST_LOG_DIR variable oeqa/sdkext: Move test cases inside cases directory oeqa/sdkext: Adds case and context modules. classes/testsdk: Migrate to use the new OESDKExtTestContext oeqa/sdkext/cases: Migrate test case to new OEQA framework oeqa/runtime: Fix TargetBuildProject instances
oeqa: Fix files handling on runtime tests.

Mariano Lopez (4):
oeqa/core: Add utils module for OEQA framework oeqa/core/decorator: Add support for OETestID and OETestTag oeqa/core/decorator: Add support for OETimeout decorator oeqa/core: Add tests for the OEQA framework

 meta/classes/populate_sdk_base.bbclass             |   9 +-
 meta/classes/rootfs-postcommands.bbclass           |  18 ++
 meta/classes/testexport.bbclass                    |   5 +
 meta/classes/testimage.bbclass                     |   4 +
 meta/classes/testsdk.bbclass                       | 162 ++++++++------
 meta/lib/oe/data.py                                |  28 +++
 meta/lib/oeqa/core/README                          |  38 ++++
 meta/lib/oeqa/core/__init__.py                     |   0
 meta/lib/oeqa/core/case.py                         |  46 ++++
 meta/lib/oeqa/core/cases/__init__.py               |   0
 meta/lib/oeqa/core/cases/example/data.json         |   1 +
 meta/lib/oeqa/core/cases/example/test_basic.py     |  20 ++
 meta/lib/oeqa/core/context.py                      | 225 ++++++++++++++++++++
 meta/lib/oeqa/core/decorator/__init__.py           |  71 +++++++
 meta/lib/oeqa/core/decorator/data.py               |  36 ++++
 meta/lib/oeqa/core/decorator/depends.py            |  94 +++++++++
 meta/lib/oeqa/core/decorator/oeid.py               |  23 ++
 meta/lib/oeqa/core/decorator/oetag.py              |  24 +++
 meta/lib/oeqa/core/decorator/oetimeout.py          |  25 +++
 meta/lib/oeqa/core/exception.py                    |  14 ++
 meta/lib/oeqa/core/loader.py                       | 235 +++++++++++++++++++++
 meta/lib/oeqa/core/runner.py                       |  76 +++++++
 meta/lib/oeqa/core/tests/__init__.py               |   0
 meta/lib/oeqa/core/tests/cases/data.py             |  20 ++
 meta/lib/oeqa/core/tests/cases/depends.py          |  38 ++++
 .../oeqa/core/tests/cases/loader/invalid/oeid.py   |  15 ++
 .../oeqa/core/tests/cases/loader/valid/another.py  |   9 +
 meta/lib/oeqa/core/tests/cases/oeid.py             |  18 ++
 meta/lib/oeqa/core/tests/cases/oetag.py            |  18 ++
 meta/lib/oeqa/core/tests/cases/timeout.py          |  18 ++
 meta/lib/oeqa/core/tests/common.py                 |  35 +++
 meta/lib/oeqa/core/tests/test_data.py              |  51 +++++
 meta/lib/oeqa/core/tests/test_decorators.py        | 135 ++++++++++++
 meta/lib/oeqa/core/tests/test_loader.py            |  86 ++++++++
 meta/lib/oeqa/core/tests/test_runner.py            |  38 ++++
 meta/lib/oeqa/core/utils/__init__.py               |   0
 meta/lib/oeqa/core/utils/misc.py                   |  37 ++++
 meta/lib/oeqa/core/utils/path.py                   |  19 ++
 meta/lib/oeqa/core/utils/test.py                   |  71 +++++++
 meta/lib/oeqa/{runtime => }/files/test.c           |   0
 meta/lib/oeqa/{runtime => }/files/test.cpp         |   0
 meta/lib/oeqa/{runtime => }/files/test.pl          |   0
 meta/lib/oeqa/{runtime => }/files/test.py          |   0
 meta/lib/oeqa/oetest.py                            |  92 +-------
 meta/lib/oeqa/runtime/buildcvs.py                  |   9 +-
 meta/lib/oeqa/runtime/buildgalculator.py           |   8 +-
 meta/lib/oeqa/runtime/buildiptables.py             |   8 +-
 meta/lib/oeqa/runtime/gcc.py                       |   4 +-
 meta/lib/oeqa/runtime/perl.py                      |   2 +-
 meta/lib/oeqa/runtime/python.py                    |   2 +-
 meta/lib/oeqa/runtime/utils/__init__.py            |   0
 meta/lib/oeqa/runtime/utils/targetbuildproject.py  |  33 +++
 meta/lib/oeqa/sdk/__init__.py                      |   3 -
 meta/lib/oeqa/sdk/case.py                          |  12 ++
 meta/lib/oeqa/sdk/{ => cases}/buildcvs.py          |  15 +-
 meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py   |  28 ++-
 meta/lib/oeqa/sdk/{ => cases}/buildiptables.py     |  16 +-
 meta/lib/oeqa/sdk/cases/gcc.py                     |  43 ++++
 meta/lib/oeqa/sdk/cases/perl.py                    |  27 +++
 meta/lib/oeqa/sdk/{ => cases}/python.py            |  25 ++-
 meta/lib/oeqa/sdk/context.py                       | 133 ++++++++++++
 .../oeqa/{runtime => sdk}/files/testsdkmakefile    |   0
 meta/lib/oeqa/sdk/gcc.py                           |  36 ----
 meta/lib/oeqa/sdk/perl.py                          |  28 ---
 meta/lib/oeqa/sdk/utils/__init__.py                |   0
 meta/lib/oeqa/sdk/utils/sdkbuildproject.py         |  45 ++++
 meta/lib/oeqa/sdkext/__init__.py                   |   3 -
 meta/lib/oeqa/sdkext/case.py                       |  21 ++
 meta/lib/oeqa/sdkext/{ => cases}/devtool.py        |  49 +++--
 meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py     |  17 +-
 meta/lib/oeqa/sdkext/context.py                    |  21 ++
 meta/lib/oeqa/utils/buildproject.py                |  52 +++++
 meta/lib/oeqa/utils/targetbuild.py                 | 135 ------------
 scripts/oe-test                                    |  98 +++++++++
 74 files changed, 2284 insertions(+), 443 deletions(-)
 create mode 100644 meta/lib/oeqa/core/README
 create mode 100644 meta/lib/oeqa/core/__init__.py
 create mode 100644 meta/lib/oeqa/core/case.py
 create mode 100644 meta/lib/oeqa/core/cases/__init__.py
 create mode 100644 meta/lib/oeqa/core/cases/example/data.json
 create mode 100644 meta/lib/oeqa/core/cases/example/test_basic.py
 create mode 100644 meta/lib/oeqa/core/context.py
 create mode 100644 meta/lib/oeqa/core/decorator/__init__.py
 create mode 100644 meta/lib/oeqa/core/decorator/data.py
 create mode 100644 meta/lib/oeqa/core/decorator/depends.py
 create mode 100644 meta/lib/oeqa/core/decorator/oeid.py
 create mode 100644 meta/lib/oeqa/core/decorator/oetag.py
 create mode 100644 meta/lib/oeqa/core/decorator/oetimeout.py
 create mode 100644 meta/lib/oeqa/core/exception.py
 create mode 100644 meta/lib/oeqa/core/loader.py
 create mode 100644 meta/lib/oeqa/core/runner.py
 create mode 100644 meta/lib/oeqa/core/tests/__init__.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/data.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/depends.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/loader/invalid/oeid.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/loader/valid/another.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/oeid.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/oetag.py
 create mode 100644 meta/lib/oeqa/core/tests/cases/timeout.py
 create mode 100644 meta/lib/oeqa/core/tests/common.py
 create mode 100755 meta/lib/oeqa/core/tests/test_data.py
 create mode 100755 meta/lib/oeqa/core/tests/test_decorators.py
 create mode 100755 meta/lib/oeqa/core/tests/test_loader.py
 create mode 100755 meta/lib/oeqa/core/tests/test_runner.py
 create mode 100644 meta/lib/oeqa/core/utils/__init__.py
 create mode 100644 meta/lib/oeqa/core/utils/misc.py
 create mode 100644 meta/lib/oeqa/core/utils/path.py
 create mode 100644 meta/lib/oeqa/core/utils/test.py
 rename meta/lib/oeqa/{runtime => }/files/test.c (100%)
 rename meta/lib/oeqa/{runtime => }/files/test.cpp (100%)
 rename meta/lib/oeqa/{runtime => }/files/test.pl (100%)
 rename meta/lib/oeqa/{runtime => }/files/test.py (100%)
 create mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
 create mode 100644 meta/lib/oeqa/runtime/utils/targetbuildproject.py
 create mode 100644 meta/lib/oeqa/sdk/case.py
 rename meta/lib/oeqa/sdk/{ => cases}/buildcvs.py (59%)
 rename meta/lib/oeqa/sdk/{ => cases}/buildgalculator.py (45%)
 rename meta/lib/oeqa/sdk/{ => cases}/buildiptables.py (58%)
 create mode 100644 meta/lib/oeqa/sdk/cases/gcc.py
 create mode 100644 meta/lib/oeqa/sdk/cases/perl.py
 rename meta/lib/oeqa/sdk/{ => cases}/python.py (46%)
 create mode 100644 meta/lib/oeqa/sdk/context.py
 rename meta/lib/oeqa/{runtime => sdk}/files/testsdkmakefile (100%)
 delete mode 100644 meta/lib/oeqa/sdk/gcc.py
 delete mode 100644 meta/lib/oeqa/sdk/perl.py
 create mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
 create mode 100644 meta/lib/oeqa/sdk/utils/sdkbuildproject.py
 create mode 100644 meta/lib/oeqa/sdkext/case.py
 rename meta/lib/oeqa/sdkext/{ => cases}/devtool.py (73%)
 rename meta/lib/oeqa/sdkext/{ => cases}/sdk_update.py (63%)
 create mode 100644 meta/lib/oeqa/sdkext/context.py
 create mode 100644 meta/lib/oeqa/utils/buildproject.py
 delete mode 100644 meta/lib/oeqa/utils/targetbuild.py
 create mode 100755 scripts/oe-test
Signature · 1 line
2.1.4