Skip to content

Refactor meta-oe, meta-perl, meta-python ptests with BBCLASSEXTEND parallel execution

Tim Orling openembedded-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest11 patches 2 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/11cli11: do not inherit ptestmeta-oe/recipes-support/cli11/cli11_2.3.2.bb+1 -20 replies
2/11span-lite: do not inherit ptestmeta-oe/recipes-support/span-lite/span-lite_0.10.3.bb+0 -10 replies
3/11ptest-packagelists-meta-oe.inc: add ptest recipesmeta-oe/conf/include/ptest-packagelists-meta-oe.inc+60 -00 replies
4/11meta-oe-ptest*-image: enable BBCLASSEXTEND parallel executionmeta-oe/recipes-core/images/meta-oe-ptest-all-image.bb meta-oe/recipes-core/images/meta-oe-ptest-fast-image.bb meta-oe/recipes-core/images/meta-oe-ptest-image.bb+68 -12 replies
5/11ptest-packagelists-meta-python.inc: add ptest recipesmeta-python/conf/include/ptest-packagelists-meta-python.inc+82 -00 replies
6/11meta-python-ptest*-image: enable BBCLASSEXTEND parallel executionmeta-python/recipes-core/images/meta-python-ptest-all-image.bb meta-python/recipes-core/images/meta-python-ptest-fast-image.bb meta-python/recipes-core/images/meta-python-ptest-image.bb+68 -10 replies
7/11python3-aspectlib: fix ptestmeta-python/recipes-devtools/python/python3-aspectlib/run-ptest meta-python/recipes-devtools/python/python3-aspectlib_2.0.0.bb+4 -20 replies
8/11ptest-packagelists-meta-perl.inc: add ptest recipesmeta-perl/conf/include/ptest-packagelists-meta-perl.inc+50 -00 replies
9/11recipes-perl/packagegroups: move to recipes-core/meta-perl/recipes-core/packagegroups/packagegroup-meta-perl.bb+0 -00 replies
10/11recipes-perl/images: move to recipes-coremeta-perl/recipes-core/images/meta-perl-base.bb meta-perl/recipes-core/images/meta-perl-image.bb meta-perl/recipes-core/images/meta-perl-ptest-image.bb+0 -00 replies
11/11meta-perl-ptest*-image: enable BBCLASSEXTEND parallel executionmeta-perl/recipes-core/images/meta-perl-ptest-all-image.bb meta-perl/recipes-core/images/meta-perl-ptest-fast-image.bb meta-perl/recipes-core/images/meta-perl-ptest-image.bb+66 -30 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/11] Refactor meta-oe, meta-perl, meta-python ptests with BBCLASSEXTEND parallel execution

This series implements parallel ptest execution, similar to "core-image-ptest: Switch to BBCLASSEXTEND parallel execution"

Refactor meta-*-ptest-image.bb for an image per ptest in the form of meta-*-ptest-image-XXX using class extensions.

* Add meta-*-ptest-all-image.bb depends on ALL meta-*-ptest-image-XXX in meta-*
* Add meta-*-ptest-fast-image.bb depends on meta-* ptests which take less than ~30s

This allows parallel execution of the tests.

The downside to this approach is the parsing time of an image generating this number of class extensions is slow but making it easier to execute and collect test data should outweigh that.

A useful advantage is that the dependencies of each ptest package are checked/tested individually.

Add to local.conf:
IMAGE_CLASSES += "testimage"

All the meta-*-ptest-image-XXX can be built with:

$ bitbake meta-*-ptest-all-image

Subsequently, al the ptest images can be run with:

$ bitbake -c testimage meta-*-ptest-all-image

You will likely want to limit the number of parallel image tests:

BB_NUMBER_THREADS = "4"

based on the number of TAP devices you have defined or the number of simultaneous QEMU sessions your server can support.

Results can be summarized with:

$ resulttool report tmp/log/oeqa/testresults.json

Call to arms:

1. When upgrading recipes that have ptest enabled, run testimage

   $ bitbake meta-*-ptest-image-<pn>
   $ bitbake -c testimage meta-*-ptest-image-<pn>
2. Help move recipes in PTEST_PROBLEMS_* to a state where all ptests pass
3. Test on architectures you care about and report problems.
4. Enable ptest in more recipes:
   * inherit ptest
   * provide a run-ptest script
   * add to conf/include/ptest-packagelists-meta-*.inc
5. Migrate python recipes which use pytest to the new "pytest --automake" methodology
6. Enable this methodology in more meta-openembedded layers.

The following changes since commit 17243e70c8309751a1129a9214899ec476ab121a:

python3-betamax: fix ptest failture of fixture and record modes (2023-03-28 10:44:00 -0700)

are available in the Git repository at:

https://git.openembedded.org/meta-openembedded-contrib timo/ptest-refactoring
http://cgit.openembedded.org/meta-openembedded-contrib/log/?h=timo/ptest-refactoring

Tim Orling (11):
cli11: do not inherit ptest
span-lite: do not inherit ptest ptest-packagelists-meta-oe.inc: add ptest recipes meta-oe-ptest*-image: enable BBCLASSEXTEND parallel execution ptest-packagelists-meta-python.inc: add ptest recipes meta-python-ptest*-image: enable BBCLASSEXTEND parallel execution
python3-aspectlib: fix ptest ptest-packagelists-meta-perl.inc: add ptest recipes recipes-perl/packagegroups: move to recipes-core/ recipes-perl/images: move to recipes-core meta-perl-ptest*-image: enable BBCLASSEXTEND parallel execution

 .../include/ptest-packagelists-meta-oe.inc    | 60 ++++++++++++++
 .../images/meta-oe-ptest-all-image.bb         | 25 ++++++
 .../images/meta-oe-ptest-fast-image.bb        |  6 ++
 .../images/meta-oe-ptest-image.bb             | 38 ++++++++-
 meta-oe/recipes-support/cli11/cli11_2.3.2.bb  |  3 +-
 .../span-lite/span-lite_0.10.3.bb             |  1 -
 .../include/ptest-packagelists-meta-perl.inc  | 50 +++++++++++
 .../images/meta-perl-base.bb                  |  0
 .../images/meta-perl-image.bb                 |  0
 .../images/meta-perl-ptest-all-image.bb       | 25 ++++++
 .../images/meta-perl-ptest-fast-image.bb      |  6 ++
 .../images/meta-perl-ptest-image.bb           | 41 ++++++++++
 .../packagegroups/packagegroup-meta-perl.bb   |  0
 .../images/meta-perl-ptest-image.bb           |  9 --
 .../ptest-packagelists-meta-python.inc        | 82 +++++++++++++++++++
 .../images/meta-python-ptest-all-image.bb     | 25 ++++++
 .../images/meta-python-ptest-fast-image.bb    |  6 ++
 .../images/meta-python-ptest-image.bb         | 38 ++++++++-
 .../python/python3-aspectlib/run-ptest        |  2 +-
 .../python/python3-aspectlib_2.0.0.bb         |  4 +-
 20 files changed, 405 insertions(+), 16 deletions(-)
 create mode 100644 meta-oe/conf/include/ptest-packagelists-meta-oe.inc
 create mode 100644 meta-oe/recipes-core/images/meta-oe-ptest-all-image.bb
 create mode 100644 meta-oe/recipes-core/images/meta-oe-ptest-fast-image.bb
 create mode 100644 meta-perl/conf/include/ptest-packagelists-meta-perl.inc
 rename meta-perl/{recipes-perl => recipes-core}/images/meta-perl-base.bb (100%)
 rename meta-perl/{recipes-perl => recipes-core}/images/meta-perl-image.bb (100%)
 create mode 100644 meta-perl/recipes-core/images/meta-perl-ptest-all-image.bb
 create mode 100644 meta-perl/recipes-core/images/meta-perl-ptest-fast-image.bb
 create mode 100644 meta-perl/recipes-core/images/meta-perl-ptest-image.bb
 rename meta-perl/{recipes-perl => recipes-core}/packagegroups/packagegroup-meta-perl.bb (100%)
 delete mode 100644 meta-perl/recipes-perl/images/meta-perl-ptest-image.bb
 create mode 100644 meta-python/conf/include/ptest-packagelists-meta-python.inc
 create mode 100644 meta-python/recipes-core/images/meta-python-ptest-all-image.bb
 create mode 100644 meta-python/recipes-core/images/meta-python-ptest-fast-image.bb
Signature · 1 line
2.34.1