RFC revive runtime/cases/_ptest.py
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest5 patches 3 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | oeqa/targetcontrol.py: simplify checking for qemu_use_kvmmeta/ | +2 | 0 replies | |
| 2/5 | runtime/cases/_ptest.py: revive itmeta/ | +30 | 0 replies | |
| 3/5 | oeqa/utils/logparser.py: add skip statusmeta/ | +3 | 0 replies | |
| 4/5 | runtime/cases/_ptest.py: add skip statusmeta/ | +6 | 0 replies | |
| 5/5 | runtime/cases/_ptest.py: rename it to ptest.pymeta/ | +0 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 0/5] revive runtime/cases/_ptest.py
Hello,
These patches can make ptest test case work, RP has suggested we write a tool to do the regression check on ptest result, I think that the use case is like:
$ bitbake <image> -ctestiamge # Suppose we add ptest to default test cases in the future # Upgrade a recipe form V1.0 to V1.1 $ bitbake <image> -ctestiamge # Run the test again
Then the regression check tool can report what's different (passed, failed, skipped) between V1.0 and V1.1.
Currently, I'm not sure about where to save the ptest results, I saved it to ${WORKDIR}/testimage/ptest_log atm, e.g.:
tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/testimage/ptest_log
But it will be removed after -cclean, then no regression check can be made, so I'd like to save the ptest results to DEPLOY_DIR_IMAGE if no objections, and make -cclean not remove them. (Or only keep the latest 2 results).
And I'm not sure where to add the regression check/tool, maybe one of:
1) Add a separate tool in oe-core/scripts, this can make it easy to do regression check among different build directories, and runtime/cases/_ptest.py can invoke it.
2) Add it to runtime/cases/_ptest.py directly
3) Add it to buildhistory
I prefer the first one, please feel free to give your comments.
// Robert
The following changes since commit ef68005a8c527e9b1d05b7769f0ec8ebe9ec3f91:
webkitgtk: Upgrade to 2.16.5 (2017-07-17 13:49:04 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/ptest
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ptest
Robert Yang (5):
oeqa/targetcontrol.py: simplify checking for qemu_use_kvm runtime/cases/_ptest.py: revive it oeqa/utils/logparser.py: add skip status runtime/cases/_ptest.py: add skip status runtime/cases/_ptest.py: rename it to ptest.py
meta/lib/oeqa/runtime/cases/_ptest.py | 103 ---------------------------------- meta/lib/oeqa/runtime/cases/ptest.py | 77 +++++++++++++++++++++++++ meta/lib/oeqa/targetcontrol.py | 5 +- meta/lib/oeqa/utils/logparser.py | 5 +- 4 files changed, 82 insertions(+), 108 deletions(-) delete mode 100644 meta/lib/oeqa/runtime/cases/_ptest.py create mode 100644 meta/lib/oeqa/runtime/cases/ptest.py
Signature · 1 line
2.11.0