CI: Make pytest export JUnitXML
PATCH 1/1 CI: Make pytest export JUnitXML
Both GitLab and Azure (and other CI systems) have native support for displaying JUnitXML test report results. The pytest framework that we use can generate these reports. Change our CI tests so that they will generate these reports and then have the respective CI platform pick them up. We write to different locations because of where each CI is (and isn't) able to easily pass things along.
--- .azure-pipelines.yml | 12 +++++++++++- .gitlab-ci.yml | 5 ++++- 2 files changed, 15 insertions(+), 2 deletions(-)
.azure-pipelines.yml +11 -1
@@ -287,7 +287,7 @@ stages: export PATH=/opt/qemu/bin:/tmp/uboot-test-hooks/bin:\${PATH} export PYTHONPATH=/tmp/uboot-test-hooks/py/travis-ci # "\${var:+"-k \$var"}" expands to "" if \$var is empty, "-k \$var" if not - ./test/py/test.py -ra -o cache_dir="\$UBOOT_TRAVIS_BUILD_DIR"/.pytest_cache --bd \${TEST_PY_BD} \${TEST_PY_ID} \${TEST_PY_TEST_SPEC:+"-k \${TEST_PY_TEST_SPEC}"} --build-dir "\$UBOOT_TRAVIS_BUILD_DIR" --report-dir "\$UBOOT_TRAVIS_BUILD_DIR" + ./test/py/test.py -ra -o cache_dir="\$UBOOT_TRAVIS_BUILD_DIR"/.pytest_cache --bd \${TEST_PY_BD} \${TEST_PY_ID} \${TEST_PY_TEST_SPEC:+"-k \${TEST_PY_TEST_SPEC}"} --build-dir "\$UBOOT_TRAVIS_BUILD_DIR" --report-dir "\$UBOOT_TRAVIS_BUILD_DIR" --junitxml=\$(System.DefaultWorkingDirectory)/results.xml # the below corresponds to .gitlab-ci.yml "after_script" rm -rf /tmp/uboot-test-hooks /tmp/venv EOF
@@ -371,6 +371,7 @@ stages: docker run "$@" --device /dev/fuse:/dev/fuse \ -v $PWD:$(work_dir) \ -v $(Pipeline.Workspace):$(Pipeline.Workspace) \ + -v $(System.DefaultWorkingDirectory):$(System.DefaultWorkingDirectory) \ -e WORK_DIR="${WORK_DIR}" \ -e TEST_PY_BD="${TEST_PY_BD}" \ -e TEST_PY_ID="${TEST_PY_ID}" \
@@ -378,6 +379,10 @@ stages: -e OVERRIDE="${OVERRIDE}" \ -e BUILD_ENV="${BUILD_ENV}" $(ci_runner_image) \ $(Pipeline.Workspace)/testsh/test.sh + - task: PublishTestResults@2 + inputs: + testResultsFormat: 'JUnit' + testResultsFiles: 'results.xml' - stage: test_py_qemu jobs:
@@ -495,6 +500,7 @@ stages: docker run "$@" --device /dev/fuse:/dev/fuse \ -v $PWD:$(work_dir) \ -v $(Pipeline.Workspace):$(Pipeline.Workspace) \ + -v $(System.DefaultWorkingDirectory):$(System.DefaultWorkingDirectory) \ -e WORK_DIR="${WORK_DIR}" \ -e TEST_PY_BD="${TEST_PY_BD}" \ -e TEST_PY_ID="${TEST_PY_ID}" \
@@ -503,6 +509,10 @@ stages: -e BUILD_ENV="${BUILD_ENV}" $(ci_runner_image) \ $(Pipeline.Workspace)/testsh/test.sh retryCountOnTaskFailure: 2 # QEMU may be too slow, etc. + - task: PublishTestResults@2 + inputs: + testResultsFormat: 'JUnit' + testResultsFiles: 'results.xml' - stage: world_build jobs:
.gitlab-ci.yml +4 -1
@@ -41,7 +41,7 @@ stages: fi after_script: - - cp -v /tmp/${TEST_PY_BD}/*.{html,css} . + - cp -v /tmp/${TEST_PY_BD}/*.{html,css,xml} . - rm -rf /tmp/uboot-test-hooks /tmp/venv script: # If we've been asked to use clang only do one configuration.
@@ -83,11 +83,14 @@ stages: ./test/py/test.py -ra --bd ${TEST_PY_BD} ${TEST_PY_ID} ${TEST_PY_TEST_SPEC:+"-k ${TEST_PY_TEST_SPEC}"} --build-dir "$UBOOT_TRAVIS_BUILD_DIR" + --junitxml=/tmp/${TEST_PY_BD}/results.xml artifacts: when: always paths: - "*.html" - "*.css" + reports: + junit: results.xml expire_in: 1 week .world_build:
Signature · 1 line
2.34.1
Quoted 8 lines · Tom Rini
Both GitLab and Azure (and other CI systems) have native support for displaying JUnitXML test report results. The pytest framework that we use can generate these reports. Change our CI tests so that they will generate these reports and then have the respective CI platform pick them up. We write to different locations because of where each CI is (and isn't) able to easily pass things along.
Quoted 6 lines
Signed-off-by Tom Rini <[email protected]>--- .azure-pipelines.yml | 12 +++++++++++- .gitlab-ci.yml | 5 ++++- 2 files changed, 15 insertions(+), 2 deletions(-)
Nice!
Quoted 8 lines · Tom Rini
Both GitLab and Azure (and other CI systems) have native support for displaying JUnitXML test report results. The pytest framework that we use can generate these reports. Change our CI tests so that they will generate these reports and then have the respective CI platform pick them up. We write to different locations because of where each CI is (and isn't) able to easily pass things along.
[...]
Applied to u-boot/master, thanks!
Signature · 1 line
Tom