Skip to contentSkip to conversation

gitlab-ci: Refactor show logs code and archive logs

PATCH In review Wainer dos Santos Moschetta 3 participants 8 messages Dec 11, 2020 – Dec 17, 2020

lore
Share & tools
lore
#1 lore raw

PATCH 1/3 tests/acceptance: Bump avocado requirements to 83.0

In reply to Wainer dos Santos Moschetta

To use Avocado's testlogs plug-in on CI it is required to use its 83.0 or greater version.

---
 tests/requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/requirements.txt +1 -1
index a1c631fa59..62e8ffd28c 100644
@@ -1,5 +1,5 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==81.0
+avocado-framework==83.0
 pycdlib==1.11.0
Signature · 1 line
2.28.0
#3 lore raw

PATCH 2/3 gitlab-ci: Refactor code that show logs of failed acceptances

In reply to Wainer dos Santos Moschetta

Replace the code (python) on after_script of the acceptance jobs that is currently used to show the logs of failed tests. Instead it is used the Avocado's testlogs plug-in which works likewise.

---
 .gitlab-ci.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
.gitlab-ci.yml +2 -1
index 98bff03b47..85aa20ffde 100644
@@ -58,13 +58,14 @@ include:
     - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
     - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
            >> ~/.config/avocado/avocado.conf
+    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL"]'
+           >> ~/.config/avocado/avocado.conf
     - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
         du -chs ${CI_PROJECT_DIR}/avocado-cache ;
       fi
     - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
   after_script:
     - cd build
-    - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP", "CANCEL")]' | xargs cat
     - du -chs ${CI_PROJECT_DIR}/avocado-cache
 
 build-system-ubuntu:
Signature · 1 line
2.28.0
#5 lore raw

PATCH 3/3 gitlab-ci: Archive logs of acceptance tests

In reply to Wainer dos Santos Moschetta

Keep the logs of acceptance tests for two days on GitLab. If you want to make it available for more time, click on the 'Keep' button on the Job page at web UI.

By default GitLab will archive artifacts only if the job succeed. Instead let's keep it on both success and failure, so it gives the opportunity to the developer/maintainer to check the error logs as well as to the logs of CANCEL tests (not shown on the job logs).

---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)
.gitlab-ci.yml +4 -0
index 85aa20ffde..bf3df843e2 100644
@@ -49,8 +49,12 @@ include:
       - ${CI_PROJECT_DIR}/avocado-cache
     policy: pull-push
   artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+    when: always
+    expire_in: 2 days
     paths:
       - build/tests/results/latest/results.xml
+      - build/tests/results/latest/test-results
     reports:
       junit: build/tests/results/latest/results.xml
   before_script:
Signature · 1 line
2.28.0
#0 lore raw

cover gitlab-ci: Refactor show logs code and archive logs

Since the Travis CI time, it has carried (commit 67892c9537d9ac500934) a python code to print the logs of acceptance tests in case of fail. Recently Avocado released a new plug-in (called "testlogs") which allow us to configure the framework to show the logs of tests which finished with a given status. For example, print the logs of tests which FAIL or CANCEL. This series is about replacing the old implementation by that native Avocado feature, on the GitLab CI.

I took that opportunity to also introduce a change in the configuration to archive the tests results files on GitLab CI for 2 days, regarless the job failed or not. If the logs are important enough to retain them, it is possible to do so by clicking on a 'Keep' button on UI; or you can simply download it to your workstation.

Here is an example of job that succeed:
https://gitlab.com/wainersm/qemu/-/jobs/905873563

And its results files can be found at:
https://gitlab.com/wainersm/qemu/-/jobs/905873563/artifacts/browse/build/tests/results/latest/test-results/

Then an example of job that failed (I changed a test, forcing its fail):
https://gitlab.com/wainersm/qemu/-/jobs/906089498

[1] https://github.com/avocado-framework/avocado/issues/4266

Wainer dos Santos Moschetta (3):
tests/acceptance: Bump avocado requirements to 83.0
gitlab-ci: Refactor code that show logs of failed acceptances
gitlab-ci: Archive logs of acceptance tests

 .gitlab-ci.yml         | 7 ++++++-
 tests/requirements.txt | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)
Signature · 1 line
2.28.0
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] tests/acceptance: Bump avocado requirements to 83.0

In reply to Wainer dos Santos Moschetta
Quoted 9 lines · Wainer dos Santos Moschetta

To use Avocado's testlogs plug-in on CI it is required to use its 83.0 or greater version.

---
 tests/requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by on v1 2/3
#4 lore raw

Re: [PATCH 2/3] gitlab-ci: Refactor code that show logs of failed acceptances

In reply to Wainer dos Santos Moschetta
Quoted 9 lines · Wainer dos Santos Moschetta

Replace the code (python) on after_script of the acceptance jobs that is currently used to show the logs of failed tests. Instead it is used the Avocado's testlogs plug-in which works likewise.

---
  .gitlab-ci.yml | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
Reviewed-by on v1 3/3
#6 lore raw

Re: [PATCH 3/3] gitlab-ci: Archive logs of acceptance tests

In reply to Wainer dos Santos Moschetta
Quoted 22 lines · Wainer dos Santos Moschetta

Keep the logs of acceptance tests for two days on GitLab. If you want to make it available for more time, click on the 'Keep' button on the Job page at web UI.

By default GitLab will archive artifacts only if the job succeed. Instead let's keep it on both success and failure, so it gives the opportunity to the developer/maintainer to check the error logs as well as to the logs of CANCEL tests (not shown on the job logs).

---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)
.gitlab-ci.yml +0 -0
index 85aa20ffde..bf3df843e2 100644
@@ -49,8 +49,12 @@ include:
       - ${CI_PROJECT_DIR}/avocado-cache
     policy: pull-push

.gitlab-ci.yml include: · line 60

Quoted 10 lines
.gitlab-ci.yml +4 -0
   artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+    when: always
+    expire_in: 2 days
     paths:
       - build/tests/results/latest/results.xml
+      - build/tests/results/latest/test-results
     reports:
       junit: build/tests/results/latest/results.xml
   before_script:
Reviewed-by on v1 3/3
#7 lore raw

Re: [PATCH 3/3] gitlab-ci: Archive logs of acceptance tests

In reply to Wainer dos Santos Moschetta
Quoted 10 lines · Wainer dos Santos Moschetta

Keep the logs of acceptance tests for two days on GitLab. If you want to make it available for more time, click on the 'Keep' button on the Job page at web UI.

By default GitLab will archive artifacts only if the job succeed. Instead let's keep it on both success and failure, so it gives the opportunity to the developer/maintainer to check the error logs as well as to the logs of CANCEL tests (not shown on the job logs).

Quoted 5 lines
---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)