Skip to contentSkip to conversation

scripts: send_qa_email: skip regression report when missing base or target

PATCH In review Alexis Lothoré 2 participants 5 messages Jun 21, 2024 – Jul 2, 2024

lore
Share & tools
lore
#0 lore raw

cover scripts: send_qa_email: skip regression report when missing base or target

Hello,
this series is a follow-up to the issue encountered in the autobuilder during the send_qa_email step. The corresponding bugzilla can be found in [1].

To be honest, I am not sure why the issue triggers only now, after inspecting the code, I would have expected this issue to start appearing a few months ago on any build not being a release (ie testing branches). My best guesses are either about a python update which changed subprocess behavior, or, more likely, some features of send_qa_email that have finally reached new branches (I am not sure when and how the yocto-autobuilder-helper branch is updated in the autobuilder).

The first commits are small fixes about issues I have encountered while sorting back my testing environment, the main fix is in the last commit.

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15503

Alexis Lothoré (3):
scripts: send_qa_email: fix invalid regex syntax warning
scripts: test_send_qa_email: fix broken tests
scripts: send_qa_email: do not try to generate a regression report
when missing base and/or target

 scripts/send_qa_email.py      | 11 +++++++----
 scripts/test_send_qa_email.py | 13 ++++++++++---
 2 files changed, 17 insertions(+), 7 deletions(-)
Signature · 1 line
2.45.2
#1 lore raw

PATCH 1/3 scripts: send_qa_email: fix invalid regex syntax warning

From: Alexis Lothoré <[email protected]>

When send_qa_email is executed with python >= 3.12, the following warnings are emitted:

/home/alexis/src/yocto_ci/yocto-autobuilder-helper/./scripts/send_qa_email.py:22: SyntaxWarning: invalid escape sequence '\d'
p = re.compile('\d{8}-\d+')
/home/alexis/src/yocto_ci/yocto-autobuilder-helper/./scripts/send_qa_email.py:67: SyntaxWarning: invalid escape sequence '\/'
tested_revision = re.match('refs\/tags\/.*\/\d+-g([a-f0-9]+)\/\d', latest_test_tag).group(1)

This warning has been introduced to replace the DeprecationWarning initially raised on such escape code in string regex. Python 3.12 changelog ([1]) states that raw string must be used in this case. Update both send_qa_email and its tests file.

[1] https://docs.python.org/dev/whatsnew/3.12.html#other-language-changes

---
 scripts/send_qa_email.py      | 4 ++--
 scripts/test_send_qa_email.py | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
scripts/send_qa_email.py +2 -2
index 8d9250d555d4..e258fa131fcb 100755
@@ -19,7 +19,7 @@ TEST_RESULTS_REPOSITORY_URL="[email protected]:yocto-testresults"
 TEST_RESULTS_DRY_RUN_REPOSITORY_URL="git://git.yoctoproject.org/yocto-testresults"
 
 def is_release_version(version):
-    p = re.compile('\d{8}-\d+')
+    p = re.compile(r'\d{8}-\d+')
     return version is not None and p.match(version) is None
 
 def get_previous_tag(targetrepodir, version):
@@ -64,7 +64,7 @@ def get_last_tested_rev_on_branch(branch, test_results_url, log):
     tags_list = subprocess.check_output(["git", "ls-remote", "--refs", "-t", test_results_url, "refs/tags/" + branch + "/*"]).decode('utf-8').strip()
     latest_test_tag=tags_list.splitlines()[-1].split()[1]
     # From test results tag, extract Poky revision
-    tested_revision = re.match('refs\/tags\/.*\/\d+-g([a-f0-9]+)\/\d', latest_test_tag).group(1)
+    tested_revision = re.match(r'refs\/tags\/.*\/\d+-g([a-f0-9]+)\/\d', latest_test_tag).group(1)
     log.info(f"Last tested revision on branch {branch} is {tested_revision}")
     return tested_revision
 
scripts/test_send_qa_email.py +2 -2
index 415d8cbbbccd..64ae80a1852d 100755
@@ -31,8 +31,8 @@ class TestVersion(unittest.TestCase):
         {"input": {"version": "4.1_M1.rc1"}, "expected": "yocto-4.0"},
         {"input": {"version": "4.1_M1.rc4"}, "expected": "yocto-4.0"},
         {"input": {"version": "4.1.rc4"}, "expected": "yocto-4.0"},
-        {"input": {"version": "yocto-5.0_M1.rc1"}, "expected": "yocto-4.3(\.\d)?"},
-        {"input": {"version": "yocto-5.0_M1.rc2"}, "expected": "yocto-4.3(\.\d)?"},
+        {"input": {"version": "yocto-5.0_M1.rc1"}, "expected": r"yocto-4.3(\.\d)?"},
+        {"input": {"version": "yocto-5.0_M1.rc2"}, "expected": r"yocto-4.3(\.\d)?"},
         {"input": {"version": "yocto-5.0_M2.rc1"}, "expected": "5.0_M1"},
     ]
 
Signature · 1 line
2.45.2
#2 lore raw

PATCH 2/3 scripts: test_send_qa_email: fix broken tests

In reply to Alexis Lothoré

From: Alexis Lothoré <[email protected]>

Commit 1dc77f590875 ("scripts: send_qa_email: add dry-run mode") updated the get_regression_base_and_target to allow it to receive a test result url. Unfortunately, this function's tests have not been updated accordingly, so they currently fail because of the missing parameter.

Add a default url in test case and update the function call with this default value.

---
 scripts/test_send_qa_email.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/test_send_qa_email.py +8 -1
index 64ae80a1852d..8b19975c31d5 100755
@@ -42,6 +42,8 @@ class TestVersion(unittest.TestCase):
         {"input": None, "expected":False}
     ]
 
+    test_results_url = "git://git.yoctoproject.org/yocto-testresults"
+
     # This data represent real data returned by utils.getcomparisonbranch
     # and the release argument passed to send-qa-email script
     regression_inputs = [
@@ -76,7 +78,12 @@ class TestVersion(unittest.TestCase):
         for data in self.regression_inputs:
             with self.subTest(data['name']):
                 base, target = send_qa_email.get_regression_base_and_target(
-                    data['input']['targetbranch'], data['input']['basebranch'], data['input']['release'], os.environ.get("POKY_PATH"), log)
+                    data['input']['targetbranch'],
+                    data['input']['basebranch'],
+                    data['input']['release'],
+                    os.environ.get("POKY_PATH"),
+                    self.test_results_url,
+                    log)
                 expected_base, expected_target = data["expected"]
                 # The comparison base can not be set statically in tests when it is supposed to be the previous tag,
                 # since the result will depend on current tags
Signature · 1 line
2.45.2
#3 lore raw

PATCH 3/3 scripts: send_qa_email: do not try to generate a regression report when missing base and/or target

In reply to Alexis Lothoré

From: Alexis Lothoré <[email protected]>

Fixes [YOCTO 15503]

When running autobuilder onm a testing or stable branch, the following exception may occur:

Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 278, in <module>

    send_qa_email()
  File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 207, in send_qa_email
    generate_regression_report(querytool, targetrepodir, regression_base, regression_target, tempdir, args.results_dir, log)
  File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 93, in generate_regression_report
    regreport = subprocess.check_output([querytool, "regression-report", base, target, '-t', resultdir])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.11/subprocess.py", line 1883, in _execute_child
    self.pid = _fork_exec(
               ^^^^^^^^^^^

TypeError: expected str, bytes or os.PathLike object, not NoneType

This issue is due to base and target revision being None, but subprocess module do not tolerate arguments being None,

Prevent the script from even trying to generate a regression report if we are missing some info about revisions to compare.

---
 scripts/send_qa_email.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
scripts/send_qa_email.py +5 -2
index e258fa131fcb..39e938284972 100755
@@ -203,8 +203,11 @@ def send_qa_email():
             utils.printheader("Processing regression report")
             try:
                 regression_base, regression_target = get_regression_base_and_target(targetbranch, basebranch, args.release, targetrepodir, test_results_url, log)
-                log.info(f"Generating regression report between {regression_base} and {regression_target}")
-                generate_regression_report(querytool, targetrepodir, regression_base, regression_target, tempdir, args.results_dir, log)
+                if regression_base and regression_target:
+                    log.info(f"Generating regression report between {regression_base} and {regression_target}")
+                    generate_regression_report(querytool, targetrepodir, regression_base, regression_target, tempdir, args.results_dir, log)
+                else:
+                    log.info("Can not guess regression base and/or target, skip regression report")
             except subprocess.CalledProcessError as e:
                 error = str(e)
                 exitcode = 1
Signature · 1 line
2.45.2