scripts: send_qa_email: skip regression report when missing base or target
cover scripts: send_qa_email: skip regression report when missing base or target
Hello,
this series is a follow-up to the issue encountered in the autobuilder during the send_qa_email step. The corresponding bugzilla can be found in [1].
To be honest, I am not sure why the issue triggers only now, after inspecting the code, I would have expected this issue to start appearing a few months ago on any build not being a release (ie testing branches). My best guesses are either about a python update which changed subprocess behavior, or, more likely, some features of send_qa_email that have finally reached new branches (I am not sure when and how the yocto-autobuilder-helper branch is updated in the autobuilder).
The first commits are small fixes about issues I have encountered while sorting back my testing environment, the main fix is in the last commit.
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15503
Alexis Lothoré (3):
scripts: send_qa_email: fix invalid regex syntax warning
scripts: test_send_qa_email: fix broken tests
scripts: send_qa_email: do not try to generate a regression report
when missing base and/or target
scripts/send_qa_email.py | 11 +++++++---- scripts/test_send_qa_email.py | 13 ++++++++++--- 2 files changed, 17 insertions(+), 7 deletions(-)
Signature · 1 line
2.45.2
PATCH 1/3 scripts: send_qa_email: fix invalid regex syntax warning
From: Alexis Lothoré <[email protected]>
When send_qa_email is executed with python >= 3.12, the following warnings are emitted:
/home/alexis/src/yocto_ci/yocto-autobuilder-helper/./scripts/send_qa_email.py:22: SyntaxWarning: invalid escape sequence '\d'
p = re.compile('\d{8}-\d+')/home/alexis/src/yocto_ci/yocto-autobuilder-helper/./scripts/send_qa_email.py:67: SyntaxWarning: invalid escape sequence '\/'
tested_revision = re.match('refs\/tags\/.*\/\d+-g([a-f0-9]+)\/\d', latest_test_tag).group(1)
This warning has been introduced to replace the DeprecationWarning initially raised on such escape code in string regex. Python 3.12 changelog ([1]) states that raw string must be used in this case. Update both send_qa_email and its tests file.
[1] https://docs.python.org/dev/whatsnew/3.12.html#other-language-changes
--- scripts/send_qa_email.py | 4 ++-- scripts/test_send_qa_email.py | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
scripts/send_qa_email.py +2 -2
@@ -19,7 +19,7 @@ TEST_RESULTS_REPOSITORY_URL="[email protected]:yocto-testresults" TEST_RESULTS_DRY_RUN_REPOSITORY_URL="git://git.yoctoproject.org/yocto-testresults" def is_release_version(version): - p = re.compile('\d{8}-\d+') + p = re.compile(r'\d{8}-\d+') return version is not None and p.match(version) is None def get_previous_tag(targetrepodir, version):
@@ -64,7 +64,7 @@ def get_last_tested_rev_on_branch(branch, test_results_url, log): tags_list = subprocess.check_output(["git", "ls-remote", "--refs", "-t", test_results_url, "refs/tags/" + branch + "/*"]).decode('utf-8').strip() latest_test_tag=tags_list.splitlines()[-1].split()[1] # From test results tag, extract Poky revision - tested_revision = re.match('refs\/tags\/.*\/\d+-g([a-f0-9]+)\/\d', latest_test_tag).group(1) + tested_revision = re.match(r'refs\/tags\/.*\/\d+-g([a-f0-9]+)\/\d', latest_test_tag).group(1) log.info(f"Last tested revision on branch {branch} is {tested_revision}") return tested_revision
scripts/test_send_qa_email.py +2 -2
@@ -31,8 +31,8 @@ class TestVersion(unittest.TestCase): {"input": {"version": "4.1_M1.rc1"}, "expected": "yocto-4.0"}, {"input": {"version": "4.1_M1.rc4"}, "expected": "yocto-4.0"}, {"input": {"version": "4.1.rc4"}, "expected": "yocto-4.0"}, - {"input": {"version": "yocto-5.0_M1.rc1"}, "expected": "yocto-4.3(\.\d)?"}, - {"input": {"version": "yocto-5.0_M1.rc2"}, "expected": "yocto-4.3(\.\d)?"}, + {"input": {"version": "yocto-5.0_M1.rc1"}, "expected": r"yocto-4.3(\.\d)?"}, + {"input": {"version": "yocto-5.0_M1.rc2"}, "expected": r"yocto-4.3(\.\d)?"}, {"input": {"version": "yocto-5.0_M2.rc1"}, "expected": "5.0_M1"}, ]
Signature · 1 line
2.45.2
PATCH 2/3 scripts: test_send_qa_email: fix broken tests
In reply to Alexis LothoréFrom: Alexis Lothoré <[email protected]>
Commit 1dc77f590875 ("scripts: send_qa_email: add dry-run mode") updated the get_regression_base_and_target to allow it to receive a test result url. Unfortunately, this function's tests have not been updated accordingly, so they currently fail because of the missing parameter.
Add a default url in test case and update the function call with this default value.
--- scripts/test_send_qa_email.py | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/test_send_qa_email.py +8 -1
@@ -42,6 +42,8 @@ class TestVersion(unittest.TestCase): {"input": None, "expected":False} ] + test_results_url = "git://git.yoctoproject.org/yocto-testresults" + # This data represent real data returned by utils.getcomparisonbranch # and the release argument passed to send-qa-email script regression_inputs = [
@@ -76,7 +78,12 @@ class TestVersion(unittest.TestCase): for data in self.regression_inputs: with self.subTest(data['name']): base, target = send_qa_email.get_regression_base_and_target( - data['input']['targetbranch'], data['input']['basebranch'], data['input']['release'], os.environ.get("POKY_PATH"), log) + data['input']['targetbranch'], + data['input']['basebranch'], + data['input']['release'], + os.environ.get("POKY_PATH"), + self.test_results_url, + log) expected_base, expected_target = data["expected"] # The comparison base can not be set statically in tests when it is supposed to be the previous tag, # since the result will depend on current tags
Signature · 1 line
2.45.2
PATCH 3/3 scripts: send_qa_email: do not try to generate a regression report when missing base and/or target
In reply to Alexis LothoréFrom: Alexis Lothoré <[email protected]>
Fixes [YOCTO 15503]
When running autobuilder onm a testing or stable branch, the following exception may occur:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 278, in <module>
send_qa_email()
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 207, in send_qa_email
generate_regression_report(querytool, targetrepodir, regression_base, regression_target, tempdir, args.results_dir, log)
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send_qa_email.py", line 93, in generate_regression_report
regreport = subprocess.check_output([querytool, "regression-report", base, target, '-t', resultdir])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib64/python3.11/subprocess.py", line 1883, in _execute_child
self.pid = _fork_exec(
^^^^^^^^^^^TypeError: expected str, bytes or os.PathLike object, not NoneType
This issue is due to base and target revision being None, but subprocess module do not tolerate arguments being None,
Prevent the script from even trying to generate a regression report if we are missing some info about revisions to compare.
--- scripts/send_qa_email.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
scripts/send_qa_email.py +5 -2
@@ -203,8 +203,11 @@ def send_qa_email(): utils.printheader("Processing regression report") try: regression_base, regression_target = get_regression_base_and_target(targetbranch, basebranch, args.release, targetrepodir, test_results_url, log) - log.info(f"Generating regression report between {regression_base} and {regression_target}") - generate_regression_report(querytool, targetrepodir, regression_base, regression_target, tempdir, args.results_dir, log) + if regression_base and regression_target: + log.info(f"Generating regression report between {regression_base} and {regression_target}") + generate_regression_report(querytool, targetrepodir, regression_base, regression_target, tempdir, args.results_dir, log) + else: + log.info("Can not guess regression base and/or target, skip regression report") except subprocess.CalledProcessError as e: error = str(e) exitcode = 1
Signature · 1 line
2.45.2
I've tested this series with stable/scarthgap-nut and can confirm that it fixes the persistent failures in send-qa-email that I've been seeing.
Steve