add gitlab-ci jobs to the daily-mail
cover add gitlab-ci jobs to the daily-mail
This patch series adds gitlab-ci job failures to the daily-mail, this includes both defconfig build failures and runtime tests failures.
Patches 1-3: gitlab-ci infrastructure for defconfigs and runtime-tests. Patches 4-5: add defconfigs and runtime-tests to per-developer
notifications.
Patches 6-8: add the formatting logic for defconfig and runtime-tests. Patches 9-10: adjust command-line arguments and enable gitlab-ci
integration.
Victor Huesca (10):
utils/daily-mail: new data: gitlab-ci jobs utils/daily-mail: new gitlab-ci data: defconfigs utils/daily-mail: new gitlab-ci data: runtime tests utils/daily-mail: add defconfig support in developers notifications utils/daily-mail: add runtime tests support in developers
notifications
utils/daily-mail: prepare the mail formatting to support gitlab-ci
results
utils/daily-mail: add formatting for defconfig results utils/daily-mail: add formatting for runtime-test results utils/daily-mail: add argument for gitlab-ci related data utils/daily-mail: enable gitlab-ci emails
utils/daily-mail | 342 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 313 insertions(+), 29 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/10 utils/daily-mail: new data: gitlab-ci jobs
Buildroot have a mirror repository on gitlab.com that is used to run continious integration tasks. Among these tasks, a complete checkup of defconfigs and runtime-tests is regularly performed.
This patch allows to retrieve tests that have failed over a given period of time. It is based on the (very imperfect) gitlab-ci API.
Due to multiple limitations of the API (pagination of results, crucial data requiring a dedicated request, etc.), this implementation performs multiple optimisation to be able to retreive the tasks needed.
--- utils/daily-mail | 157 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 156 insertions(+), 1 deletion(-)
utils/daily-mail +156 -1
@@ -7,7 +7,7 @@ import textwrap import mysmtplib as smtplib from email.mime.text import MIMEText from email.utils import formatdate -from datetime import date, timedelta +from datetime import date, timedelta, datetime import localconfig import csv from collections import defaultdict
@@ -16,6 +16,10 @@ import argparse import re import json from packaging import version +import certifi +from urllib3 import HTTPSConnectionPool +from urllib3.exceptions import HTTPError +from multiprocessing import Pool as ProcessPool sys.path.append(os.path.join(localconfig.brbase, "utils")) import getdeveloperlib # noqa: E402
@@ -27,6 +31,7 @@ baseurl = "autobuild.buildroot.net" http_baseurl = "http://" + baseurl developers = getdeveloperlib.parse_developers(localconfig.brbase) +gitlab_ci_http_pool = None def get_branches():
@@ -469,6 +474,156 @@ def get_outdated_pkgs(path): return sorted(s, key=lambda pkg: pkg['name']) +def request_paginated_url(pool, url, page_size=100, page_name='page', **request_kwargs): + ''' + Simple helper to retreive data from call paginated url. + Request's result is assumed to be json list. + ''' + request_kwargs.setdefault('fields', {}) + request_kwargs['fields']['per_page'] = page_size + results = [] + page = 1 + while "len(res) == page_size": + request_kwargs['fields'][page_name] = page + try: + req = pool.request('GET', url, **request_kwargs) + except HTTPError: + print('error: there were an error with gitlab api. ' + 'Please make sure your identification token is right and try again.') + break + if req.status != 200: + print('error: there were an error with gitlab api. ' + 'Please make sure your identification token is right and try again.') + break + res = json.loads(req.data) + results += res + page += 1 + if len(res) != page_size: + break + return results + + +def get_pipeline_details(project_id, pipeline, **request_kwargs): + pool = gitlab_ci_http_pool + pipeline_url = '/api/v4/projects/{}/pipelines/{}'.format(project_id, pipeline) + try: + pipeline_req = pool.request('GET', pipeline_url, **request_kwargs) + except HTTPError: + print('error: there were an error while fetching the gitlab-ci pipeline\'s details. ' + 'Please make sure your identification token is right and try again.') + return {} + if pipeline_req.status != 200: + print('error: there were an error while fetching the gitlab-ci pipeline\'s details. ' + 'Please make sure your identification token is right and try again.') + return {} + infos = json.loads(pipeline_req.data) + return infos + + +def get_pipeline_jobs(project_id, pipeline, kinds=None, **request_kwargs): + pool = gitlab_ci_http_pool + jobs_url = '/api/v4/projects/{}/pipelines/{}/jobs'.format(project_id, pipeline) + jobs = request_paginated_url(pool, jobs_url, **request_kwargs) + for job in jobs: + # Remove the unnecessary 'pipeline' field + del job['pipeline'] + # Backup the actual name and set the job's kind and name depending on the naming scheme + job['full_name'] = job['name'] + job['kind'] = 'other' + # Filter failed jobs then sort them + jobs = (job for job in jobs if job['status'] == 'failed') + if kinds: + jobs = (job for job in jobs if job['kind'] in kinds) + return sorted(jobs, key=lambda j: (j['name'], j['kind'])) + + +def get_gitlab_ci_pipelines(http_pool, process_pool, project_id, date, date_range=None): + ''' + Returns all gitlab-ci pipelines ran after the giver `min_date` + ''' + date_to = date + date_from = date - date_range if date_range else date + per_page = process_pool._processes + if per_page > 100: + raise ValueError('gitlab-ci supports a max pagination of 100 results per page. ' + 'The pool passed as argument must have 100 or less processes.') + fields = {'per_page': per_page} + url = '/api/v4/projects/{}/pipelines'.format(project_id) + page = 1 + pipelines = [] + + # Fetch pipelines `nb_processes` by `nb_processes` until `from_date` in reached + while True: + # List pipelines + fields['page'] = page + page += 1 + try: + cur_pipelines = http_pool.request('GET', url, fields=fields) + except HTTPError: + print('error: there were an error while fetching the gitlab-ci pipelines. ' + 'Please make sure your identification token is right and try again.') + return pipelines + if cur_pipelines.status != 200: + print('error: there were an error while fetching the gitlab-ci pipeline\'s details. ' + 'Please make sure your identification token is right and try again.') + return pipelines + cur_pipelines = json.loads(cur_pipelines.data) + + # Handle cases where the requested date is older than all pipelines + if len(cur_pipelines) == 0: + return pipelines + + # Request more infos about each pipeline (for instance the date) + pipelines_infos = [process_pool.apply_async(get_pipeline_details, (project_id, p['id'])) + for p in cur_pipelines] + pipelines_infos = [p.get() for p in pipelines_infos] + + # Add pipeline in the right period, break once the from_date is found + for p in pipelines_infos: + cur_date = datetime.strptime(p['finished_at'], '%Y-%m-%dT%H:%M:%S.%fZ').date() + if cur_date < date_from: + return pipelines + if cur_date > date_to: + continue + pipelines.append(p) + + return pipelines + + +def get_gitlab_ci(branches, date, date_range=None, kinds=None): + ''' + Retreive failed jobs from gitlab-ci grouped by pipelines + for the specified date. + If `kinds` list is provided, only jobs matching one of the + given kind will be returned. + ''' + header = {'PRIVATE-TOKEN': localconfig.gitlab_user_token} + project_id = localconfig.gitlab_project_id + failed_filter = {'fields': {'status': 'failed'}} + + # Build pools + global gitlab_ci_http_pool + gitlab_ci_http_pool = HTTPSConnectionPool('gitlab.com', port=443, cert_reqs='CERT_REQUIRED', + ca_certs=certifi.where(), timeout=30, headers=header) + process_pool = ProcessPool(processes=64) + + # Fetch pipelines 64 by 64 until `date_from` in reached + pipelines = get_gitlab_ci_pipelines(gitlab_ci_http_pool, process_pool, project_id, date, date_range) + + # Drop unnecessary branches + pipelines = [p for p in pipelines if p['ref'] in branches] + + # Get jobs ran by those pipelines + pipelines_jobs = [process_pool.apply_async(get_pipeline_jobs, (project_id, p['id'], kinds), + failed_filter) for p in pipelines] + for pipeline, jobs in zip(pipelines, pipelines_jobs): + pipeline['jobs'] = jobs.get() + + # Group pipelines by branch + pipelines = {branch: [p for p in pipelines if p['ref'] == branch] for branch in branches} + return pipelines + + def calculate_notifications(results, outdated_pkgs): ''' Prepare the notifications{} dict for the notifications to individual
Signature · 1 line
2.21.0
PATCH 2/10 utils/daily-mail: new gitlab-ci data: defconfigs
In reply to Victor HuescaThis patch adds defconfigs to the gitlab-ci data introduced by the previous patch.
--- utils/daily-mail | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
utils/daily-mail +6 -2
@@ -528,8 +528,12 @@ def get_pipeline_jobs(project_id, pipeline, kinds=None, **request_kwargs): # Remove the unnecessary 'pipeline' field del job['pipeline'] # Backup the actual name and set the job's kind and name depending on the naming scheme - job['full_name'] = job['name'] - job['kind'] = 'other' + name = job['full_name'] = job['name'] + if name.endswith('_defconfig'): + job['kind'] = 'defconfig' + job['name'] = name[:-10] + else: + job['kind'] = 'other' # Filter failed jobs then sort them jobs = (job for job in jobs if job['status'] == 'failed') if kinds:
Signature · 1 line
2.21.0
PATCH 3/10 utils/daily-mail: new gitlab-ci data: runtime tests
In reply to Victor HuescaThis patch adds runtime tests to the recognized gitlab-ci results.
--- utils/daily-mail | 3 +++ 1 file changed, 3 insertions(+)
utils/daily-mail +3 -0
@@ -532,6 +532,9 @@ def get_pipeline_jobs(project_id, pipeline, kinds=None, **request_kwargs): if name.endswith('_defconfig'): job['kind'] = 'defconfig' job['name'] = name[:-10] + elif name.startswith('tests.'): + job['kind'] = 'runtime-test' + job['name'] = name.split('.')[-1] else: job['kind'] = 'other' # Filter failed jobs then sort them
Signature · 1 line
2.21.0
PATCH 4/10 utils/daily-mail: add defconfig support in developers notifications
In reply to Victor HuescaThis patch adds defconfig job failures in the 'Notification' structure. This will allow to inform developers about failures related to the defconfigs they maintains.
This patch relys on the gitlab-ci infrasture introduced earlier on in this series as well as the patch adding support for defconfig in the getdeveloperlib already applied to master.
--- utils/daily-mail | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-)
utils/daily-mail +26 -2
@@ -77,6 +77,7 @@ class Notification: def __init__(self): self.arch_notifications = defaultdict(list) self.package_notifications = defaultdict(list) + self.defconfig_notifications = defaultdict(list) self.package_version_notification = []
@@ -166,6 +167,23 @@ def add_package_notification(branch, notifications, build_result): build_result['orphan'] = orphan +def add_defconfig_notification(branch, notifications, job): + ''' + Add to the notifications{} dict notifications that are related to + defconfig "maintainers". + ''' + orphan = True + for dev in developers: + if job['name'] in dev.defconfigs: + orphan = False + notif = notifications.setdefault(dev, Notification()) + notif.defconfig_notifications[branch].append(job) + if orphan: + job['orphan'] = True + notif = notifications.setdefault(get_orphan_developer(), Notification()) + notif.defconfig_notifications[branch].append(job) + + def add_outdated_pkg_notification(notifications, package): ''' Add to the notifications{} dict notifications that are related to
@@ -631,7 +649,7 @@ def get_gitlab_ci(branches, date, date_range=None, kinds=None): return pipelines -def calculate_notifications(results, outdated_pkgs): +def calculate_notifications(results, outdated_pkgs, gitlab_results): ''' Prepare the notifications{} dict for the notifications to individual developers, based on architecture developers, package developers,
@@ -649,6 +667,12 @@ def calculate_notifications(results, outdated_pkgs): for pkg in outdated_pkgs: add_outdated_pkg_notification(notifications, pkg) + for branch, pipelines in gitlab_results.iteritems(): + for pipeline in pipelines: + for job in pipeline['jobs']: + if job['kind'] == 'defconfig': + add_defconfig_notification(branch, notifications, job) + return notifications
@@ -691,7 +715,7 @@ def __main__(): results = {} results_by_reason = {} outdated_pkgs = get_outdated_pkgs(localconfig.pkg_stats) if 'outdated' in email_data else [] - notifications = calculate_notifications(results, outdated_pkgs) + notifications = calculate_notifications(results, outdated_pkgs, {}) smtp = smtplib.SMTP(localconfig.smtphost, localconfig.smtpport) smtp.starttls() smtp.login(localconfig.smtpuser, localconfig.smtppass)
Signature · 1 line
2.21.0
PATCH 4/10 utils/daily-mail: add defconfig support in developers notifications
Hello Victor and all,
Quoted 59 lines · Victor Huesca
This patch adds defconfig job failures in the 'Notification' structure. This will allow to inform developers about failures related to the defconfigs they maintains.
This patch relys on the gitlab-ci infrasture introduced earlier on in this series as well as the patch adding support for defconfig in the getdeveloperlib already applied to master.
Signed-off-by Victor Huesca <[email protected]>--- utils/daily-mail | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-)utils/daily-mail +19
-1@@ -77,6 +77,7 @@ class Notification: def __init__(self): self.arch_notifications = defaultdict(list) self.package_notifications = defaultdict(list) + self.defconfig_notifications = defaultdict(list) self.package_version_notification = []@@ -166,6 +167,23 @@ def add_package_notification(branch, notifications, build_result): build_result['orphan'] = orphan +def add_defconfig_notification(branch, notifications, job): + ''' + Add to the notifications{} dict notifications that are related to + defconfig "maintainers". + ''' + orphan = True + for dev in developers: + if job['name'] in dev.defconfigs: + orphan = False + notif = notifications.setdefault(dev, Notification()) + notif.defconfig_notifications[branch].append(job) + if orphan: + job['orphan'] = True + notif = notifications.setdefault(get_orphan_developer(), Notification()) + notif.defconfig_notifications[branch].append(job) + + def add_outdated_pkg_notification(notifications, package): ''' Add to the notifications{} dict notifications that are related to@@ -631,7 +649,7 @@ def get_gitlab_ci(branches, date, date_range=None, kinds=None): return pipelines -def calculate_notifications(results, outdated_pkgs): +def calculate_notifications(results, outdated_pkgs, gitlab_results): ''' Prepare the notifications{} dict for the notifications to individual developers, based on architecture developers, package developers,
utils/daily-mail def calculate_notifications(results, outdated_pkgs): · line 670
Quoted 4 lines
utils/daily-mail +1
-0@@ -649,6 +667,12 @@ def calculate_notifications(results, outdated_pkgs): for pkg in outdated_pkgs: add_outdated_pkg_notification(notifications, pkg) + for branch, pipelines in gitlab_results.iteritems():
Not sure if relevant, but dict.iteritems() does not exists anymore in Py3, and dict.items() now returns a "view"
(https://docs.python.org/3.0/whatsnew/3.0.html#views-and-iterators-instead-of-lists).
Quoted 10 lines
utils/daily-mail +5
-0+ for pipeline in pipelines: + for job in pipeline['jobs']: + if job['kind'] == 'defconfig': + add_defconfig_notification(branch, notifications, job) + return notifications
utils/daily-mail def __main__(): · line 721
Quoted 8 lines
utils/daily-mail +1
-1@@ -691,7 +715,7 @@ def __main__(): results = {} results_by_reason = {} outdated_pkgs = get_outdated_pkgs(localconfig.pkg_stats) if 'outdated' in email_data else [] - notifications = calculate_notifications(results, outdated_pkgs) + notifications = calculate_notifications(results, outdated_pkgs, {}) smtp = smtplib.SMTP(localconfig.smtphost, localconfig.smtpport) smtp.starttls() smtp.login(localconfig.smtpuser, localconfig.smtppass)
Regards,
Titouan
PATCH 5/10 utils/daily-mail: add runtime tests support in developers notifications
In reply to Victor HuescaThis patch adds runtime-test job failures in the 'Notification' structure.
This will allow to inform developers about failures related to the runtime-tests they maintains.
This patch relys on the gitlab-ci infrasture introduced earlier on in this series as well as the patch adding support for runtime-tests in the getdeveloperlib already applied to master.
--- utils/daily-mail | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
utils/daily-mail +20 -0
@@ -78,6 +78,7 @@ class Notification: self.arch_notifications = defaultdict(list) self.package_notifications = defaultdict(list) self.defconfig_notifications = defaultdict(list) + self.runtime_test_notifications = defaultdict(list) self.package_version_notification = []
@@ -184,6 +185,23 @@ def add_defconfig_notification(branch, notifications, job): notif.defconfig_notifications[branch].append(job) +def add_runtime_test_notification(branch, notifications, job): + ''' + Add to the notifications{} dict notifications that are related to + runtime-tests "maintainers". + ''' + orphan = True + for dev in developers: + if job['full_name'] in dev.runtime_tests: + orphan = False + notif = notifications.setdefault(dev, Notification()) + notif.runtime_test_notifications[branch].append(job) + if orphan: + job['orphan'] = True + notif = notifications.setdefault(get_orphan_developer(), Notification()) + notif.runtime_test_notifications[branch].append(job) + + def add_outdated_pkg_notification(notifications, package): ''' Add to the notifications{} dict notifications that are related to
@@ -672,6 +690,8 @@ def calculate_notifications(results, outdated_pkgs, gitlab_results): for job in pipeline['jobs']: if job['kind'] == 'defconfig': add_defconfig_notification(branch, notifications, job) + elif job['kind'] == 'runtime-test': + add_runtime_test_notification(branch, notifications, job) return notifications
Signature · 1 line
2.21.0
PATCH 6/10 utils/daily-mail: prepare the mail formatting to support gitlab-ci results
In reply to Victor HuescaThis patch allows to add gitlab-ci related results to mails.
Since every gitlab-ci result share the same information, a single function was preferred to factorize this part for now (it could be split once different gitlab-ci results have specific formatting).
--- utils/daily-mail | 68 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 54 insertions(+), 14 deletions(-)
utils/daily-mail +54 -14
@@ -246,6 +246,12 @@ def shrink_str(string, length, align='right', fill='...'): raise ValueError('align must be either `left`, `right` or `center`') +def fmt_duration(**timedelta_args): + '''Small helper to format a duration''' + t = datetime(1990, 1, 1) + timedelta(**timedelta_args) + return t.strftime('%Hh%Mm%Ss').lstrip('0:hms') + + def show_results(results, show_status, show_orphan=False): contents = "" # Header
@@ -314,6 +320,32 @@ def show_outdated(packages, show_orphan=False): return contents +def show_gitlab_ci_jobs(jobs, job_column_description='test-name', job_column_length=40, show_orphan=False): + # Header + contents = '' + if show_orphan: + contents += '{:>{}} | {:^61} | {:>8} | {:^5}\n'.format(job_column_description, job_column_length, + 'link to the job', 'duration', 'orph?') + contents += '{0:->{1}}-+-{0:-^61}-+-{0:->8} | {0:-^5}\n'.format('', job_column_length) + else: + contents += '{:>{}} | {:^61} | {:>8}\n'.format(job_column_description, job_column_length, + 'link to the job', 'duration') + contents += '{0:->{1}}-+-{0:-^61}-+-{0:->8}\n'.format('', job_column_length) + + # Data + for job in sorted(jobs, key=lambda j: (j['kind'], j['name'])): + name = shrink_str(job['name'], job_column_length, align='left') + duration = fmt_duration(seconds=job['duration']) + orphan = 'ORPH' if job.get('orphan') else '' + + contents += '{:>{}} | {:^61} | {:>8}'.format(name, job_column_length, job['web_url'], duration) + if show_orphan: + contents += ' | {:^5}'.format(orphan) + contents += '\n' + + return contents + + def developers_email(smtp, branches, notifications, datestr, dry_run): ''' Send the e-mails to the individual developers
@@ -387,10 +419,8 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): print "To: {}".format(dev.name) -def global_email_branch_result(results, results_by_reason, branch): - contents = "Results for branch '{}'\n".format(branch) - contents += "=====================" + "=" * len(branch) + "\n\n" - contents += "Classification of failures by reason\n" +def global_email_branch_result(results, results_by_reason): + contents = "Classification of failures by reason\n" contents += "------------------------------------\n\n" for r in results_by_reason: reason = shrink_str(r['reason'], 30)
@@ -400,11 +430,14 @@ def global_email_branch_result(results, results_by_reason, branch): contents += "Detail of failures\n" contents += "------------------\n\n" contents += show_results(results, show_status=True, show_orphan=True) - contents += "\n" return contents -def global_email(smtp, results, results_by_reason, datestr, overall, outdated_pkgs, dry_run): +def global_email_branch_gitlab_ci(pipelines): + pass # Implement this to support gitlab-ci related results + + +def global_email(smtp, branches, results, results_by_reason, datestr, overall, outdated_pkgs, gitlab_ci, dry_run): ''' Send the global e-mail to the mailing list '''
@@ -421,14 +454,21 @@ def global_email(smtp, results, results_by_reason, datestr, overall, outdated_pk if stats[3] == 0: continue contents += " {:^10} | {:^3} | {:^3} | {:^3} | {:^3} |\n".format(branch, stats[0], stats[1], stats[2], stats[3]) - if len(overall) != 0: + if any(map(len, (overall, gitlab_ci))): contents += "\n" - for branch in results.keys(): - if len(results[branch]) == 0: + for branch in branches: + res = results.get(branch, []) + git = gitlab_ci.get(branch, []) + if not any(map(len, (res, git))): continue - contents += global_email_branch_result(results[branch], results_by_reason[branch], branch) - if len(overall) != 0: - contents += "\n" + contents += "Results for branch '{}'\n".format(branch) + contents += "=====================" + "=" * len(branch) + "\n\n" + if len(res) != 0: + contents += global_email_branch_result(res, results_by_reason[branch]) + contents += "\n\n" + if len(git) != 0: + contents += global_email_branch_gitlab_ci(git) + contents += "\n\n" if outdated_pkgs: contents += "Packages having a newer version\n" contents += "===============================\n\n"
@@ -742,8 +782,8 @@ def __main__(): if 'dev' in email_dest: developers_email(smtp, branches, notifications, date_str, args.dry_run) if 'global' in email_dest: - global_email(smtp, results, results_by_reason, date_str, - overall_stats, outdated_pkgs, args.dry_run) + global_email(smtp, branches, results, results_by_reason, date_str, + overall_stats, outdated_pkgs, {}, args.dry_run) smtp.quit()
Signature · 1 line
2.21.0
PATCH 7/10 utils/daily-mail: add formatting for defconfig results
In reply to Victor HuescaThis patch uses the gitlab-ci results generic function to format defconfigs in mails.
--- utils/daily-mail | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-)
utils/daily-mail +24 -9
@@ -355,14 +355,14 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): email_from = localconfig.fromaddr subject = "[{}] Your daily results for {}".format(baseurl, datestr) contents = "Hello,\n\n" - - if notif.arch_notifications or notif.package_notifications: - contents += "Autobuilder failures\n" - contents += "====================\n\n" + if notif.arch_notifications or notif.package_notifications or notif.defconfig_notifications: + contents += "Recent build failures\n" + contents += "=====================\n\n" contents += textwrap.fill("This is the list of Buildroot build failures that occurred on {}, " - "and for which you are a registered architecture developer or package " - "developer. Please help us improving the quality of Buildroot by " - "investigating those build failures and sending patches to fix them. " + "and for which you are a registered architecture developer, package " + "developer or defconfig developer. Please help us improving the " + "quality of Buildroot by investigating those build failures and " + "sending patches to fix them. " "Thanks!".format(datestr)) contents += "\n\n" show_orphan = dev.name == ORPHAN_DEVELOPER
@@ -370,8 +370,9 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): for branch in branches: archs = notif.arch_notifications.get(branch, []) packages = notif.package_notifications.get(branch, []) + defconfigs = notif.defconfig_notifications.get(branch, []) - if len(archs) == 0 and len(packages) == 0: + if not any(map(len, (archs, packages, defconfigs))): continue contents += "Results for the '{}' branch\n".format(branch)
@@ -384,6 +385,11 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): if len(packages) != 0: contents += "Build failures related to your packages:\n\n" contents += show_results(packages, show_status=False, show_orphan=show_orphan) + contents += "\n\n" + if len(defconfigs) != 0: + contents += "Build failures related to your defconfigs:\n\n" + contents += show_gitlab_ci_jobs(defconfigs, job_column_description='defconfig', + job_column_length=33, show_orphan=show_orphan) contents += "\n"
@@ -434,7 +440,16 @@ def global_email_branch_result(results, results_by_reason): def global_email_branch_gitlab_ci(pipelines): - pass # Implement this to support gitlab-ci related results + defconfigs = [j for p in pipelines for j in p['jobs'] if j['kind'] == 'defconfig'] + + contents = "" + if len(defconfigs) != 0: + contents += 'Detail of defconfig failures\n' + contents += '----------------------------\n\n' + contents += show_gitlab_ci_jobs(defconfigs, job_column_description='defconfig', + job_column_length=33, show_orphan=True) + contents += "\n\n" + return contents def global_email(smtp, branches, results, results_by_reason, datestr, overall, outdated_pkgs, gitlab_ci, dry_run):
Signature · 1 line
2.21.0
PATCH 8/10 utils/daily-mail: add formatting for runtime-test results
In reply to Victor HuescaThis patch uses the gitlab-ci results generic function to format runtime-test in mails.
--- utils/daily-mail | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-)
utils/daily-mail +25 -7
@@ -355,14 +355,19 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): email_from = localconfig.fromaddr subject = "[{}] Your daily results for {}".format(baseurl, datestr) contents = "Hello,\n\n" - if notif.arch_notifications or notif.package_notifications or notif.defconfig_notifications: - contents += "Recent build failures\n" - contents += "=====================\n\n" + + archs = notif.arch_notifications + packages = notif.package_notifications + defconfigs = notif.defconfig_notifications + runtime_tests = notif.runtime_test_notifications + if archs or packages or defconfigs or runtime_tests: + contents += "Recent build failures and runtime-tests failures\n" + contents += "================================================\n\n" contents += textwrap.fill("This is the list of Buildroot build failures that occurred on {}, " "and for which you are a registered architecture developer, package " - "developer or defconfig developer. Please help us improving the " - "quality of Buildroot by investigating those build failures and " - "sending patches to fix them. " + "developer or defconfig developer. This list also include runtime " + "tests failures. Please help us improving the quality of Buildroot by " + "investigating those build failures and sending patches to fix them. " "Thanks!".format(datestr)) contents += "\n\n" show_orphan = dev.name == ORPHAN_DEVELOPER
@@ -371,8 +376,9 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): archs = notif.arch_notifications.get(branch, []) packages = notif.package_notifications.get(branch, []) defconfigs = notif.defconfig_notifications.get(branch, []) + runtimes = notif.runtime_test_notifications.get(branch, []) - if not any(map(len, (archs, packages, defconfigs))): + if not any(map(len, (archs, packages, defconfigs, runtimes))): continue contents += "Results for the '{}' branch\n".format(branch)
@@ -390,6 +396,11 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): contents += "Build failures related to your defconfigs:\n\n" contents += show_gitlab_ci_jobs(defconfigs, job_column_description='defconfig', job_column_length=33, show_orphan=show_orphan) + contents += "\n\n" + if len(runtimes) != 0: + contents += "Build failures related to your runtime tests:\n\n" + contents += show_gitlab_ci_jobs(runtimes, job_column_description='runtime-test', + job_column_length=25, show_orphan=show_orphan) contents += "\n"
@@ -441,6 +452,7 @@ def global_email_branch_result(results, results_by_reason): def global_email_branch_gitlab_ci(pipelines): defconfigs = [j for p in pipelines for j in p['jobs'] if j['kind'] == 'defconfig'] + runtime_tests = [j for p in pipelines for j in p['jobs'] if j['kind'] == 'runtime-test'] contents = "" if len(defconfigs) != 0:
@@ -449,6 +461,12 @@ def global_email_branch_gitlab_ci(pipelines): contents += show_gitlab_ci_jobs(defconfigs, job_column_description='defconfig', job_column_length=33, show_orphan=True) contents += "\n\n" + if len(runtime_tests) != 0: + contents += "Detail of runtime-test failures\n" + contents += "-------------------------------\n\n" + contents += show_gitlab_ci_jobs(runtime_tests, job_column_description='runtime-test', + job_column_length=25, show_orphan=True) + contents += "\n\n" return contents
Signature · 1 line
2.21.0
PATCH 9/10 utils/daily-mail: add argument for gitlab-ci related data
In reply to Victor HuescaThis patch provides two new data to the args parser to allow enabling defconfigs, runtime-tests and sanity checks in global/developer emails. These tree data are grouped together in a gitlab-ci data group as they are fetched at the same time.
This new list will be use by the next patch to enable gitlab-ci jobs in email.
--- utils/daily-mail | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
utils/daily-mail +7 -6
@@ -776,7 +776,8 @@ def parse_args(): parser.add_argument('--branches', action='store', nargs='+', help='List of branches (blank separated)') parser.add_argument('--dest', action='store', nargs='+', choices=['dev', 'global'], help='List of emails type to send (ie. global, dev)') - parser.add_argument('--data', action='store', nargs='+', choices=['autobuild', 'outdated'], + parser.add_argument('--data', action='store', nargs='+', + choices=['autobuild', 'outdated', 'defconfig', 'runtime-test'], help='List of information to add in emails (blank separated)') args = parser.parse_args() if args.date and not RE_DATE.match(args.date):
@@ -787,13 +788,13 @@ def parse_args(): def __main__(): args = parse_args() if args.date: - date_str = args.date + date_struct = datetime.strptime(args.date, '%Y-%m-%d').date() else: - yesterday = date.today() - timedelta(1) - date_str = yesterday.strftime('%Y-%m-%d') - branches = args.branches if args.branches else get_branches() + date_struct = date.today() - timedelta(1) + date_str = date_struct.isoformat() + branches = args.branches or get_branches() email_dest = set(args.dest) if args.dest else {'dev', 'global'} - email_data = set(args.data) if args.data else {'autobuild', 'outdated'} + email_data = set(args.data) if args.data else {'autobuild', 'outdated', 'defconfig', 'runtime-test'} db = _mysql.connect(host=localconfig.host, user=localconfig.user,
Signature · 1 line
2.21.0
PATCH 10/10 utils/daily-mail: enable gitlab-ci emails
In reply to Victor HuescaThis patch enable all gitlab-ci related results added by previous patches from the series in both the global and per-developer emails.
Defconfigs and runtime-tests entries are added in the per-branch results for both global and dev-mails.
--- utils/daily-mail | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
utils/daily-mail +6 -2
@@ -809,7 +809,11 @@ def __main__(): results = {} results_by_reason = {} outdated_pkgs = get_outdated_pkgs(localconfig.pkg_stats) if 'outdated' in email_data else [] - notifications = calculate_notifications(results, outdated_pkgs, {}) + if email_data & {'defconfig', 'runtime-test'}: + gitlab_ci = get_gitlab_ci(branches, date_struct, kinds=email_data) + else: + gitlab_ci = {} + notifications = calculate_notifications(results, outdated_pkgs, gitlab_ci) smtp = smtplib.SMTP(localconfig.smtphost, localconfig.smtpport) smtp.starttls() smtp.login(localconfig.smtpuser, localconfig.smtppass)
@@ -817,7 +821,7 @@ def __main__(): developers_email(smtp, branches, notifications, date_str, args.dry_run) if 'global' in email_dest: global_email(smtp, branches, results, results_by_reason, date_str, - overall_stats, outdated_pkgs, {}, args.dry_run) + overall_stats, outdated_pkgs, gitlab_ci, args.dry_run) smtp.quit()
Signature · 1 line
2.21.0
cover add gitlab-ci jobs to the daily-mail
In reply to Victor HuescaHello Victor,
Quoted 5 lines · On Thu, 29 Aug 2019 15:23:38 +0200 Victor Huesca <[email protected]>…
Victor Huesca (10):
utils/daily-mail: new data: gitlab-ci jobs utils/daily-mail: new gitlab-ci data: defconfigs utils/daily-mail: new gitlab-ci data: runtime tests utils/daily-mail: add defconfig support in developers notifications
Quoted 8 lines
utils/daily-mail: add runtime tests support in developers
notifications
utils/daily-mail: prepare the mail formatting to support gitlab-ci
results
utils/daily-mail: add formatting for defconfig results utils/daily-mail: add formatting for runtime-test results utils/daily-mail: add argument for gitlab-ci related data utils/daily-mail: enable gitlab-ci emails
Thanks for this patch series! I have made a number of changes, and applied it.
The main changes are:
- More filtering of pipelines: only failed pipelines are retrieved
- Earlier filtering of pipelines: we can filter pipelines by branch and status before getting the details for each pipeline
- Removal of the job duration in the e-mails, it is not a very useful information IMO
- Further splitting of some of the patches
So, hopefully this Saturday we should see e-mails about defconfig failures, since we have at least one still failing on master.
Thanks a lot for this work!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com