improve modularity by adding an
cover improve modularity by adding an
This patch series replace the manual argument parsing by a python argument parser. The goal is to improve the modularity to help the integration of new information to the mails.
Victor Huesca (5):
utils/daily-mail: introduce argparse to ease modularity of the script utils/daily-mail: allow to pass a date in arguments utils/daily-mail: allow to specify branches from command-line utils/daily-mail: allow to select only a subset of emails to send utils/daily-mail: allow to select only a subset of data to send in
emails
utils/daily-mail | 64 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 47 insertions(+), 17 deletions(-)
Signature · 1 line
2.21.0
PATCH 1/5 utils/daily-mail: introduce argparse to ease modularity of the script
Having an actual argument parser instead of manually checking for command-line arguments will allow future patches to add divers options to the script. This patch only introduce the parser with the 'dry-run' argument.
--- utils/daily-mail | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
utils/daily-mail +10 -5
@@ -12,6 +12,7 @@ import localconfig import csv from collections import defaultdict import math +import argparse sys.path.append(os.path.join(localconfig.brbase, "utils")) import getdeveloperlib # noqa: E402
@@ -379,7 +380,14 @@ def calculate_notifications(results): return notifications +def parse_args(): + parser = argparse.ArgumentParser() + parser.add_argument('--dry-run', action='store_true', help='Do not send email, only show output') + return parser.parse_args() + + def __main__(): + args = parse_args() yesterday = date.today() - timedelta(1) yesterday_str = yesterday.strftime('%Y-%m-%d') branches = get_branches()
@@ -392,15 +400,12 @@ def __main__(): results = get_build_results(db, yesterday_str, branches) results_by_reason = get_build_results_grouped_by_reason(db, yesterday_str, branches) notifications = calculate_notifications(results) - dry_run = False - if len(sys.argv) == 2 and sys.argv[1] == "--dry-run": - dry_run = True smtp = smtplib.SMTP(localconfig.smtphost, localconfig.smtpport) smtp.starttls() smtp.login(localconfig.smtpuser, localconfig.smtppass) - developers_email(smtp, branches, notifications, yesterday_str, dry_run) + developers_email(smtp, branches, notifications, yesterday_str, args.dry_run) global_email(smtp, results, results_by_reason, yesterday_str, - overall_stats, dry_run) + overall_stats, args.dry_run) smtp.quit()
Signature · 1 line
2.21.0
PATCH 2/5 utils/daily-mail: allow to pass a date in arguments
In reply to Victor HuescaThis allows to use a custorm date instead of the hardcoded yesterday.
--- utils/daily-mail | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
utils/daily-mail +13 -3
@@ -13,10 +13,13 @@ import csv from collections import defaultdict import math import argparse +import re sys.path.append(os.path.join(localconfig.brbase, "utils")) import getdeveloperlib # noqa: E402 +RE_DATE = re.compile(r'^\d\d\d\d-\d\d-\d\d$') + baseurl = "autobuild.buildroot.net" http_baseurl = "http://" + baseurl
@@ -383,13 +386,20 @@ def calculate_notifications(results): def parse_args(): parser = argparse.ArgumentParser() parser.add_argument('--dry-run', action='store_true', help='Do not send email, only show output') - return parser.parse_args() + parser.add_argument('--date', action='store', help='Use this date instead of yesterday') + args = parser.parse_args() + if args.date and not RE_DATE.match(args.date): + parser.error('date must be in format YYYY-MM-DD') + return args def __main__(): args = parse_args() - yesterday = date.today() - timedelta(1) - yesterday_str = yesterday.strftime('%Y-%m-%d') + if args.date: + yesterday_str = args.date + else: + yesterday = date.today() - timedelta(1) + yesterday_str = yesterday.strftime('%Y-%m-%d') branches = get_branches() db = _mysql.connect(host=localconfig.host,
Signature · 1 line
2.21.0
PATCH 3/5 utils/daily-mail: allow to specify branches from command-line
In reply to Victor HuescaThis patch allows to specify branches to include in emails instead of reading the 'branches' file. If no branch is provided, the default 'branches' file is used.
--- utils/daily-mail | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
utils/daily-mail +2 -1
@@ -387,6 +387,7 @@ def parse_args(): parser = argparse.ArgumentParser() parser.add_argument('--dry-run', action='store_true', help='Do not send email, only show output') parser.add_argument('--date', action='store', help='Use this date instead of yesterday') + parser.add_argument('--branches', action='store', nargs='+', help='List of branches (blank separated)') args = parser.parse_args() if args.date and not RE_DATE.match(args.date): parser.error('date must be in format YYYY-MM-DD')
@@ -400,7 +401,7 @@ def __main__(): else: yesterday = date.today() - timedelta(1) yesterday_str = yesterday.strftime('%Y-%m-%d') - branches = get_branches() + branches = args.branches if args.branches else get_branches() db = _mysql.connect(host=localconfig.host, user=localconfig.user,
Signature · 1 line
2.21.0
PATCH 4/5 utils/daily-mail: allow to select only a subset of emails to send
In reply to Victor HuescaThis patch allows to send the global email and the per-developer email separately by specifying the destination as command-line argument. The default behavior -- when no destination is provided -- is to select both the global and developers emails.
--- utils/daily-mail | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
utils/daily-mail +8 -3
@@ -388,6 +388,8 @@ def parse_args(): parser.add_argument('--dry-run', action='store_true', help='Do not send email, only show output') parser.add_argument('--date', action='store', help='Use this date instead of yesterday') parser.add_argument('--branches', action='store', nargs='+', help='List of branches (blank separated)') + parser.add_argument('--dest', action='store', nargs='+', choices=['dev', 'global'], + help='List of emails type to send (ie. global, dev)') args = parser.parse_args() if args.date and not RE_DATE.match(args.date): parser.error('date must be in format YYYY-MM-DD')
@@ -402,6 +404,7 @@ def __main__(): yesterday = date.today() - timedelta(1) yesterday_str = yesterday.strftime('%Y-%m-%d') branches = args.branches if args.branches else get_branches() + email_dest = set(args.dest) if args.dest else {'dev', 'global'} db = _mysql.connect(host=localconfig.host, user=localconfig.user,
@@ -414,9 +417,11 @@ def __main__(): smtp = smtplib.SMTP(localconfig.smtphost, localconfig.smtpport) smtp.starttls() smtp.login(localconfig.smtpuser, localconfig.smtppass) - developers_email(smtp, branches, notifications, yesterday_str, args.dry_run) - global_email(smtp, results, results_by_reason, yesterday_str, - overall_stats, args.dry_run) + if 'dev' in email_dest: + developers_email(smtp, branches, notifications, yesterday_str, args.dry_run) + if 'global' in email_dest: + global_email(smtp, results, results_by_reason, yesterday_str, + overall_stats, args.dry_run) smtp.quit()
Signature · 1 line
2.21.0
PATCH 5/5 utils/daily-mail: allow to select only a subset of data to send in emails
In reply to Victor HuescaThis patch adds a modular way to select a subset of the available information collected by this script. Currently the range of possibilities is limited to enable or disable the autobuild results, but it allow future patches to easily add new kind of data to this script.
--- utils/daily-mail | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-)
utils/daily-mail +17 -8
@@ -231,7 +231,7 @@ def developers_email(smtp, branches, notifications, datestr, dry_run): for dev, notif in notifications.iteritems(): to = dev.name email_from = localconfig.fromaddr - subject = "[{}] Your build results for {}".format(baseurl, datestr) + subject = "[{}] Your daily results for {}".format(baseurl, datestr) contents = "Hello,\n\n" contents += textwrap.fill("This is the list of Buildroot build failures that occurred on {}, " "and for which you are a registered architecture developer or package "
@@ -304,11 +304,12 @@ def global_email(smtp, results, results_by_reason, datestr, overall, dry_run): ''' to = "buildroot at buildroot.org" email_from = localconfig.fromaddr - subject = "[{}] Build results for {}".format(baseurl, datestr) + subject = "[{}] Daily results for {}".format(baseurl, datestr) contents = "Hello,\n\n" - contents += "Build statistics for {}\n".format(datestr) - contents += "===============================\n\n" - contents += " branch | OK | NOK | TIM | TOT |\n" + if len(overall) != 0: + contents += "Autobuild statistics for {}\n".format(datestr) + contents += "===============================\n\n" + contents += " branch | OK | NOK | TIM | TOT |\n" for branch in sorted(overall.iterkeys()): stats = overall[branch] if stats[3] == 0:
@@ -390,6 +391,8 @@ def parse_args(): parser.add_argument('--branches', action='store', nargs='+', help='List of branches (blank separated)') parser.add_argument('--dest', action='store', nargs='+', choices=['dev', 'global'], help='List of emails type to send (ie. global, dev)') + parser.add_argument('--data', action='store', nargs='+', choices=['autobuild'], + help='List of information to add in emails (blank separated)') args = parser.parse_args() if args.date and not RE_DATE.match(args.date): parser.error('date must be in format YYYY-MM-DD')
@@ -405,14 +408,20 @@ def __main__(): yesterday_str = yesterday.strftime('%Y-%m-%d') branches = args.branches if args.branches else get_branches() email_dest = set(args.dest) if args.dest else {'dev', 'global'} + email_data = set(args.data) if args.data else {'autobuild'} db = _mysql.connect(host=localconfig.host, user=localconfig.user, passwd=localconfig.passwd, db=localconfig.db) - overall_stats = get_overall_stats(db, yesterday_str, branches) - results = get_build_results(db, yesterday_str, branches) - results_by_reason = get_build_results_grouped_by_reason(db, yesterday_str, branches) + if 'autobuild' in email_data: + overall_stats = get_overall_stats(db, yesterday_str, branches) + results = get_build_results(db, yesterday_str, branches) + results_by_reason = get_build_results_grouped_by_reason(db, yesterday_str, branches) + else: + overall_stats = {} + results = {} + results_by_reason = {} notifications = calculate_notifications(results) smtp = smtplib.SMTP(localconfig.smtphost, localconfig.smtpport) smtp.starttls()
Signature · 1 line
2.21.0
Thomas Petazzoni
Appliedcover improve modularity by adding an
In reply to Victor HuescaQuoted 7 lines · On Sun, 4 Aug 2019 17:25:31 +0200 Victor Huesca <[email protected]>…
Victor Huesca (5):
utils/daily-mail: introduce argparse to ease modularity of the script utils/daily-mail: allow to pass a date in arguments utils/daily-mail: allow to specify branches from command-line utils/daily-mail: allow to select only a subset of emails to send utils/daily-mail: allow to select only a subset of data to send in
emails
Applied, thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com