Fix buildlogger
cover Fix buildlogger
Since build 999 we haven't seen any builds logged to the BuildLogs wiki page, that turns out to be because the JSON dicts keys are strings and thus the sort order isn't as expected, i.e. 999 > 1001.
This series tidies up the buildlogger some and includes a fix for this issue by converting the keys of the dicts to a list of ints and sorting that to determine the most recent build.
The following changes since commit ba01fe19440a91091fe15544ecc2809a79f19401:
nightly-qa-systemd.conf: Add create_eventlog property (2016-08-31 21:22:11 +0100)
are available in the git repository at:
git://git.yoctoproject.org/yocto-autobuilder joshuagl/buildlogger
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/log/?h=joshuagl/buildlogger
Joshua Lock (3):
buildlogger: remove old, incomplete, comment
buildlogger: don't hardcode YP AB in generated URL
buildlogger: fix sorting of recent builds
bin/buildlogger | 23 +++++++++++++---------- etc/buildlogger.conf.example | 1 + 2 files changed, 14 insertions(+), 10 deletions(-)
Signature · 1 line
2.7.4
PATCH 1/3 buildlogger: remove old, incomplete, comment
The comment isn't complete and doesn't make much sense.
--- bin/buildlogger | 6 ------ 1 file changed, 6 deletions(-)
bin/buildlogger +0 -6
@@ -11,12 +11,6 @@ __maintainer__ = "Joshua Lock" __email__ = "[email protected]" ''' -# We'd probably benefit from using some caching, but first we'd need the AB API -# to include -# -# We can set repo url, branch & commit for a bunch of repositorys. -# Do they all get built for nightly? - import codecs try: import configparser
Signature · 1 line
2.7.4
PATCH 2/3 buildlogger: don't hardcode YP AB in generated URL
In reply to Joshua LockWhen generating text for a BuildLog wiki entry we were hard-coding the Yocto Project AB URL in the entry's title, rather than that add a new variable to buildlogger.conf and use the value of this variable to construct the URL in the link.
--- bin/buildlogger | 7 +++++-- etc/buildlogger.conf.example | 1 + 2 files changed, 6 insertions(+), 2 deletions(-)
bin/buildlogger +5 -2
@@ -30,6 +30,7 @@ except ImportError: sys.exit(1) abapi = '' +abwww = '' # Wiki editing params un = '' pw = ''
@@ -49,6 +50,7 @@ def load_config(configfile): global title global wikiapi global abapi + global abwww success = False if os.path.exists(configfile):
@@ -59,6 +61,7 @@ def load_config(configfile): pw = config.get('wiki', 'password') title = config.get('wiki', 'pagetitle') wikiapi = config.get('wiki', 'apiuri') + abwww = config.get('builder', 'wwwuri') abapi = config.get('builder', 'apiuri') success = True except configparser.Error as ex:
@@ -195,8 +198,8 @@ def ab_last_build_to_entry(build_json, build_id): if prop[0] == 'commit_poky': chash = prop[1] - urlfmt = 'https://autobuilder.yoctoproject.org/main/builders/%s/builds/%s/' - url = urlfmt % (builder, buildid) + urlfmt = '%s/builders/%s/builds/%s/' + url = urlfmt % (abwww, builder, buildid) sectionfmt = '==[%s %s %s - %s %s]==' section_title = sectionfmt % (url, builder, buildid, buildbranch, chash) summaryfmt = 'Adding new BuildLog entry for build %s (%s)'
etc/buildlogger.conf.example +1 -0
@@ -5,4 +5,5 @@ pagetitle = BuildLogPageTitle apiuri = https://wiki.yoctoproject.org/wiki/api.php [builder] +wwwuri = https://autobuilder.yoctoproject.org/main apiuri = https://autobuilder.yoctoproject.org/main/json/builders/nightly/builds/_all
Signature · 1 line
2.7.4
PATCH 3/3 buildlogger: fix sorting of recent builds
In reply to Joshua LockRecent builds, >999, on the YP AutoBuilder were not being logged. This is because we sort the builds based on a string representation of their build number, yet string 1001 < string 999.
Resolve this by converting the build id's to an int, sorting them and then determining which is the highest/most recent build id.
--- bin/buildlogger | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
bin/buildlogger +8 -2
@@ -219,7 +219,7 @@ def write_last_build(buildid): # Read last logged buildid from a file def read_last_build(): - last_build = '' + last_build = '0' try: with open(cachefile, 'r') as fi: last_build = fi.readline()
@@ -254,7 +254,13 @@ def watch_for_builds(configfile): print("Failed to decode JSON: %s" % str(e)) continue - last_build = sorted(build_json.keys())[-1] + # The keys of the JSON dict are unicode strings, which makes + # sorting to find the most recent difficult (i.e. for a string type + # 999 > 1001). Convert them to ints for a simpler sort. + buildids = map(int, build_json.keys()) + buildids.sort() + last_build = str(buildids[-1]) + # If a new build is detected, post a new entry to the BuildLog if last_build != last_logged: new_entry, summary = ab_last_build_to_entry(build_json, last_build)
Signature · 1 line
2.7.4
Beth 'pidge' Flanagan
Appliedmerged into master. Thanks!
-b
Quoted 33 lines · Joshua Lock
Since build 999 we haven't seen any builds logged to the BuildLogs wiki page,
that turns out to be because the JSON dicts keys are strings and thus the sort
order isn't as expected, i.e. 999 > 1001.This series tidies up the buildlogger some and includes a fix for this issue by
converting the keys of the dicts to a list of ints and sorting that to determine
the most recent build.The following changes since commit
ba01fe19440a91091fe15544ecc2809a79f19401:nightly-qa-systemd.conf: Add
create_eventlogproperty (2016-08-31 21:22:11 +0100)are available in the git repository at:
git://git.yoctoproject.org/yocto-autobuilderjoshuagl/buildlogger
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/log/?h=joshu
agl/buildloggerJoshua Lock (3):
buildlogger: remove old, incomplete, comment
buildlogger: don't hardcode YP AB in generated URL
buildlogger: fix sorting of recent buildsbin/buildlogger | 23 +++++++++++++---------- etc/buildlogger.conf.example | 1 +
2 files changed, 14 insertions(+), 10 deletions(-)