autobuild-run: create reason file on build failures
PATCH v2 1/3 autobuild-run: create reason file on build failures
When a build fails, we calculate the reason of failure on the server side as well as client side. To solve this redundancy we only calculate on the client side and submit a file which contains the reason of failure.
---
NOTE: This series depends on work from builder-class series
Changes v1 -> v2:
- Split commit into two, one for adding reason file and one for accounting for reproducibility failures (next patch).
--- scripts/autobuild-run | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
scripts/autobuild-run +5 -2
@@ -596,6 +596,11 @@ class Builder: # not found return None + reason = get_failure_reason() + if reason: + with open(os.path.join(self.resultdir, "reason"), "w+") as reasonf: + reasonf.write("-".join(reason)) + def extract_end_log(resultfile): """Save the last part of the build log, starting from the failed package"""
@@ -604,7 +609,6 @@ class Builder: (os.path.join(self.outputdir, "logfile"), resultfile)], shell=True) - reason = get_failure_reason() if not reason: extract_last_500_lines() else:
@@ -628,7 +632,6 @@ class Builder: def copy_config_log_files(): """Recursively copy any config.log files from the failing package""" - reason = get_failure_reason() if not reason: return
Signature · 1 line
2.20.1
PATCH v2 2/3 autobuild-run: account for reproducibility failures when creating the reason file
---
Changes v1 -> v2:
- Split into separate commit for reproducibility failures
--- scripts/autobuild-run | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-)
scripts/autobuild-run +23 -10
@@ -582,16 +582,26 @@ class Builder: return def get_failure_reason(): - # Output is a tuple (package, version), or None. - lastlines = decode_bytes(subprocess.Popen( - ["tail", "-n", "3", os.path.join(self.outputdir, "logfile")], - stdout=subprocess.PIPE).communicate()[0]).splitlines() + # Output is a tuple (package, version), or None in case of package failure + # Output is "reproducible" in case of reproducibility failure + + reproducible_results = os.path.join(self.resultdir, "reproducible_results") + if os.path.exists(reproducible_results): + if os.stat(reproducible_results).st_size > 0: + reason = "reproducible" + return reason + else: + return None + else: + lastlines = decode_bytes(subprocess.Popen( + ["tail", "-n", "3", os.path.join(self.outputdir, "logfile")], + stdout=subprocess.PIPE).communicate()[0]).splitlines() - regexp = re.compile(r'make: \*\*\* .*/(?:build|toolchain)/([^/]*)/') - for line in lastlines: - m = regexp.search(line) - if m: - return m.group(1).rsplit('-', 1) + regexp = re.compile(r'make: \*\*\* .*/(?:build|toolchain)/([^/]*)/') + for line in lastlines: + m = regexp.search(line) + if m: + return m.group(1).rsplit('-', 1) # not found return None
@@ -599,7 +609,10 @@ class Builder: reason = get_failure_reason() if reason: with open(os.path.join(self.resultdir, "reason"), "w+") as reasonf: - reasonf.write("-".join(reason)) + if reason == "reproducible": + reasonf.write(reason) + else: + reasonf.write("-".join(reason)) def extract_end_log(resultfile): """Save the last part of the build log, starting from the failed package"""
Signature · 1 line
2.20.1
PATCH v2 3/3 web/import.inc.php: support reading failure reason from reason file
In reply to Atharva Lele---
NOTE: This patch is untested.
--- web/import.inc.php | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)
web/import.inc.php +15 -11
@@ -230,17 +230,21 @@ function import_result($buildid, $filename) if ($status == 0) $reason = "none"; else { - $tmp = Array(); - exec("tail -3 " . $thisbuildfinaldir . "build-end.log | grep -v '\[_all\]' | grep 'make.*: \*\*\*' | sed 's,.*\[\([^\]*\)\] Error.*,\\1,' | sed 's,.*/build/\([^/]*\)/.*,\\1,'", $tmp); - if (trim($tmp[0])) - $reason = $tmp[0]; - else { - exec("tail -1 " . $thisbuildfinaldir . "build-time.log | grep :start: | cut -d':' -f4", $tmp); - if (trim($tmp[0])) - $reason = trim($tmp[0]); - else - $reason = "unknown"; - } + if (file_exists($thisbuildfinaldir . "reason")) + $reason = trim(file_get_contents($thisbuildfinaldir . "reason", "r")); + else { + $tmp = Array(); + exec("tail -3 " . $thisbuildfinaldir . "build-end.log | grep -v '\[_all\]' | grep 'make.*: \*\*\*' | sed 's,.*\[\([^\]*\)\] Error.*,\\1,' | sed 's,.*/build/\([^/]*\)/.*,\\1,'", $tmp); + if (trim($tmp[0])) + $reason = $tmp[0]; + else { + exec("tail -1 " . $thisbuildfinaldir . "build-time.log | grep :start: | cut -d':' -f4", $tmp); + if (trim($tmp[0])) + $reason = trim($tmp[0]); + else + $reason = "unknown"; + } + } } /* Compress files that are typically too large and infrequently
Signature · 1 line
2.20.1
PATCH v2 1/3 autobuild-run: create reason file on build failures
In reply to Atharva LeleHello Atharva,
Thanks for the second iteration. One question below.
scripts/autobuild-run class Builder: · line 602
Quoted 4 lines · On Sun, 23 Jun 2019 00:11:16 +0530 Atharva Lele <[email protected]> wrote:
scripts/autobuild-run +4
-0+ reason = get_failure_reason() + if reason: + with open(os.path.join(self.resultdir, "reason"), "w+") as reasonf: + reasonf.write("-".join(reason))
Why do we have this "-".join(reason) instead of just writing the contents of the "reason" variable to the file ?
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 1/3 autobuild-run: create reason file on build failures
Quoted 4 lines · Thomas Petazzoni
Why do we have this "-".join(reason) instead of just writing the contents of the "reason" variable to the file ?
The reason to do that is to achieve the same formatting for the failure reason
as the PHP script that is "package-version"
Regards,
Atharva Lele
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190622/2507a460/attachment.html>