Fix cooker log download button
v2 cover Fix cooker log download button
This supersedes https://lists.yoctoproject.org/pipermail/toaster/2015-September/002877.html
(which was a single commit).
* Retrieve a build's log file location directly from the file system, as the one which is available at the time of the BuildStarted event is incorrect.
* Hide the "Download build log" button for builds which fail early (e.g. due to recipe parsing) and consequently have no cooker_log_path.
Changes since 986f6b04b26bba9d01393f8f4ced4e0f4385368a (toaster-next) are ingit://git.yoctoproject.org/poky-contrib, elliot/toaster/wrong_log-8373
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/wrong_log-8373
Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373
Elliot Smith (2):
toaster: Manually retrieve log file location from filesystem
toaster: Hide "Download build log" button if log doesn't exist
bitbake/lib/bb/ui/toasterui.py | 31 +++++++++++++++++++++- .../toastergui/templates/builddashboard.html | 4 ++- 2 files changed, 33 insertions(+), 2 deletions(-)
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH v2 1/2 toaster: Manually retrieve log file location from filesystem
The log file location reported by the BB_CONSOLELOG variable does not point to the log location for the current build at the time when the BuildStarted event is fired. It actually points to the location where the next build will log to. This means that the log file paths associated with a build in the cooker_log_path field are incorrect, with the result that the "Download build log" button doesn't work.
Instead, when a build starts, get the latest-dated log file and associate it with the build.
An issue explaining why this is a problem, plus steps to demonstrate it:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8411
This patch is a temporary workaround for issue 8411, as discussed in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373#c2
It fixes 8373 for now, but should be properly fixed if bitbake can provide the correct log location at the point when BuildStarted is fired.
[YOCTO #8373]
--- bitbake/lib/bb/ui/toasterui.py | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/toasterui.py +30 -1
@@ -33,6 +33,7 @@ from bb.ui.buildinfohelper import BuildInfoHelper import bb.msg import logging import os +import glob # pylint: disable=invalid-name # module properties for UI modules are read by bitbake and the contract should not be broken
@@ -61,6 +62,33 @@ def _log_settings_from_server(server): raise BaseException(error) return includelogs, loglines, consolelogfile +# TODO this is a work-around for the bug mentioned in +# https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373#c2 +# - the log file location reported by BB_CONSOLELOG at the point +# when the BuildStarted event occurs does not correspond to the +# actual log file location; +# as a work-around, we get the latest-dated log file in the same +# directory as is in the path stored in BB_CONSOLELOG +# +# consolelogfile: log file location reported by BB_CONSOLELOG +# at the point when a build starts, which isn't the actual log location; +# the assumption is that the real log file will be in the same +# directory as the reported log file +# +# returns the latest-dated *.log file in the same directory +# as consolelogfile; if there are no *.log files, this returns +# consolelogfile +def _get_real_log_file(consolelogfile): + log_file_dir = os.path.dirname(consolelogfile) + log_file_pattern = os.path.join(log_file_dir, "*.log") + log_files = glob.glob(log_file_pattern) + log_files = filter(os.path.isfile, log_files) + log_files.sort(key=os.path.getmtime) + + if len(log_files) > 0: + return log_files[-1] + else: + return consolelogfile def main(server, eventHandler, params ): helper = uihelper.BBUIHelper()
@@ -126,7 +154,8 @@ def main(server, eventHandler, params ): # the code will look into the protected variables of the event; no easy way around this if isinstance(event, bb.event.BuildStarted): - buildinfohelper.store_started_build(event, consolelogfile) + real_consolelogfile = _get_real_log_file(consolelogfile) + buildinfohelper.store_started_build(event, real_consolelogfile) if isinstance(event, (bb.build.TaskStarted, bb.build.TaskSucceeded, bb.build.TaskFailedSilent)): buildinfohelper.update_and_store_task(event)
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH v2 2/2 toaster: Hide "Download build log" button if log doesn't exist
In reply to Elliot SmithOur builds pages show all builds, but also include build requests which may have resulted in a build failure, before the build started (e.g. at the recipe parsing stage).
In such cases, the BuildStarted event is not captured by Toaster, so we have no idea where the log file for the failed build is.
The result is that a build is shown by the Toaster UI /builds/ pages, but it is really a pretend build which never went beyond being a build request, and which has no associated log file. In turn, this breaks the "Download build log" button on the build dashboard, as there's no log file associated with the build.
Fix this by hiding the "Download build log" button for builds which don't have a cooker_log_path.
[YOCTO #8373]
--- bitbake/lib/toaster/toastergui/templates/builddashboard.html | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
bitbake/lib/toaster/toastergui/templates/builddashboard.html +3 -1
@@ -37,7 +37,9 @@ <span > <i class="icon-warning-sign yellow"></i><strong><a href="#warnings" class="warning show-warnings"> {{build.warnings.count}} warning{{build.warnings.count|pluralize}}</a></strong></span> {% endif %} <span class="pull-right">Build time: <a href="{% url 'buildtime' build.pk %}">{{ build.timespent_seconds|sectohms }}</a> - <a class="btn {%if build.outcome == build.SUCCEEDED%}btn-success{%else%}btn-danger{%endif%} pull-right log" href="{% url 'build_artifact' build.id "cookerlog" build.id %}">Download build log</a> + {% if build.cooker_log_path %} + <a class="btn {%if build.outcome == build.SUCCEEDED%}btn-success{%else%}btn-danger{%endif%} pull-right log" href="{% url 'build_artifact' build.id "cookerlog" build.id %}">Download build log</a> + {% endif %} </span> {%endif%}
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Quoted 9 lines · On 05/10/2015 14:39, "[email protected] on behalf of Elliot…
This supersedes
https://lists.yoctoproject.org/pipermail/toaster/2015-September/002877.htm
l
(which was a single commit).* Retrieve a build's log file location directly from the file system, as the one which is available at the time of the BuildStarted event is incorrect.
Quoted 8 lines
* Hide the "Download build log" button for builds which fail early (e.g. due to recipe parsing) and consequently have no
cooker_log_path.Changes since
986f6b04b26bba9d01393f8f4ced4e0f4385368a(toaster-next) are ingit://git.yoctoproject.org/poky-contrib, elliot/toaster/wrong_log-8373
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/wr
ong_log-8373
These builds not only have no log: they also have no build data. That's why they should look like this:
http://www.yoctoproject.org/toaster/build-dashboard-failed-build-request.ht
ml
We should remove not only the 'download log' button, but also the left navigation and the 'build summary section'. It's all explained in this document:
https://bugzilla.yoctoproject.org/attachment.cgi?id=2781
Cheers
Belén
Quoted 33 lines
Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373
Elliot Smith (2):
toaster: Manually retrieve log file location from filesystem
toaster: Hide "Download build log" button if log doesn't exist
bitbake/lib/bb/ui/toasterui.py| 31
+++++++++++++++++++++-
.../toastergui/templates/builddashboard.html | 4 ++-2 files changed, 33 insertions(+), 2 deletions(-)--
Elliot Smith
Software Engineer
Intel OTC---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Signature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Thanks for the feedback, Belen.
Quoted 31 lines · Belen
Quoted 2 lines · On 05/10/2015 14:39, "[email protected] on behalf of Elliot…
This supersedes
https://lists.yoctoproject.org/pipermail/toaster/2015-September/002877.htm
Quoted 7 lines
l
(which was a single commit).* Retrieve a build's log file location directly from the file system, as the one which is available at the time of the BuildStarted event is incorrect.
Quoted 7 lines
* Hide the "Download build log" button for builds which fail early (e.g. due to recipe parsing) and consequently have no
cooker_log_path.Changes since
986f6b04b26bba9d01393f8f4ced4e0f4385368a(toaster-next) are ingit://git.yoctoproject.org/poky-contrib, elliot/toaster/wrong_log-8373http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/wr
Quoted 1 line
ong_log-8373
These builds not only have no log: they also have no build data. That's why they should look like this:
http://www.yoctoproject.org/toaster/build-dashboard-failed-build-request.ht
ml
Quoted 5 lines
We should remove not only the 'download log' button, but also the left navigation and the 'build summary section'. It's all explained in this document:
Could you raise a separate bug for this, please? I would prefer to keep to the scope of the original bug, which was fixing the download log button.
I hid the download log button for failed builds, as it was broken due to the bug (i.e. the log location is incorrect: in the case of failed builds, it's missing). But I'd prefer not to get into fixing everything else about failed builds as part of the same patch series.
Thanks.
Elliot
Quoted 41 lines
Cheers
Belén
Quoted 33 lines
Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373
Elliot Smith (2):
toaster: Manually retrieve log file location from filesystem
toaster: Hide "Download build log" button if log doesn't exist
bitbake/lib/bb/ui/toasterui.py| 31
+++++++++++++++++++++-
.../toastergui/templates/builddashboard.html | 4 ++-2 files changed, 33 insertions(+), 2 deletions(-)--
Elliot Smith
Software Engineer
Intel OTC---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre
Quoted 3 lines · Elliot
Could you raise a separate bug for this, please? I would prefer to keep to the scope of the original bug, which was fixing the download log button.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8443
Cheers
Belén
Sent to upstream.
Pushed to toaster-next.
Quoted 45 lines · Elliot Smith
This supersedes https://lists.yoctoproject.org/pipermail/toaster/2015-September/002877.html
(which was a single commit).* Retrieve a build's log file location directly from the file system, as the one which is available at the time of the BuildStarted event is incorrect.
* Hide the "Download build log" button for builds which fail early (e.g. due to recipe parsing) and consequently have no
cooker_log_path.Changes since
986f6b04b26bba9d01393f8f4ced4e0f4385368a(toaster-next) are ingit://git.yoctoproject.org/poky-contrib, elliot/toaster/wrong_log-8373
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/wrong_log-8373Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373
Elliot Smith (2):
toaster: Manually retrieve log file location from filesystem
toaster: Hide "Download build log" button if log doesn't existbitbake/lib/bb/ui/toasterui.py | 31 +++++++++++++++++++++- .../toastergui/templates/builddashboard.html | 4 ++- 2 files changed, 33 insertions(+), 2 deletions(-)--
Elliot Smith
Software Engineer
Intel OTC---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Signature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Signature · 3 lines
-- Regards, Ed