Build cancellation
cover Build cancellation
This is build cancelation functionality implemented by Sujith and Michael and modified by me to work on top of 'per project build directory' patchset.
NOTE: This patchset depends on 'per project build directory' patchset' v5:
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
The following changes since commit e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:
toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/toaster/project-build-dir-7880-cancel
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/project-build-dir-7880-cancel
Ed Bartosh (2):
toaster: fix conflicting migrations
toaster: use empty token
Michael Wood (9):
toaster: Move xhr calls for starting and stopping builds
toaster: xhr Update the implementation of the build cancellation
request
toaster: libtoaster Update implementation of startABuild and
cancelABuild
toaster: bldcontrol models Add a cancelling state the BuildRequest
buildinfohelper: Add handler for cancelling a build
toaster: mrb_section template Add build cancel button
toaster: models Exclude the CANCELLED builds from get_number_of_builds
toaster: runbuilds Make runbuilds aware of the build CANCELLED state
toaster: runbuilds Clean up runbuilds
Sujith H (4):
toaster: bldcontrol Add forceShutDown function to BitbakeController
toaster: update BuildEnvironmentController and BitbakeController
toaster: models Add cancelled state to build outcome
toaster: tables BuildsTable exclude cancelled builds
bitbake/lib/bb/ui/buildinfohelper.py | 19 ++- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +--- .../toaster/bldcontrol/localhostbecontroller.py | 4 +- .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++----- .../migrations/0002_add_cancelling_state.py | 19 +++ .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++ bitbake/lib/toaster/bldcontrol/models.py | 27 +++- .../orm/migrations/0006_add_cancelled_state.py | 19 +++ bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++ bitbake/lib/toaster/orm/models.py | 12 +- bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++ .../toaster/toastergui/static/js/customrecipe.js | 4 +- .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +- .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++--- .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++ .../toaster/toastergui/static/js/projectpage.js | 4 +- .../toaster/toastergui/static/js/projecttopbar.js | 6 +- .../toaster/toastergui/static/js/recipedetails.js | 4 +- bitbake/lib/toaster/toastergui/tables.py | 48 +------ bitbake/lib/toaster/toastergui/templates/base.html | 1 + .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++------------- bitbake/lib/toaster/toastergui/urls.py | 5 + 22 files changed, 507 insertions(+), 228 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py create mode 100644 bitbake/lib/toaster/toastergui/api.py create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js
--
Regards,
Ed
PATCH 1/15 toaster: bldcontrol Add forceShutDown function to BitbakeController
From: Sujith H <[email protected]>
Add forceShutDown function to BitbakeController class. This function provides a mechanism to cancel the build from toaster. An API which can be used safely to cancel build and hence shutdown running bitbake server.
[YOCTO #6787]
--- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 3 +++ 1 file changed, 3 insertions(+)
bitbake/lib/toaster/bldcontrol/bbcontroller.py +3 -0
@@ -63,6 +63,9 @@ class BitbakeController(object): task = "build" return self._runCommand(["buildTargets", targets, task]) + def forceShutDown(self): + return self._runCommand(["stateForceShutdown"]) + def getBuildEnvironmentController(**kwargs):
Signature · 1 line
2.1.4
PATCH 2/15 toaster: Move xhr calls for starting and stopping builds
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
Move the backend xhr implementation of the build request changes into it's own file and out of the ToasterTable definition.
It used to live in the views.py but in a hope of starting to collate logical groups of views move this to a new file called api.
--- bitbake/lib/toaster/toastergui/api.py | 74 ++++++++++++++++++++++++++++++++ bitbake/lib/toaster/toastergui/tables.py | 43 +------------------ bitbake/lib/toaster/toastergui/urls.py | 5 +++ 3 files changed, 80 insertions(+), 42 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/api.py
bitbake/lib/toaster/toastergui/api.py new file +74 -0
@@ -0,0 +1,74 @@ +# +# BitBake Toaster Implementation +# +# Copyright (C) 2016 Intel Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + + +# Temporary home for the UI's misc API + +from orm.models import Project, ProjectTarget +from bldcontrol.models import BuildRequest +from bldcontrol import bbcontroller +from django.http import HttpResponse, JsonResponse +from django.views.generic import View + + +class XhrBuildRequest(View): + + def get(self, request, *args, **kwargs): + return HttpResponse() + + def post(self, request, *args, **kwargs): + """ Process HTTP POSTs which make build requests """ + + project = Project.objects.get(pk=kwargs['pid']) + + if 'buildCancel' in request.POST: + for i in request.POST['buildCancel'].strip().split(" "): + try: + br = BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_QUEUED) + br.state = BuildRequest.REQ_DELETED + br.save() + except BuildRequest.DoesNotExist: + pass + + if 'buildDelete' in request.POST: + for i in request.POST['buildDelete'].strip().split(" "): + try: + BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_DELETED).delete() + except BuildRequest.DoesNotExist: + pass + + if 'targets' in request.POST: + ProjectTarget.objects.filter(project = project).delete() + s = str(request.POST['targets']) + for t in s.translate(None, ";%|\"").split(" "): + if ":" in t: + target, task = t.split(":") + else: + target = t + task = "" + ProjectTarget.objects.create(project = project, + target = target, + task = task) + project.schedule_build() + + # redirect back to builds page so any new builds in progress etc. + # are visible + response = HttpResponse() + response.status_code = 302 + response['Location'] = request.build_absolute_uri() + return response
bitbake/lib/toaster/toastergui/tables.py +1 -42
55 lines not on this page · raw message
bitbake/lib/toaster/toastergui/urls.py +5 -0
@@ -22,6 +22,7 @@ from django.views.generic import RedirectView, TemplateView from django.http import HttpResponseBadRequest from toastergui import tables from toastergui import typeaheads +from toastergui import api urlpatterns = patterns('toastergui.views', # landing page
@@ -180,6 +181,10 @@ urlpatterns = patterns('toastergui.views', url(r'^xhr_customrecipe/', 'xhr_customrecipe', name='xhr_customrecipe'), + url(r'^xhr_buildrequest/project/(?P<pid>\d+)$', + api.XhrBuildRequest.as_view(), + name='xhr_buildrequest'), + # default redirection url(r'^$', RedirectView.as_view(url='landing', permanent=True)), )
Signature · 1 line
2.1.4
PATCH 3/15 toaster: xhr Update the implementation of the build cancellation request
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
Update the implementation of the backend api for cancelling builds with the new cancelling BuildRequest state and cancelled Build state. Also added some docstring about general usage.
Co-Author: Sujith H <[email protected]>
--- bitbake/lib/toaster/toastergui/api.py | 54 +++++++++++++++++++++++++++++------ 1 file changed, 45 insertions(+), 9 deletions(-)
bitbake/lib/toaster/toastergui/api.py +45 -9
@@ -19,7 +19,7 @@ # Temporary home for the UI's misc API -from orm.models import Project, ProjectTarget +from orm.models import Project, ProjectTarget, Build from bldcontrol.models import BuildRequest from bldcontrol import bbcontroller from django.http import HttpResponse, JsonResponse
@@ -32,18 +32,54 @@ class XhrBuildRequest(View): return HttpResponse() def post(self, request, *args, **kwargs): - """ Process HTTP POSTs which make build requests """ + """ + Build control + + Entry point: /xhr_buildrequest/<project_id> + Method: POST + + Args: + id: id of build to change + buildCancel = build_request_id ... + buildDelete = id ... + targets = recipe_name ... + + Returns: + {"error": "ok"} + or + {"error": <error message>} + """ project = Project.objects.get(pk=kwargs['pid']) if 'buildCancel' in request.POST: for i in request.POST['buildCancel'].strip().split(" "): try: - br = BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_QUEUED) - br.state = BuildRequest.REQ_DELETED + br = BuildRequest.objects.get(project=project, pk=i) + + try: + bbctrl = bbcontroller.BitbakeController(br.environment) + bbctrl.forceShutDown() + except: + # We catch a bunch of exceptions here because + # this is where the server has not had time to start up + # and the build request or build is in transit between + # processes. + # We can safely just set the build as cancelled + # already as it never got started + build = br.build + build.outcome = Build.CANCELLED + build.save() + + # We now hand over to the buildinfohelper to update the + # build state once we've finished cancelling + br.state = BuildRequest.REQ_CANCELLING br.save() + except BuildRequest.DoesNotExist: - pass + return JsonResponse({'error':'No such build id %s' % i}) + + return JsonResponse({'error': 'ok'}) if 'buildDelete' in request.POST: for i in request.POST['buildDelete'].strip().split(" "):
@@ -51,6 +87,7 @@ class XhrBuildRequest(View): BuildRequest.objects.select_for_update().get(project = project, pk = i, state__lte = BuildRequest.REQ_DELETED).delete() except BuildRequest.DoesNotExist: pass + return JsonResponse({'error': 'ok' }) if 'targets' in request.POST: ProjectTarget.objects.filter(project = project).delete()
@@ -66,9 +103,8 @@ class XhrBuildRequest(View): task = task) project.schedule_build() - # redirect back to builds page so any new builds in progress etc. - # are visible + return JsonResponse({'error': 'ok' }) + response = HttpResponse() - response.status_code = 302 - response['Location'] = request.build_absolute_uri() + response.status_code = 500 return response
Signature · 1 line
2.1.4
PATCH 4/15 toaster: libtoaster Update implementation of startABuild and cancelABuild
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
Update the implementation of startABuild and cancelAbuild to reflect changes to the backend api. We now have a dedicated endpoint to make calls into so add this url to libtoaster.ctx and allow passing null in as a url value to indicate that we want to use the current project
Also:
- Fix some documentation comments
- Add the convenience of passing in an array of targets to startABuild
--- .../toaster/toastergui/static/js/customrecipe.js | 4 +-- .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +- .../lib/toaster/toastergui/static/js/libtoaster.js | 41 ++++++++++++++-------- .../toaster/toastergui/static/js/projectpage.js | 4 +-- .../toaster/toastergui/static/js/projecttopbar.js | 6 ++-- .../toaster/toastergui/static/js/recipedetails.js | 4 +-- bitbake/lib/toaster/toastergui/templates/base.html | 1 + 7 files changed, 34 insertions(+), 29 deletions(-)
bitbake/lib/toaster/toastergui/static/js/customrecipe.js +1 -3
@@ -207,9 +207,7 @@ function customRecipePageInit(ctx) { /* Trigger a build of your custom image */ $(".build-custom-image").click(function(){ - libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl, - libtoaster.ctx.projectId, - ctx.recipe.name, + libtoaster.startABuild(null, ctx.recipe.name, function(){ window.location.replace(libtoaster.ctx.projectBuildsUrl); });
bitbake/lib/toaster/toastergui/static/js/layerBtn.js +1 -2
@@ -60,8 +60,7 @@ function layerBtnsInit() { e.preventDefault(); var recipe = $(this).data('recipe-name'); - libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl, - libtoaster.ctx.projectId, recipe, + libtoaster.startABuild(null, recipe, function(){ /* Success */ window.location.replace(libtoaster.ctx.projectBuildsUrl);
bitbake/lib/toaster/toastergui/static/js/libtoaster.js +26 -15
75 lines not on this page · raw message
bitbake/lib/toaster/toastergui/static/js/projectpage.js +1 -3
@@ -232,9 +232,7 @@ function projectPageInit(ctx) { toBuild = toBuild.trim(); - libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl, - libtoaster.ctx.projectId, - toBuild, + libtoaster.startABuild(null, toBuild, function(){ /* Build request started */ window.location.replace(libtoaster.ctx.projectBuildsUrl);
bitbake/lib/toaster/toastergui/static/js/projecttopbar.js +3 -3
@@ -82,9 +82,9 @@ function projectTopBarInit(ctx) { selectedTarget = { name: newBuildTargetInput.val() }; /* Fire off the build */ - libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl, - null, selectedTarget.name, function(){ - window.location.replace(libtoaster.ctx.projectBuildsUrl); + libtoaster.startABuild(null, selectedTarget.name, + function(){ + window.location.replace(libtoaster.ctx.projectBuildsUrl); }, null); }); }
bitbake/lib/toaster/toastergui/static/js/recipedetails.js +1 -3
@@ -42,9 +42,7 @@ function recipeDetailsPageInit(ctx){ /* Trigger a build of your custom image */ $(".build-recipe-btn").click(function(){ - libtoaster.startABuild(libtoaster.ctx.projectBuildsUrl, - libtoaster.ctx.projectId, - ctx.recipe.name, + libtoaster.startABuild(null, ctx.recipe.name, function(){ window.location.replace(libtoaster.ctx.projectBuildsUrl); });
bitbake/lib/toaster/toastergui/templates/base.html +1 -0
@@ -47,6 +47,7 @@ projectBuildsUrl: {% url 'projectbuilds' project.id as pburl %}{{pburl|json}}, xhrCustomRecipeUrl : "{% url 'xhr_customrecipe' %}", projectId : {{project.id}}, + xhrBuildRequestUrl: "{% url 'xhr_buildrequest' project.id %}", {% else %} projectId : undefined, projectPageUrl : undefined,
Signature · 1 line
2.1.4
PATCH 5/15 toaster: update BuildEnvironmentController and BitbakeController
In reply to Ed BartoshFrom: Sujith H <[email protected]>
Remove getBBController function from BuildEnvironmentController. The constructor of BitbakeController is updated appropriately so that call can be made to connect to running server.
The call to startBBServer is removed from bbcontroller and handledin localhostbecontroller.
[YOCTO #6787]
--- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 26 ++++------------------ .../toaster/bldcontrol/localhostbecontroller.py | 2 +- 2 files changed, 5 insertions(+), 23 deletions(-)
bitbake/lib/toaster/bldcontrol/bbcontroller.py +4 -22
@@ -37,11 +37,12 @@ class BitbakeController(object): It is outside the scope of this class on how the server is started and aquired """ - def __init__(self, connection): - self.connection = connection + def __init__(self, be): + self.connection = bb.server.xmlrpc._create_server(be.bbaddress, + int(be.bbport))[0] def _runCommand(self, command): - result, error = self.connection.connection.runCommand(command) + result, error = self.connection.runCommand(command) if error: raise Exception(error) return result
@@ -122,25 +123,6 @@ class BuildEnvironmentController(object): """ raise NotImplementedError("FIXME: Must override setLayers") - - def getBBController(self): - """ returns a BitbakeController to an already started server; this is the point where the server - starts if needed; or reconnects to the server if we can - """ - if not self.connection: - self.be.lock = BuildEnvironment.LOCK_RUNNING - self.be.save() - - server = bb.server.xmlrpc.BitBakeXMLRPCClient() - server.initServer() - server.saveConnectionDetails("%s:%s" % (self.be.bbaddress, self.be.bbport)) - self.connection = server.establishConnection([]) - - self.be.bbtoken = self.connection.transport.connection_token - self.be.save() - - return BitbakeController(self.connection) - def getArtifact(self, path): """ This call returns an artifact identified by the 'path'. How 'path' is interpreted as up to the implementing BEC. The return MUST be a REST URL where a GET will actually return
bitbake/lib/toaster/bldcontrol/localhostbecontroller.py +1 -1
@@ -32,7 +32,7 @@ import subprocess from toastermain import settings -from bbcontroller import BuildEnvironmentController, ShellCmdException, BuildSetupException +from bbcontroller import BuildEnvironmentController, ShellCmdException, BuildSetupException, BitbakeController import logging logger = logging.getLogger("toaster")
Signature · 1 line
2.1.4
PATCH 6/15 toaster: models Add cancelled state to build outcome
In reply to Ed BartoshFrom: Sujith H <[email protected]>
A new state CANCELLED is introduced to, distinguish the state of build.
[YOCTO #6787]
--- .../orm/migrations/0006_add_cancelled_state.py | 19 +++++++++++++++++++ bitbake/lib/toaster/orm/models.py | 7 ++++++- 2 files changed, 25 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py
bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py new file +19 -0
@@ -0,0 +1,19 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('orm', '0004_provides'), + ] + + operations = [ + migrations.AlterField( + model_name='build', + name='outcome', + field=models.IntegerField(default=2, choices=[(0, b'Succeeded'), (1, b'Failed'), (2, b'In Progress'), (3, b'Cancelled')]), + ), + ]
bitbake/lib/toaster/orm/models.py +6 -1
@@ -358,11 +358,13 @@ class Build(models.Model): SUCCEEDED = 0 FAILED = 1 IN_PROGRESS = 2 + CANCELLED = 3 BUILD_OUTCOME = ( (SUCCEEDED, 'Succeeded'), (FAILED, 'Failed'), (IN_PROGRESS, 'In Progress'), + (CANCELLED, 'Cancelled'), ) search_allowed_fields = ['machine', 'cooker_log_path', "target__target", "target__target_image_file__file_name"]
@@ -389,7 +391,10 @@ class Build(models.Model): if project: builds = builds.filter(project=project) - finished_criteria = Q(outcome=Build.SUCCEEDED) | Q(outcome=Build.FAILED) + finished_criteria = \ + Q(outcome=Build.SUCCEEDED) | \ + Q(outcome=Build.FAILED) | \ + Q(outcome=Build.CANCELLED) recent_builds = list(itertools.chain( builds.filter(outcome=Build.IN_PROGRESS).order_by("-started_on"),
Signature · 1 line
2.1.4
PATCH 7/15 toaster: bldcontrol models Add a cancelling state the BuildRequest
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
To accurately reflect the state of a build request we also need a cancelling state. This is set when we've started a build and then for whatever reason cancel it, cancelling is not instantaneous so we have this state to indicate that a cancel is in progress.
Also add a state transition guard. As the state of a BuildRequest can currently be modified by three processes; Toastergui, Runbuilds/bldcontrol and the buildinofhelper we cannot say for sure which process will be running at the time of cancellation so in order to avoid one of these processes making an incorrect transition only allow transitions of state to increase.
e.g. CREATED -> QUEUED -> INPROGRESS
And to ignore such requested changes such as
INPROGRESS -> CREATED
--- .../migrations/0002_add_cancelling_state.py | 19 +++++++++++++++ bitbake/lib/toaster/bldcontrol/models.py | 27 +++++++++++++++++++++- 2 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py
bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py new file +19 -0
@@ -0,0 +1,19 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('bldcontrol', '0001_initial'), + ] + + operations = [ + migrations.AlterField( + model_name='buildrequest', + name='state', + field=models.IntegerField(default=0, choices=[(0, b'created'), (1, b'queued'), (2, b'in progress'), (3, b'completed'), (4, b'failed'), (5, b'deleted'), (6, b'cancelling'), (7, b'archive')]), + ), + ]
bitbake/lib/toaster/bldcontrol/models.py +26 -1
@@ -4,6 +4,8 @@ from django.core.validators import MaxValueValidator, MinValueValidator from django.utils.encoding import force_bytes from orm.models import Project, ProjectLayer, ProjectVariable, ProjectTarget, Build, Layer_Version +import logging +logger = logging.getLogger("toaster") # a BuildEnvironment is the equivalent of the "build/" directory on the localhost class BuildEnvironment(models.Model): SERVER_STOPPED = 0
@@ -64,7 +66,8 @@ class BuildRequest(models.Model): REQ_COMPLETED = 3 REQ_FAILED = 4 REQ_DELETED = 5 - REQ_ARCHIVE = 6 + REQ_CANCELLING = 6 + REQ_ARCHIVE = 7 REQUEST_STATE = ( (REQ_CREATED, "created"),
@@ -73,6 +76,7 @@ class BuildRequest(models.Model): (REQ_COMPLETED, "completed"), (REQ_FAILED, "failed"), (REQ_DELETED, "deleted"), + (REQ_CANCELLING, "cancelling"), (REQ_ARCHIVE, "archive"), )
@@ -85,6 +89,27 @@ class BuildRequest(models.Model): created = models.DateTimeField(auto_now_add = True) updated = models.DateTimeField(auto_now = True) + def __init__(self, *args, **kwargs): + super(BuildRequest, self).__init__(*args, **kwargs) + # Save the old state incase it's about to be modified + self.old_state = self.state + + def save(self, *args, **kwargs): + # Check that the state we're trying to set is not going backwards + # e.g. from REQ_FAILED to REQ_INPROGRESS + if self.old_state != self.state and self.old_state > self.state: + logger.warn("Invalid state change requested: " + "Cannot go from %s to %s - ignoring request" % + (BuildRequest.REQUEST_STATE[self.old_state][1], + BuildRequest.REQUEST_STATE[self.state][1]) + ) + # Set property back to the old value + self.state = self.old_state + return + + super(BuildRequest, self).save(*args, **kwargs) + + def get_duration(self): return (self.updated - self.created).total_seconds()
Signature · 1 line
2.1.4
PATCH 8/15 buildinfohelper: Add handler for cancelling a build
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
When a build is cancelled the build (action) is complete if it has been caused the request being cancelled then update the build outcome accordingly.
--- bitbake/lib/bb/ui/buildinfohelper.py | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/buildinfohelper.py +18 -1
@@ -215,6 +215,15 @@ class ORMWrapper(object): assert isinstance(errors, int) assert isinstance(warnings, int) + if build.outcome == Build.CANCELLED: + return + try: + if build.buildrequest.state == BuildRequest.REQ_CANCELLING: + return + except AttributeError: + # We may not have a buildrequest if this is a command line build + pass + outcome = Build.SUCCEEDED if errors or taskfailures: outcome = Build.FAILED
@@ -1390,9 +1399,17 @@ class BuildInfoHelper(object): be.lock = BuildEnvironment.LOCK_LOCK be.save() br = BuildRequest.objects.get(pk = br_id) + + # if we're 'done' because we got cancelled update the build outcome + if br.state == BuildRequest.REQ_CANCELLING: + logger.info("Build cancelled") + br.build.outcome = Build.CANCELLED + br.build.save() + errorcode = 0 + if errorcode == 0: # request archival of the project artifacts - br.state = BuildRequest.REQ_ARCHIVE + br.state = BuildRequest.REQ_COMPLETED else: br.state = BuildRequest.REQ_FAILED br.save()
Signature · 1 line
2.1.4
PATCH 10/15 toaster: mrb_section template Add build cancel button
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
Add the cancel build button to the mrb section template and add the event handlers to cancelABuild.
Also clean up the calls to startABuild to use the updated libtoaster methods and to make the code consistent with it's cancelABuild counterpart.
Co-Author: Sujith H <[email protected]>
--- .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++ .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++------------- 2 files changed, 149 insertions(+), 94 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js
bitbake/lib/toaster/toastergui/static/js/mrbsection.js new file +95 -0
@@ -0,0 +1,95 @@ + +function mrbSectionInit(ctx){ + + var projectBuilds; + + if (ctx.mrbType === 'project') + projectBuilds = true; + + $(".cancel-build-btn").click(function(e){ + e.preventDefault(); + + var url = $(this).data('request-url'); + var buildReqIds = $(this).data('buildrequest-id'); + var banner = $(this).parents(".alert"); + + banner.find(".progress-info").fadeOut().promise().done(function(){ + $("#cancelling-msg-" + buildReqIds).show(); + console.log("cancel build"); + libtoaster.cancelABuild(url, buildReqIds, function(){ + if (projectBuilds == false){ + /* the all builds page is not 'self updating' like thei + * project Builds + */ + window.location.reload(); + } + }, null); + }); + }); + + $(".run-again-btn").click(function(e){ + e.preventDefault(); + + var url = $(this).data('request-url'); + var target = $(this).data('target'); + + libtoaster.startABuild(url, target, function(){ + window.location.reload(); + }, null); + }); + + + var progressTimer; + + if (projectBuilds === true){ + progressTimer = window.setInterval(function() { + libtoaster.getProjectInfo(libtoaster.ctx.projectPageUrl, + function(prjInfo){ + /* These two are needed because a build can be 100% and still + * in progress due to the fact that the % done is updated at the + * start of a task so it can be doing the last task at 100% + */ + var inProgress = 0; + var allPercentDone = 0; + if (prjInfo.builds.length === 0) + return + + for (var i in prjInfo.builds){ + var build = prjInfo.builds[i]; + + if (build.outcome === "In Progress" || + $(".progress .bar").length > 0){ + /* Update the build progress */ + var percentDone; + + if (build.outcome !== "In Progress"){ + /* We have to ignore the value when it's Succeeded because it + * goes back to 0 + */ + percentDone = 100; + } else { + percentDone = build.percentDone; + inProgress++; + } + + $("#build-pc-done-" + build.id).text(percentDone); + $("#build-pc-done-title-" + build.id).attr("title", percentDone); + $("#build-pc-done-bar-" + build.id).css("width", + String(percentDone) + "%"); + + allPercentDone += percentDone; + } + } + + if (allPercentDone === (100 * prjInfo.builds.length) && !inProgress) + window.location.reload(); + + /* Our progress bar is not still showing so shutdown the polling. */ + if ($(".progress .bar").length === 0) + window.clearInterval(progressTimer); + + }); + }, 1500); + } +} +
bitbake/lib/toaster/toastergui/templates/mrb_section.html +54 -94
177 lines not on this page · raw message
Signature · 1 line
2.1.4
PATCH 11/15 toaster: models Exclude the CANCELLED builds from get_number_of_builds
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
Don't count CANCELLED builds when returning the number of builds.
--- bitbake/lib/toaster/orm/models.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
bitbake/lib/toaster/orm/models.py +4 -1
@@ -197,7 +197,10 @@ class Project(models.Model): def get_number_of_builds(self): """Return the number of builds which have ended""" - return self.build_set.filter(~Q(outcome=Build.IN_PROGRESS)).count() + return self.build_set.exclude( + Q(outcome=Build.IN_PROGRESS) | + Q(outcome=Build.CANCELLED) + ).count() def get_last_build_id(self): try:
Signature · 1 line
2.1.4
PATCH 12/15 toaster: runbuilds Make runbuilds aware of the build CANCELLED state
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
Add handlers to make sure we remove the BuildEnvironment LOCK when we have cancelled a build.
--- .../bldcontrol/management/commands/runbuilds.py | 26 ++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-)
bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py +22 -4
@@ -1,5 +1,7 @@ from django.core.management.base import NoArgsCommand, CommandError from django.db import transaction +from django.db.models import Q + from orm.models import Build, ToasterSetting, LogMessage, Target from bldcontrol.bbcontroller import getBuildEnvironmentController, ShellCmdException, BuildSetupException from bldcontrol.models import BuildRequest, BuildEnvironment, BRError, BRVariable
@@ -86,13 +88,20 @@ class Command(NoArgsCommand): def cleanup(self): from django.utils import timezone from datetime import timedelta - # environments locked for more than 30 seconds - they should be unlocked - BuildEnvironment.objects.filter(buildrequest__state__in=[BuildRequest.REQ_FAILED, BuildRequest.REQ_COMPLETED]).filter(lock=BuildEnvironment.LOCK_LOCK).filter(updated__lt = timezone.now() - timedelta(seconds = 30)).update(lock = BuildEnvironment.LOCK_FREE) - - # update all Builds that failed to start for br in BuildRequest.objects.filter(state = BuildRequest.REQ_FAILED, build__outcome = Build.IN_PROGRESS): + # environments locked for more than 30 seconds + # they should be unlocked + BuildEnvironment.objects.filter( + Q(buildrequest__state__in=[BuildRequest.REQ_FAILED, + BuildRequest.REQ_COMPLETED, + BuildRequest.REQ_CANCELLING]) & + Q(lock=BuildEnvironment.LOCK_LOCK) & + Q(updated__lt=timezone.now() - timedelta(seconds = 30)) + ).update(lock=BuildEnvironment.LOCK_FREE) + + # transpose the launch errors in ToasterExceptions br.build.outcome = Build.FAILED for brerror in br.brerror_set.all():
@@ -125,6 +134,15 @@ class Command(NoArgsCommand): br.build.save() pass + # Make sure the LOCK is removed for builds which have been fully + # cancelled + for br in BuildRequest.objects.filter( + Q(build__outcome=Build.CANCELLED) & + Q(state=BuildRequest.REQ_CANCELLING) & + ~Q(environment=None)): + br.environment.lock = BuildEnvironment.LOCK_FREE + br.environment.save() + def handle_noargs(self, **options): while True:
Signature · 1 line
2.1.4
PATCH 9/15 toaster: tables BuildsTable exclude cancelled builds
In reply to Ed BartoshFrom: Sujith H <[email protected]>
Exclude cancelled builds from showing in the builds table
[YOCTO #6787]
--- bitbake/lib/toaster/toastergui/tables.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
bitbake/lib/toaster/toastergui/tables.py +3 -2
@@ -1080,8 +1080,9 @@ class BuildsTable(ToasterTable): """ queryset = self.get_builds() - # don't include in progress builds - queryset = queryset.exclude(outcome=Build.IN_PROGRESS) + # Don't include in progress builds pr cancelled builds + queryset = queryset.exclude(Q(outcome=Build.IN_PROGRESS) | + Q(outcome=Build.CANCELLED)) # sort queryset = queryset.order_by(self.default_orderby)
Signature · 1 line
2.1.4
PATCH 13/15 toaster: runbuilds Clean up runbuilds
In reply to Ed BartoshFrom: Michael Wood <[email protected]>
- Organise the imports into logical groups
- Fix 80 col wrapping
- Remove catch all exceptions
- Log to the toaster log
- Use QuerySet functions such as .first() and Q()
--- .../bldcontrol/management/commands/runbuilds.py | 81 ++++++++++++++-------- 1 file changed, 52 insertions(+), 29 deletions(-)
bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py +52 -29
@@ -2,18 +2,25 @@ from django.core.management.base import NoArgsCommand, CommandError from django.db import transaction from django.db.models import Q +from bldcontrol.bbcontroller import getBuildEnvironmentController +from bldcontrol.bbcontroller import ShellCmdException, BuildSetupException +from bldcontrol.models import BuildRequest, BuildEnvironment +from bldcontrol.models import BRError, BRVariable + from orm.models import Build, ToasterSetting, LogMessage, Target -from bldcontrol.bbcontroller import getBuildEnvironmentController, ShellCmdException, BuildSetupException -from bldcontrol.models import BuildRequest, BuildEnvironment, BRError, BRVariable + import os import logging import time +import sys +import traceback -logger = logging.getLogger("ToasterScheduler") +logger = logging.getLogger("toaster") class Command(NoArgsCommand): args = "" - help = "Schedules and executes build requests as possible. Does not return (interrupt with Ctrl-C)" + help = "Schedules and executes build requests as possible." + "Does not return (interrupt with Ctrl-C)" @transaction.atomic
@@ -25,21 +32,19 @@ class Command(NoArgsCommand): @transaction.atomic def _selectBuildRequest(self): - br = BuildRequest.objects.filter(state = BuildRequest.REQ_QUEUED).order_by('pk')[0] - br.state = BuildRequest.REQ_INPROGRESS - br.save() + br = BuildRequest.objects.filter(state=BuildRequest.REQ_QUEUED).first() return br def schedule(self): - import traceback try: - br = None - try: - # select the build environment and the request to build - br = self._selectBuildRequest() - except IndexError as e: - #logger.debug("runbuilds: No build request") + # select the build environment and the request to build + br = self._selectBuildRequest() + if br: + br.state = BuildRequest.REQ_INPROGRESS + br.save() + else: return + try: bec = self._selectBuildEnvironment() except IndexError as e:
@@ -49,15 +54,17 @@ class Command(NoArgsCommand): logger.debug("runbuilds: No build env") return - logger.debug("runbuilds: starting build %s, environment %s" % (str(br).decode('utf-8'), bec.be)) + logger.debug("runbuilds: starting build %s, environment %s" % \ + (str(br).decode('utf-8'), bec.be)) # let the build request know where it is being executed br.environment = bec.be br.save() # this triggers an async build - bec.triggerBuild(br.brbitbake, br.brlayer_set.all(), br.brvariable_set.all(), - br.brtarget_set.all(), "%d:%d" % (br.pk, bec.be.pk)) + bec.triggerBuild(br.brbitbake, br.brlayer_set.all(), + br.brvariable_set.all(), br.brtarget_set.all(), + "%d:%d" % (br.pk, bec.be.pk)) except Exception as e: logger.error("runbuilds: Error launching build %s" % e)
@@ -88,9 +95,6 @@ class Command(NoArgsCommand): def cleanup(self): from django.utils import timezone from datetime import timedelta - # update all Builds that failed to start - - for br in BuildRequest.objects.filter(state = BuildRequest.REQ_FAILED, build__outcome = Build.IN_PROGRESS): # environments locked for more than 30 seconds # they should be unlocked BuildEnvironment.objects.filter(
@@ -102,22 +106,30 @@ class Command(NoArgsCommand): ).update(lock=BuildEnvironment.LOCK_FREE) + # update all Builds that were in progress and failed to start + for br in BuildRequest.objects.filter( + state=BuildRequest.REQ_FAILED, + build__outcome=Build.IN_PROGRESS): # transpose the launch errors in ToasterExceptions br.build.outcome = Build.FAILED for brerror in br.brerror_set.all(): logger.debug("Saving error %s" % brerror) - LogMessage.objects.create(build = br.build, level = LogMessage.EXCEPTION, message = brerror.errmsg) + LogMessage.objects.create(build=br.build, + level=LogMessage.EXCEPTION, + message=brerror.errmsg) br.build.save() - # we don't have a true build object here; hence, toasterui didn't have a change to release the BE lock + # we don't have a true build object here; hence, toasterui + # didn't have a change to release the BE lock br.environment.lock = BuildEnvironment.LOCK_FREE br.environment.save() - # update all BuildRequests without a build created for br in BuildRequest.objects.filter(build = None): - br.build = Build.objects.create(project = br.project, completed_on = br.updated, started_on = br.created) + br.build = Build.objects.create(project=br.project, + completed_on=br.updated, + started_on=br.created) br.build.outcome = Build.FAILED try: br.build.machine = br.brvariable_set.get(name='MACHINE').value
@@ -126,13 +138,16 @@ class Command(NoArgsCommand): br.save() # transpose target information for brtarget in br.brtarget_set.all(): - Target.objects.create(build=br.build, target=brtarget.target, task=brtarget.task) + Target.objects.create(build=br.build, + target=brtarget.target, + task=brtarget.task) # transpose the launch errors in ToasterExceptions for brerror in br.brerror_set.all(): - LogMessage.objects.create(build = br.build, level = LogMessage.EXCEPTION, message = brerror.errmsg) + LogMessage.objects.create(build=br.build, + level=LogMessage.EXCEPTION, + message=brerror.errmsg) br.build.save() - pass # Make sure the LOCK is removed for builds which have been fully # cancelled
@@ -148,9 +163,17 @@ class Command(NoArgsCommand): while True: try: self.cleanup() + except Exception as e: + logger.warn("runbuilds: cleanup exception %s" % str(e)) + + try: self.archive() + except Exception as e: + logger.warn("runbuilds: archive exception %s" % str(e)) + + try: self.schedule() - except: - pass + except Exception as e: + logger.warn("runbuilds: schedule exception %s" % str(e)) time.sleep(1)
Signature · 1 line
2.1.4
PATCH 14/15 toaster: fix conflicting migrations
In reply to Ed BartoshFixed by running manage.py makemigrations --merge
--- bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py | 15 +++++++++++++++ bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py
bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py new file +15 -0
@@ -0,0 +1,15 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('bldcontrol', '0002_add_cancelling_state'), + ('bldcontrol', '0002_auto_20160120_1250'), + ] + + operations = [ + ]
bitbake/lib/toaster/orm/migrations/0007_merge.py new file +15 -0
@@ -0,0 +1,15 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('orm', '0006_add_cancelled_state'), + ('orm', '0006_remove_build_build_name'), + ] + + operations = [ + ]
Signature · 1 line
2.1.4
PATCH 15/15 toaster: use empty token
In reply to Ed BartoshIf client xmlrpc token is not provided in the command line, bitbake generates random token. Server token in --server-only mode is always empty. This prevents clients with non-empty tokens to communicate with the server. Specifying empty token should prevent generation of random token and makes it possible to communicate with the server.
--- bitbake/lib/toaster/bldcontrol/localhostbecontroller.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/toaster/bldcontrol/localhostbecontroller.py +1 -1
@@ -319,7 +319,7 @@ class LocalhostBEController(BuildEnvironmentController): # run build with local bitbake. stop the server after the build. log = os.path.join(builddir, 'toaster_ui.log') self._shellcmd(['bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:-1" ' - '../bitbake/bin/bitbake %s -u toasterui >>%s 2>&1;' + '../bitbake/bin/bitbake %s -u toasterui --token="" >>%s 2>&1;' 'BITBAKE_UI="" BBSERVER=0.0.0.0:-1 %s -m)&\"' \ % (brbe, bbtargets, log, bitbake)], builddir, nowait=True)
Signature · 1 line
2.1.4
Hi,
I finally had a chance to try out the 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir' branch and I had comments:
1) use project name for build dirs? This would make it clearer which build dir has which projects in it.
2) we cannot assume I am building in ~/poky/build. There are env variables to let us find where poky is $OEROOT for instance. My bin/sh is bash but I still got the following when building from $HOME/cow/test/ when poky was in $HOME/src/poky :
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
3) if a build fails (as above) and then I cancel it, my state stays in Cancelling the build ...
4) when I cancel a build it says “Failed” it should probably say “Canceled” and still be red.
5) If I change one of my base project settings (like my download folder) the next (and following) projects should probably use my last setting as the default rather than the hardcoded default.
For instance I change my downloads from /home/…/downloads to /home/…./DL in cowMaster, when I make cowJethro, the default DL_DIR should be /home/…./DL
The following were done based on Jethro:
6) built rpi-hwup-image - then made custom image based on it. CI interface says: “Toaster has no package information for rpime. To generate package information, build rpime”
7) same true for customizing off off core-image-minimal. for both 6/7 machine = raspberrypi2.
8) trying to download the recipe file gives me the yellow error screen [Errno 2] No such file or directory:
u'/recipes-core/images/rpi-hwup-image.bb'
9) if I switch back to qemux86 as the machine, build core-image-minimal, and then customize it I also get the Toaster has no package information for cimx86me. To generate package information, build cimx86me
so it is not just due to it being a rpi image. this also gives [Errno 2] No such file or directory:
u'meta/recipes-core/images/core-image-minimal.bb'
if it try to download it.
10) after building the cimx86me, I can now download the recipe but I still do not see any packages.
11) I’m not personally fond of having the sstate and dl_dir being set on the project creation page. they are available in the bitbake var page for each project which is nice. This is more a Belén thing. just putting my opinion down.
12) if I cancel a command line build from the command line (as the little question mark suggests) the cancellation is not reflected in the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
——
So as far as upstreaming, I think we need to have the following 2 things fixed first:
1) being able to run/build from an arbitrary dir and not assuming we are in the poky tree. (should be ez)
2) fixing the custom image stuff. Since the CI is the main selling point of the 2.1 release we need to make sure we don’t break it. (hopefully not too hard?)
Other issues can be made as bugs on top of the upstream, i think.
Hope this helps,
Brian
an Intel employee
Quoted 76 lines · Ed Bartosh
This is build cancelation functionality implemented by Sujith and Michael and modified by me to work on top of 'per project build directory' patchset.
NOTE: This patchset depends on 'per project build directory' patchset' v5:
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.htmlThe following changes since commit
e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/toaster/project-build-dir-7880-cancel
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/project-build-dir-7880-cancelEd Bartosh (2):
toaster: fix conflicting migrations
toaster: use empty tokenMichael Wood (9):
toaster: Move xhr calls for starting and stopping builds
toaster: xhr Update the implementation of the build cancellation
request
toaster: libtoaster Update implementation of startABuild and
cancelABuild
toaster: bldcontrol models Add a cancelling state the BuildRequest
buildinfohelper: Add handler for cancelling a build
toaster:mrb_sectiontemplate Add build cancel button
toaster: models Exclude the CANCELLED builds fromget_number_of_builds
toaster: runbuilds Make runbuilds aware of the build CANCELLED state
toaster: runbuilds Clean up runbuildsSujith H (4):
toaster: bldcontrol Add forceShutDown function to BitbakeController
toaster: update BuildEnvironmentController and BitbakeController
toaster: models Add cancelled state to build outcome
toaster: tables BuildsTable exclude cancelled buildsbitbake/lib/bb/ui/buildinfohelper.py | 19 ++- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +--- .../toaster/bldcontrol/localhostbecontroller.py | 4 +- .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++----- .../migrations/0002_add_cancelling_state.py | 19 +++ .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++ bitbake/lib/toaster/bldcontrol/models.py | 27 +++- .../orm/migrations/0006_add_cancelled_state.py | 19 +++ bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++ bitbake/lib/toaster/orm/models.py | 12 +- bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++ .../toaster/toastergui/static/js/customrecipe.js | 4 +- .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +- .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++--- .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++ .../toaster/toastergui/static/js/projectpage.js | 4 +- .../toaster/toastergui/static/js/projecttopbar.js | 6 +- .../toaster/toastergui/static/js/recipedetails.js | 4 +- bitbake/lib/toaster/toastergui/tables.py | 48 +------ bitbake/lib/toaster/toastergui/templates/base.html | 1 + .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++------------- bitbake/lib/toaster/toastergui/urls.py | 5 + 22 files changed, 507 insertions(+), 228 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py create mode 100644 bitbake/lib/toaster/toastergui/api.py create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js--
Regards,
Ed
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
a couple of other thoughts after looking at it a little more today:
1) pid of bitbake started in proj builds dir for cleanup? do we want to save these?
bitbake.lock with lsof might mean we don't need these. however, having them might negate the lsof requirement...)
2)should poky oe-init be a #!/bin/bash if dash fails?
3) toasterui event loop should probably check for liveness of server connection…. or do we get an error on waitEvent if the server has died (segv’d, kill -9 by user, etc)?
4) add tests for start, cancel builds... this needs to be a bugzilla entry so we have some tests for it. We don't need to do a build,just make sure we can start/cancel them.
- we may want to add a success.bbclass inteo meta-oe-selftest like
the error.bbclass i added awhile ago. the error class doesn’t build anything so it’s very fast...
-b
an intel employee
Quoted 179 lines · Brian Avery
Hi,
I finally had a chance to try out the 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir' branch and I had comments:
1) use project name for build dirs? This would make it clearer which build dir has which projects in it.
2) we cannot assume I am building in ~/poky/build. There are env variables to let us find where poky is $OEROOT for instance. My bin/sh is bash but I still got the following when building from $HOME/cow/test/ when poky was in $HOME/src/poky :
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
3) if a build fails (as above) and then I cancel it, my state stays in Cancelling the build ...
4) when I cancel a build it says “Failed” it should probably say “Canceled” and still be red.
5) If I change one of my base project settings (like my download folder) the next (and following) projects should probably use my last setting as the default rather than the hardcoded default.
For instance I change my downloads from /home/…/downloads to /home/…./DL in cowMaster, when I make cowJethro, the default
DL_DIRshould be /home/…./DLThe following were done based on Jethro:
6) built rpi-hwup-image - then made custom image based on it. CI interface says: “Toaster has no package information for rpime. To generate package information, build rpime”
7) same true for customizing off off core-image-minimal. for both 6/7 machine = raspberrypi2.
8) trying to download the recipe file gives me the yellow error screen [Errno 2] No such file or directory:
u'/recipes-core/images/rpi-hwup-image.bb'9) if I switch back to qemux86 as the machine, build core-image-minimal, and then customize it I also get the Toaster has no package information for cimx86me. To generate package information, build cimx86me
so it is not just due to it being a rpi image. this also gives [Errno 2] No such file or directory:
u'meta/recipes-core/images/core-image-minimal.bb'if it try to download it.
10) after building the cimx86me, I can now download the recipe but I still do not see any packages.
11) I’m not personally fond of having the sstate and
dl_dirbeing set on the project creation page. they are available in the bitbake var page for each project which is nice. This is more a Belén thing. just putting my opinion down.12) if I cancel a command line build from the command line (as the little question mark suggests) the cancellation is not reflected in the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
——
So as far as upstreaming, I think we need to have the following 2 things fixed first:
1) being able to run/build from an arbitrary dir and not assuming we are in the poky tree. (should be ez)
2) fixing the custom image stuff. Since the CI is the main selling point of the 2.1 release we need to make sure we don’t break it. (hopefully not too hard?)
Other issues can be made as bugs on top of the upstream, i think.
Hope this helps,
Brian
an Intel employeeQuoted 76 lines · Ed Bartosh
This is build cancelation functionality implemented by Sujith and Michael and modified by me to work on top of 'per project build directory' patchset.
NOTE: This patchset depends on 'per project build directory' patchset' v5:
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.htmlThe following changes since commit
e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/toaster/project-build-dir-7880-cancel
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/project-build-dir-7880-cancelEd Bartosh (2):
toaster: fix conflicting migrations
toaster: use empty tokenMichael Wood (9):
toaster: Move xhr calls for starting and stopping builds
toaster: xhr Update the implementation of the build cancellation
request
toaster: libtoaster Update implementation of startABuild and
cancelABuild
toaster: bldcontrol models Add a cancelling state the BuildRequest
buildinfohelper: Add handler for cancelling a build
toaster:mrb_sectiontemplate Add build cancel button
toaster: models Exclude the CANCELLED builds fromget_number_of_builds
toaster: runbuilds Make runbuilds aware of the build CANCELLED state
toaster: runbuilds Clean up runbuildsSujith H (4):
toaster: bldcontrol Add forceShutDown function to BitbakeController
toaster: update BuildEnvironmentController and BitbakeController
toaster: models Add cancelled state to build outcome
toaster: tables BuildsTable exclude cancelled buildsbitbake/lib/bb/ui/buildinfohelper.py | 19 ++- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +--- .../toaster/bldcontrol/localhostbecontroller.py | 4 +- .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++----- .../migrations/0002_add_cancelling_state.py | 19 +++ .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++ bitbake/lib/toaster/bldcontrol/models.py | 27 +++- .../orm/migrations/0006_add_cancelled_state.py | 19 +++ bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++ bitbake/lib/toaster/orm/models.py | 12 +- bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++ .../toaster/toastergui/static/js/customrecipe.js | 4 +- .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +- .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++--- .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++ .../toaster/toastergui/static/js/projectpage.js | 4 +- .../toaster/toastergui/static/js/projecttopbar.js | 6 +- .../toaster/toastergui/static/js/recipedetails.js | 4 +- bitbake/lib/toaster/toastergui/tables.py | 48 +------ bitbake/lib/toaster/toastergui/templates/base.html | 1 + .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++------------- bitbake/lib/toaster/toastergui/urls.py | 5 + 22 files changed, 507 insertions(+), 228 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py create mode 100644 bitbake/lib/toaster/toastergui/api.py create mode 100644 bitbake/lib/toaster/toastergui/static/js/mrbsection.js--
Regards,
Ed
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
Brian, a few comments on your comments...
Quoted 9 lines · Brian Avery
Hi,
I finally had a chance to try out the 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir' branch and I had comments:
1) use project name for build dirs? This would make it clearer which build dir has which projects in it.
This introduces the problem of what to do if a project's name changes, which could happen in the middle of a build; it would also need additional code to sanitise the project name to produce a valid directory name. It seems simpler to me to just use the build ID, as it's not a massive stretch to figure out which build goes with which numbered directory.
...
Quoted 6 lines
11) I’m not personally fond of having the sstate and
dl_dirbeing set on the project creation page. they are available in the bitbake var page for each project which is nice. This is more a Belén thing. just putting my opinion down.
After discussion on the list, Sujith has provided a version of Ed's branch which removes these fields from the new project page:
sujith/set_dldir_sstatedirv2
Elliot
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre
Quoted 8 lines · On 19/03/2016 23:13, "[email protected] on behalf of Brian…
Hi,
I finally had a chance to try out the 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir' branch and I had comments:
1) use project name for build dirs? This would make it clearer which build dir has which projects in it.
We had a discussion about this, part of which Elliot has summarised in his previous email. It is true that it might be easier to identify the build directory of a project if we use the project name, but Toaster does not enforce unique names, it does not validate for special characters or length limits, and it allows you change the project name at any time. The problems all these could cause made us go with the project id. To ease identification, we could show the project id and the path to the build directory in the project page, unless anybody has issues with displaying this information.
Quoted 7 lines
2) we cannot assume I am building in ~/poky/build. There are env variables to let us find where poky is $OEROOT for instance. My bin/sh is bash but I still got the following when building from $HOME/cow/test/ when poky was in $HOME/src/poky :
Quoted 6 lines
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
3) if a build fails (as above) and then I cancel it, my state stays in Cancelling the build ...
This sounds like a bug: you should never be able to cancel a failed build. The 'cancel' button should only be available while bitbake is running.
Quoted 4 lines
4) when I cancel a build it says ³Failed² it should probably say ³Canceled² and still be red.
This is also a bug. Cancelled builds have their own presentation (they say "cancelled"). See page 4 of this design document
https://bugzilla.yoctoproject.org/attachment.cgi?id=2706
Quoted 5 lines
5) If I change one of my base project settings (like my download folder) the next (and following) projects should probably use my last setting as the default rather than the hardcoded default.
I am not sure I agree. The UI right now provides only per project
configuration: I wouldn't make exceptions to that rule. If we did, you would be in a project context surrounded by variables that only apply to the selected project, and all of a sudden you would have 2 variables that look the same as all others but behave differently and override the Toaster defaults.
If you want to override the Toaster defaults, you should change the default value via the Django admin interface or in the toasterconf.json file. The UI only provides access to project configuration at this time ... consistently limited ;)
Quoted 8 lines
For instance I change my downloads from /home/Š/downloads to /home/Š./DL in cowMaster, when I make cowJethro, the default
DL_DIRshould be /home/Š./DLThe following were done based on Jethro:
Quoted 7 lines
6) built rpi-hwup-image - then made custom image based on it. CI interface says: ³Toaster has no package information for rpime. To generate package information, build rpime²
7) same true for customizing off off core-image-minimal. for both 6/7 machine = raspberrypi2.
I've seen this too, but I assumed is was caused by this:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9266
Although I could be completely wrong. There is a patch under review to fix it
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
But I haven't had a chance to try it on top of Ed's branch.
Quoted 19 lines
8) trying to download the recipe file gives me the yellow error screen [Errno 2] No such file or directory:
u'/recipes-core/images/rpi-hwup-image.bb'9) if I switch back to qemux86 as the machine, build core-image-minimal, and then customize it I also get the Toaster has no package information for cimx86me. To generate package information, build cimx86me
so it is not just due to it being a rpi image. this also gives [Errno 2] No such file or directory:
u'meta/recipes-core/images/core-image-minimal.bb'if it try to download it.
Quoted 8 lines
10) after building the cimx86me, I can now download the recipe but I still do not see any packages.
11) I¹m not personally fond of having the sstate and
dl_dirbeing set on the project creation page. they are available in the bitbake var page for each project which is nice. This is more a Belén thing. just putting my opinion down.
We also discussed this (everybody complained) and decided to remove them.
Quoted 127 lines
12) if I cancel a command line build from the command line (as the little question mark suggests) the cancellation is not reflected in the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
‹‹
So as far as upstreaming, I think we need to have the following 2 things fixed first:
1) being able to run/build from an arbitrary dir and not assuming we are in the poky tree. (should be ez)
2) fixing the custom image stuff. Since the CI is the main selling point of the 2.1 release we need to make sure we don¹t break it. (hopefully not too hard?)
Other issues can be made as bugs on top of the upstream, i think.
Hope this helps,
Brian
an Intel employeeQuoted 92 lines · Ed Bartosh
This is build cancelation functionality implemented by Sujith and Michael
and modified by me to work on top of 'per project build directory' patchset.NOTE: This patchset depends on 'per project build directory' patchset' v5:
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
The following changes since commit
e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00
+0200)are available in the git repository at:
git://git.yoctoproject.org/poky-contrib
ed/toaster/project-build-dir-7880-cancelhttp://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/proje
ct-build-dir-7880-cancelEd Bartosh (2):
toaster: fix conflicting migrations
toaster: use empty tokenMichael Wood (9):
toaster: Move xhr calls for starting and stopping builds
toaster: xhr Update the implementation of the build cancellation
request
toaster: libtoaster Update implementation of startABuild and
cancelABuild
toaster: bldcontrol models Add a cancelling state the BuildRequest
buildinfohelper: Add handler for cancelling a build
toaster:mrb_sectiontemplate Add build cancel button
toaster: models Exclude the CANCELLED builds fromget_number_of_builds
toaster: runbuilds Make runbuilds aware of the build CANCELLED state
toaster: runbuilds Clean up runbuildsSujith H (4):
toaster: bldcontrol Add forceShutDown function to BitbakeController
toaster: update BuildEnvironmentController and BitbakeController
toaster: models Add cancelled state to build outcome
toaster: tables BuildsTable exclude cancelled buildsbitbake/lib/bb/ui/buildinfohelper.py | 19 ++- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +--- .../toaster/bldcontrol/localhostbecontroller.py | 4 +- .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++----- .../migrations/0002_add_cancelling_state.py | 19 +++ .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++ bitbake/lib/toaster/bldcontrol/models.py | 27 +++- .../orm/migrations/0006_add_cancelled_state.py | 19 +++ bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++ bitbake/lib/toaster/orm/models.py | 12 +- bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++ .../toaster/toastergui/static/js/customrecipe.js | 4 +- .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +- .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++--- .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++ .../toaster/toastergui/static/js/projectpage.js | 4 +- .../toaster/toastergui/static/js/projecttopbar.js | 6 +- .../toaster/toastergui/static/js/recipedetails.js | 4 +- bitbake/lib/toaster/toastergui/tables.py | 48 +------ bitbake/lib/toaster/toastergui/templates/base.html | 1 + .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++------------- bitbake/lib/toaster/toastergui/urls.py | 5 + 22 files changed, 507 insertions(+), 228 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py create mode 100644 bitbake/lib/toaster/toastergui/api.py create mode 100644bitbake/lib/toaster/toastergui/static/js/mrbsection.js
--
Regards,
Ed
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toasterSignature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Quoted 8 lines · Brian Avery
2) we cannot assume I am building in ~/poky/build. There are env variables to let us find where poky is $OEROOT for instance. My bin/sh is bash but I still got the following when building from $HOME/cow/test/ when poky was in $HOME/src/poky :
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
Should be fixed by this commit:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir&id=a3566432d7d42939a482322abb7ead6026008d49
...
Quoted 4 lines
11) I’m not personally fond of having the sstate and
dl_dirbeing set on the project creation page. they are available in the bitbake var page for each project which is nice. This is more a Belén thing. just putting my opinion down.
This should be alread fixed by Sujith. The fix is also in ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
...
Quoted 5 lines
So as far as upstreaming, I think we need to have the following 2 things fixed first:
1) being able to run/build from an arbitrary dir and not assuming we are in the poky tree. (should be ez)
yes, it should be fixed already. see my comment above.
Quoted 3 lines
2) fixing the custom image stuff. Since the CI is the main selling point of the 2.1 release we need to make sure we don’t break it. (hopefully not too hard?)
OK, I'll try to look at it.
BTW, the full patchset has 42 patches and becomes hard to maintain. We may need to consider upstreaming it in 2 or 3 steps if the above fixes are hard to do.
Regards,
Ed
Quoted 4 lines · Brian Avery
a couple of other thoughts after looking at it a little more today:
1) pid of bitbake started in proj builds dir for cleanup? do we want to save these?
not until we use them.
Quoted 2 lines
bitbake.lock with lsof might mean we don't need these. however, having them might negate the lsof requirement...)
I'll look what I can do. I was planning to reuse what I did in my old ed/toaster/bec branch: to generate shell script for cloning all repos and running the build. It would be much more easy to deal with pid files in the shell script than in current python code. However, I think it's better not to do right now as we already have quite a bit of changes and very close to release.
Quoted 1 line
2)should poky oe-init be a #!/bin/bash if dash fails?
No, we shouldn't as it works with zsh.
Quoted 3 lines
3) toasterui event loop should probably check for liveness of server connection…. or do we get an error on waitEvent if the server has died (segv’d, kill -9 by user, etc)?
We run sever only during the build. We probably need to set build state to 'Failed' if server is killed or crashed.
Regards,
Ed
Hi,
Responding to comments on my comments:
Quoted 15 lines · Belen
Quoted 8 lines · On 19/03/2016 23:13, "[email protected] on behalf of Brian…
Hi,
I finally had a chance to try out the 'poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir' branch and I had comments:
1) use project name for build dirs? This would make it clearer which build dir has which projects in it.
Quoted 10 lines
We had a discussion about this, part of which Elliot has summarised in his previous email. It is true that it might be easier to identify the build directory of a project if we use the project name, but Toaster does not enforce unique names, it does not validate for special characters or length limits, and it allows you change the project name at any time. The problems all these could cause made us go with the project id. To ease identification, we could show the project id and the path to the build directory in the project page, unless anybody has issues with displaying this information.
That would work for me. I just wanted an ez way to tie them together and i feared the proj_id would not be clear to users.
Quoted 40 lines
Quoted 7 lines
2) we cannot assume I am building in ~/poky/build. There are env variables to let us find where poky is $OEROOT for instance. My bin/sh is bash but I still got the following when building from $HOME/cow/test/ when poky was in $HOME/src/poky :
Quoted 6 lines
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
3) if a build fails (as above) and then I cancel it, my state stays in Cancelling the build ...
This sounds like a bug: you should never be able to cancel a failed build. The 'cancel' button should only be available while bitbake is running.
Quoted 4 lines
4) when I cancel a build it says ³Failed² it should probably say ³Canceled² and still be red.
This is also a bug. Cancelled builds have their own presentation (they say "cancelled"). See page 4 of this design document
https://bugzilla.yoctoproject.org/attachment.cgi?id=2706
Quoted 5 lines
5) If I change one of my base project settings (like my download folder) the next (and following) projects should probably use my last setting as the default rather than the hardcoded default.
I am not sure I agree. The UI right now provides only per project
configuration: I wouldn't make exceptions to that rule. If we did, you would be in a project context surrounded by variables that only apply to the selected project, and all of a sudden you would have 2 variables that look the same as all others but behave differently and override the Toaster defaults.
Quoted 5 lines
If you want to override the Toaster defaults, you should change the default value via the Django admin interface or in the toasterconf.json file. The UI only provides access to project configuration at this time ... consistently limited ;)
fair point. and since Elliot said they were moving off the main proj page i withdraw the suggestion.
Quoted 18 lines
Quoted 8 lines
For instance I change my downloads from /home/Š/downloads to /home/Š./DL in cowMaster, when I make cowJethro, the default
DL_DIRshould be /home/Š./DLThe following were done based on Jethro:
Quoted 7 lines
6) built rpi-hwup-image - then made custom image based on it. CI interface says: ³Toaster has no package information for rpime. To generate package information, build rpime²
7) same true for customizing off off core-image-minimal. for both 6/7 machine = raspberrypi2.
I've seen this too, but I assumed is was caused by this:
Quoted 8 lines
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9266
Although I could be completely wrong. There is a patch under review to fix it
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
But I haven't had a chance to try it on top of Ed's branch.
Could be true; i hope so :).
Quoted 24 lines
Quoted 24 lines
8) trying to download the recipe file gives me the yellow error screen [Errno 2] No such file or directory:
u'/recipes-core/images/rpi-hwup-image.bb'9) if I switch back to qemux86 as the machine, build core-image-minimal, and then customize it I also get the Toaster has no package information for cimx86me. To generate package information, build cimx86me
so it is not just due to it being a rpi image. this also gives [Errno 2] No such file or directory:
u'meta/recipes-core/images/core-image-minimal.bb'if it try to download it.
10) after building the cimx86me, I can now download the recipe but I still do not see any packages.
Quoted 135 lines
Quoted 4 lines
11) I¹m not personally fond of having the sstate and
dl_dirbeing set on the project creation page. they are available in the bitbake var page for each project which is nice. This is more a Belén thing. just putting my opinion down.We also discussed this (everybody complained) and decided to remove them.
Quoted 127 lines
12) if I cancel a command line build from the command line (as the little question mark suggests) the cancellation is not reflected in the UI. Mine got stuck at 46% when I cancelled my dropbear cli build.
‹‹
So as far as upstreaming, I think we need to have the following 2 things fixed first:
1) being able to run/build from an arbitrary dir and not assuming we are in the poky tree. (should be ez)
2) fixing the custom image stuff. Since the CI is the main selling point of the 2.1 release we need to make sure we don¹t break it. (hopefully not too hard?)
Other issues can be made as bugs on top of the upstream, i think.
Hope this helps,
Brian
an Intel employeeQuoted 84 lines · Ed Bartosh
This is build cancelation functionality implemented by Sujith and Michael
and modified by me to work on top of 'per project build directory' patchset.NOTE: This patchset depends on 'per project build directory' patchset' v5:
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004170.html
The following changes since commit
e4ba8a50641b6a2ad8ccda5fc62c61fdf7cef733:toasterui: shutdown on BuildCompleted event (2016-03-17 17:31:00
+0200)are available in the git repository at:
git://git.yoctoproject.org/poky-contrib
ed/toaster/project-build-dir-7880-cancelhttp://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/proje
ct-build-dir-7880-cancelEd Bartosh (2):
toaster: fix conflicting migrations
toaster: use empty tokenMichael Wood (9):
toaster: Move xhr calls for starting and stopping builds
toaster: xhr Update the implementation of the build cancellation
request
toaster: libtoaster Update implementation of startABuild and
cancelABuild
toaster: bldcontrol models Add a cancelling state the BuildRequest
buildinfohelper: Add handler for cancelling a build
toaster:mrb_sectiontemplate Add build cancel button
toaster: models Exclude the CANCELLED builds fromget_number_of_builds
toaster: runbuilds Make runbuilds aware of the build CANCELLED state
toaster: runbuilds Clean up runbuildsSujith H (4):
toaster: bldcontrol Add forceShutDown function to BitbakeController
toaster: update BuildEnvironmentController and BitbakeController
toaster: models Add cancelled state to build outcome
toaster: tables BuildsTable exclude cancelled buildsbitbake/lib/bb/ui/buildinfohelper.py | 19 ++- bitbake/lib/toaster/bldcontrol/bbcontroller.py | 29 +--- .../toaster/bldcontrol/localhostbecontroller.py | 4 +- .../bldcontrol/management/commands/runbuilds.py | 107 ++++++++++----- .../migrations/0002_add_cancelling_state.py | 19 +++ .../toaster/bldcontrol/migrations/0003_merge.py | 15 +++ bitbake/lib/toaster/bldcontrol/models.py | 27 +++- .../orm/migrations/0006_add_cancelled_state.py | 19 +++ bitbake/lib/toaster/orm/migrations/0007_merge.py | 15 +++ bitbake/lib/toaster/orm/models.py | 12 +- bitbake/lib/toaster/toastergui/api.py | 110 +++++++++++++++ .../toaster/toastergui/static/js/customrecipe.js | 4 +- .../lib/toaster/toastergui/static/js/layerBtn.js | 3 +- .../lib/toaster/toastergui/static/js/libtoaster.js | 41 +++--- .../lib/toaster/toastergui/static/js/mrbsection.js | 95 +++++++++++++ .../toaster/toastergui/static/js/projectpage.js | 4 +- .../toaster/toastergui/static/js/projecttopbar.js | 6 +- .../toaster/toastergui/static/js/recipedetails.js | 4 +- bitbake/lib/toaster/toastergui/tables.py | 48 +------ bitbake/lib/toaster/toastergui/templates/base.html | 1 + .../toaster/toastergui/templates/mrb_section.html | 148 ++++++++------------- bitbake/lib/toaster/toastergui/urls.py | 5 + 22 files changed, 507 insertions(+), 228 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_add_cancelling_state.py create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_merge.py create mode 100644 bitbake/lib/toaster/orm/migrations/0006_add_cancelled_state.py create mode 100644 bitbake/lib/toaster/orm/migrations/0007_merge.py create mode 100644 bitbake/lib/toaster/toastergui/api.py create mode 100644bitbake/lib/toaster/toastergui/static/js/mrbsection.js
Quoted 8 lines
--
Regards,
Ed
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
-brian
an intel employee
Hi!
Quoted 7 lines · Ed Bartosh
Quoted 4 lines · Brian Avery
a couple of other thoughts after looking at it a little more today:
1) pid of bitbake started in proj builds dir for cleanup? do we want to save these?
not until we use them.
Quoted 9 lines
Quoted 2 lines
bitbake.lock with lsof might mean we don't need these. however, having them might negate the lsof requirement...)
I'll look what I can do. I was planning to reuse what I did in my old ed/toaster/bec branch: to generate shell script for cloning all repos and running the build. It would be much more easy to deal with pid files in the shell script than in current python code. However, I think it's better not to do right now as we already have quite a bit of changes and very close to release.
k. was more of a me musing about it than anything else...
Quoted 3 lines
Quoted 1 line
2)should poky oe-init be a #!/bin/bash if dash fails?
No, we shouldn't as it works with zsh.
k. btw, how did it fail? I just ran oe-init and bitbake core-image-minimal in an ubuntu:14.04 container and it seemed fine. /bin/sh was liked to dash...
Quoted 5 lines
Quoted 3 lines
3) toasterui event loop should probably check for liveness of server connection…. or do we get an error on waitEvent if the server has died (segv’d, kill -9 by user, etc)?
We run sever only during the build. We probably need to set build state to 'Failed' if server is killed or crashed.
yep. though it can easily be a follow on bug.
Quoted 3 lines
Regards,
Ed
My thoughts:
since Ed has fixed the ../bitbake/bin issue and if
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
fixes the CI not working I'm good with upstreaming and filing additional bugs for the corner cases since this fixes alot.
TY!
-brian
-an intel employee
Quoted 5 lines · On 21/03/2016 18:07, "[email protected] on behalf of Brian…
My thoughts:
since Ed has fixed the ../bitbake/bin issue and if
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
fixes the CI not working I'm good with upstreaming and filing additional bugs for the corner cases since this fixes alot.
I finally managed to look into
poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
but my looking was brief. I can't get any builds started from the web UI:
none of the 'build' buttons seem to work. I click and nothing happens.
I can build from the command line though, and the builds do show up in Toaster.
I wonder what I am doing wrong this time ...
Cheers
Belén
what happens when starting builds from toaster?
If you are seeing :
Quoted 3 lines
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
Then Ed says ->
Should be fixed by this commit:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir&id=a3566432d7d42939a482322abb7ead6026008d49
-b
...
Quoted 28 lines · Belen
Quoted 5 lines · On 21/03/2016 18:07, "[email protected] on behalf of Brian…
My thoughts:
since Ed has fixed the ../bitbake/bin issue and if
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
fixes the CI not working I'm good with upstreaming and filing additional bugs for the corner cases since this fixes alot.I finally managed to look into
poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
but my looking was brief. I can't get any builds started from the web UI:
none of the 'build' buttons seem to work. I click and nothing happens.I can build from the command line though, and the builds do show up in Toaster.
I wonder what I am doing wrong this time ...
Cheers
Belén
Quoted 11 lines · Brian Avery
what happens when starting builds from toaster?
If you are seeing :
Quoted 3 lines
: less toaster_ui.log
bash: ../bitbake/bin/bitbake: No such file or directory
Then Ed says ->
Should be fixed by this commit:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=ed/toast
er/project-build-dir-7880-cancel-dldir_sstatedir&id=a3566432d7d42939a482322abb7ead6026008d49
Builds are now working in the branch: I can start them and cancel them. Unfortunately, something has gone funny when applying Michael's patch for the image customisation problem, so that is not fixed yet. Ed is looking into it.
Thanks!
Belén
Quoted 36 lines
-b
...Quoted 30 lines · Belen
On 21/03/2016 18:07, "[email protected] on behalf of Brian
Avery" <[email protected] on behalf ofQuoted 5 lines · [email protected]> wrote:
My thoughts:
since Ed has fixed the ../bitbake/bin issue and if
https://lists.yoctoproject.org/pipermail/toaster/2016-March/004162.html
fixes the CI not working I'm good with upstreaming and filing additional bugs for the corner cases since this fixes alot.I finally managed to look into
poky-contrib/ed/toaster/project-build-dir-7880-cancel-dldir_sstatedir
but my looking was brief. I can't get any builds started from the web UI:
none of the 'build' buttons seem to work. I click and nothing happens.I can build from the command line though, and the builds do show up in Toaster.
I wonder what I am doing wrong this time ...
Cheers
Belén