Toaster patchset
cover Toaster patchset
From: Alexandru DAMIAN <[email protected]>
Hello,
This patchset is targeted at bringing critical fixes for M3 rc2 release, including multiple show stoppers. It also adds fixes for M3 features.
The patches have been tested and reviewed on the toaster mailing list.
Can you please pull before M3 rc2 cut-off ?
Thank you,
Alex
The following changes since commit 7182cdd3c4a534a87147bb0aad7b360ffef6426b:
bitbake-layers: fix wildcard support in remove-layer (2015-02-25 23:28:15 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib adamian/20150226-submission-bb
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=adamian/20150226-submission-bb
Alexandru DAMIAN (5):
toastergui: stop covered task graph traversal early
toasterui: fix sstate task identification
toaster: order layers with same priority
toasterui: do not use transactions if the database autocommits
toastergui: add csrf tokens for AJAX calls
Belen Barros Pena (1):
toaster: fix typo in models.py
David Reyna (2):
toaster: all projects data and sorts
toaster: config var layout fixes
lib/bb/ui/buildinfohelper.py | 24 +++-- lib/toaster/bldcontrol/models.py | 2 +- lib/toaster/orm/models.py | 80 +++++++++++++--- lib/toaster/toastergui/static/css/default.css | 2 +- lib/toaster/toastergui/templates/layers.html | 1 + .../toastergui/templates/managed_builds.html | 22 +++-- .../toastergui/templates/managed_mrb_section.html | 1 + lib/toaster/toastergui/templates/projectconf.html | 23 ++--- lib/toaster/toastergui/templates/projects.html | 60 +++++++++--- lib/toaster/toastergui/templates/targets.html | 1 + lib/toaster/toastergui/views.py | 104 ++++++++++++++------- 11 files changed, 229 insertions(+), 91 deletions(-)
Signature · 1 line
1.9.1
PATCH 1/8 toaster: fix typo in models.py
From: Belen Barros Pena <[email protected]>
The typo caused the tasks table to break when a task has no help text.
--- lib/toaster/orm/models.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/toaster/orm/models.py +1 -1
@@ -366,7 +366,7 @@ class Task(models.Model): return self._helptext try: self._helptext = HelpText.objects.get(key=self.task_name, area=HelpText.VARIABLE, build=self.build).text - except HelpText.DoesNotExit: + except HelpText.DoesNotExist: self._helptext = None return self._helptext
Signature · 1 line
1.9.1
PATCH 2/8 toastergui: stop covered task graph traversal early
From: Alexandru DAMIAN <[email protected]>
This patch stops the breadth-frist graph traversal for finding covering tasks after the finding the first level of executed tasks.
The reasoning is that the first level of executed tasks is the actually relevent information on how a task was covered.
--- lib/toaster/toastergui/views.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
lib/toaster/toastergui/views.py +3 -4
@@ -357,6 +357,8 @@ def generateCoveredList2( revlist = [] ): covered_list = [ x for x in revlist if x.outcome == Task.OUTCOME_COVERED ] while len(covered_list): revlist = [ x for x in revlist if x.outcome != Task.OUTCOME_COVERED ] + if len(revlist) > 0: + return revlist newlist = _find_task_revdep_list(covered_list)
@@ -379,10 +381,7 @@ def task( request, build_id, task_id ): coveredBy = ''; if ( task.outcome == Task.OUTCOME_COVERED ): # _list = generateCoveredList( task ) - _list = generateCoveredList2( _find_task_revdep( task ) ) - coveredBy = [ ] - for t in _list: - coveredBy.append( t ) + coveredBy = sorted(generateCoveredList2( _find_task_revdep( task ) ), key = lambda x: x.recipe.name) log_head = '' log_body = '' if task.outcome == task.OUTCOME_FAILED:
Signature · 1 line
1.9.1
PATCH 3/8 toasterui: fix sstate task identification
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
This patch fixes a problem where set sstate scene tasks were not identified, causing cache attempt not being recorded.
[YOCTO #7223]
--- lib/bb/ui/buildinfohelper.py | 4 ++-- lib/toaster/orm/models.py | 13 ++++--------- 2 files changed, 6 insertions(+), 11 deletions(-)
lib/bb/ui/buildinfohelper.py +2 -2
@@ -190,8 +190,8 @@ class ORMWrapper(object): vars(task_object)[v] = task_information[v] object_changed = True - # update setscene-related information if the task was just created - if created and task_object.outcome == Task.OUTCOME_COVERED and 1 == Task.objects.related_setscene(task_object).count(): + # update setscene-related information if the task has a setscene + if task_object.outcome == Task.OUTCOME_COVERED and 1 == task_object.get_related_setscene().count(): task_object.outcome = Task.OUTCOME_CACHED object_changed = True
lib/toaster/orm/models.py +4 -9
@@ -281,10 +281,6 @@ class Target_File(models.Model): sym_target = models.ForeignKey('Target_File', related_name="symlink_set", null=True) -class TaskManager(models.Manager): - def related_setscene(self, task_object): - return Task.objects.filter(task_executed=True, build = task_object.build, recipe = task_object.recipe, task_name=task_object.task_name+"_setscene") - class Task(models.Model): SSTATE_NA = 0
@@ -339,10 +335,8 @@ class Task(models.Model): search_allowed_fields = [ "recipe__name", "recipe__version", "task_name", "logfile" ] - objects = TaskManager() - def get_related_setscene(self): - return Task.objects.related_setscene(self) + return Task.objects.filter(task_executed=True, build = self.build, recipe = self.recipe, task_name=self.task_name+"_setscene") def get_outcome_text(self): return Task.TASK_OUTCOME[self.outcome + 1][1]
@@ -377,7 +371,7 @@ class Task(models.Model): outcome = models.IntegerField(choices=TASK_OUTCOME, default=OUTCOME_NA) sstate_checksum = models.CharField(max_length=100, blank=True) path_to_sstate_obj = models.FilePathField(max_length=500, blank=True) - recipe = models.ForeignKey('Recipe', related_name='build_recipe') + recipe = models.ForeignKey('Recipe', related_name='tasks') task_name = models.CharField(max_length=100) source_url = models.FilePathField(max_length=255, blank=True) work_directory = models.FilePathField(max_length=255, blank=True)
@@ -394,7 +388,8 @@ class Task(models.Model): sstate_text = property(get_sstate_text) def __unicode__(self): - return "%d %s:%s" % (self.id, self.recipe.name, self.task_name) + return "%d(%d) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name) + class Meta: ordering = ('order', 'recipe' ,) unique_together = ('build', 'recipe', 'task_name', )
Signature · 1 line
1.9.1
PATCH 4/8 toaster: order layers with same priority
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
When we have multiple layers with the same priority, we need to order them by date, newest first. The layer version ID is a complete ordering by recent-ness, so it is suitable.
--- lib/toaster/orm/models.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/toaster/orm/models.py +1 -1
@@ -994,7 +994,7 @@ class Layer_Version(models.Model): query |= Q(projectlayer__project=project) return sorted( - Layer_Version.objects.filter(layer__name = self.layer.name).filter(query).select_related('layer_source', 'layer'), + Layer_Version.objects.filter(layer__name = self.layer.name).filter(query).select_related('layer_source', 'layer').order_by("-id"), key = lambda x: _get_ls_priority(x.layer_source), reverse = True)
Signature · 1 line
1.9.1
PATCH 5/8 toasterui: do not use transactions if the database autocommits
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Some databases, notably the SQLite3 adapter, force autocommits even if the autocommit is turned off. The behavious is tracked in this bug:
http://bugs.python.org/issue8145#msg109965
Django refuses to work with autocommit off in this case, so we have to take the same precautions when using manual transaction support.
[YOCTO #7363]
[YOCTO #7365]
--- lib/bb/ui/buildinfohelper.py | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
lib/bb/ui/buildinfohelper.py +12 -8
@@ -35,7 +35,7 @@ from bb.msg import BBLogFormatter as format from django.db import models import logging -from django.db import transaction +from django.db import transaction, connection logger = logging.getLogger("BitBake")
@@ -609,7 +609,9 @@ class BuildInfoHelper(object): self.task_order = 0 self.autocommit_step = 1 self.server = server - transaction.set_autocommit(False) + # we use manual transactions if the database doesn't autocommit on us + if not connection.features.autocommits_when_autocommit_is_off: + transaction.set_autocommit(False) self.orm_wrapper = ORMWrapper() self.has_build_history = has_build_history self.tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
@@ -929,11 +931,12 @@ class BuildInfoHelper(object): task_information['outcome'] = Task.OUTCOME_FAILED del self.internal_state['taskdata'][identifier] - # we force a sync point here, to get the progress bar to show - if self.autocommit_step % 3 == 0: - transaction.set_autocommit(True) - transaction.set_autocommit(False) - self.autocommit_step += 1 + if not connection.features.autocommits_when_autocommit_is_off: + # we force a sync point here, to get the progress bar to show + if self.autocommit_step % 3 == 0: + transaction.set_autocommit(True) + transaction.set_autocommit(False) + self.autocommit_step += 1 self.orm_wrapper.get_update_task_object(task_information, True) # must exist
@@ -1207,4 +1210,5 @@ class BuildInfoHelper(object): for event in self.internal_state['backlog']: logger.error("UNSAVED log: %s", event.msg) - transaction.set_autocommit(True) + if not connection.features.autocommits_when_autocommit_is_off: + transaction.set_autocommit(True)
Signature · 1 line
1.9.1
PATCH 6/8 toastergui: add csrf tokens for AJAX calls
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
This patch adds CSRF tokens in pages using AJAX calls in order to force Django to generate the CSRF cookie used to authenticate the call.
--- lib/toaster/toastergui/templates/layers.html | 1 + lib/toaster/toastergui/templates/managed_mrb_section.html | 1 + lib/toaster/toastergui/templates/projectconf.html | 1 + lib/toaster/toastergui/templates/targets.html | 1 + 4 files changed, 4 insertions(+)
lib/toaster/toastergui/templates/layers.html +1 -0
@@ -117,6 +117,7 @@ var tooltipUpdateText; +/* ensure csrf cookie exists {% csrf_token %} */ function _makeXHREditCall(data, onsuccess, onfail) { $.ajax( { type: "POST",
lib/toaster/toastergui/templates/managed_mrb_section.html +1 -0
@@ -156,6 +156,7 @@ <script> +/* ensure csrf cookie exists {% csrf_token %} */ function _makeXHRBuildCall(url, data, onsuccess, onfail) { $.ajax( { type: "POST",
lib/toaster/toastergui/templates/projectconf.html +1 -0
@@ -429,6 +429,7 @@ // re-assert the event handlers } + /* ensure cookie exists {% csrf_token %} */ function postEditAjaxRequest(reqdata) { var ajax = $.ajax({ type:"POST",
lib/toaster/toastergui/templates/targets.html +1 -0
@@ -120,6 +120,7 @@ var tooltipUpdateText; +/* ensure csrf cookie exists {% csrf_token %} */ function _makeXHREditCall(data, onsuccess, onfail) { $.ajax( { type: "POST",
Signature · 1 line
1.9.1
PATCH 7/8 toaster: all projects data and sorts
In reply to Alex DAMIANFrom: David Reyna <[email protected]>
Implement the 'last build' data methods, enhance variable display, add empty page and empty sort support.
[YOCTO #6682]
--- lib/toaster/bldcontrol/models.py | 2 +- lib/toaster/orm/models.py | 63 ++++++++++++++ .../toastergui/templates/managed_builds.html | 22 ++--- lib/toaster/toastergui/templates/projects.html | 60 ++++++++++--- lib/toaster/toastergui/views.py | 97 +++++++++++++++------- 5 files changed, 190 insertions(+), 54 deletions(-)
lib/toaster/bldcontrol/models.py +1 -1
@@ -106,7 +106,7 @@ class BuildRequest(models.Model): (REQ_ARCHIVE, "archive"), ) - search_allowed_fields = ("brtarget__target",) + search_allowed_fields = ("brtarget__target", "build__project__name") project = models.ForeignKey(Project) build = models.OneToOneField(Build, null = True) # TODO: toasterui should set this when Build is created
lib/toaster/orm/models.py +63 -0
69 lines not on this page · raw message
lib/toaster/toastergui/templates/managed_builds.html +12 -10
@@ -56,6 +56,13 @@ </td> <td class="target">{% for t in build.target_set.all %} <a href="{% url "builddashboard" build.id %}"> {{t.target}} </a> <br />{% endfor %}</td> <td class="machine"><a href="{% url "builddashboard" build.id %}">{{build.machine}}</a></td> + {% if MANAGED %} + <td class="project"> + {% if build.project %} + <a href="{% url 'project' build.project.id %}">{{build.project.name}}</a> + {% endif %} + </td> + {% endif %} <td class="started_on"><a href="{% url "builddashboard" build.id %}">{{build.started_on|date:"d/m/y H:i"}}</a></td> <td class="completed_on"><a href="{% url "builddashboard" build.id %}">{{build.completed_on|date:"d/m/y H:i"}}</a></td> <td class="failed_tasks error">
@@ -91,13 +98,6 @@ <a href="{%url "builddashboard" build.id%}#images">{{fstypes|get_dict_value:build.id}}</a> {% endif %} </td> - {% if MANAGED %} - <td class="project"> - {% if build.project %} - <a href="{% url 'project' build.project.id %}">{{build.project.name}}</a> - {% endif %} - </td> - {% endif %} </tr>
@@ -114,6 +114,11 @@ <td class="machine"> <a href="{% url "buildrequestdetails" buildrequest.project.id buildrequest.id %}">{{buildrequest.machine}}</a> </td> + {% if MANAGED %} + <td class="project"> + <a href="{% url 'project' buildrequest.project.id %}">{{buildrequest.project.name}}</a> + </td> + {% endif %} <td class="started_on"> <a href="{% url "buildrequestdetails" buildrequest.project.id buildrequest.id %}">{{buildrequest.created|date:"d/m/y H:i"}}</a> </td>
@@ -132,9 +137,6 @@ </td> <td class="output"> {# we have no output here #} </td> - <td class="project"> - <a href="{% url 'project' buildrequest.project.id %}">{{buildrequest.project.name}}</a> - </td> </tr> {%endif%} {% endfor %}
lib/toaster/toastergui/templates/projects.html +48 -12
73 lines not on this page · raw message
lib/toaster/toastergui/views.py +66 -31
190 lines not on this page · raw message
Signature · 1 line
1.9.1
PATCH 8/8 toaster: config var layout fixes
In reply to Alex DAMIANFrom: David Reyna <[email protected]>
Block duplicates of custom variables regardless of case, fix spacing of delete icon, no extra error msg on blank DISTRO name
[YOCTO #7359]
--- lib/toaster/toastergui/static/css/default.css | 2 +- lib/toaster/toastergui/templates/projectconf.html | 22 +++++++++++----------- 2 files changed, 12 insertions(+), 12 deletions(-)
lib/toaster/toastergui/static/css/default.css +1 -1
@@ -151,7 +151,7 @@ select { width: auto; } #dependencies_modal_form { margin-bottom: 0px; } /* Configuration styles */ -.icon-trash { color: #B94A48; font-size: 16px; padding-left: 2px; } +.icon-trash { color: #B94A48; font-size: 16px; padding-left: 5px; } .icon-trash:hover { color: #943A38; text-decoration: none; cursor: pointer; } .icon-pencil, .icon-download-alt, .icon-refresh, .icon-star-empty, .icon-star, .icon-tasks { font-size: 16px; color: #0088CC; padding-left: 2px; } .icon-pencil:hover, .icon-download-alt:hover, .icon-refresh:hover, .icon-star-empty:hover, .icon-star:hover, .icon-tasks:hover { color: #005580; text-decoration: none; cursor: pointer; }
lib/toaster/toastergui/templates/projectconf.html +11 -11
@@ -23,12 +23,12 @@ <i class="icon-pencil" id="change-distro-icon"></i> <form id="change-distro-form" style="display:none;"> <div class="input-append"> - <span id="edit-distro-name-div" class="control-group"> - <input type="text" id="new-distro" value="{{distro}}"> - <button id="apply-change-distro" class="btn" type="button">Save</button> - <button id="cancel-change-distro" type="button" class="btn btn-link">Cancel</button> + <span id="edit-distro-name-div" class="control-group"> + <input type="text" id="new-distro" value="{{distro}}"> + <button id="apply-change-distro" class="btn" type="button">Save</button> + <button id="cancel-change-distro" type="button" class="btn btn-link">Cancel</button> + </span> <span class="help-block error" id="distro-error-message"></span> - </span> </div> </form> </dd>
@@ -52,7 +52,7 @@ <i class="icon-question-sign get-help" title="Specifies additional packages to install into an image. If your build creates more than one image, the packages will be installed in <strong>all of them</strong> <br /><a href='http://www.yoctoproject.org/docs/1.6.1/ref-manual/ref-manual.html#var-IMAGE_INSTALL' target='_blank'>Read more in the manual</a>"></i> </dt> <dd class="lead"> - <span id="image_install">{% if image_install_append %}{{image_install_append}}{%else%}<i>Not set</i>{%endif%}</span> + <span id="image_install"{% if image_install_append %}{%else%} class="muted"{%endif%}>{% if image_install_append %}{{image_install_append}}{%else%}Not set{%endif%}</span> <i class="icon-pencil" id="change-image_install-icon"></i> <i class="icon-trash" id="delete-image_install-icon" {% if image_install_append %}{%else%}style="display:none;"{%endif%}></i> <form id="change-image_install-form" style="display:none;">
@@ -198,14 +198,14 @@ var existing_configvars = document.getElementsByClassName('js-config-var-name'); for (var i = 0, length = existing_configvars.length; i < length; i++) { - if (existing_configvars[i].innerHTML == variable) { + if (existing_configvars[i].innerHTML.toUpperCase() == variable.toUpperCase()) { error_msg = "This variable is already set in this page, edit its value instead"; } } var blacklist_configvars = document.getElementsByClassName('js-config-blacklist-name'); for (var i = 0, length = blacklist_configvars.length; i < length; i++) { - if (blacklist_configvars[i].value == variable.toUpperCase()) { + if (blacklist_configvars[i].value.toUpperCase() == variable.toUpperCase()) { error_msg = "You cannot edit this variable in Toaster because it is set by the build servers"; } }
@@ -257,7 +257,7 @@ if (has_spaces) { error_msg = "A valid distro name cannot include spaces"; } else if (0 == value.length) { - error_msg = "A valid distro name cannot be empty"; + error_msg = " "; } if ("" != error_msg) {
@@ -609,7 +609,7 @@ var name = $('#new-image_install').val(); postEditAjaxRequest({"configvarChange" : 'IMAGE_INSTALL_append:'+name}); $('#image_install').text(name); - $('#image_install').parent().removeClass('muted'); + $('#image_install').removeClass('muted'); $("#change-image_install-form").slideUp(function () { $('#image_install, #change-image_install-icon').show(); if (name.length > -1) {
@@ -624,7 +624,7 @@ $(this).tooltip('hide'); postEditAjaxRequest({"configvarChange" : 'IMAGE_INSTALL_append:'+''}); $('#image_install').parent().fadeOut(1000, function(){ - $('#image_install').parent().addClass('muted'); + $('#image_install').addClass('muted'); $('#image_install').text('Not set'); $('#delete-image_install-icon').hide(); $('#image_install').parent().fadeIn(1000);
Signature · 1 line
1.9.1