Convert builds/, projects/ and projectbuilds/ to ToasterTable
cover Convert builds/, projects/ and projectbuilds/ to ToasterTable
Convert some of the most important Toaster pages to ToasterTable rather than individual templates.
This involved a rewrite of the ToasterTable filter system (see tablefilter.py) to cope with date range filtering for the builds tables.
I also moved a lot of code out of the views into the model, where it seems better placed.
I also fixed a few JS issues which arose while testing, such as filters not de-selecting when there are multiple filters on a table; event handlers being lost when repopulating a table; and incorrect hiding of page elements due to table classes being used for the page title element.
Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8738
Also incidentally fixes https://bugzilla.yoctoproject.org/show_bug.cgi?id=8792,
as this functionality works correctly in ToasterTable.
Changes since 6d35574 (toaster-next) are ingit://git.yoctoproject.org/poky-contrib, elliot/toaster/tables-8738
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/tables-8738
Elliot Smith (17):
toaster: toastergui Use ToasterTable for projects page
toaster: Move image file suffix list to model
toaster: Check inferred file suffixes against list of known types
toaster: toastergui Switch projects/ view to ToasterTable
toaster: toastergui Use event delegates for hover help elements
toaster: toastergui Convert all builds page to ToasterTable
toaster: toastergui Add QuerysetFilter class
toaster: toastergui Refactor ToasterTable filtering
toaster: toastergui Switch off filter highlights when inactive
toaster: toastergui Show recent builds on all builds page
toaster: toastergui Implement date range filters for builds
toaster: toastergui Implement "today" and "yesterday" filters
toaster: toastergui Convert project builds page to ToasterTable
toaster: tests Fix Django tests for new ToasterTable pages
toaster: toastergui Don't hide all elements with .col class
toaster: toastergui Ensure filter_value updates
toaster: toastergui Streamline construction of filter objects
bitbake/lib/toaster/orm/models.py | 109 ++- .../lib/toaster/toastergui/static/js/libtoaster.js | 6 +- .../toaster/toastergui/static/js/projecttopbar.js | 9 + bitbake/lib/toaster/toastergui/static/js/table.js | 272 +++++-- bitbake/lib/toaster/toastergui/tablefilter.py | 271 +++++++ bitbake/lib/toaster/toastergui/tables.py | 826 +++++++++++++++++++-- .../toastergui/templates/baseprojectpage.html | 1 + .../toastergui/templates/builds-toastertable.html | 48 ++ .../toaster/toastergui/templates/mrb_section.html | 4 +- .../templates/projectbuilds-toastertable.html | 56 ++ .../templates/projects-toastertable.html | 36 + .../toaster/toastergui/templates/toastertable.html | 7 +- bitbake/lib/toaster/toastergui/tests.py | 255 ++++--- bitbake/lib/toaster/toastergui/urls.py | 14 +- bitbake/lib/toaster/toastergui/views.py | 79 +- bitbake/lib/toaster/toastergui/widgets.py | 94 ++- 16 files changed, 1781 insertions(+), 306 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/tablefilter.py create mode 100644 bitbake/lib/toaster/toastergui/templates/builds-toastertable.html create mode 100644 bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html create mode 100644 bitbake/lib/toaster/toastergui/templates/projects-toastertable.html
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 1/17 toaster: toastergui Use ToasterTable for projects page
The projects page uses the old approach for showing tables, which means a template for each table. This means that applying changes to ToasterTable (which is used for most tables) has no effect on the layout, styling and behaviour for these older tables, and requires additional duplicated effort.
Move the projects page to use ToasterTable instead, to remove the duplication of effort.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/tables.py | 226 ++++++++++++++++++++- .../templates/projects-toastertable.html | 36 ++++ bitbake/lib/toaster/toastergui/urls.py | 6 + 3 files changed, 265 insertions(+), 3 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/templates/projects-toastertable.html
bitbake/lib/toaster/toastergui/tables.py +223 -3
@@ -21,13 +21,12 @@ from toastergui.widgets import ToasterTable from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project -from orm.models import CustomImageRecipe, Package -from django.db.models import Q, Max +from orm.models import CustomImageRecipe, Package, Build +from django.db.models import Q, Max, Count from django.conf.urls import url from django.core.urlresolvers import reverse from django.views.generic import TemplateView - class ProjectFiltersMixin(object): """Common mixin for recipe, machine in project filters"""
@@ -633,3 +632,224 @@ class SelectPackagesTable(ToasterTable): "the package content of you custom image", static_data_name="add_rm_pkg_btn", static_data_template='{% include "pkg_add_rm_btn.html" %}') + +class ProjectsTable(ToasterTable): + """Table of projects in Toaster""" + + def __init__(self, *args, **kwargs): + super(ProjectsTable, self).__init__(*args, **kwargs) + self.default_orderby = 'updated' + self.title = 'All projects' + self.static_context_extra['Build'] = Build + + def get_context_data(self, **kwargs): + return super(ProjectsTable, self).get_context_data(**kwargs) + + def setup_queryset(self, *args, **kwargs): + queryset = Project.objects.all() + + # annotate each project with its number of builds + queryset = queryset.annotate(num_builds=Count('build')) + + # exclude the command line builds project if it has no builds + q_default_with_builds = Q(is_default=True) & Q(num_builds__gt=0) + queryset = queryset.filter(Q(is_default=False) | + q_default_with_builds) + + # order rows + queryset = queryset.order_by(self.default_orderby) + + self.queryset = queryset + + # columns: last activity on (updated) - DEFAULT, project (name), release, machine, number of builds, last build outcome, recipe (name), errors, warnings, image files + def setup_columns(self, *args, **kwargs): + name_template = ''' + {% load project_url_tag %} + <span data-project-field="name"> + <a href="{% project_url data %}"> + {{data.name}} + </a> + </span> + ''' + + last_activity_on_template = ''' + {% load project_url_tag %} + <span data-project-field="updated"> + <a href="{% project_url data %}"> + {{data.updated | date:"d/m/y H:i"}} + </a> + </span> + ''' + + release_template = ''' + <span data-project-field="release"> + {% if data.release %} + <a href="{% url 'project' data.id %}#project-details"> + {{data.release.name}} + </a> + {% elif data.is_default %} + <span class="muted">Not applicable</span> + <i class="icon-question-sign get-help hover-help" + data-original-title="This project does not have a release set. + It simply collects information about the builds you start from + the command line while Toaster is running" + style="visibility: hidden;"> + </i> + {% else %} + No release available + {% endif %} + </span> + ''' + + machine_template = ''' + <span data-project-field="machine"> + {% if data.is_default %} + <span class="muted">Not applicable</span> + <i class="icon-question-sign get-help hover-help" + data-original-title="This project does not have a machine + set. It simply collects information about the builds you + start from the command line while Toaster is running" + style="visibility: hidden;"></i> + {% else %} + <a href="{% url 'project' data.id %}#machine-distro"> + {{data.get_current_machine_name}} + </a> + {% endif %} + </span> + ''' + + number_of_builds_template = ''' + {% if data.get_number_of_builds > 0 %} + <a href="{% url 'projectbuilds' data.id %}"> + {{data.get_number_of_builds}} + </a> + {% else %} + <span class="muted">0</span> + {% endif %} + ''' + + last_build_outcome_template = ''' + {% if data.get_number_of_builds > 0 %} + <a href="{% url 'builddashboard' data.get_last_build_id %}"> + {% if data.get_last_outcome == extra.Build.SUCCEEDED %} + <i class="icon-ok-sign success"></i> + {% elif data.get_last_outcome == extra.Build.FAILED %} + <i class="icon-minus-sign error"></i> + {% endif %} + </a> + {% endif %} + ''' + + recipe_template = ''' + {% if data.get_number_of_builds > 0 %} + <a href="{% url "builddashboard" data.get_last_build_id %}"> + {{data.get_last_target}} + </a> + {% endif %} + ''' + + errors_template = ''' + {% if data.get_number_of_builds > 0 %} + <a class="errors.count error" + href="{% url "builddashboard" data.get_last_build_id %}#errors"> + {{data.get_last_errors}} error{{data.get_last_errors | pluralize}} + </a> + {% endif %} + ''' + + warnings_template = ''' + {% if data.get_number_of_builds > 0 %} + <a class="warnings.count warning" + href="{% url "builddashboard" data.get_last_build_id %}#warnings"> + {{data.get_last_warnings}} warning{{data.get_last_warnings | pluralize}} + </a> + {% endif %} + ''' + + image_files_template = ''' + {% load projecttags %} + {% if data.get_number_of_builds > 0 and data.get_last_outcome == extra.Build.SUCCEEDED %} + <a href="{% url "builddashboard" data.get_last_build_id %}#images"> + {{fstypes | get_dict_value:data.id}} + </a> + {% endif %} + ''' + + self.add_column(title='Project', + hideable=False, + orderable=True, + static_data_name='name', + static_data_template=name_template) + + self.add_column(title='Last activity on', + help_text='Starting date and time of the \ + last project build. If the project has no \ + builds, this shows the date the project was \ + created.', + hideable=True, + orderable=True, + static_data_name='updated', + static_data_template=last_activity_on_template) + + self.add_column(title='Release', + help_text='The version of the build system used by \ + the project', + hideable=False, + orderable=True, + static_data_name='release', + static_data_template=release_template) + + self.add_column(title='Machine', + help_text='The hardware currently selected for the \ + project', + hideable=False, + orderable=False, + static_data_name='machine', + static_data_template=machine_template) + + self.add_column(title='Number of builds', + help_text='The number of builds which have been run \ + for the project', + hideable=True, + orderable=False, + static_data_name='number_of_builds', + static_data_template=number_of_builds_template) + + self.add_column(title='Last build outcome', + help_text='Indicates whether the last project build \ + completed successfully or failed', + hideable=True, + orderable=False, + static_data_name='last_build_outcome', + static_data_template=last_build_outcome_template) + + self.add_column(title='Recipe', + help_text='The last recipe which was built in this \ + project', + hideable=True, + orderable=False, + static_data_name='recipe_name', + static_data_template=recipe_template) + + self.add_column(title='Errors', + help_text='The number of errors encountered during \ + the last project build (if any)', + hideable=True, + orderable=False, + static_data_name='errors', + static_data_template=errors_template) + + self.add_column(title='Warnings', + help_text='The number of warnings encountered during \ + the last project build (if any)', + hideable=True, + orderable=False, + static_data_name='warnings', + static_data_template=warnings_template) + + self.add_column(title='Image files', + help_text='', + hideable=True, + orderable=False, + static_data_name='image_files', + static_data_template=image_files_template)
bitbake/lib/toaster/toastergui/templates/projects-toastertable.html new file +36 -0
@@ -0,0 +1,36 @@ +{% extends 'base.html' %} + +{% block title %} All projects - Toaster {% endblock %} + +{% block pagecontent %} + + <div class="page-header top-air"> + <h1 data-role="page-title"></h1> + </div> + + {% url 'projects' as xhr_table_url %} + {% include 'toastertable.html' %} + + <script> + $(document).ready(function () { + var tableElt = $("#{{table_name}}"); + var titleElt = $("[data-role='page-title']"); + + tableElt.on("table-done", function (e, total, tableParams) { + var title = "All projects"; + + if (tableParams.search || tableParams.filter) { + if (total === 0) { + title = "No projects found"; + } + else if (total > 0) { + title = total + " project" + (total > 1 ? 's' : '') + " found"; + } + } + + titleElt.text(title); + }); + }); + </script> + +{% endblock %}
bitbake/lib/toaster/toastergui/urls.py +6 -0
@@ -75,8 +75,14 @@ urlpatterns = patterns('toastergui.views', url(r'^newproject/$', 'newproject', name='newproject'), + # TODO remove when new toaster table is ready url(r'^projects/$', 'projects', name='all-projects'), + # TODO move to /projects/ when new toaster table is ready + url(r'^projects-new/$', + tables.ProjectsTable.as_view(template_name="projects-toastertable.html"), + name='all-projects-new'), + url(r'^project/(?P<pid>\d+)/$', 'project', name='project'), url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'), url(r'^project/(?P<pid>\d+)/builds/$', 'projectbuilds', name='projectbuilds'),
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 2/17 toaster: Move image file suffix list to model
In reply to Elliot SmithImage file suffixes are used in the project configuration page to show a list of available image file types. This list is stored as a function in the views code.
However, this list is also needed when parsing image file paths, so that the suffixes can be shown in the "all builds" and "project builds" tables.
Move the list of valid image file suffixes to the Target_Image_File class to make is accessible in other places where it may be needed.
[YOCTO #8738]
--- bitbake/lib/toaster/orm/models.py | 9 +++++++++ bitbake/lib/toaster/toastergui/views.py | 7 +------ 2 files changed, 10 insertions(+), 6 deletions(-)
bitbake/lib/toaster/orm/models.py +9 -0
@@ -461,6 +461,15 @@ class Target(models.Model): return self.target class Target_Image_File(models.Model): + # valid suffixes for image files produced by a build + SUFFIXES = { + 'btrfs', 'cpio', 'cpio.gz', 'cpio.lz4', 'cpio.lzma', 'cpio.xz', + 'cramfs', 'elf', 'ext2', 'ext2.bz2', 'ext2.gz', 'ext2.lzma', 'ext4', + 'ext4.gz', 'ext3', 'ext3.gz', 'hddimg', 'iso', 'jffs2', 'jffs2.sum', + 'squashfs', 'squashfs-lzo', 'squashfs-xz', 'tar.bz2', 'tar.lz4', + 'tar.xz', 'tartar.gz', 'ubi', 'ubifs', 'vmdk' + } + target = models.ForeignKey(Target) file_name = models.FilePathField(max_length=254) file_size = models.IntegerField()
bitbake/lib/toaster/toastergui/views.py +1 -6
@@ -2808,12 +2808,7 @@ if True: 'all_proxy','ftp_proxy','http_proxy ','https_proxy' } - vars_fstypes = { - 'btrfs','cpio','cpio.gz','cpio.lz4','cpio.lzma','cpio.xz','cramfs', - 'elf','ext2','ext2.bz2','ext2.gz','ext2.lzma', 'ext4', 'ext4.gz', 'ext3','ext3.gz','hddimg', - 'iso','jffs2','jffs2.sum','squashfs','squashfs-lzo','squashfs-xz','tar.bz2', - 'tar.lz4','tar.xz','tartar.gz','ubi','ubifs','vmdk' - } + vars_fstypes = Target_Image_File.SUFFIXES return(vars_managed,sorted(vars_fstypes),vars_blacklist)
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 3/17 toaster: Check inferred file suffixes against list of known types
In reply to Elliot SmithThe algorithm for finding the suffix for image files produced by the build doesn't reference a list of known file suffixes, so could be prone to error.
Modify how file suffixes are parsed from the file path so that they are compared against a list of known types; if this fails, use the part of the basename of the file path after the first '.' character.
Also rationalise the places in the views code where we extract the file name extensions for builds, so they both use the same algorithm (before, the same code was duplicated in two places).
[YOCTO #8417]
--- bitbake/lib/toaster/orm/models.py | 68 ++++++++++++++++++++++++++++++-- bitbake/lib/toaster/toastergui/tables.py | 10 +++-- bitbake/lib/toaster/toastergui/views.py | 38 ++---------------- 3 files changed, 75 insertions(+), 41 deletions(-)
bitbake/lib/toaster/orm/models.py +65 -3
@@ -29,6 +29,9 @@ from django.core import validators from django.conf import settings import django.db.models.signals +import os.path +import re + import logging logger = logging.getLogger("toaster")
@@ -234,6 +237,14 @@ class Project(models.Model): except (Build.DoesNotExist,IndexError): return( "not_found" ) + def get_last_build_extensions(self): + """ + Get list of file name extensions for images produced by the most + recent build + """ + last_build = Build.objects.get(pk = self.get_last_build_id()) + return last_build.get_image_file_extensions() + def get_last_imgfiles(self): build_id = self.get_last_build_id if (-1 == build_id):
@@ -376,6 +387,57 @@ class Build(models.Model): eta += ((eta - self.started_on)*(100-completeper))/completeper return eta + def get_image_file_extensions(self): + """ + Get list of file name extensions for images produced by this build + """ + targets = Target.objects.filter(build_id = self.id) + extensions = [] + + # pattern to match against file path for building extension string + pattern = re.compile('\.([^\.]+?)$') + + for target in targets: + if (not target.is_image): + continue + + target_image_files = Target_Image_File.objects.filter(target_id = target.id) + + for target_image_file in target_image_files: + file_name = os.path.basename(target_image_file.file_name) + suffix = '' + + continue_matching = True + + # incrementally extract the suffix from the file path, + # checking it against the list of valid suffixes at each + # step; if the path is stripped of all potential suffix + # parts without matching a valid suffix, this returns all + # characters after the first '.' in the file name + while continue_matching: + matches = pattern.search(file_name) + + if None == matches: + continue_matching = False + suffix = re.sub('^\.', '', suffix) + continue + else: + suffix = matches.group(1) + suffix + + if suffix in Target_Image_File.SUFFIXES: + continue_matching = False + continue + else: + # reduce the file name and try to find the next + # segment from the path which might be part + # of the suffix + file_name = re.sub('.' + matches.group(1), '', file_name) + suffix = '.' + suffix + + if not suffix in extensions: + extensions.append(suffix) + + return ', '.join(extensions) def get_sorted_target_list(self): tgts = Target.objects.filter(build_id = self.id).order_by( 'target' );
@@ -418,7 +480,7 @@ class Build(models.Model): return self.get_outcome_text() def __str__(self): - return "%s %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()])) + return "%d %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()])) # an Artifact is anything that results from a Build, and may be of interest to the user, and is not stored elsewhere
@@ -609,7 +671,7 @@ class Task(models.Model): sstate_text = property(get_sstate_text) def __unicode__(self): - return "%s(%s) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name) + return "%d(%d) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name) class Meta: ordering = ('order', 'recipe' ,)
@@ -1260,7 +1322,7 @@ class Layer_Version(models.Model): return sorted(result, key=lambda x: x.layer.name) def __unicode__(self): - return "%s %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project") + return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project") class Meta: unique_together = ("layer_source", "up_id")
bitbake/lib/toaster/toastergui/tables.py +6 -4
@@ -661,7 +661,9 @@ class ProjectsTable(ToasterTable): self.queryset = queryset - # columns: last activity on (updated) - DEFAULT, project (name), release, machine, number of builds, last build outcome, recipe (name), errors, warnings, image files + # columns: last activity on (updated) - DEFAULT, project (name), release, + # machine, number of builds, last build outcome, recipe (name), errors, + # warnings, image files def setup_columns(self, *args, **kwargs): name_template = ''' {% load project_url_tag %}
@@ -767,10 +769,9 @@ class ProjectsTable(ToasterTable): ''' image_files_template = ''' - {% load projecttags %} {% if data.get_number_of_builds > 0 and data.get_last_outcome == extra.Build.SUCCEEDED %} <a href="{% url "builddashboard" data.get_last_build_id %}#images"> - {{fstypes | get_dict_value:data.id}} + {{data.get_last_build_extensions}} </a> {% endif %} '''
@@ -848,7 +849,8 @@ class ProjectsTable(ToasterTable): static_data_template=warnings_template) self.add_column(title='Image files', - help_text='', + help_text='The root file system types produced by \ + the last project build', hideable=True, orderable=False, static_data_name='image_files',
bitbake/lib/toaster/toastergui/views.py +4 -34
@@ -2008,23 +2008,10 @@ if True: context_date,today_begin,yesterday_begin = _add_daterange_context(queryset_all, request, {'started_on','completed_on'}) # set up list of fstypes for each build - fstypes_map = {}; + fstypes_map = {} + for build in build_info: - targets = Target.objects.filter( build_id = build.id ) - comma = ""; - extensions = ""; - for t in targets: - if ( not t.is_image ): - continue - tif = Target_Image_File.objects.filter( target_id = t.id ) - for i in tif: - s=re.sub('.*tar.bz2', 'tar.bz2', i.file_name) - if s == i.file_name: - s=re.sub('.*\.', '', i.file_name) - if None == re.search(s,extensions): - extensions += comma + s - comma = ", " - fstypes_map[build.id]=extensions + fstypes_map[build.id] = build.get_image_file_extensions() # send the data to the template context = {
@@ -3047,24 +3034,7 @@ if True: # translate the project's build target strings fstypes_map = {}; for project in project_info: - try: - targets = Target.objects.filter( build_id = project.get_last_build_id() ) - comma = ""; - extensions = ""; - for t in targets: - if ( not t.is_image ): - continue - tif = Target_Image_File.objects.filter( target_id = t.id ) - for i in tif: - s=re.sub('.*tar.bz2', 'tar.bz2', i.file_name) - if s == i.file_name: - s=re.sub('.*\.', '', i.file_name) - if None == re.search(s,extensions): - extensions += comma + s - comma = ", " - fstypes_map[project.id]=extensions - except (Target.DoesNotExist,IndexError): - fstypes_map[project.id]=project.get_last_imgfiles + fstypes_map[project.id] = project.get_last_build_extensions() context = { 'mru' : build_mru,
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 4/17 toaster: toastergui Switch projects/ view to ToasterTable
In reply to Elliot SmithRemove the old projects page and replace with the new ToasterTable-based version.
NB although the projects.html template is no longer required, it's been left in as there will be changes applied to it for the new theme. These changes will have to then be transferred from the projects.html template to projects-toastertable.html.
Similarly, the code for the projects page in views.py has been retained.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/urls.py | 9 ++------- bitbake/lib/toaster/toastergui/views.py | 24 ++++++++++++++++++++---- 2 files changed, 22 insertions(+), 11 deletions(-)
bitbake/lib/toaster/toastergui/urls.py +2 -7
@@ -74,14 +74,9 @@ urlpatterns = patterns('toastergui.views', # project URLs url(r'^newproject/$', 'newproject', name='newproject'), - - # TODO remove when new toaster table is ready - url(r'^projects/$', 'projects', name='all-projects'), - - # TODO move to /projects/ when new toaster table is ready - url(r'^projects-new/$', + url(r'^projects/$', tables.ProjectsTable.as_view(template_name="projects-toastertable.html"), - name='all-projects-new'), + name='all-projects'), url(r'^project/(?P<pid>\d+)/$', 'project', name='project'), url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'),
bitbake/lib/toaster/toastergui/views.py +20 -4
@@ -2990,9 +2990,7 @@ if True: } return render(request, "unavailable_artifact.html", context) - - - + """ @_template_renderer("projects.html") def projects(request): (pagesize, orderby) = _get_parameters_values(request, 10, 'updated:-')
@@ -3034,7 +3032,24 @@ if True: # translate the project's build target strings fstypes_map = {}; for project in project_info: - fstypes_map[project.id] = project.get_last_build_extensions() + try: + targets = Target.objects.filter( build_id = project.get_last_build_id() ) + comma = ""; + extensions = ""; + for t in targets: + if ( not t.is_image ): + continue + tif = Target_Image_File.objects.filter( target_id = t.id ) + for i in tif: + s=re.sub('.*tar.bz2', 'tar.bz2', i.file_name) + if s == i.file_name: + s=re.sub('.*\.', '', i.file_name) + if None == re.search(s,extensions): + extensions += comma + s + comma = ", " + fstypes_map[project.id]=extensions + except (Target.DoesNotExist,IndexError): + fstypes_map[project.id]=project.get_last_imgfiles context = { 'mru' : build_mru,
@@ -3092,3 +3107,4 @@ if True: _set_parameters_values(pagesize, orderby, request) return context + """
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 5/17 toaster: toastergui Use event delegates for hover help elements
In reply to Elliot Smithlibtoaster.js binds to hover help elements via their hover() and mouseout() methods. However, any elements added to the DOM after libtoaster has initialised will not have these bindings added. This causes a problem for ToasterTables which have hover-help elements (e.g. the builds/ table).
Use the on() method instead. This uses event delegation to bind a handler to any th or td elements already in the DOM, or which will be added to the DOM in future. ToasterTables can now reconstruct the table DOM and still have the correct handlers attached once the table is done.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/static/js/libtoaster.js | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
bitbake/lib/toaster/toastergui/static/js/libtoaster.js +4 -2
@@ -448,10 +448,12 @@ $(document).ready(function() { // show help bubble only on hover inside tables $(".hover-help").css("visibility","hidden"); - $("th, td").hover(function () { + + $("table").on("mouseover", "th, td", function () { $(this).find(".hover-help").css("visibility","visible"); }); - $("th, td").mouseleave(function () { + + $("table").on("mouseleave", "th, td", function () { $(this).find(".hover-help").css("visibility","hidden"); });
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 6/17 toaster: toastergui Convert all builds page to ToasterTable
In reply to Elliot SmithFor better long-term maintainability, use ToasterTable instead of Django template and view code to display the all builds page.
NB the builds.html template has been left in, as this will otherwise cause conflicts when merging the new theme.
[YOCTO #8738]
--- bitbake/lib/toaster/orm/models.py | 32 ++- bitbake/lib/toaster/toastergui/tables.py | 311 ++++++++++++++++++++- .../toastergui/templates/builds-toastertable.html | 62 ++++ bitbake/lib/toaster/toastergui/urls.py | 5 +- bitbake/lib/toaster/toastergui/views.py | 32 --- 5 files changed, 407 insertions(+), 35 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/templates/builds-toastertable.html
bitbake/lib/toaster/orm/models.py +31 -1
@@ -447,6 +447,12 @@ class Build(models.Model): return Build.BUILD_OUTCOME[int(self.outcome)][1] @property + def failed_tasks(self): + """ Get failed tasks for the build """ + tasks = self.task_build.all() + return tasks.filter(order__gt=0, outcome=Task.OUTCOME_FAILED) + + @property def errors(self): return (self.logmessage_set.filter(level=LogMessage.ERROR) | self.logmessage_set.filter(level=LogMessage.EXCEPTION) |
@@ -457,8 +463,32 @@ class Build(models.Model): return self.logmessage_set.filter(level=LogMessage.WARNING) @property + def timespent(self): + return self.completed_on - self.started_on + + @property def timespent_seconds(self): - return (self.completed_on - self.started_on).total_seconds() + return self.timespent.total_seconds() + + @property + def target_labels(self): + """ + Sorted (a-z) "target1:task, target2, target3" etc. string for all + targets in this build + """ + targets = self.target_set.all() + target_labels = [] + target_label = None + + for target in targets: + target_label = target.target + if target.task: + target_label = target_label + ':' + target.task + target_labels.append(target_label) + + target_labels.sort() + + return target_labels def get_current_status(self): """
bitbake/lib/toaster/toastergui/tables.py +310 -1
320 lines not on this page · raw message
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html new file +62 -0
62 lines not on this page · raw message
bitbake/lib/toaster/toastergui/urls.py +4 -1
@@ -27,7 +27,10 @@ urlpatterns = patterns('toastergui.views', # landing page url(r'^landing/$', 'landing', name='landing'), - url(r'^builds/$', 'builds', name='all-builds'), + url(r'^builds/$', + tables.BuildsTable.as_view(template_name="builds-toastertable.html"), + name='all-builds'), + # build info navigation url(r'^build/(?P<build_id>\d+)$', 'builddashboard', name="builddashboard"),
bitbake/lib/toaster/toastergui/views.py +0 -32
@@ -1915,34 +1915,6 @@ if True: ''' The exception raised on invalid POST requests ''' pass - # shows the "all builds" page for managed mode; it displays build requests (at least started!) instead of actual builds - # WARNING _build_list_helper() may raise a RedirectException, which - # will set the GET parameters and redirect back to the - # all-builds or projectbuilds page as appropriate; - # TODO don't use exceptions to control program flow - @_template_renderer("builds.html") - def builds(request): - # define here what parameters the view needs in the GET portion in order to - # be able to display something. 'count' and 'page' are mandatory for all views - # that use paginators. - - queryset = Build.objects.all() - - redirect_page = resolve(request.path_info).url_name - - context, pagesize, orderby = _build_list_helper(request, - queryset, - redirect_page) - # all builds page as a Project column - context['tablecols'].append({ - 'name': 'Project', - 'clclass': 'project_column' - }) - - _set_parameters_values(pagesize, orderby, request) - return context - - # helper function, to be used on "all builds" and "project builds" pages def _build_list_helper(request, queryset_all, redirect_page, pid=None): default_orderby = 'completed_on:-'
@@ -1986,10 +1958,6 @@ if True: warnings_no = Count('logmessage', only=q_warnings) ) - # add timespent field - timespent = 'completed_on - started_on' - queryset_all = queryset_all.extra(select={'timespent': timespent}) - queryset_with_search = _get_queryset(Build, queryset_all, None, search_term, ordering_string, '-completed_on')
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 7/17 toaster: toastergui Add QuerysetFilter class
In reply to Elliot SmithFilters are added to tables by passing a list of function to the ToasterTable.add_filters() method. This means a lot of code is duplicated.
Add a QuerysetFilter class which encapsulates a filter for a Queryset and provides a generic count() method.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/querysetfilter.py | 24 +++++ bitbake/lib/toaster/toastergui/tables.py | 108 +++++++---------------- bitbake/lib/toaster/toastergui/widgets.py | 16 ++-- 3 files changed, 65 insertions(+), 83 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/querysetfilter.py
bitbake/lib/toaster/toastergui/querysetfilter.py new file +24 -0
@@ -0,0 +1,24 @@ +class QuerysetFilter(object): + """ Filter for a queryset """ + + def __init__(self, criteria=None): + if criteria: + self.set_criteria(criteria) + + def set_criteria(self, criteria): + """ + criteria is an instance of django.db.models.Q; + see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects + """ + self.criteria = criteria + + def filter(self, queryset): + """ + Filter queryset according to the criteria for this filter, + returning the filtered queryset + """ + return queryset.filter(self.criteria) + + def count(self, queryset): + """ Returns a count of the elements in the filtered queryset """ + return self.filter(queryset).count()
bitbake/lib/toaster/toastergui/tables.py +30 -78
196 lines not on this page · raw message
bitbake/lib/toaster/toastergui/widgets.py +11 -5
@@ -32,6 +32,7 @@ from django.template import Context, Template from django.core.serializers.json import DjangoJSONEncoder from django.core.exceptions import FieldError from django.conf.urls import url, patterns +from toastergui.querysetfilter import QuerysetFilter import types import json
@@ -113,7 +114,8 @@ class ToasterTable(TemplateView): cls=DjangoJSONEncoder) else: for actions in self.filters[name]['filter_actions']: - actions['count'] = self.filter_actions[actions['name']](count_only=True) + queryset_filter = self.filter_actions[actions['name']] + actions['count'] = queryset_filter.count(self.queryset) # Add the "All" items filter action self.filters[name]['filter_actions'].insert(0, {
@@ -151,15 +153,18 @@ class ToasterTable(TemplateView): 'filter_actions' : filter_actions, } - def make_filter_action(self, name, title, action_function): - """ Utility to make a filter_action """ + def make_filter_action(self, name, title, queryset_filter): + """ + Utility to make a filter_action; queryset_filter is an instance + of QuerysetFilter or a function + """ action = { 'title' : title, 'name' : name, } - self.filter_actions[name] = action_function + self.filter_actions[name] = queryset_filter return action
@@ -222,7 +227,8 @@ class ToasterTable(TemplateView): return try: - self.filter_actions[filter_action]() + queryset_filter = self.filter_actions[filter_action] + self.queryset = queryset_filter.filter(self.queryset) except KeyError: # pass it to the user - programming error here raise
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 8/17 toaster: toastergui Refactor ToasterTable filtering
In reply to Elliot SmithThe filter code for ToasterTable was difficult to follow and inflexible (not allowing different types of filter, for example).
Refactor to a set of filter classes to make the structure cleaner and provide the flexibility needed for other filter types (e.g. date range filter).
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/querysetfilter.py | 7 +- bitbake/lib/toaster/toastergui/static/js/table.js | 80 +++++++++---- bitbake/lib/toaster/toastergui/tablefilter.py | 98 ++++++++++++++++ bitbake/lib/toaster/toastergui/tables.py | 132 ++++++++++++++-------- bitbake/lib/toaster/toastergui/widgets.py | 90 +++++++-------- 5 files changed, 289 insertions(+), 118 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/tablefilter.py
bitbake/lib/toaster/toastergui/querysetfilter.py +5 -2
@@ -5,7 +5,7 @@ class QuerysetFilter(object): if criteria: self.set_criteria(criteria) - def set_criteria(self, criteria): + def set_criteria(self, criteria = None): """ criteria is an instance of django.db.models.Q; see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects
@@ -17,7 +17,10 @@ class QuerysetFilter(object): Filter queryset according to the criteria for this filter, returning the filtered queryset """ - return queryset.filter(self.criteria) + if self.criteria: + return queryset.filter(self.criteria) + else: + return queryset def count(self, queryset): """ Returns a count of the elements in the filtered queryset """
bitbake/lib/toaster/toastergui/static/js/table.js +59 -21
97 lines not on this page · raw message
bitbake/lib/toaster/toastergui/tablefilter.py new file +98 -0
98 lines not on this page · raw message
bitbake/lib/toaster/toastergui/tables.py +87 -45
180 lines not on this page · raw message
bitbake/lib/toaster/toastergui/widgets.py +40 -50
144 lines not on this page · raw message
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 9/17 toaster: toastergui Switch off filter highlights when inactive
In reply to Elliot SmithIn ToasterTables with multiple columns which allow filtering (e.g. all builds), selecting one filter, then a second filter (e.g. selecting "failed builds" then "outcome" for all builds), would result in both filters being highlighted at the same time.
Fix this by removing the "active" highlight on all column filter buttons when a new filter value is submitted (via the filter modal).
NB to enable this, added a data-filter-on attribute to all column filter buttons to make them easy to select.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/static/js/table.js | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
bitbake/lib/toaster/toastergui/static/js/table.js +7 -1
@@ -248,7 +248,7 @@ function tableInit(ctx){ /* Setup the filter button */ if (col.filter_name){ - var filterBtn = $('<a href="#" role="button" class="pull-right btn btn-mini" data-toggle="modal"><i class="icon-filter filtered"></i></a>'); + var filterBtn = $('<a href="#" role="button" data-filter-on="' + col.filter_name + '" class="pull-right btn btn-mini" data-toggle="modal"><i class="icon-filter filtered"></i></a>'); filterBtn.data('filter-name', col.filter_name); filterBtn.prop('id', col.filter_name);
@@ -565,6 +565,12 @@ function tableInit(ctx){ $("#filter-modal-form-"+ctx.tableName).submit(function(e){ e.preventDefault(); + /* remove active status from all filter buttons so that only one filter + can be active at a time */ + $('[data-filter-on]').each(function (index, filterBtn) { + filterBtnActive($(filterBtn), false); + }); + tableParams.filter = $(this).find("input[type='radio']:checked").val(); var filterBtn = $("#" + tableParams.filter.split(":")[0]);
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 10/17 toaster: toastergui Show recent builds on all builds page
In reply to Elliot SmithThe recent builds section was disabled while converting the all builds page to ToasterTable.
Re-enable the recent builds area and add the data it requires to the ToasterTable context.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/tables.py | 18 +++++++++++++++++- .../toastergui/templates/builds-toastertable.html | 10 +++++----- .../lib/toaster/toastergui/templates/mrb_section.html | 2 +- 3 files changed, 23 insertions(+), 7 deletions(-)
bitbake/lib/toaster/toastergui/tables.py +17 -1
@@ -27,6 +27,7 @@ from django.db.models import Q, Max, Count from django.conf.urls import url from django.core.urlresolvers import reverse from django.views.generic import TemplateView +import itertools from toastergui.tablefilter import TableFilter, TableFilterActionToggle
@@ -889,7 +890,22 @@ class BuildsTable(ToasterTable): self.static_context_extra['Task'] = Task def get_context_data(self, **kwargs): - return super(BuildsTable, self).get_context_data(**kwargs) + context = super(BuildsTable, self).get_context_data(**kwargs) + + # for the latest builds section + queryset = Build.objects.all() + + finished_criteria = Q(outcome=Build.SUCCEEDED) | Q(outcome=Build.FAILED) + + latest_builds = itertools.chain( + queryset.filter(outcome=Build.IN_PROGRESS).order_by("-started_on"), + queryset.filter(finished_criteria).order_by("-completed_on")[:3] + ) + + context['mru'] = list(latest_builds) + context['mrb_type'] = 'all' + + return context def setup_queryset(self, *args, **kwargs): queryset = Build.objects.all()
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html +5 -5
@@ -3,13 +3,13 @@ {% block title %} All builds - Toaster {% endblock %} {% block pagecontent %} - <div class="page-header top-air"> - <h1 data-role="page-title"></h1> - </div> <div class="row-fluid"> - {# TODO need to pass this data to context #} - {#% include 'mrb_section.html' %#} + {% with mru=mru mrb_type=mrb_type %} + {% include 'mrb_section.html' %} + {% endwith %} + + <h1 class="page-header top-air" data-role="page-title"></h1> {% url 'builds' as xhr_table_url %} {% include 'toastertable.html' %}
bitbake/lib/toaster/toastergui/templates/mrb_section.html +1 -1
@@ -14,7 +14,7 @@ {% endif %} </h2> {% else %} - <div class="page-header"> + <div class="page-header top-air"> <h1> Latest builds </h1>
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 11/17 toaster: toastergui Implement date range filters for builds
In reply to Elliot SmithImplement the completed_on and started_on filtering for builds.
Also separate the name of a filter ("filter" in the querystring) from its value ("filter_value" in the querystring). This enables filtering to be defined in the querystring more intuitively, and also makes it easier to add other types of filter (e.g. by day).
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/querysetfilter.py | 3 +- bitbake/lib/toaster/toastergui/static/js/table.js | 192 +++++++++++++++++---- bitbake/lib/toaster/toastergui/tablefilter.py | 114 ++++++++++-- bitbake/lib/toaster/toastergui/tables.py | 38 +++- .../toastergui/templates/builds-toastertable.html | 32 +--- bitbake/lib/toaster/toastergui/widgets.py | 32 ++-- 6 files changed, 329 insertions(+), 82 deletions(-)
bitbake/lib/toaster/toastergui/querysetfilter.py +2 -1
@@ -2,10 +2,11 @@ class QuerysetFilter(object): """ Filter for a queryset """ def __init__(self, criteria=None): + self.criteria = None if criteria: self.set_criteria(criteria) - def set_criteria(self, criteria = None): + def set_criteria(self, criteria): """ criteria is an instance of django.db.models.Q; see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects
bitbake/lib/toaster/toastergui/static/js/table.js +163 -29
222 lines not on this page · raw message
bitbake/lib/toaster/toastergui/tablefilter.py +102 -12
152 lines not on this page · raw message
bitbake/lib/toaster/toastergui/tables.py +37 -1
@@ -29,7 +29,9 @@ from django.core.urlresolvers import reverse from django.views.generic import TemplateView import itertools -from toastergui.tablefilter import TableFilter, TableFilterActionToggle +from toastergui.tablefilter import TableFilter +from toastergui.tablefilter import TableFilterActionToggle +from toastergui.tablefilter import TableFilterActionDateRange class ProjectFilters(object): def __init__(self, project_layers):
@@ -1072,6 +1074,7 @@ class BuildsTable(ToasterTable): help_text='The date and time when the build started', hideable=True, orderable=True, + filter_name='started_on_filter', static_data_name='started_on', static_data_template=started_on_template)
@@ -1079,6 +1082,7 @@ class BuildsTable(ToasterTable): help_text='The date and time when the build finished', hideable=False, orderable=True, + filter_name='completed_on_filter', static_data_name='completed_on', static_data_template=completed_on_template)
@@ -1151,6 +1155,38 @@ class BuildsTable(ToasterTable): outcome_filter.add_action(failed_builds_filter_action) self.add_filter(outcome_filter) + # started on + started_on_filter = TableFilter( + 'started_on_filter', + 'Filter by date when build was started' + ) + + by_started_date_range_filter_action = TableFilterActionDateRange( + 'date_range', + 'Build date range', + 'started_on', + QuerysetFilter() + ) + + started_on_filter.add_action(by_started_date_range_filter_action) + self.add_filter(started_on_filter) + + # completed on + completed_on_filter = TableFilter( + 'completed_on_filter', + 'Filter by date when build was completed' + ) + + by_completed_date_range_filter_action = TableFilterActionDateRange( + 'date_range', + 'Build date range', + 'completed_on', + QuerysetFilter() + ) + + completed_on_filter.add_action(by_completed_date_range_filter_action) + self.add_filter(completed_on_filter) + # failed tasks failed_tasks_filter = TableFilter( 'failed_tasks_filter',
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html +9 -23
@@ -1,4 +1,13 @@ {% extends 'base.html' %} +{% load static %} + +{% block extraheadcontent %} + <link rel="stylesheet" href="{% static 'css/jquery-ui.min.css' %}" type='text/css'> + <link rel="stylesheet" href="{% static 'css/jquery-ui.structure.min.css' %}" type='text/css'> + <link rel="stylesheet" href="{% static 'css/jquery-ui.theme.min.css' %}" type='text/css'> + <script src="{% static 'js/jquery-ui.min.js' %}"> + </script> +{% endblock %} {% block title %} All builds - Toaster {% endblock %}
@@ -34,29 +43,6 @@ titleElt.text(title); }); - - /* {% if last_date_from and last_date_to %} - // TODO initialize the date range controls; - // this will need to be added via ToasterTable - date_init( - "started_on", - "{{last_date_from}}", - "{{last_date_to}}", - "{{dateMin_started_on}}", - "{{dateMax_started_on}}", - "{{daterange_selected}}" - ); - - date_init( - "completed_on", - "{{last_date_from}}", - "{{last_date_to}}", - "{{dateMin_completed_on}}", - "{{dateMax_completed_on}}", - "{{daterange_selected}}" - ); - {% endif %} - */ }); </script> {% endblock %}
bitbake/lib/toaster/toastergui/widgets.py +16 -16
@@ -183,13 +183,13 @@ class ToasterTable(TemplateView): return template.render(context) - def apply_filter(self, filters, **kwargs): + def apply_filter(self, filters, filter_value, **kwargs): """ Apply a filter submitted in the querystring to the ToasterTable filters: (str) in the format: - '<filter name>:<action name>!<action params>' - where <action params> is optional + '<filter name>:<action name>' + filter_value: (str) parameters to pass to the named filter <filter name> and <action name> are used to look up the correct filter in the ToasterTable's filter map; the <action params> are set on
@@ -199,15 +199,8 @@ class ToasterTable(TemplateView): self.setup_filters(**kwargs) try: - filter_name, action_name_and_params = filters.split(':') - - action_name = None - action_params = None - if re.search('!', action_name_and_params): - action_name, action_params = action_name_and_params.split('!') - action_params = urllib.unquote_plus(action_params) - else: - action_name = action_name_and_params + filter_name, action_name = filters.split(':') + action_params = urllib.unquote_plus(filter_value) except ValueError: return
@@ -217,7 +210,7 @@ class ToasterTable(TemplateView): try: table_filter = self.filter_map.get_filter(filter_name) action = table_filter.get_action(action_name) - action.set_params(action_params) + action.set_filter_params(action_params) self.queryset = action.filter(self.queryset) except KeyError: # pass it to the user - programming error here
@@ -247,13 +240,20 @@ class ToasterTable(TemplateView): def get_data(self, request, **kwargs): - """Returns the data for the page requested with the specified - parameters applied""" + """ + Returns the data for the page requested with the specified + parameters applied + + filters: filter and action name, e.g. "outcome:build_succeeded" + filter_value: value to pass to the named filter+action, e.g. "on" + (for a toggle filter) or "2015-12-11,2015-12-12" (for a date range filter) + """ page_num = request.GET.get("page", 1) limit = request.GET.get("limit", 10) search = request.GET.get("search", None) filters = request.GET.get("filter", None) + filter_value = request.GET.get("filter_value", "on") orderby = request.GET.get("orderby", None) nocache = request.GET.get("nocache", None)
@@ -285,7 +285,7 @@ class ToasterTable(TemplateView): if search: self.apply_search(search) if filters: - self.apply_filter(filters, **kwargs) + self.apply_filter(filters, filter_value, **kwargs) if orderby: self.apply_orderby(orderby)
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 12/17 toaster: toastergui Implement "today" and "yesterday" filters
In reply to Elliot SmithAdd the "today" and "yesterday" filters to the started_on and completed_on columns in the builds table.
During this work, some minor adjustments were made to the behaviour of the builds table:
* Amend filter action variable names so they're more succinct.
* Retain order in which actions are added to a filter, as this ordering is used in the UI when displaying the filter actions.
* Always show the table chrome, otherwise it's not possible to edit the columns shown until there are 10 or more results.
* Because date range searches may return no results, make sure that the search bar and "show all results" link are visible when the query returns no results.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/querysetfilter.py | 4 - bitbake/lib/toaster/toastergui/static/js/table.js | 56 +++++---- bitbake/lib/toaster/toastergui/tablefilter.py | 139 +++++++++++++++++---- bitbake/lib/toaster/toastergui/tables.py | 87 ++++++++----- .../toastergui/templates/builds-toastertable.html | 2 +- .../toaster/toastergui/templates/toastertable.html | 7 +- 6 files changed, 211 insertions(+), 84 deletions(-)
bitbake/lib/toaster/toastergui/querysetfilter.py +0 -4
@@ -22,7 +22,3 @@ class QuerysetFilter(object): return queryset.filter(self.criteria) else: return queryset - - def count(self, queryset): - """ Returns a count of the elements in the filtered queryset """ - return self.filter(queryset).count()
bitbake/lib/toaster/toastergui/static/js/table.js +31 -25
128 lines not on this page · raw message
bitbake/lib/toaster/toastergui/tablefilter.py +115 -24
213 lines not on this page · raw message
bitbake/lib/toaster/toastergui/tables.py +59 -28
191 lines not on this page · raw message
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html +1 -1
@@ -18,7 +18,7 @@ {% include 'mrb_section.html' %} {% endwith %} - <h1 class="page-header top-air" data-role="page-title"></h1> + <h1 class="page-header top-air" data-role="page-title"></h1> {% url 'builds' as xhr_table_url %} {% include 'toastertable.html' %}
bitbake/lib/toaster/toastergui/templates/toastertable.html +5 -2
@@ -32,8 +32,11 @@ <a href="#" class="add-on btn remove-search-btn-{{table_name}}" tabindex="-1"> <i class="icon-remove"></i> </a> - <button class="btn search-submit-{{table_name}}" >Search</button> - <button class="btn btn-link remove-search-btn-{{table_name}}">Show {{title|lower}} + <button class="btn search-submit-{{table_name}}"> + Search + </button> + <button class="btn btn-link show-all-{{table_name}}"> + Show {{title|lower}} </button> </form> </div>
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 13/17 toaster: toastergui Convert project builds page to ToasterTable
In reply to Elliot SmithUse the all builds ToasterTable as the basis for the project builds ToasterTable.
[YOCTO #8738]
--- .../toaster/toastergui/static/js/projecttopbar.js | 9 ++ bitbake/lib/toaster/toastergui/tables.py | 169 +++++++++++++++++---- .../toastergui/templates/baseprojectpage.html | 1 + .../toaster/toastergui/templates/mrb_section.html | 2 +- .../templates/projectbuilds-toastertable.html | 56 +++++++ bitbake/lib/toaster/toastergui/urls.py | 6 +- bitbake/lib/toaster/toastergui/views.py | 16 +- bitbake/lib/toaster/toastergui/widgets.py | 1 - 8 files changed, 223 insertions(+), 37 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html
bitbake/lib/toaster/toastergui/static/js/projecttopbar.js +9 -0
@@ -7,7 +7,10 @@ function projectTopBarInit(ctx) { var projectName = $("#project-name"); var projectNameFormToggle = $("#project-change-form-toggle"); var projectNameChangeCancel = $("#project-name-change-cancel"); + + // this doesn't exist for command-line builds var newBuildTargetInput = $("#build-input"); + var newBuildTargetBuildBtn = $("#build-button"); var selectedTarget;
@@ -42,6 +45,12 @@ function projectTopBarInit(ctx) { $(this).parent().removeClass('active'); }); + if (!newBuildTargetInput.length) { + return; + } + + /* the following only applies for non-command-line projects */ + /* Recipe build input functionality */ if (ctx.numProjectLayers > 0 && ctx.machine){ newBuildTargetInput.removeAttr("disabled");
bitbake/lib/toaster/toastergui/tables.py +143 -26
216 lines not on this page · raw message
bitbake/lib/toaster/toastergui/templates/baseprojectpage.html +1 -0
@@ -1,4 +1,5 @@ {% extends "base.html" %} + {% load projecttags %} {% load humanize %}
bitbake/lib/toaster/toastergui/templates/mrb_section.html +1 -1
@@ -6,7 +6,7 @@ {%if mru and mru.count > 0%} {%if mrb_type == 'project' %} - <h2> + <h2 class="page-header"> Latest project builds {% if project.is_default %}
bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html new file +56 -0
56 lines not on this page · raw message
bitbake/lib/toaster/toastergui/urls.py +4 -2
@@ -28,7 +28,7 @@ urlpatterns = patterns('toastergui.views', url(r'^landing/$', 'landing', name='landing'), url(r'^builds/$', - tables.BuildsTable.as_view(template_name="builds-toastertable.html"), + tables.AllBuildsTable.as_view(template_name="builds-toastertable.html"), name='all-builds'), # build info navigation
@@ -83,7 +83,9 @@ urlpatterns = patterns('toastergui.views', url(r'^project/(?P<pid>\d+)/$', 'project', name='project'), url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'), - url(r'^project/(?P<pid>\d+)/builds/$', 'projectbuilds', name='projectbuilds'), + url(r'^project/(?P<pid>\d+)/builds/$', + tables.ProjectBuildsTable.as_view(template_name="projectbuilds-toastertable.html"), + name='projectbuilds'), # the import layer is a project-specific functionality; url(r'^project/(?P<pid>\d+)/importlayer$', 'importlayer', name='importlayer'),
bitbake/lib/toaster/toastergui/views.py +9 -7
@@ -91,6 +91,7 @@ def landing(request): return render(request, 'landing.html', context) +""" # returns a list for most recent builds; def _get_latest_builds(prj=None): queryset = Build.objects.all()
@@ -101,8 +102,9 @@ def _get_latest_builds(prj=None): return list(itertools.chain( queryset.filter(outcome=Build.IN_PROGRESS).order_by("-started_on"), queryset.filter(outcome__lt=Build.IN_PROGRESS).order_by("-started_on")[:3] )) +""" - +""" # a JSON-able dict of recent builds; for use in the Project page, xhr_ updates, and other places, as needed def _project_recent_build_list(prj): data = []
@@ -131,8 +133,7 @@ def _project_recent_build_list(prj): data.append(d) return data - - +""" def objtojson(obj): from django.db.models.query import QuerySet
@@ -1915,6 +1916,7 @@ if True: ''' The exception raised on invalid POST requests ''' pass + """ # helper function, to be used on "all builds" and "project builds" pages def _build_list_helper(request, queryset_all, redirect_page, pid=None): default_orderby = 'completed_on:-'
@@ -2119,6 +2121,7 @@ if True: # merge daterange values context.update(context_date) return context, pagesize, orderby + """
@@ -2256,7 +2259,7 @@ if True: "completedbuilds": Build.objects.exclude(outcome = Build.IN_PROGRESS).filter(project_id = pid), "prj" : {"name": prj.name, }, "buildrequests" : prj.build_set.filter(outcome=Build.IN_PROGRESS), - "builds" : _project_recent_build_list(prj), + #"builds" : _project_recent_build_list(prj), "layers" : map(lambda x: { "id": x.layercommit.pk, "orderid": x.pk,
@@ -2827,10 +2830,8 @@ if True: # will set the GET parameters and redirect back to the # all-builds or projectbuilds page as appropriate; # TODO don't use exceptions to control program flow - @_template_renderer('projectbuilds.html') + """ def projectbuilds(request, pid): - prj = Project.objects.get(id = pid) - if request.method == "POST": # process any build request
@@ -2880,6 +2881,7 @@ if True: context['mru'] = _get_latest_builds(prj) return context + """ def _file_name_for_artifact(b, artifact_type, artifact_id):
bitbake/lib/toaster/toastergui/widgets.py +0 -1
@@ -61,7 +61,6 @@ class ToasterTable(TemplateView): self.total_count = 0 self.static_context_extra = {} - self.filter_actions = {} self.empty_state = "Sorry - no data found" self.default_orderby = ""
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 14/17 toaster: tests Fix Django tests for new ToasterTable pages
In reply to Elliot SmithThe Django command-line tests can no longer test the content of the projects/, builds/ and projectbuilds/ pages, as ToasterTable pages are populated by JavaScript.
Fix/remove affected tests by converting them to tests on the JSON returned by the ToasterTable.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/tests.py | 255 +++++++++++++++++++++----------- 1 file changed, 170 insertions(+), 85 deletions(-)
bitbake/lib/toaster/toastergui/tests.py +170 -85
@@ -38,11 +38,14 @@ import toastergui from toastergui.tables import SoftwareRecipesTable import json +from datetime import timedelta from bs4 import BeautifulSoup import re import string +import json PROJECT_NAME = "test project" +PROJECT_NAME2 = "test project 2" CLI_BUILDS_PROJECT_NAME = 'Command line builds' class ViewTests(TestCase):
@@ -54,14 +57,36 @@ class ViewTests(TestCase): release = Release.objects.create(name="test release", branch_name="master", bitbake_version=bbv) + release2 = Release.objects.create(name="test release 2", + branch_name="master", + bitbake_version=bbv) + self.project = Project.objects.create_project(name=PROJECT_NAME, release=release) + + Project.objects.create_project(name=PROJECT_NAME2, + release=release2) + now = timezone.now() + later = now + timedelta(days=1) build = Build.objects.create(project=self.project, started_on=now, completed_on=now) + # for testing BuildsTable + Build.objects.create(project=self.project, + started_on=now, + completed_on=now, + outcome=Build.SUCCEEDED, + machine="raspberrypi2") + + Build.objects.create(project=self.project, + started_on=later, + completed_on=later, + outcome=Build.FAILED, + machine="qemux86") + layersrc = LayerSource.objects.create(sourcetype=LayerSource.TYPE_IMPORTED) self.priority = ReleaseLayerSourcePriority.objects.create(release=release, layer_source=layersrc)
@@ -172,8 +197,7 @@ class ViewTests(TestCase): response = self.client.get(reverse('all-projects'), follow=True) self.assertEqual(response.status_code, 200) self.assertTrue(response['Content-Type'].startswith('text/html')) - self.assertTemplateUsed(response, "projects.html") - self.assertTrue(PROJECT_NAME in response.content) + self.assertTemplateUsed(response, "projects-toastertable.html") def test_get_json_call_returns_json(self): """Test for all projects output in json format"""
@@ -191,13 +215,6 @@ class ViewTests(TestCase): self.assertTrue(PROJECT_NAME in [x["name"] for x in data["rows"]]) self.assertTrue("id" in data["rows"][0]) - self.assertEqual(sorted(data["rows"][0]), - ['bitbake_version_id', 'created', 'id', - 'is_default', 'layersTypeAheadUrl', 'name', - 'num_builds', 'projectBuildsUrl', 'projectPageUrl', - 'recipesTypeAheadUrl', 'release_id', - 'short_description', 'updated', 'user_id']) - def test_typeaheads(self): """Test typeahead ReST API""" layers_url = reverse('xhr_layerstypeahead', args=(self.project.id,))
@@ -450,7 +467,7 @@ class ViewTests(TestCase): all_data = get_data(table) self.assertTrue(len(all_data['rows']) > 1, - "Cannot test on a table with < 1 row") + "Cannot test on the table %s with < 1 row" % name) if table.default_orderby: row_one = all_data['rows'][0][table.default_orderby.strip("-")]
@@ -512,16 +529,20 @@ class ViewTests(TestCase): # This is the name of the filter:action # e.g. project_filter:not_in_project filter_string = "%s:%s" % (column['filter_name'], - filter_action['name']) + filter_action['action_name']) # Now get the data with the filter applied filtered_data = get_data(table_cls(), {"filter" : filter_string}) - self.assertEqual(len(filtered_data['rows']), - int(filter_action['count']), - "We added a table filter for %s but " - "the number of rows returned was not " - "what the filter info said there " - "would be" % name) + + # date range filter actions can't specify the + # number of results they return, so their count is 0 + if filter_action['count'] != None: + self.assertEqual(len(filtered_data['rows']), + int(filter_action['count']), + "We added a table filter for %s but " + "the number of rows returned was not " + "what the filter info said there " + "would be" % name) # Test search functionality on the table
@@ -673,6 +694,10 @@ class AllProjectsPageTests(TestCase): value=self.MACHINE_NAME) project_var.save() + def _get_row_for_project(self, data, project_id): + """ Get the object representing the table data for a project """ + return [row for row in data['rows'] if row['id'] == project_id][0] + def test_default_project_hidden(self): """ The default project should be hidden if it has no builds """ params = {"count": 10, "orderby": "updated:-", "page": 1}
@@ -688,11 +713,20 @@ class AllProjectsPageTests(TestCase): self._add_build_to_default_project() params = {"count": 10, "orderby": "updated:-", "page": 1} - response = self.client.get(reverse('all-projects'), params) - self.assertTrue('tr class="data"' in response.content, - 'should be a project row in the page') - self.assertTrue(CLI_BUILDS_PROJECT_NAME in response.content, + response = self.client.get( + reverse('all-projects'), + {'format': 'json'}, + params + ) + + data = json.loads(response.content) + + # find the row for the default project + default_project_row = self._get_row_for_project(data, self.default_project.id) + + # check its name template has the correct text + self.assertEqual(default_project_row['name'], CLI_BUILDS_PROJECT_NAME, 'default project "cli builds" should be in page') def test_default_project_release(self):
@@ -706,24 +740,32 @@ class AllProjectsPageTests(TestCase): # another project to test, which should show release self._add_non_default_project() - response = self.client.get(reverse('all-projects'), follow=True) - soup = BeautifulSoup(response.content) + response = self.client.get( + reverse('all-projects'), + {'format': 'json'}, + follow=True + ) + + data = json.loads(response.content) + + # used to find the correct span in the template output + attrs = {'data-project-field': 'release'} + + # find the row for the default project + default_project_row = self._get_row_for_project(data, self.default_project.id) - # check the release cell for the default project - attrs = {'data-project': str(self.default_project.id)} - rows = soup.find_all('tr', attrs=attrs) - self.assertEqual(len(rows), 1, 'should be one row for default project') - cells = rows[0].find_all('td', attrs={'data-project-field': 'release'}) - self.assertEqual(len(cells), 1, 'should be one release cell') - text = cells[0].select('span.muted')[0].text + # check the release text for the default project + soup = BeautifulSoup(default_project_row['static:release']) + text = soup.find('span', attrs=attrs).select('span.muted')[0].text self.assertEqual(text, 'Not applicable', 'release should be not applicable for default project') + # find the row for the default project + other_project_row = self._get_row_for_project(data, self.project.id) + # check the link in the release cell for the other project - attrs = {'data-project': str(self.project.id)} - rows = soup.find_all('tr', attrs=attrs) - cells = rows[0].find_all('td', attrs={'data-project-field': 'release'}) - text = cells[0].select('a')[0].text + soup = BeautifulSoup(other_project_row['static:release']) + text = soup.find('span', attrs=attrs).select('a')[0].text.strip() self.assertEqual(text, self.release.name, 'release name should be shown for non-default project')
@@ -738,24 +780,32 @@ class AllProjectsPageTests(TestCase): # another project to test, which should show machine self._add_non_default_project() - response = self.client.get(reverse('all-projects'), follow=True) - soup = BeautifulSoup(response.content) + response = self.client.get( + reverse('all-projects'), + {'format': 'json'}, + follow=True + ) + + data = json.loads(response.content) + + # used to find the correct span in the template output + attrs = {'data-project-field': 'machine'} + + # find the row for the default project + default_project_row = self._get_row_for_project(data, self.default_project.id) # check the machine cell for the default project - attrs = {'data-project': str(self.default_project.id)} - rows = soup.find_all('tr', attrs=attrs) - self.assertEqual(len(rows), 1, 'should be one row for default project') - cells = rows[0].find_all('td', attrs={'data-project-field': 'machine'}) - self.assertEqual(len(cells), 1, 'should be one machine cell') - text = cells[0].select('span.muted')[0].text + soup = BeautifulSoup(default_project_row['static:machine']) + text = soup.find('span', attrs=attrs).select('span.muted')[0].text.strip() self.assertEqual(text, 'Not applicable', - 'machine should be not applicable for default project') + 'machine should be not applicable for default project') + + # find the row for the default project + other_project_row = self._get_row_for_project(data, self.project.id) # check the link in the machine cell for the other project - attrs = {'data-project': str(self.project.id)} - rows = soup.find_all('tr', attrs=attrs) - cells = rows[0].find_all('td', attrs={'data-project-field': 'machine'}) - text = cells[0].select('a')[0].text + soup = BeautifulSoup(other_project_row['static:machine']) + text = soup.find('span', attrs=attrs).find('a').text.strip() self.assertEqual(text, self.MACHINE_NAME, 'machine name should be shown for non-default project')
@@ -769,24 +819,33 @@ class AllProjectsPageTests(TestCase): # need a build, otherwise project doesn't display at all self._add_build_to_default_project() - # another project to test, which should show machine + # another project to test self._add_non_default_project() - response = self.client.get(reverse('all-projects'), follow=True) - soup = BeautifulSoup(response.content) + response = self.client.get( + reverse('all-projects'), + {'format': 'json'}, + follow=True + ) + + data = json.loads(response.content) - # link for default project - row = soup.find('tr', attrs={'data-project': self.default_project.id}) - cell = row.find('td', attrs={'data-project-field': 'name'}) + # find the row for the default project + default_project_row = self._get_row_for_project(data, self.default_project.id) + + # check the link on the name field + soup = BeautifulSoup(default_project_row['static:name']) expected_url = reverse('projectbuilds', args=(self.default_project.id,)) - self.assertEqual(cell.find('a')['href'], expected_url, + self.assertEqual(soup.find('a')['href'], expected_url, 'link on default project name should point to builds') - # link for other project - row = soup.find('tr', attrs={'data-project': self.project.id}) - cell = row.find('td', attrs={'data-project-field': 'name'}) + # find the row for the other project + other_project_row = self._get_row_for_project(data, self.project.id) + + # check the link for the other project + soup = BeautifulSoup(other_project_row['static:name']) expected_url = reverse('project', args=(self.project.id,)) - self.assertEqual(cell.find('a')['href'], expected_url, + self.assertEqual(soup.find('a')['href'], expected_url, 'link on project name should point to configuration') class ProjectBuildsPageTests(TestCase):
@@ -846,9 +905,9 @@ class ProjectBuildsPageTests(TestCase): def _get_rows_for_project(self, project_id): """ Helper to retrieve HTML rows for a project """ url = reverse("projectbuilds", args=(project_id,)) - response = self.client.get(url, follow=True) - soup = BeautifulSoup(response.content) - return soup.select('tr[class="data"]') + response = self.client.get(url, {'format': 'json'}, follow=True) + data = json.loads(response.content) + return data['rows'] def test_show_builds_for_project(self): """ Builds for a project should be displayed """
@@ -889,10 +948,14 @@ class ProjectBuildsPageTests(TestCase): """ Task should be shown as suffix on build name """ build = Build.objects.create(**self.project1_build_success) Target.objects.create(build=build, target='bash', task='clean') - url = reverse("projectbuilds", args=(self.project1.id,)) - response = self.client.get(url, follow=True) - result = re.findall('^ +bash:clean$', response.content, re.MULTILINE) - self.assertEqual(len(result), 2) + + url = reverse('projectbuilds', args=(self.project1.id,)) + response = self.client.get(url, {'format': 'json'}, follow=True) + data = json.loads(response.content) + cell = data['rows'][0]['static:target'] + + result = re.findall('^ +bash:clean', cell, re.MULTILINE) + self.assertEqual(len(result), 1) def test_cli_builds_hides_tabs(self): """
@@ -952,32 +1015,46 @@ class AllBuildsPageTests(TestCase): "outcome": Build.SUCCEEDED } + def _get_row_for_build(self, data, build_id): + """ Get the object representing the table data for a project """ + return [row for row in data['rows'] + if row['id'] == build_id][0] + def test_show_tasks_in_allbuilds(self): """ Task should be shown as suffix on build name """ build = Build.objects.create(**self.project1_build_success) Target.objects.create(build=build, target='bash', task='clean') + url = reverse('all-builds') - response = self.client.get(url, follow=True) - result = re.findall('bash:clean', response.content, re.MULTILINE) - self.assertEqual(len(result), 3) + response = self.client.get(url, {'format': 'json'}, follow=True) + data = json.loads(response.content) + cell = data['rows'][0]['static:target'] - def test_no_run_again_for_cli_build(self): - """ "Run again" button should not be shown for command-line builds """ - build = Build.objects.create(**self.default_project_build_success) + result = re.findall('bash:clean', cell, re.MULTILINE) + self.assertEqual(len(result), 1) + + def test_run_again(self): + """ + "Run again" button should not be shown for command-line builds, + but should be shown for other builds + """ + build1 = Build.objects.create(**self.project1_build_success) + default_build = Build.objects.create(**self.default_project_build_success) url = reverse('all-builds') response = self.client.get(url, follow=True) soup = BeautifulSoup(response.content) - attrs = {'data-latest-build-result': build.id} - result = soup.find('div', attrs=attrs) - # shouldn't see a run again button for command-line builds + attrs = {'data-latest-build-result': default_build.id} + result = soup.find('div', attrs=attrs) run_again_button = result.select('button') self.assertEqual(len(run_again_button), 0) - # should see a help icon for command-line builds - help_icon = result.select('i.get-help-green') - self.assertEqual(len(help_icon), 1) + # should see a run again button for non-command-line builds + attrs = {'data-latest-build-result': build1.id} + result = soup.find('div', attrs=attrs) + run_again_button = result.select('button') + self.assertEqual(len(run_again_button), 1) def test_tooltips_on_project_name(self): """
@@ -989,20 +1066,28 @@ class AllBuildsPageTests(TestCase): default_build = Build.objects.create(**self.default_project_build_success) url = reverse('all-builds') - response = self.client.get(url, follow=True) - soup = BeautifulSoup(response.content) + response = self.client.get(url, {'format': 'json'}, follow=True) + data = json.loads(response.content) + + # get the data row for the non-command-line builds project + other_project_row = self._get_row_for_build(data, build1.id) + + # make sure there is some HTML + soup = BeautifulSoup(other_project_row['static:project-name']) + self.assertEqual(len(soup.select('a')), 1, + 'should be a project name link') # no help icon on non-default project name - result = soup.find('tr', attrs={'data-table-build-result': build1.id}) - name = result.select('td.project-name')[0] - icons = name.select('i.get-help') + icons = soup.select('i.get-help') self.assertEqual(len(icons), 0, 'should not be a help icon for non-cli builds name') + # get the data row for the command-line builds project + default_project_row = self._get_row_for_build(data, default_build.id) + # help icon on default project name - result = soup.find('tr', attrs={'data-table-build-result': default_build.id}) - name = result.select('td.project-name')[0] - icons = name.select('i.get-help') + soup = BeautifulSoup(default_project_row['static:project-name']) + icons = soup.select('i.get-help') self.assertEqual(len(icons), 1, 'should be a help icon for cli builds name')
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 15/17 toaster: toastergui Don't hide all elements with .col class
In reply to Elliot SmithDisabling the "project" column in a ToasterTable for builds causes the recent builds area to be hidden. This is because the column hiding code hides all elements with a class matching ".<column>", regardless of where they occur on the page; and the recent builds area was using the ".project-name" class, which means it is included in the set of elements which are hidden.
Scope the element search to the table so that only elements within the table are hidden or shown.
[YOCTO #8792]
--- bitbake/lib/toaster/toastergui/static/js/table.js | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
bitbake/lib/toaster/toastergui/static/js/table.js +4 -4
@@ -313,7 +313,7 @@ function tableInit(ctx){ $("#"+ctx.tableName+" th").each(function(){ for (var i in cols_hidden){ if ($(this).hasClass(cols_hidden[i])){ - $("."+cols_hidden[i]).hide(); + table.find("."+cols_hidden[i]).hide(); $("#checkbox-"+cols_hidden[i]).removeAttr("checked"); } }
@@ -323,7 +323,7 @@ function tableInit(ctx){ * user setting. */ for (var i in defaultHiddenCols) { - $("."+defaultHiddenCols[i]).hide(); + table.find("."+defaultHiddenCols[i]).hide(); $("#checkbox-"+defaultHiddenCols[i]).removeAttr("checked"); } }
@@ -367,9 +367,9 @@ function tableInit(ctx){ var disabled_cols = []; if ($(this).prop("checked")) { - $("."+col).show(); + table.find("."+col).show(); } else { - $("."+col).hide(); + table.find("."+col).hide(); /* If we're ordered by the column we're hiding remove the order by */ if (col === tableParams.orderby || '-' + col === tableParams.orderby){
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 16/17 toaster: toastergui Ensure filter_value updates
In reply to Elliot SmithClicking on the radio button for a date range filter action populates the from and to fields for that action if they are empty.
However, because this doesn't fire "change" events, clicking on the radio button doesn't update the filter_value hidden field. This means that the date range action's filter_value parameter isn't set correctly when the filter popup is submitted.
Manually call the changeHandler() to set the filter_value whenever the radio for a date range filter is clicked.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/static/js/table.js | 32 +++++++++++++---------- 1 file changed, 18 insertions(+), 14 deletions(-)
bitbake/lib/toaster/toastergui/static/js/table.js +18 -14
@@ -485,20 +485,6 @@ function tableInit(ctx){ action.find('[data-date-to-for]').datepicker(options); inputTo.val(selectedTo); - // if the radio button is checked and one or both of the datepickers are - // empty, populate them with today's date - radio.change(function () { - var now = new Date(); - - if (inputFrom.val() === '') { - inputFrom.datepicker('setDate', now); - } - - if (inputTo.val() === '') { - inputTo.datepicker('setDate', now); - } - }); - // set filter_value based on date pickers when // one of their values changes var changeHandler = function () {
@@ -526,6 +512,24 @@ function tableInit(ctx){ inputFrom.datepicker('option', 'maxDate', inputTo.val()); }); + // if the radio button is checked and one or both of the datepickers are + // empty, populate them with today's date + radio.change(function () { + var now = new Date(); + + if (inputFrom.val() === '') { + inputFrom.datepicker('setDate', now); + } + + if (inputTo.val() === '') { + inputTo.datepicker('setDate', now); + } + + // setting the date like this doesn't fire the changeHandler to + // update the filter_value, so do that manually instead + changeHandler() + }); + return action; }
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 17/17 toaster: toastergui Streamline construction of filter objects
In reply to Elliot SmithIn line with comments from review, remove the QuerysetFilter class (redundant) and convert ProjectFilters into a class with static methods.
[YOCTO #8738]
--- bitbake/lib/toaster/toastergui/querysetfilter.py | 24 --------------- bitbake/lib/toaster/toastergui/tablefilter.py | 38 ++++++++++-------------- bitbake/lib/toaster/toastergui/tables.py | 35 +++++++++++----------- bitbake/lib/toaster/toastergui/widgets.py | 1 - 4 files changed, 32 insertions(+), 66 deletions(-) delete mode 100644 bitbake/lib/toaster/toastergui/querysetfilter.py
bitbake/lib/toaster/toastergui/querysetfilter.py deleted +0 -24
@@ -1,24 +0,0 @@ -class QuerysetFilter(object): - """ Filter for a queryset """ - - def __init__(self, criteria=None): - self.criteria = None - if criteria: - self.set_criteria(criteria) - - def set_criteria(self, criteria): - """ - criteria is an instance of django.db.models.Q; - see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects - """ - self.criteria = criteria - - def filter(self, queryset): - """ - Filter queryset according to the criteria for this filter, - returning the filtered queryset - """ - if self.criteria: - return queryset.filter(self.criteria) - else: - return queryset
bitbake/lib/toaster/toastergui/tablefilter.py +15 -23
@@ -1,7 +1,6 @@ from datetime import timedelta from django.db.models import Q, Max, Min from django.utils import dateparse, timezone -from querysetfilter import QuerysetFilter class TableFilter(object): """
@@ -97,10 +96,10 @@ class TableFilterAction(object): ToasterTable """ - def __init__(self, name, title, queryset_filter): + def __init__(self, name, title, criteria): self.name = name self.title = title - self.queryset_filter = queryset_filter + self.criteria = criteria # set in subclasses self.type = None
@@ -111,11 +110,13 @@ class TableFilterAction(object): the structure of this string depends on the type of action; it's ignored for a toggle filter action, which is just on or off """ - if not params: - return + pass def filter(self, queryset): - return self.queryset_filter.filter(queryset) + if self.criteria: + return queryset.filter(self.criteria) + else: + return queryset def to_json(self, queryset): """ Dump as a JSON object """
@@ -146,16 +147,12 @@ class TableFilterActionDay(TableFilterAction): YESTERDAY = 'yesterday' def __init__(self, name, title, field, day, - queryset_filter = QuerysetFilter(), query_helper = TableFilterQueryHelper()): + query_helper = TableFilterQueryHelper()): """ field: (string) the datetime field to filter by day: (string) "today" or "yesterday" """ - super(TableFilterActionDay, self).__init__( - name, - title, - queryset_filter - ) + super(TableFilterActionDay, self).__init__(name, title, None) self.type = 'day' self.field = field self.day = day
@@ -168,8 +165,6 @@ class TableFilterActionDay(TableFilterAction): depending on when the filtering is applied """ - criteria = None - date_str = None now = timezone.now() if self.day == self.YESTERDAY:
@@ -180,15 +175,13 @@ class TableFilterActionDay(TableFilterAction): wanted_date_str = wanted_date.strftime('%Y-%m-%d') - criteria = self.query_helper.dateStringsToQ( + self.criteria = self.query_helper.dateStringsToQ( self.field, wanted_date_str, wanted_date_str ) - self.queryset_filter.set_criteria(criteria) - - return self.queryset_filter.filter(queryset) + return queryset.filter(self.criteria) class TableFilterActionDateRange(TableFilterAction): """
@@ -197,14 +190,14 @@ class TableFilterActionDateRange(TableFilterAction): """ def __init__(self, name, title, field, - queryset_filter = QuerysetFilter(), query_helper = TableFilterQueryHelper()): + query_helper = TableFilterQueryHelper()): """ field: (string) the field to find the max/min range from in the queryset """ super(TableFilterActionDateRange, self).__init__( name, title, - queryset_filter + None ) self.type = 'daterange'
@@ -227,17 +220,16 @@ class TableFilterActionDateRange(TableFilterAction): try: date_from_str, date_to_str = params.split(',') except ValueError: - self.queryset_filter.set_criteria(None) + self.criteria = None return # one of the values required for the filter is missing, so set # it to the one which was supplied - criteria = self.query_helper.dateStringsToQ( + self.criteria = self.query_helper.dateStringsToQ( self.field, date_from_str, date_to_str ) - self.queryset_filter.set_criteria(criteria) def to_json(self, queryset): """ Dump as a JSON object """
bitbake/lib/toaster/toastergui/tables.py +17 -18
@@ -20,7 +20,6 @@ # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. from toastergui.widgets import ToasterTable -from toastergui.querysetfilter import QuerysetFilter from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project from orm.models import CustomImageRecipe, Package, Build, LogMessage, Task from orm.models import ProjectTarget
@@ -37,9 +36,13 @@ from toastergui.tablefilter import TableFilterActionDateRange from toastergui.tablefilter import TableFilterActionDay class ProjectFilters(object): - def __init__(self, project_layers): - self.in_project = QuerysetFilter(Q(layer_version__in=project_layers)) - self.not_in_project = QuerysetFilter(~Q(layer_version__in=project_layers)) + @staticmethod + def in_project(project_layers): + return Q(layer_version__in=project_layers) + + @staticmethod + def not_in_project(project_layers): + return ~(ProjectFilters.in_project(project_layers)) class LayersTable(ToasterTable): """Table of layers in Toaster"""
@@ -71,13 +74,13 @@ class LayersTable(ToasterTable): in_project_action = TableFilterActionToggle( "in_project", "Layers added to this project", - QuerysetFilter(criteria) + criteria ) not_in_project_action = TableFilterActionToggle( "not_in_project", "Layers not added to this project", - QuerysetFilter(~criteria) + ~criteria ) in_current_project_filter.add_action(in_project_action)
@@ -217,8 +220,6 @@ class MachinesTable(ToasterTable): def setup_filters(self, *args, **kwargs): project = Project.objects.get(pk=kwargs['pid']) - project_filters = ProjectFilters(self.project_layers) - in_current_project_filter = TableFilter( "in_current_project", "Filter by project machines"
@@ -227,13 +228,13 @@ class MachinesTable(ToasterTable): in_project_action = TableFilterActionToggle( "in_project", "Machines provided by layers added to this project", - project_filters.in_project + ProjectFilters.in_project(self.project_layers) ) not_in_project_action = TableFilterActionToggle( "not_in_project", "Machines provided by layers not added to this project", - project_filters.not_in_project + ProjectFilters.not_in_project(self.project_layers) ) in_current_project_filter.add_action(in_project_action)
@@ -350,8 +351,6 @@ class RecipesTable(ToasterTable): return context def setup_filters(self, *args, **kwargs): - project_filters = ProjectFilters(self.project_layers) - table_filter = TableFilter( 'in_current_project', 'Filter by project recipes'
@@ -360,13 +359,13 @@ class RecipesTable(ToasterTable): in_project_action = TableFilterActionToggle( 'in_project', 'Recipes provided by layers added to this project', - project_filters.in_project + ProjectFilters.in_project(self.project_layers) ) not_in_project_action = TableFilterActionToggle( 'not_in_project', 'Recipes provided by layers not added to this project', - project_filters.not_in_project + ProjectFilters.not_in_project(self.project_layers) ) table_filter.add_action(in_project_action)
@@ -1141,13 +1140,13 @@ class BuildsTable(ToasterTable): successful_builds_action = TableFilterActionToggle( 'successful_builds', 'Successful builds', - QuerysetFilter(Q(outcome=Build.SUCCEEDED)) + Q(outcome=Build.SUCCEEDED) ) failed_builds_action = TableFilterActionToggle( 'failed_builds', 'Failed builds', - QuerysetFilter(Q(outcome=Build.FAILED)) + Q(outcome=Build.FAILED) ) outcome_filter.add_action(successful_builds_action)
@@ -1227,13 +1226,13 @@ class BuildsTable(ToasterTable): with_failed_tasks_action = TableFilterActionToggle( 'with_failed_tasks', 'Builds with failed tasks', - QuerysetFilter(criteria) + criteria ) without_failed_tasks_action = TableFilterActionToggle( 'without_failed_tasks', 'Builds without failed tasks', - QuerysetFilter(~criteria) + ~criteria ) failed_tasks_filter.add_action(with_failed_tasks_action)
bitbake/lib/toaster/toastergui/widgets.py +0 -1
@@ -32,7 +32,6 @@ from django.template import Context, Template from django.core.serializers.json import DjangoJSONEncoder from django.core.exceptions import FieldError from django.conf.urls import url, patterns -from toastergui.querysetfilter import QuerysetFilter import types import json
Signature · 14 lines
Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Quoted 4 lines · Elliot Smith
Convert some of the most important Toaster pages to ToasterTable rather than
individual templates.
I just realised two things I missed:
1. I added a new Python file, but neglected to add the licence header.
2. I left the old (commented-out) view code in views.py.
I'll fix (1) in the branch, but I'll leave (2) until after review, in case there's some code I need to port and have missed.
Elliot
Quoted 86 lines
This involved a rewrite of the ToasterTable filter system (see
tablefilter.py)
to cope with date range filtering for the builds tables.I also moved a lot of code out of the views into the model, where it seems better placed.
I also fixed a few JS issues which arose while testing, such as filters not de-selecting when there are multiple filters on a table; event handlers being lost when repopulating a table; and incorrect hiding of page elements due to table classes being used for the page title element.
Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8738
Also incidentally fixes
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8792,
as this functionality works correctly in ToasterTable.Changes since 6d35574 (toaster-next) are in
git://git.yoctoproject.org/poky-contrib, elliot/toaster/tables-8738http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/tables-8738
Elliot Smith (17):
toaster: toastergui Use ToasterTable for projects page
toaster: Move image file suffix list to model
toaster: Check inferred file suffixes against list of known types
toaster: toastergui Switch projects/ view to ToasterTable
toaster: toastergui Use event delegates for hover help elements
toaster: toastergui Convert all builds page to ToasterTable
toaster: toastergui Add QuerysetFilter class
toaster: toastergui Refactor ToasterTable filtering
toaster: toastergui Switch off filter highlights when inactive
toaster: toastergui Show recent builds on all builds page
toaster: toastergui Implement date range filters for builds
toaster: toastergui Implement "today" and "yesterday" filters
toaster: toastergui Convert project builds page to ToasterTable
toaster: tests Fix Django tests for new ToasterTable pages
toaster: toastergui Don't hide all elements with .col class
toaster: toastergui Ensurefilter_valueupdates
toaster: toastergui Streamline construction of filter objectsbitbake/lib/toaster/orm/models.py | 109 ++- .../lib/toaster/toastergui/static/js/libtoaster.js | 6 +- .../toaster/toastergui/static/js/projecttopbar.js | 9 + bitbake/lib/toaster/toastergui/static/js/table.js | 272 +++++-- bitbake/lib/toaster/toastergui/tablefilter.py | 271 +++++++ bitbake/lib/toaster/toastergui/tables.py | 826 +++++++++++++++++++-- .../toastergui/templates/baseprojectpage.html | 1 + .../toastergui/templates/builds-toastertable.html | 48 ++ .../toaster/toastergui/templates/mrb_section.html | 4 +- .../templates/projectbuilds-toastertable.html | 56 ++ .../templates/projects-toastertable.html | 36 + .../toaster/toastergui/templates/toastertable.html | 7 +- bitbake/lib/toaster/toastergui/tests.py | 255 ++++--- bitbake/lib/toaster/toastergui/urls.py | 14 +- bitbake/lib/toaster/toastergui/views.py | 79 +- bitbake/lib/toaster/toastergui/widgets.py | 94 ++- 16 files changed, 1781 insertions(+), 306 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/tablefilter.py create mode 100644 bitbake/lib/toaster/toastergui/templates/builds-toastertable.html create mode 100644 bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html create mode 100644bitbake/lib/toaster/toastergui/templates/projects-toastertable.html
--
Elliot Smith
Software Engineer
Intel OTC---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre