Skip to contentSkip to conversation

Convert builds/, projects/ and projectbuilds/ to ToasterTable

PATCH In review Elliot Smith 1 participant 19 messages Dec 18, 2015

lore
Share & tools
lore
#0 lore raw

cover Convert builds/, projects/ and projectbuilds/ to ToasterTable

Convert some of the most important Toaster pages to ToasterTable rather than individual templates.

This involved a rewrite of the ToasterTable filter system (see tablefilter.py) to cope with date range filtering for the builds tables.

I also moved a lot of code out of the views into the model, where it seems better placed.

I also fixed a few JS issues which arose while testing, such as filters not de-selecting when there are multiple filters on a table; event handlers being lost when repopulating a table; and incorrect hiding of page elements due to table classes being used for the page title element.

Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8738

Also incidentally fixes https://bugzilla.yoctoproject.org/show_bug.cgi?id=8792,
as this functionality works correctly in ToasterTable.

Changes since 6d35574 (toaster-next) are in
git://git.yoctoproject.org/poky-contrib, elliot/toaster/tables-8738
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/tables-8738

Elliot Smith (17):
toaster: toastergui Use ToasterTable for projects page
toaster: Move image file suffix list to model
toaster: Check inferred file suffixes against list of known types
toaster: toastergui Switch projects/ view to ToasterTable
toaster: toastergui Use event delegates for hover help elements
toaster: toastergui Convert all builds page to ToasterTable
toaster: toastergui Add QuerysetFilter class
toaster: toastergui Refactor ToasterTable filtering
toaster: toastergui Switch off filter highlights when inactive
toaster: toastergui Show recent builds on all builds page
toaster: toastergui Implement date range filters for builds
toaster: toastergui Implement "today" and "yesterday" filters
toaster: toastergui Convert project builds page to ToasterTable
toaster: tests Fix Django tests for new ToasterTable pages
toaster: toastergui Don't hide all elements with .col class
toaster: toastergui Ensure filter_value updates
toaster: toastergui Streamline construction of filter objects

 bitbake/lib/toaster/orm/models.py                  | 109 ++-
 .../lib/toaster/toastergui/static/js/libtoaster.js |   6 +-
 .../toaster/toastergui/static/js/projecttopbar.js  |   9 +
 bitbake/lib/toaster/toastergui/static/js/table.js  | 272 +++++--
 bitbake/lib/toaster/toastergui/tablefilter.py      | 271 +++++++
 bitbake/lib/toaster/toastergui/tables.py           | 826 +++++++++++++++++++--
 .../toastergui/templates/baseprojectpage.html      |   1 +
 .../toastergui/templates/builds-toastertable.html  |  48 ++
 .../toaster/toastergui/templates/mrb_section.html  |   4 +-
 .../templates/projectbuilds-toastertable.html      |  56 ++
 .../templates/projects-toastertable.html           |  36 +
 .../toaster/toastergui/templates/toastertable.html |   7 +-
 bitbake/lib/toaster/toastergui/tests.py            | 255 ++++---
 bitbake/lib/toaster/toastergui/urls.py             |  14 +-
 bitbake/lib/toaster/toastergui/views.py            |  79 +-
 bitbake/lib/toaster/toastergui/widgets.py          |  94 ++-
 16 files changed, 1781 insertions(+), 306 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/tablefilter.py
 create mode 100644 bitbake/lib/toaster/toastergui/templates/builds-toastertable.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/projects-toastertable.html

--
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

#1 lore raw

PATCH 1/17 toaster: toastergui Use ToasterTable for projects page

The projects page uses the old approach for showing tables, which means a template for each table. This means that applying changes to ToasterTable (which is used for most tables) has no effect on the layout, styling and behaviour for these older tables, and requires additional duplicated effort.

Move the projects page to use ToasterTable instead, to remove the duplication of effort.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/tables.py           | 226 ++++++++++++++++++++-
 .../templates/projects-toastertable.html           |  36 ++++
 bitbake/lib/toaster/toastergui/urls.py             |   6 +
 3 files changed, 265 insertions(+), 3 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/templates/projects-toastertable.html
bitbake/lib/toaster/toastergui/tables.py +223 -3
index 3808820..e5cab48 100644
@@ -21,13 +21,12 @@
 
 from toastergui.widgets import ToasterTable
 from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project
-from orm.models import CustomImageRecipe, Package
-from django.db.models import Q, Max
+from orm.models import CustomImageRecipe, Package, Build
+from django.db.models import Q, Max, Count
 from django.conf.urls import url
 from django.core.urlresolvers import reverse
 from django.views.generic import TemplateView
 
-
 class ProjectFiltersMixin(object):
     """Common mixin for recipe, machine in project filters"""
 
@@ -633,3 +632,224 @@ class SelectPackagesTable(ToasterTable):
                         "the package content of you custom image",
                         static_data_name="add_rm_pkg_btn",
                         static_data_template='{% include "pkg_add_rm_btn.html" %}')
+
+class ProjectsTable(ToasterTable):
+    """Table of projects in Toaster"""
+
+    def __init__(self, *args, **kwargs):
+        super(ProjectsTable, self).__init__(*args, **kwargs)
+        self.default_orderby = 'updated'
+        self.title = 'All projects'
+        self.static_context_extra['Build'] = Build
+
+    def get_context_data(self, **kwargs):
+        return super(ProjectsTable, self).get_context_data(**kwargs)
+
+    def setup_queryset(self, *args, **kwargs):
+        queryset = Project.objects.all()
+
+        # annotate each project with its number of builds
+        queryset = queryset.annotate(num_builds=Count('build'))
+
+        # exclude the command line builds project if it has no builds
+        q_default_with_builds = Q(is_default=True) & Q(num_builds__gt=0)
+        queryset = queryset.filter(Q(is_default=False) |
+                                   q_default_with_builds)
+
+        # order rows
+        queryset = queryset.order_by(self.default_orderby)
+
+        self.queryset = queryset
+
+    # columns: last activity on (updated) - DEFAULT, project (name), release, machine, number of builds, last build outcome, recipe (name), errors, warnings, image files
+    def setup_columns(self, *args, **kwargs):
+        name_template = '''
+        {% load project_url_tag %}
+        <span data-project-field="name">
+          <a href="{% project_url data %}">
+            {{data.name}}
+          </a>
+        </span>
+        '''
+
+        last_activity_on_template = '''
+        {% load project_url_tag %}
+        <span data-project-field="updated">
+          <a href="{% project_url data %}">
+            {{data.updated | date:"d/m/y H:i"}}
+          </a>
+        </span>
+        '''
+
+        release_template = '''
+        <span data-project-field="release">
+          {% if data.release %}
+            <a href="{% url 'project' data.id %}#project-details">
+                {{data.release.name}}
+            </a>
+          {% elif data.is_default %}
+            <span class="muted">Not applicable</span>
+            <i class="icon-question-sign get-help hover-help"
+               data-original-title="This project does not have a release set.
+               It simply collects information about the builds you start from
+               the command line while Toaster is running"
+               style="visibility: hidden;">
+            </i>
+          {% else %}
+            No release available
+          {% endif %}
+        </span>
+        '''
+
+        machine_template = '''
+        <span data-project-field="machine">
+          {% if data.is_default %}
+            <span class="muted">Not applicable</span>
+            <i class="icon-question-sign get-help hover-help"
+               data-original-title="This project does not have a machine
+               set. It simply collects information about the builds you
+               start from the command line while Toaster is running"
+               style="visibility: hidden;"></i>
+          {% else %}
+            <a href="{% url 'project' data.id %}#machine-distro">
+              {{data.get_current_machine_name}}
+            </a>
+          {% endif %}
+        </span>
+        '''
+
+        number_of_builds_template = '''
+        {% if data.get_number_of_builds > 0 %}
+          <a href="{% url 'projectbuilds' data.id %}">
+            {{data.get_number_of_builds}}
+          </a>
+        {% else %}
+          <span class="muted">0</span>
+        {% endif %}
+        '''
+
+        last_build_outcome_template = '''
+        {% if data.get_number_of_builds > 0 %}
+          <a href="{% url 'builddashboard' data.get_last_build_id %}">
+            {% if data.get_last_outcome == extra.Build.SUCCEEDED %}
+              <i class="icon-ok-sign success"></i>
+            {% elif data.get_last_outcome == extra.Build.FAILED %}
+              <i class="icon-minus-sign error"></i>
+            {% endif %}
+          </a>
+        {% endif %}
+        '''
+
+        recipe_template = '''
+        {% if data.get_number_of_builds > 0 %}
+          <a href="{% url "builddashboard" data.get_last_build_id %}">
+            {{data.get_last_target}}
+          </a>
+        {% endif %}
+        '''
+
+        errors_template = '''
+        {% if data.get_number_of_builds > 0 %}
+          <a class="errors.count error"
+             href="{% url "builddashboard" data.get_last_build_id %}#errors">
+            {{data.get_last_errors}} error{{data.get_last_errors | pluralize}}
+          </a>
+        {% endif %}
+        '''
+
+        warnings_template = '''
+        {% if data.get_number_of_builds > 0 %}
+          <a class="warnings.count warning"
+             href="{% url "builddashboard" data.get_last_build_id %}#warnings">
+            {{data.get_last_warnings}} warning{{data.get_last_warnings | pluralize}}
+          </a>
+        {% endif %}
+        '''
+
+        image_files_template = '''
+        {% load projecttags %}
+        {% if data.get_number_of_builds > 0 and data.get_last_outcome == extra.Build.SUCCEEDED %}
+          <a href="{% url "builddashboard" data.get_last_build_id %}#images">
+            {{fstypes | get_dict_value:data.id}}
+          </a>
+        {% endif %}
+        '''
+
+        self.add_column(title='Project',
+                        hideable=False,
+                        orderable=True,
+                        static_data_name='name',
+                        static_data_template=name_template)
+
+        self.add_column(title='Last activity on',
+                        help_text='Starting date and time of the \
+                                   last project build. If the project has no \
+                                   builds, this shows the date the project was \
+                                   created.',
+                        hideable=True,
+                        orderable=True,
+                        static_data_name='updated',
+                        static_data_template=last_activity_on_template)
+
+        self.add_column(title='Release',
+                        help_text='The version of the build system used by \
+                                   the project',
+                        hideable=False,
+                        orderable=True,
+                        static_data_name='release',
+                        static_data_template=release_template)
+
+        self.add_column(title='Machine',
+                        help_text='The hardware currently selected for the \
+                                   project',
+                        hideable=False,
+                        orderable=False,
+                        static_data_name='machine',
+                        static_data_template=machine_template)
+
+        self.add_column(title='Number of builds',
+                        help_text='The number of builds which have been run \
+                                   for the project',
+                        hideable=True,
+                        orderable=False,
+                        static_data_name='number_of_builds',
+                        static_data_template=number_of_builds_template)
+
+        self.add_column(title='Last build outcome',
+                        help_text='Indicates whether the last project build \
+                                   completed successfully or failed',
+                        hideable=True,
+                        orderable=False,
+                        static_data_name='last_build_outcome',
+                        static_data_template=last_build_outcome_template)
+
+        self.add_column(title='Recipe',
+                        help_text='The last recipe which was built in this \
+                                   project',
+                        hideable=True,
+                        orderable=False,
+                        static_data_name='recipe_name',
+                        static_data_template=recipe_template)
+
+        self.add_column(title='Errors',
+                        help_text='The number of errors encountered during \
+                                   the last project build (if any)',
+                        hideable=True,
+                        orderable=False,
+                        static_data_name='errors',
+                        static_data_template=errors_template)
+
+        self.add_column(title='Warnings',
+                        help_text='The number of warnings encountered during \
+                                   the last project build (if any)',
+                        hideable=True,
+                        orderable=False,
+                        static_data_name='warnings',
+                        static_data_template=warnings_template)
+
+        self.add_column(title='Image files',
+                        help_text='',
+                        hideable=True,
+                        orderable=False,
+                        static_data_name='image_files',
+                        static_data_template=image_files_template)
bitbake/lib/toaster/toastergui/templates/projects-toastertable.html new file +36 -0
new file mode 100644
index 0000000..5814f32
@@ -0,0 +1,36 @@
+{% extends 'base.html' %}
+
+{% block title %} All projects - Toaster {% endblock %}
+
+{% block pagecontent %}
+
+  <div class="page-header top-air">
+    <h1 data-role="page-title"></h1>
+  </div>
+
+  {% url 'projects' as xhr_table_url %}
+  {% include 'toastertable.html' %}
+
+  <script>
+    $(document).ready(function () {
+      var tableElt = $("#{{table_name}}");
+      var titleElt = $("[data-role='page-title']");
+
+      tableElt.on("table-done", function (e, total, tableParams) {
+        var title = "All projects";
+
+        if (tableParams.search || tableParams.filter) {
+          if (total === 0) {
+            title = "No projects found";
+          }
+          else if (total > 0) {
+            title = total + " project" + (total > 1 ? 's' : '') + " found";
+          }
+        }
+
+        titleElt.text(title);
+      });
+    });
+  </script>
+
+{% endblock %}
bitbake/lib/toaster/toastergui/urls.py +6 -0
index 2bf2d99..da97a31 100644
@@ -75,8 +75,14 @@ urlpatterns = patterns('toastergui.views',
         url(r'^newproject/$', 'newproject', name='newproject'),
 
 
+        # TODO remove when new toaster table is ready
         url(r'^projects/$', 'projects', name='all-projects'),
 
+        # TODO move to /projects/ when new toaster table is ready
+        url(r'^projects-new/$',
+            tables.ProjectsTable.as_view(template_name="projects-toastertable.html"),
+            name='all-projects-new'),
+
         url(r'^project/(?P<pid>\d+)/$', 'project', name='project'),
         url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'),
         url(r'^project/(?P<pid>\d+)/builds/$', 'projectbuilds', name='projectbuilds'),
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#2 lore raw

PATCH 2/17 toaster: Move image file suffix list to model

In reply to Elliot Smith

Image file suffixes are used in the project configuration page to show a list of available image file types. This list is stored as a function in the views code.

However, this list is also needed when parsing image file paths, so that the suffixes can be shown in the "all builds" and "project builds" tables.

Move the list of valid image file suffixes to the Target_Image_File class to make is accessible in other places where it may be needed.

[YOCTO #8738]

---
 bitbake/lib/toaster/orm/models.py       | 9 +++++++++
 bitbake/lib/toaster/toastergui/views.py | 7 +------
 2 files changed, 10 insertions(+), 6 deletions(-)
bitbake/lib/toaster/orm/models.py +9 -0
index 6e87c54..8d5f326 100644
@@ -461,6 +461,15 @@ class Target(models.Model):
         return self.target
 
 class Target_Image_File(models.Model):
+    # valid suffixes for image files produced by a build
+    SUFFIXES = {
+        'btrfs', 'cpio', 'cpio.gz', 'cpio.lz4', 'cpio.lzma', 'cpio.xz',
+        'cramfs', 'elf', 'ext2', 'ext2.bz2', 'ext2.gz', 'ext2.lzma', 'ext4',
+        'ext4.gz', 'ext3', 'ext3.gz', 'hddimg', 'iso', 'jffs2', 'jffs2.sum',
+        'squashfs', 'squashfs-lzo', 'squashfs-xz', 'tar.bz2', 'tar.lz4',
+        'tar.xz', 'tartar.gz', 'ubi', 'ubifs', 'vmdk'
+    }
+
     target = models.ForeignKey(Target)
     file_name = models.FilePathField(max_length=254)
     file_size = models.IntegerField()
bitbake/lib/toaster/toastergui/views.py +1 -6
index bd334b9..4cd7afd 100755
@@ -2808,12 +2808,7 @@ if True:
             'all_proxy','ftp_proxy','http_proxy ','https_proxy'
             }
 
-        vars_fstypes  = {
-            'btrfs','cpio','cpio.gz','cpio.lz4','cpio.lzma','cpio.xz','cramfs',
-            'elf','ext2','ext2.bz2','ext2.gz','ext2.lzma', 'ext4', 'ext4.gz', 'ext3','ext3.gz','hddimg',
-            'iso','jffs2','jffs2.sum','squashfs','squashfs-lzo','squashfs-xz','tar.bz2',
-            'tar.lz4','tar.xz','tartar.gz','ubi','ubifs','vmdk'
-        }
+        vars_fstypes = Target_Image_File.SUFFIXES
 
         return(vars_managed,sorted(vars_fstypes),vars_blacklist)
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#3 lore raw

PATCH 3/17 toaster: Check inferred file suffixes against list of known types

In reply to Elliot Smith

The algorithm for finding the suffix for image files produced by the build doesn't reference a list of known file suffixes, so could be prone to error.

Modify how file suffixes are parsed from the file path so that they are compared against a list of known types; if this fails, use the part of the basename of the file path after the first '.' character.

Also rationalise the places in the views code where we extract the file name extensions for builds, so they both use the same algorithm (before, the same code was duplicated in two places).

[YOCTO #8417]

---
 bitbake/lib/toaster/orm/models.py        | 68 ++++++++++++++++++++++++++++++--
 bitbake/lib/toaster/toastergui/tables.py | 10 +++--
 bitbake/lib/toaster/toastergui/views.py  | 38 ++----------------
 3 files changed, 75 insertions(+), 41 deletions(-)
bitbake/lib/toaster/orm/models.py +65 -3
index 8d5f326..052dbae 100644
@@ -29,6 +29,9 @@ from django.core import validators
 from django.conf import settings
 import django.db.models.signals
 
+import os.path
+import re
+
 import logging
 logger = logging.getLogger("toaster")
 
@@ -234,6 +237,14 @@ class Project(models.Model):
         except (Build.DoesNotExist,IndexError):
             return( "not_found" )
 
+    def get_last_build_extensions(self):
+        """
+        Get list of file name extensions for images produced by the most
+        recent build
+        """
+        last_build = Build.objects.get(pk = self.get_last_build_id())
+        return last_build.get_image_file_extensions()
+
     def get_last_imgfiles(self):
         build_id = self.get_last_build_id
         if (-1 == build_id):
@@ -376,6 +387,57 @@ class Build(models.Model):
             eta += ((eta - self.started_on)*(100-completeper))/completeper
         return eta
 
+    def get_image_file_extensions(self):
+        """
+        Get list of file name extensions for images produced by this build
+        """
+        targets = Target.objects.filter(build_id = self.id)
+        extensions = []
+
+        # pattern to match against file path for building extension string
+        pattern = re.compile('\.([^\.]+?)$')
+
+        for target in targets:
+            if (not target.is_image):
+                continue
+
+            target_image_files = Target_Image_File.objects.filter(target_id = target.id)
+
+            for target_image_file in target_image_files:
+                file_name = os.path.basename(target_image_file.file_name)
+                suffix = ''
+
+                continue_matching = True
+
+                # incrementally extract the suffix from the file path,
+                # checking it against the list of valid suffixes at each
+                # step; if the path is stripped of all potential suffix
+                # parts without matching a valid suffix, this returns all
+                # characters after the first '.' in the file name
+                while continue_matching:
+                    matches = pattern.search(file_name)
+
+                    if None == matches:
+                        continue_matching = False
+                        suffix = re.sub('^\.', '', suffix)
+                        continue
+                    else:
+                        suffix = matches.group(1) + suffix
+
+                    if suffix in Target_Image_File.SUFFIXES:
+                        continue_matching = False
+                        continue
+                    else:
+                        # reduce the file name and try to find the next
+                        # segment from the path which might be part
+                        # of the suffix
+                        file_name = re.sub('.' + matches.group(1), '', file_name)
+                        suffix = '.' + suffix
+
+                if not suffix in extensions:
+                    extensions.append(suffix)
+
+        return ', '.join(extensions)
 
     def get_sorted_target_list(self):
         tgts = Target.objects.filter(build_id = self.id).order_by( 'target' );
@@ -418,7 +480,7 @@ class Build(models.Model):
             return self.get_outcome_text()
 
     def __str__(self):
-        return "%s %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
+        return "%d %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
 
 
 # an Artifact is anything that results from a Build, and may be of interest to the user, and is not stored elsewhere
@@ -609,7 +671,7 @@ class Task(models.Model):
     sstate_text  = property(get_sstate_text)
 
     def __unicode__(self):
-        return "%s(%s) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name)
+        return "%d(%d) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name)
 
     class Meta:
         ordering = ('order', 'recipe' ,)
@@ -1260,7 +1322,7 @@ class Layer_Version(models.Model):
         return sorted(result, key=lambda x: x.layer.name)
 
     def __unicode__(self):
-        return "%s %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
+        return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
 
     class Meta:
         unique_together = ("layer_source", "up_id")
bitbake/lib/toaster/toastergui/tables.py +6 -4
index e5cab48..0b1d8a2 100644
@@ -661,7 +661,9 @@ class ProjectsTable(ToasterTable):
 
         self.queryset = queryset
 
-    # columns: last activity on (updated) - DEFAULT, project (name), release, machine, number of builds, last build outcome, recipe (name), errors, warnings, image files
+    # columns: last activity on (updated) - DEFAULT, project (name), release,
+    # machine, number of builds, last build outcome, recipe (name), errors,
+    # warnings, image files
     def setup_columns(self, *args, **kwargs):
         name_template = '''
         {% load project_url_tag %}
@@ -767,10 +769,9 @@ class ProjectsTable(ToasterTable):
         '''
 
         image_files_template = '''
-        {% load projecttags %}
         {% if data.get_number_of_builds > 0 and data.get_last_outcome == extra.Build.SUCCEEDED %}
           <a href="{% url "builddashboard" data.get_last_build_id %}#images">
-            {{fstypes | get_dict_value:data.id}}
+            {{data.get_last_build_extensions}}
           </a>
         {% endif %}
         '''
@@ -848,7 +849,8 @@ class ProjectsTable(ToasterTable):
                         static_data_template=warnings_template)
 
         self.add_column(title='Image files',
-                        help_text='',
+                        help_text='The root file system types produced by \
+                                   the last project build',
                         hideable=True,
                         orderable=False,
                         static_data_name='image_files',
bitbake/lib/toaster/toastergui/views.py +4 -34
index 4cd7afd..8148623 100755
@@ -2008,23 +2008,10 @@ if True:
         context_date,today_begin,yesterday_begin = _add_daterange_context(queryset_all, request, {'started_on','completed_on'})
 
         # set up list of fstypes for each build
-        fstypes_map = {};
+        fstypes_map = {}
+
         for build in build_info:
-            targets = Target.objects.filter( build_id = build.id )
-            comma = "";
-            extensions = "";
-            for t in targets:
-                if ( not t.is_image ):
-                    continue
-                tif = Target_Image_File.objects.filter( target_id = t.id )
-                for i in tif:
-                    s=re.sub('.*tar.bz2', 'tar.bz2', i.file_name)
-                    if s == i.file_name:
-                        s=re.sub('.*\.', '', i.file_name)
-                    if None == re.search(s,extensions):
-                        extensions += comma + s
-                        comma = ", "
-            fstypes_map[build.id]=extensions
+            fstypes_map[build.id] = build.get_image_file_extensions()
 
         # send the data to the template
         context = {
@@ -3047,24 +3034,7 @@ if True:
         # translate the project's build target strings
         fstypes_map = {};
         for project in project_info:
-            try:
-                targets = Target.objects.filter( build_id = project.get_last_build_id() )
-                comma = "";
-                extensions = "";
-                for t in targets:
-                    if ( not t.is_image ):
-                        continue
-                    tif = Target_Image_File.objects.filter( target_id = t.id )
-                    for i in tif:
-                        s=re.sub('.*tar.bz2', 'tar.bz2', i.file_name)
-                        if s == i.file_name:
-                            s=re.sub('.*\.', '', i.file_name)
-                        if None == re.search(s,extensions):
-                            extensions += comma + s
-                            comma = ", "
-                fstypes_map[project.id]=extensions
-            except (Target.DoesNotExist,IndexError):
-                fstypes_map[project.id]=project.get_last_imgfiles
+            fstypes_map[project.id] = project.get_last_build_extensions()
 
         context = {
                 'mru' : build_mru,
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#4 lore raw

PATCH 4/17 toaster: toastergui Switch projects/ view to ToasterTable

In reply to Elliot Smith

Remove the old projects page and replace with the new ToasterTable-based version.

NB although the projects.html template is no longer required, it's been left in as there will be changes applied to it for the new theme. These changes will have to then be transferred from the projects.html template to projects-toastertable.html.

Similarly, the code for the projects page in views.py has been retained.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/urls.py  |  9 ++-------
 bitbake/lib/toaster/toastergui/views.py | 24 ++++++++++++++++++++----
 2 files changed, 22 insertions(+), 11 deletions(-)
bitbake/lib/toaster/toastergui/urls.py +2 -7
index da97a31..b5e9a05 100644
@@ -74,14 +74,9 @@ urlpatterns = patterns('toastergui.views',
         # project URLs
         url(r'^newproject/$', 'newproject', name='newproject'),
 
-
-        # TODO remove when new toaster table is ready
-        url(r'^projects/$', 'projects', name='all-projects'),
-
-        # TODO move to /projects/ when new toaster table is ready
-        url(r'^projects-new/$',
+        url(r'^projects/$',
             tables.ProjectsTable.as_view(template_name="projects-toastertable.html"),
-            name='all-projects-new'),
+            name='all-projects'),
 
         url(r'^project/(?P<pid>\d+)/$', 'project', name='project'),
         url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'),
bitbake/lib/toaster/toastergui/views.py +20 -4
index 8148623..a79261d 100755
@@ -2990,9 +2990,7 @@ if True:
             }
             return render(request, "unavailable_artifact.html", context)
 
-
-
-
+    """
     @_template_renderer("projects.html")
     def projects(request):
         (pagesize, orderby) = _get_parameters_values(request, 10, 'updated:-')
@@ -3034,7 +3032,24 @@ if True:
         # translate the project's build target strings
         fstypes_map = {};
         for project in project_info:
-            fstypes_map[project.id] = project.get_last_build_extensions()
+            try:
+                targets = Target.objects.filter( build_id = project.get_last_build_id() )
+                comma = "";
+                extensions = "";
+                for t in targets:
+                    if ( not t.is_image ):
+                        continue
+                    tif = Target_Image_File.objects.filter( target_id = t.id )
+                    for i in tif:
+                        s=re.sub('.*tar.bz2', 'tar.bz2', i.file_name)
+                        if s == i.file_name:
+                            s=re.sub('.*\.', '', i.file_name)
+                        if None == re.search(s,extensions):
+                            extensions += comma + s
+                            comma = ", "
+                fstypes_map[project.id]=extensions
+            except (Target.DoesNotExist,IndexError):
+                fstypes_map[project.id]=project.get_last_imgfiles
 
         context = {
                 'mru' : build_mru,
@@ -3092,3 +3107,4 @@ if True:
 
         _set_parameters_values(pagesize, orderby, request)
         return context
+    """
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#5 lore raw

PATCH 5/17 toaster: toastergui Use event delegates for hover help elements

In reply to Elliot Smith

libtoaster.js binds to hover help elements via their hover() and mouseout() methods. However, any elements added to the DOM after libtoaster has initialised will not have these bindings added. This causes a problem for ToasterTables which have hover-help elements (e.g. the builds/ table).

Use the on() method instead. This uses event delegation to bind a handler to any th or td elements already in the DOM, or which will be added to the DOM in future. ToasterTables can now reconstruct the table DOM and still have the correct handlers attached once the table is done.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/static/js/libtoaster.js | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
bitbake/lib/toaster/toastergui/static/js/libtoaster.js +4 -2
index c04f7ab..1012034 100644
@@ -448,10 +448,12 @@ $(document).ready(function() {
 
     // show help bubble only on hover inside tables
     $(".hover-help").css("visibility","hidden");
-    $("th, td").hover(function () {
+
+    $("table").on("mouseover", "th, td", function () {
         $(this).find(".hover-help").css("visibility","visible");
     });
-    $("th, td").mouseleave(function () {
+
+    $("table").on("mouseleave", "th, td", function () {
         $(this).find(".hover-help").css("visibility","hidden");
     });
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#6 lore raw

PATCH 6/17 toaster: toastergui Convert all builds page to ToasterTable

In reply to Elliot Smith

For better long-term maintainability, use ToasterTable instead of Django template and view code to display the all builds page.

NB the builds.html template has been left in, as this will otherwise cause conflicts when merging the new theme.

[YOCTO #8738]

---
 bitbake/lib/toaster/orm/models.py                  |  32 ++-
 bitbake/lib/toaster/toastergui/tables.py           | 311 ++++++++++++++++++++-
 .../toastergui/templates/builds-toastertable.html  |  62 ++++
 bitbake/lib/toaster/toastergui/urls.py             |   5 +-
 bitbake/lib/toaster/toastergui/views.py            |  32 ---
 5 files changed, 407 insertions(+), 35 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/templates/builds-toastertable.html
bitbake/lib/toaster/orm/models.py +31 -1
index 052dbae..933527d 100644
@@ -447,6 +447,12 @@ class Build(models.Model):
         return Build.BUILD_OUTCOME[int(self.outcome)][1]
 
     @property
+    def failed_tasks(self):
+        """ Get failed tasks for the build """
+        tasks = self.task_build.all()
+        return tasks.filter(order__gt=0, outcome=Task.OUTCOME_FAILED)
+
+    @property
     def errors(self):
         return (self.logmessage_set.filter(level=LogMessage.ERROR) |
                 self.logmessage_set.filter(level=LogMessage.EXCEPTION) |
@@ -457,8 +463,32 @@ class Build(models.Model):
         return self.logmessage_set.filter(level=LogMessage.WARNING)
 
     @property
+    def timespent(self):
+        return self.completed_on - self.started_on
+
+    @property
     def timespent_seconds(self):
-        return (self.completed_on - self.started_on).total_seconds()
+        return self.timespent.total_seconds()
+
+    @property
+    def target_labels(self):
+        """
+        Sorted (a-z) "target1:task, target2, target3" etc. string for all
+        targets in this build
+        """
+        targets = self.target_set.all()
+        target_labels = []
+        target_label = None
+
+        for target in targets:
+            target_label = target.target
+            if target.task:
+                target_label = target_label + ':' + target.task
+            target_labels.append(target_label)
+
+        target_labels.sort()
+
+        return target_labels
 
     def get_current_status(self):
         """
bitbake/lib/toaster/toastergui/tables.py +310 -1

320 lines not on this page · raw message

bitbake/lib/toaster/toastergui/templates/builds-toastertable.html new file +62 -0

62 lines not on this page · raw message

bitbake/lib/toaster/toastergui/urls.py +4 -1
index b5e9a05..707b7d5 100644
@@ -27,7 +27,10 @@ urlpatterns = patterns('toastergui.views',
         # landing page
         url(r'^landing/$', 'landing', name='landing'),
 
-        url(r'^builds/$', 'builds', name='all-builds'),
+        url(r'^builds/$',
+            tables.BuildsTable.as_view(template_name="builds-toastertable.html"),
+            name='all-builds'),
+
         # build info navigation
         url(r'^build/(?P<build_id>\d+)$', 'builddashboard', name="builddashboard"),
 
bitbake/lib/toaster/toastergui/views.py +0 -32
index a79261d..295773f 100755
@@ -1915,34 +1915,6 @@ if True:
         ''' The exception raised on invalid POST requests '''
         pass
 
-    # shows the "all builds" page for managed mode; it displays build requests (at least started!) instead of actual builds
-    # WARNING _build_list_helper() may raise a RedirectException, which
-    # will set the GET parameters and redirect back to the
-    # all-builds or projectbuilds page as appropriate;
-    # TODO don't use exceptions to control program flow
-    @_template_renderer("builds.html")
-    def builds(request):
-        # define here what parameters the view needs in the GET portion in order to
-        # be able to display something.  'count' and 'page' are mandatory for all views
-        # that use paginators.
-
-        queryset = Build.objects.all()
-
-        redirect_page = resolve(request.path_info).url_name
-
-        context, pagesize, orderby = _build_list_helper(request,
-                                                        queryset,
-                                                        redirect_page)
-        # all builds page as a Project column
-        context['tablecols'].append({
-            'name': 'Project',
-            'clclass': 'project_column'
-        })
-
-        _set_parameters_values(pagesize, orderby, request)
-        return context
-
-
     # helper function, to be used on "all builds" and "project builds" pages
     def _build_list_helper(request, queryset_all, redirect_page, pid=None):
         default_orderby = 'completed_on:-'
@@ -1986,10 +1958,6 @@ if True:
             warnings_no = Count('logmessage', only=q_warnings)
         )
 
-        # add timespent field
-        timespent = 'completed_on - started_on'
-        queryset_all = queryset_all.extra(select={'timespent': timespent})
-
         queryset_with_search = _get_queryset(Build, queryset_all,
                                              None, search_term,
                                              ordering_string, '-completed_on')
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#7 lore raw

PATCH 7/17 toaster: toastergui Add QuerysetFilter class

In reply to Elliot Smith

Filters are added to tables by passing a list of function to the ToasterTable.add_filters() method. This means a lot of code is duplicated.

Add a QuerysetFilter class which encapsulates a filter for a Queryset and provides a generic count() method.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/querysetfilter.py |  24 +++++
 bitbake/lib/toaster/toastergui/tables.py         | 108 +++++++----------------
 bitbake/lib/toaster/toastergui/widgets.py        |  16 ++--
 3 files changed, 65 insertions(+), 83 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/querysetfilter.py
bitbake/lib/toaster/toastergui/querysetfilter.py new file +24 -0
new file mode 100644
index 0000000..62297e9
@@ -0,0 +1,24 @@
+class QuerysetFilter(object):
+    """ Filter for a queryset """
+
+    def __init__(self, criteria=None):
+        if criteria:
+            self.set_criteria(criteria)
+
+    def set_criteria(self, criteria):
+        """
+        criteria is an instance of django.db.models.Q;
+        see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects
+        """
+        self.criteria = criteria
+
+    def filter(self, queryset):
+        """
+        Filter queryset according to the criteria for this filter,
+        returning the filtered queryset
+        """
+        return queryset.filter(self.criteria)
+
+    def count(self, queryset):
+        """ Returns a count of the elements in the filtered queryset """
+        return self.filter(queryset).count()
bitbake/lib/toaster/toastergui/tables.py +30 -78

196 lines not on this page · raw message

bitbake/lib/toaster/toastergui/widgets.py +11 -5
index 6bb3889..71b29ea 100644
@@ -32,6 +32,7 @@ from django.template import Context, Template
 from django.core.serializers.json import DjangoJSONEncoder
 from django.core.exceptions import FieldError
 from django.conf.urls import url, patterns
+from toastergui.querysetfilter import QuerysetFilter
 
 import types
 import json
@@ -113,7 +114,8 @@ class ToasterTable(TemplateView):
                               cls=DjangoJSONEncoder)
         else:
             for actions in self.filters[name]['filter_actions']:
-                actions['count'] = self.filter_actions[actions['name']](count_only=True)
+                queryset_filter = self.filter_actions[actions['name']]
+                actions['count'] = queryset_filter.count(self.queryset)
 
             # Add the "All" items filter action
             self.filters[name]['filter_actions'].insert(0, {
@@ -151,15 +153,18 @@ class ToasterTable(TemplateView):
           'filter_actions' : filter_actions,
         }
 
-    def make_filter_action(self, name, title, action_function):
-        """ Utility to make a filter_action """
+    def make_filter_action(self, name, title, queryset_filter):
+        """
+        Utility to make a filter_action; queryset_filter is an instance
+        of QuerysetFilter or a function
+        """
 
         action = {
           'title' : title,
           'name' : name,
         }
 
-        self.filter_actions[name] = action_function
+        self.filter_actions[name] = queryset_filter
 
         return action
 
@@ -222,7 +227,8 @@ class ToasterTable(TemplateView):
             return
 
         try:
-            self.filter_actions[filter_action]()
+            queryset_filter = self.filter_actions[filter_action]
+            self.queryset = queryset_filter.filter(self.queryset)
         except KeyError:
             # pass it to the user - programming error here
             raise
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#8 lore raw

PATCH 8/17 toaster: toastergui Refactor ToasterTable filtering

In reply to Elliot Smith

The filter code for ToasterTable was difficult to follow and inflexible (not allowing different types of filter, for example).

Refactor to a set of filter classes to make the structure cleaner and provide the flexibility needed for other filter types (e.g. date range filter).

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/querysetfilter.py  |   7 +-
 bitbake/lib/toaster/toastergui/static/js/table.js |  80 +++++++++----
 bitbake/lib/toaster/toastergui/tablefilter.py     |  98 ++++++++++++++++
 bitbake/lib/toaster/toastergui/tables.py          | 132 ++++++++++++++--------
 bitbake/lib/toaster/toastergui/widgets.py         |  90 +++++++--------
 5 files changed, 289 insertions(+), 118 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/tablefilter.py
bitbake/lib/toaster/toastergui/querysetfilter.py +5 -2
index 62297e9..dbae239 100644
@@ -5,7 +5,7 @@ class QuerysetFilter(object):
         if criteria:
             self.set_criteria(criteria)
 
-    def set_criteria(self, criteria):
+    def set_criteria(self, criteria = None):
         """
         criteria is an instance of django.db.models.Q;
         see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects
@@ -17,7 +17,10 @@ class QuerysetFilter(object):
         Filter queryset according to the criteria for this filter,
         returning the filtered queryset
         """
-        return queryset.filter(self.criteria)
+        if self.criteria:
+            return queryset.filter(self.criteria)
+        else:
+            return queryset
 
     def count(self, queryset):
         """ Returns a count of the elements in the filtered queryset """
bitbake/lib/toaster/toastergui/static/js/table.js +59 -21

97 lines not on this page · raw message

bitbake/lib/toaster/toastergui/tablefilter.py new file +98 -0

98 lines not on this page · raw message

bitbake/lib/toaster/toastergui/tables.py +87 -45

180 lines not on this page · raw message

bitbake/lib/toaster/toastergui/widgets.py +40 -50

144 lines not on this page · raw message

Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#9 lore raw

PATCH 9/17 toaster: toastergui Switch off filter highlights when inactive

In reply to Elliot Smith

In ToasterTables with multiple columns which allow filtering (e.g. all builds), selecting one filter, then a second filter (e.g. selecting "failed builds" then "outcome" for all builds), would result in both filters being highlighted at the same time.

Fix this by removing the "active" highlight on all column filter buttons when a new filter value is submitted (via the filter modal).

NB to enable this, added a data-filter-on attribute to all column filter buttons to make them easy to select.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/static/js/table.js | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
bitbake/lib/toaster/toastergui/static/js/table.js +7 -1
index fa01ddf..63f8a1f 100644
@@ -248,7 +248,7 @@ function tableInit(ctx){
 
       /* Setup the filter button */
       if (col.filter_name){
-        var filterBtn = $('<a href="#" role="button" class="pull-right btn btn-mini" data-toggle="modal"><i class="icon-filter filtered"></i></a>');
+        var filterBtn = $('<a href="#" role="button" data-filter-on="' + col.filter_name + '" class="pull-right btn btn-mini" data-toggle="modal"><i class="icon-filter filtered"></i></a>');
 
         filterBtn.data('filter-name', col.filter_name);
         filterBtn.prop('id', col.filter_name);
@@ -565,6 +565,12 @@ function tableInit(ctx){
   $("#filter-modal-form-"+ctx.tableName).submit(function(e){
     e.preventDefault();
 
+    /* remove active status from all filter buttons so that only one filter
+       can be active at a time */
+    $('[data-filter-on]').each(function (index, filterBtn) {
+      filterBtnActive($(filterBtn), false);
+    });
+
     tableParams.filter = $(this).find("input[type='radio']:checked").val();
 
     var filterBtn = $("#" + tableParams.filter.split(":")[0]);
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#10 lore raw

PATCH 10/17 toaster: toastergui Show recent builds on all builds page

In reply to Elliot Smith

The recent builds section was disabled while converting the all builds page to ToasterTable.

Re-enable the recent builds area and add the data it requires to the ToasterTable context.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/tables.py               | 18 +++++++++++++++++-
 .../toastergui/templates/builds-toastertable.html      | 10 +++++-----
 .../lib/toaster/toastergui/templates/mrb_section.html  |  2 +-
 3 files changed, 23 insertions(+), 7 deletions(-)
bitbake/lib/toaster/toastergui/tables.py +17 -1
index 61ea9cb..22a5a6a 100644
@@ -27,6 +27,7 @@ from django.db.models import Q, Max, Count
 from django.conf.urls import url
 from django.core.urlresolvers import reverse
 from django.views.generic import TemplateView
+import itertools
 
 from toastergui.tablefilter import TableFilter, TableFilterActionToggle
 
@@ -889,7 +890,22 @@ class BuildsTable(ToasterTable):
         self.static_context_extra['Task'] = Task
 
     def get_context_data(self, **kwargs):
-        return super(BuildsTable, self).get_context_data(**kwargs)
+        context = super(BuildsTable, self).get_context_data(**kwargs)
+
+        # for the latest builds section
+        queryset = Build.objects.all()
+
+        finished_criteria = Q(outcome=Build.SUCCEEDED) | Q(outcome=Build.FAILED)
+
+        latest_builds = itertools.chain(
+            queryset.filter(outcome=Build.IN_PROGRESS).order_by("-started_on"),
+            queryset.filter(finished_criteria).order_by("-completed_on")[:3]
+        )
+
+        context['mru'] = list(latest_builds)
+        context['mrb_type'] = 'all'
+
+        return context
 
     def setup_queryset(self, *args, **kwargs):
         queryset = Build.objects.all()
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html +5 -5
index 419d2b5..f7604fd 100644
@@ -3,13 +3,13 @@
 {% block title %} All builds - Toaster {% endblock %}
 
 {% block pagecontent %}
-  <div class="page-header top-air">
-    <h1 data-role="page-title"></h1>
-  </div>
 
   <div class="row-fluid">
-    {# TODO need to pass this data to context #}
-    {#% include 'mrb_section.html' %#}
+    {% with mru=mru mrb_type=mrb_type %}
+      {% include 'mrb_section.html' %}
+    {% endwith %}
+
+    <h1  class="page-header top-air" data-role="page-title"></h1>
 
     {% url 'builds' as xhr_table_url %}
     {% include 'toastertable.html' %}
bitbake/lib/toaster/toastergui/templates/mrb_section.html +1 -1
index bd8f991..52b3f1a 100644
@@ -14,7 +14,7 @@
       {% endif %}
       </h2>
   {% else %}
-    <div class="page-header">
+    <div class="page-header top-air">
       <h1>
       Latest builds
       </h1>
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#11 lore raw

PATCH 11/17 toaster: toastergui Implement date range filters for builds

In reply to Elliot Smith

Implement the completed_on and started_on filtering for builds.

Also separate the name of a filter ("filter" in the querystring) from its value ("filter_value" in the querystring). This enables filtering to be defined in the querystring more intuitively, and also makes it easier to add other types of filter (e.g. by day).

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/querysetfilter.py   |   3 +-
 bitbake/lib/toaster/toastergui/static/js/table.js  | 192 +++++++++++++++++----
 bitbake/lib/toaster/toastergui/tablefilter.py      | 114 ++++++++++--
 bitbake/lib/toaster/toastergui/tables.py           |  38 +++-
 .../toastergui/templates/builds-toastertable.html  |  32 +---
 bitbake/lib/toaster/toastergui/widgets.py          |  32 ++--
 6 files changed, 329 insertions(+), 82 deletions(-)
bitbake/lib/toaster/toastergui/querysetfilter.py +2 -1
index dbae239..efa8507 100644
@@ -2,10 +2,11 @@ class QuerysetFilter(object):
     """ Filter for a queryset """
 
     def __init__(self, criteria=None):
+        self.criteria = None
         if criteria:
             self.set_criteria(criteria)
 
-    def set_criteria(self, criteria = None):
+    def set_criteria(self, criteria):
         """
         criteria is an instance of django.db.models.Q;
         see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects
bitbake/lib/toaster/toastergui/static/js/table.js +163 -29

222 lines not on this page · raw message

bitbake/lib/toaster/toastergui/tablefilter.py +102 -12

152 lines not on this page · raw message

bitbake/lib/toaster/toastergui/tables.py +37 -1
index 22a5a6a..a9efc0c 100644
@@ -29,7 +29,9 @@ from django.core.urlresolvers import reverse
 from django.views.generic import TemplateView
 import itertools
 
-from toastergui.tablefilter import TableFilter, TableFilterActionToggle
+from toastergui.tablefilter import TableFilter
+from toastergui.tablefilter import TableFilterActionToggle
+from toastergui.tablefilter import TableFilterActionDateRange
 
 class ProjectFilters(object):
     def __init__(self, project_layers):
@@ -1072,6 +1074,7 @@ class BuildsTable(ToasterTable):
                         help_text='The date and time when the build started',
                         hideable=True,
                         orderable=True,
+                        filter_name='started_on_filter',
                         static_data_name='started_on',
                         static_data_template=started_on_template)
 
@@ -1079,6 +1082,7 @@ class BuildsTable(ToasterTable):
                         help_text='The date and time when the build finished',
                         hideable=False,
                         orderable=True,
+                        filter_name='completed_on_filter',
                         static_data_name='completed_on',
                         static_data_template=completed_on_template)
 
@@ -1151,6 +1155,38 @@ class BuildsTable(ToasterTable):
         outcome_filter.add_action(failed_builds_filter_action)
         self.add_filter(outcome_filter)
 
+        # started on
+        started_on_filter = TableFilter(
+            'started_on_filter',
+            'Filter by date when build was started'
+        )
+
+        by_started_date_range_filter_action = TableFilterActionDateRange(
+            'date_range',
+            'Build date range',
+            'started_on',
+            QuerysetFilter()
+        )
+
+        started_on_filter.add_action(by_started_date_range_filter_action)
+        self.add_filter(started_on_filter)
+
+        # completed on
+        completed_on_filter = TableFilter(
+            'completed_on_filter',
+            'Filter by date when build was completed'
+        )
+
+        by_completed_date_range_filter_action = TableFilterActionDateRange(
+            'date_range',
+            'Build date range',
+            'completed_on',
+            QuerysetFilter()
+        )
+
+        completed_on_filter.add_action(by_completed_date_range_filter_action)
+        self.add_filter(completed_on_filter)
+
         # failed tasks
         failed_tasks_filter = TableFilter(
             'failed_tasks_filter',
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html +9 -23
index f7604fd..2e32edb 100644
@@ -1,4 +1,13 @@
 {% extends 'base.html' %}
+{% load static %}
+
+{% block extraheadcontent %}
+  <link rel="stylesheet" href="{% static 'css/jquery-ui.min.css' %}" type='text/css'>
+  <link rel="stylesheet" href="{% static 'css/jquery-ui.structure.min.css' %}" type='text/css'>
+  <link rel="stylesheet" href="{% static 'css/jquery-ui.theme.min.css' %}" type='text/css'>
+  <script src="{% static 'js/jquery-ui.min.js' %}">
+  </script>
+{% endblock %}
 
 {% block title %} All builds - Toaster {% endblock %}
 
@@ -34,29 +43,6 @@
 
         titleElt.text(title);
       });
-
-      /* {% if last_date_from and last_date_to %}
-      // TODO initialize the date range controls;
-      // this will need to be added via ToasterTable
-      date_init(
-        "started_on",
-        "{{last_date_from}}",
-        "{{last_date_to}}",
-        "{{dateMin_started_on}}",
-        "{{dateMax_started_on}}",
-        "{{daterange_selected}}"
-      );
-
-      date_init(
-        "completed_on",
-        "{{last_date_from}}",
-        "{{last_date_to}}",
-        "{{dateMin_completed_on}}",
-        "{{dateMax_completed_on}}",
-        "{{daterange_selected}}"
-      );
-      {% endif %}
-      */
     });
   </script>
 {% endblock %}
bitbake/lib/toaster/toastergui/widgets.py +16 -16
index 8790340..47de30d 100644
@@ -183,13 +183,13 @@ class ToasterTable(TemplateView):
 
         return template.render(context)
 
-    def apply_filter(self, filters, **kwargs):
+    def apply_filter(self, filters, filter_value, **kwargs):
         """
         Apply a filter submitted in the querystring to the ToasterTable
 
         filters: (str) in the format:
-          '<filter name>:<action name>!<action params>'
-        where <action params> is optional
+          '<filter name>:<action name>'
+        filter_value: (str) parameters to pass to the named filter
 
         <filter name> and <action name> are used to look up the correct filter
         in the ToasterTable's filter map; the <action params> are set on
@@ -199,15 +199,8 @@ class ToasterTable(TemplateView):
         self.setup_filters(**kwargs)
 
         try:
-            filter_name, action_name_and_params = filters.split(':')
-
-            action_name = None
-            action_params = None
-            if re.search('!', action_name_and_params):
-                action_name, action_params = action_name_and_params.split('!')
-                action_params = urllib.unquote_plus(action_params)
-            else:
-                action_name = action_name_and_params
+            filter_name, action_name = filters.split(':')
+            action_params = urllib.unquote_plus(filter_value)
         except ValueError:
             return
 
@@ -217,7 +210,7 @@ class ToasterTable(TemplateView):
         try:
             table_filter = self.filter_map.get_filter(filter_name)
             action = table_filter.get_action(action_name)
-            action.set_params(action_params)
+            action.set_filter_params(action_params)
             self.queryset = action.filter(self.queryset)
         except KeyError:
             # pass it to the user - programming error here
@@ -247,13 +240,20 @@ class ToasterTable(TemplateView):
 
 
     def get_data(self, request, **kwargs):
-        """Returns the data for the page requested with the specified
-        parameters applied"""
+        """
+        Returns the data for the page requested with the specified
+        parameters applied
+
+        filters: filter and action name, e.g. "outcome:build_succeeded"
+        filter_value: value to pass to the named filter+action, e.g. "on"
+        (for a toggle filter) or "2015-12-11,2015-12-12" (for a date range filter)
+        """
 
         page_num = request.GET.get("page", 1)
         limit = request.GET.get("limit", 10)
         search = request.GET.get("search", None)
         filters = request.GET.get("filter", None)
+        filter_value = request.GET.get("filter_value", "on")
         orderby = request.GET.get("orderby", None)
         nocache = request.GET.get("nocache", None)
 
@@ -285,7 +285,7 @@ class ToasterTable(TemplateView):
         if search:
             self.apply_search(search)
         if filters:
-            self.apply_filter(filters, **kwargs)
+            self.apply_filter(filters, filter_value, **kwargs)
         if orderby:
             self.apply_orderby(orderby)
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#12 lore raw

PATCH 12/17 toaster: toastergui Implement "today" and "yesterday" filters

In reply to Elliot Smith

Add the "today" and "yesterday" filters to the started_on and completed_on columns in the builds table.

During this work, some minor adjustments were made to the behaviour of the builds table:

* Amend filter action variable names so they're more succinct.
* Retain order in which actions are added to a filter, as this ordering is used in the UI when displaying the filter actions.
* Always show the table chrome, otherwise it's not possible to edit the columns shown until there are 10 or more results.
* Because date range searches may return no results, make sure that the search bar and "show all results" link are visible when the query returns no results.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/querysetfilter.py   |   4 -
 bitbake/lib/toaster/toastergui/static/js/table.js  |  56 +++++----
 bitbake/lib/toaster/toastergui/tablefilter.py      | 139 +++++++++++++++++----
 bitbake/lib/toaster/toastergui/tables.py           |  87 ++++++++-----
 .../toastergui/templates/builds-toastertable.html  |   2 +-
 .../toaster/toastergui/templates/toastertable.html |   7 +-
 6 files changed, 211 insertions(+), 84 deletions(-)
bitbake/lib/toaster/toastergui/querysetfilter.py +0 -4
index efa8507..10cc988 100644
@@ -22,7 +22,3 @@ class QuerysetFilter(object):
             return queryset.filter(self.criteria)
         else:
             return queryset
-
-    def count(self, queryset):
-        """ Returns a count of the elements in the filtered queryset """
-        return self.filter(queryset).count()
bitbake/lib/toaster/toastergui/static/js/table.js +31 -25

128 lines not on this page · raw message

bitbake/lib/toaster/toastergui/tablefilter.py +115 -24

213 lines not on this page · raw message

bitbake/lib/toaster/toastergui/tables.py +59 -28

191 lines not on this page · raw message

bitbake/lib/toaster/toastergui/templates/builds-toastertable.html +1 -1
index 2e32edb..bf13a66 100644
@@ -18,7 +18,7 @@
       {% include 'mrb_section.html' %}
     {% endwith %}
 
-    <h1  class="page-header top-air" data-role="page-title"></h1>
+    <h1 class="page-header top-air" data-role="page-title"></h1>
 
     {% url 'builds' as xhr_table_url %}
     {% include 'toastertable.html' %}
bitbake/lib/toaster/toastergui/templates/toastertable.html +5 -2
index 98a715f..f0a3aed 100644
@@ -32,8 +32,11 @@
       <a href="#" class="add-on btn remove-search-btn-{{table_name}}" tabindex="-1">
         <i class="icon-remove"></i>
       </a>
-      <button class="btn search-submit-{{table_name}}" >Search</button>
-      <button class="btn btn-link remove-search-btn-{{table_name}}">Show {{title|lower}}
+      <button class="btn search-submit-{{table_name}}">
+        Search
+      </button>
+      <button class="btn btn-link show-all-{{table_name}}">
+        Show {{title|lower}}
       </button>
     </form>
   </div>
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#13 lore raw

PATCH 13/17 toaster: toastergui Convert project builds page to ToasterTable

In reply to Elliot Smith

Use the all builds ToasterTable as the basis for the project builds ToasterTable.

[YOCTO #8738]

---
 .../toaster/toastergui/static/js/projecttopbar.js  |   9 ++
 bitbake/lib/toaster/toastergui/tables.py           | 169 +++++++++++++++++----
 .../toastergui/templates/baseprojectpage.html      |   1 +
 .../toaster/toastergui/templates/mrb_section.html  |   2 +-
 .../templates/projectbuilds-toastertable.html      |  56 +++++++
 bitbake/lib/toaster/toastergui/urls.py             |   6 +-
 bitbake/lib/toaster/toastergui/views.py            |  16 +-
 bitbake/lib/toaster/toastergui/widgets.py          |   1 -
 8 files changed, 223 insertions(+), 37 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html
bitbake/lib/toaster/toastergui/static/js/projecttopbar.js +9 -0
index b6ad380..58a32a0 100644
@@ -7,7 +7,10 @@ function projectTopBarInit(ctx) {
   var projectName = $("#project-name");
   var projectNameFormToggle = $("#project-change-form-toggle");
   var projectNameChangeCancel = $("#project-name-change-cancel");
+
+  // this doesn't exist for command-line builds
   var newBuildTargetInput = $("#build-input");
+
   var newBuildTargetBuildBtn = $("#build-button");
   var selectedTarget;
 
@@ -42,6 +45,12 @@ function projectTopBarInit(ctx) {
       $(this).parent().removeClass('active');
   });
 
+  if (!newBuildTargetInput.length) {
+    return;
+  }
+
+  /* the following only applies for non-command-line projects */
+
   /* Recipe build input functionality */
   if (ctx.numProjectLayers > 0 && ctx.machine){
     newBuildTargetInput.removeAttr("disabled");
bitbake/lib/toaster/toastergui/tables.py +143 -26

216 lines not on this page · raw message

bitbake/lib/toaster/toastergui/templates/baseprojectpage.html +1 -0
index 1f45be4..b143b78 100644
@@ -1,4 +1,5 @@
 {% extends "base.html" %}
+
 {% load projecttags %}
 {% load humanize %}
 
bitbake/lib/toaster/toastergui/templates/mrb_section.html +1 -1
index 52b3f1a..2f4820c 100644
@@ -6,7 +6,7 @@
 {%if mru and mru.count > 0%}
 
   {%if mrb_type == 'project' %}
-      <h2>
+      <h2 class="page-header">
       Latest project builds
 
       {% if project.is_default %}
bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html new file +56 -0

56 lines not on this page · raw message

bitbake/lib/toaster/toastergui/urls.py +4 -2
index 707b7d5..c8c1c6a 100644
@@ -28,7 +28,7 @@ urlpatterns = patterns('toastergui.views',
         url(r'^landing/$', 'landing', name='landing'),
 
         url(r'^builds/$',
-            tables.BuildsTable.as_view(template_name="builds-toastertable.html"),
+            tables.AllBuildsTable.as_view(template_name="builds-toastertable.html"),
             name='all-builds'),
 
         # build info navigation
@@ -83,7 +83,9 @@ urlpatterns = patterns('toastergui.views',
 
         url(r'^project/(?P<pid>\d+)/$', 'project', name='project'),
         url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'),
-        url(r'^project/(?P<pid>\d+)/builds/$', 'projectbuilds', name='projectbuilds'),
+        url(r'^project/(?P<pid>\d+)/builds/$',
+            tables.ProjectBuildsTable.as_view(template_name="projectbuilds-toastertable.html"),
+            name='projectbuilds'),
 
         # the import layer is a project-specific functionality;
         url(r'^project/(?P<pid>\d+)/importlayer$', 'importlayer', name='importlayer'),
bitbake/lib/toaster/toastergui/views.py +9 -7
index 295773f..fbae36c 100755
@@ -91,6 +91,7 @@ def landing(request):
 
     return render(request, 'landing.html', context)
 
+"""
 # returns a list for most recent builds;
 def _get_latest_builds(prj=None):
     queryset = Build.objects.all()
@@ -101,8 +102,9 @@ def _get_latest_builds(prj=None):
     return list(itertools.chain(
         queryset.filter(outcome=Build.IN_PROGRESS).order_by("-started_on"),
         queryset.filter(outcome__lt=Build.IN_PROGRESS).order_by("-started_on")[:3] ))
+"""
 
-
+"""
 # a JSON-able dict of recent builds; for use in the Project page, xhr_ updates,  and other places, as needed
 def _project_recent_build_list(prj):
     data = []
@@ -131,8 +133,7 @@ def _project_recent_build_list(prj):
         data.append(d)
 
     return data
-
-
+"""
 
 def objtojson(obj):
     from django.db.models.query import QuerySet
@@ -1915,6 +1916,7 @@ if True:
         ''' The exception raised on invalid POST requests '''
         pass
 
+    """
     # helper function, to be used on "all builds" and "project builds" pages
     def _build_list_helper(request, queryset_all, redirect_page, pid=None):
         default_orderby = 'completed_on:-'
@@ -2119,6 +2121,7 @@ if True:
         # merge daterange values
         context.update(context_date)
         return context, pagesize, orderby
+    """
 
 
 
@@ -2256,7 +2259,7 @@ if True:
             "completedbuilds": Build.objects.exclude(outcome = Build.IN_PROGRESS).filter(project_id = pid),
             "prj" : {"name": prj.name, },
             "buildrequests" : prj.build_set.filter(outcome=Build.IN_PROGRESS),
-            "builds" : _project_recent_build_list(prj),
+            #"builds" : _project_recent_build_list(prj),
             "layers" :  map(lambda x: {
                         "id": x.layercommit.pk,
                         "orderid": x.pk,
@@ -2827,10 +2830,8 @@ if True:
     # will set the GET parameters and redirect back to the
     # all-builds or projectbuilds page as appropriate;
     # TODO don't use exceptions to control program flow
-    @_template_renderer('projectbuilds.html')
+    """
     def projectbuilds(request, pid):
-        prj = Project.objects.get(id = pid)
-
         if request.method == "POST":
             # process any build request
 
@@ -2880,6 +2881,7 @@ if True:
         context['mru'] = _get_latest_builds(prj)
 
         return context
+    """
 
 
     def _file_name_for_artifact(b, artifact_type, artifact_id):
bitbake/lib/toaster/toastergui/widgets.py +0 -1
index 47de30d..bc081b8 100644
@@ -61,7 +61,6 @@ class ToasterTable(TemplateView):
 
         self.total_count = 0
         self.static_context_extra = {}
-        self.filter_actions = {}
         self.empty_state = "Sorry - no data found"
         self.default_orderby = ""
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#14 lore raw

PATCH 14/17 toaster: tests Fix Django tests for new ToasterTable pages

In reply to Elliot Smith

The Django command-line tests can no longer test the content of the projects/, builds/ and projectbuilds/ pages, as ToasterTable pages are populated by JavaScript.

Fix/remove affected tests by converting them to tests on the JSON returned by the ToasterTable.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/tests.py | 255 +++++++++++++++++++++-----------
 1 file changed, 170 insertions(+), 85 deletions(-)
bitbake/lib/toaster/toastergui/tests.py +170 -85
index c927fe1..b62faff 100644
@@ -38,11 +38,14 @@ import toastergui
 
 from toastergui.tables import SoftwareRecipesTable
 import json
+from datetime import timedelta
 from bs4 import BeautifulSoup
 import re
 import string
+import json
 
 PROJECT_NAME = "test project"
+PROJECT_NAME2 = "test project 2"
 CLI_BUILDS_PROJECT_NAME = 'Command line builds'
 
 class ViewTests(TestCase):
@@ -54,14 +57,36 @@ class ViewTests(TestCase):
         release = Release.objects.create(name="test release",
                                          branch_name="master",
                                          bitbake_version=bbv)
+        release2 = Release.objects.create(name="test release 2",
+                                          branch_name="master",
+                                          bitbake_version=bbv)
+
         self.project = Project.objects.create_project(name=PROJECT_NAME,
                                                       release=release)
+
+        Project.objects.create_project(name=PROJECT_NAME2,
+                                       release=release2)
+
         now = timezone.now()
+        later = now + timedelta(days=1)
 
         build = Build.objects.create(project=self.project,
                                      started_on=now,
                                      completed_on=now)
 
+        # for testing BuildsTable
+        Build.objects.create(project=self.project,
+                             started_on=now,
+                             completed_on=now,
+                             outcome=Build.SUCCEEDED,
+                             machine="raspberrypi2")
+
+        Build.objects.create(project=self.project,
+                             started_on=later,
+                             completed_on=later,
+                             outcome=Build.FAILED,
+                             machine="qemux86")
+
         layersrc = LayerSource.objects.create(sourcetype=LayerSource.TYPE_IMPORTED)
         self.priority = ReleaseLayerSourcePriority.objects.create(release=release,
                                                                   layer_source=layersrc)
@@ -172,8 +197,7 @@ class ViewTests(TestCase):
         response = self.client.get(reverse('all-projects'), follow=True)
         self.assertEqual(response.status_code, 200)
         self.assertTrue(response['Content-Type'].startswith('text/html'))
-        self.assertTemplateUsed(response, "projects.html")
-        self.assertTrue(PROJECT_NAME in response.content)
+        self.assertTemplateUsed(response, "projects-toastertable.html")
 
     def test_get_json_call_returns_json(self):
         """Test for all projects output in json format"""
@@ -191,13 +215,6 @@ class ViewTests(TestCase):
         self.assertTrue(PROJECT_NAME in [x["name"] for x in data["rows"]])
         self.assertTrue("id" in data["rows"][0])
 
-        self.assertEqual(sorted(data["rows"][0]),
-                         ['bitbake_version_id', 'created', 'id',
-                          'is_default', 'layersTypeAheadUrl', 'name',
-                          'num_builds', 'projectBuildsUrl', 'projectPageUrl',
-                          'recipesTypeAheadUrl', 'release_id',
-                          'short_description', 'updated', 'user_id'])
-
     def test_typeaheads(self):
         """Test typeahead ReST API"""
         layers_url = reverse('xhr_layerstypeahead', args=(self.project.id,))
@@ -450,7 +467,7 @@ class ViewTests(TestCase):
             all_data = get_data(table)
 
             self.assertTrue(len(all_data['rows']) > 1,
-                            "Cannot test on a table with < 1 row")
+                            "Cannot test on the table %s with < 1 row" % name)
 
             if table.default_orderby:
                 row_one = all_data['rows'][0][table.default_orderby.strip("-")]
@@ -512,16 +529,20 @@ class ViewTests(TestCase):
                         # This is the name of the filter:action
                         # e.g. project_filter:not_in_project
                         filter_string = "%s:%s" % (column['filter_name'],
-                                                   filter_action['name'])
+                                                   filter_action['action_name'])
                         # Now get the data with the filter applied
                         filtered_data = get_data(table_cls(),
                                                  {"filter" : filter_string})
-                        self.assertEqual(len(filtered_data['rows']),
-                                         int(filter_action['count']),
-                                         "We added a table filter for %s but "
-                                         "the number of rows returned was not "
-                                         "what the filter info said there "
-                                         "would be" % name)
+
+                        # date range filter actions can't specify the
+                        # number of results they return, so their count is 0
+                        if filter_action['count'] != None:
+                            self.assertEqual(len(filtered_data['rows']),
+                                             int(filter_action['count']),
+                                             "We added a table filter for %s but "
+                                             "the number of rows returned was not "
+                                             "what the filter info said there "
+                                             "would be" % name)
 
 
             # Test search functionality on the table
@@ -673,6 +694,10 @@ class AllProjectsPageTests(TestCase):
                                                      value=self.MACHINE_NAME)
         project_var.save()
 
+    def _get_row_for_project(self, data, project_id):
+        """ Get the object representing the table data for a project """
+        return [row for row in data['rows'] if row['id'] == project_id][0]
+
     def test_default_project_hidden(self):
         """ The default project should be hidden if it has no builds """
         params = {"count": 10, "orderby": "updated:-", "page": 1}
@@ -688,11 +713,20 @@ class AllProjectsPageTests(TestCase):
         self._add_build_to_default_project()
 
         params = {"count": 10, "orderby": "updated:-", "page": 1}
-        response = self.client.get(reverse('all-projects'), params)
 
-        self.assertTrue('tr class="data"' in response.content,
-                        'should be a project row in the page')
-        self.assertTrue(CLI_BUILDS_PROJECT_NAME in response.content,
+        response = self.client.get(
+            reverse('all-projects'),
+            {'format': 'json'},
+            params
+        )
+
+        data = json.loads(response.content)
+
+        # find the row for the default project
+        default_project_row = self._get_row_for_project(data, self.default_project.id)
+
+        # check its name template has the correct text
+        self.assertEqual(default_project_row['name'], CLI_BUILDS_PROJECT_NAME,
                         'default project "cli builds" should be in page')
 
     def test_default_project_release(self):
@@ -706,24 +740,32 @@ class AllProjectsPageTests(TestCase):
         # another project to test, which should show release
         self._add_non_default_project()
 
-        response = self.client.get(reverse('all-projects'), follow=True)
-        soup = BeautifulSoup(response.content)
+        response = self.client.get(
+            reverse('all-projects'),
+            {'format': 'json'},
+            follow=True
+        )
+
+        data = json.loads(response.content)
+
+        # used to find the correct span in the template output
+        attrs = {'data-project-field': 'release'}
+
+        # find the row for the default project
+        default_project_row = self._get_row_for_project(data, self.default_project.id)
 
-        # check the release cell for the default project
-        attrs = {'data-project': str(self.default_project.id)}
-        rows = soup.find_all('tr', attrs=attrs)
-        self.assertEqual(len(rows), 1, 'should be one row for default project')
-        cells = rows[0].find_all('td', attrs={'data-project-field': 'release'})
-        self.assertEqual(len(cells), 1, 'should be one release cell')
-        text = cells[0].select('span.muted')[0].text
+        # check the release text for the default project
+        soup = BeautifulSoup(default_project_row['static:release'])
+        text = soup.find('span', attrs=attrs).select('span.muted')[0].text
         self.assertEqual(text, 'Not applicable',
                          'release should be not applicable for default project')
 
+        # find the row for the default project
+        other_project_row = self._get_row_for_project(data, self.project.id)
+
         # check the link in the release cell for the other project
-        attrs = {'data-project': str(self.project.id)}
-        rows = soup.find_all('tr', attrs=attrs)
-        cells = rows[0].find_all('td', attrs={'data-project-field': 'release'})
-        text = cells[0].select('a')[0].text
+        soup = BeautifulSoup(other_project_row['static:release'])
+        text = soup.find('span', attrs=attrs).select('a')[0].text.strip()
         self.assertEqual(text, self.release.name,
                          'release name should be shown for non-default project')
 
@@ -738,24 +780,32 @@ class AllProjectsPageTests(TestCase):
         # another project to test, which should show machine
         self._add_non_default_project()
 
-        response = self.client.get(reverse('all-projects'), follow=True)
-        soup = BeautifulSoup(response.content)
+        response = self.client.get(
+            reverse('all-projects'),
+            {'format': 'json'},
+            follow=True
+        )
+
+        data = json.loads(response.content)
+
+        # used to find the correct span in the template output
+        attrs = {'data-project-field': 'machine'}
+
+        # find the row for the default project
+        default_project_row = self._get_row_for_project(data, self.default_project.id)
 
         # check the machine cell for the default project
-        attrs = {'data-project': str(self.default_project.id)}
-        rows = soup.find_all('tr', attrs=attrs)
-        self.assertEqual(len(rows), 1, 'should be one row for default project')
-        cells = rows[0].find_all('td', attrs={'data-project-field': 'machine'})
-        self.assertEqual(len(cells), 1, 'should be one machine cell')
-        text = cells[0].select('span.muted')[0].text
+        soup = BeautifulSoup(default_project_row['static:machine'])
+        text = soup.find('span', attrs=attrs).select('span.muted')[0].text.strip()
         self.assertEqual(text, 'Not applicable',
-                         'machine should be not applicable for default project')
+            'machine should be not applicable for default project')
+
+        # find the row for the default project
+        other_project_row = self._get_row_for_project(data, self.project.id)
 
         # check the link in the machine cell for the other project
-        attrs = {'data-project': str(self.project.id)}
-        rows = soup.find_all('tr', attrs=attrs)
-        cells = rows[0].find_all('td', attrs={'data-project-field': 'machine'})
-        text = cells[0].select('a')[0].text
+        soup = BeautifulSoup(other_project_row['static:machine'])
+        text = soup.find('span', attrs=attrs).find('a').text.strip()
         self.assertEqual(text, self.MACHINE_NAME,
                          'machine name should be shown for non-default project')
 
@@ -769,24 +819,33 @@ class AllProjectsPageTests(TestCase):
         # need a build, otherwise project doesn't display at all
         self._add_build_to_default_project()
 
-        # another project to test, which should show machine
+        # another project to test
         self._add_non_default_project()
 
-        response = self.client.get(reverse('all-projects'), follow=True)
-        soup = BeautifulSoup(response.content)
+        response = self.client.get(
+            reverse('all-projects'),
+            {'format': 'json'},
+            follow=True
+        )
+
+        data = json.loads(response.content)
 
-        # link for default project
-        row = soup.find('tr', attrs={'data-project': self.default_project.id})
-        cell = row.find('td', attrs={'data-project-field': 'name'})
+        # find the row for the default project
+        default_project_row = self._get_row_for_project(data, self.default_project.id)
+
+        # check the link on the name field
+        soup = BeautifulSoup(default_project_row['static:name'])
         expected_url = reverse('projectbuilds', args=(self.default_project.id,))
-        self.assertEqual(cell.find('a')['href'], expected_url,
+        self.assertEqual(soup.find('a')['href'], expected_url,
                          'link on default project name should point to builds')
 
-        # link for other project
-        row = soup.find('tr', attrs={'data-project': self.project.id})
-        cell = row.find('td', attrs={'data-project-field': 'name'})
+        # find the row for the other project
+        other_project_row = self._get_row_for_project(data, self.project.id)
+
+        # check the link for the other project
+        soup = BeautifulSoup(other_project_row['static:name'])
         expected_url = reverse('project', args=(self.project.id,))
-        self.assertEqual(cell.find('a')['href'], expected_url,
+        self.assertEqual(soup.find('a')['href'], expected_url,
                          'link on project name should point to configuration')
 
 class ProjectBuildsPageTests(TestCase):
@@ -846,9 +905,9 @@ class ProjectBuildsPageTests(TestCase):
     def _get_rows_for_project(self, project_id):
         """ Helper to retrieve HTML rows for a project """
         url = reverse("projectbuilds", args=(project_id,))
-        response = self.client.get(url, follow=True)
-        soup = BeautifulSoup(response.content)
-        return soup.select('tr[class="data"]')
+        response = self.client.get(url, {'format': 'json'}, follow=True)
+        data = json.loads(response.content)
+        return data['rows']
 
     def test_show_builds_for_project(self):
         """ Builds for a project should be displayed """
@@ -889,10 +948,14 @@ class ProjectBuildsPageTests(TestCase):
         """ Task should be shown as suffix on build name """
         build = Build.objects.create(**self.project1_build_success)
         Target.objects.create(build=build, target='bash', task='clean')
-        url = reverse("projectbuilds", args=(self.project1.id,))
-        response = self.client.get(url, follow=True)
-        result = re.findall('^ +bash:clean$', response.content, re.MULTILINE)
-        self.assertEqual(len(result), 2)
+
+        url = reverse('projectbuilds', args=(self.project1.id,))
+        response = self.client.get(url, {'format': 'json'}, follow=True)
+        data = json.loads(response.content)
+        cell = data['rows'][0]['static:target']
+
+        result = re.findall('^ +bash:clean', cell, re.MULTILINE)
+        self.assertEqual(len(result), 1)
 
     def test_cli_builds_hides_tabs(self):
         """
@@ -952,32 +1015,46 @@ class AllBuildsPageTests(TestCase):
             "outcome": Build.SUCCEEDED
         }
 
+    def _get_row_for_build(self, data, build_id):
+        """ Get the object representing the table data for a project """
+        return [row for row in data['rows']
+                    if row['id'] == build_id][0]
+
     def test_show_tasks_in_allbuilds(self):
         """ Task should be shown as suffix on build name """
         build = Build.objects.create(**self.project1_build_success)
         Target.objects.create(build=build, target='bash', task='clean')
+
         url = reverse('all-builds')
-        response = self.client.get(url, follow=True)
-        result = re.findall('bash:clean', response.content, re.MULTILINE)
-        self.assertEqual(len(result), 3)
+        response = self.client.get(url, {'format': 'json'}, follow=True)
+        data = json.loads(response.content)
+        cell = data['rows'][0]['static:target']
 
-    def test_no_run_again_for_cli_build(self):
-        """ "Run again" button should not be shown for command-line builds """
-        build = Build.objects.create(**self.default_project_build_success)
+        result = re.findall('bash:clean', cell, re.MULTILINE)
+        self.assertEqual(len(result), 1)
+
+    def test_run_again(self):
+        """
+        "Run again" button should not be shown for command-line builds,
+        but should be shown for other builds
+        """
+        build1 = Build.objects.create(**self.project1_build_success)
+        default_build = Build.objects.create(**self.default_project_build_success)
         url = reverse('all-builds')
         response = self.client.get(url, follow=True)
         soup = BeautifulSoup(response.content)
 
-        attrs = {'data-latest-build-result': build.id}
-        result = soup.find('div', attrs=attrs)
-
         # shouldn't see a run again button for command-line builds
+        attrs = {'data-latest-build-result': default_build.id}
+        result = soup.find('div', attrs=attrs)
         run_again_button = result.select('button')
         self.assertEqual(len(run_again_button), 0)
 
-        # should see a help icon for command-line builds
-        help_icon = result.select('i.get-help-green')
-        self.assertEqual(len(help_icon), 1)
+        # should see a run again button for non-command-line builds
+        attrs = {'data-latest-build-result': build1.id}
+        result = soup.find('div', attrs=attrs)
+        run_again_button = result.select('button')
+        self.assertEqual(len(run_again_button), 1)
 
     def test_tooltips_on_project_name(self):
         """
@@ -989,20 +1066,28 @@ class AllBuildsPageTests(TestCase):
         default_build = Build.objects.create(**self.default_project_build_success)
 
         url = reverse('all-builds')
-        response = self.client.get(url, follow=True)
-        soup = BeautifulSoup(response.content)
+        response = self.client.get(url, {'format': 'json'}, follow=True)
+        data = json.loads(response.content)
+
+        # get the data row for the non-command-line builds project
+        other_project_row = self._get_row_for_build(data, build1.id)
+
+        # make sure there is some HTML
+        soup = BeautifulSoup(other_project_row['static:project-name'])
+        self.assertEqual(len(soup.select('a')), 1,
+                         'should be a project name link')
 
         # no help icon on non-default project name
-        result = soup.find('tr', attrs={'data-table-build-result': build1.id})
-        name = result.select('td.project-name')[0]
-        icons = name.select('i.get-help')
+        icons = soup.select('i.get-help')
         self.assertEqual(len(icons), 0,
                          'should not be a help icon for non-cli builds name')
 
+        # get the data row for the command-line builds project
+        default_project_row = self._get_row_for_build(data, default_build.id)
+
         # help icon on default project name
-        result = soup.find('tr', attrs={'data-table-build-result': default_build.id})
-        name = result.select('td.project-name')[0]
-        icons = name.select('i.get-help')
+        soup = BeautifulSoup(default_project_row['static:project-name'])
+        icons = soup.select('i.get-help')
         self.assertEqual(len(icons), 1,
                          'should be a help icon for cli builds name')
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#15 lore raw

PATCH 15/17 toaster: toastergui Don't hide all elements with .col class

In reply to Elliot Smith

Disabling the "project" column in a ToasterTable for builds causes the recent builds area to be hidden. This is because the column hiding code hides all elements with a class matching ".<column>", regardless of where they occur on the page; and the recent builds area was using the ".project-name" class, which means it is included in the set of elements which are hidden.

Scope the element search to the table so that only elements within the table are hidden or shown.

[YOCTO #8792]

---
 bitbake/lib/toaster/toastergui/static/js/table.js | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
bitbake/lib/toaster/toastergui/static/js/table.js +4 -4
index d77ebaf..5ffb319 100644
@@ -313,7 +313,7 @@ function tableInit(ctx){
       $("#"+ctx.tableName+" th").each(function(){
         for (var i in cols_hidden){
           if ($(this).hasClass(cols_hidden[i])){
-            $("."+cols_hidden[i]).hide();
+            table.find("."+cols_hidden[i]).hide();
             $("#checkbox-"+cols_hidden[i]).removeAttr("checked");
           }
         }
@@ -323,7 +323,7 @@ function tableInit(ctx){
          * user setting.
          */
         for (var i in defaultHiddenCols) {
-          $("."+defaultHiddenCols[i]).hide();
+          table.find("."+defaultHiddenCols[i]).hide();
           $("#checkbox-"+defaultHiddenCols[i]).removeAttr("checked");
         }
     }
@@ -367,9 +367,9 @@ function tableInit(ctx){
     var disabled_cols = [];
 
     if ($(this).prop("checked")) {
-      $("."+col).show();
+      table.find("."+col).show();
     }  else {
-      $("."+col).hide();
+      table.find("."+col).hide();
       /* If we're ordered by the column we're hiding remove the order by */
       if (col === tableParams.orderby ||
           '-' + col === tableParams.orderby){
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#16 lore raw

PATCH 16/17 toaster: toastergui Ensure filter_value updates

In reply to Elliot Smith

Clicking on the radio button for a date range filter action populates the from and to fields for that action if they are empty.

However, because this doesn't fire "change" events, clicking on the radio button doesn't update the filter_value hidden field. This means that the date range action's filter_value parameter isn't set correctly when the filter popup is submitted.

Manually call the changeHandler() to set the filter_value whenever the radio for a date range filter is clicked.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/static/js/table.js | 32 +++++++++++++----------
 1 file changed, 18 insertions(+), 14 deletions(-)
bitbake/lib/toaster/toastergui/static/js/table.js +18 -14
index 5ffb319..0c356a3 100644
@@ -485,20 +485,6 @@ function tableInit(ctx){
       action.find('[data-date-to-for]').datepicker(options);
     inputTo.val(selectedTo);
 
-    // if the radio button is checked and one or both of the datepickers are
-    // empty, populate them with today's date
-    radio.change(function () {
-      var now = new Date();
-
-      if (inputFrom.val() === '') {
-        inputFrom.datepicker('setDate', now);
-      }
-
-      if (inputTo.val() === '') {
-        inputTo.datepicker('setDate', now);
-      }
-    });
-
     // set filter_value based on date pickers when
     // one of their values changes
     var changeHandler = function () {
@@ -526,6 +512,24 @@ function tableInit(ctx){
       inputFrom.datepicker('option', 'maxDate', inputTo.val());
     });
 
+    // if the radio button is checked and one or both of the datepickers are
+    // empty, populate them with today's date
+    radio.change(function () {
+      var now = new Date();
+
+      if (inputFrom.val() === '') {
+        inputFrom.datepicker('setDate', now);
+      }
+
+      if (inputTo.val() === '') {
+        inputTo.datepicker('setDate', now);
+      }
+
+      // setting the date like this doesn't fire the changeHandler to
+      // update the filter_value, so do that manually instead
+      changeHandler()
+    });
+
     return action;
   }
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#17 lore raw

PATCH 17/17 toaster: toastergui Streamline construction of filter objects

In reply to Elliot Smith

In line with comments from review, remove the QuerysetFilter class (redundant) and convert ProjectFilters into a class with static methods.

[YOCTO #8738]

---
 bitbake/lib/toaster/toastergui/querysetfilter.py | 24 ---------------
 bitbake/lib/toaster/toastergui/tablefilter.py    | 38 ++++++++++--------------
 bitbake/lib/toaster/toastergui/tables.py         | 35 +++++++++++-----------
 bitbake/lib/toaster/toastergui/widgets.py        |  1 -
 4 files changed, 32 insertions(+), 66 deletions(-)
 delete mode 100644 bitbake/lib/toaster/toastergui/querysetfilter.py
bitbake/lib/toaster/toastergui/querysetfilter.py deleted +0 -24
deleted file mode 100644
index 10cc988..0000000
@@ -1,24 +0,0 @@
-class QuerysetFilter(object):
-    """ Filter for a queryset """
-
-    def __init__(self, criteria=None):
-        self.criteria = None
-        if criteria:
-            self.set_criteria(criteria)
-
-    def set_criteria(self, criteria):
-        """
-        criteria is an instance of django.db.models.Q;
-        see https://docs.djangoproject.com/en/1.9/ref/models/querysets/#q-objects
-        """
-        self.criteria = criteria
-
-    def filter(self, queryset):
-        """
-        Filter queryset according to the criteria for this filter,
-        returning the filtered queryset
-        """
-        if self.criteria:
-            return queryset.filter(self.criteria)
-        else:
-            return queryset
bitbake/lib/toaster/toastergui/tablefilter.py +15 -23
index 6cbf975..9dee269 100644
@@ -1,7 +1,6 @@
 from datetime import timedelta
 from django.db.models import Q, Max, Min
 from django.utils import dateparse, timezone
-from querysetfilter import QuerysetFilter
 
 class TableFilter(object):
     """
@@ -97,10 +96,10 @@ class TableFilterAction(object):
     ToasterTable
     """
 
-    def __init__(self, name, title, queryset_filter):
+    def __init__(self, name, title, criteria):
         self.name = name
         self.title = title
-        self.queryset_filter = queryset_filter
+        self.criteria = criteria
 
         # set in subclasses
         self.type = None
@@ -111,11 +110,13 @@ class TableFilterAction(object):
         the structure of this string depends on the type of action;
         it's ignored for a toggle filter action, which is just on or off
         """
-        if not params:
-            return
+        pass
 
     def filter(self, queryset):
-        return self.queryset_filter.filter(queryset)
+        if self.criteria:
+            return queryset.filter(self.criteria)
+        else:
+            return queryset
 
     def to_json(self, queryset):
         """ Dump as a JSON object """
@@ -146,16 +147,12 @@ class TableFilterActionDay(TableFilterAction):
     YESTERDAY = 'yesterday'
 
     def __init__(self, name, title, field, day,
-    queryset_filter = QuerysetFilter(), query_helper = TableFilterQueryHelper()):
+    query_helper = TableFilterQueryHelper()):
         """
         field: (string) the datetime field to filter by
         day: (string) "today" or "yesterday"
         """
-        super(TableFilterActionDay, self).__init__(
-            name,
-            title,
-            queryset_filter
-        )
+        super(TableFilterActionDay, self).__init__(name, title, None)
         self.type = 'day'
         self.field = field
         self.day = day
@@ -168,8 +165,6 @@ class TableFilterActionDay(TableFilterAction):
         depending on when the filtering is applied
         """
 
-        criteria = None
-        date_str = None
         now = timezone.now()
 
         if self.day == self.YESTERDAY:
@@ -180,15 +175,13 @@ class TableFilterActionDay(TableFilterAction):
 
         wanted_date_str = wanted_date.strftime('%Y-%m-%d')
 
-        criteria = self.query_helper.dateStringsToQ(
+        self.criteria = self.query_helper.dateStringsToQ(
             self.field,
             wanted_date_str,
             wanted_date_str
         )
 
-        self.queryset_filter.set_criteria(criteria)
-
-        return self.queryset_filter.filter(queryset)
+        return queryset.filter(self.criteria)
 
 class TableFilterActionDateRange(TableFilterAction):
     """
@@ -197,14 +190,14 @@ class TableFilterActionDateRange(TableFilterAction):
     """
 
     def __init__(self, name, title, field,
-    queryset_filter = QuerysetFilter(), query_helper = TableFilterQueryHelper()):
+    query_helper = TableFilterQueryHelper()):
         """
         field: (string) the field to find the max/min range from in the queryset
         """
         super(TableFilterActionDateRange, self).__init__(
             name,
             title,
-            queryset_filter
+            None
         )
 
         self.type = 'daterange'
@@ -227,17 +220,16 @@ class TableFilterActionDateRange(TableFilterAction):
         try:
             date_from_str, date_to_str = params.split(',')
         except ValueError:
-            self.queryset_filter.set_criteria(None)
+            self.criteria = None
             return
 
         # one of the values required for the filter is missing, so set
         # it to the one which was supplied
-        criteria = self.query_helper.dateStringsToQ(
+        self.criteria = self.query_helper.dateStringsToQ(
             self.field,
             date_from_str,
             date_to_str
         )
-        self.queryset_filter.set_criteria(criteria)
 
     def to_json(self, queryset):
         """ Dump as a JSON object """
bitbake/lib/toaster/toastergui/tables.py +17 -18
index 1449acc..ffe3357 100644
@@ -20,7 +20,6 @@
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
 from toastergui.widgets import ToasterTable
-from toastergui.querysetfilter import QuerysetFilter
 from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project
 from orm.models import CustomImageRecipe, Package, Build, LogMessage, Task
 from orm.models import ProjectTarget
@@ -37,9 +36,13 @@ from toastergui.tablefilter import TableFilterActionDateRange
 from toastergui.tablefilter import TableFilterActionDay
 
 class ProjectFilters(object):
-    def __init__(self, project_layers):
-        self.in_project = QuerysetFilter(Q(layer_version__in=project_layers))
-        self.not_in_project = QuerysetFilter(~Q(layer_version__in=project_layers))
+    @staticmethod
+    def in_project(project_layers):
+        return Q(layer_version__in=project_layers)
+
+    @staticmethod
+    def not_in_project(project_layers):
+        return ~(ProjectFilters.in_project(project_layers))
 
 class LayersTable(ToasterTable):
     """Table of layers in Toaster"""
@@ -71,13 +74,13 @@ class LayersTable(ToasterTable):
         in_project_action = TableFilterActionToggle(
             "in_project",
             "Layers added to this project",
-            QuerysetFilter(criteria)
+            criteria
         )
 
         not_in_project_action = TableFilterActionToggle(
             "not_in_project",
             "Layers not added to this project",
-            QuerysetFilter(~criteria)
+            ~criteria
         )
 
         in_current_project_filter.add_action(in_project_action)
@@ -217,8 +220,6 @@ class MachinesTable(ToasterTable):
     def setup_filters(self, *args, **kwargs):
         project = Project.objects.get(pk=kwargs['pid'])
 
-        project_filters = ProjectFilters(self.project_layers)
-
         in_current_project_filter = TableFilter(
             "in_current_project",
             "Filter by project machines"
@@ -227,13 +228,13 @@ class MachinesTable(ToasterTable):
         in_project_action = TableFilterActionToggle(
             "in_project",
             "Machines provided by layers added to this project",
-            project_filters.in_project
+            ProjectFilters.in_project(self.project_layers)
         )
 
         not_in_project_action = TableFilterActionToggle(
             "not_in_project",
             "Machines provided by layers not added to this project",
-            project_filters.not_in_project
+            ProjectFilters.not_in_project(self.project_layers)
         )
 
         in_current_project_filter.add_action(in_project_action)
@@ -350,8 +351,6 @@ class RecipesTable(ToasterTable):
         return context
 
     def setup_filters(self, *args, **kwargs):
-        project_filters = ProjectFilters(self.project_layers)
-
         table_filter = TableFilter(
             'in_current_project',
             'Filter by project recipes'
@@ -360,13 +359,13 @@ class RecipesTable(ToasterTable):
         in_project_action = TableFilterActionToggle(
             'in_project',
             'Recipes provided by layers added to this project',
-            project_filters.in_project
+            ProjectFilters.in_project(self.project_layers)
         )
 
         not_in_project_action = TableFilterActionToggle(
             'not_in_project',
             'Recipes provided by layers not added to this project',
-            project_filters.not_in_project
+            ProjectFilters.not_in_project(self.project_layers)
         )
 
         table_filter.add_action(in_project_action)
@@ -1141,13 +1140,13 @@ class BuildsTable(ToasterTable):
         successful_builds_action = TableFilterActionToggle(
             'successful_builds',
             'Successful builds',
-            QuerysetFilter(Q(outcome=Build.SUCCEEDED))
+            Q(outcome=Build.SUCCEEDED)
         )
 
         failed_builds_action = TableFilterActionToggle(
             'failed_builds',
             'Failed builds',
-            QuerysetFilter(Q(outcome=Build.FAILED))
+            Q(outcome=Build.FAILED)
         )
 
         outcome_filter.add_action(successful_builds_action)
@@ -1227,13 +1226,13 @@ class BuildsTable(ToasterTable):
         with_failed_tasks_action = TableFilterActionToggle(
             'with_failed_tasks',
             'Builds with failed tasks',
-            QuerysetFilter(criteria)
+            criteria
         )
 
         without_failed_tasks_action = TableFilterActionToggle(
             'without_failed_tasks',
             'Builds without failed tasks',
-            QuerysetFilter(~criteria)
+            ~criteria
         )
 
         failed_tasks_filter.add_action(with_failed_tasks_action)
bitbake/lib/toaster/toastergui/widgets.py +0 -1
index bc081b8..d9328d4 100644
@@ -32,7 +32,6 @@ from django.template import Context, Template
 from django.core.serializers.json import DjangoJSONEncoder
 from django.core.exceptions import FieldError
 from django.conf.urls import url, patterns
-from toastergui.querysetfilter import QuerysetFilter
 
 import types
 import json
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
#18 lore raw
In reply to Elliot Smith
Quoted 4 lines · Elliot Smith

Convert some of the most important Toaster pages to ToasterTable rather than
individual templates.

I just realised two things I missed:

1. I added a new Python file, but neglected to add the licence header.
2. I left the old (commented-out) view code in views.py.

I'll fix (1) in the branch, but I'll leave (2) until after review, in case there's some code I need to port and have missed.

Elliot

Quoted 86 lines

This involved a rewrite of the ToasterTable filter system (see tablefilter.py)
to cope with date range filtering for the builds tables.

I also moved a lot of code out of the views into the model, where it seems better placed.

I also fixed a few JS issues which arose while testing, such as filters not de-selecting when there are multiple filters on a table; event handlers being lost when repopulating a table; and incorrect hiding of page elements due to table classes being used for the page title element.

Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8738

Also incidentally fixes
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8792,
as this functionality works correctly in ToasterTable.

Changes since 6d35574 (toaster-next) are in
git://git.yoctoproject.org/poky-contrib, elliot/toaster/tables-8738

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/tables-8738

Elliot Smith (17):
toaster: toastergui Use ToasterTable for projects page
toaster: Move image file suffix list to model
toaster: Check inferred file suffixes against list of known types
toaster: toastergui Switch projects/ view to ToasterTable
toaster: toastergui Use event delegates for hover help elements
toaster: toastergui Convert all builds page to ToasterTable
toaster: toastergui Add QuerysetFilter class
toaster: toastergui Refactor ToasterTable filtering
toaster: toastergui Switch off filter highlights when inactive
toaster: toastergui Show recent builds on all builds page
toaster: toastergui Implement date range filters for builds
toaster: toastergui Implement "today" and "yesterday" filters
toaster: toastergui Convert project builds page to ToasterTable
toaster: tests Fix Django tests for new ToasterTable pages
toaster: toastergui Don't hide all elements with .col class
toaster: toastergui Ensure filter_value updates
toaster: toastergui Streamline construction of filter objects

 bitbake/lib/toaster/orm/models.py                  | 109 ++-
 .../lib/toaster/toastergui/static/js/libtoaster.js |   6 +-
 .../toaster/toastergui/static/js/projecttopbar.js  |   9 +
 bitbake/lib/toaster/toastergui/static/js/table.js  | 272 +++++--
 bitbake/lib/toaster/toastergui/tablefilter.py      | 271 +++++++
 bitbake/lib/toaster/toastergui/tables.py           | 826
+++++++++++++++++++--
 .../toastergui/templates/baseprojectpage.html      |   1 +
 .../toastergui/templates/builds-toastertable.html  |  48 ++
 .../toaster/toastergui/templates/mrb_section.html  |   4 +-
 .../templates/projectbuilds-toastertable.html      |  56 ++
 .../templates/projects-toastertable.html           |  36 +
 .../toaster/toastergui/templates/toastertable.html |   7 +-
 bitbake/lib/toaster/toastergui/tests.py            | 255 ++++---
 bitbake/lib/toaster/toastergui/urls.py             |  14 +-
 bitbake/lib/toaster/toastergui/views.py            |  79 +-
 bitbake/lib/toaster/toastergui/widgets.py          |  94 ++-
 16 files changed, 1781 insertions(+), 306 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/tablefilter.py
 create mode 100644
bitbake/lib/toaster/toastergui/templates/builds-toastertable.html
 create mode 100644
bitbake/lib/toaster/toastergui/templates/projectbuilds-toastertable.html
 create mode 100644

bitbake/lib/toaster/toastergui/templates/projects-toastertable.html

--
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Signature · 3 lines
Elliot Smith
Software Engineer
Intel Open Source Technology Centre