Skip to contentSkip to conversation

toaster: bring back the correct typeahead behaviour as a modular widget

PATCH New Ed Bartosh 1 participant 13 messages Aug 4, 2015

lore
Share & tools
lore

Ed Bartosh

author
#0 lore raw

cover toaster: bring back the correct typeahead behaviour as a modular widget

Hi,

This is a set of patches from Michael reviewed by me.

List of changes in this patchset:

- New typeahead widget
- Switch to Implementation of widget in frontend
- Utility functions in the models so that the querysets which are used for the ToasterTables are the same queries that are used for the typeaheads.
- Added/Fixed typeahead widget unit tests

Please, review.

The following changes since commit a8b723498c9a7106210db140452886894494b4d6:

bitbake: cooker: Resolve file monitoring race issues when using memres bitbake (2015-08-03 07:36:25 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/submit/michaelw/toaster/fix-typeahead-behaviour
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/submit/michaelw/toaster/fix-typeahead-behaviour

Michael Wood (12):
bitbake: toaster: orm Add util function to return the url to layerversion
bitbake: toaster: orm Add util functions to return common querysets
bitbake: toastergui: tables Use util functions for the common querysets
bitbake: toastergui: widgets Add a typeahead widget
bitbake: toastergui: Add typeaheads layers, machines, projects, recipes
bitbake: toastergui: Switch to using the new toaster typeahead widget
bitbake: toastergui: libtoaster Throw an exception no url is specified
bitbake: toasterui: views Remove unused xhr_typeahead view definition
bitbake: toastergui: views Standardise the fields project layer response
bitbake: toastergui: tests Fix and more comprehensive typeahead unittest
bitbake: toastergui: libtoaster: typeahead resiliency for slow server
bitbake: toastergui: libtoaster typeahead Add in results highlighter

 lib/toaster/orm/models.py                        |  45 ++++++-
 lib/toaster/toastergui/static/js/base.js         |  26 ++--
 lib/toaster/toastergui/static/js/importlayer.js  |  12 +-
 lib/toaster/toastergui/static/js/layerdetails.js |   2 +-
 lib/toaster/toastergui/static/js/libtoaster.js   |  37 ++++--
 lib/toaster/toastergui/static/js/projectpage.js  |   8 +-
 lib/toaster/toastergui/tables.py                 |  11 +-
 lib/toaster/toastergui/templates/base.html       |   8 +-
 lib/toaster/toastergui/templates/project.html    |   3 +-
 lib/toaster/toastergui/tests.py                  |  88 +++++++++++---
 lib/toaster/toastergui/typeaheads.py             | 145 +++++++++++++++++++++++
 lib/toaster/toastergui/urls.py                   |  19 ++-
 lib/toaster/toastergui/views.py                  |  71 +++++------
 lib/toaster/toastergui/widgets.py                |  52 ++++++++
 14 files changed, 423 insertions(+), 104 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/typeaheads.py

--
Regards,
Ed

author
#2 lore raw

PATCH 2/12 bitbake: toaster: orm Add util functions to return common querysets

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

We use these querysets when creating tables of results and also when we want to have a typeahead search. These can also form the basis of future API endpoints.

---
 lib/toaster/orm/models.py | 41 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)
lib/toaster/orm/models.py +40 -1
index 3b72f80..4f07e37 100644
@@ -20,7 +20,7 @@
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
 from django.db import models
-from django.db.models import F, Q, Avg
+from django.db.models import F, Q, Avg, Max
 from django.utils import timezone
 
 from django.core.urlresolvers import reverse
@@ -195,6 +195,45 @@ class Project(models.Model):
     def projectlayer_equivalent_set(self):
         return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch")
 
+    def get_available_machines(self):
+        """ Returns QuerySet of all Machines which are provided by the
+        Layers currently added to the Project """
+        queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set)
+        return queryset
+
+    def get_all_compatible_machines(self):
+        """ Returns QuerySet of all the compatible machines available to the
+        project including ones from Layers not currently added """
+        compatible_layers = self.compatible_layerversions()
+
+        queryset = Machine.objects.filter(layer_version__in=compatible_layers)
+        return queryset
+
+    def get_available_recipes(self):
+        """ Returns QuerySet of all Recipes which are provided by the Layers
+        currently added to the Project """
+        project_layers = self.projectlayer_equivalent_set()
+        queryset = Recipe.objects.filter(layer_version__in = project_layers)
+
+        # Copied from get_all_compatible_recipes
+        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
+        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
+        # End copy
+
+        return queryset
+
+    def get_all_compatible_recipes(self):
+        """ Returns QuerySet of all the compatible Recipes available to the
+        project including ones from Layers not currently added """
+        compatible_layerversions = self.compatible_layerversions()
+        queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions)
+
+        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
+
+        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
+        return queryset
+
+
     def schedule_build(self):
         from bldcontrol.models import BuildRequest, BRTarget, BRLayer, BRVariable, BRBitbake
         br = BuildRequest.objects.create(project = self)
Signature · 1 line
2.1.4
author
#3 lore raw

PATCH 3/12 bitbake: toastergui: tables Use util functions for the common querysets

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

Use the new utils functions on the project object to get the common querysets that we also use in tables.

---
 lib/toaster/toastergui/tables.py | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
lib/toaster/toastergui/tables.py +3 -8
index 8d5166be..e4cbec1 100644
@@ -251,9 +251,8 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin):
 
     def setup_queryset(self, *args, **kwargs):
         prj = Project.objects.get(pk = kwargs['pid'])
-        compatible_layers = prj.compatible_layerversions()
-
-        self.queryset = Machine.objects.filter(layer_version__in=compatible_layers).order_by(self.default_orderby)
+        self.queryset = prj.get_all_compatible_machines()
+        self.queryset = self.queryset.order_by(self.default_orderby)
 
     def setup_columns(self, *args, **kwargs):
 
@@ -363,11 +362,7 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin):
     def setup_queryset(self, *args, **kwargs):
         prj = Project.objects.get(pk = kwargs['pid'])
 
-        self.queryset = Recipe.objects.filter(layer_version__in = prj.compatible_layerversions())
-
-        search_maxids = map(lambda i: i[0], list(self.queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
-
-        self.queryset = self.queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
+        self.queryset = prj.get_all_compatible_recipes()
         self.queryset = self.queryset.order_by(self.default_orderby)
 
 
Signature · 1 line
2.1.4
author
#6 lore raw

PATCH 6/12 bitbake: toastergui: Switch to using the new toaster typeahead widget

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

Switch the existing typeahead inputs to use the new typeahead widget. This means we have a defined mechanism and end point for typeaheads which meets the design specification.

---
 lib/toaster/toastergui/static/js/base.js         | 26 ++++++++++++++++--------
 lib/toaster/toastergui/static/js/importlayer.js  | 12 +++++------
 lib/toaster/toastergui/static/js/layerdetails.js |  2 +-
 lib/toaster/toastergui/static/js/libtoaster.js   |  5 ++---
 lib/toaster/toastergui/static/js/projectpage.js  |  4 ++--
 lib/toaster/toastergui/templates/base.html       |  8 +++++---
 lib/toaster/toastergui/views.py                  |  6 ++++--
 7 files changed, 37 insertions(+), 26 deletions(-)
lib/toaster/toastergui/static/js/base.js +17 -9
index f253361..eba9c16 100644
@@ -89,8 +89,8 @@ function basePageInit(ctx) {
   }
 
   /* If we have a project setup the typeahead */
-  if (selectedProject.projectTargetsUrl){
-    libtoaster.makeTypeahead(newBuildTargetInput, selectedProject.projectTargetsUrl, { format: "json" }, function (item) {
+  if (selectedProject.recipesTypeAheadUrl){
+    libtoaster.makeTypeahead(newBuildTargetInput, selectedProject.recipesTypeAheadUrl, { format: "json" }, function (item) {
       selectedTarget = item;
       newBuildTargetBuildBtn.removeAttr("disabled");
     });
@@ -156,7 +156,7 @@ function basePageInit(ctx) {
       $('#project .icon-pencil').hide();
     }
 
-    libtoaster.makeTypeahead(newBuildProjectInput, selectedProject.projectsUrl, { format : "json" }, function (item) {
+    libtoaster.makeTypeahead(newBuildProjectInput, selectedProject.projectsTypeAheadUrl, { format : "json" }, function (item) {
       /* successfully selected a project */
       newBuildProjectSaveBtn.removeAttr("disabled");
       selectedProject = item;
@@ -180,13 +180,21 @@ function basePageInit(ctx) {
 
       newBuildTargetInput.removeAttr("disabled");
 
-      /* Update the typeahead to use the new selectedProject */
-      libtoaster.makeTypeahead(newBuildTargetInput, selectedProject.projectTargetsUrl, { format: "json" }, function (item) {
-        /* successfully selected a target */
-        selectedTarget = item;
-        newBuildTargetBuildBtn.removeAttr("disabled");
-      });
+      /* We've got a new project so now we need to update the
+       * target urls. We can get this from the new project's info
+       */
+      $.getJSON(selectedProject.projectPageUrl, { format: "json" },
+        function(projectInfo){
+          /* Update the typeahead to use the new selectedProject */
+          selectedProject = projectInfo;
+
+          libtoaster.makeTypeahead(newBuildTargetInput, selectedProject.recipesTypeAheadUrl, { format: "json" }, function (item) {
+              /* successfully selected a target */
+              selectedTarget = item;
+              newBuildTargetBuildBtn.removeAttr("disabled");
+          });
 
+      });
       newBuildTargetInput.val("");
 
       /* set up new form aspect */
lib/toaster/toastergui/static/js/importlayer.js +6 -6
index 50bc4dd..2fadbc0 100644
@@ -16,7 +16,7 @@ function importLayerPageInit (ctx) {
   var currentLayerDepSelection;
   var validLayerName = /^(\w|-)+$/;
 
-  libtoaster.makeTypeahead(layerDepInput, libtoaster.ctx.projectLayersUrl, { include_added: "true" }, function(item){
+  libtoaster.makeTypeahead(layerDepInput, libtoaster.ctx.layersTypeAheadUrl, { include_added: "true" }, function(item){
     currentLayerDepSelection = item;
 
     layerDepBtn.removeAttr("disabled");
@@ -26,11 +26,11 @@ function importLayerPageInit (ctx) {
   /* We automatically add "openembedded-core" layer for convenience as a
    * dependency as pretty much all layers depend on this one
    */
-  $.getJSON(libtoaster.ctx.projectLayersUrl,
-    { include_added: "true" , search: "openembedded-core", format: "json" },
+  $.getJSON(libtoaster.ctx.layersTypeAheadUrl,
+    { include_added: "true" , search: "openembedded-core" },
     function(layer) {
-    if (layer.rows.length > 0) {
-      currentLayerDepSelection = layer.rows[0];
+    if (layer.results.length > 0) {
+      currentLayerDepSelection = layer.results[0];
       layerDepBtn.click();
     }
   });
@@ -211,7 +211,7 @@ function importLayerPageInit (ctx) {
       var name = $(this).val();
 
       /* Check if the layer name exists */
-      $.getJSON(libtoaster.ctx.projectLayersUrl,
+      $.getJSON(libtoaster.ctx.layersTypeAheadUrl,
         { include_added: "true" , search: name, format: "json" },
         function(layer) {
           if (layer.rows.length > 0) {
lib/toaster/toastergui/static/js/layerdetails.js +1 -1
index be6bbcd..96372f0 100644
@@ -9,7 +9,7 @@ function layerDetailsPageInit (ctx) {
   var addRmLayerBtn = $("#add-remove-layer-btn");
 
   /* setup the dependencies typeahead */
-  libtoaster.makeTypeahead(layerDepInput, libtoaster.ctx.projectLayersUrl, { include_added: "true" }, function(item){
+  libtoaster.makeTypeahead(layerDepInput, libtoaster.ctx.layersTypeAheadUrl, { include_added: "true" }, function(item){
     currentLayerDepSelection = item;
 
     layerDepBtn.removeAttr("disabled");
lib/toaster/toastergui/static/js/libtoaster.js +2 -3
index 22377f0..51054c7 100644
@@ -19,14 +19,13 @@ var libtoaster = (function (){
     jQElement.typeahead({
         source: function(query, process){
           xhrParams.search = query;
-          xhrParams.format = "json";
           $.getJSON(xhrUrl, this.options.xhrParams, function(data){
             if (data.error !== "ok") {
               console.log("Error getting data from server "+data.error);
               return;
             }
 
-            return process (data.rows);
+            return process(data.results);
           });
         },
         updater: function(item) {
@@ -40,7 +39,7 @@ var libtoaster = (function (){
             return 0;
           }
 
-          return ~item.name.toLowerCase().indexOf(this.query.toLowerCase());
+          return 1;
         },
         highlighter: function (item) {
           if (item.hasOwnProperty('detail'))
lib/toaster/toastergui/static/js/projectpage.js +2 -2
index b7cb074..b82f740 100644
@@ -98,7 +98,7 @@ function projectPageInit(ctx) {
 
   /* Add/Rm layer functionality */
 
-  libtoaster.makeTypeahead(layerAddInput, libtoaster.ctx.projectLayersUrl, { include_added: "false" }, function(item){
+  libtoaster.makeTypeahead(layerAddInput, libtoaster.ctx.layersTypeAheadUrl, { include_added: "false" }, function(item){
     currentLayerAddSelection = item;
     layerAddBtn.removeAttr("disabled");
   });
@@ -251,7 +251,7 @@ function projectPageInit(ctx) {
     machineNameTitle.text(machineName);
   }
 
-  libtoaster.makeTypeahead(machineChangeInput, libtoaster.ctx.projectMachinesUrl, { }, function(item){
+  libtoaster.makeTypeahead(machineChangeInput, libtoaster.ctx.machinesTypeAheadUrl, { }, function(item){
     currentMachineAddSelection = item;
     machineChangeBtn.removeAttr("disabled");
   });
lib/toaster/toastergui/templates/base.html +5 -3
index c16fd65..cd4269d 100644
@@ -32,14 +32,16 @@
         jsUrl : "{% static 'js/' %}",
         htmlUrl : "{% static 'html/' %}",
         projectsUrl : "{% url 'all-projects' %}",
+        projectsTypeAheadUrl: {% url 'xhr_projectstypeahead' as prjurl%}{{prjurl|json}},
         {% if project.id %}
         projectId : {{project.id}},
         projectPageUrl : {% url 'project' project.id as purl%}{{purl|json}},
         projectName : {{project.name|json}},
-        projectTargetsUrl: {% url 'projectavailabletargets' project.id as paturl%}{{paturl|json}},
+        recipesTypeAheadUrl: {% url 'xhr_recipestypeahead' project.id as paturl%}{{paturl|json}},
+        layersTypeAheadUrl: {% url 'xhr_layerstypeahead' project.id as paturl%}{{paturl|json}},
+        machinesTypeAheadUrl: {% url 'xhr_machinestypeahead' project.id as paturl%}{{paturl|json}},
+
         projectBuildsUrl: {% url 'projectbuilds' project.id as pburl %}{{pburl|json}},
-        projectLayersUrl: {% url 'projectlayers' project.id as plurl %}{{plurl|json}},
-        projectMachinesUrl: {% url 'projectmachines' project.id as pmurl %}{{pmurl|json}},
         projectId : {{project.id}},
         {% else %}
         projectId : undefined,
lib/toaster/toastergui/views.py +4 -2
index b43a01e..b7bfb9a 100755
@@ -2235,6 +2235,8 @@ if True:
             "freqtargets": freqtargets[:5],
             "releases": map(lambda x: {"id": x.pk, "name": x.name, "description":x.description}, Release.objects.all()),
             "project_html": 1,
+            "recipesTypeAheadUrl": reverse('xhr_recipestypeahead', args=(prj.pk,)),
+            "projectBuildsUrl": reverse('projectbuilds', args=(prj.pk,)),
         }
 
         if prj.release is not None:
@@ -2784,9 +2786,9 @@ if True:
         for p in project_info.object_list:
             p.id = p.pk
             p.projectPageUrl = reverse('project', args=(p.id,))
-            p.projectLayersUrl = reverse('projectlayers', args=(p.id,))
+            p.layersTypeAheadUrl = reverse('xhr_layerstypeahead', args=(p.id,))
+            p.recipesTypeAheadUrl = reverse('xhr_recipestypeahead', args=(p.id,))
             p.projectBuildsUrl = reverse('projectbuilds', args=(p.id,))
-            p.projectTargetsUrl = reverse('projectavailabletargets', args=(p.id,))
 
         # build view-specific information; this is rendered specifically in the builds page, at the top of the page (i.e. Recent builds)
         build_mru = _get_latest_builds()
Signature · 1 line
2.1.4
author
#8 lore raw

PATCH 8/12 bitbake: toasterui: views Remove unused xhr_typeahead view definition

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

The one thing left being used in this definition was a response which contains the list of layers which would be deleted if you change the project release. This patch moves that to it's own url/endpoint and updates the frontend reference which is using it.

---
 lib/toaster/toastergui/static/js/projectpage.js |  4 +-
 lib/toaster/toastergui/templates/project.html   |  3 +-
 lib/toaster/toastergui/urls.py                  |  8 +++-
 lib/toaster/toastergui/views.py                 | 60 ++++++++++---------------
 4 files changed, 32 insertions(+), 43 deletions(-)
lib/toaster/toastergui/static/js/projectpage.js +2 -2
index b82f740..146319e 100644
@@ -406,8 +406,8 @@ function projectPageInit(ctx) {
 
     var newRelease = releaseForm.find("option:selected").data('release');
 
-    $.getJSON(ctx.typeaheadUrl,
-      { search: newRelease.id, type: "versionlayers" },
+    $.getJSON(ctx.testReleaseChangeUrl,
+      { new_release_id: newRelease.id },
       function(layers) {
         if (layers.rows.length === 0){
           /* No layers to change for this release */
lib/toaster/toastergui/templates/project.html +1 -2
index aad79b4..4cabad4 100644
@@ -11,8 +11,7 @@
 <script>
   $(document).ready(function (){
     var ctx = {
-      typeaheadUrl : "{% url 'xhr_datatypeahead' project.id %}",
-
+      testReleaseChangeUrl: "{% url 'xhr_testreleasechange' project.id %}",
     };
 
     try {
lib/toaster/toastergui/urls.py +6 -2
index b44c42f..d65ad2b 100644
@@ -126,8 +126,6 @@ urlpatterns = patterns('toastergui.views',
             name=tables.LayerMachinesTable.__name__.lower()),
 
 
-        url(r'^xhr_datatypeahead/(?P<pid>\d+)$', 'xhr_datatypeahead', name='xhr_datatypeahead'),
-        url(r'^xhr_configvaredit/(?P<pid>\d+)$', 'xhr_configvaredit', name='xhr_configvaredit'),
         # typeahead api end points
         url(r'^xhr_typeahead/(?P<pid>\d+)/layers$',
             typeaheads.LayersTypeAhead.as_view(), name='xhr_layerstypeahead'),
@@ -139,6 +137,12 @@ urlpatterns = patterns('toastergui.views',
             typeaheads.ProjectsTypeAhead.as_view(), name='xhr_projectstypeahead'),
 
 
+
+        url(r'^xhr_testreleasechange/(?P<pid>\d+)$', 'xhr_testreleasechange',
+            name='xhr_testreleasechange'),
+        url(r'^xhr_configvaredit/(?P<pid>\d+)$', 'xhr_configvaredit',
+            name='xhr_configvaredit'),
+
         url(r'^xhr_importlayer/$', 'xhr_importlayer', name='xhr_importlayer'),
         url(r'^xhr_updatelayer/$', 'xhr_updatelayer', name='xhr_updatelayer'),
 
lib/toaster/toastergui/views.py +23 -37

74 lines not on this page · raw message

Signature · 1 line
2.1.4
author
#7 lore raw

PATCH 7/12 bitbake: toastergui: libtoaster Throw an exception no url is specified

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

At a minimum for this typeahead to work we need a url parameter to call for a JSON reponse of items to filter on.

---
 lib/toaster/toastergui/static/js/libtoaster.js | 2 ++
 1 file changed, 2 insertions(+)
lib/toaster/toastergui/static/js/libtoaster.js +2 -0
index 51054c7..96e28b0 100644
@@ -15,6 +15,8 @@ var libtoaster = (function (){
    *  arg of the item.
    */
   function _makeTypeahead (jQElement, xhrUrl, xhrParams, selectedCB) {
+    if (!xhrUrl || xhrUrl.length === 0)
+      throw("No url to typeahead supplied");
 
     jQElement.typeahead({
         source: function(query, process){
Signature · 1 line
2.1.4
author
#9 lore raw

PATCH 9/12 bitbake: toastergui: views Standardise the fields project layer response

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

We need to keep consistent the field names of these objects so that we can use the object response from different calls. e.g. layer dependences or listing layers in the project.

---
 lib/toaster/toastergui/views.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/toaster/toastergui/views.py +3 -2
index 6a219ed..4563eaf 100755
@@ -2224,9 +2224,10 @@ if True:
                         "id": x.layercommit.pk,
                         "orderid": x.pk,
                         "name" : x.layercommit.layer.name,
-                        "giturl": x.layercommit.layer.vcs_url,
+                        "vcs_url": x.layercommit.layer.vcs_url,
+                        "vcs_reference" : x.layercommit.get_vcs_reference(),
                         "url": x.layercommit.layer.layer_index_url,
-                        "layerdetailurl": reverse("layerdetails", args=(prj.id, x.layercommit.pk,)),
+                        "layerdetailurl": x.layercommit.get_detailspage_url(prj.pk),
                 # This branch name is actually the release
                         "branch" : { "name" : x.layercommit.get_vcs_reference(), "layersource" : x.layercommit.up_branch.layer_source.name if x.layercommit.up_branch != None else None}},
                     prj.projectlayer_set.all().order_by("id")),
Signature · 1 line
2.1.4
author
#10 lore raw

PATCH 10/12 bitbake: toastergui: tests Fix and more comprehensive typeahead unittest

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

Fix the currently broken unit test for typeaheads and add some additional fields to check for in layers, projects, recipes and machines typeaheads.

---
 lib/toaster/toastergui/tests.py | 88 ++++++++++++++++++++++++++++++++---------
 1 file changed, 69 insertions(+), 19 deletions(-)
lib/toaster/toastergui/tests.py +69 -19
index 77e80fe..8a59af4 100644
@@ -1,7 +1,8 @@
 from django.test import TestCase
 from django.core.urlresolvers import reverse
 from orm.models import Project, Release, BitbakeVersion, Build
-from orm.models import ReleaseLayerSourcePriority, LayerSource, Layer, Layer_Version
+from orm.models import ReleaseLayerSourcePriority, LayerSource, Layer, Layer_Version, Recipe, Machine, ProjectLayer
+import json
 
 class ProvisionedProjectTestCase(TestCase):
     TEST_PROJECT_NAME = "test project"
@@ -27,7 +28,6 @@ class AllProjectsViewTestCase(ProvisionedProjectTestCase):
         self.assertTrue(response['Content-Type'].startswith('application/json'))
 
         try:
-            import json
             data = json.loads(response.content)
         except:
             self.fail("Response %s is not json-loadable" % response.content)
@@ -44,6 +44,9 @@ class AllProjectsViewTestCase(ProvisionedProjectTestCase):
 
 class ProvisionedLayersProjectTestCase(ProvisionedProjectTestCase):
     LAYER_NAME = "base-layer"
+    RECIPE_NAME = "base-recipe"
+
+
     def setUp(self):
         super(ProvisionedLayersProjectTestCase, self).setUp()
         self.layersource, created = LayerSource.objects.get_or_create(sourcetype = LayerSource.TYPE_IMPORTED)
@@ -51,6 +54,13 @@ class ProvisionedLayersProjectTestCase(ProvisionedProjectTestCase):
         self.layer, created = Layer.objects.get_or_create(name=XHRDataTypeAheadTestCase.LAYER_NAME, layer_source=self.layersource, vcs_url="/tmp/")
         self.lv, created = Layer_Version.objects.get_or_create(layer = self.layer, project = self.project, layer_source=self.layersource, commit="master")
 
+        self.recipe, created = Recipe.objects.get_or_create(layer_source=self.layersource, name=ProvisionedLayersProjectTestCase.RECIPE_NAME, version="1.2", summary="one recipe", description="recipe", layer_version=self.lv)
+
+        self.machine, created = Machine.objects.get_or_create(layer_version=self.lv, name="wisk", description="wisking machine")
+
+        ProjectLayer.objects.get_or_create(project = self.project,
+                                           layercommit = self.lv)
+
 
 class XHRDataTypeAheadTestCase(ProvisionedLayersProjectTestCase):
 
@@ -58,20 +68,60 @@ class XHRDataTypeAheadTestCase(ProvisionedLayersProjectTestCase):
         super(XHRDataTypeAheadTestCase, self).setUp()
         self.assertTrue(self.lv in self.project.compatible_layerversions())
 
-    def test_xhr_datatypeahead_layer(self):
-        response = self.client.get(reverse('xhr_datatypeahead', args=(self.project.id,)), {"type": "layerdeps"})
-        self.assertEqual(response.status_code, 200)
-        self.assertTrue(response['Content-Type'].startswith('application/json'))
-
-        try:
-            import json
-            data = json.loads(response.content)
-        except:
-            self.fail("Response %s is not json-loadable" % response.content)
-
-        self.assertTrue("error" in data)
-        self.assertEqual(data["error"], "ok")
-        self.assertTrue("list" in data)
-        self.assertTrue(len(data["list"]) > 0)
-
-        self.assertTrue(XHRDataTypeAheadTestCase.LAYER_NAME in map(lambda x: x["name"], data["list"]))
+    def test_typeaheads(self):
+        layers_url = reverse('xhr_layerstypeahead', args=(self.project.id,))
+        prj_url = reverse('xhr_projectstypeahead')
+
+        urls = [ layers_url,
+                 prj_url,
+                 reverse('xhr_recipestypeahead', args=(self.project.id,)),
+                 reverse('xhr_machinestypeahead', args=(self.project.id,)),
+               ]
+
+        def basic_reponse_check(reponse, url):
+            self.assertEqual(response.status_code, 200)
+            self.assertTrue(response['Content-Type'].startswith('application/json'))
+
+            try:
+                data = json.loads(response.content)
+            except:
+                self.fail("Response %s is not json-loadable" % response.content)
+            self.assertTrue("error" in data)
+            self.assertEqual(data["error"], "ok")
+            self.assertTrue("results" in data)
+
+            # We got a result so now check the fields
+            if len(data['results']) > 0:
+                result = data['results'][0]
+
+                self.assertTrue(len(result['name']) > 0)
+                self.assertTrue("detail" in result)
+                self.assertTrue(result['id'] > 0)
+
+                # Special check for the layers typeahead's extra fields
+                if url == layers_url:
+                    self.assertTrue(len(result['layerdetailurl']) > 0)
+                    self.assertTrue(len(result['vcs_url']) > 0)
+                    self.assertTrue(len(result['vcs_reference']) > 0)
+                # Special check for project typeahead extra fields
+                elif url == prj_url:
+                    self.assertTrue(len(result['projectPageUrl']) > 0)
+
+                return True
+
+            return False
+
+        import string
+
+        for url in urls:
+            results = False
+
+            for typeing in list(string.ascii_letters):
+                response = self.client.get(url, { 'search' : typeing })
+                results = basic_reponse_check(response, url)
+                if results:
+                    break
+
+            # After "typeing" the alpabet we should have result true
+            # from each of the urls
+            self.assertTrue(results)
Signature · 1 line
2.1.4
author
#12 lore raw

PATCH 12/12 bitbake: toastergui: libtoaster typeahead Add in results highlighter

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

Add a highlighter for the results in the typeahead, this highlights the part of the results which is currently being matched. This is a modified version of the bootstrap stock function with added escaping and the addition of the details information.

---
 lib/toaster/toastergui/static/js/libtoaster.js | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
lib/toaster/toastergui/static/js/libtoaster.js +8 -4
index f6979f6..6ec0337 100644
@@ -56,10 +56,14 @@ var libtoaster = (function (){
           return 1;
         },
         highlighter: function (item) {
-          if (item.hasOwnProperty('detail'))
-            /* Use jquery to escape the value as text into a span */
-            return $('<span></span>').text(item.name+' '+item.detail).get(0);
-          return $('<span></span>').text(item.name).get(0);
+          /* Use jquery to escape the item name and detail */
+          var current = $("<span></span>").text(item.name + ' '+item.detail);
+          current = current.html();
+
+          var query = this.query.replace(/[\-\[\]{}()*+?.,\\\^$|#\s]/g, '\\$&')
+          return current.replace(new RegExp('(' + query + ')', 'ig'), function ($1, match) {
+            return '<strong>' + match + '</strong>'
+          })
         },
         sorter: function (items) { return items; },
         xhrUrl: xhrUrl,
Signature · 1 line
2.1.4
author
#1 lore raw

PATCH 1/12 bitbake: toaster: orm Add util function to return the url to layerversion

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

Save duplicating this call and make a utility function for it on the Layer_Version object.

---
 lib/toaster/orm/models.py | 4 ++++
 1 file changed, 4 insertions(+)
lib/toaster/orm/models.py +4 -0
index 2d7ef09..3b72f80 100644
@@ -23,6 +23,7 @@ from django.db import models
 from django.db.models import F, Q, Avg
 from django.utils import timezone
 
+from django.core.urlresolvers import reverse
 
 from django.core import validators
 from django.conf import settings
@@ -1109,6 +1110,9 @@ class Layer_Version(models.Model):
             return self.up_branch.name
         return ("Cannot determine the vcs_reference for layer version %s" % vars(self))
 
+    def get_detailspage_url(self, project_id):
+        return reverse('layerdetails', args=(project_id, self.pk))
+
     def __unicode__(self):
         return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
 
Signature · 1 line
2.1.4
author
#4 lore raw

PATCH 4/12 bitbake: toastergui: widgets Add a typeahead widget

From: Michael Wood <[email protected]>

The typeahead behaviour is significantly different from searching in a table so we need a separate widget to do this.

[YOCTO #7152]

---
 lib/toaster/toastergui/widgets.py | 52 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
lib/toaster/toastergui/widgets.py +52 -0
index 0885402..2adf574 100644
@@ -354,3 +354,55 @@ class ToasterTemplateView(TemplateView):
                             content_type = "application/json; charset=utf-8")
 
         return super(ToasterTemplateView, self).get(*args, **kwargs)
+
+class ToasterTypeAhead(View):
+    """ A typeahead mechanism to support the front end typeahead widgets """
+    MAX_RESULTS = 6
+
+    class MissingFieldsException(Exception):
+        pass
+
+    def __init__(self, *args, **kwargs):
+        super(ToasterTypeAhead, self).__init__()
+
+    def get(self, request, *args, **kwargs):
+        def response(data):
+            return HttpResponse(json.dumps(data,
+                                           indent=2,
+                                           cls=DjangoJSONEncoder),
+                                content_type="application/json")
+
+        error = "ok"
+
+        search_term = request.GET.get("search", None)
+        if search_term == None:
+            # We got no search value so return empty reponse
+            return response({'error' : error , 'results': []})
+
+        try:
+            prj = Project.objects.get(pk=kwargs['pid'])
+        except KeyError:
+            prj = None
+
+        results = self.apply_search(search_term, prj, request)[:ToasterTypeAhead.MAX_RESULTS]
+
+        if len(results) > 0:
+            try:
+                self.validate_fields(results[0])
+            except MissingFieldsException as e:
+                error = e
+
+        data = { 'results' : results,
+                'error' : error,
+               }
+
+        return response(data)
+
+    def validate_fields(self, result):
+        if 'name' in result == False or 'detail' in result == False:
+            raise MissingFieldsException("name and detail are required fields")
+
+    def apply_search(self, search_term, prj):
+        """ Override this function to implement search. Return an array of
+        dictionaries with a minium of a name and detail field"""
+        pass
Signature · 1 line
2.1.4
author
#5 lore raw

PATCH 5/12 bitbake: toastergui: Add typeaheads layers, machines, projects, recipes

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

Implementation of typeahead widgets for layers machines and recipes typeahead.

[YOCTO #7152]

---
 lib/toaster/toastergui/typeaheads.py | 145 +++++++++++++++++++++++++++++++++++
 lib/toaster/toastergui/urls.py       |  11 +++
 2 files changed, 156 insertions(+)
 create mode 100644 bitbake/lib/toaster/toastergui/typeaheads.py
lib/toaster/toastergui/typeaheads.py new file +145 -0
new file mode 100644
index 0000000..d5bec58
@@ -0,0 +1,145 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2015        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from toastergui.widgets import ToasterTypeAhead
+from orm.models import Project
+from django.core.urlresolvers import reverse
+
+class LayersTypeAhead(ToasterTypeAhead):
+    """ Typeahead for layers available and not added in the current project's
+    configuration """
+    def __init__(self):
+      super(LayersTypeAhead, self).__init__()
+
+    def apply_search(self, search_term, prj, request):
+        layers = prj.compatible_layerversions()
+        layers = layers.order_by('layer__name')
+
+        # Unlike the other typeaheads we also don't want to show suggestions
+        # for layers already in the project unless required such as when adding
+        # layerdeps to a new layer.
+        if ("include_added" in request.GET and
+                request.GET['include_added'] != "true"):
+            layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set)
+
+        primary_results = layers.filter(layer__name__istartswith=search_term)
+        secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)
+
+        results = []
+
+        for layer_version in list(primary_results) + list(secondary_results):
+            vcs_reference = layer_version.get_vcs_reference()
+
+            detail = "[ %s | %s ]" % (layer_version.layer.vcs_url,
+                                      vcs_reference)
+            needed_fields = {
+                'id' : layer_version.pk,
+                'name' : layer_version.layer.name,
+                'layerdetailurl' : layer_version.get_detailspage_url(prj.pk),
+                'vcs_url' : layer_version.layer.vcs_url,
+                'vcs_reference' : vcs_reference,
+                'detail' : detail,
+            }
+
+            results.append(needed_fields)
+
+        return results
+
+class MachinesTypeAhead(ToasterTypeAhead):
+    """ Typeahead for all the machines available in the current project's
+    configuration """
+    def __init__(self):
+        super(MachinesTypeAhead, self).__init__()
+
+    def apply_search(self, search_term, prj, request):
+        machines = prj.get_available_machines()
+        machines = machines.order_by("name")
+
+        primary_results = machines.filter(name__istartswith=search_term)
+        secondary_results = machines.filter(name__icontains=search_term).exclude(pk__in=primary_results)
+        tertiary_results = machines.filter(layer_version__layer__name__icontains=search_term).exclude(pk__in=primary_results).exclude(pk__in=secondary_results)
+
+        results = []
+
+        for machine in list(primary_results) + list(secondary_results) + list(tertiary_results):
+
+            detail = "[ %s ]" % (machine.layer_version.layer.name)
+            needed_fields = {
+                'id' : machine.pk,
+                'name' : machine.name,
+                'detail' : detail,
+            }
+
+            results.append(needed_fields)
+
+        return results
+
+class RecipesTypeAhead(ToasterTypeAhead):
+    """ Typeahead for all the recipes available in the current project's
+    configuration """
+    def __init__(self):
+        super(RecipesTypeAhead, self).__init__()
+
+    def apply_search(self, search_term, prj, request):
+        recipes = prj.get_available_recipes()
+        recipes = recipes.order_by("name")
+
+
+        primary_results = recipes.filter(name__istartswith=search_term)
+        secondary_results = recipes.filter(name__icontains=search_term).exclude(pk__in=primary_results)
+        tertiary_results = recipes.filter(layer_version__layer__name__icontains=search_term).exclude(pk__in=primary_results).exclude(pk__in=secondary_results)
+
+        results = []
+
+        for recipe in list(primary_results) + list(secondary_results) + list(tertiary_results):
+
+            detail = "[ %s ]" % (recipe.layer_version.layer.name)
+            needed_fields = {
+                'id' : recipe.pk,
+                'name' : recipe.name,
+                'detail' : detail,
+            }
+
+            results.append(needed_fields)
+
+        return results
+
+class ProjectsTypeAhead(ToasterTypeAhead):
+    """ Typeahead for all the projects """
+    def __init__(self):
+        super(ProjectsTypeAhead, self).__init__()
+
+    def apply_search(self, search_term, prj, request):
+        projects = Project.objects.all().order_by("name")
+
+        primary_results = projects.filter(name__istartswith=search_term)
+        secondary_results = projects.filter(name__icontains=search_term).exclude(pk__in=primary_results)
+
+        results = []
+
+        for project in list(primary_results) + list(secondary_results):
+            needed_fields = {
+                'id' : project.pk,
+                'name' : project.name,
+                'detail' : "",
+                'projectPageUrl' : reverse('project', args=(project.pk,))
+            }
+
+            results.append(needed_fields)
+
+        return results
lib/toaster/toastergui/urls.py +11 -0
index beb4303..b44c42f 100644
@@ -21,6 +21,7 @@ from django.views.generic import RedirectView, TemplateView
 
 from django.http import HttpResponseBadRequest
 from toastergui import tables
+from toastergui import typeaheads
 
 urlpatterns = patterns('toastergui.views',
         # landing page
@@ -127,6 +128,16 @@ urlpatterns = patterns('toastergui.views',
 
         url(r'^xhr_datatypeahead/(?P<pid>\d+)$', 'xhr_datatypeahead', name='xhr_datatypeahead'),
         url(r'^xhr_configvaredit/(?P<pid>\d+)$', 'xhr_configvaredit', name='xhr_configvaredit'),
+        # typeahead api end points
+        url(r'^xhr_typeahead/(?P<pid>\d+)/layers$',
+            typeaheads.LayersTypeAhead.as_view(), name='xhr_layerstypeahead'),
+        url(r'^xhr_typeahead/(?P<pid>\d+)/machines$',
+            typeaheads.MachinesTypeAhead.as_view(), name='xhr_machinestypeahead'),
+        url(r'^xhr_typeahead/(?P<pid>\d+)/recipes$',
+            typeaheads.RecipesTypeAhead.as_view(), name='xhr_recipestypeahead'),
+        url(r'^xhr_typeahead/projects$',
+            typeaheads.ProjectsTypeAhead.as_view(), name='xhr_projectstypeahead'),
+
 
         url(r'^xhr_importlayer/$', 'xhr_importlayer', name='xhr_importlayer'),
         url(r'^xhr_updatelayer/$', 'xhr_updatelayer', name='xhr_updatelayer'),
Signature · 1 line
2.1.4
author
#11 lore raw

PATCH 11/12 bitbake: toastergui: libtoaster: typeahead resiliency for slow server

In reply to Ed Bartosh

From: Michael Wood <[email protected]>

When we have a slow request we don't want to see the typeahead results changing in the middle of typing the item. To do this we make sure that requests to the typeahead can only happen sequentially and that if results have been retrieved but the input is now empty we don't bother showing the results.

---
 lib/toaster/toastergui/static/js/libtoaster.js | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
lib/toaster/toastergui/static/js/libtoaster.js +17 -1
index 96e28b0..f6979f6 100644
@@ -18,15 +18,24 @@ var libtoaster = (function (){
     if (!xhrUrl || xhrUrl.length === 0)
       throw("No url to typeahead supplied");
 
+    var xhrReq;
+
     jQElement.typeahead({
         source: function(query, process){
           xhrParams.search = query;
-          $.getJSON(xhrUrl, this.options.xhrParams, function(data){
+
+          /* If we have a request in progress don't fire off another one*/
+          if (xhrReq)
+            xhrReq.abort();
+
+          xhrReq = $.getJSON(xhrUrl, this.options.xhrParams, function(data){
             if (data.error !== "ok") {
               console.log("Error getting data from server "+data.error);
               return;
             }
 
+            xhrReq = null;
+
             return process(data.results);
           });
         },
@@ -41,6 +50,9 @@ var libtoaster = (function (){
             return 0;
           }
 
+          if (this.$element.val().length === 0)
+            return 0;
+
           return 1;
         },
         highlighter: function (item) {
@@ -52,6 +64,7 @@ var libtoaster = (function (){
         sorter: function (items) { return items; },
         xhrUrl: xhrUrl,
         xhrParams: xhrParams,
+        xhrReq: xhrReq,
     });
 
 
@@ -530,6 +543,9 @@ $(document).ready(function() {
     });
 
     $(document).ajaxError(function(event, jqxhr, settings, errMsg){
+      if (errMsg === 'abort')
+        return;
+
       console.warn("Problem with xhr call");
       console.warn(errMsg);
       console.warn(jqxhr.responseText);
Signature · 1 line
2.1.4