Skip to contentSkip to conversation

image customization

PATCH New brian avery 2 participants 25 messages Sep 29, 2015

lore
Share & tools
lore
#0 lore raw

cover image customization

This large patch set brings the databse changes needed to support image customization as well as the skeleton of the image customization user interface. The ui for image customization is off/hidden by default. The database changes fix bugs 7967,8035,8147, and 8403.

Thanks,
Brian

The following changes since commit f78f90240a3041be426c6ba9085849352164ee8b:

bitbake: prserv/serv.py: Better messaging when starting/stopping the server with port=0 (2015-09-28 12:00:33 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib bavery/submit/michaelw/20150928_ic5.1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=bavery/submit/michaelw/20150928_ic5.1

Ed Bartosh (5):
toaster: make a workaround for old style index
toaster: create custom layer and recipes for Image customisation
toaster: implement decorator for REST responses
toaster: Add new ReST API for Image Customisation feature
toaster: Add test cases for new Image customisation features

Michael Wood (17):
toaster: tables Move the title and name into the widget
toaster: widgets ToasterTable add logger to notify when cache hit
toaster: widgets ToasterTable Add more info to search field exception
toaster: add nocache option to the ToasterTable widget
toaster: ToasterTable remove unused class definition
toaster: Add CustomImageRecipe model
toaster: add toggle for enabling image customisation feeature
toaster: Fix indentation of jsunittests view
toaster: Add ToasterTables for Image customisation feature
toaster: Add Image customisation frontend feature
toaster: Special case the openembedded-core layer to avoid duplicates
toaster: Create a relationship between build information and toaster
layers
toaster: Prioroitise the layer more generic vcs reference over the sha
toaster: tables show all recipes in the layerdetails even duplicates
toaster: buildinfohelper Create a copy of the built layer and recipe Revert "bitbake: toaster: don't re-create Target objects"
toaster: orm remove the complicated querying on the ORM

 lib/bb/ui/buildinfohelper.py                       |  95 +++++-
 lib/toaster/bldcontrol/localhostbecontroller.py    |  51 ++-
 .../0009_auto__add_field_brlayer_layer_version.py  | 180 ++++++++++
 lib/toaster/bldcontrol/models.py                   |   3 +-
 ...e__add_unique_customimagerecipe_name_project.py | 375 +++++++++++++++++++++
 lib/toaster/orm/models.py                          |  91 +++--
 lib/toaster/toastergui/static/js/customrecipe.js   |  50 +++
 lib/toaster/toastergui/static/js/layerBtn.js       |  13 +
 lib/toaster/toastergui/static/js/newcustomimage.js |  49 +++
 lib/toaster/toastergui/tables.py                   | 230 +++++++++++--
 .../toastergui/templates/baseprojectpage.html      |   7 +-
 .../toastergui/templates/customise_btn.html        |   9 +
 lib/toaster/toastergui/templates/customrecipe.html | 142 ++++++++
 .../toastergui/templates/newcustomimage.html       |  54 +++
 .../toastergui/templates/pkg_add_rm_btn.html       |  16 +
 lib/toaster/toastergui/templates/project.html      |   2 +-
 .../toastergui/templates/projecttopbar.html        |   9 +-
 lib/toaster/toastergui/tests.py                    | 187 +++++++++-
 lib/toaster/toastergui/typeaheads.py               |   5 +-
 lib/toaster/toastergui/urls.py                     |  49 ++-
 lib/toaster/toastergui/views.py                    | 195 ++++++++++-
 lib/toaster/toastergui/widgets.py                  |  50 ++-
 lib/toaster/toastermain/settings.py                |   5 +
 23 files changed, 1724 insertions(+), 143 deletions(-)
 create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py
 create mode 100644 bitbake/lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py
 create mode 100644 bitbake/lib/toaster/toastergui/static/js/customrecipe.js
 create mode 100644 bitbake/lib/toaster/toastergui/static/js/newcustomimage.js
 create mode 100644 bitbake/lib/toaster/toastergui/templates/customise_btn.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/customrecipe.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/newcustomimage.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/pkg_add_rm_btn.html
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/22 toaster: make a workaround for old style index

From: Ed Bartosh <[email protected]>

For a new style indexes 'inherits' field is used to determine if recipe is an image recipe.

As old style indexes don't have 'inherits' field this can be guessed from recipe name. Let's consider recipe an image recipe if recipe name contains '-image-'.

---
 lib/toaster/orm/models.py | 2 ++
 1 file changed, 2 insertions(+)
lib/toaster/orm/models.py +2 -0
index 883ecf4..b5c8643 100644
@@ -974,6 +974,8 @@ class LayerIndexLayerSource(LayerSource):
                 ro.file_path = ri['filepath'] + "/" + ri['filename']
                 if 'inherits' in ri:
                     ro.is_image = 'image' in ri['inherits'].split()
+                else: # workaround for old style layer index
+                    ro.is_image = "-image-" in ri['pn']
                 ro.save()
             except IntegrityError as e:
                 logger.debug("Failed saving recipe, ignoring: %s (%s:%s)" % (e, ro.layer_version, ri['filepath']+"/"+ri['filename']))
Signature · 1 line
1.9.1
#2 lore raw

PATCH 2/22 toaster: tables Move the title and name into the widget

From: Michael Wood <[email protected]>

For historical reasons this was being set in the urls definition. We can set this in the actual definition of the table and defaults in the widget.

---
 lib/toaster/toastergui/tables.py  |  2 ++
 lib/toaster/toastergui/urls.py    | 22 +++++++++++++---------
 lib/toaster/toastergui/widgets.py |  9 ++++++++-
 3 files changed, 23 insertions(+), 10 deletions(-)
lib/toaster/toastergui/tables.py +2 -0
index 92e3b5c..70e4b6d 100644
@@ -50,6 +50,7 @@ class LayersTable(ToasterTable):
     def __init__(self, *args, **kwargs):
         super(LayersTable, self).__init__(*args, **kwargs)
         self.default_orderby = "layer__name"
+        self.title = "Compatible layers"
 
     def get_context_data(self, **kwargs):
         context = super(LayersTable, self).get_context_data(**kwargs)
@@ -208,6 +209,7 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin):
     def __init__(self, *args, **kwargs):
         super(MachinesTable, self).__init__(*args, **kwargs)
         self.empty_state = "No machines maybe you need to do a build?"
+        self.title = "Compatible machines"
         self.default_orderby = "name"
 
     def get_context_data(self, **kwargs):
lib/toaster/toastergui/urls.py +13 -9
index 46e5761..55f325d 100644
@@ -87,15 +87,21 @@ urlpatterns = patterns('toastergui.views',
         # the table pages that have been converted to ToasterTable widget
         url(r'^project/(?P<pid>\d+)/machines/$',
             tables.MachinesTable.as_view(template_name="generic-toastertable-page.html"),
-            { 'table_name': tables.MachinesTable.__name__.lower(),
-              'title' : 'Compatible machines' },
             name="projectmachines"),
 
-        url(r'^project/(?P<pid>\d+)/recipes/$',
-            tables.RecipesTable.as_view(template_name="generic-toastertable-page.html"),
-            { 'table_name': tables.RecipesTable.__name__.lower(),
-              'title' : 'Compatible recipes' },
-            name="projecttargets"),
+        url(r'^project/(?P<pid>\d+)/softwarerecipes/$',
+            tables.SoftwareRecipesTable.as_view(template_name="generic-toastertable-page.html"),
+            name="projectsoftwarerecipes"),
+
+        url(r'^project/(?P<pid>\d+)/images/$',
+            tables.ImageRecipesTable.as_view(template_name="generic-toastertable-page.html"), name="projectimagerecipes"),
+
+        url(r'^project/(?P<pid>\d+)/customimages/$',
+            tables.CustomImagesTable.as_view(template_name="generic-toastertable-page.html"), name="projectcustomimages"),
+
+        url(r'^project/(?P<pid>\d+)/newcustomimage/$',
+            tables.NewCustomImagesTable.as_view(template_name="newcustomimage.html"),
+            name="newcustomimage"),
 
         url(r'^project/(?P<pid>\d+)/availablerecipes/$',
             tables.ProjectLayersRecipesTable.as_view(template_name="generic-toastertable-page.html"),
@@ -105,8 +111,6 @@ urlpatterns = patterns('toastergui.views',
 
         url(r'^project/(?P<pid>\d+)/layers/$',
             tables.LayersTable.as_view(template_name="generic-toastertable-page.html"),
-            { 'table_name': tables.LayersTable.__name__.lower(),
-              'title' : 'Compatible layers' },
             name="projectlayers"),
 
         url(r'^project/(?P<pid>\d+)/layer/(?P<layerid>\d+)$',
lib/toaster/toastergui/widgets.py +8 -1
index eb2914d..eb49692 100644
@@ -45,7 +45,7 @@ class ToasterTable(TemplateView):
         super(ToasterTable, self).__init__()
         if 'template_name' in kwargs:
             self.template_name = kwargs['template_name']
-        self.title = None
+        self.title = "Table"
         self.queryset = None
         self.columns = []
         self.filters = {}
@@ -61,6 +61,13 @@ class ToasterTable(TemplateView):
                         orderable=True,
                         field_name="id")
 
+    def get_context_data(self, **kwargs):
+        context = super(ToasterTable, self).get_context_data(**kwargs)
+        context['title'] = self.title
+        context['table_name'] =  type(self).__name__.lower()
+
+        return context
+
 
     def get(self, request, *args, **kwargs):
         if request.GET.get('format', None) == 'json':
Signature · 1 line
1.9.1
#3 lore raw

PATCH 3/22 toaster: create custom layer and recipes for Image customisation

In reply to brian avery

From: Ed Bartosh <[email protected]>

When building customised recipes toaster creates custom layer directory and puts layer.conf and custom recipes to it.

[YOCTO #8075]

---
 lib/toaster/bldcontrol/localhostbecontroller.py | 51 ++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 2 deletions(-)
lib/toaster/bldcontrol/localhostbecontroller.py +49 -2
index a9909b8..62fff1d 100644
@@ -23,9 +23,11 @@
 import os
 import sys
 import re
+import shutil
 from django.db import transaction
 from django.db.models import Q
 from bldcontrol.models import BuildEnvironment, BRLayer, BRVariable, BRTarget, BRBitbake
+from orm.models import CustomImageRecipe, Layer, Layer_Version, ProjectLayer
 import subprocess
 
 from toastermain import settings
@@ -197,7 +199,7 @@ class LocalhostBEController(BuildEnvironmentController):
         return local_checkout_path
 
 
-    def setLayers(self, bitbakes, layers):
+    def setLayers(self, bitbakes, layers, targets):
         """ a word of attention: by convention, the first layer for any build will be poky! """
 
         assert self.be.sourcedir is not None
@@ -299,6 +301,51 @@ class LocalhostBEController(BuildEnvironmentController):
         if not os.path.exists(bblayerconf):
             raise BuildSetupException("BE is not consistent: bblayers.conf file missing at %s" % bblayerconf)
 
+        # 6. create custom layer and add custom recipes to it
+        layerpath = os.path.join(self.be.sourcedir, "_meta-toaster-custom")
+        if os.path.isdir(layerpath):
+            shutil.rmtree(layerpath) # remove leftovers from previous builds
+        for target in targets:
+            try:
+                customrecipe = CustomImageRecipe.objects.get(name=target.target,
+                                                             project=bitbakes[0].req.project)
+            except CustomImageRecipe.DoesNotExist:
+                continue # not a custom recipe, skip
+
+            # create directory structure
+            for name in ("conf", "recipes"):
+                path = os.path.join(layerpath, name)
+                if not os.path.isdir(path):
+                    os.makedirs(path)
+
+            # create layer.oonf
+            config = os.path.join(layerpath, "conf", "layer.conf")
+            if not os.path.isfile(config):
+                with open(config, "w") as conf:
+                    conf.write('BBPATH .= ":${LAYERDIR}"\nBBFILES += "${LAYERDIR}/recipes/*.bb"\n')
+
+            # create recipe
+            recipe = os.path.join(layerpath, "recipes", "%s.bb" % target.target)
+            with open(recipe, "w") as recipef:
+                recipef.write("require %s\n" % customrecipe.base_recipe.recipe.file_path)
+                packages = [pkg.name for pkg in customrecipe.packages.all()]
+                if packages:
+                    recipef.write('IMAGE_INSTALL = "%s"\n' % ' '.join(packages))
+
+            # create *Layer* objects needed for build machinery to work
+            layer = Layer.objects.get_or_create(name="Toaster Custom layer",
+                                                summary="Layer for custom recipes",
+                                                vcs_url="file://%s" % layerpath)[0]
+            breq = target.req
+            lver = Layer_Version.objects.get_or_create(project=breq.project, layer=layer,
+                                                       dirpath=layerpath, build=breq.build)[0]
+            ProjectLayer.objects.get_or_create(project=breq.project, layercommit=lver,
+                                               optional=False)
+            BRLayer.objects.get_or_create(req=breq, name=layer.name, dirpath=layerpath,
+                                          giturl="file://%s" % layerpath)
+        if os.path.isdir(layerpath):
+            layerlist.append(layerpath)
+
         BuildEnvironmentController._updateBBLayers(bblayerconf, layerlist)
 
         self.islayerset = True
@@ -316,7 +363,7 @@ class LocalhostBEController(BuildEnvironmentController):
 
     def triggerBuild(self, bitbake, layers, variables, targets):
         # set up the buid environment with the needed layers
-        self.setLayers(bitbake, layers)
+        self.setLayers(bitbake, layers, targets)
         self.writeConfFile("conf/toaster-pre.conf", variables)
         self.writeConfFile("conf/toaster.conf", raw = "INHERIT+=\"toaster buildhistory\"")
 
Signature · 1 line
1.9.1
#4 lore raw

PATCH 4/22 toaster: widgets ToasterTable add logger to notify when cache hit

In reply to brian avery

From: Michael Wood <[email protected]>

Useful for debugging to remind you if you're hitting a cache or not

---
 lib/toaster/toastergui/widgets.py | 4 ++++
 1 file changed, 4 insertions(+)
lib/toaster/toastergui/widgets.py +4 -0
index eb49692..7f96370 100644
@@ -38,6 +38,9 @@ import collections
 import operator
 import re
 
+import logging
+logger = logging.getLogger("toaster")
+
 from toastergui.views import objtojson
 
 class ToasterTable(TemplateView):
@@ -264,6 +267,7 @@ class ToasterTable(TemplateView):
         data = cache.get(cache_name)
 
         if data:
+            logger.debug("Got cache data for table '%s'" % self.title)
             return data
 
         self.setup_columns(**kwargs)
Signature · 1 line
1.9.1
#5 lore raw

PATCH 5/22 toaster: widgets ToasterTable Add more info to search field exception

In reply to brian avery

From: Michael Wood <[email protected]>

If we get a search fields exception then also print out the model name

---
 lib/toaster/toastergui/widgets.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
lib/toaster/toastergui/widgets.py +2 -1
index 7f96370..aaa59c0 100644
@@ -229,7 +229,8 @@ class ToasterTable(TemplateView):
         """Creates a query based on the model's search_allowed_fields"""
 
         if not hasattr(self.queryset.model, 'search_allowed_fields'):
-            raise Exception("Err Search fields aren't defined in the model")
+            raise Exception("Search fields aren't defined in the model %s"
+                           % self.queryset.model)
 
         search_queries = []
         for st in search_term.split(" "):
Signature · 1 line
1.9.1
#6 lore raw

PATCH 6/22 toaster: add nocache option to the ToasterTable widget

In reply to brian avery

From: Michael Wood <[email protected]>

Useful for skipping the cache mechanism when debugging. Simply append nocache=true to the end of the url with the table in it.
Also adds a debug message if the table is using cached data.

---
 lib/toaster/toastergui/widgets.py | 7 +++++++
 1 file changed, 7 insertions(+)
lib/toaster/toastergui/widgets.py +7 -0
index aaa59c0..d7d5efd 100644
@@ -253,11 +253,14 @@ class ToasterTable(TemplateView):
         search = request.GET.get("search", None)
         filters = request.GET.get("filter", None)
         orderby = request.GET.get("orderby", None)
+        nocache = request.GET.get("nocache", None)
 
         # Make a unique cache name
         cache_name = self.__class__.__name__
 
         for key, val in request.GET.iteritems():
+            if key == 'nocache':
+                continue
             cache_name = cache_name + str(key) + str(val)
 
         for key, val in kwargs.iteritems():
@@ -265,6 +268,10 @@ class ToasterTable(TemplateView):
 
         # No special chars allowed in the cache name apart from dash
         cache_name = re.sub(r'[^A-Za-z0-9-]', "", cache_name)
+
+        if nocache:
+            cache.delete(cache_name)
+
         data = cache.get(cache_name)
 
         if data:
Signature · 1 line
1.9.1
#7 lore raw

PATCH 7/22 toaster: ToasterTable remove unused class definition

In reply to brian avery

From: Michael Wood <[email protected]>

Remove ToasterTemplateView as this isn't used by anything.

---
 lib/toaster/toastergui/widgets.py | 27 ---------------------------
 1 file changed, 27 deletions(-)
lib/toaster/toastergui/widgets.py +0 -27
index d7d5efd..e58dd7c 100644
@@ -349,33 +349,6 @@ class ToasterTable(TemplateView):
         return data
 
 
-class ToasterTemplateView(TemplateView):
-    # renders a instance in a template, or returns the context as json
-    # the class-equivalent of the _template_renderer decorator for views
-
-    def __init__(self, *args, **kwargs):
-        super(ToasterTemplateView, self).__init__(*args, **kwargs)
-        self.context_entries = []
-
-    def get(self, *args, **kwargs):
-        if self.request.GET.get('format', None) == 'json':
-            from django.core.urlresolvers import reverse
-            from django.shortcuts import HttpResponse
-            from views import objtojson
-            from toastergui.templatetags.projecttags import json as jsonfilter
-
-            context = self.get_context_data(**kwargs)
-
-            for x in context.keys():
-                if x not in self.context_entries:
-                    del context[x]
-
-            context["error"] = "ok"
-
-            return HttpResponse(jsonfilter(context,  default=objtojson ),
-                            content_type = "application/json; charset=utf-8")
-
-        return super(ToasterTemplateView, self).get(*args, **kwargs)
 
 class ToasterTypeAhead(View):
     """ A typeahead mechanism to support the front end typeahead widgets """
Signature · 1 line
1.9.1
#8 lore raw

PATCH 8/22 toaster: Add CustomImageRecipe model

In reply to brian avery

From: Michael Wood <[email protected]>

This model lists custom image recipes for the project. It is populated when new custom image is created. It holds reference to the base recipe and list of packages included into custom image.

For CustomImageRecipes the packages will be copied in and therefore not associated with a build so Remove the requirement for the package to have a Build.

co-author: Ed Bartosh <[email protected]>

---
 ...e__add_unique_customimagerecipe_name_project.py | 375 +++++++++++++++++++++
 lib/toaster/orm/models.py                          |  11 +-
 2 files changed, 385 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py
lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py new file +375 -0
new file mode 100644
index 0000000..6030605
@@ -0,0 +1,375 @@
+# -*- coding: utf-8 -*-
+from south.utils import datetime_utils as datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        # Adding model 'CustomImageRecipe'
+        db.create_table(u'orm_customimagerecipe', (
+            (u'id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('name', self.gf('django.db.models.fields.CharField')(max_length=100)),
+            ('base_recipe', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['orm.Recipe'])),
+            ('project', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['orm.Project'])),
+        ))
+        db.send_create_signal(u'orm', ['CustomImageRecipe'])
+
+        # Adding M2M table for field packages on 'CustomImageRecipe'
+        m2m_table_name = db.shorten_name(u'orm_customimagerecipe_packages')
+        db.create_table(m2m_table_name, (
+            ('id', models.AutoField(verbose_name='ID', primary_key=True, auto_created=True)),
+            ('customimagerecipe', models.ForeignKey(orm[u'orm.customimagerecipe'], null=False)),
+            ('package', models.ForeignKey(orm[u'orm.package'], null=False))
+        ))
+        db.create_unique(m2m_table_name, ['customimagerecipe_id', 'package_id'])
+
+        # Adding unique constraint on 'CustomImageRecipe', fields ['name', 'project']
+        db.create_unique(u'orm_customimagerecipe', ['name', 'project_id'])
+
+
+        # Changing field 'Package.build'
+        db.alter_column(u'orm_package', 'build_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['orm.Build'], null=True))
+
+    def backwards(self, orm):
+        # Removing unique constraint on 'CustomImageRecipe', fields ['name', 'project']
+        db.delete_unique(u'orm_customimagerecipe', ['name', 'project_id'])
+
+        # Deleting model 'CustomImageRecipe'
+        db.delete_table(u'orm_customimagerecipe')
+
+        # Removing M2M table for field packages on 'CustomImageRecipe'
+        db.delete_table(db.shorten_name(u'orm_customimagerecipe_packages'))
+
+
+        # Changing field 'Package.build'
+        db.alter_column(u'orm_package', 'build_id', self.gf('django.db.models.fields.related.ForeignKey')(default=None, to=orm['orm.Build']))
+
+    models = {
+        u'orm.bitbakeversion': {
+            'Meta': {'object_name': 'BitbakeVersion'},
+            'branch': ('django.db.models.fields.CharField', [], {'max_length': '32'}),
+            'dirpath': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'giturl': ('django.db.models.fields.URLField', [], {'max_length': '200'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '32'})
+        },
+        u'orm.branch': {
+            'Meta': {'unique_together': "(('layer_source', 'name'), ('layer_source', 'up_id'))", 'object_name': 'Branch'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'True', 'to': u"orm['orm.LayerSource']", 'null': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '50'}),
+            'short_description': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'}),
+            'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}),
+            'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'})
+        },
+        u'orm.build': {
+            'Meta': {'object_name': 'Build'},
+            'bitbake_version': ('django.db.models.fields.CharField', [], {'max_length': '50'}),
+            'build_name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'completed_on': ('django.db.models.fields.DateTimeField', [], {}),
+            'cooker_log_path': ('django.db.models.fields.CharField', [], {'max_length': '500'}),
+            'distro': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'distro_version': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'machine': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'outcome': ('django.db.models.fields.IntegerField', [], {'default': '2'}),
+            'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}),
+            'started_on': ('django.db.models.fields.DateTimeField', [], {})
+        },
+        u'orm.buildartifact': {
+            'Meta': {'object_name': 'BuildArtifact'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']"}),
+            'file_name': ('django.db.models.fields.FilePathField', [], {'max_length': '100'}),
+            'file_size': ('django.db.models.fields.IntegerField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'})
+        },
+        u'orm.customimagerecipe': {
+            'Meta': {'unique_together': "(('name', 'project'),)", 'object_name': 'CustomImageRecipe'},
+            'base_recipe': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Recipe']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'packages': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['orm.Package']", 'symmetrical': 'False'}),
+            'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"})
+        },
+        u'orm.helptext': {
+            'Meta': {'object_name': 'HelpText'},
+            'area': ('django.db.models.fields.IntegerField', [], {}),
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'helptext_build'", 'to': u"orm['orm.Build']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'key': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'text': ('django.db.models.fields.TextField', [], {})
+        },
+        u'orm.layer': {
+            'Meta': {'unique_together': "(('layer_source', 'up_id'), ('layer_source', 'name'))", 'object_name': 'Layer'},
+            'description': ('django.db.models.fields.TextField', [], {'default': 'None', 'null': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer_index_url': ('django.db.models.fields.URLField', [], {'max_length': '200'}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'summary': ('django.db.models.fields.TextField', [], {'default': 'None', 'null': 'True'}),
+            'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}),
+            'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}),
+            'vcs_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}),
+            'vcs_web_file_base_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}),
+            'vcs_web_tree_base_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}),
+            'vcs_web_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'})
+        },
+        u'orm.layer_version': {
+            'Meta': {'unique_together': "(('layer_source', 'up_id'),)", 'object_name': 'Layer_Version'},
+            'branch': ('django.db.models.fields.CharField', [], {'max_length': '80'}),
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'related_name': "'layer_version_build'", 'null': 'True', 'to': u"orm['orm.Build']"}),
+            'commit': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'dirpath': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '255', 'null': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'layer_version_layer'", 'to': u"orm['orm.Layer']"}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}),
+            'local_path': ('django.db.models.fields.FilePathField', [], {'default': "'/'", 'max_length': '1024'}),
+            'priority': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'project': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.Project']", 'null': 'True'}),
+            'up_branch': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.Branch']", 'null': 'True'}),
+            'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}),
+            'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'})
+        },
+        u'orm.layersource': {
+            'Meta': {'unique_together': "(('sourcetype', 'apiurl'),)", 'object_name': 'LayerSource'},
+            'apiurl': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '255', 'null': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '63'}),
+            'sourcetype': ('django.db.models.fields.IntegerField', [], {})
+        },
+        u'orm.layerversiondependency': {
+            'Meta': {'unique_together': "(('layer_source', 'up_id'),)", 'object_name': 'LayerVersionDependency'},
+            'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'dependees'", 'to': u"orm['orm.Layer_Version']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}),
+            'layer_version': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'dependencies'", 'to': u"orm['orm.Layer_Version']"}),
+            'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'})
+        },
+        u'orm.logmessage': {
+            'Meta': {'object_name': 'LogMessage'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'level': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'lineno': ('django.db.models.fields.IntegerField', [], {'null': 'True'}),
+            'message': ('django.db.models.fields.CharField', [], {'max_length': '240'}),
+            'pathname': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}),
+            'task': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Task']", 'null': 'True', 'blank': 'True'})
+        },
+        u'orm.machine': {
+            'Meta': {'unique_together': "(('layer_source', 'up_id'),)", 'object_name': 'Machine'},
+            'description': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}),
+            'layer_version': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Layer_Version']"}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}),
+            'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'})
+        },
+        u'orm.package': {
+            'Meta': {'object_name': 'Package'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']", 'null': 'True'}),
+            'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'installed_name': ('django.db.models.fields.CharField', [], {'default': "''", 'max_length': '100'}),
+            'installed_size': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'license': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'recipe': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Recipe']", 'null': 'True'}),
+            'revision': ('django.db.models.fields.CharField', [], {'max_length': '32', 'blank': 'True'}),
+            'section': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}),
+            'size': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'summary': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            'version': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'})
+        },
+        u'orm.package_dependency': {
+            'Meta': {'object_name': 'Package_Dependency'},
+            'dep_type': ('django.db.models.fields.IntegerField', [], {}),
+            'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'package_dependencies_target'", 'to': u"orm['orm.Package']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'package': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'package_dependencies_source'", 'to': u"orm['orm.Package']"}),
+            'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']", 'null': 'True'})
+        },
+        u'orm.package_file': {
+            'Meta': {'object_name': 'Package_File'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'package': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'buildfilelist_package'", 'to': u"orm['orm.Package']"}),
+            'path': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}),
+            'size': ('django.db.models.fields.IntegerField', [], {})
+        },
+        u'orm.project': {
+            'Meta': {'object_name': 'Project'},
+            'bitbake_version': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.BitbakeVersion']", 'null': 'True'}),
+            'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_default': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'release': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Release']", 'null': 'True'}),
+            'short_description': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'}),
+            'updated': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}),
+            'user_id': ('django.db.models.fields.IntegerField', [], {'null': 'True'})
+        },
+        u'orm.projectlayer': {
+            'Meta': {'unique_together': "(('project', 'layercommit'),)", 'object_name': 'ProjectLayer'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layercommit': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Layer_Version']", 'null': 'True'}),
+            'optional': ('django.db.models.fields.BooleanField', [], {'default': 'True'}),
+            'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"})
+        },
+        u'orm.projecttarget': {
+            'Meta': {'object_name': 'ProjectTarget'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}),
+            'target': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'task': ('django.db.models.fields.CharField', [], {'max_length': '100', 'null': 'True'})
+        },
+        u'orm.projectvariable': {
+            'Meta': {'object_name': 'ProjectVariable'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}),
+            'value': ('django.db.models.fields.TextField', [], {'blank': 'True'})
+        },
+        u'orm.recipe': {
+            'Meta': {'unique_together': "(('layer_version', 'file_path', 'pathflags'),)", 'object_name': 'Recipe'},
+            'bugtracker': ('django.db.models.fields.URLField', [], {'max_length': '200', 'blank': 'True'}),
+            'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            'file_path': ('django.db.models.fields.FilePathField', [], {'max_length': '255'}),
+            'homepage': ('django.db.models.fields.URLField', [], {'max_length': '200', 'blank': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_image': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}),
+            'layer_version': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'recipe_layer_version'", 'to': u"orm['orm.Layer_Version']"}),
+            'license': ('django.db.models.fields.CharField', [], {'max_length': '200', 'blank': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}),
+            'pathflags': ('django.db.models.fields.CharField', [], {'max_length': '200', 'blank': 'True'}),
+            'section': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}),
+            'summary': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}),
+            'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}),
+            'version': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'})
+        },
+        u'orm.recipe_dependency': {
+            'Meta': {'object_name': 'Recipe_Dependency'},
+            'dep_type': ('django.db.models.fields.IntegerField', [], {}),
+            'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'r_dependencies_depends'", 'to': u"orm['orm.Recipe']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'recipe': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'r_dependencies_recipe'", 'to': u"orm['orm.Recipe']"})
+        },
+        u'orm.release': {
+            'Meta': {'object_name': 'Release'},
+            'bitbake_version': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.BitbakeVersion']"}),
+            'branch_name': ('django.db.models.fields.CharField', [], {'default': "''", 'max_length': '50'}),
+            'description': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'helptext': ('django.db.models.fields.TextField', [], {'null': 'True'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '32'})
+        },
+        u'orm.releasedefaultlayer': {
+            'Meta': {'object_name': 'ReleaseDefaultLayer'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer_name': ('django.db.models.fields.CharField', [], {'default': "''", 'max_length': '100'}),
+            'release': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Release']"})
+        },
+        u'orm.releaselayersourcepriority': {
+            'Meta': {'unique_together': "(('release', 'layer_source'),)", 'object_name': 'ReleaseLayerSourcePriority'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.LayerSource']"}),
+            'priority': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'release': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Release']"})
+        },
+        u'orm.target': {
+            'Meta': {'object_name': 'Target'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'image_size': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'is_image': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'license_manifest_path': ('django.db.models.fields.CharField', [], {'max_length': '500', 'null': 'True'}),
+            'target': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'task': ('django.db.models.fields.CharField', [], {'max_length': '100', 'null': 'True'})
+        },
+        u'orm.target_file': {
+            'Meta': {'object_name': 'Target_File'},
+            'directory': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'directory_set'", 'null': 'True', 'to': u"orm['orm.Target_File']"}),
+            'group': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'inodetype': ('django.db.models.fields.IntegerField', [], {}),
+            'owner': ('django.db.models.fields.CharField', [], {'max_length': '128'}),
+            'path': ('django.db.models.fields.FilePathField', [], {'max_length': '100'}),
+            'permission': ('django.db.models.fields.CharField', [], {'max_length': '16'}),
+            'size': ('django.db.models.fields.IntegerField', [], {}),
+            'sym_target': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'symlink_set'", 'null': 'True', 'to': u"orm['orm.Target_File']"}),
+            'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']"})
+        },
+        u'orm.target_image_file': {
+            'Meta': {'object_name': 'Target_Image_File'},
+            'file_name': ('django.db.models.fields.FilePathField', [], {'max_length': '254'}),
+            'file_size': ('django.db.models.fields.IntegerField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']"})
+        },
+        u'orm.target_installed_package': {
+            'Meta': {'object_name': 'Target_Installed_Package'},
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'package': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'buildtargetlist_package'", 'to': u"orm['orm.Package']"}),
+            'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']"})
+        },
+        u'orm.task': {
+            'Meta': {'ordering': "('order', 'recipe')", 'unique_together': "(('build', 'recipe', 'task_name'),)", 'object_name': 'Task'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'task_build'", 'to': u"orm['orm.Build']"}),
+            'cpu_usage': ('django.db.models.fields.DecimalField', [], {'null': 'True', 'max_digits': '8', 'decimal_places': '2'}),
+            'disk_io': ('django.db.models.fields.IntegerField', [], {'null': 'True'}),
+            'elapsed_time': ('django.db.models.fields.DecimalField', [], {'null': 'True', 'max_digits': '8', 'decimal_places': '2'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'line_number': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'logfile': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}),
+            'message': ('django.db.models.fields.CharField', [], {'max_length': '240'}),
+            'order': ('django.db.models.fields.IntegerField', [], {'null': 'True'}),
+            'outcome': ('django.db.models.fields.IntegerField', [], {'default': '-1'}),
+            'path_to_sstate_obj': ('django.db.models.fields.FilePathField', [], {'max_length': '500', 'blank': 'True'}),
+            'recipe': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'tasks'", 'to': u"orm['orm.Recipe']"}),
+            'script_type': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'source_url': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}),
+            'sstate_checksum': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}),
+            'sstate_result': ('django.db.models.fields.IntegerField', [], {'default': '0'}),
+            'task_executed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'task_name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'work_directory': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'})
+        },
+        u'orm.task_dependency': {
+            'Meta': {'object_name': 'Task_Dependency'},
+            'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'task_dependencies_depends'", 'to': u"orm['orm.Task']"}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'task': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'task_dependencies_task'", 'to': u"orm['orm.Task']"})
+        },
+        u'orm.toastersetting': {
+            'Meta': {'object_name': 'ToasterSetting'},
+            'helptext': ('django.db.models.fields.TextField', [], {}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'max_length': '63'}),
+            'value': ('django.db.models.fields.CharField', [], {'max_length': '255'})
+        },
+        u'orm.variable': {
+            'Meta': {'object_name': 'Variable'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'variable_build'", 'to': u"orm['orm.Build']"}),
+            'changed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            'human_readable_name': ('django.db.models.fields.CharField', [], {'max_length': '200'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'variable_name': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'variable_value': ('django.db.models.fields.TextField', [], {'blank': 'True'})
+        },
+        u'orm.variablehistory': {
+            'Meta': {'object_name': 'VariableHistory'},
+            'file_name': ('django.db.models.fields.FilePathField', [], {'max_length': '255'}),
+            u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'line_number': ('django.db.models.fields.IntegerField', [], {'null': 'True'}),
+            'operation': ('django.db.models.fields.CharField', [], {'max_length': '64'}),
+            'value': ('django.db.models.fields.TextField', [], {'blank': 'True'}),
+            'variable': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'vhistory'", 'to': u"orm['orm.Variable']"})
+        }
+    }
+
+    complete_apps = ['orm']
\ No newline at end of file
lib/toaster/orm/models.py +10 -1
index b5c8643..9790630 100644
@@ -551,7 +551,7 @@ class Task_Dependency(models.Model):
 
 class Package(models.Model):
     search_allowed_fields = ['name', 'version', 'revision', 'recipe__name', 'recipe__version', 'recipe__license', 'recipe__layer_version__layer__name', 'recipe__layer_version__branch', 'recipe__layer_version__commit', 'recipe__layer_version__local_path', 'installed_name']
-    build = models.ForeignKey('Build')
+    build = models.ForeignKey('Build', null=True)
     recipe = models.ForeignKey('Recipe', null=True)
     name = models.CharField(max_length=100)
     installed_name = models.CharField(max_length=100, default='')
@@ -1172,6 +1172,15 @@ class ProjectLayer(models.Model):
     class Meta:
         unique_together = (("project", "layercommit"),)
 
+class CustomImageRecipe(models.Model):
+    name = models.CharField(max_length=100)
+    base_recipe = models.ForeignKey(Recipe)
+    packages = models.ManyToManyField(Package)
+    project = models.ForeignKey(Project)
+
+    class Meta:
+        unique_together = ("name", "project")
+
 class ProjectVariable(models.Model):
     project = models.ForeignKey(Project)
     name = models.CharField(max_length=100)
Signature · 1 line
1.9.1
#9 lore raw

PATCH 9/22 toaster: add toggle for enabling image customisation feeature

In reply to brian avery

From: Michael Wood <[email protected]>

This feature is currently under heavy development and should be used with caution.

---
 lib/toaster/toastergui/views.py     | 1 +
 lib/toaster/toastermain/settings.py | 5 +++++
 2 files changed, 6 insertions(+)
lib/toaster/toastergui/views.py +1 -0
index 8689a12..2e3b822 100755
@@ -1867,6 +1867,7 @@ def managedcontextprocessor(request):
     ret = {
         "projects": Project.objects.all(),
         "DEBUG" : toastermain.settings.DEBUG,
+        "CUSTOM_IMAGE" : toastermain.settings.CUSTOM_IMAGE,
         "TOASTER_BRANCH": toastermain.settings.TOASTER_BRANCH,
         "TOASTER_REVISION" : toastermain.settings.TOASTER_REVISION,
     }
lib/toaster/toastermain/settings.py +5 -0
index b149a5e..6439d68 100644
@@ -23,6 +23,11 @@
 
 import os, re
 
+# Temporary toggle for Image customisation
+CUSTOM_IMAGE = False
+if os.environ.get("CUSTOM_IMAGE", None) is not None:
+    CUSTOM_IMAGE = True
+
 DEBUG = True
 TEMPLATE_DEBUG = DEBUG
 
Signature · 1 line
1.9.1
#10 lore raw

PATCH 10/22 toaster: implement decorator for REST responses

In reply to brian avery

From: Ed Bartosh <[email protected]>

Implemented xhr_response decorator to decorate responses from REST methods into Django HttpResponse objects.

This decorator should reduce amount of repeated code in REST methods and make them more readable.

---
 lib/toaster/toastergui/views.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)
lib/toaster/toastergui/views.py +13 -0
index 2e3b822..95df60e 100755
@@ -45,6 +45,7 @@ from django.utils import formats
 from toastergui.templatetags.projecttags import json as jsonfilter
 import json
 from os.path import dirname
+from functools import wraps
 import itertools
 
 import magic
@@ -2314,6 +2315,18 @@ if True:
 
         return context
 
+    def xhr_response(fun):
+        """
+        Decorator for REST methods.
+        calls jsonfilter on the returned dictionary and returns result
+        as HttpResponse object of content_type application/json
+        """
+        @wraps(fun)
+        def wrapper(*args, **kwds):
+            return HttpResponse(jsonfilter(fun(*args, **kwds)),
+                                content_type="application/json")
+        return wrapper
+
     def jsunittests(request):
       """ Provides a page for the js unit tests """
       bbv = BitbakeVersion.objects.filter(branch="master").first()
Signature · 1 line
1.9.1
#11 lore raw

PATCH 11/22 toaster: Fix indentation of jsunittests view

In reply to brian avery

From: Michael Wood <[email protected]>

Fix indentation to 4 spaces

---
 lib/toaster/toastergui/views.py | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
lib/toaster/toastergui/views.py +10 -10
index 95df60e..5ea6122 100755
@@ -2328,20 +2328,20 @@ if True:
         return wrapper
 
     def jsunittests(request):
-      """ Provides a page for the js unit tests """
-      bbv = BitbakeVersion.objects.filter(branch="master").first()
-      release = Release.objects.filter(bitbake_version=bbv).first()
+        """ Provides a page for the js unit tests """
+        bbv = BitbakeVersion.objects.filter(branch="master").first()
+        release = Release.objects.filter(bitbake_version=bbv).first()
 
-      name = "_js_unit_test_prj_"
+        name = "_js_unit_test_prj_"
 
-      # If there is an existing project by this name delete it. We don't want
-      # Lots of duplicates cluttering up the projects.
-      Project.objects.filter(name=name).delete()
+        # If there is an existing project by this name delete it. We don't want
+        # Lots of duplicates cluttering up the projects.
+        Project.objects.filter(name=name).delete()
 
-      new_project = Project.objects.create_project(name=name, release=release)
+        new_project = Project.objects.create_project(name=name, release=release)
 
-      context = { 'project' : new_project }
-      return render(request, "js-unit-tests.html", context)
+        context = { 'project' : new_project }
+        return render(request, "js-unit-tests.html", context)
 
     from django.views.decorators.csrf import csrf_exempt
     @csrf_exempt
Signature · 1 line
1.9.1
#12 lore raw

PATCH 12/22 toaster: Add new ReST API for Image Customisation feature

In reply to brian avery

From: Ed Bartosh <[email protected]>

Implemented xhr_customrecipe API. To create a custom recipe from a base recipe.
Implemented xhr_customrecipe_packages API to add/remove packages to/from custom recipe.

co-authored see Signed-off-by

---
 lib/toaster/toastergui/urls.py  |  10 ++-
 lib/toaster/toastergui/views.py | 152 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 159 insertions(+), 3 deletions(-)
lib/toaster/toastergui/urls.py +9 -1
index 55f325d..b47a161 100644
@@ -152,6 +152,14 @@ urlpatterns = patterns('toastergui.views',
         # JS Unit tests
         url(r'^js-unit-tests/$', 'jsunittests', name='js-unit-tests'),
 
-        # default redirection
+        # image customisation functionality
+        url(r'^xhr_customrecipe/(?P<recipe_id>\d+)/packages/(?P<package_id>\d+|)$',
+            'xhr_customrecipe_packages', name='xhr_customrecipe_packages'),
+        url(r'^xhr_customrecipe/(?P<recipe_id>\d+)$', 'xhr_customrecipe_id',
+            name='xhr_customrecipe_id'),
+        url(r'^xhr_customrecipe/', 'xhr_customrecipe',
+            name='xhr_customrecipe'),
+
+          # default redirection
         url(r'^$', RedirectView.as_view( url= 'landing')),
 )
lib/toaster/toastergui/views.py +150 -2

169 lines not on this page · raw message

Signature · 1 line
1.9.1
#13 lore raw

PATCH 13/22 toaster: Add ToasterTables for Image customisation feature

In reply to brian avery

From: Michael Wood <[email protected]>

- Split up the recipes tables into Image recipes and Software Recipes
- Add CustomImageRecipe table
- Add SelectPackagesTable table
- Add NewCustomImagesTable table

---
 lib/toaster/toastergui/tables.py | 222 +++++++++++++++++++++++++++++++++------
 1 file changed, 188 insertions(+), 34 deletions(-)
lib/toaster/toastergui/tables.py +188 -34
index 70e4b6d..1526d59 100644
@@ -21,6 +21,7 @@
 
 from toastergui.widgets import ToasterTable
 from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project
+from orm.models import CustomImageRecipe, Package
 from django.db.models import Q, Max
 from django.conf.urls import url
 from django.core.urlresolvers import reverse
@@ -310,13 +311,20 @@ class LayerMachinesTable(MachinesTable):
 
 
 class RecipesTable(ToasterTable, ProjectFiltersMixin):
-    """Table of Recipes in Toaster"""
+    """Table of All Recipes in Toaster"""
 
     def __init__(self, *args, **kwargs):
         super(RecipesTable, self).__init__(*args, **kwargs)
         self.empty_state = "Toaster has no recipe information. To generate recipe information you can configure a layer source then run a build."
         self.default_orderby = "name"
 
+    build_col = { 'title' : "Build",
+            'help_text' : "Add or delete recipes to and from your project",
+            'hideable' : False,
+            'filter_name' : "in_current_project",
+            'static_data_name' : "add-del-layers",
+            'static_data_template' : '{% include "recipe_btn.html" %}'}
+
     def get_context_data(self, **kwargs):
         project = Project.objects.get(pk=kwargs['pid'])
         context = super(RecipesTable, self).get_context_data(**kwargs)
@@ -327,17 +335,6 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin):
 
         return context
 
-    def setup_filters(self, *args, **kwargs):
-        project = Project.objects.get(pk=kwargs['pid'])
-        self.project_layers = project.projectlayer_equivalent_set()
-
-        self.add_filter(title="Filter by project recipes",
-                        name="in_current_project",
-                        filter_actions=[
-                            self.make_filter_action("in_project", "Recipes provided by layers added to this project", self.filter_in_project),
-                            self.make_filter_action("not_in_project", "Recipes provided by layers not added to this project", self.filter_not_in_project)
-                        ])
-
 
     def setup_queryset(self, *args, **kwargs):
         prj = Project.objects.get(pk = kwargs['pid'])
@@ -348,12 +345,6 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin):
 
     def setup_columns(self, *args, **kwargs):
 
-        self.add_column(title="Recipe",
-                        help_text="Information about a single piece of software, including where to download the source, configuration options, how to compile the source files and how to package the compiled output",
-                        hideable=False,
-                        orderable=True,
-                        field_name="name")
-
         self.add_column(title="Recipe Version",
                         hidden=True,
                         field_name="version")
@@ -398,18 +389,6 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin):
         self.add_column(title="Revision",
                         field_name="layer_version__get_vcs_reference")
 
-        self.add_column(title="Build",
-                        help_text="Add or delete recipes to and from your project",
-                        hideable=False,
-                        filter_name="in_current_project",
-                        static_data_name="add-del-layers",
-                        static_data_template='{% include "recipe_btn.html" %}')
-
-        project = Project.objects.get(pk=kwargs['pid'])
-        self.add_column(title="Project compatible Layer ID",
-                        displayable = False,
-                        field_name = "projectcompatible_layer",
-                        computation = lambda x: (x.layer_version.get_equivalents_wpriority(project)[0]))
 
 class LayerRecipesTable(RecipesTable):
     """ Smaller version of the Recipes table for use in layer details """
@@ -446,10 +425,185 @@ class LayerRecipesTable(RecipesTable):
                         static_data_name="add-del-layers",
                         static_data_template=build_recipe_template)
 
-class ProjectLayersRecipesTable(RecipesTable):
-    """ Table that lists only recipes available for layers added to the project """
+class CustomImagesTable(ToasterTable):
+    """ Table to display your custom images """
+    def __init__(self, *args, **kwargs):
+        super(CustomImagesTable, self).__init__(*args, **kwargs)
+        self.title = "Custom images"
+
+    def get_context_data(self, **kwargs):
+        context = super(CustomImagesTable, self).get_context_data(**kwargs)
+        project = Project.objects.get(pk=kwargs['pid'])
+        context['project'] = project
+        context['projectlayers'] = map(lambda prjlayer: prjlayer.layercommit.id, ProjectLayer.objects.filter(project=context['project']))
+        return context
+
+    def setup_queryset(self, *args, **kwargs):
+        prj = Project.objects.get(pk = kwargs['pid'])
+        self.queryset = CustomImageRecipe.objects.filter(project=prj)
+        self.queryset = self.queryset.order_by('name')
+
+    def setup_columns(self, *args, **kwargs):
+
+        name_link_template = '''
+        <a href="{% url 'customrecipe' extra.pid data.id %}">
+          {{data.name}}
+        </a>
+        '''
+
+        self.add_column(title="Custom image",
+                        hideable=False,
+                        static_data_name="name",
+                        static_data_template=name_link_template)
+
+        self.add_column(title="Recipe file",
+                        static_data_name='recipe_file',
+                        static_data_template='')
+
+        approx_packages_template = '<a href="#imagedetails">{{data.packages.all|length}}</a>'
+        self.add_column(title="Approx packages",
+                        static_data_name='approx_packages',
+                        static_data_template=approx_packages_template)
+
+
+        build_btn_template = '''<button data-recipe-name="{{data.name}}"
+        class="btn btn-block build-recipe-btn" style="margin-top: 5px;" >
+        Build</button>'''
+
+        self.add_column(title="Build",
+                        hideable=False,
+                        static_data_name='build_custom_img',
+                        static_data_template=build_btn_template)
+
+class ImageRecipesTable(RecipesTable):
+    """ A subset of the recipes table which displayed just image recipes """
+
+    def __init__(self, *args, **kwargs):
+        super(ImageRecipesTable, self).__init__(*args, **kwargs)
+        self.title = "Compatible image recipes"
+
+    def setup_queryset(self, *args, **kwargs):
+        super(ImageRecipesTable, self).setup_queryset(*args, **kwargs)
+
+        self.queryset = self.queryset.filter(is_image=True)
+
+
+    def setup_columns(self, *args, **kwargs):
+        self.add_column(title="Image recipe",
+                        help_text="When you build an image recipe, you get an "
+                                  "image: a root file system you can"
+                                  "deploy to a machine",
+                        hideable=False,
+                        orderable=True,
+                        field_name="name")
+
+        super(ImageRecipesTable, self).setup_columns(*args, **kwargs)
+
+        self.add_column(**RecipesTable.build_col)
+
+
+class NewCustomImagesTable(ImageRecipesTable):
+    """ Table which displays Images recipes which can be customised """
+    def __init__(self, *args, **kwargs):
+        super(NewCustomImagesTable, self).__init__(*args, **kwargs)
+        self.title = "Select the image recipe you want to customise"
 
     def setup_queryset(self, *args, **kwargs):
-        super(ProjectLayersRecipesTable, self).setup_queryset(*args, **kwargs)
+        super(ImageRecipesTable, self).setup_queryset(*args, **kwargs)
+
+        self.queryset = self.queryset.filter(is_image=True)
+
+    def setup_columns(self, *args, **kwargs):
+        self.add_column(title="Image recipe",
+                        help_text="When you build an image recipe, you get an "
+                                  "image: a root file system you can"
+                                  "deploy to a machine",
+                        hideable=False,
+                        orderable=True,
+                        field_name="recipe__name")
+
+        super(ImageRecipesTable, self).setup_columns(*args, **kwargs)
+
+        self.add_column(title="Customise",
+                        hideable=False,
+                        filter_name="in_current_project",
+                        static_data_name="customise-or-add-recipe",
+                        static_data_template='{% include "customise_btn.html" %}')
+
+
+class SoftwareRecipesTable(RecipesTable):
+    """ Displays just the software recipes """
+    def __init__(self, *args, **kwargs):
+        super(SoftwareRecipesTable, self).__init__(*args, **kwargs)
+        self.title = "Compatible software recipes"
+
+    def setup_queryset(self, *args, **kwargs):
+        super(SoftwareRecipesTable, self).setup_queryset(*args, **kwargs)
+
+        self.queryset = self.queryset.filter(is_image=False)
+
+
+    def setup_columns(self, *args, **kwargs):
+        self.add_column(title="Software recipe",
+                        help_text="Information about a single piece of "
+                        "software, including where to download the source, "
+                        "configuration options, how to compile the source "
+                        "files and how to package the compiled output",
+                        hideable=False,
+                        orderable=True,
+                        field_name="name")
+
+        super(SoftwareRecipesTable, self).setup_columns(*args, **kwargs)
+
+        self.add_column(**RecipesTable.build_col)
+
+
+class SelectPackagesTable(ToasterTable):
+    """ Table to display the packages to add and remove from an image """
+
+    def __init__(self, *args, **kwargs):
+        super(SelectPackagesTable, self).__init__(*args, **kwargs)
+        self.title = "Add | Remove packages"
+
+    def setup_queryset(self, *args, **kwargs):
+        cust_recipe = CustomImageRecipe.objects.get(pk=kwargs['recipeid'])
         prj = Project.objects.get(pk = kwargs['pid'])
-        self.queryset = self.queryset.filter(layer_version__in = prj.projectlayer_equivalent_set())
+
+        current_packages = cust_recipe.packages.all()
+
+        # Get all the packages that are in the custom image
+        # Get all the packages built by builds in the current project
+        # but not those ones that are already in the custom image
+        self.queryset = Package.objects.filter(
+                            Q(pk__in=current_packages) |
+                            (Q(build__project=prj) &
+                            ~Q(name__in=current_packages.values_list('name'))))
+
+        self.queryset = self.queryset.order_by('name')
+
+        self.static_context_extra['recipe_id'] = kwargs['recipeid']
+        self.static_context_extra['current_packages'] = \
+                cust_recipe.packages.values_list('pk', flat=True)
+
+    def setup_columns(self, *args, **kwargs):
+        self.add_column(title="Package",
+                        hideable=False,
+                        orderable=True,
+                        field_name="name")
+
+        self.add_column(title="Package Version",
+                        field_name="version")
+
+        self.add_column(title="Approx Size",
+                        orderable=True,
+                        static_data_name="size",
+                        static_data_template="{% load projecttags %} \
+                        {{data.size|filtered_filesizeformat}}")
+        self.add_column(title="summary",
+                        field_name="summary")
+
+        self.add_column(title="Add | Remove",
+                        help_text="Use the add and remove buttons to modify "
+                        "the package content of you custom image",
+                        static_data_name="add_rm_pkg_btn",
+                        static_data_template='{% include "pkg_add_rm_btn.html" %}')
Signature · 1 line
1.9.1
#14 lore raw

PATCH 14/22 toaster: Add Image customisation frontend feature

In reply to brian avery

From: Michael Wood <[email protected]>

Add the Image customisation front end feature to Toaster. Caveat - This feature is currently in development and should not be enabled by default.

---
 lib/toaster/toastergui/static/js/customrecipe.js   |  50 ++++++++
 lib/toaster/toastergui/static/js/layerBtn.js       |  13 ++
 lib/toaster/toastergui/static/js/newcustomimage.js |  49 +++++++
 .../toastergui/templates/baseprojectpage.html      |   7 +-
 .../toastergui/templates/customise_btn.html        |   9 ++
 lib/toaster/toastergui/templates/customrecipe.html | 142 +++++++++++++++++++++
 .../toastergui/templates/newcustomimage.html       |  54 ++++++++
 .../toastergui/templates/pkg_add_rm_btn.html       |  16 +++
 lib/toaster/toastergui/templates/project.html      |   2 +-
 .../toastergui/templates/projecttopbar.html        |   9 +-
 lib/toaster/toastergui/urls.py                     |  17 ++-
 lib/toaster/toastergui/views.py                    |   9 ++
 12 files changed, 368 insertions(+), 9 deletions(-)
 create mode 100644 bitbake/lib/toaster/toastergui/static/js/customrecipe.js
 create mode 100644 bitbake/lib/toaster/toastergui/static/js/newcustomimage.js
 create mode 100644 bitbake/lib/toaster/toastergui/templates/customise_btn.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/customrecipe.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/newcustomimage.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/pkg_add_rm_btn.html
lib/toaster/toastergui/static/js/customrecipe.js new file +50 -0
new file mode 100644
index 0000000..4f6b304
@@ -0,0 +1,50 @@
+"use strict";
+
+function customRecipePageInit(ctx) {
+
+  var urlParams = libtoaster.parseUrlParams();
+
+  (function notificationRequest(){
+    if (urlParams.hasOwnProperty('notify') && urlParams.notify === 'new'){
+      $("#image-created-notification").show();
+    }
+  })();
+
+  $("#recipeselection").on('table-done', function(e, total, tableParams){
+    /* Table is done so now setup the click handler for the package buttons */
+    $(".add-rm-package-btn").click(function(e){
+      e.preventDefault();
+      addRemovePackage($(this), tableParams);
+    });
+  });
+
+  function addRemovePackage(pkgBtn, tableParams){
+    var pkgBtnData = pkgBtn.data();
+    var method;
+    var buttonToShow;
+
+    if (pkgBtnData.directive == 'add') {
+      method = 'PUT';
+      buttonToShow = '#package-rm-btn-' + pkgBtnData.package;
+    } else if (pkgBtnData.directive == 'remove') {
+      method = 'DELETE';
+      buttonToShow = '#package-add-btn-' + pkgBtnData.package;
+    } else {
+      throw("Unknown package directive: should be add or remove");
+    }
+
+    $.ajax({
+        type: method,
+        url: pkgBtnData.packageUrl,
+        headers: { 'X-CSRFToken' : $.cookie('csrftoken')},
+        success: function(data){
+          /* Invalidate the Add | Rm package table's current cache */
+          tableParams.nocache = true;
+          $.get(ctx.tableApiUrl, tableParams);
+          /* Swap the buttons around */
+          pkgBtn.hide();
+          $(buttonToShow).show();
+        }
+    });
+  }
+}
lib/toaster/toastergui/static/js/layerBtn.js +13 -0
index a0509f9..da0241c 100644
@@ -68,6 +68,19 @@ function layerBtnsInit(ctx) {
       });
   });
 
+
+  $(".customise-btn").unbind('click');
+  $(".customise-btn").click(function(e){
+    e.preventDefault();
+    var imgCustomModal = $("#new-custom-image-modal");
+
+    if (imgCustomModal.length == 0)
+      throw("Modal new-custom-image not found");
+
+    imgCustomModal.data('recipe', $(this).data('recipe'));
+    imgCustomModal.modal('show');
+  });
+
   /* Setup the initial state of the buttons */
 
   for (var i in ctx.projectLayers){
lib/toaster/toastergui/static/js/newcustomimage.js new file +49 -0

49 lines not on this page · raw message

lib/toaster/toastergui/templates/baseprojectpage.html +5 -2
index 668e0bf..88bf859 100644
@@ -23,8 +23,11 @@
     <ul class="nav nav-list well">
       <li><a class="nav-parent" href="{% url 'project' project.id %}">Configuration</a></li>
       <li class="nav-header">Compatible metadata</li>
-<!--  <li><a href="all-image-recipes.html">Image recipes</a></li> -->
-      <li><a href="{% url 'projecttargets' project.id %}">Recipes</a></li>
+      {% if CUSTOM_IMAGE %}
+      <li><a href="{% url 'projectcustomimages' project.id %}">Custom images</a></li>
+      {% endif %}
+      <li><a href="{% url 'projectimagerecipes' project.id %}">Image recipes</a></li>
+      <li><a href="{% url 'projectsoftwarerecipes' project.id %}">Software recipes</a></li>
       <li><a href="{% url 'projectmachines' project.id %}">Machines</a></li>
       <li><a href="{% url 'projectlayers' project.id %}">Layers</a></li>
       <li class="nav-header">Extra configuration</li>
lib/toaster/toastergui/templates/customise_btn.html new file +9 -0
new file mode 100644
index 0000000..54d05f9
@@ -0,0 +1,9 @@
+<button class="btn btn-block layer-exists-{{data.layer_version.id}} customise-btn" style="display:none;" data-recipe="{{data.id}}">
+  Customise
+</button>
+
+<button class="btn btn-block layer-add-{{data.layer_version.id}} layerbtn" data-layer='{ "id": {{data.layer_version.id}}, "name": "{{data.layer_version.layer.name}}", "layerdetailurl": "{% url 'layerdetails' extra.pid data.layer_version.id %}"}' data-directive="add">
+  <i class="icon-plus"></i>
+  Add layer
+</button>
+
lib/toaster/toastergui/templates/customrecipe.html new file +142 -0

142 lines not on this page · raw message

lib/toaster/toastergui/templates/newcustomimage.html new file +54 -0

54 lines not on this page · raw message

lib/toaster/toastergui/templates/pkg_add_rm_btn.html new file +16 -0
new file mode 100644
index 0000000..b766aea
@@ -0,0 +1,16 @@
+<button class="btn btn-block btn-danger add-rm-package-btn" id="package-rm-btn-{{data.pk}}" data-directive="remove" data-package="{{data.pk}}" data-package-url="{% url 'xhr_customrecipe_packages' extra.recipe_id data.pk %}" style="
+  {% if data.pk not in extra.current_packages %}
+    display:none
+  {% endif %}
+  ">
+  <i class="icon-trash no-tooltip"></i>
+  Remove package
+</a>
+<button class="btn btn-block add-rm-package-btn" data-directive="add" id="package-add-btn-{{data.pk}}" data-package="{{data.pk}}" data-package-url="{% url 'xhr_customrecipe_packages' extra.recipe_id data.pk %}" style="
+  {% if data.pk in extra.current_packages %}
+    display:none
+  {% endif %}
+    ">
+<i class="icon-plus"></i>
+ Add package
+</button>
lib/toaster/toastergui/templates/project.html +1 -1
index e8354fd..2f978bc 100644
@@ -67,7 +67,7 @@
 
       <div class="alert alert-info" style="display:none" id="no-most-built">
         <span class="lead">You haven't built any recipes yet</span>
-        <p style="margin-top: 10px;"><a href="{% url 'projecttargets' project.id %}">Choose a recipe to build</a></p>
+        <p style="margin-top: 10px;"><a href="{% url 'projectsoftwarerecipes' project.id %}">Choose a recipe to build</a></p>
       </div>
 
       <ul class="unstyled configuration-list" id="freq-build-list">
lib/toaster/toastergui/templates/projecttopbar.html +8 -1
index ca2741d..a3d1b88 100644
@@ -1,6 +1,6 @@
 <div class="alert alert-success lead" id="project-created-notification" style="margin-top:15px; display:none">
   <button type="button" class="close" data-dismiss="alert">×</button>
-  Your project <strong>{{project.name}}</strong> has been created. You can now <a href="{% url 'projectmachines' project.id %}">select your target machine</a> and <a href="{% url 'projecttargets' project.id %}">choose image recipes</a> to build.
+  Your project <strong>{{project.name}}</strong> has been created. You can now <a href="{% url 'projectmachines' project.id %}">select your target machine</a> and <a href="{% url 'projectsoftwarerecipes' project.id %}">choose image recipes</a> to build.
 </div>
 
 <!-- project name -->
@@ -34,6 +34,13 @@
         Import layer
       </a>
     </li>
+    {% if CUSTOM_IMAGE %}
+    <li>
+      <a href="{% url 'newcustomimage' project.id %}">
+        New custom image
+      </a>
+    </li>
+    {% endif %}
     <li class="pull-right">
       <form class="form-inline" style="margin-bottom:0px;">
         <i class="icon-question-sign get-help heading-help" data-placement="left" title="" data-original-title="Type the name of one or more recipes you want to build, separated by a space. You can also specify a task by appending a semicolon and a task name to the recipe name, like so: <code>busybox:clean</code>"></i>
lib/toaster/toastergui/urls.py +12 -5
index b47a161..a1adbb7 100644
@@ -103,16 +103,13 @@ urlpatterns = patterns('toastergui.views',
             tables.NewCustomImagesTable.as_view(template_name="newcustomimage.html"),
             name="newcustomimage"),
 
-        url(r'^project/(?P<pid>\d+)/availablerecipes/$',
-            tables.ProjectLayersRecipesTable.as_view(template_name="generic-toastertable-page.html"),
-            { 'table_name': tables.ProjectLayersRecipesTable.__name__.lower(),
-              'title' : 'Recipes available for layers in the current project' },
-            name="projectavailabletargets"),
 
         url(r'^project/(?P<pid>\d+)/layers/$',
             tables.LayersTable.as_view(template_name="generic-toastertable-page.html"),
             name="projectlayers"),
 
+
+
         url(r'^project/(?P<pid>\d+)/layer/(?P<layerid>\d+)$',
             'layerdetails', name='layerdetails'),
 
@@ -129,6 +126,16 @@ urlpatterns = patterns('toastergui.views',
             name=tables.LayerMachinesTable.__name__.lower()),
 
 
+        url(r'^project/(?P<pid>\d+)/customrecipe/(?P<recipeid>\d+)/selectpackages/$',
+            tables.SelectPackagesTable.as_view(template_name="generic-toastertable-page.html"), name="recipeselectpackages"),
+
+
+        url(r'^project/(?P<pid>\d+)/customrecipe/(?P<recipe_id>\d+)$',
+            'customrecipe',
+            name="customrecipe"),
+
+
+
         # typeahead api end points
         url(r'^xhr_typeahead/(?P<pid>\d+)/layers$',
             typeaheads.LayersTypeAhead.as_view(), name='xhr_layerstypeahead'),
lib/toaster/toastergui/views.py +9 -0
index 392e56d..b7eddf4 100755
@@ -2790,6 +2790,15 @@ if True:
 
         return(vars_managed,sorted(vars_fstypes),vars_blacklist)
 
+    def customrecipe(request, pid, recipe_id):
+        project = Project.objects.get(pk=pid)
+        context = {'project' : project,
+                   'projectlayers': [],
+                   'recipe' : CustomImageRecipe.objects.get(pk=recipe_id)
+                  }
+
+        return render(request, "customrecipe.html", context)
+
     @_template_renderer("projectconf.html")
     def projectconf(request, pid):
 
Signature · 1 line
1.9.1
#15 lore raw

PATCH 15/22 toaster: Add test cases for new Image customisation features

In reply to brian avery

From: Ed Bartosh <[email protected]>

- Adds tests for new ToasterTables
- Adds tests for new ReST API

co-author: Elliot Smith <[email protected]>
Michael Wood <[email protected]>

---
 lib/toaster/toastergui/tests.py | 187 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 182 insertions(+), 5 deletions(-)
lib/toaster/toastergui/tests.py +182 -5
index 53012b4..5d15ac9 100644
@@ -24,13 +24,21 @@
 import re
 
 from django.test import TestCase
+from django.test.client import RequestFactory
 from django.core.urlresolvers import reverse
 from django.utils import timezone
-from orm.models import Project, Release, BitbakeVersion, ProjectTarget
+
+from orm.models import Project, Release, BitbakeVersion, Build, Package
 from orm.models import ReleaseLayerSourcePriority, LayerSource, Layer, Build
 from orm.models import Layer_Version, Recipe, Machine, ProjectLayer, Target
+from orm.models import CustomImageRecipe
+from orm.models import Branch
+
+from toastergui.tables import SoftwareRecipesTable
+from django.utils import timezone
 import json
 from bs4 import BeautifulSoup
+import re
 
 PROJECT_NAME = "test project"
 
@@ -41,27 +49,58 @@ class ViewTests(TestCase):
         bbv = BitbakeVersion.objects.create(name="test bbv", giturl="/tmp/",
                                             branch="master", dirpath="")
         release = Release.objects.create(name="test release",
+                                         branch_name="master",
                                          bitbake_version=bbv)
         self.project = Project.objects.create_project(name=PROJECT_NAME,
                                                       release=release)
+        now = timezone.now()
+
+        build = Build.objects.create(project=self.project,
+                                     started_on=now,
+                                     completed_on=now)
+
         layersrc = LayerSource.objects.create(sourcetype=LayerSource.TYPE_IMPORTED)
         self.priority = ReleaseLayerSourcePriority.objects.create(release=release,
                                                                   layer_source=layersrc)
         layer = Layer.objects.create(name="base-layer", layer_source=layersrc,
                                      vcs_url="/tmp/")
 
+        branch = Branch.objects.create(name="master", layer_source=layersrc)
+
         lver = Layer_Version.objects.create(layer=layer, project=self.project,
-                                            layer_source=layersrc, commit="master")
+                                            layer_source=layersrc, commit="master",
+                                            up_branch=branch)
 
-        Recipe.objects.create(layer_source=layersrc, name="base-recipe",
-                              version="1.2", summary="one recipe",
-                              description="recipe", layer_version=lver)
+        self.recipe1 = Recipe.objects.create(layer_source=layersrc,
+                                       name="base-recipe",
+                                       version="1.2",
+                                       summary="one recipe",
+                                       description="recipe",
+                                       layer_version=lver)
 
         Machine.objects.create(layer_version=lver, name="wisk",
                                description="wisking machine")
 
         ProjectLayer.objects.create(project=self.project, layercommit=lver)
 
+
+        self.customr = CustomImageRecipe.objects.create(\
+                           name="custom recipe", project=self.project,
+                           base_recipe=self.recipe1)
+
+        self.package = Package.objects.create(name='pkg1', recipe=self.recipe1,
+                                              build=build)
+
+
+        # recipe with project for testing AvailableRecipe table
+        self.recipe2 = Recipe.objects.create(layer_source=layersrc,
+                                             name="fancy-recipe",
+                                             version="1.4",
+                                             summary="a fancy recipe",
+                                             description="fancy recipe",
+                                             layer_version=lver,
+                                             file_path='/home/foo')
+
         self.assertTrue(lver in self.project.compatible_layerversions())
 
     def test_get_base_call_returns_html(self):
@@ -183,6 +222,144 @@ class ViewTests(TestCase):
         data = json.loads(response.content)
         self.assertNotEqual(data["error"], "ok")
 
+    def test_custom_ok(self):
+        """Test successful return from ReST API xhr_customrecipe"""
+        url = reverse('xhr_customrecipe')
+        params = {'name': 'custom', 'project': self.project.id,
+                  'base': self.recipe1.id}
+        response = self.client.post(url, params)
+        self.assertEqual(response.status_code, 200)
+        data = json.loads(response.content)
+        self.assertEqual(data['error'], 'ok')
+        self.assertTrue('url' in data)
+        # get recipe from the database
+        recipe = CustomImageRecipe.objects.get(project=self.project,
+                                               name=params['name'])
+        args = (self.project.id, recipe.id,)
+        self.assertEqual(reverse('customrecipe', args=args), data['url'])
+
+    def test_custom_incomplete_params(self):
+        """Test not passing all required parameters to xhr_customrecipe"""
+        url = reverse('xhr_customrecipe')
+        for params in [{}, {'name': 'custom'},
+                       {'name': 'custom', 'project': self.project.id}]:
+            response = self.client.post(url, params)
+            self.assertEqual(response.status_code, 200)
+            data = json.loads(response.content)
+            self.assertNotEqual(data["error"], "ok")
+
+    def test_xhr_custom_wrong_project(self):
+        """Test passing wrong project id to xhr_customrecipe"""
+        url = reverse('xhr_customrecipe')
+        params = {'name': 'custom', 'project': 0, "base": self.recipe1.id}
+        response = self.client.post(url, params)
+        self.assertEqual(response.status_code, 200)
+        data = json.loads(response.content)
+        self.assertNotEqual(data["error"], "ok")
+
+    def test_xhr_custom_wrong_base(self):
+        """Test passing wrong base recipe id to xhr_customrecipe"""
+        url = reverse('xhr_customrecipe')
+        params = {'name': 'custom', 'project': self.project.id, "base": 0}
+        response = self.client.post(url, params)
+        self.assertEqual(response.status_code, 200)
+        data = json.loads(response.content)
+        self.assertNotEqual(data["error"], "ok")
+
+    def test_xhr_custom_details(self):
+        """Test getting custom recipe details"""
+        name = "custom recipe"
+        url = reverse('xhr_customrecipe_id', args=(self.customr.id,))
+        response = self.client.get(url)
+        self.assertEqual(response.status_code, 200)
+        expected = {"error": "ok",
+                    "info": {'id': self.customr.id,
+                             'name': name,
+                             'base_recipe_id': self.recipe1.id,
+                             'project_id': self.project.id,
+                            }
+                   }
+        self.assertEqual(json.loads(response.content), expected)
+
+    def test_xhr_custom_del(self):
+        """Test deleting custom recipe"""
+        name = "to be deleted"
+        recipe = CustomImageRecipe.objects.create(\
+                     name=name, project=self.project,
+                     base_recipe=self.recipe1)
+        url = reverse('xhr_customrecipe_id', args=(recipe.id,))
+        response = self.client.delete(url)
+        self.assertEqual(response.status_code, 200)
+        self.assertEqual(json.loads(response.content), {"error": "ok"})
+        # try to delete not-existent recipe
+        url = reverse('xhr_customrecipe_id', args=(recipe.id,))
+        response = self.client.delete(url)
+        self.assertEqual(response.status_code, 200)
+        self.assertNotEqual(json.loads(response.content)["error"], "ok")
+
+    def test_xhr_custom_packages(self):
+        """Test adding and deleting package to a custom recipe"""
+        url = reverse('xhr_customrecipe_packages',
+                      args=(self.customr.id, self.package.id))
+        # add self.package1 to recipe
+        response = self.client.put(url)
+        self.assertEqual(response.status_code, 200)
+        self.assertEqual(json.loads(response.content), {"error": "ok"})
+        self.assertEqual(self.customr.packages.all()[0].id, self.package.id)
+        # delete it
+        response = self.client.delete(url)
+        self.assertEqual(response.status_code, 200)
+        self.assertEqual(json.loads(response.content), {"error": "ok"})
+        self.assertFalse(self.customr.packages.all())
+        # delete it again to test error condition
+        response = self.client.delete(url)
+        self.assertEqual(response.status_code, 200)
+        self.assertNotEqual(json.loads(response.content)["error"], "ok")
+
+    def test_xhr_custom_packages_err(self):
+        """Test error conditions of xhr_customrecipe_packages"""
+        # test calls with wrong recipe id and wrong package id
+        for args in [(0, self.package.id), (self.customr.id, 0)]:
+            url = reverse('xhr_customrecipe_packages', args=args)
+            # test put and delete methods
+            for method in (self.client.put, self.client.delete):
+                response = method(url)
+                self.assertEqual(response.status_code, 200)
+                self.assertNotEqual(json.loads(response.content),
+                                    {"error": "ok"})
+
+    def test_software_recipes_table(self):
+        """Test structure returned for Software RecipesTable"""
+        table = SoftwareRecipesTable()
+        request = RequestFactory().get('/foo/', {'format': 'json'})
+        response = table.get(request, pid=self.project.id)
+        data = json.loads(response.content)
+
+        rows = data['rows']
+        row1 = next(x for x in rows if x['name'] == self.recipe1.name)
+        row1_btns = row1['static:add-del-layers']
+        row1_btns_data = row1['add-del-layers']
+        row2 = next(x for x in rows if x['name'] == self.recipe2.name)
+        row2_btns = row2['static:add-del-layers']
+        row2_btns_data = row2['add-del-layers']
+
+        self.assertEqual(response.status_code, 200, 'should be 200 OK status')
+        self.assertEqual(len(rows), 2, 'should be 2 recipes')
+
+        # check other columns have been populated correctly
+        self.assertEqual(row1['name'], self.recipe1.name)
+        self.assertEqual(row1['version'], self.recipe1.version)
+        self.assertEqual(row1['get_description_or_summary'],
+                         self.recipe1.description)
+        self.assertEqual(row1['layer_version__layer__name'],
+                         self.recipe1.layer_version.layer.name)
+        self.assertEqual(row2['name'], self.recipe2.name)
+        self.assertEqual(row2['version'], self.recipe2.version)
+        self.assertEqual(row2['get_description_or_summary'],
+                         self.recipe2.description)
+        self.assertEqual(row2['layer_version__layer__name'],
+                         self.recipe2.layer_version.layer.name)
+
 class LandingPageTests(TestCase):
     """ Tests for redirects on the landing page """
     # disable bogus pylint message error:
Signature · 1 line
1.9.1
#16 lore raw

PATCH 16/22 toaster: Special case the openembedded-core layer to avoid duplicates

In reply to brian avery

From: Michael Wood <[email protected]>

If the openembedded-core layer is specified in the toasterconf we need to treat it differently because we may also get this layer either from the layerindex source or I can also be provided locally.

---
 lib/toaster/orm/models.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
lib/toaster/orm/models.py +21 -0
index 9790630..4025702 100644
@@ -828,6 +828,7 @@ class LayerIndexLayerSource(LayerSource):
         import urllib2, urlparse, json
         import os
         proxy_settings = os.environ.get("http_proxy", None)
+        oe_core_layer = 'openembedded-core'
 
         def _get_json_response(apiurl = self.apiurl):
             _parsedurl = urlparse.urlparse(apiurl)
@@ -872,6 +873,25 @@ class LayerIndexLayerSource(LayerSource):
         if not connection.features.autocommits_when_autocommit_is_off:
             transaction.set_autocommit(False)
         for li in layers_info:
+            # Special case for the openembedded-core layer
+            if li['name'] == oe_core_layer:
+                try:
+                    # If we have an existing openembedded-core for example
+                    # from the toasterconf.json augment the info using the
+                    # layerindex rather than duplicate it
+                    oe_core_l =  Layer.objects.get(name=oe_core_layer)
+                    # Take ownership of the layer as now coming from the
+                    # layerindex
+                    oe_core_l.layer_source = self
+                    oe_core_l.up_id = li['id']
+                    oe_core_l.summary = li['summary']
+                    oe_core_l.description = li['description']
+                    oe_core_l.save()
+                    continue
+
+                except DoesNotExist:
+                    pass
+
             l, created = Layer.objects.get_or_create(layer_source = self, name = li['name'])
             l.up_id = li['id']
             l.up_date = li['updated']
@@ -882,6 +902,7 @@ class LayerIndexLayerSource(LayerSource):
             l.summary = li['summary']
             l.description = li['description']
             l.save()
+
         if not connection.features.autocommits_when_autocommit_is_off:
             transaction.set_autocommit(True)
 
Signature · 1 line
1.9.1
#17 lore raw

PATCH 17/22 toaster: Create a relationship between build information and toaster layers

In reply to brian avery

From: Michael Wood <[email protected]>

Previously this layer relationship was done by trying to match path information that came back to the buildinfohelper with trying to query for data in toaster's layers table. This rarely matched due to the lose coupling.

---
 lib/bb/ui/buildinfohelper.py                       |  30 +++-
 .../0009_auto__add_field_brlayer_layer_version.py  | 180 +++++++++++++++++++++
 lib/toaster/bldcontrol/models.py                   |   3 +-
 lib/toaster/orm/models.py                          |   2 +-
 4 files changed, 211 insertions(+), 4 deletions(-)
 create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py
lib/bb/ui/buildinfohelper.py +28 -2
index 5098448..d0efaa9 100644
@@ -94,8 +94,8 @@ class ORMWrapper(object):
 
         created = False
         if not key in vars(self)[dictname].keys():
-            vars(self)[dictname][key] = clazz.objects.create(**kwargs)
-            created = True
+            vars(self)[dictname][key], created = \
+                clazz.objects.get_or_create(**kwargs)
 
         return (vars(self)[dictname][key], created)
 
@@ -271,6 +271,17 @@ class ORMWrapper(object):
         return recipe_object
 
     def get_update_layer_version_object(self, build_obj, layer_obj, layer_version_information):
+        if isinstance(layer_obj, Layer_Version):
+            # We already found our layer version for this build so just
+            # update it with the new build information
+            logger.debug("We found our layer from toaster")
+            layer_obj.build = build_obj
+            layer_obj.local_path = layer_version_information['local_path']
+            layer_obj.commit = layer_version_information['commit']
+            layer_obj.save()
+            self.layer_version_objects.append(layer_obj)
+            return layer_obj
+
         assert isinstance(build_obj, Build)
         assert isinstance(layer_obj, Layer)
         assert 'branch' in layer_version_information
@@ -320,8 +331,15 @@ class ORMWrapper(object):
                     localdirname = os.path.join(bc.be.sourcedir, localdirname)
                 #logger.debug(1, "Localdirname %s lcal_path %s" % (localdirname, layer_information['local_path']))
                 if localdirname.startswith(layer_information['local_path']):
+                  # If the build request came from toaster this field
+                  # should contain the information from the layer_version
+                  # That created this build request.
+                    if brl.layer_version:
+                        return brl.layer_version
+
                     # we matched the BRLayer, but we need the layer_version that generated this BR; reverse of the Project.schedule_build()
                     #logger.debug(1, "Matched %s to BRlayer %s" % (pformat(layer_information["local_path"]), localdirname))
+
                     for pl in buildrequest.project.projectlayer_set.filter(layercommit__layer__name = brl.name):
                         if pl.layercommit.layer.vcs_url == brl.giturl :
                             layer = pl.layercommit.layer
@@ -674,6 +692,7 @@ class BuildInfoHelper(object):
     def __init__(self, server, has_build_history = False):
         self.internal_state = {}
         self.internal_state['taskdata'] = {}
+        self.internal_state['targets'] = []
         self.task_order = 0
         self.autocommit_step = 1
         self.server = server
@@ -752,8 +771,15 @@ class BuildInfoHelper(object):
                 if not localdirname.startswith("/"):
                     localdirname = os.path.join(bc.be.sourcedir, localdirname)
                 if path.startswith(localdirname):
+                    # If the build request came from toaster this field
+                    # should contain the information from the layer_version
+                    # That created this build request.
+                    if brl.layer_version:
+                        return brl.layer_version
+
                     #logger.warn("-- managed: matched path %s with layer %s " % (path, localdirname))
                     # we matched the BRLayer, but we need the layer_version that generated this br
+
                     for lvo in self.orm_wrapper.layer_version_objects:
                         if brl.name == lvo.layer.name:
                             return lvo
lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py new file +180 -0

181 lines not on this page · raw message

lib/toaster/bldcontrol/models.py +2 -1
index b61de58..f2493a8 100644
@@ -1,6 +1,6 @@
 from django.db import models
 from django.core.validators import MaxValueValidator, MinValueValidator
-from orm.models import Project, ProjectLayer, ProjectVariable, ProjectTarget, Build
+from orm.models import Project, ProjectLayer, ProjectVariable, ProjectTarget, Build, Layer_Version
 
 # a BuildEnvironment is the equivalent of the "build/" directory on the localhost
 class BuildEnvironment(models.Model):
@@ -137,6 +137,7 @@ class BRLayer(models.Model):
     giturl      = models.CharField(max_length = 254)
     commit      = models.CharField(max_length = 254)
     dirpath     = models.CharField(max_length = 254)
+    layer_version = models.ForeignKey(Layer_Version, null=True)
 
 class BRBitbake(models.Model):
     req         = models.ForeignKey(BuildRequest, unique = True)    # only one bitbake for a request
lib/toaster/orm/models.py +1 -1
index 4025702..8d7388e 100644
@@ -260,7 +260,7 @@ class Project(models.Model):
             for l in self.projectlayer_set.all().order_by("pk"):
                 commit = l.layercommit.get_vcs_reference()
                 print("ii Building layer ", l.layercommit.layer.name, " at vcs point ", commit)
-                BRLayer.objects.create(req = br, name = l.layercommit.layer.name, giturl = l.layercommit.layer.vcs_url, commit = commit, dirpath = l.layercommit.dirpath)
+                BRLayer.objects.create(req = br, name = l.layercommit.layer.name, giturl = l.layercommit.layer.vcs_url, commit = commit, dirpath = l.layercommit.dirpath, layer_version=l.layercommit)
 
             br.state = BuildRequest.REQ_QUEUED
             now = timezone.now()
Signature · 1 line
1.9.1
#18 lore raw

PATCH 18/22 toaster: Prioroitise the layer more generic vcs reference over the sha

In reply to brian avery

From: Michael Wood <[email protected]>

When we do a build we update the last commit value that the layer was built at However in future builds we do want to use the named reference rather than the commit sha, e.g. master/fido

---
 lib/toaster/orm/models.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/toaster/orm/models.py +2 -2
index 8d7388e..e0b31a9 100644
@@ -1155,12 +1155,12 @@ class Layer_Version(models.Model):
         return project.compatible_layerversions(layer_name = self.layer.name)
 
     def get_vcs_reference(self):
-        if self.commit is not None and len(self.commit) > 0:
-            return self.commit
         if self.branch is not None and len(self.branch) > 0:
             return self.branch
         if self.up_branch is not None:
             return self.up_branch.name
+        if self.commit is not None and len(self.commit) > 0:
+            return self.commit
         return ("Cannot determine the vcs_reference for layer version %s" % vars(self))
 
     def get_detailspage_url(self, project_id):
Signature · 1 line
1.9.1
#19 lore raw

PATCH 19/22 toaster: tables show all recipes in the layerdetails even duplicates

In reply to brian avery

From: Michael Wood <[email protected]>

For some layers we have multiple recipes for the same software with differing versions. Change to showing all versions and add a version column to the table instead.

---
 lib/toaster/toastergui/tables.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/toaster/toastergui/tables.py +4 -2
index 1526d59..3354072 100644
@@ -403,8 +403,8 @@ class LayerRecipesTable(RecipesTable):
 
 
     def setup_queryset(self, *args, **kwargs):
-        RecipesTable.setup_queryset(self, *args, **kwargs)
-        self.queryset = self.queryset.filter(layer_version__pk=int(kwargs['layerid']))
+        self.queryset = \
+                Recipe.objects.filter(layer_version__pk=int(kwargs['layerid']))
 
         self.static_context_extra['in_prj'] = ProjectLayer.objects.filter(Q(project=kwargs['pid']) & Q(layercommit=kwargs['layerid'])).count()
 
@@ -418,6 +418,8 @@ class LayerRecipesTable(RecipesTable):
         self.add_column(title="Description",
                         field_name="get_description_or_summary")
 
+        self.add_column(title="Version",
+                        field_name="version")
 
         build_recipe_template ='<button class="btn btn-block build-recipe-btn" data-recipe-name="{{data.name}}" {%if extra.in_prj == 0 %}disabled="disabled"{%endif%}>Build recipe</button>'
 
Signature · 1 line
1.9.1
#20 lore raw

PATCH 20/22 toaster: buildinfohelper Create a copy of the built layer and recipe

In reply to brian avery

From: Michael Wood <[email protected]>

Create a copy of the built layer and the recipes associated with it. This is so that the user can view the historical information about a build. i.e. a snapshot of the layer version and artifacts produced at that build.

---
 lib/bb/ui/buildinfohelper.py | 54 +++++++++++++++++++++++++++++++++-----------
 lib/toaster/orm/models.py    |  2 +-
 2 files changed, 42 insertions(+), 14 deletions(-)
lib/bb/ui/buildinfohelper.py +41 -13
index d0efaa9..e036ef6 100644
@@ -66,6 +66,7 @@ class ORMWrapper(object):
 
     def __init__(self):
         self.layer_version_objects = []
+        self.layer_version_built = []
         self.task_objects = {}
         self.recipe_objects = {}
 
@@ -254,32 +255,59 @@ class ORMWrapper(object):
 
         assert not recipe_information['file_path'].startswith("/")      # we should have layer-relative paths at all times
 
-        recipe_object, created = self._cached_get_or_create(Recipe, layer_version=recipe_information['layer_version'],
+
+        def update_recipe_obj(recipe_object):
+            object_changed = False
+            for v in vars(recipe_object):
+                if v in recipe_information.keys():
+                    object_changed = True
+                    vars(recipe_object)[v] = recipe_information[v]
+
+            if object_changed:
+                recipe_object.save()
+
+        recipe, created = self._cached_get_or_create(Recipe, layer_version=recipe_information['layer_version'],
                                      file_path=recipe_information['file_path'], pathflags = recipe_information['pathflags'])
-        if created and must_exist:
-            raise NotExisting("Recipe object created when expected to exist", recipe_information)
 
-        object_changed = False
-        for v in vars(recipe_object):
-            if v in recipe_information.keys():
-                object_changed = True
-                vars(recipe_object)[v] = recipe_information[v]
+        update_recipe_obj(recipe)
 
-        if object_changed:
-            recipe_object.save()
+        # Create a copy of the recipe for historical puposes and update it
+        for built_layer in self.layer_version_built:
+            if built_layer.layer == recipe_information['layer_version'].layer:
+                built_recipe, c = self._cached_get_or_create(Recipe,
+                        layer_version=built_layer,
+                        file_path=recipe_information['file_path'],
+                        pathflags = recipe_information['pathflags'])
+                update_recipe_obj(built_recipe)
+                break
 
-        return recipe_object
+
+
+        if created and must_exist:
+            raise NotExisting("Recipe object created when expected to exist", recipe_information)
+
+        return recipe
 
     def get_update_layer_version_object(self, build_obj, layer_obj, layer_version_information):
         if isinstance(layer_obj, Layer_Version):
             # We already found our layer version for this build so just
             # update it with the new build information
             logger.debug("We found our layer from toaster")
-            layer_obj.build = build_obj
             layer_obj.local_path = layer_version_information['local_path']
-            layer_obj.commit = layer_version_information['commit']
             layer_obj.save()
             self.layer_version_objects.append(layer_obj)
+
+            # create a new copy of this layer version as a snapshot for
+            # historical purposes
+            layer_copy, c = Layer_Version.objects.get_or_create(build=build_obj,
+                            layer=layer_obj.layer,
+                            commit=layer_version_information['commit'],
+                            local_path = layer_version_information['local_path'],
+                            )
+            logger.warning("created new historical layer version %d", layer_copy.pk)
+
+            self.layer_version_built.append(layer_copy)
+
             return layer_obj
 
         assert isinstance(build_obj, Build)
lib/toaster/orm/models.py +1 -1
index e0b31a9..9a052bf 100644
@@ -194,7 +194,7 @@ class Project(models.Model):
         if release == None:
             release = self.release
         # layers on the same branch or layers specifically set for this project
-        queryset = Layer_Version.objects.filter((Q(up_branch__name = release.branch_name) & Q(project = None)) | Q(project = self) | Q(build__project = self))
+        queryset = Layer_Version.objects.filter(((Q(up_branch__name = release.branch_name) & Q(project = None)) | Q(project = self)) & Q(build__isnull=True))
 
         if layer_name is not None:
             # we select only a layer name
Signature · 1 line
1.9.1
#21 lore raw

PATCH 21/22 Revert "bitbake: toaster: don't re-create Target objects"

In reply to brian avery

From: Michael Wood <[email protected]>

This delete followed up the foreign keys and deleted things that were not expected to be deleted.

This reverts commit 08000eb27eb8413686fb2c8daf14d234a8bff83a.

---
 lib/bb/ui/buildinfohelper.py | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
lib/bb/ui/buildinfohelper.py +16 -1
index e036ef6..9fdde29 100644
@@ -162,6 +162,8 @@ class ORMWrapper(object):
             build.bitbake_version=build_info['bitbake_version']
             build.save()
 
+            Target.objects.filter(build = build).delete()
+
         else:
             build = Build.objects.create(
                                     project = prj,
@@ -182,6 +184,19 @@ class ORMWrapper(object):
 
         return build
 
+    def create_target_objects(self, target_info):
+        assert 'build' in target_info
+        assert 'targets' in target_info
+
+        targets = []
+        for tgt_name in target_info['targets']:
+            tgt_object = Target.objects.create( build = target_info['build'],
+                                    target = tgt_name,
+                                    is_image = False,
+                                    )
+            targets.append(tgt_object)
+        return targets
+
     def update_build_object(self, build, errors, warnings, taskfailures):
         assert isinstance(build,Build)
         assert isinstance(errors, int)
@@ -911,7 +926,7 @@ class BuildInfoHelper(object):
         target_information['targets'] = event._pkgs
         target_information['build'] = build_obj
 
-        self.internal_state['targets'] = Target.objects.filter(build=target_information['build'])
+        self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
 
         # Save build configuration
         data = self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0]
Signature · 1 line
1.9.1
#22 lore raw

PATCH 22/22 toaster: orm remove the complicated querying on the ORM

In reply to brian avery

From: Michael Wood <[email protected]>

We no longer need to compute each layer_version and all the recipes which belong to this.

[YOCTO #8147]

---
 lib/toaster/orm/models.py            | 49 ++++++++++++++++++++++--------------
 lib/toaster/toastergui/typeaheads.py |  5 ++--
 2 files changed, 33 insertions(+), 21 deletions(-)
lib/toaster/orm/models.py +30 -19
index 9a052bf..5aed158 100644
@@ -191,6 +191,7 @@ class Project(models.Model):
 
     # returns a queryset of compatible layers for a project
     def compatible_layerversions(self, release = None, layer_name = None):
+        logger.warning("This function is deprecated")
         if release == None:
             release = self.release
         # layers on the same branch or layers specifically set for this project
@@ -205,45 +206,55 @@ class Project(models.Model):
 
         return queryset
 
-    def projectlayer_equivalent_set(self):
-        return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch")
+    def get_all_compatible_layer_versions(self):
+        """ Returns Queryset of all Layer_Versions which are compatible with
+        this project"""
+        queryset = Layer_Version.objects.filter(
+            (Q(up_branch__name=self.release.branch_name) & Q(build=None))
+            | Q(project=self))
+
+        return queryset
+
+    def get_project_layer_versions(self, pk=False):
+        """ Returns the Layer_Versions currently added to this project """
+        layer_versions = self.projectlayer_set.all().values('layercommit')
+
+        if pk is False:
+            return layer_versions
+        else:
+            return layer_versions.values_list('pk', flat=True)
+
 
     def get_available_machines(self):
         """ Returns QuerySet of all Machines which are provided by the
         Layers currently added to the Project """
-        queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set)
+        queryset = Machine.objects.filter(
+            layer_version__in=self.get_project_layer_versions(self))
+
         return queryset
 
     def get_all_compatible_machines(self):
         """ Returns QuerySet of all the compatible machines available to the
         project including ones from Layers not currently added """
-        compatible_layers = self.compatible_layerversions()
+        queryset = Machine.objects.filter(
+            layer_version__in=self.get_all_compatible_layer_versions())
 
-        queryset = Machine.objects.filter(layer_version__in=compatible_layers)
         return queryset
 
     def get_available_recipes(self):
-        """ Returns QuerySet of all Recipes which are provided by the Layers
-        currently added to the Project """
-        project_layers = self.projectlayer_equivalent_set()
-        queryset = Recipe.objects.filter(layer_version__in = project_layers)
-
-        # Copied from get_all_compatible_recipes
-        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
-        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
-        # End copy
+        """ Returns QuerySet of all the recipes that are provided by layers
+        added to this project """
+        queryset = Recipe.objects.filter(
+            layer_version__in=self.get_project_layer_versions())
 
         return queryset
 
     def get_all_compatible_recipes(self):
         """ Returns QuerySet of all the compatible Recipes available to the
         project including ones from Layers not currently added """
-        compatible_layerversions = self.compatible_layerversions()
-        queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions)
-
-        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
+        queryset = Recipe.objects.filter(
+            layer_version__in=self.get_all_compatible_layer_versions())
 
-        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
         return queryset
 
 
lib/toaster/toastergui/typeaheads.py +3 -2
index d5bec58..9db3182 100644
@@ -27,7 +27,7 @@ class LayersTypeAhead(ToasterTypeAhead):
       super(LayersTypeAhead, self).__init__()
 
     def apply_search(self, search_term, prj, request):
-        layers = prj.compatible_layerversions()
+        layers = prj.get_all_compatible_layer_versions()
         layers = layers.order_by('layer__name')
 
         # Unlike the other typeaheads we also don't want to show suggestions
@@ -35,7 +35,8 @@ class LayersTypeAhead(ToasterTypeAhead):
         # layerdeps to a new layer.
         if ("include_added" in request.GET and
                 request.GET['include_added'] != "true"):
-            layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set)
+            layers = layers.exclude(
+                pk__in=prj.get_project_layer_versions(pk=True))
 
         primary_results = layers.filter(layer__name__istartswith=search_term)
         secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)
Signature · 1 line
1.9.1
#23 lore raw

PATCH v2 22/22 toaster: orm remove the complicated querying on the ORM

We no longer need to compute each layer_version and all the recipes which belong to this.

[YOCTO #8147]

---
 lib/toaster/orm/models.py            | 49 ++++++++++++++++++++++--------------
 lib/toaster/toastergui/tables.py     |  2 +-
 lib/toaster/toastergui/typeaheads.py |  5 ++--
 3 files changed, 34 insertions(+), 22 deletions(-)
lib/toaster/orm/models.py +30 -19
index 9a052bf..5aed158 100644
@@ -191,6 +191,7 @@ class Project(models.Model):
 
     # returns a queryset of compatible layers for a project
     def compatible_layerversions(self, release = None, layer_name = None):
+        logger.warning("This function is deprecated")
         if release == None:
             release = self.release
         # layers on the same branch or layers specifically set for this project
@@ -205,45 +206,55 @@ class Project(models.Model):
 
         return queryset
 
-    def projectlayer_equivalent_set(self):
-        return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch")
+    def get_all_compatible_layer_versions(self):
+        """ Returns Queryset of all Layer_Versions which are compatible with
+        this project"""
+        queryset = Layer_Version.objects.filter(
+            (Q(up_branch__name=self.release.branch_name) & Q(build=None))
+            | Q(project=self))
+
+        return queryset
+
+    def get_project_layer_versions(self, pk=False):
+        """ Returns the Layer_Versions currently added to this project """
+        layer_versions = self.projectlayer_set.all().values('layercommit')
+
+        if pk is False:
+            return layer_versions
+        else:
+            return layer_versions.values_list('pk', flat=True)
+
 
     def get_available_machines(self):
         """ Returns QuerySet of all Machines which are provided by the
         Layers currently added to the Project """
-        queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set)
+        queryset = Machine.objects.filter(
+            layer_version__in=self.get_project_layer_versions(self))
+
         return queryset
 
     def get_all_compatible_machines(self):
         """ Returns QuerySet of all the compatible machines available to the
         project including ones from Layers not currently added """
-        compatible_layers = self.compatible_layerversions()
+        queryset = Machine.objects.filter(
+            layer_version__in=self.get_all_compatible_layer_versions())
 
-        queryset = Machine.objects.filter(layer_version__in=compatible_layers)
         return queryset
 
     def get_available_recipes(self):
-        """ Returns QuerySet of all Recipes which are provided by the Layers
-        currently added to the Project """
-        project_layers = self.projectlayer_equivalent_set()
-        queryset = Recipe.objects.filter(layer_version__in = project_layers)
-
-        # Copied from get_all_compatible_recipes
-        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
-        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
-        # End copy
+        """ Returns QuerySet of all the recipes that are provided by layers
+        added to this project """
+        queryset = Recipe.objects.filter(
+            layer_version__in=self.get_project_layer_versions())
 
         return queryset
 
     def get_all_compatible_recipes(self):
         """ Returns QuerySet of all the compatible Recipes available to the
         project including ones from Layers not currently added """
-        compatible_layerversions = self.compatible_layerversions()
-        queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions)
-
-        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
+        queryset = Recipe.objects.filter(
+            layer_version__in=self.get_all_compatible_layer_versions())
 
-        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
         return queryset
 
 
lib/toaster/toastergui/tables.py +1 -1
index 3354072..5cc04a2 100644
@@ -221,7 +221,7 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin):
 
     def setup_filters(self, *args, **kwargs):
         project = Project.objects.get(pk=kwargs['pid'])
-        self.project_layers = project.projectlayer_equivalent_set()
+        self.project_layers = project.get_project_layer_versions()
 
         self.add_filter(title="Filter by project machines",
                         name="in_current_project",
lib/toaster/toastergui/typeaheads.py +3 -2
index d5bec58..9db3182 100644
@@ -27,7 +27,7 @@ class LayersTypeAhead(ToasterTypeAhead):
       super(LayersTypeAhead, self).__init__()
 
     def apply_search(self, search_term, prj, request):
-        layers = prj.compatible_layerversions()
+        layers = prj.get_all_compatible_layer_versions()
         layers = layers.order_by('layer__name')
 
         # Unlike the other typeaheads we also don't want to show suggestions
@@ -35,7 +35,8 @@ class LayersTypeAhead(ToasterTypeAhead):
         # layerdeps to a new layer.
         if ("include_added" in request.GET and
                 request.GET['include_added'] != "true"):
-            layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set)
+            layers = layers.exclude(
+                pk__in=prj.get_project_layer_versions(pk=True))
 
         primary_results = layers.filter(layer__name__istartswith=search_term)
         secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)
Signature · 1 line
2.1.4
#24 lore raw

Re: [PATCH v2 22/22] toaster: orm remove the complicated querying on the ORM

Quoted 127 lines · Michael Wood

We no longer need to compute each layer_version and all the recipes which belong to this.

[YOCTO #8147]

---
  lib/toaster/orm/models.py            | 49 ++++++++++++++++++++++--------------
  lib/toaster/toastergui/tables.py     |  2 +-
  lib/toaster/toastergui/typeaheads.py |  5 ++--
  3 files changed, 34 insertions(+), 22 deletions(-)
lib/toaster/orm/models.py +30 -19
index 9a052bf..5aed158 100644
@@ -191,6 +191,7 @@ class Project(models.Model):
 
      # returns a queryset of compatible layers for a project
      def compatible_layerversions(self, release = None, layer_name = None):
+        logger.warning("This function is deprecated")
          if release == None:
              release = self.release
          # layers on the same branch or layers specifically set for this project
@@ -205,45 +206,55 @@ class Project(models.Model):
 
          return queryset
 
-    def projectlayer_equivalent_set(self):
-        return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch")
+    def get_all_compatible_layer_versions(self):
+        """ Returns Queryset of all Layer_Versions which are compatible with
+        this project"""
+        queryset = Layer_Version.objects.filter(
+            (Q(up_branch__name=self.release.branch_name) & Q(build=None))
+            | Q(project=self))
+
+        return queryset
+
+    def get_project_layer_versions(self, pk=False):
+        """ Returns the Layer_Versions currently added to this project """
+        layer_versions = self.projectlayer_set.all().values('layercommit')
+
+        if pk is False:
+            return layer_versions
+        else:
+            return layer_versions.values_list('pk', flat=True)
+
 
      def get_available_machines(self):
          """ Returns QuerySet of all Machines which are provided by the
          Layers currently added to the Project """
-        queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set)
+        queryset = Machine.objects.filter(
+            layer_version__in=self.get_project_layer_versions(self))
+
          return queryset
 
      def get_all_compatible_machines(self):
          """ Returns QuerySet of all the compatible machines available to the
          project including ones from Layers not currently added """
-        compatible_layers = self.compatible_layerversions()
+        queryset = Machine.objects.filter(
+            layer_version__in=self.get_all_compatible_layer_versions())
 
-        queryset = Machine.objects.filter(layer_version__in=compatible_layers)
          return queryset
 
      def get_available_recipes(self):
-        """ Returns QuerySet of all Recipes which are provided by the Layers
-        currently added to the Project """
-        project_layers = self.projectlayer_equivalent_set()
-        queryset = Recipe.objects.filter(layer_version__in = project_layers)
-
-        # Copied from get_all_compatible_recipes
-        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
-        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
-        # End copy
+        """ Returns QuerySet of all the recipes that are provided by layers
+        added to this project """
+        queryset = Recipe.objects.filter(
+            layer_version__in=self.get_project_layer_versions())
 
          return queryset
 
      def get_all_compatible_recipes(self):
          """ Returns QuerySet of all the compatible Recipes available to the
          project including ones from Layers not currently added """
-        compatible_layerversions = self.compatible_layerversions()
-        queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions)
-
-        search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
+        queryset = Recipe.objects.filter(
+            layer_version__in=self.get_all_compatible_layer_versions())
 
-        queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
          return queryset
 
 
lib/toaster/toastergui/tables.py +1 -1
index 3354072..5cc04a2 100644
@@ -221,7 +221,7 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin):
 
      def setup_filters(self, *args, **kwargs):
          project = Project.objects.get(pk=kwargs['pid'])
-        self.project_layers = project.projectlayer_equivalent_set()
+        self.project_layers = project.get_project_layer_versions()
 
          self.add_filter(title="Filter by project machines",
                          name="in_current_project",
lib/toaster/toastergui/typeaheads.py +1 -1
index d5bec58..9db3182 100644
@@ -27,7 +27,7 @@ class LayersTypeAhead(ToasterTypeAhead):
        super(LayersTypeAhead, self).__init__()
 
      def apply_search(self, search_term, prj, request):
-        layers = prj.compatible_layerversions()
+        layers = prj.get_all_compatible_layer_versions()
          layers = layers.order_by('layer__name')
 
          # Unlike the other typeaheads we also don't want to show suggestions

lib/toaster/toastergui/typeaheads.py class LayersTypeAhead(ToasterTypeAhead): · line 42

Quoted 9 lines
lib/toaster/toastergui/typeaheads.py +2 -1
@@ -35,7 +35,8 @@ class LayersTypeAhead(ToasterTypeAhead):
          # layerdeps to a new layer.
          if ("include_added" in request.GET and
                  request.GET['include_added'] != "true"):
-            layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set)
+            layers = layers.exclude(
+                pk__in=prj.get_project_layer_versions(pk=True))
 
          primary_results = layers.filter(layer__name__istartswith=search_term)
          secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)

We had a bit of a race condition on creating the pull request, this last patch was updated in the original branch but the submission branch was branched just before, hence v2 here.

Thanks,

Michael