image customization
cover image customization
This large patch set brings the databse changes needed to support image customization as well as the skeleton of the image customization user interface. The ui for image customization is off/hidden by default. The database changes fix bugs 7967,8035,8147, and 8403.
Thanks,
Brian
The following changes since commit f78f90240a3041be426c6ba9085849352164ee8b:
bitbake: prserv/serv.py: Better messaging when starting/stopping the server with port=0 (2015-09-28 12:00:33 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib bavery/submit/michaelw/20150928_ic5.1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=bavery/submit/michaelw/20150928_ic5.1
Ed Bartosh (5):
toaster: make a workaround for old style index
toaster: create custom layer and recipes for Image customisation
toaster: implement decorator for REST responses
toaster: Add new ReST API for Image Customisation feature
toaster: Add test cases for new Image customisation features
Michael Wood (17):
toaster: tables Move the title and name into the widget
toaster: widgets ToasterTable add logger to notify when cache hit
toaster: widgets ToasterTable Add more info to search field exception
toaster: add nocache option to the ToasterTable widget
toaster: ToasterTable remove unused class definition
toaster: Add CustomImageRecipe model
toaster: add toggle for enabling image customisation feeature
toaster: Fix indentation of jsunittests view
toaster: Add ToasterTables for Image customisation feature
toaster: Add Image customisation frontend feature
toaster: Special case the openembedded-core layer to avoid duplicates
toaster: Create a relationship between build information and toaster
layers
toaster: Prioroitise the layer more generic vcs reference over the sha
toaster: tables show all recipes in the layerdetails even duplicates
toaster: buildinfohelper Create a copy of the built layer and recipe Revert "bitbake: toaster: don't re-create Target objects"
toaster: orm remove the complicated querying on the ORM
lib/bb/ui/buildinfohelper.py | 95 +++++- lib/toaster/bldcontrol/localhostbecontroller.py | 51 ++- .../0009_auto__add_field_brlayer_layer_version.py | 180 ++++++++++ lib/toaster/bldcontrol/models.py | 3 +- ...e__add_unique_customimagerecipe_name_project.py | 375 +++++++++++++++++++++ lib/toaster/orm/models.py | 91 +++-- lib/toaster/toastergui/static/js/customrecipe.js | 50 +++ lib/toaster/toastergui/static/js/layerBtn.js | 13 + lib/toaster/toastergui/static/js/newcustomimage.js | 49 +++ lib/toaster/toastergui/tables.py | 230 +++++++++++-- .../toastergui/templates/baseprojectpage.html | 7 +- .../toastergui/templates/customise_btn.html | 9 + lib/toaster/toastergui/templates/customrecipe.html | 142 ++++++++ .../toastergui/templates/newcustomimage.html | 54 +++ .../toastergui/templates/pkg_add_rm_btn.html | 16 + lib/toaster/toastergui/templates/project.html | 2 +- .../toastergui/templates/projecttopbar.html | 9 +- lib/toaster/toastergui/tests.py | 187 +++++++++- lib/toaster/toastergui/typeaheads.py | 5 +- lib/toaster/toastergui/urls.py | 49 ++- lib/toaster/toastergui/views.py | 195 ++++++++++- lib/toaster/toastergui/widgets.py | 50 ++- lib/toaster/toastermain/settings.py | 5 + 23 files changed, 1724 insertions(+), 143 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py create mode 100644 bitbake/lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py create mode 100644 bitbake/lib/toaster/toastergui/static/js/customrecipe.js create mode 100644 bitbake/lib/toaster/toastergui/static/js/newcustomimage.js create mode 100644 bitbake/lib/toaster/toastergui/templates/customise_btn.html create mode 100644 bitbake/lib/toaster/toastergui/templates/customrecipe.html create mode 100644 bitbake/lib/toaster/toastergui/templates/newcustomimage.html create mode 100644 bitbake/lib/toaster/toastergui/templates/pkg_add_rm_btn.html
Signature · 1 line
1.9.1
PATCH 1/22 toaster: make a workaround for old style index
From: Ed Bartosh <[email protected]>
For a new style indexes 'inherits' field is used to determine if recipe is an image recipe.
As old style indexes don't have 'inherits' field this can be guessed from recipe name. Let's consider recipe an image recipe if recipe name contains '-image-'.
--- lib/toaster/orm/models.py | 2 ++ 1 file changed, 2 insertions(+)
lib/toaster/orm/models.py +2 -0
@@ -974,6 +974,8 @@ class LayerIndexLayerSource(LayerSource): ro.file_path = ri['filepath'] + "/" + ri['filename'] if 'inherits' in ri: ro.is_image = 'image' in ri['inherits'].split() + else: # workaround for old style layer index + ro.is_image = "-image-" in ri['pn'] ro.save() except IntegrityError as e: logger.debug("Failed saving recipe, ignoring: %s (%s:%s)" % (e, ro.layer_version, ri['filepath']+"/"+ri['filename']))
Signature · 1 line
1.9.1
PATCH 2/22 toaster: tables Move the title and name into the widget
From: Michael Wood <[email protected]>
For historical reasons this was being set in the urls definition. We can set this in the actual definition of the table and defaults in the widget.
--- lib/toaster/toastergui/tables.py | 2 ++ lib/toaster/toastergui/urls.py | 22 +++++++++++++--------- lib/toaster/toastergui/widgets.py | 9 ++++++++- 3 files changed, 23 insertions(+), 10 deletions(-)
lib/toaster/toastergui/tables.py +2 -0
@@ -50,6 +50,7 @@ class LayersTable(ToasterTable): def __init__(self, *args, **kwargs): super(LayersTable, self).__init__(*args, **kwargs) self.default_orderby = "layer__name" + self.title = "Compatible layers" def get_context_data(self, **kwargs): context = super(LayersTable, self).get_context_data(**kwargs)
@@ -208,6 +209,7 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin): def __init__(self, *args, **kwargs): super(MachinesTable, self).__init__(*args, **kwargs) self.empty_state = "No machines maybe you need to do a build?" + self.title = "Compatible machines" self.default_orderby = "name" def get_context_data(self, **kwargs):
lib/toaster/toastergui/urls.py +13 -9
@@ -87,15 +87,21 @@ urlpatterns = patterns('toastergui.views', # the table pages that have been converted to ToasterTable widget url(r'^project/(?P<pid>\d+)/machines/$', tables.MachinesTable.as_view(template_name="generic-toastertable-page.html"), - { 'table_name': tables.MachinesTable.__name__.lower(), - 'title' : 'Compatible machines' }, name="projectmachines"), - url(r'^project/(?P<pid>\d+)/recipes/$', - tables.RecipesTable.as_view(template_name="generic-toastertable-page.html"), - { 'table_name': tables.RecipesTable.__name__.lower(), - 'title' : 'Compatible recipes' }, - name="projecttargets"), + url(r'^project/(?P<pid>\d+)/softwarerecipes/$', + tables.SoftwareRecipesTable.as_view(template_name="generic-toastertable-page.html"), + name="projectsoftwarerecipes"), + + url(r'^project/(?P<pid>\d+)/images/$', + tables.ImageRecipesTable.as_view(template_name="generic-toastertable-page.html"), name="projectimagerecipes"), + + url(r'^project/(?P<pid>\d+)/customimages/$', + tables.CustomImagesTable.as_view(template_name="generic-toastertable-page.html"), name="projectcustomimages"), + + url(r'^project/(?P<pid>\d+)/newcustomimage/$', + tables.NewCustomImagesTable.as_view(template_name="newcustomimage.html"), + name="newcustomimage"), url(r'^project/(?P<pid>\d+)/availablerecipes/$', tables.ProjectLayersRecipesTable.as_view(template_name="generic-toastertable-page.html"),
@@ -105,8 +111,6 @@ urlpatterns = patterns('toastergui.views', url(r'^project/(?P<pid>\d+)/layers/$', tables.LayersTable.as_view(template_name="generic-toastertable-page.html"), - { 'table_name': tables.LayersTable.__name__.lower(), - 'title' : 'Compatible layers' }, name="projectlayers"), url(r'^project/(?P<pid>\d+)/layer/(?P<layerid>\d+)$',
lib/toaster/toastergui/widgets.py +8 -1
@@ -45,7 +45,7 @@ class ToasterTable(TemplateView): super(ToasterTable, self).__init__() if 'template_name' in kwargs: self.template_name = kwargs['template_name'] - self.title = None + self.title = "Table" self.queryset = None self.columns = [] self.filters = {}
@@ -61,6 +61,13 @@ class ToasterTable(TemplateView): orderable=True, field_name="id") + def get_context_data(self, **kwargs): + context = super(ToasterTable, self).get_context_data(**kwargs) + context['title'] = self.title + context['table_name'] = type(self).__name__.lower() + + return context + def get(self, request, *args, **kwargs): if request.GET.get('format', None) == 'json':
Signature · 1 line
1.9.1
PATCH 3/22 toaster: create custom layer and recipes for Image customisation
In reply to brian averyFrom: Ed Bartosh <[email protected]>
When building customised recipes toaster creates custom layer directory and puts layer.conf and custom recipes to it.
[YOCTO #8075]
--- lib/toaster/bldcontrol/localhostbecontroller.py | 51 ++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 2 deletions(-)
lib/toaster/bldcontrol/localhostbecontroller.py +49 -2
@@ -23,9 +23,11 @@ import os import sys import re +import shutil from django.db import transaction from django.db.models import Q from bldcontrol.models import BuildEnvironment, BRLayer, BRVariable, BRTarget, BRBitbake +from orm.models import CustomImageRecipe, Layer, Layer_Version, ProjectLayer import subprocess from toastermain import settings
@@ -197,7 +199,7 @@ class LocalhostBEController(BuildEnvironmentController): return local_checkout_path - def setLayers(self, bitbakes, layers): + def setLayers(self, bitbakes, layers, targets): """ a word of attention: by convention, the first layer for any build will be poky! """ assert self.be.sourcedir is not None
@@ -299,6 +301,51 @@ class LocalhostBEController(BuildEnvironmentController): if not os.path.exists(bblayerconf): raise BuildSetupException("BE is not consistent: bblayers.conf file missing at %s" % bblayerconf) + # 6. create custom layer and add custom recipes to it + layerpath = os.path.join(self.be.sourcedir, "_meta-toaster-custom") + if os.path.isdir(layerpath): + shutil.rmtree(layerpath) # remove leftovers from previous builds + for target in targets: + try: + customrecipe = CustomImageRecipe.objects.get(name=target.target, + project=bitbakes[0].req.project) + except CustomImageRecipe.DoesNotExist: + continue # not a custom recipe, skip + + # create directory structure + for name in ("conf", "recipes"): + path = os.path.join(layerpath, name) + if not os.path.isdir(path): + os.makedirs(path) + + # create layer.oonf + config = os.path.join(layerpath, "conf", "layer.conf") + if not os.path.isfile(config): + with open(config, "w") as conf: + conf.write('BBPATH .= ":${LAYERDIR}"\nBBFILES += "${LAYERDIR}/recipes/*.bb"\n') + + # create recipe + recipe = os.path.join(layerpath, "recipes", "%s.bb" % target.target) + with open(recipe, "w") as recipef: + recipef.write("require %s\n" % customrecipe.base_recipe.recipe.file_path) + packages = [pkg.name for pkg in customrecipe.packages.all()] + if packages: + recipef.write('IMAGE_INSTALL = "%s"\n' % ' '.join(packages)) + + # create *Layer* objects needed for build machinery to work + layer = Layer.objects.get_or_create(name="Toaster Custom layer", + summary="Layer for custom recipes", + vcs_url="file://%s" % layerpath)[0] + breq = target.req + lver = Layer_Version.objects.get_or_create(project=breq.project, layer=layer, + dirpath=layerpath, build=breq.build)[0] + ProjectLayer.objects.get_or_create(project=breq.project, layercommit=lver, + optional=False) + BRLayer.objects.get_or_create(req=breq, name=layer.name, dirpath=layerpath, + giturl="file://%s" % layerpath) + if os.path.isdir(layerpath): + layerlist.append(layerpath) + BuildEnvironmentController._updateBBLayers(bblayerconf, layerlist) self.islayerset = True
@@ -316,7 +363,7 @@ class LocalhostBEController(BuildEnvironmentController): def triggerBuild(self, bitbake, layers, variables, targets): # set up the buid environment with the needed layers - self.setLayers(bitbake, layers) + self.setLayers(bitbake, layers, targets) self.writeConfFile("conf/toaster-pre.conf", variables) self.writeConfFile("conf/toaster.conf", raw = "INHERIT+=\"toaster buildhistory\"")
Signature · 1 line
1.9.1
PATCH 4/22 toaster: widgets ToasterTable add logger to notify when cache hit
In reply to brian averyFrom: Michael Wood <[email protected]>
Useful for debugging to remind you if you're hitting a cache or not
--- lib/toaster/toastergui/widgets.py | 4 ++++ 1 file changed, 4 insertions(+)
lib/toaster/toastergui/widgets.py +4 -0
@@ -38,6 +38,9 @@ import collections import operator import re +import logging +logger = logging.getLogger("toaster") + from toastergui.views import objtojson class ToasterTable(TemplateView):
@@ -264,6 +267,7 @@ class ToasterTable(TemplateView): data = cache.get(cache_name) if data: + logger.debug("Got cache data for table '%s'" % self.title) return data self.setup_columns(**kwargs)
Signature · 1 line
1.9.1
PATCH 5/22 toaster: widgets ToasterTable Add more info to search field exception
In reply to brian averyFrom: Michael Wood <[email protected]>
If we get a search fields exception then also print out the model name
--- lib/toaster/toastergui/widgets.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/toaster/toastergui/widgets.py +2 -1
@@ -229,7 +229,8 @@ class ToasterTable(TemplateView): """Creates a query based on the model's search_allowed_fields""" if not hasattr(self.queryset.model, 'search_allowed_fields'): - raise Exception("Err Search fields aren't defined in the model") + raise Exception("Search fields aren't defined in the model %s" + % self.queryset.model) search_queries = [] for st in search_term.split(" "):
Signature · 1 line
1.9.1
PATCH 6/22 toaster: add nocache option to the ToasterTable widget
In reply to brian averyFrom: Michael Wood <[email protected]>
Useful for skipping the cache mechanism when debugging. Simply append nocache=true to the end of the url with the table in it.
Also adds a debug message if the table is using cached data.
--- lib/toaster/toastergui/widgets.py | 7 +++++++ 1 file changed, 7 insertions(+)
lib/toaster/toastergui/widgets.py +7 -0
@@ -253,11 +253,14 @@ class ToasterTable(TemplateView): search = request.GET.get("search", None) filters = request.GET.get("filter", None) orderby = request.GET.get("orderby", None) + nocache = request.GET.get("nocache", None) # Make a unique cache name cache_name = self.__class__.__name__ for key, val in request.GET.iteritems(): + if key == 'nocache': + continue cache_name = cache_name + str(key) + str(val) for key, val in kwargs.iteritems():
@@ -265,6 +268,10 @@ class ToasterTable(TemplateView): # No special chars allowed in the cache name apart from dash cache_name = re.sub(r'[^A-Za-z0-9-]', "", cache_name) + + if nocache: + cache.delete(cache_name) + data = cache.get(cache_name) if data:
Signature · 1 line
1.9.1
PATCH 7/22 toaster: ToasterTable remove unused class definition
In reply to brian averyFrom: Michael Wood <[email protected]>
Remove ToasterTemplateView as this isn't used by anything.
--- lib/toaster/toastergui/widgets.py | 27 --------------------------- 1 file changed, 27 deletions(-)
lib/toaster/toastergui/widgets.py +0 -27
@@ -349,33 +349,6 @@ class ToasterTable(TemplateView): return data -class ToasterTemplateView(TemplateView): - # renders a instance in a template, or returns the context as json - # the class-equivalent of the _template_renderer decorator for views - - def __init__(self, *args, **kwargs): - super(ToasterTemplateView, self).__init__(*args, **kwargs) - self.context_entries = [] - - def get(self, *args, **kwargs): - if self.request.GET.get('format', None) == 'json': - from django.core.urlresolvers import reverse - from django.shortcuts import HttpResponse - from views import objtojson - from toastergui.templatetags.projecttags import json as jsonfilter - - context = self.get_context_data(**kwargs) - - for x in context.keys(): - if x not in self.context_entries: - del context[x] - - context["error"] = "ok" - - return HttpResponse(jsonfilter(context, default=objtojson ), - content_type = "application/json; charset=utf-8") - - return super(ToasterTemplateView, self).get(*args, **kwargs) class ToasterTypeAhead(View): """ A typeahead mechanism to support the front end typeahead widgets """
Signature · 1 line
1.9.1
PATCH 8/22 toaster: Add CustomImageRecipe model
In reply to brian averyFrom: Michael Wood <[email protected]>
This model lists custom image recipes for the project. It is populated when new custom image is created. It holds reference to the base recipe and list of packages included into custom image.
For CustomImageRecipes the packages will be copied in and therefore not associated with a build so Remove the requirement for the package to have a Build.
co-author: Ed Bartosh <[email protected]>
--- ...e__add_unique_customimagerecipe_name_project.py | 375 +++++++++++++++++++++ lib/toaster/orm/models.py | 11 +- 2 files changed, 385 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py
lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py new file +375 -0
@@ -0,0 +1,375 @@ +# -*- coding: utf-8 -*- +from south.utils import datetime_utils as datetime +from south.db import db +from south.v2 import SchemaMigration +from django.db import models + + +class Migration(SchemaMigration): + + def forwards(self, orm): + # Adding model 'CustomImageRecipe' + db.create_table(u'orm_customimagerecipe', ( + (u'id', self.gf('django.db.models.fields.AutoField')(primary_key=True)), + ('name', self.gf('django.db.models.fields.CharField')(max_length=100)), + ('base_recipe', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['orm.Recipe'])), + ('project', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['orm.Project'])), + )) + db.send_create_signal(u'orm', ['CustomImageRecipe']) + + # Adding M2M table for field packages on 'CustomImageRecipe' + m2m_table_name = db.shorten_name(u'orm_customimagerecipe_packages') + db.create_table(m2m_table_name, ( + ('id', models.AutoField(verbose_name='ID', primary_key=True, auto_created=True)), + ('customimagerecipe', models.ForeignKey(orm[u'orm.customimagerecipe'], null=False)), + ('package', models.ForeignKey(orm[u'orm.package'], null=False)) + )) + db.create_unique(m2m_table_name, ['customimagerecipe_id', 'package_id']) + + # Adding unique constraint on 'CustomImageRecipe', fields ['name', 'project'] + db.create_unique(u'orm_customimagerecipe', ['name', 'project_id']) + + + # Changing field 'Package.build' + db.alter_column(u'orm_package', 'build_id', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['orm.Build'], null=True)) + + def backwards(self, orm): + # Removing unique constraint on 'CustomImageRecipe', fields ['name', 'project'] + db.delete_unique(u'orm_customimagerecipe', ['name', 'project_id']) + + # Deleting model 'CustomImageRecipe' + db.delete_table(u'orm_customimagerecipe') + + # Removing M2M table for field packages on 'CustomImageRecipe' + db.delete_table(db.shorten_name(u'orm_customimagerecipe_packages')) + + + # Changing field 'Package.build' + db.alter_column(u'orm_package', 'build_id', self.gf('django.db.models.fields.related.ForeignKey')(default=None, to=orm['orm.Build'])) + + models = { + u'orm.bitbakeversion': { + 'Meta': {'object_name': 'BitbakeVersion'}, + 'branch': ('django.db.models.fields.CharField', [], {'max_length': '32'}), + 'dirpath': ('django.db.models.fields.CharField', [], {'max_length': '255'}), + 'giturl': ('django.db.models.fields.URLField', [], {'max_length': '200'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '32'}) + }, + u'orm.branch': { + 'Meta': {'unique_together': "(('layer_source', 'name'), ('layer_source', 'up_id'))", 'object_name': 'Branch'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'True', 'to': u"orm['orm.LayerSource']", 'null': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '50'}), + 'short_description': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'}), + 'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}), + 'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}) + }, + u'orm.build': { + 'Meta': {'object_name': 'Build'}, + 'bitbake_version': ('django.db.models.fields.CharField', [], {'max_length': '50'}), + 'build_name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'completed_on': ('django.db.models.fields.DateTimeField', [], {}), + 'cooker_log_path': ('django.db.models.fields.CharField', [], {'max_length': '500'}), + 'distro': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'distro_version': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'machine': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'outcome': ('django.db.models.fields.IntegerField', [], {'default': '2'}), + 'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}), + 'started_on': ('django.db.models.fields.DateTimeField', [], {}) + }, + u'orm.buildartifact': { + 'Meta': {'object_name': 'BuildArtifact'}, + 'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']"}), + 'file_name': ('django.db.models.fields.FilePathField', [], {'max_length': '100'}), + 'file_size': ('django.db.models.fields.IntegerField', [], {}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}) + }, + u'orm.customimagerecipe': { + 'Meta': {'unique_together': "(('name', 'project'),)", 'object_name': 'CustomImageRecipe'}, + 'base_recipe': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Recipe']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'packages': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['orm.Package']", 'symmetrical': 'False'}), + 'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}) + }, + u'orm.helptext': { + 'Meta': {'object_name': 'HelpText'}, + 'area': ('django.db.models.fields.IntegerField', [], {}), + 'build': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'helptext_build'", 'to': u"orm['orm.Build']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'key': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'text': ('django.db.models.fields.TextField', [], {}) + }, + u'orm.layer': { + 'Meta': {'unique_together': "(('layer_source', 'up_id'), ('layer_source', 'name'))", 'object_name': 'Layer'}, + 'description': ('django.db.models.fields.TextField', [], {'default': 'None', 'null': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer_index_url': ('django.db.models.fields.URLField', [], {'max_length': '200'}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'summary': ('django.db.models.fields.TextField', [], {'default': 'None', 'null': 'True'}), + 'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}), + 'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}), + 'vcs_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}), + 'vcs_web_file_base_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}), + 'vcs_web_tree_base_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}), + 'vcs_web_url': ('django.db.models.fields.URLField', [], {'default': 'None', 'max_length': '200', 'null': 'True'}) + }, + u'orm.layer_version': { + 'Meta': {'unique_together': "(('layer_source', 'up_id'),)", 'object_name': 'Layer_Version'}, + 'branch': ('django.db.models.fields.CharField', [], {'max_length': '80'}), + 'build': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'related_name': "'layer_version_build'", 'null': 'True', 'to': u"orm['orm.Build']"}), + 'commit': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'dirpath': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '255', 'null': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'layer_version_layer'", 'to': u"orm['orm.Layer']"}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}), + 'local_path': ('django.db.models.fields.FilePathField', [], {'default': "'/'", 'max_length': '1024'}), + 'priority': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'project': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.Project']", 'null': 'True'}), + 'up_branch': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.Branch']", 'null': 'True'}), + 'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}), + 'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}) + }, + u'orm.layersource': { + 'Meta': {'unique_together': "(('sourcetype', 'apiurl'),)", 'object_name': 'LayerSource'}, + 'apiurl': ('django.db.models.fields.CharField', [], {'default': 'None', 'max_length': '255', 'null': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '63'}), + 'sourcetype': ('django.db.models.fields.IntegerField', [], {}) + }, + u'orm.layerversiondependency': { + 'Meta': {'unique_together': "(('layer_source', 'up_id'),)", 'object_name': 'LayerVersionDependency'}, + 'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'dependees'", 'to': u"orm['orm.Layer_Version']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}), + 'layer_version': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'dependencies'", 'to': u"orm['orm.Layer_Version']"}), + 'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}) + }, + u'orm.logmessage': { + 'Meta': {'object_name': 'LogMessage'}, + 'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'level': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'lineno': ('django.db.models.fields.IntegerField', [], {'null': 'True'}), + 'message': ('django.db.models.fields.CharField', [], {'max_length': '240'}), + 'pathname': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}), + 'task': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Task']", 'null': 'True', 'blank': 'True'}) + }, + u'orm.machine': { + 'Meta': {'unique_together': "(('layer_source', 'up_id'),)", 'object_name': 'Machine'}, + 'description': ('django.db.models.fields.CharField', [], {'max_length': '255'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}), + 'layer_version': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Layer_Version']"}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '255'}), + 'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}), + 'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}) + }, + u'orm.package': { + 'Meta': {'object_name': 'Package'}, + 'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']", 'null': 'True'}), + 'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'installed_name': ('django.db.models.fields.CharField', [], {'default': "''", 'max_length': '100'}), + 'installed_size': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'license': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'recipe': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Recipe']", 'null': 'True'}), + 'revision': ('django.db.models.fields.CharField', [], {'max_length': '32', 'blank': 'True'}), + 'section': ('django.db.models.fields.CharField', [], {'max_length': '80', 'blank': 'True'}), + 'size': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'summary': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'version': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}) + }, + u'orm.package_dependency': { + 'Meta': {'object_name': 'Package_Dependency'}, + 'dep_type': ('django.db.models.fields.IntegerField', [], {}), + 'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'package_dependencies_target'", 'to': u"orm['orm.Package']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'package': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'package_dependencies_source'", 'to': u"orm['orm.Package']"}), + 'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']", 'null': 'True'}) + }, + u'orm.package_file': { + 'Meta': {'object_name': 'Package_File'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'package': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'buildfilelist_package'", 'to': u"orm['orm.Package']"}), + 'path': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}), + 'size': ('django.db.models.fields.IntegerField', [], {}) + }, + u'orm.project': { + 'Meta': {'object_name': 'Project'}, + 'bitbake_version': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.BitbakeVersion']", 'null': 'True'}), + 'created': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'is_default': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'release': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Release']", 'null': 'True'}), + 'short_description': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'}), + 'updated': ('django.db.models.fields.DateTimeField', [], {'auto_now': 'True', 'blank': 'True'}), + 'user_id': ('django.db.models.fields.IntegerField', [], {'null': 'True'}) + }, + u'orm.projectlayer': { + 'Meta': {'unique_together': "(('project', 'layercommit'),)", 'object_name': 'ProjectLayer'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layercommit': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Layer_Version']", 'null': 'True'}), + 'optional': ('django.db.models.fields.BooleanField', [], {'default': 'True'}), + 'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}) + }, + u'orm.projecttarget': { + 'Meta': {'object_name': 'ProjectTarget'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}), + 'target': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'task': ('django.db.models.fields.CharField', [], {'max_length': '100', 'null': 'True'}) + }, + u'orm.projectvariable': { + 'Meta': {'object_name': 'ProjectVariable'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'project': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Project']"}), + 'value': ('django.db.models.fields.TextField', [], {'blank': 'True'}) + }, + u'orm.recipe': { + 'Meta': {'unique_together': "(('layer_version', 'file_path', 'pathflags'),)", 'object_name': 'Recipe'}, + 'bugtracker': ('django.db.models.fields.URLField', [], {'max_length': '200', 'blank': 'True'}), + 'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'file_path': ('django.db.models.fields.FilePathField', [], {'max_length': '255'}), + 'homepage': ('django.db.models.fields.URLField', [], {'max_length': '200', 'blank': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'is_image': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': u"orm['orm.LayerSource']", 'null': 'True'}), + 'layer_version': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'recipe_layer_version'", 'to': u"orm['orm.Layer_Version']"}), + 'license': ('django.db.models.fields.CharField', [], {'max_length': '200', 'blank': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}), + 'pathflags': ('django.db.models.fields.CharField', [], {'max_length': '200', 'blank': 'True'}), + 'section': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}), + 'summary': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'up_date': ('django.db.models.fields.DateTimeField', [], {'default': 'None', 'null': 'True'}), + 'up_id': ('django.db.models.fields.IntegerField', [], {'default': 'None', 'null': 'True'}), + 'version': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}) + }, + u'orm.recipe_dependency': { + 'Meta': {'object_name': 'Recipe_Dependency'}, + 'dep_type': ('django.db.models.fields.IntegerField', [], {}), + 'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'r_dependencies_depends'", 'to': u"orm['orm.Recipe']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'recipe': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'r_dependencies_recipe'", 'to': u"orm['orm.Recipe']"}) + }, + u'orm.release': { + 'Meta': {'object_name': 'Release'}, + 'bitbake_version': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.BitbakeVersion']"}), + 'branch_name': ('django.db.models.fields.CharField', [], {'default': "''", 'max_length': '50'}), + 'description': ('django.db.models.fields.CharField', [], {'max_length': '255'}), + 'helptext': ('django.db.models.fields.TextField', [], {'null': 'True'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '32'}) + }, + u'orm.releasedefaultlayer': { + 'Meta': {'object_name': 'ReleaseDefaultLayer'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer_name': ('django.db.models.fields.CharField', [], {'default': "''", 'max_length': '100'}), + 'release': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Release']"}) + }, + u'orm.releaselayersourcepriority': { + 'Meta': {'unique_together': "(('release', 'layer_source'),)", 'object_name': 'ReleaseLayerSourcePriority'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'layer_source': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.LayerSource']"}), + 'priority': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'release': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Release']"}) + }, + u'orm.target': { + 'Meta': {'object_name': 'Target'}, + 'build': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Build']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'image_size': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'is_image': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'license_manifest_path': ('django.db.models.fields.CharField', [], {'max_length': '500', 'null': 'True'}), + 'target': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'task': ('django.db.models.fields.CharField', [], {'max_length': '100', 'null': 'True'}) + }, + u'orm.target_file': { + 'Meta': {'object_name': 'Target_File'}, + 'directory': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'directory_set'", 'null': 'True', 'to': u"orm['orm.Target_File']"}), + 'group': ('django.db.models.fields.CharField', [], {'max_length': '128'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'inodetype': ('django.db.models.fields.IntegerField', [], {}), + 'owner': ('django.db.models.fields.CharField', [], {'max_length': '128'}), + 'path': ('django.db.models.fields.FilePathField', [], {'max_length': '100'}), + 'permission': ('django.db.models.fields.CharField', [], {'max_length': '16'}), + 'size': ('django.db.models.fields.IntegerField', [], {}), + 'sym_target': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'symlink_set'", 'null': 'True', 'to': u"orm['orm.Target_File']"}), + 'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']"}) + }, + u'orm.target_image_file': { + 'Meta': {'object_name': 'Target_Image_File'}, + 'file_name': ('django.db.models.fields.FilePathField', [], {'max_length': '254'}), + 'file_size': ('django.db.models.fields.IntegerField', [], {}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']"}) + }, + u'orm.target_installed_package': { + 'Meta': {'object_name': 'Target_Installed_Package'}, + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'package': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'buildtargetlist_package'", 'to': u"orm['orm.Package']"}), + 'target': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['orm.Target']"}) + }, + u'orm.task': { + 'Meta': {'ordering': "('order', 'recipe')", 'unique_together': "(('build', 'recipe', 'task_name'),)", 'object_name': 'Task'}, + 'build': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'task_build'", 'to': u"orm['orm.Build']"}), + 'cpu_usage': ('django.db.models.fields.DecimalField', [], {'null': 'True', 'max_digits': '8', 'decimal_places': '2'}), + 'disk_io': ('django.db.models.fields.IntegerField', [], {'null': 'True'}), + 'elapsed_time': ('django.db.models.fields.DecimalField', [], {'null': 'True', 'max_digits': '8', 'decimal_places': '2'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'line_number': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'logfile': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}), + 'message': ('django.db.models.fields.CharField', [], {'max_length': '240'}), + 'order': ('django.db.models.fields.IntegerField', [], {'null': 'True'}), + 'outcome': ('django.db.models.fields.IntegerField', [], {'default': '-1'}), + 'path_to_sstate_obj': ('django.db.models.fields.FilePathField', [], {'max_length': '500', 'blank': 'True'}), + 'recipe': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'tasks'", 'to': u"orm['orm.Recipe']"}), + 'script_type': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'source_url': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}), + 'sstate_checksum': ('django.db.models.fields.CharField', [], {'max_length': '100', 'blank': 'True'}), + 'sstate_result': ('django.db.models.fields.IntegerField', [], {'default': '0'}), + 'task_executed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'task_name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'work_directory': ('django.db.models.fields.FilePathField', [], {'max_length': '255', 'blank': 'True'}) + }, + u'orm.task_dependency': { + 'Meta': {'object_name': 'Task_Dependency'}, + 'depends_on': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'task_dependencies_depends'", 'to': u"orm['orm.Task']"}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'task': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'task_dependencies_task'", 'to': u"orm['orm.Task']"}) + }, + u'orm.toastersetting': { + 'Meta': {'object_name': 'ToasterSetting'}, + 'helptext': ('django.db.models.fields.TextField', [], {}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'name': ('django.db.models.fields.CharField', [], {'max_length': '63'}), + 'value': ('django.db.models.fields.CharField', [], {'max_length': '255'}) + }, + u'orm.variable': { + 'Meta': {'object_name': 'Variable'}, + 'build': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'variable_build'", 'to': u"orm['orm.Build']"}), + 'changed': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), + 'description': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'human_readable_name': ('django.db.models.fields.CharField', [], {'max_length': '200'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'variable_name': ('django.db.models.fields.CharField', [], {'max_length': '100'}), + 'variable_value': ('django.db.models.fields.TextField', [], {'blank': 'True'}) + }, + u'orm.variablehistory': { + 'Meta': {'object_name': 'VariableHistory'}, + 'file_name': ('django.db.models.fields.FilePathField', [], {'max_length': '255'}), + u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), + 'line_number': ('django.db.models.fields.IntegerField', [], {'null': 'True'}), + 'operation': ('django.db.models.fields.CharField', [], {'max_length': '64'}), + 'value': ('django.db.models.fields.TextField', [], {'blank': 'True'}), + 'variable': ('django.db.models.fields.related.ForeignKey', [], {'related_name': "'vhistory'", 'to': u"orm['orm.Variable']"}) + } + } + + complete_apps = ['orm']
lib/toaster/orm/models.py +10 -1
@@ -551,7 +551,7 @@ class Task_Dependency(models.Model): class Package(models.Model): search_allowed_fields = ['name', 'version', 'revision', 'recipe__name', 'recipe__version', 'recipe__license', 'recipe__layer_version__layer__name', 'recipe__layer_version__branch', 'recipe__layer_version__commit', 'recipe__layer_version__local_path', 'installed_name'] - build = models.ForeignKey('Build') + build = models.ForeignKey('Build', null=True) recipe = models.ForeignKey('Recipe', null=True) name = models.CharField(max_length=100) installed_name = models.CharField(max_length=100, default='')
@@ -1172,6 +1172,15 @@ class ProjectLayer(models.Model): class Meta: unique_together = (("project", "layercommit"),) +class CustomImageRecipe(models.Model): + name = models.CharField(max_length=100) + base_recipe = models.ForeignKey(Recipe) + packages = models.ManyToManyField(Package) + project = models.ForeignKey(Project) + + class Meta: + unique_together = ("name", "project") + class ProjectVariable(models.Model): project = models.ForeignKey(Project) name = models.CharField(max_length=100)
Signature · 1 line
1.9.1
PATCH 9/22 toaster: add toggle for enabling image customisation feeature
In reply to brian averyFrom: Michael Wood <[email protected]>
This feature is currently under heavy development and should be used with caution.
--- lib/toaster/toastergui/views.py | 1 + lib/toaster/toastermain/settings.py | 5 +++++ 2 files changed, 6 insertions(+)
lib/toaster/toastergui/views.py +1 -0
@@ -1867,6 +1867,7 @@ def managedcontextprocessor(request): ret = { "projects": Project.objects.all(), "DEBUG" : toastermain.settings.DEBUG, + "CUSTOM_IMAGE" : toastermain.settings.CUSTOM_IMAGE, "TOASTER_BRANCH": toastermain.settings.TOASTER_BRANCH, "TOASTER_REVISION" : toastermain.settings.TOASTER_REVISION, }
lib/toaster/toastermain/settings.py +5 -0
@@ -23,6 +23,11 @@ import os, re +# Temporary toggle for Image customisation +CUSTOM_IMAGE = False +if os.environ.get("CUSTOM_IMAGE", None) is not None: + CUSTOM_IMAGE = True + DEBUG = True TEMPLATE_DEBUG = DEBUG
Signature · 1 line
1.9.1
PATCH 10/22 toaster: implement decorator for REST responses
In reply to brian averyFrom: Ed Bartosh <[email protected]>
Implemented xhr_response decorator to decorate responses from REST methods into Django HttpResponse objects.
This decorator should reduce amount of repeated code in REST methods and make them more readable.
--- lib/toaster/toastergui/views.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)
lib/toaster/toastergui/views.py +13 -0
@@ -45,6 +45,7 @@ from django.utils import formats from toastergui.templatetags.projecttags import json as jsonfilter import json from os.path import dirname +from functools import wraps import itertools import magic
@@ -2314,6 +2315,18 @@ if True: return context + def xhr_response(fun): + """ + Decorator for REST methods. + calls jsonfilter on the returned dictionary and returns result + as HttpResponse object of content_type application/json + """ + @wraps(fun) + def wrapper(*args, **kwds): + return HttpResponse(jsonfilter(fun(*args, **kwds)), + content_type="application/json") + return wrapper + def jsunittests(request): """ Provides a page for the js unit tests """ bbv = BitbakeVersion.objects.filter(branch="master").first()
Signature · 1 line
1.9.1
PATCH 11/22 toaster: Fix indentation of jsunittests view
In reply to brian averyFrom: Michael Wood <[email protected]>
Fix indentation to 4 spaces
--- lib/toaster/toastergui/views.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
lib/toaster/toastergui/views.py +10 -10
@@ -2328,20 +2328,20 @@ if True: return wrapper def jsunittests(request): - """ Provides a page for the js unit tests """ - bbv = BitbakeVersion.objects.filter(branch="master").first() - release = Release.objects.filter(bitbake_version=bbv).first() + """ Provides a page for the js unit tests """ + bbv = BitbakeVersion.objects.filter(branch="master").first() + release = Release.objects.filter(bitbake_version=bbv).first() - name = "_js_unit_test_prj_" + name = "_js_unit_test_prj_" - # If there is an existing project by this name delete it. We don't want - # Lots of duplicates cluttering up the projects. - Project.objects.filter(name=name).delete() + # If there is an existing project by this name delete it. We don't want + # Lots of duplicates cluttering up the projects. + Project.objects.filter(name=name).delete() - new_project = Project.objects.create_project(name=name, release=release) + new_project = Project.objects.create_project(name=name, release=release) - context = { 'project' : new_project } - return render(request, "js-unit-tests.html", context) + context = { 'project' : new_project } + return render(request, "js-unit-tests.html", context) from django.views.decorators.csrf import csrf_exempt @csrf_exempt
Signature · 1 line
1.9.1
PATCH 12/22 toaster: Add new ReST API for Image Customisation feature
In reply to brian averyFrom: Ed Bartosh <[email protected]>
Implemented xhr_customrecipe API. To create a custom recipe from a base recipe.
Implemented xhr_customrecipe_packages API to add/remove packages to/from custom recipe.
co-authored see Signed-off-by
--- lib/toaster/toastergui/urls.py | 10 ++- lib/toaster/toastergui/views.py | 152 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 159 insertions(+), 3 deletions(-)
lib/toaster/toastergui/urls.py +9 -1
@@ -152,6 +152,14 @@ urlpatterns = patterns('toastergui.views', # JS Unit tests url(r'^js-unit-tests/$', 'jsunittests', name='js-unit-tests'), - # default redirection + # image customisation functionality + url(r'^xhr_customrecipe/(?P<recipe_id>\d+)/packages/(?P<package_id>\d+|)$', + 'xhr_customrecipe_packages', name='xhr_customrecipe_packages'), + url(r'^xhr_customrecipe/(?P<recipe_id>\d+)$', 'xhr_customrecipe_id', + name='xhr_customrecipe_id'), + url(r'^xhr_customrecipe/', 'xhr_customrecipe', + name='xhr_customrecipe'), + + # default redirection url(r'^$', RedirectView.as_view( url= 'landing')), )
lib/toaster/toastergui/views.py +150 -2
169 lines not on this page · raw message
Signature · 1 line
1.9.1
PATCH 13/22 toaster: Add ToasterTables for Image customisation feature
In reply to brian averyFrom: Michael Wood <[email protected]>
- Split up the recipes tables into Image recipes and Software Recipes
- Add CustomImageRecipe table
- Add SelectPackagesTable table
- Add NewCustomImagesTable table
--- lib/toaster/toastergui/tables.py | 222 +++++++++++++++++++++++++++++++++------ 1 file changed, 188 insertions(+), 34 deletions(-)
lib/toaster/toastergui/tables.py +188 -34
@@ -21,6 +21,7 @@ from toastergui.widgets import ToasterTable from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project +from orm.models import CustomImageRecipe, Package from django.db.models import Q, Max from django.conf.urls import url from django.core.urlresolvers import reverse
@@ -310,13 +311,20 @@ class LayerMachinesTable(MachinesTable): class RecipesTable(ToasterTable, ProjectFiltersMixin): - """Table of Recipes in Toaster""" + """Table of All Recipes in Toaster""" def __init__(self, *args, **kwargs): super(RecipesTable, self).__init__(*args, **kwargs) self.empty_state = "Toaster has no recipe information. To generate recipe information you can configure a layer source then run a build." self.default_orderby = "name" + build_col = { 'title' : "Build", + 'help_text' : "Add or delete recipes to and from your project", + 'hideable' : False, + 'filter_name' : "in_current_project", + 'static_data_name' : "add-del-layers", + 'static_data_template' : '{% include "recipe_btn.html" %}'} + def get_context_data(self, **kwargs): project = Project.objects.get(pk=kwargs['pid']) context = super(RecipesTable, self).get_context_data(**kwargs)
@@ -327,17 +335,6 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin): return context - def setup_filters(self, *args, **kwargs): - project = Project.objects.get(pk=kwargs['pid']) - self.project_layers = project.projectlayer_equivalent_set() - - self.add_filter(title="Filter by project recipes", - name="in_current_project", - filter_actions=[ - self.make_filter_action("in_project", "Recipes provided by layers added to this project", self.filter_in_project), - self.make_filter_action("not_in_project", "Recipes provided by layers not added to this project", self.filter_not_in_project) - ]) - def setup_queryset(self, *args, **kwargs): prj = Project.objects.get(pk = kwargs['pid'])
@@ -348,12 +345,6 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin): def setup_columns(self, *args, **kwargs): - self.add_column(title="Recipe", - help_text="Information about a single piece of software, including where to download the source, configuration options, how to compile the source files and how to package the compiled output", - hideable=False, - orderable=True, - field_name="name") - self.add_column(title="Recipe Version", hidden=True, field_name="version")
@@ -398,18 +389,6 @@ class RecipesTable(ToasterTable, ProjectFiltersMixin): self.add_column(title="Revision", field_name="layer_version__get_vcs_reference") - self.add_column(title="Build", - help_text="Add or delete recipes to and from your project", - hideable=False, - filter_name="in_current_project", - static_data_name="add-del-layers", - static_data_template='{% include "recipe_btn.html" %}') - - project = Project.objects.get(pk=kwargs['pid']) - self.add_column(title="Project compatible Layer ID", - displayable = False, - field_name = "projectcompatible_layer", - computation = lambda x: (x.layer_version.get_equivalents_wpriority(project)[0])) class LayerRecipesTable(RecipesTable): """ Smaller version of the Recipes table for use in layer details """
@@ -446,10 +425,185 @@ class LayerRecipesTable(RecipesTable): static_data_name="add-del-layers", static_data_template=build_recipe_template) -class ProjectLayersRecipesTable(RecipesTable): - """ Table that lists only recipes available for layers added to the project """ +class CustomImagesTable(ToasterTable): + """ Table to display your custom images """ + def __init__(self, *args, **kwargs): + super(CustomImagesTable, self).__init__(*args, **kwargs) + self.title = "Custom images" + + def get_context_data(self, **kwargs): + context = super(CustomImagesTable, self).get_context_data(**kwargs) + project = Project.objects.get(pk=kwargs['pid']) + context['project'] = project + context['projectlayers'] = map(lambda prjlayer: prjlayer.layercommit.id, ProjectLayer.objects.filter(project=context['project'])) + return context + + def setup_queryset(self, *args, **kwargs): + prj = Project.objects.get(pk = kwargs['pid']) + self.queryset = CustomImageRecipe.objects.filter(project=prj) + self.queryset = self.queryset.order_by('name') + + def setup_columns(self, *args, **kwargs): + + name_link_template = ''' + <a href="{% url 'customrecipe' extra.pid data.id %}"> + {{data.name}} + </a> + ''' + + self.add_column(title="Custom image", + hideable=False, + static_data_name="name", + static_data_template=name_link_template) + + self.add_column(title="Recipe file", + static_data_name='recipe_file', + static_data_template='') + + approx_packages_template = '<a href="#imagedetails">{{data.packages.all|length}}</a>' + self.add_column(title="Approx packages", + static_data_name='approx_packages', + static_data_template=approx_packages_template) + + + build_btn_template = '''<button data-recipe-name="{{data.name}}" + class="btn btn-block build-recipe-btn" style="margin-top: 5px;" > + Build</button>''' + + self.add_column(title="Build", + hideable=False, + static_data_name='build_custom_img', + static_data_template=build_btn_template) + +class ImageRecipesTable(RecipesTable): + """ A subset of the recipes table which displayed just image recipes """ + + def __init__(self, *args, **kwargs): + super(ImageRecipesTable, self).__init__(*args, **kwargs) + self.title = "Compatible image recipes" + + def setup_queryset(self, *args, **kwargs): + super(ImageRecipesTable, self).setup_queryset(*args, **kwargs) + + self.queryset = self.queryset.filter(is_image=True) + + + def setup_columns(self, *args, **kwargs): + self.add_column(title="Image recipe", + help_text="When you build an image recipe, you get an " + "image: a root file system you can" + "deploy to a machine", + hideable=False, + orderable=True, + field_name="name") + + super(ImageRecipesTable, self).setup_columns(*args, **kwargs) + + self.add_column(**RecipesTable.build_col) + + +class NewCustomImagesTable(ImageRecipesTable): + """ Table which displays Images recipes which can be customised """ + def __init__(self, *args, **kwargs): + super(NewCustomImagesTable, self).__init__(*args, **kwargs) + self.title = "Select the image recipe you want to customise" def setup_queryset(self, *args, **kwargs): - super(ProjectLayersRecipesTable, self).setup_queryset(*args, **kwargs) + super(ImageRecipesTable, self).setup_queryset(*args, **kwargs) + + self.queryset = self.queryset.filter(is_image=True) + + def setup_columns(self, *args, **kwargs): + self.add_column(title="Image recipe", + help_text="When you build an image recipe, you get an " + "image: a root file system you can" + "deploy to a machine", + hideable=False, + orderable=True, + field_name="recipe__name") + + super(ImageRecipesTable, self).setup_columns(*args, **kwargs) + + self.add_column(title="Customise", + hideable=False, + filter_name="in_current_project", + static_data_name="customise-or-add-recipe", + static_data_template='{% include "customise_btn.html" %}') + + +class SoftwareRecipesTable(RecipesTable): + """ Displays just the software recipes """ + def __init__(self, *args, **kwargs): + super(SoftwareRecipesTable, self).__init__(*args, **kwargs) + self.title = "Compatible software recipes" + + def setup_queryset(self, *args, **kwargs): + super(SoftwareRecipesTable, self).setup_queryset(*args, **kwargs) + + self.queryset = self.queryset.filter(is_image=False) + + + def setup_columns(self, *args, **kwargs): + self.add_column(title="Software recipe", + help_text="Information about a single piece of " + "software, including where to download the source, " + "configuration options, how to compile the source " + "files and how to package the compiled output", + hideable=False, + orderable=True, + field_name="name") + + super(SoftwareRecipesTable, self).setup_columns(*args, **kwargs) + + self.add_column(**RecipesTable.build_col) + + +class SelectPackagesTable(ToasterTable): + """ Table to display the packages to add and remove from an image """ + + def __init__(self, *args, **kwargs): + super(SelectPackagesTable, self).__init__(*args, **kwargs) + self.title = "Add | Remove packages" + + def setup_queryset(self, *args, **kwargs): + cust_recipe = CustomImageRecipe.objects.get(pk=kwargs['recipeid']) prj = Project.objects.get(pk = kwargs['pid']) - self.queryset = self.queryset.filter(layer_version__in = prj.projectlayer_equivalent_set()) + + current_packages = cust_recipe.packages.all() + + # Get all the packages that are in the custom image + # Get all the packages built by builds in the current project + # but not those ones that are already in the custom image + self.queryset = Package.objects.filter( + Q(pk__in=current_packages) | + (Q(build__project=prj) & + ~Q(name__in=current_packages.values_list('name')))) + + self.queryset = self.queryset.order_by('name') + + self.static_context_extra['recipe_id'] = kwargs['recipeid'] + self.static_context_extra['current_packages'] = \ + cust_recipe.packages.values_list('pk', flat=True) + + def setup_columns(self, *args, **kwargs): + self.add_column(title="Package", + hideable=False, + orderable=True, + field_name="name") + + self.add_column(title="Package Version", + field_name="version") + + self.add_column(title="Approx Size", + orderable=True, + static_data_name="size", + static_data_template="{% load projecttags %} \ + {{data.size|filtered_filesizeformat}}") + self.add_column(title="summary", + field_name="summary") + + self.add_column(title="Add | Remove", + help_text="Use the add and remove buttons to modify " + "the package content of you custom image", + static_data_name="add_rm_pkg_btn", + static_data_template='{% include "pkg_add_rm_btn.html" %}')
Signature · 1 line
1.9.1
PATCH 14/22 toaster: Add Image customisation frontend feature
In reply to brian averyFrom: Michael Wood <[email protected]>
Add the Image customisation front end feature to Toaster. Caveat - This feature is currently in development and should not be enabled by default.
--- lib/toaster/toastergui/static/js/customrecipe.js | 50 ++++++++ lib/toaster/toastergui/static/js/layerBtn.js | 13 ++ lib/toaster/toastergui/static/js/newcustomimage.js | 49 +++++++ .../toastergui/templates/baseprojectpage.html | 7 +- .../toastergui/templates/customise_btn.html | 9 ++ lib/toaster/toastergui/templates/customrecipe.html | 142 +++++++++++++++++++++ .../toastergui/templates/newcustomimage.html | 54 ++++++++ .../toastergui/templates/pkg_add_rm_btn.html | 16 +++ lib/toaster/toastergui/templates/project.html | 2 +- .../toastergui/templates/projecttopbar.html | 9 +- lib/toaster/toastergui/urls.py | 17 ++- lib/toaster/toastergui/views.py | 9 ++ 12 files changed, 368 insertions(+), 9 deletions(-) create mode 100644 bitbake/lib/toaster/toastergui/static/js/customrecipe.js create mode 100644 bitbake/lib/toaster/toastergui/static/js/newcustomimage.js create mode 100644 bitbake/lib/toaster/toastergui/templates/customise_btn.html create mode 100644 bitbake/lib/toaster/toastergui/templates/customrecipe.html create mode 100644 bitbake/lib/toaster/toastergui/templates/newcustomimage.html create mode 100644 bitbake/lib/toaster/toastergui/templates/pkg_add_rm_btn.html
lib/toaster/toastergui/static/js/customrecipe.js new file +50 -0
@@ -0,0 +1,50 @@ +"use strict"; + +function customRecipePageInit(ctx) { + + var urlParams = libtoaster.parseUrlParams(); + + (function notificationRequest(){ + if (urlParams.hasOwnProperty('notify') && urlParams.notify === 'new'){ + $("#image-created-notification").show(); + } + })(); + + $("#recipeselection").on('table-done', function(e, total, tableParams){ + /* Table is done so now setup the click handler for the package buttons */ + $(".add-rm-package-btn").click(function(e){ + e.preventDefault(); + addRemovePackage($(this), tableParams); + }); + }); + + function addRemovePackage(pkgBtn, tableParams){ + var pkgBtnData = pkgBtn.data(); + var method; + var buttonToShow; + + if (pkgBtnData.directive == 'add') { + method = 'PUT'; + buttonToShow = '#package-rm-btn-' + pkgBtnData.package; + } else if (pkgBtnData.directive == 'remove') { + method = 'DELETE'; + buttonToShow = '#package-add-btn-' + pkgBtnData.package; + } else { + throw("Unknown package directive: should be add or remove"); + } + + $.ajax({ + type: method, + url: pkgBtnData.packageUrl, + headers: { 'X-CSRFToken' : $.cookie('csrftoken')}, + success: function(data){ + /* Invalidate the Add | Rm package table's current cache */ + tableParams.nocache = true; + $.get(ctx.tableApiUrl, tableParams); + /* Swap the buttons around */ + pkgBtn.hide(); + $(buttonToShow).show(); + } + }); + } +}
lib/toaster/toastergui/static/js/layerBtn.js +13 -0
@@ -68,6 +68,19 @@ function layerBtnsInit(ctx) { }); }); + + $(".customise-btn").unbind('click'); + $(".customise-btn").click(function(e){ + e.preventDefault(); + var imgCustomModal = $("#new-custom-image-modal"); + + if (imgCustomModal.length == 0) + throw("Modal new-custom-image not found"); + + imgCustomModal.data('recipe', $(this).data('recipe')); + imgCustomModal.modal('show'); + }); + /* Setup the initial state of the buttons */ for (var i in ctx.projectLayers){
lib/toaster/toastergui/static/js/newcustomimage.js new file +49 -0
49 lines not on this page · raw message
lib/toaster/toastergui/templates/baseprojectpage.html +5 -2
@@ -23,8 +23,11 @@ <ul class="nav nav-list well"> <li><a class="nav-parent" href="{% url 'project' project.id %}">Configuration</a></li> <li class="nav-header">Compatible metadata</li> -<!-- <li><a href="all-image-recipes.html">Image recipes</a></li> --> - <li><a href="{% url 'projecttargets' project.id %}">Recipes</a></li> + {% if CUSTOM_IMAGE %} + <li><a href="{% url 'projectcustomimages' project.id %}">Custom images</a></li> + {% endif %} + <li><a href="{% url 'projectimagerecipes' project.id %}">Image recipes</a></li> + <li><a href="{% url 'projectsoftwarerecipes' project.id %}">Software recipes</a></li> <li><a href="{% url 'projectmachines' project.id %}">Machines</a></li> <li><a href="{% url 'projectlayers' project.id %}">Layers</a></li> <li class="nav-header">Extra configuration</li>
lib/toaster/toastergui/templates/customise_btn.html new file +9 -0
@@ -0,0 +1,9 @@ +<button class="btn btn-block layer-exists-{{data.layer_version.id}} customise-btn" style="display:none;" data-recipe="{{data.id}}"> + Customise +</button> + +<button class="btn btn-block layer-add-{{data.layer_version.id}} layerbtn" data-layer='{ "id": {{data.layer_version.id}}, "name": "{{data.layer_version.layer.name}}", "layerdetailurl": "{% url 'layerdetails' extra.pid data.layer_version.id %}"}' data-directive="add"> + <i class="icon-plus"></i> + Add layer +</button> +
lib/toaster/toastergui/templates/customrecipe.html new file +142 -0
142 lines not on this page · raw message
lib/toaster/toastergui/templates/newcustomimage.html new file +54 -0
54 lines not on this page · raw message
lib/toaster/toastergui/templates/pkg_add_rm_btn.html new file +16 -0
@@ -0,0 +1,16 @@ +<button class="btn btn-block btn-danger add-rm-package-btn" id="package-rm-btn-{{data.pk}}" data-directive="remove" data-package="{{data.pk}}" data-package-url="{% url 'xhr_customrecipe_packages' extra.recipe_id data.pk %}" style=" + {% if data.pk not in extra.current_packages %} + display:none + {% endif %} + "> + <i class="icon-trash no-tooltip"></i> + Remove package +</a> +<button class="btn btn-block add-rm-package-btn" data-directive="add" id="package-add-btn-{{data.pk}}" data-package="{{data.pk}}" data-package-url="{% url 'xhr_customrecipe_packages' extra.recipe_id data.pk %}" style=" + {% if data.pk in extra.current_packages %} + display:none + {% endif %} + "> +<i class="icon-plus"></i> + Add package +</button>
lib/toaster/toastergui/templates/project.html +1 -1
@@ -67,7 +67,7 @@ <div class="alert alert-info" style="display:none" id="no-most-built"> <span class="lead">You haven't built any recipes yet</span> - <p style="margin-top: 10px;"><a href="{% url 'projecttargets' project.id %}">Choose a recipe to build</a></p> + <p style="margin-top: 10px;"><a href="{% url 'projectsoftwarerecipes' project.id %}">Choose a recipe to build</a></p> </div> <ul class="unstyled configuration-list" id="freq-build-list">
lib/toaster/toastergui/templates/projecttopbar.html +8 -1
@@ -1,6 +1,6 @@ <div class="alert alert-success lead" id="project-created-notification" style="margin-top:15px; display:none"> <button type="button" class="close" data-dismiss="alert">×</button> - Your project <strong>{{project.name}}</strong> has been created. You can now <a href="{% url 'projectmachines' project.id %}">select your target machine</a> and <a href="{% url 'projecttargets' project.id %}">choose image recipes</a> to build. + Your project <strong>{{project.name}}</strong> has been created. You can now <a href="{% url 'projectmachines' project.id %}">select your target machine</a> and <a href="{% url 'projectsoftwarerecipes' project.id %}">choose image recipes</a> to build. </div> <!-- project name -->
@@ -34,6 +34,13 @@ Import layer </a> </li> + {% if CUSTOM_IMAGE %} + <li> + <a href="{% url 'newcustomimage' project.id %}"> + New custom image + </a> + </li> + {% endif %} <li class="pull-right"> <form class="form-inline" style="margin-bottom:0px;"> <i class="icon-question-sign get-help heading-help" data-placement="left" title="" data-original-title="Type the name of one or more recipes you want to build, separated by a space. You can also specify a task by appending a semicolon and a task name to the recipe name, like so: <code>busybox:clean</code>"></i>
lib/toaster/toastergui/urls.py +12 -5
@@ -103,16 +103,13 @@ urlpatterns = patterns('toastergui.views', tables.NewCustomImagesTable.as_view(template_name="newcustomimage.html"), name="newcustomimage"), - url(r'^project/(?P<pid>\d+)/availablerecipes/$', - tables.ProjectLayersRecipesTable.as_view(template_name="generic-toastertable-page.html"), - { 'table_name': tables.ProjectLayersRecipesTable.__name__.lower(), - 'title' : 'Recipes available for layers in the current project' }, - name="projectavailabletargets"), url(r'^project/(?P<pid>\d+)/layers/$', tables.LayersTable.as_view(template_name="generic-toastertable-page.html"), name="projectlayers"), + + url(r'^project/(?P<pid>\d+)/layer/(?P<layerid>\d+)$', 'layerdetails', name='layerdetails'),
@@ -129,6 +126,16 @@ urlpatterns = patterns('toastergui.views', name=tables.LayerMachinesTable.__name__.lower()), + url(r'^project/(?P<pid>\d+)/customrecipe/(?P<recipeid>\d+)/selectpackages/$', + tables.SelectPackagesTable.as_view(template_name="generic-toastertable-page.html"), name="recipeselectpackages"), + + + url(r'^project/(?P<pid>\d+)/customrecipe/(?P<recipe_id>\d+)$', + 'customrecipe', + name="customrecipe"), + + + # typeahead api end points url(r'^xhr_typeahead/(?P<pid>\d+)/layers$', typeaheads.LayersTypeAhead.as_view(), name='xhr_layerstypeahead'),
lib/toaster/toastergui/views.py +9 -0
@@ -2790,6 +2790,15 @@ if True: return(vars_managed,sorted(vars_fstypes),vars_blacklist) + def customrecipe(request, pid, recipe_id): + project = Project.objects.get(pk=pid) + context = {'project' : project, + 'projectlayers': [], + 'recipe' : CustomImageRecipe.objects.get(pk=recipe_id) + } + + return render(request, "customrecipe.html", context) + @_template_renderer("projectconf.html") def projectconf(request, pid):
Signature · 1 line
1.9.1
PATCH 15/22 toaster: Add test cases for new Image customisation features
In reply to brian averyFrom: Ed Bartosh <[email protected]>
- Adds tests for new ToasterTables
- Adds tests for new ReST API
co-author: Elliot Smith <[email protected]>
Michael Wood <[email protected]>
--- lib/toaster/toastergui/tests.py | 187 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 182 insertions(+), 5 deletions(-)
lib/toaster/toastergui/tests.py +182 -5
@@ -24,13 +24,21 @@ import re from django.test import TestCase +from django.test.client import RequestFactory from django.core.urlresolvers import reverse from django.utils import timezone -from orm.models import Project, Release, BitbakeVersion, ProjectTarget + +from orm.models import Project, Release, BitbakeVersion, Build, Package from orm.models import ReleaseLayerSourcePriority, LayerSource, Layer, Build from orm.models import Layer_Version, Recipe, Machine, ProjectLayer, Target +from orm.models import CustomImageRecipe +from orm.models import Branch + +from toastergui.tables import SoftwareRecipesTable +from django.utils import timezone import json from bs4 import BeautifulSoup +import re PROJECT_NAME = "test project"
@@ -41,27 +49,58 @@ class ViewTests(TestCase): bbv = BitbakeVersion.objects.create(name="test bbv", giturl="/tmp/", branch="master", dirpath="") release = Release.objects.create(name="test release", + branch_name="master", bitbake_version=bbv) self.project = Project.objects.create_project(name=PROJECT_NAME, release=release) + now = timezone.now() + + build = Build.objects.create(project=self.project, + started_on=now, + completed_on=now) + layersrc = LayerSource.objects.create(sourcetype=LayerSource.TYPE_IMPORTED) self.priority = ReleaseLayerSourcePriority.objects.create(release=release, layer_source=layersrc) layer = Layer.objects.create(name="base-layer", layer_source=layersrc, vcs_url="/tmp/") + branch = Branch.objects.create(name="master", layer_source=layersrc) + lver = Layer_Version.objects.create(layer=layer, project=self.project, - layer_source=layersrc, commit="master") + layer_source=layersrc, commit="master", + up_branch=branch) - Recipe.objects.create(layer_source=layersrc, name="base-recipe", - version="1.2", summary="one recipe", - description="recipe", layer_version=lver) + self.recipe1 = Recipe.objects.create(layer_source=layersrc, + name="base-recipe", + version="1.2", + summary="one recipe", + description="recipe", + layer_version=lver) Machine.objects.create(layer_version=lver, name="wisk", description="wisking machine") ProjectLayer.objects.create(project=self.project, layercommit=lver) + + self.customr = CustomImageRecipe.objects.create(\ + name="custom recipe", project=self.project, + base_recipe=self.recipe1) + + self.package = Package.objects.create(name='pkg1', recipe=self.recipe1, + build=build) + + + # recipe with project for testing AvailableRecipe table + self.recipe2 = Recipe.objects.create(layer_source=layersrc, + name="fancy-recipe", + version="1.4", + summary="a fancy recipe", + description="fancy recipe", + layer_version=lver, + file_path='/home/foo') + self.assertTrue(lver in self.project.compatible_layerversions()) def test_get_base_call_returns_html(self):
@@ -183,6 +222,144 @@ class ViewTests(TestCase): data = json.loads(response.content) self.assertNotEqual(data["error"], "ok") + def test_custom_ok(self): + """Test successful return from ReST API xhr_customrecipe""" + url = reverse('xhr_customrecipe') + params = {'name': 'custom', 'project': self.project.id, + 'base': self.recipe1.id} + response = self.client.post(url, params) + self.assertEqual(response.status_code, 200) + data = json.loads(response.content) + self.assertEqual(data['error'], 'ok') + self.assertTrue('url' in data) + # get recipe from the database + recipe = CustomImageRecipe.objects.get(project=self.project, + name=params['name']) + args = (self.project.id, recipe.id,) + self.assertEqual(reverse('customrecipe', args=args), data['url']) + + def test_custom_incomplete_params(self): + """Test not passing all required parameters to xhr_customrecipe""" + url = reverse('xhr_customrecipe') + for params in [{}, {'name': 'custom'}, + {'name': 'custom', 'project': self.project.id}]: + response = self.client.post(url, params) + self.assertEqual(response.status_code, 200) + data = json.loads(response.content) + self.assertNotEqual(data["error"], "ok") + + def test_xhr_custom_wrong_project(self): + """Test passing wrong project id to xhr_customrecipe""" + url = reverse('xhr_customrecipe') + params = {'name': 'custom', 'project': 0, "base": self.recipe1.id} + response = self.client.post(url, params) + self.assertEqual(response.status_code, 200) + data = json.loads(response.content) + self.assertNotEqual(data["error"], "ok") + + def test_xhr_custom_wrong_base(self): + """Test passing wrong base recipe id to xhr_customrecipe""" + url = reverse('xhr_customrecipe') + params = {'name': 'custom', 'project': self.project.id, "base": 0} + response = self.client.post(url, params) + self.assertEqual(response.status_code, 200) + data = json.loads(response.content) + self.assertNotEqual(data["error"], "ok") + + def test_xhr_custom_details(self): + """Test getting custom recipe details""" + name = "custom recipe" + url = reverse('xhr_customrecipe_id', args=(self.customr.id,)) + response = self.client.get(url) + self.assertEqual(response.status_code, 200) + expected = {"error": "ok", + "info": {'id': self.customr.id, + 'name': name, + 'base_recipe_id': self.recipe1.id, + 'project_id': self.project.id, + } + } + self.assertEqual(json.loads(response.content), expected) + + def test_xhr_custom_del(self): + """Test deleting custom recipe""" + name = "to be deleted" + recipe = CustomImageRecipe.objects.create(\ + name=name, project=self.project, + base_recipe=self.recipe1) + url = reverse('xhr_customrecipe_id', args=(recipe.id,)) + response = self.client.delete(url) + self.assertEqual(response.status_code, 200) + self.assertEqual(json.loads(response.content), {"error": "ok"}) + # try to delete not-existent recipe + url = reverse('xhr_customrecipe_id', args=(recipe.id,)) + response = self.client.delete(url) + self.assertEqual(response.status_code, 200) + self.assertNotEqual(json.loads(response.content)["error"], "ok") + + def test_xhr_custom_packages(self): + """Test adding and deleting package to a custom recipe""" + url = reverse('xhr_customrecipe_packages', + args=(self.customr.id, self.package.id)) + # add self.package1 to recipe + response = self.client.put(url) + self.assertEqual(response.status_code, 200) + self.assertEqual(json.loads(response.content), {"error": "ok"}) + self.assertEqual(self.customr.packages.all()[0].id, self.package.id) + # delete it + response = self.client.delete(url) + self.assertEqual(response.status_code, 200) + self.assertEqual(json.loads(response.content), {"error": "ok"}) + self.assertFalse(self.customr.packages.all()) + # delete it again to test error condition + response = self.client.delete(url) + self.assertEqual(response.status_code, 200) + self.assertNotEqual(json.loads(response.content)["error"], "ok") + + def test_xhr_custom_packages_err(self): + """Test error conditions of xhr_customrecipe_packages""" + # test calls with wrong recipe id and wrong package id + for args in [(0, self.package.id), (self.customr.id, 0)]: + url = reverse('xhr_customrecipe_packages', args=args) + # test put and delete methods + for method in (self.client.put, self.client.delete): + response = method(url) + self.assertEqual(response.status_code, 200) + self.assertNotEqual(json.loads(response.content), + {"error": "ok"}) + + def test_software_recipes_table(self): + """Test structure returned for Software RecipesTable""" + table = SoftwareRecipesTable() + request = RequestFactory().get('/foo/', {'format': 'json'}) + response = table.get(request, pid=self.project.id) + data = json.loads(response.content) + + rows = data['rows'] + row1 = next(x for x in rows if x['name'] == self.recipe1.name) + row1_btns = row1['static:add-del-layers'] + row1_btns_data = row1['add-del-layers'] + row2 = next(x for x in rows if x['name'] == self.recipe2.name) + row2_btns = row2['static:add-del-layers'] + row2_btns_data = row2['add-del-layers'] + + self.assertEqual(response.status_code, 200, 'should be 200 OK status') + self.assertEqual(len(rows), 2, 'should be 2 recipes') + + # check other columns have been populated correctly + self.assertEqual(row1['name'], self.recipe1.name) + self.assertEqual(row1['version'], self.recipe1.version) + self.assertEqual(row1['get_description_or_summary'], + self.recipe1.description) + self.assertEqual(row1['layer_version__layer__name'], + self.recipe1.layer_version.layer.name) + self.assertEqual(row2['name'], self.recipe2.name) + self.assertEqual(row2['version'], self.recipe2.version) + self.assertEqual(row2['get_description_or_summary'], + self.recipe2.description) + self.assertEqual(row2['layer_version__layer__name'], + self.recipe2.layer_version.layer.name) + class LandingPageTests(TestCase): """ Tests for redirects on the landing page """ # disable bogus pylint message error:
Signature · 1 line
1.9.1
PATCH 16/22 toaster: Special case the openembedded-core layer to avoid duplicates
In reply to brian averyFrom: Michael Wood <[email protected]>
If the openembedded-core layer is specified in the toasterconf we need to treat it differently because we may also get this layer either from the layerindex source or I can also be provided locally.
--- lib/toaster/orm/models.py | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
lib/toaster/orm/models.py +21 -0
@@ -828,6 +828,7 @@ class LayerIndexLayerSource(LayerSource): import urllib2, urlparse, json import os proxy_settings = os.environ.get("http_proxy", None) + oe_core_layer = 'openembedded-core' def _get_json_response(apiurl = self.apiurl): _parsedurl = urlparse.urlparse(apiurl)
@@ -872,6 +873,25 @@ class LayerIndexLayerSource(LayerSource): if not connection.features.autocommits_when_autocommit_is_off: transaction.set_autocommit(False) for li in layers_info: + # Special case for the openembedded-core layer + if li['name'] == oe_core_layer: + try: + # If we have an existing openembedded-core for example + # from the toasterconf.json augment the info using the + # layerindex rather than duplicate it + oe_core_l = Layer.objects.get(name=oe_core_layer) + # Take ownership of the layer as now coming from the + # layerindex + oe_core_l.layer_source = self + oe_core_l.up_id = li['id'] + oe_core_l.summary = li['summary'] + oe_core_l.description = li['description'] + oe_core_l.save() + continue + + except DoesNotExist: + pass + l, created = Layer.objects.get_or_create(layer_source = self, name = li['name']) l.up_id = li['id'] l.up_date = li['updated']
@@ -882,6 +902,7 @@ class LayerIndexLayerSource(LayerSource): l.summary = li['summary'] l.description = li['description'] l.save() + if not connection.features.autocommits_when_autocommit_is_off: transaction.set_autocommit(True)
Signature · 1 line
1.9.1
PATCH 17/22 toaster: Create a relationship between build information and toaster layers
In reply to brian averyFrom: Michael Wood <[email protected]>
Previously this layer relationship was done by trying to match path information that came back to the buildinfohelper with trying to query for data in toaster's layers table. This rarely matched due to the lose coupling.
--- lib/bb/ui/buildinfohelper.py | 30 +++- .../0009_auto__add_field_brlayer_layer_version.py | 180 +++++++++++++++++++++ lib/toaster/bldcontrol/models.py | 3 +- lib/toaster/orm/models.py | 2 +- 4 files changed, 211 insertions(+), 4 deletions(-) create mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py
lib/bb/ui/buildinfohelper.py +28 -2
@@ -94,8 +94,8 @@ class ORMWrapper(object): created = False if not key in vars(self)[dictname].keys(): - vars(self)[dictname][key] = clazz.objects.create(**kwargs) - created = True + vars(self)[dictname][key], created = \ + clazz.objects.get_or_create(**kwargs) return (vars(self)[dictname][key], created)
@@ -271,6 +271,17 @@ class ORMWrapper(object): return recipe_object def get_update_layer_version_object(self, build_obj, layer_obj, layer_version_information): + if isinstance(layer_obj, Layer_Version): + # We already found our layer version for this build so just + # update it with the new build information + logger.debug("We found our layer from toaster") + layer_obj.build = build_obj + layer_obj.local_path = layer_version_information['local_path'] + layer_obj.commit = layer_version_information['commit'] + layer_obj.save() + self.layer_version_objects.append(layer_obj) + return layer_obj + assert isinstance(build_obj, Build) assert isinstance(layer_obj, Layer) assert 'branch' in layer_version_information
@@ -320,8 +331,15 @@ class ORMWrapper(object): localdirname = os.path.join(bc.be.sourcedir, localdirname) #logger.debug(1, "Localdirname %s lcal_path %s" % (localdirname, layer_information['local_path'])) if localdirname.startswith(layer_information['local_path']): + # If the build request came from toaster this field + # should contain the information from the layer_version + # That created this build request. + if brl.layer_version: + return brl.layer_version + # we matched the BRLayer, but we need the layer_version that generated this BR; reverse of the Project.schedule_build() #logger.debug(1, "Matched %s to BRlayer %s" % (pformat(layer_information["local_path"]), localdirname)) + for pl in buildrequest.project.projectlayer_set.filter(layercommit__layer__name = brl.name): if pl.layercommit.layer.vcs_url == brl.giturl : layer = pl.layercommit.layer
@@ -674,6 +692,7 @@ class BuildInfoHelper(object): def __init__(self, server, has_build_history = False): self.internal_state = {} self.internal_state['taskdata'] = {} + self.internal_state['targets'] = [] self.task_order = 0 self.autocommit_step = 1 self.server = server
@@ -752,8 +771,15 @@ class BuildInfoHelper(object): if not localdirname.startswith("/"): localdirname = os.path.join(bc.be.sourcedir, localdirname) if path.startswith(localdirname): + # If the build request came from toaster this field + # should contain the information from the layer_version + # That created this build request. + if brl.layer_version: + return brl.layer_version + #logger.warn("-- managed: matched path %s with layer %s " % (path, localdirname)) # we matched the BRLayer, but we need the layer_version that generated this br + for lvo in self.orm_wrapper.layer_version_objects: if brl.name == lvo.layer.name: return lvo
lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py new file +180 -0
181 lines not on this page · raw message
lib/toaster/bldcontrol/models.py +2 -1
@@ -1,6 +1,6 @@ from django.db import models from django.core.validators import MaxValueValidator, MinValueValidator -from orm.models import Project, ProjectLayer, ProjectVariable, ProjectTarget, Build +from orm.models import Project, ProjectLayer, ProjectVariable, ProjectTarget, Build, Layer_Version # a BuildEnvironment is the equivalent of the "build/" directory on the localhost class BuildEnvironment(models.Model):
@@ -137,6 +137,7 @@ class BRLayer(models.Model): giturl = models.CharField(max_length = 254) commit = models.CharField(max_length = 254) dirpath = models.CharField(max_length = 254) + layer_version = models.ForeignKey(Layer_Version, null=True) class BRBitbake(models.Model): req = models.ForeignKey(BuildRequest, unique = True) # only one bitbake for a request
lib/toaster/orm/models.py +1 -1
@@ -260,7 +260,7 @@ class Project(models.Model): for l in self.projectlayer_set.all().order_by("pk"): commit = l.layercommit.get_vcs_reference() print("ii Building layer ", l.layercommit.layer.name, " at vcs point ", commit) - BRLayer.objects.create(req = br, name = l.layercommit.layer.name, giturl = l.layercommit.layer.vcs_url, commit = commit, dirpath = l.layercommit.dirpath) + BRLayer.objects.create(req = br, name = l.layercommit.layer.name, giturl = l.layercommit.layer.vcs_url, commit = commit, dirpath = l.layercommit.dirpath, layer_version=l.layercommit) br.state = BuildRequest.REQ_QUEUED now = timezone.now()
Signature · 1 line
1.9.1
PATCH 18/22 toaster: Prioroitise the layer more generic vcs reference over the sha
In reply to brian averyFrom: Michael Wood <[email protected]>
When we do a build we update the last commit value that the layer was built at However in future builds we do want to use the named reference rather than the commit sha, e.g. master/fido
--- lib/toaster/orm/models.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/toaster/orm/models.py +2 -2
@@ -1155,12 +1155,12 @@ class Layer_Version(models.Model): return project.compatible_layerversions(layer_name = self.layer.name) def get_vcs_reference(self): - if self.commit is not None and len(self.commit) > 0: - return self.commit if self.branch is not None and len(self.branch) > 0: return self.branch if self.up_branch is not None: return self.up_branch.name + if self.commit is not None and len(self.commit) > 0: + return self.commit return ("Cannot determine the vcs_reference for layer version %s" % vars(self)) def get_detailspage_url(self, project_id):
Signature · 1 line
1.9.1
PATCH 19/22 toaster: tables show all recipes in the layerdetails even duplicates
In reply to brian averyFrom: Michael Wood <[email protected]>
For some layers we have multiple recipes for the same software with differing versions. Change to showing all versions and add a version column to the table instead.
--- lib/toaster/toastergui/tables.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/toaster/toastergui/tables.py +4 -2
@@ -403,8 +403,8 @@ class LayerRecipesTable(RecipesTable): def setup_queryset(self, *args, **kwargs): - RecipesTable.setup_queryset(self, *args, **kwargs) - self.queryset = self.queryset.filter(layer_version__pk=int(kwargs['layerid'])) + self.queryset = \ + Recipe.objects.filter(layer_version__pk=int(kwargs['layerid'])) self.static_context_extra['in_prj'] = ProjectLayer.objects.filter(Q(project=kwargs['pid']) & Q(layercommit=kwargs['layerid'])).count()
@@ -418,6 +418,8 @@ class LayerRecipesTable(RecipesTable): self.add_column(title="Description", field_name="get_description_or_summary") + self.add_column(title="Version", + field_name="version") build_recipe_template ='<button class="btn btn-block build-recipe-btn" data-recipe-name="{{data.name}}" {%if extra.in_prj == 0 %}disabled="disabled"{%endif%}>Build recipe</button>'
Signature · 1 line
1.9.1
PATCH 20/22 toaster: buildinfohelper Create a copy of the built layer and recipe
In reply to brian averyFrom: Michael Wood <[email protected]>
Create a copy of the built layer and the recipes associated with it. This is so that the user can view the historical information about a build. i.e. a snapshot of the layer version and artifacts produced at that build.
--- lib/bb/ui/buildinfohelper.py | 54 +++++++++++++++++++++++++++++++++----------- lib/toaster/orm/models.py | 2 +- 2 files changed, 42 insertions(+), 14 deletions(-)
lib/bb/ui/buildinfohelper.py +41 -13
@@ -66,6 +66,7 @@ class ORMWrapper(object): def __init__(self): self.layer_version_objects = [] + self.layer_version_built = [] self.task_objects = {} self.recipe_objects = {}
@@ -254,32 +255,59 @@ class ORMWrapper(object): assert not recipe_information['file_path'].startswith("/") # we should have layer-relative paths at all times - recipe_object, created = self._cached_get_or_create(Recipe, layer_version=recipe_information['layer_version'], + + def update_recipe_obj(recipe_object): + object_changed = False + for v in vars(recipe_object): + if v in recipe_information.keys(): + object_changed = True + vars(recipe_object)[v] = recipe_information[v] + + if object_changed: + recipe_object.save() + + recipe, created = self._cached_get_or_create(Recipe, layer_version=recipe_information['layer_version'], file_path=recipe_information['file_path'], pathflags = recipe_information['pathflags']) - if created and must_exist: - raise NotExisting("Recipe object created when expected to exist", recipe_information) - object_changed = False - for v in vars(recipe_object): - if v in recipe_information.keys(): - object_changed = True - vars(recipe_object)[v] = recipe_information[v] + update_recipe_obj(recipe) - if object_changed: - recipe_object.save() + # Create a copy of the recipe for historical puposes and update it + for built_layer in self.layer_version_built: + if built_layer.layer == recipe_information['layer_version'].layer: + built_recipe, c = self._cached_get_or_create(Recipe, + layer_version=built_layer, + file_path=recipe_information['file_path'], + pathflags = recipe_information['pathflags']) + update_recipe_obj(built_recipe) + break - return recipe_object + + + if created and must_exist: + raise NotExisting("Recipe object created when expected to exist", recipe_information) + + return recipe def get_update_layer_version_object(self, build_obj, layer_obj, layer_version_information): if isinstance(layer_obj, Layer_Version): # We already found our layer version for this build so just # update it with the new build information logger.debug("We found our layer from toaster") - layer_obj.build = build_obj layer_obj.local_path = layer_version_information['local_path'] - layer_obj.commit = layer_version_information['commit'] layer_obj.save() self.layer_version_objects.append(layer_obj) + + # create a new copy of this layer version as a snapshot for + # historical purposes + layer_copy, c = Layer_Version.objects.get_or_create(build=build_obj, + layer=layer_obj.layer, + commit=layer_version_information['commit'], + local_path = layer_version_information['local_path'], + ) + logger.warning("created new historical layer version %d", layer_copy.pk) + + self.layer_version_built.append(layer_copy) + return layer_obj assert isinstance(build_obj, Build)
lib/toaster/orm/models.py +1 -1
@@ -194,7 +194,7 @@ class Project(models.Model): if release == None: release = self.release # layers on the same branch or layers specifically set for this project - queryset = Layer_Version.objects.filter((Q(up_branch__name = release.branch_name) & Q(project = None)) | Q(project = self) | Q(build__project = self)) + queryset = Layer_Version.objects.filter(((Q(up_branch__name = release.branch_name) & Q(project = None)) | Q(project = self)) & Q(build__isnull=True)) if layer_name is not None: # we select only a layer name
Signature · 1 line
1.9.1
PATCH 21/22 Revert "bitbake: toaster: don't re-create Target objects"
In reply to brian averyFrom: Michael Wood <[email protected]>
This delete followed up the foreign keys and deleted things that were not expected to be deleted.
This reverts commit 08000eb27eb8413686fb2c8daf14d234a8bff83a.
--- lib/bb/ui/buildinfohelper.py | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
lib/bb/ui/buildinfohelper.py +16 -1
@@ -162,6 +162,8 @@ class ORMWrapper(object): build.bitbake_version=build_info['bitbake_version'] build.save() + Target.objects.filter(build = build).delete() + else: build = Build.objects.create( project = prj,
@@ -182,6 +184,19 @@ class ORMWrapper(object): return build + def create_target_objects(self, target_info): + assert 'build' in target_info + assert 'targets' in target_info + + targets = [] + for tgt_name in target_info['targets']: + tgt_object = Target.objects.create( build = target_info['build'], + target = tgt_name, + is_image = False, + ) + targets.append(tgt_object) + return targets + def update_build_object(self, build, errors, warnings, taskfailures): assert isinstance(build,Build) assert isinstance(errors, int)
@@ -911,7 +926,7 @@ class BuildInfoHelper(object): target_information['targets'] = event._pkgs target_information['build'] = build_obj - self.internal_state['targets'] = Target.objects.filter(build=target_information['build']) + self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information) # Save build configuration data = self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0]
Signature · 1 line
1.9.1
PATCH 22/22 toaster: orm remove the complicated querying on the ORM
In reply to brian averyFrom: Michael Wood <[email protected]>
We no longer need to compute each layer_version and all the recipes which belong to this.
[YOCTO #8147]
--- lib/toaster/orm/models.py | 49 ++++++++++++++++++++++-------------- lib/toaster/toastergui/typeaheads.py | 5 ++-- 2 files changed, 33 insertions(+), 21 deletions(-)
lib/toaster/orm/models.py +30 -19
@@ -191,6 +191,7 @@ class Project(models.Model): # returns a queryset of compatible layers for a project def compatible_layerversions(self, release = None, layer_name = None): + logger.warning("This function is deprecated") if release == None: release = self.release # layers on the same branch or layers specifically set for this project
@@ -205,45 +206,55 @@ class Project(models.Model): return queryset - def projectlayer_equivalent_set(self): - return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch") + def get_all_compatible_layer_versions(self): + """ Returns Queryset of all Layer_Versions which are compatible with + this project""" + queryset = Layer_Version.objects.filter( + (Q(up_branch__name=self.release.branch_name) & Q(build=None)) + | Q(project=self)) + + return queryset + + def get_project_layer_versions(self, pk=False): + """ Returns the Layer_Versions currently added to this project """ + layer_versions = self.projectlayer_set.all().values('layercommit') + + if pk is False: + return layer_versions + else: + return layer_versions.values_list('pk', flat=True) + def get_available_machines(self): """ Returns QuerySet of all Machines which are provided by the Layers currently added to the Project """ - queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set) + queryset = Machine.objects.filter( + layer_version__in=self.get_project_layer_versions(self)) + return queryset def get_all_compatible_machines(self): """ Returns QuerySet of all the compatible machines available to the project including ones from Layers not currently added """ - compatible_layers = self.compatible_layerversions() + queryset = Machine.objects.filter( + layer_version__in=self.get_all_compatible_layer_versions()) - queryset = Machine.objects.filter(layer_version__in=compatible_layers) return queryset def get_available_recipes(self): - """ Returns QuerySet of all Recipes which are provided by the Layers - currently added to the Project """ - project_layers = self.projectlayer_equivalent_set() - queryset = Recipe.objects.filter(layer_version__in = project_layers) - - # Copied from get_all_compatible_recipes - search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id'))) - queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source') - # End copy + """ Returns QuerySet of all the recipes that are provided by layers + added to this project """ + queryset = Recipe.objects.filter( + layer_version__in=self.get_project_layer_versions()) return queryset def get_all_compatible_recipes(self): """ Returns QuerySet of all the compatible Recipes available to the project including ones from Layers not currently added """ - compatible_layerversions = self.compatible_layerversions() - queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions) - - search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id'))) + queryset = Recipe.objects.filter( + layer_version__in=self.get_all_compatible_layer_versions()) - queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source') return queryset
lib/toaster/toastergui/typeaheads.py +3 -2
@@ -27,7 +27,7 @@ class LayersTypeAhead(ToasterTypeAhead): super(LayersTypeAhead, self).__init__() def apply_search(self, search_term, prj, request): - layers = prj.compatible_layerversions() + layers = prj.get_all_compatible_layer_versions() layers = layers.order_by('layer__name') # Unlike the other typeaheads we also don't want to show suggestions
@@ -35,7 +35,8 @@ class LayersTypeAhead(ToasterTypeAhead): # layerdeps to a new layer. if ("include_added" in request.GET and request.GET['include_added'] != "true"): - layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set) + layers = layers.exclude( + pk__in=prj.get_project_layer_versions(pk=True)) primary_results = layers.filter(layer__name__istartswith=search_term) secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)
Signature · 1 line
1.9.1
PATCH v2 22/22 toaster: orm remove the complicated querying on the ORM
We no longer need to compute each layer_version and all the recipes which belong to this.
[YOCTO #8147]
--- lib/toaster/orm/models.py | 49 ++++++++++++++++++++++-------------- lib/toaster/toastergui/tables.py | 2 +- lib/toaster/toastergui/typeaheads.py | 5 ++-- 3 files changed, 34 insertions(+), 22 deletions(-)
lib/toaster/orm/models.py +30 -19
@@ -191,6 +191,7 @@ class Project(models.Model): # returns a queryset of compatible layers for a project def compatible_layerversions(self, release = None, layer_name = None): + logger.warning("This function is deprecated") if release == None: release = self.release # layers on the same branch or layers specifically set for this project
@@ -205,45 +206,55 @@ class Project(models.Model): return queryset - def projectlayer_equivalent_set(self): - return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch") + def get_all_compatible_layer_versions(self): + """ Returns Queryset of all Layer_Versions which are compatible with + this project""" + queryset = Layer_Version.objects.filter( + (Q(up_branch__name=self.release.branch_name) & Q(build=None)) + | Q(project=self)) + + return queryset + + def get_project_layer_versions(self, pk=False): + """ Returns the Layer_Versions currently added to this project """ + layer_versions = self.projectlayer_set.all().values('layercommit') + + if pk is False: + return layer_versions + else: + return layer_versions.values_list('pk', flat=True) + def get_available_machines(self): """ Returns QuerySet of all Machines which are provided by the Layers currently added to the Project """ - queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set) + queryset = Machine.objects.filter( + layer_version__in=self.get_project_layer_versions(self)) + return queryset def get_all_compatible_machines(self): """ Returns QuerySet of all the compatible machines available to the project including ones from Layers not currently added """ - compatible_layers = self.compatible_layerversions() + queryset = Machine.objects.filter( + layer_version__in=self.get_all_compatible_layer_versions()) - queryset = Machine.objects.filter(layer_version__in=compatible_layers) return queryset def get_available_recipes(self): - """ Returns QuerySet of all Recipes which are provided by the Layers - currently added to the Project """ - project_layers = self.projectlayer_equivalent_set() - queryset = Recipe.objects.filter(layer_version__in = project_layers) - - # Copied from get_all_compatible_recipes - search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id'))) - queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source') - # End copy + """ Returns QuerySet of all the recipes that are provided by layers + added to this project """ + queryset = Recipe.objects.filter( + layer_version__in=self.get_project_layer_versions()) return queryset def get_all_compatible_recipes(self): """ Returns QuerySet of all the compatible Recipes available to the project including ones from Layers not currently added """ - compatible_layerversions = self.compatible_layerversions() - queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions) - - search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id'))) + queryset = Recipe.objects.filter( + layer_version__in=self.get_all_compatible_layer_versions()) - queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source') return queryset
lib/toaster/toastergui/tables.py +1 -1
@@ -221,7 +221,7 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin): def setup_filters(self, *args, **kwargs): project = Project.objects.get(pk=kwargs['pid']) - self.project_layers = project.projectlayer_equivalent_set() + self.project_layers = project.get_project_layer_versions() self.add_filter(title="Filter by project machines", name="in_current_project",
lib/toaster/toastergui/typeaheads.py +3 -2
@@ -27,7 +27,7 @@ class LayersTypeAhead(ToasterTypeAhead): super(LayersTypeAhead, self).__init__() def apply_search(self, search_term, prj, request): - layers = prj.compatible_layerversions() + layers = prj.get_all_compatible_layer_versions() layers = layers.order_by('layer__name') # Unlike the other typeaheads we also don't want to show suggestions
@@ -35,7 +35,8 @@ class LayersTypeAhead(ToasterTypeAhead): # layerdeps to a new layer. if ("include_added" in request.GET and request.GET['include_added'] != "true"): - layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set) + layers = layers.exclude( + pk__in=prj.get_project_layer_versions(pk=True)) primary_results = layers.filter(layer__name__istartswith=search_term) secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)
Signature · 1 line
2.1.4
Re: [PATCH v2 22/22] toaster: orm remove the complicated querying on the ORM
Quoted 127 lines · Michael Wood
We no longer need to compute each
layer_versionand all the recipes which belong to this.[YOCTO #8147]
Signed-off-by Michael Wood <[email protected]>Signed-off-by brian avery <[email protected]>--- lib/toaster/orm/models.py | 49 ++++++++++++++++++++++-------------- lib/toaster/toastergui/tables.py | 2 +- lib/toaster/toastergui/typeaheads.py | 5 ++-- 3 files changed, 34 insertions(+), 22 deletions(-)lib/toaster/orm/models.py +30
-19@@ -191,6 +191,7 @@ class Project(models.Model): # returns a queryset of compatible layers for a project def compatible_layerversions(self, release = None, layer_name = None): + logger.warning("This function is deprecated") if release == None: release = self.release # layers on the same branch or layers specifically set for this project@@ -205,45 +206,55 @@ class Project(models.Model): return queryset - def projectlayer_equivalent_set(self): - return self.compatible_layerversions().filter(layer__name__in = [x.layercommit.layer.name for x in self.projectlayer_set.all()]).select_related("up_branch") + def get_all_compatible_layer_versions(self): + """ Returns Queryset of all Layer_Versions which are compatible with + this project""" + queryset = Layer_Version.objects.filter( + (Q(up_branch__name=self.release.branch_name) & Q(build=None)) + | Q(project=self)) + + return queryset + + def get_project_layer_versions(self, pk=False): + """ Returns the Layer_Versions currently added to this project """ + layer_versions = self.projectlayer_set.all().values('layercommit') + + if pk is False: + return layer_versions + else: + return layer_versions.values_list('pk', flat=True) + def get_available_machines(self): """ Returns QuerySet of all Machines which are provided by the Layers currently added to the Project """ - queryset = Machine.objects.filter(layer_version__in=self.projectlayer_equivalent_set) + queryset = Machine.objects.filter( + layer_version__in=self.get_project_layer_versions(self)) + return queryset def get_all_compatible_machines(self): """ Returns QuerySet of all the compatible machines available to the project including ones from Layers not currently added """ - compatible_layers = self.compatible_layerversions() + queryset = Machine.objects.filter( + layer_version__in=self.get_all_compatible_layer_versions()) - queryset = Machine.objects.filter(layer_version__in=compatible_layers) return queryset def get_available_recipes(self): - """ Returns QuerySet of all Recipes which are provided by the Layers - currently added to the Project """ - project_layers = self.projectlayer_equivalent_set() - queryset = Recipe.objects.filter(layer_version__in = project_layers) - - # Copied from get_all_compatible_recipes - search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id'))) - queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source') - # End copy + """ Returns QuerySet of all the recipes that are provided by layers + added to this project """ + queryset = Recipe.objects.filter( + layer_version__in=self.get_project_layer_versions()) return queryset def get_all_compatible_recipes(self): """ Returns QuerySet of all the compatible Recipes available to the project including ones from Layers not currently added """ - compatible_layerversions = self.compatible_layerversions() - queryset = Recipe.objects.filter(layer_version__in = compatible_layerversions) - - search_maxids = map(lambda i: i[0], list(queryset.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id'))) + queryset = Recipe.objects.filter( + layer_version__in=self.get_all_compatible_layer_versions()) - queryset = queryset.filter(id__in=search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source') return querysetlib/toaster/toastergui/tables.py +1
-1@@ -221,7 +221,7 @@ class MachinesTable(ToasterTable, ProjectFiltersMixin): def setup_filters(self, *args, **kwargs): project = Project.objects.get(pk=kwargs['pid']) - self.project_layers = project.projectlayer_equivalent_set() + self.project_layers = project.get_project_layer_versions() self.add_filter(title="Filter by project machines", name="in_current_project",lib/toaster/toastergui/typeaheads.py +1
-1@@ -27,7 +27,7 @@ class LayersTypeAhead(ToasterTypeAhead): super(LayersTypeAhead, self).__init__() def apply_search(self, search_term, prj, request): - layers = prj.compatible_layerversions() + layers = prj.get_all_compatible_layer_versions() layers = layers.order_by('layer__name') # Unlike the other typeaheads we also don't want to show suggestions
lib/toaster/toastergui/typeaheads.py class LayersTypeAhead(ToasterTypeAhead): · line 42
Quoted 9 lines
lib/toaster/toastergui/typeaheads.py +2
-1@@ -35,7 +35,8 @@ class LayersTypeAhead(ToasterTypeAhead): # layerdeps to a new layer. if ("include_added" in request.GET and request.GET['include_added'] != "true"): - layers = layers.exclude(pk__in=prj.projectlayer_equivalent_set) + layers = layers.exclude( + pk__in=prj.get_project_layer_versions(pk=True)) primary_results = layers.filter(layer__name__istartswith=search_term) secondary_results = layers.filter(layer__name__icontains=search_term).exclude(pk__in=primary_results)
We had a bit of a race condition on creating the pull request, this last patch was updated in the original branch but the submission branch was branched just before, hence v2 here.
Thanks,
Michael