Skip to contentSkip to conversation

toaster: project_specific_12785 enhancement set

PATCH New David Reyna 1 participant 2 messages Aug 16, 2018

lore
Share & tools
lore
#0 lore raw

cover toaster: project_specific_12785 enhancement set

From: David Reyna <[email protected]>

1) This is a large set of enhancements and fixes.

I have posted it as one patch because (a) the changes are tightly interwoven, (b) they have been debugged and tested together, and
(c) I want to conserve people's time.

2) This work was driven by two things:

(a) The opportunity to use Toaster as the YP configuration tool for third-party GUIs.

Wind River is in the process of open sourcing its Eclipse plug-ins for application and project development. The those plug-ins currently work with Wind River Workbench and Intel System Studio, and we are in the process of open sourcing them to Eclipse.org for all users.

(b) Comments from the YP Birds-of-a-Feather meeting at ELCE Prague.

There were several comments made about limitations with Toaster. Those needs aligned nicely with my personal goals with Toaster and especially the requirements of the above opportunity.

3) Here is a summary of the included content:

The primary driving enhancement:

* Bug 12785 - Support Project Specific configuration for external tools (e.g. ISS, Eclipse)

- Isolated project-specific configuration page (full Toaster context hidden)
- Support for new project, reconfigure existing project, and import existing command line project
- Ability to define variables (e.g. image recipe) and pass them back to external GUI
- Ability to execute the cloning phase, so that external GUI receive a buildable project
- Ability to call back to the external GUI when updates are completed and ready
- Compatibility of above projects with the normal full Toaster interface
- Ability to pass to a 'complete' or 'cancel' web page so that the external GUI can immediately stop that Toaster instance, and not leave dangling servers nor edit sessions open

Here are the supporting enhancements, where at least the back end is implemented:

*  Bug 12821 - Make Toaster conf changes compatible with command line usage
*  Bug 12822 - Support importing user changes to conf files into Toaster
*  Bug 12823 - Support importing user build directories into Toaster
*  Bug 12824 - Scan imported layers for content so that they are

immediately available
* Bug 12825 - show layer clone item in progress bar

Here are defects fixed:

*  Bug 12817 - builddelete.py requires explicit 'add_arguments'
*  Bug 12818 - Remove orphaned imported layers when project is deleted
*  Bug 12819 - build using selected bitbake env, not Toaster's env
*  Bug 12820 - Toaster randomizes the layer order in toaster_bblayers.conf
*  Bug 12826 - fix imported layer management

---

The following changes since commit 63a4ff7cf5f7d1671ab85800bc2212dd9cd9748d:

bitbake: checksum: sanity check path when recursively checksumming (2018-08-14 16:32:48 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib dreyna/toaster/project_specific_12785
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=dreyna/submit/dreyna/toaster/project_specific_12785

David Reyna (1):
Toaster: Implement the project-specific feature and releated
enhancements and defects.

 lib/toaster/bldcontrol/localhostbecontroller.py    | 172 ++++--
 .../bldcontrol/management/commands/runbuilds.py    |   2 +-
 .../orm/migrations/0018_project_specific.py        |  28 +
 lib/toaster/orm/models.py                          |  61 ++-
 lib/toaster/toastergui/api.py                      | 168 +++++-
 lib/toaster/toastergui/static/js/layerBtn.js       |  12 +
 lib/toaster/toastergui/static/js/libtoaster.js     | 105 ++++
 lib/toaster/toastergui/static/js/mrbsection.js     |   4 +-
 lib/toaster/toastergui/static/js/projecttopbar.js  |  22 +
 lib/toaster/toastergui/tables.py                   |   4 +
 .../toastergui/templates/base_specific.html        | 128 +++++
 .../templates/baseprojectspecificpage.html         |  48 ++
 .../templates/generic-toastertable-page.html       |   2 +-
 lib/toaster/toastergui/templates/importlayer.html  |   4 +-
 .../toastergui/templates/landing_specific.html     |  50 ++
 lib/toaster/toastergui/templates/layerdetails.html |   3 +-
 lib/toaster/toastergui/templates/mrb_section.html  |   2 +-
 .../toastergui/templates/newcustomimage.html       |   4 +-
 .../toastergui/templates/newproject_specific.html  |  95 ++++
 lib/toaster/toastergui/templates/project.html      |   7 +-
 .../toastergui/templates/project_specific.html     | 162 ++++++
 .../templates/project_specific_topbar.html         |  80 +++
 lib/toaster/toastergui/templates/projectconf.html  |   7 +-
 .../toastergui/templates/recipe_add_btn.html       |  23 +
 lib/toaster/toastergui/urls.py                     |  13 +
 lib/toaster/toastergui/views.py                    | 151 ++++++
 lib/toaster/toastergui/widgets.py                  |   6 +
 .../toastermain/management/commands/builddelete.py |   6 +-
 .../toastermain/management/commands/buildimport.py | 586 +++++++++++++++++++++
 29 files changed, 1900 insertions(+), 55 deletions(-)
 create mode 100644 bitbake/lib/toaster/orm/migrations/0018_project_specific.py
 create mode 100644 bitbake/lib/toaster/toastergui/templates/base_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/baseprojectspecificpage.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/landing_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/newproject_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/project_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/project_specific_topbar.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/recipe_add_btn.html
 mode change 100755 => 100644 bitbake/lib/toaster/toastergui/views.py
 create mode 100644 bitbake/lib/toaster/toastermain/management/commands/buildimport.py
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/1 Toaster: Implement the project-specific feature and releated enhancements and defects.

From: David Reyna <[email protected]>

Here is the primary driving enhancement:

* Bug 12785 - Support Project Specific configuration for external tools (e.g. ISS, Eclipse)

- Isolated project-specific configuration page (full Toaster context hidden)
- Support for new project, reconfigure existing project, and import existing command line project
- Ability to define variables (e.g. image recipe) and pass them back to external GUI
- Ability to execute the cloning phase, so that external GUI receive a buildable project
- Ability to call back to the external GUI when updates are completed and ready
- Compatibility of above projects with the normal full Toaster interface
- Ability to pass to a 'complete' or 'cancel' web page so that the external GUI can immediately stop that Toaster instance, and not leave dangling servers nor edit sessions open

Here are the supporting enhancements, where at least the back end is implemented:

*  Bug 12821 - Make Toaster conf changes compatible with command line usage
*  Bug 12822 - Support importing user changes to conf files into Toaster
*  Bug 12823 - Support importing user build directories into Toaster
*  Bug 12824 - Scan imported layers for content so that they are

immediately available
* Bug 12825 - show layer clone item in progress bar

Here are defects fixed:

*  Bug 12817 - builddelete.py requires explicit 'add_arguments'
*  Bug 12818 - Remove orphaned imported layers when project is deleted
*  Bug 12826 - fix imported layer management
*  Bug 12819 - build using selected bitbake env, not Toaster's env
*  Bug 12820 - Toaster randomizes the layer order in toaster_bblayers.conf

[YOCTO #12785]

---
 lib/toaster/bldcontrol/localhostbecontroller.py    | 172 ++++--
 .../bldcontrol/management/commands/runbuilds.py    |   2 +-
 .../orm/migrations/0018_project_specific.py        |  28 +
 lib/toaster/orm/models.py                          |  61 ++-
 lib/toaster/toastergui/api.py                      | 168 +++++-
 lib/toaster/toastergui/static/js/layerBtn.js       |  12 +
 lib/toaster/toastergui/static/js/libtoaster.js     | 105 ++++
 lib/toaster/toastergui/static/js/mrbsection.js     |   4 +-
 lib/toaster/toastergui/static/js/projecttopbar.js  |  22 +
 lib/toaster/toastergui/tables.py                   |   4 +
 .../toastergui/templates/base_specific.html        | 128 +++++
 .../templates/baseprojectspecificpage.html         |  48 ++
 .../templates/generic-toastertable-page.html       |   2 +-
 lib/toaster/toastergui/templates/importlayer.html  |   4 +-
 .../toastergui/templates/landing_specific.html     |  50 ++
 lib/toaster/toastergui/templates/layerdetails.html |   3 +-
 lib/toaster/toastergui/templates/mrb_section.html  |   2 +-
 .../toastergui/templates/newcustomimage.html       |   4 +-
 .../toastergui/templates/newproject_specific.html  |  95 ++++
 lib/toaster/toastergui/templates/project.html      |   7 +-
 .../toastergui/templates/project_specific.html     | 162 ++++++
 .../templates/project_specific_topbar.html         |  80 +++
 lib/toaster/toastergui/templates/projectconf.html  |   7 +-
 .../toastergui/templates/recipe_add_btn.html       |  23 +
 lib/toaster/toastergui/urls.py                     |  13 +
 lib/toaster/toastergui/views.py                    | 151 ++++++
 lib/toaster/toastergui/widgets.py                  |   6 +
 .../toastermain/management/commands/builddelete.py |   6 +-
 .../toastermain/management/commands/buildimport.py | 586 +++++++++++++++++++++
 29 files changed, 1900 insertions(+), 55 deletions(-)
 create mode 100644 bitbake/lib/toaster/orm/migrations/0018_project_specific.py
 create mode 100644 bitbake/lib/toaster/toastergui/templates/base_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/baseprojectspecificpage.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/landing_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/newproject_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/project_specific.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/project_specific_topbar.html
 create mode 100644 bitbake/lib/toaster/toastergui/templates/recipe_add_btn.html
 mode change 100755 => 100644 bitbake/lib/toaster/toastergui/views.py
 create mode 100644 bitbake/lib/toaster/toastermain/management/commands/buildimport.py
lib/toaster/bldcontrol/localhostbecontroller.py +139 -33
index 16c7c80..6bdd743 100644
@@ -27,8 +27,9 @@ import shutil
 import time
 from django.db import transaction
 from django.db.models import Q
-from bldcontrol.models import BuildEnvironment, BRLayer, BRVariable, BRTarget, BRBitbake
-from orm.models import CustomImageRecipe, Layer, Layer_Version, ProjectLayer, ToasterSetting
+from bldcontrol.models import BuildEnvironment, BuildRequest, BRLayer, BRVariable, BRTarget, BRBitbake, Build
+from orm.models import CustomImageRecipe, Layer, Layer_Version, Project, ProjectLayer, ToasterSetting
+from orm.models import signal_runbuilds
 import subprocess
 
 from toastermain import settings
@@ -38,6 +39,8 @@ from bldcontrol.bbcontroller import BuildEnvironmentController, ShellCmdExceptio
 import logging
 logger = logging.getLogger("toaster")
 
+install_dir = os.environ.get('TOASTER_DIR')
+
 from pprint import pprint, pformat
 
 class LocalhostBEController(BuildEnvironmentController):
@@ -87,10 +90,10 @@ class LocalhostBEController(BuildEnvironmentController):
         #logger.debug("localhostbecontroller: using HEAD checkout in %s" % local_checkout_path)
         return local_checkout_path
 
-
-    def setCloneStatus(self,bitbake,status,total,current):
+    def setCloneStatus(self,bitbake,status,total,current,repo_name):
         bitbake.req.build.repos_cloned=current
         bitbake.req.build.repos_to_clone=total
+        bitbake.req.build.progress_item=repo_name
         bitbake.req.build.save()
 
     def setLayers(self, bitbake, layers, targets):
@@ -100,6 +103,7 @@ class LocalhostBEController(BuildEnvironmentController):
 
         layerlist = []
         nongitlayerlist = []
+        layer_index = 0
         git_env = os.environ.copy()
         # (note: add custom environment settings here)
 
@@ -113,7 +117,7 @@ class LocalhostBEController(BuildEnvironmentController):
         if bitbake.giturl and bitbake.commit:
             gitrepos[(bitbake.giturl, bitbake.commit)] = []
             gitrepos[(bitbake.giturl, bitbake.commit)].append(
-                ("bitbake", bitbake.dirpath))
+                ("bitbake", bitbake.dirpath, 0))
 
         for layer in layers:
             # We don't need to git clone the layer for the CustomImageRecipe
@@ -124,12 +128,13 @@ class LocalhostBEController(BuildEnvironmentController):
             # If we have local layers then we don't need clone them
             # For local layers giturl will be empty
             if not layer.giturl:
-                nongitlayerlist.append(layer.layer_version.layer.local_source_dir)
+                nongitlayerlist.append( "%03d:%s" % (layer_index,layer.local_source_dir) )
                 continue
 
             if not (layer.giturl, layer.commit) in gitrepos:
                 gitrepos[(layer.giturl, layer.commit)] = []
-            gitrepos[(layer.giturl, layer.commit)].append( (layer.name, layer.dirpath) )
+            gitrepos[(layer.giturl, layer.commit)].append( (layer.name,layer.dirpath,layer_index) )
+            layer_index += 1
 
 
         logger.debug("localhostbecontroller, our git repos are %s" % pformat(gitrepos))
@@ -159,9 +164,9 @@ class LocalhostBEController(BuildEnvironmentController):
         # 3. checkout the repositories
         clone_count=0
         clone_total=len(gitrepos.keys())
-        self.setCloneStatus(bitbake,'Started',clone_total,clone_count)
+        self.setCloneStatus(bitbake,'Started',clone_total,clone_count,'')
         for giturl, commit in gitrepos.keys():
-            self.setCloneStatus(bitbake,'progress',clone_total,clone_count)
+            self.setCloneStatus(bitbake,'progress',clone_total,clone_count,gitrepos[(giturl, commit)][0][0])
             clone_count += 1
 
             localdirname = os.path.join(self.be.sourcedir, self.getGitCloneDirectory(giturl, commit))
@@ -205,16 +210,16 @@ class LocalhostBEController(BuildEnvironmentController):
                     self._shellcmd("git clone -b \"%s\" \"%s\" \"%s\" " % (bitbake.commit, bitbake.giturl, os.path.join(self.pokydirname, 'bitbake')),env=git_env)
 
             # verify our repositories
-            for name, dirpath in gitrepos[(giturl, commit)]:
+            for name, dirpath, index in gitrepos[(giturl, commit)]:
                 localdirpath = os.path.join(localdirname, dirpath)
-                logger.debug("localhostbecontroller: localdirpath expected '%s'" % localdirpath)
+                logger.debug("localhostbecontroller: localdirpath expects '%s'" % localdirpath)
                 if not os.path.exists(localdirpath):
                     raise BuildSetupException("Cannot find layer git path '%s' in checked out repository '%s:%s'. Aborting." % (localdirpath, giturl, commit))
 
                 if name != "bitbake":
-                    layerlist.append(localdirpath.rstrip("/"))
+                    layerlist.append("%03d:%s" % (index,localdirpath.rstrip("/")))
 
-        self.setCloneStatus(bitbake,'complete',clone_total,clone_count)
+        self.setCloneStatus(bitbake,'complete',clone_total,clone_count,'')
         logger.debug("localhostbecontroller: current layer list %s " % pformat(layerlist))
 
         if self.pokydirname is None and os.path.exists(os.path.join(self.be.sourcedir, "oe-init-build-env")):
@@ -232,7 +237,7 @@ class LocalhostBEController(BuildEnvironmentController):
                     customrecipe, layers)
 
                 if os.path.isdir(custom_layer_path):
-                    layerlist.append(custom_layer_path)
+                    layerlist.append("%03d:%s" % (layer_index,custom_layer_path))
 
             except CustomImageRecipe.DoesNotExist:
                 continue  # not a custom recipe, skip
@@ -240,7 +245,11 @@ class LocalhostBEController(BuildEnvironmentController):
         layerlist.extend(nongitlayerlist)
         logger.debug("\n\nset layers gives this list %s" % pformat(layerlist))
         self.islayerset = True
-        return layerlist
+
+        # restore the order of layer list for bblayers.conf
+        layerlist.sort()
+        sorted_layerlist = [l[4:] for l in layerlist]
+        return sorted_layerlist
 
     def setup_custom_image_recipe(self, customrecipe, layers):
         """ Set up toaster-custom-images layer and recipe files """
@@ -310,31 +319,115 @@ class LocalhostBEController(BuildEnvironmentController):
 
     def triggerBuild(self, bitbake, layers, variables, targets, brbe):
         layers = self.setLayers(bitbake, layers, targets)
+        is_merged_attr = bitbake.req.project.merged_attr
+
+        git_env = os.environ.copy()
+        # (note: add custom environment settings here)
+        try:
+            # insure that the project init/build uses the selected bitbake, and not Toaster's
+            del git_env['TEMPLATECONF']
+            del git_env['BBBASEDIR']
+            del git_env['BUILDDIR']
+        except KeyError:
+            pass
 
         # init build environment from the clone
-        builddir = '%s-toaster-%d' % (self.be.builddir, bitbake.req.project.id)
+        if bitbake.req.project.builddir:
+            builddir = bitbake.req.project.builddir
+        else:
+            builddir = '%s-toaster-%d' % (self.be.builddir, bitbake.req.project.id)
         oe_init = os.path.join(self.pokydirname, 'oe-init-build-env')
         # init build environment
         try:
             custom_script = ToasterSetting.objects.get(name="CUSTOM_BUILD_INIT_SCRIPT").value
             custom_script = custom_script.replace("%BUILDDIR%" ,builddir)
-            self._shellcmd("bash -c 'source %s'" % (custom_script))
+            self._shellcmd("bash -c 'source %s'" % (custom_script),env=git_env)
         except ToasterSetting.DoesNotExist:
             self._shellcmd("bash -c 'source %s %s'" % (oe_init, builddir),
-                       self.be.sourcedir)
+                       self.be.sourcedir,env=git_env)
 
         # update bblayers.conf
-        bblconfpath = os.path.join(builddir, "conf/toaster-bblayers.conf")
-        with open(bblconfpath, 'w') as bblayers:
-            bblayers.write('# line added by toaster build control\n'
-                           'BBLAYERS = "%s"' % ' '.join(layers))
-
-        # write configuration file
-        confpath = os.path.join(builddir, 'conf/toaster.conf')
-        with open(confpath, 'w') as conf:
-            for var in variables:
-                conf.write('%s="%s"\n' % (var.name, var.value))
-            conf.write('INHERIT+="toaster buildhistory"')
+        if not is_merged_attr:
+            bblconfpath = os.path.join(builddir, "conf/toaster-bblayers.conf")
+            with open(bblconfpath, 'w') as bblayers:
+                bblayers.write('# line added by toaster build control\n'
+                               'BBLAYERS = "%s"' % ' '.join(layers))
+
+            # write configuration file
+            confpath = os.path.join(builddir, 'conf/toaster.conf')
+            with open(confpath, 'w') as conf:
+                for var in variables:
+                    conf.write('%s="%s"\n' % (var.name, var.value))
+                conf.write('INHERIT+="toaster buildhistory"')
+        else:
+            # Append the Toaster-specific values directly to the bblayers.conf
+            bblconfpath = os.path.join(bitbake.req.project.builddir, "conf/bblayers.conf")
+            bblconfpath_save = os.path.join(bitbake.req.project.builddir, "conf/bblayers.conf.save")
+            shutil.copyfile(bblconfpath, bblconfpath_save)
+            with open(bblconfpath) as bblayers:
+                content = bblayers.readlines()
+            do_write = True
+            was_toaster = False
+            with open(bblconfpath,'w') as bblayers:
+                for line in content:
+                    #line = line.strip('\n')
+                    if 'TOASTER_CONFIG_PROLOG' in line:
+                        do_write = False
+                        was_toaster = True
+                    elif 'TOASTER_CONFIG_EPILOG' in line:
+                        do_write = True
+                    elif do_write:
+                        bblayers.write(line)
+                if not was_toaster:
+                    bblayers.write('\n')
+                bblayers.write('#=== TOASTER_CONFIG_PROLOG ===\n')
+                bblayers.write('BBLAYERS = "\\\n')
+                for layer in layers:
+                    bblayers.write('  %s \\\n' % layer)
+                bblayers.write('  "\n')
+                bblayers.write('#=== TOASTER_CONFIG_EPILOG ===\n')
+            # Append the Toaster-specific values directly to the local.conf
+            bbconfpath = os.path.join(bitbake.req.project.builddir, "conf/local.conf")
+            bbconfpath_save = os.path.join(bitbake.req.project.builddir, "conf/local.conf.save")
+            shutil.copyfile(bbconfpath, bbconfpath_save)
+            with open(bbconfpath) as f:
+                content = f.readlines()
+            do_write = True
+            was_toaster = False
+            with open(bbconfpath,'w') as conf:
+                for line in content:
+                    #line = line.strip('\n')
+                    if 'TOASTER_CONFIG_PROLOG' in line:
+                        do_write = False
+                        was_toaster = True
+                    elif 'TOASTER_CONFIG_EPILOG' in line:
+                        do_write = True
+                    elif do_write:
+                        conf.write(line)
+                if not was_toaster:
+                    conf.write('\n')
+                conf.write('#=== TOASTER_CONFIG_PROLOG ===\n')
+                for var in variables:
+                    if (not var.name.startswith("INTERNAL_")) and (not var.name == "BBLAYERS"):
+                        conf.write('%s="%s"\n' % (var.name, var.value))
+                conf.write('#=== TOASTER_CONFIG_EPILOG ===\n')
+
+        # If 'target' is just the project preparation target, then we are done
+        for target in targets:
+            if "_PROJECT_PREPARE_" == target.target:
+                logger.debug('localhostbecontroller: Project has been prepared. Done.')
+                # Update the Build Request and release the build environment
+                bitbake.req.state = BuildRequest.REQ_COMPLETED
+                bitbake.req.save()
+                self.be.lock = BuildEnvironment.LOCK_FREE
+                self.be.save()
+                # Close the project build and progress bar
+                bitbake.req.build.outcome = Build.SUCCEEDED
+                bitbake.req.build.save()
+                # Update the project status
+                bitbake.req.project.set_variable(Project.PROJECT_SPECIFIC_STATUS,Project.PROJECT_SPECIFIC_CLONING_SUCCESS)
+                signal_runbuilds()
+                return
 
         # clean the Toaster to build environment
         env_clean = 'unset BBPATH;' # clean BBPATH for <= YP-2.4.0
@@ -342,9 +435,14 @@ class LocalhostBEController(BuildEnvironmentController):
         # run bitbake server from the clone
         bitbake = os.path.join(self.pokydirname, 'bitbake', 'bin', 'bitbake')
         toasterlayers = os.path.join(builddir,"conf/toaster-bblayers.conf")
-        self._shellcmd('%s bash -c \"source %s %s; BITBAKE_UI="knotty" %s --read %s --read %s '
-                       '--server-only -B 0.0.0.0:0\"' % (env_clean, oe_init,
-                       builddir, bitbake, confpath, toasterlayers), self.be.sourcedir)
+        if not is_merged_attr:
+            self._shellcmd('%s bash -c \"source %s %s; BITBAKE_UI="knotty" %s --read %s --read %s '
+                           '--server-only -B 0.0.0.0:0\"' % (env_clean, oe_init,
+                           builddir, bitbake, confpath, toasterlayers), self.be.sourcedir)
+        else:
+            self._shellcmd('%s bash -c \"source %s %s; BITBAKE_UI="knotty" %s '
+                           '--server-only -B 0.0.0.0:0\"' % (env_clean, oe_init,
+                           builddir, bitbake), self.be.sourcedir)
 
         # read port number from bitbake.lock
         self.be.bbport = -1
@@ -390,12 +488,20 @@ class LocalhostBEController(BuildEnvironmentController):
         log = os.path.join(builddir, 'toaster_ui.log')
         local_bitbake = os.path.join(os.path.dirname(os.getenv('BBBASEDIR')),
                                      'bitbake')
-        self._shellcmd(['%s bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:%s" '
+        if not is_merged_attr:
+            self._shellcmd(['%s bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:%s" '
                         '%s %s -u toasterui  --read %s --read %s --token="" >>%s 2>&1;'
                         'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:%s %s -m)&\"' \
                         % (env_clean, brbe, self.be.bbport, local_bitbake, bbtargets, confpath, toasterlayers, log,
                         self.be.bbport, bitbake,)],
                         builddir, nowait=True)
+        else:
+            self._shellcmd(['%s bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:%s" '
+                        '%s %s -u toasterui  --token="" >>%s 2>&1;'
+                        'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:%s %s -m)&\"' \
+                        % (env_clean, brbe, self.be.bbport, local_bitbake, bbtargets, log,
+                        self.be.bbport, bitbake,)],
+                        builddir, nowait=True)
 
         logger.debug('localhostbecontroller: Build launched, exiting. '
                      'Follow build logs at %s' % log)
lib/toaster/bldcontrol/management/commands/runbuilds.py +1 -1
index 791e53e..6a55dd4 100644
@@ -49,7 +49,7 @@ class Command(BaseCommand):
                 # we could not find a BEC; postpone the BR
                 br.state = BuildRequest.REQ_QUEUED
                 br.save()
-                logger.debug("runbuilds: No build env")
+                logger.debug("runbuilds: No build env (%s)" % e)
                 return
 
             logger.info("runbuilds: starting build %s, environment %s" %
lib/toaster/orm/migrations/0018_project_specific.py new file +28 -0
new file mode 100644
index 0000000..084ecad
@@ -0,0 +1,28 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('orm', '0017_distro_clone'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='Project',
+            name='builddir',
+            field=models.TextField(),
+        ),
+        migrations.AddField(
+            model_name='Project',
+            name='merged_attr',
+            field=models.BooleanField(default=False)
+        ),
+        migrations.AddField(
+            model_name='Build',
+            name='progress_item',
+            field=models.CharField(max_length=40)
+        ),
+    ]
lib/toaster/orm/models.py +55 -6

102 lines not on this page · raw message

lib/toaster/toastergui/api.py +166 -2

219 lines not on this page · raw message

lib/toaster/toastergui/static/js/layerBtn.js +12 -0
index 9f9eda1..a5a6563 100644
@@ -67,6 +67,18 @@ function layerBtnsInit() {
       });
   });
 
+  $("td .set-default-recipe-btn").unbind('click');
+  $("td .set-default-recipe-btn").click(function(e){
+    e.preventDefault();
+    var recipe = $(this).data('recipe-name');
+
+    libtoaster.setDefaultImage(null, recipe,
+      function(){
+        /* Success */
+        window.location.replace(libtoaster.ctx.projectSpecificPageUrl);
+      });
+  });
+
 
   $(".customise-btn").unbind('click');
   $(".customise-btn").click(function(e){
lib/toaster/toastergui/static/js/libtoaster.js +105 -0

117 lines not on this page · raw message

lib/toaster/toastergui/static/js/mrbsection.js +3 -1
index c0c5fa9..f07ccf8 100644
@@ -86,7 +86,7 @@ function mrbSectionInit(ctx){
           if (buildFinished(build)) {
             // a build finished: reload the whole page so that the build
             // shows up in the builds table
-            window.location.reload();
+            window.location.reload(true);
           }
           else if (stateChanged(build)) {
             // update the whole template
@@ -110,6 +110,8 @@ function mrbSectionInit(ctx){
             // update the clone progress text
             selector = '#repos-cloned-percentage-' + build.id;
             $(selector).html(build.repos_cloned_percentage);
+            selector = '#repos-cloned-progressitem-' + build.id;
+            $(selector).html('('+build.progress_item+')');
 
             // update the recipe progress bar
             selector = '#repos-cloned-percentage-bar-' + build.id;
lib/toaster/toastergui/static/js/projecttopbar.js +22 -0
index 69220aa..3f9e186 100644
@@ -14,6 +14,9 @@ function projectTopBarInit(ctx) {
   var newBuildTargetBuildBtn = $("#build-button");
   var selectedTarget;
 
+  var updateProjectBtn = $("#update-project-button");
+  var cancelProjectBtn = $("#cancel-project-button");
+
   /* Project name change functionality */
   projectNameFormToggle.click(function(e){
     e.preventDefault();
@@ -89,6 +92,25 @@ function projectTopBarInit(ctx) {
     }, null);
   });
 
+  updateProjectBtn.click(function (e) {
+    e.preventDefault();
+
+    selectedTarget = { name: "_PROJECT_PREPARE_" };
+
+    /* Save current default build image, fire off the build */
+    libtoaster.updateProject(null, selectedTarget.name, newBuildTargetInput.val().trim(),
+      function(){
+        window.location.replace(libtoaster.ctx.projectSpecificPageUrl);
+    }, null);
+  });
+
+  cancelProjectBtn.click(function (e) {
+    e.preventDefault();
+
+    /* redirect to 'done/canceled' landing page */
+    window.location.replace(libtoaster.ctx.landingSpecificCancelURL);
+  });
+
   /* Call makeProjectNameValidation function */
   libtoaster.makeProjectNameValidation($("#project-name-change-input"),
       $("#hint-error-project-name"), $("#validate-project-name"),
lib/toaster/toastergui/tables.py +4 -0
index dca2fa2..03bd2ae 100644
@@ -35,6 +35,8 @@ from toastergui.tablefilter import TableFilterActionToggle
 from toastergui.tablefilter import TableFilterActionDateRange
 from toastergui.tablefilter import TableFilterActionDay
 
+import os
+
 class ProjectFilters(object):
     @staticmethod
     def in_project(project_layers):
@@ -339,6 +341,8 @@ class RecipesTable(ToasterTable):
             'filter_name' : "in_current_project",
             'static_data_name' : "add-del-layers",
             'static_data_template' : '{% include "recipe_btn.html" %}'}
+    if '1' == os.environ.get('TOASTER_PROJECTSPECIFIC'):
+            build_col['static_data_template'] = '{% include "recipe_add_btn.html" %}'
 
     def get_context_data(self, **kwargs):
         project = Project.objects.get(pk=kwargs['pid'])
lib/toaster/toastergui/templates/base_specific.html new file +128 -0

128 lines not on this page · raw message

lib/toaster/toastergui/templates/baseprojectspecificpage.html new file +48 -0

48 lines not on this page · raw message

lib/toaster/toastergui/templates/generic-toastertable-page.html +1 -1
index b3eabe1..99fbb38 100644
@@ -1,4 +1,4 @@
-{% extends "baseprojectpage.html" %}
+{% extends project_specific|yesno:"baseprojectspecificpage.html,baseprojectpage.html" %}
 {% load projecttags %}
 {% load humanize %}
 {% load static %}
lib/toaster/toastergui/templates/importlayer.html +2 -2
index 97d52c7..e0c987e 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends project_specific|yesno:"baseprojectspecificpage.html,base.html" %}
 {% load projecttags %}
 {% load humanize %}
 {% load static %}
@@ -6,7 +6,7 @@
 {% block pagecontent %}
 
 <div class="row">
-  {% include "projecttopbar.html" %}
+  {% include project_specific|yesno:"project_specific_topbar.html,projecttopbar.html" %}
   {% if project and project.release %}
   <script src="{% static 'js/layerDepsModal.js' %}"></script>
   <script src="{% static 'js/importlayer.js' %}"></script>
lib/toaster/toastergui/templates/landing_specific.html new file +50 -0

50 lines not on this page · raw message

lib/toaster/toastergui/templates/layerdetails.html +2 -1
index e0069db..1e26e31 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends project_specific|yesno:"baseprojectspecificpage.html,base.html" %}
 {% load projecttags %}
 {% load humanize %}
 {% load static %}
@@ -310,6 +310,7 @@
             {% endwith %}
             {% endwith %}
           </div>
+
         </div> <!-- end tab content -->
       </div> <!-- end tabable -->
 
lib/toaster/toastergui/templates/mrb_section.html +1 -1
index c5b9fe9..98d9fac 100644
@@ -119,7 +119,7 @@
           title="Toaster is cloning the repos required for your build">
     </span>
 
-    Cloning <span id="repos-cloned-percentage-<%:id%>"><%:repos_cloned_percentage%></span>% complete
+    Cloning <span id="repos-cloned-percentage-<%:id%>"><%:repos_cloned_percentage%></span>% complete <span id="repos-cloned-progressitem-<%:id%>">(<%:progress_item%>)</span>
 
     <%include tmpl='#cancel-template'/%>
   </div>
lib/toaster/toastergui/templates/newcustomimage.html +2 -2
index 980179a..0766e5e 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends project_specific|yesno:"baseprojectspecificpage.html,base.html" %}
 {% load projecttags %}
 {% load humanize %}
 {% load static %}
@@ -8,7 +8,7 @@
 
 <div class="row">
 
-  {% include "projecttopbar.html" %}
+  {% include project_specific|yesno:"project_specific_topbar.html,projecttopbar.html" %}
 
   <div class="col-md-12">
     {% url table_name project.id as xhr_table_url %}
lib/toaster/toastergui/templates/newproject_specific.html new file +95 -0

95 lines not on this page · raw message

lib/toaster/toastergui/templates/project.html +5 -2
index 11603d1..fa41e3c 100644
@@ -1,4 +1,4 @@
-{% extends "baseprojectpage.html" %}
+{% extends project_specific|yesno:"baseprojectspecificpage.html,baseprojectpage.html" %}
 
 {% load projecttags %}
 {% load humanize %}
@@ -18,7 +18,7 @@
     try {
       projectPageInit(ctx);
     } catch (e) {
-      document.write("Sorry, An error has occurred loading this page");
+      document.write("Sorry, An error has occurred loading this page (project):"+e);
       console.warn(e);
     }
   });
@@ -93,6 +93,7 @@
       </form>
     </div>
 
+	{% if not project_specific %}
     <div class="well well-transparent">
       <h3>Most built recipes</h3>
 
@@ -105,6 +106,7 @@
       </ul>
       <button class="btn btn-primary" id="freq-build-btn" disabled="disabled">Build selected recipes</button>
     </div>
+    {% endif %}
 
     <div class="well well-transparent">
       <h3>Project release</h3>
@@ -157,5 +159,6 @@
       <ul class="list-unstyled lead" id="layers-in-project-list">
       </ul>
   </div>
+
 </div>
 {% endblock %}
lib/toaster/toastergui/templates/project_specific.html new file +162 -0

162 lines not on this page · raw message

lib/toaster/toastergui/templates/project_specific_topbar.html new file +80 -0

80 lines not on this page · raw message

lib/toaster/toastergui/templates/projectconf.html +5 -2
index 933c588..fb20b26 100644
@@ -1,4 +1,4 @@
-{% extends "baseprojectpage.html" %}
+{% extends project_specific|yesno:"baseprojectspecificpage.html,baseprojectpage.html" %}
 {% load projecttags %}
 {% load humanize %}
 
@@ -438,8 +438,11 @@ function onEditPageUpdate(data) {
         var_context='m';
       }
     }
+    if (configvars_sorted[i][0].startsWith("INTERNAL_")) {
+        var_context='m';
+    }
     if (var_context == undefined) {
-      orightml += '<dt><span id="config_var_entry_'+configvars_sorted[i][2]+'" class="js-config-var-name"></span><span class="glyphicon glyphicon-trash js-icon-trash-config_var" id="config_var_trash_'+configvars_sorted[i][2]+'" x-data="'+configvars_sorted[i][2]+'"></span> </dt>'
+        orightml += '<dt><span id="config_var_entry_'+configvars_sorted[i][2]+'" class="js-config-var-name"></span><span class="glyphicon glyphicon-trash js-icon-trash-config_var" id="config_var_trash_'+configvars_sorted[i][2]+'" x-data="'+configvars_sorted[i][2]+'"></span> </dt>'
         orightml += '<dd class="variable-list">'
         orightml += '    <span class="lead" id="config_var_value_'+configvars_sorted[i][2]+'"></span>'
         orightml += '    <span class="glyphicon glyphicon-edit js-icon-pencil-config_var" x-data="'+configvars_sorted[i][2]+'"></span>'
lib/toaster/toastergui/templates/recipe_add_btn.html new file +23 -0
new file mode 100644
index 0000000..06c4645
@@ -0,0 +1,23 @@
+<a data-recipe-name="{{data.name}}" class="btn btn-default btn-block layer-exists-{{data.layer_version.pk}} set-default-recipe-btn" style="margin-top: 5px;
+  {% if data.layer_version.pk not in extra.current_layers %}
+    display:none;
+  {% endif %}"
+ >
+  Set recipe
+</a>
+<a class="btn btn-default btn-block layerbtn layer-add-{{data.layer_version.pk}}"
+  data-layer='{
+  "id": {{data.layer_version.pk}},
+  "name":  "{{data.layer_version.layer.name}}",
+  "layerdetailurl": "{%url "layerdetails" extra.pid data.layer_version.pk%}",
+   "xhrLayerUrl": "{% url "xhr_layer" extra.pid data.layer_version.pk %}"
+   }' data-directive="add"
+    {% if data.layer_version.pk in extra.current_layers %}
+     style="display:none;"
+    {% endif %}
+>
+  <span class="glyphicon glyphicon-plus"></span>
+  Add layer
+  <span class="glyphicon glyphicon-question-sign get-help" title="To set this
+      recipe you must first add the {{data.layer_version.layer.name}} layer to your project"></i>
+</a>
lib/toaster/toastergui/urls.py +13 -0
index e07b0ef..dc03e30 100644
@@ -116,6 +116,11 @@ urlpatterns = [
             tables.ProjectBuildsTable.as_view(template_name="projectbuilds-toastertable.html"),
             name='projectbuilds'),
 
+        url(r'^newproject_specific/(?P<pid>\d+)/$', views.newproject_specific, name='newproject_specific'),
+        url(r'^project_specific/(?P<pid>\d+)/$', views.project_specific, name='project_specific'),
+        url(r'^landing_specific/(?P<pid>\d+)/$', views.landing_specific, name='landing_specific'),
+        url(r'^landing_specific_cancel/(?P<pid>\d+)/$', views.landing_specific_cancel, name='landing_specific_cancel'),
+
         # the import layer is a project-specific functionality;
         url(r'^project/(?P<pid>\d+)/importlayer$', views.importlayer, name='importlayer'),
 
@@ -233,6 +238,14 @@ urlpatterns = [
             api.XhrBuildRequest.as_view(),
             name='xhr_buildrequest'),
 
+        url(r'^xhr_projectupdate/project/(?P<pid>\d+)$',
+            api.XhrProjectUpdate.as_view(),
+            name='xhr_projectupdate'),
+
+        url(r'^xhr_setdefaultimage/project/(?P<pid>\d+)$',
+            api.XhrSetDefaultImageUrl.as_view(),
+            name='xhr_setdefaultimage'),
+
         url(r'xhr_project/(?P<project_id>\d+)$',
             api.XhrProject.as_view(),
             name='xhr_project'),
lib/toaster/toastergui/views.py +151 -0

181 lines not on this page · raw message

lib/toaster/toastergui/widgets.py +6 -0
index a1792d9..88dff8a 100644
@@ -89,6 +89,10 @@ class ToasterTable(TemplateView):
 
         # global variables
         context['project_enable'] = ('1' == os.environ.get('TOASTER_BUILDSERVER'))
+        try:
+            context['project_specific'] = ('1' == os.environ.get('TOASTER_PROJECTSPECIFIC'))
+        except:
+            context['project_specific'] = ''
 
         return context
 
@@ -519,6 +523,8 @@ class MostRecentBuildsView(View):
                 int((build_obj.repos_cloned /
                      build_obj.repos_to_clone) * 100)
 
+            build['progress_item'] = build_obj.progress_item
+
             tasks_complete_percentage = 0
             if build_obj.outcome in (Build.SUCCEEDED, Build.FAILED):
                 tasks_complete_percentage = 100
lib/toaster/toastermain/management/commands/builddelete.py +5 -1
index 0bef8d4..bf69a8f 100644
@@ -10,8 +10,12 @@ class Command(BaseCommand):
     args    = '<buildID1 buildID2 .....>'
     help    = "Deletes selected build(s)"
 
+    def add_arguments(self, parser):
+        parser.add_argument('buildids', metavar='N', type=int, nargs='+',
+                    help="Build ID's to delete")
+
     def handle(self, *args, **options):
-        for bid in args:
+        for bid in options['buildids']:
             try:
                 b = Build.objects.get(pk = bid)
             except ObjectDoesNotExist:
lib/toaster/toastermain/management/commands/buildimport.py new file +586 -0

586 lines not on this page · raw message

Signature · 1 line
1.9.1