Skip to contentSkip to conversation

Toaster patchset

PATCH In review Alex DAMIAN 2 participants 4 messages Jan 30, 2015

lore
Share & tools
lore
#0 lore raw

cover Toaster patchset

From: Alexandru DAMIAN <[email protected]>

Hi,

This is a small patchset for toaster for fixing the startup sequence, and some build issues.

It has been reviewed on toaster mailing list, can you please pull when you can ?

Cheers,
Alex

The following changes since commit 9facf3604759b00e8fe99f929353d46f8b8ba5cb:

data_smart: Don't use mutable objects as default args (2015-01-29 22:10:53 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib adamian/20150130-submission-bb
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=adamian/20150130-submission-bb

Alexandru DAMIAN (2):
toaster: update texts for the startup sequence
toaster: new layer checkout logic

 bin/toaster                                        |   6 +-
 lib/toaster/bldcontrol/bbcontroller.py             |  13 ++-
 lib/toaster/bldcontrol/localhostbecontroller.py    | 107 +++++++++++++++------
 .../management/commands/checksettings.py           |  21 ++--
 .../bldcontrol/management/commands/loadconf.py     |   2 +-
 .../bldcontrol/management/commands/runbuilds.py    |   7 +-
 lib/toaster/bldcontrol/sshbecontroller.py          |  58 +----------
 7 files changed, 110 insertions(+), 104 deletions(-)
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/2 toaster: update texts for the startup sequence

From: Alexandru DAMIAN <[email protected]>

This patch updates the prompts to the user to better explain what Toaster is doing and the information it needs from the user.

Additionally, fixes a check in loadconf command.

[YOCTO #6785]
[YOCTO #7251]

---
 bin/toaster                                         |  6 +++---
 .../bldcontrol/management/commands/checksettings.py | 21 ++++++++++++++-------
 .../bldcontrol/management/commands/loadconf.py      |  2 +-
 3 files changed, 18 insertions(+), 11 deletions(-)
bin/toaster +3 -3
index a6d2e92..4f25293 100755
@@ -68,7 +68,7 @@ function webserverStartAll()
 		python $BBBASEDIR/lib/toaster/manage.py checksettings  --traceback || retval=1
 	fi
         if [ $retval -eq 0 ]; then
-	    echo "Starting webserver"
+	    echo "Starting webserver..."
             python $BBBASEDIR/lib/toaster/manage.py runserver "0.0.0.0:$WEB_PORT" </dev/null >${BUILDDIR}/toaster_web_$$.log 2>&1 & echo $! >${BUILDDIR}/.toastermain.pid
             sleep 1
             if ! cat "${BUILDDIR}/.toastermain.pid" | xargs -I{} kill -0 {} ; then
@@ -171,7 +171,7 @@ if [ -z "$ZSH_NAME" ] && [ `basename \"$0\"` = `basename \"$BASH_SOURCE\"` ]; th
     # Start just the web server, point the web browser to the interface, and start any Django services.
 
     if [ -n "$BUILDDIR" ]; then
-        echo -e "Error: build/ directory detected. Toaster will not start in managed mode if a build environment is detected.\nUse a clean terminal to start Toaster." 1>&2;
+	echo -e "Error: It looks like you sourced oe-init-build-env. Toaster cannot start in build mode from an oe-core build environment. You can work around this problem by starting Toaster from a new terminal window." 1>&2;
         exit 1;
     fi
 
@@ -194,7 +194,7 @@ if [ -z "$ZSH_NAME" ] && [ `basename \"$0\"` = `basename \"$BASH_SOURCE\"` ]; th
         xdg-open http://127.0.0.1:$WEB_PORT/ >/dev/null 2>&1 &
     fi
     trap trap_ctrlc SIGINT
-    echo "Running. Stop with Ctrl-C"
+    echo "Toaster is now running. You can stop it with Ctrl-C"
     while [ $RUNNING -gt 0 ]; do
 	python $BBBASEDIR/lib/toaster/manage.py runbuilds
 	sleep 1
lib/toaster/bldcontrol/management/commands/checksettings.py +14 -7
index 55f118c..cf3c3d7 100644
@@ -80,27 +80,33 @@ class Command(NoArgsCommand):
         for be in BuildEnvironment.objects.all():
             def _verify_be():
                 is_changed = False
-                print("Verifying the Build Environment type %s id %d." % (be.get_betype_display(), be.pk))
+                print("Verifying the Build Environment. If the Build Environment is completly configured, you will be asked to configure it.")
                 if len(be.sourcedir) == 0:
                     suggesteddir = self._get_suggested_sourcedir(be)
-                    be.sourcedir = raw_input(" -- Layer sources checkout directory may not be empty [guessed \"%s\"]:" % suggesteddir)
+                    if len(suggesteddir) > 0:
+                        be.sourcedir = raw_input("\nToaster needs to know in which directory it should check out the layers that will be needed for your builds.\n Toaster suggests \"%s\". If you select this directory, a layer like \"meta-yocto\" will end up in \"%s/meta-yocto\".\n Press Enter to select \"%s\" or type the full path to a different directory: " % (suggesteddir, suggesteddir, suggesteddir))
+                    else:
+                        be.sourcedir = raw_input("\nToaster needs to know in which directory it should check out the layers that will be needed for your builds. Type the full path to the directory (for example: \"%s\": " % os.environment['HOME'])
                     if len(be.sourcedir) == 0 and len(suggesteddir) > 0:
                         be.sourcedir = suggesteddir
                     is_changed = True
 
                 if not be.sourcedir.startswith("/"):
-                    be.sourcedir = raw_input(" -- Layer sources checkout directory must be an absolute path:")
+                    be.sourcedir = raw_input(" Layer sources checkout directory must be an absolute path:")
                     is_changed = True
 
                 if len(be.builddir) == 0:
                     suggesteddir = self._get_suggested_builddir(be)
-                    be.builddir = raw_input(" -- Build directory may not be empty [guessed \"%s\"]:" % suggesteddir)
+                    if len(suggesteddir) > 0:
+                        be.builddir = raw_input("\nToaster needs to know where is your build directory.\n The build directory is where all the artifacts created by your builds will be stored. Toaster suggests \"%s\".\n Press Enter to select \"%s\" or type the full path to a different directory: " % (suggesteddir, suggesteddir))
+                    else:
+                        be.builddir = raw_input("\nToaster needs to know where is your build directory.\n The build directory is where all the artifacts created by your builds will be stored. Type the full path to the directory (for example: \" %s/build\")" % os.environment['HOME'])
                     if len(be.builddir) == 0 and len(suggesteddir) > 0:
                         be.builddir = suggesteddir
                     is_changed = True
 
                 if not be.builddir.startswith("/"):
-                    be.builddir = raw_input(" -- Build directory must be an absolute path:")
+                    be.builddir = raw_input(" Build directory must be an absolute path:")
                     is_changed = True
 
 
@@ -109,6 +115,7 @@ class Command(NoArgsCommand):
                     be.save()
 
                 if is_changed and be.betype == BuildEnvironment.TYPE_LOCAL:
+                    print "\nToaster can use a SINGLE predefined configuration file to set up default project settings and layer information sources.\n Toaster will list now the configuration files that it found. Select Yes to use the desired configuration file."
                     for dirname in self._recursive_list_directories(be.sourcedir,2):
                         if os.path.exists(os.path.join(dirname, ".templateconf")):
                             import subprocess
@@ -116,13 +123,13 @@ class Command(NoArgsCommand):
                             conffilepath = os.path.join(conffilepath.strip(), "toasterconf.json")
                             candidatefilepath = os.path.join(dirname, conffilepath)
                             if os.path.exists(candidatefilepath):
-                                i = raw_input(" -- Do you want to import basic layer configuration from \"%s\" ? (y/N):" % candidatefilepath)
+                                i = raw_input("\n Found an preset configuration file \"%s\".\n Do you want to use it? (y/N):" % candidatefilepath)
                                 if len(i) and i.upper()[0] == 'Y':
                                     from loadconf import Command as LoadConfigCommand
 
                                     LoadConfigCommand()._import_layer_config(candidatefilepath)
                                     # we run lsupdates after config update
-                                    print "Layer configuration imported. Updating information from the layer source, please wait."
+                                    print "Layer configuration imported. Updating information from the layer sources, please wait.\n You can re-update any time later by running bitbake/lib/toaster/manage.py lsupdates"
                                     from django.core.management import call_command
                                     call_command("lsupdates")
 
lib/toaster/bldcontrol/management/commands/loadconf.py +1 -1
index 2257a71..e2f61e4 100644
@@ -22,7 +22,7 @@ def _get_id_for_sourcetype(s):
     for i in LayerSource.SOURCE_TYPE:
         if s == i[1]:
             return i[0]
-    raise Exception("Could not find definition for sourcetype " + s)
+    raise Exception("Could not find definition for sourcetype '%s'. Valid source types are %s" % (str(s), ', '.join(map(lambda x: "'%s'" % x[1], LayerSource.SOURCE_TYPE ))))
 
 class Command(BaseCommand):
     help = "Loads a toasterconf.json file in the database"
Signature · 1 line
1.9.1
#2 lore raw

PATCH 2/2 toaster: new layer checkout logic

From: Alexandru DAMIAN <[email protected]>

This patch implements a new layer checkout logic that brings more flexibility in getting layers under different commits work with Toaster.

The new hibrid logic will checkout separately each layer and commit id; the task execution will be delegated to the checkedout bitbake, but the data logger will be executed from the current toaster version as to bring in enough data to sustain the updates in UI models.

[YOCTO #7171]
[YOCTO #6892]

---
 lib/toaster/bldcontrol/bbcontroller.py             |  13 ++-
 lib/toaster/bldcontrol/localhostbecontroller.py    | 107 +++++++++++++++------
 .../bldcontrol/management/commands/runbuilds.py    |   7 +-
 lib/toaster/bldcontrol/sshbecontroller.py          |  58 +----------
 4 files changed, 92 insertions(+), 93 deletions(-)
lib/toaster/bldcontrol/bbcontroller.py +9 -4
index 102606e..dbfb2f3 100644
@@ -81,12 +81,17 @@ def getBuildEnvironmentController(**kwargs):
         raise Exception("FIXME: Implement BEC for type %s" % str(be.betype))
 
 
-def _getgitcheckoutdirectoryname(url):
+def _get_git_clonedirectory(url, branch):
     """ Utility that returns the last component of a git path as directory
     """
     import re
     components = re.split(r'[:\.\/]', url)
-    return components[-2] if components[-1] == "git" else components[-1]
+    base = components[-2] if components[-1] == "git" else components[-1]
+
+    if branch != "HEAD":
+        return "_%s_%s.toaster_cloned" % (base, branch)
+
+    return base
 
 
 class BuildEnvironmentController(object):
@@ -166,12 +171,12 @@ class BuildEnvironmentController(object):
         raise Exception("Must override setLayers")
 
 
-    def getBBController(self, brbe):
+    def getBBController(self):
         """ returns a BitbakeController to an already started server; this is the point where the server
             starts if needed; or reconnects to the server if we can
         """
         if not self.connection:
-            self.startBBServer(brbe)
+            self.startBBServer()
             self.be.lock = BuildEnvironment.LOCK_RUNNING
             self.be.save()
 
lib/toaster/bldcontrol/localhostbecontroller.py +76 -31

188 lines not on this page · raw message

lib/toaster/bldcontrol/management/commands/runbuilds.py +5 -2
index bdce6ee..23ee855 100644
@@ -2,7 +2,7 @@ from django.core.management.base import NoArgsCommand, CommandError
 from django.db import transaction
 from orm.models import Build
 from bldcontrol.bbcontroller import getBuildEnvironmentController, ShellCmdException, BuildSetupException
-from bldcontrol.models import BuildRequest, BuildEnvironment, BRError
+from bldcontrol.models import BuildRequest, BuildEnvironment, BRError, BRVariable
 import os
 import logging
 
@@ -47,6 +47,9 @@ class Command(NoArgsCommand):
                 return
 
             logger.debug("runbuilds: starting build %s, environment %s" % (br, bec.be))
+
+            # write the build identification variable
+            BRVariable.objects.create(req = br, name="TOASTER_BRBE", value="%d:%d" % (br.pk, bec.be.pk))
             # let the build request know where it is being executed
             br.environment = bec.be
             br.save()
@@ -56,7 +59,7 @@ class Command(NoArgsCommand):
             bec.writePreConfFile(br.brvariable_set.all())
 
             # get the bb server running with the build req id and build env id
-            bbctrl = bec.getBBController("%d:%d" % (br.pk, bec.be.pk))
+            bbctrl = bec.getBBController()
 
             # trigger the build command
             task = reduce(lambda x, y: x if len(y)== 0 else y, map(lambda y: y.task, br.brtarget_set.all()))
lib/toaster/bldcontrol/sshbecontroller.py +2 -56

71 lines not on this page · raw message

Signature · 1 line
1.9.1
#3 lore raw
In reply to Alex DAMIAN

does Dizzy need this?

- Armin

Quoted 33 lines · Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Hi,

This is a small patchset for toaster for fixing the startup sequence, and some build issues.

It has been reviewed on toaster mailing list, can you please pull when you can ?

Cheers,
Alex

The following changes since commit 9facf3604759b00e8fe99f929353d46f8b8ba5cb:

   data_smart: Don't use mutable objects as default args (2015-01-29 22:10:53 +0000)

are available in the git repository at:

   git://git.yoctoproject.org/poky-contrib adamian/20150130-submission-bb
   http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=adamian/20150130-submission-bb

Alexandru DAMIAN (2):
toaster: update texts for the startup sequence
toaster: new layer checkout logic

  bin/toaster                                        |   6 +-
  lib/toaster/bldcontrol/bbcontroller.py             |  13 ++-
  lib/toaster/bldcontrol/localhostbecontroller.py    | 107 +++++++++++++++------
  .../management/commands/checksettings.py           |  21 ++--
  .../bldcontrol/management/commands/loadconf.py     |   2 +-
  .../bldcontrol/management/commands/runbuilds.py    |   7 +-
  lib/toaster/bldcontrol/sshbecontroller.py          |  58 +----------
  7 files changed, 110 insertions(+), 104 deletions(-)