Skip to contentSkip to conversation

Webhob patches for DSI and Web modules

PATCH In review Alex DAMIAN 2 participants 96 messages Sep 24, 2013 Page 1 of 2

lore
Share & tools
lore
#0 lore raw

cover Webhob patches for DSI and Web modules

From: Alexandru DAMIAN <[email protected]>

Patches for the DSI and Web modules of Webhob, in the order of development.

DSI is the DataStore Interface, the component that records information derived from a Bitbake build into the data backend.

Web module consists of the Simple UI interface, and the API interface available to application for data query.

External components jQuery and Bootstrap are included in the sources, and respective licenses documented (APACHE-2.0 and MIT).

The following changes since commit 0fc8317c6385eb1ed69ca4522ee6424c456dbb92:

yocto-bsp: add 3.10/remove 3.8 kernel from templates (2013-09-24 12:01:47 +0100)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib webhob-poky/webhob-master-1_5_upstream_2509
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=webhob-poky/webhob-master-1_5_upstream_2509

Alexandru DAMIAN (72):
webhob: add system start/stop script
bitbake: webhob: set sane default settings
bitbake: webhob: adds dsi support for observer mode
bitbake: dsi: Translate runQueue events into Task data
bitbake: webhob: disable synchronous sqlite connection
bitbake: buildinfohelper: clean-up the imports
bitbake: buildinfohelper: clearing up task order
bitbake: dsi: saving prebuild task detailed information
bitbake: dsi: clear up the state on build completion
bitbake: dsi: event data change
bitbake: webhob: fix and cleanup start script
bitbake: dsi: refactor the BuildInfoHelper code
bitbake: webhob: record dependency info
bitbake: webhob: use defined constants for models
bitbake: dsi: fix the reading of task event information
bitbake: webhob: orm change to remove Target
bitbake: webhob: clear up ORM relations
bitbake: dsi: save detailed recipe information
bitbake: webhob: determine if a build is an image
bitbake: webhob: clean up URL structure
bitbake: webhob: add simple pages for Recipes
bitbake: webhob: enhance Simple browser navigation
bitbake: webhob: improve startup script
bitbake: webhob: extend search for multiple terms
bitbake: webhob: force bitbake server stop
bitbake: webhob: simple interface dependency list
bitbake: dsi: add feature to store package information
bitbake: webhob: add simple viewer for package information
bitbake: dsi: fix build stats data gathering
bitbake: dsi: update build object on command end
bitbake: dsi: fix sstate task information gathering
bitbake: webhob: clean up starting script
bitbake: dsi: store log information
bitbake: webhob: add simple visualisation for build errors
bitbake: webhob: store logfile and message for Tasks
bitbake: webhob: display proper information in Simple
bitbake: dsi: clear up global variables between builds
bitbake: dsi: small bugfixes in data collection
bitbake: bldviewer: add jquery and bootstrap frameworks
bitbake: webhob: add toggle column functionality to build page
bitbake: webhob: add search functionality to the Simple interface
bitbake: webhob: refactor column hiding code
bitbake: webhob: improve search functionality
bitbake: webhob: refactor Simple web interface
bitbake: webhob: simple interface CSS
bitbake: dsi: use get vars command to store configuration
bitbake: webhob: refactor CSS display in Simple
bitbake: webhob: add Configuration visualisation in Simple
bitbake: webhob: add navigation links Simple interface
bitbake: webhob: startup script fixing
bitbake: webhob: change database models and related
bitbake: webhob: navigation in the Simple interface
bitbake: dsi: save build-time package information
bitbake: webhob: Simple visualisation for the build-time packages
bitbake: webhob: store file size information
bitbake: webhob: simple visualisation for package files
bitbake: dsi: record recipe dependency
bitbake: dsi: add exception dumping code
bitbake: webhob: better help message in startup script
bitbake: dsi: add build package dependency recording
bitbake: webhob: clean up Machine table
bitbake: webhob: update API endpoints
bitbake: webhob: record task hash information
bitbake: cooker: add the inherits attribute to the dependency tree
bitbake: webhob: Clean up links in Simple UI
bitbake: dsi: read correct recipe description field
bitbake: cooker: send layer information with dependency tree
bitbake: webhob: store and display layer priorities
bitbake: webhob: collect recipe licensing info
bitbake: webhob: Simple UI interface CSS fix
bitbake: dsi: fix typos
bitbake: webhob: do not set timezones

Calin Dragomir (22):
webhob: create main WEBHOB project
webhob: create Django models for webhob
bitbake: create Data Store Interface (DSI) file
bitbake: webhob: updates to Django models
bitbake: webhob: make DSI store build information
bitbake: webhob: make DSI store task information
bitbake: webhob: view a table with all builds
bitbake: webhob: view a table with all the tasks of a build
bitbake: webhob: use buildcompleted event as the end of a build
operation
bitbake: webhob: changes to build information table
bitbake: webhob: gather buildstats for each executed task
bitbake: webhob: add layer information to the database
bitbake: webhob: mark private methods inside DSI
bitbake: webhob: create basic structure for static files
bitbake: webhob: Setup API for build model
bitbake: webhob: add ordering capabilities to build api
bitbake: webhob: validate inputs for build api
bitbake: webhob: generic view for multiple models
bitbake: webhob: improve validation code flow
bitbake: webhob: add more models to webhob API
bitbake: webhob: add search for build model
bitbake: webhob: fix ordering issue

 bitbake/bin/webhob                                 |  145 +
 bitbake/lib/bb/cooker.py                           |    2 +
 bitbake/lib/bb/ui/buildinfohelper.py               |  678 ++
 bitbake/lib/bb/ui/dsi.py                           |  569 ++
 bitbake/lib/webhob/__init__.py                     |    0
 bitbake/lib/webhob/bldviewer/__init__.py           |    0
 bitbake/lib/webhob/bldviewer/api.py                |   19 +
 .../lib/webhob/bldviewer/static/css/bootstrap.css  | 4797 +++++++++++
 .../lib/webhob/bldviewer/static/js/bootstrap.js    | 1982 +++++
 .../lib/webhob/bldviewer/static/js/jquery-2.0.3.js | 8829 ++++++++++++++++++++
 bitbake/lib/webhob/bldviewer/templates/base.html   |   30 +
 .../webhob/bldviewer/templates/basebuildpage.html  |   17 +
 .../lib/webhob/bldviewer/templates/basetable.html  |   46 +
 bitbake/lib/webhob/bldviewer/templates/bfile.html  |   24 +
 .../lib/webhob/bldviewer/templates/bpackage.html   |   44 +
 bitbake/lib/webhob/bldviewer/templates/build.html  |   45 +
 .../webhob/bldviewer/templates/configuration.html  |   20 +
 bitbake/lib/webhob/bldviewer/templates/layer.html  |   34 +
 .../lib/webhob/bldviewer/templates/package.html    |   36 +
 bitbake/lib/webhob/bldviewer/templates/recipe.html |   54 +
 bitbake/lib/webhob/bldviewer/templates/task.html   |   63 +
 .../lib/webhob/bldviewer/templatetags/__init__.py  |    0
 .../webhob/bldviewer/templatetags/projecttags.py   |    8 +
 bitbake/lib/webhob/bldviewer/urls.py               |   14 +
 bitbake/lib/webhob/bldviewer/views.py              |  220 +
 bitbake/lib/webhob/manage.py                       |   10 +
 bitbake/lib/webhob/orm/__init__.py                 |    0
 bitbake/lib/webhob/orm/models.py                   |  242 +
 bitbake/lib/webhob/orm/tests.py                    |   16 +
 bitbake/lib/webhob/orm/views.py                    |    1 +
 bitbake/lib/webhob/whbgui/__init__.py              |    0
 bitbake/lib/webhob/whbgui/static/images/yocto.jpg  |  Bin 0 -> 6582 bytes
 bitbake/lib/webhob/whbgui/templates/index.html     |   13 +
 bitbake/lib/webhob/whbgui/tests.py                 |   16 +
 bitbake/lib/webhob/whbgui/urls.py                  |    9 +
 bitbake/lib/webhob/whbgui/views.py                 |    8 +
 bitbake/lib/webhob/whbmain/__init__.py             |    0
 bitbake/lib/webhob/whbmain/settings.py             |  173 +
 bitbake/lib/webhob/whbmain/urls.py                 |   22 +
 bitbake/lib/webhob/whbmain/wsgi.py                 |   32 +
 40 files changed, 18218 insertions(+)
 create mode 100755 bitbake/bin/webhob
 create mode 100644 bitbake/lib/bb/ui/buildinfohelper.py
 create mode 100644 bitbake/lib/bb/ui/dsi.py
 create mode 100644 bitbake/lib/webhob/__init__.py
 create mode 100644 bitbake/lib/webhob/bldviewer/__init__.py
 create mode 100644 bitbake/lib/webhob/bldviewer/api.py
 create mode 100644 bitbake/lib/webhob/bldviewer/static/css/bootstrap.css
 create mode 100644 bitbake/lib/webhob/bldviewer/static/js/bootstrap.js
 create mode 100644 bitbake/lib/webhob/bldviewer/static/js/jquery-2.0.3.js
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/base.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/basebuildpage.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/basetable.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/bfile.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/bpackage.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/build.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/configuration.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/layer.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/package.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/recipe.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/task.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/__init__.py
 create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/projecttags.py
 create mode 100644 bitbake/lib/webhob/bldviewer/urls.py
 create mode 100644 bitbake/lib/webhob/bldviewer/views.py
 create mode 100755 bitbake/lib/webhob/manage.py
 create mode 100644 bitbake/lib/webhob/orm/__init__.py
 create mode 100644 bitbake/lib/webhob/orm/models.py
 create mode 100644 bitbake/lib/webhob/orm/tests.py
 create mode 100644 bitbake/lib/webhob/orm/views.py
 create mode 100644 bitbake/lib/webhob/whbgui/__init__.py
 create mode 100644 bitbake/lib/webhob/whbgui/static/images/yocto.jpg
 create mode 100644 bitbake/lib/webhob/whbgui/templates/index.html
 create mode 100644 bitbake/lib/webhob/whbgui/tests.py
 create mode 100644 bitbake/lib/webhob/whbgui/urls.py
 create mode 100644 bitbake/lib/webhob/whbgui/views.py
 create mode 100644 bitbake/lib/webhob/whbmain/__init__.py
 create mode 100644 bitbake/lib/webhob/whbmain/settings.py
 create mode 100644 bitbake/lib/webhob/whbmain/urls.py
 create mode 100644 bitbake/lib/webhob/whbmain/wsgi.py
Signature · 1 line
1.8.1.2
#1 lore raw

PATCH 1/94 webhob: create main WEBHOB project

From: Calin Dragomir <[email protected]>

This creates a basic Django project where the WEBHOB code will live. The main project had to be renamed from webhob to whbmain in order to avoid any import issues that might occur.

---
 bitbake/lib/webhob/__init__.py         |   0
 bitbake/lib/webhob/manage.py           |  10 +++
 bitbake/lib/webhob/whbmain/__init__.py |   0
 bitbake/lib/webhob/whbmain/settings.py | 155 +++++++++++++++++++++++++++++++++
 bitbake/lib/webhob/whbmain/urls.py     |  17 ++++
 bitbake/lib/webhob/whbmain/wsgi.py     |  32 +++++++
 6 files changed, 214 insertions(+)
 create mode 100644 bitbake/lib/webhob/__init__.py
 create mode 100755 bitbake/lib/webhob/manage.py
 create mode 100644 bitbake/lib/webhob/whbmain/__init__.py
 create mode 100644 bitbake/lib/webhob/whbmain/settings.py
 create mode 100644 bitbake/lib/webhob/whbmain/urls.py
 create mode 100644 bitbake/lib/webhob/whbmain/wsgi.py
bitbake/lib/webhob/__init__.py new file +0 -0
new file mode 100644
index 0000000..e69de29
bitbake/lib/webhob/manage.py new file +10 -0
new file mode 100755
index 0000000..1b2c1a7
@@ -0,0 +1,10 @@
+#!/usr/bin/env python
+import os
+import sys
+
+if __name__ == "__main__":
+    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "whbmain.settings")
+
+    from django.core.management import execute_from_command_line
+
+    execute_from_command_line(sys.argv)
bitbake/lib/webhob/whbmain/__init__.py new file +0 -0
new file mode 100644
index 0000000..e69de29
bitbake/lib/webhob/whbmain/settings.py new file +155 -0

155 lines not on this page · raw message

bitbake/lib/webhob/whbmain/urls.py new file +17 -0
new file mode 100644
index 0000000..04e0352
@@ -0,0 +1,17 @@
+from django.conf.urls import patterns, include, url
+
+# Uncomment the next two lines to enable the admin:
+# from django.contrib import admin
+# admin.autodiscover()
+
+urlpatterns = patterns('',
+    # Examples:
+    # url(r'^$', 'webhob.views.home', name='home'),
+    # url(r'^webhob/', include('webhob.foo.urls')),
+
+    # Uncomment the admin/doc line below to enable admin documentation:
+    # url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
+
+    # Uncomment the next line to enable the admin:
+    # url(r'^admin/', include(admin.site.urls)),
+)
bitbake/lib/webhob/whbmain/wsgi.py new file +32 -0
new file mode 100644
index 0000000..9595eda
@@ -0,0 +1,32 @@
+"""
+WSGI config for webhob project.
+
+This module contains the WSGI application used by Django's development server
+and any production WSGI deployments. It should expose a module-level variable
+named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
+this application via the ``WSGI_APPLICATION`` setting.
+
+Usually you will have the standard Django WSGI application here, but it also
+might make sense to replace the whole Django WSGI application with a custom one
+that later delegates to the Django one. For example, you could introduce WSGI
+middleware here, or combine a Django application with an application of another
+framework.
+
+"""
+import os
+
+# We defer to a DJANGO_SETTINGS_MODULE already in the environment. This breaks
+# if running multiple sites in the same mod_wsgi process. To fix this, use
+# mod_wsgi daemon mode with each site in its own daemon process, or use
+# os.environ["DJANGO_SETTINGS_MODULE"] = "webhob.settings"
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "wbhmain.settings")
+
+# This application object is used by any WSGI server configured to use this
+# file. This includes Django's development server, if the WSGI_APPLICATION
+# setting points here.
+from django.core.wsgi import get_wsgi_application
+application = get_wsgi_application()
+
+# Apply WSGI middleware here.
+# from helloworld.wsgi import HelloWorldApplication
+# application = HelloWorldApplication(application)
Signature · 1 line
1.8.1.2
#2 lore raw

PATCH 2/94 webhob: add system start/stop script

From: Alexandru DAMIAN <[email protected]>

Adding a script that starts/stops the webhob system. Sourced, with a parameter, it will start or stop the bitbake resident, and the webhob server, as well as point a browser to the webhob home .

---
 bitbake/bin/webhob | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 95 insertions(+)
 create mode 100755 bitbake/bin/webhob
bitbake/bin/webhob new file +95 -0
new file mode 100755
index 0000000..ccaf91f
@@ -0,0 +1,95 @@
+#!/bin/bash
+# (c) 2013 Intel Corp.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+
+# This script enables webhob event logging and
+# starts bitbake resident server
+# use as:  source webhob [start|stop]
+
+# Helper function to kill a background webhob development server
+        
+function webserverKillAllComponents() 
+{
+	local pidfile
+	for pidfile in ${BUILDDIR}/whbmain.pid; do
+		if [ -f ${pidfile} ]; then
+		while kill -0 $(< ${pidfile}) 2>/dev/null; do
+			kill -SIGTERM -`ps -p $(< ${pidfile}) -o "%r" --no-headers`	
+			sleep 1;
+		done;
+		rm  ${pidfile}
+		fi
+	done
+}
+
+# We make sure we're running in the current shell and in a good environment
+
+if [ -z "$ZSH_NAME" ] && [ `basename "$0"` = `basename $BASH_SOURCE` ]; then
+    echo "Error: This script needs to be sourced. Please run as 'source webhob [start|stop]'" 1>&2;
+    exit 1
+fi
+    
+if [ -z "$BUILDDIR" ] || [ -z `which bitbake` ]; then
+    echo "Error: Build environment is not setup or bitbake is not in path." 1>&2;
+    return 2
+fi
+
+BBBASEDIR=`dirname ${BASH_SOURCE}`/..
+
+# Determine the action. If specified by arguments, fine, if not, toggle it
+if [ "x$1" == "xstart" ] || [ "x$1" == "xstop" ]; then
+    CMD="$1"
+else
+    if [ -z "$BBSERVER" ]; then
+        CMD="start"
+    else
+        CMD="stop"
+    fi;
+fi
+
+# Make sure it's safe to run by checking bitbake lock
+
+lock=1
+(flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0 
+
+if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); then
+    echo "Error: bitbake lock state error. System may be already on." 2>&1
+    return 3 
+elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then
+    echo "Error: bitbake lock state error. System may be already off.
+manually stop system with bitbake -m / webserverKillAllComponents" 2>&1
+    return 3 
+fi
+
+
+# Execute the commands
+
+case $CMD in
+    start )
+        unset BBSERVER
+        python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0)
+        bitbake --server-only -t xmlrpc -B localhost:8200 
+        export BBSERVER=localhost:8200
+        python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
+    ;;
+    stop )
+        bitbake -m
+        unset BBSERVER
+        webserverKillAllComponents
+esac
+
+    
Signature · 1 line
1.8.1.2
#3 lore raw

PATCH 3/94 webhob: create Django models for webhob

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

Patch adds the models we need in order to store all the information we need about the builds in the database using the Django ORM.

---
 bitbake/lib/webhob/orm/__init__.py     |   0
 bitbake/lib/webhob/orm/models.py       | 162 +++++++++++++++++++++++++++++++++
 bitbake/lib/webhob/orm/tests.py        |  16 ++++
 bitbake/lib/webhob/orm/views.py        |   1 +
 bitbake/lib/webhob/whbmain/settings.py |   1 +
 5 files changed, 180 insertions(+)
 create mode 100644 bitbake/lib/webhob/orm/__init__.py
 create mode 100644 bitbake/lib/webhob/orm/models.py
 create mode 100644 bitbake/lib/webhob/orm/tests.py
 create mode 100644 bitbake/lib/webhob/orm/views.py
bitbake/lib/webhob/orm/__init__.py new file +0 -0
new file mode 100644
index 0000000..e69de29
bitbake/lib/webhob/orm/models.py new file +162 -0

162 lines not on this page · raw message

bitbake/lib/webhob/orm/tests.py new file +16 -0
new file mode 100644
index 0000000..501deb7
@@ -0,0 +1,16 @@
+"""
+This file demonstrates writing tests using the unittest module. These will pass
+when you run "manage.py test".
+
+Replace this with more appropriate tests for your application.
+"""
+
+from django.test import TestCase
+
+
+class SimpleTest(TestCase):
+    def test_basic_addition(self):
+        """
+        Tests that 1 + 1 always equals 2.
+        """
+        self.assertEqual(1 + 1, 2)
bitbake/lib/webhob/orm/views.py new file +1 -0
new file mode 100644
index 0000000..60f00ef
@@ -0,0 +1 @@
+# Create your views here.
bitbake/lib/webhob/whbmain/settings.py +1 -0
index 67aaf69..71bba4f 100644
@@ -123,6 +123,7 @@ INSTALLED_APPS = (
     # 'django.contrib.admin',
     # Uncomment the next line to enable admin documentation:
     # 'django.contrib.admindocs',
+    'orm',
 )
 
 # A sample logging configuration. The only tangible logging
Signature · 1 line
1.8.1.2
#4 lore raw

PATCH 4/94 bitbake: create Data Store Interface (DSI) file

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

The patch adds the first version of the DSI file. It uses the Knotty code for now and inserts task related information into the database using the Django ORM.
It makes use of some test objects needed to create a valid record into the database based on the definition of the Django models.

---
 bitbake/lib/bb/ui/dsi.py | 667 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 667 insertions(+)
 create mode 100644 bitbake/lib/bb/ui/dsi.py
bitbake/lib/bb/ui/dsi.py new file +667 -0
new file mode 100644
index 0000000..f832806
@@ -0,0 +1,667 @@
+#
+# BitBake DSI Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from __future__ import division
+
+import os
+import sys
+import xmlrpclib
+import logging
+import progressbar
+import signal
+import bb.msg
+import time
+import fcntl
+import struct
+import copy
+import datetime
+
+from bb.ui import uihelper
+
+logger = logging.getLogger("BitBake")
+interactive = sys.stdout.isatty()
+
+class WebHOBHelper(object):
+    
+    def __init__(self):
+        self.configure_django()
+        self.task_order = 0
+        self.tasks_information = {}
+
+    def configure_django(self):
+        import webhob.whbmain.settings as whb_django_settings
+        from django.core.management import setup_environ
+        setup_environ(whb_django_settings)
+        # Add webhob to sys path for importing modules
+        sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
+    
+    def store_build(self, uuid):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Builds, Machines, Layers
+        machine_obj = Machines.objects.create(
+                                name='Test Machine',
+                                description='This is our great test machine'
+                                )
+        machine_obj.save()
+
+        layer_obj = Layers.objects.create(
+                                name='Test Layer',
+                                branch='master_test',
+                                commit='2148u323nsau83smdmbnca3312f',
+                                priority=2,
+                                link_to_oe_core='http://test.oecore.com'
+                                )
+        layer_obj.save()
+
+        build_obj = Builds.objects.create(
+                                uuid=uuid,
+                                target='test target',
+                                host_machine='Host machine',
+                                distro='1.234',
+                                distro_version='v2',
+                                host_system='Test hosting system',
+                                host_distribution='Test host distrib',
+                                target_system=machine_obj,
+                                started_on=datetime.datetime.now(),
+                                completed_on=datetime.datetime.now(),
+                                outcome=3,
+                                errors_no=0,
+                                warnings_no=0,
+                                cpu_usage=54,
+                                disk_io=23.3232,
+                                cooker_log_path='/home/test/cooker_test',
+                                build_name='tester_build',
+                                bitbake_version='1.0.5')
+        build_obj.layers.add(layer_obj)
+        build_obj.save()
+
+        return build_obj, layer_obj
+
+    def store_started_task(self, event, build_object):
+        self.task_order += 1
+        identifier = event._package + event._task
+        self.tasks_information[identifier] = {
+                'uuid': build_object,
+                'task_executed': True,
+                'order': self.task_order,
+                'recipe': event._package,
+                'task_name': event._task,
+                'start_time': time.time(),
+                }
+
+    def write_stored_tasks(self, event, build_object, layer_object):
+        identifier = event._package + event._task
+
+        # fix this
+        if event.getDisplayName().lower() == 'succeded':
+            outcome = 0
+        else:
+            outcome = 1
+
+        self.tasks_information[identifier].update({
+                'outcome': outcome,
+                'end_time': time.time(),
+                })
+
+        task = self.tasks_information[identifier]
+
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Tasks, Recipes
+
+        recipe_obj = Recipes.objects.create(
+                                    name=task['recipe'],
+                                    version='1.0',
+                                    layer=layer_object,
+                                    summary='Short summary',
+                                    description='Longer description',
+                                    section='24',
+                                    license='license title',
+                                    licensing_info='longer license information',
+                                    homepage='http://homepage.com',
+                                    bugtracker='2456',
+                                    author='Mr.Test',
+                                    file_path='home/test/testproject/',
+                                    )
+
+        # Store finished task
+        task_obj = Tasks.objects.create(uuid=task['uuid'],
+                                        order=task['order'],
+                                        outcome=task['outcome'],
+                                        recipe=recipe_obj,
+                                        task_name=task['task_name'],
+                                        elapsed_time=task['end_time'] - task['start_time'],
+                                        # The next lines are just for testing reasons
+                                        sstate_checksum = '2134123sdscsdf232342tgfdv',
+                                        task_executed=True,
+                                        path_to_sstate_obj='/home/calin',
+                                        source_url='/',
+                                        log_file='/',
+                                        work_directory='/',
+                                        script_type=0,
+                                        file_path='/',
+                                        line_number=55,
+                                        py_stack_trace='Testing traceback',
+                                        disk_io = 23.55633,
+                                        cpu_usage=43,
+                                        errors_no=0,
+                                        warnings_no=0,
+                                        error='None',
+                                        warning='None',
+                                        sstate_result=0,
+                                        diffsigs="Testing diffsigs")
+        task_obj.save()
+        # Clean memory
+        del self.tasks_information[identifier]
+
+
+class BBProgress(progressbar.ProgressBar):
+    def __init__(self, msg, maxval):
+        self.msg = msg
+        widgets = [progressbar.Percentage(), ' ', progressbar.Bar(), ' ',
+           progressbar.ETA()]
+
+        try:
+            self._resize_default = signal.getsignal(signal.SIGWINCH)
+        except:
+            self._resize_default = None
+        progressbar.ProgressBar.__init__(self, maxval, [self.msg + ": "] + widgets)
+
+    def _handle_resize(self, signum, frame):
+        progressbar.ProgressBar._handle_resize(self, signum, frame)
+        if self._resize_default:
+            self._resize_default(signum, frame)
+    def finish(self):
+        progressbar.ProgressBar.finish(self)
+        if self._resize_default:
+            signal.signal(signal.SIGWINCH, self._resize_default)
+
+class NonInteractiveProgress(object):
+    fobj = sys.stdout
+
+    def __init__(self, msg, maxval):
+        self.msg = msg
+        self.maxval = maxval
+
+    def start(self):
+        self.fobj.write("%s..." % self.msg)
+        self.fobj.flush()
+        return self
+
+    def update(self, value):
+        pass
+
+    def finish(self):
+        self.fobj.write("done.\n")
+        self.fobj.flush()
+
+def new_progress(msg, maxval):
+    if interactive:
+        return BBProgress(msg, maxval)
+    else:
+        return NonInteractiveProgress(msg, maxval)
+
+def pluralise(singular, plural, qty):
+    if(qty == 1):
+        return singular % qty
+    else:
+        return plural % qty
+
+
+class InteractConsoleLogFilter(logging.Filter):
+    def __init__(self, tf, format):
+        self.tf = tf
+        self.format = format
+
+    def filter(self, record):
+        if record.levelno == self.format.NOTE and (record.msg.startswith("Running") or record.msg.startswith("recipe ")):
+            return False
+        self.tf.clearFooter()
+        return True
+
+class TerminalFilter(object):
+    columns = 80
+
+    def sigwinch_handle(self, signum, frame):
+        self.columns = self.getTerminalColumns()
+        if self._sigwinch_default:
+            self._sigwinch_default(signum, frame)
+
+    def getTerminalColumns(self):
+        def ioctl_GWINSZ(fd):
+            try:
+                cr = struct.unpack('hh', fcntl.ioctl(fd, self.termios.TIOCGWINSZ, '1234'))
+            except:
+                return None
+            return cr
+        cr = ioctl_GWINSZ(sys.stdout.fileno())
+        if not cr:
+            try:
+                fd = os.open(os.ctermid(), os.O_RDONLY)
+                cr = ioctl_GWINSZ(fd)
+                os.close(fd)
+            except:
+                pass
+        if not cr:
+            try:
+                cr = (env['LINES'], env['COLUMNS'])
+            except:
+                cr = (25, 80)
+        return cr[1]
+
+    def __init__(self, main, helper, console, format):
+        self.main = main
+        self.helper = helper
+        self.cuu = None
+        self.stdinbackup = None
+        self.interactive = sys.stdout.isatty()
+        self.footer_present = False
+        self.lastpids = []
+
+        if not self.interactive:
+            return
+
+        try:
+            import curses
+        except ImportError:
+            sys.exit("FATAL: The knotty ui could not load the required curses python module.")
+
+        import termios
+        self.curses = curses
+        self.termios = termios
+        try:
+            fd = sys.stdin.fileno()
+            self.stdinbackup = termios.tcgetattr(fd)
+            new = copy.deepcopy(self.stdinbackup)
+            new[3] = new[3] & ~termios.ECHO
+            termios.tcsetattr(fd, termios.TCSADRAIN, new)
+            curses.setupterm()
+            if curses.tigetnum("colors") > 2:
+                format.enable_color()
+            self.ed = curses.tigetstr("ed")
+            if self.ed:
+                self.cuu = curses.tigetstr("cuu")
+            try:
+                self._sigwinch_default = signal.getsignal(signal.SIGWINCH)
+                signal.signal(signal.SIGWINCH, self.sigwinch_handle)
+            except:
+                pass
+            self.columns = self.getTerminalColumns()
+        except:
+            self.cuu = None
+        console.addFilter(InteractConsoleLogFilter(self, format))
+
+    def clearFooter(self):
+        if self.footer_present:
+            lines = self.footer_present
+            sys.stdout.write(self.curses.tparm(self.cuu, lines))
+            sys.stdout.write(self.curses.tparm(self.ed))
+        self.footer_present = False
+
+    def updateFooter(self):
+        if not self.cuu:
+            return
+        activetasks = self.helper.running_tasks
+        failedtasks = self.helper.failed_tasks
+        runningpids = self.helper.running_pids
+        if self.footer_present and (self.lastcount == self.helper.tasknumber_current) and (self.lastpids == runningpids):
+            return
+        if self.footer_present:
+            self.clearFooter()
+        if (not self.helper.tasknumber_total or self.helper.tasknumber_current == self.helper.tasknumber_total) and not len(activetasks):
+            return
+        tasks = []
+        for t in runningpids:
+            tasks.append("%s (pid %s)" % (activetasks[t]["title"], t))
+
+        if self.main.shutdown:
+            content = "Waiting for %s running tasks to finish:" % len(activetasks)
+        elif not len(activetasks):
+            content = "No currently running tasks (%s of %s)" % (self.helper.tasknumber_current, self.helper.tasknumber_total)
+        else:
+            content = "Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total)
+        print(content)
+        lines = 1 + int(len(content) / (self.columns + 1))
+        for tasknum, task in enumerate(tasks):
+            content = "%s: %s" % (tasknum, task)
+            print(content)
+            lines = lines + 1 + int(len(content) / (self.columns + 1))
+        self.footer_present = lines
+        self.lastpids = runningpids[:]
+        self.lastcount = self.helper.tasknumber_current
+
+    def finish(self):
+        if self.stdinbackup:
+            fd = sys.stdin.fileno()
+            self.termios.tcsetattr(fd, self.termios.TCSADRAIN, self.stdinbackup)
+
+def main(server, eventHandler, params, tf = TerminalFilter):
+    print "This is running in DSI mode"
+
+    # Generate an unique ID for this build
+    # TODO: for multiple build commands this might not work as expected
+    import uuid
+    uuid = sessionid = str(uuid.uuid4())
+
+    # Get values of variables which control our output
+    includelogs, error = server.runCommand(["getVariable", "BBINCLUDELOGS"])
+    if error:
+        logger.error("Unable to get the value of BBINCLUDELOGS variable: %s" % error)
+        return 1
+    loglines, error = server.runCommand(["getVariable", "BBINCLUDELOGS_LINES"])
+    if error:
+        logger.error("Unable to get the value of BBINCLUDELOGS_LINES variable: %s" % error)
+        return 1
+    consolelogfile, error = server.runCommand(["getVariable", "BB_CONSOLELOG"])
+    if error:
+        logger.error("Unable to get the value of BB_CONSOLELOG variable: %s" % error)
+        return 1
+
+    if sys.stdin.isatty() and sys.stdout.isatty():
+        log_exec_tty = True
+    else:
+        log_exec_tty = False
+
+    helper = uihelper.BBUIHelper()
+
+    console = logging.StreamHandler(sys.stdout)
+    format_str = "%(levelname)s: %(message)s"
+    format = bb.msg.BBLogFormatter(format_str)
+    bb.msg.addDefaultlogFilter(console)
+    console.setFormatter(format)
+    logger.addHandler(console)
+
+    if consolelogfile and not params.options.show_environment:
+        bb.utils.mkdirhier(os.path.dirname(consolelogfile))
+        conlogformat = bb.msg.BBLogFormatter(format_str)
+        consolelog = logging.FileHandler(consolelogfile)
+        bb.msg.addDefaultlogFilter(consolelog)
+        consolelog.setFormatter(conlogformat)
+        logger.addHandler(consolelog)
+
+    try:
+        params.updateFromServer(server)
+        cmdline = params.parseActions()
+        if not cmdline:
+            print("Nothing to do.  Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information.")
+            return 1
+        if 'msg' in cmdline and cmdline['msg']:
+            logger.error(cmdline['msg'])
+            return 1
+
+        ret, error = server.runCommand(cmdline['action'])
+        if error:
+            logger.error("Command '%s' failed: %s" % (cmdline, error))
+            return 1
+        elif ret != True:
+            logger.error("Command '%s' failed: returned %s" % (cmdline, ret))
+            return 1
+    except xmlrpclib.Fault as x:
+        logger.error("XMLRPC Fault getting commandline:\n %s" % x)
+        return 1
+
+    parseprogress = None
+    cacheprogress = None
+    main.shutdown = 0
+    interrupted = False
+    return_value = 0
+    errors = 0
+    warnings = 0
+    taskfailures = []
+    wbhbhelper = WebHOBHelper()
+    build_object, layer_object = wbhbhelper.store_build(uuid)
+
+    termfilter = tf(main, helper, console, format)
+
+    while True:
+        try:
+            termfilter.updateFooter()
+            event = eventHandler.waitEvent(0.25)
+
+            if event is None:
+                if main.shutdown > 1:
+                    break
+                continue
+
+            helper.eventHandler(event)
+
+            if isinstance(event, bb.runqueue.runQueueExitWait):
+                if not main.shutdown:
+                    main.shutdown = 1
+
+            if isinstance(event, bb.build.TaskStarted):
+                wbhbhelper.store_started_task(event, build_object)
+
+            if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)):
+                wbhbhelper.write_stored_tasks(event, build_object, layer_object)
+
+            if isinstance(event, bb.event.LogExecTTY):
+                if log_exec_tty:
+                    tries = event.retries
+                    while tries:
+                        print("Trying to run: %s" % event.prog)
+                        if os.system(event.prog) == 0:
+                            break
+                        time.sleep(event.sleep_delay)
+                        tries -= 1
+                    if tries:
+                        continue
+                logger.warn(event.msg)
+                continue
+
+            if isinstance(event, logging.LogRecord):
+                if event.levelno >= format.ERROR:
+                    errors = errors + 1
+                    return_value = 1
+                elif event.levelno == format.WARNING:
+                    warnings = warnings + 1
+                # For "normal" logging conditions, don't show note logs from tasks
+                # but do show them if the user has changed the default log level to 
+                # include verbose/debug messages
+                if event.taskpid != 0 and event.levelno <= format.NOTE:
+                    continue
+                logger.handle(event)
+                continue
+
+            if isinstance(event, bb.build.TaskFailed):
+                return_value = 1
+                logfile = event.logfile
+                if logfile and os.path.exists(logfile):
+                    termfilter.clearFooter()
+                    bb.error("Logfile of failure stored in: %s" % logfile)
+                    if includelogs and not event.errprinted:
+                        print("Log data follows:")
+                        f = open(logfile, "r")
+                        lines = []
+                        while True:
+                            l = f.readline()
+                            if l == '':
+                                break
+                            l = l.rstrip()
+                            if loglines:
+                                lines.append(' | %s' % l)
+                                if len(lines) > int(loglines):
+                                    lines.pop(0)
+                            else:
+                                print('| %s' % l)
+                        f.close()
+                        if lines:
+                            for line in lines:
+                                print(line)
+            if isinstance(event, bb.build.TaskBase):
+                logger.info(event._message)
+                continue
+            if isinstance(event, bb.event.ParseStarted):
+                if event.total == 0:
+                    continue
+                parseprogress = new_progress("Parsing recipes", event.total).start()
+                continue
+            if isinstance(event, bb.event.ParseProgress):
+                parseprogress.update(event.current)
+                continue
+            if isinstance(event, bb.event.ParseCompleted):
+                if not parseprogress:
+                    continue
+
+                parseprogress.finish()
+                print(("Parsing of %d .bb files complete (%d cached, %d parsed). %d targets, %d skipped, %d masked, %d errors."
+                    % ( event.total, event.cached, event.parsed, event.virtuals, event.skipped, event.masked, event.errors)))
+                continue
+
+            if isinstance(event, bb.event.CacheLoadStarted):
+                cacheprogress = new_progress("Loading cache", event.total).start()
+                continue
+            if isinstance(event, bb.event.CacheLoadProgress):
+                cacheprogress.update(event.current)
+                continue
+            if isinstance(event, bb.event.CacheLoadCompleted):
+                cacheprogress.finish()
+                print("Loaded %d entries from dependency cache." % event.num_entries)
+                continue
+
+            if isinstance(event, bb.event.MultipleProviders):
+                logger.info("multiple providers are available for %s%s (%s)", event._is_runtime and "runtime " or "",
+                            event._item,
+                            ", ".join(event._candidates))
+                logger.info("consider defining a PREFERRED_PROVIDER entry to match %s", event._item)
+                continue
+            if isinstance(event, bb.event.NoProvider):
+                return_value = 1
+                errors = errors + 1
+                if event._runtime:
+                    r = "R"
+                else:
+                    r = ""
+
+                if event._dependees:
+                    logger.error("Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)", r, event._item, ", ".join(event._dependees), r)
+                else:
+                    logger.error("Nothing %sPROVIDES '%s'", r, event._item)
+                if event._reasons:
+                    for reason in event._reasons:
+                        logger.error("%s", reason)
+                continue
+
+            if isinstance(event, bb.runqueue.sceneQueueTaskStarted):
+                logger.info("Running setscene task %d of %d (%s)" % (event.stats.completed + event.stats.active + event.stats.failed + 1, event.stats.total, event.taskstring))
+                continue
+
+            if isinstance(event, bb.runqueue.runQueueTaskStarted):
+                if event.noexec:
+                    tasktype = 'noexec task'
+                else:
+                    tasktype = 'task'
+                logger.info("Running %s %s of %s (ID: %s, %s)",
+                            tasktype,
+                            event.stats.completed + event.stats.active +
+                                event.stats.failed + 1,
+                            event.stats.total, event.taskid, event.taskstring)
+                continue
+
+            if isinstance(event, bb.runqueue.runQueueTaskFailed):
+                taskfailures.append(event.taskstring)
+                logger.error("Task %s (%s) failed with exit code '%s'",
+                             event.taskid, event.taskstring, event.exitcode)
+                continue
+ 
+            if isinstance(event, bb.runqueue.sceneQueueTaskFailed):
+                logger.warn("Setscene task %s (%s) failed with exit code '%s' - real task will be run instead",
+                             event.taskid, event.taskstring, event.exitcode)
+                continue
+
+            if isinstance(event, bb.event.ConfigParsed):
+                # timestamp should be added for this
+                continue
+            
+            if isinstance(event, bb.event.RecipeParsed):
+                # timestamp should be added for this
+                continue
+                
+            if isinstance(event, bb.event.OperationStarted):
+                # timestamp should be added for this
+                continue
+
+            if isinstance(event, bb.event.OperationCompleted):
+                # timestamp should be added for this
+                # signal a complete operation
+                # calculate timing
+                continue
+
+            if isinstance(event, bb.event.DiskFull):
+                # trigger an error
+                continue
+
+            # ignore
+            if isinstance(event, (bb.event.BuildBase,
+                                  bb.event.StampUpdate,
+                                  bb.event.RecipePreFinalise,
+                                  bb.runqueue.runQueueEvent,
+                                  bb.runqueue.runQueueExitWait,
+                                  bb.event.OperationProgress,
+                                  bb.command.CommandFailed,
+                                  bb.command.CommandExit,
+                                  bb.command.CommandCompleted,
+                                  bb.cooker.CookerExit)):
+                continue
+
+            logger.error("Unknown event: %s", event)
+
+        except EnvironmentError as ioerror:
+            termfilter.clearFooter()
+            # ignore interrupted io
+            if ioerror.args[0] == 4:
+                pass
+        except KeyboardInterrupt:
+            termfilter.clearFooter()
+            if main.shutdown == 1:
+                print("\nSecond Keyboard Interrupt, stopping...\n")
+                _, error = server.runCommand(["stateStop"])
+                if error:
+                    logger.error("Unable to cleanly stop: %s" % error)
+            if main.shutdown == 0:
+                print("\nKeyboard Interrupt, closing down...\n")
+                interrupted = True
+                _, error = server.runCommand(["stateShutdown"])
+                if error:
+                    logger.error("Unable to cleanly shutdown: %s" % error)
+            main.shutdown = main.shutdown + 1
+            pass
+
+    summary = ""
+    if taskfailures:
+        summary += pluralise("\nSummary: %s task failed:",
+                             "\nSummary: %s tasks failed:", len(taskfailures))
+        for failure in taskfailures:
+            summary += "\n  %s" % failure
+    if warnings:
+        summary += pluralise("\nSummary: There was %s WARNING message shown.",
+                             "\nSummary: There were %s WARNING messages shown.", warnings)
+    if return_value:
+        summary += pluralise("\nSummary: There was %s ERROR message shown, returning a non-zero exit code.",
+                             "\nSummary: There were %s ERROR messages shown, returning a non-zero exit code.", errors)
+    if summary:
+        print(summary)
+
+    if interrupted:
+        print("Execution was interrupted, returning a non-zero exit code.")
+        if return_value == 0:
+            return_value = 1
+
+    termfilter.finish()
+
+    return return_value
+
Signature · 1 line
1.8.1.2
#5 lore raw

PATCH 5/94 bitbake: webhob: updates to Django models

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

Based on recent discussions with the team I have updated the Django models we had in order to store information more clearly. The updates include: renaming of some tables, removing of unnecessary fields, adding supplementary tables to reflect relations between objects.

---
 bitbake/lib/webhob/orm/models.py | 94 ++++++++++++++++++++--------------------
 1 file changed, 47 insertions(+), 47 deletions(-)
bitbake/lib/webhob/orm/models.py +47 -47
index 43b4319..6b8e2fa 100644
@@ -1,22 +1,18 @@
 from django.db import models
 
 
-class Builds(models.Model):
+class Build(models.Model):
 
     BUILD_OUTCOME = (
-        (0, 'Succeded'),
+        (0, 'Succeeded'),
         (1, 'Failed'),
     )
 
     uuid = models.CharField(max_length=100, unique=True)
     target = models.CharField(max_length=100)
-    host_machine = models.CharField(max_length=100)
+    machine = models.ForeignKey('Machine', related_name='build_machine')
     distro = models.CharField(max_length=100)
     distro_version = models.CharField(max_length=100)
-    host_system = models.CharField(max_length=100)
-    host_distribution = models.CharField(max_length=100)
-    target_system = models.ForeignKey('Machines', related_name='builds_machines')
-    layers = models.ManyToManyField('Layers')
     started_on = models.DateTimeField()
     completed_on = models.DateTimeField()
     outcome = models.IntegerField(choices=BUILD_OUTCOME)
@@ -29,7 +25,7 @@ class Builds(models.Model):
     bitbake_version = models.CharField(max_length=50)
 
 
-class Tasks(models.Model):
+class Task(models.Model):
 
     SSTATE_RESULT = (
         (0, 'Not Applicable'), # For rest of tasks, but they still need checking.
@@ -47,23 +43,23 @@ class Tasks(models.Model):
         (0, 'Covered'),
         (1, 'Sstate'),
         (2, 'Existing'),
-        (3, 'Succeded'),
+        (3, 'Succeeded'),
         (4, 'Failed'),
     )
 
-    uuid = models.ForeignKey(Builds, to_field='uuid', related_name='tasks_builds')
+    build = models.ForeignKey(Build, related_name='task_build')
     order = models.IntegerField()
     task_executed = models.BooleanField() # True means Executed, False means Prebuilt
     outcome = models.IntegerField(choices=TASK_OUTCOME)
     sstate_checksum = models.CharField(max_length=100)
     path_to_sstate_obj = models.FilePathField(max_length=500, blank=True)
-    recipe = models.ForeignKey('Recipes', related_name='builds_recipes')
+    recipe = models.ForeignKey('Recipe', related_name='build_recipe')
     task_name = models.CharField(max_length=100)
-    source_url = models.FilePathField(max_length=200)
-    log_file = models.FilePathField(max_length=200, blank=True)
-    work_directory = models.FilePathField(max_length=200)
+    source_url = models.FilePathField(max_length=255)
+    log_file = models.FilePathField(max_length=255, blank=True)
+    work_directory = models.FilePathField(max_length=255)
     script_type = models.IntegerField(choices=TASK_CODING)
-    file_path = models.FilePathField(max_length=200)
+    file_path = models.FilePathField(max_length=255)
     line_number = models.IntegerField()
     py_stack_trace = models.TextField()
     disk_io = models.DecimalField(max_digits=20, decimal_places=10)
@@ -74,54 +70,52 @@ class Tasks(models.Model):
     error = models.TextField()
     warning = models.TextField()
     sstate_result = models.IntegerField(choices=SSTATE_RESULT)
-    diffsigs = models.CharField(max_length=500)
 
 
-class Task_Dependencies(models.Model):
-    task = models.ForeignKey(Tasks, related_name='task_dependencies_task')
-    depends_on = models.ForeignKey(Tasks, related_name='task_dependencies_depends')
+class Task_Dependency(models.Model):
+    task = models.ForeignKey(Task, related_name='task_dependencies_task')
+    depends_on = models.ForeignKey(Task, related_name='task_dependencies_depends')
 
 
-class Targets(models.Model):
-    uuid = models.ForeignKey(Builds, to_field='uuid',  related_name='targets_builds')
+class Target(models.Model):
+    build = models.ForeignKey(Build, related_name='target_build')
     is_image = models.BooleanField()
 
 
-class Artifacts(models.Model):
-    uuid = models.ForeignKey(Builds, to_field='uuid', related_name='artifacts_builds')
-    target = models.ForeignKey(Targets, related_name='artifacts_targets')
+class Artifact(models.Model):
+    build = models.ForeignKey(Build, related_name='artifact_build')
+    target = models.ForeignKey(Target, related_name='artifact_target')
     file_name = models.CharField(max_length=100)
     file_size = models.IntegerField()
 
 
-class Packages_In_Images(models.Model):
-    package = models.ForeignKey('Packages', related_name='packages_in_images_package')
-    target = models.ForeignKey(Targets, related_name='packages_in_images_target')
+class Package_In_Image(models.Model):
+    package = models.ForeignKey('Package', related_name='package_in_image_package')
+    target = models.ForeignKey(Target, related_name='package_in_image_target')
 
 
-class Packages(models.Model):
-    package_id = models.IntegerField()
-    recipe = models.ForeignKey('Recipes', related_name='packages_recipes')
+class Package(models.Model):
+    recipe = models.ForeignKey('Recipe', related_name='package_recipe')
     name = models.CharField(max_length=100)
     version = models.CharField(max_length=100)
     size = models.IntegerField()
 
 
-class Package_Dependencies(models.Model):
-    package = models.ForeignKey(Packages, related_name='package_dependencies_package')
-    depends_on = models.ForeignKey(Packages, related_name='package_dependencies_depends')
+class Package_Dependency(models.Model):
+    package = models.ForeignKey(Package, related_name='package_dependencies_package')
+    depends_on = models.ForeignKey(Package, related_name='package_dependencies_depends')
 
 
 class Filelist(models.Model):
-    package = models.ForeignKey(Packages, related_name='filelist_packages')
-    complete_file_path = models.FilePathField(max_length=200, blank=True)
+    package = models.ForeignKey(Package, related_name='filelist_package')
+    complete_file_path = models.FilePathField(max_length=255, blank=True)
     file_size = models.IntegerField()
 
 
-class Recipes(models.Model):
+class Recipe(models.Model):
     name = models.CharField(max_length=100)
     version = models.CharField(max_length=100)
-    layer = models.ForeignKey('Layers', related_name='recipes_layers')
+    layer = models.ForeignKey('Build_Layer', related_name='recipe_build_layer')
     summary = models.CharField(max_length=100)
     description = models.CharField(max_length=100)
     section = models.CharField(max_length=100)
@@ -130,33 +124,39 @@ class Recipes(models.Model):
     homepage = models.URLField()
     bugtracker = models.URLField()
     author = models.CharField(max_length=100)
-    file_path = models.FilePathField(max_length=200)
+    file_path = models.FilePathField(max_length=255)
 
 
-class Recipe_Dependencies(models.Model):
-    recipe = models.ForeignKey(Recipes, related_name='recipe_dependencies_recipe')
-    depends_on = models.ForeignKey(Recipes, related_name='recipe_dependencies_depends')
+class Recipe_Dependency(models.Model):
+    recipe = models.ForeignKey(Recipe, related_name='recipe_dependencies_recipe')
+    depends_on = models.ForeignKey(Recipe, related_name='recipe_dependencies_depends')
 
 
-class Layers(models.Model):
+class Layer(models.Model):
     name = models.CharField(max_length=100)
+    local_path = models.FilePathField(max_length=255)
+    layer_index_url = models.URLField()
+
+
+class Build_Layer(models.Model):
+    build = models.ForeignKey(Build, related_name='build_layer_build')
+    layer = models.ForeignKey(Layer, related_name='build_layer_layer')
     branch = models.CharField(max_length=50)
     commit = models.CharField(max_length=100)
     priority = models.IntegerField()
-    link_to_oe_core = models.URLField()
 
 
-class Variables(models.Model):
-    uuid = models.ForeignKey(Builds, to_field='uuid', related_name='variables_builds')
+class Variable(models.Model):
+    build = models.ForeignKey(Build, related_name='variable_build')
     variable_name = models.CharField(max_length=100)
     variable_value = models.TextField()
-    file = models.FilePathField(max_length=200)
+    file = models.FilePathField(max_length=255)
     changed = models.BooleanField()
     human_readable_name = models.CharField(max_length=200)
     description = models.TextField()
 
 
-class Machines(models.Model):
+class Machine(models.Model):
     name = models.CharField(max_length=100)
     description = models.TextField()
 
Signature · 1 line
1.8.1.2
#6 lore raw

PATCH 6/94 bitbake: webhob: make DSI store build information

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch will enable DSI to store build information defined in the Django models. This will be done for each executed build and it will use a helper file to save this information.

Moved code from DSI interface to the helper file in to classes, one which wraps around Django model interface operations, and one that fills in information from the event system into the model data.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 127 ++++++++++++++++++++++++
 bitbake/lib/bb/ui/dsi.py             | 180 ++++-------------------------------
 2 files changed, 148 insertions(+), 159 deletions(-)
 create mode 100644 bitbake/lib/bb/ui/buildinfohelper.py
bitbake/lib/bb/ui/buildinfohelper.py new file +127 -0
new file mode 100644
index 0000000..c77248c
@@ -0,0 +1,127 @@
+import datetime
+import sys
+import uuid
+
+
+class ORMWrapper(object):
+    """ This class creates the dictionaries needed to store information in the database
+        following the format defined by the Django models. It is also used to save this
+        information in the database.
+    """
+
+    def __init__(self):
+        self.uuid = None
+
+    def get_machine_information(self, server):
+        machine_info = {}
+
+        machine_info['name'] = server.runCommand(["getVariable", "MACHINE"])[0]
+        machine_info['description'] = 'Not Available'
+
+        return machine_info
+
+    def create_machine_object(self, machine_information):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Machine
+
+        machine = Machine.objects.get_or_create(name=machine_information['name'],
+                                                description=machine_information['description'])
+
+        return machine[0]
+
+    def get_build_information(self, server, machine_obj):
+        build_info = {}
+
+        # Generate an identifier for each new build
+        self.uuid = str(uuid.uuid4())
+
+        build_info['uuid'] = self.uuid
+        build_info['target'] = 'Not Available'
+        build_info['machine'] = machine_obj
+        build_info['distro'] = server.runCommand(["getVariable", "DISTRO"])[0]
+        build_info['distro_version'] = server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
+        build_info['started_on'] = datetime.datetime.now()
+        build_info['completed_on'] = datetime.datetime.now()
+        build_info['outcome'] = 0
+        build_info['number_of_errors'] = 0
+        build_info['number_of_warnings'] = 0
+        build_info['cpu_usage'] = 0
+        build_info['disk_io'] = 0
+        build_info['cooker_log_path'] = server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
+        build_info['build_name'] = server.runCommand(["getVariable", "BUILDNAME"])[0]
+        build_info['bitbake_version'] = server.runCommand(["getVariable", "BB_VERSION"])[0]
+
+        return build_info
+
+    def create_build_object(self, build_info):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Build
+
+        build = Build.objects.create(uuid=build_info['uuid'],
+                                    target=build_info['target'],
+                                    machine=build_info['machine'],
+                                    distro=build_info['distro'],
+                                    distro_version=build_info['distro_version'],
+                                    started_on=build_info['started_on'],
+                                    completed_on=build_info['completed_on'],
+                                    outcome=build_info['outcome'],
+                                    errors_no=build_info['number_of_errors'],
+                                    warnings_no=build_info['number_of_warnings'],
+                                    cpu_usage=build_info['cpu_usage'],
+                                    disk_io=build_info['disk_io'],
+                                    cooker_log_path=build_info['cooker_log_path'],
+                                    build_name=build_info['build_name'],
+                                    bitbake_version=build_info['bitbake_version'])
+
+        return build
+
+    def update_build_object(self, build_obj, errors, warnings):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Build
+
+        outcome = 0
+        if errors or warnings:
+            outcome = 1
+
+        build = Build.objects.get(uuid=build_obj.uuid)
+        build.completed_on = datetime.datetime.now()
+        build.errors_no = errors
+        build.warnings_no = warnings
+        build.outcome = outcome
+        build.save()
+
+        return build_obj
+
+
+class BuildInfoHelper(object):
+    """ This class gathers the build information from the server and sends it
+        towards the ORM wrapper for storing in the database
+    """
+
+    def __init__(self):
+        self.configure_django()
+        self.task_order = 0
+        self.tasks_information = {}
+        self.uuid = None
+        self.transport_utils = {}
+        self.orm_wrapper = ORMWrapper()
+
+    def configure_django(self):
+        import webhob.whbmain.settings as whb_django_settings
+        from django.core.management import setup_environ
+        setup_environ(whb_django_settings)
+        # Add webhob to sys path for importing modules
+        sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
+
+    def store_started_build(self, event, server):
+
+        machine_information = self.orm_wrapper.get_machine_information(server)
+        machine_obj = self.orm_wrapper.create_machine_object(machine_information)
+
+        build_information = self.orm_wrapper.get_build_information(server, machine_obj)
+        build_obj = self.orm_wrapper.create_build_object(build_information)
+        self.transport_utils['build'] = build_obj
+
+    def update_build_information(self, event, errors, warnings):
+        build_obj = self.orm_wrapper.update_build_object(self.transport_utils['build'], errors, warnings)
+        del self.transport_utils['build']
bitbake/lib/bb/ui/dsi.py +21 -159

240 lines not on this page · raw message

Signature · 1 line
1.8.1.2
#7 lore raw

PATCH 7/94 bitbake: webhob: make DSI store task information

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch will enable DSI to store task related information. Together with the task object, other objects need to be obtained or created in order to respect the Django models defined.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 235 +++++++++++++++++++++++++++++++++--
 bitbake/lib/bb/ui/dsi.py             |   8 +-
 2 files changed, 231 insertions(+), 12 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +224 -11
index c77248c..57e404b 100644
@@ -11,6 +11,8 @@ class ORMWrapper(object):
 
     def __init__(self):
         self.uuid = None
+        self.task_order = 0
+        self.transport_utils = {}
 
     def get_machine_information(self, server):
         machine_info = {}
@@ -75,12 +77,12 @@ class ORMWrapper(object):
 
         return build
 
-    def update_build_object(self, build_obj, errors, warnings):
+    def update_build_object(self, build_obj, errors, warnings, taskfailures):
         # This needs to be imported after we have configured the Django settings file
         from webhob.orm.models import Build
 
         outcome = 0
-        if errors or warnings:
+        if errors or taskfailures:
             outcome = 1
 
         build = Build.objects.get(uuid=build_obj.uuid)
@@ -90,7 +92,212 @@ class ORMWrapper(object):
         build.outcome = outcome
         build.save()
 
-        return build_obj
+    def get_task_information(self, event):
+
+        self.task_order += 1
+
+        recipe = self.get_recipe_object(event)
+
+        task_information = {}
+        task_information['build'] = self.transport_utils['build']
+        task_information['order'] = self.task_order
+        task_information['task_executed'] = True
+        task_information['outcome'] = 0
+        task_information['sstate_checksum'] = 'Not Available'
+        task_information['path_to_sstate_obj'] = 'Not Available'
+        task_information['recipe'] = recipe
+        task_information['task_name'] = event._task
+        task_information['source_url'] = 'Not Available'
+        task_information['log_file'] = 'Not Available'
+        task_information['work_directory'] = 'Not Available'
+        task_information['script_type'] = 0
+        task_information['file_path'] = 'Not Available'
+        task_information['line_number'] = 0
+        task_information['py_stack_trace'] = 'Not Available'
+        task_information['disk_io'] = 0
+        task_information['cpu_usage'] = 0
+        task_information['elapsed_time'] = 'Not Available'
+        task_information['errors_no'] = 0
+        task_information['warnings_no'] = 0
+        task_information['error'] = 'Not Available'
+        task_information['warning'] = 'Not Available'
+        task_information['sstate_result'] = 0
+
+        return task_information
+
+    def create_task_object(self, task_information):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Task
+
+        identifier = task_information['recipe'].name + task_information['task_name']
+
+        task_object = Task.objects.get_or_create(
+                                build=task_information['build'],
+                                order=task_information['order'],
+                                task_executed=task_information['task_executed'],
+                                outcome=task_information['outcome'],
+                                sstate_checksum=task_information['sstate_checksum'],
+                                path_to_sstate_obj=task_information['path_to_sstate_obj'],
+                                recipe=task_information['recipe'],
+                                task_name=task_information['task_name'],
+                                source_url=task_information['source_url'],
+                                log_file=task_information['log_file'],
+                                work_directory=task_information['work_directory'],
+                                script_type=task_information['script_type'],
+                                file_path=task_information['file_path'],
+                                line_number=task_information['line_number'],
+                                py_stack_trace=task_information['py_stack_trace'],
+                                disk_io=task_information['disk_io'],
+                                cpu_usage=task_information['cpu_usage'],
+                                elapsed_time=task_information['elapsed_time'],
+                                errors_no=task_information['errors_no'],
+                                warnings_no=task_information['warnings_no'],
+                                error=task_information['error'],
+                                warning=task_information['warning'],
+                                sstate_result=task_information['sstate_result'])
+        task_object[0].save()
+
+        self.transport_utils[identifier] = {'object': task_object[0], 'start_time': datetime.datetime.now()}
+
+        return task_object[0]
+
+    def update_task_object(self, task_dictionary, event):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Task
+
+        task_object = task_dictionary['object']
+
+        task = Task.objects.get(build=task_object.build,
+                                task_name=task_object.task_name,
+                                recipe=task_object.recipe)
+
+        duration = datetime.datetime.now() - task_dictionary['start_time']
+        task.elapsed_time = duration.total_seconds()
+
+        outcome_info = self.get_outcome_of_task(event)
+        task.outcome = outcome_info['task_outcome']
+
+        if outcome_info.get('error', ''):
+            task.error = outcome_info['error']
+
+        #TODO: get error number
+        #TODO: get warnings number
+        #TODO: get warning information
+
+        task.save()
+
+    def get_outcome_of_task(self, event):
+        from webhob.orm.models import Task
+
+        outcome_info = {}
+
+        task_result = event.getDisplayName().lower()
+        for outcome in Task.TASK_OUTCOME:
+            if task_result == outcome[1].lower():
+                task_outcome = outcome[0]
+
+        outcome_info['task_outcome'] = task_outcome
+
+        if isinstance(event, bb.build.TaskFailed):
+            if event.logfile and os.path.exists(event.logfile):
+                outcome_info['error'] = open(event.logfile, "r").read()
+
+        return outcome_info
+
+    def get_recipe_object(self, event):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Recipe
+
+        recipe_information = self.get_recipe_information(event)
+
+        recipe_object = Recipe.objects.get_or_create(
+                                name=recipe_information['name'],
+                                version=recipe_information['version'],
+                                layer=recipe_information['layer'],
+                                summary=recipe_information['summary'],
+                                description=recipe_information['description'],
+                                section=recipe_information['section'],
+                                license=recipe_information['license'],
+                                licensing_info=recipe_information['licensing_info'],
+                                homepage=recipe_information['homepage'],
+                                bugtracker=recipe_information['bugtracker'],
+                                author=recipe_information['author'],
+                                file_path=recipe_information['file_path'])
+        recipe_object[0].save()
+
+        return recipe_object[0]
+
+    def get_recipe_information(self, event):
+
+        build_layer_obj = self.get_build_layer_object(event)
+
+        recipe_info = {}
+        recipe_info['name'] = event._package
+        recipe_info['version'] = 'Not Available'
+        recipe_info['layer'] = build_layer_obj
+        recipe_info['summary'] = 'Not Available'
+        recipe_info['description'] = 'Not Available'
+        recipe_info['section'] = 'Not Available'
+        recipe_info['license'] = 'Not Available'
+        recipe_info['licensing_info'] = 'Not Available'
+        recipe_info['homepage'] = 'Not Available'
+        recipe_info['bugtracker'] = 'Not Available'
+        recipe_info['author'] = 'Not Available'
+        recipe_info['file_path'] = 'Not Available'
+
+        return recipe_info
+
+    def get_build_layer_object(self, event):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Build_Layer
+
+        build_layer_information = self.get_build_layer_information(event)
+
+        build_layer_object = Build_Layer.objects.get_or_create(
+                                    build = build_layer_information['build'],
+                                    layer = build_layer_information['layer'],
+                                    branch = build_layer_information['branch'],
+                                    commit = build_layer_information['commit'],
+                                    priority = build_layer_information['priority'])
+        build_layer_object[0].save()
+
+        return build_layer_object[0]
+
+    def get_build_layer_information(self, event):
+
+        layer_object = self.get_layer_object(event)
+
+        build_layer_info = {}
+        build_layer_info['build'] = self.transport_utils['build']
+        build_layer_info['layer'] = layer_object
+        build_layer_info['branch'] = 'Not Available'
+        build_layer_info['commit'] = 'Not Available'
+        build_layer_info['priority'] = 0
+
+        return build_layer_info
+
+    def get_layer_object(self, event):
+        # This needs to be imported after we have configured the Django settings file
+        from webhob.orm.models import Layer
+
+        layer_information = self.get_layer_information(event)
+
+        layer_object = Layer.objects.get_or_create(
+                                name=layer_information['name'],
+                                local_path=layer_information['local_path'],
+                                layer_index_url=layer_information['layer_index_url'])
+        layer_object[0].save()
+
+        return layer_object[0]
+
+    def get_layer_information(self, event):
+
+        layer_info = {}
+        layer_info['name'] = 'Layer Test Name'
+        layer_info['local_path'] = 'Not Available'
+        layer_info['layer_index_url'] = 'Not Available'
+
+        return layer_info
 
 
 class BuildInfoHelper(object):
@@ -100,10 +307,6 @@ class BuildInfoHelper(object):
 
     def __init__(self):
         self.configure_django()
-        self.task_order = 0
-        self.tasks_information = {}
-        self.uuid = None
-        self.transport_utils = {}
         self.orm_wrapper = ORMWrapper()
 
     def configure_django(self):
@@ -120,8 +323,18 @@ class BuildInfoHelper(object):
 
         build_information = self.orm_wrapper.get_build_information(server, machine_obj)
         build_obj = self.orm_wrapper.create_build_object(build_information)
-        self.transport_utils['build'] = build_obj
+        self.orm_wrapper.transport_utils['build'] = build_obj
+
+    def update_build_information(self, event, errors, warnings, taskfailures):
+        self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures)
+        del self.orm_wrapper.transport_utils['build']
+
+    def store_started_task(self, event):
+        task_information = self.orm_wrapper.get_task_information(event)
+        task_obj = self.orm_wrapper.create_task_object(task_information)
 
-    def update_build_information(self, event, errors, warnings):
-        build_obj = self.orm_wrapper.update_build_object(self.transport_utils['build'], errors, warnings)
-        del self.transport_utils['build']
+    def update_and_store_task(self, event):
+        identifier = event._package + event._task
+        task_dictionary = self.orm_wrapper.transport_utils[identifier]
+        self.orm_wrapper.update_task_object(task_dictionary, event)
+        del self.orm_wrapper.transport_utils[identifier]
bitbake/lib/bb/ui/dsi.py +7 -1
index d7b038c..54364ce 100644
@@ -308,6 +308,12 @@ def main(server, eventHandler, params, tf = TerminalFilter):
             if isinstance(event, bb.event.BuildStarted):
                 buildinfohelper.store_started_build(event, server)
 
+            if isinstance(event, bb.build.TaskStarted):
+                buildinfohelper.store_started_task(event)
+
+            if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)):
+                buildinfohelper.update_and_store_task(event)
+
             if isinstance(event, bb.event.LogExecTTY):
                 if log_exec_tty:
                     tries = event.retries
@@ -443,7 +449,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 continue
 
             if isinstance(event, (bb.command.CommandFailed, bb.command.CommandExit, bb.command.CommandCompleted, bb.cooker.CookerExit)):
-                buildinfohelper.update_build_information(event, errors, warnings)
+                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
                 main.shutdown = 2
 
             if isinstance(event, bb.event.ConfigParsed):
Signature · 1 line
1.8.1.2
#8 lore raw

PATCH 8/94 bitbake: webhob: view a table with all builds

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch creates a page in Webhob in which you can view detailed information about each executed build. There is data about the outcome of each build, execution times, hardware usage, logs, errors, warnings and others.

---
 bitbake/lib/webhob/bldviewer/__init__.py           |  0
 bitbake/lib/webhob/bldviewer/templates/build.html  | 54 ++++++++++++++++++++++
 .../lib/webhob/bldviewer/templatetags/__init__.py  |  0
 .../webhob/bldviewer/templatetags/projecttags.py   |  8 ++++
 bitbake/lib/webhob/bldviewer/urls.py               |  6 +++
 bitbake/lib/webhob/bldviewer/views.py              | 11 +++++
 bitbake/lib/webhob/whbmain/urls.py                 |  2 +-
 7 files changed, 80 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/__init__.py
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/build.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/__init__.py
 create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/projecttags.py
 create mode 100644 bitbake/lib/webhob/bldviewer/urls.py
 create mode 100644 bitbake/lib/webhob/bldviewer/views.py
bitbake/lib/webhob/bldviewer/__init__.py new file +0 -0
new file mode 100644
index 0000000..e69de29
bitbake/lib/webhob/bldviewer/templates/build.html new file +54 -0

54 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/templatetags/__init__.py new file +0 -0
new file mode 100644
index 0000000..e69de29
bitbake/lib/webhob/bldviewer/templatetags/projecttags.py new file +8 -0
new file mode 100644
index 0000000..9ae6214
@@ -0,0 +1,8 @@
+from datetime import datetime
+from django import template
+
+register = template.Library()
+
[email protected]_tag
+def time_difference(start_time, end_time):
+    return end_time - start_time
bitbake/lib/webhob/bldviewer/urls.py new file +6 -0
new file mode 100644
index 0000000..3ec2a30
@@ -0,0 +1,6 @@
+from django.conf.urls import patterns, include, url
+
+
+urlpatterns = patterns('bldviewer.views',
+        url(r'^$', 'build', name='build'),
+)
bitbake/lib/webhob/bldviewer/views.py new file +11 -0
new file mode 100644
index 0000000..15a2d1d
@@ -0,0 +1,11 @@
+from django.shortcuts import render
+from orm.models import Build
+
+
+def build(request):
+    template = 'build.html'
+    build_info = Build.objects.all()
+
+    context = {'builds': build_info}
+
+    return render(request, template, context)
bitbake/lib/webhob/whbmain/urls.py +1 -1
index 04e0352..a15fc1b 100644
@@ -5,8 +5,8 @@ from django.conf.urls import patterns, include, url
 # admin.autodiscover()
 
 urlpatterns = patterns('',
+    url(r'^build/$', include('bldviewer.urls')),
     # Examples:
-    # url(r'^$', 'webhob.views.home', name='home'),
     # url(r'^webhob/', include('webhob.foo.urls')),
 
     # Uncomment the admin/doc line below to enable admin documentation:
Signature · 1 line
1.8.1.2
#9 lore raw

PATCH 9/94 bitbake: webhob: view a table with all the tasks of a build

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch will display all tasks executed for a build. There is detailed information about each task executed, if any. In case there were no tasks executed for a build, a warning message is displayed.

---
 bitbake/lib/webhob/bldviewer/templates/task.html | 81 ++++++++++++++++++++++++
 bitbake/lib/webhob/bldviewer/urls.py             |  1 +
 bitbake/lib/webhob/bldviewer/views.py            | 11 +++-
 bitbake/lib/webhob/whbmain/urls.py               |  2 +-
 4 files changed, 93 insertions(+), 2 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/task.html
bitbake/lib/webhob/bldviewer/templates/task.html new file +81 -0
new file mode 100644
index 0000000..26a09e9
@@ -0,0 +1,81 @@
+<html>
+
+	<head>
+		<title>WebHob Task page</title>
+	</head>
+
+<body>
+
+	<h1>WebHob Tasks</h1>
+
+	{% if not tasks %}
+		<p>No tasks were executed in this build!</p>
+	{% else %}
+
+		<table border="1">
+
+			<tr>
+			<th>Order</th>
+			<th>Task</th>
+			<th>Recipe</th>
+			<th>Recipe Version</th>
+			<th>Task Type</th>
+			<th>Outcome</th>
+			<th>Errors</th>
+			<th>Warnings</th>
+			<th>Time</th>
+			<th>Log</th>
+			<th>Work directory</th>
+			<th>CPU usage</th>
+			<th>Disk I/O</th>
+			<th>Sstate Checksum</th>
+			<th>Path to sstate object</th>
+			<th>Source URL</th>
+			<th>Script type</th>
+			<th>File path</th>
+			<th>Line number</th>
+			<th>Python Stack Trace</th>
+			<th>Sstate Result</th>
+			</tr>
+
+			{% for task in tasks %}
+
+				<tr>
+				<td>{{task.order}}</td>
+				<td>{{task.task_name}}</td>
+				<td>{{task.recipe.name}}</td>
+				<td>{{task.recipe.version}}</td>
+
+				{% if task.task_executed %}
+					<td>Executed</td>
+				{% else %}
+					<td>Prebuilt</td>
+				{% endif %}
+
+				<td>{{task.get_outcome_display}}</td>
+				<td>{{task.errors_no}}</td>
+				<td>{{task.warnings_no}}</td>
+				<td>{{task.elapsed_time}}</td>
+				<td>{{task.log_file}}</td>
+				<td>{{task.work_directory}}</td>
+				<td>{{task.cpu_usage}}</td>
+				<td>{{task.disk_io}}</td>
+				<td>{{task.sstate_checksum}}</td>
+				<td>{{task.path_to_sstate_obj}}</td>
+				<td>{{task.source_url}}</td>
+				<td>{{task.get_script_type_display}}</td>
+				<td>{{task.file_path}}</td>
+				<td>{{task.line_number}}</td>
+				<td>{{task.py_stack_trace}}</td>
+				<td>{{task.get_sstate_result_display}}</td>
+				</tr>
+
+			{% endfor %}
+
+		</table>
+
+	{% endif %}
+
+</body>
+
+</html>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
index 3ec2a30..335885c 100644
@@ -3,4 +3,5 @@ from django.conf.urls import patterns, include, url
 
 urlpatterns = patterns('bldviewer.views',
         url(r'^$', 'build', name='build'),
+        url(r'^(?P<build_id>\d+)/$', 'task', name='task'),
 )
bitbake/lib/webhob/bldviewer/views.py +10 -1
index 15a2d1d..292a638 100644
@@ -1,5 +1,5 @@
 from django.shortcuts import render
-from orm.models import Build
+from orm.models import Build, Task
 
 
 def build(request):
@@ -9,3 +9,12 @@ def build(request):
     context = {'builds': build_info}
 
     return render(request, template, context)
+
+def task(request, build_id):
+    template = 'task.html'
+
+    tasks = Task.objects.filter(build=build_id)
+
+    context = {'tasks': tasks}
+
+    return render(request, template, context)
bitbake/lib/webhob/whbmain/urls.py +1 -1
index a15fc1b..c40e7c7 100644
@@ -5,7 +5,7 @@ from django.conf.urls import patterns, include, url
 # admin.autodiscover()
 
 urlpatterns = patterns('',
-    url(r'^build/$', include('bldviewer.urls')),
+    url(r'^build/', include('bldviewer.urls')),
     # Examples:
     # url(r'^webhob/', include('webhob.foo.urls')),
 
Signature · 1 line
1.8.1.2
#10 lore raw

PATCH 10/94 bitbake: webhob: use buildcompleted event as the end of a build operation

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

The BuildCompleted event marks the end of a build, so the code will use this timestamp for computing the duration of a build operation.

---
 bitbake/lib/bb/ui/dsi.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/bb/ui/dsi.py +1 -1
index 54364ce..d5fe733 100644
@@ -448,7 +448,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
-            if isinstance(event, (bb.command.CommandFailed, bb.command.CommandExit, bb.command.CommandCompleted, bb.cooker.CookerExit)):
+            if isinstance(event, bb.event.BuildCompleted):
                 buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
                 main.shutdown = 2
 
Signature · 1 line
1.8.1.2
#11 lore raw

PATCH 11/94 bitbake: webhob: changes to build information table

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

It was decided that disk_io and cpu usage information are not very relevant for a full build, so these fields have been droped from our models and replaced with output information (image_fstypes field)

---
 bitbake/lib/bb/ui/buildinfohelper.py              | 6 ++----
 bitbake/lib/webhob/bldviewer/templates/build.html | 6 ++----
 bitbake/lib/webhob/orm/models.py                  | 3 +--
 3 files changed, 5 insertions(+), 10 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +2 -4
index 57e404b..e1ebd41 100644
@@ -47,8 +47,7 @@ class ORMWrapper(object):
         build_info['outcome'] = 0
         build_info['number_of_errors'] = 0
         build_info['number_of_warnings'] = 0
-        build_info['cpu_usage'] = 0
-        build_info['disk_io'] = 0
+        build_info['image_fstypes'] = server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0]
         build_info['cooker_log_path'] = server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
         build_info['build_name'] = server.runCommand(["getVariable", "BUILDNAME"])[0]
         build_info['bitbake_version'] = server.runCommand(["getVariable", "BB_VERSION"])[0]
@@ -69,8 +68,7 @@ class ORMWrapper(object):
                                     outcome=build_info['outcome'],
                                     errors_no=build_info['number_of_errors'],
                                     warnings_no=build_info['number_of_warnings'],
-                                    cpu_usage=build_info['cpu_usage'],
-                                    disk_io=build_info['disk_io'],
+                                    image_fstypes=build_info['image_fstypes'],
                                     cooker_log_path=build_info['cooker_log_path'],
                                     build_name=build_info['build_name'],
                                     bitbake_version=build_info['bitbake_version'])
bitbake/lib/webhob/bldviewer/templates/build.html +2 -4
index 9a2d412..1b694d6 100644
@@ -20,8 +20,7 @@
 			<th>Time</th>
 			<th>Errors</th>
 			<th>Warnings</th>
-			<th>CPU usage</th>
-			<th>Disk I/O</th>
+			<th>Output</th>
 			<th>Log</th>
 			<th>Bitbake Version</th>
 			<th>Build Name</th>
@@ -38,8 +37,7 @@
 			<td>{% time_difference build.started_on build.completed_on %}</td>
 			<td>{{build.errors_no}}</td>
 			<td>{{build.warnings_no}}</td>
-			<td>{{build.cpu_usage}}</td>
-			<td>{{build.disk_io}}</td>
+			<td>{{build.image_fstypes}}</td>
 			<td>{{build.cooker_log_path}}</td>
 			<td>{{build.bitbake_version}}</td>
 			<td>{{build.build_name}}</td>
bitbake/lib/webhob/orm/models.py +1 -2
index 6b8e2fa..f0492db 100644
@@ -18,8 +18,7 @@ class Build(models.Model):
     outcome = models.IntegerField(choices=BUILD_OUTCOME)
     errors_no = models.IntegerField()
     warnings_no = models.IntegerField()
-    cpu_usage = models.IntegerField()
-    disk_io = models.DecimalField(max_digits=20, decimal_places=10)
+    image_fstypes = models.CharField(max_length=100)
     cooker_log_path = models.CharField(max_length=500)
     build_name = models.CharField(max_length=100)
     bitbake_version = models.CharField(max_length=50)
Signature · 1 line
1.8.1.2
#12 lore raw

PATCH 12/94 bitbake: webhob: gather buildstats for each executed task

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch reads the buildstats file created for each executed task and retrieves cpu usage and disk IO information which it will store in the database.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 89 ++++++++++++++++++++++++++++++------
 bitbake/lib/bb/ui/dsi.py             |  4 +-
 bitbake/lib/webhob/orm/models.py     |  4 +-
 3 files changed, 78 insertions(+), 19 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +74 -15
index e1ebd41..6f3b5c2 100644
@@ -9,17 +9,20 @@ class ORMWrapper(object):
         information in the database.
     """
 
-    def __init__(self):
+    def __init__(self, server):
         self.uuid = None
         self.task_order = 0
         self.transport_utils = {}
+        self.server = server
 
-    def get_machine_information(self, server):
+    def get_machine_information(self):
         machine_info = {}
 
-        machine_info['name'] = server.runCommand(["getVariable", "MACHINE"])[0]
+        machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0]
         machine_info['description'] = 'Not Available'
 
+        self.transport_utils['machine'] = machine_info['name']
+
         return machine_info
 
     def create_machine_object(self, machine_information):
@@ -31,7 +34,7 @@ class ORMWrapper(object):
 
         return machine[0]
 
-    def get_build_information(self, server, machine_obj):
+    def get_build_information(self, machine_obj):
         build_info = {}
 
         # Generate an identifier for each new build
@@ -40,17 +43,17 @@ class ORMWrapper(object):
         build_info['uuid'] = self.uuid
         build_info['target'] = 'Not Available'
         build_info['machine'] = machine_obj
-        build_info['distro'] = server.runCommand(["getVariable", "DISTRO"])[0]
-        build_info['distro_version'] = server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
+        build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0]
+        build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
         build_info['started_on'] = datetime.datetime.now()
         build_info['completed_on'] = datetime.datetime.now()
         build_info['outcome'] = 0
         build_info['number_of_errors'] = 0
         build_info['number_of_warnings'] = 0
-        build_info['image_fstypes'] = server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0]
-        build_info['cooker_log_path'] = server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
-        build_info['build_name'] = server.runCommand(["getVariable", "BUILDNAME"])[0]
-        build_info['bitbake_version'] = server.runCommand(["getVariable", "BB_VERSION"])[0]
+        build_info['image_fstypes'] = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0]
+        build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
+        build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0]
+        build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0]
 
         return build_info
 
@@ -175,6 +178,10 @@ class ORMWrapper(object):
         outcome_info = self.get_outcome_of_task(event)
         task.outcome = outcome_info['task_outcome']
 
+        task_build_stats = self.get_task_build_stats(task_object)
+        task.cpu_usage = task_build_stats['cpu_usage']
+        task.disk_io = task_build_stats['disk_io']
+
         if outcome_info.get('error', ''):
             task.error = outcome_info['error']
 
@@ -202,6 +209,57 @@ class ORMWrapper(object):
 
         return outcome_info
 
+    def get_task_build_stats(self, task_object):
+        bs_path = self.get_path_information(task_object)
+        task_build_stats = self.get_build_stats_from_file(bs_path, task_object.task_name)
+
+        return task_build_stats
+
+    def get_path_information(self, task_object):
+        build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/"
+
+        tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
+        target = self.transport_utils['target']
+        machine = self.transport_utils['build'].machine.name
+        buildname = self.transport_utils['build'].build_name
+        package = task_object.recipe.name
+
+        build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target,
+                                                     machine=machine, buildname=buildname,
+                                                     package=package)
+
+        return build_stats_path
+
+    def get_build_stats_from_file(self, bs_path, task_name):
+
+        task_bs_filename = str(bs_path) + str(task_name)
+        task_bs = open(task_bs_filename, 'r')
+
+        cpu_usage = 0
+        disk_io = 0
+        startio = ''
+        endio = ''
+
+        for line in task_bs.readlines():
+            if line.startswith('CPU usage: '):
+                cpu_usage = line[11:]
+            elif line.startswith('EndTimeIO: '):
+                endio = line[11:]
+            elif line.startswith('StartTimeIO: '):
+                startio = line[13:]
+
+        task_bs.close()
+
+        if startio and endio:
+            disk_io = int(endio.strip('\n ')) - int(startio.strip('\n '))
+
+        if cpu_usage:
+            cpu_usage = float(cpu_usage.strip('% \n'))
+
+        task_build_stats = {'cpu_usage': cpu_usage, 'disk_io': disk_io}
+
+        return task_build_stats
+
     def get_recipe_object(self, event):
         # This needs to be imported after we have configured the Django settings file
         from webhob.orm.models import Recipe
@@ -303,9 +361,9 @@ class BuildInfoHelper(object):
         towards the ORM wrapper for storing in the database
     """
 
-    def __init__(self):
+    def __init__(self, server):
         self.configure_django()
-        self.orm_wrapper = ORMWrapper()
+        self.orm_wrapper = ORMWrapper(server)
 
     def configure_django(self):
         import webhob.whbmain.settings as whb_django_settings
@@ -314,14 +372,15 @@ class BuildInfoHelper(object):
         # Add webhob to sys path for importing modules
         sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
 
-    def store_started_build(self, event, server):
+    def store_started_build(self, event):
 
-        machine_information = self.orm_wrapper.get_machine_information(server)
+        machine_information = self.orm_wrapper.get_machine_information()
         machine_obj = self.orm_wrapper.create_machine_object(machine_information)
 
-        build_information = self.orm_wrapper.get_build_information(server, machine_obj)
+        build_information = self.orm_wrapper.get_build_information(machine_obj)
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.orm_wrapper.transport_utils['build'] = build_obj
+        self.orm_wrapper.transport_utils['target'] = event.getPkgs()[0]
 
     def update_build_information(self, event, errors, warnings, taskfailures):
         self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures)
bitbake/lib/bb/ui/dsi.py +2 -2
index d5fe733..fb3595b 100644
@@ -285,7 +285,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
     errors = 0
     warnings = 0
     taskfailures = []
-    buildinfohelper = BuildInfoHelper()
+    buildinfohelper = BuildInfoHelper(server)
 
     termfilter = tf(main, helper, console, format)
 
@@ -306,7 +306,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                     main.shutdown = 1
 
             if isinstance(event, bb.event.BuildStarted):
-                buildinfohelper.store_started_build(event, server)
+                buildinfohelper.store_started_build(event)
 
             if isinstance(event, bb.build.TaskStarted):
                 buildinfohelper.store_started_task(event)
bitbake/lib/webhob/orm/models.py +2 -2
index f0492db..76b7c1d 100644
@@ -61,8 +61,8 @@ class Task(models.Model):
     file_path = models.FilePathField(max_length=255)
     line_number = models.IntegerField()
     py_stack_trace = models.TextField()
-    disk_io = models.DecimalField(max_digits=20, decimal_places=10)
-    cpu_usage = models.IntegerField()
+    disk_io = models.IntegerField()
+    cpu_usage = models.DecimalField(max_digits=6, decimal_places=2)
     elapsed_time = models.CharField(max_length=50)
     errors_no = models.IntegerField()
     warnings_no = models.IntegerField()
Signature · 1 line
1.8.1.2
#13 lore raw

PATCH 13/94 bitbake: webhob: set sane default settings

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Changes the default Django settings as to point
to sane default of using sqlite by default. This
is the preffered running mode for webhob.

Changes the list of installed applications.

---
 bitbake/lib/webhob/whbmain/settings.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
bitbake/lib/webhob/whbmain/settings.py +4 -2
index 71bba4f..cd9dbe8 100644
@@ -11,8 +11,8 @@ MANAGERS = ADMINS
 
 DATABASES = {
     'default': {
-        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
-        'NAME': '',                      # Or path to database file if using sqlite3.
+        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
+        'NAME': 'webhob.sqlite',                      # Or path to database file if using sqlite3.
         'USER': '',
         'PASSWORD': '',
         'HOST': '127.0.0.1',                      # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
@@ -124,6 +124,8 @@ INSTALLED_APPS = (
     # Uncomment the next line to enable admin documentation:
     # 'django.contrib.admindocs',
     'orm',
+    'whbmain',
+    'bldviewer',
 )
 
 # A sample logging configuration. The only tangible logging
Signature · 1 line
1.8.1.2
#14 lore raw

PATCH 14/94 bitbake: webhob: adds dsi support for observer mode

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding support in the DSI user interface to remain resident and run in observer-only mode.

Modifies the webhob launch script to automatically lunch the DSI as to log all builds run under webhob setup.

---
 bitbake/bin/webhob       |  5 ++++-
 bitbake/lib/bb/ui/dsi.py | 47 ++++++++++++++---------------------------------
 2 files changed, 18 insertions(+), 34 deletions(-)
bitbake/bin/webhob +4 -1
index ccaf91f..8513a3a 100755
@@ -28,7 +28,7 @@ function webserverKillAllComponents()
 	for pidfile in ${BUILDDIR}/whbmain.pid; do
 		if [ -f ${pidfile} ]; then
 		while kill -0 $(< ${pidfile}) 2>/dev/null; do
-			kill -SIGTERM -`ps -p $(< ${pidfile}) -o "%r" --no-headers`	
+			kill -SIGTERM -$(< ${pidfile})
 			sleep 1;
 		done;
 		rm  ${pidfile}
@@ -84,9 +84,12 @@ case $CMD in
         python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0)
         bitbake --server-only -t xmlrpc -B localhost:8200 
         export BBSERVER=localhost:8200
+        bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid
         python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
     ;;
     stop )
+        kill $(< ${BUILDDIR}/dsi.pid )
+        rm ${BUILDDIR}/dsi.pid
         bitbake -m
         unset BBSERVER
         webserverKillAllComponents
bitbake/lib/bb/ui/dsi.py +14 -33

91 lines not on this page · raw message

Signature · 1 line
1.8.1.2
#15 lore raw

PATCH 15/94 bitbake: dsi: Translate runQueue events into Task data

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We're moving the capture of task running information from bb.build.Task* events to runQueue* events.

We handle runQueueTaskSkipped events to record all skipped tasks.

---
 bitbake/lib/bb/ui/dsi.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
bitbake/lib/bb/ui/dsi.py +7 -4
index 5ff5696..e527e56 100644
@@ -287,11 +287,8 @@ def main(server, eventHandler, params, tf = TerminalFilter):
             if isinstance(event, bb.event.BuildStarted):
                 buildinfohelper.store_started_build(event)
 
-            if isinstance(event, bb.build.TaskStarted):
-                buildinfohelper.store_started_task(event)
-
             if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)):
-                buildinfohelper.update_and_store_task(event)
+                continue
 
             if isinstance(event, bb.event.LogExecTTY):
                 if log_exec_tty:
@@ -416,13 +413,19 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                             event.stats.total, event.taskid, event.taskstring)
                 continue
 
+            if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.runQueueTaskSkipped)):
+                buildinfohelper.store_started_task(event)
+                continue
+
             if isinstance(event, bb.runqueue.runQueueTaskFailed):
+                buildinfohelper.update_and_store_task(event)
                 taskfailures.append(event.taskstring)
                 logger.error("Task %s (%s) failed with exit code '%s'",
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
             if isinstance(event, bb.runqueue.sceneQueueTaskFailed):
+                buildinfohelper.update_and_store_task(event)
                 logger.warn("Setscene task %s (%s) failed with exit code '%s' - real task will be run instead",
                              event.taskid, event.taskstring, event.exitcode)
                 continue
Signature · 1 line
1.8.1.2
#16 lore raw

PATCH 16/94 bitbake: webhob: disable synchronous sqlite connection

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

SQLite has very bad INSERT performance in synchronous mode. We disable the synchronous mode to improve performance only on sqlite connections.

This requires also to change the way the build info wrapper loads the Django settings.

---
 bitbake/lib/bb/ui/buildinfohelper.py   |  3 +--
 bitbake/lib/webhob/whbmain/settings.py | 10 ++++++++++
 2 files changed, 11 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +1 -2
index 6f3b5c2..e12750e 100644
@@ -366,9 +366,8 @@ class BuildInfoHelper(object):
         self.orm_wrapper = ORMWrapper(server)
 
     def configure_django(self):
+        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
         import webhob.whbmain.settings as whb_django_settings
-        from django.core.management import setup_environ
-        setup_environ(whb_django_settings)
         # Add webhob to sys path for importing modules
         sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
 
bitbake/lib/webhob/whbmain/settings.py +10 -0
index cd9dbe8..aa1d4fc 100644
@@ -20,6 +20,16 @@ DATABASES = {
     }
 }
 
+# If we're using sqlite, we need to tweak the performance a bit
+from django.db.backends.signals import connection_created
+def activate_synchronous_off(sender, connection, **kwargs):
+    if connection.vendor == 'sqlite':
+        cursor = connection.cursor()
+        cursor.execute('PRAGMA synchronous = 0;')
+connection_created.connect(activate_synchronous_off)
+#
+
+
 # Hosts/domain names that are valid for this site; required if DEBUG is False
 # See https://docs.djangoproject.com/en/1.5/ref/settings/#allowed-hosts
 ALLOWED_HOSTS = []
Signature · 1 line
1.8.1.2
#17 lore raw

PATCH 17/94 bitbake: buildinfohelper: clean-up the imports

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

As per convention, we do all imports at the beginning of file.

Also, globally set Django settings in the Django interface file.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 15 +++++----------
 1 file changed, 5 insertions(+), 10 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +5 -10
index e12750e..f7f98d8 100644
@@ -3,6 +3,11 @@ import sys
 import uuid
 
 
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
+
+import webhob.whbmain.settings as whb_django_settings
+from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer
+
 class ORMWrapper(object):
     """ This class creates the dictionaries needed to store information in the database
         following the format defined by the Django models. It is also used to save this
@@ -27,7 +32,6 @@ class ORMWrapper(object):
 
     def create_machine_object(self, machine_information):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Machine
 
         machine = Machine.objects.get_or_create(name=machine_information['name'],
                                                 description=machine_information['description'])
@@ -59,7 +63,6 @@ class ORMWrapper(object):
 
     def create_build_object(self, build_info):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Build
 
         build = Build.objects.create(uuid=build_info['uuid'],
                                     target=build_info['target'],
@@ -80,7 +83,6 @@ class ORMWrapper(object):
 
     def update_build_object(self, build_obj, errors, warnings, taskfailures):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Build
 
         outcome = 0
         if errors or taskfailures:
@@ -128,8 +130,6 @@ class ORMWrapper(object):
 
     def create_task_object(self, task_information):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Task
-
         identifier = task_information['recipe'].name + task_information['task_name']
 
         task_object = Task.objects.get_or_create(
@@ -262,7 +262,6 @@ class ORMWrapper(object):
 
     def get_recipe_object(self, event):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Recipe
 
         recipe_information = self.get_recipe_information(event)
 
@@ -305,7 +304,6 @@ class ORMWrapper(object):
 
     def get_build_layer_object(self, event):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Build_Layer
 
         build_layer_information = self.get_build_layer_information(event)
 
@@ -334,7 +332,6 @@ class ORMWrapper(object):
 
     def get_layer_object(self, event):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Layer
 
         layer_information = self.get_layer_information(event)
 
@@ -366,8 +363,6 @@ class BuildInfoHelper(object):
         self.orm_wrapper = ORMWrapper(server)
 
     def configure_django(self):
-        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
-        import webhob.whbmain.settings as whb_django_settings
         # Add webhob to sys path for importing modules
         sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
 
Signature · 1 line
1.8.1.2
#18 lore raw

PATCH 18/94 bitbake: buildinfohelper: clearing up task order

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Moved code around so that the task order is issued only on creating new task method, and not on get information.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/buildinfohelper.py +2 -1
index f7f98d8..c743751 100644
@@ -130,11 +130,12 @@ class ORMWrapper(object):
 
     def create_task_object(self, task_information):
         # This needs to be imported after we have configured the Django settings file
+        self.task_order += 1
         identifier = task_information['recipe'].name + task_information['task_name']
 
         task_object = Task.objects.get_or_create(
                                 build=task_information['build'],
-                                order=task_information['order'],
+                                order=self.task_order,
                                 task_executed=task_information['task_executed'],
                                 outcome=task_information['outcome'],
                                 sstate_checksum=task_information['sstate_checksum'],
Signature · 1 line
1.8.1.2
#19 lore raw

PATCH 19/94 bitbake: dsi: saving prebuild task detailed information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Changes to the internal structure of the database interface layer.

Clearing up the code to save information about prebuild tasks.

Clear up the event-processing fall-through code in DSI.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 54 +++++++++++++-----------------------
 bitbake/lib/bb/ui/dsi.py             |  9 ++++--
 2 files changed, 25 insertions(+), 38 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +19 -35
index c743751..a0da2e1 100644
@@ -1,6 +1,7 @@
 import datetime
 import sys
 import uuid
+import bb
 
 
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
@@ -97,19 +98,24 @@ class ORMWrapper(object):
 
     def get_task_information(self, event):
 
-        self.task_order += 1
 
         recipe = self.get_recipe_object(event)
 
         task_information = {}
         task_information['build'] = self.transport_utils['build']
-        task_information['order'] = self.task_order
-        task_information['task_executed'] = True
-        task_information['outcome'] = 0
+        task_information['outcome'] = 4
+        if isinstance(event, bb.runqueue.runQueueTaskSkipped):
+            task_information['task_executed'] = False
+            if event._skip == "covered":
+                task_information['outcome'] = 2
+            if event._skip == "existing":
+                task_information['outcome'] = 0
+        else:
+            task_information['task_executed'] = True
         task_information['sstate_checksum'] = 'Not Available'
         task_information['path_to_sstate_obj'] = 'Not Available'
         task_information['recipe'] = recipe
-        task_information['task_name'] = event._task
+        task_information['task_name'] = event.taskname
         task_information['source_url'] = 'Not Available'
         task_information['log_file'] = 'Not Available'
         task_information['work_directory'] = 'Not Available'
@@ -165,21 +171,16 @@ class ORMWrapper(object):
 
     def update_task_object(self, task_dictionary, event):
         # This needs to be imported after we have configured the Django settings file
-        from webhob.orm.models import Task
-
-        task_object = task_dictionary['object']
 
-        task = Task.objects.get(build=task_object.build,
-                                task_name=task_object.task_name,
-                                recipe=task_object.recipe)
+        task = task_dictionary['object']
 
         duration = datetime.datetime.now() - task_dictionary['start_time']
         task.elapsed_time = duration.total_seconds()
 
-        outcome_info = self.get_outcome_of_task(event)
-        task.outcome = outcome_info['task_outcome']
+        if isinstance(event, bb.runqueue.runQueueTaskCompleted):
+            task.outcome = 3
 
-        task_build_stats = self.get_task_build_stats(task_object)
+        task_build_stats = self.get_task_build_stats(task)
         task.cpu_usage = task_build_stats['cpu_usage']
         task.disk_io = task_build_stats['disk_io']
 
@@ -192,24 +193,6 @@ class ORMWrapper(object):
 
         task.save()
 
-    def get_outcome_of_task(self, event):
-        from webhob.orm.models import Task
-
-        outcome_info = {}
-
-        task_result = event.getDisplayName().lower()
-        for outcome in Task.TASK_OUTCOME:
-            if task_result == outcome[1].lower():
-                task_outcome = outcome[0]
-
-        outcome_info['task_outcome'] = task_outcome
-
-        if isinstance(event, bb.build.TaskFailed):
-            if event.logfile and os.path.exists(event.logfile):
-                outcome_info['error'] = open(event.logfile, "r").read()
-
-        return outcome_info
-
     def get_task_build_stats(self, task_object):
         bs_path = self.get_path_information(task_object)
         task_build_stats = self.get_build_stats_from_file(bs_path, task_object.task_name)
@@ -288,7 +271,7 @@ class ORMWrapper(object):
         build_layer_obj = self.get_build_layer_object(event)
 
         recipe_info = {}
-        recipe_info['name'] = event._package
+        recipe_info['name'] = event.taskpackage
         recipe_info['version'] = 'Not Available'
         recipe_info['layer'] = build_layer_obj
         recipe_info['summary'] = 'Not Available'
@@ -373,9 +356,10 @@ class BuildInfoHelper(object):
         machine_obj = self.orm_wrapper.create_machine_object(machine_information)
 
         build_information = self.orm_wrapper.get_build_information(machine_obj)
+        build_information['target'] = ' '.join(event.getPkgs())
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.orm_wrapper.transport_utils['build'] = build_obj
-        self.orm_wrapper.transport_utils['target'] = event.getPkgs()[0]
+        self.orm_wrapper.transport_utils['target'] = build_information['target']
 
     def update_build_information(self, event, errors, warnings, taskfailures):
         self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures)
@@ -386,7 +370,7 @@ class BuildInfoHelper(object):
         task_obj = self.orm_wrapper.create_task_object(task_information)
 
     def update_and_store_task(self, event):
-        identifier = event._package + event._task
+        identifier = event.taskpackage + event.taskname
         task_dictionary = self.orm_wrapper.transport_utils[identifier]
         self.orm_wrapper.update_task_object(task_dictionary, event)
         del self.orm_wrapper.transport_utils[identifier]
bitbake/lib/bb/ui/dsi.py +6 -3
index e527e56..6264ecc 100644
@@ -411,12 +411,17 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                             event.stats.completed + event.stats.active +
                                 event.stats.failed + 1,
                             event.stats.total, event.taskid, event.taskstring)
+                buildinfohelper.store_started_task(event)
                 continue
 
-            if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.runQueueTaskSkipped)):
+            if isinstance(event, (bb.runqueue.runQueueTaskStarted, bb.runqueue.runQueueTaskSkipped)):
                 buildinfohelper.store_started_task(event)
                 continue
 
+            if isinstance(event, bb.runqueue.runQueueTaskCompleted):
+                buildinfohelper.update_and_store_task(event)
+                continue
+
             if isinstance(event, bb.runqueue.runQueueTaskFailed):
                 buildinfohelper.update_and_store_task(event)
                 taskfailures.append(event.taskstring)
@@ -430,8 +435,6 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
-            if isinstance(event, bb.event.BuildCompleted):
-                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
 
             if isinstance(event, bb.event.ConfigParsed):
                 # timestamp should be added for this
Signature · 1 line
1.8.1.2
#20 lore raw

PATCH 20/94 bitbake: dsi: clear up the state on build completion

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

In order to prevent cross-data between different builds, we drop the build helper object which contains the state of the current build when the build is completed.

---
 bitbake/lib/bb/ui/dsi.py | 9 +++++++++
 1 file changed, 9 insertions(+)
bitbake/lib/bb/ui/dsi.py +9 -0
index 6264ecc..90dd732 100644
@@ -458,6 +458,15 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 # trigger an error
                 continue
 
+            if isinstance(event, (bb.event.BuildCompleted,
+                                  bb.command.CommandCompleted,
+                                  bb.command.CommandFailed,
+                                  bb.command.CommandExit)):
+                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
+                # we start a new build info
+                buildinfohelper = BuildInfoHelper(server)
+                continue
+
             # ignore
             if isinstance(event, (bb.event.BuildBase,
                                   bb.event.StampUpdate,
Signature · 1 line
1.8.1.2
#21 lore raw

PATCH 21/94 bitbake: webhob: add layer information to the database

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch adds the layer information to the database. Using the values stored in BBLAYERS it retrieves all the information needed for the database fields.
It also includes a hardcoded mapping to deal with the URLs that are different from the standard naming convention.

---
 bitbake/lib/bb/ui/buildinfohelper.py              | 45 +++++++++++++++++++++++
 bitbake/lib/webhob/bldviewer/templates/layer.html | 34 +++++++++++++++++
 bitbake/lib/webhob/bldviewer/urls.py              |  1 +
 bitbake/lib/webhob/bldviewer/views.py             | 10 ++++-
 4 files changed, 89 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/layer.html
bitbake/lib/bb/ui/buildinfohelper.py +45 -0
index a0da2e1..2da38e2 100644
@@ -336,6 +336,49 @@ class ORMWrapper(object):
 
         return layer_info
 
+    def _get_layer_dict(self, layer_path):
+
+        layer_info = {}
+        layer_name = layer_path.split('/')[-1]
+        layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/'
+        layer_url_name = self._get_url_map_name(layer_name)
+
+        layer_info['name'] = layer_name
+        layer_info['local_path'] = layer_path
+        layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name)
+
+        return layer_info
+
+    def _get_url_map_name(self, layer_name):
+        """ Some layers have a different name on openembedded.org site,
+            this method returns the correct name to use in the URL
+        """
+
+        url_name = layer_name
+        url_mapping = {'meta': 'openembedded-core'}
+
+        for key in url_mapping.keys():
+            if key == layer_name:
+                url_name = url_mapping[key]
+
+        return url_name
+
+    def _save_layer_object(self, layer_path):
+
+        layer_information = self._get_layer_dict(layer_path)
+
+        layer_object = Layer.objects.get_or_create(
+                            name=layer_information['name'],
+                            local_path=layer_information['local_path'],
+                            layer_index_url=layer_information['layer_index_url'])
+
+        layer_object[0].save()
+
+    def store_layer_info(self):
+        layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ")
+        for layer_path in layers:
+            self._save_layer_object(layer_path)
+
 
 class BuildInfoHelper(object):
     """ This class gathers the build information from the server and sends it
@@ -352,6 +395,8 @@ class BuildInfoHelper(object):
 
     def store_started_build(self, event):
 
+        self.orm_wrapper.store_layer_info()
+
         machine_information = self.orm_wrapper.get_machine_information()
         machine_obj = self.orm_wrapper.create_machine_object(machine_information)
 
bitbake/lib/webhob/bldviewer/templates/layer.html new file +34 -0
new file mode 100644
index 0000000..905a781
@@ -0,0 +1,34 @@
+<html>
+
+	<head>
+		<title>WebHob Layer page</title>
+	</head>
+
+<body>
+	<h1>WebHob Layers</h1>
+
+	<table border="1">
+
+	{% load projecttags %}
+
+		<tr>
+			<th>Name</th>
+			<th>Local Path</th>
+			<th>Layer Index URL</th>
+		</tr>
+
+		{% for layer in layers %}
+
+		<tr>
+			<td>{{layer.name}}</td>
+			<td>{{layer.local_path}}</td>
+			<td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td>
+		</tr>
+
+		{% endfor %}
+
+	</table>
+
+</body>
+
+</html>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
index 335885c..8d5dc6e 100644
@@ -3,5 +3,6 @@ from django.conf.urls import patterns, include, url
 
 urlpatterns = patterns('bldviewer.views',
         url(r'^$', 'build', name='build'),
+        url(r'^layers/$', 'layer', name='layer'),
         url(r'^(?P<build_id>\d+)/$', 'task', name='task'),
 )
bitbake/lib/webhob/bldviewer/views.py +9 -1
index 292a638..f40d396 100644
@@ -1,5 +1,5 @@
 from django.shortcuts import render
-from orm.models import Build, Task
+from orm.models import Build, Task, Layer
 
 
 def build(request):
@@ -18,3 +18,11 @@ def task(request, build_id):
     context = {'tasks': tasks}
 
     return render(request, template, context)
+
+def layer(request):
+    template = 'layer.html'
+    layer_info = Layer.objects.all()
+
+    context = {'layers': layer_info}
+
+    return render(request, template, context)
Signature · 1 line
1.8.1.2
#22 lore raw

PATCH 22/94 bitbake: webhob: mark private methods inside DSI

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch marks private methods used in the DSI.
It also cleans some old meaningless comments.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 49 +++++++++++++++---------------------
 1 file changed, 20 insertions(+), 29 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +20 -29
index 2da38e2..820e0f8 100644
@@ -32,7 +32,6 @@ class ORMWrapper(object):
         return machine_info
 
     def create_machine_object(self, machine_information):
-        # This needs to be imported after we have configured the Django settings file
 
         machine = Machine.objects.get_or_create(name=machine_information['name'],
                                                 description=machine_information['description'])
@@ -63,7 +62,6 @@ class ORMWrapper(object):
         return build_info
 
     def create_build_object(self, build_info):
-        # This needs to be imported after we have configured the Django settings file
 
         build = Build.objects.create(uuid=build_info['uuid'],
                                     target=build_info['target'],
@@ -83,7 +81,6 @@ class ORMWrapper(object):
         return build
 
     def update_build_object(self, build_obj, errors, warnings, taskfailures):
-        # This needs to be imported after we have configured the Django settings file
 
         outcome = 0
         if errors or taskfailures:
@@ -98,8 +95,7 @@ class ORMWrapper(object):
 
     def get_task_information(self, event):
 
-
-        recipe = self.get_recipe_object(event)
+        recipe = self._get_recipe_object(event)
 
         task_information = {}
         task_information['build'] = self.transport_utils['build']
@@ -135,7 +131,6 @@ class ORMWrapper(object):
         return task_information
 
     def create_task_object(self, task_information):
-        # This needs to be imported after we have configured the Django settings file
         self.task_order += 1
         identifier = task_information['recipe'].name + task_information['task_name']
 
@@ -170,7 +165,6 @@ class ORMWrapper(object):
         return task_object[0]
 
     def update_task_object(self, task_dictionary, event):
-        # This needs to be imported after we have configured the Django settings file
 
         task = task_dictionary['object']
 
@@ -180,7 +174,7 @@ class ORMWrapper(object):
         if isinstance(event, bb.runqueue.runQueueTaskCompleted):
             task.outcome = 3
 
-        task_build_stats = self.get_task_build_stats(task)
+        task_build_stats = self._get_task_build_stats(task)
         task.cpu_usage = task_build_stats['cpu_usage']
         task.disk_io = task_build_stats['disk_io']
 
@@ -193,13 +187,13 @@ class ORMWrapper(object):
 
         task.save()
 
-    def get_task_build_stats(self, task_object):
-        bs_path = self.get_path_information(task_object)
-        task_build_stats = self.get_build_stats_from_file(bs_path, task_object.task_name)
+    def _get_task_build_stats(self, task_object):
+        bs_path = self._get_path_information(task_object)
+        task_build_stats = self._get_build_stats_from_file(bs_path, task_object.task_name)
 
         return task_build_stats
 
-    def get_path_information(self, task_object):
+    def _get_path_information(self, task_object):
         build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/"
 
         tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
@@ -214,7 +208,7 @@ class ORMWrapper(object):
 
         return build_stats_path
 
-    def get_build_stats_from_file(self, bs_path, task_name):
+    def _get_build_stats_from_file(self, bs_path, task_name):
 
         task_bs_filename = str(bs_path) + str(task_name)
         task_bs = open(task_bs_filename, 'r')
@@ -244,10 +238,9 @@ class ORMWrapper(object):
 
         return task_build_stats
 
-    def get_recipe_object(self, event):
-        # This needs to be imported after we have configured the Django settings file
+    def _get_recipe_object(self, event):
 
-        recipe_information = self.get_recipe_information(event)
+        recipe_information = self._get_recipe_information(event)
 
         recipe_object = Recipe.objects.get_or_create(
                                 name=recipe_information['name'],
@@ -266,9 +259,9 @@ class ORMWrapper(object):
 
         return recipe_object[0]
 
-    def get_recipe_information(self, event):
+    def _get_recipe_information(self, event):
 
-        build_layer_obj = self.get_build_layer_object(event)
+        build_layer_obj = self._get_build_layer_object(event)
 
         recipe_info = {}
         recipe_info['name'] = event.taskpackage
@@ -286,10 +279,9 @@ class ORMWrapper(object):
 
         return recipe_info
 
-    def get_build_layer_object(self, event):
-        # This needs to be imported after we have configured the Django settings file
+    def _get_build_layer_object(self, event):
 
-        build_layer_information = self.get_build_layer_information(event)
+        build_layer_information = self._get_build_layer_information(event)
 
         build_layer_object = Build_Layer.objects.get_or_create(
                                     build = build_layer_information['build'],
@@ -301,9 +293,9 @@ class ORMWrapper(object):
 
         return build_layer_object[0]
 
-    def get_build_layer_information(self, event):
+    def _get_build_layer_information(self, event):
 
-        layer_object = self.get_layer_object(event)
+        layer_object = self._get_layer_object(event)
 
         build_layer_info = {}
         build_layer_info['build'] = self.transport_utils['build']
@@ -314,10 +306,9 @@ class ORMWrapper(object):
 
         return build_layer_info
 
-    def get_layer_object(self, event):
-        # This needs to be imported after we have configured the Django settings file
+    def _get_layer_object(self, event):
 
-        layer_information = self.get_layer_information(event)
+        layer_information = self._get_layer_information(event)
 
         layer_object = Layer.objects.get_or_create(
                                 name=layer_information['name'],
@@ -327,7 +318,7 @@ class ORMWrapper(object):
 
         return layer_object[0]
 
-    def get_layer_information(self, event):
+    def _get_layer_information(self, event):
 
         layer_info = {}
         layer_info['name'] = 'Layer Test Name'
@@ -386,10 +377,10 @@ class BuildInfoHelper(object):
     """
 
     def __init__(self, server):
-        self.configure_django()
+        self._configure_django()
         self.orm_wrapper = ORMWrapper(server)
 
-    def configure_django(self):
+    def _configure_django(self):
         # Add webhob to sys path for importing modules
         sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
 
Signature · 1 line
1.8.1.2
#23 lore raw

PATCH 23/94 bitbake: webhob: create basic structure for static files

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch creates the basic structure for the front end files needed by Webhob. It sets up a new application called whbgui which will host all the static files we need.

---
 bitbake/lib/webhob/whbgui/__init__.py             |   0
 bitbake/lib/webhob/whbgui/static/images/yocto.jpg | Bin 0 -> 6582 bytes
 bitbake/lib/webhob/whbgui/templates/index.html    |  13 +++++++++++++
 bitbake/lib/webhob/whbgui/tests.py                |  16 ++++++++++++++++
 bitbake/lib/webhob/whbgui/urls.py                 |   9 +++++++++
 bitbake/lib/webhob/whbgui/views.py                |   8 ++++++++
 bitbake/lib/webhob/whbmain/settings.py            |   3 ++-
 bitbake/lib/webhob/whbmain/urls.py                |   1 +
 8 files changed, 49 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/webhob/whbgui/__init__.py
 create mode 100644 bitbake/lib/webhob/whbgui/static/images/yocto.jpg
 create mode 100644 bitbake/lib/webhob/whbgui/templates/index.html
 create mode 100644 bitbake/lib/webhob/whbgui/tests.py
 create mode 100644 bitbake/lib/webhob/whbgui/urls.py
 create mode 100644 bitbake/lib/webhob/whbgui/views.py
bitbake/lib/webhob/whbgui/__init__.py new file +0 -0
new file mode 100644
index 0000000..e69de29
bitbake/lib/webhob/whbgui/static/images/yocto.jpg new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..5de4b7693aa330bb8220baa81567bf82c307a8ab
GIT binary patch
bitbake/lib/webhob/whbgui/templates/index.html new file +13 -0
new file mode 100644
index 0000000..80b02d9
@@ -0,0 +1,13 @@
+<html>
+<head>
+	<title>GUI Page</title>
+</head>
+<body>
+{% load staticfiles %}
+
+<img src="{% static "/static/images/yocto.jpg" %}" alt="Yocto"/>
+
+This is your basic index page!
+</body>
+
+</html>
\ No newline at end of file
bitbake/lib/webhob/whbgui/tests.py new file +16 -0
new file mode 100644
index 0000000..501deb7
@@ -0,0 +1,16 @@
+"""
+This file demonstrates writing tests using the unittest module. These will pass
+when you run "manage.py test".
+
+Replace this with more appropriate tests for your application.
+"""
+
+from django.test import TestCase
+
+
+class SimpleTest(TestCase):
+    def test_basic_addition(self):
+        """
+        Tests that 1 + 1 always equals 2.
+        """
+        self.assertEqual(1 + 1, 2)
bitbake/lib/webhob/whbgui/urls.py new file +9 -0
new file mode 100644
index 0000000..321955a
@@ -0,0 +1,9 @@
+import os
+
+from django.conf import settings
+from django.conf.urls import patterns, include, url
+
+
+urlpatterns = patterns('whbgui.views',
+        url(r'^$', 'guihome', name='guihome'),
+)
bitbake/lib/webhob/whbgui/views.py new file +8 -0
new file mode 100644
index 0000000..e53a776
@@ -0,0 +1,8 @@
+from django.shortcuts import render
+from orm.models import Build, Task
+
+
+def guihome(request):
+    template = 'index.html'
+
+    return render(request, template)
bitbake/lib/webhob/whbmain/settings.py +2 -1
index aa1d4fc..d470328 100644
@@ -128,7 +128,7 @@ INSTALLED_APPS = (
     #'django.contrib.sessions',
     #'django.contrib.sites',
     #'django.contrib.messages',
-    #'django.contrib.staticfiles',
+    'django.contrib.staticfiles',
     # Uncomment the next line to enable the admin:
     # 'django.contrib.admin',
     # Uncomment the next line to enable admin documentation:
@@ -136,6 +136,7 @@ INSTALLED_APPS = (
     'orm',
     'whbmain',
     'bldviewer',
+    'whbgui',
 )
 
 # A sample logging configuration. The only tangible logging
bitbake/lib/webhob/whbmain/urls.py +1 -0
index c40e7c7..af536aa 100644
@@ -6,6 +6,7 @@ from django.conf.urls import patterns, include, url
 
 urlpatterns = patterns('',
     url(r'^build/', include('bldviewer.urls')),
+    url(r'^gui/', include('whbgui.urls')),
     # Examples:
     # url(r'^webhob/', include('webhob.foo.urls')),
 
Signature · 1 line
1.8.1.2
#24 lore raw

PATCH 24/94 bitbake: dsi: event data change

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Changes in the BuildInfoHelper and DSI as to record task file names and task names coming through the bb.build.Task* events.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 22 +++++++++++++---------
 bitbake/lib/bb/ui/dsi.py             |  1 +
 2 files changed, 14 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +13 -9
index 820e0f8..e43aa8c 100644
@@ -116,7 +116,7 @@ class ORMWrapper(object):
         task_information['log_file'] = 'Not Available'
         task_information['work_directory'] = 'Not Available'
         task_information['script_type'] = 0
-        task_information['file_path'] = 'Not Available'
+        task_information['file_path'] = event.taskfile
         task_information['line_number'] = 0
         task_information['py_stack_trace'] = 'Not Available'
         task_information['disk_io'] = 0
@@ -132,7 +132,7 @@ class ORMWrapper(object):
 
     def create_task_object(self, task_information):
         self.task_order += 1
-        identifier = task_information['recipe'].name + task_information['task_name']
+        identifier = task_information['recipe'].file_path + task_information['task_name']
 
         task_object = Task.objects.get_or_create(
                                 build=task_information['build'],
@@ -173,14 +173,16 @@ class ORMWrapper(object):
 
         if isinstance(event, bb.runqueue.runQueueTaskCompleted):
             task.outcome = 3
+            task.save()
+
+        if isinstance(event, bb.build.TaskBase):
+            task.recipe.name = event._package
+            task.recipe.save()
 
         task_build_stats = self._get_task_build_stats(task)
         task.cpu_usage = task_build_stats['cpu_usage']
         task.disk_io = task_build_stats['disk_io']
 
-        if outcome_info.get('error', ''):
-            task.error = outcome_info['error']
-
         #TODO: get error number
         #TODO: get warnings number
         #TODO: get warning information
@@ -264,7 +266,7 @@ class ORMWrapper(object):
         build_layer_obj = self._get_build_layer_object(event)
 
         recipe_info = {}
-        recipe_info['name'] = event.taskpackage
+        recipe_info['name'] = 'N/A'
         recipe_info['version'] = 'Not Available'
         recipe_info['layer'] = build_layer_obj
         recipe_info['summary'] = 'Not Available'
@@ -275,7 +277,8 @@ class ORMWrapper(object):
         recipe_info['homepage'] = 'Not Available'
         recipe_info['bugtracker'] = 'Not Available'
         recipe_info['author'] = 'Not Available'
-        recipe_info['file_path'] = 'Not Available'
+        recipe_info['file_path'] = event.taskfile
+
 
         return recipe_info
 
@@ -406,7 +409,8 @@ class BuildInfoHelper(object):
         task_obj = self.orm_wrapper.create_task_object(task_information)
 
     def update_and_store_task(self, event):
-        identifier = event.taskpackage + event.taskname
+        identifier = event.taskfile + event.taskname
         task_dictionary = self.orm_wrapper.transport_utils[identifier]
         self.orm_wrapper.update_task_object(task_dictionary, event)
-        del self.orm_wrapper.transport_utils[identifier]
+        if isinstance(event,bb.runqueue.runQueueTaskCompleted):
+            del self.orm_wrapper.transport_utils[identifier]
bitbake/lib/bb/ui/dsi.py +1 -0
index 90dd732..22b35e2 100644
@@ -288,6 +288,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 buildinfohelper.store_started_build(event)
 
             if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)):
+                buildinfohelper.update_and_store_task(event)
                 continue
 
             if isinstance(event, bb.event.LogExecTTY):
Signature · 1 line
1.8.1.2
#25 lore raw

PATCH 25/94 bitbake: webhob: fix and cleanup start script

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Make sure that the script can start without a bitbake.lock file present.
Fixes [YOCTO #4873].

Also, clean up whitespace.

---
 bitbake/bin/webhob | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
bitbake/bin/webhob +10 -8
index 8513a3a..44a25cb 100755
@@ -21,8 +21,8 @@
 # use as:  source webhob [start|stop]
 
 # Helper function to kill a background webhob development server
-        
-function webserverKillAllComponents() 
+
+function webserverKillAllComponents()
 {
 	local pidfile
 	for pidfile in ${BUILDDIR}/whbmain.pid; do
@@ -42,7 +42,7 @@ if [ -z "$ZSH_NAME" ] && [ `basename "$0"` = `basename $BASH_SOURCE` ]; then
     echo "Error: This script needs to be sourced. Please run as 'source webhob [start|stop]'" 1>&2;
     exit 1
 fi
-    
+
 if [ -z "$BUILDDIR" ] || [ -z `which bitbake` ]; then
     echo "Error: Build environment is not setup or bitbake is not in path." 1>&2;
     return 2
@@ -64,15 +64,17 @@ fi
 # Make sure it's safe to run by checking bitbake lock
 
 lock=1
-(flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0 
+if [ -e $BUILDDIR/bitbake.lock ]; then
+    (flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0
+fi
 
 if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); then
     echo "Error: bitbake lock state error. System may be already on." 2>&1
-    return 3 
+    return 3
 elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then
     echo "Error: bitbake lock state error. System may be already off.
 manually stop system with bitbake -m / webserverKillAllComponents" 2>&1
-    return 3 
+    return 3
 fi
 
 
@@ -82,7 +84,7 @@ case $CMD in
     start )
         unset BBSERVER
         python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0)
-        bitbake --server-only -t xmlrpc -B localhost:8200 
+        bitbake --server-only -t xmlrpc -B localhost:8200
         export BBSERVER=localhost:8200
         bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid
         python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
@@ -95,4 +97,4 @@ case $CMD in
         webserverKillAllComponents
 esac
 
-    
+
Signature · 1 line
1.8.1.2
#26 lore raw

PATCH 26/94 bitbake: dsi: refactor the BuildInfoHelper code

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

This is a in-depth refactoring of the BuildInfoHelper code designed to promote clean separation of code behinnd the data interfaces.

ORMWrapper now contains only database-related code. BuildInfoHelper keeps track of the internal state, and translated the Bitbake event-based data into dictionaries that the ORM layer can process.

DSI now sets featureSets to get a complete dependency dump.

The code uses the dependency information dump to create all the recipe and task entries before actual running the build, providing safe default.

The database models have been updated to allow for NULL values instead of creating generic placeholders.

---
 bitbake/lib/bb/ui/buildinfohelper.py             | 546 ++++++++++++-----------
 bitbake/lib/bb/ui/dsi.py                         |  14 +
 bitbake/lib/webhob/bldviewer/templates/task.html |   2 +-
 bitbake/lib/webhob/orm/models.py                 |  67 +--
 4 files changed, 333 insertions(+), 296 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +283 -263
index e43aa8c..07296d9 100644
@@ -2,12 +2,13 @@ import datetime
 import sys
 import uuid
 import bb
+import re
 
 
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer
+from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package, Task_Dependency, Package_Dependency
 
 class ORMWrapper(object):
     """ This class creates the dictionaries needed to store information in the database
@@ -15,21 +16,8 @@ class ORMWrapper(object):
         information in the database.
     """
 
-    def __init__(self, server):
-        self.uuid = None
-        self.task_order = 0
-        self.transport_utils = {}
-        self.server = server
-
-    def get_machine_information(self):
-        machine_info = {}
-
-        machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0]
-        machine_info['description'] = 'Not Available'
-
-        self.transport_utils['machine'] = machine_info['name']
-
-        return machine_info
+    def __init__(self):
+        pass
 
     def create_machine_object(self, machine_information):
 
@@ -38,29 +26,6 @@ class ORMWrapper(object):
 
         return machine[0]
 
-    def get_build_information(self, machine_obj):
-        build_info = {}
-
-        # Generate an identifier for each new build
-        self.uuid = str(uuid.uuid4())
-
-        build_info['uuid'] = self.uuid
-        build_info['target'] = 'Not Available'
-        build_info['machine'] = machine_obj
-        build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0]
-        build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
-        build_info['started_on'] = datetime.datetime.now()
-        build_info['completed_on'] = datetime.datetime.now()
-        build_info['outcome'] = 0
-        build_info['number_of_errors'] = 0
-        build_info['number_of_warnings'] = 0
-        build_info['image_fstypes'] = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0]
-        build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
-        build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0]
-        build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0]
-
-        return build_info
-
     def create_build_object(self, build_info):
 
         build = Build.objects.create(uuid=build_info['uuid'],
@@ -70,14 +35,12 @@ class ORMWrapper(object):
                                     distro_version=build_info['distro_version'],
                                     started_on=build_info['started_on'],
                                     completed_on=build_info['completed_on'],
-                                    outcome=build_info['outcome'],
-                                    errors_no=build_info['number_of_errors'],
-                                    warnings_no=build_info['number_of_warnings'],
                                     image_fstypes=build_info['image_fstypes'],
                                     cooker_log_path=build_info['cooker_log_path'],
                                     build_name=build_info['build_name'],
                                     bitbake_version=build_info['bitbake_version'])
 
+
         return build
 
     def update_build_object(self, build_obj, errors, warnings, taskfailures):
@@ -93,12 +56,178 @@ class ORMWrapper(object):
         build.outcome = outcome
         build.save()
 
-    def get_task_information(self, event):
 
-        recipe = self._get_recipe_object(event)
+    def get_update_task_object(self, task_information):
+        task_object, created = Task.objects.get_or_create(
+                                build=task_information['build'],
+                                recipe=task_information['recipe'],
+                                task_name=task_information['task_name'],
+                                )
+
+        for v in vars(task_object):
+            if v in task_information.keys():
+                vars(task_object)[v] = task_information[v]
+
+        if 'start_time' in task_information.keys():
+            duration = datetime.datetime.now() - task_information['start_time']
+            task_object.elapsed_time = duration.total_seconds()
+
+        task_object.save()
+        return task_object
+
+
+    def get_update_recipe_object(self, recipe_information):
+
+        recipe_object, created = Recipe.objects.get_or_create(
+                                         build_layer=recipe_information['build_layer'],
+                                         file_path=recipe_information['file_path'])
+
+        for v in vars(recipe_object):
+            if v in recipe_information.keys():
+                vars(recipe_object)[v] = recipe_information[v]
+
+        recipe_object.save()
+
+        return recipe_object
+
+    def get_build_layer_object(self, build_layer_information):
+
+        build_layer_object = Build_Layer.objects.get_or_create(
+                                    build = build_layer_information['build'],
+                                    layer = build_layer_information['layer'],
+                                    branch = build_layer_information['branch'],
+                                    commit = build_layer_information['commit'],
+                                    priority = build_layer_information['priority']
+                                    )
+
+        build_layer_object[0].save()
+
+        return build_layer_object[0]
+
+    def get_update_layer_object(self, layer_information):
+
+        layer_object = Layer.objects.get_or_create(
+                                name=layer_information['name'],
+                                local_path=layer_information['local_path'],
+                                layer_index_url=layer_information['layer_index_url'])
+        layer_object[0].save()
+
+        return layer_object[0]
+
+
+class BuildInfoHelper(object):
+    """ This class gathers the build information from the server and sends it
+        towards the ORM wrapper for storing in the database
+        It is instantiated once per build
+        Keeps in memory all data that needs matching before writing it to the database
+    """
+
+    def __init__(self, server):
+        self._configure_django()
+        self.internal_state = {}
+        self.uuid = None
+        self.task_order = 0
+        self.server = server
+        self.orm_wrapper = ORMWrapper()
+
+    def _configure_django(self):
+        # Add webhob to sys path for importing modules
+        sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
+
+    ###################
+    ## methods to convert event/external info into objects that the ORM layer uses
+
+    def _get_machine_information(self):
+        machine_info = {}
+        machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0]
+        machine_info['description'] = 'Not Available'
+        self.internal_state['machine'] = machine_info['name']
+        return machine_info
+
+    def _get_layer_dict(self, layer_path):
+
+        layer_info = {}
+        layer_name = layer_path.split('/')[-1]
+        layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/'
+        layer_url_name = self._get_url_map_name(layer_name)
+
+        layer_info['name'] = layer_name
+        layer_info['local_path'] = layer_path
+        layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name)
+
+        return layer_info
+
+    def _get_url_map_name(self, layer_name):
+        """ Some layers have a different name on openembedded.org site,
+            this method returns the correct name to use in the URL
+        """
+
+        url_name = layer_name
+        url_mapping = {'meta': 'openembedded-core'}
+
+        for key in url_mapping.keys():
+            if key == layer_name:
+                url_name = url_mapping[key]
+
+        return url_name
+
+    def _get_layer_information(self):
+
+        layer_info = {}
+
+        return layer_info
+
+    def _get_build_layer_information(self, layer_object):
+
+        build_layer_info = {}
+        build_layer_info['build'] = self.internal_state['build']
+        build_layer_info['layer'] = layer_object
+        build_layer_info['branch'] = self._get_git_branch(layer_object.local_path)
+        build_layer_info['commit'] = self._get_git_revision(layer_object.local_path)
+        build_layer_info['priority'] = 0
+
+        return build_layer_info
+
+
+    def _get_git_branch(self, layer_path):
+        branch = os.popen('cd {path}; git branch 2>&1 | grep "^* " | tr -d "* "'.format(path=layer_path)).read()
+
+        if len(branch) != 0:
+            return branch
+        return "<unknown>"
+
+    def _get_git_revision(self, layer_path):
+        f = os.popen("cd {path}; git log -n 1 --pretty=oneline -- 2>&1".format(path=layer_path))
+        data = f.read()
+        if f.close() is None:
+            rev = data.split(" ")[0]
+            if len(rev) != 0:
+                return rev
+        return "<unknown>"
+
+
+    def _get_build_information(self, machine_obj):
+        build_info = {}
+        # Generate an identifier for each new build
+
+        build_info['uuid'] = self.uuid
+        build_info['machine'] = machine_obj
+        build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0]
+        build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
+        build_info['started_on'] = datetime.datetime.now()
+        build_info['completed_on'] = datetime.datetime.now()
+        build_info['image_fstypes'] = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0]
+        build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
+        build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0]
+        build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0]
+
+        return build_info
+
+    def _get_task_information(self, event, recipe):
+
 
         task_information = {}
-        task_information['build'] = self.transport_utils['build']
+        task_information['build'] = self.internal_state['build']
         task_information['outcome'] = 4
         if isinstance(event, bb.runqueue.runQueueTaskSkipped):
             task_information['task_executed'] = False
@@ -108,86 +237,33 @@ class ORMWrapper(object):
                 task_information['outcome'] = 0
         else:
             task_information['task_executed'] = True
-        task_information['sstate_checksum'] = 'Not Available'
-        task_information['path_to_sstate_obj'] = 'Not Available'
         task_information['recipe'] = recipe
         task_information['task_name'] = event.taskname
-        task_information['source_url'] = 'Not Available'
-        task_information['log_file'] = 'Not Available'
-        task_information['work_directory'] = 'Not Available'
-        task_information['script_type'] = 0
-        task_information['file_path'] = event.taskfile
-        task_information['line_number'] = 0
-        task_information['py_stack_trace'] = 'Not Available'
-        task_information['disk_io'] = 0
-        task_information['cpu_usage'] = 0
-        task_information['elapsed_time'] = 'Not Available'
-        task_information['errors_no'] = 0
-        task_information['warnings_no'] = 0
-        task_information['error'] = 'Not Available'
-        task_information['warning'] = 'Not Available'
-        task_information['sstate_result'] = 0
-
         return task_information
 
-    def create_task_object(self, task_information):
-        self.task_order += 1
-        identifier = task_information['recipe'].file_path + task_information['task_name']
+    def _get_build_layer_for_path(self, path):
+        def _slkey(build_layer):
+            return len(build_layer.layer.local_path)
 
-        task_object = Task.objects.get_or_create(
-                                build=task_information['build'],
-                                order=self.task_order,
-                                task_executed=task_information['task_executed'],
-                                outcome=task_information['outcome'],
-                                sstate_checksum=task_information['sstate_checksum'],
-                                path_to_sstate_obj=task_information['path_to_sstate_obj'],
-                                recipe=task_information['recipe'],
-                                task_name=task_information['task_name'],
-                                source_url=task_information['source_url'],
-                                log_file=task_information['log_file'],
-                                work_directory=task_information['work_directory'],
-                                script_type=task_information['script_type'],
-                                file_path=task_information['file_path'],
-                                line_number=task_information['line_number'],
-                                py_stack_trace=task_information['py_stack_trace'],
-                                disk_io=task_information['disk_io'],
-                                cpu_usage=task_information['cpu_usage'],
-                                elapsed_time=task_information['elapsed_time'],
-                                errors_no=task_information['errors_no'],
-                                warnings_no=task_information['warnings_no'],
-                                error=task_information['error'],
-                                warning=task_information['warning'],
-                                sstate_result=task_information['sstate_result'])
-        task_object[0].save()
-
-        self.transport_utils[identifier] = {'object': task_object[0], 'start_time': datetime.datetime.now()}
-
-        return task_object[0]
-
-    def update_task_object(self, task_dictionary, event):
-
-        task = task_dictionary['object']
-
-        duration = datetime.datetime.now() - task_dictionary['start_time']
-        task.elapsed_time = duration.total_seconds()
+        # Heuristics: we always match recipe to the deepest layer path that
+        # we can match to the recipe file path
+        for bl in sorted(self.internal_state['build_layers'], reverse=True, key=_slkey):
+            if (path.startswith(bl.layer.local_path)):
+                return bl
 
-        if isinstance(event, bb.runqueue.runQueueTaskCompleted):
-            task.outcome = 3
-            task.save()
+        #TODO: if we get here, we didn't read layers correctly
+        assert False
+        return None
 
-        if isinstance(event, bb.build.TaskBase):
-            task.recipe.name = event._package
-            task.recipe.save()
+    def _get_recipe_information_from_build_event(self, event):
 
-        task_build_stats = self._get_task_build_stats(task)
-        task.cpu_usage = task_build_stats['cpu_usage']
-        task.disk_io = task_build_stats['disk_io']
+        build_layer_obj = self._get_build_layer_for_path(re.split(':', event.taskfile)[-1])
 
-        #TODO: get error number
-        #TODO: get warnings number
-        #TODO: get warning information
+        recipe_info = {}
+        recipe_info['build_layer'] = build_layer_obj
+        recipe_info['file_path'] = re.split(':', event.taskfile)[-1]
 
-        task.save()
+        return recipe_info
 
     def _get_task_build_stats(self, task_object):
         bs_path = self._get_path_information(task_object)
@@ -199,9 +275,9 @@ class ORMWrapper(object):
         build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/"
 
         tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
-        target = self.transport_utils['target']
-        machine = self.transport_utils['build'].machine.name
-        buildname = self.transport_utils['build'].build_name
+        target = self.internal_state['target']
+        machine = self.internal_state['build'].machine.name
+        buildname = self.internal_state['build'].build_name
         package = task_object.recipe.name
 
         build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target,
@@ -240,177 +316,121 @@ class ORMWrapper(object):
 
         return task_build_stats
 
-    def _get_recipe_object(self, event):
 
-        recipe_information = self._get_recipe_information(event)
+    ################################
+    ## external available methods to store information
 
-        recipe_object = Recipe.objects.get_or_create(
-                                name=recipe_information['name'],
-                                version=recipe_information['version'],
-                                layer=recipe_information['layer'],
-                                summary=recipe_information['summary'],
-                                description=recipe_information['description'],
-                                section=recipe_information['section'],
-                                license=recipe_information['license'],
-                                licensing_info=recipe_information['licensing_info'],
-                                homepage=recipe_information['homepage'],
-                                bugtracker=recipe_information['bugtracker'],
-                                author=recipe_information['author'],
-                                file_path=recipe_information['file_path'])
-        recipe_object[0].save()
-
-        return recipe_object[0]
-
-    def _get_recipe_information(self, event):
-
-        build_layer_obj = self._get_build_layer_object(event)
-
-        recipe_info = {}
-        recipe_info['name'] = 'N/A'
-        recipe_info['version'] = 'Not Available'
-        recipe_info['layer'] = build_layer_obj
-        recipe_info['summary'] = 'Not Available'
-        recipe_info['description'] = 'Not Available'
-        recipe_info['section'] = 'Not Available'
-        recipe_info['license'] = 'Not Available'
-        recipe_info['licensing_info'] = 'Not Available'
-        recipe_info['homepage'] = 'Not Available'
-        recipe_info['bugtracker'] = 'Not Available'
-        recipe_info['author'] = 'Not Available'
-        recipe_info['file_path'] = event.taskfile
-
-
-        return recipe_info
-
-    def _get_build_layer_object(self, event):
-
-        build_layer_information = self._get_build_layer_information(event)
-
-        build_layer_object = Build_Layer.objects.get_or_create(
-                                    build = build_layer_information['build'],
-                                    layer = build_layer_information['layer'],
-                                    branch = build_layer_information['branch'],
-                                    commit = build_layer_information['commit'],
-                                    priority = build_layer_information['priority'])
-        build_layer_object[0].save()
-
-        return build_layer_object[0]
-
-    def _get_build_layer_information(self, event):
-
-        layer_object = self._get_layer_object(event)
-
-        build_layer_info = {}
-        build_layer_info['build'] = self.transport_utils['build']
-        build_layer_info['layer'] = layer_object
-        build_layer_info['branch'] = 'Not Available'
-        build_layer_info['commit'] = 'Not Available'
-        build_layer_info['priority'] = 0
-
-        return build_layer_info
-
-    def _get_layer_object(self, event):
-
-        layer_information = self._get_layer_information(event)
-
-        layer_object = Layer.objects.get_or_create(
-                                name=layer_information['name'],
-                                local_path=layer_information['local_path'],
-                                layer_index_url=layer_information['layer_index_url'])
-        layer_object[0].save()
+    def store_layer_info(self):
+        layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ")
+        self.internal_state['layers'] = []
+        for layer_path in { l for l in layers if len(l) }:
+            layer_information = self._get_layer_dict(layer_path)
+            self.internal_state['layers'].append(self.orm_wrapper.get_update_layer_object(layer_information))
 
-        return layer_object[0]
+    def store_started_build(self, event):
 
-    def _get_layer_information(self, event):
+        machine_information = self._get_machine_information()
+        machine_obj = self.orm_wrapper.create_machine_object(machine_information)
+        self.uuid = str(uuid.uuid4())
 
-        layer_info = {}
-        layer_info['name'] = 'Layer Test Name'
-        layer_info['local_path'] = 'Not Available'
-        layer_info['layer_index_url'] = 'Not Available'
+        build_information = self._get_build_information(machine_obj)
+        build_information['target'] = ' '.join(event.getPkgs())
+        build_obj = self.orm_wrapper.create_build_object(build_information)
+        self.internal_state['build'] = build_obj
+        self.internal_state['target'] = build_information['target']
+        self.internal_state['build_layers'] = []
+        for layer_object in self.internal_state['layers']:
+            build_layer_information = self._get_build_layer_information(layer_object)
+            self.internal_state['build_layers'].append(self.orm_wrapper.get_build_layer_object(build_layer_information))
 
-        return layer_info
+        del self.internal_state['layers']
 
-    def _get_layer_dict(self, layer_path):
 
-        layer_info = {}
-        layer_name = layer_path.split('/')[-1]
-        layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/'
-        layer_url_name = self._get_url_map_name(layer_name)
-
-        layer_info['name'] = layer_name
-        layer_info['local_path'] = layer_path
-        layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name)
 
-        return layer_info
+    def update_build_information(self, event, errors, warnings, taskfailures):
+        self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures)
+        del self.internal_state['build']
 
-    def _get_url_map_name(self, layer_name):
-        """ Some layers have a different name on openembedded.org site,
-            this method returns the correct name to use in the URL
-        """
+    def store_started_task(self, event):
+        identifier = re.split(':', event.taskfile)[-1] + event.taskname
 
-        url_name = layer_name
-        url_mapping = {'meta': 'openembedded-core'}
+        recipe_information = self._get_recipe_information_from_build_event(event)
+        recipe = self.orm_wrapper.get_update_recipe_object(recipe_information)
 
-        for key in url_mapping.keys():
-            if key == layer_name:
-                url_name = url_mapping[key]
+        task_information = self._get_task_information(event, recipe)
 
-        return url_name
+        self.task_order += 1
+        task_information['order'] = self.task_order
+        task_obj = self.orm_wrapper.get_update_task_object(task_information)
 
-    def _save_layer_object(self, layer_path):
+        self.internal_state[identifier] = {'start_time': datetime.datetime.now()}
 
-        layer_information = self._get_layer_dict(layer_path)
+    def update_and_store_task(self, event):
+        identifier = re.split(':', event.taskfile)[-1] + event.taskname
+        recipe_information = self._get_recipe_information_from_build_event(event)
+        recipe = self.orm_wrapper.get_update_recipe_object(recipe_information)
+        task_information = self._get_task_information(event,recipe)
+        try:
+            task_information['start_time'] = self.internal_state[identifier]['start_time']
+        except:
+            pass
 
-        layer_object = Layer.objects.get_or_create(
-                            name=layer_information['name'],
-                            local_path=layer_information['local_path'],
-                            layer_index_url=layer_information['layer_index_url'])
+        if isinstance(event, bb.runqueue.runQueueTaskCompleted):
+            task_information['outcome'] = 3     # TODO: needs to use constants
+            task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
+            task_information['cpu_usage'] = task_build_stats['cpu_usage']
+            task_information['disk_io'] = task_build_stats['disk_io']
+            del self.internal_state[identifier]
 
-        layer_object[0].save()
+        #TODO: get error number
+        #TODO: get warnings number
+        #TODO: get warning information
 
-    def store_layer_info(self):
-        layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ")
-        for layer_path in layers:
-            self._save_layer_object(layer_path)
 
+        self.orm_wrapper.get_update_task_object(task_information)
 
-class BuildInfoHelper(object):
-    """ This class gathers the build information from the server and sends it
-        towards the ORM wrapper for storing in the database
-    """
 
-    def __init__(self, server):
-        self._configure_django()
-        self.orm_wrapper = ORMWrapper(server)
 
-    def _configure_django(self):
-        # Add webhob to sys path for importing modules
-        sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
+    def store_dependency_information(self, event):
+        self.internal_state['recipes'] = {}
+        for pn in event._depgraph['pn']:
 
-    def store_started_build(self, event):
+            file_name = re.split(':', event._depgraph['pn'][pn]['filename'])[-1]
+            build_layer_obj = self._get_build_layer_for_path(re.split(':', file_name)[-1])
 
-        self.orm_wrapper.store_layer_info()
+            assert build_layer_obj is not None
 
-        machine_information = self.orm_wrapper.get_machine_information()
-        machine_obj = self.orm_wrapper.create_machine_object(machine_information)
+            recipe_info = {}
+            recipe_info['name'] = pn
+            recipe_info['version'] = event._depgraph['pn'][pn]['version']
+            recipe_info['build_layer'] = build_layer_obj
 
-        build_information = self.orm_wrapper.get_build_information(machine_obj)
-        build_information['target'] = ' '.join(event.getPkgs())
-        build_obj = self.orm_wrapper.create_build_object(build_information)
-        self.orm_wrapper.transport_utils['build'] = build_obj
-        self.orm_wrapper.transport_utils['target'] = build_information['target']
+            # TODO: dump all this info in the deptree event
+            recipe_info['summary'] = 'Not Available'
+            recipe_info['description'] = 'Not Available'
+            recipe_info['section'] = 'Not Available'
+            recipe_info['license'] = 'Not Available'
+            recipe_info['licensing_info'] = 'Not Available'
+            recipe_info['homepage'] = 'Not Available'
+            recipe_info['bugtracker'] = 'Not Available'
+            recipe_info['author'] = 'Not Available'
+            recipe_info['file_path'] = file_name
+            recipe = self.orm_wrapper.get_update_recipe_object(recipe_info)
+            self.internal_state['recipes'][pn] = recipe
 
-    def update_build_information(self, event, errors, warnings, taskfailures):
-        self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures)
-        del self.orm_wrapper.transport_utils['build']
+        def _save_a_task(taskdesc):
+            pn, taskname = re.split(r'\.', taskdesc);
+            e = event
+            e.taskname = pn
+            recipe = self.internal_state['recipes'][pn]
+            task_info = self._get_task_information(e, recipe)
+            task_info['task_name'] = taskname
+            task_obj = self.orm_wrapper.get_update_task_object(task_info)
+            return task_obj
 
-    def store_started_task(self, event):
-        task_information = self.orm_wrapper.get_task_information(event)
-        task_obj = self.orm_wrapper.create_task_object(task_information)
+        for taskdesc in event._depgraph['tdepends']:
+            _save_a_task(taskdesc)
+            for taskdesc1 in event._depgraph['tdepends'][taskdesc]:
+                _save_a_task(taskdesc1)
 
-    def update_and_store_task(self, event):
-        identifier = event.taskfile + event.taskname
-        task_dictionary = self.orm_wrapper.transport_utils[identifier]
-        self.orm_wrapper.update_task_object(task_dictionary, event)
-        if isinstance(event,bb.runqueue.runQueueTaskCompleted):
-            del self.orm_wrapper.transport_utils[identifier]
+        del self.internal_state['recipes']
bitbake/lib/bb/ui/dsi.py +14 -0
index 22b35e2..d03a7c5 100644
@@ -17,6 +17,11 @@
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
 from __future__ import division
+try:
+    import bb
+except RuntimeError as exc:
+    sys.exit(str(exc))
+
 from bb.ui import uihelper
 from bb.ui.buildinfohelper import BuildInfoHelper
 
@@ -32,6 +37,7 @@ import sys
 import time
 import xmlrpclib
 
+featureSet = [bb.cooker.CookerFeatures.HOB_EXTRA_CACHES, bb.cooker.CookerFeatures.SEND_DEPENDS_TREE]
 
 logger = logging.getLogger("BitBake")
 interactive = sys.stdout.isatty()
@@ -268,7 +274,9 @@ def main(server, eventHandler, params, tf = TerminalFilter):
     errors = 0
     warnings = 0
     taskfailures = []
+
     buildinfohelper = BuildInfoHelper(server)
+    buildinfohelper.store_layer_info()
 
     termfilter = tf(main, helper, console, format)
 
@@ -464,8 +472,10 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit)):
                 buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
+
                 # we start a new build info
                 buildinfohelper = BuildInfoHelper(server)
+                buildinfohelper.store_layer_info()
                 continue
 
             # ignore
@@ -481,6 +491,10 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                                   bb.cooker.CookerExit)):
                 continue
 
+            if isinstance(event, bb.event.DepTreeGenerated):
+                buildinfohelper.store_dependency_information(event)
+                continue
+
             logger.error("Unknown event: %s", event)
 
         except EnvironmentError as ioerror:
bitbake/lib/webhob/bldviewer/templates/task.html +1 -1
index 26a09e9..7410489 100644
@@ -64,7 +64,7 @@
 				<td>{{task.path_to_sstate_obj}}</td>
 				<td>{{task.source_url}}</td>
 				<td>{{task.get_script_type_display}}</td>
-				<td>{{task.file_path}}</td>
+				<td>{{task.recipe.file_path}}</td>
 				<td>{{task.line_number}}</td>
 				<td>{{task.py_stack_trace}}</td>
 				<td>{{task.get_sstate_result_display}}</td>
bitbake/lib/webhob/orm/models.py +35 -32

95 lines not on this page · raw message

Signature · 1 line
1.8.1.2
#27 lore raw

PATCH 27/94 bitbake: webhob: record dependency info

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

This patch adds the task dependency information to the database, and displays it in a primitive form for verification.

---
 bitbake/lib/bb/ui/buildinfohelper.py             |  9 ++++---
 bitbake/lib/webhob/bldviewer/templates/task.html | 33 +++++++++---------------
 bitbake/lib/webhob/bldviewer/views.py            |  9 ++++++-
 3 files changed, 26 insertions(+), 25 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +6 -3
index 07296d9..f5e3f25 100644
@@ -8,7 +8,9 @@ import re
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package, Task_Dependency, Package_Dependency
+from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package
+from webhob.orm.models import Task_Dependency
+
 
 class ORMWrapper(object):
     """ This class creates the dictionaries needed to store information in the database
@@ -429,8 +431,9 @@ class BuildInfoHelper(object):
             return task_obj
 
         for taskdesc in event._depgraph['tdepends']:
-            _save_a_task(taskdesc)
+            target = _save_a_task(taskdesc)
             for taskdesc1 in event._depgraph['tdepends'][taskdesc]:
-                _save_a_task(taskdesc1)
+                dep = _save_a_task(taskdesc1)
+                Task_Dependency.objects.get_or_create( task = target, depends_on = dep )
 
         del self.internal_state['recipes']
bitbake/lib/webhob/bldviewer/templates/task.html +12 -21
index 7410489..1797028 100644
@@ -17,57 +17,47 @@
 			<tr>
 			<th>Order</th>
 			<th>Task</th>
-			<th>Recipe</th>
 			<th>Recipe Version</th>
 			<th>Task Type</th>
 			<th>Outcome</th>
 			<th>Errors</th>
 			<th>Warnings</th>
 			<th>Time</th>
-			<th>Log</th>
-			<th>Work directory</th>
 			<th>CPU usage</th>
 			<th>Disk I/O</th>
-			<th>Sstate Checksum</th>
-			<th>Path to sstate object</th>
-			<th>Source URL</th>
 			<th>Script type</th>
 			<th>File path</th>
-			<th>Line number</th>
-			<th>Python Stack Trace</th>
-			<th>Sstate Result</th>
+            <th>Depends</th>
 			</tr>
 
 			{% for task in tasks %}
 
 				<tr>
 				<td>{{task.order}}</td>
-				<td>{{task.task_name}}</td>
-				<td>{{task.recipe.name}}</td>
+				<td><a name="{{task.recipe.name}}.{{task.task_name}}">
+				{{task.recipe.name}}.{{task.task_name}}</a></td>
 				<td>{{task.recipe.version}}</td>
 
 				{% if task.task_executed %}
-					<td>Executed</td>
+				<td>Executed</td>
 				{% else %}
-					<td>Prebuilt</td>
+				<td>Prebuilt</td>
 				{% endif %}
 
 				<td>{{task.get_outcome_display}}</td>
 				<td>{{task.errors_no}}</td>
 				<td>{{task.warnings_no}}</td>
 				<td>{{task.elapsed_time}}</td>
-				<td>{{task.log_file}}</td>
-				<td>{{task.work_directory}}</td>
 				<td>{{task.cpu_usage}}</td>
 				<td>{{task.disk_io}}</td>
-				<td>{{task.sstate_checksum}}</td>
-				<td>{{task.path_to_sstate_obj}}</td>
-				<td>{{task.source_url}}</td>
 				<td>{{task.get_script_type_display}}</td>
 				<td>{{task.recipe.file_path}}</td>
-				<td>{{task.line_number}}</td>
-				<td>{{task.py_stack_trace}}</td>
-				<td>{{task.get_sstate_result_display}}</td>
+                <td>
+                {% for tt in task.depends_on %}
+                    <a href="#{{tt.recipe.name}}.{{tt.task_name}}">
+                    {{tt.recipe.name}}.{{tt.task_name}}</a><br/>
+                {% endfor %}
+                </td>
 				</tr>
 
 			{% endfor %}
@@ -79,3 +69,4 @@
 </body>
 
 </html>
+
bitbake/lib/webhob/bldviewer/views.py +8 -1
index f40d396..eb901a4 100644
@@ -1,5 +1,5 @@
 from django.shortcuts import render
-from orm.models import Build, Task, Layer
+from orm.models import Build, Task, Layer, Task_Dependency
 
 
 def build(request):
@@ -14,6 +14,13 @@ def task(request, build_id):
     template = 'task.html'
 
     tasks = Task.objects.filter(build=build_id)
+    task_depends = Task_Dependency.objects.filter(task__in=tasks)
+
+    for t in tasks:
+        t.depends_on = []
+        for k in task_depends:
+            if t == k.task:
+                t.depends_on.append(k.depends_on)
 
     context = {'tasks': tasks}
 
Signature · 1 line
1.8.1.2
#28 lore raw

PATCH 28/94 bitbake: webhob: use defined constants for models

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We change the models to use constants to improve
the legibility of the code.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 13 +++++----
 bitbake/lib/webhob/orm/models.py     | 55 ++++++++++++++++++++++++------------
 2 files changed, 44 insertions(+), 24 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +7 -6
index f5e3f25..d489c87 100644
@@ -47,9 +47,9 @@ class ORMWrapper(object):
 
     def update_build_object(self, build_obj, errors, warnings, taskfailures):
 
-        outcome = 0
+        outcome = Build.SUCCEEDED
         if errors or taskfailures:
-            outcome = 1
+            outcome = Build.FAILED
 
         build = Build.objects.get(uuid=build_obj.uuid)
         build.completed_on = datetime.datetime.now()
@@ -230,13 +230,13 @@ class BuildInfoHelper(object):
 
         task_information = {}
         task_information['build'] = self.internal_state['build']
-        task_information['outcome'] = 4
+        task_information['outcome'] = Task.OUTCOME_NA
         if isinstance(event, bb.runqueue.runQueueTaskSkipped):
             task_information['task_executed'] = False
             if event._skip == "covered":
-                task_information['outcome'] = 2
+                task_information['outcome'] = Task.OUTCOME_COVERED
             if event._skip == "existing":
-                task_information['outcome'] = 0
+                task_information['outcome'] = Task.OUTCOME_EXISTING
         else:
             task_information['task_executed'] = True
         task_information['recipe'] = recipe
@@ -363,6 +363,7 @@ class BuildInfoHelper(object):
 
         self.task_order += 1
         task_information['order'] = self.task_order
+        task_information['outcome'] = Task.OUTCOME_NA
         task_obj = self.orm_wrapper.get_update_task_object(task_information)
 
         self.internal_state[identifier] = {'start_time': datetime.datetime.now()}
@@ -378,7 +379,7 @@ class BuildInfoHelper(object):
             pass
 
         if isinstance(event, bb.runqueue.runQueueTaskCompleted):
-            task_information['outcome'] = 3     # TODO: needs to use constants
+            task_information['outcome'] = Task.OUTCOME_SUCCESS     # TODO: needs to use constants
             task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
             task_information['cpu_usage'] = task_build_stats['cpu_usage']
             task_information['disk_io'] = task_build_stats['disk_io']
bitbake/lib/webhob/orm/models.py +37 -18

99 lines not on this page · raw message

Signature · 1 line
1.8.1.2
#29 lore raw

PATCH 29/94 bitbake: dsi: fix the reading of task event information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Fixing a problem where the event outcome wasn't processed. Moving all the event-type-dependent code at the same level in the class.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +14 -9
index d489c87..6eafea0 100644
@@ -231,14 +231,6 @@ class BuildInfoHelper(object):
         task_information = {}
         task_information['build'] = self.internal_state['build']
         task_information['outcome'] = Task.OUTCOME_NA
-        if isinstance(event, bb.runqueue.runQueueTaskSkipped):
-            task_information['task_executed'] = False
-            if event._skip == "covered":
-                task_information['outcome'] = Task.OUTCOME_COVERED
-            if event._skip == "existing":
-                task_information['outcome'] = Task.OUTCOME_EXISTING
-        else:
-            task_information['task_executed'] = True
         task_information['recipe'] = recipe
         task_information['task_name'] = event.taskname
         return task_information
@@ -360,10 +352,19 @@ class BuildInfoHelper(object):
         recipe = self.orm_wrapper.get_update_recipe_object(recipe_information)
 
         task_information = self._get_task_information(event, recipe)
+        task_information['outcome'] = Task.OUTCOME_NA
+
+        if isinstance(event, bb.runqueue.runQueueTaskSkipped):
+            task_information['task_executed'] = False
+            if event.reason == "covered":
+                task_information['outcome'] = Task.OUTCOME_COVERED
+            if event.reason == "existing":
+                task_information['outcome'] = Task.OUTCOME_EXISTING
+        else:
+            task_information['task_executed'] = True
 
         self.task_order += 1
         task_information['order'] = self.task_order
-        task_information['outcome'] = Task.OUTCOME_NA
         task_obj = self.orm_wrapper.get_update_task_object(task_information)
 
         self.internal_state[identifier] = {'start_time': datetime.datetime.now()}
@@ -385,6 +386,10 @@ class BuildInfoHelper(object):
             task_information['disk_io'] = task_build_stats['disk_io']
             del self.internal_state[identifier]
 
+        if isinstance(event, bb.runqueue.runQueueTaskFailed):
+            task_information['outcome'] = Task.OUTCOME_FAILED
+            del self.internal_state[identifier]
+
         #TODO: get error number
         #TODO: get warnings number
         #TODO: get warning information
Signature · 1 line
1.8.1.2
#30 lore raw

PATCH 30/94 bitbake: webhob: Setup API for build model

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

Adding REST API version 1.0 interface of WebHob.

Build call API is implemented in this patch
Documentation is here:
https://wiki.yoctoproject.org/wiki/Webhob_REST_API_Builds

---
 bitbake/lib/webhob/bldviewer/api.py   |  6 ++++
 bitbake/lib/webhob/bldviewer/views.py | 56 +++++++++++++++++++++++++++++++++++
 bitbake/lib/webhob/whbmain/urls.py    |  1 +
 3 files changed, 63 insertions(+)
 create mode 100644 bitbake/lib/webhob/bldviewer/api.py
bitbake/lib/webhob/bldviewer/api.py new file +6 -0
new file mode 100644
index 0000000..d945861
@@ -0,0 +1,6 @@
+from django.conf.urls import patterns, include, url
+
+
+urlpatterns = patterns('bldviewer.views',
+        url(r'^builds/$', 'builds', name='builds'),
+)
bitbake/lib/webhob/bldviewer/views.py +56 -0

59 lines not on this page · raw message

bitbake/lib/webhob/whbmain/urls.py +1 -0
index af536aa..14345f6 100644
@@ -6,6 +6,7 @@ from django.conf.urls import patterns, include, url
 
 urlpatterns = patterns('',
     url(r'^build/', include('bldviewer.urls')),
+    url(r'^api/1.0/', include('bldviewer.api')),
     url(r'^gui/', include('whbgui.urls')),
     # Examples:
     # url(r'^webhob/', include('webhob.foo.urls')),
Signature · 1 line
1.8.1.2
#31 lore raw

PATCH 31/94 bitbake: webhob: orm change to remove Target

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Removes the Target dependency as it is replaced
by a field in Build.

---
 bitbake/lib/webhob/orm/models.py | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)
bitbake/lib/webhob/orm/models.py +2 -11
index 83126d8..e6663a3 100644
@@ -14,6 +14,7 @@ class Build(models.Model):
 
     uuid = models.CharField(max_length=100, unique=True)
     target = models.CharField(max_length=100)
+    is_image = models.BooleanField()
     machine = models.ForeignKey('Machine', related_name='build_machine')
     distro = models.CharField(max_length=100)
     distro_version = models.CharField(max_length=100)
@@ -98,24 +99,14 @@ class Task_Dependency(models.Model):
     depends_on = models.ForeignKey(Task, related_name='task_dependencies_depends')
 
 
-class Target(models.Model):
-    build = models.ForeignKey(Build, related_name='target_build')
-    is_image = models.BooleanField()
-
-
 class Artifact(models.Model):
     build = models.ForeignKey(Build, related_name='artifact_build')
-    target = models.ForeignKey(Target, related_name='artifact_target')
     file_name = models.CharField(max_length=100)
     file_size = models.IntegerField()
 
 
-class Package_In_Image(models.Model):
-    package = models.ForeignKey('Package', related_name='package_in_image_package')
-    target = models.ForeignKey(Target, related_name='package_in_image_target')
-
-
 class Package(models.Model):
+    build = models.ForeignKey('Build', related_name='package_build')
     recipe = models.ForeignKey('Recipe', related_name='package_recipe')
     name = models.CharField(max_length=100)
     version = models.CharField(max_length=100)
Signature · 1 line
1.8.1.2
#32 lore raw

PATCH 32/94 bitbake: webhob: clear up ORM relations

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

After a bit of analysis, it's become clear that Recipes only depend on a specific Layer version, and not on the current build.

So I clear up the Build_Layer transforming it to Layer_Version and remove the Build dependency.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 61 ++++++++++++++++++------------------
 bitbake/lib/webhob/orm/models.py     |  7 ++---
 2 files changed, 33 insertions(+), 35 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +30 -31
index 6eafea0..b0b56aa 100644
@@ -8,7 +8,7 @@ import re
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package
+from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package
 from webhob.orm.models import Task_Dependency
 
 
@@ -81,7 +81,7 @@ class ORMWrapper(object):
     def get_update_recipe_object(self, recipe_information):
 
         recipe_object, created = Recipe.objects.get_or_create(
-                                         build_layer=recipe_information['build_layer'],
+                                         layer_version=recipe_information['layer_version'],
                                          file_path=recipe_information['file_path'])
 
         for v in vars(recipe_object):
@@ -92,19 +92,18 @@ class ORMWrapper(object):
 
         return recipe_object
 
-    def get_build_layer_object(self, build_layer_information):
+    def get_layer_version_object(self, layer_version_information):
 
-        build_layer_object = Build_Layer.objects.get_or_create(
-                                    build = build_layer_information['build'],
-                                    layer = build_layer_information['layer'],
-                                    branch = build_layer_information['branch'],
-                                    commit = build_layer_information['commit'],
-                                    priority = build_layer_information['priority']
+        layer_version_object = Layer_Version.objects.get_or_create(
+                                    layer = layer_version_information['layer'],
+                                    branch = layer_version_information['branch'],
+                                    commit = layer_version_information['commit'],
+                                    priority = layer_version_information['priority']
                                     )
 
-        build_layer_object[0].save()
+        layer_version_object[0].save()
 
-        return build_layer_object[0]
+        return layer_version_object[0]
 
     def get_update_layer_object(self, layer_information):
 
@@ -179,16 +178,16 @@ class BuildInfoHelper(object):
 
         return layer_info
 
-    def _get_build_layer_information(self, layer_object):
+    def _get_layer_version_information(self, layer_object):
 
-        build_layer_info = {}
-        build_layer_info['build'] = self.internal_state['build']
-        build_layer_info['layer'] = layer_object
-        build_layer_info['branch'] = self._get_git_branch(layer_object.local_path)
-        build_layer_info['commit'] = self._get_git_revision(layer_object.local_path)
-        build_layer_info['priority'] = 0
+        layer_version_info = {}
+        layer_version_info['build'] = self.internal_state['build']
+        layer_version_info['layer'] = layer_object
+        layer_version_info['branch'] = self._get_git_branch(layer_object.local_path)
+        layer_version_info['commit'] = self._get_git_revision(layer_object.local_path)
+        layer_version_info['priority'] = 0
 
-        return build_layer_info
+        return layer_version_info
 
 
     def _get_git_branch(self, layer_path):
@@ -235,13 +234,13 @@ class BuildInfoHelper(object):
         task_information['task_name'] = event.taskname
         return task_information
 
-    def _get_build_layer_for_path(self, path):
-        def _slkey(build_layer):
-            return len(build_layer.layer.local_path)
+    def _get_layer_version_for_path(self, path):
+        def _slkey(layer_version):
+            return len(layer_version.layer.local_path)
 
         # Heuristics: we always match recipe to the deepest layer path that
         # we can match to the recipe file path
-        for bl in sorted(self.internal_state['build_layers'], reverse=True, key=_slkey):
+        for bl in sorted(self.internal_state['layer_versions'], reverse=True, key=_slkey):
             if (path.startswith(bl.layer.local_path)):
                 return bl
 
@@ -251,10 +250,10 @@ class BuildInfoHelper(object):
 
     def _get_recipe_information_from_build_event(self, event):
 
-        build_layer_obj = self._get_build_layer_for_path(re.split(':', event.taskfile)[-1])
+        layer_version_obj = self._get_layer_version_for_path(re.split(':', event.taskfile)[-1])
 
         recipe_info = {}
-        recipe_info['build_layer'] = build_layer_obj
+        recipe_info['layer_version'] = layer_version_obj
         recipe_info['file_path'] = re.split(':', event.taskfile)[-1]
 
         return recipe_info
@@ -332,10 +331,10 @@ class BuildInfoHelper(object):
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.internal_state['build'] = build_obj
         self.internal_state['target'] = build_information['target']
-        self.internal_state['build_layers'] = []
+        self.internal_state['layer_versions'] = []
         for layer_object in self.internal_state['layers']:
-            build_layer_information = self._get_build_layer_information(layer_object)
-            self.internal_state['build_layers'].append(self.orm_wrapper.get_build_layer_object(build_layer_information))
+            layer_version_information = self._get_layer_version_information(layer_object)
+            self.internal_state['layer_versions'].append(self.orm_wrapper.get_layer_version_object(layer_version_information))
 
         del self.internal_state['layers']
 
@@ -404,14 +403,14 @@ class BuildInfoHelper(object):
         for pn in event._depgraph['pn']:
 
             file_name = re.split(':', event._depgraph['pn'][pn]['filename'])[-1]
-            build_layer_obj = self._get_build_layer_for_path(re.split(':', file_name)[-1])
+            layer_version_obj = self._get_layer_version_for_path(re.split(':', file_name)[-1])
 
-            assert build_layer_obj is not None
+            assert layer_version_obj is not None
 
             recipe_info = {}
             recipe_info['name'] = pn
             recipe_info['version'] = event._depgraph['pn'][pn]['version']
-            recipe_info['build_layer'] = build_layer_obj
+            recipe_info['layer_version'] = layer_version_obj
 
             # TODO: dump all this info in the deptree event
             recipe_info['summary'] = 'Not Available'
bitbake/lib/webhob/orm/models.py +3 -4
index e6663a3..91b352e 100644
@@ -127,7 +127,7 @@ class Filelist(models.Model):
 class Recipe(models.Model):
     name = models.CharField(max_length=100, null=True)
     version = models.CharField(max_length=100, null=True)
-    build_layer = models.ForeignKey('Build_Layer', related_name='recipe_build_layer')
+    layer_version = models.ForeignKey('Layer_Version', related_name='recipe_layer_version')
     summary = models.CharField(max_length=100, null=True)
     description = models.CharField(max_length=100, null=True)
     section = models.CharField(max_length=100, null=True)
@@ -150,9 +150,8 @@ class Layer(models.Model):
     layer_index_url = models.URLField()
 
 
-class Build_Layer(models.Model):
-    build = models.ForeignKey(Build, related_name='build_layer_build')
-    layer = models.ForeignKey(Layer, related_name='build_layer_layer')
+class Layer_Version(models.Model):
+    layer = models.ForeignKey(Layer, related_name='layer_version_layer')
     branch = models.CharField(max_length=50)
     commit = models.CharField(max_length=100)
     priority = models.IntegerField()
Signature · 1 line
1.8.1.2
#33 lore raw

PATCH 33/94 bitbake: dsi: save detailed recipe information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Code to save the recipe information coming in the dependency tree. Also marks recipes that generate images.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +11 -9
index b0b56aa..92f1621 100644
@@ -411,22 +411,24 @@ class BuildInfoHelper(object):
             recipe_info['name'] = pn
             recipe_info['version'] = event._depgraph['pn'][pn]['version']
             recipe_info['layer_version'] = layer_version_obj
-
-            # TODO: dump all this info in the deptree event
-            recipe_info['summary'] = 'Not Available'
-            recipe_info['description'] = 'Not Available'
-            recipe_info['section'] = 'Not Available'
-            recipe_info['license'] = 'Not Available'
+            recipe_info['summary'] = event._depgraph['pn'][pn]['summary']
+            recipe_info['license'] = event._depgraph['pn'][pn]['license']
+            recipe_info['description'] = event._depgraph['pn'][pn]['summary']
+            recipe_info['section'] = event._depgraph['pn'][pn]['section']
             recipe_info['licensing_info'] = 'Not Available'
-            recipe_info['homepage'] = 'Not Available'
-            recipe_info['bugtracker'] = 'Not Available'
+            recipe_info['homepage'] = event._depgraph['pn'][pn]['homepage']
+            recipe_info['bugtracker'] = event._depgraph['pn'][pn]['bugtracker']
             recipe_info['author'] = 'Not Available'
             recipe_info['file_path'] = file_name
             recipe = self.orm_wrapper.get_update_recipe_object(recipe_info)
+            recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits'])
             self.internal_state['recipes'][pn] = recipe
 
+        # save all task information
         def _save_a_task(taskdesc):
-            pn, taskname = re.split(r'\.', taskdesc);
+            spec = re.split(r'\.', taskdesc);
+            pn = ".".join(spec[0:-1])
+            taskname = spec[-1]
             e = event
             e.taskname = pn
             recipe = self.internal_state['recipes'][pn]
Signature · 1 line
1.8.1.2
#34 lore raw

PATCH 34/94 bitbake: webhob: determine if a build is an image

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding code to see if a build will generate a image based on the inheritance of image.bbclass in the recipe defining the build.

---
 bitbake/lib/bb/ui/buildinfohelper.py              | 4 ++++
 bitbake/lib/webhob/bldviewer/templates/build.html | 2 ++
 bitbake/lib/webhob/orm/models.py                  | 2 +-
 3 files changed, 7 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/buildinfohelper.py +4 -0
index 92f1621..16e69c5 100644
@@ -328,6 +328,7 @@ class BuildInfoHelper(object):
 
         build_information = self._get_build_information(machine_obj)
         build_information['target'] = ' '.join(event.getPkgs())
+
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.internal_state['build'] = build_obj
         self.internal_state['target'] = build_information['target']
@@ -422,6 +423,9 @@ class BuildInfoHelper(object):
             recipe_info['file_path'] = file_name
             recipe = self.orm_wrapper.get_update_recipe_object(recipe_info)
             recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits'])
+            if recipe.is_image and pn == self.internal_state['build'].target:
+                self.internal_state['build'].is_image = True
+                self.internal_state['build'].save()
             self.internal_state['recipes'][pn] = recipe
 
         # save all task information
bitbake/lib/webhob/bldviewer/templates/build.html +2 -0
index 1b694d6..cd257e0 100644
@@ -16,6 +16,7 @@
 			<th>Started On</th>
 			<th>Completed On</th>
 			<th>Target</th>
+	        <th>Image ?</th>
 			<th>Machine</th>
 			<th>Time</th>
 			<th>Errors</th>
@@ -33,6 +34,7 @@
 			<td>{{build.started_on}}</td>
 			<td>{{build.completed_on}}</td>
 			<td>{{build.target}}</td>
+	        <td>{{build.is_image}}</td>
 			<td>{{build.machine.name}}</td>
 			<td>{% time_difference build.started_on build.completed_on %}</td>
 			<td>{{build.errors_no}}</td>
bitbake/lib/webhob/orm/models.py +1 -1
index 91b352e..0d6cf15 100644
@@ -14,7 +14,7 @@ class Build(models.Model):
 
     uuid = models.CharField(max_length=100, unique=True)
     target = models.CharField(max_length=100)
-    is_image = models.BooleanField()
+    is_image = models.BooleanField(default = False)
     machine = models.ForeignKey('Machine', related_name='build_machine')
     distro = models.CharField(max_length=100)
     distro_version = models.CharField(max_length=100)
Signature · 1 line
1.8.1.2
#35 lore raw

PATCH 35/94 bitbake: webhob: clean up URL structure

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We clean up the URL structure.
* default redirect to GUI version
* basic HTML interface now available at /simple/
* clear-up simple/ paths towards object-type URLs

---
 bitbake/lib/webhob/bldviewer/urls.py | 6 ++++--
 bitbake/lib/webhob/whbmain/urls.py   | 5 ++++-
 2 files changed, 8 insertions(+), 3 deletions(-)
bitbake/lib/webhob/bldviewer/urls.py +4 -2
index 8d5dc6e..756b3fa 100644
@@ -1,8 +1,10 @@
 from django.conf.urls import patterns, include, url
+from django.views.generic.simple import redirect_to
 
 
 urlpatterns = patterns('bldviewer.views',
-        url(r'^$', 'build', name='build'),
+        url(r'^build/$', 'build', name='build'),
         url(r'^layers/$', 'layer', name='layer'),
-        url(r'^(?P<build_id>\d+)/$', 'task', name='task'),
+        url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
+        url(r'^$', redirect_to, {'url': 'build/'}),
 )
bitbake/lib/webhob/whbmain/urls.py +4 -1
index 14345f6..0e15eda 100644
@@ -1,13 +1,16 @@
 from django.conf.urls import patterns, include, url
+from django.views.generic.simple import redirect_to
+
 
 # Uncomment the next two lines to enable the admin:
 # from django.contrib import admin
 # admin.autodiscover()
 
 urlpatterns = patterns('',
-    url(r'^build/', include('bldviewer.urls')),
+    url(r'^simple/', include('bldviewer.urls')),
     url(r'^api/1.0/', include('bldviewer.api')),
     url(r'^gui/', include('whbgui.urls')),
+    url(r'^$', redirect_to, {'url': '/gui/'}),
     # Examples:
     # url(r'^webhob/', include('webhob.foo.urls')),
 
Signature · 1 line
1.8.1.2
#36 lore raw

PATCH 36/94 bitbake: webhob: add simple pages for Recipes

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We add simple pages to display database information about Layer versions and Recipes.

---
 bitbake/lib/webhob/bldviewer/templates/layer.html  |  8 ++++
 bitbake/lib/webhob/bldviewer/templates/recipe.html | 48 ++++++++++++++++++++++
 bitbake/lib/webhob/bldviewer/urls.py               |  4 +-
 bitbake/lib/webhob/bldviewer/views.py              | 15 ++++++-
 4 files changed, 72 insertions(+), 3 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/recipe.html
bitbake/lib/webhob/bldviewer/templates/layer.html +8 -0
index 905a781..24bd86b 100644
@@ -15,6 +15,7 @@
 			<th>Name</th>
 			<th>Local Path</th>
 			<th>Layer Index URL</th>
+            <th>Known Versions</th>
 		</tr>
 
 		{% for layer in layers %}
@@ -23,6 +24,13 @@
 			<td>{{layer.name}}</td>
 			<td>{{layer.local_path}}</td>
 			<td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td>
+            <td><table>
+            {% for lv in layer.versions %}
+                <tr><td>
+        <a href="/simple/layerversions/{{lv.id}}/recipes">{{lv.branch}}:{{lv.commit}} ({{lv.count}} recipes)</a>
+                </td></tr>
+            {% endfor %}
+            </table></td>
 		</tr>
 
 		{% endfor %}
bitbake/lib/webhob/bldviewer/templates/recipe.html new file +48 -0

48 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/urls.py +2 -2
index 756b3fa..b6c6796 100644
@@ -1,10 +1,10 @@
 from django.conf.urls import patterns, include, url
 from django.views.generic.simple import redirect_to
 
-
 urlpatterns = patterns('bldviewer.views',
         url(r'^build/$', 'build', name='build'),
-        url(r'^layers/$', 'layer', name='layer'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
+        url(r'^layer/$', 'layer', name='layer'),
+        url(r'^layerversions/(?P<layerversion_id>\d+)/recipes$', 'layer_versions_recipes', name='layer_versions_recipes'),
         url(r'^$', redirect_to, {'url': 'build/'}),
 )
bitbake/lib/webhob/bldviewer/views.py +14 -1
index 58d9444..de89b84 100644
@@ -1,5 +1,5 @@
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Task_Dependency
+from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe
 
 
 def build(request):
@@ -30,11 +30,24 @@ def layer(request):
     template = 'layer.html'
     layer_info = Layer.objects.all()
 
+    for li in layer_info:
+        li.versions = Layer_Version.objects.filter(layer = li)
+        for liv in li.versions:
+            liv.count = Recipe.objects.filter(layer_version__id = liv.id).count()
+
     context = {'layers': layer_info}
 
     return render(request, template, context)
 
 
+def layer_versions_recipes(request, layerversion_id):
+    template = 'recipe.html'
+    recipes = Recipe.objects.filter(layer_version__id = layerversion_id)
+    
+    context = {'recipes': recipes}
+    
+    return render(request, template, context)
+
 #### API
 
 import json
Signature · 1 line
1.8.1.2
#37 lore raw

PATCH 37/94 bitbake: webhob: add ordering capabilities to build api

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch adds the capability to order builds by a specific field of the build model.
The format used is 'field_name:order' and the default ordering is ascending.

---
 bitbake/lib/webhob/bldviewer/views.py | 9 +++++++++
 1 file changed, 9 insertions(+)
bitbake/lib/webhob/bldviewer/views.py +9 -0
index de89b84..71e95b4 100644
@@ -56,6 +56,7 @@ from django.http import HttpResponse
 
 
 def builds(request):
+    DESCENDING = 'desc'
     response_data = {}
 
     try:
@@ -68,6 +69,7 @@ def builds(request):
     except ValueError:
         offset = 0
 
+    ordering_string = request.GET.get('orderby', '')
     filter_string = request.GET.get('filter', '')
 
     if filter_string:
@@ -91,6 +93,13 @@ def builds(request):
     else:
         response_data['count'] = 0
 
+    if queryset and ordering_string:
+        column, order = ordering_string.split(':')
+        if order.lower() == DESCENDING:
+            queryset = queryset.order_by('-' + column)
+        else:
+            queryset = queryset.order_by(column)
+
     response_data['list'] = serializers.serialize('json', queryset)
 
     return HttpResponse(json.dumps(response_data), content_type='application/json')
Signature · 1 line
1.8.1.2
#38 lore raw

PATCH 38/94 bitbake: webhob: validate inputs for build api

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch validates the inputs for the build api by checking the following:
- have only one colon;
- have equal number of terms on both sides of the colon;
- left side terms must be part of the Django model fields.

---
 bitbake/lib/webhob/bldviewer/views.py | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)
bitbake/lib/webhob/bldviewer/views.py +30 -3
index 71e95b4..a71aebc 100644
@@ -52,7 +52,7 @@ def layer_versions_recipes(request, layerversion_id):
 
 import json
 from django.core import serializers
-from django.http import HttpResponse
+from django.http import HttpResponse, HttpResponseBadRequest
 
 
 def builds(request):
@@ -69,8 +69,13 @@ def builds(request):
     except ValueError:
         offset = 0
 
-    ordering_string = request.GET.get('orderby', '')
-    filter_string = request.GET.get('filter', '')
+    ordering_string, invalid = _validate_input(request.GET.get('orderby', ''))
+    if invalid:
+        return HttpResponseBadRequest()
+
+    filter_string, invalid = _validate_input(request.GET.get('filter', ''))
+    if invalid:
+        return HttpResponseBadRequest()
 
     if filter_string:
         filter_terms = _get_filtering_terms(filter_string)
@@ -111,3 +116,25 @@ def _get_filtering_terms(filter_string):
     values = search_terms[1].split(',')
 
     return dict(zip(keys, values))
+
+def _validate_input(input):
+    invalid = 0
+
+    if input:
+        input_list = input.split(":")
+
+        # Check we have only one colon
+        if len(input.split(":")) != 2:
+            invalid = 1
+
+        # Check we have an equal number of terms both sides of the colon
+        if len(input_list[0].split(',')) != len(input_list[1].split(',')):
+            invalid = 1
+
+        # Check we are looking for a valid field
+        valid_fields = Build._meta.get_all_field_names()
+        for field in input_list[0].split(','):
+            if field not in valid_fields:
+                invalid = 1
+
+    return input, invalid
Signature · 1 line
1.8.1.2
#39 lore raw

PATCH 39/94 bitbake: webhob: generic view for multiple models

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch creates a generic view that supports multiple models. It allows exploration of the fields of the models and it has validation of inputs given.

Also avoids code replication.

Conflicts:
bitbake/lib/webhob/bldviewer/views.py

---
 bitbake/lib/webhob/bldviewer/api.py   |  4 +++-
 bitbake/lib/webhob/bldviewer/views.py | 33 ++++++++++++++++++++++++---------
 2 files changed, 27 insertions(+), 10 deletions(-)
bitbake/lib/webhob/bldviewer/api.py +3 -1
index d945861..3b2d7d2 100644
@@ -2,5 +2,7 @@ from django.conf.urls import patterns, include, url
 
 
 urlpatterns = patterns('bldviewer.views',
-        url(r'^builds/$', 'builds', name='builds'),
+        url(r'^builds/$', 'model_explorer',  {'model_name':'build'}, name='builds'),
+        url(r'^tasks/$', 'model_explorer', {'model_name':'task'}, name='task'),
+        url(r'^packages/$', 'model_explorer', {'model_name':'package'}, name='package'),
 )
bitbake/lib/webhob/bldviewer/views.py +24 -9
index a71aebc..73f7ba7 100644
@@ -1,5 +1,5 @@
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe
+from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package
 
 
 def build(request):
@@ -55,9 +55,20 @@ from django.core import serializers
 from django.http import HttpResponse, HttpResponseBadRequest
 
 
-def builds(request):
+def model_explorer(request, model_name):
+
     DESCENDING = 'desc'
     response_data = {}
+    model_mapping = {
+        'build': Build,
+        'task': Task,
+        'package': Package,
+        }
+
+    if model_name not in model_mapping.keys():
+        return HttpResponseBadRequest()
+
+    model = model_mapping[model_name]
 
     try:
         limit = int(request.GET.get('limit', 0))
@@ -69,22 +80,24 @@ def builds(request):
     except ValueError:
         offset = 0
 
-    ordering_string, invalid = _validate_input(request.GET.get('orderby', ''))
+    ordering_string, invalid = _validate_input(request.GET.get('orderby', ''),
+                                               model)
     if invalid:
         return HttpResponseBadRequest()
 
-    filter_string, invalid = _validate_input(request.GET.get('filter', ''))
+    filter_string, invalid = _validate_input(request.GET.get('filter', ''),
+                                             model)
     if invalid:
         return HttpResponseBadRequest()
 
     if filter_string:
         filter_terms = _get_filtering_terms(filter_string)
         try:
-            queryset = Build.objects.filter(**filter_terms)
+            queryset = model.objects.filter(**filter_terms)
         except ValueError:
             queryset = []
     else:
-        queryset = Build.objects.all()
+        queryset = model.objects.all()
 
     if offset and limit:
         queryset = queryset[offset:(offset+limit)]
@@ -107,7 +120,8 @@ def builds(request):
 
     response_data['list'] = serializers.serialize('json', queryset)
 
-    return HttpResponse(json.dumps(response_data), content_type='application/json')
+    return HttpResponse(json.dumps(response_data),
+                        content_type='application/json')
 
 def _get_filtering_terms(filter_string):
 
@@ -117,7 +131,8 @@ def _get_filtering_terms(filter_string):
 
     return dict(zip(keys, values))
 
-def _validate_input(input):
+def _validate_input(input, model):
+
     invalid = 0
 
     if input:
@@ -132,7 +147,7 @@ def _validate_input(input):
             invalid = 1
 
         # Check we are looking for a valid field
-        valid_fields = Build._meta.get_all_field_names()
+        valid_fields = model._meta.get_all_field_names()
         for field in input_list[0].split(','):
             if field not in valid_fields:
                 invalid = 1
Signature · 1 line
1.8.1.2
#40 lore raw

PATCH 40/94 bitbake: webhob: enhance Simple browser navigation

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We make a couple of changes to allow easier navigation in the simple data display.

- all templates now inherit the common base.html
- simple navigation menu added on top of page
- there is a link between a task name in tasks page, and the recipe entry
- all tabs converted to 4-spaces

---
 bitbake/lib/webhob/bldviewer/templates/base.html   |  16 +++
 bitbake/lib/webhob/bldviewer/templates/build.html  | 103 +++++++++----------
 bitbake/lib/webhob/bldviewer/templates/layer.html  |  44 ++++----
 bitbake/lib/webhob/bldviewer/templates/recipe.html |  90 ++++++++--------
 bitbake/lib/webhob/bldviewer/templates/task.html   | 114 ++++++++++-----------
 bitbake/lib/webhob/bldviewer/urls.py               |   2 +-
 6 files changed, 180 insertions(+), 189 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/base.html
bitbake/lib/webhob/bldviewer/templates/base.html new file +16 -0
new file mode 100644
index 0000000..ab3c3b7
@@ -0,0 +1,16 @@
+<!DOCTYPE html>
+
+<html>
+	<head>
+		<title>WebHob Simple Explorer</title>
+	</head>
+
+<body>
+<div>Menu: <a href="/simple/build/">All Builds</a>&nbsp;|&nbsp;<a href="/simple/layer/">All Layers</a>
+</div>
+{% block pagecontent %}
+
+{% endblock %}
+</body>
+</html>
+
bitbake/lib/webhob/bldviewer/templates/build.html +49 -54

103 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/templates/layer.html +19 -25

60 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/templates/recipe.html +42 -48

90 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/templates/task.html +53 -61

125 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/urls.py +1 -1
index b6c6796..5f07161 100644
@@ -5,6 +5,6 @@ urlpatterns = patterns('bldviewer.views',
         url(r'^build/$', 'build', name='build'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
         url(r'^layer/$', 'layer', name='layer'),
-        url(r'^layerversions/(?P<layerversion_id>\d+)/recipes$', 'layer_versions_recipes', name='layer_versions_recipes'),
+        url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'),
         url(r'^$', redirect_to, {'url': 'build/'}),
 )
Signature · 1 line
1.8.1.2
#41 lore raw

PATCH 41/94 bitbake: webhob: improve startup script

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

A couple of modification targeted at making the
system start/stop script more resilient.

Also, it now checks for the correct Django version.

---
 bitbake/bin/webhob | 53 ++++++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 11 deletions(-)
bitbake/bin/webhob +42 -11
index 44a25cb..7b74465 100755
@@ -22,13 +22,13 @@
 
 # Helper function to kill a background webhob development server
 
-function webserverKillAllComponents()
+function webserverKillAll()
 {
 	local pidfile
 	for pidfile in ${BUILDDIR}/whbmain.pid; do
 		if [ -f ${pidfile} ]; then
 		while kill -0 $(< ${pidfile}) 2>/dev/null; do
-			kill -SIGTERM -$(< ${pidfile})
+			kill -SIGTERM -$(< ${pidfile}) 2>/dev/null
 			sleep 1;
 		done;
 		rm  ${pidfile}
@@ -36,6 +36,15 @@ function webserverKillAllComponents()
 	done
 }
 
+
+function webserverStartAll()
+{
+        python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 2)
+        python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
+        return 0
+}
+
+
 # We make sure we're running in the current shell and in a good environment
 
 if [ -z "$ZSH_NAME" ] && [ `basename "$0"` = `basename $BASH_SOURCE` ]; then
@@ -50,6 +59,16 @@ fi
 
 BBBASEDIR=`dirname ${BASH_SOURCE}`/..
 
+
+# Verify prerequisites
+
+if ! echo "import django; print (1,4,5) == django.VERSION[0:3]" | python 2>/dev/null | grep True >/dev/null; then
+    echo -e "This program needs Django 1.4.5. Please install with\n\nsudo pip install django==1.4.5"
+    return 2
+fi
+
+
+
 # Determine the action. If specified by arguments, fine, if not, toggle it
 if [ "x$1" == "xstart" ] || [ "x$1" == "xstop" ]; then
     CMD="$1"
@@ -61,6 +80,13 @@ else
     fi;
 fi
 
+NODSI=0
+if [ "x$2" == "xnodsi" ]; then
+    NODSI=1
+fi
+
+echo "The system will $CMD."
+
 # Make sure it's safe to run by checking bitbake lock
 
 lock=1
@@ -69,11 +95,11 @@ if [ -e $BUILDDIR/bitbake.lock ]; then
 fi
 
 if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); then
-    echo "Error: bitbake lock state error. System may be already on." 2>&1
+    echo "Error: bitbake lock state error. System is already on." 2>&1
     return 3
 elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then
-    echo "Error: bitbake lock state error. System may be already off.
-manually stop system with bitbake -m / webserverKillAllComponents" 2>&1
+    echo "Error: bitbake lock state error. Trying to stop a stopped system ?
+manually stop system with bitbake -m / webserverKill" 2>&1
     return 3
 fi
 
@@ -82,19 +108,24 @@ fi
 
 case $CMD in
     start )
+        webserverStartAll || return 4
         unset BBSERVER
-        python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0)
         bitbake --server-only -t xmlrpc -B localhost:8200
         export BBSERVER=localhost:8200
-        bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid
-        python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
+        if [ $NODSI == 0 ]; then        # we start the DSI only if not inhibited
+            bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid
+        fi
     ;;
     stop )
-        kill $(< ${BUILDDIR}/dsi.pid )
-        rm ${BUILDDIR}/dsi.pid
+        if [ -f ${BUILDDIR}/dsi.pid ]; then
+            kill $(< ${BUILDDIR}/dsi.pid )
+            rm ${BUILDDIR}/dsi.pid
+        fi
         bitbake -m
         unset BBSERVER
-        webserverKillAllComponents
+        webserverKillAll
+    ;;
 esac
 
+echo "Successful ${CMD}."
 
Signature · 1 line
1.8.1.2
#42 lore raw

PATCH 42/94 bitbake: webhob: improve validation code flow

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch improves the code flow in the validation method by not repeating the split method over the same list and by stoping at any invalid input encountered.

---
 bitbake/lib/webhob/bldviewer/views.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
bitbake/lib/webhob/bldviewer/views.py +4 -1
index 73f7ba7..647a1f2 100644
@@ -139,17 +139,20 @@ def _validate_input(input, model):
         input_list = input.split(":")
 
         # Check we have only one colon
-        if len(input.split(":")) != 2:
+        if len(input_list) != 2:
             invalid = 1
+            return None, invalid
 
         # Check we have an equal number of terms both sides of the colon
         if len(input_list[0].split(',')) != len(input_list[1].split(',')):
             invalid = 1
+            return None, invalid
 
         # Check we are looking for a valid field
         valid_fields = model._meta.get_all_field_names()
         for field in input_list[0].split(','):
             if field not in valid_fields:
                 invalid = 1
+                return None, invalid
 
     return input, invalid
Signature · 1 line
1.8.1.2
#43 lore raw

PATCH 43/94 bitbake: webhob: add more models to webhob API

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch enables webhob API to support recipe, layer and layer_version models.

---
 bitbake/lib/webhob/bldviewer/api.py   | 3 +++
 bitbake/lib/webhob/bldviewer/views.py | 7 +++++--
 2 files changed, 8 insertions(+), 2 deletions(-)
bitbake/lib/webhob/bldviewer/api.py +3 -0
index 3b2d7d2..9fa701c 100644
@@ -5,4 +5,7 @@ urlpatterns = patterns('bldviewer.views',
         url(r'^builds/$', 'model_explorer',  {'model_name':'build'}, name='builds'),
         url(r'^tasks/$', 'model_explorer', {'model_name':'task'}, name='task'),
         url(r'^packages/$', 'model_explorer', {'model_name':'package'}, name='package'),
+        url(r'^layers/$', 'model_explorer', {'model_name':'layer'}, name='layer'),
+        url(r'^recipes/$', 'model_explorer', {'model_name':'recipe'}, name='recipe'),
+        url(r'^layersversions/$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'),
 )
bitbake/lib/webhob/bldviewer/views.py +5 -2
index 647a1f2..3ef6940 100644
@@ -43,9 +43,9 @@ def layer(request):
 def layer_versions_recipes(request, layerversion_id):
     template = 'recipe.html'
     recipes = Recipe.objects.filter(layer_version__id = layerversion_id)
-    
+
     context = {'recipes': recipes}
-    
+
     return render(request, template, context)
 
 #### API
@@ -63,6 +63,9 @@ def model_explorer(request, model_name):
         'build': Build,
         'task': Task,
         'package': Package,
+        'layer': Layer,
+        'layerversion': Layer_Version,
+        'recipe': Recipe,
         }
 
     if model_name not in model_mapping.keys():
Signature · 1 line
1.8.1.2
#44 lore raw

PATCH 44/94 bitbake: webhob: add search for build model

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch enables search on the build model. However, the search can be executed only on a predefined list of fields allowed to be searched.

---
 bitbake/lib/webhob/bldviewer/views.py | 17 +++++++++++++++++
 bitbake/lib/webhob/orm/models.py      |  3 +++
 2 files changed, 20 insertions(+)
bitbake/lib/webhob/bldviewer/views.py +17 -0
index 3ef6940..423fdf7 100644
@@ -1,3 +1,6 @@
+import operator
+
+from django.db.models import Q
 from django.shortcuts import render
 from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package
 
@@ -93,6 +96,8 @@ def model_explorer(request, model_name):
     if invalid:
         return HttpResponseBadRequest()
 
+    search_term = request.GET.get('search', '')
+
     if filter_string:
         filter_terms = _get_filtering_terms(filter_string)
         try:
@@ -102,6 +107,9 @@ def model_explorer(request, model_name):
     else:
         queryset = model.objects.all()
 
+    if search_term:
+        queryset = _get_search_results(search_term, queryset, model)
+
     if offset and limit:
         queryset = queryset[offset:(offset+limit)]
     elif offset:
@@ -159,3 +167,12 @@ def _validate_input(input, model):
                 return None, invalid
 
     return input, invalid
+
+def _get_search_results(search_term, queryset, model):
+    q_map = map(lambda x: Q(**{x+'__icontains': search_term}),
+                model.search_allowed_fields)
+
+    search_object = reduce(operator.or_, q_map)
+    queryset = queryset.filter(search_object)
+
+    return queryset
bitbake/lib/webhob/orm/models.py +3 -0
index 0d6cf15..a4721db 100644
@@ -12,6 +12,9 @@ class Build(models.Model):
         (IN_PROGRESS, 'In Progress'),
     )
 
+    search_allowed_fields = ['target', 'machine__name',
+                             'cooker_log_path', 'image_fstypes']
+
     uuid = models.CharField(max_length=100, unique=True)
     target = models.CharField(max_length=100)
     is_image = models.BooleanField(default = False)
Signature · 1 line
1.8.1.2
#45 lore raw

PATCH 45/94 bitbake: webhob: fix ordering issue

In reply to Alex DAMIAN

From: Calin Dragomir <[email protected]>

This patch fixes the AssertionError issue that caused the build page to break.
This happened because ordering cannot be done after a slicing operation has been executed.
By placing the ordering before the slicing occurs, the issue stops appearing.

Conflicts:
bitbake/lib/webhob/bldviewer/views.py

---
 bitbake/lib/webhob/bldviewer/views.py | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
bitbake/lib/webhob/bldviewer/views.py +7 -7
index 423fdf7..a5c4387 100644
@@ -110,6 +110,13 @@ def model_explorer(request, model_name):
     if search_term:
         queryset = _get_search_results(search_term, queryset, model)
 
+    if ordering_string and queryset:
+        column, order = ordering_string.split(':')
+        if order.lower() == DESCENDING:
+            queryset = queryset.order_by('-' + column)
+        else:
+            queryset = queryset.order_by(column)
+
     if offset and limit:
         queryset = queryset[offset:(offset+limit)]
     elif offset:
@@ -122,13 +129,6 @@ def model_explorer(request, model_name):
     else:
         response_data['count'] = 0
 
-    if queryset and ordering_string:
-        column, order = ordering_string.split(':')
-        if order.lower() == DESCENDING:
-            queryset = queryset.order_by('-' + column)
-        else:
-            queryset = queryset.order_by(column)
-
     response_data['list'] = serializers.serialize('json', queryset)
 
     return HttpResponse(json.dumps(response_data),
Signature · 1 line
1.8.1.2
#46 lore raw

PATCH 46/94 bitbake: webhob: extend search for multiple terms

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We extend the search functionality as to be able to narrow down search results based on multiple terms.

---
 bitbake/lib/webhob/bldviewer/views.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
bitbake/lib/webhob/bldviewer/views.py +5 -2
index a5c4387..4f7b46c 100644
@@ -169,10 +169,13 @@ def _validate_input(input, model):
     return input, invalid
 
 def _get_search_results(search_term, queryset, model):
-    q_map = map(lambda x: Q(**{x+'__icontains': search_term}),
+    search_objects = []
+    for st in search_term.split(" "):
+        q_map = map(lambda x: Q(**{x+'__icontains': st}),
                 model.search_allowed_fields)
 
-    search_object = reduce(operator.or_, q_map)
+        search_objects.append(reduce(operator.or_, q_map))
+    search_object = reduce(operator.and_, search_objects)
     queryset = queryset.filter(search_object)
 
     return queryset
Signature · 1 line
1.8.1.2
#47 lore raw

PATCH 47/94 bitbake: webhob: force bitbake server stop

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We force a bitbake server stop by killing it on the stop command.

This will have no effect on well behaving servers since they will be stopped anyway by the bitbake -m command above

---
 bitbake/bin/webhob | 2 ++
 1 file changed, 2 insertions(+)
bitbake/bin/webhob +2 -0
index 7b74465..099f2d3 100755
@@ -124,6 +124,8 @@ case $CMD in
         bitbake -m
         unset BBSERVER
         webserverKillAll
+        # force stop any misbehaving bitbake server
+        lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 kill
     ;;
 esac
 
Signature · 1 line
1.8.1.2
#48 lore raw

PATCH 48/94 bitbake: webhob: simple interface dependency list

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We add a diff to obtain a consistent layout of the dependency field information for tasks.

---
 bitbake/lib/webhob/bldviewer/templates/task.html | 2 ++
 1 file changed, 2 insertions(+)
bitbake/lib/webhob/bldviewer/templates/task.html +2 -0
index 88c82d8..2d78668 100644
@@ -48,10 +48,12 @@
                 <td>{{task.get_script_type_display}}</td>
                 <td>{{task.recipe.file_path}}</td>
                 <td>
+            <div style="height: 3em; overflow:auto">
                 {% for tt in task.depends_on %}
                     <a href="#{{tt.recipe.name}}.{{tt.task_name}}">
                     {{tt.recipe.name}}.{{tt.task_name}}</a><br/>
                 {% endfor %}
+            </div>
                 </td>
                 </tr>
 
Signature · 1 line
1.8.1.2
#49 lore raw

PATCH 49/94 bitbake: dsi: add feature to store package information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

I'm adding code to read package information and package dependency off the buildhistory, if the buildhistory feature is enabled, and the target is an image.

There are small changes to the events trapped since we need to capture package information after the build is completed.

Minor changes to internal state handling of BuildInfoHelper.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 75 ++++++++++++++++++++++++++++++++----
 bitbake/lib/bb/ui/dsi.py             | 30 +++++++--------
 bitbake/lib/webhob/orm/models.py     |  4 +-
 3 files changed, 85 insertions(+), 24 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +68 -7
index 16e69c5..59b6fa3 100644
@@ -8,7 +8,7 @@ import re
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package
+from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, Package_Dependency
 from webhob.orm.models import Task_Dependency
 
 
@@ -42,16 +42,14 @@ class ORMWrapper(object):
                                     build_name=build_info['build_name'],
                                     bitbake_version=build_info['bitbake_version'])
 
-
         return build
 
-    def update_build_object(self, build_obj, errors, warnings, taskfailures):
+    def update_build_object(self, build, errors, warnings, taskfailures):
 
         outcome = Build.SUCCEEDED
         if errors or taskfailures:
             outcome = Build.FAILED
 
-        build = Build.objects.get(uuid=build_obj.uuid)
         build.completed_on = datetime.datetime.now()
         build.errors_no = errors
         build.warnings_no = warnings
@@ -116,6 +114,22 @@ class ORMWrapper(object):
         return layer_object[0]
 
 
+    def save_package_information(self, build_obj, packagedict, bldpkgs, recipes):
+        for p in packagedict:
+            packagedict[p]['object'] = Package.objects.create( build = build_obj,
+                                        name = p,
+                                        size = packagedict[p]['size'])
+            if p in bldpkgs:
+                packagedict[p]['object'].version = bldpkgs[p]['version']
+                packagedict[p]['object'].recipe =  recipes[bldpkgs[p]['pn']]
+                packagedict[p]['object'].save()
+
+        for p in packagedict:
+            for px in packagedict[p]['depends']:
+                Package_Dependency.objects.create( package = packagedict[p]['object'],
+                                        depends_on = packagedict[px]['object'] );
+
+
 class BuildInfoHelper(object):
     """ This class gathers the build information from the server and sends it
         towards the ORM wrapper for storing in the database
@@ -123,13 +137,14 @@ class BuildInfoHelper(object):
         Keeps in memory all data that needs matching before writing it to the database
     """
 
-    def __init__(self, server):
+    def __init__(self, server, has_build_history = False):
         self._configure_django()
         self.internal_state = {}
         self.uuid = None
         self.task_order = 0
         self.server = server
         self.orm_wrapper = ORMWrapper()
+        self.has_build_history = has_build_history
 
     def _configure_django(self):
         # Add webhob to sys path for importing modules
@@ -343,7 +358,6 @@ class BuildInfoHelper(object):
 
     def update_build_information(self, event, errors, warnings, taskfailures):
         self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures)
-        del self.internal_state['build']
 
     def store_started_task(self, event):
         identifier = re.split(':', event.taskfile)[-1] + event.taskname
@@ -398,8 +412,56 @@ class BuildInfoHelper(object):
         self.orm_wrapper.get_update_task_object(task_information)
 
 
+    def read_package_dep_data(self, event):
+        # verify that we have something to read
+        if not self.internal_state['build'].is_image or not self.has_build_history:
+            print "not collecting package info ", self.internal_state['build'].is_image, self.has_build_history
+            return
+
+        # fair warning: code correlates to buildhistory.bbclass; anything changes there, needs to chage here too
+        TOPDIR, error = self.server.runCommand(['getVariable', 'TOPDIR'])
+        MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH'])
+        TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC'])
+        MULTIMACH_TARGET_SYS, error = self.server.runCommand(['getVariable', 'MULTIMACH_TARGET_SYS'])
+        SDK_NAME, error = self.server.runCommand(['getVariable', 'SDK_NAME'])
+        BUILDHISTORY_DIR = "%s/buildhistory" % TOPDIR
+        BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, self.internal_state['build'].target)
+
+        self.internal_state['packages'] = {}
+
+        with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
+            for line in fin:
+                line = line.rstrip(";")
+                psize, px = line.split("\t")
+                punit, pname = px.split(" ")
+                self.internal_state['packages'][pname.strip()] = {'size':psize, 'depends' : []}
+
+        with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
+            p = re.compile(r' -> ')
+            for line in fin:
+                line = line.rstrip(';')
+                linesplit = p.split(line)
+                if len(linesplit) == 2:
+                    pname = linesplit[0]
+                    dependsname = linesplit[1].split(" ")[0].strip().strip(";")
+                    if not pname in self.internal_state['packages']:
+                        self.internal_state['packages'][pname] = {'size': 0, 'depends' : []}
+                    if not dependsname in self.internal_state['packages']:
+                        self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []}
+                    self.internal_state['packages'][pname]['depends'].append(dependsname)
+
+        self.orm_wrapper.save_package_information(self.internal_state['build'], self.internal_state['packages'],
+                    self.internal_state['bldpkgs'], self.internal_state['recipes'])
+
 
     def store_dependency_information(self, event):
+
+        # save build time package information
+        self.internal_state['bldpkgs'] = {}
+        for pkg  in event._depgraph['packages']:
+            self.internal_state['bldpkgs'][pkg] = event._depgraph['packages'][pkg]
+
+        # save recipe information
         self.internal_state['recipes'] = {}
         for pn in event._depgraph['pn']:
 
@@ -447,4 +509,3 @@ class BuildInfoHelper(object):
                 dep = _save_a_task(taskdesc1)
                 Task_Dependency.objects.get_or_create( task = target, depends_on = dep )
 
-        del self.internal_state['recipes']
bitbake/lib/bb/ui/dsi.py +15 -15
index d03a7c5..98d225b 100644
@@ -243,6 +243,14 @@ def main(server, eventHandler, params, tf = TerminalFilter):
 
     includelogs, loglines, consolelogfile = _log_settings_from_server(server)
 
+
+    # verify and warn
+    build_history_enabled = True
+    inheritlist, error = server.runCommand(["getVariable", "INHERIT"])
+    if not "buildhistory" in inheritlist.split(" "):
+        logger.warn("buildhistory is not enabled. Please enable INHERIT += \"buildhistory\" to see image details.")
+        build_history_enabled = False
+
     if sys.stdin.isatty() and sys.stdout.isatty():
         log_exec_tty = True
     else:
@@ -275,7 +283,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
     warnings = 0
     taskfailures = []
 
-    buildinfohelper = BuildInfoHelper(server)
+    buildinfohelper = BuildInfoHelper(server, build_history_enabled)
     buildinfohelper.store_layer_info()
 
     termfilter = tf(main, helper, console, format)
@@ -453,28 +461,20 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 # timestamp should be added for this
                 continue
 
-            if isinstance(event, bb.event.OperationStarted):
-                # timestamp should be added for this
+            if isinstance(event, (bb.event.TreeDataPreparationStarted, bb.event.TreeDataPreparationCompleted)):
                 continue
 
-            if isinstance(event, bb.event.OperationCompleted):
-                # timestamp should be added for this
-                # signal a complete operation
-                # calculate timing
-                continue
-
-            if isinstance(event, bb.event.DiskFull):
-                # trigger an error
+            if isinstance(event, (bb.event.BuildCompleted)):
+                buildinfohelper.read_package_dep_data(event)
+                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
                 continue
 
-            if isinstance(event, (bb.event.BuildCompleted,
-                                  bb.command.CommandCompleted,
+            if isinstance(event, (bb.command.CommandCompleted,
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit)):
-                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
 
                 # we start a new build info
-                buildinfohelper = BuildInfoHelper(server)
+                buildinfohelper = BuildInfoHelper(server, build_history_enabled)
                 buildinfohelper.store_layer_info()
                 continue
 
bitbake/lib/webhob/orm/models.py +2 -2
index a4721db..774cdff 100644
@@ -110,9 +110,9 @@ class Artifact(models.Model):
 
 class Package(models.Model):
     build = models.ForeignKey('Build', related_name='package_build')
-    recipe = models.ForeignKey('Recipe', related_name='package_recipe')
+    recipe = models.ForeignKey('Recipe', related_name='package_recipe', null=True)
     name = models.CharField(max_length=100)
-    version = models.CharField(max_length=100)
+    version = models.CharField(max_length=100, default="")
     size = models.IntegerField()
 
 
Signature · 1 line
1.8.1.2
#50 lore raw

PATCH 50/94 bitbake: webhob: add simple viewer for package information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

If the target built is an image, we add a link to a page which lists all the packages built for this build, with sizes, version names and recipes linked in. The package dependency information is also shown.

---
 bitbake/lib/webhob/bldviewer/templates/build.html  |  2 +-
 .../lib/webhob/bldviewer/templates/package.html    | 42 ++++++++++++++++++++++
 bitbake/lib/webhob/bldviewer/urls.py               |  1 +
 bitbake/lib/webhob/bldviewer/views.py              | 18 +++++++++-
 4 files changed, 61 insertions(+), 2 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/package.html
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
index 8ee3800..15d88a0 100644
@@ -30,7 +30,7 @@
             <td>{{build.started_on}}</td>
             <td>{{build.completed_on}}</td>
             <td>{{build.target}}</td>
-            <td>{{build.is_image}}</td>
+            <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td>
             <td>{{build.machine.name}}</td>
             <td>{% time_difference build.started_on build.completed_on %}</td>
             <td>{{build.errors_no}}</td>
bitbake/lib/webhob/bldviewer/templates/package.html new file +42 -0

42 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/urls.py +1 -0
index 5f07161..8b8b5c3 100644
@@ -4,6 +4,7 @@ from django.views.generic.simple import redirect_to
 urlpatterns = patterns('bldviewer.views',
         url(r'^build/$', 'build', name='build'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
+        url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'),
         url(r'^layer/$', 'layer', name='layer'),
         url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'),
         url(r'^$', redirect_to, {'url': 'build/'}),
bitbake/lib/webhob/bldviewer/views.py +17 -1
index 4f7b46c..99e4b99 100644
@@ -2,7 +2,7 @@ import operator
 
 from django.db.models import Q
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package
+from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package, Package_Dependency
 
 
 def build(request):
@@ -29,6 +29,22 @@ def task(request, build_id):
 
     return render(request, template, context)
 
+def package(request, build_id):
+    template = 'package.html'
+
+    packages = Package.objects.filter(build=build_id)
+    package_depends = Package_Dependency.objects.filter(package__in=packages)
+
+    for t in packages:
+        t.depends_on = []
+        for k in package_depends:
+            if t == k.package:
+                t.depends_on.append(k.depends_on)
+
+    context = {'packages': packages}
+
+    return render(request, template, context)
+
 def layer(request):
     template = 'layer.html'
     layer_info = Layer.objects.all()
Signature · 1 line
1.8.1.2
#51 lore raw

PATCH 51/94 bitbake: dsi: fix build stats data gathering

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We set the default values for build stats to be NULL so that we can distinguish the value from a legitimate 0.

Also fixing the data gathering since model field semantics changed - the version is kept on a separate field.

[YOCTO #4967]

---
 bitbake/lib/bb/ui/buildinfohelper.py | 2 +-
 bitbake/lib/webhob/orm/models.py     | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +1 -1
index 59b6fa3..1d8cb44 100644
@@ -286,7 +286,7 @@ class BuildInfoHelper(object):
         target = self.internal_state['target']
         machine = self.internal_state['build'].machine.name
         buildname = self.internal_state['build'].build_name
-        package = task_object.recipe.name
+        package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":")
 
         build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target,
                                                      machine=machine, buildname=buildname,
bitbake/lib/webhob/orm/models.py +2 -2
index 774cdff..f2cc7d9 100644
@@ -84,8 +84,8 @@ class Task(models.Model):
     script_type = models.IntegerField(choices=TASK_CODING, default=CODING_PYTHON)
     line_number = models.IntegerField(default=0)
     py_stack_trace = models.TextField(null=True)
-    disk_io = models.IntegerField(default=0)
-    cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, default=0)
+    disk_io = models.IntegerField(null=True)
+    cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, null=True)
     elapsed_time = models.CharField(max_length=50, default=0)
     errors_no = models.IntegerField(default=0)
     warnings_no = models.IntegerField(default=0)
Signature · 1 line
1.8.1.2
#52 lore raw

PATCH 52/94 bitbake: dsi: update build object on command end

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We need to make the final update to the build object on command completion, regardless of the status of the completion, in order to avoid leaving builds in the "In Progress" state.

---
 bitbake/lib/bb/ui/dsi.py | 1 +
 1 file changed, 1 insertion(+)
bitbake/lib/bb/ui/dsi.py +1 -0
index 98d225b..3e0b7ff 100644
@@ -474,6 +474,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                                   bb.command.CommandExit)):
 
                 # we start a new build info
+                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
                 buildinfohelper = BuildInfoHelper(server, build_history_enabled)
                 buildinfohelper.store_layer_info()
                 continue
Signature · 1 line
1.8.1.2
#53 lore raw

PATCH 53/94 bitbake: dsi: fix sstate task information gathering

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Fixing the information gathering for sstate-related tasks - now the start and completion events are correctly handled, and we get an update for tasks that were covered by sstate tasks.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 6 +++++-
 bitbake/lib/bb/ui/dsi.py             | 5 ++---
 2 files changed, 7 insertions(+), 4 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +5 -1
index 1d8cb44..bfd7fef 100644
@@ -67,7 +67,11 @@ class ORMWrapper(object):
         for v in vars(task_object):
             if v in task_information.keys():
                 vars(task_object)[v] = task_information[v]
+        # if we got covered by a setscene task, we're SSTATE
+        if task_object.outcome == Task.OUTCOME_COVERED and 1 == Task.objects.filter(task_executed=True, build = task_object.build, recipe = task_object.recipe, task_name=task_object.task_name+"_setscene").count():
+            task_object.outcome = Task.OUTCOME_SSTATE
 
+        # mark down duration if we have a start time
         if 'start_time' in task_information.keys():
             duration = datetime.datetime.now() - task_information['start_time']
             task_object.elapsed_time = duration.total_seconds()
@@ -393,7 +397,7 @@ class BuildInfoHelper(object):
         except:
             pass
 
-        if isinstance(event, bb.runqueue.runQueueTaskCompleted):
+        if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)):
             task_information['outcome'] = Task.OUTCOME_SUCCESS     # TODO: needs to use constants
             task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
             task_information['cpu_usage'] = task_build_stats['cpu_usage']
bitbake/lib/bb/ui/dsi.py +2 -3
index 3e0b7ff..0305121 100644
@@ -415,6 +415,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 continue
 
             if isinstance(event, bb.runqueue.sceneQueueTaskStarted):
+                buildinfohelper.store_started_task(event)
                 logger.info("Running setscene task %d of %d (%s)" % (event.stats.completed + event.stats.active + event.stats.failed + 1, event.stats.total, event.taskstring))
                 continue
 
@@ -446,10 +447,8 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
-            if isinstance(event, bb.runqueue.sceneQueueTaskFailed):
+            if isinstance(event, (bb.runqueue.sceneQueueTaskCompleted, bb.runqueue.sceneQueueTaskFailed)):
                 buildinfohelper.update_and_store_task(event)
-                logger.warn("Setscene task %s (%s) failed with exit code '%s' - real task will be run instead",
-                             event.taskid, event.taskstring, event.exitcode)
                 continue
 
 
Signature · 1 line
1.8.1.2
#54 lore raw

PATCH 54/94 bitbake: webhob: clean up starting script

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Clean up of webhob start-up script as to make sure it aborts the startup on database errors, and that it doesn't execute superflous kill commands when stopping.

---
 bitbake/bin/webhob | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
bitbake/bin/webhob +7 -3
index 099f2d3..aa35294 100755
@@ -39,9 +39,13 @@ function webserverKillAll()
 
 function webserverStartAll()
 {
-        python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 2)
+        retval=0
+        python $BBBASEDIR/lib/webhob/manage.py syncdb || retval=1
+        if [ $retval -eq 1 ]; then
+                echo "Failed db sync, stopping system start" 1>&2
+        fi
         python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
-        return 0
+        return $retval
 }
 
 
@@ -125,7 +129,7 @@ case $CMD in
         unset BBSERVER
         webserverKillAll
         # force stop any misbehaving bitbake server
-        lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 kill
+        lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 -r kill
     ;;
 esac
 
Signature · 1 line
1.8.1.2
#55 lore raw

PATCH 55/94 bitbake: dsi: store log information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding database model for storing Log Information

Adding code to store log information for errors
and warnings coming through the build system.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 41 +++++++++++++++++++++++++++++++++---
 bitbake/lib/bb/ui/dsi.py             |  9 ++++++--
 bitbake/lib/webhob/orm/models.py     | 15 +++++++++++++
 3 files changed, 60 insertions(+), 5 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +38 -3
index bfd7fef..d435f19 100644
@@ -8,9 +8,9 @@ import re
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, Package_Dependency
-from webhob.orm.models import Task_Dependency
-
+from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, LogMessage
+from webhob.orm.models import Task_Dependency, Package_Dependency
+from bb.msg import BBLogFormatter as format
 
 class ORMWrapper(object):
     """ This class creates the dictionaries needed to store information in the database
@@ -134,6 +134,18 @@ class ORMWrapper(object):
                                         depends_on = packagedict[px]['object'] );
 
 
+    def create_logmessage(self, log_information):
+        log_object = LogMessage.objects.create(
+                        build = log_information['build'],
+                        level = log_information['level'],
+                        message = log_information['message'])
+
+        for v in vars(log_object):
+            if v in log_information.keys():
+                vars(log_object)[v] = log_information[v]
+
+        return log_object.save()
+
 class BuildInfoHelper(object):
     """ This class gathers the build information from the server and sends it
         towards the ORM wrapper for storing in the database
@@ -513,3 +525,26 @@ class BuildInfoHelper(object):
                 dep = _save_a_task(taskdesc1)
                 Task_Dependency.objects.get_or_create( task = target, depends_on = dep )
 
+    def store_log_information(self, level, text):
+        log_information = {}
+        log_information['build'] = self.internal_state['build']
+        log_information['level'] = level
+        log_information['message'] = text
+        self.orm_wrapper.create_logmessage(log_information)
+
+    def store_log_event(self, event):
+        if event.levelno < format.WARNING:
+            return
+        if not 'build' in self.internal_state:
+            return
+        log_information = {}
+        log_information['build'] = self.internal_state['build']
+        if event.levelno >= format.ERROR:
+            log_information['level'] = LogMessage.ERROR
+        elif event.levelno == format.WARNING:
+            log_information['level'] = LogMessage.WARNING
+        log_information['message'] = event.msg
+        log_information['pathname'] = event.pathname
+        log_information['lineno'] = event.lineno
+        self.orm_wrapper.create_logmessage(log_information)
+
bitbake/lib/bb/ui/dsi.py +7 -2
index 0305121..78982f2 100644
@@ -332,6 +332,8 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 # include verbose/debug messages
                 if event.taskpid != 0 and event.levelno <= format.NOTE:
                     continue
+
+                buildinfohelper.store_log_event(event)
                 logger.handle(event)
                 continue
 
@@ -406,9 +408,12 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                     r = ""
 
                 if event._dependees:
-                    logger.error("Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)", r, event._item, ", ".join(event._dependees), r)
+                    text = "Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)" % (r, event._item, ", ".join(event._dependees), r)
                 else:
-                    logger.error("Nothing %sPROVIDES '%s'", r, event._item)
+                    text = "Nothing %sPROVIDES '%s'" % (r, event._item)
+
+                logger.error(text)
+                buildinfohelper.store_log_information(2, text)  # don't judge me
                 if event._reasons:
                     for reason in event._reasons:
                         logger.error("%s", reason)
bitbake/lib/webhob/orm/models.py +15 -0
index f2cc7d9..91a51ff 100644
@@ -174,3 +174,18 @@ class Machine(models.Model):
     name = models.CharField(max_length=100)
     description = models.TextField()
 
+
+class LogMessage(models.Model):
+    INFO = 0
+    WARNING = 1
+    ERROR = 2
+
+    LOG_LEVEL = ( (INFO, "info"),
+            (WARNING, "warn"),
+            (ERROR, "error") )
+
+    build = models.ForeignKey(Build)
+    level = models.IntegerField(choices=LOG_LEVEL, default=INFO)
+    message=models.CharField(max_length=240)
+    pathname = models.FilePathField(max_length=255, null=True)
+    lineno = models.IntegerField(null=True)
Signature · 1 line
1.8.1.2
#56 lore raw

PATCH 56/94 bitbake: webhob: add simple visualisation for build errors

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding Simple web error viewing in the builds page. Now the error/warning column will contain the actual content of the error.

---
 bitbake/lib/webhob/bldviewer/templates/build.html | 4 ++--
 bitbake/lib/webhob/bldviewer/views.py             | 6 ++++--
 2 files changed, 6 insertions(+), 4 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +2 -2
index 15d88a0..bef84a9 100644
@@ -33,8 +33,8 @@
             <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td>
             <td>{{build.machine.name}}</td>
             <td>{% time_difference build.started_on build.completed_on %}</td>
-            <td>{{build.errors_no}}</td>
-            <td>{{build.warnings_no}}</td>
+            <td>{% if  build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
+            <td>{% if  build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
             <td>{{build.image_fstypes}}</td>
             <td>{{build.cooker_log_path}}</td>
             <td>{{build.bitbake_version}}</td>
bitbake/lib/webhob/bldviewer/views.py +4 -2
index 99e4b99..8535963 100644
@@ -2,14 +2,16 @@ import operator
 
 from django.db.models import Q
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package, Package_Dependency
+from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage
+from orm.models import Task_Dependency, Package_Dependency
 
 
 def build(request):
     template = 'build.html'
     build_info = Build.objects.all()
+    logs = LogMessage.objects.all()
 
-    context = {'builds': build_info}
+    context = {'builds': build_info, 'logs': logs }
 
     return render(request, template, context)
 
Signature · 1 line
1.8.1.2
#57 lore raw

PATCH 57/94 bitbake: webhob: store logfile and message for Tasks

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Since we get the reference to the log file, and a corresponding message with Task* events, we store that in the database.

Corresponding changes to the model file.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 6 ++++++
 bitbake/lib/bb/ui/dsi.py             | 4 +++-
 bitbake/lib/webhob/orm/models.py     | 6 ++----
 3 files changed, 11 insertions(+), 5 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +6 -0
index d435f19..232eea8 100644
@@ -409,6 +409,12 @@ class BuildInfoHelper(object):
         except:
             pass
 
+        if 'logfile' in vars(event):
+            task_information['logfile'] = event.logfile
+
+        if '_message' in vars(event):
+            task_information['message'] = event._message
+
         if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)):
             task_information['outcome'] = Task.OUTCOME_SUCCESS     # TODO: needs to use constants
             task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
bitbake/lib/bb/ui/dsi.py +3 -1
index 78982f2..f72b1be 100644
@@ -303,7 +303,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
             if isinstance(event, bb.event.BuildStarted):
                 buildinfohelper.store_started_build(event)
 
-            if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)):
+            if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent)):
                 buildinfohelper.update_and_store_task(event)
                 continue
 
@@ -338,6 +338,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 continue
 
             if isinstance(event, bb.build.TaskFailed):
+                buildinfohelper.update_and_store_task(event)
                 return_value = 1
                 logfile = event.logfile
                 if logfile and os.path.exists(logfile):
@@ -362,6 +363,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                         if lines:
                             for line in lines:
                                 print(line)
+
             if isinstance(event, bb.build.TaskBase):
                 logger.info(event._message)
                 continue
bitbake/lib/webhob/orm/models.py +2 -4
index 91a51ff..f51f53d 100644
@@ -87,11 +87,9 @@ class Task(models.Model):
     disk_io = models.IntegerField(null=True)
     cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, null=True)
     elapsed_time = models.CharField(max_length=50, default=0)
-    errors_no = models.IntegerField(default=0)
-    warnings_no = models.IntegerField(default=0)
-    error = models.TextField(null=True)
-    warning = models.TextField(null=True)
     sstate_result = models.IntegerField(choices=SSTATE_RESULT, default=SSTATE_NA)
+    message = models.CharField(max_length=240)
+    logfile = models.FilePathField(max_length=255, null=True)
 
     class Meta:
         ordering = ('order', 'recipe' ,)
Signature · 1 line
1.8.1.2
#58 lore raw

PATCH 58/94 bitbake: webhob: display proper information in Simple

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We modify the Simple interface to match the changes in the Task file.

Also a couple of display-related changes in the Simple interface.

---
 bitbake/lib/webhob/bldviewer/templates/build.html |  4 ++--
 bitbake/lib/webhob/bldviewer/templates/task.html  | 10 +++++-----
 bitbake/lib/webhob/bldviewer/views.py             |  2 +-
 3 files changed, 8 insertions(+), 8 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +2 -2
index bef84a9..20aa310 100644
@@ -33,8 +33,8 @@
             <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td>
             <td>{{build.machine.name}}</td>
             <td>{% time_difference build.started_on build.completed_on %}</td>
-            <td>{% if  build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
-            <td>{% if  build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
+            <td>{{build.errors_no}}:{% if  build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
+            <td>{{build.warnings_no}}:{% if  build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
             <td>{{build.image_fstypes}}</td>
             <td>{{build.cooker_log_path}}</td>
             <td>{{build.bitbake_version}}</td>
bitbake/lib/webhob/bldviewer/templates/task.html +5 -5
index 2d78668..d87d261 100644
@@ -15,8 +15,8 @@
             <th>Recipe Version</th>
             <th>Task Type</th>
             <th>Outcome</th>
-            <th>Errors</th>
-            <th>Warnings</th>
+            <th>Message</th>
+            <th>Logfile</th>
             <th>Time</th>
             <th>CPU usage</th>
             <th>Disk I/O</th>
@@ -40,13 +40,13 @@
                 {% endif %}
 
                 <td>{{task.get_outcome_display}}</td>
-                <td>{{task.errors_no}}</td>
-                <td>{{task.warnings_no}}</td>
+                <td><p>{{task.message}}</td>
+                <td><a target="_fileview" href="file:///{{task.logfile}}">{{task.logfile}}</a></td>
                 <td>{{task.elapsed_time}}</td>
                 <td>{{task.cpu_usage}}</td>
                 <td>{{task.disk_io}}</td>
                 <td>{{task.get_script_type_display}}</td>
-                <td>{{task.recipe.file_path}}</td>
+                <td><a target="_fileview" href="file:///{{task.recipe.file_path}}">{{task.recipe.file_path}}</a></td>
                 <td>
             <div style="height: 3em; overflow:auto">
                 {% for tt in task.depends_on %}
bitbake/lib/webhob/bldviewer/views.py +1 -1
index 8535963..7f97312 100644
@@ -19,7 +19,7 @@ def task(request, build_id):
     template = 'task.html'
 
     tasks = Task.objects.filter(build=build_id)
-    task_depends = Task_Dependency.objects.filter(task__in=tasks)
+    task_depends = Task_Dependency.objects.filter(task__in=tasks).select_related()
 
     for t in tasks:
         t.depends_on = []
Signature · 1 line
1.8.1.2
#59 lore raw

PATCH 59/94 bitbake: dsi: clear up global variables between builds

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We're making sure that global variables that track a build data get reset when the build ends.

---
 bitbake/lib/bb/ui/dsi.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/dsi.py +5 -1
index f72b1be..c139cd2 100644
@@ -479,8 +479,12 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit)):
 
-                # we start a new build info
                 buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
+
+                # we start a new build info
+                errors = 0
+                warnings = 0
+                taskfailures = []
                 buildinfohelper = BuildInfoHelper(server, build_history_enabled)
                 buildinfohelper.store_layer_info()
                 continue
Signature · 1 line
1.8.1.2
#60 lore raw

PATCH 60/94 bitbake: dsi: small bugfixes in data collection

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

A couple of bugfixes in the data collection:
- cache the TMPDIR since it's not gonna change during the build
- CommandCompleted can be issues outside of a build, so update the build only if we're running one

---
 bitbake/lib/bb/ui/buildinfohelper.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +4 -3
index 232eea8..6f4789c 100644
@@ -161,6 +161,7 @@ class BuildInfoHelper(object):
         self.server = server
         self.orm_wrapper = ORMWrapper()
         self.has_build_history = has_build_history
+        self.tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
 
     def _configure_django(self):
         # Add webhob to sys path for importing modules
@@ -298,13 +299,12 @@ class BuildInfoHelper(object):
     def _get_path_information(self, task_object):
         build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/"
 
-        tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
         target = self.internal_state['target']
         machine = self.internal_state['build'].machine.name
         buildname = self.internal_state['build'].build_name
         package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":")
 
-        build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target,
+        build_stats_path = build_stats_format.format(tmpdir=self.tmp_dir, target=target,
                                                      machine=machine, buildname=buildname,
                                                      package=package)
 
@@ -373,7 +373,8 @@ class BuildInfoHelper(object):
 
 
     def update_build_information(self, event, errors, warnings, taskfailures):
-        self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures)
+        if 'build' in self.internal_state:
+            self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures)
 
     def store_started_task(self, event):
         identifier = re.split(':', event.taskfile)[-1] + event.taskname
Signature · 1 line
1.8.1.2
#61 lore raw

PATCH 61/94 bitbake: bldviewer: add jquery and bootstrap frameworks

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We are adding the jQuery and Bootstrap libraries, so we can add some advanced features to the Simple web interface.

jQuery license is MIT.
Bootstrap license is Apache v2.0

---
 .../lib/webhob/bldviewer/static/css/bootstrap.css  | 4797 +++++++++++
 .../lib/webhob/bldviewer/static/js/bootstrap.js    | 1982 +++++
 .../lib/webhob/bldviewer/static/js/jquery-2.0.3.js | 8829 ++++++++++++++++++++
 3 files changed, 15608 insertions(+)
 create mode 100644 bitbake/lib/webhob/bldviewer/static/css/bootstrap.css
 create mode 100644 bitbake/lib/webhob/bldviewer/static/js/bootstrap.js
 create mode 100644 bitbake/lib/webhob/bldviewer/static/js/jquery-2.0.3.js
bitbake/lib/webhob/bldviewer/static/css/bootstrap.css new file +4797 -0
new file mode 100644
index 0000000..cb62a54
@@ -0,0 +1,4797 @@
+/*!
+ * Bootstrap v3.0.0
+ *
+ * Copyright 2013 Twitter, Inc
+ * Licensed under the Apache License v2.0
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Designed and built with all the love in the world by @mdo and @fat.
+ */
+
+/*! normalize.css v2.1.0 | MIT License | git.io/normalize */
+
+article,
+aside,
+details,
+figcaption,
+figure,
+footer,
+header,
+hgroup,
+main,
+nav,
+section,
+summary {
+  display: block;
+}
+
+audio,
+canvas,
+video {
+  display: inline-block;
+}
+
+audio:not([controls]) {
+  display: none;
+  height: 0;
+}
+
+[hidden] {
+  display: none;
+}
+
+html {
+  font-family: sans-serif;
+  -webkit-text-size-adjust: 100%;
+      -ms-text-size-adjust: 100%;
+}
+
+body {
+  margin: 0;
+}
+
+a:focus {
+  outline: thin dotted;
+}
+
+a:active,
+a:hover {
+  outline: 0;
+}
+
+h1 {
+  margin: 0.67em 0;
+  font-size: 2em;
+}
+
+abbr[title] {
+  border-bottom: 1px dotted;
+}
+
+b,
+strong {
+  font-weight: bold;
+}
+
+dfn {
+  font-style: italic;
+}
+
+hr {
+  height: 0;
+  -moz-box-sizing: content-box;
+       box-sizing: content-box;
+}
+
+mark {
+  color: #000;
+  background: #ff0;
+}
+
+code,
+kbd,
+pre,
+samp {
+  font-family: monospace, serif;
+  font-size: 1em;
+}
+
+pre {
+  white-space: pre-wrap;
+}
+
+q {
+  quotes: "\201C" "\201D" "\2018" "\2019";
+}
+
+small {
+  font-size: 80%;
+}
+
+sub,
+sup {
+  position: relative;
+  font-size: 75%;
+  line-height: 0;
+  vertical-align: baseline;
+}
+
+sup {
+  top: -0.5em;
+}
+
+sub {
+  bottom: -0.25em;
+}
+
+img {
+  border: 0;
+}
+
+svg:not(:root) {
+  overflow: hidden;
+}
+
+figure {
+  margin: 0;
+}
+
+fieldset {
+  padding: 0.35em 0.625em 0.75em;
+  margin: 0 2px;
+  border: 1px solid #c0c0c0;
+}
+
+legend {
+  padding: 0;
+  border: 0;
+}
+
+button,
+input,
+select,
+textarea {
+  margin: 0;
+  font-family: inherit;
+  font-size: 100%;
+}
+
+button,
+input {
+  line-height: normal;
+}
+
+button,
+select {
+  text-transform: none;
+}
+
+button,
+html input[type="button"],
+input[type="reset"],
+input[type="submit"] {
+  cursor: pointer;
+  -webkit-appearance: button;
+}
+
+button[disabled],
+html input[disabled] {
+  cursor: default;
+}
+
+input[type="checkbox"],
+input[type="radio"] {
+  padding: 0;
+  box-sizing: border-box;
+}
+
+input[type="search"] {
+  -webkit-box-sizing: content-box;
+     -moz-box-sizing: content-box;
+          box-sizing: content-box;
+  -webkit-appearance: textfield;
+}
+
+input[type="search"]::-webkit-search-cancel-button,
+input[type="search"]::-webkit-search-decoration {
+  -webkit-appearance: none;
+}
+
+button::-moz-focus-inner,
+input::-moz-focus-inner {
+  padding: 0;
+  border: 0;
+}
+
+textarea {
+  overflow: auto;
+  vertical-align: top;
+}
+
+table {
+  border-collapse: collapse;
+  border-spacing: 0;
+}
+
+@media print {
+  * {
+    color: #000 !important;
+    text-shadow: none !important;
+    background: transparent !important;
+    box-shadow: none !important;
+  }
+  a,
+  a:visited {
+    text-decoration: underline;
+  }
+  a[href]:after {
+    content: " (" attr(href) ")";
+  }
+  abbr[title]:after {
+    content: " (" attr(title) ")";
+  }
+  .ir a:after,
+  a[href^="javascript:"]:after,
+  a[href^="#"]:after {
+    content: "";
+  }
+  pre,
+  blockquote {
+    border: 1px solid #999;
+    page-break-inside: avoid;
+  }
+  thead {
+    display: table-header-group;
+  }
+  tr,
+  img {
+    page-break-inside: avoid;
+  }
+  img {
+    max-width: 100% !important;
+  }
+  @page  {
+    margin: 2cm .5cm;
+  }
+  p,
+  h2,
+  h3 {
+    orphans: 3;
+    widows: 3;
+  }
+  h2,
+  h3 {
+    page-break-after: avoid;
+  }
+  .navbar {
+    display: none;
+  }
+  .table td,
+  .table th {
+    background-color: #fff !important;
+  }
+  .btn > .caret,
+  .dropup > .btn > .caret {
+    border-top-color: #000 !important;
+  }
+  .label {
+    border: 1px solid #000;
+  }
+  .table {
+    border-collapse: collapse !important;
+  }
+  .table-bordered th,
+  .table-bordered td {
+    border: 1px solid #ddd !important;
+  }
+}
+
+* {
+  -webkit-box-sizing: border-box;
+     -moz-box-sizing: border-box;
+          box-sizing: border-box;
+}
+
+html {
+  font-size: 62.5%;
+  -webkit-tap-highlight-color: rgba(0, 0, 0, 0);
+}
+
+body {
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+  font-size: 14px;
+  line-height: 1.428571429;
+  color: #333333;
+  background-color: #ffffff;
+}
+
+input,
+button,
+select,
+textarea {
+  font-family: inherit;
+  font-size: inherit;
+  line-height: inherit;
+}
+
+a {
+  color: #428bca;
+  text-decoration: none;
+}
+
+a:hover,
+a:focus {
+  color: #2a6496;
+  text-decoration: underline;
+}
+
+a:focus {
+  outline: thin dotted #333;
+  outline: 5px auto -webkit-focus-ring-color;
+  outline-offset: -2px;
+}
+
+img {
+  vertical-align: middle;
+}
+
+.img-responsive {
+  display: inline-block;
+  height: auto;
+  max-width: 100%;
+}
+
+.img-rounded {
+  border-radius: 6px;
+}
+
+.img-circle {
+  border-radius: 500px;
+}
+
+hr {
+  margin-top: 20px;
+  margin-bottom: 20px;
+  border: 0;
+  border-top: 1px solid #eeeeee;
+}
+
+p {
+  margin: 0 0 10px;
+}
+
+.lead {
+  margin-bottom: 20px;
+  font-size: 16.099999999999998px;
+  font-weight: 200;
+  line-height: 1.4;
+}
+
+@media (min-width: 768px) {
+  .lead {
+    font-size: 21px;
+  }
+}
+
+small {
+  font-size: 85%;
+}
+
+cite {
+  font-style: normal;
+}
+
+.text-muted {
+  color: #999999;
+}
+
+.text-primary {
+  color: #428bca;
+}
+
+.text-warning {
+  color: #c09853;
+}
+
+.text-danger {
+  color: #b94a48;
+}
+
+.text-success {
+  color: #468847;
+}
+
+.text-info {
+  color: #3a87ad;
+}
+
+.text-left {
+  text-align: left;
+}
+
+.text-right {
+  text-align: right;
+}
+
+.text-center {
+  text-align: center;
+}
+
+h1,
+h2,
+h3,
+h4,
+h5,
+h6,
+.h1,
+.h2,
+.h3,
+.h4,
+.h5,
+.h6 {
+  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
+  font-weight: 500;
+  line-height: 1.1;
+}
+
+h1 small,
+h2 small,
+h3 small,
+h4 small,
+h5 small,
+h6 small,
+.h1 small,
+.h2 small,
+.h3 small,
+.h4 small,
+.h5 small,
+.h6 small {
+  font-weight: normal;
+  line-height: 1;
+  color: #999999;
+}
+
+h1,
+h2,
+h3 {
+  margin-top: 20px;
+  margin-bottom: 10px;
+}
+
+h4,
+h5,
+h6 {
+  margin-top: 10px;
+  margin-bottom: 10px;
+}
+
+h1,
+.h1 {
+  font-size: 38px;
+}
+
+h2,
+.h2 {
+  font-size: 32px;
+}
+
+h3,
+.h3 {
+  font-size: 24px;
+}
+
+h4,
+.h4 {
+  font-size: 18px;
+}
+
+h5,
+.h5 {
+  font-size: 14px;
+}
+
+h6,
+.h6 {
+  font-size: 12px;
+}
+
+h1 small,
+.h1 small {
+  font-size: 24px;
+}
+
+h2 small,
+.h2 small {
+  font-size: 18px;
+}
+
+h3 small,
+.h3 small,
+h4 small,
+.h4 small {
+  font-size: 14px;
+}
+
+.page-header {
+  padding-bottom: 9px;
+  margin: 40px 0 20px;
+  border-bottom: 1px solid #eeeeee;
+}
+
+ul,
+ol {
+  margin-top: 0;
+  margin-bottom: 10px;
+}
+
+ul ul,
+ol ul,
+ul ol,
+ol ol {
+  margin-bottom: 0;
+}
+
+.list-unstyled {
+  padding-left: 0;
+  list-style: none;
+}
+
+.list-inline {
+  padding-left: 0;
+  list-style: none;
+}
+
+.list-inline > li {
+  display: inline-block;
+  padding-right: 5px;
+  padding-left: 5px;
+}
+
+dl {
+  margin-bottom: 20px;
+}
+
+dt,
+dd {
+  line-height: 1.428571429;
+}
+
+dt {
+  font-weight: bold;
+}
+
+dd {
+  margin-left: 0;
+}
+
+.dl-horizontal dt {
+  float: left;
+  width: 160px;
+  overflow: hidden;
+  clear: left;
+  text-align: right;
+  text-overflow: ellipsis;
+  white-space: nowrap;
+}
+
+.dl-horizontal dd {
+  margin-left: 180px;
+}
+
+.dl-horizontal dd:before,
+.dl-horizontal dd:after {
+  display: table;
+  content: " ";
+}
+
+.dl-horizontal dd:after {
+  clear: both;
+}
+
+.dl-horizontal dd:before,
+.dl-horizontal dd:after {
+  display: table;
+  content: " ";
+}
+
+.dl-horizontal dd:after {
+  clear: both;
+}
+
+abbr[title],
+abbr[data-original-title] {
+  cursor: help;
+  border-bottom: 1px dotted #999999;
+}
+
+abbr.initialism {
+  font-size: 90%;
+  text-transform: uppercase;
+}
+
+blockquote {
+  padding: 10px 20px;
+  margin: 0 0 20px;
+  border-left: 5px solid #eeeeee;
+}
+
+blockquote p {
+  font-size: 17.5px;
+  font-weight: 300;
+  line-height: 1.25;
+}
+
+blockquote p:last-child {
+  margin-bottom: 0;
+}
+
+blockquote small {
+  display: block;
+  line-height: 1.428571429;
+  color: #999999;
+}
+
+blockquote small:before {
+  content: '\2014 \00A0';
+}
+
+blockquote.pull-right {
+  float: right;
+  padding-right: 15px;
+  padding-left: 0;
+  border-right: 5px solid #eeeeee;
+  border-left: 0;
+}
+
+blockquote.pull-right p,
+blockquote.pull-right small {
+  text-align: right;
+}
+
+blockquote.pull-right small:before {
+  content: '';
+}
+
+blockquote.pull-right small:after {
+  content: '\00A0 \2014';
+}
+
+q:before,
+q:after,
+blockquote:before,
+blockquote:after {
+  content: "";
+}
+
+address {
+  display: block;
+  margin-bottom: 20px;
+  font-style: normal;
+  line-height: 1.428571429;
+}
+
+code,
+pre {
+  font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
+}
+
+code {
+  padding: 2px 4px;
+  font-size: 90%;
+  color: #c7254e;
+  white-space: nowrap;
+  background-color: #f9f2f4;
+  border-radius: 4px;
+}
+
+pre {
+  display: block;
+  padding: 9.5px;
+  margin: 0 0 10px;
+  font-size: 13px;
+  line-height: 1.428571429;
+  color: #333333;
+  word-break: break-all;
+  word-wrap: break-word;
+  background-color: #f5f5f5;
+  border: 1px solid #cccccc;
+  border-radius: 4px;
+}
+
+pre.prettyprint {
+  margin-bottom: 20px;
+}
+
+pre code {
+  padding: 0;
+  color: inherit;
+  white-space: pre-wrap;
+  background-color: transparent;
+  border: 0;
+}
+
+.pre-scrollable {
+  max-height: 340px;
+  overflow-y: scroll;
+}
+
+.container {
+  margin-right: auto;
+  margin-left: auto;
+}
+
+.container:before,
+.container:after {
+  display: table;
+  content: " ";
+}
+
+.container:after {
+  clear: both;
+}
+
+.container:before,
+.container:after {
+  display: table;
+  content: " ";
+}
+
+.container:after {
+  clear: both;
+}
+
+.row:before,
+.row:after {
+  display: table;
+  content: " ";
+}
+
+.row:after {
+  clear: both;
+}
+
+.row:before,
+.row:after {
+  display: table;
+  content: " ";
+}
+
+.row:after {
+  clear: both;
+}
+
+@media (min-width: 768px) {
+  .row {
+    margin-right: -15px;
+    margin-left: -15px;
+  }
+}
+
+.row .row {
+  margin-right: -15px;
+  margin-left: -15px;
+}
+
+.col-1,
+.col-2,
+.col-3,
+.col-4,
+.col-5,
+.col-6,
+.col-7,
+.col-8,
+.col-9,
+.col-10,
+.col-11,
+.col-12,
+.col-sm-1,
+.col-sm-2,
+.col-sm-3,
+.col-sm-4,
+.col-sm-5,
+.col-sm-6,
+.col-sm-7,
+.col-sm-8,
+.col-sm-9,
+.col-sm-10,
+.col-sm-11,
+.col-sm-12,
+.col-lg-1,
+.col-lg-2,
+.col-lg-3,
+.col-lg-4,
+.col-lg-5,
+.col-lg-6,
+.col-lg-7,
+.col-lg-8,
+.col-lg-9,
+.col-lg-10,
+.col-lg-11,
+.col-lg-12 {
+  position: relative;
+  min-height: 1px;
+  padding-right: 15px;
+  padding-left: 15px;
+}
+
+.col-1,
+.col-2,
+.col-3,
+.col-4,
+.col-5,
+.col-6,
+.col-7,
+.col-8,
+.col-9,
+.col-10,
+.col-11,
+.col-12 {
+  float: left;
+}
+
+.col-1 {
+  width: 8.333333333333332%;
+}
+
+.col-2 {
+  width: 16.666666666666664%;
+}
+
+.col-3 {
+  width: 25%;
+}
+
+.col-4 {
+  width: 33.33333333333333%;
+}
+
+.col-5 {
+  width: 41.66666666666667%;
+}
+
+.col-6 {
+  width: 50%;
+}
+
+.col-7 {
+  width: 58.333333333333336%;
+}
+
+.col-8 {
+  width: 66.66666666666666%;
+}
+
+.col-9 {
+  width: 75%;
+}
+
+.col-10 {
+  width: 83.33333333333334%;
+}
+
+.col-11 {
+  width: 91.66666666666666%;
+}
+
+.col-12 {
+  width: 100%;
+}
+
+@media (min-width: 768px) {
+  .container {
+    max-width: 728px;
+  }
+  .col-sm-1,
+  .col-sm-2,
+  .col-sm-3,
+  .col-sm-4,
+  .col-sm-5,
+  .col-sm-6,
+  .col-sm-7,
+  .col-sm-8,
+  .col-sm-9,
+  .col-sm-10,
+  .col-sm-11,
+  .col-sm-12 {
+    float: left;
+  }
+  .col-sm-1 {
+    width: 8.333333333333332%;
+  }
+  .col-sm-2 {
+    width: 16.666666666666664%;
+  }
+  .col-sm-3 {
+    width: 25%;
+  }
+  .col-sm-4 {
+    width: 33.33333333333333%;
+  }
+  .col-sm-5 {
+    width: 41.66666666666667%;
+  }
+  .col-sm-6 {
+    width: 50%;
+  }
+  .col-sm-7 {
+    width: 58.333333333333336%;
+  }
+  .col-sm-8 {
+    width: 66.66666666666666%;
+  }
+  .col-sm-9 {
+    width: 75%;
+  }
+  .col-sm-10 {
+    width: 83.33333333333334%;
+  }
+  .col-sm-11 {
+    width: 91.66666666666666%;
+  }
+  .col-sm-12 {
+    width: 100%;
+  }
+  .col-sm-push-1 {
+    left: 8.333333333333332%;
+  }
+  .col-sm-push-2 {
+    left: 16.666666666666664%;
+  }
+  .col-sm-push-3 {
+    left: 25%;
+  }
+  .col-sm-push-4 {
+    left: 33.33333333333333%;
+  }
+  .col-sm-push-5 {
+    left: 41.66666666666667%;
+  }
+  .col-sm-push-6 {
+    left: 50%;
+  }
+  .col-sm-push-7 {
+    left: 58.333333333333336%;
+  }
+  .col-sm-push-8 {
+    left: 66.66666666666666%;
+  }
+  .col-sm-push-9 {
+    left: 75%;
+  }
+  .col-sm-push-10 {
+    left: 83.33333333333334%;
+  }
+  .col-sm-push-11 {
+    left: 91.66666666666666%;
+  }
+  .col-sm-pull-1 {
+    right: 8.333333333333332%;
+  }
+  .col-sm-pull-2 {
+    right: 16.666666666666664%;
+  }
+  .col-sm-pull-3 {
+    right: 25%;
+  }
+  .col-sm-pull-4 {
+    right: 33.33333333333333%;
+  }
+  .col-sm-pull-5 {
+    right: 41.66666666666667%;
+  }
+  .col-sm-pull-6 {
+    right: 50%;
+  }
+  .col-sm-pull-7 {
+    right: 58.333333333333336%;
+  }
+  .col-sm-pull-8 {
+    right: 66.66666666666666%;
+  }
+  .col-sm-pull-9 {
+    right: 75%;
+  }
+  .col-sm-pull-10 {
+    right: 83.33333333333334%;
+  }
+  .col-sm-pull-11 {
+    right: 91.66666666666666%;
+  }
+  .col-sm-offset-1 {
+    margin-left: 8.333333333333332%;
+  }
+  .col-sm-offset-2 {
+    margin-left: 16.666666666666664%;
+  }
+  .col-sm-offset-3 {
+    margin-left: 25%;
+  }
+  .col-sm-offset-4 {
+    margin-left: 33.33333333333333%;
+  }
+  .col-sm-offset-5 {
+    margin-left: 41.66666666666667%;
+  }
+  .col-sm-offset-6 {
+    margin-left: 50%;
+  }
+  .col-sm-offset-7 {
+    margin-left: 58.333333333333336%;
+  }
+  .col-sm-offset-8 {
+    margin-left: 66.66666666666666%;
+  }
+  .col-sm-offset-9 {
+    margin-left: 75%;
+  }
+  .col-sm-offset-10 {
+    margin-left: 83.33333333333334%;
+  }
+  .col-sm-offset-11 {
+    margin-left: 91.66666666666666%;
+  }
+}
+
+@media (min-width: 992px) {
+  .container {
+    max-width: 940px;
+  }
+  .col-lg-1,
+  .col-lg-2,
+  .col-lg-3,
+  .col-lg-4,
+  .col-lg-5,
+  .col-lg-6,
+  .col-lg-7,
+  .col-lg-8,
+  .col-lg-9,
+  .col-lg-10,
+  .col-lg-11,
+  .col-lg-12 {
+    float: left;
+  }
+  .col-lg-1 {
+    width: 8.333333333333332%;
+  }
+  .col-lg-2 {
+    width: 16.666666666666664%;
+  }
+  .col-lg-3 {
+    width: 25%;
+  }
+  .col-lg-4 {
+    width: 33.33333333333333%;
+  }
+  .col-lg-5 {
+    width: 41.66666666666667%;
+  }
+  .col-lg-6 {
+    width: 50%;
+  }
+  .col-lg-7 {
+    width: 58.333333333333336%;
+  }
+  .col-lg-8 {
+    width: 66.66666666666666%;
+  }
+  .col-lg-9 {
+    width: 75%;
+  }
+  .col-lg-10 {
+    width: 83.33333333333334%;
+  }
+  .col-lg-11 {
+    width: 91.66666666666666%;
+  }
+  .col-lg-12 {
+    width: 100%;
+  }
+  .col-lg-push-1 {
+    left: 8.333333333333332%;
+  }
+  .col-lg-push-2 {
+    left: 16.666666666666664%;
+  }
+  .col-lg-push-3 {
+    left: 25%;
+  }
+  .col-lg-push-4 {
+    left: 33.33333333333333%;
+  }
+  .col-lg-push-5 {
+    left: 41.66666666666667%;
+  }
+  .col-lg-push-6 {
+    left: 50%;
+  }
+  .col-lg-push-7 {
+    left: 58.333333333333336%;
+  }
+  .col-lg-push-8 {
+    left: 66.66666666666666%;
+  }
+  .col-lg-push-9 {
+    left: 75%;
+  }
+  .col-lg-push-10 {
+    left: 83.33333333333334%;
+  }
+  .col-lg-push-11 {
+    left: 91.66666666666666%;
+  }
+  .col-lg-pull-1 {
+    right: 8.333333333333332%;
+  }
+  .col-lg-pull-2 {
+    right: 16.666666666666664%;
+  }
+  .col-lg-pull-3 {
+    right: 25%;
+  }
+  .col-lg-pull-4 {
+    right: 33.33333333333333%;
+  }
+  .col-lg-pull-5 {
+    right: 41.66666666666667%;
+  }
+  .col-lg-pull-6 {
+    right: 50%;
+  }
+  .col-lg-pull-7 {
+    right: 58.333333333333336%;
+  }
+  .col-lg-pull-8 {
+    right: 66.66666666666666%;
+  }
+  .col-lg-pull-9 {
+    right: 75%;
+  }
+  .col-lg-pull-10 {
+    right: 83.33333333333334%;
+  }
+  .col-lg-pull-11 {
+    right: 91.66666666666666%;
+  }
+  .col-lg-offset-1 {
+    margin-left: 8.333333333333332%;
+  }
+  .col-lg-offset-2 {
+    margin-left: 16.666666666666664%;
+  }
+  .col-lg-offset-3 {
+    margin-left: 25%;
+  }
+  .col-lg-offset-4 {
+    margin-left: 33.33333333333333%;
+  }
+  .col-lg-offset-5 {
+    margin-left: 41.66666666666667%;
+  }
+  .col-lg-offset-6 {
+    margin-left: 50%;
+  }
+  .col-lg-offset-7 {
+    margin-left: 58.333333333333336%;
+  }
+  .col-lg-offset-8 {
+    margin-left: 66.66666666666666%;
+  }
+  .col-lg-offset-9 {
+    margin-left: 75%;
+  }
+  .col-lg-offset-10 {
+    margin-left: 83.33333333333334%;
+  }
+  .col-lg-offset-11 {
+    margin-left: 91.66666666666666%;
+  }
+}
+
+@media (min-width: 1200px) {
+  .container {
+    max-width: 1170px;
+  }
+}
+
+table {
+  max-width: 100%;
+  background-color: transparent;
+}
+
+th {
+  text-align: left;
+}
+
+.table {
+  width: 100%;
+  margin-bottom: 20px;
+}
+
+.table thead > tr > th,
+.table tbody > tr > th,
+.table tfoot > tr > th,
+.table thead > tr > td,
+.table tbody > tr > td,
+.table tfoot > tr > td {
+  padding: 8px;
+  line-height: 1.428571429;
+  vertical-align: top;
+  border-top: 1px solid #dddddd;
+}
+
+.table thead > tr > th {
+  vertical-align: bottom;
+}
+
+.table caption + thead tr:first-child th,
+.table colgroup + thead tr:first-child th,
+.table thead:first-child tr:first-child th,
+.table caption + thead tr:first-child td,
+.table colgroup + thead tr:first-child td,
+.table thead:first-child tr:first-child td {
+  border-top: 0;
+}
+
+.table tbody + tbody {
+  border-top: 2px solid #dddddd;
+}
+
+.table .table {
+  background-color: #ffffff;
+}
+
+.table-condensed thead > tr > th,
+.table-condensed tbody > tr > th,
+.table-condensed tfoot > tr > th,
+.table-condensed thead > tr > td,
+.table-condensed tbody > tr > td,
+.table-condensed tfoot > tr > td {
+  padding: 5px;
+}
+
+.table-bordered {
+  border: 1px solid #dddddd;
+}
+
+.table-bordered > thead > tr > th,
+.table-bordered > tbody > tr > th,
+.table-bordered > tfoot > tr > th,
+.table-bordered > thead > tr > td,
+.table-bordered > tbody > tr > td,
+.table-bordered > tfoot > tr > td {
+  border: 1px solid #dddddd;
+}
+
+.table-striped > tbody > tr:nth-child(odd) > td,
+.table-striped > tbody > tr:nth-child(odd) > th {
+  background-color: #f9f9f9;
+}
+
+.table-hover > tbody > tr:hover > td,
+.table-hover > tbody > tr:hover > th {
+  background-color: #f5f5f5;
+}
+
+table col[class^="col-"] {
+  display: table-column;
+  float: none;
+}
+
+table td[class^="col-"],
+table th[class^="col-"] {
+  display: table-cell;
+  float: none;
+}
+
+.table > thead > tr > td.active,
+.table > tbody > tr > td.active,
+.table > tfoot > tr > td.active,
+.table > thead > tr > th.active,
+.table > tbody > tr > th.active,
+.table > tfoot > tr > th.active,
+.table > thead > tr.active > td,
+.table > tbody > tr.active > td,
+.table > tfoot > tr.active > td,
+.table > thead > tr.active > th,
+.table > tbody > tr.active > th,
+.table > tfoot > tr.active > th {
+  background-color: #f5f5f5;
+}
+
+.table > thead > tr > td.success,
+.table > tbody > tr > td.success,
+.table > tfoot > tr > td.success,
+.table > thead > tr > th.success,
+.table > tbody > tr > th.success,
+.table > tfoot > tr > th.success,
+.table > thead > tr.success > td,
+.table > tbody > tr.success > td,
+.table > tfoot > tr.success > td,
+.table > thead > tr.success > th,
+.table > tbody > tr.success > th,
+.table > tfoot > tr.success > th {
+  background-color: #dff0d8;
+  border-color: #d6e9c6;
+}
+
+.table > thead > tr > td.danger,
+.table > tbody > tr > td.danger,
+.table > tfoot > tr > td.danger,
+.table > thead > tr > th.danger,
+.table > tbody > tr > th.danger,
+.table > tfoot > tr > th.danger,
+.table > thead > tr.danger > td,
+.table > tbody > tr.danger > td,
+.table > tfoot > tr.danger > td,
+.table > thead > tr.danger > th,
+.table > tbody > tr.danger > th,
+.table > tfoot > tr.danger > th {
+  background-color: #f2dede;
+  border-color: #eed3d7;
+}
+
+.table > thead > tr > td.warning,
+.table > tbody > tr > td.warning,
+.table > tfoot > tr > td.warning,
+.table > thead > tr > th.warning,
+.table > tbody > tr > th.warning,
+.table > tfoot > tr > th.warning,
+.table > thead > tr.warning > td,
+.table > tbody > tr.warning > td,
+.table > tfoot > tr.warning > td,
+.table > thead > tr.warning > th,
+.table > tbody > tr.warning > th,
+.table > tfoot > tr.warning > th {
+  background-color: #fcf8e3;
+  border-color: #fbeed5;
+}
+
+.table-hover > tbody > tr > td.success:hover,
+.table-hover > tbody > tr > th.success:hover,
+.table-hover > tbody > tr.success:hover > td {
+  background-color: #d0e9c6;
+  border-color: #c9e2b3;
+}
+
+.table-hover > tbody > tr > td.danger:hover,
+.table-hover > tbody > tr > th.danger:hover,
+.table-hover > tbody > tr.danger:hover > td {
+  background-color: #ebcccc;
+  border-color: #e6c1c7;
+}
+
+.table-hover > tbody > tr > td.warning:hover,
+.table-hover > tbody > tr > th.warning:hover,
+.table-hover > tbody > tr.warning:hover > td {
+  background-color: #faf2cc;
+  border-color: #f8e5be;
+}
+
+fieldset {
+  padding: 0;
+  margin: 0;
+  border: 0;
+}
+
+legend {
+  display: block;
+  width: 100%;
+  padding: 0;
+  margin-bottom: 20px;
+  font-size: 21px;
+  line-height: inherit;
+  color: #333333;
+  border: 0;
+  border-bottom: 1px solid #e5e5e5;
+}
+
+label {
+  display: inline-block;
+  margin-bottom: 5px;
+  font-weight: bold;
+}
+
+input[type="search"] {
+  -webkit-box-sizing: border-box;
+     -moz-box-sizing: border-box;
+          box-sizing: border-box;
+}
+
+input[type="radio"],
+input[type="checkbox"] {
+  margin: 4px 0 0;
+  margin-top: 1px \9;
+  /* IE8-9 */
+
+  line-height: normal;
+}
+
+input[type="file"] {
+  display: block;
+}
+
+select[multiple],
+select[size] {
+  height: auto;
+}
+
+select optgroup {
+  font-family: inherit;
+  font-size: inherit;
+  font-style: inherit;
+}
+
+input[type="file"]:focus,
+input[type="radio"]:focus,
+input[type="checkbox"]:focus {
+  outline: thin dotted #333;
+  outline: 5px auto -webkit-focus-ring-color;
+  outline-offset: -2px;
+}
+
+input[type="number"]::-webkit-outer-spin-button,
+input[type="number"]::-webkit-inner-spin-button {
+  height: auto;
+}
+
+.form-control:-moz-placeholder {
+  color: #999999;
+}
+
+.form-control::-moz-placeholder {
+  color: #999999;
+}
+
+.form-control:-ms-input-placeholder {
+  color: #999999;
+}
+
+.form-control::-webkit-input-placeholder {
+  color: #999999;
+}
+
+.form-control {
+  display: block;
+  width: 100%;
+  height: 34px;
+  padding: 6px 12px;
+  font-size: 14px;
+  line-height: 1.428571429;
+  color: #555555;
+  vertical-align: middle;
+  background-color: #ffffff;
+  border: 1px solid #cccccc;
+  border-radius: 4px;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+  -webkit-transition: border-color ease-in-out 0.15s, box-shadow ease-in-out 0.15s;
+          transition: border-color ease-in-out 0.15s, box-shadow ease-in-out 0.15s;
+}
+
+.form-control:focus {
+  border-color: #66afe9;
+  outline: 0;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(102, 175, 233, 0.6);
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(102, 175, 233, 0.6);
+}
+
+.form-control[disabled],
+.form-control[readonly],
+fieldset[disabled] .form-control {
+  cursor: not-allowed;
+  background-color: #eeeeee;
+}
+
+textarea.form-control {
+  height: auto;
+}
+
+.form-group {
+  margin-bottom: 15px;
+}
+
+.radio,
+.checkbox {
+  display: block;
+  min-height: 20px;
+  padding-left: 20px;
+  margin-top: 10px;
+  margin-bottom: 10px;
+  vertical-align: middle;
+}
+
+.radio label,
+.checkbox label {
+  display: inline;
+  margin-bottom: 0;
+  font-weight: normal;
+  cursor: pointer;
+}
+
+.radio input[type="radio"],
+.radio-inline input[type="radio"],
+.checkbox input[type="checkbox"],
+.checkbox-inline input[type="checkbox"] {
+  float: left;
+  margin-left: -20px;
+}
+
+.radio + .radio,
+.checkbox + .checkbox {
+  margin-top: -5px;
+}
+
+.radio-inline,
+.checkbox-inline {
+  display: inline-block;
+  padding-left: 20px;
+  margin-bottom: 0;
+  font-weight: normal;
+  vertical-align: middle;
+  cursor: pointer;
+}
+
+.radio-inline + .radio-inline,
+.checkbox-inline + .checkbox-inline {
+  margin-top: 0;
+  margin-left: 10px;
+}
+
+.input-lg {
+  height: 45px;
+  padding: 10px 16px;
+  font-size: 18px;
+  line-height: 1.33;
+  border-radius: 6px;
+}
+
+.input-sm {
+  height: 30px;
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 3px;
+}
+
+select.input-lg {
+  height: 45px;
+  line-height: 45px;
+}
+
+select.input-sm {
+  height: 30px;
+  line-height: 30px;
+}
+
+textarea.input-lg,
+textarea.input-sm {
+  height: auto;
+}
+
+.has-warning .help-block,
+.has-warning .control-label {
+  color: #c09853;
+}
+
+.has-warning .form-control {
+  border-color: #c09853;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+
+.has-warning .form-control:focus {
+  border-color: #a47e3c;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e;
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e;
+}
+
+.has-warning .input-group-addon {
+  color: #c09853;
+  background-color: #fcf8e3;
+  border-color: #c09853;
+}
+
+.has-error .help-block,
+.has-error .control-label {
+  color: #b94a48;
+}
+
+.has-error .form-control {
+  border-color: #b94a48;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+
+.has-error .form-control:focus {
+  border-color: #953b39;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392;
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392;
+}
+
+.has-error .input-group-addon {
+  color: #b94a48;
+  background-color: #f2dede;
+  border-color: #b94a48;
+}
+
+.has-success .help-block,
+.has-success .control-label {
+  color: #468847;
+}
+
+.has-success .form-control {
+  border-color: #468847;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
+}
+
+.has-success .form-control:focus {
+  border-color: #356635;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b;
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b;
+}
+
+.has-success .input-group-addon {
+  color: #468847;
+  background-color: #dff0d8;
+  border-color: #468847;
+}
+
+.help-block {
+  display: block;
+  margin-top: 5px;
+  margin-bottom: 10px;
+  color: #737373;
+}
+
+.form-inline .form-control,
+.form-inline .radio,
+.form-inline .checkbox {
+  display: inline-block;
+}
+
+.form-inline .radio,
+.form-inline .checkbox {
+  margin-top: 0;
+  margin-bottom: 0;
+}
+
+.form-horizontal .control-label,
+.form-horizontal .radio-inline,
+.form-horizontal .checkbox-inline {
+  padding-top: 9px;
+}
+
+.form-horizontal .form-group:before,
+.form-horizontal .form-group:after {
+  display: table;
+  content: " ";
+}
+
+.form-horizontal .form-group:after {
+  clear: both;
+}
+
+.form-horizontal .form-group:before,
+.form-horizontal .form-group:after {
+  display: table;
+  content: " ";
+}
+
+.form-horizontal .form-group:after {
+  clear: both;
+}
+
+@media (min-width: 768px) {
+  .form-horizontal .form-group {
+    margin-right: -15px;
+    margin-left: -15px;
+  }
+}
+
+.form-horizontal .form-group .row {
+  margin-right: -15px;
+  margin-left: -15px;
+}
+
+@media (min-width: 768px) {
+  .form-horizontal .control-label {
+    text-align: right;
+  }
+}
+
+.btn {
+  display: inline-block;
+  padding: 6px 12px;
+  margin-bottom: 0;
+  font-size: 14px;
+  font-weight: 500;
+  line-height: 1.428571429;
+  text-align: center;
+  white-space: nowrap;
+  vertical-align: middle;
+  cursor: pointer;
+  border: 1px solid transparent;
+  border-radius: 4px;
+  -webkit-user-select: none;
+     -moz-user-select: none;
+      -ms-user-select: none;
+       -o-user-select: none;
+          user-select: none;
+}
+
+.btn:focus {
+  outline: thin dotted #333;
+  outline: 5px auto -webkit-focus-ring-color;
+  outline-offset: -2px;
+}
+
+.btn:hover,
+.btn:focus {
+  color: #ffffff;
+  text-decoration: none;
+}
+
+.btn:active,
+.btn.active {
+  outline: 0;
+  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
+          box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
+}
+
+.btn.disabled,
+.btn[disabled],
+fieldset[disabled] .btn {
+  pointer-events: none;
+  cursor: default;
+  opacity: 0.65;
+  filter: alpha(opacity=65);
+  -webkit-box-shadow: none;
+          box-shadow: none;
+}
+
+.btn-default {
+  color: #ffffff;
+  background-color: #474949;
+  border-color: #474949;
+}
+
+.btn-default:hover,
+.btn-default:focus,
+.btn-default:active,
+.btn-default.active {
+  background-color: #3a3c3c;
+  border-color: #2e2f2f;
+}
+
+.btn-default.disabled,
+.btn-default[disabled],
+fieldset[disabled] .btn-default,
+.btn-default.disabled:hover,
+.btn-default[disabled]:hover,
+fieldset[disabled] .btn-default:hover,
+.btn-default.disabled:focus,
+.btn-default[disabled]:focus,
+fieldset[disabled] .btn-default:focus,
+.btn-default.disabled:active,
+.btn-default[disabled]:active,
+fieldset[disabled] .btn-default:active,
+.btn-default.disabled.active,
+.btn-default[disabled].active,
+fieldset[disabled] .btn-default.active {
+  background-color: #474949;
+  border-color: #474949;
+}
+
+.btn-primary {
+  color: #ffffff;
+  background-color: #428bca;
+  border-color: #428bca;
+}
+
+.btn-primary:hover,
+.btn-primary:focus,
+.btn-primary:active,
+.btn-primary.active {
+  background-color: #357ebd;
+  border-color: #3071a9;
+}
+
+.btn-primary.disabled,
+.btn-primary[disabled],
+fieldset[disabled] .btn-primary,
+.btn-primary.disabled:hover,
+.btn-primary[disabled]:hover,
+fieldset[disabled] .btn-primary:hover,
+.btn-primary.disabled:focus,
+.btn-primary[disabled]:focus,
+fieldset[disabled] .btn-primary:focus,
+.btn-primary.disabled:active,
+.btn-primary[disabled]:active,
+fieldset[disabled] .btn-primary:active,
+.btn-primary.disabled.active,
+.btn-primary[disabled].active,
+fieldset[disabled] .btn-primary.active {
+  background-color: #428bca;
+  border-color: #428bca;
+}
+
+.btn-warning {
+  color: #ffffff;
+  background-color: #f0ad4e;
+  border-color: #f0ad4e;
+}
+
+.btn-warning:hover,
+.btn-warning:focus,
+.btn-warning:active,
+.btn-warning.active {
+  background-color: #eea236;
+  border-color: #ec971f;
+}
+
+.btn-warning.disabled,
+.btn-warning[disabled],
+fieldset[disabled] .btn-warning,
+.btn-warning.disabled:hover,
+.btn-warning[disabled]:hover,
+fieldset[disabled] .btn-warning:hover,
+.btn-warning.disabled:focus,
+.btn-warning[disabled]:focus,
+fieldset[disabled] .btn-warning:focus,
+.btn-warning.disabled:active,
+.btn-warning[disabled]:active,
+fieldset[disabled] .btn-warning:active,
+.btn-warning.disabled.active,
+.btn-warning[disabled].active,
+fieldset[disabled] .btn-warning.active {
+  background-color: #f0ad4e;
+  border-color: #f0ad4e;
+}
+
+.btn-danger {
+  color: #ffffff;
+  background-color: #d9534f;
+  border-color: #d9534f;
+}
+
+.btn-danger:hover,
+.btn-danger:focus,
+.btn-danger:active,
+.btn-danger.active {
+  background-color: #d43f3a;
+  border-color: #c9302c;
+}
+
+.btn-danger.disabled,
+.btn-danger[disabled],
+fieldset[disabled] .btn-danger,
+.btn-danger.disabled:hover,
+.btn-danger[disabled]:hover,
+fieldset[disabled] .btn-danger:hover,
+.btn-danger.disabled:focus,
+.btn-danger[disabled]:focus,
+fieldset[disabled] .btn-danger:focus,
+.btn-danger.disabled:active,
+.btn-danger[disabled]:active,
+fieldset[disabled] .btn-danger:active,
+.btn-danger.disabled.active,
+.btn-danger[disabled].active,
+fieldset[disabled] .btn-danger.active {
+  background-color: #d9534f;
+  border-color: #d9534f;
+}
+
+.btn-success {
+  color: #ffffff;
+  background-color: #5cb85c;
+  border-color: #5cb85c;
+}
+
+.btn-success:hover,
+.btn-success:focus,
+.btn-success:active,
+.btn-success.active {
+  background-color: #4cae4c;
+  border-color: #449d44;
+}
+
+.btn-success.disabled,
+.btn-success[disabled],
+fieldset[disabled] .btn-success,
+.btn-success.disabled:hover,
+.btn-success[disabled]:hover,
+fieldset[disabled] .btn-success:hover,
+.btn-success.disabled:focus,
+.btn-success[disabled]:focus,
+fieldset[disabled] .btn-success:focus,
+.btn-success.disabled:active,
+.btn-success[disabled]:active,
+fieldset[disabled] .btn-success:active,
+.btn-success.disabled.active,
+.btn-success[disabled].active,
+fieldset[disabled] .btn-success.active {
+  background-color: #5cb85c;
+  border-color: #5cb85c;
+}
+
+.btn-info {
+  color: #ffffff;
+  background-color: #5bc0de;
+  border-color: #5bc0de;
+}
+
+.btn-info:hover,
+.btn-info:focus,
+.btn-info:active,
+.btn-info.active {
+  background-color: #46b8da;
+  border-color: #31b0d5;
+}
+
+.btn-info.disabled,
+.btn-info[disabled],
+fieldset[disabled] .btn-info,
+.btn-info.disabled:hover,
+.btn-info[disabled]:hover,
+fieldset[disabled] .btn-info:hover,
+.btn-info.disabled:focus,
+.btn-info[disabled]:focus,
+fieldset[disabled] .btn-info:focus,
+.btn-info.disabled:active,
+.btn-info[disabled]:active,
+fieldset[disabled] .btn-info:active,
+.btn-info.disabled.active,
+.btn-info[disabled].active,
+fieldset[disabled] .btn-info.active {
+  background-color: #5bc0de;
+  border-color: #5bc0de;
+}
+
+.btn-link {
+  font-weight: normal;
+  color: #428bca;
+  cursor: pointer;
+  border-radius: 0;
+}
+
+.btn-link,
+.btn-link:active,
+.btn-link[disabled],
+fieldset[disabled] .btn-link {
+  background-color: transparent;
+  -webkit-box-shadow: none;
+          box-shadow: none;
+}
+
+.btn-link,
+.btn-link:hover,
+.btn-link:focus,
+.btn-link:active {
+  border-color: transparent;
+}
+
+.btn-link:hover,
+.btn-link:focus {
+  color: #2a6496;
+  text-decoration: underline;
+  background-color: transparent;
+}
+
+.btn-link[disabled]:hover,
+fieldset[disabled] .btn-link:hover,
+.btn-link[disabled]:focus,
+fieldset[disabled] .btn-link:focus {
+  color: #333333;
+  text-decoration: none;
+}
+
+.btn-lg {
+  padding: 10px 16px;
+  font-size: 18px;
+  line-height: 1.33;
+  border-radius: 6px;
+}
+
+.btn-sm,
+.btn-xs {
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 3px;
+}
+
+.btn-xs {
+  padding: 3px 5px;
+}
+
+.btn-block {
+  display: block;
+  width: 100%;
+  padding-right: 0;
+  padding-left: 0;
+}
+
+.btn-block + .btn-block {
+  margin-top: 5px;
+}
+
+input[type="submit"].btn-block,
+input[type="reset"].btn-block,
+input[type="button"].btn-block {
+  width: 100%;
+}
+
+.fade {
+  opacity: 0;
+  -webkit-transition: opacity 0.15s linear;
+          transition: opacity 0.15s linear;
+}
+
+.fade.in {
+  opacity: 1;
+}
+
+.collapse {
+  display: none;
+}
+
+.collapse.in {
+  display: block;
+}
+
+.collapsing {
+  position: relative;
+  height: 0;
+  overflow: hidden;
+  -webkit-transition: height 0.35s ease;
+          transition: height 0.35s ease;
+}
+
+.input-group {
+  position: relative;
+  display: table;
+  border-collapse: separate;
+}
+
+.input-group.col {
+  float: none;
+  padding-right: 0;
+  padding-left: 0;
+}
+
+.input-group .form-control {
+  width: 100%;
+  margin-bottom: 0;
+}
+
+.input-group-addon,
+.input-group-btn,
+.input-group .form-control {
+  display: table-cell;
+}
+
+.input-group-addon:not(:first-child):not(:last-child),
+.input-group-btn:not(:first-child):not(:last-child),
+.input-group .form-control:not(:first-child):not(:last-child) {
+  border-radius: 0;
+}
+
+.input-group-addon,
+.input-group-btn {
+  width: 1%;
+  white-space: nowrap;
+  vertical-align: middle;
+}
+
+.input-group-addon {
+  padding: 6px 12px;
+  font-size: 14px;
+  font-weight: normal;
+  line-height: 1.428571429;
+  text-align: center;
+  background-color: #eeeeee;
+  border: 1px solid #cccccc;
+  border-radius: 4px;
+  -webkit-box-sizing: border-box;
+     -moz-box-sizing: border-box;
+          box-sizing: border-box;
+}
+
+.input-group-addon.input-sm {
+  padding: 5px 10px;
+  font-size: 12px;
+  line-height: 1.5;
+  border-radius: 3px;
+}
+
+.input-group-addon.input-lg {
+  padding: 10px 16px;
+  font-size: 18px;
+  line-height: 1.33;
+  border-radius: 6px;
+}
+
+.input-group-addon input[type="radio"],
+.input-group-addon input[type="checkbox"] {
+  margin-top: 0;
+}
+
+.input-group .form-control:first-child,
+.input-group-addon:first-child,
+.input-group-btn:first-child > .btn,
+.input-group-btn:first-child > .dropdown-toggle,
+.input-group-btn:last-child > .btn:not(:last-child):not(.dropdown-toggle) {
+  border-top-right-radius: 0;
+  border-bottom-right-radius: 0;
+}
+
+.input-group-addon:first-child {
+  border-right: 0;
+}
+
+.input-group .form-control:last-child,
+.input-group-addon:last-child,
+.input-group-btn:last-child > .btn,
+.input-group-btn:last-child > .dropdown-toggle,
+.input-group-btn:first-child > .btn:not(:first-child) {
+  border-bottom-left-radius: 0;
+  border-top-left-radius: 0;
+}
+
+.input-group-addon:last-child {
+  border-left: 0;
+}
+
+.input-group-btn {
+  position: relative;
+  white-space: nowrap;
+}
+
+.input-group-btn > .btn {
+  position: relative;
+}
+
+.input-group-btn > .btn + .btn {
+  margin-left: -4px;
+}
+
+.input-group-btn > .btn:hover,
+.input-group-btn > .btn:active {
+  z-index: 2;
+}
+
+.caret {
+  display: inline-block;
+  width: 0;
+  height: 0;
+  margin-left: 2px;
+  vertical-align: middle;
+  border-top: 4px solid #000000;
+  border-right: 4px solid transparent;
+  border-left: 4px solid transparent;
+  content: "";
+}
+
+.dropdown {
+  position: relative;
+}
+
+.dropdown-menu {
+  position: absolute;
+  top: 100%;
+  left: 0;
+  z-index: 1000;
+  display: none;
+  float: left;
+  min-width: 160px;
+  padding: 5px 0;
+  margin: 2px 0 0;
+  list-style: none;
+  background-color: #ffffff;
+  border: 1px solid #cccccc;
+  border: 1px solid rgba(0, 0, 0, 0.15);
+  border-radius: 4px;
+  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
+          box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
+  background-clip: padding-box;
+}
+
+.dropdown-menu.pull-right {
+  right: 0;
+  left: auto;
+}
+
+.dropdown-menu .divider {
+  height: 1px;
+  margin: 9px 0;
+  overflow: hidden;
+  background-color: #e5e5e5;
+}
+
+.dropdown-menu > li > a {
+  display: block;
+  padding: 3px 20px;
+  clear: both;
+  font-weight: normal;
+  line-height: 1.428571429;
+  color: #333333;
+  white-space: nowrap;
+}
+
+.dropdown-menu > li > a:hover,
+.dropdown-menu > li > a:focus {
+  color: #ffffff;
+  text-decoration: none;
+  background-color: #357ebd;
+  background-image: -webkit-gradient(linear, left 0%, left 100%, from(#428bca), to(#357ebd));
+  background-image: -webkit-linear-gradient(top, #428bca, 0%, #357ebd, 100%);
+  background-image: -moz-linear-gradient(top, #428bca 0%, #357ebd 100%);
+  background-image: linear-gradient(to bottom, #428bca 0%, #357ebd 100%);
+  background-repeat: repeat-x;
+  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff357ebd', GradientType=0);
+}
+
+.dropdown-menu > .active > a,
+.dropdown-menu > .active > a:hover,
+.dropdown-menu > .active > a:focus {
+  color: #ffffff;
+  text-decoration: none;
+  background-color: #357ebd;
+  background-image: -webkit-gradient(linear, left 0%, left 100%, from(#428bca), to(#357ebd));
+  background-image: -webkit-linear-gradient(top, #428bca, 0%, #357ebd, 100%);
+  background-image: -moz-linear-gradient(top, #428bca 0%, #357ebd 100%);
+  background-image: linear-gradient(to bottom, #428bca 0%, #357ebd 100%);
+  background-repeat: repeat-x;
+  outline: 0;
+  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff357ebd', GradientType=0);
+}
+
+.dropdown-menu > .disabled > a,
+.dropdown-menu > .disabled > a:hover,
+.dropdown-menu > .disabled > a:focus {
+  color: #999999;
+}
+
+.dropdown-menu > .disabled > a:hover,
+.dropdown-menu > .disabled > a:focus {
+  text-decoration: none;
+  cursor: not-allowed;
+  background-color: transparent;
+  background-image: none;
+  filter: progid:DXImageTransform.Microsoft.gradient(enabled=false);
+}
+
+.open > .dropdown-menu {
+  display: block;
+}
+
+.open > a {
+  outline: 0;
+}
+
+.dropdown-header {
+  display: block;
+  padding: 3px 20px;
+  font-size: 12px;
+  line-height: 1.428571429;
+  color: #999999;
+}
+
+.dropdown-backdrop {
+  position: fixed;
+  top: 0;
+  right: 0;
+  bottom: 0;
+  left: 0;
+  z-index: 990;
+}
+
+.pull-right > .dropdown-menu {
+  right: 0;
+  left: auto;
+}
+
+.dropup .caret,
+.navbar-fixed-bottom .dropdown .caret {
+  border-top: 0;
+  border-bottom: 4px solid #000000;
+  content: "";
+}
+
+.dropup .dropdown-menu,
+.navbar-fixed-bottom .dropdown .dropdown-menu {
+  top: auto;
+  bottom: 100%;
+  margin-bottom: 1px;
+}
+
+.list-group {
+  padding-left: 0;
+  margin-bottom: 20px;
+}
+
+.list-group-item {
+  position: relative;
+  display: block;
+  padding: 10px 30px 10px 15px;
+  margin-bottom: -1px;
+  background-color: #ffffff;
+  border: 1px solid #dddddd;
+}
+
+.list-group-item:first-child {
+  border-top-right-radius: 4px;
+  border-top-left-radius: 4px;
+}
+
+.list-group-item:last-child {
+  margin-bottom: 0;
+  border-bottom-right-radius: 4px;
+  border-bottom-left-radius: 4px;
+}
+
+.list-group-item > .badge {
+  float: right;
+  margin-right: -15px;
+}
+
+.list-group-item-heading {
+  margin-top: 0;
+  margin-bottom: 5px;
+}
+
+.list-group-item-text {
+  margin-bottom: 0;
+  line-height: 1.3;
+}
+
+a.list-group-item .list-group-item-heading {
+  color: #333333;
+}
+
+a.list-group-item .list-group-item-text {
+  color: #555555;
+}
+
+a.list-group-item:hover,
+a.list-group-item:focus {
+  text-decoration: none;
+  background-color: #f5f5f5;
+}
+
+a.list-group-item.active {
+  z-index: 2;
+  color: #ffffff;
+  background-color: #428bca;
+  border-color: #428bca;
+}
+
+a.list-group-item.active .list-group-item-heading {
+  color: inherit;
+}
+
+a.list-group-item.active .list-group-item-text {
+  color: #e1edf7;
+}
+
+.panel {
+  padding: 15px;
+  margin-bottom: 20px;
+  background-color: #ffffff;
+  border: 1px solid #dddddd;
+  border-radius: 4px;
+  -webkit-box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
+          box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
+}
+
+.panel .list-group {
+  margin: 15px -15px -15px;
+}
+
+.panel .list-group .list-group-item {
+  border-width: 1px 0;
+}
+
+.panel .list-group .list-group-item:first-child {
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+
+.panel .list-group .list-group-item:last-child {
+  border-bottom: 0;
+}
+
+.panel-heading {
+  padding: 10px 15px;
+  margin: -15px -15px 15px;
+  background-color: #f5f5f5;
+  border-bottom: 1px solid #dddddd;
+  border-top-right-radius: 3px;
+  border-top-left-radius: 3px;
+}
+
+.panel-title {
+  margin-top: 0;
+  margin-bottom: 0;
+  font-size: 17.5px;
+  font-weight: 500;
+}
+
+.panel-title > a {
+  color: inherit;
+}
+
+.panel-footer {
+  padding: 10px 15px;
+  margin: 15px -15px -15px;
+  background-color: #f5f5f5;
+  border-top: 1px solid #dddddd;
+  border-bottom-right-radius: 3px;
+  border-bottom-left-radius: 3px;
+}
+
+.panel-primary {
+  border-color: #428bca;
+}
+
+.panel-primary .panel-heading {
+  color: #ffffff;
+  background-color: #428bca;
+  border-color: #428bca;
+}
+
+.panel-success {
+  border-color: #d6e9c6;
+}
+
+.panel-success .panel-heading {
+  color: #468847;
+  background-color: #dff0d8;
+  border-color: #d6e9c6;
+}
+
+.panel-warning {
+  border-color: #fbeed5;
+}
+
+.panel-warning .panel-heading {
+  color: #c09853;
+  background-color: #fcf8e3;
+  border-color: #fbeed5;
+}
+
+.panel-danger {
+  border-color: #eed3d7;
+}
+
+.panel-danger .panel-heading {
+  color: #b94a48;
+  background-color: #f2dede;
+  border-color: #eed3d7;
+}
+
+.panel-info {
+  border-color: #bce8f1;
+}
+
+.panel-info .panel-heading {
+  color: #3a87ad;
+  background-color: #d9edf7;
+  border-color: #bce8f1;
+}
+
+.well {
+  min-height: 20px;
+  padding: 19px;
+  margin-bottom: 20px;
+  background-color: #f5f5f5;
+  border: 1px solid #e3e3e3;
+  border-radius: 4px;
+  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+          box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
+}
+
+.well blockquote {
+  border-color: #ddd;
+  border-color: rgba(0, 0, 0, 0.15);
+}
+
+.well-lg {
+  padding: 24px;
+  border-radius: 6px;
+}
+
+.well-sm {
+  padding: 9px;
+  border-radius: 3px;
+}
+
+.close {
+  float: right;
+  font-size: 21px;
+  font-weight: bold;
+  line-height: 1;
+  color: #000000;
+  text-shadow: 0 1px 0 #ffffff;
+  opacity: 0.2;
+  filter: alpha(opacity=20);
+}
+
+.close:hover,
+.close:focus {
+  color: #000000;
+  text-decoration: none;
+  cursor: pointer;
+  opacity: 0.5;
+  filter: alpha(opacity=50);
+}
+
+button.close {
+  padding: 0;
+  cursor: pointer;
+  background: transparent;
+  border: 0;
+  -webkit-appearance: none;
+}
+
+.nav {
+  padding-left: 0;
+  margin-bottom: 0;
+  list-style: none;
+}
+
+.nav:before,
+.nav:after {
+  display: table;
+  content: " ";
+}
+
+.nav:after {
+  clear: both;
+}
+
+.nav:before,
+.nav:after {
+  display: table;
+  content: " ";
+}
+
+.nav:after {
+  clear: both;
+}
+
+.nav > li {
+  position: relative;
+  display: block;
+}
+
+.nav > li > a {
+  position: relative;
+  display: block;
+  padding: 10px 15px;
+}
+
+.nav > li > a:hover,
+.nav > li > a:focus {
+  text-decoration: none;
+  background-color: #eeeeee;
+}
+
+.nav > li.disabled > a {
+  color: #999999;
+}
+
+.nav > li.disabled > a:hover,
+.nav > li.disabled > a:focus {
+  color: #999999;
+  text-decoration: none;
+  cursor: not-allowed;
+  background-color: transparent;
+}
+
+.nav.open > a,
+.nav.open > a:hover,
+.nav.open > a:focus {
+  color: #ffffff;
+  background-color: #428bca;
+  border-color: #428bca;
+}
+
+.nav.open > a .caret,
+.nav.open > a:hover .caret,
+.nav.open > a:focus .caret {
+  border-top-color: #ffffff;
+  border-bottom-color: #ffffff;
+}
+
+.nav > .pull-right {
+  float: right;
+}
+
+.nav .nav-divider {
+  height: 1px;
+  margin: 9px 0;
+  overflow: hidden;
+  background-color: #e5e5e5;
+}
+
+.nav-tabs {
+  border-bottom: 1px solid #dddddd;
+}
+
+.nav-tabs > li {
+  float: left;
+  margin-bottom: -1px;
+}
+
+.nav-tabs > li > a {
+  margin-right: 2px;
+  line-height: 1.428571429;
+  border: 1px solid transparent;
+  border-radius: 4px 4px 0 0;
+}
+
+.nav-tabs > li > a:hover {
+  border-color: #eeeeee;
+}
+
+.nav-tabs > li.active > a,
+.nav-tabs > li.active > a:hover,
+.nav-tabs > li.active > a:focus {
+  color: #555555;
+  cursor: default;
+  background-color: #ffffff;
+  border: 1px solid #dddddd;
+  border-bottom-color: transparent;
+}
+
+.nav-tabs.nav-justified {
+  width: 100%;
+  border-bottom: 0;
+}
+
+.nav-tabs.nav-justified > li {
+  display: table-cell;
+  float: none;
+  width: 1%;
+}
+
+.nav-tabs.nav-justified > li > a {
+  text-align: center;
+}
+
+.nav-tabs.nav-justified > li > a {
+  margin-right: 0;
+  border-bottom: 1px solid #dddddd;
+}
+
+.nav-tabs.nav-justified > .active > a {
+  border-bottom-color: #ffffff;
+}
+
+.nav-pills > li {
+  float: left;
+}
+
+.nav-pills > li > a {
+  border-radius: 5px;
+}
+
+.nav-pills > li + li {
+  margin-left: 2px;
+}
+
+.nav-pills > li.active > a,
+.nav-pills > li.active > a:hover,
+.nav-pills > li.active > a:focus {
+  color: #ffffff;
+  background-color: #428bca;
+}
+
+.nav-stacked > li {
+  float: none;
+}
+
+.nav-stacked > li + li {
+  margin-top: 2px;
+  margin-left: 0;
+}
+
+.nav-justified {
+  width: 100%;
+}
+
+.nav-justified > li {
+  display: table-cell;
+  float: none;
+  width: 1%;
+}
+
+.nav-justified > li > a {
+  text-align: center;
+}
+
+.nav-tabs-justified {
+  border-bottom: 0;
+}
+
+.nav-tabs-justified > li > a {
+  margin-right: 0;
+  border-bottom: 1px solid #dddddd;
+}
+
+.nav-tabs-justified > .active > a {
+  border-bottom-color: #ffffff;
+}
+
+.tabbable:before,
+.tabbable:after {
+  display: table;
+  content: " ";
+}
+
+.tabbable:after {
+  clear: both;
+}
+
+.tabbable:before,
+.tabbable:after {
+  display: table;
+  content: " ";
+}
+
+.tabbable:after {
+  clear: both;
+}
+
+.tab-content > .tab-pane,
+.pill-content > .pill-pane {
+  display: none;
+}
+
+.tab-content > .active,
+.pill-content > .active {
+  display: block;
+}
+
+.nav .caret {
+  border-top-color: #428bca;
+  border-bottom-color: #428bca;
+}
+
+.nav a:hover .caret {
+  border-top-color: #2a6496;
+  border-bottom-color: #2a6496;
+}
+
+.nav-tabs .dropdown-menu {
+  margin-top: -1px;
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+
+.navbar {
+  position: relative;
+  min-height: 50px;
+  padding-right: 15px;
+  padding-left: 15px;
+  margin-bottom: 20px;
+  background-color: #eeeeee;
+  border-radius: 4px;
+}
+
+.navbar:before,
+.navbar:after {
+  display: table;
+  content: " ";
+}
+
+.navbar:after {
+  clear: both;
+}
+
+.navbar:before,
+.navbar:after {
+  display: table;
+  content: " ";
+}
+
+.navbar:after {
+  clear: both;
+}
+
+.navbar-nav {
+  margin-top: 10px;
+  margin-bottom: 15px;
+}
+
+.navbar-nav > li > a {
+  padding-top: 15px;
+  padding-bottom: 15px;
+  line-height: 20px;
+  color: #777777;
+  border-radius: 4px;
+}
+
+.navbar-nav > li > a:hover,
+.navbar-nav > li > a:focus {
+  color: #333333;
+  background-color: transparent;
+}
+
+.navbar-nav > .active > a,
+.navbar-nav > .active > a:hover,
+.navbar-nav > .active > a:focus {
+  color: #555555;
+  background-color: #d5d5d5;
+}
+
+.navbar-nav > .disabled > a,
+.navbar-nav > .disabled > a:hover,
+.navbar-nav > .disabled > a:focus {
+  color: #cccccc;
+  background-color: transparent;
+}
+
+.navbar-nav.pull-right {
+  width: 100%;
+}
+
+.navbar-static-top {
+  border-radius: 0;
+}
+
+.navbar-fixed-top,
+.navbar-fixed-bottom {
+  position: fixed;
+  right: 0;
+  left: 0;
+  z-index: 1030;
+  border-radius: 0;
+}
+
+.navbar-fixed-top {
+  top: 0;
+}
+
+.navbar-fixed-bottom {
+  bottom: 0;
+  margin-bottom: 0;
+}
+
+.navbar-brand {
+  display: block;
+  max-width: 200px;
+  padding: 15px 15px;
+  margin-right: auto;
+  margin-left: auto;
+  font-size: 18px;
+  font-weight: 500;
+  line-height: 20px;
+  color: #777777;
+  text-align: center;
+}
+
+.navbar-brand:hover,
+.navbar-brand:focus {
+  color: #5e5e5e;
+  text-decoration: none;
+  background-color: transparent;
+}
+
+.navbar-toggle {
+  position: relative;
+  float: right;
+  width: 48px;
+  height: 34px;
+  padding: 6px 12px;
+  margin-top: 8px;
+  margin-bottom: 8px;
+  background-color: transparent;
+  border: 1px solid #dddddd;
+  border-radius: 4px;
+}
+
+.navbar-toggle:hover,
+.navbar-toggle:focus {
+  background-color: #dddddd;
+}
+
+.navbar-toggle .icon-bar {
+  display: block;
+  width: 22px;
+  height: 2px;
+  background-color: #cccccc;
+  border-radius: 1px;
+}
+
+.navbar-toggle .icon-bar + .icon-bar {
+  margin-top: 4px;
+}
+
+.navbar-form {
+  margin-top: 8px;
+  margin-bottom: 8px;
+}
+
+.navbar-form .form-control,
+.navbar-form .radio,
+.navbar-form .checkbox {
+  display: inline-block;
+}
+
+.navbar-form .radio,
+.navbar-form .checkbox {
+  margin-top: 0;
+  margin-bottom: 0;
+}
+
+.navbar-nav > li > .dropdown-menu {
+  margin-top: 0;
+  border-top-right-radius: 0;
+  border-top-left-radius: 0;
+}
+
+.navbar-fixed-bottom .navbar-nav > li > .dropdown-menu {
+  border-bottom-right-radius: 0;
+  border-bottom-left-radius: 0;
+}
+
+.navbar-nav > .dropdown > a:hover .caret,
+.navbar-nav > .dropdown > a:focus .caret {
+  border-top-color: #333333;
+  border-bottom-color: #333333;
+}
+
+.navbar-nav > .open > a,
+.navbar-nav > .open > a:hover,
+.navbar-nav > .open > a:focus {
+  color: #555555;
+  background-color: #d5d5d5;
+}
+
+.navbar-nav > .open > a .caret,
+.navbar-nav > .open > a:hover .caret,
+.navbar-nav > .open > a:focus .caret {
+  border-top-color: #555555;
+  border-bottom-color: #555555;
+}
+
+.navbar-nav > .dropdown > a .caret {
+  border-top-color: #777777;
+  border-bottom-color: #777777;
+}
+
+.navbar-nav.pull-right > li > .dropdown-menu,
+.navbar-nav > li > .dropdown-menu.pull-right {
+  right: 0;
+  left: auto;
+}
+
+.navbar-inverse {
+  background-color: #222222;
+}
+
+.navbar-inverse .navbar-brand {
+  color: #999999;
+}
+
+.navbar-inverse .navbar-brand:hover,
+.navbar-inverse .navbar-brand:focus {
+  color: #ffffff;
+  background-color: transparent;
+}
+
+.navbar-inverse .navbar-text {
+  color: #999999;
+}
+
+.navbar-inverse .navbar-nav > li > a {
+  color: #999999;
+}
+
+.navbar-inverse .navbar-nav > li > a:hover,
+.navbar-inverse .navbar-nav > li > a:focus {
+  color: #ffffff;
+  background-color: transparent;
+}
+
+.navbar-inverse .navbar-nav > .active > a,
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

bitbake/lib/webhob/bldviewer/static/js/bootstrap.js new file +1982 -0

1,982 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/static/js/jquery-2.0.3.js new file +8829 -0

8,829 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.8.1.2

Part of the diff is not shown here. View the raw message.

#62 lore raw

PATCH 62/94 bitbake: webhob: add toggle column functionality to build page

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We add demo toggle column functionality for the Simple interface, webhob builds page.

---
 bitbake/lib/webhob/bldviewer/templates/base.html  | 15 +++++++++++++++
 bitbake/lib/webhob/bldviewer/templates/build.html |  6 ++++++
 bitbake/lib/webhob/bldviewer/views.py             |  3 ++-
 3 files changed, 23 insertions(+), 1 deletion(-)
bitbake/lib/webhob/bldviewer/templates/base.html +15 -0
index ab3c3b7..3f056a0 100644
@@ -3,9 +3,24 @@
 <html>
 	<head>
 		<title>WebHob Simple Explorer</title>
+<script src="/static/js/jquery-2.0.3.js">
+</script>
+<script src="/static/js/bootstrap.js">
+</script>
+<link href="/static/css/bootstrap.css" rel="stylesheet" type="text/css">
 	</head>
 
 <body>
+<script>
+function showhideTableColumn(i, sh) {
+    if (sh)
+        $('td:nth-child('+i+'),th:nth-child('+i+')').show();
+    else
+        $('td:nth-child('+i+'),th:nth-child('+i+')').hide();
+g
+}
+</script>
+
 <div>Menu: <a href="/simple/build/">All Builds</a>&nbsp;|&nbsp;<a href="/simple/layer/">All Layers</a>
 </div>
 {% block pagecontent %}
bitbake/lib/webhob/bldviewer/templates/build.html +6 -0
index 20aa310..e30801b 100644
@@ -5,6 +5,12 @@
 
     <table border="1">
 
+<div align="right" style="display: float">Show/Hide columns:
+<span>Output: <input type="checkbox" id="ctoutput" onchange="showhideTableColumn(10, $('#ctoutput').is(':checked'))" checked autocomplete="off"></span> |
+<span>Log: <input type="checkbox" id="ctlog" onchange="showhideTableColumn(11, $('#ctlog').is(':checked'))"checked autocomplete="off"></span> |
+<span>Build Name: <input type="checkbox" id="ctbuildname" onchange="showhideTableColumn(13, $('#ctbuildname').is(':checked'))" checked autocomplete="off"></span> |
+</div>
+
     {% load projecttags %}
 
         <tr>
bitbake/lib/webhob/bldviewer/views.py +2 -1
index 7f97312..ab69c91 100644
@@ -4,8 +4,9 @@ from django.db.models import Q
 from django.shortcuts import render
 from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage
 from orm.models import Task_Dependency, Package_Dependency
+from django.views.decorators.cache import cache_control
 
-
+@cache_control(no_store=True)
 def build(request):
     template = 'build.html'
     build_info = Build.objects.all()
Signature · 1 line
1.8.1.2
#63 lore raw

PATCH 63/94 bitbake: webhob: add search functionality to the Simple interface

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We add generic table search function for the builds table, and make it generic to be able to expand to all other table views.

---
 bitbake/lib/webhob/bldviewer/templates/base.html    | 12 +++++++++++-
 bitbake/lib/webhob/bldviewer/templates/build.html   | 12 +++++++-----
 bitbake/lib/webhob/bldviewer/templates/package.html |  2 +-
 3 files changed, 19 insertions(+), 7 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +11 -1
index 3f056a0..45f8652 100644
@@ -17,7 +17,17 @@ function showhideTableColumn(i, sh) {
         $('td:nth-child('+i+'),th:nth-child('+i+')').show();
     else
         $('td:nth-child('+i+'),th:nth-child('+i+')').hide();
-g
+}
+
+
+function filterTableRows(test) {
+    if (test.length > 0) {
+        var r = new RegExp(test);
+        $('tr.data').map( function (i, el) { if (! r.test($(el).html())) {$(el).hide()} else $(el).show()});
+    } else
+    {
+        $('tr.data').show();
+    }
 }
 </script>
 
bitbake/lib/webhob/bldviewer/templates/build.html +7 -5
index e30801b..aa92929 100644
@@ -3,14 +3,16 @@
 {% block pagecontent %}
     <h1>WebHob Builds</h1>
 
-    <table border="1">
-
-<div align="right" style="display: float">Show/Hide columns:
+<div style="padding-top:1em ;padding-bottom: 1em">
+<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
+</div>
+<div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
 <span>Output: <input type="checkbox" id="ctoutput" onchange="showhideTableColumn(10, $('#ctoutput').is(':checked'))" checked autocomplete="off"></span> |
 <span>Log: <input type="checkbox" id="ctlog" onchange="showhideTableColumn(11, $('#ctlog').is(':checked'))"checked autocomplete="off"></span> |
 <span>Build Name: <input type="checkbox" id="ctbuildname" onchange="showhideTableColumn(13, $('#ctbuildname').is(':checked'))" checked autocomplete="off"></span> |
 </div>
-
+</div>
+    <table border="1">
     {% load projecttags %}
 
         <tr>
@@ -31,7 +33,7 @@
 
         {% for build in builds %}
 
-        <tr>
+        <tr class="data">
             <td><a href="/simple/build/{{build.id}}/task/">{{build.get_outcome_display}}</a></td>
             <td>{{build.started_on}}</td>
             <td>{{build.completed_on}}</td>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -1
index d31ecbb..d1259e9 100644
@@ -19,7 +19,7 @@
 
             {% for package in packages %}
 
-                <tr>
+                <tr class="data">
                 <td><a name="#{{package.name}}">{{package.name}}</a></td>
                 <td>{{package.version}}</td>
                 <td>{{package.size}}</td>
Signature · 1 line
1.8.1.2
#64 lore raw

PATCH 64/94 bitbake: webhob: refactor column hiding code

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We're refactoring the column show/hide code to get it more generic, and easier to apply it to all table pages.

---
 bitbake/lib/webhob/bldviewer/templates/build.html | 6 +++---
 bitbake/lib/webhob/bldviewer/views.py             | 6 +++++-
 2 files changed, 8 insertions(+), 4 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +3 -3
index aa92929..5db72db 100644
@@ -7,9 +7,9 @@
 <div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
 </div>
 <div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
-<span>Output: <input type="checkbox" id="ctoutput" onchange="showhideTableColumn(10, $('#ctoutput').is(':checked'))" checked autocomplete="off"></span> |
-<span>Log: <input type="checkbox" id="ctlog" onchange="showhideTableColumn(11, $('#ctlog').is(':checked'))"checked autocomplete="off"></span> |
-<span>Build Name: <input type="checkbox" id="ctbuildname" onchange="showhideTableColumn(13, $('#ctbuildname').is(':checked'))" checked autocomplete="off"></span> |
+{% for i in hideshowcols %}
+    <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
+{% endfor %}
 </div>
 </div>
     <table border="1">
bitbake/lib/webhob/bldviewer/views.py +5 -1
index ab69c91..2ef905a 100644
@@ -12,7 +12,11 @@ def build(request):
     build_info = Build.objects.all()
     logs = LogMessage.objects.all()
 
-    context = {'builds': build_info, 'logs': logs }
+    context = {'builds': build_info, 'logs': logs , 
+        'hideshowcols' : [
+                {'name': 'Output', 'order':10},
+                {'name': 'Log', 'order':11},
+            ]}
 
     return render(request, template, context)
 
Signature · 1 line
1.8.1.2
#65 lore raw

PATCH 65/94 bitbake: webhob: improve search functionality

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

we improve the search functionality to allow for
multiple search terms separated by space.

---
 bitbake/lib/webhob/bldviewer/templates/base.html | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +5 -2
index 45f8652..12b2f01 100644
@@ -22,8 +22,11 @@ function showhideTableColumn(i, sh) {
 
 function filterTableRows(test) {
     if (test.length > 0) {
-        var r = new RegExp(test);
-        $('tr.data').map( function (i, el) { if (! r.test($(el).html())) {$(el).hide()} else $(el).show()});
+
+        var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') });
+        $('tr.data').map( function (i, el) {
+            (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show();
+        });
     } else
     {
         $('tr.data').show();
Signature · 1 line
1.8.1.2
#66 lore raw

PATCH 66/94 bitbake: webhob: refactor Simple web interface

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We're refactoring the inheritance model and code
placement in the Simple interface as to bring the
search functionality and column selection to all
table-based pages.

---
 bitbake/lib/webhob/bldviewer/templates/base.html    | 15 ++++++++++++++-
 bitbake/lib/webhob/bldviewer/templates/build.html   | 17 +++++------------
 bitbake/lib/webhob/bldviewer/templates/layer.html   |  6 ++++--
 bitbake/lib/webhob/bldviewer/templates/package.html | 12 +++++++-----
 bitbake/lib/webhob/bldviewer/templates/recipe.html  |  6 ++++--
 bitbake/lib/webhob/bldviewer/templates/task.html    |  6 ++++--
 6 files changed, 38 insertions(+), 24 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +14 -1
index 12b2f01..0d24d36 100644
@@ -36,8 +36,21 @@ function filterTableRows(test) {
 
 <div>Menu: <a href="/simple/build/">All Builds</a>&nbsp;|&nbsp;<a href="/simple/layer/">All Layers</a>
 </div>
-{% block pagecontent %}
 
+{% block pagename %}
+{% endblock %}
+
+<div style="padding-top:1em ;padding-bottom: 1em">
+<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
+</div>
+<div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
+{% for i in hideshowcols %}
+    <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
+{% endfor %}
+</div>
+</div>
+
+{% block pagetable %}
 {% endblock %}
 </body>
 </html>
bitbake/lib/webhob/bldviewer/templates/build.html +5 -12
index 5db72db..2828404 100644
@@ -1,17 +1,10 @@
 {% extends "base.html" %}
 
-{% block pagecontent %}
-    <h1>WebHob Builds</h1>
-
-<div style="padding-top:1em ;padding-bottom: 1em">
-<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
-</div>
-<div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
-{% for i in hideshowcols %}
-    <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
-{% endfor %}
-</div>
-</div>
+{% block pagename %}
+    <h1>Toaster - Builds</h1>
+{% endblock %}
+
+{% block pagetable %}
     <table border="1">
     {% load projecttags %}
 
bitbake/lib/webhob/bldviewer/templates/layer.html +4 -2
index 9d44f17..0ffdaba 100644
@@ -1,8 +1,10 @@
 {% extends "base.html" %}
 
-{% block pagecontent %}
-    <h1>WebHob Layers</h1>
+{% block pagename %}
+    <h1>Toaster - Layers</h1>
+{% endblock %}
 
+{% block pagetable %}
     <table border="1">
 
     {% load projecttags %}
bitbake/lib/webhob/bldviewer/templates/package.html +7 -5
index d1259e9..e7856ba 100644
@@ -1,8 +1,10 @@
 {% extends "base.html" %}
 
-{% block pagecontent %}
-    <h1>WebHob Packages</h1>
+{% block pagename %}
+    <h1>Toaster - Packages</h1>
+{% endblock %}
 
+{% block pagetable %}
     {% if not packages %}
         <p>No packages were build in this build!</p>
     {% else %}
@@ -29,11 +31,11 @@
             <div style="height: 3em; overflow:auto">
                     {% for d in package.depends_on %}
                     <a href="#{{d.name}}">{{d.name}}</a><br/>
-                    {% endfor %} 
+                    {% endfor %}
             </div>
                 </td>
-            
-            {% endfor %}        
+
+            {% endfor %}
 
         </table>
 
bitbake/lib/webhob/bldviewer/templates/recipe.html +4 -2
index 5ebd880..0c9c077 100644
@@ -1,8 +1,10 @@
 {% extends "base.html" %}
 
-{% block pagecontent %}
-    <h1>WebHob Recipes for a Layer</h1>
+{% block pagename %}
+    <h1>Toaster - Recipes for a Layer</h1>
+{% endblock %}
 
+{% block pagetable %}
     <table border="1">
 
     {% load projecttags %}
bitbake/lib/webhob/bldviewer/templates/task.html +4 -2
index d87d261..0ed5f9d 100644
@@ -1,8 +1,10 @@
 {% extends "base.html" %}
 
-{% block pagecontent %}
-    <h1>WebHob Tasks</h1>
+{% block pagename %}
+    <h1>Toaster - Tasks</h1>
+{% endblock %}
 
+{% block pagetable %}
     {% if not tasks %}
         <p>No tasks were executed in this build!</p>
     {% else %}
Signature · 1 line
1.8.1.2
#67 lore raw

PATCH 67/94 bitbake: webhob: simple interface CSS

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adds Bootstrap CSS features to the Simple interface, as to make it more visually appealing.

---
 bitbake/lib/webhob/bldviewer/templates/base.html   | 44 +++++++++++++---------
 bitbake/lib/webhob/bldviewer/templates/build.html  |  7 +---
 bitbake/lib/webhob/bldviewer/templates/layer.html  |  6 +--
 .../lib/webhob/bldviewer/templates/package.html    |  6 +--
 bitbake/lib/webhob/bldviewer/templates/recipe.html |  6 +--
 bitbake/lib/webhob/bldviewer/templates/task.html   |  6 +--
 6 files changed, 31 insertions(+), 44 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +26 -18
index 0d24d36..e97c8f6 100644
@@ -10,7 +10,7 @@
 <link href="/static/css/bootstrap.css" rel="stylesheet" type="text/css">
 	</head>
 
-<body>
+<body style="height: 100%">
 <script>
 function showhideTableColumn(i, sh) {
     if (sh)
@@ -22,7 +22,6 @@ function showhideTableColumn(i, sh) {
 
 function filterTableRows(test) {
     if (test.length > 0) {
-
         var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') });
         $('tr.data').map( function (i, el) {
             (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show();
@@ -33,25 +32,34 @@ function filterTableRows(test) {
     }
 }
 </script>
-
-<div>Menu: <a href="/simple/build/">All Builds</a>&nbsp;|&nbsp;<a href="/simple/layer/">All Layers</a>
-</div>
-
-{% block pagename %}
-{% endblock %}
-
-<div style="padding-top:1em ;padding-bottom: 1em">
-<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
-</div>
-<div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
-{% for i in hideshowcols %}
-    <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
-{% endfor %}
-</div>
+<div style="position:fixed; top:0; z-index: 999; width: 100%; background:white"> 
+    <ul class="nav nav-tabs">
+        <li><a href="/simple/build/">All Builds</a></li>
+        <li><a href="/simple/layer/">All Layers</a></li>
+    </ul>
+    
+    {% block pagename %}
+    {% endblock %}
+    
+    <div style="padding-top:1em ;padding-bottom: 1em">
+    <div align="left" style="display:inline-block; width: 49%; margin-left: 2em"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
+    </div>
+    {% if hideshowcols %}
+    <div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
+    {% for i in hideshowcols %}
+        <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
+    {% endfor %}
+    </div>
+    {% endif %}
+    </div>
 </div>
-
+<div style="padding-top: 13em; width:98%; align:center; margin:auto;">
+    <table class="table table-striped table-condensed">
 {% block pagetable %}
 {% endblock %}
+    </table>
+</div>
+<div class="navbar"><br/>About Toaster | Yocto Project </div>
 </body>
 </html>
 
bitbake/lib/webhob/bldviewer/templates/build.html +1 -6
index 2828404..20c415b 100644
@@ -5,9 +5,8 @@
 {% endblock %}
 
 {% block pagetable %}
-    <table border="1">
-    {% load projecttags %}
 
+    {% load projecttags %}
         <tr>
             <th>Outcome</th>
             <th>Started On</th>
@@ -23,9 +22,7 @@
             <th>Bitbake Version</th>
             <th>Build Name</th>
         </tr>
-
         {% for build in builds %}
-
         <tr class="data">
             <td><a href="/simple/build/{{build.id}}/task/">{{build.get_outcome_display}}</a></td>
             <td>{{build.started_on}}</td>
@@ -43,8 +40,6 @@
         </tr>
 
         {% endfor %}
-
-    </table>
 {% endblock %}
 
 
bitbake/lib/webhob/bldviewer/templates/layer.html +1 -5
index 0ffdaba..5b317ba 100644
@@ -5,8 +5,6 @@
 {% endblock %}
 
 {% block pagetable %}
-    <table border="1">
-
     {% load projecttags %}
 
         <tr>
@@ -18,7 +16,7 @@
 
         {% for layer in layers %}
 
-        <tr>
+        <tr class="data">
             <td>{{layer.name}}</td>
             <td>{{layer.local_path}}</td>
             <td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td>
@@ -33,6 +31,4 @@
 
         {% endfor %}
 
-    </table>
-
 {% endblock %}
bitbake/lib/webhob/bldviewer/templates/package.html +1 -5
index e7856ba..f70b022 100644
@@ -9,8 +9,6 @@
         <p>No packages were build in this build!</p>
     {% else %}
 
-        <table border="1">
-
             <tr>
             <th>Name</th>
             <th>Version</th>
@@ -21,7 +19,7 @@
 
             {% for package in packages %}
 
-                <tr class="data">
+            <tr class="data">
                 <td><a name="#{{package.name}}">{{package.name}}</a></td>
                 <td>{{package.version}}</td>
                 <td>{{package.size}}</td>
@@ -37,8 +35,6 @@
 
             {% endfor %}
 
-        </table>
-
     {% endif %}
 
 {% endblock %}
bitbake/lib/webhob/bldviewer/templates/recipe.html +1 -5
index 0c9c077..515053a 100644
@@ -5,8 +5,6 @@
 {% endblock %}
 
 {% block pagetable %}
-    <table border="1">
-
     {% load projecttags %}
 
         <tr>
@@ -24,7 +22,7 @@
 
         {% for recipe in recipes %}
 
-        <tr>
+        <tr class="data">
             <td><a name="{{recipe.name}}">{{recipe.name}}</a></td>
             <td>{{recipe.version}}</td>
             <td>{{recipe.summary}}</td>
@@ -39,6 +37,4 @@
 
         {% endfor %}
 
-    </table>
-
 {% endblock %}
bitbake/lib/webhob/bldviewer/templates/task.html +1 -5
index 0ed5f9d..6d06d0f 100644
@@ -9,8 +9,6 @@
         <p>No tasks were executed in this build!</p>
     {% else %}
 
-        <table border="1">
-
             <tr>
             <th>Order</th>
             <th>Task</th>
@@ -29,7 +27,7 @@
 
             {% for task in tasks %}
 
-                <tr>
+            <tr class="data">
                 <td>{{task.order}}</td>
                 <td><a name="{{task.recipe.name}}.{{task.task_name}}">
                 <a href="/simple/layerversions/{{task.recipe.layer_version_id}}/recipes/#{{task.recipe.name}}">{{task.recipe.name}}</a>.{{task.task_name}}</a></td>
@@ -61,8 +59,6 @@
 
             {% endfor %}
 
-        </table>
-
     {% endif %}
 
 {% endblock %}
Signature · 1 line
1.8.1.2
#68 lore raw

PATCH 68/94 bitbake: dsi: use get vars command to store configuration

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Patch to store variable configuration for a build using the getAllKeysWithFlags command. Only variables that are not functions are saved.

Minor fixes to the Variable model as to match data we receive.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 15 +++++++++++++--
 bitbake/lib/webhob/orm/models.py     |  6 +++---
 2 files changed, 16 insertions(+), 5 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +13 -2
index 6f4789c..89d5f15 100644
@@ -9,7 +9,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
 from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, LogMessage
-from webhob.orm.models import Task_Dependency, Package_Dependency
+from webhob.orm.models import Task_Dependency, Package_Dependency, Variable
 from bb.msg import BBLogFormatter as format
 
 class ORMWrapper(object):
@@ -146,6 +146,15 @@ class ORMWrapper(object):
 
         return log_object.save()
 
+    def save_build_variables(self, build_obj, vardump):
+        for k in vardump:
+            if not bool(vardump[k]['func']):
+                Variable.objects.create( build = build_obj,
+                    variable_name = k,
+                    variable_value = vardump[k]['v'],
+                    description = vardump[k]['doc'])
+
+
 class BuildInfoHelper(object):
     """ This class gathers the build information from the server and sends it
         towards the ORM wrapper for storing in the database
@@ -363,13 +372,15 @@ class BuildInfoHelper(object):
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.internal_state['build'] = build_obj
         self.internal_state['target'] = build_information['target']
+        # Load layer information for the build
         self.internal_state['layer_versions'] = []
         for layer_object in self.internal_state['layers']:
             layer_version_information = self._get_layer_version_information(layer_object)
             self.internal_state['layer_versions'].append(self.orm_wrapper.get_layer_version_object(layer_version_information))
 
         del self.internal_state['layers']
-
+        # Save build configuration
+        self.orm_wrapper.save_build_variables(build_obj, self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0])
 
 
     def update_build_information(self, event, errors, warnings, taskfailures):
bitbake/lib/webhob/orm/models.py +3 -3
index f51f53d..1d67e40 100644
@@ -161,11 +161,11 @@ class Layer_Version(models.Model):
 class Variable(models.Model):
     build = models.ForeignKey(Build, related_name='variable_build')
     variable_name = models.CharField(max_length=100)
-    variable_value = models.TextField()
+    variable_value = models.TextField(null=True)
     file = models.FilePathField(max_length=255)
-    changed = models.BooleanField()
+    changed = models.BooleanField(default=False)
     human_readable_name = models.CharField(max_length=200)
-    description = models.TextField()
+    description = models.TextField(null=True)
 
 
 class Machine(models.Model):
Signature · 1 line
1.8.1.2
#69 lore raw

PATCH 69/94 bitbake: webhob: refactor CSS display in Simple

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Refactoring CSS display in Simple to separate
the generic layout to the table display code.

With the new layout we can add the generic display to more than just table pages.

---
 bitbake/lib/webhob/bldviewer/templates/base.html   | 51 ++++------------------
 .../lib/webhob/bldviewer/templates/basetable.html  | 46 +++++++++++++++++++
 bitbake/lib/webhob/bldviewer/templates/build.html  |  2 +-
 bitbake/lib/webhob/bldviewer/templates/layer.html  |  2 +-
 .../lib/webhob/bldviewer/templates/package.html    |  2 +-
 bitbake/lib/webhob/bldviewer/templates/recipe.html |  2 +-
 bitbake/lib/webhob/bldviewer/templates/task.html   |  2 +-
 7 files changed, 59 insertions(+), 48 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/basetable.html
bitbake/lib/webhob/bldviewer/templates/base.html +8 -43
index e97c8f6..31abeb1 100644
@@ -2,7 +2,7 @@
 
 <html>
 	<head>
-		<title>WebHob Simple Explorer</title>
+		<title>Toaster Simple Explorer</title>
 <script src="/static/js/jquery-2.0.3.js">
 </script>
 <script src="/static/js/bootstrap.js">
@@ -11,55 +11,20 @@
 	</head>
 
 <body style="height: 100%">
-<script>
-function showhideTableColumn(i, sh) {
-    if (sh)
-        $('td:nth-child('+i+'),th:nth-child('+i+')').show();
-    else
-        $('td:nth-child('+i+'),th:nth-child('+i+')').hide();
-}
-
-
-function filterTableRows(test) {
-    if (test.length > 0) {
-        var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') });
-        $('tr.data').map( function (i, el) {
-            (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show();
-        });
-    } else
-    {
-        $('tr.data').show();
-    }
-}
-</script>
-<div style="position:fixed; top:0; z-index: 999; width: 100%; background:white"> 
+<div style="width:100%; height: 100%; position:absolute">
+<div style="width: 100%; height: 3em" class="nav">
     <ul class="nav nav-tabs">
         <li><a href="/simple/build/">All Builds</a></li>
         <li><a href="/simple/layer/">All Layers</a></li>
     </ul>
-    
-    {% block pagename %}
-    {% endblock %}
-    
-    <div style="padding-top:1em ;padding-bottom: 1em">
-    <div align="left" style="display:inline-block; width: 49%; margin-left: 2em"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
-    </div>
-    {% if hideshowcols %}
-    <div align="right" style="display: inline-block; width: 49%">Show/Hide columns:
-    {% for i in hideshowcols %}
-        <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
-    {% endfor %}
-    </div>
-    {% endif %}
-    </div>
 </div>
-<div style="padding-top: 13em; width:98%; align:center; margin:auto;">
-    <table class="table table-striped table-condensed">
-{% block pagetable %}
+
+<div style="overflow-y:scroll; width: 100%; position: absolute; top: 3em; bottom:70px ">
+{% block pagecontent %}
 {% endblock %}
-    </table>
 </div>
-<div class="navbar"><br/>About Toaster | Yocto Project </div>
+<div class="navbar" style="position: absolute; bottom: 0; width:100%"><br/>About Toaster | Yocto Project </div>
+</div>
 </body>
 </html>
 
bitbake/lib/webhob/bldviewer/templates/basetable.html new file +46 -0

46 lines not on this page · raw message

bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
index 20c415b..53271a7 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends "basetable.html" %}
 
 {% block pagename %}
     <h1>Toaster - Builds</h1>
bitbake/lib/webhob/bldviewer/templates/layer.html +1 -1
index 5b317ba..508f254 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends "basetable.html" %}
 
 {% block pagename %}
     <h1>Toaster - Layers</h1>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -1
index f70b022..622c23d 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends "basetable.html" %}
 
 {% block pagename %}
     <h1>Toaster - Packages</h1>
bitbake/lib/webhob/bldviewer/templates/recipe.html +1 -1
index 515053a..e855625 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends "basetable.html" %}
 
 {% block pagename %}
     <h1>Toaster - Recipes for a Layer</h1>
bitbake/lib/webhob/bldviewer/templates/task.html +1 -1
index 6d06d0f..05d8e12 100644
@@ -1,4 +1,4 @@
-{% extends "base.html" %}
+{% extends "basetable.html" %}
 
 {% block pagename %}
     <h1>Toaster - Tasks</h1>
Signature · 1 line
1.8.1.2
#70 lore raw

PATCH 70/94 bitbake: webhob: add Configuration visualisation in Simple

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We add a page to show the build configuration variables in the Simple display.

We add a bit of navigation menu to allow moving between pages of the same build.

---
 .../webhob/bldviewer/templates/configuration.html  | 28 ++++++++++++++++++++++
 .../lib/webhob/bldviewer/templates/package.html    |  7 +++++-
 bitbake/lib/webhob/bldviewer/templates/task.html   |  5 ++++
 bitbake/lib/webhob/bldviewer/urls.py               |  1 +
 bitbake/lib/webhob/bldviewer/views.py              | 12 +++++++---
 5 files changed, 49 insertions(+), 4 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/configuration.html
bitbake/lib/webhob/bldviewer/templates/configuration.html new file +28 -0
new file mode 100644
index 0000000..2f197c0
@@ -0,0 +1,28 @@
+{% extends "basetable.html" %}
+
+{% block pagename %}
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
+  <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
+  <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
+</ul>
+     <h1>Toaster - Build Configuration </h1>
+{% endblock %}
+
+{% block pagetable %}
+
+            <tr>
+            <th>Name</th>
+            <th>Value</th>
+            <th>Description</th>
+            </tr>
+
+            {% for variable in configuration %}
+
+            <tr class="data">
+                <td>{{variable.variable_name}}</td>
+                <td>{{variable.variable_value}}</td>
+                <td>{% if variable.description %}{{variable.description}}{% endif %}</td>
+            {% endfor %}
+
+{% endblock %}
bitbake/lib/webhob/bldviewer/templates/package.html +6 -1
index 622c23d..339b411 100644
@@ -1,7 +1,12 @@
 {% extends "basetable.html" %}
 
 {% block pagename %}
-    <h1>Toaster - Packages</h1>
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
+  <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
+  <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
+</ul>
+     <h1>Toaster - Packages</h1>
 {% endblock %}
 
 {% block pagetable %}
bitbake/lib/webhob/bldviewer/templates/task.html +5 -0
index 05d8e12..4939531 100644
@@ -1,6 +1,11 @@
 {% extends "basetable.html" %}
 
 {% block pagename %}
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
+  <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
+  <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
+</ul>
     <h1>Toaster - Tasks</h1>
 {% endblock %}
 
bitbake/lib/webhob/bldviewer/urls.py +1 -0
index 8b8b5c3..8f9bc0d 100644
@@ -5,6 +5,7 @@ urlpatterns = patterns('bldviewer.views',
         url(r'^build/$', 'build', name='build'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
         url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'),
+        url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'),
         url(r'^layer/$', 'layer', name='layer'),
         url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'),
         url(r'^$', redirect_to, {'url': 'build/'}),
bitbake/lib/webhob/bldviewer/views.py +9 -3
index 2ef905a..77be08b 100644
@@ -2,7 +2,7 @@ import operator
 
 from django.db.models import Q
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage
+from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage, Variable
 from orm.models import Task_Dependency, Package_Dependency
 from django.views.decorators.cache import cache_control
 
@@ -32,10 +32,16 @@ def task(request, build_id):
             if t == k.task:
                 t.depends_on.append(k.depends_on)
 
-    context = {'tasks': tasks}
+    context = {'build': build_id, 'tasks': tasks}
 
     return render(request, template, context)
 
+def configuration(request, build_id):
+    template = 'configuration.html'
+    variables = Variable.objects.filter(build=build_id)
+    context = {'build': build_id, 'configuration' : variables}
+    return render(request, template, context)
+
 def package(request, build_id):
     template = 'package.html'
 
@@ -48,7 +54,7 @@ def package(request, build_id):
             if t == k.package:
                 t.depends_on.append(k.depends_on)
 
-    context = {'packages': packages}
+    context = {'build' : build_id ,'packages': packages}
 
     return render(request, template, context)
 
Signature · 1 line
1.8.1.2
#71 lore raw

PATCH 71/94 bitbake: webhob: add navigation links Simple interface

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding a couple of navigation links on top of pages so that we provide easier movement between pages and a bit more context.

---
 bitbake/lib/webhob/bldviewer/templates/build.html         | 2 +-
 bitbake/lib/webhob/bldviewer/templates/configuration.html | 1 +
 bitbake/lib/webhob/bldviewer/templates/package.html       | 1 +
 bitbake/lib/webhob/bldviewer/templates/recipe.html        | 3 +++
 bitbake/lib/webhob/bldviewer/templates/task.html          | 1 +
 bitbake/lib/webhob/bldviewer/views.py                     | 6 ++++--
 6 files changed, 11 insertions(+), 3 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
index 53271a7..bbd283e 100644
@@ -24,7 +24,7 @@
         </tr>
         {% for build in builds %}
         <tr class="data">
-            <td><a href="/simple/build/{{build.id}}/task/">{{build.get_outcome_display}}</a></td>
+            <td><a href="/simple/build/{{build.id}}/configuration/">{{build.get_outcome_display}}</a></td>
             <td>{{build.started_on}}</td>
             <td>{{build.completed_on}}</td>
             <td>{{build.target}}</td>
bitbake/lib/webhob/bldviewer/templates/configuration.html +1 -0
index 2f197c0..4db4765 100644
@@ -2,6 +2,7 @@
 
 {% block pagename %}
 <ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Build {{build}} : </a></li>
   <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
   <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
   <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -0
index 339b411..b4091c6 100644
@@ -2,6 +2,7 @@
 
 {% block pagename %}
 <ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Build {{build}} : </a></li>
   <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
   <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
   <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
bitbake/lib/webhob/bldviewer/templates/recipe.html +3 -0
index e855625..32fcdb8 100644
@@ -1,6 +1,9 @@
 {% extends "basetable.html" %}
 
 {% block pagename %}
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Layer  {{layer_version.layer.name}}&nbsp;:&nbsp;{{layer_version.branch}}&nbsp;:&nbsp;{{layer_version.commit}}</a></li>
+</ul>
     <h1>Toaster - Recipes for a Layer</h1>
 {% endblock %}
 
bitbake/lib/webhob/bldviewer/templates/task.html +1 -0
index 4939531..9736b4a 100644
@@ -2,6 +2,7 @@
 
 {% block pagename %}
 <ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Build {{build}} : </a></li>
   <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
   <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
   <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
bitbake/lib/webhob/bldviewer/views.py +4 -2
index 77be08b..cb7cfc7 100644
@@ -12,7 +12,7 @@ def build(request):
     build_info = Build.objects.all()
     logs = LogMessage.objects.all()
 
-    context = {'builds': build_info, 'logs': logs , 
+    context = {'builds': build_info, 'logs': logs ,
         'hideshowcols' : [
                 {'name': 'Output', 'order':10},
                 {'name': 'Log', 'order':11},
@@ -76,7 +76,9 @@ def layer_versions_recipes(request, layerversion_id):
     template = 'recipe.html'
     recipes = Recipe.objects.filter(layer_version__id = layerversion_id)
 
-    context = {'recipes': recipes}
+    context = {'recipes': recipes,
+            'layer_version' : Layer_Version.objects.filter( id = layerversion_id )[0]
+    }
 
     return render(request, template, context)
 
Signature · 1 line
1.8.1.2
#72 lore raw

PATCH 72/94 bitbake: webhob: startup script fixing

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Let's not start the webserver if the database sync went wrong.

---
 bitbake/bin/webhob | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
bitbake/bin/webhob +2 -1
index aa35294..0c4d9ea 100755
@@ -43,8 +43,9 @@ function webserverStartAll()
         python $BBBASEDIR/lib/webhob/manage.py syncdb || retval=1
         if [ $retval -eq 1 ]; then
                 echo "Failed db sync, stopping system start" 1>&2
+        else
+            python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
         fi
-        python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
         return $retval
 }
 
Signature · 1 line
1.8.1.2
#73 lore raw

PATCH 73/94 bitbake: webhob: change database models and related

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We change the database models as to allow storage of data based on new Package design:

* there is a separate Target model that will hold each of the targets of a build.
* the package information from an image file is stored in Target_Package model
* the package information from the build process is stored in Build_Package model
* there is a provision to store dependency type between Packages
* the code in the DSI has been altered to cope with the new separate Targets for each build
* the code in Simple interface has been changed to provision for the new model code.

---
 bitbake/lib/bb/ui/buildinfohelper.py               | 153 +++++++++++----------
 bitbake/lib/bb/ui/dsi.py                           |   5 +-
 bitbake/lib/webhob/bldviewer/templates/build.html  |   2 +-
 .../lib/webhob/bldviewer/templates/package.html    |   4 +-
 bitbake/lib/webhob/bldviewer/urls.py               |   1 +
 bitbake/lib/webhob/bldviewer/views.py              |  11 +-
 bitbake/lib/webhob/orm/models.py                   |  71 +++++++---
 7 files changed, 151 insertions(+), 96 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +84 -69
index 89d5f15..0860d2a 100644
@@ -1,6 +1,5 @@
 import datetime
 import sys
-import uuid
 import bb
 import re
 
@@ -8,8 +7,9 @@ import re
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, LogMessage
-from webhob.orm.models import Task_Dependency, Package_Dependency, Variable
+from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage
+from webhob.orm.models import Target_Package, Build_Package, Variable
+from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency
 from bb.msg import BBLogFormatter as format
 
 class ORMWrapper(object):
@@ -30,20 +30,30 @@ class ORMWrapper(object):
 
     def create_build_object(self, build_info):
 
-        build = Build.objects.create(uuid=build_info['uuid'],
-                                    target=build_info['target'],
+        build = Build.objects.create(
                                     machine=build_info['machine'],
                                     distro=build_info['distro'],
                                     distro_version=build_info['distro_version'],
                                     started_on=build_info['started_on'],
                                     completed_on=build_info['completed_on'],
-                                    image_fstypes=build_info['image_fstypes'],
                                     cooker_log_path=build_info['cooker_log_path'],
                                     build_name=build_info['build_name'],
                                     bitbake_version=build_info['bitbake_version'])
 
         return build
 
+    def create_target_objects(self, target_info):
+        targets = []
+        for tgt_name in target_info['targets']:
+            tgt_object = Target.objects.create( build = target_info['build'],
+                                    target = tgt_name,
+                                    is_image = False,
+                                    image_fstypes = "",
+                                    file_name = "",
+                                    file_size = 0);
+            targets.append(tgt_object)
+        return targets
+
     def update_build_object(self, build, errors, warnings, taskfailures):
 
         outcome = Build.SUCCEEDED
@@ -118,9 +128,9 @@ class ORMWrapper(object):
         return layer_object[0]
 
 
-    def save_package_information(self, build_obj, packagedict, bldpkgs, recipes):
+    def save_target_package_information(self, target_obj, packagedict, bldpkgs, recipes):
         for p in packagedict:
-            packagedict[p]['object'] = Package.objects.create( build = build_obj,
+            packagedict[p]['object'] = Target_Package.objects.create( target = target_obj,
                                         name = p,
                                         size = packagedict[p]['size'])
             if p in bldpkgs:
@@ -130,7 +140,7 @@ class ORMWrapper(object):
 
         for p in packagedict:
             for px in packagedict[p]['depends']:
-                Package_Dependency.objects.create( package = packagedict[p]['object'],
+                Target_Package_Dependency.objects.create( package = packagedict[p]['object'],
                                         depends_on = packagedict[px]['object'] );
 
 
@@ -165,7 +175,6 @@ class BuildInfoHelper(object):
     def __init__(self, server, has_build_history = False):
         self._configure_django()
         self.internal_state = {}
-        self.uuid = None
         self.task_order = 0
         self.server = server
         self.orm_wrapper = ORMWrapper()
@@ -252,7 +261,6 @@ class BuildInfoHelper(object):
         build_info = {}
         # Generate an identifier for each new build
 
-        build_info['uuid'] = self.uuid
         build_info['machine'] = machine_obj
         build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0]
         build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
@@ -307,15 +315,17 @@ class BuildInfoHelper(object):
 
     def _get_path_information(self, task_object):
         build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/"
+        build_stats_path = []
 
-        target = self.internal_state['target']
-        machine = self.internal_state['build'].machine.name
-        buildname = self.internal_state['build'].build_name
-        package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":")
+        for t in self.internal_state['targets']:
+            target = t.target
+            machine = self.internal_state['build'].machine.name
+            buildname = self.internal_state['build'].build_name
+            package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":")
 
-        build_stats_path = build_stats_format.format(tmpdir=self.tmp_dir, target=target,
+            build_stats_path.append(build_stats_format.format(tmpdir=self.tmp_dir, target=target,
                                                      machine=machine, buildname=buildname,
-                                                     package=package)
+                                                     package=package))
 
         return build_stats_path
 
@@ -364,14 +374,19 @@ class BuildInfoHelper(object):
 
         machine_information = self._get_machine_information()
         machine_obj = self.orm_wrapper.create_machine_object(machine_information)
-        self.uuid = str(uuid.uuid4())
 
         build_information = self._get_build_information(machine_obj)
-        build_information['target'] = ' '.join(event.getPkgs())
 
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.internal_state['build'] = build_obj
-        self.internal_state['target'] = build_information['target']
+
+        # create target information
+        target_information = {}
+        target_information['targets'] = event.getPkgs()
+        target_information['build'] = build_obj
+
+        self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
+
         # Load layer information for the build
         self.internal_state['layer_versions'] = []
         for layer_object in self.internal_state['layers']:
@@ -428,7 +443,7 @@ class BuildInfoHelper(object):
             task_information['message'] = event._message
 
         if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)):
-            task_information['outcome'] = Task.OUTCOME_SUCCESS     # TODO: needs to use constants
+            task_information['outcome'] = Task.OUTCOME_SUCCESS
             task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
             task_information['cpu_usage'] = task_build_stats['cpu_usage']
             task_information['disk_io'] = task_build_stats['disk_io']
@@ -438,54 +453,52 @@ class BuildInfoHelper(object):
             task_information['outcome'] = Task.OUTCOME_FAILED
             del self.internal_state[identifier]
 
-        #TODO: get error number
-        #TODO: get warnings number
-        #TODO: get warning information
-
-
         self.orm_wrapper.get_update_task_object(task_information)
 
 
-    def read_package_dep_data(self, event):
-        # verify that we have something to read
-        if not self.internal_state['build'].is_image or not self.has_build_history:
-            print "not collecting package info ", self.internal_state['build'].is_image, self.has_build_history
-            return
-
-        # fair warning: code correlates to buildhistory.bbclass; anything changes there, needs to chage here too
-        TOPDIR, error = self.server.runCommand(['getVariable', 'TOPDIR'])
-        MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH'])
-        TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC'])
-        MULTIMACH_TARGET_SYS, error = self.server.runCommand(['getVariable', 'MULTIMACH_TARGET_SYS'])
-        SDK_NAME, error = self.server.runCommand(['getVariable', 'SDK_NAME'])
-        BUILDHISTORY_DIR = "%s/buildhistory" % TOPDIR
-        BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, self.internal_state['build'].target)
-
-        self.internal_state['packages'] = {}
-
-        with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
-            for line in fin:
-                line = line.rstrip(";")
-                psize, px = line.split("\t")
-                punit, pname = px.split(" ")
-                self.internal_state['packages'][pname.strip()] = {'size':psize, 'depends' : []}
-
-        with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
-            p = re.compile(r' -> ')
-            for line in fin:
-                line = line.rstrip(';')
-                linesplit = p.split(line)
-                if len(linesplit) == 2:
-                    pname = linesplit[0]
-                    dependsname = linesplit[1].split(" ")[0].strip().strip(";")
-                    if not pname in self.internal_state['packages']:
-                        self.internal_state['packages'][pname] = {'size': 0, 'depends' : []}
-                    if not dependsname in self.internal_state['packages']:
-                        self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []}
-                    self.internal_state['packages'][pname]['depends'].append(dependsname)
-
-        self.orm_wrapper.save_package_information(self.internal_state['build'], self.internal_state['packages'],
-                    self.internal_state['bldpkgs'], self.internal_state['recipes'])
+    def read_target_package_dep_data(self, event):
+        # for all targets
+        for target in self.internal_state['targets']:
+            # verify that we have something to read
+            if not target.is_image or not self.has_build_history:
+                print "not collecting package info ", target.is_image, self.has_build_history
+                break
+
+            # fair warning: code correlates to buildhistory.bbclass; anything changes there, needs to chage here too
+            TOPDIR, error = self.server.runCommand(['getVariable', 'TOPDIR'])
+            MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH'])
+            TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC'])
+            MULTIMACH_TARGET_SYS, error = self.server.runCommand(['getVariable', 'MULTIMACH_TARGET_SYS'])
+            SDK_NAME, error = self.server.runCommand(['getVariable', 'SDK_NAME'])
+            BUILDHISTORY_DIR = "%s/buildhistory" % TOPDIR
+            BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, target.target)
+
+            self.internal_state['packages'] = {}
+
+            with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
+                for line in fin:
+                    line = line.rstrip(";")
+                    psize, px = line.split("\t")
+                    punit, pname = px.split(" ")
+                    self.internal_state['packages'][pname.strip()] = {'size':psize, 'depends' : []}
+
+            with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
+                p = re.compile(r' -> ')
+                for line in fin:
+                    line = line.rstrip(';')
+                    linesplit = p.split(line)
+                    if len(linesplit) == 2:
+                        pname = linesplit[0]
+                        dependsname = linesplit[1].split(" ")[0].strip().strip(";")
+                        if not pname in self.internal_state['packages']:
+                            self.internal_state['packages'][pname] = {'size': 0, 'depends' : []}
+                        if not dependsname in self.internal_state['packages']:
+                            self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []}
+                        self.internal_state['packages'][pname]['depends'].append(dependsname)
+
+            self.orm_wrapper.save_target_package_information(target,
+                        self.internal_state['packages'],
+                        self.internal_state['bldpkgs'], self.internal_state['recipes'])
 
 
     def store_dependency_information(self, event):
@@ -519,9 +532,11 @@ class BuildInfoHelper(object):
             recipe_info['file_path'] = file_name
             recipe = self.orm_wrapper.get_update_recipe_object(recipe_info)
             recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits'])
-            if recipe.is_image and pn == self.internal_state['build'].target:
-                self.internal_state['build'].is_image = True
-                self.internal_state['build'].save()
+            if recipe.is_image:
+                for t in self.internal_state['targets']:
+                    if pn == t.target:
+                        t.is_image = True
+                        t.save()
             self.internal_state['recipes'][pn] = recipe
 
         # save all task information
bitbake/lib/bb/ui/dsi.py +4 -1
index c139cd2..3284ea2 100644
@@ -471,7 +471,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 continue
 
             if isinstance(event, (bb.event.BuildCompleted)):
-                buildinfohelper.read_package_dep_data(event)
+                buildinfohelper.read_target_package_dep_data(event)
                 buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
                 continue
 
@@ -531,6 +531,9 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                     logger.error("Unable to cleanly shutdown: %s" % error)
             main.shutdown = main.shutdown + 1
             pass
+        except Exception as e:
+            print(e)
+            pass
 
     summary = ""
     if taskfailures:
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
index bbd283e..eed81a3 100644
@@ -27,7 +27,7 @@
             <td><a href="/simple/build/{{build.id}}/configuration/">{{build.get_outcome_display}}</a></td>
             <td>{{build.started_on}}</td>
             <td>{{build.completed_on}}</td>
-            <td>{{build.target}}</td>
+            <td>{% for t in build.target_set.all %}<a href="/simple/build/{{build.id}}/target/{{t.id}}">{{t.target}}</a><br/>{% endfor %}</td>
             <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td>
             <td>{{build.machine.name}}</td>
             <td>{% time_difference build.started_on build.completed_on %}</td>
bitbake/lib/webhob/bldviewer/templates/package.html +2 -2
index b4091c6..7764f05 100644
@@ -7,12 +7,12 @@
   <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
   <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
 </ul>
-     <h1>Toaster - Packages</h1>
+     <h1>Toaster - Target Install Packages</h1>
 {% endblock %}
 
 {% block pagetable %}
     {% if not packages %}
-        <p>No packages were build in this build!</p>
+        <p>No packages were recorded for this target!</p>
     {% else %}
 
             <tr>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
index 8f9bc0d..11bdbe9 100644
@@ -4,6 +4,7 @@ from django.views.generic.simple import redirect_to
 urlpatterns = patterns('bldviewer.views',
         url(r'^build/$', 'build', name='build'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
+        url(r'^build/(?P<build_id>\d+)/targets/$', 'task', name='task'),
         url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'),
         url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'),
         url(r'^layer/$', 'layer', name='layer'),
bitbake/lib/webhob/bldviewer/views.py +6 -5
index cb7cfc7..ab222ac 100644
@@ -2,14 +2,15 @@ import operator
 
 from django.db.models import Q
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage, Variable
-from orm.models import Task_Dependency, Package_Dependency
+from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable
+from orm.models import Task_Dependency, Target_Package_Dependency
 from django.views.decorators.cache import cache_control
 
 @cache_control(no_store=True)
 def build(request):
     template = 'build.html'
     build_info = Build.objects.all()
+
     logs = LogMessage.objects.all()
 
     context = {'builds': build_info, 'logs': logs ,
@@ -45,8 +46,8 @@ def configuration(request, build_id):
 def package(request, build_id):
     template = 'package.html'
 
-    packages = Package.objects.filter(build=build_id)
-    package_depends = Package_Dependency.objects.filter(package__in=packages)
+    packages = Target_Package.objects.filter(build=build_id)
+    package_depends = Target_Package_Dependency.objects.filter(package__in=packages)
 
     for t in packages:
         t.depends_on = []
@@ -96,7 +97,7 @@ def model_explorer(request, model_name):
     model_mapping = {
         'build': Build,
         'task': Task,
-        'package': Package,
+        'package': Target_Package,
         'layer': Layer,
         'layerversion': Layer_Version,
         'recipe': Recipe,
bitbake/lib/webhob/orm/models.py +53 -18

104 lines not on this page · raw message

Signature · 1 line
1.8.1.2
#74 lore raw

PATCH 74/94 bitbake: webhob: navigation in the Simple interface

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Modifies the navigation in the Simple interface to allow new multiple targets per build model.

Refactoring the code to load relationships as to be based on Django's RelationshipManager automatic follower.

---
 bitbake/lib/webhob/bldviewer/templates/base.html   |  4 ++--
 .../webhob/bldviewer/templates/configuration.html  | 13 ++----------
 .../lib/webhob/bldviewer/templates/package.html    |  2 +-
 bitbake/lib/webhob/bldviewer/templates/task.html   | 19 +++++-------------
 bitbake/lib/webhob/bldviewer/urls.py               | 10 +++++-----
 bitbake/lib/webhob/bldviewer/views.py              | 23 +++++-----------------
 bitbake/lib/webhob/orm/models.py                   |  6 +++++-
 bitbake/lib/webhob/whbmain/urls.py                 |  2 +-
 8 files changed, 26 insertions(+), 53 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +2 -2
index 31abeb1..1a2278a 100644
@@ -14,8 +14,8 @@
 <div style="width:100%; height: 100%; position:absolute">
 <div style="width: 100%; height: 3em" class="nav">
     <ul class="nav nav-tabs">
-        <li><a href="/simple/build/">All Builds</a></li>
-        <li><a href="/simple/layer/">All Layers</a></li>
+        <li><a href="/simple/builds/">All Builds</a></li>
+        <li><a href="/simple/layers/">All Layers</a></li>
     </ul>
 </div>
 
bitbake/lib/webhob/bldviewer/templates/configuration.html +2 -11
index 4db4765..052c37c 100644
@@ -1,15 +1,6 @@
-{% extends "basetable.html" %}
-
-{% block pagename %}
-<ul class="nav nav-tabs" style="display: inline-block">
-  <li><a>Build {{build}} : </a></li>
-  <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
-  <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
-  <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
-</ul>
-     <h1>Toaster - Build Configuration </h1>
-{% endblock %}
+{% extends "basebuildpage.html" %}
 
+{% block pagetitle %}Configuration{% endblock %}
 {% block pagetable %}
 
             <tr>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -1
index 7764f05..cba21e1 100644
@@ -1,4 +1,4 @@
-{% extends "basetable.html" %}
+{% extends "basebuildpage.html" %}
 
 {% block pagename %}
 <ul class="nav nav-tabs" style="display: inline-block">
bitbake/lib/webhob/bldviewer/templates/task.html +5 -14
index 9736b4a..65d75cb 100644
@@ -1,15 +1,6 @@
-{% extends "basetable.html" %}
-
-{% block pagename %}
-<ul class="nav nav-tabs" style="display: inline-block">
-  <li><a>Build {{build}} : </a></li>
-  <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
-  <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
-  <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
-</ul>
-    <h1>Toaster - Tasks</h1>
-{% endblock %}
+{% extends "basebuildpage.html" %}
 
+{% block pagetitle %}Tasks{% endblock %}
 {% block pagetable %}
     {% if not tasks %}
         <p>No tasks were executed in this build!</p>
@@ -55,9 +46,9 @@
                 <td><a target="_fileview" href="file:///{{task.recipe.file_path}}">{{task.recipe.file_path}}</a></td>
                 <td>
             <div style="height: 3em; overflow:auto">
-                {% for tt in task.depends_on %}
-                    <a href="#{{tt.recipe.name}}.{{tt.task_name}}">
-                    {{tt.recipe.name}}.{{tt.task_name}}</a><br/>
+                {% for tt in task.task_dependencies_task.all %}
+                    <a href="#{{tt.depends_on.recipe.name}}.{{tt.depends_on.task_name}}">
+                    {{tt.depends_on.recipe.name}}.{{tt.depends_on.task_name}}</a><br/>
                 {% endfor %}
             </div>
                 </td>
bitbake/lib/webhob/bldviewer/urls.py +5 -5
index 11bdbe9..d49f8a5 100644
@@ -2,12 +2,12 @@ from django.conf.urls import patterns, include, url
 from django.views.generic.simple import redirect_to
 
 urlpatterns = patterns('bldviewer.views',
-        url(r'^build/$', 'build', name='build'),
+        url(r'^builds/$', 'build', name='build'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
-        url(r'^build/(?P<build_id>\d+)/targets/$', 'task', name='task'),
-        url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'),
+        url(r'^build/(?P<build_id>\d+)/packages/$', 'bpackage', name='bpackage'),
+        url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/packages/$', 'tpackage', name='tpackage'),
         url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'),
-        url(r'^layer/$', 'layer', name='layer'),
+        url(r'^layers/$', 'layer', name='layer'),
         url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'),
-        url(r'^$', redirect_to, {'url': 'build/'}),
+        url(r'^$', redirect_to, {'url': 'builds/'}),
 )
bitbake/lib/webhob/bldviewer/views.py +5 -18
index ab222ac..cfc2a89 100644
@@ -25,37 +25,24 @@ def task(request, build_id):
     template = 'task.html'
 
     tasks = Task.objects.filter(build=build_id)
-    task_depends = Task_Dependency.objects.filter(task__in=tasks).select_related()
 
-    for t in tasks:
-        t.depends_on = []
-        for k in task_depends:
-            if t == k.task:
-                t.depends_on.append(k.depends_on)
-
-    context = {'build': build_id, 'tasks': tasks}
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'tasks': tasks}
 
     return render(request, template, context)
 
 def configuration(request, build_id):
     template = 'configuration.html'
     variables = Variable.objects.filter(build=build_id)
-    context = {'build': build_id, 'configuration' : variables}
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables}
     return render(request, template, context)
 
-def package(request, build_id):
+def tpackage(request, build_id, target_id):
     template = 'package.html'
 
-    packages = Target_Package.objects.filter(build=build_id)
+    packages = Target_Package.objects.filter(target=target_id)
     package_depends = Target_Package_Dependency.objects.filter(package__in=packages)
 
-    for t in packages:
-        t.depends_on = []
-        for k in package_depends:
-            if t == k.package:
-                t.depends_on.append(k.depends_on)
-
-    context = {'build' : build_id ,'packages': packages}
+    context = {'build' : Build.objects.filter(pk=build_id)[0],'packages': packages}
 
     return render(request, template, context)
 
bitbake/lib/webhob/orm/models.py +5 -1
index 4b5c413..f3d988e 100644
@@ -1,4 +1,5 @@
 from django.db import models
+from django.utils.encoding import python_2_unicode_compatible
 
 
 class Build(models.Model):
@@ -27,7 +28,7 @@ class Build(models.Model):
     build_name = models.CharField(max_length=100)
     bitbake_version = models.CharField(max_length=50)
 
-
+@python_2_unicode_compatible
 class Target(models.Model):
     search_allowed_fields = ['target', 'image_fstypes', 'file_name']
     build = models.ForeignKey(Build)
@@ -37,6 +38,9 @@ class Target(models.Model):
     file_name = models.CharField(max_length=100)
     file_size = models.IntegerField()
 
+    def __str__(self):
+        return self.target
+
 
 class Task(models.Model):
 
bitbake/lib/webhob/whbmain/urls.py +1 -1
index 0e15eda..73f9370 100644
@@ -10,7 +10,7 @@ urlpatterns = patterns('',
     url(r'^simple/', include('bldviewer.urls')),
     url(r'^api/1.0/', include('bldviewer.api')),
     url(r'^gui/', include('whbgui.urls')),
-    url(r'^$', redirect_to, {'url': '/gui/'}),
+    url(r'^$', redirect_to, {'url': '/simple/'}),
     # Examples:
     # url(r'^webhob/', include('webhob.foo.urls')),
 
Signature · 1 line
1.8.1.2
#75 lore raw

PATCH 75/94 bitbake: dsi: save build-time package information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Saving the build-time package information in the database. Slight additions to the ORM model as to record more information.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 20 ++++++++++++++++++++
 bitbake/lib/bb/ui/dsi.py             |  5 +++++
 bitbake/lib/webhob/orm/models.py     |  8 ++++++--
 3 files changed, 31 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +20 -0
index 0860d2a..06ca703 100644
@@ -156,6 +156,20 @@ class ORMWrapper(object):
 
         return log_object.save()
 
+
+    def save_build_package_information(self, build_obj, package_info, recipes):
+        bp_object = Build_Package.objects.create( build = build_obj,
+                                       recipe = recipes[package_info['PN']],
+                                       name = package_info['PKG'],
+                                       version = package_info['PKGV'],
+                                       revision = package_info['PKGR'],
+                                       summary = package_info['SUMMARY'],
+                                       description = package_info['DESCRIPTION'],
+                                       section = package_info['SECTION'],
+                                       license = package_info['LICENSE'],
+                                       )
+        return bp_object
+
     def save_build_variables(self, build_obj, vardump):
         for k in vardump:
             if not bool(vardump[k]['func']):
@@ -558,6 +572,12 @@ class BuildInfoHelper(object):
                 dep = _save_a_task(taskdesc1)
                 Task_Dependency.objects.get_or_create( task = target, depends_on = dep )
 
+    def store_build_package_information(self, event):
+        package_info = event.data
+        self.orm_wrapper.save_build_package_information(self.internal_state['build'],
+                            package_info, self.internal_state['recipes'])
+
+
     def store_log_information(self, level, text):
         log_information = {}
         log_information['build'] = self.internal_state['build']
bitbake/lib/bb/ui/dsi.py +5 -0
index 3284ea2..643e98e 100644
@@ -489,6 +489,11 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 buildinfohelper.store_layer_info()
                 continue
 
+            if isinstance(event, bb.event.MetadataEvent):
+                if event.type == "SinglePackageInfo":
+                    buildinfohelper.store_build_package_information(event)
+                continue
+
             # ignore
             if isinstance(event, (bb.event.BuildBase,
                                   bb.event.StampUpdate,
bitbake/lib/webhob/orm/models.py +6 -2
index f3d988e..7e57c1d 100644
@@ -115,8 +115,12 @@ class Build_Package(models.Model):
     recipe = models.ForeignKey('Recipe', null=True)
     name = models.CharField(max_length=100)
     version = models.CharField(max_length=100, default="")
-    size = models.IntegerField()
-    license = models.CharField(max_length=200, null=True)
+    revision = models.CharField(max_length=32, default="")
+    summary = models.CharField(max_length=200, default="")
+    description = models.CharField(max_length=200, default="")
+    size = models.IntegerField(default=0)
+    section = models.CharField(max_length=80, null=True)
+    license = models.CharField(max_length=80, null=True)
 
 class Build_Package_Dependency(models.Model):
     TYPE_DEPENDS = 0
Signature · 1 line
1.8.1.2
#76 lore raw

PATCH 76/94 bitbake: webhob: Simple visualisation for the build-time packages

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding pages and views to allow viewing the data collected about the build-time packages.

---
 .../webhob/bldviewer/templates/basebuildpage.html  | 17 ++++++++++
 .../lib/webhob/bldviewer/templates/bpackage.html   | 36 ++++++++++++++++++++++
 .../lib/webhob/bldviewer/templates/package.html    | 10 ------
 bitbake/lib/webhob/bldviewer/views.py              |  9 +++++-
 4 files changed, 61 insertions(+), 11 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/basebuildpage.html
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/bpackage.html
bitbake/lib/webhob/bldviewer/templates/basebuildpage.html new file +17 -0
new file mode 100644
index 0000000..478019b
@@ -0,0 +1,17 @@
+{% extends "basetable.html" %}
+
+{% block pagename %}
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Build {{build.target_set.all|join:"&nbsp;"}} at {{build.started_on}} : </a></li>
+  <li><a href="/simple/build/{{build.id}}/task/"> Tasks </a></li>
+  <li><a href="/simple/build/{{build.id}}/packages/"> Build Packages </a></li>
+    {% for t in build.target_set.all %}
+        {% if t.is_image %}
+  <li><a href="/simple/build/{{build.id}}/target/{{t.pk}}/packages/"> Packages for {{t.target}} </a> </li>
+        {% endif %}
+    {% endfor %}
+  <li><a href="/simple/build/{{build.id}}/configuration/"> Configuration </a> </li>
+</ul>
+     <h1>Toaster - Build {% block pagetitle %} {% endblock %}</h1>
+{% endblock %}
+
bitbake/lib/webhob/bldviewer/templates/bpackage.html new file +36 -0
new file mode 100644
index 0000000..4e6cb00
@@ -0,0 +1,36 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetable %}
+    {% if not packages %}
+        <p>No packages were recorded for this target!</p>
+    {% else %}
+
+            <tr>
+            <th>Name</th>
+            <th>Version</th>
+            <th>Revision</th>
+            <th>Recipe</th>
+            <th>Summary</th>
+            <th>Section</th>
+            <th>Description</th>
+            <th>License</th>
+            </tr>
+
+            {% for package in packages %}
+
+            <tr class="data">
+                <td><a name="#{{package.name}}">{{package.name}}</a></td>
+                <td>{{package.version}}</td>
+                <td>{{package.revision}}</td>
+                <td><a href="/simple/layerversions/{{package.recipe.layer_version_id}}/recipes/#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td>
+                
+            <td>{{package.summary}}</td>
+            <td>{{package.section}}</td>
+            <td>{{package.description}}</td>
+            <td>{{package.license}}</td>
+
+            {% endfor %}
+
+    {% endif %}
+
+{% endblock %}
bitbake/lib/webhob/bldviewer/templates/package.html +0 -10
index cba21e1..4ce25ae 100644
@@ -1,15 +1,5 @@
 {% extends "basebuildpage.html" %}
 
-{% block pagename %}
-<ul class="nav nav-tabs" style="display: inline-block">
-  <li><a>Build {{build}} : </a></li>
-  <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li>
-  <li><a href="/simple/build/{{build}}/package/"> Package </a> </li>
-  <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
-</ul>
-     <h1>Toaster - Target Install Packages</h1>
-{% endblock %}
-
 {% block pagetable %}
     {% if not packages %}
         <p>No packages were recorded for this target!</p>
bitbake/lib/webhob/bldviewer/views.py +8 -1
index cfc2a89..a1da8b3 100644
@@ -3,7 +3,7 @@ import operator
 from django.db.models import Q
 from django.shortcuts import render
 from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable
-from orm.models import Task_Dependency, Target_Package_Dependency
+from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package
 from django.views.decorators.cache import cache_control
 
 @cache_control(no_store=True)
@@ -36,6 +36,13 @@ def configuration(request, build_id):
     context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables}
     return render(request, template, context)
 
+
+def bpackage(request, build_id):
+    template = 'bpackage.html'
+    packages = Build_Package.objects.filter(build = build_id)
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'packages' : packages}
+    return render(request, template, context)
+
 def tpackage(request, build_id, target_id):
     template = 'package.html'
 
Signature · 1 line
1.8.1.2
#77 lore raw

PATCH 77/94 bitbake: webhob: store file size information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding code to store file size information for each of the build package. Reassembling information from a pair of events for each package - first contains file and size information, and the second contains package metadata.

Cleanup of the database models to hold file size info.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 24 +++++++++++++++++++++---
 bitbake/lib/bb/ui/dsi.py             |  2 ++
 bitbake/lib/webhob/orm/models.py     | 12 ++++++------
 3 files changed, 29 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +21 -3
index 06ca703..8b0e720 100644
@@ -8,7 +8,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
 from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage
-from webhob.orm.models import Target_Package, Build_Package, Variable
+from webhob.orm.models import Target_Package, Build_Package, Variable, Build_File
 from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency
 from bb.msg import BBLogFormatter as format
 
@@ -157,7 +157,8 @@ class ORMWrapper(object):
         return log_object.save()
 
 
-    def save_build_package_information(self, build_obj, package_info, recipes):
+    def save_build_package_information(self, build_obj, package_info, recipes, files):
+        # create and save the object
         bp_object = Build_Package.objects.create( build = build_obj,
                                        recipe = recipes[package_info['PN']],
                                        name = package_info['PKG'],
@@ -168,6 +169,14 @@ class ORMWrapper(object):
                                        section = package_info['SECTION'],
                                        license = package_info['LICENSE'],
                                        )
+        # save any attached file information
+        if bp_object.name in files.keys():
+            for path, size in files[bp_object.name]:
+                fo = Build_File.objects.create( bpackage = bp_object,
+                                        path = path,
+                                        size = size )
+            del files[bp_object.name]
+
         return bp_object
 
     def save_build_variables(self, build_obj, vardump):
@@ -575,8 +584,17 @@ class BuildInfoHelper(object):
     def store_build_package_information(self, event):
         package_info = event.data
         self.orm_wrapper.save_build_package_information(self.internal_state['build'],
-                            package_info, self.internal_state['recipes'])
+                            package_info,
+                            self.internal_state['recipes'],
+                            self.internal_state['package_files'])
+
+
+    def store_package_file_information(self, event):
+        if not 'package_files' in self.internal_state.keys():
+            self.internal_state['package_files'] = {}
 
+        data = event._data
+        self.internal_state['package_files'][data['PKG']] = data['FILES']
 
     def store_log_information(self, level, text):
         log_information = {}
bitbake/lib/bb/ui/dsi.py +2 -0
index 643e98e..e5447ec 100644
@@ -492,6 +492,8 @@ def main(server, eventHandler, params, tf = TerminalFilter):
             if isinstance(event, bb.event.MetadataEvent):
                 if event.type == "SinglePackageInfo":
                     buildinfohelper.store_build_package_information(event)
+                elif event.type == "PackageFileSize":
+                    buildinfohelper.store_package_file_information(event)
                 continue
 
             # ignore
bitbake/lib/webhob/orm/models.py +6 -6
index 7e57c1d..aa368fe 100644
@@ -157,15 +157,15 @@ class Target_Package_Dependency(models.Model):
     dep_type = models.IntegerField(choices=DEPENDS_TYPE)
 
 
-class Build_Filelist(models.Model):
+class Build_File(models.Model):
     bpackage = models.ForeignKey(Build_Package, related_name='filelist_bpackage')
-    complete_file_path = models.FilePathField(max_length=255, blank=True)
-    file_size = models.IntegerField()
+    path = models.FilePathField(max_length=255, blank=True)
+    size = models.IntegerField()
 
-class Target_Filelist(models.Model):
+class Target_File(models.Model):
     tpackage = models.ForeignKey(Target_Package, related_name='filelist_tpackage')
-    complete_file_path = models.FilePathField(max_length=255, blank=True)
-    file_size = models.IntegerField()
+    path = models.FilePathField(max_length=255, blank=True)
+    size = models.IntegerField()
 
 
 class Recipe(models.Model):
Signature · 1 line
1.8.1.2
#78 lore raw

PATCH 78/94 bitbake: webhob: simple visualisation for package files

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding a simple table visualisation for data collected for package content - file names and file sizes.

---
 bitbake/lib/webhob/bldviewer/templates/bfile.html  | 24 ++++++++++++++++++++++
 .../lib/webhob/bldviewer/templates/bpackage.html   |  7 +++----
 bitbake/lib/webhob/bldviewer/urls.py               |  1 +
 bitbake/lib/webhob/bldviewer/views.py              |  9 ++++++--
 4 files changed, 35 insertions(+), 6 deletions(-)
 create mode 100644 bitbake/lib/webhob/bldviewer/templates/bfile.html
bitbake/lib/webhob/bldviewer/templates/bfile.html new file +24 -0
new file mode 100644
index 0000000..d4943c0
@@ -0,0 +1,24 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetitle %}Files for package {{files.0.bpackage.name}} {% endblock %}
+{% block pagetable %}
+    {% if not files %}
+        <p>No files were recorded for this package!</p>
+    {% else %}
+
+            <tr>
+            <th>Name</th>
+            <th>Size</th>
+            </tr>
+
+            {% for file in files %}
+
+            <tr class="data">
+            <td>{{file.path}}</td>
+            <td>{{file.size}}</td>
+
+            {% endfor %}
+
+    {% endif %}
+
+{% endblock %}
bitbake/lib/webhob/bldviewer/templates/bpackage.html +3 -4
index 4e6cb00..91e5f6f 100644
@@ -1,5 +1,6 @@
 {% extends "basebuildpage.html" %}
 
+{% block pagetitle %}Packages{% endblock %}
 {% block pagetable %}
     {% if not packages %}
         <p>No packages were recorded for this target!</p>
@@ -8,7 +9,6 @@
             <tr>
             <th>Name</th>
             <th>Version</th>
-            <th>Revision</th>
             <th>Recipe</th>
             <th>Summary</th>
             <th>Section</th>
@@ -19,9 +19,8 @@
             {% for package in packages %}
 
             <tr class="data">
-                <td><a name="#{{package.name}}">{{package.name}}</a></td>
-                <td>{{package.version}}</td>
-                <td>{{package.revision}}</td>
+                <td><a name="#{{package.name}}" href="/simple/build/{{build.pk}}/package/{{package.pk}}/files/">{{package.name}} ({{package.filelist_bpackage.count}} files)</a></td>
+                <td>{{package.version}}-{{package.revision}}</td>
                 <td><a href="/simple/layerversions/{{package.recipe.layer_version_id}}/recipes/#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td>
                 
             <td>{{package.summary}}</td>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
index d49f8a5..d6dd280 100644
@@ -5,6 +5,7 @@ urlpatterns = patterns('bldviewer.views',
         url(r'^builds/$', 'build', name='build'),
         url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
         url(r'^build/(?P<build_id>\d+)/packages/$', 'bpackage', name='bpackage'),
+        url(r'^build/(?P<build_id>\d+)/package/(?P<package_id>\d+)/files/$', 'bfile', name='bfile'),
         url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/packages/$', 'tpackage', name='tpackage'),
         url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'),
         url(r'^layers/$', 'layer', name='layer'),
bitbake/lib/webhob/bldviewer/views.py +7 -2
index a1da8b3..4ff23d3 100644
@@ -3,7 +3,7 @@ import operator
 from django.db.models import Q
 from django.shortcuts import render
 from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable
-from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package
+from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package, Build_File
 from django.views.decorators.cache import cache_control
 
 @cache_control(no_store=True)
@@ -36,13 +36,18 @@ def configuration(request, build_id):
     context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables}
     return render(request, template, context)
 
-
 def bpackage(request, build_id):
     template = 'bpackage.html'
     packages = Build_Package.objects.filter(build = build_id)
     context = {'build': Build.objects.filter(pk=build_id)[0], 'packages' : packages}
     return render(request, template, context)
 
+def bfile(request, build_id, package_id):
+    template = 'bfile.html'
+    files = Build_File.objects.filter(bpackage = package_id)
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'files' : files}
+    return render(request, template, context)
+
 def tpackage(request, build_id, target_id):
     template = 'package.html'
 
Signature · 1 line
1.8.1.2
#79 lore raw

PATCH 79/94 bitbake: dsi: record recipe dependency

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Add code to record recipe dependencies, for DEPENDS and RDEPENDS relationships.

Caveat emptor: relationships with ASSUME_PROVIDED
PNs will not be recorded, as they don't have real
recipe objects.

Adds code to the Simple interface to allow visualisation of the recorded data.

---
 bitbake/lib/bb/ui/buildinfohelper.py               | 23 +++++++++++++++++++++-
 bitbake/lib/webhob/bldviewer/templates/recipe.html |  9 +++++++++
 bitbake/lib/webhob/orm/models.py                   | 11 +++++++++--
 3 files changed, 40 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +22 -1
index 8b0e720..cf947da 100644
@@ -9,7 +9,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 import webhob.whbmain.settings as whb_django_settings
 from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage
 from webhob.orm.models import Target_Package, Build_Package, Variable, Build_File
-from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency
+from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency, Recipe_Dependency
 from bb.msg import BBLogFormatter as format
 
 class ORMWrapper(object):
@@ -562,6 +562,27 @@ class BuildInfoHelper(object):
                         t.save()
             self.internal_state['recipes'][pn] = recipe
 
+        # save recipe dependency
+        try:
+            # buildtime
+            for recipe in event._depgraph['depends']:
+                target = self.internal_state['recipes'][recipe]
+                for dep in event._depgraph['depends'][recipe]:
+                    dependency = self.internal_state['recipes'][dep]
+                    Recipe_Dependency.objects.get_or_create( recipe = target,
+                            depends_on = dependency, dep_type = Recipe_Dependency.TYPE_DEPENDS)
+
+            # runtime
+            for recipe in event._depgraph['rdepends-pn']:
+                target = self.internal_state['recipes'][recipe]
+                for dep in event._depgraph['rdepends-pn'][recipe]:
+                    dependency = self.internal_state['recipes'][dep]
+                    Recipe_Dependency.objects.get_or_create( recipe = target,
+                            depends_on = dependency, dep_type = Recipe_Dependency.TYPE_RDEPENDS)
+
+        except KeyError:    # we'll not get recipes for key w/ values listed in ASSUME_PROVIDED
+            pass
+
         # save all task information
         def _save_a_task(taskdesc):
             spec = re.split(r'\.', taskdesc);
bitbake/lib/webhob/bldviewer/templates/recipe.html +9 -0
index 32fcdb8..e81883f 100644
@@ -22,6 +22,8 @@
             <th>Bugtracker</th>
             <th>Author</th>
             <th>File_path</th>
+            <th style="width: 30em">Recipe Dependency</th>
+
 
         {% for recipe in recipes %}
 
@@ -36,6 +38,13 @@
             <td>{{recipe.bugtracker}}</td>
             <td>{{recipe.author}}</td>
             <td>{{recipe.file_path}}</td>
+            <td>
+        <div style="height: 5em; overflow:auto">
+            {% for rr in recipe.r_dependencies_recipe.all %}
+                <a href="#{{rr.depends_on.name}}">{{rr.depends_on.name}}</a><br/>
+            {% endfor %}
+        </div>
+            </td>
         </tr>
 
         {% endfor %}
bitbake/lib/webhob/orm/models.py +9 -2
index aa368fe..7ce0d1f 100644
@@ -184,9 +184,16 @@ class Recipe(models.Model):
 
 
 class Recipe_Dependency(models.Model):
-    recipe = models.ForeignKey(Recipe, related_name='recipe_dependencies_recipe')
-    depends_on = models.ForeignKey(Recipe, related_name='recipe_dependencies_depends')
+    TYPE_DEPENDS = 0
+    TYPE_RDEPENDS = 1
 
+    DEPENDS_TYPE = (
+        (TYPE_DEPENDS, "depends"),
+        (TYPE_RDEPENDS, "rdepends"),
+    )
+    recipe = models.ForeignKey(Recipe, related_name='r_dependencies_recipe')
+    depends_on = models.ForeignKey(Recipe, related_name='r_dependencies_depends')
+    dep_type = models.IntegerField(choices=DEPENDS_TYPE)
 
 class Layer(models.Model):
     name = models.CharField(max_length=100)
Signature · 1 line
1.8.1.2
#80 lore raw

PATCH 80/94 bitbake: dsi: add exception dumping code

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Dump detailed information about unexpected exceptions during a run. This has effect only when running DSI from command line, since in background mode everything is dumped to /dev/null

---
 bitbake/lib/bb/ui/dsi.py | 2 ++
 1 file changed, 2 insertions(+)
bitbake/lib/bb/ui/dsi.py +2 -0
index e5447ec..22cff5d 100644
@@ -540,6 +540,8 @@ def main(server, eventHandler, params, tf = TerminalFilter):
             pass
         except Exception as e:
             print(e)
+            import traceback
+            traceback.print_exc()
             pass
 
     summary = ""
Signature · 1 line
1.8.1.2
#81 lore raw

PATCH 81/94 bitbake: webhob: better help message in startup script

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Improved the error during stop message in the startup script as the previous message was confusing to the users.

---
 bitbake/bin/webhob | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
bitbake/bin/webhob +8 -1
index 0c4d9ea..7bc9759 100755
@@ -104,7 +104,14 @@ if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] );
     return 3
 elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then
     echo "Error: bitbake lock state error. Trying to stop a stopped system ?
-manually stop system with bitbake -m / webserverKill" 2>&1
+If you think the system is hanged up, you can try to manually stop system with the commands
+
+# BBSERVER=localhost:8200 bitbake -m
+
+and
+
+# webserverKillAll
+" 2>&1
     return 3
 fi
 
Signature · 1 line
1.8.1.2
#82 lore raw

PATCH 82/94 bitbake: dsi: add build package dependency recording

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Adding code to store all types of package dependencies that a build time package can have.

Added code to visualise the dependency listing in
the Simple interface.

Addded code to record content size for the
build time packages.

---
 bitbake/lib/bb/ui/buildinfohelper.py               | 33 ++++++++++++++++++++++
 .../lib/webhob/bldviewer/templates/bpackage.html   | 13 +++++++--
 bitbake/lib/webhob/orm/models.py                   | 16 ++++++++---
 3 files changed, 56 insertions(+), 6 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +33 -0
index cf947da..30aa098 100644
@@ -166,6 +166,7 @@ class ORMWrapper(object):
                                        revision = package_info['PKGR'],
                                        summary = package_info['SUMMARY'],
                                        description = package_info['DESCRIPTION'],
+                                       size = package_info['PKGSIZE'],
                                        section = package_info['SECTION'],
                                        license = package_info['LICENSE'],
                                        )
@@ -177,6 +178,38 @@ class ORMWrapper(object):
                                         size = size )
             del files[bp_object.name]
 
+        # save soft dependency information
+        if package_info['RDEPENDS'] is not None:
+            for p in package_info['RDEPENDS'].split(" "):
+                if not p.startswith("("):
+                    Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                        depends_on = p, dep_type = Build_Package_Dependency.TYPE_RDEPENDS)
+        if package_info['RDEPENDS'] is not None:
+            for p in package_info['RPROVIDES'].split(" "):
+                if not p.startswith("("):
+                    Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                        depends_on = p, dep_type = Build_Package_Dependency.TYPE_RPROVIDES)
+        if package_info['RDEPENDS'] is not None:
+            for p in package_info['RRECOMMENDS'].split(" "):
+                if not p.startswith("("):
+                    Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                        depends_on = p, dep_type = Build_Package_Dependency.TYPE_RRECOMMENDS)
+        if package_info['RDEPENDS'] is not None:
+            for p in package_info['RSUGGESTS'].split(" "):
+                if not p.startswith("("):
+                    Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                        depends_on = p, dep_type = Build_Package_Dependency.TYPE_RSUGGESTS)
+        if package_info['RDEPENDS'] is not None:
+            for p in package_info['RREPLACES'].split(" "):
+                if not p.startswith("("):
+                    Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                        depends_on = p, dep_type = Build_Package_Dependency.TYPE_RREPLACES)
+        if package_info['RDEPENDS'] is not None:
+            for p in package_info['RCONFLICTS'].split(" "):
+                if not p.startswith("("):
+                    Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                        depends_on = p, dep_type = Build_Package_Dependency.TYPE_RCONFLICTS)
+
         return bp_object
 
     def save_build_variables(self, build_obj, vardump):
bitbake/lib/webhob/bldviewer/templates/bpackage.html +11 -2
index 91e5f6f..46a256c 100644
@@ -13,7 +13,9 @@
             <th>Summary</th>
             <th>Section</th>
             <th>Description</th>
+            <th>Size</th>
             <th>License</th>
+            <th>Dependencies List (all)</th>
             </tr>
 
             {% for package in packages %}
@@ -22,12 +24,19 @@
                 <td><a name="#{{package.name}}" href="/simple/build/{{build.pk}}/package/{{package.pk}}/files/">{{package.name}} ({{package.filelist_bpackage.count}} files)</a></td>
                 <td>{{package.version}}-{{package.revision}}</td>
                 <td><a href="/simple/layerversions/{{package.recipe.layer_version_id}}/recipes/#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td>
-                
+
             <td>{{package.summary}}</td>
             <td>{{package.section}}</td>
             <td>{{package.description}}</td>
+            <td>{{package.size}}</td>
             <td>{{package.license}}</td>
-
+            <td>
+        <div style="height: 3em">
+            {% for bpd in package.bpackage_dependencies_package.all %}
+                {{bpd.dep_type}}: {{bpd.depends_on}} <br/>
+            {% endfor %}
+        </div>
+            </td>
             {% endfor %}
 
     {% endif %}
bitbake/lib/webhob/orm/models.py +12 -4
index 7ce0d1f..2018020 100644
@@ -123,14 +123,22 @@ class Build_Package(models.Model):
     license = models.CharField(max_length=80, null=True)
 
 class Build_Package_Dependency(models.Model):
-    TYPE_DEPENDS = 0
-    TYPE_RDEPENDS = 1
+    TYPE_RDEPENDS = 0
+    TYPE_RPROVIDES = 1
+    TYPE_RRECOMMENDS = 2
+    TYPE_RSUGGESTS = 3
+    TYPE_RREPLACES = 4
+    TYPE_RCONFLICTS = 5
     DEPENDS_TYPE = (
-        (TYPE_DEPENDS, "depends"),
         (TYPE_RDEPENDS, "rdepends"),
+        (TYPE_RPROVIDES, "rprovides"),
+        (TYPE_RRECOMMENDS, "rrecommends"),
+        (TYPE_RSUGGESTS, "rsuggests"),
+        (TYPE_RREPLACES, "rreplaces"),
+        (TYPE_RCONFLICTS, "rconflicts"),
     )
     package = models.ForeignKey(Build_Package, related_name='bpackage_dependencies_package')
-    depends_on = models.ForeignKey(Build_Package, related_name='bpackage_dependencies_depends')
+    depends_on = models.CharField(max_length=100)   # soft dependency
     dep_type = models.IntegerField(choices=DEPENDS_TYPE)
 
 
Signature · 1 line
1.8.1.2
#83 lore raw

PATCH 83/94 bitbake: webhob: clean up Machine table

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Removing the Machine table in the favour of a column in the Build table since it complicated the queries without any benefit in reduced data structure.

---
 bitbake/lib/bb/ui/buildinfohelper.py              | 26 +++++------------------
 bitbake/lib/webhob/bldviewer/templates/build.html |  2 +-
 bitbake/lib/webhob/orm/models.py                  |  9 ++------
 3 files changed, 8 insertions(+), 29 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +5 -21
index 30aa098..b5c2dc0 100644
@@ -7,7 +7,7 @@ import re
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
 
 import webhob.whbmain.settings as whb_django_settings
-from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage
+from webhob.orm.models import Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage
 from webhob.orm.models import Target_Package, Build_Package, Variable, Build_File
 from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency, Recipe_Dependency
 from bb.msg import BBLogFormatter as format
@@ -21,12 +21,6 @@ class ORMWrapper(object):
     def __init__(self):
         pass
 
-    def create_machine_object(self, machine_information):
-
-        machine = Machine.objects.get_or_create(name=machine_information['name'],
-                                                description=machine_information['description'])
-
-        return machine[0]
 
     def create_build_object(self, build_info):
 
@@ -244,13 +238,6 @@ class BuildInfoHelper(object):
     ###################
     ## methods to convert event/external info into objects that the ORM layer uses
 
-    def _get_machine_information(self):
-        machine_info = {}
-        machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0]
-        machine_info['description'] = 'Not Available'
-        self.internal_state['machine'] = machine_info['name']
-        return machine_info
-
     def _get_layer_dict(self, layer_path):
 
         layer_info = {}
@@ -313,11 +300,11 @@ class BuildInfoHelper(object):
         return "<unknown>"
 
 
-    def _get_build_information(self, machine_obj):
+    def _get_build_information(self):
         build_info = {}
         # Generate an identifier for each new build
 
-        build_info['machine'] = machine_obj
+        build_info['machine'] = self.server.runCommand(["getVariable", "MACHINE"])[0]
         build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0]
         build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
         build_info['started_on'] = datetime.datetime.now()
@@ -375,7 +362,7 @@ class BuildInfoHelper(object):
 
         for t in self.internal_state['targets']:
             target = t.target
-            machine = self.internal_state['build'].machine.name
+            machine = self.internal_state['build'].machine
             buildname = self.internal_state['build'].build_name
             package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":")
 
@@ -428,10 +415,7 @@ class BuildInfoHelper(object):
 
     def store_started_build(self, event):
 
-        machine_information = self._get_machine_information()
-        machine_obj = self.orm_wrapper.create_machine_object(machine_information)
-
-        build_information = self._get_build_information(machine_obj)
+        build_information = self._get_build_information()
 
         build_obj = self.orm_wrapper.create_build_object(build_information)
         self.internal_state['build'] = build_obj
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
index eed81a3..dd0e5c0 100644
@@ -29,7 +29,7 @@
             <td>{{build.completed_on}}</td>
             <td>{% for t in build.target_set.all %}<a href="/simple/build/{{build.id}}/target/{{t.id}}">{{t.target}}</a><br/>{% endfor %}</td>
             <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td>
-            <td>{{build.machine.name}}</td>
+            <td>{{build.machine}}</td>
             <td>{% time_difference build.started_on build.completed_on %}</td>
             <td>{{build.errors_no}}:{% if  build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
             <td>{{build.warnings_no}}:{% if  build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
bitbake/lib/webhob/orm/models.py +2 -7
index 2018020..66cbdc8 100644
@@ -13,10 +13,10 @@ class Build(models.Model):
         (IN_PROGRESS, 'In Progress'),
     )
 
-    search_allowed_fields = ['machine__name',
+    search_allowed_fields = ['machine',
                              'cooker_log_path']
 
-    machine = models.ForeignKey('Machine', related_name='build_machine')
+    machine = models.CharField(max_length=100)
     distro = models.CharField(max_length=100)
     distro_version = models.CharField(max_length=100)
     started_on = models.DateTimeField()
@@ -226,11 +226,6 @@ class Variable(models.Model):
     description = models.TextField(null=True)
 
 
-class Machine(models.Model):
-    name = models.CharField(max_length=100)
-    description = models.TextField()
-
-
 class LogMessage(models.Model):
     INFO = 0
     WARNING = 1
Signature · 1 line
1.8.1.2
#84 lore raw

PATCH 84/94 bitbake: webhob: update API endpoints

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

Updating the API endpoints as described by the documentation.

The documentation updated here:
https://wiki.yoctoproject.org/wiki/REST_API_Contracts

---
 bitbake/lib/webhob/bldviewer/api.py   | 20 ++++++++++++++------
 bitbake/lib/webhob/bldviewer/views.py | 15 ++++++++++++---
 2 files changed, 26 insertions(+), 9 deletions(-)
bitbake/lib/webhob/bldviewer/api.py +14 -6
index 9fa701c..2e58313 100644
@@ -2,10 +2,18 @@ from django.conf.urls import patterns, include, url
 
 
 urlpatterns = patterns('bldviewer.views',
-        url(r'^builds/$', 'model_explorer',  {'model_name':'build'}, name='builds'),
-        url(r'^tasks/$', 'model_explorer', {'model_name':'task'}, name='task'),
-        url(r'^packages/$', 'model_explorer', {'model_name':'package'}, name='package'),
-        url(r'^layers/$', 'model_explorer', {'model_name':'layer'}, name='layer'),
-        url(r'^recipes/$', 'model_explorer', {'model_name':'recipe'}, name='recipe'),
-        url(r'^layersversions/$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'),
+        url(r'^builds$', 'model_explorer',  {'model_name':'build'}, name='builds'),
+        url(r'^targets$', 'model_explorer',  {'model_name':'target'}, name='targets'),
+        url(r'^tasks$', 'model_explorer', {'model_name':'task'}, name='task'),
+        url(r'^task_dependencies$', 'model_explorer',  {'model_name':'task_dependency'}, name='task_dependencies'),
+        url(r'^packages$', 'model_explorer',  {'model_name':'build_package'}, name='build_packages'),
+        url(r'^package_dependencies$', 'model_explorer',  {'model_name':'build_package_dependency'}, name='build_package_dependencies'),
+        url(r'^target_packages$', 'model_explorer',  {'model_name':'target_package'}, name='target_packages'),
+        url(r'^package_files$', 'model_explorer',  {'model_name':'build_file'}, name='build_files'),
+        url(r'^layers$', 'model_explorer', {'model_name':'layer'}, name='layer'),
+        url(r'^layerversions$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'),
+        url(r'^recipes$', 'model_explorer', {'model_name':'recipe'}, name='recipe'),
+        url(r'^recipe_dependencies$', 'model_explorer',  {'model_name':'recipe_dependency'}, name='recipe_dependencies'),
+        url(r'^variables$', 'model_explorer',  {'model_name':'variable'}, name='variables'),
+        url(r'^logmessages$', 'model_explorer',  {'model_name':'logmessage'}, name='logmessages'),
 )
bitbake/lib/webhob/bldviewer/views.py +12 -3
index 4ff23d3..98099ee 100644
@@ -2,8 +2,8 @@ import operator
 
 from django.db.models import Q
 from django.shortcuts import render
-from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable
-from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package, Build_File
+from orm.models import Build, Target, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable
+from orm.models import Task_Dependency, Recipe_Dependency, Build_Package, Build_File, Build_Package_Dependency
 from django.views.decorators.cache import cache_control
 
 @cache_control(no_store=True)
@@ -95,11 +95,20 @@ def model_explorer(request, model_name):
     response_data = {}
     model_mapping = {
         'build': Build,
+        'target': Target,
+        'target_package': Target_Package,
         'task': Task,
-        'package': Target_Package,
+        'task_dependency': Task_Dependency,
+        'package': Build_Package,
         'layer': Layer,
         'layerversion': Layer_Version,
         'recipe': Recipe,
+        'recipe_dependency': Recipe_Dependency,
+        'build_package': Build_Package,
+        'build_package_dependency': Build_Package_Dependency,
+        'build_file': Build_File,
+        'variable': Variable,
+        'logmessage': LogMessage,
         }
 
     if model_name not in model_mapping.keys():
Signature · 1 line
1.8.1.2
#85 lore raw

PATCH 85/94 bitbake: webhob: record task hash information

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We record the task hash for all queue tasks.

Displaying the task hash in the Simple interface.

---
 bitbake/lib/bb/ui/buildinfohelper.py             | 5 +++++
 bitbake/lib/webhob/bldviewer/templates/task.html | 2 ++
 2 files changed, 7 insertions(+)
bitbake/lib/bb/ui/buildinfohelper.py +5 -0
index b5c2dc0..4ffa0d9 100644
@@ -324,6 +324,11 @@ class BuildInfoHelper(object):
         task_information['outcome'] = Task.OUTCOME_NA
         task_information['recipe'] = recipe
         task_information['task_name'] = event.taskname
+        try:
+            # some tasks don't come with a hash. and that's ok
+            task_information['sstate_checksum'] = event.taskhash
+        except AttributeError:
+            pass
         return task_information
 
     def _get_layer_version_for_path(self, path):
bitbake/lib/webhob/bldviewer/templates/task.html +2 -0
index 65d75cb..25715fe 100644
@@ -11,6 +11,7 @@
             <th>Task</th>
             <th>Recipe Version</th>
             <th>Task Type</th>
+            <th>Checksum</th>
             <th>Outcome</th>
             <th>Message</th>
             <th>Logfile</th>
@@ -36,6 +37,7 @@
                 <td>Prebuilt</td>
                 {% endif %}
 
+                <td>{{task.sstate_checksum}}</td>
                 <td>{{task.get_outcome_display}}</td>
                 <td><p>{{task.message}}</td>
                 <td><a target="_fileview" href="file:///{{task.logfile}}">{{task.logfile}}</a></td>
Signature · 1 line
1.8.1.2