Webhob patches for DSI and Web modules
cover Webhob patches for DSI and Web modules
From: Alexandru DAMIAN <[email protected]>
Patches for the DSI and Web modules of Webhob, in the order of development.
DSI is the DataStore Interface, the component that records information derived from a Bitbake build into the data backend.
Web module consists of the Simple UI interface, and the API interface available to application for data query.
External components jQuery and Bootstrap are included in the sources, and respective licenses documented (APACHE-2.0 and MIT).
The following changes since commit 0fc8317c6385eb1ed69ca4522ee6424c456dbb92:
yocto-bsp: add 3.10/remove 3.8 kernel from templates (2013-09-24 12:01:47 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib webhob-poky/webhob-master-1_5_upstream_2509
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=webhob-poky/webhob-master-1_5_upstream_2509
Alexandru DAMIAN (72):
webhob: add system start/stop script
bitbake: webhob: set sane default settings
bitbake: webhob: adds dsi support for observer mode
bitbake: dsi: Translate runQueue events into Task data
bitbake: webhob: disable synchronous sqlite connection
bitbake: buildinfohelper: clean-up the imports
bitbake: buildinfohelper: clearing up task order
bitbake: dsi: saving prebuild task detailed information
bitbake: dsi: clear up the state on build completion
bitbake: dsi: event data change
bitbake: webhob: fix and cleanup start script
bitbake: dsi: refactor the BuildInfoHelper code
bitbake: webhob: record dependency info
bitbake: webhob: use defined constants for models
bitbake: dsi: fix the reading of task event information
bitbake: webhob: orm change to remove Target
bitbake: webhob: clear up ORM relations
bitbake: dsi: save detailed recipe information
bitbake: webhob: determine if a build is an image
bitbake: webhob: clean up URL structure
bitbake: webhob: add simple pages for Recipes
bitbake: webhob: enhance Simple browser navigation
bitbake: webhob: improve startup script
bitbake: webhob: extend search for multiple terms
bitbake: webhob: force bitbake server stop
bitbake: webhob: simple interface dependency list
bitbake: dsi: add feature to store package information
bitbake: webhob: add simple viewer for package information
bitbake: dsi: fix build stats data gathering
bitbake: dsi: update build object on command end
bitbake: dsi: fix sstate task information gathering
bitbake: webhob: clean up starting script
bitbake: dsi: store log information
bitbake: webhob: add simple visualisation for build errors
bitbake: webhob: store logfile and message for Tasks
bitbake: webhob: display proper information in Simple
bitbake: dsi: clear up global variables between builds
bitbake: dsi: small bugfixes in data collection
bitbake: bldviewer: add jquery and bootstrap frameworks
bitbake: webhob: add toggle column functionality to build page
bitbake: webhob: add search functionality to the Simple interface
bitbake: webhob: refactor column hiding code
bitbake: webhob: improve search functionality
bitbake: webhob: refactor Simple web interface
bitbake: webhob: simple interface CSS
bitbake: dsi: use get vars command to store configuration
bitbake: webhob: refactor CSS display in Simple
bitbake: webhob: add Configuration visualisation in Simple
bitbake: webhob: add navigation links Simple interface
bitbake: webhob: startup script fixing
bitbake: webhob: change database models and related
bitbake: webhob: navigation in the Simple interface
bitbake: dsi: save build-time package information
bitbake: webhob: Simple visualisation for the build-time packages
bitbake: webhob: store file size information
bitbake: webhob: simple visualisation for package files
bitbake: dsi: record recipe dependency
bitbake: dsi: add exception dumping code
bitbake: webhob: better help message in startup script
bitbake: dsi: add build package dependency recording
bitbake: webhob: clean up Machine table
bitbake: webhob: update API endpoints
bitbake: webhob: record task hash information
bitbake: cooker: add the inherits attribute to the dependency tree
bitbake: webhob: Clean up links in Simple UI
bitbake: dsi: read correct recipe description field
bitbake: cooker: send layer information with dependency tree
bitbake: webhob: store and display layer priorities
bitbake: webhob: collect recipe licensing info
bitbake: webhob: Simple UI interface CSS fix
bitbake: dsi: fix typos
bitbake: webhob: do not set timezones
Calin Dragomir (22):
webhob: create main WEBHOB project
webhob: create Django models for webhob
bitbake: create Data Store Interface (DSI) file
bitbake: webhob: updates to Django models
bitbake: webhob: make DSI store build information
bitbake: webhob: make DSI store task information
bitbake: webhob: view a table with all builds
bitbake: webhob: view a table with all the tasks of a build
bitbake: webhob: use buildcompleted event as the end of a build
operation
bitbake: webhob: changes to build information table
bitbake: webhob: gather buildstats for each executed task
bitbake: webhob: add layer information to the database
bitbake: webhob: mark private methods inside DSI
bitbake: webhob: create basic structure for static files
bitbake: webhob: Setup API for build model
bitbake: webhob: add ordering capabilities to build api
bitbake: webhob: validate inputs for build api
bitbake: webhob: generic view for multiple models
bitbake: webhob: improve validation code flow
bitbake: webhob: add more models to webhob API
bitbake: webhob: add search for build model
bitbake: webhob: fix ordering issue
bitbake/bin/webhob | 145 + bitbake/lib/bb/cooker.py | 2 + bitbake/lib/bb/ui/buildinfohelper.py | 678 ++ bitbake/lib/bb/ui/dsi.py | 569 ++ bitbake/lib/webhob/__init__.py | 0 bitbake/lib/webhob/bldviewer/__init__.py | 0 bitbake/lib/webhob/bldviewer/api.py | 19 + .../lib/webhob/bldviewer/static/css/bootstrap.css | 4797 +++++++++++ .../lib/webhob/bldviewer/static/js/bootstrap.js | 1982 +++++ .../lib/webhob/bldviewer/static/js/jquery-2.0.3.js | 8829 ++++++++++++++++++++ bitbake/lib/webhob/bldviewer/templates/base.html | 30 + .../webhob/bldviewer/templates/basebuildpage.html | 17 + .../lib/webhob/bldviewer/templates/basetable.html | 46 + bitbake/lib/webhob/bldviewer/templates/bfile.html | 24 + .../lib/webhob/bldviewer/templates/bpackage.html | 44 + bitbake/lib/webhob/bldviewer/templates/build.html | 45 + .../webhob/bldviewer/templates/configuration.html | 20 + bitbake/lib/webhob/bldviewer/templates/layer.html | 34 + .../lib/webhob/bldviewer/templates/package.html | 36 + bitbake/lib/webhob/bldviewer/templates/recipe.html | 54 + bitbake/lib/webhob/bldviewer/templates/task.html | 63 + .../lib/webhob/bldviewer/templatetags/__init__.py | 0 .../webhob/bldviewer/templatetags/projecttags.py | 8 + bitbake/lib/webhob/bldviewer/urls.py | 14 + bitbake/lib/webhob/bldviewer/views.py | 220 + bitbake/lib/webhob/manage.py | 10 + bitbake/lib/webhob/orm/__init__.py | 0 bitbake/lib/webhob/orm/models.py | 242 + bitbake/lib/webhob/orm/tests.py | 16 + bitbake/lib/webhob/orm/views.py | 1 + bitbake/lib/webhob/whbgui/__init__.py | 0 bitbake/lib/webhob/whbgui/static/images/yocto.jpg | Bin 0 -> 6582 bytes bitbake/lib/webhob/whbgui/templates/index.html | 13 + bitbake/lib/webhob/whbgui/tests.py | 16 + bitbake/lib/webhob/whbgui/urls.py | 9 + bitbake/lib/webhob/whbgui/views.py | 8 + bitbake/lib/webhob/whbmain/__init__.py | 0 bitbake/lib/webhob/whbmain/settings.py | 173 + bitbake/lib/webhob/whbmain/urls.py | 22 + bitbake/lib/webhob/whbmain/wsgi.py | 32 + 40 files changed, 18218 insertions(+) create mode 100755 bitbake/bin/webhob create mode 100644 bitbake/lib/bb/ui/buildinfohelper.py create mode 100644 bitbake/lib/bb/ui/dsi.py create mode 100644 bitbake/lib/webhob/__init__.py create mode 100644 bitbake/lib/webhob/bldviewer/__init__.py create mode 100644 bitbake/lib/webhob/bldviewer/api.py create mode 100644 bitbake/lib/webhob/bldviewer/static/css/bootstrap.css create mode 100644 bitbake/lib/webhob/bldviewer/static/js/bootstrap.js create mode 100644 bitbake/lib/webhob/bldviewer/static/js/jquery-2.0.3.js create mode 100644 bitbake/lib/webhob/bldviewer/templates/base.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/basebuildpage.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/basetable.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/bfile.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/bpackage.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/build.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/configuration.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/layer.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/package.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/recipe.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/task.html create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/__init__.py create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/projecttags.py create mode 100644 bitbake/lib/webhob/bldviewer/urls.py create mode 100644 bitbake/lib/webhob/bldviewer/views.py create mode 100755 bitbake/lib/webhob/manage.py create mode 100644 bitbake/lib/webhob/orm/__init__.py create mode 100644 bitbake/lib/webhob/orm/models.py create mode 100644 bitbake/lib/webhob/orm/tests.py create mode 100644 bitbake/lib/webhob/orm/views.py create mode 100644 bitbake/lib/webhob/whbgui/__init__.py create mode 100644 bitbake/lib/webhob/whbgui/static/images/yocto.jpg create mode 100644 bitbake/lib/webhob/whbgui/templates/index.html create mode 100644 bitbake/lib/webhob/whbgui/tests.py create mode 100644 bitbake/lib/webhob/whbgui/urls.py create mode 100644 bitbake/lib/webhob/whbgui/views.py create mode 100644 bitbake/lib/webhob/whbmain/__init__.py create mode 100644 bitbake/lib/webhob/whbmain/settings.py create mode 100644 bitbake/lib/webhob/whbmain/urls.py create mode 100644 bitbake/lib/webhob/whbmain/wsgi.py
Signature · 1 line
1.8.1.2
PATCH 1/94 webhob: create main WEBHOB project
From: Calin Dragomir <[email protected]>
This creates a basic Django project where the WEBHOB code will live. The main project had to be renamed from webhob to whbmain in order to avoid any import issues that might occur.
--- bitbake/lib/webhob/__init__.py | 0 bitbake/lib/webhob/manage.py | 10 +++ bitbake/lib/webhob/whbmain/__init__.py | 0 bitbake/lib/webhob/whbmain/settings.py | 155 +++++++++++++++++++++++++++++++++ bitbake/lib/webhob/whbmain/urls.py | 17 ++++ bitbake/lib/webhob/whbmain/wsgi.py | 32 +++++++ 6 files changed, 214 insertions(+) create mode 100644 bitbake/lib/webhob/__init__.py create mode 100755 bitbake/lib/webhob/manage.py create mode 100644 bitbake/lib/webhob/whbmain/__init__.py create mode 100644 bitbake/lib/webhob/whbmain/settings.py create mode 100644 bitbake/lib/webhob/whbmain/urls.py create mode 100644 bitbake/lib/webhob/whbmain/wsgi.py
bitbake/lib/webhob/__init__.py new file +0 -0
bitbake/lib/webhob/manage.py new file +10 -0
@@ -0,0 +1,10 @@ +#!/usr/bin/env python +import os +import sys + +if __name__ == "__main__": + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "whbmain.settings") + + from django.core.management import execute_from_command_line + + execute_from_command_line(sys.argv)
bitbake/lib/webhob/whbmain/__init__.py new file +0 -0
bitbake/lib/webhob/whbmain/settings.py new file +155 -0
155 lines not on this page · raw message
bitbake/lib/webhob/whbmain/urls.py new file +17 -0
@@ -0,0 +1,17 @@ +from django.conf.urls import patterns, include, url + +# Uncomment the next two lines to enable the admin: +# from django.contrib import admin +# admin.autodiscover() + +urlpatterns = patterns('', + # Examples: + # url(r'^$', 'webhob.views.home', name='home'), + # url(r'^webhob/', include('webhob.foo.urls')), + + # Uncomment the admin/doc line below to enable admin documentation: + # url(r'^admin/doc/', include('django.contrib.admindocs.urls')), + + # Uncomment the next line to enable the admin: + # url(r'^admin/', include(admin.site.urls)), +)
bitbake/lib/webhob/whbmain/wsgi.py new file +32 -0
@@ -0,0 +1,32 @@ +""" +WSGI config for webhob project. + +This module contains the WSGI application used by Django's development server +and any production WSGI deployments. It should expose a module-level variable +named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover +this application via the ``WSGI_APPLICATION`` setting. + +Usually you will have the standard Django WSGI application here, but it also +might make sense to replace the whole Django WSGI application with a custom one +that later delegates to the Django one. For example, you could introduce WSGI +middleware here, or combine a Django application with an application of another +framework. + +""" +import os + +# We defer to a DJANGO_SETTINGS_MODULE already in the environment. This breaks +# if running multiple sites in the same mod_wsgi process. To fix this, use +# mod_wsgi daemon mode with each site in its own daemon process, or use +# os.environ["DJANGO_SETTINGS_MODULE"] = "webhob.settings" +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "wbhmain.settings") + +# This application object is used by any WSGI server configured to use this +# file. This includes Django's development server, if the WSGI_APPLICATION +# setting points here. +from django.core.wsgi import get_wsgi_application +application = get_wsgi_application() + +# Apply WSGI middleware here. +# from helloworld.wsgi import HelloWorldApplication +# application = HelloWorldApplication(application)
Signature · 1 line
1.8.1.2
PATCH 2/94 webhob: add system start/stop script
From: Alexandru DAMIAN <[email protected]>
Adding a script that starts/stops the webhob system. Sourced, with a parameter, it will start or stop the bitbake resident, and the webhob server, as well as point a browser to the webhob home .
--- bitbake/bin/webhob | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100755 bitbake/bin/webhob
bitbake/bin/webhob new file +95 -0
@@ -0,0 +1,95 @@ +#!/bin/bash +# (c) 2013 Intel Corp. + +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +# This script enables webhob event logging and +# starts bitbake resident server +# use as: source webhob [start|stop] + +# Helper function to kill a background webhob development server + +function webserverKillAllComponents() +{ + local pidfile + for pidfile in ${BUILDDIR}/whbmain.pid; do + if [ -f ${pidfile} ]; then + while kill -0 $(< ${pidfile}) 2>/dev/null; do + kill -SIGTERM -`ps -p $(< ${pidfile}) -o "%r" --no-headers` + sleep 1; + done; + rm ${pidfile} + fi + done +} + +# We make sure we're running in the current shell and in a good environment + +if [ -z "$ZSH_NAME" ] && [ `basename "$0"` = `basename $BASH_SOURCE` ]; then + echo "Error: This script needs to be sourced. Please run as 'source webhob [start|stop]'" 1>&2; + exit 1 +fi + +if [ -z "$BUILDDIR" ] || [ -z `which bitbake` ]; then + echo "Error: Build environment is not setup or bitbake is not in path." 1>&2; + return 2 +fi + +BBBASEDIR=`dirname ${BASH_SOURCE}`/.. + +# Determine the action. If specified by arguments, fine, if not, toggle it +if [ "x$1" == "xstart" ] || [ "x$1" == "xstop" ]; then + CMD="$1" +else + if [ -z "$BBSERVER" ]; then + CMD="start" + else + CMD="stop" + fi; +fi + +# Make sure it's safe to run by checking bitbake lock + +lock=1 +(flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0 + +if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); then + echo "Error: bitbake lock state error. System may be already on." 2>&1 + return 3 +elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then + echo "Error: bitbake lock state error. System may be already off. +manually stop system with bitbake -m / webserverKillAllComponents" 2>&1 + return 3 +fi + + +# Execute the commands + +case $CMD in + start ) + unset BBSERVER + python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0) + bitbake --server-only -t xmlrpc -B localhost:8200 + export BBSERVER=localhost:8200 + python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid + ;; + stop ) + bitbake -m + unset BBSERVER + webserverKillAllComponents +esac + +
Signature · 1 line
1.8.1.2
PATCH 3/94 webhob: create Django models for webhob
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
Patch adds the models we need in order to store all the information we need about the builds in the database using the Django ORM.
--- bitbake/lib/webhob/orm/__init__.py | 0 bitbake/lib/webhob/orm/models.py | 162 +++++++++++++++++++++++++++++++++ bitbake/lib/webhob/orm/tests.py | 16 ++++ bitbake/lib/webhob/orm/views.py | 1 + bitbake/lib/webhob/whbmain/settings.py | 1 + 5 files changed, 180 insertions(+) create mode 100644 bitbake/lib/webhob/orm/__init__.py create mode 100644 bitbake/lib/webhob/orm/models.py create mode 100644 bitbake/lib/webhob/orm/tests.py create mode 100644 bitbake/lib/webhob/orm/views.py
bitbake/lib/webhob/orm/__init__.py new file +0 -0
bitbake/lib/webhob/orm/models.py new file +162 -0
162 lines not on this page · raw message
bitbake/lib/webhob/orm/tests.py new file +16 -0
@@ -0,0 +1,16 @@ +""" +This file demonstrates writing tests using the unittest module. These will pass +when you run "manage.py test". + +Replace this with more appropriate tests for your application. +""" + +from django.test import TestCase + + +class SimpleTest(TestCase): + def test_basic_addition(self): + """ + Tests that 1 + 1 always equals 2. + """ + self.assertEqual(1 + 1, 2)
bitbake/lib/webhob/orm/views.py new file +1 -0
@@ -0,0 +1 @@ +# Create your views here.
bitbake/lib/webhob/whbmain/settings.py +1 -0
@@ -123,6 +123,7 @@ INSTALLED_APPS = ( # 'django.contrib.admin', # Uncomment the next line to enable admin documentation: # 'django.contrib.admindocs', + 'orm', ) # A sample logging configuration. The only tangible logging
Signature · 1 line
1.8.1.2
PATCH 4/94 bitbake: create Data Store Interface (DSI) file
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
The patch adds the first version of the DSI file. It uses the Knotty code for now and inserts task related information into the database using the Django ORM.
It makes use of some test objects needed to create a valid record into the database based on the definition of the Django models.
--- bitbake/lib/bb/ui/dsi.py | 667 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 667 insertions(+) create mode 100644 bitbake/lib/bb/ui/dsi.py
bitbake/lib/bb/ui/dsi.py new file +667 -0
@@ -0,0 +1,667 @@ +# +# BitBake DSI Implementation +# +# Copyright (C) 2013 Intel Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +from __future__ import division + +import os +import sys +import xmlrpclib +import logging +import progressbar +import signal +import bb.msg +import time +import fcntl +import struct +import copy +import datetime + +from bb.ui import uihelper + +logger = logging.getLogger("BitBake") +interactive = sys.stdout.isatty() + +class WebHOBHelper(object): + + def __init__(self): + self.configure_django() + self.task_order = 0 + self.tasks_information = {} + + def configure_django(self): + import webhob.whbmain.settings as whb_django_settings + from django.core.management import setup_environ + setup_environ(whb_django_settings) + # Add webhob to sys path for importing modules + sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob')) + + def store_build(self, uuid): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Builds, Machines, Layers + machine_obj = Machines.objects.create( + name='Test Machine', + description='This is our great test machine' + ) + machine_obj.save() + + layer_obj = Layers.objects.create( + name='Test Layer', + branch='master_test', + commit='2148u323nsau83smdmbnca3312f', + priority=2, + link_to_oe_core='http://test.oecore.com' + ) + layer_obj.save() + + build_obj = Builds.objects.create( + uuid=uuid, + target='test target', + host_machine='Host machine', + distro='1.234', + distro_version='v2', + host_system='Test hosting system', + host_distribution='Test host distrib', + target_system=machine_obj, + started_on=datetime.datetime.now(), + completed_on=datetime.datetime.now(), + outcome=3, + errors_no=0, + warnings_no=0, + cpu_usage=54, + disk_io=23.3232, + cooker_log_path='/home/test/cooker_test', + build_name='tester_build', + bitbake_version='1.0.5') + build_obj.layers.add(layer_obj) + build_obj.save() + + return build_obj, layer_obj + + def store_started_task(self, event, build_object): + self.task_order += 1 + identifier = event._package + event._task + self.tasks_information[identifier] = { + 'uuid': build_object, + 'task_executed': True, + 'order': self.task_order, + 'recipe': event._package, + 'task_name': event._task, + 'start_time': time.time(), + } + + def write_stored_tasks(self, event, build_object, layer_object): + identifier = event._package + event._task + + # fix this + if event.getDisplayName().lower() == 'succeded': + outcome = 0 + else: + outcome = 1 + + self.tasks_information[identifier].update({ + 'outcome': outcome, + 'end_time': time.time(), + }) + + task = self.tasks_information[identifier] + + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Tasks, Recipes + + recipe_obj = Recipes.objects.create( + name=task['recipe'], + version='1.0', + layer=layer_object, + summary='Short summary', + description='Longer description', + section='24', + license='license title', + licensing_info='longer license information', + homepage='http://homepage.com', + bugtracker='2456', + author='Mr.Test', + file_path='home/test/testproject/', + ) + + # Store finished task + task_obj = Tasks.objects.create(uuid=task['uuid'], + order=task['order'], + outcome=task['outcome'], + recipe=recipe_obj, + task_name=task['task_name'], + elapsed_time=task['end_time'] - task['start_time'], + # The next lines are just for testing reasons + sstate_checksum = '2134123sdscsdf232342tgfdv', + task_executed=True, + path_to_sstate_obj='/home/calin', + source_url='/', + log_file='/', + work_directory='/', + script_type=0, + file_path='/', + line_number=55, + py_stack_trace='Testing traceback', + disk_io = 23.55633, + cpu_usage=43, + errors_no=0, + warnings_no=0, + error='None', + warning='None', + sstate_result=0, + diffsigs="Testing diffsigs") + task_obj.save() + # Clean memory + del self.tasks_information[identifier] + + +class BBProgress(progressbar.ProgressBar): + def __init__(self, msg, maxval): + self.msg = msg + widgets = [progressbar.Percentage(), ' ', progressbar.Bar(), ' ', + progressbar.ETA()] + + try: + self._resize_default = signal.getsignal(signal.SIGWINCH) + except: + self._resize_default = None + progressbar.ProgressBar.__init__(self, maxval, [self.msg + ": "] + widgets) + + def _handle_resize(self, signum, frame): + progressbar.ProgressBar._handle_resize(self, signum, frame) + if self._resize_default: + self._resize_default(signum, frame) + def finish(self): + progressbar.ProgressBar.finish(self) + if self._resize_default: + signal.signal(signal.SIGWINCH, self._resize_default) + +class NonInteractiveProgress(object): + fobj = sys.stdout + + def __init__(self, msg, maxval): + self.msg = msg + self.maxval = maxval + + def start(self): + self.fobj.write("%s..." % self.msg) + self.fobj.flush() + return self + + def update(self, value): + pass + + def finish(self): + self.fobj.write("done.\n") + self.fobj.flush() + +def new_progress(msg, maxval): + if interactive: + return BBProgress(msg, maxval) + else: + return NonInteractiveProgress(msg, maxval) + +def pluralise(singular, plural, qty): + if(qty == 1): + return singular % qty + else: + return plural % qty + + +class InteractConsoleLogFilter(logging.Filter): + def __init__(self, tf, format): + self.tf = tf + self.format = format + + def filter(self, record): + if record.levelno == self.format.NOTE and (record.msg.startswith("Running") or record.msg.startswith("recipe ")): + return False + self.tf.clearFooter() + return True + +class TerminalFilter(object): + columns = 80 + + def sigwinch_handle(self, signum, frame): + self.columns = self.getTerminalColumns() + if self._sigwinch_default: + self._sigwinch_default(signum, frame) + + def getTerminalColumns(self): + def ioctl_GWINSZ(fd): + try: + cr = struct.unpack('hh', fcntl.ioctl(fd, self.termios.TIOCGWINSZ, '1234')) + except: + return None + return cr + cr = ioctl_GWINSZ(sys.stdout.fileno()) + if not cr: + try: + fd = os.open(os.ctermid(), os.O_RDONLY) + cr = ioctl_GWINSZ(fd) + os.close(fd) + except: + pass + if not cr: + try: + cr = (env['LINES'], env['COLUMNS']) + except: + cr = (25, 80) + return cr[1] + + def __init__(self, main, helper, console, format): + self.main = main + self.helper = helper + self.cuu = None + self.stdinbackup = None + self.interactive = sys.stdout.isatty() + self.footer_present = False + self.lastpids = [] + + if not self.interactive: + return + + try: + import curses + except ImportError: + sys.exit("FATAL: The knotty ui could not load the required curses python module.") + + import termios + self.curses = curses + self.termios = termios + try: + fd = sys.stdin.fileno() + self.stdinbackup = termios.tcgetattr(fd) + new = copy.deepcopy(self.stdinbackup) + new[3] = new[3] & ~termios.ECHO + termios.tcsetattr(fd, termios.TCSADRAIN, new) + curses.setupterm() + if curses.tigetnum("colors") > 2: + format.enable_color() + self.ed = curses.tigetstr("ed") + if self.ed: + self.cuu = curses.tigetstr("cuu") + try: + self._sigwinch_default = signal.getsignal(signal.SIGWINCH) + signal.signal(signal.SIGWINCH, self.sigwinch_handle) + except: + pass + self.columns = self.getTerminalColumns() + except: + self.cuu = None + console.addFilter(InteractConsoleLogFilter(self, format)) + + def clearFooter(self): + if self.footer_present: + lines = self.footer_present + sys.stdout.write(self.curses.tparm(self.cuu, lines)) + sys.stdout.write(self.curses.tparm(self.ed)) + self.footer_present = False + + def updateFooter(self): + if not self.cuu: + return + activetasks = self.helper.running_tasks + failedtasks = self.helper.failed_tasks + runningpids = self.helper.running_pids + if self.footer_present and (self.lastcount == self.helper.tasknumber_current) and (self.lastpids == runningpids): + return + if self.footer_present: + self.clearFooter() + if (not self.helper.tasknumber_total or self.helper.tasknumber_current == self.helper.tasknumber_total) and not len(activetasks): + return + tasks = [] + for t in runningpids: + tasks.append("%s (pid %s)" % (activetasks[t]["title"], t)) + + if self.main.shutdown: + content = "Waiting for %s running tasks to finish:" % len(activetasks) + elif not len(activetasks): + content = "No currently running tasks (%s of %s)" % (self.helper.tasknumber_current, self.helper.tasknumber_total) + else: + content = "Currently %s running tasks (%s of %s):" % (len(activetasks), self.helper.tasknumber_current, self.helper.tasknumber_total) + print(content) + lines = 1 + int(len(content) / (self.columns + 1)) + for tasknum, task in enumerate(tasks): + content = "%s: %s" % (tasknum, task) + print(content) + lines = lines + 1 + int(len(content) / (self.columns + 1)) + self.footer_present = lines + self.lastpids = runningpids[:] + self.lastcount = self.helper.tasknumber_current + + def finish(self): + if self.stdinbackup: + fd = sys.stdin.fileno() + self.termios.tcsetattr(fd, self.termios.TCSADRAIN, self.stdinbackup) + +def main(server, eventHandler, params, tf = TerminalFilter): + print "This is running in DSI mode" + + # Generate an unique ID for this build + # TODO: for multiple build commands this might not work as expected + import uuid + uuid = sessionid = str(uuid.uuid4()) + + # Get values of variables which control our output + includelogs, error = server.runCommand(["getVariable", "BBINCLUDELOGS"]) + if error: + logger.error("Unable to get the value of BBINCLUDELOGS variable: %s" % error) + return 1 + loglines, error = server.runCommand(["getVariable", "BBINCLUDELOGS_LINES"]) + if error: + logger.error("Unable to get the value of BBINCLUDELOGS_LINES variable: %s" % error) + return 1 + consolelogfile, error = server.runCommand(["getVariable", "BB_CONSOLELOG"]) + if error: + logger.error("Unable to get the value of BB_CONSOLELOG variable: %s" % error) + return 1 + + if sys.stdin.isatty() and sys.stdout.isatty(): + log_exec_tty = True + else: + log_exec_tty = False + + helper = uihelper.BBUIHelper() + + console = logging.StreamHandler(sys.stdout) + format_str = "%(levelname)s: %(message)s" + format = bb.msg.BBLogFormatter(format_str) + bb.msg.addDefaultlogFilter(console) + console.setFormatter(format) + logger.addHandler(console) + + if consolelogfile and not params.options.show_environment: + bb.utils.mkdirhier(os.path.dirname(consolelogfile)) + conlogformat = bb.msg.BBLogFormatter(format_str) + consolelog = logging.FileHandler(consolelogfile) + bb.msg.addDefaultlogFilter(consolelog) + consolelog.setFormatter(conlogformat) + logger.addHandler(consolelog) + + try: + params.updateFromServer(server) + cmdline = params.parseActions() + if not cmdline: + print("Nothing to do. Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information.") + return 1 + if 'msg' in cmdline and cmdline['msg']: + logger.error(cmdline['msg']) + return 1 + + ret, error = server.runCommand(cmdline['action']) + if error: + logger.error("Command '%s' failed: %s" % (cmdline, error)) + return 1 + elif ret != True: + logger.error("Command '%s' failed: returned %s" % (cmdline, ret)) + return 1 + except xmlrpclib.Fault as x: + logger.error("XMLRPC Fault getting commandline:\n %s" % x) + return 1 + + parseprogress = None + cacheprogress = None + main.shutdown = 0 + interrupted = False + return_value = 0 + errors = 0 + warnings = 0 + taskfailures = [] + wbhbhelper = WebHOBHelper() + build_object, layer_object = wbhbhelper.store_build(uuid) + + termfilter = tf(main, helper, console, format) + + while True: + try: + termfilter.updateFooter() + event = eventHandler.waitEvent(0.25) + + if event is None: + if main.shutdown > 1: + break + continue + + helper.eventHandler(event) + + if isinstance(event, bb.runqueue.runQueueExitWait): + if not main.shutdown: + main.shutdown = 1 + + if isinstance(event, bb.build.TaskStarted): + wbhbhelper.store_started_task(event, build_object) + + if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)): + wbhbhelper.write_stored_tasks(event, build_object, layer_object) + + if isinstance(event, bb.event.LogExecTTY): + if log_exec_tty: + tries = event.retries + while tries: + print("Trying to run: %s" % event.prog) + if os.system(event.prog) == 0: + break + time.sleep(event.sleep_delay) + tries -= 1 + if tries: + continue + logger.warn(event.msg) + continue + + if isinstance(event, logging.LogRecord): + if event.levelno >= format.ERROR: + errors = errors + 1 + return_value = 1 + elif event.levelno == format.WARNING: + warnings = warnings + 1 + # For "normal" logging conditions, don't show note logs from tasks + # but do show them if the user has changed the default log level to + # include verbose/debug messages + if event.taskpid != 0 and event.levelno <= format.NOTE: + continue + logger.handle(event) + continue + + if isinstance(event, bb.build.TaskFailed): + return_value = 1 + logfile = event.logfile + if logfile and os.path.exists(logfile): + termfilter.clearFooter() + bb.error("Logfile of failure stored in: %s" % logfile) + if includelogs and not event.errprinted: + print("Log data follows:") + f = open(logfile, "r") + lines = [] + while True: + l = f.readline() + if l == '': + break + l = l.rstrip() + if loglines: + lines.append(' | %s' % l) + if len(lines) > int(loglines): + lines.pop(0) + else: + print('| %s' % l) + f.close() + if lines: + for line in lines: + print(line) + if isinstance(event, bb.build.TaskBase): + logger.info(event._message) + continue + if isinstance(event, bb.event.ParseStarted): + if event.total == 0: + continue + parseprogress = new_progress("Parsing recipes", event.total).start() + continue + if isinstance(event, bb.event.ParseProgress): + parseprogress.update(event.current) + continue + if isinstance(event, bb.event.ParseCompleted): + if not parseprogress: + continue + + parseprogress.finish() + print(("Parsing of %d .bb files complete (%d cached, %d parsed). %d targets, %d skipped, %d masked, %d errors." + % ( event.total, event.cached, event.parsed, event.virtuals, event.skipped, event.masked, event.errors))) + continue + + if isinstance(event, bb.event.CacheLoadStarted): + cacheprogress = new_progress("Loading cache", event.total).start() + continue + if isinstance(event, bb.event.CacheLoadProgress): + cacheprogress.update(event.current) + continue + if isinstance(event, bb.event.CacheLoadCompleted): + cacheprogress.finish() + print("Loaded %d entries from dependency cache." % event.num_entries) + continue + + if isinstance(event, bb.event.MultipleProviders): + logger.info("multiple providers are available for %s%s (%s)", event._is_runtime and "runtime " or "", + event._item, + ", ".join(event._candidates)) + logger.info("consider defining a PREFERRED_PROVIDER entry to match %s", event._item) + continue + if isinstance(event, bb.event.NoProvider): + return_value = 1 + errors = errors + 1 + if event._runtime: + r = "R" + else: + r = "" + + if event._dependees: + logger.error("Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)", r, event._item, ", ".join(event._dependees), r) + else: + logger.error("Nothing %sPROVIDES '%s'", r, event._item) + if event._reasons: + for reason in event._reasons: + logger.error("%s", reason) + continue + + if isinstance(event, bb.runqueue.sceneQueueTaskStarted): + logger.info("Running setscene task %d of %d (%s)" % (event.stats.completed + event.stats.active + event.stats.failed + 1, event.stats.total, event.taskstring)) + continue + + if isinstance(event, bb.runqueue.runQueueTaskStarted): + if event.noexec: + tasktype = 'noexec task' + else: + tasktype = 'task' + logger.info("Running %s %s of %s (ID: %s, %s)", + tasktype, + event.stats.completed + event.stats.active + + event.stats.failed + 1, + event.stats.total, event.taskid, event.taskstring) + continue + + if isinstance(event, bb.runqueue.runQueueTaskFailed): + taskfailures.append(event.taskstring) + logger.error("Task %s (%s) failed with exit code '%s'", + event.taskid, event.taskstring, event.exitcode) + continue + + if isinstance(event, bb.runqueue.sceneQueueTaskFailed): + logger.warn("Setscene task %s (%s) failed with exit code '%s' - real task will be run instead", + event.taskid, event.taskstring, event.exitcode) + continue + + if isinstance(event, bb.event.ConfigParsed): + # timestamp should be added for this + continue + + if isinstance(event, bb.event.RecipeParsed): + # timestamp should be added for this + continue + + if isinstance(event, bb.event.OperationStarted): + # timestamp should be added for this + continue + + if isinstance(event, bb.event.OperationCompleted): + # timestamp should be added for this + # signal a complete operation + # calculate timing + continue + + if isinstance(event, bb.event.DiskFull): + # trigger an error + continue + + # ignore + if isinstance(event, (bb.event.BuildBase, + bb.event.StampUpdate, + bb.event.RecipePreFinalise, + bb.runqueue.runQueueEvent, + bb.runqueue.runQueueExitWait, + bb.event.OperationProgress, + bb.command.CommandFailed, + bb.command.CommandExit, + bb.command.CommandCompleted, + bb.cooker.CookerExit)): + continue + + logger.error("Unknown event: %s", event) + + except EnvironmentError as ioerror: + termfilter.clearFooter() + # ignore interrupted io + if ioerror.args[0] == 4: + pass + except KeyboardInterrupt: + termfilter.clearFooter() + if main.shutdown == 1: + print("\nSecond Keyboard Interrupt, stopping...\n") + _, error = server.runCommand(["stateStop"]) + if error: + logger.error("Unable to cleanly stop: %s" % error) + if main.shutdown == 0: + print("\nKeyboard Interrupt, closing down...\n") + interrupted = True + _, error = server.runCommand(["stateShutdown"]) + if error: + logger.error("Unable to cleanly shutdown: %s" % error) + main.shutdown = main.shutdown + 1 + pass + + summary = "" + if taskfailures: + summary += pluralise("\nSummary: %s task failed:", + "\nSummary: %s tasks failed:", len(taskfailures)) + for failure in taskfailures: + summary += "\n %s" % failure + if warnings: + summary += pluralise("\nSummary: There was %s WARNING message shown.", + "\nSummary: There were %s WARNING messages shown.", warnings) + if return_value: + summary += pluralise("\nSummary: There was %s ERROR message shown, returning a non-zero exit code.", + "\nSummary: There were %s ERROR messages shown, returning a non-zero exit code.", errors) + if summary: + print(summary) + + if interrupted: + print("Execution was interrupted, returning a non-zero exit code.") + if return_value == 0: + return_value = 1 + + termfilter.finish() + + return return_value +
Signature · 1 line
1.8.1.2
PATCH 5/94 bitbake: webhob: updates to Django models
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
Based on recent discussions with the team I have updated the Django models we had in order to store information more clearly. The updates include: renaming of some tables, removing of unnecessary fields, adding supplementary tables to reflect relations between objects.
--- bitbake/lib/webhob/orm/models.py | 94 ++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 47 deletions(-)
bitbake/lib/webhob/orm/models.py +47 -47
@@ -1,22 +1,18 @@ from django.db import models -class Builds(models.Model): +class Build(models.Model): BUILD_OUTCOME = ( - (0, 'Succeded'), + (0, 'Succeeded'), (1, 'Failed'), ) uuid = models.CharField(max_length=100, unique=True) target = models.CharField(max_length=100) - host_machine = models.CharField(max_length=100) + machine = models.ForeignKey('Machine', related_name='build_machine') distro = models.CharField(max_length=100) distro_version = models.CharField(max_length=100) - host_system = models.CharField(max_length=100) - host_distribution = models.CharField(max_length=100) - target_system = models.ForeignKey('Machines', related_name='builds_machines') - layers = models.ManyToManyField('Layers') started_on = models.DateTimeField() completed_on = models.DateTimeField() outcome = models.IntegerField(choices=BUILD_OUTCOME)
@@ -29,7 +25,7 @@ class Builds(models.Model): bitbake_version = models.CharField(max_length=50) -class Tasks(models.Model): +class Task(models.Model): SSTATE_RESULT = ( (0, 'Not Applicable'), # For rest of tasks, but they still need checking.
@@ -47,23 +43,23 @@ class Tasks(models.Model): (0, 'Covered'), (1, 'Sstate'), (2, 'Existing'), - (3, 'Succeded'), + (3, 'Succeeded'), (4, 'Failed'), ) - uuid = models.ForeignKey(Builds, to_field='uuid', related_name='tasks_builds') + build = models.ForeignKey(Build, related_name='task_build') order = models.IntegerField() task_executed = models.BooleanField() # True means Executed, False means Prebuilt outcome = models.IntegerField(choices=TASK_OUTCOME) sstate_checksum = models.CharField(max_length=100) path_to_sstate_obj = models.FilePathField(max_length=500, blank=True) - recipe = models.ForeignKey('Recipes', related_name='builds_recipes') + recipe = models.ForeignKey('Recipe', related_name='build_recipe') task_name = models.CharField(max_length=100) - source_url = models.FilePathField(max_length=200) - log_file = models.FilePathField(max_length=200, blank=True) - work_directory = models.FilePathField(max_length=200) + source_url = models.FilePathField(max_length=255) + log_file = models.FilePathField(max_length=255, blank=True) + work_directory = models.FilePathField(max_length=255) script_type = models.IntegerField(choices=TASK_CODING) - file_path = models.FilePathField(max_length=200) + file_path = models.FilePathField(max_length=255) line_number = models.IntegerField() py_stack_trace = models.TextField() disk_io = models.DecimalField(max_digits=20, decimal_places=10)
@@ -74,54 +70,52 @@ class Tasks(models.Model): error = models.TextField() warning = models.TextField() sstate_result = models.IntegerField(choices=SSTATE_RESULT) - diffsigs = models.CharField(max_length=500) -class Task_Dependencies(models.Model): - task = models.ForeignKey(Tasks, related_name='task_dependencies_task') - depends_on = models.ForeignKey(Tasks, related_name='task_dependencies_depends') +class Task_Dependency(models.Model): + task = models.ForeignKey(Task, related_name='task_dependencies_task') + depends_on = models.ForeignKey(Task, related_name='task_dependencies_depends') -class Targets(models.Model): - uuid = models.ForeignKey(Builds, to_field='uuid', related_name='targets_builds') +class Target(models.Model): + build = models.ForeignKey(Build, related_name='target_build') is_image = models.BooleanField() -class Artifacts(models.Model): - uuid = models.ForeignKey(Builds, to_field='uuid', related_name='artifacts_builds') - target = models.ForeignKey(Targets, related_name='artifacts_targets') +class Artifact(models.Model): + build = models.ForeignKey(Build, related_name='artifact_build') + target = models.ForeignKey(Target, related_name='artifact_target') file_name = models.CharField(max_length=100) file_size = models.IntegerField() -class Packages_In_Images(models.Model): - package = models.ForeignKey('Packages', related_name='packages_in_images_package') - target = models.ForeignKey(Targets, related_name='packages_in_images_target') +class Package_In_Image(models.Model): + package = models.ForeignKey('Package', related_name='package_in_image_package') + target = models.ForeignKey(Target, related_name='package_in_image_target') -class Packages(models.Model): - package_id = models.IntegerField() - recipe = models.ForeignKey('Recipes', related_name='packages_recipes') +class Package(models.Model): + recipe = models.ForeignKey('Recipe', related_name='package_recipe') name = models.CharField(max_length=100) version = models.CharField(max_length=100) size = models.IntegerField() -class Package_Dependencies(models.Model): - package = models.ForeignKey(Packages, related_name='package_dependencies_package') - depends_on = models.ForeignKey(Packages, related_name='package_dependencies_depends') +class Package_Dependency(models.Model): + package = models.ForeignKey(Package, related_name='package_dependencies_package') + depends_on = models.ForeignKey(Package, related_name='package_dependencies_depends') class Filelist(models.Model): - package = models.ForeignKey(Packages, related_name='filelist_packages') - complete_file_path = models.FilePathField(max_length=200, blank=True) + package = models.ForeignKey(Package, related_name='filelist_package') + complete_file_path = models.FilePathField(max_length=255, blank=True) file_size = models.IntegerField() -class Recipes(models.Model): +class Recipe(models.Model): name = models.CharField(max_length=100) version = models.CharField(max_length=100) - layer = models.ForeignKey('Layers', related_name='recipes_layers') + layer = models.ForeignKey('Build_Layer', related_name='recipe_build_layer') summary = models.CharField(max_length=100) description = models.CharField(max_length=100) section = models.CharField(max_length=100)
@@ -130,33 +124,39 @@ class Recipes(models.Model): homepage = models.URLField() bugtracker = models.URLField() author = models.CharField(max_length=100) - file_path = models.FilePathField(max_length=200) + file_path = models.FilePathField(max_length=255) -class Recipe_Dependencies(models.Model): - recipe = models.ForeignKey(Recipes, related_name='recipe_dependencies_recipe') - depends_on = models.ForeignKey(Recipes, related_name='recipe_dependencies_depends') +class Recipe_Dependency(models.Model): + recipe = models.ForeignKey(Recipe, related_name='recipe_dependencies_recipe') + depends_on = models.ForeignKey(Recipe, related_name='recipe_dependencies_depends') -class Layers(models.Model): +class Layer(models.Model): name = models.CharField(max_length=100) + local_path = models.FilePathField(max_length=255) + layer_index_url = models.URLField() + + +class Build_Layer(models.Model): + build = models.ForeignKey(Build, related_name='build_layer_build') + layer = models.ForeignKey(Layer, related_name='build_layer_layer') branch = models.CharField(max_length=50) commit = models.CharField(max_length=100) priority = models.IntegerField() - link_to_oe_core = models.URLField() -class Variables(models.Model): - uuid = models.ForeignKey(Builds, to_field='uuid', related_name='variables_builds') +class Variable(models.Model): + build = models.ForeignKey(Build, related_name='variable_build') variable_name = models.CharField(max_length=100) variable_value = models.TextField() - file = models.FilePathField(max_length=200) + file = models.FilePathField(max_length=255) changed = models.BooleanField() human_readable_name = models.CharField(max_length=200) description = models.TextField() -class Machines(models.Model): +class Machine(models.Model): name = models.CharField(max_length=100) description = models.TextField()
Signature · 1 line
1.8.1.2
PATCH 6/94 bitbake: webhob: make DSI store build information
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch will enable DSI to store build information defined in the Django models. This will be done for each executed build and it will use a helper file to save this information.
Moved code from DSI interface to the helper file in to classes, one which wraps around Django model interface operations, and one that fills in information from the event system into the model data.
--- bitbake/lib/bb/ui/buildinfohelper.py | 127 ++++++++++++++++++++++++ bitbake/lib/bb/ui/dsi.py | 180 ++++------------------------------- 2 files changed, 148 insertions(+), 159 deletions(-) create mode 100644 bitbake/lib/bb/ui/buildinfohelper.py
bitbake/lib/bb/ui/buildinfohelper.py new file +127 -0
@@ -0,0 +1,127 @@ +import datetime +import sys +import uuid + + +class ORMWrapper(object): + """ This class creates the dictionaries needed to store information in the database + following the format defined by the Django models. It is also used to save this + information in the database. + """ + + def __init__(self): + self.uuid = None + + def get_machine_information(self, server): + machine_info = {} + + machine_info['name'] = server.runCommand(["getVariable", "MACHINE"])[0] + machine_info['description'] = 'Not Available' + + return machine_info + + def create_machine_object(self, machine_information): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Machine + + machine = Machine.objects.get_or_create(name=machine_information['name'], + description=machine_information['description']) + + return machine[0] + + def get_build_information(self, server, machine_obj): + build_info = {} + + # Generate an identifier for each new build + self.uuid = str(uuid.uuid4()) + + build_info['uuid'] = self.uuid + build_info['target'] = 'Not Available' + build_info['machine'] = machine_obj + build_info['distro'] = server.runCommand(["getVariable", "DISTRO"])[0] + build_info['distro_version'] = server.runCommand(["getVariable", "DISTRO_VERSION"])[0] + build_info['started_on'] = datetime.datetime.now() + build_info['completed_on'] = datetime.datetime.now() + build_info['outcome'] = 0 + build_info['number_of_errors'] = 0 + build_info['number_of_warnings'] = 0 + build_info['cpu_usage'] = 0 + build_info['disk_io'] = 0 + build_info['cooker_log_path'] = server.runCommand(["getVariable", "BB_CONSOLELOG"])[0] + build_info['build_name'] = server.runCommand(["getVariable", "BUILDNAME"])[0] + build_info['bitbake_version'] = server.runCommand(["getVariable", "BB_VERSION"])[0] + + return build_info + + def create_build_object(self, build_info): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Build + + build = Build.objects.create(uuid=build_info['uuid'], + target=build_info['target'], + machine=build_info['machine'], + distro=build_info['distro'], + distro_version=build_info['distro_version'], + started_on=build_info['started_on'], + completed_on=build_info['completed_on'], + outcome=build_info['outcome'], + errors_no=build_info['number_of_errors'], + warnings_no=build_info['number_of_warnings'], + cpu_usage=build_info['cpu_usage'], + disk_io=build_info['disk_io'], + cooker_log_path=build_info['cooker_log_path'], + build_name=build_info['build_name'], + bitbake_version=build_info['bitbake_version']) + + return build + + def update_build_object(self, build_obj, errors, warnings): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Build + + outcome = 0 + if errors or warnings: + outcome = 1 + + build = Build.objects.get(uuid=build_obj.uuid) + build.completed_on = datetime.datetime.now() + build.errors_no = errors + build.warnings_no = warnings + build.outcome = outcome + build.save() + + return build_obj + + +class BuildInfoHelper(object): + """ This class gathers the build information from the server and sends it + towards the ORM wrapper for storing in the database + """ + + def __init__(self): + self.configure_django() + self.task_order = 0 + self.tasks_information = {} + self.uuid = None + self.transport_utils = {} + self.orm_wrapper = ORMWrapper() + + def configure_django(self): + import webhob.whbmain.settings as whb_django_settings + from django.core.management import setup_environ + setup_environ(whb_django_settings) + # Add webhob to sys path for importing modules + sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob')) + + def store_started_build(self, event, server): + + machine_information = self.orm_wrapper.get_machine_information(server) + machine_obj = self.orm_wrapper.create_machine_object(machine_information) + + build_information = self.orm_wrapper.get_build_information(server, machine_obj) + build_obj = self.orm_wrapper.create_build_object(build_information) + self.transport_utils['build'] = build_obj + + def update_build_information(self, event, errors, warnings): + build_obj = self.orm_wrapper.update_build_object(self.transport_utils['build'], errors, warnings) + del self.transport_utils['build']
bitbake/lib/bb/ui/dsi.py +21 -159
240 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH 7/94 bitbake: webhob: make DSI store task information
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch will enable DSI to store task related information. Together with the task object, other objects need to be obtained or created in order to respect the Django models defined.
--- bitbake/lib/bb/ui/buildinfohelper.py | 235 +++++++++++++++++++++++++++++++++-- bitbake/lib/bb/ui/dsi.py | 8 +- 2 files changed, 231 insertions(+), 12 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +224 -11
@@ -11,6 +11,8 @@ class ORMWrapper(object): def __init__(self): self.uuid = None + self.task_order = 0 + self.transport_utils = {} def get_machine_information(self, server): machine_info = {}
@@ -75,12 +77,12 @@ class ORMWrapper(object): return build - def update_build_object(self, build_obj, errors, warnings): + def update_build_object(self, build_obj, errors, warnings, taskfailures): # This needs to be imported after we have configured the Django settings file from webhob.orm.models import Build outcome = 0 - if errors or warnings: + if errors or taskfailures: outcome = 1 build = Build.objects.get(uuid=build_obj.uuid)
@@ -90,7 +92,212 @@ class ORMWrapper(object): build.outcome = outcome build.save() - return build_obj + def get_task_information(self, event): + + self.task_order += 1 + + recipe = self.get_recipe_object(event) + + task_information = {} + task_information['build'] = self.transport_utils['build'] + task_information['order'] = self.task_order + task_information['task_executed'] = True + task_information['outcome'] = 0 + task_information['sstate_checksum'] = 'Not Available' + task_information['path_to_sstate_obj'] = 'Not Available' + task_information['recipe'] = recipe + task_information['task_name'] = event._task + task_information['source_url'] = 'Not Available' + task_information['log_file'] = 'Not Available' + task_information['work_directory'] = 'Not Available' + task_information['script_type'] = 0 + task_information['file_path'] = 'Not Available' + task_information['line_number'] = 0 + task_information['py_stack_trace'] = 'Not Available' + task_information['disk_io'] = 0 + task_information['cpu_usage'] = 0 + task_information['elapsed_time'] = 'Not Available' + task_information['errors_no'] = 0 + task_information['warnings_no'] = 0 + task_information['error'] = 'Not Available' + task_information['warning'] = 'Not Available' + task_information['sstate_result'] = 0 + + return task_information + + def create_task_object(self, task_information): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Task + + identifier = task_information['recipe'].name + task_information['task_name'] + + task_object = Task.objects.get_or_create( + build=task_information['build'], + order=task_information['order'], + task_executed=task_information['task_executed'], + outcome=task_information['outcome'], + sstate_checksum=task_information['sstate_checksum'], + path_to_sstate_obj=task_information['path_to_sstate_obj'], + recipe=task_information['recipe'], + task_name=task_information['task_name'], + source_url=task_information['source_url'], + log_file=task_information['log_file'], + work_directory=task_information['work_directory'], + script_type=task_information['script_type'], + file_path=task_information['file_path'], + line_number=task_information['line_number'], + py_stack_trace=task_information['py_stack_trace'], + disk_io=task_information['disk_io'], + cpu_usage=task_information['cpu_usage'], + elapsed_time=task_information['elapsed_time'], + errors_no=task_information['errors_no'], + warnings_no=task_information['warnings_no'], + error=task_information['error'], + warning=task_information['warning'], + sstate_result=task_information['sstate_result']) + task_object[0].save() + + self.transport_utils[identifier] = {'object': task_object[0], 'start_time': datetime.datetime.now()} + + return task_object[0] + + def update_task_object(self, task_dictionary, event): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Task + + task_object = task_dictionary['object'] + + task = Task.objects.get(build=task_object.build, + task_name=task_object.task_name, + recipe=task_object.recipe) + + duration = datetime.datetime.now() - task_dictionary['start_time'] + task.elapsed_time = duration.total_seconds() + + outcome_info = self.get_outcome_of_task(event) + task.outcome = outcome_info['task_outcome'] + + if outcome_info.get('error', ''): + task.error = outcome_info['error'] + + #TODO: get error number + #TODO: get warnings number + #TODO: get warning information + + task.save() + + def get_outcome_of_task(self, event): + from webhob.orm.models import Task + + outcome_info = {} + + task_result = event.getDisplayName().lower() + for outcome in Task.TASK_OUTCOME: + if task_result == outcome[1].lower(): + task_outcome = outcome[0] + + outcome_info['task_outcome'] = task_outcome + + if isinstance(event, bb.build.TaskFailed): + if event.logfile and os.path.exists(event.logfile): + outcome_info['error'] = open(event.logfile, "r").read() + + return outcome_info + + def get_recipe_object(self, event): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Recipe + + recipe_information = self.get_recipe_information(event) + + recipe_object = Recipe.objects.get_or_create( + name=recipe_information['name'], + version=recipe_information['version'], + layer=recipe_information['layer'], + summary=recipe_information['summary'], + description=recipe_information['description'], + section=recipe_information['section'], + license=recipe_information['license'], + licensing_info=recipe_information['licensing_info'], + homepage=recipe_information['homepage'], + bugtracker=recipe_information['bugtracker'], + author=recipe_information['author'], + file_path=recipe_information['file_path']) + recipe_object[0].save() + + return recipe_object[0] + + def get_recipe_information(self, event): + + build_layer_obj = self.get_build_layer_object(event) + + recipe_info = {} + recipe_info['name'] = event._package + recipe_info['version'] = 'Not Available' + recipe_info['layer'] = build_layer_obj + recipe_info['summary'] = 'Not Available' + recipe_info['description'] = 'Not Available' + recipe_info['section'] = 'Not Available' + recipe_info['license'] = 'Not Available' + recipe_info['licensing_info'] = 'Not Available' + recipe_info['homepage'] = 'Not Available' + recipe_info['bugtracker'] = 'Not Available' + recipe_info['author'] = 'Not Available' + recipe_info['file_path'] = 'Not Available' + + return recipe_info + + def get_build_layer_object(self, event): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Build_Layer + + build_layer_information = self.get_build_layer_information(event) + + build_layer_object = Build_Layer.objects.get_or_create( + build = build_layer_information['build'], + layer = build_layer_information['layer'], + branch = build_layer_information['branch'], + commit = build_layer_information['commit'], + priority = build_layer_information['priority']) + build_layer_object[0].save() + + return build_layer_object[0] + + def get_build_layer_information(self, event): + + layer_object = self.get_layer_object(event) + + build_layer_info = {} + build_layer_info['build'] = self.transport_utils['build'] + build_layer_info['layer'] = layer_object + build_layer_info['branch'] = 'Not Available' + build_layer_info['commit'] = 'Not Available' + build_layer_info['priority'] = 0 + + return build_layer_info + + def get_layer_object(self, event): + # This needs to be imported after we have configured the Django settings file + from webhob.orm.models import Layer + + layer_information = self.get_layer_information(event) + + layer_object = Layer.objects.get_or_create( + name=layer_information['name'], + local_path=layer_information['local_path'], + layer_index_url=layer_information['layer_index_url']) + layer_object[0].save() + + return layer_object[0] + + def get_layer_information(self, event): + + layer_info = {} + layer_info['name'] = 'Layer Test Name' + layer_info['local_path'] = 'Not Available' + layer_info['layer_index_url'] = 'Not Available' + + return layer_info class BuildInfoHelper(object):
@@ -100,10 +307,6 @@ class BuildInfoHelper(object): def __init__(self): self.configure_django() - self.task_order = 0 - self.tasks_information = {} - self.uuid = None - self.transport_utils = {} self.orm_wrapper = ORMWrapper() def configure_django(self):
@@ -120,8 +323,18 @@ class BuildInfoHelper(object): build_information = self.orm_wrapper.get_build_information(server, machine_obj) build_obj = self.orm_wrapper.create_build_object(build_information) - self.transport_utils['build'] = build_obj + self.orm_wrapper.transport_utils['build'] = build_obj + + def update_build_information(self, event, errors, warnings, taskfailures): + self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures) + del self.orm_wrapper.transport_utils['build'] + + def store_started_task(self, event): + task_information = self.orm_wrapper.get_task_information(event) + task_obj = self.orm_wrapper.create_task_object(task_information) - def update_build_information(self, event, errors, warnings): - build_obj = self.orm_wrapper.update_build_object(self.transport_utils['build'], errors, warnings) - del self.transport_utils['build'] + def update_and_store_task(self, event): + identifier = event._package + event._task + task_dictionary = self.orm_wrapper.transport_utils[identifier] + self.orm_wrapper.update_task_object(task_dictionary, event) + del self.orm_wrapper.transport_utils[identifier]
bitbake/lib/bb/ui/dsi.py +7 -1
@@ -308,6 +308,12 @@ def main(server, eventHandler, params, tf = TerminalFilter): if isinstance(event, bb.event.BuildStarted): buildinfohelper.store_started_build(event, server) + if isinstance(event, bb.build.TaskStarted): + buildinfohelper.store_started_task(event) + + if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)): + buildinfohelper.update_and_store_task(event) + if isinstance(event, bb.event.LogExecTTY): if log_exec_tty: tries = event.retries
@@ -443,7 +449,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): continue if isinstance(event, (bb.command.CommandFailed, bb.command.CommandExit, bb.command.CommandCompleted, bb.cooker.CookerExit)): - buildinfohelper.update_build_information(event, errors, warnings) + buildinfohelper.update_build_information(event, errors, warnings, taskfailures) main.shutdown = 2 if isinstance(event, bb.event.ConfigParsed):
Signature · 1 line
1.8.1.2
PATCH 8/94 bitbake: webhob: view a table with all builds
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch creates a page in Webhob in which you can view detailed information about each executed build. There is data about the outcome of each build, execution times, hardware usage, logs, errors, warnings and others.
--- bitbake/lib/webhob/bldviewer/__init__.py | 0 bitbake/lib/webhob/bldviewer/templates/build.html | 54 ++++++++++++++++++++++ .../lib/webhob/bldviewer/templatetags/__init__.py | 0 .../webhob/bldviewer/templatetags/projecttags.py | 8 ++++ bitbake/lib/webhob/bldviewer/urls.py | 6 +++ bitbake/lib/webhob/bldviewer/views.py | 11 +++++ bitbake/lib/webhob/whbmain/urls.py | 2 +- 7 files changed, 80 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/webhob/bldviewer/__init__.py create mode 100644 bitbake/lib/webhob/bldviewer/templates/build.html create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/__init__.py create mode 100644 bitbake/lib/webhob/bldviewer/templatetags/projecttags.py create mode 100644 bitbake/lib/webhob/bldviewer/urls.py create mode 100644 bitbake/lib/webhob/bldviewer/views.py
bitbake/lib/webhob/bldviewer/__init__.py new file +0 -0
bitbake/lib/webhob/bldviewer/templates/build.html new file +54 -0
54 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/templatetags/__init__.py new file +0 -0
bitbake/lib/webhob/bldviewer/templatetags/projecttags.py new file +8 -0
@@ -0,0 +1,8 @@ +from datetime import datetime +from django import template + +register = template.Library() + [email protected]_tag +def time_difference(start_time, end_time): + return end_time - start_time
bitbake/lib/webhob/bldviewer/urls.py new file +6 -0
@@ -0,0 +1,6 @@ +from django.conf.urls import patterns, include, url + + +urlpatterns = patterns('bldviewer.views', + url(r'^$', 'build', name='build'), +)
bitbake/lib/webhob/bldviewer/views.py new file +11 -0
@@ -0,0 +1,11 @@ +from django.shortcuts import render +from orm.models import Build + + +def build(request): + template = 'build.html' + build_info = Build.objects.all() + + context = {'builds': build_info} + + return render(request, template, context)
bitbake/lib/webhob/whbmain/urls.py +1 -1
@@ -5,8 +5,8 @@ from django.conf.urls import patterns, include, url # admin.autodiscover() urlpatterns = patterns('', + url(r'^build/$', include('bldviewer.urls')), # Examples: - # url(r'^$', 'webhob.views.home', name='home'), # url(r'^webhob/', include('webhob.foo.urls')), # Uncomment the admin/doc line below to enable admin documentation:
Signature · 1 line
1.8.1.2
PATCH 9/94 bitbake: webhob: view a table with all the tasks of a build
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch will display all tasks executed for a build. There is detailed information about each task executed, if any. In case there were no tasks executed for a build, a warning message is displayed.
--- bitbake/lib/webhob/bldviewer/templates/task.html | 81 ++++++++++++++++++++++++ bitbake/lib/webhob/bldviewer/urls.py | 1 + bitbake/lib/webhob/bldviewer/views.py | 11 +++- bitbake/lib/webhob/whbmain/urls.py | 2 +- 4 files changed, 93 insertions(+), 2 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/task.html
bitbake/lib/webhob/bldviewer/templates/task.html new file +81 -0
@@ -0,0 +1,81 @@ +<html> + + <head> + <title>WebHob Task page</title> + </head> + +<body> + + <h1>WebHob Tasks</h1> + + {% if not tasks %} + <p>No tasks were executed in this build!</p> + {% else %} + + <table border="1"> + + <tr> + <th>Order</th> + <th>Task</th> + <th>Recipe</th> + <th>Recipe Version</th> + <th>Task Type</th> + <th>Outcome</th> + <th>Errors</th> + <th>Warnings</th> + <th>Time</th> + <th>Log</th> + <th>Work directory</th> + <th>CPU usage</th> + <th>Disk I/O</th> + <th>Sstate Checksum</th> + <th>Path to sstate object</th> + <th>Source URL</th> + <th>Script type</th> + <th>File path</th> + <th>Line number</th> + <th>Python Stack Trace</th> + <th>Sstate Result</th> + </tr> + + {% for task in tasks %} + + <tr> + <td>{{task.order}}</td> + <td>{{task.task_name}}</td> + <td>{{task.recipe.name}}</td> + <td>{{task.recipe.version}}</td> + + {% if task.task_executed %} + <td>Executed</td> + {% else %} + <td>Prebuilt</td> + {% endif %} + + <td>{{task.get_outcome_display}}</td> + <td>{{task.errors_no}}</td> + <td>{{task.warnings_no}}</td> + <td>{{task.elapsed_time}}</td> + <td>{{task.log_file}}</td> + <td>{{task.work_directory}}</td> + <td>{{task.cpu_usage}}</td> + <td>{{task.disk_io}}</td> + <td>{{task.sstate_checksum}}</td> + <td>{{task.path_to_sstate_obj}}</td> + <td>{{task.source_url}}</td> + <td>{{task.get_script_type_display}}</td> + <td>{{task.file_path}}</td> + <td>{{task.line_number}}</td> + <td>{{task.py_stack_trace}}</td> + <td>{{task.get_sstate_result_display}}</td> + </tr> + + {% endfor %} + + </table> + + {% endif %} + +</body> + +</html>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
@@ -3,4 +3,5 @@ from django.conf.urls import patterns, include, url urlpatterns = patterns('bldviewer.views', url(r'^$', 'build', name='build'), + url(r'^(?P<build_id>\d+)/$', 'task', name='task'), )
bitbake/lib/webhob/bldviewer/views.py +10 -1
@@ -1,5 +1,5 @@ from django.shortcuts import render -from orm.models import Build +from orm.models import Build, Task def build(request):
@@ -9,3 +9,12 @@ def build(request): context = {'builds': build_info} return render(request, template, context) + +def task(request, build_id): + template = 'task.html' + + tasks = Task.objects.filter(build=build_id) + + context = {'tasks': tasks} + + return render(request, template, context)
bitbake/lib/webhob/whbmain/urls.py +1 -1
@@ -5,7 +5,7 @@ from django.conf.urls import patterns, include, url # admin.autodiscover() urlpatterns = patterns('', - url(r'^build/$', include('bldviewer.urls')), + url(r'^build/', include('bldviewer.urls')), # Examples: # url(r'^webhob/', include('webhob.foo.urls')),
Signature · 1 line
1.8.1.2
PATCH 10/94 bitbake: webhob: use buildcompleted event as the end of a build operation
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
The BuildCompleted event marks the end of a build, so the code will use this timestamp for computing the duration of a build operation.
--- bitbake/lib/bb/ui/dsi.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/bb/ui/dsi.py +1 -1
@@ -448,7 +448,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): event.taskid, event.taskstring, event.exitcode) continue - if isinstance(event, (bb.command.CommandFailed, bb.command.CommandExit, bb.command.CommandCompleted, bb.cooker.CookerExit)): + if isinstance(event, bb.event.BuildCompleted): buildinfohelper.update_build_information(event, errors, warnings, taskfailures) main.shutdown = 2
Signature · 1 line
1.8.1.2
PATCH 11/94 bitbake: webhob: changes to build information table
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
It was decided that disk_io and cpu usage information are not very relevant for a full build, so these fields have been droped from our models and replaced with output information (image_fstypes field)
--- bitbake/lib/bb/ui/buildinfohelper.py | 6 ++---- bitbake/lib/webhob/bldviewer/templates/build.html | 6 ++---- bitbake/lib/webhob/orm/models.py | 3 +-- 3 files changed, 5 insertions(+), 10 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +2 -4
@@ -47,8 +47,7 @@ class ORMWrapper(object): build_info['outcome'] = 0 build_info['number_of_errors'] = 0 build_info['number_of_warnings'] = 0 - build_info['cpu_usage'] = 0 - build_info['disk_io'] = 0 + build_info['image_fstypes'] = server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0] build_info['cooker_log_path'] = server.runCommand(["getVariable", "BB_CONSOLELOG"])[0] build_info['build_name'] = server.runCommand(["getVariable", "BUILDNAME"])[0] build_info['bitbake_version'] = server.runCommand(["getVariable", "BB_VERSION"])[0]
@@ -69,8 +68,7 @@ class ORMWrapper(object): outcome=build_info['outcome'], errors_no=build_info['number_of_errors'], warnings_no=build_info['number_of_warnings'], - cpu_usage=build_info['cpu_usage'], - disk_io=build_info['disk_io'], + image_fstypes=build_info['image_fstypes'], cooker_log_path=build_info['cooker_log_path'], build_name=build_info['build_name'], bitbake_version=build_info['bitbake_version'])
bitbake/lib/webhob/bldviewer/templates/build.html +2 -4
@@ -20,8 +20,7 @@ <th>Time</th> <th>Errors</th> <th>Warnings</th> - <th>CPU usage</th> - <th>Disk I/O</th> + <th>Output</th> <th>Log</th> <th>Bitbake Version</th> <th>Build Name</th>
@@ -38,8 +37,7 @@ <td>{% time_difference build.started_on build.completed_on %}</td> <td>{{build.errors_no}}</td> <td>{{build.warnings_no}}</td> - <td>{{build.cpu_usage}}</td> - <td>{{build.disk_io}}</td> + <td>{{build.image_fstypes}}</td> <td>{{build.cooker_log_path}}</td> <td>{{build.bitbake_version}}</td> <td>{{build.build_name}}</td>
bitbake/lib/webhob/orm/models.py +1 -2
@@ -18,8 +18,7 @@ class Build(models.Model): outcome = models.IntegerField(choices=BUILD_OUTCOME) errors_no = models.IntegerField() warnings_no = models.IntegerField() - cpu_usage = models.IntegerField() - disk_io = models.DecimalField(max_digits=20, decimal_places=10) + image_fstypes = models.CharField(max_length=100) cooker_log_path = models.CharField(max_length=500) build_name = models.CharField(max_length=100) bitbake_version = models.CharField(max_length=50)
Signature · 1 line
1.8.1.2
PATCH 12/94 bitbake: webhob: gather buildstats for each executed task
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch reads the buildstats file created for each executed task and retrieves cpu usage and disk IO information which it will store in the database.
--- bitbake/lib/bb/ui/buildinfohelper.py | 89 ++++++++++++++++++++++++++++++------ bitbake/lib/bb/ui/dsi.py | 4 +- bitbake/lib/webhob/orm/models.py | 4 +- 3 files changed, 78 insertions(+), 19 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +74 -15
@@ -9,17 +9,20 @@ class ORMWrapper(object): information in the database. """ - def __init__(self): + def __init__(self, server): self.uuid = None self.task_order = 0 self.transport_utils = {} + self.server = server - def get_machine_information(self, server): + def get_machine_information(self): machine_info = {} - machine_info['name'] = server.runCommand(["getVariable", "MACHINE"])[0] + machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0] machine_info['description'] = 'Not Available' + self.transport_utils['machine'] = machine_info['name'] + return machine_info def create_machine_object(self, machine_information):
@@ -31,7 +34,7 @@ class ORMWrapper(object): return machine[0] - def get_build_information(self, server, machine_obj): + def get_build_information(self, machine_obj): build_info = {} # Generate an identifier for each new build
@@ -40,17 +43,17 @@ class ORMWrapper(object): build_info['uuid'] = self.uuid build_info['target'] = 'Not Available' build_info['machine'] = machine_obj - build_info['distro'] = server.runCommand(["getVariable", "DISTRO"])[0] - build_info['distro_version'] = server.runCommand(["getVariable", "DISTRO_VERSION"])[0] + build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0] + build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0] build_info['started_on'] = datetime.datetime.now() build_info['completed_on'] = datetime.datetime.now() build_info['outcome'] = 0 build_info['number_of_errors'] = 0 build_info['number_of_warnings'] = 0 - build_info['image_fstypes'] = server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0] - build_info['cooker_log_path'] = server.runCommand(["getVariable", "BB_CONSOLELOG"])[0] - build_info['build_name'] = server.runCommand(["getVariable", "BUILDNAME"])[0] - build_info['bitbake_version'] = server.runCommand(["getVariable", "BB_VERSION"])[0] + build_info['image_fstypes'] = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0] + build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0] + build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0] + build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0] return build_info
@@ -175,6 +178,10 @@ class ORMWrapper(object): outcome_info = self.get_outcome_of_task(event) task.outcome = outcome_info['task_outcome'] + task_build_stats = self.get_task_build_stats(task_object) + task.cpu_usage = task_build_stats['cpu_usage'] + task.disk_io = task_build_stats['disk_io'] + if outcome_info.get('error', ''): task.error = outcome_info['error']
@@ -202,6 +209,57 @@ class ORMWrapper(object): return outcome_info + def get_task_build_stats(self, task_object): + bs_path = self.get_path_information(task_object) + task_build_stats = self.get_build_stats_from_file(bs_path, task_object.task_name) + + return task_build_stats + + def get_path_information(self, task_object): + build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/" + + tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0] + target = self.transport_utils['target'] + machine = self.transport_utils['build'].machine.name + buildname = self.transport_utils['build'].build_name + package = task_object.recipe.name + + build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target, + machine=machine, buildname=buildname, + package=package) + + return build_stats_path + + def get_build_stats_from_file(self, bs_path, task_name): + + task_bs_filename = str(bs_path) + str(task_name) + task_bs = open(task_bs_filename, 'r') + + cpu_usage = 0 + disk_io = 0 + startio = '' + endio = '' + + for line in task_bs.readlines(): + if line.startswith('CPU usage: '): + cpu_usage = line[11:] + elif line.startswith('EndTimeIO: '): + endio = line[11:] + elif line.startswith('StartTimeIO: '): + startio = line[13:] + + task_bs.close() + + if startio and endio: + disk_io = int(endio.strip('\n ')) - int(startio.strip('\n ')) + + if cpu_usage: + cpu_usage = float(cpu_usage.strip('% \n')) + + task_build_stats = {'cpu_usage': cpu_usage, 'disk_io': disk_io} + + return task_build_stats + def get_recipe_object(self, event): # This needs to be imported after we have configured the Django settings file from webhob.orm.models import Recipe
@@ -303,9 +361,9 @@ class BuildInfoHelper(object): towards the ORM wrapper for storing in the database """ - def __init__(self): + def __init__(self, server): self.configure_django() - self.orm_wrapper = ORMWrapper() + self.orm_wrapper = ORMWrapper(server) def configure_django(self): import webhob.whbmain.settings as whb_django_settings
@@ -314,14 +372,15 @@ class BuildInfoHelper(object): # Add webhob to sys path for importing modules sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob')) - def store_started_build(self, event, server): + def store_started_build(self, event): - machine_information = self.orm_wrapper.get_machine_information(server) + machine_information = self.orm_wrapper.get_machine_information() machine_obj = self.orm_wrapper.create_machine_object(machine_information) - build_information = self.orm_wrapper.get_build_information(server, machine_obj) + build_information = self.orm_wrapper.get_build_information(machine_obj) build_obj = self.orm_wrapper.create_build_object(build_information) self.orm_wrapper.transport_utils['build'] = build_obj + self.orm_wrapper.transport_utils['target'] = event.getPkgs()[0] def update_build_information(self, event, errors, warnings, taskfailures): self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures)
bitbake/lib/bb/ui/dsi.py +2 -2
@@ -285,7 +285,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): errors = 0 warnings = 0 taskfailures = [] - buildinfohelper = BuildInfoHelper() + buildinfohelper = BuildInfoHelper(server) termfilter = tf(main, helper, console, format)
@@ -306,7 +306,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): main.shutdown = 1 if isinstance(event, bb.event.BuildStarted): - buildinfohelper.store_started_build(event, server) + buildinfohelper.store_started_build(event) if isinstance(event, bb.build.TaskStarted): buildinfohelper.store_started_task(event)
bitbake/lib/webhob/orm/models.py +2 -2
@@ -61,8 +61,8 @@ class Task(models.Model): file_path = models.FilePathField(max_length=255) line_number = models.IntegerField() py_stack_trace = models.TextField() - disk_io = models.DecimalField(max_digits=20, decimal_places=10) - cpu_usage = models.IntegerField() + disk_io = models.IntegerField() + cpu_usage = models.DecimalField(max_digits=6, decimal_places=2) elapsed_time = models.CharField(max_length=50) errors_no = models.IntegerField() warnings_no = models.IntegerField()
Signature · 1 line
1.8.1.2
PATCH 13/94 bitbake: webhob: set sane default settings
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Changes the default Django settings as to point
to sane default of using sqlite by default. This
is the preffered running mode for webhob.
Changes the list of installed applications.
--- bitbake/lib/webhob/whbmain/settings.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
bitbake/lib/webhob/whbmain/settings.py +4 -2
@@ -11,8 +11,8 @@ MANAGERS = ADMINS DATABASES = { 'default': { - 'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'. - 'NAME': '', # Or path to database file if using sqlite3. + 'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'. + 'NAME': 'webhob.sqlite', # Or path to database file if using sqlite3. 'USER': '', 'PASSWORD': '', 'HOST': '127.0.0.1', # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
@@ -124,6 +124,8 @@ INSTALLED_APPS = ( # Uncomment the next line to enable admin documentation: # 'django.contrib.admindocs', 'orm', + 'whbmain', + 'bldviewer', ) # A sample logging configuration. The only tangible logging
Signature · 1 line
1.8.1.2
PATCH 14/94 bitbake: webhob: adds dsi support for observer mode
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding support in the DSI user interface to remain resident and run in observer-only mode.
Modifies the webhob launch script to automatically lunch the DSI as to log all builds run under webhob setup.
--- bitbake/bin/webhob | 5 ++++- bitbake/lib/bb/ui/dsi.py | 47 ++++++++++++++--------------------------------- 2 files changed, 18 insertions(+), 34 deletions(-)
bitbake/bin/webhob +4 -1
@@ -28,7 +28,7 @@ function webserverKillAllComponents() for pidfile in ${BUILDDIR}/whbmain.pid; do if [ -f ${pidfile} ]; then while kill -0 $(< ${pidfile}) 2>/dev/null; do - kill -SIGTERM -`ps -p $(< ${pidfile}) -o "%r" --no-headers` + kill -SIGTERM -$(< ${pidfile}) sleep 1; done; rm ${pidfile}
@@ -84,9 +84,12 @@ case $CMD in python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0) bitbake --server-only -t xmlrpc -B localhost:8200 export BBSERVER=localhost:8200 + bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid ;; stop ) + kill $(< ${BUILDDIR}/dsi.pid ) + rm ${BUILDDIR}/dsi.pid bitbake -m unset BBSERVER webserverKillAllComponents
bitbake/lib/bb/ui/dsi.py +14 -33
91 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH 15/94 bitbake: dsi: Translate runQueue events into Task data
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We're moving the capture of task running information from bb.build.Task* events to runQueue* events.
We handle runQueueTaskSkipped events to record all skipped tasks.
--- bitbake/lib/bb/ui/dsi.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
bitbake/lib/bb/ui/dsi.py +7 -4
@@ -287,11 +287,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): if isinstance(event, bb.event.BuildStarted): buildinfohelper.store_started_build(event) - if isinstance(event, bb.build.TaskStarted): - buildinfohelper.store_started_task(event) - if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)): - buildinfohelper.update_and_store_task(event) + continue if isinstance(event, bb.event.LogExecTTY): if log_exec_tty:
@@ -416,13 +413,19 @@ def main(server, eventHandler, params, tf = TerminalFilter): event.stats.total, event.taskid, event.taskstring) continue + if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.runQueueTaskSkipped)): + buildinfohelper.store_started_task(event) + continue + if isinstance(event, bb.runqueue.runQueueTaskFailed): + buildinfohelper.update_and_store_task(event) taskfailures.append(event.taskstring) logger.error("Task %s (%s) failed with exit code '%s'", event.taskid, event.taskstring, event.exitcode) continue if isinstance(event, bb.runqueue.sceneQueueTaskFailed): + buildinfohelper.update_and_store_task(event) logger.warn("Setscene task %s (%s) failed with exit code '%s' - real task will be run instead", event.taskid, event.taskstring, event.exitcode) continue
Signature · 1 line
1.8.1.2
PATCH 16/94 bitbake: webhob: disable synchronous sqlite connection
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
SQLite has very bad INSERT performance in synchronous mode. We disable the synchronous mode to improve performance only on sqlite connections.
This requires also to change the way the build info wrapper loads the Django settings.
--- bitbake/lib/bb/ui/buildinfohelper.py | 3 +-- bitbake/lib/webhob/whbmain/settings.py | 10 ++++++++++ 2 files changed, 11 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +1 -2
@@ -366,9 +366,8 @@ class BuildInfoHelper(object): self.orm_wrapper = ORMWrapper(server) def configure_django(self): + os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings - from django.core.management import setup_environ - setup_environ(whb_django_settings) # Add webhob to sys path for importing modules sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
bitbake/lib/webhob/whbmain/settings.py +10 -0
@@ -20,6 +20,16 @@ DATABASES = { } } +# If we're using sqlite, we need to tweak the performance a bit +from django.db.backends.signals import connection_created +def activate_synchronous_off(sender, connection, **kwargs): + if connection.vendor == 'sqlite': + cursor = connection.cursor() + cursor.execute('PRAGMA synchronous = 0;') +connection_created.connect(activate_synchronous_off) +# + + # Hosts/domain names that are valid for this site; required if DEBUG is False # See https://docs.djangoproject.com/en/1.5/ref/settings/#allowed-hosts ALLOWED_HOSTS = []
Signature · 1 line
1.8.1.2
PATCH 17/94 bitbake: buildinfohelper: clean-up the imports
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
As per convention, we do all imports at the beginning of file.
Also, globally set Django settings in the Django interface file.
--- bitbake/lib/bb/ui/buildinfohelper.py | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +5 -10
@@ -3,6 +3,11 @@ import sys import uuid +os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") + +import webhob.whbmain.settings as whb_django_settings +from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer + class ORMWrapper(object): """ This class creates the dictionaries needed to store information in the database following the format defined by the Django models. It is also used to save this
@@ -27,7 +32,6 @@ class ORMWrapper(object): def create_machine_object(self, machine_information): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Machine machine = Machine.objects.get_or_create(name=machine_information['name'], description=machine_information['description'])
@@ -59,7 +63,6 @@ class ORMWrapper(object): def create_build_object(self, build_info): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Build build = Build.objects.create(uuid=build_info['uuid'], target=build_info['target'],
@@ -80,7 +83,6 @@ class ORMWrapper(object): def update_build_object(self, build_obj, errors, warnings, taskfailures): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Build outcome = 0 if errors or taskfailures:
@@ -128,8 +130,6 @@ class ORMWrapper(object): def create_task_object(self, task_information): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Task - identifier = task_information['recipe'].name + task_information['task_name'] task_object = Task.objects.get_or_create(
@@ -262,7 +262,6 @@ class ORMWrapper(object): def get_recipe_object(self, event): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Recipe recipe_information = self.get_recipe_information(event)
@@ -305,7 +304,6 @@ class ORMWrapper(object): def get_build_layer_object(self, event): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Build_Layer build_layer_information = self.get_build_layer_information(event)
@@ -334,7 +332,6 @@ class ORMWrapper(object): def get_layer_object(self, event): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Layer layer_information = self.get_layer_information(event)
@@ -366,8 +363,6 @@ class BuildInfoHelper(object): self.orm_wrapper = ORMWrapper(server) def configure_django(self): - os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") - import webhob.whbmain.settings as whb_django_settings # Add webhob to sys path for importing modules sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
Signature · 1 line
1.8.1.2
PATCH 18/94 bitbake: buildinfohelper: clearing up task order
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Moved code around so that the task order is issued only on creating new task method, and not on get information.
--- bitbake/lib/bb/ui/buildinfohelper.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/buildinfohelper.py +2 -1
@@ -130,11 +130,12 @@ class ORMWrapper(object): def create_task_object(self, task_information): # This needs to be imported after we have configured the Django settings file + self.task_order += 1 identifier = task_information['recipe'].name + task_information['task_name'] task_object = Task.objects.get_or_create( build=task_information['build'], - order=task_information['order'], + order=self.task_order, task_executed=task_information['task_executed'], outcome=task_information['outcome'], sstate_checksum=task_information['sstate_checksum'],
Signature · 1 line
1.8.1.2
PATCH 19/94 bitbake: dsi: saving prebuild task detailed information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Changes to the internal structure of the database interface layer.
Clearing up the code to save information about prebuild tasks.
Clear up the event-processing fall-through code in DSI.
--- bitbake/lib/bb/ui/buildinfohelper.py | 54 +++++++++++++----------------------- bitbake/lib/bb/ui/dsi.py | 9 ++++-- 2 files changed, 25 insertions(+), 38 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +19 -35
@@ -1,6 +1,7 @@ import datetime import sys import uuid +import bb os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings")
@@ -97,19 +98,24 @@ class ORMWrapper(object): def get_task_information(self, event): - self.task_order += 1 recipe = self.get_recipe_object(event) task_information = {} task_information['build'] = self.transport_utils['build'] - task_information['order'] = self.task_order - task_information['task_executed'] = True - task_information['outcome'] = 0 + task_information['outcome'] = 4 + if isinstance(event, bb.runqueue.runQueueTaskSkipped): + task_information['task_executed'] = False + if event._skip == "covered": + task_information['outcome'] = 2 + if event._skip == "existing": + task_information['outcome'] = 0 + else: + task_information['task_executed'] = True task_information['sstate_checksum'] = 'Not Available' task_information['path_to_sstate_obj'] = 'Not Available' task_information['recipe'] = recipe - task_information['task_name'] = event._task + task_information['task_name'] = event.taskname task_information['source_url'] = 'Not Available' task_information['log_file'] = 'Not Available' task_information['work_directory'] = 'Not Available'
@@ -165,21 +171,16 @@ class ORMWrapper(object): def update_task_object(self, task_dictionary, event): # This needs to be imported after we have configured the Django settings file - from webhob.orm.models import Task - - task_object = task_dictionary['object'] - task = Task.objects.get(build=task_object.build, - task_name=task_object.task_name, - recipe=task_object.recipe) + task = task_dictionary['object'] duration = datetime.datetime.now() - task_dictionary['start_time'] task.elapsed_time = duration.total_seconds() - outcome_info = self.get_outcome_of_task(event) - task.outcome = outcome_info['task_outcome'] + if isinstance(event, bb.runqueue.runQueueTaskCompleted): + task.outcome = 3 - task_build_stats = self.get_task_build_stats(task_object) + task_build_stats = self.get_task_build_stats(task) task.cpu_usage = task_build_stats['cpu_usage'] task.disk_io = task_build_stats['disk_io']
@@ -192,24 +193,6 @@ class ORMWrapper(object): task.save() - def get_outcome_of_task(self, event): - from webhob.orm.models import Task - - outcome_info = {} - - task_result = event.getDisplayName().lower() - for outcome in Task.TASK_OUTCOME: - if task_result == outcome[1].lower(): - task_outcome = outcome[0] - - outcome_info['task_outcome'] = task_outcome - - if isinstance(event, bb.build.TaskFailed): - if event.logfile and os.path.exists(event.logfile): - outcome_info['error'] = open(event.logfile, "r").read() - - return outcome_info - def get_task_build_stats(self, task_object): bs_path = self.get_path_information(task_object) task_build_stats = self.get_build_stats_from_file(bs_path, task_object.task_name)
@@ -288,7 +271,7 @@ class ORMWrapper(object): build_layer_obj = self.get_build_layer_object(event) recipe_info = {} - recipe_info['name'] = event._package + recipe_info['name'] = event.taskpackage recipe_info['version'] = 'Not Available' recipe_info['layer'] = build_layer_obj recipe_info['summary'] = 'Not Available'
@@ -373,9 +356,10 @@ class BuildInfoHelper(object): machine_obj = self.orm_wrapper.create_machine_object(machine_information) build_information = self.orm_wrapper.get_build_information(machine_obj) + build_information['target'] = ' '.join(event.getPkgs()) build_obj = self.orm_wrapper.create_build_object(build_information) self.orm_wrapper.transport_utils['build'] = build_obj - self.orm_wrapper.transport_utils['target'] = event.getPkgs()[0] + self.orm_wrapper.transport_utils['target'] = build_information['target'] def update_build_information(self, event, errors, warnings, taskfailures): self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures)
@@ -386,7 +370,7 @@ class BuildInfoHelper(object): task_obj = self.orm_wrapper.create_task_object(task_information) def update_and_store_task(self, event): - identifier = event._package + event._task + identifier = event.taskpackage + event.taskname task_dictionary = self.orm_wrapper.transport_utils[identifier] self.orm_wrapper.update_task_object(task_dictionary, event) del self.orm_wrapper.transport_utils[identifier]
bitbake/lib/bb/ui/dsi.py +6 -3
@@ -411,12 +411,17 @@ def main(server, eventHandler, params, tf = TerminalFilter): event.stats.completed + event.stats.active + event.stats.failed + 1, event.stats.total, event.taskid, event.taskstring) + buildinfohelper.store_started_task(event) continue - if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.runQueueTaskSkipped)): + if isinstance(event, (bb.runqueue.runQueueTaskStarted, bb.runqueue.runQueueTaskSkipped)): buildinfohelper.store_started_task(event) continue + if isinstance(event, bb.runqueue.runQueueTaskCompleted): + buildinfohelper.update_and_store_task(event) + continue + if isinstance(event, bb.runqueue.runQueueTaskFailed): buildinfohelper.update_and_store_task(event) taskfailures.append(event.taskstring)
@@ -430,8 +435,6 @@ def main(server, eventHandler, params, tf = TerminalFilter): event.taskid, event.taskstring, event.exitcode) continue - if isinstance(event, bb.event.BuildCompleted): - buildinfohelper.update_build_information(event, errors, warnings, taskfailures) if isinstance(event, bb.event.ConfigParsed): # timestamp should be added for this
Signature · 1 line
1.8.1.2
PATCH 20/94 bitbake: dsi: clear up the state on build completion
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
In order to prevent cross-data between different builds, we drop the build helper object which contains the state of the current build when the build is completed.
--- bitbake/lib/bb/ui/dsi.py | 9 +++++++++ 1 file changed, 9 insertions(+)
bitbake/lib/bb/ui/dsi.py +9 -0
@@ -458,6 +458,15 @@ def main(server, eventHandler, params, tf = TerminalFilter): # trigger an error continue + if isinstance(event, (bb.event.BuildCompleted, + bb.command.CommandCompleted, + bb.command.CommandFailed, + bb.command.CommandExit)): + buildinfohelper.update_build_information(event, errors, warnings, taskfailures) + # we start a new build info + buildinfohelper = BuildInfoHelper(server) + continue + # ignore if isinstance(event, (bb.event.BuildBase, bb.event.StampUpdate,
Signature · 1 line
1.8.1.2
PATCH 21/94 bitbake: webhob: add layer information to the database
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch adds the layer information to the database. Using the values stored in BBLAYERS it retrieves all the information needed for the database fields.
It also includes a hardcoded mapping to deal with the URLs that are different from the standard naming convention.
--- bitbake/lib/bb/ui/buildinfohelper.py | 45 +++++++++++++++++++++++ bitbake/lib/webhob/bldviewer/templates/layer.html | 34 +++++++++++++++++ bitbake/lib/webhob/bldviewer/urls.py | 1 + bitbake/lib/webhob/bldviewer/views.py | 10 ++++- 4 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/layer.html
bitbake/lib/bb/ui/buildinfohelper.py +45 -0
@@ -336,6 +336,49 @@ class ORMWrapper(object): return layer_info + def _get_layer_dict(self, layer_path): + + layer_info = {} + layer_name = layer_path.split('/')[-1] + layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/' + layer_url_name = self._get_url_map_name(layer_name) + + layer_info['name'] = layer_name + layer_info['local_path'] = layer_path + layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name) + + return layer_info + + def _get_url_map_name(self, layer_name): + """ Some layers have a different name on openembedded.org site, + this method returns the correct name to use in the URL + """ + + url_name = layer_name + url_mapping = {'meta': 'openembedded-core'} + + for key in url_mapping.keys(): + if key == layer_name: + url_name = url_mapping[key] + + return url_name + + def _save_layer_object(self, layer_path): + + layer_information = self._get_layer_dict(layer_path) + + layer_object = Layer.objects.get_or_create( + name=layer_information['name'], + local_path=layer_information['local_path'], + layer_index_url=layer_information['layer_index_url']) + + layer_object[0].save() + + def store_layer_info(self): + layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ") + for layer_path in layers: + self._save_layer_object(layer_path) + class BuildInfoHelper(object): """ This class gathers the build information from the server and sends it
@@ -352,6 +395,8 @@ class BuildInfoHelper(object): def store_started_build(self, event): + self.orm_wrapper.store_layer_info() + machine_information = self.orm_wrapper.get_machine_information() machine_obj = self.orm_wrapper.create_machine_object(machine_information)
bitbake/lib/webhob/bldviewer/templates/layer.html new file +34 -0
@@ -0,0 +1,34 @@ +<html> + + <head> + <title>WebHob Layer page</title> + </head> + +<body> + <h1>WebHob Layers</h1> + + <table border="1"> + + {% load projecttags %} + + <tr> + <th>Name</th> + <th>Local Path</th> + <th>Layer Index URL</th> + </tr> + + {% for layer in layers %} + + <tr> + <td>{{layer.name}}</td> + <td>{{layer.local_path}}</td> + <td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td> + </tr> + + {% endfor %} + + </table> + +</body> + +</html>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
@@ -3,5 +3,6 @@ from django.conf.urls import patterns, include, url urlpatterns = patterns('bldviewer.views', url(r'^$', 'build', name='build'), + url(r'^layers/$', 'layer', name='layer'), url(r'^(?P<build_id>\d+)/$', 'task', name='task'), )
bitbake/lib/webhob/bldviewer/views.py +9 -1
@@ -1,5 +1,5 @@ from django.shortcuts import render -from orm.models import Build, Task +from orm.models import Build, Task, Layer def build(request):
@@ -18,3 +18,11 @@ def task(request, build_id): context = {'tasks': tasks} return render(request, template, context) + +def layer(request): + template = 'layer.html' + layer_info = Layer.objects.all() + + context = {'layers': layer_info} + + return render(request, template, context)
Signature · 1 line
1.8.1.2
PATCH 22/94 bitbake: webhob: mark private methods inside DSI
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch marks private methods used in the DSI.
It also cleans some old meaningless comments.
--- bitbake/lib/bb/ui/buildinfohelper.py | 49 +++++++++++++++--------------------- 1 file changed, 20 insertions(+), 29 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +20 -29
@@ -32,7 +32,6 @@ class ORMWrapper(object): return machine_info def create_machine_object(self, machine_information): - # This needs to be imported after we have configured the Django settings file machine = Machine.objects.get_or_create(name=machine_information['name'], description=machine_information['description'])
@@ -63,7 +62,6 @@ class ORMWrapper(object): return build_info def create_build_object(self, build_info): - # This needs to be imported after we have configured the Django settings file build = Build.objects.create(uuid=build_info['uuid'], target=build_info['target'],
@@ -83,7 +81,6 @@ class ORMWrapper(object): return build def update_build_object(self, build_obj, errors, warnings, taskfailures): - # This needs to be imported after we have configured the Django settings file outcome = 0 if errors or taskfailures:
@@ -98,8 +95,7 @@ class ORMWrapper(object): def get_task_information(self, event): - - recipe = self.get_recipe_object(event) + recipe = self._get_recipe_object(event) task_information = {} task_information['build'] = self.transport_utils['build']
@@ -135,7 +131,6 @@ class ORMWrapper(object): return task_information def create_task_object(self, task_information): - # This needs to be imported after we have configured the Django settings file self.task_order += 1 identifier = task_information['recipe'].name + task_information['task_name']
@@ -170,7 +165,6 @@ class ORMWrapper(object): return task_object[0] def update_task_object(self, task_dictionary, event): - # This needs to be imported after we have configured the Django settings file task = task_dictionary['object']
@@ -180,7 +174,7 @@ class ORMWrapper(object): if isinstance(event, bb.runqueue.runQueueTaskCompleted): task.outcome = 3 - task_build_stats = self.get_task_build_stats(task) + task_build_stats = self._get_task_build_stats(task) task.cpu_usage = task_build_stats['cpu_usage'] task.disk_io = task_build_stats['disk_io']
@@ -193,13 +187,13 @@ class ORMWrapper(object): task.save() - def get_task_build_stats(self, task_object): - bs_path = self.get_path_information(task_object) - task_build_stats = self.get_build_stats_from_file(bs_path, task_object.task_name) + def _get_task_build_stats(self, task_object): + bs_path = self._get_path_information(task_object) + task_build_stats = self._get_build_stats_from_file(bs_path, task_object.task_name) return task_build_stats - def get_path_information(self, task_object): + def _get_path_information(self, task_object): build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/" tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
@@ -214,7 +208,7 @@ class ORMWrapper(object): return build_stats_path - def get_build_stats_from_file(self, bs_path, task_name): + def _get_build_stats_from_file(self, bs_path, task_name): task_bs_filename = str(bs_path) + str(task_name) task_bs = open(task_bs_filename, 'r')
@@ -244,10 +238,9 @@ class ORMWrapper(object): return task_build_stats - def get_recipe_object(self, event): - # This needs to be imported after we have configured the Django settings file + def _get_recipe_object(self, event): - recipe_information = self.get_recipe_information(event) + recipe_information = self._get_recipe_information(event) recipe_object = Recipe.objects.get_or_create( name=recipe_information['name'],
@@ -266,9 +259,9 @@ class ORMWrapper(object): return recipe_object[0] - def get_recipe_information(self, event): + def _get_recipe_information(self, event): - build_layer_obj = self.get_build_layer_object(event) + build_layer_obj = self._get_build_layer_object(event) recipe_info = {} recipe_info['name'] = event.taskpackage
@@ -286,10 +279,9 @@ class ORMWrapper(object): return recipe_info - def get_build_layer_object(self, event): - # This needs to be imported after we have configured the Django settings file + def _get_build_layer_object(self, event): - build_layer_information = self.get_build_layer_information(event) + build_layer_information = self._get_build_layer_information(event) build_layer_object = Build_Layer.objects.get_or_create( build = build_layer_information['build'],
@@ -301,9 +293,9 @@ class ORMWrapper(object): return build_layer_object[0] - def get_build_layer_information(self, event): + def _get_build_layer_information(self, event): - layer_object = self.get_layer_object(event) + layer_object = self._get_layer_object(event) build_layer_info = {} build_layer_info['build'] = self.transport_utils['build']
@@ -314,10 +306,9 @@ class ORMWrapper(object): return build_layer_info - def get_layer_object(self, event): - # This needs to be imported after we have configured the Django settings file + def _get_layer_object(self, event): - layer_information = self.get_layer_information(event) + layer_information = self._get_layer_information(event) layer_object = Layer.objects.get_or_create( name=layer_information['name'],
@@ -327,7 +318,7 @@ class ORMWrapper(object): return layer_object[0] - def get_layer_information(self, event): + def _get_layer_information(self, event): layer_info = {} layer_info['name'] = 'Layer Test Name'
@@ -386,10 +377,10 @@ class BuildInfoHelper(object): """ def __init__(self, server): - self.configure_django() + self._configure_django() self.orm_wrapper = ORMWrapper(server) - def configure_django(self): + def _configure_django(self): # Add webhob to sys path for importing modules sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob'))
Signature · 1 line
1.8.1.2
PATCH 23/94 bitbake: webhob: create basic structure for static files
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch creates the basic structure for the front end files needed by Webhob. It sets up a new application called whbgui which will host all the static files we need.
--- bitbake/lib/webhob/whbgui/__init__.py | 0 bitbake/lib/webhob/whbgui/static/images/yocto.jpg | Bin 0 -> 6582 bytes bitbake/lib/webhob/whbgui/templates/index.html | 13 +++++++++++++ bitbake/lib/webhob/whbgui/tests.py | 16 ++++++++++++++++ bitbake/lib/webhob/whbgui/urls.py | 9 +++++++++ bitbake/lib/webhob/whbgui/views.py | 8 ++++++++ bitbake/lib/webhob/whbmain/settings.py | 3 ++- bitbake/lib/webhob/whbmain/urls.py | 1 + 8 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/webhob/whbgui/__init__.py create mode 100644 bitbake/lib/webhob/whbgui/static/images/yocto.jpg create mode 100644 bitbake/lib/webhob/whbgui/templates/index.html create mode 100644 bitbake/lib/webhob/whbgui/tests.py create mode 100644 bitbake/lib/webhob/whbgui/urls.py create mode 100644 bitbake/lib/webhob/whbgui/views.py
bitbake/lib/webhob/whbgui/__init__.py new file +0 -0
bitbake/lib/webhob/whbgui/static/images/yocto.jpg new file binary +0 -0
bitbake/lib/webhob/whbgui/templates/index.html new file +13 -0
@@ -0,0 +1,13 @@ +<html> +<head> + <title>GUI Page</title> +</head> +<body> +{% load staticfiles %} + +<img src="{% static "/static/images/yocto.jpg" %}" alt="Yocto"/> + +This is your basic index page! +</body> + +</html>
bitbake/lib/webhob/whbgui/tests.py new file +16 -0
@@ -0,0 +1,16 @@ +""" +This file demonstrates writing tests using the unittest module. These will pass +when you run "manage.py test". + +Replace this with more appropriate tests for your application. +""" + +from django.test import TestCase + + +class SimpleTest(TestCase): + def test_basic_addition(self): + """ + Tests that 1 + 1 always equals 2. + """ + self.assertEqual(1 + 1, 2)
bitbake/lib/webhob/whbgui/urls.py new file +9 -0
@@ -0,0 +1,9 @@ +import os + +from django.conf import settings +from django.conf.urls import patterns, include, url + + +urlpatterns = patterns('whbgui.views', + url(r'^$', 'guihome', name='guihome'), +)
bitbake/lib/webhob/whbgui/views.py new file +8 -0
@@ -0,0 +1,8 @@ +from django.shortcuts import render +from orm.models import Build, Task + + +def guihome(request): + template = 'index.html' + + return render(request, template)
bitbake/lib/webhob/whbmain/settings.py +2 -1
@@ -128,7 +128,7 @@ INSTALLED_APPS = ( #'django.contrib.sessions', #'django.contrib.sites', #'django.contrib.messages', - #'django.contrib.staticfiles', + 'django.contrib.staticfiles', # Uncomment the next line to enable the admin: # 'django.contrib.admin', # Uncomment the next line to enable admin documentation:
@@ -136,6 +136,7 @@ INSTALLED_APPS = ( 'orm', 'whbmain', 'bldviewer', + 'whbgui', ) # A sample logging configuration. The only tangible logging
bitbake/lib/webhob/whbmain/urls.py +1 -0
@@ -6,6 +6,7 @@ from django.conf.urls import patterns, include, url urlpatterns = patterns('', url(r'^build/', include('bldviewer.urls')), + url(r'^gui/', include('whbgui.urls')), # Examples: # url(r'^webhob/', include('webhob.foo.urls')),
Signature · 1 line
1.8.1.2
PATCH 24/94 bitbake: dsi: event data change
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Changes in the BuildInfoHelper and DSI as to record task file names and task names coming through the bb.build.Task* events.
--- bitbake/lib/bb/ui/buildinfohelper.py | 22 +++++++++++++--------- bitbake/lib/bb/ui/dsi.py | 1 + 2 files changed, 14 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +13 -9
@@ -116,7 +116,7 @@ class ORMWrapper(object): task_information['log_file'] = 'Not Available' task_information['work_directory'] = 'Not Available' task_information['script_type'] = 0 - task_information['file_path'] = 'Not Available' + task_information['file_path'] = event.taskfile task_information['line_number'] = 0 task_information['py_stack_trace'] = 'Not Available' task_information['disk_io'] = 0
@@ -132,7 +132,7 @@ class ORMWrapper(object): def create_task_object(self, task_information): self.task_order += 1 - identifier = task_information['recipe'].name + task_information['task_name'] + identifier = task_information['recipe'].file_path + task_information['task_name'] task_object = Task.objects.get_or_create( build=task_information['build'],
@@ -173,14 +173,16 @@ class ORMWrapper(object): if isinstance(event, bb.runqueue.runQueueTaskCompleted): task.outcome = 3 + task.save() + + if isinstance(event, bb.build.TaskBase): + task.recipe.name = event._package + task.recipe.save() task_build_stats = self._get_task_build_stats(task) task.cpu_usage = task_build_stats['cpu_usage'] task.disk_io = task_build_stats['disk_io'] - if outcome_info.get('error', ''): - task.error = outcome_info['error'] - #TODO: get error number #TODO: get warnings number #TODO: get warning information
@@ -264,7 +266,7 @@ class ORMWrapper(object): build_layer_obj = self._get_build_layer_object(event) recipe_info = {} - recipe_info['name'] = event.taskpackage + recipe_info['name'] = 'N/A' recipe_info['version'] = 'Not Available' recipe_info['layer'] = build_layer_obj recipe_info['summary'] = 'Not Available'
@@ -275,7 +277,8 @@ class ORMWrapper(object): recipe_info['homepage'] = 'Not Available' recipe_info['bugtracker'] = 'Not Available' recipe_info['author'] = 'Not Available' - recipe_info['file_path'] = 'Not Available' + recipe_info['file_path'] = event.taskfile + return recipe_info
@@ -406,7 +409,8 @@ class BuildInfoHelper(object): task_obj = self.orm_wrapper.create_task_object(task_information) def update_and_store_task(self, event): - identifier = event.taskpackage + event.taskname + identifier = event.taskfile + event.taskname task_dictionary = self.orm_wrapper.transport_utils[identifier] self.orm_wrapper.update_task_object(task_dictionary, event) - del self.orm_wrapper.transport_utils[identifier] + if isinstance(event,bb.runqueue.runQueueTaskCompleted): + del self.orm_wrapper.transport_utils[identifier]
bitbake/lib/bb/ui/dsi.py +1 -0
@@ -288,6 +288,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): buildinfohelper.store_started_build(event) if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)): + buildinfohelper.update_and_store_task(event) continue if isinstance(event, bb.event.LogExecTTY):
Signature · 1 line
1.8.1.2
PATCH 25/94 bitbake: webhob: fix and cleanup start script
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Make sure that the script can start without a bitbake.lock file present.
Fixes [YOCTO #4873].
Also, clean up whitespace.
--- bitbake/bin/webhob | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
bitbake/bin/webhob +10 -8
@@ -21,8 +21,8 @@ # use as: source webhob [start|stop] # Helper function to kill a background webhob development server - -function webserverKillAllComponents() + +function webserverKillAllComponents() { local pidfile for pidfile in ${BUILDDIR}/whbmain.pid; do
@@ -42,7 +42,7 @@ if [ -z "$ZSH_NAME" ] && [ `basename "$0"` = `basename $BASH_SOURCE` ]; then echo "Error: This script needs to be sourced. Please run as 'source webhob [start|stop]'" 1>&2; exit 1 fi - + if [ -z "$BUILDDIR" ] || [ -z `which bitbake` ]; then echo "Error: Build environment is not setup or bitbake is not in path." 1>&2; return 2
@@ -64,15 +64,17 @@ fi # Make sure it's safe to run by checking bitbake lock lock=1 -(flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0 +if [ -e $BUILDDIR/bitbake.lock ]; then + (flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0 +fi if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); then echo "Error: bitbake lock state error. System may be already on." 2>&1 - return 3 + return 3 elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then echo "Error: bitbake lock state error. System may be already off. manually stop system with bitbake -m / webserverKillAllComponents" 2>&1 - return 3 + return 3 fi
@@ -82,7 +84,7 @@ case $CMD in start ) unset BBSERVER python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0) - bitbake --server-only -t xmlrpc -B localhost:8200 + bitbake --server-only -t xmlrpc -B localhost:8200 export BBSERVER=localhost:8200 bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid
@@ -95,4 +97,4 @@ case $CMD in webserverKillAllComponents esac - +
Signature · 1 line
1.8.1.2
PATCH 26/94 bitbake: dsi: refactor the BuildInfoHelper code
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
This is a in-depth refactoring of the BuildInfoHelper code designed to promote clean separation of code behinnd the data interfaces.
ORMWrapper now contains only database-related code. BuildInfoHelper keeps track of the internal state, and translated the Bitbake event-based data into dictionaries that the ORM layer can process.
DSI now sets featureSets to get a complete dependency dump.
The code uses the dependency information dump to create all the recipe and task entries before actual running the build, providing safe default.
The database models have been updated to allow for NULL values instead of creating generic placeholders.
--- bitbake/lib/bb/ui/buildinfohelper.py | 546 ++++++++++++----------- bitbake/lib/bb/ui/dsi.py | 14 + bitbake/lib/webhob/bldviewer/templates/task.html | 2 +- bitbake/lib/webhob/orm/models.py | 67 +-- 4 files changed, 333 insertions(+), 296 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +283 -263
@@ -2,12 +2,13 @@ import datetime import sys import uuid import bb +import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer +from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package, Task_Dependency, Package_Dependency class ORMWrapper(object): """ This class creates the dictionaries needed to store information in the database
@@ -15,21 +16,8 @@ class ORMWrapper(object): information in the database. """ - def __init__(self, server): - self.uuid = None - self.task_order = 0 - self.transport_utils = {} - self.server = server - - def get_machine_information(self): - machine_info = {} - - machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0] - machine_info['description'] = 'Not Available' - - self.transport_utils['machine'] = machine_info['name'] - - return machine_info + def __init__(self): + pass def create_machine_object(self, machine_information):
@@ -38,29 +26,6 @@ class ORMWrapper(object): return machine[0] - def get_build_information(self, machine_obj): - build_info = {} - - # Generate an identifier for each new build - self.uuid = str(uuid.uuid4()) - - build_info['uuid'] = self.uuid - build_info['target'] = 'Not Available' - build_info['machine'] = machine_obj - build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0] - build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0] - build_info['started_on'] = datetime.datetime.now() - build_info['completed_on'] = datetime.datetime.now() - build_info['outcome'] = 0 - build_info['number_of_errors'] = 0 - build_info['number_of_warnings'] = 0 - build_info['image_fstypes'] = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0] - build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0] - build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0] - build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0] - - return build_info - def create_build_object(self, build_info): build = Build.objects.create(uuid=build_info['uuid'],
@@ -70,14 +35,12 @@ class ORMWrapper(object): distro_version=build_info['distro_version'], started_on=build_info['started_on'], completed_on=build_info['completed_on'], - outcome=build_info['outcome'], - errors_no=build_info['number_of_errors'], - warnings_no=build_info['number_of_warnings'], image_fstypes=build_info['image_fstypes'], cooker_log_path=build_info['cooker_log_path'], build_name=build_info['build_name'], bitbake_version=build_info['bitbake_version']) + return build def update_build_object(self, build_obj, errors, warnings, taskfailures):
@@ -93,12 +56,178 @@ class ORMWrapper(object): build.outcome = outcome build.save() - def get_task_information(self, event): - recipe = self._get_recipe_object(event) + def get_update_task_object(self, task_information): + task_object, created = Task.objects.get_or_create( + build=task_information['build'], + recipe=task_information['recipe'], + task_name=task_information['task_name'], + ) + + for v in vars(task_object): + if v in task_information.keys(): + vars(task_object)[v] = task_information[v] + + if 'start_time' in task_information.keys(): + duration = datetime.datetime.now() - task_information['start_time'] + task_object.elapsed_time = duration.total_seconds() + + task_object.save() + return task_object + + + def get_update_recipe_object(self, recipe_information): + + recipe_object, created = Recipe.objects.get_or_create( + build_layer=recipe_information['build_layer'], + file_path=recipe_information['file_path']) + + for v in vars(recipe_object): + if v in recipe_information.keys(): + vars(recipe_object)[v] = recipe_information[v] + + recipe_object.save() + + return recipe_object + + def get_build_layer_object(self, build_layer_information): + + build_layer_object = Build_Layer.objects.get_or_create( + build = build_layer_information['build'], + layer = build_layer_information['layer'], + branch = build_layer_information['branch'], + commit = build_layer_information['commit'], + priority = build_layer_information['priority'] + ) + + build_layer_object[0].save() + + return build_layer_object[0] + + def get_update_layer_object(self, layer_information): + + layer_object = Layer.objects.get_or_create( + name=layer_information['name'], + local_path=layer_information['local_path'], + layer_index_url=layer_information['layer_index_url']) + layer_object[0].save() + + return layer_object[0] + + +class BuildInfoHelper(object): + """ This class gathers the build information from the server and sends it + towards the ORM wrapper for storing in the database + It is instantiated once per build + Keeps in memory all data that needs matching before writing it to the database + """ + + def __init__(self, server): + self._configure_django() + self.internal_state = {} + self.uuid = None + self.task_order = 0 + self.server = server + self.orm_wrapper = ORMWrapper() + + def _configure_django(self): + # Add webhob to sys path for importing modules + sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob')) + + ################### + ## methods to convert event/external info into objects that the ORM layer uses + + def _get_machine_information(self): + machine_info = {} + machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0] + machine_info['description'] = 'Not Available' + self.internal_state['machine'] = machine_info['name'] + return machine_info + + def _get_layer_dict(self, layer_path): + + layer_info = {} + layer_name = layer_path.split('/')[-1] + layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/' + layer_url_name = self._get_url_map_name(layer_name) + + layer_info['name'] = layer_name + layer_info['local_path'] = layer_path + layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name) + + return layer_info + + def _get_url_map_name(self, layer_name): + """ Some layers have a different name on openembedded.org site, + this method returns the correct name to use in the URL + """ + + url_name = layer_name + url_mapping = {'meta': 'openembedded-core'} + + for key in url_mapping.keys(): + if key == layer_name: + url_name = url_mapping[key] + + return url_name + + def _get_layer_information(self): + + layer_info = {} + + return layer_info + + def _get_build_layer_information(self, layer_object): + + build_layer_info = {} + build_layer_info['build'] = self.internal_state['build'] + build_layer_info['layer'] = layer_object + build_layer_info['branch'] = self._get_git_branch(layer_object.local_path) + build_layer_info['commit'] = self._get_git_revision(layer_object.local_path) + build_layer_info['priority'] = 0 + + return build_layer_info + + + def _get_git_branch(self, layer_path): + branch = os.popen('cd {path}; git branch 2>&1 | grep "^* " | tr -d "* "'.format(path=layer_path)).read() + + if len(branch) != 0: + return branch + return "<unknown>" + + def _get_git_revision(self, layer_path): + f = os.popen("cd {path}; git log -n 1 --pretty=oneline -- 2>&1".format(path=layer_path)) + data = f.read() + if f.close() is None: + rev = data.split(" ")[0] + if len(rev) != 0: + return rev + return "<unknown>" + + + def _get_build_information(self, machine_obj): + build_info = {} + # Generate an identifier for each new build + + build_info['uuid'] = self.uuid + build_info['machine'] = machine_obj + build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0] + build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0] + build_info['started_on'] = datetime.datetime.now() + build_info['completed_on'] = datetime.datetime.now() + build_info['image_fstypes'] = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0] + build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0] + build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0] + build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0] + + return build_info + + def _get_task_information(self, event, recipe): + task_information = {} - task_information['build'] = self.transport_utils['build'] + task_information['build'] = self.internal_state['build'] task_information['outcome'] = 4 if isinstance(event, bb.runqueue.runQueueTaskSkipped): task_information['task_executed'] = False
@@ -108,86 +237,33 @@ class ORMWrapper(object): task_information['outcome'] = 0 else: task_information['task_executed'] = True - task_information['sstate_checksum'] = 'Not Available' - task_information['path_to_sstate_obj'] = 'Not Available' task_information['recipe'] = recipe task_information['task_name'] = event.taskname - task_information['source_url'] = 'Not Available' - task_information['log_file'] = 'Not Available' - task_information['work_directory'] = 'Not Available' - task_information['script_type'] = 0 - task_information['file_path'] = event.taskfile - task_information['line_number'] = 0 - task_information['py_stack_trace'] = 'Not Available' - task_information['disk_io'] = 0 - task_information['cpu_usage'] = 0 - task_information['elapsed_time'] = 'Not Available' - task_information['errors_no'] = 0 - task_information['warnings_no'] = 0 - task_information['error'] = 'Not Available' - task_information['warning'] = 'Not Available' - task_information['sstate_result'] = 0 - return task_information - def create_task_object(self, task_information): - self.task_order += 1 - identifier = task_information['recipe'].file_path + task_information['task_name'] + def _get_build_layer_for_path(self, path): + def _slkey(build_layer): + return len(build_layer.layer.local_path) - task_object = Task.objects.get_or_create( - build=task_information['build'], - order=self.task_order, - task_executed=task_information['task_executed'], - outcome=task_information['outcome'], - sstate_checksum=task_information['sstate_checksum'], - path_to_sstate_obj=task_information['path_to_sstate_obj'], - recipe=task_information['recipe'], - task_name=task_information['task_name'], - source_url=task_information['source_url'], - log_file=task_information['log_file'], - work_directory=task_information['work_directory'], - script_type=task_information['script_type'], - file_path=task_information['file_path'], - line_number=task_information['line_number'], - py_stack_trace=task_information['py_stack_trace'], - disk_io=task_information['disk_io'], - cpu_usage=task_information['cpu_usage'], - elapsed_time=task_information['elapsed_time'], - errors_no=task_information['errors_no'], - warnings_no=task_information['warnings_no'], - error=task_information['error'], - warning=task_information['warning'], - sstate_result=task_information['sstate_result']) - task_object[0].save() - - self.transport_utils[identifier] = {'object': task_object[0], 'start_time': datetime.datetime.now()} - - return task_object[0] - - def update_task_object(self, task_dictionary, event): - - task = task_dictionary['object'] - - duration = datetime.datetime.now() - task_dictionary['start_time'] - task.elapsed_time = duration.total_seconds() + # Heuristics: we always match recipe to the deepest layer path that + # we can match to the recipe file path + for bl in sorted(self.internal_state['build_layers'], reverse=True, key=_slkey): + if (path.startswith(bl.layer.local_path)): + return bl - if isinstance(event, bb.runqueue.runQueueTaskCompleted): - task.outcome = 3 - task.save() + #TODO: if we get here, we didn't read layers correctly + assert False + return None - if isinstance(event, bb.build.TaskBase): - task.recipe.name = event._package - task.recipe.save() + def _get_recipe_information_from_build_event(self, event): - task_build_stats = self._get_task_build_stats(task) - task.cpu_usage = task_build_stats['cpu_usage'] - task.disk_io = task_build_stats['disk_io'] + build_layer_obj = self._get_build_layer_for_path(re.split(':', event.taskfile)[-1]) - #TODO: get error number - #TODO: get warnings number - #TODO: get warning information + recipe_info = {} + recipe_info['build_layer'] = build_layer_obj + recipe_info['file_path'] = re.split(':', event.taskfile)[-1] - task.save() + return recipe_info def _get_task_build_stats(self, task_object): bs_path = self._get_path_information(task_object)
@@ -199,9 +275,9 @@ class ORMWrapper(object): build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/" tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0] - target = self.transport_utils['target'] - machine = self.transport_utils['build'].machine.name - buildname = self.transport_utils['build'].build_name + target = self.internal_state['target'] + machine = self.internal_state['build'].machine.name + buildname = self.internal_state['build'].build_name package = task_object.recipe.name build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target,
@@ -240,177 +316,121 @@ class ORMWrapper(object): return task_build_stats - def _get_recipe_object(self, event): - recipe_information = self._get_recipe_information(event) + ################################ + ## external available methods to store information - recipe_object = Recipe.objects.get_or_create( - name=recipe_information['name'], - version=recipe_information['version'], - layer=recipe_information['layer'], - summary=recipe_information['summary'], - description=recipe_information['description'], - section=recipe_information['section'], - license=recipe_information['license'], - licensing_info=recipe_information['licensing_info'], - homepage=recipe_information['homepage'], - bugtracker=recipe_information['bugtracker'], - author=recipe_information['author'], - file_path=recipe_information['file_path']) - recipe_object[0].save() - - return recipe_object[0] - - def _get_recipe_information(self, event): - - build_layer_obj = self._get_build_layer_object(event) - - recipe_info = {} - recipe_info['name'] = 'N/A' - recipe_info['version'] = 'Not Available' - recipe_info['layer'] = build_layer_obj - recipe_info['summary'] = 'Not Available' - recipe_info['description'] = 'Not Available' - recipe_info['section'] = 'Not Available' - recipe_info['license'] = 'Not Available' - recipe_info['licensing_info'] = 'Not Available' - recipe_info['homepage'] = 'Not Available' - recipe_info['bugtracker'] = 'Not Available' - recipe_info['author'] = 'Not Available' - recipe_info['file_path'] = event.taskfile - - - return recipe_info - - def _get_build_layer_object(self, event): - - build_layer_information = self._get_build_layer_information(event) - - build_layer_object = Build_Layer.objects.get_or_create( - build = build_layer_information['build'], - layer = build_layer_information['layer'], - branch = build_layer_information['branch'], - commit = build_layer_information['commit'], - priority = build_layer_information['priority']) - build_layer_object[0].save() - - return build_layer_object[0] - - def _get_build_layer_information(self, event): - - layer_object = self._get_layer_object(event) - - build_layer_info = {} - build_layer_info['build'] = self.transport_utils['build'] - build_layer_info['layer'] = layer_object - build_layer_info['branch'] = 'Not Available' - build_layer_info['commit'] = 'Not Available' - build_layer_info['priority'] = 0 - - return build_layer_info - - def _get_layer_object(self, event): - - layer_information = self._get_layer_information(event) - - layer_object = Layer.objects.get_or_create( - name=layer_information['name'], - local_path=layer_information['local_path'], - layer_index_url=layer_information['layer_index_url']) - layer_object[0].save() + def store_layer_info(self): + layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ") + self.internal_state['layers'] = [] + for layer_path in { l for l in layers if len(l) }: + layer_information = self._get_layer_dict(layer_path) + self.internal_state['layers'].append(self.orm_wrapper.get_update_layer_object(layer_information)) - return layer_object[0] + def store_started_build(self, event): - def _get_layer_information(self, event): + machine_information = self._get_machine_information() + machine_obj = self.orm_wrapper.create_machine_object(machine_information) + self.uuid = str(uuid.uuid4()) - layer_info = {} - layer_info['name'] = 'Layer Test Name' - layer_info['local_path'] = 'Not Available' - layer_info['layer_index_url'] = 'Not Available' + build_information = self._get_build_information(machine_obj) + build_information['target'] = ' '.join(event.getPkgs()) + build_obj = self.orm_wrapper.create_build_object(build_information) + self.internal_state['build'] = build_obj + self.internal_state['target'] = build_information['target'] + self.internal_state['build_layers'] = [] + for layer_object in self.internal_state['layers']: + build_layer_information = self._get_build_layer_information(layer_object) + self.internal_state['build_layers'].append(self.orm_wrapper.get_build_layer_object(build_layer_information)) - return layer_info + del self.internal_state['layers'] - def _get_layer_dict(self, layer_path): - layer_info = {} - layer_name = layer_path.split('/')[-1] - layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/' - layer_url_name = self._get_url_map_name(layer_name) - - layer_info['name'] = layer_name - layer_info['local_path'] = layer_path - layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name) - return layer_info + def update_build_information(self, event, errors, warnings, taskfailures): + self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures) + del self.internal_state['build'] - def _get_url_map_name(self, layer_name): - """ Some layers have a different name on openembedded.org site, - this method returns the correct name to use in the URL - """ + def store_started_task(self, event): + identifier = re.split(':', event.taskfile)[-1] + event.taskname - url_name = layer_name - url_mapping = {'meta': 'openembedded-core'} + recipe_information = self._get_recipe_information_from_build_event(event) + recipe = self.orm_wrapper.get_update_recipe_object(recipe_information) - for key in url_mapping.keys(): - if key == layer_name: - url_name = url_mapping[key] + task_information = self._get_task_information(event, recipe) - return url_name + self.task_order += 1 + task_information['order'] = self.task_order + task_obj = self.orm_wrapper.get_update_task_object(task_information) - def _save_layer_object(self, layer_path): + self.internal_state[identifier] = {'start_time': datetime.datetime.now()} - layer_information = self._get_layer_dict(layer_path) + def update_and_store_task(self, event): + identifier = re.split(':', event.taskfile)[-1] + event.taskname + recipe_information = self._get_recipe_information_from_build_event(event) + recipe = self.orm_wrapper.get_update_recipe_object(recipe_information) + task_information = self._get_task_information(event,recipe) + try: + task_information['start_time'] = self.internal_state[identifier]['start_time'] + except: + pass - layer_object = Layer.objects.get_or_create( - name=layer_information['name'], - local_path=layer_information['local_path'], - layer_index_url=layer_information['layer_index_url']) + if isinstance(event, bb.runqueue.runQueueTaskCompleted): + task_information['outcome'] = 3 # TODO: needs to use constants + task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information)) + task_information['cpu_usage'] = task_build_stats['cpu_usage'] + task_information['disk_io'] = task_build_stats['disk_io'] + del self.internal_state[identifier] - layer_object[0].save() + #TODO: get error number + #TODO: get warnings number + #TODO: get warning information - def store_layer_info(self): - layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ") - for layer_path in layers: - self._save_layer_object(layer_path) + self.orm_wrapper.get_update_task_object(task_information) -class BuildInfoHelper(object): - """ This class gathers the build information from the server and sends it - towards the ORM wrapper for storing in the database - """ - def __init__(self, server): - self._configure_django() - self.orm_wrapper = ORMWrapper(server) - def _configure_django(self): - # Add webhob to sys path for importing modules - sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'webhob')) + def store_dependency_information(self, event): + self.internal_state['recipes'] = {} + for pn in event._depgraph['pn']: - def store_started_build(self, event): + file_name = re.split(':', event._depgraph['pn'][pn]['filename'])[-1] + build_layer_obj = self._get_build_layer_for_path(re.split(':', file_name)[-1]) - self.orm_wrapper.store_layer_info() + assert build_layer_obj is not None - machine_information = self.orm_wrapper.get_machine_information() - machine_obj = self.orm_wrapper.create_machine_object(machine_information) + recipe_info = {} + recipe_info['name'] = pn + recipe_info['version'] = event._depgraph['pn'][pn]['version'] + recipe_info['build_layer'] = build_layer_obj - build_information = self.orm_wrapper.get_build_information(machine_obj) - build_information['target'] = ' '.join(event.getPkgs()) - build_obj = self.orm_wrapper.create_build_object(build_information) - self.orm_wrapper.transport_utils['build'] = build_obj - self.orm_wrapper.transport_utils['target'] = build_information['target'] + # TODO: dump all this info in the deptree event + recipe_info['summary'] = 'Not Available' + recipe_info['description'] = 'Not Available' + recipe_info['section'] = 'Not Available' + recipe_info['license'] = 'Not Available' + recipe_info['licensing_info'] = 'Not Available' + recipe_info['homepage'] = 'Not Available' + recipe_info['bugtracker'] = 'Not Available' + recipe_info['author'] = 'Not Available' + recipe_info['file_path'] = file_name + recipe = self.orm_wrapper.get_update_recipe_object(recipe_info) + self.internal_state['recipes'][pn] = recipe - def update_build_information(self, event, errors, warnings, taskfailures): - self.orm_wrapper.update_build_object(self.orm_wrapper.transport_utils['build'], errors, warnings, taskfailures) - del self.orm_wrapper.transport_utils['build'] + def _save_a_task(taskdesc): + pn, taskname = re.split(r'\.', taskdesc); + e = event + e.taskname = pn + recipe = self.internal_state['recipes'][pn] + task_info = self._get_task_information(e, recipe) + task_info['task_name'] = taskname + task_obj = self.orm_wrapper.get_update_task_object(task_info) + return task_obj - def store_started_task(self, event): - task_information = self.orm_wrapper.get_task_information(event) - task_obj = self.orm_wrapper.create_task_object(task_information) + for taskdesc in event._depgraph['tdepends']: + _save_a_task(taskdesc) + for taskdesc1 in event._depgraph['tdepends'][taskdesc]: + _save_a_task(taskdesc1) - def update_and_store_task(self, event): - identifier = event.taskfile + event.taskname - task_dictionary = self.orm_wrapper.transport_utils[identifier] - self.orm_wrapper.update_task_object(task_dictionary, event) - if isinstance(event,bb.runqueue.runQueueTaskCompleted): - del self.orm_wrapper.transport_utils[identifier] + del self.internal_state['recipes']
bitbake/lib/bb/ui/dsi.py +14 -0
@@ -17,6 +17,11 @@ # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. from __future__ import division +try: + import bb +except RuntimeError as exc: + sys.exit(str(exc)) + from bb.ui import uihelper from bb.ui.buildinfohelper import BuildInfoHelper
@@ -32,6 +37,7 @@ import sys import time import xmlrpclib +featureSet = [bb.cooker.CookerFeatures.HOB_EXTRA_CACHES, bb.cooker.CookerFeatures.SEND_DEPENDS_TREE] logger = logging.getLogger("BitBake") interactive = sys.stdout.isatty()
@@ -268,7 +274,9 @@ def main(server, eventHandler, params, tf = TerminalFilter): errors = 0 warnings = 0 taskfailures = [] + buildinfohelper = BuildInfoHelper(server) + buildinfohelper.store_layer_info() termfilter = tf(main, helper, console, format)
@@ -464,8 +472,10 @@ def main(server, eventHandler, params, tf = TerminalFilter): bb.command.CommandFailed, bb.command.CommandExit)): buildinfohelper.update_build_information(event, errors, warnings, taskfailures) + # we start a new build info buildinfohelper = BuildInfoHelper(server) + buildinfohelper.store_layer_info() continue # ignore
@@ -481,6 +491,10 @@ def main(server, eventHandler, params, tf = TerminalFilter): bb.cooker.CookerExit)): continue + if isinstance(event, bb.event.DepTreeGenerated): + buildinfohelper.store_dependency_information(event) + continue + logger.error("Unknown event: %s", event) except EnvironmentError as ioerror:
bitbake/lib/webhob/bldviewer/templates/task.html +1 -1
@@ -64,7 +64,7 @@ <td>{{task.path_to_sstate_obj}}</td> <td>{{task.source_url}}</td> <td>{{task.get_script_type_display}}</td> - <td>{{task.file_path}}</td> + <td>{{task.recipe.file_path}}</td> <td>{{task.line_number}}</td> <td>{{task.py_stack_trace}}</td> <td>{{task.get_sstate_result_display}}</td>
bitbake/lib/webhob/orm/models.py +35 -32
95 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH 27/94 bitbake: webhob: record dependency info
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
This patch adds the task dependency information to the database, and displays it in a primitive form for verification.
--- bitbake/lib/bb/ui/buildinfohelper.py | 9 ++++--- bitbake/lib/webhob/bldviewer/templates/task.html | 33 +++++++++--------------- bitbake/lib/webhob/bldviewer/views.py | 9 ++++++- 3 files changed, 26 insertions(+), 25 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +6 -3
@@ -8,7 +8,9 @@ import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package, Task_Dependency, Package_Dependency +from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package +from webhob.orm.models import Task_Dependency + class ORMWrapper(object): """ This class creates the dictionaries needed to store information in the database
@@ -429,8 +431,9 @@ class BuildInfoHelper(object): return task_obj for taskdesc in event._depgraph['tdepends']: - _save_a_task(taskdesc) + target = _save_a_task(taskdesc) for taskdesc1 in event._depgraph['tdepends'][taskdesc]: - _save_a_task(taskdesc1) + dep = _save_a_task(taskdesc1) + Task_Dependency.objects.get_or_create( task = target, depends_on = dep ) del self.internal_state['recipes']
bitbake/lib/webhob/bldviewer/templates/task.html +12 -21
@@ -17,57 +17,47 @@ <tr> <th>Order</th> <th>Task</th> - <th>Recipe</th> <th>Recipe Version</th> <th>Task Type</th> <th>Outcome</th> <th>Errors</th> <th>Warnings</th> <th>Time</th> - <th>Log</th> - <th>Work directory</th> <th>CPU usage</th> <th>Disk I/O</th> - <th>Sstate Checksum</th> - <th>Path to sstate object</th> - <th>Source URL</th> <th>Script type</th> <th>File path</th> - <th>Line number</th> - <th>Python Stack Trace</th> - <th>Sstate Result</th> + <th>Depends</th> </tr> {% for task in tasks %} <tr> <td>{{task.order}}</td> - <td>{{task.task_name}}</td> - <td>{{task.recipe.name}}</td> + <td><a name="{{task.recipe.name}}.{{task.task_name}}"> + {{task.recipe.name}}.{{task.task_name}}</a></td> <td>{{task.recipe.version}}</td> {% if task.task_executed %} - <td>Executed</td> + <td>Executed</td> {% else %} - <td>Prebuilt</td> + <td>Prebuilt</td> {% endif %} <td>{{task.get_outcome_display}}</td> <td>{{task.errors_no}}</td> <td>{{task.warnings_no}}</td> <td>{{task.elapsed_time}}</td> - <td>{{task.log_file}}</td> - <td>{{task.work_directory}}</td> <td>{{task.cpu_usage}}</td> <td>{{task.disk_io}}</td> - <td>{{task.sstate_checksum}}</td> - <td>{{task.path_to_sstate_obj}}</td> - <td>{{task.source_url}}</td> <td>{{task.get_script_type_display}}</td> <td>{{task.recipe.file_path}}</td> - <td>{{task.line_number}}</td> - <td>{{task.py_stack_trace}}</td> - <td>{{task.get_sstate_result_display}}</td> + <td> + {% for tt in task.depends_on %} + <a href="#{{tt.recipe.name}}.{{tt.task_name}}"> + {{tt.recipe.name}}.{{tt.task_name}}</a><br/> + {% endfor %} + </td> </tr> {% endfor %}
@@ -79,3 +69,4 @@ </body> </html> +
bitbake/lib/webhob/bldviewer/views.py +8 -1
@@ -1,5 +1,5 @@ from django.shortcuts import render -from orm.models import Build, Task, Layer +from orm.models import Build, Task, Layer, Task_Dependency def build(request):
@@ -14,6 +14,13 @@ def task(request, build_id): template = 'task.html' tasks = Task.objects.filter(build=build_id) + task_depends = Task_Dependency.objects.filter(task__in=tasks) + + for t in tasks: + t.depends_on = [] + for k in task_depends: + if t == k.task: + t.depends_on.append(k.depends_on) context = {'tasks': tasks}
Signature · 1 line
1.8.1.2
PATCH 28/94 bitbake: webhob: use defined constants for models
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We change the models to use constants to improve
the legibility of the code.
--- bitbake/lib/bb/ui/buildinfohelper.py | 13 +++++---- bitbake/lib/webhob/orm/models.py | 55 ++++++++++++++++++++++++------------ 2 files changed, 44 insertions(+), 24 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +7 -6
@@ -47,9 +47,9 @@ class ORMWrapper(object): def update_build_object(self, build_obj, errors, warnings, taskfailures): - outcome = 0 + outcome = Build.SUCCEEDED if errors or taskfailures: - outcome = 1 + outcome = Build.FAILED build = Build.objects.get(uuid=build_obj.uuid) build.completed_on = datetime.datetime.now()
@@ -230,13 +230,13 @@ class BuildInfoHelper(object): task_information = {} task_information['build'] = self.internal_state['build'] - task_information['outcome'] = 4 + task_information['outcome'] = Task.OUTCOME_NA if isinstance(event, bb.runqueue.runQueueTaskSkipped): task_information['task_executed'] = False if event._skip == "covered": - task_information['outcome'] = 2 + task_information['outcome'] = Task.OUTCOME_COVERED if event._skip == "existing": - task_information['outcome'] = 0 + task_information['outcome'] = Task.OUTCOME_EXISTING else: task_information['task_executed'] = True task_information['recipe'] = recipe
@@ -363,6 +363,7 @@ class BuildInfoHelper(object): self.task_order += 1 task_information['order'] = self.task_order + task_information['outcome'] = Task.OUTCOME_NA task_obj = self.orm_wrapper.get_update_task_object(task_information) self.internal_state[identifier] = {'start_time': datetime.datetime.now()}
@@ -378,7 +379,7 @@ class BuildInfoHelper(object): pass if isinstance(event, bb.runqueue.runQueueTaskCompleted): - task_information['outcome'] = 3 # TODO: needs to use constants + task_information['outcome'] = Task.OUTCOME_SUCCESS # TODO: needs to use constants task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information)) task_information['cpu_usage'] = task_build_stats['cpu_usage'] task_information['disk_io'] = task_build_stats['disk_io']
bitbake/lib/webhob/orm/models.py +37 -18
99 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH 29/94 bitbake: dsi: fix the reading of task event information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Fixing a problem where the event outcome wasn't processed. Moving all the event-type-dependent code at the same level in the class.
--- bitbake/lib/bb/ui/buildinfohelper.py | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +14 -9
@@ -231,14 +231,6 @@ class BuildInfoHelper(object): task_information = {} task_information['build'] = self.internal_state['build'] task_information['outcome'] = Task.OUTCOME_NA - if isinstance(event, bb.runqueue.runQueueTaskSkipped): - task_information['task_executed'] = False - if event._skip == "covered": - task_information['outcome'] = Task.OUTCOME_COVERED - if event._skip == "existing": - task_information['outcome'] = Task.OUTCOME_EXISTING - else: - task_information['task_executed'] = True task_information['recipe'] = recipe task_information['task_name'] = event.taskname return task_information
@@ -360,10 +352,19 @@ class BuildInfoHelper(object): recipe = self.orm_wrapper.get_update_recipe_object(recipe_information) task_information = self._get_task_information(event, recipe) + task_information['outcome'] = Task.OUTCOME_NA + + if isinstance(event, bb.runqueue.runQueueTaskSkipped): + task_information['task_executed'] = False + if event.reason == "covered": + task_information['outcome'] = Task.OUTCOME_COVERED + if event.reason == "existing": + task_information['outcome'] = Task.OUTCOME_EXISTING + else: + task_information['task_executed'] = True self.task_order += 1 task_information['order'] = self.task_order - task_information['outcome'] = Task.OUTCOME_NA task_obj = self.orm_wrapper.get_update_task_object(task_information) self.internal_state[identifier] = {'start_time': datetime.datetime.now()}
@@ -385,6 +386,10 @@ class BuildInfoHelper(object): task_information['disk_io'] = task_build_stats['disk_io'] del self.internal_state[identifier] + if isinstance(event, bb.runqueue.runQueueTaskFailed): + task_information['outcome'] = Task.OUTCOME_FAILED + del self.internal_state[identifier] + #TODO: get error number #TODO: get warnings number #TODO: get warning information
Signature · 1 line
1.8.1.2
PATCH 30/94 bitbake: webhob: Setup API for build model
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
Adding REST API version 1.0 interface of WebHob.
Build call API is implemented in this patch
Documentation is here:
https://wiki.yoctoproject.org/wiki/Webhob_REST_API_Builds
--- bitbake/lib/webhob/bldviewer/api.py | 6 ++++ bitbake/lib/webhob/bldviewer/views.py | 56 +++++++++++++++++++++++++++++++++++ bitbake/lib/webhob/whbmain/urls.py | 1 + 3 files changed, 63 insertions(+) create mode 100644 bitbake/lib/webhob/bldviewer/api.py
bitbake/lib/webhob/bldviewer/api.py new file +6 -0
@@ -0,0 +1,6 @@ +from django.conf.urls import patterns, include, url + + +urlpatterns = patterns('bldviewer.views', + url(r'^builds/$', 'builds', name='builds'), +)
bitbake/lib/webhob/bldviewer/views.py +56 -0
59 lines not on this page · raw message
bitbake/lib/webhob/whbmain/urls.py +1 -0
@@ -6,6 +6,7 @@ from django.conf.urls import patterns, include, url urlpatterns = patterns('', url(r'^build/', include('bldviewer.urls')), + url(r'^api/1.0/', include('bldviewer.api')), url(r'^gui/', include('whbgui.urls')), # Examples: # url(r'^webhob/', include('webhob.foo.urls')),
Signature · 1 line
1.8.1.2
PATCH 31/94 bitbake: webhob: orm change to remove Target
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Removes the Target dependency as it is replaced
by a field in Build.
--- bitbake/lib/webhob/orm/models.py | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-)
bitbake/lib/webhob/orm/models.py +2 -11
@@ -14,6 +14,7 @@ class Build(models.Model): uuid = models.CharField(max_length=100, unique=True) target = models.CharField(max_length=100) + is_image = models.BooleanField() machine = models.ForeignKey('Machine', related_name='build_machine') distro = models.CharField(max_length=100) distro_version = models.CharField(max_length=100)
@@ -98,24 +99,14 @@ class Task_Dependency(models.Model): depends_on = models.ForeignKey(Task, related_name='task_dependencies_depends') -class Target(models.Model): - build = models.ForeignKey(Build, related_name='target_build') - is_image = models.BooleanField() - - class Artifact(models.Model): build = models.ForeignKey(Build, related_name='artifact_build') - target = models.ForeignKey(Target, related_name='artifact_target') file_name = models.CharField(max_length=100) file_size = models.IntegerField() -class Package_In_Image(models.Model): - package = models.ForeignKey('Package', related_name='package_in_image_package') - target = models.ForeignKey(Target, related_name='package_in_image_target') - - class Package(models.Model): + build = models.ForeignKey('Build', related_name='package_build') recipe = models.ForeignKey('Recipe', related_name='package_recipe') name = models.CharField(max_length=100) version = models.CharField(max_length=100)
Signature · 1 line
1.8.1.2
PATCH 32/94 bitbake: webhob: clear up ORM relations
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
After a bit of analysis, it's become clear that Recipes only depend on a specific Layer version, and not on the current build.
So I clear up the Build_Layer transforming it to Layer_Version and remove the Build dependency.
--- bitbake/lib/bb/ui/buildinfohelper.py | 61 ++++++++++++++++++------------------ bitbake/lib/webhob/orm/models.py | 7 ++--- 2 files changed, 33 insertions(+), 35 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +30 -31
@@ -8,7 +8,7 @@ import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Build_Layer, Layer, Package +from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package from webhob.orm.models import Task_Dependency
@@ -81,7 +81,7 @@ class ORMWrapper(object): def get_update_recipe_object(self, recipe_information): recipe_object, created = Recipe.objects.get_or_create( - build_layer=recipe_information['build_layer'], + layer_version=recipe_information['layer_version'], file_path=recipe_information['file_path']) for v in vars(recipe_object):
@@ -92,19 +92,18 @@ class ORMWrapper(object): return recipe_object - def get_build_layer_object(self, build_layer_information): + def get_layer_version_object(self, layer_version_information): - build_layer_object = Build_Layer.objects.get_or_create( - build = build_layer_information['build'], - layer = build_layer_information['layer'], - branch = build_layer_information['branch'], - commit = build_layer_information['commit'], - priority = build_layer_information['priority'] + layer_version_object = Layer_Version.objects.get_or_create( + layer = layer_version_information['layer'], + branch = layer_version_information['branch'], + commit = layer_version_information['commit'], + priority = layer_version_information['priority'] ) - build_layer_object[0].save() + layer_version_object[0].save() - return build_layer_object[0] + return layer_version_object[0] def get_update_layer_object(self, layer_information):
@@ -179,16 +178,16 @@ class BuildInfoHelper(object): return layer_info - def _get_build_layer_information(self, layer_object): + def _get_layer_version_information(self, layer_object): - build_layer_info = {} - build_layer_info['build'] = self.internal_state['build'] - build_layer_info['layer'] = layer_object - build_layer_info['branch'] = self._get_git_branch(layer_object.local_path) - build_layer_info['commit'] = self._get_git_revision(layer_object.local_path) - build_layer_info['priority'] = 0 + layer_version_info = {} + layer_version_info['build'] = self.internal_state['build'] + layer_version_info['layer'] = layer_object + layer_version_info['branch'] = self._get_git_branch(layer_object.local_path) + layer_version_info['commit'] = self._get_git_revision(layer_object.local_path) + layer_version_info['priority'] = 0 - return build_layer_info + return layer_version_info def _get_git_branch(self, layer_path):
@@ -235,13 +234,13 @@ class BuildInfoHelper(object): task_information['task_name'] = event.taskname return task_information - def _get_build_layer_for_path(self, path): - def _slkey(build_layer): - return len(build_layer.layer.local_path) + def _get_layer_version_for_path(self, path): + def _slkey(layer_version): + return len(layer_version.layer.local_path) # Heuristics: we always match recipe to the deepest layer path that # we can match to the recipe file path - for bl in sorted(self.internal_state['build_layers'], reverse=True, key=_slkey): + for bl in sorted(self.internal_state['layer_versions'], reverse=True, key=_slkey): if (path.startswith(bl.layer.local_path)): return bl
@@ -251,10 +250,10 @@ class BuildInfoHelper(object): def _get_recipe_information_from_build_event(self, event): - build_layer_obj = self._get_build_layer_for_path(re.split(':', event.taskfile)[-1]) + layer_version_obj = self._get_layer_version_for_path(re.split(':', event.taskfile)[-1]) recipe_info = {} - recipe_info['build_layer'] = build_layer_obj + recipe_info['layer_version'] = layer_version_obj recipe_info['file_path'] = re.split(':', event.taskfile)[-1] return recipe_info
@@ -332,10 +331,10 @@ class BuildInfoHelper(object): build_obj = self.orm_wrapper.create_build_object(build_information) self.internal_state['build'] = build_obj self.internal_state['target'] = build_information['target'] - self.internal_state['build_layers'] = [] + self.internal_state['layer_versions'] = [] for layer_object in self.internal_state['layers']: - build_layer_information = self._get_build_layer_information(layer_object) - self.internal_state['build_layers'].append(self.orm_wrapper.get_build_layer_object(build_layer_information)) + layer_version_information = self._get_layer_version_information(layer_object) + self.internal_state['layer_versions'].append(self.orm_wrapper.get_layer_version_object(layer_version_information)) del self.internal_state['layers']
@@ -404,14 +403,14 @@ class BuildInfoHelper(object): for pn in event._depgraph['pn']: file_name = re.split(':', event._depgraph['pn'][pn]['filename'])[-1] - build_layer_obj = self._get_build_layer_for_path(re.split(':', file_name)[-1]) + layer_version_obj = self._get_layer_version_for_path(re.split(':', file_name)[-1]) - assert build_layer_obj is not None + assert layer_version_obj is not None recipe_info = {} recipe_info['name'] = pn recipe_info['version'] = event._depgraph['pn'][pn]['version'] - recipe_info['build_layer'] = build_layer_obj + recipe_info['layer_version'] = layer_version_obj # TODO: dump all this info in the deptree event recipe_info['summary'] = 'Not Available'
bitbake/lib/webhob/orm/models.py +3 -4
@@ -127,7 +127,7 @@ class Filelist(models.Model): class Recipe(models.Model): name = models.CharField(max_length=100, null=True) version = models.CharField(max_length=100, null=True) - build_layer = models.ForeignKey('Build_Layer', related_name='recipe_build_layer') + layer_version = models.ForeignKey('Layer_Version', related_name='recipe_layer_version') summary = models.CharField(max_length=100, null=True) description = models.CharField(max_length=100, null=True) section = models.CharField(max_length=100, null=True)
@@ -150,9 +150,8 @@ class Layer(models.Model): layer_index_url = models.URLField() -class Build_Layer(models.Model): - build = models.ForeignKey(Build, related_name='build_layer_build') - layer = models.ForeignKey(Layer, related_name='build_layer_layer') +class Layer_Version(models.Model): + layer = models.ForeignKey(Layer, related_name='layer_version_layer') branch = models.CharField(max_length=50) commit = models.CharField(max_length=100) priority = models.IntegerField()
Signature · 1 line
1.8.1.2
PATCH 33/94 bitbake: dsi: save detailed recipe information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Code to save the recipe information coming in the dependency tree. Also marks recipes that generate images.
--- bitbake/lib/bb/ui/buildinfohelper.py | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +11 -9
@@ -411,22 +411,24 @@ class BuildInfoHelper(object): recipe_info['name'] = pn recipe_info['version'] = event._depgraph['pn'][pn]['version'] recipe_info['layer_version'] = layer_version_obj - - # TODO: dump all this info in the deptree event - recipe_info['summary'] = 'Not Available' - recipe_info['description'] = 'Not Available' - recipe_info['section'] = 'Not Available' - recipe_info['license'] = 'Not Available' + recipe_info['summary'] = event._depgraph['pn'][pn]['summary'] + recipe_info['license'] = event._depgraph['pn'][pn]['license'] + recipe_info['description'] = event._depgraph['pn'][pn]['summary'] + recipe_info['section'] = event._depgraph['pn'][pn]['section'] recipe_info['licensing_info'] = 'Not Available' - recipe_info['homepage'] = 'Not Available' - recipe_info['bugtracker'] = 'Not Available' + recipe_info['homepage'] = event._depgraph['pn'][pn]['homepage'] + recipe_info['bugtracker'] = event._depgraph['pn'][pn]['bugtracker'] recipe_info['author'] = 'Not Available' recipe_info['file_path'] = file_name recipe = self.orm_wrapper.get_update_recipe_object(recipe_info) + recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits']) self.internal_state['recipes'][pn] = recipe + # save all task information def _save_a_task(taskdesc): - pn, taskname = re.split(r'\.', taskdesc); + spec = re.split(r'\.', taskdesc); + pn = ".".join(spec[0:-1]) + taskname = spec[-1] e = event e.taskname = pn recipe = self.internal_state['recipes'][pn]
Signature · 1 line
1.8.1.2
PATCH 34/94 bitbake: webhob: determine if a build is an image
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding code to see if a build will generate a image based on the inheritance of image.bbclass in the recipe defining the build.
--- bitbake/lib/bb/ui/buildinfohelper.py | 4 ++++ bitbake/lib/webhob/bldviewer/templates/build.html | 2 ++ bitbake/lib/webhob/orm/models.py | 2 +- 3 files changed, 7 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/buildinfohelper.py +4 -0
@@ -328,6 +328,7 @@ class BuildInfoHelper(object): build_information = self._get_build_information(machine_obj) build_information['target'] = ' '.join(event.getPkgs()) + build_obj = self.orm_wrapper.create_build_object(build_information) self.internal_state['build'] = build_obj self.internal_state['target'] = build_information['target']
@@ -422,6 +423,9 @@ class BuildInfoHelper(object): recipe_info['file_path'] = file_name recipe = self.orm_wrapper.get_update_recipe_object(recipe_info) recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits']) + if recipe.is_image and pn == self.internal_state['build'].target: + self.internal_state['build'].is_image = True + self.internal_state['build'].save() self.internal_state['recipes'][pn] = recipe # save all task information
bitbake/lib/webhob/bldviewer/templates/build.html +2 -0
@@ -16,6 +16,7 @@ <th>Started On</th> <th>Completed On</th> <th>Target</th> + <th>Image ?</th> <th>Machine</th> <th>Time</th> <th>Errors</th>
@@ -33,6 +34,7 @@ <td>{{build.started_on}}</td> <td>{{build.completed_on}}</td> <td>{{build.target}}</td> + <td>{{build.is_image}}</td> <td>{{build.machine.name}}</td> <td>{% time_difference build.started_on build.completed_on %}</td> <td>{{build.errors_no}}</td>
bitbake/lib/webhob/orm/models.py +1 -1
@@ -14,7 +14,7 @@ class Build(models.Model): uuid = models.CharField(max_length=100, unique=True) target = models.CharField(max_length=100) - is_image = models.BooleanField() + is_image = models.BooleanField(default = False) machine = models.ForeignKey('Machine', related_name='build_machine') distro = models.CharField(max_length=100) distro_version = models.CharField(max_length=100)
Signature · 1 line
1.8.1.2
PATCH 35/94 bitbake: webhob: clean up URL structure
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We clean up the URL structure.
* default redirect to GUI version
* basic HTML interface now available at /simple/
* clear-up simple/ paths towards object-type URLs
--- bitbake/lib/webhob/bldviewer/urls.py | 6 ++++-- bitbake/lib/webhob/whbmain/urls.py | 5 ++++- 2 files changed, 8 insertions(+), 3 deletions(-)
bitbake/lib/webhob/bldviewer/urls.py +4 -2
@@ -1,8 +1,10 @@ from django.conf.urls import patterns, include, url +from django.views.generic.simple import redirect_to urlpatterns = patterns('bldviewer.views', - url(r'^$', 'build', name='build'), + url(r'^build/$', 'build', name='build'), url(r'^layers/$', 'layer', name='layer'), - url(r'^(?P<build_id>\d+)/$', 'task', name='task'), + url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), + url(r'^$', redirect_to, {'url': 'build/'}), )
bitbake/lib/webhob/whbmain/urls.py +4 -1
@@ -1,13 +1,16 @@ from django.conf.urls import patterns, include, url +from django.views.generic.simple import redirect_to + # Uncomment the next two lines to enable the admin: # from django.contrib import admin # admin.autodiscover() urlpatterns = patterns('', - url(r'^build/', include('bldviewer.urls')), + url(r'^simple/', include('bldviewer.urls')), url(r'^api/1.0/', include('bldviewer.api')), url(r'^gui/', include('whbgui.urls')), + url(r'^$', redirect_to, {'url': '/gui/'}), # Examples: # url(r'^webhob/', include('webhob.foo.urls')),
Signature · 1 line
1.8.1.2
PATCH 36/94 bitbake: webhob: add simple pages for Recipes
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We add simple pages to display database information about Layer versions and Recipes.
--- bitbake/lib/webhob/bldviewer/templates/layer.html | 8 ++++ bitbake/lib/webhob/bldviewer/templates/recipe.html | 48 ++++++++++++++++++++++ bitbake/lib/webhob/bldviewer/urls.py | 4 +- bitbake/lib/webhob/bldviewer/views.py | 15 ++++++- 4 files changed, 72 insertions(+), 3 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/recipe.html
bitbake/lib/webhob/bldviewer/templates/layer.html +8 -0
@@ -15,6 +15,7 @@ <th>Name</th> <th>Local Path</th> <th>Layer Index URL</th> + <th>Known Versions</th> </tr> {% for layer in layers %}
@@ -23,6 +24,13 @@ <td>{{layer.name}}</td> <td>{{layer.local_path}}</td> <td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td> + <td><table> + {% for lv in layer.versions %} + <tr><td> + <a href="/simple/layerversions/{{lv.id}}/recipes">{{lv.branch}}:{{lv.commit}} ({{lv.count}} recipes)</a> + </td></tr> + {% endfor %} + </table></td> </tr> {% endfor %}
bitbake/lib/webhob/bldviewer/templates/recipe.html new file +48 -0
48 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/urls.py +2 -2
@@ -1,10 +1,10 @@ from django.conf.urls import patterns, include, url from django.views.generic.simple import redirect_to - urlpatterns = patterns('bldviewer.views', url(r'^build/$', 'build', name='build'), - url(r'^layers/$', 'layer', name='layer'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), + url(r'^layer/$', 'layer', name='layer'), + url(r'^layerversions/(?P<layerversion_id>\d+)/recipes$', 'layer_versions_recipes', name='layer_versions_recipes'), url(r'^$', redirect_to, {'url': 'build/'}), )
bitbake/lib/webhob/bldviewer/views.py +14 -1
@@ -1,5 +1,5 @@ from django.shortcuts import render -from orm.models import Build, Task, Layer, Task_Dependency +from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe def build(request):
@@ -30,11 +30,24 @@ def layer(request): template = 'layer.html' layer_info = Layer.objects.all() + for li in layer_info: + li.versions = Layer_Version.objects.filter(layer = li) + for liv in li.versions: + liv.count = Recipe.objects.filter(layer_version__id = liv.id).count() + context = {'layers': layer_info} return render(request, template, context) +def layer_versions_recipes(request, layerversion_id): + template = 'recipe.html' + recipes = Recipe.objects.filter(layer_version__id = layerversion_id) + + context = {'recipes': recipes} + + return render(request, template, context) + #### API import json
Signature · 1 line
1.8.1.2
PATCH 37/94 bitbake: webhob: add ordering capabilities to build api
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch adds the capability to order builds by a specific field of the build model.
The format used is 'field_name:order' and the default ordering is ascending.
--- bitbake/lib/webhob/bldviewer/views.py | 9 +++++++++ 1 file changed, 9 insertions(+)
bitbake/lib/webhob/bldviewer/views.py +9 -0
@@ -56,6 +56,7 @@ from django.http import HttpResponse def builds(request): + DESCENDING = 'desc' response_data = {} try:
@@ -68,6 +69,7 @@ def builds(request): except ValueError: offset = 0 + ordering_string = request.GET.get('orderby', '') filter_string = request.GET.get('filter', '') if filter_string:
@@ -91,6 +93,13 @@ def builds(request): else: response_data['count'] = 0 + if queryset and ordering_string: + column, order = ordering_string.split(':') + if order.lower() == DESCENDING: + queryset = queryset.order_by('-' + column) + else: + queryset = queryset.order_by(column) + response_data['list'] = serializers.serialize('json', queryset) return HttpResponse(json.dumps(response_data), content_type='application/json')
Signature · 1 line
1.8.1.2
PATCH 38/94 bitbake: webhob: validate inputs for build api
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch validates the inputs for the build api by checking the following:
- have only one colon;
- have equal number of terms on both sides of the colon;
- left side terms must be part of the Django model fields.
--- bitbake/lib/webhob/bldviewer/views.py | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)
bitbake/lib/webhob/bldviewer/views.py +30 -3
@@ -52,7 +52,7 @@ def layer_versions_recipes(request, layerversion_id): import json from django.core import serializers -from django.http import HttpResponse +from django.http import HttpResponse, HttpResponseBadRequest def builds(request):
@@ -69,8 +69,13 @@ def builds(request): except ValueError: offset = 0 - ordering_string = request.GET.get('orderby', '') - filter_string = request.GET.get('filter', '') + ordering_string, invalid = _validate_input(request.GET.get('orderby', '')) + if invalid: + return HttpResponseBadRequest() + + filter_string, invalid = _validate_input(request.GET.get('filter', '')) + if invalid: + return HttpResponseBadRequest() if filter_string: filter_terms = _get_filtering_terms(filter_string)
@@ -111,3 +116,25 @@ def _get_filtering_terms(filter_string): values = search_terms[1].split(',') return dict(zip(keys, values)) + +def _validate_input(input): + invalid = 0 + + if input: + input_list = input.split(":") + + # Check we have only one colon + if len(input.split(":")) != 2: + invalid = 1 + + # Check we have an equal number of terms both sides of the colon + if len(input_list[0].split(',')) != len(input_list[1].split(',')): + invalid = 1 + + # Check we are looking for a valid field + valid_fields = Build._meta.get_all_field_names() + for field in input_list[0].split(','): + if field not in valid_fields: + invalid = 1 + + return input, invalid
Signature · 1 line
1.8.1.2
PATCH 39/94 bitbake: webhob: generic view for multiple models
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch creates a generic view that supports multiple models. It allows exploration of the fields of the models and it has validation of inputs given.
Also avoids code replication.
Conflicts:
bitbake/lib/webhob/bldviewer/views.py
--- bitbake/lib/webhob/bldviewer/api.py | 4 +++- bitbake/lib/webhob/bldviewer/views.py | 33 ++++++++++++++++++++++++--------- 2 files changed, 27 insertions(+), 10 deletions(-)
bitbake/lib/webhob/bldviewer/api.py +3 -1
@@ -2,5 +2,7 @@ from django.conf.urls import patterns, include, url urlpatterns = patterns('bldviewer.views', - url(r'^builds/$', 'builds', name='builds'), + url(r'^builds/$', 'model_explorer', {'model_name':'build'}, name='builds'), + url(r'^tasks/$', 'model_explorer', {'model_name':'task'}, name='task'), + url(r'^packages/$', 'model_explorer', {'model_name':'package'}, name='package'), )
bitbake/lib/webhob/bldviewer/views.py +24 -9
@@ -1,5 +1,5 @@ from django.shortcuts import render -from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe +from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package def build(request):
@@ -55,9 +55,20 @@ from django.core import serializers from django.http import HttpResponse, HttpResponseBadRequest -def builds(request): +def model_explorer(request, model_name): + DESCENDING = 'desc' response_data = {} + model_mapping = { + 'build': Build, + 'task': Task, + 'package': Package, + } + + if model_name not in model_mapping.keys(): + return HttpResponseBadRequest() + + model = model_mapping[model_name] try: limit = int(request.GET.get('limit', 0))
@@ -69,22 +80,24 @@ def builds(request): except ValueError: offset = 0 - ordering_string, invalid = _validate_input(request.GET.get('orderby', '')) + ordering_string, invalid = _validate_input(request.GET.get('orderby', ''), + model) if invalid: return HttpResponseBadRequest() - filter_string, invalid = _validate_input(request.GET.get('filter', '')) + filter_string, invalid = _validate_input(request.GET.get('filter', ''), + model) if invalid: return HttpResponseBadRequest() if filter_string: filter_terms = _get_filtering_terms(filter_string) try: - queryset = Build.objects.filter(**filter_terms) + queryset = model.objects.filter(**filter_terms) except ValueError: queryset = [] else: - queryset = Build.objects.all() + queryset = model.objects.all() if offset and limit: queryset = queryset[offset:(offset+limit)]
@@ -107,7 +120,8 @@ def builds(request): response_data['list'] = serializers.serialize('json', queryset) - return HttpResponse(json.dumps(response_data), content_type='application/json') + return HttpResponse(json.dumps(response_data), + content_type='application/json') def _get_filtering_terms(filter_string):
@@ -117,7 +131,8 @@ def _get_filtering_terms(filter_string): return dict(zip(keys, values)) -def _validate_input(input): +def _validate_input(input, model): + invalid = 0 if input:
@@ -132,7 +147,7 @@ def _validate_input(input): invalid = 1 # Check we are looking for a valid field - valid_fields = Build._meta.get_all_field_names() + valid_fields = model._meta.get_all_field_names() for field in input_list[0].split(','): if field not in valid_fields: invalid = 1
Signature · 1 line
1.8.1.2
PATCH 40/94 bitbake: webhob: enhance Simple browser navigation
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We make a couple of changes to allow easier navigation in the simple data display.
- all templates now inherit the common base.html
- simple navigation menu added on top of page
- there is a link between a task name in tasks page, and the recipe entry
- all tabs converted to 4-spaces
--- bitbake/lib/webhob/bldviewer/templates/base.html | 16 +++ bitbake/lib/webhob/bldviewer/templates/build.html | 103 +++++++++---------- bitbake/lib/webhob/bldviewer/templates/layer.html | 44 ++++---- bitbake/lib/webhob/bldviewer/templates/recipe.html | 90 ++++++++-------- bitbake/lib/webhob/bldviewer/templates/task.html | 114 ++++++++++----------- bitbake/lib/webhob/bldviewer/urls.py | 2 +- 6 files changed, 180 insertions(+), 189 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/base.html
bitbake/lib/webhob/bldviewer/templates/base.html new file +16 -0
@@ -0,0 +1,16 @@ +<!DOCTYPE html> + +<html> + <head> + <title>WebHob Simple Explorer</title> + </head> + +<body> +<div>Menu: <a href="/simple/build/">All Builds</a> | <a href="/simple/layer/">All Layers</a> +</div> +{% block pagecontent %} + +{% endblock %} +</body> +</html> +
bitbake/lib/webhob/bldviewer/templates/build.html +49 -54
103 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/templates/layer.html +19 -25
60 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/templates/recipe.html +42 -48
90 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/templates/task.html +53 -61
125 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/urls.py +1 -1
@@ -5,6 +5,6 @@ urlpatterns = patterns('bldviewer.views', url(r'^build/$', 'build', name='build'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), url(r'^layer/$', 'layer', name='layer'), - url(r'^layerversions/(?P<layerversion_id>\d+)/recipes$', 'layer_versions_recipes', name='layer_versions_recipes'), + url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'), url(r'^$', redirect_to, {'url': 'build/'}), )
Signature · 1 line
1.8.1.2
PATCH 41/94 bitbake: webhob: improve startup script
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
A couple of modification targeted at making the
system start/stop script more resilient.
Also, it now checks for the correct Django version.
--- bitbake/bin/webhob | 53 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 11 deletions(-)
bitbake/bin/webhob +42 -11
@@ -22,13 +22,13 @@ # Helper function to kill a background webhob development server -function webserverKillAllComponents() +function webserverKillAll() { local pidfile for pidfile in ${BUILDDIR}/whbmain.pid; do if [ -f ${pidfile} ]; then while kill -0 $(< ${pidfile}) 2>/dev/null; do - kill -SIGTERM -$(< ${pidfile}) + kill -SIGTERM -$(< ${pidfile}) 2>/dev/null sleep 1; done; rm ${pidfile}
@@ -36,6 +36,15 @@ function webserverKillAllComponents() done } + +function webserverStartAll() +{ + python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 2) + python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid + return 0 +} + + # We make sure we're running in the current shell and in a good environment if [ -z "$ZSH_NAME" ] && [ `basename "$0"` = `basename $BASH_SOURCE` ]; then
@@ -50,6 +59,16 @@ fi BBBASEDIR=`dirname ${BASH_SOURCE}`/.. + +# Verify prerequisites + +if ! echo "import django; print (1,4,5) == django.VERSION[0:3]" | python 2>/dev/null | grep True >/dev/null; then + echo -e "This program needs Django 1.4.5. Please install with\n\nsudo pip install django==1.4.5" + return 2 +fi + + + # Determine the action. If specified by arguments, fine, if not, toggle it if [ "x$1" == "xstart" ] || [ "x$1" == "xstop" ]; then CMD="$1"
@@ -61,6 +80,13 @@ else fi; fi +NODSI=0 +if [ "x$2" == "xnodsi" ]; then + NODSI=1 +fi + +echo "The system will $CMD." + # Make sure it's safe to run by checking bitbake lock lock=1
@@ -69,11 +95,11 @@ if [ -e $BUILDDIR/bitbake.lock ]; then fi if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); then - echo "Error: bitbake lock state error. System may be already on." 2>&1 + echo "Error: bitbake lock state error. System is already on." 2>&1 return 3 elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then - echo "Error: bitbake lock state error. System may be already off. -manually stop system with bitbake -m / webserverKillAllComponents" 2>&1 + echo "Error: bitbake lock state error. Trying to stop a stopped system ? +manually stop system with bitbake -m / webserverKill" 2>&1 return 3 fi
@@ -82,19 +108,24 @@ fi case $CMD in start ) + webserverStartAll || return 4 unset BBSERVER - python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 0) bitbake --server-only -t xmlrpc -B localhost:8200 export BBSERVER=localhost:8200 - bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid - python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid + if [ $NODSI == 0 ]; then # we start the DSI only if not inhibited + bitbake --observe-only -u dsi >/dev/null 2>&1 & echo $! >${BUILDDIR}/dsi.pid + fi ;; stop ) - kill $(< ${BUILDDIR}/dsi.pid ) - rm ${BUILDDIR}/dsi.pid + if [ -f ${BUILDDIR}/dsi.pid ]; then + kill $(< ${BUILDDIR}/dsi.pid ) + rm ${BUILDDIR}/dsi.pid + fi bitbake -m unset BBSERVER - webserverKillAllComponents + webserverKillAll + ;; esac +echo "Successful ${CMD}."
Signature · 1 line
1.8.1.2
PATCH 42/94 bitbake: webhob: improve validation code flow
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch improves the code flow in the validation method by not repeating the split method over the same list and by stoping at any invalid input encountered.
--- bitbake/lib/webhob/bldviewer/views.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
bitbake/lib/webhob/bldviewer/views.py +4 -1
@@ -139,17 +139,20 @@ def _validate_input(input, model): input_list = input.split(":") # Check we have only one colon - if len(input.split(":")) != 2: + if len(input_list) != 2: invalid = 1 + return None, invalid # Check we have an equal number of terms both sides of the colon if len(input_list[0].split(',')) != len(input_list[1].split(',')): invalid = 1 + return None, invalid # Check we are looking for a valid field valid_fields = model._meta.get_all_field_names() for field in input_list[0].split(','): if field not in valid_fields: invalid = 1 + return None, invalid return input, invalid
Signature · 1 line
1.8.1.2
PATCH 43/94 bitbake: webhob: add more models to webhob API
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch enables webhob API to support recipe, layer and layer_version models.
--- bitbake/lib/webhob/bldviewer/api.py | 3 +++ bitbake/lib/webhob/bldviewer/views.py | 7 +++++-- 2 files changed, 8 insertions(+), 2 deletions(-)
bitbake/lib/webhob/bldviewer/api.py +3 -0
@@ -5,4 +5,7 @@ urlpatterns = patterns('bldviewer.views', url(r'^builds/$', 'model_explorer', {'model_name':'build'}, name='builds'), url(r'^tasks/$', 'model_explorer', {'model_name':'task'}, name='task'), url(r'^packages/$', 'model_explorer', {'model_name':'package'}, name='package'), + url(r'^layers/$', 'model_explorer', {'model_name':'layer'}, name='layer'), + url(r'^recipes/$', 'model_explorer', {'model_name':'recipe'}, name='recipe'), + url(r'^layersversions/$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'), )
bitbake/lib/webhob/bldviewer/views.py +5 -2
@@ -43,9 +43,9 @@ def layer(request): def layer_versions_recipes(request, layerversion_id): template = 'recipe.html' recipes = Recipe.objects.filter(layer_version__id = layerversion_id) - + context = {'recipes': recipes} - + return render(request, template, context) #### API
@@ -63,6 +63,9 @@ def model_explorer(request, model_name): 'build': Build, 'task': Task, 'package': Package, + 'layer': Layer, + 'layerversion': Layer_Version, + 'recipe': Recipe, } if model_name not in model_mapping.keys():
Signature · 1 line
1.8.1.2
PATCH 44/94 bitbake: webhob: add search for build model
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch enables search on the build model. However, the search can be executed only on a predefined list of fields allowed to be searched.
--- bitbake/lib/webhob/bldviewer/views.py | 17 +++++++++++++++++ bitbake/lib/webhob/orm/models.py | 3 +++ 2 files changed, 20 insertions(+)
bitbake/lib/webhob/bldviewer/views.py +17 -0
@@ -1,3 +1,6 @@ +import operator + +from django.db.models import Q from django.shortcuts import render from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package
@@ -93,6 +96,8 @@ def model_explorer(request, model_name): if invalid: return HttpResponseBadRequest() + search_term = request.GET.get('search', '') + if filter_string: filter_terms = _get_filtering_terms(filter_string) try:
@@ -102,6 +107,9 @@ def model_explorer(request, model_name): else: queryset = model.objects.all() + if search_term: + queryset = _get_search_results(search_term, queryset, model) + if offset and limit: queryset = queryset[offset:(offset+limit)] elif offset:
@@ -159,3 +167,12 @@ def _validate_input(input, model): return None, invalid return input, invalid + +def _get_search_results(search_term, queryset, model): + q_map = map(lambda x: Q(**{x+'__icontains': search_term}), + model.search_allowed_fields) + + search_object = reduce(operator.or_, q_map) + queryset = queryset.filter(search_object) + + return queryset
bitbake/lib/webhob/orm/models.py +3 -0
@@ -12,6 +12,9 @@ class Build(models.Model): (IN_PROGRESS, 'In Progress'), ) + search_allowed_fields = ['target', 'machine__name', + 'cooker_log_path', 'image_fstypes'] + uuid = models.CharField(max_length=100, unique=True) target = models.CharField(max_length=100) is_image = models.BooleanField(default = False)
Signature · 1 line
1.8.1.2
PATCH 45/94 bitbake: webhob: fix ordering issue
In reply to Alex DAMIANFrom: Calin Dragomir <[email protected]>
This patch fixes the AssertionError issue that caused the build page to break.
This happened because ordering cannot be done after a slicing operation has been executed.
By placing the ordering before the slicing occurs, the issue stops appearing.
Conflicts:
bitbake/lib/webhob/bldviewer/views.py
--- bitbake/lib/webhob/bldviewer/views.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
bitbake/lib/webhob/bldviewer/views.py +7 -7
@@ -110,6 +110,13 @@ def model_explorer(request, model_name): if search_term: queryset = _get_search_results(search_term, queryset, model) + if ordering_string and queryset: + column, order = ordering_string.split(':') + if order.lower() == DESCENDING: + queryset = queryset.order_by('-' + column) + else: + queryset = queryset.order_by(column) + if offset and limit: queryset = queryset[offset:(offset+limit)] elif offset:
@@ -122,13 +129,6 @@ def model_explorer(request, model_name): else: response_data['count'] = 0 - if queryset and ordering_string: - column, order = ordering_string.split(':') - if order.lower() == DESCENDING: - queryset = queryset.order_by('-' + column) - else: - queryset = queryset.order_by(column) - response_data['list'] = serializers.serialize('json', queryset) return HttpResponse(json.dumps(response_data),
Signature · 1 line
1.8.1.2
PATCH 46/94 bitbake: webhob: extend search for multiple terms
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We extend the search functionality as to be able to narrow down search results based on multiple terms.
--- bitbake/lib/webhob/bldviewer/views.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
bitbake/lib/webhob/bldviewer/views.py +5 -2
@@ -169,10 +169,13 @@ def _validate_input(input, model): return input, invalid def _get_search_results(search_term, queryset, model): - q_map = map(lambda x: Q(**{x+'__icontains': search_term}), + search_objects = [] + for st in search_term.split(" "): + q_map = map(lambda x: Q(**{x+'__icontains': st}), model.search_allowed_fields) - search_object = reduce(operator.or_, q_map) + search_objects.append(reduce(operator.or_, q_map)) + search_object = reduce(operator.and_, search_objects) queryset = queryset.filter(search_object) return queryset
Signature · 1 line
1.8.1.2
PATCH 47/94 bitbake: webhob: force bitbake server stop
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We force a bitbake server stop by killing it on the stop command.
This will have no effect on well behaving servers since they will be stopped anyway by the bitbake -m command above
--- bitbake/bin/webhob | 2 ++ 1 file changed, 2 insertions(+)
bitbake/bin/webhob +2 -0
@@ -124,6 +124,8 @@ case $CMD in bitbake -m unset BBSERVER webserverKillAll + # force stop any misbehaving bitbake server + lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 kill ;; esac
Signature · 1 line
1.8.1.2
PATCH 48/94 bitbake: webhob: simple interface dependency list
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We add a diff to obtain a consistent layout of the dependency field information for tasks.
--- bitbake/lib/webhob/bldviewer/templates/task.html | 2 ++ 1 file changed, 2 insertions(+)
bitbake/lib/webhob/bldviewer/templates/task.html +2 -0
@@ -48,10 +48,12 @@ <td>{{task.get_script_type_display}}</td> <td>{{task.recipe.file_path}}</td> <td> + <div style="height: 3em; overflow:auto"> {% for tt in task.depends_on %} <a href="#{{tt.recipe.name}}.{{tt.task_name}}"> {{tt.recipe.name}}.{{tt.task_name}}</a><br/> {% endfor %} + </div> </td> </tr>
Signature · 1 line
1.8.1.2
PATCH 49/94 bitbake: dsi: add feature to store package information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
I'm adding code to read package information and package dependency off the buildhistory, if the buildhistory feature is enabled, and the target is an image.
There are small changes to the events trapped since we need to capture package information after the build is completed.
Minor changes to internal state handling of BuildInfoHelper.
--- bitbake/lib/bb/ui/buildinfohelper.py | 75 ++++++++++++++++++++++++++++++++---- bitbake/lib/bb/ui/dsi.py | 30 +++++++-------- bitbake/lib/webhob/orm/models.py | 4 +- 3 files changed, 85 insertions(+), 24 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +68 -7
@@ -8,7 +8,7 @@ import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package +from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, Package_Dependency from webhob.orm.models import Task_Dependency
@@ -42,16 +42,14 @@ class ORMWrapper(object): build_name=build_info['build_name'], bitbake_version=build_info['bitbake_version']) - return build - def update_build_object(self, build_obj, errors, warnings, taskfailures): + def update_build_object(self, build, errors, warnings, taskfailures): outcome = Build.SUCCEEDED if errors or taskfailures: outcome = Build.FAILED - build = Build.objects.get(uuid=build_obj.uuid) build.completed_on = datetime.datetime.now() build.errors_no = errors build.warnings_no = warnings
@@ -116,6 +114,22 @@ class ORMWrapper(object): return layer_object[0] + def save_package_information(self, build_obj, packagedict, bldpkgs, recipes): + for p in packagedict: + packagedict[p]['object'] = Package.objects.create( build = build_obj, + name = p, + size = packagedict[p]['size']) + if p in bldpkgs: + packagedict[p]['object'].version = bldpkgs[p]['version'] + packagedict[p]['object'].recipe = recipes[bldpkgs[p]['pn']] + packagedict[p]['object'].save() + + for p in packagedict: + for px in packagedict[p]['depends']: + Package_Dependency.objects.create( package = packagedict[p]['object'], + depends_on = packagedict[px]['object'] ); + + class BuildInfoHelper(object): """ This class gathers the build information from the server and sends it towards the ORM wrapper for storing in the database
@@ -123,13 +137,14 @@ class BuildInfoHelper(object): Keeps in memory all data that needs matching before writing it to the database """ - def __init__(self, server): + def __init__(self, server, has_build_history = False): self._configure_django() self.internal_state = {} self.uuid = None self.task_order = 0 self.server = server self.orm_wrapper = ORMWrapper() + self.has_build_history = has_build_history def _configure_django(self): # Add webhob to sys path for importing modules
@@ -343,7 +358,6 @@ class BuildInfoHelper(object): def update_build_information(self, event, errors, warnings, taskfailures): self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures) - del self.internal_state['build'] def store_started_task(self, event): identifier = re.split(':', event.taskfile)[-1] + event.taskname
@@ -398,8 +412,56 @@ class BuildInfoHelper(object): self.orm_wrapper.get_update_task_object(task_information) + def read_package_dep_data(self, event): + # verify that we have something to read + if not self.internal_state['build'].is_image or not self.has_build_history: + print "not collecting package info ", self.internal_state['build'].is_image, self.has_build_history + return + + # fair warning: code correlates to buildhistory.bbclass; anything changes there, needs to chage here too + TOPDIR, error = self.server.runCommand(['getVariable', 'TOPDIR']) + MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH']) + TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC']) + MULTIMACH_TARGET_SYS, error = self.server.runCommand(['getVariable', 'MULTIMACH_TARGET_SYS']) + SDK_NAME, error = self.server.runCommand(['getVariable', 'SDK_NAME']) + BUILDHISTORY_DIR = "%s/buildhistory" % TOPDIR + BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, self.internal_state['build'].target) + + self.internal_state['packages'] = {} + + with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin: + for line in fin: + line = line.rstrip(";") + psize, px = line.split("\t") + punit, pname = px.split(" ") + self.internal_state['packages'][pname.strip()] = {'size':psize, 'depends' : []} + + with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin: + p = re.compile(r' -> ') + for line in fin: + line = line.rstrip(';') + linesplit = p.split(line) + if len(linesplit) == 2: + pname = linesplit[0] + dependsname = linesplit[1].split(" ")[0].strip().strip(";") + if not pname in self.internal_state['packages']: + self.internal_state['packages'][pname] = {'size': 0, 'depends' : []} + if not dependsname in self.internal_state['packages']: + self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []} + self.internal_state['packages'][pname]['depends'].append(dependsname) + + self.orm_wrapper.save_package_information(self.internal_state['build'], self.internal_state['packages'], + self.internal_state['bldpkgs'], self.internal_state['recipes']) + def store_dependency_information(self, event): + + # save build time package information + self.internal_state['bldpkgs'] = {} + for pkg in event._depgraph['packages']: + self.internal_state['bldpkgs'][pkg] = event._depgraph['packages'][pkg] + + # save recipe information self.internal_state['recipes'] = {} for pn in event._depgraph['pn']:
@@ -447,4 +509,3 @@ class BuildInfoHelper(object): dep = _save_a_task(taskdesc1) Task_Dependency.objects.get_or_create( task = target, depends_on = dep ) - del self.internal_state['recipes']
bitbake/lib/bb/ui/dsi.py +15 -15
@@ -243,6 +243,14 @@ def main(server, eventHandler, params, tf = TerminalFilter): includelogs, loglines, consolelogfile = _log_settings_from_server(server) + + # verify and warn + build_history_enabled = True + inheritlist, error = server.runCommand(["getVariable", "INHERIT"]) + if not "buildhistory" in inheritlist.split(" "): + logger.warn("buildhistory is not enabled. Please enable INHERIT += \"buildhistory\" to see image details.") + build_history_enabled = False + if sys.stdin.isatty() and sys.stdout.isatty(): log_exec_tty = True else:
@@ -275,7 +283,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): warnings = 0 taskfailures = [] - buildinfohelper = BuildInfoHelper(server) + buildinfohelper = BuildInfoHelper(server, build_history_enabled) buildinfohelper.store_layer_info() termfilter = tf(main, helper, console, format)
@@ -453,28 +461,20 @@ def main(server, eventHandler, params, tf = TerminalFilter): # timestamp should be added for this continue - if isinstance(event, bb.event.OperationStarted): - # timestamp should be added for this + if isinstance(event, (bb.event.TreeDataPreparationStarted, bb.event.TreeDataPreparationCompleted)): continue - if isinstance(event, bb.event.OperationCompleted): - # timestamp should be added for this - # signal a complete operation - # calculate timing - continue - - if isinstance(event, bb.event.DiskFull): - # trigger an error + if isinstance(event, (bb.event.BuildCompleted)): + buildinfohelper.read_package_dep_data(event) + buildinfohelper.update_build_information(event, errors, warnings, taskfailures) continue - if isinstance(event, (bb.event.BuildCompleted, - bb.command.CommandCompleted, + if isinstance(event, (bb.command.CommandCompleted, bb.command.CommandFailed, bb.command.CommandExit)): - buildinfohelper.update_build_information(event, errors, warnings, taskfailures) # we start a new build info - buildinfohelper = BuildInfoHelper(server) + buildinfohelper = BuildInfoHelper(server, build_history_enabled) buildinfohelper.store_layer_info() continue
bitbake/lib/webhob/orm/models.py +2 -2
@@ -110,9 +110,9 @@ class Artifact(models.Model): class Package(models.Model): build = models.ForeignKey('Build', related_name='package_build') - recipe = models.ForeignKey('Recipe', related_name='package_recipe') + recipe = models.ForeignKey('Recipe', related_name='package_recipe', null=True) name = models.CharField(max_length=100) - version = models.CharField(max_length=100) + version = models.CharField(max_length=100, default="") size = models.IntegerField()
Signature · 1 line
1.8.1.2
PATCH 50/94 bitbake: webhob: add simple viewer for package information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
If the target built is an image, we add a link to a page which lists all the packages built for this build, with sizes, version names and recipes linked in. The package dependency information is also shown.
--- bitbake/lib/webhob/bldviewer/templates/build.html | 2 +- .../lib/webhob/bldviewer/templates/package.html | 42 ++++++++++++++++++++++ bitbake/lib/webhob/bldviewer/urls.py | 1 + bitbake/lib/webhob/bldviewer/views.py | 18 +++++++++- 4 files changed, 61 insertions(+), 2 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/package.html
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
@@ -30,7 +30,7 @@ <td>{{build.started_on}}</td> <td>{{build.completed_on}}</td> <td>{{build.target}}</td> - <td>{{build.is_image}}</td> + <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td> <td>{{build.machine.name}}</td> <td>{% time_difference build.started_on build.completed_on %}</td> <td>{{build.errors_no}}</td>
bitbake/lib/webhob/bldviewer/templates/package.html new file +42 -0
42 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/urls.py +1 -0
@@ -4,6 +4,7 @@ from django.views.generic.simple import redirect_to urlpatterns = patterns('bldviewer.views', url(r'^build/$', 'build', name='build'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), + url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'), url(r'^layer/$', 'layer', name='layer'), url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'), url(r'^$', redirect_to, {'url': 'build/'}),
bitbake/lib/webhob/bldviewer/views.py +17 -1
@@ -2,7 +2,7 @@ import operator from django.db.models import Q from django.shortcuts import render -from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package +from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package, Package_Dependency def build(request):
@@ -29,6 +29,22 @@ def task(request, build_id): return render(request, template, context) +def package(request, build_id): + template = 'package.html' + + packages = Package.objects.filter(build=build_id) + package_depends = Package_Dependency.objects.filter(package__in=packages) + + for t in packages: + t.depends_on = [] + for k in package_depends: + if t == k.package: + t.depends_on.append(k.depends_on) + + context = {'packages': packages} + + return render(request, template, context) + def layer(request): template = 'layer.html' layer_info = Layer.objects.all()
Signature · 1 line
1.8.1.2
PATCH 51/94 bitbake: dsi: fix build stats data gathering
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We set the default values for build stats to be NULL so that we can distinguish the value from a legitimate 0.
Also fixing the data gathering since model field semantics changed - the version is kept on a separate field.
[YOCTO #4967]
--- bitbake/lib/bb/ui/buildinfohelper.py | 2 +- bitbake/lib/webhob/orm/models.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +1 -1
@@ -286,7 +286,7 @@ class BuildInfoHelper(object): target = self.internal_state['target'] machine = self.internal_state['build'].machine.name buildname = self.internal_state['build'].build_name - package = task_object.recipe.name + package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":") build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target, machine=machine, buildname=buildname,
bitbake/lib/webhob/orm/models.py +2 -2
@@ -84,8 +84,8 @@ class Task(models.Model): script_type = models.IntegerField(choices=TASK_CODING, default=CODING_PYTHON) line_number = models.IntegerField(default=0) py_stack_trace = models.TextField(null=True) - disk_io = models.IntegerField(default=0) - cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, default=0) + disk_io = models.IntegerField(null=True) + cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, null=True) elapsed_time = models.CharField(max_length=50, default=0) errors_no = models.IntegerField(default=0) warnings_no = models.IntegerField(default=0)
Signature · 1 line
1.8.1.2
PATCH 52/94 bitbake: dsi: update build object on command end
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We need to make the final update to the build object on command completion, regardless of the status of the completion, in order to avoid leaving builds in the "In Progress" state.
--- bitbake/lib/bb/ui/dsi.py | 1 + 1 file changed, 1 insertion(+)
bitbake/lib/bb/ui/dsi.py +1 -0
@@ -474,6 +474,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): bb.command.CommandExit)): # we start a new build info + buildinfohelper.update_build_information(event, errors, warnings, taskfailures) buildinfohelper = BuildInfoHelper(server, build_history_enabled) buildinfohelper.store_layer_info() continue
Signature · 1 line
1.8.1.2
PATCH 53/94 bitbake: dsi: fix sstate task information gathering
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Fixing the information gathering for sstate-related tasks - now the start and completion events are correctly handled, and we get an update for tasks that were covered by sstate tasks.
--- bitbake/lib/bb/ui/buildinfohelper.py | 6 +++++- bitbake/lib/bb/ui/dsi.py | 5 ++--- 2 files changed, 7 insertions(+), 4 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +5 -1
@@ -67,7 +67,11 @@ class ORMWrapper(object): for v in vars(task_object): if v in task_information.keys(): vars(task_object)[v] = task_information[v] + # if we got covered by a setscene task, we're SSTATE + if task_object.outcome == Task.OUTCOME_COVERED and 1 == Task.objects.filter(task_executed=True, build = task_object.build, recipe = task_object.recipe, task_name=task_object.task_name+"_setscene").count(): + task_object.outcome = Task.OUTCOME_SSTATE + # mark down duration if we have a start time if 'start_time' in task_information.keys(): duration = datetime.datetime.now() - task_information['start_time'] task_object.elapsed_time = duration.total_seconds()
@@ -393,7 +397,7 @@ class BuildInfoHelper(object): except: pass - if isinstance(event, bb.runqueue.runQueueTaskCompleted): + if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)): task_information['outcome'] = Task.OUTCOME_SUCCESS # TODO: needs to use constants task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information)) task_information['cpu_usage'] = task_build_stats['cpu_usage']
bitbake/lib/bb/ui/dsi.py +2 -3
@@ -415,6 +415,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): continue if isinstance(event, bb.runqueue.sceneQueueTaskStarted): + buildinfohelper.store_started_task(event) logger.info("Running setscene task %d of %d (%s)" % (event.stats.completed + event.stats.active + event.stats.failed + 1, event.stats.total, event.taskstring)) continue
@@ -446,10 +447,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): event.taskid, event.taskstring, event.exitcode) continue - if isinstance(event, bb.runqueue.sceneQueueTaskFailed): + if isinstance(event, (bb.runqueue.sceneQueueTaskCompleted, bb.runqueue.sceneQueueTaskFailed)): buildinfohelper.update_and_store_task(event) - logger.warn("Setscene task %s (%s) failed with exit code '%s' - real task will be run instead", - event.taskid, event.taskstring, event.exitcode) continue
Signature · 1 line
1.8.1.2
PATCH 54/94 bitbake: webhob: clean up starting script
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Clean up of webhob start-up script as to make sure it aborts the startup on database errors, and that it doesn't execute superflous kill commands when stopping.
--- bitbake/bin/webhob | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
bitbake/bin/webhob +7 -3
@@ -39,9 +39,13 @@ function webserverKillAll() function webserverStartAll() { - python $BBBASEDIR/lib/webhob/manage.py syncdb || (echo "Failed db sync, stopping system start" 1>&2 && return 2) + retval=0 + python $BBBASEDIR/lib/webhob/manage.py syncdb || retval=1 + if [ $retval -eq 1 ]; then + echo "Failed db sync, stopping system start" 1>&2 + fi python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid - return 0 + return $retval }
@@ -125,7 +129,7 @@ case $CMD in unset BBSERVER webserverKillAll # force stop any misbehaving bitbake server - lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 kill + lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 -r kill ;; esac
Signature · 1 line
1.8.1.2
PATCH 55/94 bitbake: dsi: store log information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding database model for storing Log Information
Adding code to store log information for errors
and warnings coming through the build system.
--- bitbake/lib/bb/ui/buildinfohelper.py | 41 +++++++++++++++++++++++++++++++++--- bitbake/lib/bb/ui/dsi.py | 9 ++++++-- bitbake/lib/webhob/orm/models.py | 15 +++++++++++++ 3 files changed, 60 insertions(+), 5 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +38 -3
@@ -8,9 +8,9 @@ import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, Package_Dependency -from webhob.orm.models import Task_Dependency - +from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, LogMessage +from webhob.orm.models import Task_Dependency, Package_Dependency +from bb.msg import BBLogFormatter as format class ORMWrapper(object): """ This class creates the dictionaries needed to store information in the database
@@ -134,6 +134,18 @@ class ORMWrapper(object): depends_on = packagedict[px]['object'] ); + def create_logmessage(self, log_information): + log_object = LogMessage.objects.create( + build = log_information['build'], + level = log_information['level'], + message = log_information['message']) + + for v in vars(log_object): + if v in log_information.keys(): + vars(log_object)[v] = log_information[v] + + return log_object.save() + class BuildInfoHelper(object): """ This class gathers the build information from the server and sends it towards the ORM wrapper for storing in the database
@@ -513,3 +525,26 @@ class BuildInfoHelper(object): dep = _save_a_task(taskdesc1) Task_Dependency.objects.get_or_create( task = target, depends_on = dep ) + def store_log_information(self, level, text): + log_information = {} + log_information['build'] = self.internal_state['build'] + log_information['level'] = level + log_information['message'] = text + self.orm_wrapper.create_logmessage(log_information) + + def store_log_event(self, event): + if event.levelno < format.WARNING: + return + if not 'build' in self.internal_state: + return + log_information = {} + log_information['build'] = self.internal_state['build'] + if event.levelno >= format.ERROR: + log_information['level'] = LogMessage.ERROR + elif event.levelno == format.WARNING: + log_information['level'] = LogMessage.WARNING + log_information['message'] = event.msg + log_information['pathname'] = event.pathname + log_information['lineno'] = event.lineno + self.orm_wrapper.create_logmessage(log_information) +
bitbake/lib/bb/ui/dsi.py +7 -2
@@ -332,6 +332,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): # include verbose/debug messages if event.taskpid != 0 and event.levelno <= format.NOTE: continue + + buildinfohelper.store_log_event(event) logger.handle(event) continue
@@ -406,9 +408,12 @@ def main(server, eventHandler, params, tf = TerminalFilter): r = "" if event._dependees: - logger.error("Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)", r, event._item, ", ".join(event._dependees), r) + text = "Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)" % (r, event._item, ", ".join(event._dependees), r) else: - logger.error("Nothing %sPROVIDES '%s'", r, event._item) + text = "Nothing %sPROVIDES '%s'" % (r, event._item) + + logger.error(text) + buildinfohelper.store_log_information(2, text) # don't judge me if event._reasons: for reason in event._reasons: logger.error("%s", reason)
bitbake/lib/webhob/orm/models.py +15 -0
@@ -174,3 +174,18 @@ class Machine(models.Model): name = models.CharField(max_length=100) description = models.TextField() + +class LogMessage(models.Model): + INFO = 0 + WARNING = 1 + ERROR = 2 + + LOG_LEVEL = ( (INFO, "info"), + (WARNING, "warn"), + (ERROR, "error") ) + + build = models.ForeignKey(Build) + level = models.IntegerField(choices=LOG_LEVEL, default=INFO) + message=models.CharField(max_length=240) + pathname = models.FilePathField(max_length=255, null=True) + lineno = models.IntegerField(null=True)
Signature · 1 line
1.8.1.2
PATCH 56/94 bitbake: webhob: add simple visualisation for build errors
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding Simple web error viewing in the builds page. Now the error/warning column will contain the actual content of the error.
--- bitbake/lib/webhob/bldviewer/templates/build.html | 4 ++-- bitbake/lib/webhob/bldviewer/views.py | 6 ++++-- 2 files changed, 6 insertions(+), 4 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +2 -2
@@ -33,8 +33,8 @@ <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td> <td>{{build.machine.name}}</td> <td>{% time_difference build.started_on build.completed_on %}</td> - <td>{{build.errors_no}}</td> - <td>{{build.warnings_no}}</td> + <td>{% if build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> + <td>{% if build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> <td>{{build.image_fstypes}}</td> <td>{{build.cooker_log_path}}</td> <td>{{build.bitbake_version}}</td>
bitbake/lib/webhob/bldviewer/views.py +4 -2
@@ -2,14 +2,16 @@ import operator from django.db.models import Q from django.shortcuts import render -from orm.models import Build, Task, Layer, Layer_Version, Task_Dependency, Recipe, Package, Package_Dependency +from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage +from orm.models import Task_Dependency, Package_Dependency def build(request): template = 'build.html' build_info = Build.objects.all() + logs = LogMessage.objects.all() - context = {'builds': build_info} + context = {'builds': build_info, 'logs': logs } return render(request, template, context)
Signature · 1 line
1.8.1.2
PATCH 57/94 bitbake: webhob: store logfile and message for Tasks
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Since we get the reference to the log file, and a corresponding message with Task* events, we store that in the database.
Corresponding changes to the model file.
--- bitbake/lib/bb/ui/buildinfohelper.py | 6 ++++++ bitbake/lib/bb/ui/dsi.py | 4 +++- bitbake/lib/webhob/orm/models.py | 6 ++---- 3 files changed, 11 insertions(+), 5 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +6 -0
@@ -409,6 +409,12 @@ class BuildInfoHelper(object): except: pass + if 'logfile' in vars(event): + task_information['logfile'] = event.logfile + + if '_message' in vars(event): + task_information['message'] = event._message + if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)): task_information['outcome'] = Task.OUTCOME_SUCCESS # TODO: needs to use constants task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
bitbake/lib/bb/ui/dsi.py +3 -1
@@ -303,7 +303,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): if isinstance(event, bb.event.BuildStarted): buildinfohelper.store_started_build(event) - if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent, bb.build.TaskFailed)): + if isinstance(event, (bb.build.TaskSucceeded, bb.build.TaskFailedSilent)): buildinfohelper.update_and_store_task(event) continue
@@ -338,6 +338,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): continue if isinstance(event, bb.build.TaskFailed): + buildinfohelper.update_and_store_task(event) return_value = 1 logfile = event.logfile if logfile and os.path.exists(logfile):
@@ -362,6 +363,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): if lines: for line in lines: print(line) + if isinstance(event, bb.build.TaskBase): logger.info(event._message) continue
bitbake/lib/webhob/orm/models.py +2 -4
@@ -87,11 +87,9 @@ class Task(models.Model): disk_io = models.IntegerField(null=True) cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, null=True) elapsed_time = models.CharField(max_length=50, default=0) - errors_no = models.IntegerField(default=0) - warnings_no = models.IntegerField(default=0) - error = models.TextField(null=True) - warning = models.TextField(null=True) sstate_result = models.IntegerField(choices=SSTATE_RESULT, default=SSTATE_NA) + message = models.CharField(max_length=240) + logfile = models.FilePathField(max_length=255, null=True) class Meta: ordering = ('order', 'recipe' ,)
Signature · 1 line
1.8.1.2
PATCH 58/94 bitbake: webhob: display proper information in Simple
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We modify the Simple interface to match the changes in the Task file.
Also a couple of display-related changes in the Simple interface.
--- bitbake/lib/webhob/bldviewer/templates/build.html | 4 ++-- bitbake/lib/webhob/bldviewer/templates/task.html | 10 +++++----- bitbake/lib/webhob/bldviewer/views.py | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +2 -2
@@ -33,8 +33,8 @@ <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td> <td>{{build.machine.name}}</td> <td>{% time_difference build.started_on build.completed_on %}</td> - <td>{% if build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> - <td>{% if build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> + <td>{{build.errors_no}}:{% if build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> + <td>{{build.warnings_no}}:{% if build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> <td>{{build.image_fstypes}}</td> <td>{{build.cooker_log_path}}</td> <td>{{build.bitbake_version}}</td>
bitbake/lib/webhob/bldviewer/templates/task.html +5 -5
@@ -15,8 +15,8 @@ <th>Recipe Version</th> <th>Task Type</th> <th>Outcome</th> - <th>Errors</th> - <th>Warnings</th> + <th>Message</th> + <th>Logfile</th> <th>Time</th> <th>CPU usage</th> <th>Disk I/O</th>
@@ -40,13 +40,13 @@ {% endif %} <td>{{task.get_outcome_display}}</td> - <td>{{task.errors_no}}</td> - <td>{{task.warnings_no}}</td> + <td><p>{{task.message}}</td> + <td><a target="_fileview" href="file:///{{task.logfile}}">{{task.logfile}}</a></td> <td>{{task.elapsed_time}}</td> <td>{{task.cpu_usage}}</td> <td>{{task.disk_io}}</td> <td>{{task.get_script_type_display}}</td> - <td>{{task.recipe.file_path}}</td> + <td><a target="_fileview" href="file:///{{task.recipe.file_path}}">{{task.recipe.file_path}}</a></td> <td> <div style="height: 3em; overflow:auto"> {% for tt in task.depends_on %}
bitbake/lib/webhob/bldviewer/views.py +1 -1
@@ -19,7 +19,7 @@ def task(request, build_id): template = 'task.html' tasks = Task.objects.filter(build=build_id) - task_depends = Task_Dependency.objects.filter(task__in=tasks) + task_depends = Task_Dependency.objects.filter(task__in=tasks).select_related() for t in tasks: t.depends_on = []
Signature · 1 line
1.8.1.2
PATCH 59/94 bitbake: dsi: clear up global variables between builds
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We're making sure that global variables that track a build data get reset when the build ends.
--- bitbake/lib/bb/ui/dsi.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/dsi.py +5 -1
@@ -479,8 +479,12 @@ def main(server, eventHandler, params, tf = TerminalFilter): bb.command.CommandFailed, bb.command.CommandExit)): - # we start a new build info buildinfohelper.update_build_information(event, errors, warnings, taskfailures) + + # we start a new build info + errors = 0 + warnings = 0 + taskfailures = [] buildinfohelper = BuildInfoHelper(server, build_history_enabled) buildinfohelper.store_layer_info() continue
Signature · 1 line
1.8.1.2
PATCH 60/94 bitbake: dsi: small bugfixes in data collection
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
A couple of bugfixes in the data collection:
- cache the TMPDIR since it's not gonna change during the build
- CommandCompleted can be issues outside of a build, so update the build only if we're running one
--- bitbake/lib/bb/ui/buildinfohelper.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +4 -3
@@ -161,6 +161,7 @@ class BuildInfoHelper(object): self.server = server self.orm_wrapper = ORMWrapper() self.has_build_history = has_build_history + self.tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0] def _configure_django(self): # Add webhob to sys path for importing modules
@@ -298,13 +299,12 @@ class BuildInfoHelper(object): def _get_path_information(self, task_object): build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/" - tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0] target = self.internal_state['target'] machine = self.internal_state['build'].machine.name buildname = self.internal_state['build'].build_name package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":") - build_stats_path = build_stats_format.format(tmpdir=tmp_dir, target=target, + build_stats_path = build_stats_format.format(tmpdir=self.tmp_dir, target=target, machine=machine, buildname=buildname, package=package)
@@ -373,7 +373,8 @@ class BuildInfoHelper(object): def update_build_information(self, event, errors, warnings, taskfailures): - self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures) + if 'build' in self.internal_state: + self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures) def store_started_task(self, event): identifier = re.split(':', event.taskfile)[-1] + event.taskname
Signature · 1 line
1.8.1.2
PATCH 61/94 bitbake: bldviewer: add jquery and bootstrap frameworks
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We are adding the jQuery and Bootstrap libraries, so we can add some advanced features to the Simple web interface.
jQuery license is MIT.
Bootstrap license is Apache v2.0
--- .../lib/webhob/bldviewer/static/css/bootstrap.css | 4797 +++++++++++ .../lib/webhob/bldviewer/static/js/bootstrap.js | 1982 +++++ .../lib/webhob/bldviewer/static/js/jquery-2.0.3.js | 8829 ++++++++++++++++++++ 3 files changed, 15608 insertions(+) create mode 100644 bitbake/lib/webhob/bldviewer/static/css/bootstrap.css create mode 100644 bitbake/lib/webhob/bldviewer/static/js/bootstrap.js create mode 100644 bitbake/lib/webhob/bldviewer/static/js/jquery-2.0.3.js
bitbake/lib/webhob/bldviewer/static/css/bootstrap.css new file +4797 -0
@@ -0,0 +1,4797 @@ +/*! + * Bootstrap v3.0.0 + * + * Copyright 2013 Twitter, Inc + * Licensed under the Apache License v2.0 + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Designed and built with all the love in the world by @mdo and @fat. + */ + +/*! normalize.css v2.1.0 | MIT License | git.io/normalize */ + +article, +aside, +details, +figcaption, +figure, +footer, +header, +hgroup, +main, +nav, +section, +summary { + display: block; +} + +audio, +canvas, +video { + display: inline-block; +} + +audio:not([controls]) { + display: none; + height: 0; +} + +[hidden] { + display: none; +} + +html { + font-family: sans-serif; + -webkit-text-size-adjust: 100%; + -ms-text-size-adjust: 100%; +} + +body { + margin: 0; +} + +a:focus { + outline: thin dotted; +} + +a:active, +a:hover { + outline: 0; +} + +h1 { + margin: 0.67em 0; + font-size: 2em; +} + +abbr[title] { + border-bottom: 1px dotted; +} + +b, +strong { + font-weight: bold; +} + +dfn { + font-style: italic; +} + +hr { + height: 0; + -moz-box-sizing: content-box; + box-sizing: content-box; +} + +mark { + color: #000; + background: #ff0; +} + +code, +kbd, +pre, +samp { + font-family: monospace, serif; + font-size: 1em; +} + +pre { + white-space: pre-wrap; +} + +q { + quotes: "\201C" "\201D" "\2018" "\2019"; +} + +small { + font-size: 80%; +} + +sub, +sup { + position: relative; + font-size: 75%; + line-height: 0; + vertical-align: baseline; +} + +sup { + top: -0.5em; +} + +sub { + bottom: -0.25em; +} + +img { + border: 0; +} + +svg:not(:root) { + overflow: hidden; +} + +figure { + margin: 0; +} + +fieldset { + padding: 0.35em 0.625em 0.75em; + margin: 0 2px; + border: 1px solid #c0c0c0; +} + +legend { + padding: 0; + border: 0; +} + +button, +input, +select, +textarea { + margin: 0; + font-family: inherit; + font-size: 100%; +} + +button, +input { + line-height: normal; +} + +button, +select { + text-transform: none; +} + +button, +html input[type="button"], +input[type="reset"], +input[type="submit"] { + cursor: pointer; + -webkit-appearance: button; +} + +button[disabled], +html input[disabled] { + cursor: default; +} + +input[type="checkbox"], +input[type="radio"] { + padding: 0; + box-sizing: border-box; +} + +input[type="search"] { + -webkit-box-sizing: content-box; + -moz-box-sizing: content-box; + box-sizing: content-box; + -webkit-appearance: textfield; +} + +input[type="search"]::-webkit-search-cancel-button, +input[type="search"]::-webkit-search-decoration { + -webkit-appearance: none; +} + +button::-moz-focus-inner, +input::-moz-focus-inner { + padding: 0; + border: 0; +} + +textarea { + overflow: auto; + vertical-align: top; +} + +table { + border-collapse: collapse; + border-spacing: 0; +} + +@media print { + * { + color: #000 !important; + text-shadow: none !important; + background: transparent !important; + box-shadow: none !important; + } + a, + a:visited { + text-decoration: underline; + } + a[href]:after { + content: " (" attr(href) ")"; + } + abbr[title]:after { + content: " (" attr(title) ")"; + } + .ir a:after, + a[href^="javascript:"]:after, + a[href^="#"]:after { + content: ""; + } + pre, + blockquote { + border: 1px solid #999; + page-break-inside: avoid; + } + thead { + display: table-header-group; + } + tr, + img { + page-break-inside: avoid; + } + img { + max-width: 100% !important; + } + @page { + margin: 2cm .5cm; + } + p, + h2, + h3 { + orphans: 3; + widows: 3; + } + h2, + h3 { + page-break-after: avoid; + } + .navbar { + display: none; + } + .table td, + .table th { + background-color: #fff !important; + } + .btn > .caret, + .dropup > .btn > .caret { + border-top-color: #000 !important; + } + .label { + border: 1px solid #000; + } + .table { + border-collapse: collapse !important; + } + .table-bordered th, + .table-bordered td { + border: 1px solid #ddd !important; + } +} + +* { + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; +} + +html { + font-size: 62.5%; + -webkit-tap-highlight-color: rgba(0, 0, 0, 0); +} + +body { + font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; + font-size: 14px; + line-height: 1.428571429; + color: #333333; + background-color: #ffffff; +} + +input, +button, +select, +textarea { + font-family: inherit; + font-size: inherit; + line-height: inherit; +} + +a { + color: #428bca; + text-decoration: none; +} + +a:hover, +a:focus { + color: #2a6496; + text-decoration: underline; +} + +a:focus { + outline: thin dotted #333; + outline: 5px auto -webkit-focus-ring-color; + outline-offset: -2px; +} + +img { + vertical-align: middle; +} + +.img-responsive { + display: inline-block; + height: auto; + max-width: 100%; +} + +.img-rounded { + border-radius: 6px; +} + +.img-circle { + border-radius: 500px; +} + +hr { + margin-top: 20px; + margin-bottom: 20px; + border: 0; + border-top: 1px solid #eeeeee; +} + +p { + margin: 0 0 10px; +} + +.lead { + margin-bottom: 20px; + font-size: 16.099999999999998px; + font-weight: 200; + line-height: 1.4; +} + +@media (min-width: 768px) { + .lead { + font-size: 21px; + } +} + +small { + font-size: 85%; +} + +cite { + font-style: normal; +} + +.text-muted { + color: #999999; +} + +.text-primary { + color: #428bca; +} + +.text-warning { + color: #c09853; +} + +.text-danger { + color: #b94a48; +} + +.text-success { + color: #468847; +} + +.text-info { + color: #3a87ad; +} + +.text-left { + text-align: left; +} + +.text-right { + text-align: right; +} + +.text-center { + text-align: center; +} + +h1, +h2, +h3, +h4, +h5, +h6, +.h1, +.h2, +.h3, +.h4, +.h5, +.h6 { + font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; + font-weight: 500; + line-height: 1.1; +} + +h1 small, +h2 small, +h3 small, +h4 small, +h5 small, +h6 small, +.h1 small, +.h2 small, +.h3 small, +.h4 small, +.h5 small, +.h6 small { + font-weight: normal; + line-height: 1; + color: #999999; +} + +h1, +h2, +h3 { + margin-top: 20px; + margin-bottom: 10px; +} + +h4, +h5, +h6 { + margin-top: 10px; + margin-bottom: 10px; +} + +h1, +.h1 { + font-size: 38px; +} + +h2, +.h2 { + font-size: 32px; +} + +h3, +.h3 { + font-size: 24px; +} + +h4, +.h4 { + font-size: 18px; +} + +h5, +.h5 { + font-size: 14px; +} + +h6, +.h6 { + font-size: 12px; +} + +h1 small, +.h1 small { + font-size: 24px; +} + +h2 small, +.h2 small { + font-size: 18px; +} + +h3 small, +.h3 small, +h4 small, +.h4 small { + font-size: 14px; +} + +.page-header { + padding-bottom: 9px; + margin: 40px 0 20px; + border-bottom: 1px solid #eeeeee; +} + +ul, +ol { + margin-top: 0; + margin-bottom: 10px; +} + +ul ul, +ol ul, +ul ol, +ol ol { + margin-bottom: 0; +} + +.list-unstyled { + padding-left: 0; + list-style: none; +} + +.list-inline { + padding-left: 0; + list-style: none; +} + +.list-inline > li { + display: inline-block; + padding-right: 5px; + padding-left: 5px; +} + +dl { + margin-bottom: 20px; +} + +dt, +dd { + line-height: 1.428571429; +} + +dt { + font-weight: bold; +} + +dd { + margin-left: 0; +} + +.dl-horizontal dt { + float: left; + width: 160px; + overflow: hidden; + clear: left; + text-align: right; + text-overflow: ellipsis; + white-space: nowrap; +} + +.dl-horizontal dd { + margin-left: 180px; +} + +.dl-horizontal dd:before, +.dl-horizontal dd:after { + display: table; + content: " "; +} + +.dl-horizontal dd:after { + clear: both; +} + +.dl-horizontal dd:before, +.dl-horizontal dd:after { + display: table; + content: " "; +} + +.dl-horizontal dd:after { + clear: both; +} + +abbr[title], +abbr[data-original-title] { + cursor: help; + border-bottom: 1px dotted #999999; +} + +abbr.initialism { + font-size: 90%; + text-transform: uppercase; +} + +blockquote { + padding: 10px 20px; + margin: 0 0 20px; + border-left: 5px solid #eeeeee; +} + +blockquote p { + font-size: 17.5px; + font-weight: 300; + line-height: 1.25; +} + +blockquote p:last-child { + margin-bottom: 0; +} + +blockquote small { + display: block; + line-height: 1.428571429; + color: #999999; +} + +blockquote small:before { + content: '\2014 \00A0'; +} + +blockquote.pull-right { + float: right; + padding-right: 15px; + padding-left: 0; + border-right: 5px solid #eeeeee; + border-left: 0; +} + +blockquote.pull-right p, +blockquote.pull-right small { + text-align: right; +} + +blockquote.pull-right small:before { + content: ''; +} + +blockquote.pull-right small:after { + content: '\00A0 \2014'; +} + +q:before, +q:after, +blockquote:before, +blockquote:after { + content: ""; +} + +address { + display: block; + margin-bottom: 20px; + font-style: normal; + line-height: 1.428571429; +} + +code, +pre { + font-family: Monaco, Menlo, Consolas, "Courier New", monospace; +} + +code { + padding: 2px 4px; + font-size: 90%; + color: #c7254e; + white-space: nowrap; + background-color: #f9f2f4; + border-radius: 4px; +} + +pre { + display: block; + padding: 9.5px; + margin: 0 0 10px; + font-size: 13px; + line-height: 1.428571429; + color: #333333; + word-break: break-all; + word-wrap: break-word; + background-color: #f5f5f5; + border: 1px solid #cccccc; + border-radius: 4px; +} + +pre.prettyprint { + margin-bottom: 20px; +} + +pre code { + padding: 0; + color: inherit; + white-space: pre-wrap; + background-color: transparent; + border: 0; +} + +.pre-scrollable { + max-height: 340px; + overflow-y: scroll; +} + +.container { + margin-right: auto; + margin-left: auto; +} + +.container:before, +.container:after { + display: table; + content: " "; +} + +.container:after { + clear: both; +} + +.container:before, +.container:after { + display: table; + content: " "; +} + +.container:after { + clear: both; +} + +.row:before, +.row:after { + display: table; + content: " "; +} + +.row:after { + clear: both; +} + +.row:before, +.row:after { + display: table; + content: " "; +} + +.row:after { + clear: both; +} + +@media (min-width: 768px) { + .row { + margin-right: -15px; + margin-left: -15px; + } +} + +.row .row { + margin-right: -15px; + margin-left: -15px; +} + +.col-1, +.col-2, +.col-3, +.col-4, +.col-5, +.col-6, +.col-7, +.col-8, +.col-9, +.col-10, +.col-11, +.col-12, +.col-sm-1, +.col-sm-2, +.col-sm-3, +.col-sm-4, +.col-sm-5, +.col-sm-6, +.col-sm-7, +.col-sm-8, +.col-sm-9, +.col-sm-10, +.col-sm-11, +.col-sm-12, +.col-lg-1, +.col-lg-2, +.col-lg-3, +.col-lg-4, +.col-lg-5, +.col-lg-6, +.col-lg-7, +.col-lg-8, +.col-lg-9, +.col-lg-10, +.col-lg-11, +.col-lg-12 { + position: relative; + min-height: 1px; + padding-right: 15px; + padding-left: 15px; +} + +.col-1, +.col-2, +.col-3, +.col-4, +.col-5, +.col-6, +.col-7, +.col-8, +.col-9, +.col-10, +.col-11, +.col-12 { + float: left; +} + +.col-1 { + width: 8.333333333333332%; +} + +.col-2 { + width: 16.666666666666664%; +} + +.col-3 { + width: 25%; +} + +.col-4 { + width: 33.33333333333333%; +} + +.col-5 { + width: 41.66666666666667%; +} + +.col-6 { + width: 50%; +} + +.col-7 { + width: 58.333333333333336%; +} + +.col-8 { + width: 66.66666666666666%; +} + +.col-9 { + width: 75%; +} + +.col-10 { + width: 83.33333333333334%; +} + +.col-11 { + width: 91.66666666666666%; +} + +.col-12 { + width: 100%; +} + +@media (min-width: 768px) { + .container { + max-width: 728px; + } + .col-sm-1, + .col-sm-2, + .col-sm-3, + .col-sm-4, + .col-sm-5, + .col-sm-6, + .col-sm-7, + .col-sm-8, + .col-sm-9, + .col-sm-10, + .col-sm-11, + .col-sm-12 { + float: left; + } + .col-sm-1 { + width: 8.333333333333332%; + } + .col-sm-2 { + width: 16.666666666666664%; + } + .col-sm-3 { + width: 25%; + } + .col-sm-4 { + width: 33.33333333333333%; + } + .col-sm-5 { + width: 41.66666666666667%; + } + .col-sm-6 { + width: 50%; + } + .col-sm-7 { + width: 58.333333333333336%; + } + .col-sm-8 { + width: 66.66666666666666%; + } + .col-sm-9 { + width: 75%; + } + .col-sm-10 { + width: 83.33333333333334%; + } + .col-sm-11 { + width: 91.66666666666666%; + } + .col-sm-12 { + width: 100%; + } + .col-sm-push-1 { + left: 8.333333333333332%; + } + .col-sm-push-2 { + left: 16.666666666666664%; + } + .col-sm-push-3 { + left: 25%; + } + .col-sm-push-4 { + left: 33.33333333333333%; + } + .col-sm-push-5 { + left: 41.66666666666667%; + } + .col-sm-push-6 { + left: 50%; + } + .col-sm-push-7 { + left: 58.333333333333336%; + } + .col-sm-push-8 { + left: 66.66666666666666%; + } + .col-sm-push-9 { + left: 75%; + } + .col-sm-push-10 { + left: 83.33333333333334%; + } + .col-sm-push-11 { + left: 91.66666666666666%; + } + .col-sm-pull-1 { + right: 8.333333333333332%; + } + .col-sm-pull-2 { + right: 16.666666666666664%; + } + .col-sm-pull-3 { + right: 25%; + } + .col-sm-pull-4 { + right: 33.33333333333333%; + } + .col-sm-pull-5 { + right: 41.66666666666667%; + } + .col-sm-pull-6 { + right: 50%; + } + .col-sm-pull-7 { + right: 58.333333333333336%; + } + .col-sm-pull-8 { + right: 66.66666666666666%; + } + .col-sm-pull-9 { + right: 75%; + } + .col-sm-pull-10 { + right: 83.33333333333334%; + } + .col-sm-pull-11 { + right: 91.66666666666666%; + } + .col-sm-offset-1 { + margin-left: 8.333333333333332%; + } + .col-sm-offset-2 { + margin-left: 16.666666666666664%; + } + .col-sm-offset-3 { + margin-left: 25%; + } + .col-sm-offset-4 { + margin-left: 33.33333333333333%; + } + .col-sm-offset-5 { + margin-left: 41.66666666666667%; + } + .col-sm-offset-6 { + margin-left: 50%; + } + .col-sm-offset-7 { + margin-left: 58.333333333333336%; + } + .col-sm-offset-8 { + margin-left: 66.66666666666666%; + } + .col-sm-offset-9 { + margin-left: 75%; + } + .col-sm-offset-10 { + margin-left: 83.33333333333334%; + } + .col-sm-offset-11 { + margin-left: 91.66666666666666%; + } +} + +@media (min-width: 992px) { + .container { + max-width: 940px; + } + .col-lg-1, + .col-lg-2, + .col-lg-3, + .col-lg-4, + .col-lg-5, + .col-lg-6, + .col-lg-7, + .col-lg-8, + .col-lg-9, + .col-lg-10, + .col-lg-11, + .col-lg-12 { + float: left; + } + .col-lg-1 { + width: 8.333333333333332%; + } + .col-lg-2 { + width: 16.666666666666664%; + } + .col-lg-3 { + width: 25%; + } + .col-lg-4 { + width: 33.33333333333333%; + } + .col-lg-5 { + width: 41.66666666666667%; + } + .col-lg-6 { + width: 50%; + } + .col-lg-7 { + width: 58.333333333333336%; + } + .col-lg-8 { + width: 66.66666666666666%; + } + .col-lg-9 { + width: 75%; + } + .col-lg-10 { + width: 83.33333333333334%; + } + .col-lg-11 { + width: 91.66666666666666%; + } + .col-lg-12 { + width: 100%; + } + .col-lg-push-1 { + left: 8.333333333333332%; + } + .col-lg-push-2 { + left: 16.666666666666664%; + } + .col-lg-push-3 { + left: 25%; + } + .col-lg-push-4 { + left: 33.33333333333333%; + } + .col-lg-push-5 { + left: 41.66666666666667%; + } + .col-lg-push-6 { + left: 50%; + } + .col-lg-push-7 { + left: 58.333333333333336%; + } + .col-lg-push-8 { + left: 66.66666666666666%; + } + .col-lg-push-9 { + left: 75%; + } + .col-lg-push-10 { + left: 83.33333333333334%; + } + .col-lg-push-11 { + left: 91.66666666666666%; + } + .col-lg-pull-1 { + right: 8.333333333333332%; + } + .col-lg-pull-2 { + right: 16.666666666666664%; + } + .col-lg-pull-3 { + right: 25%; + } + .col-lg-pull-4 { + right: 33.33333333333333%; + } + .col-lg-pull-5 { + right: 41.66666666666667%; + } + .col-lg-pull-6 { + right: 50%; + } + .col-lg-pull-7 { + right: 58.333333333333336%; + } + .col-lg-pull-8 { + right: 66.66666666666666%; + } + .col-lg-pull-9 { + right: 75%; + } + .col-lg-pull-10 { + right: 83.33333333333334%; + } + .col-lg-pull-11 { + right: 91.66666666666666%; + } + .col-lg-offset-1 { + margin-left: 8.333333333333332%; + } + .col-lg-offset-2 { + margin-left: 16.666666666666664%; + } + .col-lg-offset-3 { + margin-left: 25%; + } + .col-lg-offset-4 { + margin-left: 33.33333333333333%; + } + .col-lg-offset-5 { + margin-left: 41.66666666666667%; + } + .col-lg-offset-6 { + margin-left: 50%; + } + .col-lg-offset-7 { + margin-left: 58.333333333333336%; + } + .col-lg-offset-8 { + margin-left: 66.66666666666666%; + } + .col-lg-offset-9 { + margin-left: 75%; + } + .col-lg-offset-10 { + margin-left: 83.33333333333334%; + } + .col-lg-offset-11 { + margin-left: 91.66666666666666%; + } +} + +@media (min-width: 1200px) { + .container { + max-width: 1170px; + } +} + +table { + max-width: 100%; + background-color: transparent; +} + +th { + text-align: left; +} + +.table { + width: 100%; + margin-bottom: 20px; +} + +.table thead > tr > th, +.table tbody > tr > th, +.table tfoot > tr > th, +.table thead > tr > td, +.table tbody > tr > td, +.table tfoot > tr > td { + padding: 8px; + line-height: 1.428571429; + vertical-align: top; + border-top: 1px solid #dddddd; +} + +.table thead > tr > th { + vertical-align: bottom; +} + +.table caption + thead tr:first-child th, +.table colgroup + thead tr:first-child th, +.table thead:first-child tr:first-child th, +.table caption + thead tr:first-child td, +.table colgroup + thead tr:first-child td, +.table thead:first-child tr:first-child td { + border-top: 0; +} + +.table tbody + tbody { + border-top: 2px solid #dddddd; +} + +.table .table { + background-color: #ffffff; +} + +.table-condensed thead > tr > th, +.table-condensed tbody > tr > th, +.table-condensed tfoot > tr > th, +.table-condensed thead > tr > td, +.table-condensed tbody > tr > td, +.table-condensed tfoot > tr > td { + padding: 5px; +} + +.table-bordered { + border: 1px solid #dddddd; +} + +.table-bordered > thead > tr > th, +.table-bordered > tbody > tr > th, +.table-bordered > tfoot > tr > th, +.table-bordered > thead > tr > td, +.table-bordered > tbody > tr > td, +.table-bordered > tfoot > tr > td { + border: 1px solid #dddddd; +} + +.table-striped > tbody > tr:nth-child(odd) > td, +.table-striped > tbody > tr:nth-child(odd) > th { + background-color: #f9f9f9; +} + +.table-hover > tbody > tr:hover > td, +.table-hover > tbody > tr:hover > th { + background-color: #f5f5f5; +} + +table col[class^="col-"] { + display: table-column; + float: none; +} + +table td[class^="col-"], +table th[class^="col-"] { + display: table-cell; + float: none; +} + +.table > thead > tr > td.active, +.table > tbody > tr > td.active, +.table > tfoot > tr > td.active, +.table > thead > tr > th.active, +.table > tbody > tr > th.active, +.table > tfoot > tr > th.active, +.table > thead > tr.active > td, +.table > tbody > tr.active > td, +.table > tfoot > tr.active > td, +.table > thead > tr.active > th, +.table > tbody > tr.active > th, +.table > tfoot > tr.active > th { + background-color: #f5f5f5; +} + +.table > thead > tr > td.success, +.table > tbody > tr > td.success, +.table > tfoot > tr > td.success, +.table > thead > tr > th.success, +.table > tbody > tr > th.success, +.table > tfoot > tr > th.success, +.table > thead > tr.success > td, +.table > tbody > tr.success > td, +.table > tfoot > tr.success > td, +.table > thead > tr.success > th, +.table > tbody > tr.success > th, +.table > tfoot > tr.success > th { + background-color: #dff0d8; + border-color: #d6e9c6; +} + +.table > thead > tr > td.danger, +.table > tbody > tr > td.danger, +.table > tfoot > tr > td.danger, +.table > thead > tr > th.danger, +.table > tbody > tr > th.danger, +.table > tfoot > tr > th.danger, +.table > thead > tr.danger > td, +.table > tbody > tr.danger > td, +.table > tfoot > tr.danger > td, +.table > thead > tr.danger > th, +.table > tbody > tr.danger > th, +.table > tfoot > tr.danger > th { + background-color: #f2dede; + border-color: #eed3d7; +} + +.table > thead > tr > td.warning, +.table > tbody > tr > td.warning, +.table > tfoot > tr > td.warning, +.table > thead > tr > th.warning, +.table > tbody > tr > th.warning, +.table > tfoot > tr > th.warning, +.table > thead > tr.warning > td, +.table > tbody > tr.warning > td, +.table > tfoot > tr.warning > td, +.table > thead > tr.warning > th, +.table > tbody > tr.warning > th, +.table > tfoot > tr.warning > th { + background-color: #fcf8e3; + border-color: #fbeed5; +} + +.table-hover > tbody > tr > td.success:hover, +.table-hover > tbody > tr > th.success:hover, +.table-hover > tbody > tr.success:hover > td { + background-color: #d0e9c6; + border-color: #c9e2b3; +} + +.table-hover > tbody > tr > td.danger:hover, +.table-hover > tbody > tr > th.danger:hover, +.table-hover > tbody > tr.danger:hover > td { + background-color: #ebcccc; + border-color: #e6c1c7; +} + +.table-hover > tbody > tr > td.warning:hover, +.table-hover > tbody > tr > th.warning:hover, +.table-hover > tbody > tr.warning:hover > td { + background-color: #faf2cc; + border-color: #f8e5be; +} + +fieldset { + padding: 0; + margin: 0; + border: 0; +} + +legend { + display: block; + width: 100%; + padding: 0; + margin-bottom: 20px; + font-size: 21px; + line-height: inherit; + color: #333333; + border: 0; + border-bottom: 1px solid #e5e5e5; +} + +label { + display: inline-block; + margin-bottom: 5px; + font-weight: bold; +} + +input[type="search"] { + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; +} + +input[type="radio"], +input[type="checkbox"] { + margin: 4px 0 0; + margin-top: 1px \9; + /* IE8-9 */ + + line-height: normal; +} + +input[type="file"] { + display: block; +} + +select[multiple], +select[size] { + height: auto; +} + +select optgroup { + font-family: inherit; + font-size: inherit; + font-style: inherit; +} + +input[type="file"]:focus, +input[type="radio"]:focus, +input[type="checkbox"]:focus { + outline: thin dotted #333; + outline: 5px auto -webkit-focus-ring-color; + outline-offset: -2px; +} + +input[type="number"]::-webkit-outer-spin-button, +input[type="number"]::-webkit-inner-spin-button { + height: auto; +} + +.form-control:-moz-placeholder { + color: #999999; +} + +.form-control::-moz-placeholder { + color: #999999; +} + +.form-control:-ms-input-placeholder { + color: #999999; +} + +.form-control::-webkit-input-placeholder { + color: #999999; +} + +.form-control { + display: block; + width: 100%; + height: 34px; + padding: 6px 12px; + font-size: 14px; + line-height: 1.428571429; + color: #555555; + vertical-align: middle; + background-color: #ffffff; + border: 1px solid #cccccc; + border-radius: 4px; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); + -webkit-transition: border-color ease-in-out 0.15s, box-shadow ease-in-out 0.15s; + transition: border-color ease-in-out 0.15s, box-shadow ease-in-out 0.15s; +} + +.form-control:focus { + border-color: #66afe9; + outline: 0; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(102, 175, 233, 0.6); + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(102, 175, 233, 0.6); +} + +.form-control[disabled], +.form-control[readonly], +fieldset[disabled] .form-control { + cursor: not-allowed; + background-color: #eeeeee; +} + +textarea.form-control { + height: auto; +} + +.form-group { + margin-bottom: 15px; +} + +.radio, +.checkbox { + display: block; + min-height: 20px; + padding-left: 20px; + margin-top: 10px; + margin-bottom: 10px; + vertical-align: middle; +} + +.radio label, +.checkbox label { + display: inline; + margin-bottom: 0; + font-weight: normal; + cursor: pointer; +} + +.radio input[type="radio"], +.radio-inline input[type="radio"], +.checkbox input[type="checkbox"], +.checkbox-inline input[type="checkbox"] { + float: left; + margin-left: -20px; +} + +.radio + .radio, +.checkbox + .checkbox { + margin-top: -5px; +} + +.radio-inline, +.checkbox-inline { + display: inline-block; + padding-left: 20px; + margin-bottom: 0; + font-weight: normal; + vertical-align: middle; + cursor: pointer; +} + +.radio-inline + .radio-inline, +.checkbox-inline + .checkbox-inline { + margin-top: 0; + margin-left: 10px; +} + +.input-lg { + height: 45px; + padding: 10px 16px; + font-size: 18px; + line-height: 1.33; + border-radius: 6px; +} + +.input-sm { + height: 30px; + padding: 5px 10px; + font-size: 12px; + line-height: 1.5; + border-radius: 3px; +} + +select.input-lg { + height: 45px; + line-height: 45px; +} + +select.input-sm { + height: 30px; + line-height: 30px; +} + +textarea.input-lg, +textarea.input-sm { + height: auto; +} + +.has-warning .help-block, +.has-warning .control-label { + color: #c09853; +} + +.has-warning .form-control { + border-color: #c09853; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); +} + +.has-warning .form-control:focus { + border-color: #a47e3c; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e; + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #dbc59e; +} + +.has-warning .input-group-addon { + color: #c09853; + background-color: #fcf8e3; + border-color: #c09853; +} + +.has-error .help-block, +.has-error .control-label { + color: #b94a48; +} + +.has-error .form-control { + border-color: #b94a48; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); +} + +.has-error .form-control:focus { + border-color: #953b39; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392; + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #d59392; +} + +.has-error .input-group-addon { + color: #b94a48; + background-color: #f2dede; + border-color: #b94a48; +} + +.has-success .help-block, +.has-success .control-label { + color: #468847; +} + +.has-success .form-control { + border-color: #468847; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075); +} + +.has-success .form-control:focus { + border-color: #356635; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b; + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #7aba7b; +} + +.has-success .input-group-addon { + color: #468847; + background-color: #dff0d8; + border-color: #468847; +} + +.help-block { + display: block; + margin-top: 5px; + margin-bottom: 10px; + color: #737373; +} + +.form-inline .form-control, +.form-inline .radio, +.form-inline .checkbox { + display: inline-block; +} + +.form-inline .radio, +.form-inline .checkbox { + margin-top: 0; + margin-bottom: 0; +} + +.form-horizontal .control-label, +.form-horizontal .radio-inline, +.form-horizontal .checkbox-inline { + padding-top: 9px; +} + +.form-horizontal .form-group:before, +.form-horizontal .form-group:after { + display: table; + content: " "; +} + +.form-horizontal .form-group:after { + clear: both; +} + +.form-horizontal .form-group:before, +.form-horizontal .form-group:after { + display: table; + content: " "; +} + +.form-horizontal .form-group:after { + clear: both; +} + +@media (min-width: 768px) { + .form-horizontal .form-group { + margin-right: -15px; + margin-left: -15px; + } +} + +.form-horizontal .form-group .row { + margin-right: -15px; + margin-left: -15px; +} + +@media (min-width: 768px) { + .form-horizontal .control-label { + text-align: right; + } +} + +.btn { + display: inline-block; + padding: 6px 12px; + margin-bottom: 0; + font-size: 14px; + font-weight: 500; + line-height: 1.428571429; + text-align: center; + white-space: nowrap; + vertical-align: middle; + cursor: pointer; + border: 1px solid transparent; + border-radius: 4px; + -webkit-user-select: none; + -moz-user-select: none; + -ms-user-select: none; + -o-user-select: none; + user-select: none; +} + +.btn:focus { + outline: thin dotted #333; + outline: 5px auto -webkit-focus-ring-color; + outline-offset: -2px; +} + +.btn:hover, +.btn:focus { + color: #ffffff; + text-decoration: none; +} + +.btn:active, +.btn.active { + outline: 0; + -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125); + box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125); +} + +.btn.disabled, +.btn[disabled], +fieldset[disabled] .btn { + pointer-events: none; + cursor: default; + opacity: 0.65; + filter: alpha(opacity=65); + -webkit-box-shadow: none; + box-shadow: none; +} + +.btn-default { + color: #ffffff; + background-color: #474949; + border-color: #474949; +} + +.btn-default:hover, +.btn-default:focus, +.btn-default:active, +.btn-default.active { + background-color: #3a3c3c; + border-color: #2e2f2f; +} + +.btn-default.disabled, +.btn-default[disabled], +fieldset[disabled] .btn-default, +.btn-default.disabled:hover, +.btn-default[disabled]:hover, +fieldset[disabled] .btn-default:hover, +.btn-default.disabled:focus, +.btn-default[disabled]:focus, +fieldset[disabled] .btn-default:focus, +.btn-default.disabled:active, +.btn-default[disabled]:active, +fieldset[disabled] .btn-default:active, +.btn-default.disabled.active, +.btn-default[disabled].active, +fieldset[disabled] .btn-default.active { + background-color: #474949; + border-color: #474949; +} + +.btn-primary { + color: #ffffff; + background-color: #428bca; + border-color: #428bca; +} + +.btn-primary:hover, +.btn-primary:focus, +.btn-primary:active, +.btn-primary.active { + background-color: #357ebd; + border-color: #3071a9; +} + +.btn-primary.disabled, +.btn-primary[disabled], +fieldset[disabled] .btn-primary, +.btn-primary.disabled:hover, +.btn-primary[disabled]:hover, +fieldset[disabled] .btn-primary:hover, +.btn-primary.disabled:focus, +.btn-primary[disabled]:focus, +fieldset[disabled] .btn-primary:focus, +.btn-primary.disabled:active, +.btn-primary[disabled]:active, +fieldset[disabled] .btn-primary:active, +.btn-primary.disabled.active, +.btn-primary[disabled].active, +fieldset[disabled] .btn-primary.active { + background-color: #428bca; + border-color: #428bca; +} + +.btn-warning { + color: #ffffff; + background-color: #f0ad4e; + border-color: #f0ad4e; +} + +.btn-warning:hover, +.btn-warning:focus, +.btn-warning:active, +.btn-warning.active { + background-color: #eea236; + border-color: #ec971f; +} + +.btn-warning.disabled, +.btn-warning[disabled], +fieldset[disabled] .btn-warning, +.btn-warning.disabled:hover, +.btn-warning[disabled]:hover, +fieldset[disabled] .btn-warning:hover, +.btn-warning.disabled:focus, +.btn-warning[disabled]:focus, +fieldset[disabled] .btn-warning:focus, +.btn-warning.disabled:active, +.btn-warning[disabled]:active, +fieldset[disabled] .btn-warning:active, +.btn-warning.disabled.active, +.btn-warning[disabled].active, +fieldset[disabled] .btn-warning.active { + background-color: #f0ad4e; + border-color: #f0ad4e; +} + +.btn-danger { + color: #ffffff; + background-color: #d9534f; + border-color: #d9534f; +} + +.btn-danger:hover, +.btn-danger:focus, +.btn-danger:active, +.btn-danger.active { + background-color: #d43f3a; + border-color: #c9302c; +} + +.btn-danger.disabled, +.btn-danger[disabled], +fieldset[disabled] .btn-danger, +.btn-danger.disabled:hover, +.btn-danger[disabled]:hover, +fieldset[disabled] .btn-danger:hover, +.btn-danger.disabled:focus, +.btn-danger[disabled]:focus, +fieldset[disabled] .btn-danger:focus, +.btn-danger.disabled:active, +.btn-danger[disabled]:active, +fieldset[disabled] .btn-danger:active, +.btn-danger.disabled.active, +.btn-danger[disabled].active, +fieldset[disabled] .btn-danger.active { + background-color: #d9534f; + border-color: #d9534f; +} + +.btn-success { + color: #ffffff; + background-color: #5cb85c; + border-color: #5cb85c; +} + +.btn-success:hover, +.btn-success:focus, +.btn-success:active, +.btn-success.active { + background-color: #4cae4c; + border-color: #449d44; +} + +.btn-success.disabled, +.btn-success[disabled], +fieldset[disabled] .btn-success, +.btn-success.disabled:hover, +.btn-success[disabled]:hover, +fieldset[disabled] .btn-success:hover, +.btn-success.disabled:focus, +.btn-success[disabled]:focus, +fieldset[disabled] .btn-success:focus, +.btn-success.disabled:active, +.btn-success[disabled]:active, +fieldset[disabled] .btn-success:active, +.btn-success.disabled.active, +.btn-success[disabled].active, +fieldset[disabled] .btn-success.active { + background-color: #5cb85c; + border-color: #5cb85c; +} + +.btn-info { + color: #ffffff; + background-color: #5bc0de; + border-color: #5bc0de; +} + +.btn-info:hover, +.btn-info:focus, +.btn-info:active, +.btn-info.active { + background-color: #46b8da; + border-color: #31b0d5; +} + +.btn-info.disabled, +.btn-info[disabled], +fieldset[disabled] .btn-info, +.btn-info.disabled:hover, +.btn-info[disabled]:hover, +fieldset[disabled] .btn-info:hover, +.btn-info.disabled:focus, +.btn-info[disabled]:focus, +fieldset[disabled] .btn-info:focus, +.btn-info.disabled:active, +.btn-info[disabled]:active, +fieldset[disabled] .btn-info:active, +.btn-info.disabled.active, +.btn-info[disabled].active, +fieldset[disabled] .btn-info.active { + background-color: #5bc0de; + border-color: #5bc0de; +} + +.btn-link { + font-weight: normal; + color: #428bca; + cursor: pointer; + border-radius: 0; +} + +.btn-link, +.btn-link:active, +.btn-link[disabled], +fieldset[disabled] .btn-link { + background-color: transparent; + -webkit-box-shadow: none; + box-shadow: none; +} + +.btn-link, +.btn-link:hover, +.btn-link:focus, +.btn-link:active { + border-color: transparent; +} + +.btn-link:hover, +.btn-link:focus { + color: #2a6496; + text-decoration: underline; + background-color: transparent; +} + +.btn-link[disabled]:hover, +fieldset[disabled] .btn-link:hover, +.btn-link[disabled]:focus, +fieldset[disabled] .btn-link:focus { + color: #333333; + text-decoration: none; +} + +.btn-lg { + padding: 10px 16px; + font-size: 18px; + line-height: 1.33; + border-radius: 6px; +} + +.btn-sm, +.btn-xs { + padding: 5px 10px; + font-size: 12px; + line-height: 1.5; + border-radius: 3px; +} + +.btn-xs { + padding: 3px 5px; +} + +.btn-block { + display: block; + width: 100%; + padding-right: 0; + padding-left: 0; +} + +.btn-block + .btn-block { + margin-top: 5px; +} + +input[type="submit"].btn-block, +input[type="reset"].btn-block, +input[type="button"].btn-block { + width: 100%; +} + +.fade { + opacity: 0; + -webkit-transition: opacity 0.15s linear; + transition: opacity 0.15s linear; +} + +.fade.in { + opacity: 1; +} + +.collapse { + display: none; +} + +.collapse.in { + display: block; +} + +.collapsing { + position: relative; + height: 0; + overflow: hidden; + -webkit-transition: height 0.35s ease; + transition: height 0.35s ease; +} + +.input-group { + position: relative; + display: table; + border-collapse: separate; +} + +.input-group.col { + float: none; + padding-right: 0; + padding-left: 0; +} + +.input-group .form-control { + width: 100%; + margin-bottom: 0; +} + +.input-group-addon, +.input-group-btn, +.input-group .form-control { + display: table-cell; +} + +.input-group-addon:not(:first-child):not(:last-child), +.input-group-btn:not(:first-child):not(:last-child), +.input-group .form-control:not(:first-child):not(:last-child) { + border-radius: 0; +} + +.input-group-addon, +.input-group-btn { + width: 1%; + white-space: nowrap; + vertical-align: middle; +} + +.input-group-addon { + padding: 6px 12px; + font-size: 14px; + font-weight: normal; + line-height: 1.428571429; + text-align: center; + background-color: #eeeeee; + border: 1px solid #cccccc; + border-radius: 4px; + -webkit-box-sizing: border-box; + -moz-box-sizing: border-box; + box-sizing: border-box; +} + +.input-group-addon.input-sm { + padding: 5px 10px; + font-size: 12px; + line-height: 1.5; + border-radius: 3px; +} + +.input-group-addon.input-lg { + padding: 10px 16px; + font-size: 18px; + line-height: 1.33; + border-radius: 6px; +} + +.input-group-addon input[type="radio"], +.input-group-addon input[type="checkbox"] { + margin-top: 0; +} + +.input-group .form-control:first-child, +.input-group-addon:first-child, +.input-group-btn:first-child > .btn, +.input-group-btn:first-child > .dropdown-toggle, +.input-group-btn:last-child > .btn:not(:last-child):not(.dropdown-toggle) { + border-top-right-radius: 0; + border-bottom-right-radius: 0; +} + +.input-group-addon:first-child { + border-right: 0; +} + +.input-group .form-control:last-child, +.input-group-addon:last-child, +.input-group-btn:last-child > .btn, +.input-group-btn:last-child > .dropdown-toggle, +.input-group-btn:first-child > .btn:not(:first-child) { + border-bottom-left-radius: 0; + border-top-left-radius: 0; +} + +.input-group-addon:last-child { + border-left: 0; +} + +.input-group-btn { + position: relative; + white-space: nowrap; +} + +.input-group-btn > .btn { + position: relative; +} + +.input-group-btn > .btn + .btn { + margin-left: -4px; +} + +.input-group-btn > .btn:hover, +.input-group-btn > .btn:active { + z-index: 2; +} + +.caret { + display: inline-block; + width: 0; + height: 0; + margin-left: 2px; + vertical-align: middle; + border-top: 4px solid #000000; + border-right: 4px solid transparent; + border-left: 4px solid transparent; + content: ""; +} + +.dropdown { + position: relative; +} + +.dropdown-menu { + position: absolute; + top: 100%; + left: 0; + z-index: 1000; + display: none; + float: left; + min-width: 160px; + padding: 5px 0; + margin: 2px 0 0; + list-style: none; + background-color: #ffffff; + border: 1px solid #cccccc; + border: 1px solid rgba(0, 0, 0, 0.15); + border-radius: 4px; + -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175); + box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175); + background-clip: padding-box; +} + +.dropdown-menu.pull-right { + right: 0; + left: auto; +} + +.dropdown-menu .divider { + height: 1px; + margin: 9px 0; + overflow: hidden; + background-color: #e5e5e5; +} + +.dropdown-menu > li > a { + display: block; + padding: 3px 20px; + clear: both; + font-weight: normal; + line-height: 1.428571429; + color: #333333; + white-space: nowrap; +} + +.dropdown-menu > li > a:hover, +.dropdown-menu > li > a:focus { + color: #ffffff; + text-decoration: none; + background-color: #357ebd; + background-image: -webkit-gradient(linear, left 0%, left 100%, from(#428bca), to(#357ebd)); + background-image: -webkit-linear-gradient(top, #428bca, 0%, #357ebd, 100%); + background-image: -moz-linear-gradient(top, #428bca 0%, #357ebd 100%); + background-image: linear-gradient(to bottom, #428bca 0%, #357ebd 100%); + background-repeat: repeat-x; + filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff357ebd', GradientType=0); +} + +.dropdown-menu > .active > a, +.dropdown-menu > .active > a:hover, +.dropdown-menu > .active > a:focus { + color: #ffffff; + text-decoration: none; + background-color: #357ebd; + background-image: -webkit-gradient(linear, left 0%, left 100%, from(#428bca), to(#357ebd)); + background-image: -webkit-linear-gradient(top, #428bca, 0%, #357ebd, 100%); + background-image: -moz-linear-gradient(top, #428bca 0%, #357ebd 100%); + background-image: linear-gradient(to bottom, #428bca 0%, #357ebd 100%); + background-repeat: repeat-x; + outline: 0; + filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff428bca', endColorstr='#ff357ebd', GradientType=0); +} + +.dropdown-menu > .disabled > a, +.dropdown-menu > .disabled > a:hover, +.dropdown-menu > .disabled > a:focus { + color: #999999; +} + +.dropdown-menu > .disabled > a:hover, +.dropdown-menu > .disabled > a:focus { + text-decoration: none; + cursor: not-allowed; + background-color: transparent; + background-image: none; + filter: progid:DXImageTransform.Microsoft.gradient(enabled=false); +} + +.open > .dropdown-menu { + display: block; +} + +.open > a { + outline: 0; +} + +.dropdown-header { + display: block; + padding: 3px 20px; + font-size: 12px; + line-height: 1.428571429; + color: #999999; +} + +.dropdown-backdrop { + position: fixed; + top: 0; + right: 0; + bottom: 0; + left: 0; + z-index: 990; +} + +.pull-right > .dropdown-menu { + right: 0; + left: auto; +} + +.dropup .caret, +.navbar-fixed-bottom .dropdown .caret { + border-top: 0; + border-bottom: 4px solid #000000; + content: ""; +} + +.dropup .dropdown-menu, +.navbar-fixed-bottom .dropdown .dropdown-menu { + top: auto; + bottom: 100%; + margin-bottom: 1px; +} + +.list-group { + padding-left: 0; + margin-bottom: 20px; +} + +.list-group-item { + position: relative; + display: block; + padding: 10px 30px 10px 15px; + margin-bottom: -1px; + background-color: #ffffff; + border: 1px solid #dddddd; +} + +.list-group-item:first-child { + border-top-right-radius: 4px; + border-top-left-radius: 4px; +} + +.list-group-item:last-child { + margin-bottom: 0; + border-bottom-right-radius: 4px; + border-bottom-left-radius: 4px; +} + +.list-group-item > .badge { + float: right; + margin-right: -15px; +} + +.list-group-item-heading { + margin-top: 0; + margin-bottom: 5px; +} + +.list-group-item-text { + margin-bottom: 0; + line-height: 1.3; +} + +a.list-group-item .list-group-item-heading { + color: #333333; +} + +a.list-group-item .list-group-item-text { + color: #555555; +} + +a.list-group-item:hover, +a.list-group-item:focus { + text-decoration: none; + background-color: #f5f5f5; +} + +a.list-group-item.active { + z-index: 2; + color: #ffffff; + background-color: #428bca; + border-color: #428bca; +} + +a.list-group-item.active .list-group-item-heading { + color: inherit; +} + +a.list-group-item.active .list-group-item-text { + color: #e1edf7; +} + +.panel { + padding: 15px; + margin-bottom: 20px; + background-color: #ffffff; + border: 1px solid #dddddd; + border-radius: 4px; + -webkit-box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05); + box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05); +} + +.panel .list-group { + margin: 15px -15px -15px; +} + +.panel .list-group .list-group-item { + border-width: 1px 0; +} + +.panel .list-group .list-group-item:first-child { + border-top-right-radius: 0; + border-top-left-radius: 0; +} + +.panel .list-group .list-group-item:last-child { + border-bottom: 0; +} + +.panel-heading { + padding: 10px 15px; + margin: -15px -15px 15px; + background-color: #f5f5f5; + border-bottom: 1px solid #dddddd; + border-top-right-radius: 3px; + border-top-left-radius: 3px; +} + +.panel-title { + margin-top: 0; + margin-bottom: 0; + font-size: 17.5px; + font-weight: 500; +} + +.panel-title > a { + color: inherit; +} + +.panel-footer { + padding: 10px 15px; + margin: 15px -15px -15px; + background-color: #f5f5f5; + border-top: 1px solid #dddddd; + border-bottom-right-radius: 3px; + border-bottom-left-radius: 3px; +} + +.panel-primary { + border-color: #428bca; +} + +.panel-primary .panel-heading { + color: #ffffff; + background-color: #428bca; + border-color: #428bca; +} + +.panel-success { + border-color: #d6e9c6; +} + +.panel-success .panel-heading { + color: #468847; + background-color: #dff0d8; + border-color: #d6e9c6; +} + +.panel-warning { + border-color: #fbeed5; +} + +.panel-warning .panel-heading { + color: #c09853; + background-color: #fcf8e3; + border-color: #fbeed5; +} + +.panel-danger { + border-color: #eed3d7; +} + +.panel-danger .panel-heading { + color: #b94a48; + background-color: #f2dede; + border-color: #eed3d7; +} + +.panel-info { + border-color: #bce8f1; +} + +.panel-info .panel-heading { + color: #3a87ad; + background-color: #d9edf7; + border-color: #bce8f1; +} + +.well { + min-height: 20px; + padding: 19px; + margin-bottom: 20px; + background-color: #f5f5f5; + border: 1px solid #e3e3e3; + border-radius: 4px; + -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05); + box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05); +} + +.well blockquote { + border-color: #ddd; + border-color: rgba(0, 0, 0, 0.15); +} + +.well-lg { + padding: 24px; + border-radius: 6px; +} + +.well-sm { + padding: 9px; + border-radius: 3px; +} + +.close { + float: right; + font-size: 21px; + font-weight: bold; + line-height: 1; + color: #000000; + text-shadow: 0 1px 0 #ffffff; + opacity: 0.2; + filter: alpha(opacity=20); +} + +.close:hover, +.close:focus { + color: #000000; + text-decoration: none; + cursor: pointer; + opacity: 0.5; + filter: alpha(opacity=50); +} + +button.close { + padding: 0; + cursor: pointer; + background: transparent; + border: 0; + -webkit-appearance: none; +} + +.nav { + padding-left: 0; + margin-bottom: 0; + list-style: none; +} + +.nav:before, +.nav:after { + display: table; + content: " "; +} + +.nav:after { + clear: both; +} + +.nav:before, +.nav:after { + display: table; + content: " "; +} + +.nav:after { + clear: both; +} + +.nav > li { + position: relative; + display: block; +} + +.nav > li > a { + position: relative; + display: block; + padding: 10px 15px; +} + +.nav > li > a:hover, +.nav > li > a:focus { + text-decoration: none; + background-color: #eeeeee; +} + +.nav > li.disabled > a { + color: #999999; +} + +.nav > li.disabled > a:hover, +.nav > li.disabled > a:focus { + color: #999999; + text-decoration: none; + cursor: not-allowed; + background-color: transparent; +} + +.nav.open > a, +.nav.open > a:hover, +.nav.open > a:focus { + color: #ffffff; + background-color: #428bca; + border-color: #428bca; +} + +.nav.open > a .caret, +.nav.open > a:hover .caret, +.nav.open > a:focus .caret { + border-top-color: #ffffff; + border-bottom-color: #ffffff; +} + +.nav > .pull-right { + float: right; +} + +.nav .nav-divider { + height: 1px; + margin: 9px 0; + overflow: hidden; + background-color: #e5e5e5; +} + +.nav-tabs { + border-bottom: 1px solid #dddddd; +} + +.nav-tabs > li { + float: left; + margin-bottom: -1px; +} + +.nav-tabs > li > a { + margin-right: 2px; + line-height: 1.428571429; + border: 1px solid transparent; + border-radius: 4px 4px 0 0; +} + +.nav-tabs > li > a:hover { + border-color: #eeeeee; +} + +.nav-tabs > li.active > a, +.nav-tabs > li.active > a:hover, +.nav-tabs > li.active > a:focus { + color: #555555; + cursor: default; + background-color: #ffffff; + border: 1px solid #dddddd; + border-bottom-color: transparent; +} + +.nav-tabs.nav-justified { + width: 100%; + border-bottom: 0; +} + +.nav-tabs.nav-justified > li { + display: table-cell; + float: none; + width: 1%; +} + +.nav-tabs.nav-justified > li > a { + text-align: center; +} + +.nav-tabs.nav-justified > li > a { + margin-right: 0; + border-bottom: 1px solid #dddddd; +} + +.nav-tabs.nav-justified > .active > a { + border-bottom-color: #ffffff; +} + +.nav-pills > li { + float: left; +} + +.nav-pills > li > a { + border-radius: 5px; +} + +.nav-pills > li + li { + margin-left: 2px; +} + +.nav-pills > li.active > a, +.nav-pills > li.active > a:hover, +.nav-pills > li.active > a:focus { + color: #ffffff; + background-color: #428bca; +} + +.nav-stacked > li { + float: none; +} + +.nav-stacked > li + li { + margin-top: 2px; + margin-left: 0; +} + +.nav-justified { + width: 100%; +} + +.nav-justified > li { + display: table-cell; + float: none; + width: 1%; +} + +.nav-justified > li > a { + text-align: center; +} + +.nav-tabs-justified { + border-bottom: 0; +} + +.nav-tabs-justified > li > a { + margin-right: 0; + border-bottom: 1px solid #dddddd; +} + +.nav-tabs-justified > .active > a { + border-bottom-color: #ffffff; +} + +.tabbable:before, +.tabbable:after { + display: table; + content: " "; +} + +.tabbable:after { + clear: both; +} + +.tabbable:before, +.tabbable:after { + display: table; + content: " "; +} + +.tabbable:after { + clear: both; +} + +.tab-content > .tab-pane, +.pill-content > .pill-pane { + display: none; +} + +.tab-content > .active, +.pill-content > .active { + display: block; +} + +.nav .caret { + border-top-color: #428bca; + border-bottom-color: #428bca; +} + +.nav a:hover .caret { + border-top-color: #2a6496; + border-bottom-color: #2a6496; +} + +.nav-tabs .dropdown-menu { + margin-top: -1px; + border-top-right-radius: 0; + border-top-left-radius: 0; +} + +.navbar { + position: relative; + min-height: 50px; + padding-right: 15px; + padding-left: 15px; + margin-bottom: 20px; + background-color: #eeeeee; + border-radius: 4px; +} + +.navbar:before, +.navbar:after { + display: table; + content: " "; +} + +.navbar:after { + clear: both; +} + +.navbar:before, +.navbar:after { + display: table; + content: " "; +} + +.navbar:after { + clear: both; +} + +.navbar-nav { + margin-top: 10px; + margin-bottom: 15px; +} + +.navbar-nav > li > a { + padding-top: 15px; + padding-bottom: 15px; + line-height: 20px; + color: #777777; + border-radius: 4px; +} + +.navbar-nav > li > a:hover, +.navbar-nav > li > a:focus { + color: #333333; + background-color: transparent; +} + +.navbar-nav > .active > a, +.navbar-nav > .active > a:hover, +.navbar-nav > .active > a:focus { + color: #555555; + background-color: #d5d5d5; +} + +.navbar-nav > .disabled > a, +.navbar-nav > .disabled > a:hover, +.navbar-nav > .disabled > a:focus { + color: #cccccc; + background-color: transparent; +} + +.navbar-nav.pull-right { + width: 100%; +} + +.navbar-static-top { + border-radius: 0; +} + +.navbar-fixed-top, +.navbar-fixed-bottom { + position: fixed; + right: 0; + left: 0; + z-index: 1030; + border-radius: 0; +} + +.navbar-fixed-top { + top: 0; +} + +.navbar-fixed-bottom { + bottom: 0; + margin-bottom: 0; +} + +.navbar-brand { + display: block; + max-width: 200px; + padding: 15px 15px; + margin-right: auto; + margin-left: auto; + font-size: 18px; + font-weight: 500; + line-height: 20px; + color: #777777; + text-align: center; +} + +.navbar-brand:hover, +.navbar-brand:focus { + color: #5e5e5e; + text-decoration: none; + background-color: transparent; +} + +.navbar-toggle { + position: relative; + float: right; + width: 48px; + height: 34px; + padding: 6px 12px; + margin-top: 8px; + margin-bottom: 8px; + background-color: transparent; + border: 1px solid #dddddd; + border-radius: 4px; +} + +.navbar-toggle:hover, +.navbar-toggle:focus { + background-color: #dddddd; +} + +.navbar-toggle .icon-bar { + display: block; + width: 22px; + height: 2px; + background-color: #cccccc; + border-radius: 1px; +} + +.navbar-toggle .icon-bar + .icon-bar { + margin-top: 4px; +} + +.navbar-form { + margin-top: 8px; + margin-bottom: 8px; +} + +.navbar-form .form-control, +.navbar-form .radio, +.navbar-form .checkbox { + display: inline-block; +} + +.navbar-form .radio, +.navbar-form .checkbox { + margin-top: 0; + margin-bottom: 0; +} + +.navbar-nav > li > .dropdown-menu { + margin-top: 0; + border-top-right-radius: 0; + border-top-left-radius: 0; +} + +.navbar-fixed-bottom .navbar-nav > li > .dropdown-menu { + border-bottom-right-radius: 0; + border-bottom-left-radius: 0; +} + +.navbar-nav > .dropdown > a:hover .caret, +.navbar-nav > .dropdown > a:focus .caret { + border-top-color: #333333; + border-bottom-color: #333333; +} + +.navbar-nav > .open > a, +.navbar-nav > .open > a:hover, +.navbar-nav > .open > a:focus { + color: #555555; + background-color: #d5d5d5; +} + +.navbar-nav > .open > a .caret, +.navbar-nav > .open > a:hover .caret, +.navbar-nav > .open > a:focus .caret { + border-top-color: #555555; + border-bottom-color: #555555; +} + +.navbar-nav > .dropdown > a .caret { + border-top-color: #777777; + border-bottom-color: #777777; +} + +.navbar-nav.pull-right > li > .dropdown-menu, +.navbar-nav > li > .dropdown-menu.pull-right { + right: 0; + left: auto; +} + +.navbar-inverse { + background-color: #222222; +} + +.navbar-inverse .navbar-brand { + color: #999999; +} + +.navbar-inverse .navbar-brand:hover, +.navbar-inverse .navbar-brand:focus { + color: #ffffff; + background-color: transparent; +} + +.navbar-inverse .navbar-text { + color: #999999; +} + +.navbar-inverse .navbar-nav > li > a { + color: #999999; +} + +.navbar-inverse .navbar-nav > li > a:hover, +.navbar-inverse .navbar-nav > li > a:focus { + color: #ffffff; + background-color: transparent; +} + +.navbar-inverse .navbar-nav > .active > a,
Lines of this file are not shown. View the raw message.
bitbake/lib/webhob/bldviewer/static/js/bootstrap.js new file +1982 -0
1,982 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/static/js/jquery-2.0.3.js new file +8829 -0
8,829 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.8.1.2
Part of the diff is not shown here. View the raw message.
PATCH 62/94 bitbake: webhob: add toggle column functionality to build page
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We add demo toggle column functionality for the Simple interface, webhob builds page.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 15 +++++++++++++++ bitbake/lib/webhob/bldviewer/templates/build.html | 6 ++++++ bitbake/lib/webhob/bldviewer/views.py | 3 ++- 3 files changed, 23 insertions(+), 1 deletion(-)
bitbake/lib/webhob/bldviewer/templates/base.html +15 -0
@@ -3,9 +3,24 @@ <html> <head> <title>WebHob Simple Explorer</title> +<script src="/static/js/jquery-2.0.3.js"> +</script> +<script src="/static/js/bootstrap.js"> +</script> +<link href="/static/css/bootstrap.css" rel="stylesheet" type="text/css"> </head> <body> +<script> +function showhideTableColumn(i, sh) { + if (sh) + $('td:nth-child('+i+'),th:nth-child('+i+')').show(); + else + $('td:nth-child('+i+'),th:nth-child('+i+')').hide(); +g +} +</script> + <div>Menu: <a href="/simple/build/">All Builds</a> | <a href="/simple/layer/">All Layers</a> </div> {% block pagecontent %}
bitbake/lib/webhob/bldviewer/templates/build.html +6 -0
@@ -5,6 +5,12 @@ <table border="1"> +<div align="right" style="display: float">Show/Hide columns: +<span>Output: <input type="checkbox" id="ctoutput" onchange="showhideTableColumn(10, $('#ctoutput').is(':checked'))" checked autocomplete="off"></span> | +<span>Log: <input type="checkbox" id="ctlog" onchange="showhideTableColumn(11, $('#ctlog').is(':checked'))"checked autocomplete="off"></span> | +<span>Build Name: <input type="checkbox" id="ctbuildname" onchange="showhideTableColumn(13, $('#ctbuildname').is(':checked'))" checked autocomplete="off"></span> | +</div> + {% load projecttags %} <tr>
bitbake/lib/webhob/bldviewer/views.py +2 -1
@@ -4,8 +4,9 @@ from django.db.models import Q from django.shortcuts import render from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage from orm.models import Task_Dependency, Package_Dependency +from django.views.decorators.cache import cache_control - +@cache_control(no_store=True) def build(request): template = 'build.html' build_info = Build.objects.all()
Signature · 1 line
1.8.1.2
PATCH 63/94 bitbake: webhob: add search functionality to the Simple interface
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We add generic table search function for the builds table, and make it generic to be able to expand to all other table views.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 12 +++++++++++- bitbake/lib/webhob/bldviewer/templates/build.html | 12 +++++++----- bitbake/lib/webhob/bldviewer/templates/package.html | 2 +- 3 files changed, 19 insertions(+), 7 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +11 -1
@@ -17,7 +17,17 @@ function showhideTableColumn(i, sh) { $('td:nth-child('+i+'),th:nth-child('+i+')').show(); else $('td:nth-child('+i+'),th:nth-child('+i+')').hide(); -g +} + + +function filterTableRows(test) { + if (test.length > 0) { + var r = new RegExp(test); + $('tr.data').map( function (i, el) { if (! r.test($(el).html())) {$(el).hide()} else $(el).show()}); + } else + { + $('tr.data').show(); + } } </script>
bitbake/lib/webhob/bldviewer/templates/build.html +7 -5
@@ -3,14 +3,16 @@ {% block pagecontent %} <h1>WebHob Builds</h1> - <table border="1"> - -<div align="right" style="display: float">Show/Hide columns: +<div style="padding-top:1em ;padding-bottom: 1em"> +<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> +</div> +<div align="right" style="display: inline-block; width: 49%">Show/Hide columns: <span>Output: <input type="checkbox" id="ctoutput" onchange="showhideTableColumn(10, $('#ctoutput').is(':checked'))" checked autocomplete="off"></span> | <span>Log: <input type="checkbox" id="ctlog" onchange="showhideTableColumn(11, $('#ctlog').is(':checked'))"checked autocomplete="off"></span> | <span>Build Name: <input type="checkbox" id="ctbuildname" onchange="showhideTableColumn(13, $('#ctbuildname').is(':checked'))" checked autocomplete="off"></span> | </div> - +</div> + <table border="1"> {% load projecttags %} <tr>
@@ -31,7 +33,7 @@ {% for build in builds %} - <tr> + <tr class="data"> <td><a href="/simple/build/{{build.id}}/task/">{{build.get_outcome_display}}</a></td> <td>{{build.started_on}}</td> <td>{{build.completed_on}}</td>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -1
@@ -19,7 +19,7 @@ {% for package in packages %} - <tr> + <tr class="data"> <td><a name="#{{package.name}}">{{package.name}}</a></td> <td>{{package.version}}</td> <td>{{package.size}}</td>
Signature · 1 line
1.8.1.2
PATCH 64/94 bitbake: webhob: refactor column hiding code
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We're refactoring the column show/hide code to get it more generic, and easier to apply it to all table pages.
--- bitbake/lib/webhob/bldviewer/templates/build.html | 6 +++--- bitbake/lib/webhob/bldviewer/views.py | 6 +++++- 2 files changed, 8 insertions(+), 4 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +3 -3
@@ -7,9 +7,9 @@ <div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> </div> <div align="right" style="display: inline-block; width: 49%">Show/Hide columns: -<span>Output: <input type="checkbox" id="ctoutput" onchange="showhideTableColumn(10, $('#ctoutput').is(':checked'))" checked autocomplete="off"></span> | -<span>Log: <input type="checkbox" id="ctlog" onchange="showhideTableColumn(11, $('#ctlog').is(':checked'))"checked autocomplete="off"></span> | -<span>Build Name: <input type="checkbox" id="ctbuildname" onchange="showhideTableColumn(13, $('#ctbuildname').is(':checked'))" checked autocomplete="off"></span> | +{% for i in hideshowcols %} + <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> | +{% endfor %} </div> </div> <table border="1">
bitbake/lib/webhob/bldviewer/views.py +5 -1
@@ -12,7 +12,11 @@ def build(request): build_info = Build.objects.all() logs = LogMessage.objects.all() - context = {'builds': build_info, 'logs': logs } + context = {'builds': build_info, 'logs': logs , + 'hideshowcols' : [ + {'name': 'Output', 'order':10}, + {'name': 'Log', 'order':11}, + ]} return render(request, template, context)
Signature · 1 line
1.8.1.2
PATCH 65/94 bitbake: webhob: improve search functionality
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
we improve the search functionality to allow for
multiple search terms separated by space.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +5 -2
@@ -22,8 +22,11 @@ function showhideTableColumn(i, sh) { function filterTableRows(test) { if (test.length > 0) { - var r = new RegExp(test); - $('tr.data').map( function (i, el) { if (! r.test($(el).html())) {$(el).hide()} else $(el).show()}); + + var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') }); + $('tr.data').map( function (i, el) { + (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show(); + }); } else { $('tr.data').show();
Signature · 1 line
1.8.1.2
PATCH 66/94 bitbake: webhob: refactor Simple web interface
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We're refactoring the inheritance model and code
placement in the Simple interface as to bring the
search functionality and column selection to all
table-based pages.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 15 ++++++++++++++- bitbake/lib/webhob/bldviewer/templates/build.html | 17 +++++------------ bitbake/lib/webhob/bldviewer/templates/layer.html | 6 ++++-- bitbake/lib/webhob/bldviewer/templates/package.html | 12 +++++++----- bitbake/lib/webhob/bldviewer/templates/recipe.html | 6 ++++-- bitbake/lib/webhob/bldviewer/templates/task.html | 6 ++++-- 6 files changed, 38 insertions(+), 24 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +14 -1
@@ -36,8 +36,21 @@ function filterTableRows(test) { <div>Menu: <a href="/simple/build/">All Builds</a> | <a href="/simple/layer/">All Layers</a> </div> -{% block pagecontent %} +{% block pagename %} +{% endblock %} + +<div style="padding-top:1em ;padding-bottom: 1em"> +<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> +</div> +<div align="right" style="display: inline-block; width: 49%">Show/Hide columns: +{% for i in hideshowcols %} + <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> | +{% endfor %} +</div> +</div> + +{% block pagetable %} {% endblock %} </body> </html>
bitbake/lib/webhob/bldviewer/templates/build.html +5 -12
@@ -1,17 +1,10 @@ {% extends "base.html" %} -{% block pagecontent %} - <h1>WebHob Builds</h1> - -<div style="padding-top:1em ;padding-bottom: 1em"> -<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> -</div> -<div align="right" style="display: inline-block; width: 49%">Show/Hide columns: -{% for i in hideshowcols %} - <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> | -{% endfor %} -</div> -</div> +{% block pagename %} + <h1>Toaster - Builds</h1> +{% endblock %} + +{% block pagetable %} <table border="1"> {% load projecttags %}
bitbake/lib/webhob/bldviewer/templates/layer.html +4 -2
@@ -1,8 +1,10 @@ {% extends "base.html" %} -{% block pagecontent %} - <h1>WebHob Layers</h1> +{% block pagename %} + <h1>Toaster - Layers</h1> +{% endblock %} +{% block pagetable %} <table border="1"> {% load projecttags %}
bitbake/lib/webhob/bldviewer/templates/package.html +7 -5
@@ -1,8 +1,10 @@ {% extends "base.html" %} -{% block pagecontent %} - <h1>WebHob Packages</h1> +{% block pagename %} + <h1>Toaster - Packages</h1> +{% endblock %} +{% block pagetable %} {% if not packages %} <p>No packages were build in this build!</p> {% else %}
@@ -29,11 +31,11 @@ <div style="height: 3em; overflow:auto"> {% for d in package.depends_on %} <a href="#{{d.name}}">{{d.name}}</a><br/> - {% endfor %} + {% endfor %} </div> </td> - - {% endfor %} + + {% endfor %} </table>
bitbake/lib/webhob/bldviewer/templates/recipe.html +4 -2
@@ -1,8 +1,10 @@ {% extends "base.html" %} -{% block pagecontent %} - <h1>WebHob Recipes for a Layer</h1> +{% block pagename %} + <h1>Toaster - Recipes for a Layer</h1> +{% endblock %} +{% block pagetable %} <table border="1"> {% load projecttags %}
bitbake/lib/webhob/bldviewer/templates/task.html +4 -2
@@ -1,8 +1,10 @@ {% extends "base.html" %} -{% block pagecontent %} - <h1>WebHob Tasks</h1> +{% block pagename %} + <h1>Toaster - Tasks</h1> +{% endblock %} +{% block pagetable %} {% if not tasks %} <p>No tasks were executed in this build!</p> {% else %}
Signature · 1 line
1.8.1.2
PATCH 67/94 bitbake: webhob: simple interface CSS
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adds Bootstrap CSS features to the Simple interface, as to make it more visually appealing.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 44 +++++++++++++--------- bitbake/lib/webhob/bldviewer/templates/build.html | 7 +--- bitbake/lib/webhob/bldviewer/templates/layer.html | 6 +-- .../lib/webhob/bldviewer/templates/package.html | 6 +-- bitbake/lib/webhob/bldviewer/templates/recipe.html | 6 +-- bitbake/lib/webhob/bldviewer/templates/task.html | 6 +-- 6 files changed, 31 insertions(+), 44 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +26 -18
@@ -10,7 +10,7 @@ <link href="/static/css/bootstrap.css" rel="stylesheet" type="text/css"> </head> -<body> +<body style="height: 100%"> <script> function showhideTableColumn(i, sh) { if (sh)
@@ -22,7 +22,6 @@ function showhideTableColumn(i, sh) { function filterTableRows(test) { if (test.length > 0) { - var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') }); $('tr.data').map( function (i, el) { (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show();
@@ -33,25 +32,34 @@ function filterTableRows(test) { } } </script> - -<div>Menu: <a href="/simple/build/">All Builds</a> | <a href="/simple/layer/">All Layers</a> -</div> - -{% block pagename %} -{% endblock %} - -<div style="padding-top:1em ;padding-bottom: 1em"> -<div align="left" style="display:inline-block; width: 49%"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> -</div> -<div align="right" style="display: inline-block; width: 49%">Show/Hide columns: -{% for i in hideshowcols %} - <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> | -{% endfor %} -</div> +<div style="position:fixed; top:0; z-index: 999; width: 100%; background:white"> + <ul class="nav nav-tabs"> + <li><a href="/simple/build/">All Builds</a></li> + <li><a href="/simple/layer/">All Layers</a></li> + </ul> + + {% block pagename %} + {% endblock %} + + <div style="padding-top:1em ;padding-bottom: 1em"> + <div align="left" style="display:inline-block; width: 49%; margin-left: 2em"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> + </div> + {% if hideshowcols %} + <div align="right" style="display: inline-block; width: 49%">Show/Hide columns: + {% for i in hideshowcols %} + <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> | + {% endfor %} + </div> + {% endif %} + </div> </div> - +<div style="padding-top: 13em; width:98%; align:center; margin:auto;"> + <table class="table table-striped table-condensed"> {% block pagetable %} {% endblock %} + </table> +</div> +<div class="navbar"><br/>About Toaster | Yocto Project </div> </body> </html>
bitbake/lib/webhob/bldviewer/templates/build.html +1 -6
@@ -5,9 +5,8 @@ {% endblock %} {% block pagetable %} - <table border="1"> - {% load projecttags %} + {% load projecttags %} <tr> <th>Outcome</th> <th>Started On</th>
@@ -23,9 +22,7 @@ <th>Bitbake Version</th> <th>Build Name</th> </tr> - {% for build in builds %} - <tr class="data"> <td><a href="/simple/build/{{build.id}}/task/">{{build.get_outcome_display}}</a></td> <td>{{build.started_on}}</td>
@@ -43,8 +40,6 @@ </tr> {% endfor %} - - </table> {% endblock %}
bitbake/lib/webhob/bldviewer/templates/layer.html +1 -5
@@ -5,8 +5,6 @@ {% endblock %} {% block pagetable %} - <table border="1"> - {% load projecttags %} <tr>
@@ -18,7 +16,7 @@ {% for layer in layers %} - <tr> + <tr class="data"> <td>{{layer.name}}</td> <td>{{layer.local_path}}</td> <td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td>
@@ -33,6 +31,4 @@ {% endfor %} - </table> - {% endblock %}
bitbake/lib/webhob/bldviewer/templates/package.html +1 -5
@@ -9,8 +9,6 @@ <p>No packages were build in this build!</p> {% else %} - <table border="1"> - <tr> <th>Name</th> <th>Version</th>
@@ -21,7 +19,7 @@ {% for package in packages %} - <tr class="data"> + <tr class="data"> <td><a name="#{{package.name}}">{{package.name}}</a></td> <td>{{package.version}}</td> <td>{{package.size}}</td>
@@ -37,8 +35,6 @@ {% endfor %} - </table> - {% endif %} {% endblock %}
bitbake/lib/webhob/bldviewer/templates/recipe.html +1 -5
@@ -5,8 +5,6 @@ {% endblock %} {% block pagetable %} - <table border="1"> - {% load projecttags %} <tr>
@@ -24,7 +22,7 @@ {% for recipe in recipes %} - <tr> + <tr class="data"> <td><a name="{{recipe.name}}">{{recipe.name}}</a></td> <td>{{recipe.version}}</td> <td>{{recipe.summary}}</td>
@@ -39,6 +37,4 @@ {% endfor %} - </table> - {% endblock %}
bitbake/lib/webhob/bldviewer/templates/task.html +1 -5
@@ -9,8 +9,6 @@ <p>No tasks were executed in this build!</p> {% else %} - <table border="1"> - <tr> <th>Order</th> <th>Task</th>
@@ -29,7 +27,7 @@ {% for task in tasks %} - <tr> + <tr class="data"> <td>{{task.order}}</td> <td><a name="{{task.recipe.name}}.{{task.task_name}}"> <a href="/simple/layerversions/{{task.recipe.layer_version_id}}/recipes/#{{task.recipe.name}}">{{task.recipe.name}}</a>.{{task.task_name}}</a></td>
@@ -61,8 +59,6 @@ {% endfor %} - </table> - {% endif %} {% endblock %}
Signature · 1 line
1.8.1.2
PATCH 68/94 bitbake: dsi: use get vars command to store configuration
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Patch to store variable configuration for a build using the getAllKeysWithFlags command. Only variables that are not functions are saved.
Minor fixes to the Variable model as to match data we receive.
--- bitbake/lib/bb/ui/buildinfohelper.py | 15 +++++++++++++-- bitbake/lib/webhob/orm/models.py | 6 +++--- 2 files changed, 16 insertions(+), 5 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +13 -2
@@ -9,7 +9,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, LogMessage -from webhob.orm.models import Task_Dependency, Package_Dependency +from webhob.orm.models import Task_Dependency, Package_Dependency, Variable from bb.msg import BBLogFormatter as format class ORMWrapper(object):
@@ -146,6 +146,15 @@ class ORMWrapper(object): return log_object.save() + def save_build_variables(self, build_obj, vardump): + for k in vardump: + if not bool(vardump[k]['func']): + Variable.objects.create( build = build_obj, + variable_name = k, + variable_value = vardump[k]['v'], + description = vardump[k]['doc']) + + class BuildInfoHelper(object): """ This class gathers the build information from the server and sends it towards the ORM wrapper for storing in the database
@@ -363,13 +372,15 @@ class BuildInfoHelper(object): build_obj = self.orm_wrapper.create_build_object(build_information) self.internal_state['build'] = build_obj self.internal_state['target'] = build_information['target'] + # Load layer information for the build self.internal_state['layer_versions'] = [] for layer_object in self.internal_state['layers']: layer_version_information = self._get_layer_version_information(layer_object) self.internal_state['layer_versions'].append(self.orm_wrapper.get_layer_version_object(layer_version_information)) del self.internal_state['layers'] - + # Save build configuration + self.orm_wrapper.save_build_variables(build_obj, self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0]) def update_build_information(self, event, errors, warnings, taskfailures):
bitbake/lib/webhob/orm/models.py +3 -3
@@ -161,11 +161,11 @@ class Layer_Version(models.Model): class Variable(models.Model): build = models.ForeignKey(Build, related_name='variable_build') variable_name = models.CharField(max_length=100) - variable_value = models.TextField() + variable_value = models.TextField(null=True) file = models.FilePathField(max_length=255) - changed = models.BooleanField() + changed = models.BooleanField(default=False) human_readable_name = models.CharField(max_length=200) - description = models.TextField() + description = models.TextField(null=True) class Machine(models.Model):
Signature · 1 line
1.8.1.2
PATCH 69/94 bitbake: webhob: refactor CSS display in Simple
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Refactoring CSS display in Simple to separate
the generic layout to the table display code.
With the new layout we can add the generic display to more than just table pages.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 51 ++++------------------ .../lib/webhob/bldviewer/templates/basetable.html | 46 +++++++++++++++++++ bitbake/lib/webhob/bldviewer/templates/build.html | 2 +- bitbake/lib/webhob/bldviewer/templates/layer.html | 2 +- .../lib/webhob/bldviewer/templates/package.html | 2 +- bitbake/lib/webhob/bldviewer/templates/recipe.html | 2 +- bitbake/lib/webhob/bldviewer/templates/task.html | 2 +- 7 files changed, 59 insertions(+), 48 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/basetable.html
bitbake/lib/webhob/bldviewer/templates/base.html +8 -43
@@ -2,7 +2,7 @@ <html> <head> - <title>WebHob Simple Explorer</title> + <title>Toaster Simple Explorer</title> <script src="/static/js/jquery-2.0.3.js"> </script> <script src="/static/js/bootstrap.js">
@@ -11,55 +11,20 @@ </head> <body style="height: 100%"> -<script> -function showhideTableColumn(i, sh) { - if (sh) - $('td:nth-child('+i+'),th:nth-child('+i+')').show(); - else - $('td:nth-child('+i+'),th:nth-child('+i+')').hide(); -} - - -function filterTableRows(test) { - if (test.length > 0) { - var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') }); - $('tr.data').map( function (i, el) { - (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show(); - }); - } else - { - $('tr.data').show(); - } -} -</script> -<div style="position:fixed; top:0; z-index: 999; width: 100%; background:white"> +<div style="width:100%; height: 100%; position:absolute"> +<div style="width: 100%; height: 3em" class="nav"> <ul class="nav nav-tabs"> <li><a href="/simple/build/">All Builds</a></li> <li><a href="/simple/layer/">All Layers</a></li> </ul> - - {% block pagename %} - {% endblock %} - - <div style="padding-top:1em ;padding-bottom: 1em"> - <div align="left" style="display:inline-block; width: 49%; margin-left: 2em"> Search: <input type="search" id="filterstring" style="width: 30em" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off"> - </div> - {% if hideshowcols %} - <div align="right" style="display: inline-block; width: 49%">Show/Hide columns: - {% for i in hideshowcols %} - <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> | - {% endfor %} - </div> - {% endif %} - </div> </div> -<div style="padding-top: 13em; width:98%; align:center; margin:auto;"> - <table class="table table-striped table-condensed"> -{% block pagetable %} + +<div style="overflow-y:scroll; width: 100%; position: absolute; top: 3em; bottom:70px "> +{% block pagecontent %} {% endblock %} - </table> </div> -<div class="navbar"><br/>About Toaster | Yocto Project </div> +<div class="navbar" style="position: absolute; bottom: 0; width:100%"><br/>About Toaster | Yocto Project </div> +</div> </body> </html>
bitbake/lib/webhob/bldviewer/templates/basetable.html new file +46 -0
46 lines not on this page · raw message
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
@@ -1,4 +1,4 @@ -{% extends "base.html" %} +{% extends "basetable.html" %} {% block pagename %} <h1>Toaster - Builds</h1>
bitbake/lib/webhob/bldviewer/templates/layer.html +1 -1
@@ -1,4 +1,4 @@ -{% extends "base.html" %} +{% extends "basetable.html" %} {% block pagename %} <h1>Toaster - Layers</h1>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -1
@@ -1,4 +1,4 @@ -{% extends "base.html" %} +{% extends "basetable.html" %} {% block pagename %} <h1>Toaster - Packages</h1>
bitbake/lib/webhob/bldviewer/templates/recipe.html +1 -1
@@ -1,4 +1,4 @@ -{% extends "base.html" %} +{% extends "basetable.html" %} {% block pagename %} <h1>Toaster - Recipes for a Layer</h1>
bitbake/lib/webhob/bldviewer/templates/task.html +1 -1
@@ -1,4 +1,4 @@ -{% extends "base.html" %} +{% extends "basetable.html" %} {% block pagename %} <h1>Toaster - Tasks</h1>
Signature · 1 line
1.8.1.2
PATCH 70/94 bitbake: webhob: add Configuration visualisation in Simple
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We add a page to show the build configuration variables in the Simple display.
We add a bit of navigation menu to allow moving between pages of the same build.
--- .../webhob/bldviewer/templates/configuration.html | 28 ++++++++++++++++++++++ .../lib/webhob/bldviewer/templates/package.html | 7 +++++- bitbake/lib/webhob/bldviewer/templates/task.html | 5 ++++ bitbake/lib/webhob/bldviewer/urls.py | 1 + bitbake/lib/webhob/bldviewer/views.py | 12 +++++++--- 5 files changed, 49 insertions(+), 4 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/configuration.html
bitbake/lib/webhob/bldviewer/templates/configuration.html new file +28 -0
@@ -0,0 +1,28 @@ +{% extends "basetable.html" %} + +{% block pagename %} +<ul class="nav nav-tabs" style="display: inline-block"> + <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> + <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> + <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> +</ul> + <h1>Toaster - Build Configuration </h1> +{% endblock %} + +{% block pagetable %} + + <tr> + <th>Name</th> + <th>Value</th> + <th>Description</th> + </tr> + + {% for variable in configuration %} + + <tr class="data"> + <td>{{variable.variable_name}}</td> + <td>{{variable.variable_value}}</td> + <td>{% if variable.description %}{{variable.description}}{% endif %}</td> + {% endfor %} + +{% endblock %}
bitbake/lib/webhob/bldviewer/templates/package.html +6 -1
@@ -1,7 +1,12 @@ {% extends "basetable.html" %} {% block pagename %} - <h1>Toaster - Packages</h1> +<ul class="nav nav-tabs" style="display: inline-block"> + <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> + <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> + <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> +</ul> + <h1>Toaster - Packages</h1> {% endblock %} {% block pagetable %}
bitbake/lib/webhob/bldviewer/templates/task.html +5 -0
@@ -1,6 +1,11 @@ {% extends "basetable.html" %} {% block pagename %} +<ul class="nav nav-tabs" style="display: inline-block"> + <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> + <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> + <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> +</ul> <h1>Toaster - Tasks</h1> {% endblock %}
bitbake/lib/webhob/bldviewer/urls.py +1 -0
@@ -5,6 +5,7 @@ urlpatterns = patterns('bldviewer.views', url(r'^build/$', 'build', name='build'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'), + url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'), url(r'^layer/$', 'layer', name='layer'), url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'), url(r'^$', redirect_to, {'url': 'build/'}),
bitbake/lib/webhob/bldviewer/views.py +9 -3
@@ -2,7 +2,7 @@ import operator from django.db.models import Q from django.shortcuts import render -from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage +from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage, Variable from orm.models import Task_Dependency, Package_Dependency from django.views.decorators.cache import cache_control
@@ -32,10 +32,16 @@ def task(request, build_id): if t == k.task: t.depends_on.append(k.depends_on) - context = {'tasks': tasks} + context = {'build': build_id, 'tasks': tasks} return render(request, template, context) +def configuration(request, build_id): + template = 'configuration.html' + variables = Variable.objects.filter(build=build_id) + context = {'build': build_id, 'configuration' : variables} + return render(request, template, context) + def package(request, build_id): template = 'package.html'
@@ -48,7 +54,7 @@ def package(request, build_id): if t == k.package: t.depends_on.append(k.depends_on) - context = {'packages': packages} + context = {'build' : build_id ,'packages': packages} return render(request, template, context)
Signature · 1 line
1.8.1.2
PATCH 71/94 bitbake: webhob: add navigation links Simple interface
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding a couple of navigation links on top of pages so that we provide easier movement between pages and a bit more context.
--- bitbake/lib/webhob/bldviewer/templates/build.html | 2 +- bitbake/lib/webhob/bldviewer/templates/configuration.html | 1 + bitbake/lib/webhob/bldviewer/templates/package.html | 1 + bitbake/lib/webhob/bldviewer/templates/recipe.html | 3 +++ bitbake/lib/webhob/bldviewer/templates/task.html | 1 + bitbake/lib/webhob/bldviewer/views.py | 6 ++++-- 6 files changed, 11 insertions(+), 3 deletions(-)
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
@@ -24,7 +24,7 @@ </tr> {% for build in builds %} <tr class="data"> - <td><a href="/simple/build/{{build.id}}/task/">{{build.get_outcome_display}}</a></td> + <td><a href="/simple/build/{{build.id}}/configuration/">{{build.get_outcome_display}}</a></td> <td>{{build.started_on}}</td> <td>{{build.completed_on}}</td> <td>{{build.target}}</td>
bitbake/lib/webhob/bldviewer/templates/configuration.html +1 -0
@@ -2,6 +2,7 @@ {% block pagename %} <ul class="nav nav-tabs" style="display: inline-block"> + <li><a>Build {{build}} : </a></li> <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -0
@@ -2,6 +2,7 @@ {% block pagename %} <ul class="nav nav-tabs" style="display: inline-block"> + <li><a>Build {{build}} : </a></li> <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
bitbake/lib/webhob/bldviewer/templates/recipe.html +3 -0
@@ -1,6 +1,9 @@ {% extends "basetable.html" %} {% block pagename %} +<ul class="nav nav-tabs" style="display: inline-block"> + <li><a>Layer {{layer_version.layer.name}} : {{layer_version.branch}} : {{layer_version.commit}}</a></li> +</ul> <h1>Toaster - Recipes for a Layer</h1> {% endblock %}
bitbake/lib/webhob/bldviewer/templates/task.html +1 -0
@@ -2,6 +2,7 @@ {% block pagename %} <ul class="nav nav-tabs" style="display: inline-block"> + <li><a>Build {{build}} : </a></li> <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li>
bitbake/lib/webhob/bldviewer/views.py +4 -2
@@ -12,7 +12,7 @@ def build(request): build_info = Build.objects.all() logs = LogMessage.objects.all() - context = {'builds': build_info, 'logs': logs , + context = {'builds': build_info, 'logs': logs , 'hideshowcols' : [ {'name': 'Output', 'order':10}, {'name': 'Log', 'order':11},
@@ -76,7 +76,9 @@ def layer_versions_recipes(request, layerversion_id): template = 'recipe.html' recipes = Recipe.objects.filter(layer_version__id = layerversion_id) - context = {'recipes': recipes} + context = {'recipes': recipes, + 'layer_version' : Layer_Version.objects.filter( id = layerversion_id )[0] + } return render(request, template, context)
Signature · 1 line
1.8.1.2
PATCH 72/94 bitbake: webhob: startup script fixing
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Let's not start the webserver if the database sync went wrong.
--- bitbake/bin/webhob | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
bitbake/bin/webhob +2 -1
@@ -43,8 +43,9 @@ function webserverStartAll() python $BBBASEDIR/lib/webhob/manage.py syncdb || retval=1 if [ $retval -eq 1 ]; then echo "Failed db sync, stopping system start" 1>&2 + else + python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid fi - python $BBBASEDIR/lib/webhob/manage.py runserver </dev/null >${BUILDDIR}/whbmain.log 2>&1 & echo $! >${BUILDDIR}/whbmain.pid return $retval }
Signature · 1 line
1.8.1.2
PATCH 73/94 bitbake: webhob: change database models and related
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We change the database models as to allow storage of data based on new Package design:
* there is a separate Target model that will hold each of the targets of a build.
* the package information from an image file is stored in Target_Package model
* the package information from the build process is stored in Build_Package model
* there is a provision to store dependency type between Packages
* the code in the DSI has been altered to cope with the new separate Targets for each build
* the code in Simple interface has been changed to provision for the new model code.
--- bitbake/lib/bb/ui/buildinfohelper.py | 153 +++++++++++---------- bitbake/lib/bb/ui/dsi.py | 5 +- bitbake/lib/webhob/bldviewer/templates/build.html | 2 +- .../lib/webhob/bldviewer/templates/package.html | 4 +- bitbake/lib/webhob/bldviewer/urls.py | 1 + bitbake/lib/webhob/bldviewer/views.py | 11 +- bitbake/lib/webhob/orm/models.py | 71 +++++++--- 7 files changed, 151 insertions(+), 96 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +84 -69
@@ -1,6 +1,5 @@ import datetime import sys -import uuid import bb import re
@@ -8,8 +7,9 @@ import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Package, LogMessage -from webhob.orm.models import Task_Dependency, Package_Dependency, Variable +from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage +from webhob.orm.models import Target_Package, Build_Package, Variable +from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency from bb.msg import BBLogFormatter as format class ORMWrapper(object):
@@ -30,20 +30,30 @@ class ORMWrapper(object): def create_build_object(self, build_info): - build = Build.objects.create(uuid=build_info['uuid'], - target=build_info['target'], + build = Build.objects.create( machine=build_info['machine'], distro=build_info['distro'], distro_version=build_info['distro_version'], started_on=build_info['started_on'], completed_on=build_info['completed_on'], - image_fstypes=build_info['image_fstypes'], cooker_log_path=build_info['cooker_log_path'], build_name=build_info['build_name'], bitbake_version=build_info['bitbake_version']) return build + def create_target_objects(self, target_info): + targets = [] + for tgt_name in target_info['targets']: + tgt_object = Target.objects.create( build = target_info['build'], + target = tgt_name, + is_image = False, + image_fstypes = "", + file_name = "", + file_size = 0); + targets.append(tgt_object) + return targets + def update_build_object(self, build, errors, warnings, taskfailures): outcome = Build.SUCCEEDED
@@ -118,9 +128,9 @@ class ORMWrapper(object): return layer_object[0] - def save_package_information(self, build_obj, packagedict, bldpkgs, recipes): + def save_target_package_information(self, target_obj, packagedict, bldpkgs, recipes): for p in packagedict: - packagedict[p]['object'] = Package.objects.create( build = build_obj, + packagedict[p]['object'] = Target_Package.objects.create( target = target_obj, name = p, size = packagedict[p]['size']) if p in bldpkgs:
@@ -130,7 +140,7 @@ class ORMWrapper(object): for p in packagedict: for px in packagedict[p]['depends']: - Package_Dependency.objects.create( package = packagedict[p]['object'], + Target_Package_Dependency.objects.create( package = packagedict[p]['object'], depends_on = packagedict[px]['object'] );
@@ -165,7 +175,6 @@ class BuildInfoHelper(object): def __init__(self, server, has_build_history = False): self._configure_django() self.internal_state = {} - self.uuid = None self.task_order = 0 self.server = server self.orm_wrapper = ORMWrapper()
@@ -252,7 +261,6 @@ class BuildInfoHelper(object): build_info = {} # Generate an identifier for each new build - build_info['uuid'] = self.uuid build_info['machine'] = machine_obj build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0] build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
@@ -307,15 +315,17 @@ class BuildInfoHelper(object): def _get_path_information(self, task_object): build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/" + build_stats_path = [] - target = self.internal_state['target'] - machine = self.internal_state['build'].machine.name - buildname = self.internal_state['build'].build_name - package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":") + for t in self.internal_state['targets']: + target = t.target + machine = self.internal_state['build'].machine.name + buildname = self.internal_state['build'].build_name + package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":") - build_stats_path = build_stats_format.format(tmpdir=self.tmp_dir, target=target, + build_stats_path.append(build_stats_format.format(tmpdir=self.tmp_dir, target=target, machine=machine, buildname=buildname, - package=package) + package=package)) return build_stats_path
@@ -364,14 +374,19 @@ class BuildInfoHelper(object): machine_information = self._get_machine_information() machine_obj = self.orm_wrapper.create_machine_object(machine_information) - self.uuid = str(uuid.uuid4()) build_information = self._get_build_information(machine_obj) - build_information['target'] = ' '.join(event.getPkgs()) build_obj = self.orm_wrapper.create_build_object(build_information) self.internal_state['build'] = build_obj - self.internal_state['target'] = build_information['target'] + + # create target information + target_information = {} + target_information['targets'] = event.getPkgs() + target_information['build'] = build_obj + + self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information) + # Load layer information for the build self.internal_state['layer_versions'] = [] for layer_object in self.internal_state['layers']:
@@ -428,7 +443,7 @@ class BuildInfoHelper(object): task_information['message'] = event._message if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)): - task_information['outcome'] = Task.OUTCOME_SUCCESS # TODO: needs to use constants + task_information['outcome'] = Task.OUTCOME_SUCCESS task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information)) task_information['cpu_usage'] = task_build_stats['cpu_usage'] task_information['disk_io'] = task_build_stats['disk_io']
@@ -438,54 +453,52 @@ class BuildInfoHelper(object): task_information['outcome'] = Task.OUTCOME_FAILED del self.internal_state[identifier] - #TODO: get error number - #TODO: get warnings number - #TODO: get warning information - - self.orm_wrapper.get_update_task_object(task_information) - def read_package_dep_data(self, event): - # verify that we have something to read - if not self.internal_state['build'].is_image or not self.has_build_history: - print "not collecting package info ", self.internal_state['build'].is_image, self.has_build_history - return - - # fair warning: code correlates to buildhistory.bbclass; anything changes there, needs to chage here too - TOPDIR, error = self.server.runCommand(['getVariable', 'TOPDIR']) - MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH']) - TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC']) - MULTIMACH_TARGET_SYS, error = self.server.runCommand(['getVariable', 'MULTIMACH_TARGET_SYS']) - SDK_NAME, error = self.server.runCommand(['getVariable', 'SDK_NAME']) - BUILDHISTORY_DIR = "%s/buildhistory" % TOPDIR - BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, self.internal_state['build'].target) - - self.internal_state['packages'] = {} - - with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin: - for line in fin: - line = line.rstrip(";") - psize, px = line.split("\t") - punit, pname = px.split(" ") - self.internal_state['packages'][pname.strip()] = {'size':psize, 'depends' : []} - - with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin: - p = re.compile(r' -> ') - for line in fin: - line = line.rstrip(';') - linesplit = p.split(line) - if len(linesplit) == 2: - pname = linesplit[0] - dependsname = linesplit[1].split(" ")[0].strip().strip(";") - if not pname in self.internal_state['packages']: - self.internal_state['packages'][pname] = {'size': 0, 'depends' : []} - if not dependsname in self.internal_state['packages']: - self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []} - self.internal_state['packages'][pname]['depends'].append(dependsname) - - self.orm_wrapper.save_package_information(self.internal_state['build'], self.internal_state['packages'], - self.internal_state['bldpkgs'], self.internal_state['recipes']) + def read_target_package_dep_data(self, event): + # for all targets + for target in self.internal_state['targets']: + # verify that we have something to read + if not target.is_image or not self.has_build_history: + print "not collecting package info ", target.is_image, self.has_build_history + break + + # fair warning: code correlates to buildhistory.bbclass; anything changes there, needs to chage here too + TOPDIR, error = self.server.runCommand(['getVariable', 'TOPDIR']) + MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH']) + TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC']) + MULTIMACH_TARGET_SYS, error = self.server.runCommand(['getVariable', 'MULTIMACH_TARGET_SYS']) + SDK_NAME, error = self.server.runCommand(['getVariable', 'SDK_NAME']) + BUILDHISTORY_DIR = "%s/buildhistory" % TOPDIR + BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, target.target) + + self.internal_state['packages'] = {} + + with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin: + for line in fin: + line = line.rstrip(";") + psize, px = line.split("\t") + punit, pname = px.split(" ") + self.internal_state['packages'][pname.strip()] = {'size':psize, 'depends' : []} + + with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin: + p = re.compile(r' -> ') + for line in fin: + line = line.rstrip(';') + linesplit = p.split(line) + if len(linesplit) == 2: + pname = linesplit[0] + dependsname = linesplit[1].split(" ")[0].strip().strip(";") + if not pname in self.internal_state['packages']: + self.internal_state['packages'][pname] = {'size': 0, 'depends' : []} + if not dependsname in self.internal_state['packages']: + self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []} + self.internal_state['packages'][pname]['depends'].append(dependsname) + + self.orm_wrapper.save_target_package_information(target, + self.internal_state['packages'], + self.internal_state['bldpkgs'], self.internal_state['recipes']) def store_dependency_information(self, event):
@@ -519,9 +532,11 @@ class BuildInfoHelper(object): recipe_info['file_path'] = file_name recipe = self.orm_wrapper.get_update_recipe_object(recipe_info) recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits']) - if recipe.is_image and pn == self.internal_state['build'].target: - self.internal_state['build'].is_image = True - self.internal_state['build'].save() + if recipe.is_image: + for t in self.internal_state['targets']: + if pn == t.target: + t.is_image = True + t.save() self.internal_state['recipes'][pn] = recipe # save all task information
bitbake/lib/bb/ui/dsi.py +4 -1
@@ -471,7 +471,7 @@ def main(server, eventHandler, params, tf = TerminalFilter): continue if isinstance(event, (bb.event.BuildCompleted)): - buildinfohelper.read_package_dep_data(event) + buildinfohelper.read_target_package_dep_data(event) buildinfohelper.update_build_information(event, errors, warnings, taskfailures) continue
@@ -531,6 +531,9 @@ def main(server, eventHandler, params, tf = TerminalFilter): logger.error("Unable to cleanly shutdown: %s" % error) main.shutdown = main.shutdown + 1 pass + except Exception as e: + print(e) + pass summary = "" if taskfailures:
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
@@ -27,7 +27,7 @@ <td><a href="/simple/build/{{build.id}}/configuration/">{{build.get_outcome_display}}</a></td> <td>{{build.started_on}}</td> <td>{{build.completed_on}}</td> - <td>{{build.target}}</td> + <td>{% for t in build.target_set.all %}<a href="/simple/build/{{build.id}}/target/{{t.id}}">{{t.target}}</a><br/>{% endfor %}</td> <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td> <td>{{build.machine.name}}</td> <td>{% time_difference build.started_on build.completed_on %}</td>
bitbake/lib/webhob/bldviewer/templates/package.html +2 -2
@@ -7,12 +7,12 @@ <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> </ul> - <h1>Toaster - Packages</h1> + <h1>Toaster - Target Install Packages</h1> {% endblock %} {% block pagetable %} {% if not packages %} - <p>No packages were build in this build!</p> + <p>No packages were recorded for this target!</p> {% else %} <tr>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
@@ -4,6 +4,7 @@ from django.views.generic.simple import redirect_to urlpatterns = patterns('bldviewer.views', url(r'^build/$', 'build', name='build'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), + url(r'^build/(?P<build_id>\d+)/targets/$', 'task', name='task'), url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'), url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'), url(r'^layer/$', 'layer', name='layer'),
bitbake/lib/webhob/bldviewer/views.py +6 -5
@@ -2,14 +2,15 @@ import operator from django.db.models import Q from django.shortcuts import render -from orm.models import Build, Task, Layer, Layer_Version, Recipe, Package, LogMessage, Variable -from orm.models import Task_Dependency, Package_Dependency +from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable +from orm.models import Task_Dependency, Target_Package_Dependency from django.views.decorators.cache import cache_control @cache_control(no_store=True) def build(request): template = 'build.html' build_info = Build.objects.all() + logs = LogMessage.objects.all() context = {'builds': build_info, 'logs': logs ,
@@ -45,8 +46,8 @@ def configuration(request, build_id): def package(request, build_id): template = 'package.html' - packages = Package.objects.filter(build=build_id) - package_depends = Package_Dependency.objects.filter(package__in=packages) + packages = Target_Package.objects.filter(build=build_id) + package_depends = Target_Package_Dependency.objects.filter(package__in=packages) for t in packages: t.depends_on = []
@@ -96,7 +97,7 @@ def model_explorer(request, model_name): model_mapping = { 'build': Build, 'task': Task, - 'package': Package, + 'package': Target_Package, 'layer': Layer, 'layerversion': Layer_Version, 'recipe': Recipe,
bitbake/lib/webhob/orm/models.py +53 -18
104 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH 74/94 bitbake: webhob: navigation in the Simple interface
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Modifies the navigation in the Simple interface to allow new multiple targets per build model.
Refactoring the code to load relationships as to be based on Django's RelationshipManager automatic follower.
--- bitbake/lib/webhob/bldviewer/templates/base.html | 4 ++-- .../webhob/bldviewer/templates/configuration.html | 13 ++---------- .../lib/webhob/bldviewer/templates/package.html | 2 +- bitbake/lib/webhob/bldviewer/templates/task.html | 19 +++++------------- bitbake/lib/webhob/bldviewer/urls.py | 10 +++++----- bitbake/lib/webhob/bldviewer/views.py | 23 +++++----------------- bitbake/lib/webhob/orm/models.py | 6 +++++- bitbake/lib/webhob/whbmain/urls.py | 2 +- 8 files changed, 26 insertions(+), 53 deletions(-)
bitbake/lib/webhob/bldviewer/templates/base.html +2 -2
@@ -14,8 +14,8 @@ <div style="width:100%; height: 100%; position:absolute"> <div style="width: 100%; height: 3em" class="nav"> <ul class="nav nav-tabs"> - <li><a href="/simple/build/">All Builds</a></li> - <li><a href="/simple/layer/">All Layers</a></li> + <li><a href="/simple/builds/">All Builds</a></li> + <li><a href="/simple/layers/">All Layers</a></li> </ul> </div>
bitbake/lib/webhob/bldviewer/templates/configuration.html +2 -11
@@ -1,15 +1,6 @@ -{% extends "basetable.html" %} - -{% block pagename %} -<ul class="nav nav-tabs" style="display: inline-block"> - <li><a>Build {{build}} : </a></li> - <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> - <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> - <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> -</ul> - <h1>Toaster - Build Configuration </h1> -{% endblock %} +{% extends "basebuildpage.html" %} +{% block pagetitle %}Configuration{% endblock %} {% block pagetable %} <tr>
bitbake/lib/webhob/bldviewer/templates/package.html +1 -1
@@ -1,4 +1,4 @@ -{% extends "basetable.html" %} +{% extends "basebuildpage.html" %} {% block pagename %} <ul class="nav nav-tabs" style="display: inline-block">
bitbake/lib/webhob/bldviewer/templates/task.html +5 -14
@@ -1,15 +1,6 @@ -{% extends "basetable.html" %} - -{% block pagename %} -<ul class="nav nav-tabs" style="display: inline-block"> - <li><a>Build {{build}} : </a></li> - <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> - <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> - <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> -</ul> - <h1>Toaster - Tasks</h1> -{% endblock %} +{% extends "basebuildpage.html" %} +{% block pagetitle %}Tasks{% endblock %} {% block pagetable %} {% if not tasks %} <p>No tasks were executed in this build!</p>
@@ -55,9 +46,9 @@ <td><a target="_fileview" href="file:///{{task.recipe.file_path}}">{{task.recipe.file_path}}</a></td> <td> <div style="height: 3em; overflow:auto"> - {% for tt in task.depends_on %} - <a href="#{{tt.recipe.name}}.{{tt.task_name}}"> - {{tt.recipe.name}}.{{tt.task_name}}</a><br/> + {% for tt in task.task_dependencies_task.all %} + <a href="#{{tt.depends_on.recipe.name}}.{{tt.depends_on.task_name}}"> + {{tt.depends_on.recipe.name}}.{{tt.depends_on.task_name}}</a><br/> {% endfor %} </div> </td>
bitbake/lib/webhob/bldviewer/urls.py +5 -5
@@ -2,12 +2,12 @@ from django.conf.urls import patterns, include, url from django.views.generic.simple import redirect_to urlpatterns = patterns('bldviewer.views', - url(r'^build/$', 'build', name='build'), + url(r'^builds/$', 'build', name='build'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), - url(r'^build/(?P<build_id>\d+)/targets/$', 'task', name='task'), - url(r'^build/(?P<build_id>\d+)/package/$', 'package', name='package'), + url(r'^build/(?P<build_id>\d+)/packages/$', 'bpackage', name='bpackage'), + url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/packages/$', 'tpackage', name='tpackage'), url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'), - url(r'^layer/$', 'layer', name='layer'), + url(r'^layers/$', 'layer', name='layer'), url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'), - url(r'^$', redirect_to, {'url': 'build/'}), + url(r'^$', redirect_to, {'url': 'builds/'}), )
bitbake/lib/webhob/bldviewer/views.py +5 -18
@@ -25,37 +25,24 @@ def task(request, build_id): template = 'task.html' tasks = Task.objects.filter(build=build_id) - task_depends = Task_Dependency.objects.filter(task__in=tasks).select_related() - for t in tasks: - t.depends_on = [] - for k in task_depends: - if t == k.task: - t.depends_on.append(k.depends_on) - - context = {'build': build_id, 'tasks': tasks} + context = {'build': Build.objects.filter(pk=build_id)[0], 'tasks': tasks} return render(request, template, context) def configuration(request, build_id): template = 'configuration.html' variables = Variable.objects.filter(build=build_id) - context = {'build': build_id, 'configuration' : variables} + context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables} return render(request, template, context) -def package(request, build_id): +def tpackage(request, build_id, target_id): template = 'package.html' - packages = Target_Package.objects.filter(build=build_id) + packages = Target_Package.objects.filter(target=target_id) package_depends = Target_Package_Dependency.objects.filter(package__in=packages) - for t in packages: - t.depends_on = [] - for k in package_depends: - if t == k.package: - t.depends_on.append(k.depends_on) - - context = {'build' : build_id ,'packages': packages} + context = {'build' : Build.objects.filter(pk=build_id)[0],'packages': packages} return render(request, template, context)
bitbake/lib/webhob/orm/models.py +5 -1
@@ -1,4 +1,5 @@ from django.db import models +from django.utils.encoding import python_2_unicode_compatible class Build(models.Model):
@@ -27,7 +28,7 @@ class Build(models.Model): build_name = models.CharField(max_length=100) bitbake_version = models.CharField(max_length=50) - +@python_2_unicode_compatible class Target(models.Model): search_allowed_fields = ['target', 'image_fstypes', 'file_name'] build = models.ForeignKey(Build)
@@ -37,6 +38,9 @@ class Target(models.Model): file_name = models.CharField(max_length=100) file_size = models.IntegerField() + def __str__(self): + return self.target + class Task(models.Model):
bitbake/lib/webhob/whbmain/urls.py +1 -1
@@ -10,7 +10,7 @@ urlpatterns = patterns('', url(r'^simple/', include('bldviewer.urls')), url(r'^api/1.0/', include('bldviewer.api')), url(r'^gui/', include('whbgui.urls')), - url(r'^$', redirect_to, {'url': '/gui/'}), + url(r'^$', redirect_to, {'url': '/simple/'}), # Examples: # url(r'^webhob/', include('webhob.foo.urls')),
Signature · 1 line
1.8.1.2
PATCH 75/94 bitbake: dsi: save build-time package information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Saving the build-time package information in the database. Slight additions to the ORM model as to record more information.
--- bitbake/lib/bb/ui/buildinfohelper.py | 20 ++++++++++++++++++++ bitbake/lib/bb/ui/dsi.py | 5 +++++ bitbake/lib/webhob/orm/models.py | 8 ++++++-- 3 files changed, 31 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +20 -0
@@ -156,6 +156,20 @@ class ORMWrapper(object): return log_object.save() + + def save_build_package_information(self, build_obj, package_info, recipes): + bp_object = Build_Package.objects.create( build = build_obj, + recipe = recipes[package_info['PN']], + name = package_info['PKG'], + version = package_info['PKGV'], + revision = package_info['PKGR'], + summary = package_info['SUMMARY'], + description = package_info['DESCRIPTION'], + section = package_info['SECTION'], + license = package_info['LICENSE'], + ) + return bp_object + def save_build_variables(self, build_obj, vardump): for k in vardump: if not bool(vardump[k]['func']):
@@ -558,6 +572,12 @@ class BuildInfoHelper(object): dep = _save_a_task(taskdesc1) Task_Dependency.objects.get_or_create( task = target, depends_on = dep ) + def store_build_package_information(self, event): + package_info = event.data + self.orm_wrapper.save_build_package_information(self.internal_state['build'], + package_info, self.internal_state['recipes']) + + def store_log_information(self, level, text): log_information = {} log_information['build'] = self.internal_state['build']
bitbake/lib/bb/ui/dsi.py +5 -0
@@ -489,6 +489,11 @@ def main(server, eventHandler, params, tf = TerminalFilter): buildinfohelper.store_layer_info() continue + if isinstance(event, bb.event.MetadataEvent): + if event.type == "SinglePackageInfo": + buildinfohelper.store_build_package_information(event) + continue + # ignore if isinstance(event, (bb.event.BuildBase, bb.event.StampUpdate,
bitbake/lib/webhob/orm/models.py +6 -2
@@ -115,8 +115,12 @@ class Build_Package(models.Model): recipe = models.ForeignKey('Recipe', null=True) name = models.CharField(max_length=100) version = models.CharField(max_length=100, default="") - size = models.IntegerField() - license = models.CharField(max_length=200, null=True) + revision = models.CharField(max_length=32, default="") + summary = models.CharField(max_length=200, default="") + description = models.CharField(max_length=200, default="") + size = models.IntegerField(default=0) + section = models.CharField(max_length=80, null=True) + license = models.CharField(max_length=80, null=True) class Build_Package_Dependency(models.Model): TYPE_DEPENDS = 0
Signature · 1 line
1.8.1.2
PATCH 76/94 bitbake: webhob: Simple visualisation for the build-time packages
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding pages and views to allow viewing the data collected about the build-time packages.
--- .../webhob/bldviewer/templates/basebuildpage.html | 17 ++++++++++ .../lib/webhob/bldviewer/templates/bpackage.html | 36 ++++++++++++++++++++++ .../lib/webhob/bldviewer/templates/package.html | 10 ------ bitbake/lib/webhob/bldviewer/views.py | 9 +++++- 4 files changed, 61 insertions(+), 11 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/basebuildpage.html create mode 100644 bitbake/lib/webhob/bldviewer/templates/bpackage.html
bitbake/lib/webhob/bldviewer/templates/basebuildpage.html new file +17 -0
@@ -0,0 +1,17 @@ +{% extends "basetable.html" %} + +{% block pagename %} +<ul class="nav nav-tabs" style="display: inline-block"> + <li><a>Build {{build.target_set.all|join:" "}} at {{build.started_on}} : </a></li> + <li><a href="/simple/build/{{build.id}}/task/"> Tasks </a></li> + <li><a href="/simple/build/{{build.id}}/packages/"> Build Packages </a></li> + {% for t in build.target_set.all %} + {% if t.is_image %} + <li><a href="/simple/build/{{build.id}}/target/{{t.pk}}/packages/"> Packages for {{t.target}} </a> </li> + {% endif %} + {% endfor %} + <li><a href="/simple/build/{{build.id}}/configuration/"> Configuration </a> </li> +</ul> + <h1>Toaster - Build {% block pagetitle %} {% endblock %}</h1> +{% endblock %} +
bitbake/lib/webhob/bldviewer/templates/bpackage.html new file +36 -0
@@ -0,0 +1,36 @@ +{% extends "basebuildpage.html" %} + +{% block pagetable %} + {% if not packages %} + <p>No packages were recorded for this target!</p> + {% else %} + + <tr> + <th>Name</th> + <th>Version</th> + <th>Revision</th> + <th>Recipe</th> + <th>Summary</th> + <th>Section</th> + <th>Description</th> + <th>License</th> + </tr> + + {% for package in packages %} + + <tr class="data"> + <td><a name="#{{package.name}}">{{package.name}}</a></td> + <td>{{package.version}}</td> + <td>{{package.revision}}</td> + <td><a href="/simple/layerversions/{{package.recipe.layer_version_id}}/recipes/#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td> + + <td>{{package.summary}}</td> + <td>{{package.section}}</td> + <td>{{package.description}}</td> + <td>{{package.license}}</td> + + {% endfor %} + + {% endif %} + +{% endblock %}
bitbake/lib/webhob/bldviewer/templates/package.html +0 -10
@@ -1,15 +1,5 @@ {% extends "basebuildpage.html" %} -{% block pagename %} -<ul class="nav nav-tabs" style="display: inline-block"> - <li><a>Build {{build}} : </a></li> - <li><a href="/simple/build/{{build}}/task/"> Tasks </a></li> - <li><a href="/simple/build/{{build}}/package/"> Package </a> </li> - <li><a href="/simple/build/{{build}}/configuration/"> Configuration </a> </li> -</ul> - <h1>Toaster - Target Install Packages</h1> -{% endblock %} - {% block pagetable %} {% if not packages %} <p>No packages were recorded for this target!</p>
bitbake/lib/webhob/bldviewer/views.py +8 -1
@@ -3,7 +3,7 @@ import operator from django.db.models import Q from django.shortcuts import render from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable -from orm.models import Task_Dependency, Target_Package_Dependency +from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package from django.views.decorators.cache import cache_control @cache_control(no_store=True)
@@ -36,6 +36,13 @@ def configuration(request, build_id): context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables} return render(request, template, context) + +def bpackage(request, build_id): + template = 'bpackage.html' + packages = Build_Package.objects.filter(build = build_id) + context = {'build': Build.objects.filter(pk=build_id)[0], 'packages' : packages} + return render(request, template, context) + def tpackage(request, build_id, target_id): template = 'package.html'
Signature · 1 line
1.8.1.2
PATCH 77/94 bitbake: webhob: store file size information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding code to store file size information for each of the build package. Reassembling information from a pair of events for each package - first contains file and size information, and the second contains package metadata.
Cleanup of the database models to hold file size info.
--- bitbake/lib/bb/ui/buildinfohelper.py | 24 +++++++++++++++++++++--- bitbake/lib/bb/ui/dsi.py | 2 ++ bitbake/lib/webhob/orm/models.py | 12 ++++++------ 3 files changed, 29 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +21 -3
@@ -8,7 +8,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage -from webhob.orm.models import Target_Package, Build_Package, Variable +from webhob.orm.models import Target_Package, Build_Package, Variable, Build_File from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency from bb.msg import BBLogFormatter as format
@@ -157,7 +157,8 @@ class ORMWrapper(object): return log_object.save() - def save_build_package_information(self, build_obj, package_info, recipes): + def save_build_package_information(self, build_obj, package_info, recipes, files): + # create and save the object bp_object = Build_Package.objects.create( build = build_obj, recipe = recipes[package_info['PN']], name = package_info['PKG'],
@@ -168,6 +169,14 @@ class ORMWrapper(object): section = package_info['SECTION'], license = package_info['LICENSE'], ) + # save any attached file information + if bp_object.name in files.keys(): + for path, size in files[bp_object.name]: + fo = Build_File.objects.create( bpackage = bp_object, + path = path, + size = size ) + del files[bp_object.name] + return bp_object def save_build_variables(self, build_obj, vardump):
@@ -575,8 +584,17 @@ class BuildInfoHelper(object): def store_build_package_information(self, event): package_info = event.data self.orm_wrapper.save_build_package_information(self.internal_state['build'], - package_info, self.internal_state['recipes']) + package_info, + self.internal_state['recipes'], + self.internal_state['package_files']) + + + def store_package_file_information(self, event): + if not 'package_files' in self.internal_state.keys(): + self.internal_state['package_files'] = {} + data = event._data + self.internal_state['package_files'][data['PKG']] = data['FILES'] def store_log_information(self, level, text): log_information = {}
bitbake/lib/bb/ui/dsi.py +2 -0
@@ -492,6 +492,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): if isinstance(event, bb.event.MetadataEvent): if event.type == "SinglePackageInfo": buildinfohelper.store_build_package_information(event) + elif event.type == "PackageFileSize": + buildinfohelper.store_package_file_information(event) continue # ignore
bitbake/lib/webhob/orm/models.py +6 -6
@@ -157,15 +157,15 @@ class Target_Package_Dependency(models.Model): dep_type = models.IntegerField(choices=DEPENDS_TYPE) -class Build_Filelist(models.Model): +class Build_File(models.Model): bpackage = models.ForeignKey(Build_Package, related_name='filelist_bpackage') - complete_file_path = models.FilePathField(max_length=255, blank=True) - file_size = models.IntegerField() + path = models.FilePathField(max_length=255, blank=True) + size = models.IntegerField() -class Target_Filelist(models.Model): +class Target_File(models.Model): tpackage = models.ForeignKey(Target_Package, related_name='filelist_tpackage') - complete_file_path = models.FilePathField(max_length=255, blank=True) - file_size = models.IntegerField() + path = models.FilePathField(max_length=255, blank=True) + size = models.IntegerField() class Recipe(models.Model):
Signature · 1 line
1.8.1.2
PATCH 78/94 bitbake: webhob: simple visualisation for package files
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding a simple table visualisation for data collected for package content - file names and file sizes.
--- bitbake/lib/webhob/bldviewer/templates/bfile.html | 24 ++++++++++++++++++++++ .../lib/webhob/bldviewer/templates/bpackage.html | 7 +++---- bitbake/lib/webhob/bldviewer/urls.py | 1 + bitbake/lib/webhob/bldviewer/views.py | 9 ++++++-- 4 files changed, 35 insertions(+), 6 deletions(-) create mode 100644 bitbake/lib/webhob/bldviewer/templates/bfile.html
bitbake/lib/webhob/bldviewer/templates/bfile.html new file +24 -0
@@ -0,0 +1,24 @@ +{% extends "basebuildpage.html" %} + +{% block pagetitle %}Files for package {{files.0.bpackage.name}} {% endblock %} +{% block pagetable %} + {% if not files %} + <p>No files were recorded for this package!</p> + {% else %} + + <tr> + <th>Name</th> + <th>Size</th> + </tr> + + {% for file in files %} + + <tr class="data"> + <td>{{file.path}}</td> + <td>{{file.size}}</td> + + {% endfor %} + + {% endif %} + +{% endblock %}
bitbake/lib/webhob/bldviewer/templates/bpackage.html +3 -4
@@ -1,5 +1,6 @@ {% extends "basebuildpage.html" %} +{% block pagetitle %}Packages{% endblock %} {% block pagetable %} {% if not packages %} <p>No packages were recorded for this target!</p>
@@ -8,7 +9,6 @@ <tr> <th>Name</th> <th>Version</th> - <th>Revision</th> <th>Recipe</th> <th>Summary</th> <th>Section</th>
@@ -19,9 +19,8 @@ {% for package in packages %} <tr class="data"> - <td><a name="#{{package.name}}">{{package.name}}</a></td> - <td>{{package.version}}</td> - <td>{{package.revision}}</td> + <td><a name="#{{package.name}}" href="/simple/build/{{build.pk}}/package/{{package.pk}}/files/">{{package.name}} ({{package.filelist_bpackage.count}} files)</a></td> + <td>{{package.version}}-{{package.revision}}</td> <td><a href="/simple/layerversions/{{package.recipe.layer_version_id}}/recipes/#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td> <td>{{package.summary}}</td>
bitbake/lib/webhob/bldviewer/urls.py +1 -0
@@ -5,6 +5,7 @@ urlpatterns = patterns('bldviewer.views', url(r'^builds/$', 'build', name='build'), url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'), url(r'^build/(?P<build_id>\d+)/packages/$', 'bpackage', name='bpackage'), + url(r'^build/(?P<build_id>\d+)/package/(?P<package_id>\d+)/files/$', 'bfile', name='bfile'), url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/packages/$', 'tpackage', name='tpackage'), url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'), url(r'^layers/$', 'layer', name='layer'),
bitbake/lib/webhob/bldviewer/views.py +7 -2
@@ -3,7 +3,7 @@ import operator from django.db.models import Q from django.shortcuts import render from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable -from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package +from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package, Build_File from django.views.decorators.cache import cache_control @cache_control(no_store=True)
@@ -36,13 +36,18 @@ def configuration(request, build_id): context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables} return render(request, template, context) - def bpackage(request, build_id): template = 'bpackage.html' packages = Build_Package.objects.filter(build = build_id) context = {'build': Build.objects.filter(pk=build_id)[0], 'packages' : packages} return render(request, template, context) +def bfile(request, build_id, package_id): + template = 'bfile.html' + files = Build_File.objects.filter(bpackage = package_id) + context = {'build': Build.objects.filter(pk=build_id)[0], 'files' : files} + return render(request, template, context) + def tpackage(request, build_id, target_id): template = 'package.html'
Signature · 1 line
1.8.1.2
PATCH 79/94 bitbake: dsi: record recipe dependency
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Add code to record recipe dependencies, for DEPENDS and RDEPENDS relationships.
Caveat emptor: relationships with ASSUME_PROVIDED
PNs will not be recorded, as they don't have real
recipe objects.
Adds code to the Simple interface to allow visualisation of the recorded data.
--- bitbake/lib/bb/ui/buildinfohelper.py | 23 +++++++++++++++++++++- bitbake/lib/webhob/bldviewer/templates/recipe.html | 9 +++++++++ bitbake/lib/webhob/orm/models.py | 11 +++++++++-- 3 files changed, 40 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +22 -1
@@ -9,7 +9,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage from webhob.orm.models import Target_Package, Build_Package, Variable, Build_File -from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency +from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency, Recipe_Dependency from bb.msg import BBLogFormatter as format class ORMWrapper(object):
@@ -562,6 +562,27 @@ class BuildInfoHelper(object): t.save() self.internal_state['recipes'][pn] = recipe + # save recipe dependency + try: + # buildtime + for recipe in event._depgraph['depends']: + target = self.internal_state['recipes'][recipe] + for dep in event._depgraph['depends'][recipe]: + dependency = self.internal_state['recipes'][dep] + Recipe_Dependency.objects.get_or_create( recipe = target, + depends_on = dependency, dep_type = Recipe_Dependency.TYPE_DEPENDS) + + # runtime + for recipe in event._depgraph['rdepends-pn']: + target = self.internal_state['recipes'][recipe] + for dep in event._depgraph['rdepends-pn'][recipe]: + dependency = self.internal_state['recipes'][dep] + Recipe_Dependency.objects.get_or_create( recipe = target, + depends_on = dependency, dep_type = Recipe_Dependency.TYPE_RDEPENDS) + + except KeyError: # we'll not get recipes for key w/ values listed in ASSUME_PROVIDED + pass + # save all task information def _save_a_task(taskdesc): spec = re.split(r'\.', taskdesc);
bitbake/lib/webhob/bldviewer/templates/recipe.html +9 -0
@@ -22,6 +22,8 @@ <th>Bugtracker</th> <th>Author</th> <th>File_path</th> + <th style="width: 30em">Recipe Dependency</th> + {% for recipe in recipes %}
@@ -36,6 +38,13 @@ <td>{{recipe.bugtracker}}</td> <td>{{recipe.author}}</td> <td>{{recipe.file_path}}</td> + <td> + <div style="height: 5em; overflow:auto"> + {% for rr in recipe.r_dependencies_recipe.all %} + <a href="#{{rr.depends_on.name}}">{{rr.depends_on.name}}</a><br/> + {% endfor %} + </div> + </td> </tr> {% endfor %}
bitbake/lib/webhob/orm/models.py +9 -2
@@ -184,9 +184,16 @@ class Recipe(models.Model): class Recipe_Dependency(models.Model): - recipe = models.ForeignKey(Recipe, related_name='recipe_dependencies_recipe') - depends_on = models.ForeignKey(Recipe, related_name='recipe_dependencies_depends') + TYPE_DEPENDS = 0 + TYPE_RDEPENDS = 1 + DEPENDS_TYPE = ( + (TYPE_DEPENDS, "depends"), + (TYPE_RDEPENDS, "rdepends"), + ) + recipe = models.ForeignKey(Recipe, related_name='r_dependencies_recipe') + depends_on = models.ForeignKey(Recipe, related_name='r_dependencies_depends') + dep_type = models.IntegerField(choices=DEPENDS_TYPE) class Layer(models.Model): name = models.CharField(max_length=100)
Signature · 1 line
1.8.1.2
PATCH 80/94 bitbake: dsi: add exception dumping code
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Dump detailed information about unexpected exceptions during a run. This has effect only when running DSI from command line, since in background mode everything is dumped to /dev/null
--- bitbake/lib/bb/ui/dsi.py | 2 ++ 1 file changed, 2 insertions(+)
bitbake/lib/bb/ui/dsi.py +2 -0
@@ -540,6 +540,8 @@ def main(server, eventHandler, params, tf = TerminalFilter): pass except Exception as e: print(e) + import traceback + traceback.print_exc() pass summary = ""
Signature · 1 line
1.8.1.2
PATCH 81/94 bitbake: webhob: better help message in startup script
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Improved the error during stop message in the startup script as the previous message was confusing to the users.
--- bitbake/bin/webhob | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
bitbake/bin/webhob +8 -1
@@ -104,7 +104,14 @@ if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/whbmain.pid ] ); return 3 elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/whbmain.pid ] ) ; then echo "Error: bitbake lock state error. Trying to stop a stopped system ? -manually stop system with bitbake -m / webserverKill" 2>&1 +If you think the system is hanged up, you can try to manually stop system with the commands + +# BBSERVER=localhost:8200 bitbake -m + +and + +# webserverKillAll +" 2>&1 return 3 fi
Signature · 1 line
1.8.1.2
PATCH 82/94 bitbake: dsi: add build package dependency recording
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Adding code to store all types of package dependencies that a build time package can have.
Added code to visualise the dependency listing in
the Simple interface.
Addded code to record content size for the
build time packages.
--- bitbake/lib/bb/ui/buildinfohelper.py | 33 ++++++++++++++++++++++ .../lib/webhob/bldviewer/templates/bpackage.html | 13 +++++++-- bitbake/lib/webhob/orm/models.py | 16 ++++++++--- 3 files changed, 56 insertions(+), 6 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +33 -0
@@ -166,6 +166,7 @@ class ORMWrapper(object): revision = package_info['PKGR'], summary = package_info['SUMMARY'], description = package_info['DESCRIPTION'], + size = package_info['PKGSIZE'], section = package_info['SECTION'], license = package_info['LICENSE'], )
@@ -177,6 +178,38 @@ class ORMWrapper(object): size = size ) del files[bp_object.name] + # save soft dependency information + if package_info['RDEPENDS'] is not None: + for p in package_info['RDEPENDS'].split(" "): + if not p.startswith("("): + Build_Package_Dependency.objects.get_or_create( package = bp_object, + depends_on = p, dep_type = Build_Package_Dependency.TYPE_RDEPENDS) + if package_info['RDEPENDS'] is not None: + for p in package_info['RPROVIDES'].split(" "): + if not p.startswith("("): + Build_Package_Dependency.objects.get_or_create( package = bp_object, + depends_on = p, dep_type = Build_Package_Dependency.TYPE_RPROVIDES) + if package_info['RDEPENDS'] is not None: + for p in package_info['RRECOMMENDS'].split(" "): + if not p.startswith("("): + Build_Package_Dependency.objects.get_or_create( package = bp_object, + depends_on = p, dep_type = Build_Package_Dependency.TYPE_RRECOMMENDS) + if package_info['RDEPENDS'] is not None: + for p in package_info['RSUGGESTS'].split(" "): + if not p.startswith("("): + Build_Package_Dependency.objects.get_or_create( package = bp_object, + depends_on = p, dep_type = Build_Package_Dependency.TYPE_RSUGGESTS) + if package_info['RDEPENDS'] is not None: + for p in package_info['RREPLACES'].split(" "): + if not p.startswith("("): + Build_Package_Dependency.objects.get_or_create( package = bp_object, + depends_on = p, dep_type = Build_Package_Dependency.TYPE_RREPLACES) + if package_info['RDEPENDS'] is not None: + for p in package_info['RCONFLICTS'].split(" "): + if not p.startswith("("): + Build_Package_Dependency.objects.get_or_create( package = bp_object, + depends_on = p, dep_type = Build_Package_Dependency.TYPE_RCONFLICTS) + return bp_object def save_build_variables(self, build_obj, vardump):
bitbake/lib/webhob/bldviewer/templates/bpackage.html +11 -2
@@ -13,7 +13,9 @@ <th>Summary</th> <th>Section</th> <th>Description</th> + <th>Size</th> <th>License</th> + <th>Dependencies List (all)</th> </tr> {% for package in packages %}
@@ -22,12 +24,19 @@ <td><a name="#{{package.name}}" href="/simple/build/{{build.pk}}/package/{{package.pk}}/files/">{{package.name}} ({{package.filelist_bpackage.count}} files)</a></td> <td>{{package.version}}-{{package.revision}}</td> <td><a href="/simple/layerversions/{{package.recipe.layer_version_id}}/recipes/#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td> - + <td>{{package.summary}}</td> <td>{{package.section}}</td> <td>{{package.description}}</td> + <td>{{package.size}}</td> <td>{{package.license}}</td> - + <td> + <div style="height: 3em"> + {% for bpd in package.bpackage_dependencies_package.all %} + {{bpd.dep_type}}: {{bpd.depends_on}} <br/> + {% endfor %} + </div> + </td> {% endfor %} {% endif %}
bitbake/lib/webhob/orm/models.py +12 -4
@@ -123,14 +123,22 @@ class Build_Package(models.Model): license = models.CharField(max_length=80, null=True) class Build_Package_Dependency(models.Model): - TYPE_DEPENDS = 0 - TYPE_RDEPENDS = 1 + TYPE_RDEPENDS = 0 + TYPE_RPROVIDES = 1 + TYPE_RRECOMMENDS = 2 + TYPE_RSUGGESTS = 3 + TYPE_RREPLACES = 4 + TYPE_RCONFLICTS = 5 DEPENDS_TYPE = ( - (TYPE_DEPENDS, "depends"), (TYPE_RDEPENDS, "rdepends"), + (TYPE_RPROVIDES, "rprovides"), + (TYPE_RRECOMMENDS, "rrecommends"), + (TYPE_RSUGGESTS, "rsuggests"), + (TYPE_RREPLACES, "rreplaces"), + (TYPE_RCONFLICTS, "rconflicts"), ) package = models.ForeignKey(Build_Package, related_name='bpackage_dependencies_package') - depends_on = models.ForeignKey(Build_Package, related_name='bpackage_dependencies_depends') + depends_on = models.CharField(max_length=100) # soft dependency dep_type = models.IntegerField(choices=DEPENDS_TYPE)
Signature · 1 line
1.8.1.2
PATCH 83/94 bitbake: webhob: clean up Machine table
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Removing the Machine table in the favour of a column in the Build table since it complicated the queries without any benefit in reduced data structure.
--- bitbake/lib/bb/ui/buildinfohelper.py | 26 +++++------------------ bitbake/lib/webhob/bldviewer/templates/build.html | 2 +- bitbake/lib/webhob/orm/models.py | 9 ++------ 3 files changed, 8 insertions(+), 29 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +5 -21
@@ -7,7 +7,7 @@ import re os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webhob.whbmain.settings") import webhob.whbmain.settings as whb_django_settings -from webhob.orm.models import Machine, Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage +from webhob.orm.models import Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage from webhob.orm.models import Target_Package, Build_Package, Variable, Build_File from webhob.orm.models import Task_Dependency, Build_Package_Dependency, Target_Package_Dependency, Recipe_Dependency from bb.msg import BBLogFormatter as format
@@ -21,12 +21,6 @@ class ORMWrapper(object): def __init__(self): pass - def create_machine_object(self, machine_information): - - machine = Machine.objects.get_or_create(name=machine_information['name'], - description=machine_information['description']) - - return machine[0] def create_build_object(self, build_info):
@@ -244,13 +238,6 @@ class BuildInfoHelper(object): ################### ## methods to convert event/external info into objects that the ORM layer uses - def _get_machine_information(self): - machine_info = {} - machine_info['name'] = self.server.runCommand(["getVariable", "MACHINE"])[0] - machine_info['description'] = 'Not Available' - self.internal_state['machine'] = machine_info['name'] - return machine_info - def _get_layer_dict(self, layer_path): layer_info = {}
@@ -313,11 +300,11 @@ class BuildInfoHelper(object): return "<unknown>" - def _get_build_information(self, machine_obj): + def _get_build_information(self): build_info = {} # Generate an identifier for each new build - build_info['machine'] = machine_obj + build_info['machine'] = self.server.runCommand(["getVariable", "MACHINE"])[0] build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0] build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0] build_info['started_on'] = datetime.datetime.now()
@@ -375,7 +362,7 @@ class BuildInfoHelper(object): for t in self.internal_state['targets']: target = t.target - machine = self.internal_state['build'].machine.name + machine = self.internal_state['build'].machine buildname = self.internal_state['build'].build_name package = task_object.recipe.name + "-" + task_object.recipe.version.strip(":")
@@ -428,10 +415,7 @@ class BuildInfoHelper(object): def store_started_build(self, event): - machine_information = self._get_machine_information() - machine_obj = self.orm_wrapper.create_machine_object(machine_information) - - build_information = self._get_build_information(machine_obj) + build_information = self._get_build_information() build_obj = self.orm_wrapper.create_build_object(build_information) self.internal_state['build'] = build_obj
bitbake/lib/webhob/bldviewer/templates/build.html +1 -1
@@ -29,7 +29,7 @@ <td>{{build.completed_on}}</td> <td>{% for t in build.target_set.all %}<a href="/simple/build/{{build.id}}/target/{{t.id}}">{{t.target}}</a><br/>{% endfor %}</td> <td>{% if build.is_image %} <a href="/simple/build/{{build.id}}/package/">{{build.is_image}}</a>{% else %} {{build.is_image}} {% endif %}</td> - <td>{{build.machine.name}}</td> + <td>{{build.machine}}</td> <td>{% time_difference build.started_on build.completed_on %}</td> <td>{{build.errors_no}}:{% if build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td> <td>{{build.warnings_no}}:{% if build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
bitbake/lib/webhob/orm/models.py +2 -7
@@ -13,10 +13,10 @@ class Build(models.Model): (IN_PROGRESS, 'In Progress'), ) - search_allowed_fields = ['machine__name', + search_allowed_fields = ['machine', 'cooker_log_path'] - machine = models.ForeignKey('Machine', related_name='build_machine') + machine = models.CharField(max_length=100) distro = models.CharField(max_length=100) distro_version = models.CharField(max_length=100) started_on = models.DateTimeField()
@@ -226,11 +226,6 @@ class Variable(models.Model): description = models.TextField(null=True) -class Machine(models.Model): - name = models.CharField(max_length=100) - description = models.TextField() - - class LogMessage(models.Model): INFO = 0 WARNING = 1
Signature · 1 line
1.8.1.2
PATCH 84/94 bitbake: webhob: update API endpoints
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
Updating the API endpoints as described by the documentation.
The documentation updated here:
https://wiki.yoctoproject.org/wiki/REST_API_Contracts
--- bitbake/lib/webhob/bldviewer/api.py | 20 ++++++++++++++------ bitbake/lib/webhob/bldviewer/views.py | 15 ++++++++++++--- 2 files changed, 26 insertions(+), 9 deletions(-)
bitbake/lib/webhob/bldviewer/api.py +14 -6
@@ -2,10 +2,18 @@ from django.conf.urls import patterns, include, url urlpatterns = patterns('bldviewer.views', - url(r'^builds/$', 'model_explorer', {'model_name':'build'}, name='builds'), - url(r'^tasks/$', 'model_explorer', {'model_name':'task'}, name='task'), - url(r'^packages/$', 'model_explorer', {'model_name':'package'}, name='package'), - url(r'^layers/$', 'model_explorer', {'model_name':'layer'}, name='layer'), - url(r'^recipes/$', 'model_explorer', {'model_name':'recipe'}, name='recipe'), - url(r'^layersversions/$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'), + url(r'^builds$', 'model_explorer', {'model_name':'build'}, name='builds'), + url(r'^targets$', 'model_explorer', {'model_name':'target'}, name='targets'), + url(r'^tasks$', 'model_explorer', {'model_name':'task'}, name='task'), + url(r'^task_dependencies$', 'model_explorer', {'model_name':'task_dependency'}, name='task_dependencies'), + url(r'^packages$', 'model_explorer', {'model_name':'build_package'}, name='build_packages'), + url(r'^package_dependencies$', 'model_explorer', {'model_name':'build_package_dependency'}, name='build_package_dependencies'), + url(r'^target_packages$', 'model_explorer', {'model_name':'target_package'}, name='target_packages'), + url(r'^package_files$', 'model_explorer', {'model_name':'build_file'}, name='build_files'), + url(r'^layers$', 'model_explorer', {'model_name':'layer'}, name='layer'), + url(r'^layerversions$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'), + url(r'^recipes$', 'model_explorer', {'model_name':'recipe'}, name='recipe'), + url(r'^recipe_dependencies$', 'model_explorer', {'model_name':'recipe_dependency'}, name='recipe_dependencies'), + url(r'^variables$', 'model_explorer', {'model_name':'variable'}, name='variables'), + url(r'^logmessages$', 'model_explorer', {'model_name':'logmessage'}, name='logmessages'), )
bitbake/lib/webhob/bldviewer/views.py +12 -3
@@ -2,8 +2,8 @@ import operator from django.db.models import Q from django.shortcuts import render -from orm.models import Build, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable -from orm.models import Task_Dependency, Target_Package_Dependency, Build_Package, Build_File +from orm.models import Build, Target, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable +from orm.models import Task_Dependency, Recipe_Dependency, Build_Package, Build_File, Build_Package_Dependency from django.views.decorators.cache import cache_control @cache_control(no_store=True)
@@ -95,11 +95,20 @@ def model_explorer(request, model_name): response_data = {} model_mapping = { 'build': Build, + 'target': Target, + 'target_package': Target_Package, 'task': Task, - 'package': Target_Package, + 'task_dependency': Task_Dependency, + 'package': Build_Package, 'layer': Layer, 'layerversion': Layer_Version, 'recipe': Recipe, + 'recipe_dependency': Recipe_Dependency, + 'build_package': Build_Package, + 'build_package_dependency': Build_Package_Dependency, + 'build_file': Build_File, + 'variable': Variable, + 'logmessage': LogMessage, } if model_name not in model_mapping.keys():
Signature · 1 line
1.8.1.2
PATCH 85/94 bitbake: webhob: record task hash information
In reply to Alex DAMIANFrom: Alexandru DAMIAN <[email protected]>
We record the task hash for all queue tasks.
Displaying the task hash in the Simple interface.
--- bitbake/lib/bb/ui/buildinfohelper.py | 5 +++++ bitbake/lib/webhob/bldviewer/templates/task.html | 2 ++ 2 files changed, 7 insertions(+)
bitbake/lib/bb/ui/buildinfohelper.py +5 -0
@@ -324,6 +324,11 @@ class BuildInfoHelper(object): task_information['outcome'] = Task.OUTCOME_NA task_information['recipe'] = recipe task_information['task_name'] = event.taskname + try: + # some tasks don't come with a hash. and that's ok + task_information['sstate_checksum'] = event.taskhash + except AttributeError: + pass return task_information def _get_layer_version_for_path(self, path):
bitbake/lib/webhob/bldviewer/templates/task.html +2 -0
@@ -11,6 +11,7 @@ <th>Task</th> <th>Recipe Version</th> <th>Task Type</th> + <th>Checksum</th> <th>Outcome</th> <th>Message</th> <th>Logfile</th>
@@ -36,6 +37,7 @@ <td>Prebuilt</td> {% endif %} + <td>{{task.sstate_checksum}}</td> <td>{{task.get_outcome_display}}</td> <td><p>{{task.message}}</td> <td><a target="_fileview" href="file:///{{task.logfile}}">{{task.logfile}}</a></td>
Signature · 1 line
1.8.1.2