Webhob: helper based on webservice
cover Webhob: helper based on webservice
This helper is a bitbake ui, based on soaplib 1.0 lib to implement a webservice interface.
Using 'bitbake -u webhob_webserivce ip:port' to startup this help.
I defined 2 webservice methods, they are runCommand(param) and getEvent().
Different languages can remotely call the 2 methods to communicate with bitbake. such as php, java, js, python and so on.
1. runCommand(param) method is responsible for sending commands to bitbake server.
the param format is:
{
'function' : string (required, function name)
'param_type': array(string) (optional, params types, values only are 'string', 'list' or 'bool')
'params' : array(string) (optional, string param corresponds to itself,
list type corresponds to 'str1 str2 str3 ....', bool type corresponds to 'true' or 'false'
)
}
some examples:
1). to call bitbake server.runCommand(["initCooker"])
param style is:
{
'function': 'initCokker'
}
2). to call bitbake server.runCommand(["getVariable", "BBLAYERS"])
param style is:
{
'function' : 'getVariable'
'param_type' : ['string']
'params' : ['BBLASYERS']
}
3). to call bitbake server.runCommand(["buildTargets", list(recipe_queue), 'build'])
param style is:
{
'function' : 'buildTargets'
'param_type' : ['list','string']
'params' : ['recipe1 recipe2 recipe2 recipe4 ....', 'build']
}
2. getEvent() is respinsible for getting the event results generated by bitbake event objects, it returns a standard json format data. some examples:
1) to get some async configurations:
[{'event': 'ConfigFilesFound',
'values': 'poky-bleeding poky-lsb poky poky-tiny defaultsetup',
'variable': 'distro'},
{'event': 'CommandCompleted'}
]
[{'event': 'ConfigFilesFound',
'values': 'atom-pc routerstationpro mpc8315e-rdb beagleboard qemumips qemuarm qemux86-64 qemux86 qemuppc',
'variable': 'machine'},
{'event': 'CommandCompleted'}
'values': 'i586 x86_64 i686',
'variable': 'machine-sdk'},
{'event': 'CommandCompleted'}
]
[{'event': 'FilesMatchingFound',
'matches': 'rootfs_rpm.bbclass rootfs_deb.bbclass rootfs_ipk.bbclass',
'pattern': 'rootfs_'},
{'event': 'CommandCompleted'}
]
2) some recipe progress values
[{'current': 24,
'event': 'TreeDataPreparationProgress',
'msg': 'Preparing tree data: 24/1105',
'total': 1105},
{'current': 25,
'event': 'TreeDataPreparationProgress',
'msg': 'Preparing tree data: 25/1105',
'total': 1105},
{'current': 26,
'event': 'TreeDataPreparationProgress',
'msg': 'Preparing tree data: 26/1105',
'total': 1105},
{'current': 27,
'event': 'TreeDataPreparationProgress',
'msg': 'Preparing tree data: 27/1105',
'total': 1105},
{'current': 28,
'event': 'TreeDataPreparationProgress',
'msg': 'Preparing tree data: 28/1105',
'total': 1105},
{'current': 29,
'event': 'TreeDataPreparationProgress',
'msg': 'Preparing tree data: 29/1105',
'total': 1105}]
3) some task building values
[{'event': 'TaskSucceeded',
'message': 'package libpcre-native-8.21-r1: task do_populate_sysroot: Succeeded',
'package': 'libpcre-native-8.21-r1',
'pid': 30382,
'task': 'do_populate_sysroot'},
{'event': 'runQueueTaskCompleted',
'pid': 0,
'taskid': 753,
'taskstring': 'virtual:native:/home/xiaotong/workspace/python/poky-contrib/meta/recipes-support/libpcre/libpcre_8.21.bb, do_populate_sysroot'},
{'event': 'runQueueTaskStarted',
'noexec': False,
'pid': 0,
'stats': {'active': 0, 'completed': 324, 'failed': 0},
'taskid': 709,
'taskstring': 'virtual:native:/home/xiaotong/workspace/python/poky-contrib/meta/recipes-support/db/db_5.1.19.bb, do_populate_sysroot'},
{'event': 'TaskStarted',
'message': 'package db-native-5.1.19-r4: task do_populate_sysroot: Started',
'package': 'db-native-5.1.19-r4',
'pid': 30447,The following changes since commit 3bf8069100e54153d13319dc32ca089327b6a139:
documentation/yocto-project-qs/yocto-project-qs.xml: added quotes (2012-05-01 21:00:36 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib xtlv/webhob-webservice
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=xtlv/webhob-webservice
Xiaotong lv (1):
Webhob: a general helper interface based on webservice
bitbake/lib/bb/ui/crumbs/webserviceeventhandler.py | 190 ++++++++++++++++++++ bitbake/lib/bb/ui/webhob_webservice.py | 141 +++++++++++++++ 2 files changed, 331 insertions(+), 0 deletions(-) create mode 100644 bitbake/lib/bb/ui/crumbs/webserviceeventhandler.py create mode 100755 bitbake/lib/bb/ui/webhob_webservice.py
Signature · 1 line
1.7.4.4
PATCH 1/1 Webhob: a general helper interface based on webservice
using command: 'bitbake -u webhob_webservice' to start webservice helper. defining 2 methods of webservice , they are runCommand(param) and getEvent(), then different languages can use webservice client to remote call them. runCommand(params) method to send command to bitbake server. params format is:
{
'function':string
'param_type':array(string)
'params':array(string)
}getEvent() method handles bitbake object event then returns a general json format data.
--- bitbake/lib/bb/ui/crumbs/webserviceeventhandler.py | 190 ++++++++++++++++++++ bitbake/lib/bb/ui/webhob_webservice.py | 141 +++++++++++++++ 2 files changed, 331 insertions(+), 0 deletions(-) create mode 100644 bitbake/lib/bb/ui/crumbs/webserviceeventhandler.py create mode 100755 bitbake/lib/bb/ui/webhob_webservice.py
bitbake/lib/bb/ui/crumbs/webserviceeventhandler.py new file +190 -0
@@ -0,0 +1,190 @@ +#!/usr/bin/env python +# +# Copyright (C) 2011 Intel Corporation +# +# Authored by Lv Xiaotong <[email protected]> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +import logging + +class WSEventHandler: + def __init__ (self): + self.ret_value = {} + + def clear_ret_value(self): + if self.ret_value: + self.ret_value = {} + + #To convert the list in recipe and packages tree model into string type. + #so that the tree model can be converted into standard json data + def treemodel_list_tostring(self, data): + for k, v in data.iteritems(): + if isinstance(v, dict): + data[k] = self.treemodel_list_tostring(v) + elif isinstance(v, list): + data[k] = ' '.join(v) + else: + data[k] = v + return data + + def handle_event(self, event): + if not event: + return + + self.clear_ret_value() + + if isinstance(event, bb.event.PackageInfo): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["pkginfolist"] = self.treemodel_list_tostring(event._pkginfolist) + elif isinstance(event, bb.event.SanityCheckPassed): + self.ret_value["event"] = bb.event.getName(event) + + #Handler these logs, do we need to do some logging filtering under logging.INFO level. + #cause maybe some loggings are unuseful but this can affect the webservice efficiency. + #and client user experience + elif isinstance(event, logging.LogRecord): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["msg"] = event.msg + self.ret_value["getMessage"] = event.getMessage() + self.ret_value["logging_error"] = logging.ERROR + self.ret_value["logging_warning"] = logging.WARNING + self.ret_value["levelno"] = event.levelno + + elif isinstance(event, bb.event.TargetsTreeGenerated): + self.ret_value["event"] = bb.event.getName(event) + if event._model: + self.ret_value["model"] = self.treemodel_list_tostring(event._model) + + elif isinstance(event, bb.event.ConfigFilesFound): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["variable"] = event._variable + self.ret_value["values"] = ' '.join(event._values) + + elif isinstance(event, bb.event.ConfigFilePathFound): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["path"] = event._path + + elif isinstance(event, bb.event.FilesMatchingFound): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["pattern"] = event._pattern + self.ret_value["matches"] = ' '.join(event._matches) + + elif isinstance(event, bb.command.CommandCompleted): + self.ret_value["event"] = bb.event.getName(event) + + elif isinstance(event, bb.command.CommandFailed): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["exitcode"] = event.exitcode + self.ret_value["error"] = event.error + + elif isinstance(event, (bb.event.ParseStarted, + bb.event.CacheLoadStarted, + bb.event.TreeDataPreparationStarted) + ): + self.ret_value["event"] = bb.event.getName(event) + if self.ret_value["event"] != 'TreeDataPreparationStarted': + self.ret_value["total"] = event.total + + elif isinstance(event, (bb.event.ParseProgress, + bb.event.CacheLoadProgress, + bb.event.TreeDataPreparationProgress) + ): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["current"] = event.current + self.ret_value["total"] = event.total + self.ret_value["msg"] = event.msg + + elif isinstance(event, (bb.event.ParseCompleted, + bb.event.CacheLoadCompleted, + bb.event.TreeDataPreparationCompleted) + ): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["total"] = event.total + self.ret_value["msg"] = event.msg + + elif isinstance(event, bb.event.NoProvider): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["dependees"] = ' '.join(event._dependees) + self.ret_value["reasons"] = ' '.join(event._reasons) + self.ret_value["runtime"] = event._runtime + self.ret_value["item"] = event._item + + elif isinstance(event, bb.event.MultipleProviders): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["candidates"] = ' '.join(event._candidates) + self.ret_value["runtime"] = event._runtime + self.ret_value["item"] = event._item + + elif isinstance(event, bb.event.BuildStarted): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["name"] = event._name + self.ret_value["msg"] = event.msg + self.ret_value["failures"] = event._failures + self.ret_value["pkgs"] = ' '.join(event._pkgs) + self.ret_value["pid"] = event.pid + + elif isinstance(event, bb.event.BuildCompleted): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["name"] = event._name + self.ret_value["msg"] = event.msg + self.ret_value["failures"] = event._failures + self.ret_value["total"] = event.total + self.ret_value["pkgs"] = ' '.join(event._pkgs) + self.ret_value["pid"] = event.pid + + elif isinstance(event, (bb.build.TaskBase, + bb.build.TaskStarted, + bb.build.TaskSucceeded) + ): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["task"] = event._task + self.ret_value["message"] = event._message + self.ret_value["package"] = event._package + self.ret_value["pid"] = event.pid + + if isinstance(event, bb.build.TaskFailed): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["logfile"] = event.logfile + self.ret_value["errprinted"] = event.errprinted + + elif isinstance(event, (bb.runqueue.runQueueTaskStarted, + bb.runqueue.sceneQueueTaskStarted) + ): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["noexec"] = event.noexec + self.ret_value["taskstring"] = event.taskstring + self.ret_value["taskid"] = event.taskid + self.ret_value["stats"] = {'completed':event.stats.completed, + 'active':event.stats.active, + 'failed':event.stats.failed + } + self.ret_value["pid"] = event.pid + + elif isinstance(event, bb.runqueue.runQueueTaskCompleted): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["taskstring"] = event.taskstring + self.ret_value["pid"] = event.pid + self.ret_value["taskid"] = event.taskid + + elif isinstance(event, (bb.runqueue.runQueueTaskFailed, + bb.runqueue.sceneQueueTaskFailed) + ): + self.ret_value["event"] = bb.event.getName(event) + self.ret_value["taskid"] = event.taskid + self.ret_value["exitcode"] = event.exitcode + self.ret_value["taskstring"] = event.taskstring + self.ret_value["pid"] = event.pid + + return self.ret_value
bitbake/lib/bb/ui/webhob_webservice.py new file +141 -0
141 lines not on this page · raw message
Signature · 1 line
1.7.4.4
Is there any feedback for this piece of helper code?
--
Shane
Xiaotong lv wrote on 2012-06-07:
Quoted 115 lines
This helper is a bitbake ui, based on soaplib 1.0 lib to implement a webservice interface. Using 'bitbake -u
webhob_webserivceip:port' to startup this help. I defined 2 webservice methods, they are runCommand(param) andgetEvent(). Different languages can remotely call the 2 methods to communicate with bitbake. such as php, java, js, python and so on. 1. runCommand(param) method is responsible for sending commands to bitbake server.
the param format is:
{
'function' : string (required, function name)
'param_type': array(string) (optional, params types, values
only are 'string', 'list' or 'bool') 'params' : array(string)
(optional, string param
corresponds to itself,
list type corresponds to 'str1 str2 str3 ....', bool type
corresponds to 'true' or 'false') } some examples: 1). to call bitbake server.runCommand(["initCooker"]) param style is: { 'function': 'initCokker' } 2). to call bitbake server.runCommand(["getVariable", "BBLAYERS"]) param style is: { 'function' : 'getVariable' 'param_type' : ['string'] 'params' : ['BBLASYERS'] } 3). to call bitbake server.runCommand(["buildTargets", list(recipe_queue), 'build']) param style is: { 'function' : 'buildTargets' 'param_type' : ['list','string'] 'params' : ['recipe1 recipe2 recipe2 recipe4 ....', 'build'] } 2. getEvent() is respinsible for getting the event results generated by bitbake event objects, it returns a standard json format data. some examples: 1) to get some async configurations: [{'event': 'ConfigFilesFound', 'values': 'poky-bleeding poky-lsb poky poky-tiny defaultsetup', 'variable': 'distro'}, {'event': 'CommandCompleted'} ] [{'event': 'ConfigFilesFound', 'values': 'atom-pc routerstationpro mpc8315e-rdb beagleboard qemumips qemuarm qemux86-64 qemux86 qemuppc', 'variable': 'machine'}, {'event': 'CommandCompleted'} 'values': 'i586 x86_64 i686', 'variable': 'machine-sdk'}, {'event': 'CommandCompleted'} ] [{'event': 'FilesMatchingFound', 'matches': 'rootfs_rpm.bbclass rootfs_deb.bbclass rootfs_ipk.bbclass', 'pattern': 'rootfs_'}, {'event': 'CommandCompleted'} ] 2) some recipe progress values [{'current': 24, 'event': 'TreeDataPreparationProgress', 'msg': 'Preparing tree data: 24/1105', 'total': 1105}, {'current': 25, 'event': 'TreeDataPreparationProgress', 'msg': 'Preparing tree data: 25/1105', 'total': 1105}, {'current': 26, 'event': 'TreeDataPreparationProgress', 'msg': 'Preparing tree data: 26/1105', 'total': 1105}, {'current': 27, 'event': 'TreeDataPreparationProgress', 'msg': 'Preparing tree data: 27/1105', 'total': 1105}, {'current': 28, 'event': 'TreeDataPreparationProgress', 'msg': 'Preparing tree data: 28/1105', 'total': 1105}, {'current': 29, 'event': 'TreeDataPreparationProgress', 'msg': 'Preparing tree data: 29/1105', 'total': 1105}] 3) some task building values [{'event': 'TaskSucceeded', 'message': 'package libpcre-native-8.21-r1: task do_populate_sysroot: Succeeded', 'package': 'libpcre-native-8.21-r1', 'pid': 30382, 'task': 'do_populate_sysroot'}, {'event': 'runQueueTaskCompleted', 'pid': 0, 'taskid': 753, 'taskstring': 'virtual:native:/home/xiaotong/workspace/python/poky-contrib/meta/recipes -support/libpcre/libpcre_8.21.bb, do_populate_sysroot'}, {'event': 'runQueueTaskStarted', 'noexec': False, 'pid': 0, 'stats': {'active': 0, 'completed': 324, 'failed': 0}, 'taskid': 709, 'taskstring': 'virtual:native:/home/xiaotong/workspace/python/poky-contrib/meta/recipes -support/db/db_5.1.19.bb, do_populate_sysroot'}, {'event': 'TaskStarted', 'message': 'package db-native-5.1.19-r4: task do_populate_sysroot: Started', 'package': 'db-native-5.1.19-r4', 'pid': 30447,The following changes since commit
3bf8069100e54153d13319dc32ca089327b6a139:
documentation/yocto-project-qs/yocto-project-qs.xml: added quotes
(2012-05-01 21:00:36 +0100)are available in the git repository at:
git://git.yoctoproject.org/poky-contribxtlv/webhob-webservice
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=xtlv/webhob-webs
erv iceXiaotong lv (1):
Webhob: a general helper interface based on webservicebitbake/lib/bb/ui/crumbs/webserviceeventhandler.py | 190++++++++++++++++++++
bitbake/lib/bb/ui/webhob_webservice.py| 141 +++++++++++++++ 2 files changed, 331 insertions(+), 0 deletions(-) create mode 100644bitbake/lib/bb/ui/crumbs/webserviceeventhandler.pycreate mode 100755bitbake/lib/bb/ui/webhob_webservice.py