Skip to contentSkip to conversation

Fixes for toastergui

PATCH In review Ed Bartosh 3 participants 9 messages Sep 30, 2015 – Oct 6, 2015

lore
Share & tools
lore

Ed Bartosh

author
#0 lore raw

v4 cover Fixes for toastergui

Hi,

This patchset fixes creation of Target objects for command line builds and couple of usability fixes for toastergui module.

Changes in v2: used better name for the API: get_or_create_targets. Changes in v3: made sure 'build' task is not set in Target objects.
rebased pathset on top of poky-contrib:toaster-next
Changes in v4: removed deletion of Target objects that was brought by rebase
rebased on toaster-next

The following changes since commit 4220ebce223fa7cd0865c793b7dd3ffe07b8e130:

toaster: Special case the openembedded-core layer to avoid duplicates (2015-09-30 15:12:40 +0300)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/target-task
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/target-task

Ed Bartosh (6):
event: add new property to BuildBase class
event: add new parameter to Build* event APIs
toaster: add task to the target_information
toaster: add get_or_create_targets API
toaster: ignore ReachableStamps event
toaster: use meaningful logging levels

 bitbake/lib/bb/cooker.py             |  8 ++++----
 bitbake/lib/bb/event.py              | 18 +++++++++++++-----
 bitbake/lib/bb/ui/buildinfohelper.py | 37 ++++++++++++++++++++----------------
 bitbake/lib/bb/ui/toasterui.py       | 11 ++++++-----
 4 files changed, 44 insertions(+), 30 deletions(-)

--
Ed

author
#1 lore raw

PATCH v4 1/6 event: add new property to BuildBase class

Added 'task' property to the base class of BuildStarted and BuildCompleted classes to store bitbake task if it's specified.

This is done as without task Build* events can't fully represent bitbake build. Task information is needed by UI classes to properly show or process Build* events.

---
 bitbake/lib/bb/event.py | 8 ++++++++
 1 file changed, 8 insertions(+)
bitbake/lib/bb/event.py +8 -0
index 366bc41..68496e5 100644
@@ -331,6 +331,7 @@ class BuildBase(Event):
     def __init__(self, n, p, failures = 0):
         self._name = n
         self._pkgs = p
+        self._task = None
         Event.__init__(self)
         self._failures = failures
 
@@ -352,6 +353,12 @@ class BuildBase(Event):
     def setCfg(self, cfg):
         self.data = cfg
 
+    def getTask(self):
+        return self._task
+
+    def setTask(self, task):
+        self._task = task
+
     def getFailures(self):
         """
         Return the number of failed packages
@@ -361,6 +368,7 @@ class BuildBase(Event):
     pkgs = property(getPkgs, setPkgs, None, "pkgs property")
     name = property(getName, setName, None, "name property")
     cfg = property(getCfg, setCfg, None, "cfg property")
+    task = property(getTask, setTask, None, "task property")
 
 
 
Signature · 1 line
2.1.4
author
#2 lore raw

PATCH v4 2/6 event: add new parameter to Build* event APIs

In reply to Ed Bartosh

Added 't'(task) parameter to BuildStarted and BuildCompleted event APIs. Modified existing calls of above mentioned APIs:
passed bitbake task name as a parameter.

---
 bitbake/lib/bb/cooker.py |  8 ++++----
 bitbake/lib/bb/event.py  | 12 ++++++------
 2 files changed, 10 insertions(+), 10 deletions(-)
bitbake/lib/bb/cooker.py +4 -4
index a0d7d59..ef81c3a 100644
@@ -1313,7 +1313,7 @@ class BBCooker:
         taskdata.add_provider(self.data, self.recipecache, item)
 
         buildname = self.data.getVar("BUILDNAME", True)
-        bb.event.fire(bb.event.BuildStarted(buildname, [item]), self.expanded_data)
+        bb.event.fire(bb.event.BuildStarted(buildname, [item], task), self.expanded_data)
 
         # Execute the runqueue
         if not task.startswith("do_"):
@@ -1345,7 +1345,7 @@ class BBCooker:
                 return False
 
             if not retval:
-                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, item, failures, interrupted), self.expanded_data)
+                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, item, task, failures, interrupted), self.expanded_data)
                 self.command.finishAsyncCommand(msg)
                 return False
             if retval is True:
@@ -1381,7 +1381,7 @@ class BBCooker:
                 return False
 
             if not retval:
-                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, targets, failures, interrupted), self.data)
+                bb.event.fire(bb.event.BuildCompleted(len(rq.rqdata.runq_fnid), buildname, targets, task, failures, interrupted), self.data)
                 self.command.finishAsyncCommand(msg)
                 return False
             if retval is True:
@@ -1394,7 +1394,7 @@ class BBCooker:
         taskdata, runlist, fulltargetlist = self.buildTaskData(targets, task, self.configuration.abort)
 
         buildname = self.data.getVar("BUILDNAME", False)
-        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist), self.data)
+        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist, task), self.data)
 
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         if 'universe' in targets:
bitbake/lib/bb/event.py +6 -6
index 68496e5..bf67ee9 100644
@@ -328,10 +328,10 @@ class StampUpdate(Event):
 class BuildBase(Event):
     """Base class for bbmake run events"""
 
-    def __init__(self, n, p, failures = 0):
+    def __init__(self, n, p, t, failures = 0):
         self._name = n
         self._pkgs = p
-        self._task = None
+        self._task = t
         Event.__init__(self)
         self._failures = failures
 
@@ -376,19 +376,19 @@ class BuildBase(Event):
 
 class BuildStarted(BuildBase, OperationStarted):
     """bbmake build run started"""
-    def __init__(self, n, p, failures = 0):
+    def __init__(self, n, p, t, failures = 0):
         OperationStarted.__init__(self, "Building Started")
-        BuildBase.__init__(self, n, p, failures)
+        BuildBase.__init__(self, n, p, t, failures)
 
 class BuildCompleted(BuildBase, OperationCompleted):
     """bbmake build run completed"""
-    def __init__(self, total, n, p, failures=0, interrupted=0):
+    def __init__(self, total, n, p, t, failures=0, interrupted=0):
         if not failures:
             OperationCompleted.__init__(self, total, "Building Succeeded")
         else:
             OperationCompleted.__init__(self, total, "Building Failed")
         self._interrupted = interrupted
-        BuildBase.__init__(self, n, p, failures)
+        BuildBase.__init__(self, n, p, t, failures)
 
 class DiskFull(Event):
     """Disk full case build aborted"""
Signature · 1 line
2.1.4
author
#3 lore raw

PATCH v4 3/6 toaster: add task to the target_information

In reply to Ed Bartosh

Added task information from BuildStarted event to the target_information structure. Made sure 'build' task is not set in Target objects as it shouldn't be shown in Toaster UI.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 6 ++++++
 1 file changed, 6 insertions(+)
bitbake/lib/bb/ui/buildinfohelper.py +6 -0
index 9fdde29..34a8694 100644
@@ -925,6 +925,12 @@ class BuildInfoHelper(object):
         target_information = {}
         target_information['targets'] = event._pkgs
         target_information['build'] = build_obj
+        task = getattr(event, 'task', '')
+        if task.startswith('do_'):
+            task = task[3:]
+        if task == 'build':
+            task = ''
+        target_information['task'] = task
 
         self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
 
Signature · 1 line
2.1.4
author
#4 lore raw

PATCH v4 4/6 toaster: add get_or_create_targets API

In reply to Ed Bartosh

Target objects are created before the build if build is started from UI in build mode. However, in analysis mode Target objects don't exist and need to be created using information from bitbake events.

Added new API call get_or_create_targets to retrive existing target objects or create them if they don't exist yet.

---
 bitbake/lib/bb/ui/buildinfohelper.py | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +14 -15
index 34a8694..793418a 100644
@@ -162,8 +162,6 @@ class ORMWrapper(object):
             build.bitbake_version=build_info['bitbake_version']
             build.save()
 
-            Target.objects.filter(build = build).delete()
-
         else:
             build = Build.objects.create(
                                     project = prj,
@@ -184,18 +182,19 @@ class ORMWrapper(object):
 
         return build
 
-    def create_target_objects(self, target_info):
-        assert 'build' in target_info
-        assert 'targets' in target_info
-
-        targets = []
-        for tgt_name in target_info['targets']:
-            tgt_object = Target.objects.create( build = target_info['build'],
-                                    target = tgt_name,
-                                    is_image = False,
-                                    )
-            targets.append(tgt_object)
-        return targets
+    @staticmethod
+    def get_or_create_targets(target_info):
+        result = []
+        for target in target_info['targets']:
+            obj, created = Target.objects.get_or_create(build=target_info['build'],
+                                                        target=target)
+            if created:
+                obj.is_image = False
+                if target_info['task']:
+                    obj.task = target_info['task']
+                obj.save()
+            result.append(obj)
+        return result
 
     def update_build_object(self, build, errors, warnings, taskfailures):
         assert isinstance(build,Build)
@@ -932,7 +931,7 @@ class BuildInfoHelper(object):
             task = ''
         target_information['task'] = task
 
-        self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
+        self.internal_state['targets'] = self.orm_wrapper.get_or_create_targets(target_information)
 
         # Save build configuration
         data = self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0]
Signature · 1 line
2.1.4
author
#5 lore raw

PATCH v4 5/6 toaster: ignore ReachableStamps event

In reply to Ed Bartosh

Added ReachableStamps event to the list of known events to ignore. This should stop toaster throwing error message:
ERROR: Unknown event: <bb.event.ReachableStamps>

---
 bitbake/lib/bb/ui/toasterui.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/toasterui.py +2 -1
index e0c278b..3891e63 100644
@@ -308,7 +308,8 @@ def main(server, eventHandler, params ):
                                   bb.event.OperationProgress,
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit,
-                                  bb.command.CommandCompleted)):
+                                  bb.command.CommandCompleted,
+                                  bb.event.ReachableStamps)):
                 continue
 
             if isinstance(event, bb.event.DepTreeGenerated):
Signature · 1 line
2.1.4
author
#6 lore raw

PATCH v4 6/6 toaster: use meaningful logging levels

In reply to Ed Bartosh

Changed logging levels to more appropriate ones.

---
 bitbake/lib/bb/ui/toasterui.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
bitbake/lib/bb/ui/toasterui.py +4 -4
index 3891e63..dbe0d09 100644
@@ -130,14 +130,14 @@ def main(server, eventHandler, params ):
 
             if isinstance(event, (bb.build.TaskStarted, bb.build.TaskSucceeded, bb.build.TaskFailedSilent)):
                 buildinfohelper.update_and_store_task(event)
-                logger.warn("Logfile for task %s", event.logfile)
+                logger.info("Logfile for task %s", event.logfile)
                 continue
 
             if isinstance(event, bb.build.TaskBase):
                 logger.info(event._message)
 
             if isinstance(event, bb.event.LogExecTTY):
-                logger.warn(event.msg)
+                logger.info(event.msg)
                 continue
 
             if isinstance(event, logging.LogRecord):
@@ -316,7 +316,7 @@ def main(server, eventHandler, params ):
                 buildinfohelper.store_dependency_information(event)
                 continue
 
-            logger.error("Unknown event: %s", event)
+            logger.warn("Unknown event: %s", event)
             return_value += 1
 
         except EnvironmentError as ioerror:
@@ -336,7 +336,7 @@ def main(server, eventHandler, params ):
             if tb is not None:
                 curr = tb
                 while curr is not None:
-                    logger.warn("Error data dump %s\n%s\n" , traceback.format_tb(curr,1), pformat(curr.tb_frame.f_locals))
+                    logger.error("Error data dump %s\n%s\n" , traceback.format_tb(curr,1), pformat(curr.tb_frame.f_locals))
                     curr = curr.tb_next
 
             # save them to database, if possible; if it fails, we already logged to console.
Signature · 1 line
2.1.4
#7 lore raw
In reply to Ed Bartosh
Quoted 18 lines · On 30/09/2015 14:30, "[email protected] on behalf of Ed…

Hi,

This patchset fixes creation of Target objects for command line builds and couple of usability fixes for toastergui module.

Changes in v2: used better name for the API: get_or_create_targets. Changes in v3: made sure 'build' task is not set in Target objects.
rebased pathset on top of poky-contrib:toaster-next
Changes in v4: removed deletion of Target objects that was brought by rebase
rebased on toaster-next

The following changes since commit
4220ebce223fa7cd0865c793b7dd3ffe07b8e130:

toaster: Special case the openembedded-core layer to avoid duplicates
(2015-09-30 15:12:40 +0300)

Quoted 6 lines

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/target-task

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/target
-task

Now tasks appear in both modes, for builds in progress and builds completed, so the UI looks good to me.

Thanks!

Belén

Quoted 24 lines

Ed Bartosh (6):
event: add new property to BuildBase class
event: add new parameter to Build* event APIs
toaster: add task to the target_information
toaster: add get_or_create_targets API
toaster: ignore ReachableStamps event
toaster: use meaningful logging levels

bitbake/lib/bb/cooker.py             |  8 ++++----
bitbake/lib/bb/event.py              | 18 +++++++++++++-----
bitbake/lib/bb/ui/buildinfohelper.py | 37
++++++++++++++++++++----------------
bitbake/lib/bb/ui/toasterui.py       | 11 ++++++-----
4 files changed, 44 insertions(+), 30 deletions(-)

--
Ed

Signature · 4 lines
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
#8 lore raw
Quoted 6 lines · Belen
Quoted 2 lines · On 30/09/2015 14:30, "[email protected] on behalf of Ed…

This patchset fixes creation of Target objects for command line builds and couple of usability fixes for toastergui module.

I submitted this to bitbake-devel:
http://lists.openembedded.org/pipermail/bitbake-devel/2015-October/006489.html

Elliot

Quoted 60 lines
Quoted 11 lines

Changes in v2: used better name for the API: get_or_create_targets. Changes in v3: made sure 'build' task is not set in Target objects.
rebased pathset on top of poky-contrib:toaster-next
Changes in v4: removed deletion of Target objects that was brought by rebase
rebased on toaster-next

The following changes since commit
4220ebce223fa7cd0865c793b7dd3ffe07b8e130:

Quoted 8 lines

toaster: Special case the openembedded-core layer to avoid duplicates
(2015-09-30 15:12:40 +0300)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/target-task

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/target

Quoted 1 line

-task

Now tasks appear in both modes, for builds in progress and builds completed, so the UI looks good to me.

Thanks!

Belén

Quoted 16 lines

Ed Bartosh (6):
event: add new property to BuildBase class
event: add new parameter to Build* event APIs
toaster: add task to the target_information
toaster: add get_or_create_targets API
toaster: ignore ReachableStamps event
toaster: use meaningful logging levels

bitbake/lib/bb/cooker.py             |  8 ++++----
bitbake/lib/bb/event.py              | 18 +++++++++++++-----
bitbake/lib/bb/ui/buildinfohelper.py | 37
++++++++++++++++++++----------------
bitbake/lib/bb/ui/toasterui.py       | 11 ++++++-----
4 files changed, 44 insertions(+), 30 deletions(-)
Quoted 8 lines

--
Ed

--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster

--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster

Signature · 3 lines
Elliot Smith
Software Engineer
Intel Open Source Technology Centre