Skip to contentSkip to conversation

Make toasterui to work in build mode

PATCH Superseded Ed Bartosh 2 participants 8 messages Feb 25, 2016 – Feb 26, 2016

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Ed Bartosh

author
#0 lore raw

cover Make toasterui to work in build mode

Hi,

This patchset is a preparation for the #7880 fix. As we're going to get rid of running bitbake server we need toasterui to work in build mode, i.e. with bitbake <target> -u toasterui.

If you want to test this functionality you can do the following:
- start Toaster
- kill bitbake server and observer processes
- unset BBSERVER
- run bitbake <target> -u toasterui

Expected outcome: you should see commandline build in UI without bitbake server
and observer running.

I've discovered nasty bug in handling setEventMask by bitbake server while working on this. It causes ignoring of the event mask set by toasterui. The result of this is that list of events in _ev_list variable is different from the actual list of events processed by toasterui. Here is the difference:

-bb.command.CommandExit
+bb.event.BuildCompleted
+bb.event.BuildStarted
+bb.event.ConfigParsed
+bb.event.DepTreeGenerated
-bb.runqueue.runQueueExitWait
+bb.event.RecipeParsed
+bb.event.SanityCheck
+bb.event.SanityCheckPassed
+bb.event.TreeDataPreparationCompleted
+bb.event.TreeDataPreparationStarted
+bb.runqueue.runQueueTaskCompleted
+bb.runqueue.runQueueTaskSkipped
+bb.runqueue.sceneQueueTaskCompleted

The fix for this bug is also included in this patchset.

The following changes since commit 029cc9b513864376024cd5fcbb2724e618e92090:

toaster: custom breadcrumb for the default project (2016-02-24 13:29:00 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/toasterui-in-build-mode
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/toasterui-in-build-mode

Ed Bartosh (6):
toasterui: reformat list of events
toasterui: update list of events
command: make setEventMask readonly
toasterui: check if setEventMask succeeded
toasterui: make toasterui to work in build mode
toasterui: exit on final events

 bitbake/lib/bb/command.py      |  1 +
 bitbake/lib/bb/ui/toasterui.py | 76 ++++++++++++++++++++++++++++++++++--------
 2 files changed, 63 insertions(+), 14 deletions(-)

--
Regards,
Ed

author
#1 lore raw

PATCH 1/6 toasterui: reformat list of events

Reformatted and reordered list of events to make changes easily and see them clearly in the diffs.

Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/ui/toasterui.py | 36 +++++++++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 9 deletions(-)
bitbake/lib/bb/ui/toasterui.py +27 -9
index 32b1889..377526e 100644
@@ -92,15 +92,33 @@ def _close_build_log(build_log):
         build_log.close()
         logger.removeHandler(build_log)
 
-_evt_list = [ "bb.runqueue.runQueueExitWait", "bb.event.LogExecTTY", "logging.LogRecord",
-              "bb.build.TaskFailed", "bb.build.TaskBase", "bb.event.ParseStarted",
-              "bb.event.ParseProgress", "bb.event.ParseCompleted", "bb.event.CacheLoadStarted",
-              "bb.event.CacheLoadProgress", "bb.event.CacheLoadCompleted", "bb.command.CommandFailed",
-              "bb.command.CommandExit", "bb.command.CommandCompleted",  "bb.cooker.CookerExit",
-              "bb.event.MultipleProviders", "bb.event.NoProvider", "bb.runqueue.sceneQueueTaskStarted",
-              "bb.runqueue.runQueueTaskStarted", "bb.runqueue.runQueueTaskFailed", "bb.runqueue.sceneQueueTaskFailed",
-              "bb.event.BuildBase", "bb.build.TaskStarted", "bb.build.TaskSucceeded", "bb.build.TaskFailedSilent",
-              "bb.event.MetadataEvent"]
+_evt_list = [
+    "bb.build.TaskBase",
+    "bb.build.TaskFailed",
+    "bb.build.TaskFailedSilent",
+    "bb.build.TaskStarted",
+    "bb.build.TaskSucceeded",
+    "bb.command.CommandCompleted",
+    "bb.command.CommandExit",
+    "bb.command.CommandFailed",
+    "bb.cooker.CookerExit",
+    "bb.event.BuildBase",
+    "bb.event.CacheLoadCompleted",
+    "bb.event.CacheLoadProgress",
+    "bb.event.CacheLoadStarted",
+    "bb.event.LogExecTTY",
+    "bb.event.MetadataEvent",
+    "bb.event.MultipleProviders",
+    "bb.event.NoProvider",
+    "bb.event.ParseCompleted",
+    "bb.event.ParseProgress",
+    "bb.event.ParseStarted",
+    "bb.runqueue.runQueueExitWait",
+    "bb.runqueue.runQueueTaskFailed",
+    "bb.runqueue.runQueueTaskStarted",
+    "bb.runqueue.sceneQueueTaskFailed",
+    "bb.runqueue.sceneQueueTaskStarted",
+    "logging.LogRecord"]
 
 def main(server, eventHandler, params):
     # set to a logging.FileHandler instance when a build starts;
Signature · 1 line
2.1.4
author
#2 lore raw

PATCH 2/6 toasterui: update list of events

In reply to Ed Bartosh

Removed events not used in the code from the list. Added events that are used in the code.

Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/ui/toasterui.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/toasterui.py +12 -2
index 377526e..53b9d92 100644
@@ -99,13 +99,16 @@ _evt_list = [
     "bb.build.TaskStarted",
     "bb.build.TaskSucceeded",
     "bb.command.CommandCompleted",
-    "bb.command.CommandExit",
     "bb.command.CommandFailed",
     "bb.cooker.CookerExit",
     "bb.event.BuildBase",
+    "bb.event.BuildCompleted",
+    "bb.event.BuildStarted",
     "bb.event.CacheLoadCompleted",
     "bb.event.CacheLoadProgress",
     "bb.event.CacheLoadStarted",
+    "bb.event.ConfigParsed",
+    "bb.event.DepTreeGenerated",
     "bb.event.LogExecTTY",
     "bb.event.MetadataEvent",
     "bb.event.MultipleProviders",

@@ -113,9 +116,16 @@ _evt_list = [
     "bb.event.ParseCompleted",
     "bb.event.ParseProgress",
     "bb.event.ParseStarted",
-    "bb.runqueue.runQueueExitWait",
+    "bb.event.RecipeParsed",
+    "bb.event.SanityCheck",
+    "bb.event.SanityCheckPassed",
+    "bb.event.TreeDataPreparationCompleted",
+    "bb.event.TreeDataPreparationStarted",
+    "bb.runqueue.runQueueTaskCompleted",
     "bb.runqueue.runQueueTaskFailed",
+    "bb.runqueue.runQueueTaskSkipped",
     "bb.runqueue.runQueueTaskStarted",
+    "bb.runqueue.sceneQueueTaskCompleted",
     "bb.runqueue.sceneQueueTaskFailed",
     "bb.runqueue.sceneQueueTaskStarted",
     "logging.LogRecord"]
Signature · 1 line
2.1.4
author
#3 lore raw

PATCH 3/6 command: make setEventMask readonly

In reply to Ed Bartosh

Executing setEventMask command when bitbake server is in readonly mode causes runCommand to fail with the following error:
'Not able to execute not readonly commands in readonly mode'

Set readonly attribute for setEventMask command to make it working for Toaster UI. This should not do any harm as this command doesn't influence cooker state.

Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/command.py | 1 +
 1 file changed, 1 insertion(+)
bitbake/lib/bb/command.py +1 -0
index 74106d1..0559ffc 100644
@@ -279,6 +279,7 @@ class CommandsSync:
         mask = params[3]
         return bb.event.set_UIHmask(handlerNum, llevel, debug_domains, mask)
     setEventMask.needconfig = False
+    setEventMask.readonly = True
 
     def setFeatures(self, command, params):
         """
Signature · 1 line
2.1.4
author
#4 lore raw

PATCH 4/6 toasterui: check if setEventMask succeeded

In reply to Ed Bartosh

Currently toasterui ignores return value of setEventMask command, which created confusing difference between set of events set by this command and the real set used in the code.

Checked if setEventMask succeeded. Print error message and exit if it's not.

Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/ui/toasterui.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/toasterui.py +4 -2
index 53b9d92..30e0cf8 100644
@@ -154,8 +154,10 @@ def main(server, eventHandler, params):
     logger.addHandler(console)
     logger.setLevel(logging.INFO)
     llevel, debug_domains = bb.msg.constructLogOptions()
-    server.runCommand(["setEventMask", server.getEventHandle(), llevel, debug_domains, _evt_list])
-
+    result, error = server.runCommand(["setEventMask", server.getEventHandle(), llevel, debug_domains, _evt_list])
+    if not result or error:
+        logger.error("can't set event mask: %s", error)
+        return 1
 
     # verify and warn
     build_history_enabled = True
Signature · 1 line
2.1.4
author
#5 lore raw

PATCH 5/6 toasterui: make toasterui to work in build mode

In reply to Ed Bartosh

Currently toasterui works only in observer mode. This is artificial limitation which was made to support current toaster design. As we decided to stop using bitbake server we'll need to run toasterui also in build mode.

[YOCTO #7880]

Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/ui/toasterui.py | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
bitbake/lib/bb/ui/toasterui.py +17 -2
index 30e0cf8..f8fb211 100644
@@ -168,8 +168,23 @@ def main(server, eventHandler, params):
         build_history_enabled = False
 
     if not params.observe_only:
-        logger.error("ToasterUI can only work in observer mode")
-        return 1
+        params.updateFromServer(server)
+        params.updateToServer(server, os.environ.copy())
+        cmdline = params.parseActions()
+        if not cmdline:
+            print("Nothing to do.  Use 'bitbake world' to build everything, or run 'bitbake --help' for usage information.")
+            return 1
+        if 'msg' in cmdline and cmdline['msg']:
+            logger.error(cmdline['msg'])
+            return 1
+
+        ret, error = server.runCommand(cmdline['action'])
+        if error:
+            logger.error("Command '%s' failed: %s" % (cmdline, error))
+            return 1
+        elif ret != True:
+            logger.error("Command '%s' failed: returned %s" % (cmdline, ret))
+            return 1
 
     # set to 1 when toasterui needs to shut down
     main.shutdown = 0
Signature · 1 line
2.1.4
author
#6 lore raw

PATCH 6/6 toasterui: exit on final events

In reply to Ed Bartosh

Toasterui exits only if bitbake observer shuts down. In build mode it should exit when build is done.

Made toasterui exit on bb.command.CommandCompleted, bb.command.CommandFailed and bb.command.CommandExit events when it's running in build mode.

Signed-off-by Ed Bartosh <[email protected]>
---
 bitbake/lib/bb/ui/toasterui.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
bitbake/lib/bb/ui/toasterui.py +4 -1
index f8fb211..757a89f 100644
@@ -388,7 +388,10 @@ def main(server, eventHandler, params):
             if isinstance(event, (bb.command.CommandCompleted,
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit)):
-                errorcode = 0
+                if params.observe_only:
+                    errorcode = 0
+                else:
+                    main.shutdown = 1
 
                 continue
Signature · 1 line
2.1.4
#7 lore raw
In reply to Ed Bartosh

I saw these errors/warnings when I ran bitbake as you suggested (clean build directory):

WARNING: buildhistory is not enabled. Please enable INHERIT += "buildhistory" to see image details.
ERROR: Layer version information not found; Check if the bitbake server was configured to inherit toaster.bbclass.
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>
WARNING: Could not match layer version for recipe path /home/ell/dev/toaster/poky/meta/ : []

But I only get these if I manually kill bitbake and unset BBSERVER; if I just run Toaster as per usual, it works fine.

I can't see a reason why this shouldn't be merged, then, as it doesn't affect Toaster's normal operation. However, I'd like Brian's opinion before I submit the patches to bitbake-devel.

Elliot

Quoted 76 lines · Ed Bartosh

Hi,

This patchset is a preparation for the #7880 fix. As we're going to get rid of
running bitbake server we need toasterui to work in build mode, i.e. with bitbake <target> -u toasterui.

If you want to test this functionality you can do the following:
- start Toaster
- kill bitbake server and observer processes
- unset BBSERVER
- run bitbake <target> -u toasterui

Expected outcome: you should see commandline build in UI without bitbake server
and observer running.

I've discovered nasty bug in handling setEventMask by bitbake server while working on this. It causes ignoring of the event mask set by toasterui. The result
of this is that list of events in _ev_list variable is different from the actual
list of events processed by toasterui. Here is the difference:

(quoted diff) +12 -2
-bb.command.CommandExit
+bb.event.BuildCompleted
+bb.event.BuildStarted
+bb.event.ConfigParsed
+bb.event.DepTreeGenerated
-bb.runqueue.runQueueExitWait
+bb.event.RecipeParsed
+bb.event.SanityCheck
+bb.event.SanityCheckPassed
+bb.event.TreeDataPreparationCompleted
+bb.event.TreeDataPreparationStarted
+bb.runqueue.runQueueTaskCompleted
+bb.runqueue.runQueueTaskSkipped
+bb.runqueue.sceneQueueTaskCompleted

The fix for this bug is also included in this patchset.

The following changes since commit
029cc9b513864376024cd5fcbb2724e618e92090:

toaster: custom breadcrumb for the default project (2016-02-24 13:29:00
+0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib
ed/toaster/toasterui-in-build-mode

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/toasterui-in-build-mode

Ed Bartosh (6):
toasterui: reformat list of events
toasterui: update list of events
command: make setEventMask readonly
toasterui: check if setEventMask succeeded
toasterui: make toasterui to work in build mode
toasterui: exit on final events

 bitbake/lib/bb/command.py      |  1 +
 bitbake/lib/bb/ui/toasterui.py | 76
++++++++++++++++++++++++++++++++++--------
 2 files changed, 63 insertions(+), 14 deletions(-)

--
Regards,
Ed
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster

Signature · 3 lines
Elliot Smith
Software Engineer
Intel Open Source Technology Centre