Capture command-line builds which fail due to bad targets
cover Capture command-line builds which fail due to bad targets
Toaster currently can't catch build failures which are due to bad targets (e.g. "bitbake ecky-thump"), as the bitbake run doesn't reach the BuildStarted event in these cases.
Add a new BuildInit event which fires as soon as buildTargets() or buildFile() is invoked, and make the build object at this point. When BuildStarted fires, collect layer information, build variables etc. as previously.
Backwards-compatibility is maintained for jethro by still making a build object on BuildStarted events. jethro builds which fail early will therefore not be captured by the new code, but existing functionality should not be broken either.
To test:
1. Start Toaster.
2. On the command-line, run a build with a bad target like "bitbake ecky-thump".
3. The build should register as a failure on the "all builds" page.
The following changes since commit fb8349b97c736d6633b2d79a4003dd86f760bdb1 (toaster-next):
toaster: bin/toaster Fix noweb command (2016-04-21 14:19:05 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib elliot/toaster/failed_builds-8440-v6
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/failed_builds-8440-v6
Elliot Smith (2):
cooker: add BuildInit event
toasterui: use BuildInit to capture builds with bad targets
bitbake/lib/bb/cooker.py | 6 ++++++ bitbake/lib/bb/event.py | 6 +++++- bitbake/lib/bb/ui/buildinfohelper.py | 17 ++++++++++------- bitbake/lib/bb/ui/toasterui.py | 19 ++++++++----------- 4 files changed, 29 insertions(+), 19 deletions(-)
--
1.9.3
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 1/2 cooker: add BuildInit event
In situations where a bitbake run fails before the build properly starts and BuildStarted is fired, a UI has no way to get at the targets passed to the build. This makes it difficult for the UI to report on the targets which failed.
Fire a BuildInit event before running buildTargets() or buildFile(). This enables a UI to capture targets passed to buildTargets(), even if the build fails (e.g. the targets themselves are invalid).
[YOCTO #8440]
--- bitbake/lib/bb/cooker.py | 6 ++++++ bitbake/lib/bb/event.py | 6 +++++- 2 files changed, 11 insertions(+), 1 deletion(-)
bitbake/lib/bb/cooker.py +6 -0
@@ -1264,6 +1264,7 @@ class BBCooker: """ Build the file matching regexp buildfile """ + bb.event.fire(bb.event.BuildInit(), self.expanded_data) # Too many people use -b because they think it's how you normally # specify a target to be built, so show a warning
@@ -1367,6 +1368,11 @@ class BBCooker: Attempt to build the targets specified """ + # build a list of target:task strings in format + # ['zlib:do_build', 'mpfr-native:do_build', ...] + packages = map(lambda target: "%s:do_%s" % (target, task), targets) + bb.event.fire(bb.event.BuildInit(packages), self.expanded_data) + def buildTargetsIdle(server, rq, abort): msg = None interrupted = 0
bitbake/lib/bb/event.py +5 -1
@@ -373,7 +373,11 @@ class BuildBase(Event): - +class BuildInit(BuildBase): + """buildFile or buildTargets was invoked""" + def __init__(self, p=[]): + name = None + BuildBase.__init__(self, name, p) class BuildStarted(BuildBase, OperationStarted): """bbmake build run started"""
Signature · 12 lines
1.9.3 --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
PATCH 2/2 toasterui: use BuildInit to capture builds with bad targets
In reply to Elliot SmithIf a build has bad targets which cause bitbake to fail before BuildStarted is fired, we are unable to represent this in Toaster, as the build was not detected at all.
Use the BuildInit event so we can capture these failures.
BuildStarted is retained as the event which causes a build object to be created, to support older versions of bitbake which don't have BuildInit.
[YOCTO #8440]
--- bitbake/lib/bb/ui/buildinfohelper.py | 17 ++++++++++------- bitbake/lib/bb/ui/toasterui.py | 19 ++++++++----------- 2 files changed, 18 insertions(+), 18 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +10 -7
@@ -1017,7 +1017,17 @@ class BuildInfoHelper(object): self.internal_state['build'] = build_obj + # create target information + target_information = {} + target_information['targets'] = event._pkgs + target_information['build'] = build_obj + + self.internal_state['targets'] = self.orm_wrapper.get_or_create_targets(target_information) + + def save_build_layers_and_variables(self, event): # save layer version information for this build + build_obj = self.internal_state['build'] + if not 'lvs' in self.internal_state: logger.error("Layer version information not found; Check if the bitbake server was configured to inherit toaster.bbclass.") else:
@@ -1026,13 +1036,6 @@ class BuildInfoHelper(object): del self.internal_state['lvs'] - # create target information - target_information = {} - target_information['targets'] = event._pkgs - target_information['build'] = build_obj - - self.internal_state['targets'] = self.orm_wrapper.get_or_create_targets(target_information) - # Save build configuration data = self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0]
bitbake/lib/bb/ui/toasterui.py +8 -11
@@ -103,6 +103,7 @@ _evt_list = [ "bb.command.CommandFailed", "bb.cooker.CookerExit", "bb.event.BuildCompleted", + "bb.event.BuildInit", "bb.event.BuildStarted", "bb.event.CacheLoadCompleted", "bb.event.CacheLoadProgress",
@@ -231,19 +232,15 @@ def main(server, eventHandler, params): # pylint: disable=protected-access # the code will look into the protected variables of the event; no easy way around this - # we treat ParseStarted as the first event of toaster-triggered - # builds; that way we get the Build Configuration included in the log - # and any errors that occur before BuildStarted is fired - if isinstance(event, bb.event.ParseStarted): + # create a build object in buildinfohelper from either BuildInit (if available) + # or BuildStarted (for jethro and previous versions) + if isinstance(event, (bb.event.BuildStarted, bb.event.BuildInit)): if not (build_log and build_log_file_path): build_log, build_log_file_path = _open_build_log(log_dir) - continue - - if isinstance(event, bb.event.BuildStarted): - if not (build_log and build_log_file_path): - build_log, build_log_file_path = _open_build_log(log_dir) - - buildinfohelper.store_started_build(event, build_log_file_path) + buildinfohelper.store_started_build(event, build_log_file_path) + # get additional data from BuildStarted + if isinstance(event, bb.event.BuildStarted): + buildinfohelper.save_build_layers_and_variables(event) continue if isinstance(event, (bb.build.TaskStarted, bb.build.TaskSucceeded, bb.build.TaskFailedSilent)):
Signature · 12 lines
1.9.3 --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Quoted 31 lines · On 25/04/2016 16:26, "[email protected] on behalf of Elliot…
Toaster currently can't catch build failures which are due to bad targets (e.g.
"bitbake ecky-thump"), as the bitbake run doesn't reach the BuildStarted event in these cases.Add a new BuildInit event which fires as soon as
buildTargets()orbuildFile()
is invoked, and make the build object at this point. When BuildStarted fires,
collect layer information, build variables etc. as previously.Backwards-compatibility is maintained for jethro by still making a build object
on BuildStarted events. jethro builds which fail early will therefore not be
captured by the new code, but existing functionality should not be broken either.To test:
1. Start Toaster.
2. On the command-line, run a build with a bad target like "bitbake ecky-thump".
3. The build should register as a failure on the "all builds" page.The following changes since commit
fb8349b97c736d6633b2d79a4003dd86f760bdb1
(toaster-next):toaster: bin/toaster Fix noweb command (2016-04-21 14:19:05 +0100)
Quoted 7 lines
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib
elliot/toaster/failed_builds-8440-v6http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/fa
iled_builds-8440-v6
This works for me. Thanks!
Belén
Quoted 30 lines
Elliot Smith (2):
cooker: add BuildInit event
toasterui: use BuildInit to capture builds with bad targetsbitbake/lib/bb/cooker.py | 6 ++++++ bitbake/lib/bb/event.py | 6 +++++- bitbake/lib/bb/ui/buildinfohelper.py | 17 ++++++++++------- bitbake/lib/bb/ui/toasterui.py | 19 ++++++++----------- 4 files changed, 29 insertions(+), 19 deletions(-)--
1.9.3---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Signature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Thanks - pushed to toaster-next
Quoted 76 lines · Belen
Quoted 31 lines · On 25/04/2016 16:26, "[email protected] on behalf of Elliot…
Toaster currently can't catch build failures which are due to bad targets (e.g.
"bitbake ecky-thump"), as the bitbake run doesn't reach the BuildStarted event in these cases.Add a new BuildInit event which fires as soon as
buildTargets()orbuildFile()
is invoked, and make the build object at this point. When BuildStarted fires,
collect layer information, build variables etc. as previously.Backwards-compatibility is maintained for jethro by still making a build object
on BuildStarted events. jethro builds which fail early will therefore not be
captured by the new code, but existing functionality should not be broken either.To test:
1. Start Toaster.
2. On the command-line, run a build with a bad target like "bitbake ecky-thump".
3. The build should register as a failure on the "all builds" page.The following changes since commit
fb8349b97c736d6633b2d79a4003dd86f760bdb1
(toaster-next):toaster: bin/toaster Fix noweb command (2016-04-21 14:19:05 +0100)
Quoted 7 lines
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib
elliot/toaster/failed_builds-8440-v6http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/fa
iled_builds-8440-v6This works for me. Thanks!
Belén
Quoted 29 lines
Elliot Smith (2):
cooker: add BuildInit event
toasterui: use BuildInit to capture builds with bad targetsbitbake/lib/bb/cooker.py | 6 ++++++ bitbake/lib/bb/event.py | 6 +++++- bitbake/lib/bb/ui/buildinfohelper.py | 17 ++++++++++------- bitbake/lib/bb/ui/toasterui.py | 19 ++++++++----------- 4 files changed, 29 insertions(+), 19 deletions(-)--
1.9.3---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Signature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster