bitbake-layers: print the recipe's depends that crosses a layer boundary
cover bitbake-layers: print the recipe's depends that crosses a layer boundary
The following changes since commit 8d6e55bf2192524bda84138b5356a2791adbbe61:
prserv: add LOCALCOUNT to AUTOINCs migration feature (2013-01-22 16:01:32 +0000)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib robert/bitbake-layers
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/bitbake-layers
Robert Yang (2):
bitbake-layers: fix get_file_layer
bitbake-layers: print the recipe's depends that crosses a layer
boundary
bitbake/bin/bitbake-layers | 136 +++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 135 insertions(+), 1 deletions(-)
PATCH 1/2 bitbake-layers: fix get_file_layer
The get_file_layer can't handle the nested layer correctly, e.g.:
meta-intel/conf/layer.confmeta-intel/meta-cedartrail/conf/layer.conf
They are two layers, the file:
meta-intel/common/recipes-bsp/amt/lms_7.1.20.bb
belongs to meta-intel's layer, but the get_file_layer() may return meta-cedartrail accordig to BBLAYERS' order, since it uses:
for layerdir in self.bblayers:
if regex.match(os.path.join(layerdir, 'test')):
return self.get_layer_name(layerdir)
which doesn't care the filename, re-match the filename would fix the problem.
[YOCTO #3387]
--- bitbake/bin/bitbake-layers | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
bitbake/bin/bitbake-layers +2 -1
@@ -26,6 +26,7 @@ import os import sys import fnmatch from collections import defaultdict +import re bindir = os.path.dirname(__file__) topdir = os.path.dirname(bindir)
@@ -458,7 +459,7 @@ build results (as the layer priority order has effectively changed). for layer, _, regex, _ in self.bbhandler.cooker.status.bbfile_config_priorities: if regex.match(filename): for layerdir in self.bblayers: - if regex.match(os.path.join(layerdir, 'test')): + if regex.match(os.path.join(layerdir, 'test')) and re.match(layerdir, filename): return self.get_layer_name(layerdir) return "?"
Signature · 1 line
1.7.1
PATCH 2/2 bitbake-layers: print the recipe's depends that crosses a layer boundary
In reply to Robert YangFigure out the dependency between recipes that crosses a layer boundary
* Introduction:
- For the .bb file, we need check the following depends:
Quoted 1 line
Check the DEPENDS, RDEPENDS and inherits according to the
cooker_data.
> The cooker_data doesn't have the info about "require/include xxx",
so we need check them manually.
- For the .bbclass, .inc and .conf file, we can't get their file depends from the cooker_data, we need check them manually.
* Usage:
$ bitbake-layer show-cross-depends
I don't like the name "show-cross-depends", figure-out-cross-depends might be better, but most of the commands are "show-xxx" for bitbake-layers.
* Output: (snipped)
/work/poky/meta/recipes-support/libusb/libusb-compat_0.1.4.bb inherits /work/poky/meta-yocto/classes/poky-sanity.bbclass /work/poky/meta-intel/meta-cedartrail/recipes-cdv-media/images/core-image-cdv-media.bb requires /work/poky/meta/recipes-sato/images/core-image-sato.bb /work/poky/meta-yocto/recipes-core/tiny-init/tiny-init.bb RDEPENDS /work/poky/meta/recipes-core/busybox/busybox_1.20.2.bb /work/poky/meta-intel/common/recipes-bsp/amt/lms_7.1.20.bb DEPENDS /work/poky/meta/recipes-devtools/autoconf/autoconf_2.69.bb /work/poky/meta/classes/distrodata.bbclass includes /work/poky/meta-yocto/conf/distro/include/package_regex.inc /work/poky/meta-intel/meta-sys940x/conf/machine/sys940x.conf requires /work/poky/meta/conf/machine/include/tune-atom.inc
[YOCTO #3387]
--- bitbake/bin/bitbake-layers | 133 ++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 133 insertions(+), 0 deletions(-)
bitbake/bin/bitbake-layers +133 -0
@@ -542,6 +542,139 @@ Recipes are listed with the bbappends that apply to them as subitems. notappended.append(basename) return appended, notappended + def do_show_cross_depends(self, args): + """figure out the dependency between recipes that crosses a layer boundary. + +usage: show-cross-depends + +Figure out the dependency between recipes that crosses a layer boundary. + +NOTE: +The .bbappend file can impact the dependency. +""" + self.bbhandler.prepare() + pkg_fn = self.bbhandler.cooker_data.pkg_fn + bbpath = str(self.bbhandler.config_data.getVar('BBPATH', True)) + self.require_re = re.compile(r"require\s+(.+)") + self.include_re = re.compile(r"include\s+(.+)") + self.inherit_re = re.compile(r"inherit\s+(.+)") + + # The bb's DEPENDS and RDEPENDS + for f in pkg_fn: + f = bb.cache.Cache.virtualfn2realfn(f)[0] + # Get the layername that the file is in + layername = self.get_file_layer(f) + + # The DEPENDS + deps = self.bbhandler.cooker_data.deps[f] + for pn in deps: + if pn in self.bbhandler.cooker_data.pkg_pn: + best = bb.providers.findBestProvider(pn, + self.bbhandler.cooker.configuration.data, + self.bbhandler.cooker_data, + self.bbhandler.cooker_data.pkg_pn) + self.check_cross_depends("DEPENDS", layername, f, best[3]) + + # The RDPENDS + all_rdeps = self.bbhandler.cooker_data.rundeps[f].values() + # Remove the duplicated or null one. + sorted_rdeps = {} + # The all_rdeps is the list in list, so we need two for loops + for k1 in all_rdeps: + for k2 in k1: + sorted_rdeps[k2] = 1 + all_rdeps = sorted_rdeps.keys() + for rdep in all_rdeps: + all_p = bb.providers.getRuntimeProviders(self.bbhandler.cooker_data, rdep) + if all_p: + best = bb.providers.filterProvidersRunTime(all_p, rdep, + self.bbhandler.cooker.configuration.data, + self.bbhandler.cooker_data)[0][0] + self.check_cross_depends("RDEPENDS", layername, f, best) + + # The inherit class + cls_re = re.compile('classes/') + if f in self.bbhandler.cooker_data.inherits: + inherits = self.bbhandler.cooker_data.inherits[f] + for cls in inherits: + # The inherits' format is [classes/cls, /path/to/classes/cls] + # ignore the classes/cls. + if not cls_re.match(cls): + inherit_layername = self.get_file_layer(cls) + if inherit_layername != layername: + logger.plain("%s inherits %s" % (f, cls)) + + # The 'require/include xxx' in the bb file + pv_re = re.compile(r"\${PV}") + fnfile = open(f, 'r') + line = fnfile.readline() + while line: + m, keyword = self.match_require_include(line) + # Found the 'require/include xxxx' + if m: + needed_file = m.group(1) + # Replace the ${PV} with the real PV + if pv_re.search(needed_file) and f in self.bbhandler.cooker_data.pkg_pepvpr: + pv = self.bbhandler.cooker_data.pkg_pepvpr[f][1] + needed_file = re.sub(r"\${PV}", pv, needed_file) + self.print_cross_files(bbpath, keyword, layername, f, needed_file) + line = fnfile.readline() + fnfile.close() + + # The "require/include xxx" in conf/machine/*.conf, .inc and .bbclass + conf_re = re.compile(".*/conf/machine/[^\/]*\.conf$") + inc_re = re.compile(".*\.inc$") + # The "inherit xxx" in .bbclass + bbclass_re = re.compile(".*\.bbclass$") + for layerdir in self.bblayers: + layername = self.get_layer_name(layerdir) + for dirpath, dirnames, filenames in os.walk(layerdir): + for name in filenames: + f = os.path.join(dirpath, name) + s = conf_re.match(f) or inc_re.match(f) or bbclass_re.match(f) + if s: + ffile = open(f, 'r') + line = ffile.readline() + while line: + m, keyword = self.match_require_include(line) + # Only bbclass has the "inherit xxx" here. + bbclass="" + if not m and f.endswith(".bbclass"): + m, keyword = self.match_inherit(line) + bbclass=".bbclass" + # Find a 'require/include xxxx' + if m: + self.print_cross_files(bbpath, keyword, layername, f, m.group(1) + bbclass) + line = ffile.readline() + ffile.close() + + def print_cross_files(self, bbpath, keyword, layername, f, needed_filename): + """Print the depends that crosses a layer boundary""" + needed_file = bb.utils.which(bbpath, needed_filename) + if needed_file: + # Which layer is this file from + needed_layername = self.get_file_layer(needed_file) + if needed_layername != layername: + logger.plain("%s %s %s" %(f, keyword, needed_file)) + def match_inherit(self, line): + """Match the inherit xxx line""" + return (self.inherit_re.match(line), "inherits") + + def match_require_include(self, line): + """Match the require/include xxx line""" + m = self.require_re.match(line) + keyword = "requires" + if not m: + m = self.include_re.match(line) + keyword = "includes" + return (m, keyword) + + def check_cross_depends(self, keyword, layername, f, needed_file): + """Print the DEPENDS/RDEPENDS file that crosses a layer boundary""" + best_realfn = bb.cache.Cache.virtualfn2realfn(needed_file)[0] + needed_layername = self.get_file_layer(best_realfn) + if needed_layername != layername: + logger.plain("%s %s %s" % (f, keyword, best_realfn)) if __name__ == '__main__': sys.exit(main(sys.argv[1:]) or 0)
Signature · 1 line
1.7.1
Hi Robert,
Quoted 8 lines · On Friday 25 January 2013 16:35:08 Robert Yang wrote:
The following changes since commit
8d6e55bf2192524bda84138b5356a2791adbbe61:prserv: add LOCALCOUNT to AUTOINCs migration feature (2013-01-22 16:01:32
+0000)are available in the git repository at:
git://git.pokylinux.org/poky-contribrobert/bitbake-layers
Quoted 6 lines
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/bitbake-layers
Robert Yang (2):
bitbake-layers: fixget_file_layer
bitbake-layers: print the recipe's depends that crosses a layer
boundary
Looks good, thanks for implementing this. I think we might be able to improve the output formatting for this subcommand in a similar manner to the other subcommands a while ago (i.e. avoiding long paths but with an option to go back to the full path format - see show-overlayed and its -f option for an example) however this can be done at a later date.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Hi Pual,
Thank you very much, I've filed a defect for tracking this:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=3824
Bug 3824 - bitbake-layers: let show-cross-depends avoid long path
// Robert
Quoted 27 lines · Paul Eggleton
Hi Robert,
Quoted 8 lines · On Friday 25 January 2013 16:35:08 Robert Yang wrote:
The following changes since commit
8d6e55bf2192524bda84138b5356a2791adbbe61:prserv: add LOCALCOUNT to AUTOINCs migration feature (2013-01-22 16:01:32
+0000)are available in the git repository at:
git://git.pokylinux.org/poky-contribrobert/bitbake-layersQuoted 6 lines
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/bitbake-layers
Robert Yang (2):
bitbake-layers: fixget_file_layer
bitbake-layers: print the recipe's depends that crosses a layer
boundaryLooks good, thanks for implementing this. I think we might be able to improve the output formatting for this subcommand in a similar manner to the other subcommands a while ago (i.e. avoiding long paths but with an option to go back to the full path format - see show-overlayed and its -f option for an example) however this can be done at a later date.
Cheers,
Paul