Buildhistory fixes
cover Buildhistory fixes
An optimisation for image package size list collection, a fix for image dependency graph filtering, and a minor comment grammar fix.
The following changes since commit d60806e56aed2f62f6a0e030a564f7fdc4a1314d:
build-appliance-image: Exclude DDATETIME from task signature (2016-04-03 15:51:10 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/buildhistory-fixes
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/buildhistory-fixes
Paul Eggleton (3):
classes/buildhistory: optimise getting package size list classes/buildhistory: fix filtering of depends-nokernel.dot classes/buildhistory: fix grammar in comments
meta/classes/buildhistory.bbclass | 17 +++++------------ scripts/oe-pkgdata-util | 26 +++++++++++++++++++++----- 2 files changed, 26 insertions(+), 17 deletions(-)
Signature · 1 line
2.5.5
PATCH 1/3 classes/buildhistory: optimise getting package size list
Invoking oe-pkgdata-util in turn for every package in the list was slow with a large image. Modify oe-pkgdata-util's read-value command to take an option to read the list of packages from a file, as well as prefix the value with the package name; we can then use this to get all of the package sizes at once. This reduces the time to gather this information from minutes to just a second or two.
Fixes [YOCTO #7339].
--- meta/classes/buildhistory.bbclass | 13 +++---------- scripts/oe-pkgdata-util | 26 +++++++++++++++++++++----- 2 files changed, 24 insertions(+), 15 deletions(-)
meta/classes/buildhistory.bbclass +3 -10
@@ -3,7 +3,7 @@ # # Based in part on testlab.bbclass and packagehistory.bbclass # -# Copyright (C) 2011-2014 Intel Corporation +# Copyright (C) 2011-2016 Intel Corporation # Copyright (C) 2007-2011 Koen Kooi <[email protected]> #
@@ -416,15 +416,8 @@ buildhistory_get_installed() { rm $1/depends.tmp # Produce installed package sizes list - printf "" > $1/installed-package-sizes.tmp - cat $pkgcache | while read pkg pkgfile pkgarch - do - size=`oe-pkgdata-util -p ${PKGDATA_DIR} read-value "PKGSIZE" ${pkg}_${pkgarch}` - if [ "$size" != "" ] ; then - echo "$size $pkg" >> $1/installed-package-sizes.tmp - fi - done - cat $1/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > $1/installed-package-sizes.txt + oe-pkgdata-util -p ${PKGDATA_DIR} read-value "PKGSIZE" -n -f $pkgcache > $1/installed-package-sizes.tmp + cat $1/installed-package-sizes.tmp | awk '{print $2 "\tKiB " $1}' | sort -n -r > $1/installed-package-sizes.txt rm $1/installed-package-sizes.tmp # We're now done with the cache, delete it
scripts/oe-pkgdata-util +21 -5
@@ -161,8 +161,18 @@ def glob(args): def read_value(args): # Handle both multiple arguments and multiple values within an arg (old syntax) packages = [] - for pkgitem in args.pkg: - packages.extend(pkgitem.split()) + if args.file: + with open(args.file, 'r') as f: + for line in f: + splitline = line.split() + if splitline: + packages.append(splitline[0]) + else: + for pkgitem in args.pkg: + packages.extend(pkgitem.split()) + if not packages: + logger.error("No packages specified") + sys.exit(1) def readvar(pkgdata_file, valuename): val = ""
@@ -187,9 +197,13 @@ def read_value(args): qvar = "%s_%s" % (args.valuename, mappedpkg) # PKGSIZE is now in bytes, but we we want it in KB pkgsize = (int(readvar(revlink, qvar)) + 1024 // 2) // 1024 - print("%d" % pkgsize) + value = "%d" % pkgsize + else: + value = readvar(revlink, qvar) + if args.prefix_name: + print('%s %s' % (pkg_name, value)) else: - print(readvar(revlink, qvar)) + print(value) def lookup_pkglist(pkgs, pkgdata_dir, reverse): if reverse:
@@ -465,7 +479,9 @@ def main(): help='Read any pkgdata value for one or more packages', description='Reads the named value from the pkgdata files for the specified packages') parser_read_value.add_argument('valuename', help='Name of the value to look up') - parser_read_value.add_argument('pkg', nargs='+', help='Runtime package name to look up') + parser_read_value.add_argument('pkg', nargs='*', help='Runtime package name to look up') + parser_read_value.add_argument('-f', '--file', help='Read package names from the specified file (one per line, first field only)') + parser_read_value.add_argument('-n', '--prefix-name', help='Prefix output with package name', action='store_true') parser_read_value.set_defaults(func=read_value) parser_glob = subparsers.add_parser('glob',
Signature · 1 line
2.5.5
PATCH 2/3 classes/buildhistory: fix filtering of depends-nokernel.dot
In reply to Paul EggletonFor images we produce a number of filtered dependency .dot files for readability, the first of which is depends-nokernel.dot which filters out just the kernel itself (not kernel modules). Unfortunately the filter specifications hadn't been updated for the dash-to-underscore removal or the 4.x kernel upgrade, thus the filtering wasn't actually doing anything.
--- meta/classes/buildhistory.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/buildhistory.bbclass +1 -1
@@ -425,7 +425,7 @@ buildhistory_get_installed() { if [ "$2" != "sdk" ] ; then # Produce some cut-down graphs (for readability) - grep -v kernel_image $1/depends.dot | grep -v kernel-2 | grep -v kernel-3 > $1/depends-nokernel.dot + grep -v kernel-image $1/depends.dot | grep -v kernel-3 | grep -v kernel-4 > $1/depends-nokernel.dot grep -v libc6 $1/depends-nokernel.dot | grep -v libgcc > $1/depends-nokernel-nolibc.dot grep -v update- $1/depends-nokernel-nolibc.dot > $1/depends-nokernel-nolibc-noupdate.dot grep -v kernel-module $1/depends-nokernel-nolibc-noupdate.dot > $1/depends-nokernel-nolibc-noupdate-nomodules.dot
Signature · 1 line
2.5.5
PATCH 3/3 classes/buildhistory: fix grammar in comments
In reply to Paul EggletonFix a minor grammatical error in the comments here.
--- meta/classes/buildhistory.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/buildhistory.bbclass +1 -1
@@ -42,7 +42,7 @@ BUILDHISTORY_COMMIT_AUTHOR ?= "buildhistory <buildhistory@${DISTRO}>" BUILDHISTORY_PUSH_REPO ?= "" SSTATEPOSTINSTFUNCS_append = " buildhistory_emit_pkghistory" -# We want to avoid influence the signatures of sstate tasks - first the function itself: +# We want to avoid influencing the signatures of sstate tasks - first the function itself: sstate_install[vardepsexclude] += "buildhistory_emit_pkghistory" # then the value added to SSTATEPOSTINSTFUNCS: SSTATEPOSTINSTFUNCS[vardepvalueexclude] .= "| buildhistory_emit_pkghistory"
Signature · 1 line
2.5.5