icecc improvements
cover icecc improvements
Fix for icecc inherit causing allarch and native recipes to have different signatures for different MACHINES.
Make blacklist more strict and start with empty one - old entries seem all to be resolved already.
The following changes since commit 8163854adf87ac42a8f08ee25685d0ce1efb4724:
oe-selftest: separated the SStateBase and SStateTests in different modules (2014-01-16 12:18:44 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib jansa/icecc
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=jansa/icecc
Martin Jansa (4):
icecc: Don't replace non-empty PARALLEL_MAKE with empty
ICECC_PARALLEL_MAKE
icecc: use bb.utils.which also for 'as'
icecc: Fix allarch and native recipes having different signatures
icecc: use exact match in blacklists, re-start with empty
system_package_blacklist
meta/classes/icecc.bbclass | 46 +++++++++++++++++++++++++++++----------------- 1 file changed, 29 insertions(+), 17 deletions(-)
Signature · 1 line
1.8.5.2
PATCH 1/4 icecc: Don't replace non-empty PARALLEL_MAKE with empty ICECC_PARALLEL_MAKE
* it's needed for use-case like this:
# Inherit icecc here, so that all builders have the same sstate signatures INHERIT_DISTRO += "icecc"
# and then disable its function by default (so that people still need to explicity
# enable it in local.conf if they have configured icecc and want to use it.
# You need to set _empty_ value in local.conf to enable icecc function:
# ICECC_DISABLED = ""
ICECC_DISABLED ??= "1"
* so default ICECC_PARALLEL_MAKE is still empty, but we want build
to respect our PARALLEL_MAKE, unfortunately we cannot do something
like ICECC_PARALLEL_MAKE ??= "${PARALLEL_MAKE}", because that would
cause PARALLEL_MAKE to reference itself.--- meta/classes/icecc.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/icecc.bbclass +1 -1
@@ -147,7 +147,7 @@ def icc_version(bb, d): return "" parallel = d.getVar('ICECC_PARALLEL_MAKE') or "" - if not d.getVar('PARALLEL_MAKE') == "": + if not d.getVar('PARALLEL_MAKE') == "" and parallel: d.setVar("PARALLEL_MAKE", parallel) if icc_is_native(bb, d):
Signature · 1 line
1.8.5.2
PATCH 2/4 icecc: use bb.utils.which also for 'as'
In reply to Martin Jansa* it was introduced in
commit 3a842ec52e7d010767b13bdcb5629ac07b3ee9e7
Author: Dmitry Eremin-Solenikov <[email protected]>
Date: Fri Sep 16 10:55:16 2011 +0400
Subject: icecc.bbclass: replace with updated version
without any explanation in which case
${ICECC_CC} -print-prog-name=asis returning as in current working directory, but will keep old behavior just in case
--- meta/classes/icecc.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
meta/classes/icecc.bbclass +6 -1
@@ -251,6 +251,8 @@ set_icecc_env() { ICECC_CC="${@icc_get_and_check_tool(bb, d, "gcc")}" ICECC_CXX="${@icc_get_and_check_tool(bb, d, "g++")}" + # cannot use icc_get_and_check_tool here because it assumes as without target_sys prefix + ICECC_WHICH_AS="${@bb.utils.which(os.getenv('PATH'), 'as')}" if [ ! -x "${ICECC_CC}" -o ! -x "${ICECC_CXX}" ] then bbwarn "Cannot use icecc: could not get ICECC_CC or ICECC_CXX"
@@ -266,9 +268,12 @@ set_icecc_env() { fi ICECC_AS="`${ICECC_CC} -print-prog-name=as`" + # for target recipes should return something like: + # /OE/tmp-eglibc/sysroots/x86_64-linux/usr/libexec/arm920tt-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/4.8.2/as + # and just "as" for native, if it returns "as" in current directory (for whatever reason) use "as" from PATH if [ "`dirname "${ICECC_AS}"`" = "." ] then - ICECC_AS="`which as`" + ICECC_AS="${ICECC_WHICH_AS}" fi if [ ! -f "${ICECC_VERSION}.done" ]
Signature · 1 line
1.8.5.2
PATCH 3/4 icecc: Fix allarch and native recipes having different signatures
In reply to Martin Jansa* for different MACHINES
* is there more elegant way to have "overridable" function so that signature handler properly uses only the branch without STAGING_BINDIR_TOOLCHAIN?
--- meta/classes/icecc.bbclass | 4 ++++ 1 file changed, 4 insertions(+)
meta/classes/icecc.bbclass +4 -0
@@ -142,6 +142,8 @@ def icc_is_native(bb, d): bb.data.inherits_class("cross", d) or \ bb.data.inherits_class("native", d); +# Don't pollute allarch signatures with TARGET_FPU +icc_version[vardepsexclude] += "TARGET_FPU" def icc_version(bb, d): if use_icc(bb, d) == "no": return ""
@@ -182,6 +184,8 @@ def icc_get_external_tool(bb, d, tool): target_prefix = d.expand('${TARGET_PREFIX}') return os.path.join(external_toolchain_bindir, '%s%s' % (target_prefix, tool)) +# Don't pollute native signatures with target TUNE_PKGARCH through STAGING_BINDIR_TOOLCHAIN +icc_get_tool[vardepsexclude] += "STAGING_BINDIR_TOOLCHAIN" def icc_get_tool(bb, d, tool): if icc_is_native(bb, d): return bb.utils.which(os.getenv("PATH"), tool)
Signature · 1 line
1.8.5.2
PATCH 4/4 icecc: use exact match in blacklists, re-start with empty system_package_blacklist
In reply to Martin Jansa* unify debug messages a bit
* old implementation allowed partial match in blacklist, it's safer to explicitly list exact matches
* I was able to build all entries from system_package_blacklist with icecc enabled, lets assume that they were already resolved by newer versions (we've fixed a lot of parallel issues in recipes which were detected even without icecc and this list is very old).
--- meta/classes/icecc.bbclass | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-)
meta/classes/icecc.bbclass +18 -15
@@ -95,36 +95,39 @@ def use_icc(bb,d): if icc_is_allarch(bb, d): return "no" - package_tmp = d.expand('${PN}') + pn = d.getVar('PN', True) - system_class_blacklist = [ "none" ] + system_class_blacklist = [] user_class_blacklist = (d.getVar('ICECC_USER_CLASS_BL') or "none").split() package_class_blacklist = system_class_blacklist + user_class_blacklist for black in package_class_blacklist: if bb.data.inherits_class(black, d): - #bb.note(package_tmp, ' class ', black, ' found in blacklist, disable icecc') + bb.debug(1, "%s: class %s found in blacklist, disable icecc" % (pn, black)) return "no" - #"system" package blacklist contains a list of packages that can not distribute compile tasks - #for one reason or the other - system_package_blacklist = [ "uclibc", "glibc", "gcc", "bind", "u-boot", "dhcp-forwarder", "enchant", "connman", "orbit2" ] + # "system" recipe blacklist contains a list of packages that can not distribute compile tasks + # for one reason or the other + # this is the old list (which doesn't seem to be valid anymore, because I was able to build + # all these with icecc enabled) + # system_package_blacklist = [ "uclibc", "glibc", "gcc", "bind", "u-boot", "dhcp-forwarder", "enchant", "connman", "orbit2" ] + # when adding new entry, please document why (how it failed) so that we can re-evaluate it later + # e.g. when there is new version + system_package_blacklist = [] user_package_blacklist = (d.getVar('ICECC_USER_PACKAGE_BL') or "").split() user_package_whitelist = (d.getVar('ICECC_USER_PACKAGE_WL') or "").split() package_blacklist = system_package_blacklist + user_package_blacklist - for black in package_blacklist: - if black in package_tmp: - #bb.note(package_tmp, ' found in blacklist, disable icecc') - return "no" + if pn in package_blacklist: + bb.debug(1, "%s: found in blacklist, disable icecc" % pn) + return "no" - for white in user_package_whitelist: - if white in package_tmp: - bb.debug(1, package_tmp, " ", d.expand('${PV})'), " found in whitelist, enable icecc") - return "yes" + if pn in user_package_whitelist: + bb.debug(1, "%s: found in whitelist, enable icecc" % pn) + return "yes" if d.getVar('PARALLEL_MAKE') == "": - bb.debug(1, package_tmp, " ", d.expand('${PV}'), " has empty PARALLEL_MAKE, disable icecc") + bb.debug(1, "%s: has empty PARALLEL_MAKE, disable icecc" % pn) return "no" return "yes"
Signature · 1 line
1.8.5.2