fixes for package_qa_walk() and buildpaths
cover fixes for package_qa_walk() and buildpaths
The following changes since commit 4038970f8ce27ac0d7a0afe2cdaa9a65108dfff5:
os-release: fix do_compile() when RPM signing is enabled (2015-10-07 00:08:07 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/buildpaths
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/buildpaths
Robert Yang (8):
insane.bbclass: do_package_qa depends on WARN_QA and ERROR_QA insane.bbclass: make package_qa_walk() can print all the messages insane.bbclass: buildpaths: ignore .pyc files insane.bbclass: skip buildpath qa check when set
INHIBIT_PACKAGE_STRIP
insane.bbclass: fix for ignoring debug files package.bbclass: strip archive files autotools.bbclass: use relative path to run configure script glibc-initial.inc: use relative path to run configure
meta/classes/autotools.bbclass | 4 +++- meta/classes/insane.bbclass | 32 ++++++++++++++++++++++++----- meta/classes/package.bbclass | 18 +++++++++++++++- meta/recipes-core/glibc/glibc-initial.inc | 2 +- 4 files changed, 48 insertions(+), 8 deletions(-)
Signature · 1 line
1.7.9.5
PATCH 1/8 insane.bbclass: do_package_qa depends on WARN_QA and ERROR_QA
do_package_qa() should rerun when WARN_QA or ERROR_QA changed
[YOCTO #8445]
--- meta/classes/insane.bbclass | 1 + 1 file changed, 1 insertion(+)
meta/classes/insane.bbclass +1 -0
@@ -1113,6 +1113,7 @@ python do_package_qa () { } do_package_qa[rdeptask] = "do_packagedata" +do_package_qa[vardeps] += "${WARN_QA} ${ERROR_QA}" addtask do_package_qa after do_packagedata do_package before do_build SSTATETASKS += "do_package_qa"
Signature · 1 line
1.7.9.5
PATCH 2/8 insane.bbclass: make package_qa_walk() can print all the messages
In reply to Robert Yang* If more than one files have the same QA issue, package_qa_walk() can only print the last one, others are overrided, for example:
messages["host-user-contaminated"] = "foo1" messages["host-user-contaminated"] = "foo2"
Only foo2 will be printed, this patch fixes the issue.
* Remove unused parameter "path" from package_qa_walk()
The path is a useless parameter in package_qa_walk() since it has been redined inside.
[YOCTO #8436]
--- meta/classes/insane.bbclass | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
meta/classes/insane.bbclass +17 -3
@@ -749,8 +749,15 @@ def package_qa_check_staged(path,d): return sane +# Merge src dict's content into dest +def package_qa_merge_dict(dest, src): + if len(dest) == len(src) and dest != src: + for w in src: + if dest[w] != src[w]: + dest[w] = src[w] + '\n' + dest[w] + # Walk over all files in a directory and call func -def package_qa_walk(path, warnfuncs, errorfuncs, skip, package, d): +def package_qa_walk(warnfuncs, errorfuncs, skip, package, d): import oe.qa #if this will throw an exception, then fix the dict above
@@ -766,9 +773,17 @@ def package_qa_walk(path, warnfuncs, errorfuncs, skip, package, d): except: elf = None for func in warnfuncs: + warnings_orig = warnings.copy() func(path, package, d, elf, warnings) + # warnings[foo] = "foo1" might be overrided by + # warnings [foo] = "foo2", check and merge. + package_qa_merge_dict(warnings, warnings_orig) for func in errorfuncs: + errors_orig = errors.copy() func(path, package, d, elf, errors) + # errors[foo] = "foo1" might be overrided by + # errors [foo] = "foo2", check and merge. + package_qa_merge_dict(errors, errors_orig) for w in warnings: package_qa_handle_error(w, warnings[w], d)
@@ -1094,8 +1109,7 @@ python do_package_qa () { package_qa_handle_error("pkgname", "%s doesn't match the [a-z0-9.+-]+ regex" % package, d) - path = "%s/%s" % (pkgdest, package) - if not package_qa_walk(path, warnchecks, errorchecks, skip, package, d): + if not package_qa_walk(warnchecks, errorchecks, skip, package, d): walk_sane = False if not package_qa_check_rdepends(package, pkgdest, skip, taskdeps, packages, d): rdepends_sane = False
Signature · 1 line
1.7.9.5
PATCH 3/8 insane.bbclass: buildpaths: ignore .pyc files
In reply to Robert YangThe .pyc files usually contain build path like:
i586-poky-linux-gcc -m32 -march=i586 --sysroot=/buildarea/lyang1/test_p/tmp/sysroots/qemux86 And:
_PYTHON_PROJECT_BASE=/buildarea/lyang1/test_p/tmp/work/i586-poky-linux/python/2.7.9-r1/build
They are safe, so ignore them.
This can reduce 1065 QA issues for buildpaths in a world build.
[YOCTO #8446]
--- meta/classes/insane.bbclass | 4 ++++ 1 file changed, 4 insertions(+)
meta/classes/insane.bbclass +4 -0
@@ -576,6 +576,10 @@ def package_qa_check_buildpaths(path, name, d, elf, messages): if path.find(".debug") != -1: return + # Ignore .pyc files, not interesting + if path.endswith(".pyc"): + return + # Ignore symlinks if os.path.islink(path): return
Signature · 1 line
1.7.9.5
PATCH 4/8 insane.bbclass: skip buildpath qa check when set INHIBIT_PACKAGE_STRIP
In reply to Robert YangThere are build paths in the file when set INHIBIT_PACKAGE_STRIP.
[YOCTO #8458]
--- meta/classes/insane.bbclass | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
meta/classes/insane.bbclass +5 -2
@@ -572,8 +572,11 @@ def package_qa_check_buildpaths(path, name, d, elf, messages): """ Check for build paths inside target files and error if not found in the whitelist """ - # Ignore .debug files, not interesting - if path.find(".debug") != -1: + # Ignore: + # - .debug files + # - which sets INHIBIT_PACKAGE_STRIP + if path.find(".debug") != -1 or \ + d.getVar('INHIBIT_PACKAGE_STRIP', True) == "1": return # Ignore .pyc files, not interesting
Signature · 1 line
1.7.9.5
PATCH 5/8 insane.bbclass: fix for ignoring debug files
In reply to Robert YangSome recipes put debug files in "usr/src/debug/" rather than ".debug".
--- meta/classes/insane.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/insane.bbclass +1 -1
@@ -575,7 +575,7 @@ def package_qa_check_buildpaths(path, name, d, elf, messages): # Ignore: # - .debug files # - which sets INHIBIT_PACKAGE_STRIP - if path.find(".debug") != -1 or \ + if path.find(".debug") != -1 or path.find("/src/debug/") != -1 or \ d.getVar('INHIBIT_PACKAGE_STRIP', True) == "1": return
Signature · 1 line
1.7.9.5
PATCH 6/8 package.bbclass: strip archive files
In reply to Robert YangFor saving space and removing build path in .a files.
[YOCTO #8457]
--- meta/classes/package.bbclass | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
meta/classes/package.bbclass +17 -1
@@ -870,12 +870,25 @@ python split_and_strip_files () { return type + def isArchive(path): + ret, result = oe.utils.getstatusoutput("file \"%s\"" % path.replace("\"", "\\\"")) + if ret: + msg = "split_and_strip_files: 'file %s' failed" % path + package_qa_handle_error("split-strip", msg, d) + return False + + if "current ar archive" in result: + return True + else: + return False + # # First lets figure out all of the files we may have to process ... do this only once! # elffiles = {} symlinks = {} kernmods = [] + archives = [] inodes = {} libdir = os.path.abspath(dvar + os.sep + d.getVar("libdir", True)) baselibdir = os.path.abspath(dvar + os.sep + d.getVar("base_libdir", True))
@@ -886,6 +899,9 @@ python split_and_strip_files () { if file.endswith(".ko") and file.find("/lib/modules/") != -1: kernmods.append(file) continue + if file.endswith('.a') and isArchive(file): + archives.append(file) + continue # Skip debug files if debugappend and file.endswith(debugappend):
@@ -1019,7 +1035,7 @@ python split_and_strip_files () { elf_file = int(elffiles[file]) #bb.note("Strip %s" % file) sfiles.append((file, elf_file, strip)) - for f in kernmods: + for f in kernmods + archives: sfiles.append((f, 16, strip)) oe.utils.multiprocess_exec(sfiles, oe.package.runstrip)
Signature · 1 line
1.7.9.5
PATCH 7/8 autotools.bbclass: use relative path to run configure script
In reply to Robert YangThe absolute path (/path/to/configure) caused VPATH in Makefile to be an absolute path, and then it will be in elf files, use relative path to run configure can fix the problem.
This will reduce 3127 buildpaths QA issues in a world buld (8957 -> 5830).
[YOCTO #8437]
--- meta/classes/autotools.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes/autotools.bbclass +3 -1
@@ -79,7 +79,9 @@ CONFIGUREOPT_DEPTRACK ?= "--disable-dependency-tracking" oe_runconf () { - cfgscript="${S}/configure" + # Use relative path for S to avoid buildpaths in files + relative_s=`python -c "import os; print os.path.relpath('${S}', '.')"` + cfgscript="$relative_s/configure" if [ -x "$cfgscript" ] ; then bbnote "Running $cfgscript ${CONFIGUREOPTS} ${EXTRA_OECONF} $@" set +e
Signature · 1 line
1.7.9.5
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 7 lines · Robert Yang
oe_runconf() {meta/classes/autotools.bbclass +2
-1- cfgscript="${S}/configure" + # Use relative path for S to avoid buildpaths in files + relative_s=`python -c "import os; print os.path.relpath('${S}','.')"`
+ cfgscript="$relative_s/configure"
Why not just use inline python here, rather than calling out to an external python process at task execution time?
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 11 lines · Christopher Larson
Quoted 7 lines · Robert Yang
oe_runconf() {meta/classes/autotools.bbclass +2
-1- cfgscript="${S}/configure" + # Use relative path for S to avoid buildpaths in files + relative_s=`python -c "import os; print os.path.relpath('${S}','.')"`
+ cfgscript="$relative_s/configure"
Quoted 2 lines
Why not just use inline python here, rather than calling out to an external python process at task execution time?
Oh, nevermind, of course oe_runconf can be run from any number of paths :)
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 11 lines · Christopher Larson
+ # Use relative path for S to avoid buildpaths in files
Quoted 7 lines
Quoted 4 lines
+ relative_s=`python -c "import os; print os.path.relpath('${S}',
'.')"`
+ cfgscript="$relative_s/configure"Why not just use inline python here, rather than calling out to an external python process at task execution time?
Oh, nevermind, of course
oe_runconfcan be run from any number of paths :)
Why not use realpath?
ross@flashheart ~/Yocto/poky/meta/recipes-devtools/qemu (mut)
$ realpath --relative-to=. ~/Yocto/poky/meta/recipes-support/mpfr/ ../../recipes-support/mpfr
Ross
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Hi Chris and Ross,
Quoted 10 lines · Ross
On 8 October 2015 at 22:00, Christopher Larson <[email protected] <mailto:[email protected]>> wrote:
+ # Use relative path for S to avoid buildpaths in files
+ relative_s=`python -c "import os; print
os.path.relpath('${S}', '.')"`
+ cfgscript="$relative_s/configure"
Quoted 5 lines
Why not just use inline python here, rather than calling out to an external python process at task execution time?
Oh, nevermind, of course
oe_runconfcan be run from any number of paths :)
Yes, "inline python" is executed at parsing time, but we need it be a building time path.
Quoted 7 lines
Why not use realpath?
ross@flashheart ~/Yocto/poky/meta/recipes-devtools/qemu (mut)
$ realpath --relative-to=. ~/Yocto/poky/meta/recipes-support/mpfr/ ../../recipes-support/mpfr
I'm afraid that realpath is not generally existed on the build host ?
// Robert
Quoted 2 lines
Ross
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 2 lines · Robert Yang
I'm afraid that realpath is not generally existed on the build host ?
It's part of coreutils on my Debian stable box. I guess the real question is "does it exist on Centos 6?". [looks]. Hmm, it's not present on that.
Ross
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 9 lines · Ross
On 9 October 2015 at 02:28, Robert Yang <[email protected] <mailto:[email protected]>> wrote:
I'm afraid that realpath is not generally existed on the build host ?
It's part of coreutils on my Debian stable box. I guess the real question is "does it exist on Centos 6?". [looks]. Hmm, it's not present on that.
Ubuntu 14.03 desktop doesn't have it, either.
// Robert
Quoted 2 lines
Ross
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 10 lines · On Friday 09 October 2015 19:30:07 Robert Yang wrote:
Quoted 8 lines · Ross
On 9 October 2015 at 02:28, Robert Yang <[email protected]
<mailto:[email protected]>> wrote:
I'm afraid that realpath is not generally existed on the build host ?It's part of coreutils on my Debian stable box. I guess the real question is "does it exist on Centos 6?". [looks]. Hmm, it's not present on that.
Ubuntu 14.03 desktop doesn't have it, either.
Does readlink -f suffice ?
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 7 lines · Paul Eggleton
Quoted 6 lines · On Friday 09 October 2015 19:30:07 Robert Yang wrote:
Quoted 5 lines · Ross
On 9 October 2015 at 02:28, Robert Yang <[email protected]
<mailto:[email protected]>> wrote:
I'm afraid that realpath is not generally existed on the build host ?
Quoted 6 lines
Quoted 4 lines
Quoted 3 lines
It's part of coreutils on my Debian stable box. I guess the real question is "does it exist on Centos 6?". [looks]. Hmm, it's not present on that.
Ubuntu 14.03 desktop doesn't have it, either.
Does readlink -f suffice ?
Hi Paul,
We deed change an absolute path to relative path here, readlink -f does the opposite ?
// Robert
Quoted 4 lines
Cheers,
Paul
Re: [PATCH 7/8] autotools.bbclass: use relative path to run configure script
Quoted 14 lines · On Tuesday 13 October 2015 15:17:45 Robert Yang wrote:
Quoted 13 lines · Paul Eggleton
Quoted 12 lines · On Friday 09 October 2015 19:30:07 Robert Yang wrote:
Quoted 10 lines · Ross
On 9 October 2015 at 02:28, Robert Yang <[email protected]
<mailto:[email protected]>> wrote:
I'm afraid that realpath is not generally existed on the build host ?It's part of coreutils on my Debian stable box. I guess the real question
is "does it exist on Centos 6?". [looks]. Hmm, it's not present on that.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Ubuntu 14.03 desktop doesn't have it, either.
Does readlink -f suffice ?
Hi Paul,
We deed change an absolute path to relative path here, readlink -f does the opposite ?
Ah, my misunderstanding - sorry for the noise.
Cheers,
Paul
Signature · 2 lines
Paul Eggleton Intel Open Source Technology Centre
PATCH 8/8 glibc-initial.inc: use relative path to run configure
In reply to Robert YangAvoid potential build path in output files.
--- meta/recipes-core/glibc/glibc-initial.inc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/glibc/glibc-initial.inc +1 -1
@@ -11,7 +11,7 @@ TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TCBOOTSTRAP}" do_configure () { (cd ${S} && gnu-configize) || die "failure in running gnu-configize" find ${S} -name "configure" | xargs touch - ${S}/configure --host=${TARGET_SYS} --build=${BUILD_SYS} \ + ../git/configure --host=${TARGET_SYS} --build=${BUILD_SYS} \ --prefix=/usr \ --without-cvs --disable-sanity-checks \ --with-headers=${STAGING_DIR_TARGET}${includedir} \
Signature · 1 line
1.7.9.5