shlibs providers improvements
cover shlibs providers improvements
Fixes for [YOCTO #4628]
Richard had few more ideas to improve it even more, but this is good start.
Most important changes:
* PRIVATE_LIBS aren't duplicitly added to RDEPENDS (when recipe provides own libfoo it shouldn't get runtime dependency on global libfoo shlib provider)
* warning is shown for undeterministic providers (multiple packages recorded to provide the same library) - recipe maintainers can decide if such library should be considered private or make sure that only one package is built (most common case are gst plugins where 0.10 and 1.* provide the same gst modules, but as long as nobody tries to linke directly to some gst plugin we shouldn't list them as shlib providers).
* list files which require given shlib, often I was surprised to see that something got runtime dependency on libfoo, showing which binary is linked to it makes it easier to debug when you have only log.do_package and package directory is already gone (e.g. analyzing test-dependencies.sh script outputs)
The following changes since commit 8163854adf87ac42a8f08ee25685d0ce1efb4724:
oe-selftest: separated the SStateBase and SStateTests in different modules (2014-01-16 12:18:44 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib jansa/shlib-providers
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=jansa/shlib-providers
Martin Jansa (5):
package.bbclass: move reading shlibs providers to separate function package.bbclass: show warning when package is trying to provide
already provided shlib
package.bbclass: add SHLIBSSEARCHDIRS to define where to search for
shlib providers
package.bbclass: Don't search for prividers of PRIVATE_LIBS package.bbclass: Show which files require given dependency in debug
output
meta/classes/package.bbclass | 102 +++++++++++++++++++++++++++++++------------ 1 file changed, 75 insertions(+), 27 deletions(-)
Signature · 1 line
1.8.5.2
PATCH 1/5 package.bbclass: move reading shlibs providers to separate function
* prepare for reading shlibs providers only from dependency tree of current recipe
[YOCTO #4628]
--- meta/classes/package.bbclass | 43 +++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 20 deletions(-)
meta/classes/package.bbclass +23 -20
@@ -1352,6 +1352,28 @@ python package_do_shlibs() { # Take shared lock since we're only reading, not writing lf = bb.utils.lockfile(d.expand("${PACKAGELOCK}")) + def read_shlib_providers(): + list_re = re.compile('^(.*)\.list$') + # Go from least to most specific since the last one found wins + for dir in reversed(shlibs_dirs): + if not os.path.exists(dir): + continue + for file in os.listdir(dir): + m = list_re.match(file) + if m: + dep_pkg = m.group(1) + fd = open(os.path.join(dir, file)) + lines = fd.readlines() + fd.close() + ver_file = os.path.join(dir, dep_pkg + '.ver') + lib_ver = None + if os.path.exists(ver_file): + fd = open(ver_file) + lib_ver = fd.readline().rstrip() + fd.close() + for l in lines: + shlib_provider[l.rstrip()] = (dep_pkg, lib_ver) + def linux_so(file): needs_ldconfig = False cmd = d.getVar('OBJDUMP', True) + " -p " + pipes.quote(file) + " 2>/dev/null"
@@ -1496,26 +1518,7 @@ python package_do_shlibs() { postinst += d.getVar('ldconfig_postinst_fragment', True) d.setVar('pkg_postinst_%s' % pkg, postinst) - list_re = re.compile('^(.*)\.list$') - # Go from least to most specific since the last one found wins - for dir in reversed(shlibs_dirs): - if not os.path.exists(dir): - continue - for file in os.listdir(dir): - m = list_re.match(file) - if m: - dep_pkg = m.group(1) - fd = open(os.path.join(dir, file)) - lines = fd.readlines() - fd.close() - ver_file = os.path.join(dir, dep_pkg + '.ver') - lib_ver = None - if os.path.exists(ver_file): - fd = open(ver_file) - lib_ver = fd.readline().rstrip() - fd.close() - for l in lines: - shlib_provider[l.rstrip()] = (dep_pkg, lib_ver) + read_shlib_providers() bb.utils.unlockfile(lf)
Signature · 1 line
1.8.5.2
PATCH 2/5 package.bbclass: show warning when package is trying to provide already provided shlib
In reply to Martin Jansa* move read_shlib_providers before registering package as provider and don't change provider if it already exists, show warning instead
[YOCTO #4628]
--- meta/classes/package.bbclass | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +10 -2
@@ -1356,6 +1356,7 @@ python package_do_shlibs() { list_re = re.compile('^(.*)\.list$') # Go from least to most specific since the last one found wins for dir in reversed(shlibs_dirs): + bb.debug(2, "Reading shlib providers in %s" % (dir)) if not os.path.exists(dir): continue for file in os.listdir(dir):
@@ -1471,6 +1472,8 @@ python package_do_shlibs() { needed = {} shlib_provider = {} + read_shlib_providers() + for pkg in packages.split(): private_libs = d.getVar('PRIVATE_LIBS_' + pkg, True) or d.getVar('PRIVATE_LIBS', True) needs_ldconfig = False
@@ -1504,6 +1507,12 @@ python package_do_shlibs() { if len(sonames): fd = open(shlibs_file, 'w') for s in sonames: + if s in shlib_provider: + (old_pkg, old_pkgver) = shlib_provider[s] + if old_pkg != pkg: + bb.warn('%s-%s is already registered as shlib provider for %s, ignoring %s-%s trying to register the same' % (old_pkg, old_pkgver, s, pkg, pkgver)) + continue + bb.debug(1, 'registering %s-%s as shlib provider for %s' % (pkg, pkgver, s)) fd.write(s + '\n') shlib_provider[s] = (pkg, pkgver) fd.close()
@@ -1517,8 +1526,7 @@ python package_do_shlibs() { postinst = '#!/bin/sh\n' postinst += d.getVar('ldconfig_postinst_fragment', True) d.setVar('pkg_postinst_%s' % pkg, postinst) - - read_shlib_providers() + bb.debug(1, 'LIBNAMES: pkg %s sonames %s' % (pkg, sonames)) bb.utils.unlockfile(lf)
Signature · 1 line
1.8.5.2
Re: [PATCH 2/5] package.bbclass: show warning when package is trying to provide already provided shlib
Quoted 32 lines · Martin Jansa
* move
read_shlib_providersbefore registering package as provider and don't change provider if it already exists, show warning instead[YOCTO #4628]
Signed-off-by Martin Jansa <[email protected]>--- meta/classes/package.bbclass | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)meta/classes/package.bbclass +3
-0@@ -1356,6 +1356,7 @@ python package_do_shlibs() { list_re = re.compile('^(.*)\.list$') # Go from least to most specific since the last one found wins for dir in reversed(shlibs_dirs): + bb.debug(2, "Reading shlib providers in %s" % (dir)) if not os.path.exists(dir): continue for file in os.listdir(dir):@@ -1471,6 +1472,8 @@ python package_do_shlibs() { needed = {} shlib_provider = {} + read_shlib_providers() + for pkg in packages.split(): private_libs = d.getVar('PRIVATE_LIBS_' + pkg, True) or d.getVar('PRIVATE_LIBS', True) needs_ldconfig = False
meta/classes/package.bbclass python package_do_shlibs() { · line 1514
Quoted 8 lines
meta/classes/package.bbclass +5
-0@@ -1504,6 +1507,12 @@ python package_do_shlibs() { if len(sonames): fd = open(shlibs_file, 'w') for s in sonames: + if s in shlib_provider: + (old_pkg, old_pkgver) = shlib_provider[s] + if old_pkg != pkg: + bb.warn('%s-%s is already registered as shlib provider for %s, ignoring %s-%s trying to register the same' % (old_pkg, old_pkgver, s, pkg, pkgver)) + continue
I'll drop continue and update warning message, changing registered provider for last built is still undeterministic but slightly better e.g. when changing packaging and the shlib is moved between packages (I've run into this in efl, when moving to split packages and they were still getting runtime dependency on old efl package).
Quoted 17 lines
meta/classes/package.bbclass +2
-2+ bb.debug(1, 'registering %s-%s as shlib provider for %s' % (pkg, pkgver, s)) fd.write(s + '\n') shlib_provider[s] = (pkg, pkgver) fd.close()@@ -1517,8 +1526,7 @@ python package_do_shlibs() { postinst = '#!/bin/sh\n' postinst += d.getVar('ldconfig_postinst_fragment', True) d.setVar('pkg_postinst_%s' % pkg, postinst) - - read_shlib_providers() + bb.debug(1, 'LIBNAMES: pkg %s sonames %s' % (pkg, sonames)) bb.utils.unlockfile(lf)Signature · 1 line
1.8.5.2
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
PATCH v2 2/5 package.bbclass: show warning when package is trying to provide already provided shlib
In reply to Martin Jansa* move read_shlib_providers before registering package as provider and don't change provider if it already exists, show warning instead
[YOCTO #4628]
--- meta/classes/package.bbclass | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +9 -2
@@ -1356,6 +1356,7 @@ python package_do_shlibs() { list_re = re.compile('^(.*)\.list$') # Go from least to most specific since the last one found wins for dir in reversed(shlibs_dirs): + bb.debug(2, "Reading shlib providers in %s" % (dir)) if not os.path.exists(dir): continue for file in os.listdir(dir):
@@ -1471,6 +1472,8 @@ python package_do_shlibs() { needed = {} shlib_provider = {} + read_shlib_providers() + for pkg in packages.split(): private_libs = d.getVar('PRIVATE_LIBS_' + pkg, True) or d.getVar('PRIVATE_LIBS', True) needs_ldconfig = False
@@ -1504,6 +1507,11 @@ python package_do_shlibs() { if len(sonames): fd = open(shlibs_file, 'w') for s in sonames: + if s in shlib_provider: + (old_pkg, old_pkgver) = shlib_provider[s] + if old_pkg != pkg: + bb.warn('%s-%s was registered as shlib provider for %s, changing it to %s-%s because it was built later' % (old_pkg, old_pkgver, s, pkg, pkgver)) + bb.debug(1, 'registering %s-%s as shlib provider for %s' % (pkg, pkgver, s)) fd.write(s + '\n') shlib_provider[s] = (pkg, pkgver) fd.close()
@@ -1517,8 +1525,7 @@ python package_do_shlibs() { postinst = '#!/bin/sh\n' postinst += d.getVar('ldconfig_postinst_fragment', True) d.setVar('pkg_postinst_%s' % pkg, postinst) - - read_shlib_providers() + bb.debug(1, 'LIBNAMES: pkg %s sonames %s' % (pkg, sonames)) bb.utils.unlockfile(lf)
Signature · 1 line
1.8.5.2
Re: [PATCHv2 2/5] package.bbclass: show warning when package is trying to provide already provided shlib
Quoted 2 lines · Martin Jansa
* move
read_shlib_providersbefore registering package as provider and don't change provider if it already exists, show warning instead
This new patch doesn't seem to match the commit message any more.
p.
Quoted 5 lines
meta/classes/package.bbclass +5
-0+ if s in shlib_provider: + (old_pkg, old_pkgver) = shlib_provider[s] + if old_pkg != pkg: + bb.warn('%s-%s was registered as shlib provider for %s, changing it to %s-%s because it was built later' % (old_pkg, old_pkgver, s, pkg, pkgver)) + bb.debug(1, 'registering %s-%s as shlib provider for %s' % (pkg, pkgver, s))
PATCH v2 3/5 package.bbclass: add SHLIBSSEARCHDIRS to define where to search for shlib providers
* when package contains some files matching "^.*\.so", but in directory not default linker search paths (e.g. /opt/package/bundled-lib/libfoo.so) don't register it as libfoo provider, because it's possible that there is different package providing libfoo.so in ${libdir} and that would be better shlib provider for other packages to depend on
* recipes providing libs intentionally in some other directory can define own SHLIBSSEARCHDIRS value
[YOCTO #4628]
--- meta/classes/package.bbclass | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
meta/classes/package.bbclass +23 -3
@@ -1322,6 +1322,9 @@ python package_do_filedeps() { SHLIBSDIRS = "${PKGDATA_DIR}/${MLPREFIX}shlibs" SHLIBSWORKDIR = "${PKGDESTWORK}/${MLPREFIX}shlibs" +# default search path when searching for shlibs provided by package +SHLIBSSEARCHDIRS ?= "${baselib} ${libdir}" + python package_do_shlibs() { import re, pipes
@@ -1332,6 +1335,20 @@ python package_do_shlibs() { lib_re = re.compile("^.*\.so") libdir_re = re.compile(".*/%s$" % d.getVar('baselib', True)) + + shlibs_search_dirs = d.getVar('SHLIBSSEARCHDIRS', True) + shlibs_search_dirs_re_txt = "" + for dir in shlibs_search_dirs.split(' '): + # strip leading and trailing slash, it's added in regexp + if dir.endswith("/"): + dir = dir[:-1] + if dir.startswith("/"): + dir = dir[1:] + if shlibs_search_dirs_re_txt: + shlibs_search_dirs_re_txt += "|" + shlibs_search_dirs_re_txt += "(^.*/%s/.*$)" % dir + shlibs_search_dirs_re = re.compile(shlibs_search_dirs_re_txt) + bb.debug(2, "will use following RE to search for provides sonames %s" % shlibs_search_dirs_re_txt) packages = d.getVar('PACKAGES', True) targetos = d.getVar('TARGET_OS', True)
@@ -1390,9 +1407,12 @@ python package_do_shlibs() { if m: this_soname = m.group(1) if not this_soname in sonames: - # if library is private (only used by package) then do not build shlib for it - if not private_libs or -1 == private_libs.find(this_soname): - sonames.append(this_soname) + if shlibs_search_dirs_re.match(file): + # if library is private (only used by package) then do not build shlib for it + if not private_libs or -1 == private_libs.find(this_soname): + sonames.append(this_soname) + else: + bb.debug(2, "ignoring soname %s from %s, because path doesn't match %s" % (this_soname, file, shlibs_search_dirs_re_txt)) if libdir_re.match(os.path.dirname(file)): needs_ldconfig = True if snap_symlinks and (os.path.basename(file) != this_soname):
Signature · 1 line
1.8.5.2
Re: [PATCHv2 3/5] package.bbclass: add SHLIBSSEARCHDIRS to define where to search for shlib providers
Quoted 7 lines · Martin Jansa
* when package contains some files matching "^.*\.so", but in directory not default linker search paths (e.g.
/opt/package/bundled-lib/libfoo.so) don't register it as libfoo provider, because it's possible that there is different package providing libfoo.so in ${libdir} and that would be better shlib provider for other packages to depend on
* recipes providing libs intentionally in some other directory can define own SHLIBSSEARCHDIRS value
This one is the same as v1, sent by accident when sending v3 of 2/5.
Quoted 61 lines
[YOCTO #4628]
Signed-off-by Martin Jansa <[email protected]>--- meta/classes/package.bbclass | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)meta/classes/package.bbclass +23
-3@@ -1322,6 +1322,9 @@ python package_do_filedeps() { SHLIBSDIRS = "${PKGDATA_DIR}/${MLPREFIX}shlibs" SHLIBSWORKDIR = "${PKGDESTWORK}/${MLPREFIX}shlibs" +# default search path when searching for shlibs provided by package +SHLIBSSEARCHDIRS ?= "${baselib} ${libdir}" + python package_do_shlibs() { import re, pipes@@ -1332,6 +1335,20 @@ python package_do_shlibs() { lib_re = re.compile("^.*\.so") libdir_re = re.compile(".*/%s$" % d.getVar('baselib', True)) + + shlibs_search_dirs = d.getVar('SHLIBSSEARCHDIRS', True) + shlibs_search_dirs_re_txt = "" + for dir in shlibs_search_dirs.split(' '): + # strip leading and trailing slash, it's added in regexp + if dir.endswith("/"): + dir = dir[:-1] + if dir.startswith("/"): + dir = dir[1:] + if shlibs_search_dirs_re_txt: + shlibs_search_dirs_re_txt += "|" + shlibs_search_dirs_re_txt += "(^.*/%s/.*$)" % dir + shlibs_search_dirs_re = re.compile(shlibs_search_dirs_re_txt) + bb.debug(2, "will use following RE to search for provides sonames %s" % shlibs_search_dirs_re_txt) packages = d.getVar('PACKAGES', True) targetos = d.getVar('TARGET_OS', True)@@ -1390,9 +1407,12 @@ python package_do_shlibs() { if m: this_soname = m.group(1) if not this_soname in sonames: - # if library is private (only used by package) then do not build shlib for it - if not private_libs or -1 == private_libs.find(this_soname): - sonames.append(this_soname) + if shlibs_search_dirs_re.match(file): + # if library is private (only used by package) then do not build shlib for it + if not private_libs or -1 == private_libs.find(this_soname): + sonames.append(this_soname) + else: + bb.debug(2, "ignoring soname %s from %s, because path doesn't match %s" % (this_soname, file, shlibs_search_dirs_re_txt)) if libdir_re.match(os.path.dirname(file)): needs_ldconfig = True if snap_symlinks and (os.path.basename(file) != this_soname):Signature · 1 line
1.8.5.2
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
PATCH v3 2/5 package.bbclass: show warning when package is providing already provided shlib
In reply to Phil Blundell* move read_shlib_providers before registering package as provider and show warning when different package tries to provide something already provided.
[YOCTO #4628]
--- meta/classes/package.bbclass | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +9 -2
@@ -1356,6 +1356,7 @@ python package_do_shlibs() { list_re = re.compile('^(.*)\.list$') # Go from least to most specific since the last one found wins for dir in reversed(shlibs_dirs): + bb.debug(2, "Reading shlib providers in %s" % (dir)) if not os.path.exists(dir): continue for file in os.listdir(dir):
@@ -1471,6 +1472,8 @@ python package_do_shlibs() { needed = {} shlib_provider = {} + read_shlib_providers() + for pkg in packages.split(): private_libs = d.getVar('PRIVATE_LIBS_' + pkg, True) or d.getVar('PRIVATE_LIBS', True) needs_ldconfig = False
@@ -1504,6 +1507,11 @@ python package_do_shlibs() { if len(sonames): fd = open(shlibs_file, 'w') for s in sonames: + if s in shlib_provider: + (old_pkg, old_pkgver) = shlib_provider[s] + if old_pkg != pkg: + bb.warn('%s-%s was registered as shlib provider for %s, changing it to %s-%s because it was built later' % (old_pkg, old_pkgver, s, pkg, pkgver)) + bb.debug(1, 'registering %s-%s as shlib provider for %s' % (pkg, pkgver, s)) fd.write(s + '\n') shlib_provider[s] = (pkg, pkgver) fd.close()
@@ -1517,8 +1525,7 @@ python package_do_shlibs() { postinst = '#!/bin/sh\n' postinst += d.getVar('ldconfig_postinst_fragment', True) d.setVar('pkg_postinst_%s' % pkg, postinst) - - read_shlib_providers() + bb.debug(1, 'LIBNAMES: pkg %s sonames %s' % (pkg, sonames)) bb.utils.unlockfile(lf)
Signature · 1 line
1.8.5.2
PATCH 3/5 package.bbclass: add SHLIBSSEARCHDIRS to define where to search for shlib providers
In reply to Martin Jansa* when package contains some files matching "^.*\.so", but in directory not default linker search paths (e.g. /opt/package/bundled-lib/libfoo.so) don't register it as libfoo provider, because it's possible that there is different package providing libfoo.so in ${libdir} and that would be better shlib provider for other packages to depend on
* recipes providing libs intentionally in some other directory can define own SHLIBSSEARCHDIRS value
[YOCTO #4628]
--- meta/classes/package.bbclass | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
meta/classes/package.bbclass +23 -3
@@ -1322,6 +1322,9 @@ python package_do_filedeps() { SHLIBSDIRS = "${PKGDATA_DIR}/${MLPREFIX}shlibs" SHLIBSWORKDIR = "${PKGDESTWORK}/${MLPREFIX}shlibs" +# default search path when searching for shlibs provided by package +SHLIBSSEARCHDIRS ?= "${baselib} ${libdir}" + python package_do_shlibs() { import re, pipes
@@ -1332,6 +1335,20 @@ python package_do_shlibs() { lib_re = re.compile("^.*\.so") libdir_re = re.compile(".*/%s$" % d.getVar('baselib', True)) + + shlibs_search_dirs = d.getVar('SHLIBSSEARCHDIRS', True) + shlibs_search_dirs_re_txt = "" + for dir in shlibs_search_dirs.split(' '): + # strip leading and trailing slash, it's added in regexp + if dir.endswith("/"): + dir = dir[:-1] + if dir.startswith("/"): + dir = dir[1:] + if shlibs_search_dirs_re_txt: + shlibs_search_dirs_re_txt += "|" + shlibs_search_dirs_re_txt += "(^.*/%s/.*$)" % dir + shlibs_search_dirs_re = re.compile(shlibs_search_dirs_re_txt) + bb.debug(2, "will use following RE to search for provides sonames %s" % shlibs_search_dirs_re_txt) packages = d.getVar('PACKAGES', True) targetos = d.getVar('TARGET_OS', True)
@@ -1390,9 +1407,12 @@ python package_do_shlibs() { if m: this_soname = m.group(1) if not this_soname in sonames: - # if library is private (only used by package) then do not build shlib for it - if not private_libs or -1 == private_libs.find(this_soname): - sonames.append(this_soname) + if shlibs_search_dirs_re.match(file): + # if library is private (only used by package) then do not build shlib for it + if not private_libs or -1 == private_libs.find(this_soname): + sonames.append(this_soname) + else: + bb.debug(2, "ignoring soname %s from %s, because path doesn't match %s" % (this_soname, file, shlibs_search_dirs_re_txt)) if libdir_re.match(os.path.dirname(file)): needs_ldconfig = True if snap_symlinks and (os.path.basename(file) != this_soname):
Signature · 1 line
1.8.5.2
Re: [PATCH 3/5] package.bbclass: add SHLIBSSEARCHDIRS to define where to search for shlib providers
Quoted 8 lines · Martin Jansa
* when package contains some files matching "^.*\.so", but in directory not default linker search paths (e.g.
/opt/package/bundled-lib/libfoo.so) don't register it as libfoo provider, because it's possible that there is different package providing libfoo.so in ${libdir} and that would be better shlib provider for other packages to depend on
* recipes providing libs intentionally in some other directory can define own SHLIBSSEARCHDIRS value
Quoted 6 lines
[YOCTO #4628]
Signed-off-by Martin Jansa <[email protected]>--- meta/classes/package.bbclass | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
Doesn't this mean we lose automatic dependencies for libraries not in the default system paths after this change?
We should really be respecting the libraries RPATH in this code and searching the places it will look...
Cheers,
Richard
Re: [PATCH 3/5] package.bbclass: add SHLIBSSEARCHDIRS to define where to search for shlib providers
Quoted 11 lines · Richard Purdie
Quoted 10 lines · Martin Jansa
* when package contains some files matching "^.*\.so", but in directory not default linker search paths (e.g.
/opt/package/bundled-lib/libfoo.so) don't register it as libfoo provider, because it's possible that there is different package providing libfoo.so in ${libdir} and that would be better shlib provider for other packages to depend on
* recipes providing libs intentionally in some other directory can define own SHLIBSSEARCHDIRS value[YOCTO #4628]
Quoted 7 lines
Quoted 4 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/classes/package.bbclass | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)Doesn't this mean we lose automatic dependencies for libraries not in the default system paths after this change?
True, recipes with such libraries would need to append to SHLIBSSEARCHDIRS
Quoted 2 lines
We should really be respecting the libraries RPATH in this code and searching the places it will look...
I agree we should respect RPATH, but sometimes it is more complicated, because your own binaries can have RPATH pointing to /opt/package/bundled-lib/libfoo.so, but you don't want any other recipe to find your libfoo.so, because there is preferred "system" /usr/lib/libfoo.so
So in ideal case we should respect combination of default search path, ld.so.conf, RPATH and PRIVATE_LIBS :/.
I don't mind if you drop this one from patchset, it's the most controversial one and can be work-arounded by carefully maintained PRIVATE_LIBS list.
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
PATCH 4/5 package.bbclass: Don't search for prividers of PRIVATE_LIBS
In reply to Martin Jansa* split PRIVATE_LIBS and don't use find(), so that libfoo cannot be found in PRIVATE_LIBS = "libfoobar"
--- meta/classes/package.bbclass | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +11 -2
@@ -1409,7 +1409,7 @@ python package_do_shlibs() { if not this_soname in sonames: if shlibs_search_dirs_re.match(file): # if library is private (only used by package) then do not build shlib for it - if not private_libs or -1 == private_libs.find(this_soname): + if not private_libs or this_soname not in private_libs: sonames.append(this_soname) else: bb.debug(2, "ignoring soname %s from %s, because path doesn't match %s" % (this_soname, file, shlibs_search_dirs_re_txt))
@@ -1495,7 +1495,8 @@ python package_do_shlibs() { read_shlib_providers() for pkg in packages.split(): - private_libs = d.getVar('PRIVATE_LIBS_' + pkg, True) or d.getVar('PRIVATE_LIBS', True) + private_libs = d.getVar('PRIVATE_LIBS_' + pkg, True) or d.getVar('PRIVATE_LIBS', True) or "" + private_libs = private_libs.split() needs_ldconfig = False bb.debug(2, "calculating shlib provides for %s" % pkg)
@@ -1566,6 +1567,14 @@ python package_do_shlibs() { deps = list() for n in needed[pkg]: + # if n is in private libraries, don't try to search provider for it + # this could cause problem in case some abc.bb provides private + # /opt/abc/lib/libfoo.so.1 and contains /usr/bin/abc depending on system library libfoo.so.1 + # but skipping it is still better alternative than providing own + # version and then adding runtime dependency for the same system library + if private_libs and n in private_libs: + bb.debug(2, '%s: Dependency %s covered by PRIVATE_LIBS' % (pkg, n)) + continue if n in shlib_provider.keys(): (dep_pkg, ver_needed) = shlib_provider[n]
Signature · 1 line
1.8.5.2
PATCH 5/5 package.bbclass: Show which files require given dependency in debug output
In reply to Martin Jansa* when log.do_package shows some unexpected dependency, people usually need to grep package directory to find which binary was creating that dependency, show it directly in the debug output
--- meta/classes/package.bbclass | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +10 -2
@@ -1403,6 +1403,9 @@ python package_do_shlibs() { if m: if m.group(1) not in needed[pkg]: needed[pkg].append(m.group(1)) + if m.group(1) not in needed_from: + needed_from[m.group(1)] = [] + needed_from[m.group(1)].append(file) m = re.match("\s+SONAME\s+([^\s]*)", l) if m: this_soname = m.group(1)
@@ -1478,6 +1481,10 @@ python package_do_shlibs() { needed[pkg] = [] if name and name not in needed[pkg]: needed[pkg].append(name) + if name not in needed_from: + needed_from[name] = [] + if lafile and lafile not in needed_from[name]: + needed_from[name].append(lafile) #bb.note("Adding %s for %s" % (name, pkg)) if d.getVar('PACKAGE_SNAP_LIB_SYMLINKS', True) == "1":
@@ -1491,6 +1498,7 @@ python package_do_shlibs() { use_ldconfig = False needed = {} + needed_from = {} shlib_provider = {} read_shlib_providers()
@@ -1578,7 +1586,7 @@ python package_do_shlibs() { if n in shlib_provider.keys(): (dep_pkg, ver_needed) = shlib_provider[n] - bb.debug(2, '%s: Dependency %s requires package %s' % (pkg, n, dep_pkg)) + bb.debug(2, '%s: Dependency %s requires package %s (used by files: %s)' % (pkg, n, dep_pkg, needed_from[n])) if dep_pkg == pkg: continue
@@ -1590,7 +1598,7 @@ python package_do_shlibs() { if not dep in deps: deps.append(dep) else: - bb.note("Couldn't find shared library provider for %s" % n) + bb.note("Couldn't find shared library provider for %s, used by files: %s" % (n, needed_from[n])) deps_file = os.path.join(pkgdest, pkg + ".shlibdeps") if os.path.exists(deps_file):
Signature · 1 line
1.8.5.2