package.bbclass: allow shell-style wildcards in PRIVATE_LIBS
PATCH 1/1 package.bbclass: allow shell-style wildcards in PRIVATE_LIBS
PRIVATE_LIBS is used to exclude 'private' libraries from getting added to automatic runtime dependency resolution. This variable currently has to list all libraries by name, which becomes a maintenance issue if the list of such libraries frequently changes, or is very large.
This change allows using shell-style wildcards in the variable, similar to how FILES lists what gets packaged.
--- meta/classes/package.bbclass | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
meta/classes/package.bbclass +4 -2
@@ -1646,7 +1646,8 @@ python package_do_shlibs() { prov = (this_soname, ldir, pkgver) if not prov in sonames: # if library is private (only used by package) then do not build shlib for it - if not private_libs or this_soname not in private_libs: + import fnmatch + if not private_libs or len([i for i in private_libs if fnmatch.fnmatch(this_soname, i)]) == 0: sonames.add(prov) if libdir_re.match(os.path.dirname(file)): needs_ldconfig = True
@@ -1829,7 +1830,8 @@ python package_do_shlibs() { # /opt/abc/lib/libfoo.so.1 and contains /usr/bin/abc depending on system library libfoo.so.1 # but skipping it is still better alternative than providing own # version and then adding runtime dependency for the same system library - if private_libs and n[0] in private_libs: + import fnmatch + if private_libs and len([i for i in private_libs if fnmatch.fnmatch(n[0], i)]) > 0: bb.debug(2, '%s: Dependency %s covered by PRIVATE_LIBS' % (pkg, n[0])) continue if n[0] in shlib_provider.keys():
Signature · 1 line
2.17.1
Hi Alexander,
Quoted 18 lines · Alexander Kanavin
PRIVATE_LIBSis used to exclude 'private' libraries from getting added to automatic runtime dependency resolution. This variable currently has to list all libraries by name, which becomes a maintenance issue if the list of such libraries frequently changes, or is very large.This change allows using shell-style wildcards in the variable, similar to how FILES lists what gets packaged.
Signed-off-by Alexander Kanavin <[email protected]>--- meta/classes/package.bbclass | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)meta/classes/package.bbclass +0
-0
meta/classes/package.bbclass python package_do_shlibs() { · line 1650
Quoted 6 lines
meta/classes/package.bbclass +2
-1@@ -1646,7 +1646,8 @@ python package_do_shlibs() { prov = (this_soname, ldir, pkgver) if not prov in sonames: # if library is private (only used by package) then do not build shlib for it - if not private_libs or this_soname not in private_libs: + import fnmatch + if not private_libs or len([i for i in private_libs if fnmatch.fnmatch(this_soname, i)]) == 0:
We only need to know if this_soname is matching one of the patterns, we don't need to check for each and every pattern.
Something like:
for pattern in private_libs:
if fnmatch.fnmatch(this_soname, pattern)]) == 0: sonames.add(prov) break
would be possible?
AFAICT, private_libs is either the content of PRIVATE_LIBS_PN or PRIVATE_LIBS or "" then passed to .split(). This should be safe for the forloop without a check.
meta/classes/package.bbclass python package_do_shlibs() { · line 1834
Quoted 10 lines
meta/classes/package.bbclass +2
-1sonames.add(prov) if libdir_re.match(os.path.dirname(file)): needs_ldconfig = True@@ -1829,7 +1830,8 @@ python package_do_shlibs() { # /opt/abc/lib/libfoo.so.1 and contains /usr/bin/abc depending on system library libfoo.so.1 # but skipping it is still better alternative than providing own # version and then adding runtime dependency for the same system library - if private_libs and n[0] in private_libs: + import fnmatch + if private_libs and len([i for i in private_libs if fnmatch.fnmatch(n[0], i)]) > 0:
Same here?
Just a suggestion :)
BR,
Quentin
Signature · 3 lines
StreamUnlimited Engineering GmbH High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria [email protected], www.streamunlimited.com
Quoted 11 lines · Quentin Schulz
Quoted 1 line
# if library is private (only used by package) thendo not build shlib for it
Quoted 1 line
- if not
private_libsorthis_sonamenot in
private_libs:Review
Quoted 2 lines
(quoted diff) +2
-0+ import fnmatch + if not private_libs or len([i for i in private_libsif
fnmatch.fnmatch(this_soname, i)]) == 0:We only need to know if
this_sonameis matching one of the patterns, we don't need to check for each and every pattern.
Quoted 9 lines
Something like:
for pattern in
private_libs:if fnmatch.fnmatch(this_soname, pattern)]) == 0: sonames.add(prov) breakwould be possible?
Yes, although there should only be very few patterns, and most of the time just one: *.so*. Not worth it I'd say. There are other spots in do_package which truly slow down things and could use a bit of optimization.
Alex
Quoted 15 lines · Quentin Schulz
meta/classes/package.bbclass python package_do_shlibs() { · line 1648
Quoted 5 lines
+++ b/meta/classes/package.bbclass
meta/classes/package.bbclass +0
-0@@ -1646,7 +1646,8 @@ python package_do_shlibs() { prov = (this_soname, ldir, pkgver) if not prov in sonames: # if library is private (only used by package) thendo not build shlib for it
Review python package_do_shlibs() { · line 1649
Quoted 1 line
(quoted diff) +0
-1- if not private_libs or this_soname not in
private_libs:Review python package_do_shlibs() { · line 1650
Quoted 2 lines
(quoted diff) +2
-0+ import fnmatch + if not private_libs or len([i for i in private_libsif
fnmatch.fnmatch(this_soname, i)]) == 0:We only need to know if
this_sonameis matching one of the patterns, we don't need to check for each and every pattern.
Quoted 9 lines
Something like:
for pattern in
private_libs:if fnmatch.fnmatch(this_soname, pattern)]) == 0: sonames.add(prov) breakwould be possible?
Actually, this suggested code snippet looks altogether wrong to me. We do need to check the file against every pattern, and only if it matches none of them, it gets added to sonames. Note the original code says 'this_soname not in private_libs', which also means going over every item in the list.
Alex
Hi Alex,
Quoted 25 lines · Alexander Kanavin
Quoted 22 lines · Quentin Schulz
meta/classes/package.bbclass python package_do_shlibs() { · line 1648
Quoted 5 lines
+++ b/meta/classes/package.bbclass
meta/classes/package.bbclass +0
-0@@ -1646,7 +1646,8 @@ python package_do_shlibs() { prov = (this_soname, ldir, pkgver) if not prov in sonames: # if library is private (only used by package) thendo not build shlib for it
Review python package_do_shlibs() { · line 1649
Quoted 1 line
(quoted diff) +0
-1- if not private_libs or this_soname not in
private_libs:Review python package_do_shlibs() { · line 1650
Quoted 2 lines
(quoted diff) +2
-0+ import fnmatch + if not private_libs or len([i for i in private_libsif
fnmatch.fnmatch(this_soname, i)]) == 0:We only need to know if
this_sonameis matching one of the patterns, we don't need to check for each and every pattern.Something like:
for pattern in
private_libs:if fnmatch.fnmatch(this_soname, pattern)]) == 0: sonames.add(prov) break
Quoted 4 lines
Quoted 2 lines
would be possible?
Actually, this suggested code snippet looks altogether wrong to me. We do
Indeed.
Quoted 2 lines
need to check the file against every pattern, and only if it matches none of them, it gets added to sonames. Note the original code says '
this_soname
So we just need to invert the logic actually.
matched = False
for pattern in private_libs:
if fnmatch.fnmatch(this_soname, pattern)]) == 0: matched = True break
if not matched:
sonames.add(prov)
Quoted 2 lines
not in
private_libs', which also means going over every item in the list.
I'd hope Python devs did optimization on this one and stopped at the first match in the list, but I'm not one of them so...
BR,
Quentin
Signature · 3 lines
StreamUnlimited Engineering GmbH High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria [email protected], www.streamunlimited.com
Quoted 11 lines · Quentin Schulz
So we just need to invert the logic actually.
matched = False
for pattern inprivate_libs:
iffnmatch.fnmatch(this_soname, pattern)]) == 0:
matched = True
breakif not matched:
sonames.add(prov)
I think you are looking for the "for...else" feature in Python :) Still, the original code is not adding any lines at all while yours adds eight so I'd just keep the patch as it is.
Alex
Hi Alex,
Quoted 11 lines · Alexander Kanavin
Quoted 8 lines · Quentin Schulz
So we just need to invert the logic actually.
matched = False
for pattern inprivate_libs:
iffnmatch.fnmatch(this_soname, pattern)]) == 0:
matched = True
break
Quoted 5 lines
Quoted 3 lines
if not matched:
sonames.add(prov)I think you are looking for the "for...else" feature in Python :) Still,
I'm discovering this one, thanks!
Quoted 3 lines
the original code is not adding any lines at all while yours adds eight so I'd just keep the patch as it is.
Added lines vs added execution time.
Anyway, glad to have this feature one day in Yocto, I was about to use it in some project and was upset for having to name each and every lib in the package.
Thanks!
Quentin
Signature · 3 lines
StreamUnlimited Engineering GmbH High Tech Campus Vienna, Gutheil-Schoder-Gasse 10, 1100 Vienna, Austria [email protected], www.streamunlimited.com
Quoted 7 lines · Quentin Schulz
Quoted 1 line
the original code is not adding any lines at all while yours adds eight
so
Quoted 2 lines
I'd just keep the patch as it is.
Added lines vs added execution time.
I am fairly certain do_package spends the bulk of its time elsewhere, so you are really optimizing the wrong spot here :)
Alex