Skip to contentSkip to conversation

multilib/recipes: Use new RecipePostKeyExpansion event

PATCH In review Richard Purdie 3 participants 8 messages May 26, 2020 – May 27, 2020

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/5 multilib/recipes: Use new RecipePostKeyExpansion event

There are issues with multilib due to the ordering of events where some functions see the remapped multilib dependencies and some do not. A significant problem is that the multilib class needs to make some changes before key expansion and some afterwards but by using existing event handlers, some code sees things in a partially translated state, leading to bugs.

This patch changes things to use a new event handler from bitbake which makes the ordering of the changes explcit.

The challenge in doing this is that it breaks some existing anonymous python and dyanmic assignments. In some cases these used to be translated and no longer are, meaning MLPREFIX has to be added. In some cases these are now translated and the MLPREFIX can be removed.

This change does now make it very clear when MLPREFIX is required and when it is not, its just the migration path which is harder. The patch changes the small number of cases where fixes are needed.

In particular, where a variable like RDEPENDS is conditionally extended (e.g. with an override), MLPREFIX is now required.

This patch also reverts:
base: Revert 'base.bbclass: considering multilib when setting LICENSE_EXCLUSION'

This reverts 6597130256a1609c3e05ec5891aceaf549c37985 as the changes to multilib datastore handling mean its no longer necessary.

---
 meta/classes/base.bbclass                     |  3 +-
 meta/classes/fontcache.bbclass                |  2 +-
 meta/classes/multilib.bbclass                 | 28 ++++++++++++---
 meta/lib/oe/classextend.py                    | 35 +++++++++++++++++--
 meta/recipes-core/glibc/glibc-package.inc     |  2 +-
 .../packagegroups/packagegroup-base.bb        |  8 ++---
 meta/recipes-core/psplash/psplash_git.bb      |  5 +--
 meta/recipes-devtools/perl/perl_5.30.2.bb     |  2 +-
 meta/recipes-devtools/python/python3_3.8.2.bb | 14 ++++----
 .../packagegroup-core-full-cmdline.bb         | 10 +++---
 meta/recipes-graphics/mesa/mesa.inc           | 10 +++---
 .../alsa/alsa-plugins_1.2.1.bb                |  6 ++--
 meta/recipes-support/boost/boost.inc          | 10 ++++--
 13 files changed, 95 insertions(+), 40 deletions(-)
meta/classes/base.bbclass +1 -2
index 7aa2e144eb7..4c681cc870d 100644
@@ -570,8 +570,7 @@ python () {
                 if unskipped_pkgs:
                     for pkg in skipped_pkgs:
                         bb.debug(1, "Skipping the package %s at do_rootfs because of incompatible license(s): %s" % (pkg, ' '.join(skipped_pkgs[pkg])))
-                        mlprefix = d.getVar('MLPREFIX')
-                        d.setVar('LICENSE_EXCLUSION-' + mlprefix + pkg, ' '.join(skipped_pkgs[pkg]))
+                        d.setVar('LICENSE_EXCLUSION-' + pkg, ' '.join(skipped_pkgs[pkg]))
                     for pkg in unskipped_pkgs:
                         bb.debug(1, "Including the package %s" % pkg)
                 else:
meta/classes/fontcache.bbclass +1 -1
index 97e7f17f00e..624a420a0de 100644
@@ -7,7 +7,7 @@ PACKAGE_WRITE_DEPS += "qemu-native"
 inherit qemu
 
 FONT_PACKAGES ??= "${PN}"
-FONT_EXTRA_RDEPENDS ?= "fontconfig-utils"
+FONT_EXTRA_RDEPENDS ?= "${MLPREFIX}fontconfig-utils"
 FONTCONFIG_CACHE_DIR ?= "${localstatedir}/cache/fontconfig"
 FONTCONFIG_CACHE_PARAMS ?= "-v"
 # You can change this to e.g. FC_DEBUG=16 to debug fc-cache issues,
meta/classes/multilib.bbclass +23 -5
index ee677da1e20..9f726e45371 100644
@@ -91,13 +91,12 @@ addhandler multilib_virtclass_handler
 multilib_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
 
 python __anonymous () {
-    variant = d.getVar("BBEXTENDVARIANT")
-
-    import oe.classextend
+    if bb.data.inherits_class('image', d):
+        variant = d.getVar("BBEXTENDVARIANT")
+        import oe.classextend
 
-    clsextend = oe.classextend.ClassExtender(variant, d)
+        clsextend = oe.classextend.ClassExtender(variant, d)
 
-    if bb.data.inherits_class('image', d):
         clsextend.map_depends_variable("PACKAGE_INSTALL")
         clsextend.map_depends_variable("LINGUAS_INSTALL")
         clsextend.map_depends_variable("RDEPENDS")
@@ -109,6 +108,22 @@ python __anonymous () {
         bb.build.deltask('do_populate_sdk', d)
         bb.build.deltask('do_populate_sdk_ext', d)
         return
+}
+
+python multilib_virtclass_handler_postkeyexp () {
+    cls = d.getVar("BBEXTENDCURR")
+    variant = d.getVar("BBEXTENDVARIANT")
+    if cls != "multilib" or not variant:
+        return
+
+    variant = d.getVar("BBEXTENDVARIANT")
+
+    import oe.classextend
+
+    clsextend = oe.classextend.ClassExtender(variant, d)
+
+    if bb.data.inherits_class('image', d):
+        return
 
     clsextend.map_depends_variable("DEPENDS")
     clsextend.map_variable("PROVIDES")
@@ -129,6 +144,9 @@ python __anonymous () {
     reset_alternative_priority(d)
 }
 
+addhandler multilib_virtclass_handler_postkeyexp
+multilib_virtclass_handler_postkeyexp[eventmask] = "bb.event.RecipePostKeyExpansion"
+
 def reset_alternative_priority(d):
     if not bb.data.inherits_class('update-alternatives', d):
         return
meta/lib/oe/classextend.py +32 -3
index f02fbe9fbad..e1049ce3e88 100644
@@ -4,11 +4,21 @@
 
 import collections
 
+def get_packages(d):
+    pkgs = d.getVar("PACKAGES_NONML")
+    extcls = d.getVar("EXTENDERCLASS")
+    return extcls.rename_packages_internal(pkgs)
+
+def get_depends(varprefix, d):
+    extcls = d.getVar("EXTENDERCLASS")
+    return extcls.map_depends_variable(varprefix + "_NONML")
+
 class ClassExtender(object):
     def __init__(self, extname, d):
         self.extname = extname
         self.d = d
         self.pkgs_mapping = []
+        self.d.setVar("EXTENDERCLASS", self)
 
     def extend_name(self, name):
         if name.startswith("kernel-") or name == "virtual/kernel":
@@ -24,7 +34,7 @@ class ClassExtender(object):
             if not subs.startswith(self.extname):
                 return "virtual/" + self.extname + "-" + subs
             return name
-        if name.startswith("/"):
+        if name.startswith("/") or (name.startswith("${") and name.endswith("}")):
             return name
         if not name.startswith(self.extname):
             return self.extname + "-" + name
@@ -89,8 +99,14 @@ class ClassExtender(object):
         for dep in deps:
             newdeps[self.map_depends(dep)] = deps[dep]
 
-        self.d.setVar(varname, bb.utils.join_deps(newdeps, False).replace("EXTENDPKGV", "${EXTENDPKGV}"))
+        if not varname.endswith("_NONML"):
+            #if varname == "DEPENDS":
+            self.d.renameVar(varname, varname + "_NONML")
+            self.d.setVar(varname, "${@oe.classextend.get_depends('%s', d)}" % varname)
+            self.d.appendVarFlag(varname, "vardeps", " " + varname + "_NONML")
+        ret = bb.utils.join_deps(newdeps, False).replace("EXTENDPKGV", "${EXTENDPKGV}")
         self.d.setVar("EXTENDPKGV", orig)
+        return ret
 
     def map_packagevars(self):
         for pkg in (self.d.getVar("PACKAGES").split() + [""]):
@@ -109,10 +125,23 @@ class ClassExtender(object):
                continue
             self.pkgs_mapping.append([pkg, self.extend_name(pkg)])
 
-        self.d.setVar("PACKAGES", " ".join([row[1] for row in self.pkgs_mapping]))
+        self.d.renameVar("PACKAGES", "PACKAGES_NONML")
+        self.d.setVar("PACKAGES", "${@oe.classextend.get_packages(d)}")
+
+    def rename_packages_internal(self, pkgs):
+        self.pkgs_mapping = []
+        for pkg in (self.d.expand(pkgs) or "").split():
+            if pkg.startswith(self.extname):
+               self.pkgs_mapping.append([pkg.split(self.extname + "-")[1], pkg])
+               continue
+            self.pkgs_mapping.append([pkg, self.extend_name(pkg)])
+
+        return " ".join([row[1] for row in self.pkgs_mapping])
 
     def rename_package_variables(self, variables):
         for pkg_mapping in self.pkgs_mapping:
+            if pkg_mapping[0].startswith("${") and pkg_mapping[0].endswith("}"):
+                continue
             for subs in variables:
                 self.d.renameVar("%s_%s" % (subs, pkg_mapping[0]), "%s_%s" % (subs, pkg_mapping[1]))
 
meta/recipes-core/glibc/glibc-package.inc +1 -1
index aa8e0592169..ff25fd41875 100644
@@ -24,7 +24,7 @@ libc_baselibs_append = " ${@oe.utils.conditional('ARCH_DYNAMIC_LOADER', '', '',
 INSANE_SKIP_${PN}_append_aarch64 = " libdir"
 
 FILES_${PN} = "${libc_baselibs} ${libexecdir}/*"
-RRECOMMENDS_${PN} = "${@bb.utils.filter('DISTRO_FEATURES', 'ldconfig', d)}"
+RRECOMMENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'ldconfig', '${MLPREFIX}ldconfig', '', d)}"
 FILES_ldconfig = "${base_sbindir}/ldconfig ${sysconfdir}/ld.so.conf"
 FILES_ldd = "${bindir}/ldd"
 FILES_libsegfault = "${base_libdir}/libSegFault*"
meta/recipes-core/packagegroups/packagegroup-base.bb +4 -4
index 1f802da09b7..90b79adfdcd 100644
@@ -110,16 +110,16 @@ python __anonymous () {
     machine_features= set(d.getVar("MACHINE_FEATURES").split())
 
     if "bluetooth" in distro_features and not "bluetooth" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
-        d.setVar("ADD_BT", "packagegroup-base-bluetooth")
+        d.setVar("ADD_BT", "${MLPREFIX}packagegroup-base-bluetooth")
 
     if "wifi" in distro_features and not "wifi" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
-        d.setVar("ADD_WIFI", "packagegroup-base-wifi")
+        d.setVar("ADD_WIFI", "${MLPREFIX}packagegroup-base-wifi")
 
     if "3g" in distro_features and not "3g" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
-        d.setVar("ADD_3G", "packagegroup-base-3g")
+        d.setVar("ADD_3G", "${MLPREFIX}packagegroup-base-3g")
 
     if "nfc" in distro_features and not "nfc" in machine_features and ("usbhost" in machine_features):
-        d.setVar("ADD_NFC", "packagegroup-base-nfc")
+        d.setVar("ADD_NFC", "${MLPREFIX}packagegroup-base-nfc")
 }
 
 #
meta/recipes-core/psplash/psplash_git.bb +3 -2
index 22c71f099b8..44f0007daf0 100644
@@ -22,6 +22,7 @@ SPLASH_IMAGES = "file://psplash-poky-img.h;outsuffix=default"
 python __anonymous() {
     oldpkgs = d.getVar("PACKAGES").split()
     splashfiles = d.getVar('SPLASH_IMAGES').split()
+    mlprefix = d.getVar('MLPREFIX') or ''
     pkgs = []
     localpaths = []
     for uri in splashfiles:
@@ -46,9 +47,9 @@ python __anonymous() {
     # Set these so that we have less work to do in do_compile and do_install_append
     d.setVar("SPLASH_INSTALL", " ".join(pkgs))
     d.setVar("SPLASH_LOCALPATHS", " ".join(localpaths))
+    for p in pkgs:
+        d.prependVar("PACKAGES", "%s%s " % (mlprefix, p))
 
-    d.prependVar("PACKAGES", "%s " % (" ".join(pkgs)))
-    mlprefix = d.getVar('MLPREFIX') or ''
     pn = d.getVar('PN') or ''
     for p in pkgs:
         ep = '%s%s' % (mlprefix, p)
meta/recipes-devtools/perl/perl_5.30.2.bb +1 -1
index 778c420b2ee..26138ea9e55 100644
@@ -328,7 +328,7 @@ python split_perl_packages () {
 
 python() {
     if d.getVar('CLASSOVERRIDE') == "class-target":
-        d.setVar("PACKAGES_DYNAMIC", "^perl-module-.*(?<!native)$")
+        d.setVar("PACKAGES_DYNAMIC", "^${MLPREFIX}perl-module-.*(?<!native)$")
     elif d.getVar('CLASSOVERRIDE') == "class-native":
         d.setVar("PACKAGES_DYNAMIC", "^perl-module-.*-native$")
     elif d.getVar('CLASSOVERRIDE') == "class-nativesdk":
meta/recipes-devtools/python/python3_3.8.2.bb +7 -7
index a4a16fd495f..0474f07214f 100644
@@ -311,8 +311,8 @@ do_create_manifest[depends] += "${PN}:do_patch"
 
 # manual dependency additions
 RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules"
-RRECOMMENDS_${PN}-crypt_append_class-target = " openssl ca-certificates"
-RRECOMMENDS_${PN}-crypt_append_class-nativesdk = " openssl ca-certificates"
+RRECOMMENDS_${PN}-crypt_append_class-target = " ${MLPREFIX}openssl ${MLPREFIX}ca-certificates"
+RRECOMMENDS_${PN}-crypt_append_class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX}ca-certificates"
 
 # For historical reasons PN is empty and provided by python3-modules
 FILES_${PN} = ""
@@ -322,7 +322,7 @@ FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
 FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}"
 
 # provide python-pyvenv from python3-venv
-RPROVIDES_${PN}-venv += "python3-pyvenv"
+RPROVIDES_${PN}-venv += "${MLPREFIX}python3-pyvenv"
 
 # package libpython3
 PACKAGES =+ "libpython3 libpython3-staticdev"
@@ -333,8 +333,8 @@ INSANE_SKIP_${PN}-dev += "dev-elf"
 # catch all the rest (unsorted)
 PACKAGES += "${PN}-misc"
 RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs python3-pydoc python3-pickle python3-audio"
-RDEPENDS_${PN}-modules_append_class-target = " python3-misc"
-RDEPENDS_${PN}-modules_append_class-nativesdk = " python3-misc"
+RDEPENDS_${PN}-modules_append_class-target = " ${MLPREFIX}python3-misc"
+RDEPENDS_${PN}-modules_append_class-nativesdk = " ${MLPREFIX}python3-misc"
 FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN} ${libdir}/python${PYTHON_MAJMIN}/lib-dynload"
 
 # catch manpage
@@ -348,5 +348,5 @@ RDEPENDS_${PN}-ptest_append_libc-glibc = " locale-base-tr-tr.iso-8859-9"
 RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk tk-lib', '', d)}"
 RDEPENDS_${PN}-dev = ""
 
-RDEPENDS_${PN}-tests_append_class-target = " bash"
-RDEPENDS_${PN}-tests_append_class-nativesdk = " bash"
+RDEPENDS_${PN}-tests_append_class-target = " ${MLPREFIX}bash"
+RDEPENDS_${PN}-tests_append_class-nativesdk = " ${MLPREFIX}bash"
meta/recipes-extended/packagegroups/packagegroup-core-full-cmdline.bb +6 -4
index 15a8e6dedc3..16c2f9f2aa9 100644
@@ -32,21 +32,23 @@ python __anonymous () {
     namemap["packagegroup-core-full-cmdline-sys-services"] = "packagegroup-core-sys-services"
 
     packages = d.getVar("PACKAGES").split()
+    mlprefix = d.getVar("MLPREFIX")
     for pkg in packages:
+        pkg2 = pkg[len(mlprefix):]
         if pkg.endswith('-dev'):
-            mapped = namemap.get(pkg[:-4], None)
+            mapped = namemap.get(pkg2[:-4], None)
             if mapped:
                 mapped += '-dev'
         elif pkg.endswith('-dbg'):
-            mapped = namemap.get(pkg[:-4], None)
+            mapped = namemap.get(pkg2[:-4], None)
             if mapped:
                 mapped += '-dbg'
         else:
-            mapped = namemap.get(pkg, None)
+            mapped = namemap.get(pkg2, None)
 
         if mapped:
             oldtaskname = mapped.replace("packagegroup-core", "task-core")
-            mapstr = " %s %s" % (mapped, oldtaskname)
+            mapstr = " %s%s %s%s" % (mlprefix, mapped, mlprefix, oldtaskname)
             d.appendVar("RPROVIDES_%s" % pkg, mapstr)
             d.appendVar("RREPLACES_%s" % pkg, mapstr)
             d.appendVar("RCONFLICTS_%s" % pkg, mapstr)
meta/recipes-graphics/mesa/mesa.inc +6 -4
index fede691d6ff..bb43a9a8b65 100644
@@ -212,18 +212,20 @@ python __anonymous() {
               ("gles", "libgles3",)):
         if not p[0] in pkgconfig:
             continue
-        fullp = p[1] + "-mesa"
-        pkgs = " ".join(p[1:])
+        mlprefix = d.getVar("MLPREFIX")
+        fullp = mlprefix + p[1] + "-mesa"
+        mlprefix = d.getVar("MLPREFIX")
+        pkgs = " ".join(mlprefix + x for x in p[1:])
         d.setVar("DEBIAN_NOAUTONAME_" + fullp, "1")
         d.appendVar("RREPLACES_" + fullp, pkgs)
         d.appendVar("RPROVIDES_" + fullp, pkgs)
         d.appendVar("RCONFLICTS_" + fullp, pkgs)
 
-        d.appendVar("RRECOMMENDS_" + fullp, " mesa-megadriver")
+        d.appendVar("RRECOMMENDS_" + fullp, " ${MLPREFIX}mesa-megadriver")
 
         # For -dev, the first element is both the Debian and original name
         fullp += "-dev"
-        pkgs = p[1] + "-dev"
+        pkgs = mlprefix + p[1] + "-dev"
         d.setVar("DEBIAN_NOAUTONAME_" + fullp, "1")
         d.appendVar("RREPLACES_" + fullp, pkgs)
         d.appendVar("RPROVIDES_" + fullp, pkgs)
meta/recipes-multimedia/alsa/alsa-plugins_1.2.1.bb +3 -3
index 9882e12763a..d092b158f26 100644
@@ -167,7 +167,7 @@ FILES_${PN}-pulseaudio-conf += "\
 "
 
 RDEPENDS_${PN}-pulseaudio-conf += "\
-        libasound-module-conf-pulse \
-        libasound-module-ctl-pulse \
-        libasound-module-pcm-pulse \
+        ${MLPREFIX}libasound-module-conf-pulse \
+        ${MLPREFIX}libasound-module-ctl-pulse \
+        ${MLPREFIX}libasound-module-pcm-pulse \
 "
meta/recipes-support/boost/boost.inc +7 -3
index 8eb9494381f..ca140d595fd 100644
@@ -62,12 +62,16 @@ PACKAGES = "${PN}-dbg ${BOOST_PACKAGES}"
 python __anonymous () {
     packages = []
     extras = []
+    mlprefix = d.getVar("MLPREFIX")
     for lib in d.getVar('BOOST_LIBS').split():
         extras.append("--with-%s" % lib)
-        pkg = "boost-%s" % lib.replace("_", "-")
-        packages.append(pkg)
+        pkg = "boost-%s" % (lib.replace("_", "-"))
+        packages.append(mlprefix + pkg)
         if not d.getVar("FILES_%s" % pkg):
-                d.setVar("FILES_%s" % pkg, "${libdir}/libboost_%s*.so.*" % lib)
+                d.setVar("FILES_%s%s" % (mlprefix, pkg), "${libdir}/libboost_%s*.so.*" % lib)
+        else:
+                d.setVar("FILES_%s%s" % (mlprefix, pkg), d.getVar("FILES_%s" % pkg))
+
     d.setVar("BOOST_PACKAGES", " ".join(packages))
     d.setVar("BJAM_EXTRA", " ".join(extras))
 }
Signature · 1 line
2.25.1
#1 lore raw

PATCH 2/5 ltp: Add missing dependencies on coreutils, bc, e2fsprogs and gdb

When the tests are run we see messages like:

/opt/ltp/testcases/bin/run_cpuctl_stress_test.sh: line 242: nice: command not found /opt/ltp/testcases/bin/run_cpuctl_test_fj.sh: line 66: tac: command not found vma05 1 TCONF: 'gdb' not found
memcg_failcnt 1 TCONF: 'bc' not found
Owner=nobody; perms=-rw-------; sudo: lsattr: command not found

so add missing dependencies to avoid these.

---
 meta/recipes-extended/ltp/ltp_20200515.bb | 4 ++++
 1 file changed, 4 insertions(+)
meta/recipes-extended/ltp/ltp_20200515.bb +4 -0
index 317a4e461a7..0304a40185b 100644
@@ -77,13 +77,17 @@ do_install(){
 RDEPENDS_${PN} = "\
     attr \
     bash \
+    bc \
+    coreutils \
     cpio \
     cronie \
     curl \
+    e2fsprogs \
     e2fsprogs-mke2fs \
     expect \
     file \
     gawk \
+    gdb \
     gzip \
     iproute2 \
     ldd \
Signature · 1 line
2.25.1
#2 lore raw

PATCH 3/5 resulttool/report: Remove leftover debugging

In reply to Richard Purdie

I've long since wondered why there was some odd output in result reports, remove the leftover debug which was causing it.

---
 scripts/lib/resulttool/report.py | 1 -
 1 file changed, 1 deletion(-)
scripts/lib/resulttool/report.py +0 -1
index 7ceceac802c..f0ca50ebe24 100644
@@ -96,7 +96,6 @@ class ResultsTextReport(object):
         if 'ltpresult.sections' in result and suite not in result['ltpresult.sections']:
             try:
                 _, suite, suite1, test = k.split(".", 3)
-                print("split2: %s %s %s" % (suite, suite1, test))
                 if suite + "." + suite1 in result['ltpresult.sections']:
                     suite = suite + "." + suite1
             except ValueError:
Signature · 1 line
2.25.1
#3 lore raw

PATCH 4/5 resulttool/log: Add ability to dump ltp logs as well as ptest

In reply to Richard Purdie

Currently only ptest logs are accessible with the log command, this adds support so the ltp logs can be extracted too.

---
 scripts/lib/resulttool/log.py         | 21 ++++++++++++++-------
 scripts/lib/resulttool/resultutils.py | 22 ++++++++++++++--------
 2 files changed, 28 insertions(+), 15 deletions(-)
scripts/lib/resulttool/log.py +14 -7
index f1bfd99500b..eb3927ec82a 100644
@@ -34,13 +34,17 @@ def log(args, logger):
         return 1
 
     for _, run_name, _, r in resultutils.test_run_results(results):
-        if args.dump_ptest and 'ptestresult.sections' in r:
-            for name, ptest in r['ptestresult.sections'].items():
-                logdata = resultutils.ptestresult_get_log(r, name)
+        if args.dump_ptest:
+            for sectname in ['ptestresult.sections', 'ltpposixresult.sections', 'ltpresult.sections']:
+             if sectname in r:
+              for name, ptest in r[sectname].items():
+                logdata = resultutils.generic_get_log(sectname, r, name)
                 if logdata is not None:
                     dest_dir = args.dump_ptest
                     if args.prepend_run:
                         dest_dir = os.path.join(dest_dir, run_name)
+                    if not sectname.startswith("ptest"):
+                        dest_dir = os.path.join(dest_dir, sectname.split(".")[0])
 
                     os.makedirs(dest_dir, exist_ok=True)
                     dest = os.path.join(dest_dir, '%s.log' % name)
@@ -49,10 +53,13 @@ def log(args, logger):
                         f.write(logdata)
 
         if args.raw_ptest:
-            rawlog = resultutils.ptestresult_get_rawlogs(r)
-            if rawlog is not None:
-                print(rawlog)
-            else:
+            found = False
+            for sectname in ['ptestresult.rawlogs', 'ltpposixresult.rawlogs', 'ltpresult.rawlogs']:
+                rawlog = resultutils.generic_get_rawlogs(sectname, r)
+                if rawlog is not None:
+                    print(rawlog)
+                    found = True
+            if not found:
                 print('Raw ptest logs not found')
                 return 1
 
scripts/lib/resulttool/resultutils.py +14 -8
index 5fec01f6f34..8917022d36e 100644
@@ -130,23 +130,29 @@ def decode_log(logdata):
             return data.decode("utf-8", errors='ignore')
     return None
 
-def ptestresult_get_log(results, section):
-    if 'ptestresult.sections' not in results:
+def generic_get_log(sectionname, results, section):
+    if sectionname not in results:
         return None
-    if section not in results['ptestresult.sections']:
+    if section not in results[sectionname]:
         return None
 
-    ptest = results['ptestresult.sections'][section]
+    ptest = results[sectionname][section]
     if 'log' not in ptest:
         return None
     return decode_log(ptest['log'])
 
-def ptestresult_get_rawlogs(results):
-    if 'ptestresult.rawlogs' not in results:
+def ptestresult_get_log(results, section):
+    return generic_get_log('ptestresuls.sections', results, section)
+
+def generic_get_rawlogs(sectname, results):
+    if sectname not in results:
         return None
-    if 'log' not in results['ptestresult.rawlogs']:
+    if 'log' not in results[sectname]:
         return None
-    return decode_log(results['ptestresult.rawlogs']['log'])
+    return decode_log(results[sectname]['log'])
+
+def ptestresult_get_rawlogs(results):
+    return generic_get_rawlogs('ptestresult.rawlogs', results)
 
 def save_resultsdata(results, destdir, fn="testresults.json", ptestjson=False, ptestlogs=False):
     for res in results:
Signature · 1 line
2.25.1
#4 lore raw

PATCH 5/5 ltp: Exclude the memcg_stress tests due to timeout problems

In reply to Richard Purdie

This test runs for 900s, we often see tests killed after 300s without output which makes the test results unreliable and inconsistent. The easiest solution for now is to skip this long running test, patching it out wth sed.

---
 meta/recipes-extended/ltp/ltp_20200515.bb | 4 ++++
 1 file changed, 4 insertions(+)
meta/recipes-extended/ltp/ltp_20200515.bb +4 -0
index 0304a40185b..67c75903204 100644
@@ -72,6 +72,10 @@ do_install(){
          -e 's@[^ ]*-fdebug-prefix-map=[^ "]*@@g' \
          -e 's@[^ ]*-fmacro-prefix-map=[^ "]*@@g' \
          -e 's@[^ ]*--sysroot=[^ "]*@@g' 
+
+    # The controllers memcg_stree test seems to cause us hangs and takes 900s
+    # (maybe we expect more regular output?), anyhow, skip it
+    sed -e '/^memcg_stress/d' -i ${D}${prefix}/runtest/controllers
 }
 
 RDEPENDS_${PN} = "\
Signature · 1 line
2.25.1
#5 lore raw
In reply to Richard Purdie

The one question I did have is there are a couple of places where you add the ML prefix for nativesdk classes. If we don't have them, it shouldn't hurt anything, but do we actually support multilib nativesdk packages?

Quoted 424 lines · Richard Purdie

There are issues with multilib due to the ordering of events where some functions see the remapped multilib dependencies and some do not. A significant problem is that the multilib class needs to make some changes before key expansion and some afterwards but by using existing event handlers, some code sees things in a partially translated state, leading to bugs.

This patch changes things to use a new event handler from bitbake which makes the ordering of the changes explcit.

The challenge in doing this is that it breaks some existing anonymous python and dyanmic assignments. In some cases these used to be translated and no longer are, meaning MLPREFIX has to be added. In some cases these are now translated and the MLPREFIX can be removed.

This change does now make it very clear when MLPREFIX is required and when it is not, its just the migration path which is harder. The patch changes the small number of cases where fixes are needed.

In particular, where a variable like RDEPENDS is conditionally extended (e.g. with an override), MLPREFIX is now required.

This patch also reverts:
base: Revert 'base.bbclass: considering multilib when setting LICENSE_EXCLUSION'

This reverts 6597130256a1609c3e05ec5891aceaf549c37985 as the changes to multilib datastore handling mean its no longer necessary.

---
  meta/classes/base.bbclass                     |  3 +-
  meta/classes/fontcache.bbclass                |  2 +-
  meta/classes/multilib.bbclass                 | 28 ++++++++++++---
  meta/lib/oe/classextend.py                    | 35 +++++++++++++++++--
  meta/recipes-core/glibc/glibc-package.inc     |  2 +-
  .../packagegroups/packagegroup-base.bb        |  8 ++---
  meta/recipes-core/psplash/psplash_git.bb      |  5 +--
  meta/recipes-devtools/perl/perl_5.30.2.bb     |  2 +-
  meta/recipes-devtools/python/python3_3.8.2.bb | 14 ++++----
  .../packagegroup-core-full-cmdline.bb         | 10 +++---
  meta/recipes-graphics/mesa/mesa.inc           | 10 +++---
  .../alsa/alsa-plugins_1.2.1.bb                |  6 ++--
  meta/recipes-support/boost/boost.inc          | 10 ++++--
  13 files changed, 95 insertions(+), 40 deletions(-)
meta/classes/base.bbclass +1 -2
index 7aa2e144eb7..4c681cc870d 100644
@@ -570,8 +570,7 @@ python () {
                  if unskipped_pkgs:
                      for pkg in skipped_pkgs:
                          bb.debug(1, "Skipping the package %s at do_rootfs because of incompatible license(s): %s" % (pkg, ' '.join(skipped_pkgs[pkg])))
-                        mlprefix = d.getVar('MLPREFIX')
-                        d.setVar('LICENSE_EXCLUSION-' + mlprefix + pkg, ' '.join(skipped_pkgs[pkg]))
+                        d.setVar('LICENSE_EXCLUSION-' + pkg, ' '.join(skipped_pkgs[pkg]))
                      for pkg in unskipped_pkgs:
                          bb.debug(1, "Including the package %s" % pkg)
                  else:
meta/classes/fontcache.bbclass +1 -1
index 97e7f17f00e..624a420a0de 100644
@@ -7,7 +7,7 @@ PACKAGE_WRITE_DEPS += "qemu-native"
  inherit qemu
 
  FONT_PACKAGES ??= "${PN}"
-FONT_EXTRA_RDEPENDS ?= "fontconfig-utils"
+FONT_EXTRA_RDEPENDS ?= "${MLPREFIX}fontconfig-utils"
  FONTCONFIG_CACHE_DIR ?= "${localstatedir}/cache/fontconfig"
  FONTCONFIG_CACHE_PARAMS ?= "-v"
  # You can change this to e.g. FC_DEBUG=16 to debug fc-cache issues,
meta/classes/multilib.bbclass +23 -5
index ee677da1e20..9f726e45371 100644
@@ -91,13 +91,12 @@ addhandler multilib_virtclass_handler
  multilib_virtclass_handler[eventmask] = "bb.event.RecipePreFinalise"
 
  python __anonymous () {
-    variant = d.getVar("BBEXTENDVARIANT")
-
-    import oe.classextend
+    if bb.data.inherits_class('image', d):
+        variant = d.getVar("BBEXTENDVARIANT")
+        import oe.classextend
 
-    clsextend = oe.classextend.ClassExtender(variant, d)
+        clsextend = oe.classextend.ClassExtender(variant, d)
 
-    if bb.data.inherits_class('image', d):
          clsextend.map_depends_variable("PACKAGE_INSTALL")
          clsextend.map_depends_variable("LINGUAS_INSTALL")
          clsextend.map_depends_variable("RDEPENDS")
@@ -109,6 +108,22 @@ python __anonymous () {
          bb.build.deltask('do_populate_sdk', d)
          bb.build.deltask('do_populate_sdk_ext', d)
          return
+}
+
+python multilib_virtclass_handler_postkeyexp () {
+    cls = d.getVar("BBEXTENDCURR")
+    variant = d.getVar("BBEXTENDVARIANT")
+    if cls != "multilib" or not variant:
+        return
+
+    variant = d.getVar("BBEXTENDVARIANT")
+
+    import oe.classextend
+
+    clsextend = oe.classextend.ClassExtender(variant, d)
+
+    if bb.data.inherits_class('image', d):
+        return
 
      clsextend.map_depends_variable("DEPENDS")
      clsextend.map_variable("PROVIDES")
@@ -129,6 +144,9 @@ python __anonymous () {
      reset_alternative_priority(d)
  }
 
+addhandler multilib_virtclass_handler_postkeyexp
+multilib_virtclass_handler_postkeyexp[eventmask] = "bb.event.RecipePostKeyExpansion"
+
  def reset_alternative_priority(d):
      if not bb.data.inherits_class('update-alternatives', d):
          return
meta/lib/oe/classextend.py +32 -3
index f02fbe9fbad..e1049ce3e88 100644
@@ -4,11 +4,21 @@
 
  import collections
 
+def get_packages(d):
+    pkgs = d.getVar("PACKAGES_NONML")
+    extcls = d.getVar("EXTENDERCLASS")
+    return extcls.rename_packages_internal(pkgs)
+
+def get_depends(varprefix, d):
+    extcls = d.getVar("EXTENDERCLASS")
+    return extcls.map_depends_variable(varprefix + "_NONML")
+
  class ClassExtender(object):
      def __init__(self, extname, d):
          self.extname = extname
          self.d = d
          self.pkgs_mapping = []
+        self.d.setVar("EXTENDERCLASS", self)
 
      def extend_name(self, name):
          if name.startswith("kernel-") or name == "virtual/kernel":
@@ -24,7 +34,7 @@ class ClassExtender(object):
              if not subs.startswith(self.extname):
                  return "virtual/" + self.extname + "-" + subs
              return name
-        if name.startswith("/"):
+        if name.startswith("/") or (name.startswith("${") and name.endswith("}")):
              return name
          if not name.startswith(self.extname):
              return self.extname + "-" + name
@@ -89,8 +99,14 @@ class ClassExtender(object):
          for dep in deps:
              newdeps[self.map_depends(dep)] = deps[dep]
 
-        self.d.setVar(varname, bb.utils.join_deps(newdeps, False).replace("EXTENDPKGV", "${EXTENDPKGV}"))
+        if not varname.endswith("_NONML"):
+            #if varname == "DEPENDS":
+            self.d.renameVar(varname, varname + "_NONML")
+            self.d.setVar(varname, "${@oe.classextend.get_depends('%s', d)}" % varname)
+            self.d.appendVarFlag(varname, "vardeps", " " + varname + "_NONML")
+        ret = bb.utils.join_deps(newdeps, False).replace("EXTENDPKGV", "${EXTENDPKGV}")
          self.d.setVar("EXTENDPKGV", orig)
+        return ret
 
      def map_packagevars(self):
          for pkg in (self.d.getVar("PACKAGES").split() + [""]):
@@ -109,10 +125,23 @@ class ClassExtender(object):
                 continue
              self.pkgs_mapping.append([pkg, self.extend_name(pkg)])
 
-        self.d.setVar("PACKAGES", " ".join([row[1] for row in self.pkgs_mapping]))
+        self.d.renameVar("PACKAGES", "PACKAGES_NONML")
+        self.d.setVar("PACKAGES", "${@oe.classextend.get_packages(d)}")
+
+    def rename_packages_internal(self, pkgs):
+        self.pkgs_mapping = []
+        for pkg in (self.d.expand(pkgs) or "").split():
+            if pkg.startswith(self.extname):
+               self.pkgs_mapping.append([pkg.split(self.extname + "-")[1], pkg])
+               continue
+            self.pkgs_mapping.append([pkg, self.extend_name(pkg)])
+
+        return " ".join([row[1] for row in self.pkgs_mapping])
 
      def rename_package_variables(self, variables):
          for pkg_mapping in self.pkgs_mapping:
+            if pkg_mapping[0].startswith("${") and pkg_mapping[0].endswith("}"):
+                continue
              for subs in variables:
                  self.d.renameVar("%s_%s" % (subs, pkg_mapping[0]), "%s_%s" % (subs, pkg_mapping[1]))
 
meta/recipes-core/glibc/glibc-package.inc +1 -1
index aa8e0592169..ff25fd41875 100644
@@ -24,7 +24,7 @@ libc_baselibs_append = " ${@oe.utils.conditional('ARCH_DYNAMIC_LOADER', '', '',
  INSANE_SKIP_${PN}_append_aarch64 = " libdir"
 
  FILES_${PN} = "${libc_baselibs} ${libexecdir}/*"
-RRECOMMENDS_${PN} = "${@bb.utils.filter('DISTRO_FEATURES', 'ldconfig', d)}"
+RRECOMMENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'ldconfig', '${MLPREFIX}ldconfig', '', d)}"
  FILES_ldconfig = "${base_sbindir}/ldconfig ${sysconfdir}/ld.so.conf"
  FILES_ldd = "${bindir}/ldd"
  FILES_libsegfault = "${base_libdir}/libSegFault*"
meta/recipes-core/packagegroups/packagegroup-base.bb +4 -4
index 1f802da09b7..90b79adfdcd 100644
@@ -110,16 +110,16 @@ python __anonymous () {
      machine_features= set(d.getVar("MACHINE_FEATURES").split())
 
      if "bluetooth" in distro_features and not "bluetooth" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
-        d.setVar("ADD_BT", "packagegroup-base-bluetooth")
+        d.setVar("ADD_BT", "${MLPREFIX}packagegroup-base-bluetooth")
 
      if "wifi" in distro_features and not "wifi" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
-        d.setVar("ADD_WIFI", "packagegroup-base-wifi")
+        d.setVar("ADD_WIFI", "${MLPREFIX}packagegroup-base-wifi")
 
      if "3g" in distro_features and not "3g" in machine_features and ("pcmcia" in machine_features or "pci" in machine_features or "usbhost" in machine_features):
-        d.setVar("ADD_3G", "packagegroup-base-3g")
+        d.setVar("ADD_3G", "${MLPREFIX}packagegroup-base-3g")
 
      if "nfc" in distro_features and not "nfc" in machine_features and ("usbhost" in machine_features):
-        d.setVar("ADD_NFC", "packagegroup-base-nfc")
+        d.setVar("ADD_NFC", "${MLPREFIX}packagegroup-base-nfc")
  }
 
  #
meta/recipes-core/psplash/psplash_git.bb +3 -2
index 22c71f099b8..44f0007daf0 100644
@@ -22,6 +22,7 @@ SPLASH_IMAGES = "file://psplash-poky-img.h;outsuffix=default"
  python __anonymous() {
      oldpkgs = d.getVar("PACKAGES").split()
      splashfiles = d.getVar('SPLASH_IMAGES').split()
+    mlprefix = d.getVar('MLPREFIX') or ''
      pkgs = []
      localpaths = []
      for uri in splashfiles:
@@ -46,9 +47,9 @@ python __anonymous() {
      # Set these so that we have less work to do in do_compile and do_install_append
      d.setVar("SPLASH_INSTALL", " ".join(pkgs))
      d.setVar("SPLASH_LOCALPATHS", " ".join(localpaths))
+    for p in pkgs:
+        d.prependVar("PACKAGES", "%s%s " % (mlprefix, p))
 
-    d.prependVar("PACKAGES", "%s " % (" ".join(pkgs)))
-    mlprefix = d.getVar('MLPREFIX') or ''
      pn = d.getVar('PN') or ''
      for p in pkgs:
          ep = '%s%s' % (mlprefix, p)
meta/recipes-devtools/perl/perl_5.30.2.bb +1 -1
index 778c420b2ee..26138ea9e55 100644
@@ -328,7 +328,7 @@ python split_perl_packages () {
 
  python() {
      if d.getVar('CLASSOVERRIDE') == "class-target":
-        d.setVar("PACKAGES_DYNAMIC", "^perl-module-.*(?<!native)$")
+        d.setVar("PACKAGES_DYNAMIC", "^${MLPREFIX}perl-module-.*(?<!native)$")
      elif d.getVar('CLASSOVERRIDE') == "class-native":
          d.setVar("PACKAGES_DYNAMIC", "^perl-module-.*-native$")
      elif d.getVar('CLASSOVERRIDE') == "class-nativesdk":
meta/recipes-devtools/python/python3_3.8.2.bb +7 -7
index a4a16fd495f..0474f07214f 100644
@@ -311,8 +311,8 @@ do_create_manifest[depends] += "${PN}:do_patch"
 
  # manual dependency additions
  RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python3-modules"
-RRECOMMENDS_${PN}-crypt_append_class-target = " openssl ca-certificates"
-RRECOMMENDS_${PN}-crypt_append_class-nativesdk = " openssl ca-certificates"
+RRECOMMENDS_${PN}-crypt_append_class-target = " ${MLPREFIX}openssl ${MLPREFIX}ca-certificates"
+RRECOMMENDS_${PN}-crypt_append_class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX}ca-certificates"
 
  # For historical reasons PN is empty and provided by python3-modules
  FILES_${PN} = ""
@@ -322,7 +322,7 @@ FILES_${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
  FILES_${PN}-idle += "${bindir}/idle3 ${bindir}/idle${PYTHON_MAJMIN}"
 
  # provide python-pyvenv from python3-venv
-RPROVIDES_${PN}-venv += "python3-pyvenv"
+RPROVIDES_${PN}-venv += "${MLPREFIX}python3-pyvenv"
 
  # package libpython3
  PACKAGES =+ "libpython3 libpython3-staticdev"
@@ -333,8 +333,8 @@ INSANE_SKIP_${PN}-dev += "dev-elf"
  # catch all the rest (unsorted)
  PACKAGES += "${PN}-misc"
  RDEPENDS_${PN}-misc += "python3-core python3-email python3-codecs python3-pydoc python3-pickle python3-audio"
-RDEPENDS_${PN}-modules_append_class-target = " python3-misc"
-RDEPENDS_${PN}-modules_append_class-nativesdk = " python3-misc"
+RDEPENDS_${PN}-modules_append_class-target = " ${MLPREFIX}python3-misc"
+RDEPENDS_${PN}-modules_append_class-nativesdk = " ${MLPREFIX}python3-misc"
  FILES_${PN}-misc = "${libdir}/python${PYTHON_MAJMIN} ${libdir}/python${PYTHON_MAJMIN}/lib-dynload"
 
  # catch manpage
@@ -348,5 +348,5 @@ RDEPENDS_${PN}-ptest_append_libc-glibc = " locale-base-tr-tr.iso-8859-9"
  RDEPENDS_${PN}-tkinter += "${@bb.utils.contains('PACKAGECONFIG', 'tk', 'tk tk-lib', '', d)}"
  RDEPENDS_${PN}-dev = ""
 
-RDEPENDS_${PN}-tests_append_class-target = " bash"
-RDEPENDS_${PN}-tests_append_class-nativesdk = " bash"
+RDEPENDS_${PN}-tests_append_class-target = " ${MLPREFIX}bash"
+RDEPENDS_${PN}-tests_append_class-nativesdk = " ${MLPREFIX}bash"
meta/recipes-extended/packagegroups/packagegroup-core-full-cmdline.bb +6 -4
index 15a8e6dedc3..16c2f9f2aa9 100644
@@ -32,21 +32,23 @@ python __anonymous () {
      namemap["packagegroup-core-full-cmdline-sys-services"] = "packagegroup-core-sys-services"
 
      packages = d.getVar("PACKAGES").split()
+    mlprefix = d.getVar("MLPREFIX")
      for pkg in packages:
+        pkg2 = pkg[len(mlprefix):]
          if pkg.endswith('-dev'):
-            mapped = namemap.get(pkg[:-4], None)
+            mapped = namemap.get(pkg2[:-4], None)
              if mapped:
                  mapped += '-dev'
          elif pkg.endswith('-dbg'):
-            mapped = namemap.get(pkg[:-4], None)
+            mapped = namemap.get(pkg2[:-4], None)
              if mapped:
                  mapped += '-dbg'
          else:
-            mapped = namemap.get(pkg, None)
+            mapped = namemap.get(pkg2, None)
 
          if mapped:
              oldtaskname = mapped.replace("packagegroup-core", "task-core")
-            mapstr = " %s %s" % (mapped, oldtaskname)
+            mapstr = " %s%s %s%s" % (mlprefix, mapped, mlprefix, oldtaskname)
              d.appendVar("RPROVIDES_%s" % pkg, mapstr)
              d.appendVar("RREPLACES_%s" % pkg, mapstr)
              d.appendVar("RCONFLICTS_%s" % pkg, mapstr)
meta/recipes-graphics/mesa/mesa.inc +6 -4
index fede691d6ff..bb43a9a8b65 100644
@@ -212,18 +212,20 @@ python __anonymous() {
                ("gles", "libgles3",)):
          if not p[0] in pkgconfig:
              continue
-        fullp = p[1] + "-mesa"
-        pkgs = " ".join(p[1:])
+        mlprefix = d.getVar("MLPREFIX")
+        fullp = mlprefix + p[1] + "-mesa"
+        mlprefix = d.getVar("MLPREFIX")
+        pkgs = " ".join(mlprefix + x for x in p[1:])
          d.setVar("DEBIAN_NOAUTONAME_" + fullp, "1")
          d.appendVar("RREPLACES_" + fullp, pkgs)
          d.appendVar("RPROVIDES_" + fullp, pkgs)
          d.appendVar("RCONFLICTS_" + fullp, pkgs)
 
-        d.appendVar("RRECOMMENDS_" + fullp, " mesa-megadriver")
+        d.appendVar("RRECOMMENDS_" + fullp, " ${MLPREFIX}mesa-megadriver")
 
          # For -dev, the first element is both the Debian and original name
          fullp += "-dev"
-        pkgs = p[1] + "-dev"
+        pkgs = mlprefix + p[1] + "-dev"
          d.setVar("DEBIAN_NOAUTONAME_" + fullp, "1")
          d.appendVar("RREPLACES_" + fullp, pkgs)
          d.appendVar("RPROVIDES_" + fullp, pkgs)
meta/recipes-multimedia/alsa/alsa-plugins_1.2.1.bb +3 -3
index 9882e12763a..d092b158f26 100644
@@ -167,7 +167,7 @@ FILES_${PN}-pulseaudio-conf += "\
  "
 
  RDEPENDS_${PN}-pulseaudio-conf += "\
-        libasound-module-conf-pulse \
-        libasound-module-ctl-pulse \
-        libasound-module-pcm-pulse \
+        ${MLPREFIX}libasound-module-conf-pulse \
+        ${MLPREFIX}libasound-module-ctl-pulse \
+        ${MLPREFIX}libasound-module-pcm-pulse \
  "
meta/recipes-support/boost/boost.inc +7 -3
index 8eb9494381f..ca140d595fd 100644
@@ -62,12 +62,16 @@ PACKAGES = "${PN}-dbg ${BOOST_PACKAGES}"
  python __anonymous () {
      packages = []
      extras = []
+    mlprefix = d.getVar("MLPREFIX")
      for lib in d.getVar('BOOST_LIBS').split():
          extras.append("--with-%s" % lib)
-        pkg = "boost-%s" % lib.replace("_", "-")
-        packages.append(pkg)
+        pkg = "boost-%s" % (lib.replace("_", "-"))
+        packages.append(mlprefix + pkg)
          if not d.getVar("FILES_%s" % pkg):
-                d.setVar("FILES_%s" % pkg, "${libdir}/libboost_%s*.so.*" % lib)
+                d.setVar("FILES_%s%s" % (mlprefix, pkg), "${libdir}/libboost_%s*.so.*" % lib)
+        else:
+                d.setVar("FILES_%s%s" % (mlprefix, pkg), d.getVar("FILES_%s" % pkg))
+
      d.setVar("BOOST_PACKAGES", " ".join(packages))
      d.setVar("BJAM_EXTRA", " ".join(extras))
  }
Signature · 2 lines
Jeremy A. Puhlman
[email protected]
#6 lore raw
Quoted 5 lines · Jeremy Puhlman

The one question I did have is there are a couple of places where you add the ML prefix for
nativesdk classes. If we don't have them, it shouldn't hurt anything, but do we actually support
multilib nativesdk packages?

nativesdk actually uses the same mechanism to extend the package names as multilib does so for nativesdk, MLPREFIX="nativesdk-".

So no, we don't support mulitlib nativesdk but nativesdk is a kind of multilib in some ways as it also sets MLPREFIX.

Cheers,

Richard

#7 lore raw
In reply to Richard Purdie

Hi Richard,

Quoted 18 lines · Richard Purdie

There are issues with multilib due to the ordering of events where some functions see the remapped multilib dependencies and some do not. A significant problem is that the multilib class needs to make some changes before key expansion and some afterwards but by using existing event handlers, some code sees things in a partially translated state, leading to bugs.

This patch changes things to use a new event handler from bitbake which makes the ordering of the changes explcit.

The challenge in doing this is that it breaks some existing anonymous python and dyanmic assignments. In some cases these used to be translated and no longer are, meaning MLPREFIX has to be added. In some cases these are now translated and the MLPREFIX can be removed.

This change does now make it very clear when MLPREFIX is required and when it is not, its just the migration path which is harder. The patch changes the small number of cases where fixes are needed.

Quoted 9 lines

In particular, where a variable like RDEPENDS is conditionally extended (e.g. with an override), MLPREFIX is now required.

This patch also reverts:
base: Revert 'base.bbclass: considering multilib when setting LICENSE_EXCLUSION'

This reverts 6597130256a1609c3e05ec5891aceaf549c37985 as the changes to multilib datastore handling mean its no longer necessary.

Is it possible to link the bugzilla entry in the commit log?

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13865

Otherwise, the main question is: do builds actually show warnings or fail if MLPREFIX is forgotten in some places? I'm thinking about other layers needing similar changes.

In either case, maybe a proper "note" in the migration steps for the impacted release would be a good addition I think?

Some nitpicking/cosmetic review:

[...]

meta/lib/oe/classextend.py class ClassExtender(object): · line 103

Quoted 3 lines
meta/lib/oe/classextend.py +2 -1
-        self.d.setVar(varname, bb.utils.join_deps(newdeps, False).replace("EXTENDPKGV", "${EXTENDPKGV}"))
+        if not varname.endswith("_NONML"):
+            #if varname == "DEPENDS":

Is it an actual comment or is it a leftover of some debugging session?

[...]

meta/recipes-graphics/mesa/mesa.inc python __anonymous() { · line 217

Quoted 13 lines
meta/recipes-graphics/mesa/mesa.inc +3 -2
index fede691d6ff..bb43a9a8b65 100644
@@ -212,18 +212,20 @@ python __anonymous() {
               ("gles", "libgles3",)):
         if not p[0] in pkgconfig:
             continue
-        fullp = p[1] + "-mesa"
-        pkgs = " ".join(p[1:])
+        mlprefix = d.getVar("MLPREFIX")
+        fullp = mlprefix + p[1] + "-mesa"
+        mlprefix = d.getVar("MLPREFIX")

Duplicate mlprefix = d.getVar("MLPREFIX"). Any reason for that?

[...]

Thanks a bunch for the patches, I know it was painful. Quentin