Skip to contentSkip to conversation

SDK buildhistory fixes

PATCH New Laurentiu Palcu 1 participant 6 messages Mar 18, 2014 – Mar 19, 2014

lore
Share & tools
lore
#0 lore raw

cover SDK buildhistory fixes

The buildhistory dependency files for target/host SDK packages were not properly created because the wrapper function called, list_installed_packages(), was always looking in the image rootfs.

This patchset will rename the old wrapper function to image_list_installed_packages() and create a new one, for SDK stuff, sdk_list_installed_packages().

The changes in package_manager.py, even though they appear to be lots, its the same code moved around from one class to a newly created PkgsList class. So, the logic remains the same.

Tested for all backends (buildhistory activated) with the following:

bitbake core-image-sato && bitbake -c populate_sdk core-image-sato

laurentiu

The following changes since commit 6bbb179cc526c86631dfcb140e3dd51a8c07a52d:

bitbake: runqueue: More carefully handle the sigchld handler (2014-03-18 10:23:13 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/buildhistory_sdk_dep_files_fix
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/buildhistory_sdk_dep_files_fix

Laurentiu Palcu (4):
package_manager.py: create separate class for installed packages
listing
rootfs.py, sdk.py: adjust/create the wrappers for creating installed
packages list
image.bbclass, license.bbclass: adjust the name of
list_installed_packages()
buildhistory.bbclass: create proper dependency files for SDK

 meta/classes/buildhistory.bbclass |   40 ++--
 meta/classes/image.bbclass        |    4 +-
 meta/classes/license.bbclass      |    4 +-
 meta/lib/oe/package_manager.py    |  376 +++++++++++++++++++++----------------
 meta/lib/oe/rootfs.py             |   19 +-
 meta/lib/oe/sdk.py                |   18 ++
 6 files changed, 264 insertions(+), 197 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/4 package_manager.py: create separate class for installed packages listing

This commit creates a new class that has the only purpose to generate various listings of installed packages in the rootfs.

Basically, the methods involved in listing the installed packages, that were part of each backend PM class implementation, were moved to this new class.

This change avoids instantiating a new PM object just to get the list of installed packages in a certain rootfs.

---
 meta/lib/oe/package_manager.py |  376 ++++++++++++++++++++++------------------
 1 file changed, 211 insertions(+), 165 deletions(-)
meta/lib/oe/package_manager.py +211 -165
index 1279b50..0426b23 100644
@@ -213,6 +213,213 @@ class DpkgIndexer(Indexer):
                 return(result)
 
 
+class PkgsList(object):
+    __metaclass__ = ABCMeta
+
+    def __init__(self, d, rootfs_dir):
+        self.d = d
+        self.rootfs_dir = rootfs_dir
+
+    @abstractmethod
+    def list(self, format=None):
+        pass
+
+
+class RpmPkgsList(PkgsList):
+    def __init__(self, d, rootfs_dir, arch_var=None, os_var=None):
+        super(RpmPkgsList, self).__init__(d, rootfs_dir)
+
+        self.rpm_cmd = bb.utils.which(os.getenv('PATH'), "rpm")
+        self.image_rpmlib = os.path.join(self.rootfs_dir, 'var/lib/rpm')
+
+        self.ml_prefix_list, self.ml_os_list = \
+            RpmIndexer(d, rootfs_dir).get_ml_prefix_and_os_list(arch_var, os_var)
+
+    '''
+    Translate the RPM/Smart format names to the OE multilib format names
+    '''
+    def _pkg_translate_smart_to_oe(self, pkg, arch):
+        new_pkg = pkg
+        fixed_arch = arch.replace('_', '-')
+        found = 0
+        for mlib in self.ml_prefix_list:
+            for cmp_arch in self.ml_prefix_list[mlib]:
+                fixed_cmp_arch = cmp_arch.replace('_', '-')
+                if fixed_arch == fixed_cmp_arch:
+                    if mlib == 'default':
+                        new_pkg = pkg
+                        new_arch = cmp_arch
+                    else:
+                        new_pkg = mlib + '-' + pkg
+                        # We need to strip off the ${mlib}_ prefix on the arch
+                        new_arch = cmp_arch.replace(mlib + '_', '')
+
+                    # Workaround for bug 3565. Simply look to see if we
+                    # know of a package with that name, if not try again!
+                    filename = os.path.join(self.d.getVar('PKGDATA_DIR', True),
+                                            'runtime-reverse',
+                                            new_pkg)
+                    if os.path.exists(filename):
+                        found = 1
+                        break
+
+            if found == 1 and fixed_arch == fixed_cmp_arch:
+                break
+        #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
+        return new_pkg, new_arch
+
+    def _list_pkg_deps(self):
+        cmd = [bb.utils.which(os.getenv('PATH'), "rpmresolve"),
+               "-t", self.image_rpmlib]
+
+        try:
+            output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip()
+        except subprocess.CalledProcessError as e:
+            bb.fatal("Cannot get the package dependencies. Command '%s' "
+                     "returned %d:\n%s" % (' '.join(cmd), e.returncode, e.output))
+
+        return output
+
+    def list(self, format=None):
+        if format == "deps":
+            return self._list_pkg_deps()
+
+        cmd = self.rpm_cmd + ' --root ' + self.rootfs_dir
+        cmd += ' -D "_dbpath /var/lib/rpm" -qa'
+        cmd += " --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]'"
+
+        try:
+            # bb.note(cmd)
+            tmp_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip()
+
+            rpm_db_locks = glob.glob('%s/var/lib/rpm/__db.*' % self.rootfs_dir)
+            for f in rpm_db_locks:
+                bb.utils.remove(f, True)
+        except subprocess.CalledProcessError as e:
+            bb.fatal("Cannot get the installed packages list. Command '%s' "
+                     "returned %d:\n%s" % (cmd, e.returncode, e.output))
+
+        output = list()
+        for line in tmp_output.split('\n'):
+            if len(line.strip()) == 0:
+                continue
+            pkg = line.split()[0]
+            arch = line.split()[1]
+            ver = line.split()[2]
+            pkgorigin = line.split()[3]
+            new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
+
+            if format == "arch":
+                output.append('%s %s' % (new_pkg, new_arch))
+            elif format == "file":
+                output.append('%s %s %s' % (new_pkg, pkgorigin, new_arch))
+            elif format == "ver":
+                output.append('%s %s %s' % (new_pkg, new_arch, ver))
+            else:
+                output.append('%s' % (new_pkg))
+
+            output.sort()
+
+        return '\n'.join(output)
+
+
+class OpkgPkgsList(PkgsList):
+    def __init__(self, d, rootfs_dir, config_file):
+        super(OpkgPkgsList, self).__init__(d, rootfs_dir)
+
+        self.opkg_cmd = bb.utils.which(os.getenv('PATH'), "opkg-cl")
+        self.opkg_args = "-f %s -o %s " % (config_file, rootfs_dir)
+        self.opkg_args += self.d.getVar("OPKG_ARGS", True)
+
+    def list(self, format=None):
+        opkg_query_cmd = bb.utils.which(os.getenv('PATH'), "opkg-query-helper.py")
+
+        if format == "arch":
+            cmd = "%s %s status | %s -a" % \
+                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
+        elif format == "file":
+            cmd = "%s %s status | %s -f" % \
+                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
+        elif format == "ver":
+            cmd = "%s %s status | %s -v" % \
+                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
+        elif format == "deps":
+            cmd = "%s %s status | %s" % \
+                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
+        else:
+            cmd = "%s %s list_installed | cut -d' ' -f1" % \
+                (self.opkg_cmd, self.opkg_args)
+
+        try:
+            output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip()
+        except subprocess.CalledProcessError as e:
+            bb.fatal("Cannot get the installed packages list. Command '%s' "
+                     "returned %d:\n%s" % (cmd, e.returncode, e.output))
+
+        if output and format == "file":
+            tmp_output = ""
+            for line in output.split('\n'):
+                pkg, pkg_file, pkg_arch = line.split()
+                full_path = os.path.join(self.rootfs_dir, pkg_arch, pkg_file)
+                if os.path.exists(full_path):
+                    tmp_output += "%s %s %s\n" % (pkg, full_path, pkg_arch)
+                else:
+                    tmp_output += "%s %s %s\n" % (pkg, pkg_file, pkg_arch)
+
+            output = tmp_output
+
+        return output
+
+
+class DpkgPkgsList(PkgsList):
+    def list(self, format=None):
+        cmd = [bb.utils.which(os.getenv('PATH'), "dpkg-query"),
+               "--admindir=%s/var/lib/dpkg" % self.rootfs_dir,
+               "-W"]
+
+        if format == "arch":
+            cmd.append("-f=${Package} ${PackageArch}\n")
+        elif format == "file":
+            cmd.append("-f=${Package} ${Package}_${Version}_${Architecture}.deb ${PackageArch}\n")
+        elif format == "ver":
+            cmd.append("-f=${Package} ${PackageArch} ${Version}\n")
+        elif format == "deps":
+            cmd.append("-f=Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n")
+        else:
+            cmd.append("-f=${Package}\n")
+
+        try:
+            output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip()
+        except subprocess.CalledProcessError as e:
+            bb.fatal("Cannot get the installed packages list. Command '%s' "
+                     "returned %d:\n%s" % (' '.join(cmd), e.returncode, e.output))
+
+        if format == "file":
+            tmp_output = ""
+            for line in tuple(output.split('\n')):
+                pkg, pkg_file, pkg_arch = line.split()
+                full_path = os.path.join(self.rootfs_dir, pkg_arch, pkg_file)
+                if os.path.exists(full_path):
+                    tmp_output += "%s %s %s\n" % (pkg, full_path, pkg_arch)
+                else:
+                    tmp_output += "%s %s %s\n" % (pkg, pkg_file, pkg_arch)
+
+            output = tmp_output
+        elif format == "deps":
+            opkg_query_cmd = bb.utils.which(os.getenv('PATH'), "opkg-query-helper.py")
+
+            try:
+                output = subprocess.check_output("echo -e '%s' | %s" %
+                                                 (output, opkg_query_cmd),
+                                                 stderr=subprocess.STDOUT,
+                                                 shell=True)
+            except subprocess.CalledProcessError as e:
+                bb.fatal("Cannot compute packages dependencies. Command '%s' "
+                         "returned %d:\n%s" % (e.cmd, e.returncode, e.output))
+
+        return output
+
+
 class PackageManager(object):
     """
     This is an abstract class. Do not instantiate this directly.
@@ -360,10 +567,10 @@ class RpmPM(PackageManager):
             bb.utils.mkdirhier(self.d.expand('${T}/saved'))
 
         self.indexer = RpmIndexer(self.d, self.deploy_dir)
+        self.pkgs_list = RpmPkgsList(self.d, self.deploy_dir, arch_var, os_var)
 
         self.ml_prefix_list, self.ml_os_list = self.indexer.get_ml_prefix_and_os_list(arch_var, os_var)
 
-
     def insert_feeds_uris(self):
         if self.feed_uris == "":
             return
@@ -442,39 +649,6 @@ class RpmPM(PackageManager):
             bb.fatal("Could not invoke smart. Command "
                      "'%s' returned %d:\n%s" % (cmd, e.returncode, e.output))
 
-    '''
-    Translate the RPM/Smart format names to the OE multilib format names
-    '''
-    def _pkg_translate_smart_to_oe(self, pkg, arch):
-        new_pkg = pkg
-        fixed_arch = arch.replace('_', '-')
-        found = 0
-        for mlib in self.ml_prefix_list:
-            for cmp_arch in self.ml_prefix_list[mlib]:
-                fixed_cmp_arch = cmp_arch.replace('_', '-')
-                if fixed_arch == fixed_cmp_arch:
-                    if mlib == 'default':
-                        new_pkg = pkg
-                        new_arch = cmp_arch
-                    else:
-                        new_pkg = mlib + '-' + pkg
-                        # We need to strip off the ${mlib}_ prefix on the arch
-                        new_arch = cmp_arch.replace(mlib + '_', '')
-
-                    # Workaround for bug 3565. Simply look to see if we
-                    # know of a package with that name, if not try again!
-                    filename = os.path.join(self.d.getVar('PKGDATA_DIR', True),
-                                            'runtime-reverse',
-                                            new_pkg)
-                    if os.path.exists(filename):
-                        found = 1
-                        break
-
-            if found == 1 and fixed_arch == fixed_cmp_arch:
-                break
-        #bb.note('%s, %s -> %s, %s' % (pkg, arch, new_pkg, new_arch))
-        return new_pkg, new_arch
-
     def _search_pkg_name_in_feeds(self, pkg, feed_archs):
         for arch in feed_archs:
             arch = arch.replace('-', '_')
@@ -815,56 +989,8 @@ class RpmPM(PackageManager):
                             self.image_rpmlib,
                             symlinks=True)
 
-    def _list_pkg_deps(self):
-        cmd = [bb.utils.which(os.getenv('PATH'), "rpmresolve"),
-               "-t", self.image_rpmlib]
-
-        try:
-            output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip()
-        except subprocess.CalledProcessError as e:
-            bb.fatal("Cannot get the package dependencies. Command '%s' "
-                     "returned %d:\n%s" % (' '.join(cmd), e.returncode, e.output))
-
-        return output
-
     def list_installed(self, format=None):
-        if format == "deps":
-            return self._list_pkg_deps()
-
-        cmd = self.rpm_cmd + ' --root ' + self.target_rootfs
-        cmd += ' -D "_dbpath /var/lib/rpm" -qa'
-        cmd += " --qf '[%{NAME} %{ARCH} %{VERSION} %{PACKAGEORIGIN}\n]'"
-
-        try:
-            # bb.note(cmd)
-            tmp_output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip()
-            self._unlock_rpm_db()
-        except subprocess.CalledProcessError as e:
-            bb.fatal("Cannot get the installed packages list. Command '%s' "
-                     "returned %d:\n%s" % (cmd, e.returncode, e.output))
-
-        output = list()
-        for line in tmp_output.split('\n'):
-            if len(line.strip()) == 0:
-                continue
-            pkg = line.split()[0]
-            arch = line.split()[1]
-            ver = line.split()[2]
-            pkgorigin = line.split()[3]
-            new_pkg, new_arch = self._pkg_translate_smart_to_oe(pkg, arch)
-
-            if format == "arch":
-                output.append('%s %s' % (new_pkg, new_arch))
-            elif format == "file":
-                output.append('%s %s %s' % (new_pkg, pkgorigin, new_arch))
-            elif format == "ver":
-                output.append('%s %s %s' % (new_pkg, new_arch, ver))
-            else:
-                output.append('%s' % (new_pkg))
-
-            output.sort()
-
-        return '\n'.join(output)
+        return self.pkgs_list.list(format)
 
     '''
     If incremental install, we need to determine what we've got,
@@ -1207,43 +1333,7 @@ class OpkgPM(PackageManager):
         bb.utils.mkdirhier(self.opkg_dir)
 
     def list_installed(self, format=None):
-        opkg_query_cmd = bb.utils.which(os.getenv('PATH'), "opkg-query-helper.py")
-
-        if format == "arch":
-            cmd = "%s %s status | %s -a" % \
-                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
-        elif format == "file":
-            cmd = "%s %s status | %s -f" % \
-                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
-        elif format == "ver":
-            cmd = "%s %s status | %s -v" % \
-                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
-        elif format == "deps":
-            cmd = "%s %s status | %s" % \
-                (self.opkg_cmd, self.opkg_args, opkg_query_cmd)
-        else:
-            cmd = "%s %s list_installed | cut -d' ' -f1" % \
-                (self.opkg_cmd, self.opkg_args)
-
-        try:
-            output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=True).strip()
-        except subprocess.CalledProcessError as e:
-            bb.fatal("Cannot get the installed packages list. Command '%s' "
-                     "returned %d:\n%s" % (cmd, e.returncode, e.output))
-
-        if output and format == "file":
-            tmp_output = ""
-            for line in output.split('\n'):
-                pkg, pkg_file, pkg_arch = line.split()
-                full_path = os.path.join(self.deploy_dir, pkg_arch, pkg_file)
-                if os.path.exists(full_path):
-                    tmp_output += "%s %s %s\n" % (pkg, full_path, pkg_arch)
-                else:
-                    tmp_output += "%s %s %s\n" % (pkg, pkg_file, pkg_arch)
-
-            output = tmp_output
-
-        return output
+        return OpkgPkgsList(self.d, self.target_rootfs, self.config_file).list(format)
 
     def handle_bad_recommendations(self):
         bad_recommendations = self.d.getVar("BAD_RECOMMENDATIONS", True) or ""
@@ -1594,51 +1684,7 @@ class DpkgPM(PackageManager):
                      "returned %d:\n%s" % (cmd, e.returncode, e.output))
 
     def list_installed(self, format=None):
-        cmd = [bb.utils.which(os.getenv('PATH'), "dpkg-query"),
-               "--admindir=%s/var/lib/dpkg" % self.target_rootfs,
-               "-W"]
-
-        if format == "arch":
-            cmd.append("-f=${Package} ${PackageArch}\n")
-        elif format == "file":
-            cmd.append("-f=${Package} ${Package}_${Version}_${Architecture}.deb ${PackageArch}\n")
-        elif format == "ver":
-            cmd.append("-f=${Package} ${PackageArch} ${Version}\n")
-        elif format == "deps":
-            cmd.append("-f=Package: ${Package}\nDepends: ${Depends}\nRecommends: ${Recommends}\n\n")
-        else:
-            cmd.append("-f=${Package}\n")
-
-        try:
-            output = subprocess.check_output(cmd, stderr=subprocess.STDOUT).strip()
-        except subprocess.CalledProcessError as e:
-            bb.fatal("Cannot get the installed packages list. Command '%s' "
-                     "returned %d:\n%s" % (' '.join(cmd), e.returncode, e.output))
-
-        if format == "file":
-            tmp_output = ""
-            for line in tuple(output.split('\n')):
-                pkg, pkg_file, pkg_arch = line.split()
-                full_path = os.path.join(self.deploy_dir, pkg_arch, pkg_file)
-                if os.path.exists(full_path):
-                    tmp_output += "%s %s %s\n" % (pkg, full_path, pkg_arch)
-                else:
-                    tmp_output += "%s %s %s\n" % (pkg, pkg_file, pkg_arch)
-
-            output = tmp_output
-        elif format == "deps":
-            opkg_query_cmd = bb.utils.which(os.getenv('PATH'), "opkg-query-helper.py")
-
-            try:
-                output = subprocess.check_output("echo -e '%s' | %s" %
-                                                 (output, opkg_query_cmd),
-                                                 stderr=subprocess.STDOUT,
-                                                 shell=True)
-            except subprocess.CalledProcessError as e:
-                bb.fatal("Cannot compute packages dependencies. Command '%s' "
-                         "returned %d:\n%s" % (e.cmd, e.returncode, e.output))
-
-        return output
+        return DpkgPkgsList(self.d, self.target_rootfs).list()
 
 
 def generate_index_files(d):
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH 2/4 rootfs.py, sdk.py: adjust/create the wrappers for creating installed packages list

In reply to Laurentiu Palcu

Since we created a new PkgsList object that will deal with listing the installed packages in a rootfs, use the new class both for images and SDKs in the wrapper functions.

The old list_installed_packages() wrapper listed only the packages inside an image rootfs. It didn't deal with target/host SDK rootfs's.

---
 meta/lib/oe/rootfs.py |   19 ++++---------------
 meta/lib/oe/sdk.py    |   18 ++++++++++++++++++
 2 files changed, 22 insertions(+), 15 deletions(-)
meta/lib/oe/rootfs.py +4 -15
index 30a1321..0e6c8bc 100644
@@ -709,28 +709,17 @@ def create_rootfs(d, manifest_dir=None):
     os.environ.update(env_bkp)
 
 
-def list_installed_packages(d, format=None, rootfs_dir=None):
+def image_list_installed_packages(d, format=None, rootfs_dir=None):
     if not rootfs_dir:
         rootfs_dir = d.getVar('IMAGE_ROOTFS', True)
 
     img_type = d.getVar('IMAGE_PKGTYPE', True)
     if img_type == "rpm":
-        return RpmPM(d,
-                     rootfs_dir,
-                     d.getVar('TARGET_VENDOR', True)
-                     ).list_installed(format)
+        return RpmPkgsList(d, rootfs_dir).list(format)
     elif img_type == "ipk":
-        return OpkgPM(d,
-                      rootfs_dir,
-                      d.getVar("IPKGCONF_TARGET", True),
-                      d.getVar("ALL_MULTILIB_PACKAGE_ARCHS", True)
-                      ).list_installed(format)
+        return OpkgPkgsList(d, rootfs_dir, d.getVar("IPKGCONF_TARGET", True)).list(format)
     elif img_type == "deb":
-        return DpkgPM(d,
-                      rootfs_dir,
-                      d.getVar('PACKAGE_ARCHS', True),
-                      d.getVar('DPKG_ARCH', True)
-                      ).list_installed(format)
+        return DpkgPkgsList(d, rootfs_dir).list(format)
 
 if __name__ == "__main__":
     """
meta/lib/oe/sdk.py +18 -0
index 01a1807..5643199 100644
@@ -289,6 +289,24 @@ class DpkgSdk(Sdk):
         bb.utils.remove(os.path.join(self.sdk_output, "var"), True)
 
 
+def sdk_list_installed_packages(d, target, format=None, rootfs_dir=None):
+    if rootfs_dir is None:
+        sdk_output = d.getVar('SDK_OUTPUT', True)
+        target_path = d.getVar('SDKTARGETSYSROOT', True).strip('/')
+
+        rootfs_dir = [sdk_output, os.path.join(sdk_output, target_path)][target is True]
+
+    img_type = d.getVar('IMAGE_PKGTYPE', True)
+    if img_type == "rpm":
+        arch_var = ["SDK_PACKAGE_ARCHS", None][target is True]
+        os_var = ["SDK_OS", None][target is True]
+        return RpmPkgsList(d, rootfs_dir, arch_var, os_var).list(format)
+    elif img_type == "ipk":
+        conf_file_var = ["IPKGCONF_SDK", "IPKGCONF_Target"][target is True]
+        return OpkgPkgsList(d, rootfs_dir, d.getVar(conf_file_var, True)).list(format)
+    elif img_type == "deb":
+        return DpkgPkgsList(d, rootfs_dir).list(format)
+
 def populate_sdk(d, manifest_dir=None):
     env_bkp = os.environ.copy()
 
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH 3/4 image.bbclass, license.bbclass: adjust the name of list_installed_packages()

In reply to Laurentiu Palcu

The old wrapper got renamed to image_list_installed_packages().

---
 meta/classes/image.bbclass   |    4 ++--
 meta/classes/license.bbclass |    4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
meta/classes/image.bbclass +2 -2
index 51d16d7..9a04288 100644
@@ -330,9 +330,9 @@ make_zimage_symlink_relative () {
 }
 
 python write_image_manifest () {
-    from oe.rootfs import list_installed_packages
+    from oe.rootfs import image_list_installed_packages
     with open(d.getVar('IMAGE_MANIFEST', True), 'w+') as image_manifest:
-        image_manifest.write(list_installed_packages(d, 'ver'))
+        image_manifest.write(image_list_installed_packages(d, 'ver'))
 }
 
 # Make login manager(s) enable automatic login.
meta/classes/license.bbclass +2 -2
index a0b877d..08f0665 100644
@@ -20,9 +20,9 @@ python write_package_manifest() {
 	# Get list of installed packages
 	license_image_dir = d.expand('${LICENSE_DIRECTORY}/${IMAGE_NAME}')
 	bb.utils.mkdirhier(license_image_dir)
-	from oe.rootfs import list_installed_packages
+	from oe.rootfs import image_list_installed_packages
 	open(os.path.join(license_image_dir, 'package.manifest'),
-         'w+').write(list_installed_packages(d))
+         'w+').write(image_list_installed_packages(d))
 }
 
 license_create_manifest() {
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH 4/4 buildhistory.bbclass: create proper dependency files for SDK

In reply to Laurentiu Palcu

The old functions were calling the list_installed_packages() wrapper function that only listed the packages in an image rootfs. Even for target/host SDK. Also, a python crash was possible if 'bitbake -c populate_sdk core-image-*' was called without calling 'bitbake core-image-*' first. That's because the wrapper was always looking into the image rootfs...

This commit fixes the problem and calls the right wrapper for image/sdk.

---
 meta/classes/buildhistory.bbclass |   40 +++++++++++++++++++++++++------------
 1 file changed, 27 insertions(+), 13 deletions(-)
meta/classes/buildhistory.bbclass +27 -13
index 5d0a229..262095f 100644
@@ -313,22 +313,36 @@ def write_pkghistory(pkginfo, d):
             if os.path.exists(filevarpath):
                 os.unlink(filevarpath)
 
-python buildhistory_list_installed() {
-    from oe.rootfs import list_installed_packages
+#
+# rootfs_type can be: image, sdk_target, sdk_host
+#
+def buildhistory_list_installed(d, rootfs_type="image"):
+    from oe.rootfs import image_list_installed_packages
+    from oe.sdk import sdk_list_installed_packages
+
+    process_list = [('file', 'bh_installed_pkgs.txt'),\
+                    ('deps', 'bh_installed_pkgs_deps.txt')]
 
-    pkgs_list_file = os.path.join(d.getVar('WORKDIR', True),
-                                  "bh_installed_pkgs.txt")
+    for output_type, output_file in process_list:
+        output_file_full = os.path.join(d.getVar('WORKDIR', True), output_file)
 
-    with open(pkgs_list_file, 'w') as pkgs_list:
-        pkgs_list.write(list_installed_packages(d, 'file'))
+        with open(output_file_full, 'w') as output:
+            if rootfs_type == "image":
+                output.write(image_list_installed_packages(d, output_type))
+            else:
+                output.write(sdk_list_installed_packages(d, rootfs_type == "sdk_target", output_type))
 
-    pkgs_deps_file = os.path.join(d.getVar('WORKDIR', True),
-                                  "bh_installed_pkgs_deps.txt")
+python buildhistory_list_installed_image() {
+    buildhistory_list_installed(d)
+}
 
-    with open(pkgs_deps_file, 'w') as pkgs_deps:
-        pkgs_deps.write(list_installed_packages(d, 'deps'))
+python buildhistory_list_installed_sdk_target() {
+    buildhistory_list_installed(d, "sdk_target")
 }
 
+python buildhistory_list_installed_sdk_host() {
+    buildhistory_list_installed(d, "sdk_host")
+}
 
 buildhistory_get_installed() {
 	mkdir -p $1
@@ -471,15 +485,15 @@ END
 }
 
 # By prepending we get in before the removal of packaging files
-ROOTFS_POSTPROCESS_COMMAND =+ " buildhistory_list_installed ;\
+ROOTFS_POSTPROCESS_COMMAND =+ " buildhistory_list_installed_image ;\
                                 buildhistory_get_image_installed ; "
 
 IMAGE_POSTPROCESS_COMMAND += " buildhistory_get_imageinfo ; "
 
 # We want these to be the last run so that we get called after complementary package installation
-POPULATE_SDK_POST_TARGET_COMMAND_append = " buildhistory_list_installed ;\
+POPULATE_SDK_POST_TARGET_COMMAND_append = " buildhistory_list_installed_sdk_target ;\
                                             buildhistory_get_sdk_installed_target ; "
-POPULATE_SDK_POST_HOST_COMMAND_append = " buildhistory_list_installed ;\
+POPULATE_SDK_POST_HOST_COMMAND_append = " buildhistory_list_installed_sdk_host ;\
                                           buildhistory_get_sdk_installed_host ; "
 
 SDK_POSTPROCESS_COMMAND += "buildhistory_get_sdkinfo ; "
Signature · 1 line
1.7.9.5
#5 lore raw

v2 cover SDK buildhistory fixes (cover letter only)

In reply to Laurentiu Palcu

v2:
* I passed deploy_dir instead of target_rootfs to RpmPkgsList constructor... Strangely enough, this could be caught only when building core-image-minimal (which I didn't use in my tests).

The buildhistory dependency files for target/host SDK packages were not properly created because the wrapper function called, list_installed_packages(), was always looking in the image rootfs.

This patchset will rename the old wrapper function to image_list_installed_packages() and create a new one, for SDK stuff, sdk_list_installed_packages().

The changes in package_manager.py, even though they appear to be lots, its the same code moved around from one class to a newly created PkgsList class. So, the logic remains the same.

Tested for all backends (buildhistory activated) with the following:

bitbake core-image-sato && bitbake -c populate_sdk core-image-sato

laurentiu

The following changes since commit 0150bc30d3674301631c2e9b6c64e01058fd1070:

bitbake: runqueue: Really fix sigchld handling (2014-03-18 23:05:53 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/buildhistory_sdk_dep_files_fix
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/buildhistory_sdk_dep_files_fix

Laurentiu Palcu (4):
package_manager.py: create separate class for installed packages
listing
rootfs.py, sdk.py: adjust/create the wrappers for creating installed
packages list
image.bbclass, license.bbclass: adjust the name of
list_installed_packages()
buildhistory.bbclass: create proper dependency files for SDK

 meta/classes/buildhistory.bbclass |   40 ++--
 meta/classes/image.bbclass        |    4 +-
 meta/classes/license.bbclass      |    4 +-
 meta/lib/oe/package_manager.py    |  376 +++++++++++++++++++++----------------
 meta/lib/oe/rootfs.py             |   19 +-
 meta/lib/oe/sdk.py                |   18 ++
 6 files changed, 264 insertions(+), 197 deletions(-)
Signature · 1 line
1.7.9.5