Skip to contentSkip to conversation

lib/oe/package-manager: skip processing installed-pkgs with empty globs

PATCH In review Claus Stovgaard 1 participant 3 messages Oct 7, 2024 – Oct 11, 2024 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 lib/oe/package-manager: skip processing installed-pkgs with empty globs

We can skip processing the installed-pkgs file if globs is empty. This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY returns an empty string. If globs is an empty string the result from processing with empty glob in oe-pkgdata-util will always be 0 packages to install.

Instead of return early on this we just skip and still generate the locale archive if needed.

---
 meta/lib/oe/package_manager/__init__.py | 76 ++++++++++++-------------
 1 file changed, 37 insertions(+), 39 deletions(-)
meta/lib/oe/package_manager/__init__.py +37 -39
index d3b2317894..2100a97c12 100644
@@ -365,45 +365,43 @@ class PackageManager(object, metaclass=ABCMeta):
                 for complementary_linguas in (self.d.getVar('IMAGE_LINGUAS_COMPLEMENTARY') or "").split():
                     globs += (" " + complementary_linguas) % lang
 
-        if globs is None:
-            return
-
-        # we need to write the list of installed packages to a file because the
-        # oe-pkgdata-util reads it from a file
-        with tempfile.NamedTemporaryFile(mode="w+", prefix="installed-pkgs") as installed_pkgs:
-            pkgs = self.list_installed()
-
-            provided_pkgs = set()
-            for pkg in pkgs.values():
-                provided_pkgs |= set(pkg.get('provs', []))
-
-            output = oe.utils.format_pkg_list(pkgs, "arch")
-            installed_pkgs.write(output)
-            installed_pkgs.flush()
-
-            cmd = ["oe-pkgdata-util",
-                   "-p", self.d.getVar('PKGDATA_DIR'), "glob", installed_pkgs.name,
-                   globs]
-            exclude = self.d.getVar('PACKAGE_EXCLUDE_COMPLEMENTARY')
-            if exclude:
-                cmd.extend(['--exclude=' + '|'.join(exclude.split())])
-            try:
-                bb.note('Running %s' % cmd)
-                proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-                stdout, stderr = proc.communicate()
-                if stderr: bb.note(stderr.decode("utf-8"))
-                complementary_pkgs = stdout.decode("utf-8")
-                complementary_pkgs = set(complementary_pkgs.split())
-                skip_pkgs = sorted(complementary_pkgs & provided_pkgs)
-                install_pkgs = sorted(complementary_pkgs - provided_pkgs)
-                bb.note("Installing complementary packages ... %s (skipped already provided packages %s)" % (
-                    ' '.join(install_pkgs),
-                    ' '.join(skip_pkgs)))
-                self.install(install_pkgs, hard_depends_only=True)
-            except subprocess.CalledProcessError as e:
-                bb.fatal("Could not compute complementary packages list. Command "
-                         "'%s' returned %d:\n%s" %
-                         (' '.join(cmd), e.returncode, e.output.decode("utf-8")))
+        if globs:
+            # we need to write the list of installed packages to a file because the
+            # oe-pkgdata-util reads it from a file
+            with tempfile.NamedTemporaryFile(mode="w+", prefix="installed-pkgs") as installed_pkgs:
+                pkgs = self.list_installed()
+
+                provided_pkgs = set()
+                for pkg in pkgs.values():
+                    provided_pkgs |= set(pkg.get('provs', []))
+
+                output = oe.utils.format_pkg_list(pkgs, "arch")
+                installed_pkgs.write(output)
+                installed_pkgs.flush()
+
+                cmd = ["oe-pkgdata-util",
+                    "-p", self.d.getVar('PKGDATA_DIR'), "glob", installed_pkgs.name,
+                    globs]
+                exclude = self.d.getVar('PACKAGE_EXCLUDE_COMPLEMENTARY')
+                if exclude:
+                    cmd.extend(['--exclude=' + '|'.join(exclude.split())])
+                try:
+                    bb.note('Running %s' % cmd)
+                    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+                    stdout, stderr = proc.communicate()
+                    if stderr: bb.note(stderr.decode("utf-8"))
+                    complementary_pkgs = stdout.decode("utf-8")
+                    complementary_pkgs = set(complementary_pkgs.split())
+                    skip_pkgs = sorted(complementary_pkgs & provided_pkgs)
+                    install_pkgs = sorted(complementary_pkgs - provided_pkgs)
+                    bb.note("Installing complementary packages ... %s (skipped already provided packages %s)" % (
+                        ' '.join(install_pkgs),
+                        ' '.join(skip_pkgs)))
+                    self.install(install_pkgs, hard_depends_only=True)
+                except subprocess.CalledProcessError as e:
+                    bb.fatal("Could not compute complementary packages list. Command "
+                            "'%s' returned %d:\n%s" %
+                            (' '.join(cmd), e.returncode, e.output.decode("utf-8")))
 
         if self.d.getVar('IMAGE_LOCALES_ARCHIVE') == '1':
             target_arch = self.d.getVar('TARGET_ARCH')
Signature · 1 line
2.34.1
#1 lore raw
Quoted 121 lines · Claus Stovgaard

We can skip processing the installed-pkgs file if globs is empty. This is the case if self.d.getVar for IMAGE_INSTALL_COMPLEMENTARY returns an empty string. If globs is an empty string the result from processing with empty glob in oe-pkgdata-util will always be 0 packages
to install.

Instead of return early on this we just skip and still generate the locale archive if needed.

---
 meta/lib/oe/package_manager/__init__.py | 76 ++++++++++++----------- --

 1 file changed, 37 insertions(+), 39 deletions(-)
meta/lib/oe/package_manager/__init__.py +0 -0
index d3b2317894..2100a97c12 100644
@@ -365,45 +365,43 @@ class PackageManager(object,

metaclass=ABCMeta):
                 for complementary_linguas in (self.d.getVar('IMAGE_LINGUAS_COMPLEMENTARY') or "").split():
                     globs += (" " + complementary_linguas) % lang
-        if globs is None:
-            return
-
-        # we need to write the list of installed packages to a file because the
-        # oe-pkgdata-util reads it from a file
-        with tempfile.NamedTemporaryFile(mode="w+", prefix="installed-pkgs") as installed_pkgs:
-            pkgs = self.list_installed()

meta/lib/oe/package_manager/__init__.py +0 -11
-
-            provided_pkgs = set()
-            for pkg in pkgs.values():
-                provided_pkgs |= set(pkg.get('provs', []))
-
-            output = oe.utils.format_pkg_list(pkgs, "arch")
-            installed_pkgs.write(output)
-            installed_pkgs.flush()
-
-            cmd = ["oe-pkgdata-util",
-                   "-p", self.d.getVar('PKGDATA_DIR'), "glob",

installed_pkgs.name,
-                   globs]
-            exclude = self.d.getVar('PACKAGE_EXCLUDE_COMPLEMENTARY') -            if exclude:
-                cmd.extend(['--exclude=' + '|'.join(exclude.split())])
-            try:
-                bb.note('Running %s' % cmd)
-                proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-                stdout, stderr = proc.communicate()
-                if stderr: bb.note(stderr.decode("utf-8")) -                complementary_pkgs = stdout.decode("utf-8") -                complementary_pkgs = set(complementary_pkgs.split()) -                skip_pkgs = sorted(complementary_pkgs & provided_pkgs)
-                install_pkgs = sorted(complementary_pkgs - provided_pkgs)
-                bb.note("Installing complementary packages ... %s (skipped already provided packages %s)" % (
-                    ' '.join(install_pkgs),
-                    ' '.join(skip_pkgs)))
-                self.install(install_pkgs, hard_depends_only=True) -            except subprocess.CalledProcessError as e:
-                bb.fatal("Could not compute complementary packages list. Command "
-                         "'%s' returned %d:\n%s" %
-                         (' '.join(cmd), e.returncode, e.output.decode("utf-8")))
+        if globs:
+            # we need to write the list of installed packages to a file because the
+            # oe-pkgdata-util reads it from a file
+            with tempfile.NamedTemporaryFile(mode="w+", prefix="installed-pkgs") as installed_pkgs:
+                pkgs = self.list_installed()

meta/lib/oe/package_manager/__init__.py +11 -0
+
+                provided_pkgs = set()
+                for pkg in pkgs.values():
+                    provided_pkgs |= set(pkg.get('provs', []))
+
+                output = oe.utils.format_pkg_list(pkgs, "arch")
+                installed_pkgs.write(output)
+                installed_pkgs.flush()
+
+                cmd = ["oe-pkgdata-util",
+                    "-p", self.d.getVar('PKGDATA_DIR'), "glob",

installed_pkgs.name,
+                    globs]
+                exclude = self.d.getVar('PACKAGE_EXCLUDE_COMPLEMENTARY')
+                if exclude:
+                    cmd.extend(['--exclude=' + '|'.join(exclude.split())])
+                try:
+                    bb.note('Running %s' % cmd)
+                    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+                    stdout, stderr = proc.communicate() +                    if stderr: bb.note(stderr.decode("utf-8")) +                    complementary_pkgs = stdout.decode("utf-8") +                    complementary_pkgs = set(complementary_pkgs.split())
+                    skip_pkgs = sorted(complementary_pkgs & provided_pkgs)
+                    install_pkgs = sorted(complementary_pkgs - provided_pkgs)
+                    bb.note("Installing complementary packages ... %s (skipped already provided packages %s)" % (
+                        ' '.join(install_pkgs),
+                        ' '.join(skip_pkgs)))
+                    self.install(install_pkgs, hard_depends_only=True)
+                except subprocess.CalledProcessError as e:

Quoted 8 lines

+                    bb.fatal("Could not compute complementary packages list. Command "
+                            "'%s' returned %d:\n%s" % +                            (' '.join(cmd), e.returncode, e.output.decode("utf-8")))

         if self.d.getVar('IMAGE_LOCALES_ARCHIVE') == '1':
             target_arch = self.d.getVar('TARGET_ARCH')

Hi Ross.

Now you commented on v2 of the patch. Hope you think v3 is okay.

/Claus

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw
Quoted 7 lines · Claus Stovgaard via lists.openembedded.org

Hi Ross.

Now you commented on v2 of the patch. Hope you think v3 is okay.

/Claus

Nevermind. I can see Richard har picked it up in OE-core and poky- contrib.

/Claus