Skip to contentSkip to conversation

oeqa/selftest/lic_checksum: Cleanup changes to emptytest include

PATCH New Nathan Rossi 1 participant 4 messages Nov 14, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/4 oeqa/selftest/lic_checksum: Cleanup changes to emptytest include

From: Nathan Rossi <[email protected]>

Config written to the emptytest include file is invalid after the test has cleaned up its temporary directories resulting in the emptytest recipe potentially having invalid content when parsed by successive bitbake runs.

This presents the following error in tests after lic_checksum execution, e.g. 'oe-selftest -r lic_checksum recipetool'

ERROR: .../emptytest.bb: Unable to get checksum for emptytest SRC_URI entry tmpn_nyosnq: file could not be found

Remove the recipe include content once the bitbake runs are completed in each lic_checksum test case.

(cherry picked from commit 087df767a64b271b503d714df3df6d8b3caad1c0)
---
 meta/lib/oeqa/selftest/cases/lic_checksum.py | 2 ++
 1 file changed, 2 insertions(+)
meta/lib/oeqa/selftest/cases/lic_checksum.py +2 -0
index 8f1226e6a5..bc0a2b5d8e 100644
@@ -26,6 +26,7 @@ LIC_FILES_CHKSUM = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
 """ % (urllib.parse.quote(lic_path), urllib.parse.quote(lic_path)))
         result = bitbake(bitbake_cmd)
+        self.delete_recipeinc('emptytest')
 
 
     # Verify that changing a license file that has an absolute path causes
@@ -51,5 +52,6 @@ SRC_URI = "file://%s;md5=d41d8cd98f00b204e9800998ecf8427e"
             f.write("data")
 
         result = bitbake(bitbake_cmd, ignore_status=True)
+        self.delete_recipeinc('emptytest')
         if error_msg not in result.output:
             raise AssertionError(result.output)

---
2.38.1

#1 lore raw

PATCH 2/4 oeqa/selftest/minidebuginfo: Create selftest for minidebuginfo

From: Nathan Rossi <[email protected]>

Add a new selftest to validate minidebuginfo support. This selftest builds a complete target image with PACKAGE_MINIDEBUGINFO enabled. ELFs included in the image are expected to have minidebuginfo included in the resulting executables and shared libraries, the self test validates this by unpacking the image and checking for the associated ".gnu_debugdata" section on busybox and libc ELFs.

(cherry picked from commit 5063a31ad05b75ec6ac12158fe759e81fcdb1585)

---
The commit from master relies on the 'target_sys' parameter of runCmd, this was introduced in afa3d3ba00b40fd29e9852eeaa2c2c9b68f18659 (langdale and later). That commit could be backported alongside this commit however it adds '/bin' to the PATH by default which may cause issues (?). As such this commit is modified to add the TARGET_SYS path into the PATH environment passed into the runCmd call.

---
 meta/lib/oeqa/selftest/cases/minidebuginfo.py | 49 +++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/minidebuginfo.py
meta/lib/oeqa/selftest/cases/minidebuginfo.py new file +49 -0
new file mode 100644
index 0000000000..414dad64a3
@@ -0,0 +1,49 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+import os
+import subprocess
+import tempfile
+import shutil
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, runCmd
+
+
+class Minidebuginfo(OESelftestTestCase):
+    def test_minidebuginfo(self):
+        target_sys = get_bb_var("TARGET_SYS")
+        binutils = "binutils-cross-{}".format(get_bb_var("TARGET_ARCH"))
+
+        self.write_config("""
+PACKAGE_MINIDEBUGINFO = "1"
+IMAGE_FSTYPES = "tar.bz2"
+""")
+        bitbake("core-image-minimal {}:do_addto_recipe_sysroot".format(binutils))
+
+        deploy_dir = get_bb_var("DEPLOY_DIR_IMAGE")
+        native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", binutils)
+        readelf = get_bb_var("READELF", "core-image-minimal")
+
+        # add usr/bin/${TARGET_SYS} to PATH
+        env = os.environ.copy()
+        paths = [os.path.join(native_sysroot, "usr", "bin", target_sys)]
+        paths += env["PATH"].split(":")
+        env["PATH"] = ":".join(paths)
+
+        # confirm that executables and shared libraries contain an ELF section
+        # ".gnu_debugdata" which stores minidebuginfo.
+        with tempfile.TemporaryDirectory(prefix = "unpackfs-") as unpackedfs:
+            filename = os.path.join(deploy_dir, "core-image-minimal-{}.tar.bz2".format(self.td["MACHINE"]))
+            shutil.unpack_archive(filename, unpackedfs)
+
+            r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "bin", "busybox")],
+                    native_sysroot = native_sysroot, env = env)
+            self.assertIn(".gnu_debugdata", r.output)
+
+            r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "lib", "libc.so.6")],
+                    native_sysroot = native_sysroot, env = env)
+            self.assertIn(".gnu_debugdata", r.output)
+

---
2.38.1

#2 lore raw

PATCH 3/4 glibc-locale: Do not INHIBIT_DEFAULT_DEPS

In reply to Nathan Rossi

From: Nathan Rossi <[email protected]>

The glibc-locale recipe already partially depends on the base depends in order to satisfy the do_package dependency on binutils. However since commit d6ffd683bf6 NM has defaulted to gcc-nm, meaning do_package depends on gcc (for minidebuginfo).

Whilst the do_package task could handle having the dependencies explicitly defined (either in glibc-locale or in package.bbclass), setting these would require some amount of conditional dependency configuration (cross/crosssdk/etc.). Since both binutils and gcc are already dependencies of virtual/libc (although compilerlibs is not), having glibc-locale not inhibit the default depends simplifies the handling of this situation for both glibc-locale and package.bbclass.

(cherry picked from commit 8a40d0a6039e87a5b4b26a0e84dd797fe5c75cba)
---
 meta/recipes-core/glibc/glibc-locale.inc | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
meta/recipes-core/glibc/glibc-locale.inc +3 -8
index 7c14abfe99..7f70b3ca4f 100644
@@ -5,14 +5,9 @@ SUMMARY = "Locale data from glibc"
 BPN = "glibc"
 LOCALEBASEPN = "${MLPREFIX}glibc"
 
-# glibc-collateral.inc inhibits all default deps, but do_package needs objcopy
-# ERROR: objcopy failed with exit code 127 (cmd was 'i586-webos-linux-objcopy' --only-keep-debug 'glibc-locale/2.17-r0/package/usr/lib/gconv/IBM1166.so' 'glibc-locale/2.17-r0/package/usr/lib/gconv/.debug/IBM1166.so')
-# ERROR: Function failed: split_and_strip_files
-BINUTILSDEP = "virtual/${MLPREFIX}${TARGET_PREFIX}binutils:do_populate_sysroot"
-BINUTILSDEP:class-nativesdk = "virtual/${TARGET_PREFIX}binutils-crosssdk:do_populate_sysroot"
-do_package[depends] += "${BINUTILSDEP}"
-
-DEPENDS += "virtual/libc"
+# Do not inhibit default deps, do_package requires binutils/gcc for
+# objcopy/gcc-nm and glibc-locale depends on virtual/libc directly.
+INHIBIT_DEFAULT_DEPS = ""
 
 # Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
 # is set. The idea is to avoid running localedef on the target (at first boot)

---
2.38.1

#3 lore raw

PATCH 4/4 package: Fix handling of minidebuginfo with newer binutils

In reply to Nathan Rossi

From: Nathan Rossi <[email protected]>

Newer versions of binutils (2.38+) have changed how the "--only-keep-debug" of objcopy behaves when stripping non-debug sections from an ELF.

https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80

This change causes associated sections to be correctly marked as NOBITS with the section contents removed from the output. The side effect is that this causes issues with objcopy's ability to perform symbol and relocation stripping (-S/--strip-all) on the debug split ELF, such that with some object files (e.g. kernel modules) objcopy fails to strip symbols/relocations with an error like the following:

.../.debug/nls_cp950.ko[.rodata]: file truncated

Because of this it is now problematic to generate minidebuginfo for these types of ELF objects. However it is not typically useful to inject minidebuginfo into these types of ELFs, and other distributions (e.g. Fedora, referring to find-debuginfo.sh of debugedit) only insert minidebuginfo into executables and shared libraries.

This change causes the minidebuginfo injection to only apply to EXEC/DYN type ELFs, which limits the injection to executables and shared libraires.

Additionally this change fixes the parsing of the sections from the "readelf -W -S" output which was not accounting for the section index column having leading spaces for single digit index values e.g. "[ 1]".

(cherry picked from commit 2084cfcb3d15db3e02637f1cd63ab9c997f38a65)
---
 meta/classes/package.bbclass | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
meta/classes/package.bbclass +18 -3
index 97e97d2703..8b11fdd155 100644
@@ -484,16 +484,31 @@ def inject_minidebuginfo(file, dvar, dv, d):
         bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file))
         return
 
+    # minidebuginfo does not make sense to apply to ELF objects other than
+    # executables and shared libraries, skip applying the minidebuginfo
+    # generation for objects like kernel modules.
+    for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines():
+        if not line.strip().startswith("Type:"):
+            continue
+        elftype = line.split(":")[1].strip()
+        if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]):
+            bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file))
+            return
+        break
+
     # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo.
     # We will exclude all of these from minidebuginfo to save space.
     remove_section_names = []
     for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines():
-        fields = line.split()
-        if len(fields) < 8:
+        # strip the leading "  [ 1]" section index to allow splitting on space
+        if ']' not in line:
+            continue
+        fields = line[line.index(']') + 1:].split()
+        if len(fields) < 7:
             continue
         name = fields[0]
         type = fields[1]
-        flags = fields[7]
+        flags = fields[6]
         # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them
         if name.startswith('.debug_'):
             continue

---
2.38.1