oeqa/selftest/minidebuginfo: Create selftest for minidebuginfo
PATCH 1/3 oeqa/selftest/minidebuginfo: Create selftest for minidebuginfo
From: Nathan Rossi <[email protected]>
Add a new selftest to validate minidebuginfo support. This selftest builds a complete target image with PACKAGE_MINIDEBUGINFO enabled. ELFs included in the image are expected to have minidebuginfo included in the resulting executables and shared libraries, the self test validates this by unpacking the image and checking for the associated ".gnu_debugdata" section on busybox and libc ELFs.
--- meta/lib/oeqa/selftest/cases/minidebuginfo.py | 43 +++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases/minidebuginfo.py
meta/lib/oeqa/selftest/cases/minidebuginfo.py new file +43 -0
@@ -0,0 +1,43 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# +import os +import subprocess +import tempfile +import shutil + +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, get_bb_var, runCmd + + +class Minidebuginfo(OESelftestTestCase): + def test_minidebuginfo(self): + target_sys = get_bb_var("TARGET_SYS") + binutils = "binutils-cross-{}".format(get_bb_var("TARGET_ARCH")) + + self.write_config(""" +PACKAGE_MINIDEBUGINFO = "1" +IMAGE_FSTYPES = "tar.bz2" +""") + bitbake("core-image-minimal {}:do_addto_recipe_sysroot".format(binutils)) + + deploy_dir = get_bb_var("DEPLOY_DIR_IMAGE") + native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", binutils) + readelf = get_bb_var("READELF", "core-image-minimal") + + # confirm that executables and shared libraries contain an ELF section + # ".gnu_debugdata" which stores minidebuginfo. + with tempfile.TemporaryDirectory(prefix = "unpackfs-") as unpackedfs: + filename = os.path.join(deploy_dir, "core-image-minimal-{}.tar.bz2".format(self.td["MACHINE"])) + shutil.unpack_archive(filename, unpackedfs) + + r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "bin", "busybox")], + native_sysroot = native_sysroot, target_sys = target_sys) + self.assertIn(".gnu_debugdata", r.output) + + r = runCmd([readelf, "-W", "-S", os.path.join(unpackedfs, "lib", "libc.so.6")], + native_sysroot = native_sysroot, target_sys = target_sys) + self.assertIn(".gnu_debugdata", r.output) +
---
2.37.2
PATCH 2/3 glibc-locale: Do not INHIBIT_DEFAULT_DEPS
From: Nathan Rossi <[email protected]>
The glibc-locale recipe already partially depends on the base depends in order to satisfy the do_package dependency on binutils. However since commit d6ffd683bf6 NM has defaulted to gcc-nm, meaning do_package depends on gcc (for minidebuginfo).
Whilst the do_package task could handle having the dependencies explicitly defined (either in glibc-locale or in package.bbclass), setting these would require some amount of conditional dependency configuration (cross/crosssdk/etc.). Since both binutils and gcc are already dependencies of virtual/libc (although compilerlibs is not), having glibc-locale not inhibit the default depends simplifies the handling of this situation for both glibc-locale and package.bbclass.
--- meta/recipes-core/glibc/glibc-locale.inc | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
meta/recipes-core/glibc/glibc-locale.inc +3 -8
@@ -5,14 +5,9 @@ SUMMARY = "Locale data from glibc" BPN = "glibc" LOCALEBASEPN = "${MLPREFIX}glibc" -# glibc-collateral.inc inhibits all default deps, but do_package needs objcopy -# ERROR: objcopy failed with exit code 127 (cmd was 'i586-webos-linux-objcopy' --only-keep-debug 'glibc-locale/2.17-r0/package/usr/lib/gconv/IBM1166.so' 'glibc-locale/2.17-r0/package/usr/lib/gconv/.debug/IBM1166.so') -# ERROR: Function failed: split_and_strip_files -BINUTILSDEP = "virtual/${MLPREFIX}${TARGET_PREFIX}binutils:do_populate_sysroot" -BINUTILSDEP:class-nativesdk = "virtual/${TARGET_PREFIX}binutils-crosssdk:do_populate_sysroot" -do_package[depends] += "${BINUTILSDEP}" - -DEPENDS += "virtual/libc" +# Do not inhibit default deps, do_package requires binutils/gcc for +# objcopy/gcc-nm and glibc-locale depends on virtual/libc directly. +INHIBIT_DEFAULT_DEPS = "" # Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION # is set. The idea is to avoid running localedef on the target (at first boot)
---
2.37.2
PATCH 3/3 package: Fix handling of minidebuginfo with newer binutils
In reply to Nathan RossiFrom: Nathan Rossi <[email protected]>
Newer versions of binutils (2.38+) have changed how the "--only-keep-debug" of objcopy behaves when stripping non-debug sections from an ELF.
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80
This change causes associated sections to be correctly marked as NOBITS with the section contents removed from the output. The side effect is that this causes issues with objcopy's ability to perform symbol and relocation stripping (-S/--strip-all) on the debug split ELF, such that with some object files (e.g. kernel modules) objcopy fails to strip symbols/relocations with an error like the following:
.../.debug/nls_cp950.ko[.rodata]: file truncated
Because of this it is now problematic to generate minidebuginfo for these types of ELF objects. However it is not typically useful to inject minidebuginfo into these types of ELFs, and other distributions (e.g. Fedora, referring to find-debuginfo.sh of debugedit) only insert minidebuginfo into executables and shared libraries.
This change causes the minidebuginfo injection to only apply to EXEC/DYN type ELFs, which limits the injection to executables and shared libraires.
Additionally this change fixes the parsing of the sections from the "readelf -W -S" output which was not accounting for the section index column having leading spaces for single digit index values e.g. "[ 1]".
--- meta/classes-global/package.bbclass | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
meta/classes-global/package.bbclass +18 -3
@@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d): bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file)) return + # minidebuginfo does not make sense to apply to ELF objects other than + # executables and shared libraries, skip applying the minidebuginfo + # generation for objects like kernel modules. + for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines(): + if not line.strip().startswith("Type:"): + continue + elftype = line.split(":")[1].strip() + if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]): + bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file)) + return + break + # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo. # We will exclude all of these from minidebuginfo to save space. remove_section_names = [] for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines(): - fields = line.split() - if len(fields) < 8: + # strip the leading " [ 1]" section index to allow splitting on space + if ']' not in line: + continue + fields = line[line.index(']') + 1:].split() + if len(fields) < 7: continue name = fields[0] type = fields[1] - flags = fields[7] + flags = fields[6] # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them if name.startswith('.debug_'): continue
---
2.37.2
Re: [OE-core] [PATCH 3/3] package: Fix handling of minidebuginfo with newer binutils
Hello,
This series causes the following failure on the autobuilders:
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4354/steps/14/logs/stdio
2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_bindir: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_machine: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_alternatives: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_basic: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_binary: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_extlayer: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_func: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_glob: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_todir_glob: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_invalid: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_orig: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_patch: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_postinstall: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_renamed: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_script: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_subdir: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_todir: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_wildcard: ERROR
Quoted 87 lines · Nathan Rossi
From: Nathan Rossi <[email protected]>
Newer versions of binutils (2.38+) have changed how the "--only-keep-debug" of objcopy behaves when stripping non-debug sections from an ELF.
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80
This change causes associated sections to be correctly marked as NOBITS with the section contents removed from the output. The side effect is that this causes issues with objcopy's ability to perform symbol and relocation stripping (-S/--strip-all) on the debug split ELF, such that with some object files (e.g. kernel modules) objcopy fails to strip symbols/relocations with an error like the following:
.../.debug/nls_cp950.ko[.rodata]: file truncated
Because of this it is now problematic to generate minidebuginfo for these types of ELF objects. However it is not typically useful to inject minidebuginfo into these types of ELFs, and other distributions (e.g. Fedora, referring to
find-debuginfo.shof debugedit) only insert minidebuginfo into executables and shared libraries.This change causes the minidebuginfo injection to only apply to EXEC/DYN type ELFs, which limits the injection to executables and shared libraires.
Additionally this change fixes the parsing of the sections from the "readelf -W -S" output which was not accounting for the section index column having leading spaces for single digit index values e.g. "[ 1]".
Signed-off-by Nathan Rossi <[email protected]>--- meta/classes-global/package.bbclass | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)meta/classes-global/package.bbclass +18
-4@@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d): bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file)) return + # minidebuginfo does not make sense to apply to ELF objects other than + # executables and shared libraries, skip applying the minidebuginfo + # generation for objects like kernel modules. + for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines(): + if not line.strip().startswith("Type:"): + continue + elftype = line.split(":")[1].strip() + if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]): + bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file)) + return + break + # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo. # We will exclude all of these from minidebuginfo to save space. remove_section_names = [] for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines(): - fields = line.split() - if len(fields) < 8: + # strip the leading " [ 1]" section index to allow splitting on space + if ']' not in line: + continue + fields = line[line.index(']') + 1:].split() + if len(fields) < 7: continue name = fields[0] type = fields[1] - flags = fields[7] + flags = fields[6] # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them if name.startswith('.debug_'): continue ---2.37.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172617): https://lists.openembedded.org/g/openembedded-core/message/172617
Mute This Topic: https://lists.openembedded.org/mt/94752153/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH 3/3] package: Fix handling of minidebuginfo with newer binutils
Quoted 6 lines · Alexandre Belloni
Hello,
This series causes the following failure on the autobuilders:
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/4354/steps/14/logs/stdio
I looked into these failures, the SRC_URI error appears to be an issue with the lic_checksum test itself, and it presents without this series. I have posted a patch to resolve it
https://lists.openembedded.org/g/openembedded-core/message/172945.
The other errors around "test.bb" bash hash changes do not appear to be errors from the recipetools bitbake execution (note the "2 ERROR messages"). I suspect they are just captured during the fetch selftest cases in a common stderr buffer? I was unable to reproduce them locally when running fetch/recipetool.
Regards,
Nathan
Quoted 115 lines
2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_bindir: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_add_machine: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_alternatives: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_basic: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_binary: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_extlayer: ERROR 2022-11-06 00:37:54,099 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_func: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_glob: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_inst_todir_glob: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_invalid: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_orig: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_patch: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_postinstall: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_renamed: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_script: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_subdir: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_todir: ERROR 2022-11-06 00:37:54,100 - oe-selftest - INFO - RESULTS - recipetool.RecipetoolAppendTests.test_recipetool_appendfile_wildcard: ERRORQuoted 79 lines · Nathan Rossi
From: Nathan Rossi <[email protected]>
Newer versions of binutils (2.38+) have changed how the "--only-keep-debug" of objcopy behaves when stripping non-debug sections from an ELF.
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80
This change causes associated sections to be correctly marked as NOBITS with the section contents removed from the output. The side effect is that this causes issues with objcopy's ability to perform symbol and relocation stripping (-S/--strip-all) on the debug split ELF, such that with some object files (e.g. kernel modules) objcopy fails to strip symbols/relocations with an error like the following:
.../.debug/nls_cp950.ko[.rodata]: file truncated
Because of this it is now problematic to generate minidebuginfo for these types of ELF objects. However it is not typically useful to inject minidebuginfo into these types of ELFs, and other distributions (e.g. Fedora, referring to
find-debuginfo.shof debugedit) only insert minidebuginfo into executables and shared libraries.This change causes the minidebuginfo injection to only apply to EXEC/DYN type ELFs, which limits the injection to executables and shared libraires.
Additionally this change fixes the parsing of the sections from the "readelf -W -S" output which was not accounting for the section index column having leading spaces for single digit index values e.g. "[ 1]".
Signed-off-by Nathan Rossi <[email protected]>--- meta/classes-global/package.bbclass | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)meta/classes-global/package.bbclass +18
-4@@ -490,16 +490,31 @@ def inject_minidebuginfo(file, dvar, dv, d): bb.debug(1, 'ELF file {} has no debuginfo, skipping minidebuginfo injection'.format(file)) return + # minidebuginfo does not make sense to apply to ELF objects other than + # executables and shared libraries, skip applying the minidebuginfo + # generation for objects like kernel modules. + for line in subprocess.check_output([readelf, '-h', debugfile], universal_newlines=True).splitlines(): + if not line.strip().startswith("Type:"): + continue + elftype = line.split(":")[1].strip() + if not any(elftype.startswith(i) for i in ["EXEC", "DYN"]): + bb.debug(1, 'ELF file {} is not executable/shared, skipping minidebuginfo injection'.format(file)) + return + break + # Find non-allocated PROGBITS, NOTE, and NOBITS sections in the debuginfo. # We will exclude all of these from minidebuginfo to save space. remove_section_names = [] for line in subprocess.check_output([readelf, '-W', '-S', debugfile], universal_newlines=True).splitlines(): - fields = line.split() - if len(fields) < 8: + # strip the leading " [ 1]" section index to allow splitting on space + if ']' not in line: + continue + fields = line[line.index(']') + 1:].split() + if len(fields) < 7: continue name = fields[0] type = fields[1] - flags = fields[7] + flags = fields[6] # .debug_ sections will be removed by objcopy -S so no need to explicitly remove them if name.startswith('.debug_'): continue ---2.37.2
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172617): https://lists.openembedded.org/g/openembedded-core/message/172617
Mute This Topic: https://lists.openembedded.org/mt/94752153/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com