package.bbclass: improve auto debug package splitting order
cover package.bbclass: improve auto debug package splitting order
Hi RP,
Is it necessary to split multiple sub dbg packages or only one dbg is allowed? If the former, this fix makes sense.
I know in oe-core, almost one dbg package per recipe, but in oe, 21 recipes have extra sub dbg package and 18 of them did not set NOAUTOPACKAGEDEBUG = "1".
So rather than modify them one by one tediously, how about to improve auto debug package splitting order? If ${PN}-dbg exists, split the unsplit debug file to ${PN}-dbg, if ${PN}-dbg does not exist, still split them to first dbg package.
//Hongxu
The following changes since commit d180dc758710c7259d45eeb9304e7284a8fd8825:
oeqa/selftest/recipetool: fix non-determinism in cmake test (2018-09-13 16:28:26 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib hongxu/dbg-split
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=hongxu/dbg-split
Hongxu Jia (2):
package.bbclass: improve auto debug package splitting order selftest/package: improve case to test debug package splitting
.../selftest-hardlink/selftest-hardlink.bb | 4 +++- meta/classes/package.bbclass | 10 +++++++- meta/lib/oeqa/selftest/cases/package.py | 27 ++++++++++++++++------ 3 files changed, 32 insertions(+), 9 deletions(-)
Signature · 1 line
2.7.4
PATCH 1/2 package.bbclass: improve auto debug package splitting order
Since `da5ec06 package: Add auto package splitting of .debug files' applied in oe-core, the first dbg package will have all debug files automatically.
If there are several sub dbg packages, except the first one, the others are empty.
Even though var-NOAUTOPACKAGEDEBUG is provided for manual control, but many recipes (such as in oe) requires to pick up a few debug files into sub dbg packages, and then split the rest to ${PN}-dbg. If set NOAUTOPACKAGEDEBUG = "1", they have to do the extra splitting manually.
So improve splitting order, if ${PN}-dbg exists, split the unsplit debug file to ${PN}-dbg, if ${PN}-dbg does not exist, still split them to first dbg package.
--- meta/classes/package.bbclass | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
meta/classes/package.bbclass +9 -1
@@ -1175,6 +1175,14 @@ python populate_packages () { if "/.debug/" in path or path.endswith("/.debug"): debug.append(path) + pn_dbg = pn + "-dbg" + # If ${PN}-dbg does not exist, use first *-dbg to replace + if pn_dbg not in package_list: + for pkg in package_list: + if pkg.endswith("-dbg"): + pn_dbg = pkg + break + for pkg in package_list: root = os.path.join(pkgdest, pkg) bb.utils.mkdirhier(root)
@@ -1188,7 +1196,7 @@ python populate_packages () { origfiles = filesvar.split() files, symlink_paths = files_from_filevars(origfiles) - if autodebug and pkg.endswith("-dbg"): + if autodebug and pn_dbg == pkg: files.extend(debug) for file in files:
Signature · 1 line
2.7.4
PATCH 2/2 selftest/package: improve case to test debug package splitting
In reply to Hongxu JiaTweak recipe selftest-hardlink
- Add ${PN}-bin, ${PN}-bin-dbg
Tweak case test_gdb_hardlink_debug
- Test gdb debug package separately, verify the debug package splitting works correctly.
[Test without commit `package.bbclass: improve auto debug package splitting order']
2018-09-14 15:58:06,233 - oe-selftest - INFO - ======================================================================
2018-09-14 15:58:06,233 - oe-selftest - INFO - FAIL: test_gdb_hardlink_debug (package.PackageTests)
2018-09-14 15:58:06,233 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-09-14 15:58:06,233 - oe-selftest - INFO - Traceback (most recent call last):
File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 123, in test_gdb_hardlink_debug
self._test_gdb_debug(packages, binaries)
File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 162, in _test_gdb_debug
self.fail('GDB %s failed' % binary)AssertionError: GDB /usr/libexec/hello3 failed
[Test without commit `package.bbclass: improve auto debug package splitting order']
--- .../selftest-hardlink/selftest-hardlink.bb | 4 +++- meta/lib/oeqa/selftest/cases/package.py | 27 ++++++++++++++++------ 2 files changed, 23 insertions(+), 8 deletions(-)
meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb +3 -1
@@ -26,5 +26,7 @@ do_install () { } RDEPENDS_${PN}-gdb += "gdb" -PACKAGES =+ "${PN}-gdb" +PACKAGES =+ "${PN}-gdb ${PN}-bin ${PN}-bin-dbg" FILES_${PN}-gdb = "${bindir}/gdb.sh" +FILES_${PN}-bin-dbg = "${bindir}/.debug/*" +
meta/lib/oeqa/selftest/cases/package.py +20 -7
@@ -115,9 +115,25 @@ class PackageTests(OESelftestTestCase): # Verify gdb to read symbols from separated debug hardlink file correctly def test_gdb_hardlink_debug(self): - features = 'IMAGE_INSTALL_append = " selftest-hardlink"\n' - features += 'IMAGE_INSTALL_append = " selftest-hardlink-dbg"\n' - features += 'IMAGE_INSTALL_append = " selftest-hardlink-gdb"\n' + packages = ['selftest-hardlink', + 'selftest-hardlink-dbg', + 'selftest-hardlink-gdb'] + binaries = ['/usr/libexec/hello3', + '/usr/libexec/hello4'] + self._test_gdb_debug(packages, binaries) + + packages = ['selftest-hardlink-bin', + 'selftest-hardlink-bin-dbg', + 'selftest-hardlink-gdb'] + binaries = ['/usr/bin/hello1', + '/usr/bin/hello2'] + self._test_gdb_debug(packages, binaries) + + def _test_gdb_debug(self, packages, binaries): + features = "" + for pkg in packages: + features += 'IMAGE_INSTALL_append = " %s"\n' % pkg + self.write_config(features) bitbake("core-image-minimal")
@@ -141,9 +157,6 @@ class PackageTests(OESelftestTestCase): return False with runqemu('core-image-minimal') as qemu: - for binary in ['/usr/bin/hello1', - '/usr/bin/hello2', - '/usr/libexec/hello3', - '/usr/libexec/hello4']: + for binary in binaries: if not gdbtest(qemu, binary): self.fail('GDB %s failed' % binary)
Signature · 1 line
2.7.4
PATCH v2 2/2 selftest/package: improve case to test debug package splitting
Tweak recipe selftest-hardlink
- Add ${PN}-bin, ${PN}-bin-dbg
Tweak test_preserve_sparse_hardlinks
- s/selftest-hardlink/selftest-hardlink-bin/
Tweak case test_gdb_hardlink_debug
- Test gdb debug package separately, verify the debug package splitting works correctly.
[Test without commit `package.bbclass: improve auto debug package splitting order']
2018-09-14 15:58:06,233 - oe-selftest - INFO - ======================================================================
2018-09-14 15:58:06,233 - oe-selftest - INFO - FAIL: test_gdb_hardlink_debug (package.PackageTests)
2018-09-14 15:58:06,233 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-09-14 15:58:06,233 - oe-selftest - INFO - Traceback (most recent call last):
File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 123, in test_gdb_hardlink_debug
self._test_gdb_debug(packages, binaries)
File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 162, in _test_gdb_debug
self.fail('GDB %s failed' % binary)AssertionError: GDB /usr/libexec/hello3 failed
[Test without commit `package.bbclass: improve auto debug package splitting order']
--- .../selftest-hardlink/selftest-hardlink.bb | 4 ++- meta/lib/oeqa/selftest/cases/package.py | 31 +++++++++++++++------- 2 files changed, 25 insertions(+), 10 deletions(-)
meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb +3 -1
@@ -26,5 +26,7 @@ do_install () { } RDEPENDS_${PN}-gdb += "gdb" -PACKAGES =+ "${PN}-gdb" +PACKAGES =+ "${PN}-gdb ${PN}-bin ${PN}-bin-dbg" FILES_${PN}-gdb = "${bindir}/gdb.sh" +FILES_${PN}-bin-dbg = "${bindir}/.debug/*" +
meta/lib/oeqa/selftest/cases/package.py +22 -9
@@ -98,10 +98,10 @@ class PackageTests(OESelftestTestCase): def checkfiles(): # Recipe creates 4 hardlinked files, there is a copy in package/ and a copy in packages-split/ # so expect 8 in total. - self.assertEqual(os.stat(dest + "/selftest-hardlink" + bindir + "/hello1").st_nlink, 8) + self.assertEqual(os.stat(dest + "/selftest-hardlink-bin" + bindir + "/hello1").st_nlink, 8) # Test a sparse file remains sparse - sparsestat = os.stat(dest + "/selftest-hardlink" + bindir + "/sparsetest") + sparsestat = os.stat(dest + "/selftest-hardlink-bin" + bindir + "/sparsetest") self.assertEqual(sparsestat.st_blocks, 0) self.assertEqual(sparsestat.st_size, 1048576)
@@ -115,9 +115,25 @@ class PackageTests(OESelftestTestCase): # Verify gdb to read symbols from separated debug hardlink file correctly def test_gdb_hardlink_debug(self): - features = 'IMAGE_INSTALL_append = " selftest-hardlink"\n' - features += 'IMAGE_INSTALL_append = " selftest-hardlink-dbg"\n' - features += 'IMAGE_INSTALL_append = " selftest-hardlink-gdb"\n' + packages = ['selftest-hardlink', + 'selftest-hardlink-dbg', + 'selftest-hardlink-gdb'] + binaries = ['/usr/libexec/hello3', + '/usr/libexec/hello4'] + self._test_gdb_debug(packages, binaries) + + packages = ['selftest-hardlink-bin', + 'selftest-hardlink-bin-dbg', + 'selftest-hardlink-gdb'] + binaries = ['/usr/bin/hello1', + '/usr/bin/hello2'] + self._test_gdb_debug(packages, binaries) + + def _test_gdb_debug(self, packages, binaries): + features = "" + for pkg in packages: + features += 'IMAGE_INSTALL_append = " %s"\n' % pkg + self.write_config(features) bitbake("core-image-minimal")
@@ -141,9 +157,6 @@ class PackageTests(OESelftestTestCase): return False with runqemu('core-image-minimal') as qemu: - for binary in ['/usr/bin/hello1', - '/usr/bin/hello2', - '/usr/libexec/hello3', - '/usr/libexec/hello4']: + for binary in binaries: if not gdbtest(qemu, binary): self.fail('GDB %s failed' % binary)
Signature · 1 line
2.7.4
Quoted 1 line · Hongxu Jia
Hi RP,
Ping RP,
//Hongxu
Quoted 32 lines
Is it necessary to split multiple sub dbg packages or only one dbg is allowed? If the former, this fix makes sense.
I know in oe-core, almost one dbg package per recipe, but in oe, 21 recipes have extra sub dbg package and 18 of them did not set NOAUTOPACKAGEDEBUG = "1".
So rather than modify them one by one tediously, how about to improve auto debug package splitting order? If ${PN}-dbg exists, split the unsplit debug file to ${PN}-dbg, if ${PN}-dbg does not exist, still split them to first dbg package.
//Hongxu
The following changes since commit
d180dc758710c7259d45eeb9304e7284a8fd8825:oeqa/selftest/recipetool: fix non-determinism in cmake test (2018-09-13 16:28:26 +0100)are available in the git repository at:
git://git.openembedded.org/openembedded-core-contribhongxu/dbg-split
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=hongxu/dbg-splitHongxu Jia (2):
package.bbclass: improve auto debug package splitting order selftest/package: improve case to test debug package splitting.../selftest-hardlink/selftest-hardlink.bb | 4 +++- meta/classes/package.bbclass | 10 +++++++- meta/lib/oeqa/selftest/cases/package.py | 27 ++++++++++++++++------ 3 files changed, 32 insertions(+), 9 deletions(-)
Quoted 4 lines · Hongxu Jia
Hi RP,
Is it necessary to split multiple sub dbg packages or only one dbg is allowed? If the former, this fix makes sense.
Only one is recommended and the code only works properly in some cases for one. I'd like to be in a position where we could have that as the only thing we supported but its simply not practical in some cases.
The debug package dependency handling probably needs a rethink but that would be 2.7 development cycle, its too late for 2.6 now.
Quoted 8 lines
I know in oe-core, almost one dbg package per recipe, but in oe, 21 recipes have extra sub dbg package and 18 of them did not set NOAUTOPACKAGEDEBUG = "1".
So rather than modify them one by one tediously, how about to improve auto debug package splitting order? If ${PN}-dbg exists, split the unsplit debug file to ${PN}-dbg, if ${PN}-dbg does not exist, still split them to first dbg package.
I don't really want to encourage multiple -dbg packages per recipe and I am a little concerned about making changes to the core packaging code with the release coming up. Fixing the recipes, although tedious may be safer/better rather than encouraging people to do things which aren't well supported.
Cheers,
Richard
Quoted 20 lines · On 2018年09月17日 20:06, [email protected] wrote:
Quoted 4 lines · Hongxu Jia
Hi RP,
Is it necessary to split multiple sub dbg packages or only one dbg is allowed? If the former, this fix makes sense.
Only one is recommended and the code only works properly in some cases for one. I'd like to be in a position where we could have that as the only thing we supported but its simply not practical in some cases.
The debug package dependency handling probably needs a rethink but that would be 2.7 development cycle, its too late for 2.6 now.
Quoted 8 lines
I know in oe-core, almost one dbg package per recipe, but in oe, 21 recipes have extra sub dbg package and 18 of them did not set NOAUTOPACKAGEDEBUG = "1".
So rather than modify them one by one tediously, how about to improve auto debug package splitting order? If ${PN}-dbg exists, split the unsplit debug file to ${PN}-dbg, if ${PN}-dbg does not exist, still split them to first dbg package.
Quoted 5 lines
I don't really want to encourage multiple -dbg packages per recipe and I am a little concerned about making changes to the core packaging code with the release coming up. Fixing the recipes, although tedious may be safer/better rather than encouraging people to do things which aren't well supported.
Got it, I will fix affected recipes
Thanks for replying
//Hongxu
Quoted 3 lines
Cheers,
Richard