rootfs_ipk, image: Add debug capture support

If ${IMAGE_BUILD_DEBUG} is set, construct a parallel tree containing the debug data for the packages that have been installed in the rootfs, then tar it up and deploy it alongside the rootfs images.

Signed-off-by Phil Blundell <[email protected]>

---
 meta/classes/image.bbclass      |    9 +++++++++
 meta/classes/rootfs_ipk.bbclass |   15 +++++++++++++++
 meta/conf/bitbake.conf          |    1 +
 3 files changed, 25 insertions(+)

meta/classes/image.bbclass +9 -0

index ab212b3..c82fba1 100644
@@ -445,3 +445,12 @@ do_package_write_deb[noexec] = "1"
 do_package_write_rpm[noexec] = "1"
 
 addtask rootfs before do_build
+
+fakeroot do_capture_debug() {
+	if [ "${IMAGE_BUILD_DEBUG}" = "1" -a -n "${IMAGE_ROOTFS_DBG}" -a -d "${IMAGE_ROOTFS_DBG}" ]; then
+		tar czf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.debug.tar.gz -C ${IMAGE_ROOTFS_DBG} .
+		[ "${IMAGE_NAME}" == "${IMAGE_LINK_NAME}" ] || ln -sf ${IMAGE_NAME}.debug.tar.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.debug.tar.gz
+	fi
+}
+
+addtask capture_debug before do_build after do_rootfs

meta/classes/rootfs_ipk.bbclass +8 -0

index 46e8d60..e4e74dd 100644
@@ -9,6 +9,8 @@ EXTRAOPKGCONFIG ?= ""
 ROOTFS_PKGMANAGE = "opkg opkg-collateral ${EXTRAOPKGCONFIG}"
 ROOTFS_PKGMANAGE_BOOTSTRAP  = "run-postinsts"
 
+IMAGE_BUILD_DEBUG ?= "0"
+
 do_rootfs[depends] += "opkg-native:do_populate_sysroot opkg-utils-native:do_populate_sysroot"
 do_rootfs[recrdeptask] += "do_package_write_ipk"
 

@@ -17,6 +19,7 @@ do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
 IPKG_ARGS = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS} --force-overwrite"
 # The _POST version also works when constructing the matching SDK
 IPKG_ARGS_POST = "-f ${IPKGCONF_TARGET} -o $INSTALL_ROOTFS_IPK --force-overwrite"
+IPKG_ARGS_DBG = "-f ${IPKGCONF_TARGET} -o ${IMAGE_ROOTFS_DBG} --force-overwrite"
 
 OPKG_PREPROCESS_COMMANDS = "package_update_index_ipk; package_generate_ipkg_conf"
 

@@ -88,6 +91,18 @@ fakeroot rootfs_ipk_do_rootfs () {
 		fi
 	fi
 
+	if [ "${IMAGE_BUILD_DEBUG}" = "1" -a -n "${IMAGE_ROOTFS_DBG}" ]; then
+		all_pkgs=`opkg-cl ${IPKG_ARGS} list-installed | awk '{ print $1 }'`
+		[ "${IMAGE_ROOTFS_DBG}" -ef "${IMAGE_ROOTFS}" ] || rm -rf ${IMAGE_ROOTFS_DBG}
+		mkdir -p ${IMAGE_ROOTFS_DBG}${opkglibdir}
+		opkg-cl ${IPKG_ARGS_DBG} update

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Quoted 3 lines · Richard Purdie

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Ah, possibly. I'll have a look. Can you give me a pointer to the mechanism you were thinking of?

Quoted 7 lines · Phil Blundell

Quoted 3 lines · Richard Purdie

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Ah, possibly. I'll have a look. Can you give me a pointer to the mechanism you were thinking of?

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition of dbg packages.

Or IMAGE_FEATURES += "dbg-img" ;-)

Quoted 1 line · Otavio Salvador

Or IMAGE_FEATURES += "dbg-img" ;-)

Where is that implemented? I couldn't find any obvious reference to dbg-img in any classes.

Quoted 4 lines · Phil Blundell

Quoted 1 line · Otavio Salvador

Or IMAGE_FEATURES += "dbg-img" ;-)

Where is that implemented? I couldn't find any obvious reference to dbg-img in any classes.

Sorry by cause a missunderstanding ... I was just pointing to a possible new feature you could add.

Quoted 8 lines · Otavio Salvador

Quoted 4 lines · Phil Blundell

Quoted 1 line · Otavio Salvador

Or IMAGE_FEATURES += "dbg-img" ;-)

Where is that implemented? I couldn't find any obvious reference to dbg-img in any classes.

Sorry by cause a missunderstanding ... I was just pointing to a possible new feature you could add.

that feature is already implemented though, in dbg-pkgs. If you don't mean dbg-img should do something other/more than what dbg-pkgs does?

Quoted 4 lines · Martin Ertsås

Quoted 3 lines · Otavio Salvador

Quoted 2 lines · Phil Blundell

Quoted 1 line · Otavio Salvador

Or IMAGE_FEATURES += "dbg-img" ;-)

Quoted 7 lines

Quoted 5 lines

Quoted 2 lines

Where is that implemented? I couldn't find any obvious reference to dbg-img in any classes.

Sorry by cause a missunderstanding ... I was just pointing to a possible new feature you could add.

that feature is already implemented though, in dbg-pkgs. If you don't mean dbg-img should do something other/more than what dbg-pkgs does?

To build a separated tar with the symbols, not on the image itself.

Quoted 10 lines · Otavio Salvador

Quoted 9 lines · Martin Ertsås

Quoted 8 lines · Otavio Salvador

Quoted 4 lines · Phil Blundell

Quoted 1 line · Otavio Salvador

Or IMAGE_FEATURES += "dbg-img" ;-)

Where is that implemented? I couldn't find any obvious reference to dbg-img in any classes.

Sorry by cause a missunderstanding ... I was just pointing to a possible new feature you could add.

Quoted 4 lines

Quoted 2 lines

that feature is already implemented though, in dbg-pkgs. If you don't mean dbg-img should do something other/more than what dbg-pkgs does?

To build a separated tar with the symbols, not on the image itself.

With only the symbols? Or a separate image with symbols, side by side with the stripped one? In the later case you could just add a export IMAGE_BASENAME = "my-dbg-image" in the image recipe. If you mean with only the symbols I don't think we have any features for that right now.

Quoted 7 lines · Phil Blundell

Quoted 3 lines · Richard Purdie

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Ah, possibly. I'll have a look. Can you give me a pointer to the mechanism you were thinking of?

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition of dbg packages.

Quoted 7 lines · Phil Blundell

Quoted 3 lines · Richard Purdie

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Ah, possibly. I'll have a look. Can you give me a pointer to the mechanism you were thinking of?

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition of dbg packages.

Quoted 11 lines · Richard Purdie

Quoted 7 lines · Phil Blundell

Quoted 3 lines · Richard Purdie

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Ah, possibly. I'll have a look. Can you give me a pointer to the mechanism you were thinking of?

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition of dbg packages.

I've somewhat belatedly gotten around to using this and encountered a couple of issuettes:

1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf which was causing ${PN} to be dragged into the image whenever ${PN}-dbg was installed. This is undesirable because it means that, for example, any image which includes sysvinit-pidof will end up installing sysvinit and this might cause quite drastic changes to the behaviour of the resulting image.

2. Having fixed the above I'm now left with a couple of file conflicts in the -dbg packages. For example, sysvinit and util-linux both ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to install /sbin/.debug/sulogin. This is unfortunate if you have an image which uses, say, sysvinit-pidof and util-linux-blkid.

I'm not quite sure what the right way to fix (2) is. I suppose in an ideal world the -dbg packages would be separated in the same way the parent binary packages are, but that doesn't look entirely straightforward to arrange.

Quoted 11 lines · Richard Purdie

Quoted 7 lines · Phil Blundell

Quoted 3 lines · Richard Purdie

Hasn't Paul added a general mechanism for doing this so we could add this feature a level higher so that it could be used by all packaging formats?

Ah, possibly. I'll have a look. Can you give me a pointer to the mechanism you were thinking of?

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition of dbg packages.

I've somewhat belatedly gotten around to using this and encountered a couple of issuettes:

1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf which was causing ${PN} to be dragged into the image whenever ${PN}-dbg was installed. This is undesirable because it means that, for example, any image which includes sysvinit-pidof will end up installing sysvinit and this might cause quite drastic changes to the behaviour of the resulting image.

2. Having fixed the above I'm now left with a couple of file conflicts in the -dbg packages. For example, sysvinit and util-linux both ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to install /sbin/.debug/sulogin. This is unfortunate if you have an image which uses, say, sysvinit-pidof and util-linux-blkid.

I'm not quite sure what the right way to fix (2) is. I suppose in an ideal world the -dbg packages would be separated in the same way the parent binary packages are, but that doesn't look entirely straightforward to arrange.

p.

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Is this a problem that they should have used the update-alternatives for sulogin? (Sounds like it might be a security issue though...) This would avoid the .debug conflict.

Quoted 3 lines · Mark Hatle

Is this a problem that they should have used the update-alternatives for sulogin? (Sounds like it might be a security issue though...) This would avoid the .debug conflict.

I dunno. It seems a bit sad to force use of update-alternatives just to deconflict the .debug data (since the binaries in question would never be installed at the same time anyway), and in general it seems like we want less update-alternatives rather than more of it.

But I guess that would indeed solve the problem and it's probably the lowest-effort thing that does. So maybe that's the way ahead.

p.

The alternative of course is to crease special -dbg packages for the two conflicting items. I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

Quoted 2 lines · Mark Hatle

The alternative of course is to crease special -dbg packages for the two conflicting items. I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

Yeah, indeed, that's what I suggested in my original email. At the time I thought that would be hard to arrange (in the general case), but having given it some further consideration perhaps it isn't all that bad after all.

p.

Quoted 8 lines · Phil Blundell

Quoted 2 lines · Mark Hatle

The alternative of course is to crease special -dbg packages for the two conflicting items. I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

Yeah, indeed, that's what I suggested in my original email. At the time I thought that would be hard to arrange (in the general case), but having given it some further consideration perhaps it isn't all that bad after all.

I certainly wouldn't be against an enhancement that tries to match up the binaries to their debug and create suitably named -dbg packages. The only tricky part is what to do with the associated sources? Since those are not arranged according to binaries, but generally for the whole recipe.

Quoted 10 lines · Mark Hatle

Quoted 8 lines · Phil Blundell

Quoted 2 lines · Mark Hatle

The alternative of course is to crease special -dbg packages for the two conflicting items. I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

Yeah, indeed, that's what I suggested in my original email. At the time I thought that would be hard to arrange (in the general case), but having given it some further consideration perhaps it isn't all that bad after all.

Quoted 4 lines

I certainly wouldn't be against an enhancement that tries to match up the binaries to their debug and create suitably named -dbg packages. The only tricky part is what to do with the associated sources? Since those are not arranged according to binaries, but generally for the whole recipe.

The sources can stay where they are, in ${PN}-dbg. There won't be any conflict there because the installed files are already namespaced by recipe.

Quoted 16 lines · Phil Blundell

Quoted 14 lines · Mark Hatle

Quoted 8 lines · Phil Blundell

Quoted 2 lines · Mark Hatle

The alternative of course is to crease special -dbg packages for the two conflicting items. I.e. foo-dbg, foo-sulogin-dbg, bar-dbg and bar-sulogin-dbg...

Yeah, indeed, that's what I suggested in my original email. At the time I thought that would be hard to arrange (in the general case), but having given it some further consideration perhaps it isn't all that bad after all.

I certainly wouldn't be against an enhancement that tries to match up the binaries to their debug and create suitably named -dbg packages. The only tricky part is what to do with the associated sources? Since those are not arranged according to binaries, but generally for the whole recipe.

Quoted 3 lines

The sources can stay where they are, in ${PN}-dbg. There won't be any conflict there because the installed files are already namespaced by recipe.

Maybe add a ${PN}-source with it? So any -dbg could rdepends on it?

--
Otavio Salvador O.S. Systems
E-mail: [email protected] http://www.ossystems.com.br
Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br

Quoted 2 lines

Something like IMAGE_FEATURES += "dbg-pkgs" should trigger the addition of dbg packages.

I've somewhat belatedly gotten around to using this and encountered a couple of issuettes:

1. There was a spurious-looking RRECOMMENDS_${PN}-dbg in bitbake.conf which was causing ${PN} to be dragged into the image whenever ${PN}-dbg was installed. This is undesirable because it means that, for example, any image which includes sysvinit-pidof will end up installing sysvinit and this might cause quite drastic changes to the behaviour of the resulting image.

2. Having fixed the above I'm now left with a couple of file conflicts in the -dbg packages. For example, sysvinit and util-linux both ship /sbin/sulogin and so sysvinit-dbg and util-linux-dbg both want to install /sbin/.debug/sulogin. This is unfortunate if you have an image which uses, say, sysvinit-pidof and util-linux-blkid.

I'm not quite sure what the right way to fix (2) is. I suppose in an ideal world the -dbg packages would be separated in the same way the parent binary packages are, but that doesn't look entirely straightforward to arrange.

On Thu, Apr 25, 2013 at 7:12 AM, Phil Blundell <[email protected]> wrote:

I'm not quite sure what the right way to fix (2) is. I suppose in an
ideal world the -dbg packages would be separated in the same way the
parent binary packages are, but that doesn't look entirely straightforward to arrange.

I had implemented something along these lines back in oe classic, when
I was at MontaVista. See
http://git.openembedded.org/openembedded/tree/classes/package_dbg.bbclass for that implementation. I haven't touched it or tried using it in some time, however.

a) Should we ditch FILES_${PN}-dbg and have it auto constructed from any .debug directories? This would be a rather nice automation/cleanup. I can't see a pressing reason not to do this.

b) Consider splitting it to a -dbg package per package where there are debug files as per above. There are pros and cons to this and I'm torn, see c).

c) Consider splitting the debug source into a separate package. If we did do b), the question is where should the sources go?

a) A binary from package X segfaults. You want to get good gdb data for why. You therefore install X-dbg. X links against Y. You therefore want the symbols/code for Y too so you can trace into the problem which may be in Y.

b) You want to compile against X, you install X-dev. You expect anything X needs to link with (e.g. through any .pc file) to also be present.

c) You want to rebuild X and hence install X-dev to ensure the build dependencies are present.

d) There was once an idea that you could do something like install core-image-minimal-dev to get the equivalent of core-image-minimal with dev-pkgs. I think we've found better ways to do this kind of thing now?

Quoted 9 lines · Chris Larson

On Thu, Apr 25, 2013 at 7:12 AM, Phil Blundell <[email protected]> wrote:

I'm not quite sure what the right way to fix (2) is. I suppose in an
ideal world the -dbg packages would be separated in the same way the
parent binary packages are, but that doesn't look entirely straightforward to arrange.

Quoted 3 lines

I had implemented something along these lines back in oe classic, when
I was at MontaVista. See
http://git.openembedded.org/openembedded/tree/classes/package_dbg.bbclass for that implementation. I haven't touched it or tried using it in some time, however.

I have to admit I've been wondering if we shouldn't overhaul the -dbg part of the system a bit. There are a few things I've been wondering about:

a) Should we ditch FILES_${PN}-dbg and have it auto constructed from any .debug directories? This would be a rather nice automation/cleanup. I can't see a pressing reason not to do this.

b) Consider splitting it to a -dbg package per package where there are debug files as per above. There are pros and cons to this and I'm torn, see c).

c) Consider splitting the debug source into a separate package. If we did do b), the question is where should the sources go?

In the bigger picture we have various dependency chain issues as the -dbg and -dev dependency changes are horrible. The question has always been whether X-dbg or X-dev should be usable to pull in sensible dependencies.

Thinking about the scenarios you might use these in:

a) A binary from package X segfaults. You want to get good gdb data for why. You therefore install X-dbg. X links against Y. You therefore want the symbols/code for Y too so you can trace into the problem which may be in Y.

b) You want to compile against X, you install X-dev. You expect anything X needs to link with (e.g. through any .pc file) to also be present.

c) You want to rebuild X and hence install X-dev to ensure the build dependencies are present.

d) There was once an idea that you could do something like install core-image-minimal-dev to get the equivalent of core-image-minimal with dev-pkgs. I think we've found better ways to do this kind of thing now?

In a/b/c, the usability fail is if you try to gdb/compile, find a dependency missing, install it and repeat this cycle several times over.

Given d) is dead, thankfully, does that let us rip some code out and improve the dependencies?

Cheers,

Richard

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core