Skip to contentSkip to conversation

Debugging using sysroots and GDB

DISCUSSION Pascal Bach 5 participants 12 messages Feb 23, 2016 – Mar 8, 2016

lore
Share & tools
lore
#0 lore raw

Hi Everybody

Currently debugging using sysroots seems to work as long as the work folder containing the original source is available.
Once this work dir is gone the debugger is no longer able to find the source code. This is especially confusing if some packages are taken from sstate and others are built in the current projects, because for some libraries the sources can be found and for others not. It is hard at first to figure out the reason why one works and the other not.

In order to make it easier to debug using sysroots I propose to modify the how the sysroots is built to do somethins similar to rootfs:

1. Changing the source reference in the debug symbols to point to the source under /usr/src/debug
2. Copy the sources into /usr/src/debug under sysroots similar to how it is done for a debug rootfs.
3. Document that the user needs to add the following to the .gdbinit to make it work:

    ```
    set sysroot /project/oe/build/tmp/sysroots/<target_sysrtoot>
    set substitute-path /usr/src/debug /project/oe/build/tmp/sysroots/<target_sysrtoot>/usr/src/debug
    ```

This would also allow the sysroot to be relocatable as it is kind of standalone and everything required to debug is included.

What do you think? Does this proposal make sense, or did I miss something and this is completly unnecessary and there is an easier way already working?

Regards
Pascal

#1 lore raw

ping!
No comment on this topic?

Am 23.02.2016 um 15:09 schrieb Pascal Bach:

Quoted 21 lines

Hi Everybody

Currently debugging using sysroots seems to work as long as the work folder containing the original source is available.
Once this work dir is gone the debugger is no longer able to find the source code. This is especially confusing if some packages are taken from sstate and others are built in the current projects, because for some libraries the sources can be found and for others not. It is hard at first to figure out the reason why one works and the other not.

In order to make it easier to debug using sysroots I propose to modify the how the sysroots is built to do somethins similar to rootfs:

1. Changing the source reference in the debug symbols to point to the source under /usr/src/debug
2. Copy the sources into /usr/src/debug under sysroots similar to how it is done for a debug rootfs.
3. Document that the user needs to add the following to the .gdbinit to make it work:

    ```
    set sysroot /project/oe/build/tmp/sysroots/<target_sysrtoot>
    set substitute-path /usr/src/debug /project/oe/build/tmp/sysroots/<target_sysrtoot>/usr/src/debug
    ```

This would also allow the sysroot to be relocatable as it is kind of standalone and everything required to debug is included.

What do you think? Does this proposal make sense, or did I miss something and this is completly unnecessary and there is an easier way already working?

Regards
Pascal

#2 lore raw
In reply to Pascal Bach
Quoted 25 lines · Pascal Bach

Currently debugging using sysroots seems to work as long as the work folder containing the original source is available.
Once this work dir is gone the debugger is no longer able to find the source code. This is especially confusing if some packages are taken from sstate and others are built in the current projects, because for some libraries the sources can be found and for others not. It is hard at first to figure out the reason why one works and the other not.

In order to make it easier to debug using sysroots I propose to modify the how the sysroots is built to do somethins similar to rootfs:

1. Changing the source reference in the debug symbols to point to the source under /usr/src/debug
2. Copy the sources into /usr/src/debug under sysroots similar to how it is done for a debug rootfs.
3. Document that the user needs to add the following to the .gdbinit to make it work:

    ```
    set sysroot /project/oe/build/tmp/sysroots/<target_sysrtoot>
    set substitute-path /usr/src/debug
/project/oe/build/tmp/sysroots/<target_sysrtoot>/usr/src/debug
    ```
Quoted 6 lines

This would also allow the sysroot to be relocatable as it is kind of standalone and everything required to debug is included.

What do you think? Does this proposal make sense, or did I miss something and this is completly unnecessary and there is an easier way already working?

It really depends on your view of what the sysroot is used for. We intentionally strip a variety of things out of it as things work today basically for size/performance reasons. If we start putting debugging source in there, it will become huge and this will image the size of things like eSDK and affect the speed of operations like building from sstate.

We don't really support debugging from the sysroot. We could talk about changing that but I suspect people wouldn't like the performance penalty.

You might ask "ok, can we make it optional?". The trouble with that is more code paths which people will complain we're not testing and consequently an increased test matrix and even slower patch merging cycles.

So the question is, is this something we want to support this way?

Cheers,

Richard

#3 lore raw
Quoted 6 lines

It really depends on your view of what the sysroot is used for. We intentionally strip a variety of things out of it as things work today basically for size/performance reasons. If we start putting debugging source in there, it will become huge and this will image the size of things like eSDK and affect the speed of operations like building from sstate.

From my point of view debugging from sysroots is not mandatory if I can find a suitable alternative. One thing I want to avoid is to install -dbg packages to my image.

What I currently do is the following:

1. Enable Debugfs generation using: IMAGE_GEN_DEBUGFS = "1"
2. Switch debug split style to: PACKAGE_DEBUG_SPLIT_STYLE = "debug-file-directory"

After building the image I end up with a rootfs and a rootfs-dbg

3. Extract both rootfs and rootfs-dbg somewhere (/home/projects)
4. Setup gdb with the following .gdbinit:

```
set sysroot /home/projects/rootfs
set debug-file-directory /home/projects/rootfs-dbg/usr/lib/debug
set substitute-path /usr/src/debug /home/projects/rootfs-dbg/usr/src/debug
```

With this setup I'm able to debug trough all libraries on the system. So so far so good.

However for this to work I need to change the PACKAGE_DEBUG_SPLIT_STYLE to a non default and this made me wonder if I missed something.
- Is there a way to get this setup working with the default ".debug" style of OE?
- If not why is the .debug style the default instead of "debug-file-directory"?
- Is there any documentation? I was unable to find anything more than [1] in the Yocto Mega manual.
- Is there some best practice I missed?

[1] http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#platdev-gdb-remotedebug-setup

Quoted 3 lines

We don't really support debugging from the sysroot. We could talk about changing that but I suspect people wouldn't like the performance penalty.

Ok if this is not supported is there any documentation on whats the recommended debugging method?

Quoted 5 lines

You might ask "ok, can we make it optional?". The trouble with that is more code paths which people will complain we're not testing and consequently an increased test matrix and even slower patch merging cycles.

I agree, there should only be one recommended and supported way to do debugging and this way should be well documented.

Quoted 6 lines

So the question is, is this something we want to support this way?

Cheers,

Richard

Pascal

#4 lore raw
Quoted 36 lines · Pascal Bach
Quoted 10 lines

It really depends on your view of what the sysroot is used for. We intentionally strip a variety of things out of it as things work today
basically for size/performance reasons. If we start putting debugging
source in there, it will become huge and this will image the size of
things like eSDK and affect the speed of operations like building from
sstate.

From my point of view debugging from sysroots is not mandatory if I can find a suitable alternative.
One thing I want to avoid is to install -dbg packages to my image.

What I currently do is the following:

1. Enable Debugfs generation using: IMAGE_GEN_DEBUGFS = "1"
2. Switch debug split style to: PACKAGE_DEBUG_SPLIT_STYLE = "debug -file-directory"

After building the image I end up with a rootfs and a rootfs-dbg

3. Extract both rootfs and rootfs-dbg somewhere (/home/projects)
4. Setup gdb with the following .gdbinit:

```
set sysroot /home/projects/rootfs
set debug-file-directory /home/projects/rootfs-dbg/usr/lib/debug
set substitute-path /usr/src/debug /home/projects/rootfs
-dbg/usr/src/debug
```

With this setup I'm able to debug trough all libraries on the system. So so far so good.

Quoted 10 lines

However for this to work I need to change the PACKAGE_DEBUG_SPLIT_STYLE to a non default and this made me wonder if I missed something.
- Is there a way to get this setup working with the default ".debug" style of OE?
- If not why is the .debug style the default instead of "debug-file -directory"?
- Is there any documentation? I was unable to find anything more than
[1] in the Yocto Mega manual.
- Is there some best practice I missed?

Using this method is a supported approach. I'm a little surprised .debug directories don't work. Have you tried just setting:

set sysroot /home/projects/rootfs-dbg
set substitute-path /usr/src/debug /home/projects/rootfs -bbg/usr/src/debug

as I believe it should work with IMAGE_GEN_DEBUGFS=1 "out the box". If it doesn't that is a bug and we should try and look into fixing that.

Cheers,

Richard

#5 lore raw

Hi Richard

Quoted 6 lines

Using this method is a supported approach. I'm a little surprised .debug directories don't work. Have you tried just setting:

set sysroot /home/projects/rootfs-dbg
set substitute-path /usr/src/debug /home/projects/rootfs -bbg/usr/src/debug

This doesn't seem to work (tried only with dizzy).
The only way I could get this to work is to extract both the rootfs and the rootfs-dbg into the same place and then set sysroot to that location. Is this the way it is supposed to be?

Quoted 3 lines

as I believe it should work with IMAGE_GEN_DEBUGFS=1 "out the box". If it doesn't that is a bug and we should try and look into fixing that.

I'm currently using dizzy. I will give it another try with jethro next week maybe it is working there.

Pascal

#6 lore raw
Quoted 10 lines · Pascal Bach

Hi Richard

Quoted 6 lines

Using this method is a supported approach. I'm a little surprised .debug directories don't work. Have you tried just setting:

set sysroot /home/projects/rootfs-dbg
set substitute-path /usr/src/debug /home/projects/rootfs -bbg/usr/src/debug

This doesn't seem to work (tried only with dizzy).
The only way I could get this to work is to extract both the rootfs and

the rootfs-dbg into the same place and then set sysroot to that location.

Quoted 1 line

Is this the way it is supposed to be?

Yes

Quoted 5 lines
Quoted 4 lines

as I believe it should work with IMAGE_GEN_DEBUGFS=1 "out the box". If it doesn't that is a bug and we should try and look into fixing that.

I'm currently using dizzy. I will give it another try with jethro next

week maybe it is working there.

Quoted 7 lines

Pascal
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#7 lore raw
Quoted 6 lines
Quoted 3 lines

The only way I could get this to work is to extract both the rootfs and the rootfs-dbg into the same place and then set sysroot to that location. Is this the way it is supposed to be?

Yes

Ok I was hoping to keep them seperate.
This because I'm using the rootfs for NFS and by having the debug symbols in a seperate rootfs-dbg directory they would not be visible from the target. By setting PACKAGE_DEBUG_SPLIT_STYLE = "debug-file-directory" I got this working.

#8 lore raw
In reply to Richard Purdie
Quoted 3 lines · Richard Purdie
Quoted 2 lines · Pascal Bach

Currently debugging using sysroots seems to work as long as the work folder containing the original source is available.

I am using sysroot for debugging too. I have no

* INHERIT += "rm_work"

and added

INHIBIT_SYSROOT_STRIP = "1"

to my local.conf

I use this approach for very long time now and from my point of view it has one BIG advantage: I can use these settings for productive images and start remote debug within seconds without further rebuilding.
OK - default compiler optimization sometimes causes headaches but it works for solving >80% of my cases. In case I want to debug software that I have developed setting optimization recipe wise helps or I debug that on PC or target itself.

How much time is really saved by stripping sysroot?

Andreas

#9 lore raw
Quoted 2 lines · Andreas Müller

How much time is really saved by stripping sysroot?

It was more about reducing the size of the sstate than build performance. It's a toggle for precisely this reason: some users may not care about the space saving but want to use the sysroot for gdb.

Ross

#10 lore raw
In reply to Andreas Müller

Hi Andreas

Quoted 10 lines

I am using sysroot for debugging too. I have no

* INHERIT += "rm_work"

and added

INHIBIT_SYSROOT_STRIP = "1"

to my local.conf

Quoted 8 lines

I use this approach for very long time now and from my point of view it has one BIG advantage: I can use these settings for productive images and start remote debug within seconds without further rebuilding.
OK - default compiler optimization sometimes causes headaches but it works for solving >80% of my cases. In case I want to debug software that I have developed setting optimization recipe wise helps or I debug that on PC or target itself.

How do you deal with the case where there is no source in your work dir because the package was taken from sstate? The only solution I found so far is to build without sstate :(

Pascal

#11 lore raw
Quoted 19 lines · Pascal Bach

Hi Andreas

Quoted 10 lines

I am using sysroot for debugging too. I have no

* INHERIT += "rm_work"

and added

INHIBIT_SYSROOT_STRIP = "1"

to my local.conf

Quoted 8 lines

I use this approach for very long time now and from my point of view it has one BIG advantage: I can use these settings for productive images and start remote debug within seconds without further rebuilding.
OK - default compiler optimization sometimes causes headaches but it works for solving >80% of my cases. In case I want to debug software that I have developed setting optimization recipe wise helps or I debug that on PC or target itself.

Quoted 3 lines

How do you deal with the case where there is no source in your work dir because the package was taken from sstate? The only solution I found so far is to build without sstate :(

To be honest: I never had that - maybe because I am keeping a complete tmp for each machine I am working with.

Andreas