Status of external toolchain support
I am starting work on a new target and thought I'd take a look at using a crosstool-NG external toolchain and I have a couple of questions about what is supposed to be supported.
I have successfully built an arm toolchain using eglibc and nptl (Thanks Yann!).
When configuring buildroot I set all my options the same as the toolchain, except for thread support where nptl was not an option. Does it matter that they don't match up? Should you even be able to set thread library when using an external toolchain?
When using crosstools who should build gdb? If buildroot builds it I have to make sure my toolchain is not read only? Can I just let crosstools build gdb for the host and only get buildroot to make gdb-server?
Thanks
Will
Will, All,
Quoted 7 lines · On Wednesday 12 May 2010 19:15:38 Will Wagner wrote:
I am starting work on a new target and thought I'd take a look at using a crosstool-NG external toolchain and I have a couple of questions about what is supposed to be supported.
When configuring buildroot I set all my options the same as the toolchain, except for thread support where nptl was not an option. Does it matter that they don't match up?
I'm not sure I understand the question... :-/
Buildroot _shall_ be configured to match the external toolchain features.
It is not possible to extract the external toolchain configuration prior to displaying the config menu, as the path to the toolchain is entered in that menu.
So one has to tell buildroot what is or is not supported by the external toolchain. These options are then checked for by the buildroot makefiles, and if they do not match, the build stops.
Quoted 2 lines
Should you even be able to set
thread library when using an external toolchain?
An external toolchain can be built to not support threading. It's an option in crosstool-NG, at least. Maybe other means to build toolchain also offer this possibility.
So yes, we also have to select threading model in buildroot, and it has to match the external toolchain capabilities.
Quoted 3 lines
When using crosstools who should build gdb? If buildroot builds it I have to make sure my toolchain is not read only? Can I just let crosstools build gdb for the host and only get buildroot to make gdb-server?
The same entity shall build both the cross gdb and the gdbserver, as those two are tightly coupled, because of the protocol to exchange information between them.
So either you get buildroot to build both, or crosstool-NG. You can't mix.
Definitely, gdb is on the border line, because it's a valuable tool to have in a toolchain, but it's not really part of the toolchain either, that's why it's possible to build a cross gdb and a gdbserver with crostool-NG. But there is currently no way to have buildroot copy the gdbserver from the debug-root to the staging/target dires of buildroot. I'm working to tightly integrate crosstool-NG as a backend to generate the toolchains, and once that's done, it will be possible to (although not at the first inclusion of this work in BR tree).
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Quoted 3 lines · Yann E. MORIN
The same entity shall build both the cross gdb and the gdbserver, as those two are tightly coupled, because of the protocol to exchange information between them.
Sorry, I don't understand where that limitation comes from.
The gdb remote protocol is standardized. It's plain ASCII, and I can't see how it would be affected by the choice of toolchain. There are plenty of third-party gdb servers that are not only built by different entities, but the server isn't even built from gdb sources.
As long as as the gdb server and client versions are compatible (e.g. they're using the same protocol) they should work fine.
Have you seen protocol failures betwee gdb server/client due to the fact that a different toolchain was used to build them? If so, my understand is that's a bug and should be reported to the gdb maintainers.
Signature · 3 lines
Grant Edwards grant.b.edwards Yow! I invented skydiving
at in 1989!
gmail.com Quoted 4 lines
Quoted 4 lines
Should you even be able to set
thread library when using an external toolchain?
Quoted 8 lines
An external toolchain can be built to not support threading. It's an option in crosstool-NG, at least. Maybe other means to build toolchain also offer this possibility.
So yes, we also have to select threading model in buildroot, and it has to match the external toolchain capabilities.
I understand that options that affect packages have to be set the same. I can also see that you need to know if the toolchain supports threads or not, however once you have an external toolchain surely the packages you build with it do not care what version of threading is used.
If you need to tell buildroot the type of threading used (linuxthread old/new or nptl) then currently buildroot does not check that they match up, as I have an nptl toolchain but am unable to select that in buildroot so it is set to linuxthreads new, but I got no error.
Will, All,
Quoted 7 lines · On Wednesday 12 May 2010 23:36:59 Will Wagner wrote:
Quoted 2 lines
So yes, we also have to select threading model in buildroot, and it has to match the external toolchain capabilities.
I understand that options that affect packages have to be set the same. I can also see that you need to know if the toolchain supports threads or not, however once you have an external toolchain surely the packages you build with it do not care what version of threading is used.
The only two (non-toolchain) packages, gdb and dmalloc, that depends on threads, refer to !BR2_PTHREADS_NONE.
Quoted 4 lines
If you need to tell buildroot the type of threading used (linuxthread old/new or nptl) then currently buildroot does not check that they match up, as I have an nptl toolchain but am unable to select that in buildroot so it is set to linuxthreads new, but I got no error.
Oh, I see what you mean:
- the prompt for the threading model is wrong when dealing with external
toolchains: we only need to know if threads are supported or not
- a check is missing to check coherency btw external toolchain threading capabilties, and buildroot configuration.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Grant, All,
Quoted 6 lines · On Wednesday 12 May 2010 23:34:41 Grant Edwards wrote:
Quoted 3 lines · Yann E. MORIN
The same entity shall build both the cross gdb and the gdbserver, as those two are tightly coupled, because of the protocol to exchange information between them.
Sorry, I don't understand where that limitation comes from.
Quoted 7 lines
The gdb remote protocol is standardized. It's plain ASCII, and I can't see how it would be affected by the choice of toolchain. There are plenty of third-party gdb servers that are not only built by different entities, but the server isn't even built from gdb sources.
As long as as the gdb server and client versions are compatible (e.g. they're using the same protocol) they should work fine.
That's what I meant. And there have been quite some reports where the cross gdb and the gdbserver came from two gdb versions, and they did not speak the same protocol.
Quoted 4 lines
Have you seen protocol failures betwee gdb server/client due to the fact that a different toolchain was used to build them? If so, my understand is that's a bug and should be reported to the gdb maintainers.
That was not what I meant. Sorry for the confusion. I was trying to say:
"build the cross gdb *and* the gdbserver from the same version of gdb "so there is no protocol mis-match between the two
Of course, the gdbserver stub in the kernel, in u-boot, in redboot, in my jtag ICE, and in many other places are not built from the same source.
And in that case, it can happen that using too old/new a cross gdb with them will lead to issues, as they might not use the same protocol.
Yes, I've seen that happen, with an "old" jtag ICE, and a "recent" gdb.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Quoted 3 lines · Yann E. MORIN
That was not what I meant. Sorry for the confusion. I was trying to say:
"build the cross gdb *and* the gdbserver from the same version of gdb "so there is no protocol mis-match between the two
That's good advice. In theory, they don't have to be _exactly_ the same version as long as there weren't incompatible protocol changes between those two versions. But, figuring out where the incompatible protocol changes are is probably a lot more work that just using the same version for server and client.
Quoted 7 lines
Of course, the gdbserver stub in the kernel, in u-boot, in redboot, in my jtag ICE, and in many other places are not built from the same source.
And in that case, it can happen that using too old/new a cross gdb with them will lead to issues, as they might not use the same protocol.
Yes, I've seen that happen, with an "old" jtag ICE, and a "recent" gdb.
No doubt. There is some effort to make gdb/protocol changes so that things are backwards compatible, but it's not always successful.
If you're building both the server and client, the safe thing to do is always build them from the same version.
Signature · 3 lines
Grant Edwards grant.b.edwards Yow! Th' MIND is the Pizza
at Palace of th' SOUL
gmail.com Quoted 10 lines · Yann E. MORIN
Quoted 4 lines
If you need to tell buildroot the type of threading used (linuxthread old/new or nptl) then currently buildroot does not check that they match up, as I have an nptl toolchain but am unable to select that in buildroot so it is set to linuxthreads new, but I got no error.
Oh, I see what you mean:
- the prompt for the threading model is wrong when dealing with external
toolchains: we only need to know if threads are supported or not
- a check is missing to check coherency btw external toolchain threading capabilties, and buildroot configuration.
I think the problem is that the file toolchain\Config.in.2 is configured to only allow BR2_THREADS_NATIVE (NPTL threading option) to be selected if uclibc was set to the latest snapshot. This was a change made in February. It seems like this selection should also be allowed if glibc is the C library.
Quoted 4 lines
Regards,
Yann E. MORIN.
Mark
Hello,
Quoted 3 lines · On Wed, 12 May 2010 18:15:38 +0100 Will Wagner <[email protected]> wrote:
When configuring buildroot I set all my options the same as the toolchain, except for thread support where nptl was not an option. Does it matter that they don't match up?
It matters if some packages do depend on these options. The only reason why we have BR2_INET_RPC, BR2_INET_IPV6 and other similar options in the external toolchain case is because some packages do depend on them. As we can't detect their value at menuconfig time, we have to ask the user to provide the appropriate values according to its toolchain configuration.
Quoted 1 line
Should you even be able to set thread library when using an external toolchain?
The thread type (none, linuxthreads, linuxthreads old and nptl) can already be set from the menuconfig interface in external toolchain mode.
However, the "NPTL" selection isn't visible, as it depends on BR2_UCLIBC_VERSION_SNAPSHOT. So we should probably :
1. Make BR2_PTHREADS_NATIVE depends on BR2_UCLIBC_VERSION_SNAPSHOT *OR* BR2_TOOLCHAIN EXTERNAL
2. Improve toolchain/external-toolchain/ext-tool.mk so that it checks that the value selected for the thread implementation actually matches the one available in the external toolchain.
Quoted 3 lines
When using crosstools who should build gdb? If buildroot builds it I have to make sure my toolchain is not read only? Can I just let crosstools build gdb for the host and only get buildroot to make gdb-server?
I always build the cross gdb and gdbserver with crosstool-ng. This is an area in which I haven't clarified the relation between Buildroot and the external toolchain.
We should probably :
*) Hide the option to build the cross-gdb in Buildroot in external
toolchain mode, making the assumption that the external toolchain should provide its own cross gdb.
*) Add an option to ask Buildroot to either build a new gdbserver, or
copy the one available for the external toolchain (so that we are sure its version matches the version of the cross-gdb in the external toolchain)
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Thomas, All,
Quoted 4 lines · On Thursday 13 May 2010 17:22:02 Thomas Petazzoni wrote:
Quoted 1 line · On Wed, 12 May 2010 18:15:38 +0100 Will Wagner <[email protected]> wrote:
Should you even be able to set thread library when using an external toolchain?
Quoted 8 lines
The thread type (none, linuxthreads, linuxthreads old and nptl) can already be set from the menuconfig interface in external toolchain mode.
However, the "NPTL" selection isn't visible, as it depends on
BR2_UCLIBC_VERSION_SNAPSHOT. So we should probably :1. Make
BR2_PTHREADS_NATIVEdepends onBR2_UCLIBC_VERSION_SNAPSHOT*OR*BR2_TOOLCHAINEXTERNAL
Not really needed in fact, as what packages really want to know is whether the toolchain supports threads or not. The threading model (LinuxThreads or native) does not really matter, in fact.
Quoted 3 lines
2. Improve
toolchain/external-toolchain/ext-tool.mkso that it checks that the value selected for the thread implementation actually matches the one available in the external toolchain.
That is missing, yes.
Quoted 8 lines
Quoted 3 lines
When using crosstools who should build gdb? If buildroot builds it I have to make sure my toolchain is not read only? Can I just let crosstools build gdb for the host and only get buildroot to make gdb-server?
I always build the cross gdb and gdbserver with crosstool-ng. This is an area in which I haven't clarified the relation between Buildroot and the external toolchain.
Quoted 5 lines
We should probably :
*) Hide the option to build the cross-gdb in Buildroot in external
toolchain mode, making the assumption that the external toolchain should provide its own cross gdb.
With a purely external toolchain, we can't make this assumption sanely. So in this case, it's better to keep the option to build cross-gdb (and gdbserver).
Quoted 4 lines
*) Add an option to ask Buildroot to either build a new gdbserver, or
copy the one available for the external toolchain (so that we are sure its version matches the version of the cross-gdb in the external toolchain)
In the work I'm doing, I'm preparing the fact that cross-gdb + gdbserver can be built with crosstool-NG *or* buildroot:
- if buildroot builds them, then they get disabled in crosstool-NG.
- if crosstool-NG builds them, then gdbserver is copied to staging/target, and the crossgdb is in the toolchain path.
Only the first is currently handled, not the second part.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Quoted 6 lines · On Thu, 13 May 2010 19:21:50 +0200 "Yann E. MORIN"…
Quoted 2 lines
1. Make
BR2_PTHREADS_NATIVEdepends onBR2_UCLIBC_VERSION_SNAPSHOT*OR*BR2_TOOLCHAINEXTERNALNot really needed in fact, as what packages really want to know is whether the toolchain supports threads or not. The threading model (LinuxThreads or native) does not really matter, in fact.
Yeah, in theory, yes. However, I have seen packages such as Webkit that did compile with a NPTL-powered libc and not with a linuxthreads-powered libc. See 596bcb63fb26052b86c1271913747e701883dbfa and https://bugs.busybox.net/show_bug.cgi?id=1405. But, admitted, it's more a bug in uClibc than a real issue.
Quoted 9 lines
Quoted 5 lines
We should probably :
*) Hide the option to build the cross-gdb in Buildroot in external
toolchain mode, making the assumption that the external toolchain should provide its own cross gdb.With a purely external toolchain, we can't make this assumption sanely. So in this case, it's better to keep the option to build cross-gdb (and gdbserver).
Why not, but then we should properly handle the case where the toolchain has a cross-gdb and the user has selected to build one. How should we handle this ?
Quoted 5 lines
Quoted 4 lines
*) Add an option to ask Buildroot to either build a new gdbserver, or
copy the one available for the external toolchain (so that we are sure its version matches the version of the cross-gdb in the external toolchain)
Quoted 7 lines
In the work I'm doing, I'm preparing the fact that cross-gdb + gdbserver can be built with crosstool-NG *or* buildroot:
- if buildroot builds them, then they get disabled in crosstool-NG.
- if crosstool-NG builds them, then gdbserver is copied to staging/target, and the crossgdb is in the toolchain path.Only the first is currently handled, not the second part.
Yes, this is ok, but even if we're going to add Crosstool-NG as a backend, I would like to keep the external toolchain support, and thus improve it to fix those issues.
Regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Thomas, All,
Quoted 8 lines · On Thursday 13 May 2010 22:08:51 Thomas Petazzoni wrote:
Quoted 3 lines · On Thu, 13 May 2010 19:21:50 +0200 "Yann E. MORIN"…
Not really needed in fact, as what packages really want to know is whether the toolchain supports threads or not. The threading model (LinuxThreads or native) does not really matter, in fact.
Yeah, in theory, yes. However, I have seen packages such as Webkit that did compile with a NPTL-powered libc and not with a linuxthreads-powered libc.
Ah... Webkit...
Quoted 3 lines
See
596bcb63fb26052b86c1271913747e701883dbfa
and https://bugs.busybox.net/show_bug.cgi?id=1405. But, admitted, it's more a bug in uClibc than a real issue.
I see. Thanks for the pointer. I guess that we then need to get the NPTL selection with external toolchain as well.
Quoted 6 lines
Quoted 3 lines
With a purely external toolchain, we can't make this assumption sanely. So in this case, it's better to keep the option to build cross-gdb (and gdbserver).
Why not, but then we should properly handle the case where the toolchain has a cross-gdb and the user has selected to build one. How should we handle this ?
Keep config options as-is, and then add the coherency check, that is if the user did select to build cross-gdb, but it is already present alongside the cross-gcc, then bail out.
Quoted 11 lines
Quoted 7 lines
In the work I'm doing, I'm preparing the fact that cross-gdb + gdbserver can be built with crosstool-NG *or* buildroot:
- if buildroot builds them, then they get disabled in crosstool-NG.
- if crosstool-NG builds them, then gdbserver is copied to staging/target, and the crossgdb is in the toolchain path.Only the first is currently handled, not the second part.
Yes, this is ok, but even if we're going to add Crosstool-NG as a backend, I would like to keep the external toolchain support, and thus improve it to fix those issues.
Oh, I'm not removing external toolchain at all! In fact, I even build upon the existing code. So we can share as much as possible (because, after all, using crostool-NG as a backend is just a kind of external toolchain, but one that we can force some options of, rather than checking afterwards).
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'