Skip to contentSkip to conversation

Help to fix Microblaze issues in Buildroot

Buildroot Thomas Petazzoni 4 participants 13 messages Oct 6, 2013 – Nov 20, 2013 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

Jan, Chris, Alvaro, Spenser,

You are amongst the four persons who contributed Microblaze related changes to Buildroot. We are currently having various build problems in our autobuilders caused by the Microblaze support, and it would be nice if you could help us fixing those issues and generally maintain the Microblaze support in a good shape in Buildroot.

At the moment, here are the latest autobuilder issues we have on Microblaze:

* libglib2 fails to build
http://autobuild.buildroot.org/results/32f/32fca009984ca0de4b4026ff6b4029104d0396c5/build-end.log

undefined reference to `__sync_fetch_and_or_4'

probably caused by the compiler not providing the right intrinsics, or libglib2 not falling back to other solutions when the intrinsics are not available.

* nettle fails to build
http://autobuild.buildroot.org/results/20b/20b2e2573985a687be29220a7df3b11a32894ba2/build-end.log

sha512-compress.c: In function '_nettle_sha512_compress':
sha512-compress.c:180:1: internal compiler error: in reload, at reload1.c:1059 Please submit a full bug report,

compiler bug, apparently.

* openssl fails to build
http://autobuild.buildroot.org/results/a050faaabbf89f9d3f324b821da84f8991a5aebb/build-end.log

sha512.c: In function 'sha512_block_data_order':
sha512.c:585:2: internal compiler error: in reload, at reload1.c:1059 Please submit a full bug report,
with preprocessed source if appropriate.

looks pretty much the same compiler bug as the one triggered by the nettle build.

* cairo fails to build
http://autobuild.buildroot.org/results/7054d168729c4e257000db93ab5137af1e1583cc/build-end.log

dragon.c: In function 'do_dragon':
dragon.c:162:1: internal compiler error: in gen_reg_rtx, at emit-rtl.c:862 Please submit a full bug report,
with preprocessed source if appropriate.

another compiler error.

Maybe those problems can be solved by using another Microblaze external toolchain, but I couldn't find any other available on the Web. Or maybe we should remove the external Microblaze toolchains and instead support Microblaze in the internal Buildroot toolchain backend?

Thanks for your help,

Thomas

For reference, the last 30 Microblaze build failures were:

+---------------------+----------------------------------------------------------------------------------+-----------------+
| 2013-10-06 05:31:03 | http://autobuild.buildroot.org/results/e570445aafab51ce497da9e0d864157f50ff75d7/ | libglib2-2.36.3 |
| 2013-10-05 22:37:36 | http://autobuild.buildroot.org/results/20b2e2573985a687be29220a7df3b11a32894ba2/ | nettle-2.7.1    |
| 2013-10-05 21:59:39 | http://autobuild.buildroot.org/results/32fca009984ca0de4b4026ff6b4029104d0396c5/ | libglib2-2.36.3 |
| 2013-10-05 19:32:44 | http://autobuild.buildroot.org/results/166ab2a8c2efffa0b518f07d7f1dcdfe8c85614f/ | nettle-2.7.1    |
| 2013-10-05 17:39:45 | http://autobuild.buildroot.org/results/4ec0f495f4ff77bb35e7b1ab8446671c8d260cd6/ | libglib2-2.36.3 |
| 2013-10-05 09:44:38 | http://autobuild.buildroot.org/results/ce1c9a2ca389c2fb1c8955c75212737c7d57a803/ | cairo-1.12.10   |
| 2013-10-05 07:21:48 | http://autobuild.buildroot.org/results/b1d6d56d2c8bcd3df47e2685ed8e7de8742a7c73/ | nettle-2.7.1    |
| 2013-10-05 01:45:57 | http://autobuild.buildroot.org/results/a050faaabbf89f9d3f324b821da84f8991a5aebb/ | openssl-1.0.1e  |
| 2013-10-04 23:20:14 | http://autobuild.buildroot.org/results/ac8992a75b97f0baab55fe6e93ed0a8eeca08bcf/ | openssl-1.0.1e  |
| 2013-10-04 18:15:46 | http://autobuild.buildroot.org/results/80a0d575d2afac933c8cb2fb596a0347c50e4912/ | libglib2-2.36.3 |
| 2013-10-04 15:36:19 | http://autobuild.buildroot.org/results/9a3027a69dac63b4e621529eff18035530796e7e/ | nettle-2.7.1    |
| 2013-10-04 12:14:21 | http://autobuild.buildroot.org/results/4611c835d904fb5728ae5531bcc41ed33e1f5de6/ | nettle-2.7.1    |
| 2013-10-04 10:34:37 | http://autobuild.buildroot.org/results/c1332dd02817e25ab8ecd0ceab8c4953b65c9f50/ | pcsc-lite-1.8.6 |
| 2013-10-04 09:39:00 | http://autobuild.buildroot.org/results/517e2754256d39864d38befeeccd54c153e28971/ | libglib2-2.36.3 |
| 2013-10-03 14:04:25 | http://autobuild.buildroot.org/results/7054d168729c4e257000db93ab5137af1e1583cc/ | cairo-1.12.10   |
| 2013-10-02 05:50:46 | http://autobuild.buildroot.org/results/0a26b484f1cc01d6c150f6f855b2b8c7b03d585b/ | libglib2-2.36.3 |
| 2013-10-02 00:53:35 | http://autobuild.buildroot.org/results/2b10358b3540f9a12fc777bc37afbfb6072d2a47/ | binutils-2.21   |
| 2013-10-01 22:25:57 | http://autobuild.buildroot.org/results/5f691a60ffed41f0f4099179a07d7fc6c0251823/ | openssl-1.0.1e  |
| 2013-10-01 19:14:35 | http://autobuild.buildroot.org/results/d9115507421d121b0c317a2d1b52ef33199e913f/ | libglib2-2.36.3 |
| 2013-10-01 11:35:25 | http://autobuild.buildroot.org/results/0ad3e345fe1a9666bc18562549621aa28987b846/ | libglib2-2.36.3 |
| 2013-10-01 06:48:37 | http://autobuild.buildroot.org/results/1a373593d5f9d55f2e7e9113ecdb4adb8e6baaa7/ | libglib2-2.36.3 |
| 2013-09-30 18:09:09 | http://autobuild.buildroot.org/results/a44af8dee49fea322824f99da31b82c828728476/ | libglib2-2.36.3 |
| 2013-09-30 11:54:37 | http://autobuild.buildroot.org/results/74284d1087fc5677787882681c7f38a8f19e4402/ | openssl-1.0.1e  |
| 2013-09-29 14:55:24 | http://autobuild.buildroot.org/results/af53648189dd9ea40ea20d3e173e6ff391d73279/ | nettle-2.7.1    |
| 2013-09-29 13:47:53 | http://autobuild.buildroot.org/results/7cb55407f7f3e156cc04fd66da284a6c4178c776/ | libglib2-2.36.3 |
| 2013-09-29 12:54:59 | http://autobuild.buildroot.org/results/5031115f6e2c8e6c32309bab3d982b9e55e69fc3/ | libglib2-2.36.3 |
| 2013-09-29 10:57:54 | http://autobuild.buildroot.org/results/f12f692a4cf24e93d71f511362704e57766e2127/ | libglib2-2.36.3 |
| 2013-09-29 07:56:39 | http://autobuild.buildroot.org/results/c486ba7eebd2bbbef69b3a68323e0329cc65f79d/ | openssl-1.0.1e  |
| 2013-09-29 07:31:02 | http://autobuild.buildroot.org/results/f0dd5afd1e49e967d60c7ec7e62a924942e9a3ea/ | libglib2-2.36.3 |
| 2013-09-29 00:39:25 | http://autobuild.buildroot.org/results/767177028f4d0b53a3b9898f293526b234394463/ | libglib2-2.36.3 |
+---------------------+----------------------------------------------------------------------------------+-----------------+
Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#2 lore raw
In reply to a message not in the archive

Hi all

Sorry for the big delay on this issue.

Now that I've spent a few hours looking at the code, I can firmly say this is a compiler bug.

I've studied beecrypt's sha384.c code (openssl and others implement exactly the same code from a common source) and I've managed to reduce the code to what I think is a minimal example of bug triggering code.

After requesting gcc to preprocess the source, the same error still happens. Almost any minimal change I've tried to the preprocessed source makes the error go away (indexes or values seem to be irrelevant, I've just mantained the numbers the code came from). An optimization flag is required to trigger the bug (-Os, -O1,2,3). I haven't been successful trying to find which specific optimization flag is the one that causes the error.

I've manually merged both (original and preprocessed files) to ease the analysis of the original source code and the resulting code. I'm sorry I can't provide any more information, I really don't have any grasp of gcc's internals.

Best regards

~buildroot/output/build/beecrypt-4.2.1 (bug-microblaze)$ cat minimal-preprocessed.c
# 1 "minimal.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "minimal.c"
typedef unsigned long long int uint64_t;
# 3 "minimal.c"
/*
#define ROTR64(x, s) (((x) >> (s)) | ((x) << (64 - (s))))

#define S(x,s) ROTR64(x, s)
#define CH(x,y,z) ((x&(y^z))^z)
#define MAJ(x,y,z) (((x|y)&z)|(x&y))
#define SIG0(x) (S(x,28) ^ S(x,34) ^ S(x,39))
#define SIG1(x) (S(x,14) ^ S(x,18) ^ S(x,41))

#define ROUND(a,b,c,d,e,f,g,h,w,k)    \
    temp = h + SIG1(e) + CH(e,f,g) + k + w;    \
    h = temp + SIG0(a) + MAJ(a,b,c);    \
    d += temp

*/
# 16 "minimal.c"
void sha384Process(uint64_t *hh)
{
uint64_t a, b, c, d, e, f, g, h, temp;
uint64_t *w;
const uint64_t *k;
unsigned char t;

/* Expansion of ROUND(f,g,h,a,b,c,d,e,w[75],k[75]); */
temp = e + ((((b) >> (14)) | ((b) << (64 - (14)))) ^ (((b) >> (18)) | ((b)
<< (64 - (18)))) ^ (((b) >> (41)) | ((b) << (64 - (41))))) + ((b&(c^d))^d)
+ k[75] + w[75];
e = temp + ((((f) >> (28)) | ((f) << (64 - (28)))) ^ (((f) >> (34)) | ((f)
<< (64 - (34)))) ^ (((f) >> (39)) | ((f) << (64 - (39))))) + (((f|g)&h)|(f&g));
a += temp;

/* Expansion of ROUND(e,f,g,h,a,b,c,d,w[76],k[76]); */
temp = d + ((((a) >> (14)) | ((a) << (64 - (14)))) ^ (((a) >> (18)) | ((a)
<< (64 - (18)))) ^ (((a) >> (41)) | ((a) << (64 - (41))))) + ((a&(b^c))^c)
+ k[76] + w[76];
d = temp + ((((e) >> (28)) | ((e) << (64 - (28)))) ^ (((e) >> (34)) | ((e)
<< (64 - (34)))) ^ (((e) >> (39)) | ((e) << (64 - (39))))) + (((e|f)&g)|(e&f));
h += temp;

/* Expansion of ROUND(d,e,f,g,h,a,b,c,w[77],k[77]); */
temp = c + ((((h) >> (14)) | ((h) << (64 - (14)))) ^ (((h) >> (18)) | ((h)
<< (64 - (18)))) ^ (((h) >> (41)) | ((h) << (64 - (41))))) + ((h&(a^b))^b)
+ k[77] + w[77];
c = temp + ((((d) >> (28)) | ((d) << (64 - (28)))) ^ (((d) >> (34)) | ((d)
<< (64 - (34)))) ^ (((d) >> (39)) | ((d) << (64 - (39))))) + (((d|e)&f)|(d&e));
g += temp;

/* Expansion of ROUND(c,d,e,f,g,h,a,b,w[78],k[78]); */
temp = b + ((((g) >> (14)) | ((g) << (64 - (14)))) ^ (((g) >> (18)) | ((g)
<< (64 - (18)))) ^ (((g) >> (41)) | ((g) << (64 - (41))))) + ((g&(h^a))^a)
+ k[78] + w[78];
b = temp + ((((c) >> (28)) | ((c) << (64 - (28)))) ^ (((c) >> (34)) | ((c)
<< (64 - (34)))) ^ (((c) >> (39)) | ((c) << (64 - (39))))) + (((c|d)&e)|(c&d));
f += temp;

hh[0] += a;
hh[1] += b;
hh[2] += c;
hh[3] += d;
hh[4] += e;
hh[5] += f;
hh[6] += g;
hh[7] += h;
}

~buildroot/output/build/beecrypt-4.2.1 (bug-microblaze)$ ../../host/usr/bin/microblazeel-unknown-linux-gnu-gcc -Os -c minimal-preprocessed.c
minimal.c: In function 'sha384Process':
minimal.c:51:1: internal compiler error: in reload,@reload1.c:1059 Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.

2013/10/6 Alvaro Gamez <[email protected]>

Quoted 166 lines

Hi Thomas, hi all!

The last three errors seem to be the same. It's apparently triggered by a loop in which several macros are called one after the other, repeatedly.

Although I don't know much (nothing, really) about gcc, I also think it's a compiler bug, so maybe the best way of action is, as you said, supporting Microblaze directly on the internal toolchain, without relaying on Xilinx's.

Looking a bit on files reload1.c and emit-rtl.c, I can only tell it's got something to do with pseudo registers, which I don't know what they are, but maybe that gives someone an idea of what the problem is.

Anyway, I'll try to replicate the issue on the next few days and will try to at least make a minimal (and preprocessed) code example, just in case we want to continue using Xilinx's toolchain and we can patch some code to workaround that or provide a more extensive bug report to Xilinx.

Regards

2013/10/6 Thomas Petazzoni <[email protected]>

Quoted 130 lines

Jan, Chris, Alvaro, Spenser,

You are amongst the four persons who contributed Microblaze related changes to Buildroot. We are currently having various build problems in our autobuilders caused by the Microblaze support, and it would be nice if you could help us fixing those issues and generally maintain the Microblaze support in a good shape in Buildroot.

At the moment, here are the latest autobuilder issues we have on Microblaze:

* libglib2 fails to build

http://autobuild.buildroot.org/results/32f/32fca009984ca0de4b4026ff6b4029104d0396c5/build-end.log

undefined reference to `__sync_fetch_and_or_4'

probably caused by the compiler not providing the right intrinsics, or libglib2 not falling back to other solutions when the intrinsics are not available.

* nettle fails to build

http://autobuild.buildroot.org/results/20b/20b2e2573985a687be29220a7df3b11a32894ba2/build-end.log

sha512-compress.c: In function '_nettle_sha512_compress':
sha512-compress.c:180:1: internal compiler error: in reload, at
reload1.c:1059
Please submit a full bug report,

compiler bug, apparently.

* openssl fails to build

http://autobuild.buildroot.org/results/a050faaabbf89f9d3f324b821da84f8991a5aebb/build-end.log

sha512.c: In function 'sha512_block_data_order':
sha512.c:585:2: internal compiler error: in reload, at reload1.c:1059 Please submit a full bug report,
with preprocessed source if appropriate.

looks pretty much the same compiler bug as the one triggered by the nettle build.

* cairo fails to build

http://autobuild.buildroot.org/results/7054d168729c4e257000db93ab5137af1e1583cc/build-end.log

dragon.c: In function 'do_dragon':
dragon.c:162:1: internal compiler error: in gen_reg_rtx, at
emit-rtl.c:862
Please submit a full bug report,
with preprocessed source if appropriate.

another compiler error.

Maybe those problems can be solved by using another Microblaze external toolchain, but I couldn't find any other available on the Web. Or maybe we should remove the external Microblaze toolchains and instead support Microblaze in the internal Buildroot toolchain backend?

Thanks for your help,

Thomas

For reference, the last 30 Microblaze build failures were:

+---------------------+----------------------------------------------------------------------------------+-----------------+
| 2013-10-06 05:31:03 |
http://autobuild.buildroot.org/results/e570445aafab51ce497da9e0d864157f50ff75d7/| libglib2-2.36.3 |
| 2013-10-05 22:37:36 |
http://autobuild.buildroot.org/results/20b2e2573985a687be29220a7df3b11a32894ba2/| nettle-2.7.1    |
| 2013-10-05 21:59:39 |
http://autobuild.buildroot.org/results/32fca009984ca0de4b4026ff6b4029104d0396c5/| libglib2-2.36.3 |
| 2013-10-05 19:32:44 |
http://autobuild.buildroot.org/results/166ab2a8c2efffa0b518f07d7f1dcdfe8c85614f/| nettle-2.7.1    |
| 2013-10-05 17:39:45 |
http://autobuild.buildroot.org/results/4ec0f495f4ff77bb35e7b1ab8446671c8d260cd6/| libglib2-2.36.3 |
| 2013-10-05 09:44:38 |
http://autobuild.buildroot.org/results/ce1c9a2ca389c2fb1c8955c75212737c7d57a803/| cairo-1.12.10   |
| 2013-10-05 07:21:48 |
http://autobuild.buildroot.org/results/b1d6d56d2c8bcd3df47e2685ed8e7de8742a7c73/| nettle-2.7.1    |
| 2013-10-05 01:45:57 |
http://autobuild.buildroot.org/results/a050faaabbf89f9d3f324b821da84f8991a5aebb/| openssl-1.0.1e  |
| 2013-10-04 23:20:14 |
http://autobuild.buildroot.org/results/ac8992a75b97f0baab55fe6e93ed0a8eeca08bcf/| openssl-1.0.1e  |
| 2013-10-04 18:15:46 |
http://autobuild.buildroot.org/results/80a0d575d2afac933c8cb2fb596a0347c50e4912/| libglib2-2.36.3 |
| 2013-10-04 15:36:19 |
http://autobuild.buildroot.org/results/9a3027a69dac63b4e621529eff18035530796e7e/| nettle-2.7.1    |
| 2013-10-04 12:14:21 |
http://autobuild.buildroot.org/results/4611c835d904fb5728ae5531bcc41ed33e1f5de6/| nettle-2.7.1    |
| 2013-10-04 10:34:37 |
http://autobuild.buildroot.org/results/c1332dd02817e25ab8ecd0ceab8c4953b65c9f50/| pcsc-lite-1.8.6 |
| 2013-10-04 09:39:00 |
http://autobuild.buildroot.org/results/517e2754256d39864d38befeeccd54c153e28971/| libglib2-2.36.3 |
| 2013-10-03 14:04:25 |
http://autobuild.buildroot.org/results/7054d168729c4e257000db93ab5137af1e1583cc/| cairo-1.12.10   |
| 2013-10-02 05:50:46 |
http://autobuild.buildroot.org/results/0a26b484f1cc01d6c150f6f855b2b8c7b03d585b/| libglib2-2.36.3 |
| 2013-10-02 00:53:35 |
http://autobuild.buildroot.org/results/2b10358b3540f9a12fc777bc37afbfb6072d2a47/| binutils-2.21   |
| 2013-10-01 22:25:57 |
http://autobuild.buildroot.org/results/5f691a60ffed41f0f4099179a07d7fc6c0251823/| openssl-1.0.1e  |
| 2013-10-01 19:14:35 |
http://autobuild.buildroot.org/results/d9115507421d121b0c317a2d1b52ef33199e913f/| libglib2-2.36.3 |
| 2013-10-01 11:35:25 |
http://autobuild.buildroot.org/results/0ad3e345fe1a9666bc18562549621aa28987b846/| libglib2-2.36.3 |
| 2013-10-01 06:48:37 |
http://autobuild.buildroot.org/results/1a373593d5f9d55f2e7e9113ecdb4adb8e6baaa7/| libglib2-2.36.3 |
| 2013-09-30 18:09:09 |
http://autobuild.buildroot.org/results/a44af8dee49fea322824f99da31b82c828728476/| libglib2-2.36.3 |
| 2013-09-30 11:54:37 |
http://autobuild.buildroot.org/results/74284d1087fc5677787882681c7f38a8f19e4402/| openssl-1.0.1e  |
| 2013-09-29 14:55:24 |
http://autobuild.buildroot.org/results/af53648189dd9ea40ea20d3e173e6ff391d73279/| nettle-2.7.1    |
| 2013-09-29 13:47:53 |
http://autobuild.buildroot.org/results/7cb55407f7f3e156cc04fd66da284a6c4178c776/| libglib2-2.36.3 |
| 2013-09-29 12:54:59 |
http://autobuild.buildroot.org/results/5031115f6e2c8e6c32309bab3d982b9e55e69fc3/| libglib2-2.36.3 |
| 2013-09-29 10:57:54 |
http://autobuild.buildroot.org/results/f12f692a4cf24e93d71f511362704e57766e2127/| libglib2-2.36.3 |
| 2013-09-29 07:56:39 |
http://autobuild.buildroot.org/results/c486ba7eebd2bbbef69b3a68323e0329cc65f79d/| openssl-1.0.1e  |
| 2013-09-29 07:31:02 |
http://autobuild.buildroot.org/results/f0dd5afd1e49e967d60c7ec7e62a924942e9a3ea/| libglib2-2.36.3 |
| 2013-09-29 00:39:25 |
http://autobuild.buildroot.org/results/767177028f4d0b53a3b9898f293526b234394463/| libglib2-2.36.3 |
Quoted 7 lines

+---------------------+----------------------------------------------------------------------------------+-----------------+

--
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

--
?lvaro G?mez Machado

Signature · 4 lines
?lvaro G?mez Machado
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/2c6a76f8/attachment-0001.html>
#3 lore raw

Dear Alvaro Gamez,

Quoted 16 lines · Alvaro Gamez

Sorry for the big delay on this issue.

Now that I've spent a few hours looking at the code, I can firmly say this is a compiler bug.

I've studied beecrypt's sha384.c code (openssl and others implement exactly the same code from a common source) and I've managed to reduce the code to what I think is a minimal example of bug triggering code.

After requesting gcc to preprocess the source, the same error still happens. Almost any minimal change I've tried to the preprocessed source makes the error go away (indexes or values seem to be irrelevant, I've just mantained the numbers the code came from). An optimization flag is required to trigger the bug (-Os, -O1,2,3). I haven't been successful trying to find which specific optimization flag is the one that causes the error.

Quoted 4 lines

I've manually merged both (original and preprocessed files) to ease the analysis of the original source code and the resulting code. I'm sorry I can't provide any more information, I really don't have any grasp of gcc's internals.

Thanks! In fact, Spenser submitted patches to add support for Microblaze in the internal toolchain backend. This is useful because the pre-built external toolchains that we have been using until now are quite old, and not maintained by Xilinx.

Therefore, the plan is, I believe:

(1) Merge Spenser's patches to add Microblaze internal toolchain support.

(2) Remove the broken Microblaze external toolchains

(3) Hopefully, convince Xilinx to also provide pre-built external toolchains that are up-to-date. This will have to be done by people having contacts within Xilinx.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#4 lore raw

Hi,

Quoted 30 lines · Thomas Petazzoni

Dear Alvaro Gamez,

Quoted 16 lines · Alvaro Gamez

Sorry for the big delay on this issue.

Now that I've spent a few hours looking at the code, I can firmly say this is a compiler bug.

I've studied beecrypt's sha384.c code (openssl and others implement exactly the same code from a common source) and I've managed to reduce the code to what I think is a minimal example of bug triggering code.

After requesting gcc to preprocess the source, the same error still happens. Almost any minimal change I've tried to the preprocessed source makes the error go away (indexes or values seem to be irrelevant, I've just mantained the numbers the code came from). An optimization flag is required to trigger the bug (-Os, -O1,2,3). I haven't been successful trying to find which specific optimization flag is the one that causes the error.

Quoted 4 lines

I've manually merged both (original and preprocessed files) to ease the analysis of the original source code and the resulting code. I'm sorry I can't provide any more information, I really don't have any grasp of gcc's internals.

Thanks! In fact, Spenser submitted patches to add support for Microblaze in the internal toolchain backend. This is useful because the pre-built external toolchains that we have been using until now are quite old, and not maintained by Xilinx.

Quoted 4 lines

Therefore, the plan is, I believe:

(1) Merge Spenser's patches to add Microblaze internal toolchain support.

It would be nice if someone could verify that the compiler errors from these autobuilds are gone when using Spenser's toolchain. Alvaro, are you interested in trying this?

Thanks,
Thomas

#5 lore raw

Hi,

2013/11/20 Thomas De Schampheleire <[email protected]>

Quoted 10 lines
Quoted 5 lines

Therefore, the plan is, I believe:

(1) Merge Spenser's patches to add Microblaze internal toolchain support.

It would be nice if someone could verify that the compiler errors from these autobuilds are gone when using Spenser's toolchain. Alvaro, are you interested in trying this?

Sure, I can try to compile this minimal example first as well as a full buildroot system. That won't take much time. I guess I will need some patches that have been sent to the list or that are available in patchwork?

Regards

Signature · 4 lines
?lvaro G?mez Machado
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/bc2d389a/attachment.html>
#6 lore raw
In reply to Thomas Petazzoni

Hi Thomas,

2013/11/20 Thomas Petazzoni <[email protected]>

Quoted 11 lines

Therefore, the plan is, I believe:

(1) Merge Spenser's patches to add Microblaze internal toolchain support.

(2) Remove the broken Microblaze external toolchains

(3) Hopefully, convince Xilinx to also provide pre-built external toolchains that are up-to-date. This will have to be done by people having contacts within Xilinx.

Now that I take the time to revisit Xilinx' website, I think maybe we're doing something wrong...

Old Xilinx git repository,
http://git.xilinx.com/?p=microblaze-gnu.git;a=tree;f=binaries now redirect to github: https://github.com/xilinx and seems to be quite up to date with upstream gcc.

There seems to be also a later version of GNU Tools available at
http://www.xilinx.com/guest_resources/gnu/

- GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2,
newlib 1.19, eglibc-2.18) -
mb_gnu_20131023.tar.gz<http://www.xilinx.com/guest_resources/member/mb_gnu/mb_gnu_20131023.tar.gz>

I don't know how up to date are these downloads, but it may be worth a try? Although if we're going ahead with Spenser's patches maybe we can/(want to?) avoid Xilinx altogether?

Signature · 4 lines
?lvaro G?mez Machado
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/21064ffe/attachment.html>
#7 lore raw

Dear Alvaro Gamez,

Quoted 7 lines · Alvaro Gamez

Now that I take the time to revisit Xilinx' website, I think maybe we're doing something wrong...

Old Xilinx git repository,
http://git.xilinx.com/?p=microblaze-gnu.git;a=tree;f=binaries now redirect to github: https://github.com/xilinx and seems to be quite up to date with upstream gcc.

These repositories are exactly the repositories that Spenser patches are using for the internal toolchain backend.

See:

http://patchwork.ozlabs.org/patch/291045/ (binutils)
http://patchwork.ozlabs.org/patch/291046/ (gcc)
http://patchwork.ozlabs.org/patch/291047/ (glibc)
http://patchwork.ozlabs.org/patch/291048/ (gdb)

Quoted 10 lines

There seems to be also a later version of GNU Tools available at
http://www.xilinx.com/guest_resources/gnu/

- GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2,
newlib 1.19, eglibc-2.18) -
mb_gnu_20131023.tar.gz<http://www.xilinx.com/guest_resources/member/mb_gnu/mb_gnu_20131023.tar.gz>

I don't know how up to date are these downloads, but it may be worth a try? Although if we're going ahead with Spenser's patches maybe we can/(want to?) avoid Xilinx altogether?

We are not avoiding Xilinx altogether at all, please look at Spenser patches.

*However* the tarballs you're pointing at at
http://www.xilinx.com/guest_resources/gnu/ look interesting for pre-built external toolchains. I'm currently downloading "GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2, newlib 1.19, eglibc-2.18)", and I'll have a look at what it is exactly.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#8 lore raw

Thomas,

Quoted 29 lines
Quoted 7 lines

Now that I take the time to revisit Xilinx' website, I think maybe we're doing something wrong...

Old Xilinx git repository,
http://git.xilinx.com/?p=microblaze-gnu.git;a=tree;f=binaries now redirect to github: https://github.com/xilinx and seems to be quite up to date with upstream gcc.

These repositories are exactly the repositories that Spenser patches are using for the internal toolchain backend.

See:

http://patchwork.ozlabs.org/patch/291045/ (binutils)
http://patchwork.ozlabs.org/patch/291046/ (gcc)
http://patchwork.ozlabs.org/patch/291047/ (glibc)
http://patchwork.ozlabs.org/patch/291048/ (gdb)

Quoted 10 lines

There seems to be also a later version of GNU Tools available at
http://www.xilinx.com/guest_resources/gnu/

- GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2,
newlib 1.19, eglibc-2.18) -
mb_gnu_20131023.tar.gz<http://www.xilinx.com/guest_resources/member/mb_gnu/mb_gnu_20131023.tar.gz>

I don't know how up to date are these downloads, but it may be worth a try? Although if we're going ahead with Spenser's patches maybe we can/(want to?) avoid Xilinx altogether?

Quoted 8 lines

We are not avoiding Xilinx altogether at all, please look at Spenser patches.

*However* the tarballs you're pointing at at
http://www.xilinx.com/guest_resources/gnu/ look interesting for pre-built external toolchains. I'm currently downloading "GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2, newlib 1.19, eglibc-2.18)", and I'll have a look at what it is exactly.

I just downloaded the mb_gnu_20131023.tar.gz to see what it is all about. These are just the sources for the toolchain not a binary external toolchain. It may be useful to match these versions in the internal toolchain.

Thanks,
Spenser

Signature · 3 lines
Spenser Gilliland
Computer Engineer
Doctoral Candidate
#9 lore raw
In reply to Thomas Petazzoni

Hi again

Well, I have some good news and some bad news.

I've applied Spenser's patches and after some fixes I've managed to test the same build that caused beecrypt (and openssl and others) to break.

The good news is that beecrypt has compiled succesfully, so this seems to be definitely the way to go to support latest Xilinx devices and such.

The bad news is that Spenser's patches require some work. Most of it is easy and I've already done it myself, I can resubmit these patches with the little changes I made to have it fully working.

However, there's a detail that affects other buildroot parts.

gcc.mk file requires the source code to be downloaded as a .tar.bz2 file.

However, the new github helper is only able to download .tar.gz files. I don't think this is due to the new github helper but to github itself, that only provides gziped versions of its files.

gcc.mk defines HOST_GCC_EXTRACT_CMDS as:

        $(BZCAT) $(DL_DIR)/$(GCC_SOURCE) | \
                $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) \
                --exclude='libjava/*' \
                --exclude='libgo/*' \
                --exclude='gcc/testsuite/*' \
                --exclude='libstdc++-v3/testsuite/*' \
                $(TAR_OPTIONS) -
        mkdir -p $(@D)/libstdc++-v3/testsuite/
        echo "all:" > $(@D)/libstdc++-v3/testsuite/Makefile.in
        echo "install:" >> $(@D)/libstdc++-v3/testsuite/Makefile.in

So... now I have the question:

Is it necessary to manually invoke bzcat? Why don't we let tar deal with the matter of deciding which uncompresser to use, since it now by defaults can detect it without the need to pass any 'z' or 'j' parameter?

2013/11/20 Thomas Petazzoni <[email protected]>

Quoted 52 lines

Dear Alvaro Gamez,

Quoted 7 lines · Alvaro Gamez

Now that I take the time to revisit Xilinx' website, I think maybe we're doing something wrong...

Old Xilinx git repository,
http://git.xilinx.com/?p=microblaze-gnu.git;a=tree;f=binaries now redirect to github: https://github.com/xilinx and seems to be quite up to date with upstream gcc.

These repositories are exactly the repositories that Spenser patches are using for the internal toolchain backend.

See:

http://patchwork.ozlabs.org/patch/291045/ (binutils)
http://patchwork.ozlabs.org/patch/291046/ (gcc)
http://patchwork.ozlabs.org/patch/291047/ (glibc)
http://patchwork.ozlabs.org/patch/291048/ (gdb)

Quoted 6 lines

There seems to be also a later version of GNU Tools available at
http://www.xilinx.com/guest_resources/gnu/

- GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2, newlib 1.19, eglibc-2.18) -
mb_gnu_20131023.tar.gz<

http://www.xilinx.com/guest_resources/member/mb_gnu/mb_gnu_20131023.tar.gz

Quoted 5 lines

I don't know how up to date are these downloads, but it may be worth a try? Although if we're going ahead with Spenser's patches maybe we can/(want to?) avoid Xilinx altogether?

We are not avoiding Xilinx altogether at all, please look at Spenser patches.

*However* the tarballs you're pointing at at
http://www.xilinx.com/guest_resources/gnu/ look interesting for pre-built external toolchains. I'm currently downloading "GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2, newlib 1.19, eglibc-2.18)", and I'll have a look at what it is exactly.

Thanks!

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Signature · 4 lines
?lvaro G?mez Machado
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/b2034d2c/attachment.html>
#10 lore raw

Alvaro,

Quoted 6 lines · Alvaro Gamez

Hi again

Well, I have some good news and some bad news.

I've applied Spenser's patches and after some fixes I've managed to test

the same build that caused beecrypt (and openssl and others) to break.

Quoted 2 lines

The good news is that beecrypt has compiled succesfully, so this seems to

be definitely the way to go to support latest Xilinx devices and such.

Quoted 2 lines

The bad news is that Spenser's patches require some work. Most of it is

easy and I've already done it myself, I can resubmit these patches with the little changes I made to have it fully working.

Quoted 6 lines

However, there's a detail that affects other buildroot parts.

gcc.mk file requires the source code to be downloaded as a .tar.bz2 file.

However, the new github helper is only able to download .tar.gz files. I

don't think this is due to the new github helper but to github itself, that only provides gziped versions of its files.

Quoted 14 lines

gcc.mk defines HOST_GCC_EXTRACT_CMDS as:

        $(BZCAT) $(DL_DIR)/$(GCC_SOURCE) | \
                $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) \
                --exclude='libjava/*' \
                --exclude='libgo/*' \
                --exclude='gcc/testsuite/*' \
                --exclude='libstdc++-v3/testsuite/*' \
                $(TAR_OPTIONS) -
        mkdir -p $(@D)/libstdc++-v3/testsuite/
        echo "all:" > $(@D)/libstdc++-v3/testsuite/Makefile.in
        echo "install:" >> $(@D)/libstdc++-v3/testsuite/Makefile.in
Quoted 3 lines

So... now I have the question:

Is it necessary to manually invoke bzcat? Why don't we let tar deal with

the matter of deciding which uncompresser to use, since it now by defaults can detect it without the need to pass any 'z' or 'j' parameter?

Quoted 33 lines

2013/11/20 Thomas Petazzoni <[email protected]>

Quoted 29 lines

Dear Alvaro Gamez,

Quoted 7 lines · Alvaro Gamez

Now that I take the time to revisit Xilinx' website, I think maybe we're doing something wrong...

Old Xilinx git repository,
http://git.xilinx.com/?p=microblaze-gnu.git;a=tree;f=binaries now redirect to github: https://github.com/xilinx and seems to be quite up to date with upstream gcc.

These repositories are exactly the repositories that Spenser patches are using for the internal toolchain backend.

See:

http://patchwork.ozlabs.org/patch/291045/ (binutils)
http://patchwork.ozlabs.org/patch/291046/ (gcc)
http://patchwork.ozlabs.org/patch/291047/ (glibc)
http://patchwork.ozlabs.org/patch/291048/ (gdb)

Quoted 6 lines

There seems to be also a later version of GNU Tools available at
http://www.xilinx.com/guest_resources/gnu/

- GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2, newlib 1.19, eglibc-2.18) -
mb_gnu_20131023.tar.gz<

http://www.xilinx.com/guest_resources/member/mb_gnu/mb_gnu_20131023.tar.gz>

Quoted 25 lines
Quoted 14 lines
Quoted 4 lines

I don't know how up to date are these downloads, but it may be worth a try? Although if we're going ahead with Spenser's patches maybe we can/(want to?) avoid Xilinx altogether?

We are not avoiding Xilinx altogether at all, please look at Spenser patches.

*However* the tarballs you're pointing at at
http://www.xilinx.com/guest_resources/gnu/ look interesting for pre-built external toolchains. I'm currently downloading "GNU Tools for MicroBlaze (gcc 4.6.4, gdb 7.6.0, binutils 2.23.2, newlib 1.19, eglibc-2.18)", and I'll have a look at what it is exactly.

Quoted 7 lines

Thanks!

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Quoted 2 lines

--
?lvaro G?mez Machado

Alvaro,

You need to apply the arc patch set as well as this set. We harmonized the patches to make sure there could both be applied cleanly.

Thanks,
Spenser
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/63a3b84c/attachment.html>

#11 lore raw
In reply to Alvaro Gamez

Dear Alvaro Gamez,

Quoted 3 lines · Alvaro Gamez

The bad news is that Spenser's patches require some work. Most of it is easy and I've already done it myself, I can resubmit these patches with the little changes I made to have it fully working.

Did you realize that Spenser patches apply on top of the patches from Mischa Jonker?

Quoted 22 lines

However, there's a detail that affects other buildroot parts.

gcc.mk file requires the source code to be downloaded as a .tar.bz2
file.

However, the new github helper is only able to download .tar.gz files. I don't think this is due to the new github helper but to github itself, that only provides gziped versions of its files.

gcc.mk defines HOST_GCC_EXTRACT_CMDS as:

        $(BZCAT) $(DL_DIR)/$(GCC_SOURCE) | \
                $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) \
                --exclude='libjava/*' \
                --exclude='libgo/*' \
                --exclude='gcc/testsuite/*' \
                --exclude='libstdc++-v3/testsuite/*' \
                $(TAR_OPTIONS) -
        mkdir -p $(@D)/libstdc++-v3/testsuite/
        echo "all:" > $(@D)/libstdc++-v3/testsuite/Makefile.in
        echo "install:" >> $(@D)/libstdc++-v3/testsuite/Makefile.in
Quoted 6 lines

So... now I have the question:

Is it necessary to manually invoke bzcat? Why don't we let tar deal with the matter of deciding which uncompresser to use, since it now by defaults can detect it without the need to pass any 'z' or 'j' parameter?

For example, this specific problem is solved by Mischa Jonker patches. See http://patchwork.ozlabs.org/patch/290981/.

See also the cover letter from Spenser at
http://lists.busybox.net/pipermail/buildroot/2013-November/082590.html,
which explains what are the dependencies of his patch series.

Which other problems have you seen?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#12 lore raw

Hi, Spenser, Thomas

Sorry, no, I didn't read much and run wildly and foolishly to apply as few patches as possible, for I've been very disconnected from buildroot development for months, so I wasn't really following the dependencies between patches. In retrospect, that was unwise.

For what it's worth, if by simply renaming and recompressing some files I've managed to build a microblaze system with only Spenser's patches applied to master... that seems to mean that everything is working really great.

Thank you!

2013/11/20 Thomas Petazzoni <[email protected]>

Quoted 55 lines

Dear Alvaro Gamez,

Quoted 3 lines · Alvaro Gamez

The bad news is that Spenser's patches require some work. Most of it is easy and I've already done it myself, I can resubmit these patches with the little changes I made to have it fully working.

Did you realize that Spenser patches apply on top of the patches from Mischa Jonker?

Quoted 22 lines

However, there's a detail that affects other buildroot parts.

gcc.mk file requires the source code to be downloaded as a .tar.bz2
file.

However, the new github helper is only able to download .tar.gz files. I don't think this is due to the new github helper but to github itself, that only provides gziped versions of its files.

gcc.mk defines HOST_GCC_EXTRACT_CMDS as:

        $(BZCAT) $(DL_DIR)/$(GCC_SOURCE) | \
                $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $(@D) \
                --exclude='libjava/*' \
                --exclude='libgo/*' \
                --exclude='gcc/testsuite/*' \
                --exclude='libstdc++-v3/testsuite/*' \
                $(TAR_OPTIONS) -
        mkdir -p $(@D)/libstdc++-v3/testsuite/
        echo "all:" > $(@D)/libstdc++-v3/testsuite/Makefile.in
        echo "install:" >> $(@D)/libstdc++-v3/testsuite/Makefile.in
Quoted 6 lines

So... now I have the question:

Is it necessary to manually invoke bzcat? Why don't we let tar deal with the matter of deciding which uncompresser to use, since it now by defaults can detect it without the need to pass any 'z' or 'j' parameter?

For example, this specific problem is solved by Mischa Jonker patches. See http://patchwork.ozlabs.org/patch/290981/.

See also the cover letter from Spenser at
http://lists.busybox.net/pipermail/buildroot/2013-November/082590.html,
which explains what are the dependencies of his patch series.

Which other problems have you seen?

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Signature · 4 lines
?lvaro G?mez Machado
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131120/8b954276/attachment.html>
#13 lore raw

Dear Alvaro Gamez,

Quoted 9 lines · Alvaro Gamez

Sorry, no, I didn't read much and run wildly and foolishly to apply as few patches as possible, for I've been very disconnected from buildroot development for months, so I wasn't really following the dependencies between patches. In retrospect, that was unwise.

For what it's worth, if by simply renaming and recompressing some files I've managed to build a microblaze system with only Spenser's patches applied to master... that seems to mean that everything is working really great.

Great, thanks for the testing!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com