Skip to contentSkip to conversation

buildman with distcc

U-Boot York Sun 2 participants 12 messages Nov 13, 2014 – Nov 20, 2014

lore
Share & tools
lore

York Sun

author
#0 lore raw

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

York

author
#1 lore raw
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

York

#2 lore raw

Hi York,

Quoted 11 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

Regards,
Simon

author
#3 lore raw
Quoted 12 lines · Simon Glass

Hi York,

Quoted 9 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

Quoted 6 lines
Quoted 2 lines

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

buildman --list-tool-chains cannot detect the toolchain if doing so.

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

York

#4 lore raw

Hi York,

Quoted 13 lines · York Sun
Quoted 12 lines · Simon Glass

Hi York,

Quoted 9 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command line. But that should not affect operation.

Quoted 3 lines

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

Indeed. I wonder why that doesn't work with buildman?

Regards,
Simon

author
#5 lore raw
Quoted 19 lines · Simon Glass

Hi York,

Quoted 16 lines · York Sun
Quoted 15 lines · Simon Glass

Hi York,

Quoted 11 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command line. But that should not affect operation.

The path in ./.buildman is partial. buildman appends other stuff to make a complete path. I need to put double quote for "distcc <my path prefix>". I don't see how that could work.

Quoted 6 lines
Quoted 4 lines

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

Indeed. I wonder why that doesn't work with buildman?

Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a checkout.

York

#6 lore raw

Hi York,

Quoted 24 lines · York Sun
Quoted 23 lines · Simon Glass

Hi York,

Quoted 20 lines · York Sun
Quoted 18 lines · Simon Glass

Hi York,

Quoted 11 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command line. But that should not affect operation.

The path in ./.buildman is partial. buildman appends other stuff to make a complete path. I need to put double quote for "distcc <my path prefix>". I don't see how that could work.

It doesn't have to - if you give a complete path then it will use it. You might need to change things in toolchain.py to make it work. See MakeEnvironment() which sets up the environment for Make - CROSS_COMPILE, PATH.

Yes, one problem is that buildman uses objsize to find out the code/data size. How can that work with distcc?

Quoted 10 lines
Quoted 7 lines
Quoted 5 lines

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

Indeed. I wonder why that doesn't work with buildman?

Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a checkout.

Well if you are using that feature out certainly can't use MAKEALL since it doesn't support it. If you leave off the branch (-b) buildman will build the current source and not check anything out.

Regards,
Simon

author
#7 lore raw
Quoted 36 lines · Simon Glass

Hi York,

Quoted 24 lines · York Sun
Quoted 23 lines · Simon Glass

Hi York,

Quoted 20 lines · York Sun
Quoted 18 lines · Simon Glass

Hi York,

Quoted 11 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command line. But that should not affect operation.

The path in ./.buildman is partial. buildman appends other stuff to make a complete path. I need to put double quote for "distcc <my path prefix>". I don't see how that could work.

Quoted 7 lines

It doesn't have to - if you give a complete path then it will use it. You might need to change things in toolchain.py to make it work. See MakeEnvironment() which sets up the environment for Make - CROSS_COMPILE, PATH.

Yes, one problem is that buildman uses objsize to find out the code/data size. How can that work with distcc?

Probably it won't.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

Indeed. I wonder why that doesn't work with buildman?

Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a checkout.

Well if you are using that feature out certainly can't use MAKEALL since it doesn't support it. If you leave off the branch (-b) buildman will build the current source and not check anything out.

I like to use buildman to build all the commits.

Can I force CC with buildman?

York

#8 lore raw

Hi York,

Quoted 42 lines · York Sun
Quoted 41 lines · Simon Glass

Hi York,

Quoted 24 lines · York Sun
Quoted 23 lines · Simon Glass

Hi York,

Quoted 20 lines · York Sun
Quoted 18 lines · Simon Glass

Hi York,

Quoted 11 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command line. But that should not affect operation.

The path in ./.buildman is partial. buildman appends other stuff to make a complete path. I need to put double quote for "distcc <my path prefix>". I don't see how that could work.

It doesn't have to - if you give a complete path then it will use it. You might need to change things in toolchain.py to make it work. See MakeEnvironment() which sets up the environment for Make - CROSS_COMPILE, PATH.

Quoted 4 lines
Quoted 2 lines

Yes, one problem is that buildman uses objsize to find out the code/data size. How can that work with distcc?

Probably it won't.

I wonder how the build copes with things like 'objcopy' then, which are in the Makefile.

Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
Quoted 7 lines

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

Indeed. I wonder why that doesn't work with buildman?

Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a checkout.

Quoted 8 lines
Quoted 4 lines

Well if you are using that feature out certainly can't use MAKEALL since it doesn't support it. If you leave off the branch (-b) buildman will build the current source and not check anything out.

I like to use buildman to build all the commits.

Can I force CC with buildman?

If you add that feature you can. But maybe you just need to force CROSS_COMPILE?

Regards,
Simon

author
#9 lore raw
Quoted 72 lines · Simon Glass

Hi York,

Quoted 42 lines · York Sun
Quoted 41 lines · Simon Glass

Hi York,

Quoted 24 lines · York Sun
Quoted 23 lines · Simon Glass

Hi York,

Quoted 20 lines · York Sun
Quoted 18 lines · Simon Glass

Hi York,

Quoted 11 lines · York Sun
Quoted 6 lines · York Sun

Simon,

Is it possible to use buildman with distcc? I am trying to speed up compiling. Using MAKEALL or make I can specify CROSS_COMPILE="distcc <path to toolchain". I don't know how to do this with buildman.

I meant to use buildman with multiple architectures, like 'arm & freescale' powerpc aarch64 together.

I wonder what would happen if you put it in the '[toolchain]' section of ~/.buildman ?

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

buildman --list-tool-chains cannot detect the toolchain if doing so.

No, it works by looking for filenames rather than running the command line. But that should not affect operation.

The path in ./.buildman is partial. buildman appends other stuff to make a complete path. I need to put double quote for "distcc <my path prefix>". I don't see how that could work.

It doesn't have to - if you give a complete path then it will use it. You might need to change things in toolchain.py to make it work. See MakeEnvironment() which sets up the environment for Make - CROSS_COMPILE, PATH.

Quoted 4 lines
Quoted 2 lines

Yes, one problem is that buildman uses objsize to find out the code/data size. How can that work with distcc?

Probably it won't.

I wonder how the build copes with things like 'objcopy' then, which are in the Makefile.

Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

If not using buildman, I can modify Makefile to prefix distcc to CC and keep using MAKEALL.

Indeed. I wonder why that doesn't work with buildman?

Doesn't buildman checkout each commit? Modifying the Makefile doesn't survive a checkout.

Well if you are using that feature out certainly can't use MAKEALL since it doesn't support it. If you leave off the branch (-b) buildman will build the current source and not check anything out.

Quoted 5 lines
Quoted 3 lines

I like to use buildman to build all the commits.

Can I force CC with buildman?

If you add that feature you can. But maybe you just need to force CROSS_COMPILE?

I can change the env['CROSS_COMPILE'] in toolchain.py and make distcc work. But I am seeing odd distribution. Comparing with MAKEALL, with single thread and 50 jobs, the distribution is mostly to localhost. Not sure how buildman calls make. Can you point me to the right direction?

York

author
#10 lore raw
Quoted 9 lines · York Sun
Quoted 2 lines · Simon Glass

If you add that feature you can. But maybe you just need to force CROSS_COMPILE?

I can change the env['CROSS_COMPILE'] in toolchain.py and make distcc work. But I am seeing odd distribution. Comparing with MAKEALL, with single thread and 50 jobs, the distribution is mostly to localhost. Not sure how buildman calls make. Can you point me to the right direction?

Nah, don't bother. I tried a build. It doesn't speed up enough to pursue this way, even with 6 servers. I may pick up in the future if I have to do this way.

York

#11 lore raw

Hi York,

Quoted 5 lines · York Sun
Quoted 4 lines · York Sun
Quoted 2 lines · Simon Glass

If you add that feature you can. But maybe you just need to force CROSS_COMPILE?

Quoted 8 lines
Quoted 5 lines

I can change the env['CROSS_COMPILE'] in toolchain.py and make distcc work. But I am seeing odd distribution. Comparing with MAKEALL, with single thread and 50 jobs, the distribution is mostly to localhost. Not sure how buildman calls make. Can you point me to the right direction?

Nah, don't bother. I tried a build. It doesn't speed up enough to pursue this way, even with 6 servers. I may pick up in the future if I have to do this way.

OK.

Regards,
Simon