Preparatory work on toolchains - v3

Hello All!

These are preparatory work on toolchain, laying ground to later add crosstool-NG as a backend. Currently, these are rather an RFC, even if it works OK for me.

Changes since v2:
- rebased againts latest tree
- enhance commit message for 3rd patch

Changes since v1:
- rebased against latest tree
- patches formatted to detect renames/copies

Yann E. MORIN (5):
toolchain: move helper functions from external toolchain
toolchain: rename external toolchain dir
toolchain: move makefile includes
toolchain: move buildroot config files
toolchain: rename base target for external toolchains

 Makefile                                           |   17 +-
 toolchain/Config.in                                |   16 +-
 toolchain/external-toolchain/ext-tool.mk           |  356 --------------------
 toolchain/helpers.mk                               |  244 ++++++++++++++
 toolchain/toolchain-buildroot.mk                   |   12 +
 .../{Config.in.1 => toolchain-buildroot/Config.in} |    2 +-
 toolchain/toolchain-buildroot/Config.in.2          |   37 ++
 toolchain/{Config.in.2 => toolchain-common.in}     |   46 +--
 toolchain/toolchain-external.mk                    |   14 +
 .../Config.in                                      |    0
 .../Config.in.2                                    |    0
 toolchain/toolchain-external/ext-tool.mk           |  115 +++++++
 12 files changed, 440 insertions(+), 419 deletions(-)
 delete mode 100644 toolchain/external-toolchain/ext-tool.mk
 create mode 100644 toolchain/helpers.mk
 create mode 100644 toolchain/toolchain-buildroot.mk
 rename toolchain/{Config.in.1 => toolchain-buildroot/Config.in} (82%)
 create mode 100644 toolchain/toolchain-buildroot/Config.in.2
 rename toolchain/{Config.in.2 => toolchain-common.in} (84%)
 create mode 100644 toolchain/toolchain-external.mk
 rename toolchain/{external-toolchain => toolchain-external}/Config.in (100%)
 rename toolchain/{external-toolchain => toolchain-external}/Config.in.2 (100%)
 create mode 100644 toolchain/toolchain-external/ext-tool.mk

The helper functions used for external toolchains may also be useful to alternate toolchain backends (currently, the external toolchain is the sole user).

---
 Makefile                                 |    1 +
 toolchain/external-toolchain/ext-tool.mk |  241 -----------------------------
 toolchain/helpers.mk                     |  244 ++++++++++++++++++++++++++++++
 3 files changed, 245 insertions(+), 241 deletions(-)
 create mode 100644 toolchain/helpers.mk

index d6bcb36..e02b7db 100644
@@ -306,6 +306,7 @@ include toolchain/mklibs/mklibs.mk
 include toolchain/sstrip/sstrip.mk
 include toolchain/uClibc/uclibc.mk
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
+include toolchain/helpers.mk
 include toolchain/*/*.mk
 endif
 

1.7.1

Rename the external toolchain directory.
When new backends are here, it will be easier to sort them out if they are all prefixed the same way.

---
 toolchain/Config.in                                |    4 ++--
 .../Config.in                                      |    0
 .../Config.in.2                                    |    0
 .../ext-tool.mk                                    |    0
 4 files changed, 2 insertions(+), 2 deletions(-)
 rename toolchain/{external-toolchain => toolchain-external}/Config.in (100%)
 rename toolchain/{external-toolchain => toolchain-external}/Config.in.2 (100%)
 rename toolchain/{external-toolchain => toolchain-external}/ext-tool.mk (100%)

index ee94143..52b9d8b 100644
@@ -17,7 +17,7 @@ config BR2_TOOLCHAIN_EXTERNAL
 endchoice
 
 source "toolchain/Config.in.1"
-source "toolchain/external-toolchain/Config.in"
+source "toolchain/toolchain-external/Config.in"
 
 # we want gdb config in the middle of both source and external
 # toolchains, but mconf won't let us source the same file twice,

@@ -26,6 +26,6 @@ source "toolchain/gdb/Config.in"
 comment "Common Toolchain Options"
 
 source "toolchain/Config.in.2"
-source "toolchain/external-toolchain/Config.in.2"
+source "toolchain/toolchain-external/Config.in.2"
 
 endmenu

similarity index 100%
rename from toolchain/external-toolchain/Config.in
rename to toolchain/toolchain-external/Config.in

similarity index 100%
rename from toolchain/external-toolchain/Config.in.2
rename to toolchain/toolchain-external/Config.in.2

similarity index 100%
rename from toolchain/external-toolchain/ext-tool.mk
rename to toolchain/toolchain-external/ext-tool.mk

1.7.1

Including a bunch of Makefiles with wildcard makes it impossible to add new toolchain backends. Avoid that by namely including needed files.

The external toolchain still needs to include all the toolchain/*/*.mk sub-makefiles, as they are needed to build a toolchain that runs on the target. It is to be noted that the cross-toolchain is not built in this case, as the make-targets to build the cross-toolchain are not present in the $(BASE_TARGETS) variable, which is later used to create the dependency rules.

Also, the comment 'Explicit ordering' has been removed, as it is mis- leading. It is make's responsibility to create the proper ordering based on the dependency rules it finds in the Makefiles

---
 Makefile                         |   16 ++--------------
 toolchain/toolchain-buildroot.mk |   12 ++++++++++++
 toolchain/toolchain-external.mk  |   14 ++++++++++++++
 3 files changed, 28 insertions(+), 14 deletions(-)
 create mode 100644 toolchain/toolchain-buildroot.mk
 create mode 100644 toolchain/toolchain-external.mk

index e02b7db..7f6da17 100644
@@ -293,21 +293,9 @@ all: world
 # each selected package to TARGETS if that package was selected
 # in the .config file.
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
-# avoid pulling in external toolchain which is broken for toplvl parallel builds
-# Explicit ordering:
-include toolchain/dependencies/dependencies.mk
-include toolchain/binutils/binutils.mk
-include toolchain/ccache/ccache.mk
-include toolchain/elf2flt/elf2flt.mk
-include toolchain/gcc/gcc-uclibc-4.x.mk
-include toolchain/gdb/gdb.mk
-include toolchain/kernel-headers/kernel-headers.mk
-include toolchain/mklibs/mklibs.mk
-include toolchain/sstrip/sstrip.mk
-include toolchain/uClibc/uclibc.mk
+include toolchain/toolchain-buildroot.mk
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
-include toolchain/helpers.mk
-include toolchain/*/*.mk
+include toolchain/toolchain-external.mk
 endif
 
 include package/*/*.mk

new file mode 100644
index 0000000..d879697
@@ -0,0 +1,12 @@
+# Include files required for the internal toolchain backend
+
+include toolchain/dependencies/dependencies.mk
+include toolchain/binutils/binutils.mk
+include toolchain/ccache/ccache.mk
+include toolchain/elf2flt/elf2flt.mk
+include toolchain/gcc/gcc-uclibc-4.x.mk
+include toolchain/gdb/gdb.mk
+include toolchain/kernel-headers/kernel-headers.mk
+include toolchain/mklibs/mklibs.mk
+include toolchain/sstrip/sstrip.mk
+include toolchain/uClibc/uclibc.mk

new file mode 100644
index 0000000..6f1f641
@@ -0,0 +1,14 @@
+# Required includes for the external toolchain backend
+
+include toolchain/helpers.mk
+include toolchain/binutils/binutils.mk
+include toolchain/ccache/ccache.mk
+include toolchain/dependencies/dependencies.mk
+include toolchain/elf2flt/elf2flt.mk
+include toolchain/gcc/gcc-uclibc-4.x.mk
+include toolchain/gdb/gdb.mk
+include toolchain/kernel-headers/kernel-headers.mk
+include toolchain/mklibs/mklibs.mk
+include toolchain/sstrip/sstrip.mk
+include toolchain/toolchain-external/ext-tool.mk
+include toolchain/uClibc/uclibc.mk

1.7.1

Handle the internal toolchain backend mechanism the same way we handle other backends.

---
 toolchain/Config.in                                |   12 +----
 .../{Config.in.1 => toolchain-buildroot/Config.in} |    2 +-
 toolchain/toolchain-buildroot/Config.in.2          |   37 ++++++++++++++++
 toolchain/{Config.in.2 => toolchain-common.in}     |   46 ++++----------------
 4 files changed, 50 insertions(+), 47 deletions(-)
 rename toolchain/{Config.in.1 => toolchain-buildroot/Config.in} (82%)
 create mode 100644 toolchain/toolchain-buildroot/Config.in.2
 rename toolchain/{Config.in.2 => toolchain-common.in} (84%)

index 52b9d8b..fc9a048 100644
@@ -16,16 +16,10 @@ config BR2_TOOLCHAIN_EXTERNAL
 
 endchoice
 
-source "toolchain/Config.in.1"
+source "toolchain/toolchain-buildroot/Config.in"
 source "toolchain/toolchain-external/Config.in"
-
-# we want gdb config in the middle of both source and external
-# toolchains, but mconf won't let us source the same file twice,
-# so put it here instead
-source "toolchain/gdb/Config.in"
-comment "Common Toolchain Options"
-
-source "toolchain/Config.in.2"
+source "toolchain/toolchain-common.in"
+source "toolchain/toolchain-buildroot/Config.in.2"
 source "toolchain/toolchain-external/Config.in.2"
 
 endmenu

similarity index 82%
rename from toolchain/Config.in.1
rename to toolchain/toolchain-buildroot/Config.in
index 5c26858..a9dd192 100644
@@ -1,4 +1,4 @@
-#
+# Config entries for internal toolchain backend
 
 if BR2_TOOLCHAIN_BUILDROOT
 source "toolchain/kernel-headers/Config.in"

new file mode 100644
index 0000000..512a608
@@ -0,0 +1,37 @@
+# Buildroot backend specific options
+
+if BR2_TOOLCHAIN_BUILDROOT
+source "toolchain/elf2flt/Config.in"
+source "toolchain/mklibs/Config.in"
+source "toolchain/sstrip/Config.in"
+
+config BR2_ENABLE_MULTILIB
+	bool "Enable multilib support?"
+	help
+	    Build libraries to support different ABIs.
+
+config BR2_VFP_FLOAT
+	bool "Use ARM Vector Floating Point unit"
+	depends on !BR2_SOFT_FLOAT
+	depends on BR2_arm || BR2_armeb
+	help
+	  Setting this option will enable the "-mfpu=vfp" option.
+	  If your ARM CPU has a Vector Floating Point Unit (VFP)
+	  and the toolchain supports the option, then the
+	  code can be optimized.
+
+	  Most people will answer N.
+
+config BR2_CROSS_TOOLCHAIN_TARGET_UTILS
+	bool "Include target utils in cross toolchain"
+	default y
+	help
+	  When using buildroot to build a deployable cross toolchain,
+	  it is handy to include certain target apps with that toolchain
+	  as a convenience.
+	  Examples include ldd, gdbserver, and strace.
+
+	  Answer Y if you want these apps (if built) copied into the
+	  cross toolchain dir under <arch>-linux-uclibc/target_utils/.
+
+endif

1.7.1

It is misleeading to name the external toolchain make target 'uclibc', as external toolchains can be based on other C libraries (eg. glibc or eglibc).

---
 Makefile                                 |    2 +-
 toolchain/toolchain-external/ext-tool.mk |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

index 7f6da17..d3ee730 100644
@@ -230,7 +230,7 @@ PREFERRED_LIB_FLAGS:=--enable-static --enable-shared
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
 BASE_TARGETS:=uclibc-configured binutils cross_compiler uclibc-target-utils kernel-headers
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
-BASE_TARGETS:=uclibc
+BASE_TARGETS:=toolchain-prepare
 endif
 TARGETS:=
 

index 6251582..5f6714a 100644
@@ -37,7 +37,7 @@
 #  cross-compiler binaries remains external, all libraries and headers
 #  are imported into the Buildroot tree.
 
-uclibc: dependencies $(STAMP_DIR)/ext-toolchain-installed
+toolchain-prepare: dependencies $(STAMP_DIR)/ext-toolchain-installed
 
 LIB_EXTERNAL_LIBS=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)

1.7.1

I understand your point about uclibc not being a suitable generic name, but I'm not sure toolchain-prepare is the ideal name. To me, what this target does is guarantees the presence of the C run time library (whether that be uClibc, glibc, etc). Might "c-rtl" or similar be a better name? (Of course, in order to generate the RTL, the toolchain will need to be there, as usual).

I can also imagine a scenario where one is building such a minimal system that one needs the toolchain, but not the RTL. What are good target names to guarantee the toolchain, and the RTL, as separate targets?

Mitch.

Mitch, All,

OK, I agree that we can drop this patch for now, and do the rename later. "c-rtl" is not that descriptive either... Even in-contect, it took me a while to see what it meant...

Maybe smthg like:
fill-me-that-fscking-staging-and-target-with-the-fscking-runtime-from-toolchain-and-dont-forget-headers-and-statics-for-staging-please-oh-and-build-the-toolchain-if-needed-while-youre-at-it-if-you-dont-care-you-are-such-a-lovely-pet

Hehe! Yes, I like that one! :-)

[ Thomas, I bet you, this time! :) ]

Well, it will require some thinking...

Regards,
Yann E. MORIN.

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

This is supposed to be computer science, acronyms only may be applied -

How about:
My-Software, Vital-C, Run-Time. Direct, Digital Loading libraries? or which would be MSVCRT.DLL as an acronym?

Something short, sweet and descriptive; such as found in en_OpenSSL:
EXP-EDH-RSA-DES-CBC-SHA
Which, believe it or not, is the plain text version of an acronym used. Tack a: BR2- on the front of something like that and call it decision made. ;-)

Mike

Got my vote :-)

Seriously, how about just "c"? The idea being that once you've made that target, you can build things written in C. After all, we already have packages with dependencies of "php", "python", "lua" and "microperl"...

Mitch.