Skip to contentSkip to conversation

Toolchain updates: package infra, musl support, Linaro updates, glibc fixes

PATCH New Thomas Petazzoni 5 participants 58 messages Oct 8, 2013 – Oct 11, 2013 Kbuild

lore
Share & tools
lore
#0 lore raw

v3 cover Toolchain updates: package infra, musl support, Linaro updates, glibc fixes

Hello,

Here is a new version of the toolchain fixes and updates. Careful review of patches 19 and 20 would be appreciated.

Peter, the other patches (before 19 and 20) can be merged even if there is still ongoing discussion on patches 19 and 20.

Changes since v2:

* Different order for the patches: the intrusive modification of the package infrastructure to support package-based 'make external-deps' is pushed at the end of the series, since it's most likely to be the commit causing the biggest amount of discussion/testing.

* Two fixes for the internal glibc build logic have been added.

* The helpers.mk is no longer moved into toolchain/toolchain-external/. Even though moving it in this location is not causing any problem, it is not very logical, since it's also used by the package/glibc/glibc.mk file, part of the internal toolchain backend.

* Some minor cleanup of the helpers.mk file.

Changes since RFCv1:

* Remove remaining references to the Crosstool-NG backend in the source code, noticed by Thomas De Schampheleire.

* Update the manual after Crosstool-NG backend removal

* Added Acked-by on the relevant patches.

* Ensure 'helpers.mk' is properly include from the right place after being moved into toolchain-external. Noticed by Luca Ceresoli.

* Fixed a few typos in comments, noticed by Thomas De Schampheleire.

* Renamed <pkg>_SOURCE_ADDONS to <pkg>_EXTRA_DOWNLOADS as suggested by Luca Ceresoli, and added documentation for this new variable in the Buildroot manual.

* The support for musl-based external toolchains has been integrated in this patch set, together with the fix for the hardcoded ld-linux-arm.so problem that was noticed in the v1 of the musl patch set.

* Updates for the Linaro ARM and AArch64 toolchains have been added.

* A fix for the Linaro ARM related bug #6452 has been added.

Thanks,

Thomas

Thomas Petazzoni (20):
glibc: both eglibc and glibc need host-gawk
glibc: fix glibc build by creating an empty gnu/stubs.h
package: add a <pkg>_EXTRA_DOWNLOADS variable
toolchain: introduce a virtual package
toolchain-buildroot: convert to the package infrastructure
toolchain-external: convert to the package infrastructure
dependencies: remove useless targets
toolchain-external: conditionalize the installation of libraries
toolchain: modify the wildcard logic for shared libraries copying
toolchain: fix the helpers.mk comment
toolchain: do not check largefile, wchar, IPv6 and locale for glibc
toolchains
toolchain-external: add support for musl C library
toolchain-external: improve help text of some options
busybox: add patches to fix build with the musl C library
toolchain-external: update Linaro ARM toolchain
toolchain-external: update Linaro AArch64 toolchain
toolchain-external: improve target library copy logic
toolchain-external: fix Linaro ARM toolchain support
package: package-based implementation of source, external-deps and
legal-info
package: remove useless SHOW_EXTERNAL_DEPS support

 Makefile                                           |  70 ++-----
 docs/manual/adding-packages-generic.txt            |   6 +
 ...01-libbb-use-poll.h-instead-of-sys-poll.h.patch |  32 +++
 ...onfig-include-linux-if_slip.h-instead-of-.patch |  33 ++++
 ...libiproute-use-linux-if_packet.h-instead-.patch |  36 ++++
 package/glibc/glibc.mk                             |  27 ++-
 package/pkg-download.mk                            |  44 +----
 package/pkg-generic.mk                             |  22 ++-
 support/dependencies/dependencies.mk               |   8 -
 toolchain/Config.in                                |   9 +
 toolchain/helpers.mk                               |  46 +++--
 toolchain/toolchain-buildroot.mk                   |   9 -
 .../toolchain-buildroot/toolchain-buildroot.mk     |  17 ++
 toolchain/toolchain-external.mk                    |   4 -
 toolchain/toolchain-external/Config.in             |  61 ++++--
 .../{ext-tool.mk => toolchain-external.mk}         | 220 +++++++++++----------
 toolchain/toolchain/toolchain.mk                   |  18 ++
 17 files changed, 391 insertions(+), 271 deletions(-)
 create mode 100644 package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch
 create mode 100644 package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch
 create mode 100644 package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch
 delete mode 100644 toolchain/toolchain-buildroot.mk
 create mode 100644 toolchain/toolchain-buildroot/toolchain-buildroot.mk
 delete mode 100644 toolchain/toolchain-external.mk
 rename toolchain/toolchain-external/{ext-tool.mk => toolchain-external.mk} (82%)
 create mode 100644 toolchain/toolchain/toolchain.mk
Signature · 1 line
1.8.1.2
#1 lore raw

PATCH v3 1/20 glibc: both eglibc and glibc need host-gawk

Until now, only the eglibc build was pulling host-gawk as a dependency, but after more testing, it turns out that the glibc build also requires host-gawk.

---
 package/glibc/glibc.mk | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
package/glibc/glibc.mk +1 -6
index 48ff0f7..a9e89f1 100644
@@ -21,12 +21,7 @@ GLIBC_LICENSE_FILES = $(addprefix $(GLIBC_SRC_SUBDIR)/,COPYING COPYING.LIB LICEN
 
 # Before (e)glibc is configured, we must have the first stage
 # cross-compiler and the kernel headers
-GLIBC_DEPENDENCIES = host-gcc-initial linux-headers
-
-# eglibc also needs host-gawk
-ifeq ($(BR2_TOOLCHAIN_BUILDROOT_EGLIBC),y)
-GLIBC_DEPENDENCIES += host-gawk
-endif
+GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-gawk
 
 # Before (e)glibc is built, we must have the second stage cross-compiler
 glibc-build: host-gcc-intermediate
Signature · 1 line
1.8.1.2
#2 lore raw

PATCH v3 1/20 glibc: both eglibc and glibc need host-gawk

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> Until now, only the eglibc build was pulling host-gawk as a Thomas> dependency, but after more testing, it turns out that the glibc build Thomas> also requires host-gawk.

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#3 lore raw

PATCH v3 2/20 glibc: fix glibc build by creating an empty gnu/stubs.h

In reply to Thomas Petazzoni

The stubs.h header is not installed by install-headers, but is needed for the gcc build. An empty stubs.h will work, as explained in
http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick is used by Crosstool-NG.

---
 package/glibc/glibc.mk | 12 ++++++++++++
 1 file changed, 12 insertions(+)
package/glibc/glibc.mk +12 -0
index a9e89f1..4d89caf 100644
@@ -47,6 +47,17 @@ else ifeq ($(BR2_MIPS_OABI32),y)
 GLIBC_EXTRA_CFLAGS += -mabi=32
 endif
 
+# The stubs.h header is not installed by install-headers, but is
+# needed for the gcc build. An empty stubs.h will work, as explained
+# in http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick
+# is used by Crosstool-NG.
+ifeq ($(BR2_TOOLCHAIN_BUILDROOT_GLIBC),y)
+define GLIBC_ADD_MISSING_STUB_H
+	mkdir -p $(STAGING_DIR)/usr/include/gnu
+	touch $(STAGING_DIR)/usr/include/gnu/stubs.h
+endef
+endif
+
 # Even though we use the autotools-package infrastructure, we have to
 # override the default configure commands for several reasons:
 #
@@ -89,6 +100,7 @@ define GLIBC_CONFIGURE_CMDS
 	cp $(@D)/build/csu/crt1.o $(STAGING_DIR)/usr/lib/
 	cp $(@D)/build/csu/crti.o $(STAGING_DIR)/usr/lib/
 	cp $(@D)/build/csu/crtn.o $(STAGING_DIR)/usr/lib/
+	$(GLIBC_ADD_MISSING_STUB_H)
 	$(TARGET_CROSS)gcc -nostdlib \
 		-nostartfiles -shared -x c /dev/null -o $(STAGING_DIR)/usr/lib/libc.so
 endef
Signature · 1 line
1.8.1.2
#4 lore raw

PATCH v3 2/20 glibc: fix glibc build by creating an empty gnu/stubs.h

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> The stubs.h header is not installed by install-headers, but is needed Thomas> for the gcc build. An empty stubs.h will work, as explained in Thomas> http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick is Thomas> used by Crosstool-NG.

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#5 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

In reply to Thomas Petazzoni

Converting the external toolchain logic into a package raises a very special use case that wasn't handled by the package infrastructure:
the Blackfin toolchain is delivered as two tarballs instead of one. Unfortunately <pkg>_SOURCE only allows to pass one tarball name.

However, we really want both tarballs to be known by the package infrastructure, so that the normal 'source' and 'external-deps' mechanism work fine.

In order to achieve this, we add a <pkg>_EXTRA_DOWNLOADS variable, which allows a package to list other stuff it would like to see downloaded, but that are otherwise not used by the package infrastructure itself: it is up to the package to do it by itself.

---
 docs/manual/adding-packages-generic.txt | 6 ++++++
 package/pkg-generic.mk                  | 1 +
 2 files changed, 7 insertions(+)
docs/manual/adding-packages-generic.txt +6 -0
index cb067bd..59a2684 100644
@@ -248,6 +248,12 @@ information is (assuming the package name is +libfoo+) :
     +LIBFOO_SITE=/opt/software/libfoo.tar.gz+ +
     +LIBFOO_SITE=$(TOPDIR)/../src/libfoo/+
 
+* +LIBFOO_EXTRA_DOWNLOADS+ lists a number of additional files that
+  Buildroot should download from +LIBFOO_SITE+ in addition to the main
+  +LIBFOO_SOURCE+ (which usually is a tarball). Buildroot will not do
+  anything with those additional files, except download files: it will
+  be up to the package recipe to use them from +$(DL_DIR)+.
+
 * +LIBFOO_SITE_METHOD+ determines the method used to fetch or copy the
   package source code. In many cases, Buildroot guesses the method
   from the contents of +LIBFOO_SITE+ and setting +LIBFOO_SITE_METHOD+
package/pkg-generic.mk +1 -0
index bfc4dc1..a46457c 100644
@@ -40,6 +40,7 @@ ifeq ($(DL_MODE),DOWNLOAD)
 	fi
 endif
 	$(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE:/=)/$($(PKG)_SOURCE)))
+	$(foreach p,$($(PKG)_EXTRA_DOWNLOADS),$(call DOWNLOAD,$($(PKG)_SITE:/=)/$(p))$(sep))
 	$(foreach p,$($(PKG)_PATCH),\
 		$(if $(findstring ://,$(p)),\
 			$(call DOWNLOAD,$(p)),\
Signature · 1 line
1.8.1.2
#6 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> Converting the external toolchain logic into a package raises a very Thomas> special use case that wasn't handled by the package infrastructure:
Thomas> the Blackfin toolchain is delivered as two tarballs instead of Thomas> one. Unfortunately <pkg>_SOURCE only allows to pass one tarball name.

Thomas> However, we really want both tarballs to be known by the package Thomas> infrastructure, so that the normal 'source' and 'external-deps' Thomas> mechanism work fine.

Thomas> In order to achieve this, we add a <pkg>_EXTRA_DOWNLOADS variable, Thomas> which allows a package to list other stuff it would like to see Thomas> downloaded, but that are otherwise not used by the package Thomas> infrastructure itself: it is up to the package to do it by itself.

I think it should be possible to support a space seperated list of sources in <pkg>_SOURCE, similar to how we do it for <pkg>_PATCH without breaking backwards compatibility, but I won't delay this series because of it - So committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#7 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

Dear Peter Korsgaard,

Quoted 9 lines · Peter Korsgaard

Thomas> In order to achieve this, we add a <pkg>_EXTRA_DOWNLOADS variable, Thomas> which allows a package to list other stuff it would like to see Thomas> downloaded, but that are otherwise not used by the package Thomas> infrastructure itself: it is up to the package to do it by itself.

I think it should be possible to support a space seperated list of sources in <pkg>_SOURCE, similar to how we do it for <pkg>_PATCH without breaking backwards compatibility, but I won't delay this series because of it - So committed, thanks.

This has been discussed in a prior version of this patch set. The problem is that <pkg>_SOURCE is not only about downloading, but also about extracting.

So, what would <pkg>_SOURCE do? Download all the files listed, but extract only the first one? This seems like a weird semantic. I think I prefer the following clear semantic:

<pkg>_SOURCE is one file, that gets downloaded and then extracted into the source directory of the package.

<pkg>_PATCH is a list of files or URLs, that get downloaded and apply as patches into the source directory

<pkg>_EXTRA_DOWNLOADS is a list of files that get downloaded, and nothing else. The package is responsible for doing something with them.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#8 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hi,

Quoted 4 lines
Quoted 4 lines

I think it should be possible to support a space seperated list of sources in <pkg>_SOURCE, similar to how we do it for <pkg>_PATCH without breaking backwards compatibility, but I won't delay this series because of it - So committed, thanks.

Thomas> This has been discussed in a prior version of this patch set. The Thomas> problem is that <pkg>_SOURCE is not only about downloading, but also Thomas> about extracting.

Thomas> So, what would <pkg>_SOURCE do? Download all the files listed, Thomas> but extract only the first one? This seems like a weird Thomas> semantic. I think I prefer the following clear semantic:

Thomas> <pkg>_SOURCE is one file, that gets downloaded and then extracted into Thomas> the source directory of the package.

Thomas> <pkg>_PATCH is a list of files or URLs, that get downloaded and apply Thomas> as patches into the source directory

 Thomas>  <pkg>_EXTRA_DOWNLOADS is a list of files that get downloaded,
 Thomas>  and nothing else. The package is responsible for doing
 Thomas>  something with them.

Well, I don't feel strongly about it, but no matter what we do packages with multple sources will be "special" (Luckily they are very uncommon), so you always have to do something explicit to use (extract) the extra sources.

But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner than just saying that only the first element in _SOURCE will be automatically extracted.

Signature · 1 line
Bye, Peter Korsgaard
#9 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

Dear Peter Korsgaard,

Quoted 14 lines · Peter Korsgaard

Thomas> This has been discussed in a prior version of this patch set. The Thomas> problem is that <pkg>_SOURCE is not only about downloading, but also Thomas> about extracting.

Thomas> So, what would <pkg>_SOURCE do? Download all the files listed, Thomas> but extract only the first one? This seems like a weird Thomas> semantic. I think I prefer the following clear semantic:

Thomas> <pkg>_SOURCE is one file, that gets downloaded and then extracted into Thomas> the source directory of the package.

Thomas> <pkg>_PATCH is a list of files or URLs, that get downloaded and apply Thomas> as patches into the source directory

Quoted 8 lines
 Thomas>  <pkg>_EXTRA_DOWNLOADS is a list of files that get downloaded,
 Thomas>  and nothing else. The package is responsible for doing
 Thomas>  something with them.

Well, I don't feel strongly about it, but no matter what we do packages with multple sources will be "special" (Luckily they are very uncommon), so you always have to do something explicit to use (extract) the extra sources.

Sure.

Quoted 3 lines

But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner than just saying that only the first element in _SOURCE will be automatically extracted.

Well, I find it cleaner, but if you prefer to have it in _SOURCE, I can certainly modify the patch to extend the semantic of _SOURCE instead. Let me know what you decide so that I can respin the patch set if needed.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#10 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hi,

Quoted 3 lines
Quoted 3 lines

But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner than just saying that only the first element in _SOURCE will be automatically extracted.

Thomas> Well, I find it cleaner, but if you prefer to have it in Thomas> _SOURCE, I can certainly modify the patch to extend the Thomas> semantic of _SOURCE instead. Let me know what you decide so Thomas> that I can respin the patch set if needed.

Just leave it for now. _EXTRA_DOWNLOADS is ok.

Signature · 1 line
Bye, Peter Korsgaard
#11 lore raw

PATCH v3 3/20 package: add a <pkg>_EXTRA_DOWNLOADS variable

Dear Peter Korsgaard,

Quoted 10 lines · Peter Korsgaard
Quoted 3 lines
Quoted 3 lines

But I don't particulary find a special _EXTRA_DOWNLOADS any cleaner than just saying that only the first element in _SOURCE will be automatically extracted.

Thomas> Well, I find it cleaner, but if you prefer to have it in Thomas> _SOURCE, I can certainly modify the patch to extend the Thomas> semantic of _SOURCE instead. Let me know what you decide so Thomas> that I can respin the patch set if needed.

Just leave it for now. _EXTRA_DOWNLOADS is ok.

I don't mind at all changing this, just let me know if I should do it. You're mail was not clear about what you really wanted to happen. I must say I don't have a really strong opinion about this, so either way is fine with me.

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#12 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

In reply to Thomas Petazzoni

This commit introduces a virtual package called 'toolchain', located in 'toolchain/toolchain', which simply depends on 'toolchain-buildroot' or 'toolchain-external' depending on the selected toolchain backend.

For now, toolchain-buildroot and toolchain-external are still manual make targets, but the following patches convert those backends to use the package infrastructure as well.

In addition to this:

* The main Makefile is modified to always make BASE_TARGETS point to this new toolchain virtual package.

* The main Makefile is changed to include all the toolchain/*/*.mk
files: the toolchain virtual package, and the toolchain-buildroot and toolchain-external directories.

* The dependency of the toolchain on prepare dirs and dependencies is moved to the toolchain virtual package. It is moved as a prerequisite of the "toolchain-source" rule to ensure that all directories are prepared before we even start extracting the toolchain.

---
 Makefile                         | 19 ++++---------------
 toolchain/toolchain/toolchain.mk | 18 ++++++++++++++++++
 2 files changed, 22 insertions(+), 15 deletions(-)
 create mode 100644 toolchain/toolchain/toolchain.mk
Makefile +4 -15
index 2fb1168..f266e2d 100644
@@ -236,11 +236,7 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 #
 ################################################################################
 
-ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
-BASE_TARGETS += toolchain-buildroot
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
-BASE_TARGETS += toolchain-external
-endif
+BASE_TARGETS = toolchain
 
 TARGETS:=
 
@@ -317,11 +313,8 @@ include support/dependencies/dependencies.mk
 # We also need the various per-package makefiles, which also add
 # each selected package to TARGETS if that package was selected
 # in the .config file.
-ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
-include toolchain/toolchain-buildroot.mk
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
-include toolchain/toolchain-external.mk
-endif
+include toolchain/helpers.mk
+include toolchain/*/*.mk
 
 # Include the package override file if one has been provided in the
 # configuration.
@@ -388,16 +381,12 @@ $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
-$(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
-
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
 
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
-toolchain: prepare dirs dependencies $(BASE_TARGETS)
-
-world: toolchain $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(TARGETS_ALL)
 
 .PHONY: all world toolchain dirs clean distclean source outputmakefile \
 	legal-info legal-info-prepare legal-info-clean printvars \
toolchain/toolchain/toolchain.mk new file +18 -0
new file mode 100644
index 0000000..44ed629
@@ -0,0 +1,18 @@
+################################################################################
+#
+# toolchain
+#
+################################################################################
+
+TOOLCHAIN_SOURCE =
+
+ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
+TOOLCHAIN_DEPENDENCIES += toolchain-buildroot
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
+TOOLCHAIN_DEPENDENCIES += toolchain-external
+endif
+
+$(eval $(generic-package))
+
+toolchain-source: prepare dirs dependencies $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
+
Signature · 1 line
1.8.1.2
#13 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Hi Thomas,

Makefile $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) % · line 389

Quoted 7 lines · Thomas Petazzoni

[...]
prepare: $(BUILD_DIR)/buildroot-config/auto.conf

Makefile +1 -3
-toolchain: prepare dirs dependencies $(BASE_TARGETS)
-
-world: toolchain $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(TARGETS_ALL)

Why do you change that? It seems to me better to use the "toolchain" target as dependency instead of $(BASE_TARGETS)
IMHO $(BASE_TARGETS) is an implementation detail of the new toolchain virtual package so it's better to use inside the new toolchain virtual package.

Quoted 1 line

[...]

Best regards

Signature · 1 line
Fabio Porcedda
#14 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Dear Fabio Porcedda,

Quoted 7 lines · Fabio Porcedda

Hi Thomas,

Quoted 3 lines · Thomas Petazzoni

[...]
prepare: $(BUILD_DIR)/buildroot-config/auto.conf

Quoted 10 lines

Makefile $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) % · line 389

Quoted 4 lines
Makefile +1 -3
-toolchain: prepare dirs dependencies $(BASE_TARGETS)
-
-world: toolchain $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(TARGETS_ALL)

Why do you change that? It seems to me better to use the "toolchain" target as dependency instead of $(BASE_TARGETS)
IMHO $(BASE_TARGETS) is an implementation detail of the new toolchain virtual package so it's better to use inside the new toolchain virtual package.

BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure why you say BASE_TARGETS is an implementation detail of the new toolchain virtual package. BASE_TARGETS now never has any other value than just "toolchain".

Of course, we might want to get rid of BASE_TARGETS later on, but for the moment, BASE_TARGETS is still used in other places, like when building the list of packages for which make source or make external-deps should be executed:

PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS)) PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))

Ultimately, toolchain will no longer need to be part of BASE_TARGETS as it will be handled just as a normal package dependency (done by your patches). But I'm trying to do things step by step, progressively. Therefore, in the end, I believe the world rule will no even contain "toolchain" and $(TARGETS_ALL) but just $(PACKAGES), but we can't change the world in one day :-)

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#15 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Quoted 21 lines · Thomas Petazzoni

Dear Fabio Porcedda,

Quoted 7 lines · Fabio Porcedda

Hi Thomas,

Quoted 3 lines · Thomas Petazzoni

[...]
prepare: $(BUILD_DIR)/buildroot-config/auto.conf

Quoted 10 lines

Makefile $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) % · line 389

Quoted 4 lines
Makefile +1 -3
-toolchain: prepare dirs dependencies $(BASE_TARGETS)
-
-world: toolchain $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(TARGETS_ALL)

Why do you change that? It seems to me better to use the "toolchain" target as dependency instead of $(BASE_TARGETS)
IMHO $(BASE_TARGETS) is an implementation detail of the new toolchain virtual package so it's better to use inside the new toolchain virtual package.

Quoted 4 lines

BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure why you say BASE_TARGETS is an implementation detail of the new toolchain virtual package. BASE_TARGETS now never has any other value than just "toolchain".

Sorry, I've missed that part, now I understand better.

Quoted 7 lines

Of course, we might want to get rid of BASE_TARGETS later on, but for the moment, BASE_TARGETS is still used in other places, like when building the list of packages for which make source or make external-deps should be executed:

PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS)) PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))

Ok I understand.

Quoted 5 lines

Ultimately, toolchain will no longer need to be part of BASE_TARGETS as it will be handled just as a normal package dependency (done by your patches). But I'm trying to do things step by step, progressively. Therefore, in the end, I believe the world rule will no even contain "toolchain" and $(TARGETS_ALL) but just $(PACKAGES),

Well about the world rule in my patches in the end it contain only the "target-post-image" because it's the last step, but that's another story.

Quoted 2 lines

but we can't
change the world in one day :-)

That's for sure :)

Thanks for these toolchains improvements

Signature · 1 line
Fabio Porcedda
#16 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Dear Fabio Porcedda,

Quoted 6 lines · Fabio Porcedda
Quoted 4 lines

BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure why you say BASE_TARGETS is an implementation detail of the new toolchain virtual package. BASE_TARGETS now never has any other value than just "toolchain".

Sorry, I've missed that part, now I understand better.

No problem :)

Quoted 9 lines
Quoted 5 lines

Ultimately, toolchain will no longer need to be part of BASE_TARGETS as it will be handled just as a normal package dependency (done by your patches). But I'm trying to do things step by step, progressively. Therefore, in the end, I believe the world rule will no even contain "toolchain" and $(TARGETS_ALL) but just $(PACKAGES),

Well about the world rule in my patches in the end it contain only the "target-post-image" because it's the last step, but that's another story.

Yes, indeed. Seen your patches, I think they look good on principle, I need to test them/have a more detailed look.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#17 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Quoted 19 lines · Thomas Petazzoni

Dear Fabio Porcedda,

Quoted 6 lines · Fabio Porcedda
Quoted 4 lines

BASE_TARGETS = toolchain, so there are exactly equivalent. I'm not sure why you say BASE_TARGETS is an implementation detail of the new toolchain virtual package. BASE_TARGETS now never has any other value than just "toolchain".

Sorry, I've missed that part, now I understand better.

No problem :)

Quoted 6 lines
Quoted 5 lines

Ultimately, toolchain will no longer need to be part of BASE_TARGETS as it will be handled just as a normal package dependency (done by your patches). But I'm trying to do things step by step, progressively. Therefore, in the end, I believe the world rule will no even contain "toolchain" and $(TARGETS_ALL) but just $(PACKAGES),

Quoted 6 lines
Quoted 3 lines

Well about the world rule in my patches in the end it contain only the "target-post-image" because it's the last step, but that's another story.

Yes, indeed. Seen your patches, I think they look good on principle, I need to test them/have a more detailed look.

About those patch set I'll send a new version rebased over your latest changes.

Best regards

Signature · 1 line
Fabio Porcedda
#18 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

In reply to Thomas Petazzoni
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> This commit introduces a virtual package called 'toolchain', located Thomas> in 'toolchain/toolchain', which simply depends on Thomas> 'toolchain-buildroot' or 'toolchain-external' depending on the Thomas> selected toolchain backend.

[snip]

Thomas> @@ -317,11 +313,8 @@ include support/dependencies/dependencies.mk

 Thomas>  # We also need the various per-package makefiles, which also add
 Thomas>  # each selected package to TARGETS if that package was selected
 Thomas>  # in the .config file.

Thomas> -ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y) Thomas> -include toolchain/toolchain-buildroot.mk Thomas> -else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y) Thomas> -include toolchain/toolchain-external.mk Thomas> -endif
Thomas> +include toolchain/helpers.mk Thomas> +include toolchain/*/*.mk

That's no good. toolchain/*/*.mk doesn't pick up up toolchain/toolchain-{buildroot,external}.mk:

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

toolchain undefined Extracting
toolchain undefined Patching toolchain//toolchain

make: *** No rule to make target toolchain-buildroot', needed by toolchain-depends'. Stop.

Did you mean to simply do 'include toolchain/*.mk' instead of those two lines?

Signature · 1 line
Bye, Peter Korsgaard
#19 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Dear Peter Korsgaard,

Quoted 21 lines · Peter Korsgaard
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> This commit introduces a virtual package called 'toolchain', located Thomas> in 'toolchain/toolchain', which simply depends on Thomas> 'toolchain-buildroot' or 'toolchain-external' depending on the Thomas> selected toolchain backend.

[snip]

Thomas> @@ -317,11 +313,8 @@ include support/dependencies/dependencies.mk

 Thomas>  # We also need the various per-package makefiles, which also add
 Thomas>  # each selected package to TARGETS if that package was selected
 Thomas>  # in the .config file.

Thomas> -ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y) Thomas> -include toolchain/toolchain-buildroot.mk Thomas> -else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y) Thomas> -include toolchain/toolchain-external.mk Thomas> -endif
Thomas> +include toolchain/helpers.mk Thomas> +include toolchain/*/*.mk

Quoted 8 lines

That's no good. toolchain/*/*.mk doesn't pick up up toolchain/toolchain-{buildroot,external}.mk:

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

toolchain undefined Extracting
toolchain undefined Patching toolchain//toolchain

make: *** No rule to make target toolchain-buildroot', needed by toolchain-depends'. Stop.

Did you mean to simply do 'include toolchain/*.mk' instead of those two lines?

No. The toolchain/toolchain-{buildroot,external}.mk file get removed in PATCH 5 and PATCH 6 respectively. So the problem is just a bisection
problem: I haven't tested the series after each commit.

Do you want me to respin with this bisection issue fixed?

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#20 lore raw

PATCH v3 4/20 toolchain: introduce a virtual package

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hi,

Quoted 2 lines
Quoted 2 lines

Did you mean to simply do 'include toolchain/*.mk' instead of those two lines?

Thomas> No. The toolchain/toolchain-{buildroot,external}.mk file get Thomas> removed in PATCH 5 and PATCH 6 respectively. So the problem is Thomas> just a bisection problem: I haven't tested the series after Thomas> each commit.

Tsk tsk ;)

Thomas> Do you want me to respin with this bisection issue fixed?

No, lets just leave it as it is - Thanks.

Signature · 1 line
Bye, Peter Korsgaard
#21 lore raw

PATCH v3 5/20 toolchain-buildroot: convert to the package infrastructure

In reply to Thomas Petazzoni

This commit converts the toolchain-buildroot logic to the package infrastructure. The package is fairly simple as it only defines BUILDROOT_LIBC, and depends on host-gcc-final to get the overall internal toolchain build logic started.

---
 toolchain/toolchain-buildroot.mk                     |  9 ---------
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 17 +++++++++++++++++
 2 files changed, 17 insertions(+), 9 deletions(-)
 delete mode 100644 toolchain/toolchain-buildroot.mk
 create mode 100644 toolchain/toolchain-buildroot/toolchain-buildroot.mk
toolchain/toolchain-buildroot.mk deleted +0 -9
deleted file mode 100644
index 3a05800..0000000
@@ -1,9 +0,0 @@
-# Triggerring the build of the host-gcc-final will automaticaly do the
-# build of binutils, uClibc, kernel headers and all the intermediate
-# gcc steps.
-
-include toolchain/helpers.mk
-
-BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
-
-toolchain-buildroot: host-gcc-final
toolchain/toolchain-buildroot/toolchain-buildroot.mk new file +17 -0
new file mode 100644
index 0000000..ced430e
@@ -0,0 +1,17 @@
+################################################################################
+#
+# toolchain-buildroot
+#
+################################################################################
+
+TOOLCHAIN_BUILDROOT_SOURCE =
+
+BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
+
+# Trigerring the build of the host-gcc-final will automatically do the
+# build of binutils, uClibc, kernel headers and all the intermediate
+# gcc steps.
+
+TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final
+
+$(eval $(generic-package))
Signature · 1 line
1.8.1.2
#22 lore raw

PATCH v3 5/20 toolchain-buildroot: convert to the package infrastructure

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> This commit converts the toolchain-buildroot logic to the package Thomas> infrastructure. The package is fairly simple as it only defines Thomas> BUILDROOT_LIBC, and depends on host-gcc-final to get the overall Thomas> internal toolchain build logic started.

It's more like getting it completed, but ok - Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#23 lore raw

PATCH v3 5/20 toolchain-buildroot: convert to the package infrastructure

In reply to Thomas Petazzoni

Hi Thomas,

Quoted 38 lines · Thomas Petazzoni

This commit converts the toolchain-buildroot logic to the package infrastructure. The package is fairly simple as it only defines BUILDROOT_LIBC, and depends on host-gcc-final to get the overall internal toolchain build logic started.

---
 toolchain/toolchain-buildroot.mk                     |  9 ---------
 toolchain/toolchain-buildroot/toolchain-buildroot.mk | 17 +++++++++++++++++
 2 files changed, 17 insertions(+), 9 deletions(-)
 delete mode 100644 toolchain/toolchain-buildroot.mk
 create mode 100644 toolchain/toolchain-buildroot/toolchain-buildroot.mk
toolchain/toolchain-buildroot.mk deleted +0 -9
deleted file mode 100644
index 3a05800..0000000
@@ -1,9 +0,0 @@
-# Triggerring the build of the host-gcc-final will automaticaly do the
-# build of binutils, uClibc, kernel headers and all the intermediate
-# gcc steps.
-
-include toolchain/helpers.mk
-
-BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
-
-toolchain-buildroot: host-gcc-final
toolchain/toolchain-buildroot/toolchain-buildroot.mk new file +3 -0
new file mode 100644
index 0000000..ced430e
@@ -0,0 +1,17 @@
+################################################################################
+#
+# toolchain-buildroot

toolchain/toolchain-buildroot/toolchain-buildroot.mk line 13

Quoted 10 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk new file +10 -0
+#
+################################################################################
+
+TOOLCHAIN_BUILDROOT_SOURCE =
+
+BUILDROOT_LIBC = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_LIBC))
+
+# Trigerring the build of the host-gcc-final will automatically do the
+# build of binutils, uClibc, kernel headers and all the intermediate
+# gcc steps.

It seems you miscorrected 'triggerring' after my comment on the first version of your patch (and I did not yet have/take time to review v2 and v3). It should be:
Triggering (double g, single r instead of the other way around).

Best regards,
Thomas

#24 lore raw

PATCH v3 5/20 toolchain-buildroot: convert to the package infrastructure

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas De Schampheleire <[email protected]> writes:

Hi,

Quoted 3 lines

toolchain/toolchain-buildroot/toolchain-buildroot.mk line 13

Quoted 3 lines
toolchain/toolchain-buildroot/toolchain-buildroot.mk new file +3 -0
+# Trigerring the build of the host-gcc-final will automatically do the
+# build of binutils, uClibc, kernel headers and all the intermediate
+# gcc steps.

Thomas> It seems you miscorrected 'triggerring' after my comment on the first Thomas> version of your patch (and I did not yet have/take time to review v2 Thomas> and v3). It should be:
Thomas> Triggering (double g, single r instead of the other way around).

Ahh yes, will fix - Thanks.

Signature · 1 line
Bye, Peter Korsgaard
#25 lore raw

PATCH v3 6/20 toolchain-external: convert to the package infrastructure

In reply to Thomas Petazzoni

This commit converts the 'toolchain-external' logic to the package infrastructure.

The TOOLCHAIN_EXTERNAL_DIR variable (which points to where the toolchain is located) is renamed to TOOLCHAIN_EXTERNAL_INSTALL_DIR, because the former conflicts with the package infrastructure (which defines the <pkg>_DIR variable for each package as pointing to its build directory).

The new _EXTRA_DOWNLOADS mechanism is used for Blackfin toolchains.

The extract, configuration and installation steps are converted inside the <pkg>_EXTRACT_CMDS, <pkg>_CONFIGURE_CMDS and <pkg>_INSTALL_STAGING_CMDS.

---
 toolchain/toolchain-external.mk                    |   4 -
 .../{ext-tool.mk => toolchain-external.mk}         | 154 +++++++++------------
 2 files changed, 69 insertions(+), 89 deletions(-)
 delete mode 100644 toolchain/toolchain-external.mk
 rename toolchain/toolchain-external/{ext-tool.mk => toolchain-external.mk} (86%)
toolchain/toolchain-external.mk deleted +0 -4
deleted file mode 100644
index 2f43db7..0000000
@@ -1,4 +0,0 @@
-# Required includes for the external toolchain backend
-
-include toolchain/helpers.mk
-include toolchain/toolchain-external/ext-tool.mk
toolchain/toolchain-external/toolchain-external.mk renamed from toolchain/toolchain-external/ext-tool.mk +69 -85

280 lines not on this page · raw message

Signature · 1 line
1.8.1.2
#26 lore raw

PATCH v3 6/20 toolchain-external: convert to the package infrastructure

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> This commit converts the 'toolchain-external' logic to the package Thomas> infrastructure.

Thomas> The TOOLCHAIN_EXTERNAL_DIR variable (which points to where the Thomas> toolchain is located) is renamed to TOOLCHAIN_EXTERNAL_INSTALL_DIR, Thomas> because the former conflicts with the package infrastructure (which Thomas> defines the <pkg>_DIR variable for each package as pointing to its Thomas> build directory).

Thomas> The new _EXTRA_DOWNLOADS mechanism is used for Blackfin toolchains.

Thomas> The extract, configuration and installation steps are converted inside Thomas> the <pkg>_EXTRACT_CMDS, <pkg>_CONFIGURE_CMDS and
Thomas> <pkg>_INSTALL_STAGING_CMDS.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]> Thomas> Acked-by: Luca Ceresoli <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#27 lore raw

PATCH v3 7/20 dependencies: remove useless targets

In reply to Thomas Petazzoni

The dependencies-source, dependencies-clean and dependencies-dirclean targets are not needed, as long as 'dependencies' is not used in the <pkg>_DEPENDENCIES of a package.

---
 support/dependencies/dependencies.mk | 8 --------
 1 file changed, 8 deletions(-)
support/dependencies/dependencies.mk +0 -8
index 97f85e6..155a909 100644
@@ -33,14 +33,6 @@ dependencies: HOSTCC=$(HOSTCC_NOCCACHE)
 dependencies: HOSTCXX=$(HOSTCXX_NOCCACHE)
 dependencies: core-dependencies $(DEPENDENCIES_HOST_PREREQ)
 
-dependencies-source:
-
-dependencies-clean:
-	rm -f $(SSTRIP_TARGET)
-
-dependencies-dirclean:
-	true
-
 ################################################################################
 #
 # Toplevel Makefile options
Signature · 1 line
1.8.1.2
#28 lore raw

PATCH v3 7/20 dependencies: remove useless targets

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> The dependencies-source, dependencies-clean and dependencies-dirclean Thomas> targets are not needed, as long as 'dependencies' is not used in the Thomas> <pkg>_DEPENDENCIES of a package.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#29 lore raw

PATCH v3 8/20 toolchain-external: conditionalize the installation of libraries

In reply to Thomas Petazzoni

The external toolchain code makes the assumption that all C libraries have a ld*.so, libc.so, libcrypt.so, libdl.so, libgcc_s.so, libm.so, libnsl.so, libresolv.so, libutil.so, and when thread support is enabled, libpthread.so, etc.

However, this is not the case with the musl C library, which integrates all the functionalities in a single libc.so file. In preparation of the support of the musl library, we make the current value of LIB_EXTERNAL_LIBS conditional to glibc or uClibc.

The addition of additional libraries through BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS is kept outside the condition, at the end.

---
 toolchain/toolchain-external/toolchain-external.mk | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
toolchain/toolchain-external/toolchain-external.mk +11 -8
index 85fc341..aaf4994 100644
@@ -54,8 +54,16 @@
 #  $(HOST_DIR)/usr/bin like for the internal toolchains, and the rest
 #  of Buildroot is handled identical for the 2 toolchain types.
 
-LIB_EXTERNAL_LIBS=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
-LIB_EXTERNAL_LIBS+=$(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS))
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
+LIB_EXTERNAL_LIBS+=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
+ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
+LIB_EXTERNAL_LIBS+=libpthread.so
+ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
+LIB_EXTERNAL_LIBS+=libthread_db.so
+endif # gdbserver
+endif # ! no threads
+endif
+
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
 LIB_EXTERNAL_LIBS+=libnss_files.so libnss_dns.so
 endif
@@ -64,12 +72,7 @@ ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
 USR_LIB_EXTERNAL_LIBS+=libstdc++.so
 endif
 
-ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
-LIB_EXTERNAL_LIBS+=libpthread.so
-ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
-LIB_EXTERNAL_LIBS+=libthread_db.so
-endif # gdbserver
-endif # ! no threads
+LIB_EXTERNAL_LIBS+=$(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS))
 
 # Details about sysroot directory selection.
 #
Signature · 1 line
1.8.1.2
#30 lore raw

PATCH v3 8/20 toolchain-external: conditionalize the installation of libraries

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> The external toolchain code makes the assumption that all C libraries Thomas> have a ld*.so, libc.so, libcrypt.so, libdl.so, libgcc_s.so, libm.so, Thomas> libnsl.so, libresolv.so, libutil.so, and when thread support is Thomas> enabled, libpthread.so, etc.

Thomas> However, this is not the case with the musl C library, which Thomas> integrates all the functionalities in a single libc.so file. In Thomas> preparation of the support of the musl library, we make the current Thomas> value of LIB_EXTERNAL_LIBS conditional to glibc or uClibc.

Thomas> The addition of additional libraries through
Thomas> BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS is kept outside the condition, at Thomas> the end.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#31 lore raw

PATCH v3 9/20 toolchain: modify the wildcard logic for shared libraries copying

In reply to Thomas Petazzoni

Until now, the copy_toolchain_lib_root function took as argument the base name of a library (e.g: libm.so), and was assuming that the usual scheme libm.so.<x> being a symbolic link to the real library was used.

However, with musl based toolchains, the C library is named libc.so directly, with no symbolic link at all. Therefore, this commit changes the copy_toolchain_lib_root to move the responsibility of using a wildcard or not after the library name the caller's responsibility.

So, all the existing LIB_EXTERNAL_LIBS values are modified to have a .* at the end, so that the behavior is effectively unchanged.

---
 package/glibc/glibc.mk                             |  8 ++++----
 toolchain/helpers.mk                               |  2 +-
 toolchain/toolchain-external/toolchain-external.mk | 10 +++++-----
 3 files changed, 10 insertions(+), 10 deletions(-)
package/glibc/glibc.mk +4 -4
index 4d89caf..0dab492 100644
@@ -112,12 +112,12 @@ endef
 #
 
 GLIBC_LIBS_LIB = \
-	ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so	   \
-	libnsl.so libpthread.so libresolv.so librt.so libutil.so   \
-	libnss_files.so libnss_dns.so
+	ld*.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.*        \
+	libnsl.so.* libpthread.so.* libresolv.so.* librt.so.* libutil.so.*   \
+	libnss_files.so.* libnss_dns.so.*
 
 ifeq ($(BR2_PACKAGE_GDB_SERVER),y)
-GLIBC_LIBS_LIB += libthread_db.so
+GLIBC_LIBS_LIB += libthread_db.so.*
 endif
 
 define GLIBC_INSTALL_TARGET_CMDS
toolchain/helpers.mk +1 -1
index 27b6710..37f2cf9 100644
@@ -54,7 +54,7 @@ copy_toolchain_lib_root = \
 		$${ARCH_SYSROOT_DIR}/$${ARCH_LIB_DIR} \
 		$${ARCH_SYSROOT_DIR}/usr/$${ARCH_LIB_DIR} \
 		$${SUPPORT_LIB_DIR} ; do \
-		LIBSPATH=`find $${dir} -maxdepth 1 -name "$${LIB}.*" 2>/dev/null` ; \
+		LIBSPATH=`find $${dir} -maxdepth 1 -name "$${LIB}" 2>/dev/null` ; \
 		if test -n "$${LIBSPATH}" ; then \
 			break ; \
 		fi \
toolchain/toolchain-external/toolchain-external.mk +5 -5
index aaf4994..2bfe64a 100644
@@ -55,21 +55,21 @@
 #  of Buildroot is handled identical for the 2 toolchain types.
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
-LIB_EXTERNAL_LIBS+=ld*.so libc.so libcrypt.so libdl.so libgcc_s.so libm.so libnsl.so libresolv.so librt.so libutil.so
+LIB_EXTERNAL_LIBS+=ld*.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
-LIB_EXTERNAL_LIBS+=libpthread.so
+LIB_EXTERNAL_LIBS+=libpthread.so.*
 ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
-LIB_EXTERNAL_LIBS+=libthread_db.so
+LIB_EXTERNAL_LIBS+=libthread_db.so.*
 endif # gdbserver
 endif # ! no threads
 endif
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
-LIB_EXTERNAL_LIBS+=libnss_files.so libnss_dns.so
+LIB_EXTERNAL_LIBS+=libnss_files.so.* libnss_dns.so.*
 endif
 
 ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
-USR_LIB_EXTERNAL_LIBS+=libstdc++.so
+USR_LIB_EXTERNAL_LIBS+=libstdc++.so.*
 endif
 
 LIB_EXTERNAL_LIBS+=$(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS))
Signature · 1 line
1.8.1.2
#32 lore raw

PATCH v3 9/20 toolchain: modify the wildcard logic for shared libraries copying

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> Until now, the copy_toolchain_lib_root function took as argument the Thomas> base name of a library (e.g: libm.so), and was assuming that the usual Thomas> scheme libm.so.<x> being a symbolic link to the real library was used.

Thomas> However, with musl based toolchains, the C library is named libc.so Thomas> directly, with no symbolic link at all. Therefore, this commit changes Thomas> the copy_toolchain_lib_root to move the responsibility of using a Thomas> wildcard or not after the library name the caller's responsibility.

Thomas> So, all the existing LIB_EXTERNAL_LIBS values are modified to have a Thomas> .* at the end, so that the behavior is effectively unchanged.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#33 lore raw

PATCH v3 10/20 toolchain: fix the helpers.mk comment

In reply to Thomas Petazzoni

This commit fixes various typos and mistakes in the comment at the top of the helpers.mk file:

* usefull -> useful

* The optional stripping of libraries no longer exists, so there's no reason to mention it.

* Indicate that the copy_toolchain_lib_root function is also used by the glibc package, not only by the external toolchain logic.

* Separate more clearly the top comment introducing the entire file, from the comment introducing the first function.

---
 toolchain/helpers.mk | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
toolchain/helpers.mk +7 -5
index 37f2cf9..d9051ae 100644
@@ -1,9 +1,11 @@
-# This Makefile fragment declares helper functions, usefull to handle
+# This Makefile fragment declares helper functions, useful to handle
 # non- buildroot-built toolchains, eg. purely external toolchains.
-#
-# Copy a toolchain library and its symbolic links from the sysroot
-# directory to the target directory. Also optionaly strips the
-# library.
+
+# The copy_toolchain_lib_root function copies a toolchain library and
+# its symbolic links from the sysroot directory to the target
+# directory. Note that this function is used both the external
+# toolchain logic, and the glibc package, so care must be taken when
+# changing this function.
 #
 # Most toolchains (CodeSourcery ones) have their libraries either in
 # /lib or /usr/lib relative to their ARCH_SYSROOT_DIR, so we search
Signature · 1 line
1.8.1.2
#34 lore raw

PATCH v3 10/20 toolchain: fix the helpers.mk comment

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> This commit fixes various typos and mistakes in the comment at the top Thomas> of the helpers.mk file:

Thomas> * usefull -> useful

Thomas> * The optional stripping of libraries no longer exists, so there's no
Thomas> reason to mention it.

Thomas> * Indicate that the copy_toolchain_lib_root function is also used by
Thomas> the glibc package, not only by the external toolchain logic.

Thomas> * Separate more clearly the top comment introducing the entire file,
Thomas> from the comment introducing the first function.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]> Thomas> ---

 Thomas>  toolchain/helpers.mk | 12 +++++++-----

Thomas> 1 file changed, 7 insertions(+), 5 deletions(-)

Thomas> diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk Thomas> index 37f2cf9..d9051ae 100644
Thomas> --- a/toolchain/helpers.mk
Thomas> +++ b/toolchain/helpers.mk
Thomas> @@ -1,9 +1,11 @@
Thomas> -# This Makefile fragment declares helper functions, usefull to handle Thomas> +# This Makefile fragment declares helper functions, useful to handle Thomas> # non- buildroot-built toolchains, eg. purely external toolchains.

This comment is still not quite true.

I've changed it to simply say:

This Makefile fragment declares toolchain related helper functions.

Thomas> -#
Thomas> -# Copy a toolchain library and its symbolic links from the sysroot Thomas> -# directory to the target directory. Also optionaly strips the Thomas> -# library.
Thomas> +
Thomas> +# The copy_toolchain_lib_root function copies a toolchain library and Thomas> +# its symbolic links from the sysroot directory to the target Thomas> +# directory. Note that this function is used both the external

s/both/by both/

Signature · 1 line
Bye, Peter Korsgaard
#35 lore raw

PATCH v3 10/20 toolchain: fix the helpers.mk comment

In reply to Thomas Petazzoni
Quoted 25 lines · Thomas Petazzoni

This commit fixes various typos and mistakes in the comment at the top of the helpers.mk file:

* usefull -> useful

* The optional stripping of libraries no longer exists, so there's no reason to mention it.

* Indicate that the copy_toolchain_lib_root function is also used by the glibc package, not only by the external toolchain logic.

* Separate more clearly the top comment introducing the entire file, from the comment introducing the first function.

---
 toolchain/helpers.mk | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
toolchain/helpers.mk +0 -1
index 37f2cf9..d9051ae 100644
@@ -1,9 +1,11 @@
-# This Makefile fragment declares helper functions, usefull to handle

toolchain/helpers.mk line 6

Quoted 10 lines
toolchain/helpers.mk +5 -4
+# This Makefile fragment declares helper functions, useful to handle
 # non- buildroot-built toolchains, eg. purely external toolchains.
-#
-# Copy a toolchain library and its symbolic links from the sysroot
-# directory to the target directory. Also optionaly strips the
-# library.
+
+# The copy_toolchain_lib_root function copies a toolchain library and
+# its symbolic links from the sysroot directory to the target
+# directory. Note that this function is used both the external

used for

toolchain/helpers.mk line 7

Quoted 1 line
toolchain/helpers.mk +1 -0
+# toolchain logic, and the glibc package, so care must be taken when

I think the comma before 'and the' is not needed.

toolchain/helpers.mk

Quoted 5 lines
toolchain/helpers.mk +1 -1
+# changing this function.
 #
 # Most toolchains (CodeSourcery ones) have their libraries either in
 # /lib or /usr/lib relative to their ARCH_SYSROOT_DIR, so we search
--

Best regards,
Thomas

#36 lore raw

PATCH v3 11/20 toolchain: do not check largefile, wchar, IPv6 and locale for glibc toolchains

In reply to Thomas Petazzoni

The check_glibc function contained checks to verify that the user had properly enabled the largefile, IPv6, locale and wchar options, to match how glibc is configured. This was useful when the support for glibc external toolchains was introduced, but since then, we added the BR2_TOOLCHAIN_USES_GLIBC symbol that automatically enables largefile, IPv6, locale and wchar support when a glibc or eglibc external toolchain is used. Therefore, many of the check_glibc checks are useless now, so we can remove these.

---
 toolchain/helpers.mk | 4 ----
 1 file changed, 4 deletions(-)
toolchain/helpers.mk +0 -4
index d9051ae..b530a02 100644
@@ -206,11 +206,7 @@ check_glibc = \
 		echo "Incorrect selection of the C library"; \
 		exit -1; \
 	fi; \
-	$(call check_glibc_feature,BR2_LARGEFILE,Large file support) ;\
-	$(call check_glibc_feature,BR2_INET_IPV6,IPv6 support) ;\
-	$(call check_glibc_feature,BR2_ENABLE_LOCALE,Locale support) ;\
 	$(call check_glibc_feature,BR2_USE_MMU,MMU support) ;\
-	$(call check_glibc_feature,BR2_USE_WCHAR,Wide char support) ;\
 	$(call check_glibc_rpc_feature,$${SYSROOT_DIR})
 
 #
Signature · 1 line
1.8.1.2
#37 lore raw

PATCH v3 11/20 toolchain: do not check largefile, wchar, IPv6 and locale for glibc toolchains

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> The check_glibc function contained checks to verify that the user had Thomas> properly enabled the largefile, IPv6, locale and wchar options, to Thomas> match how glibc is configured. This was useful when the support for Thomas> glibc external toolchains was introduced, but since then, we added the Thomas> BR2_TOOLCHAIN_USES_GLIBC symbol that automatically enables largefile, Thomas> IPv6, locale and wchar support when a glibc or eglibc external Thomas> toolchain is used. Therefore, many of the check_glibc checks are Thomas> useless now, so we can remove these.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#38 lore raw

PATCH v3 12/20 toolchain-external: add support for musl C library

In reply to Thomas Petazzoni

This commit adds support for external toolchains based on the musl C library, as available from http://www.musl-libc.org.

Note that the pre-built musl toolchains available from
http://musl.codu.org/ are not working for the moment, since they lack sysroot support. However, this problem has been reported to the maintainer, who has already added sysroot support in his scripts at
https://bitbucket.org/GregorR/musl-cross, and therefore the next version of the pre-built toolchains should work with Buildroot out-of-the-box. In the mean time, the musl-cross script must be used to build the toolchain.

---
 toolchain/Config.in                                |  9 +++++++++
 toolchain/helpers.mk                               | 11 +++++++++++
 toolchain/toolchain-external/Config.in             | 12 ++++++++++++
 toolchain/toolchain-external/toolchain-external.mk | 18 +++++++++++++++++-
 4 files changed, 49 insertions(+), 1 deletion(-)
toolchain/Config.in +9 -0
index 0f9a7dd..3980d79 100644
@@ -15,6 +15,15 @@ config BR2_TOOLCHAIN_USES_GLIBC
 config BR2_TOOLCHAIN_USES_UCLIBC
 	bool
 
+config BR2_TOOLCHAIN_USES_MUSL
+	bool
+	select BR2_LARGEFILE
+	select BR2_INET_IPV6
+	select BR2_USE_WCHAR
+	select BR2_ENABLE_LOCALE
+	select BR2_TOOLCHAIN_HAS_THREADS
+	select BR2_TOOLCHAIN_HAS_THREADS_DEBUG
+
 choice
 	prompt "Toolchain type"
 	help
toolchain/helpers.mk +11 -0
index b530a02..dfeef76 100644
@@ -210,6 +210,17 @@ check_glibc = \
 	$(call check_glibc_rpc_feature,$${SYSROOT_DIR})
 
 #
+# Check that the selected C library is really musl
+#
+# $1: sysroot directory
+check_musl = \
+	SYSROOT_DIR="$(strip $1)"; \
+	if test ! -f $${SYSROOT_DIR}/lib/libc.so -o -e $${SYSROOT_DIR}/lib/libm.so ; then \
+		echo "Incorrect selection of the C library" ; \
+		exit -1; \
+	fi
+
+#
 # Check the conformity of Buildroot configuration with regard to the
 # uClibc configuration of the external toolchain, for a particular
 # feature.
toolchain/toolchain-external/Config.in +12 -0
index c53577e..89c799e 100644
@@ -860,6 +860,10 @@ config BR2_TOOLCHAIN_EXTERNAL_UCLIBC
 	bool
 	select BR2_TOOLCHAIN_USES_UCLIBC
 
+config BR2_TOOLCHAIN_EXTERNAL_MUSL
+	bool
+	select BR2_TOOLCHAIN_USES_MUSL
+
 if BR2_TOOLCHAIN_EXTERNAL_CUSTOM
 
 choice
@@ -882,6 +886,13 @@ config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC
 	  eglibc configured to exclude key features may cause build failures to
 	  some packages.
 
+config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
+	bool "musl"
+	select BR2_TOOLCHAIN_EXTERNAL_MUSL
+	help
+	  Select this option if your external toolchain uses the
+	  'musl' C library, available from http://www.musl-libc.org/.
+
 endchoice
 
 if BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC
@@ -952,6 +963,7 @@ endif # BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC
 config BR2_TOOLCHAIN_EXTERNAL_INET_RPC
 	bool "Toolchain has RPC support?"
 	select BR2_TOOLCHAIN_HAS_NATIVE_RPC
+	depends on !BR2_TOOLCHAIN_EXTERNAL_MUSL
 	default y if BR2_TOOLCHAIN_EXTERNAL_GLIBC
 	help
 	  Select this option if your external toolchain supports
toolchain/toolchain-external/toolchain-external.mk +17 -1
index 2bfe64a..b2266b6 100644
@@ -68,6 +68,10 @@ ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y)
 LIB_EXTERNAL_LIBS+=libnss_files.so.* libnss_dns.so.*
 endif
 
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y)
+LIB_EXTERNAL_LIBS += libc.so libgcc_s.so.*
+endif
+
 ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
 USR_LIB_EXTERNAL_LIBS+=libstdc++.so.*
 endif
@@ -353,7 +357,7 @@ endif
 define TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS
 	$(Q)$(call check_cross_compiler_exists,$(TOOLCHAIN_EXTERNAL_CC))
 	$(Q)LIBC_A_LOCATION=`readlink -f $$(LANG=C $(TOOLCHAIN_EXTERNAL_CC) -print-file-name=libc.a)` ; \
-	SYSROOT_DIR=`echo $${LIBC_A_LOCATION} | sed -r -e 's:usr/lib(32|64)?/(.*/)?libc\.a::'` ; \
+	SYSROOT_DIR=`echo $${LIBC_A_LOCATION} | sed -r -e 's:(usr/)?lib(32|64)?/(.*/)?libc\.a::'` ; \
 	if test -z "$${SYSROOT_DIR}" ; then \
 		@echo "External toolchain doesn't support --sysroot. Cannot use." ; \
 		exit 1 ; \
@@ -368,11 +372,23 @@ define TOOLCHAIN_EXTERNAL_CONFIGURE_CMDS
 	fi ; \
 	if test "$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC)" = "y" ; then \
 		$(call check_uclibc,$${SYSROOT_DIR}) ; \
+	elif test "$(BR2_TOOLCHAIN_EXTERNAL_MUSL)" = "y" ; then \
+		$(call check_musl,$${SYSROOT_DIR}) ; \
 	else \
 		$(call check_glibc,$${SYSROOT_DIR}) ; \
 	fi
 endef
 
+# With the musl C library, the libc.so library directly plays the role
+# of the dynamic library loader. We just need to create a symbolic
+# link to libc.so with the appropriate name.
+ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y)
+define TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
+	ln -sf libc.so $(TARGET_DIR)/lib/ld-musl-$(ARCH).so.1
+endef
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
+endif
+
 # Integration of the toolchain into Buildroot: find the main sysroot
 # and the variant-specific sysroot, then copy the needed libraries to
 # the $(TARGET_DIR) and copy the whole sysroot (libraries and headers)
Signature · 1 line
1.8.1.2
#39 lore raw

PATCH v3 12/20 toolchain-external: add support for musl C library

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> This commit adds support for external toolchains based on the musl C Thomas> library, as available from http://www.musl-libc.org.

Thomas> Note that the pre-built musl toolchains available from Thomas> http://musl.codu.org/ are not working for the moment, since they lack Thomas> sysroot support. However, this problem has been reported to the Thomas> maintainer, who has already added sysroot support in his scripts at Thomas> https://bitbucket.org/GregorR/musl-cross, and therefore the next Thomas> version of the pre-built toolchains should work with Buildroot Thomas> out-of-the-box. In the mean time, the musl-cross script must be used Thomas> to build the toolchain.

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]>

Thomas> +# Check that the selected C library is really musl

s/is really/really is/

Committed with that fixed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#40 lore raw

PATCH v3 13/20 toolchain-external: improve help text of some options

In reply to Thomas Petazzoni
---
 toolchain/toolchain-external/Config.in | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
toolchain/toolchain-external/Config.in +11 -4
index 89c799e..67f4fc7 100644
@@ -876,15 +876,22 @@ config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_UCLIBC
 	# For the time being, we assume that all custom external
 	# toolchains have shadow password support.
 	select BR2_TOOLCHAIN_HAS_SHADOW_PASSWORDS
+	help
+	  Select this option if your external toolchain uses the
+	  uClibc C library (available from http://www.uclibc.org/).
 
 config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC
 	bool "glibc/eglibc"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
 	help
-	  Note: eglibc is a variant of glibc that (among other things) can be
-	  configured to exclude some of its features. Using a toolchain with
-	  eglibc configured to exclude key features may cause build failures to
-	  some packages.
+	  Select this option if your external toolchain uses the GNU C
+	  library (available from https://www.gnu.org/software/libc/)
+	  or its variant the eglibc library (http://www.eglibc.org/).
+
+	  Note: eglibc is a variant of glibc that (among other things)
+	  can be configured to exclude some of its features. Using a
+	  toolchain with eglibc configured to exclude key features may
+	  cause build failures to some packages.
 
 config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
 	bool "musl"
Signature · 1 line
1.8.1.2
#41 lore raw

PATCH v3 13/20 toolchain-external: improve help text of some options

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]> Thomas> ---

 Thomas>  toolchain/toolchain-external/Config.in | 15 +++++++++++----

Thomas> 1 file changed, 11 insertions(+), 4 deletions(-)

[snip]

 Thomas> +	  Note: eglibc is a variant of glibc that (among other things)
 Thomas> +	  can be configured to exclude some of its features. Using a
 Thomas> +	  toolchain with eglibc configured to exclude key features may
 Thomas> +	  cause build failures to some packages.

Thomas> config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
Thomas> bool "musl"

No musl help text?

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#42 lore raw

PATCH v3 13/20 toolchain-external: improve help text of some options

Dear Peter Korsgaard,

Quoted 14 lines · Peter Korsgaard
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> Signed-off-by: Thomas Petazzoni <[email protected]> Thomas> ---

 Thomas>  toolchain/toolchain-external/Config.in | 15 +++++++++++----

Thomas> 1 file changed, 11 insertions(+), 4 deletions(-)

[snip]

 Thomas> +	  Note: eglibc is a variant of glibc that (among other things)
 Thomas> +	  can be configured to exclude some of its features. Using a
 Thomas> +	  toolchain with eglibc configured to exclude key features may
 Thomas> +	  cause build failures to some packages.
Quoted 4 lines

Thomas> config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL
Thomas> bool "musl"

No musl help text?

Damn, will send a follow up patch.

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#43 lore raw

PATCH v3 14/20 busybox: add patches to fix build with the musl C library

In reply to Thomas Petazzoni

Those patches have been submitted upstream.

---
 ...01-libbb-use-poll.h-instead-of-sys-poll.h.patch | 32 +++++++++++++++++++
 ...onfig-include-linux-if_slip.h-instead-of-.patch | 33 ++++++++++++++++++++
 ...libiproute-use-linux-if_packet.h-instead-.patch | 36 ++++++++++++++++++++++
 3 files changed, 101 insertions(+)
 create mode 100644 package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch
 create mode 100644 package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch
 create mode 100644 package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch
package/busybox/1.21.1/0001-libbb-use-poll.h-instead-of-sys-poll.h.patch new file +32 -0
new file mode 100644
index 0000000..089d858
@@ -0,0 +1,32 @@
+From 676452a92d1fc4f85e8d89930d7729197c031026 Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <[email protected]>
+Date: Sat, 5 Oct 2013 15:51:56 +0200
+Subject: [PATCH 1/3] libbb: use <poll.h> instead of <sys/poll.h>
+
+As the pol() manpage recommends, include the <poll.h> header instead
+of <sys/poll.h>. This allows to get rid of gazillions of warnings when
+building Busybox against the musl C library, which prints a warning
+when the internal header <sys/poll.h> is included directly instead of
+the <poll.h> header.
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+---
+ include/libbb.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/include/libbb.h b/include/libbb.h
+index e520060..f03f454 100644
+--- a/include/libbb.h
++++ b/include/libbb.h
+@@ -37,7 +37,7 @@
+ #include <libgen.h> /* dirname,basename */
+ #undef basename
+ #define basename dont_use_basename
+-#include <sys/poll.h>
++#include <poll.h>
+ #include <sys/ioctl.h>
+ #include <sys/mman.h>
+ #include <sys/socket.h>
+-- 
+1.8.1.2
+
package/busybox/1.21.1/0002-network-ifconfig-include-linux-if_slip.h-instead-of-.patch new file +33 -0
new file mode 100644
index 0000000..ed35719
@@ -0,0 +1,33 @@
+From 5eae213031a3ef88fe8d9d0a387f44264ea635cc Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <[email protected]>
+Date: Sat, 5 Oct 2013 15:53:16 +0200
+Subject: [PATCH 2/3] network/ifconfig: include <linux/if_slip.h> instead of
+ <net/if_slip.h>
+
+The musl C library doesn't provide the <net/if_slip.h> since userspace
+applications can just as well use the <linux/if_slip.h> kernel header.
+
+This commit fixes the build of the ifconfig applet with the musl C
+library.
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+---
+ networking/ifconfig.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/networking/ifconfig.c b/networking/ifconfig.c
+index 782374b..c90ed6b 100644
+--- a/networking/ifconfig.c
++++ b/networking/ifconfig.c
+@@ -56,7 +56,7 @@
+ #endif
+ 
+ #if ENABLE_FEATURE_IFCONFIG_SLIP
+-# include <net/if_slip.h>
++# include <linux/if_slip.h>
+ #endif
+ 
+ /* I don't know if this is needed for busybox or not.  Anyone? */
+-- 
+1.8.1.2
+
package/busybox/1.21.1/0003-networking-libiproute-use-linux-if_packet.h-instead-.patch new file +36 -0
new file mode 100644
index 0000000..7b74656
@@ -0,0 +1,36 @@
+From d4fec31889ad660a58dab633c511221feb66e817 Mon Sep 17 00:00:00 2001
+From: Thomas Petazzoni <[email protected]>
+Date: Sat, 5 Oct 2013 15:55:06 +0200
+Subject: [PATCH 3/3] networking/libiproute: use <linux/if_packet.h> instead of
+ <net/if_packet.h>
+
+The musl C library doesn't provide the <net/if_packet.h> since the
+corresponding kernel headers <linux/if_packet.h> already provides the
+necessary definitions. Replacing <net/if_packet.h> by
+<linux/if_packet.h> also removes the need to include
+<netpacket/packet.h>
+
+This commit fixes the build of iplink with the musl C library.
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+---
+ networking/libiproute/iplink.c | 3 +--
+ 1 file changed, 1 insertion(+), 2 deletions(-)
+
+diff --git a/networking/libiproute/iplink.c b/networking/libiproute/iplink.c
+index bad2017..32ccb1c 100644
+--- a/networking/libiproute/iplink.c
++++ b/networking/libiproute/iplink.c
+@@ -5,8 +5,7 @@
+  * Licensed under GPLv2 or later, see file LICENSE in this source tree.
+  */
+ #include <net/if.h>
+-#include <net/if_packet.h>
+-#include <netpacket/packet.h>
++#include <linux/if_packet.h>
+ #include <netinet/if_ether.h>
+ 
+ #include "ip_common.h"  /* #include "libbb.h" is inside */
+-- 
+1.8.1.2
+
Signature · 1 line
1.8.1.2
#44 lore raw

PATCH v3 14/20 busybox: add patches to fix build with the musl C library

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thomas> Those patches have been submitted upstream.

Committed this and 15, 16, 17, 18 - Thanks.

Signature · 1 line
Bye, Peter Korsgaard
#45 lore raw

PATCH v3 15/20 toolchain-external: update Linaro ARM toolchain

In reply to Thomas Petazzoni
---
 toolchain/toolchain-external/Config.in             | 20 ++++++++++----------
 toolchain/toolchain-external/toolchain-external.mk |  6 +++---
 2 files changed, 13 insertions(+), 13 deletions(-)
toolchain/toolchain-external/Config.in +10 -10
index 67f4fc7..ecfe17c 100644
@@ -3,8 +3,8 @@ if BR2_TOOLCHAIN_EXTERNAL
 choice
 	prompt "Toolchain"
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
-	bool "Linaro 2013.08"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09
+	bool "Linaro 2013.09"
 	depends on BR2_arm
 	depends on BR2_GCC_TARGET_ARCH = "armv7-a"
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
@@ -15,7 +15,7 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
 	select BR2_HOSTARCH_NEEDS_IA32_LIBS
 	help
 	  Linaro toolchain for the ARM architecture. It uses Linaro
-	  GCC 2013.08 (based on gcc 4.8), Linaro GDB 2013.05 (based on
+	  GCC 2013.09 (based on gcc 4.8), Linaro GDB 2013.05 (based on
 	  GDB 7.6), eglibc 2.17, Binutils 2013.06 (based on 2.23). It
 	  generates code that runs on all Cortex-A profile devices,
 	  but tuned for the Cortex-A9. The code generated is Thumb 2,
@@ -24,8 +24,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
 
 	  To use this toolchain, you must disable soft float usage.
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
-	bool "Linaro 2013.07"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
+	bool "Linaro 2013.08"
 	depends on BR2_arm
 	depends on BR2_GCC_TARGET_ARCH = "armv7-a"
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
@@ -36,7 +36,7 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
 	select BR2_HOSTARCH_NEEDS_IA32_LIBS
 	help
 	  Linaro toolchain for the ARM architecture. It uses Linaro
-	  GCC 2013.07 (based on gcc 4.8), Linaro GDB 2013.05 (based on
+	  GCC 2013.08 (based on gcc 4.8), Linaro GDB 2013.05 (based on
 	  GDB 7.6), eglibc 2.17, Binutils 2013.06 (based on 2.23). It
 	  generates code that runs on all Cortex-A profile devices,
 	  but tuned for the Cortex-A9. The code generated is Thumb 2,
@@ -45,8 +45,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
 
 	  To use this toolchain, you must disable soft float usage.
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06
-	bool "Linaro 2013.06"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
+	bool "Linaro 2013.07"
 	depends on BR2_arm
 	depends on BR2_GCC_TARGET_ARCH = "armv7-a"
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
@@ -57,7 +57,7 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06
 	select BR2_HOSTARCH_NEEDS_IA32_LIBS
 	help
 	  Linaro toolchain for the ARM architecture. It uses Linaro
-	  GCC 2013.06 (based on gcc 4.8), Linaro GDB 2013.05 (based on
+	  GCC 2013.07 (based on gcc 4.8), Linaro GDB 2013.05 (based on
 	  GDB 7.6), eglibc 2.17, Binutils 2013.06 (based on 2.23). It
 	  generates code that runs on all Cortex-A profile devices,
 	  but tuned for the Cortex-A9. The code generated is Thumb 2,
@@ -816,9 +816,9 @@ config BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX
 
 config BR2_TOOLCHAIN_EXTERNAL_PREFIX
 	string
+	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09
 	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08
 	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07
-	default "arm-linux-gnueabihf"	 if BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201109
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201203
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201305
toolchain/toolchain-external/toolchain-external.mk +3 -3
index b2266b6..e38ee48 100644
@@ -239,15 +239,15 @@ define TOOLCHAIN_EXTERNAL_FIXUP_CMDS
 	mv $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/armv5te/* $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/
 	rm -rf $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/arago-2011.09/
 endef
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_06),y)
-TOOLCHAIN_EXTERNAL_SITE = https://releases.linaro.org/13.06/components/toolchain/binaries/
-TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.06_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.07/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.07-1_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.08/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux.tar.xz
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09),y)
+TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.09/components/toolchain/binaries/
+TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.09_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_MIPS201203),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/mips-linux-gnu/
 TOOLCHAIN_EXTERNAL_SOURCE = mips-2012.03-63-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2
Signature · 1 line
1.8.1.2
#46 lore raw

PATCH v3 16/20 toolchain-external: update Linaro AArch64 toolchain

In reply to Thomas Petazzoni
---
 toolchain/toolchain-external/Config.in             | 14 +++++++-------
 toolchain/toolchain-external/toolchain-external.mk |  6 +++---
 2 files changed, 10 insertions(+), 10 deletions(-)
toolchain/toolchain-external/Config.in +7 -7
index ecfe17c..0d42041 100644
@@ -730,8 +730,8 @@ config BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEBE_V2
 	  Toolchain for the Microblaze architecture, from
 	  http://wiki.xilinx.com/mb-gnu-tools
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
-	bool "Linaro AArch64 13.08"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_09
+	bool "Linaro AArch64 13.09"
 	depends on BR2_aarch64
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
@@ -742,8 +742,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
 	  Toolchain for the AArch64 architecture, from
 	  http://www.linaro.org/engineering/armv8/
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
-	bool "Linaro AArch64 13.07"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
+	bool "Linaro AArch64 13.08"
 	depends on BR2_aarch64
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
@@ -754,8 +754,8 @@ config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
 	  Toolchain for the AArch64 architecture, from
 	  http://www.linaro.org/engineering/armv8/
 
-config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_06
-	bool "Linaro AArch64 13.04"
+config BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
+	bool "Linaro AArch64 13.07"
 	depends on BR2_aarch64
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
 	select BR2_TOOLCHAIN_EXTERNAL_GLIBC
@@ -824,9 +824,9 @@ config BR2_TOOLCHAIN_EXTERNAL_PREFIX
 	default "arm-none-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201305
 	default "arm-arago-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_ARAGO_ARMV7A_201109
 	default "arm-arago-linux-gnueabi" if BR2_TOOLCHAIN_EXTERNAL_ARAGO_ARMV5TE_201109
+	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_09
 	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08
 	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07
-	default "aarch64-linux-gnu"      if BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_06
 	default "microblazeel-unknown-linux-gnu" if BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEEL_14_3
 	default "microblazeel-unknown-linux-gnu" if BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEEL_V2
 	default "microblaze-unknown-linux-gnu" if BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEBE_14_3
toolchain/toolchain-external/toolchain-external.mk +3 -3
index e38ee48..73a761b 100644
@@ -310,15 +310,15 @@ TOOLCHAIN_EXTERNAL_SOURCE = lin32-microblaze-unknown-linux-gnu_14.3_early.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_XILINX_MICROBLAZEBE_V2),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sources.buildroot.net/
 TOOLCHAIN_EXTERNAL_SOURCE = microblaze-unknown-linux-gnu.tgz
-else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_06),y)
-TOOLCHAIN_EXTERNAL_SITE = https://releases.linaro.org/13.06/components/toolchain/binaries/
-TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.06_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_07),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.07/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.07-1_linux.tar.xz
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_08),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.08/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.08_linux.tar.xz
+else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64_13_09),y)
+TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.09/components/toolchain/binaries/
+TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-aarch64-linux-gnu-4.8-2013.09_linux.tar.xz
 else
 # Custom toolchain
 TOOLCHAIN_EXTERNAL_SITE = $(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))
Signature · 1 line
1.8.1.2
#47 lore raw

PATCH v3 17/20 toolchain-external: improve target library copy logic

In reply to Thomas Petazzoni

The copy_toolchain_lib_root function is responsible for copying a given library (and its symbolic link) to the target filesystem. To do so, it looks for the library in various locations, and then iterates over the symbolic link all the way to the library, copying them as needed to the target filesystem.

However, the latest Linaro toolchains bring an interesting use case:
the lib/ directory in the toolchain is organized as follows:

- ld-linux.so.3 -> arm-linux-gnueabi/ld-2.17...so
- ld-linux-armhf.so.3 -> arm-linux-gnueabihf/ld-2.17...so
- arm-linux-gnueabi/
- all ARMv4T soft float libraries
- arm-linux-gnueabihf/
- all ARMv7 hard float libraries

In order to match what we do with all other toolchains, we want all those libraries and symbolic links to be copied directly under $(TARGET_DIR)/lib. This commit does that by adjusting the copy logic.

This is part of the fix for bug #6452 (eglibc from Linaro 2013.07 not copied to target correctly).

---
 toolchain/helpers.mk | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
toolchain/helpers.mk +10 -7
index dfeef76..f7e70cd 100644
@@ -61,21 +61,24 @@ copy_toolchain_lib_root = \
 			break ; \
 		fi \
 	done ; \
+	mkdir -p $(TARGET_DIR)/$${DESTDIR}; \
 	for LIBPATH in $${LIBSPATH} ; do \
-		LIBNAME=`basename $${LIBPATH}`; \
-		LIBDIR=`dirname $${LIBPATH}` ; \
-		while test \! -z "$${LIBNAME}" ; do \
-			LIBPATH=$${LIBDIR}/$${LIBNAME} ; \
+		while true ; do \
+			LIBNAME=`basename $${LIBPATH}`; \
+			LIBDIR=`dirname $${LIBPATH}` ; \
+			LINKTARGET=`readlink $${LIBPATH}` ; \
 			rm -fr $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
-			mkdir -p $(TARGET_DIR)/$${DESTDIR}; \
 			if test -h $${LIBPATH} ; then \
-				cp -d $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/; \
+				ln -sf `basename $${LINKTARGET}` $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME} ; \
 			elif test -f $${LIBPATH}; then \
 				$(INSTALL) -D -m0755 $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
 			else \
 				exit -1; \
 			fi; \
-			LIBNAME="`readlink $${LIBPATH}`"; \
+			if test -z "$${LINKTARGET}" ; then \
+				break ; \
+			fi ; \
+			LIBPATH="`readlink -f $${LIBPATH}`"; \
 		done; \
 	done; \
  \
Signature · 1 line
1.8.1.2
#48 lore raw

PATCH v3 18/20 toolchain-external: fix Linaro ARM toolchain support

In reply to Thomas Petazzoni

This commit fixes bug #6452 (eglibc from Linaro 2013.07 not copied to target correctly) by:

* Copying only the relevant library loader to the target on ARMhf (i.e ld-linux-armhf.so and not ld.so*). This is needed since Linaro toolchains provide two library loaders, one ARMv7 hf, and one ARMv4 soft-float.

* Making sure a $(TARGET_DIR)/lib/arm-linux-gnueabihf/ symbolic link to $(TARGET_DIR)/lib/ exists, since the dynamic loader of Linaro toolchains expects libraries to be found in $(TARGET_DIR)/lib/arm-linux-gnueabihf/.

---
 toolchain/toolchain-external/toolchain-external.mk | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
toolchain/toolchain-external/toolchain-external.mk +16 -1
index 73a761b..b5b1ce7 100644
@@ -55,7 +55,12 @@
 #  of Buildroot is handled identical for the 2 toolchain types.
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
-LIB_EXTERNAL_LIBS+=ld*.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
+LIB_EXTERNAL_LIBS+=libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
+ifeq ($(BR2_ARM_EABIHF),y)
+LIB_EXTERNAL_LIBS+=ld-linux-armhf.so.*
+else
+LIB_EXTERNAL_LIBS+=ld*.so.*
+endif
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
 LIB_EXTERNAL_LIBS+=libpthread.so.*
 ifneq ($(BR2_PACKAGE_GDB_SERVER)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
@@ -216,6 +221,13 @@ TOOLCHAIN_EXTERNAL_CFLAGS += -msoft-float
 TOOLCHAIN_EXTERNAL_WRAPPER_ARGS += -DBR_SOFTFLOAT=1
 endif
 
+# The Linaro ARMhf toolchain expects the libraries in
+# /lib/arm-linux-gnueabihf, but Buildroot copies them to /lib, so we
+# need to create a symbolic link.
+define TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
+	ln -sf . $(TARGET_DIR)/lib/arm-linux-gnueabihf
+endef
+
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM201109),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/arm-none-linux-gnueabi/
 TOOLCHAIN_EXTERNAL_SOURCE = arm-2011.09-70-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
@@ -242,12 +254,15 @@ endef
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_07),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.07/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.07-1_linux.tar.xz
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_08),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.08/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.08_linux.tar.xz
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_LINARO_2013_09),y)
 TOOLCHAIN_EXTERNAL_SITE = http://releases.linaro.org/13.09/components/toolchain/binaries/
 TOOLCHAIN_EXTERNAL_SOURCE = gcc-linaro-arm-linux-gnueabihf-4.8-2013.09_linux.tar.xz
+TOOLCHAIN_EXTERNAL_POST_INSTALL_STAGING_HOOKS += TOOLCHAIN_EXTERNAL_LINARO_ARMHF_SYMLINK
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_MIPS201203),y)
 TOOLCHAIN_EXTERNAL_SITE = http://sourcery.mentor.com/public/gnu_toolchain/mips-linux-gnu/
 TOOLCHAIN_EXTERNAL_SOURCE = mips-2012.03-63-mips-linux-gnu-i686-pc-linux-gnu.tar.bz2
Signature · 1 line
1.8.1.2
#49 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

In reply to Thomas Petazzoni

Until now, the make source, make external-deps and make legal-info where relying on enumerating the packages by looking at $(TARGETS) which contains only the target packages and not the host packages. Thanks to the TARGET_HOST_DEPS and HOST_DEPS variables, it was doing a two-level resolution of host package dependencies, but it was not entirely correct since the dependency chain might be deeper than that: some packages could be missed.

From an idea of Arnout, this patch introduces in each package additional targets <pkg>-all-source, <pkg>-all-legal-info and <pkg>-all-external-deps that executes the 'source', 'legal-info' and 'external-deps' targets for this package and all its dependencies, be they target or host dependencies.

This provides a much cleaner implementation of this mechanism, which is also more robust.

In order to achieve this, this patch also separates the "package" targets from other targets: instead of mixing them both in the global TARGETS variable, the new PACKAGES variable contains the name of all packages that are enabled in the configuration, while TARGETS is only used for additional things to be done (target-finalize, etc.). This separation is needed so that we don't try to use targets (such as <foo>-all-external-deps) on things that are not packages. Some further cleanups in this direction are possible, this commit takes a relatively minimal approach for now.

---
 Makefile               | 53 +++++++++++++-------------------------------------
 package/pkg-generic.mk | 16 ++++++++++++++-
 2 files changed, 28 insertions(+), 41 deletions(-)
Makefile +13 -40
index f266e2d..702e7d4 100644
@@ -238,8 +238,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 
 BASE_TARGETS = toolchain
 
-TARGETS:=
-
 # silent mode requested?
 QUIET:=$(if $(findstring s,$(MAKEFLAGS)),-q)
 
@@ -310,9 +308,6 @@ endif
 include package/Makefile.in
 include support/dependencies/dependencies.mk
 
-# We also need the various per-package makefiles, which also add
-# each selected package to TARGETS if that package was selected
-# in the .config file.
 include toolchain/helpers.mk
 include toolchain/*/*.mk
 
@@ -349,34 +344,11 @@ include fs/common.mk
 
 TARGETS+=target-post-image
 
-TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
-TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
-TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
-
-# host-* dependencies have to be handled specially, as those aren't
-# visible in Kconfig and hence not added to a variable like TARGETS.
-# instead, find all the host-* targets listed in each <PKG>_DEPENDENCIES
-# variable for each enabled target.
-# Notice: this only works for newstyle gentargets/autotargets packages
-TARGETS_HOST_DEPS = $(sort $(filter host-%,$(foreach dep,\
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS))),\
-		$($(dep)))))
-# Host packages can in turn have their own dependencies. Likewise find
-# all the package names listed in the HOST_<PKG>_DEPENDENCIES for each
-# host package found above. Ideally this should be done recursively until
-# no more packages are found, but that's hard to do in make, so limit to
-# 1 level for now.
-HOST_DEPS = $(sort $(foreach dep,\
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS_HOST_DEPS))),\
-		$($(dep))))
-HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS)))
-
-TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\
-		$(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS))))
-
-# all targets depend on the crosscompiler and it's prerequisites
-$(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
+PACKAGES_CLEAN:=$(patsubst %,%-clean,$(PACKAGES))
+PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))
+PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))
+PACKAGES_DIRCLEAN:=$(patsubst %,%-dirclean,$(PACKAGES))
+PACKAGES_LEGAL_INFO:=$(patsubst %,%-all-legal-info,$(PACKAGES) $(BASE_TARGETS))
 
 dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
@@ -386,12 +358,13 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
-world: $(BASE_TARGETS) $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
 .PHONY: all world toolchain dirs clean distclean source outputmakefile \
 	legal-info legal-info-prepare legal-info-clean printvars \
-	$(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \
-	$(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \
+	$(BASE_TARGETS) $(PACKAGES) $(TARGETS) \
+	$(PACKAGES_CLEAN) $(PACKAGES_DIRCLEAN) $(PACKAGES_SOURCE) $(PACKAGES_LEGAL_INFO) \
+	$(PACKAGES_EXTERNAL_DEPS) \
 	$(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
@@ -562,10 +535,10 @@ target-post-image:
 toolchain-eclipse-register:
 	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
 
-source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE)
+source: dirs $(PACKAGES_SOURCE)
 
 external-deps:
-	@$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
+	@$(MAKE) -s $(EXTRAMAKEARGS) $(PACKAGES_EXTERNAL_DEPS) | sort -u
 
 legal-info-clean:
 	@rm -fr $(LEGAL_INFO_DIR)
@@ -580,7 +553,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
 	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
 
 legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
-		$(TARGETS_LEGAL_INFO)
+		$(PACKAGES_LEGAL_INFO)
 	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
 	@if [ -r $(LEGAL_WARNINGS) ]; then \
 		cat support/legal-info/README.warnings-header \
@@ -590,7 +563,7 @@ legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
 	@rm -f $(LEGAL_WARNINGS)
 
 show-targets:
-	@echo $(TARGETS)
+	@echo $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
 else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
package/pkg-generic.mk +15 -1
index a46457c..bd6169c 100644
@@ -424,6 +424,20 @@ endif
 $(1)-show-depends:
 			@echo $$($(2)_DEPENDENCIES)
 
+$(1)-all-source: 	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-source) $(1)-source
+
+$(1)-external-deps:
+ifneq ($$($(2)_SOURCE),)
+			@echo $$($(2)_SOURCE)
+endif
+ifneq ($$($(2)_EXTRA_DOWNLOADS),y)
+			@echo $$($(2)_EXTRA_DOWNLOADS)
+endif
+
+$(1)-all-external-deps:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-external-deps) $(1)-external-deps
+
+$(1)-all-legal-info:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-legal-info) $(1)-legal-info
+
 $(1)-uninstall:		$(1)-configure $$($(2)_TARGET_UNINSTALL)
 
 $(1)-clean:		$(1)-uninstall \
@@ -528,7 +542,7 @@ endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
 # configuration
 ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
-TARGETS += $(1)
+PACKAGES += $(1)
 PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
 PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
 PACKAGES_USERS += $$($(2)_USERS)$$(sep)
Signature · 1 line
1.8.1.2
#50 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

Quoted 47 lines · Thomas Petazzoni

Until now, the make source, make external-deps and make legal-info where relying on enumerating the packages by looking at $(TARGETS) which contains only the target packages and not the host packages. Thanks to the TARGET_HOST_DEPS and HOST_DEPS variables, it was doing a two-level resolution of host package dependencies, but it was not entirely correct since the dependency chain might be deeper than that: some packages could be missed.

Quoted 1 line

From an idea of Arnout, this patch introduces in each package

additional targets <pkg>-all-source, <pkg>-all-legal-info and <pkg>-all-external-deps that executes the 'source', 'legal-info' and 'external-deps' targets for this package and all its dependencies, be they target or host dependencies.

This provides a much cleaner implementation of this mechanism, which is also more robust.

In order to achieve this, this patch also separates the "package" targets from other targets: instead of mixing them both in the global TARGETS variable, the new PACKAGES variable contains the name of all packages that are enabled in the configuration, while TARGETS is only used for additional things to be done (target-finalize, etc.). This separation is needed so that we don't try to use targets (such as <foo>-all-external-deps) on things that are not packages. Some further cleanups in this direction are possible, this commit takes a relatively minimal approach for now.

---
  Makefile               | 53 +++++++++++++-------------------------------------
  package/pkg-generic.mk | 16 ++++++++++++++-
  2 files changed, 28 insertions(+), 41 deletions(-)
Makefile +0 -2
index f266e2d..702e7d4 100644
@@ -238,8 +238,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 
  BASE_TARGETS = toolchain
 
-TARGETS:=
-
  # silent mode requested?
  QUIET:=$(if $(findstring s,$(MAKEFLAGS)),-q)
 

Makefile endif · line 310

Quoted 3 lines
Makefile +0 -0
@@ -310,9 +308,6 @@ endif
  include package/Makefile.in
  include support/dependencies/dependencies.mk
 

You could remove this empty line as well.

Quoted 29 lines
Makefile +0 -22
-# We also need the various per-package makefiles, which also add
-# each selected package to TARGETS if that package was selected
-# in the .config file.
  include toolchain/helpers.mk
  include toolchain/*/*.mk
 
@@ -349,34 +344,11 @@ include fs/common.mk
 
  TARGETS+=target-post-image
 
-TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
-TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
-TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))
-
-# host-* dependencies have to be handled specially, as those aren't
-# visible in Kconfig and hence not added to a variable like TARGETS.
-# instead, find all the host-* targets listed in each <PKG>_DEPENDENCIES
-# variable for each enabled target.
-# Notice: this only works for newstyle gentargets/autotargets packages
-TARGETS_HOST_DEPS = $(sort $(filter host-%,$(foreach dep,\
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS))),\
-		$($(dep)))))
-# Host packages can in turn have their own dependencies. Likewise find
-# all the package names listed in the HOST_<PKG>_DEPENDENCIES for each
-# host package found above. Ideally this should be done recursively until
-# no more packages are found, but that's hard to do in make, so limit to
-# 1 level for now.
-HOST_DEPS = $(sort $(foreach dep,\

Makefile include fs/common.mk · line 347

Quoted 10 lines
Makefile +1 -9
-		$(addsuffix _DEPENDENCIES,$(call UPPERCASE,$(TARGETS_HOST_DEPS))),\
-		$($(dep))))
-HOST_SOURCE += $(addsuffix -source,$(sort $(TARGETS_HOST_DEPS) $(HOST_DEPS)))
-
-TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\
-		$(TARGETS) $(BASE_TARGETS) $(TARGETS_HOST_DEPS) $(HOST_DEPS))))
-
-# all targets depend on the crosscompiler and it's prerequisites
-$(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
+PACKAGES_CLEAN:=$(patsubst %,%-clean,$(PACKAGES))

Can you make these match the coding style? PACKAGES_CLEAN = ...

Makefile include fs/common.mk · line 348

Quoted 1 line
Makefile +1 -0
+PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))

BASE_TARGETS is just "toolchain", right? And that is already part of
the dependencies of PACKAGES, right? So why is that still needed here?

Makefile include fs/common.mk · line 350

Quoted 2 lines
Makefile +2 -0
+PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))
+PACKAGES_DIRCLEAN:=$(patsubst %,%-dirclean,$(PACKAGES))

The PACKAGES_DIRCLEAN is really redundant, it is only used to define
them as .PHONY but that is not done for e.g. -build.

Quoted 59 lines
Makefile +9 -7
+PACKAGES_LEGAL_INFO:=$(patsubst %,%-all-legal-info,$(PACKAGES) $(BASE_TARGETS))
 
  dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
  	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
@@ -386,12 +358,13 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 
  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
-world: $(BASE_TARGETS) $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
  .PHONY: all world toolchain dirs clean distclean source outputmakefile \
  	legal-info legal-info-prepare legal-info-clean printvars \
-	$(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \
-	$(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \
+	$(BASE_TARGETS) $(PACKAGES) $(TARGETS) \
+	$(PACKAGES_CLEAN) $(PACKAGES_DIRCLEAN) $(PACKAGES_SOURCE) $(PACKAGES_LEGAL_INFO) \
+	$(PACKAGES_EXTERNAL_DEPS) \
  	$(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
  	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
@@ -562,10 +535,10 @@ target-post-image:
  toolchain-eclipse-register:
  	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
 
-source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE)
+source: dirs $(PACKAGES_SOURCE)
 
  external-deps:
-	@$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
+	@$(MAKE) -s $(EXTRAMAKEARGS) $(PACKAGES_EXTERNAL_DEPS) | sort -u
 
  legal-info-clean:
  	@rm -fr $(LEGAL_INFO_DIR)
@@ -580,7 +553,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
  	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
 
  legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
-		$(TARGETS_LEGAL_INFO)
+		$(PACKAGES_LEGAL_INFO)
  	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
  	@if [ -r $(LEGAL_WARNINGS) ]; then \
  		cat support/legal-info/README.warnings-header \
@@ -590,7 +563,7 @@ legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
  	@rm -f $(LEGAL_WARNINGS)
 
  show-targets:
-	@echo $(TARGETS)
+	@echo $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
  else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
package/pkg-generic.mk +0 -0
index a46457c..bd6169c 100644
@@ -424,6 +424,20 @@ endif
  $(1)-show-depends:
  			@echo $$($(2)_DEPENDENCIES)

package/pkg-generic.mk endif · line 435

Quoted 10 lines
package/pkg-generic.mk +9 -0
 
+$(1)-all-source: 	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-source) $(1)-source
+
+$(1)-external-deps:
+ifneq ($$($(2)_SOURCE),)
+			@echo $$($(2)_SOURCE)
+endif
+ifneq ($$($(2)_EXTRA_DOWNLOADS),y)
+			@echo $$($(2)_EXTRA_DOWNLOADS)
+endif
  While you're at it, you could add $(2)_PATCH.

Are the double dollars really needed? They're not used in -all-source
so why would you use them here...

Quoted 9 lines
package/pkg-generic.mk +5 -0
+
+$(1)-all-external-deps:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-external-deps) $(1)-external-deps
+
+$(1)-all-legal-info:	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-legal-info) $(1)-legal-info
+
  $(1)-uninstall:		$(1)-configure $$($(2)_TARGET_UNINSTALL)
 
  $(1)-clean:		$(1)-uninstall \

package/pkg-generic.mk endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),) · line 548

Quoted 8 lines
package/pkg-generic.mk +1 -1
@@ -528,7 +542,7 @@ endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
  # configuration
  ifeq ($$($$($(2)_KCONFIG_VAR)),y)
 
-TARGETS += $(1)
+PACKAGES += $(1)
  PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
  PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
  PACKAGES_USERS += $$($(2)_USERS)$$(sep)

Shouldn't you do something similar in fs/ ? Otherwise 'make source'
will not download e.g. mtd.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#51 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

Arnout,

Thanks a lot for your review!

Quoted 6 lines · Arnout Vandecappelle

Makefile endif · line 310

Quoted 4 lines
Makefile +0 -0
@@ -310,9 +308,6 @@ endif
  include package/Makefile.in
  include support/dependencies/dependencies.mk
 

You could remove this empty line as well.

Right.

Quoted 5 lines

Makefile include fs/common.mk · line 347

Quoted 3 lines
Makefile +1 -2
-# all targets depend on the crosscompiler and it's prerequisites
-$(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
+PACKAGES_CLEAN:=$(patsubst %,%-clean,$(PACKAGES))

Can you make these match the coding style? PACKAGES_CLEAN = ...

Yes, certainly.

Quoted 4 lines

Makefile include fs/common.mk · line 348

Quoted 1 line
Makefile +1 -0
+PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))

BASE_TARGETS is just "toolchain", right? And that is already part of
the dependencies of PACKAGES, right? So why is that still needed here?

Yes, BASE_TARGETS is just toolchain now. However no, "toolchain" is not yet part of the dependencies of all packages. This is something the patch series from Fabio Porcedda is doing, and I don't want to solve all problems in this patch set :)

The main reason I've kept BASE_TARGETS for now is because I knew Fabio would be cleaning up that further with his patch set.

Quoted 5 lines

Makefile include fs/common.mk · line 350

Quoted 2 lines
Makefile +2 -0
+PACKAGES_EXTERNAL_DEPS:=$(patsubst %,%-all-external-deps,$(PACKAGES) $(BASE_TARGETS))
+PACKAGES_DIRCLEAN:=$(patsubst %,%-dirclean,$(PACKAGES))

The PACKAGES_DIRCLEAN is really redundant, it is only used to define
them as .PHONY but that is not done for e.g. -build.

True, will remove.

Quoted 60 lines
Quoted 60 lines
Makefile +9 -7
+PACKAGES_LEGAL_INFO:=$(patsubst %,%-all-legal-info,$(PACKAGES) $(BASE_TARGETS))
 
  dirs: $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
  	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
@@ -386,12 +358,13 @@ $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 
  prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
-world: $(BASE_TARGETS) $(TARGETS_ALL)
+world: $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
  .PHONY: all world toolchain dirs clean distclean source outputmakefile \
  	legal-info legal-info-prepare legal-info-clean printvars \
-	$(BASE_TARGETS) $(TARGETS) $(TARGETS_ALL) \
-	$(TARGETS_CLEAN) $(TARGETS_DIRCLEAN) $(TARGETS_SOURCE) $(TARGETS_LEGAL_INFO) \
+	$(BASE_TARGETS) $(PACKAGES) $(TARGETS) \
+	$(PACKAGES_CLEAN) $(PACKAGES_DIRCLEAN) $(PACKAGES_SOURCE) $(PACKAGES_LEGAL_INFO) \
+	$(PACKAGES_EXTERNAL_DEPS) \
  	$(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
  	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
@@ -562,10 +535,10 @@ target-post-image:
  toolchain-eclipse-register:
  	./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
 
-source: dirs $(TARGETS_SOURCE) $(HOST_SOURCE)
+source: dirs $(PACKAGES_SOURCE)
 
  external-deps:
-	@$(MAKE) -Bs DL_MODE=SHOW_EXTERNAL_DEPS $(EXTRAMAKEARGS) source | sort -u
+	@$(MAKE) -s $(EXTRAMAKEARGS) $(PACKAGES_EXTERNAL_DEPS) | sort -u
 
  legal-info-clean:
  	@rm -fr $(LEGAL_INFO_DIR)
@@ -580,7 +553,7 @@ legal-info-prepare: $(LEGAL_INFO_DIR)
  	@cp $(BUILDROOT_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
 
  legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
-		$(TARGETS_LEGAL_INFO)
+		$(PACKAGES_LEGAL_INFO)
  	@cat support/legal-info/README.header >>$(LEGAL_REPORT)
  	@if [ -r $(LEGAL_WARNINGS) ]; then \
  		cat support/legal-info/README.warnings-header \
@@ -590,7 +563,7 @@ legal-info: dirs legal-info-clean legal-info-prepare $(REDIST_SOURCES_DIR) \
  	@rm -f $(LEGAL_WARNINGS)
 
  show-targets:
-	@echo $(TARGETS)
+	@echo $(BASE_TARGETS) $(PACKAGES) $(TARGETS)
 
  else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
package/pkg-generic.mk +0 -0
index a46457c..bd6169c 100644
@@ -424,6 +424,20 @@ endif
  $(1)-show-depends:
  			@echo $$($(2)_DEPENDENCIES)
 
Quoted 11 lines

package/pkg-generic.mk endif · line 435

Quoted 9 lines
package/pkg-generic.mk +9 -0
+$(1)-all-source: 	$(foreach p,$($(2)_DEPENDENCIES),$(p)-all-source) $(1)-source
+
+$(1)-external-deps:
+ifneq ($$($(2)_SOURCE),)
+			@echo $$($(2)_SOURCE)
+endif
+ifneq ($$($(2)_EXTRA_DOWNLOADS),y)
+			@echo $$($(2)_EXTRA_DOWNLOADS)
+endif
  While you're at it, you could add $(2)_PATCH.

Ahh, yes, true. I thought about it at some point, and then forgot.

Quoted 2 lines

Are the double dollars really needed? They're not used in -all-source
so why would you use them here...

I'll check that, maybe not.

Quoted 8 lines

package/pkg-generic.mk endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),) · line 548

Quoted 5 lines
package/pkg-generic.mk +1 -1
-TARGETS += $(1)
+PACKAGES += $(1)
  PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
  PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
  PACKAGES_USERS += $$($(2)_USERS)$$(sep)

Shouldn't you do something similar in fs/ ? Otherwise 'make source'
will not download e.g. mtd.

That's a weakness of this patch series. I believe I might need to convert the fs/ stuff to packages (after all they are packages that have dependencies, and install something in the images/ directory), so that the legal-info/source/external-deps logic works for them as well, without doing hacks. What do you think about this?

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#52 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

Quoted 7 lines · Thomas Petazzoni
Quoted 2 lines
Quoted 2 lines

Shouldn't you do something similar in fs/ ? Otherwise 'make source'
will not download e.g. mtd.

That's a weakness of this patch series. I believe I might need to convert the fs/ stuff to packages (after all they are packages that have dependencies, and install something in the images/ directory), so that the legal-info/source/external-deps logic works for them as well, without doing hacks. What do you think about this?

I don't think it's useful to make them packages, because they are
really different. Adding the propagation of -all-source etc. to the dependencies can easily be done in ROOTFS_TARGET_INTERNAL.

In a later stage, we could consider to move the (few) common parts into
some common package/rootfs infrastructure, but the gain is limited I expect.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#53 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

Dear Arnout Vandecappelle,

Quoted 3 lines · Arnout Vandecappelle

I don't think it's useful to make them packages, because they are
really different. Adding the propagation of -all-source etc. to the dependencies can easily be done in ROOTFS_TARGET_INTERNAL.

Yeah, that's indeed another option. We're already doing it with $(1)-show-depends.

Quoted 2 lines

In a later stage, we could consider to move the (few) common parts into
some common package/rootfs infrastructure, but the gain is limited I expect.

Note that we have another strange location for a dependency on a
package: the target-root-passwd target, which sets the root password, might depend on the host-mkpasswd package. So while normally, we consider all the target-<foo> targets in system/system.mk as useless for 'make source', 'make external-deps' and so on, it's not really the case for this one. Except that host-mkpasswd in fact has all its source code directly in package/mkpasswd/ (we don't download anything), and it's used purely on the host, so maybe we can ignore this problem?

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#54 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

Quoted 14 lines · Thomas Petazzoni

Dear Arnout Vandecappelle,

Quoted 3 lines · Arnout Vandecappelle

I don't think it's useful to make them packages, because they are
really different. Adding the propagation of -all-source etc. to the dependencies can easily be done in ROOTFS_TARGET_INTERNAL.

Yeah, that's indeed another option. We're already doing it with $(1)-show-depends.

Quoted 2 lines

In a later stage, we could consider to move the (few) common parts into
some common package/rootfs infrastructure, but the gain is limited I expect.

Quoted 8 lines

Note that we have another strange location for a dependency on a
package: the target-root-passwd target, which sets the root password, might depend on the host-mkpasswd package. So while normally, we consider all the target-<foo> targets in system/system.mk as useless for 'make source', 'make external-deps' and so on, it's not really the case for this one. Except that host-mkpasswd in fact has all its source code directly in package/mkpasswd/ (we don't download anything), and it's used purely on the host, so maybe we can ignore this problem?

For now, yes. If we ever refactor the common parts from fs and
pkg-generic, then we can also include the system stuff in there.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#55 lore raw

PATCH v3 19/20 package: package-based implementation of source, external-deps and legal-info

In reply to Thomas Petazzoni
Quoted 11 lines · Thomas Petazzoni

Arnout,

Thanks a lot for your review!

On Thu, 10 Oct 2013 19:24:12 +0200, Arnout Vandecappelle wrote:
[...]

Quoted 4 lines
Quoted 1 line

+PACKAGES_SOURCE:=$(patsubst %,%-all-source,$(PACKAGES) $(BASE_TARGETS))

BASE_TARGETS is just "toolchain", right? And that is already part of
the dependencies of PACKAGES, right? So why is that still needed here?

Quoted 7 lines

Yes, BASE_TARGETS is just toolchain now. However no, "toolchain" is not yet part of the dependencies of all packages. This is something the patch series from Fabio Porcedda is doing, and I don't want to solve all problems in this patch set :)

The main reason I've kept BASE_TARGETS for now is because I knew Fabio would be cleaning up that further with his patch set.

Sure, I will send a rebased patch set when this patch set is merged.

Quoted 1 line

[...]

Regards

Signature · 1 line
Fabio Porcedda
#56 lore raw

PATCH v3 20/20 package: remove useless SHOW_EXTERNAL_DEPS support

In reply to Thomas Petazzoni

Now that a package-based 'make external-deps' support has been added, the DL_MODE=SHOW_EXTERNAL_DEPS mechanism is no longer needed, so some clean up is done in this commit.

---
 package/pkg-download.mk | 44 ++------------------------------------------
 package/pkg-generic.mk  |  5 +----
 2 files changed, 3 insertions(+), 46 deletions(-)
package/pkg-download.mk +2 -42
index 8e4a1ec..208c83c 100644
@@ -18,9 +18,8 @@ SCP := $(call qstrip,$(BR2_SCP)) $(QUIET)
 SSH := $(call qstrip,$(BR2_SSH)) $(QUIET)
 LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
 
-# Default spider mode is 'DOWNLOAD'. Other possible values are 'SOURCE_CHECK'
-# used by the _source-check target and 'SHOW_EXTERNAL_DEPS', used by the
-# external-deps target.
+# Default spider mode is 'DOWNLOAD'. The possible value is
+# 'SOURCE_CHECK' used by the _source-check target.
 DL_MODE=DOWNLOAD
 
 # Override BR2_DL_DIR if shell variable defined
@@ -66,11 +65,6 @@ domainseparator=$(if $(1),$(1),/)
 # The SOURCE_CHECK_* helpers are in charge of simply checking that the source
 # is available for download. This can be used to make sure one will be able
 # to get all the sources needed for one's build configuration.
-#
-# The SHOW_EXTERNAL_DEPS_* helpers simply output to the console the names
-# of the files that will be downloaded, or path and revision of the
-# source repositories, producing a list of all the "external dependencies"
-# of a given build configuration.
 ################################################################################
 
 # Try a shallow clone - but that only works if the version is a ref (tag or
@@ -101,11 +95,6 @@ define SOURCE_CHECK_GIT
   $(GIT) ls-remote --heads $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_GIT
-	echo $($(PKG)_SOURCE)
-endef
-
-
 define DOWNLOAD_BZR
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	$(BZR) export $(DL_DIR)/$($(PKG)_SOURCE) $($(PKG)_SITE) -r $($(PKG)_DL_VERSION)
@@ -115,10 +104,6 @@ define SOURCE_CHECK_BZR
 	$(BZR) ls --quiet $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_BZR
-	echo $($(PKG)_SOURCE)
-endef
-
 define DOWNLOAD_CVS
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
@@ -134,10 +119,6 @@ define SOURCE_CHECK_CVS
 	$(CVS) -d:pserver:anonymous:@$(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) login
 endef
 
-define SHOW_EXTERNAL_DEPS_CVS
-	echo $($(PKG)_SOURCE)
-endef
-
 define DOWNLOAD_SVN
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
@@ -151,10 +132,6 @@ define SOURCE_CHECK_SVN
   $(SVN) ls $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_SVN
-  echo $($(PKG)_SOURCE)
-endef
-
 # SCP URIs should be of the form scp://[user@]host:filepath
 # Note that filepath is relative to the user's home directory, so you may want
 # to prepend the path with a slash: scp://[user@]host:/absolutepath
@@ -167,11 +144,6 @@ define SOURCE_CHECK_SCP
 	$(SSH) $(call domain,$(1),:) ls '$(call notdomain,$(1),:)' > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_SCP
-	echo $(2)
-endef
-
-
 define DOWNLOAD_HG
 	test -e $(DL_DIR)/$($(PKG)_SOURCE) || \
 	(pushd $(DL_DIR) > /dev/null && \
@@ -188,10 +160,6 @@ define SOURCE_CHECK_HG
   $(HG) incoming --force -l1 $($(PKG)_SITE) > /dev/null
 endef
 
-define SHOW_EXTERNAL_DEPS_HG
-  echo $($(PKG)_SOURCE)
-endef
-
 # Download a file using wget. Only download the file if it doesn't
 # already exist in the download directory. If the download fails,
 # remove the file (because wget -O creates a 0-byte file even if the
@@ -209,10 +177,6 @@ define SOURCE_CHECK_WGET
   $(WGET) --spider '$(call qstrip,$(1))'
 endef
 
-define SHOW_EXTERNAL_DEPS_WGET
-  echo $(2)
-endef
-
 define DOWNLOAD_LOCALFILES
 	test -e $(DL_DIR)/$(2) || \
 		$(LOCALFILES) $(call stripurischeme,$(call qstrip,$(1))) $(DL_DIR)
@@ -222,10 +186,6 @@ define SOURCE_CHECK_LOCALFILES
   test -e $(call stripurischeme,$(call qstrip,$(1)))
 endef
 
-define SHOW_EXTERNAL_DEPS_LOCALFILES
-  echo $(2)
-endef
-
 ################################################################################
 # DOWNLOAD -- Download helper. Will try to download source from:
 # 1) BR2_PRIMARY_SITE if enabled
package/pkg-generic.mk +1 -4
index bd6169c..1e09efb 100644
@@ -71,13 +71,10 @@ $(BUILD_DIR)/%/.stamp_rsynced:
 	rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D)
 	$(Q)touch $@
 
-# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
-# packages
+# Handle the SOURCE_CHECK case for rsynced packages
 $(BUILD_DIR)/%/.stamp_rsync_sourced:
 ifeq ($(DL_MODE),SOURCE_CHECK)
 	test -d $(SRCDIR)
-else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS)
-	echo "file://$(SRCDIR)"
 else
 	@true # Nothing to do to source a local package
 endif
Signature · 1 line
1.8.1.2
#57 lore raw

PATCH v3 20/20 package: remove useless SHOW_EXTERNAL_DEPS support

Quoted 5 lines · Thomas Petazzoni

Now that a package-based 'make external-deps' support has been added, the DL_MODE=SHOW_EXTERNAL_DEPS mechanism is no longer needed, so some clean up is done in this commit.

Looks good. Except:

[snip]

Quoted 6 lines
package/pkg-generic.mk +0 -0
index bd6169c..1e09efb 100644
@@ -71,13 +71,10 @@ $(BUILD_DIR)/%/.stamp_rsynced:
  	rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D)

package/pkg-generic.mk $(BUILD_DIR)/%/.stamp_rsynced: · line 80

Quoted 10 lines
package/pkg-generic.mk +1 -4
  	$(Q)touch $@
 
-# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
-# packages
+# Handle the SOURCE_CHECK case for rsynced packages
  $(BUILD_DIR)/%/.stamp_rsync_sourced:
  ifeq ($(DL_MODE),SOURCE_CHECK)
  	test -d $(SRCDIR)
-else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS)
-	echo "file://$(SRCDIR)"

This implies that you did the wrong thing in your previous patch. the
-show-external-deps target should contain a check for _OVERRIDE_SRCDIR.

Regards,
Arnout

Quoted 4 lines
package/pkg-generic.mk +0 -0
  else
  	@true # Nothing to do to source a local package
  endif
Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F