Prototype implementation of per-package out of tree build
Hello,
(Sorry for the previous posting that only had the first two patches. I was in the train in the middle of the mountains, and the 3G connection was interrupted for a too long time.)
There has recently been some interest in improving the support for overriden packages by doing out of tree build of all packages. Since I had the beginning of a prototype working, I cleaned it up, made numerous fixes to make it work apparently properly, and share it with the community.
The following patches should be considered as a prototype, I haven't tested all packages (of course), and there are quite certainly some corner case issues that haven't been solved. However, I think it's a good time to get more testing in order to find, and solve those issues.
It should also not be considered yet that we want this feature in Buildroot. It is again only a prototype, which aims at solving the remaining problems, and see if we think it is a good idea to have this in Buildroot or not.
For those who want to test, the 'out-of-tree' branch is available at:
git://git.free-electrons.com/users/thomas-petazzoni/buildroot.git
Here is a quick summary of what this patch set does:
* The tarballs are now extracted in $(O)/src/<pkg>-<version>
* Each tarball is extracted and patched only once, even if there is both a target build and a host build. Once a package has been extracted and patched, its source code is changed to not have write permissions, in order to catch misbehaving packages in terms of out of tree build.
* The builds continue to be done in $(O)/build/<pkg>-<version> for target packages and $(O)/build/host-<pkg>-<version> for host packages.
* A <pkg>_SUPPORTS_OUT_OF_TREE variable defines whether the package supports out of tree build or not. By default, it's set to NO, in which case the package source code is rsynced from its source directory to the build directory at the beginning of the configure step. It is set to YES by both the autotools and cmake package infrastructures (which have been modified to do out of tree build), and by some packages that use the generic infrastructure but nonetheless support out of tree build.
* The override source directory mechanism is updated to take advantage of this out of tree support. Now, if you override a package that supports out of tree build, the source code is no longer rsynced to the build directory. This makes the override source directory mechanism a lot more usable for big packages such as the Linux kernel.
Amongst the known remaining issues to solve are:
* The semantic of the 'make <pkg>-dirclean' command. Should it remove the build directory and the source directory? Only the build directory? If it removes both (which the proposed prototype currently does), it is strange because you remove one build variant (host or target) and the source directory, but the other build variant (target or host) remains.
* I am not sure on how we want to handle <pkg>-reconfigure and <pkg>-rebuild. I guess we want to tell the user to do his changes in the package source directory, output/src/<pkg>-<version>. But all files in this source directory are marked read-only, so it's not very practical.
In addition to this, for packages that don't support out of tree build, we do a rsync at the beginning of the configure step. So if the user does "make <pkg>-reconfigure", then the source code gets rsynced again from the source directory to the build directory, but not if the user does "make <pkg>-rebuild".
Comments, ideas, suggestions are welcome.
Best regards,
Thomas
RFC 1/14 Add a new "src" directory in the output directory
This new directory will be used to extract the source directory of the different packages, as part of the out of tree support.
Note that we need to explicitly re-add write permissions on the contents of this directory, because write permissions are removed from the source code of packages in order to ensure that the packages are not incorrectly modifying their source directory during an out of tree build. We also re-add write permissions to the build directory since some packages copy files from their source directory to the build directory, causing rm to fail due to lack of permissions.
--- Makefile | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
Makefile +5 -3
@@ -260,6 +260,7 @@ GENERATE_LOCALE=$(call qstrip,$(BR2_GENERATE_LOCALE)) STAMP_DIR:=$(BASE_DIR)/stamps BINARIES_DIR:=$(BASE_DIR)/images +SRC_DIR:=$(BASE_DIR)/src TARGET_DIR:=$(BASE_DIR)/target TOOLCHAIN_DIR=$(BASE_DIR)/toolchain TARGET_SKELETON=$(TOPDIR)/system/skeleton
@@ -383,7 +384,7 @@ TARGETS_LEGAL_INFO:=$(patsubst %,%-legal-info,\ $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) % dirs: $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \ - $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) + $(HOST_DIR) $(BINARIES_DIR) $(SRC_DIR) $(STAMP_DIR) $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
@@ -409,7 +410,7 @@ world: toolchain $(TARGETS_ALL) # dependencies anywhere else # ############################################################# -$(TOOLCHAIN_DIR) $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR): +$(TOOLCHAIN_DIR) $(BUILD_DIR) $(HOST_DIR) $(BINARIES_DIR) $(SRC_DIR) $(STAMP_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR): @mkdir -p $@ $(STAGING_DIR):
@@ -704,9 +705,10 @@ ifeq ($(NEED_WRAPPER),y) endif clean: + -chmod u+w -R $(SRC_DIR)/* $(BUILD_DIR)/* rm -rf $(STAGING_DIR) $(TARGET_DIR) $(BINARIES_DIR) $(HOST_DIR) \ $(STAMP_DIR) $(BUILD_DIR) $(TOOLCHAIN_DIR) $(BASE_DIR)/staging \ - $(LEGAL_INFO_DIR) + $(LEGAL_INFO_DIR) $(SRC_DIR) distclean: clean ifeq ($(DL_DIR),$(TOPDIR)/dl)
Signature · 1 line
1.7.9.5
RFC 1/14 Add a new "src" directory in the output directory
Hello Thomas,
(I repost my remarks there in order to keep all discussion in same thread)
Quoted 21 lines · On Monday 21 January 2013 00:52:10 Thomas Petazzoni wrote:
This new directory will be used to extract the source directory of the different packages, as part of the out of tree support.
Note that we need to explicitly re-add write permissions on the contents of this directory, because write permissions are removed from the source code of packages in order to ensure that the packages are not incorrectly modifying their source directory during an out of tree build. We also re-add write permissions to the build directory since some packages copy files from their source directory to the build directory, causing rm to fail due to lack of permissions.
Signed-off-by Thomas Petazzoni <[email protected]>--- Makefile | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)Makefile +0
-0
Makefile GENERATE_LOCALE=$(call qstrip,$(BR2_GENERATE_LOCALE)) · line 263
Quoted 4 lines
Makefile +1
-0@@ -260,6 +260,7 @@ GENERATE_LOCALE=$(call qstrip,$(BR2_GENERATE_LOCALE)) STAMP_DIR:=$(BASE_DIR)/stamps BINARIES_DIR:=$(BASE_DIR)/images +SRC_DIR:=$(BASE_DIR)/src
IMHO, $(SRC_DIR) should be in $(TOPDIR) to be shared between different configurations. In add $(SRC_DIR) have to customizable in case I would like to clearly separate two projects.
I think, my workspace would look like this:
br/srcs-projet1
br/srcs-projet2
br/output-project1-configA
br/output-project1-configB
br/output-project2-configA
br/output-project2-configB
Finally SRC_DIR should be handled like DL_DIR.
[...]
Signature · 1 line
J?r?me Pouiller
RFC 1/14 Add a new "src" directory in the output directory
Dear J?r?me Pouiller,
Quoted 3 lines · J?r?me Pouiller
IMHO, $(SRC_DIR) should be in $(TOPDIR) to be shared between different configurations. In add $(SRC_DIR) have to customizable in case I would like to clearly separate two projects.
I have indeed thought about making the source code shareable across rebuilds and across projects, but since we apply patches to the source code, it seems a bit complicated to do in a reliable way.
What happens when you bump your Buildroot version and the patches for a given package change? You would have to manually clean up the source directories. Not sure we want to do that, at least for now, but the opinion of others would be interesting here.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
RFC 1/14 Add a new "src" directory in the output directory
Quoted 10 lines · On Monday 21 January 2013 18:16:40 Thomas Petazzoni wrote:
Dear J?r?me Pouiller,
Quoted 3 lines · J?r?me Pouiller
IMHO, $(SRC_DIR) should be in $(TOPDIR) to be shared between different configurations. In add $(SRC_DIR) have to customizable in case I would like to clearly separate two projects.
I have indeed thought about making the source code shareable across rebuilds and across projects, but since we apply patches to the source code, it seems a bit complicated to do in a reliable way.
Hmm... If source is not shared across builds, what is advantage of build out of sources?
I see two advantages:
- No duplication of source between host and target. But finaly, only a few
(and small) package are concerned. Shall we care about this?
- To make build of a package outside of Buildroot easier. In this case, did
you consider to build in a subdirectory of source? It does not allow to share source between host and target but may be sufficient and may solve a bunch of questions.
Quoted 4 lines
What happens when you bump your Buildroot version and the patches for a given package change? You would have to manually clean up the source directories. Not sure we want to do that, at least for now, but the opinion of others would be interesting here.
Currently, when you bump your version of Buildroot, you have to clean all. Buildroot may provide a rule to also cleaning source directory.
Regards,
Signature · 1 line
J?r?me Pouiller
RFC 1/14 Add a new "src" directory in the output directory
Dear J?r?me Pouiller,
Quoted 10 lines · J?r?me Pouiller
Hmm... If source is not shared across builds, what is advantage of build out of sources?
I see two advantages:
- No duplication of source between host and target. But finaly, only a few
(and small) package are concerned. Shall we care about this?
- To make build of a package outside of Buildroot easier. In this case, did
you consider to build in a subdirectory of source? It does not allow to share source between host and target but may be sufficient and may solve a bunch of questions.
This last point is the main reason. For now, the LINUX_OVERRIDE_SRCDIR mechanism is barely usable because it does a complete rsync of the Linux source tree, which is very annoying. So, switching all packages to out of tree build sounds like an interesting solution.
Quoted 6 lines
Quoted 4 lines
What happens when you bump your Buildroot version and the patches for a given package change? You would have to manually clean up the source directories. Not sure we want to do that, at least for now, but the opinion of others would be interesting here.
Currently, when you bump your version of Buildroot, you have to clean all. Buildroot may provide a rule to also cleaning source directory.
In my opinion, it makes the whole thing a lot more complicated to understand. At the moment:
make clean = removes the output directory
That's explained in just one easy line.
If we start putting the source trees outside of the output directory, things get a lot more complicated for new users. They have to understand there is one tree for the build results, one tree for the sources, understand when and why to clean which of the trees and so on. To me, it's not ok.
Remember, simplicity of use is a *key* point of Buildroot. We are very attentive about this.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
RFC 1/14 Add a new "src" directory in the output directory
Hello Thomas,
Quoted 15 lines · On Tuesday 22 January 2013 17:49:02 Thomas Petazzoni wrote:
Dear J?r?me Pouiller,
Quoted 12 lines · J?r?me Pouiller
Hmm... If source is not shared across builds, what is advantage of build out of sources?
I see two advantages:
- No duplication of source between host and target. But finaly, only a few(and small) package are concerned. Shall we care about this?
- To make build of a package outside of Buildroot easier. In this case, did
Quoted 8 lines
Quoted 3 lines
you consider to build in a subdirectory of source? It does not allow to share source between host and target but may be sufficient and may solve a bunch of questions.
This last point is the main reason. For now, the
LINUX_OVERRIDE_SRCDIRmechanism is barely usable because it does a complete rsync of the Linux source tree, which is very annoying. So, switching all packages to out of tree build sounds like an interesting solution.
And what about building in a subdirectory of source directory ?
[...]
Quoted 2 lines
Remember, simplicity of use is a *key* point of Buildroot. We are very attentive about this.
Ok, I share your point of view about simplicity.
Regards,
Signature · 1 line
J?r?me Pouiller
RFC 1/14 Add a new "src" directory in the output directory
Dear J?r?me Pouiller,
Quoted 6 lines · J?r?me Pouiller
Quoted 5 lines
This last point is the main reason. For now, the
LINUX_OVERRIDE_SRCDIRmechanism is barely usable because it does a complete rsync of the Linux source tree, which is very annoying. So, switching all packages to out of tree build sounds like an interesting solution.And what about building in a subdirectory of source directory ?
I am not sure why you think it would solve any problem, but it is anyway not possible for packages for which <pkg>_OVERRIDE_SRCDIR is
used: we certainly do *not* want to build in a subdirectory of the source directory.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
RFC 1/14 Add a new "src" directory in the output directory
Hello Thomas,
Quoted 9 lines · On Wednesday 23 January 2013 16:45:36 Thomas Petazzoni wrote:
Dear J?r?me Pouiller,
Quoted 6 lines · J?r?me Pouiller
Quoted 5 lines
This last point is the main reason. For now, the
LINUX_OVERRIDE_SRCDIRmechanism is barely usable because it does a complete rsync of the Linux source tree, which is very annoying. So, switching all packages to out of tree build sounds like an interesting solution.
Quoted 6 lines
Quoted 1 line
And what about building in a subdirectory of source directory ?
I am not sure why you think it would solve any problem, but it is anyway not possible for packages for which <pkg>_OVERRIDE_SRCDIR is
used: we certainly do *not* want to build in a subdirectory of the source directory.
Mainly because packages not compiling out-of-source won't need to be rsynced between SRCDIR ans BUILDDIR.
Next it may solve questions about of dirclean behaviour.
Finally because I like idea that a directory contains all files related to a package.
Sure, overrided packages have be to handled differently.
But it is only an idea, not sure it is a good idea :-) .
Signature · 1 line
J?r?me Pouiller
RFC 2/14 package infrastructure: move subdir support to autotools
In reply to Thomas PetazzoniThe <pkg>_SUBDIR variable was currently somewhat handled at the generic package infrastructure level, with the $(PKG)_SRCDIR variable taking into account this SUBDIR. But in fact, this <pkg>_SUBDIR variable only makes sense in the context of the autotools infrastructure, to indicate that the configure script is not located at the root of the source tree, but instead in a sub-directory. In the case of generic packages, the package .mk file already has enough freedom to handle bizarre situations where the Makefile or other parts of the build system may not be located at the root of the package source tree.
In order to simplify the introduction of out of tree support, we therefore move the SUBDIR support as an autotools infrastructure specific feature.
--- package/pkg-autotools.mk | 30 ++++++++++++++++++++---------- package/pkg-generic.mk | 11 +---------- 2 files changed, 21 insertions(+), 20 deletions(-)
package/pkg-autotools.mk +20 -10
@@ -70,6 +70,16 @@ ifndef $(2)_MAKE endif endif +ifndef $(2)_SUBDIR + ifdef $(3)_SUBDIR + $(2)_SUBDIR = $$($(3)_SUBDIR) + else + $(2)_SUBDIR ?= + endif +endif + +$(2)_SRCSUBDIR = $$($(2)_SRCDIR)/$($(2)_SUBDIR) + $(2)_CONF_ENV ?= $(2)_CONF_OPT ?= $(2)_MAKE_ENV ?=
@@ -93,7 +103,7 @@ ifeq ($(5),target) # Configure package for target define $(2)_CONFIGURE_CMDS - (cd $$($$(PKG)_SRCDIR) && rm -rf config.cache && \ + (cd $$($$(PKG)_SRCSUBDIR) && rm -rf config.cache && \ $$(TARGET_CONFIGURE_OPTS) \ $$(TARGET_CONFIGURE_ARGS) \ $$($$(PKG)_CONF_ENV) \
@@ -120,7 +130,7 @@ else # because it often relies on host tools which may or may not be # installed. define $(2)_CONFIGURE_CMDS - (cd $$($$(PKG)_SRCDIR) && rm -rf config.cache; \ + (cd $$($$(PKG)_SRCSUBDIR) && rm -rf config.cache; \ $$(HOST_CONFIGURE_OPTS) \ CFLAGS="$$(HOST_CFLAGS)" \ LDFLAGS="$$(HOST_LDFLAGS)" \
@@ -216,11 +226,11 @@ endif ifndef $(2)_BUILD_CMDS ifeq ($(5),target) define $(2)_BUILD_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_SRCDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_SRCSUBDIR) endef else define $(2)_BUILD_CMDS - $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_SRCDIR) + $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_SRCSUBDIR) endef endif endif
@@ -231,7 +241,7 @@ endif # ifndef $(2)_INSTALL_CMDS define $(2)_INSTALL_CMDS - $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -C $$($$(PKG)_SRCDIR) install + $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -C $$($$(PKG)_SRCSUBDIR) install endef endif
@@ -241,7 +251,7 @@ endif # ifndef $(2)_INSTALL_STAGING_CMDS define $(2)_INSTALL_STAGING_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_STAGING_OPT) -C $$($$(PKG)_SRCDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_STAGING_OPT) -C $$($$(PKG)_SRCSUBDIR) for i in $$$$(find $(STAGING_DIR)/usr/lib* -name "*.la"); do \ cp -f $$$$i $$$$i~; \ $$(SED) "s:\(['= ]\)/usr:\\1$(STAGING_DIR)/usr:g" $$$$i; \
@@ -255,7 +265,7 @@ endif # ifndef $(2)_INSTALL_TARGET_CMDS define $(2)_INSTALL_TARGET_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_TARGET_OPT) -C $$($$(PKG)_SRCDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_TARGET_OPT) -C $$($$(PKG)_SRCSUBDIR) endef endif
@@ -265,7 +275,7 @@ endif # ifndef $(2)_CLEAN_CMDS define $(2)_CLEAN_CMDS - -$$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_CLEAN_OPT) -C $$($$(PKG)_SRCDIR) + -$$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_CLEAN_OPT) -C $$($$(PKG)_SRCSUBDIR) endef endif
@@ -275,7 +285,7 @@ endif # ifndef $(2)_UNINSTALL_STAGING_CMDS define $(2)_UNINSTALL_STAGING_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_UNINSTALL_STAGING_OPT) -C $$($$(PKG)_SRCDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_UNINSTALL_STAGING_OPT) -C $$($$(PKG)_SRCSUBDIR) endef endif
@@ -288,7 +298,7 @@ endif # ifndef $(2)_UNINSTALL_TARGET_CMDS define $(2)_UNINSTALL_TARGET_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -k $$($$(PKG)_UNINSTALL_TARGET_OPT) -C $$($$(PKG)_SRCDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -k $$($$(PKG)_UNINSTALL_TARGET_OPT) -C $$($$(PKG)_SRCSUBDIR) endef endif
package/pkg-generic.mk +1 -10
@@ -218,16 +218,7 @@ endif $(2)_BASE_NAME = $(1)-$$($(2)_VERSION) $(2)_DL_DIR = $$(DL_DIR)/$$($(2)_BASE_NAME) $(2)_DIR = $$(BUILD_DIR)/$$($(2)_BASE_NAME) - -ifndef $(2)_SUBDIR - ifdef $(3)_SUBDIR - $(2)_SUBDIR = $$($(3)_SUBDIR) - else - $(2)_SUBDIR ?= - endif -endif - -$(2)_SRCDIR = $$($(2)_DIR)/$$($(2)_SUBDIR) +$(2)_SRCDIR = $$($(2)_DIR) $(2)_BUILDDIR ?= $$($(2)_SRCDIR) ifneq ($$($(2)_OVERRIDE_SRCDIR),)
Signature · 1 line
1.7.9.5
RFC 3/14 generic infrastructure: out of tree build support
In reply to Thomas PetazzoniThis commit adds the out of tree support to the core package infrastructure.
In details:
* The stamp_downloaded, stamp_extracted and stamp_patched are now created in the source directory, since those steps are relative to the source itself, and not to the build process.
* The rsynced and rsync_overrive_sourced steps are removed, as overriden packages are now naturally handled by the package infrastructure.
* In the patch step, at the end, we mark all files in the package source directory as non-writable. This is done to ensure that the out-of-tree build process really doesn't mess up with the source code (which is now shared by the target and host builds).
* In the configure step, if the package doesn't support out of tree build, we rsync its source code into the build directory.
* We now define the $(2)_RAW_BASE_NAME variable in addition to $(2)_BASE_NAME. The former is always "<pkg>-<version>" regarding of whether we're building a target or a host package, while the latter is "<pkg>-<version>" for target packages, and "host-<pkg>-<version>" for host packages. The $(2)_RAW_BASE_NAME variable will be used to name the source directory of packages, since we have only one, shared for both the target and host build.
* $(2)_BUILDDIR is no longer equal to $(2)_SRCDIR, and is now defined to $(BUILD_DIR)/$(2)_BASE_NAME, i.e output/build/(host-)?<pkg>-<version>. $(2)_SRCDIR, in turn, is equal to output/src/<pkg>-<version> if the package has not been overriden, otherwise it is set to the overriden location.
* By default, $(2)_SUPPORTS_OUT_OF_TREE is set to NO, so that the copy of the sources to the build directory is done. It can be overriden on a per-package basis (for example, the Linux, Barebox or U-Boot generic packages support out of tree build) or in some specialized package infrastructure.
* The location of the stamp files is now defined relatively to either the build directory or the source directory. We consider that the download, extract and patch steps are related to the sources, and therefore the stamp file is created in the source directory of the package. The other steps are consider to be build related.
* The default extract command is changed to extract to the source directory.
* The sequencing of steps is slightly modified for various reasons:
- As the autoreconf step will be moved as a post patch hook, we need to have a special handling of the host-autoconf, host-automake and host-libtool dependencies. Instead of depending on them in the $(1)-depends step, we depend on them in the $(1)-patch step. We can't move all dependencies to the $(1)-patch step, otherwise we would break the RTAI packaging. See commit a473a616d27e46951a23d90249a31b08006098bf.
- dirclean is now implemented differently depending on whether the package has an overriden source directory or not. When the package is normal (not overriden), both the source directory and build directory are removed. When the package is overriden, only the build directory is removed.
- the rsync step for overriden packages no longer exists. It is now part of the natural handling of packaging that don't support out of tree build.
* The legal info infrastructure is changed to force copy license files. Since the files in the source directories are read-only, if we run twice the 'make legal-info' command, it fails at copying the license files. Forcing the copy makes it work properly.
--- package/pkg-generic.mk | 128 ++++++++++++++++++++++++++---------------------- package/pkg-utils.mk | 2 +- 2 files changed, 71 insertions(+), 59 deletions(-)
package/pkg-generic.mk +70 -58
@@ -25,7 +25,7 @@ ################################################################################ # Retrieve the archive -$(BUILD_DIR)/%/.stamp_downloaded: +$(SRC_DIR)/%/.stamp_downloaded: ifeq ($(DL_MODE),DOWNLOAD) # Only show the download message if it isn't already downloaded $(Q)if test ! -e $(DL_DIR)/$($(PKG)_SOURCE); then \
@@ -48,7 +48,7 @@ ifeq ($(DL_MODE),DOWNLOAD) endif # Unpack the archive -$(BUILD_DIR)/%/.stamp_extracted: +$(SRC_DIR)/%/.stamp_extracted: @$(call MESSAGE,"Extracting") $(Q)mkdir -p $(@D) $($(PKG)_EXTRACT_CMDS)
@@ -57,32 +57,13 @@ $(BUILD_DIR)/%/.stamp_extracted: $(foreach hook,$($(PKG)_POST_EXTRACT_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ -# Rsync the source directory if the <pkg>_OVERRIDE_SRCDIR feature is -# used. -$(BUILD_DIR)/%/.stamp_rsynced: - @$(call MESSAGE,"Syncing from source dir $(SRCDIR)") - @test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1) - rsync -au $(SRCDIR)/ $(@D) - $(Q)touch $@ - -# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced -# packages -$(BUILD_DIR)/%/.stamp_rsync_sourced: -ifeq ($(DL_MODE),SOURCE_CHECK) - test -d $(SRCDIR) -else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS) - echo "file://$(SRCDIR)" -else - @true # Nothing to do to source a local package -endif - # Patch # # The RAWNAME variable is the lowercased package name, which allows to # find the package directory (typically package/<pkgname>) and the # prefix of the patches -$(BUILD_DIR)/%/.stamp_patched: NAMEVER = $(RAWNAME)-$($(PKG)_VERSION) -$(BUILD_DIR)/%/.stamp_patched: +$(SRC_DIR)/%/.stamp_patched: NAMEVER = $(RAWNAME)-$($(PKG)_VERSION) +$(SRC_DIR)/%/.stamp_patched: @$(call MESSAGE,"Patching $($(PKG)_DIR_PREFIX)/$(RAWNAME)") $(foreach hook,$($(PKG)_PRE_PATCH_HOOKS),$(call $(hook))$(sep)) $(foreach p,$($(PKG)_PATCH),support/scripts/apply-patches.sh $(@D) $(DL_DIR) $(p)$(sep))
@@ -100,9 +81,15 @@ $(BUILD_DIR)/%/.stamp_patched: ) $(foreach hook,$($(PKG)_POST_PATCH_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + chmod a-w -R $(@D) # Configure $(BUILD_DIR)/%/.stamp_configured: + @mkdir -p $(@D) + @if test "$($(PKG)_SUPPORTS_OUT_OF_TREE)" != "YES"; then \ + rsync -au $($(PKG)_SRCDIR)/* $(@D)/ ; \ + chmod u+w -R $(@D)/ ; \ + fi $(foreach hook,$($(PKG)_PRE_CONFIGURE_HOOKS),$(call $(hook))$(sep)) @$(call MESSAGE,"Configuring") $($(PKG)_CONFIGURE_CMDS)
@@ -167,10 +154,26 @@ $(BUILD_DIR)/%/.stamp_uninstalled: $(if $(BR2_INIT_SYSV)$(BR2_INIT_BUSYBOX),\ $($(PKG)_UNINSTALL_INIT_SYSV)) -# Remove package sources +# Remove package build directory $(BUILD_DIR)/%/.stamp_dircleaned: rm -Rf $(@D) +# Remove package source directory +$(SRC_DIR)/%/.stamp_dircleaned: + -chmod u+w -R $(@D) + rm -Rf $(@D) + +# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for overriden +# packages +$(BUILD_DIR)/%/.stamp_override_sourced: +ifeq ($(DL_MODE),SOURCE_CHECK) + test -d $($(PKG)_SRCDIR) +else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS) + echo "file://$($(PKG)_SRCDIR)" +else + @true # Nothing to do to source a local package +endif + ################################################################################ # inner-generic-package -- generates the make targets needed to build a # generic package
@@ -215,11 +218,10 @@ else $(2)_VERSION = $(subst /,_,$($(2)_VERSION)) endif -$(2)_BASE_NAME = $(1)-$$($(2)_VERSION) +$(2)_BASE_NAME = $$($(2)_NAME)-$$($(2)_VERSION) +$(2)_RAW_BASE_NAME = $$($(2)_RAWNAME)-$$($(2)_VERSION) $(2)_DL_DIR = $$(DL_DIR)/$$($(2)_BASE_NAME) -$(2)_DIR = $$(BUILD_DIR)/$$($(2)_BASE_NAME) -$(2)_SRCDIR = $$($(2)_DIR) -$(2)_BUILDDIR ?= $$($(2)_SRCDIR) +$(2)_BUILDDIR = $$(BUILD_DIR)/$$($(2)_BASE_NAME) ifneq ($$($(2)_OVERRIDE_SRCDIR),) $(2)_VERSION = custom
@@ -260,6 +262,12 @@ $(2)_OVERRIDE_SRCDIR = $($(2)_SITE) endif endif +ifeq ($$($(2)_OVERRIDE_SRCDIR),) +$(2)_SRCDIR = $$(SRC_DIR)/$$($(2)_RAW_BASE_NAME) +else +$(2)_SRCDIR = $$($(2)_OVERRIDE_SRCDIR) +endif + ifndef $(2)_LICENSE ifdef $(3)_LICENSE $(2)_LICENSE = $($(3)_LICENSE)
@@ -282,6 +290,13 @@ endif $(2)_REDISTRIBUTE ?= YES +ifndef $(2)_SUPPORTS_OUT_OF_TREE + ifdef $(3)_SUPPORTS_OUT_OF_TREE + $(2)_SUPPORTS_OUT_OF_TREE = $($(3)_SUPPORTS_OUT_OF_TREE) + endif +endif + +$(2)_SUPPORTS_OUT_OF_TREE ?= NO $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
@@ -291,25 +306,25 @@ $(2)_INSTALL_TARGET ?= YES $(2)_DIR_PREFIX = $(if $(4),$(4),$(TOP_SRCDIR)/package) # define sub-target stamps -$(2)_TARGET_INSTALL_TARGET = $$($(2)_DIR)/.stamp_target_installed -$(2)_TARGET_INSTALL_STAGING = $$($(2)_DIR)/.stamp_staging_installed -$(2)_TARGET_INSTALL_IMAGES = $$($(2)_DIR)/.stamp_images_installed -$(2)_TARGET_INSTALL_HOST = $$($(2)_DIR)/.stamp_host_installed -$(2)_TARGET_BUILD = $$($(2)_DIR)/.stamp_built -$(2)_TARGET_CONFIGURE = $$($(2)_DIR)/.stamp_configured -$(2)_TARGET_RSYNC = $$($(2)_DIR)/.stamp_rsynced -$(2)_TARGET_RSYNC_SOURCE = $$($(2)_DIR)/.stamp_rsync_sourced -$(2)_TARGET_PATCH = $$($(2)_DIR)/.stamp_patched -$(2)_TARGET_EXTRACT = $$($(2)_DIR)/.stamp_extracted -$(2)_TARGET_SOURCE = $$($(2)_DIR)/.stamp_downloaded -$(2)_TARGET_UNINSTALL = $$($(2)_DIR)/.stamp_uninstalled -$(2)_TARGET_CLEAN = $$($(2)_DIR)/.stamp_cleaned -$(2)_TARGET_DIRCLEAN = $$($(2)_DIR)/.stamp_dircleaned +$(2)_TARGET_INSTALL_TARGET = $$($(2)_BUILDDIR)/.stamp_target_installed +$(2)_TARGET_INSTALL_STAGING = $$($(2)_BUILDDIR)/.stamp_staging_installed +$(2)_TARGET_INSTALL_IMAGES = $$($(2)_BUILDDIR)/.stamp_images_installed +$(2)_TARGET_INSTALL_HOST = $$($(2)_BUILDDIR)/.stamp_host_installed +$(2)_TARGET_BUILD = $$($(2)_BUILDDIR)/.stamp_built +$(2)_TARGET_CONFIGURE = $$($(2)_BUILDDIR)/.stamp_configured +$(2)_TARGET_PATCH = $$($(2)_SRCDIR)/.stamp_patched +$(2)_TARGET_EXTRACT = $$($(2)_SRCDIR)/.stamp_extracted +$(2)_TARGET_SOURCE = $$($(2)_SRCDIR)/.stamp_downloaded +$(2)_TARGET_OVERRIDE_SOURCE = $$($(2)_BUILDDIR)/.stamp_override_sourced +$(2)_TARGET_UNINSTALL = $$($(2)_BUILDDIR)/.stamp_uninstalled +$(2)_TARGET_CLEAN = $$($(2)_BUILDDIR)/.stamp_cleaned +$(2)_TARGET_SRCDIRCLEAN = $$($(2)_SRCDIR)/.stamp_dircleaned +$(2)_TARGET_BUILDDIRCLEAN = $$($(2)_BUILDDIR)/.stamp_dircleaned # default extract command $(2)_EXTRACT_CMDS ?= \ $$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $(DL_DIR)/$$($(2)_SOURCE) | \ - $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $$($(2)_DIR) $(TAR_OPTIONS) -) + $(TAR) $(TAR_STRIP_COMPONENTS)=1 -C $$($(2)_SRCDIR) $(TAR_OPTIONS) -) # post-steps hooks $(2)_POST_DOWNLOAD_HOOKS ?=
@@ -370,7 +385,8 @@ ifeq ($$($(2)_OVERRIDE_SRCDIR),) $(1)-configure: $(1)-patch $(1)-depends \ $$($(2)_TARGET_CONFIGURE) -$(1)-patch: $(1)-extract $$($(2)_TARGET_PATCH) +$(1)-patch: $(1)-extract $$($(2)_TARGET_PATCH) \ + $(filter host-automake host-autoconf host-libtool,$$($(2)_DEPENDENCIES)) $(1)-extract: $(1)-source \ $$($(2)_TARGET_EXTRACT)
@@ -378,22 +394,24 @@ $(1)-extract: $(1)-source \ $(1)-depends: $$($(2)_DEPENDENCIES) $(1)-source: $$($(2)_TARGET_SOURCE) + +$(1)-dirclean: $$($(2)_TARGET_SRCDIRCLEAN) \ + $$($(2)_TARGET_BUILDDIRCLEAN) else # In the package override case, the sequence of steps -# source, by rsyncing # depends # configure $(1)-configure: $(1)-depends \ $$($(2)_TARGET_CONFIGURE) -$(1)-depends: $(1)-rsync $$($(2)_DEPENDENCIES) +$(1)-depends: $$($(2)_DEPENDENCIES) -$(1)-patch: $(1)-rsync -$(1)-extract: $(1)-rsync +$(1)-patch: +$(1)-extract: -$(1)-rsync: $$($(2)_TARGET_RSYNC) +$(1)-source: $$($(2)_TARGET_OVERRIDE_SOURCE) -$(1)-source: $$($(2)_TARGET_RSYNC_SOURCE) +$(1)-dirclean: $$($(2)_TARGET_BUILDDIRCLEAN) endif $(1)-show-depends:
@@ -404,8 +422,6 @@ $(1)-uninstall: $(1)-configure $$($(2)_TARGET_UNINSTALL) $(1)-clean: $(1)-uninstall \ $$($(2)_TARGET_CLEAN) -$(1)-dirclean: $$($(2)_TARGET_DIRCLEAN) - $(1)-clean-for-rebuild: ifneq ($$($(2)_OVERRIDE_SRCDIR),) rm -f $$($(2)_TARGET_RSYNC)
@@ -431,17 +447,13 @@ $$($(2)_TARGET_INSTALL_IMAGES): PKG=$(2) $$($(2)_TARGET_INSTALL_HOST): PKG=$(2) $$($(2)_TARGET_BUILD): PKG=$(2) $$($(2)_TARGET_CONFIGURE): PKG=$(2) -$$($(2)_TARGET_RSYNC): SRCDIR=$$($(2)_OVERRIDE_SRCDIR) -$$($(2)_TARGET_RSYNC): PKG=$(2) -$$($(2)_TARGET_RSYNC_SOURCE): SRCDIR=$$($(2)_OVERRIDE_SRCDIR) -$$($(2)_TARGET_RSYNC_SOURCE): PKG=$(2) $$($(2)_TARGET_PATCH): PKG=$(2) $$($(2)_TARGET_PATCH): RAWNAME=$(patsubst host-%,%,$(1)) $$($(2)_TARGET_EXTRACT): PKG=$(2) $$($(2)_TARGET_SOURCE): PKG=$(2) +$$($(2)_TARGET_OVERRIDE_SOURCE): PKG=$(2) $$($(2)_TARGET_UNINSTALL): PKG=$(2) $$($(2)_TARGET_CLEAN): PKG=$(2) -$$($(2)_TARGET_DIRCLEAN): PKG=$(2) # Compute the name of the Kconfig option that correspond to the # package being enabled. We handle three cases: the special Linux
@@ -488,7 +500,7 @@ ifeq ($(call qstrip,$$($(2)_LICENSE_FILES)),) @$(call legal-warning-pkg,$$($(2)_RAWNAME),cannot save license ($(2)_LICENSE_FILES not defined)) else @for F in $$($(2)_LICENSE_FILES); do \ - $(call legal-license-file,$$($(2)_RAWNAME),$$$${F},$$($(2)_DIR)/$$$${F}); \ + $(call legal-license-file,$$($(2)_RAWNAME),$$$${F},$$($(2)_SRCDIR)/$$$${F}); \ done endif ifeq ($$($(2)_REDISTRIBUTE),YES)
package/pkg-utils.mk +1 -1
@@ -107,5 +107,5 @@ define legal-license-file # pkg, filename, file-fullpath cat $(3) >>$(LEGAL_LICENSES_TXT) && \ echo >>$(LEGAL_LICENSES_TXT) && \ mkdir -p $(LICENSE_FILES_DIR)/$(1)/ && \ - cp $(3) $(LICENSE_FILES_DIR)/$(1)/ + cp -f $(3) $(LICENSE_FILES_DIR)/$(1)/ endef
Signature · 1 line
1.7.9.5
RFC 3/14 generic infrastructure: out of tree build support
On 01/21/13 00:52, Thomas Petazzoni wrote:
[snip]
Quoted 13 lines
# Patch
#
# The RAWNAME variable is the lowercased package name, which allows to# find the package directory (typically package/<pkgname>) and the # prefix of the patches -$(BUILD_DIR)/%/.stamp_patched: NAMEVER = $(RAWNAME)-$($(PKG)_VERSION) -$(BUILD_DIR)/%/.stamp_patched: +$(SRC_DIR)/%/.stamp_patched: NAMEVER = $(RAWNAME)-$($(PKG)_VERSION) +$(SRC_DIR)/%/.stamp_patched: @$(call MESSAGE,"Patching $($(PKG)_DIR_PREFIX)/$(RAWNAME)") $(foreach hook,$($(PKG)_PRE_PATCH_HOOKS),$(call $(hook))$(sep)) $(foreach p,$($(PKG)_PATCH),support/scripts/apply-patches.sh $(@D) $(DL_DIR) $(p)$(sep))
package/pkg-generic.mk $(BUILD_DIR)/%/.stamp_patched: · line 82
Quoted 2 lines
package/pkg-generic.mk +0
-0@@ -100,9 +81,15 @@ $(BUILD_DIR)/%/.stamp_patched: ) $(foreach hook,$($(PKG)_POST_PATCH_HOOKS),$(call $(hook))$(sep))
Ouch, we forgot something: there may be a HOST_FOO_PRE_PATCH_HOOKS,HOST_FOO_PATCH or HOST_FOO_POST_PATCH_HOOKS that is different from its non-host equivalent... Perhaps there should be a check that they are the same (or the host version is not defined).
package/pkg-generic.mk $(BUILD_DIR)/%/.stamp_patched: · line 84
Quoted 2 lines
package/pkg-generic.mk +1
-0$(Q)touch $@ + chmod a-w -R $(@D)
I would do this before touching the stamp file.
[snip]
package/pkg-generic.mk $(BUILD_DIR)/%/.stamp_uninstalled: · line 175
Quoted 10 lines
package/pkg-generic.mk +10
-0+# Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for overriden +# packages +$(BUILD_DIR)/%/.stamp_override_sourced: +ifeq ($(DL_MODE),SOURCE_CHECK) + test -d $($(PKG)_SRCDIR) +else ifeq ($(DL_MODE),SHOW_EXTERNAL_DEPS) + echo "file://$($(PKG)_SRCDIR)" +else + @true # Nothing to do to source a local package +endif
Why did you move this down? It was better when it was close to the rest
of the source stuff.
package/pkg-generic.mk else · line 222
Quoted 11 lines
package/pkg-generic.mk +3
-1+ ################################################################################ # inner-generic-package -- generates the make targets needed to build a # generic package@@ -215,11 +218,10 @@ else $(2)_VERSION = $(subst /,_,$($(2)_VERSION)) endif -$(2)_BASE_NAME = $(1)-$$($(2)_VERSION) +$(2)_BASE_NAME = $$($(2)_NAME)-$$($(2)_VERSION) +$(2)_RAW_BASE_NAME = $$($(2)_RAWNAME)-$$($(2)_VERSION)
Whitespace seems to be a bit messed up here. Or it could just be the
mixing of tabs and spaces.
package/pkg-generic.mk else · line 220
Quoted 2 lines
package/pkg-generic.mk +0
-1$(2)_DL_DIR = $$(DL_DIR)/$$($(2)_BASE_NAME) -$(2)_DIR = $$(BUILD_DIR)/$$($(2)_BASE_NAME)
Err, $(PKG)_DIR is used in a lot of places: barebox, uboot, linux,
ocf-linux, rtai, xenomai, bash, busybox, host-dbus, dhcp, dmalloc, dnsmasq, fbv, file, hostapd, ... OK I'm tired of typing all this, just do git grep '[A-Z0-9]_DIR\b' -- \*.mk | sed 's/\(TARGET\|HOST\|STAGING\|DL\)_DIR//g' | grep '[A-Z0-9]_DIR\b'
I think these should be fixed up first in separate patches (as far as
possible). It's not always clear whether they should be changed into the SRCDIR or BUILDDIR.
Regards,
Arnout
Quoted 6 lines
package/pkg-generic.mk +1
-2-$(2)_SRCDIR = $$($(2)_DIR) -$(2)_BUILDDIR ?= $$($(2)_SRCDIR) +$(2)_BUILDDIR = $$(BUILD_DIR)/$$($(2)_BASE_NAME) ifneq ($$($(2)_OVERRIDE_SRCDIR),) $(2)_VERSION = custom
[snip]
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 4/14 autotools infrastructure: support out of tree build
In reply to Thomas PetazzoniAll autotools packages are supposed to support out of tree build, so we mark this at the package infrastructure level, and adjust the infrastructure to execute the configure script from the build directory.
--- package/pkg-autotools.mk | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-)
package/pkg-autotools.mk +13 -14
@@ -78,8 +78,6 @@ ifndef $(2)_SUBDIR endif endif -$(2)_SRCSUBDIR = $$($(2)_SRCDIR)/$($(2)_SUBDIR) - $(2)_CONF_ENV ?= $(2)_CONF_OPT ?= $(2)_MAKE_ENV ?=
@@ -91,6 +89,7 @@ $(2)_INSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) install $(2)_CLEAN_OPT ?= clean $(2)_UNINSTALL_STAGING_OPT ?= DESTDIR=$$(STAGING_DIR) uninstall $(2)_UNINSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) uninstall +$(2)_SUPPORTS_OUT_OF_TREE = YES #
@@ -103,11 +102,11 @@ ifeq ($(5),target) # Configure package for target define $(2)_CONFIGURE_CMDS - (cd $$($$(PKG)_SRCSUBDIR) && rm -rf config.cache && \ + (cd $$($$(PKG)_BUILDDIR) && rm -rf config.cache && \ $$(TARGET_CONFIGURE_OPTS) \ $$(TARGET_CONFIGURE_ARGS) \ $$($$(PKG)_CONF_ENV) \ - ./configure \ + $$($$(PKG)_SRCDIR)/$$($$(PKG)_SUBDIR)/configure \ --target=$$(GNU_TARGET_NAME) \ --host=$$(GNU_TARGET_NAME) \ --build=$$(GNU_HOST_NAME) \
@@ -130,12 +129,12 @@ else # because it often relies on host tools which may or may not be # installed. define $(2)_CONFIGURE_CMDS - (cd $$($$(PKG)_SRCSUBDIR) && rm -rf config.cache; \ + (cd $$($$(PKG)_BUILDDIR) && rm -rf config.cache; \ $$(HOST_CONFIGURE_OPTS) \ CFLAGS="$$(HOST_CFLAGS)" \ LDFLAGS="$$(HOST_LDFLAGS)" \ $$($$(PKG)_CONF_ENV) \ - ./configure \ + $$($$(PKG)_SRCDIR)/$$($$(PKG)_SUBDIR)/configure \ --prefix="$$(HOST_DIR)/usr" \ --sysconfdir="$$(HOST_DIR)/etc" \ --enable-shared --disable-static \
@@ -226,11 +225,11 @@ endif ifndef $(2)_BUILD_CMDS ifeq ($(5),target) define $(2)_BUILD_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_SRCSUBDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_BUILDDIR) endef else define $(2)_BUILD_CMDS - $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_SRCSUBDIR) + $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_MAKE_OPT) -C $$($$(PKG)_BUILDDIR) endef endif endif
@@ -241,7 +240,7 @@ endif # ifndef $(2)_INSTALL_CMDS define $(2)_INSTALL_CMDS - $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -C $$($$(PKG)_SRCSUBDIR) install + $$(HOST_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -C $$($$(PKG)_BUILDDIR) install endef endif
@@ -251,7 +250,7 @@ endif # ifndef $(2)_INSTALL_STAGING_CMDS define $(2)_INSTALL_STAGING_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_STAGING_OPT) -C $$($$(PKG)_SRCSUBDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_STAGING_OPT) -C $$($$(PKG)_BUILDDIR) for i in $$$$(find $(STAGING_DIR)/usr/lib* -name "*.la"); do \ cp -f $$$$i $$$$i~; \ $$(SED) "s:\(['= ]\)/usr:\\1$(STAGING_DIR)/usr:g" $$$$i; \
@@ -265,7 +264,7 @@ endif # ifndef $(2)_INSTALL_TARGET_CMDS define $(2)_INSTALL_TARGET_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_TARGET_OPT) -C $$($$(PKG)_SRCSUBDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_INSTALL_TARGET_OPT) -C $$($$(PKG)_BUILDDIR) endef endif
@@ -275,7 +274,7 @@ endif # ifndef $(2)_CLEAN_CMDS define $(2)_CLEAN_CMDS - -$$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_CLEAN_OPT) -C $$($$(PKG)_SRCSUBDIR) + -$$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_CLEAN_OPT) -C $$($$(PKG)_BUILDDIR) endef endif
@@ -285,7 +284,7 @@ endif # ifndef $(2)_UNINSTALL_STAGING_CMDS define $(2)_UNINSTALL_STAGING_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_UNINSTALL_STAGING_OPT) -C $$($$(PKG)_SRCSUBDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) $$($$(PKG)_UNINSTALL_STAGING_OPT) -C $$($$(PKG)_BUILDDIR) endef endif
@@ -298,7 +297,7 @@ endif # ifndef $(2)_UNINSTALL_TARGET_CMDS define $(2)_UNINSTALL_TARGET_CMDS - $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -k $$($$(PKG)_UNINSTALL_TARGET_OPT) -C $$($$(PKG)_SRCSUBDIR) + $$(TARGET_MAKE_ENV) $$($$(PKG)_MAKE_ENV) $$($$(PKG)_MAKE) -k $$($$(PKG)_UNINSTALL_TARGET_OPT) -C $$($$(PKG)_BUILDDIR) endef endif
Signature · 1 line
1.7.9.5
RFC 4/14 autotools infrastructure: support out of tree build
Quoted 24 lines · Thomas Petazzoni
All autotools packages are supposed to support out of tree build, so we mark this at the package infrastructure level, and adjust the infrastructure to execute the configure script from the build directory.
Signed-off-by Thomas Petazzoni<[email protected]>--- package/pkg-autotools.mk | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-)package/pkg-autotools.mk +0
-2@@ -78,8 +78,6 @@ ifndef $(2)_SUBDIR endif endif -$(2)_SRCSUBDIR = $$($(2)_SRCDIR)/$($(2)_SUBDIR) - $(2)_CONF_ENV ?= $(2)_CONF_OPT ?= $(2)_MAKE_ENV ?=
package/pkg-autotools.mk $(2)_INSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) install · line 92
Quoted 4 lines
package/pkg-autotools.mk +1
-0@@ -91,6 +89,7 @@ $(2)_INSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) install $(2)_CLEAN_OPT ?= clean $(2)_UNINSTALL_STAGING_OPT ?= DESTDIR=$$(STAGING_DIR) uninstall $(2)_UNINSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) uninstall +$(2)_SUPPORTS_OUT_OF_TREE = YES
Out of tree build may be broken for some packages, so I would make this
?= YES.
Quoted 7 lines
package/pkg-autotools.mk +0
-0#@@ -103,11 +102,11 @@ ifeq ($(5),target) # Configure package for target define $(2)_CONFIGURE_CMDS
package/pkg-autotools.mk ifeq ($(5),target) · line 112
Quoted 10 lines
package/pkg-autotools.mk +2
-2- (cd $$($$(PKG)_SRCSUBDIR)&& rm -rf config.cache&& \ + (cd $$($$(PKG)_BUILDDIR)&& rm -rf config.cache&& \ $$(TARGET_CONFIGURE_OPTS) \ $$(TARGET_CONFIGURE_ARGS) \ $$($$(PKG)_CONF_ENV) \ - ./configure \ + $$($$(PKG)_SRCDIR)/$$($$(PKG)_SUBDIR)/configure \ --target=$$(GNU_TARGET_NAME) \ --host=$$(GNU_TARGET_NAME) \ --build=$$(GNU_HOST_NAME) \
Hm, not sure I like this. The build system may refer to something
outside the subdir and may get confused by the changed path. So I'd do an mkdir -p of $($(PKG)_BUILDDIR)/$($(PKG)_SUBDIR) and build there.
Regards,
Arnout
[snip]
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 5/14 autotools infrastructure: do the autoreconf as a post patch step
In reply to Thomas PetazzoniDoing the autoreconf step as a pre-configure hook doesn't work properly, because the source directory is read-only during the configure step. And in fact, the autoreconf process modifies the source code, so it is quite logical to do it as part of the patching process rather than the configuration process.
--- package/pkg-autotools.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/pkg-autotools.mk +1 -1
@@ -214,7 +214,7 @@ $(2)_DEPENDENCIES ?= $(filter-out host-automake host-autoconf host-libtool $(1), ifeq ($$($(2)_AUTORECONF),YES) -$(2)_PRE_CONFIGURE_HOOKS += AUTORECONF_HOOK +$(2)_POST_PATCH_HOOKS += AUTORECONF_HOOK $(2)_DEPENDENCIES += host-automake host-autoconf host-libtool endif
Signature · 1 line
1.7.9.5
RFC 5/14 autotools infrastructure: do the autoreconf as a post patch step
Quoted 5 lines · Thomas Petazzoni
Doing the autoreconf step as a pre-configure hook doesn't work properly, because the source directory is read-only during the configure step. And in fact, the autoreconf process modifies the source code, so it is quite logical to do it as part of the patching process rather than the configuration process.
It was in fact moved from POST_PATCH to PRE_CONFIGURE in commit89d1ad91fe4b1b65f0e902f94aba99a6cefbf631, and the PRE_CONFIGURE_HOOKS were added specifically for this purpose. Unfortunately, the commit gives no explanation as to why it was needed.
Regards,
Arnout
Quoted 19 lines
Signed-off-by Thomas Petazzoni<[email protected]>--- package/pkg-autotools.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)package/pkg-autotools.mk +1
-1@@ -214,7 +214,7 @@ $(2)_DEPENDENCIES ?= $(filter-out host-automake host-autoconf host-libtool $(1), ifeq ($$($(2)_AUTORECONF),YES) -$(2)_PRE_CONFIGURE_HOOKS += AUTORECONF_HOOK +$(2)_POST_PATCH_HOOKS += AUTORECONF_HOOK $(2)_DEPENDENCIES += host-automake host-autoconf host-libtool endif
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 5/14 autotools infrastructure: do the autoreconf as a post patch step
Dear Arnout Vandecappelle,
Quoted 11 lines · Arnout Vandecappelle
Quoted 5 lines · Thomas Petazzoni
Doing the autoreconf step as a pre-configure hook doesn't work properly, because the source directory is read-only during the configure step. And in fact, the autoreconf process modifies the source code, so it is quite logical to do it as part of the patching process rather than the configuration process.
It was in fact moved from
POST_PATCHtoPRE_CONFIGUREin commit89d1ad91fe4b1b65f0e902f94aba99a6cefbf631, and thePRE_CONFIGURE_HOOKSwere added specifically for this purpose. Unfortunately, the commit gives no explanation as to why it was needed.
Hum, right. Interesting. I've looked at the commits around this one, and also the e-mails around the one through which this patch was submitted, but it didn't give any clue. The other patches around are version bumps and other, seemingly unrelated things.
It's even more disappointing that I was amongst the people giving a Acked-by on this patch...
One reason that might explain this is the lack of ordering guarantees on post patch hooks. For example, if you have a post patch hook that applies Debian patches, and a post patch hook to do the autoreconf, you quite certainly want the Debian patches hook to be executed before the autoreconf hook. But I have absolutely no idea if this is the problem that we were trying to fix here.
That said, I still believe that the autoreconf thing belongs to the patch step. It is really a modification of the source code itself, and it is common to both the target build and host build. So maybe the autotools infrastructure needs a special hook in the generic infrastructure (rather than a normal post patch hook), to ensure that the autoreconf step gets executed after all post patch hooks?
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
RFC 5/14 autotools infrastructure: do the autoreconf as a post patch step
Quoted 6 lines · Thomas Petazzoni
One reason that might explain this is the lack of ordering guarantees on post patch hooks. For example, if you have a post patch hook that applies Debian patches, and a post patch hook to do the autoreconf, you quite certainly want the Debian patches hook to be executed before the autoreconf hook. But I have absolutely no idea if this is the problem that we were trying to fix here.
That sounds like a likely explanation...
Quoted 3 lines
That said, I still believe that the autoreconf thing belongs to the patch step. It is really a modification of the source code itself, and it is common to both the target build and host build.
I completely agree.
Quoted 4 lines
So maybe the
autotools infrastructure needs a special hook in the generic infrastructure (rather than a normal post patch hook), to ensure that the autoreconf step gets executed after all post patch hooks?
Or maybe (just thinking out loud here) there could be the possibility
for the package infrastructure to add additional steps in the dependency chain. Something like
ifeq ($$($(2)_AUTORECONF),YES)
$(2)_TARGET_AUTORECONFIGURE = $$($(3)_SRCDIR)/.stamp_autoreconfigured $(1)-autoreconfigure: $(1)-patch $$($(2)_TARGET_AUTORECONFIGURE) $(1)-configure: $(1)-autoreconfigure
endif
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
RFC 5/14 autotools infrastructure: do the autoreconf as a post patch step
Dear Arnout Vandecappelle,
Quoted 9 lines · Arnout Vandecappelle
Quoted 4 lines
So maybe the
autotools infrastructure needs a special hook in the generic infrastructure (rather than a normal post patch hook), to ensure that the autoreconf step gets executed after all post patch hooks?Or maybe (just thinking out loud here) there could be the
possibility for the package infrastructure to add additional steps in the dependency chain. Something like
Quoted 5 lines
ifeq ($$($(2)_AUTORECONF),YES)
$(2)_TARGET_AUTORECONFIGURE = $$($(3)_SRCDIR)/.stamp_autoreconfigured $(1)-autoreconfigure: $(1)-patch $$($(2)_TARGET_AUTORECONFIGURE) $(1)-configure: $(1)-autoreconfigure
endif
Sounds interesting. I'll try to experiment around this idea.
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
RFC 6/14 cmake infrastructure: support out of tree build
In reply to Thomas PetazzoniThe cmake infrastructure almost naturally supports out of tree build. The only thing is that the <pkg>_BUILDDIR and <pkg>_SRCDIR variables are now defined by pkg-generic.
--- package/pkg-cmake.mk | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
package/pkg-cmake.mk +1 -3
@@ -46,9 +46,7 @@ $(2)_INSTALL_HOST_OPT ?= install $(2)_INSTALL_STAGING_OPT ?= DESTDIR=$$(STAGING_DIR) install $(2)_INSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) install $(2)_CLEAN_OPT ?= clean - -$(2)_SRCDIR = $$($(2)_DIR)/$($(2)_SUBDIR) -$(2)_BUILDDIR = $$($(2)_SRCDIR) +$(2)_SUPPORTS_OUT_OF_TREE = YES # # Configure step. Only define it if not already defined by the package
Signature · 1 line
1.7.9.5
RFC 7/14 busybox: support out of tree build
In reply to Thomas Petazzoni--- package/busybox/busybox.mk | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
package/busybox/busybox.mk +13 -13
@@ -15,6 +15,8 @@ BUSYBOX_SOURCE = busybox-$(BUSYBOX_VERSION).tar.bz2 BUSYBOX_LICENSE = GPLv2 BUSYBOX_LICENSE_FILES = LICENSE +BUSYBOX_SUPPORTS_OUT_OF_TREE = YES + BUSYBOX_CFLAGS = \ $(TARGET_CFLAGS) \ -I$(LINUX_HEADERS_DIR)/include
@@ -32,7 +34,7 @@ BUSYBOX_CFLAGS += -I$(STAGING_DIR)/usr/include/tirpc/ BUSYBOX_CFLAGS_busybox += -ltirpc endif -BUSYBOX_BUILD_CONFIG = $(BUSYBOX_DIR)/.config +BUSYBOX_BUILD_CONFIG = $(BUSYBOX_BUILDDIR)/.config # Allows the build system to tweak CFLAGS BUSYBOX_MAKE_ENV = \ $(TARGET_MAKE_ENV) \
@@ -173,10 +175,8 @@ define BUSYBOX_INSTALL_WATCHDOG_SCRIPT endef endif -# We do this here to avoid busting a modified .config in configure -BUSYBOX_POST_EXTRACT_HOOKS += BUSYBOX_COPY_CONFIG - define BUSYBOX_CONFIGURE_CMDS + $(BUSYBOX_COPY_CONFIG) $(BUSYBOX_SET_LARGEFILE) $(BUSYBOX_SET_IPV6) $(BUSYBOX_PREFER_STATIC)
@@ -188,15 +188,15 @@ define BUSYBOX_CONFIGURE_CMDS $(BUSYBOX_SET_INIT) $(BUSYBOX_SET_WATCHDOG) @yes "" | $(MAKE) ARCH=$(KERNEL_ARCH) CROSS_COMPILE="$(TARGET_CROSS)" \ - -C $(@D) oldconfig + -C $(BUSYBOX_SRCDIR) O=$(@D) oldconfig endef define BUSYBOX_BUILD_CMDS - $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(@D) + $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(BUSYBOX_SRCDIR) O=$(@D) endef define BUSYBOX_INSTALL_TARGET_CMDS - $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(@D) install + $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(BUSYBOX_SRCDIR) O=$(@D) install if [ ! -f $(TARGET_DIR)/usr/share/udhcpc/default.script ]; then \ $(INSTALL) -m 0755 -D package/busybox/udhcpc.script \ $(TARGET_DIR)/usr/share/udhcpc/default.script; \
@@ -208,20 +208,20 @@ define BUSYBOX_INSTALL_TARGET_CMDS endef define BUSYBOX_UNINSTALL_TARGET_CMDS - $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(@D) uninstall + $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(BUSYBOX_SRCDIR) O=$(@D) uninstall endef define BUSYBOX_CLEAN_CMDS - $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(@D) clean + $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(BUSYBOX_SRCDIR) O=$(@D) clean endef $(eval $(generic-package)) busybox-menuconfig busybox-xconfig busybox-gconfig: busybox-patch - $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(BUSYBOX_DIR) \ - $(subst busybox-,,$@) - rm -f $(BUSYBOX_DIR)/.stamp_built - rm -f $(BUSYBOX_DIR)/.stamp_target_installed + $(BUSYBOX_MAKE_ENV) $(MAKE) $(BUSYBOX_MAKE_OPTS) -C $(BUSYBOX_SRCDIR) \ + O=$(BUSYBOX_BUILDDIR) $(subst busybox-,,$@) + rm -f $(BUSYBOX_BUILDDIR)/.stamp_built + rm -f $(BUSYBOX_BUILDDIR)/.stamp_target_installed busybox-update-config: cp -f $(BUSYBOX_BUILD_CONFIG) $(BUSYBOX_CONFIG_FILE)
Signature · 1 line
1.7.9.5
RFC 8/14 mtd: support out of tree build
In reply to Thomas Petazzoni--- package/mtd/mtd.mk | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
package/mtd/mtd.mk +5 -3
@@ -9,6 +9,8 @@ MTD_SITE = ftp://ftp.infradead.org/pub/mtd-utils MTD_LICENSE = GPLv2 MTD_LICENSE_FILES = COPYING +MTD_SUPPORTS_OUT_OF_TREE = YES + ifeq ($(BR2_PACKAGE_MTD_MKFSJFFS2),y) MTD_DEPENDENCIES = zlib lzo endif
@@ -21,11 +23,11 @@ HOST_MTD_DEPENDENCIES = host-zlib host-lzo host-e2fsprogs define HOST_MTD_BUILD_CMDS $(HOST_CONFIGURE_OPTS) $(MAKE1) \ - CROSS= BUILDDIR=$(@D) WITHOUT_XATTR=1 -C $(@D) + CROSS= BUILDDIR=$(@D) WITHOUT_XATTR=1 -C $(MTD_SRCDIR) endef define HOST_MTD_INSTALL_CMDS - $(MAKE1) BUILDDIR=$(@D) DESTDIR=$(HOST_DIR) -C $(@D) install + $(MAKE1) BUILDDIR=$(@D) DESTDIR=$(HOST_DIR) -C $(MTD_SRCDIR) install endef MKFS_JFFS2 = $(HOST_DIR)/usr/sbin/mkfs.jffs2
@@ -77,7 +79,7 @@ ifneq ($(MTD_TARGETS_y),) define MTD_BUILD_CMDS $(TARGET_CONFIGURE_OPTS) $(MAKE1) CROSS=$(TARGET_CROSS) \ - BUILDDIR=$(@D) WITHOUT_XATTR=1 WITHOUT_LARGEFILE=1 -C $(@D) \ + BUILDDIR=$(@D) WITHOUT_XATTR=1 WITHOUT_LARGEFILE=1 -C $(MTD_SRCDIR) \ $(addprefix $(@D)/,$(MTD_TARGETS_y)) endef
Signature · 1 line
1.7.9.5
RFC 9/14 barebox: support out of tree build
In reply to Thomas PetazzoniFor Barebox, a minor patch is needed to make out-of-tree build possible when the source directory is read-only. This patch can probably be merged upstream.
--- ...-build-when-source-directory-is-read-only.patch | 42 ++++++++++++++++++++ boot/barebox/barebox.mk | 7 ++-- 2 files changed, 46 insertions(+), 3 deletions(-) create mode 100644 boot/barebox/barebox-genenv-fix-build-when-source-directory-is-read-only.patch
boot/barebox/barebox-genenv-fix-build-when-source-directory-is-read-only.patch new file +42 -0
@@ -0,0 +1,42 @@ +From e4bc9ba958c2aba5b2830724457f3a70f014cfc5 Mon Sep 17 00:00:00 2001 +From: Thomas Petazzoni <[email protected]> +Date: Sun, 20 Jan 2013 17:25:55 +0100 +Subject: [PATCH] genenv: fix build when source directory is read-only + +When doing an out-of-tree build where the source directory of Barebox +is read-only (i.e all files have no write permission), the build of +Barebox halts in scripts/genenv, because it removes files for which it +doesn't have write permissions. Those files are being removed from the +temporary directory .barebox_default_env/ once the environnement is +generated. However, since the files have been copied from the source +directory, they carry the permissions without write access. This leads +a regular rm to wait interactively for the user to confirm the +suppression, even those removal is possible since we have write access +to the containing directory. + +To solve this, we explicitly re-add write permissions on the copied +files and directories into this temporary directory. + +Signed-off-by: Thomas Petazzoni <[email protected]> +--- + scripts/genenv | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/scripts/genenv b/scripts/genenv +index 374db6d..3d79c63 100755 +--- a/scripts/genenv ++++ b/scripts/genenv +@@ -24,6 +24,10 @@ for i in $*; do + done + ) + ++# Ensure we have write permissions, in case the source directory is ++# read-only. ++chmod +w -R $tempdir ++ + find $tempdir -name '.svn' -o -name '*~' | xargs --no-run-if-empty rm -r + + $objtree/scripts/bareboxenv -s $tempdir $target +-- +1.7.9.5 +
boot/barebox/barebox.mk +4 -3
@@ -23,6 +23,7 @@ endif BAREBOX_DEPENDENCIES = host-lzop BAREBOX_LICENSE = GPLv2 with exceptions BAREBOX_LICENSE_FILES = COPYING +BAREBOX_SUPPORTS_OUT_OF_TREE = YES ifneq ($(call qstrip,$(BR2_TARGET_BAREBOX_CUSTOM_PATCH_DIR)),) define BAREBOX_APPLY_CUSTOM_PATCHES
@@ -50,14 +51,14 @@ BAREBOX_MAKE_FLAGS = ARCH=$(BAREBOX_ARCH) CROSS_COMPILE="$(CCACHE) $(TARGET_CROS ifeq ($(BR2_TARGET_BAREBOX_USE_DEFCONFIG),y) -BAREBOX_SOURCE_CONFIG = $(@D)/arch/$(BAREBOX_ARCH)/configs/$(call qstrip,$(BR2_TARGET_BAREBOX_BOARD_DEFCONFIG))_defconfig +BAREBOX_SOURCE_CONFIG = $(BAREBOX_SRCDIR)/arch/$(BAREBOX_ARCH)/configs/$(call qstrip,$(BR2_TARGET_BAREBOX_BOARD_DEFCONFIG))_defconfig else ifeq ($(BR2_TARGET_BAREBOX_USE_CUSTOM_CONFIG),y) BAREBOX_SOURCE_CONFIG = $(BR2_TARGET_BAREBOX_CUSTOM_CONFIG_FILE) endif define BAREBOX_CONFIGURE_CMDS - cp $(BAREBOX_SOURCE_CONFIG) $(@D)/arch/$(BAREBOX_ARCH)/configs/buildroot_defconfig - $(TARGET_MAKE_ENV) $(MAKE) $(BAREBOX_MAKE_FLAGS) -C $(@D) buildroot_defconfig + cp $(BAREBOX_SOURCE_CONFIG) $(@D)/.config + yes '' | $(TARGET_MAKE_ENV) $(MAKE) $(BAREBOX_MAKE_FLAGS) -C $(BAREBOX_SRCDIR) O=$(@D) oldconfig endef ifeq ($(BR2_TARGET_BAREBOX_BAREBOXENV),y)
Signature · 1 line
1.7.9.5
RFC 10/14 uboot: support out of tree build
In reply to Thomas Petazzoni--- boot/uboot/uboot.mk | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
boot/uboot/uboot.mk +4 -3
@@ -10,6 +10,7 @@ UBOOT_LICENSE = GPLv2+ UBOOT_LICENSE_FILES = COPYING UBOOT_INSTALL_IMAGES = YES +UBOOT_SUPPORTS_OUT_OF_TREE = YES ifeq ($(UBOOT_VERSION),custom) # Handle custom U-Boot tarballs as specified by the configuration
@@ -71,8 +72,8 @@ UBOOT_POST_PATCH_HOOKS += UBOOT_APPLY_CUSTOM_PATCHES endif define UBOOT_CONFIGURE_CMDS - $(TARGET_CONFIGURE_OPTS) $(UBOOT_CONFIGURE_OPTS) \ - $(MAKE) -C $(@D) $(UBOOT_MAKE_OPTS) \ + $(TARGET_CONFIGURE_OPTS) $(UBOOT_CONFIGURE_OPTS) \ + $(MAKE) -C $(UBOOT_SRCDIR) O=$(@D) $(UBOOT_MAKE_OPTS) \ $(UBOOT_BOARD_NAME)_config @echo >> $(@D)/include/config.h @echo "/* Add a wrapper around the values Buildroot sets. */" >> $(@D)/include/config.h
@@ -91,7 +92,7 @@ endef define UBOOT_BUILD_CMDS $(TARGET_CONFIGURE_OPTS) $(UBOOT_CONFIGURE_OPTS) \ - $(MAKE) -C $(@D) $(UBOOT_MAKE_OPTS) \ + $(MAKE) -C $(UBOOT_SRCDIR) O=$(@D) $(UBOOT_MAKE_OPTS) \ $(UBOOT_MAKE_TARGET) endef
Signature · 1 line
1.7.9.5
RFC 11/14 linux: support out of tree build
In reply to Thomas PetazzoniSupporting out of tree build in the Linux package requires a few changes:
* A clarification of KERNEL_ARCH_PATH, which used to point to $(O)/build/linux-<version>/arch/<architecture>/. Instead, we now have KERNEL_ARCH_SRCDIR (pointing to the architecture directory in the sources) and KERNEL_ARCH_BUILDDIR (pointing to the architectire directory in the build directory)
* The custom Device Tree source file (if any) is now copied through a post-patch hook, rather than during the build step. This is needed because copying a Device Tree source file in the source tree requires the source tree to be writable, which is no longer the case after the patch step. And copying something into the source tree *is* a patching action, so doing this in the patch step looks like the right thing to do. The only consequence of this is that this action will no longer be done if LINUX_OVERRIDE_SRCDIR is defined, because the extract/patch steps are not done on packages for which the source directory is overriden. But it doesn't seem like a big issue since, if the user has overriden the kernel source directory, we assume that (s)he can store the Device Tree source file in the kernel source directory itself.
* We no longer use the trick of creating a buildroot_defconfig, since this requires creating a buildroot_defconfig file in the source tree during the configure step. And it doesn't work with out-of-tree because the source tree is read only during the configure step (and onwards). So instead, we copy the defconfig to .config, and run a silent oldconfig on it.
--- linux/linux.mk | 47 ++++++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 19 deletions(-)
linux/linux.mk +28 -19
@@ -35,6 +35,7 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH)) LINUX_INSTALL_IMAGES = YES LINUX_DEPENDENCIES += host-module-init-tools +LINUX_SUPPORTS_OUT_OF_TREE = YES ifeq ($(BR2_LINUX_KERNEL_UBOOT_IMAGE),y) LINUX_DEPENDENCIES += host-uboot-tools
@@ -106,22 +107,25 @@ endif # for bzImage, arch/i386 and arch/x86_64 do not exist when copying the # defconfig file. ifeq ($(KERNEL_ARCH),i386) -KERNEL_ARCH_PATH=$(LINUX_DIR)/arch/x86 +KERNEL_ARCH_SUBDIR=arch/x86 else ifeq ($(KERNEL_ARCH),x86_64) -KERNEL_ARCH_PATH=$(LINUX_DIR)/arch/x86 +KERNEL_ARCH_SUBDIR=arch/x86 else -KERNEL_ARCH_PATH=$(LINUX_DIR)/arch/$(KERNEL_ARCH) +KERNEL_ARCH_SUBDIR=arch/$(KERNEL_ARCH) endif +KERNEL_ARCH_SRCDIR=$(LINUX_SRCDIR)/$(KERNEL_ARCH_SUBDIR) +KERNEL_ARCH_BUILDDIR=$(LINUX_BUILDDIR)/$(KERNEL_ARCH_SUBDIR) + ifeq ($(BR2_LINUX_KERNEL_VMLINUX),y) -LINUX_IMAGE_PATH=$(LINUX_DIR)/$(LINUX_IMAGE_NAME) +LINUX_IMAGE_PATH=$(LINUX_BUILDDIR)/$(LINUX_IMAGE_NAME) else ifeq ($(BR2_LINUX_KERNEL_VMLINUZ),y) -LINUX_IMAGE_PATH=$(LINUX_DIR)/$(LINUX_IMAGE_NAME) +LINUX_IMAGE_PATH=$(LINUX_BUILDDIR)/$(LINUX_IMAGE_NAME) else ifeq ($(KERNEL_ARCH),avr32) -LINUX_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/images/$(LINUX_IMAGE_NAME) +LINUX_IMAGE_PATH=$(KERNEL_ARCH_BUILDDIR)/boot/images/$(LINUX_IMAGE_NAME) else -LINUX_IMAGE_PATH=$(KERNEL_ARCH_PATH)/boot/$(LINUX_IMAGE_NAME) +LINUX_IMAGE_PATH=$(KERNEL_ARCH_BUILDDIR)/boot/$(LINUX_IMAGE_NAME) endif endif # BR2_LINUX_KERNEL_VMLINUX
@@ -150,15 +154,22 @@ LINUX_POST_PATCH_HOOKS += LINUX_APPLY_PATCHES ifeq ($(BR2_LINUX_KERNEL_USE_DEFCONFIG),y) -KERNEL_SOURCE_CONFIG = $(KERNEL_ARCH_PATH)/configs/$(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG))_defconfig +KERNEL_SOURCE_CONFIG = $(KERNEL_ARCH_SRCDIR)/configs/$(call qstrip,$(BR2_LINUX_KERNEL_DEFCONFIG))_defconfig else ifeq ($(BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG),y) KERNEL_SOURCE_CONFIG = $(BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE) endif +define LINUX_COPY_DTS + cp $(BR2_LINUX_KERNEL_CUSTOM_DTS_PATH) $(KERNEL_ARCH_SRCDIR)/boot/dts/) +endef + +ifneq ($(BR2_LINUX_KERNEL_USE_CUSTOM_DTS),) +LINUX_POST_PATCH_HOOKS += LINUX_COPY_DTS +endif + define LINUX_CONFIGURE_CMDS - cp $(KERNEL_SOURCE_CONFIG) $(KERNEL_ARCH_PATH)/configs/buildroot_defconfig - $(TARGET_MAKE_ENV) $(MAKE1) $(LINUX_MAKE_FLAGS) -C $(@D) buildroot_defconfig - rm $(KERNEL_ARCH_PATH)/configs/buildroot_defconfig + cp $(KERNEL_SOURCE_CONFIG) $(@D)/.config + yes '' | $(TARGET_MAKE_ENV) $(MAKE1) $(LINUX_MAKE_FLAGS) -C $(LINUX_SRCDIR) O=$(@D) oldconfig $(if $(BR2_ARM_EABI), $(call KCONFIG_ENABLE_OPT,CONFIG_AEABI,$(@D)/.config), $(call KCONFIG_DISABLE_OPT,CONFIG_AEABI,$(@D)/.config))
@@ -194,8 +205,8 @@ endef define LINUX_INSTALL_DTB # dtbs moved from arch/$ARCH/boot to arch/$ARCH/boot/dts since 3.8-rc1 cp $(addprefix \ - $(KERNEL_ARCH_PATH)/boot/$(if $(wildcard \ - $(addprefix $(KERNEL_ARCH_PATH)/boot/dts/,$(KERNEL_DTBS))),dts/),$(KERNEL_DTBS)) \ + $(KERNEL_ARCH_BUILDDIR)/boot/$(if $(wildcard \ + $(addprefix $(KERNEL_ARCH_BUILDDIR)/boot/dts/,$(KERNEL_DTBS))),dts/),$(KERNEL_DTBS)) \ $(BINARIES_DIR)/ endef endif
@@ -204,11 +215,11 @@ endif ifeq ($(BR2_LINUX_KERNEL_APPENDED_DTB),y) # dtbs moved from arch/$ARCH/boot to arch/$ARCH/boot/dts since 3.8-rc1 define LINUX_APPEND_DTB - if [ -e $(KERNEL_ARCH_PATH)/boot/$(KERNEL_DTS_NAME).dtb ]; then \ - cat $(KERNEL_ARCH_PATH)/boot/$(KERNEL_DTS_NAME).dtb; \ + if [ -e $(KERNEL_ARCH_BUILDDIR)/boot/$(KERNEL_DTS_NAME).dtb ]; then \ + cat $(KERNEL_ARCH_BUILDDIR)/boot/$(KERNEL_DTS_NAME).dtb; \ else \ - cat $(KERNEL_ARCH_PATH)/boot/dts/$(KERNEL_DTS_NAME).dtb; \ - fi >> $(KERNEL_ARCH_PATH)/boot/zImage + cat $(KERNEL_ARCH_BUILDDIR)/boot/dts/$(KERNEL_DTS_NAME).dtb; \ + fi >> $(KERNEL_ARCH_BUILDDIR)/boot/zImage endef ifeq ($(BR2_LINUX_KERNEL_APPENDED_UIMAGE),y) # We need to generate the uImage here after that so that the uImage is
@@ -220,8 +231,6 @@ endif # Compilation. We make sure the kernel gets rebuilt when the # configuration has changed. define LINUX_BUILD_CMDS - $(if $(BR2_LINUX_KERNEL_USE_CUSTOM_DTS), - cp $(BR2_LINUX_KERNEL_CUSTOM_DTS_PATH) $(KERNEL_ARCH_PATH)/boot/dts/) $(TARGET_MAKE_ENV) $(MAKE) $(LINUX_MAKE_FLAGS) -C $(@D) $(LINUX_IMAGE_TARGET) @if grep -q "CONFIG_MODULES=y" $(@D)/.config; then \ $(TARGET_MAKE_ENV) $(MAKE) $(LINUX_MAKE_FLAGS) -C $(@D) modules ; \
Signature · 1 line
1.7.9.5
RFC 12/14 autoconf: fix out-of-tree build
In reply to Thomas PetazzoniEven though one might think that autoconf should have been the best example of a clean and nice example of a correct out of tree build implementation, this is not the case. The autoconf build process makes modification to the source tree, for some obscure updating process of manual pages.
For now, we workaround this problem by re-adding write permissions on the man/ subdirectory of autoconf sources, but on the long term, a proper, upstreamable solution would be better.
--- package/autoconf/autoconf.mk | 10 ++++++++++ 1 file changed, 10 insertions(+)
package/autoconf/autoconf.mk +10 -0
@@ -20,6 +20,16 @@ HOST_AUTOCONF_CONF_ENV = ac_cv_path_M4=$(HOST_DIR)/usr/bin/m4 \ HOST_AUTOCONF_DEPENDENCIES = host-m4 host-libtool +# The autoconf build process writes to its source directory, which is +# very stupid, but forces us to give write permissions to one +# directory. +define AUTOCONF_FIXUP_PERMISSIONS + chmod u+w -R $(AUTOCONF_SRCDIR)/man +endef + +AUTOCONF_PRE_CONFIGURE_HOOKS += AUTOCONF_FIXUP_PERMISSIONS +HOST_AUTOCONF_PRE_CONFIGURE_HOOKS += AUTOCONF_FIXUP_PERMISSIONS + $(eval $(autotools-package)) $(eval $(host-autotools-package))
Signature · 1 line
1.7.9.5
RFC 13/14 cmake: support out of tree build
In reply to Thomas Petazzoni--- package/cmake/cmake.mk | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
package/cmake/cmake.mk +3 -1
@@ -1,11 +1,13 @@ CMAKE_VERSION = 2.8.9 CMAKE_SITE = http://www.cmake.org/files/v2.8/ +CMAKE_SUPPORTS_OUT_OF_TREE = YES + define HOST_CMAKE_CONFIGURE_CMDS (cd $(@D); \ LDFLAGS="$(HOST_LDFLAGS)" \ CFLAGS="$(HOST_CFLAGS)" \ - ./bootstrap --prefix=$(HOST_DIR)/usr --parallel=$(PARALLEL_JOBS) \ + $(CMAKE_SRCDIR)/bootstrap --prefix=$(HOST_DIR)/usr --parallel=$(PARALLEL_JOBS) \ ) endef
Signature · 1 line
1.7.9.5
RFC 14/14 makedevs: support out of tree build
In reply to Thomas PetazzoniNothing needs to be done, the source is already directly taken from package/makedevs/, so it just works.
--- package/makedevs/makedevs.mk | 2 ++ 1 file changed, 2 insertions(+)
package/makedevs/makedevs.mk +2 -0
@@ -8,6 +8,8 @@ MAKEDEVS_SOURCE = HOST_MAKEDEVS_SOURCE = +MAKEDEVS_SUPPORTS_OUT_OF_TREE = YES + define MAKEDEVS_BUILD_CMDS $(TARGET_CC) $(TARGET_CFLAGS) $(TARGET_LDFLAGS) \ package/makedevs/makedevs.c -o $(@D)/makedevs
Signature · 1 line
1.7.9.5
[Buildroot] [RFC v1] Prototype implementation of per-package out oftree build
In reply to Thomas PetazzoniQuoted 20 lines
-----Original Message-----
From: buildroot-bounces at busybox.net [mailto:buildroot- bounces at busybox.net] On Behalf Of Thomas Petazzoni
Sent: Monday, 21 January 2013 10:52
To: buildroot at uclibc.org
Subject: [Buildroot] [RFC v1] Prototype implementation of per- package out oftree buildHello,
(Sorry for the previous posting that only had the first two patches. I was in the train in the middle of the mountains, and the 3G connection was interrupted for a too long time.)
There has recently been some interest in improving the support for overriden packages by doing out of tree build of all packages. Since I had the beginning of a prototype working, I cleaned it up, made numerous fixes to make it work apparently properly, and share it with the community.
Quoted 4 lines
The following patches should be considered as a prototype, I haven't tested all packages (of course), and there are quite certainly some corner case issues that haven't been solved. However, I think it's a good time to get more testing in order to find, and solve those
issues.
Quoted 63 lines
It should also not be considered yet that we want this feature in Buildroot. It is again only a prototype, which aims at solving the remaining problems, and see if we think it is a good idea to have this in Buildroot or not.
For those who want to test, the 'out-of-tree' branch is available at:
git://git.free-electrons.com/users/thomas-petazzoni/buildroot.gitHere is a quick summary of what this patch set does:
* The tarballs are now extracted in $(O)/src/<pkg>-<version>
* Each tarball is extracted and patched only once, even if there is both a target build and a host build. Once a package has been extracted and patched, its source code is changed to not have write permissions, in order to catch misbehaving packages in terms of out of tree build.
* The builds continue to be done in $(O)/build/<pkg>-<version> for target packages and $(O)/build/host-<pkg>-<version> for host packages.
* A <pkg>_SUPPORTS_OUT_OF_TREE variable defines whether the
package
supports out of tree build or not. By default, it's set to NO, in which case the package source code is rsynced from its source directory to the build directory at the beginning of the configure step. It is set to YES by both the autotools and cmake package infrastructures (which have been modified to do out of tree build), and by some packages that use the generic infrastructure but nonetheless support out of tree build.* The override source directory mechanism is updated to take advantage of this out of tree support. Now, if you override a package that supports out of tree build, the source code is no longer rsynced to the build directory. This makes the override source directory mechanism a lot more usable for big packages such as the Linux kernel.
Amongst the known remaining issues to solve are:
* The semantic of the 'make <pkg>-dirclean' command. Should it
remove
the build directory and the source directory? Only the build directory? If it removes both (which the proposed prototype currently does), it is strange because you remove one build variant (host or target) and the source directory, but the other build variant (target or host) remains.* I am not sure on how we want to handle <pkg>-reconfigure and <pkg>-rebuild. I guess we want to tell the user to do his changes in the package source directory, output/src/<pkg>-<version>. But all files in this source directory are marked read-only, so it's not very practical.
In addition to this, for packages that don't support out of tree build, we do a rsync at the beginning of the configure step. So if the user does "make <pkg>-reconfigure", then the source code gets rsynced again from the source directory to the build directory, but not if the user does "make <pkg>-rebuild".
Quoted 5 lines
Comments, ideas, suggestions are welcome.
Best regards,
Thomas
Hi Thomas,
This looks fairly similar to what I was proposing so I'm personally all for it. Here are a few of my thoughts about it:
* There's one more feature which I think would complete the Linux out-of-tree build support for me, and that is the ability to specify the <PKG>_OVERRIDE_SRCDIR value via the configuration. I've attached a patch for how this might look.
* I like the idea making the source tree read-only to keep the build honest but I think this removes one of the major benefits of having a source tree in the first place, that is being able to modify and play around with it. This leads to the <pkg>-rebuild not making sense as you point out. I think it might be best to simply not enforce this restriction (even if it means some packages could abuse that).
* Do we need a source directory when a package doesn't support out-of-tree building? The downside is additional disk space though that might not be that serious of a concern and it might just come down to a choice of aesthetics I suppose.
* As far as <pkg>-dirclean goes, I would say remove both build and source directories. If we only remove the build directory it will in effect not be all that different to <pkg>-clean. The only other real option would be to have a separate source directory for each target variant.
Cheers,
Przemyslaw Wrzos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 001-linux-external-source-dir.patch
Type: application/octet-stream
Size: 2748 bytes
Desc: 001-linux-external-source-dir.patch
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130121/9c9d6a30/attachment.obj>
[Buildroot] [RFC v1] Prototype implementation of per-package out oftree build
Dear Przemyslaw Wrzos,
Quoted 2 lines · Przemyslaw Wrzos
This looks fairly similar to what I was proposing so I'm personally all for it. Here are a few of my thoughts about it:
Yes, indeed it looks similar, but I had already started working on this prototype some time ago, and I had the feeling that it was already a bit more complete (support in the autotools and cmake infrastructure for out of tree build, etc.).
Quoted 4 lines
* There's one more feature which I think would complete the Linux out-of-tree build support for me, and that is the ability to specify the <PKG>_OVERRIDE_SRCDIR value via the configuration. I've attached a patch for how this might look.
In general, I am not really favorable. You want it for Linux, but then the next guy will want it for Qt, the other guy for X.org server, the other guy for OpenSSH, etc. And then we have gazillions of configuration options to customize the source directory... while we have the much more generic OVERRIDE_SRCDIR mechanism. I really would like to avoid the profusion of such configuration options.
That said, for packages like Linux, U-Boot or Barebox, we already give the possibility of building from a custom Git tree or a custom tarball location, so we could decide that those packages are "special", and we could therefore add the "custom source directory location" option to them.
But I really want to put a limit to the type of packages on which we add such options, because I don't want to see this being added more or less randomly to each and every package in Buildroot.
Quoted 6 lines
* I like the idea making the source tree read-only to keep the build honest but I think this removes one of the major benefits of having a source tree in the first place, that is being able to modify and play around with it. This leads to the <pkg>-rebuild not making sense as you point out. I think it might be best to simply not enforce this restriction (even if it means some packages could abuse that).
Indeed, as I pointed in my introduction e-mail, having the source directory read-only has some drawbacks. However, since the source directory is shared between the host build and target build of a given package, ensuring that the configuration/build steps of the package do not mess with the source code is quite important. Otherwise, we could see cases of pollution of the host build into the target build or vice versa.
Quoted 4 lines
* Do we need a source directory when a package doesn't support out-of-tree building? The downside is additional disk space though that might not be that serious of a concern and it might just come down to a choice of aesthetics I suppose.
The idea with this out-of-tree support is to make the "overriden packages" more "normal". So if you add another special case for packages that don't support out of tree build, the entire goal is a bit lost.
For example, will you tell users that if they want to do a quick and dirty modification of a package source code, they should go in $(O)/src/foo-1.0 if foo supports out of tree build or $(O)/build/foo-1.0 if foo doesn't? Seems really not logical.
In general, I have the feeling that the biggest packages use a reasonable build system, and therefore are capable of doing out of tree build.
Quoted 5 lines
* As far as <pkg>-dirclean goes, I would say remove both build and source directories. If we only remove the build directory it will in effect not be all that different to <pkg>-clean. The only other real option would be to have a separate source directory for each target variant.
The problem is that if you have a package that gets build for the host and the target, you have:
output/src/foo-1.0 output/build/foo-1.0 output/build/host-foo-1.0
Then, you do:
make foo-dirclean
So, output/src/foo-1.0 and output/build/foo-1.0 are removed, but output/build/host-foo-1.0 remains. So we have the build directory for a given package but no longer the corresponding source directory. Doesn't seem to be really nice. No?
Thanks a lot for your very interesting feedback on this, good to see that the discussion is starting!
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
[Buildroot] [RFC v1] Prototype implementation of per-package out oftree build
Am 21.01.2013 18:26, schrieb Thomas Petazzoni:
Quoted 28 lines
Dear Przemyslaw Wrzos,
Quoted 2 lines · Przemyslaw Wrzos
This looks fairly similar to what I was proposing so I'm personally all for it. Here are a few of my thoughts about it:
Yes, indeed it looks similar, but I had already started working on this prototype some time ago, and I had the feeling that it was already a bit more complete (support in the autotools and cmake infrastructure for out of tree build, etc.).
Quoted 4 lines
* There's one more feature which I think would complete the Linux out-of-tree build support for me, and that is the ability to specify the <PKG>_OVERRIDE_SRCDIR value via the configuration. I've attached a patch for how this might look.
In general, I am not really favorable. You want it for Linux, but then the next guy will want it for Qt, the other guy for X.org server, the other guy for OpenSSH, etc. And then we have gazillions of configuration options to customize the source directory... while we have the much more generic
OVERRIDE_SRCDIRmechanism. I really would like to avoid the profusion of such configuration options.That said, for packages like Linux, U-Boot or Barebox, we already give the possibility of building from a custom Git tree or a custom tarball location, so we could decide that those packages are "special", and we could therefore add the "custom source directory location" option to them.
Quoted 3 lines
But I really want to put a limit to the type of packages on which we add such options, because I don't want to see this being added more or less randomly to each and every package in Buildroot.
Once upon a time I thought of such a feature, but I don't think it can be done in a good way. And since we have the OVERRIDE, I do not see any need for it. Maybe even some of the special handlings for Linux, U-Boot and Barebox could be deprecated, but I did not think about this in detail.
Maybe it's worth thinking about extending the OVERRIDE mechanism to override SOURCE and SITE, too.
Quoted 45 lines
Quoted 7 lines
* I like the idea making the source tree read-only to keep the build honest but I think this removes one of the major benefits of having a source tree in the first place, that is being able to modify and play around with it. This leads to the <pkg>-rebuild not making sense as you point out. I think it might be best to simply not enforce this restriction (even if it means some packages could abuse that).
Indeed, as I pointed in my introduction e-mail, having the source directory read-only has some drawbacks. However, since the source directory is shared between the host build and target build of a given package, ensuring that the configuration/build steps of the package do not mess with the source code is quite important. Otherwise, we could see cases of pollution of the host build into the target build or vice versa.
Quoted 4 lines
* Do we need a source directory when a package doesn't support out-of-tree building? The downside is additional disk space though that might not be that serious of a concern and it might just come down to a choice of aesthetics I suppose.
The idea with this out-of-tree support is to make the "overriden packages" more "normal". So if you add another special case for packages that don't support out of tree build, the entire goal is a bit lost.
For example, will you tell users that if they want to do a quick and dirty modification of a package source code, they should go in $(O)/src/foo-1.0 if foo supports out of tree build or $(O)/build/foo-1.0 if foo doesn't? Seems really not logical.
In general, I have the feeling that the biggest packages use a reasonable build system, and therefore are capable of doing out of tree build.
Quoted 5 lines
* As far as <pkg>-dirclean goes, I would say remove both build and source directories. If we only remove the build directory it will in effect not be all that different to <pkg>-clean. The only other real option would be to have a separate source directory for each target variant.
The problem is that if you have a package that gets build for the host and the target, you have:
output/src/foo-1.0 output/build/foo-1.0 output/build/host-foo-1.0
Quoted 8 lines
Then, you do:
make foo-dirclean
So, output/src/foo-1.0 and output/build/foo-1.0 are removed, but output/build/host-foo-1.0 remains. So we have the build directory for a given package but no longer the corresponding source directory. Doesn't seem to be really nice. No?
To me it would seem most natural if output/build/host-foo-1.0 would be removed, too.
Similar issues might occur with foo-reconfigure and friends. If they share one source they would both have to be updated in such a case.
Regards
Stephan
Quoted 7 lines
Thanks a lot for your very interesting feedback on this, good to see that the discussion is starting!
Thanks,
Thomas
Signature · 4 lines
reLinux - Stephan Hoffmann Am Schmidtgrund 124 50765 K?ln Tel. +49.221.95595-19 Fax: -64 www.reLinux.de sho at reLinux.de
On 01/21/13 00:52, Thomas Petazzoni wrote:
[snip]
Quoted 6 lines
* The semantic of the 'make <pkg>-dirclean' command. Should it remove the build directory and the source directory? Only the build directory? If it removes both (which the proposed prototype currently does), it is strange because you remove one build variant (host or target) and the source directory, but the other build variant (target or host) remains.
I think the typical usecase for <pkg>-dirclean is when the patches have
changed. So clearly the source should be removed as well. As mentioned by Stephan, Ideally the host-<pkg> build dir should be removed as well, but I think this is not that important because the newly patched files will have a more recent timestamp so the corresponding host-binaries should be rebuilt automatically.
Quoted 5 lines
* I am not sure on how we want to handle <pkg>-reconfigure and <pkg>-rebuild. I guess we want to tell the user to do his changes in the package source directory, output/src/<pkg>-<version>. But all files in this source directory are marked read-only, so it's not very practical.
First of all, <pkg>-re{configure,build} is often used withOVERRIDE_SRCDIR so there you don't have a problem.
If people do edit directly in the output/src directory, I think it is
_very_ good that they have to 'chmod +w' first, as a reminder that editing in output/src is a bit risky. And once you've chmod'ed it, it stays writable because the chmod a-w is done at the end of the patch step.
So I don't think there's an issue for <pkg>-re{configure,build}.
Quoted 5 lines
In addition to this, for packages that don't support out of tree build, we do a rsync at the beginning of the configure step. So if the user does "make <pkg>-reconfigure", then the source code gets rsynced again from the source directory to the build directory, but not if the user does "make <pkg>-rebuild".
That's not so good. It could be solved by using the same mechanism that
is currently used by OVERRIDE_SRCDIR: add an intermediate <pkg>-rsync target and stamp file, and remove the stamp file in <pkg>-clean-for-re{configure,build}.
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F