Makefile: unset MAKEFLAGS

MAKEFLAGS can carry options that make Buildroot failed, eg. '-jN'. To avoid this, unset the MAKEFLAGS environment variable.

Dear Samuel Martin,

Quoted 2 lines · Samuel Martin

MAKEFLAGS can carry options that make Buildroot failed, eg. '-jN'. To avoid this, unset the MAKEFLAGS environment variable.

Does unsetting MAKEFLAGS really allows to prevent top-level parallel build on Buildroot?

Or does it only prevents MAKEFLAGS from leaking to the sub-make calls that we do to build packages?

Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux development, consulting, training and support.
http://free-electrons.com

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Samuel" == Samuel Martin <[email protected]> writes:

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Samuel" == Samuel Martin <[email protected]> writes:

Samuel> MAKEFLAGS can carry options that make Buildroot failed, eg. '-jN'. Samuel> To avoid this, unset the MAKEFLAGS environment variable.

Unfortunately this also drops '-s' from child make invocations, making silent builds significantly more noisy.

We have .NOTPARALLEL: in the top level Makefile, isn't that enough to ensure funky MAKEFLAGS settings? Anything else you encountered?

MAKEFLAGS can carry options that make Buildroot failed, eg. '-jN'. To avoid this, unset the MAKEFLAGS environment variable.

Signed-off-by Samuel Martin <[email protected]>

---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

Makefile +0 -0

index 015fbdf..11a7b70 100644

Hi,

Quoted 13 lines · Samuel Martin

MAKEFLAGS can carry options that make Buildroot failed, eg. '-jN'. To avoid this, unset the MAKEFLAGS environment variable.

Signed-off-by Samuel Martin <[email protected]>

---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

Makefile +0 -0

index 015fbdf..11a7b70 100644

Makefile unexport CFLAGS · line 213

Quoted 7 lines

Makefile +1 -0

@@ -207,6 +207,7 @@ unexport CFLAGS
 unexport CXXFLAGS
 unexport GREP_OPTIONS
 unexport CONFIG_SITE
+unexport MAKEFLAGS
 unexport QMAKESPEC
 unexport TERMINFO
 

What is the strategy with respect to cleaning up the user's environment when building buildroot?
Because there are a number of other variables that users can have (and do have) that corrupt the build, for example:

C_INCLUDE_PATH
CPLUS_INCLUDE_PATH
LIBRARY_PATH
LD_LIBRARY_PATH
PERL5LIB
GCC_EXEC_PREFIX

In the twisted environments that I'm working in, I'm unsetting these from a wrapper around buildroot make. However, it seems that there already are a number of cleanups done inside buildroot itself, so it makes sense to add the above variables to the list.

What do you think?

Quoted 41 lines · Thomas De Schampheleire

Hi,

Quoted 13 lines · Samuel Martin

MAKEFLAGS can carry options that make Buildroot failed, eg. '-jN'. To avoid this, unset the MAKEFLAGS environment variable.

Signed-off-by Samuel Martin <[email protected]>

---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

Makefile +0 -0

index 015fbdf..11a7b70 100644

Makefile unexport CFLAGS · line 213

Quoted 7 lines

Makefile +1 -0

@@ -207,6 +207,7 @@ unexport CFLAGS
 unexport CXXFLAGS
 unexport GREP_OPTIONS
 unexport CONFIG_SITE
+unexport MAKEFLAGS
 unexport QMAKESPEC
 unexport TERMINFO
 

What is the strategy with respect to cleaning up the user's environment when building buildroot?
Because there are a number of other variables that users can have (and do have) that corrupt the build, for example:

C_INCLUDE_PATH
CPLUS_INCLUDE_PATH
LIBRARY_PATH
LD_LIBRARY_PATH
PERL5LIB
GCC_EXEC_PREFIX

In the twisted environments that I'm working in, I'm unsetting these from a wrapper around buildroot make. However, it seems that there already are a number of cleanups done inside buildroot itself, so it makes sense to add the above variables to the list.

Quoted 1 line

What do you think?

How about going all the way: clean out the environment completely, and explicitly add env vars to builders as needed. That would result in a more deterministic build environment, IMHO.

How about going all the way: clean out the environment completely, and explicitly add env vars to builders as needed. That would result in a more deterministic build environment, IMHO.

Quoted 3 lines · Bj?rn Forsman

How about going all the way: clean out the environment completely, and explicitly add env vars to builders as needed. That would result in a more deterministic build environment, IMHO.

I don't think this works: there are a bunch of environment variables that we accept (to tune the uClibc config file, the Busybox config file, the host gcc to use, etc.). And we also accept the feature that all BR2_<foo> values can be overridden using environment variables. So I don't think wiping out the entire environment is possible. However, we could /maybe/ (if that's possible), wipe out the entire environment *except* the BR2_<foo> variables, and the few other variables that we explicitly accept. But I'm not sure how to achieve that.

Thomas, All,

Quoted 5 lines · On Thursday 11 July 2013 13:46:46 Thomas Petazzoni wrote:

Quoted 3 lines · Bj?rn Forsman

How about going all the way: clean out the environment completely, and explicitly add env vars to builders as needed. That would result in a more deterministic build environment, IMHO.

Quoted 8 lines

I don't think this works: there are a bunch of environment variables that we accept (to tune the uClibc config file, the Busybox config file, the host gcc to use, etc.). And we also accept the feature that all BR2_<foo> values can be overridden using environment variables. So I don't think wiping out the entire environment is possible. However, we could /maybe/ (if that's possible), wipe out the entire environment *except* the BR2_<foo> variables, and the few other variables that we explicitly accept. But I'm not sure how to achieve that.

Even that would not work. For example, I have LD_PRELOAD set to tsocksify all network connections, without which I could not download anything.

So, even though LD_PRELOAD looks like a good candidate to dump, it can be valid in some cases.

I believe it is much better that Buildroot chokes on a select list of variables, warn about a few others, and accept the rest.

But with this policy, all existing 'unexports' in the Makefile should be removed, and it's up to the user to fix his stuff.
In corporate environments, for one, this is not always feasible. Many developers will build with buildroot as part of an overall build process, and will not know anything about buildroot itself nor its requirements. So either the overall build process should clear the user's environment before starting the buildroot build, or buildroot itself should clean up what it knows is problematic.

Since there already are several variables cleared in that way, I'm more in favor in going down that path and adding some more troublesome variables, rather than removing these bits altogether.

Thomas, All,

Quoted 5 lines · On Thursday 11 July 2013 13:46:46 Thomas Petazzoni wrote:

Quoted 3 lines · Bj?rn Forsman

How about going all the way: clean out the environment completely, and explicitly add env vars to builders as needed. That would result in a more deterministic build environment, IMHO.

Quoted 8 lines

I don't think this works: there are a bunch of environment variables that we accept (to tune the uClibc config file, the Busybox config file, the host gcc to use, etc.). And we also accept the feature that all BR2_<foo> values can be overridden using environment variables. So I don't think wiping out the entire environment is possible. However, we could /maybe/ (if that's possible), wipe out the entire environment *except* the BR2_<foo> variables, and the few other variables that we explicitly accept. But I'm not sure how to achieve that.

Even that would not work. For example, I have LD_PRELOAD set to tsocksify all network connections, without which I could not download anything.

So, even though LD_PRELOAD looks like a good candidate to dump, it can be valid in some cases.

I believe it is much better that Buildroot chokes on a select list of variables, warn about a few others, and accept the rest.

Quoted 2 lines

I believe it is much better that Buildroot chokes on a select list of variables, warn about a few others, and accept the rest.

If you think about it, the downloader is actually a bit different from the builder. At some point Buildroot will (like all other build systems) grow support for checking each downloaded file against checksums specified in the package build file. And once you have that, it doesn't really matter how dirty the environment in the *downloader* is, because it will be guaranteed to match the checksum, or fail. It is only in the *builder* that the environment must be clean.

Are there any reasons for allowing env vars, other than BR2_*, to slip through from host and into the Buildroot *builder* processes? I cannot think of any.

Dear Bj?rn Forsman,

Quoted 10 lines · Bj?rn Forsman

Quoted 2 lines

I believe it is much better that Buildroot chokes on a select list of variables, warn about a few others, and accept the rest.

If you think about it, the downloader is actually a bit different from the builder. At some point Buildroot will (like all other build systems) grow support for checking each downloaded file against checksums specified in the package build file. And once you have that, it doesn't really matter how dirty the environment in the *downloader* is, because it will be guaranteed to match the checksum, or fail. It is only in the *builder* that the environment must be clean.

I am not sure to understand why you separate the download side and the build side here. Both are done with Buildroot makefiles, there is nothing different in terms of environment variables between those steps.

Quoted 3 lines

Are there any reasons for allowing env vars, other than BR2_*, to slip through from host and into the Buildroot *builder* processes? I cannot think of any.

Yes, we allow passing HOSTCC, UCLIBC_CONFIG_FILE, BUSYBOX_CONFIG_FILE, and a bunch of other environment variables (see the Buildroot manual).

Also, some environment variables like LD_LIBRARY_PATH or LD_PRELOAD should just be usable by the user, if he has some funky setup that requires those environment variables to hold special values.

What is the strategy with respect to cleaning up the user's environment when building buildroot?
Because there are a number of other variables that users can have (and do have) that corrupt the build, for example:

C_INCLUDE_PATH
CPLUS_INCLUDE_PATH
LIBRARY_PATH
LD_LIBRARY_PATH
PERL5LIB
GCC_EXEC_PREFIX

In the twisted environments that I'm working in, I'm unsetting these from a wrapper around buildroot make. However, it seems that there already are a number of cleanups done inside buildroot itself, so it makes sense to add the above variables to the list.

What do you think?

Dear Thomas De Schampheleire,

Quoted 17 lines · Thomas De Schampheleire

What is the strategy with respect to cleaning up the user's environment when building buildroot?
Because there are a number of other variables that users can have (and do have) that corrupt the build, for example:

C_INCLUDE_PATH
CPLUS_INCLUDE_PATH
LIBRARY_PATH
LD_LIBRARY_PATH
PERL5LIB
GCC_EXEC_PREFIX

In the twisted environments that I'm working in, I'm unsetting these from a wrapper around buildroot make. However, it seems that there already are a number of cleanups done inside buildroot itself, so it makes sense to add the above variables to the list.

Quoted 1 line

What do you think?

We have are a bit inconsistent on this. Some variables are unset from the main Makefile, and a bunch of others are checked in support/dependencies/dependencies.sh. For example, LD_LIBRARY_PATH which you mentioned get checked in support/dependencies/dependencies.sh.

Looks like a bit of cleanup in this area might be useful.

dependencies.sh does not complain on the existence of LD_LIBRARY_PATH in the environment, only on the presence of the current working directory inside of it. But of course, if we decide to unexport this variable, then checking its contents is no longer needed. Is there any valid use case for someone setting LD_LIBRARY_PATH during a buildroot build?

For the check on the presence of the current working dir in PATH, we could question whether it makes sense to raise a message and exit, or to manipulate PATH during the build and continue.

Another variable checked in dependencies.sh is PERL_MM_OPT. This seems a good candidate to add to the existing list in Makefile.

Is there any particular reason why some unexports are done always, and some are done inside a check on BR2_HAVE_DOT_CONFIG ?