buildroot openssl compile error

I am using buildroot to cross-compile an embedded ARM system. Everything else works fine, but I can not get openssl to compile (I am trying to add openssh and it requires openssl and zlib). It can not find zlib.h, although I have zlib1g-dev installed on my build machine (a 386 Ubuntu system).
I have tried manually configuring openssl (./Configure --with-zlib-include=/usr/include), with no success.
Here is the error I get:

making all in crypto/comp...
make[3]: Entering directory `/home/brad/2240/buildroot-2009.02/build_arm/openssl -0.9.8g/crypto/comp'
/opt/arm-linux/usr/bin/arm-linux-gcc -I.. -I../.. -I../../include -fPIC -DOPENSS
L_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLF
CN_H -DTERMIO -O3 -fomit-frame-pointer -Wall -c -o comp_lib.o comp_lib.c
/opt/arm-linux/usr/bin/arm-linux-gcc -I.. -I../.. -I../../include -fPIC -DOPENSS
L_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLF
CN_H -DTERMIO -O3 -fomit-frame-pointer -Wall -c -o comp_err.o comp_err.c
/opt/arm-linux/usr/bin/arm-linux-gcc -I.. -I../.. -I../../include -fPIC -DOPENSS
L_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLF
CN_H -DTERMIO -O3 -fomit-frame-pointer -Wall -c -o c_rle.o c_rle.c
/opt/arm-linux/usr/bin/arm-linux-gcc -I.. -I../.. -I../../include -fPIC -DOPENSS
L_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLF
CN_H -DTERMIO -O3 -fomit-frame-pointer -Wall -c -o c_zlib.o c_zlib.c
c_zlib.c:25:18: error: zlib.h: No such file or directory

Thanks in advance for any help with this.
...Brad

Hi,

Brad> I am using buildroot to cross-compile an embedded ARM Brad> system. Everything else works fine, but I can not get openssl to Brad> compile (I am trying to add openssh and it requires openssl and Brad> zlib). It can not find zlib.h, although I have zlib1g-dev installed on Brad> my build machine (a 386 Ubuntu system).
Brad> I have tried manually configuring openssl (./Configure Brad> --with-zlib-include=/usr/include), with no success.
Brad> Here is the error I get:

Any special reason why you cannot use dropbear?

Is this with an internal or external toolchain?

There's no -I$(STAGING_DIR)/usr/include in the gcc invocation, so that probably fails with an external toolchain.

Brad> /opt/arm-linux/usr/bin/arm-linux-gcc -I.. -I../.. -I../../include Brad> -fPIC -DOPENSS
Brad> L_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN Brad> -DHAVE_DLF
Brad> CN_H -DTERMIO -O3 -fomit-frame-pointer -Wall -c -o c_zlib.o c_zlib.c
Brad> c_zlib.c:25:18: error: zlib.h: No such file or directory

If not, do you have a build_arm/staging_dir/usr/include/zlib.h? Please post your .config

Bye, Peter Korsgaard

Le Mon, 13 Apr 2009 20:03:50 +0200,
Peter Korsgaard <[email protected]> a ?crit :

There's no need for -I$(STAGING_DIR)/usr/include with an external toolchain if --sysroot $(STAGING_DIR) is passed (at least my experiments confirm this). See r25879.

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

Hello Thomas,
I tried to build the latest (4/13/09) svn version of buildroot with my external buildroot toolchain (cross-compiler for arm). I assume the fix described in r25879 is in this version?
I still get an error compiling openssl:

/opt/arm-linux/usr/bin/arm-linux-gcc -I.. -I../.. -I../../include -fPIC -DOPENSS
L_PIC -DZLIB_SHARED -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLF
CN_H -DTERMIO -O3 -fomit-frame-pointer -Wall -c -o c_zlib.o c_zlib.c
c_zlib.c:25:18: error: zlib.h: No such file or directory

I do not see a --sysroot gcc argument. There is a build_arm/staging_dir/usr/include/zlib.h.
Please advise me as to how I can build openssl within buildroot using an external toolchain.

Thanks,
Brad

Hi Brad,

Le Mon, 13 Apr 2009 22:10:39 -0500,
Brad Johnson <[email protected]> a ?crit :

Thanks for the report. I've just run the compilation of openssl to reproduce the issue. I strongly suspect that the following part of openssl.mk is the culprit :

$(OPENSSL_TARGET_BUILD):

        $(MAKE1) CC=$(TARGET_CC) -C $(OPENSSL_DIR) all build-shared
        $(MAKE1) CC=$(TARGET_CC) -C $(OPENSSL_DIR) do_linux-shared
        touch $@

Because by passing CC=$(TARGET_CC) it overrides the value of CC passed in $(TARGET_CONFIGURE_ARGS), which is CC="$(TARGET_CC) $(TARGET_CFLAGS)".

I'm looking into this.

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

Le Tue, 14 Apr 2009 09:09:07 +0200,
Thomas Petazzoni <[email protected]> a ?crit :

Here, using CC="$(TARGET_CC) $(TARGET_CFLAGS)" allows openssl to find zlib.h properly. However, it fails later with make thinking that the -Os and --sysroot options are for him (and not for gcc). The openssl build system looks strange, I don't have the time to look at time now. Maybe tomorrow.

Sincerly,

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

Le Tue, 14 Apr 2009 10:41:34 +0200,
Thomas Petazzoni <[email protected]> a ?crit :

I had a quick look, and the openssl build system is definitely strange. I don't see how one can pass custom CFLAGS to this build system (and this is needed to pass --sysroot $(STAGING_DIR)).

The Configure script usage is :

==================================================================
Usage: Configure [no-<cipher> ...] [enable-<cipher> ...] [experimental-<cipher> ...] [-Dxxx] [-lxxx] [-Lxxx] [-fxxx] [-Kxxx] [no-hw-xxx|no-hw] [[no-]threads] [[no-]shared] [[no-]zlib|zlib-dynamic] [enable-montasm] [no-asm] [no-dso] [no-krb5] [386] [--prefix=DIR] [--openssldir=OPENSSLDIR] [--with-xxx[=vvv]] [--test-sanity] os/compiler[:flags]
==================================================================

I tried to use os/compiler[:flags] to pass flags, but didn't succeed.

On
http://www.therareair.com/2009/01/01/tutorial-how-to-compile-openssl-for-the-iphone/,
where they explain how to compile OpenSSL for the iPhone, they suggest to directly hack the Configure script to add the proper CFLAGS.

Any suggestion on this ?

Thanks!

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

Thomas> Le Tue, 14 Apr 2009 10:41:34 +0200,
Thomas> Thomas Petazzoni <[email protected]> a ?crit :

Thomas> I had a quick look, and the openssl build system is Thomas> definitely strange. I don't see how one can pass custom Thomas> CFLAGS to this build system (and this is needed to pass Thomas> --sysroot $(STAGING_DIR)).

From a quick look it seems to just be a matter of adding CFLAG='$(TARGET_CFLAGS) <extra flags>' on the make command line, where <extra flags> are the flags openssl uses internally (something like -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLCFN -DHAVE_DLFCN_H -DTERMIO).

The question is why is all of a sudden failed? I thought I had seen positive reports with external toolchains after r23458, where openssl was changed to Makefile.autotools.in format. Notice that we used to patch up the Makefile with sed to change the compiler flags.

I'm wondering if it wouldn't be simpler to use a wrapper script around gcc / g++ when using external toolchains, which would add -sysroot and -I$(STAGING_DIR)/include -I$(STAGING_DIR)/usr/include to the argument list instead of all of this.

Bye, Peter Korsgaard

Le Wed, 15 Apr 2009 14:57:38 +0200,
Peter Korsgaard <[email protected]> a ?crit :

How do I know, in a proper way, what these <extra flags> are ? (I mean, without hardcoding them in Buildroot's .mk file)

I have no idea, I've never myself compiled openssl with external toolchain.

I'd say that something like
http://sources.busybox.net/index.py/trunk/buildroot/package/openssl/openssl.mk?revision=22690&view=markup
(before the Makefile.autotools.in conversion) was probably working since the Configure script was patched with the value of $(TARGET_CFLAGS).

What's your suggestion to work around the issue ?

-I$(STAGING_DIR)/include and -I$(STAGING_DIR)/usr/include are not needed when --sysroot $(STAGING) is given.

Moreover, the --sysroot option is just part of $(TARGET_CFLAGS), which might as well contain other flags (march, mtune, mcpu, Os, O2 or whatever), so the .mk should be written so that the build process of all packages really take into account $(TARGET_CFLAGS). And if they do, then there's no point in adding a wrapper script specially for the --sysroot case.

Cheers,

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

Hi,

Thomas> How do I know, in a proper way, what these <extra flags> are ? (I mean, Thomas> without hardcoding them in Buildroot's .mk file)

You don't. Either you do like we used to do and fix the makefile with sed (s/^CFLAG = /CFLAG = -sysroot .../) and live with the fact that openssl likes to compile with -O3 no mather what optimization level you have selected in BR, or you hardcode the list in openssl.mk (it probably doesn't change that often).

Thomas> I have no idea, I've never myself compiled openssl with Thomas> external toolchain.

Ahh ok.

Thomas> What's your suggestion to work around the issue ?

One of the options above (hardcoded list or sed invocation).

Thomas> Moreover, the --sysroot option is just part of Thomas> $(TARGET_CFLAGS), which might as well contain other flags Thomas> (march, mtune, mcpu, Os, O2 or whatever), so the .mk should Thomas> be written so that the build process of all packages really Thomas> take into account $(TARGET_CFLAGS). And if they do, then Thomas> there's no point in adding a wrapper script specially for the Thomas> --sysroot case.

Ok.

Bye, Peter Korsgaard

Ok this patch fixes it for me.
Tested plain snapshot with an external toolchain and it fails, patch and
it's fixed, i suggest Thomas gives it a spin too.
Also moved c_rehash removal to always since it depends on perl and i
doubt anyone uses it.
However there are other failures i've seen with external toolchain support.
First in line was sstrip, it doesn't get built when external toolchain
is on and it's not a standard toolchain tool so it should be strip-only
when using external or build it as usual.
Second is openntpd that doesn't like my toolchain prefix.
And i left it there, it's not like i'm too interested in external
toolchains ;-)
Enjoy.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: buildroot-openssl-external-toolchain.patch
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20090415/1005056b/attachment.diff>

Adding this in $(OPENSSL_TARGET_CONFIGURE) just before the end:
$(SED) "s:-march=[-a-z0-9] ::" -e "s:-mcpu=[-a-z0-9] ::g" $(OPENSSL_DIR)/Makefile
$(SED) "s:-O[0-9]:$(TARGET_CFLAGS):" $(OPENSSL_DIR)/Makefile Gets rid of the "guesstimate" optimizations openssl does and we win on size optimization while at it.

For ARM EABI old size:
1542542 libcrypto.so.0.9.8
296769 libssl.so.0.9.8

New size:
1435414 libcrypto.so.0.9.8
279543 libssl.so.0.9.8

Regards.

Hello Peter,
What I have is a buildroot-built toolchain, but I initially installed the toolchain outside the buildroot tree so it could be shared by other developers (BR2_STAGING_DIR=/opt/arm-linux). I then changed it to use the externally installed toolchain, so other developers could start from the same shared config files and build a root file system without having to rebuild the toolchain every time.
I do have a build_arm/staging_dir/usr/include/zlib.h, but no /opt/arm-linux/usr/include/zlib.h.
And as I explain this, I think I understand that my problem is that when I add new packages (like ssh) to the buildroot config, I probably need to rebuild the externally installed toolchain so the proper header files get installed there? At the time, it seemed to me that it would be capable of finding the internal build_arm/staging_dir/usr/include/zlib.h, but it could be that it now only includes the external usr/include location?
I have attached my .config file.
Thanks,
Brad

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: buildroot.config
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20090413/6964a653/attachment-0001.diff>

Le Sat, 11 Apr 2009 22:46:59 -0500,
Brad Johnson <[email protected]> a ?crit :

This issue has been fixed by r26226.

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com