Skip to contentSkip to conversation

openssl: support building the binary without MMU

PATCH New Benoît Thébaudeau 4 participants 14 messages Jun 19, 2015 – Jul 3, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 openssl: support building the binary without MMU

The commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU" prevented the openssl binary from being built without MMU in order to fix a build failure without fork(). However, openssl is designed to support the lack of fork() with -DHAVE_FORK=0, so allow the openssl binary to be enabled without MMU thanks to this option.

---
 package/openssl/Config.in  | 3 ---
 package/openssl/openssl.mk | 4 ++++
 2 files changed, 4 insertions(+), 3 deletions(-)
package/openssl/Config.in +0 -3
index 07859ca..d147c07 100644
@@ -13,15 +13,12 @@ if BR2_PACKAGE_OPENSSL
 
 config BR2_PACKAGE_OPENSSL_BIN
 	bool "openssl binary"
-	# uses fork()
-	depends on BR2_USE_MMU
 	depends on !BR2_STATIC_LIBS
 	help
 	  Install the openssl binary to the target file system. This is a
 	  command line tool for doing various crypthographic stuff.
 
 comment "openssl binary needs a toolchain w/ dynamic library"
-	depends on BR2_USE_MMU
 	depends on BR2_STATIC_LIBS
 
 config BR2_PACKAGE_OPENSSL_ENGINES
package/openssl/openssl.mk +4 -0
index fd8904d..34a9830 100644
@@ -22,6 +22,10 @@ endef
 OPENSSL_PRE_CONFIGURE_HOOKS += OPENSSL_DISABLE_APPS
 endif
 
+ifeq ($(BR2_USE_MMU),)
+OPENSSL_CFLAGS += -DHAVE_FORK=0
+endif
+
 ifeq ($(BR2_PACKAGE_CRYPTODEV_LINUX),y)
 OPENSSL_CFLAGS += -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS
 OPENSSL_DEPENDENCIES += cryptodev-linux
Signature · 1 line
2.1.4
#1 lore raw

PATCH 2/2 openssl: always build apps

Now that building the openssl binary without MMU is supported, the only reason left for not building apps if the openssl binary is disabled is to save build time. Moreover, the commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU", which added this behavior, had a side effect: the scripts from apps (CA.pl, CA.sh and tsget) and the default configuration file (openssl.cnf) were no longer installed, which is not advertized by the BR2_PACKAGE_OPENSSL_BIN option. CA.pl and CA.sh use the openssl binary, so not installing them without the latter would make sense. But tsget does not use the openssl binary, and openssl.cnf can be used by libcrypto, so it is preferable to handle BR2_PACKAGE_OPENSSL_BIN like before the commit mentioned above, i.e. to always build and install apps and to just remove the openssl binary afterwards if needed, which the current commit does.

---
 package/openssl/openssl.mk | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
package/openssl/openssl.mk +7 -8
index 34a9830..d450f1e 100644
@@ -14,14 +14,6 @@ HOST_OPENSSL_DEPENDENCIES = host-zlib
 OPENSSL_TARGET_ARCH = generic32
 OPENSSL_CFLAGS = $(TARGET_CFLAGS)
 
-ifeq ($(BR2_PACKAGE_OPENSSL_BIN),)
-define OPENSSL_DISABLE_APPS
-	$(SED) '/^build_apps/! s/build_apps//' $(@D)/Makefile.org
-	$(SED) '/^DIRS=/ s/apps//' $(@D)/Makefile.org
-endef
-OPENSSL_PRE_CONFIGURE_HOOKS += OPENSSL_DISABLE_APPS
-endif
-
 ifeq ($(BR2_USE_MMU),)
 OPENSSL_CFLAGS += -DHAVE_FORK=0
 endif
@@ -140,6 +132,13 @@ endef
 OPENSSL_POST_INSTALL_TARGET_HOOKS += OPENSSL_INSTALL_FIXUPS_SHARED
 endif
 
+ifneq ($(BR2_PACKAGE_OPENSSL_BIN),y)
+define OPENSSL_REMOVE_OPENSSL_BIN
+	rm -f $(TARGET_DIR)/usr/bin/openssl
+endef
+OPENSSL_POST_INSTALL_TARGET_HOOKS += OPENSSL_REMOVE_OPENSSL_BIN
+endif
+
 ifneq ($(BR2_PACKAGE_OPENSSL_ENGINES),y)
 define OPENSSL_REMOVE_OPENSSL_ENGINES
 	rm -rf $(TARGET_DIR)/usr/lib/engines
Signature · 1 line
2.1.4
#2 lore raw

PATCH 2/2 openssl: always build apps

Dear Beno?t Th?baudeau,

Quoted 9 lines · Beno?t Th?baudeau

Now that building the openssl binary without MMU is supported, the only reason left for not building apps if the openssl binary is disabled is to save build time. Moreover, the commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU", which added this behavior, had a side effect: the scripts from apps (CA.pl, CA.sh and tsget) and the default configuration file (openssl.cnf) were no longer installed, which is not advertized by the BR2_PACKAGE_OPENSSL_BIN option. CA.pl and CA.sh use the openssl binary, so not installing them without the latter would make sense. But tsget

But that's exactly what you're doing here: CA.pl and CA.sh are now installed, even if the openssl binary is not. Also, all the c_* programs in /etc/ssl/misc/ are installed, they call the openssl tool, but the openssl tool is not installed.

tsget and CA.pl are perl scripts, so they cannot work if you don't have a perl interpreter on the target anyway.

So, maybe we need something like:

ifeq ($(BR2_PACKAGE_PERL),)
define OPENSSL_REMOVE_PERL_SCRIPTS

	$(RM) -f $(TARGET_DIR)/etc/ssl/misc/{CA.pl,tsget}

endef
OPENSSL_POST_INSTALL_TARGET_HOOKS += OPENSSL_REMOVE_PERL_SCRIPTS endif

ifeq ($(BR2_PACKAGE_OPENSSL_BIN),)
define OPENSSL_REMOVE_BIN

	$(RM) -f $(TARGET_DIR)/usr/bin/openssl
	$(RM) -f $(TARGET_DIR)/etc/ssl/misc/{c_*,CA.pl,CA.sh}

endef
OPENSSL_POST_INSTALL_TARGET_HOOKS += OPENSSL_REMOVE_BIN endif

No?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#3 lore raw

PATCH 2/2 openssl: always build apps

Dear Thomas Petazzoni,

Quoted 30 lines · Thomas Petazzoni
Quoted 9 lines · Beno?t Th?baudeau

Now that building the openssl binary without MMU is supported, the only reason left for not building apps if the openssl binary is disabled is to save build time. Moreover, the commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU", which added this behavior, had a side effect: the scripts from apps (CA.pl, CA.sh and tsget) and the default configuration file (openssl.cnf) were no longer installed, which is not advertized by the BR2_PACKAGE_OPENSSL_BIN option. CA.pl and CA.sh use the openssl binary, so not installing them without the latter would make sense. But tsget

But that's exactly what you're doing here: CA.pl and CA.sh are now installed, even if the openssl binary is not. Also, all the c_* programs in /etc/ssl/misc/ are installed, they call the openssl tool, but the openssl tool is not installed.

tsget and CA.pl are perl scripts, so they cannot work if you don't have a perl interpreter on the target anyway.

So, maybe we need something like:

ifeq ($(BR2_PACKAGE_PERL),)
define OPENSSL_REMOVE_PERL_SCRIPTS

	$(RM) -f $(TARGET_DIR)/etc/ssl/misc/{CA.pl,tsget}

endef
OPENSSL_POST_INSTALL_TARGET_HOOKS += OPENSSL_REMOVE_PERL_SCRIPTS endif

ifeq ($(BR2_PACKAGE_OPENSSL_BIN),)
define OPENSSL_REMOVE_BIN

Quoted 7 lines
	$(RM) -f $(TARGET_DIR)/usr/bin/openssl
	$(RM) -f $(TARGET_DIR)/etc/ssl/misc/{c_*,CA.pl,CA.sh}

endef
OPENSSL_POST_INSTALL_TARGET_HOOKS += OPENSSL_REMOVE_BIN endif

No?

Yes. My intent was to keep the rules minimal, but I agree that it is better as you suggest. I will send a v2.

I have a question about the management of the scripts depending on Perl, though. Doing as you suggest hides this behavior in the .mk, so the users won't know that they have a choice just by looking at the configuration. Do you think that it does not really matter, or that a comment or a depends on / select should be added to the Config.in?

Best regards,
Beno?t

#4 lore raw

PATCH 2/2 openssl: always build apps

Dear Beno?t Th?baudeau,

Quoted 2 lines · Beno?t Th?baudeau

Yes. My intent was to keep the rules minimal, but I agree that it is better as you suggest. I will send a v2.

Great, thanks.

Quoted 5 lines

I have a question about the management of the scripts depending on Perl, though. Doing as you suggest hides this behavior in the .mk, so the users won't know that they have a choice just by looking at the configuration. Do you think that it does not really matter, or that a comment or a depends on / select should be added to the Config.in?

We generally try to not clutter menuconfig with too many comments/options: we can't make visible in menuconfig every little possible dependency. That's why we use a lot of "automatic optional dependencies": a package automatically uses another package if it is available, without having this dependency visible from a menuconfig/Config.in point of view.

I think it's a bit the same here. If you know you need that specific Perl script from OpenSSL, then we assume that you are smart enough to realize that you might need to enable a Perl interpreter for your target.

Yes: we do believe our users are smart! :-)

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#5 lore raw

PATCH 2/2 openssl: always build apps

Quoted 15 lines · Thomas Petazzoni

Dear Beno?t Th?baudeau,

Quoted 2 lines · Beno?t Th?baudeau

Yes. My intent was to keep the rules minimal, but I agree that it is better as you suggest. I will send a v2.

Great, thanks.

Quoted 5 lines

I have a question about the management of the scripts depending on Perl, though. Doing as you suggest hides this behavior in the .mk, so the users won't know that they have a choice just by looking at the configuration. Do you think that it does not really matter, or that a comment or a depends on / select should be added to the Config.in?

Quoted 6 lines

We generally try to not clutter menuconfig with too many comments/options: we can't make visible in menuconfig every little possible dependency. That's why we use a lot of "automatic optional dependencies": a package automatically uses another package if it is available, without having this dependency visible from a menuconfig/Config.in point of view.

We (or at least, I) do like to have such a thing mentioned in the help text,
however. The help text does not clutter the menus so there is no reason to be terse there.

Regards,
Arnout

Quoted 12 lines

I think it's a bit the same here. If you know you need that specific Perl script from OpenSSL, then we assume that you are smart enough to realize that you might need to enable a Perl interpreter for your target.

Yes: we do believe our users are smart! :-)

Thanks,

Thomas

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#6 lore raw

PATCH 2/2 openssl: always build apps

Arnout,

Quoted 10 lines · Arnout Vandecappelle
Quoted 6 lines

We generally try to not clutter menuconfig with too many comments/options: we can't make visible in menuconfig every little possible dependency. That's why we use a lot of "automatic optional dependencies": a package automatically uses another package if it is available, without having this dependency visible from a menuconfig/Config.in point of view.

We (or at least, I) do like to have such a thing mentioned in the help text,
however. The help text does not clutter the menus so there is no reason to be terse there.

I am perfectly fine with having more details in the Config.in help text.

However, do we really want to explicitly list all automatic optional dependencies in the Config.in help text? I believe it would definitely be hard to maintain. So the Config.in help text should probably be mostly used for things that aren't clearly visible by reading the .mk file. And automatic optional dependencies are usually pretty explicit by looking at the .mk file.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#7 lore raw

PATCH 2/2 openssl: always build apps

Quoted 15 lines · Thomas Petazzoni

Arnout,

Quoted 10 lines · Arnout Vandecappelle
Quoted 6 lines

We generally try to not clutter menuconfig with too many comments/options: we can't make visible in menuconfig every little possible dependency. That's why we use a lot of "automatic optional dependencies": a package automatically uses another package if it is available, without having this dependency visible from a menuconfig/Config.in point of view.

We (or at least, I) do like to have such a thing mentioned in the help text,
however. The help text does not clutter the menus so there is no reason to be terse there.

Quoted 5 lines

I am perfectly fine with having more details in the Config.in help text.

However, do we really want to explicitly list all automatic optional dependencies in the Config.in help text? I believe it would definitely be hard to maintain.

Why is it hard to maintain? When you add an automatic dependency to the .mk
file, add it to the Config.in as well. No biggie.

That said, I didn't mean that it should necessarily be added for everything.
But certainly for things that are not immediately obvious. E.g. python support obviously requires python, openssl support is also pretty obvious. Although even there I wouldn't mind mentioning it in the help text.

In this case, on the other hand, it's a tool called 'tsget' that needs perl -
not so obvious.

Quoted 4 lines

So the Config.in help text should probably be
mostly used for things that aren't clearly visible by reading the .mk file. And automatic optional dependencies are usually pretty explicit by looking at the .mk file.

Ugh, I wouldn't like it if we expect from our users that they not only read all
the .mk files of the packages they want to use, but also understand what it all means.

Regards,
Arnout

Quoted 5 lines

Best regards,

Thomas

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#8 lore raw

PATCH 2/2 openssl: always build apps

Arnout,

Quoted 6 lines · Arnout Vandecappelle
Quoted 3 lines

However, do we really want to explicitly list all automatic optional dependencies in the Config.in help text? I believe it would definitely be hard to maintain.

Why is it hard to maintain? When you add an automatic dependency to the .mk
file, add it to the Config.in as well. No biggie.

Well, just another thing to keep in sync. If we were to do that, then we could just as well not do any automatic optional dependency, and always have a Config.in sub-option to enable the dependency.

Quoted 7 lines

That said, I didn't mean that it should necessarily be added for everything.
But certainly for things that are not immediately obvious. E.g. python support obviously requires python, openssl support is also pretty obvious. Although even there I wouldn't mind mentioning it in the help text.

In this case, on the other hand, it's a tool called 'tsget' that needs perl -
not so obvious.

True.

Quoted 8 lines
Quoted 4 lines

So the Config.in help text should probably be
mostly used for things that aren't clearly visible by reading the .mk file. And automatic optional dependencies are usually pretty explicit by looking at the .mk file.

Ugh, I wouldn't like it if we expect from our users that they not only read all
the .mk files of the packages they want to use, but also understand what it all means.

But in practice, it's very much the case today, due to these automatic optional dependency handling that we use.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#9 lore raw

PATCH 1/2 openssl: support building the binary without MMU

In reply to Benoît Thébaudeau

Dear Beno?t Th?baudeau,

Quoted 7 lines · Beno?t Th?baudeau

The commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU" prevented the openssl binary from being built without MMU in order to fix a build failure without fork(). However, openssl is designed to support the lack of fork() with -DHAVE_FORK=0, so allow the openssl binary to be enabled without MMU thanks to this option.

Hum. But then it means that we have propagated the MMU dependency to all reverse dependencies of openssl (i.e packages that depend on openssl), but they may no longer need this dependency anymore.

There is a fairly huge list of packages that "select BR2_PACKAGE_OPENSSL". How can we find out which ones will now build fine without MMU support ?

By the way, do you have a specific need for openssl on no-MMU platforms? If so, on which platform? We don't have many no-MMU users, so it's good when they speak up :-)

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#10 lore raw

PATCH 1/2 openssl: support building the binary without MMU

Quoted 9 lines · Thomas Petazzoni

Dear Beno?t Th?baudeau,

Quoted 6 lines · Beno?t Th?baudeau

The commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU" prevented the openssl binary from being built without MMU in order to fix a build failure without fork(). However, openssl is designed to support the lack of fork() with -DHAVE_FORK=0, so allow the openssl binary to be enabled without MMU thanks to this option.

Quoted 5 lines
Quoted 1 line

Hum. But then it means that we have propagated the MMU dependency to all reverse dependencies of openssl (i.e packages that depend on openssl), but they may no longer need this dependency anymore.

This is only about BR2_PACKAGE_OPENSSL_BIN, which is not selected by any package.

Regards,
Arnout

Quoted 13 lines

There is a fairly huge list of packages that "select BR2_PACKAGE_OPENSSL". How can we find out which ones will now build fine without MMU support ?

By the way, do you have a specific need for openssl on no-MMU platforms? If so, on which platform? We don't have many no-MMU users, so it's good when they speak up :-)

Thanks,

Thomas

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#11 lore raw

PATCH 1/2 openssl: support building the binary without MMU

Dear Arnout Vandecappelle, Thomas Petazzoni,

Quoted 10 lines · Arnout Vandecappelle
Quoted 9 lines · Thomas Petazzoni

Dear Beno?t Th?baudeau,

Quoted 6 lines · Beno?t Th?baudeau

The commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU" prevented the openssl binary from being built without MMU in order to fix a build failure without fork(). However, openssl is designed to support the lack of fork() with -DHAVE_FORK=0, so allow the openssl binary to be enabled without MMU thanks to this option.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Hum. But then it means that we have propagated the MMU dependency to all reverse dependencies of openssl (i.e packages that depend on openssl), but they may no longer need this dependency anymore.

This is only about BR2_PACKAGE_OPENSSL_BIN, which is not selected by any package.

It's indeed only about the binary, not about the library.

Quoted 7 lines
Quoted 7 lines

There is a fairly huge list of packages that "select BR2_PACKAGE_OPENSSL". How can we find out which ones will now build fine without MMU support ?

By the way, do you have a specific need for openssl on no-MMU platforms? If so, on which platform? We don't have many no-MMU users, so it's good when they speak up :-)

Actually, no. To make the full story short, I was comparing Buildroot 2011.08 to the current master, and I noticed that openssl.cnf and some other files had disappeared in the built target files (see 2/2), and the investigation led me to the commit 720893b62510438237b9923d744dd079ddb4f67d, which led to 1/2 here.

Best regards,
Beno?t

#12 lore raw

PATCH 1/2 openssl: support building the binary without MMU

In reply to Arnout Vandecappelle

Arnout,

Quoted 5 lines · Arnout Vandecappelle
Quoted 3 lines

Hum. But then it means that we have propagated the MMU dependency to all reverse dependencies of openssl (i.e packages that depend on openssl), but they may no longer need this dependency anymore.

This is only about BR2_PACKAGE_OPENSSL_BIN, which is not selected by any package.

I obviously missed that. Thanks for noticing.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#13 lore raw

PATCH 1/2 openssl: support building the binary without MMU

In reply to Benoît Thébaudeau

Dear Beno?t Th?baudeau,

Quoted 11 lines · Beno?t Th?baudeau

The commit 720893b62510438237b9923d744dd079ddb4f67d "openssl: disable apps for NOMMU" prevented the openssl binary from being built without MMU in order to fix a build failure without fork(). However, openssl is designed to support the lack of fork() with -DHAVE_FORK=0, so allow the openssl binary to be enabled without MMU thanks to this option.

---
 package/openssl/Config.in  | 3 ---
 package/openssl/openssl.mk | 4 ++++
 2 files changed, 4 insertions(+), 3 deletions(-)

Applied, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com