libopenssl: bump version to 1.1.1a

- remove all parallel build patches (openssl build-system changed)

- rebased 0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch to apply to Configurations/unix-Makefile.tmpl (Makefile template)

- removed 0002-cryptodev-Fix-issue-with-signature-generation.patch (upstream applied)

- rebased 0003-Reproducible-build-do-not-leak-compiler-path.patch to apply to crypto/build.info (Makefile template)

- fix musl/uclibc build failure, use '-DOPENSSL_NO_ASYNC'

- remove legacy enable-tlsext configure option

- fix host library install path

- change legacy INSTALL_PREFIX to DESTDIR

- remove 'libraries gets installed read only, so strip fails' workaround (not needed anymore)

- change engine directory from /usr/lib/engines to /usr/lib/engines-1.1

- change license file hash, no license change, only the following hint was removed:

Actually both licenses are BSD-style Open Source licenses. In case of any license issues related to OpenSSL please contact openssl-core at openssl.org.

- fix host-libopenssl compile setting rpath as decribed in libopenssl-1.1.0h/NOTES.UNIX

Signed-off-by Peter Seiderer <[email protected]>

---
Changes v3 -> v4:
- bump version to 1.1.1a
- remove all parallel build patches hash file entries
- re-remove 0004-Revert-util-dofile.pl-only-quote-stuff-that-actually.patch (upstream applied)
- fix hist library install path
- removed 0002-cryptodev-Fix-issue-with-signature-generation.patch (upstram applied)
- remove follow up patch for openssh (not longer needed since version bump to 7.9p1, see https://www.openssh.com/releasenotes.html
Portability)

Changes v2 -> v3:
- no changes

Changes v1 -> v2:
- add OPENSSL_NO_ASYNC workaround for musl compile too (suggested by Bernd Kuhls)

- fix host-libopenssl compile (reported by Ryan Coe) by setting rpath (suggested by Ryan Coe)

- fix 0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch and 0003-Reproducible-build-do-not-leak-compiler-path.patch to apply to the Makefile templates (instead of re-generated Makefile) (reported by Ryan Coe)

- add 0004-Revert-util-dofile.pl-only-quote-stuff-that-actually.patch (suggested by Bernd Kuhls)

Notes:

- There was a previous attempt to bump the openssl version by David Mosberger <[email protected]>. I could not find the corresponding patch in patchwork or on the mailing list, only a reply by Arnout Vandecappelle (see [2]) and the answer by David Mosberger (see [3]).

- Compile checked packages (depending explicit on libopenssl or host-libopenssl):
O.k:
- hostapd
- libpjsip
- mosquitto
- wpa_supplicant

Failure:
- softether/host-softether

- Compile checked packages (depending on openssl or host-openssl):
O.k.:
- alljoyn-base
- apr
- apr-util
- freeswitch
- openssh

Failure:
- android-tools
- apache (CMake configure errro, unrelated?)

[2] http://lists.busybox.net/pipermail/buildroot/2017-August/200859.html
[3] http://lists.busybox.net/pipermail/buildroot/2017-August/200898.html

---
  ...building-manpages-if-we-re-not-going.patch |  34 +-
  ...-Fix-issue-with-signature-generation.patch | 450 ------------------
  ...ible-build-do-not-leak-compiler-path.patch |  31 +-
  package/libopenssl/libopenssl.hash            |  15 +-
  package/libopenssl/libopenssl.mk              |  41 +-
  5 files changed, 61 insertions(+), 510 deletions(-)

delete mode 100644 package/libopenssl/0002-cryptodev-Fix-issue-with-signature-generation.patch

I'm getting a build failure in wget with the patch.? The failure is:

checking for library containing psl_builtin... no
configure: WARNING: *** libpsl was not found. Fallback to builtin cookie checking. checking for ZLIB... yes
checking for OPENSSL... no
configure: error: Package requirements (openssl) were not met:

Package 'openssl', required by 'virtual:world', not found

Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix.

Alternatively, you may set the environment variables OPENSSL_CFLAGS and OPENSSL_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
make: *** [package/pkg-generic.mk:223:
/home/ryan/devel/buildroot/output/build/wget-1.20.1/.stamp_configured] Error 1

The following tweak makes things build again for me:

package/libopenssl/libopenssl.mk +1 -1

index e461e08126..8046995720 100644
@@ -74,7 +74,7 @@ define HOST_LIBOPENSSL_CONFIGURE_CMDS
 ???? ??? ./config \
 ???? ??? --prefix=$(HOST_DIR) \
 ???? ??? --openssldir=$(HOST_DIR)/etc/ssl \
-??? ??? --libdir=$(HOST_DIR)/lib \
+??? ??? --libdir=lib \
 ???? ??? -Wl,-rpath,'$(HOST_DIR)/lib' \
 ???? ??? shared \
 ???? ??? zlib-dynamic \

Quoted 36 lines · Ryan Coe

I'm getting a build failure in wget with the patch.? The failure is:

checking for library containing psl_builtin... no
configure: WARNING: *** libpsl was not found. Fallback to builtin cookie checking. checking for ZLIB... yes
checking for OPENSSL... no
configure: error: Package requirements (openssl) were not met:

Package 'openssl', required by 'virtual:world', not found

Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix.

Alternatively, you may set the environment variables OPENSSL_CFLAGS and OPENSSL_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
make: *** [package/pkg-generic.mk:223:
/home/ryan/devel/buildroot/output/build/wget-1.20.1/.stamp_configured] Error 1

The following tweak makes things build again for me:

package/libopenssl/libopenssl.mk +1 -1

index e461e08126..8046995720 100644
@@ -74,7 +74,7 @@ define HOST_LIBOPENSSL_CONFIGURE_CMDS
 ???? ??? ./config \
 ???? ??? --prefix=$(HOST_DIR) \
 ???? ??? --openssldir=$(HOST_DIR)/etc/ssl \
-??? ??? --libdir=$(HOST_DIR)/lib \
+??? ??? --libdir=lib \
 ???? ??? -Wl,-rpath,'$(HOST_DIR)/lib' \
 ???? ??? shared \
 ???? ??? zlib-dynamic \

package/libopenssl/libopenssl.mk define LIBOPENSSL_CONFIGURE_CMDS · line 96

Quoted 8 lines

package/libopenssl/libopenssl.mk +1 -1

@@ -90,7 +90,7 @@ define LIBOPENSSL_CONFIGURE_CMDS
 ???? ??? ??? linux-$(LIBOPENSSL_TARGET_ARCH) \
 ???? ??? ??? --prefix=/usr \
 ???? ??? ??? --openssldir=/etc/ssl \
-??? ??? ??? --libdir=/lib \
+??? ??? ??? --libdir=lib \
 ???? ??? ??? $(if $(BR2_TOOLCHAIN_HAS_THREADS),threads,no-threads) \
 ???? ??? ??? $(if $(BR2_STATIC_LIBS),no-shared,shared) \
 ???? ??? ??? no-rc5 \  

That is definitely not good, because it breaks things in other places.

I think it's worth a try to remove the --libdir entirely instead, upstream is
supposed to have fixed the libdir handling to correspond to autotools conventions. And if it doesn't work, we should fix it with an upstreamable patch.

Regards,
Arnout

- remove all parallel build patches (openssl build-system changed)

- rebased 0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch to apply to Configurations/unix-Makefile.tmpl (Makefile template)

- removed 0002-cryptodev-Fix-issue-with-signature-generation.patch (upstream applied)

- rebased 0003-Reproducible-build-do-not-leak-compiler-path.patch to apply to crypto/build.info (Makefile template)

- fix musl/uclibc build failure, use '-DOPENSSL_NO_ASYNC'

- remove legacy enable-tlsext configure option

- fix host library install path

- change legacy INSTALL_PREFIX to DESTDIR

- remove 'libraries gets installed read only, so strip fails' workaround (not needed anymore)

- change engine directory from /usr/lib/engines to /usr/lib/engines-1.1

- change license file hash, no license change, only the following hint was removed:

Actually both licenses are BSD-style Open Source licenses. In case of any license issues related to OpenSSL please contact openssl-core at openssl.org.

- fix host-libopenssl compile setting rpath as decribed in libopenssl-1.1.0h/NOTES.UNIX

Signed-off-by Peter Seiderer <[email protected]>

---
Changes v3 -> v4:
- bump version to 1.1.1a
- remove all parallel build patches hash file entries
- re-remove 0004-Revert-util-dofile.pl-only-quote-stuff-that-actually.patch (upstream applied)
- fix hist library install path
- removed 0002-cryptodev-Fix-issue-with-signature-generation.patch (upstram applied)
- remove follow up patch for openssh (not longer needed since version bump to 7.9p1, see https://www.openssh.com/releasenotes.html
Portability)

Changes v2 -> v3:
- no changes

Changes v1 -> v2:
- add OPENSSL_NO_ASYNC workaround for musl compile too (suggested by Bernd Kuhls)

- fix host-libopenssl compile (reported by Ryan Coe) by setting rpath (suggested by Ryan Coe)

- fix 0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch and 0003-Reproducible-build-do-not-leak-compiler-path.patch to apply to the Makefile templates (instead of re-generated Makefile) (reported by Ryan Coe)

- add 0004-Revert-util-dofile.pl-only-quote-stuff-that-actually.patch (suggested by Bernd Kuhls)

Notes:

- There was a previous attempt to bump the openssl version by David Mosberger <[email protected]>. I could not find the corresponding patch in patchwork or on the mailing list, only a reply by Arnout Vandecappelle (see [2]) and the answer by David Mosberger (see [3]).

- Compile checked packages (depending explicit on libopenssl or host-libopenssl):
O.k:
- hostapd
- libpjsip
- mosquitto
- wpa_supplicant

Failure:
- softether/host-softether

- Compile checked packages (depending on openssl or host-openssl):
O.k.:
- alljoyn-base
- apr
- apr-util
- freeswitch
- openssh

Failure:
- android-tools
- apache (CMake configure errro, unrelated?)

[2] http://lists.busybox.net/pipermail/buildroot/2017-August/200859.html
[3] http://lists.busybox.net/pipermail/buildroot/2017-August/200898.html
---

Quoted 23 lines

- Compile checked packages (depending explicit on libopenssl or host-libopenssl):
O.k:
- hostapd
- libpjsip
- mosquitto
- wpa_supplicant

Failure:
- softether/host-softether

- Compile checked packages (depending on openssl or host-openssl):
O.k.:
- alljoyn-base
- apr
- apr-util
- freeswitch
- openssh

Failure:
- android-tools
- apache (CMake configure errro, unrelated?)

Quoted 3 lines

[2] http://lists.busybox.net/pipermail/buildroot/2017-August/200859.html
[3] http://lists.busybox.net/pipermail/buildroot/2017-August/200898.html
---

Just some worries, I was trying to port openssl 1.1.x for one of the project and there were some issues with API compatibility for example for openssh (and mey be some other like curl, wget), it was an year ago

and may be mostly the packages switched to openssl 1.1.x API but it is not better to make openssl 1.1.x as one of the openssl provider to do not break other packages ?

Best regards,
Vadim Kochan
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Hello Vadim,

On Wed, 16 Jan 2019 15:58:21 +0200, Vadim Kochan <[email protected]> wrote:

[...]

Quoted 23 lines

Quoted 23 lines

- Compile checked packages (depending explicit on libopenssl or host-libopenssl):
O.k:
- hostapd
- libpjsip
- mosquitto
- wpa_supplicant

Failure:
- softether/host-softether

- Compile checked packages (depending on openssl or host-openssl):
O.k.:
- alljoyn-base
- apr
- apr-util
- freeswitch
- openssh

Failure:
- android-tools
- apache (CMake configure errro, unrelated?)

Quoted 7 lines

Quoted 3 lines

[2] http://lists.busybox.net/pipermail/buildroot/2017-August/200859.html
[3] http://lists.busybox.net/pipermail/buildroot/2017-August/200898.html
---

Just some worries, I was trying to port openssl 1.1.x for one of the project and there were some issues with API compatibility for example for openssh (and mey be some other like curl, wget), it was an year ago

Openssh and wget are fixed (by update) already, did not yet test curl...but I think the situation improves as some major linux distributions did already the openssl update (or work on the update)...

Quoted 3 lines

and may be mostly the packages switched to openssl 1.1.x API but it is not better to make openssl 1.1.x as one of the openssl provider to do not break other packages ?

Maybe the way to go in case to much packages will fail with openssl 1.1.x...

Identified ~200 buidlroot packages to (optional) depend on openssl...

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

At some point, we'll just have to let an autobuilder detect the issues. Some
issues anyway only become apparent at link time, so when building a dependency of a dependency...

We have an autobuilder running on an openssl-bump branch and Patrick is fixing
the issues that come out. But at some point we'll need to just bite the bullet and accept the whole thing.

I do hope that it could still go into 2019.02.

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

Hi,

Quoted 7 lines

At some point, we'll just have to let an autobuilder detect the issues. Some
issues anyway only become apparent at link time, so when building a dependency of a dependency...

We have an autobuilder running on an openssl-bump branch and Patrick is fixing
the issues that come out. But at some point we'll need to just bite the bullet and accept the whole thing.

Quoted 1 line

I do hope that it could still go into 2019.02.

If that is our goal, then we might as well merge it ASAP to get more test coverage?

What is the killer feature why we need openssl-1.1.x in 2019.02.x? TLS 1.3 or just that 1.0.2 goes EOL at the end of the year?

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

Quoted 7 lines

Quoted 1 line

I do hope that it could still go into 2019.02.

If that is our goal, then we might as well merge it ASAP to get more test coverage?

What is the killer feature why we need openssl-1.1.x in 2019.02.x? TLS 1.3 or just that 1.0.2 goes EOL at the end of the year?

Both, I guess. Also that the whole known world (including Debian stable,
released June 2017) has already switched to 1.1.x.

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

Hi,

Quoted 5 lines

Quoted 5 lines

Quoted 1 line

I do hope that it could still go into 2019.02.

If that is our goal, then we might as well merge it ASAP to get more test coverage?

Quoted 5 lines

Quoted 2 lines

What is the killer feature why we need openssl-1.1.x in 2019.02.x? TLS 1.3 or just that 1.0.2 goes EOL at the end of the year?

Both, I guess. Also that the whole known world (including Debian stable,
released June 2017) has already switched to 1.1.x.

With 1.1.x I take it you mean 1.1.0, right? The 1.1.1 series (strange numbering convention) was only released back in September.

1.1.0 is not a LTS so not suitable for 2019.02.x. 1.0.2 is supported until the end of 2019, so that is IMHO still OK for 2019.02.

I agree that having 1.1.1x in 2019.02.x would be nice, but a pity that quite some packages break with it (even if the whole known world has switched to it).

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

Quoted 5 lines

Quoted 2 lines

Both, I guess. Also that the whole known world (including Debian stable,
released June 2017) has already switched to 1.1.x.

With 1.1.x I take it you mean 1.1.0, right? The 1.1.1 series (strange numbering convention) was only released back in September.

With 1.1.x I mean 1.1.0 or 1.1.1. The breakage comes from the 1.1.0 bump; .0 to
.1 is fairly smooth.

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Arnout" == Arnout Vandecappelle <[email protected]> writes:

Hi,

Quoted 5 lines

Quoted 5 lines

Quoted 2 lines

Both, I guess. Also that the whole known world (including Debian stable,
released June 2017) has already switched to 1.1.x.

With 1.1.x I take it you mean 1.1.0, right? The 1.1.1 series (strange numbering convention) was only released back in September.

Quoted 2 lines

With 1.1.x I mean 1.1.0 or 1.1.1. The breakage comes from the 1.1.0 bump; .0 to
.1 is fairly smooth.

Ok. 1.1.0 was released 2.5 years ago (August 2016). How come we still need fixes for quite some packages? All abandoned software? Should we get rid of some of those packages?