lz4: Changing how LZ4 library handling is done

This change will expect the system to have LZ4 libraries and headers installed by default. We still carry a bundled LZ4 library, which must now be explicitly enabled through providing --enable-bundled-lz4 to ./configure. Otherwise, as before, --disable-lz4 will completely remove any LZ4 support.

Also improve the autoconf code slightly, to use AS_HELP_STRING() where needed and wrap some strings/values with [] where it was missing in the LZ4 segment of ./confiugre.ac.

---
 Changes.rst            |  8 +++++++
 configure.ac           | 57 ++++++++++++++++++++++++++------------------------
 src/compat/Makefile.am |  5 ++++-
 3 files changed, 42 insertions(+), 28 deletions(-)

index 53a14438..128f148a 100644
@@ -308,6 +308,14 @@ User-visible Changes
 
 Maintainer-visible changes
 --------------------------
+- OpenVPN will not use the bundled LZ4 library by default if a system
+  library have not been found.  The bundled library needs to be enabled
+  explicitly by adding --enable-bundled-lz4 to ./configure.  This is to
+  remove any ambiguity of which library is being used.  And defaulting to
+  use the system library is best from a security perspective; this way the
+  LZ4 library can be updated externally without requiring OpenVPN to be
+  rebuilt and packaged.
+
 - OpenVPN no longer supports building with crypto support, but without TLS
   support.  As a consequence, OPENSSL_CRYPTO_{CFLAGS,LIBS} and
   OPENSSL_SSL_{CFLAGS,LIBS} have been merged into OPENSSL_{CFLAGS,LIBS}.  This

index 444a8f8b..3dc25f39 100644
@@ -26,5 +26,8 @@ libcompat_la_SOURCES = \
 	compat-daemon.c \
 	compat-inet_ntop.c \
 	compat-inet_pton.c \
-	compat-lz4.c compat-lz4.h \
 	compat-versionhelpers.h
+
+if ENABLE_BUNDLED_LZ4
+libcompat_la_SOURCES += compat-lz4.c compat-lz4.h
+endif

2.13.5

ACK.

Imho, this is a very good step towards moving the bundled lib apart. The bundled lib is still there, so still usable by the maintainer who wants that, but it basically becomes legacy.

Cheers,

Antonio Quartulli

Hi,

I'm totally missing the *reason* why you want to change this

We don't have --enable-bundled-compat flags for the rest of the things in compat/ either, don't we?

Also, I can't see consensus that "remove the bundled lz4" is the way to go - this was on the plate for the hackathon to discuss. You and Antonio are convinced that this is a good way forward, applying a very specific Linux-distro-based mindset to it ("missing libraries are a problem of the package builder, why should we care?") - please listen to my arguments: there are people out there that build OpenVPN from source (tarball or git), and they are looking at library dependencies with a slightly different view.

gert

USENET is *not* the non-clickable part of WWW!
                                                           //www.muc.de/~gert/
Gert Doering - Munich, Germany                             [email protected]...
fax: +49-89-35655025                        [email protected]...

Bundled libraries are considered to be bad, as it requires active maintenance. Just look at which version we ship in OpenVPN (1.6.0) and we've had two rebase patches on the -devel ML which have been ignored, first updating to v1.7.4.2 [0] and then v1.7.5 [1]. I have not bothered yet to send a new rebase request to the latest v1.8.0 [2]. That is an awfully bad track record.

[0] Dec 15, 2016
<https://www.mail-archive.com/[email protected]/msg13575.html>
[1] Feb 21, 2017
<https://www.mail-archive.com/[email protected]/msg14137.html>
[2] Aug 17, 2017 <https://github.com/lz4/lz4/releases/tag/v1.8.0>

If we even manage to miss that there are CVE fixes in the LZ4 library, then we've in even bigger trouble. Especially if we're so bad at updating just the maintenance releases.

Yes, I can already hear you say that updates caused by CVEs will be handled more promptly. But from my point of view, that's just words. We do not have any processes for handling bundled libraries nor response times nor who and how updates in bundled libraries are tracked.

Somehow I expected that argument from you, Gert. But that is truly comparing apples and oranges.

LZ4 is a large library compared to the compat functions we do provide:

  1605  compat-basename.c
  2208  compat-inet_ntop.c
  2261  compat-inet_pton.c
  2353  compat-daemon.c
  3210  compat-gettimeofday.c
  4057  compat-dirname.c
 49760  compat-lz4.c

The code complexity of compat-dirname.c (which is the second largest compat code) is negligible compared to compat-lz4.c So the risk of a security issue should be considerably lower in these other functions we add.

Secondly, we only use our compat-*.c functions if the underlying system does not carry those features - most commonly, these are libc related functions. So if a systems libc does not carry a function we need, we need to have a wrapper for a specific function. And strictly speaking, LZ4 is not a requirement for OpenVPN to function as an application.

Hence the --enable-bundled-lz4. If you do not have or want to build LZ4 yourself first, then you can use our bundled LZ4 - with the risks that implies. But this is done explicitly, so a developer or package maintainer is forced to take a decision here first.

But equally important, if anyone is going to build OpenVPN from source, what is the chances that they will not be able to build LZ4 from source before building OpenVPN?

And we already have several other external libraries we depend on which we do not carry a compat-* version for ... LZO being the most obvious one, as it is a 1:1 comparison to LZ4. Then there is openssl/mbedtls. But we do have feature specific dependencies pkcs11-helper, p11kit, which is available on all our supported platforms. Then there is libpam (for *nix). And Linux can add libselinux and libsystemd into the mix as well.

This patch actually aligns LZ4 to be treated more equally to LZO, with the distinction that we do have --enable-bundled-lz4 - at least for some time forward.

And some more arguments why bundled libraries are bad, here even from a FreeBSD perspective:
<https://www.freebsd.org/doc/en/books/porters-handbook/bundled-libs.html>

A couple of other with more generic perspectives:
<http://www.professionalsecurity.co.uk/products/computer-systems-and-it-security-news/library-bundles-a-game-of-chance/>
<https://blog.flameeyes.eu/2009/03/bundling-libraries-the-curse-of-the-ancients/>

And the perspective from a few Linux distros:
<https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies>
<https://fedoraproject.org/wiki/Bundled_Libraries?rd=Packaging:Bundled_Libraries>

kind regards,

David Sommerseth
OpenVPN Technologies, Inc

Just a correction. We bundle lz4-1.7.1, not 1.6.0. That was a left-over from my testing of the API update patch. But that doesn't change my argument that much.

kind regards,

David Sommerseth
OpenVPN Technologies, Inc