Skip to contentSkip to conversation

add support for engine keys

PATCH In review James Bottomley 4 participants 33 messages May 28, 2020 – Jun 24, 2020

lore
Share & tools
lore
#0 lore raw

v6 cover add support for engine keys

This update exposes the new configuration file addition to the openssl initialization as a separate patch. This means that the behaviour of pontentially reading from a configuration file is global rather than contingent on the engine parameter being present and thus is now fully consistent. I've added this as a separate patch, since it's a separate feature.

---

Engine keys are an openssl concept for a key file which can only be understood by an engine (usually because it's been wrapped by the engine itself). We use this for TPM engine keys, so you can either generate them within your TPM or wrap them from existing private keys.
Once wrapped, the keys will only function in the TPM that generated
them, so it means the VPN keys are tied to the physical platform, which is very useful. Engine keys have to be loaded via a specific callback, so use this as a fallback in openvpn if an engine is specified and if the PEM read of the private key fails.

Adding a unit test for this type of key proved particularly
problematic: there's apparently no simple engine you can use to check the functionality, so after a bit of googling, I just wrote one as part of the test. You can see that the unit test converts an existing key to engine format (which is simply changing the PEM guards), tries to start openvpn with the key and verifies that the engine methods are called and the password correctly retrieved. To make the test simple, it relies on openssl detecting a mismatch between the certificate and the key after the key has been loaded rather than going on to bring up an openvpn loop, but I think that's sufficient to test out the engine patch fully.

---

James Bottomley (3):
openssl: add engine method for loading the key
crypto_openssl: add initialization to pick up local configuration Add unit tests for engine keys

 configure.ac                                  |   5 +
 src/openvpn/crypto_openssl.c                  |  61 +++++++++++
 src/openvpn/crypto_openssl.h                  |  12 +++
 src/openvpn/ssl_openssl.c                     |   5 +
 tests/unit_tests/Makefile.am                  |   3 +
 tests/unit_tests/engine-key/Makefile.am       |  24 +++++
 .../engine-key/check_engine_keys.sh           |  30 ++++++
 tests/unit_tests/engine-key/libtestengine.c   | 101 ++++++++++++++++++
 tests/unit_tests/engine-key/openssl.cnf.in    |  12 +++
 9 files changed, 253 insertions(+)
 create mode 100644 tests/unit_tests/engine-key/Makefile.am
 create mode 100755 tests/unit_tests/engine-key/check_engine_keys.sh
 create mode 100644 tests/unit_tests/engine-key/libtestengine.c
 create mode 100644 tests/unit_tests/engine-key/openssl.cnf.in
Signature · 1 line
2.26.2
#1 lore raw

PATCH v6 1/3 openssl: add engine method for loading the key

As well as doing crypto acceleration, engines can also be used to load key files. If the engine is set, and the private key loading fails for bio methods, this patch makes openvpn try to get the engine to load the key. If that succeeds, we end up using an engine based key. This can be used with the openssl tpm engines to make openvpn use a TPM wrapped key file.

---

v2: add better configuration guarding

v4: - use crypto_msg() instead of raw openssl prints
- remove ENGINE_init/finish(). Openvpn already initializes the engine so doing a second initialization is wrong.
- don't clear the openssl errors from the BIO_read failure just in case they might be useful.
- ad ui.h include for openssl 1.1 build failure

v5: add engine init

---
 src/openvpn/crypto_openssl.c | 56 ++++++++++++++++++++++++++++++++++++
 src/openvpn/crypto_openssl.h | 12 ++++++++
 src/openvpn/ssl_openssl.c    |  5 ++++
 3 files changed, 73 insertions(+)
src/openvpn/crypto_openssl.c +56 -0
index a5b2c45a..4ac77fde 100644
@@ -63,6 +63,7 @@
 #endif
 
 #if HAVE_OPENSSL_ENGINE
+#include <openssl/ui.h>
 #include <openssl/engine.h>
 
 static bool engine_initialized = false; /* GLOBAL */
@@ -1070,4 +1071,59 @@ memcmp_constant_time(const void *a, const void *b, size_t size)
 {
     return CRYPTO_memcmp(a, b, size);
 }
+
+#if HAVE_OPENSSL_ENGINE
+static int
+ui_reader(UI *ui, UI_STRING *uis)
+{
+    SSL_CTX *ctx = UI_get0_user_data(ui);
+
+    if (UI_get_string_type(uis) == UIT_PROMPT) {
+        pem_password_cb *cb = SSL_CTX_get_default_passwd_cb(ctx);
+        void *d = SSL_CTX_get_default_passwd_cb_userdata(ctx);
+        char password[64];
+
+        cb(password, sizeof(password), 0, d);
+        UI_set_result(ui, uis, password);
+
+        return 1;
+    }
+    return 0;
+}
+#endif
+
+EVP_PKEY *
+engine_load_key(const char *file, SSL_CTX *ctx)
+{
+#if HAVE_OPENSSL_ENGINE
+    UI_METHOD *ui;
+    EVP_PKEY *pkey;
+
+    if (!engine_persist)
+        return NULL;
+
+    /* this will print out the error from BIO_read */
+    crypto_msg(M_INFO, "PEM_read_bio failed, now trying engine method to load private key");
+
+    ui = UI_create_method("openvpn");
+    if (!ui) {
+	crypto_msg(M_FATAL, "Engine UI creation failed");
+        return NULL;
+    }
+
+    UI_method_set_reader(ui, ui_reader);
+
+    ENGINE_init(engine_persist);
+    pkey = ENGINE_load_private_key(engine_persist, file, ui, ctx);
+    ENGINE_finish(engine_persist);
+    if (!pkey)
+	crypto_msg(M_FATAL, "Engine could not load key file");
+
+    UI_destroy_method(ui);
+    return pkey;
+#else
+    return NULL;
+#endif
+}
+
 #endif /* ENABLE_CRYPTO_OPENSSL */
src/openvpn/crypto_openssl.h +12 -0
index 64754480..7449fbd3 100644
@@ -107,4 +107,16 @@ cipher_kt_var_key_size(const cipher_kt_t *cipher)
     return EVP_CIPHER_flags(cipher) & EVP_CIPH_VARIABLE_LENGTH;
 }
 
+/**
+ * Load a key file from an engine
+ *
+ * @param file	The engine file to load
+ * @param ui	The UI method for the password prompt
+ * @param data	The data to pass to the UI method
+ *
+ * @return	The private key if successful or NULL if not
+ */
+EVP_PKEY *
+engine_load_key(const char *file, SSL_CTX *ctx);
+
 #endif /* CRYPTO_OPENSSL_H_ */
src/openvpn/ssl_openssl.c +5 -0
index 06c836da..a489053b 100644
@@ -1020,6 +1020,11 @@ tls_ctx_load_priv_file(struct tls_root_ctx *ctx, const char *priv_key_file,
     pkey = PEM_read_bio_PrivateKey(in, NULL,
                                    SSL_CTX_get_default_passwd_cb(ctx->ctx),
                                    SSL_CTX_get_default_passwd_cb_userdata(ctx->ctx));
+    if (!pkey)
+    {
+        pkey = engine_load_key(priv_key_file, ctx->ctx);
+    }
+
     if (!pkey || !SSL_CTX_use_PrivateKey(ssl_ctx, pkey))
     {
 #ifdef ENABLE_MANAGEMENT
Signature · 1 line
2.26.2
#4 lore raw

PATCH v6 2/3 crypto_openssl: add initialization to pick up local configuration

In reply to James Bottomley

The test programme for the new openssl engine code requires overriding the system default configuration file to point to the location of the test engine. Add an initialization stanza that makes this behaviour universal, so now anyone running openvpn configured with openssl can specify their own configuration file with the OPENSSL_CONF environment variable.

---
 src/openvpn/crypto_openssl.c | 5 +++++
 1 file changed, 5 insertions(+)
src/openvpn/crypto_openssl.c +5 -0
index 4ac77fde..fd57edd2 100644
@@ -149,6 +149,11 @@ crypto_init_lib_engine(const char *engine_name)
 void
 crypto_init_lib(void)
 {
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL);
+#else
+    OPENSSL_config(NULL);
+#endif
     /*
      * If you build the OpenSSL library and OpenVPN with
      * CRYPTO_MDEBUG, you will get a listing of OpenSSL
Signature · 1 line
2.26.2
#12 lore raw

PATCH v6 3/3 Add unit tests for engine keys

In reply to James Bottomley

Testing engines is problematic, so one of the prerequisites built for the tests is a simple openssl engine that reads a non-standard PEM guarded key. The test is simply can we run a client/server configuration with the usual sample key replaced by an engine key. The trivial engine prints out some operations and we check for these in the log to make sure the engine was used to load the key and that it correctly got the password.

---
v5: do not hard code dynamic library extension into openssl.cnf (MacOS)
v4: add OPENSSL_config(NULL) so debian checks will work
v3: added this patch

---
 configure.ac                                  |   5 +
 tests/unit_tests/Makefile.am                  |   3 +
 tests/unit_tests/engine-key/Makefile.am       |  24 +++++
 .../engine-key/check_engine_keys.sh           |  30 ++++++
 tests/unit_tests/engine-key/libtestengine.c   | 101 ++++++++++++++++++
 tests/unit_tests/engine-key/openssl.cnf.in    |  12 +++
 6 files changed, 175 insertions(+)
 create mode 100644 tests/unit_tests/engine-key/Makefile.am
 create mode 100755 tests/unit_tests/engine-key/check_engine_keys.sh
 create mode 100644 tests/unit_tests/engine-key/libtestengine.c
 create mode 100644 tests/unit_tests/engine-key/openssl.cnf.in
configure.ac +5 -0
index 273a8d1b..92d4eeba 100644
@@ -1387,6 +1387,10 @@ AM_CONDITIONAL([GIT_CHECKOUT], [test "${GIT_CHECKOUT}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_AUTH_PAM], [test "${enable_plugin_auth_pam}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_DOWN_ROOT], [test "${enable_plugin_down_root}" = "yes"])
 AM_CONDITIONAL([HAVE_LD_WRAP_SUPPORT], [test "${have_ld_wrap_support}" = "yes"])
+AM_CONDITIONAL([OPENSSL_ENGINE], [test "${have_openssl_engine}" = "yes"])
+
+shrext=$shrext_cmds
+AC_SUBST([shrext])
 
 sampledir="\$(docdir)/sample"
 AC_SUBST([plugindir])
@@ -1448,6 +1452,7 @@ AC_CONFIG_FILES([
         tests/unit_tests/openvpn/Makefile
         tests/unit_tests/plugins/Makefile
         tests/unit_tests/plugins/auth-pam/Makefile
+	tests/unit_tests/engine-key/Makefile
 	sample/Makefile
 ])
 AC_CONFIG_FILES([tests/t_client.sh], [chmod +x tests/t_client.sh])
tests/unit_tests/Makefile.am +3 -0
index 33fefaac..f27cd90f 100644
@@ -2,4 +2,7 @@ AUTOMAKE_OPTIONS = foreign
 
 if ENABLE_UNITTESTS
 SUBDIRS = example_test openvpn plugins
+if OPENSSL_ENGINE
+SUBDIRS += engine-key
+endif
 endif
tests/unit_tests/engine-key/Makefile.am new file +24 -0
new file mode 100644
index 00000000..05f56bfd
@@ -0,0 +1,24 @@
+AUTOMAKE_OPTIONS = foreign
+
+check_LTLIBRARIES = libtestengine.la
+conffiles = openssl.cnf
+
+TESTS_ENVIRONMENT = srcdir="$(abs_srcdir)"; \
+	builddir="$(abs_builddir)"; \
+	top_builddir="$(top_builddir)"; \
+	top_srcdir="$(top_srcdir)"; \
+	export srcdir builddir top_builddir top_srcdir;
+
+TESTS = check_engine_keys.sh
+check_engine_keys.sh: $(conffiles)
+
+clean-local:
+	rm -f $(conffiles)
+
+$(builddir)/%.cnf: $(srcdir)/%.cnf.in
+	sed 's/SHREXT/@shrext@/' < $< > $@
+
+libtestengine_la_SOURCES = libtestengine.c
+libtestengine_la_LDFLAGS = @TEST_LDFLAGS@ -rpath /lib -avoid-version -module -shared -export-dynamic
+libtestengine_la_CFLAGS = @TEST_CFLAGS@ -I$(openvpn_srcdir) -I$(compat_srcdir)
+
tests/unit_tests/engine-key/check_engine_keys.sh new file +30 -0
new file mode 100755
index 00000000..e0c9d7b0
@@ -0,0 +1,30 @@
+#!/bin/sh
+
+OPENSSL_CONF="${builddir}/openssl.cnf"
+export OPENSSL_CONF
+
+password='AT3S4PASSWD'
+
+key="${builddir}/client.key"
+pwdfile="${builddir}/passwd"
+
+# create an engine key for us
+sed 's/PRIVATE KEY/TEST ENGINE KEY/' < ${top_srcdir}/sample/sample-keys/client.key > ${key}
+echo "$password" > $pwdfile
+
+# note here we've induced a mismatch in the client key and the server
+# cert which openvpn should report and die.  Check that it does.  Note
+# also that this mismatch depends on openssl not openvpn, so it is
+# somewhat fragile
+${top_builddir}/src/openvpn/openvpn --cd ${top_srcdir}/sample --config sample-config-files/loopback-server --engine testengine --key ${key} --askpass $pwdfile > log.txt 2>&1
+
+# first off check we died because of a key mismatch.  If this doesn't
+# pass, suspect openssl of returning different messages and update the
+# test accordingly
+grep -q 'X509_check_private_key:key values mismatch' log.txt || { echo "Key mismatch not detected"; exit 1; }
+
+# now look for the engine prints (these are under our control)
+grep -q 'ENGINE: engine_init called' log.txt || { echo "Engine initialization not detected"; exit 1; }
+grep -q 'ENGINE: engine_load_key called' log.txt || { echo "Key was not loaded from engine"; exit 1; }
+grep -q "ENGINE: engine_load_key got password ${password}" log.txt || { echo "Key password was not retrieved by the engine"; exit 1; }
+exit 0
tests/unit_tests/engine-key/libtestengine.c new file +101 -0

101 lines not on this page · raw message

tests/unit_tests/engine-key/openssl.cnf.in new file +12 -0
new file mode 100644
index 00000000..93edd483
@@ -0,0 +1,12 @@
+HOME		= .
+openssl_conf	= openssl_init
+
+[req]
+[openssl_init]
+engines		= engines_section
+
+[engines_section]
+testengine	= testengine_section
+
+[testengine_section]
+dynamic_path	= $ENV::srcdir/.libs/libtestengineSHREXT
Signature · 1 line
2.26.2
#13 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Am 29.05.20 um 00:59 schrieb James Bottomley:

Quoted 244 lines

Testing engines is problematic, so one of the prerequisites built for the tests is a simple openssl engine that reads a non-standard PEM guarded key. The test is simply can we run a client/server configuration with the usual sample key replaced by an engine key. The trivial engine prints out some operations and we check for these in the log to make sure the engine was used to load the key and that it correctly got the password.

---
v5: do not hard code dynamic library extension into openssl.cnf (MacOS)
v4: add OPENSSL_config(NULL) so debian checks will work
v3: added this patch

---
 configure.ac                                  |   5 +
 tests/unit_tests/Makefile.am                  |   3 +
 tests/unit_tests/engine-key/Makefile.am       |  24 +++++
 .../engine-key/check_engine_keys.sh           |  30 ++++++
 tests/unit_tests/engine-key/libtestengine.c   | 101 ++++++++++++++++++
 tests/unit_tests/engine-key/openssl.cnf.in    |  12 +++
 6 files changed, 175 insertions(+)
 create mode 100644 tests/unit_tests/engine-key/Makefile.am
 create mode 100755 tests/unit_tests/engine-key/check_engine_keys.sh
 create mode 100644 tests/unit_tests/engine-key/libtestengine.c
 create mode 100644 tests/unit_tests/engine-key/openssl.cnf.in
configure.ac +5 -0
index 273a8d1b..92d4eeba 100644
@@ -1387,6 +1387,10 @@ AM_CONDITIONAL([GIT_CHECKOUT], [test "${GIT_CHECKOUT}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_AUTH_PAM], [test "${enable_plugin_auth_pam}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_DOWN_ROOT], [test "${enable_plugin_down_root}" = "yes"])
 AM_CONDITIONAL([HAVE_LD_WRAP_SUPPORT], [test "${have_ld_wrap_support}" = "yes"])
+AM_CONDITIONAL([OPENSSL_ENGINE], [test "${have_openssl_engine}" = "yes"])
+
+shrext=$shrext_cmds
+AC_SUBST([shrext])
 
 sampledir="\$(docdir)/sample"
 AC_SUBST([plugindir])
@@ -1448,6 +1452,7 @@ AC_CONFIG_FILES([
         tests/unit_tests/openvpn/Makefile
         tests/unit_tests/plugins/Makefile
         tests/unit_tests/plugins/auth-pam/Makefile
+	tests/unit_tests/engine-key/Makefile
 	sample/Makefile
 ])
 AC_CONFIG_FILES([tests/t_client.sh], [chmod +x tests/t_client.sh])
tests/unit_tests/Makefile.am +3 -0
index 33fefaac..f27cd90f 100644
@@ -2,4 +2,7 @@ AUTOMAKE_OPTIONS = foreign
 
 if ENABLE_UNITTESTS
 SUBDIRS = example_test openvpn plugins
+if OPENSSL_ENGINE
+SUBDIRS += engine-key
+endif
 endif
tests/unit_tests/engine-key/Makefile.am new file +24 -0
new file mode 100644
index 00000000..05f56bfd
@@ -0,0 +1,24 @@
+AUTOMAKE_OPTIONS = foreign
+
+check_LTLIBRARIES = libtestengine.la
+conffiles = openssl.cnf
+
+TESTS_ENVIRONMENT = srcdir="$(abs_srcdir)"; \
+	builddir="$(abs_builddir)"; \
+	top_builddir="$(top_builddir)"; \
+	top_srcdir="$(top_srcdir)"; \
+	export srcdir builddir top_builddir top_srcdir;
+
+TESTS = check_engine_keys.sh
+check_engine_keys.sh: $(conffiles)
+
+clean-local:
+	rm -f $(conffiles)
+
+$(builddir)/%.cnf: $(srcdir)/%.cnf.in
+	sed 's/SHREXT/@shrext@/' < $< > $@
+
+libtestengine_la_SOURCES = libtestengine.c
+libtestengine_la_LDFLAGS = @TEST_LDFLAGS@ -rpath /lib -avoid-version -module -shared -export-dynamic
+libtestengine_la_CFLAGS = @TEST_CFLAGS@ -I$(openvpn_srcdir) -I$(compat_srcdir)
+
tests/unit_tests/engine-key/check_engine_keys.sh new file +30 -0
new file mode 100755
index 00000000..e0c9d7b0
@@ -0,0 +1,30 @@
+#!/bin/sh
+
+OPENSSL_CONF="${builddir}/openssl.cnf"
+export OPENSSL_CONF
+
+password='AT3S4PASSWD'
+
+key="${builddir}/client.key"
+pwdfile="${builddir}/passwd"
+
+# create an engine key for us
+sed 's/PRIVATE KEY/TEST ENGINE KEY/' < ${top_srcdir}/sample/sample-keys/client.key > ${key}
+echo "$password" > $pwdfile
+
+# note here we've induced a mismatch in the client key and the server
+# cert which openvpn should report and die.  Check that it does.  Note
+# also that this mismatch depends on openssl not openvpn, so it is
+# somewhat fragile
+${top_builddir}/src/openvpn/openvpn --cd ${top_srcdir}/sample --config sample-config-files/loopback-server --engine testengine --key ${key} --askpass $pwdfile > log.txt 2>&1
+
+# first off check we died because of a key mismatch.  If this doesn't
+# pass, suspect openssl of returning different messages and update the
+# test accordingly
+grep -q 'X509_check_private_key:key values mismatch' log.txt || { echo "Key mismatch not detected"; exit 1; }
+
+# now look for the engine prints (these are under our control)
+grep -q 'ENGINE: engine_init called' log.txt || { echo "Engine initialization not detected"; exit 1; }
+grep -q 'ENGINE: engine_load_key called' log.txt || { echo "Key was not loaded from engine"; exit 1; }
+grep -q "ENGINE: engine_load_key got password ${password}" log.txt || { echo "Key password was not retrieved by the engine"; exit 1; }
+exit 0
tests/unit_tests/engine-key/libtestengine.c new file +101 -0
new file mode 100644
index 00000000..46ec1e33
@@ -0,0 +1,101 @@
+#include <string.h>
+#include <openssl/engine.h>
+#include <openssl/evp.h>
+#include <openssl/pem.h>
+
+static char *engine_id = "testengine";
+static char *engine_name = "Engine for testing openvpn engine key support";
+
+static int is_initialized = 0;
+
+static int engine_init(ENGINE *e)
+{
+	is_initialized = 1;
+	fprintf(stderr, "ENGINE: engine_init called\n");
+	return 1;
+}
+
+static int engine_finish(ENGINE *e)
+{
+	fprintf(stderr, "ENGINE: engine_finsh called\n");
+	is_initialized = 0;
+	return 1;
+}
+
+static EVP_PKEY *engine_load_key(ENGINE *e, const char *key_id,
+				 UI_METHOD *ui_method, void *cb_data)
+{
+	BIO *b;
+	EVP_PKEY *pkey;
+	PKCS8_PRIV_KEY_INFO *p8inf;
+	UI *ui;
+	char auth[256];
+
+	fprintf(stderr, "ENGINE: engine_load_key called\n");
+
+	if (!is_initialized) {
+		fprintf(stderr, "Load Key called without correct initialization\n");
+		return NULL;
+	}
+	b = BIO_new_file(key_id, "r");
+	if (!b) {
+		fprintf(stderr, "File %s does not exist or cannot be read\n", key_id);
+		return 0;
+	}
+	/* Basically read an EVP_PKEY private key file with different
+	 * PEM guards --- we are a test engine */
+	p8inf = PEM_ASN1_read_bio((d2i_of_void *)d2i_PKCS8_PRIV_KEY_INFO,
+				 "TEST ENGINE KEY", b,
+				 NULL, NULL, NULL);
+	BIO_free(b);
+	if (!p8inf) {
+		fprintf(stderr, "Failed to read engine private key\n");
+		return NULL;
+	}
+	pkey = EVP_PKCS82PKEY(p8inf);
+
+	/* now we have a private key, pretend it had a password
+	 * this verifies the password makes it through openvpn OK */
+	ui = UI_new();
+
+	if (ui_method)
+		UI_set_method(ui, ui_method);
+
+	UI_add_user_data(ui, cb_data);
+
+	if (UI_add_input_string(ui, "enter test engine key",
+				UI_INPUT_FLAG_DEFAULT_PWD,
+				auth, 0, sizeof(auth)) == 0) {
+		fprintf(stderr, "UI_add_input_string failed\n");
+		goto out;
+	}
+
+	if (UI_process(ui)) {
+		fprintf(stderr, "UI_process failed\n");
+		goto out;
+	}
+
+	fprintf(stderr, "ENGINE: engine_load_key got password %s\n", auth);
+
+ out:
+	UI_free(ui);
+
+	return pkey;
+}
+
+
+static int engine_bind_fn(ENGINE *e, const char *id)
+{
+	if (id && strcmp(id, engine_id) != 0)
+		return 0;
+	if (!ENGINE_set_id(e, engine_id) ||
+	    !ENGINE_set_name(e, engine_name) ||
+	    !ENGINE_set_init_function(e, engine_init) ||
+	    !ENGINE_set_finish_function(e, engine_finish) ||
+	    !ENGINE_set_load_privkey_function(e, engine_load_key))
+		return 0;
+	return 1;
+}
+
+IMPLEMENT_DYNAMIC_CHECK_FN()
+IMPLEMENT_DYNAMIC_BIND_FN(engine_bind_fn)
tests/unit_tests/engine-key/openssl.cnf.in new file +2 -0
new file mode 100644
index 00000000..93edd483
@@ -0,0 +1,12 @@
+HOME		= .
+openssl_conf	= openssl_init

tests/unit_tests/engine-key/openssl.cnf.in line 12

Quoted 10 lines
tests/unit_tests/engine-key/openssl.cnf.in new file +10 -0
+
+[req]
+[openssl_init]
+engines		= engines_section
+
+[engines_section]
+testengine	= testengine_section
+
+[testengine_section]
+dynamic_path	= $ENV::srcdir/.libs/libtestengineSHREXT

Somehow this turns into a autoconf/automake nightmare:

[testengine_section]
dynamic_path = $ENV::srcdir/.libs/libtestengine`test .odule = .yes
SHREXTSHREXT echo .so || echo .dylib`

is what I end up in the openssl.cnf

The generated Makefile already has:

$(builddir)/%.cnf: $(srcdir)/%.cnf.in
sed 's/SHREXT/test .$module = .yes && echo .so || echo .dylib/' < $<

Quoted 1 line

$@

Maybe instead of using that use magic useing two different section like testengineso and testenginedylib and then makeing the engine name in the script dynamic?

Also for some reason the makefile still builds it now as .so instead of .dylib and if I manually put the full path in the openssl.cnf the test works:

dynamic_path =
/Users/arne/oss/openvpn-ossl/tests/unit_tests/engine-key/.libs/libtestengine.so

Any idea? I can also help to get this mess resolve but maybe you have a quick idea what is wrong.

Arne

Acked-by on v6 1/3
#2 lore raw

Re: [Openvpn-devel] [PATCH v6 1/3] openssl: add engine method for loading the key

In reply to James Bottomley

Am 29.05.20 um 00:59 schrieb James Bottomley:

Quoted 7 lines

As well as doing crypto acceleration, engines can also be used to load key files. If the engine is set, and the private key loading fails for bio methods, this patch makes openvpn try to get the engine to load the key. If that succeeds, we end up using an engine based key. This can be used with the openssl tpm engines to make openvpn use a TPM wrapped key file.

This looks good now.

Arne

Acked-by on v6 2/3
#5 lore raw

Re: [Openvpn-devel] [PATCH v6 2/3] crypto_openssl: add initialization to pick up local configuration

In reply to James Bottomley

Am 29.05.20 um 00:59 schrieb James Bottomley:

Quoted 1 line

The test programme for the new openssl engine code requires overriding

                   ^^^

Unrelated to this commit but I wondered about this spelling and it seems that the British programme spelling usually is a TV programme while program is used in computer context. AE just uses program for both.

Quoted 5 lines

the system default configuration file to point to the location of the test engine. Add an initialization stanza that makes this behaviour universal, so now anyone running openvpn configured with openssl can specify their own configuration file with the OPENSSL_CONF environment variable.

I also tested that with a configuration file with an invalid statement this does not abort OpenVPN (the bug we had a few weeks ago)

Arne

#14 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

In reply to Arne Schwabe

On Fri, 2020-06-05 at 13:31 +0200, Arne Schwabe wrote:
[...]

Quoted 1 line

Somehow this turns into a autoconf/automake nightmare:

Heh, got to say autoconf is a bit of a nightmare for its more esoteric features because the docs usually don't cover them and you end up having to take the opinion of the internet, which is somewhat, er, diverse.

Quoted 4 lines

[testengine_section]
dynamic_path = $ENV::srcdir/.libs/libtestengine`test .odule =
.yes SHREXTSHREXT echo .so || echo .dylib`

Quoted 8 lines

is what I end up in the openssl.cnf

The generated Makefile already has:

$(builddir)/%.cnf: $(srcdir)/%.cnf.in
sed 's/SHREXT/`test .$module = .yes && echo .so || echo
.dylib`/' < $<

Quoted 1 line

$@

Hm, so SHREXT is dynamic on MAC platforms. If I replace the single quote with double, that should allow the shell expansion.

Quoted 3 lines

Maybe instead of using that use magic useing two different section like testengineso and testenginedylib and then makeing the engine name in the script dynamic?

I'll investigate, but openssl has a fairly tight binding between the name of the dynamic object and the name of the engine. It's very difficult (but not I perhaps impossible) to have an engine name different from the DSO name.

Quoted 2 lines

Also for some reason the makefile still builds it now as .so instead of .dylib

That's likely a libtool flag issue. If you update this in Makefile.am:

libtestengine_la_LDFLAGS = -rpath /lib -avoid-version -module -shared -export-dynamic

does it build a .dylib?

Quoted 9 lines

and if I manually put the full path in the openssl.cnf the
test works:

dynamic_path =
/Users/arne/oss/openvpn-ossl/tests/unit_tests/engine- key/.libs/libtestengine.so

Any idea? I can also help to get this mess resolve but maybe you have a quick idea what is wrong.

My money would be on the idea that environment specifiers in openssl conf files are only supported on UNIX. I'll get it to do a full path substitution on build to fix this.

James

#15 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Quoted 15 lines · James Bottomley

On Fri, 2020-06-05 at 13:31 +0200, Arne Schwabe wrote:
[...]

Quoted 1 line

Somehow this turns into a autoconf/automake nightmare:

Heh, got to say autoconf is a bit of a nightmare for its more esoteric features because the docs usually don't cover them and you end up having to take the opinion of the internet, which is somewhat, er, diverse.

Quoted 6 lines

[testengine_section]
dynamic_path = $ENV::srcdir/.libs/libtestengine`test .odule
= .yes SHREXTSHREXT echo .so || echo .dylib`

is what I end up in the openssl.cnf

Quoted 8 lines
Quoted 5 lines

The generated Makefile already has:

$(builddir)/%.cnf: $(srcdir)/%.cnf.in
sed 's/SHREXT/`test .$module = .yes && echo .so || echo
.dylib`/' < $< $@

Hm, so SHREXT is dynamic on MAC platforms. If I replace the single quote with double, that should allow the shell expansion.

The internet thinks the fix is to add eval to the hunk in configure.ac:

eval shrext=$shrext_cmds
AC_SUBST([shrext])

I tried it out and it seems to work.

Quoted 13 lines
Quoted 3 lines

Maybe instead of using that use magic useing two different section like testengineso and testenginedylib and then makeing the engine name in the script dynamic?

I'll investigate, but openssl has a fairly tight binding between the name of the dynamic object and the name of the engine. It's very difficult (but not I perhaps impossible) to have an engine name different from the DSO name.

Quoted 3 lines

Also for some reason the makefile still builds it now as .so instead
of .dylib

Quoted 7 lines

That's likely a libtool flag issue. If you update this in Makefile.am:

libtestengine_la_LDFLAGS = -rpath /lib -avoid-version -module -shared -export-dynamic

does it build a .dylib?

So if this works, with the eval above and full path substitution in the openssl.cnf.in I think I might have a solution.

James

#16 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Am 05.06.20 um 19:19 schrieb James Bottomley:

Quoted 25 lines · James Bottomley
Quoted 15 lines · Arne Schwabe

[...]

Quoted 1 line

Somehow this turns into a autoconf/automake nightmare:

Heh, got to say autoconf is a bit of a nightmare for its more esoteric features because the docs usually don't cover them and you end up having to take the opinion of the internet, which is somewhat, er, diverse.

Quoted 6 lines

[testengine_section]
dynamic_path = $ENV::srcdir/.libs/libtestengine`test .odule
= .yes SHREXTSHREXT echo .so || echo .dylib`

is what I end up in the openssl.cnf

Quoted 8 lines
Quoted 5 lines

The generated Makefile already has:

$(builddir)/%.cnf: $(srcdir)/%.cnf.in
sed 's/SHREXT/`test .$module = .yes && echo .so || echo
.dylib`/' < $< $@

Hm, so SHREXT is dynamic on MAC platforms. If I replace the single quote with double, that should allow the shell expansion.

Quoted 6 lines

The internet thinks the fix is to add eval to the hunk in configure.ac:

eval shrext=$shrext_cmds
AC_SUBST([shrext])

I tried it out and it seems to work.

[testengine_section]
dynamic_path = $ENV::srcdir/.libs/libtestengine.dylib

Yes, works here.

Quoted 17 lines
Quoted 17 lines
Quoted 4 lines

Maybe instead of using that use magic useing two different section like testengineso and testenginedylib and then makeing the engine name in the script dynamic?

I'll investigate, but openssl has a fairly tight binding between the name of the dynamic object and the name of the engine. It's very difficult (but not I perhaps impossible) to have an engine name different from the DSO name.

Quoted 3 lines

Also for some reason the makefile still builds it now as .so instead
of .dylib

That's likely a libtool flag issue. If you update this in Makefile.am:

Quoted 7 lines
Quoted 4 lines

libtestengine_la_LDFLAGS = -rpath /lib -avoid-version -module -shared -export-dynamic

does it build a .dylib?

So if this works, with the eval above and full path substitution in the openssl.cnf.in I think I might have a solution.

Unfortunately not. Still creates the .so file. If everything fails we might just do a if mac then cp from .so to .dylib as part of the test run. It is ugly but might be better than to wait forever on this patch.

Arne

#17 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Hi,

Quoted 6 lines · Arne Schwabe
Quoted 2 lines

So if this works, with the eval above and full path substitution in the openssl.cnf.in I think I might have a solution.

Unfortunately not. Still creates the .so file. If everything fails we might just do a if mac then cp from .so to .dylib as part of the test run. It is ugly but might be better than to wait forever on this patch.

Naive question - can we not just use a .so on Mac? Like, consistent, build .so, request .so from openssl.cnf?

(Haven't dug into shlib weirdness on MacOS yet. But if you want, I can tell stories about shlib on AIX. Which will make you ask for lots of alcoholic beverages so you can forget again...)

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#18 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Am 06.06.20 um 10:41 schrieb Gert Doering:

Quoted 10 lines

Hi,

Quoted 6 lines · Arne Schwabe
Quoted 2 lines

So if this works, with the eval above and full path substitution in the openssl.cnf.in I think I might have a solution.

Unfortunately not. Still creates the .so file. If everything fails we might just do a if mac then cp from .so to .dylib as part of the test run. It is ugly but might be better than to wait forever on this patch.

Quoted 6 lines

Naive question - can we not just use a .so on Mac? Like, consistent, build .so, request .so from openssl.cnf?

(Haven't dug into shlib weirdness on MacOS yet. But if you want, I can tell stories about shlib on AIX. Which will make you ask for lots of alcoholic beverages so you can forget again...)

I thought I tested that and it didn't work but I tested it again and that also works.

Arne

#19 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Hi,

Quoted 5 lines · Arne Schwabe
Quoted 2 lines

Naive question - can we not just use a .so on Mac? Like, consistent, build .so, request .so from openssl.cnf?

I thought I tested that and it didn't work but I tested it again and that also works.

Then let's just do that, and simplifiy the test stuff :-)

(Building a "production quality" OpenSSL engine would need to do what the platform wants, but for "this is really just to test the OpenVPN code", I think "less autoconf hoops is better")

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#20 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Quoted 6 lines · Gert Doering

Hi,

Quoted 3 lines · Arne Schwabe
Quoted 2 lines

Naive question - can we not just use a .so on Mac? Like, consistent, build .so, request .so from openssl.cnf?

Quoted 8 lines
Quoted 2 lines

I thought I tested that and it didn't work but I tested it again and that also works.

Then let's just do that, and simplifiy the test stuff :-)

(Building a "production quality" OpenSSL engine would need to do what the platform wants, but for "this is really just to test the OpenVPN code", I think "less autoconf hoops is better")

Well, I am intellectually curious since I maintain the TPM engine for openssl. But Macs don't have TPMs so no-one is ever going to complain
...

The only remaining problem is the lack of environment variable support in MAC openssl which I'll fix by using the absolute path. If it works I think below is the replacement patch.

James

---8>8>8><8<8<8----

From: James Bottomley <[email protected]...>
Subject: [PATCH v7 3/3] Add unit tests for engine keys

Testing engines is problematic, so one of the prerequisites built for the tests is a simple openssl engine that reads a non-standard PEM guarded key. The test is simply can we run a client/server configuration with the usual sample key replaced by an engine key. The trivial engine prints out some operations and we check for these in the log to make sure the engine was used to load the key and that it correctly got the password.

---
v7: add absolute path instead of env variable in openssl.cnf (MacOS)
v5: do not hard code dynamic library extension into openssl.cnf (MacOS)
v4: add OPENSSL_config(NULL) so debian checks will work
v3: added this patch

---
 configure.ac                                  |   6 ++
 tests/unit_tests/Makefile.am                  |   3 +
 tests/unit_tests/engine-key/Makefile.am       |  24 +++++
 .../engine-key/check_engine_keys.sh           |  30 ++++++
 tests/unit_tests/engine-key/libtestengine.c   | 101 ++++++++++++++++++
 tests/unit_tests/engine-key/openssl.cnf.in    |  12 +++
 6 files changed, 176 insertions(+)
 create mode 100644 tests/unit_tests/engine-key/Makefile.am
 create mode 100755 tests/unit_tests/engine-key/check_engine_keys.sh
 create mode 100644 tests/unit_tests/engine-key/libtestengine.c
 create mode 100644 tests/unit_tests/engine-key/openssl.cnf.in
configure.ac +6 -0
index 273a8d1b..3e8a541d 100644
@@ -1387,6 +1387,11 @@ AM_CONDITIONAL([GIT_CHECKOUT], [test "${GIT_CHECKOUT}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_AUTH_PAM], [test "${enable_plugin_auth_pam}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_DOWN_ROOT], [test "${enable_plugin_down_root}" = "yes"])
 AM_CONDITIONAL([HAVE_LD_WRAP_SUPPORT], [test "${have_ld_wrap_support}" = "yes"])
+AM_CONDITIONAL([OPENSSL_ENGINE], [test "${have_openssl_engine}" = "yes"])
+
+module=yes
+eval shrext=$shrext_cmds
+AC_SUBST([shrext])
 
 sampledir="\$(docdir)/sample"
 AC_SUBST([plugindir])
@@ -1448,6 +1453,7 @@ AC_CONFIG_FILES([
         tests/unit_tests/openvpn/Makefile
         tests/unit_tests/plugins/Makefile
         tests/unit_tests/plugins/auth-pam/Makefile
+	tests/unit_tests/engine-key/Makefile
 	sample/Makefile
 ])
 AC_CONFIG_FILES([tests/t_client.sh], [chmod +x tests/t_client.sh])
tests/unit_tests/Makefile.am +3 -0
index 33fefaac..f27cd90f 100644
@@ -2,4 +2,7 @@ AUTOMAKE_OPTIONS = foreign
 
 if ENABLE_UNITTESTS
 SUBDIRS = example_test openvpn plugins
+if OPENSSL_ENGINE
+SUBDIRS += engine-key
+endif
 endif
tests/unit_tests/engine-key/Makefile.am new file +24 -0
new file mode 100644
index 00000000..e8a473a8
@@ -0,0 +1,24 @@
+AUTOMAKE_OPTIONS = foreign
+
+check_LTLIBRARIES = libtestengine.la
+conffiles = openssl.cnf
+
+TESTS_ENVIRONMENT = srcdir="$(abs_srcdir)"; \
+	builddir="$(abs_builddir)"; \
+	top_builddir="$(top_builddir)"; \
+	top_srcdir="$(top_srcdir)"; \
+	export srcdir builddir top_builddir top_srcdir;
+
+TESTS = check_engine_keys.sh
+check_engine_keys.sh: $(conffiles)
+
+clean-local:
+	rm -f $(conffiles)
+
+$(builddir)/%.cnf: $(srcdir)/%.cnf.in
+	sed "s|SHREXT|@shrext@|;s|ABSBUILDDIR|$(abs_builddir)|" < $< > $@
+
+libtestengine_la_SOURCES = libtestengine.c
+libtestengine_la_LDFLAGS = @TEST_LDFLAGS@ -avoid-version -module -shared -export-dynamic
+libtestengine_la_CFLAGS = @TEST_CFLAGS@ -I$(openvpn_srcdir) -I$(compat_srcdir)
+
tests/unit_tests/engine-key/check_engine_keys.sh new file +30 -0
new file mode 100755
index 00000000..e0c9d7b0
@@ -0,0 +1,30 @@
+#!/bin/sh
+
+OPENSSL_CONF="${builddir}/openssl.cnf"
+export OPENSSL_CONF
+
+password='AT3S4PASSWD'
+
+key="${builddir}/client.key"
+pwdfile="${builddir}/passwd"
+
+# create an engine key for us
+sed 's/PRIVATE KEY/TEST ENGINE KEY/' < ${top_srcdir}/sample/sample-keys/client.key > ${key}
+echo "$password" > $pwdfile
+
+# note here we've induced a mismatch in the client key and the server
+# cert which openvpn should report and die.  Check that it does.  Note
+# also that this mismatch depends on openssl not openvpn, so it is
+# somewhat fragile
+${top_builddir}/src/openvpn/openvpn --cd ${top_srcdir}/sample --config sample-config-files/loopback-server --engine testengine --key ${key} --askpass $pwdfile > log.txt 2>&1
+
+# first off check we died because of a key mismatch.  If this doesn't
+# pass, suspect openssl of returning different messages and update the
+# test accordingly
+grep -q 'X509_check_private_key:key values mismatch' log.txt || { echo "Key mismatch not detected"; exit 1; }
+
+# now look for the engine prints (these are under our control)
+grep -q 'ENGINE: engine_init called' log.txt || { echo "Engine initialization not detected"; exit 1; }
+grep -q 'ENGINE: engine_load_key called' log.txt || { echo "Key was not loaded from engine"; exit 1; }
+grep -q "ENGINE: engine_load_key got password ${password}" log.txt || { echo "Key password was not retrieved by the engine"; exit 1; }
+exit 0
tests/unit_tests/engine-key/libtestengine.c new file +101 -0

101 lines not on this page · raw message

tests/unit_tests/engine-key/openssl.cnf.in new file +12 -0
new file mode 100644
index 00000000..40d328ce
@@ -0,0 +1,12 @@
+HOME		= .
+openssl_conf	= openssl_init
+
+[req]
+[openssl_init]
+engines		= engines_section
+
+[engines_section]
+testengine	= testengine_section
+
+[testengine_section]
+dynamic_path	= ABSBUILDDIR/.libs/libtestengineSHREXT
Signature · 1 line
2.26.2
#21 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Am 06.06.20 um 17:22 schrieb James Bottomley:

Quoted 16 lines · Gert Doering
Quoted 6 lines

Hi,

Quoted 3 lines · Arne Schwabe
Quoted 2 lines

Naive question - can we not just use a .so on Mac? Like, consistent, build .so, request .so from openssl.cnf?

Quoted 8 lines
Quoted 2 lines

I thought I tested that and it didn't work but I tested it again and that also works.

Then let's just do that, and simplifiy the test stuff :-)

(Building a "production quality" OpenSSL engine would need to do what the platform wants, but for "this is really just to test the OpenVPN code", I think "less autoconf hoops is better")

Quoted 8 lines

Well, I am intellectually curious since I maintain the TPM engine for openssl. But Macs don't have TPMs so no-one is ever going to complain
...

The only remaining problem is the lack of environment variable support in MAC openssl which I'll fix by using the absolute path. If it works I think below is the replacement patch.

Looks more like one step forward, two back:
openssl.cnf absolute path looks good:

dynamic_path =
/Users/arne/oss/openvpn-ossl/tests/unit_tests/engine-key/.libs/libtestengine.so

But now it has .so there. Alos the .libs directory now has neither .so nor .dylib

engine-key% find .
.
./libtestengine_la-libtestengine.o
./Makefile
./libtestengine.la
./libtestengine_la-libtestengine.lo
./openssl.cnf
./log.txt
./.deps
./.deps/libtestengine_la-libtestengine.Plo
./.libs
./.libs/libtestengine_la-libtestengine.o
./.libs/libtestengine.la
./.libs/libtestengine.a
./passwd
./client.key

(This is an out of tree build so only generated files here)

Make log:

make check
[...]
Making check in engine-key
Makefile:451: .deps/libtestengine_la-libtestengine.Plo: No such file or directory
/Applications/Xcode.app/Contents/Developer/usr/bin/make libtestengine.la
/bin/sh ../../../libtool --tag=CC --mode=compile clang
-DHAVE_CONFIG_H -I.
-I../../../../openvpn-git/tests/unit_tests/engine-key -I../../..
-I../../../include -I/usr/local/opt/[email protected].../include
-I../../../../openvpn-git/include -I/usr/local/Cellar/cmocka/1.1.5/include -I -I -Wall -Wno-unused-parameter -Wno-unused-function -O2 -g -Wall -std=c99 -MT libtestengine_la-libtestengine.lo -MD -MP -MF .deps/libtestengine_la-libtestengine.Tpo -c -o libtestengine_la-libtestengine.lo test -f 'libtestengine.c' || echo '../../../../openvpn-git/tests/unit_tests/engine-key/'libtestengine.c
libtool: compile: clang -DHAVE_CONFIG_H -I. -I../../../../openvpn-git/tests/unit_tests/engine-key -I../../.. -I../../../include -I/usr/local/opt/[email protected].../include -I../../../../openvpn-git/include -I/usr/local/Cellar/cmocka/1.1.5/include -I -I -Wall -Wno-unused-parameter -Wno-unused-function -O2 -g -Wall -std=c99 -MT libtestengine_la-libtestengine.lo -MD -MP -MF .deps/libtestengine_la-libtestengine.Tpo -c ../../../../openvpn-git/tests/unit_tests/engine-key/libtestengine.c -fno-common -DPIC -o .libs/libtestengine_la-libtestengine.o
libtool: compile: clang -DHAVE_CONFIG_H -I. -I../../../../openvpn-git/tests/unit_tests/engine-key -I../../.. -I../../../include -I/usr/local/opt/[email protected].../include -I../../../../openvpn-git/include -I/usr/local/Cellar/cmocka/1.1.5/include -I -I -Wall -Wno-unused-parameter -Wno-unused-function -O2 -g -Wall -std=c99 -MT libtestengine_la-libtestengine.lo -MD -MP -MF .deps/libtestengine_la-libtestengine.Tpo -c ../../../../openvpn-git/tests/unit_tests/engine-key/libtestengine.c -o libtestengine_la-libtestengine.o >/dev/null 2>&1
mv -f .deps/libtestengine_la-libtestengine.Tpo .deps/libtestengine_la-libtestengine.Plo
/bin/sh ../../../libtool --tag=CC --mode=link clang
-I/usr/local/opt/[email protected].../include -I../../../../openvpn-git/include
-I/usr/local/Cellar/cmocka/1.1.5/include -I -I -Wall -Wno-unused-parameter -Wno-unused-function -O2 -g -Wall -std=c99 -L/usr/local/opt/[email protected].../lib -lcrypto -lssl -L/usr/local/Cellar/cmocka/1.1.5/lib -lcmocka -avoid-version -module -shared -export-dynamic -o libtestengine.la libtestengine_la-libtestengine.lo -lresolv
libtool: link: ar cru .libs/libtestengine.a .libs/libtestengine_la-libtestengine.o
libtool: link: ranlib .libs/libtestengine.a
libtool: link: ( cd ".libs" && rm -f "libtestengine.la" && ln -s "../libtestengine.la" "libtestengine.la" ) /Applications/Xcode.app/Contents/Developer/usr/bin/make check-TESTS
sed "s|SHREXT|.so|;s|ABSBUILDDIR|/Users/arne/oss/openvpn-ossl/tests/unit_tests/engine-key|" < ../../../../openvpn-git/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf

#3 lore raw

[Openvpn-devel] [PATCH applied] Re: openssl: add engine method for loading the key

In reply to James Bottomley

Your patch has been applied to the master branch.

commit 8155f8aa0a8d801dd0d84e0ec844b9ca4c225df7

Author: James Bottomley
Date:   Thu May 28 15:59:18 2020 -0700
     openssl: add engine method for loading the key

     Signed-off-by: James Bottomley <[email protected]...>
     Acked-by: Arne Schwabe <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg19937.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering

#6 lore raw

[Openvpn-devel] [PATCH applied] Re: crypto_openssl: add initialization to pick up local configuration

In reply to James Bottomley

Your patch has been applied to the master branch.

commit a4071b20115c7d4e808df81169a986e65cec4efa

Author: James Bottomley
Date:   Thu May 28 15:59:19 2020 -0700
     crypto_openssl: add initialization to pick up local configuration

     Signed-off-by: James Bottomley <[email protected]...>
     Acked-by: Arne Schwabe <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg19936.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering

#22 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

In reply to James Bottomley

Hi,

Quoted 3 lines · James Bottomley

The only remaining problem is the lack of environment variable support in MAC openssl which I'll fix by using the absolute path. If it works I think below is the replacement patch.

That patch seems to go the "complicated" route, not the "easy" one...?

configure.ac AM_CONDITIONAL([GIT_CHECKOUT], [test "${GIT_CHECKOUT}" = "yes"]) · line 1394

Quoted 2 lines
configure.ac +2 -0
+eval shrext=$shrext_cmds
+AC_SUBST([shrext])

... I hoped we can get rid of this again...

And maybe then we can just have "libtestengine.so" target in Makefile?

Quoted 2 lines

+[testengine_section]
+dynamic_path = ABSBUILDDIR/.libs/libtestengineSHREXT

And no more SHREXT here either...

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#23 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Quoted 9 lines · Gert Doering

Hi,

Quoted 5 lines · James Bottomley

The only remaining problem is the lack of environment variable support
in MAC openssl which I'll fix by using the absolute path. If it works
I think below is the replacement patch.

Quoted 7 lines

That patch seems to go the "complicated" route, not the "easy" one...?

Review

Quoted 2 lines
(quoted diff) +2 -0
+eval shrext=$shrext_cmds
+AC_SUBST([shrext])

... I hoped we can get rid of this again...

In theory I can get around this using the libtool -shrext .so on the command line. Someone would have to check on windows that this is acceptable because they really like .dll as the extension. If it works on windows then the below patch should do it.

James

Quoted 9 lines

And maybe then we can just have "libtestengine.so" target in Makefile?

Quoted 2 lines

+[testengine_section]
+dynamic_path = ABSBUILDDIR/.libs/libtestengineSHREXT

And no more SHREXT here either...

gert

---8>8>8><8<8<8---
From 5e1fb464d4669882d11f52d314e7830d556c8f12 Mon Sep 17 00:00:00 2001
From: James Bottomley <[email protected]...>
Date: Wed, 24 Jan 2018 17:14:19 -0800
Subject: [PATCH] Add unit tests for engine keys

Testing engines is problematic, so one of the prerequisites built for the tests is a simple openssl engine that reads a non-standard PEM guarded key. The test is simply can we run a client/server configuration with the usual sample key replaced by an engine key. The trivial engine prints out some operations and we check for these in the log to make sure the engine was used to load the key and that it correctly got the password.

---
v6: add absolute path instead of env variable in openssl.cnf (MacOS)
v5: do not hard code dynamic library extension into openssl.cnf (MacOS)
v4: add OPENSSL_config(NULL) so debian checks will work
v3: added this patch

---
 configure.ac                                  |   2 +
 tests/unit_tests/Makefile.am                  |   3 +
 tests/unit_tests/engine-key/Makefile.am       |  24 +++++
 .../engine-key/check_engine_keys.sh           |  30 ++++++
 tests/unit_tests/engine-key/libtestengine.c   | 101 ++++++++++++++++++
 tests/unit_tests/engine-key/openssl.cnf.in    |  12 +++
 6 files changed, 172 insertions(+)
 create mode 100644 tests/unit_tests/engine-key/Makefile.am
 create mode 100755 tests/unit_tests/engine-key/check_engine_keys.sh
 create mode 100644 tests/unit_tests/engine-key/libtestengine.c
 create mode 100644 tests/unit_tests/engine-key/openssl.cnf.in
configure.ac +2 -0
index 273a8d1b..53b7a967 100644
@@ -1387,6 +1387,7 @@ AM_CONDITIONAL([GIT_CHECKOUT], [test "${GIT_CHECKOUT}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_AUTH_PAM], [test "${enable_plugin_auth_pam}" = "yes"])
 AM_CONDITIONAL([ENABLE_PLUGIN_DOWN_ROOT], [test "${enable_plugin_down_root}" = "yes"])
 AM_CONDITIONAL([HAVE_LD_WRAP_SUPPORT], [test "${have_ld_wrap_support}" = "yes"])
+AM_CONDITIONAL([OPENSSL_ENGINE], [test "${have_openssl_engine}" = "yes"])
 
 sampledir="\$(docdir)/sample"
 AC_SUBST([plugindir])
@@ -1448,6 +1449,7 @@ AC_CONFIG_FILES([
         tests/unit_tests/openvpn/Makefile
         tests/unit_tests/plugins/Makefile
         tests/unit_tests/plugins/auth-pam/Makefile
+	tests/unit_tests/engine-key/Makefile
 	sample/Makefile
 ])
 AC_CONFIG_FILES([tests/t_client.sh], [chmod +x tests/t_client.sh])
tests/unit_tests/Makefile.am +3 -0
index 33fefaac..f27cd90f 100644
@@ -2,4 +2,7 @@ AUTOMAKE_OPTIONS = foreign
 
 if ENABLE_UNITTESTS
 SUBDIRS = example_test openvpn plugins
+if OPENSSL_ENGINE
+SUBDIRS += engine-key
+endif
 endif
tests/unit_tests/engine-key/Makefile.am new file +24 -0
new file mode 100644
index 00000000..857459ed
@@ -0,0 +1,24 @@
+AUTOMAKE_OPTIONS = foreign
+
+check_LTLIBRARIES = libtestengine.la
+conffiles = openssl.cnf
+
+TESTS_ENVIRONMENT = srcdir="$(abs_srcdir)"; \
+	builddir="$(abs_builddir)"; \
+	top_builddir="$(top_builddir)"; \
+	top_srcdir="$(top_srcdir)"; \
+	export srcdir builddir top_builddir top_srcdir;
+
+TESTS = check_engine_keys.sh
+check_engine_keys.sh: $(conffiles)
+
+clean-local:
+	rm -f $(conffiles)
+
+$(builddir)/%.cnf: $(srcdir)/%.cnf.in
+	sed "s|ABSBUILDDIR|$(abs_builddir)|" < $< > $@
+
+libtestengine_la_SOURCES = libtestengine.c
+libtestengine_la_LDFLAGS = @TEST_LDFLAGS@ -rpath /lib -shrext .so
+libtestengine_la_CFLAGS = @TEST_CFLAGS@ -I$(openvpn_srcdir) -I$(compat_srcdir)
+
tests/unit_tests/engine-key/check_engine_keys.sh new file +30 -0
new file mode 100755
index 00000000..e0c9d7b0
@@ -0,0 +1,30 @@
+#!/bin/sh
+
+OPENSSL_CONF="${builddir}/openssl.cnf"
+export OPENSSL_CONF
+
+password='AT3S4PASSWD'
+
+key="${builddir}/client.key"
+pwdfile="${builddir}/passwd"
+
+# create an engine key for us
+sed 's/PRIVATE KEY/TEST ENGINE KEY/' < ${top_srcdir}/sample/sample-keys/client.key > ${key}
+echo "$password" > $pwdfile
+
+# note here we've induced a mismatch in the client key and the server
+# cert which openvpn should report and die.  Check that it does.  Note
+# also that this mismatch depends on openssl not openvpn, so it is
+# somewhat fragile
+${top_builddir}/src/openvpn/openvpn --cd ${top_srcdir}/sample --config sample-config-files/loopback-server --engine testengine --key ${key} --askpass $pwdfile > log.txt 2>&1
+
+# first off check we died because of a key mismatch.  If this doesn't
+# pass, suspect openssl of returning different messages and update the
+# test accordingly
+grep -q 'X509_check_private_key:key values mismatch' log.txt || { echo "Key mismatch not detected"; exit 1; }
+
+# now look for the engine prints (these are under our control)
+grep -q 'ENGINE: engine_init called' log.txt || { echo "Engine initialization not detected"; exit 1; }
+grep -q 'ENGINE: engine_load_key called' log.txt || { echo "Key was not loaded from engine"; exit 1; }
+grep -q "ENGINE: engine_load_key got password ${password}" log.txt || { echo "Key password was not retrieved by the engine"; exit 1; }
+exit 0
tests/unit_tests/engine-key/libtestengine.c new file +101 -0

101 lines not on this page · raw message

tests/unit_tests/engine-key/openssl.cnf.in new file +12 -0
new file mode 100644
index 00000000..5eda9fa9
@@ -0,0 +1,12 @@
+HOME		= .
+openssl_conf	= openssl_init
+
+[req]
+[openssl_init]
+engines		= engines_section
+
+[engines_section]
+testengine	= testengine_section
+
+[testengine_section]
+dynamic_path	= ABSBUILDDIR/.libs/libtestengine.so
Signature · 1 line
2.26.2
#24 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Hi,

Quoted 9 lines · James Bottomley
Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+eval shrext=$shrext_cmds
+AC_SUBST([shrext])

... I hoped we can get rid of this again...

In theory I can get around this using the libtool -shrext .so on the command line. Someone would have to check on windows that this is acceptable because they really like .dll as the extension. If it works on windows then the below patch should do it.

Oh, Windows.

Production builds for windows are done with MinGW on Ubuntu (cross-builds) and we do not run any unit tests there.

Lev does run MSVC builds, but he's not using "make", so I'm not sure he ever runs "make check"...

What would be needed to actually run this test on windows? Or any of our tests? "MinGW/Cygwin on windows"? Has anyone built with that since XP? (My first openvpn.exe was built with MinGW/Cygwin, but that was painful)

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#7 lore raw

Re: [Openvpn-devel] [PATCH v6 2/3] crypto_openssl: add initialization to pick up local configuration

In reply to James Bottomley

Hi,

src/openvpn/crypto_openssl.c crypto_init_lib_engine(const char *engine_name) · line 156

Quoted 13 lines
src/openvpn/crypto_openssl.c +5 -0
index 4ac77fde..fd57edd2 100644
@@ -149,6 +149,11 @@ crypto_init_lib_engine(const char *engine_name)
 void
 crypto_init_lib(void)
 {
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL);
+#else
+    OPENSSL_config(NULL);
+#endif

This is failing on some of the Travis builds, with the error message

1191 crypto_openssl.c: In function `crypto_init_lib':
1192 crypto_openssl.c:155:5: error: implicit declaration of function OPENSSL_config'; did you mean OPENSSL_init'? [-Werror=implicit-function-declaration]
1193 OPENSSL_config(NULL);
1194 ^~~~~~~~~~~~~~

https://travis-ci.org/github/OpenVPN/openvpn/builds/695633823

#8 lore raw

Re: [Openvpn-devel] [PATCH v6 2/3] crypto_openssl: add initialization to pick up local configuration

Quoted 29 lines · Gert Doering

Hi,

Quoted 6 lines
src/openvpn/crypto_openssl.c +0 -0
index 4ac77fde..fd57edd2 100644

src/openvpn/crypto_openssl.c crypto_init_lib_engine(const char · line 157

Quoted 9 lines
src/openvpn/crypto_openssl.c +5 -0
@@ -149,6 +149,11 @@ crypto_init_lib_engine(const char
 *engine_name)
 void
 crypto_init_lib(void)
 {
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CONFIG, NULL);
+#else
+    OPENSSL_config(NULL);
+#endif

This is failing on some of the Travis builds, with the error message

1191 crypto_openssl.c: In function `crypto_init_lib':
1192 crypto_openssl.c:155:5: error: implicit declaration of function OPENSSL_config'; did you mean OPENSSL_init'? [-Werror=implicit- function-declaration]
1193 OPENSSL_config(NULL);
1194 ^~~~~~~~~~~~~~

https://travis-ci.org/github/OpenVPN/openvpn/builds/695633823

Quoted 7 lines

From what I can see, this is for the builds with "openssl 1.0.1u" and "openssl 1.0.2u" on Bionic.

I'm not sure if we intend to support these versions, but this needs to be fixed - either the code needs to be adjusted or the travis build matrix (for master).

Sorry about that. Best guess is it's missing an include for openssl/conf.h. You don't need that today because pretty much every other openssl header includes it, but that may not always have been so.

Does the below patch fix it? If it does, it should probably be folded into the other patch. It should be safe because openssl/conf.h has existed for every version of openssl you support.

James

---8>8>8><8<8<8---
From: James Bottomley <[email protected]...>
Subject: [PATCH] crypto_openssl: add include for openssl/conf.h

Fix build failure on older versions of openssl.

---
 src/openvpn/crypto_openssl.c | 1 +
 1 file changed, 1 insertion(+)
src/openvpn/crypto_openssl.c +1 -0
index fd57edd2..94b6d85b 100644
@@ -43,6 +43,7 @@
 #include "crypto_backend.h"
 #include "openssl_compat.h"
 
+#include <openssl/conf.h>
 #include <openssl/des.h>
 #include <openssl/err.h>
 #include <openssl/evp.h>
Signature · 1 line
2.26.2
Acked-by
#9 lore raw

Re: [Openvpn-devel] [PATCH v6 2/3] crypto_openssl: add initialization to pick up local configuration

Quoted 9 lines

Sorry about that. Best guess is it's missing an include for openssl/conf.h. You don't need that today because pretty much every other openssl header includes it, but that may not always have been so.

Does the below patch fix it? If it does, it should probably be folded into the other patch. It should be safe because openssl/conf.h has existed for every version of openssl you support.

I am also puzzeld by this. On my local machine the OpenSSL 1.0.2 buildis fine without this extra include.

But I am ACKing this alone on the basis that including the conf.h header is the corect thing to do (the man page says it should be included)

It fixes the travis build error
(https://travis-ci.org/github/schwabe/openvpn/builds/695947378) but I do not really understand why OpenSSL behaves different there.

So

Quoted 27 lines

James

---8>8>8><8<8<8---
From: James Bottomley <[email protected]...>
Subject: [PATCH] crypto_openssl: add include for openssl/conf.h

Fix build failure on older versions of openssl.

---
 src/openvpn/crypto_openssl.c | 1 +
 1 file changed, 1 insertion(+)
src/openvpn/crypto_openssl.c +1 -0
index fd57edd2..94b6d85b 100644
@@ -43,6 +43,7 @@
 #include "crypto_backend.h"
 #include "openssl_compat.h"
 
+#include <openssl/conf.h>
 #include <openssl/des.h>
 #include <openssl/err.h>
 #include <openssl/evp.h>
#10 lore raw

Re: [Openvpn-devel] [PATCH v6 2/3] crypto_openssl: add initialization to pick up local configuration

пн, 8 июн. 2020 г. в 15:06, Arne Schwabe <[email protected]...>:

Quoted 14 lines
Quoted 9 lines

Sorry about that. Best guess is it's missing an include for openssl/conf.h. You don't need that today because pretty much every other openssl header includes it, but that may not always have been so.

Does the below patch fix it? If it does, it should probably be folded into the other patch. It should be safe because openssl/conf.h has existed for every version of openssl you support.

I am also puzzeld by this. On my local machine the OpenSSL 1.0.2 buildis fine without this extra include.

Quoted 7 lines

But I am ACKing this alone on the basis that including the conf.h header is the corect thing to do (the man page says it should be included)

It fixes the travis build error
(https://travis-ci.org/github/schwabe/openvpn/builds/695947378) but I do not really understand why OpenSSL behaves different there.

it might happen if includes are mixed from OS openssl and custom openssl. I'll have a look (what is include path in travis).

Quoted 39 lines

So

Quoted 20 lines

James

---8>8>8><8<8<8---
From: James Bottomley <[email protected]...>
Subject: [PATCH] crypto_openssl: add include for openssl/conf.h

Fix build failure on older versions of openssl.

---
 src/openvpn/crypto_openssl.c | 1 +
 1 file changed, 1 insertion(+)
src/openvpn/crypto_openssl.c +0 -0
index fd57edd2..94b6d85b 100644

src/openvpn/crypto_openssl.c line 49

Quoted 7 lines
src/openvpn/crypto_openssl.c +1 -0
@@ -43,6 +43,7 @@
 #include "crypto_backend.h"
 #include "openssl_compat.h"
 
+#include <openssl/conf.h>
 #include <openssl/des.h>
 #include <openssl/err.h>
 #include <openssl/evp.h>

_______________________________________________
Openvpn-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/openvpn-devel

#11 lore raw

[Openvpn-devel] [PATCH applied] Re: crypto_openssl: add include for openssl/conf.h

In reply to James Bottomley

Your patch has been applied to the master branch.

commit 25266ebba97d7f4169f902b8b0d3c38eaa4c43a4

Author: James Bottomley
Date:   Sun Jun 7 15:10:58 2020 -0700
     crypto_openssl: add include for openssl/conf.h

     Signed-off-by: James Bottomley <[email protected]...>
     Acked-by: Arne Schwabe <[email protected]...>
     Message-Id: <[email protected]...>
     URL: https://www.mail-archive.com/[email protected]/msg19996.html
     Signed-off-by: Gert Doering <[email protected]...>

--
kind regards,

Gert Doering

#25 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

In reply to James Bottomley

Hi,

Quoted 8 lines · James Bottomley

Testing engines is problematic, so one of the prerequisites built for the tests is a simple openssl engine that reads a non-standard PEM guarded key. The test is simply can we run a client/server configuration with the usual sample key replaced by an engine key. The trivial engine prints out some operations and we check for these in the log to make sure the engine was used to load the key and that it correctly got the password.

Quoted 7 lines

---
v5: do not hard code dynamic library extension into openssl.cnf (MacOS)
v4: add OPENSSL_config(NULL) so debian checks will work
v3: added this patch
---

So, how can we continue with this one? The engine key code is in, the unit test code not.

Arne, James, can we converge on something here?

I would like something "as simple as possible", aka "little configure magic"
- if that means "MacOS engine is built and loaded as .so and not .dynlib, but the test passes" I would prefer it over "more configure.ac magic" - but if this is what it takes, I'm fine as well.

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#26 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Quoted 21 lines · Gert Doering

Hi,

Quoted 9 lines · James Bottomley

Testing engines is problematic, so one of the prerequisites built for the tests is a simple openssl engine that reads a non-standard PEM guarded key. The test is simply can we run a client/server configuration with the usual sample key replaced by an engine key. The trivial engine prints out some operations and we check for these in the log to make sure the engine was used to load the key and that it correctly got the password.

Quoted 8 lines

om>

---
v5: do not hard code dynamic library extension into openssl.cnf (MacOS)
v4: add OPENSSL_config(NULL) so debian checks will work
v3: added this patch
---

Quoted 4 lines

So, how can we continue with this one? The engine key code is in, the unit test code not.

Arne, James, can we converge on something here?

Could someone just test the proposed updated v6 patch on a Mac?

https://sourceforge.net/p/openvpn/mailman/message/37031113/

If it works then we have something that functions on both platforms
and we don't need to worry about windows until someone runs unit tests there.

Quoted 6 lines

I would like something "as simple as possible", aka "little configure magic" - if that means "MacOS engine is built and loaded as .so and not .dynlib, but the test passes" I would prefer it over "more configure.ac magic"
-
but if this is what it takes, I'm fine as well.

Right, that's what the patch in the url does: uses .so on both mac and linux.

James

#27 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Hi,

Quoted 5 lines · James Bottomley
Quoted 1 line

Arne, James, can we converge on something here?

Could someone just test the proposed updated v6 patch on a Mac?

https://sourceforge.net/p/openvpn/mailman/message/37031113/

Took Arne and me half a day ("the macos build slave did not have cmocka, and it went downhill from there") but I can now confirm that it works and passes the tests.

This is on a Mojave MacMini with "brew" openssl 1.1.1c and your patch "v6":

Making check in engine-key
/Applications/Xcode.app/Contents/Developer/usr/bin/make libtestengine.la
...
/bin/sh ../../../libtool --tag=CC --mode=link gcc -I/usr/local/Cellar/[email protected].../1.1.1c/include -I../../../../openvpn.git/include -I/usr/local/Cellar/cmocka/1.1.5/include -I -I -Wall -Wno-unused-parameter -Wno-unused-function -g -O2 -std=c99 -L/usr/local/Cellar/[email protected].../1.1.1c/lib -lcrypto -lssl -L/usr/local/Cellar/cmocka/1.1.5/lib -lcmocka -rpath /lib -shrext .so -o libtestengine.la libtestengine_la-libtestengine.lo -lresolv
libtool: link: gcc -dynamiclib -Wl,-undefined -Wl,dynamic_lookup -o .libs/libtestengine.0.so .libs/libtestengine_la-libtestengine.o -L/usr/local/Cellar/[email protected].../1.1.1c/lib -lcrypto -lssl -L/usr/local/Cellar/cmocka/1.1.5/lib -lcmocka -lresolv -g -O2 -install_name /lib/libtestengine.0.so -compatibility_version 1 -current_version 1.0 -Wl,-single_module
libtool: link: (cd ".libs" && rm -f "libtestengine.so" && ln -s "libtestengine.0.so" "libtestengine.so")
libtool: link: ar cru .libs/libtestengine.a libtestengine_la-libtestengine.o
libtool: link: ranlib .libs/libtestengine.a
libtool: link: ( cd ".libs" && rm -f "libtestengine.la" && ln -s "../libtestengine.la" "libtestengine.la" ) /Applications/Xcode.app/Contents/Developer/usr/bin/make check-TESTS
sed "s|ABSBUILDDIR|/Users/gert/openvpn-build-mojave/tests/unit_tests/engine-key|" < ../../../../openvpn.git/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf
PASS: check_engine_keys.sh
=============
1 test passed
=============

(I'm not sure where it is actually *calling* the "check_engine_keys.sh", but it says "test passed!" and I see ".so" up there)

[..]

Quoted 2 lines

Right, that's what the patch in the url does: uses .so on both mac and linux.

I got all confused with your "v6" patch and your "v7" patch, which did other things, and but did not have a "v6" in the history :-)

*This* patch definitely looks like it's doing the right thing on MacOS.

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#28 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Quoted 51 lines · Gert Doering

Hi,

Quoted 5 lines · James Bottomley
Quoted 1 line

Arne, James, can we converge on something here?

Could someone just test the proposed updated v6 patch on a Mac?

https://sourceforge.net/p/openvpn/mailman/message/37031113/

Took Arne and me half a day ("the macos build slave did not have cmocka, and it went downhill from there") but I can now confirm that it works and passes the tests.

This is on a Mojave MacMini with "brew" openssl 1.1.1c and your patch "v6":

Making check in engine-key /Applications/Xcode.app/Contents/Developer/usr/bin/make libtestengin e.la
...
/bin/sh ../../../libtool --tag=CC --mode=link gcc -I/usr/local/Cel
lar/[email protected].../1.1.1c/include -I../../../../openvpn.git/include
-I/usr/local/Cellar/cmocka/1.1.5/include -I -I -Wall -Wno-unused- parameter -Wno-unused-function -g -O2 -std=c99 -L/usr/local/Cellar/op
[email protected].../1.1.1c/lib -lcrypto -lssl -
L/usr/local/Cellar/cmocka/1.1.5/lib -lcmocka -rpath /lib -shrext .so -o libtestengine.la libtestengine_la-libtestengine.lo - lresolv
libtool: link: gcc -dynamiclib -Wl,-undefined -Wl,dynamic_lookup -o
.libs/libtestengine.0.so .libs/libtestengine_la-libtestengine.o -L
/usr/local/Cellar/[email protected].../1.1.1c/lib -lcrypto -lssl
-L/usr/local/Cellar/cmocka/1.1.5/lib -lcmocka -lresolv -g -O2 -
install_name /lib/libtestengine.0.so -compatibility_version 1 -current_version 1.0 -Wl,-single_module
libtool: link: (cd ".libs" && rm -f "libtestengine.so" && ln -s "libtestengine.0.so" "libtestengine.so")
libtool: link: ar cru .libs/libtestengine.a libtestengine_la- libtestengine.o
libtool: link: ranlib .libs/libtestengine.a
libtool: link: ( cd ".libs" && rm -f "libtestengine.la" && ln -s "../libtestengine.la" "libtestengine.la" ) /Applications/Xcode.app/Contents/Developer/usr/bin/make check-TESTS sed "s|ABSBUILDDIR|/Users/gert/openvpn-build- mojave/tests/unit_tests/engine-key|" < ../../../../openvpn.git/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf
PASS: check_engine_keys.sh
=============
1 test passed
=============

Quoted 3 lines

(I'm not sure where it is actually *calling* the "check_engine_keys.sh",
but it says "test passed!" and I see ".so" up there)

Yes, it's using the standard test runner, so if it says PASS/FAIL:
<script> it definitely called <script>

Quoted 9 lines

[..]

Quoted 2 lines

Right, that's what the patch in the url does: uses .so on both mac and linux.

I got all confused with your "v6" patch and your "v7" patch, which did other things, and but did not have a "v6" in the history :-)

*This* patch definitely looks like it's doing the right thing on MacOS.

Great, thanks ... I'll rewrap it and submit as a separate patch email with cover letter.

James

#29 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Hi,

Quoted 4 lines · James Bottomley
Quoted 4 lines

[..]

Quoted 2 lines

Right, that's what the patch in the url does: uses .so on both mac and linux.

Quoted 8 lines
Quoted 5 lines

I got all confused with your "v6" patch and your "v7" patch, which did other things, and but did not have a "v6" in the history :-)

*This* patch definitely looks like it's doing the right thing on MacOS.

Great, thanks ... I'll rewrap it and submit as a separate patch email with cover letter.

I was about to just merge what I have, but it exploded on me on FreeBSD now - and I can't really figure out why (yet)...

libtool: link: (cd ".libs" && rm -f "libtestengine.so.0" && ln -s "libtestengine.so.0.0.0" "libtestengine.so.0")
libtool: link: (cd ".libs" && rm -f "libtestengine.so" && ln -s "libtestengine.so.0.0.0" "libtestengine.so")
libtool: link: ar cru .libs/libtestengine.a libtestengine_la-libtestengine.o
libtool: link: ranlib .libs/libtestengine.a
libtool: link: ( cd ".libs" && rm -f "libtestengine.la" && ln -s "../libtestengine.la" "libtestengine.la" ) make check-TESTS
make[5]: don't know how to make openssl.cnf. Stop

make[5]: stopped in /home/gert/src/openvpn-maint/test-build-master-fbsd/tests/unit_tests/engine-key

it seems as if there is an implicit rule "somewhere" that Linux make has and FreeBSD make not how to build openssl.cnf from $(srcdir)/openssl.cnf.in
- on Linux, when I remove "openssl.cnf" from the target dir and re-run "make check", I get

make[5]: Entering directory '/rhome/gert/src/openvpn-maint/test-build-master-ossl111/tests/unit_tests/engine-key'
sed "s|ABSBUILDDIR|/rhome/gert/src/openvpn-maint/test-build-master-ossl111/tests/unit_tests/engine-key|" < ../../../../openvpn/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf
PASS: check_engine_keys.sh

Ah. If I run "gmake check", it also passes on FreeBSD...

gmake[5]: Entering directory '/home/gert/src/openvpn-maint/test-build-master-fbsd/tests/unit_tests/engine-key'
sed "s|ABSBUILDDIR|/home/gert/src/openvpn-maint/test-build-master-fbsd/tests/unit_tests/engine-key|" < ../../../../openvpn/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf
PASS: check_engine_keys.sh
=============
1 test passed
=============

... so the engine part *itself* is fine, just the "openssl.cnf from .in" doesn't work with BSD make...

We need a v7 (or v8?) :-(

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#30 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Quoted 69 lines · Gert Doering

Hi,

Quoted 9 lines · James Bottomley
Quoted 9 lines

[..]

Quoted 3 lines

Right, that's what the patch in the url does: uses .so on both mac
and linux.

I got all confused with your "v6" patch and your "v7" patch, which
did other things, and but did not have a "v6" in the history :-)

Quoted 6 lines
Quoted 2 lines

*This* patch definitely looks like it's doing the right thing on MacOS.

Great, thanks ... I'll rewrap it and submit as a separate patch email
with cover letter.

I was about to just merge what I have, but it exploded on me on FreeBSD
now - and I can't really figure out why (yet)...

libtool: link: (cd ".libs" && rm -f "libtestengine.so.0" && ln -s "libtestengine.so.0.0.0" "libtestengine.so.0")
libtool: link: (cd ".libs" && rm -f "libtestengine.so" && ln -s "libtestengine.so.0.0.0" "libtestengine.so")
libtool: link: ar cru .libs/libtestengine.a libtestengine_la- libtestengine.o
libtool: link: ranlib .libs/libtestengine.a
libtool: link: ( cd ".libs" && rm -f "libtestengine.la" && ln -s "../libtestengine.la" "libtestengine.la" )
make check-TESTS
make[5]: don't know how to make openssl.cnf. Stop

make[5]: stopped in /home/gert/src/openvpn-maint/test-build-master- fbsd/tests/unit_tests/engine-key

it seems as if there is an implicit rule "somewhere" that Linux make has
and FreeBSD make not how to build openssl.cnf from $(srcdir)/openssl.cnf.in
- on Linux, when I remove "openssl.cnf" from the target dir and re- run
"make check", I get

make[5]: Entering directory '/rhome/gert/src/openvpn-maint/test- build-master-ossl111/tests/unit_tests/engine-key'
sed "s|ABSBUILDDIR|/rhome/gert/src/openvpn-maint/test-build-master- ossl111/tests/unit_tests/engine-key|" < ../../../../openvpn/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf
PASS: check_engine_keys.sh

Ah. If I run "gmake check", it also passes on FreeBSD...

gmake[5]: Entering directory '/home/gert/src/openvpn-maint/test- build-master-fbsd/tests/unit_tests/engine-key'
sed "s|ABSBUILDDIR|/home/gert/src/openvpn-maint/test-build-master- fbsd/tests/unit_tests/engine-key|" < ../../../../openvpn/tests/unit_tests/engine-key/openssl.cnf.in > openssl.cnf
PASS: check_engine_keys.sh
=============
1 test passed
=============

Quoted 5 lines

... so the engine part *itself* is fine, just the "openssl.cnf from .in" doesn't work with BSD make...

We need a v7 (or v8?) :-(

That will be my fault. I assumed automake always ran with gnu make, so the .cnf rule is in gnu make syntax at the bottom of (Makefile.am):

$(builddir)/%.cnf: $(srcdir)/%.cnf.in

	sed "s|ABSBUILDDIR|$(abs_builddir)|" < $< > $@

If I just spell everything out and don't use the percent wildcard, it should work even on legacy make.

James

#31 lore raw

Re: [Openvpn-devel] [PATCH v6 3/3] Add unit tests for engine keys

Hi,

Quoted 1 line · James Bottomley

That will be my fault. I assumed automake always ran with gnu make,

No :-) (and let's not start a gnu make vs bsd make vs. cmake vs. ant
discussion now :-) ).

Specifically, we run automake on linux systems before doing release tarballs, so the target host doesn't even *need* automake installed - but if you want, automake runs just fine on BSD, producing "standard makefiles" that work fine with BSD make.

Quoted 8 lines

so the .cnf rule is in gnu make syntax at the bottom of (Makefile.am):

$(builddir)/%.cnf: $(srcdir)/%.cnf.in

	sed "s|ABSBUILDDIR|$(abs_builddir)|" < $< > $@

If I just spell everything out and don't use the percent wildcard, it should work even on legacy make.

Ah, there it is :) - I think "non-GNU" make doesn't handle these in combination with paths.

"man make" on FreeBSD says:

The $@ and $< variables are more or less universally portable, as is the $(MAKE) variable. Basic use of suffix rules (for files only in the current directory, not trying to chain transformations together, etc.) is also reasonably portable.

so

$(builddir)/openvpn.cnf: $(srcdir)/openvpn.cnf.in

	sed "s|ABSBUILDDIR|$(abs_builddir)|" < $< > $@

should work.

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...
#32 lore raw

Re: [Openvpn-devel] engine-test on FreeBSD still not working

Hi,

Quoted 6 lines · Gert Doering

so

$(builddir)/openvpn.cnf: $(srcdir)/openvpn.cnf.in

	sed "s|ABSBUILDDIR|$(abs_builddir)|" < $< > $@

should work.

It doesn't :-(

It worked for my tests of that patch, because the openssl.cnf generated with "gmake test" was still in that directory - but today I created new build directories to test --enable-small, and it doesn't - seems it is not understanding $(builddir)/openvpn.cnf: as destination, nor does "< $<" work.

What works is this:

openssl.cnf: $(srcdir)/openssl.cnf.in

        sed "s|ABSBUILDDIR|$(abs_builddir)|" < $(srcdir)/openssl.cnf.in > $@

(tested both for clean in-tree and out-of-tree builds)

Can we have a new (LAST!!!) patch, please...

gert

Signature · 6 lines
"If was one thing all people took for granted, was conviction that if you 
 feed honest figures into a computer, honest figures come out. Never doubted 
 it myself till I met a computer with a sense of humor."
                             Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany                             [email protected]...