Skip to contentSkip to conversation

Add support for OpenSSL Provider API

PATCH Superseded Eddie Kovsky 5 participants 15 messages Apr 29 – Oct 28

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 Add support for OpenSSL Provider API

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Changes in v3:
- Removed Kconfig option
- Changed macro symbol from CONFIG_OPENSSL_NO_DEPRECATED to USE_PKCS11_PROVIDER or USE_PKCS11_ENGINE
v2: https://lore.kernel.org/u-boot/[email protected]/

Changes in v2:
- Remove default for new Kconfig option
- Use #ifdef instead of IS_ENABLED macro
- Remove comment after #endif
- Remove unrelated checkpatch cleanup of 'sslErr' variable name
v1: https://lore.kernel.org/u-boot/[email protected]/

---
 doc/build/gcc.rst       |   4 +-
 lib/aes/aes-encrypt.c   |   4 +-
 lib/rsa/rsa-sign.c      | 102 ++++++++++++++++++++++++++++++++++++++--
 tools/docker/Dockerfile |   1 +
 4 files changed, 103 insertions(+), 8 deletions(-)
doc/build/gcc.rst +2 -2
index 1fef718ceecb..29a6a632e7e3 100644
@@ -25,8 +25,8 @@ Depending on the build targets further packages maybe needed
 
     sudo apt-get install bc bison build-essential coccinelle \
       device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
-      libgnutls28-dev libguestfs-tools libncurses-dev \
-      libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
+      libgnutls28-dev libguestfs-tools libncurses-dev libpython3-dev \
+      libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl pkcs11-provider \
       pkg-config python3 python3-asteval python3-coverage python3-filelock \
       python3-pkg-resources python3-pycryptodome python3-pyelftools \
       python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
lib/aes/aes-encrypt.c +3 -1
index 90e1407b4f09..4fc4ce232478 100644
@@ -16,7 +16,9 @@
 #include <openssl/err.h>
 #include <openssl/ssl.h>
 #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+# include <openssl/engine.h>
+#endif
 #include <uboot_aes.h>
 
 #if OPENSSL_VERSION_NUMBER >= 0x10000000L
lib/rsa/rsa-sign.c +97 -5

234 lines not on this page · raw message

tools/docker/Dockerfile +1 -0
index 73bf6cdd2c52..50e98e83dc20 100644
@@ -122,6 +122,7 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
 	openssl \
 	picocom \
 	parted \
+	pkcs11-provider \
 	pkg-config \
 	python-is-python3 \
 	python3 \
Signature · 1 line
2.53.0
Reviewed-by on v4 1/1
#1 lore raw

Hi Eddie,

Thank you for the patch.

Quoted 17 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

Quoted 7 lines

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

Quoted 1 line

---

#2 lore raw
In reply to Eddie Kovsky

Hi Eddie,

Quoted 15 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

Quoted 6 lines

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

But does it actually use a provider or an engine to begin with? I don't see test/py/tests/test_vboot.py calling mkimage with the -N argument. What are the tests (or command) you ran to validate this? I briefly saw the CI failed in v3 because a package was missing, but wasn't it simply because the headers or provider libraries which are now necessary for building lib/rsa/rsa-sign.c were not present? The logs aren't available anymore unfortunately. If that's the case, then that's also an issue. We shouldn't need to install providers if we aren't going to use any? Yes, I know that we currently cannot compile if we don't have openssl-devel-engine (on Fedora), but if we can improve the situation, we should.

How did you test (locally is fine) with providers?

Quoted 2 lines

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

Please do not forget to add the colon after Tested-by so it actually makes it a git trailer instead of just text.

Quoted 47 lines

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Changes in v3:
- Removed Kconfig option
- Changed macro symbol from CONFIG_OPENSSL_NO_DEPRECATED to USE_PKCS11_PROVIDER or USE_PKCS11_ENGINE
v2: https://lore.kernel.org/u-boot/[email protected]/

Changes in v2:
- Remove default for new Kconfig option
- Use #ifdef instead of IS_ENABLED macro
- Remove comment after #endif
- Remove unrelated checkpatch cleanup of 'sslErr' variable name
v1: https://lore.kernel.org/u-boot/[email protected]/

---
  doc/build/gcc.rst       |   4 +-
  lib/aes/aes-encrypt.c   |   4 +-
  lib/rsa/rsa-sign.c      | 102 ++++++++++++++++++++++++++++++++++++++--
  tools/docker/Dockerfile |   1 +
  4 files changed, 103 insertions(+), 8 deletions(-)
doc/build/gcc.rst +2 -2
index 1fef718ceecb..29a6a632e7e3 100644
@@ -25,8 +25,8 @@ Depending on the build targets further packages maybe needed
 
      sudo apt-get install bc bison build-essential coccinelle \
        device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
-      libgnutls28-dev libguestfs-tools libncurses-dev \
-      libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
+      libgnutls28-dev libguestfs-tools libncurses-dev libpython3-dev \
+      libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl pkcs11-provider \
        pkg-config python3 python3-asteval python3-coverage python3-filelock \
        python3-pkg-resources python3-pycryptodome python3-pyelftools \
        python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
lib/aes/aes-encrypt.c +0 -0
index 90e1407b4f09..4fc4ce232478 100644

lib/aes/aes-encrypt.c line 21

Quoted 7 lines
lib/aes/aes-encrypt.c +3 -1
@@ -16,7 +16,9 @@
  #include <openssl/err.h>
  #include <openssl/ssl.h>
  #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+# include <openssl/engine.h>
+#endif

Considering there are no other changes in this file, is this include actually needed?

Quoted 14 lines
lib/aes/aes-encrypt.c +0 -0
  #include <uboot_aes.h>
 
  #if OPENSSL_VERSION_NUMBER >= 0x10000000L
lib/rsa/rsa-sign.c +2 -1
index 0e38c9e802fd..f456f3c58e65 100644
@@ -19,7 +19,47 @@
  #include <openssl/err.h>
  #include <openssl/ssl.h>
  #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if OPENSSL_VERSION_MAJOR >= 3
+# define USE_PKCS11_PROVIDER

lib/rsa/rsa-sign.c line 33

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+# include <err.h>
+# include <openssl/provider.h>
+# include <openssl/store.h>
+#else
+# if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+#  define USE_PKCS11_ENGINE
+#  include <openssl/engine.h>
+# endif
+#endif
+

Sorry but that's a NACK.

As far as I can tell, this effectively disables using engines when your host openssl version is 3.0+, which we currently support (and I currently use it). You can have this for 4.0+ as engine support has been removed, but not for 3.x.

lib/rsa/rsa-sign.c line 43

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+#ifdef USE_PKCS11_PROVIDER
+#define ERR(cond, fmt, ...)				\
+	do {						\
+		bool __cond = (cond);			\
+		drain_openssl_errors(__LINE__, 0);	\
+		if (__cond) {				\
+			errx(1, fmt, ## __VA_ARGS__);	\
+		}					\
+	} while (0)
+

Is this really related to the PKCS11 provider? I think there's a mix between "using the provider API" and "using the pkcs11 provider".

lib/rsa/rsa-sign.c line 54

Quoted 11 lines
lib/rsa/rsa-sign.c +11 -0
+static void drain_openssl_errors(int l, int silent)
+{
+	const char *file;
+	char buf[120];
+	int e, line;
+
+	if (ERR_peek_error() == 0)
+		return;
+	if (!silent)
+		fprintf(stderr, "At main.c:%d:\n", l);
+

main.c?

Quoted 37 lines
lib/rsa/rsa-sign.c +12 -2
+	while ((e = ERR_peek_error_line(&file, &line))) {
+		ERR_error_string(e, buf);
+		if (!silent)
+			fprintf(stderr, "- SSL %s: %s:%d\n", buf, file, line);
+		ERR_get_error();
+	}
+}
+#endif
 
  static int rsa_err(const char *msg)
  {
@@ -94,10 +134,11 @@ err_cert:
   *
   * @keydir:	Key prefix
   * @name	Name of key
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
+#ifdef USE_PKCS11_ENGINE
  static int rsa_engine_get_pub_key(const char *keydir, const char *name,
  				  ENGINE *engine, EVP_PKEY **evpp)
  {
@@ -157,21 +198,24 @@ static int rsa_engine_get_pub_key(const char *keydir, const char *name,
 
  	return 0;
  }
+#endif
 
  /**
   * rsa_get_pub_key() - read a public key
   *
   * @keydir:	Directory containing the key (PEM file) or key prefix (engine)
   * @name	Name of key file (will have a .crt extension)
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider

lib/rsa/rsa-sign.c static int rsa_engine_get_pub_key(const char *keydir, const char *name, · line 218

Quoted 10 lines
lib/rsa/rsa-sign.c +2 -0
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
  static int rsa_get_pub_key(const char *keydir, const char *name,
  			   ENGINE *engine, EVP_PKEY **evpp)
  {
+#ifdef USE_PKCS11_ENGINE
  	if (engine)
  		return rsa_engine_get_pub_key(keydir, name, engine, evpp);
+#endif

We should probably at least print something when engines aren't supported but the engine parameter is passed, or maybe even fail as that's an incorrect configuration.

Quoted 6 lines
lib/rsa/rsa-sign.c +0 -0
  	return rsa_pem_get_pub_key(keydir, name, evpp);
  }
 
@@ -207,13 +251,44 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
  		return -ENOENT;
  	}

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 262

Quoted 10 lines
lib/rsa/rsa-sign.c +9 -0
 
+#ifdef USE_PKCS11_PROVIDER
+	EVP_PKEY *private_key = NULL;
+	OSSL_STORE_CTX *store;
+
+	if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
+		ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
+	if (!OSSL_PROVIDER_try_load(NULL, "default", true))
+		ERR(1, "OSSL_PROVIDER_try_load(default)");
+

Why are we unconditionally loading providers and specifically the pkcs11 one? As far as I could tell we should know from the URI whether a key from a provider is requested by checking if a colon is in the URI. And we can automatically load the requested provider based on that?

Otherwise the easiest answer to this could simply be: "let the user configure this externally via OPENSSL_CONF environment variable".

Do we also need to update tools/mkimage.c (and doc/mkimage.1) to remove -N engine from usage if the tool doesn't support it? Do we also need to update doc/usage/fit/signature.rst to explain how to use pkcs11 provider?

FYI, we (my employer) are migrating from engines to providers for signing FIT images with binman so I'm starting to have a look on how to do this now.

Cheers,
Quentin

#3 lore raw
Quoted 36 lines · Quentin Schulz

Hi Eddie,

Quoted 15 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

Quoted 6 lines

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

But does it actually use a provider or an engine to begin with? I don't see test/py/tests/test_vboot.py calling mkimage with the -N argument. What are the tests (or command) you ran to validate this? I briefly saw the CI failed in v3 because a package was missing, but wasn't it simply because the headers or provider libraries which are now necessary for building lib/rsa/rsa-sign.c were not present? The logs aren't available anymore unfortunately. If that's the case, then that's also an issue. We shouldn't need to install providers if we aren't going to use any? Yes, I know that we currently cannot compile if we don't have openssl-devel-engine (on Fedora), but if we can improve the situation, we should.

Quoted 2 lines

How did you test (locally is fine) with providers?

The FIT signature verification tests are documented here:

https://docs.u-boot.org/en/latest/usage/fit/signature.html#u-boot-fit-signature-verification

The tests currently fail in build environments (like Fedora) that don't have engine support. This is how we originally became aware of the API issue last year.

❯ ./test/py/test.py --bd sandbox --build -k vboot
+make O=u-boot/build-sandbox -s sandbox_defconfig
+make O=u-boot/build-sandbox -s -j8
In file included from tools/generated/lib/aes/aes-encrypt.c:1:
../tools/../lib/aes/aes-encrypt.c:19:10: fatal error: openssl/engine.h: No such file or directory 19 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
In file included from tools/generated/lib/rsa/rsa-sign.c:1:
../tools/../lib/rsa/rsa-sign.c:22:10: fatal error: openssl/engine.h: No such file or directory 22 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.

Github provides the Azure CI pipeline as a free service. I wouldn't expect them to retain logs at that tier.

Quoted 6 lines
Quoted 2 lines

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

Please do not forget to add the colon after Tested-by so it actually makes it a git trailer instead of just text.

That's an unlucky typo because I added the trailers manually.

Quoted 48 lines
Quoted 48 lines

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Changes in v3:
- Removed Kconfig option
- Changed macro symbol from CONFIG_OPENSSL_NO_DEPRECATED to USE_PKCS11_PROVIDER or USE_PKCS11_ENGINE
v2: https://lore.kernel.org/u-boot/[email protected]/

Changes in v2:
- Remove default for new Kconfig option
- Use #ifdef instead of IS_ENABLED macro
- Remove comment after #endif
- Remove unrelated checkpatch cleanup of 'sslErr' variable name
v1: https://lore.kernel.org/u-boot/[email protected]/

---
  doc/build/gcc.rst       |   4 +-
  lib/aes/aes-encrypt.c   |   4 +-
  lib/rsa/rsa-sign.c      | 102 ++++++++++++++++++++++++++++++++++++++--
  tools/docker/Dockerfile |   1 +
  4 files changed, 103 insertions(+), 8 deletions(-)
doc/build/gcc.rst +2 -2
index 1fef718ceecb..29a6a632e7e3 100644
@@ -25,8 +25,8 @@ Depending on the build targets further packages maybe needed
      sudo apt-get install bc bison build-essential coccinelle \
        device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
-      libgnutls28-dev libguestfs-tools libncurses-dev \
-      libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
+      libgnutls28-dev libguestfs-tools libncurses-dev libpython3-dev \
+      libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl pkcs11-provider \
        pkg-config python3 python3-asteval python3-coverage python3-filelock \
        python3-pkg-resources python3-pycryptodome python3-pyelftools \
        python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
lib/aes/aes-encrypt.c +0 -0
index 90e1407b4f09..4fc4ce232478 100644
@@ -16,7 +16,9 @@
  #include <openssl/err.h>
  #include <openssl/ssl.h>
Quoted 9 lines

lib/aes/aes-encrypt.c line 21

Quoted 5 lines
lib/aes/aes-encrypt.c +3 -1
  #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+# include <openssl/engine.h>
+#endif

Considering there are no other changes in this file, is this include actually needed?

The header is still needed to maintain backward compatibility for engine support. I maintained the #ifdef style to be consistent with its usage elsewhere, based on previous feedback.

Quoted 26 lines
Quoted 13 lines
lib/aes/aes-encrypt.c +0 -0
  #include <uboot_aes.h>
  #if OPENSSL_VERSION_NUMBER >= 0x10000000L
lib/rsa/rsa-sign.c +2 -1
index 0e38c9e802fd..f456f3c58e65 100644
@@ -19,7 +19,47 @@
  #include <openssl/err.h>
  #include <openssl/ssl.h>
  #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if OPENSSL_VERSION_MAJOR >= 3
+# define USE_PKCS11_PROVIDER

lib/rsa/rsa-sign.c line 33

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+# include <err.h>
+# include <openssl/provider.h>
+# include <openssl/store.h>
+#else
+# if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+#  define USE_PKCS11_ENGINE
+#  include <openssl/engine.h>
+# endif
+#endif
+

Sorry but that's a NACK.

Quoted 5 lines

As far as I can tell, this effectively disables using engines when your host openssl version is 3.0+, which we currently support (and I currently use it). You can have this for 4.0+ as engine support has been removed, but not for 3.x.

You don't get a choice here. OpenSSL deprecated the Engine API starting with Version 3.0, and removed it completely starting with Version 4.0. If the environment you're building U-Boot in has a recent version of OpenSSL then Engines are no longer available to you.

Quoted 5 lines
Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+#ifdef USE_PKCS11_PROVIDER
+#define ERR(cond, fmt, ...)				\
+	do {						\
+		bool __cond = (cond);			\
+		drain_openssl_errors(__LINE__, 0);	\
Quoted 9 lines

lib/rsa/rsa-sign.c line 43

Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+		if (__cond) {				\
+			errx(1, fmt, ## __VA_ARGS__);	\
+		}					\
+	} while (0)
+

Is this really related to the PKCS11 provider? I think there's a mix between "using the provider API" and "using the pkcs11 provider".

It's a helper macro used for the Provider path when OpenSSL is v3 or later.

Quoted 5 lines
Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+static void drain_openssl_errors(int l, int silent)
+{
+	const char *file;
+	char buf[120];
+	int e, line;
Quoted 9 lines

lib/rsa/rsa-sign.c line 54

Quoted 6 lines
lib/rsa/rsa-sign.c +6 -0
+
+	if (ERR_peek_error() == 0)
+		return;
+	if (!silent)
+		fprintf(stderr, "At main.c:%d:\n", l);
+

main.c?

Not clear what you are referring to here.

Quoted 40 lines
Quoted 40 lines
lib/rsa/rsa-sign.c +12 -2
+	while ((e = ERR_peek_error_line(&file, &line))) {
+		ERR_error_string(e, buf);
+		if (!silent)
+			fprintf(stderr, "- SSL %s: %s:%d\n", buf, file, line);
+		ERR_get_error();
+	}
+}
+#endif
  static int rsa_err(const char *msg)
  {
@@ -94,10 +134,11 @@ err_cert:
   *
   * @keydir:	Key prefix
   * @name	Name of key
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
+#ifdef USE_PKCS11_ENGINE
  static int rsa_engine_get_pub_key(const char *keydir, const char *name,
  				  ENGINE *engine, EVP_PKEY **evpp)
  {
@@ -157,21 +198,24 @@ static int rsa_engine_get_pub_key(const char *keydir, const char *name,
  	return 0;
  }
+#endif
  /**
   * rsa_get_pub_key() - read a public key
   *
   * @keydir:	Directory containing the key (PEM file) or key prefix (engine)
   * @name	Name of key file (will have a .crt extension)
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
  static int rsa_get_pub_key(const char *keydir, const char *name,
  			   ENGINE *engine, EVP_PKEY **evpp)
  {
Quoted 9 lines

lib/rsa/rsa-sign.c static int rsa_engine_get_pub_key(const char *keydir, const char *name, · line 218

Quoted 4 lines
lib/rsa/rsa-sign.c +2 -0
+#ifdef USE_PKCS11_ENGINE
  	if (engine)
  		return rsa_engine_get_pub_key(keydir, name, engine, evpp);
+#endif

We should probably at least print something when engines aren't supported but the engine parameter is passed, or maybe even fail as that's an incorrect configuration.

We discussed this in a previous revision and decided the comment was sufficient. The correct path will be taken depending on which version of the API is available.

Quoted 27 lines
Quoted 4 lines
lib/rsa/rsa-sign.c +0 -0
  	return rsa_pem_get_pub_key(keydir, name, evpp);
  }
@@ -207,13 +251,44 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
  		return -ENOENT;

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 261

Quoted 10 lines
lib/rsa/rsa-sign.c +9 -0
  	}
+#ifdef USE_PKCS11_PROVIDER
+	EVP_PKEY *private_key = NULL;
+	OSSL_STORE_CTX *store;
+
+	if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
+		ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
+	if (!OSSL_PROVIDER_try_load(NULL, "default", true))
+		ERR(1, "OSSL_PROVIDER_try_load(default)");
+

Why are we unconditionally loading providers and specifically the pkcs11 one? As far as I could tell we should know from the URI whether a key from a provider is requested by checking if a colon is in the URI. And we can automatically load the requested provider based on that?

Otherwise the easiest answer to this could simply be: "let the user configure this externally via OPENSSL_CONF environment variable".

Do we also need to update tools/mkimage.c (and doc/mkimage.1) to remove -N engine from usage if the tool doesn't support it? Do we also need to update doc/usage/fit/signature.rst to explain how to use pkcs11 provider?

Quoted 6 lines

FYI, we (my employer) are migrating from engines to providers for signing FIT images with binman so I'm starting to have a look on how to do this now.

Cheers,
Quentin

Additional changes may be needed down the road. I'm sure additional patches will be welcome. The scope of this change is to introduce support for the new API while continuing to support developers who are still building in environments using the deprecated Engine API.

Eddie

#4 lore raw

Hi Eddie,

Quoted 61 lines · Eddie Kovsky
Quoted 36 lines · Quentin Schulz

Hi Eddie,

Quoted 15 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

Quoted 6 lines

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

But does it actually use a provider or an engine to begin with? I don't see test/py/tests/test_vboot.py calling mkimage with the -N argument. What are the tests (or command) you ran to validate this? I briefly saw the CI failed in v3 because a package was missing, but wasn't it simply because the headers or provider libraries which are now necessary for building lib/rsa/rsa-sign.c were not present? The logs aren't available anymore unfortunately. If that's the case, then that's also an issue. We shouldn't need to install providers if we aren't going to use any? Yes, I know that we currently cannot compile if we don't have openssl-devel-engine (on Fedora), but if we can improve the situation, we should.

Quoted 2 lines

How did you test (locally is fine) with providers?

The FIT signature verification tests are documented here:

https://docs.u-boot.org/en/latest/usage/fit/signature.html#u-boot-fit-signature-verification

The tests currently fail in build environments (like Fedora) that don't have engine support. This is how we originally became aware of the API issue last year.

❯ ./test/py/test.py --bd sandbox --build -k vboot
+make O=u-boot/build-sandbox -s sandbox_defconfig
+make O=u-boot/build-sandbox -s -j8
In file included from tools/generated/lib/aes/aes-encrypt.c:1:
../tools/../lib/aes/aes-encrypt.c:19:10: fatal error: openssl/engine.h: No such file or directory 19 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
In file included from tools/generated/lib/rsa/rsa-sign.c:1:
../tools/../lib/rsa/rsa-sign.c:22:10: fatal error: openssl/engine.h: No such file or directory 22 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.

Quoted 3 lines

Github provides the Azure CI pipeline as a free service. I wouldn't expect them to retain logs at that tier.

As said, I don't think we are testing OpenSSL providers in test/py. rsa-sign.c and aes-encrypt.c can work perfectly fine without using engines, but currently it requires the engine.h header file to compile (and if missing, then the functions, macros, constants defined in that header file will need to be compiled out like done in this patch) even if it doesn't use engines at runtime.

The issue is not that OpenSSL is built without engine support, it's rather that Fedora has decided to stop shipping openssl/engine.h by default. For some reason, they still compile OpenSSL with engine support (I don't understand why). You are fixing a build issue, sure, and we must fix it, but I don't think this patch is doing it the proper way. So I am asking again, did you test with an actual OpenSSL provider (and no, the implicit file: scheme "provider" doesn't count)?

See https://lore.kernel.org/u-boot/[email protected]/T/#m7454283f474aaee33736e1ae7154569da846f14a

Quoted 10 lines
Quoted 7 lines
Quoted 3 lines

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

Please do not forget to add the colon after Tested-by so it actually makes it a git trailer instead of just text.

That's an unlucky typo because I added the trailers manually.

It happens :)

For your info, with b4 (https://b4.docs.kernel.org/en/latest/contributor/overview.html) you can run b4 trailers -u (if you started the series with b4, though you can probably create a new branch from an existing series with b4 prep --from-thread?) and it'll automatically fetch the trailers from answers on the ML and add them to the appropriate patches.

Quoted 58 lines
Quoted 48 lines
Quoted 48 lines

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Changes in v3:
- Removed Kconfig option
- Changed macro symbol from CONFIG_OPENSSL_NO_DEPRECATED to USE_PKCS11_PROVIDER or USE_PKCS11_ENGINE
v2: https://lore.kernel.org/u-boot/[email protected]/

Changes in v2:
- Remove default for new Kconfig option
- Use #ifdef instead of IS_ENABLED macro
- Remove comment after #endif
- Remove unrelated checkpatch cleanup of 'sslErr' variable name
v1: https://lore.kernel.org/u-boot/[email protected]/

---
   doc/build/gcc.rst       |   4 +-
   lib/aes/aes-encrypt.c   |   4 +-
   lib/rsa/rsa-sign.c      | 102 ++++++++++++++++++++++++++++++++++++++--
   tools/docker/Dockerfile |   1 +
   4 files changed, 103 insertions(+), 8 deletions(-)
doc/build/gcc.rst +2 -2
index 1fef718ceecb..29a6a632e7e3 100644
@@ -25,8 +25,8 @@ Depending on the build targets further packages maybe needed
       sudo apt-get install bc bison build-essential coccinelle \
         device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
-      libgnutls28-dev libguestfs-tools libncurses-dev \
-      libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
+      libgnutls28-dev libguestfs-tools libncurses-dev libpython3-dev \
+      libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl pkcs11-provider \
         pkg-config python3 python3-asteval python3-coverage python3-filelock \
         python3-pkg-resources python3-pycryptodome python3-pyelftools \
         python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
lib/aes/aes-encrypt.c +0 -0
index 90e1407b4f09..4fc4ce232478 100644
@@ -16,7 +16,9 @@
   #include <openssl/err.h>
   #include <openssl/ssl.h>
Quoted 9 lines

lib/aes/aes-encrypt.c line 21

Quoted 5 lines
lib/aes/aes-encrypt.c +3 -1
   #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+# include <openssl/engine.h>
+#endif

Considering there are no other changes in this file, is this include actually needed?

Quoted 4 lines

The header is still needed to maintain backward compatibility for engine support. I maintained the #ifdef style to be consistent with its usage elsewhere, based on previous feedback.

What am I trying to say here is no symbol in the file comes from openssl/engine.h and I can compile the file perfectly fine without this header included, so we should probably get rid of this include.

Quoted 26 lines
Quoted 26 lines
Quoted 13 lines
lib/aes/aes-encrypt.c +0 -0
   #include <uboot_aes.h>
   #if OPENSSL_VERSION_NUMBER >= 0x10000000L
lib/rsa/rsa-sign.c +2 -1
index 0e38c9e802fd..f456f3c58e65 100644
@@ -19,7 +19,47 @@
   #include <openssl/err.h>
   #include <openssl/ssl.h>
   #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if OPENSSL_VERSION_MAJOR >= 3
+# define USE_PKCS11_PROVIDER

lib/rsa/rsa-sign.c line 33

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+# include <err.h>
+# include <openssl/provider.h>
+# include <openssl/store.h>
+#else
+# if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+#  define USE_PKCS11_ENGINE
+#  include <openssl/engine.h>
+# endif
+#endif
+

Sorry but that's a NACK.

Quoted 7 lines
Quoted 5 lines

As far as I can tell, this effectively disables using engines when your host openssl version is 3.0+, which we currently support (and I currently use it). You can have this for 4.0+ as engine support has been removed, but not for 3.x.

You don't get a choice here. OpenSSL deprecated the Engine API starting

Providers and engines can cohabit in OpenSSL 3.x, so that is possible (I tested it locally with the binman test suite, with both providers and engines local test suites working with the same OpenSSL version (Fedora 44 + openssl-devel-engine).

Quoted 4 lines

with Version 3.0, and removed it completely starting with Version 4.0. If the environment you're building U-Boot in has a recent version of OpenSSL then Engines are no longer available to you.

This only happens because Fedora (and not Debian, or ArchLinux) decided to stop shipping an engine.h header file (while still compiling OpenSSL with engine support).

I can use engines with OpenSSL 3.x, with the current state in U-Boot, so you cannot unilaterally decide to break that.

What you can do is either remove support for OpenSSL engines entirely when it's factually not possible to build mkimage with support for it (and no, checking for OpenSSL 3.x is not enough). That is, when OpenSSL is built/shipped without engine support (that is, for Fedora, unless openssl-devel-engine is installed). Or migrate engine support to use the provider API (by e.g. prepending org.openssl.engine: in the OSSL_STORE scheme) for OpenSSL 3.x. The user API must be identical though, what works with engines on OpenSSL 1.x an OpenSSL 3.x with engine support via the provider API must be the same. I would suggest to NOT go for the second option as I don't think it's worth the added complexity.

Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+#ifdef USE_PKCS11_PROVIDER
+#define ERR(cond, fmt, ...)				\
+	do {						\
+		bool __cond = (cond);			\
+		drain_openssl_errors(__LINE__, 0);	\
Quoted 9 lines

lib/rsa/rsa-sign.c line 43

Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+		if (__cond) {				\
+			errx(1, fmt, ## __VA_ARGS__);	\
+		}					\
+	} while (0)
+

Is this really related to the PKCS11 provider? I think there's a mix between "using the provider API" and "using the pkcs11 provider".

Quoted 3 lines

It's a helper macro used for the Provider path when OpenSSL is v3 or later.

I'm arguing about the macro being called USE_PKCS11_PROVIDER while having nothing to do with pkcs11. It's misleading.

Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+static void drain_openssl_errors(int l, int silent)
+{
+	const char *file;
+	char buf[120];
+	int e, line;
Quoted 9 lines

lib/rsa/rsa-sign.c line 54

Quoted 6 lines
lib/rsa/rsa-sign.c +6 -0
+
+	if (ERR_peek_error() == 0)
+		return;
+	if (!silent)
+		fprintf(stderr, "At main.c:%d:\n", l);
+

main.c?

Quoted 2 lines

Not clear what you are referring to here.

The error message would state "At main.c:" which is absolutely not related to any file this is called from. I guess maybe you want __FILE__ here instead?

Quoted 50 lines
Quoted 40 lines
Quoted 40 lines
lib/rsa/rsa-sign.c +12 -2
+	while ((e = ERR_peek_error_line(&file, &line))) {
+		ERR_error_string(e, buf);
+		if (!silent)
+			fprintf(stderr, "- SSL %s: %s:%d\n", buf, file, line);
+		ERR_get_error();
+	}
+}
+#endif
   static int rsa_err(const char *msg)
   {
@@ -94,10 +134,11 @@ err_cert:
    *
    * @keydir:	Key prefix
    * @name	Name of key
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider
    * @evpp	Returns EVP_PKEY object, or NULL on failure
    * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
    */
+#ifdef USE_PKCS11_ENGINE
   static int rsa_engine_get_pub_key(const char *keydir, const char *name,
   				  ENGINE *engine, EVP_PKEY **evpp)
   {
@@ -157,21 +198,24 @@ static int rsa_engine_get_pub_key(const char *keydir, const char *name,
   	return 0;
   }
+#endif
   /**
    * rsa_get_pub_key() - read a public key
    *
    * @keydir:	Directory containing the key (PEM file) or key prefix (engine)
    * @name	Name of key file (will have a .crt extension)
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider
    * @evpp	Returns EVP_PKEY object, or NULL on failure
    * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
    */
   static int rsa_get_pub_key(const char *keydir, const char *name,
   			   ENGINE *engine, EVP_PKEY **evpp)
   {
Quoted 9 lines

lib/rsa/rsa-sign.c static int rsa_engine_get_pub_key(const char *keydir, const char *name, · line 218

Quoted 4 lines
lib/rsa/rsa-sign.c +2 -0
+#ifdef USE_PKCS11_ENGINE
   	if (engine)
   		return rsa_engine_get_pub_key(keydir, name, engine, evpp);
+#endif

We should probably at least print something when engines aren't supported but the engine parameter is passed, or maybe even fail as that's an incorrect configuration.

Quoted 4 lines

We discussed this in a previous revision and decided the comment was sufficient. The correct path will be taken depending on which version of the API is available.

mkimage will still happily accept the -N argument which sets the engine parameter, even if engines aren't supported.

I think if you have a file key with the same name as in the engine, you'll then sign with the wrong key.

Think

mkimage -N myengine -g key

and you have a key.key in the local directory. If you compile with OpenSSL engines supported, you'll essentially use myengine:key, if not, you'll use key.key, without mkimage even notifying you by using the same command.

Quoted 30 lines
Quoted 30 lines
Quoted 4 lines
lib/rsa/rsa-sign.c +0 -0
   	return rsa_pem_get_pub_key(keydir, name, evpp);
   }
@@ -207,13 +251,44 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
   		return -ENOENT;

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 261

Quoted 10 lines
lib/rsa/rsa-sign.c +9 -0
   	}
+#ifdef USE_PKCS11_PROVIDER
+	EVP_PKEY *private_key = NULL;
+	OSSL_STORE_CTX *store;
+
+	if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
+		ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
+	if (!OSSL_PROVIDER_try_load(NULL, "default", true))
+		ERR(1, "OSSL_PROVIDER_try_load(default)");
+

Why are we unconditionally loading providers and specifically the pkcs11 one? As far as I could tell we should know from the URI whether a key from a provider is requested by checking if a colon is in the URI. And we can automatically load the requested provider based on that?

Otherwise the easiest answer to this could simply be: "let the user configure this externally via OPENSSL_CONF environment variable".

Do we also need to update tools/mkimage.c (and doc/mkimage.1) to remove -N engine from usage if the tool doesn't support it? Do we also need to update doc/usage/fit/signature.rst to explain how to use pkcs11 provider?

FYI, we (my employer) are migrating from engines to providers for signing FIT images with binman so I'm starting to have a look on how to do this now.

Quoted 9 lines
Quoted 3 lines

Cheers,
Quentin

Additional changes may be needed down the road. I'm sure additional patches will be welcome. The scope of this change is to introduce support for the new API while continuing to support developers who are still building in environments using the deprecated Engine API.

But it's not doing that? This is removing support for OpenSSL engines on all systems shipping OpenSSL 3.x? We cannot accept patches that break things under the pretense there can be follow-up patches. I'm not asking support for signing with OpenSSL providers in this series, I'm asking you to not break existing users.

I'm still unsure what the goal of this patch is. It says it adds support for OpenSSL provider API but I don't think provider support was actually tested. It also says U-Boot cannot be built on Fedora anymore due to engine.h missing. Both can be be separate topics. The urgent task is to fix building on Fedora, supporting providers can then be tackled later on (by either you, or someone else; e.g. I already have support for providers locally, just need to write nice commit logs and wait on what this patch will become here). I commented out everything related to engines and ran the testSignSimple, testSignExactFIT, testSignNonFit, testSignMissingMkimage and testSplPubkeyDtb to check whether using keys was still supported (meaning, we aren't forced to migrate to the OSSL_STORE interface for simple keys).

I'd be fine with guarding all engine support with
if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
and making sure we at least warn (or fail) if an engine is asked by the user but we don't support it, so that you can build on Fedora again without openssl-devel-engine.

Cheers,
Quentin

#5 lore raw

Hi Quentin and Eddie,

Quoted 84 lines · Quentin Schulz

Hi Eddie,

Quoted 61 lines · Eddie Kovsky
Quoted 36 lines · Quentin Schulz

Hi Eddie,

Quoted 15 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

Quoted 6 lines

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

But does it actually use a provider or an engine to begin with? I don't see test/py/tests/test_vboot.py calling mkimage with the -N argument. What are the tests (or command) you ran to validate this? I briefly saw the CI failed in v3 because a package was missing, but wasn't it simply because the headers or provider libraries which are now necessary for building lib/rsa/rsa-sign.c were not present? The logs aren't available anymore unfortunately. If that's the case, then that's also an issue. We shouldn't need to install providers if we aren't going to use any? Yes, I know that we currently cannot compile if we don't have openssl-devel-engine (on Fedora), but if we can improve the situation, we should.

Quoted 2 lines

How did you test (locally is fine) with providers?

The FIT signature verification tests are documented here:

https://docs.u-boot.org/en/latest/usage/fit/signature.html#u-boot-fit-signature-verification

The tests currently fail in build environments (like Fedora) that don't have engine support. This is how we originally became aware of the API issue last year.

❯ ./test/py/test.py --bd sandbox --build -k vboot
+make O=u-boot/build-sandbox -s sandbox_defconfig
+make O=u-boot/build-sandbox -s -j8
In file included from tools/generated/lib/aes/aes-encrypt.c:1:
../tools/../lib/aes/aes-encrypt.c:19:10: fatal error: openssl/engine.h: No such file or directory 19 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
In file included from tools/generated/lib/rsa/rsa-sign.c:1:
../tools/../lib/rsa/rsa-sign.c:22:10: fatal error: openssl/engine.h: No such file or directory 22 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.

Quoted 3 lines

Github provides the Azure CI pipeline as a free service. I wouldn't expect them to retain logs at that tier.

As said, I don't think we are testing OpenSSL providers in test/py. rsa-sign.c and aes-encrypt.c can work perfectly fine without using engines, but currently it requires the engine.h header file to compile (and if missing, then the functions, macros, constants defined in that header file will need to be compiled out like done in this patch) even if it doesn't use engines at runtime.

The issue is not that OpenSSL is built without engine support, it's rather that Fedora has decided to stop shipping openssl/engine.h by default. For some reason, they still compile OpenSSL with engine support (I don't understand why). You are fixing a build issue, sure, and we must fix it, but I don't think this patch is doing it the proper way. So I am asking again, did you test with an actual OpenSSL provider (and no, the implicit file: scheme "provider" doesn't count)?

Quoted 5 lines

See
https://lore.kernel.org/u-boot/[email protected]/T/#m7454283f474aaee33736e1ae7154569da846f14a

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Tested-by Enric Balletbo i Serra <[email protected]>

Chiming in here as one of the folks testing this series. I'll admit I got completely lost in the mailing list jungle trying to follow this massive wall of messages, but I finally dug into the logs and ran some more hands-on tests.

I think the scope of this patch was strictly a backend API migration to unblock U-Boot compilation on distros like Fedora, not supporting real providers out of the gate.

To verify that Eddie's patch successfully switches the underlying architecture to the OpenSSL 3.x Provider API, I ran a local system trace using a file-based path (-k .). The strace confirmed that the legacy engine hooks are bypassed and the provider module is used instead. The underlying provider plumbing itself is completely solid for standard file-based keys [1].

With that, plus the tests passing in CI, I assumed the new backend plumbing worked as intended. From a functional standpoint, the outcome is identical: your images get signed securely just like before. The only difference is the internal machinery OpenSSL uses under the hood.

But you are entirely right. I didn't test against a real hardware engine initially, and looking closer, I agree the current patch does break backward compatibility for anyone using a real Engine on OpenSSL 3.x.

I did a run using SoftHSM to see exactly why:

Before this patch: Passing -N pkcs11 successfully triggered the legacy engine. The signing succeeded.

With this patch: The OpenSSL 3.x version check unconditionally hijacks the flow, ignores the -N engine flag, and forces execution down the Provider path. The strict OSSL_STORE_open parser receives the mangled string and instantly fails with ENOENT because is unconditionally appending /[key-name-hint].key.

My understanding is that to fix the Fedora build issues without breaking existing workflows, Eddie just needs to adjust the control flow for v4 like you suggested:

If the user explicitly commands an engine via -N, prioritize the legacy engine path—guarding the engine headers and blocks with #ifndef OPENSSL_NO_ENGINE instead of a blanket OpenSSL version check.

If no engine flag is passed, route seamlessly into the new Provider API backend.

This fixes the compilation bottleneck on some distros, completely preserves legacy engine setups, and allows real providers URI string-mangling issue to be handled as clean, independent follow-up patches.

Does it makes sense?

Cheers,
Enric Balletbo

[1]

 $ strace -f -e trace=openat ./tools/mkimage -F -k . -K mock.dtb -r
fitImage 2>&1 | grep -iE "pkcs11|ossl-modules|\.key"
openat(AT_FDCWD, "/etc/pki/tls/openssl.d/pkcs11-provider.conf", O_RDONLY) = 7
openat(AT_FDCWD, "/usr/lib64/ossl-modules/pkcs11.so", O_RDONLY|O_CLOEXEC) = 5
openat(AT_FDCWD, "./dev.key", O_RDONLY) = 5
FIT description: Dummy FIT image for testing providers
Created:         Wed Jun  3 19:26:03 2026
 Image 0 (kernel-1)
  Description:  Dummy Kernel
  Created:      Wed Jun  3 19:26:03 2026
  Type:         Kernel Image
  Compression:  uncompressed
  Data Size:    100 Bytes = 0.10 KiB = 0.00 MiB
  Architecture: ARM
  OS:           Linux
  Load Address: 0x80008000
  Entry Point:  0x80008000
  Hash algo:    sha256
  Hash value:   cd00e292c5970d3c5e2f0ffa5171e555bc46bfc4faddfb4a418b6840b86e79a3
 Default Configuration: 'config-1'
 Configuration 0 (config-1)
  Description:  Boot configuration
  Kernel:       kernel-1
  Sign algo:    sha256,rsa2048:dev
  Sign value:
4b88172afa850fb98d4c81662b043fedfebd7db55be8011aa464570e58289cf58939ea3adfafd9dd808362cfbd830fccaf806bbdc3d221922ec4e59a478a526904b1f39afa28d770b2808ed727825953eb8e4f1387faf3396e6f0e8d1a5be1c4b698e94e383d466371b8469638a3d3e1e1a3385eb4752c7554d09289face527d60262f274c8ab12ae28069a51dbce19505578e391e2fc25a9df915b6917cf057b5b2ecd9ede7db81a0be1cb106db801f75500562902f45fbed9df5b87c61cabc854cf429d25f5977813bdb239b38e27ba461b752c612068ae1dc2566d3456d2396828bfaea1a946b77d0479ee720574d6068409353e4554d0dd5c56f1a0a6228
  Timestamp:    Wed Jun  3 19:26:16 2026
Signature written to 'fitImage', node '/configurations/config-1/signature-1'
Public key written to 'mock.dtb', node '/signature/key-dev'

./tools/fit_check_sign -f fitImage -k mock.dtb
Verifying Hash Integrity for node 'config-1'... sha256,rsa2048:dev+ Verified OK, loading images
## Loading kernel (any) from FIT Image at 7f0d23b65000 ...
Using 'config-1' configuration
Verifying Hash Integrity ...
sha256,rsa2048:dev+
OK

Trying 'kernel-1' kernel subimage

     Description:  Dummy Kernel
     Created:      Wed Jun  3 19:26:03 2026
     Type:         Kernel Image
     Compression:  uncompressed
     Data Size:    100 Bytes = 0.10 KiB = 0.00 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x80008000
     Entry Point:  0x80008000
     Hash algo:    sha256
     Hash value:

cd00e292c5970d3c5e2f0ffa5171e555bc46bfc4faddfb4a418b6840b86e79a3
Verifying Hash Integrity ...
sha256+
OK

Decrypting Data ...
OK

Loading Kernel Image to 0
## Loading fdt (any) from FIT Image at 7f0d23b65000 ...
Using 'config-1' configuration
Verifying Hash Integrity ...
sha256,rsa2048:dev+
OK

Could not find subimage node type 'fdt'
## Loading ramdisk (any) from FIT Image at 7f0d23b65000 ...
Using 'config-1' configuration
Verifying Hash Integrity ...
sha256,rsa2048:dev+
OK

Could not find subimage node type 'ramdisk'
Signature check OK

Quoted 329 lines
Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Tested-by Mark Kettenis <[email protected]>

Please do not forget to add the colon after Tested-by so it actually makes it a git trailer instead of just text.

That's an unlucky typo because I added the trailers manually.

It happens :)

For your info, with b4
(https://b4.docs.kernel.org/en/latest/contributor/overview.html) you can run b4 trailers -u (if you started the series with b4, though you can probably create a new branch from an existing series with b4 prep --from-thread?) and it'll automatically fetch the trailers from answers on the ML and add them to the appropriate patches.

Quoted 58 lines
Quoted 48 lines
Quoted 48 lines

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Changes in v3:
- Removed Kconfig option
- Changed macro symbol from CONFIG_OPENSSL_NO_DEPRECATED to USE_PKCS11_PROVIDER or USE_PKCS11_ENGINE
v2: https://lore.kernel.org/u-boot/[email protected]/

Changes in v2:
- Remove default for new Kconfig option
- Use #ifdef instead of IS_ENABLED macro
- Remove comment after #endif
- Remove unrelated checkpatch cleanup of 'sslErr' variable name
v1: https://lore.kernel.org/u-boot/[email protected]/

---
   doc/build/gcc.rst       |   4 +-
   lib/aes/aes-encrypt.c   |   4 +-
   lib/rsa/rsa-sign.c      | 102 ++++++++++++++++++++++++++++++++++++++--
   tools/docker/Dockerfile |   1 +
   4 files changed, 103 insertions(+), 8 deletions(-)
doc/build/gcc.rst +2 -2
index 1fef718ceecb..29a6a632e7e3 100644
@@ -25,8 +25,8 @@ Depending on the build targets further packages maybe needed
       sudo apt-get install bc bison build-essential coccinelle \
         device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
-      libgnutls28-dev libguestfs-tools libncurses-dev \
-      libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
+      libgnutls28-dev libguestfs-tools libncurses-dev libpython3-dev \
+      libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl pkcs11-provider \
         pkg-config python3 python3-asteval python3-coverage python3-filelock \
         python3-pkg-resources python3-pycryptodome python3-pyelftools \
         python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
lib/aes/aes-encrypt.c +0 -0
index 90e1407b4f09..4fc4ce232478 100644
@@ -16,7 +16,9 @@
   #include <openssl/err.h>
   #include <openssl/ssl.h>
Quoted 9 lines

lib/aes/aes-encrypt.c line 21

Quoted 5 lines
lib/aes/aes-encrypt.c +3 -1
   #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+# include <openssl/engine.h>
+#endif

Considering there are no other changes in this file, is this include actually needed?

Quoted 4 lines

The header is still needed to maintain backward compatibility for engine support. I maintained the #ifdef style to be consistent with its usage elsewhere, based on previous feedback.

What am I trying to say here is no symbol in the file comes from openssl/engine.h and I can compile the file perfectly fine without this header included, so we should probably get rid of this include.

Quoted 26 lines
Quoted 26 lines
Quoted 13 lines
lib/aes/aes-encrypt.c +0 -0
   #include <uboot_aes.h>
   #if OPENSSL_VERSION_NUMBER >= 0x10000000L
lib/rsa/rsa-sign.c +2 -1
index 0e38c9e802fd..f456f3c58e65 100644
@@ -19,7 +19,47 @@
   #include <openssl/err.h>
   #include <openssl/ssl.h>
   #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if OPENSSL_VERSION_MAJOR >= 3
+# define USE_PKCS11_PROVIDER

lib/rsa/rsa-sign.c line 33

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+# include <err.h>
+# include <openssl/provider.h>
+# include <openssl/store.h>
+#else
+# if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+#  define USE_PKCS11_ENGINE
+#  include <openssl/engine.h>
+# endif
+#endif
+

Sorry but that's a NACK.

Quoted 7 lines
Quoted 5 lines

As far as I can tell, this effectively disables using engines when your host openssl version is 3.0+, which we currently support (and I currently use it). You can have this for 4.0+ as engine support has been removed, but not for 3.x.

You don't get a choice here. OpenSSL deprecated the Engine API starting

Providers and engines can cohabit in OpenSSL 3.x, so that is possible (I tested it locally with the binman test suite, with both providers and engines local test suites working with the same OpenSSL version (Fedora 44 + openssl-devel-engine).

Quoted 4 lines

with Version 3.0, and removed it completely starting with Version 4.0. If the environment you're building U-Boot in has a recent version of OpenSSL then Engines are no longer available to you.

This only happens because Fedora (and not Debian, or ArchLinux) decided to stop shipping an engine.h header file (while still compiling OpenSSL with engine support).

I can use engines with OpenSSL 3.x, with the current state in U-Boot, so you cannot unilaterally decide to break that.

What you can do is either remove support for OpenSSL engines entirely when it's factually not possible to build mkimage with support for it (and no, checking for OpenSSL 3.x is not enough). That is, when OpenSSL is built/shipped without engine support (that is, for Fedora, unless openssl-devel-engine is installed). Or migrate engine support to use the provider API (by e.g. prepending org.openssl.engine: in the OSSL_STORE scheme) for OpenSSL 3.x. The user API must be identical though, what works with engines on OpenSSL 1.x an OpenSSL 3.x with engine support via the provider API must be the same. I would suggest to NOT go for the second option as I don't think it's worth the added complexity.

Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+#ifdef USE_PKCS11_PROVIDER
+#define ERR(cond, fmt, ...)                                \
+   do {                                            \
+           bool __cond = (cond);                   \
+           drain_openssl_errors(__LINE__, 0);      \
Quoted 9 lines

lib/rsa/rsa-sign.c line 43

Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+           if (__cond) {                           \
+                   errx(1, fmt, ## __VA_ARGS__);   \
+           }                                       \
+   } while (0)
+

Is this really related to the PKCS11 provider? I think there's a mix between "using the provider API" and "using the pkcs11 provider".

Quoted 3 lines

It's a helper macro used for the Provider path when OpenSSL is v3 or later.

I'm arguing about the macro being called USE_PKCS11_PROVIDER while having nothing to do with pkcs11. It's misleading.

Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+static void drain_openssl_errors(int l, int silent)
+{
+   const char *file;
+   char buf[120];
+   int e, line;
Quoted 9 lines

lib/rsa/rsa-sign.c line 54

Quoted 6 lines
lib/rsa/rsa-sign.c +6 -0
+
+   if (ERR_peek_error() == 0)
+           return;
+   if (!silent)
+           fprintf(stderr, "At main.c:%d:\n", l);
+

main.c?

Quoted 2 lines

Not clear what you are referring to here.

The error message would state "At main.c:" which is absolutely not related to any file this is called from. I guess maybe you want __FILE__ here instead?

Quoted 50 lines
Quoted 40 lines
Quoted 40 lines
lib/rsa/rsa-sign.c +12 -2
+   while ((e = ERR_peek_error_line(&file, &line))) {
+           ERR_error_string(e, buf);
+           if (!silent)
+                   fprintf(stderr, "- SSL %s: %s:%d\n", buf, file, line);
+           ERR_get_error();
+   }
+}
+#endif
   static int rsa_err(const char *msg)
   {
@@ -94,10 +134,11 @@ err_cert:
    *
    * @keydir:      Key prefix
    * @name Name of key
- * @engine Engine to use
+ * @engine Engine to use or NULL when using pkcs11 provider
    * @evpp Returns EVP_PKEY object, or NULL on failure
    * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
    */
+#ifdef USE_PKCS11_ENGINE
   static int rsa_engine_get_pub_key(const char *keydir, const char *name,
                              ENGINE *engine, EVP_PKEY **evpp)
   {
@@ -157,21 +198,24 @@ static int rsa_engine_get_pub_key(const char *keydir, const char *name,
    return 0;
   }
+#endif
   /**
    * rsa_get_pub_key() - read a public key
    *
    * @keydir:      Directory containing the key (PEM file) or key prefix (engine)
    * @name Name of key file (will have a .crt extension)
- * @engine Engine to use
+ * @engine Engine to use or NULL when using pkcs11 provider
    * @evpp Returns EVP_PKEY object, or NULL on failure
    * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
    */
   static int rsa_get_pub_key(const char *keydir, const char *name,
                       ENGINE *engine, EVP_PKEY **evpp)
   {
Quoted 9 lines

lib/rsa/rsa-sign.c static int rsa_engine_get_pub_key(const char *keydir, const char *name, · line 218

Quoted 4 lines
lib/rsa/rsa-sign.c +2 -0
+#ifdef USE_PKCS11_ENGINE
    if (engine)
            return rsa_engine_get_pub_key(keydir, name, engine, evpp);
+#endif

We should probably at least print something when engines aren't supported but the engine parameter is passed, or maybe even fail as that's an incorrect configuration.

Quoted 4 lines

We discussed this in a previous revision and decided the comment was sufficient. The correct path will be taken depending on which version of the API is available.

mkimage will still happily accept the -N argument which sets the engine parameter, even if engines aren't supported.

I think if you have a file key with the same name as in the engine, you'll then sign with the wrong key.

Think

mkimage -N myengine -g key

and you have a key.key in the local directory. If you compile with OpenSSL engines supported, you'll essentially use myengine:key, if not, you'll use key.key, without mkimage even notifying you by using the same command.

Quoted 30 lines
Quoted 30 lines
Quoted 4 lines
lib/rsa/rsa-sign.c +0 -0
    return rsa_pem_get_pub_key(keydir, name, evpp);
   }
@@ -207,13 +251,44 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
            return -ENOENT;

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 261

Quoted 10 lines
lib/rsa/rsa-sign.c +9 -0
    }
+#ifdef USE_PKCS11_PROVIDER
+   EVP_PKEY *private_key = NULL;
+   OSSL_STORE_CTX *store;
+
+   if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
+           ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
+   if (!OSSL_PROVIDER_try_load(NULL, "default", true))
+           ERR(1, "OSSL_PROVIDER_try_load(default)");
+

Why are we unconditionally loading providers and specifically the pkcs11 one? As far as I could tell we should know from the URI whether a key from a provider is requested by checking if a colon is in the URI. And we can automatically load the requested provider based on that?

Otherwise the easiest answer to this could simply be: "let the user configure this externally via OPENSSL_CONF environment variable".

Do we also need to update tools/mkimage.c (and doc/mkimage.1) to remove -N engine from usage if the tool doesn't support it? Do we also need to update doc/usage/fit/signature.rst to explain how to use pkcs11 provider?

FYI, we (my employer) are migrating from engines to providers for signing FIT images with binman so I'm starting to have a look on how to do this now.

Quoted 9 lines
Quoted 3 lines

Cheers,
Quentin

Additional changes may be needed down the road. I'm sure additional patches will be welcome. The scope of this change is to introduce support for the new API while continuing to support developers who are still building in environments using the deprecated Engine API.

But it's not doing that? This is removing support for OpenSSL engines on all systems shipping OpenSSL 3.x? We cannot accept patches that break things under the pretense there can be follow-up patches. I'm not asking support for signing with OpenSSL providers in this series, I'm asking you to not break existing users.

I'm still unsure what the goal of this patch is. It says it adds support for OpenSSL provider API but I don't think provider support was actually tested. It also says U-Boot cannot be built on Fedora anymore due to engine.h missing. Both can be be separate topics. The urgent task is to fix building on Fedora, supporting providers can then be tackled later on (by either you, or someone else; e.g. I already have support for providers locally, just need to write nice commit logs and wait on what this patch will become here). I commented out everything related to engines and ran the testSignSimple, testSignExactFIT, testSignNonFit, testSignMissingMkimage and testSplPubkeyDtb to check whether using keys was still supported (meaning, we aren't forced to migrate to the OSSL_STORE interface for simple keys).

I'd be fine with guarding all engine support with
if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0) and making sure we at least warn (or fail) if an engine is asked by the user but we don't support it, so that you can build on Fedora again without openssl-devel-engine.

Cheers,
Quentin

#6 lore raw

Hi Enric,

Thanks for chiming in.

Quoted 11 lines · Enric Balletbo i Serra

Hi Quentin and Eddie,

Quoted 8 lines · Quentin Schulz

Hi Eddie,

Quoted 4 lines · Eddie Kovsky
Quoted 3 lines · Quentin Schulz

Hi Eddie,

On 4/29/26 8:02 PM, Eddie Kovsky wrote:

[...]

Quoted 9 lines

My understanding is that to fix the Fedora build issues without breaking existing workflows, Eddie just needs to adjust the control flow for v4 like you suggested:

If the user explicitly commands an engine via -N, prioritize the legacy engine path—guarding the engine headers and blocks with #ifndef OPENSSL_NO_ENGINE instead of a blanket OpenSSL version check.

If no engine flag is passed, route seamlessly into the new Provider API backend.

Fix the build issue by checking whether OpenSSL engines are supported (and no, a version check is not enough for OpenSSL 3.x (it is for OpenSSL 4.x though)). If they are, keep the same logic as today. If they aren't, fail hard when an engine is requested, otherwise keep the same logic as today. Make use of #ifdef to guard code that cannot compile when engine support is disabled and you should be good to go. This will still fail unit tests because we are testing and building a dummy engine regardless of support. But last time I checked (years ago), many unrelated unit tests are also failing on Fedora, so it wouldn't be a deal breaker for me (and I have some "interesting" ideas on how to handle this in a later commit).

Also, please do not use USE_PKCS11_PROVIDER when it has nothing to do with PKCS11 provider. I understand this was imported from the kernel, but they do only support PKCS11 as far as I could tell, so it makes sense for them.

I honestly don't want to see the Provider/Store API being used only for loading simple keys. It already works without the provider API and isn't deprecated as far as I know? Let someone who's interested (my employer is, therefore I have to be :) ) in supporting OpenSSL providers do the work of implementing support for OpenSSL providers, with unit tests, backward compatibility with engines through the OpenSSL provider API, etc...

For PKCS11, we have some interesting logic for the OpenSSL engine, and I think we may want something similar for the OpenSSL provider.

Note that OpenSSL 1.x is still used in some distros (Bullseye, Slackware 15, AlmaLinux/RockyLinux 8 according to pkgs.org) so I think it's still premature to entirely remove OpenSSL engine support (and OpenSSL 1.x doesn't have the provider API).

Quoted 6 lines

This fixes the compilation bottleneck on some distros, completely preserves legacy engine setups, and allows real providers URI string-mangling issue to be handled as clean, independent follow-up patches.

We need proper unit tests for supporting OpenSSL providers (and not only a local key using the Provider/Store API). I'll repeat it once more, I have local WIP (but tested) patches for OpenSSL provider support, so I don't think it makes sense for someone to redo the work. The fact this was started back in November 2025 and it hasn't been really pushed hard since kinda hinted at this not being a big issue for RH (or they keep their stuff downstream and upstream can take the time it'll take), hence why I was waiting for Eddie to answer/send a new version (I've also been swamped at work for a few weeks, which should end soon hopefully, so it also worked out for me to wait on Eddie).

Quoted 2 lines

Does it makes sense?

Somewhat. I still don't understand the obsession with the Provider API when the issue (as I understand it) is "I cannot compile OpenSSL engine support and I don't need it".

Cheers,
Quentin

#7 lore raw

Hi Quentin,

Quoted 28 lines · Quentin Schulz

Hi Enric,

Thanks for chiming in.

Quoted 11 lines · Enric Balletbo i Serra

Hi Quentin and Eddie,

Quoted 8 lines · Quentin Schulz

Hi Eddie,

Quoted 4 lines · Eddie Kovsky
Quoted 3 lines · Quentin Schulz

Hi Eddie,

On 4/29/26 8:02 PM, Eddie Kovsky wrote:

[...]

Quoted 9 lines

My understanding is that to fix the Fedora build issues without breaking existing workflows, Eddie just needs to adjust the control flow for v4 like you suggested:

If the user explicitly commands an engine via -N, prioritize the legacy engine path—guarding the engine headers and blocks with #ifndef OPENSSL_NO_ENGINE instead of a blanket OpenSSL version check.

If no engine flag is passed, route seamlessly into the new Provider API backend.

Quoted 12 lines

Fix the build issue by checking whether OpenSSL engines are supported (and no, a version check is not enough for OpenSSL 3.x (it is for OpenSSL 4.x though)). If they are, keep the same logic as today. If they aren't, fail hard when an engine is requested, otherwise keep the same logic as today. Make use of #ifdef to guard code that cannot compile when engine support is disabled and you should be good to go. This will still fail unit tests because we are testing and building a dummy engine regardless of support. But last time I checked (years ago), many unrelated unit tests are also failing on Fedora, so it wouldn't be a deal breaker for me (and I have some "interesting" ideas on how to handle this in a later commit).

This makes perfect sense to me. Since you already have a provider series in your local queue, pivoting v4 to strictly implement these compile-guards looks like the best path forward. I also appreciate the clarification regarding the unit tests, knowing that the expected dummy engine test failures block the patch, as they did before.

Quoted 5 lines

Also, please do not use USE_PKCS11_PROVIDER when it has nothing to do with PKCS11 provider. I understand this was imported from the kernel, but they do only support PKCS11 as far as I could tell, so it makes sense for them.

I agree now that I have a bit more knowledge.

Quoted 7 lines

I honestly don't want to see the Provider/Store API being used only for loading simple keys. It already works without the provider API and isn't deprecated as far as I know? Let someone who's interested (my employer is, therefore I have to be :) ) in supporting OpenSSL providers do the work of implementing support for OpenSSL providers, with unit tests, backward compatibility with engines through the OpenSSL provider API, etc...

Agreed if protecting backward compatibility for OpenSSL 1.x distributions are critical, and we definitely shouldn't break those existing legacy engine setups.

Quoted 15 lines

For PKCS11, we have some interesting logic for the OpenSSL engine, and I think we may want something similar for the OpenSSL provider.

Note that OpenSSL 1.x is still used in some distros (Bullseye, Slackware 15, AlmaLinux/RockyLinux 8 according to pkgs.org) so I think it's still premature to entirely remove OpenSSL engine support (and OpenSSL 1.x doesn't have the provider API).

Quoted 6 lines

This fixes the compilation bottleneck on some distros, completely preserves legacy engine setups, and allows real providers URI string-mangling issue to be handled as clean, independent follow-up patches.

Quoted 11 lines

We need proper unit tests for supporting OpenSSL providers (and not only a local key using the Provider/Store API). I'll repeat it once more, I have local WIP (but tested) patches for OpenSSL provider support, so I don't think it makes sense for someone to redo the work. The fact this was started back in November 2025 and it hasn't been really pushed hard since kinda hinted at this not being a big issue for RH (or they keep their stuff downstream and upstream can take the time it'll take), hence why I was waiting for Eddie to answer/send a new version (I've also been swamped at work for a few weeks, which should end soon hopefully, so it also worked out for me to wait on Eddie).

Knowing that, there is no reason to duplicate your efforts. Please cc me when you send it upstream, as I'm interested.

Quoted 7 lines
Quoted 2 lines

Does it makes sense?

Somewhat. I still don't understand the obsession with the Provider API when the issue (as I understand it) is "I cannot compile OpenSSL engine support and I don't need it".

To clarify the "obsession" point, which I think is an unfortunate take, my impression was that this wasn't just about throwing a quick, blind hack at a Fedora compiler error. The build failure was simply the trigger.

Eddie put significant effort into researching the modern OpenSSL 3.x API, studying how to phase out legacy, deprecated components, looking at the U-Boot's testing infrastructure, and trying to proactively modernize U-Boot's backend based on established upstream kernel patterns.

I agree, now, and this is the beauty of discussions, that the kernel approach is not enough due to its narrow scope. U-Boot’s edge cases make a full API migration heavier than expected. However, trying to actively clean up technical debt is a valuable goal, and characterizing it as a mere obsession with a build error can be quite discouraging for new contributors putting in real work to improve the codebase.

That said, if Eddie is willing to do so, keeping that patch focused strictly on the #ifndef compile-guards looks like the right approach to me too, and it leaves the runway entirely clear for your provider migration patches later.

Cheers,
Enric

Quoted 3 lines

Cheers,
Quentin

#8 lore raw

Hi Enric, Eddie,

Quoted 20 lines · Enric Balletbo i Serra

Hi Quentin,

Quoted 17 lines · Quentin Schulz

Hi Enric,

Thanks for chiming in.

Quoted 11 lines · Enric Balletbo i Serra

Hi Quentin and Eddie,

Quoted 8 lines · Quentin Schulz

Hi Eddie,

Quoted 4 lines · Eddie Kovsky
Quoted 3 lines · Quentin Schulz

Hi Eddie,

On 4/29/26 8:02 PM, Eddie Kovsky wrote:

[...]

Quoted 8 lines
Quoted 7 lines
Quoted 2 lines

Does it makes sense?

Somewhat. I still don't understand the obsession with the Provider API when the issue (as I understand it) is "I cannot compile OpenSSL engine support and I don't need it".

Quoted 5 lines

To clarify the "obsession" point, which I think is an unfortunate take, my impression was that this wasn't just about throwing a quick, blind hack at a Fedora compiler error. The build failure was simply the trigger.

I think "no engine support anymore, so providers must be supported" was a shortcut many could have taken. I understand (and really appreciate) the intent of trying to improve the situation instead of "just" patching a build failure and be done with it (after all, I myself just installed openssl-devel-engine when I got the build error instead of putting in the work Eddie did).

Quoted 6 lines

Eddie put significant effort into researching the modern OpenSSL 3.x API, studying how to phase out legacy, deprecated components, looking at the U-Boot's testing infrastructure, and trying to proactively modernize U-Boot's backend based on established upstream kernel patterns.

Quoted 8 lines

I agree, now, and this is the beauty of discussions, that the kernel approach is not enough due to its narrow scope. U-Boot’s edge cases make a full API migration heavier than expected. However, trying to actively clean up technical debt is a valuable goal, and characterizing it as a mere obsession with a build error can be quite discouraging for new contributors putting in real work to improve the codebase.

I do realize that "obsession" was too strong of a word and I apologize to Eddie and you. I appreciate the feedback and will try to do better.

Cheers,
Quentin

#9 lore raw
In reply to Enric Balletbo i Serra
Quoted 225 lines · Enric Balletbo i Serra

Hi Quentin and Eddie,

Quoted 84 lines · Quentin Schulz

Hi Eddie,

Quoted 61 lines · Eddie Kovsky
Quoted 36 lines · Quentin Schulz

Hi Eddie,

Quoted 15 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

Quoted 6 lines

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

But does it actually use a provider or an engine to begin with? I don't see test/py/tests/test_vboot.py calling mkimage with the -N argument. What are the tests (or command) you ran to validate this? I briefly saw the CI failed in v3 because a package was missing, but wasn't it simply because the headers or provider libraries which are now necessary for building lib/rsa/rsa-sign.c were not present? The logs aren't available anymore unfortunately. If that's the case, then that's also an issue. We shouldn't need to install providers if we aren't going to use any? Yes, I know that we currently cannot compile if we don't have openssl-devel-engine (on Fedora), but if we can improve the situation, we should.

Quoted 2 lines

How did you test (locally is fine) with providers?

The FIT signature verification tests are documented here:

https://docs.u-boot.org/en/latest/usage/fit/signature.html#u-boot-fit-signature-verification

The tests currently fail in build environments (like Fedora) that don't have engine support. This is how we originally became aware of the API issue last year.

❯ ./test/py/test.py --bd sandbox --build -k vboot
+make O=u-boot/build-sandbox -s sandbox_defconfig
+make O=u-boot/build-sandbox -s -j8
In file included from tools/generated/lib/aes/aes-encrypt.c:1:
../tools/../lib/aes/aes-encrypt.c:19:10: fatal error: openssl/engine.h: No such file or directory 19 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
In file included from tools/generated/lib/rsa/rsa-sign.c:1:
../tools/../lib/rsa/rsa-sign.c:22:10: fatal error: openssl/engine.h: No such file or directory 22 | #include <openssl/engine.h>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.

Quoted 3 lines

Github provides the Azure CI pipeline as a free service. I wouldn't expect them to retain logs at that tier.

As said, I don't think we are testing OpenSSL providers in test/py. rsa-sign.c and aes-encrypt.c can work perfectly fine without using engines, but currently it requires the engine.h header file to compile (and if missing, then the functions, macros, constants defined in that header file will need to be compiled out like done in this patch) even if it doesn't use engines at runtime.

The issue is not that OpenSSL is built without engine support, it's rather that Fedora has decided to stop shipping openssl/engine.h by default. For some reason, they still compile OpenSSL with engine support (I don't understand why). You are fixing a build issue, sure, and we must fix it, but I don't think this patch is doing it the proper way. So I am asking again, did you test with an actual OpenSSL provider (and no, the implicit file: scheme "provider" doesn't count)?

Quoted 5 lines

See
https://lore.kernel.org/u-boot/[email protected]/T/#m7454283f474aaee33736e1ae7154569da846f14a

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Tested-by Enric Balletbo i Serra <[email protected]>

Chiming in here as one of the folks testing this series. I'll admit I got completely lost in the mailing list jungle trying to follow this massive wall of messages, but I finally dug into the logs and ran some more hands-on tests.

I think the scope of this patch was strictly a backend API migration to unblock U-Boot compilation on distros like Fedora, not supporting real providers out of the gate.

To verify that Eddie's patch successfully switches the underlying architecture to the OpenSSL 3.x Provider API, I ran a local system trace using a file-based path (-k .). The strace confirmed that the legacy engine hooks are bypassed and the provider module is used instead. The underlying provider plumbing itself is completely solid for standard file-based keys [1].

With that, plus the tests passing in CI, I assumed the new backend plumbing worked as intended. From a functional standpoint, the outcome is identical: your images get signed securely just like before. The only difference is the internal machinery OpenSSL uses under the hood.

But you are entirely right. I didn't test against a real hardware engine initially, and looking closer, I agree the current patch does break backward compatibility for anyone using a real Engine on OpenSSL 3.x.

I did a run using SoftHSM to see exactly why:

Before this patch: Passing -N pkcs11 successfully triggered the legacy engine. The signing succeeded.

With this patch: The OpenSSL 3.x version check unconditionally hijacks the flow, ignores the -N engine flag, and forces execution down the Provider path. The strict OSSL_STORE_open parser receives the mangled string and instantly fails with ENOENT because is unconditionally appending /[key-name-hint].key.

My understanding is that to fix the Fedora build issues without breaking existing workflows, Eddie just needs to adjust the control flow for v4 like you suggested:

If the user explicitly commands an engine via -N, prioritize the legacy engine path—guarding the engine headers and blocks with #ifndef OPENSSL_NO_ENGINE instead of a blanket OpenSSL version check.

If no engine flag is passed, route seamlessly into the new Provider API backend.

This fixes the compilation bottleneck on some distros, completely preserves legacy engine setups, and allows real providers URI string-mangling issue to be handled as clean, independent follow-up patches.

Does it makes sense?

Cheers,
Enric Balletbo

[1]

 $ strace -f -e trace=openat ./tools/mkimage -F -k . -K mock.dtb -r
fitImage 2>&1 | grep -iE "pkcs11|ossl-modules|\.key"
openat(AT_FDCWD, "/etc/pki/tls/openssl.d/pkcs11-provider.conf", O_RDONLY) = 7
openat(AT_FDCWD, "/usr/lib64/ossl-modules/pkcs11.so", O_RDONLY|O_CLOEXEC) = 5
openat(AT_FDCWD, "./dev.key", O_RDONLY) = 5
FIT description: Dummy FIT image for testing providers
Created:         Wed Jun  3 19:26:03 2026
 Image 0 (kernel-1)
  Description:  Dummy Kernel
  Created:      Wed Jun  3 19:26:03 2026
  Type:         Kernel Image
  Compression:  uncompressed
  Data Size:    100 Bytes = 0.10 KiB = 0.00 MiB
  Architecture: ARM
  OS:           Linux
  Load Address: 0x80008000
  Entry Point:  0x80008000
  Hash algo:    sha256
  Hash value:   cd00e292c5970d3c5e2f0ffa5171e555bc46bfc4faddfb4a418b6840b86e79a3
 Default Configuration: 'config-1'
 Configuration 0 (config-1)
  Description:  Boot configuration
  Kernel:       kernel-1
  Sign algo:    sha256,rsa2048:dev
  Sign value:
4b88172afa850fb98d4c81662b043fedfebd7db55be8011aa464570e58289cf58939ea3adfafd9dd808362cfbd830fccaf806bbdc3d221922ec4e59a478a526904b1f39afa28d770b2808ed727825953eb8e4f1387faf3396e6f0e8d1a5be1c4b698e94e383d466371b8469638a3d3e1e1a3385eb4752c7554d09289face527d60262f274c8ab12ae28069a51dbce19505578e391e2fc25a9df915b6917cf057b5b2ecd9ede7db81a0be1cb106db801f75500562902f45fbed9df5b87c61cabc854cf429d25f5977813bdb239b38e27ba461b752c612068ae1dc2566d3456d2396828bfaea1a946b77d0479ee720574d6068409353e4554d0dd5c56f1a0a6228
  Timestamp:    Wed Jun  3 19:26:16 2026
Signature written to 'fitImage', node '/configurations/config-1/signature-1'
Public key written to 'mock.dtb', node '/signature/key-dev'

./tools/fit_check_sign -f fitImage -k mock.dtb
Verifying Hash Integrity for node 'config-1'... sha256,rsa2048:dev+ Verified OK, loading images
## Loading kernel (any) from FIT Image at 7f0d23b65000 ...
Using 'config-1' configuration
Verifying Hash Integrity ...
sha256,rsa2048:dev+
OK

Trying 'kernel-1' kernel subimage

     Description:  Dummy Kernel
     Created:      Wed Jun  3 19:26:03 2026
     Type:         Kernel Image
     Compression:  uncompressed
     Data Size:    100 Bytes = 0.10 KiB = 0.00 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x80008000
     Entry Point:  0x80008000
     Hash algo:    sha256
     Hash value:

cd00e292c5970d3c5e2f0ffa5171e555bc46bfc4faddfb4a418b6840b86e79a3
Verifying Hash Integrity ...
sha256+
OK

Decrypting Data ...
OK

Loading Kernel Image to 0
## Loading fdt (any) from FIT Image at 7f0d23b65000 ...
Using 'config-1' configuration
Verifying Hash Integrity ...
sha256,rsa2048:dev+
OK

Could not find subimage node type 'fdt'
## Loading ramdisk (any) from FIT Image at 7f0d23b65000 ...
Using 'config-1' configuration
Verifying Hash Integrity ...
sha256,rsa2048:dev+
OK

Quoted 4 lines

Could not find subimage node type 'ramdisk'
Signature check OK

The root cause of the hardware key signing failures can be traced back to the master branch. In lib/rsa/rsa-sign.c around line 306 we have this function that decides which function to call to read a private key based on testing if the engine pointer is valid.

    static int rsa_get_priv_key(const char *keydir, const char *name,
                    const char *keyfile, ENGINE *engine, EVP_PKEY **evpp)
    {
        if (engine)
            return rsa_engine_get_priv_key(keydir, name, keyfile, engine,
                            evpp);
        return rsa_pem_get_priv_key(keydir, name, keyfile, evpp);
    }

The problem is that only the first function rsa_engine_get_priv_key() includes the code necessary to correctly parse a PKCS11 URI. So as it's currently implemented if you are signing with a hardware key and are unlucky enough to fail to initialize the engine pointer, then you will take the fall through path to rsa_pem_get_priv_key(). This function appends ".key" to your URI string, making your private key unusable and resulting in signing errors like this:

    $ ./tools/mkimage -F -N pkcs11 -k "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=e35503459cfe8566;token=uboot-token;id=%00;object=uboot-key;type=private;pin-value=1234" -K mock.dtb -r fitImage
    Couldn't open RSA private key: 'pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=e35503459cfe8566;token=uboot-token;id=%00;object=uboot-key;type=private;pin-value=1234/dev.key': No such file or directory
                                                                      ^^^^^^^^

An unintended side effect of my initial patch is that it forces the caller to take the second path IF the Provider API is available on the system, even if the engine.h header is also present. This happened because I was following the convention of equating OpenSSL version numbers with APIs.

    +#if OPENSSL_VERSION_MAJOR >= 3
    +# define USE_PKCS11_PROVIDER
    +# include <err.h>
    +# include <openssl/provider.h>
    +# include <openssl/store.h>
    +#else
    +# if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
    +#  define USE_PKCS11_ENGINE
    +#  include <openssl/engine.h>
    +# endif
    +#endif

    --->8

    static int rsa_get_priv_key(const char *keydir, const char *name,
                    const char *keyfile, ENGINE *engine, EVP_PKEY **evpp)
    {
    +#ifdef USE_PKCS11_ENGINE
        if (engine)
            return rsa_engine_get_priv_key(keydir, name, keyfile, engine,
                            evpp);
    +#endif
        return rsa_pem_get_priv_key(keydir, name, keyfile, evpp);
    }

Hardware key signing isn't broken in environments that have an Engine API provided by an older release of OpenSSL. RHEL 8 still ships OpenSSL 1, and hardware key signing works as expected with my patch:

    # openssl version
    OpenSSL 1.1.1k  FIPS 25 Mar 2021
    # cat /etc/redhat-release
    Red Hat Enterprise Linux release 8.8 (Ootpa)
    # git apply 0001-Add-support-for-OpenSSL-Provider-API.patch
    --->8

    # ./tools/mkimage -F -N pkcs11 -k "pkcs11:model=SoftHSM%20v2;manufacturer=SoftHSM%20project;serial=779ba2fecf837067;token=uboot-token;object=uboot-key;pin-value=1234" -K mock.dtb -r fitImage

    FIT description: Dummy FIT image for testing providers
    Created:         Wed Jul 22 11:21:59 2026
    --->8
    Signature written to 'fitImage', node '/configurations/config-1/signature-1'
    Public key written to 'mock.dtb', node '/signature/key-dev'

Probably the easiest way to resolve this is, as Enric suggested, to decouple the OpenSSL version from the Provider name. Although technically correct, it doesn't cover the use case where the engine.h header is available alongside the Provider API.

However, hardware key signing can still fail with the current implementation. That needs to be cleaned up, probably in a separate patch. After digging through the Git history I couldn't determine why rsa_get_priv_key() is implemented this way. There's a lot of duplication between the two functions it might call, but simply duplicating the PKCS11 URI code would not be a good solution. So I'll need to think about that some more.

Eddie

#10 lore raw
In reply to Eddie Kovsky

Hi Eddie,

Thanks for working on this, this was a good base for me to work on supporting pkcs11 and custom OpenSSL providers for signing FIT images with binman.

Quoted 114 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Changes in v3:
- Removed Kconfig option
- Changed macro symbol from CONFIG_OPENSSL_NO_DEPRECATED to USE_PKCS11_PROVIDER or USE_PKCS11_ENGINE
v2: https://lore.kernel.org/u-boot/[email protected]/

Changes in v2:
- Remove default for new Kconfig option
- Use #ifdef instead of IS_ENABLED macro
- Remove comment after #endif
- Remove unrelated checkpatch cleanup of 'sslErr' variable name
v1: https://lore.kernel.org/u-boot/[email protected]/

---
  doc/build/gcc.rst       |   4 +-
  lib/aes/aes-encrypt.c   |   4 +-
  lib/rsa/rsa-sign.c      | 102 ++++++++++++++++++++++++++++++++++++++--
  tools/docker/Dockerfile |   1 +
  4 files changed, 103 insertions(+), 8 deletions(-)
doc/build/gcc.rst +2 -2
index 1fef718ceecb..29a6a632e7e3 100644
@@ -25,8 +25,8 @@ Depending on the build targets further packages maybe needed
 
      sudo apt-get install bc bison build-essential coccinelle \
        device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
-      libgnutls28-dev libguestfs-tools libncurses-dev \
-      libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
+      libgnutls28-dev libguestfs-tools libncurses-dev libpython3-dev \
+      libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl pkcs11-provider \
        pkg-config python3 python3-asteval python3-coverage python3-filelock \
        python3-pkg-resources python3-pycryptodome python3-pyelftools \
        python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
lib/aes/aes-encrypt.c +3 -1
index 90e1407b4f09..4fc4ce232478 100644
@@ -16,7 +16,9 @@
  #include <openssl/err.h>
  #include <openssl/ssl.h>
  #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+# include <openssl/engine.h>
+#endif
  #include <uboot_aes.h>
 
  #if OPENSSL_VERSION_NUMBER >= 0x10000000L
lib/rsa/rsa-sign.c +25 -1
index 0e38c9e802fd..f456f3c58e65 100644
@@ -19,7 +19,47 @@
  #include <openssl/err.h>
  #include <openssl/ssl.h>
  #include <openssl/evp.h>
-#include <openssl/engine.h>
+#if OPENSSL_VERSION_MAJOR >= 3
+# define USE_PKCS11_PROVIDER
+# include <err.h>
+# include <openssl/provider.h>
+# include <openssl/store.h>
+#else
+# if !defined(OPENSSL_NO_ENGINE) && !defined(OPENSSL_NO_DEPRECATED_3_0)
+#  define USE_PKCS11_ENGINE
+#  include <openssl/engine.h>
+# endif
+#endif
+
+#ifdef USE_PKCS11_PROVIDER
+#define ERR(cond, fmt, ...)				\
+	do {						\
+		bool __cond = (cond);			\
+		drain_openssl_errors(__LINE__, 0);	\
+		if (__cond) {				\
+			errx(1, fmt, ## __VA_ARGS__);	\
+		}					\
+	} while (0)
+
+static void drain_openssl_errors(int l, int silent)
+{
+	const char *file;

lib/rsa/rsa-sign.c line 56

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+	char buf[120];
+	int e, line;
+
+	if (ERR_peek_error() == 0)
+		return;
+	if (!silent)
+		fprintf(stderr, "At main.c:%d:\n", l);
+
+	while ((e = ERR_peek_error_line(&file, &line))) {
+		ERR_error_string(e, buf);

buf must be at least 256 bytes long, and it's 120 here. Either increase buf[] size to 256 or use ERR_error_string_n(e, buf, 120), c.f. https://docs.openssl.org/3.0/man3/ERR_error_string/#description.

lib/rsa/rsa-sign.c line 63

Quoted 7 lines
lib/rsa/rsa-sign.c +6 -0
+		if (!silent)
+			fprintf(stderr, "- SSL %s: %s:%d\n", buf, file, line);
+		ERR_get_error();
+	}
+}
+#endif
 

OK so this was somehow unclear to me this was imported from Linux even though you stated that in your commit log. In the kernel, the macro and function are stored in scripts/ssl-common.h, I'm wondering if we can import it in a similar way (separate file) so that it's easy to keep track of changes made to the file in Linux and backport them to U-Boot whenever necessary. The origin is at least clear that way.

lib/rsa/rsa-sign.c err_cert: · line 137

Quoted 8 lines
lib/rsa/rsa-sign.c +1 -1
  static int rsa_err(const char *msg)
  {
@@ -94,10 +134,11 @@ err_cert:
   *
   * @keydir:	Key prefix
   * @name	Name of key
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider

No, because this function doesn't exist whenever provider support is enabled. Since I would like to support both providers and engines in parallel (see the other review), this function can actually be built even if provider support is enabled. However, we shouldn't be calling this function with a provider. If engine is NULL, this function isn't called, see rsa_get_pub_key().

Quoted 10 lines
lib/rsa/rsa-sign.c +1 -0
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
+#ifdef USE_PKCS11_ENGINE
  static int rsa_engine_get_pub_key(const char *keydir, const char *name,
  				  ENGINE *engine, EVP_PKEY **evpp)
  {
@@ -157,21 +198,24 @@ static int rsa_engine_get_pub_key(const char *keydir, const char *name,
 
  	return 0;

lib/rsa/rsa-sign.c static int rsa_engine_get_pub_key(const char *keydir, const char *name, · line 208

Quoted 10 lines
lib/rsa/rsa-sign.c +2 -1
  }
+#endif
 
  /**
   * rsa_get_pub_key() - read a public key
   *
   * @keydir:	Directory containing the key (PEM file) or key prefix (engine)
   * @name	Name of key file (will have a .crt extension)
- * @engine	Engine to use
+ * @engine	Engine to use or NULL when using pkcs11 provider

Well, a provider isn't an engine, so I think it's pretty clear engine is supposed to be NULL when you don't want to use an engine? Also, this applies to all providers, not only pkcs11.

Quoted 14 lines
lib/rsa/rsa-sign.c +2 -0
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
  static int rsa_get_pub_key(const char *keydir, const char *name,
  			   ENGINE *engine, EVP_PKEY **evpp)
  {
+#ifdef USE_PKCS11_ENGINE
  	if (engine)
  		return rsa_engine_get_pub_key(keydir, name, engine, evpp);
+#endif
  	return rsa_pem_get_pub_key(keydir, name, evpp);
  }
 

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 257

Quoted 7 lines
lib/rsa/rsa-sign.c +4 -0
@@ -207,13 +251,44 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
  		return -ENOENT;
  	}
 
+#ifdef USE_PKCS11_PROVIDER
+	EVP_PKEY *private_key = NULL;
+	OSSL_STORE_CTX *store;
+

This is sneaky here because it's not in the git context, but above here there is:

"""

          if (keydir && name)
                  snprintf(path, sizeof(path), "%s/%s.key", keydir, name);
          else if (keyfile)
                  snprintf(path, sizeof(path), "%s", keyfile);
          else
                  return -EINVAL;

          f = fopen(path, "r");
          if (!f) {
                  fprintf(stderr, "Couldn't open RSA private key: '%s': 
%s\n",
                          path, strerror(errno));
                  return -ENOENT;
          }

"""

This means this function will fail if path cannot be found on disk, which I am assuming is the case for every provider. I'm wondering if you actually tested this with a provider or only with a file? (the OSSL_STORE_open() supports files in path but also providers (with the scheme (e.g. pkcs11:) as prefix)) (my grep-fu may be lacking but nothing in test/ seems to do anything with pkcs11?) If your only worry is to build with an OpenSSL without engine support but have no real interest in supporting providers, then just ifdef the engine support and let provider support be done by someone else (I'm looking into it now for signing FIT images, I've got something locally already).

Also, we have some logic for generating the proper path for pkcs11 from keydir, name and keyfile for the pkcs11 engine, c.f. rsa_engine_get_pub_key(), I'm assuming we may want something similar for pkcs11 provider.

Locally, I've gone for:

"""

         if (keydir && !strncmp(pkcs11_schema, keydir, 
strlen(pkcs11_schema))) {
                 // Create the URI for the PKCS11 provider
                 if (strstr(keydir, "object=") || strstr(keydir, "id="))
                        snprintf(path, sizeof(path), "%s;type=private",
                                  keydir);
                 else
                         snprintf(path, sizeof(path),
                                  "%s;object=%s;type=private",
                                   keydir, name);
          } else if (keydir && strstr(keydir, ":")) {
                 // Create the URI for specified provider from keydir 
and name
                 snprintf(path, sizeof(path), "%s%s", keydir,
                          name ?: "");
          } else if (keydir && name) {
                 snprintf(path, sizeof(path), "%s/%s.key", keydir, name);
          } else if (keyfile) {
                  snprintf(path, sizeof(path), "%s", keyfile);
          } else {
                  return -EINVAL;
          }

"""

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 262

Quoted 5 lines
lib/rsa/rsa-sign.c +5 -0
+	if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
+		ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
+	if (!OSSL_PROVIDER_try_load(NULL, "default", true))
+		ERR(1, "OSSL_PROVIDER_try_load(default)");
+

You can ignore this with

export OPENSSL_CONF=/path/to/my/openssl.conf

"""
openssl_conf = openssl_init

[openssl_init]
providers = providers_section

[providers_section]
default = default_provider
pkcs11 = pkcs11_provider

[default_provider]
activate=1

[pkcs11_provider]
activate = 1
"""

This will allow supporting any provider, not only pkcs11 and allow to build without pkcs11-provider installed if another provider is desired. Note that you're modifying code used to sign with RSA, it's not necessarily pkcs11 (unlike in the kernel from where you took inspiration which is explicitly pkcs11-only). This also allows to pass a pin for the pkcs11 provider without U-Boot needing to know about it.

Quoted 4 lines
lib/rsa/rsa-sign.c +4 -0
+	store = OSSL_STORE_open(path, NULL, NULL, NULL, NULL);
+	ERR(!store, "OSSL_STORE_open");
+
+	while (!OSSL_STORE_eof(store)) {

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 276

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+		OSSL_STORE_INFO *info = OSSL_STORE_load(store);
+
+		if (!info) {
+			drain_openssl_errors(__LINE__, 0);
+			continue;
+		}
+		if (OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY) {
+			private_key = OSSL_STORE_INFO_get1_PKEY(info);
+			ERR(!private_key, "OSSL_STORE_INFO_get1_PKEY");
+		}

I have been told that calling OSSL_STORE_expect() with the expected type should make it easier to get the object we want directly.

Something like

OSSL_STORE_expect(store, OSSL_STORE_INFO_PKEY);

right after OSSL_STORE_open(). I'm assuming this should allow us to get rid of the OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY check since OSS_STORE_load() would only return objects of expected type.

c.f. https://docs.openssl.org/3.0/man3/OSSL_STORE_expect/#description

I've not tested this myself yet.

Quoted 7 lines
lib/rsa/rsa-sign.c +7 -0
+		OSSL_STORE_INFO_free(info);
+		if (private_key)
+			break;
+	}
+	OSSL_STORE_close(store);
+
+	*evpp = private_key;

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 292

Quoted 10 lines
lib/rsa/rsa-sign.c +2 -1
+#else
  	if (!PEM_read_PrivateKey(f, evpp, NULL, path)) {
  		rsa_err("Failure reading private key");
  		fclose(f);
  		return -EPROTO;
  	}
  	fclose(f);
-
+#endif
  	return 0;

Not much is actually shared in rsa_pem_get_priv_key between provider and no provider support so I think it may make sense to ifdef the whole content of the function or maybe the whole function even (or move the content into other functions).

Quoted 13 lines
lib/rsa/rsa-sign.c +1 -0
  }
 
@@ -226,6 +301,7 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
   * @evpp	Returns EVP_PKEY object, or NULL on failure
   * Return: 0 if ok, -ve on error (in which case *evpp will be set to NULL)
   */
+#ifdef USE_PKCS11_ENGINE
  static int rsa_engine_get_priv_key(const char *keydir, const char *name,
  				   const char *keyfile,
  				   ENGINE *engine, EVP_PKEY **evpp)
@@ -293,22 +369,25 @@ static int rsa_engine_get_priv_key(const char *keydir, const char *name,
 
  	return 0;

lib/rsa/rsa-sign.c static int rsa_engine_get_priv_key(const char *keydir, const char *name, · line 379

Quoted 10 lines
lib/rsa/rsa-sign.c +2 -1
  }
+#endif
 
  /**
   * rsa_get_priv_key() - read a private key
   *
   * @keydir:	Directory containing the key (PEM file) or key prefix (engine)
   * @name	Name of key
- * @engine	Engine to use for signing
+ * @engine	Engine to use or NULL when using pkcs11 provider

Same remark as above.

Cheers,
Quentin

#11 lore raw

Hi Eddie,

Quoted 1 line · Quentin Schulz

On 4/29/26 8:02 PM, Eddie Kovsky wrote:

[...]>> +    store = OSSL_STORE_open(path, NULL, NULL, NULL, NULL);

Quoted 18 lines

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 276

Quoted 13 lines

+    ERR(!store, "OSSL_STORE_open");

lib/rsa/rsa-sign.c +12 -0
+
+    while (!OSSL_STORE_eof(store)) {
+        OSSL_STORE_INFO *info = OSSL_STORE_load(store);
+
+        if (!info) {
+            drain_openssl_errors(__LINE__, 0);
+            continue;
+        }
+        if (OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY) {
+            private_key = OSSL_STORE_INFO_get1_PKEY(info);
+            ERR(!private_key, "OSSL_STORE_INFO_get1_PKEY");
+        }

I have been told that calling OSSL_STORE_expect() with the expected type should make it easier to get the object we want directly.

Something like

Quoted 8 lines

OSSL_STORE_expect(store, OSSL_STORE_INFO_PKEY);

right after OSSL_STORE_open(). I'm assuming this should allow us to get rid of the OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY check since OSS_STORE_load() would only return objects of expected type.

c.f. https://eur02.safelinks.protection.outlook.com/? url=https%3A%2F%2Fdocs.openssl.org%2F3.0%2Fman3%2FOSSL_STORE_expect%2F%23description&data=05%7C02%7Cquentin.schulz%40cherry.de%7C02bd298ec294469f172a08deb65a8a6d%7C5e0e1b5221b54e7b83bb514ec460677e%7C0%7C0%7C639148697211162781%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=v8m2m1Mi5UXizJjVHHUE2EPryifJfMTnDgrSgFZx1b8%3D&reserved=0

I've not tested this myself yet.

This was actually required for my custom provider. When you don't set the expected type, your provider store context's OSSL_STORE_PARAM_EXPECT param gets initialized to zero which means "return all supported OSSL_STORE_INFO_* objects except for OSSL_STORE_INFO_NAME". Our custom provider didn't implement that (it does now) and would simply say that what's requested is invalid. I'm unsure whether this means our custom provider was broken or if we should cater for this corner case and simply request a store its private key?

On a more general topic, I cannot build anything including the openssl/engine.h header without openssl-devel-engine on Fedora. This means tools/binman/test/fit/dummy-rsa-engine.c but also CST (https://gitlab.apertis.org/pkg/imx-code-signing-tool/-/blob/debian/unstable/code/back_end-ssl/src/engine.c?ref_type=heads#L11) which means we cannot run the binman test suite on Fedora without openssl-devel-engine package installed, or anything that requires CST, e.g. the nxp-imx8mcst nodes in binman. I'll try to figure out a way to handle the dummy-rsa-engine.c and be able to skip engine/provider tests if unsupported.

Someone will need to have a look at adding support for providers to CST. We download and build code from apertis.org but I'm unsure whether we're really supposed to use that or report bugs there. Debian points to https://salsa.debian.org/collabora-team/imx-code-signing-tool instead so maybe we should report there? They seem to be an import from NXP, so adding Peng in Cc, maybe they know who to ask for provider support (or at the very least, remove engine support if engine support is disabled in OpenSSL so it can compile just fine).

Cheers,
Quentin

#12 lore raw
In reply to Eddie Kovsky

Forgot to report something I fixed locally.

On 4/29/26 8:02 PM, Eddie Kovsky wrote:
[...]

Quoted 24 lines
lib/rsa/rsa-sign.c +20 -0
@@ -207,13 +251,44 @@ static int rsa_pem_get_priv_key(const char *keydir, const char *name,
  		return -ENOENT;
  	}
 
+#ifdef USE_PKCS11_PROVIDER
+	EVP_PKEY *private_key = NULL;
+	OSSL_STORE_CTX *store;
+
+	if (!OSSL_PROVIDER_try_load(NULL, "pkcs11", true))
+		ERR(1, "OSSL_PROVIDER_try_load(pkcs11)");
+	if (!OSSL_PROVIDER_try_load(NULL, "default", true))
+		ERR(1, "OSSL_PROVIDER_try_load(default)");
+
+	store = OSSL_STORE_open(path, NULL, NULL, NULL, NULL);
+	ERR(!store, "OSSL_STORE_open");
+
+	while (!OSSL_STORE_eof(store)) {
+		OSSL_STORE_INFO *info = OSSL_STORE_load(store);
+
+		if (!info) {
+			drain_openssl_errors(__LINE__, 0);
+			continue;
+		}
+		if (OSSL_STORE_INFO_get_type(info) == OSSL_STORE_INFO_PKEY) {

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 283

Quoted 10 lines
lib/rsa/rsa-sign.c +10 -0
+			private_key = OSSL_STORE_INFO_get1_PKEY(info);
+			ERR(!private_key, "OSSL_STORE_INFO_get1_PKEY");
+		}
+		OSSL_STORE_INFO_free(info);
+		if (private_key)
+			break;
+	}
+	OSSL_STORE_close(store);
+
+	*evpp = private_key;

If we reach here without actually finding a private_key, we'll return 0 a few lines down which is definitely not what we want to do. I'm suggesting:

if (!private_key)

    return -EINVAL;

Maybe it should be -ENOENT like for when we don't find the key on disk (see first line in git context in this hunk), because for some reason our logic in tools/image-host.c specifies that missing keys is allowed (???????).

lib/rsa/rsa-sign.c static int rsa_pem_get_priv_key(const char *keydir, const char *name, · line 294

Quoted 12 lines
lib/rsa/rsa-sign.c +2 -1
+#else
  	if (!PEM_read_PrivateKey(f, evpp, NULL, path)) {
  		rsa_err("Failure reading private key");
  		fclose(f);
  		return -EPROTO;
  	}
  	fclose(f);
-
+#endif
  	return 0;
  }
 

Cheers,
Quentin

#13 lore raw
In reply to Eddie Kovsky
Quoted 23 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

Quoted 7 lines

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Hey folks. So I think at this point we're still expecting a v5, and that should be able to be merged, yes? If so, is there some update on status there, or some way the rest of the community can help out? I think this is something that will really need to be sorted out for the v2027.01 release or a lot more projects are going to start having problems building U-Boot. Thanks!

Signature · 1 line
Tom
#14 lore raw

Hi Tom, Eddie,

Quoted 33 lines · Tom Rini
Quoted 23 lines · Eddie Kovsky

The Engine API has been deprecated since the release of OpenSSL 3.0. End users have been advised to migrate to the new Provider interface. Several distributions have already removed support for engines, which is preventing U-Boot from being compiled in those environments.

Add support for the Provider API while continuing to support the existing Engine API on distros shipping older releases of OpenSSL.

This is based on similar work contributed by Jan Stancek updating Linux to use the Provider interface.

commit 558bdc45dfb2669e1741384a0c80be9c82fa052c
Author: Jan Stancek <[email protected]>
Date: Fri Sep 20 19:52:48 2024 +0300

sign-file,extract-cert: use pkcs11 provider for OPENSSL MAJOR >= 3

The changes have been tested with the FIT signature verification vboot tests on Fedora 42 and Debian 13. All 30 tests pass with both the legacy Engine library installed and with the Provider API.

Tested-by Enric Balletbo i Serra <[email protected]> Tested-by Mark Kettenis <[email protected]>

Quoted 7 lines

---
Changes in v4:
- Add comment that @engine pointer is null when using pkcs11 provider
- Remove extra line break
- Add pkcs11-provider package to build dependencies
v3: https://lore.kernel.org/u-boot/[email protected]/

Quoted 7 lines

Hey folks. So I think at this point we're still expecting a v5, and that should be able to be merged, yes? If so, is there some update on status there, or some way the rest of the community can help out? I think this is something that will really need to be sorted out for the v2027.01 release or a lot more projects are going to start having problems building U-Boot. Thanks!

I think we really should focus on gating the engine code for now, so we can compile on Fedora 44 without openssl-devel-engine, and the other distros that have switched to OpenSSL 4 already (Fedora 45, Yocto, etc...).

Note that Yocto has decided to take the v4 of this patch a few months back, c.f. https://git.openembedded.org/openembedded-core/commit/?id=e7692ccf13484c8064f45dbaecd67ed8ea119cbc

We can then spend more time implementing actual OpenSSL provider support later, as this will be a new feature and doesn't have time pressure (well, for upstream U-Boot that is :) ).

Cheers,
Quentin