Skip to contentSkip to conversation

sign-file.c:149:17: warning: implicit declaration of function ‘ENGINE_load_builtin_engines’

DISCUSSION Bagas Sanjaya 3 participants 9 messages Nov 16, 2023 – Nov 30, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

Quoted 173 lines

This feels like the linker is confused about where to find libssl.so.3 and/or libcrypto.so.3 for scripts/sign-file.c :

HOSTCC scripts/sign-file
scripts/sign-file.c: In function ‘read_private_key’:
scripts/sign-file.c:149:17: warning: implicit declaration of function ‘ENGINE_load_builtin_engines’ [-Wimplicit-function-declaration]

  149 |                 ENGINE_load_builtin_engines();
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sign-file.c:151:21: warning: implicit declaration of function ‘ENGINE_by_id’ [-Wimplicit-function-declaration]
  151 |                 e = ENGINE_by_id("pkcs11");
      |                     ^~~~~~~~~~~~

scripts/sign-file.c:151:19: warning: assignment to ‘ENGINE *’ {aka ‘struct engine_st *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
151 | e = ENGINE_by_id("pkcs11");
| ^
scripts/sign-file.c:153:21: warning: implicit declaration of function ‘ENGINE_init’ [-Wimplicit-function-declaration]
153 | if (ENGINE_init(e))

      |                     ^~~~~~~~~~~
scripts/sign-file.c:158:30: warning: implicit declaration of function ‘ENGINE_ctrl_cmd_string’ [-Wimplicit-function-declaration]
  158 |                         ERR(!ENGINE_ctrl_cmd_string(e, "PIN", key_pass, 0),
      |                              ^~~~~~~~~~~~~~~~~~~~~~

scripts/sign-file.c:114:32: note: in definition of macro ‘ERR’
114 | bool __cond = (cond); \
| ^~~~
scripts/sign-file.c:160:31: warning: implicit declaration of function ‘ENGINE_load_private_key’ [-Wimplicit-function-declaration]
160 | private_key = ENGINE_load_private_key(e, private_key_name,

      |                               ^~~~~~~~~~~~~~~~~~~~~~~

scripts/sign-file.c:160:29: warning: assignment to ‘EVP_PKEY *’ {aka ‘struct evp_pkey_st *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
160 | private_key = ENGINE_load_private_key(e, private_key_name,
| ^
/usr/local/bin/ld: /var/tmp/root/ccqqhtVK.o: in function `main':
sign-file.c:(.text.startup+0x346): undefined reference to `ENGINE_load_builtin_engines'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x357): undefined reference to `ENGINE_by_id'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x37e): undefined reference to `ENGINE_init'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x3a8): undefined reference to `ENGINE_ctrl_cmd_string'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x3d3): undefined reference to `ENGINE_load_private_key'
collect2: error: ld returned 1 exit status
make[1]: *** [scripts/Makefile.host:111: scripts/sign-file] Error 1
make: *** [Makefile:1273: scripts] Error 2

If I try to just deal with the sign-file.c by itself I see that I can compile and link fine with OpenSSL 3.1.4 :

e#
e# which openssl
/usr/local/bin/openssl
e#
e# ldd /usr/local/bin/openssl

        linux-vdso.so.1 (0x00007ffcfb3f7000)
        libssl.so.3 => /usr/local/lib/libssl.so.3 (0x00007f6f3b2fb000)
        libcrypto.so.3 => /usr/local/lib/libcrypto.so.3 (0x00007f6f3ac00000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6f3b2d3000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6f3aa1f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6f3b3e5000)

e#
e# openssl version
OpenSSL 3.1.4 24 Oct 2023 (Library: OpenSSL 3.1.4 24 Oct 2023) e#

I can compile the source fine ( with strange warnings ) with the
latest OpenSSL bits :

e# /usr/local/gcc13/bin/gcc -c -g -O0 -m64 -I/usr/local/include -I/usr/local/include/openssl -o scripts/sign-file.o scripts/sign-file.c scripts/sign-file.c: In function ‘read_private_key’:
scripts/sign-file.c:149:17: warning: implicit declaration of function ‘ENGINE_load_builtin_engines’ [-Wimplicit-function-declaration]

  149 |                 ENGINE_load_builtin_engines();
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sign-file.c:151:21: warning: implicit declaration of function ‘ENGINE_by_id’ [-Wimplicit-function-declaration]
  151 |                 e = ENGINE_by_id("pkcs11");
      |                     ^~~~~~~~~~~~

scripts/sign-file.c:151:19: warning: assignment to ‘ENGINE *’ {aka ‘struct engine_st *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
151 | e = ENGINE_by_id("pkcs11");
| ^
scripts/sign-file.c:117:25: warning: implicit declaration of function ‘errx’ [-Wimplicit-function-declaration]

  117 |                         errx(1, fmt, ## __VA_ARGS__);   \
      |                         ^~~~
scripts/sign-file.c:152:17: note: in expansion of macro ‘ERR’
  152 |                 ERR(!e, "Load PKCS#11 ENGINE");
      |                 ^~~
scripts/sign-file.c:153:21: warning: implicit declaration of function ‘ENGINE_init’ [-Wimplicit-function-declaration]
  153 |                 if (ENGINE_init(e))
      |                     ^~~~~~~~~~~
scripts/sign-file.c:158:30: warning: implicit declaration of function ‘ENGINE_ctrl_cmd_string’ [-Wimplicit-function-declaration]
  158 |                         ERR(!ENGINE_ctrl_cmd_string(e, "PIN", key_pass, 0),
      |                              ^~~~~~~~~~~~~~~~~~~~~~

scripts/sign-file.c:114:32: note: in definition of macro ‘ERR’
114 | bool __cond = (cond); \
| ^~~~
scripts/sign-file.c:160:31: warning: implicit declaration of function ‘ENGINE_load_private_key’ [-Wimplicit-function-declaration]
160 | private_key = ENGINE_load_private_key(e, private_key_name,

      |                               ^~~~~~~~~~~~~~~~~~~~~~~

scripts/sign-file.c:160:29: warning: assignment to ‘EVP_PKEY *’ {aka ‘struct evp_pkey_st *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
160 | private_key = ENGINE_load_private_key(e, private_key_name,
| ^
e#

Then we link gently :

e# /usr/local/gcc13/bin/gcc -g -O0 -m64 -L/usr/local/include -L/usr/local/include/openssl -Wl,-rpath=/usr/local/lib,-enable-new-dtags -o scripts/sign-file scripts/sign-file.o -lssl -lcrypto e#

e# ls -l scripts/sign-file
-rwxr-xr-x 1 root root 37800 Nov 16 16:22 scripts/sign-file e#
e# readelf -delV scripts/sign-file | grep -E 'NEED|NAME|PATH'
[10] .gnu.version_r VERNEED 0000000000400be8 00000be8

 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/usr/local/lib]
 0x000000006ffffffe (VERNEED)            0x400be8
 0x000000006fffffff (VERNEEDNUM)         2
e# 
e# ldd scripts/sign-file
        linux-vdso.so.1 (0x00007ffed8579000)
        libssl.so.3 => /usr/local/lib/libssl.so.3 (0x00007fde99d2e000)
        libcrypto.so.3 => /usr/local/lib/libcrypto.so.3 (0x00007fde99800000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fde9961f000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fde99600000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fde99e18000)

e#
e# scripts/sign-file -foo
scripts/sign-file: invalid option -- 'f'
Usage: scripts/sign-file [-dp] <hash algo> <key> <x509> <module> [<dest>]

       scripts/sign-file -s <raw sig> <hash algo> <x509> <module> [<dest>]

e#

* * * T H E R E F O R E I T C O M P I L E S F I N E * * *

However my Linux kernel compile fails on that :

e# /usr/bin/time -p /usr/bin/nice -n +18 make 2>&1 | tee ../linux-6.1.62.make_j1.log
HOSTCC scripts/sign-file
scripts/sign-file.c: In function ‘read_private_key’:
scripts/sign-file.c:149:17: warning: implicit declaration of function ‘ENGINE_load_builtin_engines’ [-Wimplicit-function-declaration]

  149 |                 ENGINE_load_builtin_engines();
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/sign-file.c:151:21: warning: implicit declaration of function ‘ENGINE_by_id’ [-Wimplicit-function-declaration]
  151 |                 e = ENGINE_by_id("pkcs11");
      |                     ^~~~~~~~~~~~

scripts/sign-file.c:151:19: warning: assignment to ‘ENGINE *’ {aka ‘struct engine_st *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
151 | e = ENGINE_by_id("pkcs11");
| ^
scripts/sign-file.c:153:21: warning: implicit declaration of function ‘ENGINE_init’ [-Wimplicit-function-declaration]
153 | if (ENGINE_init(e))

      |                     ^~~~~~~~~~~
scripts/sign-file.c:158:30: warning: implicit declaration of function ‘ENGINE_ctrl_cmd_string’ [-Wimplicit-function-declaration]
  158 |                         ERR(!ENGINE_ctrl_cmd_string(e, "PIN", key_pass, 0),
      |                              ^~~~~~~~~~~~~~~~~~~~~~

scripts/sign-file.c:114:32: note: in definition of macro ‘ERR’
114 | bool __cond = (cond); \
| ^~~~
scripts/sign-file.c:160:31: warning: implicit declaration of function ‘ENGINE_load_private_key’ [-Wimplicit-function-declaration]
160 | private_key = ENGINE_load_private_key(e, private_key_name,

      |                               ^~~~~~~~~~~~~~~~~~~~~~~

scripts/sign-file.c:160:29: warning: assignment to ‘EVP_PKEY *’ {aka ‘struct evp_pkey_st *’} from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
160 | private_key = ENGINE_load_private_key(e, private_key_name,
| ^
/usr/local/bin/ld: /var/tmp/root/ccqqhtVK.o: in function `main':
sign-file.c:(.text.startup+0x346): undefined reference to `ENGINE_load_builtin_engines'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x357): undefined reference to `ENGINE_by_id'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x37e): undefined reference to `ENGINE_init'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x3a8): undefined reference to `ENGINE_ctrl_cmd_string'
/usr/local/bin/ld: sign-file.c:(.text.startup+0x3d3): undefined reference to `ENGINE_load_private_key'
collect2: error: ld returned 1 exit status
make[1]: *** [scripts/Makefile.host:111: scripts/sign-file] Error 1
make: *** [Makefile:1273: scripts] Error 2
real 0.92
user 0.74
sys 0.18
e#

Quoted 7 lines

Looks like some confusion with the linker ?

Setting LDFLAGS=-rpath=/usr/local/openssl/lib -enable-new-dtag seems to mean nothing.

Not related to https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

Thanks.

[1]: https://bugzilla.kernel.org/show_bug.cgi?id=218154

Signature · 1 line
An old man doll... just what I always wanted! - Clara
#1 lore raw
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 8 lines
Quoted 2 lines

Not related to https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

It fails to compile in the usual way when trying to build the kernel.

Dennis

#2 lore raw
Quoted 15 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 8 lines
Quoted 2 lines

Not related to https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

Quoted 6 lines

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

It fails to compile in the usual way when trying to build the kernel.

Hi Thorsten and all,

AFAIK there is no reply from kbuild people (maybe they missed this bug?).

As for the error itself, let me clarify. The reporter (Dennis) have a build problem with /usr/local version of OpenSSL library. He installed it (presumably) alongside with system version (which is installed to /usr), hence I called the /usr/local version as custom one (IDK if that version is vanilla OpenSSL or not). Maybe am I missing something?

Thanks.

Signature · 1 line
An old man doll... just what I always wanted! - Clara
#3 lore raw
Quoted 20 lines · Bagas Sanjaya
Quoted 19 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 8 lines
Quoted 2 lines

Not related to https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

Quoted 6 lines
Quoted 2 lines

It fails to compile in the usual way when trying to build the kernel.

Hi Thorsten and all,

AFAIK there is no reply from kbuild people (maybe they missed this bug?).

I suspect no one is even looking at it. Just us :)

Quoted 7 lines

As for the error itself, let me clarify. The reporter (Dennis) have a build problem with /usr/local version of OpenSSL library. He installed it (presumably) alongside with system version (which is installed to /usr), hence I called the /usr/local version as custom one (IDK if that version is vanilla OpenSSL or not). Maybe am I missing something?

Well, today we have OpenSSL 3.2.0 published :

https://www.openssl.org/source/

So I will try again and this time I will begin with a very stripped down install of Debian stable. When I mean stripped down I mean there will be nothing in the base install except reasonable TCP/IP network support and the ability to FTP a file into it. I can extract a tarball of a decent compiler that I bootstrapped myself with excellent results :

    https://gcc.gnu.org/pipermail/gcc-testresults/2023-August/794816.html

This will be done on real hardware. Not a virtual machine of any type where that should make no difference at all. Anyone running ESXi would see that same results I am seeing anyways.

I want to see if the old sign-file.c code is simply using deprecated calls and then rewrite/patch as needed. This means the OpenSSL involved will be the production 3.2.0 release published today.

Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

#4 lore raw
In reply to Dennis Clarke
Quoted 12 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 6 lines
Quoted 3 lines

Not related to
https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

Quoted 7 lines
Quoted 3 lines

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

This has all the hallmarks of an openssl compiled without engine support; is the symbol OPENSSL_NO_ENGINE set? And which distro did you get this library from?

James

#5 lore raw
Quoted 21 lines · James Bottomley
Quoted 12 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 6 lines
Quoted 3 lines

Not related to
https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

Quoted 7 lines
Quoted 3 lines

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

Quoted 6 lines

This has all the hallmarks of an openssl compiled without engine support; is the symbol OPENSSL_NO_ENGINE set? And which distro did you get this library from?

James

I am looking into this. The code will likely age into some deprecated calls and I think that I may be way out on the edge here. However the code will need a pile of ifndef stuff and then call the correct future looking calls for OpenSSL 3.x etc etc etc ... the usual stuff

Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

#6 lore raw
Quoted 27 lines · Dennis Clarke
Quoted 26 lines · James Bottomley
Quoted 12 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 6 lines
Quoted 3 lines

Not related to
https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

Quoted 8 lines
Quoted 3 lines

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

This has all the hallmarks of an openssl compiled without engine support; is the symbol OPENSSL_NO_ENGINE set?  And which distro did you get this library from?

Quoted 5 lines
Quoted 2 lines

James

I am looking into this. The code will likely age into some deprecated calls and I think that I may be way out on the edge here.

So you did build without engine support ...

Quoted 3 lines

However the code will need a pile of ifndef stuff and then call the
correct future looking calls for OpenSSL 3.x etc etc etc ... the usual stuff

Well, not really: openssl is highly configurable and if it gets configured wrongly, stuff like this happens. That's why distros have a fairly inclusive configuration and they stick to it. No-one can cope with the combinatoric explosion of openssl configuration possibilities (even though they have ifdefs for most of them) so the only way is really to fix a standard configuration and assume you're building for it.

Openssl has been talking for ages about removing engine support, but they've been unable to do so due to the rather slow pace of conversion of their own engines. I anticipate this code can be removed in favour of the pkcs11 provider long before openssl actually manages to remove engines.

James

#7 lore raw
Quoted 11 lines · James Bottomley
Quoted 10 lines · Dennis Clarke
Quoted 9 lines · James Bottomley
Quoted 8 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 2 lines
Quoted 2 lines

Not related to
https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I

.
. <snip>
.

Quoted 5 lines
Quoted 3 lines

I am looking into this. The code will likely age into some deprecated calls and I think that I may be way out on the edge here.

So you did build without engine support ...

Yep.

--prefix=/usr/local no-asm shared no-engine no-hw threads zlib
sctp enable-weak-ssl-ciphers -DPEDANTIC -D_REENTRANT

So there we see the "no-engine" option. That pretty much kicks the sign-file.c code to the curb.

Quoted 6 lines
Quoted 3 lines

However the code will need a pile of ifndef stuff and then call the
correct future looking calls for OpenSSL 3.x etc etc etc ... the usual stuff

Well, not really: openssl is highly configurable and if it gets configured wrongly, stuff like this happens.

Well, not "wrongly". More like "not the usual off the shelf stuff".

Quoted 6 lines

That's why distros have a
fairly inclusive configuration and they stick to it. No-one can cope with the combinatoric explosion of openssl configuration possibilities (even though they have ifdefs for most of them) so the only way is really to fix a standard configuration and assume you're building for it.

Seems clear to me.

Quoted 7 lines

Openssl has been talking for ages about removing engine support, but they've been unable to do so due to the rather slow pace of conversion of their own engines. I anticipate this code can be removed in favour of the pkcs11 provider long before openssl actually manages to remove engines.

James

Well I thank you for the clarity here. I still feel that sign-file.c needs a bit of a rewrite and I guess the old expression "patches are welcome" works here.

Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken

#8 lore raw
In reply to James Bottomley
Quoted 21 lines · James Bottomley
Quoted 12 lines · Dennis Clarke
Quoted 4 lines · Bagas Sanjaya

Hi,

I notice a bug report on Bugzilla [1]. Quoting from it:

<snip>

Quoted 6 lines
Quoted 3 lines

Not related to
https://bugzilla.kernel.org/show_bug.cgi?id=215750 but I *feel* that this code needs a hug.

See Bugzilla for the full thread.

Quoted 7 lines
Quoted 3 lines

AFAIK, this looks like a bug when the kernel is compiled against custom (non-system) version of OpenSSL library.

I do not know what you could possibly mean. There is nothing "custom" about OpenSSL. For that matter the gcc compiler I am using was also built by me. Works fine. The sign-file.c source compiles fine.

Quoted 6 lines

This has all the hallmarks of an openssl compiled without engine support; is the symbol OPENSSL_NO_ENGINE set? And which distro did you get this library from?

James

Its okay.

I can fix it.

To work with new OpenSSL 3.2.0 and old stuff .. for a while ....

Signature · 4 lines
--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken