Skip to contentSkip to conversation

openssl: update from 3.0.8 to 3.1.0

PATCH Randy MacLeod 3 participants 6 messages Mar 15, 2023 – Mar 16, 2023 3 messages not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive
Quoted 3 lines · Randy MacLeod via lists.openembedded.org
Quoted 1 line

From the NEWS.md file:

   ### Major changes between OpenSSL 3.0 and OpenSSL 3.1.0 [14 Mar 2023]

I know we're in feature freeze but this openssl update that came out on March 14th looks interesting.

I've sent it in before local tests complete so the Europeans / Brits can put the update through the YP autobuilder tests.

The update was pretty simple and the code change isn't huge.

There's a likely very desirable FIPS 140 updated mentioned in:

https://www.openssl.org/blog/blog/2023/03/07/OpenSSL3.1Release/

    OpenSSL 3.1 is mostly a small increment over the functionality available in OpenSSL 3.0. The main changes are:

*

A FIPS 140-3 compliant FIPS Provider.

The FIPS provider has been upgraded to be FIPS 140-3 compliant. ...

Local builds are still underway but going as well as expected. See below for what I have on hand so far.

../Randy

$ cat ../b/ossl-3.1.0/openssl-buildall.log
BUILDALL-QEMU LOG FOR openssl
START TIME: 2023-03-14_18:09:37
HOSTNAME: yow-fedora-builder1
HOST OS: Fedora Linux 37 (Server Edition)
HOST KERNEL: 6.0.14-300.fc37.x86_64
===============
BUILD RESULTS:
[glibc]
PASS: qemumips
PASS: qemumips64
PASS: qemuppc
PASS: qemuriscv32
PASS: qemuriscv64
PASS: qemux86
PASS: qemux86-64
PASS: qemuarm
PASS: qemuarm64
PASS: qemuarmv5
FAIL: qemuloongarch64
PASS: qemuppc64
[musl]
PASS: qemumips
PASS: qemumips64
PASS: qemuppc
FAIL: qemuriscv32
PASS: qemuriscv64
PASS: qemux86
PASS: qemux86-64

...

ptest are also still executing but seem to be in good shape:

Poky (Yocto Project Reference Distro) 4.1+snapshot-c9c1d6af8f00ad11cadd400c860a439349ce5b7d qemux86-64 /dev/ttyS0

qemux86-64 login: root

root@qemux86-64:~# ptest-runner openssl
START: ptest-runner
2023-03-15T04:46
BEGIN: /usr/lib/openssl/ptest
SKIP: 00-prep_fipsmodule_cnf.t .. skipped: FIPS module config file only supported in a fips build
Files=1, Tests=0,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.38 cusr  0.11 csys =  0.51 CPU)
Result: NOTESTS
PASS: 01-test_abort.t .................... ok
SKIP: 01-test_fipsmodule_cnf.t ........... skipped: Test only supported in a fips build
PASS: 01-test_sanity.t ................... ok
PASS: 01-test_symbol_presence.t .......... ok
PASS: 01-test_test.t ..................... ok
PASS: 02-test_errstr.t ................... ok
PASS: 02-test_internal_context.t ......... ok
PASS: 02-test_internal_ctype.t ........... ok
PASS: 02-test_internal_exts.t ............ ok
PASS: 02-test_internal_keymgmt.t ......... ok
PASS: 02-test_internal_provider.t ........ ok
PASS: 02-test_lhash.t .................... ok
PASS: 02-test_localetest.t ............... ok
PASS: 02-test_ordinals.t ................. ok
PASS: 02-test_sparse_array.t ............. ok
PASS: 02-test_stack.t .................... ok
PASS: 03-test_exdata.t ................... ok
SKIP: 03-test_fipsinstall.t .............. skipped: Test only supported in a fips build
PASS: 03-test_internal_asn1.t ............ ok
PASS: 03-test_internal_asn1_dsa.t ........ ok
PASS: 03-test_internal_bn.t .............. ok
PASS: 03-test_internal_chacha.t .......... ok
PASS: 03-test_internal_curve448.t ........ ok
PASS: 03-test_internal_ec.t .............. ok
PASS: 03-test_internal_ffc.t ............. ok
PASS: 03-test_internal_mdc2.t ............ ok
PASS: 03-test_internal_modes.t ........... ok
PASS: 03-test_internal_namemap.t ......... ok
PASS: 03-test_internal_poly1305.t ........ ok
PASS: 03-test_internal_rsa_sp800_56b.t ... ok
PASS: 03-test_internal_siphash.t ......... ok
PASS: 03-test_internal_sm2.t ............. ok
PASS: 03-test_internal_sm3.t ............. ok
PASS: 03-test_internal_sm4.t ............. ok
PASS: 03-test_internal_ssl_cert_table.t .. ok
PASS: 03-test_internal_x509.t ............ ok
PASS: 03-test_params_api.t ............... ok
PASS: 03-test_property.t ................. ok
PASS: 03-test_ui.t ....................... ok
PASS: 04-test_asn1_decode.t .............. ok
PASS: 04-test_asn1_encode.t .............. ok
PASS: 04-test_asn1_string_table.t ........ ok
PASS: 04-test_bio_callback.t ............. ok
PASS: 04-test_bio_core.t ................. ok
PASS: 04-test_bioprint.t ................. ok
PASS: 04-test_conf.t ..................... ok
PASS: 04-test_encoder_decoder.t .......... ok
PASS: 04-test_encoder_decoder_legacy.t ... ok
PASS: 04-test_err.t ...................... ok
PASS: 04-test_hexstring.t ................ ok
PASS: 04-test_nodefltctx.t ............... ok
PASS: 04-test_param_build.t .............. ok
PASS: 04-test_params.t ................... ok
PASS: 04-test_params_conversion.t ........ ok
PASS: 04-test_pem_read_depr.t ............ ok
PASS: 04-test_pem_reading.t .............. ok
PASS: 04-test_provfetch.t ................ ok
PASS: 04-test_provider.t ................. ok
PASS: 04-test_provider_fallback.t ........ ok
PASS: 04-test_provider_pkey.t ............ ok
PASS: 04-test_punycode.t ................. ok
PASS: 04-test_upcalls.t .................. ok
PASS: 05-test_bf.t ....................... ok
PASS: 05-test_cast.t ..................... ok
PASS: 05-test_cmac.t ..................... ok
PASS: 05-test_des.t ...................... ok
PASS: 05-test_hmac.t ..................... ok
PASS: 05-test_idea.t ..................... ok
PASS: 05-test_pbe.t ...................... ok
PASS: 05-test_rand.t ..................... ok
PASS: 05-test_rc2.t ...................... ok
PASS: 05-test_rc4.t ...................... ok
SKIP: 05-test_rc5.t ...................... skipped: rc5 is not supported by this OpenSSL build
PASS: 06-test_algorithmid.t .............. ok
PASS: 06-test_rdcpu_sanity.t ............. ok
PASS: 10-test_bn.t ....................... ok
PASS: 10-test_exp.t ...................... ok
PASS: 15-test_dh.t ....................... ok
PASS: 15-test_dsa.t ...................... ok
PASS: 15-test_dsaparam.t ................. ok
PASS: 15-test_ec.t ....................... ok
PASS: 15-test_ecdsa.t .................... ok
PASS: 15-test_ecparam.t .................. ok
PASS: 15-test_gendh.t .................... ok
PASS: 15-test_gendhparam.t ............... ok
PASS: 15-test_gendsa.t ................... ok
PASS: 15-test_genec.t .................... ok
PASS: 15-test_genrsa.t ................... ok
PASS: 15-test_mp_rsa.t ................... ok
PASS: 15-test_out_option.t ............... ok
PASS: 15-test_rsa.t ...................... ok
PASS: 15-test_rsaoaep.t .................. ok
PASS: 15-test_rsapss.t ................... ok
PASS: 15-test_sha.t ...................... ok
PASS: 20-test_app.t ...................... ok
SKIP: 20-test_cli_fips.t ................. skipped: Test only supported in a fips build with security checks
PASS: 20-test_dgst.t ..................... ok
PASS: 20-test_dhparam.t .................. ok
PASS: 20-test_dhparam_check.t ............ ok
PASS: 20-test_enc.t ...................... ok
PASS: 20-test_enc_more.t ................. ok
PASS: 20-test_kdf.t ...................... ok
PASS: 20-test_legacy_okay.t .............. ok
PASS: 20-test_mac.t ...................... ok
PASS: 20-test_passwd.t ................... ok
PASS: 20-test_pkeyutl.t .................. ok
PASS: 20-test_rand_config.t .............. ok
PASS: 20-test_spkac.t .................... ok
PASS: 25-test_crl.t ...................... ok
PASS: 25-test_d2i.t ...................... ok
PASS: 25-test_eai_data.t ................. ok
PASS: 25-test_pkcs7.t .................... ok
PASS: 25-test_req.t ...................... ok
PASS: 25-test_rusext.t ................... ok
PASS: 25-test_sid.t ...................... ok
PASS: 25-test_verify.t ................... ok
PASS: 25-test_verify_store.t ............. ok
PASS: 25-test_x509.t ..................... ok
SKIP: 30-test_acvp.t ..................... skipped: ACVP is not supported by this test
PASS: 30-test_aesgcm.t ................... ok
PASS: 30-test_afalg.t .................... ok
PASS: 30-test_defltfips.t ................ ok
PASS: 30-test_engine.t ................... ok
PASS: 30-test_evp.t ...................... ok
PASS: 30-test_evp_extra.t ................ ok
PASS: 30-test_evp_fetch_prov.t ........... ok
PASS: 30-test_evp_kdf.t .................. ok
PASS: 30-test_evp_libctx.t ............... ok
PASS: 30-test_evp_pkey_dparam.t .......... ok
PASS: 30-test_evp_pkey_provided.t ........ ok
PASS: 30-test_pbelu.t .................... ok
PASS: 30-test_pkey_meth.t ................ ok
PASS: 30-test_pkey_meth_kdf.t ............ ok
PASS: 30-test_prov_config.t .............. ok
PASS: 30-test_provider_status.t .......... ok
PASS: 40-test_rehash.t ................... ok
PASS: 60-test_x509_check_cert_pkey.t ..... ok
PASS: 60-test_x509_dup_cert.t ............ ok
PASS: 60-test_x509_store.t ............... ok
PASS: 60-test_x509_time.t ................ ok
PASS: 61-test_bio_prefix.t ............... ok
PASS: 61-test_bio_readbuffer.t ........... ok
PASS: 65-test_cmp_asn.t .................. ok
PASS: 65-test_cmp_client.t ............... ok
PASS: 65-test_cmp_ctx.t .................. ok
PASS: 65-test_cmp_hdr.t .................. ok
PASS: 65-test_cmp_msg.t .................. ok
PASS: 65-test_cmp_protect.t .............. ok
PASS: 65-test_cmp_server.t ............... ok
PASS: 65-test_cmp_status.t ............... ok
PASS: 65-test_cmp_vfy.t .................. ok
PASS: 66-test_ossl_store.t ............... ok
PASS: 70-test_asyncio.t .................. ok
...
#3 lore raw
In reply to a message not in the archive

The ptests all passed or were skipped as you'd hope and expect from openssl. Results below.

../Randy

Quoted 4 lines · Randy MacLeod via lists.openembedded.org
Quoted 3 lines · Randy MacLeod via lists.openembedded.org
Quoted 1 line

From the NEWS.md file:

Quoted 12 lines

PASS: 65-test_cmp_ctx.t .................. ok
PASS: 65-test_cmp_hdr.t .................. ok
PASS: 65-test_cmp_msg.t .................. ok
PASS: 65-test_cmp_protect.t .............. ok
PASS: 65-test_cmp_server.t ............... ok
PASS: 65-test_cmp_status.t ............... ok
PASS: 65-test_cmp_vfy.t .................. ok
PASS: 66-test_ossl_store.t ............... ok
PASS: 70-test_asyncio.t .................. ok
...

PASS: 70-test_bad_dtls.t ................. ok
PASS: 70-test_clienthello.t .............. ok
PASS: 70-test_comp.t ..................... ok
PASS: 70-test_key_share.t ................ ok
PASS: 70-test_packet.t ................... ok
PASS: 70-test_recordlen.t ................ ok
PASS: 70-test_renegotiation.t ............ ok
PASS: 70-test_servername.t ............... ok
PASS: 70-test_sslcbcpadding.t ............ ok
PASS: 70-test_sslcertstatus.t ............ ok
PASS: 70-test_sslextension.t ............. ok
PASS: 70-test_sslmessages.t .............. ok
PASS: 70-test_sslrecords.t ............... ok
PASS: 70-test_sslsessiontick.t ........... ok
PASS: 70-test_sslsigalgs.t ............... ok
PASS: 70-test_sslsignature.t ............. ok
PASS: 70-test_sslskewith0p.t ............. ok
PASS: 70-test_sslversions.t .............. ok
PASS: 70-test_sslvertol.t ................ ok
PASS: 70-test_tls13alerts.t .............. ok
PASS: 70-test_tls13cookie.t .............. ok
PASS: 70-test_tls13downgrade.t ........... ok
PASS: 70-test_tls13hrr.t ................. ok
PASS: 70-test_tls13kexmodes.t ............ ok
PASS: 70-test_tls13messages.t ............ ok
PASS: 70-test_tls13psk.t ................. ok
PASS: 70-test_tlsextms.t ................. ok
PASS: 70-test_verify_extra.t ............. ok
PASS: 70-test_wpacket.t .................. ok
PASS: 71-test_ssl_ctx.t .................. ok
PASS: 79-test_http.t ..................... ok
PASS: 80-test_ca.t ....................... ok
PASS: 80-test_cipherbytes.t .............. ok
PASS: 80-test_cipherlist.t ............... ok
PASS: 80-test_ciphername.t ............... ok
PASS: 80-test_cmp_http.t ................. ok
PASS: 80-test_cms.t ...................... ok
PASS: 80-test_cmsapi.t ................... ok
PASS: 80-test_ct.t ....................... ok
PASS: 80-test_dane.t ..................... ok
PASS: 80-test_dtls.t ..................... ok
PASS: 80-test_dtls_mtu.t ................. ok
PASS: 80-test_dtlsv1listen.t ............. ok
PASS: 80-test_ocsp.t ..................... ok
PASS: 80-test_pkcs12.t ................... ok
PASS: 80-test_ssl_new.t .................. ok
PASS: 80-test_ssl_old.t .................. ok
PASS: 80-test_ssl_test_ctx.t ............. ok
PASS: 80-test_sslcorrupt.t ............... ok
PASS: 80-test_tsa.t ...................... ok
PASS: 80-test_x509aux.t .................. ok
PASS: 81-test_cmp_cli.t .................. ok
PASS: 90-test_asn1_time.t ................ ok
PASS: 90-test_async.t .................... ok
PASS: 90-test_bio_enc.t .................. ok
PASS: 90-test_bio_memleak.t .............. ok
PASS: 90-test_constant_time.t ............ ok
PASS: 90-test_fatalerr.t ................. ok
SKIP: 90-test_fipsload.t ................. skipped: Test is disabled with disabled fips
PASS: 90-test_gmdiff.t ................... ok
PASS: 90-test_ige.t ...................... ok
PASS: 90-test_includes.t ................. ok
PASS: 90-test_memleak.t .................. ok
PASS: 90-test_overhead.t ................. ok
PASS: 90-test_secmem.t ................... ok
PASS: 90-test_shlibload.t ................ ok
PASS: 90-test_srp.t ...................... ok
PASS: 90-test_sslapi.t ................... ok
PASS: 90-test_sslbuffers.t ............... ok
PASS: 90-test_store.t .................... ok
PASS: 90-test_sysdefault.t ............... ok
PASS: 90-test_threads.t .................. ok
PASS: 90-test_time_offset.t .............. ok
PASS: 90-test_tls13ccs.t ................. ok
PASS: 90-test_tls13encryption.t .......... ok
PASS: 90-test_tls13secrets.t ............. ok
PASS: 90-test_traceapi.t ................. ok
PASS: 90-test_v3name.t ................... ok
PASS: 91-test_pkey_check.t ............... ok
SKIP: 95-test_external_gost_engine.t ..... skipped: No external tests in this configuration
SKIP: 95-test_external_krb5.t ............ skipped: No external tests in this configuration
SKIP: 95-test_external_oqsprovider.t ..... skipped: No external tests in this configuration
SKIP: 95-test_external_pyca.t ............ skipped: No external tests in this configuration
SKIP: 95-test_external_tlsfuzzer.t ....... skipped: No external tests in this configuration
PASS: 99-test_ecstress.t ................. ok
PASS: 99-test_fuzz_asn1.t ................ ok
PASS: 99-test_fuzz_asn1parse.t ........... ok
PASS: 99-test_fuzz_bignum.t .............. ok
PASS: 99-test_fuzz_bndiv.t ............... ok
PASS: 99-test_fuzz_client.t .............. ok
PASS: 99-test_fuzz_cmp.t ................. ok
PASS: 99-test_fuzz_cms.t ................. ok
PASS: 99-test_fuzz_conf.t ................ ok
PASS: 99-test_fuzz_crl.t ................. ok
PASS: 99-test_fuzz_ct.t .................. ok
PASS: 99-test_fuzz_server.t .............. ok
PASS: 99-test_fuzz_x509.t ................ ok
All tests successful.
Files=250, Tests=3354, 588 wallclock secs (11.78 usr  1.17 sys + 465.74 cusr 114.46 csys = 593.15 CPU)
Result: PASS
DURATION: 589
END: /usr/lib/openssl/ptest
2023-03-15T04:56
STOP: ptest-runner
TOTAL: 1 FAIL: 0

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#5 lore raw
In reply to a message not in the archive
Quoted 2 lines · Randy MacLeod via lists.openembedded.org

The ptests all passed or were skipped as you'd hope and expect from openssl. Results below.

The autobuilder shows that the updated openssl isn't reproducible:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2552/steps/12/logs/stdio

I have something more urgent to deal with next so if anyone wants to pick this up, just say so, so that we don't duplicate effort.

../Randy

#6 lore raw
Quoted 11 lines · Randy MacLeod

On 2023-03-15 09:58, Randy MacLeod via lists.openembedded.org wrote:

Quoted 3 lines

The ptests all passed or were skipped as you'd hope and expect from openssl.
 Results below.

The autobuilder shows that the updated openssl isn't reproducible:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2552/steps/12/logs/stdio
I have something more urgent to deal with next so if anyone wants to pick this up,
 just say so, so that we don't duplicate effort.

I'm not saying I'll fix this but I did take a look just to get an idea of what is wrong.

The diffoscope output is here:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230315-7znox1mp/packages/diff-html/

and we see an assembler file has random strings in it.

In the openssl source this is generated by a perl script, rypto/modes/asm/aes-gcm-avx512.pl

Wearing suitable safety equipment to stop it burning, inside the perl script you can see references like:

my $rndsuffix = &random_string();

I think this is done to avoid symbol clashes but I couldn't see anything that would actually clash so more understanding of that this is doing is needed. I wondered if we could set some kind of perl seeding to make it deterministic.

I'll reply again if I learn anything more.

Cheers,

Richard

#7 lore raw
Quoted 6 lines · Richard Purdie

In the openssl source this is generated by a perl script, rypto/modes/asm/aes-gcm-avx512.pl

Wearing suitable safety equipment to stop it burning, inside the perl script you can see references like:

Quoted 8 lines

my $rndsuffix = &random_string();

I think this is done to avoid symbol clashes but I couldn't see anything that would actually clash so more understanding of that this is doing is needed. I wondered if we could set some kind of perl seeding to make it deterministic.

I'll reply again if I learn anything more.

Even if it's not a real fix, avx-512 support can be more or less disabled by default, as support for that is limited to high end server grade chips. We haven't enabled avx/avx2 yet :)

Alex

#8 lore raw
Quoted 15 lines · Alexander Kanavin
Quoted 13 lines · Richard Purdie

In the openssl source this is generated by a perl script, rypto/modes/asm/aes-gcm-avx512.pl

Wearing suitable safety equipment to stop it burning, inside the perl script you can see references like:

my $rndsuffix = &random_string();

I think this is done to avoid symbol clashes but I couldn't see anything that would actually clash so more understanding of that this is doing is needed. I wondered if we could set some kind of perl seeding to make it deterministic.

Quoted 5 lines
Quoted 1 line

I'll reply again if I learn anything more.

Even if it's not a real fix, avx-512 support can be more or less disabled by default, as support for that is limited to high end server grade chips. We haven't enabled avx/avx2 yet :)

I think the fix might be straight forward and just to add something like:

# Ensure the local labels are reproduicble
srand(10000);

I'll put that into testing, see how it looks.

Cheers,

Richard