openssl: update from 3.0.8 to 3.1.0
Quoted 3 lines · Randy MacLeod via lists.openembedded.org
Quoted 1 line
From the NEWS.md file:
### Major changes between OpenSSL 3.0 and OpenSSL 3.1.0 [14 Mar 2023]
I know we're in feature freeze but this openssl update that came out on March 14th looks interesting.
I've sent it in before local tests complete so the Europeans / Brits can put the update through the YP autobuilder tests.
The update was pretty simple and the code change isn't huge.
There's a likely very desirable FIPS 140 updated mentioned in:
https://www.openssl.org/blog/blog/2023/03/07/OpenSSL3.1Release/
OpenSSL 3.1 is mostly a small increment over the functionality available in OpenSSL 3.0. The main changes are:
*
A FIPS 140-3 compliant FIPS Provider.
The FIPS provider has been upgraded to be FIPS 140-3 compliant. ...
Local builds are still underway but going as well as expected. See below for what I have on hand so far.
../Randy
$ cat ../b/ossl-3.1.0/openssl-buildall.log BUILDALL-QEMU LOG FOR openssl START TIME: 2023-03-14_18:09:37 HOSTNAME: yow-fedora-builder1 HOST OS: Fedora Linux 37 (Server Edition) HOST KERNEL: 6.0.14-300.fc37.x86_64 =============== BUILD RESULTS: [glibc] PASS: qemumips PASS: qemumips64 PASS: qemuppc PASS: qemuriscv32 PASS: qemuriscv64 PASS: qemux86 PASS: qemux86-64 PASS: qemuarm PASS: qemuarm64 PASS: qemuarmv5 FAIL: qemuloongarch64 PASS: qemuppc64 [musl] PASS: qemumips PASS: qemumips64 PASS: qemuppc FAIL: qemuriscv32 PASS: qemuriscv64 PASS: qemux86 PASS: qemux86-64
...
ptest are also still executing but seem to be in good shape:
Poky (Yocto Project Reference Distro) 4.1+snapshot-c9c1d6af8f00ad11cadd400c860a439349ce5b7d qemux86-64 /dev/ttyS0
qemux86-64 login: root
root@qemux86-64:~# ptest-runner openssl START: ptest-runner 2023-03-15T04:46 BEGIN: /usr/lib/openssl/ptest SKIP: 00-prep_fipsmodule_cnf.t .. skipped: FIPS module config file only supported in a fips build Files=1, Tests=0, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.38 cusr 0.11 csys = 0.51 CPU) Result: NOTESTS PASS: 01-test_abort.t .................... ok SKIP: 01-test_fipsmodule_cnf.t ........... skipped: Test only supported in a fips build PASS: 01-test_sanity.t ................... ok PASS: 01-test_symbol_presence.t .......... ok PASS: 01-test_test.t ..................... ok PASS: 02-test_errstr.t ................... ok PASS: 02-test_internal_context.t ......... ok PASS: 02-test_internal_ctype.t ........... ok PASS: 02-test_internal_exts.t ............ ok PASS: 02-test_internal_keymgmt.t ......... ok PASS: 02-test_internal_provider.t ........ ok PASS: 02-test_lhash.t .................... ok PASS: 02-test_localetest.t ............... ok PASS: 02-test_ordinals.t ................. ok PASS: 02-test_sparse_array.t ............. ok PASS: 02-test_stack.t .................... ok PASS: 03-test_exdata.t ................... ok SKIP: 03-test_fipsinstall.t .............. skipped: Test only supported in a fips build PASS: 03-test_internal_asn1.t ............ ok PASS: 03-test_internal_asn1_dsa.t ........ ok PASS: 03-test_internal_bn.t .............. ok PASS: 03-test_internal_chacha.t .......... ok PASS: 03-test_internal_curve448.t ........ ok PASS: 03-test_internal_ec.t .............. ok PASS: 03-test_internal_ffc.t ............. ok PASS: 03-test_internal_mdc2.t ............ ok PASS: 03-test_internal_modes.t ........... ok PASS: 03-test_internal_namemap.t ......... ok PASS: 03-test_internal_poly1305.t ........ ok PASS: 03-test_internal_rsa_sp800_56b.t ... ok PASS: 03-test_internal_siphash.t ......... ok PASS: 03-test_internal_sm2.t ............. ok PASS: 03-test_internal_sm3.t ............. ok PASS: 03-test_internal_sm4.t ............. ok PASS: 03-test_internal_ssl_cert_table.t .. ok PASS: 03-test_internal_x509.t ............ ok PASS: 03-test_params_api.t ............... ok PASS: 03-test_property.t ................. ok PASS: 03-test_ui.t ....................... ok PASS: 04-test_asn1_decode.t .............. ok PASS: 04-test_asn1_encode.t .............. ok PASS: 04-test_asn1_string_table.t ........ ok PASS: 04-test_bio_callback.t ............. ok PASS: 04-test_bio_core.t ................. ok PASS: 04-test_bioprint.t ................. ok PASS: 04-test_conf.t ..................... ok PASS: 04-test_encoder_decoder.t .......... ok PASS: 04-test_encoder_decoder_legacy.t ... ok PASS: 04-test_err.t ...................... ok PASS: 04-test_hexstring.t ................ ok PASS: 04-test_nodefltctx.t ............... ok PASS: 04-test_param_build.t .............. ok PASS: 04-test_params.t ................... ok PASS: 04-test_params_conversion.t ........ ok PASS: 04-test_pem_read_depr.t ............ ok PASS: 04-test_pem_reading.t .............. ok PASS: 04-test_provfetch.t ................ ok PASS: 04-test_provider.t ................. ok PASS: 04-test_provider_fallback.t ........ ok PASS: 04-test_provider_pkey.t ............ ok PASS: 04-test_punycode.t ................. ok PASS: 04-test_upcalls.t .................. ok PASS: 05-test_bf.t ....................... ok PASS: 05-test_cast.t ..................... ok PASS: 05-test_cmac.t ..................... ok PASS: 05-test_des.t ...................... ok PASS: 05-test_hmac.t ..................... ok PASS: 05-test_idea.t ..................... ok PASS: 05-test_pbe.t ...................... ok PASS: 05-test_rand.t ..................... ok PASS: 05-test_rc2.t ...................... ok PASS: 05-test_rc4.t ...................... ok SKIP: 05-test_rc5.t ...................... skipped: rc5 is not supported by this OpenSSL build PASS: 06-test_algorithmid.t .............. ok PASS: 06-test_rdcpu_sanity.t ............. ok PASS: 10-test_bn.t ....................... ok PASS: 10-test_exp.t ...................... ok PASS: 15-test_dh.t ....................... ok PASS: 15-test_dsa.t ...................... ok PASS: 15-test_dsaparam.t ................. ok PASS: 15-test_ec.t ....................... ok PASS: 15-test_ecdsa.t .................... ok PASS: 15-test_ecparam.t .................. ok PASS: 15-test_gendh.t .................... ok PASS: 15-test_gendhparam.t ............... ok PASS: 15-test_gendsa.t ................... ok PASS: 15-test_genec.t .................... ok PASS: 15-test_genrsa.t ................... ok PASS: 15-test_mp_rsa.t ................... ok PASS: 15-test_out_option.t ............... ok PASS: 15-test_rsa.t ...................... ok PASS: 15-test_rsaoaep.t .................. ok PASS: 15-test_rsapss.t ................... ok PASS: 15-test_sha.t ...................... ok PASS: 20-test_app.t ...................... ok SKIP: 20-test_cli_fips.t ................. skipped: Test only supported in a fips build with security checks PASS: 20-test_dgst.t ..................... ok PASS: 20-test_dhparam.t .................. ok PASS: 20-test_dhparam_check.t ............ ok PASS: 20-test_enc.t ...................... ok PASS: 20-test_enc_more.t ................. ok PASS: 20-test_kdf.t ...................... ok PASS: 20-test_legacy_okay.t .............. ok PASS: 20-test_mac.t ...................... ok PASS: 20-test_passwd.t ................... ok PASS: 20-test_pkeyutl.t .................. ok PASS: 20-test_rand_config.t .............. ok PASS: 20-test_spkac.t .................... ok PASS: 25-test_crl.t ...................... ok PASS: 25-test_d2i.t ...................... ok PASS: 25-test_eai_data.t ................. ok PASS: 25-test_pkcs7.t .................... ok PASS: 25-test_req.t ...................... ok PASS: 25-test_rusext.t ................... ok PASS: 25-test_sid.t ...................... ok PASS: 25-test_verify.t ................... ok PASS: 25-test_verify_store.t ............. ok PASS: 25-test_x509.t ..................... ok SKIP: 30-test_acvp.t ..................... skipped: ACVP is not supported by this test PASS: 30-test_aesgcm.t ................... ok PASS: 30-test_afalg.t .................... ok PASS: 30-test_defltfips.t ................ ok PASS: 30-test_engine.t ................... ok PASS: 30-test_evp.t ...................... ok PASS: 30-test_evp_extra.t ................ ok PASS: 30-test_evp_fetch_prov.t ........... ok PASS: 30-test_evp_kdf.t .................. ok PASS: 30-test_evp_libctx.t ............... ok PASS: 30-test_evp_pkey_dparam.t .......... ok PASS: 30-test_evp_pkey_provided.t ........ ok PASS: 30-test_pbelu.t .................... ok PASS: 30-test_pkey_meth.t ................ ok PASS: 30-test_pkey_meth_kdf.t ............ ok PASS: 30-test_prov_config.t .............. ok PASS: 30-test_provider_status.t .......... ok PASS: 40-test_rehash.t ................... ok PASS: 60-test_x509_check_cert_pkey.t ..... ok PASS: 60-test_x509_dup_cert.t ............ ok PASS: 60-test_x509_store.t ............... ok PASS: 60-test_x509_time.t ................ ok PASS: 61-test_bio_prefix.t ............... ok PASS: 61-test_bio_readbuffer.t ........... ok PASS: 65-test_cmp_asn.t .................. ok PASS: 65-test_cmp_client.t ............... ok PASS: 65-test_cmp_ctx.t .................. ok PASS: 65-test_cmp_hdr.t .................. ok PASS: 65-test_cmp_msg.t .................. ok PASS: 65-test_cmp_protect.t .............. ok PASS: 65-test_cmp_server.t ............... ok PASS: 65-test_cmp_status.t ............... ok PASS: 65-test_cmp_vfy.t .................. ok PASS: 66-test_ossl_store.t ............... ok PASS: 70-test_asyncio.t .................. ok ...
The ptests all passed or were skipped as you'd hope and expect from openssl. Results below.
../Randy
Quoted 4 lines · Randy MacLeod via lists.openembedded.org
Quoted 3 lines · Randy MacLeod via lists.openembedded.org
Quoted 1 line
From the NEWS.md file:
Quoted 12 lines
PASS: 65-test_cmp_ctx.t .................. ok
PASS: 65-test_cmp_hdr.t .................. ok
PASS: 65-test_cmp_msg.t .................. ok
PASS: 65-test_cmp_protect.t .............. ok
PASS: 65-test_cmp_server.t ............... ok
PASS: 65-test_cmp_status.t ............... ok
PASS: 65-test_cmp_vfy.t .................. ok
PASS: 66-test_ossl_store.t ............... ok
PASS: 70-test_asyncio.t .................. ok
...
PASS: 70-test_bad_dtls.t ................. ok
PASS: 70-test_clienthello.t .............. ok
PASS: 70-test_comp.t ..................... ok
PASS: 70-test_key_share.t ................ ok
PASS: 70-test_packet.t ................... ok
PASS: 70-test_recordlen.t ................ ok
PASS: 70-test_renegotiation.t ............ ok
PASS: 70-test_servername.t ............... ok
PASS: 70-test_sslcbcpadding.t ............ ok
PASS: 70-test_sslcertstatus.t ............ ok
PASS: 70-test_sslextension.t ............. ok
PASS: 70-test_sslmessages.t .............. ok
PASS: 70-test_sslrecords.t ............... ok
PASS: 70-test_sslsessiontick.t ........... ok
PASS: 70-test_sslsigalgs.t ............... ok
PASS: 70-test_sslsignature.t ............. ok
PASS: 70-test_sslskewith0p.t ............. ok
PASS: 70-test_sslversions.t .............. ok
PASS: 70-test_sslvertol.t ................ ok
PASS: 70-test_tls13alerts.t .............. ok
PASS: 70-test_tls13cookie.t .............. ok
PASS: 70-test_tls13downgrade.t ........... ok
PASS: 70-test_tls13hrr.t ................. ok
PASS: 70-test_tls13kexmodes.t ............ ok
PASS: 70-test_tls13messages.t ............ ok
PASS: 70-test_tls13psk.t ................. ok
PASS: 70-test_tlsextms.t ................. ok
PASS: 70-test_verify_extra.t ............. ok
PASS: 70-test_wpacket.t .................. ok
PASS: 71-test_ssl_ctx.t .................. ok
PASS: 79-test_http.t ..................... ok
PASS: 80-test_ca.t ....................... ok
PASS: 80-test_cipherbytes.t .............. ok
PASS: 80-test_cipherlist.t ............... ok
PASS: 80-test_ciphername.t ............... ok
PASS: 80-test_cmp_http.t ................. ok
PASS: 80-test_cms.t ...................... ok
PASS: 80-test_cmsapi.t ................... ok
PASS: 80-test_ct.t ....................... ok
PASS: 80-test_dane.t ..................... ok
PASS: 80-test_dtls.t ..................... ok
PASS: 80-test_dtls_mtu.t ................. ok
PASS: 80-test_dtlsv1listen.t ............. ok
PASS: 80-test_ocsp.t ..................... ok
PASS: 80-test_pkcs12.t ................... ok
PASS: 80-test_ssl_new.t .................. ok
PASS: 80-test_ssl_old.t .................. ok
PASS: 80-test_ssl_test_ctx.t ............. ok
PASS: 80-test_sslcorrupt.t ............... ok
PASS: 80-test_tsa.t ...................... ok
PASS: 80-test_x509aux.t .................. ok
PASS: 81-test_cmp_cli.t .................. ok
PASS: 90-test_asn1_time.t ................ ok
PASS: 90-test_async.t .................... ok
PASS: 90-test_bio_enc.t .................. ok
PASS: 90-test_bio_memleak.t .............. ok
PASS: 90-test_constant_time.t ............ ok
PASS: 90-test_fatalerr.t ................. ok
SKIP: 90-test_fipsload.t ................. skipped: Test is disabled with disabled fips
PASS: 90-test_gmdiff.t ................... ok
PASS: 90-test_ige.t ...................... ok
PASS: 90-test_includes.t ................. ok
PASS: 90-test_memleak.t .................. ok
PASS: 90-test_overhead.t ................. ok
PASS: 90-test_secmem.t ................... ok
PASS: 90-test_shlibload.t ................ ok
PASS: 90-test_srp.t ...................... ok
PASS: 90-test_sslapi.t ................... ok
PASS: 90-test_sslbuffers.t ............... ok
PASS: 90-test_store.t .................... ok
PASS: 90-test_sysdefault.t ............... ok
PASS: 90-test_threads.t .................. ok
PASS: 90-test_time_offset.t .............. ok
PASS: 90-test_tls13ccs.t ................. ok
PASS: 90-test_tls13encryption.t .......... ok
PASS: 90-test_tls13secrets.t ............. ok
PASS: 90-test_traceapi.t ................. ok
PASS: 90-test_v3name.t ................... ok
PASS: 91-test_pkey_check.t ............... ok
SKIP: 95-test_external_gost_engine.t ..... skipped: No external tests in this configuration
SKIP: 95-test_external_krb5.t ............ skipped: No external tests in this configuration
SKIP: 95-test_external_oqsprovider.t ..... skipped: No external tests in this configuration
SKIP: 95-test_external_pyca.t ............ skipped: No external tests in this configuration
SKIP: 95-test_external_tlsfuzzer.t ....... skipped: No external tests in this configuration
PASS: 99-test_ecstress.t ................. ok
PASS: 99-test_fuzz_asn1.t ................ ok
PASS: 99-test_fuzz_asn1parse.t ........... ok
PASS: 99-test_fuzz_bignum.t .............. ok
PASS: 99-test_fuzz_bndiv.t ............... ok
PASS: 99-test_fuzz_client.t .............. ok
PASS: 99-test_fuzz_cmp.t ................. ok
PASS: 99-test_fuzz_cms.t ................. ok
PASS: 99-test_fuzz_conf.t ................ ok
PASS: 99-test_fuzz_crl.t ................. ok
PASS: 99-test_fuzz_ct.t .................. ok
PASS: 99-test_fuzz_server.t .............. ok
PASS: 99-test_fuzz_x509.t ................ ok
All tests successful.
Files=250, Tests=3354, 588 wallclock secs (11.78 usr 1.17 sys + 465.74 cusr 114.46 csys = 593.15 CPU)
Result: PASS
DURATION: 589
END: /usr/lib/openssl/ptest
2023-03-15T04:56
STOP: ptest-runner
TOTAL: 1 FAIL: 0
Signature · 2 lines
# Randy MacLeod # Wind River Linux
Quoted 2 lines · Randy MacLeod via lists.openembedded.org
The ptests all passed or were skipped as you'd hope and expect from openssl. Results below.
The autobuilder shows that the updated openssl isn't reproducible:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2552/steps/12/logs/stdio
I have something more urgent to deal with next so if anyone wants to pick this up, just say so, so that we don't duplicate effort.
../Randy
Quoted 11 lines · Randy MacLeod
On 2023-03-15 09:58, Randy MacLeod via lists.openembedded.org wrote:
Quoted 3 lines
The ptests all passed or were skipped as you'd hope and expect from openssl.
Results below.The autobuilder shows that the updated openssl isn't reproducible:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2552/steps/12/logs/stdio
I have something more urgent to deal with next so if anyone wants to pick this up,
just say so, so that we don't duplicate effort.
I'm not saying I'll fix this but I did take a look just to get an idea of what is wrong.
The diffoscope output is here:
http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230315-7znox1mp/packages/diff-html/
and we see an assembler file has random strings in it.
In the openssl source this is generated by a perl script, rypto/modes/asm/aes-gcm-avx512.pl
Wearing suitable safety equipment to stop it burning, inside the perl script you can see references like:
my $rndsuffix = &random_string();
I think this is done to avoid symbol clashes but I couldn't see anything that would actually clash so more understanding of that this is doing is needed. I wondered if we could set some kind of perl seeding to make it deterministic.
I'll reply again if I learn anything more.
Cheers,
Richard
Quoted 6 lines · Richard Purdie
In the openssl source this is generated by a perl script,
rypto/modes/asm/aes-gcm-avx512.plWearing suitable safety equipment to stop it burning, inside the perl script you can see references like:
Quoted 8 lines
my $rndsuffix =
&random_string();I think this is done to avoid symbol clashes but I couldn't see anything that would actually clash so more understanding of that this is doing is needed. I wondered if we could set some kind of perl seeding to make it deterministic.
I'll reply again if I learn anything more.
Even if it's not a real fix, avx-512 support can be more or less disabled by default, as support for that is limited to high end server grade chips. We haven't enabled avx/avx2 yet :)
Alex
Quoted 15 lines · Alexander Kanavin
Quoted 13 lines · Richard Purdie
In the openssl source this is generated by a perl script,
rypto/modes/asm/aes-gcm-avx512.plWearing suitable safety equipment to stop it burning, inside the perl script you can see references like:
my $rndsuffix =
&random_string();I think this is done to avoid symbol clashes but I couldn't see anything that would actually clash so more understanding of that this is doing is needed. I wondered if we could set some kind of perl seeding to make it deterministic.
Quoted 5 lines
Quoted 1 line
I'll reply again if I learn anything more.
Even if it's not a real fix, avx-512 support can be more or less disabled by default, as support for that is limited to high end server grade chips. We haven't enabled avx/avx2 yet :)
I think the fix might be straight forward and just to add something like:
# Ensure the local labels are reproduicble
srand(10000);
I'll put that into testing, see how it looks.
Cheers,
Richard