openssl: Remove the c_rehash shell re-implementation
PATCH v3 1/1 openssl: Remove the c_rehash shell re-implementation
We had a c_rehash shell re-implementation being used for the native package however the ca-certificates now uses the openssl rehash internal application so there is no use for the c_rehash anymore.
---
Changes in v3:
- remove c_rehash completely
- fix ca-certificates recipe comment
Changes in v2:
- updated commit log
.../openssl/openssl/openssl-c_rehash.sh | 222 ------------------ .../openssl/openssl_1.1.1a.bb | 13 +- .../ca-certificates_20190110.bb | 2 +- 3 files changed, 2 insertions(+), 235 deletions(-) delete mode 100644 meta/recipes-connectivity/openssl/openssl/openssl-c_rehash.sh
meta/recipes-connectivity/openssl/openssl/openssl-c_rehash.sh deleted +0 -222
@@ -1,222 +0,0 @@ -#!/bin/sh -# -# Ben Secrest <[email protected]> -# -# sh c_rehash script, scan all files in a directory -# and add symbolic links to their hash values. -# -# based on the c_rehash perl script distributed with openssl -# -# LICENSE: See OpenSSL license -# ^^acceptable?^^ -# - -# default certificate location -DIR=/etc/openssl - -# for filetype bitfield -IS_CERT=$(( 1 << 0 )) -IS_CRL=$(( 1 << 1 )) - - -# check to see if a file is a certificate file or a CRL file -# arguments: -# 1. the filename to be scanned -# returns: -# bitfield of file type; uses ${IS_CERT} and ${IS_CRL} -# -check_file() -{ - local IS_TYPE=0 - - # make IFS a newline so we can process grep output line by line - local OLDIFS=${IFS} - IFS=$( printf "\n" ) - - # XXX: could be more efficient to have two 'grep -m' but is -m portable? - for LINE in $( grep '^-----BEGIN .*-----' ${1} ) - do - if echo ${LINE} \ - | grep -q -E '^-----BEGIN (X509 |TRUSTED )?CERTIFICATE-----' - then - IS_TYPE=$(( ${IS_TYPE} | ${IS_CERT} )) - - if [ $(( ${IS_TYPE} & ${IS_CRL} )) -ne 0 ] - then - break - fi - elif echo ${LINE} | grep -q '^-----BEGIN X509 CRL-----' - then - IS_TYPE=$(( ${IS_TYPE} | ${IS_CRL} )) - - if [ $(( ${IS_TYPE} & ${IS_CERT} )) -ne 0 ] - then - break - fi - fi - done - - # restore IFS - IFS=${OLDIFS} - - return ${IS_TYPE} -} - - -# -# use openssl to fingerprint a file -# arguments: -# 1. the filename to fingerprint -# 2. the method to use (x509, crl) -# returns: -# none -# assumptions: -# user will capture output from last stage of pipeline -# -fingerprint() -{ - ${SSL_CMD} ${2} -fingerprint -noout -in ${1} | sed 's/^.*=//' | tr -d ':' -} - - -# -# link_hash - create links to certificate files -# arguments: -# 1. the filename to create a link for -# 2. the type of certificate being linked (x509, crl) -# returns: -# 0 on success, 1 otherwise -# -link_hash() -{ - local FINGERPRINT=$( fingerprint ${1} ${2} ) - local HASH=$( ${SSL_CMD} ${2} -hash -noout -in ${1} ) - local SUFFIX=0 - local LINKFILE='' - local TAG='' - - if [ ${2} = "crl" ] - then - TAG='r' - fi - - LINKFILE=${HASH}.${TAG}${SUFFIX} - - while [ -f ${LINKFILE} ] - do - if [ ${FINGERPRINT} = $( fingerprint ${LINKFILE} ${2} ) ] - then - echo "NOTE: Skipping duplicate file ${1}" >&2 - return 1 - fi - - SUFFIX=$(( ${SUFFIX} + 1 )) - LINKFILE=${HASH}.${TAG}${SUFFIX} - done - - echo "${3} => ${LINKFILE}" - - # assume any system with a POSIX shell will either support symlinks or - # do something to handle this gracefully - ln -s ${3} ${LINKFILE} - - return 0 -} - - -# hash_dir create hash links in a given directory -hash_dir() -{ - echo "Doing ${1}" - - cd ${1} - - ls -1 * 2>/dev/null | while read FILE - do - if echo ${FILE} | grep -q -E '^[[:xdigit:]]{8}\.r?[[:digit:]]+$' \ - && [ -h "${FILE}" ] - then - rm ${FILE} - fi - done - - ls -1 *.pem *.cer *.crt *.crl 2>/dev/null | while read FILE - do - REAL_FILE=${FILE} - # if we run on build host then get to the real files in rootfs - if [ -n "${SYSROOT}" -a -h ${FILE} ] - then - FILE=$( readlink ${FILE} ) - # check the symlink is absolute (or dangling in other word) - if [ "x/" = "x$( echo ${FILE} | cut -c1 -)" ] - then - REAL_FILE=${SYSROOT}/${FILE} - fi - fi - - check_file ${REAL_FILE} - local FILE_TYPE=${?} - local TYPE_STR='' - - if [ $(( ${FILE_TYPE} & ${IS_CERT} )) -ne 0 ] - then - TYPE_STR='x509' - elif [ $(( ${FILE_TYPE} & ${IS_CRL} )) -ne 0 ] - then - TYPE_STR='crl' - else - echo "NOTE: ${FILE} does not contain a certificate or CRL: skipping" >&2 - continue - fi - - link_hash ${REAL_FILE} ${TYPE_STR} ${FILE} - done -} - - -# choose the name of an ssl application -if [ -n "${OPENSSL}" ] -then - SSL_CMD=$(which ${OPENSSL} 2>/dev/null) -else - SSL_CMD=/usr/bin/openssl - OPENSSL=${SSL_CMD} - export OPENSSL -fi - -# fix paths -PATH=${PATH}:${DIR}/bin -export PATH - -# confirm existance/executability of ssl command -if ! [ -x ${SSL_CMD} ] -then - echo "${0}: rehashing skipped ('openssl' program not available)" >&2 - exit 0 -fi - -# determine which directories to process -old_IFS=$IFS -if [ ${#} -gt 0 ] -then - IFS=':' - DIRLIST=${*} -elif [ -n "${SSL_CERT_DIR}" ] -then - DIRLIST=$SSL_CERT_DIR -else - DIRLIST=${DIR}/certs -fi - -IFS=':' - -# process directories -for CERT_DIR in ${DIRLIST} -do - if [ -d ${CERT_DIR} -a -w ${CERT_DIR} ] - then - IFS=$old_IFS - hash_dir ${CERT_DIR} - IFS=':' - fi -done
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb +1 -12
@@ -13,7 +13,6 @@ DEPENDS = "hostperl-runtime-native" SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz \ file://run-ptest \ - file://openssl-c_rehash.sh \ file://0001-skip-test_symbol_presence.patch \ file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \ file://afalg.patch \
@@ -149,12 +148,6 @@ do_install_append_class-native () { SSL_CERT_DIR=${libdir}/ssl-1.1/certs \ SSL_CERT_FILE=${libdir}/ssl-1.1/cert.pem \ OPENSSL_ENGINES=${libdir}/ssl-1.1/engines - - # Install a custom version of c_rehash that can handle sysroots properly. - # This version is used for example when installing ca-certificates during - # image creation. - install -Dm 0755 ${WORKDIR}/openssl-c_rehash.sh ${D}${bindir}/c_rehash - sed -i -e 's,/etc/openssl,${sysconfdir}/ssl,g' ${D}${bindir}/c_rehash } do_install_append_class-nativesdk () {
@@ -196,7 +189,7 @@ FILES_libcrypto = "${libdir}/libcrypto${SOLIBS}" FILES_libssl = "${libdir}/libssl${SOLIBS}" FILES_openssl-conf = "${sysconfdir}/ssl/openssl.cnf" FILES_${PN}-engines = "${libdir}/engines-1.1" -FILES_${PN}-misc = "${libdir}/ssl-1.1/misc ${bindir}/c_rehash" +FILES_${PN}-misc = "${libdir}/ssl-1.1/misc" FILES_${PN} =+ "${libdir}/ssl-1.1/*" FILES_${PN}_append_class-nativesdk = " ${SDKPATHNATIVE}/environment-setup.d/openssl.sh"
@@ -211,7 +204,3 @@ RREPLACES_openssl-conf = "openssl10-conf" RCONFLICTS_openssl-conf = "openssl10-conf" BBCLASSEXTEND = "native nativesdk" - -inherit multilib_script - -MULTILIB_SCRIPTS = "${PN}-bin:${bindir}/c_rehash"
meta/recipes-support/ca-certificates/ca-certificates_20190110.bb +1 -1
@@ -11,7 +11,7 @@ LIC_FILES_CHKSUM = "file://debian/copyright;md5=aeb420429b1659507e0a5a1b123e8308 DEPENDS = "" DEPENDS_class-native = "openssl-native" DEPENDS_class-nativesdk = "openssl-native" -# Need c_rehash from openssl and run-parts from debianutils +# Need rehash from openssl and run-parts from debianutils PACKAGE_WRITE_DEPS += "openssl-native debianutils-native" SRCREV = "c28799b138b044c963d24c4a69659b6e5486e3be"
Signature · 1 line
2.21.0
Just FYI:
This won't work if someone uses only openssl10 in the image (a bit difficult to do with current master, but still some people have to do it), in case someone needs it as well, here is how Gentoo makes ca-certificate backwards compatible with old openssl:
https://github.com/gentoo/gentoo/commit/03f9b674ca3315198c72849e8dd77583974759c2#diff-1801ddca78e57240592ef16b1c5262e7
more details in:
https://bugs.gentoo.org/653382
Basically using (good old) shell c_rehash implementation from separate recipe:
https://github.com/gentoo/gentoo/blob/master/app-misc/c_rehash/c_rehash-1.7-r1.ebuild
instead of "openssl rehash" in update-ca-certificates script.
Cheers,
Quoted 327 lines · Otavio Salvador
We had a
c_rehashshell re-implementation being used for the native package however the ca-certificates now uses the openssl rehash internal application so there is no use for thec_rehashanymore.Signed-off-by Otavio Salvador <[email protected]>---
Changes in v3:
- removec_rehashcompletely
- fix ca-certificates recipe commentChanges in v2:
- updated commit log.../openssl/openssl/openssl-c_rehash.sh | 222 ------------------ .../openssl/openssl_1.1.1a.bb | 13 +- .../ca-certificates_20190110.bb | 2 +- 3 files changed, 2 insertions(+), 235 deletions(-) delete mode 100644
meta/recipes-connectivity/openssl/openssl/openssl-c_rehash.shmeta/recipes-connectivity/openssl/openssl/openssl-c_rehash.sh deleted +0
-36@@ -1,222 +0,0 @@ -#!/bin/sh -# -# Ben Secrest <[email protected]> -# -# sh c_rehash script, scan all files in a directory -# and add symbolic links to their hash values. -# -# based on the c_rehash perl script distributed with openssl -# -# LICENSE: See OpenSSL license -# ^^acceptable?^^ -# - -# default certificate location -DIR=/etc/openssl - -# for filetype bitfield -IS_CERT=$(( 1 << 0 )) -IS_CRL=$(( 1 << 1 )) - - -# check to see if a file is a certificate file or a CRL file -# arguments: -# 1. the filename to be scanned -# returns: -# bitfield of file type; uses ${IS_CERT} and ${IS_CRL} -# -check_file() -{ - local IS_TYPE=0 - - # make IFS a newline so we can process grep output line by line - local OLDIFS=${IFS} - IFS=$( printf "\n" ) - - # XXX: could be more efficient to have two 'grep -m' but is -mportable?
meta/recipes-connectivity/openssl/openssl/openssl-c_rehash.sh deleted +0
-42- for LINE in $( grep '^-----BEGIN .*-----' ${1} ) - do - if echo ${LINE} \ - | grep -q -E '^-----BEGIN (X509 |TRUSTED )?CERTIFICATE-----' - then - IS_TYPE=$(( ${IS_TYPE} | ${IS_CERT} )) - - if [ $(( ${IS_TYPE} & ${IS_CRL} )) -ne 0 ] - then - break - fi - elif echo ${LINE} | grep -q '^-----BEGIN X509 CRL-----' - then - IS_TYPE=$(( ${IS_TYPE} | ${IS_CRL} )) - - if [ $(( ${IS_TYPE} & ${IS_CERT} )) -ne 0 ] - then - break - fi - fi - done - - # restore IFS - IFS=${OLDIFS} - - return ${IS_TYPE} -} - - -# -# use openssl to fingerprint a file -# arguments: -# 1. the filename to fingerprint -# 2. the method to use (x509, crl) -# returns: -# none -# assumptions: -# user will capture output from last stage of pipeline -# -fingerprint() -{ - ${SSL_CMD} ${2} -fingerprint -noout -in ${1} | sed 's/^.*=//' | tr -d':'
meta/recipes-connectivity/openssl/openssl/openssl-c_rehash.sh deleted +0
-90-} - - -# -# link_hash - create links to certificate files -# arguments: -# 1. the filename to create a link for -# 2. the type of certificate being linked (x509, crl) -# returns: -# 0 on success, 1 otherwise -# -link_hash() -{ - local FINGERPRINT=$( fingerprint ${1} ${2} ) - local HASH=$( ${SSL_CMD} ${2} -hash -noout -in ${1} ) - local SUFFIX=0 - local LINKFILE='' - local TAG='' - - if [ ${2} = "crl" ] - then - TAG='r' - fi - - LINKFILE=${HASH}.${TAG}${SUFFIX} - - while [ -f ${LINKFILE} ] - do - if [ ${FINGERPRINT} = $( fingerprint ${LINKFILE} ${2} ) ] - then - echo "NOTE: Skipping duplicate file ${1}" >&2 - return 1 - fi - - SUFFIX=$(( ${SUFFIX} + 1 )) - LINKFILE=${HASH}.${TAG}${SUFFIX} - done - - echo "${3} => ${LINKFILE}" - - # assume any system with a POSIX shell will either support symlinks or - # do something to handle this gracefully - ln -s ${3} ${LINKFILE} - - return 0 -} - - -# hash_dir create hash links in a given directory -hash_dir() -{ - echo "Doing ${1}" - - cd ${1} - - ls -1 * 2>/dev/null | while read FILE - do - if echo ${FILE} | grep -q -E '^[[:xdigit:]]{8}\.r?[[:digit:]]+$' \ - && [ -h "${FILE}" ] - then - rm ${FILE} - fi - done - - ls -1 *.pem *.cer *.crt *.crl 2>/dev/null | while read FILE - do - REAL_FILE=${FILE} - # if we run on build host then get to the real files in rootfs - if [ -n "${SYSROOT}" -a -h ${FILE} ] - then - FILE=$( readlink ${FILE} ) - # check the symlink is absolute (or dangling in other word) - if [ "x/" = "x$( echo ${FILE} | cut -c1 -)" ] - then - REAL_FILE=${SYSROOT}/${FILE} - fi - fi - - check_file ${REAL_FILE} - local FILE_TYPE=${?} - local TYPE_STR='' - - if [ $(( ${FILE_TYPE} & ${IS_CERT} )) -ne 0 ] - then - TYPE_STR='x509' - elif [ $(( ${FILE_TYPE} & ${IS_CRL} )) -ne 0 ] - then - TYPE_STR='crl' - else - echo "NOTE: ${FILE} does not contain a certificate or CRL:skipping" >&2
meta/recipes-connectivity/openssl/openssl/openssl-c_rehash.sh deleted +0
-54- continue - fi - - link_hash ${REAL_FILE} ${TYPE_STR} ${FILE} - done -} - - -# choose the name of an ssl application -if [ -n "${OPENSSL}" ] -then - SSL_CMD=$(which ${OPENSSL} 2>/dev/null) -else - SSL_CMD=/usr/bin/openssl - OPENSSL=${SSL_CMD} - export OPENSSL -fi - -# fix paths -PATH=${PATH}:${DIR}/bin -export PATH - -# confirm existance/executability of ssl command -if ! [ -x ${SSL_CMD} ] -then - echo "${0}: rehashing skipped ('openssl' program not available)" >&2 - exit 0 -fi - -# determine which directories to process -old_IFS=$IFS -if [ ${#} -gt 0 ] -then - IFS=':' - DIRLIST=${*} -elif [ -n "${SSL_CERT_DIR}" ] -then - DIRLIST=$SSL_CERT_DIR -else - DIRLIST=${DIR}/certs -fi - -IFS=':' - -# process directories -for CERT_DIR in ${DIRLIST} -do - if [ -d ${CERT_DIR} -a -w ${CERT_DIR} ] - then - IFS=$old_IFS - hash_dir ${CERT_DIR} - IFS=':' - fi -donemeta/recipes-connectivity/openssl/openssl_1.1.1a.bb +0
-1@@ -13,7 +13,6 @@ DEPENDS = "hostperl-runtime-native" SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz \ file://run-ptest \ - file://openssl-c_rehash.sh \ file://0001-skip-test_symbol_presence.patch \ file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \file://afalg.patch \meta/recipes-connectivity/openssl/openssl_1.1.1a.bb +0
-2@@ -149,12 +148,6 @@ do_install_append_class-native () { SSL_CERT_DIR=${libdir}/ssl-1.1/certs \ SSL_CERT_FILE=${libdir}/ssl-1.1/cert.pem \ OPENSSL_ENGINES=${libdir}/ssl-1.1/engines - - # Install a custom version of c_rehash that can handle sysrootsproperly.
- # This version is used for example when installing ca-certificates
duringmeta/recipes-connectivity/openssl/openssl_1.1.1a.bb +0
-2- # image creation. - install -Dm 0755 ${WORKDIR}/openssl-c_rehash.sh${D}${bindir}/c_rehash
- sed -i -e 's,/etc/openssl,${sysconfdir}/ssl,g'
${D}${bindir}/c_rehash
}do_install_append_class-nativesdk () {
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb +1
-1@@ -196,7 +189,7 @@ FILES_libcrypto = "${libdir}/libcrypto${SOLIBS}" FILES_libssl = "${libdir}/libssl${SOLIBS}" FILES_openssl-conf = "${sysconfdir}/ssl/openssl.cnf" FILES_${PN}-engines = "${libdir}/engines-1.1" -FILES_${PN}-misc = "${libdir}/ssl-1.1/misc ${bindir}/c_rehash" +FILES_${PN}-misc = "${libdir}/ssl-1.1/misc" FILES_${PN} =+ "${libdir}/ssl-1.1/*" FILES_${PN}_append_class-nativesdk = "
${SDKPATHNATIVE}/environment-setup.d/openssl.sh"meta/recipes-connectivity/openssl/openssl_1.1.1a.bb +0
-4@@ -211,7 +204,3 @@ RREPLACES_openssl-conf = "openssl10-conf" RCONFLICTS_openssl-conf = "openssl10-conf" BBCLASSEXTEND = "native nativesdk" - -inherit multilib_script - -MULTILIB_SCRIPTS = "${PN}-bin:${bindir}/c_rehash"(quoted diff) +0
-0ca-certificates_20190110.bb b/meta/recipes-support/ca-certificates/ ca-certificates_20190110.bb
indexb9f57900c8..4c0425302f100644meta/recipes-support/ca-certificates/ca-certificates_20190110.bb +0
-0@@ -11,7 +11,7 @@ LIC_FILES_CHKSUM ="file://debian/copyright;md5=aeb420429b1659507e0a5a1b123e8308
DEPENDS = ""
DEPENDS_class-native = "openssl-native" DEPENDS_class-nativesdk = "openssl-native"meta/recipes-support/ca-certificates/ca-certificates_20190110.bb +1
-2-# Need c_rehash from openssl and run-parts from debianutils +# Need rehash from openssl and run-parts from debianutils PACKAGE_WRITE_DEPS += "openssl-native debianutils-native" SRCREV = "c28799b138b044c963d24c4a69659b6e5486e3be" --2.21.0
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Hello Martin,
Quoted 4 lines · Martin Jansa
This won't work if someone uses only openssl10 in the image (a bit difficult to do with current master, but still some people have to do it), in case someone needs it as well, here is how Gentoo makes ca-certificate backwards compatible with old openssl:
https://github.com/gentoo/gentoo/commit/03f9b674ca3315198c72849e8dd77583974759c2#diff-1801ddca78e57240592ef16b1c5262e7
more details in:
https://bugs.gentoo.org/653382
On OE-Core case, even if someone uses openssl10 as the library provider, it will still use the openssl 1.1 binaries under target. See:
+PACKAGE_PREPROCESS_FUNCS += "openssl_package_preprocess"
+
+# openssl 1.0 development files and executable binaries clash with openssl 1.1 +# files when installed into target rootfs. So we don't put them into +# packages, but they continue to be provided via target sysroot for +# cross-compilation on the host, if some software still depends on openssl 1.0. +openssl_package_preprocess () {
+ for file in find ${PKGD} -name *.h -o -name *.pc -o -name *.so; do
+ rm $file
+ done
+ rm ${PKGD}${bindir}/openssl
+ rm ${PKGD}${bindir}/c_rehash
+ rmdir ${PKGD}${bindir}+
+}
So for OE-Core, it is not supported. The rootfs generation will still work, it seems.
Signature · 3 lines
Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750