External CSL toolchain improvements
cover External CSL toolchain improvements
See the commit message summaries for details, but this syncs up my local external-csl-toolchain recipe with upstream, which includes general cleanup and use of base_libdir to fix issues with mips64, splits out libstdc++-staticdev, adds CSL_VER_MAIN to target recipe signatures, and fixes and uses precompiled locales rather than regenerating them.
Most of this is straightforward, but I would appreciate review in particular of the "libc-package: rework 'precompiled' locale handling". Thanks.
The following changes since commit 26d822ad059dd4839cc6fee2a65714e73cc146e1:
nfs-utils: don't try to unload nfsd module (2012-05-01 14:28:47 +0100)
are available in the git repository at:
git://github.com/kergoth/oe-core external-csl-improvements
Christopher Larson (8):
csl-versions.inc: capture version in signatures
external-csl-toolchain: split out a libstdc++-staticdev package
external-csl-toolchain: silence .a/.debug QA warnings
external-csl-toolchain: cleanup, simplify, use base_libdir
external-csl-toolchain: update SUPPORTED to match toolchain
tclibc-eglibc: drop hardcoded locale-base-en-us/en-gb
libc-package: rework ''precompiled' locale handling
tcmode-external-csl: use the existing precompiled locale files
meta/classes/libc-package.bbclass | 96 ++++---- meta/conf/distro/include/csl-versions.inc | 4 + meta/conf/distro/include/tclibc-eglibc.inc | 12 +- meta/conf/distro/include/tcmode-external-csl.inc | 4 + meta/recipes-core/meta/external-csl-toolchain.bb | 68 +++--- .../meta/external-csl-toolchain/SUPPORTED | 257 +++++++++++++++++++- 6 files changed, 346 insertions(+), 95 deletions(-)
Christopher Larson (8):
csl-versions.inc: capture version in signatures
external-csl-toolchain: split out a libstdc++-staticdev package
external-csl-toolchain: silence .a/.debug QA warnings
external-csl-toolchain: cleanup, simplify, use base_libdir
external-csl-toolchain: update SUPPORTED to match toolchain
tclibc-eglibc: drop hardcoded locale-base-en-us/en-gb
libc-package: rework ''precompiled' locale handling
tcmode-external-csl: use the existing precompiled locale files
meta/classes/libc-package.bbclass | 96 ++++---- meta/conf/distro/include/csl-versions.inc | 4 + meta/conf/distro/include/tclibc-eglibc.inc | 12 +- meta/conf/distro/include/tcmode-external-csl.inc | 4 + meta/recipes-core/meta/external-csl-toolchain.bb | 68 +++--- .../meta/external-csl-toolchain/SUPPORTED | 257 +++++++++++++++++++- 6 files changed, 346 insertions(+), 95 deletions(-)
Signature · 1 line
1.7.7
PATCH 1/8 csl-versions.inc: capture version in signatures
We want to ensure that changing external toolchain version will change the metadata checksums of target recipes. This will do so via ensuring that any variable which references TOOLCHAIN_OPTIONS also pulls in the toolchain version variables.
--- meta/conf/distro/include/csl-versions.inc | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
meta/conf/distro/include/csl-versions.inc +4 -0
@@ -103,3 +103,7 @@ python csl_version_handler () { d.setVar('CSL_VER_GDB', csl_get_gdb_version(ld)) } addhandler csl_version_handler + +# Ensure that any variable which includes the --sysroot (CC, CXX, etc) also +# depends on the toolchain version +TOOLCHAIN_OPTIONS[vardeps] += "CSL_VER_MAIN CSL_VER_GCC"
Signature · 1 line
1.7.7
PATCH 2/8 external-csl-toolchain: split out a libstdc++-staticdev package
In reply to Christopher LarsonThis silences one of the QA warnings.
--- meta/recipes-core/meta/external-csl-toolchain.bb | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
meta/recipes-core/meta/external-csl-toolchain.bb +4 -4
@@ -88,7 +88,7 @@ external_toolchain_sysroot_adjust() { fi } -PACKAGES =+ "libgcc libgcc-dev libstdc++ libstdc++-dev linux-libc-headers linux-libc-headers-dev gdbserver gdbserver-dbg" +PACKAGES =+ "libgcc libgcc-dev libstdc++ libstdc++-dev libstdc++-staticdev linux-libc-headers linux-libc-headers-dev gdbserver gdbserver-dbg" INSANE_SKIP_libgcc = "1" INSANE_SKIP_libstdc++ = "1"
@@ -111,6 +111,7 @@ PKGV_libgcc = "${CSL_VER_GCC}" PKGV_libgcc-dev = "${CSL_VER_GCC}" PKGV_libstdc++ = "${CSL_VER_GCC}" PKGV_libstdc++-dev = "${CSL_VER_GCC}" +PKGV_libstdc++-staticdev = "${CSL_VER_GCC}" PKGV_linux-libc-headers = "${CSL_VER_KERNEL}" PKGV_linux-libc-headers-dev = "${CSL_VER_KERNEL}" PKGV_gdbserver = "${CSL_VER_GDB}"
@@ -122,9 +123,8 @@ FILES_libstdc++ = "${libdir}/libstdc++.so.*" FILES_libstdc++-dev = "${includedir}/c++/${PV} \ ${libdir}/libstdc++.so \ ${libdir}/libstdc++.la \ - ${libdir}/libstdc++.a \ - ${libdir}/libsupc++.la \ - ${libdir}/libsupc++.a" + ${libdir}/libsupc++.la" +FILES_libstdc++-staticdev = "${libdir}/libstdc++.a ${libdir}/libsupc++.a" FILES_linux-libc-headers = "${includedir}/asm* \ ${includedir}/linux \ ${includedir}/mtd \
Signature · 1 line
1.7.7
PATCH 3/8 external-csl-toolchain: silence .a/.debug QA warnings
In reply to Christopher Larson--- meta/recipes-core/meta/external-csl-toolchain.bb | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)
meta/recipes-core/meta/external-csl-toolchain.bb +2 -3
@@ -90,9 +90,8 @@ external_toolchain_sysroot_adjust() { PACKAGES =+ "libgcc libgcc-dev libstdc++ libstdc++-dev libstdc++-staticdev linux-libc-headers linux-libc-headers-dev gdbserver gdbserver-dbg" -INSANE_SKIP_libgcc = "1" -INSANE_SKIP_libstdc++ = "1" -INSANE_SKIP_gdbserver = "1" +# This test should be fixed to ignore .a files in .debug dirs +INSANE_SKIP_${PN}-dbg = "staticdev" PKG_${PN} = "eglibc" PKG_${PN}-dev = "eglibc-dev"
Signature · 1 line
1.7.7
PATCH 4/8 external-csl-toolchain: cleanup, simplify, use base_libdir
In reply to Christopher Larson--- meta/recipes-core/meta/external-csl-toolchain.bb | 55 +++++++++++----------- 1 files changed, 27 insertions(+), 28 deletions(-)
meta/recipes-core/meta/external-csl-toolchain.bb +27 -28
@@ -24,36 +24,40 @@ PROVIDES += "\ virtual/linux-libc-headers \ " PV = "${CSL_VER_MAIN}" -PR = "r3" +PR = "r5" #SRC_URI = "http://www.codesourcery.com/public/gnu_toolchain/${CSL_TARGET_SYS}/arm-${PV}-${TARGET_PREFIX}i686-pc-linux-gnu.tar.bz2" SRC_URI = "file://SUPPORTED" do_install() { - install -d ${D}${sysconfdir} ${D}${bindir} ${D}${sbindir} ${D}${base_bindir} ${D}${libdir} - install -d ${D}${base_libdir} ${D}${base_sbindir} ${D}${datadir} ${D}/usr + # Use optimized files if available + sysroot="${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc" + if [ -d $sysroot/${CSL_TARGET_CORE} ]; then + sysroot="$sysroot/${CSL_TARGET_CORE}" + fi + + cp -a $sysroot${base_libdir}/. ${D}${base_libdir} + cp -a $sysroot/etc/. ${D}${sysconfdir} + cp -a $sysroot/sbin/. ${D}${base_sbindir} - if [ -d ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE} ]; then - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/lib/. ${D}${base_libdir} - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/etc/. ${D}${sysconfdir} - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/sbin/. ${D}${base_sbindir} - if [ ! -e ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/include ]; then - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/include ${D}/usr/ + install -d ${D}/usr + for usr_element in bin libexec sbin share ${base_libdir}; do + usr_path=$sysroot/usr/$usr_element + cp -a $usr_path ${D}/usr/ + done + for datadir_element in man info; do + datadir_path=$sysroot/usr/$datadir_element + if [ -e $datadir_path ]; then + cp -a $datadir_path ${D}${datadir}/ fi - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/. ${D}/usr/ - else - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/lib/. ${D}${base_libdir} - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/etc/. ${D}${sysconfdir} - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/sbin/. ${D}${base_sbindir} - cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/. ${D}/usr/ - fi + done - if [ -e ${D}${prefix}/info ]; then - mv ${D}${prefix}/info ${D}${infodir} - fi - if [ -e ${D}${prefix}/man ]; then - mv ${D}${prefix}/man ${D}${mandir} + # Some toolchains have headers under the core specific area + if [ -e $sysroot/usr/include ]; then + cp -a $sysroot/usr/include/. ${D}${includedir} + else + cp -a $sysroot/../usr/include/. ${D}${includedir} fi rm ${D}${sysconfdir}/rpc
@@ -61,18 +65,13 @@ do_install() { mv ${D}${libdir}/bin/* ${D}${bindir}/ if [ -e ${D}${libdir}/bin/.debug ]; then - install -d ${D}${bindir}/.debug mv ${D}${libdir}/bin/.debug/* ${D}${bindir}/.debug/ fi ln -s ../../bin/gdbserver ${D}${libdir}/bin/sysroot-gdbserver sed -i -e 's/__packed/__attribute__ ((packed))/' ${D}${includedir}/mtd/ubi-user.h - sed -i -e "s# /lib# ../../lib#g" -e "s# /usr/lib# .#g" ${D}${libdir}/libc.so - sed -i -e "s# /lib# ../../lib#g" -e "s# /usr/lib# .#g" ${D}${libdir}/libpthread.so -} - -do_install_locale_append () { - rm -r ${D}${datadir}/locale ${D}${libdir}/locale + sed -i -e "s# ${base_libdir}# ../..${base_libdir}#g" -e "s# ${libdir}# .#g" ${D}${libdir}/libc.so + sed -i -e "s# ${base_libdir}# ../..${base_libdir}#g" -e "s# ${libdir}# .#g" ${D}${libdir}/libpthread.so } SYSROOT_PREPROCESS_FUNCS += "external_toolchain_sysroot_adjust"
Signature · 1 line
1.7.7
PATCH 6/8 tclibc-eglibc: drop hardcoded locale-base-en-us/en-gb
In reply to Christopher LarsonThis is handled by IMAGE_LINGUAS, so hardcoding it here is at best unnecessary duplication.
--- meta/conf/distro/include/tclibc-eglibc.inc | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-)
meta/conf/distro/include/tclibc-eglibc.inc +5 -7
@@ -19,10 +19,10 @@ CXXFLAGS += "-fvisibility-inlines-hidden" LIBC_DEPENDENCIES = "libsegfault \ eglibc \ - eglibc-dbg \ - eglibc-dev \ - eglibc-utils \ - eglibc-thread-db \ + eglibc-dbg \ + eglibc-dev \ + eglibc-utils \ + eglibc-thread-db \ ${@get_libc_locales_dependencies(d)}" LIBC_LOCALE_DEPENDENCIES = "\
@@ -30,9 +30,7 @@ LIBC_LOCALE_DEPENDENCIES = "\ eglibc-gconv-ibm850 \ eglibc-gconv-cp1252 \ eglibc-gconv-iso8859-1 \ - eglibc-gconv-iso8859-15 \ - locale-base-en-us \ - locale-base-en-gb" + eglibc-gconv-iso8859-15" def get_libc_locales_dependencies(d): if 'libc-locales' in (d.getVar('DISTRO_FEATURES', True) or '').split() :
Signature · 1 line
1.7.7
PATCH 5/8 external-csl-toolchain: update SUPPORTED to match toolchain
In reply to Christopher Larson--- .../meta/external-csl-toolchain/SUPPORTED | 257 +++++++++++++++++++- 1 files changed, 255 insertions(+), 2 deletions(-)
meta/recipes-core/meta/external-csl-toolchain/SUPPORTED +255 -2
@@ -1,4 +1,257 @@ -en_GB.UTF-8 UTF-8 +aa_DJ ISO-8859-1 +aa_ER UTF-8 +aa_ER@saaho UTF-8 +aa_ET UTF-8 +af_ZA ISO-8859-1 +am_ET UTF-8 +an_ES ISO-8859-15 +ar_AE ISO-8859-6 +ar_BH ISO-8859-6 +ar_DZ ISO-8859-6 +ar_EG ISO-8859-6 +ar_IN UTF-8 +ar_IQ ISO-8859-6 +ar_JO ISO-8859-6 +ar_KW ISO-8859-6 +ar_LB ISO-8859-6 +ar_LY ISO-8859-6 +ar_MA ISO-8859-6 +ar_OM ISO-8859-6 +ar_QA ISO-8859-6 +ar_SA ISO-8859-6 +ar_SD ISO-8859-6 +ar_SY ISO-8859-6 +ar_TN ISO-8859-6 +ar_YE ISO-8859-6 +ast_ES ISO-8859-15 +be_BY CP1251 +be_BY@latin UTF-8 +ber_DZ UTF-8 +ber_MA UTF-8 +bg_BG CP1251 +bn_BD UTF-8 +bn_IN UTF-8 +bo_CN UTF-8 +bo_IN UTF-8 +br_FR ISO-8859-1 +br_FR@euro ISO-8859-15 +bs_BA ISO-8859-2 +byn_ER UTF-8 +ca_AD ISO-8859-15 +ca_ES ISO-8859-1 +ca_ES@euro ISO-8859-15 +ca_FR ISO-8859-15 +ca_IT ISO-8859-15 +crh_UA UTF-8 +cs_CZ ISO-8859-2 +csb_PL UTF-8 +cy_GB ISO-8859-14 +da_DK ISO-8859-1 +de_AT ISO-8859-1 +de_AT@euro ISO-8859-15 +de_BE ISO-8859-1 +de_BE@euro ISO-8859-15 +de_CH ISO-8859-1 +de_DE ISO-8859-1 +de_DE@euro ISO-8859-15 +de_LU ISO-8859-1 +de_LU@euro ISO-8859-15 +dv_MV UTF-8 +dz_BT UTF-8 +el_GR ISO-8859-7 +el_CY ISO-8859-7 +en_AG UTF-8 +en_AU ISO-8859-1 +en_BW ISO-8859-1 +en_CA ISO-8859-1 +en_DK ISO-8859-1 en_GB ISO-8859-1 -en_US.UTF-8 UTF-8 +en_HK ISO-8859-1 +en_IE ISO-8859-1 +en_IE@euro ISO-8859-15 +en_IN UTF-8 +en_NG UTF-8 +en_NZ ISO-8859-1 +en_PH ISO-8859-1 +en_SG ISO-8859-1 en_US ISO-8859-1 +en_ZA ISO-8859-1 +en_ZW ISO-8859-1 +es_AR ISO-8859-1 +es_BO ISO-8859-1 +es_CL ISO-8859-1 +es_CO ISO-8859-1 +es_DO ISO-8859-1 +es_EC ISO-8859-1 +es_ES ISO-8859-1 +es_ES@euro ISO-8859-15 +es_GT ISO-8859-1 +es_HN ISO-8859-1 +es_MX ISO-8859-1 +es_NI ISO-8859-1 +es_PA ISO-8859-1 +es_PE ISO-8859-1 +es_PR ISO-8859-1 +es_PY ISO-8859-1 +es_SV ISO-8859-1 +es_US ISO-8859-1 +es_UY ISO-8859-1 +es_VE ISO-8859-1 +et_EE ISO-8859-1 +eu_ES ISO-8859-1 +eu_ES@euro ISO-8859-15 +fa_IR UTF-8 +fi_FI ISO-8859-1 +fi_FI@euro ISO-8859-15 +fil_PH UTF-8 +fo_FO ISO-8859-1 +fr_BE ISO-8859-1 +fr_BE@euro ISO-8859-15 +fr_CA ISO-8859-1 +fr_CH ISO-8859-1 +fr_FR ISO-8859-1 +fr_FR@euro ISO-8859-15 +fr_LU ISO-8859-1 +fr_LU@euro ISO-8859-15 +fur_IT UTF-8 +fy_NL UTF-8 +fy_DE UTF-8 +ga_IE ISO-8859-1 +ga_IE@euro ISO-8859-15 +gd_GB ISO-8859-15 +gez_ER UTF-8 +gez_ER@abegede UTF-8 +gez_ET UTF-8 +gez_ET@abegede UTF-8 +gl_ES ISO-8859-1 +gl_ES@euro ISO-8859-15 +gu_IN UTF-8 +gv_GB ISO-8859-1 +ha_NG UTF-8 +he_IL ISO-8859-8 +hi_IN UTF-8 +hne_IN UTF-8 +hr_HR ISO-8859-2 +hsb_DE ISO-8859-2 +ht_HT UTF-8 +hu_HU ISO-8859-2 +hy_AM UTF-8 +id_ID ISO-8859-1 +ig_NG UTF-8 +ik_CA UTF-8 +is_IS ISO-8859-1 +it_CH ISO-8859-1 +it_IT ISO-8859-1 +it_IT@euro ISO-8859-15 +iu_CA UTF-8 +iw_IL ISO-8859-8 +ka_GE GEORGIAN-PS +kk_KZ PT154 +kk_KZ RK1048 +kl_GL ISO-8859-1 +km_KH UTF-8 +kn_IN UTF-8 +ks_IN UTF-8 +ks_IN@devanagari UTF-8 +ku_TR ISO-8859-9 +kw_GB ISO-8859-1 +ky_KG UTF-8 +lg_UG ISO-8859-10 +li_BE UTF-8 +li_NL UTF-8 +lo_LA UTF-8 +lt_LT ISO-8859-13 +lv_LV ISO-8859-13 +mai_IN UTF-8 +mg_MG ISO-8859-15 +mi_NZ ISO-8859-13 +mk_MK ISO-8859-5 +ml_IN UTF-8 +mn_MN UTF-8 +mr_IN UTF-8 +ms_MY ISO-8859-1 +mt_MT ISO-8859-3 +my_MM UTF-8 +nan_TW@latin UTF-8 +nb_NO ISO-8859-1 +nds_DE UTF-8 +nds_NL UTF-8 +ne_NP UTF-8 +nl_AW UTF-8 +nl_BE ISO-8859-1 +nl_BE@euro ISO-8859-15 +nl_NL ISO-8859-1 +nl_NL@euro ISO-8859-15 +nn_NO ISO-8859-1 +nr_ZA UTF-8 +nso_ZA UTF-8 +oc_FR ISO-8859-1 +om_ET UTF-8 +om_KE ISO-8859-1 +or_IN UTF-8 +pa_IN UTF-8 +pa_PK UTF-8 +pap_AN UTF-8 +pl_PL ISO-8859-2 +ps_AF UTF-8 +pt_BR ISO-8859-1 +pt_PT ISO-8859-1 +pt_PT@euro ISO-8859-15 +ro_RO ISO-8859-2 +ru_RU ISO-8859-5 +ru_UA KOI8-U +rw_RW UTF-8 +sa_IN UTF-8 +sc_IT UTF-8 +sd_IN UTF-8 +sd_IN@devanagari UTF-8 +se_NO UTF-8 +shs_CA UTF-8 +si_LK UTF-8 +sid_ET UTF-8 +sk_SK ISO-8859-2 +sl_SI ISO-8859-2 +so_DJ ISO-8859-1 +so_ET UTF-8 +so_KE ISO-8859-1 +so_SO ISO-8859-1 +sq_AL ISO-8859-1 +sr_ME UTF-8 +sr_RS UTF-8 +sr_RS@latin UTF-8 +ss_ZA UTF-8 +st_ZA ISO-8859-1 +sv_FI ISO-8859-1 +sv_FI@euro ISO-8859-15 +sv_SE ISO-8859-1 +ta_IN UTF-8 +te_IN UTF-8 +tg_TJ KOI8-T +th_TH TIS-620 +ti_ER UTF-8 +ti_ET UTF-8 +tig_ER UTF-8 +tk_TM UTF-8 +tl_PH ISO-8859-1 +tn_ZA UTF-8 +tr_CY ISO-8859-9 +tr_TR ISO-8859-9 +ts_ZA UTF-8 +ug_CN UTF-8 +uk_UA KOI8-U +ur_PK UTF-8 +uz_UZ ISO-8859-1 +uz_UZ@cyrillic UTF-8 +ve_ZA UTF-8 +vi_VN UTF-8 +wa_BE ISO-8859-1 +wa_BE@euro ISO-8859-15 +wo_SN UTF-8 +xh_ZA ISO-8859-1 +yi_US CP1255 +yo_NG UTF-8 +zh_CN GB2312 +zh_HK BIG5-HKSCS +zh_SG GB2312 +zh_TW BIG5 +zu_ZA ISO-8859-1
Signature · 1 line
1.7.7
PATCH 7/8 libc-package: rework 'precompiled' locale handling
In reply to Christopher LarsonThere were a couple problems with the handling of precompiled locales.
- it gathered the list of locales from the directories - this breaks due to the naming mismatch, e.g. en_US.UTF-8 vs en_US.utf8.
- it retained its hardcoded assumption that the non-suffixed locale (en_US, as opposed to en_US.*) is UTF-8, while the others are otherwise. Hardcoding this is both inflexible and just plain wrong for some toolchains. It's most common in desktop distros for 'en_US' to be non-utf8, and ''en_US.UTF-8' is utf8, and this is the case in some external toolchains as well.
The code now uses the SUPPORTED file to hold the knowledge it needs. This file not only holds the list of locales to generate, but also maps the locale names to the charsets they correspond to. The code now uses this to assemble its charset map, falling back to the '.' suffix as charset when the locale is not in the map. For precompiled, it now uses the locale->charset knowledge it has, thereby allowing non-utf8 non-suffixed locale names, whereas for non-precompiled, it reverts to the previous assumption, renaming the utf8 locale and forcibly suffixing the others.
So, a person maintaining an external toolchain recipe is responsible for ensuring that the SUPPORTED file they provide matches up with the compiled locales in the toolchain, if they want to utilize precompiled locales.
I believe in the long term the compiled case should do the same thing precompiled does, and use SUPPORTED or a similar mechanism to encode the knowledge, and if people want all the non-suffixed names to be utf8, they can change that file to do so. This would avoid the hardcoded assumption in the code, as well as consolidating the behavior between the compiled and precompiled cases.
--- meta/classes/libc-package.bbclass | 96 +++++++++++++++++------------------- 1 files changed, 45 insertions(+), 51 deletions(-)
meta/classes/libc-package.bbclass +45 -51
@@ -207,40 +207,30 @@ python package_do_split_gconvs () { dot_re = re.compile("(.*)\.(.*)") -#GLIBC_GENERATE_LOCALES var specifies which locales to be supported, empty or "all" means all locales - if use_bin != "precompiled": - supported = d.getVar('GLIBC_GENERATE_LOCALES', True) - if not supported or supported == "all": - f = open(base_path_join(d.getVar('WORKDIR', True), "SUPPORTED"), "r") - supported = f.readlines() - f.close() - else: - supported = supported.split() - supported = map(lambda s:s.replace(".", " ") + "\n", supported) - else: - supported = [] - full_bin_path = d.getVar('PKGD', True) + binary_locales_dir - for dir in os.listdir(full_bin_path): - dbase = dir.split(".") - d2 = " " - if len(dbase) > 1: - d2 = "." + dbase[1].upper() + " " - supported.append(dbase[0] + d2) + # Read in supported locales and associated encodings + supported = {} + with open(base_path_join(d.getVar('WORKDIR', True), "SUPPORTED")) as f: + for line in f.readlines(): + try: + locale, charset = line.rstrip().split() + except ValueError: + continue + supported[locale] = charset - # Collate the locales by base and encoding - utf8_only = int(d.getVar('LOCALE_UTF8_ONLY', True) or 0) - encodings = {} - for l in supported: - l = l[:-1] - (locale, charset) = l.split(" ") - if utf8_only and charset != 'UTF-8': - continue - m = dot_re.match(locale) - if m: - locale = m.group(1) - if not encodings.has_key(locale): - encodings[locale] = [] - encodings[locale].append(charset) + # GLIBC_GENERATE_LOCALES var specifies which locales to be generated. empty or "all" means all locales + to_generate = d.getVar('GLIBC_GENERATE_LOCALES', True) + if not to_generate or to_generate == 'all': + to_generate = supported.keys() + else: + to_generate = to_generate.split() + for locale in to_generate: + if locale not in supported: + if '.' in locale: + charset = locale.split('.')[1] + else: + charset = 'UTF-8' + bb.warn("Unsupported locale '%s', assuming encoding '%s'" % (locale, charset)) + supported[locale] = charset def output_locale_source(name, pkgname, locale, encoding): d.setVar('RDEPENDS_%s' % pkgname, 'localedef %s-localedata-%s %s-charmap-%s' % \
@@ -271,7 +261,7 @@ python package_do_split_gconvs () { use_cross_localedef = d.getVar("LOCALE_GENERATION_WITH_CROSS-LOCALEDEF", True) or "0" if use_cross_localedef == "1": - target_arch = d.getVar('TARGET_ARCH', True) + target_arch = d.getVar('TARGET_ARCH', True) locale_arch_options = { \ "arm": " --uint32-align=4 --little-endian ", \ "powerpc": " --uint32-align=4 --big-endian ", \
@@ -334,25 +324,29 @@ python package_do_split_gconvs () { bb.note("preparing tree for binary locale generation") bb.build.exec_func("do_prep_locale_tree", d) - # Reshuffle names so that UTF-8 is preferred over other encodings - non_utf8 = [] - for l in encodings.keys(): - if len(encodings[l]) == 1: - output_locale(l, l, encodings[l][0]) - if encodings[l][0] != "UTF-8": - non_utf8.append(l) + utf8_only = int(d.getVar('LOCALE_UTF8_ONLY', True) or 0) + encodings = {} + for locale in to_generate: + charset = supported[locale] + if utf8_only and charset != 'UTF-8': + continue + + m = dot_re.match(locale) + if m: + base = m.group(1) else: - if "UTF-8" in encodings[l]: - output_locale(l, l, "UTF-8") - encodings[l].remove("UTF-8") - else: - non_utf8.append(l) - for e in encodings[l]: - output_locale('%s.%s' % (l, e), l, e) + base = locale - if non_utf8 != [] and use_bin != "precompiled": - bb.note("the following locales are supported only in legacy encodings:") - bb.note(" " + " ".join(non_utf8)) + # Precompiled locales are kept as is, obeying SUPPORTED, while + # others are adjusted, ensuring that the non-suffixed locales + # are utf-8, while the suffixed are not. + if use_bin == "precompiled": + output_locale(locale, base, charset) + else: + if charset == 'UTF-8': + output_locale(base, base, charset) + else: + output_locale('%s.%s' % (base, charset), base, charset) if use_bin == "compile": makefile = base_path_join(d.getVar("WORKDIR", True), "locale-tree", "Makefile")
Signature · 1 line
1.7.7
PATCH 8/8 tcmode-external-csl: use the existing precompiled locale files
In reply to Christopher Larson--- meta/conf/distro/include/tcmode-external-csl.inc | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
meta/conf/distro/include/tcmode-external-csl.inc +4 -0
@@ -35,6 +35,10 @@ PREFERRED_PROVIDER_virtual/libiconv = "external-csl-toolchain" PREFERRED_PROVIDER_glibc-thread-db = "external-csl-toolchain" PREFERRED_PROVIDER_virtual/linux-libc-headers = "external-csl-toolchain" +# No need to re-compile the locale files +GLIBC_INTERNAL_USE_BINARY_LOCALE = "precompiled" +ENABLE_BINARY_LOCALE_GENERATION = "" + TARGET_CPPFLAGS_prepend = " -isystem${EXTERNAL_TOOLCHAIN}/${TARGET_SYS}/include " TARGET_LDFLAGS_prepend = " -L${EXTERNAL_TOOLCHAIN}/${TARGET_SYS}/lib -Wl,-rpath-link,${EXTERNAL_TOOLCHAIN}/${TARGET_SYS}/lib "
Signature · 1 line
1.7.7
Re: [PATCH 4/8] external-csl-toolchain: cleanup, simplify, use base_libdir
In reply to Christopher LarsonQuoted 58 lines · Christopher Larson
Signed-off-by Christopher Larson <[email protected]>---
meta/recipes-core/meta/external-csl-toolchain.bb | 55 +++++++++++-----------1 files changed, 27 insertions(+), 28 deletions(-)meta/recipes-core/meta/external-csl-toolchain.bb +0
-0@@ -24,36 +24,40 @@ PROVIDES += "\ virtual/linux-libc-headers \"
PV = "${CSL_VER_MAIN}"meta/recipes-core/meta/external-csl-toolchain.bb +1
-1-PR = "r3" +PR = "r5"SRC_URI = "file://SUPPORTED"
do_install() {
- install -d ${D}${sysconfdir} ${D}${bindir} ${D}${sbindir} ${D}${base_bindir} ${D}${libdir}
- install -d ${D}${base_libdir} ${D}${base_sbindir} ${D}${datadir} ${D}/usr
+ # Use optimized files if available
+ sysroot="${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc"
+ if [ -d $sysroot/${CSL_TARGET_CORE} ]; then
+ sysroot="$sysroot/${CSL_TARGET_CORE}"
+ fi
+
+ cp -a $sysroot${base_libdir}/. ${D}${base_libdir}
+ cp -a $sysroot/etc/. ${D}${sysconfdir}
+ cp -a $sysroot/sbin/. ${D}${base_sbindir}- if [ -d ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE} ]; then
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/lib/. ${D}${base_libdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/etc/. ${D}${sysconfdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/sbin/. ${D}${base_sbindir}
- if [ ! -e ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/include ]; then
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/include ${D}/usr/
+ install -d ${D}/usr
+ forusr_elementin bin libexec sbin share ${base_libdir}; do
+ usr_path=$sysroot/usr/$usr_element
+ cp -a $usr_path ${D}/usr/
+ done
+ fordatadir_elementin man info; do
+ datadir_path=$sysroot/usr/$datadir_element
+ if [ -e $datadir_path ]; then
+ cp -a $datadir_path ${D}${datadir}/
fi
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/. ${D}/usr/
- else
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/lib/. ${D}${base_libdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/etc/. ${D}${sysconfdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/sbin/. ${D}${base_sbindir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/. ${D}/usr/
- fi
+ done
Quoted 11 lines
- if [ -e ${D}${prefix}/info ]; then
- mv ${D}${prefix}/info ${D}${infodir}
- fi
- if [ -e ${D}${prefix}/man ]; then
- mv ${D}${prefix}/man ${D}${mandir}
+ # Some toolchains have headers under the core specific area
+ if [ -e $sysroot/usr/include ]; then
+ cp -a $sysroot/usr/include/. ${D}${includedir}
+ else
+ cp -a $sysroot/../usr/include/. ${D}${includedir} fi
can we avoid using /usr/ explicitly here
Quoted 31 lines
rm ${D}${sysconfdir}/rpc
meta/recipes-core/meta/external-csl-toolchain.bb +0
-0@@ -61,18 +65,13 @@ do_install() {mv ${D}${libdir}/bin/* ${D}${bindir}/
if [ -e ${D}${libdir}/bin/.debug ]; then
- install -d ${D}${bindir}/.debug
mv ${D}${libdir}/bin/.debug/* ${D}${bindir}/.debug/ fi
ln -s ../../bin/gdbserver ${D}${libdir}/bin/sysroot-gdbserversed -i -e 's/__packed/__attribute__ ((packed))/'
${D}${includedir}/mtd/ubi-user.h
- sed -i -e "s# /lib# ../../lib#g" -e "s# /usr/lib# .#g" ${D}${libdir}/libc.so
- sed -i -e "s# /lib# ../../lib#g" -e "s# /usr/lib# .#g" ${D}${libdir}/libpthread.someta/recipes-core/meta/external-csl-toolchain.bb +2
-4-} - -do_install_locale_append () { - rm -r ${D}${datadir}/locale ${D}${libdir}/locale + sed -i -e "s# ${base_libdir}# ../..${base_libdir}#g" -e "s# ${libdir}# .#g" ${D}${libdir}/libc.so + sed -i -e "s# ${base_libdir}# ../..${base_libdir}#g" -e "s# ${libdir}# .#g" ${D}${libdir}/libpthread.so}
SYSROOT_PREPROCESS_FUNCS += "
external_toolchain_sysroot_adjust" --
1.7.7_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [PATCH 8/8] tcmode-external-csl: use the existing precompiled locale files
In reply to Christopher Larsonmeta/conf/distro/include/tcmode-external-csl.inc PREFERRED_PROVIDER_virtual/libiconv = "external-csl-toolchain" · line 40
Quoted 3 lines · Christopher Larson
meta/conf/distro/include/tcmode-external-csl.inc +3
-0+# No need to re-compile the locale files +GLIBC_INTERNAL_USE_BINARY_LOCALE = "precompiled" +ENABLE_BINARY_LOCALE_GENERATION = ""
should this be 0
Re: [PATCH 8/8] tcmode-external-csl: use the existing precompiled locale files
Quoted 6 lines · Khem Raj
meta/conf/distro/include/tcmode-external-csl.inc PREFERRED_PROVIDER_virtual/libiconv = "external-csl-toolchain" · line 40
Quoted 3 lines · Christopher Larson
meta/conf/distro/include/tcmode-external-csl.inc +3
-0+# No need to re-compile the locale files +GLIBC_INTERNAL_USE_BINARY_LOCALE = "precompiled" +ENABLE_BINARY_LOCALE_GENERATION = ""should this be 0
Either 0 or the empty string have the same effect. I suppose I could change it to 0 for consistency, though.
Signature · 1 line
Christopher Larson
Re: [PATCH 4/8] external-csl-toolchain: cleanup, simplify, use base_libdir
In reply to Khem RajQuoted 59 lines · Khem Raj
Quoted 58 lines · Christopher Larson
Signed-off-by Christopher Larson <[email protected]>---
meta/recipes-core/meta/external-csl-toolchain.bb | 55 +++++++++++-----------1 files changed, 27 insertions(+), 28 deletions(-)meta/recipes-core/meta/external-csl-toolchain.bb +0
-0@@ -24,36 +24,40 @@ PROVIDES += "\ virtual/linux-libc-headers \"
PV = "${CSL_VER_MAIN}"meta/recipes-core/meta/external-csl-toolchain.bb +1
-1-PR = "r3" +PR = "r5"SRC_URI = "file://SUPPORTED"
do_install() {
- install -d ${D}${sysconfdir} ${D}${bindir} ${D}${sbindir} ${D}${base_bindir} ${D}${libdir}
- install -d ${D}${base_libdir} ${D}${base_sbindir} ${D}${datadir} ${D}/usr
+ # Use optimized files if available
+ sysroot="${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc"
+ if [ -d $sysroot/${CSL_TARGET_CORE} ]; then
+ sysroot="$sysroot/${CSL_TARGET_CORE}"
+ fi
+
+ cp -a $sysroot${base_libdir}/. ${D}${base_libdir}
+ cp -a $sysroot/etc/. ${D}${sysconfdir}
+ cp -a $sysroot/sbin/. ${D}${base_sbindir}- if [ -d ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE} ]; then
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/lib/. ${D}${base_libdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/etc/. ${D}${sysconfdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/sbin/. ${D}${base_sbindir}
- if [ ! -e ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/include ]; then
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/include ${D}/usr/
+ install -d ${D}/usr
+ forusr_elementin bin libexec sbin share ${base_libdir}; do
+ usr_path=$sysroot/usr/$usr_element
+ cp -a $usr_path ${D}/usr/
+ done
+ fordatadir_elementin man info; do
+ datadir_path=$sysroot/usr/$datadir_element
+ if [ -e $datadir_path ]; then
+ cp -a $datadir_path ${D}${datadir}/
fi
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/. ${D}/usr/
- else
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/lib/. ${D}${base_libdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/etc/. ${D}${sysconfdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/sbin/. ${D}${base_sbindir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/. ${D}/usr/
- fi
+ done
Quoted 13 lines
Quoted 11 lines
- if [ -e ${D}${prefix}/info ]; then
- mv ${D}${prefix}/info ${D}${infodir}
- fi
- if [ -e ${D}${prefix}/man ]; then
- mv ${D}${prefix}/man ${D}${mandir}
+ # Some toolchains have headers under the core specific area
+ if [ -e $sysroot/usr/include ]; then
+ cp -a $sysroot/usr/include/. ${D}${includedir}
+ else
+ cp -a $sysroot/../usr/include/. ${D}${includedir} fican we avoid using
/usr/explicitly here
If you change prefix, you *will* break things. The glibc in the external toolchain expects a particular layout. Changing that layout will break those expectations. I expect a better approach would be to use prefix and then add an explicit error if prefix is not /usr, but files copied will be identical either way.
Signature · 1 line
Christopher Larson
Re: [PATCH 4/8] external-csl-toolchain: cleanup, simplify, use base_libdir
Quoted 74 lines · Chris Larson
Quoted 59 lines · Khem Raj
Quoted 58 lines · Christopher Larson
Signed-off-by Christopher Larson <[email protected]>---
meta/recipes-core/meta/external-csl-toolchain.bb | 55 +++++++++++-----------1 files changed, 27 insertions(+), 28 deletions(-)meta/recipes-core/meta/external-csl-toolchain.bb +0
-0@@ -24,36 +24,40 @@ PROVIDES += "\ virtual/linux-libc-headers \"
PV = "${CSL_VER_MAIN}"meta/recipes-core/meta/external-csl-toolchain.bb +1
-1-PR = "r3" +PR = "r5"SRC_URI = "file://SUPPORTED"
do_install() {
- install -d ${D}${sysconfdir} ${D}${bindir} ${D}${sbindir} ${D}${base_bindir} ${D}${libdir}
- install -d ${D}${base_libdir} ${D}${base_sbindir} ${D}${datadir} ${D}/usr
+ # Use optimized files if available
+ sysroot="${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc"
+ if [ -d $sysroot/${CSL_TARGET_CORE} ]; then
+ sysroot="$sysroot/${CSL_TARGET_CORE}"
+ fi
+
+ cp -a $sysroot${base_libdir}/. ${D}${base_libdir}
+ cp -a $sysroot/etc/. ${D}${sysconfdir}
+ cp -a $sysroot/sbin/. ${D}${base_sbindir}- if [ -d ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE} ]; then
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/lib/. ${D}${base_libdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/etc/. ${D}${sysconfdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/sbin/. ${D}${base_sbindir}
- if [ ! -e ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/include ]; then
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/include ${D}/usr/
+ install -d ${D}/usr
+ forusr_elementin bin libexec sbin share ${base_libdir}; do
+ usr_path=$sysroot/usr/$usr_element
+ cp -a $usr_path ${D}/usr/
+ done
+ fordatadir_elementin man info; do
+ datadir_path=$sysroot/usr/$datadir_element
+ if [ -e $datadir_path ]; then
+ cp -a $datadir_path ${D}${datadir}/
fi
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/${CSL_TARGET_CORE}/usr/. ${D}/usr/
- else
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/lib/. ${D}${base_libdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/etc/. ${D}${sysconfdir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/sbin/. ${D}${base_sbindir}
- cp -a ${EXTERNAL_TOOLCHAIN}/${CSL_TARGET_SYS}/libc/usr/. ${D}/usr/
- fi
+ doneQuoted 13 lines
Quoted 11 lines
- if [ -e ${D}${prefix}/info ]; then
- mv ${D}${prefix}/info ${D}${infodir}
- fi
- if [ -e ${D}${prefix}/man ]; then
- mv ${D}${prefix}/man ${D}${mandir}
+ # Some toolchains have headers under the core specific area
+ if [ -e $sysroot/usr/include ]; then
+ cp -a $sysroot/usr/include/. ${D}${includedir}
+ else
+ cp -a $sysroot/../usr/include/. ${D}${includedir} fican we avoid using
/usr/explicitly here
Quoted 5 lines
If you change prefix, you *will* break things. The glibc in the external toolchain expects a particular layout. Changing that layout will break those expectations. I expect a better approach would be to use prefix and then add an explicit error if prefix is not /usr, but files copied will be identical either way.
I'll start work on another commit to apply after this series which does this.
Signature · 1 line
Christopher Larson
Saul Wold
AppliedQuoted 44 lines · Christopher Larson
See the commit message summaries for details, but this syncs up my local external-csl-toolchain recipe with upstream, which includes general cleanup and use of
base_libdirto fix issues with mips64, splits out libstdc++-staticdev, addsCSL_VER_MAINto target recipe signatures, and fixes and uses precompiled locales rather than regenerating them.Most of this is straightforward, but I would appreciate review in particular of the "libc-package: rework 'precompiled' locale handling". Thanks.
The following changes since commit
26d822ad059dd4839cc6fee2a65714e73cc146e1:nfs-utils: don't try to unload nfsd module (2012-05-01 14:28:47 +0100)are available in the git repository at:
git://github.com/kergoth/oe-coreexternal-csl-improvementsChristopher Larson (8):
csl-versions.inc: capture version in signatures
external-csl-toolchain: split out a libstdc++-staticdev package
external-csl-toolchain: silence .a/.debug QA warnings
external-csl-toolchain: cleanup, simplify, usebase_libdir
external-csl-toolchain: update SUPPORTED to match toolchain
tclibc-eglibc: drop hardcoded locale-base-en-us/en-gb
libc-package: rework ''precompiled' locale handling
tcmode-external-csl: use the existing precompiled locale filesmeta/classes/libc-package.bbclass | 96 ++++---- meta/conf/distro/include/csl-versions.inc | 4 + meta/conf/distro/include/tclibc-eglibc.inc | 12 +- meta/conf/distro/include/tcmode-external-csl.inc | 4 + meta/recipes-core/meta/external-csl-toolchain.bb | 68 +++--- .../meta/external-csl-toolchain/SUPPORTED | 257 +++++++++++++++++++- 6 files changed, 346 insertions(+), 95 deletions(-)Christopher Larson (8):
csl-versions.inc: capture version in signatures
external-csl-toolchain: split out a libstdc++-staticdev package
external-csl-toolchain: silence .a/.debug QA warnings
external-csl-toolchain: cleanup, simplify, usebase_libdir
external-csl-toolchain: update SUPPORTED to match toolchain
tclibc-eglibc: drop hardcoded locale-base-en-us/en-gb
libc-package: rework ''precompiled' locale handling
tcmode-external-csl: use the existing precompiled locale files
Quoted 8 lines
meta/classes/libc-package.bbclass | 96 ++++---- meta/conf/distro/include/csl-versions.inc | 4 + meta/conf/distro/include/tclibc-eglibc.inc | 12 +- meta/conf/distro/include/tcmode-external-csl.inc | 4 + meta/recipes-core/meta/external-csl-toolchain.bb | 68 +++--- .../meta/external-csl-toolchain/SUPPORTED | 257 +++++++++++++++++++- 6 files changed, 346 insertions(+), 95 deletions(-)
Merged into OE-Core
Thanks
Sau!