musl: support multilib
cover musl: support multilib
From: "mark.yang" <[email protected]>
This series makes musl work with multilib ([YOCTO #13122]) and reverts the multilib guard in musl.inc (4da308d37a).
What it fixes:
- gcc wrote the 64-bit musl loader as /usr/lib64/ld-musl-*.so.1, but musl installs it in nonarch_base_libdir. 64-bit programs could not start, and meson's sanity check failed.
- The multilib header wrapper used bits/wordsize.h, which glibc has and musl does not, so oe_multilib_header did not support musl. On musl the wrapper now checks which of bits/alltypes-32.h and -64.h the sysroot has, and uses the pointer size when it has both.
- Both variants installed /usr/bin/ldd as a link to their own libc.so, and rpm rejected them as a file conflict. ldd is now a script in an ldd package, as in glibc. It runs the loader with the same ELF class as the file.
Tested on qemuarm64 + lib32 armv7ahf and qemux86-64 + lib32 x86, with usrmerge:
- ipk: core-image-minimal with lib32 packages, and lib32-core-image-minimal, on both machines
- rpm: core-image-minimal with lib32 packages on qemuarm64
- ipk: core-image-minimal on qemuarm64 without multilib
In qemu, 64-bit and 32-bit programs ran side by side, and ldd picked the right loader for each. I also ran the commands from the oeqa ldd test by hand, and they passed in every image.
Still open: SWIG recipes that read wrapped headers need -D__MHWORDSIZE. python3-m2crypto (meta-python) is the one I found. I will send that fix to meta-python.
mark.yang (6):
gcc: point the musl dynamic linker at nonarch_base_libdir
musl: wrap the bits headers for multilib multilib_header_wrapper.h: support musl
multilib_header: wrap musl headers for multilib builds
musl: make ldd work with multilib
Revert "musl: Show error when used with multilibs"
meta/classes-recipe/cross-canadian.bbclass | 1 + meta/classes-recipe/cross.bbclass | 1 + meta/classes-recipe/crosssdk.bbclass | 1 + meta/classes-recipe/multilib_header.bbclass | 2 +- meta/recipes-core/musl/musl.inc | 6 --- meta/recipes-core/musl/musl/ldd | 18 +++++++++ meta/recipes-core/musl/musl_git.bb | 21 ++++++++-- meta/recipes-devtools/gcc/gcc-common.inc | 1 + .../gcc/gcc-configure-common.inc | 2 + .../gcc/gcc-multilib-config.inc | 20 ++++++---- ...AMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch | 39 ++++++++++--------- ...s-fix-v4bx-to-linker-to-support-EABI.patch | 2 +- scripts/multilib_header_wrapper.h | 22 +++++++++-- 13 files changed, 95 insertions(+), 41 deletions(-) create mode 100644 meta/recipes-core/musl/musl/ldd
PATCH 1/6 gcc: point the musl dynamic linker at nonarch_base_libdir
From: "mark.yang" <[email protected]>
Make gcc embed the musl loader path from nonarch_base_libdir instead of base_libdir.
In a multilib build the 64-bit variant got its own base_libdir (/usr/lib64), so its binaries asked for /usr/lib64/ld-musl-aarch64.so.1. The musl recipe installs every variant's loader in nonarch_base_libdir (/usr/lib here), so that file does not exist. With the musl.inc guard removed, meson's compiler sanity check fails under qemu-user (qemuarm64 + lib32, usrmerge):
qemu-aarch64: Could not open '/usr/lib64/ld-musl-aarch64.so.1': No such file or directory
Reference: https://errors.yoctoproject.org/Errors/Details/973525/
The musl wiki wants exactly /lib/ld-musl-$ARCH.so.1 in PT_INTERP, so that a binary runs on any musl system, and says the loader must not move to /lib64 or anywhere else. The loader name already carries the architecture, so all of them can share one directory.
nonarch_base_libdir keeps the no-lib64 rule, and without usrmerge it is the wiki path itself. With usrmerge it becomes /usr/lib, which non-multilib usrmerge builds already use.
Reference: https://wiki.musl-libc.org/guidelines-for-distributions
gcc-cross-aarch64 lib32-gcc-cross-arm before /usr/lib64/ld-musl-aarch64.so.1 /usr/lib/ld-musl-armhf.so.1 after /usr/lib/ld-musl-aarch64.so.1 unchanged
glibc builds keep their default loader path.
- 0007: MUSL_DYNAMIC_LINKER* use a new SYSTEMLIBS_NONARCH_DIR. The cross classes set target_nonarch_base_libdir for it, next to target_base_libdir.
- gcc-multilib-config: stop rewriting the musl loader per multilib, which 24c4e01892 ("gcc: also relocate the musl loader") added. It gave the x86-64 target and SDK gcc /lib64/ld-musl-x86_64.so.1. Only nativesdk-gcc keeps it, for the %r relocation marker.
--- meta/classes-recipe/cross-canadian.bbclass | 1 + meta/classes-recipe/cross.bbclass | 1 + meta/classes-recipe/crosssdk.bbclass | 1 + meta/recipes-devtools/gcc/gcc-common.inc | 1 + .../gcc/gcc-configure-common.inc | 2 + .../gcc/gcc-multilib-config.inc | 20 ++++++---- ...AMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch | 39 ++++++++++--------- ...s-fix-v4bx-to-linker-to-support-EABI.patch | 2 +- 8 files changed, 40 insertions(+), 27 deletions(-)
meta/classes-recipe/cross-canadian.bbclass +1 -0
@@ -145,6 +145,7 @@ target_base_prefix := "${root_prefix}" target_prefix := "${prefix}" target_exec_prefix := "${exec_prefix}" target_base_libdir = "${target_base_prefix}/${baselib}" +target_nonarch_base_libdir := "${nonarch_base_libdir}" target_libdir = "${target_exec_prefix}/${baselib}" target_includedir := "${includedir}"
meta/classes-recipe/cross.bbclass +1 -0
@@ -59,6 +59,7 @@ target_base_prefix := "${root_prefix}" target_prefix := "${prefix}" target_exec_prefix := "${exec_prefix}" target_base_libdir = "${target_base_prefix}/${baselib}" +target_nonarch_base_libdir := "${nonarch_base_libdir}" target_libdir = "${target_exec_prefix}/${baselib}" target_includedir := "${includedir}"
meta/classes-recipe/crosssdk.bbclass +1 -0
@@ -42,6 +42,7 @@ TARGET_FPU = "" target_libdir = "${SDKPATHNATIVE}${libdir_nativesdk}" target_includedir = "${SDKPATHNATIVE}${includedir_nativesdk}" target_base_libdir = "${SDKPATHNATIVE}${base_libdir_nativesdk}" +target_nonarch_base_libdir = "${SDKPATHNATIVE}${base_libdir_nativesdk}" target_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}" target_exec_prefix = "${SDKPATHNATIVE}${prefix_nativesdk}" baselib = "lib"
meta/recipes-devtools/gcc/gcc-common.inc +1 -0
@@ -111,6 +111,7 @@ B ?= "${WORKDIR}/gcc-${PV}/build.${HOST_SYS}.${TARGET_SYS}" target_includedir ?= "${includedir}" target_libdir ?= "${libdir}" target_base_libdir ?= "${base_libdir}" +target_nonarch_base_libdir ?= "${nonarch_base_libdir}" target_prefix ?= "${prefix}" # We need to ensure that for the shared work directory, the do_patch signatures match
meta/recipes-devtools/gcc/gcc-configure-common.inc +2 -0
@@ -72,6 +72,7 @@ CPPFLAGS = "" SYSTEMHEADERS = "${target_includedir}" SYSTEMLIBS = "${target_base_libdir}/" SYSTEMLIBS1 = "${target_libdir}/" +SYSTEMLIBS_NONARCH = "${target_nonarch_base_libdir}/" SYSTEMLOCALHEADERS = "${target_prefix}/local/include" do_configure:prepend () {
@@ -84,6 +85,7 @@ do_configure:prepend () { #define STANDARD_STARTFILE_PREFIX_1 "${SYSTEMLIBS}" #define STANDARD_STARTFILE_PREFIX_2 "${SYSTEMLIBS1}" #define SYSTEMLIBS_DIR "${SYSTEMLIBS}" +#define SYSTEMLIBS_NONARCH_DIR "${SYSTEMLIBS_NONARCH}" #define LOCAL_INCLUDE_DIR "${SYSTEMLOCALHEADERS}" #endif /* ! GCC_DEFAULTS_H */ _EOF
meta/recipes-devtools/gcc/gcc-multilib-config.inc +12 -8
@@ -123,15 +123,19 @@ python gcc_multilib_setup() { r'\1' + wrap_libdir(libdirx32) + r'\3'), (r'^(#define\s*UCLIBC_DYNAMIC_LINKER\b\s*)(\S+)(\s*\".*\")$', r'\1' + wrap_libdir(libdir32) + r'\3'), - (r'^(#define\s*MUSL_DYNAMIC_LINKER32\s*)(\S+)(\s*\".*\")$', - r'\1' + wrap_libdir(libdir32) + r'\3'), - (r'^(#define\s*MUSL_DYNAMIC_LINKER64\s*)(\S+)(\s*\".*\")$', - r'\1' + wrap_libdir(libdir64) + r'\3'), - (r'^(#define\s*MUSL_DYNAMIC_LINKERX32\s*)(\S+)(\s*\".*\")$', - r'\1' + wrap_libdir(libdirx32) + r'\3'), - (r'^(#define\s*MUSL_DYNAMIC_LINKER\b\s*)(\S+)(\s*\".*\")$', - r'\1' + wrap_libdir(libdir32) + r'\3'), ] + # musl loaders keep SYSTEMLIBS_NONARCH_DIR, nativesdk-gcc needs %r + if pn == "nativesdk-gcc": + substs += [ + (r'^(#define\s*MUSL_DYNAMIC_LINKER32\s*)(\S+)(\s*\".*\")$', + r'\1' + wrap_libdir(libdir32) + r'\3'), + (r'^(#define\s*MUSL_DYNAMIC_LINKER64\s*)(\S+)(\s*\".*\")$', + r'\1' + wrap_libdir(libdir64) + r'\3'), + (r'^(#define\s*MUSL_DYNAMIC_LINKERX32\s*)(\S+)(\s*\".*\")$', + r'\1' + wrap_libdir(libdirx32) + r'\3'), + (r'^(#define\s*MUSL_DYNAMIC_LINKER\b\s*)(\S+)(\s*\".*\")$', + r'\1' + wrap_libdir(libdir32) + r'\3'), + ] for (i, line) in enumerate(filelines): for subst in substs:
meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch +21 -18
@@ -12,8 +12,11 @@ SH, sparc, alpha for possible future support (if any) Removes the do_headerfix task in metadata +MUSL_DYNAMIC_LINKER uses SYSTEMLIBS_NONARCH_DIR instead. + Upstream-Status: Inappropriate [OE configuration] Signed-off-by: Khem Raj <[email protected]> +Signed-off-by: mark.yang <[email protected]> --- gcc/config/aarch64/aarch64-linux.h | 4 ++-- gcc/config/alpha/linux-elf.h | 4 ++--
@@ -47,7 +50,7 @@ index 116bb4e69f3..c0de4900dc2 100644 #undef MUSL_DYNAMIC_LINKER -#define MUSL_DYNAMIC_LINKER "/lib/ld-musl-aarch64%{mbig-endian:_be}%{mabi=ilp32:_ilp32}.so.1" -+#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_DIR "ld-musl-aarch64%{mbig-endian:_be}%{mabi=ilp32:_ilp32}.so.1" ++#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_NONARCH_DIR "ld-musl-aarch64%{mbig-endian:_be}%{mabi=ilp32:_ilp32}.so.1" #undef ASAN_CC1_SPEC #define ASAN_CC1_SPEC "%{%:sanitize(address):-funwind-tables}"
@@ -86,7 +89,7 @@ index 8ef5643b5d6..76893197773 100644 #endif #define MUSL_DYNAMIC_LINKER \ - "/lib/ld-musl-arm" MUSL_DYNAMIC_LINKER_E "%{mfloat-abi=hard:hf}%{mfdpic:-fdpic}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-arm" MUSL_DYNAMIC_LINKER_E "%{mfloat-abi=hard:hf}%{mfdpic:-fdpic}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-arm" MUSL_DYNAMIC_LINKER_E "%{mfloat-abi=hard:hf}%{mfdpic:-fdpic}.so.1" /* At this point, bpabi.h will have clobbered LINK_SPEC. We want to use the GNU/Linux version, not the generic BPABI version. */
@@ -116,7 +119,7 @@ index efbd978c610..9ea0261b0fc 100644 #undef MUSL_DYNAMIC_LINKER -#define MUSL_DYNAMIC_LINKER "/lib/ld-musl-i386.so.1" -+#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_DIR "ld-musl-i386.so.1" ++#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_NONARCH_DIR "ld-musl-i386.so.1" diff --git a/gcc/config/i386/linux64.h b/gcc/config/i386/linux64.h index 24170ee2845..733e3e4945a 100644 --- a/gcc/config/i386/linux64.h
@@ -134,13 +137,13 @@ index 24170ee2845..733e3e4945a 100644 #undef MUSL_DYNAMIC_LINKER32 -#define MUSL_DYNAMIC_LINKER32 "/lib/ld-musl-i386.so.1" -+#define MUSL_DYNAMIC_LINKER32 SYSTEMLIBS_DIR "ld-musl-i386.so.1" ++#define MUSL_DYNAMIC_LINKER32 SYSTEMLIBS_NONARCH_DIR "ld-musl-i386.so.1" #undef MUSL_DYNAMIC_LINKER64 -#define MUSL_DYNAMIC_LINKER64 "/lib/ld-musl-x86_64.so.1" -+#define MUSL_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-musl-x86_64.so.1" ++#define MUSL_DYNAMIC_LINKER64 SYSTEMLIBS_NONARCH_DIR "ld-musl-x86_64.so.1" #undef MUSL_DYNAMIC_LINKERX32 -#define MUSL_DYNAMIC_LINKERX32 "/lib/ld-musl-x32.so.1" -+#define MUSL_DYNAMIC_LINKERX32 SYSTEMLIBS_DIR "ld-musl-x32.so.1" ++#define MUSL_DYNAMIC_LINKERX32 SYSTEMLIBS_NONARCH_DIR "ld-musl-x32.so.1" diff --git a/gcc/config/linux.h b/gcc/config/linux.h index d6280a4d4dd..c79d4902f85 100644 --- a/gcc/config/linux.h
@@ -178,7 +181,7 @@ index fbc75a90ad5..38c239940bd 100644 #undef MUSL_DYNAMIC_LINKER #define MUSL_DYNAMIC_LINKER \ - "/lib/ld-musl-loongarch" ABI_GRLEN_SPEC MUSL_ABI_SPEC ".so.1" -+ SYSTEMLIBS_DIR "ld-musl-loongarch" ABI_GRLEN_SPEC MUSL_ABI_SPEC ".so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-loongarch" ABI_GRLEN_SPEC MUSL_ABI_SPEC ".so.1" #undef GNU_USER_TARGET_LINK_SPEC #define GNU_USER_TARGET_LINK_SPEC \
@@ -200,7 +203,7 @@ index 7b09cde7693..8bfafaf87f7 100644 #undef MUSL_DYNAMIC_LINKER -#define MUSL_DYNAMIC_LINKER "/lib/ld-musl-microblaze" MUSL_DYNAMIC_LINKER_E ".so.1" -+#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_DIR "ld-musl-microblaze" MUSL_DYNAMIC_LINKER_E ".so.1" ++#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_NONARCH_DIR "ld-musl-microblaze" MUSL_DYNAMIC_LINKER_E ".so.1" #undef SUBTARGET_EXTRA_SPECS #define SUBTARGET_EXTRA_SPECS \
@@ -236,14 +239,14 @@ index ae15d490108..0281650009d 100644 #undef MUSL_DYNAMIC_LINKER32 #define MUSL_DYNAMIC_LINKER32 \ - "/lib/ld-musl-mips%{mips32r6|mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-mips%{mips32r6|mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-mips%{mips32r6|mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" #undef MUSL_DYNAMIC_LINKER64 #define MUSL_DYNAMIC_LINKER64 \ - "/lib/ld-musl-mips64%{mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-mips64%{mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-mips64%{mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" #define MUSL_DYNAMIC_LINKERN32 \ - "/lib/ld-musl-mipsn32%{mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-mipsn32%{mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-mipsn32%{mips64r6:r6}%{EL:el}%{msoft-float:-sf}.so.1" #define BIONIC_DYNAMIC_LINKERN32 "/system/bin/linker32" #define GNU_USER_DYNAMIC_LINKERN32 \
@@ -265,7 +268,7 @@ index 9060c940a44..c7712db194a 100644 #undef MUSL_DYNAMIC_LINKER -#define MUSL_DYNAMIC_LINKER "/lib/ld-musl-riscv" XLEN_SPEC MUSL_ABI_SUFFIX ".so.1" -+#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_DIR "ld-musl-riscv" XLEN_SPEC MUSL_ABI_SUFFIX ".so.1" ++#define MUSL_DYNAMIC_LINKER SYSTEMLIBS_NONARCH_DIR "ld-musl-riscv" XLEN_SPEC MUSL_ABI_SUFFIX ".so.1" #define ICACHE_FLUSH_FUNC "__riscv_flush_icache"
@@ -295,11 +298,11 @@ index 0316d8cb65d..8de5eb8bb49 100644 #undef MUSL_DYNAMIC_LINKER32 #define MUSL_DYNAMIC_LINKER32 \ - "/lib/ld-musl-powerpc" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-powerpc" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-powerpc" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" #undef MUSL_DYNAMIC_LINKER64 #define MUSL_DYNAMIC_LINKER64 \ - "/lib/ld-musl-powerpc64" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-powerpc64" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-powerpc64" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" #undef DEFAULT_ASM_ENDIAN #if (TARGET_DEFAULT & MASK_LITTLE_ENDIAN)
@@ -316,7 +319,7 @@ index afb5c6ad867..3eb6c814ced 100644 #undef MUSL_DYNAMIC_LINKER #define MUSL_DYNAMIC_LINKER \ - "/lib/ld-musl-powerpc" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" -+ SYSTEMLIBS_DIR "ld-musl-powerpc" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-powerpc" MUSL_DYNAMIC_LINKER_E "%{msoft-float:-sf}.so.1" #ifndef GNU_USER_DYNAMIC_LINKER #define GNU_USER_DYNAMIC_LINKER GLIBC_DYNAMIC_LINKER
@@ -335,10 +338,10 @@ index 2c3bca5dfcb..1332624abad 100644 #undef MUSL_DYNAMIC_LINKER32 -#define MUSL_DYNAMIC_LINKER32 "/lib/ld-musl-s390.so.1" -+#define MUSL_DYNAMIC_LINKER32 SYSTEMLIBS_DIR "ld-musl-s390.so.1" ++#define MUSL_DYNAMIC_LINKER32 SYSTEMLIBS_NONARCH_DIR "ld-musl-s390.so.1" #undef MUSL_DYNAMIC_LINKER64 -#define MUSL_DYNAMIC_LINKER64 "/lib/ld-musl-s390x.so.1" -+#define MUSL_DYNAMIC_LINKER64 SYSTEMLIBS_DIR "ld-musl-s390x.so.1" ++#define MUSL_DYNAMIC_LINKER64 SYSTEMLIBS_NONARCH_DIR "ld-musl-s390x.so.1" #undef LINK_SPEC #define LINK_SPEC \
@@ -351,7 +354,7 @@ index 78a7905a919..28493851982 100644 #undef MUSL_DYNAMIC_LINKER #define MUSL_DYNAMIC_LINKER \ - "/lib/ld-musl-sh" MUSL_DYNAMIC_LINKER_E MUSL_DYNAMIC_LINKER_FP \ -+ SYSTEMLIBS_DIR "ld-musl-sh" MUSL_DYNAMIC_LINKER_E MUSL_DYNAMIC_LINKER_FP \ ++ SYSTEMLIBS_NONARCH_DIR "ld-musl-sh" MUSL_DYNAMIC_LINKER_E MUSL_DYNAMIC_LINKER_FP \ "%{mfdpic:-fdpic}.so.1" -#define GLIBC_DYNAMIC_LINKER "/lib/ld-linux.so.2"
meta/recipes-devtools/gcc/gcc/0009-gcc-armv4-pass-fix-v4bx-to-linker-to-support-EABI.patch +1 -1
@@ -23,7 +23,7 @@ index 76893197773..12c334463c6 100644 +++ b/gcc/config/arm/linux-eabi.h @@ -91,10 +91,14 @@ #define MUSL_DYNAMIC_LINKER \ - SYSTEMLIBS_DIR "ld-musl-arm" MUSL_DYNAMIC_LINKER_E "%{mfloat-abi=hard:hf}%{mfdpic:-fdpic}.so.1" + SYSTEMLIBS_NONARCH_DIR "ld-musl-arm" MUSL_DYNAMIC_LINKER_E "%{mfloat-abi=hard:hf}%{mfdpic:-fdpic}.so.1" +/* For armv4 we pass --fix-v4bx to linker to support EABI */ +#undef TARGET_FIX_V4BX_SPEC
PATCH 2/6 musl: wrap the bits headers for multilib
In reply to [email protected]From: "mark.yang" <[email protected]>
Pass every installed bits/*.h through oe_multilib_header, as glibc-package.inc does for the glibc ones.
Unlike glibc, musl keeps all of its arch headers in bits/. Which of them differ depends on the arch pair and the musl release, so all of them are wrapped instead of keeping a list.
--- meta/recipes-core/musl/musl_git.bb | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
meta/recipes-core/musl/musl_git.bb +8 -1
@@ -2,7 +2,7 @@ # Released under the MIT license (see COPYING.MIT for the terms) require musl.inc -inherit linuxloader +inherit linuxloader multilib_header SRCREV = "c4e1bb3994c14ed5112c894d15a451bf00f0d501"
@@ -69,6 +69,13 @@ do_install() { echo "${libdir}" >> ${D}${sysconfdir}/ld-musl-${MUSL_LDSO_ARCH}.path rm -f ${D}${bindir}/ldd ${D}${GLIBC_LDSO} ln -rs ${D}${libdir}/libc.so ${D}${bindir}/ldd + musl_multilib_bits +} + +musl_multilib_bits () { + for h in ${D}${includedir}/bits/*.h; do + oe_multilib_header bits/${h##*/} + done } FILES:${PN} += "${nonarch_base_libdir}/ld-musl-${MUSL_LDSO_ARCH}.so.1 ${sysconfdir}/ld-musl-${MUSL_LDSO_ARCH}.path"
PATCH 3/6 multilib_header_wrapper.h: support musl
In reply to [email protected]From: "mark.yang" <[email protected]>
Pick the -32/-64 variant on musl from the musl headers in the sysroot, or from __SIZEOF_POINTER__ when both variants are there. glibc multilib is unchanged.
This brings back __MHWORDSIZE, dropped in 3a6ba47ede. musl defines __WORDSIZE in sys/reg.h and sys/user.h, and a wrapper included after them would redefine it. The first wrapper in a translation unit sets it as follows:
1. Already defined: used as is. SWIG has no __has_include, so musl recipes that run it over wrapped headers pass -D__MHWORDSIZE=<bits>.
2. Only bits/alltypes-32.h or only bits/alltypes-64.h: 32 or 64. This is a recipe sysroot, which has one musl variant.
3. bits/alltypes.h with both variants: __SIZEOF_POINTER__ * 8. This is an SDK sysroot.
4. Otherwise: __WORDSIZE from bits/wordsize.h, as before.
musl installs bits/alltypes-32.h or -64.h now that its bits headers are wrapped.
bpf needs no __bpf__ check. 6d8c7e3426 removed the old one, which set 64. A lib32 recipe that builds eBPF with clang -target bpf gets 8-byte pointers, so the pointer size alone would pick the -64.h headers. Its sysroot has only the -32.h ones, and rule 2 picks those.
--- scripts/multilib_header_wrapper.h | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
scripts/multilib_header_wrapper.h +18 -4
@@ -5,9 +5,23 @@* */ -#include <bits/wordsize.h> +#ifndef __MHWORDSIZE +# if defined(__has_include) +# if __has_include(<bits/alltypes-32.h>) && !__has_include(<bits/alltypes-64.h>) +# define __MHWORDSIZE 32 /* musl, 32-bit only */ +# elif __has_include(<bits/alltypes-64.h>) && !__has_include(<bits/alltypes-32.h>) +# define __MHWORDSIZE 64 /* musl, 64-bit only */ +# elif __has_include(<bits/alltypes.h>) +# define __MHWORDSIZE (__SIZEOF_POINTER__ * 8) /* musl, both (SDK) */ +# endif +# endif +# ifndef __MHWORDSIZE +# include <bits/wordsize.h> /* glibc */ +# define __MHWORDSIZE __WORDSIZE +# endif +#endif -#if __WORDSIZE ==32 +#if __MHWORDSIZE ==32 #ifdef _MIPS_SIM
@@ -23,9 +37,9 @@#include <ENTER_HEADER_FILENAME_HERE-32.h> #endif -#elif __WORDSIZE ==64 +#elif __MHWORDSIZE ==64 #include <ENTER_HEADER_FILENAME_HERE-64.h> #else #error "Unknown __WORDSIZE detected" -#endif /* matches #if __WORDSIZE == 32 */ +#endif /* matches #if __MHWORDSIZE == 32 */
PATCH 4/6 multilib_header: wrap musl headers for multilib builds
In reply to [email protected]From: "mark.yang" <[email protected]>
Let oe_multilib_header wrap headers on musl when multilib is enabled. dad1c27463 skipped musl because it was not multilib, and because the wrapper needed bits/wordsize.h. The wrapper no longer needs it.
Non-multilib musl builds keep their plain headers. Only multilib puts two variants of a header into one sysroot.
With the musl.inc multilib guard removed, a qemuarm64 + lib32 SDK with both variants silently replaced 20 asm/*.h headers from linux-libc-headers and 17 musl bits headers by the lib32 copies. Both recipes already pass them to oe_multilib_header.
--- meta/classes-recipe/multilib_header.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/multilib_header.bbclass +1 -1
@@ -15,7 +15,7 @@ oe_multilib_header() { case ${HOST_OS} in *-musl*) - return + [ -n "${MULTILIB_VARIANTS}" ] || return 0 ;; *) esac
PATCH 5/6 musl: make ldd work with multilib
In reply to [email protected]From: "mark.yang" <[email protected]>
Replace the /usr/bin/ldd link to libc.so with a script, and ship it in its own ldd package.
The script and its packaging follow glibc:
- RTLDLIST holds the loader of every variant, built with oe.utils.all_multilib_tune_values() as in glibc-ld.inc. do_install substitutes it into the script, so both variants ship the same file.
- ldd uses the first installed loader in RTLDLIST with the same ELF class as the file.
- The package is named ldd, as in glibc-package.inc. RDEPENDS on ldd need no RPROVIDES. The oeqa runtime ldd test checks for this package name, so it now runs on musl.
glibc's ldd finds that loader by running each one with --verify. In open_verify(), the loader compares e_ident[EI_CLASS] of the file with its own class and exits with 1 when they differ.
The musl loader has no --verify. Without a check in the script, the user has to run the loader that matches each program by hand:
/lib/ld-musl-aarch64.so.1 --list app64 /lib/ld-musl-armhf.so.1 --list app32
In a mixed image, /usr/bin holds programs of both classes, and the file name does not show which.
The script therefore makes the same comparison as glibc. It reads byte 4 (EI_CLASS) of the file and of each loader with dd:
dd if=app64 bs=1 skip=4 count=1 -> 2 (ELFCLASS64) dd if=app32 bs=1 skip=4 count=1 -> 1 (ELFCLASS32) dd if=/lib/ld-musl-aarch64.so.1 bs=1 skip=4 count=1 -> 2 (ELFCLASS64) dd if=/lib/ld-musl-armhf.so.1 bs=1 skip=4 count=1 -> 1 (ELFCLASS32)
ldd then picks the aarch64 loader for app64 and the armhf loader for app32. This gives the two commands above.
A loader's own header holds the class it was built for. glibc uses the same value as the constant ELFW(CLASS).
The command follows Alpine. Its musl APKBUILD writes ldd as 'exec /lib/$LDSO --list "$@"'.
musl upstream also recommends this form over the argv[0] link.
Reference: https://www.openwall.com/lists/musl/2023/04/17/6
When no loader matches, as for a non-ELF file, the first installed loader in RTLDLIST reports the error.
musl no longer provides ldd. As with glibc, an image gets it only when something installs or depends on it.
--- meta/recipes-core/musl/musl/ldd | 18 ++++++++++++++++++ meta/recipes-core/musl/musl_git.bb | 12 ++++++++++-- 2 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-core/musl/musl/ldd
meta/recipes-core/musl/musl/ldd new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/sh + +RTLDLIST="@RTLDLIST@" + +elfclass() { + dd if="$1" bs=1 skip=4 count=1 2>/dev/null +} + +class=$(elfclass "$1") +first= +for rtld in $RTLDLIST; do + [ -x "$rtld" ] || continue + [ -n "$first" ] || first=$rtld + if [ "$(elfclass "$rtld")" = "$class" ]; then + exec "$rtld" --list "$@" + fi +done +exec "$first" --list "$@"
meta/recipes-core/musl/musl_git.bb +10 -2
@@ -15,6 +15,7 @@ SRC_URI = "git://git.musl-libc.org/musl;branch=master \ file://0002-ldso-Use-syslibdir-and-libdir-as-default-pathes-to-l.patch \ file://0003-tools-version.sh-fall-back-to-VERSION-when-git-descr.patch \ file://0004-configure-use-ffp-exception-behavior-maytrap-when-su.patch \ + file://ldd \ " PROVIDES += "virtual/libc virtual/libiconv virtual/libintl virtual/crypt"
@@ -28,6 +29,9 @@ DEPENDS = "virtual/cross-binutils \ " GLIBC_LDSO = "${@get_glibc_loader(d)}" MUSL_LDSO_ARCH = "${@get_musl_loader_arch(d)}" +MUSL_GETLOADER = "${@get_musl_loader(d)}" +RTLDLIST = "${@oe.utils.all_multilib_tune_values(d, 'MUSL_GETLOADER')}" +RTLDLIST[vardepvalue] = "${RTLDLIST}" export CROSS_COMPILE = "${TARGET_PREFIX}"
@@ -68,7 +72,8 @@ do_install() { echo "${base_libdir}" > ${D}${sysconfdir}/ld-musl-${MUSL_LDSO_ARCH}.path echo "${libdir}" >> ${D}${sysconfdir}/ld-musl-${MUSL_LDSO_ARCH}.path rm -f ${D}${bindir}/ldd ${D}${GLIBC_LDSO} - ln -rs ${D}${libdir}/libc.so ${D}${bindir}/ldd + install -m 0755 ${UNPACKDIR}/ldd ${D}${bindir}/ldd + sed -i -e 's#@RTLDLIST@#${RTLDLIST}#' ${D}${bindir}/ldd musl_multilib_bits }
@@ -78,6 +83,9 @@ musl_multilib_bits () { done } +PACKAGES =+ "ldd" + +FILES:ldd = "${bindir}/ldd" FILES:${PN} += "${nonarch_base_libdir}/ld-musl-${MUSL_LDSO_ARCH}.so.1 ${sysconfdir}/ld-musl-${MUSL_LDSO_ARCH}.path" FILES:${PN}-staticdev = "${libdir}/libc.a" FILES:${PN}-dev =+ "${libdir}/libcrypt.a ${libdir}/libdl.a ${libdir}/libm.a \
@@ -87,7 +95,7 @@ FILES:${PN}-dev =+ "${libdir}/libcrypt.a ${libdir}/libdl.a ${libdir}/libm.a \ RDEPENDS:${PN}-dev += "linux-libc-headers-dev bsd-headers-dev libssp-nonshared-staticdev" RPROVIDES:${PN}-dev += "libc-dev virtual-libc-dev" -RPROVIDES:${PN} += "ldd rtld(GNU_HASH)" +RPROVIDES:${PN} += "rtld(GNU_HASH)" LEAD_SONAME = "libc.so" INSANE_SKIP:${PN}-dev = "staticdev"
PATCH 6/6 Revert "musl: Show error when used with multilibs"
In reply to [email protected]From: "mark.yang" <[email protected]>
This reverts commit 4da308d37aa51231b617fedb401076c13cafd89a.
musl now works with multilib. The loader search path follows base_libdir (df98368e23), and the earlier patches in this series handle the rest:
- gcc points the musl dynamic linker at nonarch_base_libdir.
- The musl bits headers get multilib wrappers.
- ldd picks the loader that matches the ELF class of the file.
With these, lib32 programs built against musl install and run next to the 64-bit ones in the same image.
[YOCTO #13122]
--- meta/recipes-core/musl/musl.inc | 6 ------ 1 file changed, 6 deletions(-)
meta/recipes-core/musl/musl.inc +0 -6
@@ -29,9 +29,3 @@ ARM_INSTRUCTION_SET:armv4 = "arm" B = "${WORKDIR}/build" do_configure[cleandirs] = "${B}" - -python () { - multilibs = d.getVar("MULTILIBS") - if multilibs: - raise bb.parse.SkipRecipe("musl is not compatible with multilibs") -}