Skip to contentSkip to conversation

rust: Use llvm-config from native llvm-rust on rust target builds

PATCH In review Khem Raj 2 participants 5 messages Jan 14, 2024 – Jan 15, 2024

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 rust: Use llvm-config from native llvm-rust on rust target builds

target rust recipe builds ( cross compile ) calls llvm-config from target sysroot which works ok as long as C++ runtime it needs is available on build host e.g. libstdc++ etc. which is commonly the case, however when using clang and llvm runtime this falters since it should be using libc++ from native sysroot and if this does not exist on build machine this fails to find libc++ shared object and llvm-config fails to run. This ensures that llvm-config version in use is correctly relocated and can use shared libraries from native sysroot correctly. Adding ORIGIN to sysroot will look for the .so in same dir as the binary and there is the libc++.so.1 copied in place

Fixes rust build with clang compiler.

| /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or director y

| thread 'main' panicked at llvm.rs:551:19:
| command did not execute successfully: "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config" "--version"
| expected success, got: exit status: 127
---
 meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
meta/recipes-devtools/rust/rust_1.74.1.bb +6 -2
index 30543ada7db..2dffe009827 100644
@@ -198,9 +198,13 @@ rust_runx () {
     # Copy the natively built llvm-config into the target so we can run it. Horrible,
     # but works!
     if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} != ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
-        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
+        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
+        mkdir -p ${tgtdir}
         cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
-        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
+        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
+            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
+        fi
+        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
     fi
 
     oe_cargo_fix_env
Signature · 1 line
2.43.0
#1 lore raw
Quoted 34 lines · Khem Raj

target rust recipe builds ( cross compile ) calls llvm-config from target sysroot which works ok as long as C++ runtime it needs is available on build host e.g. libstdc++ etc. which is commonly the case, however when using clang and llvm runtime this falters since it should be using libc++ from native sysroot and if this does not exist on build machine this fails to find libc++ shared object and llvm-config fails to run. This ensures that llvm-config version in use is correctly relocated and can use shared libraries from native sysroot correctly. Adding ORIGIN to sysroot will look for the .so in same dir as the binary and there is the libc++.so.1 copied in place

Fixes rust build with clang compiler.

Quoted 1 line

/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or director

y

Quoted 3 lines

thread 'main' panicked at llvm.rs:551:19:
command did not execute successfully: "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config" "--version" expected success, got: exit status: 127

---
 meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
meta/recipes-devtools/rust/rust_1.74.1.bb +1 -1
index 30543ada7db..2dffe009827 100644
@@ -198,9 +198,13 @@ rust_runx () {
     # Copy the natively built llvm-config into the target so we can run it. Horrible,
     # but works!
     if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} != ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
-        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
+        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`

meta/recipes-devtools/rust/rust_1.74.1.bb rust_runx () { · line 210

Quoted 10 lines
meta/recipes-devtools/rust/rust_1.74.1.bb +5 -1
+        mkdir -p ${tgtdir}
         cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
-        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
+        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
+            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
+        fi
+        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
     fi
 
     oe_cargo_fix_env

Copying a native library into the target sysroot goes beyond what I'm comfortable with even for this horrible hack with llvm-config.

Since it seems to be finding it by RPATH, can you not just add an RPATH to the native sysroot in the binary?

Cheers,

Richard

author
#2 lore raw

Hi

Quoted 52 lines · Richard Purdie
Quoted 11 lines · Khem Raj

target rust recipe builds ( cross compile ) calls llvm-config from target sysroot which works ok as long as C++ runtime it needs is available on build host e.g. libstdc++ etc. which is commonly the case, however when using clang and llvm runtime this falters since it should be using libc++ from native sysroot and if this does not exist on build machine this fails to find libc++ shared object and llvm-config fails to run. This ensures that llvm-config version in use is correctly relocated and can use shared libraries from native sysroot correctly. Adding ORIGIN to sysroot will look for the .so in same dir as the binary and there is the libc++.so.1 copied in place

Quoted 3 lines

Fixes rust build with clang compiler.

Quoted 1 line

/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config:
error while loading shared libraries: libc++.so.1: cannot open shared
object file: No such file or director

Quoted 3 lines

y

Quoted 2 lines

thread 'main' panicked at llvm.rs:551:19:
command did not execute successfully:

"/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config" "--version"

Review

Quoted 8 lines
Quoted 1 line

expected success, got: exit status: 127

---
 meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/meta/recipes-devtools/rust/rust_1.74.1.bb

meta/recipes-devtools/rust/rust_1.74.1.bb rust_runx () { · line 198

Quoted 5 lines

index 30543ada7db..2dffe009827 100644

meta/recipes-devtools/rust/rust_1.74.1.bb +0 -0
@@ -198,9 +198,13 @@ rust_runx () {
     # Copy the natively built llvm-config into the target so we can run

it. Horrible,

meta/recipes-devtools/rust/rust_1.74.1.bb rust_runx () { · line 200

Quoted 2 lines
meta/recipes-devtools/rust/rust_1.74.1.bb +0 -0
     # but works!
     if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=

${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then

Quoted 11 lines
meta/recipes-devtools/rust/rust_1.74.1.bb +6 -2
-        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
+        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
+        mkdir -p ${tgtdir}
         cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
-        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
+        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
+            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
+        fi
+        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
     fi
 
Quoted 4 lines

meta/recipes-devtools/rust/rust_1.74.1.bb rust_runx () { · line 210

Quoted 1 line
meta/recipes-devtools/rust/rust_1.74.1.bb +0 -0
     oe_cargo_fix_env

Copying a native library into the target sysroot goes beyond what I'm comfortable with even for this horrible hack with llvm-config.

It’s just supporting the original hack to work properly I don’t think it’s any worse that the original hack

Quoted 4 lines

Since it seems to be finding it by RPATH, can you not just add an RPATH to the native sysroot in the binary?

It does not work and I don’t know why but seems rpath in llvm-config affects what paths it spits out

Quoted 8 lines

Cheers,

Richard

#3 lore raw
Quoted 75 lines · Khem Raj

Hi

Quoted 60 lines · Richard Purdie
Quoted 59 lines · Khem Raj

target rust recipe builds ( cross compile ) calls llvm-config from
target sysroot which works ok as long as C++ runtime it needs is available on build host e.g. libstdc++ etc. which is commonly the case, however when using clang and llvm runtime this falters since
it should be using libc++ from native sysroot and if this does not
exist on build machine this fails to find libc++ shared object and
llvm-config fails to run. This ensures that llvm-config version in
use is correctly relocated and can use shared libraries from native
sysroot correctly. Adding ORIGIN to sysroot will look for the .so in
same dir as the binary and there is the libc++.so.1 copied in place

Fixes rust build with clang compiler.

Quoted 4 lines

/mnt/b/yoe/master/build/tmp/work/riscv64-yoe- linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-
config: error while loading shared libraries: libc++.so.1:
cannot open shared object file: No such file or director

y

Quoted 6 lines

thread 'main' panicked at llvm.rs:551:19:
command did not execute successfully:
"/mnt/b/yoe/master/build/tmp/work/riscv64-yoe- linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm- config" "--version"
expected success, got: exit status: 127

---
  meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--

  1 file changed, 6 insertions(+), 2 deletions(-)
meta/recipes-devtools/rust/rust_1.74.1.bb +0 -0
index 30543ada7db..2dffe009827 100644
@@ -198,9 +198,13 @@ rust_runx () {
       # Copy the natively built llvm-config into the target so we
 can run it. Horrible,
       # but works!

      if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} != ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
-        mkdir -p dirname ${RUST_ALTERNATE_EXE_PATH} +        tgtdir=dirname ${RUST_ALTERNATE_EXE_PATH}
+        mkdir -p ${tgtdir}
          cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
-        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
+        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then +            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/ +        fi

Quoted 9 lines
Quoted 4 lines

+        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH} fi

      oe_cargo_fix_env

Copying a native library into the target sysroot goes beyond what I'm
comfortable with even for this horrible hack with llvm-config.

Quoted 2 lines

It’s just supporting the original hack to work properly I don’t think it’s any worse that the original hack

Putting a non-target library into the target sysroot could cause all kinds of problems if the target and native are similar enough the libraries can be seen and confused by the linker. I do see that as a big and potentially very dangerous difference.

Quoted 10 lines
Quoted 6 lines

Since it seems to be finding it by RPATH, can you not just add an RPATH
to the native sysroot in the binary?

It does not work and I don’t know why but seems rpath in llvm-config affects what paths it spits out

I am reluctant to take this without a better understanding of what is going on here. We may need to fix the lvm-config issues properly.

Cheers,

Richard

author
#4 lore raw
Quoted 77 lines · Richard Purdie
Quoted 75 lines · Khem Raj

Hi

Quoted 69 lines · Richard Purdie
Quoted 55 lines · Khem Raj

target rust recipe builds ( cross compile ) calls llvm-config from
target sysroot which works ok as long as C++ runtime it needs is available on build host e.g. libstdc++ etc. which is commonly the case, however when using clang and llvm runtime this falters since
it should be using libc++ from native sysroot and if this does not
exist on build machine this fails to find libc++ shared object and
llvm-config fails to run. This ensures that llvm-config version in
use is correctly relocated and can use shared libraries from native
sysroot correctly. Adding ORIGIN to sysroot will look for the .so in
same dir as the binary and there is the libc++.so.1 copied in place

Fixes rust build with clang compiler.

Quoted 4 lines

/mnt/b/yoe/master/build/tmp/work/riscv64-yoe- linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-
config: error while loading shared libraries: libc++.so.1:
cannot open shared object file: No such file or director

y

Quoted 6 lines

thread 'main' panicked at llvm.rs:551:19:
command did not execute successfully:
"/mnt/b/yoe/master/build/tmp/work/riscv64-yoe- linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm- config" "--version"
expected success, got: exit status: 127

---
  meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)
meta/recipes-devtools/rust/rust_1.74.1.bb +0 -0
index 30543ada7db..2dffe009827 100644
@@ -198,9 +198,13 @@ rust_runx () {
      # Copy the natively built llvm-config into the target so we
 can run it. Horrible,
      # but works!
      if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
 ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ];

then

meta/recipes-devtools/rust/rust_1.74.1.bb +2 -1
-        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
+        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
+        mkdir -p ${tgtdir}
          cp ${RUST_ALTERNATE_EXE_PATH_NATIVE}

${RUST_ALTERNATE_EXE_PATH}

meta/recipes-devtools/rust/rust_1.74.1.bb rust_runx () { · line 210

Quoted 8 lines
meta/recipes-devtools/rust/rust_1.74.1.bb +4 -1
-        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
+        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
+            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
+        fi
+        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
      fi
 
      oe_cargo_fix_env

Copying a native library into the target sysroot goes beyond what I'm
comfortable with even for this horrible hack with llvm-config.

Quoted 7 lines
Quoted 2 lines

It’s just supporting the original hack to work properly I don’t think it’s any worse that the original hack

Putting a non-target library into the target sysroot could cause all kinds of problems if the target and native are similar enough the libraries can be seen and confused by the linker. I do see that as a big and potentially very dangerous difference.

Agreed. although the conditions it must meet to fail make it quite unlikely since it goes into specific directory llvm-rust/bin where llvm-rust binaries are installed
under sysroot, and none of those target binaries use RPATH except llvm-config but there is a chance.

Quoted 9 lines
Quoted 9 lines
Quoted 7 lines

Since it seems to be finding it by RPATH, can you not just add an RPATH
to the native sysroot in the binary?

Quoted 6 lines
Quoted 2 lines

It does not work and I don’t know why but seems rpath in llvm-config affects what paths it spits out

I am reluctant to take this without a better understanding of what is going on here. We may need to fix the lvm-config issues properly.

I think if we use ORIGIN relative paths into RPATH, it works ok, previously I tried to hardcode native sysroot paths. I will send a revised version to replace
it with RPATH

Quoted 4 lines

Cheers,

Richard